From patchwork Wed Feb 7 17:40:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 13548800 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 69CDBC4828D for ; Wed, 7 Feb 2024 17:41:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B71176B0072; Wed, 7 Feb 2024 12:41:08 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B20A16B0074; Wed, 7 Feb 2024 12:41:08 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9A60C6B0075; Wed, 7 Feb 2024 12:41:08 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 8A1F46B0072 for ; Wed, 7 Feb 2024 12:41:08 -0500 (EST) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 53AEA1A0987 for ; Wed, 7 Feb 2024 17:41:08 +0000 (UTC) X-FDA: 81765723816.19.1AB8D8E Received: from mail-yw1-f179.google.com (mail-yw1-f179.google.com [209.85.128.179]) by imf11.hostedemail.com (Postfix) with ESMTP id 94D9540012 for ; Wed, 7 Feb 2024 17:41:06 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=none ("invalid DKIM record") header.d=soleen.com header.s=google header.b="hs/Qa+sw"; dmarc=none; spf=none (imf11.hostedemail.com: domain of pasha.tatashin@soleen.com has no SPF policy when checking 209.85.128.179) smtp.mailfrom=pasha.tatashin@soleen.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1707327666; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=si3iSYQzW/wS10JKxA6PCTvsO4f4p9drxSnRwVUScZU=; b=d/BkFJUDwcVzx8gTOnKw2Xmv23XcVmGP1QtGRFBVHcRkucCsncU4reEPl9lkZ0WTVyhgv1 d5I5QNNWajdUhsRh9j9GoPqgXZeo2RiV1Y4wTweSRIOSmjpynG+71uQR6u2dUGIPHZbL5b uopYrdci5VFWTzNJZS2REDxq+ji2CMY= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=none ("invalid DKIM record") header.d=soleen.com header.s=google header.b="hs/Qa+sw"; dmarc=none; spf=none (imf11.hostedemail.com: domain of pasha.tatashin@soleen.com has no SPF policy when checking 209.85.128.179) smtp.mailfrom=pasha.tatashin@soleen.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1707327666; a=rsa-sha256; cv=none; b=ElCNaLd1bHKNNusZBqYekziLsvqN1cRtbdyrL0fBCzrAgaawkRa5R+QmTNT6zM+0nlAn+J PsMT26ZYhJw35//XfmTJq/eEV5e27S3ay+jyU+qT7+EZTV7tcCYj7RVHCApQ6o3+jrIPPp swDn1QIhxJ3ycyL6x3t2D0lPORJvRw4= Received: by mail-yw1-f179.google.com with SMTP id 00721157ae682-60492003050so8625697b3.0 for ; Wed, 07 Feb 2024 09:41:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; t=1707327665; x=1707932465; darn=kvack.org; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:from:to:cc:subject:date:message-id:reply-to; bh=si3iSYQzW/wS10JKxA6PCTvsO4f4p9drxSnRwVUScZU=; b=hs/Qa+sw+VX42Q9NT/nQdpIHeY4Rno/p9JqiHqeWNV+Jw+AIUIKKFVlRufevVk9jkf Nflkb4s3gm7ftVK1FlHZ/bbI1scwP5oNbASHufKrshT2tgkvP1YB4DdJgu8Dcg3aA6Ln QvVhKvxQRb91BSCHu2/1u6PkQRHwGqN6IvHu6hPe5LZa6SQf6BWdBNh7T1xC1vcAOyrb 61stjdmHtlv+ClinYZxfQzq2uSoEfBkcf3d6C7pptxZV17oId7ISev9DLzwx3fNQbYe6 02qN/TTTvLJnIjyp30u+ltYMXr2FyM0MSib6JUlcOrsJJhw5Fjfu8fmfMJ00Pr/VWLtp bm7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707327665; x=1707932465; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=si3iSYQzW/wS10JKxA6PCTvsO4f4p9drxSnRwVUScZU=; b=obdixsV4W/N436pLjn4tixFuIZlxJ9q5kerI1Z+v8Xd+ONPFmTqQQYjpcA2iVWHCRt LzcNWFFRA5NeyGKh13YgGMHSmi3yjfp1vVf/oETnDrFlF83z47Bzg5x1jVosQNGIRId+ MRs8dXzXkk9jqliiN1o6LGfXK1jwTNOY5nwIvY1Xsb5QM1I7SdXlV0VM5dc42J1Gf55u 0c/BAYbjqDPgMk4Vp3dHWKgPZSycGkBh2cg9SqTl+hol6UBqxyunKX4+2XodfrLLzijX 94St06y1uf6eg1/bqiA8l8MBCcTK4818HZS9p3vxRPLhBgJXwxxUeRmxh32yp/3XdYjo yuyg== X-Forwarded-Encrypted: i=1; AJvYcCUvL17o8ZPp3qAcyDT8g+48fFf1NKswsqmjYqt+uYeEVCns+bZrY0EA254mpKmCeT30CJlbBSOcJv8wjhrp1Kuwg7c= X-Gm-Message-State: AOJu0YxJ5C7UBpkGAmb8q+c4WobI976YZjYk1scAIEKOY5KKJHiFFmvZ gfWMhpBpsyGg3kL+tYLyRPKDVXVMMn4MvoDWtFsBmra9k3Kkqdi2P4okAzPlrPU= X-Google-Smtp-Source: AGHT+IEEHU0Zq486UjgccXih/doHmjMJ7ii4hv+46i8KhFxxFLmJqhnzWQ3qz8k37AV1xp4vGAJu6A== X-Received: by 2002:a0d:d641:0:b0:604:9fd4:a0ec with SMTP id y62-20020a0dd641000000b006049fd4a0ecmr668771ywd.5.1707327665531; Wed, 07 Feb 2024 09:41:05 -0800 (PST) X-Forwarded-Encrypted: i=1; AJvYcCUkpA1e8or7NcUr/WsWjrzjdXlKnYP7Brox9OXywH4gDbvfn48o3nQvHRI3pVbQX4O+aXkOe6P1Hn3x8IUM5XVB16c5XA7JXuzZH5fDUuhC3U9N1bfEbkfjgV7/xqboIkaeaOm372F/MkZZVEq0wEc2DJuJkyXvpBTbqVylv+rD5YWo0/G+Kg3OD5182kxWb9kzCZ+UE/n0vaD+CRG2C1kuzz4SmrQ6NdqQbHPuKYn6WxVgQS1Scls5HHdhjkF410H1Jnetb2wUH9Vej4DTNp8scc0RM4g+s3dGiuedxcYwU/O4lWzNLDnUo/WA9DCIOYKmjzvnmxeBED3dOwxSJ7FMF3P7X+3ERyu7sWGAfJMo4dSW5siDWi6HZiZ5MqA9leEV73ZV1Xm1wucVQ9wsQHZulJm4UYSoAVTYaQ9clHXTJ+boA8TvSi+pQFaIkTmzILrZcCnP/v9+gZF2G3b2d2KFGmqYSgG0gGaeKeptMhY5zBZ5TXiR5vMzoVLgPLdposs2r8L4pNbOEWF0zi6izX845ny9/Y8ba709mNUr9BsTXTdpXOH32jdwZUB1rRUPT4dFPtw6nyl8BmXWI3DBiZnpBu47RgCKYnZfBSXFi2q0ZTfGpY6vkE0+pNUelxzhMwP9YImd8mz8ORz8gfk0sj2DMsm151dgQmX/uC9RcMuoN0PSwo42yRz7KspGOG5eic/E2f2nSxwgMyWgDE8l4BZ6AqGYqlwpCJ66J1Li+1ikKsc722qVzr+fdFigDGJDZehHFCM8B1EdQjx8ub/jH9/P26lDifl1VjFAc3AA4Kvvsyo6yhiOJofANrUTNPO6SqS0jzDMXufYMQZIoY+HTiBjvj6aFjyMPao63IIQYiOrKr6uAeYBxKTyEUYAz6+FQVqHMG28i+uK+/VCIFl4wemd2x3yQ34RfasOYy5M2vSNaEUDOn3DeO3cOzE+rN ZYkpbwNh uQLZgO5OgShpDB5RC7nx4T4vJMHBLV4gdTrXKpcq5xKTKPg4L5nfJnheTA51MSe4dZHDfknxqft58Yp2qBaw3tvo3OmilhNVPPJyw5F7OKhNnu0H7PXSEV9GbfUQKNfMuv2lOfWpHHyZl+skgBoEfMVKO2/jKOJPnWm8ioObQVd6L2344T1EtRFasW0zYPwG9TQRQE+RZfpHFZr/v5NAy7nVe2HWfQmZ+TNHgtBJ8saOFbD+lqsTZ7loaKSaE51S/rACmqjiUEXw9S+tAJKyFQ8s6HaQikjgoPGoTP71vIid3a11OqK/EgMvScjkeqG0vU9tuvtXKomcd+gn/P3WY+qXKHpd9ClUllsbc/g4f39izeoO+NuBgrOyc8DAKyHdSA7OvsYgwTXHlopChDAFWRMic7m+I6ke8MGRJUTc5D9/kmWyxIE8uXkDBqWdSKuVjeQqEFojdrrbHmALXwMRaHOqgrUm+TD21Sq/eQwjWSfH+Z Received: from soleen.c.googlers.com.com (249.240.85.34.bc.googleusercontent.com. [34.85.240.249]) by smtp.gmail.com with ESMTPSA id e10-20020a37db0a000000b007854018044bsm696310qki.134.2024.02.07.09.41.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Feb 2024 09:41:05 -0800 (PST) From: Pasha Tatashin To: akpm@linux-foundation.org, alim.akhtar@samsung.com, alyssa@rosenzweig.io, asahi@lists.linux.dev, baolu.lu@linux.intel.com, bhelgaas@google.com, cgroups@vger.kernel.org, corbet@lwn.net, david@redhat.com, dwmw2@infradead.org, hannes@cmpxchg.org, heiko@sntech.de, iommu@lists.linux.dev, jernej.skrabec@gmail.com, jonathanh@nvidia.com, joro@8bytes.org, krzysztof.kozlowski@linaro.org, linux-doc@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-rockchip@lists.infradead.org, linux-samsung-soc@vger.kernel.org, linux-sunxi@lists.linux.dev, linux-tegra@vger.kernel.org, lizefan.x@bytedance.com, marcan@marcan.st, mhiramat@kernel.org, m.szyprowski@samsung.com, pasha.tatashin@soleen.com, paulmck@kernel.org, rdunlap@infradead.org, robin.murphy@arm.com, samuel@sholland.org, suravee.suthikulpanit@amd.com, sven@svenpeter.dev, thierry.reding@gmail.com, tj@kernel.org, tomas.mudrunka@gmail.com, vdumpa@nvidia.com, wens@csie.org, will@kernel.org, yu-cheng.yu@intel.com, rientjes@google.com, bagasdotme@gmail.com, mkoutny@suse.com Subject: [PATCH v4 00/10] IOMMU memory observability Date: Wed, 7 Feb 2024 17:40:52 +0000 Message-ID: <20240207174102.1486130-1-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.43.0.594.gd9cf4e227d-goog MIME-Version: 1.0 X-Rspamd-Queue-Id: 94D9540012 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: f99ncrxuerj8yw5df5z36twk7jffnxq1 X-HE-Tag: 1707327666-856893 X-HE-Meta: U2FsdGVkX1/A+TZtftLwz+GoKBlxiFVROmSSFjZxV5TrD/eyYmfWArI1l6/jFPC5yWs2segVTuBKYRIMpigTq4hZs/qrTM3uz+Vppfsq8bg1l2Dq3XN5QmVYWpTvOWw0kl4cP8uH6qVw0DffuO9rXZzwzlxnaqK7CgGr+oec9lpDxybiB/j36cCsu43PBZV1LnGPE+QId0dZ2AqcNlc1Fi3kKm8Tn6IgDKfsUn6G9Zjt6qh81U41i615FWM2CleNlf6OQfxr6qdXveXooo19Ve4HvbWbBBATID7/hycAM8gv6n/zkpmg+3ANZ79yvtS49BIZ9S45ErHq+k4is3mCH25nqtOSus/Y0W/GHBvXdHCc+aRb/d6yCiGRIlDSzRtfcSSeBcgd7ZiSU6AO3KB/edgCKGBdyetkf1c9TNM2EtGa3MutFc/LJOonVmGUOYFNKbnbUcGAje2HGyuRY5mqJ/sULFqVWO/heiCdRNHZo0wq2uz3R+acjITwEpiDiuzNb3skRsjTT0VcjninZz65bGQirTfODCVHdm2XuWSdxHOXUoggWlqE1UnokaDtmNyo+SouvVHS2ZB3xhlCF/UZeIzN1NT24/PR0egdNtPLrXN3xq5PCF+bTRZ7zF2RcQH+0rVO/bJP4IfVLxG0U4BJ1b8GtGmii3BY/7hdxb4FML8xynbo1x7szeVF3+iwvd2k5XrQs+TVKgiuR1D/ZhGAafRT99ROPrFSCf/de6U4AhvUS/YfpdiQHMRlr2a+2RbpcLMK6IeUjl0hJuieKNn/TWFWQuWKTUTJBvcKcdgQ+wOXMZcfhkF7HSRCGj9e0+gFlh0ekklwmNF47eeM3Mz8jbhUkIwS4gwHf0tjrfVMO70cHIFDc1cLsvu9UvGINd04jKAa1It4pPHMgg9uc9b5gteZ+NYaRFrj3O0nAvi31RlfUZtfDWZE9VpxEJHucMyFsw00R9RDGldESHf/p9t BBZr4RFv Xn7ShMWxkvVuDofTuz6XatkxEcqIBQVEBYxfeZ23msWNhwoEN1upvSbaUyABnk1BQFsXqTUu61dx6NV4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: ---------------------------------------------------------------------- Changelog ---------------------------------------------------------------------- v4: - Synced with v6.8-rc3 - Updated commit log for "iommu: account IOMMU allocated memory" as suggested by Michal Koutný - Added more Acked-bys David Rientjes and Thierry Reding - Added Tested-by Bagas Sanjaya. v3: - Sync with v6.7-rc7 - Addressed comments from David Rientjes: s/pages/page/, added unlikely() into the branches, expanded comment for iommu_free_pages_list(). - Added Acked-bys: David Rientjes v2: - Added Reviewed-by Janne Grunau - Sync with 6.7.0-rc3 - Separated form the series patches: vhost-vdpa: account iommu allocations https://lore.kernel.org/all/20231130200447.2319543-1-pasha.tatashin@soleen.com vfio: account iommu allocations https://lore.kernel.org/all/20231130200900.2320829-1-pasha.tatashin@soleen.com as suggested by Jason Gunthorpe - Fixed SPARC build issue detected by kernel test robot - Drop the following patches as they do account iommu page tables: iommu/dma: use page allocation function provided by iommu-pages.h iommu/fsl: use page allocation function provided by iommu-pages.h iommu/iommufd: use page allocation function provided by iommu-pages.h as suggested by Robin Murphy. These patches are not related to IOMMU page tables. We might need to do a separate work to support DMA observability. - Remove support iommu/io-pgtable-arm-v7s as the 2nd level pages are under a page size, thanks Robin Murphy for pointing this out. ---------------------------------------------------------------------- Description ---------------------------------------------------------------------- IOMMU subsystem may contain state that is in gigabytes. Majority of that state is iommu page tables. Yet, there is currently, no way to observe how much memory is actually used by the iommu subsystem. This patch series solves this problem by adding both observability to all pages that are allocated by IOMMU, and also accountability, so admins can limit the amount if via cgroups. The system-wide observability is using /proc/meminfo: SecPageTables: 438176 kB Contains IOMMU and KVM memory. Per-node observability: /sys/devices/system/node/nodeN/meminfo Node N SecPageTables: 422204 kB Contains IOMMU and KVM memory in the given NUMA node. Per-node IOMMU only observability: /sys/devices/system/node/nodeN/vmstat nr_iommu_pages 105555 Contains number of pages IOMMU allocated in the given node. Accountability: using sec_pagetables cgroup-v2 memory.stat entry. With the change, iova_stress[1] stops as limit is reached: $ ./iova_stress iova space: 0T free memory: 497G iova space: 1T free memory: 495G iova space: 2T free memory: 493G iova space: 3T free memory: 491G stops as limit is reached. This series encorporates suggestions that came from the discussion at LPC [2]. ---------------------------------------------------------------------- [1] https://github.com/soleen/iova_stress [2] https://lpc.events/event/17/contributions/1466 ---------------------------------------------------------------------- Previous versions v1: https://lore.kernel.org/all/20231128204938.1453583-1-pasha.tatashin@soleen.com v2: https://lore.kernel.org/linux-mm/20231130201504.2322355-1-pasha.tatashin@soleen.com v3: https://lore.kernel.org/all/20231226200205.562565-1-pasha.tatashin@soleen.com ---------------------------------------------------------------------- Pasha Tatashin (10): iommu/vt-d: add wrapper functions for page allocations iommu/amd: use page allocation function provided by iommu-pages.h iommu/io-pgtable-arm: use page allocation function provided by iommu-pages.h iommu/io-pgtable-dart: use page allocation function provided by iommu-pages.h iommu/exynos: use page allocation function provided by iommu-pages.h iommu/rockchip: use page allocation function provided by iommu-pages.h iommu/sun50i: use page allocation function provided by iommu-pages.h iommu/tegra-smmu: use page allocation function provided by iommu-pages.h iommu: observability of the IOMMU allocations iommu: account IOMMU allocated memory Documentation/admin-guide/cgroup-v2.rst | 2 +- Documentation/filesystems/proc.rst | 4 +- drivers/iommu/amd/amd_iommu.h | 8 - drivers/iommu/amd/init.c | 91 +++++---- drivers/iommu/amd/io_pgtable.c | 13 +- drivers/iommu/amd/io_pgtable_v2.c | 20 +- drivers/iommu/amd/iommu.c | 13 +- drivers/iommu/exynos-iommu.c | 14 +- drivers/iommu/intel/dmar.c | 10 +- drivers/iommu/intel/iommu.c | 47 ++--- drivers/iommu/intel/iommu.h | 2 - drivers/iommu/intel/irq_remapping.c | 10 +- drivers/iommu/intel/pasid.c | 12 +- drivers/iommu/intel/svm.c | 7 +- drivers/iommu/io-pgtable-arm.c | 15 +- drivers/iommu/io-pgtable-dart.c | 37 ++-- drivers/iommu/iommu-pages.h | 236 ++++++++++++++++++++++++ drivers/iommu/rockchip-iommu.c | 14 +- drivers/iommu/sun50i-iommu.c | 7 +- drivers/iommu/tegra-smmu.c | 18 +- include/linux/mmzone.h | 5 +- mm/vmstat.c | 3 + 22 files changed, 397 insertions(+), 191 deletions(-) create mode 100644 drivers/iommu/iommu-pages.h