From patchwork Thu Feb 22 17:39:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 13567698 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 81C22C54791 for ; Thu, 22 Feb 2024 17:39:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F10A66B0071; Thu, 22 Feb 2024 12:39:48 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id EC0D86B007E; Thu, 22 Feb 2024 12:39:48 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D627D6B0080; Thu, 22 Feb 2024 12:39:48 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id C28A16B0071 for ; Thu, 22 Feb 2024 12:39:48 -0500 (EST) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 91DC11C112E for ; Thu, 22 Feb 2024 17:39:48 +0000 (UTC) X-FDA: 81820152456.06.3A8FE21 Received: from mail-qt1-f175.google.com (mail-qt1-f175.google.com [209.85.160.175]) by imf29.hostedemail.com (Postfix) with ESMTP id AA20B120012 for ; Thu, 22 Feb 2024 17:39:46 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=soleen-com.20230601.gappssmtp.com header.s=20230601 header.b=1m1qi91a; dmarc=pass (policy=none) header.from=soleen.com; spf=pass (imf29.hostedemail.com: domain of pasha.tatashin@soleen.com designates 209.85.160.175 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1708623586; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=SmC4SQg9RJFTyxACzu+4WUHQYE29YjgzDeLpVcyOgV4=; b=Fc/0nRVQs9iJAe8eHXnsuE8LM5lN3yj4EacSb3+LPw02Wna4uC4G3ot0JYCQOb4M7ZcdB6 /vwU8p92okpe/50NtZP7Sa++uqUnRl7oCKLSxsUDL0Kp/nG7LjPI/H0aaEQma+mgdW1IAp 7DZ1utXqoYBw6BevUxSOFaJuZwltjbs= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=soleen-com.20230601.gappssmtp.com header.s=20230601 header.b=1m1qi91a; dmarc=pass (policy=none) header.from=soleen.com; spf=pass (imf29.hostedemail.com: domain of pasha.tatashin@soleen.com designates 209.85.160.175 as permitted sender) smtp.mailfrom=pasha.tatashin@soleen.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1708623586; a=rsa-sha256; cv=none; b=XkkyGQSiU6LzrxE8LDckuaDjRKjIWL6FPQIFURpk0xLKWeL99jPxGkucnJPnL0v2D1S3Np auS7wmmCKdxMRjdNPOt0aeh9zntPzFcA4l/rbEGUhxmpEVy4g9DEj9gJsf3YBCLQt3Gi/j EXywiA3i5Elcy0N4TMud6KPKKsq3t08= Received: by mail-qt1-f175.google.com with SMTP id d75a77b69052e-42a029c8e62so14481051cf.1 for ; Thu, 22 Feb 2024 09:39:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen-com.20230601.gappssmtp.com; s=20230601; t=1708623586; x=1709228386; darn=kvack.org; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:from:to:cc:subject:date:message-id:reply-to; bh=SmC4SQg9RJFTyxACzu+4WUHQYE29YjgzDeLpVcyOgV4=; b=1m1qi91aqcCsY6fLDhRN7uURWjPMLEOIOsv9PXzDOr/PQVP7q7DTH9f6zYOZ581qVK mVJD5gfYFOBkYqnTIN20T3p+nxNxgKvZCi5j39zP/DwtMogbzX8gw7JYNk/QgZdDJKx2 jSAofUflYcW69o7/G6c7TNJeh5m6NejmYWML6xoZbaL8CjnLpDeyouK6LrSosOKZc8Vd uSbDTWTCt4UliJ8+kdpUJv/gsu0HK7niwv9qhlyPR2n6vWFIYVcplGPLt5TUygINBPi5 M4n0+imMyBgpJrIFnpPyC8kJYpY2rsPEbx49z2Vq3OfjFID4DcVm+mDK/mvNk45m2KqI 76vg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708623586; x=1709228386; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=SmC4SQg9RJFTyxACzu+4WUHQYE29YjgzDeLpVcyOgV4=; b=Ni9MiAXotXHlt5T99tWA3zg9UuikZ22jY31aYrauXI3oEgc7mjUeeneMTMC4yZ7IvE H7YV8a6ulsPgWCVW95rH3Xz9+QzYetru0QyFzMllU48XrSV3IdxihCQVvY7o+/tZVw82 1n92cB0rKU/rXWXSBoKF0ZqdGVgrSeP8nM6tGBr11Sgn1FJMtDjAW7uJuOhUJcrp9YHX y9jnhprxsVKSBpC3wL+WdpFMhFampJFz8ajzFOGiA9qkzNfd33rKd/x3C/sOjG56Vv1t APMKYsWmZst07TnWv8feCeR92mGN0s9m5v5cFq/hqgnt7q1SXAsaJU/WEjK3xJDwxsqh qzvQ== X-Forwarded-Encrypted: i=1; AJvYcCWXirIJ8tXXYSNXPQt+h9Akr8bmvE2LJWqcbDNGmGpLYaZvBjUSPdvPLX9wskVLtt1MX4ZIH/eWH/JLl2MXB34tSEM= X-Gm-Message-State: AOJu0Yz0+zO0fn565z/v8pDf2lAg69YgOxn4mNjSPD0OVqy+96sDBJX3 nlwwCiUw0hxNmF6uYh+uwWNNVcR6LA7jKsUkZ3/Zt5Jp5B3FX7m2FRtoAOe4Gwc= X-Google-Smtp-Source: AGHT+IHQazQ2VKGKYiiCB/F+URR0eEgUMsL+eJQ7E97VwpEkplwTztlYNQqqnsB6QCcvUbBxCire4A== X-Received: by 2002:a05:622a:3d0:b0:42c:71ea:9b28 with SMTP id k16-20020a05622a03d000b0042c71ea9b28mr28604094qtx.32.1708623585634; Thu, 22 Feb 2024 09:39:45 -0800 (PST) Received: from soleen.c.googlers.com.com (249.240.85.34.bc.googleusercontent.com. [34.85.240.249]) by smtp.gmail.com with ESMTPSA id f17-20020ac86ed1000000b0042e5ab6f24fsm259682qtv.7.2024.02.22.09.39.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 22 Feb 2024 09:39:45 -0800 (PST) From: Pasha Tatashin To: akpm@linux-foundation.org, alim.akhtar@samsung.com, alyssa@rosenzweig.io, asahi@lists.linux.dev, baolu.lu@linux.intel.com, bhelgaas@google.com, cgroups@vger.kernel.org, corbet@lwn.net, david@redhat.com, dwmw2@infradead.org, hannes@cmpxchg.org, heiko@sntech.de, iommu@lists.linux.dev, jernej.skrabec@gmail.com, jonathanh@nvidia.com, joro@8bytes.org, krzysztof.kozlowski@linaro.org, linux-doc@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-rockchip@lists.infradead.org, linux-samsung-soc@vger.kernel.org, linux-sunxi@lists.linux.dev, linux-tegra@vger.kernel.org, lizefan.x@bytedance.com, marcan@marcan.st, mhiramat@kernel.org, m.szyprowski@samsung.com, pasha.tatashin@soleen.com, paulmck@kernel.org, rdunlap@infradead.org, robin.murphy@arm.com, samuel@sholland.org, suravee.suthikulpanit@amd.com, sven@svenpeter.dev, thierry.reding@gmail.com, tj@kernel.org, tomas.mudrunka@gmail.com, vdumpa@nvidia.com, wens@csie.org, will@kernel.org, yu-cheng.yu@intel.com, rientjes@google.com, bagasdotme@gmail.com, mkoutny@suse.com Subject: [PATCH v5 00/11] IOMMU memory observability Date: Thu, 22 Feb 2024 17:39:26 +0000 Message-ID: <20240222173942.1481394-1-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.44.0.rc0.258.g7320e95886-goog MIME-Version: 1.0 X-Rspamd-Queue-Id: AA20B120012 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: fkmk918gwcqoxi9yhih3g1mfr7tbsmhz X-HE-Tag: 1708623586-581925 X-HE-Meta: U2FsdGVkX18o8DyGFYLJTWvm3Drmk6qNNVA0ydSLNANaTJcuM6Sxj+rLlRloeS7WzyA/8m6fObsv8wo+oL2eia9EeoR0EC54j+NyRQ3mh9+L8DsW7hkb+1ZLXM20KhLzSplJ7AQ7aWjdt4lUFhQnAhMIU8thv7i4uBRmamHWzPCOsmJo364KV1pOEhCVErGgueXtADfe03hxjjlDVWLDPC6afeplqaRnbz6REyMEEgtkjOCm+1vWhkXYvXuzilLnI79g07mVLGTYQTU1X7NpHhE/a/G5XMvlvMKVZeWyu51kUsqVG0bSDBZnbdkN7691LEUgYns4QSXJkqV7WhDLK4SKrVGWSolLRdYQNiKZcwy1O/jC+MPkU4GCFrQ+Kwfi7+ThUztwjxAlPfGaJ0LLZy2QiR5nNL55nj75cnyhOgocFpaLSAfN3LeofF3AM2oG5hbs8QsksqEkbUXvRu38hWUoTzz360w7xcPMlery4esofLa/TiaVXQ/jK6HsHN8eoNvXqA5OG3gynxPxX1KyD+run/OF+NpJAfaConP+HGq046Bu6/aV9DfNXetQ9CgRPC4J5Svznz6TNw7aytqhfL14g0iOfyFqe/TexYo9wTkSsTXNx8//ZiDqqrQ9arRJUAvmGvu4RKfPgmzUvlGtXhHybrr+w3TonE6yedErDFMwElGzGKa4yV4BrjlPEtI6RKrlvsPGqDQEdA4aVto1GkfjQOAskD/cHVghlRMCrKHB6jeOxjzrvOe0E6upuNtCk0cHln99XxwvhblXUPV5xwujz79iZ7sJXgd+Jf2u5Ycw82lrmV22En1aTqhZqNT0Q0pWH4H9lIoITHlZx4pM21M+FRMawk+is4UEKHiaDRBkFv+a57a2bIY9j2gOhiJVYbFXGQ2X5zbcRE5zlwCYv47jpZfX1Ng/5dnFnBrgF2DpNJtK1SH3qHgMno8TawzqaJIued+UtueiG00Qyc9 TXLq3PAj sXlE19nDEpyRg7k3+uya+S3NUpkpjFS17yziYmEBCzDayRuAU84Ag/E2E3tO4Hm9fgo8e0Tq7s2ptfMNBpA4wbnlw+wTP6ZkdFkTfbilwj1KTJZx+J18iv2YAkFqZ09hVt7vc3iyuk3GTXnJ8lFQER/IBFn8Z+6IwEyJeeGb1ezQ0p83iC5u5LGDOtCGWBYGt66Y01vT6NAZbbyyqNyjuqoLJRawOZgntL9GC2symN6ekvrQdk9IuBiBzzliKzST4UR54Zx7ckO9KKEiX84J6euwwupOFxbdy5P3BIU4bH6N5Eap4Tyu5AYPFX23bEIBEfSh8tj+kJ332il324YI4mQLtcJRwyU+dXArn4vFH5vZZk+uelGgd9VKGnEM4rgS4A9QPYldQWSWXWS+Nr040IPbYlRnHczmu6ZtMj3ej1uqYJAEiFWyMAk8qgCQGqqGSHw5yeltqeWxcyjBeo859kZhqsQzhWd1o6V6MY2fC+ybUlt4ss1Xoc01omOnRmRrE2hQvtQq6lZ7A3zH6B/OVhpZ/1EEY/uk0bbiwRMm84qZI3PGJtb6euXreuVG4zfH+GjmwIdhjfwstsZVb4s4XKD9ckVfkil7ScA7bU8VsBY9qinYTuEx7uLaBK9PPyjLKy6W7zIx9Uf8X4GLaSXJItnzVjGOMw1mGpq6HKv/mrbu/gNXaxyWzH+HHZDHqrHDOv/bEuvOz8GXvTFQZxKfOMHIaNFDorbg4Q/GBijw1F+vy7gflUqIGmLzJaY2Sz2ySg1y4z2WYeRuyJX5IX8eoZX3rSg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: ---------------------------------------------------------------------- Changelog ---------------------------------------------------------------------- v5: - Synced with v6.8-rc5 - Added: Acked-by: Marek Szyprowski - Added: Acked-by: Jernej Skrabec - Addressed review comments from Robin Murphy: Updated the header comment in iommu-pages.h Removed __iommu_alloc_pages_node(), invoke iommu_alloc_pages_node directly. Removed unused: __iommu_alloc_page_node() Removed __iommu_free_page() Renamed: iommu_free_pages_list() -> iommu_put_pages_list() Added missing iommu_put_pages_list() to dma-iommu.c in iommu/dma: use iommu_put_pages_list() to releae freelist v4: - Synced with v6.8-rc3 - Updated commit log for "iommu: account IOMMU allocated memory" as suggested by Michal Koutný - Added more Acked-bys David Rientjes and Thierry Reding - Added Tested-by Bagas Sanjaya. v3: - Sync with v6.7-rc7 - Addressed comments from David Rientjes: s/pages/page/, added unlikely() into the branches, expanded comment for iommu_free_pages_list(). - Added Acked-bys: David Rientjes v2: - Added Reviewed-by Janne Grunau - Sync with 6.7.0-rc3 - Separated form the series patches: vhost-vdpa: account iommu allocations https://lore.kernel.org/all/20231130200447.2319543-1-pasha.tatashin@soleen.com vfio: account iommu allocations https://lore.kernel.org/all/20231130200900.2320829-1-pasha.tatashin@soleen.com as suggested by Jason Gunthorpe - Fixed SPARC build issue detected by kernel test robot - Drop the following patches as they do account iommu page tables: iommu/dma: use page allocation function provided by iommu-pages.h iommu/fsl: use page allocation function provided by iommu-pages.h iommu/iommufd: use page allocation function provided by iommu-pages.h as suggested by Robin Murphy. These patches are not related to IOMMU page tables. We might need to do a separate work to support DMA observability. - Remove support iommu/io-pgtable-arm-v7s as the 2nd level pages are under a page size, thanks Robin Murphy for pointing this out. ---------------------------------------------------------------------- Description ---------------------------------------------------------------------- IOMMU subsystem may contain state that is in gigabytes. Majority of that state is iommu page tables. Yet, there is currently, no way to observe how much memory is actually used by the iommu subsystem. This patch series solves this problem by adding both observability to all pages that are allocated by IOMMU, and also accountability, so admins can limit the amount if via cgroups. The system-wide observability is using /proc/meminfo: SecPageTables: 438176 kB Contains IOMMU and KVM memory. Per-node observability: /sys/devices/system/node/nodeN/meminfo Node N SecPageTables: 422204 kB Contains IOMMU and KVM memory in the given NUMA node. Per-node IOMMU only observability: /sys/devices/system/node/nodeN/vmstat nr_iommu_pages 105555 Contains number of pages IOMMU allocated in the given node. Accountability: using sec_pagetables cgroup-v2 memory.stat entry. With the change, iova_stress[1] stops as limit is reached: $ ./iova_stress iova space: 0T free memory: 497G iova space: 1T free memory: 495G iova space: 2T free memory: 493G iova space: 3T free memory: 491G stops as limit is reached. This series encorporates suggestions that came from the discussion at LPC [2]. ---------------------------------------------------------------------- [1] https://github.com/soleen/iova_stress [2] https://lpc.events/event/17/contributions/1466 ---------------------------------------------------------------------- Previous versions v1: https://lore.kernel.org/all/20231128204938.1453583-1-pasha.tatashin@soleen.com v2: https://lore.kernel.org/linux-mm/20231130201504.2322355-1-pasha.tatashin@soleen.com v3: https://lore.kernel.org/all/20231226200205.562565-1-pasha.tatashin@soleen.com v4: https://lore.kernel.org/all/20240207174102.1486130-1-pasha.tatashin@soleen.com ---------------------------------------------------------------------- Pasha Tatashin (11): iommu/vt-d: add wrapper functions for page allocations iommu/dma: use iommu_put_pages_list() to releae freelist iommu/amd: use page allocation function provided by iommu-pages.h iommu/io-pgtable-arm: use page allocation function provided by iommu-pages.h iommu/io-pgtable-dart: use page allocation function provided by iommu-pages.h iommu/exynos: use page allocation function provided by iommu-pages.h iommu/rockchip: use page allocation function provided by iommu-pages.h iommu/sun50i: use page allocation function provided by iommu-pages.h iommu/tegra-smmu: use page allocation function provided by iommu-pages.h iommu: observability of the IOMMU allocations iommu: account IOMMU allocated memory Documentation/admin-guide/cgroup-v2.rst | 2 +- Documentation/filesystems/proc.rst | 4 +- drivers/iommu/amd/amd_iommu.h | 8 - drivers/iommu/amd/init.c | 91 ++++++------ drivers/iommu/amd/io_pgtable.c | 13 +- drivers/iommu/amd/io_pgtable_v2.c | 20 +-- drivers/iommu/amd/iommu.c | 13 +- drivers/iommu/dma-iommu.c | 7 +- drivers/iommu/exynos-iommu.c | 14 +- drivers/iommu/intel/dmar.c | 16 +- drivers/iommu/intel/iommu.c | 47 ++---- drivers/iommu/intel/iommu.h | 2 - drivers/iommu/intel/irq_remapping.c | 16 +- drivers/iommu/intel/pasid.c | 18 +-- drivers/iommu/intel/svm.c | 11 +- drivers/iommu/io-pgtable-arm.c | 15 +- drivers/iommu/io-pgtable-dart.c | 37 ++--- drivers/iommu/iommu-pages.h | 186 ++++++++++++++++++++++++ drivers/iommu/rockchip-iommu.c | 14 +- drivers/iommu/sun50i-iommu.c | 7 +- drivers/iommu/tegra-smmu.c | 18 ++- include/linux/mmzone.h | 5 +- mm/vmstat.c | 3 + 23 files changed, 361 insertions(+), 206 deletions(-) create mode 100644 drivers/iommu/iommu-pages.h