From patchwork Mon Feb 5 15:32:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Robin Murphy X-Patchwork-Id: 13545693 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C844AC4828D for ; Mon, 5 Feb 2024 15:32:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 02C6E6B0085; Mon, 5 Feb 2024 10:32:57 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id EF8866B0087; Mon, 5 Feb 2024 10:32:56 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DE7DD6B0088; Mon, 5 Feb 2024 10:32:56 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id CD2A26B0085 for ; Mon, 5 Feb 2024 10:32:56 -0500 (EST) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 4658E120A57 for ; Mon, 5 Feb 2024 15:32:56 +0000 (UTC) X-FDA: 81758143152.10.300361F Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf02.hostedemail.com (Postfix) with ESMTP id 55CD580026 for ; Mon, 5 Feb 2024 15:32:54 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf02.hostedemail.com: domain of robin.murphy@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=robin.murphy@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1707147174; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=unFrpB4e3+scnqjyvzclIGP74f4AAsET/Bk0JpL7ly8=; b=zIaJVWz8olUQR5PW9KsZAtRYI16c9YRtyTa7TKKvVtEOV5LDaSOMVaSf7V9CmlM1X2jTAg Iz/ugL/exZwjclc6YFrOqEBHqdZlad2DoBx/nsIIctKG5E6qVjww7eLMAyxZIrSiW6BtKm YABvVOBwUzsYXAt2GTW/kKLnljjSNVc= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf02.hostedemail.com: domain of robin.murphy@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=robin.murphy@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1707147174; a=rsa-sha256; cv=none; b=3TPaC6TbbzEBuFREnjUKb4z5cJuDpjsD6Jz0Pf1HcLs265fMbu9iyoxMERMuy+oGzmBy1N krfqjd9CvAOG08PNAb543n7sAe8Weqiv/FDhBb5O6jXROnIxe7qP+AUCtp2p5Esu/tc/Yp nM0MtVNIv1g+5EHB6hRb2nmNh4m7xEo= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 14C211FB; Mon, 5 Feb 2024 07:33:36 -0800 (PST) Received: from e121345-lin.cambridge.arm.com (e121345-lin.cambridge.arm.com [10.1.196.40]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 483403F5A1; Mon, 5 Feb 2024 07:32:52 -0800 (PST) From: Robin Murphy To: joro@8bytes.org Cc: will@kernel.org, pasha.tatashin@soleen.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-mm@kvack.org, rientjes@google.com, yosryahmed@google.com, john.g.garry@oracle.com Subject: [PATCH v3 3/3] iommu/iova: use named kmem_cache for iova magazines Date: Mon, 5 Feb 2024 15:32:41 +0000 Message-Id: X-Mailer: git-send-email 2.39.2.101.g768bb238c484.dirty In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: 55CD580026 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: iejf5ktcrp6zi97we7igshe8cm4s9dh9 X-HE-Tag: 1707147174-694800 X-HE-Meta: U2FsdGVkX19cohhBEP+JN7v/unLQcxeiMov2KA4r6SyMFS79xHkd3Mb+7sPMqASD9wOPy90PNFiziyqAI2ys6WKGO8EzzIWavJcqXHkRvPgXPGES52J3/oIh0qPZY3+s00LGw3GD1+YV/vmRbdL/Y4S3ptNyeIRtxXQncEHfWsXfhf33MlwcZiIPO0zYOqhekmmggb2fw3xHuyY0tuKnGFsLVdrMiAOVBRB05WDQG5NIUoEc6K7io683BCDAwqfsX0fkIBwwDm0jXNeUfYGttjWUwkp7AxluL4AcqyS0sNqT5VLMe1/p6QY+rOU5ZpS9PUDDlIVa366E2LYgZyImAPFw0ves47lFbURGGSN/Dfx1H3qjvgkhE/xLJBDJnAIHK8Mbv8+BPEQOcMNMBp6NLi6AwQQe7mXnM3X/8ESEoYKOBRUNPEfElcgFEg7gKDQpyDc+s1BbSFK6d4LcWqdmXpZwe15z+xkTQFn6zY0+ttkzR/GCWD5UFk9morKf0qx/qYFN5DqtplL14zTnFDE31tDpHHuIZdTurBAwHUyXhAxzL2Cfikc2r/cdEZt6ZZNIbxf2bhdHGZvlISOelVZ7VEGOP3ANA3IZ2g/Pf18m7KeWRdXnVamTi9zOgdJCJ7ORUzYR46+JOXTZgWtMx7B9a+zQ9Mi1xHQejxjKHUE4ZAytuU7/PhFI/tiZPPCMp7U9m09eoDw0NS8AkVEXZJ19nZUkSc6pBZSVVsNPUxMTye/DI09dSAQ4Wz131Xjbomh5OQJiayoaWf/uS9q0DCVIlooXKtlsCkmfTuaX1d6mmaZyjhwjsH2CUDG0eEW16zzB9QMPmU63iPz3fe4m03dyF7eWvkNlDwrpI2g/FkRVwFV7Y9yUgg1d5C07c3gJb9jqK3IZi1yr2imu3FvGADz9KclWXyxUkLQF2DU12hrqnbhXGIr1lbYfeD/Q+tjhjeiTF6rnnTWqwH8M+rdpFP/ /UCLhe5A BzNByODbzkyLiZYA7HsETsS07uis/6f/PvG+dys5Zt3J9FYVBKX2Oh3JDkaabkXbcBnqURVElowBflDiufHD6VIbLy+HdcgUFb/8hcQmWP2RXEEvQhKiv2YzXqJm82jea1GyHArsKe73XsCHAuheskEUdMLUNAr0El6QDLyLQ0gIUeAml3nwEHCHwdlSb/QnX6HT4H7nUlG9+ASKVsRFS9hMrdeUa6Y9SoKo+Bd6s9kskl6wswO4bBAN9fxINXbOmOF41/Zojk7r9IeY2PlbKScDvyx+uQ35QxLlU X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Pasha Tatashin The magazine buffers can take gigabytes of kmem memory, dominating all other allocations. For observability purpose create named slab cache so the iova magazine memory overhead can be clearly observed. With this change: > slabtop -o | head Active / Total Objects (% used) : 869731 / 952904 (91.3%) Active / Total Slabs (% used) : 103411 / 103974 (99.5%) Active / Total Caches (% used) : 135 / 211 (64.0%) Active / Total Size (% used) : 395389.68K / 411430.20K (96.1%) Minimum / Average / Maximum Object : 0.02K / 0.43K / 8.00K OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME 244412 244239 99% 1.00K 61103 4 244412K iommu_iova_magazine 91636 88343 96% 0.03K 739 124 2956K kmalloc-32 75744 74844 98% 0.12K 2367 32 9468K kernfs_node_cache On this machine it is now clear that magazine use 242M of kmem memory. Acked-by: David Rientjes Signed-off-by: Pasha Tatashin [ rm: adjust to rework of iova_cache_{get,put} ] Signed-off-by: Robin Murphy Reviewed-by: John Garry Reviewed-by: Jerry Snitselaar --- drivers/iommu/iova.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/drivers/iommu/iova.c b/drivers/iommu/iova.c index b5de865ee50b..d59d0ea2fd21 100644 --- a/drivers/iommu/iova.c +++ b/drivers/iommu/iova.c @@ -590,6 +590,8 @@ struct iova_rcache { struct delayed_work work; }; +static struct kmem_cache *iova_magazine_cache; + unsigned long iova_rcache_range(void) { return PAGE_SIZE << (IOVA_RANGE_CACHE_MAX_SIZE - 1); @@ -599,7 +601,7 @@ static struct iova_magazine *iova_magazine_alloc(gfp_t flags) { struct iova_magazine *mag; - mag = kmalloc(sizeof(*mag), flags); + mag = kmem_cache_alloc(iova_magazine_cache, flags); if (mag) mag->size = 0; @@ -608,7 +610,7 @@ static struct iova_magazine *iova_magazine_alloc(gfp_t flags) static void iova_magazine_free(struct iova_magazine *mag) { - kfree(mag); + kmem_cache_free(iova_magazine_cache, mag); } static void @@ -953,6 +955,12 @@ int iova_cache_get(void) if (!iova_cache) goto out_err; + iova_magazine_cache = kmem_cache_create("iommu_iova_magazine", + sizeof(struct iova_magazine), + 0, SLAB_HWCACHE_ALIGN, NULL); + if (!iova_magazine_cache) + goto out_err; + err = cpuhp_setup_state_multi(CPUHP_IOMMU_IOVA_DEAD, "iommu/iova:dead", NULL, iova_cpuhp_dead); if (err) { @@ -968,6 +976,7 @@ int iova_cache_get(void) out_err: kmem_cache_destroy(iova_cache); + kmem_cache_destroy(iova_magazine_cache); mutex_unlock(&iova_cache_mutex); return err; } @@ -984,6 +993,7 @@ void iova_cache_put(void) if (!iova_cache_users) { cpuhp_remove_multi_state(CPUHP_IOMMU_IOVA_DEAD); kmem_cache_destroy(iova_cache); + kmem_cache_destroy(iova_magazine_cache); } mutex_unlock(&iova_cache_mutex); }