From patchwork Mon Feb 5 15:32:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Robin Murphy X-Patchwork-Id: 13545691 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5A00AC4828D for ; Mon, 5 Feb 2024 15:32:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 584646B0080; Mon, 5 Feb 2024 10:32:50 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5359E6B0082; Mon, 5 Feb 2024 10:32:50 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 275226B0083; Mon, 5 Feb 2024 10:32:50 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 0DA6B6B0080 for ; Mon, 5 Feb 2024 10:32:50 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id D07D5A183A for ; Mon, 5 Feb 2024 15:32:49 +0000 (UTC) X-FDA: 81758142858.23.8849425 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf05.hostedemail.com (Postfix) with ESMTP id 36E1410000C for ; Mon, 5 Feb 2024 15:32:48 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf05.hostedemail.com: domain of robin.murphy@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=robin.murphy@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1707147168; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yY+4M7kMt2BVJso2Lto96zv05XHdud4jM/u756+6AO4=; b=Kt0VtsXUlfBb81FzfrplvuzEQIdTZR5tuq4T9Yr165ZqW6hmiWdODiGvUdoY2CC0xsLyL8 YmzDJnMKkia8/RPka2/fI7NQ3MPwCyMQ4WNt4rPSzecd5pAidN5tF1ZVz9myFYGZe7eKMe U65maGW9kE3E4nxhClapP20uGTiLehQ= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf05.hostedemail.com: domain of robin.murphy@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=robin.murphy@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1707147168; a=rsa-sha256; cv=none; b=GEzgRiu8c3LheBRVpmWfQC4sWAghilNykAupzwp6r1JwVsaB9IJXcREt8RfkNvRZfVHM/u 3RynwrMVT6hTIo7PlxGPidkTliSygFLgEzInPGDgfLLxDjwBzRIinSafZwK2R4kZ7V8mpb mxeSiBwfBSZp/pYOhexoeb4sbjM1soE= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 36CBDDA7; Mon, 5 Feb 2024 07:33:30 -0800 (PST) Received: from e121345-lin.cambridge.arm.com (e121345-lin.cambridge.arm.com [10.1.196.40]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 6A7DF3F5A1; Mon, 5 Feb 2024 07:32:46 -0800 (PST) From: Robin Murphy To: joro@8bytes.org Cc: will@kernel.org, pasha.tatashin@soleen.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-mm@kvack.org, rientjes@google.com, yosryahmed@google.com, john.g.garry@oracle.com Subject: [PATCH v3 1/3] iommu/iova: Tidy up iova_cache_get() failure Date: Mon, 5 Feb 2024 15:32:39 +0000 Message-Id: X-Mailer: git-send-email 2.39.2.101.g768bb238c484.dirty In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 36E1410000C X-Stat-Signature: 69o7ezzocz8d799bjwn6ngwcd8kzuzup X-HE-Tag: 1707147168-446324 X-HE-Meta: U2FsdGVkX1/uPhVBu0l9lcSbz+MUeHndHM1svwDeCABCDWPc+ZeVjkrnJ7UnYEASAl+rL9M77oDYdD67I4HHpD9uXU5ubJdJndBBmbgf5dEEuUxPjq5o9UlbakgPw7PkYg6QnCnBpP74B6k9zTonIsFsIW57hmpbJJXfTyIkzqdvKBERBp4+mDNosIbTUoeIpg7elE4/P+JMQjnu4ngCnBGW6t/but1EDZQvM6nDyuqT8xlcAtb7BECGJlHSmm8V9irk5pzo6K67eUvmQJDtdTjLTo8YrEeizYMEF5C0yBm1ghpSNpNWOi/QBvTQVPDpkLoiOzVQklMe++/iQt7vu8X8s7KOmBUAsuhkGNUDgHCJ58NHkSd2OZ0y/FiXUgBGgbu3yvmM0BkSJzpYXOp3vf5p+Nw+56J03+6fvKGUArhMmHdYVvyk1U74pk/qnpr0+9ZwtKa+scqYxWVJzfkj0zidpxxmRgaz+EkDAnVNKa1UjKcylLHIyB+Oq/4rdxXwZ7H18y/H88iwuReJjZH64zBSrlzvK0JWJ36sh0CZdgQmaMDJD+wJb9SmsJWsDTO0Lh1q9v9kl7BscDF+uwzIKHU3wTP9SDlPJxOl3mD3iPmu26/6fEpOu/ubdbdRo5YwumGXZyePELsNvs8vvkBIvQLSC+3cTC7bWFtO52MRBXE38csjD6655PNKXAoI+mxng1N85YCaURNYkzSJrNLLE/Oxy5M8w05bPb+uyGZspKLn6eW6sbRZIGZYopvq4YsOjkmdiY2hYoOzyPx0oh3n9FFrK6OMgK1apIqvXCvR9q9d1Wa1MoELP6y3ucKa5O44ShgJDQKHyAlc0JXE/5Lr56ghrwuy9pxRKxyrozFy7QY7kTRGtIDw16K8azYv1wTiP2LXX9lTPvamGpcLlYSbe8zAdH4+iwjvmiYyxWzBdPjn/vxXmz332eP4nPH20d+XdbeyBCd8qxw+AS8u0fH udVaQsXo sw09m+qgvClctXrXn4u0R39vBnls7WlyIeT6RqblRhkpFi5g1gTUfKr4a8QoLsh55fnKasEj8cQble1sMmfXbe17UG4zPSR/t45oKxiUqZaGTTjtqtv1xoNT8QRrwit+hNZxBL6tInjpt5ER/IsXzzh3cS94hcCkM4JQUqvc6lR2F7W7mz7xF4j3D8d7pXf2Pes/ObrFKzerimRE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Failure handling in iova_cache_get() is a little messy, and we'd like to add some more to it, so let's tidy up a bit first. By leaving the hotplug handler until last we can take advantage of kmem_cache_destroy() being NULL-safe to have a single cleanup label. We can also improve the error reporting, noting that kmem_cache_create() already screams if it fails, so that one is redundant. Signed-off-by: Robin Murphy Acked-by: David Rientjes Reviewed-by: Pasha Tatashin Reviewed-by: John Garry Reviewed-by: Jerry Snitselaar --- drivers/iommu/iova.c | 33 ++++++++++++++++----------------- 1 file changed, 16 insertions(+), 17 deletions(-) diff --git a/drivers/iommu/iova.c b/drivers/iommu/iova.c index d30e453d0fb4..cf95001d85c0 100644 --- a/drivers/iommu/iova.c +++ b/drivers/iommu/iova.c @@ -254,26 +254,20 @@ static void free_iova_mem(struct iova *iova) int iova_cache_get(void) { + int err = -ENOMEM; + mutex_lock(&iova_cache_mutex); if (!iova_cache_users) { - int ret; + iova_cache = kmem_cache_create("iommu_iova", sizeof(struct iova), 0, + SLAB_HWCACHE_ALIGN, NULL); + if (!iova_cache) + goto out_err; - ret = cpuhp_setup_state_multi(CPUHP_IOMMU_IOVA_DEAD, "iommu/iova:dead", NULL, - iova_cpuhp_dead); - if (ret) { - mutex_unlock(&iova_cache_mutex); - pr_err("Couldn't register cpuhp handler\n"); - return ret; - } - - iova_cache = kmem_cache_create( - "iommu_iova", sizeof(struct iova), 0, - SLAB_HWCACHE_ALIGN, NULL); - if (!iova_cache) { - cpuhp_remove_multi_state(CPUHP_IOMMU_IOVA_DEAD); - mutex_unlock(&iova_cache_mutex); - pr_err("Couldn't create iova cache\n"); - return -ENOMEM; + err = cpuhp_setup_state_multi(CPUHP_IOMMU_IOVA_DEAD, "iommu/iova:dead", + NULL, iova_cpuhp_dead); + if (err) { + pr_err("IOVA: Couldn't register cpuhp handler: %pe\n", ERR_PTR(err)); + goto out_err; } } @@ -281,6 +275,11 @@ int iova_cache_get(void) mutex_unlock(&iova_cache_mutex); return 0; + +out_err: + kmem_cache_destroy(iova_cache); + mutex_unlock(&iova_cache_mutex); + return err; } EXPORT_SYMBOL_GPL(iova_cache_get); From patchwork Mon Feb 5 15:32:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Robin Murphy X-Patchwork-Id: 13545692 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1C3F7C48295 for ; Mon, 5 Feb 2024 15:32:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EE8CA6B0082; Mon, 5 Feb 2024 10:32:51 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E84736B0083; Mon, 5 Feb 2024 10:32:51 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CAE696B0085; Mon, 5 Feb 2024 10:32:51 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id B9A476B0082 for ; Mon, 5 Feb 2024 10:32:51 -0500 (EST) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 748281201DB for ; Mon, 5 Feb 2024 15:32:51 +0000 (UTC) X-FDA: 81758142942.06.523B61A Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf09.hostedemail.com (Postfix) with ESMTP id B06AC14002E for ; Mon, 5 Feb 2024 15:32:49 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf09.hostedemail.com: domain of robin.murphy@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=robin.murphy@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1707147169; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iYOjIPp5YF4pHRIdEAilCHpp38B3kyUgFoGeW/y2Mfw=; b=xeZAVpFyNeeNKxr/lyaRkukNKygUTI39AysxkXzo2sI6Jqv2podL3y/wqCW6YiPqdmOmt+ fWi+mhioI4BjQM3D/i6bEtGcW5gyRkgUFn6F25elG3HlzMbLKMGYqMn9/1IwFskEKjbXBI rYY7iy21iLk6wIcvlYAX6/zH2xwn6XU= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf09.hostedemail.com: domain of robin.murphy@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=robin.murphy@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1707147169; a=rsa-sha256; cv=none; b=LEEFr2Aivf7gWae6bNNmUqkiwjSr38AEkSRmdlUct5HIlPe06uU3YSogcnjKbzo8q9obqz +aswXK/vsVIuOX6aPDNI1xUD79UufAVpYkyxsOMfT5Hjiq+DABXgUaYpG7FP2o5kdjSB5S BDlIwlcdaCXuDea8ME0+attJ79loDVs= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9287C12FC; Mon, 5 Feb 2024 07:33:31 -0800 (PST) Received: from e121345-lin.cambridge.arm.com (e121345-lin.cambridge.arm.com [10.1.196.40]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id C60753F5A1; Mon, 5 Feb 2024 07:32:47 -0800 (PST) From: Robin Murphy To: joro@8bytes.org Cc: will@kernel.org, pasha.tatashin@soleen.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-mm@kvack.org, rientjes@google.com, yosryahmed@google.com, john.g.garry@oracle.com Subject: [PATCH v3 2/3] iommu/iova: Reorganise some code Date: Mon, 5 Feb 2024 15:32:40 +0000 Message-Id: X-Mailer: git-send-email 2.39.2.101.g768bb238c484.dirty In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: grxw5ffewu33dscqa6nw3f683aky4o1y X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: B06AC14002E X-HE-Tag: 1707147169-861162 X-HE-Meta: U2FsdGVkX1/c9w5p6VCzlfgacjds4t7lQd+qubot0LAovE1qFUT0d9BcLZxcISENyJ9Xb167LE5EXiXU5YgwBNOlQyHjAsBW71EOkxmH2oYrMSqHJQ5L8LmqQkKEKX607Mdgcto64OSVf2pf8zS9rfOnk3bmN+HdCTkyNLZi88WZYXyIfiiY1zkpggKYzAAlouZZenuDKTfILNkm5/YlJOqE+UEc6o4qtqhovZJj62onC/XEi35zRfC9GO4AXRvBfDnaVd5b9bzXFvZamaSsmN9lMSu+V3Fw2gTKDJCj+45xmWuRw3ofY0M54hCVIoi8UlPRydaQVO8ONgZCO29EDp6TykRUboDT0IXQ/mHKwPAL/Upfut3A4SA4HgC5e5NtMu2wqrCI3E031NmNJrmEH1XP/VMZfrMrxiH4AquVMyph3S6zYUidzKcsJr9CjosmMmCe32fuqzb/moD+J2Ukp595VBTcNjMNDC2qpwaZ+358vxTNVRflECZuWedZbikrTpSgjWmIwH1CXSjHQZbGEBpH2G2d49tJDw3tOg6Tut0aXwtnDb0K8gEg7HNVKXNBeY46LKT/564lyJy5bxsOQmNH8yQzZi0MQNyBsDutDdIDY/ZxLWyAZH4toO7dVdpFi91V6MP0MeA9LOHvxnB2X9NoeUJn3VwV46lHtnsCr4mu2RdKOtbW5PEWGCt8s8ceuYdvEYl6e3hV4wcRUQhAhmyUJQUOukv0uSROUtSsbDlKH1uAi/UyaJq684RWDJM0vBcB+OEp+J9f27gsJTF7/XQnFwv2M8d/xsx72byWR7OedsOKog4WQoNkaJBXdTPG1M2NzOznZN+e88ghGilfbRAwDzgUv5FD8l/VoDDCEPSPIjnvJg2z20m4mG6YFmhGY20OCyNLL0bahngVIjyH9+BTiQSfg81qqFNFwFnJwOH5LhI4rfRXlINtam+trAzbcRxvwUECRTBzvzPEDGa Zlnuiqoj GIAp03ra4kUAk80TZw8OaJmQzm9b6+FRcGdMXM8bjfUZQbljwg4ZVvww35Wx0JeXtGuiZrjoDXBD28S0vJCtApb5cwMZQiPEb1iilgOjfMFtLe2qX+BGvnBS6DKxSUc1XDGxUO+lv0Po/wWumCOlqNxiRdq9sc2UsIuUD8MrJibEMc78UF3RxyPIaYg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The iova_cache_{get,put}() calls really represent top-level lifecycle management for the whole IOVA library, so it's long been rather confusing to have them buried right in the middle of the allocator implementation details. Move them to a more expected position at the end of the file, where it will then also be easier to expand them. With this, we can also move the rcache hotplug handler (plus another stray function) into the rcache portion of the file. Signed-off-by: Robin Murphy Acked-by: David Rientjes Reviewed-by: Pasha Tatashin Reviewed-by: John Garry Reviewed-by: Jerry Snitselaar --- drivers/iommu/iova.c | 128 +++++++++++++++++++++---------------------- 1 file changed, 64 insertions(+), 64 deletions(-) diff --git a/drivers/iommu/iova.c b/drivers/iommu/iova.c index cf95001d85c0..b5de865ee50b 100644 --- a/drivers/iommu/iova.c +++ b/drivers/iommu/iova.c @@ -24,24 +24,8 @@ static bool iova_rcache_insert(struct iova_domain *iovad, static unsigned long iova_rcache_get(struct iova_domain *iovad, unsigned long size, unsigned long limit_pfn); -static void free_cpu_cached_iovas(unsigned int cpu, struct iova_domain *iovad); static void free_iova_rcaches(struct iova_domain *iovad); - -unsigned long iova_rcache_range(void) -{ - return PAGE_SIZE << (IOVA_RANGE_CACHE_MAX_SIZE - 1); -} - -static int iova_cpuhp_dead(unsigned int cpu, struct hlist_node *node) -{ - struct iova_domain *iovad; - - iovad = hlist_entry_safe(node, struct iova_domain, cpuhp_dead); - - free_cpu_cached_iovas(cpu, iovad); - return 0; -} - +static void free_cpu_cached_iovas(unsigned int cpu, struct iova_domain *iovad); static void free_global_cached_iovas(struct iova_domain *iovad); static struct iova *to_iova(struct rb_node *node) @@ -252,53 +236,6 @@ static void free_iova_mem(struct iova *iova) kmem_cache_free(iova_cache, iova); } -int iova_cache_get(void) -{ - int err = -ENOMEM; - - mutex_lock(&iova_cache_mutex); - if (!iova_cache_users) { - iova_cache = kmem_cache_create("iommu_iova", sizeof(struct iova), 0, - SLAB_HWCACHE_ALIGN, NULL); - if (!iova_cache) - goto out_err; - - err = cpuhp_setup_state_multi(CPUHP_IOMMU_IOVA_DEAD, "iommu/iova:dead", - NULL, iova_cpuhp_dead); - if (err) { - pr_err("IOVA: Couldn't register cpuhp handler: %pe\n", ERR_PTR(err)); - goto out_err; - } - } - - iova_cache_users++; - mutex_unlock(&iova_cache_mutex); - - return 0; - -out_err: - kmem_cache_destroy(iova_cache); - mutex_unlock(&iova_cache_mutex); - return err; -} -EXPORT_SYMBOL_GPL(iova_cache_get); - -void iova_cache_put(void) -{ - mutex_lock(&iova_cache_mutex); - if (WARN_ON(!iova_cache_users)) { - mutex_unlock(&iova_cache_mutex); - return; - } - iova_cache_users--; - if (!iova_cache_users) { - cpuhp_remove_multi_state(CPUHP_IOMMU_IOVA_DEAD); - kmem_cache_destroy(iova_cache); - } - mutex_unlock(&iova_cache_mutex); -} -EXPORT_SYMBOL_GPL(iova_cache_put); - /** * alloc_iova - allocates an iova * @iovad: - iova domain in question @@ -653,6 +590,11 @@ struct iova_rcache { struct delayed_work work; }; +unsigned long iova_rcache_range(void) +{ + return PAGE_SIZE << (IOVA_RANGE_CACHE_MAX_SIZE - 1); +} + static struct iova_magazine *iova_magazine_alloc(gfp_t flags) { struct iova_magazine *mag; @@ -989,5 +931,63 @@ static void free_global_cached_iovas(struct iova_domain *iovad) spin_unlock_irqrestore(&rcache->lock, flags); } } + +static int iova_cpuhp_dead(unsigned int cpu, struct hlist_node *node) +{ + struct iova_domain *iovad; + + iovad = hlist_entry_safe(node, struct iova_domain, cpuhp_dead); + + free_cpu_cached_iovas(cpu, iovad); + return 0; +} + +int iova_cache_get(void) +{ + int err = -ENOMEM; + + mutex_lock(&iova_cache_mutex); + if (!iova_cache_users) { + iova_cache = kmem_cache_create("iommu_iova", sizeof(struct iova), 0, + SLAB_HWCACHE_ALIGN, NULL); + if (!iova_cache) + goto out_err; + + err = cpuhp_setup_state_multi(CPUHP_IOMMU_IOVA_DEAD, "iommu/iova:dead", + NULL, iova_cpuhp_dead); + if (err) { + pr_err("IOVA: Couldn't register cpuhp handler: %pe\n", ERR_PTR(err)); + goto out_err; + } + } + + iova_cache_users++; + mutex_unlock(&iova_cache_mutex); + + return 0; + +out_err: + kmem_cache_destroy(iova_cache); + mutex_unlock(&iova_cache_mutex); + return err; +} +EXPORT_SYMBOL_GPL(iova_cache_get); + +void iova_cache_put(void) +{ + mutex_lock(&iova_cache_mutex); + if (WARN_ON(!iova_cache_users)) { + mutex_unlock(&iova_cache_mutex); + return; + } + iova_cache_users--; + if (!iova_cache_users) { + cpuhp_remove_multi_state(CPUHP_IOMMU_IOVA_DEAD); + kmem_cache_destroy(iova_cache); + } + mutex_unlock(&iova_cache_mutex); +} +EXPORT_SYMBOL_GPL(iova_cache_put); + MODULE_AUTHOR("Anil S Keshavamurthy "); MODULE_LICENSE("GPL"); From patchwork Mon Feb 5 15:32:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Robin Murphy X-Patchwork-Id: 13545693 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C844AC4828D for ; Mon, 5 Feb 2024 15:32:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 02C6E6B0085; Mon, 5 Feb 2024 10:32:57 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id EF8866B0087; Mon, 5 Feb 2024 10:32:56 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DE7DD6B0088; Mon, 5 Feb 2024 10:32:56 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id CD2A26B0085 for ; Mon, 5 Feb 2024 10:32:56 -0500 (EST) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 4658E120A57 for ; Mon, 5 Feb 2024 15:32:56 +0000 (UTC) X-FDA: 81758143152.10.300361F Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf02.hostedemail.com (Postfix) with ESMTP id 55CD580026 for ; Mon, 5 Feb 2024 15:32:54 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf02.hostedemail.com: domain of robin.murphy@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=robin.murphy@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1707147174; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=unFrpB4e3+scnqjyvzclIGP74f4AAsET/Bk0JpL7ly8=; b=zIaJVWz8olUQR5PW9KsZAtRYI16c9YRtyTa7TKKvVtEOV5LDaSOMVaSf7V9CmlM1X2jTAg Iz/ugL/exZwjclc6YFrOqEBHqdZlad2DoBx/nsIIctKG5E6qVjww7eLMAyxZIrSiW6BtKm YABvVOBwUzsYXAt2GTW/kKLnljjSNVc= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf02.hostedemail.com: domain of robin.murphy@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=robin.murphy@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1707147174; a=rsa-sha256; cv=none; b=3TPaC6TbbzEBuFREnjUKb4z5cJuDpjsD6Jz0Pf1HcLs265fMbu9iyoxMERMuy+oGzmBy1N krfqjd9CvAOG08PNAb543n7sAe8Weqiv/FDhBb5O6jXROnIxe7qP+AUCtp2p5Esu/tc/Yp nM0MtVNIv1g+5EHB6hRb2nmNh4m7xEo= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 14C211FB; Mon, 5 Feb 2024 07:33:36 -0800 (PST) Received: from e121345-lin.cambridge.arm.com (e121345-lin.cambridge.arm.com [10.1.196.40]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 483403F5A1; Mon, 5 Feb 2024 07:32:52 -0800 (PST) From: Robin Murphy To: joro@8bytes.org Cc: will@kernel.org, pasha.tatashin@soleen.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-mm@kvack.org, rientjes@google.com, yosryahmed@google.com, john.g.garry@oracle.com Subject: [PATCH v3 3/3] iommu/iova: use named kmem_cache for iova magazines Date: Mon, 5 Feb 2024 15:32:41 +0000 Message-Id: X-Mailer: git-send-email 2.39.2.101.g768bb238c484.dirty In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: 55CD580026 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: iejf5ktcrp6zi97we7igshe8cm4s9dh9 X-HE-Tag: 1707147174-694800 X-HE-Meta: U2FsdGVkX19cohhBEP+JN7v/unLQcxeiMov2KA4r6SyMFS79xHkd3Mb+7sPMqASD9wOPy90PNFiziyqAI2ys6WKGO8EzzIWavJcqXHkRvPgXPGES52J3/oIh0qPZY3+s00LGw3GD1+YV/vmRbdL/Y4S3ptNyeIRtxXQncEHfWsXfhf33MlwcZiIPO0zYOqhekmmggb2fw3xHuyY0tuKnGFsLVdrMiAOVBRB05WDQG5NIUoEc6K7io683BCDAwqfsX0fkIBwwDm0jXNeUfYGttjWUwkp7AxluL4AcqyS0sNqT5VLMe1/p6QY+rOU5ZpS9PUDDlIVa366E2LYgZyImAPFw0ves47lFbURGGSN/Dfx1H3qjvgkhE/xLJBDJnAIHK8Mbv8+BPEQOcMNMBp6NLi6AwQQe7mXnM3X/8ESEoYKOBRUNPEfElcgFEg7gKDQpyDc+s1BbSFK6d4LcWqdmXpZwe15z+xkTQFn6zY0+ttkzR/GCWD5UFk9morKf0qx/qYFN5DqtplL14zTnFDE31tDpHHuIZdTurBAwHUyXhAxzL2Cfikc2r/cdEZt6ZZNIbxf2bhdHGZvlISOelVZ7VEGOP3ANA3IZ2g/Pf18m7KeWRdXnVamTi9zOgdJCJ7ORUzYR46+JOXTZgWtMx7B9a+zQ9Mi1xHQejxjKHUE4ZAytuU7/PhFI/tiZPPCMp7U9m09eoDw0NS8AkVEXZJ19nZUkSc6pBZSVVsNPUxMTye/DI09dSAQ4Wz131Xjbomh5OQJiayoaWf/uS9q0DCVIlooXKtlsCkmfTuaX1d6mmaZyjhwjsH2CUDG0eEW16zzB9QMPmU63iPz3fe4m03dyF7eWvkNlDwrpI2g/FkRVwFV7Y9yUgg1d5C07c3gJb9jqK3IZi1yr2imu3FvGADz9KclWXyxUkLQF2DU12hrqnbhXGIr1lbYfeD/Q+tjhjeiTF6rnnTWqwH8M+rdpFP/ /UCLhe5A BzNByODbzkyLiZYA7HsETsS07uis/6f/PvG+dys5Zt3J9FYVBKX2Oh3JDkaabkXbcBnqURVElowBflDiufHD6VIbLy+HdcgUFb/8hcQmWP2RXEEvQhKiv2YzXqJm82jea1GyHArsKe73XsCHAuheskEUdMLUNAr0El6QDLyLQ0gIUeAml3nwEHCHwdlSb/QnX6HT4H7nUlG9+ASKVsRFS9hMrdeUa6Y9SoKo+Bd6s9kskl6wswO4bBAN9fxINXbOmOF41/Zojk7r9IeY2PlbKScDvyx+uQ35QxLlU X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Pasha Tatashin The magazine buffers can take gigabytes of kmem memory, dominating all other allocations. For observability purpose create named slab cache so the iova magazine memory overhead can be clearly observed. With this change: > slabtop -o | head Active / Total Objects (% used) : 869731 / 952904 (91.3%) Active / Total Slabs (% used) : 103411 / 103974 (99.5%) Active / Total Caches (% used) : 135 / 211 (64.0%) Active / Total Size (% used) : 395389.68K / 411430.20K (96.1%) Minimum / Average / Maximum Object : 0.02K / 0.43K / 8.00K OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME 244412 244239 99% 1.00K 61103 4 244412K iommu_iova_magazine 91636 88343 96% 0.03K 739 124 2956K kmalloc-32 75744 74844 98% 0.12K 2367 32 9468K kernfs_node_cache On this machine it is now clear that magazine use 242M of kmem memory. Acked-by: David Rientjes Signed-off-by: Pasha Tatashin [ rm: adjust to rework of iova_cache_{get,put} ] Signed-off-by: Robin Murphy Reviewed-by: John Garry Reviewed-by: Jerry Snitselaar --- drivers/iommu/iova.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/drivers/iommu/iova.c b/drivers/iommu/iova.c index b5de865ee50b..d59d0ea2fd21 100644 --- a/drivers/iommu/iova.c +++ b/drivers/iommu/iova.c @@ -590,6 +590,8 @@ struct iova_rcache { struct delayed_work work; }; +static struct kmem_cache *iova_magazine_cache; + unsigned long iova_rcache_range(void) { return PAGE_SIZE << (IOVA_RANGE_CACHE_MAX_SIZE - 1); @@ -599,7 +601,7 @@ static struct iova_magazine *iova_magazine_alloc(gfp_t flags) { struct iova_magazine *mag; - mag = kmalloc(sizeof(*mag), flags); + mag = kmem_cache_alloc(iova_magazine_cache, flags); if (mag) mag->size = 0; @@ -608,7 +610,7 @@ static struct iova_magazine *iova_magazine_alloc(gfp_t flags) static void iova_magazine_free(struct iova_magazine *mag) { - kfree(mag); + kmem_cache_free(iova_magazine_cache, mag); } static void @@ -953,6 +955,12 @@ int iova_cache_get(void) if (!iova_cache) goto out_err; + iova_magazine_cache = kmem_cache_create("iommu_iova_magazine", + sizeof(struct iova_magazine), + 0, SLAB_HWCACHE_ALIGN, NULL); + if (!iova_magazine_cache) + goto out_err; + err = cpuhp_setup_state_multi(CPUHP_IOMMU_IOVA_DEAD, "iommu/iova:dead", NULL, iova_cpuhp_dead); if (err) { @@ -968,6 +976,7 @@ int iova_cache_get(void) out_err: kmem_cache_destroy(iova_cache); + kmem_cache_destroy(iova_magazine_cache); mutex_unlock(&iova_cache_mutex); return err; } @@ -984,6 +993,7 @@ void iova_cache_put(void) if (!iova_cache_users) { cpuhp_remove_multi_state(CPUHP_IOMMU_IOVA_DEAD); kmem_cache_destroy(iova_cache); + kmem_cache_destroy(iova_magazine_cache); } mutex_unlock(&iova_cache_mutex); }