From patchwork Mon Jun 12 15:31:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13276860 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BF170C88CB2 for ; Mon, 12 Jun 2023 15:32:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5D22A8E0008; Mon, 12 Jun 2023 11:32:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5826F8E0002; Mon, 12 Jun 2023 11:32:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 44AA98E0008; Mon, 12 Jun 2023 11:32:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 35E478E0002 for ; Mon, 12 Jun 2023 11:32:29 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id D787DA027E for ; Mon, 12 Jun 2023 15:32:28 +0000 (UTC) X-FDA: 80894487576.14.179E041 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf21.hostedemail.com (Postfix) with ESMTP id DBA521C001B for ; Mon, 12 Jun 2023 15:32:26 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=none; spf=pass (imf21.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686583947; a=rsa-sha256; cv=none; b=ytD2jaEk05+luPy1HWzqPdxtwdURLbtmfqhpVEqtpKdTpszaz8lsCGHY2HG8GreZ4yTIhb +U6Cycxk9aterEsgfNsj5iWGFsqBu99I74UKZu571iQJUi4cCg+zjAwVKY+AmuRAQd0vRZ rwJG8ciLQ4FkhtIAca0/+ZKtI3lrJqY= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=none; spf=pass (imf21.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686583947; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=98XE5a7BhLm7ZzEjWpMumSTI8jrHUeYv75m7kBYZXFY=; b=3jyRWphnbel38Lsz7b3ojjR9Bc1Kfdv5uTQkJJXRu0fMvm/nHi3oyXVdzRAQOze2ZU8p+3 pen+BDrhxCRAgpmpnz2BrWcxUfrN3ZNP6NOcXgVxYaxloGWbMiimR5Fg5yppAPqtdnvHnf 1+kg5ZCDg6Ct3JUl79T9N9vCrpx8QwM= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 0333762ADF; Mon, 12 Jun 2023 15:32:26 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 76150C4339B; Mon, 12 Jun 2023 15:32:21 +0000 (UTC) From: Catalin Marinas To: Andrew Morton , Linus Torvalds Cc: Christoph Hellwig , Robin Murphy , Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Jonathan Cameron , Vlastimil Babka , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v7 04/17] mm/slab: Limit kmalloc() minimum alignment to dma_get_cache_alignment() Date: Mon, 12 Jun 2023 16:31:48 +0100 Message-Id: <20230612153201.554742-5-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230612153201.554742-1-catalin.marinas@arm.com> References: <20230612153201.554742-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: DBA521C001B X-Stat-Signature: oqwkj7z78jdamu4zsnqau71ecdsm5fc3 X-HE-Tag: 1686583946-514937 X-HE-Meta: U2FsdGVkX1+GSdJyVsz2H5JIKY4C+30Q+/rZNT1qW5aRoOIr87R2qQtiTcSr26opJ3JK6uyMBRF0F8546LQ/pq84U45QKAlv3ewwMUP+/kzLCombYIQ4ZizqMZKia5ugXOyGi1fTws/jGDAaQUA0L9k7bP7/p9dcJbGN0meyFJlwSGC8iQChLGrCUuQUfvKPbtrSdqkMR8FoKB5VdN6xJuBEjMVrDV7k+JoKMRljKX/NEgTtqvYpwifx/jPeK0by8Hqna7XjPn5zEtnGlJ0eEAGIepa8KnUfWw+dTT2g8JGmJyZhjD6AiKjr9Q3IqR2ifR1Rjnv1RmvqwfLyFRXznhbia1+XXqzlf60EBcZsFuIaBl+8ZNSkMj7VvhlBape3IbJc/wy6NBJizMBRqpUxfRFgBRsFys3bpfE5nmwKsWGgZxrekNEMFwxoOi185VhDSHsQN731jxRb1rmKC3zh9Zu81If5DuiQ3aLWsPe+6nCPfTTdyqEsM5D+ca2QTf5149HN9ww7c9LXZQeO4Y0F41tEmsfKLoqj6G+VCIujGJkaKpL4Ama/c8+VVlBIQ/JESyb3tZa9vobksr6IVxR6u/I++Lt4Bcexr/WyHj1jyGkeNkY0L7XZpDFN31UuxGvkqwnPSZHI8k4kCtbj5g6Bm92soebGWV95eaCAOFIygttGNZLbeOix2INFkJCVqtY7bkF8w9pGN4gk9o+f4nsN+nivm2YQirXnRsAngc/oLCiyZpSkmLTnQVH4iEUZFyf7mbwTLrX+uDLxSnkhq1cvC2wK0qzvwzgR5YamZM6fOLk3ygmEX/I2cFX260c0ffM/Z7DA//mQrr2cLCKinGTGUw46fcEOeYdxVyzmfbu3856fFx7PKn9qpQFHfHttt6QS5JTGTtVjC6Uyh3mVz8PNSisseFNjU0mDV5Mz78DZnH1d0GAcgrzO5OD/jzOzpNzOfWgTz/syp73uRPGktSn xDpw1m6s 8qDT/gU1lSO7d2ZXhnchqA0xnp2OPwUFAYVmOIXjRKHdesfTYg5G2T2ftghoQQy2x1ZI3OQO9XfZLCW6NTAjesMwOGBQSrU7HneEpYucrBDy0955BoOKLX8i3KnwLcUmdsO6y+kHse4cFi49fDU4IbAqmGbgVDuseHbpSWJBYYmny4jMVl+Lg3Ea5JN2AtOn3Lflaq8tRj2IaHPJPGx7Ifg9SNl9O1IF9l4tblNRoNIZ/qPQ7nNbNdxR4MjGd36iM5EdcvzMXbVGt9+5fXX/ytVhUvRL1CwhT80EQBZpjEJHOKcYMiqMWZcQWwTfUh6p+0w7GdMYsfWNVbI/I8E/QQkizUJ8H67LW0d6H90LK6+0xpCQ7rrS7l+L77DIJgCgfTsUuea6cpwyJ7htb+F8iPJWyJ24BaWBvJF+RG2/wg5/UmkRLMSG63A9UXroGnZidKLmqgDLgCUwqB8VwpDwqIfkelnCtD8A+iWEa7Oce2MMSkBbr1oCZsae8iH2e4ZGiW27zJqnOPaIsXSaShLy4T7haOUf7aihr54xDu8yp9k/YSCbXZoN4D7EPI3utG4vmd+kZwUuzuAwbs392c71UWikH3A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Do not create kmalloc() caches which are not aligned to dma_get_cache_alignment(). There is no functional change since for current architectures defining ARCH_DMA_MINALIGN, ARCH_KMALLOC_MINALIGN equals ARCH_DMA_MINALIGN (and dma_get_cache_alignment()). On architectures without a specific ARCH_DMA_MINALIGN, dma_get_cache_alignment() is 1, so no change to the kmalloc() caches. Signed-off-by: Catalin Marinas Reviewed-by: Vlastimil Babka Cc: Andrew Morton Cc: Christoph Hellwig Cc: Robin Murphy Tested-by: Isaac J. Manjarres --- mm/slab_common.c | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/mm/slab_common.c b/mm/slab_common.c index 7f069159aee2..7c6475847fdf 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -17,6 +17,7 @@ #include #include #include +#include #include #include #include @@ -862,9 +863,18 @@ void __init setup_kmalloc_cache_index_table(void) } } +static unsigned int __kmalloc_minalign(void) +{ + return dma_get_cache_alignment(); +} + void __init new_kmalloc_cache(int idx, enum kmalloc_cache_type type, slab_flags_t flags) { + unsigned int minalign = __kmalloc_minalign(); + unsigned int aligned_size = kmalloc_info[idx].size; + int aligned_idx = idx; + if ((KMALLOC_RECLAIM != KMALLOC_NORMAL) && (type == KMALLOC_RECLAIM)) { flags |= SLAB_RECLAIM_ACCOUNT; } else if (IS_ENABLED(CONFIG_MEMCG_KMEM) && (type == KMALLOC_CGROUP)) { @@ -877,9 +887,17 @@ new_kmalloc_cache(int idx, enum kmalloc_cache_type type, slab_flags_t flags) flags |= SLAB_CACHE_DMA; } - kmalloc_caches[type][idx] = create_kmalloc_cache( - kmalloc_info[idx].name[type], - kmalloc_info[idx].size, flags); + if (minalign > ARCH_KMALLOC_MINALIGN) { + aligned_size = ALIGN(aligned_size, minalign); + aligned_idx = __kmalloc_index(aligned_size, false); + } + + if (!kmalloc_caches[type][aligned_idx]) + kmalloc_caches[type][aligned_idx] = create_kmalloc_cache( + kmalloc_info[aligned_idx].name[type], + aligned_size, flags); + if (idx != aligned_idx) + kmalloc_caches[type][idx] = kmalloc_caches[type][aligned_idx]; /* * If CONFIG_MEMCG_KMEM is enabled, disable cache merging for