From patchwork Wed May 31 15:48:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13262454 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 26FFBC77B7A for ; Wed, 31 May 2023 15:48:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B46958E0005; Wed, 31 May 2023 11:48:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AF70E8E0001; Wed, 31 May 2023 11:48:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9BE6B8E0005; Wed, 31 May 2023 11:48:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 8E8418E0001 for ; Wed, 31 May 2023 11:48:49 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 49A8D16029D for ; Wed, 31 May 2023 15:48:49 +0000 (UTC) X-FDA: 80850983178.08.FFB9F5E Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf22.hostedemail.com (Postfix) with ESMTP id 81BE4C0020 for ; Wed, 31 May 2023 15:48:47 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=none; spf=pass (imf22.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1685548127; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yoW/ReeYfZSTzC2TZN9dmK/8gq9+nyhtUUc+/CgA4gM=; b=TdMsCRk1kPhsMljrEX1njk7ooKOc362faqlsdLoFP/qjati7hvw7VyLR9M/NZTtohSOhC3 raqKZRkaWhkD1qrIN4PeqljqODk4Y9FFvNtH7jph81rEWwq+TTmuDE76rFMRPbI5aUeMv2 9NKD5bjpfbJcTiMPMJGcpf3OvMEZ5/Y= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1685548127; a=rsa-sha256; cv=none; b=2e4yLokbZPnSeVI66gfaQgVMPYjEh468L8bBZYsWki0Ror5UO/jedmM3RKgM8T4L8LjO0m f2aDeVEEO/7RgxKJYGs4zJcIWc96tLULIKLJV+++f3wwh6y4/AMbuOydNZAoZvf4r7qlEr hD7ZWZlGFDdoVgbo8nO5fxKFBcposU0= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=none; spf=pass (imf22.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id B2C546380B; Wed, 31 May 2023 15:48:46 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 69AD3C4339E; Wed, 31 May 2023 15:48:42 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Christoph Hellwig , Robin Murphy Cc: Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Jonathan Cameron , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v6 01/17] mm/slab: Decouple ARCH_KMALLOC_MINALIGN from ARCH_DMA_MINALIGN Date: Wed, 31 May 2023 16:48:20 +0100 Message-Id: <20230531154836.1366225-2-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230531154836.1366225-1-catalin.marinas@arm.com> References: <20230531154836.1366225-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Stat-Signature: zopfwupbqjowpuqqdcfb157o1k648z3g X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 81BE4C0020 X-Rspam-User: X-HE-Tag: 1685548127-769303 X-HE-Meta: U2FsdGVkX19TDTxAIy0ZgL8f6FmipVA/6s73Zf71xZGNC6ojKqdxiJxOe1YFXvW14kmWunGxEDAcMpjpNUCFjo3yz02UeHxFHSbZaaGN1VV+WrEHGyUebR8sY5tHjkt8GiGliK288TgTmFe3xMAyvs1v6AYz30pcO/fByaoWA+KVzkEn3eoyn9zFN5lj2h/DCPAS4zCVp2Y0nep5LPPBFk354z6+Nl6OcV5Yl8UoC0LqMdlaeFGcjVnH+UT8r9w81QnYQ8Y56RCKmP/LTpOuK75BSuUemXzf8uGLb5hBnMCtt21tgp5CJOaex/2qtLXbuZAk6XYJGlzj876qAUw9rmyf9GvgbnoANk4MWct4j7pEb4RhAewIrG8j+z+jI20lQBXj4YIOLQYHULdW2EfKC0mgHhWdbdT7ML8lcHykEi6iFcdLME1cRBqau3MjD1ijd8kawIG1bvsy5grnTGIavLoztWM/t+IvjwZ6h8Iq50k6rGsGGptZIvhKFtmk1aQd1fOf2TjaAwcWFRj0Uyr8as8b2BEmgpIkNWrY83CfcDgzK45gboXwY95c8bEB+IJzJO9fwYhERDa7z45/EwtxBttzIkLOJ4ZtQcNhnA41EpIe7FkJA/EKU6/jmWh5bIwH2WcIiu0BMmfUX+yWmuTObRuKPQsWYE4vfhtDlr/p4U4EUAN7BZ8jYj5gsxXbGvy9F3emKsptc6THDf7guampGR/fXrC5/Tr1RlKrNFHNXjleiBmnHDTJvmVbmpxllfwTeIS4T8ALGAvtc1ti2T6849CtnxMiNyL3ganyfOAsXlxD+RgU4hSh7Gw2yrPG1oBpNX+g+yrADDv7tB9oATPbZuMqUFaD0I0zRrZdQ8/065c7H2XdZrFchSzz+d8K/dUdZNvaZ1fl0dhU5R6Dl7oBvbxrbMqDjUJ9ktHeMfV4jobO8+ZRDm/EjaWsEcXNwHjgKSiuWeCVr8miZdyduXB 4TF1wJdy E27+zHfG5BMyc8rsJoNh64l4+jnkHl5jIEckhXcRRnVoxIr/7MkZSvKhFKUDwnnXBvP0Bc3YwgQBC31ucOmHWDwHhEcL7EViBl29z9vH1Tg9q4yW1I8derX6qQ13YnjVPo0fUL9vGxI3NjyaXnGqUEd/8m0s7g5LO6cHuA51VWhgk6LFr5o+DSA1hhtWxDGYQSe6Y9iyhleu0L66mkSzDGyYQfXbMDml9jjDEB0uHDwTRwNVV7pQq7yyZItYp4tFzraH8VkzY9jvLRs3nALWvZ336KbNgdaQx7LCcuMVaCDcEWnBwxsNYGFsbWftDyERKiHWbAn95D394MJm1E3gzD8lmWqNKS4wSU/raQhbMl5hynjaOfv2FN6mC6qwtuMMtZb3I7wZpAzGMDHgmT7Q0Rg/yFAn9p/iwB+7FjpmK54VpNgWJozPQBvAhEUOMlq2ORMryePcEr184/P/SrbX8wB4wnoOj5o0Oy/oaNLvxZ1SopXe9Ht5xIiTkiaXPBngYN2rivtA3Pdr1bp49GZjxME3UX1iFScW0VfvZ9wC1vR1pGix/Ucvdfc6Qxdi5B6aKlzXu X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: In preparation for supporting a kmalloc() minimum alignment smaller than the arch DMA alignment, decouple the two definitions. This requires that either the kmalloc() caches are aligned to a (run-time) cache-line size or the DMA API bounces unaligned kmalloc() allocations. Subsequent patches will implement both options. After this patch, ARCH_DMA_MINALIGN is expected to be used in static alignment annotations and defined by an architecture to be the maximum alignment for all supported configurations/SoCs in a single Image. Architectures opting in to a smaller ARCH_KMALLOC_MINALIGN will need to define its value in the arch headers. Since ARCH_DMA_MINALIGN is now always defined, adjust the #ifdef in dma_get_cache_alignment() so that there is no change for architectures not requiring a minimum DMA alignment. Signed-off-by: Catalin Marinas Cc: Andrew Morton Cc: Christoph Hellwig Cc: Robin Murphy --- include/linux/dma-mapping.h | 2 +- include/linux/slab.h | 14 +++++++++++--- 2 files changed, 12 insertions(+), 4 deletions(-) diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h index 0ee20b764000..3288a1339271 100644 --- a/include/linux/dma-mapping.h +++ b/include/linux/dma-mapping.h @@ -545,7 +545,7 @@ static inline int dma_set_min_align_mask(struct device *dev, static inline int dma_get_cache_alignment(void) { -#ifdef ARCH_DMA_MINALIGN +#ifdef ARCH_HAS_DMA_MINALIGN return ARCH_DMA_MINALIGN; #endif return 1; diff --git a/include/linux/slab.h b/include/linux/slab.h index 6b3e155b70bf..50dcf9cfbf62 100644 --- a/include/linux/slab.h +++ b/include/linux/slab.h @@ -235,12 +235,20 @@ void kmem_dump_obj(void *object); * alignment larger than the alignment of a 64-bit integer. * Setting ARCH_DMA_MINALIGN in arch headers allows that. */ -#if defined(ARCH_DMA_MINALIGN) && ARCH_DMA_MINALIGN > 8 +#ifdef ARCH_DMA_MINALIGN +#define ARCH_HAS_DMA_MINALIGN +#if ARCH_DMA_MINALIGN > 8 && !defined(ARCH_KMALLOC_MINALIGN) #define ARCH_KMALLOC_MINALIGN ARCH_DMA_MINALIGN -#define KMALLOC_MIN_SIZE ARCH_DMA_MINALIGN -#define KMALLOC_SHIFT_LOW ilog2(ARCH_DMA_MINALIGN) +#endif #else +#define ARCH_DMA_MINALIGN __alignof__(unsigned long long) +#endif + +#ifndef ARCH_KMALLOC_MINALIGN #define ARCH_KMALLOC_MINALIGN __alignof__(unsigned long long) +#elif ARCH_KMALLOC_MINALIGN > 8 +#define KMALLOC_MIN_SIZE ARCH_KMALLOC_MINALIGN +#define KMALLOC_SHIFT_LOW ilog2(KMALLOC_MIN_SIZE) #endif /* From patchwork Wed May 31 15:48:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13262455 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3C938C77B73 for ; Wed, 31 May 2023 15:48:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B1F508E0006; Wed, 31 May 2023 11:48:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id ACFF08E0001; Wed, 31 May 2023 11:48:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9BE1C8E0006; Wed, 31 May 2023 11:48:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 8E5BE8E0001 for ; Wed, 31 May 2023 11:48:53 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 6AC2816024F for ; Wed, 31 May 2023 15:48:53 +0000 (UTC) X-FDA: 80850983346.17.EA589CA Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf02.hostedemail.com (Postfix) with ESMTP id CA84E80012 for ; Wed, 31 May 2023 15:48:51 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf02.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1685548131; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Qms+9sEtyjEiS/3cZZb5NgzaCqfRLp/F+aac0bZ7Tw0=; b=x9OVG9SXlba3eX8My0mqfcKjfXmKeYtYG1RrfoptQcuZTDQ0i3H8vjB0BZjMLhn6lQKgaH jGMZAxmzCDBvwzRViop10XW38cNaYRgGTqFe9FDWARJsoMaUM+k57IWGeyLaXkrYgOrpSF pjxsywzJWARuv7v9k4LHarmx/MLH3Ss= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf02.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1685548131; a=rsa-sha256; cv=none; b=I4VauLwTLvHjY6och43sO0o1Lgt0KBsIKnc4KuIlY5KdrGRXldvcMfflHyGM+2dnBgx7Mb Akq7SEl1ahBChtqT0/mV8crLxLPd4NPUpS6fx7Y74GLpLJD3QHE4cTtoPb4Pmc0AXqRZ6J ZjSN97ruyuTtFqJHdadWwwXoPmZWfyk= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id CF557638F6; Wed, 31 May 2023 15:48:50 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 84008C433AA; Wed, 31 May 2023 15:48:46 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Christoph Hellwig , Robin Murphy Cc: Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Jonathan Cameron , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v6 02/17] dma: Allow dma_get_cache_alignment() to be overridden by the arch code Date: Wed, 31 May 2023 16:48:21 +0100 Message-Id: <20230531154836.1366225-3-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230531154836.1366225-1-catalin.marinas@arm.com> References: <20230531154836.1366225-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: 6bfgurhmdhosrpjsismy64tcrmy1gx7z X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: CA84E80012 X-HE-Tag: 1685548131-213327 X-HE-Meta: U2FsdGVkX19vDfMbBMYqHU0Xx1tBMshqFktZbiH5JSGsaBQUZpUZkKsqP/GoOxIFc04xI3WXBswj2Gdg/2xH5SOuCz0wgGlVNd3Vg9nht3BkqOhKf37rsywgRNNt6QC2sgflLhi+So4zxqU0Q7Ejvmt9OL2IbxeGYlBhapoQVnaecxmgtEWWGHVr+L2hYAMeHVIcAl/B1peChIOTQ3hheC+AXZS1g9XCQhwezkukps2O2U48xmgw+0YLA1Vig131xNOb5BRziLYEgKOj6j/t6qnISbNIerLGpQU+xIzoblXbeGL48MPswL08gntB+c6NnwhDT+F6LQE9Q3zR3VD6VmcUepor9lzOwtfYug8K0tREfDVxLrZ6w6rQBkHv97YSVZzhR3cN5lLACbl/609qyAewD2S0v3feIvzclfPwJO5W2r2mvUxv43ZefLSEikxEDQn0T/dU8BZAfyNEX+70Pi8ejD/HWiVaYpyyVt8ZCQ1iJ9yXPyji4nm+ZyXKdCmdaqjqbV43tmnD6lyg4C97aj4oeoLfthct0/CLuvNL3KAyRcFaQcIKagEBSL+uNewVos0zoLUTtmW7sboBN0Hm5WB/6PJw8bAUE1cfNlFAtbO5XZkjlA0dGx+44rYvTjp7W5WbtwqnK2ggFzAVWEdnza7l6V9rdpjQW2D7UXzCx7ZGz/YjlpZZ4w5PhDudXbPPTd9XOdjigkd+0gHLOcnouITRBhjv2TEO9IZJzavLXHwEIRdz3oW5yLVWXOniW3pNiFwDeKssz4bxjBQ3ykQKBtkuXiSwM+qsy9nNDNqt/heiZUrDYyht/dNIHIWGN4ANd3i3jCRVIiBBnPlMzHgtjl0aGAq2g3MkH9s7TafK0oDOewVSVF42TkB7qclNKZxfK5zGQEcUUOTF46IRhCMeJMkyLACYzDqUBtqlWX7GpajHR1DS9OzcFqpcnpC+0vd+Z5JCp/ddzy39dVoJYe+ MrKA6dye TbT+vbn340CKWChS5LgvkEoYkoxYbYCRwmO87Eap5chEa+o8oBFTRjD24EnYct4WfQDmeAtMhVciOG4GfB03ourhaxoCR4lP2pmJbrd1EqaC51+753xki/IVEUxR4tJyEWYbQwPyPM5NpEaYAfhAkQOjPM3D1X+OuP25BCn2UbdT549+4rA/69P0f3suctgdKPqWtWlgC1cTKq8DViK7HoUrD71KaBpTj26FVv/M0LLjzbPhT7O+Bqhy8gd89GVcbj0V+IRwTFKTgCWGJO5tP06Tt0cTqBKfNo2szNSQnmEjuQdSCVVlpOVV/uIbkcafgQ7Cix1MGsbs9c9s5bRxeRvBL0bkiiF0sfB9FEpbm+rmTalYooJLCZ0fhDOD0jwKUP0NhnH6+UsPm6+EziTNwuLlpz+q7Xp62uHtvTEIt5V/B0AItrncoNLanPSKebeeVwdVKC9N9NxY3Zu7BIrmuaPrDB4babbgkg3lUMwHqooUUcPObxuHhWwfLCh5tUi1PJsO171gr0KCvaRB+At5RBRO9jWzfXDcC5y0KKTCCSEIkljLR6/AT0UhwJoOPcsN4C/mP X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On arm64, ARCH_DMA_MINALIGN is larger than most cache line size configurations deployed. Allow an architecture to override dma_get_cache_alignment() in order to return a run-time probed value (e.g. cache_line_size()). Signed-off-by: Catalin Marinas Reviewed-by: Christoph Hellwig Cc: Robin Murphy Cc: Will Deacon --- include/linux/dma-mapping.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h index 3288a1339271..c41019289223 100644 --- a/include/linux/dma-mapping.h +++ b/include/linux/dma-mapping.h @@ -543,6 +543,7 @@ static inline int dma_set_min_align_mask(struct device *dev, return 0; } +#ifndef dma_get_cache_alignment static inline int dma_get_cache_alignment(void) { #ifdef ARCH_HAS_DMA_MINALIGN @@ -550,6 +551,7 @@ static inline int dma_get_cache_alignment(void) #endif return 1; } +#endif static inline void *dmam_alloc_coherent(struct device *dev, size_t size, dma_addr_t *dma_handle, gfp_t gfp) From patchwork Wed May 31 15:48:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13262456 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A34F3C77B7A for ; Wed, 31 May 2023 15:48:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2BC468E0007; Wed, 31 May 2023 11:48:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 21CD38E0001; Wed, 31 May 2023 11:48:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 132CF8E0007; Wed, 31 May 2023 11:48:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 047DB8E0001 for ; Wed, 31 May 2023 11:48:58 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id A09731402B0 for ; Wed, 31 May 2023 15:48:57 +0000 (UTC) X-FDA: 80850983514.03.FD41A23 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf29.hostedemail.com (Postfix) with ESMTP id DC54D120014 for ; Wed, 31 May 2023 15:48:55 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=none; spf=pass (imf29.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1685548136; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=35xPKuIoBzBBtPc7oLgirz1u4PdrAIKlSY99QdYwnw8=; b=ULKZrOAewXWi2dFKjyVmdgrPOP6CfMIQEtQ8GZRBMcPrnmr71D26kja1hjcl7Nrh0NWSLm F4jOoaYIHFjp1Ufiew6P93oS4Ug53JyQkrrUuCEGsPkFxmqEYKvK30UA+afBZIFW83NS9n SLAc6NQkZwo33h/9U/rQpXpyPgdNuOU= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1685548136; a=rsa-sha256; cv=none; b=TMVw/7bcsncG2QNXwZY/x5mazFM6AaCws4wQuZsylsiX1ZcfH77E1dNKekz0Cdy/xpKFIT 3JphQlDOzNFXMZmNSWvPVtHxE0kH8KAN7QIOGkzWh+zAx2mcp4HUTv+b4y3pmYaud30oMa fGl6o6O2cr2KjVyI1RfV34xqTucxywY= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=none; spf=pass (imf29.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id DFEAE63D98; Wed, 31 May 2023 15:48:54 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9F0FAC433EF; Wed, 31 May 2023 15:48:50 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Christoph Hellwig , Robin Murphy Cc: Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Jonathan Cameron , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v6 03/17] mm/slab: Simplify create_kmalloc_cache() args and make it static Date: Wed, 31 May 2023 16:48:22 +0100 Message-Id: <20230531154836.1366225-4-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230531154836.1366225-1-catalin.marinas@arm.com> References: <20230531154836.1366225-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Stat-Signature: a1djorr5n1uod9hb6yrfc4gwpnuqwt4w X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: DC54D120014 X-Rspam-User: X-HE-Tag: 1685548135-14324 X-HE-Meta: U2FsdGVkX19ZiSL6h59aV29BG+T78s9RmihAzHCH9Ise7/EjYhkAGgWx5GdOlImuQUPwhMR6dDA97f/iSEfqxzk1Nm6N34l8wdj6ET9jcS9EgU3Qxo1X5oytRbx39ffE8oZzCExxkUw0qN2SYFQidLC19AO26Uyu1AlKxY1Jx68hz38e/N8+uLfSxug83mlNfY/PNTO+WLonWgGRPErpHd51//sYEhJsZRpSF1pLvm8v+NP88MWahSvJ1QU19oMPLTthpcboZaUULrEdFIYF7on8p9HI2Q/njGTTb6VAXplHALnwVQj/min7HvhIqbgb+K8XPJeO/KcHSLA+UdEZdBgei38ZtH6t01j4k8mhqs470FdhvuhARJVqEDnaP7RMVT2vuXeMxaekZvhilGl88JoQAl8c1wpj5LmXRmC81jDww9YtxrpPdQtaPRVIQz9m+BHGZsX81toevhhRDB+UKIZw+6cwPlT6pScEv2RZt6i7L0HXpX6weGXASpOFrCFu0N7H0q4+h4Bwbwvo+/lFjKHcN1mDeIq9QylGMS5NQD3ffFdBeyHM7eW9qJkDq8sIdo4/O+TFdX7KMbG6S/FAPC6Wx1TQXMwZpkFLgnYIYoPnwoer7C92zZfFSOZreykHNiCvLwDLnxXf+FFPo9h03Ey0ak97IHVlBkpcBT5k2tG7ZEsuj5GmQO3g6uXUY8celLE4t++ITR3xNdN3AwkwG1KiOTc09Xq5wU5+cyNkI+pbeq257Zg4tVO9XjsZLEROOGycDrEHPUH+/4P/AZ7ARCH1dmCO+ctFwXEpabxXanFQKjREdlESp9yV2bmoFWdgw/v39xG6CIq7Rj5sz8/VEJcdd1NLAyagw0YYDV/ipeKA6RBEGpIX2pZNJjDrSoF4NT6yddh1fcKzOaidRl573ylBDnAnpNLUr61Kaj9FsDbsaTIzKvYQ64T/4lFniCh/t34omJoBrNfbMkyvbEP NGzr67Mn 9bEduY6aWWuuSY4AR0uz/8a8R9/wKVnoBHt7kjk0ZWLlHvtPlm8w3ou4R2SKOLrFbcqhTS6q3VQrrn3atlRixGt1xZWrlB9Fc1CIkTMCUQiJsk++Ekjhe+k9OEBOw4CpeViHlHpop+sqY8A44QL5L25Hlhjt7jg3C2XemUnGrDKv/Bm3Bysi4rFhq/lSJ8JNB+I4MpOkmUjJ1dfd1xH3miwuihFSrU/igzm9ufJB0Xt8d0vimXaM0L7aV9O3ERKDTLpn7x+UKw0/wKuClE2TDMZnea0gSWIdsldtnuf343ZKZAEXlTDkmgnHlZ2OwJS327jGnz9h3yqwsBqetxeoi+xt938ABpLbiPeW1fCVBxqZscZiBEidDssMdFH1kvQY7ULl0BACwu70Ye3ARIM365ct6lVlkpymbjiBbnyVRE46vnYQWUPdFDaAzX0qKqXLDz1u6GudtQWnhzsQYGHWXNYHoDMAhIrNBI0e+qFYTsG9vhsbjGXyC7UIdYHIaARc7sRlsKBA5Wv8m+zJb4Vcly9R69fzpTHCYzNcXP+ZblVlC8lSG3F3UNz6enr7Nbb6toCn6 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: In the slab variant of kmem_cache_init(), call new_kmalloc_cache() instead of initialising the kmalloc_caches array directly. With this, create_kmalloc_cache() is now only called from new_kmalloc_cache() in the same file, so make it static. In addition, the useroffset argument is always 0 while usersize is the same as size. Remove them. Signed-off-by: Catalin Marinas Cc: Andrew Morton Reviewed-by: Vlastimil Babka --- mm/slab.c | 6 +----- mm/slab.h | 5 ++--- mm/slab_common.c | 14 ++++++-------- 3 files changed, 9 insertions(+), 16 deletions(-) diff --git a/mm/slab.c b/mm/slab.c index bb57f7fdbae1..b7817dcba63e 100644 --- a/mm/slab.c +++ b/mm/slab.c @@ -1240,11 +1240,7 @@ void __init kmem_cache_init(void) * Initialize the caches that provide memory for the kmem_cache_node * structures first. Without this, further allocations will bug. */ - kmalloc_caches[KMALLOC_NORMAL][INDEX_NODE] = create_kmalloc_cache( - kmalloc_info[INDEX_NODE].name[KMALLOC_NORMAL], - kmalloc_info[INDEX_NODE].size, - ARCH_KMALLOC_FLAGS, 0, - kmalloc_info[INDEX_NODE].size); + new_kmalloc_cache(INDEX_NODE, KMALLOC_NORMAL, ARCH_KMALLOC_FLAGS); slab_state = PARTIAL_NODE; setup_kmalloc_cache_index_table(); diff --git a/mm/slab.h b/mm/slab.h index f01ac256a8f5..592590fcddae 100644 --- a/mm/slab.h +++ b/mm/slab.h @@ -255,9 +255,8 @@ gfp_t kmalloc_fix_flags(gfp_t flags); /* Functions provided by the slab allocators */ int __kmem_cache_create(struct kmem_cache *, slab_flags_t flags); -struct kmem_cache *create_kmalloc_cache(const char *name, unsigned int size, - slab_flags_t flags, unsigned int useroffset, - unsigned int usersize); +void __init new_kmalloc_cache(int idx, enum kmalloc_cache_type type, + slab_flags_t flags); extern void create_boot_cache(struct kmem_cache *, const char *name, unsigned int size, slab_flags_t flags, unsigned int useroffset, unsigned int usersize); diff --git a/mm/slab_common.c b/mm/slab_common.c index 607249785c07..7f069159aee2 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -658,17 +658,16 @@ void __init create_boot_cache(struct kmem_cache *s, const char *name, s->refcount = -1; /* Exempt from merging for now */ } -struct kmem_cache *__init create_kmalloc_cache(const char *name, - unsigned int size, slab_flags_t flags, - unsigned int useroffset, unsigned int usersize) +static struct kmem_cache *__init create_kmalloc_cache(const char *name, + unsigned int size, + slab_flags_t flags) { struct kmem_cache *s = kmem_cache_zalloc(kmem_cache, GFP_NOWAIT); if (!s) panic("Out of memory when creating slab %s\n", name); - create_boot_cache(s, name, size, flags | SLAB_KMALLOC, useroffset, - usersize); + create_boot_cache(s, name, size, flags | SLAB_KMALLOC, 0, size); list_add(&s->list, &slab_caches); s->refcount = 1; return s; @@ -863,7 +862,7 @@ void __init setup_kmalloc_cache_index_table(void) } } -static void __init +void __init new_kmalloc_cache(int idx, enum kmalloc_cache_type type, slab_flags_t flags) { if ((KMALLOC_RECLAIM != KMALLOC_NORMAL) && (type == KMALLOC_RECLAIM)) { @@ -880,8 +879,7 @@ new_kmalloc_cache(int idx, enum kmalloc_cache_type type, slab_flags_t flags) kmalloc_caches[type][idx] = create_kmalloc_cache( kmalloc_info[idx].name[type], - kmalloc_info[idx].size, flags, 0, - kmalloc_info[idx].size); + kmalloc_info[idx].size, flags); /* * If CONFIG_MEMCG_KMEM is enabled, disable cache merging for From patchwork Wed May 31 15:48:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13262457 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 961F2C77B7A for ; Wed, 31 May 2023 15:49:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 37E6B8E0008; Wed, 31 May 2023 11:49:02 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 32E7C8E0001; Wed, 31 May 2023 11:49:02 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1F6518E0008; Wed, 31 May 2023 11:49:02 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 11D6F8E0001 for ; Wed, 31 May 2023 11:49:02 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 8461B120286 for ; Wed, 31 May 2023 15:49:01 +0000 (UTC) X-FDA: 80850983682.18.E2AA8F6 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf11.hostedemail.com (Postfix) with ESMTP id D428740010 for ; Wed, 31 May 2023 15:48:59 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf11.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1685548139; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0mqLCBdpgkfqDUcqHwwIym+BakJfQ9CLI5FzmbFJWTk=; b=yJ/kFAUArFGd7RqCqx8E07Q3TAgWO8KiqEb+0t5mt++2Lo/i2XwvHMIW5Q8oKBciWDWYpz TRahR7pdBacF0+fiVztgRDOv/FM1YmvS6O/IJ7OGiKRjMuwLE9rkG+ly/LPE4dqoiJyyyn 9fFcjpcOOgLYjR8u58qWr+GXbzhi8HU= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf11.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1685548139; a=rsa-sha256; cv=none; b=tEb3U6wKvWpXSi+KMtg44rL1nLHBUjxK8JK+yldyIFSHSVKcL0mzwv8cxbZ/C5twKzGqa7 dwfFHWht47swvTMl/WmPJUio6wMMWy0sbXVf6w4DkWjWfh5v9fsCPB6+tmMnX4eQBsPlR5 QC6X0FoNQ+rCZkn0MlT8/kYQ6JuEPEM= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id EEBDA63D92; Wed, 31 May 2023 15:48:58 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id ACC7CC433AF; Wed, 31 May 2023 15:48:54 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Christoph Hellwig , Robin Murphy Cc: Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Jonathan Cameron , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v6 04/17] mm/slab: Limit kmalloc() minimum alignment to dma_get_cache_alignment() Date: Wed, 31 May 2023 16:48:23 +0100 Message-Id: <20230531154836.1366225-5-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230531154836.1366225-1-catalin.marinas@arm.com> References: <20230531154836.1366225-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: g8w6nauwzgaerxecgb8q8e94f1ic4m9d X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: D428740010 X-HE-Tag: 1685548139-708932 X-HE-Meta: U2FsdGVkX1/IBEoS5sWg1S1PAh8LtGWpEJqgQfG/2Sphyh5fFPbE3YKuOyIosW3OzFns8u0R+vb6nur4aqpzt21E0O1a/B22gm4FA4Mlt3ZWl2KLbyzoF5dHrGsO27F2u2CB98x0KuE5QXX/xDjR6UmuerYmchFeNemPmbiebnvHFYZl7hv1+F7VEHVIjcJ+8tUF+vLZY2RgrReHsGXOwOUWA+XkYF5AInOjUUHxljcTwU3t8ON4IUF1Gv1R5oJ+akqavI051xIK34hMu/gBjb22dDRYztpKLDjC9LlN99U6VqUQZf0t4h7R8+FIYnNZ/Yu/Uqkxg5todaLgS02goD+AC0vEFYaioaO/r0Ds7VMFv+JYe8GhSpVwOOk2e0D21XzuIXTjDP/EPp19YEF2wsMabz0ChEa4chm8kLiPTcaqV9pVuZyX4N7CEXd49lRPItLNdjrqCIKlxUP6CBMWW8CozPY3YX9YCH3ah5GHyUaBc/mPA/t4Pzi/6o54h8O38chkwGUK2uk79MfuDOQi9amh2GHdFNMMsZierdh8PRAfkqqq9a8azMQ57abzMwJtXmrWyeyH+5CIhB/spoa7ii9kxyE3IOoRy4Tpo1z3gV77FRK2DzS0BbOSSyJkexygHw85CNfm5y9sljIGeqoluTIrpJD06djAjDIgr1q37MmhXFt6MSDIBe5y5y5CQSDR9LF2m7mhwhf6oQAz3nxekOEcB1AvKDKDxAWIzC5+Rl97Kijp1thcvl+8VZg0kt7FBb6xyCR9W04fDyk0bx3c1tL5n8ECvp5N6yvxBbS5nZxjBT+jrZbT2o0fIVdffaEJN4at4ENqi3gtIwAWLHt/Z/Oaoa7HafST9crbKt+HDcrntUdL/iDZVs40XSKCKrpqUtd795Ucc890sZt+GTFtxwJHcgeIJS0Z5XZ6n1gnGCGQrioF2m/VnhohHvij4m1j9c/QdMg0A0WjdNJGcgP xx6yjGsi WHNKy1FspEPYaKaxb1Wo/bRLGQsJwETVH61rL+Y4dKP0ptYtXupdLO7h25j5qxNVoAVQwPokHBoHObmwjsx5F+Glyj3teWuKZiuh/AEz5RJKtwFu4WRSv+ROjtySIMR9jIMbm2qGc2WL4LmO3blvOnuwytb+E81ERmjzDGJpI7Cs9I60s406ceLHl/yTGDtjK/2/c7KUPXed2r2c3bKxjKjvGlHd+xbIPQbVrTUQRLxjKwejJNa8amSKPG8F5ghbHqQI5D7RPM+itZj0Ulj8vWljY5s4j1hBxZ43lNYGj5Y3fZl7DD39Rn6fk0G9mvA7HHjaRzxaPUC8s5/onGYMB1h9ccddSqEUtJb1VM7kfZaZnYMUli2A2g4BYK9Fwhm8u2LV1/2VkYSlaaq09LsZwv4WmtgD/shFtjaHwxpaKbdsHpI6hkglDva0ZmOyolUJadK2FCh6Jr0JDsOXbdvQNyfHJNFWXu62A8IHDOz8fAR7R7AYcuB6cAYAIMQ7JO2s4SEO8knZkoITsbRLYEz4demMOUr816IcadoQCcOEPrgfSwfVH2H+alvxsCOvedHn1HFr1 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Do not create kmalloc() caches which are not aligned to dma_get_cache_alignment(). There is no functional change since for current architectures defining ARCH_DMA_MINALIGN, ARCH_KMALLOC_MINALIGN equals ARCH_DMA_MINALIGN (and dma_get_cache_alignment()). On architectures without a specific ARCH_DMA_MINALIGN, dma_get_cache_alignment() is 1, so no change to the kmalloc() caches. If an architecture selects ARCH_HAS_DMA_CACHE_LINE_SIZE (introduced previously), the kmalloc() caches will be aligned to a cache line size. Signed-off-by: Catalin Marinas Cc: Andrew Morton Cc: Christoph Hellwig Cc: Robin Murphy Reviewed-by: Vlastimil Babka --- mm/slab_common.c | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/mm/slab_common.c b/mm/slab_common.c index 7f069159aee2..7c6475847fdf 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -17,6 +17,7 @@ #include #include #include +#include #include #include #include @@ -862,9 +863,18 @@ void __init setup_kmalloc_cache_index_table(void) } } +static unsigned int __kmalloc_minalign(void) +{ + return dma_get_cache_alignment(); +} + void __init new_kmalloc_cache(int idx, enum kmalloc_cache_type type, slab_flags_t flags) { + unsigned int minalign = __kmalloc_minalign(); + unsigned int aligned_size = kmalloc_info[idx].size; + int aligned_idx = idx; + if ((KMALLOC_RECLAIM != KMALLOC_NORMAL) && (type == KMALLOC_RECLAIM)) { flags |= SLAB_RECLAIM_ACCOUNT; } else if (IS_ENABLED(CONFIG_MEMCG_KMEM) && (type == KMALLOC_CGROUP)) { @@ -877,9 +887,17 @@ new_kmalloc_cache(int idx, enum kmalloc_cache_type type, slab_flags_t flags) flags |= SLAB_CACHE_DMA; } - kmalloc_caches[type][idx] = create_kmalloc_cache( - kmalloc_info[idx].name[type], - kmalloc_info[idx].size, flags); + if (minalign > ARCH_KMALLOC_MINALIGN) { + aligned_size = ALIGN(aligned_size, minalign); + aligned_idx = __kmalloc_index(aligned_size, false); + } + + if (!kmalloc_caches[type][aligned_idx]) + kmalloc_caches[type][aligned_idx] = create_kmalloc_cache( + kmalloc_info[aligned_idx].name[type], + aligned_size, flags); + if (idx != aligned_idx) + kmalloc_caches[type][idx] = kmalloc_caches[type][aligned_idx]; /* * If CONFIG_MEMCG_KMEM is enabled, disable cache merging for From patchwork Wed May 31 15:48:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13262458 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0D54EC77B73 for ; Wed, 31 May 2023 15:49:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9E6CA8E0001; Wed, 31 May 2023 11:49:05 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 996AE6B0074; Wed, 31 May 2023 11:49:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 885DD8E0001; Wed, 31 May 2023 11:49:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 799C16B0072 for ; Wed, 31 May 2023 11:49:05 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 1432F1C6EAC for ; Wed, 31 May 2023 15:49:05 +0000 (UTC) X-FDA: 80850983850.08.41AE376 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf24.hostedemail.com (Postfix) with ESMTP id 693B4180024 for ; Wed, 31 May 2023 15:49:03 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf24.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1685548143; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=X9GvMJYtQNhC/CV+bvo1nqxSzdMjEtDAvkUK7+TAINo=; b=pl2C9eADssJ7bYv3NuxNlCcTGg2utvSS0mxWMbys82X3P5F98Hm19gHeawxfMXtgFGakwg x21txakJP8O3EMk3nqVt8ShplwbUdSYSxn8KtiQAdjCyUP0wdN/WeSW2hbfsC3+jXjIXu9 nu9DgZjf6taNjx/zfLVvLj44H7MgLBQ= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf24.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1685548143; a=rsa-sha256; cv=none; b=XdePTn1ks7GthhQLQI2J9ktFPboPimeWEbs24296uaKirD4NoW7l2nNkNTvU92zrRQVA4x PB5XmHRuB/B11Vq8dLZGM/sH102ik67ADWonpJdQ2Sc12OdIvEfgPyH90iCO10PCXaBrM7 QVhz7WYCpxqxHnvAWWyrPFqs1+K41+s= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 9BBC860F0D; Wed, 31 May 2023 15:49:02 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id C08ABC433EF; Wed, 31 May 2023 15:48:58 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Christoph Hellwig , Robin Murphy Cc: Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Jonathan Cameron , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v6 05/17] drivers/base: Use ARCH_DMA_MINALIGN instead of ARCH_KMALLOC_MINALIGN Date: Wed, 31 May 2023 16:48:24 +0100 Message-Id: <20230531154836.1366225-6-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230531154836.1366225-1-catalin.marinas@arm.com> References: <20230531154836.1366225-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 693B4180024 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: hz7h7qbk4i69xmcfn96myi9qgoa39s9u X-HE-Tag: 1685548143-437405 X-HE-Meta: U2FsdGVkX1+pFcKJwGethd9OffS+LcrfEIjzDxWbGY1L66/4EO4KJA9nHWeVDTbAjsKtFQakLiPHJN9+/2si7838ZqZ4rZqF7bdAgX+4cjkzYF4d6o0GynBm3jpRgHLaJXoDNXMLk3eJswA0HjZTkPNQZZJplAEh/OJbcYm3qyxT8GCb8NBHj9YDf5IupG/wxAXo/UAbyApxQnOx5BxBVN6yda+5eCdS5usxfXaE6DPUbTEYj92Ddo3G98UyfwbKjfjCdMeYxke4sgKRZtWmE4H3ARlOK/ywn0tgd5ldMJnt3ZBknPr6s+LF890tyxJcnjriRphK78g7ptsMIWIb+FqmSYXHjJ1nn/aQywnprUp8eRZ8qF848gkO4PylA4imVC8MXcfSAbY95VjZTg5uGTU1z+xvMXmk5ywt2C3NDdSn6ytjI/2jrO4Jmcn8/VKTJB2IO+rKvRrveY57FwoUerkw4lHimRGhLFHuR6HJ2fP4qqUPABSjEHCPUeeRvQTtC1FAfpl6skfLPzMDAFUCmbJUy2hoqj6KxRxf+hQFwGsRDVVpUWU1NXVUu5CMTTb0+B9j+YndUAoI4VUiUOjcp/WxzNCDZVbr9bgWvS5eJ5+YznzDwc/Y81KpQNR2Sqve4zV0c8b+B+k1mlbOOqDpNhpzuhE7Tju62cn/lr5vi1wE/DzQ26fkezJSFhhKxGVC2W/tJqj1rnjtPp4cgXSXpVdA5LpoPdsw1EzNvS9r29x7H+FlDKwtAWT+/BQZLKRlCsM/BM34SUTm3dGKt1EbGFTYnnA8OCKfT0SdOYOla1oSo8rKpAZES6bdfMTWyO5wXNxkJteh5Pm46OyTNIPUr6sRrUwNk6XGpV15zf1Njgys8ptguhPYuCad0zRokz7E4TPekg2eGJU5JcMomwg4o2ntc/cgO91J6fEyWSQ2h+WPyis2p+gSHd6Fck10OKY27zNmWSPe5c8Srj4EjOw MsmuZpWB 4Dmg50Kgw9AlTr3pvuPuzLFtHPYaF0yaAHkiJ2/nrZucayTA0AucA4qKvvGu6WDh6UadZHTjQA6LPw43dcRM9GrXQMfIZYLrT33noJ7FNw4OFd3CN+/ReZflJvo+qyAjJyepnBgm/ByUwvJCzRA30ZqAOV6e9HZem9MWvahi+NPJSyD6Y06JDoTcM4CFnyaO65UWWZJ4dLqVtJ6gTjQnazO5P1hZQ9QTscX5jCyTtw45XzY1jxjjlqGaod9wS8GKObre/imQl0LtfpzbKExD2bZmmk75nYss6puL6TLz9dmEgdMqejdjeszEO/Rj1hpaoo4rR5yNrblu31sJ/clU7mMHRwl2fO756o7WXTbua1o3PzqWoiu4DCeXSI2N3oIWQquzdL8a+Hcmymp6vaQH24GBOnAUzQe52j/6UA8MH3k7XWwsbbfwvuwK24XCgblrJ2QZ+C2wvfWLwllbFlNi45plDJpQkImLnG2f6qrLfwrjxzpcWLRFsqBeJFUK+A9ivmLPpOWGXqGKxqezcNdIUqIjXcQSnJagCTvf9GX+c9CBzYbcTNfm/RIKZnfM2ZBtyFf6Ttl+s6U4JQlDY58+gkLeGjm7KFgKc5PSrdC/Lp+6fYXS11fTFusaDEQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: ARCH_DMA_MINALIGN represents the minimum (static) alignment for safe DMA operations while ARCH_KMALLOC_MINALIGN is the minimum kmalloc() objects alignment. Signed-off-by: Catalin Marinas Acked-by: Greg Kroah-Hartman Cc: "Rafael J. Wysocki" --- drivers/base/devres.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/base/devres.c b/drivers/base/devres.c index 5c998cfac335..3df0025d12aa 100644 --- a/drivers/base/devres.c +++ b/drivers/base/devres.c @@ -29,10 +29,10 @@ struct devres { * Some archs want to perform DMA into kmalloc caches * and need a guaranteed alignment larger than * the alignment of a 64-bit integer. - * Thus we use ARCH_KMALLOC_MINALIGN here and get exactly the same - * buffer alignment as if it was allocated by plain kmalloc(). + * Thus we use ARCH_DMA_MINALIGN for data[] which will force the same + * alignment for struct devres when allocated by kmalloc(). */ - u8 __aligned(ARCH_KMALLOC_MINALIGN) data[]; + u8 __aligned(ARCH_DMA_MINALIGN) data[]; }; struct devres_group { From patchwork Wed May 31 15:48:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13262459 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AF679C77B7A for ; Wed, 31 May 2023 15:49:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 48F8E6B0071; Wed, 31 May 2023 11:49:10 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3F1528E0009; Wed, 31 May 2023 11:49:10 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2B9EE6B0074; Wed, 31 May 2023 11:49:10 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 1EA826B0071 for ; Wed, 31 May 2023 11:49:10 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id E37F01C7216 for ; Wed, 31 May 2023 15:49:09 +0000 (UTC) X-FDA: 80850984018.21.D13AEDD Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf29.hostedemail.com (Postfix) with ESMTP id 2E568120010 for ; Wed, 31 May 2023 15:49:07 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf29.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1685548148; a=rsa-sha256; cv=none; b=tv5dIXFzDGBJPTMWmuzJZxMaYtPEb5uvSZ+OPBpP8qFU+BlOg70cJxtljGm08eRWdQ3BBl HIubzlnMv6Gz/Z+taDF3VygiVRH0Ma92noC6b+uBWRUhxDy5ZLv7//bNzaETCe9n73Eagm R2klhROjwz+XyZicg7egA02Cf0eeigI= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf29.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1685548148; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=S59+slMdeGHVr9gGv1wpxhrhYrhnmUIhONDI6UrLqhg=; b=LRElfdAdYZptdiEJwQhrM1wSWTEn2rl3ckoZChDem4AkaPHTX2azm3TiaVm9dLKIEwHPcx Oy07dqhZqiV65kMDyEQfhgMfTmASyXX3AXugc1BMm5GywQEWjOGrDeXu6PubjgVVhsVUJZ AwZ1Vd8K35rpsr7/5CbNcBjSwxW5R8I= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 19F0363DA4; Wed, 31 May 2023 15:49:07 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id CD567C4339E; Wed, 31 May 2023 15:49:02 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Christoph Hellwig , Robin Murphy Cc: Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Jonathan Cameron , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v6 06/17] drivers/gpu: Use ARCH_DMA_MINALIGN instead of ARCH_KMALLOC_MINALIGN Date: Wed, 31 May 2023 16:48:25 +0100 Message-Id: <20230531154836.1366225-7-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230531154836.1366225-1-catalin.marinas@arm.com> References: <20230531154836.1366225-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 2E568120010 X-Stat-Signature: io1kwudi8q5do5h55pumqhn11n9yq7he X-Rspam-User: X-HE-Tag: 1685548147-129092 X-HE-Meta: U2FsdGVkX18QQ1ie+38xFNwmy4SmQIpbVwHPtwizOCddDwQ0Lk9dUgGOktURq4R7F15QsFhP51Acb1N2uZHtHG+9rcOOn65O5k+6MV8WfObL8RkNfZlyXSTKT0dxOw8OTnQNhRV5Az5KyZu/WZ2AeG0RMAZIVSzy9xzjV0Ke/s2Kx9S/ZZwRQpBMIJ2h+quRFe4ciYDfpwJ/rM5sWEv/YZgpZ0RsUmxHLTg0nwqW7ylLX6A96edNZ4Fhb73gwdVL6L8Oo8FOLSTrWAWiwEEJwiqDLgm0/n9TOkYbp6lBL2mqw41QcvlAaKfIIJYw6gCAr7IgGOQ9kteTmhbcXKtxpk7i9tyicZDxUEpGlqCfS6AQXgkOBKexb9V7YayRl/ZS9veC7Wiuvt0dRBPLxq0CwFl7PwZj3YC9DVgkxe/m6F5x0EAe9okQJBN3PIYohTLLprdgzenXSUsgRLOSI6rpShR4SuwIXSYAdQNbtCYtJ8WNPlWhRNgHJJnvDaGIp8TzqGDVLtbntVhEkY21cCbS0kCnUOsXQebwFH8XH0yl96vVn5emXCUF4uUk6Fr/HkSQ/t6aEzNFzWPe8MMm2kU0nHsW1OoHKtPUu2aG6WcQx1wYXf8S8ygyZuXGfbQ3KrrtnYhJBxvneknHIPU0HB2cWraOgQLq1KiSw5EdRB5KfGoL4L+Rq3/4VWdZ+C0Z/rcconTpPMyMw0xWICPut2TThvoUzdJHnSQ+O5rJhV+auzrPWjmKE4M3vVmZrT/uf/d7u04oXKoNJlE85QqdkFLVL9twZpc3ct9nDSRann6ZfyK0IleuhCzVqxYjcH+IAQZ6vRlyviM0o0RgM9j2GiL8exbB/5ACY+bhFJyfft7fFCYYf9r5fSTnRrzz1PgjZzkCXk8HEp9gYJCf1xZXAyBMTH1ofUCb7iEI9CMbfeKLgDX7EHlUsrCrdykcWkjsdufNgdO1OCfaeruZhs7ZuRb 2wzPkFin NDrza1KRcIunQglW+/TNoYyEoSq8zSEnycPPIGjFGWdPJ/G4jAhRKng2dxqjlxfRQuVVnCLQoxU2lcjtq+rzAH4DNcokfUFY9v5ehZzh32SkYh8KFJAuddhFeEAN+zgtLDLciGQ+eEUDN8/4dVikm0giFOZaI9jIc4iMalIm+Ub52MlXybsOlIl5ApZ43jH7aINAVqlE+C6xEZGe7mbDw2FTASF2vJ8b7bMpP0NC/uQoD9WCNyY4DCPcqRDRR5AGhgKlysm644tfPgx3+sPFKZnoPobM4Ey1zU55LgumegvglDOmCvg3MxWr8YYykseibtPeLhCUugjX1PvEdJudvwyIFN4/GA5EwnMBw7NoUslk/1nIyvuWdwZb8eBS5W8TmoBB8DK/K3O0sIa5GYGKKF3piCwWjGUAhY088iDAE0toQ0Omz/2vrAlRCK/f+Vojny5txdoeasQaJAcJ8vSxxZ2k54AkijAOeEUbEL2F00uqDwA/M+nFRNDLXRh4u9oWbmyfw X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: ARCH_DMA_MINALIGN represents the minimum (static) alignment for safe DMA operations while ARCH_KMALLOC_MINALIGN is the minimum kmalloc() objects alignment. Signed-off-by: Catalin Marinas Cc: Daniel Vetter --- drivers/gpu/drm/drm_managed.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/gpu/drm/drm_managed.c b/drivers/gpu/drm/drm_managed.c index 4cf214de50c4..3a5802f60e65 100644 --- a/drivers/gpu/drm/drm_managed.c +++ b/drivers/gpu/drm/drm_managed.c @@ -49,10 +49,10 @@ struct drmres { * Some archs want to perform DMA into kmalloc caches * and need a guaranteed alignment larger than * the alignment of a 64-bit integer. - * Thus we use ARCH_KMALLOC_MINALIGN here and get exactly the same - * buffer alignment as if it was allocated by plain kmalloc(). + * Thus we use ARCH_DMA_MINALIGN for data[] which will force the same + * alignment for struct drmres when allocated by kmalloc(). */ - u8 __aligned(ARCH_KMALLOC_MINALIGN) data[]; + u8 __aligned(ARCH_DMA_MINALIGN) data[]; }; static void free_dr(struct drmres *dr) From patchwork Wed May 31 15:48:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13262460 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9A14BC77B7A for ; Wed, 31 May 2023 15:49:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3570E8E000A; Wed, 31 May 2023 11:49:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 306F68E0009; Wed, 31 May 2023 11:49:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1F6B48E000A; Wed, 31 May 2023 11:49:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 1106B8E0009 for ; Wed, 31 May 2023 11:49:14 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id CFB3C40267 for ; Wed, 31 May 2023 15:49:13 +0000 (UTC) X-FDA: 80850984186.14.8D8F9BA Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf03.hostedemail.com (Postfix) with ESMTP id 3818220029 for ; Wed, 31 May 2023 15:49:11 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=none; spf=pass (imf03.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1685548152; a=rsa-sha256; cv=none; b=NpYkzXRKDV2Teff7YRPSoDgTCKuhKUbIdg4X45J4gSfMlvFwOVCrGeDvGLVfKtrNALRSXq Nkr9nTwBUWBKHKgbQ60wK+WwilIwgy7tiFi02Qx0WCOu3TJgoHd9S6doiO/TDJxt/qnq0N fAW+zn1b/Kyd/PUsyveJ4XVgP0dm/V0= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=none; spf=pass (imf03.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1685548152; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZmV86imfVMATseoJlCRYow7aoFQuclO2qwE8MVqrGZI=; b=qaa9dmFi+MdxyUVETatRXlOgQAgi0m6n5NZAFmBLWuh1OKG2gZFadXg8KOpnnJY7OYTW0U /a89pPf39KrmdcVTqPorFArY6kSrnSrsw7S1poWVhU8WB3tcWO0mHeklrZ6SVEshdCaGF/ qzy1o6nHpBs24RvC9RCpNX285Du7+gU= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 52424638F6; Wed, 31 May 2023 15:49:11 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D9AD7C433AA; Wed, 31 May 2023 15:49:06 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Christoph Hellwig , Robin Murphy Cc: Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Jonathan Cameron , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v6 07/17] drivers/usb: Use ARCH_DMA_MINALIGN instead of ARCH_KMALLOC_MINALIGN Date: Wed, 31 May 2023 16:48:26 +0100 Message-Id: <20230531154836.1366225-8-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230531154836.1366225-1-catalin.marinas@arm.com> References: <20230531154836.1366225-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 3818220029 X-Stat-Signature: xx1u689nrco15s4xec6pzc5tes5iu1qx X-HE-Tag: 1685548151-329178 X-HE-Meta: U2FsdGVkX1/iQJPH940Er9tPM5gTBNBaxRqMwGgBBeZ2wMU5eNwk2X/1OPGvIiaV3GHR6cmebpEWUCO12YsjDSE2XXheBNGGR9eGCBAnLzw9ZYcTCOxIjj1a1dwqNYzfNNhdswwLFtOewvsFMN33ZQqv4djuLF/jCt5LCmKOT81uinXNpjvBMlVX5UroHFR1cJgBb33YpacNm2wJMe1XOyjonGpN4uz5t9eLMLBSdxUmIBmO3smfWIX2zYlhzWk7rzGRa2o3nF6oaapRygI9uhx4uDvRsR5l5xK9gl7+VKcMDKQ8TvvOjPkaY3SFq0Bn2Z06ov8nrUDhoRjhFDAA7/o0cZz7RLb5PUMd4x891N49j/vGREGkYADeUtC8VGSTwAgYfzvuZADi0z+RBr3Ykcs5SGvg7P071VN2mai4r1UioYT5w6HwFILjPdoq/jgyE2g7XUN7gebva9kVBIcausiCo0wR4hcS+Idvfd3slVdgcFbDXEA+XlDmpryVrdxpXEpIfztzsMcJ6FTjeMj7Jk/mzkSefSCzUV5TRpvYsqJ1v7G0MP70d8EhAXP1MR+4+n09Z5Vg+4TPxLdkC+1UijbZ35tGQpqFsE1fHfzLO4j/7zYwpEaqgGed9lWXYiVGqeY7xHu2yGYbLqgpOlJE9EIDfeluBXLzX5sDclOfWENEPI2LyMe3IO5VpRoi0dqlmzZgglo2hJE+YkHPjEVR3E+A3i+Hm1hFjZWX/T7BkX6+QVy4lPKDn7j9qfGJUgpefYrZgtefaOz04uTMsPrg8iJe8hKsIgYKvertYUFqghLN2znit6lYqmbJjfTmPnTukWIRw8+uzf7nX98abnsdmCNZLCGA7d9umcofkfs5WZthsTmu25sR8VHeGNS6PlPLIeo9Nvj4rkXLNM1tdMhGtuYygnXxt7I4ZfqqV0nJaqCd0SsRqdwM0zyIv+dfH28TbGYmQxCazCHu7HVsUWs kYjMr4Ll FDseBzNQZ3olT9YW3bXVKMjAtXIwfjoMEt1f34Nxuhcf7FsS0pvYoY9E8RCQzTu/Xx7CmTvjg7G04+N+DCy3AOoP4hKLut0iROaBfHd2/TPHThZF8tupI/wHjWYMiuYnBk2s83We5d0hvH7TxmQKT/W3A932TsyirHqv8/x9JUzwft8XSnvFaQHn+Le4Vpp+RqVCIO4d6E2rRMNLltc9U0XM1+p37Rn8HEMT7tXeRazN/GacK4z9Ze5Pn1RdIvL+qlKfUE4sTwCTu7+nTatlWZcGkdsUtZW/j/a4iCyc1E4uGNGMTFU+qc3D29OPWS1i+/TK3xrRvM9Z2YdIci9TkWG/AB3gxaMCsKnN9S4UyAL0qS3notKSWcq1MGNlML1viebJgFtA8sHi8xCBTI/gZYg9b58xMnXt/IDAgqpbsp6JmXgDNf5BnUNP/SectiRgfDHSbnezYgs1x2YyExiz3yyjhZHFQdGHy6PRHless/RCrxMh7TCkxAQ08A80pke77WrwOqZBb9R3EvH+IaFvtChc/Cj6Gz6JtiSFZZiM7M/FrJkqd54vPuKCf8OSRVSupu5C0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: ARCH_DMA_MINALIGN represents the minimum (static) alignment for safe DMA operations while ARCH_KMALLOC_MINALIGN is the minimum kmalloc() objects alignment. Signed-off-by: Catalin Marinas Acked-by: Greg Kroah-Hartman --- drivers/usb/core/buffer.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/drivers/usb/core/buffer.c b/drivers/usb/core/buffer.c index fbb087b728dc..e21d8d106977 100644 --- a/drivers/usb/core/buffer.c +++ b/drivers/usb/core/buffer.c @@ -34,13 +34,13 @@ void __init usb_init_pool_max(void) { /* * The pool_max values must never be smaller than - * ARCH_KMALLOC_MINALIGN. + * ARCH_DMA_MINALIGN. */ - if (ARCH_KMALLOC_MINALIGN <= 32) + if (ARCH_DMA_MINALIGN <= 32) ; /* Original value is okay */ - else if (ARCH_KMALLOC_MINALIGN <= 64) + else if (ARCH_DMA_MINALIGN <= 64) pool_max[0] = 64; - else if (ARCH_KMALLOC_MINALIGN <= 128) + else if (ARCH_DMA_MINALIGN <= 128) pool_max[0] = 0; /* Don't use this pool */ else BUILD_BUG(); /* We don't allow this */ From patchwork Wed May 31 15:48:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13262461 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A968AC7EE31 for ; Wed, 31 May 2023 15:49:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 41A3A8E0008; Wed, 31 May 2023 11:49:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3CAB68E0003; Wed, 31 May 2023 11:49:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 292528E0008; Wed, 31 May 2023 11:49:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 17D708E0003 for ; Wed, 31 May 2023 11:49:18 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id E8215ADCFD for ; Wed, 31 May 2023 15:49:17 +0000 (UTC) X-FDA: 80850984354.29.E19ABE7 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf14.hostedemail.com (Postfix) with ESMTP id 4196210001A for ; Wed, 31 May 2023 15:49:16 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=none; spf=pass (imf14.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1685548156; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+fgUtZ7z2bEcsaerci/jffnaSF7K1ZtF0r/2TxWeNAQ=; b=LzhCnp1fBClzPdXzdYT4tiFTDQ4AYEPkUunrnhK6v+RwiyvoRlRyuXmuoSd4n4Ht917Ret S8qunu1fWuRdwINZ/pTHihYDnptKyxURRIqLQD0i9jzxdRS7TEuc4TnA10+FUisKFNuzQb Phr3GqE5zAyV1y72WZtajrJRL8CJnFw= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1685548156; a=rsa-sha256; cv=none; b=iKalRi1szyIjv+L8HfRTHz8stMQfQVuAAk7gfK3pG2HxGO1RcHaoz7EkyMiQE8VL2HpNBQ FyUliKjdFeTdQ055uj8caEQ83PTQJjPvQnaLJJZ97L7s8b0uI9tdgtPWPqlCjHHMQIq/AC euXo7nktGJN9seLv9bjHAlMuSSOTntA= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=none; spf=pass (imf14.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 7016663D92; Wed, 31 May 2023 15:49:15 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 25B5EC4339E; Wed, 31 May 2023 15:49:11 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Christoph Hellwig , Robin Murphy Cc: Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Jonathan Cameron , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v6 08/17] drivers/spi: Use ARCH_DMA_MINALIGN instead of ARCH_KMALLOC_MINALIGN Date: Wed, 31 May 2023 16:48:27 +0100 Message-Id: <20230531154836.1366225-9-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230531154836.1366225-1-catalin.marinas@arm.com> References: <20230531154836.1366225-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 4196210001A X-Rspam-User: X-Stat-Signature: omfndbngbmanxwgw9gqfqnzu4m4bfihh X-Rspamd-Server: rspam03 X-HE-Tag: 1685548156-953570 X-HE-Meta: U2FsdGVkX18rHd4C9a6CWeJ4niqHMCczbxESTvW5496wimyldhJHFP2+UfajETWDO38i/Kiigd2qjpBKhRSUdTDfrxXiuoS7kR2ANZk0ES1CyIzJ8ILF6Rsn0D8lSuabq8JO/GZrqyXZcVFHpehIhEAO2CORvetuqVQlHnIMi9DoKMeXju6oEK+BDdjwuA5cahjRqplwE5v1WkgLhJDdN+fqV21OkYN3LluUOkCjxnu/Zhue5s0rN/umYRzCorDExF0TvxumBzGIaeNTD2YAxWjQrYGLpMYGCQH5diSLkRy77l2lbPpr9BoH3OYsMBVpUGAZxZ9tQsCZ+U5BjlduOPy0x5M7olsEQoTCsshQLMJhzXk/U8N/lFRJqNP1mA1BbI+QnYIcv3hdZG+Iwll0Y0XACAgKSueUNYfLlOdWlCIDE+B+JZjmFN9RSaca/gsXWHBkLV4DGscJO8gSSJgIWuwxXC5M1PfR6wYNcZD46m//XQj3N/KJmu6A7FuQraFUT5O1qOuM1DkoOWJBNYFtrJlKl0YeWcyPmGWHFps9X14tzxHbo+PW8EMZm+no76DuCHa2Pi/010iFMZHv9hyKJEPOYCFWPr2LQzhpfFZybLZywRuzIwXJndnq/1p9OY/nUZR+xo2nU0t4pY7JiKSoJMWaLRnECxk1ociubI+hASjYPHwHYzEFDq8eyTmyis7MLHNuJjNucruR9Fp+uRqeeqwe8K8Vy3MkePY9OMKHmItO7SvATwBPiPneGvAmSuPOIp7aEzERi4plK4jQpUAJuU3/tg6a6mKCyABH4k42LtqnZMc9xPtpNrtQKuvHXCWvw0WRCHpibqPXcUZA7qdZdYB6qHga7hbCyZPE8LnbBnmH7odsKn9KxbRnRPWd47l6Fa83HqGkrziNMyD97fDOdGpOU/HuIVQ3p9fKyvTvYHfQW0QqZDpBOvzwowqPA1DX7Fo0VK5r8Ej3MUlMTpy fw+HvPv2 EbCDgDIq4zdgPFxCFO0GmipT4BSulDCz3b0rqvcDWaWl6Jo+YImIbfW3+RX6Cuna2uXC84v9drjC6RnBbG8YR71ngNqf5TGm8m880lSq5EcBWT73+yus0FRhJ/DqhWJuzINl4F1x6auq/INRlihm5XM21IYWL9gS4Qjhd5YisXKkun+h0DS7zO+dQIliYlP9LPIf6A7+M0bn4e38YxyXfCTuu3VJK7nzFCt5GhkB9y+DBZFb8ozAOi7bGoG+l8qSfuW7gG6SGWiEUX/WQFIQv/uZkc0Esggzmio8mHGH4B14Xi/e/DgnxwH0DiZnhhh+I7XDa3UJmeMd1GEnQ8YYoQ0pG4caFqblG1HZ/m3gMlcr/uyKgUpAqooRx1cgUlczybWJGCAdIugTrzY1iHPPpQPFh0rE5F5ZFK0bzfBPf0vRkD6jXI2zUFomu4zqunomplo723T/RoIuocXaDJi8JxGtCxohKt6NLh4QKSezbPeFX07uePl1yAi+Do0XGeQNRScd/Ld5LZaiNnOdl2JQ4QuuSIbV4ukqUWCSvb8aO+EISgaryWfBw4ZdVPOUVZFo2/pIC X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: ARCH_DMA_MINALIGN represents the minimum (static) alignment for safe DMA operations while ARCH_KMALLOC_MINALIGN is the minimum kmalloc() objects alignment. Signed-off-by: Catalin Marinas Acked-by: Mark Brown --- drivers/spi/spidev.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/spi/spidev.c b/drivers/spi/spidev.c index 39d94c850839..8d009275a59d 100644 --- a/drivers/spi/spidev.c +++ b/drivers/spi/spidev.c @@ -237,7 +237,7 @@ static int spidev_message(struct spidev_data *spidev, /* Ensure that also following allocations from rx_buf/tx_buf will meet * DMA alignment requirements. */ - unsigned int len_aligned = ALIGN(u_tmp->len, ARCH_KMALLOC_MINALIGN); + unsigned int len_aligned = ALIGN(u_tmp->len, ARCH_DMA_MINALIGN); k_tmp->len = u_tmp->len; From patchwork Wed May 31 15:48:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13262462 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9D8F2C77B73 for ; Wed, 31 May 2023 15:49:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3A4D88E0009; Wed, 31 May 2023 11:49:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 354DB8E0003; Wed, 31 May 2023 11:49:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 244588E0009; Wed, 31 May 2023 11:49:22 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 13A518E0003 for ; Wed, 31 May 2023 11:49:22 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id E9A481C7216 for ; Wed, 31 May 2023 15:49:21 +0000 (UTC) X-FDA: 80850984522.15.22C3E06 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf12.hostedemail.com (Postfix) with ESMTP id 4DB5B40018 for ; Wed, 31 May 2023 15:49:20 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf12.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1685548160; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=P3GKCRqSal+FoaPUZC73tQFjQIUPiL6KpjJp1J/vlzY=; b=obyb3MU0/+brEIkxX12KdoUks2k/17+Vxr3zOd9T2BKhBmvM0fG4tfq5bvczA5kid0aD49 iq9+pjdxPUZ7SndZejpuPqiZxNQIwYlBHNw9ldj3kOUgXgrueelGDgW86y3ceiBmKGorsP mqOTJWLjNw4yiKFNELoRMtdWrADDjKY= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf12.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1685548160; a=rsa-sha256; cv=none; b=tmJr1EcZGVHkX06R21tnZg6ZOy5Puf5dDOi7zbgRqhRYEvWRX5poucQGQeDz2PIKDP2ODF 9x6w3j75xBd4LSlRdFMjq8vUdu5m1TBkPDmUR2TbC+RdvkUYneifVyDbXsOQJsmS3eqGyf yvCWvTbGZbMGhD0aabAQZbp4AgEVGPQ= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 7CFF563DA1; Wed, 31 May 2023 15:49:19 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3EBEEC433AA; Wed, 31 May 2023 15:49:15 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Christoph Hellwig , Robin Murphy Cc: Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Jonathan Cameron , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v6 09/17] dm-crypt: Use ARCH_DMA_MINALIGN instead of ARCH_KMALLOC_MINALIGN Date: Wed, 31 May 2023 16:48:28 +0100 Message-Id: <20230531154836.1366225-10-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230531154836.1366225-1-catalin.marinas@arm.com> References: <20230531154836.1366225-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 4DB5B40018 X-Stat-Signature: 58o9hqx5zae7mxqg4b6mmchqejjjqtqq X-HE-Tag: 1685548160-612513 X-HE-Meta: U2FsdGVkX18xhvOEt6dkXaQcnQ4N6XFH0vNbN7YG2aU77l0tMlVrQ7HQHvJUzvHU2RYaeaMHxOp3/2tr8Tpw99I3xHRe/9lLoLfSJ11JKZSLfaxIGJJp+maf/2rRB4vhbgqgIvpyDll/FNw7i9TITDEk9RDsMLLmun6pzdk/Lv39E6LTrCJ8c/XR0UjT5fj/94LfWBESIPwJRFvqaLjceAMmug/UaVwdgWJCzwP4igCZJ2kE+vT5gPc25vrGxKOQUj248g1DGYzdBVCYGf/Eah5fGn1oss9HpKrjm4gMdJzHCJMJ1EigBxtDHmzAAPVfV6J8HzttcMuVih9IVaSmfPXD1GL6tG1d7nyHBB0T78fvxXjR8x9L3UjXpAGF1uGjPTzhk5zsDLL9cHgzHuoMFODv0NhphDCaIXKRJ9t392geohsl+Qrxyq1jU0cK8b8maxKUNWmu+YeqUp6H3kMC1lgqM4yWbYiZxg0Eom5xzjCqbQAdXOyDyx5as7dtAFay6Tztz8BSr4AtmZbLcLwHqzS1JTG7GSIy4+RlafBPHA+OXnzcwAcMf9ib3qwfCMRzjGzvg9ORj/F3BE2mObO9EuIZw3d2gqwbiDWhFfMZQH0bIxzn5BcI1zmzizyohcfdCLxY6x8HzoRZcoDRtdxQsvgwXvu59C3AwKIUfgmfG5vLj+6LtE5HlHTKtA8hCXOrIlIikzy0+Ml+NDRSKNBRjXJW9JiHzPdihxiVSOgzgXs8cMmxdgw/Aj8WWH+jRoG34nukMKhillbDIPcDyISAiyipwF8896WoXqVIpG9VcE3zzntE9uEFWb9ZAaS5Mo3HNsAaU73R5RQJnn+ICtrMVoe9B2KyBItPcOpcji8bMzfHy8ieuxjgmCfG1iEvJkNpZRV2+/uKmJ/mr9wk8RM8tBz82CJ98m76G2YaiSRedELubzzi3JcAXHGAPGUJE973dhHUcO/WgnQJi+fIuYe J9HYBPra n20VHMcindtiJeuJv+SJVw5MZ2/c/RrNDMwM4KHkcwQr8kCsEcxwftGkg8YdY9UXR9nHBdICY3qk4EUJv8ffWRhhIVvelfGx7zJ06sqMK6e0gLsr0vLqj7QNb0TzleCuVY2RcjLejiOFJ182Fzf/g0C2X0mhVZyTuC5IwSYgJdX+yozSXdFJwxCh/5wMcOQivqhd8s+XHenSh+PLlh5vM+j3F4zQvK1EdhYJg8aY3ta3N9wsPzA79LiQqED1vtYddECX5XhCGC2mlKuTalYV28Q0An/DS2sntTnQCj1gu3dTC5vMJXsRlNCX1JBSlQjgBKP9Ad7mzqrCq6zIQvBFyraNHWgmLCkD9yQhKYqGBOJi9CmHNGWt+iI1XuuK3VIGnI63cPGQnyrP8T6/V61dQT13RsuePT8M10MtmasREmCR4eZ5wU9VouRbqWO/YkRBdAXRbnqcrAda2SkTSMhPJfN/Jradm8ebyKlujlthgeStafjbw2vq20kaYH7ceEfyjYBe4jxo+XCZpMXrIoMehBN+cOnjJCIz0V7lI6/qcuSH27RlQ13kKBQcBd/FCC/NygBq4Uvx07eVvryZBsiQV6Wvunw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: ARCH_DMA_MINALIGN represents the minimum (static) alignment for safe DMA operations while ARCH_KMALLOC_MINALIGN is the minimum kmalloc() objects alignment. Signed-off-by: Catalin Marinas Cc: Alasdair Kergon Cc: Mike Snitzer --- drivers/md/dm-crypt.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/md/dm-crypt.c b/drivers/md/dm-crypt.c index 8b47b913ee83..ebbd8f7db880 100644 --- a/drivers/md/dm-crypt.c +++ b/drivers/md/dm-crypt.c @@ -3256,7 +3256,7 @@ static int crypt_ctr(struct dm_target *ti, unsigned int argc, char **argv) cc->per_bio_data_size = ti->per_io_data_size = ALIGN(sizeof(struct dm_crypt_io) + cc->dmreq_start + additional_req_size, - ARCH_KMALLOC_MINALIGN); + ARCH_DMA_MINALIGN); ret = mempool_init(&cc->page_pool, BIO_MAX_VECS, crypt_page_alloc, crypt_page_free, cc); if (ret) { From patchwork Wed May 31 15:48:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13262463 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F292CC77B7A for ; Wed, 31 May 2023 15:49:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9237C8E0005; Wed, 31 May 2023 11:49:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8D2B08E0003; Wed, 31 May 2023 11:49:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7C2788E0005; Wed, 31 May 2023 11:49:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 6D9EC8E0003 for ; Wed, 31 May 2023 11:49:26 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 486B11A0278 for ; Wed, 31 May 2023 15:49:26 +0000 (UTC) X-FDA: 80850984732.28.83F9636 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf19.hostedemail.com (Postfix) with ESMTP id 88A951A0030 for ; Wed, 31 May 2023 15:49:24 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf19.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1685548164; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UsSmHn96n500LAHNvSPWZPr8sPKm3BwiSbEVR6vxi/k=; b=IEgPbAtxelbOWNL+UnworJVI3bPv40c3R+xRIhUGR/5CPG9tljyZbgkT9nw+7GX3zSFOJP SlChOw5rmQhgTAYgEh3G72vqfSvdZLkdouBg67soOP65A3+kag5AoAojQTtBFNuFhd3zmH EidvpGlXPS3UmtWwJ+SRAMGlN7C/Slg= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf19.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1685548164; a=rsa-sha256; cv=none; b=haPTcsdYTntrAblScypOngH4lwIk382aBG1db4lc5uPQMhGPiS1lEgcCyaaAkOXAuH7G9+ suCemTnQU88aC3wtCn8tD02BFC9o9anWFCJtqjQ7EY4KGZe09Uud+/QwVfsfI8QvxBtHYa 2j/Q5NzUmZPRKz35sSf5QLAraU8UMME= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id A801E63D5F; Wed, 31 May 2023 15:49:23 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4C384C433A4; Wed, 31 May 2023 15:49:19 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Christoph Hellwig , Robin Murphy Cc: Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Jonathan Cameron , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org, Lars-Peter Clausen Subject: [PATCH v6 10/17] iio: core: Use ARCH_DMA_MINALIGN instead of ARCH_KMALLOC_MINALIGN Date: Wed, 31 May 2023 16:48:29 +0100 Message-Id: <20230531154836.1366225-11-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230531154836.1366225-1-catalin.marinas@arm.com> References: <20230531154836.1366225-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 88A951A0030 X-Stat-Signature: npw8kkho16qf5efaau956sub8hazpfkf X-Rspam-User: X-HE-Tag: 1685548164-897336 X-HE-Meta: U2FsdGVkX1/a31x+cY/f7iemM+pIfG7L5r4xGt5lxoNjmApS3KbRWwXQg+F4N988FYaCipE8SnjFOokAGUaIfXck95D986HecA6br+kwau0A8nbnmBfvMVY9B2RyEsqWvAGqHNLJbt4S7/sko+ZJEoPHNeVcVDMGWKm8RnQv4DJJWmL7kt1DUIeyRcVFvOgElUIhse1IfSOfPyUhx2mUe5f0Pt2Xd8qlluTFSwFJjzs150fEyoJKlf3Uzwn3Y/4gWAW/OtPLum5cyZAqRBquurib09DAGjQmhU9HALzMita2mjZLchV4/Q+JJcswyQWsIw4ajPMB+cLoh3twDcB2wU9mHS4KipD7P4C+bRcp/fXK+XMbRGejVxCYSBCDQXOlVCuflKvVjFSi77vH6WqnlERJuxQIHlR+rAW0yhjVFGP7Oj3HprxQAxJbcWQGgt2OG1LiLy4a47Qdqo0by7zFePJlv7B/kLP7jsebFaFB9FP7kSOIvmIsxrp8F5y5M2iofYcFsh94uSV5vbuicJU3Z3HQgwAagboAfVL8E9+iNaBz7Ln/EyIx9iGkNwnMZ8if1XgLGC8nDT/PGkwQn0Pp9lVGdei8M9CCvpcTf5OC50rzo4IpC1bA/PJiwmjoTnVSsnnrup+/Mi2dhvziKEWWiU/FN29Cphsz2Kj9eihoeeb/zEs0OlL/CA5lSD97iNgmANsNX0zjzIdwbqzcEImS7lns99/bTdqAvfLb7irl+YF/0o8tCVgcULU6qelOXi3rNZ4zkEVIdwC4xYWOCgSRgyR+MofAVgC60SQ2KS2/rSvTuyUaJQxaYR7FIQ+xyod1sD/z0ife0Ft+zqzBdTioj2isIH/+7gbs4mMU3qrnIOxgPio9/UDMS1AEUR5IOiPd/5ZphJ7amtENOyuKqAizd5QgQoEFWzTGwPfL0s9MXTfE5qWscMdwBRZUvb/cvZSHCAPksDVIAfWziKS50DW cB+vtZEs k8nLbyfjrSI/PVOou48mLubxEbOwK37y4F41VE/BiTcb+gQXDifAAigPncfN9muG6yOHiJO9dpEWnGpmskgEtgHICi/gJSMv4eLenP8x9glb/lVN1FQ7SZ0Ob0294EjYmbWNi5JqHJjj1S6L7CL48Qz9z7cAvSriz84VCmvmnYVnYHlYPsVgULsCtCJiHxDKnxQj22Xj4hrhSdl5pvPXee04dvkSyqAB15TMjmvFkyjR+FsDh7WwymIgrx7JamnXhzBcp9jeAnGX34B5xV8SklntpH+srmp9n3ez0TaQsZdNuSIswl/thci2GvfHfqvHiTWOdtPiRkXBlK6wVXEcpaj4cubnxx/kg1wCO2hSFaxf9u11qmg+VH29+1isdvfhYM06UdIWCnxOijc2ysyV3gQtnMXZlr92rl2h1RN+z2JhjyCtnsevtizudGFy9nqU2w2d6asCcSPrIdbP8UdwzliCjHdj1HxJ1DBEhERfCElBEOWmT3Y/PvznRleqIpih8uhE6K3v3BJuhAn5San7aB5yVMZkokZWkvjT91vDRtmA+slAXhzB0C9msoEAKx2lcMT+4 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: ARCH_DMA_MINALIGN represents the minimum (static) alignment for safe DMA operations while ARCH_KMALLOC_MINALIGN is the minimum kmalloc() objects alignment. Signed-off-by: Catalin Marinas Cc: Jonathan Cameron Cc: Lars-Peter Clausen --- include/linux/iio/iio.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/linux/iio/iio.h b/include/linux/iio/iio.h index 81413cd3a3e7..d28a5e8097e4 100644 --- a/include/linux/iio/iio.h +++ b/include/linux/iio/iio.h @@ -722,7 +722,7 @@ static inline void *iio_device_get_drvdata(const struct iio_dev *indio_dev) * must not share cachelines with the rest of the structure, thus making * them safe for use with non-coherent DMA. */ -#define IIO_DMA_MINALIGN ARCH_KMALLOC_MINALIGN +#define IIO_DMA_MINALIGN ARCH_DMA_MINALIGN struct iio_dev *iio_device_alloc(struct device *parent, int sizeof_priv); /* The information at the returned address is guaranteed to be cacheline aligned */ From patchwork Wed May 31 15:48:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13262464 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C0058C77B7A for ; Wed, 31 May 2023 15:49:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 552F08E000B; Wed, 31 May 2023 11:49:34 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 502D78E0003; Wed, 31 May 2023 11:49:34 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3CC608E000B; Wed, 31 May 2023 11:49:34 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 2DF3F8E0003 for ; Wed, 31 May 2023 11:49:34 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 8AFB6AE2CA for ; Wed, 31 May 2023 15:49:30 +0000 (UTC) X-FDA: 80850984900.13.93AFD87 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf17.hostedemail.com (Postfix) with ESMTP id A2BBC4001D for ; Wed, 31 May 2023 15:49:28 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=none; spf=pass (imf17.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1685548168; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Cf1L6WmjPMAYm9aSbUscBDbAibSbBWO2SXPG1VqERas=; b=lZL77tzWDMoP//yBjvTw3zz0n/N9i+kOCfIWR5UmrkgHwpHAsn7p380RvK8R2d1MfZYloJ YwDJm0o5TnkzxXhw6OO+eSW6xsltqNWa7wBwEjg3slgH/1/VHz74Vns2xwvPlRYia9OEo+ rE071srPyJLvLSlTerNmbqCrCcbjTEQ= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=none; spf=pass (imf17.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1685548168; a=rsa-sha256; cv=none; b=OLd1Znalbg5rM8M/27I/GGi5RxLYNii1ETTNMoMDRwnNbwAaZFkvDrKZebOWx0+g10U4eU XXwWc4rgy+rngWkyzlUjynm4mzdL6RO734tjUIFQn3XgEoC01O/R9RsJEOdDGcbREaZK3a AqspcYZOTLxPfwjypUPaZxfHrBhaJ3Q= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id BF7E063DA3; Wed, 31 May 2023 15:49:27 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7CE58C433A8; Wed, 31 May 2023 15:49:23 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Christoph Hellwig , Robin Murphy Cc: Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Jonathan Cameron , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v6 11/17] arm64: Allow kmalloc() caches aligned to the smaller cache_line_size() Date: Wed, 31 May 2023 16:48:30 +0100 Message-Id: <20230531154836.1366225-12-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230531154836.1366225-1-catalin.marinas@arm.com> References: <20230531154836.1366225-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: A2BBC4001D X-Rspam-User: X-Stat-Signature: y38wkjapibpn1cthfbc5erkc6x1g6e37 X-Rspamd-Server: rspam01 X-HE-Tag: 1685548168-949582 X-HE-Meta: U2FsdGVkX1+PKug+khGGY2ZCjC7kD7DUoAwg9HKVqWUb5DxLN7IzKZk31r5SewLA38cVavr2pR+8e7WhklxROF7FdJr1cUkuvJVPQxtikM2bzJJvNj4xJqRFvPdis9IZjlfRJS7stHFhU4xqXDcb5wYWvUOeC+Murq9o8bRb4MjU2BVL/OvGOPzBjRJCu5SbDG1eto1EhTRrK8/1im1x+Aty5i+S65BZjtygNZiNDESGgbI6UIMIBJPgeYJ88w4kVBCN1PtLeux4+EbHJhG2LJMCHN8dGLL1c6lAXQxf/eapXvSVj7nH3PW6z1j46t6zc5vFTmiVpIW1L8sbJrtT7l1zLpVKQ7HMiACnNBsmpGsQvG61WC5vY/5uUJN2NbguzmfjOzlh7lUZxzpaJ6e0N0ZIaKplvh3K1kORUcNK+gbLxRxPSnm5dDZO9JG1YcWE1qR6TK98V3eti8VOksySKorZZeNZDEj+M1ZL+iLywHcs4MV+XvbUD0GBqPkTo+mvMjGMIFHaUT80wJIM65BgPlh4mrSaFcOTSQ+lULudzUyVdZlcNOtQW1gqeelCQ5aI0VvoChi+AH/QE/+iMPBvMxKGxsVUX1FI45JjJ+1Q1ZUD5dmjkVzOCx0T2E3hy5WaIhqz+zThuD7IXaacL0hMDQX9eASybOBwvadALs3J0asRY2js7NwVuXjsCJNZjPFZQSwUhFsS0CgkUduKefAAmiQgCXCCd4xFEeKFDOLD+apvh4ugAzkiH8+S0tWWX6YlUcYlamahKBvy5YQSFUDe8LXxQGPsj3xkQQhUWo8md79wBDJOtc9o/dHrJzMKC4loPTMT4qW8uh55vkafQT9QqBAc9nWtmO9Jx9vPXF0WL4I6G2u2Pz7NeoTApkHsp1hEU2OqT58fNNjHvIy6wUR9q6GIIEb9T+FM+zH/FE+y91PsmKnzO02IuRb1AY9yvWl2EzjaKwLkidR/cimg/e4 +wgmJD+f 46Q0hRpZLqL4XEAyH0qnQX3Bnf74k0ReJ3BJNadWWx0cPT67zP0pHOHyU/GS55bG7MWvZ+eFwTmnoetXP2I47dsUd6GkNEhOH5VJJafxEvb0ZXMYbxhrvWu0EpDJDPTPviDSLAlFWT6NN2Ymz4l+k25OODY42il83IA5NSc0iC1ni+rB1ToKYM7l1BRAWSKYSoWDiSW0J0DW8NHaDXvDpqNvckSYbtdcvbp+8h6Mk/7wmgNqRdyBb5LsXLAxVglXM0KAZBcAyVBzcbTRixtNVg1qi3cxd2KO2ZJtwqd9+YxsJl/1E05UF2DG+RZztgkHx4Pq9oKruImZUIIn2/G7NfXBpjxib02hEav8iR8AORBEBGxviw1iuKQMHXPaU/uDAnwkn+SMC/fpTGRRiYsd3HtA/vqznKC8sdclaeKTG9KgdoDYi7OeCEfSmCLqHg2s0YSxozfFxoFDDahplwNNRVMbr5HxLM8AFsmMJv27u4JNzBHyFzZAQI/bMP+mCsJQg9cKQ47KjZu12DSEYW/71APKDY4YJwdDZe791SmrB3jSnLtU5ELdSOVg6vALA9DeJyjxF X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On arm64, ARCH_DMA_MINALIGN is 128, larger than the cache line size on most of the current platforms (typically 64). Define ARCH_KMALLOC_MINALIGN to 8 (the default for architectures without their own ARCH_DMA_MINALIGN) and override dma_get_cache_alignment() to return cache_line_size(), probed at run-time. The kmalloc() caches will be limited to the cache line size. This will allow the additional kmalloc-{64,192} caches on most arm64 platforms. Signed-off-by: Catalin Marinas Cc: Will Deacon --- arch/arm64/include/asm/cache.h | 3 +++ 1 file changed, 3 insertions(+) diff --git a/arch/arm64/include/asm/cache.h b/arch/arm64/include/asm/cache.h index a51e6e8f3171..ceb368d33bf4 100644 --- a/arch/arm64/include/asm/cache.h +++ b/arch/arm64/include/asm/cache.h @@ -33,6 +33,7 @@ * the CPU. */ #define ARCH_DMA_MINALIGN (128) +#define ARCH_KMALLOC_MINALIGN (8) #ifndef __ASSEMBLY__ @@ -90,6 +91,8 @@ static inline int cache_line_size_of_cpu(void) int cache_line_size(void); +#define dma_get_cache_alignment cache_line_size + /* * Read the effective value of CTR_EL0. * From patchwork Wed May 31 15:48:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13262465 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 53812C7EE31 for ; Wed, 31 May 2023 15:49:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E53C28E000C; Wed, 31 May 2023 11:49:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DDDBC8E0003; Wed, 31 May 2023 11:49:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CA4B38E000C; Wed, 31 May 2023 11:49:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id BB6B78E0003 for ; Wed, 31 May 2023 11:49:37 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 54E571A02AC for ; Wed, 31 May 2023 15:49:34 +0000 (UTC) X-FDA: 80850985068.04.761DC01 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf19.hostedemail.com (Postfix) with ESMTP id 9A8BE1A0028 for ; Wed, 31 May 2023 15:49:32 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf19.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1685548172; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6iELFwMU1BdH//ruMwUvhKCV6FH3nK4Bm5fzg3cJms4=; b=cCskYdOW3MrlS+WNn9BKuQd0n9RMMevlv8Nc3LtPuaqKFO0g/pPbv8zfn9eG2jXnU9mfJ6 qimQldh6OmfSLYWryd7LDrT/OA5fevjBqohO3uEMV+PmpBzcWWVoO3hRoYhO1MWQHmbgP6 0rKicNjnHeb0V9D6nE+iENxgasxKbRY= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf19.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1685548172; a=rsa-sha256; cv=none; b=nvoUxXCG9I8oe5k9hE9jXBtt5G5zYagmb5rlXVZ/AEyysS4DIeEhz9Ssbbrld2pmMxsj5P D/qBmCPnJRhAZgDCJbg8ZNSEkgKe0ol2ZVB18xihxIP3deX/ENOSofLzHp92Jn/A1YmdKS VMgyIgB+uUg05xtjT2P6ykBS+yacVF8= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id A59A663D9F; Wed, 31 May 2023 15:49:31 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8F1CDC433AE; Wed, 31 May 2023 15:49:27 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Christoph Hellwig , Robin Murphy Cc: Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Jonathan Cameron , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v6 12/17] scatterlist: Add dedicated config for DMA flags Date: Wed, 31 May 2023 16:48:31 +0100 Message-Id: <20230531154836.1366225-13-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230531154836.1366225-1-catalin.marinas@arm.com> References: <20230531154836.1366225-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: nrg34sybhx97ymgo5phw4h7kba9fzdbc X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 9A8BE1A0028 X-HE-Tag: 1685548172-959327 X-HE-Meta: U2FsdGVkX1/WauxI3p8iyKJML7YFKE9PTKb62frnIoeGDhK3vW3TFVg4FpCcOTl2UXns7eqWN4PQjNNcX01nO9/Wdjvq+O2RF4KPOKfJnKtAzNkgxJTaOZAj7CaKwASBHzuP4d7hLQUKTuXjpZIC2M6klJXnr/6D0ErSWWP4TNljpDe/GFPqc6V2uyKkBXzQFkU6eRXMTLqlH5KsMIS9Wu112t0tXBFQF1t+O/eT64pl2r6PeOqiEU+ZbSfwvlFHgExmmjwEoafDMk52bLJJeyhYhCojWsXeh+84rNRNK4Z+xGF5xQ4XhHgbcwigMQsyOoeQrdWeS4/7X+s5Dk+/sWohtLXmCvdhMCQLdR2uhw3Wv5m/Cqx1vrFQCsIXfDnzWXFvlbGcENfq4X0x69yuYrA0ALfUayw1xuwFrKg4p5eEvO3xJ9akfg/gjD2v+3TX5YI7Giov/gRJJ1NLK9JQLkseTLRbAoqBBalBIt+/4xYLYULm/06eARpvA5g4Qg51zk2mXi9cwiVbTY4lVgSB5p6s1NwAfpX6LuRvePQi8djQXHzdeiwwJrDynpxvOXrmJvYWp2XY5XBfFgPyVyry4NcWgDNWRSnyQF4NKLzWvL9BH+tikyJs3mRvWnuWT7/y185tWMgxu3yYlYmTaQ8dMcs6AV3CLFJ4kd/6rBcDummakBiAtXyRcEg1dSQ8kFGpWoREbKk9kqKTnrbN3oTIq70G3CXY6STir/bmqqUa8siCIlCl3ZNclgxYrWD67yN4PyM91Z2PX6VDxC+1a5yRAEzameYRU8u6d4eJiUA3UTAjV9ilEyss//U/s2z18mjkQllSplovvdXRGvdE1Icfh3znge6l/35cqKFfEFx2xcWXIVvvTawFJ6hEmi2o7YsYv/YJWJl0BXV14ulX3LXR7+2WKCAyaYydkk73NBh5nZnnYwtfshcciwHXFXVJ24o1eSQp8BDgWW+gSNV8XBP WbJDPatZ k486RcPLhcP5MxZ73jxsVeco+QDsGE/WEPDbOlf3ylq+nNMIJb4sYN2VTFoKqWfcwDEu7xRL8+v8IeZiPK0pUl4mwmkYNOoqdCYiER9qEl84xEku6xgFzJ0kUXcPw83xjQ8O86b+bYhgugA2y6Hny2f/13+PKNs/w/O9gzIfQMupzOVnhfSVnNheNKAeuSGygoDvrNTLTHSDLRnzG3NHAISm5Oty26ithexxQxPoIWxqZX5qw9zqbn6NEjbqw1uXVUucX9I728JHD2W6UD3lSO//XJ4oGEanU5a1BAbeZCDayYTJKo4CDFnWsQEEAXImBA/MtKo0LxZyIbht8UphJxgHQszZOSWMgpAZssQZvho107ZNlRzt3o3aNSWummVXygLV4Ec11n/9D4EmMrRf2hejyI/IiQX9xY6IN0N10DeFyrei1/txmTHv4clxK/vVmPzkRzVT2Y7ucLre1UhOGjurZYPZzLj01/x6P X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Robin Murphy The DMA flags field will be useful for users beyond PCI P2P, so upgrade to its own dedicated config option. Signed-off-by: Robin Murphy [catalin.marinas@arm.com: use #ifdef CONFIG_NEED_SG_DMA_FLAGS in scatterlist.h] [catalin.marinas@arm.com: update PCI_P2PDMA dma_flags comment in scatterlist.h] Signed-off-by: Catalin Marinas Reviewed-by: Christoph Hellwig --- drivers/pci/Kconfig | 1 + include/linux/scatterlist.h | 13 ++++++------- kernel/dma/Kconfig | 3 +++ 3 files changed, 10 insertions(+), 7 deletions(-) diff --git a/drivers/pci/Kconfig b/drivers/pci/Kconfig index 9309f2469b41..3c07d8d214b3 100644 --- a/drivers/pci/Kconfig +++ b/drivers/pci/Kconfig @@ -168,6 +168,7 @@ config PCI_P2PDMA # depends on 64BIT select GENERIC_ALLOCATOR + select NEED_SG_DMA_FLAGS help Enableѕ drivers to do PCI peer-to-peer transactions to and from BARs that are exposed in other devices that are the part of diff --git a/include/linux/scatterlist.h b/include/linux/scatterlist.h index 375a5e90d86a..19833fd4113b 100644 --- a/include/linux/scatterlist.h +++ b/include/linux/scatterlist.h @@ -16,7 +16,7 @@ struct scatterlist { #ifdef CONFIG_NEED_SG_DMA_LENGTH unsigned int dma_length; #endif -#ifdef CONFIG_PCI_P2PDMA +#ifdef CONFIG_NEED_SG_DMA_FLAGS unsigned int dma_flags; #endif }; @@ -249,12 +249,11 @@ static inline void sg_unmark_end(struct scatterlist *sg) } /* - * CONFGI_PCI_P2PDMA depends on CONFIG_64BIT which means there is 4 bytes - * in struct scatterlist (assuming also CONFIG_NEED_SG_DMA_LENGTH is set). - * Use this padding for DMA flags bits to indicate when a specific - * dma address is a bus address. + * One 64-bit architectures there is a 4-byte padding in struct scatterlist + * (assuming also CONFIG_NEED_SG_DMA_LENGTH is set). Use this padding for DMA + * flags bits to indicate when a specific dma address is a bus address. */ -#ifdef CONFIG_PCI_P2PDMA +#ifdef CONFIG_NEED_SG_DMA_FLAGS #define SG_DMA_BUS_ADDRESS (1 << 0) @@ -312,7 +311,7 @@ static inline void sg_dma_unmark_bus_address(struct scatterlist *sg) { } -#endif +#endif /* CONFIG_NEED_SG_DMA_FLAGS */ /** * sg_phys - Return physical address of an sg entry diff --git a/kernel/dma/Kconfig b/kernel/dma/Kconfig index 6677d0e64d27..acc6f231259c 100644 --- a/kernel/dma/Kconfig +++ b/kernel/dma/Kconfig @@ -24,6 +24,9 @@ config DMA_OPS_BYPASS config ARCH_HAS_DMA_MAP_DIRECT bool +config NEED_SG_DMA_FLAGS + bool + config NEED_SG_DMA_LENGTH bool From patchwork Wed May 31 15:48:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13262476 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 03D8BC7EE2F for ; Wed, 31 May 2023 15:49:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 94AF88E000D; Wed, 31 May 2023 11:49:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8FAC28E0003; Wed, 31 May 2023 11:49:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 726458E000D; Wed, 31 May 2023 11:49:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 654FE8E0003 for ; Wed, 31 May 2023 11:49:39 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 28794802C0 for ; Wed, 31 May 2023 15:49:39 +0000 (UTC) X-FDA: 80850985278.18.CEF738C Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf09.hostedemail.com (Postfix) with ESMTP id 4FABA140019 for ; Wed, 31 May 2023 15:49:36 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf09.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1685548177; a=rsa-sha256; cv=none; b=OBWStHF2Pr4z+YRI4v80iuGNHCvCmkDFY/g8wMa9q/fqq4EYqYmZT2MFUp3yyw9xX6UKhT GJNI4tD8ZsEzOS+Sz86m3RQ3YrRt+jVW9feAjQMcbWgAebhZDbNthWOimQZC71Fj8/Rr6+ 5PHkPYYT23AdDp9Aeyh5RRL2Rfj/8rQ= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf09.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1685548177; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=u1UyWoVJ6Ep1SVIdrbWokxdwP4Oi3eNvYJYbfxSDnZ0=; b=o4lXYBvRMbnfLbkePqPZHj0GaaXmtOZ1oghS1q1AcTSnqi9/CfUXdArvvues97fN9cuIjj BlvRUZMNXrh/yFhxtca6etjODMoE0Oe3ieTxhb4QOAxShuSIbs1Yrmi5gaeq4kPiXR/1fj CqgJKNbGuSADjD21CEuptKsd3SWZbuU= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 556EA63D5F; Wed, 31 May 2023 15:49:36 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id BC8BDC433A1; Wed, 31 May 2023 15:49:31 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Christoph Hellwig , Robin Murphy Cc: Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Jonathan Cameron , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org, Jerry Snitselaar , Logan Gunthorpe Subject: [PATCH v6 13/17] dma-mapping: Name SG DMA flag helpers consistently Date: Wed, 31 May 2023 16:48:32 +0100 Message-Id: <20230531154836.1366225-14-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230531154836.1366225-1-catalin.marinas@arm.com> References: <20230531154836.1366225-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 4FABA140019 X-Stat-Signature: nre4hx18xcxbezcud57nrgdg47jnm7xa X-Rspam-User: X-HE-Tag: 1685548176-55773 X-HE-Meta: U2FsdGVkX19dQIbELjEsZpBDEc9qkwiZQcCjJkBFdb+9yLfRAIQFx0tRvfs6qacuSv/w7SFbMHQSkDcbSiJ+WOs722WRvJRprK9e9AKh+Pz4XMJ4KGC/iSVk4rGaKPIuYUXdqJxYPJlUcDwwlUHbAcG2tS75bjBHQ5RdS08+7Cwoo7fQY/khoj2bgVVXnbkmH2zfpN4ydU8icdvP2M3vkk3wGGB9IFPgKU6Kvac3DjP/CNPF1FRLq+POQpMbxZZFVoVK3wcz8DhVbRnPPCmxBUjbkTEIUVMVmwmsknzlu9PetM9PHZy5j8O6PW1vWbJV2Riffy2WeJIzDIXTgLS2srOCDRXo4VW2rCRlDMx9fXbtUmtVCny7vWRzziCI3H3DH0NPZndhjy66BiNE0TehJISh0gsCQYCqiEFzuwBFVx4UP9rXxNO0EhT9APzFNYo8SpyOPHLnAHvT51cxS9gAjQ9Ld4orGZHiyhw1b9x2VujETGJ9UUaWpFhngsKrQdY6pJxL3lF46/t8obJVK/FJ0mpqx/NZOBtUd0zSzWf71QbOgKx7J1L/jcZD63btWWgiQwHs6IohC2xw1lMmKKFaueBEaS/nzHwUnYhbPak4bfVcWjREzrmlNdECyJMXH9NYAPM1dxTFJy48pDoe5lKS7brTyLmNuD23xo8mNTZ/NBRQQO+CtEWhf8jPnT3aXXCfI9ukLdLHzhkOO0lehJfVsjXlxCpjkuXtEXytvwvsiWIW4VsdrYJEIJV+2CmUabHfIJli68t82s4KvIA0xbSJJXnkNNgbXF8/J/F9vEfGJYnMO51kkifaw6HNU5ynAu+M2IIoDJfW0+OrPHE5SP+uZH50uQrV0yOcxiVs4Hu7gjKJ1UGLq2a8qTa2lLdR0OsKaHKrQDs01wavxZpVu3b2KDLmy1Ud+2cLh5Vpo+9mG2KGSNmz7XrbyDXNv97DtWWdarxHsenjGHU+W/Ci23V tOn5PwZw k28uPEx6VkfinAz8w86FbMNprKb1aorGaYwYlhpTTinVFC1PjGQF0MfyuU2d/ai+8n+6y0URXnFWZvbQbooN2mPfvqLiwjxJudQCA1C4PlbLlj5Hyf+QYSyadm7OA8ly61/eB8rKKj+ZH6jv+OEAnE0q9WHxphypftuWdjvxZArsczIZdG/h1YOhOH0gEwemR+SpcRK4qWXdh7blOu+0Bs+ydodJylcSP8L6CDPVF1XJfVwhtbw1SpTWnkJxp3NAUckIoBUD1aN+HAikUqK8NCxGZFvlUkg+2ejQ1YhrZyPYn59A9EtfYL6jO1cZB97sjsa9QXXk7N30skafPnuVFpiF9DVATQ6TH4gR1n1jOpdALMlsA3wEv7HyNUkHJ5SOgRSM/NBwulO7kpT0VvsCBsI/0pSgtK0S8ZjWNkK1xF7mno+ihPSoyCUzSpPTfPBe1fmrTM8qMyqfNvDfgzwQlROLc2vyXs4RnsUn44x5fvNQd8EWeqYfO27eVbgjDUJWGIyyTAe3YajaNJ2pIiaUw3CJgILjxIkDmag0eu5VpHCOZqs8kNj/Ckm2UiI7azkY/L42QSIQMTPD2hCmPatbcaCXvtgbUDRJFRD1D X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Robin Murphy sg_is_dma_bus_address() is inconsistent with the naming pattern of its corresponding setters and its own kerneldoc, so take the majority vote and rename it sg_dma_is_bus_address() (and fix up the missing underscores in the kerneldoc too). This gives us a nice clear pattern where SG DMA flags are SG_DMA_, and the helpers for acting on them are sg_dma__(). Signed-off-by: Robin Murphy Reviewed-by: Christoph Hellwig Reviewed-by: Jerry Snitselaar Reviewed-by: Logan Gunthorpe Link: https://lore.kernel.org/r/fa2eca2862c7ffc41b50337abffb2dfd2864d3ea.1685036694.git.robin.murphy@arm.com Signed-off-by: Catalin Marinas --- drivers/iommu/dma-iommu.c | 8 ++++---- drivers/iommu/iommu.c | 2 +- include/linux/scatterlist.h | 8 ++++---- kernel/dma/direct.c | 2 +- 4 files changed, 10 insertions(+), 10 deletions(-) diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c index 7a9f0b0bddbd..b8bba4aa196f 100644 --- a/drivers/iommu/dma-iommu.c +++ b/drivers/iommu/dma-iommu.c @@ -1080,7 +1080,7 @@ static int __finalise_sg(struct device *dev, struct scatterlist *sg, int nents, sg_dma_address(s) = DMA_MAPPING_ERROR; sg_dma_len(s) = 0; - if (sg_is_dma_bus_address(s)) { + if (sg_dma_is_bus_address(s)) { if (i > 0) cur = sg_next(cur); @@ -1136,7 +1136,7 @@ static void __invalidate_sg(struct scatterlist *sg, int nents) int i; for_each_sg(sg, s, nents, i) { - if (sg_is_dma_bus_address(s)) { + if (sg_dma_is_bus_address(s)) { sg_dma_unmark_bus_address(s); } else { if (sg_dma_address(s) != DMA_MAPPING_ERROR) @@ -1329,7 +1329,7 @@ static void iommu_dma_unmap_sg(struct device *dev, struct scatterlist *sg, * just have to be determined. */ for_each_sg(sg, tmp, nents, i) { - if (sg_is_dma_bus_address(tmp)) { + if (sg_dma_is_bus_address(tmp)) { sg_dma_unmark_bus_address(tmp); continue; } @@ -1343,7 +1343,7 @@ static void iommu_dma_unmap_sg(struct device *dev, struct scatterlist *sg, nents -= i; for_each_sg(tmp, tmp, nents, i) { - if (sg_is_dma_bus_address(tmp)) { + if (sg_dma_is_bus_address(tmp)) { sg_dma_unmark_bus_address(tmp); continue; } diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index f1dcfa3f1a1b..eb620552967b 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -2567,7 +2567,7 @@ ssize_t iommu_map_sg(struct iommu_domain *domain, unsigned long iova, len = 0; } - if (sg_is_dma_bus_address(sg)) + if (sg_dma_is_bus_address(sg)) goto next; if (len) { diff --git a/include/linux/scatterlist.h b/include/linux/scatterlist.h index 19833fd4113b..2f06178996ba 100644 --- a/include/linux/scatterlist.h +++ b/include/linux/scatterlist.h @@ -258,7 +258,7 @@ static inline void sg_unmark_end(struct scatterlist *sg) #define SG_DMA_BUS_ADDRESS (1 << 0) /** - * sg_dma_is_bus address - Return whether a given segment was marked + * sg_dma_is_bus_address - Return whether a given segment was marked * as a bus address * @sg: SG entry * @@ -266,13 +266,13 @@ static inline void sg_unmark_end(struct scatterlist *sg) * Returns true if sg_dma_mark_bus_address() has been called on * this segment. **/ -static inline bool sg_is_dma_bus_address(struct scatterlist *sg) +static inline bool sg_dma_is_bus_address(struct scatterlist *sg) { return sg->dma_flags & SG_DMA_BUS_ADDRESS; } /** - * sg_dma_mark_bus address - Mark the scatterlist entry as a bus address + * sg_dma_mark_bus_address - Mark the scatterlist entry as a bus address * @sg: SG entry * * Description: @@ -300,7 +300,7 @@ static inline void sg_dma_unmark_bus_address(struct scatterlist *sg) #else -static inline bool sg_is_dma_bus_address(struct scatterlist *sg) +static inline bool sg_dma_is_bus_address(struct scatterlist *sg) { return false; } diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c index 5595d1d5cdcc..d29cade048db 100644 --- a/kernel/dma/direct.c +++ b/kernel/dma/direct.c @@ -463,7 +463,7 @@ void dma_direct_unmap_sg(struct device *dev, struct scatterlist *sgl, int i; for_each_sg(sgl, sg, nents, i) { - if (sg_is_dma_bus_address(sg)) + if (sg_dma_is_bus_address(sg)) sg_dma_unmark_bus_address(sg); else dma_direct_unmap_page(dev, sg->dma_address, From patchwork Wed May 31 15:48:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13262477 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D9DE1C77B7A for ; Wed, 31 May 2023 15:49:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 79D54900002; Wed, 31 May 2023 11:49:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 74CC68E0003; Wed, 31 May 2023 11:49:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 61480900002; Wed, 31 May 2023 11:49:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 513858E0003 for ; Wed, 31 May 2023 11:49:43 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 08CC040277 for ; Wed, 31 May 2023 15:49:43 +0000 (UTC) X-FDA: 80850985446.08.5C35188 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf28.hostedemail.com (Postfix) with ESMTP id 54963C001C for ; Wed, 31 May 2023 15:49:41 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=none; spf=pass (imf28.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1685548181; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=trBjKA2P3t7Lv44FdRPNpP5ETnPfkIlOly8ehL5hPTw=; b=b88AmAW0nMxrolBn6lKgNzkgylpj38S2V40TcbjMdDlicck6apt453WYgys3saU42MZ4Gt YO1CDUsiC5NeYR2G29UhigAxPKb8jBuAA3G1q2QlSibokUWMBT7gsKi5xbqEqgkNC2qlVO tbHtMQuWSZ4cJ8XE5cC9bHjeRAwnF2E= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1685548181; a=rsa-sha256; cv=none; b=5DVb4BA7mVTMV2JFfw8XDC78YwivqpXW1Z5/5SdTtdNThEjdgUdlYtE8jh5yUF2f5wEYer etdMHzSW2nxSdqBK2SAYBlPCQIieQxPrmOY6mrdbIJ6knu+YtBrDqoE0R/Ig/hJae3imY2 iQpr06EfWj3jFLXF2VRyjCMMSXr9j9Q= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=none; spf=pass (imf28.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 6AFC663D9B; Wed, 31 May 2023 15:49:40 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 26227C4339E; Wed, 31 May 2023 15:49:36 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Christoph Hellwig , Robin Murphy Cc: Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Jonathan Cameron , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v6 14/17] dma-mapping: Force bouncing if the kmalloc() size is not cache-line-aligned Date: Wed, 31 May 2023 16:48:33 +0100 Message-Id: <20230531154836.1366225-15-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230531154836.1366225-1-catalin.marinas@arm.com> References: <20230531154836.1366225-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 54963C001C X-Rspam-User: X-Stat-Signature: nyixsqouzi41eb97rers1hqa4in99tb4 X-Rspamd-Server: rspam03 X-HE-Tag: 1685548181-792124 X-HE-Meta: U2FsdGVkX1/ElOjxQGD7K+OztTcG+Ej/I795vBD8mIh8x+jqA5X+gYum5UpLB7eKv95rEa6vYi9xiLMXFFQeO0qkl3yuj6anyHJpu3EILL/pT+yAx42WI+8ybEZ6gyDH3KNg+ghaIQ6o+y/j/Wcl9uZca3PM2S3tfuLFuUuESPquYUA7LrONOoeh5InpK4hzikCqu25AQZg8elzAi6/oZqZZ1rDqw9Hxwriv30Vt+7qr6D2Pl5YCwRsiK96+3i8yEZkDuHs0zE+4ndK557YCd/RaoZGWCuLVMVV1ZdGrJYAQLOZpvslUNqCyTL2mQtfChWhVsx6sqGDMy6EKxsrRKd1NV0MyAOgQQaoVZQGtFjnpFOawrXEkFlus1Vc8wKpWJ/dgKKY0qif6PfllYVuodqtFzJgL6nVi747PoB2wgIcHQ+jwo2vY+NimB+5AupjFjnX6+/fjYy+96B2a6etXKfele4E6v+TxeOM1CjZuiqs458sJbtrTZJMRXN+FjXKVeX5CkkqqkFajo/78CJbtVQfAIoNSM6+H9dRplwbnCD5WJvviis7vTbh/E2NuggXoWHp8BV97Gps9WRnjN6vC9k+B+IzZIj1xL5BzwXX3EktqYQDJFmXllpRT5DC9LqrLDv9A/7KFkC4pM+1EsJM7rsz5Jlgn5QfTHouXxaJbd/pbotXvt2CW6pCv4M/xVko8sMSCsqrEUDAew5mKPKx4PCRybIpGyOkCPE5UsJZk8E9kFKRse5N0pZFmEQgxNcPjXwU9bzwhUIkWGsYVL+OotZIPfSdsl0mHZh8tuV5ejMbLloiSs0SaP7QFj/YQnnwfAkHqyap9HFJAUetdI4olE0ghEciiE+DRDRF4ukKeJDgvYYZQTYe/cvRsAbHPt7NUYbKTzK5vrg+3mJ8BVSNZcipSWML5RjVLeHspoCFdZaUZ6zFNpb7Gz73qkdjMTBlrC9NV3YsuA0gBNR4T8Eg O8sjcLRx bMsoagtuwUAfQVGaPnKOHqQQrVQSdBbIKZ18NEXM4GvIYfYhsbiCCP7fQ6ot4/8iUkduEKSwmIU7HVf4Y9EY7DLKoDdMAJ9j0QaNnVGOy+tzXbrPXvBo9pbKf+JkHOw40XshszRYW7kzU2GWi8bWmtzpfiMdsPYf11RJ8vgEuRyzIuWE3P2mdv3fpLjMu6jCN5Z/EHF+poEYSJZQ7TfnIuSxSN6HdoPrqmEaG6tSQ+1hARx5HTkDLbEolaPd7T8F0OYhF130iKDFMzopV53SMlE3eqg/pfjtEo9fCq4+tSou8I+qM9FIDFljuSz1wCJ7ZkyCko9T6xsoCjWIx1VyrXzb2ga47dCTt5aXjs6tAbwH7Fe77dhVVOO4Z9QAD97FZCfvrdnUxT1GzXI+ZwK5TaEvvLeAo4eTCQwa3zHo1iO2vddARq3cMS7J41b/tWo6QJxDpRR0zh+XJ8aRYNAJWPIsNPw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: For direct DMA, if the size is small enough to have originated from a kmalloc() cache below ARCH_DMA_MINALIGN, check its alignment against dma_get_cache_alignment() and bounce if necessary. For larger sizes, it is the responsibility of the DMA API caller to ensure proper alignment. At this point, the kmalloc() caches are properly aligned but this will change in a subsequent patch. Architectures can opt in by selecting ARCH_WANT_KMALLOC_DMA_BOUNCE. Signed-off-by: Catalin Marinas Reviewed-by: Christoph Hellwig Cc: Robin Murphy --- include/linux/dma-map-ops.h | 61 +++++++++++++++++++++++++++++++++++++ kernel/dma/Kconfig | 4 +++ kernel/dma/direct.h | 3 +- 3 files changed, 67 insertions(+), 1 deletion(-) diff --git a/include/linux/dma-map-ops.h b/include/linux/dma-map-ops.h index 31f114f486c4..9bf19b5bf755 100644 --- a/include/linux/dma-map-ops.h +++ b/include/linux/dma-map-ops.h @@ -8,6 +8,7 @@ #include #include +#include struct cma; @@ -277,6 +278,66 @@ static inline bool dev_is_dma_coherent(struct device *dev) } #endif /* CONFIG_ARCH_HAS_DMA_COHERENCE_H */ +/* + * Check whether potential kmalloc() buffers are safe for non-coherent DMA. + */ +static inline bool dma_kmalloc_safe(struct device *dev, + enum dma_data_direction dir) +{ + /* + * If DMA bouncing of kmalloc() buffers is disabled, the kmalloc() + * caches have already been aligned to a DMA-safe size. + */ + if (!IS_ENABLED(CONFIG_DMA_BOUNCE_UNALIGNED_KMALLOC)) + return true; + + /* + * kmalloc() buffers are DMA-safe irrespective of size if the device + * is coherent or the direction is DMA_TO_DEVICE (non-desctructive + * cache maintenance and benign cache line evictions). + */ + if (dev_is_dma_coherent(dev) || dir == DMA_TO_DEVICE) + return true; + + return false; +} + +/* + * Check whether the given size, assuming it is for a kmalloc()'ed buffer, is + * sufficiently aligned for non-coherent DMA. + */ +static inline bool dma_kmalloc_size_aligned(size_t size) +{ + /* + * Larger kmalloc() sizes are guaranteed to be aligned to + * ARCH_DMA_MINALIGN. + */ + if (size >= 2 * ARCH_DMA_MINALIGN || + IS_ALIGNED(kmalloc_size_roundup(size), dma_get_cache_alignment())) + return true; + + return false; +} + +/* + * Check whether the given object size may have originated from a kmalloc() + * buffer with a slab alignment below the DMA-safe alignment and needs + * bouncing for non-coherent DMA. The pointer alignment is not considered and + * in-structure DMA-safe offsets are the responsibility of the caller. Such + * code should use the static ARCH_DMA_MINALIGN for compiler annotations. + * + * The heuristics can have false positives, bouncing unnecessarily, though the + * buffers would be small. False negatives are theoretically possible if, for + * example, multiple small kmalloc() buffers are coalesced into a larger + * buffer that passes the alignment check. There are no such known constructs + * in the kernel. + */ +static inline bool dma_kmalloc_needs_bounce(struct device *dev, size_t size, + enum dma_data_direction dir) +{ + return !dma_kmalloc_safe(dev, dir) && !dma_kmalloc_size_aligned(size); +} + void *arch_dma_alloc(struct device *dev, size_t size, dma_addr_t *dma_handle, gfp_t gfp, unsigned long attrs); void arch_dma_free(struct device *dev, size_t size, void *cpu_addr, diff --git a/kernel/dma/Kconfig b/kernel/dma/Kconfig index acc6f231259c..abea1823fe21 100644 --- a/kernel/dma/Kconfig +++ b/kernel/dma/Kconfig @@ -90,6 +90,10 @@ config SWIOTLB bool select NEED_DMA_MAP_STATE +config DMA_BOUNCE_UNALIGNED_KMALLOC + bool + depends on SWIOTLB + config DMA_RESTRICTED_POOL bool "DMA Restricted Pool" depends on OF && OF_RESERVED_MEM && SWIOTLB diff --git a/kernel/dma/direct.h b/kernel/dma/direct.h index e38ffc5e6bdd..97ec892ea0b5 100644 --- a/kernel/dma/direct.h +++ b/kernel/dma/direct.h @@ -94,7 +94,8 @@ static inline dma_addr_t dma_direct_map_page(struct device *dev, return swiotlb_map(dev, phys, size, dir, attrs); } - if (unlikely(!dma_capable(dev, dma_addr, size, true))) { + if (unlikely(!dma_capable(dev, dma_addr, size, true)) || + dma_kmalloc_needs_bounce(dev, size, dir)) { if (is_pci_p2pdma_page(page)) return DMA_MAPPING_ERROR; if (is_swiotlb_active(dev)) From patchwork Wed May 31 15:48:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13262478 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5AFA0C77B7A for ; Wed, 31 May 2023 15:49:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E2C6E8E0003; Wed, 31 May 2023 11:49:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DDCA1900003; Wed, 31 May 2023 11:49:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CA4228E000E; Wed, 31 May 2023 11:49:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id BD6088E0003 for ; Wed, 31 May 2023 11:49:47 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 83F6B16028D for ; Wed, 31 May 2023 15:49:47 +0000 (UTC) X-FDA: 80850985614.21.895012E Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf27.hostedemail.com (Postfix) with ESMTP id 9BB494001B for ; Wed, 31 May 2023 15:49:45 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=none; spf=pass (imf27.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1685548185; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=djcU7uQ6jO6NWmeykoF7MFcFDUy8GO+UVTvXmdJHI6s=; b=sFXyBs3Ti0rsbHlJyHqmDebbVqpvsNbJLP3aMn6Jg3mzzoVYl0HzOGHXTvGWCTvZpqI4+B UADS+3lchbNUvzsBTjJpqu6wTNT9c2wJ0VfwGQ6dq/VLzKJYUv5VflUYO4gXEb0+1kl6BQ IOmMgx5ciGI792w4zdatSmINp3WyV7U= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=none; spf=pass (imf27.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1685548185; a=rsa-sha256; cv=none; b=dFuCFnIgrcJ+mWRhB2UYC+eHwd/heH9SkkH0/EGI2eIrllZi38mHcqD6lWvATY+Mn+rSsQ MLkd1q4yX63HJ9CP2MZSjiC1nVuBIpKpS5X4zafRLXRZ5CEmSdu+i+szByyJFb8Fi7Tp4o NdTIbudI0Iw3wJR9S2J50Pi/5/pVCiU= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 757AF62B59; Wed, 31 May 2023 15:49:44 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 35ADEC433A1; Wed, 31 May 2023 15:49:40 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Christoph Hellwig , Robin Murphy Cc: Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Jonathan Cameron , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v6 15/17] iommu/dma: Force bouncing if the size is not cacheline-aligned Date: Wed, 31 May 2023 16:48:34 +0100 Message-Id: <20230531154836.1366225-16-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230531154836.1366225-1-catalin.marinas@arm.com> References: <20230531154836.1366225-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 9BB494001B X-Rspam-User: X-Stat-Signature: xpng5uj4xiu8f6bgpytk474q6y444km7 X-Rspamd-Server: rspam01 X-HE-Tag: 1685548185-979301 X-HE-Meta: U2FsdGVkX1/qndESNkDdPu5V6YmpEpMU19nUs3TJTp11dQu+8hOKEcL27QFom7JW/6t/l25SqLb1WUy9W5sUuuohhXPs1wno9oBTBVpufUDe/DWEjYLNfsr+2VlEc8IwFZFdq9qx07C3T+dEO90rgVlKa2MqRhdnDimyj80+3OpGFtKepWpa4mMyyKdYaPIgl+0gMlNToFKdAEy1fmIGb403wUcwKN8kOQ4f5V1HRZ1ABq0Z86/tiaRRkeYWFSsx9gjr3kxyqRZUkQ5ZDev5tbWkJggwaJjuBVdanjiswo7xGxhJrVLt7FN/ZHfnEu0s2iP3g9LXtQLSQBiKl4emwtRM9wr05fwUzWdrdimI/MSwHneljQARZjOHz1++AVgau3wTkcUYP74wKzhH9MGoO0uLZkS3q39vB+i1BpGt9cvxw8MlOqQ8dwuXgNhw4gKpUSY+T8jEpuNFVA05tK+R/YsRVuX8o6fTgE619GUxvuj+Fsw+PzU9XPjVdPkQp8Pwk7pm+hlPwRs9PeFJ7LzogkgPBpkfFAANPtUz6VBGgZlt9sbH6Clhri/T7pJitr2ql58uDlBI9k8SP4YXKGQYazFabJzALVwBpweOKmUbx6p0rZuhst774ljVbxZNQA9aokyeTlVj+swv3e1gHsdY+fVMH0F3IFKhebgLpSdI4Lcg5lwirdlOQd3GpWuYt3yGwhb3kpNOOkPfN9w0/QE93yb8KRhmJlY8m0jO37BJ44+bOeaxt0i1UWkJM9Fof0zvcswHEeUwaK0v4YC4ZNk9WxehZMku9WhOUXvfLhhQvTqb7lF4LYYGwFNkH8KbFcuy1Cz6YebmeH+HMCehlPuaseAPH91ojVjjvE4HDl6cFJT1ue1vjT0kr0fsHAQSmIJ43Bn0LBByQSXR93vsbnTLZ0cvBm2irsMhfvYQXp38W6TRZcknyG+EOhJJ1Um2A46JCFURNv0l8f23hn20x9j gM2qp4Hy Sdy4xaNuuI3Xl/o9gIjdIodCzx/cah/9E8wfPl3OWW53MrIfIbhODq26tfSlbi7bsrtKUr7iuxk17U0j8jZpEPrTxOp1Zb92M6t4uqDcOI0x8cyqS3abVRuNKJftPVDrfktPvL00wVNk23eN3ovJiKAJOHt4gF2uaWQN6zV6ZFVRfD6b6dji9kZgxCb65dUu5gzJdVkxL1Wh4A/zk7/D2WRFGtpdxYwi44nue0apn0zM9kjpY9Lr88Lq44lOHhxb2HA7QcYu2wU95NVgJPgjBdIVsd6gg0ho9mZ3mky+QW+MOT9Xi8eVKLP2V8xE7nMz8hSCw8JnaGBb19B1oeApl0UqcqLCF084PlFvs0iyHyLA+nFd93WawHy8nDVEvHOeUHI6LeeLj8oTMsGlT6rz2a9YD7klt8iXaTQeepVk96UB/JH2fCXpuisnFcg8KbhPAcWYoREnF+FqBlH0boWLUR8Nw6r2JlfnreMnEW/1deYTl7RYQ0q1ZEKxFrGqZNe7Jf0ig0K/PV2nYBKt9+pEUmcBgcw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Similarly to the direct DMA, bounce small allocations as they may have originated from a kmalloc() cache not safe for DMA. Unlike the direct DMA, iommu_dma_map_sg() cannot call iommu_dma_map_sg_swiotlb() for all non-coherent devices as this would break some cases where the iova is expected to be contiguous (dmabuf). Instead, scan the scatterlist for any small sizes and only go the swiotlb path if any element of the list needs bouncing (note that iommu_dma_map_page() would still only bounce those buffers which are not DMA-aligned). To avoid scanning the scatterlist on the 'sync' operations, introduce an SG_DMA_SWIOTLB flag set by iommu_dma_map_sg_swiotlb(). The dev_use_swiotlb() function together with the newly added dev_use_sg_swiotlb() now check for both untrusted devices and unaligned kmalloc() buffers (suggested by Robin Murphy). Signed-off-by: Catalin Marinas Reviewed-by: Robin Murphy Cc: Joerg Roedel Cc: Christoph Hellwig --- drivers/iommu/Kconfig | 1 + drivers/iommu/dma-iommu.c | 50 ++++++++++++++++++++++++++++++------- include/linux/scatterlist.h | 41 ++++++++++++++++++++++++++++-- 3 files changed, 81 insertions(+), 11 deletions(-) diff --git a/drivers/iommu/Kconfig b/drivers/iommu/Kconfig index db98c3f86e8c..670eff7a8e11 100644 --- a/drivers/iommu/Kconfig +++ b/drivers/iommu/Kconfig @@ -152,6 +152,7 @@ config IOMMU_DMA select IOMMU_IOVA select IRQ_MSI_IOMMU select NEED_SG_DMA_LENGTH + select NEED_SG_DMA_FLAGS if SWIOTLB # Shared Virtual Addressing config IOMMU_SVA diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c index b8bba4aa196f..6eaac5123839 100644 --- a/drivers/iommu/dma-iommu.c +++ b/drivers/iommu/dma-iommu.c @@ -520,9 +520,38 @@ static bool dev_is_untrusted(struct device *dev) return dev_is_pci(dev) && to_pci_dev(dev)->untrusted; } -static bool dev_use_swiotlb(struct device *dev) +static bool dev_use_swiotlb(struct device *dev, size_t size, + enum dma_data_direction dir) { - return IS_ENABLED(CONFIG_SWIOTLB) && dev_is_untrusted(dev); + return IS_ENABLED(CONFIG_SWIOTLB) && + (dev_is_untrusted(dev) || + dma_kmalloc_needs_bounce(dev, size, dir)); +} + +static bool dev_use_sg_swiotlb(struct device *dev, struct scatterlist *sg, + int nents, enum dma_data_direction dir) +{ + struct scatterlist *s; + int i; + + if (!IS_ENABLED(CONFIG_SWIOTLB)) + return false; + + if (dev_is_untrusted(dev)) + return true; + + /* + * If kmalloc() buffers are not DMA-safe for this device and + * direction, check the individual lengths in the sg list. If any + * element is deemed unsafe, use the swiotlb for bouncing. + */ + if (!dma_kmalloc_safe(dev, dir)) { + for_each_sg(sg, s, nents, i) + if (!dma_kmalloc_size_aligned(s->length)) + return true; + } + + return false; } /** @@ -922,7 +951,7 @@ static void iommu_dma_sync_single_for_cpu(struct device *dev, { phys_addr_t phys; - if (dev_is_dma_coherent(dev) && !dev_use_swiotlb(dev)) + if (dev_is_dma_coherent(dev) && !dev_use_swiotlb(dev, size, dir)) return; phys = iommu_iova_to_phys(iommu_get_dma_domain(dev), dma_handle); @@ -938,7 +967,7 @@ static void iommu_dma_sync_single_for_device(struct device *dev, { phys_addr_t phys; - if (dev_is_dma_coherent(dev) && !dev_use_swiotlb(dev)) + if (dev_is_dma_coherent(dev) && !dev_use_swiotlb(dev, size, dir)) return; phys = iommu_iova_to_phys(iommu_get_dma_domain(dev), dma_handle); @@ -956,7 +985,7 @@ static void iommu_dma_sync_sg_for_cpu(struct device *dev, struct scatterlist *sg; int i; - if (dev_use_swiotlb(dev)) + if (sg_dma_use_swiotlb(sgl)) for_each_sg(sgl, sg, nelems, i) iommu_dma_sync_single_for_cpu(dev, sg_dma_address(sg), sg->length, dir); @@ -972,7 +1001,7 @@ static void iommu_dma_sync_sg_for_device(struct device *dev, struct scatterlist *sg; int i; - if (dev_use_swiotlb(dev)) + if (sg_dma_use_swiotlb(sgl)) for_each_sg(sgl, sg, nelems, i) iommu_dma_sync_single_for_device(dev, sg_dma_address(sg), @@ -998,7 +1027,8 @@ static dma_addr_t iommu_dma_map_page(struct device *dev, struct page *page, * If both the physical buffer start address and size are * page aligned, we don't need to use a bounce page. */ - if (dev_use_swiotlb(dev) && iova_offset(iovad, phys | size)) { + if (dev_use_swiotlb(dev, size, dir) && + iova_offset(iovad, phys | size)) { void *padding_start; size_t padding_size, aligned_size; @@ -1166,6 +1196,8 @@ static int iommu_dma_map_sg_swiotlb(struct device *dev, struct scatterlist *sg, struct scatterlist *s; int i; + sg_dma_mark_swiotlb(sg); + for_each_sg(sg, s, nents, i) { sg_dma_address(s) = iommu_dma_map_page(dev, sg_page(s), s->offset, s->length, dir, attrs); @@ -1210,7 +1242,7 @@ static int iommu_dma_map_sg(struct device *dev, struct scatterlist *sg, goto out; } - if (dev_use_swiotlb(dev)) + if (dev_use_sg_swiotlb(dev, sg, nents, dir)) return iommu_dma_map_sg_swiotlb(dev, sg, nents, dir, attrs); if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC)) @@ -1315,7 +1347,7 @@ static void iommu_dma_unmap_sg(struct device *dev, struct scatterlist *sg, struct scatterlist *tmp; int i; - if (dev_use_swiotlb(dev)) { + if (sg_dma_use_swiotlb(sg)) { iommu_dma_unmap_sg_swiotlb(dev, sg, nents, dir, attrs); return; } diff --git a/include/linux/scatterlist.h b/include/linux/scatterlist.h index 2f06178996ba..69d87e312263 100644 --- a/include/linux/scatterlist.h +++ b/include/linux/scatterlist.h @@ -251,11 +251,13 @@ static inline void sg_unmark_end(struct scatterlist *sg) /* * One 64-bit architectures there is a 4-byte padding in struct scatterlist * (assuming also CONFIG_NEED_SG_DMA_LENGTH is set). Use this padding for DMA - * flags bits to indicate when a specific dma address is a bus address. + * flags bits to indicate when a specific dma address is a bus address or the + * buffer may have been bounced via SWIOTLB. */ #ifdef CONFIG_NEED_SG_DMA_FLAGS -#define SG_DMA_BUS_ADDRESS (1 << 0) +#define SG_DMA_BUS_ADDRESS (1 << 0) +#define SG_DMA_SWIOTLB (1 << 1) /** * sg_dma_is_bus_address - Return whether a given segment was marked @@ -298,6 +300,34 @@ static inline void sg_dma_unmark_bus_address(struct scatterlist *sg) sg->dma_flags &= ~SG_DMA_BUS_ADDRESS; } +/** + * sg_dma_use_swiotlb - Return whether the scatterlist was marked for SWIOTLB + * bouncing + * @sg: SG entry + * + * Description: + * Returns true if the scatterlist was marked for SWIOTLB bouncing. Not all + * elements may have been bounced, so the caller would have to check + * individual SG entries with is_swiotlb_buffer(). + */ +static inline bool sg_dma_use_swiotlb(struct scatterlist *sg) +{ + return sg->dma_flags & SG_DMA_SWIOTLB; +} + +/** + * sg_dma_use_swiotlb - Mark the scatterlist for SWIOTLB bouncing + * @sg: SG entry + * + * Description: + * Marks a a scatterlist for SWIOTLB bounce. Not all SG entries may be + * bounced. + */ +static inline void sg_dma_mark_swiotlb(struct scatterlist *sg) +{ + sg->dma_flags |= SG_DMA_SWIOTLB; +} + #else static inline bool sg_dma_is_bus_address(struct scatterlist *sg) @@ -310,6 +340,13 @@ static inline void sg_dma_mark_bus_address(struct scatterlist *sg) static inline void sg_dma_unmark_bus_address(struct scatterlist *sg) { } +static inline bool sg_dma_use_swiotlb(struct scatterlist *sg) +{ + return false; +} +static inline void sg_dma_mark_swiotlb(struct scatterlist *sg) +{ +} #endif /* CONFIG_NEED_SG_DMA_FLAGS */ From patchwork Wed May 31 15:48:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13262479 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 06121C77B7A for ; Wed, 31 May 2023 15:49:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 971A78E0001; Wed, 31 May 2023 11:49:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8FA328E0007; Wed, 31 May 2023 11:49:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7C26B8E0008; Wed, 31 May 2023 11:49:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 6E03B8E0007 for ; Wed, 31 May 2023 11:49:51 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 2FD01140289 for ; Wed, 31 May 2023 15:49:51 +0000 (UTC) X-FDA: 80850985782.15.B2CFD0C Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf26.hostedemail.com (Postfix) with ESMTP id 7E38714001E for ; Wed, 31 May 2023 15:49:49 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf26.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1685548189; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RYwKwBbFVhKuYYjB0bZAfV9ae8PLZlXqN8QaXAdpqek=; b=rRZm+5QhuFfQup2AaSkuZy63rqw9cX+ZUCTNJ1y5MuKU8zVAmaM3qfZrgNh8lgwHY4wSQ5 P2B5u1/UJC01jWaRhX/e8rVtCo23bX0xZpXssmgC8fzBMphTXsFuKqqBxojeYq8R2786bg TWXaqJvL9eW3itzPun72M4GtiOdEeVA= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf26.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1685548189; a=rsa-sha256; cv=none; b=11qQ6Wsbw0OXqXzcyQQ2pjLAbBF45s1nzgQch092TLGjbkt2xqcP1tPuIRh4ELoAACWYGq uE/NYdnRAS7U7u4vxzhYfVP1kkaBUeaiZxhj0avf0WMD4ZtZmZQgIS41aYPS9+Kkw1FBG5 Kc6sRFNpqfh2WSavx18/UcGgeOBLuzc= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 9568F6380B; Wed, 31 May 2023 15:49:48 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4278DC433AA; Wed, 31 May 2023 15:49:44 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Christoph Hellwig , Robin Murphy Cc: Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Jonathan Cameron , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v6 16/17] mm: slab: Reduce the kmalloc() minimum alignment if DMA bouncing possible Date: Wed, 31 May 2023 16:48:35 +0100 Message-Id: <20230531154836.1366225-17-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230531154836.1366225-1-catalin.marinas@arm.com> References: <20230531154836.1366225-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 7E38714001E X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: 53o8o3y5fbz8exgi8er9biy48q59ciyo X-HE-Tag: 1685548189-5007 X-HE-Meta: U2FsdGVkX1+vOpXgb1GIzdwKFY4xc7oVx2EzwZdPdJxbbXo6QH8c7fG4VzJT++R+bdTIgIHPtqrX18xmRBGBII025S+V2+rQBfQUxt5+4KqRVq+iihJRf6GYE6JYw85dGZzGG3jt43eKwfmDmoEPJzzaRHLUzMkD2m7LqBu+70ZiBpHoqlYjyB3aQIVQnv4fKesk6KiZQIl0KudD1IpFyZKZqeNm0Y9M5M5icnWrZY68RK5+LDscx6hR6uz+Zmh849hmC4/ebbdUqHqq+u284OGkPlCamGMo0FpxxMxuMnhdFfDyfxYGPdOkZ5QXJWort05XJ5dzkJ89J8b0Hw0uh2YMwlvRMDp42+sOzLkijiV6UikKXNQmXYaAEZywfyV6Atw32MNQ0aoKlL1Uu/HRbN8mK/XQd7a4QEulYdGINvgHZWlMr8zQiBIJAW7PL0SROCUXyKmq9Iv/RFSwod4ObSFbotmWmUQcBjarKP6vPNDOZYxgI6J4RITIf+qedxiXrAVjb8yS/bCRgFYwhEteEknmpTc0Af+7TU0A8Mks48/BxLOFGyTHMLPvQiu3TGXDxpck+LRc8ymCv5rip7d9KNfj/PqD0t/ayBulgZ2j87x4cMTjUcEV1JbFjfJ3L6BdfKvaDIEXDgLP1wcxqf71SqnQWhiBYsF2YtrVhOwWcyTyVi5s0DMJayNo5ebCPo8l2qQVV/u31Pd0P/vNPqgIgO2p+RFc2C7p6XGZU9PMI5oO6rMWnLrRj+nupSMMI6pQ7oR3PedL/uEqMKPUXTAutMGxyFWCMoHDy3WrloLdqn9ZUws0idxB9x2/XOPJkzzLu2UdGjzTBvawEAEkYNI5PZELaQeFpfR00iDppNkHYs2416TTf43prbo4ZYkAtu1U6Ef4Rx5HV08n439xLXmLBHGhei/h0eckkiUUeLoBPVG65EbnmYy2U0eI72GzlJIwWOpEAxIoCACTeTSpWs4 +ZiqWDyg LEEVgnymETCAM7VCDSlS40+Ho7m+CdeR7v10u/ZNbP7zg9+yB3GecCglOOMiSKJ/ApZM6Z8NhPV7Qjdgm6+FBwfWDbcVUS/M2/ip3hi1M8DcBNSoXZCFnvuvzs+kPbmSwtm2lzDR4RjmL9EnrUtxrNDqGMRLKcUPBkDao35c61vrmUfB8GKB0lFPuEWCnw8Hr0AJXBbpS38cz7ivmvZPwqOaNkWsP9UrvIrMxJM5MJ0BDHODr1EuK5T2sKKYL7Ac+Itmfo3ArY7ctsgzKyH8m7kFmcccBfqr12+pvKGTm4AK5DsY1Sjj/rD+M9LjB8fS+/Xte5dEHhKa2mJlyypEQopbZxC8u7ZLOLNZqSsY+tuJIRnVMFjbgmNTa/3BYJS1w1PhydhFL0nsooJwtNogW+XSJ+YbjpoNsXoUObWC/cII7+xhyjaLj/epqEm2udKs7oatjJ/jRq8UIT/SOMQYrV0PtVKtRltrJeZBAoqfDMj2rZuYSwfCWY1vrRlfrvxyIgIPgtTKZE3IGhn90VlVVNvlTtA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: If an architecture opted in to DMA bouncing of unaligned kmalloc() buffers (ARCH_WANT_KMALLOC_DMA_BOUNCE), reduce the minimum kmalloc() cache alignment below cache-line size to ARCH_KMALLOC_MINALIGN. Signed-off-by: Catalin Marinas Cc: Andrew Morton Cc: Christoph Hellwig Cc: Robin Murphy Reviewed-by: Vlastimil Babka --- mm/slab_common.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/mm/slab_common.c b/mm/slab_common.c index 7c6475847fdf..fe46459a8b77 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -18,6 +18,7 @@ #include #include #include +#include #include #include #include @@ -863,10 +864,19 @@ void __init setup_kmalloc_cache_index_table(void) } } +#ifdef CONFIG_DMA_BOUNCE_UNALIGNED_KMALLOC +static unsigned int __kmalloc_minalign(void) +{ + if (io_tlb_default_mem.nslabs) + return ARCH_KMALLOC_MINALIGN; + return dma_get_cache_alignment(); +} +#else static unsigned int __kmalloc_minalign(void) { return dma_get_cache_alignment(); } +#endif void __init new_kmalloc_cache(int idx, enum kmalloc_cache_type type, slab_flags_t flags) From patchwork Wed May 31 15:48:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13262480 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0340BC77B73 for ; Wed, 31 May 2023 15:49:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 94C248E0007; Wed, 31 May 2023 11:49:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8FB068E0006; Wed, 31 May 2023 11:49:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7C2B18E0007; Wed, 31 May 2023 11:49:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 6A9EE8E0006 for ; Wed, 31 May 2023 11:49:55 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 2EB2BA023F for ; Wed, 31 May 2023 15:49:55 +0000 (UTC) X-FDA: 80850985950.05.A8C36F1 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf25.hostedemail.com (Postfix) with ESMTP id 77A30A0016 for ; Wed, 31 May 2023 15:49:53 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf25.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1685548193; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QcxrCBYims99sTaWtbsc4IRs+7tDysGBphQN4oXq1fE=; b=6aBG2ZZwYC+t4YCaDdBYplmUF8K5hUOMNLAYxXIGkVyhNAGXVw8NvV6Ps8sQVxK3EyCTsC YCBRY5bXDpJf4Noe8z9xJdiodtdfYeDG8beLPSJNUZQXoh2fOcdVUsIni9ZCcxwftodusF m5oPT9SPLE8FwN8Ktn0RO3lNHeREa+g= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf25.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1685548193; a=rsa-sha256; cv=none; b=Cav062TCxkj1bOMxXjciD5KwcnsGHUb1i6AMPCt3mDBCg/7E8aLpXaehmSOdjPW0svvyW2 v6ZGPOshFeE1y/nx5356GROhHZU1CKfvyp6e96jd7c7PiGglvIj9js+2y7m4EBMVzcxFWA fzMvGESkRx5VHZX2GMNmcCaMVUrXqd8= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id A144F6380B; Wed, 31 May 2023 15:49:52 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 677E6C433A0; Wed, 31 May 2023 15:49:48 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Christoph Hellwig , Robin Murphy Cc: Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Jonathan Cameron , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v6 17/17] arm64: Enable ARCH_WANT_KMALLOC_DMA_BOUNCE for arm64 Date: Wed, 31 May 2023 16:48:36 +0100 Message-Id: <20230531154836.1366225-18-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230531154836.1366225-1-catalin.marinas@arm.com> References: <20230531154836.1366225-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 77A30A0016 X-Stat-Signature: can86yaz8pfgx1gra5wmusfotabhr1px X-HE-Tag: 1685548193-213493 X-HE-Meta: U2FsdGVkX1+XGfc26OptAO8IUk9nJ45RvzwuGthgxxz6YDLWe/vX5bRIUlzCnPQxhZDz+7JdBy+BR3wCEE+n+/53FUXXwt/Jxi5Y1NOBSFNLMRpBbcUm+3MvEM/eG54B25E1id1ttPHOdZeup1sVzwn+XretCoRYGIK7BupKAgPlzmEB267cLXF1gdaIMm7rpjatSqqje9NjB+/vjdgVmrnX3i0RimbT30Ay22mS5RJgBmFfVi7nQj3hd6xK3LnFponu6A5UzEX9urg2Jjj6Sam5SPLiBfs1fzltJ671xQbtEF7f43D2Ow7LRTH+IQt/peFrmAZzmFUAMXd123jOQBAyZAlWQyGcpTK3phLPhS6p7fGp0tAJ3dlZGzUkQdkz3rrMhQcvk2Ac2FflYJFj/xmpLmEGfE0Y+anuZCKGAi1a4wGuv9YWvORiMrU74y3j38eC2S7A+LN8h/uL3IjLojVz7HZOun2UXOLtNv4WZyHD1TbqweZbOwzFVGogOQp9NPffoiZgrhSzADBRnb4L55jEnkYpy1P4fdmbDCB+H9hhNz5oNhc0w/DL1D/6vdE16eYcqCC6EwWfFzLT1HbqZY4dp+gwRG/Twa1dGErtsn7ounzdh86C5jnouxMZ4SkeF3vG3AylIrnC1OkMg1Wk14WvXAfo2FViKK51TKLes2V0nbOPqkQl/Gep9/0lddR8m9XHiUX62EFpCpIgCkhCCgCJXOWYbgqzI064YN5uKH6ZeE2mMzv9v/bAW+cy8gHSMTNpLi0FCcMh/B5DzQLAGfyqRHfyqpZKGLOuKYTHF2vPlDpeZeL9zB4GM/Cl7GUwqkbE8XbDjsP8nEOK+alSEV1Vt5HCED2yw2TOwVZdMsDEB5iBKxPSXuGIrtNoyOLrsRK8Q2AvPGsDhonarun5D7Qts5Ma3AU1hIdnJc5XkaNK2Ss1K6oJuzCR8/VgMdBdf6SESRSCoxo7Rs9//UL uGctM+7i OAXFwWbwb2/b+qb54DE59gqRFwglkK/2oQwYtY8BUndBgfXzdPnzUokdqTox74JCO1ZcfGLJqSz6Graj86qM1z2lDUlYfWh/FhvMRcKaB4kWNg5kgr/1nDktAM9XCrfHhRCPFjTyAMtYJsAFrSZBzypikJ2pvX2O7HWqwrKJ+z1GTdBHmvDNV6hVKrb0wQxUNQLsdTIeNlikCSipaL96s8PB/8ocF4XRBUt2z1k9s5s8gnZIMbBu6Cuetxpm7GdddoBMbo/wKfbfTCVAF+dyY/2MFFKMwW2cMM7LgvRK4Rany2bTAXYbS5RpPnJECypJ6HxzEfgIkiHoxR/HZZvelVHaPz2v0kqgpaly+/nB02YiIoKixlVV6iY6OCSHK3WjfLFvZ27/ESHbB6uc4uRAcFvxPygp6BV5bKB/MpvJCP/8iNAFaUd8HGN38QSnfW0+B1dkBWL9ig4Q8wJRzPmNgrGWtE8AuG8iWkx9LlhQ1qIz/olHuNo64KN1ZjetMbmrCguwxAMmWxL5JqUAg8tnWqZ+HBdt0naAGHeZIqtn8jNLUlsrUsE2OHqzV/crsymFy32Mi X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: With the DMA bouncing of unaligned kmalloc() buffers now in place, enable it for arm64 to allow the kmalloc-{8,16,32,48,96} caches. In addition, always create the swiotlb buffer even when the end of RAM is within the 32-bit physical address range (the swiotlb buffer can still be disabled on the kernel command line). Signed-off-by: Catalin Marinas Cc: Will Deacon --- arch/arm64/Kconfig | 1 + arch/arm64/mm/init.c | 7 ++++++- 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index b1201d25a8a4..af42871431c0 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -120,6 +120,7 @@ config ARM64 select CRC32 select DCACHE_WORD_ACCESS select DYNAMIC_FTRACE if FUNCTION_TRACER + select DMA_BOUNCE_UNALIGNED_KMALLOC select DMA_DIRECT_REMAP select EDAC_SUPPORT select FRAME_POINTER diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c index 66e70ca47680..3ac2e9d79ce4 100644 --- a/arch/arm64/mm/init.c +++ b/arch/arm64/mm/init.c @@ -442,7 +442,12 @@ void __init bootmem_init(void) */ void __init mem_init(void) { - swiotlb_init(max_pfn > PFN_DOWN(arm64_dma_phys_limit), SWIOTLB_VERBOSE); + bool swiotlb = max_pfn > PFN_DOWN(arm64_dma_phys_limit); + + if (IS_ENABLED(CONFIG_DMA_BOUNCE_UNALIGNED_KMALLOC)) + swiotlb = true; + + swiotlb_init(swiotlb, SWIOTLB_VERBOSE); /* this will put all unused low memory onto the freelists */ memblock_free_all();