From patchwork Mon Jun 12 15:31:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13276857 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9ECBDC88CB2 for ; Mon, 12 Jun 2023 15:32:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0F0C48E0005; Mon, 12 Jun 2023 11:32:21 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0A1E18E0002; Mon, 12 Jun 2023 11:32:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EAA3E8E0005; Mon, 12 Jun 2023 11:32:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id DCEB68E0002 for ; Mon, 12 Jun 2023 11:32:20 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id E75331402B3 for ; Mon, 12 Jun 2023 15:32:16 +0000 (UTC) X-FDA: 80894487072.09.AAE315B Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf08.hostedemail.com (Postfix) with ESMTP id 05942160023 for ; Mon, 12 Jun 2023 15:32:13 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=none; spf=pass (imf08.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686583934; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UUvO9n2es88yTASKvPq57Bk5K7ngON6nudF22+6AUl4=; b=tzVhxAiM8R1IOmlLVgsc/cNAe0pVg1+UUgxER8vk1AWNbeW2cGXYKOGDzyU3/2h6s/29Oq dHhGf76V8ji9vyRLvDXweH9CwUu5el296cCPTC7fxYa1UUBLINI4HCRsgXQpzGXXXKwqZl yy0OlxQXv9x1W63cqCugjYrye7FKz1U= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=none; spf=pass (imf08.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686583934; a=rsa-sha256; cv=none; b=OIQfvEg1MEYxZVYSY3kHt5qSOFrh7Fm9riCH7BijDDVrxt8r//weR3bs6rXN9GfIjmHBP7 ihcv1T5yuDoNMc7y4dJ/0OrozLhxDmh6+V6Hbk0uoJEyeNi5wob5q8CDC5kStc+4vNgROx GFfrQ7SUHLmwALO1S2x7eHExNs2OUjE= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 01CC7617E7; Mon, 12 Jun 2023 15:32:13 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 589BBC433A0; Mon, 12 Jun 2023 15:32:08 +0000 (UTC) From: Catalin Marinas To: Andrew Morton , Linus Torvalds Cc: Christoph Hellwig , Robin Murphy , Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Jonathan Cameron , Vlastimil Babka , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v7 01/17] mm/slab: Decouple ARCH_KMALLOC_MINALIGN from ARCH_DMA_MINALIGN Date: Mon, 12 Jun 2023 16:31:45 +0100 Message-Id: <20230612153201.554742-2-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230612153201.554742-1-catalin.marinas@arm.com> References: <20230612153201.554742-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 05942160023 X-Rspam-User: X-Stat-Signature: dtonk9k1b9hxqyu4qyx7dc6qa5j1pyz8 X-Rspamd-Server: rspam01 X-HE-Tag: 1686583933-578805 X-HE-Meta: U2FsdGVkX19GX30t6xvUTUerte+7/qIi/GmFegx4GoBytRBXn8CKamxH3ZRi8kFqaJ/xDYWI8Li219DFiGEaqmO93Bd8tiZltcd0+VUmpcDFGkQ9XtDnv99Trfvf2CunTfFKWnzVtVpvigrLXXISttgbcdhXT/rQsmHsqR26+NhKtv0CTyX4doh0U8DkP1oReMNI+EkkhadIgW0bVOHt5rHYOP7/DiXTdcRLpLm2xB51wO2xM54ZARL8mOUMVshtZfMNe4r6iMA73Bo+T2vTydHwxcVjNYIXk+SgYd68eNnFzYjwY4nDnD7z3Gte6ac7XO6ETU+ikvs8olQuxpZ0Ya4rfIT9OqJ6Nv1R/zivM3XOQYyJ+xDFPF0Kg/adNFoBZJ9YW8yWfYrdqsZ53WpcCFE8J6UV/RvwVO6vi1OSZptBFwgHpbm5JYtlmvEM8QwXNux7GLTmlOUCpepfhLkqneMHV4M6t6v3NVxxOj4mD5w2Gj7C7fAgNOjJ7zDMr/N21LET4d2mv1HzmuTI0GUrxmiO/KL1mz1fkU2oCCh11PjDuG8G7cTyhzFYH3SO+zt5LPTB+ogxArlMXjWs/Uz/0cIZLLNs1qFn9556ydFeEMbN1plYXbhm9tcWiyq9Yc4HBmQkMGJ/qpYTg3iHOPDbLRcyB+50HJoKmEzcNQciaRt3bj9rCSTr/oATDYOMUSFDIMVWozEinSL02LJLs3Zl/uy50AYTP2ZOTEroMpEpZQLYtymWP/rdskXqauMLlAQ39QolOje0XrROapXeOJDA3QzihnCDG2ICgX/kc4AqG+XOLJCjfssMI5C/HGCmhQ4wraZBjC1GeHNJjMfOJaGeKQIiP4/xvUuIQ0d34JpCfrjVdeT26YlZkr4gT+ZKPJ31F9DE7s6c+/kI+jBoichEdF3BkuWCKmeSeSBvNYgDrOhMROs9zU2bt60KzJ0mhOFxdwHTB0JQrNpW+StpzMv oNRklool YfnR9aRsxf3eeYR/0yRtuQi5AeeIlHh+x2CRq7RksGvVtuFr0GR7W8BOlLsLwfi55LNDKbBrZlyOt2ijtAlVnF+JdLg3nVNB4ZRLS3Gx+4A5Zi+0ToBQfaoSwbW1TsiOx0yAG/nsc1uuFCPJIT/AOWAWH/9mQSoiMigAFhG/C4fUjoxTJ4pWhGfcQYbr2XIdvd7JKFKy9ZI8JCIps+4TUX4Euyu2K4tOGMcycpIXojIOH18u187YM1htKAOwBpUS603l5yn94Pt4sR1AKh5Rhup62Ug0nqVcgiMfV+AcJmqJmO12Q/eE6C+0iHZC8Jf+0t5Ji08+KtoHyzyFAu2GReUFk+tabNta29VroI+kW4odhF7/T1+Ex7aQou50p6zcSaummdt1z2hJAHZVLUYbENKb57T3qwEKK9L7VSAyikSeQ5qsRpYtov92nrdTgm+E1c+7y96wzMGcdAKD6805foipAMnimEVTaT2uhgPfCuW2NbsjPbDLi+9N4Ngh79y82jQvXaET1Rjel+xHEjDcq7PEWDqsyw5TTjtTCaOnuCoG6ZIArZYVb4IgsWko3+0cVIIkjhzQYwFlWT24F1Gg+HiTt8Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: In preparation for supporting a kmalloc() minimum alignment smaller than the arch DMA alignment, decouple the two definitions. This requires that either the kmalloc() caches are aligned to a (run-time) cache-line size or the DMA API bounces unaligned kmalloc() allocations. Subsequent patches will implement both options. After this patch, ARCH_DMA_MINALIGN is expected to be used in static alignment annotations and defined by an architecture to be the maximum alignment for all supported configurations/SoCs in a single Image. Architectures opting in to a smaller ARCH_KMALLOC_MINALIGN will need to define its value in the arch headers. Since ARCH_DMA_MINALIGN is now always defined, adjust the #ifdef in dma_get_cache_alignment() so that there is no change for architectures not requiring a minimum DMA alignment. Signed-off-by: Catalin Marinas Cc: Andrew Morton Cc: Vlastimil Babka Cc: Christoph Hellwig Cc: Robin Murphy Tested-by: Isaac J. Manjarres Acked-by: Vlastimil Babka --- include/linux/cache.h | 6 ++++++ include/linux/dma-mapping.h | 3 ++- include/linux/slab.h | 14 ++++++++++---- 3 files changed, 18 insertions(+), 5 deletions(-) diff --git a/include/linux/cache.h b/include/linux/cache.h index 5da1bbd96154..9900d20b76c2 100644 --- a/include/linux/cache.h +++ b/include/linux/cache.h @@ -98,4 +98,10 @@ struct cacheline_padding { #define CACHELINE_PADDING(name) #endif +#ifdef ARCH_DMA_MINALIGN +#define ARCH_HAS_DMA_MINALIGN +#else +#define ARCH_DMA_MINALIGN __alignof__(unsigned long long) +#endif + #endif /* __LINUX_CACHE_H */ diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h index 0ee20b764000..a50375331eac 100644 --- a/include/linux/dma-mapping.h +++ b/include/linux/dma-mapping.h @@ -2,6 +2,7 @@ #ifndef _LINUX_DMA_MAPPING_H #define _LINUX_DMA_MAPPING_H +#include #include #include #include @@ -545,7 +546,7 @@ static inline int dma_set_min_align_mask(struct device *dev, static inline int dma_get_cache_alignment(void) { -#ifdef ARCH_DMA_MINALIGN +#ifdef ARCH_HAS_DMA_MINALIGN return ARCH_DMA_MINALIGN; #endif return 1; diff --git a/include/linux/slab.h b/include/linux/slab.h index 6b3e155b70bf..ca53425e9b32 100644 --- a/include/linux/slab.h +++ b/include/linux/slab.h @@ -12,6 +12,7 @@ #ifndef _LINUX_SLAB_H #define _LINUX_SLAB_H +#include #include #include #include @@ -235,12 +236,17 @@ void kmem_dump_obj(void *object); * alignment larger than the alignment of a 64-bit integer. * Setting ARCH_DMA_MINALIGN in arch headers allows that. */ -#if defined(ARCH_DMA_MINALIGN) && ARCH_DMA_MINALIGN > 8 +#ifdef ARCH_HAS_DMA_MINALIGN +#if ARCH_DMA_MINALIGN > 8 && !defined(ARCH_KMALLOC_MINALIGN) #define ARCH_KMALLOC_MINALIGN ARCH_DMA_MINALIGN -#define KMALLOC_MIN_SIZE ARCH_DMA_MINALIGN -#define KMALLOC_SHIFT_LOW ilog2(ARCH_DMA_MINALIGN) -#else +#endif +#endif + +#ifndef ARCH_KMALLOC_MINALIGN #define ARCH_KMALLOC_MINALIGN __alignof__(unsigned long long) +#elif ARCH_KMALLOC_MINALIGN > 8 +#define KMALLOC_MIN_SIZE ARCH_KMALLOC_MINALIGN +#define KMALLOC_SHIFT_LOW ilog2(KMALLOC_MIN_SIZE) #endif /* From patchwork Mon Jun 12 15:31:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13276856 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 01302C7EE2F for ; Mon, 12 Jun 2023 15:32:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8698E8E0003; Mon, 12 Jun 2023 11:32:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 81A448E0002; Mon, 12 Jun 2023 11:32:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 709B48E0003; Mon, 12 Jun 2023 11:32:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 5F6358E0002 for ; Mon, 12 Jun 2023 11:32:20 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 3C00D1602A8 for ; Mon, 12 Jun 2023 15:32:20 +0000 (UTC) X-FDA: 80894487240.26.5BACBA4 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf18.hostedemail.com (Postfix) with ESMTP id 6204F1C000C for ; Mon, 12 Jun 2023 15:32:17 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=none; spf=pass (imf18.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686583938; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7leEv/np438Rje1x+yfLLaxCOj/l02Ozp19Q4FMwCtg=; b=nOrFHsiNhz7OzYRC12IMEsIpwEbNOwcJaLdnFlxw6MLxqTvVIY+FkLDsT/dfk2xPV0jIZM SoAHIaDO7YWqNxbt5kaRBtUQRxxc0l7sKEEUTmy9jykg/t/gd/27R11d+UCayGDM1lWjvx +XdckaNpJGSGjWydflqM2ZIU3pMIpTE= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686583938; a=rsa-sha256; cv=none; b=Og5kvKw7PYhrupoQt8TdWhUM7cNVYSCwAmY+gJwuwgRBPHlVa8RbTe/w4j6TG128w9XrWF BC/FvJREfDc5woOzVPqEuCwUre5soyr7AqrcEwisE5wawneE3YcaJg+jHmDPqDLLMj2bcJ 8xDhS2g6r7P7k0SOSfzJeVKj0a3eKqk= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=none; spf=pass (imf18.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 52A1462ADD; Mon, 12 Jun 2023 15:32:17 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id CA261C4339C; Mon, 12 Jun 2023 15:32:12 +0000 (UTC) From: Catalin Marinas To: Andrew Morton , Linus Torvalds Cc: Christoph Hellwig , Robin Murphy , Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Jonathan Cameron , Vlastimil Babka , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v7 02/17] dma: Allow dma_get_cache_alignment() to be overridden by the arch code Date: Mon, 12 Jun 2023 16:31:46 +0100 Message-Id: <20230612153201.554742-3-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230612153201.554742-1-catalin.marinas@arm.com> References: <20230612153201.554742-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Stat-Signature: h9imt9k3nwei6k7rjmwohtdxfe37fnsp X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 6204F1C000C X-Rspam-User: X-HE-Tag: 1686583937-579134 X-HE-Meta: U2FsdGVkX19C4Q+wDC4Pgz/ZCCR45MQ3CyX3Z41ljvOyegEBOhtL4aV8te7hZbBzmc94Bw4BIGtpJ7I4ib0AnGgbOARs5QyA+0DR6oPFfoegQl+HqouR5poDF4881YL6r9sAaLvNxM90ST8GkgwJZiI4wXwqLwK7AP+7nG+denWAh1G7gJ9BBF8jT7LHjqjfssDM+X/6QiNkkKZSGUQ5rEZCOqlO1zMe+1egTnNSWL3OcOeYI1SrQOphJzNOyf/8TA4/OPqL85kTvLGRwLHFT74gCM35qPHu/gEH1xFKZMGFZs06TtRglFTYpOu9TAFQVI8TSySn/E9BmVZWCwNQhjhWgwSza2lAaXPdULYGV/cWnz3JWCeA3ZxK0GGd89RvP7C3ns4L5cgDrpC2uqhapS+MiJcf1hjK7YIx1Rqq+RjjOXrKyuDO3z+ME5GxtUTbMgc0lxFZQ/zkhRitR0sWJ1Wa5XOKpiERKgbP9NS261kQp6vk7LqFZ98N98WigE7kVEEnGLR49o2Ag5V32GtncfhplZTKVJIgtWdEz6P9pCSVkYNWlQ5kC9u3n71WSocZ5GaPWzKePcrb3CkNbVYiLYr4pCntrc+tlSjshZE+1I08NVVxfKykjpI/V8IJkLxJMYXOOp91jC00V3HtV6QOkGuhaRDMqayIcGkrJL10lDVJM+S0u/dFbtHf9KiL81Vtea+oC8tLhUt6eTz2v8i3+I+nAZhOoUV6eVXfusha0nXSy2WAoMlaiES7O7kK7bXumUIgKR/KjqdTdAsdWJrZXWj5rml3fprYQdxgUdkluhKEUs7KsaW7YpoREeT2gW+W0dIMz6Sw3+w1OgGPWOSnisMr/uLIyZwSqr9B/j5iOTMVOhjENjGb9VdF+3Ej62BDiJ15qBlTJzebaQygXpubtvjTlaSe/zIFag2fhTSmxxGi8AMQmC5fKUTI7l2/EmqEfLi9m9Iy/7uRkFNcEKQ kEbJoyvv mpQt+uo1fK1kD96wO8a7NsCXwoHXCKda5/hx6hNYzhOdrI1Y5O7woKI85fcp3HEpiFV3e54fdv014S1Fb/IIFGBEXd6UbNfKaM2ywU68i/+rbltLsHUu+ju1B7J4Pv4s6ENKjAf8B0kyrwLdXXKVUON9+Qfz0eEV9U27+Mi0+d8bUjvqT2KL7QwVxJl0Bs20niO9RPDQSYZ3UcolpHfGOS7LL6jJhka2HNpCrihwODaNaHeiey65KLkJsh8EL8XrDseNxPabm8uC+dxPfhtbYOLWcMljUbP2lU3U9+aaAwP4HhL0ZlXOXruKMApx9vJ78pqBVM8MSJcgFW/dfONbo/tcNj+Hv+3WAhwE7zmnYSHEEWlTtFKRd227tGXvIKVYL5TlLbStABEI2bGDXRLPPCj9r9W/iCd1sGLckiMvY+16LAEvAHMW3XVvFrUM/IXGT/eZ2PzLr2z3QvQu+EYXqe+uetcIBnzHX/i+2hlrBxsNkQg2vHeTi59KlfxrJ+4fWMvCp1B1my0FdN8ftoJiKUtCMKEcUHWH2+miboEzDbeInjGp+3P1DnIqdvg47mgnD5b4dYxo8dnikgj8ngFqJA/EKBg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On arm64, ARCH_DMA_MINALIGN is larger than most cache line size configurations deployed. Allow an architecture to override dma_get_cache_alignment() in order to return a run-time probed value (e.g. cache_line_size()). Signed-off-by: Catalin Marinas Reviewed-by: Christoph Hellwig Cc: Robin Murphy Cc: Will Deacon Tested-by: Isaac J. Manjarres --- include/linux/dma-mapping.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h index a50375331eac..e13050eb9777 100644 --- a/include/linux/dma-mapping.h +++ b/include/linux/dma-mapping.h @@ -544,6 +544,7 @@ static inline int dma_set_min_align_mask(struct device *dev, return 0; } +#ifndef dma_get_cache_alignment static inline int dma_get_cache_alignment(void) { #ifdef ARCH_HAS_DMA_MINALIGN @@ -551,6 +552,7 @@ static inline int dma_get_cache_alignment(void) #endif return 1; } +#endif static inline void *dmam_alloc_coherent(struct device *dev, size_t size, dma_addr_t *dma_handle, gfp_t gfp) From patchwork Mon Jun 12 15:31:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13276859 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DE236C88CB6 for ; Mon, 12 Jun 2023 15:32:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 81DEE8E0007; Mon, 12 Jun 2023 11:32:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 77E418E0002; Mon, 12 Jun 2023 11:32:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5F8448E0007; Mon, 12 Jun 2023 11:32:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 523ED8E0002 for ; Mon, 12 Jun 2023 11:32:25 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 2806114026B for ; Mon, 12 Jun 2023 15:32:25 +0000 (UTC) X-FDA: 80894487450.03.FF487A5 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf24.hostedemail.com (Postfix) with ESMTP id 9182418002B for ; Mon, 12 Jun 2023 15:32:22 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=none; spf=pass (imf24.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686583942; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3Jj4EF+XtrFLmw7b5wkN2F+9P7xlNswk5uVqJStOCqw=; b=TpMxknr8NBXLYZEWA45Cupos9zta2Pa8C4hWKt/gsGsuHQ4+WEqU4OTp4POa283bzCr2Wg iINJwjtLaczFVYZn2nP+wpUJCQP5+519PeQoiZo5ZM8fjfg+j40rqDqo0ZzTglkzEU1yDt I7DX6n6TczbGTpfOn8ihPv2BshOvP9Y= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686583942; a=rsa-sha256; cv=none; b=oQa1BZb/KSRbWJ98rZmGdl0ADXbXvBd3EIP9nx54CJDXJGyYBYXdl/t2FEehfCBXsTUWsm ZZIYrrY6cjRUAiLBoY274wy20EpT05FD/hhDt7vNNYzvZwmLclZKOdpuYXFmwWw8WH12oY edulX0VfmU3jApj0PKPZJfYr3AguZVU= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=none; spf=pass (imf24.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id A2A6E617E7; Mon, 12 Jun 2023 15:32:21 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 259A9C433EF; Mon, 12 Jun 2023 15:32:17 +0000 (UTC) From: Catalin Marinas To: Andrew Morton , Linus Torvalds Cc: Christoph Hellwig , Robin Murphy , Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Jonathan Cameron , Vlastimil Babka , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v7 03/17] mm/slab: Simplify create_kmalloc_cache() args and make it static Date: Mon, 12 Jun 2023 16:31:47 +0100 Message-Id: <20230612153201.554742-4-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230612153201.554742-1-catalin.marinas@arm.com> References: <20230612153201.554742-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 9182418002B X-Rspam-User: X-Stat-Signature: jwoia7jfgrpuifb3z7foykyfqzopydwn X-Rspamd-Server: rspam03 X-HE-Tag: 1686583942-466310 X-HE-Meta: U2FsdGVkX1/twk1KHBM2bPf2NxoZS6qJxCs8XxMlevmuuZKFKFcNgyziwBVDOUg/QxTG7IvaHwo5PdqcWCM56/WqLtLVmx3czIjhwnYMh0OATbOmbV7U+IgPct6eGPWqi2fnLmAAGv9amvO/MYajuRPWJq/VzUkF5jOwNz+2qGMVKvja2mUWobzTzel00y7v8pmCU3074BH2ZkGzpWqqBLUEbF3tlEiEUv+b0JHDs1BQ3+D5DdsU2ALSSthlSoStunhhP7gxfejQKgbwgQhx4dbW54mDNOaZifq1qMjIk8NyOqybiDXEuMRQOCrBz79XzuJLQmmUw1qn2bAD4U3rlROSaUrVSKo+VK4uurWsjSNzpqFV0iQFAmdoJPKYleotOeE4HvLia12JTtpQB/SPrlpN60/q0VFeGtSy2wWNjvGWhx7x/cVz5vlEG8jfhjeAfBFZzy3iAuOOG/GkfiZJxuaL8TNFF1Sg594nMoZWTBEfL8udNLLDadyoMEw/yOFeFtynVWRcz2t5Ic/Xgackrj619rE7jQfVZZfHBZ5mhhgdCVIpjQpEHWJc9RquSVa9hPuWCGsTQIi10EYakDaAFZouSBMRRLweUo6bWZcIrDPw4dpowHY2c8ysLxySpqVCVQl8+D2usOpMGkPmNLqz8V5fevAZZ92Ae+7Qp+/Hio961ZSI3Zb+1sienLLcXCQRbYs6q1mp2xGPPXtVv7bKlzFXyxYX/UElXhAk5q6CWjnrXD+JBr7ECIDMCWO/kDfTP207FIr1XGsROmlfRW+cVlcg2a3E5LZhU8ohyqhvyw0gjod9NeUliOkfKro91Xke5izxw1O0XOScVpghs9cBFveGthXLOoLBDeJU2VVFCJzgU5brTbn9lDKLi5JjFK4onSuHuO+N5QUtvcMknIP5cm+e8K9A61dpiY/b901Og84bcCCxOyHMyoXv5cGR8WzaEvBEXfZLACAckHU9QBb +ls7q652 S6XUu5U/eNhGoILYSXPi6FDr/9YMqkqtvgsUc5QleR1kbNoXZre1dXom1TFT6t9vp3UpuvXUPCr6X2opeplclDMJZ9NJ1xC6k0lxnWRYJHs8IQYTbvYfume+krCSlE74eZSkZeXJBjIvCqA7wZb0NZ9JC4Xr7LZNaqrzyjV2iosutMwurYFc0uoCe8tjIBMa4d9KZIM6OE5jIKWzOmMWnLh1dqmrFnR0u/LrOEFiKU+FwzWwOdAZ+xj3Y0hxxpa3yP7Ijra1YHCPl9iLPDzbjRcXurefRmjN/jImDuAX9yd0rtgY7QtPY2Nmslebjj0PqKWMs6AdN9tKcF3K5VvlwSxbbNkmhvCSTY9v4MCM1uy32a3Gzeh3cK23ddPX8JVISgr8hpBVqYTV0dVJPntf1SQDbEmLVwmo9BC6eYXh3fXswC7HzCrkqYYq+n2KBQszOG7VhKnerTPQwgtfWCjN/7bmCLmBUr3juzEzCk9Wj0mk0CmbTrQhR8plmMDPIF8i0WRsovBrdZ9uk2l/3GhAPunAesg7tyJDcXwh+93i5eR1Eyz7/pgLEocnEyc84zjXsvZyWcTr48afYaCoWJZHQjdPVSA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: In the slab variant of kmem_cache_init(), call new_kmalloc_cache() instead of initialising the kmalloc_caches array directly. With this, create_kmalloc_cache() is now only called from new_kmalloc_cache() in the same file, so make it static. In addition, the useroffset argument is always 0 while usersize is the same as size. Remove them. Signed-off-by: Catalin Marinas Reviewed-by: Vlastimil Babka Cc: Andrew Morton Tested-by: Isaac J. Manjarres --- mm/slab.c | 6 +----- mm/slab.h | 5 ++--- mm/slab_common.c | 14 ++++++-------- 3 files changed, 9 insertions(+), 16 deletions(-) diff --git a/mm/slab.c b/mm/slab.c index bb57f7fdbae1..b7817dcba63e 100644 --- a/mm/slab.c +++ b/mm/slab.c @@ -1240,11 +1240,7 @@ void __init kmem_cache_init(void) * Initialize the caches that provide memory for the kmem_cache_node * structures first. Without this, further allocations will bug. */ - kmalloc_caches[KMALLOC_NORMAL][INDEX_NODE] = create_kmalloc_cache( - kmalloc_info[INDEX_NODE].name[KMALLOC_NORMAL], - kmalloc_info[INDEX_NODE].size, - ARCH_KMALLOC_FLAGS, 0, - kmalloc_info[INDEX_NODE].size); + new_kmalloc_cache(INDEX_NODE, KMALLOC_NORMAL, ARCH_KMALLOC_FLAGS); slab_state = PARTIAL_NODE; setup_kmalloc_cache_index_table(); diff --git a/mm/slab.h b/mm/slab.h index f01ac256a8f5..592590fcddae 100644 --- a/mm/slab.h +++ b/mm/slab.h @@ -255,9 +255,8 @@ gfp_t kmalloc_fix_flags(gfp_t flags); /* Functions provided by the slab allocators */ int __kmem_cache_create(struct kmem_cache *, slab_flags_t flags); -struct kmem_cache *create_kmalloc_cache(const char *name, unsigned int size, - slab_flags_t flags, unsigned int useroffset, - unsigned int usersize); +void __init new_kmalloc_cache(int idx, enum kmalloc_cache_type type, + slab_flags_t flags); extern void create_boot_cache(struct kmem_cache *, const char *name, unsigned int size, slab_flags_t flags, unsigned int useroffset, unsigned int usersize); diff --git a/mm/slab_common.c b/mm/slab_common.c index 607249785c07..7f069159aee2 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -658,17 +658,16 @@ void __init create_boot_cache(struct kmem_cache *s, const char *name, s->refcount = -1; /* Exempt from merging for now */ } -struct kmem_cache *__init create_kmalloc_cache(const char *name, - unsigned int size, slab_flags_t flags, - unsigned int useroffset, unsigned int usersize) +static struct kmem_cache *__init create_kmalloc_cache(const char *name, + unsigned int size, + slab_flags_t flags) { struct kmem_cache *s = kmem_cache_zalloc(kmem_cache, GFP_NOWAIT); if (!s) panic("Out of memory when creating slab %s\n", name); - create_boot_cache(s, name, size, flags | SLAB_KMALLOC, useroffset, - usersize); + create_boot_cache(s, name, size, flags | SLAB_KMALLOC, 0, size); list_add(&s->list, &slab_caches); s->refcount = 1; return s; @@ -863,7 +862,7 @@ void __init setup_kmalloc_cache_index_table(void) } } -static void __init +void __init new_kmalloc_cache(int idx, enum kmalloc_cache_type type, slab_flags_t flags) { if ((KMALLOC_RECLAIM != KMALLOC_NORMAL) && (type == KMALLOC_RECLAIM)) { @@ -880,8 +879,7 @@ new_kmalloc_cache(int idx, enum kmalloc_cache_type type, slab_flags_t flags) kmalloc_caches[type][idx] = create_kmalloc_cache( kmalloc_info[idx].name[type], - kmalloc_info[idx].size, flags, 0, - kmalloc_info[idx].size); + kmalloc_info[idx].size, flags); /* * If CONFIG_MEMCG_KMEM is enabled, disable cache merging for From patchwork Mon Jun 12 15:31:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13276860 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BF170C88CB2 for ; Mon, 12 Jun 2023 15:32:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5D22A8E0008; Mon, 12 Jun 2023 11:32:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5826F8E0002; Mon, 12 Jun 2023 11:32:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 44AA98E0008; Mon, 12 Jun 2023 11:32:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 35E478E0002 for ; Mon, 12 Jun 2023 11:32:29 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id D787DA027E for ; Mon, 12 Jun 2023 15:32:28 +0000 (UTC) X-FDA: 80894487576.14.179E041 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf21.hostedemail.com (Postfix) with ESMTP id DBA521C001B for ; Mon, 12 Jun 2023 15:32:26 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=none; spf=pass (imf21.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686583947; a=rsa-sha256; cv=none; b=ytD2jaEk05+luPy1HWzqPdxtwdURLbtmfqhpVEqtpKdTpszaz8lsCGHY2HG8GreZ4yTIhb +U6Cycxk9aterEsgfNsj5iWGFsqBu99I74UKZu571iQJUi4cCg+zjAwVKY+AmuRAQd0vRZ rwJG8ciLQ4FkhtIAca0/+ZKtI3lrJqY= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=none; spf=pass (imf21.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686583947; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=98XE5a7BhLm7ZzEjWpMumSTI8jrHUeYv75m7kBYZXFY=; b=3jyRWphnbel38Lsz7b3ojjR9Bc1Kfdv5uTQkJJXRu0fMvm/nHi3oyXVdzRAQOze2ZU8p+3 pen+BDrhxCRAgpmpnz2BrWcxUfrN3ZNP6NOcXgVxYaxloGWbMiimR5Fg5yppAPqtdnvHnf 1+kg5ZCDg6Ct3JUl79T9N9vCrpx8QwM= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 0333762ADF; Mon, 12 Jun 2023 15:32:26 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 76150C4339B; Mon, 12 Jun 2023 15:32:21 +0000 (UTC) From: Catalin Marinas To: Andrew Morton , Linus Torvalds Cc: Christoph Hellwig , Robin Murphy , Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Jonathan Cameron , Vlastimil Babka , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v7 04/17] mm/slab: Limit kmalloc() minimum alignment to dma_get_cache_alignment() Date: Mon, 12 Jun 2023 16:31:48 +0100 Message-Id: <20230612153201.554742-5-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230612153201.554742-1-catalin.marinas@arm.com> References: <20230612153201.554742-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: DBA521C001B X-Stat-Signature: oqwkj7z78jdamu4zsnqau71ecdsm5fc3 X-HE-Tag: 1686583946-514937 X-HE-Meta: U2FsdGVkX1+GSdJyVsz2H5JIKY4C+30Q+/rZNT1qW5aRoOIr87R2qQtiTcSr26opJ3JK6uyMBRF0F8546LQ/pq84U45QKAlv3ewwMUP+/kzLCombYIQ4ZizqMZKia5ugXOyGi1fTws/jGDAaQUA0L9k7bP7/p9dcJbGN0meyFJlwSGC8iQChLGrCUuQUfvKPbtrSdqkMR8FoKB5VdN6xJuBEjMVrDV7k+JoKMRljKX/NEgTtqvYpwifx/jPeK0by8Hqna7XjPn5zEtnGlJ0eEAGIepa8KnUfWw+dTT2g8JGmJyZhjD6AiKjr9Q3IqR2ifR1Rjnv1RmvqwfLyFRXznhbia1+XXqzlf60EBcZsFuIaBl+8ZNSkMj7VvhlBape3IbJc/wy6NBJizMBRqpUxfRFgBRsFys3bpfE5nmwKsWGgZxrekNEMFwxoOi185VhDSHsQN731jxRb1rmKC3zh9Zu81If5DuiQ3aLWsPe+6nCPfTTdyqEsM5D+ca2QTf5149HN9ww7c9LXZQeO4Y0F41tEmsfKLoqj6G+VCIujGJkaKpL4Ama/c8+VVlBIQ/JESyb3tZa9vobksr6IVxR6u/I++Lt4Bcexr/WyHj1jyGkeNkY0L7XZpDFN31UuxGvkqwnPSZHI8k4kCtbj5g6Bm92soebGWV95eaCAOFIygttGNZLbeOix2INFkJCVqtY7bkF8w9pGN4gk9o+f4nsN+nivm2YQirXnRsAngc/oLCiyZpSkmLTnQVH4iEUZFyf7mbwTLrX+uDLxSnkhq1cvC2wK0qzvwzgR5YamZM6fOLk3ygmEX/I2cFX260c0ffM/Z7DA//mQrr2cLCKinGTGUw46fcEOeYdxVyzmfbu3856fFx7PKn9qpQFHfHttt6QS5JTGTtVjC6Uyh3mVz8PNSisseFNjU0mDV5Mz78DZnH1d0GAcgrzO5OD/jzOzpNzOfWgTz/syp73uRPGktSn xDpw1m6s 8qDT/gU1lSO7d2ZXhnchqA0xnp2OPwUFAYVmOIXjRKHdesfTYg5G2T2ftghoQQy2x1ZI3OQO9XfZLCW6NTAjesMwOGBQSrU7HneEpYucrBDy0955BoOKLX8i3KnwLcUmdsO6y+kHse4cFi49fDU4IbAqmGbgVDuseHbpSWJBYYmny4jMVl+Lg3Ea5JN2AtOn3Lflaq8tRj2IaHPJPGx7Ifg9SNl9O1IF9l4tblNRoNIZ/qPQ7nNbNdxR4MjGd36iM5EdcvzMXbVGt9+5fXX/ytVhUvRL1CwhT80EQBZpjEJHOKcYMiqMWZcQWwTfUh6p+0w7GdMYsfWNVbI/I8E/QQkizUJ8H67LW0d6H90LK6+0xpCQ7rrS7l+L77DIJgCgfTsUuea6cpwyJ7htb+F8iPJWyJ24BaWBvJF+RG2/wg5/UmkRLMSG63A9UXroGnZidKLmqgDLgCUwqB8VwpDwqIfkelnCtD8A+iWEa7Oce2MMSkBbr1oCZsae8iH2e4ZGiW27zJqnOPaIsXSaShLy4T7haOUf7aihr54xDu8yp9k/YSCbXZoN4D7EPI3utG4vmd+kZwUuzuAwbs392c71UWikH3A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Do not create kmalloc() caches which are not aligned to dma_get_cache_alignment(). There is no functional change since for current architectures defining ARCH_DMA_MINALIGN, ARCH_KMALLOC_MINALIGN equals ARCH_DMA_MINALIGN (and dma_get_cache_alignment()). On architectures without a specific ARCH_DMA_MINALIGN, dma_get_cache_alignment() is 1, so no change to the kmalloc() caches. Signed-off-by: Catalin Marinas Reviewed-by: Vlastimil Babka Cc: Andrew Morton Cc: Christoph Hellwig Cc: Robin Murphy Tested-by: Isaac J. Manjarres --- mm/slab_common.c | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/mm/slab_common.c b/mm/slab_common.c index 7f069159aee2..7c6475847fdf 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -17,6 +17,7 @@ #include #include #include +#include #include #include #include @@ -862,9 +863,18 @@ void __init setup_kmalloc_cache_index_table(void) } } +static unsigned int __kmalloc_minalign(void) +{ + return dma_get_cache_alignment(); +} + void __init new_kmalloc_cache(int idx, enum kmalloc_cache_type type, slab_flags_t flags) { + unsigned int minalign = __kmalloc_minalign(); + unsigned int aligned_size = kmalloc_info[idx].size; + int aligned_idx = idx; + if ((KMALLOC_RECLAIM != KMALLOC_NORMAL) && (type == KMALLOC_RECLAIM)) { flags |= SLAB_RECLAIM_ACCOUNT; } else if (IS_ENABLED(CONFIG_MEMCG_KMEM) && (type == KMALLOC_CGROUP)) { @@ -877,9 +887,17 @@ new_kmalloc_cache(int idx, enum kmalloc_cache_type type, slab_flags_t flags) flags |= SLAB_CACHE_DMA; } - kmalloc_caches[type][idx] = create_kmalloc_cache( - kmalloc_info[idx].name[type], - kmalloc_info[idx].size, flags); + if (minalign > ARCH_KMALLOC_MINALIGN) { + aligned_size = ALIGN(aligned_size, minalign); + aligned_idx = __kmalloc_index(aligned_size, false); + } + + if (!kmalloc_caches[type][aligned_idx]) + kmalloc_caches[type][aligned_idx] = create_kmalloc_cache( + kmalloc_info[aligned_idx].name[type], + aligned_size, flags); + if (idx != aligned_idx) + kmalloc_caches[type][idx] = kmalloc_caches[type][aligned_idx]; /* * If CONFIG_MEMCG_KMEM is enabled, disable cache merging for From patchwork Mon Jun 12 15:31:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13276861 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9DE28C88CB6 for ; Mon, 12 Jun 2023 15:32:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1653B8E0009; Mon, 12 Jun 2023 11:32:34 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 116C68E0002; Mon, 12 Jun 2023 11:32:34 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F1F8C8E0009; Mon, 12 Jun 2023 11:32:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id D656E8E0002 for ; Mon, 12 Jun 2023 11:32:33 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 649081A0260 for ; Mon, 12 Jun 2023 15:32:33 +0000 (UTC) X-FDA: 80894487786.28.E384438 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf29.hostedemail.com (Postfix) with ESMTP id 68D1E12001F for ; Mon, 12 Jun 2023 15:32:31 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf29.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686583951; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pIB3SvnuE4oH3PVP9l8vEoEBsTlH+YYK7Kp4bwnkxMs=; b=TA25ya45tYZp0grWwxQh3Jjk0uupE+x4cOnTNhIqgXjW0PQcR21jJ6QE/Z2SDaBCRuU1du 40BYhyb2JNdXqh4r7xl1zzhrn/2rfgMSiNtRgE8nTo6L/W1GD3UgjS2tpSBLHWbU1D+Uqk hPSZhe6WxcxkI67MsLravqAjxIH+3W8= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf29.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686583951; a=rsa-sha256; cv=none; b=zi6QvKZzuQa4FmZh6mLKWEbhhKhgxozx1+Qau6Z9ZpB1m0XilpwXE3RM3zPmDtiJtIV38t JJsB2/RML5OZLZq1UlJidn1iuKeQeEcS1Q1HnSk3jkLSvyU82KqT3hg6iteqFcb15xcD/g c2dnUzbOvaBJ54I9pQYM1q3Y7F1Yci8= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 7551B61FB6; Mon, 12 Jun 2023 15:32:30 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id CAAB8C433D2; Mon, 12 Jun 2023 15:32:25 +0000 (UTC) From: Catalin Marinas To: Andrew Morton , Linus Torvalds Cc: Christoph Hellwig , Robin Murphy , Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Jonathan Cameron , Vlastimil Babka , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v7 05/17] drivers/base: Use ARCH_DMA_MINALIGN instead of ARCH_KMALLOC_MINALIGN Date: Mon, 12 Jun 2023 16:31:49 +0100 Message-Id: <20230612153201.554742-6-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230612153201.554742-1-catalin.marinas@arm.com> References: <20230612153201.554742-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 68D1E12001F X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: gsfoe4xtowh5snakz5883b393mfwdieh X-HE-Tag: 1686583951-734198 X-HE-Meta: U2FsdGVkX1/uv4sVh+Pskjd/VJ4k4L20EFuQ88feKS9QRaXqjL+ej1NoHS6aZtW0BX6fM/l65pn+TWh7T/wLEwEwsQAqxfIlYz/aCbfAPJU04FOEt0UJOgdkLMdfatBrhThfv1k/dJ14Z/5SUEGeVwYJDxmZ/qQ6rQ54TIr3BoMJdr4mNkhxV9FSyEZfXPhK+ScDqlMLy2TXXcta1LI2tvyF0drcSuvEFX6LAbI5rrFaEkovH3KtZwm6XzXDSLXjS0p8VJA31OQLdJ5PXD8PWxhjPyDS23u82ZLTqV6dFiqkGIblGNn2pZ7xwu/6iuOa0BjhtfLw49MM/z8YeY8Z6ApmVMWl1grH5mNxkVkMhM6nGfrJ+9GFydcwBL4UZPau9+kk3QJG6XcTQEkv5hUKcJzI9UabFZ/Ncl6AOBxla3Df/Vc461d3GWn//h8zKO98cll+RXO3p4FxfjGz6CLYa4fWdtpSPiDz4q9fQHI7/kEf7UDLmpfDZuJ2TFBfEknxva5TgBjFYP/QuoCxoJnIwpyLIiQaaQHw0QdLV+T+8xbCyTisw/WC4FGtr5Yai7namtBusaH8yQI36fZmHRiJXUzrD4/8/Jg6Em4K/cayujdkpK/KUwYbxaNUlWWOvrNV+ps/Kn0MMSJeD8+OAu5OoNwff/2FUVbEn4dFx1Gqx8u24RkkQk2bLygic+u7tbG5d/qnM3dXNy2+W1E70tOKZ5FMJyBI1Wxzz80Fwxc29LbhlXfma9fOXSB5hJw+QR48ok739XB2KztkkpbCMhs1u5C9HJE+Xm1sGg583dfE0Sy9pVhSjxFzC3m2umAhvsx8/IKaE0hlxyY5BsFSVtRgkQWkew0PB0x5xFPUabIR/9Ff1MUs6H340EmaBSRjUbcq3ivLFjlzNLcoEpHBvmZd2cs9fV2J3wjUCmsSnyRqyFZDXsKQumhtryoFmL5I+ntpz2mSVsV/qDqW1cEoP9f p1ClPuYp UgLX4iR9bntbpKKUyGMF4HXvhToGUPvZaBq/LEthl2g7jz5iDvV8cJH05ol0i/VXwzdfJbAcoc46ZVcCmzqXdL1z9kOuPqok2BehVKJTLeym76e8MfL45baWeg2ygLTLFE/8pYCaiigKisphWBOrRs/Vcbx/9HeQcSYBdsrmfmzLIn8ifHdjF01my7YGSwgcj7Uw3mGJuwege2MuX53MjYiM109TYuX3yqLHKcYNolbMEG362XzNmZxqFdenDH9xsynmtoyXc8ibPjj+jpDM/Rv1qSTBuYg/sVvcEYFoZKY/9yuM28n6neQv4shHMQ7oIlr92lGBozPz4Te0FICFx0loFHeeYDacVIoOxvTtKaunwKBNIJmtp+yT4WosImB85K4hwN+zR1DlyDmdAWXVux18WWWGIfMluy3ioUXmMqANTbXXRw1cnbxzhpSMoN7s5vI04QXLXi+AjnujI7Blu8sD08lVtnkTr8xw7EjX3+9IlJiHxRLVBPG9uqunMoyax90lrK2CIZEulSjm+qRKW65uTt1w9jC4TO6g4K9UtQjEAc8ZtRlbx2j6c3euzAD2KXwB5FVbZOcZPYtK4zkMuGAclpOtGkWKeTFb5JJuv1h6oQOL1+i2tjPckTO3pg037hdxwEZNT/h+RHaGPMXcQxHnJZg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: ARCH_DMA_MINALIGN represents the minimum (static) alignment for safe DMA operations while ARCH_KMALLOC_MINALIGN is the minimum kmalloc() objects alignment. Signed-off-by: Catalin Marinas Acked-by: Greg Kroah-Hartman Cc: "Rafael J. Wysocki" Tested-by: Isaac J. Manjarres --- drivers/base/devres.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/base/devres.c b/drivers/base/devres.c index 5c998cfac335..3df0025d12aa 100644 --- a/drivers/base/devres.c +++ b/drivers/base/devres.c @@ -29,10 +29,10 @@ struct devres { * Some archs want to perform DMA into kmalloc caches * and need a guaranteed alignment larger than * the alignment of a 64-bit integer. - * Thus we use ARCH_KMALLOC_MINALIGN here and get exactly the same - * buffer alignment as if it was allocated by plain kmalloc(). + * Thus we use ARCH_DMA_MINALIGN for data[] which will force the same + * alignment for struct devres when allocated by kmalloc(). */ - u8 __aligned(ARCH_KMALLOC_MINALIGN) data[]; + u8 __aligned(ARCH_DMA_MINALIGN) data[]; }; struct devres_group { From patchwork Mon Jun 12 15:31:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13276862 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 788F2C7EE2F for ; Mon, 12 Jun 2023 15:32:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 13E028E000A; Mon, 12 Jun 2023 11:32:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0EFE68E0002; Mon, 12 Jun 2023 11:32:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EF8788E000A; Mon, 12 Jun 2023 11:32:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id E1BB48E0002 for ; Mon, 12 Jun 2023 11:32:37 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id A7B23C028F for ; Mon, 12 Jun 2023 15:32:37 +0000 (UTC) X-FDA: 80894487954.11.81B97B5 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf15.hostedemail.com (Postfix) with ESMTP id ED071A0017 for ; Mon, 12 Jun 2023 15:32:35 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf15.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686583956; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bcILxfP7Z0mS5m2l+jEBqFRdY0jwbrbm2JIuHvtSng4=; b=Inv8aoEomArAnVXHYqzfU0euGrCVPflzV4ZNLlH/m/MqAQbeg1jnS3Q2vVhKFzyblDQzni Mxi9JHSHN9NYAszAYfOR5EY6thY23NBPyj1MdJ7ZXDvcpC3BLoEPrUATtRqt7MQsOW0e3e R42Msgx89yD2VP6fzMCAkGPyfbqhmYU= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf15.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686583956; a=rsa-sha256; cv=none; b=Ny9TzSI0NW25dVvbfMUGRJ4RyqwtN2f+HssONWPKBRi9A5LROPsUOIqlB8fDTpO72NmzbC lch1WpT5CSryk6dm+Ia9Qrv1QI0TIKOVhmGYBOUsqAstO1ej8v5SYHDEjPSpNfNd+OvHVW JjuibARc47l2jKgFG3a2m4yEFb8o3oE= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id D6B39618D8; Mon, 12 Jun 2023 15:32:34 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 49096C433A4; Mon, 12 Jun 2023 15:32:30 +0000 (UTC) From: Catalin Marinas To: Andrew Morton , Linus Torvalds Cc: Christoph Hellwig , Robin Murphy , Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Jonathan Cameron , Vlastimil Babka , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v7 06/17] drivers/gpu: Use ARCH_DMA_MINALIGN instead of ARCH_KMALLOC_MINALIGN Date: Mon, 12 Jun 2023 16:31:50 +0100 Message-Id: <20230612153201.554742-7-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230612153201.554742-1-catalin.marinas@arm.com> References: <20230612153201.554742-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: ED071A0017 X-Stat-Signature: ma6957t9hd3wo17qq3kh6y6zmnfy5eom X-Rspam-User: X-HE-Tag: 1686583955-993408 X-HE-Meta: U2FsdGVkX1/jHhxhdI3RE0PuEhDE9DWQTuWe5MZDQCb4qBEMepFTplXyo2EPTnTrU/v6/d0QpUpHyi/VjrI8nkCVacL0PAYNxzzfCdEDqMq7dgNAENaQINAZzUvhQTP7s4M8p2AV3aJWD+WvY98q2PsVlac9v+WiOAM4KU+hmUWv06gdtp4Qiqm0b8WLom/Jh8DDB3gVOBeXxjpEG2m1ASKaIq5oqRDWpVK1bTSFXi1boqkjjRV/r6iPB+GSDg24XBZoqt68YpPv1X26Eg+196i5Xk/AENPOpfzxhjpRfwY5OOMBu71hvGO2EK1rIZHnWgWhM4LVObR+Ph6Is6yQipn6NhwgYQm8t2aXIrcDqwIMKvdOzBlp1KyVUEHJk2j3YrkuA1kxT1MROZe7fUKSUMpQ/0lv49BKdgGPxviSqKMKk7Xphye9JByU5ovfrNqeOR/yGRklfZ0xS+YBdWcnig3sr/Y26MwVFDZOcRmem+coCPMzctLw0xaDu8/8R5VJ8bkdWwCcyK5IllHRcjTURqSSuLnDbPGz88tYKVUvCj2AqSgMyR1bKvrZWF/J+f+k+nsaoOFLPnH21Mdu7RorKuAJ4mrO9YICUIdvrgwtUccNol9ZiL6ZAsYKo7pBkr90L8qlwfKd3wjCrlPH+iCiqyQlnXC/SBAhJoBbDJi1sLMg6p/x21AFfTA5N4IyuUqwtWU5M1VX9oQ0FKIj3XB9fh7QBdGhcEtBK9WIlK7ov2Tmt6PnzO09xY97Aoc/cyszbwmbv3wg3PYmPQFKed9/wLgj6xAl3XWNvumkjoPA+zXr+GgSyTn5Y80IntK25Ke5VDgQwRe6KTqUdeQxjBAm909TBKr5rlpfZ3EI3rOg37/T2X149TQafIIZ6eIpM8K9A6EWxooJqieC4g03zpx4jGol2pVi54U2x1mdli+0256bi48oOJT+duSAGT8fkuWVXZCz7ffJgz81Fzukq+o kcNNsTOM hdkjQB/hYfOnd2YNGC3bHrgEeUEaejZDyV14lCCZ0KsTcNXvZYwR3nmBIuBTM7Yyk7eoMpcS87YrupwAmM+lp1if2uy0K/aqWCTHQQE992xraRtAFlYLVkn6HxcURS6mC7DwdeCUd0Ymb9iDXz98ftZvc+HmTJTP7FTMdbTygb2i+TVng36G5t5rhaugCWAPVJ7bSMimNaRiojpTkmJSLMNRsFep/jkpaoYywQcOy0BsMjd18OFVAEeTETTlXv6iXsxW2XUQBTDrlzzlkwBdLqcA/wmVlChpUwzgqDav/awEIMy6R7BrTk7d59sabZd9GeSwhh1F5iXNYDThlA36693rBFvvDs1DwRd6pMifkmdnJTZhYaskAYL5ktH2stVIXgZYlNERIrmsoUaL03BiJcl6osjgNogsOWML80QJyw1HougfSDxePLsEn1TavxtKC6VMQHP+IsMwzz2+lzUPJjAM6UlSUEc1HoxjPuB49cblqw68JXlFoYC6jLoEje2FYQ08yd0b/HiObB2g4GaGyZuQaXA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: ARCH_DMA_MINALIGN represents the minimum (static) alignment for safe DMA operations while ARCH_KMALLOC_MINALIGN is the minimum kmalloc() objects alignment. Signed-off-by: Catalin Marinas Cc: Daniel Vetter Tested-by: Isaac J. Manjarres --- drivers/gpu/drm/drm_managed.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/gpu/drm/drm_managed.c b/drivers/gpu/drm/drm_managed.c index 4cf214de50c4..3a5802f60e65 100644 --- a/drivers/gpu/drm/drm_managed.c +++ b/drivers/gpu/drm/drm_managed.c @@ -49,10 +49,10 @@ struct drmres { * Some archs want to perform DMA into kmalloc caches * and need a guaranteed alignment larger than * the alignment of a 64-bit integer. - * Thus we use ARCH_KMALLOC_MINALIGN here and get exactly the same - * buffer alignment as if it was allocated by plain kmalloc(). + * Thus we use ARCH_DMA_MINALIGN for data[] which will force the same + * alignment for struct drmres when allocated by kmalloc(). */ - u8 __aligned(ARCH_KMALLOC_MINALIGN) data[]; + u8 __aligned(ARCH_DMA_MINALIGN) data[]; }; static void free_dr(struct drmres *dr) From patchwork Mon Jun 12 15:31:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13276863 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1C71EC88CB4 for ; Mon, 12 Jun 2023 15:32:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B29D4900002; Mon, 12 Jun 2023 11:32:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id ADA338E0002; Mon, 12 Jun 2023 11:32:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9C8E8900002; Mon, 12 Jun 2023 11:32:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 884028E0002 for ; Mon, 12 Jun 2023 11:32:42 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 681381202BA for ; Mon, 12 Jun 2023 15:32:42 +0000 (UTC) X-FDA: 80894488164.17.B50F2B7 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf17.hostedemail.com (Postfix) with ESMTP id 6824D4002A for ; Mon, 12 Jun 2023 15:32:39 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=none; spf=pass (imf17.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686583960; a=rsa-sha256; cv=none; b=U3lGToSjxeHMuFZvNJ5QSXzbybytpTbqKWtNFQOe5qdrQl9/x+VM/6pqVe5NnrQUG0galk 9KNM/Su+D3EovkvRoQzx+7eNZLHFOW79e2xnxWjzFiUPzFMlY2t5btlymuFl1vW7V7XviF 5+4js4eNWZCHQ3YwGYnpHsNZh3qYj7g= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=none; spf=pass (imf17.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686583960; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vZStyUc7LlYOT8NMiiLXCqHwidPc7KVMpmBSzGU4hdI=; b=WgZqm8UBcjDmgqLWU8WqXCTLOxHGwzQcrgDWpt0zw5BxKRfPtbTJBj/GRS05ENRwkDNAZW O4vb36ajjNJmBV8sSsT4EhnvuuN+zsES4sP48akMQWLl5QaS3e/iYN3gIojDy02udcq3bv r+fPF+77sEK9YYcA4cb6VyPkK++P1go= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 3028862AE3; Mon, 12 Jun 2023 15:32:39 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A227AC433AE; Mon, 12 Jun 2023 15:32:34 +0000 (UTC) From: Catalin Marinas To: Andrew Morton , Linus Torvalds Cc: Christoph Hellwig , Robin Murphy , Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Jonathan Cameron , Vlastimil Babka , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v7 07/17] drivers/usb: Use ARCH_DMA_MINALIGN instead of ARCH_KMALLOC_MINALIGN Date: Mon, 12 Jun 2023 16:31:51 +0100 Message-Id: <20230612153201.554742-8-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230612153201.554742-1-catalin.marinas@arm.com> References: <20230612153201.554742-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 6824D4002A X-Stat-Signature: zfk7tidhwt81rjsptwhm86dg7qdd4fws X-HE-Tag: 1686583959-206865 X-HE-Meta: U2FsdGVkX196fGvWY87i24k0aK7L1zQLeSBE+yRSQaZS/X0Hhv2C6447SlB3rEC4iFGVZ30PDO80vPSveDjgnihAsHwv4YCxXR2euhmNY09gKpR79ivayfIyalZKVktgl4nv5EB6gn6HCAKYbhB8ePD5EFhJHnEw6KLcOTDlLzSbfBAnpmQ8OTwU4MYsg0UXV+cccMlDYyN05AczHeTmZevJ75Pvuuk75Xd8Wyq2iz4a53URfNCkylmlPe/gdbb/DIanzBUTNZ9YlvVx+pez8Rz8uy39qmW0kIqEL/ZzxAxxj/83un/NTNs6+6prjYZbZ5JJ747Zo6vUVLx7t2L/KGxWG9Q0thfHo2AF9TBX+vRToxgGNVdH5XP+Rjor3AXmSLKurLnZJJ1euXFgIXBbVg78wo079vSwFbwqfGyWPfXIUHDUIfkZvjZ+8daSVsoZmo9KqWJd+IBLmuwKCmxUwXAUs5p9WO94K6UmPzBSMFuhOjCAt19p09qYoNZ9Cl7luMuRGCE7czn+feyKzzIg2QXdXIV4YM71JlknS4evPK7+IYQy9RZXLf0zZPPSYIkaFDhDDfUllU6geN/Rrd4HGVfn1DKg4Mgs6oTYk2Up8dcZhZrUVMwKbFZ6S55R761F0fjfJPle8AKJ5jYLerVoLmB2PFLCQv3V9ueLdvYrxpQv26PbRGWm8lefN3RSptwlJa6CkSOxGFAjQVAXuKmc7r58EUZsNBpQRGQm1dd9VF/HWAsCo5/L1rdJFjPZA89HN85oZm9koRBjeDF4+Gnfl8Zb63F6rANQKj3S9+ZtxZWb75IsOA+e+EEZixfRm3rYwmkiJhIc5M6EjkiJQzcbyjVr7sEBp6EsRhtIvlazHr6kAo7f8uMHF69cbGty9ciB+1irFwdq4WrJfVobfsmK5EzcSERfJaIW+jiOJtjl4sZRjgfilRZijCfgl/1jJnSvl4Ov/OLIRIPCH8c8WRB G1CunB0G oN2zisLnTMFpyaj1X79+ozkmvaKzGsi+AD4nejPOeRz1yEoCTrLa2dX1hmwpucHYSYeQEDPWX1vV51gUNHReVzMp0aY+7zxeyQDJ0rSnN2nmVEqkxKvv5jtubeRv3HSgQBiP0HRuNyS4OS2EzY4JF85l2wc9U0336GZa6zqFUAt536qJs4hdspIWf5N8dzgwFkOBrQ2iseF0X02db1rtrxQ+RwpY7hlhkGTjvn8M2teF5Y+RJDzxoXSYXStUDWs/reQ+qRcTxtAVpUGbwmLP/1509gGUBt4wNw/OX+WAmvIYokwhD53sY2YD9RZfSeKjZjp0SGPV3tvyHF/JiRcLx/3dbbPaLpOxX71Y/2I57njw5o1WkulRmqv5mUqCM+cc89wlG0U1qAJweIK+4bR64cjxlts+K+KCcak3IN+Ds33bMk4qxYbQ53cv5suqvgkmW3FJ8JUyHuqEKn3Dwlh3H+7O8N6GW7uWLX+l9UljHcGFXpP3LoCgGvz2nQDSi7wgkIt9cC+n7aZWH9PK1M1lJx5QFVmZgkebrQbJm7BUt0GRlFO9Coc8dXNrX8rrE1GzR5xzmJOPH0QzjYkLRoXDI3a9Btg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: ARCH_DMA_MINALIGN represents the minimum (static) alignment for safe DMA operations while ARCH_KMALLOC_MINALIGN is the minimum kmalloc() objects alignment. Signed-off-by: Catalin Marinas Acked-by: Greg Kroah-Hartman Tested-by: Isaac J. Manjarres --- drivers/usb/core/buffer.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/drivers/usb/core/buffer.c b/drivers/usb/core/buffer.c index fbb087b728dc..e21d8d106977 100644 --- a/drivers/usb/core/buffer.c +++ b/drivers/usb/core/buffer.c @@ -34,13 +34,13 @@ void __init usb_init_pool_max(void) { /* * The pool_max values must never be smaller than - * ARCH_KMALLOC_MINALIGN. + * ARCH_DMA_MINALIGN. */ - if (ARCH_KMALLOC_MINALIGN <= 32) + if (ARCH_DMA_MINALIGN <= 32) ; /* Original value is okay */ - else if (ARCH_KMALLOC_MINALIGN <= 64) + else if (ARCH_DMA_MINALIGN <= 64) pool_max[0] = 64; - else if (ARCH_KMALLOC_MINALIGN <= 128) + else if (ARCH_DMA_MINALIGN <= 128) pool_max[0] = 0; /* Don't use this pool */ else BUILD_BUG(); /* We don't allow this */ From patchwork Mon Jun 12 15:31:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13276864 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 44DC9C88CB4 for ; Mon, 12 Jun 2023 15:32:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DE95F8E0007; Mon, 12 Jun 2023 11:32:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D98578E0002; Mon, 12 Jun 2023 11:32:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C878A8E0007; Mon, 12 Jun 2023 11:32:46 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id BADEB8E0002 for ; Mon, 12 Jun 2023 11:32:46 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 93D751402A4 for ; Mon, 12 Jun 2023 15:32:46 +0000 (UTC) X-FDA: 80894488332.30.FB138F0 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf30.hostedemail.com (Postfix) with ESMTP id 6DC588000A for ; Mon, 12 Jun 2023 15:32:44 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf30.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686583964; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=je9aVxGjxQ3eIhH8J5HxA/vH3cf85JkONJ2g3lGrGh8=; b=qpiqZEAybZXFWTrIH/WTraaTwY98pNx2AjXvKeuvnZiINnGV0tW5vFjGF7OyCeGb6i+FhO ltXNyBCGG7fj8pxu46VzL32HpyRLWSyT15RnugE9WgJ2rYqAONnXtrruDtmkPQasrhOvWq S2s1b9sby8x41Qmz1L7O06jzhwd3oUk= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf30.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686583964; a=rsa-sha256; cv=none; b=2ErWMQ9QC9dhMX//KdfonM9+W0rITDMQGzXYqJ58+s+bkPl92vrq++07dqU5A8CFjgw0jb bQTaENx2RJMu/aNGPW/NfKytiEk+wQ65px41xIUccFykiGkF2RvLRlryErO1vpNYR3KAY3 m2RHzomaSWpKk6kYcaffFdKysEEwT90= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 7EBDB62AE6; Mon, 12 Jun 2023 15:32:43 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id F3091C433D2; Mon, 12 Jun 2023 15:32:38 +0000 (UTC) From: Catalin Marinas To: Andrew Morton , Linus Torvalds Cc: Christoph Hellwig , Robin Murphy , Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Jonathan Cameron , Vlastimil Babka , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v7 08/17] drivers/spi: Use ARCH_DMA_MINALIGN instead of ARCH_KMALLOC_MINALIGN Date: Mon, 12 Jun 2023 16:31:52 +0100 Message-Id: <20230612153201.554742-9-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230612153201.554742-1-catalin.marinas@arm.com> References: <20230612153201.554742-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 6DC588000A X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: ca5p4cmdgwaq9e677634ntnpedmmfed7 X-HE-Tag: 1686583964-475057 X-HE-Meta: U2FsdGVkX1+1iBgO71r+YGgk9/R90GVdHwDm55GSdvED3N+oipYE30Oq7oFeSKLCtnb4Zdm7mpi60zMllsaDH5eLl4/15XNF627AoEezXGbvr1fRxYSEdLG+V9Kny8AgdlqWMzfGhE5TTQyQtoLka3GVNhrX4P8MgkE0vvBzQ+ssFjenvrHC0Je04IzhntHNEUpNrWufZ570O1L/Y2a+lnHb2b/P6JEd0AIzSC5fINYyxKNaDlK5TuNmbMHbKOKH/lj4/B77wsT9oRDbHNKpvomqLEFHkjvZYmXfJxrTc0s9lh9mREx97ZV0HnD8xjR5JfhpOlY6bJ7jBT3/FVNn1p6Lp514LTQTcEFYtJwJDN9+dluVzrGGANLlXZXcGcdcIhyeGX9xEQOkyI5OkMfTAYO6aKNNtEgtA+0+DMnl4JCMXsxa9SjXKccATKYh0hXrIVxvoGPuNKvrZ0qkju6XRBGiQcvUJBDyyDFwZHoiZEodmcKizmXKP1CxjH4c194s3Ana6/a6GIskY6Z6poA5WCNSGu2e7MduBMSEfZzJcCr1KeUksqCf1/aijhAuEPTXnckupqcgZyHdHOXSQXc6JyThvJTcEHVIL9tpfnKEo7ZQjWTjjmzRsvdhHkkAZOgE/7E41uSzqGHqBh45t81W70YOhMi1slbCYOv+EYMl9KZqruWL/MabhNiMc/gPCux9RfyhrKJQhXkzOtaeqYLGxzoDVibaLw7+5LryDavat4uemexThjVYpk6wIVGJwwMlCuk1tAa2FsFme99k61iG+yimVP2uO2D6c1MVb5jUyU3KL8qGHsxMZbgb/ukC3iKNjVNqs5Zu2iSxmsYVP79UbfJOJtqCiKDgZFIISBQ8umrPWtWSbD36LNHj9GNjSkIvrpCdSM/3Q7o0RVxB85YYFHPaGbVTCDK8ZYNx0Lh3OPilTo+nr247Ehk1uB95wS1a++ofj2db1GNVfyOSp69 39rkZHU1 +LaA8tuB0qrb/D2iIChfu/c6mLDl8AO6Y49DnRh8bhGjuxFmLPdCc+aK8eZDjv9jyITzhpelSdmHoeQkz/3fWq2L5uc5mxxSj3sEFx3eoYd7vWR8Cself6MPc92thVjCRKfyTeJWEM1k6qSR4rsdG6zPHZSbKYUOh8+TgbQ6LMR7zmWrSUTLeQfsvElkeC+MuHWdQ8drJRdcIv2YSS4wbvzSQHqBm2w+OtWc+MHpAOXnIQIcYBut2aj/CZTOnyJkwDkPGald+hWLw71ZMPlusZOsdCJbnFcncg/NLyELlglB3mfs2mrR1AWf5f/+Ea0XWT8cp4tCuMGY0R8M/3oy1k/zw8J50oYa87FFBo7SpJl9JiZUt8TMRoj+cjlbsIVqw34xZWaz+ObKu/dWAMUln5gLqI/wC1f+0KvJB5XMs3c1T2msulKMiPadZph16UcfxXfZY5FNA/knkpG9BRVUlrV4iMxIJZD8PEXJ6CvKuLLCjRUjviLFdS4vV9qAfDD4D2sazclnLXE9UtDQCUgCTDZ1NYI1FMuWX+/5lA7jKGlrPFteWySXqQD4WCfiAznYPi092NPSoU2m9UuTQd9H4STjVow== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: ARCH_DMA_MINALIGN represents the minimum (static) alignment for safe DMA operations while ARCH_KMALLOC_MINALIGN is the minimum kmalloc() objects alignment. Signed-off-by: Catalin Marinas Acked-by: Mark Brown Tested-by: Isaac J. Manjarres --- drivers/spi/spidev.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/spi/spidev.c b/drivers/spi/spidev.c index 39d94c850839..8d009275a59d 100644 --- a/drivers/spi/spidev.c +++ b/drivers/spi/spidev.c @@ -237,7 +237,7 @@ static int spidev_message(struct spidev_data *spidev, /* Ensure that also following allocations from rx_buf/tx_buf will meet * DMA alignment requirements. */ - unsigned int len_aligned = ALIGN(u_tmp->len, ARCH_KMALLOC_MINALIGN); + unsigned int len_aligned = ALIGN(u_tmp->len, ARCH_DMA_MINALIGN); k_tmp->len = u_tmp->len; From patchwork Mon Jun 12 15:31:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13276865 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3EB3C88CB4 for ; Mon, 12 Jun 2023 15:32:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5361C8E000B; Mon, 12 Jun 2023 11:32:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4E5BB8E0002; Mon, 12 Jun 2023 11:32:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3871E8E000B; Mon, 12 Jun 2023 11:32:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 2B2678E0002 for ; Mon, 12 Jun 2023 11:32:51 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id F31A51C794E for ; Mon, 12 Jun 2023 15:32:50 +0000 (UTC) X-FDA: 80894488500.13.75AE97D Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf15.hostedemail.com (Postfix) with ESMTP id C91CEA002C for ; Mon, 12 Jun 2023 15:32:48 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=none; spf=pass (imf15.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686583968; a=rsa-sha256; cv=none; b=zNtIJsLpxQayGno+TaHbZ7vAmZTX2f2R1fbF43CIJv6xIkhuwKE+3qMLpmaKbRyQ1rjtLT czVjq23pMMvG8r0UXiFlE0GDFNGwKvXHDx5TV2CMDVjaQrG2wRLagTibnfBkfnjzz9OPpr Toz2l5RAb8VsA6AC/joSGzv9DpRBbFk= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=none; spf=pass (imf15.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686583968; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mmajKpEVlHnP6XYxFnPEi7/dHxMZM/L92V9DBRbuy/o=; b=Iw3fKgt5DGG01+QwOCN4XZ7iqNb3QWLjgVggMjo5m7at7I9wl1MMBpnfyiz0u0evrxu3d5 RUY0aXAgral76oIKxmeqOaku7fq0xrOtrZLeKLJw8SCPBGoFx+b1wmVryVLwgAu9Pdl0CR M2gkgqkh8iS8k+kBM/AIJtMcVwMgkxw= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id DD86F629B3; Mon, 12 Jun 2023 15:32:47 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5183DC433A0; Mon, 12 Jun 2023 15:32:43 +0000 (UTC) From: Catalin Marinas To: Andrew Morton , Linus Torvalds Cc: Christoph Hellwig , Robin Murphy , Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Jonathan Cameron , Vlastimil Babka , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v7 09/17] dm-crypt: Use ARCH_DMA_MINALIGN instead of ARCH_KMALLOC_MINALIGN Date: Mon, 12 Jun 2023 16:31:53 +0100 Message-Id: <20230612153201.554742-10-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230612153201.554742-1-catalin.marinas@arm.com> References: <20230612153201.554742-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: C91CEA002C X-Stat-Signature: wgkfh7r1cj9gthpntmh961z7gg4h1wiw X-HE-Tag: 1686583968-295161 X-HE-Meta: U2FsdGVkX19fwm7ZVoxIibgFD+mS72NHQorQvBvRXhuooy7ZA6qFNV76YaIWhvM3eHyrLr86GTA3QxQkMTmmPl5Uy3fCoNZuWLcNsqdfGPJ592L0W5rzAlUbZ5E8Q8pQBgrXOJtnedWHrKiawN0/+SgvoGl06G18J/82aFqdYKEmZxgVLtYugJX/5OC4T5jbb1GKeR+vZvfGKDs3er9/4UEf+Qa/g9pMjYQpi8s9wkUMLu+xOnUp6r+RGebcyI1oZi4CyD2jz4mt0L51X2zlRhVuRStsIdlII4N83TLveMVwK54szmyLk5U45EIkK/mA1pY0tVX1g93AlEBXewMT90rItAMA1r4MDmiY60wfJ+fQ20/A6EOdpXDcparBNq9jch5E/lXiSQ56h3wOotrrGniF33KYPBZUC+kiy/NzEKUfGSjXH7fzSAizzYL7dXD8sfw48HUuqldMG+/1CaAhPwpYWa32fw2fgOX37U3z5DVnn/gSTOeyL9dMQTHy0whxQ150oLVh6ozEJuxuHfkt6ykCC8J9j0SjpAVD8OH7jnARQvU+FBMpqZ1PelZVavk/DiyALAWj7f/392ExwJMoWke5xuhU/+2xjUYhS5dlZ3CWXt1FIhYaOigfMG67uXzL7VZJAJHQDQ5eJuJGb86gUtk0VnCRGh5aF8JRVGXnPk9My7d1qyVm1G7erUh1hdZ3gUkAL2fjRCdaGcxKbRtuZCrAgT59ZXG7vqB3RCddlQy9SrmPkG+Kfi3Hb1JnGeQPIpLOLGvB6Lghh2QMfl/lslxk2HgIQ+E/VAgHC6r1ZHuKfbEgXux+YvM6wcfWEyYRj0hKYXQDFox8QqMLHVa+pFOKG01hO4jYsPBj2VF+RfUaMp90eViOzgce10ao6lD5Dx+BfxzrDUAEHCihnHI4XgJJgbmnfWe2zxpWXRCH5chKBkyN4CgDoZP7w4aL/b7K+dqNejxAKyOhlYYoVBN tp8xUnEJ rRpv3QXU6e+zVe7pBZ6Qoc5TtVrnpoGSQOfvMjwwTQagk5VaMhu3snGjOuAfBWr6af92pY2ViR3Ko5G8ROEAq19P6kjie9vryjEfjuR2Kyty5ReHEr5ws0mOu5y7axCMKqRsYcCG0uYe46x8F5S19jcHmBo9kVytiu7QE9xIAYDC1k0JK7RY0HMfs2iSN8e4/sAyRHRQpLux6RkHw8qWcOyHuDwaW8tz1lleBBAiiNfmBrNmMFOpHJXGlz6N7e/49ykL6zqWkkijmPKiPq3mvFCD6kXNisflGtO6LMuq1R6ggK6TZTPRCFAzpeKR6HAtoo+2axEf4/ZlRqWyEu4UTypRxe7FtggS+TyVhIqewD7kY+FeUafoOCO+Yy67MezCHkdiarcjurQTvKbMcbMgN6wwL5jGfgv6K4oF99HG5VGbmGUjPkyd6V7gCflvGrN6CR1XI5E4ZBkFWSSYwoeSzBmjYeUM3L8kjD0XiTpXJ6X2zrnQUNbVFKN68EuLXwESjLCTzGdhK0dkfqhHlZx7M7nlHWvLYtzRU2IFxfeR5EJbiiKxNwMWncUowlCs7hiknc2Wa0hBUNq19mQ4+j4hK1yYkx9Y9CrnTKLEXslwMBp3S2/g= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: ARCH_DMA_MINALIGN represents the minimum (static) alignment for safe DMA operations while ARCH_KMALLOC_MINALIGN is the minimum kmalloc() objects alignment. Signed-off-by: Catalin Marinas Cc: Alasdair Kergon Cc: Mike Snitzer Tested-by: Isaac J. Manjarres --- drivers/md/dm-crypt.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/md/dm-crypt.c b/drivers/md/dm-crypt.c index 8b47b913ee83..ebbd8f7db880 100644 --- a/drivers/md/dm-crypt.c +++ b/drivers/md/dm-crypt.c @@ -3256,7 +3256,7 @@ static int crypt_ctr(struct dm_target *ti, unsigned int argc, char **argv) cc->per_bio_data_size = ti->per_io_data_size = ALIGN(sizeof(struct dm_crypt_io) + cc->dmreq_start + additional_req_size, - ARCH_KMALLOC_MINALIGN); + ARCH_DMA_MINALIGN); ret = mempool_init(&cc->page_pool, BIO_MAX_VECS, crypt_page_alloc, crypt_page_free, cc); if (ret) { From patchwork Mon Jun 12 15:31:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13276866 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A57C9C7EE2F for ; Mon, 12 Jun 2023 15:32:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 361EE940007; Mon, 12 Jun 2023 11:32:57 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 311A78E0002; Mon, 12 Jun 2023 11:32:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1B3CB940007; Mon, 12 Jun 2023 11:32:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 0C2B78E0002 for ; Mon, 12 Jun 2023 11:32:57 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id BEA77A029F for ; Mon, 12 Jun 2023 15:32:56 +0000 (UTC) X-FDA: 80894488752.02.BA53C1F Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf05.hostedemail.com (Postfix) with ESMTP id 7772810000D for ; Mon, 12 Jun 2023 15:32:53 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf05.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686583973; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bcQNhX6sCx2xBmy5xR9JctGZhUOcAA0mG4HoiKwBkGs=; b=q5rLIhHvAQdc2DpQOrMiF7L7+eEg8Zd7r/VGx9sdQe4xAIl/QI81PNsVRAMpt8rijjW4cR kS47m/hoMvpjbdw1c6PKv3upBx0aG7pJHIrkGcSuIBwAlZ26t0XjDSCsOiS4v81TcpC4Aa lX5Q25h/CNUagaZ5zC0QsN2GPCYpPAU= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf05.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686583973; a=rsa-sha256; cv=none; b=7/pSDTLotyr7/ol/D52XBz7Sut5Zk0wo+jbIfpBEJWqq7o8dgt5bJ7iK8sBy1wn+QJlGY0 2327FGTbAqVrUGrP4/swQJuoYzv46l8JjoN1GOqOShAvFhZTrXOJb0R8uoUMiDW7xnQ9qb 71iu+tZ0g5RPh8xEtsUqyNmyBoermWE= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 9030C62AC4; Mon, 12 Jun 2023 15:32:52 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A922AC433AA; Mon, 12 Jun 2023 15:32:47 +0000 (UTC) From: Catalin Marinas To: Andrew Morton , Linus Torvalds Cc: Christoph Hellwig , Robin Murphy , Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Jonathan Cameron , Vlastimil Babka , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org, Jonathan Cameron , Lars-Peter Clausen Subject: [PATCH v7 10/17] iio: core: Use ARCH_DMA_MINALIGN instead of ARCH_KMALLOC_MINALIGN Date: Mon, 12 Jun 2023 16:31:54 +0100 Message-Id: <20230612153201.554742-11-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230612153201.554742-1-catalin.marinas@arm.com> References: <20230612153201.554742-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 7772810000D X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: hx9zfic47j5ju18gz1xkt9ne4gjc43g7 X-HE-Tag: 1686583973-769246 X-HE-Meta: U2FsdGVkX1+X9WjYu+sG7ZzJJ1Owm2GLW0JyM8wv8AcA5kz9S96tNu7E/+z0lDJoIMMYVXPOfnH/YQYOGWfk+q7Cp+I4hsrmLB9yzYRUGpR3pRKNyvWrc3WlPQJbs4AqRJxmHOalULXhNhJ1hZk2/3wTsfjFdKpXrwk1AHmGlwwU2gusWmmfCb+xmjTEM/Vr2zEcesBnClBH6WaL9e4TsQIPE/b8h9us71WFtvmfKNIoHh2ODtbfLU5wYSqaNDulGrBBb7TbM5XAxl7uVnrx/e8+cobIGVTqS0t0Csuoh2LRLnpAl8SHq+lkGe/VPhS4LQW715mIvZ0tGmyRoOD837zP/aB5D5jktNDcXKo7gcJp8V8rFA2VOoAsBlpDc59bXk9I8ItuuukRsha91yE+lY5HyAmr/S9J881DqxpQMNAYG+Rh0uF76VW1Df+UjbVxPlGSOEThnP9ZpvC31AdG+niPLfBZY/3EmpzJz7AUEbdcd2yq8skXOIZMx60U6fShO+dBbBVUdCQmaxV64GIQr/c7ogrpkTrXgk7+dlRHjz6ykrKXz+zavRbGN2vbU6lCLV6ZlELuil4Y5PUtboJSzLF9tY0a8gicSLlj9ne3prnhYgmUW9kNukT2teTvgMuBPuQ338F9UKUfIhS0g+Zpl1xjbW5g815PlZNFGTC8kQPx3rnKHIRPN3heCGCmBS1F7AG3Enxb/4i0MX6SYRSZdifCFnNqqAgI40nEepuoAQR+6RrCSDf/czBpOSOlvoyg4r93NN3xU98yjXDXpI4ArN7U4gnnxhrq+IRiLKFKg5+XkX4t/UCWN5kgxmoAWzIOWgHcjU2nVVoN4pW4DeKe1MBldscYETs6nbM6eSq/B5oD4RtdT89qvTjZVMKHlMrN45biR5oS2OPG1Za/12ocwWAl/xgPL3qQrsom/yoaNShK1z+vGcJ4PAzcCtWBHK5cgf52mVsBWglUIhEn7pm ZIcsyBrr iLl3En57NIcVOMYRcRpBWVNZIz7GDzqep1yW7UeIvHlbb0qSabcZXVyIOagtlGhD1T0yI0NsH/rKivIiUcNxUvyWibCuGNGpOfQsnLECWeiwD4+NBK67dqQmjNiOF0Vy9DRyVpdm+E++Kgm2wRzS3eiJzzpxVU1NUx+cz0GW4v33LVNKIidvHiAuuT7F28l0o8ZnpWpyd6VnLJhyCpTSx0YoWgkvpNROLbmWklMNiXc5Pm+Z5/aGpFIUPt53A35qYP/2Uqy20k5DZmJX0xKGgzQ6ezIBNcSgXW5EgYF0auyyAJ1s9UdgwM8zE4ZZatxbHdnnP2/I8t5/DTj0yAZ2m3ENU+Sagg5R0iuYq8emrj+6JFtTranNSHhlkGHMjbsfMdn56M9/weYky1XBCBqZkIrC/fP6XfkHqFjwpj+AglDPV1OIRhtecgVDEoxm26VStXmXDmjYjc1LHjCpRQzmOnw2+hFQfE3Us1vCovIFxiMumkoxMKCgRNMaeMZw1kk5mu9fxxOX9nuqrKc1YLhTaXF8AtTQydi4M1SsppLQsAVj7AjeGjWFXrfZE/g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: ARCH_DMA_MINALIGN represents the minimum (static) alignment for safe DMA operations while ARCH_KMALLOC_MINALIGN is the minimum kmalloc() objects alignment. Signed-off-by: Catalin Marinas Acked-by: Jonathan Cameron Cc: Lars-Peter Clausen Tested-by: Isaac J. Manjarres --- include/linux/iio/iio.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/linux/iio/iio.h b/include/linux/iio/iio.h index 81413cd3a3e7..d28a5e8097e4 100644 --- a/include/linux/iio/iio.h +++ b/include/linux/iio/iio.h @@ -722,7 +722,7 @@ static inline void *iio_device_get_drvdata(const struct iio_dev *indio_dev) * must not share cachelines with the rest of the structure, thus making * them safe for use with non-coherent DMA. */ -#define IIO_DMA_MINALIGN ARCH_KMALLOC_MINALIGN +#define IIO_DMA_MINALIGN ARCH_DMA_MINALIGN struct iio_dev *iio_device_alloc(struct device *parent, int sizeof_priv); /* The information at the returned address is guaranteed to be cacheline aligned */ From patchwork Mon Jun 12 15:31:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13276867 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7D454C88CB2 for ; Mon, 12 Jun 2023 15:33:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1668B8E0002; Mon, 12 Jun 2023 11:33:00 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0EF14940008; Mon, 12 Jun 2023 11:33:00 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EF8638E000C; Mon, 12 Jun 2023 11:32:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id DE3E08E0002 for ; Mon, 12 Jun 2023 11:32:59 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id AE46DA02A8 for ; Mon, 12 Jun 2023 15:32:59 +0000 (UTC) X-FDA: 80894488878.30.4CF50B4 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf09.hostedemail.com (Postfix) with ESMTP id D96D214001D for ; Mon, 12 Jun 2023 15:32:57 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf09.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686583977; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XZxYG1ZyKNCbRegZE+KJoS+5roUkNS5nMBDKAO377Q4=; b=i/kOLcD5+6asDPYaRh5uhE8JckW2VWSCJtzErqiBOjEs63E3VMP6hp4UZrIm3B5S1GFvcI W9u7SJ1e2VuUCB0+rj5z6EoFA25ckjJFFk3YQAyCj7mbTgKM17QArGcIY8s3Nf6c4D9Dga AJtcQ+q4zerI9QEK+YKxPRdhvdo/3Bk= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf09.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686583977; a=rsa-sha256; cv=none; b=mf6NGChCoYYmc9DezVKoWe2A9sjKdNRQrdR4p6cx8Rwnp8mpfVZFo2nPWmOk19PEGgLINl MQGfaQKJGdTaiETZRd0LBTo5JWGQGHMZbD30xunrqwfRo5jWcvt4PKLep0oggKnbp3esDr 4RerxHE7cpvunN7g+SB3grTLImk9NqE= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id E31C762AD9; Mon, 12 Jun 2023 15:32:56 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 65A00C433AE; Mon, 12 Jun 2023 15:32:52 +0000 (UTC) From: Catalin Marinas To: Andrew Morton , Linus Torvalds Cc: Christoph Hellwig , Robin Murphy , Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Jonathan Cameron , Vlastimil Babka , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v7 11/17] arm64: Allow kmalloc() caches aligned to the smaller cache_line_size() Date: Mon, 12 Jun 2023 16:31:55 +0100 Message-Id: <20230612153201.554742-12-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230612153201.554742-1-catalin.marinas@arm.com> References: <20230612153201.554742-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: D96D214001D X-Stat-Signature: cto64oq5f3rwnpa7mgyaxsrj7wub9gjn X-HE-Tag: 1686583977-728785 X-HE-Meta: U2FsdGVkX19qDKl9P/OViEOt1vB2Oew2EVBaD9uCmBIEUUT9eQf7DEUgZeFqUjBLIy8wOUTO9NbmmuxyVkRSIS9hRCu8bvvK6ECsSCGSe+pcCfhyDpCK53VcB0vG1IMRs5O1ErITy7TfQDfwJEmE9uQyw4cH9odkfCiTHqihHMYVpFRkLi7hOH9Gw4+2t2PtB9zoxWeEb7DcorlFf+GCqeS84XGViPspjSo7pVP1AA9epSpt1j0TGCh6Wp7/3ON5C3RcAKIQGXajeWq/THFk109Bg8E5KRBXzSK1FJi3BAJzLKmKky+TTakcVV+UpCW1iZcx3cDNV0clb/JgVuVauzVIfsCc2Z81yiZGdosyJY5Mn03nR5rDGe1ngG3mn+g/NNiu67HsJ+3iGDC1imKWswfIHi54788J0XCTdZKzLROXW+I49mHoPV+r0DKC4Zoc/nzgCmz3sh4Il/8822DJFjdMtotbzR2N6qz3vvDVdiVOu4OvwRNDckzghTD1NGio5Jev7JgqUfp/oWASRVRXG9ihd83Vvj0fIhDCKRjtmcbUN53xB1cQ9/Z++X+pg8KMl/JuF77iwThRFMKS4TPdCNOJBhr+L/Acaq4IoBL9Ft0t6URzNUDbxBgFYEOtqoC18ELp8eKqfuPxXGxpdIUfMhuIhYBI8WrHT8r67IJwOO7/GKztSiv7YkG+vFYApGojllWqurSi5geuJTSzX6RjgbWiWkOiIbFNUlyrdEzHHZcjU9Q/z/RtIHEYzfcC52izxoSlCiLaDWdJdj417Tu06SvZhfacJlTR1pX9SqcJ8iL+9qvkVIDicGWM52DCPKusE4xYuZfLZveB132d9bO6PRY0qj7p9FPsI6QsgAT3na7wfYyiUNNfmFfQNU93Z/7D2YG0JQ2llJGFs14rUBy+hO3TEZzAY+2s1w5PEmo5kP0UJZUrjfTzgPxQG+60eObwcyazfjO9adgTk8NUmLe kSa060Fe D/Z65Q9+TJUaNUIJ67sPV2wabW0c+gztJ3Xd489vquudzR8DEOMuAaQzx9AYrm+WQeGK5BMC2kdCY0nLcnMrHWBuzO8zEBP+8PwRUdZedegsAgNt+g/hScFToD1vz/HZTAiVhP68w+RlpZ7MShhMpN8MmVjHiCoJpOp2xapb9iJeUX0GK6Wo6rLFU27lMjKL6LxsCuIz93V8jYQy2gB2IDUOFyTH1rGoU9sz2pY7c/vI9sPF5Bs1Xmf53q/KSJYWAdxepgUPiHEHQDnel0FHF8bjg1P4cdYc0iD+iyC61Oq00crj2yRkH2E75EKox8FwG8sudPaO2mqOQVOqAonBeN8946jEPNTomODwzE95h9WGK9Bwv65Xwaqg6AUKYcqzZNjTv/zh9cEbQbH7Z9NnqBeRlSrKyX8sAZodY1B+WjVa/US0D2dx9uvPQU/oM4SBZa8cYRwvENb/xIitNte95BLuCdDqxAFBusnHolYIA54lc9iq+qFbkLZq4Q5AZvj4Wqz2c0ziDm4zDPb6oHrkm1AajiwDEpqddSWDqbyu3bwS+qKSlkfarVUUHwcynotg3E1ae+sST3ckas7DPY7L51++9aQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On arm64, ARCH_DMA_MINALIGN is 128, larger than the cache line size on most of the current platforms (typically 64). Define ARCH_KMALLOC_MINALIGN to 8 (the default for architectures without their own ARCH_DMA_MINALIGN) and override dma_get_cache_alignment() to return cache_line_size(), probed at run-time. The kmalloc() caches will be limited to the cache line size. This will allow the additional kmalloc-{64,192} caches on most arm64 platforms. Signed-off-by: Catalin Marinas Cc: Will Deacon Tested-by: Isaac J. Manjarres --- arch/arm64/include/asm/cache.h | 3 +++ 1 file changed, 3 insertions(+) diff --git a/arch/arm64/include/asm/cache.h b/arch/arm64/include/asm/cache.h index a51e6e8f3171..ceb368d33bf4 100644 --- a/arch/arm64/include/asm/cache.h +++ b/arch/arm64/include/asm/cache.h @@ -33,6 +33,7 @@ * the CPU. */ #define ARCH_DMA_MINALIGN (128) +#define ARCH_KMALLOC_MINALIGN (8) #ifndef __ASSEMBLY__ @@ -90,6 +91,8 @@ static inline int cache_line_size_of_cpu(void) int cache_line_size(void); +#define dma_get_cache_alignment cache_line_size + /* * Read the effective value of CTR_EL0. * From patchwork Mon Jun 12 15:31:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13276868 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E5ECAC7EE2F for ; Mon, 12 Jun 2023 15:33:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7F5498E0008; Mon, 12 Jun 2023 11:33:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7A57A8E0005; Mon, 12 Jun 2023 11:33:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 694718E0008; Mon, 12 Jun 2023 11:33:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 59AA98E0005 for ; Mon, 12 Jun 2023 11:33:04 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 2D81A1A0256 for ; Mon, 12 Jun 2023 15:33:04 +0000 (UTC) X-FDA: 80894489088.07.BDF2C12 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf20.hostedemail.com (Postfix) with ESMTP id 3CC3D1C0027 for ; Mon, 12 Jun 2023 15:33:01 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=none; spf=pass (imf20.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686583982; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=n8jxFfpCZRM4nPaOR8hcZPkWObYK7GNY+z1x0KsyXks=; b=UxdTOLELhUhcQe/bRpGbRZYSis/pRcg4LNjKFV7/2664Iq7NEVNtJaGqCcSDaA+HgdPSHq OP6BdMFR0FLYZc0uzC75MBdZ5qIOv2Wf6fh3z6NIN6DMm3SfpslJp1foxH85Hq0xZ61Eoc 7JL767rbLz1F9EJFeUeUISbeX9c3s5Y= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686583982; a=rsa-sha256; cv=none; b=WhLaxx+4Un/REAFhLOc+UjmFxCgf8NyhKiFp3NNSoFol4tnl3shH+EBzVJmwFakMkHOIPE 2OnhD44opPdbOx4q0/mPyE4bkLAXSjAzCI6pQOvzeTKKEUeSPwYtyp+YGV8YotUSxRTy0/ wueHZylYaugowZDxN59h0j7/18jDQqE= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=none; spf=pass (imf20.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 606DB62AE9; Mon, 12 Jun 2023 15:33:01 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id BA072C4339C; Mon, 12 Jun 2023 15:32:56 +0000 (UTC) From: Catalin Marinas To: Andrew Morton , Linus Torvalds Cc: Christoph Hellwig , Robin Murphy , Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Jonathan Cameron , Vlastimil Babka , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v7 12/17] scatterlist: Add dedicated config for DMA flags Date: Mon, 12 Jun 2023 16:31:56 +0100 Message-Id: <20230612153201.554742-13-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230612153201.554742-1-catalin.marinas@arm.com> References: <20230612153201.554742-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 3CC3D1C0027 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: 5xrfrtoib3n98c58jttkrwi8g58dur9e X-HE-Tag: 1686583981-269518 X-HE-Meta: U2FsdGVkX1++61vQ9fUm1rZIqWkOXnuaOZFq0KGLZMXjcsapVPk7wVE46PAiY6Itxv5+kvIdhMUqwiApEqy9y88cYOYEh79vwXT8+DJwwSGTkHBNHSlMWEOvj5NIM/sFrqH6ft05f+Nj0nqYCG/dEWrEsCqT8zbobYi7Gj0GuMzWgfNGSzO4MMO/mlUG5NyftObgx2DNPOf4pasC88lVdWNEy/jzzD/Jda/x551w5H/f32rFVzcoST5BjpF+4QUOuiTU5vwUuWBLvemkXlBlfP2AWIR8OfppJ2DfqOCqewCubKJC+Ho63A1r89jF0NlWPfAQ8E/4gg6UI7vulbF4FmBzhPe+5pnp+YViTyu8Cz4tGJeLNzsfx/kkive0a5QjuMS9wdz+CKDXrrwezfFEp57vjJxIbfTOBkMlfU3jP6cLoGU/5V1uBsq80iNe70ADgdv4nK+UQBXIILubDwpYqfKWgyjKUiGe1A+ZpzLE5l4x9LY0ntzINcbkUtOeLfnX0YuVGKdcB05CDK7Z++YuhrwDHZScSGt00jUpPy8d8tsw1BjbuTVzybNMLL2dI9Ykj8wBHqH+sAYZcfHSWcrvLDIKk5fAmFWFBqrXi5nV3Zu18DyybluytWffUwuXOormo8VWGRhS4HGNSXo3GrzV9edwvK619CpSgBpb+gXykp4+IzBdmIK5FiPypGTnjwFUO/ztdRt9IWhGMMR0SuqWGEb0mhR9bM1QgpWvJK57LXOGQG3CaN+Nb7zmA1FC+eITlTQjQhaI8t8ccvf8JzdfhaeEsyNytwoblz5esct69bDfzRf9yG5uBizwFp9Ou2hqVQ+LqF8GD1LJlVp2FXN/9kCKJEt7dgA8fQPO+trChKgemlyXO2IwG2RQRu/w3UznakabsdraJdUra2n/auJ8kh9ymCLOoPz/er/DjE8k7SyxFIgpPD6VijcgvTBcEJTuWEiIrqttqGbuJixEont KNGUtDTr 5CJYbrUBnmIUkImakmCtc0SV/Li7KWItd7v86ZHW596iBuXJ8qVvWjl1DHgnkJh/bwixJFS8B/X0RaZec30c7XwzGmV1cqa1tzxnTqhsrqkJpl+0hX5NdrMjLBzeDhaxdDLUKuwOrNKFIFBW349vmSfBeO+4cZEFLliyX4gTOLx+V2ZesUCM8cqpihlPh9ngAs30Ok/d2SuSWlO08YB6xruqHZ7D/pLcPGmRD7C2Rv5rfkL41fbp551hQKZMe0Its649pHp1ZR/Hpe6Hq201Y0wUv2rLIh97xHmsrexKAG/UUmb9E6S8fK0zJtwLMn+PeNCz69h/SP+5wNltiZrG1B4XTRwPm8G2xItMxIv3BB7NkW7JOESOh7cSN4XPQf93YUejFOlYI2h9RntENZIxwO6lywo6PAgMwU/MIREvetzPmcYiEvvZezjkNSEMLEGp0UFVu6ys9wsgS34dXH2PPPVbM+SVZiEAy1h8I7nwON+Rd6jym6VJQXAKS6Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Robin Murphy The DMA flags field will be useful for users beyond PCI P2P, so upgrade to its own dedicated config option. Signed-off-by: Robin Murphy [catalin.marinas@arm.com: use #ifdef CONFIG_NEED_SG_DMA_FLAGS in scatterlist.h] [catalin.marinas@arm.com: update PCI_P2PDMA dma_flags comment in scatterlist.h] Signed-off-by: Catalin Marinas Reviewed-by: Christoph Hellwig Tested-by: Isaac J. Manjarres --- drivers/pci/Kconfig | 1 + include/linux/scatterlist.h | 13 ++++++------- kernel/dma/Kconfig | 3 +++ 3 files changed, 10 insertions(+), 7 deletions(-) diff --git a/drivers/pci/Kconfig b/drivers/pci/Kconfig index 9309f2469b41..3c07d8d214b3 100644 --- a/drivers/pci/Kconfig +++ b/drivers/pci/Kconfig @@ -168,6 +168,7 @@ config PCI_P2PDMA # depends on 64BIT select GENERIC_ALLOCATOR + select NEED_SG_DMA_FLAGS help Enableѕ drivers to do PCI peer-to-peer transactions to and from BARs that are exposed in other devices that are the part of diff --git a/include/linux/scatterlist.h b/include/linux/scatterlist.h index 375a5e90d86a..19833fd4113b 100644 --- a/include/linux/scatterlist.h +++ b/include/linux/scatterlist.h @@ -16,7 +16,7 @@ struct scatterlist { #ifdef CONFIG_NEED_SG_DMA_LENGTH unsigned int dma_length; #endif -#ifdef CONFIG_PCI_P2PDMA +#ifdef CONFIG_NEED_SG_DMA_FLAGS unsigned int dma_flags; #endif }; @@ -249,12 +249,11 @@ static inline void sg_unmark_end(struct scatterlist *sg) } /* - * CONFGI_PCI_P2PDMA depends on CONFIG_64BIT which means there is 4 bytes - * in struct scatterlist (assuming also CONFIG_NEED_SG_DMA_LENGTH is set). - * Use this padding for DMA flags bits to indicate when a specific - * dma address is a bus address. + * One 64-bit architectures there is a 4-byte padding in struct scatterlist + * (assuming also CONFIG_NEED_SG_DMA_LENGTH is set). Use this padding for DMA + * flags bits to indicate when a specific dma address is a bus address. */ -#ifdef CONFIG_PCI_P2PDMA +#ifdef CONFIG_NEED_SG_DMA_FLAGS #define SG_DMA_BUS_ADDRESS (1 << 0) @@ -312,7 +311,7 @@ static inline void sg_dma_unmark_bus_address(struct scatterlist *sg) { } -#endif +#endif /* CONFIG_NEED_SG_DMA_FLAGS */ /** * sg_phys - Return physical address of an sg entry diff --git a/kernel/dma/Kconfig b/kernel/dma/Kconfig index 6677d0e64d27..acc6f231259c 100644 --- a/kernel/dma/Kconfig +++ b/kernel/dma/Kconfig @@ -24,6 +24,9 @@ config DMA_OPS_BYPASS config ARCH_HAS_DMA_MAP_DIRECT bool +config NEED_SG_DMA_FLAGS + bool + config NEED_SG_DMA_LENGTH bool From patchwork Mon Jun 12 15:31:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13276869 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 56FC9C7EE2F for ; Mon, 12 Jun 2023 15:33:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EAB7E8E000C; Mon, 12 Jun 2023 11:33:10 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E5C528E0005; Mon, 12 Jun 2023 11:33:10 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CFC858E000C; Mon, 12 Jun 2023 11:33:10 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id BFF528E0005 for ; Mon, 12 Jun 2023 11:33:10 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 7FD9412027A for ; Mon, 12 Jun 2023 15:33:10 +0000 (UTC) X-FDA: 80894489340.30.D4D3DEC Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf23.hostedemail.com (Postfix) with ESMTP id 2FCD614002F for ; Mon, 12 Jun 2023 15:33:06 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf23.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686583987; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9Lo5qv7nUQqFKTvriR5A+h+u9COGpgdN5rIa6SgcmHc=; b=S86hJu+wsdq1ZsxelpGhSfn5n+vFeJ7dbZSuLnc6h37CBMd2xxMcjVxpMBEwIMykM4nsqT FxD+CTf9C1e0CKb9Inifh3hp4VXOHCXimwyHbUXKJ0TfZvfdBOOrz48bWboRABxjioz6w5 clyBm0bTlZ2oqtKdy1Iegb+lhvA6U60= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf23.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686583987; a=rsa-sha256; cv=none; b=D9gpPXmwA66VKUmcv4BQ0NZmqF8Fd5alzQdLG04Q8G2wmC31siF/ZWxeAXTN0fFdNRrMS/ 5nuiZxUZufAI94ZTaVXPK3fnj/keIl66o9YY3bsHtFFebWb9Sxq5OAlZpgOK7vXAZuBWfE GN2xdcJYpd02VcxtJWIpEnNJWOO/GAg= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 111CD62AEB; Mon, 12 Jun 2023 15:33:06 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2EF3FC433A8; Mon, 12 Jun 2023 15:33:01 +0000 (UTC) From: Catalin Marinas To: Andrew Morton , Linus Torvalds Cc: Christoph Hellwig , Robin Murphy , Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Jonathan Cameron , Vlastimil Babka , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org, Jerry Snitselaar , Logan Gunthorpe Subject: [PATCH v7 13/17] dma-mapping: Name SG DMA flag helpers consistently Date: Mon, 12 Jun 2023 16:31:57 +0100 Message-Id: <20230612153201.554742-14-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230612153201.554742-1-catalin.marinas@arm.com> References: <20230612153201.554742-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 2FCD614002F X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: od7fq77yf3tgeuqt35dffcx5tuu8md3p X-HE-Tag: 1686583986-18093 X-HE-Meta: U2FsdGVkX1+6qFJ/LCeoM+ofsGlYgl1NFoyyfuHbEzc532OyYeUxwXiY5t+E1oojddhUUOEOAcxtz3wa4B9YRoGvHheqanPiDHQ/uiIlLWDov8E0HJPdbdt23LFYZbUA90azqxaySd0OmpS5yWW8Mo0aurbUOZPPMJzui6sKakjtfKXlEpTgFrmnDgAKpt5gIJTfQIADqGL6Aoz0ivglCT8m/dPebUxh7+b4L+igl4Bd1DGoEluUnl1d0JZ4mjUQEF0BjHlaTbo32be1l1nqw8dmJHz5OsFzQknc5tKKy0bnT1ef6/YY50O6ZgI+WjJ01hCR2TI6DcwIt8CWI0K2yVp5N8V7ol7fryaJjp59Mr3dqgPQwCJQvNVPGWgklfD2Y4Iw1pezDCpLREo84rqA8mjDqR2TH+sb6SNEcSFnceKwOJ+RQphX9JUJ1/JWd4rHjIEkQkMIBO9ZNQftct71SvSdLwxfKHlvyH3g3CwiLuZi/qv4L+mOcSP7LDhr71ZWlennKeWeurtT9WA22n8pj29e1RitkHl5W36YL3fsKJHdup5xK1PPvtQtHDvIl5Toty7/UNvu+KS7QukTmMjVN9YoMqweDcDr4kmB1wzM324/mOZSh7a8YYgsZvLnkgkSt3V0D8YfatRhrq0ZQ+IyX7VQYHJVxmLxyDlU9pf1Xoxy4O7RVPKzo3/jIm9+t9NinScTB0mUwaBa/tmfBehm8gm5oYrHB/bPXC7OR/L/8M+nRPnjgUSLQJRk3lcT84Z3mR++RSl/M371aXZLKQyxKVwWBn5JT8cbZR1kbvLc+3M5jramdkoJjCY25qh+rrsoWhDHMVWNz77+AEVeP3wj873abSyaGwUI0pq6zeAXiVzF/0FAuFllz2vmnzrpUh/H0ekxB0BhpZbcGfTs6PM0L9SG8x2BAuCZ2AhUGoGfBV1n0nz+Mya5f6zusoXq+XYQYbS1zUzBt1Mr452MtG5 AGo0o+zD KRG/bLqCiCI5pjwyruAkYIU4CQ8ZnqJvQmUMf02bPsU1pqNgduywh2w3+PdHchhv2JaJ6DocbH+uetJtnNxGSu29QgJ4XUl+hnpVo4q122v51cgI1KqnsVc9Rz0KjQYWN7bySDCgj0DleIEgtzV+nqLa9VkHbfvfKwU3JlTOIE2TKvbhDCNQX+tTjk5+UmmheZxgQ/mOh5TMJ4+nEujSx/WjU0L6bCEIzZ7hNw25SYvB+DP+9Ewp/d00atqswgaKz3GF/sIt3S1trQcoPIElE+tfptwFDGcaaSg21dOltgAwwWBzOk3usBdARkEhk3mHmlS8w3Wj7MVevfCoRL6riXFRf3Bw53SPfEWmGkqxAbWSo95AOFWr2S7entbnP5UgQmMyKKq0pO7BABrvQDnLggF2KclgghMfWF81zVT51MRdbXJi/NRwOln05ZLrErXhcWTgR3DMJ/cDDtVY4JXTSNKwtvvuCGnoW7LFL5SyfD+GAWFJ8HeQ0FAIT2HXLRvy9Vo9zd9TEdsDrBIf2yM0uSmLP1izJHjuWpbRRukS1xWway+lko7i5pVipJznWh6suJSL5C607LLjpoakMRl25hHXAeX6BVvojzrLgAyqozzPt3G1POhQNQmDFP3VR52JESeyI X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Robin Murphy sg_is_dma_bus_address() is inconsistent with the naming pattern of its corresponding setters and its own kerneldoc, so take the majority vote and rename it sg_dma_is_bus_address() (and fix up the missing underscores in the kerneldoc too). This gives us a nice clear pattern where SG DMA flags are SG_DMA_, and the helpers for acting on them are sg_dma__(). Signed-off-by: Robin Murphy Reviewed-by: Christoph Hellwig Reviewed-by: Jerry Snitselaar Reviewed-by: Logan Gunthorpe Link: https://lore.kernel.org/r/fa2eca2862c7ffc41b50337abffb2dfd2864d3ea.1685036694.git.robin.murphy@arm.com Signed-off-by: Catalin Marinas Tested-by: Isaac J. Manjarres --- drivers/iommu/dma-iommu.c | 8 ++++---- drivers/iommu/iommu.c | 2 +- include/linux/scatterlist.h | 8 ++++---- kernel/dma/direct.c | 2 +- 4 files changed, 10 insertions(+), 10 deletions(-) diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c index 7a9f0b0bddbd..b8bba4aa196f 100644 --- a/drivers/iommu/dma-iommu.c +++ b/drivers/iommu/dma-iommu.c @@ -1080,7 +1080,7 @@ static int __finalise_sg(struct device *dev, struct scatterlist *sg, int nents, sg_dma_address(s) = DMA_MAPPING_ERROR; sg_dma_len(s) = 0; - if (sg_is_dma_bus_address(s)) { + if (sg_dma_is_bus_address(s)) { if (i > 0) cur = sg_next(cur); @@ -1136,7 +1136,7 @@ static void __invalidate_sg(struct scatterlist *sg, int nents) int i; for_each_sg(sg, s, nents, i) { - if (sg_is_dma_bus_address(s)) { + if (sg_dma_is_bus_address(s)) { sg_dma_unmark_bus_address(s); } else { if (sg_dma_address(s) != DMA_MAPPING_ERROR) @@ -1329,7 +1329,7 @@ static void iommu_dma_unmap_sg(struct device *dev, struct scatterlist *sg, * just have to be determined. */ for_each_sg(sg, tmp, nents, i) { - if (sg_is_dma_bus_address(tmp)) { + if (sg_dma_is_bus_address(tmp)) { sg_dma_unmark_bus_address(tmp); continue; } @@ -1343,7 +1343,7 @@ static void iommu_dma_unmap_sg(struct device *dev, struct scatterlist *sg, nents -= i; for_each_sg(tmp, tmp, nents, i) { - if (sg_is_dma_bus_address(tmp)) { + if (sg_dma_is_bus_address(tmp)) { sg_dma_unmark_bus_address(tmp); continue; } diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index f1dcfa3f1a1b..eb620552967b 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -2567,7 +2567,7 @@ ssize_t iommu_map_sg(struct iommu_domain *domain, unsigned long iova, len = 0; } - if (sg_is_dma_bus_address(sg)) + if (sg_dma_is_bus_address(sg)) goto next; if (len) { diff --git a/include/linux/scatterlist.h b/include/linux/scatterlist.h index 19833fd4113b..2f06178996ba 100644 --- a/include/linux/scatterlist.h +++ b/include/linux/scatterlist.h @@ -258,7 +258,7 @@ static inline void sg_unmark_end(struct scatterlist *sg) #define SG_DMA_BUS_ADDRESS (1 << 0) /** - * sg_dma_is_bus address - Return whether a given segment was marked + * sg_dma_is_bus_address - Return whether a given segment was marked * as a bus address * @sg: SG entry * @@ -266,13 +266,13 @@ static inline void sg_unmark_end(struct scatterlist *sg) * Returns true if sg_dma_mark_bus_address() has been called on * this segment. **/ -static inline bool sg_is_dma_bus_address(struct scatterlist *sg) +static inline bool sg_dma_is_bus_address(struct scatterlist *sg) { return sg->dma_flags & SG_DMA_BUS_ADDRESS; } /** - * sg_dma_mark_bus address - Mark the scatterlist entry as a bus address + * sg_dma_mark_bus_address - Mark the scatterlist entry as a bus address * @sg: SG entry * * Description: @@ -300,7 +300,7 @@ static inline void sg_dma_unmark_bus_address(struct scatterlist *sg) #else -static inline bool sg_is_dma_bus_address(struct scatterlist *sg) +static inline bool sg_dma_is_bus_address(struct scatterlist *sg) { return false; } diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c index 5595d1d5cdcc..d29cade048db 100644 --- a/kernel/dma/direct.c +++ b/kernel/dma/direct.c @@ -463,7 +463,7 @@ void dma_direct_unmap_sg(struct device *dev, struct scatterlist *sgl, int i; for_each_sg(sgl, sg, nents, i) { - if (sg_is_dma_bus_address(sg)) + if (sg_dma_is_bus_address(sg)) sg_dma_unmark_bus_address(sg); else dma_direct_unmap_page(dev, sg->dma_address, From patchwork Mon Jun 12 15:31:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13276870 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B4701C88CB6 for ; Mon, 12 Jun 2023 15:33:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4E7A0940008; Mon, 12 Jun 2023 11:33:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 497A28E0005; Mon, 12 Jun 2023 11:33:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3604F940008; Mon, 12 Jun 2023 11:33:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 2155D8E0005 for ; Mon, 12 Jun 2023 11:33:14 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id EDCF3140299 for ; Mon, 12 Jun 2023 15:33:13 +0000 (UTC) X-FDA: 80894489466.21.AF10567 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf01.hostedemail.com (Postfix) with ESMTP id 756304001A for ; Mon, 12 Jun 2023 15:33:11 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf01.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686583991; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UAWloagYHC8zwKSOoXFeJLmx1VhZfhH4Y6rt98gl2io=; b=38ul5Wg3p79C617e+/4CLzLcC/heZPEVJbpvTABCqnO9VCLTrPVZ9I4BEFGsXEHssY1J9u hfVLBB313Lp5Yz6ZDC0HZCTlqcbFP1EYfZ2aWGkGGVBkc2r8cv2mAKe/x0MWczhCfjXw8N DhCEe7oJYM6XqGIWfYk0/1OM9CWHroE= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf01.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686583991; a=rsa-sha256; cv=none; b=FOWDhdVzRZGIjWyBhfU4fqqY2WsPh3bKMyRJMeUgjMFxOCsvJtC3OL1VnfEuU6P7/DEx9T TeqBavZYe2ykk3JxxqvI65/WLeSUgb6vQLzFb+7EYzox8+blAr9bIK4cy7W+bUQ5M4LN/e 7PV2EKvLdi7wrX8SOsIW2TZm6FAEbHs= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 6DB2E617A4; Mon, 12 Jun 2023 15:33:10 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D1F8DC433D2; Mon, 12 Jun 2023 15:33:05 +0000 (UTC) From: Catalin Marinas To: Andrew Morton , Linus Torvalds Cc: Christoph Hellwig , Robin Murphy , Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Jonathan Cameron , Vlastimil Babka , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v7 14/17] dma-mapping: Force bouncing if the kmalloc() size is not cache-line-aligned Date: Mon, 12 Jun 2023 16:31:58 +0100 Message-Id: <20230612153201.554742-15-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230612153201.554742-1-catalin.marinas@arm.com> References: <20230612153201.554742-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 756304001A X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: n9pg4pwtiqx1fyecg1x1jrka361z6e75 X-HE-Tag: 1686583991-786500 X-HE-Meta: U2FsdGVkX1+8zZ12hTjavDUla91St9ypO4xXOaBLevBcq2dlTNaq6RaeS7h+MMsROEQqwTHTC32D6WlDuTSeDEndjV5Vx90DgUAaasdXBmNxsMntbmh6KRlJQTJoZz1sDbIYt1BKKszwi4N9TdM32lAdSPVKMoyTmjzhiP38Ns8eV0NLK8+rutwVA7SMrRccSlT9S9NfYN8H2ZlkSlzegKal1pEuwJfhVVWVlbSQt7WZwLOLL1rxZ56+9H06LRkoDgz04apuMQBWOBN0Nm+B4lgFpftOURNi0NIexDQ9pogTPgzKwKDBCEFN5N8mvLlP3KiZvh0vY2Uku0w6ZlgKiiMe6g3v5nFVaFS04+5XVcjxUfln18KruzOXVPOMW5N/ijysJzIRsNH6Anw+6qQd1Fib43oKy9A2148NRUXO0hIS13sTjVrRSeZIMufb1DbO+LIXKUPDKmyK7whv/8ICAhrXPmNOVCutv9lyYbtvRX/8eAP6fiOWhXVDWYS0+kDZiqYs+PhozsThzN0p+ZmshFvpyFt+n4tWWROipF+y6D22HEOuFhYqLX/gbwyqrZTq1sjFoNFQhpXTswflB51o4qp9aA9C2Xl/ai+OnKfcc7PoRtZqsBbdm+J5FiVlR3pyKwQOGpqhSfw7WcS8MiSNP36k2fb331uA6uQylNLJyW6Q1orOt2QIZ0IJWAL6ocM4cda/m18dZcC9ZitbQwNdXLelwvhks8qVhWM2mVoiUs/B6K7NDxCP+IHMNbF/kAw25n1zTqit/r8Kn+ShDfCQnmb43xfzlaIkcjrZKeJnyMdRjegrijSu274Y/jEhhZBmUecgULvo03Ug3p76Sevfzo+DlL+NyHcCKywpCjdqy1quV9h5oDfLLyQTU9mMBgus5lgRFeATE3BKo0+JhltMYNbKCVRlWGFLWSuFZghfhqgxZbaWJPpTEcla3PnWFv1RLtpoo5eb+LiP50oPiKw 5Zf/UkWP 9o+6mC2MtdRijAqimZomhMzQL6IxIwLYG0/9bK8sglzAw93M7MEnpKYj9RuoZObFUu2BY0utkKnSIiL79BdQcCZtLhBhEFnJwGV4+BG0fuoft7gB1hfsVce5H1fNRrYrRBxMFeHS3Y5TW6ZLqqJMVVzCwiWASMwTYnArP1y5or7cB6q2zkfg1J6fjENBSUAyUqHPikzHXuHchaE0cf7rwIrT+S20/cCQvNv+QLlHk8mmlrwbF4jAd0i0gBTiiEU5AeDuSMFjfT/qIuwuJoSYz9iFgQJm766oxs+Fdw4gaA1FFwZkNtxrkMrrLOaxqh3bT7ZG6nJ0JN845Tv4nPrM8GrZq2yo77wlyiSEouh27dKDEFRRJKAepPjF8kKXb2b3H0lN0noYs07gciikEV6tidz9E6XTBFm5p7K2XZ5ZkoUAbqowB69KFnkasUMRRWNp7yryu6v0wfp7PUzsFnwaPBn8KR2Q6mlxWMWtubx1AbF0kY3M= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: For direct DMA, if the size is small enough to have originated from a kmalloc() cache below ARCH_DMA_MINALIGN, check its alignment against dma_get_cache_alignment() and bounce if necessary. For larger sizes, it is the responsibility of the DMA API caller to ensure proper alignment. At this point, the kmalloc() caches are properly aligned but this will change in a subsequent patch. Architectures can opt in by selecting DMA_BOUNCE_UNALIGNED_KMALLOC. Signed-off-by: Catalin Marinas Reviewed-by: Christoph Hellwig Reviewed-by: Robin Murphy Tested-by: Isaac J. Manjarres --- include/linux/dma-map-ops.h | 61 +++++++++++++++++++++++++++++++++++++ kernel/dma/Kconfig | 4 +++ kernel/dma/direct.h | 3 +- 3 files changed, 67 insertions(+), 1 deletion(-) diff --git a/include/linux/dma-map-ops.h b/include/linux/dma-map-ops.h index 31f114f486c4..9bf19b5bf755 100644 --- a/include/linux/dma-map-ops.h +++ b/include/linux/dma-map-ops.h @@ -8,6 +8,7 @@ #include #include +#include struct cma; @@ -277,6 +278,66 @@ static inline bool dev_is_dma_coherent(struct device *dev) } #endif /* CONFIG_ARCH_HAS_DMA_COHERENCE_H */ +/* + * Check whether potential kmalloc() buffers are safe for non-coherent DMA. + */ +static inline bool dma_kmalloc_safe(struct device *dev, + enum dma_data_direction dir) +{ + /* + * If DMA bouncing of kmalloc() buffers is disabled, the kmalloc() + * caches have already been aligned to a DMA-safe size. + */ + if (!IS_ENABLED(CONFIG_DMA_BOUNCE_UNALIGNED_KMALLOC)) + return true; + + /* + * kmalloc() buffers are DMA-safe irrespective of size if the device + * is coherent or the direction is DMA_TO_DEVICE (non-desctructive + * cache maintenance and benign cache line evictions). + */ + if (dev_is_dma_coherent(dev) || dir == DMA_TO_DEVICE) + return true; + + return false; +} + +/* + * Check whether the given size, assuming it is for a kmalloc()'ed buffer, is + * sufficiently aligned for non-coherent DMA. + */ +static inline bool dma_kmalloc_size_aligned(size_t size) +{ + /* + * Larger kmalloc() sizes are guaranteed to be aligned to + * ARCH_DMA_MINALIGN. + */ + if (size >= 2 * ARCH_DMA_MINALIGN || + IS_ALIGNED(kmalloc_size_roundup(size), dma_get_cache_alignment())) + return true; + + return false; +} + +/* + * Check whether the given object size may have originated from a kmalloc() + * buffer with a slab alignment below the DMA-safe alignment and needs + * bouncing for non-coherent DMA. The pointer alignment is not considered and + * in-structure DMA-safe offsets are the responsibility of the caller. Such + * code should use the static ARCH_DMA_MINALIGN for compiler annotations. + * + * The heuristics can have false positives, bouncing unnecessarily, though the + * buffers would be small. False negatives are theoretically possible if, for + * example, multiple small kmalloc() buffers are coalesced into a larger + * buffer that passes the alignment check. There are no such known constructs + * in the kernel. + */ +static inline bool dma_kmalloc_needs_bounce(struct device *dev, size_t size, + enum dma_data_direction dir) +{ + return !dma_kmalloc_safe(dev, dir) && !dma_kmalloc_size_aligned(size); +} + void *arch_dma_alloc(struct device *dev, size_t size, dma_addr_t *dma_handle, gfp_t gfp, unsigned long attrs); void arch_dma_free(struct device *dev, size_t size, void *cpu_addr, diff --git a/kernel/dma/Kconfig b/kernel/dma/Kconfig index acc6f231259c..abea1823fe21 100644 --- a/kernel/dma/Kconfig +++ b/kernel/dma/Kconfig @@ -90,6 +90,10 @@ config SWIOTLB bool select NEED_DMA_MAP_STATE +config DMA_BOUNCE_UNALIGNED_KMALLOC + bool + depends on SWIOTLB + config DMA_RESTRICTED_POOL bool "DMA Restricted Pool" depends on OF && OF_RESERVED_MEM && SWIOTLB diff --git a/kernel/dma/direct.h b/kernel/dma/direct.h index e38ffc5e6bdd..97ec892ea0b5 100644 --- a/kernel/dma/direct.h +++ b/kernel/dma/direct.h @@ -94,7 +94,8 @@ static inline dma_addr_t dma_direct_map_page(struct device *dev, return swiotlb_map(dev, phys, size, dir, attrs); } - if (unlikely(!dma_capable(dev, dma_addr, size, true))) { + if (unlikely(!dma_capable(dev, dma_addr, size, true)) || + dma_kmalloc_needs_bounce(dev, size, dir)) { if (is_pci_p2pdma_page(page)) return DMA_MAPPING_ERROR; if (is_swiotlb_active(dev)) From patchwork Mon Jun 12 15:31:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13276871 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C02AAC7EE2F for ; Mon, 12 Jun 2023 15:33:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 66E6B8E0009; Mon, 12 Jun 2023 11:33:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 61DFE8E0005; Mon, 12 Jun 2023 11:33:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4BF548E0009; Mon, 12 Jun 2023 11:33:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 3D1958E0005 for ; Mon, 12 Jun 2023 11:33:18 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 9B0D340288 for ; Mon, 12 Jun 2023 15:33:17 +0000 (UTC) X-FDA: 80894489634.08.B899B77 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf27.hostedemail.com (Postfix) with ESMTP id AF54B4000E for ; Mon, 12 Jun 2023 15:33:15 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf27.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686583995; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5qLXjBEatpN3Ri0xxgcy2oBKkS0dTlGY3nafxGrdKuY=; b=0ippCduo7WojyKbXSAp7ZZ0OaSTq98cVRCdKeMoKr5HC8a2uKoKchLpx49PBt1R5BwnSNX 4G3hgaf7oX2JDaYNibl5d0RgyLW1cSoJz6m2T0rDbTeEiNTeKfJYwZ+AKddx7BD/EJLP72 oSdxYl9t8HnPVs0fyoJgi3vwgvtJGmU= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf27.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686583995; a=rsa-sha256; cv=none; b=dncPw64UZSWsk/UZXeJcAbA2VqWS2GQ68NnTrTNUMBxuya7vuH7xKFsRAo2cdzEPw0xjN4 kggAbyhJUN8l10luw0wxLlWHHbbIG+t1Z/sMgvq1f8JMoRzTnhFyFMN5LF512hPJ+2GvOk Pgj/3f4ZVtqnLfF8SQawk+Y2O00NM/I= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id DA03261FB6; Mon, 12 Jun 2023 15:33:14 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3352CC433B3; Mon, 12 Jun 2023 15:33:10 +0000 (UTC) From: Catalin Marinas To: Andrew Morton , Linus Torvalds Cc: Christoph Hellwig , Robin Murphy , Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Jonathan Cameron , Vlastimil Babka , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v7 15/17] iommu/dma: Force bouncing if the size is not cacheline-aligned Date: Mon, 12 Jun 2023 16:31:59 +0100 Message-Id: <20230612153201.554742-16-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230612153201.554742-1-catalin.marinas@arm.com> References: <20230612153201.554742-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: AF54B4000E X-Stat-Signature: 7aaqi7oiret4b8q5qzo5r4tddhuw83bd X-HE-Tag: 1686583995-34743 X-HE-Meta: U2FsdGVkX1/3saP4fLRIvQ5VMpB/ynb1CdJXkGQjfQf7azJpHLm+4NNu8UDcNsY1P8mBwUQQP/XtKk0+JSLz1aQq+yEU9AfzehLlZhL0t81v7Q+Rdd5yf69MQBUZ26iggZpZ1l2rVjg4ewI8Ji3W5ZjkspzOS0wVmBbX+//drrFYdWWsmTzY7csyNE6gwW02FSKTOMICscI2qwKuFgo8OCCDY2wUH4bJahSydM1g+md8Nv/YuSOrxIDNpJ/C9Im1y23W6KSe0xEkdmgzYV4s9LciSpNoTRW0ImqB06Y3MlJ2vGHpefH/hPn1vvJJDVOnbZSoa8CWdtiKKefedri3Tei+DcDevfa4ki1orsptMwuhC1K0QBhuAtsEslhTL9V0cJlbtqI99LuXY7wQKxFvIuvK8UNeBl+2WPzKBPdBMUE+9o32t52tJ5dEvJIFSxr3H8rPbz14uhn4JkKvcZbBma2QivYv3JlNdbhcEK3nus6vn5KKJ+Si1Ozp5U4oMrlYXqF/rABp45sq+YffAv0+mv5p0NoQApU+WeDpQAAhQzfpMbCXVEpzUxzyTElsZefkOY7wY0ysxGGh9eDnIxrNSyYBTgXoG7qrIrZKOi997CB3SKosCl3fcu8BVwt7y8gSDsq1tuOPCG+jLnEIl3xie9FwTwl1JWJRM46tffRSU+B3HL736Apnucj/YimUxGqfqy/vbTUTYtyYCI75UsJPjmPg4ewp+m1QOZd+EGR426bI36Jnd19K+R7xRa+i6EPdauFfp+HPt3MxnYNwHB4etaQcqUqv0/xhsV1D/EjAPyQ0Ct2XUhcyabhc1gw6QSMIWtVBX1xlvRdsWFCSSumdgV6VW3rsrnDAnYtHMxSi6QUSdsLmJRU8SUjDFgdTkn+Fni8vrnEZ1ZgOuMdo0+JJBl014sD+7EjojgTuq0GC9oE6K108pu0wa3RkOUj2Q/ZvVIcMfWybY7HOIau2bLK tfjk7m0f 2pO8Km+cDv3JiOCwF8WOAh2JWvPAbT/0ymHcv37GexRgKISM9X9NVGnuNLG1VMXdfCweOo9tiR2//ACiofA0MavD/SJz3Kl3a11aHVm9uBKjFW+mD0LhTDFaXZKAkIDUmec1kC4NWw5Bq98DuyFSadJmXey19gosK0ujUrM/cXystlyLm0UKtbTWdMd/Zw8ApVOmXN+OEy8q+5VmNJSmJ5jta+XLY4PtvkCIydOkMFAsgAn+EgqZ7G+wdEm6+NCxl0w253OIjwJILN7eCCxsMuGY9XPuxjBnFyPfS1C8r/i3prKW5NUlJVHq9Lxhg+Gr0Ob192ZaD8mqtLBA4SC6nHaP3aVCHwAoVBOntmccBEJNBNdJu9lvoyf43DXD/+eq+6XpKT6sfy/jfcGwRYCTtL++76BsNFcThAhUH+WTG79KULnAqKBPO0F8Y2Z/enhJqV5J6SZuvAXgL743fSxODbfxeY09x3KKuNopnZsrSRau+Ou3MGOxW66JpF7JIPJ/pX14tEqhMdwW2a+FXIYecRMXfotc2o6O2jOQ2taPhM6tuxd4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Similarly to the direct DMA, bounce small allocations as they may have originated from a kmalloc() cache not safe for DMA. Unlike the direct DMA, iommu_dma_map_sg() cannot call iommu_dma_map_sg_swiotlb() for all non-coherent devices as this would break some cases where the iova is expected to be contiguous (dmabuf). Instead, scan the scatterlist for any small sizes and only go the swiotlb path if any element of the list needs bouncing (note that iommu_dma_map_page() would still only bounce those buffers which are not DMA-aligned). To avoid scanning the scatterlist on the 'sync' operations, introduce an SG_DMA_SWIOTLB flag set by iommu_dma_map_sg_swiotlb(). The dev_use_swiotlb() function together with the newly added dev_use_sg_swiotlb() now check for both untrusted devices and unaligned kmalloc() buffers (suggested by Robin Murphy). Signed-off-by: Catalin Marinas Reviewed-by: Robin Murphy Cc: Joerg Roedel Cc: Christoph Hellwig Tested-by: Isaac J. Manjarres --- drivers/iommu/Kconfig | 1 + drivers/iommu/dma-iommu.c | 50 ++++++++++++++++++++++++++++++------- include/linux/scatterlist.h | 41 ++++++++++++++++++++++++++++-- 3 files changed, 81 insertions(+), 11 deletions(-) diff --git a/drivers/iommu/Kconfig b/drivers/iommu/Kconfig index db98c3f86e8c..670eff7a8e11 100644 --- a/drivers/iommu/Kconfig +++ b/drivers/iommu/Kconfig @@ -152,6 +152,7 @@ config IOMMU_DMA select IOMMU_IOVA select IRQ_MSI_IOMMU select NEED_SG_DMA_LENGTH + select NEED_SG_DMA_FLAGS if SWIOTLB # Shared Virtual Addressing config IOMMU_SVA diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c index b8bba4aa196f..e86ae462cade 100644 --- a/drivers/iommu/dma-iommu.c +++ b/drivers/iommu/dma-iommu.c @@ -520,9 +520,38 @@ static bool dev_is_untrusted(struct device *dev) return dev_is_pci(dev) && to_pci_dev(dev)->untrusted; } -static bool dev_use_swiotlb(struct device *dev) +static bool dev_use_swiotlb(struct device *dev, size_t size, + enum dma_data_direction dir) { - return IS_ENABLED(CONFIG_SWIOTLB) && dev_is_untrusted(dev); + return IS_ENABLED(CONFIG_SWIOTLB) && + (dev_is_untrusted(dev) || + dma_kmalloc_needs_bounce(dev, size, dir)); +} + +static bool dev_use_sg_swiotlb(struct device *dev, struct scatterlist *sg, + int nents, enum dma_data_direction dir) +{ + struct scatterlist *s; + int i; + + if (!IS_ENABLED(CONFIG_SWIOTLB)) + return false; + + if (dev_is_untrusted(dev)) + return true; + + /* + * If kmalloc() buffers are not DMA-safe for this device and + * direction, check the individual lengths in the sg list. If any + * element is deemed unsafe, use the swiotlb for bouncing. + */ + if (!dma_kmalloc_safe(dev, dir)) { + for_each_sg(sg, s, nents, i) + if (!dma_kmalloc_size_aligned(s->length)) + return true; + } + + return false; } /** @@ -922,7 +951,7 @@ static void iommu_dma_sync_single_for_cpu(struct device *dev, { phys_addr_t phys; - if (dev_is_dma_coherent(dev) && !dev_use_swiotlb(dev)) + if (dev_is_dma_coherent(dev) && !dev_use_swiotlb(dev, size, dir)) return; phys = iommu_iova_to_phys(iommu_get_dma_domain(dev), dma_handle); @@ -938,7 +967,7 @@ static void iommu_dma_sync_single_for_device(struct device *dev, { phys_addr_t phys; - if (dev_is_dma_coherent(dev) && !dev_use_swiotlb(dev)) + if (dev_is_dma_coherent(dev) && !dev_use_swiotlb(dev, size, dir)) return; phys = iommu_iova_to_phys(iommu_get_dma_domain(dev), dma_handle); @@ -956,7 +985,7 @@ static void iommu_dma_sync_sg_for_cpu(struct device *dev, struct scatterlist *sg; int i; - if (dev_use_swiotlb(dev)) + if (sg_dma_is_swiotlb(sgl)) for_each_sg(sgl, sg, nelems, i) iommu_dma_sync_single_for_cpu(dev, sg_dma_address(sg), sg->length, dir); @@ -972,7 +1001,7 @@ static void iommu_dma_sync_sg_for_device(struct device *dev, struct scatterlist *sg; int i; - if (dev_use_swiotlb(dev)) + if (sg_dma_is_swiotlb(sgl)) for_each_sg(sgl, sg, nelems, i) iommu_dma_sync_single_for_device(dev, sg_dma_address(sg), @@ -998,7 +1027,8 @@ static dma_addr_t iommu_dma_map_page(struct device *dev, struct page *page, * If both the physical buffer start address and size are * page aligned, we don't need to use a bounce page. */ - if (dev_use_swiotlb(dev) && iova_offset(iovad, phys | size)) { + if (dev_use_swiotlb(dev, size, dir) && + iova_offset(iovad, phys | size)) { void *padding_start; size_t padding_size, aligned_size; @@ -1166,6 +1196,8 @@ static int iommu_dma_map_sg_swiotlb(struct device *dev, struct scatterlist *sg, struct scatterlist *s; int i; + sg_dma_mark_swiotlb(sg); + for_each_sg(sg, s, nents, i) { sg_dma_address(s) = iommu_dma_map_page(dev, sg_page(s), s->offset, s->length, dir, attrs); @@ -1210,7 +1242,7 @@ static int iommu_dma_map_sg(struct device *dev, struct scatterlist *sg, goto out; } - if (dev_use_swiotlb(dev)) + if (dev_use_sg_swiotlb(dev, sg, nents, dir)) return iommu_dma_map_sg_swiotlb(dev, sg, nents, dir, attrs); if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC)) @@ -1315,7 +1347,7 @@ static void iommu_dma_unmap_sg(struct device *dev, struct scatterlist *sg, struct scatterlist *tmp; int i; - if (dev_use_swiotlb(dev)) { + if (sg_dma_is_swiotlb(sg)) { iommu_dma_unmap_sg_swiotlb(dev, sg, nents, dir, attrs); return; } diff --git a/include/linux/scatterlist.h b/include/linux/scatterlist.h index 2f06178996ba..ec46d8e8e49d 100644 --- a/include/linux/scatterlist.h +++ b/include/linux/scatterlist.h @@ -251,11 +251,13 @@ static inline void sg_unmark_end(struct scatterlist *sg) /* * One 64-bit architectures there is a 4-byte padding in struct scatterlist * (assuming also CONFIG_NEED_SG_DMA_LENGTH is set). Use this padding for DMA - * flags bits to indicate when a specific dma address is a bus address. + * flags bits to indicate when a specific dma address is a bus address or the + * buffer may have been bounced via SWIOTLB. */ #ifdef CONFIG_NEED_SG_DMA_FLAGS -#define SG_DMA_BUS_ADDRESS (1 << 0) +#define SG_DMA_BUS_ADDRESS (1 << 0) +#define SG_DMA_SWIOTLB (1 << 1) /** * sg_dma_is_bus_address - Return whether a given segment was marked @@ -298,6 +300,34 @@ static inline void sg_dma_unmark_bus_address(struct scatterlist *sg) sg->dma_flags &= ~SG_DMA_BUS_ADDRESS; } +/** + * sg_dma_is_swiotlb - Return whether the scatterlist was marked for SWIOTLB + * bouncing + * @sg: SG entry + * + * Description: + * Returns true if the scatterlist was marked for SWIOTLB bouncing. Not all + * elements may have been bounced, so the caller would have to check + * individual SG entries with is_swiotlb_buffer(). + */ +static inline bool sg_dma_is_swiotlb(struct scatterlist *sg) +{ + return sg->dma_flags & SG_DMA_SWIOTLB; +} + +/** + * sg_dma_mark_swiotlb - Mark the scatterlist for SWIOTLB bouncing + * @sg: SG entry + * + * Description: + * Marks a a scatterlist for SWIOTLB bounce. Not all SG entries may be + * bounced. + */ +static inline void sg_dma_mark_swiotlb(struct scatterlist *sg) +{ + sg->dma_flags |= SG_DMA_SWIOTLB; +} + #else static inline bool sg_dma_is_bus_address(struct scatterlist *sg) @@ -310,6 +340,13 @@ static inline void sg_dma_mark_bus_address(struct scatterlist *sg) static inline void sg_dma_unmark_bus_address(struct scatterlist *sg) { } +static inline bool sg_dma_is_swiotlb(struct scatterlist *sg) +{ + return false; +} +static inline void sg_dma_mark_swiotlb(struct scatterlist *sg) +{ +} #endif /* CONFIG_NEED_SG_DMA_FLAGS */ From patchwork Mon Jun 12 15:32:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13276872 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 189EAC7EE2F for ; Mon, 12 Jun 2023 15:33:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B28F18E000B; Mon, 12 Jun 2023 11:33:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AD94F8E0005; Mon, 12 Jun 2023 11:33:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9C84F8E000B; Mon, 12 Jun 2023 11:33:22 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 8F3758E0005 for ; Mon, 12 Jun 2023 11:33:22 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 2A5E1C0280 for ; Mon, 12 Jun 2023 15:33:22 +0000 (UTC) X-FDA: 80894489844.17.DBBA25F Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf05.hostedemail.com (Postfix) with ESMTP id 545E510000C for ; Mon, 12 Jun 2023 15:33:19 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf05.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686584000; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+tCZdLyULJntPoisjA5Kvnhf6MMZALveOM8+g4gPdAo=; b=8DJrJiZr1tyfL2l0JU0uw2NrBI09oM4oIQSbsbx5vYJIGCXIBD2lEsWE9iuuKowA0OddAT aNuKw0B0ri5t13+r3ywMB8G8/FDOiF6iaal3rPglb3kWQgQKJVGN6SrHgZEiPuZWRo4ikc kTcmzNBqdIWLp4G1AgNSieWClU04kys= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf05.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686584000; a=rsa-sha256; cv=none; b=Lm+pHzoJ2/cTOS5Rlm1FFS/PO4bMNfr+NZ37FfJYIqV86lN3zduO/CbMRsjet2v+TpV89Y AQjwpZegHdDrN75cGsdNZ4vfpNX2q7oPUAMGpaNKukmEbq0of5zc4ApqK66NVbSEybO/p8 zjrxqh1n6YRuVVF46VYsW5nhMW/QqXI= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 4DE23618D8; Mon, 12 Jun 2023 15:33:19 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id AD3B6C433A1; Mon, 12 Jun 2023 15:33:14 +0000 (UTC) From: Catalin Marinas To: Andrew Morton , Linus Torvalds Cc: Christoph Hellwig , Robin Murphy , Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Jonathan Cameron , Vlastimil Babka , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v7 16/17] mm: slab: Reduce the kmalloc() minimum alignment if DMA bouncing possible Date: Mon, 12 Jun 2023 16:32:00 +0100 Message-Id: <20230612153201.554742-17-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230612153201.554742-1-catalin.marinas@arm.com> References: <20230612153201.554742-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: 545hp5zr848f5cgwd6pxtoh8ihbj5idk X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 545E510000C X-HE-Tag: 1686583999-111584 X-HE-Meta: U2FsdGVkX1+gl8udmUKj5siwnIpdy8pVmRX228xWYue1oDRsmIKMwY8JRBxRShNFZXwtK/y/AlbsgncspKyIGqAG/L4/5OEfdoQmI3cIjRW/VeS3fEm6FcjUfK91sION0qyCx4S30I9/duf8Rv8jWAd/yoTCdUSQx7FAxHe7xGE7+umDDMIqeI0c+kGhpKtKG5h5tG3HyDLnDPoW6Ih/B0hBEHtypB4vrr3or1E7K7y7fS5no5xABMZI4R6jEiNK90mJXuXyzxT12200Yh6SwAFlqXqyq5vbZTEES6ONtwzV73QsJiJ94rwAbhuaNsWrnm8alGkZ7yfOpt3kbFmdGbgzQWIOLTri/FNZWJAGYIpUphmIUqXR6xKf+3fk0wFGbMhoQCmUOjoVD3KOIeISNDID7cf4MWgQ/NRG9u3/m6BiqY09Qg/EL+H6HpC5BOKa+Fu7Fc8v5Ypi9qYp41Ql/RBMZB3s4eXgwf+iqA5Hq90g9hpE+EgmkCARvN13OaeFe9l9/vBg28ST01P5Od41Fa3AffPTqlRDJEDRlgbdcS8xIoRfcRyqDUs3HCxl7QeDu/oB2eQUpqfHmPP2BGMIssIUAyLVRMwCIntF17q9rTlv1qoFohkJDH1PJsC0XNzNXTwwKD76kNNKJkR0V9HSqUzDPRPn5rc4kZtYSpLeO77q0tVgfx0eeGJ95gvry5OYmx2qZ3y+CA2OqwBqhuk5fR5y3+TtAz3tcp9bn0PtPYIIzBGkbe/GA+ABoX+UsPF1xP9tR1riuAo+7Ds7pKIKOHiMnXgbqnfyv86woK5/J0kjBlAh9wRl0rSBUIMSXAbx4GF2gKKlUKkEnHC2OTwXRt8kM6ak8cWUBjyFvNJWKT9GgBADG58o6d1OX9XlYBI7eGWAKMMwhamZsYB+9O7Hn37boAnKsAfDHEiRMCNqFNwtgIFxW/nH0U4tY7SRWMy3yrBbfYow7zAh7ro+Sfy vpkGArct usU9XLxR31iiidQDBrVDxxFJGB3qXY476Ttr4ZQu0U+NpoHh7jcCceck73Abk/1KFWq/rT9I9HLVl4j/HnX8TfeEtFuNP2EXMO/dyCp5Zh1GxpMj3mVq8dMpVIsjq3Tl/PukroRz+TEcyj408NeSnj2N2ERLGeq4TGy57J4eQ/LHUe9VAp2AQ0JFiSVq8p22dg6J+pxmc1g6pzFdv4Vfwy8s5sKID18vJGdLS2yEvsRLv4SSi3+ymIE8OcW5IljUoMyW9I7G8GU/WKDFGRWyOgC/N+zOFyMahdIYrmt7Af+7pog6VE4VOxXB/g1JlnIZABDwp1hHbe8kPcchbLJYjFaF/4PndNxY8OL8tSUDoqCuBr+thf2xWXG8USSJ+8KoRye9PD/E6JrXKv2iv/4PBbDBeNvlH7ZJjL7IRwMTQdAS4ykJxC2ZModAMbN+tOLOYIeqlO4omT+cdp7iT+fRbiBF0J0XZ1YT+NXf/4D1+ecBZngBK9Dfn2F++iJBQYR+vv7EcOLez4Mt5mi/E8bmC3MlocheC+Y2tT7J4X2URIw6dPH8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: If an architecture opted in to DMA bouncing of unaligned kmalloc() buffers (ARCH_WANT_KMALLOC_DMA_BOUNCE), reduce the minimum kmalloc() cache alignment below cache-line size to ARCH_KMALLOC_MINALIGN. Signed-off-by: Catalin Marinas Reviewed-by: Vlastimil Babka Cc: Andrew Morton Cc: Christoph Hellwig Cc: Robin Murphy Tested-by: Isaac J. Manjarres --- mm/slab_common.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/mm/slab_common.c b/mm/slab_common.c index 7c6475847fdf..43c008165f56 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -18,6 +18,7 @@ #include #include #include +#include #include #include #include @@ -865,6 +866,10 @@ void __init setup_kmalloc_cache_index_table(void) static unsigned int __kmalloc_minalign(void) { +#ifdef CONFIG_DMA_BOUNCE_UNALIGNED_KMALLOC + if (io_tlb_default_mem.nslabs) + return ARCH_KMALLOC_MINALIGN; +#endif return dma_get_cache_alignment(); } From patchwork Mon Jun 12 15:32:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13276873 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C8CBFC7EE2F for ; Mon, 12 Jun 2023 15:33:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6E3BF8E0005; Mon, 12 Jun 2023 11:33:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 69395940009; Mon, 12 Jun 2023 11:33:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 55D528E000D; Mon, 12 Jun 2023 11:33:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 3D1218E0005 for ; Mon, 12 Jun 2023 11:33:26 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 1FE79AF5D4 for ; Mon, 12 Jun 2023 15:33:26 +0000 (UTC) X-FDA: 80894490012.08.F0DDAEA Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf06.hostedemail.com (Postfix) with ESMTP id 764B1180006 for ; Mon, 12 Jun 2023 15:33:24 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf06.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686584004; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ohstg6Y6DkD+7eeT8TqXPEnbed0FH6PfCyBYZpswvgo=; b=MewcNe3eSRVC8io1f3Rt/IN9HPfz89v6OVT/FyasmBwhEs2q6Cmgz0jLNk4eJY87brett1 m7YCQFSlhTxs1AFNNTDgNON20mv+Y1UIa2kLRX8ra45xskzVPX625LBXgG79Hgu766Haey gMWHpMBqg2oGUxs0OKa4+wZ647/5PKI= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf06.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686584004; a=rsa-sha256; cv=none; b=L6opSX7MixD4RptWwlyuLVvVszmo+1SH6ssVhkXgY9AiYoOUnmj31aF8JrioiCV3dVlUjq X7AG+1qn2bd/5I3QxvmeE6WQ+5AvdprooGo4CThCU+GvayceJTIVfkLLGH6xnOUI/y1yoB cVmsAgtBBaFJO20Tv0jgnkJH5dtA/NM= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id A03F5617E7; Mon, 12 Jun 2023 15:33:23 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 216D4C433D2; Mon, 12 Jun 2023 15:33:18 +0000 (UTC) From: Catalin Marinas To: Andrew Morton , Linus Torvalds Cc: Christoph Hellwig , Robin Murphy , Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Jonathan Cameron , Vlastimil Babka , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v7 17/17] arm64: Enable ARCH_WANT_KMALLOC_DMA_BOUNCE for arm64 Date: Mon, 12 Jun 2023 16:32:01 +0100 Message-Id: <20230612153201.554742-18-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230612153201.554742-1-catalin.marinas@arm.com> References: <20230612153201.554742-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 764B1180006 X-Stat-Signature: bshd56sfgwuq68ickc41cj9n6uz89dh7 X-HE-Tag: 1686584004-929576 X-HE-Meta: U2FsdGVkX1/pxaZPCpFM8PCvDfLBvn1uz74w132rYZjigEuj3l+caA2AEx+/zAubeXjDveTvRXmUCZisV9mUNJ5nvnj8/xfkouiPrBzJNC0U2hXlQMw46As9gks4dTgdCcvgbJ/CerxwlKA80z1PiShiOwer/RTZYP3jAVWncOaSQ7h/dvYV/IUVeuN3ONnP5jMFGnzgnSnz97fy24CqbWaliIze8Fc4JdNfinKYOVeka9vqov/QU7z0/zup3DjaAfJKTYFwF6OtRruS/xGgR/LECOPq6nxkCYoVq2HvhMtNnNagnCfRZDwCJGAplIWTWWPFP/gsdInqmfhLDfsecUhUnh6zU2Y/TSwMkQUEafBZRUGaGxE90ouQZrGBbQmDamGQVQ0dJV97R+pEcoKcZ0CfZqPabwJCiRWTFrUKaLqQntgVPHT0DrMPWU5KpPxK1VhRBkHSqafRgMYYBTe5u13k2CGySIeb/3cJYkHVvstaqXfCazZhpLyFA1Ctu/8y/rd1tcY9MAUYV9V8iKrC+VoN+ChFWmMwC/bcZil1sMw4kBAN8GBGT37U5pvazlDpDYcUBcK/Ny1UT9O2YD32eH8HnRFto+8+DsX5hdKQOYVzw1x3u6etnXcd3uFCsRLxY6mfIGdK4lz4vMAL6svFDbewPv+RLAjoJwBiwgNQxHJHwxogSqyn+6hB3XHHQAQbH7XzEaZEoRT49lkh2/bzLbZnQ9BABgglY97pdBPsi52lYgBYpojzjjT4yo15foz2R+zZcZSyeNbtJtHMxcC20gHd2FojGC5dUCmHmWSrle9fKyUi2RiQvUmfLmBsC9XkSrJf7s4QkYAVLlI79l+4yyuWAzXGKv9ZlylamLUPp8ZfSfBXGL9wVsMo7FmjjuPkcuKPe/ZFbiV+bd2HcAcwUout8IxXYI/JcZcxQPU0v7zGZzJqmjgrUSH3eBlEzRMzcumz58JV/gvVRvbb82y /DoVr4So Tzhwt76j7BXN2sMo2/qaeNl0ubV7kwXvlcGxJHRrSDTPJ7AmOKly6fz7oqVHgFsEekHGywncNHjYUSdX1gBwmRmA1Tiqt4dO4FKjDUzYYT2yg4i1ZqH3+9DuW5UOZs0Ntzno4XAsZcNkKFUAJSnmiY9J9kItSSKp70Glhb0zMzrpHvRH2U+daomgUSuCfDgile5OU7ftxJMXUpBjiIuTZ8j4hohbh88cUt2RjRRKNcup0hHm4Bl/pfeRZmPEmi/Tp7+boHJlhr2F2SiZEI4z8dPfn5TO24nzMu7UilMcf46XUGdec3PZljRLOkxRagIPv6PmJsIEaUuO0SPpzprRjW5RHRlVm8zJxFDtr4CtnRaodPMcOYhnmVnRGPZkIohvlS5uCYmvp6Wb6QsC/VZT549/jhAQdjF8C6QPhdIQIbAM3k1NrsIQzRX2rRbDdIK0IGqJYaq+EOi37FzlXpnnsO0lbAeBe2C+zQfSI9KKsvXiowbLmlUDJyigk1baQG1h6U42X2qfjousDuPcAI2f1y5PXkWSEq2l9Pc/3Us8sZRhCiJuc3Lm5YJfqNQ+b1ZbqhYW56n0VCCLWGt36CINgu1kjAw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: With the DMA bouncing of unaligned kmalloc() buffers now in place, enable it for arm64 to allow the kmalloc-{8,16,32,48,96} caches. In addition, always create the swiotlb buffer even when the end of RAM is within the 32-bit physical address range (the swiotlb buffer can still be disabled on the kernel command line). Signed-off-by: Catalin Marinas Cc: Will Deacon Tested-by: Isaac J. Manjarres --- arch/arm64/Kconfig | 1 + arch/arm64/mm/init.c | 7 ++++++- 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index b1201d25a8a4..af42871431c0 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -120,6 +120,7 @@ config ARM64 select CRC32 select DCACHE_WORD_ACCESS select DYNAMIC_FTRACE if FUNCTION_TRACER + select DMA_BOUNCE_UNALIGNED_KMALLOC select DMA_DIRECT_REMAP select EDAC_SUPPORT select FRAME_POINTER diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c index 66e70ca47680..3ac2e9d79ce4 100644 --- a/arch/arm64/mm/init.c +++ b/arch/arm64/mm/init.c @@ -442,7 +442,12 @@ void __init bootmem_init(void) */ void __init mem_init(void) { - swiotlb_init(max_pfn > PFN_DOWN(arm64_dma_phys_limit), SWIOTLB_VERBOSE); + bool swiotlb = max_pfn > PFN_DOWN(arm64_dma_phys_limit); + + if (IS_ENABLED(CONFIG_DMA_BOUNCE_UNALIGNED_KMALLOC)) + swiotlb = true; + + swiotlb_init(swiotlb, SWIOTLB_VERBOSE); /* this will put all unused low memory onto the freelists */ memblock_free_all();