From patchwork Wed May 24 17:18:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13254340 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 456A6C77B73 for ; Wed, 24 May 2023 17:19:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A5783900004; Wed, 24 May 2023 13:19:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A07A5900002; Wed, 24 May 2023 13:19:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8CF65900004; Wed, 24 May 2023 13:19:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 7AA36900002 for ; Wed, 24 May 2023 13:19:19 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 58030408DD for ; Wed, 24 May 2023 17:19:19 +0000 (UTC) X-FDA: 80825809638.05.BA04D1D Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf26.hostedemail.com (Postfix) with ESMTP id 73889140023 for ; Wed, 24 May 2023 17:19:17 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf26.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684948757; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yoW/ReeYfZSTzC2TZN9dmK/8gq9+nyhtUUc+/CgA4gM=; b=2aa/9rtMYPXWU8npyxUTIzOc/VhPsnEGaH/Vu3K643eyL5ruNjc/nM3RYWjZjclYKYpnAY mpPfQV3N9KTv2xQW3e3XWtkH6e/Vy5yhEVMhWSwj4JV5jhNcgcVbZq4FzRpFAgIXsYKY3k uuGawUKfIzIIgd1qhFV6eMp+i6sD5TI= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf26.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684948757; a=rsa-sha256; cv=none; b=cAfrsqMiTQP2nMJLHI2ToPX7olXXJqFUljFUkzyhJ4X6hyhyJjnAdDWf9j7sMG0XdlnxBl RmP4nPm0RN918WFdywr9CFDHmCXT9zDA5j2tuLNHthJ1LYtdUxLJiB7d+F+bjClgQ9yHjX x4iXOk3qisp3L9VxxSvTWTtwwt7bNR8= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id BBED363E21; Wed, 24 May 2023 17:19:14 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8063AC4339C; Wed, 24 May 2023 17:19:10 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Christoph Hellwig , Robin Murphy Cc: Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 01/15] mm/slab: Decouple ARCH_KMALLOC_MINALIGN from ARCH_DMA_MINALIGN Date: Wed, 24 May 2023 18:18:50 +0100 Message-Id: <20230524171904.3967031-2-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230524171904.3967031-1-catalin.marinas@arm.com> References: <20230524171904.3967031-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: jb311octpxnxsfea5at8rapcztza51pt X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 73889140023 X-HE-Tag: 1684948757-755798 X-HE-Meta: U2FsdGVkX1+fBX0Yg1bP3hYbzsXtkD9kDND7lYXPtXnDvGM4cgXz9n6yFOeiPPbcUpCZ6L4ndc9+AJ0Xw7QiWqHYQwjIEbf+VbDP3Afkt29XXz6HpJCjgpgY56MUqgd/cD7i9q/jLkoKDHNxzQPt87JCg1oJE44/pP7de9DFn/5r7tmAagwkwh4VkfGJbQkB3RQulesIMsZpRrEA8VhmfFHWbNPNig55njmFqc0gMOlFXc94p8ZnbjKtsuyWsiEkAH32qOoMZK9G4obGxIpu40yt7z6RKDpM0C7xW2tckeKf7mWHPon5p8qDlQVhSukomcz6w7OcmmMf9TplaXxkD9nDiXHanL20yLHoHPUBvdYnJ2h/5hggFIAdV+4wzaWzDhcR+LhHB3kP3C9I2V1fLnRonTutgMOZG0L7LEk9WmFQvvX/SZB6tyCV5IozV/HKflkDGO3GpKfR/BP361as5303pqYAckZL+c3bQ2dzpFOTcCBdSD3lC3ON16Q9AAIZR2hTwRxiANmVVSQzQ7L+xL5asTw9TNMK2zdPism1wuyR5IGWxyh9QzOgwJqsrscb3dxzOZXfvVEvBLLLqSlaHxfWhajYkcBKNv6KHS6an+wJEuJhnx/xYIHrddnUtLMjWYcBS543LBNiVy5VtESkVJHBEbDvMrdRpHyez/iW6/kaln403ABjizdxMk4HOlHULL9eMPOttuVgFVtbmgcgBKxe2ptcSAev9dpnHEb7LbPQNlxRzbyNnz0A6IWEODZOMS7coUY7M3Afs8DiXZ6Xf60qaLjhLlKMTRrlMRv+/h2BHzbKtG/Bb1f24uHvpmdhYEEfCWte2nvOgCB6KWheG/3nIk4FtafilHZ1FNc1MLx1ONRz1gdmBUHKgU74FvG4CW+l25lvPLFiBFhyE73WHpd4bbCUsczSfJuor0oDNS5KXH02OIBR2olRhnTtoFeA2rL6iu5zBz8+Q3NFkcI mHQUoeqz IS/3Y9wsabsSRPx0ZBgwmZsQxnDosYUok5nYS0FXlon3fCfHkTVk8pOwky5ycu6IHkNZJjqAQ3K87EgTxX1HM1wEt6y4ckTYBOTGAtfsT5ylexubcGJwR6OzUFt+ZpAT2AM0T5ZX0LdnqD3fo4QMaOucVulbD6d64AcVnIgvjMn26QlB82rdIuk1Dn2TnQp9ykb4kE5EaA3vWQRWetYQB1s/oCFPMTMSfHgOB8Rrs0N61yfXvgEUQZknLJD12+oUL179hakyW7oAM6axICdEJ9TgEv3z7J2W/7TFzZ9hb5EnJoLjbspaWcLIigeymeHparIdwbO+EywJTFWLFZ4pXijz7nSfV8r+a69SvUtIN/9xT+3KS6ccj8MEeKHkRKL9pJJAYi0WPuaq/4WKBJsMIFqLy3VNWSzVyXD3T7vVzd/5ZOtc6Dw9XJESHR6thBFvhHv/r33c1ieXn5tXkdZlQfj1WMHWOCRzj9d/7WlzRPFG4CMXZD+7Ot33gBS2q68WYPxcpBroKjCmMiWjJ6IRnXAPg/mdVCNOPgCyTDaAUOfcGJ7RTqIAWVGyQ4DRMWeHEcQ23 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: In preparation for supporting a kmalloc() minimum alignment smaller than the arch DMA alignment, decouple the two definitions. This requires that either the kmalloc() caches are aligned to a (run-time) cache-line size or the DMA API bounces unaligned kmalloc() allocations. Subsequent patches will implement both options. After this patch, ARCH_DMA_MINALIGN is expected to be used in static alignment annotations and defined by an architecture to be the maximum alignment for all supported configurations/SoCs in a single Image. Architectures opting in to a smaller ARCH_KMALLOC_MINALIGN will need to define its value in the arch headers. Since ARCH_DMA_MINALIGN is now always defined, adjust the #ifdef in dma_get_cache_alignment() so that there is no change for architectures not requiring a minimum DMA alignment. Signed-off-by: Catalin Marinas Cc: Andrew Morton Cc: Christoph Hellwig Cc: Robin Murphy --- include/linux/dma-mapping.h | 2 +- include/linux/slab.h | 14 +++++++++++--- 2 files changed, 12 insertions(+), 4 deletions(-) diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h index 0ee20b764000..3288a1339271 100644 --- a/include/linux/dma-mapping.h +++ b/include/linux/dma-mapping.h @@ -545,7 +545,7 @@ static inline int dma_set_min_align_mask(struct device *dev, static inline int dma_get_cache_alignment(void) { -#ifdef ARCH_DMA_MINALIGN +#ifdef ARCH_HAS_DMA_MINALIGN return ARCH_DMA_MINALIGN; #endif return 1; diff --git a/include/linux/slab.h b/include/linux/slab.h index 6b3e155b70bf..50dcf9cfbf62 100644 --- a/include/linux/slab.h +++ b/include/linux/slab.h @@ -235,12 +235,20 @@ void kmem_dump_obj(void *object); * alignment larger than the alignment of a 64-bit integer. * Setting ARCH_DMA_MINALIGN in arch headers allows that. */ -#if defined(ARCH_DMA_MINALIGN) && ARCH_DMA_MINALIGN > 8 +#ifdef ARCH_DMA_MINALIGN +#define ARCH_HAS_DMA_MINALIGN +#if ARCH_DMA_MINALIGN > 8 && !defined(ARCH_KMALLOC_MINALIGN) #define ARCH_KMALLOC_MINALIGN ARCH_DMA_MINALIGN -#define KMALLOC_MIN_SIZE ARCH_DMA_MINALIGN -#define KMALLOC_SHIFT_LOW ilog2(ARCH_DMA_MINALIGN) +#endif #else +#define ARCH_DMA_MINALIGN __alignof__(unsigned long long) +#endif + +#ifndef ARCH_KMALLOC_MINALIGN #define ARCH_KMALLOC_MINALIGN __alignof__(unsigned long long) +#elif ARCH_KMALLOC_MINALIGN > 8 +#define KMALLOC_MIN_SIZE ARCH_KMALLOC_MINALIGN +#define KMALLOC_SHIFT_LOW ilog2(KMALLOC_MIN_SIZE) #endif /* From patchwork Wed May 24 17:18:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13254341 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B8541C7EE2D for ; Wed, 24 May 2023 17:19:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 15603900005; Wed, 24 May 2023 13:19:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 108E4900002; Wed, 24 May 2023 13:19:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F1123900005; Wed, 24 May 2023 13:19:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id D9005900002 for ; Wed, 24 May 2023 13:19:21 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 6F576AE4DC for ; Wed, 24 May 2023 17:19:21 +0000 (UTC) X-FDA: 80825809722.13.BF4B5AA Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf04.hostedemail.com (Postfix) with ESMTP id C0AB140015 for ; Wed, 24 May 2023 17:19:19 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf04.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684948759; a=rsa-sha256; cv=none; b=pI4bMmcbFb6N8UrWcTDforq3TOnm/sOuDfxxpEwCkUtq8ma6IheIKconqTy8b8KJ7AlyWE rFH7koRs51A88t+nqiJg8esVdhL888pZvVipcOHq0EdpKs5KbBxZ6asrF6uHTz7EomZRz6 rKJKtcXdPIQgcR5ZuGPSO9GfXDTu1/8= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf04.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684948759; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=P1eswU29vM0nF+2OrASKsW9QuG/ezSkB9CLotuLXI1Q=; b=lslq3d3maMi1B1k1o3gHV7K/Zp0yDMV7/hCSsuKvEzgfXKafbEd1Zv0fAZrM+cv3Ztod6K fdvhJqOt1AVAUs5Lzc5Rg4ilV/IQfQQhOwWlwmuPZpabZCdZneP7/OTUH0XYeZuYFkDfPv 1gxEArpqiOYDfqzK+5uYokqnxgRbc1s= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id D6553638D6; Wed, 24 May 2023 17:19:18 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 932BDC433EF; Wed, 24 May 2023 17:19:14 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Christoph Hellwig , Robin Murphy Cc: Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 02/15] dma: Allow dma_get_cache_alignment() to be overridden by the arch code Date: Wed, 24 May 2023 18:18:51 +0100 Message-Id: <20230524171904.3967031-3-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230524171904.3967031-1-catalin.marinas@arm.com> References: <20230524171904.3967031-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: C0AB140015 X-Stat-Signature: 1hp5fgricktjrmobt4a3z5d3bj18ks4e X-Rspam-User: X-HE-Tag: 1684948759-692635 X-HE-Meta: U2FsdGVkX18SQlUwPUc+zJJTuvrhY2nJhEwK610FnyMtxJ6FbRDElYhwpfEFK8GCEEXh1/SKj/ifiP7nKNeZOkV9tlXrCdR/IuOyVKpd+txlcM0FLHgoD7//aThBDgarK927IycBLS9u3M7D78Mj8XnuT3kF3XSLHeVBiM+M7pK1+mHxUfDcs3vhQdjvqXEwF9dYnvJFcHgk0GRaKeumvh6fNNXli6zX9LR3LKnTEuY7eNLRykjbrTIiEh7PAnEGtg/7S1kD22t/JusTnTPzRP0m+o7oXedQb5g4dG53EBib/kOqWiw32wwpcLKWmnz5PtgRaiTacAYF7yWI3LFyd62eUM+rO+woOoEmbxV+PwUgMY83VBmKRNOJ6CWu9snqoTUMn3Omk8A50a24EFPbO3hUyVV8Bnw3l7KocSYgUCazyaGtBZeGA5yNUOQIm2e4/JzLjF768i1YHMrIv0gi/FHSqok1XUbKCCL/BjlEX/As0oKGwrV5eZK8S8hg20lj2lqGVvQPn8mtbW0PZa+6lJ7zsztXHqse4mbibiafTfoGV/8fq2wSw8QhfnjC8LVBL1YeND69tthLLwUFgG4p35+iQxJ+ssr8G5++ng8jJbmAK2BKd29m3/7JuLpdfkVHSYhc+ibGdOFeMKRV9Kc2oozFpyvz7mJ3CkrksNJPgu5kvWLC9e9Uybb/30PobteLiJ4PKE0Cz6NP5pVNt6NCMB9eY3Rs8b2NmfV9pbjasnDbUSOWOsuXErIs07Js85DodEsmzXayfH0u/tzwR2P7CSUa+vkamwKNqwMbyPd8YM2uhx1ivjI7EHShpmY+SNotHZ+Y6u9aND36Y33ajmFkCuD4T4xqq4EgYZBgAZqOdJRJ2JYn1S93VoyPusHcpiX8IZC0ce2zBleDz1eUl6N5FE1WMVU4sFKXjNReLuLoHbILQ1QGmvMxoycR4J7wtSstGr4JxHz+tsM837LtnK3 kjlzrQig KuS9sISsr+Hljc+KyBk6YA5kZ97i4MuozpT94Lwl9kM653gWj71N+WkfPv+JIHr7A3lr6PlYijXwGk/ifuFVKrvE0Dz+m8wwKSJep5CUpzQJ8pbl2iXBDRQt35/Hl9kWARrTvjTzyPUlDkBIhRYJdQJzYGdNz/P5et29h8lkq3BjmXrKMDkYWfzy8mtt0yN8L2dEZUAV04zcNYLA0ANPDbaAqqcxDHHniass2GMCWVjoB/FXPCzwtTLqktAB3NgSawfZ+fKve43P+qmMmsxlzLppjZgg0k7Tm+AkoMVCh7wle8/Hm3KVpiRlS/jklMyfCQ2Ezj+giPxGfyrn4yQEsmASD57D3J/MF4S7MVWA56V75ztMKIuATaFkkGvGuvXuVd9Io/rVyWHYBw2ALNBurNgXbUf3Bt/fs6oCtTZMr2/od6CwTAPP95tiMiEYgEhKT/+hbhYArMIR6+hi2Pj3hhmfLFoGT/BOO6bagWzEKH+aByFi/Zz8c4mRzeeOiovpSwsJY+8PlWE2jIS9/QD09bko117Lax1EQ9cJ6BTvtBgJi1Z9tC2DxDuhzdha/hcnfFuSk X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On arm64, ARCH_DMA_MINALIGN is larger than most cache line size configurations deployed. Allow an architecture to override dma_get_cache_alignment() in order to return a run-time probed value (e.g. cache_line_size()). Signed-off-by: Catalin Marinas Cc: Christoph Hellwig Cc: Robin Murphy Cc: Will Deacon Reviewed-by: Christoph Hellwig --- include/linux/dma-mapping.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h index 3288a1339271..c41019289223 100644 --- a/include/linux/dma-mapping.h +++ b/include/linux/dma-mapping.h @@ -543,6 +543,7 @@ static inline int dma_set_min_align_mask(struct device *dev, return 0; } +#ifndef dma_get_cache_alignment static inline int dma_get_cache_alignment(void) { #ifdef ARCH_HAS_DMA_MINALIGN @@ -550,6 +551,7 @@ static inline int dma_get_cache_alignment(void) #endif return 1; } +#endif static inline void *dmam_alloc_coherent(struct device *dev, size_t size, dma_addr_t *dma_handle, gfp_t gfp) From patchwork Wed May 24 17:18:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13254342 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2F059C77B73 for ; Wed, 24 May 2023 17:19:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C2B4D900006; Wed, 24 May 2023 13:19:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BDCC2900002; Wed, 24 May 2023 13:19:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AA448900006; Wed, 24 May 2023 13:19:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 9C5D9900002 for ; Wed, 24 May 2023 13:19:27 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 4C7D11C7862 for ; Wed, 24 May 2023 17:19:27 +0000 (UTC) X-FDA: 80825809974.08.3EB1F6D Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf17.hostedemail.com (Postfix) with ESMTP id 796B240022 for ; Wed, 24 May 2023 17:19:25 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=none; spf=pass (imf17.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684948765; a=rsa-sha256; cv=none; b=uHXZ5ZmlH5Wynqmlie3D4kXCnf3svAA234sMMpCdrjhvhAjCA+EPS3OHpZ5K5lx/WssS3/ rt6Zdf7Q2TWNCJAj22QZ7I9xV7bciiVQbdOykURkZ8Mer0F7bxZGvTfa9nuqfskc6+e5/r xRlTxtEGANjcAOCk1CbmIgQ3N6VY24M= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=none; spf=pass (imf17.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684948765; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=35xPKuIoBzBBtPc7oLgirz1u4PdrAIKlSY99QdYwnw8=; b=PuiOaOhWjsTfmEJ1Zh5PyIRurfoSE5Z2UQWmOnFmFO0IM+GvyGrFqYcWrp+ljRqgQMeZKv sqjgo17zNvOedw5PBEXZZuFnLLCvdOhpQTFaWBzdiH45KdlcnoL8lWfiNekrnzoo9MOcLr w+gy3NvhyHkMmkjUq8dkS60J6bDXO3A= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id E6D3E63F94; Wed, 24 May 2023 17:19:22 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A7145C4339B; Wed, 24 May 2023 17:19:18 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Christoph Hellwig , Robin Murphy Cc: Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 03/15] mm/slab: Simplify create_kmalloc_cache() args and make it static Date: Wed, 24 May 2023 18:18:52 +0100 Message-Id: <20230524171904.3967031-4-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230524171904.3967031-1-catalin.marinas@arm.com> References: <20230524171904.3967031-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 796B240022 X-Stat-Signature: bkibafccnqbhu1oizp395ycj4gfxfz4u X-HE-Tag: 1684948765-743468 X-HE-Meta: U2FsdGVkX1/B3O59GHtwLav8SWVxks3zVsD2N+18EdikBJgxfacuGN9ATe18Agd6odWnXG3XJF0R7tJeAnN2kwnDwTZXZkbPuA5QHUUWq2/C0DVcqrcS84OCTdgqUeWR49wJHv24kbZNKMolqncjX5bLdZ3OARmcUaoaCNmQYSqpcAVtMxAk3rgWFXW738km43Cii6lL66MGKbeDs/rL1l4af/9BPaSiT+B2HbS4/S4Sg+2hPx6BRGQXuJkeevoO6CIwvcqiTRZ8uVCB8lPWckP130kUStLPd8v3qSfXCmKFpCjhixR+RZejHyfH2SuytpvkWzJf2Kgx9UFmQq3wUKCsCC2lopT9jZKls/mf+h0nPJGXQBZNogSY8lhPms9dOZZ1cgLq4a7htcVvFlAFGL1vhfa5zz22oAWeTvr5u7YT8tlm2f+G8dtVlOEBn9L0zyqknbgoKnx7bvc+Tu1gMpD6F/75uqAi/uVzgQ0m31zxWLNZie0EaOXIrzW4Dql7QCKMDyMUrHm/1pTM+llyVdQfFe0I51RPkSDXHpgs17c/UEc7/z5+rgookeLiArciSr5MifVcsHaFB31tzLg/Ib2pAvasdupjxGXbiWXG8hTKZqakYBK4qHgzFPcZZzeItLaQF4Ep8BNeF2H23A5dzV6bMSZZl+AaY49NnsZLaCjoCyzqLHcE+GUw4Ma7Sg6Tz1ERy5M7GrfYOYxvtVN/LcQMxhHQZolxZg+Pk4Te57oOBzQ0RU1hWPltz5n23ugZrcAG/xjcPrE7RoBdRATg2TpnyikLG7jihuG9oZ1sygBwLpYyfsdPG+ZAq9Rzh5EPa653M/tceL0HdpGuIOM3tBRE2lLc+WwDMUhRIfk26kasdYaG6PYN4gjmh67WKTfNyOyPLr68SU+oy0G/r72oXSljDZkVhbESMLo07oRAIBAjwevgJsHS6phYokDVat7uGWflcSWWqVUhdipe0HZ pzRL6ehM O4pw2Vu68mb2x2g5nFzW0hIbM40eViXo3bv5I9ZAkNdUpFYaQl/DrNvkrKJq6y5FCpUiRIG8V6fEJ1S0uVMJzH/eQPMBAzIFg0C4OFaSwAvrfq50xEbO4ouWTiKzMfFcjFzb2zMycm5l+TGsSRgYaQOow/R+Pdmqrh/P96F20CzeycYsq9kcQCBUOjJvRgfWuAfXXBDthiWcZpQ8Io/OosGfmFXZFBKKCtJmKtw1ZuKKvCVHl2utLQiPMc2H+r9hRTDpBhv2K4Y1DVSsGlVdZ5DGF+9yB5iO356L1JhyqzcZCgDX4FQ6RMZLz89sXLQMdsYiW8g62Er3JpvrV9kFvEY6OaGckjn8WRosq2WGZ80ltic5jH3LxCe4tsg6q9mkHHxYL3wSZlZRJlNDmkQPQwZ6GwLJTbodi6liuK12cgkkWPYPDqpgw7EAr01PAyRYuaEsfbw12wpvDFxN8Hc/71JOEhD0G3w9PMfPGKlmPZ1CQNgxOazCxVBQmbpfCioCfOKRTU4U79UgsUFXx5eEvGbc/+/+wMPmoQQdov3FmhbRn0frUBT9/CDDBItPCokatVN9d X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: In the slab variant of kmem_cache_init(), call new_kmalloc_cache() instead of initialising the kmalloc_caches array directly. With this, create_kmalloc_cache() is now only called from new_kmalloc_cache() in the same file, so make it static. In addition, the useroffset argument is always 0 while usersize is the same as size. Remove them. Signed-off-by: Catalin Marinas Cc: Andrew Morton --- mm/slab.c | 6 +----- mm/slab.h | 5 ++--- mm/slab_common.c | 14 ++++++-------- 3 files changed, 9 insertions(+), 16 deletions(-) diff --git a/mm/slab.c b/mm/slab.c index bb57f7fdbae1..b7817dcba63e 100644 --- a/mm/slab.c +++ b/mm/slab.c @@ -1240,11 +1240,7 @@ void __init kmem_cache_init(void) * Initialize the caches that provide memory for the kmem_cache_node * structures first. Without this, further allocations will bug. */ - kmalloc_caches[KMALLOC_NORMAL][INDEX_NODE] = create_kmalloc_cache( - kmalloc_info[INDEX_NODE].name[KMALLOC_NORMAL], - kmalloc_info[INDEX_NODE].size, - ARCH_KMALLOC_FLAGS, 0, - kmalloc_info[INDEX_NODE].size); + new_kmalloc_cache(INDEX_NODE, KMALLOC_NORMAL, ARCH_KMALLOC_FLAGS); slab_state = PARTIAL_NODE; setup_kmalloc_cache_index_table(); diff --git a/mm/slab.h b/mm/slab.h index f01ac256a8f5..592590fcddae 100644 --- a/mm/slab.h +++ b/mm/slab.h @@ -255,9 +255,8 @@ gfp_t kmalloc_fix_flags(gfp_t flags); /* Functions provided by the slab allocators */ int __kmem_cache_create(struct kmem_cache *, slab_flags_t flags); -struct kmem_cache *create_kmalloc_cache(const char *name, unsigned int size, - slab_flags_t flags, unsigned int useroffset, - unsigned int usersize); +void __init new_kmalloc_cache(int idx, enum kmalloc_cache_type type, + slab_flags_t flags); extern void create_boot_cache(struct kmem_cache *, const char *name, unsigned int size, slab_flags_t flags, unsigned int useroffset, unsigned int usersize); diff --git a/mm/slab_common.c b/mm/slab_common.c index 607249785c07..7f069159aee2 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -658,17 +658,16 @@ void __init create_boot_cache(struct kmem_cache *s, const char *name, s->refcount = -1; /* Exempt from merging for now */ } -struct kmem_cache *__init create_kmalloc_cache(const char *name, - unsigned int size, slab_flags_t flags, - unsigned int useroffset, unsigned int usersize) +static struct kmem_cache *__init create_kmalloc_cache(const char *name, + unsigned int size, + slab_flags_t flags) { struct kmem_cache *s = kmem_cache_zalloc(kmem_cache, GFP_NOWAIT); if (!s) panic("Out of memory when creating slab %s\n", name); - create_boot_cache(s, name, size, flags | SLAB_KMALLOC, useroffset, - usersize); + create_boot_cache(s, name, size, flags | SLAB_KMALLOC, 0, size); list_add(&s->list, &slab_caches); s->refcount = 1; return s; @@ -863,7 +862,7 @@ void __init setup_kmalloc_cache_index_table(void) } } -static void __init +void __init new_kmalloc_cache(int idx, enum kmalloc_cache_type type, slab_flags_t flags) { if ((KMALLOC_RECLAIM != KMALLOC_NORMAL) && (type == KMALLOC_RECLAIM)) { @@ -880,8 +879,7 @@ new_kmalloc_cache(int idx, enum kmalloc_cache_type type, slab_flags_t flags) kmalloc_caches[type][idx] = create_kmalloc_cache( kmalloc_info[idx].name[type], - kmalloc_info[idx].size, flags, 0, - kmalloc_info[idx].size); + kmalloc_info[idx].size, flags); /* * If CONFIG_MEMCG_KMEM is enabled, disable cache merging for From patchwork Wed May 24 17:18:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13254343 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 818A6C77B7C for ; Wed, 24 May 2023 17:19:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E4FA7900007; Wed, 24 May 2023 13:19:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DD93F900002; Wed, 24 May 2023 13:19:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CC683900007; Wed, 24 May 2023 13:19:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id BD750900002 for ; Wed, 24 May 2023 13:19:29 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 981321C787B for ; Wed, 24 May 2023 17:19:29 +0000 (UTC) X-FDA: 80825810058.13.89349F0 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf18.hostedemail.com (Postfix) with ESMTP id E60AE1C0007 for ; Wed, 24 May 2023 17:19:27 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=none; spf=pass (imf18.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684948768; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0mqLCBdpgkfqDUcqHwwIym+BakJfQ9CLI5FzmbFJWTk=; b=XqcrOvRYFL9GWUgT+fHxRQyCHv38d6YIBVd6K+dALsFEKywjAKPnwCv1KnHIvp5oplRy2N Ftl0rGPzRhsfx9xW1ZgTxbsN/DXQaODN8XCbqB2Jnkmvnwy5f/tIlMefBVowLoYB2JVcni 5ISRUEFlPUbkmjeg4MEIdv9qAWWFqiE= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684948768; a=rsa-sha256; cv=none; b=QyiZxa+pesF22U/RAsi9ji7AdoqeEdpqjqpiIMWJocwWiCsxyOTPHnV43u+yhvpjz7xBj/ tOat6ntO99paz9WXAmpkn2IcpbYOXwkxPY/zfIQLvHK5KAf/2MLkXA8NFFHfPw0EjoOZqy WUvff354UwlsCT0tWFl7CYK4EnKcmyg= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=none; spf=pass (imf18.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 0CBFB638D6; Wed, 24 May 2023 17:19:27 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id BAAE6C433EF; Wed, 24 May 2023 17:19:22 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Christoph Hellwig , Robin Murphy Cc: Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 04/15] mm/slab: Limit kmalloc() minimum alignment to dma_get_cache_alignment() Date: Wed, 24 May 2023 18:18:53 +0100 Message-Id: <20230524171904.3967031-5-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230524171904.3967031-1-catalin.marinas@arm.com> References: <20230524171904.3967031-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: E60AE1C0007 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: g7gqm885kr9zyh4p7r8t4yrxn99e3eez X-HE-Tag: 1684948767-795912 X-HE-Meta: U2FsdGVkX1/xftYLrbeauha6Ii5lsk5V3hFQEsfJJnC89JGLFty4kk9qWXJRTfH8xKMF9aKwUuCI6T7Y4Uy+MqW+kIXrPlIEQGnlf4O7PPylkjboJ12KJEUfkA6XsYovjWdKBToyMTR8Jsv6z3kcIroSlgXPDBpSev9vWVj3rugKe/INs5s4U2Y6Hb/a5IJEWjzfsWKOikFZnk/wiP0StHHZP5LZJ1S/1x78yKsCuXQriOajNT/YaW5U7FVThiGAEmqp7W5OwnRpA781R48F+yUuwMEdC7XE8FJ8ycQWx5f+5N457B7IArMbmY5BYL5kpafzrVzE6gCDFK9bMTLLCWKCyV0ubC4Z2RKS2p+yiKpouXx+K2vnvxQduidq2jOuJ7zs5tBQttZAVAz0gEe4Sqev1rA8JfZfmDbXQthExIxSyIuzcgz2ide05THC1yRxMf3kPRyNCygNTW9rvCg3mro677H02lajdTwJVekiKhIyHHfHE8wBD6Ey1r9DtZf6RnErfzrwqrb9uvgSC8Ns7s5jU5Jpf83MXiasLBYkf/wGRE8tHXnglhioED7QrjE5daYfQM/FtDrtZXMss2zBVABJGTWzD33YmS/SgeK67YBtA4sc9dNO4T443KZ/auIQ8is9afdqgW2G44zIFzn1YHcaDPRXa0sauHBL4PNZXo9bAQXoOZcjgxE6s2wpYendXnm7Ho78VFHK+q7i/BWoQnrdaD7354vT+vfichPRdUxgSdlEgXoEykV0vxlR4LV28id6dw2zSUsFZ6V1YaBQ0mPB2K4bQAaaAy3luo4dBS7HnuPPfArzJYdZRn4lSQODu7SsKSqNZKlLQQjFxsm69+mnPfwzhzm0tIUh90mG4h4uFQ2H8hnh13n/D6XNd/yb/tTkQ8wMjIep3/GuY/WxLYADF24S3Rm4U8OYFM+GSBpnxe+HFV0AQLowxSjhRoEsR4QWfX4ZIFaauYJNI5h cr7s/OuD bQzDKmdpcdKi+DBbB1IQATemFzj3HLXhxSGRQ4VDJX7UwfcK1xJ48WPsH+LxDSkO+jRQ6BVB7t9DSXyDTPSpPNul9wqrJ16bWTs2IgRHuBQ6yImq6XlU3Vn7yX4j0dBzvdsoAPVIZYNn9GsYMZZYnqgg8nsJmLxLgD/2tzsRxwt0R6aHvxPQdVXwbDPvCUJ1TD9iKv8Z0lN5RtwMmJqFcpe2fVMBDo/sHp/ijam0LIkkFJDd3piG6tbe4M2RJ+MxVSOT5pN/MgCbkmTrBoIMPEYmkuhHhLMD+QSwLImRISczPajR4kq1bUL+A4AOjn/t0Iya9ekPzEs7unAwpn7DN75G2OSKF+CTBxV0VO8DRXAlqSJ0TrBO0K4nh9+VSw7x+pbXghW+Gc+1x/O/Q6yGW0s7dDekOHCyc3a0niXw48793MM1TIOkYwgYsHMbEOkAGTqLgIZP99RWxY728hfA0T3UtxVVIIEuEgFLakHQ7SmkO8YRYx7LHC5SftU8FRNkka3wT1IKHbjHUK+BiEMJfhMh8eRaB/AkY6wxDJZd+DZA++O6BANshneUpisWt3dqBBdzt X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Do not create kmalloc() caches which are not aligned to dma_get_cache_alignment(). There is no functional change since for current architectures defining ARCH_DMA_MINALIGN, ARCH_KMALLOC_MINALIGN equals ARCH_DMA_MINALIGN (and dma_get_cache_alignment()). On architectures without a specific ARCH_DMA_MINALIGN, dma_get_cache_alignment() is 1, so no change to the kmalloc() caches. If an architecture selects ARCH_HAS_DMA_CACHE_LINE_SIZE (introduced previously), the kmalloc() caches will be aligned to a cache line size. Signed-off-by: Catalin Marinas Cc: Andrew Morton Cc: Christoph Hellwig Cc: Robin Murphy --- mm/slab_common.c | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/mm/slab_common.c b/mm/slab_common.c index 7f069159aee2..7c6475847fdf 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -17,6 +17,7 @@ #include #include #include +#include #include #include #include @@ -862,9 +863,18 @@ void __init setup_kmalloc_cache_index_table(void) } } +static unsigned int __kmalloc_minalign(void) +{ + return dma_get_cache_alignment(); +} + void __init new_kmalloc_cache(int idx, enum kmalloc_cache_type type, slab_flags_t flags) { + unsigned int minalign = __kmalloc_minalign(); + unsigned int aligned_size = kmalloc_info[idx].size; + int aligned_idx = idx; + if ((KMALLOC_RECLAIM != KMALLOC_NORMAL) && (type == KMALLOC_RECLAIM)) { flags |= SLAB_RECLAIM_ACCOUNT; } else if (IS_ENABLED(CONFIG_MEMCG_KMEM) && (type == KMALLOC_CGROUP)) { @@ -877,9 +887,17 @@ new_kmalloc_cache(int idx, enum kmalloc_cache_type type, slab_flags_t flags) flags |= SLAB_CACHE_DMA; } - kmalloc_caches[type][idx] = create_kmalloc_cache( - kmalloc_info[idx].name[type], - kmalloc_info[idx].size, flags); + if (minalign > ARCH_KMALLOC_MINALIGN) { + aligned_size = ALIGN(aligned_size, minalign); + aligned_idx = __kmalloc_index(aligned_size, false); + } + + if (!kmalloc_caches[type][aligned_idx]) + kmalloc_caches[type][aligned_idx] = create_kmalloc_cache( + kmalloc_info[aligned_idx].name[type], + aligned_size, flags); + if (idx != aligned_idx) + kmalloc_caches[type][idx] = kmalloc_caches[type][aligned_idx]; /* * If CONFIG_MEMCG_KMEM is enabled, disable cache merging for From patchwork Wed May 24 17:18:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13254344 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A8BA4C77B7C for ; Wed, 24 May 2023 17:19:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2DDFA900008; Wed, 24 May 2023 13:19:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2BDC0900002; Wed, 24 May 2023 13:19:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1A2F6900008; Wed, 24 May 2023 13:19:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 0D7F1900002 for ; Wed, 24 May 2023 13:19:35 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id B421E40341 for ; Wed, 24 May 2023 17:19:34 +0000 (UTC) X-FDA: 80825810268.21.323B50A Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf07.hostedemail.com (Postfix) with ESMTP id 163634001E for ; Wed, 24 May 2023 17:19:31 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf07.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684948772; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=X9GvMJYtQNhC/CV+bvo1nqxSzdMjEtDAvkUK7+TAINo=; b=3G9qhmsXTuUE8ceOxgzd2yF2OT5/3GAzkBK6FizxnnFeijHLCcHRCA5Rw3DseCpOeZajs/ 8u2e10EUOkNe9cg6Nj3/EQTUt9m3DdCIKur4MWDCV5LPesrVx+5pdkUPue3/Zvgm7M0eBw X+eKXYJKqoQ7HQTJ7ZEMVG/RdSrxock= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf07.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684948772; a=rsa-sha256; cv=none; b=aGvZhUzqWLohOTcDCMrr6bxIK2wEPOnfePaqHJqawQIVcDti+9zAk4vYhQOexUbE5YM+B2 Mopa1jyGzx61NIXS7/FZPBQ4N8T17bkOYdZzGMfwMPfRPVKky+tVhnQb4Dpw0eO2fBR1P8 j0ESmUZxB5p7Tp6mSGRvDT05+1g1c08= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 21A0463E8C; Wed, 24 May 2023 17:19:31 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D422AC4339B; Wed, 24 May 2023 17:19:26 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Christoph Hellwig , Robin Murphy Cc: Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 05/15] drivers/base: Use ARCH_DMA_MINALIGN instead of ARCH_KMALLOC_MINALIGN Date: Wed, 24 May 2023 18:18:54 +0100 Message-Id: <20230524171904.3967031-6-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230524171904.3967031-1-catalin.marinas@arm.com> References: <20230524171904.3967031-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 163634001E X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: a7p8st9k4gt745a91xxqf4b496ykjq85 X-HE-Tag: 1684948771-414398 X-HE-Meta: U2FsdGVkX1/YVvPOZB0Wg9ENM84Ob03FoWYncaMDs3ARkpqauHqm7jcD40xqVHNaZqqg4BhSL1gjDTvZqkMWvbwuo/UI7RKzFHxvcJwl/g5XjRCnj1PyzYfT0sVysjBjzj5coZ+zKOT+yctFf/5AKhvB0D5M2Xmg3nQ9T49QrFqRwD7AIfEQEX+0xYlZ7iYCc0OpUa9tsY6R6OCabZddte0Frt/C8ABMDDb3hsPVJH2iMkpu0W3l5LQkJYM5qcOnEC3GLF09n+DYtZdPOK2Vl3+z9z3U6XDRDEHFwgcPmT9CzegMgWUl+f3F0elfL3PJ7vIJJJ/Yl7h2YdU2POJ3Q6ydyNkPicD4F/7qFvYBBTP5+yMP3ux/x1Wif/Vy3vHdpm853Bh5ZJ9NlDRqdPTkhUKc+y5Op7mRGtxkrFgfDMTja/nscO+UNrw3xA14jaB7ShP5WnL9y4LBWu6nG+pckD8SycFKK9B84cHFGM5Y2LdESnHmQ0aS9IiVqTpBpQ8p7jZsaBlwAmd0hRUF+XmHNQ3n6r/czTWJy+DNpMGNWDEoVDehXju6TOxFJXQmAy6JFkxRs8nBxd5BmWTf7F+6Qp0XzhrxNNAu/4DAqBqXQaRIosLKYsuCncjO1bJLXJNDJs9WzXhwyzkcIbHJ2hn6KeW3zAADoUw+OFrgovkBvN8MsUAKiktkBUMbiiDucnRPSIYhkNiOdklHq6n7g1UxXrCarw1tvwlfZH09hPBkdRHutuT8tLvp5Iyt1ZiTC4jkxBCoHsa9ae8zcG7cNz0jFc3gbOnY+8+uNcwBm282DJNQtSXzR76If4Mk3ionf61Z2NtIQWr3pKKB5zepQh3FJjdGuy19NB56vXYb+NuA8jwLLHz3eVuD4QNp5/LNH30CQ3U5ovGM/jtF1gPQrTVSFP8UvcVhomDvzMxK6+7jkS/XEHBxq4YcxwkRqL60MFG1RkuIgpw+Y4UhDDOOkWz YxEl4Qdw jWlgyUeWJNuGczb7Ui5oibLZY12mdQK5ThlKH6jXSfE6P3oNKM7NpF0KnHhZHPVgqb7IQXpVx8+yta+fiPSsgFNvSIIJL5cH+Pf3dI7Snfm/fH1Pwt3BjXaAy76GNLAUZ7WHP7qq64ZPdCmgzY/Gwel/bmMwJF0JSnXl05HT4hLn17GnbXQgTDmfrparBK3QWBtrkb97pBA7yeLB62/DmZkLic2BpAZijNPLb/o1oWFUE0oj8VNeG/oRHbpFn9uLeTrWkHd+N105PxoU5A3rs5i0znj/DhzAkDhWwkvmZptl4xj59cSDAFRjEr0HXkL3BycYPVrPzsTk/fV0WHfy40rq8EqTUSXhFV3t7iaqzu8jDr3trTFCKUeLmkBqBf8JE0mGa2iRMRZxB5LVmQWu+jNlpRd3P/wv+HVXr2K+UJdsqL0ziPtOWIchqc5ye3XY3h9IuL9Ui1tWBVb5l1zzqRT+L0/oS0PLo8r3rKO+8mzm8bod7IyAR2mZ8poPICfCTmbMFtVPeNIGkbLU1eJIZHIxyjGmp8IaI+BYDEoL+/IzxGyIhrjH+4a3U3iJFL6FI0TixsG9dneXhYlUBeRUeUTWubeg7OpgqMGpWTJUuR8QtLx9JvpA9AiLh4w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: ARCH_DMA_MINALIGN represents the minimum (static) alignment for safe DMA operations while ARCH_KMALLOC_MINALIGN is the minimum kmalloc() objects alignment. Signed-off-by: Catalin Marinas Acked-by: Greg Kroah-Hartman Cc: "Rafael J. Wysocki" --- drivers/base/devres.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/base/devres.c b/drivers/base/devres.c index 5c998cfac335..3df0025d12aa 100644 --- a/drivers/base/devres.c +++ b/drivers/base/devres.c @@ -29,10 +29,10 @@ struct devres { * Some archs want to perform DMA into kmalloc caches * and need a guaranteed alignment larger than * the alignment of a 64-bit integer. - * Thus we use ARCH_KMALLOC_MINALIGN here and get exactly the same - * buffer alignment as if it was allocated by plain kmalloc(). + * Thus we use ARCH_DMA_MINALIGN for data[] which will force the same + * alignment for struct devres when allocated by kmalloc(). */ - u8 __aligned(ARCH_KMALLOC_MINALIGN) data[]; + u8 __aligned(ARCH_DMA_MINALIGN) data[]; }; struct devres_group { From patchwork Wed May 24 17:18:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13254345 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8E968C77B73 for ; Wed, 24 May 2023 17:19:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2B62F900009; Wed, 24 May 2023 13:19:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 265BA900002; Wed, 24 May 2023 13:19:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 15539900009; Wed, 24 May 2023 13:19:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 08846900002 for ; Wed, 24 May 2023 13:19:39 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id D33DEC0978 for ; Wed, 24 May 2023 17:19:38 +0000 (UTC) X-FDA: 80825810436.08.843242C Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf15.hostedemail.com (Postfix) with ESMTP id 39067A0008 for ; Wed, 24 May 2023 17:19:36 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf15.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684948777; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=S59+slMdeGHVr9gGv1wpxhrhYrhnmUIhONDI6UrLqhg=; b=JvvhcG3BY8i/ieWa8ebsv0+pcI3fh9vj+PwtruLe9kDyQvB/XOhXGI1kQ7Ch/6LDYyn+6E AJzGgmnSS2sVAKwaPesSxzIkOomfj8ADh9WDdWFME7uvcCuBnsfVJnT4MLsS/d8YeaRU9o 17WV0EeVvTsiqPez+LMviZpi1Pu4vII= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf15.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684948777; a=rsa-sha256; cv=none; b=loz2c0sV/eNqnYi0C2m1KgS+RJ7Wwx4mrYXoAw67U+5iKDhk8ww6aOZsNFJCot/J7M7NWY qeHSfa4VFNhPqjb8weRQVtDf8HrlnMdiMwdsIsFIMTpp9zaTmds0ePUlGJ28qpzVN8OtWm mvSXUu8Rbi1xb9PO7PdonE2ucgMu1ig= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 3A7E163E28; Wed, 24 May 2023 17:19:35 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E6EE9C433A0; Wed, 24 May 2023 17:19:30 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Christoph Hellwig , Robin Murphy Cc: Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 06/15] drivers/gpu: Use ARCH_DMA_MINALIGN instead of ARCH_KMALLOC_MINALIGN Date: Wed, 24 May 2023 18:18:55 +0100 Message-Id: <20230524171904.3967031-7-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230524171904.3967031-1-catalin.marinas@arm.com> References: <20230524171904.3967031-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 39067A0008 X-Stat-Signature: hxfghc8zmxn7g9z3wtmicj7dufoampg5 X-HE-Tag: 1684948776-459884 X-HE-Meta: U2FsdGVkX1/YXh8NjpPfOwQE1vtv+l6FQGbb2j26X68x6A+FrqxGgZNv8yKRDSL+xiArYDhW1HFmkHUTvv9mYakf07lU9MbsKlmKO5YbXmeBVgE1wioAXnNpM/0ym3jwyqWUBpkREqyA6bx8quhJBptWC/CZU7h2RLJvdFUy8U6igqLdsWuKVEda8ygTQuy6auOZGYam9ed7juCYtnaP74qN7vdlK5UEngCwB3k8WhdfWCz+SLymD3ylRbwC2yqwAEJzLHFp2cu6vBjfXQBmXhO+ohna4t8Z3lxNuoFawaRtv8P8yMWjz9Aj3gxkCgdkab+UbnJcqvdEXifS89Vgu1dx2WOT5ta9yJgNan+N2EMxmOUzOvxJ/bE0HfPOa4viJkrW1S/yKr+hJ/xT6sXimgTpmw5Qvin/D4ywiBI4H8WCZsDmCGeLGfZbqXcBW3/ZyWmEtm9tJm4+Ekc+lVvCrw9BjXLU+WqVFoe0MJsF34MLHmRYqRg4L5ijJ+39SJep5UWMpgQ+fL5XHaiuNu4mPH8CrPQbaB1MLunXDdohEvb5Q1LWLFP1fyGJaaJe1rLc9K2l9Mkof7yOfX8xyXrXjOl4HH/ZPoX9RusYR3ArbPzxFeb6Wt4q6VCEy21aGWCACwB+ZMyilTMlCUYC2nF30dks6YOD5aC/4QM46hcIYu+72VKyA34j/F4p5MbHTCV/rz9C5gPnVvtexyKEz7Q8J8m1sI/cRpR28R1K3D0OfbonNSCpCGvaRz9eKjkjbshT4A5itv680OZ5EP7HG1GFjaFePJ2JKWDfwCs7lvtscn5zcuMof0OH/IITeh5VSGoyaOcqjE9EcyqS9rVEYEllyhaN8GLuea/f2FNEDFoW3K0t025sdvlXMSQUCkJLKC4PSaxVV6FdqnVclUwoMlSggp7TgpG1NP1ZHNClpEVgPMgkvDG6GxBVp6CDpMgNRm0Psr1NdqhYDgLo8I8WNnl lOqjZ69k N09tIw3b82QVv63MYdTRg9NvJBGR64HDvEG34uuxdlII0Dfv0+RagVFS+G3bHkANggLku8jjlbT0j5H8+Kme0L39vmMjKVRQyEk6ubc+G4FOd+Tvx0PFhp4CnESUO/8+VCQvMDY+GtUH/KHi6HtZKETp0vWwLvNHotKpeAinhglvF8HCuzl1wSVRMlfNLZiRCslYgyNt/q12n5m4UTpTHC8MXJ/HCZmHUw/MFfM4Smmpif8giNwPTb/G07KShCQBsmwyljqeskhYvNs5I5vsSJmQw+NQQenPflW4VrHGaJfU0C7Temve31yWkrvzXD+Ucut875r2yF/uf1VW+dad1rQTt1GRYUYNBFaAz0rnEflI1odKmo6lAcn+BhwcMRhVVj4vnj9fa1PTCTGkFhBEVFeGCUR+abSxzNn6JERu+Hmz1D0FhypDTKCR0WTLhrrWWl6YO0QwR8KHWgZjpNCaZqSjpNmYd0pQsDAzpS9qocQkIVOE//zBMYgiqBVjyTs0luXeh X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: ARCH_DMA_MINALIGN represents the minimum (static) alignment for safe DMA operations while ARCH_KMALLOC_MINALIGN is the minimum kmalloc() objects alignment. Signed-off-by: Catalin Marinas Cc: Daniel Vetter --- drivers/gpu/drm/drm_managed.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/gpu/drm/drm_managed.c b/drivers/gpu/drm/drm_managed.c index 4cf214de50c4..3a5802f60e65 100644 --- a/drivers/gpu/drm/drm_managed.c +++ b/drivers/gpu/drm/drm_managed.c @@ -49,10 +49,10 @@ struct drmres { * Some archs want to perform DMA into kmalloc caches * and need a guaranteed alignment larger than * the alignment of a 64-bit integer. - * Thus we use ARCH_KMALLOC_MINALIGN here and get exactly the same - * buffer alignment as if it was allocated by plain kmalloc(). + * Thus we use ARCH_DMA_MINALIGN for data[] which will force the same + * alignment for struct drmres when allocated by kmalloc(). */ - u8 __aligned(ARCH_KMALLOC_MINALIGN) data[]; + u8 __aligned(ARCH_DMA_MINALIGN) data[]; }; static void free_dr(struct drmres *dr) From patchwork Wed May 24 17:18:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13254346 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DBB39C77B7C for ; Wed, 24 May 2023 17:19:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7723A90000A; Wed, 24 May 2023 13:19:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7210F900002; Wed, 24 May 2023 13:19:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 60FD490000A; Wed, 24 May 2023 13:19:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 52812900002 for ; Wed, 24 May 2023 13:19:42 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 0001B120A02 for ; Wed, 24 May 2023 17:19:41 +0000 (UTC) X-FDA: 80825810604.11.AE0697C Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf11.hostedemail.com (Postfix) with ESMTP id 39C5B40005 for ; Wed, 24 May 2023 17:19:40 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf11.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684948780; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZmV86imfVMATseoJlCRYow7aoFQuclO2qwE8MVqrGZI=; b=OFntC3g+5HI/dxvE3WP/tx4UalrJ9XINHQN9zbowMvYis5W5dhWx0c/MjdiObl4V9+mMMe 3gHzN88ITC6ND+08Zt0znY1tWGjwOotUYtyvyzCrzabrXExxCo4TAbc+PXDqxXmtN4kv0V 4BvmdEO9rKiCWyFgrV5QjR7HfaNS7G8= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf11.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684948780; a=rsa-sha256; cv=none; b=1QoE0WKes4mCg+QfyOM/6EX28/PuNG2us5CYWh+xJ0Xebb26aN2et+nW9QD/OJgrb88gDE 6hN9m06deb07CBf3WX+wuTuZppaHvGD+Gd4Dw5YdLfMetyw3FTEK+gWU09GqzuZSw6jIqP owZJoo4eg5TGfL6+6NSnoe3w5QWXaPY= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 54378638B6; Wed, 24 May 2023 17:19:39 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0AFB4C433D2; Wed, 24 May 2023 17:19:34 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Christoph Hellwig , Robin Murphy Cc: Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 07/15] drivers/usb: Use ARCH_DMA_MINALIGN instead of ARCH_KMALLOC_MINALIGN Date: Wed, 24 May 2023 18:18:56 +0100 Message-Id: <20230524171904.3967031-8-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230524171904.3967031-1-catalin.marinas@arm.com> References: <20230524171904.3967031-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 39C5B40005 X-Stat-Signature: z6wki585shm5aj4rymja18grqhafux4g X-HE-Tag: 1684948780-735271 X-HE-Meta: U2FsdGVkX186i78peWxHQKvdxUrDuHCKXsh4dKTYmvX4VqCHfTVxOl2/tadf8z8cbPWHeIK8XhuOHkG7ynYOIVkUKOE9/8cELMh0vcyEQ0s19tG0V/mJSpBk5ay4vOj3WE5iAITHQsRm4XsWGPQgQPBE0StHfh+VCe0EJnICWMX+7X2KACLg1C6rYU/MifksMHqL1JoAr1clLJxHo/Rp0iX3LQ76e7NYM03Xng4YgNTjZsHRskzos1ECVjXMel87bKKtEFq5JkmKg1GDN19Dby7FZ6iaQYlm+YPvbWDMjFALElvQsy5MiAa8qJfaHXEEhZfc6tuMKE9Zph/G15ha2DsBVFkRUoi938TPk7hxPbq5H+55J95uejlSHMtCcunlvWiDo0UWDz+Pa9EFS3/L7TfH1Q6S5MTp0fwP/ZEnGG+nUW7BN/nj99PmqagiGl5VBuoC9x5o0348Ox3QPaVksK290NVqyGDIPdUHZ2HOYrUwx3Zs1/YIeTglHuPAZTu72+aIiEnbOF+Shl6WNB6oJ2Z+955s24dS3N9zIM221egBzJaic60Uej2lqHkv4uVwYdx4gSY8DVYvgMChzNWQHfcZbpsjjSJFOpT+rXLYwQuP6NwiFLByo0C5Pn2cpelj1S8QA4ZlTiN2ZlQ/GD81ueJB0kLFZPr6YGfq5XhFi6HQlY+2quh9KqJ89CT3f+yIkhoGyqCkdXwv95HBr/zwNIUsEHzPhEvu8v4pfcsNAbqei6Tp/xCyc7I8RUaloT6mxgNmP+Em7l4YWdPXpep9Dh4dz2TGSVlud6P1xHcqK1y0MjYr5XWj1mrPVrrKsixwRtzm5AoDwBW9KhlzuPQ/wx3NyFA7/MH0zLr8t7g5SOd2JhcQkxqHMqiwFlKbbg89pxUk9JUFxNp4rux142tBRKF1Y5bSFOXtp8thGq5FpmiF542ajPXZWpaJn1dJ51QeVvtsrVsM0zHqQ3Xp+xT 8imMVKdK gKhq/yo7ME72kb9u15eRrbWZh+y6WJok0vSYiIx7VIELtNRbe8RjC7uH04h3zRpm9kLFxUWKoE9cGVvVesjB2bvyY3AjEObIYAoM7bD+MK9CO4H9fGIwtmFzeiKvFbqu6S4kq8QZSkeA71t3nrcJy3IHYPDxhXTvD7juHo8WAyIxxb7uVfF+ToChapSl1WjJ/MbDYqcjWQrDGII1zzoECKzs+jzx4Qlkj9bqQD4Sm+EiaRqNbgO8Dcd0F1QU9QUZQEhbF0UdIXFN2X/SZTf36J3Jx40aEdB/x6zMqL3qGYnSXUqO1Fn7YSUYO1XE8cmA08sIbhVuyXIwL58QQ+OpLNksEAK4hKtuD4T1UX0av3pa56RBO4lVGiNxvBZe1BiDbgAOnfr3Azc7I8zI7wFlAbNVgzJje3djb6fd1u6Z+EbwCf9iopdeV0L+m4UDG1SOj8aKUkJnTGlk03mSG4fBkoZxjD7oC3kHSgu2fFMfLr78MkAv24j6mHq0mfZWZh0AJMF0qN8T6g0RsTU9jQP8SsjNsxGr3DD6dieXwNs99rwB/BdDpWCIMAP9haABMkDHKZhxQ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: ARCH_DMA_MINALIGN represents the minimum (static) alignment for safe DMA operations while ARCH_KMALLOC_MINALIGN is the minimum kmalloc() objects alignment. Signed-off-by: Catalin Marinas Acked-by: Greg Kroah-Hartman --- drivers/usb/core/buffer.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/drivers/usb/core/buffer.c b/drivers/usb/core/buffer.c index fbb087b728dc..e21d8d106977 100644 --- a/drivers/usb/core/buffer.c +++ b/drivers/usb/core/buffer.c @@ -34,13 +34,13 @@ void __init usb_init_pool_max(void) { /* * The pool_max values must never be smaller than - * ARCH_KMALLOC_MINALIGN. + * ARCH_DMA_MINALIGN. */ - if (ARCH_KMALLOC_MINALIGN <= 32) + if (ARCH_DMA_MINALIGN <= 32) ; /* Original value is okay */ - else if (ARCH_KMALLOC_MINALIGN <= 64) + else if (ARCH_DMA_MINALIGN <= 64) pool_max[0] = 64; - else if (ARCH_KMALLOC_MINALIGN <= 128) + else if (ARCH_DMA_MINALIGN <= 128) pool_max[0] = 0; /* Don't use this pool */ else BUILD_BUG(); /* We don't allow this */ From patchwork Wed May 24 17:18:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13254347 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1091EC7EE2E for ; Wed, 24 May 2023 17:19:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8D0B790000B; Wed, 24 May 2023 13:19:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 880F8900002; Wed, 24 May 2023 13:19:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7498E90000B; Wed, 24 May 2023 13:19:46 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 669E6900002 for ; Wed, 24 May 2023 13:19:46 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 3347CAE58D for ; Wed, 24 May 2023 17:19:46 +0000 (UTC) X-FDA: 80825810772.25.6335DB9 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf14.hostedemail.com (Postfix) with ESMTP id 79FA8100009 for ; Wed, 24 May 2023 17:19:44 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf14.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684948784; a=rsa-sha256; cv=none; b=hCszq3yYkcOKzJvFhRHgu9Y4cEcB6I1oxTYF9ulksX+Qs0Se5qINeSYgwzoAx9D/TwUmFa mvwy8jIRIIfhx61jG5XYxjmQiSlsKTpI3K2Q2W8jWawEEEvfScbg3ccYcaykK7SfNrGpCO +5Nn2BQrHF2sdiVpBJHq00GmyFKiyus= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf14.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684948784; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+fgUtZ7z2bEcsaerci/jffnaSF7K1ZtF0r/2TxWeNAQ=; b=5s4GcyQvOg/gTBlri6LOZ92Iq33FeZKR43gdVDp6DksK+E9e3/K+jqa2yVrZVRfVJLd7cf DQOT+UnNXkx3jrsb02gmHBrm2VuGTaEwhW8L4arIMwFJUvKkYcLrtvTqIwM6Vl9kKqSKP+ fL6SJ7umproJngM6lIUpg6ump1/w4eY= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 6FD2663F99; Wed, 24 May 2023 17:19:43 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2979DC433EF; Wed, 24 May 2023 17:19:39 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Christoph Hellwig , Robin Murphy Cc: Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 08/15] drivers/spi: Use ARCH_DMA_MINALIGN instead of ARCH_KMALLOC_MINALIGN Date: Wed, 24 May 2023 18:18:57 +0100 Message-Id: <20230524171904.3967031-9-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230524171904.3967031-1-catalin.marinas@arm.com> References: <20230524171904.3967031-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 79FA8100009 X-Stat-Signature: gfysed3rmtzcu45495c4g8woy7m9j7ac X-Rspam-User: X-HE-Tag: 1684948784-747603 X-HE-Meta: U2FsdGVkX1+EiuVDPgCtSLQJrIDjOE6p/Ol3fSMJLyAwzjn5C9j8VEtnfg7tqNDDwWtMoh+NwJYE7KMc1agw6yVNadttxLeiFVP+DHNLK1MX4i5QksZsKkRIEIOEYPHz6wyQx758LhrUC6NGABtVSb9qarSO9wjCj52tR9Wad46sO4PEUJTQdsVYsuH0s4HyHsYcCgyxSPxKeu7dRkIMVKa4/ouuc4M/IcEDP7NcDwnYg+6uFHaWm/F8E7Pw48i9fA9Gz5ylJsUZks1xrxxJCHBcM6PktKlPthbdAl/ZSxlmgFoT2Vz8Re5CNlaBsY7Hqb8IR+U5KWYrNhvUNClibNr2QwQFNXrE3gjHns+Gd4N3EU0GNDNKjqnGjHTmUIlAGQOvKMbwRyKYlRKvrnIXJ7AgOHCd70ZSNUGjU5Y3Y8EwGF5k8By5rGK0MehW0V1b1kVya4CUNjftN1mBlXZvXej7yXyqYbO/p+4oc4jxzU7un84uGU+5UG6T6AXl2HI/GU/hS41SfXo8yvXQU6lmEBOGJ7liyVNjlu8UXSUiyGggofuX5tn14uyGrdfl3XBPI+aG2iFBsO/XznRNbzW/auS5Dm3gDlk7bG1Matp1V2/GYjHNZAXohRGKN6JX8nusfa/toCavb9R2Cm2BJI6YC/fugvvWcQIfh8rUFijt6RRZIEa8y90jLSyFKFZgUIN3mdgsREcBcGWKPdmRJTZ8noBcHSAiyrqlO1OLg0/1KFg/WlLuMWhC9OcRShnoJD7b2avQ7xC4RgAQPnozH0nriOIy5bLoccNoxX/lTQ0UKL/Ulp2Jyo2qtRe2E0AndF8MON620PtlB5XFALU+A/GsG/Rf+kxfDybmw1U4M+glJafhgcF3bML3R+JeRifKCF3J/nkOgQHL0GhOva5qjEkGZlLVj06w10Gkf5tIs2F29gM6pYnZfIEprVg9tSEOLIw4/2GvrqXT11JNwBGl8OB PdAONHPJ 0BUYc4KuOuWCPr9IBNfE9CidoH7Xc8vpMLJOcg8hE8vu/laelDYqkFzGsOv/ymi5HZPYKPZvTRHmjFEEPfrFpjxq+U4wUW2/lO/bLrff68I+VLrs8J6JYTPwtwrzwm6KZzAg/m5XbMH7rKybtvVi3bNBAFFLruD3ziUrGANqvEmtNB105r9ebAa1a+uZu3d1AYI0WUEvVmUPyNpq4toeLAiDeV2YEqXq2VWcExNv6pcipJizGs4EVLD8S0HAdxSJuMlKGH6FG/bXr+mkSnUnIWcULo2DHX9jPN/kQdAYjxI2V6m7wR3HFrTBV7ph1LoNXrCkZ4Jv4EBP2HCEzNOPktvdqh4ijPtqVrw7y5aeb9qI8gmKPh8FZJU8v1jf87AdJwZPS6kZP8h/eMbq7Oh7CRn9qyvwWFWl3/3QHIBXSsyyT4g8VtGFOkeCZpUtPhjt5I4XrooXthryWRfMlOOZd0nWR3kZudL6GEg+Qebc73hNHHUKEIDimo46cTbBNIZnojZRW2NMZ2O4byQ5WUeDdPmDjDr3lOm7Ov5TmMnnVrUqOLHG1bDBOYu0JznLOyDIkzvEp X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: ARCH_DMA_MINALIGN represents the minimum (static) alignment for safe DMA operations while ARCH_KMALLOC_MINALIGN is the minimum kmalloc() objects alignment. Signed-off-by: Catalin Marinas Acked-by: Mark Brown --- drivers/spi/spidev.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/spi/spidev.c b/drivers/spi/spidev.c index 39d94c850839..8d009275a59d 100644 --- a/drivers/spi/spidev.c +++ b/drivers/spi/spidev.c @@ -237,7 +237,7 @@ static int spidev_message(struct spidev_data *spidev, /* Ensure that also following allocations from rx_buf/tx_buf will meet * DMA alignment requirements. */ - unsigned int len_aligned = ALIGN(u_tmp->len, ARCH_KMALLOC_MINALIGN); + unsigned int len_aligned = ALIGN(u_tmp->len, ARCH_DMA_MINALIGN); k_tmp->len = u_tmp->len; From patchwork Wed May 24 17:18:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13254348 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 30C44C77B7C for ; Wed, 24 May 2023 17:19:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C7A3E90000C; Wed, 24 May 2023 13:19:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C29CC900002; Wed, 24 May 2023 13:19:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B190290000C; Wed, 24 May 2023 13:19:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id A2DF9900002 for ; Wed, 24 May 2023 13:19:50 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 7952612093F for ; Wed, 24 May 2023 17:19:50 +0000 (UTC) X-FDA: 80825810940.17.BA29F49 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf25.hostedemail.com (Postfix) with ESMTP id 71A1CA000F for ; Wed, 24 May 2023 17:19:48 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf25.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684948788; a=rsa-sha256; cv=none; b=z4vN0P376TFBlJ3oJXoRfX8uu1wgT58NyXCcuv8dfdhilBqzq3ThMkhusH531y1qD178Rh gqvlBuBZBPjg6/1yijAnKTT+KSrmcr8s0oOTpFJABgRpXra8hJYQQdRaWcWcWkLYQjLDJa 4bw4L0e6//aX1JXLK+LZVS8ra7BDHI8= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf25.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684948788; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=P3GKCRqSal+FoaPUZC73tQFjQIUPiL6KpjJp1J/vlzY=; b=BS+YdxQgyofOhX5TQ69UcxP51QVsNaw83woeOYBSdZQ9s+tnhuNSHwHmaQ7dDo2uvY0KaA +rXxLbzZImRYXqyc4fQ4iv/kbPKjwMv2D+ExtGhuJULqjlVhwuGZqgiHK6gAEyfjsmutdO MJD0LgO7/oeupkNq5/KJxeufNkigTuw= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 8C70263F87; Wed, 24 May 2023 17:19:47 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 444B1C433A8; Wed, 24 May 2023 17:19:43 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Christoph Hellwig , Robin Murphy Cc: Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 09/15] drivers/md: Use ARCH_DMA_MINALIGN instead of ARCH_KMALLOC_MINALIGN Date: Wed, 24 May 2023 18:18:58 +0100 Message-Id: <20230524171904.3967031-10-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230524171904.3967031-1-catalin.marinas@arm.com> References: <20230524171904.3967031-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 71A1CA000F X-Stat-Signature: 7qn9u1jqabembnhm4uiebbdbrzeu65ip X-Rspam-User: X-HE-Tag: 1684948788-964966 X-HE-Meta: U2FsdGVkX1/s9udOW2nw02ipt8Ou2Xfnaa+osRrXgccmYsyBWYQJ+55O1xO4dWbBb5NQUs4/x01SKSd6YmSn0N9vX0aaAnBqw1XtYrIjy7lQhhzf/xAj9rJCpZ8spEHiU1ifRhg5UR3RSn2HDC/HxAwMR+HvQkgGMbL6yB9U+M2kTJpRDkK11l3qwH6lUTuJhV29dEG9eZCBNMMNmyp+udQEl30zh81TBkjFqcUooDv9hWtMqSJKQW6xz6k5WdRFV1X3HsvbyqwukDMBEvuD8Zy44WVhcdcug1g8CFiiu1basZ/5xiU/zKt3eVJPns7RznUVOc8SR7SYx4MoGDpCdO1Bi7dqIGp8rLN7L5seoC6xFlJATgBY9gmHwCtIU1PTO8E4xhMSO1qSEcw0DQO7l6Ne7vhM7ghZskXpcqDam8BsCrMQM1fG+jV1NFySEN2DmvkQGb6EbvC+T6L6dLRFdsD3p2dEubHjAwUydna9Ga2YTtNa4Coa6lzQgxjM7YXRW5lxy+AXa0UKoc0Banoaxm3PD8DunsdM8CId4PPQC+C1hmn3aBLninQg4LWTJeL9AN9zIMebTIU3Pj3/YkLGJIZmLajUi2n99IkxqlvIMxwyQg+bZ0aiUzEQeVpBb0KIGpX6eAX+TUxvuTk42Ss/8FGbbEdJBzGYKfgH2RHS5ZtxKXJwCrfddW65sj6fws9a6udXCYyhKCRMPcCz68RYFylM/d1nK3OgAPDb0qYZliHMaE60nowwsOLvL0uxnJg3SjzkQ1TrsF/BFJvdpfJXDr5a7eGVdHp3tDQdkweRaQseB2zv8p6SWg2V+aHNtwEdgpwH5AY3HgOGmxpc4UywQKwD7mzDua9K8JRH2k0XY1fCxtJIX77IvPCMAfqwkxXK8yyq82PRSfMFBr7On1B6gbVqf8NQmaG6o680DSowSROPap3HnxSgtiB/F0gc71z4Dqoo18wuPFodpF7iGPC QESYNKgc z88XyXSpTezDrKWLo60JpIRMR0tydxpV2/HJonG4TSR26qweEBnefEDmnksaBixw/C6kZXoCQJ3l38oaTjwzNCDaJutUPiAUL14Lw+iWk4o+TxYoA+B/dEN/549raqAoALFe+Kyzdh0gpt+aBprcie5KKjLesW1UZR6i7zk2+eBBkdiS+8ZFzvzPgTy1QF6WzOlKb0iDX5G5JBZQg9ycCveC8QCsLJVUmRQgs9Hp1b7HUXYoGJEMtP0FNWAzaTqRRvJ/Jy6Ya+FUrIQPRPYOLvsDRpCkhKnLJ4Sp/JJObIecc2Cc/xjcex2ChAC96Vu/N0M41Czj+/Y7h6qSa5msfYbUPOvkLCimL3Zq8m1m3ID5Psqvn05sVmpTUY936mhE19wG1PSM99e77lZSjU29WdIpQHNCJ6BujW5m+OWnQHI3wq7Pr1ePqKgxHYK4p3N7sLlBrb41SnD8DTbNP04D3NVSb4zCu1o7L1FKNGMsEMuVjyUTdt19uDbY2hDIHMLWlSbZWQDBv6Re0Gj0DJWgVyZmT6rZlXc9BGTFuhoOL3A2Vzoy1bNlIrDF+idNmvIlT3GGIPSH05uOs9fJ77/q4/7t5Hw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: ARCH_DMA_MINALIGN represents the minimum (static) alignment for safe DMA operations while ARCH_KMALLOC_MINALIGN is the minimum kmalloc() objects alignment. Signed-off-by: Catalin Marinas Cc: Alasdair Kergon Cc: Mike Snitzer --- drivers/md/dm-crypt.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/md/dm-crypt.c b/drivers/md/dm-crypt.c index 8b47b913ee83..ebbd8f7db880 100644 --- a/drivers/md/dm-crypt.c +++ b/drivers/md/dm-crypt.c @@ -3256,7 +3256,7 @@ static int crypt_ctr(struct dm_target *ti, unsigned int argc, char **argv) cc->per_bio_data_size = ti->per_io_data_size = ALIGN(sizeof(struct dm_crypt_io) + cc->dmreq_start + additional_req_size, - ARCH_KMALLOC_MINALIGN); + ARCH_DMA_MINALIGN); ret = mempool_init(&cc->page_pool, BIO_MAX_VECS, crypt_page_alloc, crypt_page_free, cc); if (ret) { From patchwork Wed May 24 17:18:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13254349 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 23852C77B7C for ; Wed, 24 May 2023 17:19:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9BA7190000D; Wed, 24 May 2023 13:19:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 96AAE900002; Wed, 24 May 2023 13:19:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 85A4490000D; Wed, 24 May 2023 13:19:54 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 769A8900002 for ; Wed, 24 May 2023 13:19:54 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 522C580ACC for ; Wed, 24 May 2023 17:19:54 +0000 (UTC) X-FDA: 80825811108.29.0FBB9B4 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf01.hostedemail.com (Postfix) with ESMTP id 8A3FC40013 for ; Wed, 24 May 2023 17:19:52 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf01.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684948792; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Cf1L6WmjPMAYm9aSbUscBDbAibSbBWO2SXPG1VqERas=; b=1n29m0e0qZgmzfhJhn0aNSAe2nYqcpuUgSiU0GXQsvj/CPyECSWbNGNI9Dg/xDldX+IQnU c+jIG+6Ef/0KdELxwseKlGuKmUgSOJnH0WI1IxZ7ErhF+CxYL9nKNMxruWDokpfgwqTDQD 2VOA3iBCBI3YeL9NR5zS+iqFkBmpSog= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf01.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684948792; a=rsa-sha256; cv=none; b=bF6OEkc9X40cpEux42E7l6BWJqh/Nbc79i94T9Qd3aj4XRTIpEOYMsLtkT74FInGPxaCOR t5rUjp9JcHVzAtGRDtlwK8YMLy72fUVnhLudbi95myn3W7/G+eeanO5PndY8F2DRwJnsHJ 9ZhXQQDx4SjpMSy7/sMmKKjWgLXevr8= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 9E60163E27; Wed, 24 May 2023 17:19:51 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5E95DC433A7; Wed, 24 May 2023 17:19:47 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Christoph Hellwig , Robin Murphy Cc: Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 10/15] arm64: Allow kmalloc() caches aligned to the smaller cache_line_size() Date: Wed, 24 May 2023 18:18:59 +0100 Message-Id: <20230524171904.3967031-11-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230524171904.3967031-1-catalin.marinas@arm.com> References: <20230524171904.3967031-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 8A3FC40013 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: ik9opr731qxab4wywkkyrihhxf7xxw3m X-HE-Tag: 1684948792-799068 X-HE-Meta: U2FsdGVkX1/Ijm13UsezJNNHwtntTfPVJDS46wMhebwHeTloK1kJFvUjdPbso2Co4N8xmYhVp0553gXm2gDuQw/Cuq7Yj0XkCZSsdCgoKmc9XywiAkSjDj7Te+DmVOt6Qd7v9ZwU5UWT/DCdYqBygZJtZ9Scfu8cjCGgkz75YnrEi6QY2FCtOtUEwiiu0Sn5L0U+Hp3T1ZAxEZxS4fFk+ID5jhDltcQuVnGGFLAjZdAf1YBtufWpMLz6YEZi8DmN582grCyLpGTl/h8qcSCcPsoVApZb45G8layKm9ugINdczablVCftfwNi4peGIzHhOSKc4vZPX6u64s6G9A29LHwKXbUL48qlzxpKTkUrK7F3mGeAmxtXNAgA76iUDUooYFvLv/gBQFruFhv5elN9kXu6F/pPFESj5+I7CEKuz4JYoreqv1Ynsocomfc0Wxap3LSnvwp07rpymB3/m1EW1SCcPH2lNwOiRAfVfEOuwUDXb722LCAVl9K2kZsv9OQXhdk/GI8IVvNBX3BVth1k6ZY73pNoCwR+oQkKN0A3wzJNPo+U0XBGuWaXFZ+t/Li3cSUrq+vH6hmsr+YGalnl5Bb3TaS6/yRYbM4avTXzOcJfvGxXiZbHUs6NXMC4E93XGn8l1tcvs+RDSZINiSXwdwdqK0KRV1orrl50jstJwnkGsMywgB2zDu4QMQFpyjeqk0BBp6cq7UQ/Evlzz3n2oyBhfz3+nf3xUTz4g+hKqz7FR2PKSkI/mnFrpXdqlXAQDSNhBGK8yGxqRxBTghO0qG34aPcyFYx9PHQUepr2IC0MryxPZMkNfDPkqxW648LJRyGuMZMSxgTjjv38GHOVgj+A46349AfJzna2qv4HrvApvtM6VVXXwsG46P4R2DoTQ+b8+ZlxfYWyrx3s4oqB0W+x1b8VHVSPJuDyS0e+qJEr7Pi5EqWtXxcyLTTvRZZ2EjrZbdC03uf3vjuJPfb 0BIgc/p5 uYccbHYAEr3uwIit9l8FB8KknMGVsM+k5xQIYUUOqI6UO5FyHVSrqwa0ZwjuQERbjeVw6wtoz3/Svbga45DIsfAiSt6aw+JfjDQV7ILw87RHMxEEItiWshvhQuY4C4qdPlWN30tSk/xOuxfhKHpbeZhmCnjrwMYopGq/n3dSSfeWDzj5PXl/M0E29/zQkXaPqvmFeYo/V/kny8a/ZkF7TZyrK0XmEexXUzyth64v17Jf/RnEs8wzEWEiupl3aB7mHwKH2uw8SWH6GOaItbRjH0UvVdNi3nQk9O8kFmThHlHTskR9T4iz75UL7HtYLtc1Yo+aRQDf3BbCdYAp9WUmN7n28h1HyFLv5JIWzQjWz9ZHHi2ezkEAKFaZdnLp0ImkPWU+opl2xRzrVEo+qfMfycw2FqPggUyA2g6kvM18P4+i0fAtgK/X3vd+vCpbC3C+JgCGRJ/uHVoJuFh0GmqYIkaKxaDne2s1Do+x5E+1ah5BSSJ3inbs0cqEjaDg4mV1DWqhPWprgjojnGuQfjbFjTo9XM9+3hmhOLnnaEM2QnrNemB7IK8QKIgfv7KSojjoEfqlu X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On arm64, ARCH_DMA_MINALIGN is 128, larger than the cache line size on most of the current platforms (typically 64). Define ARCH_KMALLOC_MINALIGN to 8 (the default for architectures without their own ARCH_DMA_MINALIGN) and override dma_get_cache_alignment() to return cache_line_size(), probed at run-time. The kmalloc() caches will be limited to the cache line size. This will allow the additional kmalloc-{64,192} caches on most arm64 platforms. Signed-off-by: Catalin Marinas Cc: Will Deacon --- arch/arm64/include/asm/cache.h | 3 +++ 1 file changed, 3 insertions(+) diff --git a/arch/arm64/include/asm/cache.h b/arch/arm64/include/asm/cache.h index a51e6e8f3171..ceb368d33bf4 100644 --- a/arch/arm64/include/asm/cache.h +++ b/arch/arm64/include/asm/cache.h @@ -33,6 +33,7 @@ * the CPU. */ #define ARCH_DMA_MINALIGN (128) +#define ARCH_KMALLOC_MINALIGN (8) #ifndef __ASSEMBLY__ @@ -90,6 +91,8 @@ static inline int cache_line_size_of_cpu(void) int cache_line_size(void); +#define dma_get_cache_alignment cache_line_size + /* * Read the effective value of CTR_EL0. * From patchwork Wed May 24 17:19:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13254350 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4774BC77B7C for ; Wed, 24 May 2023 17:19:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DB26690000E; Wed, 24 May 2023 13:19:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D6289900002; Wed, 24 May 2023 13:19:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C2B8390000E; Wed, 24 May 2023 13:19:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id B14FC900002 for ; Wed, 24 May 2023 13:19:58 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 8017316028D for ; Wed, 24 May 2023 17:19:58 +0000 (UTC) X-FDA: 80825811276.27.0935B58 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf14.hostedemail.com (Postfix) with ESMTP id BE4A0100018 for ; Wed, 24 May 2023 17:19:56 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf14.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684948796; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jcO3Jclmw9mB7L7BfaIYzVvXEyR0RpDGNHlTTAHR8/g=; b=GVGobseF3qb+2BnW275/ZTpStHiitUluhJw0G8KChV3ixH9otec99UzuAgRTMnz7xGM7gb GCIvxEFq1v4taeN648FArGdrwpEgQmTVR/ZqxRMIT+6SGU1zE3UFMmKBVbGGCh1KBLeTEy 1rdpVWn/uDOS3iuIJxvWKWQEx5HtJ54= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf14.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684948796; a=rsa-sha256; cv=none; b=oSv9Y0MhAUUdGy99AZ7dP9T1qJElyYc+Oo6tdi4neYXfz5ux+f7XrjZwdkvpKfE1f0P8OX HRIiT44Ipd+LE6vnwayNd+Zu5UdJH3M65ZSNitE+9ftyk+y3Y1EwKTg4CtrBL7AgXCcDgq t5LUB5k8OwaEBzjwjnY4f/TmqJV9hmc= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id EA09D63C6C; Wed, 24 May 2023 17:19:55 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 768AFC433A8; Wed, 24 May 2023 17:19:51 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Christoph Hellwig , Robin Murphy Cc: Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 11/15] scatterlist: Add dedicated config for DMA flags Date: Wed, 24 May 2023 18:19:00 +0100 Message-Id: <20230524171904.3967031-12-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230524171904.3967031-1-catalin.marinas@arm.com> References: <20230524171904.3967031-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: BE4A0100018 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: ztpo9xx1hb1nfg8pzi3xa15myenhob93 X-HE-Tag: 1684948796-956292 X-HE-Meta: U2FsdGVkX1/B5gf3QQ8WbkgSMsOn7Lx38cH+/mBm21ckxcs0m/pDNTM0pbP5RuSTL6ax6X3MTi3oC3OFXoLBRJX2SXQK0owDbgXBrul3fo8BEZGe5ow0Nk8WMuGYviBZ+8Zbk0Gk5nT2mvEzOKy3vNRk/vw1YHjbGQYx1E/RcdLhaLvYD9d+8sFDPNzgn5I2Z7dFLaUog3XijYZh2H0W1D0NlVF/SR/P2khmsb0blx3Mc+ZUqfu6kI6jOKg8TavTOWY3ViR6SRtLQY3albuyp4A3xX02Q1L+3MrQs8yhMKkipdf6kDnV2iUnpcu1oMnAyZuBxX07eg2OnU9G76Fuvzq3WhQNTnEQ9+jbFyBv/JUlsOLI9gO6lRSP4ahdthjsUGI6Ryx0tCraN4x4fROZN+LASL2lSnm2lJX15vOSGsbv2sn1zQWtUXXe25GSlaLyGfkHcf+bKr5l7fpbygTq0EeLFAdJWvlEp6ISC/qLFfiF8ENVzbUjaKM+WlWseQBU4gUt0zOisnv2VMq3IGxq7lb0G64G3wZ9fs39XTC5ADuvJcV0ImGVGFv4FQKa/2EqxNea7TDROkr9rxtpo1WvjmYYKnc19/VCPeO4NjJ8KBLwJspn5toQD81jTJ28Be3R2KNJ/dE77Eo2OwqZnMwpuBRtx+RQAN6CKnl+BiLfHxoePIg41UHcFDWb/CJR4diit2GKfOWnm+V5g2ACJJ6azS7IY0nh/z/nfjimPR0A9LqHxi0y0+f+Sftqg4AydjgFNB0i8OFyC/RE4KVy7q5P0urlAJvrjbbQDZWAX/o7JTOc1wB1DO31j7bUUH0SjfF5f9CKn2YaQshvcSvFa2AKF9Y1b9MHKFCGVaTrKe4UuEwoWO32ezHe8X4kfifYnRtQZhLzNs2JIZ0YY8v/VK/YV0lm8v4KlR1GQ+gNy8JpCdMS6sXNIFCmSurXK47/WWCpYsoHkmTrzvdd+FdLr1w OtRZ3E59 /7fIl2qm1cLtdKyDNsyqFPdeGXKc+5uarA0juxQf3CX32ytVGPwhFFRNfn3PMREdAsM4LQgs8qd75QH547ugw0tS663vwk0WbNfsXIwctZtuUIYEpbw6NplVKmufiLoqFyXbJ3ePA6oq/x3uds9+qWgsxBMy6WLQR0YEea9CGiQyRIn1DTIfs0z3f+iSA0iscY/iB/fMea0mNwoK7cmMUAb2I50qBN1Ej5rzeTdWC3c/4NLUvB5pcU/0+/Pn3sqmQX++A4SLO+Ul6pdXrG8CoK1I1gz5Fs3f7lZfQ+pGqtw5qOx/2+LNONAGX8QBCleA2cBIsLi+ymqfVBoW0sYUBLwBWcQmtBTQp64A7ZNKMJL6dVShnM0K4Rfku0V/rKs9IrROhIHRK1YX7iKB8VzDQ/stccSrl8l/UoeM8BGDS4Zbvvgxtm9DwNVePSqRixLDGzX5/mcn7XFWAxnojH2eFpYdceCnLLFFL9jjWSdEFPnOI9DXV/9u58Y1sNe3XVu8eolg6 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Robin Murphy The DMA flags field will be useful for users beyond PCI P2P, so upgrade to its own dedicated config option. Signed-off-by: Robin Murphy Signed-off-by: Catalin Marinas Reviewed-by: Christoph Hellwig --- drivers/pci/Kconfig | 1 + include/linux/scatterlist.h | 4 ++-- kernel/dma/Kconfig | 3 +++ 3 files changed, 6 insertions(+), 2 deletions(-) diff --git a/drivers/pci/Kconfig b/drivers/pci/Kconfig index 9309f2469b41..3c07d8d214b3 100644 --- a/drivers/pci/Kconfig +++ b/drivers/pci/Kconfig @@ -168,6 +168,7 @@ config PCI_P2PDMA # depends on 64BIT select GENERIC_ALLOCATOR + select NEED_SG_DMA_FLAGS help Enableѕ drivers to do PCI peer-to-peer transactions to and from BARs that are exposed in other devices that are the part of diff --git a/include/linux/scatterlist.h b/include/linux/scatterlist.h index 375a5e90d86a..87aaf8b5cdb4 100644 --- a/include/linux/scatterlist.h +++ b/include/linux/scatterlist.h @@ -16,7 +16,7 @@ struct scatterlist { #ifdef CONFIG_NEED_SG_DMA_LENGTH unsigned int dma_length; #endif -#ifdef CONFIG_PCI_P2PDMA +#ifdef CONFIG_NEED_SG_DMA_FLAGS unsigned int dma_flags; #endif }; @@ -249,7 +249,7 @@ static inline void sg_unmark_end(struct scatterlist *sg) } /* - * CONFGI_PCI_P2PDMA depends on CONFIG_64BIT which means there is 4 bytes + * CONFIG_PCI_P2PDMA depends on CONFIG_64BIT which means there is 4 bytes * in struct scatterlist (assuming also CONFIG_NEED_SG_DMA_LENGTH is set). * Use this padding for DMA flags bits to indicate when a specific * dma address is a bus address. diff --git a/kernel/dma/Kconfig b/kernel/dma/Kconfig index 6677d0e64d27..acc6f231259c 100644 --- a/kernel/dma/Kconfig +++ b/kernel/dma/Kconfig @@ -24,6 +24,9 @@ config DMA_OPS_BYPASS config ARCH_HAS_DMA_MAP_DIRECT bool +config NEED_SG_DMA_FLAGS + bool + config NEED_SG_DMA_LENGTH bool From patchwork Wed May 24 17:19:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13254351 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3BAA5C7EE31 for ; Wed, 24 May 2023 17:20:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CC79D90000F; Wed, 24 May 2023 13:20:02 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C77E2900002; Wed, 24 May 2023 13:20:02 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B400590000F; Wed, 24 May 2023 13:20:02 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id A5A8E900002 for ; Wed, 24 May 2023 13:20:02 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 7F4B6C08AA for ; Wed, 24 May 2023 17:20:02 +0000 (UTC) X-FDA: 80825811444.07.2C8E7FC Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf13.hostedemail.com (Postfix) with ESMTP id C4CBE20011 for ; Wed, 24 May 2023 17:20:00 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf13.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684948800; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=trBjKA2P3t7Lv44FdRPNpP5ETnPfkIlOly8ehL5hPTw=; b=PhdSsGw6QXbW5hlc2SWmD3w/ZxaYsV4o/OXauaUD3bTd9QQtGhBT3PwnkkIP+PegZIeCOp URKtrg1cu82s85EQA9VDgKscJeGLy7bgfL+F7B77iha1WjYiedWaHYmY71fD2wr68MCTHv oMWRcAH9RFv4+xnHQiE9MRpMnOCehIM= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf13.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684948800; a=rsa-sha256; cv=none; b=ni6uSCTyIFvAW0zrH4MEJuvOScuXAXpvieYAZrH6ezsi486efXnYhVIpKGAr04w0I4B5t6 aarHph0HCrRYZ6dxYQOhSq2d0A78vEUjBy52MxJX2lw0DaY6PY9ZT8r24WG4BpS9Vt/nl8 09lYkKnF++X3lMT1Rtswt6KATyJICqo= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id D8365638D6; Wed, 24 May 2023 17:19:59 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9AC60C433D2; Wed, 24 May 2023 17:19:55 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Christoph Hellwig , Robin Murphy Cc: Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 12/15] dma-mapping: Force bouncing if the kmalloc() size is not cache-line-aligned Date: Wed, 24 May 2023 18:19:01 +0100 Message-Id: <20230524171904.3967031-13-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230524171904.3967031-1-catalin.marinas@arm.com> References: <20230524171904.3967031-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: C4CBE20011 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: 48imtgq3f3syg1myhqbxqdqxssbz4e6k X-HE-Tag: 1684948800-922560 X-HE-Meta: U2FsdGVkX18YaUs2/5q3zCNrTmAWy+r/GINU4UxIm4fB3sakxTVjtp30h2Av349WKaFhaT7PO46QXSZc5tG7AQOXbO/OrTvdYPw0kOh89yVNDZGlJGC75I6dTDVLzdoFdf3K261Yx4CMdd4D9YJVpgAMPq1rcTu36QXELQnggksA+sEGwzP3sc8ygF7EWe1o38UiGlLhnmtT1B+6EDBhMH9z5GFgxzvtQGackDvGVlnmmPzUxue1gxgqyVsqkSrQs/UlzPecFoGwgdObfaZPqcm/lASqyAa0O4HBnkl72L04u/XJU5dyJvvRG3ojOTKFb4Wr/Cm0GvXeuoVsmebqzSkD+hXSbfKA6aMQ8PxOzTwiZoDtim9i2OGacKkbGmey5eW8hWYM7UvqnY8SAu29zHs5lKr/9cr+sp+YUocm77O6DznKx3p185uxXLkNjxV2DbdCvyiH2L0EhPdLOK3ZXxIekgcFUad1mDAV48QrX0j+n46D+5HxfnZIGp5jAjZYpu3nQEZ8PheieuPqJsF5dx/l7vlydldbKnLt7z8ZcvMFpr25MRCQ8s8uy2OMnladY5P8V5btUWDeSAdL5dhwTbhSAWYpHJXzLWQfJZ55HHRbM5qayEaCbDq7ox2eof7r3eAP3f1XH/x1NL7wLqXdUPZKuFVe2zASREIYPGwIB6CT1aaEApxjl7x3qkU0g1LblP191OZyiJx0cwvShONmflwujML9Lo2m+S63AcDsrY2siNUswJWlUskCjb8mLeTTQeK6LEBhgfsz4lP4LCCxYr65wU+fxvLmHPL8paqAtMfEl8Dry78TM3gEWv2yTtMpd6jseqfnuk21DjPRGz6YjDgssb1aMl6OrUuae2Hzuxlrgm7ei8Iy4MGc4Qja+WIlCcLUkGJ2lgUB/+C1Egm8HwySYuWzBAfkKXcbPrK/K60SDRF6jW5+/tMs0UQMWNMKGevrnSRxMTwK7iYNv77 wICySYMA AaZ4VKgONkBkVoxcfZAR8WpoFfKN03aN2R1DOai6vp1MqxtVNkWUzBZmXJgUS5zd04/gtAEPQHg2g5LzFrX3TU+RjIF46QRHaZ6n6hIQWgUK0JT7Xp/1ghO3QrufPUFOBVdPPvRfj7mt5JdBhIFlZfg9oD9o1ZlJCF48+V8ROLtFsBs1wXd6k3SBbNwxi0apIAxPqkywoimRhuCmc8gl1XdCT4obw+UbvNipdmtRmJY2lRGSbBOtEDsSiFlGYDb91Ppk8HQuV+DajcnZoCVXBjySeLfenBoWbJP7TDoHPo8hKI20R68AAAzJ4BAEySa1KJNsIzsfu3+MkZG7LjuBpVggWYawh9ipHqfyxwDGrVXMn9trYo99vAf8sNbmEudsFuWlXtJxVYyfS2DltdkrOUCAhHg/1wvyuqekE11kdwzuRwGtc8UEzIjHt7YgKyFyAVPBDqmMBdhLeOqEaFGwPttfGrQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: For direct DMA, if the size is small enough to have originated from a kmalloc() cache below ARCH_DMA_MINALIGN, check its alignment against dma_get_cache_alignment() and bounce if necessary. For larger sizes, it is the responsibility of the DMA API caller to ensure proper alignment. At this point, the kmalloc() caches are properly aligned but this will change in a subsequent patch. Architectures can opt in by selecting ARCH_WANT_KMALLOC_DMA_BOUNCE. Signed-off-by: Catalin Marinas Reviewed-by: Christoph Hellwig Cc: Robin Murphy Reviewed-by: Robin Murphy --- include/linux/dma-map-ops.h | 61 +++++++++++++++++++++++++++++++++++++ kernel/dma/Kconfig | 4 +++ kernel/dma/direct.h | 3 +- 3 files changed, 67 insertions(+), 1 deletion(-) diff --git a/include/linux/dma-map-ops.h b/include/linux/dma-map-ops.h index 31f114f486c4..9bf19b5bf755 100644 --- a/include/linux/dma-map-ops.h +++ b/include/linux/dma-map-ops.h @@ -8,6 +8,7 @@ #include #include +#include struct cma; @@ -277,6 +278,66 @@ static inline bool dev_is_dma_coherent(struct device *dev) } #endif /* CONFIG_ARCH_HAS_DMA_COHERENCE_H */ +/* + * Check whether potential kmalloc() buffers are safe for non-coherent DMA. + */ +static inline bool dma_kmalloc_safe(struct device *dev, + enum dma_data_direction dir) +{ + /* + * If DMA bouncing of kmalloc() buffers is disabled, the kmalloc() + * caches have already been aligned to a DMA-safe size. + */ + if (!IS_ENABLED(CONFIG_DMA_BOUNCE_UNALIGNED_KMALLOC)) + return true; + + /* + * kmalloc() buffers are DMA-safe irrespective of size if the device + * is coherent or the direction is DMA_TO_DEVICE (non-desctructive + * cache maintenance and benign cache line evictions). + */ + if (dev_is_dma_coherent(dev) || dir == DMA_TO_DEVICE) + return true; + + return false; +} + +/* + * Check whether the given size, assuming it is for a kmalloc()'ed buffer, is + * sufficiently aligned for non-coherent DMA. + */ +static inline bool dma_kmalloc_size_aligned(size_t size) +{ + /* + * Larger kmalloc() sizes are guaranteed to be aligned to + * ARCH_DMA_MINALIGN. + */ + if (size >= 2 * ARCH_DMA_MINALIGN || + IS_ALIGNED(kmalloc_size_roundup(size), dma_get_cache_alignment())) + return true; + + return false; +} + +/* + * Check whether the given object size may have originated from a kmalloc() + * buffer with a slab alignment below the DMA-safe alignment and needs + * bouncing for non-coherent DMA. The pointer alignment is not considered and + * in-structure DMA-safe offsets are the responsibility of the caller. Such + * code should use the static ARCH_DMA_MINALIGN for compiler annotations. + * + * The heuristics can have false positives, bouncing unnecessarily, though the + * buffers would be small. False negatives are theoretically possible if, for + * example, multiple small kmalloc() buffers are coalesced into a larger + * buffer that passes the alignment check. There are no such known constructs + * in the kernel. + */ +static inline bool dma_kmalloc_needs_bounce(struct device *dev, size_t size, + enum dma_data_direction dir) +{ + return !dma_kmalloc_safe(dev, dir) && !dma_kmalloc_size_aligned(size); +} + void *arch_dma_alloc(struct device *dev, size_t size, dma_addr_t *dma_handle, gfp_t gfp, unsigned long attrs); void arch_dma_free(struct device *dev, size_t size, void *cpu_addr, diff --git a/kernel/dma/Kconfig b/kernel/dma/Kconfig index acc6f231259c..abea1823fe21 100644 --- a/kernel/dma/Kconfig +++ b/kernel/dma/Kconfig @@ -90,6 +90,10 @@ config SWIOTLB bool select NEED_DMA_MAP_STATE +config DMA_BOUNCE_UNALIGNED_KMALLOC + bool + depends on SWIOTLB + config DMA_RESTRICTED_POOL bool "DMA Restricted Pool" depends on OF && OF_RESERVED_MEM && SWIOTLB diff --git a/kernel/dma/direct.h b/kernel/dma/direct.h index e38ffc5e6bdd..97ec892ea0b5 100644 --- a/kernel/dma/direct.h +++ b/kernel/dma/direct.h @@ -94,7 +94,8 @@ static inline dma_addr_t dma_direct_map_page(struct device *dev, return swiotlb_map(dev, phys, size, dir, attrs); } - if (unlikely(!dma_capable(dev, dma_addr, size, true))) { + if (unlikely(!dma_capable(dev, dma_addr, size, true)) || + dma_kmalloc_needs_bounce(dev, size, dir)) { if (is_pci_p2pdma_page(page)) return DMA_MAPPING_ERROR; if (is_swiotlb_active(dev)) From patchwork Wed May 24 17:19:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13254352 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E1F29C77B73 for ; Wed, 24 May 2023 17:20:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 833DD900010; Wed, 24 May 2023 13:20:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7E4B1900002; Wed, 24 May 2023 13:20:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6ABD3900010; Wed, 24 May 2023 13:20:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 5BF4B900002 for ; Wed, 24 May 2023 13:20:07 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 26C4F80B2C for ; Wed, 24 May 2023 17:20:06 +0000 (UTC) X-FDA: 80825811654.28.FF61F64 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf05.hostedemail.com (Postfix) with ESMTP id 132E8100002 for ; Wed, 24 May 2023 17:20:04 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=none; spf=pass (imf05.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684948805; a=rsa-sha256; cv=none; b=Sk/2HiEph5s6NwvnumqkPPX0VL4VH4m5VGL0IBiWMpzLLfvytSh7xFopcmun9T++J09RTi uQwqP0NfHK2gbSlmwsh4l24bJrQzquGdCznFP66UY8p5PXHT1eDx0jIZavvXu93UVqCxaL isAZYWt1+CTlobQli5ROEaZb2IuP1NY= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=none; spf=pass (imf05.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684948805; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KKonPp9ec0Dwumt9xX2iBUJbTxrdMfCA1fM7B0ZFm7k=; b=GcXjifo9pcqToWx3ZQ8EAOrpp5quh570MY0hiwB6r19QTfG2PzIQOPo59PEeqSfnJCgLjs LUXA+Xkawv0ObwUUND3Zl2CVH16PqjaqF9VNyyrVIQD2j7BEEDDnvgg6cLErUKxCz0zpwi udr9l/FAMn6rUYsR3Mi7JCoBBytj0oE= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 2580B63FA4; Wed, 24 May 2023 17:20:04 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id ACB15C433B0; Wed, 24 May 2023 17:19:59 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Christoph Hellwig , Robin Murphy Cc: Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 13/15] iommu/dma: Force bouncing if the size is not cacheline-aligned Date: Wed, 24 May 2023 18:19:02 +0100 Message-Id: <20230524171904.3967031-14-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230524171904.3967031-1-catalin.marinas@arm.com> References: <20230524171904.3967031-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 132E8100002 X-Stat-Signature: zxwyu3o98jtf7r4qixebkkw4r6tcpepo X-HE-Tag: 1684948804-988601 X-HE-Meta: U2FsdGVkX19zvRIoFQUJEqa6T7uBQK24RlJc9yuH4ZmWooE0eUaSP+d9h7Ebmv5azN8DpLw1vG21/2yc8cT01/e40vvAPuGEyT65PeNnwzLT6oqTic8DpfoFSxV6reYUr+79zLAC7Yy4myj7yMdGelS+cPmK0KuNYbKnDJ1aucT4K/DOGNztUStcYFpNaDvbx2QPSiG17f+Z99Z+sYU4MPCt/ijXJKC/GaKqF72SPF/BnA01c19d+5cnWB1LFEK0JjNwYoDU4CkyCeoU97TMCSUytP88q1h4mGz6+DFPwbAgIvdga6j8/hpTBu5Pv/UT1+H+BqXJJIILFAwOak9sHbdc+x48+twNMFTsULp0JaivVmst/KF7AfpRrS5unlDoRstxzuz+AbgWzZBQTeuodnWvFYTBG9aQcagvEpdxIlj0beSu6E0kwW/WkZiAlGOlitGYksYAjhlz/Bl/KimvHH4iC7EltApYARO0gNCtHXjlgdJJ0gPn+wFjJWX0QDm7+e2WuUVlmkjcvWE2tqjTZ8WygYaTGd8uF4FLKm1LroRemw69eq2FXd37zLLzOJUWW6og1vSJnmeAQ/G/wN1szymYMu6W0bm/ryWpQa3Wg4YND34rKwuQB4OBaxv0X3emXz43C/JU6bvdJUXclxc6C2oONzWEENqHnjyd5Y17KG0kDugTOXPMPQj7qN/XisbfHFSv+GYDPBhMoqwlIjKFOVomf4FpOEyDtX8zMPSrcVBLeOY+ctThl517wW/WsOI2kMXnh9sZ/n3JO94hiHiDrEweGwcZGiW5nrPhfBep0Kd1XFSueduBQzkIR16bYqjwCBtzyUEMyBSYPezN5m6wYrlUcvenDsfeU8O1ds+OQ/0IkVPldD+98xXnZ/nD/IsaUDGqeE7ojKC/y5Dbc3RISlNak7iJOAH6OnSHSHCvFNVMN/VT8NLiibTnHmR92Ouko3VaEfTYjcVo/qVEoF0 OI2L/Ow+ bsB3uZRsjwhUJshkGPaZkkJkRfBTuHtOp5qOR4L1fUOVzVNdA/7yiaJ6VCOVj1EnakGcKU3UIxUGlPFH0hMF0htywILN4uJ9a6eXuM4Q7ZR/xWAEfHzcpaMPLya+I9qleYCShfiB5WfSqlQfTgx5cfWxZw7weafolUaZFX2NaSLgDsMtyde0jmE+mcSHL6X5Jb8ewvKD3o1+RYybpfCPCOpbcJvJpbh5n0H2LbnIYie5JmSnisMmkotDXGcdrdofjVM6EL/cm2ZyLJSMUiS8Ky1iiL21REQ6WOqOhM8Eq9bLhqubWxBADe6AABnd7a7Dktq31otKNu8LprqZMy11+5ZZpI6ukJw571hAD6KSgJ974SHJq1uDpD3Nxe5AuVETCNmEI3RbYRDADzCI6GVhY1X4ljoAzU48NHeCHw7wW5zOrPCDpVVkUf3873l0kV58vyeiKzaKRBDhDzeSq03e+PGCc6YvGO5ExxmfVWR3aoJM2JzgOj4iwYVzMZVBGEBtOysIuA0AZSLKZ3OlehiNl2wA4Dw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Similarly to the direct DMA, bounce small allocations as they may have originated from a kmalloc() cache not safe for DMA. Unlike the direct DMA, iommu_dma_map_sg() cannot call iommu_dma_map_sg_swiotlb() for all non-coherent devices as this would break some cases where the iova is expected to be contiguous (dmabuf). Instead, scan the scatterlist for any small sizes and only go the swiotlb path if any element of the list needs bouncing (note that iommu_dma_map_page() would still only bounce those buffers which are not DMA-aligned). To avoid scanning the scatterlist on the 'sync' operations, introduce an SG_DMA_USE_SWIOTLB flag set by iommu_dma_map_sg_swiotlb(). The dev_use_swiotlb() function together with the newly added dev_use_sg_swiotlb() now check for both untrusted devices and unaligned kmalloc() buffers (suggested by Robin Murphy). Signed-off-by: Catalin Marinas Cc: Joerg Roedel Cc: Christoph Hellwig Cc: Robin Murphy Reviewed-by: Robin Murphy --- drivers/iommu/Kconfig | 1 + drivers/iommu/dma-iommu.c | 50 ++++++++++++++++++++++++++++++------- include/linux/scatterlist.h | 25 +++++++++++++++++-- 3 files changed, 65 insertions(+), 11 deletions(-) diff --git a/drivers/iommu/Kconfig b/drivers/iommu/Kconfig index db98c3f86e8c..670eff7a8e11 100644 --- a/drivers/iommu/Kconfig +++ b/drivers/iommu/Kconfig @@ -152,6 +152,7 @@ config IOMMU_DMA select IOMMU_IOVA select IRQ_MSI_IOMMU select NEED_SG_DMA_LENGTH + select NEED_SG_DMA_FLAGS if SWIOTLB # Shared Virtual Addressing config IOMMU_SVA diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c index 7a9f0b0bddbd..24a8b8c2368c 100644 --- a/drivers/iommu/dma-iommu.c +++ b/drivers/iommu/dma-iommu.c @@ -520,9 +520,38 @@ static bool dev_is_untrusted(struct device *dev) return dev_is_pci(dev) && to_pci_dev(dev)->untrusted; } -static bool dev_use_swiotlb(struct device *dev) +static bool dev_use_swiotlb(struct device *dev, size_t size, + enum dma_data_direction dir) { - return IS_ENABLED(CONFIG_SWIOTLB) && dev_is_untrusted(dev); + return IS_ENABLED(CONFIG_SWIOTLB) && + (dev_is_untrusted(dev) || + dma_kmalloc_needs_bounce(dev, size, dir)); +} + +static bool dev_use_sg_swiotlb(struct device *dev, struct scatterlist *sg, + int nents, enum dma_data_direction dir) +{ + struct scatterlist *s; + int i; + + if (!IS_ENABLED(CONFIG_SWIOTLB)) + return false; + + if (dev_is_untrusted(dev)) + return true; + + /* + * If kmalloc() buffers are not DMA-safe for this device and + * direction, check the individual lengths in the sg list. If any + * element is deemed unsafe, use the swiotlb for bouncing. + */ + if (!dma_kmalloc_safe(dev, dir)) { + for_each_sg(sg, s, nents, i) + if (!dma_kmalloc_size_aligned(s->length)) + return true; + } + + return false; } /** @@ -922,7 +951,7 @@ static void iommu_dma_sync_single_for_cpu(struct device *dev, { phys_addr_t phys; - if (dev_is_dma_coherent(dev) && !dev_use_swiotlb(dev)) + if (dev_is_dma_coherent(dev) && !dev_use_swiotlb(dev, size, dir)) return; phys = iommu_iova_to_phys(iommu_get_dma_domain(dev), dma_handle); @@ -938,7 +967,7 @@ static void iommu_dma_sync_single_for_device(struct device *dev, { phys_addr_t phys; - if (dev_is_dma_coherent(dev) && !dev_use_swiotlb(dev)) + if (dev_is_dma_coherent(dev) && !dev_use_swiotlb(dev, size, dir)) return; phys = iommu_iova_to_phys(iommu_get_dma_domain(dev), dma_handle); @@ -956,7 +985,7 @@ static void iommu_dma_sync_sg_for_cpu(struct device *dev, struct scatterlist *sg; int i; - if (dev_use_swiotlb(dev)) + if (sg_is_dma_use_swiotlb(sgl)) for_each_sg(sgl, sg, nelems, i) iommu_dma_sync_single_for_cpu(dev, sg_dma_address(sg), sg->length, dir); @@ -972,7 +1001,7 @@ static void iommu_dma_sync_sg_for_device(struct device *dev, struct scatterlist *sg; int i; - if (dev_use_swiotlb(dev)) + if (sg_is_dma_use_swiotlb(sgl)) for_each_sg(sgl, sg, nelems, i) iommu_dma_sync_single_for_device(dev, sg_dma_address(sg), @@ -998,7 +1027,8 @@ static dma_addr_t iommu_dma_map_page(struct device *dev, struct page *page, * If both the physical buffer start address and size are * page aligned, we don't need to use a bounce page. */ - if (dev_use_swiotlb(dev) && iova_offset(iovad, phys | size)) { + if (dev_use_swiotlb(dev, size, dir) && + iova_offset(iovad, phys | size)) { void *padding_start; size_t padding_size, aligned_size; @@ -1166,6 +1196,8 @@ static int iommu_dma_map_sg_swiotlb(struct device *dev, struct scatterlist *sg, struct scatterlist *s; int i; + sg_dma_mark_use_swiotlb(sg); + for_each_sg(sg, s, nents, i) { sg_dma_address(s) = iommu_dma_map_page(dev, sg_page(s), s->offset, s->length, dir, attrs); @@ -1210,7 +1242,7 @@ static int iommu_dma_map_sg(struct device *dev, struct scatterlist *sg, goto out; } - if (dev_use_swiotlb(dev)) + if (dev_use_sg_swiotlb(dev, sg, nents, dir)) return iommu_dma_map_sg_swiotlb(dev, sg, nents, dir, attrs); if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC)) @@ -1315,7 +1347,7 @@ static void iommu_dma_unmap_sg(struct device *dev, struct scatterlist *sg, struct scatterlist *tmp; int i; - if (dev_use_swiotlb(dev)) { + if (sg_is_dma_use_swiotlb(sg)) { iommu_dma_unmap_sg_swiotlb(dev, sg, nents, dir, attrs); return; } diff --git a/include/linux/scatterlist.h b/include/linux/scatterlist.h index 87aaf8b5cdb4..330a157c5501 100644 --- a/include/linux/scatterlist.h +++ b/include/linux/scatterlist.h @@ -248,6 +248,29 @@ static inline void sg_unmark_end(struct scatterlist *sg) sg->page_link &= ~SG_END; } +#define SG_DMA_BUS_ADDRESS (1 << 0) +#define SG_DMA_USE_SWIOTLB (1 << 1) + +#ifdef CONFIG_SWIOTLB +static inline bool sg_is_dma_use_swiotlb(struct scatterlist *sg) +{ + return sg->dma_flags & SG_DMA_USE_SWIOTLB; +} + +static inline void sg_dma_mark_use_swiotlb(struct scatterlist *sg) +{ + sg->dma_flags |= SG_DMA_USE_SWIOTLB; +} +#else +static inline bool sg_is_dma_use_swiotlb(struct scatterlist *sg) +{ + return false; +} +static inline void sg_dma_mark_use_swiotlb(struct scatterlist *sg) +{ +} +#endif + /* * CONFIG_PCI_P2PDMA depends on CONFIG_64BIT which means there is 4 bytes * in struct scatterlist (assuming also CONFIG_NEED_SG_DMA_LENGTH is set). @@ -256,8 +279,6 @@ static inline void sg_unmark_end(struct scatterlist *sg) */ #ifdef CONFIG_PCI_P2PDMA -#define SG_DMA_BUS_ADDRESS (1 << 0) - /** * sg_dma_is_bus address - Return whether a given segment was marked * as a bus address From patchwork Wed May 24 17:19:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13254353 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 817F1C77B7C for ; Wed, 24 May 2023 17:20:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1F204900011; Wed, 24 May 2023 13:20:11 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1A2CC900002; Wed, 24 May 2023 13:20:11 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F39D7900011; Wed, 24 May 2023 13:20:10 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id E4E97900002 for ; Wed, 24 May 2023 13:20:10 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id B4A174097F for ; Wed, 24 May 2023 17:20:10 +0000 (UTC) X-FDA: 80825811780.21.01C48DD Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf08.hostedemail.com (Postfix) with ESMTP id 0DA4E160029 for ; Wed, 24 May 2023 17:20:08 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=none; spf=pass (imf08.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684948809; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RYwKwBbFVhKuYYjB0bZAfV9ae8PLZlXqN8QaXAdpqek=; b=siBB1ZhhcNNPx6zEztb/+DjzWygrmq+71OefiiTuhmjQ8MJgL4KPEC4/8dACQ8Bbmp3pJA t7f7D00hKhRsfIlwGbWUoJH7K5BIsLQ39LxR50a6LOLoyT6KBsLTuRjCQOhaTMTyaYDF9d nDKgG1Xz5Khmnqmruw0j6slbb8ifaBc= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684948809; a=rsa-sha256; cv=none; b=yUp8WOHgAyo33llRhXIEjIuYeWsU4d1EC+YYAUXHiG2SU4WAIJlcCmxSZwr90/mR/qinHj ncF9yTYfc0Nhn9pAMBTkl6lpwo7cLadc7HRc59fGBiawHMGJ4FWYGfZMqIZmfOn0dusaIV EG9ygvoKnvrmvAaAUBZ6s4lCmvpgQT8= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=none; spf=pass (imf08.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 23C8363F80; Wed, 24 May 2023 17:20:08 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id DC7BDC4339B; Wed, 24 May 2023 17:20:03 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Christoph Hellwig , Robin Murphy Cc: Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 14/15] mm: slab: Reduce the kmalloc() minimum alignment if DMA bouncing possible Date: Wed, 24 May 2023 18:19:03 +0100 Message-Id: <20230524171904.3967031-15-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230524171904.3967031-1-catalin.marinas@arm.com> References: <20230524171904.3967031-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 0DA4E160029 X-Rspam-User: X-Stat-Signature: shn6fqcco6buppw64i6w9mfagiqkt8ji X-Rspamd-Server: rspam03 X-HE-Tag: 1684948808-361777 X-HE-Meta: U2FsdGVkX1/7qNBbnPMBRzNKoPKKlTAxUJJIQWvI3cDR4OpwxD5KC1RiLwP93QqX+lzWYttJu7m6epicBRgZCICPVwp8VFybg2qGhZS/QBj0FR91Qy52qj6bYL91m/dnlp9VHl7lPl/CY/bsnu6hpNajjlrkLQWT1BulsEzxxUvvzGvlTtzYwrw3l6h9Rurc0dC4APLwSvfYacjr3Kp6bNToIEPHMDM5SxniFZ1Rf6/8pUttz8eLn7CO8ccy7/m1i4LrfZCVagHpS2urDDVosfmuQ2hOi/34Mc3DUUUIml0RF2w9SpM5LlcTqGMateGDC8/2lsHHlKwKBtYVHtvoVbiO4NvoMh12IwG0VfEMKsdYRzgOu0KJ2OhcKsNRzqxjT2fpAXrUnXiKLY+rnpwmlAaWR0ulZJTlA0aMkEkx2s25dK7zWy7IauC23/d8Ko+2Bj8WEP6faCqDYeaFM/CoyBCaOriTBNmgrZiauG6Zo4OVhXwmFGIuKI7edKBO5RjHMNtJuoYdgu4byJhNGWPib/n2QGxVl5PjaefDGmWL+RQtUP94JimLYlhsPcGAX1J2ojU5p+qYNCky9zQpFvAqP1+o0V20sypOGRVQd/uW5oNCVw5Tt/iQxAiu8jJCYUuv88vRZgOOHQgCHftq25KyeQIcRh3nUmIyt26YIwBoNO87dau4vPGUeBXBmPjyGia2nTifafqWtC8Yk0M+KsP/ffMVr/VVmn2dHIVAX/PRrW3HZ3vRQr4mStleD6BUI4vamUpXgFIAx/2nzLdl0vA7sMPMzc515VCBnIP1LSlfZQg9XwybUNCyO1YkrYmA4xt3j9g1s+/o9vMVqTe3ORNCQYJFqZVnsaZk4R01O51leg+bhafeGSNesAevkI8B3NhZuEvnE3MevrLITb+kMmPbT9zUEWMTieIT5DIMhBvn6lhvk3ifUWDd8cY3KDGsaeIzEelde6udBKx+vF9RDvB SkckaedE t6aly/02bR1dkzQz0aNmDdhnc4yk0AGdec7Q63f2C9hEe1/G5fyZjHaqUxkv5PtQVKTxhrROiSAiZtLun/cawBCmLNr8OQ9AebQaHho59WNoEiSWndUoyKkm8c0K/DtTayJZeru5xCSPZJrCVDGJXWcfOkpVkh03hT3x4lFER6/djB3WXmfig3hUBsEn1gLaMajNwh9j1blXG9Z9jhHH/a1b+/gCbMxlpz37mFd8G4+ggx0xZ/1z4L0ouVRSwJgMooUyJKXsiwOo1vNzF17xG44G5WT5W4Y7duh5juWFLwIiRLq79y6eRlbE4RSxVFhJV2pkuG+p8RnkiI91pcGyF7byMm0aEp/5KNT4tYwwjU/chLOUwk30ZvC5AcGH3ZsRqJOR+3oAqgG0DKpm4VZLx5fkGKlLm5in8bFA91lXftb2vRbbNyVxK+jU9Dq3TUazi07nqt5zYPt5MHeCiEuK7SKulaig1nfEKuugoSTXM0eh9MrJ6rCCL7IPRfbPaiOFdN6ZMVN5xVSj2EjjpK5eEq3VSBA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: If an architecture opted in to DMA bouncing of unaligned kmalloc() buffers (ARCH_WANT_KMALLOC_DMA_BOUNCE), reduce the minimum kmalloc() cache alignment below cache-line size to ARCH_KMALLOC_MINALIGN. Signed-off-by: Catalin Marinas Cc: Andrew Morton Cc: Christoph Hellwig Cc: Robin Murphy --- mm/slab_common.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/mm/slab_common.c b/mm/slab_common.c index 7c6475847fdf..fe46459a8b77 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -18,6 +18,7 @@ #include #include #include +#include #include #include #include @@ -863,10 +864,19 @@ void __init setup_kmalloc_cache_index_table(void) } } +#ifdef CONFIG_DMA_BOUNCE_UNALIGNED_KMALLOC +static unsigned int __kmalloc_minalign(void) +{ + if (io_tlb_default_mem.nslabs) + return ARCH_KMALLOC_MINALIGN; + return dma_get_cache_alignment(); +} +#else static unsigned int __kmalloc_minalign(void) { return dma_get_cache_alignment(); } +#endif void __init new_kmalloc_cache(int idx, enum kmalloc_cache_type type, slab_flags_t flags) From patchwork Wed May 24 17:19:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13254354 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BF844C77B7C for ; Wed, 24 May 2023 17:20:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 611A2900012; Wed, 24 May 2023 13:20:15 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5E82E900002; Wed, 24 May 2023 13:20:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4FDDB900012; Wed, 24 May 2023 13:20:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 3B1B2900002 for ; Wed, 24 May 2023 13:20:15 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 05AED1A0A58 for ; Wed, 24 May 2023 17:20:15 +0000 (UTC) X-FDA: 80825811990.10.8155722 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf01.hostedemail.com (Postfix) with ESMTP id 14D3E40013 for ; Wed, 24 May 2023 17:20:12 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf01.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684948813; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QcxrCBYims99sTaWtbsc4IRs+7tDysGBphQN4oXq1fE=; b=a5wbVOyVu+R42JZwNuASQS11bQegshBwSAeAoOeuUc8/vCBmc2xfH13aC9QAdpthGUiVdl ugfSCsxCpJNtJBsIHn2ZtAUCOhXiJCoG7+j/AKf4EAwE5M/MXJpmp55GApUmpz6/+lrkI6 dQ7dfy0jONdh6zEeegSaYJ2UPK55HRs= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf01.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684948813; a=rsa-sha256; cv=none; b=HafLykPaINuKGJhrxXwu7Gyt6QVITY3PICUDNwyA7zkhlZdw6OXXQoZPAI28HUPvoj+rnC BfSF3YWSMK1z8o5+PZh6BiDwf484kuLAJs4pKi4mrr7ieITha36JMvrS2zUDrL8cDEb6kP nnVLnfOGN//wbnuEyMs898PhE2ZhA1I= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 3CA2E638B6; Wed, 24 May 2023 17:20:12 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id EEE70C433AC; Wed, 24 May 2023 17:20:07 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Christoph Hellwig , Robin Murphy Cc: Arnd Bergmann , Greg Kroah-Hartman , Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 15/15] arm64: Enable ARCH_WANT_KMALLOC_DMA_BOUNCE for arm64 Date: Wed, 24 May 2023 18:19:04 +0100 Message-Id: <20230524171904.3967031-16-catalin.marinas@arm.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230524171904.3967031-1-catalin.marinas@arm.com> References: <20230524171904.3967031-1-catalin.marinas@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 14D3E40013 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: qwfazsn5fa33x6dnkjz7exeycz7xkggj X-HE-Tag: 1684948812-586172 X-HE-Meta: U2FsdGVkX1+M/h3UyWogu9jKfbRcFAK8UtR+eJiKMller1AlqMg+Qgv3T1nXoe+3ZbahWMN7JFTjuDrWqfILjZ8b3zfEVBMVO7CEWciHBl2leDsiuepJYXYTPByaIjPO+lcN36G40aPJACBwppsLyE1MVAW23cjlyDBcaOJf8QtMyup511WbrZSva2/ZBNQSZ+0F3XPtvu1jblyVUtxLfD4NNKCit3SyGX0E4Rm71RQJOatR1RsirCmW5Jr8Z+wXpe+eyZkM/8dU2SyIXytrTc1DDNlct4VP+8Pe6/Y14u7aypn1/9hdGlPlAimbodzj2qXb3TrIMEFK0K4Kshv5h2xaLw3dPkH2l8DinGjPmBfLwjzW6i7pYzmj3A0nhIJWUleJo9j5DzW2PGZaOEsURpdRNpgrbL1JWp9M6r6UFn2b3ugph6+KsQipuDjuQfVXaKw8jmGio1rC7SinCORPe6YyhSEQNkH0RxWHyM0vrz/F16oPLn0X3x7JXmq0tpNKlvrm24N5hwXAdbrtkNXzZGDoN2Iy0drPnUYhD1h4M7cp8kisW0PGLAdulrr8Vyx6Rel7eFO8hp75TO14lKq88Rr/yKWdc+rgUz+24zbcHAhSa6foCYramOgiGi56v62/K/KJlH++IdvCEFoT8e48u4RaanmbDd6VTl81N3xGp34QO8SaYGwt5mAeuSwE44hflNtMRu/tk80VaoJp/JBMdShHdvLU05f6eUL6tld/JHglb6wqVhS8bZk6pGOoDdO72k87n8H/2hIshxrjYC3APLw+/qva4N1Ctd6/liFIu/ACQS0heTkv7vVNCE0J1RwXbPRvPejEfbX3c48zhLDt/9JnPI8EXGBSx6A7jUZxYbbJog58zXbRqdKYKOiVybcpyYSp1+ten8kk074VsUK6+fHtsPWh3IFfsoRSSiD3G17BvY/8R5JmWG3ya9eLxurAwv4f70QDz0BScKqVwLh gJk554qh Lq8QXKC28HDXp3MljCnIJSTRqcRVEQwl6YwCKZlfbz20QQCbW/gBU9KrIaOOU6yoGY6JjpQ3zj1bzmZuh+2DnGIw9ZGkKVW6dHPRbMyE09uzLpBED5t9/RSZzHECCXvr3LJVphWutZvdD0J/gpJuxGZGP9JZq1wJ1IQ/uypcUV5vQMV5mhnd06ulq0JlIJtKEbPKitFd34XUOPI4fVPZy4No8j+iMRYOUpeaVp6yV33VidSwmXRu6yKdU23tzadZq4sbKe0ORwmIGLfhpoZCjykKkkLx/1t0xClf7gJHo/7gMBy7MyOPPAkeSD/GaJcTh0cqF3JbJSn1fQFwhEENjMmsqe0FZ3z7tSh71f3XkvHSg0datuqqphVBn72yy5t1WmZygouAwwoaIf1p57c963IpFDo96ZYwgkX50FfcIhx7hfz5RJNycULAexvnhtKFxVUP8azuWiLnzsJ6yXeNj0YcdVWPBBdZz2dk6f3v2TBUM91+2FODAEMFF3dnbt9fcxIWtqKscZhe4HbNtFX9hg1hCuYUktN1ADZpliGxwSbz/nJv8T14W+Mn5gXEO/Jra6Q6s X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: With the DMA bouncing of unaligned kmalloc() buffers now in place, enable it for arm64 to allow the kmalloc-{8,16,32,48,96} caches. In addition, always create the swiotlb buffer even when the end of RAM is within the 32-bit physical address range (the swiotlb buffer can still be disabled on the kernel command line). Signed-off-by: Catalin Marinas Cc: Will Deacon --- arch/arm64/Kconfig | 1 + arch/arm64/mm/init.c | 7 ++++++- 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index b1201d25a8a4..af42871431c0 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -120,6 +120,7 @@ config ARM64 select CRC32 select DCACHE_WORD_ACCESS select DYNAMIC_FTRACE if FUNCTION_TRACER + select DMA_BOUNCE_UNALIGNED_KMALLOC select DMA_DIRECT_REMAP select EDAC_SUPPORT select FRAME_POINTER diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c index 66e70ca47680..3ac2e9d79ce4 100644 --- a/arch/arm64/mm/init.c +++ b/arch/arm64/mm/init.c @@ -442,7 +442,12 @@ void __init bootmem_init(void) */ void __init mem_init(void) { - swiotlb_init(max_pfn > PFN_DOWN(arm64_dma_phys_limit), SWIOTLB_VERBOSE); + bool swiotlb = max_pfn > PFN_DOWN(arm64_dma_phys_limit); + + if (IS_ENABLED(CONFIG_DMA_BOUNCE_UNALIGNED_KMALLOC)) + swiotlb = true; + + swiotlb_init(swiotlb, SWIOTLB_VERBOSE); /* this will put all unused low memory onto the freelists */ memblock_free_all();