From patchwork Sun Nov 6 22:01:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13033621 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 04F96C4332F for ; Sun, 6 Nov 2022 22:01:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 97B4C8E0003; Sun, 6 Nov 2022 17:01:58 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 952168E0001; Sun, 6 Nov 2022 17:01:58 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7F3E38E0003; Sun, 6 Nov 2022 17:01:58 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 70BD98E0001 for ; Sun, 6 Nov 2022 17:01:58 -0500 (EST) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 39957804B9 for ; Sun, 6 Nov 2022 22:01:58 +0000 (UTC) X-FDA: 80104390716.17.647ED54 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by imf14.hostedemail.com (Postfix) with ESMTP id 9B3FB100003 for ; Sun, 6 Nov 2022 22:01:56 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 2B864B80D3A; Sun, 6 Nov 2022 22:01:55 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2F350C43143; Sun, 6 Nov 2022 22:01:50 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Arnd Bergmann , Christoph Hellwig , Greg Kroah-Hartman Cc: Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Robin Murphy , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v3 01/13] mm/slab: Decouple ARCH_KMALLOC_MINALIGN from ARCH_DMA_MINALIGN Date: Sun, 6 Nov 2022 22:01:31 +0000 Message-Id: <20221106220143.2129263-2-catalin.marinas@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221106220143.2129263-1-catalin.marinas@arm.com> References: <20221106220143.2129263-1-catalin.marinas@arm.com> MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1667772116; a=rsa-sha256; cv=none; b=KN9YXXH+ipUxhHohAa+xIISIV3302L2kmWBOyZKD2mxALuGq0+rspwfNImoqE9Z/jD70sr rHmx4kMq7q2yAinUwwHNmsvjIxvycMj9XZ8ZaChmy/GN/xkPRnXw/gyFauAVEfqZE6jLac W39J9+xuJWS2DVuLXnoTjX822yorihI= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=none; spf=pass (imf14.hostedemail.com: domain of cmarinas@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1667772116; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pu9pl4T1W8DhVGZtHHfFfqYRWCs6qPYIUbfiu3tZOSM=; b=rttRxG1Cw4jy3D/tjPfGSzu5OULYbKy3/KmWgWpD8deaBG18AqAJ22W/bM4HiHHh92MkI+ dhNGLC/o5gW25oi3AcwDaUnWW0+eEX/U9xZ9rJBCEtWO5WKd6BvETiu4qnmJbbSGmlumky /ckNDekCI4bsUAuUL8ZGE9V5I6tARgk= Authentication-Results: imf14.hostedemail.com; dkim=none; spf=pass (imf14.hostedemail.com: domain of cmarinas@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) X-Rspamd-Server: rspam12 X-Rspam-User: X-Stat-Signature: 5dcnga19c394fy9tdwbnryd5jyysntjn X-Rspamd-Queue-Id: 9B3FB100003 X-HE-Tag: 1667772116-546115 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: In preparation for supporting a kmalloc() minimum alignment smaller than the arch DMA alignment, decouple the two definitions. This requires that the DMA API bounces smaller kmalloc() allocations, hence the smaller ARCH_KMALLOC_MINALIGN is only enabled if a new CONFIG_DMA_BOUNCE_UNALIGNED_KMALLOC option is enabled (to be introduced in a subsequent patch). After this patch, ARCH_DMA_MINALIGN is expected to be used in static alignment annotations and defined by an architecture to be the maximum alignment for all supported configurations/SoCs in a single Image. ARCH_KMALLOC_MINALIGN becomes the default sizeof(unsigned long long). Since ARCH_DMA_MINALIGN is now always defined, adjust the #ifdef in dma_get_cache_alignment() so that there is no change for architectures not requiring a minimum DMA alignment. Signed-off-by: Catalin Marinas Cc: Andrew Morton Cc: Christoph Hellwig Cc: Robin Murphy --- include/linux/dma-mapping.h | 2 +- include/linux/slab.h | 14 +++++++++++--- 2 files changed, 12 insertions(+), 4 deletions(-) diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h index 0ee20b764000..3288a1339271 100644 --- a/include/linux/dma-mapping.h +++ b/include/linux/dma-mapping.h @@ -545,7 +545,7 @@ static inline int dma_set_min_align_mask(struct device *dev, static inline int dma_get_cache_alignment(void) { -#ifdef ARCH_DMA_MINALIGN +#ifdef ARCH_HAS_DMA_MINALIGN return ARCH_DMA_MINALIGN; #endif return 1; diff --git a/include/linux/slab.h b/include/linux/slab.h index 90877fcde70b..b104d63e5456 100644 --- a/include/linux/slab.h +++ b/include/linux/slab.h @@ -221,12 +221,20 @@ void kmem_dump_obj(void *object); * alignment larger than the alignment of a 64-bit integer. * Setting ARCH_DMA_MINALIGN in arch headers allows that. */ -#if defined(ARCH_DMA_MINALIGN) && ARCH_DMA_MINALIGN > 8 +#ifdef ARCH_DMA_MINALIGN +#define ARCH_HAS_DMA_MINALIGN +#if ARCH_DMA_MINALIGN > 8 && !defined(CONFIG_DMA_BOUNCE_UNALIGNED_KMALLOC) #define ARCH_KMALLOC_MINALIGN ARCH_DMA_MINALIGN -#define KMALLOC_MIN_SIZE ARCH_DMA_MINALIGN -#define KMALLOC_SHIFT_LOW ilog2(ARCH_DMA_MINALIGN) +#endif #else +#define ARCH_DMA_MINALIGN __alignof__(unsigned long long) +#endif + +#ifndef ARCH_KMALLOC_MINALIGN #define ARCH_KMALLOC_MINALIGN __alignof__(unsigned long long) +#else +#define KMALLOC_MIN_SIZE ARCH_KMALLOC_MINALIGN +#define KMALLOC_SHIFT_LOW ilog2(KMALLOC_MIN_SIZE) #endif /* From patchwork Sun Nov 6 22:01:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13033622 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6BA10C4332F for ; Sun, 6 Nov 2022 22:02:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D72E48E0005; Sun, 6 Nov 2022 17:02:01 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D23288E0001; Sun, 6 Nov 2022 17:02:01 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C11CF8E0005; Sun, 6 Nov 2022 17:02:01 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id B50F58E0001 for ; Sun, 6 Nov 2022 17:02:01 -0500 (EST) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 744C71208E5 for ; Sun, 6 Nov 2022 22:02:01 +0000 (UTC) X-FDA: 80104390842.10.FADABFE Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by imf22.hostedemail.com (Postfix) with ESMTP id D9ADEC000A for ; Sun, 6 Nov 2022 22:02:00 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 585E2B80D38; Sun, 6 Nov 2022 22:01:59 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 37993C433B5; Sun, 6 Nov 2022 22:01:54 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Arnd Bergmann , Christoph Hellwig , Greg Kroah-Hartman Cc: Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Robin Murphy , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v3 02/13] dma-mapping: Force bouncing if the kmalloc() size is not cacheline-aligned Date: Sun, 6 Nov 2022 22:01:32 +0000 Message-Id: <20221106220143.2129263-3-catalin.marinas@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221106220143.2129263-1-catalin.marinas@arm.com> References: <20221106220143.2129263-1-catalin.marinas@arm.com> MIME-Version: 1.0 ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1667772121; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PvipVnjKjzWqPzfgQSoQHSuW6iWoAw45fZnbTXFvWx4=; b=UJ+cHo7Mr5NmUBPVUpxaTD/1RHAFPfmmUfwirXjSRwKGNywb9aNsedGezhacZijE1h5oTj k9+3u001a8Rebu/YVCJdbKvKE6ga0NanO9/ge/ndBB+YO9+fjpg7gvkQJj4XGnOUKIcM95 bhCMQUAv6KaD14HuRO8Y7E3TlOcnRno= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=none; spf=pass (imf22.hostedemail.com: domain of cmarinas@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1667772121; a=rsa-sha256; cv=none; b=IotFKiXMU5bVJPS6V1pTzDCrSHYnPn8S6H8Sdj98bmKbH+a+NQ3Jb65a/IoreUJvcmYfs3 9hkZ1JmnQrcNQTZ1Dbrmpx59MYj6FYvArkzk4Iq+UDkG+QwCXz3d+uRgG38m74M0cT5LeK cdL0W5qYKKum21vdYXpKkcPy2O+/kLs= Authentication-Results: imf22.hostedemail.com; dkim=none; spf=pass (imf22.hostedemail.com: domain of cmarinas@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) X-Rspam-User: X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: D9ADEC000A X-Stat-Signature: 9oodq7r9e41zsnab87jnjpka4tzwop8n X-HE-Tag: 1667772120-740431 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: For direct DMA, if the size is small enough to have originated from a kmalloc() cache below ARCH_DMA_MINALIGN, check its alignment against cache_line_size() and bounce if necessary. For larger sizes, it is the responsibility of the DMA API caller to ensure proper alignment. Signed-off-by: Catalin Marinas Cc: Christoph Hellwig Cc: Robin Murphy --- include/linux/dma-map-ops.h | 27 +++++++++++++++++++++++++++ kernel/dma/direct.h | 3 ++- 2 files changed, 29 insertions(+), 1 deletion(-) diff --git a/include/linux/dma-map-ops.h b/include/linux/dma-map-ops.h index d678afeb8a13..785f7aa90f57 100644 --- a/include/linux/dma-map-ops.h +++ b/include/linux/dma-map-ops.h @@ -8,6 +8,7 @@ #include #include +#include struct cma; @@ -275,6 +276,32 @@ static inline bool dev_is_dma_coherent(struct device *dev) } #endif /* CONFIG_ARCH_HAS_DMA_COHERENCE_H */ +/* + * Check whether the given size, assuming it is for a kmalloc()'ed object, is + * safe for non-coherent DMA or needs bouncing. + */ +static inline bool dma_kmalloc_needs_bounce(struct device *dev, size_t size, + enum dma_data_direction dir) +{ + /* + * No need for bouncing if coherent DMA or the direction is + * DMA_TO_DEVICE. + */ + if (!IS_ENABLED(CONFIG_DMA_BOUNCE_UNALIGNED_KMALLOC) || + dir == DMA_TO_DEVICE || dev_is_dma_coherent(dev)) + return false; + + /* + * Larger kmalloc() sizes are guaranteed to be aligned to + * ARCH_DMA_MINALIGN. + */ + if (size >= 2 * ARCH_DMA_MINALIGN || + IS_ALIGNED(kmalloc_size_roundup(size), dma_get_cache_alignment())) + return false; + + return true; +} + void *arch_dma_alloc(struct device *dev, size_t size, dma_addr_t *dma_handle, gfp_t gfp, unsigned long attrs); void arch_dma_free(struct device *dev, size_t size, void *cpu_addr, diff --git a/kernel/dma/direct.h b/kernel/dma/direct.h index e38ffc5e6bdd..97ec892ea0b5 100644 --- a/kernel/dma/direct.h +++ b/kernel/dma/direct.h @@ -94,7 +94,8 @@ static inline dma_addr_t dma_direct_map_page(struct device *dev, return swiotlb_map(dev, phys, size, dir, attrs); } - if (unlikely(!dma_capable(dev, dma_addr, size, true))) { + if (unlikely(!dma_capable(dev, dma_addr, size, true)) || + dma_kmalloc_needs_bounce(dev, size, dir)) { if (is_pci_p2pdma_page(page)) return DMA_MAPPING_ERROR; if (is_swiotlb_active(dev)) From patchwork Sun Nov 6 22:01:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13033623 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AA0B7C433FE for ; Sun, 6 Nov 2022 22:02:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 44B9A8E0006; Sun, 6 Nov 2022 17:02:04 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3D4288E0001; Sun, 6 Nov 2022 17:02:04 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 275748E0006; Sun, 6 Nov 2022 17:02:04 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 1AE788E0001 for ; Sun, 6 Nov 2022 17:02:04 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id CB70F1208E5 for ; Sun, 6 Nov 2022 22:02:03 +0000 (UTC) X-FDA: 80104390926.08.B835DBB Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf18.hostedemail.com (Postfix) with ESMTP id 782571C0005 for ; Sun, 6 Nov 2022 22:02:03 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 8877560DD5; Sun, 6 Nov 2022 22:02:02 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 49473C4347C; Sun, 6 Nov 2022 22:01:58 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Arnd Bergmann , Christoph Hellwig , Greg Kroah-Hartman Cc: Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Robin Murphy , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v3 03/13] iommu/dma: Force bouncing of the size is not cacheline-aligned Date: Sun, 6 Nov 2022 22:01:33 +0000 Message-Id: <20221106220143.2129263-4-catalin.marinas@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221106220143.2129263-1-catalin.marinas@arm.com> References: <20221106220143.2129263-1-catalin.marinas@arm.com> MIME-Version: 1.0 ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=none; spf=pass (imf18.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1667772123; a=rsa-sha256; cv=none; b=7SejZy3OxXMDmUGwzTknjYpjokhgxMw/hH4kwbzM1wtOuzP+WFMeNBwh9FbxRjgpS/bk+K L5J+gRcOTXJ9ykszatf3mZnc+twgfbGc2yWCmpiDINaRp3Wt7e324zpsuWoL2WvOP1ISUA IoJHNEcs2nmLMp24hFZupC4reiM0KVI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1667772123; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=87uBk1AAuXY6JFqVGEykvsue21YAiRpJqc5IBvHuWMY=; b=M21QPVw6aXVkeJh6uKh7HwYgerI9EDlVmlIM5rxz8gapRkYobaXfli8ijDDXFbXIA7pusx xmldtrKpxE9M6C22pRG9KmgbB15l4bUlqLelgJOP336T1qersyPzEV+1vCbWMat3Vf9bRr wReEzTwRoqWcQmQM21pNA4Ek8s6RPC8= X-Stat-Signature: citn5j6ntshrt9qdt4dhti7kd1gnz7y6 X-Rspamd-Queue-Id: 782571C0005 Authentication-Results: imf18.hostedemail.com; dkim=none; spf=pass (imf18.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) X-Rspamd-Server: rspam05 X-Rspam-User: X-HE-Tag: 1667772123-634794 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Similarly to the direct DMA, bounce small allocations as they may have originated from a kmalloc() cache not safe for DMA. Unlike the direct DMA, iommu_dma_map_sg() cannot call iommu_dma_map_sg_swiotlb() for all non-coherent devices as this would break some cases where the iova is expected to be contiguous (dmabuf). Instead, scan the scatterlist for any small sizes and only go the swiotlb path if any element of the list needs bouncing (note that iommu_dma_map_page() would still only bounce those buffers which are not DMA-aligned). To avoid scanning the scatterlist on the 'sync' operations, introduce a SG_DMA_BOUNCED flag set during the iommu_dma_map_sg() call (suggested by Robin Murphy). Signed-off-by: Catalin Marinas Cc: Joerg Roedel Cc: Christoph Hellwig Cc: Robin Murphy Signed-off-by: Robin Murphy --- Not entirely sure about this approach but here it is. And it needs better testing. drivers/iommu/dma-iommu.c | 12 ++++++++---- include/linux/dma-map-ops.h | 23 +++++++++++++++++++++++ include/linux/scatterlist.h | 27 ++++++++++++++++++++++++--- 3 files changed, 55 insertions(+), 7 deletions(-) diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c index 9297b741f5e8..8c80dffe0337 100644 --- a/drivers/iommu/dma-iommu.c +++ b/drivers/iommu/dma-iommu.c @@ -948,7 +948,7 @@ static void iommu_dma_sync_sg_for_cpu(struct device *dev, struct scatterlist *sg; int i; - if (dev_use_swiotlb(dev)) + if (dev_use_swiotlb(dev) || sg_is_dma_bounced(sgl)) for_each_sg(sgl, sg, nelems, i) iommu_dma_sync_single_for_cpu(dev, sg_dma_address(sg), sg->length, dir); @@ -964,7 +964,7 @@ static void iommu_dma_sync_sg_for_device(struct device *dev, struct scatterlist *sg; int i; - if (dev_use_swiotlb(dev)) + if (dev_use_swiotlb(dev) || sg_is_dma_bounced(sgl)) for_each_sg(sgl, sg, nelems, i) iommu_dma_sync_single_for_device(dev, sg_dma_address(sg), @@ -990,7 +990,8 @@ static dma_addr_t iommu_dma_map_page(struct device *dev, struct page *page, * If both the physical buffer start address and size are * page aligned, we don't need to use a bounce page. */ - if (dev_use_swiotlb(dev) && iova_offset(iovad, phys | size)) { + if ((dev_use_swiotlb(dev) && iova_offset(iovad, phys | size)) || + dma_kmalloc_needs_bounce(dev, size, dir)) { void *padding_start; size_t padding_size, aligned_size; @@ -1202,7 +1203,10 @@ static int iommu_dma_map_sg(struct device *dev, struct scatterlist *sg, goto out; } - if (dev_use_swiotlb(dev)) + if (dma_sg_kmalloc_needs_bounce(dev, sg, nents, dir)) + sg_dma_mark_bounced(sg); + + if (dev_use_swiotlb(dev) || sg_is_dma_bounced(sg)) return iommu_dma_map_sg_swiotlb(dev, sg, nents, dir, attrs); if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC)) diff --git a/include/linux/dma-map-ops.h b/include/linux/dma-map-ops.h index 785f7aa90f57..e747a46261d4 100644 --- a/include/linux/dma-map-ops.h +++ b/include/linux/dma-map-ops.h @@ -302,6 +302,29 @@ static inline bool dma_kmalloc_needs_bounce(struct device *dev, size_t size, return true; } +/* + * Return true if any of the scatterlist elements needs bouncing due to + * potentially originating from a small kmalloc() cache. + */ +static inline bool dma_sg_kmalloc_needs_bounce(struct device *dev, + struct scatterlist *sg, int nents, + enum dma_data_direction dir) +{ + struct scatterlist *s; + int i; + + if (!IS_ENABLED(CONFIG_DMA_BOUNCE_UNALIGNED_KMALLOC) || + dir == DMA_TO_DEVICE || dev_is_dma_coherent(dev)) + return false; + + for_each_sg(sg, s, nents, i) { + if (dma_kmalloc_needs_bounce(dev, s->length, dir)) + return true; + } + + return false; +} + void *arch_dma_alloc(struct device *dev, size_t size, dma_addr_t *dma_handle, gfp_t gfp, unsigned long attrs); void arch_dma_free(struct device *dev, size_t size, void *cpu_addr, diff --git a/include/linux/scatterlist.h b/include/linux/scatterlist.h index 375a5e90d86a..f16cf040fe2c 100644 --- a/include/linux/scatterlist.h +++ b/include/linux/scatterlist.h @@ -16,7 +16,7 @@ struct scatterlist { #ifdef CONFIG_NEED_SG_DMA_LENGTH unsigned int dma_length; #endif -#ifdef CONFIG_PCI_P2PDMA +#if defined(CONFIG_PCI_P2PDMA) || defined(CONFIG_DMA_BOUNCE_UNALIGNED_KMALLOC) unsigned int dma_flags; #endif }; @@ -248,6 +248,29 @@ static inline void sg_unmark_end(struct scatterlist *sg) sg->page_link &= ~SG_END; } +#define SG_DMA_BUS_ADDRESS (1 << 0) +#define SG_DMA_BOUNCED (1 << 1) + +#ifdef CONFIG_DMA_BOUNCE_UNALIGNED_KMALLOC +static inline bool sg_is_dma_bounced(struct scatterlist *sg) +{ + return sg->dma_flags & SG_DMA_BOUNCED; +} + +static inline void sg_dma_mark_bounced(struct scatterlist *sg) +{ + sg->dma_flags |= SG_DMA_BOUNCED; +} +#else +static inline bool sg_is_dma_bounced(struct scatterlist *sg) +{ + return false; +} +static inline void sg_dma_mark_bounced(struct scatterlist *sg) +{ +} +#endif + /* * CONFGI_PCI_P2PDMA depends on CONFIG_64BIT which means there is 4 bytes * in struct scatterlist (assuming also CONFIG_NEED_SG_DMA_LENGTH is set). @@ -256,8 +279,6 @@ static inline void sg_unmark_end(struct scatterlist *sg) */ #ifdef CONFIG_PCI_P2PDMA -#define SG_DMA_BUS_ADDRESS (1 << 0) - /** * sg_dma_is_bus address - Return whether a given segment was marked * as a bus address From patchwork Sun Nov 6 22:01:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13033624 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 29754C4332F for ; Sun, 6 Nov 2022 22:02:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B77918E0007; Sun, 6 Nov 2022 17:02:09 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B26F58E0001; Sun, 6 Nov 2022 17:02:09 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9EF668E0007; Sun, 6 Nov 2022 17:02:09 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 925D98E0001 for ; Sun, 6 Nov 2022 17:02:09 -0500 (EST) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 57B00AAABF for ; Sun, 6 Nov 2022 22:02:09 +0000 (UTC) X-FDA: 80104391178.22.54169BC Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by imf30.hostedemail.com (Postfix) with ESMTP id CB81C80006 for ; Sun, 6 Nov 2022 22:02:08 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 5C4DCB80D3D; Sun, 6 Nov 2022 22:02:07 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5D987C4314A; Sun, 6 Nov 2022 22:02:02 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Arnd Bergmann , Christoph Hellwig , Greg Kroah-Hartman Cc: Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Robin Murphy , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v3 04/13] mm/slab: Allow kmalloc() minimum alignment fallback to dma_get_cache_alignment() Date: Sun, 6 Nov 2022 22:01:34 +0000 Message-Id: <20221106220143.2129263-5-catalin.marinas@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221106220143.2129263-1-catalin.marinas@arm.com> References: <20221106220143.2129263-1-catalin.marinas@arm.com> MIME-Version: 1.0 ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1667772129; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hb/JMHhuvRMmjdAuUa9IyIxKSZUdxwl7w/JwapC56BM=; b=1Jq5gFA0cNeZld9aeRGAbwg5clgMhcpgJ/02MUt3I+ljTFjdiL207T+ie5n75HTYuar1Hx kpvGXYFWZfaMupuBBBenxjaWlyUrc2vg8zf6a17vxkVxFtKOaGtJiFSE3Wq1B3BxElF29o dqbDtxOPShZsmv9I58GjVZ8kGng5yzw= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=none; spf=pass (imf30.hostedemail.com: domain of cmarinas@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1667772129; a=rsa-sha256; cv=none; b=Fve3BrztlSKkZcEv9XnuQKcRZHQrNUcYBmKfOyBXtFM3DwzP4NlhACeTbhDkZrOqf8KyPC tElvH96TSLbaB+nDKroAspV7cXyasE5A7XCfeNwBXVvDMmrW/w/ZIWBn+QUereDhhvNdJl 7ky2uzT26FBrBdLQVF+KeeKqee4lxXI= Authentication-Results: imf30.hostedemail.com; dkim=none; spf=pass (imf30.hostedemail.com: domain of cmarinas@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) X-Rspam-User: X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: CB81C80006 X-Stat-Signature: 76qx4ns7ebqss39q5c8wopgcx6j31ioq X-HE-Tag: 1667772128-500277 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: For architecture with ARCH_DMA_MINALIGN larger than ARCH_KMALLOC_MINALIGN, if no default swiotlb buffer is available, fall back to a kmalloc() minimum alignment which is safe for DMA. Signed-off-by: Catalin Marinas Cc: Andrew Morton --- I'm not particularly fond of the slab_common.c code probing the default swiotlb. In one incarnation I had a __weak arch_kmalloc_minalign() overridden by the arch code but decided there wasn't anything arm64 specific in there. mm/slab.c | 6 +----- mm/slab.h | 2 ++ mm/slab_common.c | 40 +++++++++++++++++++++++++++++++++++----- 3 files changed, 38 insertions(+), 10 deletions(-) diff --git a/mm/slab.c b/mm/slab.c index 59c8e28f7b6a..6e31eb027ef6 100644 --- a/mm/slab.c +++ b/mm/slab.c @@ -1241,11 +1241,7 @@ void __init kmem_cache_init(void) * Initialize the caches that provide memory for the kmem_cache_node * structures first. Without this, further allocations will bug. */ - kmalloc_caches[KMALLOC_NORMAL][INDEX_NODE] = create_kmalloc_cache( - kmalloc_info[INDEX_NODE].name[KMALLOC_NORMAL], - kmalloc_info[INDEX_NODE].size, - ARCH_KMALLOC_FLAGS, 0, - kmalloc_info[INDEX_NODE].size); + new_kmalloc_cache(INDEX_NODE, KMALLOC_NORMAL, ARCH_KMALLOC_FLAGS); slab_state = PARTIAL_NODE; setup_kmalloc_cache_index_table(); diff --git a/mm/slab.h b/mm/slab.h index 0202a8c2f0d2..d0460e0f6760 100644 --- a/mm/slab.h +++ b/mm/slab.h @@ -288,6 +288,8 @@ int __kmem_cache_create(struct kmem_cache *, slab_flags_t flags); struct kmem_cache *create_kmalloc_cache(const char *name, unsigned int size, slab_flags_t flags, unsigned int useroffset, unsigned int usersize); +void __init new_kmalloc_cache(int idx, enum kmalloc_cache_type type, + slab_flags_t flags); extern void create_boot_cache(struct kmem_cache *, const char *name, unsigned int size, slab_flags_t flags, unsigned int useroffset, unsigned int usersize); diff --git a/mm/slab_common.c b/mm/slab_common.c index 33b1886b06eb..b62f27c2dda7 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -17,6 +17,8 @@ #include #include #include +#include +#include #include #include #include @@ -852,9 +854,30 @@ void __init setup_kmalloc_cache_index_table(void) } } -static void __init +static unsigned int __kmalloc_minalign(void) +{ + int cache_align = dma_get_cache_alignment(); + + /* + * If CONFIG_DMA_BOUNCE_UNALIGNED_KMALLOC is not enabled, + * ARCH_KMALLOC_MINALIGN matches ARCH_DMA_MINALIGN. + */ + if (!IS_ENABLED(CONFIG_DMA_BOUNCE_UNALIGNED_KMALLOC) || + cache_align < ARCH_KMALLOC_MINALIGN || io_tlb_default_mem.nslabs) + return ARCH_KMALLOC_MINALIGN; + + pr_info_once("No default DMA bounce buffer, increasing the kmalloc() minimum alignment to %d\n", + cache_align); + return cache_align; +} + +void __init new_kmalloc_cache(int idx, enum kmalloc_cache_type type, slab_flags_t flags) { + unsigned int minalign = __kmalloc_minalign(); + unsigned int aligned_size = kmalloc_info[idx].size; + int aligned_idx = idx; + if (type == KMALLOC_RECLAIM) { flags |= SLAB_RECLAIM_ACCOUNT; } else if (IS_ENABLED(CONFIG_MEMCG_KMEM) && (type == KMALLOC_CGROUP)) { @@ -867,10 +890,17 @@ new_kmalloc_cache(int idx, enum kmalloc_cache_type type, slab_flags_t flags) flags |= SLAB_CACHE_DMA; } - kmalloc_caches[type][idx] = create_kmalloc_cache( - kmalloc_info[idx].name[type], - kmalloc_info[idx].size, flags, 0, - kmalloc_info[idx].size); + if (minalign > ARCH_KMALLOC_MINALIGN) { + aligned_size = ALIGN(aligned_size, minalign); + aligned_idx = __kmalloc_index(aligned_size, false); + } + + if (!kmalloc_caches[type][aligned_idx]) + kmalloc_caches[type][aligned_idx] = create_kmalloc_cache( + kmalloc_info[aligned_idx].name[type], + aligned_size, flags, 0, aligned_size); + if (idx != aligned_idx) + kmalloc_caches[type][idx] = kmalloc_caches[type][aligned_idx]; /* * If CONFIG_MEMCG_KMEM is enabled, disable cache merging for From patchwork Sun Nov 6 22:01:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13033625 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 78F63C4332F for ; Sun, 6 Nov 2022 22:02:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 18B958E0008; Sun, 6 Nov 2022 17:02:14 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 13C408E0001; Sun, 6 Nov 2022 17:02:14 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0041E8E0008; Sun, 6 Nov 2022 17:02:13 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id E84AD8E0001 for ; Sun, 6 Nov 2022 17:02:13 -0500 (EST) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 94516120306 for ; Sun, 6 Nov 2022 22:02:13 +0000 (UTC) X-FDA: 80104391346.10.73DA1B9 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by imf09.hostedemail.com (Postfix) with ESMTP id DCD9F140009 for ; Sun, 6 Nov 2022 22:02:12 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 70413B80D28; Sun, 6 Nov 2022 22:02:11 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6E53CC4314C; Sun, 6 Nov 2022 22:02:06 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Arnd Bergmann , Christoph Hellwig , Greg Kroah-Hartman Cc: Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Robin Murphy , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v3 05/13] mm/slab: Simplify create_kmalloc_cache() args and make it static Date: Sun, 6 Nov 2022 22:01:35 +0000 Message-Id: <20221106220143.2129263-6-catalin.marinas@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221106220143.2129263-1-catalin.marinas@arm.com> References: <20221106220143.2129263-1-catalin.marinas@arm.com> MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1667772133; a=rsa-sha256; cv=none; b=RbQ3UQCV7Qv6mVEQt7Fwc7gJOLMRaD/dOmf+oIAJmEPrUG6c7mA6n8SvwQsceEyYvf9yGw PuMLbIU6FpIkEa1FVSi3WodpndYq6F23Ue2ImlXUxcgZ9SKp69f2yI3obXsk3UBvJSZagx Imk67u2vUWq6fV8o8KbMwR/wgIRX+AY= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=none; spf=pass (imf09.hostedemail.com: domain of cmarinas@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1667772133; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dz+0r9meuLU/UDJ1RjlIT+qRV+NlDYya9KwUINVbNNY=; b=HWjrHLKDnujBT6D8E49ZKNG2Ll2ZRIJrm1mXEayl4hjGnLhGTVFxypsVCEmA3aY9hgoTf6 V8CLRK6bCmeAV3HW+2kADTY8kv3mOyaWB8Evwffp2JSksF6SLTLyiRxcX7IQp2f43j396g Kg61mQNZv6voQvK4RTM9fDXE9flV268= X-Stat-Signature: prbt36bt74fazxa7us5joxpd7uneq3b8 X-Rspam-User: Authentication-Results: imf09.hostedemail.com; dkim=none; spf=pass (imf09.hostedemail.com: domain of cmarinas@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) X-Rspamd-Queue-Id: DCD9F140009 X-Rspamd-Server: rspam09 X-HE-Tag: 1667772132-8807 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: create_kmalloc_cache() is now only called from new_kmalloc_cache() in the same file, so make it static. In addition, the useroffset argument is always 0 while usersize is the same as size. Remove them. Signed-off-by: Catalin Marinas Cc: Andrew Morton --- mm/slab.h | 3 --- mm/slab_common.c | 11 +++++------ 2 files changed, 5 insertions(+), 9 deletions(-) diff --git a/mm/slab.h b/mm/slab.h index d0460e0f6760..93ba93e7b772 100644 --- a/mm/slab.h +++ b/mm/slab.h @@ -285,9 +285,6 @@ gfp_t kmalloc_fix_flags(gfp_t flags); /* Functions provided by the slab allocators */ int __kmem_cache_create(struct kmem_cache *, slab_flags_t flags); -struct kmem_cache *create_kmalloc_cache(const char *name, unsigned int size, - slab_flags_t flags, unsigned int useroffset, - unsigned int usersize); void __init new_kmalloc_cache(int idx, enum kmalloc_cache_type type, slab_flags_t flags); extern void create_boot_cache(struct kmem_cache *, const char *name, diff --git a/mm/slab_common.c b/mm/slab_common.c index b62f27c2dda7..3fe3f4ad1362 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -654,17 +654,16 @@ void __init create_boot_cache(struct kmem_cache *s, const char *name, s->refcount = -1; /* Exempt from merging for now */ } -struct kmem_cache *__init create_kmalloc_cache(const char *name, - unsigned int size, slab_flags_t flags, - unsigned int useroffset, unsigned int usersize) +static struct kmem_cache *__init create_kmalloc_cache(const char *name, + unsigned int size, + slab_flags_t flags) { struct kmem_cache *s = kmem_cache_zalloc(kmem_cache, GFP_NOWAIT); if (!s) panic("Out of memory when creating slab %s\n", name); - create_boot_cache(s, name, size, flags | SLAB_KMALLOC, useroffset, - usersize); + create_boot_cache(s, name, size, flags | SLAB_KMALLOC, 0, size); kasan_cache_create_kmalloc(s); list_add(&s->list, &slab_caches); s->refcount = 1; @@ -898,7 +897,7 @@ new_kmalloc_cache(int idx, enum kmalloc_cache_type type, slab_flags_t flags) if (!kmalloc_caches[type][aligned_idx]) kmalloc_caches[type][aligned_idx] = create_kmalloc_cache( kmalloc_info[aligned_idx].name[type], - aligned_size, flags, 0, aligned_size); + aligned_size, flags); if (idx != aligned_idx) kmalloc_caches[type][idx] = kmalloc_caches[type][aligned_idx]; From patchwork Sun Nov 6 22:01:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13033626 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1288FC4332F for ; Sun, 6 Nov 2022 22:02:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 757BE8E0009; Sun, 6 Nov 2022 17:02:16 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 72F778E0001; Sun, 6 Nov 2022 17:02:16 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5D09F8E0009; Sun, 6 Nov 2022 17:02:16 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 500EB8E0001 for ; Sun, 6 Nov 2022 17:02:16 -0500 (EST) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 7B826808D1 for ; Sun, 6 Nov 2022 22:02:15 +0000 (UTC) X-FDA: 80104391430.14.DA8BFB1 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf29.hostedemail.com (Postfix) with ESMTP id 1C44312000B for ; Sun, 6 Nov 2022 22:02:14 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 61DA260DCC; Sun, 6 Nov 2022 22:02:14 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 86607C433C1; Sun, 6 Nov 2022 22:02:10 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Arnd Bergmann , Christoph Hellwig , Greg Kroah-Hartman Cc: Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Robin Murphy , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v3 06/13] dma: Allow the smaller cache_line_size() returned by dma_get_cache_alignment() Date: Sun, 6 Nov 2022 22:01:36 +0000 Message-Id: <20221106220143.2129263-7-catalin.marinas@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221106220143.2129263-1-catalin.marinas@arm.com> References: <20221106220143.2129263-1-catalin.marinas@arm.com> MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1667772135; a=rsa-sha256; cv=none; b=mT02KP3wLSZ2Rfp/XA+HFIyX+C3Yntt81Py1p1ka5cZRWlnBgH87WCNYa0gd7Vy5UXQzmU /3CGry2wP/q0F0GGNB6un1SOIhYh6iK9QVaJCkYox0cZ/6oIh5fT06oy4JB+DMQKdHEAZA vrF/a9qlC0e25GhgNFwsClECaKvtLjk= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=none; spf=pass (imf29.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1667772135; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LkwQjw/L+UDhC5i4BYbU5Y3t1NP9s28WtDY6yeWPLcY=; b=T8//dMFSgVHQmnvJIZl1+y9to3zK8Kwr3vyjXYCbDyjuMQ5J+JMnAxoBFkjve7eKbEv9ON HE+DbOPANFs7Tlz7KbhvttWbbvxbyiJ9v8dlivPW2xT0VpEnaDHeWmA95Zi/sFZvyM1c3q yQiNJWAvarJFVY+UEWYYFiFjbptlDkQ= Authentication-Results: imf29.hostedemail.com; dkim=none; spf=pass (imf29.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 1C44312000B X-Stat-Signature: d1ah8zgxagrdbynqdnj8yb11wfnozqpf X-HE-Tag: 1667772134-192311 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On architectures like arm64, ARCH_DMA_MINALIGN is larger than the majority of cache line size configurations. Allow an architecture to opt in to dma_get_cache_alignment() returning such smaller size and select the option for arm64. Signed-off-by: Catalin Marinas Cc: Christoph Hellwig Cc: Robin Murphy Cc: Will Deacon --- Is there any architecture where ARCH_DMA_MINALIGN is larger than cache_line_size()? We could avoid another Kconfig entry. arch/arm64/Kconfig | 1 + include/linux/dma-mapping.h | 2 ++ kernel/dma/Kconfig | 6 ++++++ 3 files changed, 9 insertions(+) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 505c8a1ccbe0..3991cb7b8a33 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -22,6 +22,7 @@ config ARM64 select ARCH_HAS_CURRENT_STACK_POINTER select ARCH_HAS_DEBUG_VIRTUAL select ARCH_HAS_DEBUG_VM_PGTABLE + select ARCH_HAS_DMA_CACHE_LINE_SIZE select ARCH_HAS_DMA_PREP_COHERENT select ARCH_HAS_ACPI_TABLE_UPGRADE if ACPI select ARCH_HAS_FAST_MULTIPLIER diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h index 3288a1339271..b29124341317 100644 --- a/include/linux/dma-mapping.h +++ b/include/linux/dma-mapping.h @@ -545,6 +545,8 @@ static inline int dma_set_min_align_mask(struct device *dev, static inline int dma_get_cache_alignment(void) { + if (IS_ENABLED(CONFIG_ARCH_HAS_DMA_CACHE_LINE_SIZE)) + return cache_line_size(); #ifdef ARCH_HAS_DMA_MINALIGN return ARCH_DMA_MINALIGN; #endif diff --git a/kernel/dma/Kconfig b/kernel/dma/Kconfig index 56866aaa2ae1..d6fab8e3cbae 100644 --- a/kernel/dma/Kconfig +++ b/kernel/dma/Kconfig @@ -76,6 +76,12 @@ config ARCH_HAS_DMA_PREP_COHERENT config ARCH_HAS_FORCE_DMA_UNENCRYPTED bool +config ARCH_HAS_DMA_CACHE_LINE_SIZE + bool + help + Select if the architecture has non-coherent DMA and + cache_line_size() is a safe alignment for DMA buffers. + config SWIOTLB bool select NEED_DMA_MAP_STATE From patchwork Sun Nov 6 22:01:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13033627 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C8ACCC4332F for ; Sun, 6 Nov 2022 22:02:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 695B28E000A; Sun, 6 Nov 2022 17:02:21 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 61DE68E0001; Sun, 6 Nov 2022 17:02:21 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4E5C38E000A; Sun, 6 Nov 2022 17:02:21 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 41C3C8E0001 for ; Sun, 6 Nov 2022 17:02:21 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 1A3AF1C5F3A for ; Sun, 6 Nov 2022 22:02:21 +0000 (UTC) X-FDA: 80104391682.02.10F7AE0 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf28.hostedemail.com (Postfix) with ESMTP id A23C4C0004 for ; Sun, 6 Nov 2022 22:02:19 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id D6DB960DE9; Sun, 6 Nov 2022 22:02:18 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 95051C43147; Sun, 6 Nov 2022 22:02:14 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Arnd Bergmann , Christoph Hellwig , Greg Kroah-Hartman Cc: Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Robin Murphy , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v3 07/13] drivers/base: Use ARCH_DMA_MINALIGN instead of ARCH_KMALLOC_MINALIGN Date: Sun, 6 Nov 2022 22:01:37 +0000 Message-Id: <20221106220143.2129263-8-catalin.marinas@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221106220143.2129263-1-catalin.marinas@arm.com> References: <20221106220143.2129263-1-catalin.marinas@arm.com> MIME-Version: 1.0 ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf28.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1667772139; a=rsa-sha256; cv=none; b=RUBUWyEi5WQ3b+iRFtQGq98JTh1bneQkK2j72gxPgwlFIZr8emVksQxGa6G3L91PiTlufG 8SPa0SKs6qwh0wm6RypWh6eF/QWaDkhyHb1THZK3cVsO4u3XKOp2LxryTsOMJyvWfg9BzF roB+W4bL3FY6DplkXc4f0wSKCeLaRek= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1667772139; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XDCcRXwOo+xEIqxIRMdutr9HbBFr73cl4jAeW3k2o08=; b=Y+EQMOPmdHKodnN/wv7dPfq9vlBP1pKsvHQkNgP9fdLqDY5owbzdy7DFQoYUt+tC7+lU5G 2yHq3atPl6prX+cs+FbrX7KdIIgerT/ho2YsbOyEOiTr/r27jWaTolL6Ufq1GiWfuuFeVm ggETwq4JABcABRKnpZXjd7wvCipLBLQ= X-Stat-Signature: q4inbewxa3wz8ndn3pt3jcbq6cj7rza9 X-Rspamd-Queue-Id: A23C4C0004 Authentication-Results: imf28.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf28.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org X-Rspam-User: X-Rspamd-Server: rspam06 X-HE-Tag: 1667772139-728638 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: ARCH_DMA_MINALIGN represents the minimum (static) alignment for safe DMA operations while ARCH_KMALLOC_MINALIGN is the minimum kmalloc() objects alignment. Signed-off-by: Catalin Marinas Cc: Greg Kroah-Hartman Cc: "Rafael J. Wysocki" --- drivers/base/devres.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/base/devres.c b/drivers/base/devres.c index 4ab2b50ee38f..5dbfa3c31af6 100644 --- a/drivers/base/devres.c +++ b/drivers/base/devres.c @@ -29,10 +29,10 @@ struct devres { * Some archs want to perform DMA into kmalloc caches * and need a guaranteed alignment larger than * the alignment of a 64-bit integer. - * Thus we use ARCH_KMALLOC_MINALIGN here and get exactly the same - * buffer alignment as if it was allocated by plain kmalloc(). + * Thus we use ARCH_DMA_MINALIGN for data[] which will force the same + * alignment for struct devres when allocated by kmalloc(). */ - u8 __aligned(ARCH_KMALLOC_MINALIGN) data[]; + u8 __aligned(ARCH_DMA_MINALIGN) data[]; }; struct devres_group { From patchwork Sun Nov 6 22:01:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13033628 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5E944C4332F for ; Sun, 6 Nov 2022 22:02:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 052B58E000B; Sun, 6 Nov 2022 17:02:26 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0039C8E0001; Sun, 6 Nov 2022 17:02:25 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E34728E000B; Sun, 6 Nov 2022 17:02:25 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id D5E208E0001 for ; Sun, 6 Nov 2022 17:02:25 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id ADC41A0628 for ; Sun, 6 Nov 2022 22:02:25 +0000 (UTC) X-FDA: 80104391850.23.DC8B71C Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by imf03.hostedemail.com (Postfix) with ESMTP id 40D0320008 for ; Sun, 6 Nov 2022 22:02:24 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id B9058B80D3E; Sun, 6 Nov 2022 22:02:23 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A7ECBC43149; Sun, 6 Nov 2022 22:02:18 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Arnd Bergmann , Christoph Hellwig , Greg Kroah-Hartman Cc: Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Robin Murphy , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v3 08/13] drivers/gpu: Use ARCH_DMA_MINALIGN instead of ARCH_KMALLOC_MINALIGN Date: Sun, 6 Nov 2022 22:01:38 +0000 Message-Id: <20221106220143.2129263-9-catalin.marinas@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221106220143.2129263-1-catalin.marinas@arm.com> References: <20221106220143.2129263-1-catalin.marinas@arm.com> MIME-Version: 1.0 ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1667772145; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=S59+slMdeGHVr9gGv1wpxhrhYrhnmUIhONDI6UrLqhg=; b=IYvJXsryy48E1jF2rlWsUM8gPt4NiiRKTA6oac2mSRG7neBZsOFCnfL6I6m+UAhmX52v8r Zbfq7d4r8c2vHLLCVdfA60HKZU2noliNcusqUh6owSv5GQqmkep4bFuy8098J4OVLxReIx gv2DR0F7T+4hHN4Sv/6l61/gQJ8HSSc= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf03.hostedemail.com: domain of cmarinas@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1667772145; a=rsa-sha256; cv=none; b=Sob71nNCfb2CAXa+3Bt28bZUmzcK8SpTMxoBnvhI+en9f461L+XRhswwv63dFmwG2fPpTw eZC7nHBCH5ELnemg0zfHq+qrr6YiqkU35u+QwK/ZZqQPSczEFl1eZRcg/3HAw8mr04/9S1 WPLj/CaFBtVqFbHJe72aeOgrMKayAU8= X-Rspamd-Queue-Id: 40D0320008 X-Rspam-User: X-Rspamd-Server: rspam08 Authentication-Results: imf03.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf03.hostedemail.com: domain of cmarinas@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=cmarinas@kernel.org X-Stat-Signature: exdau45qtnzmgiq6yhw85qirpapq3kuf X-HE-Tag: 1667772144-308814 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: ARCH_DMA_MINALIGN represents the minimum (static) alignment for safe DMA operations while ARCH_KMALLOC_MINALIGN is the minimum kmalloc() objects alignment. Signed-off-by: Catalin Marinas Cc: Daniel Vetter --- drivers/gpu/drm/drm_managed.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/gpu/drm/drm_managed.c b/drivers/gpu/drm/drm_managed.c index 4cf214de50c4..3a5802f60e65 100644 --- a/drivers/gpu/drm/drm_managed.c +++ b/drivers/gpu/drm/drm_managed.c @@ -49,10 +49,10 @@ struct drmres { * Some archs want to perform DMA into kmalloc caches * and need a guaranteed alignment larger than * the alignment of a 64-bit integer. - * Thus we use ARCH_KMALLOC_MINALIGN here and get exactly the same - * buffer alignment as if it was allocated by plain kmalloc(). + * Thus we use ARCH_DMA_MINALIGN for data[] which will force the same + * alignment for struct drmres when allocated by kmalloc(). */ - u8 __aligned(ARCH_KMALLOC_MINALIGN) data[]; + u8 __aligned(ARCH_DMA_MINALIGN) data[]; }; static void free_dr(struct drmres *dr) From patchwork Sun Nov 6 22:01:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13033629 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8065EC433FE for ; Sun, 6 Nov 2022 22:02:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D253F8E000C; Sun, 6 Nov 2022 17:02:29 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CD47F8E0001; Sun, 6 Nov 2022 17:02:29 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B9C6D8E000C; Sun, 6 Nov 2022 17:02:29 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id AC2688E0001 for ; Sun, 6 Nov 2022 17:02:29 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 8FD9EA06B4 for ; Sun, 6 Nov 2022 22:02:29 +0000 (UTC) X-FDA: 80104392018.08.5CED29C Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by imf14.hostedemail.com (Postfix) with ESMTP id 1E274100004 for ; Sun, 6 Nov 2022 22:02:28 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id C2D38B80D48; Sun, 6 Nov 2022 22:02:27 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id CA6E9C433D6; Sun, 6 Nov 2022 22:02:22 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Arnd Bergmann , Christoph Hellwig , Greg Kroah-Hartman Cc: Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Robin Murphy , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v3 09/13] drivers/usb: Use ARCH_DMA_MINALIGN instead of ARCH_KMALLOC_MINALIGN Date: Sun, 6 Nov 2022 22:01:39 +0000 Message-Id: <20221106220143.2129263-10-catalin.marinas@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221106220143.2129263-1-catalin.marinas@arm.com> References: <20221106220143.2129263-1-catalin.marinas@arm.com> MIME-Version: 1.0 ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=none; spf=pass (imf14.hostedemail.com: domain of cmarinas@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1667772149; a=rsa-sha256; cv=none; b=JxzbKKKt6i7Gdm8+3VD607Z9Fslep0G02uUeoJJcjtr3f3WlwseMXTbTgvLMSUzCu7YG+a pN3nxsmZGeH6Ik9lKn/uekJjd6YwGlt6RyusdvpDUrxudkA9mXTvY/LPurFQt2kIh9iq1i 0Nn38ZhPD9UKf0h7WI486EiFhQOPooM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1667772149; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=rq19jP8KoV6cfikLmjA5jQ41EC5pXyhnoRUI2FjO7tU=; b=7tUrP5aGu8WfqktgpahB3woPG75mtrDf77F8l+XkOqq74eW0BrYlUBk8+BzvujM/mLXSYv ACxg497qVKp7bSOBWA6AmXQN1QdZ2267MrtpNm+3XjFUWlI4EsSBbhjFPLR+k1fsintzwC vXpF+wSHeRjSegYUuWQmo6jukAX0ILk= X-Stat-Signature: wwn64pwyte16w3dgti5pa3ot5px4f1e1 X-Rspamd-Queue-Id: 1E274100004 Authentication-Results: imf14.hostedemail.com; dkim=none; spf=pass (imf14.hostedemail.com: domain of cmarinas@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) X-Rspamd-Server: rspam05 X-Rspam-User: X-HE-Tag: 1667772148-681355 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: ARCH_DMA_MINALIGN represents the minimum (static) alignment for safe DMA operations while ARCH_KMALLOC_MINALIGN is the minimum kmalloc() objects alignment. Signed-off-by: Catalin Marinas Cc: Greg Kroah-Hartman --- drivers/usb/core/buffer.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/drivers/usb/core/buffer.c b/drivers/usb/core/buffer.c index fbb087b728dc..e21d8d106977 100644 --- a/drivers/usb/core/buffer.c +++ b/drivers/usb/core/buffer.c @@ -34,13 +34,13 @@ void __init usb_init_pool_max(void) { /* * The pool_max values must never be smaller than - * ARCH_KMALLOC_MINALIGN. + * ARCH_DMA_MINALIGN. */ - if (ARCH_KMALLOC_MINALIGN <= 32) + if (ARCH_DMA_MINALIGN <= 32) ; /* Original value is okay */ - else if (ARCH_KMALLOC_MINALIGN <= 64) + else if (ARCH_DMA_MINALIGN <= 64) pool_max[0] = 64; - else if (ARCH_KMALLOC_MINALIGN <= 128) + else if (ARCH_DMA_MINALIGN <= 128) pool_max[0] = 0; /* Don't use this pool */ else BUILD_BUG(); /* We don't allow this */ From patchwork Sun Nov 6 22:01:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13033630 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 61089C433FE for ; Sun, 6 Nov 2022 22:02:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0C89B8E0007; Sun, 6 Nov 2022 17:02:34 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 051848E0001; Sun, 6 Nov 2022 17:02:33 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E82718E0007; Sun, 6 Nov 2022 17:02:33 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id DA0FD8E0001 for ; Sun, 6 Nov 2022 17:02:33 -0500 (EST) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id AA0BAC0459 for ; Sun, 6 Nov 2022 22:02:33 +0000 (UTC) X-FDA: 80104392186.22.BD864E2 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by imf23.hostedemail.com (Postfix) with ESMTP id 2383B140003 for ; Sun, 6 Nov 2022 22:02:32 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id CB7D1B80D2E; Sun, 6 Nov 2022 22:02:31 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D8178C43470; Sun, 6 Nov 2022 22:02:26 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Arnd Bergmann , Christoph Hellwig , Greg Kroah-Hartman Cc: Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Robin Murphy , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v3 10/13] drivers/spi: Use ARCH_DMA_MINALIGN instead of ARCH_KMALLOC_MINALIGN Date: Sun, 6 Nov 2022 22:01:40 +0000 Message-Id: <20221106220143.2129263-11-catalin.marinas@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221106220143.2129263-1-catalin.marinas@arm.com> References: <20221106220143.2129263-1-catalin.marinas@arm.com> MIME-Version: 1.0 ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1667772153; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XS26Kwt1TS+tA8636dGXdSqogXOL8caJbJ1kDBldBR0=; b=jKPjw7tqWrMcvLF1AvS5EUs4BUZvsecsR3wt/H5F0nfZcFtDw8flZnn8MsPOiEkuUpv6+7 abZRJDTImI4uRJ66/CvmtIfQ8qZWQC2US3JJ5yH/x1X8dD9+MkPOekE0budtitKnXz0Z8p IhG1XveAamRSjCn3sjYI92NWo/dKHgw= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=none; spf=pass (imf23.hostedemail.com: domain of cmarinas@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1667772153; a=rsa-sha256; cv=none; b=W9a4Z0M+2dFW1zTDS3SSxwutee7257jP8v0bcEVv1JR0yPCMFrI5Wa6RH9VL8ttCyjoRmQ m1/TdnZuw6/+sDZ4MzW3QRL3iM0BAGyEl+iZ6uakGbw/xhgoW1iqad6H995BvYSnX2Ozbi VyscwbvZATdSW3HLAayilBhNAr8SQX8= Authentication-Results: imf23.hostedemail.com; dkim=none; spf=pass (imf23.hostedemail.com: domain of cmarinas@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) X-Rspam-User: X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 2383B140003 X-Stat-Signature: 5mrufgmi7ap4tpadmaqiuoxwtfxdbmcs X-HE-Tag: 1667772152-558810 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: ARCH_DMA_MINALIGN represents the minimum (static) alignment for safe DMA operations while ARCH_KMALLOC_MINALIGN is the minimum kmalloc() objects alignment. Signed-off-by: Catalin Marinas Cc: Mark Brown Acked-by: Mark Brown --- drivers/spi/spidev.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/spi/spidev.c b/drivers/spi/spidev.c index b2775d82d2d7..f74486240b1f 100644 --- a/drivers/spi/spidev.c +++ b/drivers/spi/spidev.c @@ -228,7 +228,7 @@ static int spidev_message(struct spidev_data *spidev, /* Ensure that also following allocations from rx_buf/tx_buf will meet * DMA alignment requirements. */ - unsigned int len_aligned = ALIGN(u_tmp->len, ARCH_KMALLOC_MINALIGN); + unsigned int len_aligned = ALIGN(u_tmp->len, ARCH_DMA_MINALIGN); k_tmp->len = u_tmp->len; From patchwork Sun Nov 6 22:01:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13033631 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 104FCC433FE for ; Sun, 6 Nov 2022 22:02:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A3DE48E000D; Sun, 6 Nov 2022 17:02:36 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9EEFE8E0001; Sun, 6 Nov 2022 17:02:36 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 88EA28E000D; Sun, 6 Nov 2022 17:02:36 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 79E408E0001 for ; Sun, 6 Nov 2022 17:02:36 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 56325AABB5 for ; Sun, 6 Nov 2022 22:02:36 +0000 (UTC) X-FDA: 80104392312.23.3D22812 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf03.hostedemail.com (Postfix) with ESMTP id EB26C20002 for ; Sun, 6 Nov 2022 22:02:35 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 27EA860DEA; Sun, 6 Nov 2022 22:02:35 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E4456C43148; Sun, 6 Nov 2022 22:02:30 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Arnd Bergmann , Christoph Hellwig , Greg Kroah-Hartman Cc: Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Robin Murphy , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v3 11/13] crypto: Use ARCH_DMA_MINALIGN instead of ARCH_KMALLOC_MINALIGN Date: Sun, 6 Nov 2022 22:01:41 +0000 Message-Id: <20221106220143.2129263-12-catalin.marinas@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221106220143.2129263-1-catalin.marinas@arm.com> References: <20221106220143.2129263-1-catalin.marinas@arm.com> MIME-Version: 1.0 ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=none; spf=pass (imf03.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1667772156; a=rsa-sha256; cv=none; b=7zZD20CYqtj8/3NUzT4BaXjKqkIC4Df+NmFHq+5MixFrxCtN2EtXNnXU2N6YH8yvKtgEGx rZCEcuCzhXmhjSr2IBsdt/9uLns18anTlfqhaws2+Y733+QQkpcyAbabUZMvwnmHGUEQuI BGEd57blDj/9FC0swi05axco419e7IM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1667772156; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gAQSh36C9KqkKi5xK50DhdofwN1JfPGKoXpI0Fg8VOQ=; b=Cm5HhcM0V0VOq2KDpYIGE9mjtN2gf+gahAamC4fBX8JjbK4jcJ8WqCEr+TJJqsd4t5GgYK o/aC9u996W2RGIDdyrfGtu2bcn/SPHMEQRJDRYJbWyZMWxGtPQ3AxYYomdqkgKDVyAM4l6 3K5ccW9AOhEeoWVtk19mgqw+VNts2qs= X-Stat-Signature: r53d9zps8ddjtaw7ysfuh8fhs91tp7xe X-Rspamd-Queue-Id: EB26C20002 Authentication-Results: imf03.hostedemail.com; dkim=none; spf=pass (imf03.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) X-Rspamd-Server: rspam05 X-Rspam-User: X-HE-Tag: 1667772155-293021 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: ARCH_DMA_MINALIGN represents the minimum (static) alignment for safe DMA operations while ARCH_KMALLOC_MINALIGN is the minimum kmalloc() alignment. This will ensure that the static alignment of various structures or members of those structures( e.g. __ctx[] in struct aead_request) is safe for DMA. Note that sizeof such structures becomes aligned to ARCH_DMA_MINALIGN and kmalloc() will honour such alignment, so there is no confusion for the compiler. Signed-off-by: Catalin Marinas Cc: Herbert Xu Cc: Ard Biesheuvel --- I know Herbert NAK'ed this patch but I'm still keeping it here temporarily, until we agree on some refactoring at the crypto code. FTR, I don't think there's anything wrong with this patch since kmalloc() will return ARCH_DMA_MINALIGN-aligned objects if the sizeof such objects is a multiple of ARCH_DMA_MINALIGN (side-effect of CRYPTO_MINALIGN_ATTR). include/linux/crypto.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/linux/crypto.h b/include/linux/crypto.h index 2324ab6f1846..654b9c355575 100644 --- a/include/linux/crypto.h +++ b/include/linux/crypto.h @@ -167,7 +167,7 @@ * maintenance for non-coherent DMA (cache invalidation in particular) does not * affect data that may be accessed by the CPU concurrently. */ -#define CRYPTO_MINALIGN ARCH_KMALLOC_MINALIGN +#define CRYPTO_MINALIGN ARCH_DMA_MINALIGN #define CRYPTO_MINALIGN_ATTR __attribute__ ((__aligned__(CRYPTO_MINALIGN))) From patchwork Sun Nov 6 22:01:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13033632 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 95347C4332F for ; Sun, 6 Nov 2022 22:02:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 387118E000E; Sun, 6 Nov 2022 17:02:42 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 336E18E0001; Sun, 6 Nov 2022 17:02:42 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 226C88E000E; Sun, 6 Nov 2022 17:02:42 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 172A88E0001 for ; Sun, 6 Nov 2022 17:02:42 -0500 (EST) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id C0342A093B for ; Sun, 6 Nov 2022 22:02:40 +0000 (UTC) X-FDA: 80104392480.07.7CA2F8A Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf17.hostedemail.com (Postfix) with ESMTP id 6B85E40002 for ; Sun, 6 Nov 2022 22:02:40 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id A78D660DF0; Sun, 6 Nov 2022 22:02:39 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id F14A5C43149; Sun, 6 Nov 2022 22:02:34 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Arnd Bergmann , Christoph Hellwig , Greg Kroah-Hartman Cc: Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Robin Murphy , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v3 12/13] drivers/md: Use ARCH_DMA_MINALIGN instead of ARCH_KMALLOC_MINALIGN Date: Sun, 6 Nov 2022 22:01:42 +0000 Message-Id: <20221106220143.2129263-13-catalin.marinas@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221106220143.2129263-1-catalin.marinas@arm.com> References: <20221106220143.2129263-1-catalin.marinas@arm.com> MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1667772160; a=rsa-sha256; cv=none; b=EGPgul39j7JrS0MVmSqC4N95YflLPkm3r5Ch6f7r9PpbELdKvTKqaWxuIq9B87jf8Ls0D3 WzOToF7HDNvgOEbLaZkTJt7cnVjcP0J4IEb3YdT4AYES4sCishiDrsWp4idY3o3p3Rz1P1 yCYG0OptAE57OQRFgemoNbJilXg3igs= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=none; spf=pass (imf17.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1667772160; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pDhz7XEXjwzYmuPQaO3jG7tdtt5Kxl4xIUNvSZj+Ak4=; b=K2KEDPHdXz9wEpM/1as60zSBOVCcdj5wVqBcpu11QHLoDnhPZUL/3Lsi+VYHq6Am1aQ054 XgiiFwcmpHbeJMm752EzTdBKYN/k6FzkHzCmmQaHeR87gTrDleKH5RYjkrAXRIJfgvm4b/ ZZp4dyHSQ8XZQFNrgITXdxlH8h9eyjU= X-Stat-Signature: 49h4hjfnr38fqu4ruo7sgsqyi8zfpp56 X-Rspamd-Queue-Id: 6B85E40002 Authentication-Results: imf17.hostedemail.com; dkim=none; spf=pass (imf17.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) X-Rspam-User: X-Rspamd-Server: rspam11 X-HE-Tag: 1667772160-669871 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: ARCH_DMA_MINALIGN represents the minimum (static) alignment for safe DMA operations while ARCH_KMALLOC_MINALIGN is the minimum kmalloc() objects alignment. Signed-off-by: Catalin Marinas Cc: Alasdair Kergon Cc: Mike Snitzer --- This is somewhat related to the previous crypto patch. The dm_crypt_io structure is supposed to be CRYPTO_MINALIGN-aligned. drivers/md/dm-crypt.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/md/dm-crypt.c b/drivers/md/dm-crypt.c index 159c6806c19b..4f1bd5b348c3 100644 --- a/drivers/md/dm-crypt.c +++ b/drivers/md/dm-crypt.c @@ -3250,7 +3250,7 @@ static int crypt_ctr(struct dm_target *ti, unsigned int argc, char **argv) cc->per_bio_data_size = ti->per_io_data_size = ALIGN(sizeof(struct dm_crypt_io) + cc->dmreq_start + additional_req_size, - ARCH_KMALLOC_MINALIGN); + ARCH_DMA_MINALIGN); ret = mempool_init(&cc->page_pool, BIO_MAX_VECS, crypt_page_alloc, crypt_page_free, cc); if (ret) { From patchwork Sun Nov 6 22:01:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13033633 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A1498C4332F for ; Sun, 6 Nov 2022 22:02:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 44B068E0003; Sun, 6 Nov 2022 17:02:46 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3FA578E0001; Sun, 6 Nov 2022 17:02:46 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2E9918E0003; Sun, 6 Nov 2022 17:02:46 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 1E8C08E0001 for ; Sun, 6 Nov 2022 17:02:46 -0500 (EST) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id E45621A0B55 for ; Sun, 6 Nov 2022 22:02:45 +0000 (UTC) X-FDA: 80104392690.07.A26A1E6 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by imf06.hostedemail.com (Postfix) with ESMTP id 5CB96180003 for ; Sun, 6 Nov 2022 22:02:45 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id EFA1CB80D4A; Sun, 6 Nov 2022 22:02:43 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0A241C4314C; Sun, 6 Nov 2022 22:02:38 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Arnd Bergmann , Christoph Hellwig , Greg Kroah-Hartman Cc: Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Robin Murphy , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v3 13/13] dma: arm64: Add CONFIG_DMA_BOUNCE_UNALIGNED_KMALLOC and enable it for arm64 Date: Sun, 6 Nov 2022 22:01:43 +0000 Message-Id: <20221106220143.2129263-14-catalin.marinas@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221106220143.2129263-1-catalin.marinas@arm.com> References: <20221106220143.2129263-1-catalin.marinas@arm.com> MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1667772165; a=rsa-sha256; cv=none; b=IQp/Xn+h6fmqzO2UWiuiwqEPd3QZ1XqmCfZt/32HI2rZlN/bFsb272KQKvAYQJ4ALBhgha N6GHtpQtKBi+6K9jWTnMR37Kdq9N+ZKlW+tGuMcCe59LlhDhbqV/v5BgZCAgmy5UoRvu9c jXaTgStf4MocOBJwmIRGTAGsGI1J9q4= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=none; spf=pass (imf06.hostedemail.com: domain of cmarinas@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1667772165; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TdcqJYQWWtwEVe1kRjJzVtKvGp4qZ6haY9zrYh4QD40=; b=ulTUMoypgNGkNWJz3q2nEVwFC5DkdMFkcurYYmqByBVWlcSE92jeG7UAp9+ROo8QPmLbx7 oIvqrQKenRCz7wKpdro5oH0XVNL0IrVhD19xw0hpZndFeF5MsPXSNn4MFQUBvkETTEvTQS 23Sud7pMTclL8Ei3MfH3yyVUNod1yLM= X-Stat-Signature: mz8ee5jyiu1t5io8dsswosy65yocxk9o X-Rspamd-Queue-Id: 5CB96180003 Authentication-Results: imf06.hostedemail.com; dkim=none; spf=pass (imf06.hostedemail.com: domain of cmarinas@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) X-Rspam-User: X-Rspamd-Server: rspam11 X-HE-Tag: 1667772165-812418 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: With all the infrastructure in place for bouncing small kmalloc() buffers, add the corresponding Kconfig entry and select it for arm64. Signed-off-by: Catalin Marinas Cc: Will Deacon Cc: Christoph Hellwig Cc: Robin Murphy --- arch/arm64/Kconfig | 1 + kernel/dma/Kconfig | 8 ++++++++ 2 files changed, 9 insertions(+) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 3991cb7b8a33..f889cf16e6ab 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -100,6 +100,7 @@ config ARM64 select ARCH_WANT_FRAME_POINTERS select ARCH_WANT_HUGE_PMD_SHARE if ARM64_4K_PAGES || (ARM64_16K_PAGES && !ARM64_VA_BITS_36) select ARCH_WANT_HUGETLB_PAGE_OPTIMIZE_VMEMMAP + select ARCH_WANT_KMALLOC_DMA_BOUNCE select ARCH_WANT_LD_ORPHAN_WARN select ARCH_WANTS_NO_INSTR select ARCH_WANTS_THP_SWAP if ARM64_4K_PAGES diff --git a/kernel/dma/Kconfig b/kernel/dma/Kconfig index d6fab8e3cbae..b56e76371023 100644 --- a/kernel/dma/Kconfig +++ b/kernel/dma/Kconfig @@ -86,6 +86,14 @@ config SWIOTLB bool select NEED_DMA_MAP_STATE +config ARCH_WANT_KMALLOC_DMA_BOUNCE + bool + +config DMA_BOUNCE_UNALIGNED_KMALLOC + def_bool y + depends on ARCH_WANT_KMALLOC_DMA_BOUNCE + depends on SWIOTLB && !SLOB + config DMA_RESTRICTED_POOL bool "DMA Restricted Pool" depends on OF && OF_RESERVED_MEM && SWIOTLB