From patchwork Sun Nov 6 22:01:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Catalin Marinas X-Patchwork-Id: 13033620 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 98030C4332F for ; Sun, 6 Nov 2022 22:01:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 97B958E0002; Sun, 6 Nov 2022 17:01:53 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 92B478E0001; Sun, 6 Nov 2022 17:01:53 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 81A548E0002; Sun, 6 Nov 2022 17:01:53 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 73FF38E0001 for ; Sun, 6 Nov 2022 17:01:53 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 41712A9FAE for ; Sun, 6 Nov 2022 22:01:53 +0000 (UTC) X-FDA: 80104390506.02.35EA36C Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by imf21.hostedemail.com (Postfix) with ESMTP id 95E691C000A for ; Sun, 6 Nov 2022 22:01:52 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 187FFB80D29; Sun, 6 Nov 2022 22:01:51 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 21614C433D6; Sun, 6 Nov 2022 22:01:45 +0000 (UTC) From: Catalin Marinas To: Linus Torvalds , Arnd Bergmann , Christoph Hellwig , Greg Kroah-Hartman Cc: Will Deacon , Marc Zyngier , Andrew Morton , Herbert Xu , Ard Biesheuvel , Isaac Manjarres , Saravana Kannan , Alasdair Kergon , Daniel Vetter , Joerg Roedel , Mark Brown , Mike Snitzer , "Rafael J. Wysocki" , Robin Murphy , linux-mm@kvack.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Subject: [PATCH v3 00/13] mm, dma, arm64: Reduce ARCH_KMALLOC_MINALIGN to 8 Date: Sun, 6 Nov 2022 22:01:30 +0000 Message-Id: <20221106220143.2129263-1-catalin.marinas@arm.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1667772112; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references; bh=cxOJ9puB8gs9iabUeTE9SfpHk/Ez9PWbsn5DvtR5Tg4=; b=kLt4SUoEOyWfqcOoYu2mDjW8v9CVvtx3NHhDE/7eD/5w3wahXZON4ioYEqPk5ns1wv4Le/ rlRuFVpkVdrQykY4ULVKGT4cACUG4ARTDYOX8gNF0epOkdpvfxMCdd4heUcGAQnPv19QNO 9liBLax+DKWKICk0jQdO1xghpv6KZKw= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=none; spf=pass (imf21.hostedemail.com: domain of cmarinas@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1667772113; a=rsa-sha256; cv=none; b=vmuf8dNztrl3HmYTbMMn9KPWWKDkd2bsRUrmemPUdGoWrWb7u+BWgjgH0PbtoxlIq5ZuzP YXtl1pNoUy9C8QYXuMldrGx909zDKX4yxh1BiZ8HSQE7NiGCY9W1CW8WU6+4YiebAgxbZ+ ZE+w98R168jaM8CdC9Q7/167B1gQ7cY= Authentication-Results: imf21.hostedemail.com; dkim=none; spf=pass (imf21.hostedemail.com: domain of cmarinas@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) X-Rspam-User: X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 95E691C000A X-Stat-Signature: wij1wpfxq44z9dmufjia93uqcodxjttx X-HE-Tag: 1667772112-690136 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hi, That's the third attempt at reducing the kmalloc() minimum alignment on arm64 below the ARCH_DMA_MINALIGN of 128. The first version was not aggressive enough, limiting ARCH_KMALLOC_MINALIGN to 64 while the second version added an explicit __GFP_PACKED flag. This third version reduces ARCH_KMALLOC_MINALIGN to 8 while defining ARCH_DMA_MINALIGN for all platforms and using it instead of the former in places where we need a static alignment (structure or members align attributes). The first patch decouples the kmalloc() and DMA alignment, though this only takes effect after the Kconfig entry is enabled by the last patch. Patches 2 and 3 add bouncing via the swiotlb if any of the sizes are small enough to have originated from an unaligned kmalloc() cache. Not entirely sure whether my approach for iommu bouncing is correct, so open to suggestions. Patch 4 is a fallback in case there is no swiotlb buffer. Together with patch 6, we can still get a smaller kmalloc() minalign of 64 (typical cache line size) rather than 128 on arm64. If we improve the bouncing to use the DMA coherent pool, this run-time __kmalloc_minalign() can go away. Patch 5 is some cleanup following the refactoring in patch 4. Patches 7-12 change some ARCH_KMALLOC_MINALIGN uses to ARCH_DMA_MINALIGN. The crypto changes have been rejected by Herbert previously but I still included them here until the crypto code is refactored. The last patch enables the bouncing for arm64. Thanks. Catalin Marinas (13): mm/slab: Decouple ARCH_KMALLOC_MINALIGN from ARCH_DMA_MINALIGN dma-mapping: Force bouncing if the kmalloc() size is not cacheline-aligned iommu/dma: Force bouncing of the size is not cacheline-aligned mm/slab: Allow kmalloc() minimum alignment fallback to dma_get_cache_alignment() mm/slab: Simplify create_kmalloc_cache() args and make it static dma: Allow the smaller cache_line_size() returned by dma_get_cache_alignment() drivers/base: Use ARCH_DMA_MINALIGN instead of ARCH_KMALLOC_MINALIGN drivers/gpu: Use ARCH_DMA_MINALIGN instead of ARCH_KMALLOC_MINALIGN drivers/usb: Use ARCH_DMA_MINALIGN instead of ARCH_KMALLOC_MINALIGN drivers/spi: Use ARCH_DMA_MINALIGN instead of ARCH_KMALLOC_MINALIGN crypto: Use ARCH_DMA_MINALIGN instead of ARCH_KMALLOC_MINALIGN drivers/md: Use ARCH_DMA_MINALIGN instead of ARCH_KMALLOC_MINALIGN dma: arm64: Add CONFIG_DMA_BOUNCE_UNALIGNED_KMALLOC and enable it for arm64 arch/arm64/Kconfig | 2 ++ drivers/base/devres.c | 6 ++--- drivers/gpu/drm/drm_managed.c | 6 ++--- drivers/iommu/dma-iommu.c | 12 ++++++--- drivers/md/dm-crypt.c | 2 +- drivers/spi/spidev.c | 2 +- drivers/usb/core/buffer.c | 8 +++--- include/linux/crypto.h | 2 +- include/linux/dma-map-ops.h | 50 +++++++++++++++++++++++++++++++++++ include/linux/dma-mapping.h | 4 ++- include/linux/scatterlist.h | 27 ++++++++++++++++--- include/linux/slab.h | 14 +++++++--- kernel/dma/Kconfig | 14 ++++++++++ kernel/dma/direct.h | 3 ++- mm/slab.c | 6 +---- mm/slab.h | 5 ++-- mm/slab_common.c | 49 +++++++++++++++++++++++++++------- 17 files changed, 169 insertions(+), 43 deletions(-)