From patchwork Fri Dec 29 08:18:17 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 10136861 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id A3B236037D for ; Fri, 29 Dec 2017 08:44:10 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8D6DB2DAA4 for ; Fri, 29 Dec 2017 08:44:10 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 805E92DEEC; Fri, 29 Dec 2017 08:44:10 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_MED autolearn=unavailable version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [65.50.211.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 463D52DAA4 for ; Fri, 29 Dec 2017 08:44:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=76zAUZHw0VF7CkEGeTPRcYQGQbXgQMm+QSZE+QwtLSU=; b=l+CbYvWskLgFIBex0dKEGyG9gS 7J7M3jNTcL43NvnFIgfrU893OFT7gAHOns20sJU2a3KmpCQk/RyZbLye8dhFQarvd2Nv2X1zUJ5aQ Ga9/qOvNj7eBLSX/sMhiZKR26Y9VGG2oW5GUY0EdUaTetutsxRMFQ8KHOD1Vh8QkxGq8NzHBBT4bY 3YIE5DSeVwPvzuL0qbQYJMaJNQzDzmA21vCFf3VK4ki3D2idsHtcTNkakF7vSx2Hk/Gt7sjXWK7Gk NqfkcL48zLDvR5NJcYvaDlCDWSkzUCte2ZiUe5p5Di+PSBP7RiSey937ZtUU3xAygpjJeikGTYxry I0DOplkw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.89 #1 (Red Hat Linux)) id 1eUqGq-0003TX-TR; Fri, 29 Dec 2017 08:44:04 +0000 Received: from 77.117.237.29.wireless.dyn.drei.com ([77.117.237.29] helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.89 #1 (Red Hat Linux)) id 1eUptm-0000XA-7n; Fri, 29 Dec 2017 08:20:15 +0000 From: Christoph Hellwig To: iommu@lists.linux-foundation.org Subject: [PATCH 13/67] dma-mapping: move swiotlb arch helpers to a new header Date: Fri, 29 Dec 2017 09:18:17 +0100 Message-Id: <20171229081911.2802-14-hch@lst.de> X-Mailer: git-send-email 2.14.2 In-Reply-To: <20171229081911.2802-1-hch@lst.de> References: <20171229081911.2802-1-hch@lst.de> X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-mips@linux-mips.org, linux-ia64@vger.kernel.org, linux-sh@vger.kernel.org, sparclinux@vger.kernel.org, Guan Xuetao , linux-arch@vger.kernel.org, linux-s390@vger.kernel.org, linux-c6x-dev@linux-c6x.org, linux-hexagon@vger.kernel.org, x86@kernel.org, linux-snps-arc@lists.infradead.org, adi-buildroot-devel@lists.sourceforge.net, linux-m68k@lists.linux-m68k.org, patches@groups.riscv.org, linux-metag@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Michal Simek , linux-parisc@vger.kernel.org, linux-cris-kernel@axis.com, linux-kernel@vger.kernel.org, linux-alpha@vger.kernel.org, linuxppc-dev@lists.ozlabs.org MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP phys_to_dma, dma_to_phys and dma_capable are helpers published by architecture code for use of swiotlb and xen-swiotlb only. Drivers are not supposed to use these directly, but use the DMA API instead. Move these to a new asm/dma-direct.h helper, included by a linux/dma-direct.h wrapper that provides the default linear mapping unless the architecture wants to override it. Signed-off-by: Christoph Hellwig --- MAINTAINERS | 1 + arch/Kconfig | 4 +++ arch/arm/Kconfig | 1 + arch/arm/include/asm/dma-direct.h | 36 ++++++++++++++++++++++ arch/arm/include/asm/dma-mapping.h | 31 ------------------- arch/arm64/include/asm/dma-mapping.h | 22 ------------- arch/arm64/mm/dma-mapping.c | 2 +- arch/ia64/include/asm/dma-mapping.h | 18 ----------- arch/mips/Kconfig | 2 ++ arch/mips/include/asm/dma-direct.h | 1 + arch/mips/include/asm/dma-mapping.h | 8 ----- .../include/asm/mach-cavium-octeon/dma-coherence.h | 8 +++++ arch/mips/include/asm/mach-generic/dma-coherence.h | 12 -------- .../include/asm/mach-loongson64/dma-coherence.h | 8 +++++ arch/powerpc/Kconfig | 1 + arch/powerpc/include/asm/dma-direct.h | 29 +++++++++++++++++ arch/powerpc/include/asm/dma-mapping.h | 25 --------------- arch/tile/include/asm/dma-mapping.h | 18 ----------- arch/unicore32/include/asm/dma-mapping.h | 18 ----------- arch/x86/Kconfig | 1 + arch/x86/include/asm/dma-direct.h | 30 ++++++++++++++++++ arch/x86/include/asm/dma-mapping.h | 26 ---------------- arch/x86/kernel/amd_gart_64.c | 1 + arch/x86/kernel/pci-dma.c | 2 +- arch/x86/kernel/pci-nommu.c | 2 +- arch/x86/kernel/pci-swiotlb.c | 2 +- arch/x86/mm/mem_encrypt.c | 2 +- arch/x86/pci/sta2x11-fixup.c | 1 + arch/xtensa/include/asm/dma-mapping.h | 10 ------ drivers/crypto/marvell/cesa.c | 1 + drivers/mtd/nand/qcom_nandc.c | 1 + drivers/xen/swiotlb-xen.c | 2 +- include/linux/dma-direct.h | 32 +++++++++++++++++++ lib/swiotlb.c | 2 +- 34 files changed, 165 insertions(+), 195 deletions(-) create mode 100644 arch/arm/include/asm/dma-direct.h create mode 100644 arch/mips/include/asm/dma-direct.h create mode 100644 arch/powerpc/include/asm/dma-direct.h create mode 100644 arch/x86/include/asm/dma-direct.h create mode 100644 include/linux/dma-direct.h diff --git a/MAINTAINERS b/MAINTAINERS index a6e86e20761e..7521b063b499 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -4340,6 +4340,7 @@ F: lib/dma-noop.c F: lib/dma-virt.c F: drivers/base/dma-mapping.c F: drivers/base/dma-coherent.c +F: include/linux/dma-direct.h F: include/linux/dma-mapping.h DME1737 HARDWARE MONITOR DRIVER diff --git a/arch/Kconfig b/arch/Kconfig index 400b9e1b2f27..3edf118ad777 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -938,6 +938,10 @@ config STRICT_MODULE_RWX and non-text memory will be made non-executable. This provides protection against certain security exploits (e.g. writing to text) +# select if the architecture provides an asm/dma-direct.h header +config ARCH_HAS_PHYS_TO_DMA + bool + config ARCH_HAS_REFCOUNT bool help diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig index 51c8df561077..00d889a37965 100644 --- a/arch/arm/Kconfig +++ b/arch/arm/Kconfig @@ -8,6 +8,7 @@ config ARM select ARCH_HAS_DEVMEM_IS_ALLOWED select ARCH_HAS_ELF_RANDOMIZE select ARCH_HAS_SET_MEMORY + select ARCH_HAS_PHYS_TO_DMA select ARCH_HAS_STRICT_KERNEL_RWX if MMU && !XIP_KERNEL select ARCH_HAS_STRICT_MODULE_RWX if MMU select ARCH_HAS_TICK_BROADCAST if GENERIC_CLOCKEVENTS_BROADCAST diff --git a/arch/arm/include/asm/dma-direct.h b/arch/arm/include/asm/dma-direct.h new file mode 100644 index 000000000000..5b0a8a421894 --- /dev/null +++ b/arch/arm/include/asm/dma-direct.h @@ -0,0 +1,36 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef ASM_ARM_DMA_DIRECT_H +#define ASM_ARM_DMA_DIRECT_H 1 + +static inline dma_addr_t phys_to_dma(struct device *dev, phys_addr_t paddr) +{ + unsigned int offset = paddr & ~PAGE_MASK; + return pfn_to_dma(dev, __phys_to_pfn(paddr)) + offset; +} + +static inline phys_addr_t dma_to_phys(struct device *dev, dma_addr_t dev_addr) +{ + unsigned int offset = dev_addr & ~PAGE_MASK; + return __pfn_to_phys(dma_to_pfn(dev, dev_addr)) + offset; +} + +static inline bool dma_capable(struct device *dev, dma_addr_t addr, size_t size) +{ + u64 limit, mask; + + if (!dev->dma_mask) + return 0; + + mask = *dev->dma_mask; + + limit = (mask + 1) & ~mask; + if (limit && size > limit) + return 0; + + if ((addr | (addr + size - 1)) & ~mask) + return 0; + + return 1; +} + +#endif /* ASM_ARM_DMA_DIRECT_H */ diff --git a/arch/arm/include/asm/dma-mapping.h b/arch/arm/include/asm/dma-mapping.h index daf837423a76..5fb1b7fbdfbe 100644 --- a/arch/arm/include/asm/dma-mapping.h +++ b/arch/arm/include/asm/dma-mapping.h @@ -109,37 +109,6 @@ static inline bool is_device_dma_coherent(struct device *dev) return dev->archdata.dma_coherent; } -static inline dma_addr_t phys_to_dma(struct device *dev, phys_addr_t paddr) -{ - unsigned int offset = paddr & ~PAGE_MASK; - return pfn_to_dma(dev, __phys_to_pfn(paddr)) + offset; -} - -static inline phys_addr_t dma_to_phys(struct device *dev, dma_addr_t dev_addr) -{ - unsigned int offset = dev_addr & ~PAGE_MASK; - return __pfn_to_phys(dma_to_pfn(dev, dev_addr)) + offset; -} - -static inline bool dma_capable(struct device *dev, dma_addr_t addr, size_t size) -{ - u64 limit, mask; - - if (!dev->dma_mask) - return 0; - - mask = *dev->dma_mask; - - limit = (mask + 1) & ~mask; - if (limit && size > limit) - return 0; - - if ((addr | (addr + size - 1)) & ~mask) - return 0; - - return 1; -} - static inline void dma_mark_clean(void *addr, size_t size) { } /** diff --git a/arch/arm64/include/asm/dma-mapping.h b/arch/arm64/include/asm/dma-mapping.h index eada887a93bf..400fa67d3b5a 100644 --- a/arch/arm64/include/asm/dma-mapping.h +++ b/arch/arm64/include/asm/dma-mapping.h @@ -50,28 +50,6 @@ static inline bool is_device_dma_coherent(struct device *dev) return dev->archdata.dma_coherent; } -static inline dma_addr_t phys_to_dma(struct device *dev, phys_addr_t paddr) -{ - dma_addr_t dev_addr = (dma_addr_t)paddr; - - return dev_addr - ((dma_addr_t)dev->dma_pfn_offset << PAGE_SHIFT); -} - -static inline phys_addr_t dma_to_phys(struct device *dev, dma_addr_t dev_addr) -{ - phys_addr_t paddr = (phys_addr_t)dev_addr; - - return paddr + ((phys_addr_t)dev->dma_pfn_offset << PAGE_SHIFT); -} - -static inline bool dma_capable(struct device *dev, dma_addr_t addr, size_t size) -{ - if (!dev->dma_mask) - return false; - - return addr + size - 1 <= *dev->dma_mask; -} - static inline void dma_mark_clean(void *addr, size_t size) { } diff --git a/arch/arm64/mm/dma-mapping.c b/arch/arm64/mm/dma-mapping.c index b45c5bcaeccb..f3a637b98487 100644 --- a/arch/arm64/mm/dma-mapping.c +++ b/arch/arm64/mm/dma-mapping.c @@ -24,7 +24,7 @@ #include #include #include -#include +#include #include #include #include diff --git a/arch/ia64/include/asm/dma-mapping.h b/arch/ia64/include/asm/dma-mapping.h index c1bab526a046..eabee56d995c 100644 --- a/arch/ia64/include/asm/dma-mapping.h +++ b/arch/ia64/include/asm/dma-mapping.h @@ -27,22 +27,4 @@ static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus) return platform_dma_get_ops(NULL); } -static inline bool dma_capable(struct device *dev, dma_addr_t addr, size_t size) -{ - if (!dev->dma_mask) - return 0; - - return addr + size - 1 <= *dev->dma_mask; -} - -static inline dma_addr_t phys_to_dma(struct device *dev, phys_addr_t paddr) -{ - return paddr; -} - -static inline phys_addr_t dma_to_phys(struct device *dev, dma_addr_t daddr) -{ - return daddr; -} - #endif /* _ASM_IA64_DMA_MAPPING_H */ diff --git a/arch/mips/Kconfig b/arch/mips/Kconfig index 350a990fc719..4b0c26b2e9b7 100644 --- a/arch/mips/Kconfig +++ b/arch/mips/Kconfig @@ -429,6 +429,7 @@ config MACH_LOONGSON32 config MACH_LOONGSON64 bool "Loongson-2/3 family of machines" + select ARCH_HAS_PHYS_TO_DMA select SYS_SUPPORTS_ZBOOT help This enables the support of Loongson-2/3 family of machines. @@ -877,6 +878,7 @@ config MIKROTIK_RB532 config CAVIUM_OCTEON_SOC bool "Cavium Networks Octeon SoC based boards" select CEVT_R4K + select ARCH_HAS_PHYS_TO_DMA select ARCH_PHYS_ADDR_T_64BIT select DMA_COHERENT select SYS_SUPPORTS_64BIT_KERNEL diff --git a/arch/mips/include/asm/dma-direct.h b/arch/mips/include/asm/dma-direct.h new file mode 100644 index 000000000000..f32f15530aba --- /dev/null +++ b/arch/mips/include/asm/dma-direct.h @@ -0,0 +1 @@ +#include diff --git a/arch/mips/include/asm/dma-mapping.h b/arch/mips/include/asm/dma-mapping.h index 0d9418d264f9..676c14cfc580 100644 --- a/arch/mips/include/asm/dma-mapping.h +++ b/arch/mips/include/asm/dma-mapping.h @@ -17,14 +17,6 @@ static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus) return mips_dma_map_ops; } -static inline bool dma_capable(struct device *dev, dma_addr_t addr, size_t size) -{ - if (!dev->dma_mask) - return false; - - return addr + size <= *dev->dma_mask; -} - static inline void dma_mark_clean(void *addr, size_t size) {} #define arch_setup_dma_ops arch_setup_dma_ops diff --git a/arch/mips/include/asm/mach-cavium-octeon/dma-coherence.h b/arch/mips/include/asm/mach-cavium-octeon/dma-coherence.h index 9110988b92a1..f00833acb626 100644 --- a/arch/mips/include/asm/mach-cavium-octeon/dma-coherence.h +++ b/arch/mips/include/asm/mach-cavium-octeon/dma-coherence.h @@ -61,6 +61,14 @@ static inline void plat_post_dma_flush(struct device *dev) { } +static inline bool dma_capable(struct device *dev, dma_addr_t addr, size_t size) +{ + if (!dev->dma_mask) + return false; + + return addr + size <= *dev->dma_mask; +} + dma_addr_t phys_to_dma(struct device *dev, phys_addr_t paddr); phys_addr_t dma_to_phys(struct device *dev, dma_addr_t daddr); diff --git a/arch/mips/include/asm/mach-generic/dma-coherence.h b/arch/mips/include/asm/mach-generic/dma-coherence.h index 61addb1677e9..8ad7a40ca786 100644 --- a/arch/mips/include/asm/mach-generic/dma-coherence.h +++ b/arch/mips/include/asm/mach-generic/dma-coherence.h @@ -70,16 +70,4 @@ static inline void plat_post_dma_flush(struct device *dev) } #endif -#ifdef CONFIG_SWIOTLB -static inline dma_addr_t phys_to_dma(struct device *dev, phys_addr_t paddr) -{ - return paddr; -} - -static inline phys_addr_t dma_to_phys(struct device *dev, dma_addr_t daddr) -{ - return daddr; -} -#endif - #endif /* __ASM_MACH_GENERIC_DMA_COHERENCE_H */ diff --git a/arch/mips/include/asm/mach-loongson64/dma-coherence.h b/arch/mips/include/asm/mach-loongson64/dma-coherence.h index 1602a9e9e8c2..5cfda8f893e9 100644 --- a/arch/mips/include/asm/mach-loongson64/dma-coherence.h +++ b/arch/mips/include/asm/mach-loongson64/dma-coherence.h @@ -17,6 +17,14 @@ struct device; +static inline bool dma_capable(struct device *dev, dma_addr_t addr, size_t size) +{ + if (!dev->dma_mask) + return false; + + return addr + size <= *dev->dma_mask; +} + extern dma_addr_t phys_to_dma(struct device *dev, phys_addr_t paddr); extern phys_addr_t dma_to_phys(struct device *dev, dma_addr_t daddr); static inline dma_addr_t plat_map_dma_mem(struct device *dev, void *addr, diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig index c51e6ce42e7a..887285eb684a 100644 --- a/arch/powerpc/Kconfig +++ b/arch/powerpc/Kconfig @@ -139,6 +139,7 @@ config PPC select ARCH_HAS_ELF_RANDOMIZE select ARCH_HAS_FORTIFY_SOURCE select ARCH_HAS_GCOV_PROFILE_ALL + select ARCH_HAS_PHYS_TO_DMA select ARCH_HAS_PMEM_API if PPC64 select ARCH_HAS_SCALED_CPUTIME if VIRT_CPU_ACCOUNTING_NATIVE select ARCH_HAS_SG_CHAIN diff --git a/arch/powerpc/include/asm/dma-direct.h b/arch/powerpc/include/asm/dma-direct.h new file mode 100644 index 000000000000..a5b59c765426 --- /dev/null +++ b/arch/powerpc/include/asm/dma-direct.h @@ -0,0 +1,29 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef ASM_POWERPC_DMA_DIRECT_H +#define ASM_POWERPC_DMA_DIRECT_H 1 + +static inline bool dma_capable(struct device *dev, dma_addr_t addr, size_t size) +{ +#ifdef CONFIG_SWIOTLB + struct dev_archdata *sd = &dev->archdata; + + if (sd->max_direct_dma_addr && addr + size > sd->max_direct_dma_addr) + return false; +#endif + + if (!dev->dma_mask) + return false; + + return addr + size - 1 <= *dev->dma_mask; +} + +static inline dma_addr_t phys_to_dma(struct device *dev, phys_addr_t paddr) +{ + return paddr + get_dma_offset(dev); +} + +static inline phys_addr_t dma_to_phys(struct device *dev, dma_addr_t daddr) +{ + return daddr - get_dma_offset(dev); +} +#endif /* ASM_POWERPC_DMA_DIRECT_H */ diff --git a/arch/powerpc/include/asm/dma-mapping.h b/arch/powerpc/include/asm/dma-mapping.h index 592c7f418aa0..f6ab51205a85 100644 --- a/arch/powerpc/include/asm/dma-mapping.h +++ b/arch/powerpc/include/asm/dma-mapping.h @@ -112,31 +112,6 @@ extern int dma_set_mask(struct device *dev, u64 dma_mask); extern u64 __dma_get_required_mask(struct device *dev); -static inline bool dma_capable(struct device *dev, dma_addr_t addr, size_t size) -{ -#ifdef CONFIG_SWIOTLB - struct dev_archdata *sd = &dev->archdata; - - if (sd->max_direct_dma_addr && addr + size > sd->max_direct_dma_addr) - return false; -#endif - - if (!dev->dma_mask) - return false; - - return addr + size - 1 <= *dev->dma_mask; -} - -static inline dma_addr_t phys_to_dma(struct device *dev, phys_addr_t paddr) -{ - return paddr + get_dma_offset(dev); -} - -static inline phys_addr_t dma_to_phys(struct device *dev, dma_addr_t daddr) -{ - return daddr - get_dma_offset(dev); -} - #define ARCH_HAS_DMA_MMAP_COHERENT #endif /* __KERNEL__ */ diff --git a/arch/tile/include/asm/dma-mapping.h b/arch/tile/include/asm/dma-mapping.h index 97ad62878290..75b8aaa4e70b 100644 --- a/arch/tile/include/asm/dma-mapping.h +++ b/arch/tile/include/asm/dma-mapping.h @@ -44,26 +44,8 @@ static inline void set_dma_offset(struct device *dev, dma_addr_t off) dev->archdata.dma_offset = off; } -static inline dma_addr_t phys_to_dma(struct device *dev, phys_addr_t paddr) -{ - return paddr; -} - -static inline phys_addr_t dma_to_phys(struct device *dev, dma_addr_t daddr) -{ - return daddr; -} - static inline void dma_mark_clean(void *addr, size_t size) {} -static inline bool dma_capable(struct device *dev, dma_addr_t addr, size_t size) -{ - if (!dev->dma_mask) - return 0; - - return addr + size - 1 <= *dev->dma_mask; -} - #define HAVE_ARCH_DMA_SET_MASK 1 int dma_set_mask(struct device *dev, u64 mask); diff --git a/arch/unicore32/include/asm/dma-mapping.h b/arch/unicore32/include/asm/dma-mapping.h index ac608c2f6af6..5cb250bf2d8c 100644 --- a/arch/unicore32/include/asm/dma-mapping.h +++ b/arch/unicore32/include/asm/dma-mapping.h @@ -25,24 +25,6 @@ static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus) return &swiotlb_dma_map_ops; } -static inline bool dma_capable(struct device *dev, dma_addr_t addr, size_t size) -{ - if (dev && dev->dma_mask) - return addr + size - 1 <= *dev->dma_mask; - - return 1; -} - -static inline dma_addr_t phys_to_dma(struct device *dev, phys_addr_t paddr) -{ - return paddr; -} - -static inline phys_addr_t dma_to_phys(struct device *dev, dma_addr_t daddr) -{ - return daddr; -} - static inline void dma_mark_clean(void *addr, size_t size) {} #endif /* __KERNEL__ */ diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index d4fc98c50378..f6f4328103c0 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -54,6 +54,7 @@ config X86 select ARCH_HAS_FORTIFY_SOURCE select ARCH_HAS_GCOV_PROFILE_ALL select ARCH_HAS_KCOV if X86_64 + select ARCH_HAS_PHYS_TO_DMA select ARCH_HAS_PMEM_API if X86_64 # Causing hangs/crashes, see the commit that added this change for details. select ARCH_HAS_REFCOUNT diff --git a/arch/x86/include/asm/dma-direct.h b/arch/x86/include/asm/dma-direct.h new file mode 100644 index 000000000000..1295bc622ebe --- /dev/null +++ b/arch/x86/include/asm/dma-direct.h @@ -0,0 +1,30 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef ASM_X86_DMA_DIRECT_H +#define ASM_X86_DMA_DIRECT_H 1 + +#include + +#ifdef CONFIG_X86_DMA_REMAP /* Platform code defines bridge-specific code */ +bool dma_capable(struct device *dev, dma_addr_t addr, size_t size); +dma_addr_t phys_to_dma(struct device *dev, phys_addr_t paddr); +phys_addr_t dma_to_phys(struct device *dev, dma_addr_t daddr); +#else +static inline bool dma_capable(struct device *dev, dma_addr_t addr, size_t size) +{ + if (!dev->dma_mask) + return 0; + + return addr + size - 1 <= *dev->dma_mask; +} + +static inline dma_addr_t phys_to_dma(struct device *dev, phys_addr_t paddr) +{ + return __sme_set(paddr); +} + +static inline phys_addr_t dma_to_phys(struct device *dev, dma_addr_t daddr) +{ + return __sme_clr(daddr); +} +#endif /* CONFIG_X86_DMA_REMAP */ +#endif /* ASM_X86_DMA_DIRECT_H */ diff --git a/arch/x86/include/asm/dma-mapping.h b/arch/x86/include/asm/dma-mapping.h index 0350d99bb8fd..dfdc9357a349 100644 --- a/arch/x86/include/asm/dma-mapping.h +++ b/arch/x86/include/asm/dma-mapping.h @@ -12,7 +12,6 @@ #include #include #include -#include #ifdef CONFIG_ISA # define ISA_DMA_BIT_MASK DMA_BIT_MASK(24) @@ -42,31 +41,6 @@ extern void dma_generic_free_coherent(struct device *dev, size_t size, void *vaddr, dma_addr_t dma_addr, unsigned long attrs); -#ifdef CONFIG_X86_DMA_REMAP /* Platform code defines bridge-specific code */ -extern bool dma_capable(struct device *dev, dma_addr_t addr, size_t size); -extern dma_addr_t phys_to_dma(struct device *dev, phys_addr_t paddr); -extern phys_addr_t dma_to_phys(struct device *dev, dma_addr_t daddr); -#else - -static inline bool dma_capable(struct device *dev, dma_addr_t addr, size_t size) -{ - if (!dev->dma_mask) - return 0; - - return addr + size - 1 <= *dev->dma_mask; -} - -static inline dma_addr_t phys_to_dma(struct device *dev, phys_addr_t paddr) -{ - return __sme_set(paddr); -} - -static inline phys_addr_t dma_to_phys(struct device *dev, dma_addr_t daddr) -{ - return __sme_clr(daddr); -} -#endif /* CONFIG_X86_DMA_REMAP */ - static inline unsigned long dma_alloc_coherent_mask(struct device *dev, gfp_t gfp) { diff --git a/arch/x86/kernel/amd_gart_64.c b/arch/x86/kernel/amd_gart_64.c index cc0e8bc0ea3f..ecd486cb06ab 100644 --- a/arch/x86/kernel/amd_gart_64.c +++ b/arch/x86/kernel/amd_gart_64.c @@ -31,6 +31,7 @@ #include #include #include +#include #include #include #include diff --git a/arch/x86/kernel/pci-dma.c b/arch/x86/kernel/pci-dma.c index 599d7462eccc..8439e6de6156 100644 --- a/arch/x86/kernel/pci-dma.c +++ b/arch/x86/kernel/pci-dma.c @@ -1,5 +1,5 @@ // SPDX-License-Identifier: GPL-2.0 -#include +#include #include #include #include diff --git a/arch/x86/kernel/pci-nommu.c b/arch/x86/kernel/pci-nommu.c index 8ac9d1fe0373..8b8ff98486de 100644 --- a/arch/x86/kernel/pci-nommu.c +++ b/arch/x86/kernel/pci-nommu.c @@ -1,7 +1,7 @@ // SPDX-License-Identifier: GPL-2.0 /* Fallback functions when the main IOMMU code is not compiled in. This code is roughly equivalent to i386. */ -#include +#include #include #include #include diff --git a/arch/x86/kernel/pci-swiotlb.c b/arch/x86/kernel/pci-swiotlb.c index 53bd05ea90d8..9d3e35c33d94 100644 --- a/arch/x86/kernel/pci-swiotlb.c +++ b/arch/x86/kernel/pci-swiotlb.c @@ -6,7 +6,7 @@ #include #include #include -#include +#include #include #include diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c index d9a9e9fc75dd..764b916ef7da 100644 --- a/arch/x86/mm/mem_encrypt.c +++ b/arch/x86/mm/mem_encrypt.c @@ -15,7 +15,7 @@ #include #include #include -#include +#include #include #include diff --git a/arch/x86/pci/sta2x11-fixup.c b/arch/x86/pci/sta2x11-fixup.c index 53d600217973..75577c1490c4 100644 --- a/arch/x86/pci/sta2x11-fixup.c +++ b/arch/x86/pci/sta2x11-fixup.c @@ -26,6 +26,7 @@ #include #include #include +#include #include #define STA2X11_SWIOTLB_SIZE (4*1024*1024) diff --git a/arch/xtensa/include/asm/dma-mapping.h b/arch/xtensa/include/asm/dma-mapping.h index 153bf2370988..44098800dad7 100644 --- a/arch/xtensa/include/asm/dma-mapping.h +++ b/arch/xtensa/include/asm/dma-mapping.h @@ -23,14 +23,4 @@ static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus) return &xtensa_dma_map_ops; } -static inline dma_addr_t phys_to_dma(struct device *dev, phys_addr_t paddr) -{ - return (dma_addr_t)paddr; -} - -static inline phys_addr_t dma_to_phys(struct device *dev, dma_addr_t daddr) -{ - return (phys_addr_t)daddr; -} - #endif /* _XTENSA_DMA_MAPPING_H */ diff --git a/drivers/crypto/marvell/cesa.c b/drivers/crypto/marvell/cesa.c index 293832488cc9..3a0c40081ffb 100644 --- a/drivers/crypto/marvell/cesa.c +++ b/drivers/crypto/marvell/cesa.c @@ -24,6 +24,7 @@ #include #include #include +#include /* XXX: drivers shall never use this directly! */ #include #include #include diff --git a/drivers/mtd/nand/qcom_nandc.c b/drivers/mtd/nand/qcom_nandc.c index 2656c1ac5646..411cdfd12a85 100644 --- a/drivers/mtd/nand/qcom_nandc.c +++ b/drivers/mtd/nand/qcom_nandc.c @@ -23,6 +23,7 @@ #include #include #include +#include /* XXX: drivers shall never use this directly! */ /* NANDc reg offsets */ #define NAND_FLASH_CMD 0x00 diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c index 82fc54f8eb77..5bb72d3f8337 100644 --- a/drivers/xen/swiotlb-xen.c +++ b/drivers/xen/swiotlb-xen.c @@ -36,7 +36,7 @@ #define pr_fmt(fmt) "xen:" KBUILD_MODNAME ": " fmt #include -#include +#include #include #include #include diff --git a/include/linux/dma-direct.h b/include/linux/dma-direct.h new file mode 100644 index 000000000000..2cc1b6558944 --- /dev/null +++ b/include/linux/dma-direct.h @@ -0,0 +1,32 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _LINUX_DMA_DIRECT_H +#define _LINUX_DMA_DIRECT_H 1 + +#include + +#ifdef CONFIG_ARCH_HAS_PHYS_TO_DMA +#include +#else +static inline dma_addr_t phys_to_dma(struct device *dev, phys_addr_t paddr) +{ + dma_addr_t dev_addr = (dma_addr_t)paddr; + + return dev_addr - ((dma_addr_t)dev->dma_pfn_offset << PAGE_SHIFT); +} + +static inline phys_addr_t dma_to_phys(struct device *dev, dma_addr_t dev_addr) +{ + phys_addr_t paddr = (phys_addr_t)dev_addr; + + return paddr + ((phys_addr_t)dev->dma_pfn_offset << PAGE_SHIFT); +} + +static inline bool dma_capable(struct device *dev, dma_addr_t addr, size_t size) +{ + if (!dev->dma_mask) + return false; + + return addr + size - 1 <= *dev->dma_mask; +} +#endif /* !CONFIG_ARCH_HAS_PHYS_TO_DMA */ +#endif /* _LINUX_DMA_DIRECT_H */ diff --git a/lib/swiotlb.c b/lib/swiotlb.c index cea19aaf303c..6583f3512386 100644 --- a/lib/swiotlb.c +++ b/lib/swiotlb.c @@ -18,7 +18,7 @@ */ #include -#include +#include #include #include #include