From patchwork Wed Jul 6 23:15:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Heiko Stuebner X-Patchwork-Id: 12908772 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1AB00C43334 for ; Wed, 6 Jul 2022 23:16:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=nNrgiCGIv27fw2TXTYi2DBPbGT7Kkj+i02GMbe7Shbg=; b=Fpul6nMXx6m/eZ rzokKRx9T+FbbxXX7lV/ybs/MkAYo/34oxb7iosl639eHQSw0g01iorc6ADLmKS8/RgoyJmTlXlAc 0MGNBSxp2JTj4fp/7e4ooa4ZWbWq+Ii3wqOrPdG+50clPUopDL96al7SdS7vDHdqtazSvg0+/bIYw Oe8hwL6Dfo86WD62F7aMXdEu4JAoSvL0XddOL9BSD4mOUGYFHYQiO4FiKgU8/jivbdcAdyB9Xu1iV B3PBkGNq2GOeJiQRXX+2kSDX4I0QFlBeX6ftvOIuwukFCKRcvS3N1aFaEOss/prBS9MXtvA/81Hzz jXA6jmCQ7fQDeRA5MQdg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1o9EFK-00CiLf-66; Wed, 06 Jul 2022 23:15:50 +0000 Received: from gloria.sntech.de ([185.11.138.130]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1o9EFF-00CiJ1-J6 for linux-riscv@lists.infradead.org; Wed, 06 Jul 2022 23:15:48 +0000 Received: from ip5b412258.dynamic.kabel-deutschland.de ([91.65.34.88] helo=phil.lan) by gloria.sntech.de with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1o9EFD-0002yn-C5; Thu, 07 Jul 2022 01:15:43 +0200 From: Heiko Stuebner To: palmer@dabbelt.com, paul.walmsley@sifive.com Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, wefu@redhat.com, guoren@kernel.org, cmuellner@linux.com, philipp.tomsich@vrull.eu, hch@lst.de, samuel@sholland.org, atishp@atishpatra.org, anup@brainfault.org, mick@ics.forth.gr, robh+dt@kernel.org, krzk+dt@kernel.org, devicetree@vger.kernel.org, drew@beagleboard.org, rdunlap@infradead.org, Heiko Stuebner , Rob Herring Subject: [PATCH v7 1/4] of: also handle dma-noncoherent in of_dma_is_coherent() Date: Thu, 7 Jul 2022 01:15:33 +0200 Message-Id: <20220706231536.2041855-2-heiko@sntech.de> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220706231536.2041855-1-heiko@sntech.de> References: <20220706231536.2041855-1-heiko@sntech.de> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220706_161545_715802_6791FD00 X-CRM114-Status: GOOD ( 15.24 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org of_dma_is_coherent() currently expects the architecture to be non-coherent and some devices being coherent getting marked as such with the dma-coherent devicetree property. For PowerPC CONFIG_OF_DMA_DEFAULT_COHERENT was added which currently makes of_dma_is_coherent() always return true but doesn't handle the case of the architecture being coherent but some devices not. So modify the function to also check for dma-noncoherent and set a suitable default return value. If CONFIG_OF_DMA_DEFAULT_COHERENT is set the value starts with true and finding dma-noncoherent will set it to false and without CONFIG_OF_DMA_DEFAULT_COHERENT, the behaviour is reversed. Reviewed-by: Christoph Hellwig Reviewed-by: Rob Herring Reviewed-by: Guo Ren Signed-off-by: Heiko Stuebner --- drivers/of/address.c | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/drivers/of/address.c b/drivers/of/address.c index 94f017d808c4..96f0a12e507c 100644 --- a/drivers/of/address.c +++ b/drivers/of/address.c @@ -1045,26 +1045,29 @@ phys_addr_t __init of_dma_get_max_cpu_address(struct device_node *np) * * It returns true if "dma-coherent" property was found * for this device in the DT, or if DMA is coherent by - * default for OF devices on the current platform. + * default for OF devices on the current platform and no + * "dma-noncoherent" property was found for this device. */ bool of_dma_is_coherent(struct device_node *np) { struct device_node *node; - - if (IS_ENABLED(CONFIG_OF_DMA_DEFAULT_COHERENT)) - return true; + bool is_coherent = IS_ENABLED(CONFIG_OF_DMA_DEFAULT_COHERENT); node = of_node_get(np); while (node) { if (of_property_read_bool(node, "dma-coherent")) { - of_node_put(node); - return true; + is_coherent = true; + break; + } + if (of_property_read_bool(node, "dma-noncoherent")) { + is_coherent = false; + break; } node = of_get_next_dma_parent(node); } of_node_put(node); - return false; + return is_coherent; } EXPORT_SYMBOL_GPL(of_dma_is_coherent); From patchwork Wed Jul 6 23:15:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Heiko Stuebner X-Patchwork-Id: 12908774 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E3638CCA480 for ; Wed, 6 Jul 2022 23:16:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=zjdbbaeYgRvvj3m1iOcaXossBior7d7/69sWpIhEVR8=; b=st6th8T49+vG9A ruCaPFlhauN/iPj7Pq3iTYmJRRollATyrAZfjAR+KRBGIRxb5k3ReiQ5pvdYhpWblirYYQvY8tM4U 3oZCgUtVznD2RkTjE0saOE84+qBMUURmhzyidiapZraPAgj3+HveLY9wjc0+OFKgQ6eCgsf1/mkQz 6OQ2uyr8+baKm5RPnO32QQKnOD3ttRs6M2INCNzROJtgG6sj4HGmJMV8X9pLOv8C31z8InS6WcVMW N4goQbY/Ce1v7MC01R//yJ53BYe7WR9BFT8+6/uChomscAvChAqyt+N+/7LTBALhV0AYK2QMpcCRp 2/4PTD5VqiMmb+/8PG+Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1o9EFM-00CiMJ-0N; Wed, 06 Jul 2022 23:15:52 +0000 Received: from gloria.sntech.de ([185.11.138.130]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1o9EFF-00CiJE-UR for linux-riscv@lists.infradead.org; Wed, 06 Jul 2022 23:15:48 +0000 Received: from ip5b412258.dynamic.kabel-deutschland.de ([91.65.34.88] helo=phil.lan) by gloria.sntech.de with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1o9EFD-0002yn-UG; Thu, 07 Jul 2022 01:15:43 +0200 From: Heiko Stuebner To: palmer@dabbelt.com, paul.walmsley@sifive.com Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, wefu@redhat.com, guoren@kernel.org, cmuellner@linux.com, philipp.tomsich@vrull.eu, hch@lst.de, samuel@sholland.org, atishp@atishpatra.org, anup@brainfault.org, mick@ics.forth.gr, robh+dt@kernel.org, krzk+dt@kernel.org, devicetree@vger.kernel.org, drew@beagleboard.org, rdunlap@infradead.org, Heiko Stuebner , Rob Herring Subject: [PATCH v7 2/4] dt-bindings: riscv: document cbom-block-size Date: Thu, 7 Jul 2022 01:15:34 +0200 Message-Id: <20220706231536.2041855-3-heiko@sntech.de> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220706231536.2041855-1-heiko@sntech.de> References: <20220706231536.2041855-1-heiko@sntech.de> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220706_161546_013530_D42864E0 X-CRM114-Status: UNSURE ( 9.96 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org The Zicbom operates on a block-size defined for the cpu-core, which does not necessarily match other cache-sizes used. So add the necessary property for the system to know the core's block-size. Reviewed-by: Anup Patel Reviewed-by: Guo Ren Acked-by: Rob Herring Signed-off-by: Heiko Stuebner --- Documentation/devicetree/bindings/riscv/cpus.yaml | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/Documentation/devicetree/bindings/riscv/cpus.yaml b/Documentation/devicetree/bindings/riscv/cpus.yaml index d632ac76532e..873dd12f6e89 100644 --- a/Documentation/devicetree/bindings/riscv/cpus.yaml +++ b/Documentation/devicetree/bindings/riscv/cpus.yaml @@ -63,6 +63,11 @@ properties: - riscv,sv48 - riscv,none + riscv,cbom-block-size: + $ref: /schemas/types.yaml#/definitions/uint32 + description: + The blocksize in bytes for the Zicbom cache operations. + riscv,isa: description: Identifies the specific RISC-V instruction set architecture From patchwork Wed Jul 6 23:15:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Heiko Stuebner X-Patchwork-Id: 12908776 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C1E5FCCA47C for ; Wed, 6 Jul 2022 23:16:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=GfyjT+ZVoRVagaQkk8Zc10FXugOAVVIf7YGF/yOVdbk=; b=l2sdrgeu+CB0vp sepbLu511qsujGgsOfipfmt3+OCti10t7nQlLxDy0HAgKbbz535fiUEj7C8Dp3gdoQvnbflKhpEzK 1aaMO390Ja6Jigcy18THShBjKNxFgJP3c1t+68OPXagdynBcUeA4o+o2CUXonTwIqU5BfLTQ+xoQg 9ij0I7QyMwL/fAj1Qhf6KBXV7x+feVVHEmhz3QkoFNW0DSbSyNXqUU3L14C2HZ+4yQUIfdUydvAQf H5DSsEjv4Q2jYD1LeY7JvOoQlecgzwzWjQPWeQXYFVi0BpRDyNk1CGyIPCLLK5+xeQDVeSW74CWBd 6CaeaHRw3sa/oFGsjajQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1o9EFT-00CiQi-71; Wed, 06 Jul 2022 23:15:59 +0000 Received: from gloria.sntech.de ([185.11.138.130]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1o9EFI-00CiKj-W0 for linux-riscv@lists.infradead.org; Wed, 06 Jul 2022 23:15:51 +0000 Received: from ip5b412258.dynamic.kabel-deutschland.de ([91.65.34.88] helo=phil.lan) by gloria.sntech.de with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1o9EFE-0002yn-HX; Thu, 07 Jul 2022 01:15:44 +0200 From: Heiko Stuebner To: palmer@dabbelt.com, paul.walmsley@sifive.com Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, wefu@redhat.com, guoren@kernel.org, cmuellner@linux.com, philipp.tomsich@vrull.eu, hch@lst.de, samuel@sholland.org, atishp@atishpatra.org, anup@brainfault.org, mick@ics.forth.gr, robh+dt@kernel.org, krzk+dt@kernel.org, devicetree@vger.kernel.org, drew@beagleboard.org, rdunlap@infradead.org, Heiko Stuebner , Atish Patra Subject: [PATCH v7 3/4] riscv: Add support for non-coherent devices using zicbom extension Date: Thu, 7 Jul 2022 01:15:35 +0200 Message-Id: <20220706231536.2041855-4-heiko@sntech.de> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220706231536.2041855-1-heiko@sntech.de> References: <20220706231536.2041855-1-heiko@sntech.de> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220706_161549_234345_F80DAD05 X-CRM114-Status: GOOD ( 28.20 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org The Zicbom ISA-extension was ratified in november 2021 and introduces instructions for dcache invalidate, clean and flush operations. Implement cache management operations for non-coherent devices based on them. Of course not all cores will support this, so implement an alternative-based mechanism that replaces empty instructions with ones done around Zicbom instructions. As discussed in previous versions, assume the platform being coherent by default so that non-coherent devices need to get marked accordingly by firmware. Reviewed-by: Christoph Hellwig Signed-off-by: Heiko Stuebner Cc: Christoph Hellwig Cc: Atish Patra Cc: Guo Ren Cc: Anup Patel Reviewed-by: Guo Ren --- arch/riscv/Kconfig | 31 ++++++++ arch/riscv/Makefile | 4 + arch/riscv/include/asm/cache.h | 4 + arch/riscv/include/asm/cacheflush.h | 10 +++ arch/riscv/include/asm/errata_list.h | 19 ++++- arch/riscv/include/asm/hwcap.h | 1 + arch/riscv/kernel/cpu.c | 1 + arch/riscv/kernel/cpufeature.c | 24 ++++++ arch/riscv/kernel/setup.c | 2 + arch/riscv/mm/Makefile | 1 + arch/riscv/mm/dma-noncoherent.c | 112 +++++++++++++++++++++++++++ 11 files changed, 208 insertions(+), 1 deletion(-) create mode 100644 arch/riscv/mm/dma-noncoherent.c diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index 32ffef9f6e5b..897ae28abf81 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -113,6 +113,7 @@ config RISCV select MODULES_USE_ELF_RELA if MODULES select MODULE_SECTIONS if MODULES select OF + select OF_DMA_DEFAULT_COHERENT select OF_EARLY_FLATTREE select OF_IRQ select PCI_DOMAINS_GENERIC if PCI @@ -218,6 +219,14 @@ config PGTABLE_LEVELS config LOCKDEP_SUPPORT def_bool y +config RISCV_DMA_NONCOHERENT + bool + select ARCH_HAS_DMA_PREP_COHERENT + select ARCH_HAS_SYNC_DMA_FOR_DEVICE + select ARCH_HAS_SYNC_DMA_FOR_CPU + select ARCH_HAS_SETUP_DMA_OPS + select DMA_DIRECT_REMAP + source "arch/riscv/Kconfig.socs" source "arch/riscv/Kconfig.erratas" @@ -376,6 +385,28 @@ config RISCV_ISA_SVPBMT If you don't know what to do here, say Y. +config CC_HAS_ZICBOM + bool + default y if 64BIT && $(cc-option,-mabi=lp64 -march=rv64ima_zicbom) + default y if 32BIT && $(cc-option,-mabi=ilp32 -march=rv32ima_zicbom) + +config RISCV_ISA_ZICBOM + bool "Zicbom extension support for non-coherent DMA operation" + depends on CC_HAS_ZICBOM + depends on !XIP_KERNEL + select RISCV_DMA_NONCOHERENT + select RISCV_ALTERNATIVE + default y + help + Adds support to dynamically detect the presence of the ZICBOM + extension (Cache Block Management Operations) and enable its + usage. + + The Zicbom extension can be used to handle for example + non-coherent DMA support on devices that need it. + + If you don't know what to do here, say Y. + config FPU bool "FPU support" default y diff --git a/arch/riscv/Makefile b/arch/riscv/Makefile index 34cf8a598617..fbaabc98b3d2 100644 --- a/arch/riscv/Makefile +++ b/arch/riscv/Makefile @@ -56,6 +56,10 @@ riscv-march-$(CONFIG_RISCV_ISA_C) := $(riscv-march-y)c toolchain-need-zicsr-zifencei := $(call cc-option-yn, -march=$(riscv-march-y)_zicsr_zifencei) riscv-march-$(toolchain-need-zicsr-zifencei) := $(riscv-march-y)_zicsr_zifencei +# Check if the toolchain supports Zicbom extension +toolchain-supports-zicbom := $(call cc-option-yn, -march=$(riscv-march-y)_zicbom) +riscv-march-$(toolchain-supports-zicbom) := $(riscv-march-y)_zicbom + KBUILD_CFLAGS += -march=$(subst fd,,$(riscv-march-y)) KBUILD_AFLAGS += -march=$(riscv-march-y) diff --git a/arch/riscv/include/asm/cache.h b/arch/riscv/include/asm/cache.h index 9b58b104559e..d3036df23ccb 100644 --- a/arch/riscv/include/asm/cache.h +++ b/arch/riscv/include/asm/cache.h @@ -11,6 +11,10 @@ #define L1_CACHE_BYTES (1 << L1_CACHE_SHIFT) +#ifdef CONFIG_RISCV_DMA_NONCOHERENT +#define ARCH_DMA_MINALIGN L1_CACHE_BYTES +#endif + /* * RISC-V requires the stack pointer to be 16-byte aligned, so ensure that * the flat loader aligns it accordingly. diff --git a/arch/riscv/include/asm/cacheflush.h b/arch/riscv/include/asm/cacheflush.h index 23ff70350992..a60acaecfeda 100644 --- a/arch/riscv/include/asm/cacheflush.h +++ b/arch/riscv/include/asm/cacheflush.h @@ -42,6 +42,16 @@ void flush_icache_mm(struct mm_struct *mm, bool local); #endif /* CONFIG_SMP */ +#ifdef CONFIG_RISCV_ISA_ZICBOM +void riscv_init_cbom_blocksize(void); +#else +static inline void riscv_init_cbom_blocksize(void) { } +#endif + +#ifdef CONFIG_RISCV_DMA_NONCOHERENT +void riscv_noncoherent_supported(void); +#endif + /* * Bits in sys_riscv_flush_icache()'s flags argument. */ diff --git a/arch/riscv/include/asm/errata_list.h b/arch/riscv/include/asm/errata_list.h index 398e351e7002..79d89aeeaa6c 100644 --- a/arch/riscv/include/asm/errata_list.h +++ b/arch/riscv/include/asm/errata_list.h @@ -20,7 +20,8 @@ #endif #define CPUFEATURE_SVPBMT 0 -#define CPUFEATURE_NUMBER 1 +#define CPUFEATURE_ZICBOM 1 +#define CPUFEATURE_NUMBER 2 #ifdef __ASSEMBLY__ @@ -87,6 +88,22 @@ asm volatile(ALTERNATIVE( \ #define ALT_THEAD_PMA(_val) #endif +#define ALT_CMO_OP(_op, _start, _size, _cachesize) \ +asm volatile(ALTERNATIVE( \ + __nops(5), \ + "mv a0, %1\n\t" \ + "j 2f\n\t" \ + "3:\n\t" \ + "cbo." __stringify(_op) " (a0)\n\t" \ + "add a0, a0, %0\n\t" \ + "2:\n\t" \ + "bltu a0, %2, 3b\n\t", 0, \ + CPUFEATURE_ZICBOM, CONFIG_RISCV_ISA_ZICBOM) \ + : : "r"(_cachesize), \ + "r"((unsigned long)(_start) & ~((_cachesize) - 1UL)), \ + "r"((unsigned long)(_start) + (_size)) \ + : "a0") + #endif /* __ASSEMBLY__ */ #endif diff --git a/arch/riscv/include/asm/hwcap.h b/arch/riscv/include/asm/hwcap.h index 4e2486881840..6044e402003d 100644 --- a/arch/riscv/include/asm/hwcap.h +++ b/arch/riscv/include/asm/hwcap.h @@ -53,6 +53,7 @@ extern unsigned long elf_hwcap; enum riscv_isa_ext_id { RISCV_ISA_EXT_SSCOFPMF = RISCV_ISA_EXT_BASE, RISCV_ISA_EXT_SVPBMT, + RISCV_ISA_EXT_ZICBOM, RISCV_ISA_EXT_ID_MAX = RISCV_ISA_EXT_MAX, }; diff --git a/arch/riscv/kernel/cpu.c b/arch/riscv/kernel/cpu.c index fba9e9f46a8c..0365557f7122 100644 --- a/arch/riscv/kernel/cpu.c +++ b/arch/riscv/kernel/cpu.c @@ -89,6 +89,7 @@ int riscv_of_parent_hartid(struct device_node *node) static struct riscv_isa_ext_data isa_ext_arr[] = { __RISCV_ISA_EXT_DATA(sscofpmf, RISCV_ISA_EXT_SSCOFPMF), __RISCV_ISA_EXT_DATA(svpbmt, RISCV_ISA_EXT_SVPBMT), + __RISCV_ISA_EXT_DATA(zicbom, RISCV_ISA_EXT_ZICBOM), __RISCV_ISA_EXT_DATA("", RISCV_ISA_EXT_MAX), }; diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c index 6a40cb8134bd..d01a792a7201 100644 --- a/arch/riscv/kernel/cpufeature.c +++ b/arch/riscv/kernel/cpufeature.c @@ -12,6 +12,7 @@ #include #include #include +#include #include #include #include @@ -199,6 +200,7 @@ void __init riscv_fill_hwcap(void) } else { SET_ISA_EXT_MAP("sscofpmf", RISCV_ISA_EXT_SSCOFPMF); SET_ISA_EXT_MAP("svpbmt", RISCV_ISA_EXT_SVPBMT); + SET_ISA_EXT_MAP("zicbom", RISCV_ISA_EXT_ZICBOM); } #undef SET_ISA_EXT_MAP } @@ -259,6 +261,25 @@ static bool __init_or_module cpufeature_probe_svpbmt(unsigned int stage) return false; } +static bool __init_or_module cpufeature_probe_zicbom(unsigned int stage) +{ +#ifdef CONFIG_RISCV_ISA_ZICBOM + switch (stage) { + case RISCV_ALTERNATIVES_EARLY_BOOT: + return false; + default: + if (riscv_isa_extension_available(NULL, ZICBOM)) { + riscv_noncoherent_supported(); + return true; + } else { + return false; + } + } +#endif + + return false; +} + /* * Probe presence of individual extensions. * @@ -273,6 +294,9 @@ static u32 __init_or_module cpufeature_probe(unsigned int stage) if (cpufeature_probe_svpbmt(stage)) cpu_req_feature |= (1U << CPUFEATURE_SVPBMT); + if (cpufeature_probe_zicbom(stage)) + cpu_req_feature |= (1U << CPUFEATURE_ZICBOM); + return cpu_req_feature; } diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c index f0f36a4a0e9b..95ef6e2bf45c 100644 --- a/arch/riscv/kernel/setup.c +++ b/arch/riscv/kernel/setup.c @@ -22,6 +22,7 @@ #include #include +#include #include #include #include @@ -296,6 +297,7 @@ void __init setup_arch(char **cmdline_p) #endif riscv_fill_hwcap(); + riscv_init_cbom_blocksize(); apply_boot_alternatives(); } diff --git a/arch/riscv/mm/Makefile b/arch/riscv/mm/Makefile index ac7a25298a04..d76aabf4b94d 100644 --- a/arch/riscv/mm/Makefile +++ b/arch/riscv/mm/Makefile @@ -30,3 +30,4 @@ endif endif obj-$(CONFIG_DEBUG_VIRTUAL) += physaddr.o +obj-$(CONFIG_RISCV_DMA_NONCOHERENT) += dma-noncoherent.o diff --git a/arch/riscv/mm/dma-noncoherent.c b/arch/riscv/mm/dma-noncoherent.c new file mode 100644 index 000000000000..a8dc0bd9078d --- /dev/null +++ b/arch/riscv/mm/dma-noncoherent.c @@ -0,0 +1,112 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * RISC-V specific functions to support DMA for non-coherent devices + * + * Copyright (c) 2021 Western Digital Corporation or its affiliates. + */ + +#include +#include +#include +#include +#include +#include + +static unsigned int riscv_cbom_block_size = L1_CACHE_BYTES; +static bool noncoherent_supported; + +void arch_sync_dma_for_device(phys_addr_t paddr, size_t size, + enum dma_data_direction dir) +{ + void *vaddr = phys_to_virt(paddr); + + switch (dir) { + case DMA_TO_DEVICE: + ALT_CMO_OP(clean, vaddr, size, riscv_cbom_block_size); + break; + case DMA_FROM_DEVICE: + ALT_CMO_OP(clean, vaddr, size, riscv_cbom_block_size); + break; + case DMA_BIDIRECTIONAL: + ALT_CMO_OP(flush, vaddr, size, riscv_cbom_block_size); + break; + default: + break; + } +} + +void arch_sync_dma_for_cpu(phys_addr_t paddr, size_t size, + enum dma_data_direction dir) +{ + void *vaddr = phys_to_virt(paddr); + + switch (dir) { + case DMA_TO_DEVICE: + break; + case DMA_FROM_DEVICE: + case DMA_BIDIRECTIONAL: + ALT_CMO_OP(flush, vaddr, size, riscv_cbom_block_size); + break; + default: + break; + } +} + +void arch_dma_prep_coherent(struct page *page, size_t size) +{ + void *flush_addr = page_address(page); + + ALT_CMO_OP(flush, flush_addr, size, riscv_cbom_block_size); +} + +void arch_setup_dma_ops(struct device *dev, u64 dma_base, u64 size, + const struct iommu_ops *iommu, bool coherent) +{ + WARN_TAINT(!coherent && riscv_cbom_block_size > ARCH_DMA_MINALIGN, + TAINT_CPU_OUT_OF_SPEC, + "%s %s: ARCH_DMA_MINALIGN smaller than riscv,cbom-block-size (%d < %d)", + dev_driver_string(dev), dev_name(dev), + ARCH_DMA_MINALIGN, riscv_cbom_block_size); + + WARN_TAINT(!coherent && !noncoherent_supported, TAINT_CPU_OUT_OF_SPEC, + "%s %s: device non-coherent but no non-coherent operations supported", + dev_driver_string(dev), dev_name(dev)); + + dev->dma_coherent = coherent; +} + +#ifdef CONFIG_RISCV_ISA_ZICBOM +void riscv_init_cbom_blocksize(void) +{ + struct device_node *node; + int ret; + u32 val; + + for_each_of_cpu_node(node) { + int hartid = riscv_of_processor_hartid(node); + int cbom_hartid; + + if (hartid < 0) + continue; + + /* set block-size for cbom extension if available */ + ret = of_property_read_u32(node, "riscv,cbom-block-size", &val); + if (ret) + continue; + + if (!riscv_cbom_block_size) { + riscv_cbom_block_size = val; + cbom_hartid = hartid; + } else { + if (riscv_cbom_block_size != val) + pr_warn("cbom-block-size mismatched between harts %d and %d\n", + cbom_hartid, hartid); + } + } +} +#endif + +void riscv_noncoherent_supported(void) +{ + noncoherent_supported = true; +} From patchwork Wed Jul 6 23:15:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Heiko Stuebner X-Patchwork-Id: 12908773 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8CB2FC43334 for ; Wed, 6 Jul 2022 23:16:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=irXMZPlQRut1a0EpH8yDZE93al/G/rl/5BRSlEM1tKU=; b=3kMf0z5eg2xfLM +I4a7Ph4eyK7iJUNeDLBA8ZAeUqIjlPe4Ppn6yO/B1UmvXXLUFIYjedoVOcP3J1daAfz/+FdnmXXm cOIHjOeoxAFYs1slbeZc3u1qPMOetWFkcqj7Cef8Qll/ayjsRR2EQoAn3e/ufvdwmMpcAGtTMx/ni 3QWJUcItrsngwGuYdruB2sIse2IO0nqMA5wjNILXg7IritlOveK6KUz7Ioh4hE7x8buf6+4a0UjkR ekE3pf0wxhY7gCqN0Pe7sJnQNcvRoHmH4b/ZKAmcaxwi/HafzWGT/e38Yekr4nFpJ0iuevgCodVH9 kHaQf+uxr8zlEgWZcqxQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1o9EFO-00CiMz-6q; Wed, 06 Jul 2022 23:15:54 +0000 Received: from gloria.sntech.de ([185.11.138.130]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1o9EFH-00CiK1-2M for linux-riscv@lists.infradead.org; Wed, 06 Jul 2022 23:15:48 +0000 Received: from ip5b412258.dynamic.kabel-deutschland.de ([91.65.34.88] helo=phil.lan) by gloria.sntech.de with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1o9EFF-0002yn-3v; Thu, 07 Jul 2022 01:15:45 +0200 From: Heiko Stuebner To: palmer@dabbelt.com, paul.walmsley@sifive.com Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, wefu@redhat.com, guoren@kernel.org, cmuellner@linux.com, philipp.tomsich@vrull.eu, hch@lst.de, samuel@sholland.org, atishp@atishpatra.org, anup@brainfault.org, mick@ics.forth.gr, robh+dt@kernel.org, krzk+dt@kernel.org, devicetree@vger.kernel.org, drew@beagleboard.org, rdunlap@infradead.org, Heiko Stuebner Subject: [PATCH v7 4/4] riscv: implement cache-management errata for T-Head SoCs Date: Thu, 7 Jul 2022 01:15:36 +0200 Message-Id: <20220706231536.2041855-5-heiko@sntech.de> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220706231536.2041855-1-heiko@sntech.de> References: <20220706231536.2041855-1-heiko@sntech.de> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220706_161547_174990_73CDC708 X-CRM114-Status: GOOD ( 13.72 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org The T-Head C906 and C910 implement a scheme for handling cache operations different from the generic Zicbom extension. Add an errata for it next to the generic dma coherency ops. Reviewed-by: Samuel Holland Tested-by: Samuel Holland Reviewed-by: Guo Ren Signed-off-by: Heiko Stuebner --- arch/riscv/Kconfig.erratas | 11 +++++++ arch/riscv/errata/thead/errata.c | 20 ++++++++++++ arch/riscv/include/asm/errata_list.h | 48 +++++++++++++++++++++++++--- 3 files changed, 74 insertions(+), 5 deletions(-) diff --git a/arch/riscv/Kconfig.erratas b/arch/riscv/Kconfig.erratas index 457ac72c9b36..3223e533fd87 100644 --- a/arch/riscv/Kconfig.erratas +++ b/arch/riscv/Kconfig.erratas @@ -55,4 +55,15 @@ config ERRATA_THEAD_PBMT If you don't know what to do here, say "Y". +config ERRATA_THEAD_CMO + bool "Apply T-Head cache management errata" + depends on ERRATA_THEAD + select RISCV_DMA_NONCOHERENT + default y + help + This will apply the cache management errata to handle the + non-standard handling on non-coherent operations on T-Head SoCs. + + If you don't know what to do here, say "Y". + endmenu diff --git a/arch/riscv/errata/thead/errata.c b/arch/riscv/errata/thead/errata.c index b37b6fedd53b..202c83f677b2 100644 --- a/arch/riscv/errata/thead/errata.c +++ b/arch/riscv/errata/thead/errata.c @@ -27,6 +27,23 @@ static bool errata_probe_pbmt(unsigned int stage, return false; } +static bool errata_probe_cmo(unsigned int stage, + unsigned long arch_id, unsigned long impid) +{ +#ifdef CONFIG_ERRATA_THEAD_CMO + if (arch_id != 0 || impid != 0) + return false; + + if (stage == RISCV_ALTERNATIVES_EARLY_BOOT) + return false; + + riscv_noncoherent_supported(); + return true; +#else + return false; +#endif +} + static u32 thead_errata_probe(unsigned int stage, unsigned long archid, unsigned long impid) { @@ -35,6 +52,9 @@ static u32 thead_errata_probe(unsigned int stage, if (errata_probe_pbmt(stage, archid, impid)) cpu_req_errata |= (1U << ERRATA_THEAD_PBMT); + if (errata_probe_cmo(stage, archid, impid)) + cpu_req_errata |= (1U << ERRATA_THEAD_CMO); + return cpu_req_errata; } diff --git a/arch/riscv/include/asm/errata_list.h b/arch/riscv/include/asm/errata_list.h index 79d89aeeaa6c..19a771085781 100644 --- a/arch/riscv/include/asm/errata_list.h +++ b/arch/riscv/include/asm/errata_list.h @@ -16,7 +16,8 @@ #ifdef CONFIG_ERRATA_THEAD #define ERRATA_THEAD_PBMT 0 -#define ERRATA_THEAD_NUMBER 1 +#define ERRATA_THEAD_CMO 1 +#define ERRATA_THEAD_NUMBER 2 #endif #define CPUFEATURE_SVPBMT 0 @@ -88,17 +89,54 @@ asm volatile(ALTERNATIVE( \ #define ALT_THEAD_PMA(_val) #endif +/* + * dcache.ipa rs1 (invalidate, physical address) + * | 31 - 25 | 24 - 20 | 19 - 15 | 14 - 12 | 11 - 7 | 6 - 0 | + * 0000001 01010 rs1 000 00000 0001011 + * dache.iva rs1 (invalida, virtual address) + * 0000001 00110 rs1 000 00000 0001011 + * + * dcache.cpa rs1 (clean, physical address) + * | 31 - 25 | 24 - 20 | 19 - 15 | 14 - 12 | 11 - 7 | 6 - 0 | + * 0000001 01001 rs1 000 00000 0001011 + * dcache.cva rs1 (clean, virtual address) + * 0000001 00100 rs1 000 00000 0001011 + * + * dcache.cipa rs1 (clean then invalidate, physical address) + * | 31 - 25 | 24 - 20 | 19 - 15 | 14 - 12 | 11 - 7 | 6 - 0 | + * 0000001 01011 rs1 000 00000 0001011 + * dcache.civa rs1 (... virtual address) + * 0000001 00111 rs1 000 00000 0001011 + * + * sync.s (make sure all cache operations finished) + * | 31 - 25 | 24 - 20 | 19 - 15 | 14 - 12 | 11 - 7 | 6 - 0 | + * 0000000 11001 00000 000 00000 0001011 + */ +#define THEAD_inval_A0 ".long 0x0265000b" +#define THEAD_clean_A0 ".long 0x0245000b" +#define THEAD_flush_A0 ".long 0x0275000b" +#define THEAD_SYNC_S ".long 0x0190000b" + #define ALT_CMO_OP(_op, _start, _size, _cachesize) \ -asm volatile(ALTERNATIVE( \ - __nops(5), \ +asm volatile(ALTERNATIVE_2( \ + __nops(6), \ "mv a0, %1\n\t" \ "j 2f\n\t" \ "3:\n\t" \ "cbo." __stringify(_op) " (a0)\n\t" \ "add a0, a0, %0\n\t" \ "2:\n\t" \ - "bltu a0, %2, 3b\n\t", 0, \ - CPUFEATURE_ZICBOM, CONFIG_RISCV_ISA_ZICBOM) \ + "bltu a0, %2, 3b\n\t" \ + "nop", 0, CPUFEATURE_ZICBOM, CONFIG_RISCV_ISA_ZICBOM, \ + "mv a0, %1\n\t" \ + "j 2f\n\t" \ + "3:\n\t" \ + THEAD_##_op##_A0 "\n\t" \ + "add a0, a0, %0\n\t" \ + "2:\n\t" \ + "bltu a0, %2, 3b\n\t" \ + THEAD_SYNC_S, THEAD_VENDOR_ID, \ + ERRATA_THEAD_CMO, CONFIG_ERRATA_THEAD_CMO) \ : : "r"(_cachesize), \ "r"((unsigned long)(_start) & ~((_cachesize) - 1UL)), \ "r"((unsigned long)(_start) + (_size)) \