From patchwork Tue Sep 6 10:21:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Prabhakar X-Patchwork-Id: 12967229 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 00E00C38145 for ; Tue, 6 Sep 2022 10:22:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=IFNVoNAy/N7MZBwEYNq+reOmDOk2pU/B8KSXqJ9tOUA=; b=THiXHs4cw+XaY+ fye05pxyAxE6bOxwsGsUqNmhgfbx9O2R6otiVMyg2BFepRWoi41Ik50iS4q/s9tahWqF409IzfMBB uv+my1BXzkdG56vg7jKd81I2Fh/GPzoY+0jtzPJjy7zuZsstTttkSJj8grjt/oqfe6X+zCV1IuC3z ICh7sip5sPh5hZO7lBjcdDBElGGiEGkvYcrVcpsf7OMjeI8TfRgiYP6ub3q9g4RpJQD5hLcKepnUj yIrXaxUTjwcwY9rlXDJFemmgU0jDFYJ9LhYtnt2ULDAkCfOtqH1zViEqM7w2SgS6I+iPqhThyeYt/ zptkCu+LHO423zUQjP2Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oVVip-00CNiE-4p; Tue, 06 Sep 2022 10:22:23 +0000 Received: from relmlor1.renesas.com ([210.160.252.171] helo=relmlie5.idc.renesas.com) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oVVii-00CNai-Vi for linux-riscv@lists.infradead.org; Tue, 06 Sep 2022 10:22:19 +0000 X-IronPort-AV: E=Sophos;i="5.93,293,1654527600"; d="scan'208";a="131840223" Received: from unknown (HELO relmlir6.idc.renesas.com) ([10.200.68.152]) by relmlie5.idc.renesas.com with ESMTP; 06 Sep 2022 19:22:13 +0900 Received: from localhost.localdomain (unknown [10.226.36.204]) by relmlir6.idc.renesas.com (Postfix) with ESMTP id 0B43B42174C1; Tue, 6 Sep 2022 19:22:09 +0900 (JST) From: Lad Prabhakar To: Paul Walmsley , Palmer Dabbelt , Albert Ou Cc: Atish Patra , Anup Patel , Geert Uytterhoeven , Conor Dooley , linux-riscv@lists.infradead.org, linux-renesas-soc@vger.kernel.org, linux-kernel@vger.kernel.org, Prabhakar , Biju Das , Lad Prabhakar Subject: [RFC PATCH 1/2] riscv: vendors: andes: Add support to configure the PMA regions Date: Tue, 6 Sep 2022 11:21:53 +0100 Message-Id: <20220906102154.32526-2-prabhakar.mahadev-lad.rj@bp.renesas.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20220906102154.32526-1-prabhakar.mahadev-lad.rj@bp.renesas.com> References: <20220906102154.32526-1-prabhakar.mahadev-lad.rj@bp.renesas.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220906_032217_165834_EE8681D4 X-CRM114-Status: GOOD ( 21.19 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org The Andes AX45MP core has a Programmable Physical Memory Attributes (PMA) block that allows dynamic adjustment of memory attributes in the runtime. It contains a configurable amount of PMA entries implemented as CSR registers to control the attributes of memory locations in interest. Below are the memory attributes supported: * Device, Non-bufferable * Device, bufferable * Memory, Non-cacheable, Non-bufferable * Memory, Non-cacheable, Bufferable * Memory, Write-back, No-allocate * Memory, Write-back, Read-allocate * Memory, Write-back, Write-allocate * Memory, Write-back, Read and Write-allocate This patch adds support to configure the memory attributes of the memory regions as passed from the core node. Currently the OpenSBI code implements support for "Memory, Non-cacheable, Non-bufferable" option with SBI_EXT_ANDES_SET_PMA. More info about PMA (section 10.3): http://www.andestech.com/wp-content/uploads/AX45MP-1C-Rev.-5.0.0-Datasheet.pdf Signed-off-by: Lad Prabhakar --- Note: the current implementation only supports "Memory, Non-cacheable, Bufferable" option. --- arch/riscv/Kbuild | 2 + arch/riscv/include/asm/sbi.h | 1 + arch/riscv/vendors/Makefile | 3 + arch/riscv/vendors/andes/Makefile | 3 + arch/riscv/vendors/andes/ax45mp.c | 93 ++++++++++++++++++++++++++ arch/riscv/vendors/andes/include/sbi.h | 27 ++++++++ 6 files changed, 129 insertions(+) create mode 100644 arch/riscv/vendors/Makefile create mode 100644 arch/riscv/vendors/andes/Makefile create mode 100644 arch/riscv/vendors/andes/ax45mp.c create mode 100644 arch/riscv/vendors/andes/include/sbi.h diff --git a/arch/riscv/Kbuild b/arch/riscv/Kbuild index afa83e307a2e..d6821f660fc3 100644 --- a/arch/riscv/Kbuild +++ b/arch/riscv/Kbuild @@ -5,6 +5,8 @@ obj-$(CONFIG_BUILTIN_DTB) += boot/dts/ obj-y += errata/ obj-$(CONFIG_KVM) += kvm/ +obj-y += vendors/ + obj-$(CONFIG_ARCH_HAS_KEXEC_PURGATORY) += purgatory/ # for cleaning diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h index 2a0ef738695e..10a7c855d125 100644 --- a/arch/riscv/include/asm/sbi.h +++ b/arch/riscv/include/asm/sbi.h @@ -37,6 +37,7 @@ enum sbi_ext_id { /* Vendor extensions must lie within this range */ SBI_EXT_VENDOR_START = 0x09000000, + SBI_EXT_ANDES = 0x0900031E, SBI_EXT_VENDOR_END = 0x09FFFFFF, }; diff --git a/arch/riscv/vendors/Makefile b/arch/riscv/vendors/Makefile new file mode 100644 index 000000000000..0a5a5541d2a3 --- /dev/null +++ b/arch/riscv/vendors/Makefile @@ -0,0 +1,3 @@ +# SPDX-License-Identifier: GPL-2.0 + +obj-$(CONFIG_ARCH_R9A07G043) += andes/ diff --git a/arch/riscv/vendors/andes/Makefile b/arch/riscv/vendors/andes/Makefile new file mode 100644 index 000000000000..60fa8226c4a3 --- /dev/null +++ b/arch/riscv/vendors/andes/Makefile @@ -0,0 +1,3 @@ +# SPDX-License-Identifier: GPL-2.0 + +obj-$(CONFIG_ARCH_R9A07G043) += ax45mp.o diff --git a/arch/riscv/vendors/andes/ax45mp.c b/arch/riscv/vendors/andes/ax45mp.c new file mode 100644 index 000000000000..931cba754f41 --- /dev/null +++ b/arch/riscv/vendors/andes/ax45mp.c @@ -0,0 +1,93 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * AX45MP setup + * - PMA + * + * Copyright (C) 2022 Renesas Electronics Corp. + * + */ + +#include +#include + +#include + +#include "include/sbi.h" + +#define ANDES_AX45MP_MAX_PMA_REGIONS 16 + +struct pma_arg_t { + phys_addr_t offset; + unsigned long vaddr; + size_t size; + size_t entry_id; +}; + +static long sbi_set_pma(void *arg) +{ + phys_addr_t offset = ((struct pma_arg_t *)arg)->offset; + unsigned long vaddr = ((struct pma_arg_t *)arg)->vaddr; + size_t entry_id = ((struct pma_arg_t *)arg)->entry_id; + size_t size = ((struct pma_arg_t *)arg)->size; + struct sbiret ret; + + ret = sbi_ecall(SBI_EXT_ANDES, SBI_EXT_ANDES_SET_PMA, offset, vaddr, size, entry_id, 0, 0); + + return ret.value; +} + +static unsigned long cpu_nocache_area_set(unsigned long start, + unsigned long size, + unsigned long entry_id) +{ + struct pma_arg_t pma_arg; + unsigned long ret = 0; + + pma_arg.offset = start; + pma_arg.size = size; + pma_arg.vaddr = start + size; + pma_arg.entry_id = entry_id; + ret = sbi_set_pma(&pma_arg); + + return ret; +} + +static void ax45mp_configure_pma_regions(struct device_node *np, int count) +{ + u64 start, size; + unsigned int i; + + for (i = 0 ; i < count ; i++) { + of_property_read_u64_index(np, "pma-regions", (i << 1), &start); + of_property_read_u64_index(np, "pma-regions", (i << 1) + 1, &size); + cpu_nocache_area_set(start, size, i); + } +} + +static const struct of_device_id ax45mp_ids[] = { + { .compatible = "andestech,ax45mp" }, + { /* sentinel */ } +}; + +static int __init ax45mp_init(void) +{ + struct device_node *np; + int count; + + np = of_find_matching_node(NULL, ax45mp_ids); + if (!np) + return -ENODEV; + + count = of_property_count_elems_of_size(np, "pma-regions", + sizeof(u32) * 4); + if (count < 0) + return 0; + + if (count > ANDES_AX45MP_MAX_PMA_REGIONS) + return -EINVAL; + + ax45mp_configure_pma_regions(np, count); + + return 0; +} +arch_initcall(ax45mp_init); diff --git a/arch/riscv/vendors/andes/include/sbi.h b/arch/riscv/vendors/andes/include/sbi.h new file mode 100644 index 000000000000..6dcd215bb5f8 --- /dev/null +++ b/arch/riscv/vendors/andes/include/sbi.h @@ -0,0 +1,27 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ + +#ifndef __RISCV_ANDES_SBI_H +#define __RISCV_ANDES_SBI_H + +enum sbi_ext_andes_fid { + SBI_EXT_ANDES_GET_MCACHE_CTL_STATUS = 0, + SBI_EXT_ANDES_GET_MMISC_CTL_STATUS, + SBI_EXT_ANDES_SET_MCACHE_CTL, + SBI_EXT_ANDES_SET_MMISC_CTL, + SBI_EXT_ANDES_ICACHE_OP, + SBI_EXT_ANDES_DCACHE_OP, + SBI_EXT_ANDES_L1CACHE_I_PREFETCH, + SBI_EXT_ANDES_L1CACHE_D_PREFETCH, + SBI_EXT_ANDES_NON_BLOCKING_LOAD_STORE, + SBI_EXT_ANDES_WRITE_AROUND, + SBI_EXT_ANDES_SET_PMA, + SBI_EXT_ANDES_FREE_PMA, + SBI_EXT_ANDES_PROBE_PMA, + SBI_EXT_ANDES_DCACHE_WBINVAL_ALL, + SBI_EXT_ANDES_GET_MICM_CTL_STATUS, + SBI_EXT_ANDES_GET_MDCM_CTL_STATUS, + SBI_EXT_ANDES_GET_MMSC_CTL_STATUS, + SBI_EXT_ANDES_GET_MISA_CTL_STATUS, +}; + +#endif From patchwork Tue Sep 6 10:21:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Prabhakar X-Patchwork-Id: 12967230 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C1FD7ECAAA1 for ; Tue, 6 Sep 2022 10:22:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=z5X4VImdGIu8gP47NpR6G8jOgqY2EzY1VtaBlIz76+Q=; b=k18JSSYzEnULdw LPnvMKZyZZ3soh/IuDYt+zeIJqKgF3epYugXgXzfvHCQdZ81g6pIdzdFRHIP4QtIBeN20M+cxTD4v Jx+Zf8G8H4lsR1I5PwASSoaYxJNbmjy3zKkLGFfBkYrU7lsUi1Y++XCrsDseJzkaS2lTt76egjBCP /hE1ZgXQlVkDRqh+GAFNAfCzCuvA45ZWrm7qcjtreRGR0cKr4MA0y1QiRIn00CfnsVvnzqkrRexUM ri4ytIK0oMgprwehzYhhssy29HNgHNxF79BtwFUWd/tCwMKQZgHkdmPtu5qWVbjbb0KkgB0PlTJki pfW9Bujr7PB5VKyAnOQQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oVViw-00CNmH-TG; Tue, 06 Sep 2022 10:22:31 +0000 Received: from relmlor2.renesas.com ([210.160.252.172] helo=relmlie6.idc.renesas.com) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oVVik-00CNUL-D2 for linux-riscv@lists.infradead.org; Tue, 06 Sep 2022 10:22:21 +0000 X-IronPort-AV: E=Sophos;i="5.93,293,1654527600"; d="scan'208";a="133888055" Received: from unknown (HELO relmlir6.idc.renesas.com) ([10.200.68.152]) by relmlie6.idc.renesas.com with ESMTP; 06 Sep 2022 19:22:18 +0900 Received: from localhost.localdomain (unknown [10.226.36.204]) by relmlir6.idc.renesas.com (Postfix) with ESMTP id 5344742174CA; Tue, 6 Sep 2022 19:22:14 +0900 (JST) From: Lad Prabhakar To: Paul Walmsley , Palmer Dabbelt , Albert Ou Cc: Atish Patra , Anup Patel , Geert Uytterhoeven , Conor Dooley , linux-riscv@lists.infradead.org, linux-renesas-soc@vger.kernel.org, linux-kernel@vger.kernel.org, Prabhakar , Biju Das , Lad Prabhakar Subject: [RFC PATCH 2/2] riscv: vendors: andes: Add support for non-cohernet dma Date: Tue, 6 Sep 2022 11:21:54 +0100 Message-Id: <20220906102154.32526-3-prabhakar.mahadev-lad.rj@bp.renesas.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20220906102154.32526-1-prabhakar.mahadev-lad.rj@bp.renesas.com> References: <20220906102154.32526-1-prabhakar.mahadev-lad.rj@bp.renesas.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220906_032218_715990_175F1F73 X-CRM114-Status: GOOD ( 24.14 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org On the AX45MP core, cache coherency is a specification option so it may not be supported. In this case DMA will fail. As a workaround, firstly we allocate a global dma coherent pool from which DMA allocations are taken and marked as non-cacheable in the PMA region as specified in the device tree. Synchronization callbacks are implemented to synchronize when doing DMA transactions. This feature is based on the work posted [0] by Vincent Chen for the Andes AndeStart RISC-V CPU. [0] https://lore.kernel.org/lkml/1540982130-28248-1-git-send-email-vincentc@andestech.com/ Signed-off-by: Lad Prabhakar --- arch/riscv/vendors/andes/Makefile | 1 + arch/riscv/vendors/andes/ax45mp_cache.c | 296 ++++++++++++++++++ arch/riscv/vendors/andes/ax45mp_nocache_dma.c | 65 ++++ arch/riscv/vendors/andes/include/proc.h | 9 + 4 files changed, 371 insertions(+) create mode 100644 arch/riscv/vendors/andes/ax45mp_cache.c create mode 100644 arch/riscv/vendors/andes/ax45mp_nocache_dma.c create mode 100644 arch/riscv/vendors/andes/include/proc.h diff --git a/arch/riscv/vendors/andes/Makefile b/arch/riscv/vendors/andes/Makefile index 60fa8226c4a3..f9ec92b92455 100644 --- a/arch/riscv/vendors/andes/Makefile +++ b/arch/riscv/vendors/andes/Makefile @@ -1,3 +1,4 @@ # SPDX-License-Identifier: GPL-2.0 obj-$(CONFIG_ARCH_R9A07G043) += ax45mp.o +obj-$(CONFIG_ARCH_R9A07G043) += ax45mp_cache.o ax45mp_nocache_dma.o diff --git a/arch/riscv/vendors/andes/ax45mp_cache.c b/arch/riscv/vendors/andes/ax45mp_cache.c new file mode 100644 index 000000000000..6941821e7e4a --- /dev/null +++ b/arch/riscv/vendors/andes/ax45mp_cache.c @@ -0,0 +1,296 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * non-coherent cache functions for AX45MP + * + * Copyright (C) 2022 Renesas Electronics Corp. + * + */ + +#include +#include + +#include + +#include "include/sbi.h" + +/* D-cache operation */ +#define CCTL_L1D_VA_INVAL 0 +#define CCTL_L1D_VA_WB 1 + +/* L2 cache */ +#define L2_CACHE_CTL_CEN_MASK 1 + +/* L2 cache registers */ +#define L2C_REG_CTL_OFFSET 0x8 +#define L2C_REG_C0_CMD_OFFSET 0x40 +#define L2C_REG_C0_ACC_OFFSET 0x48 +#define L2C_REG_STATUS_OFFSET 0x80 + +/* L2 CCTL status */ +#define CCTL_L2_STATUS_IDLE 0 + +/* L2 CCTL status cores mask */ +#define CCTL_L2_STATUS_C0_MASK 0xf + +/* L2 cache operation */ +#define CCTL_L2_PA_INVAL 0x8 +#define CCTL_L2_PA_WB 0x9 + +#define L2C_HPM_PER_CORE_OFFSET 0x8 +#define L2C_REG_PER_CORE_OFFSET 0x10 +#define CCTL_L2_STATUS_PER_CORE_OFFSET 4 + +#define L2C_REG_CN_CMD_OFFSET(n) \ + (L2C_REG_C0_CMD_OFFSET + ((n) * L2C_REG_PER_CORE_OFFSET)) +#define L2C_REG_CN_ACC_OFFSET(n) \ + (L2C_REG_C0_ACC_OFFSET + ((n) * L2C_REG_PER_CORE_OFFSET)) +#define CCTL_L2_STATUS_CN_MASK(n) \ + (CCTL_L2_STATUS_C0_MASK << ((n) * CCTL_L2_STATUS_PER_CORE_OFFSET)) + +#define MICM_CFG_ISZ_OFFSET 6 +#define MICM_CFG_ISZ_MASK (0x7 << MICM_CFG_ISZ_OFFSET) + +#define MDCM_CFG_DSZ_OFFSET 6 +#define MDCM_CFG_DSZ_MASK (0x7 << MDCM_CFG_DSZ_OFFSET) + +#define CCTL_REG_UCCTLBEGINADDR_NUM 0x80b +#define CCTL_REG_UCCTLCOMMAND_NUM 0x80c + +#define MCACHE_CTL_CCTL_SUEN_OFFSET 8 +#define MMSC_CFG_CCTLCSR_OFFSET 16 +#define MISA_20_OFFSET 20 + +#define MCACHE_CTL_CCTL_SUEN_MASK (0x1 << MCACHE_CTL_CCTL_SUEN_OFFSET) +#define MMSC_CFG_CCTLCSR_MASK (0x1 << MMSC_CFG_CCTLCSR_OFFSET) +#define MISA_20_MASK (0x1 << MISA_20_OFFSET) + +#define MAX_CACHE_LINE_SIZE 256 + +struct ax45mp_cache_info { + bool init_done; + int dcache_line_size; +}; + +static DEFINE_PER_CPU(struct ax45mp_cache_info, cpu_cache_info) = { + .init_done = 0, + .dcache_line_size = SZ_64, +}; + +static void __iomem *l2c_base; + +static uint32_t cpu_get_mcache_ctl_status(void) +{ + struct sbiret ret; + + ret = sbi_ecall(SBI_EXT_ANDES, SBI_EXT_ANDES_GET_MCACHE_CTL_STATUS, 0, 0, 0, 0, 0, 0); + return ret.value; +} + +static uint32_t cpu_get_micm_cfg_status(void) +{ + struct sbiret ret; + + ret = sbi_ecall(SBI_EXT_ANDES, SBI_EXT_ANDES_GET_MICM_CTL_STATUS, 0, 0, 0, 0, 0, 0); + return ret.value; +} + +static uint32_t cpu_get_mdcm_cfg_status(void) +{ + struct sbiret ret; + + ret = sbi_ecall(SBI_EXT_ANDES, SBI_EXT_ANDES_GET_MDCM_CTL_STATUS, 0, 0, 0, 0, 0, 0); + return ret.value; +} + +static uint32_t cpu_get_mmsc_cfg_status(void) +{ + struct sbiret ret; + + ret = sbi_ecall(SBI_EXT_ANDES, SBI_EXT_ANDES_GET_MMSC_CTL_STATUS, 0, 0, 0, 0, 0, 0); + return ret.value; +} + +static uint32_t cpu_get_misa_cfg_status(void) +{ + struct sbiret ret; + + ret = sbi_ecall(SBI_EXT_ANDES, SBI_EXT_ANDES_GET_MISA_CTL_STATUS, 0, 0, 0, 0, 0, 0); + return ret.value; +} + +static void fill_cpu_cache_info(struct ax45mp_cache_info *cpu_ci) +{ + struct cpu_cacheinfo *this_cpu_ci = + get_cpu_cacheinfo(smp_processor_id()); + struct cacheinfo *this_leaf = this_cpu_ci->info_list; + unsigned int i; + + for (i = 0; i < this_cpu_ci->num_leaves ; i++, this_leaf++) { + if (this_leaf->type == CACHE_TYPE_DATA) + cpu_ci->dcache_line_size = this_leaf->coherency_line_size; + } + + cpu_ci->init_done = true; +} + +static inline int get_cache_line_size(void) +{ + struct ax45mp_cache_info *cpu_ci = + &per_cpu(cpu_cache_info, smp_processor_id()); + + if (unlikely(!cpu_ci->init_done)) + fill_cpu_cache_info(cpu_ci); + return cpu_ci->dcache_line_size; +} + +static uint32_t cpu_l2c_get_cctl_status(void) +{ + return readl((void *)(l2c_base + L2C_REG_STATUS_OFFSET)); +} + +static uint32_t cpu_l2c_ctl_status(void) +{ + return readl((void *)(l2c_base + L2C_REG_CTL_OFFSET)); +} + +static bool cpu_cache_controlable(void) +{ + return (((cpu_get_micm_cfg_status() & MICM_CFG_ISZ_MASK) || + (cpu_get_mdcm_cfg_status() & MDCM_CFG_DSZ_MASK)) && + (cpu_get_misa_cfg_status() & MISA_20_MASK) && + (cpu_get_mmsc_cfg_status() & MMSC_CFG_CCTLCSR_MASK) && + (cpu_get_mcache_ctl_status() & MCACHE_CTL_CCTL_SUEN_MASK)); +} + +static void cpu_dcache_wb_range(unsigned long start, + unsigned long end, + int line_size) +{ + bool ucctl_ok = false; + unsigned long pa; + int mhartid = 0; +#ifdef CONFIG_SMP + mhartid = smp_processor_id(); +#endif + + ucctl_ok = cpu_cache_controlable(); + + while (end > start) { + if (ucctl_ok) { + csr_write(CCTL_REG_UCCTLBEGINADDR_NUM, start); + csr_write(CCTL_REG_UCCTLCOMMAND_NUM, CCTL_L1D_VA_WB); + } + + if (l2c_base && (cpu_l2c_ctl_status() & L2_CACHE_CTL_CEN_MASK)) { + pa = virt_to_phys((void *)start); + writel(pa, (void *)(l2c_base + L2C_REG_CN_ACC_OFFSET(mhartid))); + writel(CCTL_L2_PA_WB, (void *)(l2c_base + L2C_REG_CN_CMD_OFFSET(mhartid))); + while ((cpu_l2c_get_cctl_status() & + CCTL_L2_STATUS_CN_MASK(mhartid)) != CCTL_L2_STATUS_IDLE) + ; + } + + start += line_size; + } +} + +static void cpu_dcache_inval_range(unsigned long start, + unsigned long end, + int line_size) +{ + bool ucctl_ok = false; + unsigned long pa; + int mhartid = 0; +#ifdef CONFIG_SMP + mhartid = smp_processor_id(); +#endif + + ucctl_ok = cpu_cache_controlable(); + + while (end > start) { + if (ucctl_ok) { + csr_write(CCTL_REG_UCCTLBEGINADDR_NUM, start); + csr_write(CCTL_REG_UCCTLCOMMAND_NUM, CCTL_L1D_VA_INVAL); + } + + if (l2c_base && (cpu_l2c_ctl_status() & L2_CACHE_CTL_CEN_MASK)) { + pa = virt_to_phys((void *)start); + writel(pa, (void *)(l2c_base + L2C_REG_CN_ACC_OFFSET(mhartid))); + writel(CCTL_L2_PA_INVAL, + (void *)(l2c_base + L2C_REG_CN_CMD_OFFSET(mhartid))); + while ((cpu_l2c_get_cctl_status() & + CCTL_L2_STATUS_CN_MASK(mhartid)) != CCTL_L2_STATUS_IDLE) + ; + } + + start += line_size; + } +} + +void cpu_dma_inval_range(unsigned long start, unsigned long end) +{ + unsigned long line_size = get_cache_line_size(); + char cache_buf[2][MAX_CACHE_LINE_SIZE] = { 0 }; + unsigned long old_start = start; + unsigned long old_end = end; + unsigned long flags; + + if (unlikely(start == end)) + return; + + start = start & (~(line_size - 1)); + end = ((end + line_size - 1) & (~(line_size - 1))); + + local_irq_save(flags); + if (unlikely(start != old_start)) + memcpy(&cache_buf[0][0], (void *)start, line_size); + + if (unlikely(end != old_end)) + memcpy(&cache_buf[1][0], (void *)(old_end & (~(line_size - 1))), line_size); + + cpu_dcache_inval_range(start, end, line_size); + + if (unlikely(start != old_start)) + memcpy((void *)start, &cache_buf[0][0], (old_start & (line_size - 1))); + + if (unlikely(end != old_end)) + memcpy((void *)(old_end + 1), + &cache_buf[1][(old_end & (line_size - 1)) + 1], + end - old_end - 1); + + local_irq_restore(flags); +} +EXPORT_SYMBOL(cpu_dma_inval_range); + +void cpu_dma_wb_range(unsigned long start, unsigned long end) +{ + unsigned long flags; + unsigned long line_size = get_cache_line_size(); + + local_irq_save(flags); + start = start & (~(line_size - 1)); + cpu_dcache_wb_range(start, end, line_size); + local_irq_restore(flags); +} +EXPORT_SYMBOL(cpu_dma_wb_range); + +static const struct of_device_id ax45mp_cache_ids[] = { + { .compatible = "andestech,ax45mp-cache" }, + { /* sentinel */ } +}; + +static int __init ax45mp_cache_init(void) +{ + struct device_node *np; + + np = of_find_matching_node(NULL, ax45mp_cache_ids); + if (!np) + return -ENODEV; + + l2c_base = of_iomap(np, 0); + if (!l2c_base) + return -ENOMEM; + + return 0; +} +arch_initcall(ax45mp_cache_init); diff --git a/arch/riscv/vendors/andes/ax45mp_nocache_dma.c b/arch/riscv/vendors/andes/ax45mp_nocache_dma.c new file mode 100644 index 000000000000..263012f89d0b --- /dev/null +++ b/arch/riscv/vendors/andes/ax45mp_nocache_dma.c @@ -0,0 +1,65 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * DMA callbacks implemented for the non-coherent cache + * on AX45MP + * + * Copyright (C) 2022 Renesas Electronics Corp. + * + */ + +#include +#include + +#include "include/proc.h" + +static inline void cache_op(phys_addr_t paddr, size_t size, + void (*fn)(unsigned long start, unsigned long end)) +{ + unsigned long start; + + start = (unsigned long)phys_to_virt(paddr); + fn(start, start + size); +} + +void arch_sync_dma_for_device(phys_addr_t paddr, + size_t size, enum dma_data_direction dir) +{ + switch (dir) { + case DMA_FROM_DEVICE: + cache_op(paddr, size, cpu_dma_inval_range); + break; + case DMA_TO_DEVICE: + case DMA_BIDIRECTIONAL: + cache_op(paddr, size, cpu_dma_wb_range); + break; + default: + BUG(); + } +} + +void arch_sync_dma_for_cpu(phys_addr_t paddr, + size_t size, enum dma_data_direction dir) +{ + switch (dir) { + case DMA_TO_DEVICE: + break; + case DMA_FROM_DEVICE: + case DMA_BIDIRECTIONAL: + cache_op(paddr, size, cpu_dma_inval_range); + break; + default: + BUG(); + } +} + +void *arch_dma_alloc(struct device *dev, size_t size, dma_addr_t *handle, + gfp_t gfp, unsigned long attrs) +{ + return dma_alloc_from_global_coherent(dev, size, handle); +} + +void arch_dma_free(struct device *dev, size_t size, void *vaddr, + dma_addr_t handle, unsigned long attrs) +{ + dma_release_from_global_coherent(0, vaddr); +} diff --git a/arch/riscv/vendors/andes/include/proc.h b/arch/riscv/vendors/andes/include/proc.h new file mode 100644 index 000000000000..ace9bcfccf6b --- /dev/null +++ b/arch/riscv/vendors/andes/include/proc.h @@ -0,0 +1,9 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ + +#ifndef __RISCV_ANDES_PROC_H +#define __RISCV_ANDES_PROC_H + +void cpu_dma_inval_range(unsigned long start, unsigned long end); +void cpu_dma_wb_range(unsigned long start, unsigned long end); + +#endif