From patchwork Wed Jun 14 10:47:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Lad, Prabhakar" X-Patchwork-Id: 13279910 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 96F52EB64DB for ; Wed, 14 Jun 2023 10:48:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=852g8akAVZrkP6EouUAdDf9sHJGih+Trvj5dY50/xpY=; b=qay2kHkL2GGIrM WoRgVkhHHKfZOeH812mKkTMJImBLRstnqIXY2kIufww7egv3tKWoM+6VVw0yoTBCPtGp3cyeAe+2I d4V7RMr6933oagpIXUcX8nzFo5a15BOWMbFp3z1pn6uerr23e6TY0HKv3RAXVlKSiacQUk5DmzO3u 9Hjo2/7g4jff0RJbDvz6+ELSoin4e7kVRg8w5VEHnuvYL6fl9xBFxdcweaEFmnjI/M4ErPQ64bKFn xzPLJwAzL/3P/PfIFxjVsemtOiaGCsk/c8V/Wz0LJxXd+Zex1SKM0NSMC13GRufc68cGKEzf8tNuP JBEUgGPK0aFhnIWVSdmw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1q9O32-00BHGL-1z; Wed, 14 Jun 2023 10:48:20 +0000 Received: from mail-wm1-x336.google.com ([2a00:1450:4864:20::336]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1q9O2u-00BH72-2p for linux-riscv@lists.infradead.org; Wed, 14 Jun 2023 10:48:16 +0000 Received: by mail-wm1-x336.google.com with SMTP id 5b1f17b1804b1-3f8d61cb36cso5663985e9.1 for ; Wed, 14 Jun 2023 03:48:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1686739690; x=1689331690; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=pPOutq+8CWbrJU3SgNwlkuco/YMLb70czPF19YdjQ84=; b=PXefaV9AIZoNXXdIp6G3C0693Iy2I99E5I89bqjI4BiJqSUzbYNfRTF0VUGkmNysRd 0MU/0yFQEsCQAeJ/xGOguW0vcbB+b2YR0Cc6UeG4BZ2oPiSJ64offwa7dt6dywKoevZc XGDfce7Rmx7SElHUpc3HqYYzj4aFUR2d9h7rFzTaPwfycNagfEJrz81yk3MdbG+sMz0U K6+luzbvqJ6oA4gtdimBc270xH9yDH4H6KH2o/qK6+ro3y+cXDZrBLgeY3rK3X9Icp5o ewds0QSrTrn0rvd3o6uE0FHH0BPcloqvTeZHDeBE9CdtFnYw2MPUi4b+V+LSlJaFhQD9 +Z7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686739690; x=1689331690; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=pPOutq+8CWbrJU3SgNwlkuco/YMLb70czPF19YdjQ84=; b=MZ2WpePmAL4I2f7sb3Ty5NZoofw0i7ViwctW46TCL+2IWTDxzCqXCTdJTstb5NYr0y 98JKLgbo71H/vxUDqTgLWjkuCMreQEDyvCrK3CNP4G5N0tozF7S+FJhxQ3ruFxTp2ymJ 0C3QD98RKI0ZxY5b9cCJPQSIxodO35sR3yaBz9n1yjWfuwx+So4QJZnI2QWeIdtkTshf tOeANcyWXMYa2JKvWRwjWTXojP9uWqOjwXEiG1XZk9jR74n3eLXkPqzsWoKIQrHJQoRG i1bDmyLITTh2454MhAgFg5Wk4DMkI1ppfjVT4vfEs8wsaghr2IUFaQMPbtsjM7oalDvk RXig== X-Gm-Message-State: AC+VfDxQgvrknhpTRHxnuytDlKwMuqTbnSR+YOzVJdSAFML1FqCUcA99 UJ8hMAxDghtVYDBwculH074= X-Google-Smtp-Source: ACHHUZ65w8FF7k+qOiBaIEK+FhAvhodNYjOScGIbt7jTVSO1vJu9oD00CHZ1GN7HHFjzOGpyssIVwg== X-Received: by 2002:a7b:c047:0:b0:3f8:1a00:7dd1 with SMTP id u7-20020a7bc047000000b003f81a007dd1mr7445833wmc.27.1686739690314; Wed, 14 Jun 2023 03:48:10 -0700 (PDT) Received: from prasmi.home ([2a00:23c8:2501:c701:64d6:8737:b80d:a298]) by smtp.gmail.com with ESMTPSA id y10-20020a1c4b0a000000b003f5ffba9ae1sm17154904wma.24.2023.06.14.03.48.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 14 Jun 2023 03:48:09 -0700 (PDT) From: Prabhakar X-Google-Original-From: Prabhakar To: Arnd Bergmann , Conor Dooley , Geert Uytterhoeven , Guo Ren , Andrew Jones , Paul Walmsley , Palmer Dabbelt , Albert Ou , Samuel Holland , linux-riscv@lists.infradead.org, Christoph Hellwig Cc: Rob Herring , Krzysztof Kozlowski , devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, linux-renesas-soc@vger.kernel.org, Prabhakar , Biju Das , Lad Prabhakar Subject: [PATCH v9 5/6] cache: Add L2 cache management for Andes AX45MP RISC-V core Date: Wed, 14 Jun 2023 11:47:58 +0100 Message-Id: <20230614104759.228372-6-prabhakar.mahadev-lad.rj@bp.renesas.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230614104759.228372-1-prabhakar.mahadev-lad.rj@bp.renesas.com> References: <20230614104759.228372-1-prabhakar.mahadev-lad.rj@bp.renesas.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230614_034812_957880_991F7C23 X-CRM114-Status: GOOD ( 28.04 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org From: Lad Prabhakar I/O Coherence Port (IOCP) provides an AXI interface for connecting external non-caching masters, such as DMA controllers. The accesses from IOCP are coherent with D-Caches and L2 Cache. IOCP is a specification option and is disabled on the Renesas RZ/Five SoC due to this reason IP blocks using DMA will fail. The Andes AX45MP core has a Programmable Physical Memory Attributes (PMA) block that allows dynamic adjustment of memory attributes in the runtime. It contains a configurable amount of PMA entries implemented as CSR registers to control the attributes of memory locations in interest. Below are the memory attributes supported: * Device, Non-bufferable * Device, bufferable * Memory, Non-cacheable, Non-bufferable * Memory, Non-cacheable, Bufferable * Memory, Write-back, No-allocate * Memory, Write-back, Read-allocate * Memory, Write-back, Write-allocate * Memory, Write-back, Read and Write-allocate More info about PMA (section 10.3): Link: http://www.andestech.com/wp-content/uploads/AX45MP-1C-Rev.-5.0.0-Datasheet.pdf As a workaround for SoCs with IOCP disabled CMO needs to be handled by software. Firstly OpenSBI configures the memory region as "Memory, Non-cacheable, Bufferable" and passes this region as a global shared dma pool as a DT node. With DMA_GLOBAL_POOL enabled all DMA allocations happen from this region and synchronization callbacks are implemented to synchronize when doing DMA transactions. Example PMA region passes as a DT node from OpenSBI: reserved-memory { #address-cells = <2>; #size-cells = <2>; ranges; pma_resv0@58000000 { compatible = "shared-dma-pool"; reg = <0x0 0x58000000 0x0 0x08000000>; no-map; linux,dma-default; }; }; Signed-off-by: Lad Prabhakar Reviewed-by: Conor Dooley --- v8 -> v9 * Dropped exporting CMO functions as we no more used ALTERNATIVE_X() macro * Now using the riscv_noncoherent_register_cache_ops() for registering CMO ops * Added RB tag from Conor v7 -> v8 * Dropped function pointer usage * Now exporting the functions for clean/inval/flush * Switched to using early_initcall instead of arch_initcall * Dropped entry for "include/cache" from MAINTAINERS * Dropped dependency of RISCV on AX45MP_L2_CACHE * Returning error in case of cache line mismatch * Renamed clean/inval/flush functions v6 -> v7 * Implemented flush callback * Dropped using riscv_dma_noncoherent_cmo_ops v5 -> v6 * Moved driver to cache folder * Switched to new API for CMO v4 -> v5 * Dropped code for configuring L2 cache * Dropped code for configuring PMA * Updated commit message * Added comments * Changed static branch enable/disable order RFC v3 -> v4 * Made use of runtime patching instead of compile time * Now just exposing single function ax45mp_no_iocp_cmo() for CMO handling * Added a check to make sure cache line size is always 64 bytes * Renamed folder rzf -> rzfive * Improved Kconfig description * Dropped L2 cache configuration * Dropped unnecessary casts * Fixed comments pointed by Geert. --- MAINTAINERS | 7 ++ drivers/Kconfig | 2 + drivers/Makefile | 1 + drivers/cache/Kconfig | 11 ++ drivers/cache/Makefile | 3 + drivers/cache/ax45mp_cache.c | 229 +++++++++++++++++++++++++++++++++++ 6 files changed, 253 insertions(+) create mode 100644 drivers/cache/Kconfig create mode 100644 drivers/cache/Makefile create mode 100644 drivers/cache/ax45mp_cache.c diff --git a/MAINTAINERS b/MAINTAINERS index 55ac73793856..899452038a5b 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -20073,6 +20073,13 @@ S: Supported T: git git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging.git F: drivers/staging/ +STANDALONE CACHE CONTROLLER DRIVERS +M: Conor Dooley +L: linux-riscv@lists.infradead.org +S: Maintained +T: git https://git.kernel.org/pub/scm/linux/kernel/git/conor/linux.git/ +F: drivers/cache + STARFIRE/DURALAN NETWORK DRIVER M: Ion Badulescu S: Odd Fixes diff --git a/drivers/Kconfig b/drivers/Kconfig index 514ae6b24cb2..2ae1b6707c2c 100644 --- a/drivers/Kconfig +++ b/drivers/Kconfig @@ -15,6 +15,8 @@ source "drivers/base/Kconfig" source "drivers/bus/Kconfig" +source "drivers/cache/Kconfig" + source "drivers/connector/Kconfig" source "drivers/firmware/Kconfig" diff --git a/drivers/Makefile b/drivers/Makefile index 7241d80a7b29..23eb201fe18a 100644 --- a/drivers/Makefile +++ b/drivers/Makefile @@ -11,6 +11,7 @@ ifdef building_out_of_srctree MAKEFLAGS += --include-dir=$(srctree) endif +obj-y += cache/ obj-y += irqchip/ obj-y += bus/ diff --git a/drivers/cache/Kconfig b/drivers/cache/Kconfig new file mode 100644 index 000000000000..a57677f908f3 --- /dev/null +++ b/drivers/cache/Kconfig @@ -0,0 +1,11 @@ +# SPDX-License-Identifier: GPL-2.0 +menu "Cache Drivers" + +config AX45MP_L2_CACHE + bool "Andes Technology AX45MP L2 Cache controller" + depends on RISCV_DMA_NONCOHERENT + select RISCV_NONSTANDARD_CACHE_OPS + help + Support for the L2 cache controller on Andes Technology AX45MP platforms. + +endmenu diff --git a/drivers/cache/Makefile b/drivers/cache/Makefile new file mode 100644 index 000000000000..2012e7fb978d --- /dev/null +++ b/drivers/cache/Makefile @@ -0,0 +1,3 @@ +# SPDX-License-Identifier: GPL-2.0 + +obj-$(CONFIG_AX45MP_L2_CACHE) += ax45mp_cache.o diff --git a/drivers/cache/ax45mp_cache.c b/drivers/cache/ax45mp_cache.c new file mode 100644 index 000000000000..df868872aabf --- /dev/null +++ b/drivers/cache/ax45mp_cache.c @@ -0,0 +1,229 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * non-coherent cache functions for Andes AX45MP + * + * Copyright (C) 2023 Renesas Electronics Corp. + */ + +#include +#include +#include +#include +#include + +#include + +/* L2 cache registers */ +#define AX45MP_L2C_REG_CTL_OFFSET 0x8 + +#define AX45MP_L2C_REG_C0_CMD_OFFSET 0x40 +#define AX45MP_L2C_REG_C0_ACC_OFFSET 0x48 +#define AX45MP_L2C_REG_STATUS_OFFSET 0x80 + +/* D-cache operation */ +#define AX45MP_CCTL_L1D_VA_INVAL 0 /* Invalidate an L1 cache entry */ +#define AX45MP_CCTL_L1D_VA_WB 1 /* Write-back an L1 cache entry */ + +/* L2 CCTL status */ +#define AX45MP_CCTL_L2_STATUS_IDLE 0 + +/* L2 CCTL status cores mask */ +#define AX45MP_CCTL_L2_STATUS_C0_MASK 0xf + +/* L2 cache operation */ +#define AX45MP_CCTL_L2_PA_INVAL 0x8 /* Invalidate an L2 cache entry */ +#define AX45MP_CCTL_L2_PA_WB 0x9 /* Write-back an L2 cache entry */ + +#define AX45MP_L2C_REG_PER_CORE_OFFSET 0x10 +#define AX45MP_CCTL_L2_STATUS_PER_CORE_OFFSET 4 + +#define AX45MP_L2C_REG_CN_CMD_OFFSET(n) \ + (AX45MP_L2C_REG_C0_CMD_OFFSET + ((n) * AX45MP_L2C_REG_PER_CORE_OFFSET)) +#define AX45MP_L2C_REG_CN_ACC_OFFSET(n) \ + (AX45MP_L2C_REG_C0_ACC_OFFSET + ((n) * AX45MP_L2C_REG_PER_CORE_OFFSET)) +#define AX45MP_CCTL_L2_STATUS_CN_MASK(n) \ + (AX45MP_CCTL_L2_STATUS_C0_MASK << ((n) * AX45MP_CCTL_L2_STATUS_PER_CORE_OFFSET)) + +#define AX45MP_CCTL_REG_UCCTLBEGINADDR_NUM 0x80b +#define AX45MP_CCTL_REG_UCCTLCOMMAND_NUM 0x80c + +#define AX45MP_CACHE_LINE_SIZE 64 + +struct ax45mp_priv { + void __iomem *l2c_base; + u32 ax45mp_cache_line_size; +}; + +static struct ax45mp_priv ax45mp_priv; + +/* L2 Cache operations */ +static inline uint32_t ax45mp_cpu_l2c_get_cctl_status(void) +{ + return readl(ax45mp_priv.l2c_base + AX45MP_L2C_REG_STATUS_OFFSET); +} + +static void ax45mp_cpu_cache_operation(unsigned long start, unsigned long end, + unsigned long line_size, unsigned int l1_op, + unsigned int l2_op) +{ + void __iomem *base = ax45mp_priv.l2c_base; + int mhartid = smp_processor_id(); + unsigned long pa; + + while (end > start) { + csr_write(AX45MP_CCTL_REG_UCCTLBEGINADDR_NUM, start); + csr_write(AX45MP_CCTL_REG_UCCTLCOMMAND_NUM, l1_op); + + pa = virt_to_phys((void *)start); + writel(pa, base + AX45MP_L2C_REG_CN_ACC_OFFSET(mhartid)); + writel(l2_op, base + AX45MP_L2C_REG_CN_CMD_OFFSET(mhartid)); + while ((ax45mp_cpu_l2c_get_cctl_status() & + AX45MP_CCTL_L2_STATUS_CN_MASK(mhartid)) != + AX45MP_CCTL_L2_STATUS_IDLE) + ; + + start += line_size; + } +} + +/* Write-back L1 and L2 cache entry */ +static inline void ax45mp_cpu_dcache_wb_range(unsigned long start, unsigned long end, + unsigned long line_size) +{ + ax45mp_cpu_cache_operation(start, end, line_size, + AX45MP_CCTL_L1D_VA_WB, + AX45MP_CCTL_L2_PA_WB); +} + +/* Invalidate the L1 and L2 cache entry */ +static inline void ax45mp_cpu_dcache_inval_range(unsigned long start, unsigned long end, + unsigned long line_size) +{ + ax45mp_cpu_cache_operation(start, end, line_size, + AX45MP_CCTL_L1D_VA_INVAL, + AX45MP_CCTL_L2_PA_INVAL); +} + +static void ax45mp_dma_cache_inv(phys_addr_t paddr, unsigned long size) +{ + unsigned long start = (unsigned long)phys_to_virt(paddr); + char cache_buf[2][AX45MP_CACHE_LINE_SIZE]; + unsigned long end = start + size; + unsigned long old_start = start; + unsigned long old_end = end; + unsigned long line_size; + unsigned long flags; + + if (unlikely(start == end)) + return; + + line_size = ax45mp_priv.ax45mp_cache_line_size; + + memset(&cache_buf, 0x0, sizeof(cache_buf)); + start = start & (~(line_size - 1)); + end = ((end + line_size - 1) & (~(line_size - 1))); + + local_irq_save(flags); + if (unlikely(start != old_start)) + memcpy(&cache_buf[0][0], (void *)start, line_size); + + if (unlikely(end != old_end)) + memcpy(&cache_buf[1][0], (void *)(old_end & (~(line_size - 1))), line_size); + + ax45mp_cpu_dcache_inval_range(start, end, line_size); + + if (unlikely(start != old_start)) + memcpy((void *)start, &cache_buf[0][0], (old_start & (line_size - 1))); + + local_irq_restore(flags); +} + +static void ax45mp_dma_cache_wback(phys_addr_t paddr, unsigned long size) +{ + unsigned long start = (unsigned long)phys_to_virt(paddr); + unsigned long end = start + size; + unsigned long line_size; + unsigned long flags; + + line_size = ax45mp_priv.ax45mp_cache_line_size; + start = start & (~(line_size - 1)); + local_irq_save(flags); + ax45mp_cpu_dcache_wb_range(start, end, line_size); + local_irq_restore(flags); +} + +static void ax45mp_dma_cache_wback_inv(phys_addr_t paddr, unsigned long size) +{ + ax45mp_dma_cache_wback(paddr, size); + ax45mp_dma_cache_inv(paddr, size); +} + +static int ax45mp_get_l2_line_size(struct device_node *np) +{ + int ret; + + ret = of_property_read_u32(np, "cache-line-size", &ax45mp_priv.ax45mp_cache_line_size); + if (ret) { + pr_err("Failed to get cache-line-size, defaulting to 64 bytes\n"); + return ret; + } + + if (ax45mp_priv.ax45mp_cache_line_size != AX45MP_CACHE_LINE_SIZE) { + pr_err("Expected cache-line-size to be 64 bytes (found:%u)\n", + ax45mp_priv.ax45mp_cache_line_size); + return -EINVAL; + } + + return 0; +} + +static const struct riscv_cache_ops ax45mp_cmo_ops = { + .clean = &ax45mp_dma_cache_wback, + .inval = &ax45mp_dma_cache_inv, + .flush = &ax45mp_dma_cache_wback_inv, +}; + +static const struct of_device_id ax45mp_cache_ids[] = { + { .compatible = "andestech,ax45mp-cache" }, + { /* sentinel */ } +}; + +static int __init ax45mp_cache_init(void) +{ + struct device_node *np; + struct resource res; + int ret; + + np = of_find_matching_node(NULL, ax45mp_cache_ids); + if (!of_device_is_available(np)) + return -ENODEV; + + ret = of_address_to_resource(np, 0, &res); + if (ret) + return ret; + + /* + * If IOCP is present on the Andes AX45MP core riscv_cbom_block_size + * will be 0 for sure, so we can definitely rely on it. If + * riscv_cbom_block_size = 0 we don't need to handle CMO using SW any + * more so we just return success here and only if its being set we + * continue further in the probe path. + */ + if (!riscv_cbom_block_size) + return 0; + + ax45mp_priv.l2c_base = ioremap(res.start, resource_size(&res)); + if (!ax45mp_priv.l2c_base) + return -ENOMEM; + + ret = ax45mp_get_l2_line_size(np); + if (ret) { + iounmap(ax45mp_priv.l2c_base); + return ret; + } + + riscv_noncoherent_register_cache_ops(&ax45mp_cmo_ops); + + return 0; +} +early_initcall(ax45mp_cache_init);