From patchwork Wed Jun 12 11:22:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daire McNamara X-Patchwork-Id: 13694850 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 48F5DC27C53 for ; Wed, 12 Jun 2024 11:23:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:CC:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=I/mjgfL1teLx9txAHnI5TEyYFfP5me+J92uTr/n2NxY=; b=QMPP9Gg3p+DRJ7 SVwY9uTQFtDmCsfv4TTvK2r8DwBl0fRjpMAyXNEkfxrj3anP5MsTkI5MjsbPU/Bb1eHJ/BWpRxoC1 GkhwdU0aNTIPJ9izi03pH8ZV4jQhnNAxCGy4ptuaoVERbWg86yITe4OL0guy7tdTRf2uc9YnT3Zx2 MfSOCM6otirS8RzKNz+kIbqgiXrfRbsaQ3uTsLvpV3H/CaOMok+TClac9+hohfMXgz7PL/9NSdQdF LgG6qEDjygFV29UAlHsLUdScoKTdc8Xcr7gtKpMJqup25ueHN6wkEfHqZPkc9KnDLdiw3YZiUFjAY a4OR3dbmzshpNIsAAyIQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sHM4j-0000000CQZ0-1aNX; Wed, 12 Jun 2024 11:23:33 +0000 Received: from esa.microchip.iphmx.com ([68.232.153.233]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sHM4c-0000000CQSc-1sFD for linux-riscv@lists.infradead.org; Wed, 12 Jun 2024 11:23:29 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=microchip.com; i=@microchip.com; q=dns/txt; s=mchp; t=1718191406; x=1749727406; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=SxYj/0mHTwUjjUz8Q77KoZhEj4ZQpQBhQ0pJODzMeP0=; b=JisW9tDT+VmgkrWU1l0PI27nG8xF3jT+ulz/x8JagJsR6b0mgm2v/f1u ylL4nRnpNphBh7+4mbIdNK5aioaPs23kbsJ3ONFVIuuOGJ6hIUVfmOCro KsgQ/7h8l3mrNvpDRDFJzEsrV+EQiUuRxZhbAvGfxldbxHkxXcYaP1YPt FmRcmEaea3ZYcRkPz7CB0ndG9dqDU08jVZcp0tvVd/+DId8nk4y+H5w50 offjRQ7x2ThpFOgQM166BVAuRVo1nJV5kxqN/qVAHici4Y0n4X2NHtNfI 2zRoavcxN0hwoKgPwg6NWagsc5nwjM3DyF7fiTeyuvnOqZ3glh0+w/lDK Q==; X-CSE-ConnectionGUID: hzPuRHFvRM+Wy8rrRTyffQ== X-CSE-MsgGUID: KnmZP0C1RsqmhkdEl31Jxw== X-IronPort-AV: E=Sophos;i="6.08,233,1712646000"; d="scan'208";a="29761711" X-Amp-Result: SKIPPED(no attachment in message) Received: from unknown (HELO email.microchip.com) ([170.129.1.10]) by esa1.microchip.iphmx.com with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 12 Jun 2024 04:23:23 -0700 Received: from chn-vm-ex01.mchp-main.com (10.10.85.143) by chn-vm-ex04.mchp-main.com (10.10.85.152) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 12 Jun 2024 04:22:48 -0700 Received: from daire-X570.microchip.com (10.10.85.11) by chn-vm-ex01.mchp-main.com (10.10.85.143) with Microsoft SMTP Server id 15.1.2507.35 via Frontend Transport; Wed, 12 Jun 2024 04:22:46 -0700 From: To: , CC: , , , , , , , , , Subject: [PATCH v3 2/3] PCI: microchip: Fix inbound address translation tables Date: Wed, 12 Jun 2024 12:22:12 +0100 Message-ID: <20240612112213.2734748-3-daire.mcnamara@microchip.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240612112213.2734748-1-daire.mcnamara@microchip.com> References: <20240612112213.2734748-1-daire.mcnamara@microchip.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240612_042326_600916_DD006FA7 X-CRM114-Status: GOOD ( 21.24 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org From: Daire McNamara On Microchip PolarFire SoC the PCIe Root Port can be behind one of three general purpose Fabric Interface Controller (FIC) buses that encapsulates an AXI-S bus. Depending on which FIC(s) the Root Port is connected through to CPU space, and what address translation is done by that FIC, the Root Port driver's inbound address translation may vary. For all current supported designs and all future expected designs, inbound address translation done by a FIC on PolarFire SoC varies depending on whether PolarFire SoC in operating in dma-coherent mode or dma-noncoherent mode. The setup of the outbound address translation tables in the root port driver only needs to handle these two cases. Setup the inbound address translation tables to one of two address translations, depending on whether the rootport is marked as dma-coherent or dma-noncoherent. Fixes: 6f15a9c9f941 ("PCI: microchip: Add Microchip Polarfire PCIe controller driver") Signed-off-by: Daire McNamara --- drivers/pci/controller/pcie-microchip-host.c | 97 +++++++++++++++++++- 1 file changed, 92 insertions(+), 5 deletions(-) diff --git a/drivers/pci/controller/pcie-microchip-host.c b/drivers/pci/controller/pcie-microchip-host.c index 853adce24492..d5021333e2aa 100644 --- a/drivers/pci/controller/pcie-microchip-host.c +++ b/drivers/pci/controller/pcie-microchip-host.c @@ -30,6 +30,9 @@ #define MC_PCIE_BRIDGE_ADDR (MC_PCIE1_BRIDGE_ADDR) #define MC_PCIE_CTRL_ADDR (MC_PCIE1_CTRL_ADDR) +#define MC_MAX_NUM_INBOUND_WINDOWS 8 +#define MPFS_NC_BOUNCE_ADDR 0x80000000 + /* PCIe Bridge Phy Regs */ #define PCIE_PCI_IRQ_DW0 0xa8 #define MSIX_CAP_MASK BIT(31) @@ -105,6 +108,7 @@ #define ATR0_AXI4_SLV0_TRSL_PARAM 0x810u #define PCIE_TX_RX_INTERFACE 0x00000000u #define PCIE_CONFIG_INTERFACE 0x00000001u +#define TRSL_ID_AXI4_MASTER_0 0x00000004u #define ATR_ENTRY_SIZE 32 @@ -931,6 +935,89 @@ static int mc_pcie_init_irq_domains(struct mc_pcie *port) return mc_allocate_msi_domains(port); } +static void mc_pcie_setup_inbound_atr(int window_index, u64 axi_addr, u64 pcie_addr, size_t size) +{ + void __iomem *bridge_base_addr = port->axi_base_addr + MC_PCIE_BRIDGE_ADDR; + u32 table_offset = window_index * ATR_ENTRY_SIZE; + u32 atr_sz; + u32 val; + + atr_sz = ilog2(size) - 1; + atr_sz &= GENMASK(5, 0); + + val = lower_32_bits(pcie_addr) & GENMASK(31, 12); + val |= (atr_sz << ATR_SIZE_SHIFT); + val |= ATR_IMPL_ENABLE; + writel(val, bridge_base_addr + table_offset + ATR0_PCIE_WIN0_SRCADDR_PARAM); + + writel(upper_32_bits(pcie_addr), bridge_base_addr + table_offset + + ATR0_PCIE_WIN0_SRC_ADDR); + + writel(lower_32_bits(axi_addr), bridge_base_addr + table_offset + + ATR0_PCIE_WIN0_TRSL_ADDR_LSB); + writel(upper_32_bits(axi_addr), bridge_base_addr + table_offset + + ATR0_PCIE_WIN0_TRSL_ADDR_UDW); + + writel(TRSL_ID_AXI4_MASTER_0, bridge_base_addr + table_offset + + ATR0_PCIE_WIN0_TRSL_PARAM); +} + +static int mc_pcie_setup_inbound_ranges(struct platform_device *pdev, struct mc_pcie *port) +{ + struct device *dev = &pdev->dev; + struct device_node *dn = dev->of_node; + struct of_range_parser parser; + struct of_range range; + int atr_index = 0; + + /* + * MPFS PCIe root port is 32-bit only, behind a Fabric Interface + * Controller FPGA logic block which contains the AXI-S interface. + * + * From the point of view of the PCIe root port, There are only + * two supported Root Port configurations + * + * Configuration 1: for use with fully coherent designs; supports a + * window from 0x0 (CPU space) to specified PCIe space. + * + * Configuration 2: for use with non-coherent designs; supports two + * 1 Gb wide windows to CPU space; one mapping cpu space 0 to pcie + * space 0x80000000 and mapping cpu space 0x40000000 to pcie + * space 0xc0000000. This cfg needs two windows because of how + * the MSI space is allocated in the AXI-S range on MPFS. + * + * The FIC interface outside the PCIe block *must* complete the inbound + * address translation as per MCHP MPFS FPGA design guidelines. + */ + if (device_property_read_bool(dev, "dma-noncoherent")) { + /* + * Always need same two tables in this case. Need two tables + * due to hardware interactions between address and size. + */ + mc_pcie_setup_inbound_atr(0, 0, MPFS_NC_BOUNCE_ADDR, SZ_1G); + mc_pcie_setup_inbound_atr(1, SZ_1G, MPFS_NC_BOUNCE_ADDR + SZ_1G, SZ_1G); + } else { + /* Find any dma-ranges */ + if (of_pci_dma_range_parser_init(&parser, dn)) { + /* No dma-range property - setup default */ + mc_pcie_setup_inbound_atr(0, 0, 0, SZ_4G); + return 0; + } + + for_each_of_range(&parser, &range) { + if (atr_index >= MC_MAX_NUM_INBOUND_WINDOWS) { + dev_err(dev, "too many inbound ranges; %d available tables\n", + MC_MAX_NUM_INBOUND_WINDOWS); + return -EINVAL; + } + mc_pcie_setup_inbound_atr(atr_index, 0, range.pci_addr, range.size); + atr_index++; + } + } + + return 0; +} + static void mc_pcie_setup_window(void __iomem *bridge_base_addr, u32 index, phys_addr_t axi_addr, phys_addr_t pci_addr, u64 size) @@ -962,11 +1049,6 @@ static void mc_pcie_setup_window(void __iomem *bridge_base_addr, u32 index, val = upper_32_bits(pci_addr); writel(val, bridge_base_addr + (index * ATR_ENTRY_SIZE) + ATR0_AXI4_SLV0_TRSL_ADDR_UDW); - - val = readl(bridge_base_addr + ATR0_PCIE_WIN0_SRCADDR_PARAM); - val |= (ATR0_PCIE_ATR_SIZE << ATR0_PCIE_ATR_SIZE_SHIFT); - writel(val, bridge_base_addr + ATR0_PCIE_WIN0_SRCADDR_PARAM); - writel(0, bridge_base_addr + ATR0_PCIE_WIN0_SRC_ADDR); } static int mc_pcie_setup_windows(struct platform_device *pdev, @@ -1129,6 +1211,11 @@ static int mc_platform_init(struct pci_config_window *cfg) if (ret) return ret; + /* Configure inbound translation tables */ + ret = mc_pcie_setup_inbound_ranges(pdev, port); + if (ret) + return ret; + /* Address translation is up; safe to enable interrupts */ ret = mc_init_interrupts(pdev, port); if (ret)