From patchwork Fri Oct 23 19:56:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vidya Sagar X-Patchwork-Id: 11854495 X-Patchwork-Delegate: lorenzo.pieralisi@arm.com Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0FAAA92C for ; Fri, 23 Oct 2020 19:57:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D95CB22254 for ; Fri, 23 Oct 2020 19:57:14 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=nvidia.com header.i=@nvidia.com header.b="MUPvimuP" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1750069AbgJWT5L (ORCPT ); Fri, 23 Oct 2020 15:57:11 -0400 Received: from hqnvemgate26.nvidia.com ([216.228.121.65]:17013 "EHLO hqnvemgate26.nvidia.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755224AbgJWT5L (ORCPT ); Fri, 23 Oct 2020 15:57:11 -0400 Received: from hqmail.nvidia.com (Not Verified[216.228.121.13]) by hqnvemgate26.nvidia.com (using TLS: TLSv1.2, AES256-SHA) id ; Fri, 23 Oct 2020 12:56:50 -0700 Received: from HQMAIL101.nvidia.com (172.20.187.10) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Fri, 23 Oct 2020 19:57:08 +0000 Received: from vidyas-desktop.nvidia.com (10.124.1.5) by mail.nvidia.com (172.20.187.10) with Microsoft SMTP Server id 15.0.1473.3 via Frontend Transport; Fri, 23 Oct 2020 19:57:04 +0000 From: Vidya Sagar To: , , , , , , , CC: , , , , , Subject: [PATCH 1/3] PCI: of: Warn if non-prefetchable memory aperture size is > 32-bit Date: Sat, 24 Oct 2020 01:26:53 +0530 Message-ID: <20201023195655.11242-2-vidyas@nvidia.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20201023195655.11242-1-vidyas@nvidia.com> References: <20201023195655.11242-1-vidyas@nvidia.com> X-NVConfidentiality: public MIME-Version: 1.0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nvidia.com; s=n1; t=1603483010; bh=dZDk41IQVmye1tm/HGRwkE9LdTLqCSACL+m5qHhAk84=; h=From:To:CC:Subject:Date:Message-ID:X-Mailer:In-Reply-To: References:X-NVConfidentiality:MIME-Version:Content-Type; b=MUPvimuPnqr3XPUjB4q6/Dj7ubZaHhEo0+gV52RMHfZ4mYxdou9Zqu5NgiqjRHMuS FXaHoyWLhYg5ehKsTCM1IPfbDoPskyW8it7uf78Rr6m7b8yJkWHAeDB358HKOD+I4/ 5mmb6v7pHFH7GPFUlMuZxx6LjIeZ5ztneBe9u6sfYgJlTSqvi9stNYUW++rnw2Xula C9FKqzGpWq2PRyPziAshq+DYdbP26/jtJ1uzkTfd1IzAfitSPjPyVA+AelpnvW0YTA Wq61YOuYyP+K7xzDTXXsaf6X0Qz7GUqUXMzffgJNHaq05ZxgJ32RhL+QIiy3w+lanK ZWgt4uOWwtKDQ== Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org As per PCIe spec r5.0, sec 7.5.1.3.8 only 32-bit BAR registers are defined for non-prefetchable memory and hence a warning should be reported when the size of them go beyond 32-bits. Signed-off-by: Vidya Sagar Reviewed-by: Rob Herring --- drivers/pci/of.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/drivers/pci/of.c b/drivers/pci/of.c index ac24cd5439a9..5ea472ae22ac 100644 --- a/drivers/pci/of.c +++ b/drivers/pci/of.c @@ -556,6 +556,11 @@ static int pci_parse_request_of_pci_ranges(struct device *dev, break; case IORESOURCE_MEM: res_valid |= !(res->flags & IORESOURCE_PREFETCH); + + if (!(res->flags & IORESOURCE_PREFETCH)) + if (upper_32_bits(resource_size(res))) + dev_warn(dev, "Memory resource size exceeds max for 32 bits\n"); + break; } } From patchwork Fri Oct 23 19:56:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vidya Sagar X-Patchwork-Id: 11854497 X-Patchwork-Delegate: lorenzo.pieralisi@arm.com Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EBB1092C for ; Fri, 23 Oct 2020 19:57:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C33B522255 for ; Fri, 23 Oct 2020 19:57:16 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=nvidia.com header.i=@nvidia.com header.b="S6afGSdJ" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755238AbgJWT5Q (ORCPT ); Fri, 23 Oct 2020 15:57:16 -0400 Received: from hqnvemgate26.nvidia.com ([216.228.121.65]:17019 "EHLO hqnvemgate26.nvidia.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755237AbgJWT5P (ORCPT ); Fri, 23 Oct 2020 15:57:15 -0400 Received: from hqmail.nvidia.com (Not Verified[216.228.121.13]) by hqnvemgate26.nvidia.com (using TLS: TLSv1.2, AES256-SHA) id ; Fri, 23 Oct 2020 12:56:55 -0700 Received: from HQMAIL101.nvidia.com (172.20.187.10) by HQMAIL101.nvidia.com (172.20.187.10) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Fri, 23 Oct 2020 19:57:14 +0000 Received: from vidyas-desktop.nvidia.com (10.124.1.5) by mail.nvidia.com (172.20.187.10) with Microsoft SMTP Server id 15.0.1473.3 via Frontend Transport; Fri, 23 Oct 2020 19:57:09 +0000 From: Vidya Sagar To: , , , , , , , CC: , , , , , Subject: [PATCH 2/3] PCI: dwc: Add support to program ATU for >4GB memory aperture sizes Date: Sat, 24 Oct 2020 01:26:54 +0530 Message-ID: <20201023195655.11242-3-vidyas@nvidia.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20201023195655.11242-1-vidyas@nvidia.com> References: <20201023195655.11242-1-vidyas@nvidia.com> X-NVConfidentiality: public MIME-Version: 1.0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nvidia.com; s=n1; t=1603483015; bh=Lcu11GtGa94b8F15yYcftGZjf+99kinzBPZEJ0Hk4Nc=; h=From:To:CC:Subject:Date:Message-ID:X-Mailer:In-Reply-To: References:X-NVConfidentiality:MIME-Version:Content-Type; b=S6afGSdJjwBWM4IYoOg6hqyhHO1q+IAH0Qmj+4FXMNQhCp87Rf8E5jKNYPautjK82 J3fO7M6lYltynXhUkhQ5V9sl9TfStISjSnaS6MEd0fH5yW1xxxJrzKsNsibfDUozl8 gKUxAlVHarA0u8V8UUqDgRtFL13w10JM8YzhvzKL6MxGQhfU5Oppplu4Pwapv4BrmN hxX0y8sxa8Nz1tEC76d4y9W03zsM38+ReY25ijlE4bD7Az7AxW0g+VEh4nLdSFmCWp lfaJvC6lIslRJ5RIM2/DAszdTtB2J6OvcgLxiT1O4v5k1a6OLUpCyxAnAgov7tCETn ExpAMNY0s30Sg== Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Add support to program the ATU to enable translations for >4GB sizes of the prefetchable memory apertures. Signed-off-by: Vidya Sagar Reviewed-by: Rob Herring --- drivers/pci/controller/dwc/pcie-designware.c | 12 +++++++----- drivers/pci/controller/dwc/pcie-designware.h | 3 ++- 2 files changed, 9 insertions(+), 6 deletions(-) diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c index c2dea8fc97c8..b5e438b70cd5 100644 --- a/drivers/pci/controller/dwc/pcie-designware.c +++ b/drivers/pci/controller/dwc/pcie-designware.c @@ -228,7 +228,7 @@ static void dw_pcie_writel_ob_unroll(struct dw_pcie *pci, u32 index, u32 reg, static void dw_pcie_prog_outbound_atu_unroll(struct dw_pcie *pci, u8 func_no, int index, int type, u64 cpu_addr, u64 pci_addr, - u32 size) + u64 size) { u32 retries, val; u64 limit_addr = cpu_addr + size - 1; @@ -245,8 +245,10 @@ static void dw_pcie_prog_outbound_atu_unroll(struct dw_pcie *pci, u8 func_no, lower_32_bits(pci_addr)); dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_UPPER_TARGET, upper_32_bits(pci_addr)); - dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL1, - type | PCIE_ATU_FUNC_NUM(func_no)); + val = type | PCIE_ATU_FUNC_NUM(func_no); + val = upper_32_bits(size - 1) ? + val | PCIE_ATU_INCREASE_REGION_SIZE : val; + dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL1, val); dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL2, PCIE_ATU_ENABLE); @@ -267,7 +269,7 @@ static void dw_pcie_prog_outbound_atu_unroll(struct dw_pcie *pci, u8 func_no, static void __dw_pcie_prog_outbound_atu(struct dw_pcie *pci, u8 func_no, int index, int type, u64 cpu_addr, - u64 pci_addr, u32 size) + u64 pci_addr, u64 size) { u32 retries, val; @@ -311,7 +313,7 @@ static void __dw_pcie_prog_outbound_atu(struct dw_pcie *pci, u8 func_no, } void dw_pcie_prog_outbound_atu(struct dw_pcie *pci, int index, int type, - u64 cpu_addr, u64 pci_addr, u32 size) + u64 cpu_addr, u64 pci_addr, u64 size) { __dw_pcie_prog_outbound_atu(pci, 0, index, type, cpu_addr, pci_addr, size); diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h index 9d2f511f13fa..e7f441441db2 100644 --- a/drivers/pci/controller/dwc/pcie-designware.h +++ b/drivers/pci/controller/dwc/pcie-designware.h @@ -84,6 +84,7 @@ #define PCIE_ATU_REGION_INDEX1 0x1 #define PCIE_ATU_REGION_INDEX0 0x0 #define PCIE_ATU_CR1 0x904 +#define PCIE_ATU_INCREASE_REGION_SIZE BIT(13) #define PCIE_ATU_TYPE_MEM 0x0 #define PCIE_ATU_TYPE_IO 0x2 #define PCIE_ATU_TYPE_CFG0 0x4 @@ -295,7 +296,7 @@ void dw_pcie_upconfig_setup(struct dw_pcie *pci); int dw_pcie_wait_for_link(struct dw_pcie *pci); void dw_pcie_prog_outbound_atu(struct dw_pcie *pci, int index, int type, u64 cpu_addr, u64 pci_addr, - u32 size); + u64 size); void dw_pcie_prog_ep_outbound_atu(struct dw_pcie *pci, u8 func_no, int index, int type, u64 cpu_addr, u64 pci_addr, u32 size); From patchwork Fri Oct 23 19:56:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vidya Sagar X-Patchwork-Id: 11854499 X-Patchwork-Delegate: lorenzo.pieralisi@arm.com Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7EBB592C for ; Fri, 23 Oct 2020 19:57:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 584EC22210 for ; Fri, 23 Oct 2020 19:57:25 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=nvidia.com header.i=@nvidia.com header.b="C4dkruRO" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755252AbgJWT5W (ORCPT ); Fri, 23 Oct 2020 15:57:22 -0400 Received: from hqnvemgate24.nvidia.com ([216.228.121.143]:1732 "EHLO hqnvemgate24.nvidia.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S374621AbgJWT5W (ORCPT ); Fri, 23 Oct 2020 15:57:22 -0400 Received: from hqmail.nvidia.com (Not Verified[216.228.121.13]) by hqnvemgate24.nvidia.com (using TLS: TLSv1.2, AES256-SHA) id ; Fri, 23 Oct 2020 12:57:30 -0700 Received: from HQMAIL101.nvidia.com (172.20.187.10) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Fri, 23 Oct 2020 19:57:20 +0000 Received: from vidyas-desktop.nvidia.com (10.124.1.5) by mail.nvidia.com (172.20.187.10) with Microsoft SMTP Server id 15.0.1473.3 via Frontend Transport; Fri, 23 Oct 2020 19:57:16 +0000 From: Vidya Sagar To: , , , , , , , CC: , , , , , Subject: [PATCH 3/3] PCI: dwc: Add support to handle prefetchable memory mapping Date: Sat, 24 Oct 2020 01:26:55 +0530 Message-ID: <20201023195655.11242-4-vidyas@nvidia.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20201023195655.11242-1-vidyas@nvidia.com> References: <20201023195655.11242-1-vidyas@nvidia.com> X-NVConfidentiality: public MIME-Version: 1.0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nvidia.com; s=n1; t=1603483050; bh=TUW5Amy+q6/UT7sHDyTCeSoRQZL9+Z8R+mBZmXHdcD8=; h=From:To:CC:Subject:Date:Message-ID:X-Mailer:In-Reply-To: References:X-NVConfidentiality:MIME-Version:Content-Type; b=C4dkruROMqmRF8Ck0Eo3/CA6pjeJz8GCksYr7Z4mBWfG958FUYnMuNREIw9ytvBVi mrMiuoS3/QuUbevtBGOMx+PhySPGVe7hNVJ5qVTWDgf18cuv5oMnKVYTtdrHBIcAYo AtqOOhakNT6XgEx9FDreRgos4a10N5PvP+NSjOc3wlwBrV3Om1a5u9b/vLWbQfff6q n3HnZKMdPBjOOw47ikB1bDk1Ok4eDd3jReF2F1X1N+gU0f44GKHkNbGCdMaD+fIOek AXQGD9AUoZkq99yrPNgypvsy7h3ZUdV32kK8mbuT/HL7mVQx7TvLDQFJ6gxTNCq7L2 ebvVNZ6c9mpdw== Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org DWC sub-system currently doesn't differentiate between prefetchable and non-prefetchable memory aperture entries in the 'ranges' property and provides ATU mapping only for the first memory aperture entry out of all the entries present. This was introduced by the commit 0f71c60ffd26 ("PCI: dwc: Remove storing of PCI resources"). Mapping for a memory apreture is required if its CPU address and the bus address are different and the current mechanism works only if the memory aperture which needs mapping happens to be the first entry. It doesn't work either if the memory aperture that needs mapping is not the first entry or if both prefetchable and non-prefetchable apertures need mapping. This patch fixes this issue by differentiating between prefetchable and non-prefetchable apertures in the 'ranges' property there by removing the dependency on the order in which they are specified and adds support for mapping prefetchable aperture using ATU region-3 if required. Fixes: 0f71c60ffd26 ("PCI: dwc: Remove storing of PCI resources") Link: http://patchwork.ozlabs.org/project/linux-pci/patch/20200513190855.23318-1-vidyas@nvidia.com/ Signed-off-by: Vidya Sagar --- Changes from previous versions: * Addressed Rob's comments and as part of that split the patch into three sub-patches * Rewrote commit subject and description * Addressed review comments from Lorenzo .../pci/controller/dwc/pcie-designware-host.c | 39 ++++++++++++++++--- drivers/pci/controller/dwc/pcie-designware.h | 1 + 2 files changed, 34 insertions(+), 6 deletions(-) diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c index 674f32db85ca..a1f319ccd816 100644 --- a/drivers/pci/controller/dwc/pcie-designware-host.c +++ b/drivers/pci/controller/dwc/pcie-designware-host.c @@ -529,9 +529,39 @@ static struct pci_ops dw_pcie_ops = { .write = pci_generic_config_write, }; +static void dw_pcie_setup_mem_atu(struct pcie_port *pp, + struct resource_entry *win) +{ + struct dw_pcie *pci = to_dw_pcie_from_pp(pp); + + /* Check for prefetchable memory aperture */ + if (win->res->flags & IORESOURCE_PREFETCH) { + /* Number of view ports must at least be 4 to enable mapping */ + if (pci->num_viewport < 4) { + dev_warn(pci->dev, + "Insufficient ATU regions to map Prefetchable memory\n"); + } else { + dw_pcie_prog_outbound_atu(pci, + PCIE_ATU_REGION_INDEX3, + PCIE_ATU_TYPE_MEM, + win->res->start, + win->res->start - win->offset, + resource_size(win->res)); + } + } else { /* Non-prefetchable memory aperture */ + dw_pcie_prog_outbound_atu(pci, + PCIE_ATU_REGION_INDEX0, + PCIE_ATU_TYPE_MEM, + win->res->start, + win->res->start - win->offset, + resource_size(win->res)); + } +} + void dw_pcie_setup_rc(struct pcie_port *pp) { u32 val, ctrl, num_ctrls; + struct resource_entry *win; struct dw_pcie *pci = to_dw_pcie_from_pp(pp); /* @@ -586,13 +616,10 @@ void dw_pcie_setup_rc(struct pcie_port *pp) * ATU, so we should not program the ATU here. */ if (pp->bridge->child_ops == &dw_child_pcie_ops) { - struct resource_entry *entry = - resource_list_first_type(&pp->bridge->windows, IORESOURCE_MEM); + resource_list_for_each_entry(win, &pp->bridge->windows) + if (resource_type(win->res) == IORESOURCE_MEM) + dw_pcie_setup_mem_atu(pp, win); - dw_pcie_prog_outbound_atu(pci, PCIE_ATU_REGION_INDEX0, - PCIE_ATU_TYPE_MEM, entry->res->start, - entry->res->start - entry->offset, - resource_size(entry->res)); if (pci->num_viewport > 2) dw_pcie_prog_outbound_atu(pci, PCIE_ATU_REGION_INDEX2, PCIE_ATU_TYPE_IO, pp->io_base, diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h index e7f441441db2..21dd06831b50 100644 --- a/drivers/pci/controller/dwc/pcie-designware.h +++ b/drivers/pci/controller/dwc/pcie-designware.h @@ -80,6 +80,7 @@ #define PCIE_ATU_VIEWPORT 0x900 #define PCIE_ATU_REGION_INBOUND BIT(31) #define PCIE_ATU_REGION_OUTBOUND 0 +#define PCIE_ATU_REGION_INDEX3 0x3 #define PCIE_ATU_REGION_INDEX2 0x2 #define PCIE_ATU_REGION_INDEX1 0x1 #define PCIE_ATU_REGION_INDEX0 0x0