From patchwork Fri Apr 28 22:34:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jim Quinlan X-Patchwork-Id: 13226861 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7C9A6C77B7C for ; Fri, 28 Apr 2023 22:35:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346801AbjD1WfP (ORCPT ); Fri, 28 Apr 2023 18:35:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46178 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346502AbjD1WfM (ORCPT ); Fri, 28 Apr 2023 18:35:12 -0400 Received: from mail-pg1-x534.google.com (mail-pg1-x534.google.com [IPv6:2607:f8b0:4864:20::534]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1655D1BDA; Fri, 28 Apr 2023 15:35:11 -0700 (PDT) Received: by mail-pg1-x534.google.com with SMTP id 41be03b00d2f7-51f597c975fso307529a12.0; Fri, 28 Apr 2023 15:35:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1682721310; x=1685313310; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=l9wGE8lEckhtyHamfE+Si7Tf+FLdP4R/QrNZ2YYLB78=; b=LI/HpU3bqYqcgk2yc4XDsCLqOVxkrLPD5Pkhnv2x1LYPgqioxl6C5Qn5gN3TgfXZjR 9XXkIUrl5d8LfQlZopOs3AALUvRiB5x7v5pSef/xfe/rcvl4TKT4vmVs2NXv1ebM7cbq gygXPmzFqT33qaYYzRzX3QGCgHGghC34yVRLvpc3scIRkBSSGJ+ZJRolXAui6fxwvAtB f5tvvzqeto780Vr92cbOKacRwzCC8NquZEbzs3IjjV9ois4v+Tm89ErdK9X7zP9hZg5R 7esj4Vw1MXnR+Jm5bY0fcVNfcNunb8ecvLH1qkED+GEZJv9H7b21XTh3QtB8w4vjcVrE QttA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682721310; x=1685313310; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=l9wGE8lEckhtyHamfE+Si7Tf+FLdP4R/QrNZ2YYLB78=; b=GAeXDs17PrlGVz19RwoXQj4+zzYrfHYUl/cWR7ufXoRtvhOdggYUHl60R7IbpUSQ8l AAFRU2UcUF9D+vRnxJ0+EnVYb8HtH73GEPSKivZLr97F/PqMPnEsSWosVdlZE2JoLp2I qBLXCtpMG9b+2dI7VgG+wva8VKDnyV5XSRbD2omwCJV2rzirumk4sRbzRQ1do2POIyIu /4SUDCFcHN/0hnK7sj4fZerOUI4h5N2o7ZTKQ4Arg/WDh/Vt2EHOARW4W0vi1llSJbd5 j5439K/LV9lOCs3R2SkaIETzngMMog286v6bf+IPYeLuTAfLKoVp3vmonjSOfa3sgLs6 JaKg== X-Gm-Message-State: AC+VfDwL98wtqawIWqJmlyigcZvUUYZkvViFFv5rHCe6X1Cpc/lzqoZA b8mKVoEaDXMZEsJvNIN/mf3oVBKqr/c= X-Google-Smtp-Source: ACHHUZ55F9CSNezNW/MWfvmvq/nt+l7GWDbpbZmg2w+R0j60UqA/yPhdGbb90HXYMl/rEYcTXlGx+A== X-Received: by 2002:a17:902:6b86:b0:1a2:6257:36b9 with SMTP id p6-20020a1709026b8600b001a2625736b9mr6058610plk.31.1682721310308; Fri, 28 Apr 2023 15:35:10 -0700 (PDT) Received: from stbsrv-and-01.and.broadcom.net ([192.19.144.250]) by smtp.gmail.com with ESMTPSA id j8-20020a170902690800b001a1b66af22fsm13657847plk.62.2023.04.28.15.35.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Apr 2023 15:35:09 -0700 (PDT) From: Jim Quinlan To: linux-pci@vger.kernel.org, Nicolas Saenz Julienne , Bjorn Helgaas , Lorenzo Pieralisi , Cyril Brulebois , Phil Elwell , bcm-kernel-feedback-list@broadcom.com, jim2101024@gmail.com, james.quinlan@broadcom.com Cc: Florian Fainelli , Lorenzo Pieralisi , =?utf-8?q?Krzysztof_Wilczy=C5=84?= =?utf-8?q?ski?= , Rob Herring , Krzysztof Kozlowski , linux-arm-kernel@lists.infradead.org (moderated list:BROADCOM BCM7XXX ARM ARCHITECTURE), linux-rpi-kernel@lists.infradead.org (moderated list:BROADCOM BCM2711/BCM2835 ARM ARCHITECTURE), devicetree@vger.kernel.org (open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS), linux-kernel@vger.kernel.org (open list) Subject: [PATCH v4 1/5] dt-bindings: PCI: brcmstb: brcm,{enable-l1ss,completion-timeout-us} props Date: Fri, 28 Apr 2023 18:34:55 -0400 Message-Id: <20230428223500.23337-2-jim2101024@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230428223500.23337-1-jim2101024@gmail.com> References: <20230428223500.23337-1-jim2101024@gmail.com> Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org This commit introduces two new properties: brcm,enable-l1ss (bool): The Broadcom STB/CM PCIe HW -- a core that is also used by RPi SOCs -- requires the driver probe() to deliberately place the HW one of three CLKREQ# modes: (a) CLKREQ# driven by the RC unconditionally (b) CLKREQ# driven by the EP for ASPM L0s, L1 (c) Bidirectional CLKREQ#, as used for L1 Substates (L1SS). The HW+driver can tell the difference between downstream devices that need (a) and (b), but does not know when to configure (c). All devices should work fine when the driver chooses (a) or (b), but (c) may be desired to realize the extra power savings that L1SS offers. So we introduce the boolean "brcm,enable-l1ss" property to inform the driver that (c) is desired. Setting this property only makes sense when the downstream device is L1SS-capable and the OS is configured to activate this mode (e.g. policy==superpowersave). This property is already present in the Raspian version of Linux, but the upstream driver implementaion that follows adds more details and discerns between (a) and (b). brcm,completion-timeout-us (u32): Our HW will cause a CPU abort on any PCI transaction completion abort error. It makes sense then to increase the timeout value for this type of error in hopes that the response is merely delayed. Further, L1SS-capable devices may have a long L1SS exit time and may require a custom timeout value: we've been asked by our customers to make this configurable for just this reason. Signed-off-by: Jim Quinlan Reviewed-by: Rob Herring --- .../devicetree/bindings/pci/brcm,stb-pcie.yaml | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/Documentation/devicetree/bindings/pci/brcm,stb-pcie.yaml b/Documentation/devicetree/bindings/pci/brcm,stb-pcie.yaml index 7e15aae7d69e..239cc95545bd 100644 --- a/Documentation/devicetree/bindings/pci/brcm,stb-pcie.yaml +++ b/Documentation/devicetree/bindings/pci/brcm,stb-pcie.yaml @@ -64,6 +64,22 @@ properties: aspm-no-l0s: true + brcm,enable-l1ss: + description: Indicates that PCIe L1SS power savings + are desired, the downstream device is L1SS-capable, and the + OS has been configured to enable this mode. For boards + using a mini-card connector, this mode may not meet the + TCRLon maximum time of 400ns, as specified in 3.2.5.2.5 + of the PCI Express Mini CEM 2.0 specification. + type: boolean + + brcm,completion-timeout-us: + description: Number of microseconds before PCI transaction + completion timeout abort is signalled. + minimum: 16 + default: 1000000 + maximum: 19884107 + brcm,scb-sizes: description: u64 giving the 64bit PCIe memory viewport size of a memory controller. There may be up to From patchwork Fri Apr 28 22:34:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jim Quinlan X-Patchwork-Id: 13226862 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 65B01C7EE23 for ; Fri, 28 Apr 2023 22:35:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346502AbjD1WfQ (ORCPT ); Fri, 28 Apr 2023 18:35:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46274 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346802AbjD1WfP (ORCPT ); Fri, 28 Apr 2023 18:35:15 -0400 Received: from mail-pj1-x102b.google.com (mail-pj1-x102b.google.com [IPv6:2607:f8b0:4864:20::102b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 146FD4EEC; Fri, 28 Apr 2023 15:35:13 -0700 (PDT) Received: by mail-pj1-x102b.google.com with SMTP id 98e67ed59e1d1-24735727c83so341507a91.3; Fri, 28 Apr 2023 15:35:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1682721313; x=1685313313; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=MGCuOvlYjAihsXcy4j2O4xddc7TDjVJgSaiz9+vHtp8=; b=QyJ7WcgcyHe4qsX3BUwQ+ekd+GNyZBsu3Ff1X48QmBVrQFBe4FbIZ7sK8MN63oFQHr jF/D5DvNIdKvuNkTrvFYyq1vRnzhqBri9Jzf58wpKWBqtNBp/4D9DKAyDzZlBu6BWt0z 6Yo20AM5KCAwJm2E170fcoyjuwTwSRpj9W2SmCueyOJQHfzq8Gntrkmr5HJ68GhoxRGi KWpNLgL2iHgEEzO/uivvb0r+o8THmC2gSDFS/hzTiPmRjYuPmHM3x3uHnwuXDhNw3MhX xCIwMivof5+FTy0xj2JmarGnvYXB6eKii7veSpW4qezpCineeywJoFWBiNVEhtT6P5wa Y+wA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682721313; x=1685313313; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=MGCuOvlYjAihsXcy4j2O4xddc7TDjVJgSaiz9+vHtp8=; b=S5yUYh6iKFmtcorTf4C01UiZ0U8UFXCuM3NrEimGf1Nu1TLAt1z237ZLpySn5JxJ7u fhrS5N/KrBP0mTIQ4sGKKaemVdKRte6Z2ItK5ujIeA1S172t2MZToRAy/IPk6O3U+aRj 2xdsCofP28VhZqsxTw0zYP7JvpsNaFsUQwLVi4JyqUwGYrxapkT1+DLhD0v+m3cECPkq jjVIzxADjY3ScdnKS4RNnVMvb7QycwbJA/ERI6xAUcOmJ2jOfB9d2rcoADfknXBVRqG7 56tiTZSF8S8nVRf3OjV+GDD/NGMB3DsAPmx+xzrCNBNIKqJI1BVPmLSOV+ij3ldi+3HI LJqA== X-Gm-Message-State: AC+VfDy6Tjj3nMM5ZXTFlavl+b9VpnXYBq6qpNxWRIKJrojxFxMLO5OM Smc5M9FwhooccPZFOpekees+yGEd0lo= X-Google-Smtp-Source: ACHHUZ4PEIx62HVF0KATes6ZI4nSOmplsqYUffl2I9mxh5P+eHr+IhaIax8goRdmgxHc2VkJmShQRQ== X-Received: by 2002:a17:90a:ae84:b0:240:f8a6:55c7 with SMTP id u4-20020a17090aae8400b00240f8a655c7mr6810947pjq.20.1682721313164; Fri, 28 Apr 2023 15:35:13 -0700 (PDT) Received: from stbsrv-and-01.and.broadcom.net ([192.19.144.250]) by smtp.gmail.com with ESMTPSA id j8-20020a170902690800b001a1b66af22fsm13657847plk.62.2023.04.28.15.35.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Apr 2023 15:35:12 -0700 (PDT) From: Jim Quinlan To: linux-pci@vger.kernel.org, Nicolas Saenz Julienne , Bjorn Helgaas , Lorenzo Pieralisi , Cyril Brulebois , Phil Elwell , bcm-kernel-feedback-list@broadcom.com, jim2101024@gmail.com, james.quinlan@broadcom.com Cc: Florian Fainelli , Lorenzo Pieralisi , =?utf-8?q?Krzysztof_Wilczy=C5=84?= =?utf-8?q?ski?= , Rob Herring , linux-rpi-kernel@lists.infradead.org (moderated list:BROADCOM BCM2711/BCM2835 ARM ARCHITECTURE), linux-arm-kernel@lists.infradead.org (moderated list:BROADCOM BCM2711/BCM2835 ARM ARCHITECTURE), linux-kernel@vger.kernel.org (open list) Subject: [PATCH v4 2/5] PCI: brcmstb: Configure HW CLKREQ# mode appropriate for downstream device Date: Fri, 28 Apr 2023 18:34:56 -0400 Message-Id: <20230428223500.23337-3-jim2101024@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230428223500.23337-1-jim2101024@gmail.com> References: <20230428223500.23337-1-jim2101024@gmail.com> Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org The Broadcom STB/CM PCIe HW core, which is also used in RPi SOCs, must be deliberately set by the RC probe() into one of three mutually exclusive modes: (a) No CLKREQ# expected or required, refclk is always available. (b) CLKREQ# is expected to be driven by downstream device when needed. (c) Bidirectional CLKREQ# for L1SS capable devices. Previously, only (b) was supported by the driver, as almost all STB/CM boards operate in this mode. But now there is interest in activating L1SS power savings from STB/CM customers, and also interest in accommodating mode (a) for designs such as the RPi CM4 with IO board. The HW+driver is able to tell us when mode (a) or (b) is needed. All devices should be functional using the RC-driver selected (a) or (b) mode. For those with L1SS-capable devices that desire the power savings that come with mode (c) we rely on the DT prop "brcm,enable-l1ss". It would be nice to do this automatically but there is no easy way to determine this at the time the PCI RC driver executes its probe(). Using this mode only makes sense when the downstream device is L1SS-capable and the OS has been configured to activate L1SS (e.g. policy==powersupersave). The "brcm,enable-l1ss" property has already been in use by Raspian Linux, but this implementation adds more details and discerns between (a) and (b) automatically. Link: https://bugzilla.kernel.org/show_bug.cgi?id=217276 Tested-by: Florian Fainelli Signed-off-by: Jim Quinlan --- drivers/pci/controller/pcie-brcmstb.c | 69 +++++++++++++++++++++++---- 1 file changed, 59 insertions(+), 10 deletions(-) diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c index edf283e2b5dd..c4b076ea5180 100644 --- a/drivers/pci/controller/pcie-brcmstb.c +++ b/drivers/pci/controller/pcie-brcmstb.c @@ -48,10 +48,17 @@ #define PCIE_RC_CFG_PRIV1_LINK_CAPABILITY 0x04dc #define PCIE_RC_CFG_PRIV1_LINK_CAPABILITY_ASPM_SUPPORT_MASK 0xc00 +#define PCIE_RC_CFG_PRIV1_ROOT_CAP 0x4f8 +#define PCIE_RC_CFG_PRIV1_ROOT_CAP_L1SS_MODE_MASK 0xf8 + #define PCIE_RC_DL_MDIO_ADDR 0x1100 #define PCIE_RC_DL_MDIO_WR_DATA 0x1104 #define PCIE_RC_DL_MDIO_RD_DATA 0x1108 +#define PCIE_0_RC_PL_PHY_DBG_CLKREQ2_0 0x1e30 +#define CLKREQ2_0_CLKREQ_IN_CNT_MASK 0x3f000000 +#define CLKREQ2_0_CLKREQ_IN_MASK 0x40000000 + #define PCIE_MISC_MISC_CTRL 0x4008 #define PCIE_MISC_MISC_CTRL_PCIE_RCB_64B_MODE_MASK 0x80 #define PCIE_MISC_MISC_CTRL_PCIE_RCB_MPS_MODE_MASK 0x400 @@ -121,9 +128,12 @@ #define PCIE_MISC_HARD_PCIE_HARD_DEBUG 0x4204 #define PCIE_MISC_HARD_PCIE_HARD_DEBUG_CLKREQ_DEBUG_ENABLE_MASK 0x2 +#define PCIE_MISC_HARD_PCIE_HARD_DEBUG_L1SS_ENABLE_MASK 0x200000 #define PCIE_MISC_HARD_PCIE_HARD_DEBUG_SERDES_IDDQ_MASK 0x08000000 #define PCIE_BMIPS_MISC_HARD_PCIE_HARD_DEBUG_SERDES_IDDQ_MASK 0x00800000 - +#define PCIE_CLKREQ_MASK \ + (PCIE_MISC_HARD_PCIE_HARD_DEBUG_CLKREQ_DEBUG_ENABLE_MASK | \ + PCIE_MISC_HARD_PCIE_HARD_DEBUG_L1SS_ENABLE_MASK) #define PCIE_INTR2_CPU_BASE 0x4300 #define PCIE_MSI_INTR2_BASE 0x4500 @@ -1024,13 +1034,58 @@ static int brcm_pcie_setup(struct brcm_pcie *pcie) return 0; } +static void brcm_config_clkreq(struct brcm_pcie *pcie) +{ + bool l1ss = of_property_read_bool(pcie->np, "brcm,enable-l1ss"); + void __iomem *base = pcie->base; + u32 clkreq_set, tmp = readl(base + PCIE_0_RC_PL_PHY_DBG_CLKREQ2_0); + bool clkreq_in_seen; + + /* + * We have "seen" CLKREQ# if it is asserted or has been in the past. + * Note that the CLKREQ_IN_MASK is 1 if CLKREQ# is asserted. + */ + clkreq_in_seen = !!(tmp & CLKREQ2_0_CLKREQ_IN_MASK) || + !!FIELD_GET(CLKREQ2_0_CLKREQ_IN_CNT_MASK, tmp); + + /* Start with safest setting where we provide refclk regardless */ + clkreq_set = readl(pcie->base + PCIE_MISC_HARD_PCIE_HARD_DEBUG) & + ~PCIE_CLKREQ_MASK; + + if (l1ss && IS_ENABLED(CONFIG_PCIEASPM)) { + /* + * Note: For boards using a mini-card connector, this mode + * may not meet the TCRLon maximum time of 400ns, as + * specified in 3.2.5.2.5 of the PCI Express Mini CEM 2.0 + * specification. + */ + clkreq_set |= PCIE_MISC_HARD_PCIE_HARD_DEBUG_L1SS_ENABLE_MASK; + dev_info(pcie->dev, "bi-dir CLKREQ# for L1SS power savings"); + } else { + if (clkreq_in_seen && IS_ENABLED(CONFIG_PCIEASPM)) { + clkreq_set |= PCIE_MISC_HARD_PCIE_HARD_DEBUG_CLKREQ_DEBUG_ENABLE_MASK; + dev_info(pcie->dev, "uni-dir CLKREQ# for L0s, L1 ASPM\n"); + } else { + dev_info(pcie->dev, "CLKREQ# ignored; no ASPM\n"); + /* Might as well unadvertise ASPM */ + tmp = readl(base + PCIE_RC_CFG_PRIV1_LINK_CAPABILITY) & + ~PCIE_RC_CFG_PRIV1_LINK_CAPABILITY_ASPM_SUPPORT_MASK; + writel(tmp, base + PCIE_RC_CFG_PRIV1_LINK_CAPABILITY); + } + /* Setting the field to 2 unadvertises L1SS support */ + tmp = readl(base + PCIE_RC_CFG_PRIV1_ROOT_CAP); + u32p_replace_bits(&tmp, 2, PCIE_RC_CFG_PRIV1_ROOT_CAP_L1SS_MODE_MASK); + writel(tmp, base + PCIE_RC_CFG_PRIV1_ROOT_CAP); + } + writel(clkreq_set, pcie->base + PCIE_MISC_HARD_PCIE_HARD_DEBUG); +} + static int brcm_pcie_start_link(struct brcm_pcie *pcie) { struct device *dev = pcie->dev; void __iomem *base = pcie->base; u16 nlw, cls, lnksta; bool ssc_good = false; - u32 tmp; int ret, i; /* Unassert the fundamental reset */ @@ -1055,6 +1110,8 @@ static int brcm_pcie_start_link(struct brcm_pcie *pcie) return -ENODEV; } + brcm_config_clkreq(pcie); + if (pcie->gen) brcm_pcie_set_gen(pcie, pcie->gen); @@ -1073,14 +1130,6 @@ static int brcm_pcie_start_link(struct brcm_pcie *pcie) pci_speed_string(pcie_link_speed[cls]), nlw, ssc_good ? "(SSC)" : "(!SSC)"); - /* - * Refclk from RC should be gated with CLKREQ# input when ASPM L0s,L1 - * is enabled => setting the CLKREQ_DEBUG_ENABLE field to 1. - */ - tmp = readl(base + PCIE_MISC_HARD_PCIE_HARD_DEBUG); - tmp |= PCIE_MISC_HARD_PCIE_HARD_DEBUG_CLKREQ_DEBUG_ENABLE_MASK; - writel(tmp, base + PCIE_MISC_HARD_PCIE_HARD_DEBUG); - return 0; } From patchwork Fri Apr 28 22:34:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jim Quinlan X-Patchwork-Id: 13226863 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5718AC77B61 for ; Fri, 28 Apr 2023 22:35:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346813AbjD1WfT (ORCPT ); Fri, 28 Apr 2023 18:35:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46238 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346810AbjD1WfR (ORCPT ); Fri, 28 Apr 2023 18:35:17 -0400 Received: from mail-pj1-x1031.google.com (mail-pj1-x1031.google.com [IPv6:2607:f8b0:4864:20::1031]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DA58C4ED2; Fri, 28 Apr 2023 15:35:16 -0700 (PDT) Received: by mail-pj1-x1031.google.com with SMTP id 98e67ed59e1d1-2496863c2c7so457635a91.1; Fri, 28 Apr 2023 15:35:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1682721316; x=1685313316; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=KlHfoAK9yIIL7Skn5B+cEJbm8dBEJYFGzxR8exxzgXQ=; b=Kb+u4fJA7dD3ma7mh7ur3/WOGuTpaYWQFbTJ15nPcFDNiF+ONXnzW8oq6ZHX1Vup+0 seIwnPvE87ivhg8uQliuz2f5wPrcotvoo2mge2XTb8+v/rh7BX4DbPsX3Ntin7c+h/ax rYjnbirkxtlMiXz8s4dV/dmhUNqFbDTx19EsQ6LDuSEHNn0qNQhhY+7i1KdWRjD/e6wC x+yFh30Qrk17VtQmIeqQIsHJayPMxkkXNya5vvQmc5OPTvW77dMwYGo5UoW8xoDF/i8x /OeK7BaMftDsattR2p3RpxgnYorlwGLUbLnwNxdrN428PAPyHXqA4LsKc5Mg+82N1XdV iLJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682721316; x=1685313316; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=KlHfoAK9yIIL7Skn5B+cEJbm8dBEJYFGzxR8exxzgXQ=; b=lfGz87+RmfV6CiBtH4/D1G2xzBarWV0skayM8IW8/NBtd8pBXfDRNFLOh9PDaJcYor /AH544+CUgYZt1/AGnlcF3jBIheB4i1ylu7eR0tLm2b9fWAmxE4lTnIb4NtWPk9sziCR aHI30PfE0Zmmr4XEbK7uI477mxbtshXxaExyYKOWGXnv+A/rA0PYCOKvf2etxksYR7eh dq+nbzCQozRNyLdP4Arienk5lBAoH2WJrd0w1H9r2Jgz9mk9UP0Vz59h4oAuXtOHhsfx 5bReHLMrWWC8oU3vLdRGlSP6mPVeLvPxmYQQ17nj3pOvG1NlZWWcuMJXXc8+K/IHh6x1 UrOA== X-Gm-Message-State: AC+VfDwelaZFOBrQPbadVq/0oIHLNKlWkkHBcBJAMKws3u8FAd5EZsoV vWFrHFcsIGzSyJhEboi+9Loyzyy6rCE= X-Google-Smtp-Source: ACHHUZ68ybGXSJgG5K8t8W9riO0/4ScMAw2d8sx0MAvypN9nDxB1x2uEHSKe4GewhEAxoXQFTg1YHA== X-Received: by 2002:a17:90b:1a81:b0:247:7def:236a with SMTP id ng1-20020a17090b1a8100b002477def236amr7109657pjb.34.1682721315980; Fri, 28 Apr 2023 15:35:15 -0700 (PDT) Received: from stbsrv-and-01.and.broadcom.net ([192.19.144.250]) by smtp.gmail.com with ESMTPSA id j8-20020a170902690800b001a1b66af22fsm13657847plk.62.2023.04.28.15.35.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Apr 2023 15:35:15 -0700 (PDT) From: Jim Quinlan To: linux-pci@vger.kernel.org, Nicolas Saenz Julienne , Bjorn Helgaas , Lorenzo Pieralisi , Cyril Brulebois , Phil Elwell , bcm-kernel-feedback-list@broadcom.com, jim2101024@gmail.com, james.quinlan@broadcom.com Cc: Florian Fainelli , Lorenzo Pieralisi , =?utf-8?q?Krzysztof_Wilczy=C5=84?= =?utf-8?q?ski?= , Rob Herring , linux-rpi-kernel@lists.infradead.org (moderated list:BROADCOM BCM2711/BCM2835 ARM ARCHITECTURE), linux-arm-kernel@lists.infradead.org (moderated list:BROADCOM BCM2711/BCM2835 ARM ARCHITECTURE), linux-kernel@vger.kernel.org (open list) Subject: [PATCH v4 3/5] PCI: brcmstb: Set PCIe transaction completion timeout Date: Fri, 28 Apr 2023 18:34:57 -0400 Message-Id: <20230428223500.23337-4-jim2101024@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230428223500.23337-1-jim2101024@gmail.com> References: <20230428223500.23337-1-jim2101024@gmail.com> Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Since the STB PCIe HW will cause a CPU abort on a PCIe transaction completion timeout abort, we might as well extend the default timeout limit. Further, different devices and systems may requires a larger or smaller amount commensurate with their L1SS exit time, so the property "brcm,completion-timeout-us" may be used to set a custom timeout value. Tested-by: Florian Fainelli Signed-off-by: Jim Quinlan --- drivers/pci/controller/pcie-brcmstb.c | 30 +++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c index c4b076ea5180..c2cb683447ac 100644 --- a/drivers/pci/controller/pcie-brcmstb.c +++ b/drivers/pci/controller/pcie-brcmstb.c @@ -1080,6 +1080,35 @@ static void brcm_config_clkreq(struct brcm_pcie *pcie) writel(clkreq_set, pcie->base + PCIE_MISC_HARD_PCIE_HARD_DEBUG); } +static void brcm_config_completion_timeout(struct brcm_pcie *pcie) +{ + /* TIMEOUT register is two registers before RGR1_SW_INIT_1 */ + const char *fmt = "brcm,completion-timeout-us clamped to region [%u..%u]\n"; + const unsigned int REG_OFFSET = PCIE_RGR1_SW_INIT_1(pcie) - 8; + const u32 timeout_us_min = 16; + const u32 timeout_us_max = 19884107; + u32 timeout_us = 1000000; /* Our default, 1 second */ + int rval, ret; + + ret = of_property_read_u32(pcie->np, "brcm,completion-timeout-us", + &timeout_us); + if (ret && ret != -EINVAL) + dev_err(pcie->dev, "malformed/invalid 'brcm,completion-timeout-us'\n"); + + /* If needed, clamp the requested timeout value and issue a warning */ + if (timeout_us < timeout_us_min) { + timeout_us = timeout_us_min; + dev_warn(pcie->dev, fmt, timeout_us_min, timeout_us_max); + } else if (timeout_us > timeout_us_max) { + timeout_us = timeout_us_max; + dev_warn(pcie->dev, fmt, timeout_us_min, timeout_us_max); + } + + /* Each unit in timeout register is 1/216,000,000 seconds */ + rval = 216 * timeout_us; + writel(rval, pcie->base + REG_OFFSET); +} + static int brcm_pcie_start_link(struct brcm_pcie *pcie) { struct device *dev = pcie->dev; @@ -1110,6 +1139,7 @@ static int brcm_pcie_start_link(struct brcm_pcie *pcie) return -ENODEV; } + brcm_config_completion_timeout(pcie); brcm_config_clkreq(pcie); if (pcie->gen) From patchwork Fri Apr 28 22:34:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jim Quinlan X-Patchwork-Id: 13226864 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 186F6C77B61 for ; Fri, 28 Apr 2023 22:35:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346865AbjD1Wff (ORCPT ); Fri, 28 Apr 2023 18:35:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46468 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346547AbjD1Wfb (ORCPT ); Fri, 28 Apr 2023 18:35:31 -0400 Received: from mail-pl1-x632.google.com (mail-pl1-x632.google.com [IPv6:2607:f8b0:4864:20::632]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0D04749D2; Fri, 28 Apr 2023 15:35:19 -0700 (PDT) Received: by mail-pl1-x632.google.com with SMTP id d9443c01a7336-1a6bc48aec8so3909125ad.2; Fri, 28 Apr 2023 15:35:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1682721319; x=1685313319; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=soSZ2SRcxkgwO6bRCgJ7t3tsyaulOFN/8ZMK9B68Z4o=; b=cqMzvh19jnoipsC+X8oNLX6h03W5oevo5LyC8x31tSxe1fyOjrADCqD4XhPQ9ytN2m WwikptL1oi9hYiGuOr10dTeODrfI7HlcwagUq2kxUxZ+i1k2A6/qP8e9uulYY7RH6tBd O4d0maGA3SyB5Gm+E0DhRGGqCCi+GcoEZItiZpgvc1aRxpYLkHrhNMMINYWUSWIfgYol XSzWAiQb8MsGLB6WkljcwlmXuAhAafk69stum9UkuebsAKaeh3kbgPKKGbgAlYrcK/dN xo961KahNe9Fb2IeWfROSEg/TmNQ7wEylOoSHObHeYsr9kvjThoRcUPg4wzTny4k1E6x h5mQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682721319; x=1685313319; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=soSZ2SRcxkgwO6bRCgJ7t3tsyaulOFN/8ZMK9B68Z4o=; b=H81oH1owO75D67ZwacoARVYUgwBJHWZlJv10RfyZasAVB4mMBl88C8AIZXQmRcXEu2 sOvHF707hUEwGXug4BxH9j+pBB+3EA9YiSQ1JDxvTXbp/jHP2bR9rBsbMcJoLjiGwRzZ NbcjBhNP5FtptgQcHpybxdXcLyBiNB+FaE7/MvDh883nP3F1p9U8L+O9zW4MxCC+H0M1 wBLcIzXLiDFdDJ6Oat5D+kCST7McqBqmnAGikiEn8DzNGjZ9zrOz09GMbza4JtKWnfba h6ix5y4C7DhCgE0sRqfg88vF9Yadc/sEVP6dLTr7SafhCbWnU92+dzSlPsB+NRzM7LHG K+PA== X-Gm-Message-State: AC+VfDwaoH81BpnaLmecsvZkk9bnRbsxPEP31OOszgYfe1gunYex1Pbw YeG90HLXlW/LE2TxIur4c2uVeGnEXiM= X-Google-Smtp-Source: ACHHUZ4PiExedhiiNp0qLo/UOua0tCni9cce60v18358xqFHAGRhITwr+7v6IRsdRkdk0+e5MUialg== X-Received: by 2002:a17:902:b20e:b0:1a1:d366:b085 with SMTP id t14-20020a170902b20e00b001a1d366b085mr6517084plr.21.1682721318911; Fri, 28 Apr 2023 15:35:18 -0700 (PDT) Received: from stbsrv-and-01.and.broadcom.net ([192.19.144.250]) by smtp.gmail.com with ESMTPSA id j8-20020a170902690800b001a1b66af22fsm13657847plk.62.2023.04.28.15.35.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Apr 2023 15:35:18 -0700 (PDT) From: Jim Quinlan To: linux-pci@vger.kernel.org, Nicolas Saenz Julienne , Bjorn Helgaas , Lorenzo Pieralisi , Cyril Brulebois , Phil Elwell , bcm-kernel-feedback-list@broadcom.com, jim2101024@gmail.com, james.quinlan@broadcom.com Cc: Florian Fainelli , Lorenzo Pieralisi , =?utf-8?q?Krzysztof_Wilczy=C5=84?= =?utf-8?q?ski?= , Rob Herring , linux-rpi-kernel@lists.infradead.org (moderated list:BROADCOM BCM2711/BCM2835 ARM ARCHITECTURE), linux-arm-kernel@lists.infradead.org (moderated list:BROADCOM BCM2711/BCM2835 ARM ARCHITECTURE), linux-kernel@vger.kernel.org (open list) Subject: [PATCH v4 4/5] PCI: brcmstb: Don't assume 2711 bootloader leaves PERST# asserted Date: Fri, 28 Apr 2023 18:34:58 -0400 Message-Id: <20230428223500.23337-5-jim2101024@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230428223500.23337-1-jim2101024@gmail.com> References: <20230428223500.23337-1-jim2101024@gmail.com> Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org The current PCIe driver assumes PERST# is asserted when probe() is invoked. The reasons are as follows: (1) One Broadcom SOC (7278) causes a panic if the PERST# register is written during this time window. (2) If PERST# is deasserted at Linux probe() time, experience and QA suspend/resume tests have shown that some endpoint devices fail if the PERST# is pulsed (deasserted => asserted => deasserted) quickly in this fashion, even though the timing is in accordance with their datasheets. (3) Keeping things in reset tends to save power, if for some reason the PCIe driver is not yet present. Broadcom STB and CM SOCs bootloaders always have PERST# asserted at probe() time. This is not necessarily the case for the 2711/RPi bootloader. In addition, there is a failing test case [1] that may be caused by a deasserted PERST#. Finally, Raspian version of Linux does assert PERST# at probe() time. So, for 2711/RPi SOCs, do what Raspian does and assert PERST#. [1] https://lore.kernel.org/linux-pci/20230411165919.23955-1-jim2101024@gmail.com/T/#m39ebab8bc2827b2304aeeff470a6c6a58f46f987 Signed-off-by: Jim Quinlan --- drivers/pci/controller/pcie-brcmstb.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c index c2cb683447ac..c486f4b979cc 100644 --- a/drivers/pci/controller/pcie-brcmstb.c +++ b/drivers/pci/controller/pcie-brcmstb.c @@ -884,6 +884,11 @@ static int brcm_pcie_setup(struct brcm_pcie *pcie) /* Reset the bridge */ pcie->bridge_sw_init_set(pcie, 1); + + /* Ensure that PERST# is asserted; some bootloaders may deassert it. */ + if (pcie->type == BCM2711) + pcie->perst_set(pcie, 1); + usleep_range(100, 200); /* Take the bridge out of reset */ From patchwork Fri Apr 28 22:34:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jim Quinlan X-Patchwork-Id: 13226865 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8D784C77B61 for ; Fri, 28 Apr 2023 22:35:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346547AbjD1Wfi (ORCPT ); Fri, 28 Apr 2023 18:35:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46314 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346818AbjD1Wfe (ORCPT ); Fri, 28 Apr 2023 18:35:34 -0400 Received: from mail-pl1-x636.google.com (mail-pl1-x636.google.com [IPv6:2607:f8b0:4864:20::636]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 31FAE5BAC; Fri, 28 Apr 2023 15:35:23 -0700 (PDT) Received: by mail-pl1-x636.google.com with SMTP id d9443c01a7336-1a6f0d8cdfeso3892695ad.2; Fri, 28 Apr 2023 15:35:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1682721322; x=1685313322; h=references:in-reply-to:message-id:date:subject:cc:to:from:from:to :cc:subject:date:message-id:reply-to; bh=oyCMutM3XR7UqQNXvyE3UlGy3eP0WxqAenzYZpxmAUk=; b=ZqOHNSSwc1p+SXnbXXR8VyjHYjV1nHxeFEw8EYt7f/hreRJrimkKUvP/Skpxkud6sv 9ZqvzOBRZAVFccK69sJC93Ej3w3Oj/RBgClqub2cC110RUnOe1S/XyRuHDfV7DnlzEQc rC6a5pJcZwB4PiWGqgkqaS74FMkojE4742aCw6ZoOIpfsoo0L9/kzyA1nZTt/OCg79nc HH3wwnXEfzpZMwaOvU2Qnx75IXcHd/8/25W/6dclbpZegPKfZuPkeVBJWSx07sW+oE47 KdMzdFCyD3t//70b2A7rYFhpvVaI2UukR7VQCeqSVVXGtvfroX125wpM9QR8bvZzixK0 qSwQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682721322; x=1685313322; h=references:in-reply-to:message-id:date:subject:cc:to:from :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=oyCMutM3XR7UqQNXvyE3UlGy3eP0WxqAenzYZpxmAUk=; b=RcjDU8H3fTcRCgDXQP8Ga5zFqgEHo5G0U374DOU+mHioDpYePJl1R48j+Z7z20C81v g50fa1zRq1CWBSDejBGuFpzgW7iPJFu3HD3I2mNUrfX8gkdxIn2LUKziI6ko/rzJfLVq FZijW5BijHwiSdvA/ASHbEbnL5hm/dHuKre7Z5UFqyztB8bZgBPbjMD9z1MNFnLeFCLx My7jeh/6uICyoqoAk3ex//y4FFLCPt+AVBzU5drzEL7ncwP8AxIugT1G+5wzALQ1wGgG mGOBzWBl20g9bl8FPEGd5E0a6P3hBcdsFBwG15Rxxsy/aUJzdDBaBEDbFF5gSRlJhY8W 0ikQ== X-Gm-Message-State: AC+VfDwMhdSLdtM9DuSUKDnEnbNr3kajFTY60tsB5QH8Uu/gngi97iWb GZr/Nhf934QnyA7XOo/hCfT4mKO2HFw= X-Google-Smtp-Source: ACHHUZ7+91atWxFkw5CuAQIQHCd0j3x7oUzOTb3fUr+/wDoDCx5E0tjtJeTEjJJqUKsUEOdUqHe/4Q== X-Received: by 2002:a17:903:2451:b0:1a9:baaf:7ed with SMTP id l17-20020a170903245100b001a9baaf07edmr6753096pls.66.1682721322215; Fri, 28 Apr 2023 15:35:22 -0700 (PDT) Received: from stbsrv-and-01.and.broadcom.net ([192.19.144.250]) by smtp.gmail.com with ESMTPSA id j8-20020a170902690800b001a1b66af22fsm13657847plk.62.2023.04.28.15.35.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Apr 2023 15:35:21 -0700 (PDT) From: Jim Quinlan To: linux-pci@vger.kernel.org, Nicolas Saenz Julienne , Bjorn Helgaas , Lorenzo Pieralisi , Cyril Brulebois , Phil Elwell , bcm-kernel-feedback-list@broadcom.com, jim2101024@gmail.com, james.quinlan@broadcom.com Cc: Florian Fainelli , Lorenzo Pieralisi , =?utf-8?q?Krzysztof_Wilczy=C5=84?= =?utf-8?q?ski?= , Rob Herring , linux-rpi-kernel@lists.infradead.org (moderated list:BROADCOM BCM2711/BCM2835 ARM ARCHITECTURE), linux-arm-kernel@lists.infradead.org (moderated list:BROADCOM BCM2711/BCM2835 ARM ARCHITECTURE), linux-kernel@vger.kernel.org (open list) Subject: [PATCH v4 5/5] PCI: brcmstb: Remove stale comment Date: Fri, 28 Apr 2023 18:34:59 -0400 Message-Id: <20230428223500.23337-6-jim2101024@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230428223500.23337-1-jim2101024@gmail.com> References: <20230428223500.23337-1-jim2101024@gmail.com> Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org A comment says that Multi-MSI is not supported by the driver. A past commit [1] added this feature, so the comment is incorrect and is removed. [1] commit 198acab1772f22f2 ("PCI: brcmstb: Enable Multi-MSI") Signed-off-by: Jim Quinlan --- drivers/pci/controller/pcie-brcmstb.c | 1 - 1 file changed, 1 deletion(-) diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c index c486f4b979cc..25f11f03fa09 100644 --- a/drivers/pci/controller/pcie-brcmstb.c +++ b/drivers/pci/controller/pcie-brcmstb.c @@ -449,7 +449,6 @@ static struct irq_chip brcm_msi_irq_chip = { }; static struct msi_domain_info brcm_msi_domain_info = { - /* Multi MSI is supported by the controller, but not by this driver */ .flags = (MSI_FLAG_USE_DEF_DOM_OPS | MSI_FLAG_USE_DEF_CHIP_OPS | MSI_FLAG_MULTI_PCI_MSI), .chip = &brcm_msi_irq_chip,