| Message ID | 1651801629-30223-7-git-send-email-hongxing.zhu@nxp.com (mailing list archive) |
|---|---|
| State | New, archived |
| Delegated to: | Lorenzo Pieralisi |
| Headers | show |
| Series | PCI: imx6: refine codes and add compliance tests mode support | expand |
Am Freitag, dem 06.05.2022 um 09:47 +0800 schrieb Richard Zhu: > Since i.MX PCIe doesn't support hot-plug, reduce power consumption > as much as possible by disabling clocks and regulators and returning > error when the link is down. > > Signed-off-by: Richard Zhu <hongxing.zhu@nxp.com> > --- > drivers/pci/controller/dwc/pci-imx6.c | 19 +++++++++++++++---- > 1 file changed, 15 insertions(+), 4 deletions(-) > > diff --git a/drivers/pci/controller/dwc/pci-imx6.c b/drivers/pci/controller/dwc/pci-imx6.c > index 3ce3993d5797..d122c12193a6 100644 > --- a/drivers/pci/controller/dwc/pci-imx6.c > +++ b/drivers/pci/controller/dwc/pci-imx6.c > @@ -845,7 +845,9 @@ static int imx6_pcie_start_link(struct dw_pcie *pci) > /* Start LTSSM. */ > imx6_pcie_ltssm_enable(dev); > > - dw_pcie_wait_for_link(pci); > + ret = dw_pcie_wait_for_link(pci); > + if (ret) > + goto err_out; This adds back error handling that has been intentionally removed in f81f095e8771 ("PCI: imx6: Allow to probe when dw_pcie_wait_for_link() fails"). While I agree that disabling the clocks and regulators is the right thing to do when we don't manage to get a link, we should still allow the driver to probe, so please add a "ret = 0" to this newly added non-fatal error paths. > > if (pci->link_gen == 2) { > /* Allow Gen2 mode after the link is up. */ > @@ -876,12 +878,14 @@ static int imx6_pcie_start_link(struct dw_pcie *pci) > ret = imx6_pcie_wait_for_speed_change(imx6_pcie); > if (ret) { > dev_err(dev, "Failed to bring link up!\n"); > - goto err_reset_phy; > + goto err_out; > } > } > > /* Make sure link training is finished as well! */ > - dw_pcie_wait_for_link(pci); > + ret = dw_pcie_wait_for_link(pci); > + if (ret) > + goto err_out; > } else { > dev_info(dev, "Link: Gen2 disabled\n"); > } > @@ -890,11 +894,18 @@ static int imx6_pcie_start_link(struct dw_pcie *pci) > dev_info(dev, "Link up, Gen%i\n", tmp & PCI_EXP_LNKSTA_CLS); > return 0; > > -err_reset_phy: > +err_out: > dev_dbg(dev, "PHY DEBUG_R0=0x%08x DEBUG_R1=0x%08x\n", > dw_pcie_readl_dbi(pci, PCIE_PORT_DEBUG0), > dw_pcie_readl_dbi(pci, PCIE_PORT_DEBUG1)); > imx6_pcie_reset_phy(imx6_pcie); > + imx6_pcie_clk_disable(imx6_pcie); > + if (imx6_pcie->phy != NULL) { Please use the more common if (imx6_pcie->phy) here. Regards, Lucas > + phy_power_off(imx6_pcie->phy); > + phy_exit(imx6_pcie->phy); > + } > + if (imx6_pcie->vpcie) > + regulator_disable(imx6_pcie->vpcie); > return ret; > } >
> -----Original Message----- > From: Lucas Stach <l.stach@pengutronix.de> > Sent: 2022年6月8日 15:35 > To: Hongxing Zhu <hongxing.zhu@nxp.com>; bhelgaas@google.com; > robh+dt@kernel.org; broonie@kernel.org; lorenzo.pieralisi@arm.com; > jingoohan1@gmail.com; festevam@gmail.com; > francesco.dolcini@toradex.com > Cc: linux-pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org; > linux-kernel@vger.kernel.org; kernel@pengutronix.de; dl-linux-imx > <linux-imx@nxp.com> > Subject: Re: [PATCH v9 6/8] PCI: imx6: Disable clocks and regulators after link is > down > > Am Freitag, dem 06.05.2022 um 09:47 +0800 schrieb Richard Zhu: > > Since i.MX PCIe doesn't support hot-plug, reduce power consumption as > > much as possible by disabling clocks and regulators and returning > > error when the link is down. > > > > Signed-off-by: Richard Zhu <hongxing.zhu@nxp.com> > > --- > > drivers/pci/controller/dwc/pci-imx6.c | 19 +++++++++++++++---- > > 1 file changed, 15 insertions(+), 4 deletions(-) > > > > diff --git a/drivers/pci/controller/dwc/pci-imx6.c > > b/drivers/pci/controller/dwc/pci-imx6.c > > index 3ce3993d5797..d122c12193a6 100644 > > --- a/drivers/pci/controller/dwc/pci-imx6.c > > +++ b/drivers/pci/controller/dwc/pci-imx6.c > > @@ -845,7 +845,9 @@ static int imx6_pcie_start_link(struct dw_pcie *pci) > > /* Start LTSSM. */ > > imx6_pcie_ltssm_enable(dev); > > > > - dw_pcie_wait_for_link(pci); > > + ret = dw_pcie_wait_for_link(pci); > > + if (ret) > > + goto err_out; > > This adds back error handling that has been intentionally removed in > f81f095e8771 ("PCI: imx6: Allow to probe when dw_pcie_wait_for_link() fails"). > While I agree that disabling the clocks and regulators is the right thing to do > when we don't manage to get a link, we should still allow the driver to probe, > so please add a "ret = 0" to this newly added non-fatal error paths. > Thanks for your review comments. There would be a long latency if the link is down and probe is finished successfully. Since the dw_pcie_wait_for_link() would be invoked twice in every driver probe and resume operation later. Each dw_pcie_wait_for_link() would consume about 90,000*10 ~ 100,000*10 u-seconds. I'm afraid that such a long latency would bring bad user experience. Here are the logs when probe is allowed when PCIe link is down: [ 55.045954][ T1835] imx6q-pcie 5f000000.pcie: PM: calling imx6_pcie_resume_noirq.742dfa074b40dca7ca925f0c49c905ec.cfi_jt+0x0/0x8 @ 1835, parent: bus@5f000000 ... [ 56.074566][ T1835] imx6q-pcie 5f000000.pcie: Phy link never came up [ 57.074816][ T1835] imx6q-pcie 5f000000.pcie: Phy link never came up ... [ 57.182300][ T1835] imx6q-pcie 5f000000.pcie: PM: imx6_pcie_resume_noirq.742dfa074b40dca7ca925f0c49c905ec.cfi_jt+0x0/0x8 returned 0 after 2136334 usecs [ 57.182347][ T1835] imx6q-pcie 5f010000.pcie: PM: calling imx6_pcie_resume_noirq.742dfa074b40dca7ca925f0c49c905ec.cfi_jt+0x0/0x8 @ 1835, parent: bus@5f000000 ... [ 58.210584][ T1835] imx6q-pcie 5f010000.pcie: Phy link never came up [ 59.210831][ T1835] imx6q-pcie 5f010000.pcie: Phy link never came up ... [ 59.318313][ T1835] imx6q-pcie 5f010000.pcie: PM: imx6_pcie_resume_noirq.742dfa074b40dca7ca925f0c49c905ec.cfi_jt+0x0/0x8 returned 0 after 2135949 usecs So, I'm prefer that it's better to let the probe failed when link is down. How do you think about that? > > > > if (pci->link_gen == 2) { > > /* Allow Gen2 mode after the link is up. */ @@ -876,12 +878,14 > @@ > > static int imx6_pcie_start_link(struct dw_pcie *pci) > > ret = imx6_pcie_wait_for_speed_change(imx6_pcie); > > if (ret) { > > dev_err(dev, "Failed to bring link up!\n"); > > - goto err_reset_phy; > > + goto err_out; > > } > > } > > > > /* Make sure link training is finished as well! */ > > - dw_pcie_wait_for_link(pci); > > + ret = dw_pcie_wait_for_link(pci); > > + if (ret) > > + goto err_out; > > } else { > > dev_info(dev, "Link: Gen2 disabled\n"); > > } > > @@ -890,11 +894,18 @@ static int imx6_pcie_start_link(struct dw_pcie > *pci) > > dev_info(dev, "Link up, Gen%i\n", tmp & PCI_EXP_LNKSTA_CLS); > > return 0; > > > > -err_reset_phy: > > +err_out: > > dev_dbg(dev, "PHY DEBUG_R0=0x%08x DEBUG_R1=0x%08x\n", > > dw_pcie_readl_dbi(pci, PCIE_PORT_DEBUG0), > > dw_pcie_readl_dbi(pci, PCIE_PORT_DEBUG1)); > > imx6_pcie_reset_phy(imx6_pcie); > > + imx6_pcie_clk_disable(imx6_pcie); > > + if (imx6_pcie->phy != NULL) { > > Please use the more common if (imx6_pcie->phy) here. > Okay. Thanks. Best Regards Richard Zhu > Regards, > Lucas > > > + phy_power_off(imx6_pcie->phy); > > + phy_exit(imx6_pcie->phy); > > + } > > + if (imx6_pcie->vpcie) > > + regulator_disable(imx6_pcie->vpcie); > > return ret; > > } > > >
On Thu, Jun 09, 2022 at 06:17:46AM +0000, Hongxing Zhu wrote: > > -----Original Message----- > > From: Lucas Stach <l.stach@pengutronix.de> > > Sent: 2022年6月8日 15:35 > > To: Hongxing Zhu <hongxing.zhu@nxp.com>; bhelgaas@google.com; > > robh+dt@kernel.org; broonie@kernel.org; lorenzo.pieralisi@arm.com; > > jingoohan1@gmail.com; festevam@gmail.com; > > francesco.dolcini@toradex.com > > Cc: linux-pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org; > > linux-kernel@vger.kernel.org; kernel@pengutronix.de; dl-linux-imx > > <linux-imx@nxp.com> > > Subject: Re: [PATCH v9 6/8] PCI: imx6: Disable clocks and regulators after link is > > down > > > > Am Freitag, dem 06.05.2022 um 09:47 +0800 schrieb Richard Zhu: > > > Since i.MX PCIe doesn't support hot-plug, reduce power consumption as > > > much as possible by disabling clocks and regulators and returning > > > error when the link is down. > > > > > > Signed-off-by: Richard Zhu <hongxing.zhu@nxp.com> > > > --- > > > drivers/pci/controller/dwc/pci-imx6.c | 19 +++++++++++++++---- > > > 1 file changed, 15 insertions(+), 4 deletions(-) > > > > > > diff --git a/drivers/pci/controller/dwc/pci-imx6.c > > > b/drivers/pci/controller/dwc/pci-imx6.c > > > index 3ce3993d5797..d122c12193a6 100644 > > > --- a/drivers/pci/controller/dwc/pci-imx6.c > > > +++ b/drivers/pci/controller/dwc/pci-imx6.c > > > @@ -845,7 +845,9 @@ static int imx6_pcie_start_link(struct dw_pcie *pci) > > > /* Start LTSSM. */ > > > imx6_pcie_ltssm_enable(dev); > > > > > > - dw_pcie_wait_for_link(pci); > > > + ret = dw_pcie_wait_for_link(pci); > > > + if (ret) > > > + goto err_out; > > > > This adds back error handling that has been intentionally removed in > > f81f095e8771 ("PCI: imx6: Allow to probe when dw_pcie_wait_for_link() fails"). > > While I agree that disabling the clocks and regulators is the right thing to do > > when we don't manage to get a link, we should still allow the driver to probe, > > so please add a "ret = 0" to this newly added non-fatal error paths. > > > Thanks for your review comments. > There would be a long latency if the link is down and probe is finished > successfully. > Since the dw_pcie_wait_for_link() would be invoked twice in every driver probe > and resume operation later. Each dw_pcie_wait_for_link() would consume about > 90,000*10 ~ 100,000*10 u-seconds. I'm afraid that such a long latency would > bring bad user experience. > > Here are the logs when probe is allowed when PCIe link is down: > [ 55.045954][ T1835] imx6q-pcie 5f000000.pcie: PM: calling imx6_pcie_resume_noirq.742dfa074b40dca7ca925f0c49c905ec.cfi_jt+0x0/0x8 @ 1835, parent: bus@5f000000 > ... > [ 56.074566][ T1835] imx6q-pcie 5f000000.pcie: Phy link never came up > [ 57.074816][ T1835] imx6q-pcie 5f000000.pcie: Phy link never came up > ... > [ 57.182300][ T1835] imx6q-pcie 5f000000.pcie: PM: imx6_pcie_resume_noirq.742dfa074b40dca7ca925f0c49c905ec.cfi_jt+0x0/0x8 returned 0 after 2136334 usecs > > [ 57.182347][ T1835] imx6q-pcie 5f010000.pcie: PM: calling imx6_pcie_resume_noirq.742dfa074b40dca7ca925f0c49c905ec.cfi_jt+0x0/0x8 @ 1835, parent: bus@5f000000 > ... > [ 58.210584][ T1835] imx6q-pcie 5f010000.pcie: Phy link never came up > [ 59.210831][ T1835] imx6q-pcie 5f010000.pcie: Phy link never came up > ... > [ 59.318313][ T1835] imx6q-pcie 5f010000.pcie: PM: imx6_pcie_resume_noirq.742dfa074b40dca7ca925f0c49c905ec.cfi_jt+0x0/0x8 returned 0 after 2135949 usecs > > So, I'm prefer that it's better to let the probe failed when link is down. > How do you think about that? I think that recently Bjorn mentioned some concern with this approach, and I agree with him. I think that the probe of the PCIe root port should not fail if the link is down. What is the reason for such a long wait in dw_pcie_wait_for_link()? Is this slowing down the resume process as a whole? Why called twice? (I'm not familiar with that part of the code) Francesco
Am Donnerstag, dem 09.06.2022 um 06:17 +0000 schrieb Hongxing Zhu: > > -----Original Message----- > > From: Lucas Stach <l.stach@pengutronix.de> > > Sent: 2022年6月8日 15:35 > > To: Hongxing Zhu <hongxing.zhu@nxp.com>; bhelgaas@google.com; > > robh+dt@kernel.org; broonie@kernel.org; lorenzo.pieralisi@arm.com; > > jingoohan1@gmail.com; festevam@gmail.com; > > francesco.dolcini@toradex.com > > Cc: linux-pci@vger.kernel.org; > > linux-arm-kernel@lists.infradead.org; > > linux-kernel@vger.kernel.org; kernel@pengutronix.de; dl-linux-imx > > <linux-imx@nxp.com> > > Subject: Re: [PATCH v9 6/8] PCI: imx6: Disable clocks and > > regulators after link is > > down > > > > Am Freitag, dem 06.05.2022 um 09:47 +0800 schrieb Richard Zhu: > > > Since i.MX PCIe doesn't support hot-plug, reduce power > > > consumption as > > > much as possible by disabling clocks and regulators and returning > > > error when the link is down. > > > > > > Signed-off-by: Richard Zhu <hongxing.zhu@nxp.com> > > > --- > > > drivers/pci/controller/dwc/pci-imx6.c | 19 +++++++++++++++---- > > > 1 file changed, 15 insertions(+), 4 deletions(-) > > > > > > diff --git a/drivers/pci/controller/dwc/pci-imx6.c > > > b/drivers/pci/controller/dwc/pci-imx6.c > > > index 3ce3993d5797..d122c12193a6 100644 > > > --- a/drivers/pci/controller/dwc/pci-imx6.c > > > +++ b/drivers/pci/controller/dwc/pci-imx6.c > > > @@ -845,7 +845,9 @@ static int imx6_pcie_start_link(struct > > > dw_pcie *pci) > > > /* Start LTSSM. */ > > > imx6_pcie_ltssm_enable(dev); > > > > > > - dw_pcie_wait_for_link(pci); > > > + ret = dw_pcie_wait_for_link(pci); > > > + if (ret) > > > + goto err_out; > > > > This adds back error handling that has been intentionally removed > > in > > f81f095e8771 ("PCI: imx6: Allow to probe when > > dw_pcie_wait_for_link() fails"). > > While I agree that disabling the clocks and regulators is the right > > thing to do > > when we don't manage to get a link, we should still allow the > > driver to probe, > > so please add a "ret = 0" to this newly added non-fatal error > > paths. > > > Thanks for your review comments. > There would be a long latency if the link is down and probe is > finished > successfully. > Since the dw_pcie_wait_for_link() would be invoked twice in every > driver probe > and resume operation later. Each dw_pcie_wait_for_link() would > consume about > 90,000*10 ~ 100,000*10 u-seconds. I'm afraid that such a long > latency would > bring bad user experience. > > Here are the logs when probe is allowed when PCIe link is down: > [ 55.045954][ T1835] imx6q-pcie 5f000000.pcie: PM: calling > imx6_pcie_resume_noirq.742dfa074b40dca7ca925f0c49c905ec.cfi_jt+0x0/0x > 8 @ 1835, parent: bus@5f000000 > ... > [ 56.074566][ T1835] imx6q-pcie 5f000000.pcie: Phy link never came > up > [ 57.074816][ T1835] imx6q-pcie 5f000000.pcie: Phy link never came > up > ... > [ 57.182300][ T1835] imx6q-pcie 5f000000.pcie: PM: > imx6_pcie_resume_noirq.742dfa074b40dca7ca925f0c49c905ec.cfi_jt+0x0/0x > 8 returned 0 after 2136334 usecs > > [ 57.182347][ T1835] imx6q-pcie 5f010000.pcie: PM: calling > imx6_pcie_resume_noirq.742dfa074b40dca7ca925f0c49c905ec.cfi_jt+0x0/0x > 8 @ 1835, parent: bus@5f000000 > ... > [ 58.210584][ T1835] imx6q-pcie 5f010000.pcie: Phy link never came > up > [ 59.210831][ T1835] imx6q-pcie 5f010000.pcie: Phy link never came > up > ... > [ 59.318313][ T1835] imx6q-pcie 5f010000.pcie: PM: > imx6_pcie_resume_noirq.742dfa074b40dca7ca925f0c49c905ec.cfi_jt+0x0/0x > 8 returned 0 after 2135949 usecs > > So, I'm prefer that it's better to let the probe failed when link is > down. > How do you think about that? It different from the behavior of other platforms, that still show the root bridge in lspci, even if the link is down. I've seen people confused by this behavior. Come to think about it: does lspci work when all the clocks are disabled? The latency in the probe path is not that relevant, as it is done asynchronous, so it doesn't stall the boot process. You have a point on the suspend/resume path however. Maybe we need to remember the link state in suspend to know if resume should even do anything useful? Regards, Lucas > > > > > > > if (pci->link_gen == 2) { > > > /* Allow Gen2 mode after the link is up. */ @@ - > > > 876,12 +878,14 > > @@ > > > static int imx6_pcie_start_link(struct dw_pcie *pci) > > > ret = > > > imx6_pcie_wait_for_speed_change(imx6_pcie); > > > if (ret) { > > > dev_err(dev, "Failed to bring > > > link up!\n"); > > > - goto err_reset_phy; > > > + goto err_out; > > > } > > > } > > > > > > /* Make sure link training is finished as well! > > > */ > > > - dw_pcie_wait_for_link(pci); > > > + ret = dw_pcie_wait_for_link(pci); > > > + if (ret) > > > + goto err_out; > > > } else { > > > dev_info(dev, "Link: Gen2 disabled\n"); > > > } > > > @@ -890,11 +894,18 @@ static int imx6_pcie_start_link(struct > > > dw_pcie > > *pci) > > > dev_info(dev, "Link up, Gen%i\n", tmp & > > > PCI_EXP_LNKSTA_CLS); > > > return 0; > > > > > > -err_reset_phy: > > > +err_out: > > > dev_dbg(dev, "PHY DEBUG_R0=0x%08x DEBUG_R1=0x%08x\n", > > > dw_pcie_readl_dbi(pci, PCIE_PORT_DEBUG0), > > > dw_pcie_readl_dbi(pci, PCIE_PORT_DEBUG1)); > > > imx6_pcie_reset_phy(imx6_pcie); > > > + imx6_pcie_clk_disable(imx6_pcie); > > > + if (imx6_pcie->phy != NULL) { > > > > Please use the more common if (imx6_pcie->phy) here. > > > Okay. Thanks. > > Best Regards > Richard Zhu > > > Regards, > > Lucas > > > > > + phy_power_off(imx6_pcie->phy); > > > + phy_exit(imx6_pcie->phy); > > > + } > > > + if (imx6_pcie->vpcie) > > > + regulator_disable(imx6_pcie->vpcie); > > > return ret; > > > } > > > > > >
> -----Original Message----- > From: Lucas Stach <l.stach@pengutronix.de> > Sent: 2022年6月9日 15:55 > To: Hongxing Zhu <hongxing.zhu@nxp.com>; bhelgaas@google.com; > robh+dt@kernel.org; broonie@kernel.org; lorenzo.pieralisi@arm.com; > jingoohan1@gmail.com; festevam@gmail.com; > francesco.dolcini@toradex.com > Cc: linux-pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org; > linux-kernel@vger.kernel.org; kernel@pengutronix.de; dl-linux-imx > <linux-imx@nxp.com> > Subject: Re: [PATCH v9 6/8] PCI: imx6: Disable clocks and regulators after link is > down > > Am Donnerstag, dem 09.06.2022 um 06:17 +0000 schrieb Hongxing Zhu: > > > -----Original Message----- > > > From: Lucas Stach <l.stach@pengutronix.de> > > > Sent: 2022年6月8日 15:35 > > > To: Hongxing Zhu <hongxing.zhu@nxp.com>; bhelgaas@google.com; > > > robh+dt@kernel.org; broonie@kernel.org; lorenzo.pieralisi@arm.com; > > > jingoohan1@gmail.com; festevam@gmail.com; > > > francesco.dolcini@toradex.com > > > Cc: linux-pci@vger.kernel.org; > > > linux-arm-kernel@lists.infradead.org; > > > linux-kernel@vger.kernel.org; kernel@pengutronix.de; dl-linux-imx > > > <linux-imx@nxp.com> > > > Subject: Re: [PATCH v9 6/8] PCI: imx6: Disable clocks and regulators > > > after link is down > > > > > > Am Freitag, dem 06.05.2022 um 09:47 +0800 schrieb Richard Zhu: > > > > Since i.MX PCIe doesn't support hot-plug, reduce power consumption > > > > as much as possible by disabling clocks and regulators and > > > > returning error when the link is down. > > > > > > > > Signed-off-by: Richard Zhu <hongxing.zhu@nxp.com> > > > > --- > > > > drivers/pci/controller/dwc/pci-imx6.c | 19 +++++++++++++++---- > > > > 1 file changed, 15 insertions(+), 4 deletions(-) > > > > > > > > diff --git a/drivers/pci/controller/dwc/pci-imx6.c > > > > b/drivers/pci/controller/dwc/pci-imx6.c > > > > index 3ce3993d5797..d122c12193a6 100644 > > > > --- a/drivers/pci/controller/dwc/pci-imx6.c > > > > +++ b/drivers/pci/controller/dwc/pci-imx6.c > > > > @@ -845,7 +845,9 @@ static int imx6_pcie_start_link(struct dw_pcie > > > > *pci) > > > > /* Start LTSSM. */ > > > > imx6_pcie_ltssm_enable(dev); > > > > > > > > - dw_pcie_wait_for_link(pci); > > > > + ret = dw_pcie_wait_for_link(pci); > > > > + if (ret) > > > > + goto err_out; > > > > > > This adds back error handling that has been intentionally removed in > > > f81f095e8771 ("PCI: imx6: Allow to probe when > > > dw_pcie_wait_for_link() fails"). > > > While I agree that disabling the clocks and regulators is the right > > > thing to do when we don't manage to get a link, we should still > > > allow the driver to probe, so please add a "ret = 0" to this newly > > > added non-fatal error paths. > > > > > Thanks for your review comments. > > There would be a long latency if the link is down and probe is > > finished > > successfully. > > Since the dw_pcie_wait_for_link() would be invoked twice in every > > driver probe > > and resume operation later. Each dw_pcie_wait_for_link() would > > consume about > > 90,000*10 ~ 100,000*10 u-seconds. I'm afraid that such a long latency > > would > > bring bad user experience. > > > > Here are the logs when probe is allowed when PCIe link is down: > > [ 55.045954][ T1835] imx6q-pcie 5f000000.pcie: PM: calling > > > imx6_pcie_resume_noirq.742dfa074b40dca7ca925f0c49c905ec.cfi_jt+0x0/0x > > 8 @ 1835, parent: bus@5f000000 > > ... > > [ 56.074566][ T1835] imx6q-pcie 5f000000.pcie: Phy link never came > > up > > [ 57.074816][ T1835] imx6q-pcie 5f000000.pcie: Phy link never came > > up > > ... > > [ 57.182300][ T1835] imx6q-pcie 5f000000.pcie: PM: > > > imx6_pcie_resume_noirq.742dfa074b40dca7ca925f0c49c905ec.cfi_jt+0x0/0x > > 8 returned 0 after 2136334 usecs > > > > [ 57.182347][ T1835] imx6q-pcie 5f010000.pcie: PM: calling > > > imx6_pcie_resume_noirq.742dfa074b40dca7ca925f0c49c905ec.cfi_jt+0x0/0x > > 8 @ 1835, parent: bus@5f000000 > > ... > > [ 58.210584][ T1835] imx6q-pcie 5f010000.pcie: Phy link never came > > up > > [ 59.210831][ T1835] imx6q-pcie 5f010000.pcie: Phy link never came > > up > > ... > > [ 59.318313][ T1835] imx6q-pcie 5f010000.pcie: PM: > > > imx6_pcie_resume_noirq.742dfa074b40dca7ca925f0c49c905ec.cfi_jt+0x0/0x > > 8 returned 0 after 2135949 usecs > > > > So, I'm prefer that it's better to let the probe failed when link is > > down. > > How do you think about that? > > It different from the behavior of other platforms, that still show the root bridge > in lspci, even if the link is down. I've seen people confused by this behavior. > Come to think about it: does lspci work when all the clocks are disabled? > > The latency in the probe path is not that relevant, as it is done asynchronous, > so it doesn't stall the boot process. You have a point on the suspend/resume > path however. Maybe we need to remember the link state in suspend to know > if resume should even do anything useful? > This sounds good. Thus, one flag is required to specify the link is up or not before suspend. If the link is down, the imx6_pcie_start_link() shouldn't invoked at all when resume back. Then we can avoid the long latency and keep probe finished when link is down. In this scenario, we can let probe successfully, although the link is down. And the last patch can be dropped, since the clocks/regulators are always on in this case. Thanks for your suggestion and inspiration. Best Regards Richard Zhu > Regards, > Lucas > > > > > > > > > > > if (pci->link_gen == 2) { > > > > /* Allow Gen2 mode after the link is up. */ @@ - > > > > 876,12 +878,14 > > > @@ > > > > static int imx6_pcie_start_link(struct dw_pcie *pci) > > > > ret = > > > > imx6_pcie_wait_for_speed_change(imx6_pcie); > > > > if (ret) { > > > > dev_err(dev, "Failed to bring link up!\n"); > > > > - goto err_reset_phy; > > > > + goto err_out; > > > > } > > > > } > > > > > > > > /* Make sure link training is finished as well! > > > > */ > > > > - dw_pcie_wait_for_link(pci); > > > > + ret = dw_pcie_wait_for_link(pci); > > > > + if (ret) > > > > + goto err_out; > > > > } else { > > > > dev_info(dev, "Link: Gen2 disabled\n"); > > > > } > > > > @@ -890,11 +894,18 @@ static int imx6_pcie_start_link(struct > > > > dw_pcie > > > *pci) > > > > dev_info(dev, "Link up, Gen%i\n", tmp & PCI_EXP_LNKSTA_CLS); > > > > return 0; > > > > > > > > -err_reset_phy: > > > > +err_out: > > > > dev_dbg(dev, "PHY DEBUG_R0=0x%08x DEBUG_R1=0x%08x\n", > > > > dw_pcie_readl_dbi(pci, PCIE_PORT_DEBUG0), > > > > dw_pcie_readl_dbi(pci, PCIE_PORT_DEBUG1)); > > > > imx6_pcie_reset_phy(imx6_pcie); > > > > + imx6_pcie_clk_disable(imx6_pcie); > > > > + if (imx6_pcie->phy != NULL) { > > > > > > Please use the more common if (imx6_pcie->phy) here. > > > > > Okay. Thanks. > > > > Best Regards > > Richard Zhu > > > > > Regards, > > > Lucas > > > > > > > + phy_power_off(imx6_pcie->phy); > > > > + phy_exit(imx6_pcie->phy); > > > > + } > > > > + if (imx6_pcie->vpcie) > > > > + regulator_disable(imx6_pcie->vpcie); > > > > return ret; > > > > } > > > > > > > > > >
> -----Original Message----- > From: Francesco Dolcini <francesco.dolcini@toradex.com> > Sent: 2022年6月9日 15:53 > To: Hongxing Zhu <hongxing.zhu@nxp.com> > Cc: Lucas Stach <l.stach@pengutronix.de>; bhelgaas@google.com; > robh+dt@kernel.org; broonie@kernel.org; lorenzo.pieralisi@arm.com; > jingoohan1@gmail.com; festevam@gmail.com; > francesco.dolcini@toradex.com; linux-pci@vger.kernel.org; > linux-arm-kernel@lists.infradead.org; linux-kernel@vger.kernel.org; > kernel@pengutronix.de; dl-linux-imx <linux-imx@nxp.com> > Subject: Re: [PATCH v9 6/8] PCI: imx6: Disable clocks and regulators after link is > down > > On Thu, Jun 09, 2022 at 06:17:46AM +0000, Hongxing Zhu wrote: > > > -----Original Message----- > > > From: Lucas Stach <l.stach@pengutronix.de> > > > Sent: 2022年6月8日 15:35 > > > To: Hongxing Zhu <hongxing.zhu@nxp.com>; bhelgaas@google.com; > > > robh+dt@kernel.org; broonie@kernel.org; lorenzo.pieralisi@arm.com; > > > jingoohan1@gmail.com; festevam@gmail.com; > > > francesco.dolcini@toradex.com > > > Cc: linux-pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org; > > > linux-kernel@vger.kernel.org; kernel@pengutronix.de; dl-linux-imx > > > <linux-imx@nxp.com> > > > Subject: Re: [PATCH v9 6/8] PCI: imx6: Disable clocks and regulators > > > after link is down > > > > > > Am Freitag, dem 06.05.2022 um 09:47 +0800 schrieb Richard Zhu: > > > > Since i.MX PCIe doesn't support hot-plug, reduce power consumption > > > > as much as possible by disabling clocks and regulators and > > > > returning error when the link is down. > > > > > > > > Signed-off-by: Richard Zhu <hongxing.zhu@nxp.com> > > > > --- > > > > drivers/pci/controller/dwc/pci-imx6.c | 19 +++++++++++++++---- > > > > 1 file changed, 15 insertions(+), 4 deletions(-) > > > > > > > > diff --git a/drivers/pci/controller/dwc/pci-imx6.c > > > > b/drivers/pci/controller/dwc/pci-imx6.c > > > > index 3ce3993d5797..d122c12193a6 100644 > > > > --- a/drivers/pci/controller/dwc/pci-imx6.c > > > > +++ b/drivers/pci/controller/dwc/pci-imx6.c > > > > @@ -845,7 +845,9 @@ static int imx6_pcie_start_link(struct dw_pcie > *pci) > > > > /* Start LTSSM. */ > > > > imx6_pcie_ltssm_enable(dev); > > > > > > > > - dw_pcie_wait_for_link(pci); > > > > + ret = dw_pcie_wait_for_link(pci); > > > > + if (ret) > > > > + goto err_out; > > > > > > This adds back error handling that has been intentionally removed in > > > f81f095e8771 ("PCI: imx6: Allow to probe when dw_pcie_wait_for_link() > fails"). > > > While I agree that disabling the clocks and regulators is the right > > > thing to do when we don't manage to get a link, we should still > > > allow the driver to probe, so please add a "ret = 0" to this newly added > non-fatal error paths. > > > > > Thanks for your review comments. > > There would be a long latency if the link is down and probe is > > finished successfully. > > Since the dw_pcie_wait_for_link() would be invoked twice in every > > driver probe and resume operation later. Each dw_pcie_wait_for_link() > > would consume about > > 90,000*10 ~ 100,000*10 u-seconds. I'm afraid that such a long latency > > would bring bad user experience. > > > > Here are the logs when probe is allowed when PCIe link is down: > > [ 55.045954][ T1835] imx6q-pcie 5f000000.pcie: PM: calling > imx6_pcie_resume_noirq.742dfa074b40dca7ca925f0c49c905ec.cfi_jt+0x0/0x > 8 @ 1835, parent: bus@5f000000 > > ... > > [ 56.074566][ T1835] imx6q-pcie 5f000000.pcie: Phy link never came up > > [ 57.074816][ T1835] imx6q-pcie 5f000000.pcie: Phy link never came up > > ... > > [ 57.182300][ T1835] imx6q-pcie 5f000000.pcie: PM: > imx6_pcie_resume_noirq.742dfa074b40dca7ca925f0c49c905ec.cfi_jt+0x0/0x > 8 returned 0 after 2136334 usecs > > > > [ 57.182347][ T1835] imx6q-pcie 5f010000.pcie: PM: calling > imx6_pcie_resume_noirq.742dfa074b40dca7ca925f0c49c905ec.cfi_jt+0x0/0x > 8 @ 1835, parent: bus@5f000000 > > ... > > [ 58.210584][ T1835] imx6q-pcie 5f010000.pcie: Phy link never came up > > [ 59.210831][ T1835] imx6q-pcie 5f010000.pcie: Phy link never came up > > ... > > [ 59.318313][ T1835] imx6q-pcie 5f010000.pcie: PM: > imx6_pcie_resume_noirq.742dfa074b40dca7ca925f0c49c905ec.cfi_jt+0x0/0x > 8 returned 0 after 2135949 usecs > > > > So, I'm prefer that it's better to let the probe failed when link is down. > > How do you think about that? > > I think that recently Bjorn mentioned some concern with this approach, and I > agree with him. > I think that the probe of the PCIe root port should not fail if the link is down. > > What is the reason for such a long wait in dw_pcie_wait_for_link()? Is this > slowing down the resume process as a whole? Why called twice? (I'm not > familiar with that part of the code) > Thanks for your concerns. To avoid a corner link down issue, iMX PCIe driver force link up GEN1 speed firstly, then try to link up the highest speed later with one speed exchange. So, the dw_pcie_wait_for_link() would be invoked twice in iMX PCIe driver. Lucas and I try to figure out one method to avoid the long latency when resume back from suspend mode when link is down and probe is succeed. Best Regards Richard Zhu > Francesco
diff --git a/drivers/pci/controller/dwc/pci-imx6.c b/drivers/pci/controller/dwc/pci-imx6.c index 3ce3993d5797..d122c12193a6 100644 --- a/drivers/pci/controller/dwc/pci-imx6.c +++ b/drivers/pci/controller/dwc/pci-imx6.c @@ -845,7 +845,9 @@ static int imx6_pcie_start_link(struct dw_pcie *pci) /* Start LTSSM. */ imx6_pcie_ltssm_enable(dev); - dw_pcie_wait_for_link(pci); + ret = dw_pcie_wait_for_link(pci); + if (ret) + goto err_out; if (pci->link_gen == 2) { /* Allow Gen2 mode after the link is up. */ @@ -876,12 +878,14 @@ static int imx6_pcie_start_link(struct dw_pcie *pci) ret = imx6_pcie_wait_for_speed_change(imx6_pcie); if (ret) { dev_err(dev, "Failed to bring link up!\n"); - goto err_reset_phy; + goto err_out; } } /* Make sure link training is finished as well! */ - dw_pcie_wait_for_link(pci); + ret = dw_pcie_wait_for_link(pci); + if (ret) + goto err_out; } else { dev_info(dev, "Link: Gen2 disabled\n"); } @@ -890,11 +894,18 @@ static int imx6_pcie_start_link(struct dw_pcie *pci) dev_info(dev, "Link up, Gen%i\n", tmp & PCI_EXP_LNKSTA_CLS); return 0; -err_reset_phy: +err_out: dev_dbg(dev, "PHY DEBUG_R0=0x%08x DEBUG_R1=0x%08x\n", dw_pcie_readl_dbi(pci, PCIE_PORT_DEBUG0), dw_pcie_readl_dbi(pci, PCIE_PORT_DEBUG1)); imx6_pcie_reset_phy(imx6_pcie); + imx6_pcie_clk_disable(imx6_pcie); + if (imx6_pcie->phy != NULL) { + phy_power_off(imx6_pcie->phy); + phy_exit(imx6_pcie->phy); + } + if (imx6_pcie->vpcie) + regulator_disable(imx6_pcie->vpcie); return ret; }
Since i.MX PCIe doesn't support hot-plug, reduce power consumption as much as possible by disabling clocks and regulators and returning error when the link is down. Signed-off-by: Richard Zhu <hongxing.zhu@nxp.com> --- drivers/pci/controller/dwc/pci-imx6.c | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-)