| Message ID | 20211122104156.518063-1-maz@kernel.org (mailing list archive) |
|---|---|
| State | Superseded |
| Delegated to: | Lorenzo Pieralisi |
| Headers | show |
| Series | [v2] PCI: apple: Follow the PCIe specifications when resetting the port | expand |
On Monday 22 November 2021 10:41:56 Marc Zyngier wrote: > While the Apple PCIe driver works correctly when directly booted > from the firmware, it fails to initialise when the kernel is booted > from a bootloader using PCIe such as u-boot. > > That's beacuse we're missing a proper reset of the port (we only > clear the reset, but never assert it). > > The PCIe spec requirements are two-fold: > > - #PERST must be asserted before setting up the clocks, and > stay asserted for at least 100us (Tperst-clk). > > - Once #PERST is deasserted, the OS must wait for at least 100ms > "from the end of a Conventional Reset" before we can start talking > to the devices > > Implementing this results in a booting system. > > Fixes: 1e33888fbe44 ("PCI: apple: Add initial hardware bring-up") > Signed-off-by: Marc Zyngier <maz@kernel.org> > Cc: Alyssa Rosenzweig <alyssa@rosenzweig.io> > Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> > Cc: Bjorn Helgaas <bhelgaas@google.com> > Cc: Pali Rohár <pali@kernel.org> Looks good, but see comment below. Acked-by: Pali Rohár <pali@kernel.org> > --- > drivers/pci/controller/pcie-apple.c | 10 ++++++++++ > 1 file changed, 10 insertions(+) > > diff --git a/drivers/pci/controller/pcie-apple.c b/drivers/pci/controller/pcie-apple.c > index 1bf4d75b61be..957960a733c4 100644 > --- a/drivers/pci/controller/pcie-apple.c > +++ b/drivers/pci/controller/pcie-apple.c > @@ -539,13 +539,23 @@ static int apple_pcie_setup_port(struct apple_pcie *pcie, > > rmw_set(PORT_APPCLK_EN, port->base + PORT_APPCLK); > > + /* Engage #PERST before setting up the clock */ > + gpiod_set_value(reset, 0); > + > ret = apple_pcie_setup_refclk(pcie, port); > if (ret < 0) > return ret; > > + /* The minimal Tperst-clk value is 100us (PCIe CMS r2.0, 2.6.2) */ > + usleep_range(100, 200); > + > + /* Deassert #PERST */ > rmw_set(PORT_PERST_OFF, port->base + PORT_PERST); > gpiod_set_value(reset, 1); + Luca Just one comment. PERST# (PCIe Reset) is active-low signal. De-asserting means to really set value to 1. But there was a discussion that de-asserting should be done by call: gpiod_set_value(reset, 0); https://lore.kernel.org/linux-pci/51be082a-ff10-8a19-5648-f279aabcac51@lucaceresoli.net/ Could we make this new pcie-apple.c driver to use gpiod_set_value(reset, 0) for de-asserting, like in other drivers? > > + /* Wait for 100ms after #PERST deassertion (PCIe r2.0, 6.6.1) */ > + msleep(100); > + > ret = readl_relaxed_poll_timeout(port->base + PORT_STATUS, stat, > stat & PORT_STATUS_READY, 100, 250000); > if (ret < 0) { > -- > 2.30.2 >
On Mon, 22 Nov 2021 12:03:47 +0000, Pali Rohár <pali@kernel.org> wrote: > > On Monday 22 November 2021 10:41:56 Marc Zyngier wrote: > > While the Apple PCIe driver works correctly when directly booted > > from the firmware, it fails to initialise when the kernel is booted > > from a bootloader using PCIe such as u-boot. > > > > That's beacuse we're missing a proper reset of the port (we only > > clear the reset, but never assert it). > > > > The PCIe spec requirements are two-fold: > > > > - #PERST must be asserted before setting up the clocks, and > > stay asserted for at least 100us (Tperst-clk). > > > > - Once #PERST is deasserted, the OS must wait for at least 100ms > > "from the end of a Conventional Reset" before we can start talking > > to the devices > > > > Implementing this results in a booting system. > > > > Fixes: 1e33888fbe44 ("PCI: apple: Add initial hardware bring-up") > > Signed-off-by: Marc Zyngier <maz@kernel.org> > > Cc: Alyssa Rosenzweig <alyssa@rosenzweig.io> > > Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> > > Cc: Bjorn Helgaas <bhelgaas@google.com> > > Cc: Pali Rohár <pali@kernel.org> > > Looks good, but see comment below. > > Acked-by: Pali Rohár <pali@kernel.org> Thanks for that. > > > --- > > drivers/pci/controller/pcie-apple.c | 10 ++++++++++ > > 1 file changed, 10 insertions(+) > > > > diff --git a/drivers/pci/controller/pcie-apple.c b/drivers/pci/controller/pcie-apple.c > > index 1bf4d75b61be..957960a733c4 100644 > > --- a/drivers/pci/controller/pcie-apple.c > > +++ b/drivers/pci/controller/pcie-apple.c > > @@ -539,13 +539,23 @@ static int apple_pcie_setup_port(struct apple_pcie *pcie, > > > > rmw_set(PORT_APPCLK_EN, port->base + PORT_APPCLK); > > > > + /* Engage #PERST before setting up the clock */ > > + gpiod_set_value(reset, 0); > > + > > ret = apple_pcie_setup_refclk(pcie, port); > > if (ret < 0) > > return ret; > > > > + /* The minimal Tperst-clk value is 100us (PCIe CMS r2.0, 2.6.2) */ > > + usleep_range(100, 200); > > + > > + /* Deassert #PERST */ > > rmw_set(PORT_PERST_OFF, port->base + PORT_PERST); > > gpiod_set_value(reset, 1); > > + Luca > > Just one comment. PERST# (PCIe Reset) is active-low signal. De-asserting > means to really set value to 1. > > But there was a discussion that de-asserting should be done by call: > gpiod_set_value(reset, 0); > > https://lore.kernel.org/linux-pci/51be082a-ff10-8a19-5648-f279aabcac51@lucaceresoli.net/ > > Could we make this new pcie-apple.c driver to use gpiod_set_value(reset, 0) > for de-asserting, like in other drivers? I guess it depends whether you care about the assertion or the signal itself. I think we may have a bug in the way the GPIOs are handled at the moment, as it makes no difference whether I register the GPIO are active high or active low... I guess that will be yet another thing to debug, but in the meantime we have a reliable reset. Thanks, M.
Hi Marc, On 22/11/21 15:43, Marc Zyngier wrote: > On Mon, 22 Nov 2021 12:03:47 +0000, > Pali Rohár <pali@kernel.org> wrote: >> >> On Monday 22 November 2021 10:41:56 Marc Zyngier wrote: >>> While the Apple PCIe driver works correctly when directly booted >>> from the firmware, it fails to initialise when the kernel is booted >>> from a bootloader using PCIe such as u-boot. >>> >>> That's beacuse we're missing a proper reset of the port (we only >>> clear the reset, but never assert it). >>> >>> The PCIe spec requirements are two-fold: >>> >>> - #PERST must be asserted before setting up the clocks, and >>> stay asserted for at least 100us (Tperst-clk). >>> >>> - Once #PERST is deasserted, the OS must wait for at least 100ms >>> "from the end of a Conventional Reset" before we can start talking >>> to the devices >>> >>> Implementing this results in a booting system. >>> >>> Fixes: 1e33888fbe44 ("PCI: apple: Add initial hardware bring-up") >>> Signed-off-by: Marc Zyngier <maz@kernel.org> >>> Cc: Alyssa Rosenzweig <alyssa@rosenzweig.io> >>> Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> >>> Cc: Bjorn Helgaas <bhelgaas@google.com> >>> Cc: Pali Rohár <pali@kernel.org> >> >> Looks good, but see comment below. >> >> Acked-by: Pali Rohár <pali@kernel.org> > > Thanks for that. > >> >>> --- >>> drivers/pci/controller/pcie-apple.c | 10 ++++++++++ >>> 1 file changed, 10 insertions(+) >>> >>> diff --git a/drivers/pci/controller/pcie-apple.c b/drivers/pci/controller/pcie-apple.c >>> index 1bf4d75b61be..957960a733c4 100644 >>> --- a/drivers/pci/controller/pcie-apple.c >>> +++ b/drivers/pci/controller/pcie-apple.c >>> @@ -539,13 +539,23 @@ static int apple_pcie_setup_port(struct apple_pcie *pcie, >>> >>> rmw_set(PORT_APPCLK_EN, port->base + PORT_APPCLK); >>> >>> + /* Engage #PERST before setting up the clock */ >>> + gpiod_set_value(reset, 0); >>> + >>> ret = apple_pcie_setup_refclk(pcie, port); >>> if (ret < 0) >>> return ret; >>> >>> + /* The minimal Tperst-clk value is 100us (PCIe CMS r2.0, 2.6.2) */ >>> + usleep_range(100, 200); >>> + >>> + /* Deassert #PERST */ >>> rmw_set(PORT_PERST_OFF, port->base + PORT_PERST); >>> gpiod_set_value(reset, 1); >> >> + Luca >> >> Just one comment. PERST# (PCIe Reset) is active-low signal. De-asserting >> means to really set value to 1. >> >> But there was a discussion that de-asserting should be done by call: >> gpiod_set_value(reset, 0); >> >> https://lore.kernel.org/linux-pci/51be082a-ff10-8a19-5648-f279aabcac51@lucaceresoli.net/ >> >> Could we make this new pcie-apple.c driver to use gpiod_set_value(reset, 0) >> for de-asserting, like in other drivers? I agree it should be done right from the beginning since this is a new driver. Fixing it later is a painful process. > I guess it depends whether you care about the assertion or the signal > itself. I think we may have a bug in the way the GPIOs are handled at > the moment, as it makes no difference whether I register the GPIO are > active high or active low... > > I guess that will be yet another thing to debug, but in the meantime > we have a reliable reset. Strange, in my case the "active low" pin polarity is correctly picked up from device tree by the gpiolib code, thus using gpio_set_value(gpiod, 1) asserts the pin as it should, resulting in an electrically low pin.
> Date: Mon, 22 Nov 2021 14:43:37 +0000 > From: Marc Zyngier <maz@kernel.org> > > On Mon, 22 Nov 2021 12:03:47 +0000, > Pali Rohár <pali@kernel.org> wrote: > > > > On Monday 22 November 2021 10:41:56 Marc Zyngier wrote: > > > While the Apple PCIe driver works correctly when directly booted > > > from the firmware, it fails to initialise when the kernel is booted > > > from a bootloader using PCIe such as u-boot. > > > > > > That's beacuse we're missing a proper reset of the port (we only > > > clear the reset, but never assert it). > > > > > > The PCIe spec requirements are two-fold: > > > > > > - #PERST must be asserted before setting up the clocks, and > > > stay asserted for at least 100us (Tperst-clk). > > > > > > - Once #PERST is deasserted, the OS must wait for at least 100ms > > > "from the end of a Conventional Reset" before we can start talking > > > to the devices > > > > > > Implementing this results in a booting system. > > > > > > Fixes: 1e33888fbe44 ("PCI: apple: Add initial hardware bring-up") > > > Signed-off-by: Marc Zyngier <maz@kernel.org> > > > Cc: Alyssa Rosenzweig <alyssa@rosenzweig.io> > > > Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> > > > Cc: Bjorn Helgaas <bhelgaas@google.com> > > > Cc: Pali Rohár <pali@kernel.org> > > > > Looks good, but see comment below. > > > > Acked-by: Pali Rohár <pali@kernel.org> > > Thanks for that. > > > > > > --- > > > drivers/pci/controller/pcie-apple.c | 10 ++++++++++ > > > 1 file changed, 10 insertions(+) > > > > > > diff --git a/drivers/pci/controller/pcie-apple.c b/drivers/pci/controller/pcie-apple.c > > > index 1bf4d75b61be..957960a733c4 100644 > > > --- a/drivers/pci/controller/pcie-apple.c > > > +++ b/drivers/pci/controller/pcie-apple.c > > > @@ -539,13 +539,23 @@ static int apple_pcie_setup_port(struct apple_pcie *pcie, > > > > > > rmw_set(PORT_APPCLK_EN, port->base + PORT_APPCLK); > > > > > > + /* Engage #PERST before setting up the clock */ > > > + gpiod_set_value(reset, 0); > > > + > > > ret = apple_pcie_setup_refclk(pcie, port); > > > if (ret < 0) > > > return ret; > > > > > > + /* The minimal Tperst-clk value is 100us (PCIe CMS r2.0, 2.6.2) */ > > > + usleep_range(100, 200); > > > + > > > + /* Deassert #PERST */ > > > rmw_set(PORT_PERST_OFF, port->base + PORT_PERST); > > > gpiod_set_value(reset, 1); > > > > + Luca > > > > Just one comment. PERST# (PCIe Reset) is active-low signal. De-asserting > > means to really set value to 1. > > > > But there was a discussion that de-asserting should be done by call: > > gpiod_set_value(reset, 0); > > > > https://lore.kernel.org/linux-pci/51be082a-ff10-8a19-5648-f279aabcac51@lucaceresoli.net/ > > > > Could we make this new pcie-apple.c driver to use gpiod_set_value(reset, 0) > > for de-asserting, like in other drivers? > > I guess it depends whether you care about the assertion or the signal > itself. I think we may have a bug in the way the GPIOs are handled at > the moment, as it makes no difference whether I register the GPIO are > active high or active low... That's unfortunate. But maybe that's an opportunity to fix the devicetree to use GPIO_ACTIVE_LOW for these GPIOs?
Luca, On Mon, 22 Nov 2021 21:32:15 +0000, Luca Ceresoli <luca@lucaceresoli.net> wrote: > > >> Just one comment. PERST# (PCIe Reset) is active-low signal. De-asserting > >> means to really set value to 1. > >> > >> But there was a discussion that de-asserting should be done by call: > >> gpiod_set_value(reset, 0); > >> > >> https://lore.kernel.org/linux-pci/51be082a-ff10-8a19-5648-f279aabcac51@lucaceresoli.net/ > >> > >> Could we make this new pcie-apple.c driver to use gpiod_set_value(reset, 0) > >> for de-asserting, like in other drivers? > > I agree it should be done right from the beginning since this is a new > driver. Fixing it later is a painful process. No more painful than anything else. At this stage, using a positive or negative polarity is immaterial, as there is no core infrastructure making any use of this behaviour (every single driver must reinvent its own square wheel). If such an infrastructure existed, that'd indeed be a requirement. For now, this is merely a convention. > > I guess it depends whether you care about the assertion or the signal > > itself. I think we may have a bug in the way the GPIOs are handled at > > the moment, as it makes no difference whether I register the GPIO are > > active high or active low... > > > > I guess that will be yet another thing to debug, but in the meantime > > we have a reliable reset. > > Strange, in my case the "active low" pin polarity is correctly picked up > from device tree by the gpiolib code, thus using gpio_set_value(gpiod, > 1) asserts the pin as it should, resulting in an electrically low pin. As I said, this looks like a bug, probably in the M1 DT. I'll try to look into it when I get the time. M.
On Mon, 22 Nov 2021 21:50:48 +0000, Mark Kettenis <mark.kettenis@xs4all.nl> wrote: > > > Date: Mon, 22 Nov 2021 14:43:37 +0000 > > From: Marc Zyngier <maz@kernel.org> > > > > On Mon, 22 Nov 2021 12:03:47 +0000, > > Pali Rohár <pali@kernel.org> wrote: > > > > > > On Monday 22 November 2021 10:41:56 Marc Zyngier wrote: > > > > While the Apple PCIe driver works correctly when directly booted > > > > from the firmware, it fails to initialise when the kernel is booted > > > > from a bootloader using PCIe such as u-boot. > > > > > > > > That's beacuse we're missing a proper reset of the port (we only > > > > clear the reset, but never assert it). > > > > > > > > The PCIe spec requirements are two-fold: > > > > > > > > - #PERST must be asserted before setting up the clocks, and > > > > stay asserted for at least 100us (Tperst-clk). > > > > > > > > - Once #PERST is deasserted, the OS must wait for at least 100ms > > > > "from the end of a Conventional Reset" before we can start talking > > > > to the devices > > > > > > > > Implementing this results in a booting system. > > > > > > > > Fixes: 1e33888fbe44 ("PCI: apple: Add initial hardware bring-up") > > > > Signed-off-by: Marc Zyngier <maz@kernel.org> > > > > Cc: Alyssa Rosenzweig <alyssa@rosenzweig.io> > > > > Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> > > > > Cc: Bjorn Helgaas <bhelgaas@google.com> > > > > Cc: Pali Rohár <pali@kernel.org> > > > > > > Looks good, but see comment below. > > > > > > Acked-by: Pali Rohár <pali@kernel.org> > > > > Thanks for that. > > > > > > > > > --- > > > > drivers/pci/controller/pcie-apple.c | 10 ++++++++++ > > > > 1 file changed, 10 insertions(+) > > > > > > > > diff --git a/drivers/pci/controller/pcie-apple.c b/drivers/pci/controller/pcie-apple.c > > > > index 1bf4d75b61be..957960a733c4 100644 > > > > --- a/drivers/pci/controller/pcie-apple.c > > > > +++ b/drivers/pci/controller/pcie-apple.c > > > > @@ -539,13 +539,23 @@ static int apple_pcie_setup_port(struct apple_pcie *pcie, > > > > > > > > rmw_set(PORT_APPCLK_EN, port->base + PORT_APPCLK); > > > > > > > > + /* Engage #PERST before setting up the clock */ > > > > + gpiod_set_value(reset, 0); > > > > + > > > > ret = apple_pcie_setup_refclk(pcie, port); > > > > if (ret < 0) > > > > return ret; > > > > > > > > + /* The minimal Tperst-clk value is 100us (PCIe CMS r2.0, 2.6.2) */ > > > > + usleep_range(100, 200); > > > > + > > > > + /* Deassert #PERST */ > > > > rmw_set(PORT_PERST_OFF, port->base + PORT_PERST); > > > > gpiod_set_value(reset, 1); > > > > > > + Luca > > > > > > Just one comment. PERST# (PCIe Reset) is active-low signal. De-asserting > > > means to really set value to 1. > > > > > > But there was a discussion that de-asserting should be done by call: > > > gpiod_set_value(reset, 0); > > > > > > https://lore.kernel.org/linux-pci/51be082a-ff10-8a19-5648-f279aabcac51@lucaceresoli.net/ > > > > > > Could we make this new pcie-apple.c driver to use gpiod_set_value(reset, 0) > > > for de-asserting, like in other drivers? > > > > I guess it depends whether you care about the assertion or the signal > > itself. I think we may have a bug in the way the GPIOs are handled at > > the moment, as it makes no difference whether I register the GPIO are > > active high or active low... > > That's unfortunate. But maybe that's an opportunity to fix the > devicetree to use GPIO_ACTIVE_LOW for these GPIOs? Indeed. The following hack does the right thing, and I can then reverse the polarity of the reset in the Linux driver. Of course, it breaks u-boot at the same time (and I suspect OpenBSD would be equally affected). So if we are going down that road, we may need a flag day where all the moving parts change. I don't really mind not being able to boot older kernels, but this goes beyond Linux at this point. Thanks, M. diff --git a/arch/arm64/boot/dts/apple/t8103.dtsi b/arch/arm64/boot/dts/apple/t8103.dtsi index d2e9afde3729..cad1ab920304 100644 --- a/arch/arm64/boot/dts/apple/t8103.dtsi +++ b/arch/arm64/boot/dts/apple/t8103.dtsi @@ -10,6 +10,7 @@ #include <dt-bindings/interrupt-controller/apple-aic.h> #include <dt-bindings/interrupt-controller/irq.h> #include <dt-bindings/pinctrl/apple.h> +#include <dt-bindings/gpio/gpio.h> / { compatible = "apple,t8103", "apple,arm-platform"; @@ -293,7 +294,7 @@ pcie0: pcie@690000000 { port00: pci@0,0 { device_type = "pci"; reg = <0x0 0x0 0x0 0x0 0x0>; - reset-gpios = <&pinctrl_ap 152 0>; + reset-gpios = <&pinctrl_ap 152 GPIO_ACTIVE_LOW>; max-link-speed = <2>; #address-cells = <3>; @@ -313,7 +314,7 @@ port00: pci@0,0 { port01: pci@1,0 { device_type = "pci"; reg = <0x800 0x0 0x0 0x0 0x0>; - reset-gpios = <&pinctrl_ap 153 0>; + reset-gpios = <&pinctrl_ap 153 GPIO_ACTIVE_LOW>; max-link-speed = <2>; #address-cells = <3>; @@ -333,7 +334,7 @@ port01: pci@1,0 { port02: pci@2,0 { device_type = "pci"; reg = <0x1000 0x0 0x0 0x0 0x0>; - reset-gpios = <&pinctrl_ap 33 0>; + reset-gpios = <&pinctrl_ap 33 GPIO_ACTIVE_LOW>; max-link-speed = <1>; #address-cells = <3>;
On Tuesday 23 November 2021 12:24:10 Marc Zyngier wrote: > On Mon, 22 Nov 2021 21:50:48 +0000, > Mark Kettenis <mark.kettenis@xs4all.nl> wrote: > > > > > Date: Mon, 22 Nov 2021 14:43:37 +0000 > > > From: Marc Zyngier <maz@kernel.org> > > > > > > On Mon, 22 Nov 2021 12:03:47 +0000, > > > Pali Rohár <pali@kernel.org> wrote: > > > > > > > > On Monday 22 November 2021 10:41:56 Marc Zyngier wrote: > > > > > While the Apple PCIe driver works correctly when directly booted > > > > > from the firmware, it fails to initialise when the kernel is booted > > > > > from a bootloader using PCIe such as u-boot. > > > > > > > > > > That's beacuse we're missing a proper reset of the port (we only > > > > > clear the reset, but never assert it). > > > > > > > > > > The PCIe spec requirements are two-fold: > > > > > > > > > > - #PERST must be asserted before setting up the clocks, and > > > > > stay asserted for at least 100us (Tperst-clk). > > > > > > > > > > - Once #PERST is deasserted, the OS must wait for at least 100ms > > > > > "from the end of a Conventional Reset" before we can start talking > > > > > to the devices > > > > > > > > > > Implementing this results in a booting system. > > > > > > > > > > Fixes: 1e33888fbe44 ("PCI: apple: Add initial hardware bring-up") > > > > > Signed-off-by: Marc Zyngier <maz@kernel.org> > > > > > Cc: Alyssa Rosenzweig <alyssa@rosenzweig.io> > > > > > Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> > > > > > Cc: Bjorn Helgaas <bhelgaas@google.com> > > > > > Cc: Pali Rohár <pali@kernel.org> > > > > > > > > Looks good, but see comment below. > > > > > > > > Acked-by: Pali Rohár <pali@kernel.org> > > > > > > Thanks for that. > > > > > > > > > > > > --- > > > > > drivers/pci/controller/pcie-apple.c | 10 ++++++++++ > > > > > 1 file changed, 10 insertions(+) > > > > > > > > > > diff --git a/drivers/pci/controller/pcie-apple.c b/drivers/pci/controller/pcie-apple.c > > > > > index 1bf4d75b61be..957960a733c4 100644 > > > > > --- a/drivers/pci/controller/pcie-apple.c > > > > > +++ b/drivers/pci/controller/pcie-apple.c > > > > > @@ -539,13 +539,23 @@ static int apple_pcie_setup_port(struct apple_pcie *pcie, > > > > > > > > > > rmw_set(PORT_APPCLK_EN, port->base + PORT_APPCLK); > > > > > > > > > > + /* Engage #PERST before setting up the clock */ > > > > > + gpiod_set_value(reset, 0); > > > > > + > > > > > ret = apple_pcie_setup_refclk(pcie, port); > > > > > if (ret < 0) > > > > > return ret; > > > > > > > > > > + /* The minimal Tperst-clk value is 100us (PCIe CMS r2.0, 2.6.2) */ > > > > > + usleep_range(100, 200); > > > > > + > > > > > + /* Deassert #PERST */ > > > > > rmw_set(PORT_PERST_OFF, port->base + PORT_PERST); > > > > > gpiod_set_value(reset, 1); > > > > > > > > + Luca > > > > > > > > Just one comment. PERST# (PCIe Reset) is active-low signal. De-asserting > > > > means to really set value to 1. > > > > > > > > But there was a discussion that de-asserting should be done by call: > > > > gpiod_set_value(reset, 0); > > > > > > > > https://lore.kernel.org/linux-pci/51be082a-ff10-8a19-5648-f279aabcac51@lucaceresoli.net/ > > > > > > > > Could we make this new pcie-apple.c driver to use gpiod_set_value(reset, 0) > > > > for de-asserting, like in other drivers? > > > > > > I guess it depends whether you care about the assertion or the signal > > > itself. I think we may have a bug in the way the GPIOs are handled at > > > the moment, as it makes no difference whether I register the GPIO are > > > active high or active low... > > > > That's unfortunate. But maybe that's an opportunity to fix the > > devicetree to use GPIO_ACTIVE_LOW for these GPIOs? > > Indeed. The following hack does the right thing, and I can then > reverse the polarity of the reset in the Linux driver. Of course, it > breaks u-boot at the same time (and I suspect OpenBSD would be equally > affected). U-Boot has its own copy of DTS files, right? So it can be fixed in U-Boot driver together with U-Boot DTS file at the same time. > So if we are going down that road, we may need a flag day where all > the moving parts change. I don't really mind not being able to boot > older kernels, but this goes beyond Linux at this point. > > Thanks, > > M. > > diff --git a/arch/arm64/boot/dts/apple/t8103.dtsi b/arch/arm64/boot/dts/apple/t8103.dtsi > index d2e9afde3729..cad1ab920304 100644 > --- a/arch/arm64/boot/dts/apple/t8103.dtsi > +++ b/arch/arm64/boot/dts/apple/t8103.dtsi > @@ -10,6 +10,7 @@ > #include <dt-bindings/interrupt-controller/apple-aic.h> > #include <dt-bindings/interrupt-controller/irq.h> > #include <dt-bindings/pinctrl/apple.h> > +#include <dt-bindings/gpio/gpio.h> > > / { > compatible = "apple,t8103", "apple,arm-platform"; > @@ -293,7 +294,7 @@ pcie0: pcie@690000000 { > port00: pci@0,0 { > device_type = "pci"; > reg = <0x0 0x0 0x0 0x0 0x0>; > - reset-gpios = <&pinctrl_ap 152 0>; > + reset-gpios = <&pinctrl_ap 152 GPIO_ACTIVE_LOW>; > max-link-speed = <2>; > > #address-cells = <3>; > @@ -313,7 +314,7 @@ port00: pci@0,0 { > port01: pci@1,0 { > device_type = "pci"; > reg = <0x800 0x0 0x0 0x0 0x0>; > - reset-gpios = <&pinctrl_ap 153 0>; > + reset-gpios = <&pinctrl_ap 153 GPIO_ACTIVE_LOW>; > max-link-speed = <2>; > > #address-cells = <3>; > @@ -333,7 +334,7 @@ port01: pci@1,0 { > port02: pci@2,0 { > device_type = "pci"; > reg = <0x1000 0x0 0x0 0x0 0x0>; > - reset-gpios = <&pinctrl_ap 33 0>; > + reset-gpios = <&pinctrl_ap 33 GPIO_ACTIVE_LOW>; > max-link-speed = <1>; > > #address-cells = <3>; > > -- > Without deviation from the norm, progress is not possible.
> Date: Tue, 23 Nov 2021 08:48:50 +0000 > From: Marc Zyngier <maz@kernel.org> > > Luca, > > On Mon, 22 Nov 2021 21:32:15 +0000, > Luca Ceresoli <luca@lucaceresoli.net> wrote: > > > > >> Just one comment. PERST# (PCIe Reset) is active-low signal. De-asserting > > >> means to really set value to 1. > > >> > > >> But there was a discussion that de-asserting should be done by call: > > >> gpiod_set_value(reset, 0); > > >> > > >> https://lore.kernel.org/linux-pci/51be082a-ff10-8a19-5648-f279aabcac51@lucaceresoli.net/ > > >> > > >> Could we make this new pcie-apple.c driver to use gpiod_set_value(reset, 0) > > >> for de-asserting, like in other drivers? > > > > I agree it should be done right from the beginning since this is a new > > driver. Fixing it later is a painful process. > > No more painful than anything else. At this stage, using a positive or > negative polarity is immaterial, as there is no core infrastructure > making any use of this behaviour (every single driver must reinvent > its own square wheel). If such an infrastructure existed, that'd > indeed be a requirement. For now, this is merely a convention. > > > > I guess it depends whether you care about the assertion or the signal > > > itself. I think we may have a bug in the way the GPIOs are handled at > > > the moment, as it makes no difference whether I register the GPIO are > > > active high or active low... > > > > > > I guess that will be yet another thing to debug, but in the meantime > > > we have a reliable reset. > > > > Strange, in my case the "active low" pin polarity is correctly picked up > > from device tree by the gpiolib code, thus using gpio_set_value(gpiod, > > 1) asserts the pin as it should, resulting in an electrically low pin. > > As I said, this looks like a bug, probably in the M1 DT. I'll try to > look into it when I get the time. So the diff below is what the changes look like for U-Boot. The U-Boot Apple PCIe driver has not been submitted upstream yet, so making this change is no problem. diff --git a/arch/arm/dts/t8103-j274.dts b/arch/arm/dts/t8103-j274.dts index aef1ae29b6..3777337033 100644 --- a/arch/arm/dts/t8103-j274.dts +++ b/arch/arm/dts/t8103-j274.dts @@ -65,7 +65,7 @@ device_type = "pci"; reg = <0x0 0x0 0x0 0x0 0x0>; pwren-gpios = <&smc 13 0>; - reset-gpios = <&pinctrl_ap 152 0>; + reset-gpios = <&pinctrl_ap 152 GPIO_ACTIVE_LOW>; max-link-speed = <2>; #address-cells = <3>; @@ -76,7 +76,7 @@ pci1: pci@1,0 { device_type = "pci"; reg = <0x800 0x0 0x0 0x0 0x0>; - reset-gpios = <&pinctrl_ap 153 0>; + reset-gpios = <&pinctrl_ap 153 GPIO_ACTIVE_LOW>; max-link-speed = <2>; #address-cells = <3>; @@ -87,7 +87,7 @@ pci2: pci@2,0 { device_type = "pci"; reg = <0x1000 0x0 0x0 0x0 0x0>; - reset-gpios = <&pinctrl_ap 33 0>; + reset-gpios = <&pinctrl_ap 33 GPIO_ACTIVE_LOW>; max-link-speed = <1>; #address-cells = <3>; diff --git a/drivers/pci/pcie_apple.c b/drivers/pci/pcie_apple.c index bef6043adb..89eec70d81 100644 --- a/drivers/pci/pcie_apple.c +++ b/drivers/pci/pcie_apple.c @@ -210,7 +210,7 @@ static int apple_pcie_setup_port(struct apple_pcie_priv *pcie, unsigned idx) return 0; dm_gpio_set_dir_flags(&pcie->perstn[idx], GPIOD_IS_OUT); - dm_gpio_set_value(&pcie->perstn[idx], 0); + dm_gpio_set_value(&pcie->perstn[idx], 1); rmwl(0, PORT_APPCLK_EN, pcie->base_port[idx] + PORT_APPCLK); @@ -221,7 +221,7 @@ static int apple_pcie_setup_port(struct apple_pcie_priv *pcie, unsigned idx) apple_pcie_port_pwren(pcie, idx); rmwl(0, PORT_PERST_OFF, pcie->base_port[idx] + PORT_PERST); - dm_gpio_set_value(&pcie->perstn[idx], 1); + dm_gpio_set_value(&pcie->perstn[idx], 0); res = readl_poll_timeout(pcie->base_port[idx] + PORT_STATUS, stat, (stat & PORT_STATUS_READY), 100, 250000); if (res < 0) {
> Date: Tue, 23 Nov 2021 12:24:10 +0000 > From: Marc Zyngier <maz@kernel.org> > > On Mon, 22 Nov 2021 21:50:48 +0000, > Mark Kettenis <mark.kettenis@xs4all.nl> wrote: > > > > > Date: Mon, 22 Nov 2021 14:43:37 +0000 > > > From: Marc Zyngier <maz@kernel.org> > > > > > > On Mon, 22 Nov 2021 12:03:47 +0000, > > > Pali Rohár <pali@kernel.org> wrote: > > > > > > > > On Monday 22 November 2021 10:41:56 Marc Zyngier wrote: > > > > > While the Apple PCIe driver works correctly when directly booted > > > > > from the firmware, it fails to initialise when the kernel is booted > > > > > from a bootloader using PCIe such as u-boot. > > > > > > > > > > That's beacuse we're missing a proper reset of the port (we only > > > > > clear the reset, but never assert it). > > > > > > > > > > The PCIe spec requirements are two-fold: > > > > > > > > > > - #PERST must be asserted before setting up the clocks, and > > > > > stay asserted for at least 100us (Tperst-clk). > > > > > > > > > > - Once #PERST is deasserted, the OS must wait for at least 100ms > > > > > "from the end of a Conventional Reset" before we can start talking > > > > > to the devices > > > > > > > > > > Implementing this results in a booting system. > > > > > > > > > > Fixes: 1e33888fbe44 ("PCI: apple: Add initial hardware bring-up") > > > > > Signed-off-by: Marc Zyngier <maz@kernel.org> > > > > > Cc: Alyssa Rosenzweig <alyssa@rosenzweig.io> > > > > > Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> > > > > > Cc: Bjorn Helgaas <bhelgaas@google.com> > > > > > Cc: Pali Rohár <pali@kernel.org> > > > > > > > > Looks good, but see comment below. > > > > > > > > Acked-by: Pali Rohár <pali@kernel.org> > > > > > > Thanks for that. > > > > > > > > > > > > --- > > > > > drivers/pci/controller/pcie-apple.c | 10 ++++++++++ > > > > > 1 file changed, 10 insertions(+) > > > > > > > > > > diff --git a/drivers/pci/controller/pcie-apple.c b/drivers/pci/controller/pcie-apple.c > > > > > index 1bf4d75b61be..957960a733c4 100644 > > > > > --- a/drivers/pci/controller/pcie-apple.c > > > > > +++ b/drivers/pci/controller/pcie-apple.c > > > > > @@ -539,13 +539,23 @@ static int apple_pcie_setup_port(struct apple_pcie *pcie, > > > > > > > > > > rmw_set(PORT_APPCLK_EN, port->base + PORT_APPCLK); > > > > > > > > > > + /* Engage #PERST before setting up the clock */ > > > > > + gpiod_set_value(reset, 0); > > > > > + > > > > > ret = apple_pcie_setup_refclk(pcie, port); > > > > > if (ret < 0) > > > > > return ret; > > > > > > > > > > + /* The minimal Tperst-clk value is 100us (PCIe CMS r2.0, 2.6.2) */ > > > > > + usleep_range(100, 200); > > > > > + > > > > > + /* Deassert #PERST */ > > > > > rmw_set(PORT_PERST_OFF, port->base + PORT_PERST); > > > > > gpiod_set_value(reset, 1); > > > > > > > > + Luca > > > > > > > > Just one comment. PERST# (PCIe Reset) is active-low signal. De-asserting > > > > means to really set value to 1. > > > > > > > > But there was a discussion that de-asserting should be done by call: > > > > gpiod_set_value(reset, 0); > > > > > > > > https://lore.kernel.org/linux-pci/51be082a-ff10-8a19-5648-f279aabcac51@lucaceresoli.net/ > > > > > > > > Could we make this new pcie-apple.c driver to use gpiod_set_value(reset, 0) > > > > for de-asserting, like in other drivers? > > > > > > I guess it depends whether you care about the assertion or the signal > > > itself. I think we may have a bug in the way the GPIOs are handled at > > > the moment, as it makes no difference whether I register the GPIO are > > > active high or active low... > > > > That's unfortunate. But maybe that's an opportunity to fix the > > devicetree to use GPIO_ACTIVE_LOW for these GPIOs? > > Indeed. The following hack does the right thing, and I can then > reverse the polarity of the reset in the Linux driver. Of course, it > breaks u-boot at the same time (and I suspect OpenBSD would be equally > affected). As I said in my other reply (that clearly crossed this message) the U-Boot PCIe driver has not been upstreamed yet. OpenBSD (and also NetBSD) rely on the configuration done by U-Boot and don't mess with the reset GPIO at this moment. > So if we are going down that road, we may need a flag day where all > the moving parts change. I don't really mind not being able to boot > older kernels, but this goes beyond Linux at this point. So I don't think this is needed. The earlier we fix this the better! > diff --git a/arch/arm64/boot/dts/apple/t8103.dtsi b/arch/arm64/boot/dts/apple/t8103.dtsi > index d2e9afde3729..cad1ab920304 100644 > --- a/arch/arm64/boot/dts/apple/t8103.dtsi > +++ b/arch/arm64/boot/dts/apple/t8103.dtsi > @@ -10,6 +10,7 @@ > #include <dt-bindings/interrupt-controller/apple-aic.h> > #include <dt-bindings/interrupt-controller/irq.h> > #include <dt-bindings/pinctrl/apple.h> > +#include <dt-bindings/gpio/gpio.h> > > / { > compatible = "apple,t8103", "apple,arm-platform"; > @@ -293,7 +294,7 @@ pcie0: pcie@690000000 { > port00: pci@0,0 { > device_type = "pci"; > reg = <0x0 0x0 0x0 0x0 0x0>; > - reset-gpios = <&pinctrl_ap 152 0>; > + reset-gpios = <&pinctrl_ap 152 GPIO_ACTIVE_LOW>; > max-link-speed = <2>; > > #address-cells = <3>; > @@ -313,7 +314,7 @@ port00: pci@0,0 { > port01: pci@1,0 { > device_type = "pci"; > reg = <0x800 0x0 0x0 0x0 0x0>; > - reset-gpios = <&pinctrl_ap 153 0>; > + reset-gpios = <&pinctrl_ap 153 GPIO_ACTIVE_LOW>; > max-link-speed = <2>; > > #address-cells = <3>; > @@ -333,7 +334,7 @@ port01: pci@1,0 { > port02: pci@2,0 { > device_type = "pci"; > reg = <0x1000 0x0 0x0 0x0 0x0>; > - reset-gpios = <&pinctrl_ap 33 0>; > + reset-gpios = <&pinctrl_ap 33 GPIO_ACTIVE_LOW>; > max-link-speed = <1>; > > #address-cells = <3>; > > -- > Without deviation from the norm, progress is not possible. > > _______________________________________________ > linux-arm-kernel mailing list > linux-arm-kernel@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
On Tue, 23 Nov 2021 12:27:19 +0000, Pali Rohár <pali@kernel.org> wrote: > > U-Boot has its own copy of DTS files, right? So it can be fixed in > U-Boot driver together with U-Boot DTS file at the same time. No, there is only a single DTS, solely provided by either u-boot or the earlier firmware. The version in the kernel tree is purely for documentation purposes. M.
diff --git a/drivers/pci/controller/pcie-apple.c b/drivers/pci/controller/pcie-apple.c index 1bf4d75b61be..957960a733c4 100644 --- a/drivers/pci/controller/pcie-apple.c +++ b/drivers/pci/controller/pcie-apple.c @@ -539,13 +539,23 @@ static int apple_pcie_setup_port(struct apple_pcie *pcie, rmw_set(PORT_APPCLK_EN, port->base + PORT_APPCLK); + /* Engage #PERST before setting up the clock */ + gpiod_set_value(reset, 0); + ret = apple_pcie_setup_refclk(pcie, port); if (ret < 0) return ret; + /* The minimal Tperst-clk value is 100us (PCIe CMS r2.0, 2.6.2) */ + usleep_range(100, 200); + + /* Deassert #PERST */ rmw_set(PORT_PERST_OFF, port->base + PORT_PERST); gpiod_set_value(reset, 1); + /* Wait for 100ms after #PERST deassertion (PCIe r2.0, 6.6.1) */ + msleep(100); + ret = readl_relaxed_poll_timeout(port->base + PORT_STATUS, stat, stat & PORT_STATUS_READY, 100, 250000); if (ret < 0) {
While the Apple PCIe driver works correctly when directly booted from the firmware, it fails to initialise when the kernel is booted from a bootloader using PCIe such as u-boot. That's beacuse we're missing a proper reset of the port (we only clear the reset, but never assert it). The PCIe spec requirements are two-fold: - #PERST must be asserted before setting up the clocks, and stay asserted for at least 100us (Tperst-clk). - Once #PERST is deasserted, the OS must wait for at least 100ms "from the end of a Conventional Reset" before we can start talking to the devices Implementing this results in a booting system. Fixes: 1e33888fbe44 ("PCI: apple: Add initial hardware bring-up") Signed-off-by: Marc Zyngier <maz@kernel.org> Cc: Alyssa Rosenzweig <alyssa@rosenzweig.io> Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> Cc: Bjorn Helgaas <bhelgaas@google.com> Cc: Pali Rohár <pali@kernel.org> --- drivers/pci/controller/pcie-apple.c | 10 ++++++++++ 1 file changed, 10 insertions(+)