| Message ID | 20230321201313.2507539-14-sean.anderson@seco.com |
|---|---|
| State | Superseded |
| Headers | show |
| Series | phy: Add support for Lynx 10G SerDes | expand |
On Tue, Mar 21, 2023 at 04:13:12PM -0400, Sean Anderson wrote: > This adds serdes support to the LS1088ARDB. I have tested the QSGMII > ports as well as the two 10G ports. The SFP slot is now fully supported, > instead of being modeled as a fixed-link. > > Linux hangs around when the serdes is initialized if the si5341 is > enabled with the in-tree driver, so I have modeled it as a two fixed > clocks instead. There are a few registers in the QIXIS FPGA which > control the SFP GPIOs; I have modeled them as discrete GPIO controllers > for now. I never saw the AQR105 interrupt fire; not sure what was going > on, but I have removed it to force polling. So you didn't see the interrupt fire even without these patches? I just tested this on a LS1088ARDB and it works. root@localhost:~# cat /proc/interrupts | grep extirq 99: 5 ls-extirq 2 Level 0x0000000008b97000:00 root@localhost:~# ip link set dev endpmac2 up root@localhost:~# cat /proc/interrupts | grep extirq 99: 6 ls-extirq 2 Level 0x0000000008b97000:00 root@localhost:~# ip link set dev endpmac2 down root@localhost:~# cat /proc/interrupts | grep extirq 99: 7 ls-extirq 2 Level 0x0000000008b97000:00 Please don't just remove things. Ioana
On 3/24/23 09:17, Ioana Ciornei wrote: > On Tue, Mar 21, 2023 at 04:13:12PM -0400, Sean Anderson wrote: >> This adds serdes support to the LS1088ARDB. I have tested the QSGMII >> ports as well as the two 10G ports. The SFP slot is now fully supported, >> instead of being modeled as a fixed-link. >> >> Linux hangs around when the serdes is initialized if the si5341 is >> enabled with the in-tree driver, so I have modeled it as a two fixed >> clocks instead. There are a few registers in the QIXIS FPGA which >> control the SFP GPIOs; I have modeled them as discrete GPIO controllers >> for now. I never saw the AQR105 interrupt fire; not sure what was going >> on, but I have removed it to force polling. > > So you didn't see the interrupt fire even without these patches? Not sure. I went to check this, and discovered I could no longer get the link to come up in Linux, even on v6.0 (before the rate adaptation tuff). I see the LEDs blinking in U-Boot, so presumably it's some configuration problem. I'm going to look into this further when I have more time. > I just tested this on a LS1088ARDB and it works. > > root@localhost:~# cat /proc/interrupts | grep extirq > 99: 5 ls-extirq 2 Level 0x0000000008b97000:00 > root@localhost:~# ip link set dev endpmac2 up > root@localhost:~# cat /proc/interrupts | grep extirq > 99: 6 ls-extirq 2 Level 0x0000000008b97000:00 > root@localhost:~# ip link set dev endpmac2 down > root@localhost:~# cat /proc/interrupts | grep extirq > 99: 7 ls-extirq 2 Level 0x0000000008b97000:00 > > Please don't just remove things. Well, polling isn't the worst thing for a single interface... I do remember having a problem with the interrupt. If this series works with interrupts enabled, I can leave it in. Did you have a chance to look at the core (patches 7 and 8) of this series? Does it make sense to you? Am I missing something which would allow switching from 1G->10G? --Sean
On 3/27/23 14:15, Sean Anderson wrote: > On 3/24/23 09:17, Ioana Ciornei wrote: >> On Tue, Mar 21, 2023 at 04:13:12PM -0400, Sean Anderson wrote: >>> This adds serdes support to the LS1088ARDB. I have tested the QSGMII >>> ports as well as the two 10G ports. The SFP slot is now fully supported, >>> instead of being modeled as a fixed-link. >>> >>> Linux hangs around when the serdes is initialized if the si5341 is >>> enabled with the in-tree driver, so I have modeled it as a two fixed >>> clocks instead. There are a few registers in the QIXIS FPGA which >>> control the SFP GPIOs; I have modeled them as discrete GPIO controllers >>> for now. I never saw the AQR105 interrupt fire; not sure what was going >>> on, but I have removed it to force polling. >> >> So you didn't see the interrupt fire even without these patches? > > Not sure. I went to check this, and discovered I could no longer get the > link to come up in Linux, even on v6.0 (before the rate adaptation > tuff). I see the LEDs blinking in U-Boot, so presumably it's some > configuration problem. I'm going to look into this further when I have > more time. I figured it out. I forgot the dpmac2 was 10G only (no rate adaptation for the AQR105). And the interrupt does not fire on net/main: # cat /proc/interrupts | grep extirq 22: 0 0 0 0 0 0 0 0 ls-extirq 2 Level 0x0000000008b97000:00 Inspecting the phy manually shows the link coming up. By removing the interrupt, the link comes up as usual. I wanted to look into this further, but the IRQ goes through the QIXIS and the firmware source isn't available so I wasn't able to do so. If you'd like, I can try probing the signal (to see where the problem is), but I won't have time for a bit. >> I just tested this on a LS1088ARDB and it works. >> >> root@localhost:~# cat /proc/interrupts | grep extirq >> 99: 5 ls-extirq 2 Level 0x0000000008b97000:00 >> root@localhost:~# ip link set dev endpmac2 up >> root@localhost:~# cat /proc/interrupts | grep extirq >> 99: 6 ls-extirq 2 Level 0x0000000008b97000:00 >> root@localhost:~# ip link set dev endpmac2 down >> root@localhost:~# cat /proc/interrupts | grep extirq >> 99: 7 ls-extirq 2 Level 0x0000000008b97000:00 >> >> Please don't just remove things. > > Well, polling isn't the worst thing for a single interface... I do > remember having a problem with the interrupt. If this series works > with interrupts enabled, I can leave it in. Anyway, given that interrupts seem to be broken for some boards? some configurations? I would like to keep polling. > Did you have a chance to look at the core (patches 7 and 8) of this > series? Does it make sense to you? Am I missing something which would > allow switching from 1G->10G? --Sean
On Mon, Mar 27, 2023 at 02:15:47PM -0400, Sean Anderson wrote: > On 3/24/23 09:17, Ioana Ciornei wrote: > > On Tue, Mar 21, 2023 at 04:13:12PM -0400, Sean Anderson wrote: > >> This adds serdes support to the LS1088ARDB. I have tested the QSGMII > >> ports as well as the two 10G ports. The SFP slot is now fully supported, > >> instead of being modeled as a fixed-link. > >> > >> Linux hangs around when the serdes is initialized if the si5341 is > >> enabled with the in-tree driver, so I have modeled it as a two fixed > >> clocks instead. There are a few registers in the QIXIS FPGA which > >> control the SFP GPIOs; I have modeled them as discrete GPIO controllers > >> for now. I never saw the AQR105 interrupt fire; not sure what was going > >> on, but I have removed it to force polling. > > > > So you didn't see the interrupt fire even without these patches? > > Not sure. I went to check this, and discovered I could no longer get the > link to come up in Linux, even on v6.0 (before the rate adaptation > tuff). I see the LEDs blinking in U-Boot, so presumably it's some > configuration problem. I'm going to look into this further when I have > more time. > > > I just tested this on a LS1088ARDB and it works. > > > > root@localhost:~# cat /proc/interrupts | grep extirq > > 99: 5 ls-extirq 2 Level 0x0000000008b97000:00 > > root@localhost:~# ip link set dev endpmac2 up > > root@localhost:~# cat /proc/interrupts | grep extirq > > 99: 6 ls-extirq 2 Level 0x0000000008b97000:00 > > root@localhost:~# ip link set dev endpmac2 down > > root@localhost:~# cat /proc/interrupts | grep extirq > > 99: 7 ls-extirq 2 Level 0x0000000008b97000:00 > > > > Please don't just remove things. > > Well, polling isn't the worst thing for a single interface... I do > remember having a problem with the interrupt. If this series works > with interrupts enabled, I can leave it in. > > Did you have a chance to look at the core (patches 7 and 8) of this > series? Does it make sense to you? Am I missing something which would > allow switching from 1G->10G? > For a bit of context, I also attempted dynamic switching from 1G to 10G on my own even before this patch set but I did not get a link up on the PCS (CDR lock was there through). Pretty much the same state as you. What I propose is to take this whole endeavor step by step. I am also interrested in getting this feature to actually work but I just didn't have the time to investigate in depth was is missing. And without the dynamic switching I cannot say that I find the addition of the SerDes PHY driver useful. I have the Lynx 10G on my TODO list but I still have some other tasks on the Lynx 28G for the next 2-3 weeks. Once I get those done, I will look closer at the patches. In the meantime, some small thigs from this patch set can be submitted separately. For example, describing the SFP cage on the LS1088ARDB. I still have some small questions on the DTS implementation for the gpio controllers but I would be able to submit this myself if you do not find the time (with your authorship of course). Ioana
On 3/28/23 05:25, Ioana Ciornei wrote: > On Mon, Mar 27, 2023 at 02:15:47PM -0400, Sean Anderson wrote: >> On 3/24/23 09:17, Ioana Ciornei wrote: >> > On Tue, Mar 21, 2023 at 04:13:12PM -0400, Sean Anderson wrote: >> >> This adds serdes support to the LS1088ARDB. I have tested the QSGMII >> >> ports as well as the two 10G ports. The SFP slot is now fully supported, >> >> instead of being modeled as a fixed-link. >> >> >> >> Linux hangs around when the serdes is initialized if the si5341 is >> >> enabled with the in-tree driver, so I have modeled it as a two fixed >> >> clocks instead. There are a few registers in the QIXIS FPGA which >> >> control the SFP GPIOs; I have modeled them as discrete GPIO controllers >> >> for now. I never saw the AQR105 interrupt fire; not sure what was going >> >> on, but I have removed it to force polling. >> > >> > So you didn't see the interrupt fire even without these patches? >> >> Not sure. I went to check this, and discovered I could no longer get the >> link to come up in Linux, even on v6.0 (before the rate adaptation >> tuff). I see the LEDs blinking in U-Boot, so presumably it's some >> configuration problem. I'm going to look into this further when I have >> more time. >> >> > I just tested this on a LS1088ARDB and it works. >> > >> > root@localhost:~# cat /proc/interrupts | grep extirq >> > 99: 5 ls-extirq 2 Level 0x0000000008b97000:00 >> > root@localhost:~# ip link set dev endpmac2 up >> > root@localhost:~# cat /proc/interrupts | grep extirq >> > 99: 6 ls-extirq 2 Level 0x0000000008b97000:00 >> > root@localhost:~# ip link set dev endpmac2 down >> > root@localhost:~# cat /proc/interrupts | grep extirq >> > 99: 7 ls-extirq 2 Level 0x0000000008b97000:00 >> > >> > Please don't just remove things. >> >> Well, polling isn't the worst thing for a single interface... I do >> remember having a problem with the interrupt. If this series works >> with interrupts enabled, I can leave it in. >> >> Did you have a chance to look at the core (patches 7 and 8) of this >> series? Does it make sense to you? Am I missing something which would >> allow switching from 1G->10G? >> > > For a bit of context, I also attempted dynamic switching from 1G to 10G > on my own even before this patch set but I did not get a link up on the > PCS (CDR lock was there through). Pretty much the same state as you. > > What I propose is to take this whole endeavor step by step. > I am also interrested in getting this feature to actually work but I > just didn't have the time to investigate in depth was is missing. > And without the dynamic switching I cannot say that I find the addition > of the SerDes PHY driver useful. Well, it's still useful for supporting 1G and 10G. I touched on this in the cover letter, but there are conflicting PLL defaults on the LS1046A. If you use SRDS_PRCTL 1133, the PLL mapping is 2211. But for 3333 it's 2222. This means PLL2 is used for both 1G and 10G, but no reference frequency can work for both. This will cause the PBL to enter a reset loop, since it wants the PLLs to lock before booting, and this happens before any user configuration. To get around this, we disconnected RESET_REQ_B on our board, and we use this driver to configure the PLLs correctly for whatever SRDS_PRCTL we boot up with. This way we can have two RCWs for 1G and 10G configuration. > I have the Lynx 10G on my TODO list but I still have some other tasks > on the Lynx 28G for the next 2-3 weeks. Once I get those done, I will > look closer at the patches. OK, thanks. > In the meantime, some small thigs from this patch set can be submitted > separately. For example, describing the SFP cage on the LS1088ARDB. I'll have a look at this. I suppose I will also split off the IRQ thing. > I still have some small questions on the DTS implementation for the gpio > controllers but I would be able to submit this myself if you do not find > the time (with your authorship of course). I am going to do another revision to address the GPIO binding problem, so please ask away. --Sean
diff --git a/arch/arm64/boot/dts/freescale/fsl-ls1088a-rdb.dts b/arch/arm64/boot/dts/freescale/fsl-ls1088a-rdb.dts index ee8e932628d1..ede537b644e8 100644 --- a/arch/arm64/boot/dts/freescale/fsl-ls1088a-rdb.dts +++ b/arch/arm64/boot/dts/freescale/fsl-ls1088a-rdb.dts @@ -10,17 +10,55 @@ /dts-v1/; +#include <dt-bindings/gpio/gpio.h> + #include "fsl-ls1088a.dtsi" / { model = "LS1088A RDB Board"; compatible = "fsl,ls1088a-rdb", "fsl,ls1088a"; + + clk_100mhz: clock-100mhz { + compatible = "fixed-clock"; + #clock-cells = <0>; + clock-frequency = <100000000>; + }; + + clk_156mhz: clock-156mhz { + compatible = "fixed-clock"; + #clock-cells = <0>; + clock-frequency = <156250000>; + }; + + sfp_slot: sfp { + compatible = "sff,sfp"; + i2c-bus = <&sfp_i2c>; + los-gpios = <&los_stat 5 GPIO_ACTIVE_HIGH>; + tx-fault-gpios = <&los_stat 4 GPIO_ACTIVE_HIGH>; + tx-disable-gpios = <&brdcfg9 4 GPIO_ACTIVE_HIGH>; + }; +}; + +&serdes1 { + clocks = <&clk_100mhz>, <&clk_156mhz>; + clock-names = "ref0", "ref1"; + fsl,unused-lanes-reserved; + status = "okay"; +}; + +&dpmac1 { + managed = "in-band-status"; + pcs-handle = <&pcs1>; + phys = <&serdes1_C>; + sfp = <&sfp_slot>; }; &dpmac2 { phy-handle = <&mdio2_aquantia_phy>; phy-connection-type = "10gbase-r"; + managed = "in-band-status"; pcs-handle = <&pcs2>; + phys = <&serdes1_D>; }; &dpmac3 { @@ -28,6 +66,7 @@ &dpmac3 { phy-connection-type = "qsgmii"; managed = "in-band-status"; pcs-handle = <&pcs3_0>; + phys = <&serdes1_A>; }; &dpmac4 { @@ -35,6 +74,7 @@ &dpmac4 { phy-connection-type = "qsgmii"; managed = "in-band-status"; pcs-handle = <&pcs3_1>; + phys = <&serdes1_A>; }; &dpmac5 { @@ -42,6 +82,7 @@ &dpmac5 { phy-connection-type = "qsgmii"; managed = "in-band-status"; pcs-handle = <&pcs3_2>; + phys = <&serdes1_A>; }; &dpmac6 { @@ -49,6 +90,7 @@ &dpmac6 { phy-connection-type = "qsgmii"; managed = "in-band-status"; pcs-handle = <&pcs3_3>; + phys = <&serdes1_A>; }; &dpmac7 { @@ -56,6 +98,7 @@ &dpmac7 { phy-connection-type = "qsgmii"; managed = "in-band-status"; pcs-handle = <&pcs7_0>; + phys = <&serdes1_B>; }; &dpmac8 { @@ -63,6 +106,7 @@ &dpmac8 { phy-connection-type = "qsgmii"; managed = "in-band-status"; pcs-handle = <&pcs7_1>; + phys = <&serdes1_B>; }; &dpmac9 { @@ -70,6 +114,7 @@ &dpmac9 { phy-connection-type = "qsgmii"; managed = "in-band-status"; pcs-handle = <&pcs7_2>; + phys = <&serdes1_B>; }; &dpmac10 { @@ -77,6 +122,7 @@ &dpmac10 { phy-connection-type = "qsgmii"; managed = "in-band-status"; pcs-handle = <&pcs7_3>; + phys = <&serdes1_B>; }; &emdio1 { @@ -128,7 +174,6 @@ &emdio2 { mdio2_aquantia_phy: ethernet-phy@0 { compatible = "ethernet-phy-ieee802.3-c45"; - interrupts-extended = <&extirq 2 IRQ_TYPE_LEVEL_LOW>; reg = <0x0>; }; }; @@ -171,6 +216,12 @@ rtc@51 { interrupts-extended = <&extirq 0 IRQ_TYPE_LEVEL_LOW>; }; }; + + sfp_i2c: i2c@6 { + #address-cells = <1>; + #size-cells = <0>; + reg = <0x6>; + }; }; }; @@ -185,8 +236,31 @@ nand@0,0 { }; fpga: board-control@2,0 { - compatible = "fsl,ls1088ardb-fpga", "fsl,fpga-qixis"; + #address-cells = <1>; + #size-cells = <1>; + compatible = "fsl,ls1088ardb-fpga", "fsl,fpga-qixis", + "simple-bus"; reg = <0x2 0x0 0x0000100>; + ranges = <0x0 0x2 0x0 0x0000100>; + + los_stat: gpio-controller@1d { + #gpio-cells = <2>; + compatible = "fsl,fpga-qixis-los-stat", + "ni,169445-nand-gpio"; + reg = <0x1d 0x1>; + reg-names = "dat"; + gpio-controller; + no-output; + }; + + brdcfg9: gpio-controller@59 { + #gpio-cells = <2>; + compatible = "fsl,fpga-qixis-brdcfg9", + "ni,169445-nand-gpio"; + reg = <0x59 0x1>; + reg-names = "dat"; + gpio-controller; + }; }; }; @@ -203,6 +277,10 @@ &esdhc { status = "okay"; }; +&pcs_mdio1 { + status = "okay"; +}; + &pcs_mdio2 { status = "okay"; };
This adds serdes support to the LS1088ARDB. I have tested the QSGMII ports as well as the two 10G ports. The SFP slot is now fully supported, instead of being modeled as a fixed-link. Linux hangs around when the serdes is initialized if the si5341 is enabled with the in-tree driver, so I have modeled it as a two fixed clocks instead. There are a few registers in the QIXIS FPGA which control the SFP GPIOs; I have modeled them as discrete GPIO controllers for now. I never saw the AQR105 interrupt fire; not sure what was going on, but I have removed it to force polling. To enable serdes support, the DPC needs to set the macs to MAC_LINK_TYPE_BACKPLANE. All MACs using the same QSGMII should be converted at once. Additionally, in order to change interface types, the MC firmware must support DPAA2_MAC_FEATURE_PROTOCOL_CHANGE. Signed-off-by: Sean Anderson <sean.anderson@seco.com> --- (no changes since v10) Changes in v10: - Move serdes bindings to SoC dtsi - Use "descriptions" instead of "bindings" - Don't use /clocks - Add missing gpio-controller properties Changes in v9: - Add fsl,unused-lanes-reserved to allow a gradual transition, depending on the mac link type. - Remove unused clocks - Fix some phy mode node names - phy-type -> fsl,phy Changes in v8: - Rename serdes phy handles like the LS1046A - Add SFP slot binding - Fix incorrect lane ordering (it's backwards on the LS1088A just like it is in the LS1046A). - Fix duplicated lane 2 (it should have been lane 3). - Fix incorrectly-documented value for XFI1. - Remove interrupt for aquantia phy. It never fired for whatever reason, preventing the link from coming up. - Add GPIOs for QIXIS FPGA. - Enable MAC1 PCS - Remove si5341 binding Changes in v4: - Convert to new bindings .../boot/dts/freescale/fsl-ls1088a-rdb.dts | 82 ++++++++++++++++++- 1 file changed, 80 insertions(+), 2 deletions(-)