| Message ID | 20220508224848.2384723-1-hauke@hauke-m.de (mailing list archive) |
|---|---|
| Headers | show |
| Series | net: dsa: realtek: rtl8365mb: Add SGMII and HSGMII support | expand |
> This was tested on a Buffalo WSR-2533DHP2. This is a board using a > Mediatek MT7622 SoC and its 2.5G Ethernet MAC connected over a 2.5G > HSGMII link to a RTL8367S switch providing 5 1G ports. > This is the only board I have using this switch. > > With the DSA_TAG_PROTO_RTL8_4 tag format the TCP checksum for all TCP > send packets is wrong. It is set to 0x83c6. The mac driver probably > should do some TCP checksum offload, but it does not work. 0x83c6 might be yout tcp pseudo ip header sum. > When I used the DSA_TAG_PROTO_RTL8_4T tag format the send packets are > ok, but it looks like the system does TCP Large receive offload, but > does not update the TCP checksum correctly. I see that multiple received > TCP packets are combined into one (using tcpdump on switch port on > device). The switch tag is also correctly removed. tcpdump complains > that the checksum is wrong, it was updated somewhere, but it is wrong. > > Does anyone know what could be wrong here and how to fix this? The good news, it is a known problem: https://patchwork.kernel.org/project/netdevbpf/patch/20220411230305.28951-1-luizluca@gmail.com/ (there are some interesting discussions) https://patchwork.kernel.org/project/netdevbpf/patch/20220416052737.25509-1-luizluca@gmail.com/ (merged) The bad news, there is no way to enable checksum offload with mediatek+realtek. You'll need to disable almost any type of offload. For any tag before the IP header, if your driver/HW does not support a way to set where the IP header starts and the offload HW does not understand the tag protocol, the offload HW will keep the pseudo ip header sum. And for tags after the payload, the offload HW will blend the tag with the payload, generating bad checksums when the switch removes the tag. You can do that from userland, using ethtool on the master port before the bridge is up, or patching the driver. You can try this patch (written for MT7620 but it is easy to adapt it to upstream mtk_eth_soc.c). https://github.com/luizluca/openwrt/commit/d799bd363f902bf3b9c595972a1b9280a0b61dca . I never submitted that upstream because I don't have the HW to test it (Arinç tested a modified version in an MT7621) and I didn't investigate how much those extra ifs in ndo_features_check will cost in performance when the driver does support the tag (using a mediatek switch). And the DSA_TAG_PROTO_RTL8_4T already paid off. It was added exactly as a way to test checksum errors. Probably no offload will work for tags that are after the payload if the offload HW does not already know that tag (e.g. same vendors). DSA_TAG_PROTO_RTL8_4T works because it calculates the checksum in software before the tag is added. However, during my tests, I never tested TCP Large receive offload. > This uses the rtl8367s-sgmii.bin firmware file. I extracted it from a > GPL driver source code with a GPL notice on top. I do not have the > source code of this firmware. You can download it here: > https://hauke-m.de/files/rtl8367/rtl8367s-sgmii.bin > Here are some information about the source: > https://hauke-m.de/files/rtl8367/rtl8367s-sgmii.txt > > This file does not look like intentional GPL. It would be nice if > Realtek could send this file or a similar version to the linux-firmware > repository under a license which allows redistribution. I do not have > any contact at Realtek, if someone has a contact there it would be nice > if we can help me on this topic. > > Hauke Mehrtens (4): > net: dsa: realtek: rtl8365mb: Fix interface type mask > net: dsa: realtek: rtl8365mb: Get chip option > net: dsa: realtek: rtl8365mb: Add setting MTU > net: dsa: realtek: rtl8365mb: Add SGMII and HSGMII support > > drivers/net/dsa/realtek/rtl8365mb.c | 444 ++++++++++++++++++++++++++-- > 1 file changed, 413 insertions(+), 31 deletions(-) > > -- > 2.30.2 >
> > Hauke Mehrtens (4): > > net: dsa: realtek: rtl8365mb: Fix interface type mask > > net: dsa: realtek: rtl8365mb: Get chip option > > net: dsa: realtek: rtl8365mb: Add setting MTU > > net: dsa: realtek: rtl8365mb: Add SGMII and HSGMII support I didn't get these two, although patchwork got them: net: dsa: realtek: rtl8365mb: Get chip option net: dsa: realtek: rtl8365mb: Add SGMII and HSGMII support
On 5/9/2022 12:38 AM, Luiz Angelo Daros de Luca wrote: >>> Hauke Mehrtens (4): >>> net: dsa: realtek: rtl8365mb: Fix interface type mask >>> net: dsa: realtek: rtl8365mb: Get chip option >>> net: dsa: realtek: rtl8365mb: Add setting MTU >>> net: dsa: realtek: rtl8365mb: Add SGMII and HSGMII support > > I didn't get these two, although patchwork got them: > > net: dsa: realtek: rtl8365mb: Get chip option > net: dsa: realtek: rtl8365mb: Add SGMII and HSGMII support Probably yet another instance of poor interaction between gmail.com and vger.kernel.org, I got all of them in my inbox.
On Mon, May 09, 2022 at 08:36:19AM -0700, Florian Fainelli wrote: > On 5/9/2022 12:38 AM, Luiz Angelo Daros de Luca wrote: > > > > Hauke Mehrtens (4): > > > > net: dsa: realtek: rtl8365mb: Fix interface type mask > > > > net: dsa: realtek: rtl8365mb: Get chip option > > > > net: dsa: realtek: rtl8365mb: Add setting MTU > > > > net: dsa: realtek: rtl8365mb: Add SGMII and HSGMII support > > > > I didn't get these two, although patchwork got them: > > > > net: dsa: realtek: rtl8365mb: Get chip option > > net: dsa: realtek: rtl8365mb: Add SGMII and HSGMII support > > Probably yet another instance of poor interaction between gmail.com and > vger.kernel.org, I got all of them in my inbox. > -- > Florian But you were copied to the emails, Luiz wasn't. I'm also having trouble receiving emails from the mailing list, I get them with a huge delay (days).
On 5/9/2022 8:40 AM, Vladimir Oltean wrote: > On Mon, May 09, 2022 at 08:36:19AM -0700, Florian Fainelli wrote: >> On 5/9/2022 12:38 AM, Luiz Angelo Daros de Luca wrote: >>>>> Hauke Mehrtens (4): >>>>> net: dsa: realtek: rtl8365mb: Fix interface type mask >>>>> net: dsa: realtek: rtl8365mb: Get chip option >>>>> net: dsa: realtek: rtl8365mb: Add setting MTU >>>>> net: dsa: realtek: rtl8365mb: Add SGMII and HSGMII support >>> >>> I didn't get these two, although patchwork got them: >>> >>> net: dsa: realtek: rtl8365mb: Get chip option >>> net: dsa: realtek: rtl8365mb: Add SGMII and HSGMII support >> >> Probably yet another instance of poor interaction between gmail.com and >> vger.kernel.org, I got all of them in my inbox. >> -- >> Florian > > But you were copied to the emails, Luiz wasn't. Yes, that much is true. > I'm also having trouble receiving emails from the mailing list, I get > them with a huge delay (days). Time to switch to a different toolset maybe: https://josefbacik.github.io/kernel/2021/10/18/lei-and-b4.html
On Mon, May 09, 2022 at 08:41:38AM -0700, Florian Fainelli wrote: > On 5/9/2022 8:40 AM, Vladimir Oltean wrote: > > On Mon, May 09, 2022 at 08:36:19AM -0700, Florian Fainelli wrote: > > > On 5/9/2022 12:38 AM, Luiz Angelo Daros de Luca wrote: > > > > > > Hauke Mehrtens (4): > > > > > > net: dsa: realtek: rtl8365mb: Fix interface type mask > > > > > > net: dsa: realtek: rtl8365mb: Get chip option > > > > > > net: dsa: realtek: rtl8365mb: Add setting MTU > > > > > > net: dsa: realtek: rtl8365mb: Add SGMII and HSGMII support > > > > > > > > I didn't get these two, although patchwork got them: > > > > > > > > net: dsa: realtek: rtl8365mb: Get chip option > > > > net: dsa: realtek: rtl8365mb: Add SGMII and HSGMII support > > > > > > Probably yet another instance of poor interaction between gmail.com and > > > vger.kernel.org, I got all of them in my inbox. > > > -- > > > Florian > > > > But you were copied to the emails, Luiz wasn't. > > Yes, that much is true. > > > I'm also having trouble receiving emails from the mailing list, I get > > them with a huge delay (days). > > Time to switch to a different toolset maybe: > https://josefbacik.github.io/kernel/2021/10/18/lei-and-b4.html > -- > Florian I guess I'll finally try lei out, thanks.
Hi Hauke, Thanks for your series. On Mon, May 09, 2022 at 12:48:44AM +0200, Hauke Mehrtens wrote: > This was tested on a Buffalo WSR-2533DHP2. This is a board using a > Mediatek MT7622 SoC and its 2.5G Ethernet MAC connected over a 2.5G > HSGMII link to a RTL8367S switch providing 5 1G ports. > This is the only board I have using this switch. > > With the DSA_TAG_PROTO_RTL8_4 tag format the TCP checksum for all TCP > send packets is wrong. It is set to 0x83c6. The mac driver probably > should do some TCP checksum offload, but it does not work. > > When I used the DSA_TAG_PROTO_RTL8_4T tag format the send packets are > ok, but it looks like the system does TCP Large receive offload, but > does not update the TCP checksum correctly. I see that multiple received > TCP packets are combined into one (using tcpdump on switch port on > device). The switch tag is also correctly removed. tcpdump complains > that the checksum is wrong, it was updated somewhere, but it is wrong. > > Does anyone know what could be wrong here and how to fix this? > > This uses the rtl8367s-sgmii.bin firmware file. I extracted it from a > GPL driver source code with a GPL notice on top. I do not have the > source code of this firmware. You can download it here: > https://hauke-m.de/files/rtl8367/rtl8367s-sgmii.bin > Here are some information about the source: > https://hauke-m.de/files/rtl8367/rtl8367s-sgmii.txt > > This file does not look like intentional GPL. It would be nice if > Realtek could send this file or a similar version to the linux-firmware > repository under a license which allows redistribution. I do not have > any contact at Realtek, if someone has a contact there it would be nice > if we can help me on this topic. Let me follow up on this. It will take some days so please be patient. However, in the worst case, I think that the license header in that file is pretty explicit: You did not enter any particular licensing agreement with Realtek, and therefore the code (including those bytes of SGMII firmware) are licensed to you under the GPL. IANAL but it seems quite clear, no? > > Hauke Mehrtens (4): > net: dsa: realtek: rtl8365mb: Fix interface type mask > net: dsa: realtek: rtl8365mb: Get chip option > net: dsa: realtek: rtl8365mb: Add setting MTU > net: dsa: realtek: rtl8365mb: Add SGMII and HSGMII support > > drivers/net/dsa/realtek/rtl8365mb.c | 444 ++++++++++++++++++++++++++-- > 1 file changed, 413 insertions(+), 31 deletions(-) > > -- > 2.30.2 >
On 5/9/22 08:28, Luiz Angelo Daros de Luca wrote: >> This was tested on a Buffalo WSR-2533DHP2. This is a board using a >> Mediatek MT7622 SoC and its 2.5G Ethernet MAC connected over a 2.5G >> HSGMII link to a RTL8367S switch providing 5 1G ports. >> This is the only board I have using this switch. >> >> With the DSA_TAG_PROTO_RTL8_4 tag format the TCP checksum for all TCP >> send packets is wrong. It is set to 0x83c6. The mac driver probably >> should do some TCP checksum offload, but it does not work. > > 0x83c6 might be yout tcp pseudo ip header sum. Yes this is the default checksum. I see the same checksum when listening on my laptop for packets it sends out and where the NIC does checksum offloading. >> When I used the DSA_TAG_PROTO_RTL8_4T tag format the send packets are >> ok, but it looks like the system does TCP Large receive offload, but >> does not update the TCP checksum correctly. I see that multiple received >> TCP packets are combined into one (using tcpdump on switch port on >> device). The switch tag is also correctly removed. tcpdump complains >> that the checksum is wrong, it was updated somewhere, but it is wrong. >> >> Does anyone know what could be wrong here and how to fix this? > > The good news, it is a known problem: > > https://patchwork.kernel.org/project/netdevbpf/patch/20220411230305.28951-1-luizluca@gmail.com/ > (there are some interesting discussions) > https://patchwork.kernel.org/project/netdevbpf/patch/20220416052737.25509-1-luizluca@gmail.com/ > (merged) Thanks for the links. > The bad news, there is no way to enable checksum offload with > mediatek+realtek. You'll need to disable almost any type of offload. > For any tag before the IP header, if your driver/HW does not support a > way to set where the IP header starts and the offload HW does not > understand the tag protocol, the offload HW will keep the pseudo ip > header sum. And for tags after the payload, the offload HW will blend > the tag with the payload, generating bad checksums when the switch > removes the tag. > > You can do that from userland, using ethtool on the master port before > the bridge is up, or patching the driver. You can try this patch > (written for MT7620 but it is easy to adapt it to upstream > mtk_eth_soc.c). > https://github.com/luizluca/openwrt/commit/d799bd363f902bf3b9c595972a1b9280a0b61dca > . I never submitted that upstream because I don't have the HW to test > it (Arinç tested a modified version in an MT7621) and I didn't > investigate how much those extra ifs in ndo_features_check will cost > in performance when the driver does support the tag (using a mediatek > switch). Thanks for the tip to remove the checksum offloading before adding the device to the bridge, I was already wondering why it did not work when I deactivated the checksum offloading later. Thanks for the link. I also have one device with a MT7531 switch, but it is sued in production. I will check this in the next days. > And the DSA_TAG_PROTO_RTL8_4T already paid off. It was added exactly > as a way to test checksum errors. Probably no offload will work for > tags that are after the payload if the offload HW does not already > know that tag (e.g. same vendors). DSA_TAG_PROTO_RTL8_4T works because > it calculates the checksum in software before the tag is added. > However, during my tests, I never tested TCP Large receive offload. I accidentally tested TCP Large receive offload. ;-) The driver has a special DMA flag for to tell the MAC that there is a Mediatek tag in the packet. >> This uses the rtl8367s-sgmii.bin firmware file. I extracted it from a >> GPL driver source code with a GPL notice on top. I do not have the >> source code of this firmware. You can download it here: >> https://hauke-m.de/files/rtl8367/rtl8367s-sgmii.bin >> Here are some information about the source: >> https://hauke-m.de/files/rtl8367/rtl8367s-sgmii.txt >> >> This file does not look like intentional GPL. It would be nice if >> Realtek could send this file or a similar version to the linux-firmware >> repository under a license which allows redistribution. I do not have >> any contact at Realtek, if someone has a contact there it would be nice >> if we can help me on this topic. >> >> Hauke Mehrtens (4): >> net: dsa: realtek: rtl8365mb: Fix interface type mask >> net: dsa: realtek: rtl8365mb: Get chip option >> net: dsa: realtek: rtl8365mb: Add setting MTU >> net: dsa: realtek: rtl8365mb: Add SGMII and HSGMII support >> >> drivers/net/dsa/realtek/rtl8365mb.c | 444 ++++++++++++++++++++++++++-- >> 1 file changed, 413 insertions(+), 31 deletions(-) Hauke