[RFC,v2,00/11] dwmac-meson8b Ethernet RX delay configuration

Message

Martin Blumenstingl April 29, 2020, 8:16 p.m. UTC

The Ethernet TX performance has been historically bad on Meson8b and
Meson8m2 SoCs because high packet loss was seen. I found out that this
was related (yet again) to the RGMII TX delay configuration.
In the process of discussing the big picture (and not just a single
patch) [0] with Andrew I discovered that the IP block behind the
dwmac-meson8b driver actually seems to support the configuration of the
RGMII RX delay (at least on the Meson8b SoC generation).

Since I sent the last RFC I got additional documentation from Jianxin
(many thanks!). Also I have discovered some more interesting details:
- Meson8b Odroid-C1 requires an RX delay (by either the PHY or the MAC)
  Based on the vendor u-boot code (not upstream) I assume that it will
  be the same for all Meson8b and Meson8m2 boards
- Khadas VIM2 seems to have the RX delay built into the PCB trace
  length. When I enable the RX delay on the PHY or MAC I can't get any
  data through. I expect that we will have the same situation on all
  GXBB, GXM, AXG, G12A, G12B and SM1 boards


Changes since RFC v1 at [1]:
- add support for the timing adjustment clock input (dt-bindings and
  in the driver) thanks to the input from the unnamed Ethernet engineer
  at Amlogic. This is the missing link between the fclk_div2 clock and
  the Ethernet controller on Meson8b (no traffic would flow if that
  clock was disabled)
- add support fot the amlogic,rx-delay-ns property. The only supported
  values so far are 0ns and 2ns. The registers seem to allow more
  precise timing adjustments, but I could not make that work so far.
- add more register documentation (for the new RX delay bits) and
  unified the placement of existing register documentation. Again,
  thanks to Jianxin and the unnamed Ethernet engineer at Amlogic
- DO NOT MERGE: .dts patches to show the conversion of the Meson8b
  and Meson8m2 boards to "rgmii-id". I didn't have time for all arm64
  patches yet, but these will switch to phy-mode = "rgmii-txid" with
  amlogic,rx-delay-ns = <0> (because the delay seems to be provided by
  the PCB trace length).


[0] https://patchwork.kernel.org/patch/11309891/
[1] https://patchwork.kernel.org/cover/11310719/


Martin Blumenstingl (11):
  dt-bindings: net: meson-dwmac: Add the amlogic,rx-delay-ns property
  dt-bindings: net: dwmac-meson: Document the "timing-adjustment" clock
  net: stmmac: dwmac-meson8b: use FIELD_PREP instead of open-coding it
  net: stmmac: dwmac-meson8b: Move the documentation for the TX delay
  net: stmmac: dwmac-meson8b: Add the PRG_ETH0_ADJ_* bits
  net: stmmac: dwmac-meson8b: Fetch the "timing-adjustment" clock
  net: stmmac: dwmac-meson8b: Make the clock enabling code re-usable
  net: stmmac: dwmac-meson8b: add support for the RX delay configuration
  arm64: dts: amlogic: Add the Ethernet "timing-adjustment" clock
  ARM: dts: meson: Add the Ethernet "timing-adjustment" clock
  ARM: dts: meson: Switch existing boards with RGMII PHY to "rgmii-id"

 .../bindings/net/amlogic,meson-dwmac.yaml     |  23 ++-
 arch/arm/boot/dts/meson8b-odroidc1.dts        |   3 +-
 arch/arm/boot/dts/meson8b.dtsi                |   5 +-
 arch/arm/boot/dts/meson8m2-mxiii-plus.dts     |   4 +-
 arch/arm/boot/dts/meson8m2.dtsi               |   5 +-
 arch/arm64/boot/dts/amlogic/meson-axg.dtsi    |   6 +-
 .../boot/dts/amlogic/meson-g12-common.dtsi    |   6 +-
 arch/arm64/boot/dts/amlogic/meson-gxbb.dtsi   |   5 +-
 arch/arm64/boot/dts/amlogic/meson-gxl.dtsi    |   5 +-
 .../ethernet/stmicro/stmmac/dwmac-meson8b.c   | 140 ++++++++++++++----
 10 files changed, 150 insertions(+), 52 deletions(-)

Comments

Andrew Lunn April 29, 2020, 9:29 p.m. UTC | #1

> - Khadas VIM2 seems to have the RX delay built into the PCB trace
>   length. When I enable the RX delay on the PHY or MAC I can't get any
>   data through. I expect that we will have the same situation on all
>   GXBB, GXM, AXG, G12A, G12B and SM1 boards

Hi Martin

Can you actually see this on the PCB? The other possibility is that
the bootloader is configuring something, which is not getting
overridden when linux starts up.

	   Andrew

Martin Blumenstingl May 1, 2020, 2:49 p.m. UTC | #2

Hi Andrew,

On Wed, Apr 29, 2020 at 11:29 PM Andrew Lunn <[e-mail]> wrote:
>
> > - Khadas VIM2 seems to have the RX delay built into the PCB trace
> >   length. When I enable the RX delay on the PHY or MAC I can't get any
> >   data through. I expect that we will have the same situation on all
> >   GXBB, GXM, AXG, G12A, G12B and SM1 boards
>
> Hi Martin
>
> Can you actually see this on the PCB? The other possibility is that
> the bootloader is configuring something, which is not getting
> overridden when linux starts up.
at least it doesn't jump straight into my eye.
I checked in u-boot and Linux, and for both the RX delay is disabled
in the PHY as well as in the MAC.

The schematics of the Khadas VIM2 also show the the RX delay in the
PHY is turned off by pin-strapping, see page 7 on the right: [0]
It's the same for the Khadas VIM3 schematics, also on page 7: [1]
There are also high resolution images of the Khadas VIM3 online so you
can look at it yourself (I couldn't find any for the Khadas VIM2 which
is what I have): [2]

I agree that we need to get an answer to the RX delay question on the
arm64 SoCs.
If there's no way to find out from the existing resources then I can
contact Khadas and ask them about the PCB trace length on VIM2, VIM3
and VIM3L (these are the ones with RGMII PHYs).

For the older SoCs the RX delay has to be provided by either the MAC
or the PHY and right now we're not configuring it.
We cannot simply enable the RX delay at the PHY level because the
bootloader enables it in the MAC (so we have to turn it off there).
So it would be great if you could still review this series.


Martin


[0] https://dl.khadas.com/Hardware/VIM2/Schematic/VIM2_V12_Sch.pdf
[1] https://dl.khadas.com/Hardware/VIM3/Schematic/VIM3_V12_Sch.pdf
[2] https://forum.khadas.com/t/khadas-vim3-is-launching-on-24-june/4103