| Message ID | 20230201145845.2312060-1-o.rempel@pengutronix.de (mailing list archive) |
|---|---|
| Headers | show |
| Series | net: add EEE support for KSZ9477 and AR8035 with i.MX6 | expand |
On Wed, Feb 01, 2023 at 03:58:22PM +0100, Oleksij Rempel wrote: > With this patch series we provide EEE control for KSZ9477 family of switches and > AR8035 with i.MX6 configuration. > According to my tests, on a system with KSZ8563 switch and 100Mbit idle link, > we consume 0,192W less power per port if EEE is enabled. What is the code flow through the kernel with EEE? I wasn't able to find a good explanation about it. Is it advertised by default, if supported? I guess phy_advertise_supported() does that. But is that desirable? Doesn't EEE cause undesired latency for MAC-level PTP timestamping on an otherwise idle link?
On Sat, Feb 04, 2023 at 02:13:32AM +0200, Vladimir Oltean wrote: > On Wed, Feb 01, 2023 at 03:58:22PM +0100, Oleksij Rempel wrote: > > With this patch series we provide EEE control for KSZ9477 family of switches and > > AR8035 with i.MX6 configuration. > > According to my tests, on a system with KSZ8563 switch and 100Mbit idle link, > > we consume 0,192W less power per port if EEE is enabled. > > What is the code flow through the kernel with EEE? I wasn't able to find > a good explanation about it. > > Is it advertised by default, if supported? I guess phy_advertise_supported() > does that. Ack. > But is that desirable? Doesn't EEE cause undesired latency for MAC-level > PTP timestamping on an otherwise idle link? Theoretically, MAC controls Low Power Idle states and even with some "Wake" latency should be fully aware of actual ingress and egress time stamps. Practically, right now I do not have such HW to confirm it. My project is affected by EEE in different ways: - with EEE PTP has too much jitter - without EEE, the devices consumes too much power in standby mode with WoL enabled. Even switching to 10BaseT less power as 100BaseTX with EEE would do. My view is probably biased by my environment - PTP is relatively rare use case. EEE saves power (0,2W+0,2W per link in my case). Summary power saving of all devices is potentially equal to X amount of power plants. So, EEE should be enabled by default. Beside, flow control (enabled by default) affects PTP in some cases too. May be ptp4l should warn about this options? We should be able to detect it from user space. Regards, Oleksij
On Mon, Feb 06, 2023 at 06:47:13AM +0100, Oleksij Rempel wrote: > On Sat, Feb 04, 2023 at 02:13:32AM +0200, Vladimir Oltean wrote: > > On Wed, Feb 01, 2023 at 03:58:22PM +0100, Oleksij Rempel wrote: > > > With this patch series we provide EEE control for KSZ9477 family of switches and > > > AR8035 with i.MX6 configuration. > > > According to my tests, on a system with KSZ8563 switch and 100Mbit idle link, > > > we consume 0,192W less power per port if EEE is enabled. > > > > What is the code flow through the kernel with EEE? I wasn't able to find > > a good explanation about it. > > > > Is it advertised by default, if supported? I guess phy_advertise_supported() > > does that. > > Ack. > > > But is that desirable? Doesn't EEE cause undesired latency for MAC-level > > PTP timestamping on an otherwise idle link? > > Theoretically, MAC controls Low Power Idle states and even with some > "Wake" latency should be fully aware of actual ingress and egress time > stamps. I'm not sure if this is also true with Atheros SmartEEE, where the PHY is the one who enters LPI mode and buffers packets until it wakes up? > > Practically, right now I do not have such HW to confirm it. My project > is affected by EEE in different ways: Doesn't FEC support PTP? > - with EEE PTP has too much jitter > - without EEE, the devices consumes too much power in standby mode with > WoL enabled. Even switching to 10BaseT less power as 100BaseTX with > EEE would do. > > My view is probably biased by my environment - PTP is relatively rare > use case. EEE saves power (0,2W+0,2W per link in my case). Summary power > saving of all devices is potentially equal to X amount of power plants. > So, EEE should be enabled by default. I'm not contesting the value of EEE. Just wondering whether it's best for the kernel, rather than user space, to enable it by default. > > Beside, flow control (enabled by default) affects PTP in some cases too. You are probably talking about the fact that flow control may affect end-to-end delay measurements (across switches in a LAN). Yes, but EEE (or at least SmartEEE) may affect peer-to-peer delay measurements, which I see as worse. > > May be ptp4l should warn about this options? We should be able to detect > it from user space. This isn't necessarily a bad idea, even though it would end up renegotiating and losing the link. Maybe it would be good to drag Richard Cochran into the discussion too. After all he's the one who should agree what should and what shouldn't ptp4l be concerned with. > > Regards, > Oleksij > -- > Pengutronix e.K. | | > Steuerwalder Str. 21 | http://www.pengutronix.de/ | > 31137 Hildesheim, Germany | Phone: +49-5121-206917-0 | > Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-5555 |
> > > What is the code flow through the kernel with EEE? I wasn't able to find > > > a good explanation about it. > > > > > > Is it advertised by default, if supported? I guess phy_advertise_supported() > > > does that. The old flow is poorly defined. If the MAC supports EEE, it should call phy_init_eee(). That looks at the results of auto-neg and returns if EEE has been negotiated or not. However, i'm not aware of any code which disables by default the advertisement of EEE, or actually enables the negotiation of EEE. So there are probably a number of PHYs which are EEE capable, connected to a MAC driver which does not call phy_init_eee() and are advertising EEE and negotiating EEE. There might also be a subset of that which are actually doing EEE, despite not calling phy_init_eee(). So the current code is not good, and there is a danger we introduce power regressions as we sort this out. The current MAC/PHY API is pretty broken. We probably should be handling this similar to pause. A MAC which supports pause should call phy_support_asym_pause() or phy_support_sym_pause() which will cause the PHY to advertise its supported Pause modes. So we might want to add a phy_support_eee()? We then want the result of EEE negotiation available in phydev for when the link_adjust() callback is called. A quick look at a few MAC drivers seems to indicate many are getting it wrong and don't actually wait for the result of the auto-neg.... Andrew
On Mon, Feb 06, 2023 at 04:10:38PM +0200, Vladimir Oltean wrote: > On Mon, Feb 06, 2023 at 06:47:13AM +0100, Oleksij Rempel wrote: > > On Sat, Feb 04, 2023 at 02:13:32AM +0200, Vladimir Oltean wrote: > > > On Wed, Feb 01, 2023 at 03:58:22PM +0100, Oleksij Rempel wrote: > > > > With this patch series we provide EEE control for KSZ9477 family of switches and > > > > AR8035 with i.MX6 configuration. > > > > According to my tests, on a system with KSZ8563 switch and 100Mbit idle link, > > > > we consume 0,192W less power per port if EEE is enabled. > > > > > > What is the code flow through the kernel with EEE? I wasn't able to find > > > a good explanation about it. > > > > > > Is it advertised by default, if supported? I guess phy_advertise_supported() > > > does that. > > > > Ack. > > > > > But is that desirable? Doesn't EEE cause undesired latency for MAC-level > > > PTP timestamping on an otherwise idle link? > > > > Theoretically, MAC controls Low Power Idle states and even with some > > "Wake" latency should be fully aware of actual ingress and egress time > > stamps. > > I'm not sure if this is also true with Atheros SmartEEE, where the PHY > is the one who enters LPI mode and buffers packets until it wakes up? Yes, you right. With SmartEEE without MAC assistance, PTP will be broken. At the same time, if MAC is PTP and EEE capable, the same PHY with SmartEEE disabled should work just fine. > > Practically, right now I do not have such HW to confirm it. My project > > is affected by EEE in different ways: > > Doesn't FEC support PTP? FEC do supports PTP, but do not support EEE on i.MX6/7 variants. > > - with EEE PTP has too much jitter > > - without EEE, the devices consumes too much power in standby mode with > > WoL enabled. Even switching to 10BaseT less power as 100BaseTX with > > EEE would do. > > > > My view is probably biased by my environment - PTP is relatively rare > > use case. EEE saves power (0,2W+0,2W per link in my case). Summary power > > saving of all devices is potentially equal to X amount of power plants. > > So, EEE should be enabled by default. > > I'm not contesting the value of EEE. Just wondering whether it's best > for the kernel, rather than user space, to enable it by default. I woulds say, at the end the switch will decide what functionality will be advertised. Other nodes should just tell what capabilities they support. > > > > Beside, flow control (enabled by default) affects PTP in some cases too. > > You are probably talking about the fact that flow control may affect > end-to-end delay measurements (across switches in a LAN). Yes, but EEE > (or at least SmartEEE) may affect peer-to-peer delay measurements, which > I see as worse. I agree. User space should be notified some how about SmartEEE functionality. Especially if it is incompatible with some other functionality like PTP. It took me some time to understand why my PTP sync was so unstable. SmartEEE was just silently enabled by HW and no EEE related information was provided to user space. > > May be ptp4l should warn about this options? We should be able to detect > > it from user space. > > This isn't necessarily a bad idea, even though it would end up > renegotiating and losing the link. My idea was to inform the user, not actively do what ever is needed. It can conflict with other services or make system administrator scratch the head without understanding why things magically happen. > Maybe it would be good to drag Richard Cochran into the discussion too. > After all he's the one who should agree what should and what shouldn't > ptp4l be concerned with. ACK. Regards, Oleksij
On Mon, Feb 06, 2023 at 04:39:52PM +0100, Andrew Lunn wrote: > > > > What is the code flow through the kernel with EEE? I wasn't able to find > > > > a good explanation about it. > > > > > > > > Is it advertised by default, if supported? I guess phy_advertise_supported() > > > > does that. > > The old flow is poorly defined. If the MAC supports EEE, it should > call phy_init_eee(). That looks at the results of auto-neg and returns > if EEE has been negotiated or not. > > However, i'm not aware of any code which disables by default the > advertisement of EEE, or actually enables the negotiation of EEE. So > there are probably a number of PHYs which are EEE capable, connected > to a MAC driver which does not call phy_init_eee() and are advertising > EEE and negotiating EEE. There might also be a subset of that which > are actually doing EEE, despite not calling phy_init_eee(). > > So the current code is not good, and there is a danger we introduce > power regressions as we sort this out. > > The current MAC/PHY API is pretty broken. We probably should be > handling this similar to pause. A MAC which supports pause should call > phy_support_asym_pause() or phy_support_sym_pause() which will cause > the PHY to advertise its supported Pause modes. So we might want to > add a phy_support_eee()? We then want the result of EEE negotiation > available in phydev for when the link_adjust() callback is called. Good point. SmartEEE will be probably a bit more challenging. If MAC do not advertise EEE support, SmartEEE can be enabled. But it would break PTP if SmartEEE is active. Except SmartEEE capable PHY implements own PTP support. In any case, user space will need extra information to identify potential issues. > A quick look at a few MAC drivers seems to indicate many are getting > it wrong and don't actually wait for the result of the auto-neg.... Some ethernet driver trying to do own EEE state detection, and doing false positive detection on not supported states - for example half duplex.
> SmartEEE will be probably a bit more challenging. If MAC do not > advertise EEE support, SmartEEE can be enabled. But it would break PTP > if SmartEEE is active. Except SmartEEE capable PHY implements own PTP > support. In any case, user space will need extra information to > identify potential issues. If we have a MAC driver which does not implement the ethtool set_eee() and get_eee() ops, and a dev->phydev with the SmartEEE flag set, we could have net/ethtool/eee.c call direct into phylib. As for PTP and EEE, maybe we want the core PTP code to try calling get_eee() and at minimum issue a warning if it is enabled, if it thinks MAC PTP is being used? Andrew
changes v4: - remove following helpers: mmd_eee_cap_to_ethtool_sup_t mmd_eee_adv_to_ethtool_adv_t ethtool_adv_to_mmd_eee_adv_t and port drivers from this helpers to linkmode helpers. - rebase against latest net-next - port phy_init_eee() to genphy_c45_eee_is_active() changes v3: - rework some parts of EEE infrastructure and move it to c45 code. - add supported_eee storage and start using it in EEE code and by the micrel driver. - add EEE support for ar8035 PHY - add SmartEEE support to FEC i.MX series. changes v2: - use phydev->supported instead of reading MII_BMSR regiaster - fix @get_eee > @set_eee With this patch series we provide EEE control for KSZ9477 family of switches and AR8035 with i.MX6 configuration. According to my tests, on a system with KSZ8563 switch and 100Mbit idle link, we consume 0,192W less power per port if EEE is enabled. Oleksij Rempel (23): net: dsa: microchip: enable EEE support net: phy: add genphy_c45_read_eee_abilities() function net: phy: micrel: add ksz9477_get_features() net: phy: export phy_check_valid() function net: phy: add genphy_c45_ethtool_get/set_eee() support net: phy: c22: migrate to genphy_c45_write_eee_adv() net: phy: c45: migrate to genphy_c45_write_eee_adv() net: phy: migrate phy_init_eee() to genphy_c45_eee_is_active() net: phy: start using genphy_c45_ethtool_get/set_eee() net: phy: add driver specific get/set_eee support net: phy: at803x: implement ethtool access to SmartEEE functionality net: phy: at803x: ar8035: fix EEE support for half duplex links net: phy: add PHY specifica flag to signal SmartEEE support net: phy: at803x: add PHY_SMART_EEE flag to AR8035 net: phy: add phy_has_smarteee() helper net: fec: add support for PHYs with SmartEEE support e1000e: replace EEE ethtool helpers to linkmode variants igb: replace EEE ethtool helpers to linkmode variants igc: replace EEE ethtool helpers to linkmode variants tg3: replace EEE ethtool helpers to linkmode variants r8152: replace EEE ethtool helpers to linkmode variants net: usb: ax88179_178a: replace EEE ethtool helpers to linkmode variants net: mdio: drop EEE ethtool helpers in favor to linkmode variants drivers/net/dsa/microchip/ksz_common.c | 66 ++++ drivers/net/ethernet/broadcom/tg3.c | 9 +- drivers/net/ethernet/freescale/fec_main.c | 22 +- drivers/net/ethernet/intel/e1000e/ethtool.c | 16 +- drivers/net/ethernet/intel/igb/igb_ethtool.c | 23 +- drivers/net/ethernet/intel/igc/igc_ethtool.c | 12 +- drivers/net/phy/at803x.c | 142 ++++++++- drivers/net/phy/micrel.c | 21 ++ drivers/net/phy/phy-c45.c | 298 ++++++++++++++++++- drivers/net/phy/phy.c | 155 ++-------- drivers/net/phy/phy_device.c | 26 +- drivers/net/usb/ax88179_178a.c | 24 +- drivers/net/usb/r8152.c | 34 ++- include/linux/mdio.h | 136 ++++----- include/linux/phy.h | 28 ++ include/uapi/linux/mdio.h | 8 + 16 files changed, 760 insertions(+), 260 deletions(-)