| Message ID | 1619162596-23846-1-git-send-email-moshe@nvidia.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | Extend module EEPROM API | expand |
> From: Moshe Shemesh [mailto:moshe@nvidia.com] > Sent: Friday, April 23, 2021 12:23 AM > To: Michal Kubecek <mkubecek@suse.cz>; Andrew Lunn > <andrew@lunn.ch>; Jakub Kicinski <kuba@kernel.org>; Don Bollinger > <don@thebollingers.org>; netdev@vger.kernel.org > Cc: Vladyslav Tarasiuk <vladyslavt@nvidia.com>; Moshe Shemesh > <moshe@nvidia.com> > Subject: [PATCH ethtool-next 0/4] Extend module EEPROM API > > Ethtool supports module EEPROM dumps via the `ethtool -m <dev>` > command. > But in current state its functionality is limited - offset and length parameters, > which are used to specify a linear desired region of EEPROM data to dump, is > not enough, considering emergence of complex module EEPROM layouts > such as CMIS 4.0. > > Moreover, CMIS 4.0 extends the amount of pages that may be accessible by > introducing another parameter for page addressing - banks. > Besides, currently module EEPROM is represented as a chunk of > concatenated pages, where lower 128 bytes of all pages, except page 00h, > are omitted. Offset and length are used to address parts of this fake linear > memory. But in practice drivers, which implement > get_module_info() and get_module_eeprom() ethtool ops still calculate > page number and set I2C address on their own. > > This series adds support in `ethtool -m` of dumping an arbitrary page > specified by page number, bank number and I2C address. Implement netlink > handler for `ethtool -m` in order to make such requests to the kernel and > extend CLI by adding corresponding parameters. > New command line format: > ethtool -m <dev> [hex on|off] [raw on|off] [offset N] [length N] [page N] > [bank N] [i2c N] > > Netlink infrastructure works on per-page basis and allows dumps of a single > page at once. But in case user requests human-readable output, which > currently may require more than one page, userspace can make such > additional calls to kernel on demand and place pages in a linked list. > It allows to get pages from cache on demand and pass them to refactored > SFF decoders. > > Vladyslav Tarasiuk (4): > ethtool: Add netlink handler for getmodule (-m) > ethtool: Refactor human-readable module EEPROM output for new API > ethtool: Rename QSFP-DD identifiers to use CMIS 4.0 > ethtool: Update manpages to reflect changes to getmodule (-m) command > > Makefile.am | 3 +- > qsfp-dd.c => cmis4.c | 220 +++++++++++--------- > cmis4.h | 128 ++++++++++++ > ethtool.8.in | 14 ++ > ethtool.c | 4 + > internal.h | 12 ++ > list.h | 34 ++++ > netlink/desc-ethtool.c | 13 ++ > netlink/extapi.h | 2 + > netlink/module-eeprom.c | 438 > ++++++++++++++++++++++++++++++++++++++++ > qsfp-dd.h | 125 ------------ > qsfp.c | 129 +++++++----- > qsfp.h | 52 ++--- > sff-common.c | 3 + > sff-common.h | 3 +- > 15 files changed, 876 insertions(+), 304 deletions(-) rename qsfp-dd.c => > cmis4.c (55%) create mode 100644 cmis4.h create mode 100644 list.h create > mode 100644 netlink/module-eeprom.c delete mode 100644 qsfp-dd.h > > -- > 2.26.2 Will there be an ethtool option to write to module eeprom in CMIS format? There are routine functions to configure the devices that require writing appropriate values to various registers. Byte 26 allows software control of low power mode, squelch and software reset. Page 10h is full of Lane and Data Path Control registers. Beyond the spec, but allowed by the spec, there are vendor specific capabilities like firmware download that require bulk write (up to 128 bytes per write). Using the full capabilities of these devices will require write capability. Ethtool is the only path that will be allowed. Don
> There are routine functions to configure the devices that require writing > appropriate values to various registers. Byte 26 allows software control of > low power mode, squelch and software reset. Page 10h is full of Lane and > Data Path Control registers. These all sounds like foot canons when in user space control. I would expect the MAC and or SFP driver to make use of these features, no need to export them to user space, at least not in a raw format. I could however imagine ethtool commands to manipulate specific features, passing the request to the MAC to perform, so it knows what is going on. > Beyond the spec, but allowed by the spec, there are vendor specific > capabilities like firmware download that require bulk write (up to 128 bytes > per write). This one is not so easy. Since it is vendor specific, we need to consider how to actually make it vendor generic from Ethtool, or maybe devlink. Maybe code in the kernel which matches on the vendor string in the SFP EEPROM, and provides a standardized API towards ethtool, and does whatever magic is needed towards the SFP. But it gets messy when you don't have direct access to the SFP, there is a layer of firmware in the middle, which is often the case. Andrew
> > There are routine functions to configure the devices that require > > writing appropriate values to various registers. Byte 26 allows > > software control of low power mode, squelch and software reset. Page > > 10h is full of Lane and Data Path Control registers. > > These all sounds like foot canons when in user space control. I would expect > the MAC and or SFP driver to make use of these features, no need to export > them to user space, at least not in a raw format. I could however imagine > ethtool commands to manipulate specific features, passing the request to > the MAC to perform, so it knows what is going on. > > > Beyond the spec, but allowed by the spec, there are vendor specific > > capabilities like firmware download that require bulk write (up to 128 > > bytes per write). > > This one is not so easy. Since it is vendor specific, we need to consider how to > actually make it vendor generic from Ethtool, or maybe devlink. Maybe code > in the kernel which matches on the vendor string in the SFP EEPROM, and > provides a standardized API towards ethtool, and does whatever magic is > needed towards the SFP. But it gets messy when you don't have direct > access to the SFP, there is a layer of firmware in the middle, which is often > the case. > > Andrew Here we go again... It is my experience that there are far more capabilities in these devices than will ever be captured in ethtool. Module vendors can provide additional value to their customers by putting innovative features into their modules, and providing software applications to take advantage of those features. These features don't necessarily impact the network stack. They may be used to draw additional diagnostic data from the devices, or to enable management features like flashing colored lights built into custom modules. I've written code to do these and more things which are unique to one vendor, and valued by their customers. I'm asking for a generic interface that allows read/write access to arbitrary registers. Put the warnings in the documentation, limit access to it, but make it available. Don
> Here we go again... It is my experience that there are far more > capabilities in these devices than will ever be captured in ethtool. Module > vendors can provide additional value to their customers by putting > innovative features into their modules, and providing software applications > to take advantage of those features. These features don't necessarily > impact the network stack. They may be used to draw additional diagnostic > data from the devices, or to enable management features like flashing > colored lights built into custom modules. I've written code to do these and > more things which are unique to one vendor, and valued by their customers. Sorry, but not going to happen. Ethernet PHYs can have lots of addition registers on stop of what 802.3 specifies. Vendors do add additional functionality. And we do support some of it, like configuring downshift, energy detect power down, cable diagnostics in a generic manor. And we are open to adding more such features. People can contribute code which multiple vendors might implement. We then have well documented APIs which are the same across different vendors. For LEDs you should be using the Linux LED class drivers. The Ethernet PHYs are slowly moving that way. Very slowly :-( Both MAC and Ethernet PHY drivers have tunables. I would suggest using this concept, but applied to SFPs. This is how most of the additional PHY features are supported. But first you need to add an SFP driver framework, which matches on the fields in the EEPROM to load the driver specific to an SFP. That then gives you a place to add such code. Andrew