| Message ID | cover.1721851988.git.pabeni@redhat.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | net: introduce TX shaping H/W offload API | expand |
Wed, Jul 24, 2024 at 10:24:46PM CEST, pabeni@redhat.com wrote: >We have a plurality of shaping-related drivers API, but none flexible >enough to meet existing demand from vendors[1]. > >This series introduces new device APIs to configure in a flexible way >TX shaping H/W offload. The new functionality is exposed via a newly From what I understand, and please correct me if I'm wrong, this patchset is about HW shaper configuration. Basically it provides new UAPI to configure the HW shaper. So Why you say "offload"? I don't see anything being offloaded here. Also, from the previous discussions, I gained impression that the goal of this work is to replace multiple driver apis for the shaper and consolidate it under new one. I don't see anything like this in this patchset. Do you plan it as a follow-up? Do you have RFC for that step as well? >defined generic netlink interface and includes introspection >capabilities. Some self-tests are included, on top of a dummy >netdevsim implementation, and a basic implementation for the iavf >driver. > >Some usage examples: > >* Configure shaping on a given queue: > >./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \ > --do set --json '{"ifindex":'$IFINDEX', > "shaper": {"handle": > {"scope": "queue", "id":'$QUEUEID' }, > "bw-max": 2000000 }}' > >* Container B/W sharing > >The orchestration infrastructure wants to group the >container-related queues under a RR scheduling and limit the aggregate >bandwidth: > >./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \ > --do group --json '{"ifindex":'$IFINDEX', > "inputs": [ > {"handle": {"scope": "queue", "id":'$QID1' }, > "weight": '$W1'}, > {"handle": {"scope": "queue", "id":'$QID2' }, > "weight": '$W2'}], > {"handle": {"scope": "queue", "id":'$QID3' }, > "weight": '$W3'}], > "output": { "handle": {"scope":"netdev"}, > "output": { "handle": {"scope":"netdev"}, > "bw-max": 10000000}}' >{'handle': {'id': 0, 'scope': 'netdev'}} > >* Delegation > >A container wants to set a B/W limit on 2 of its own queues: > >./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \ > --do group --json '{"ifindex":'$IFINDEX', > "inputs": [ > {"handle": {"scope": "queue", "id":'$QID1' }, > "weight": '$W1'}, > {"handle": {"scope": "queue", "id":'$QID2' }, > "weight": '$W2'}], > "output": { "handle": {"scope":"detached"}, > "bw-max": 5000000}}' >{'handle': {'id': 0, 'scope': 'detached'}} > >* Cleanup: > >Deleting a single queue shaper: > >./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \ > --do delete --json \ > '{"ifindex":'$IFINDEX', > "handle": {"scope": "queue", "id":'$QID1' }}' > >Deleting the last shaper under a group deletes the group, too: > >./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \ > --do delete --json \ > '{"ifindex":'$IFINDEX', > "handle": {"scope": "queue", "id":'$QID2' }}' >./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \ > --do get --json '{"ifindex":'$IF', > "handle": { "scope": "detached", "id": 0}}' >Netlink error: Invalid argument >nl_len = 80 (64) nl_flags = 0x300 nl_type = 2 > error: -22 > extack: {'msg': "Can't find shaper for handle 10000000"} > >Changes from RFC v1: > - set() and delete() ops operate on a single shaper > - added group() op to allow grouping and nesting > - split the NL implementation into multiple patches to help reviewing > >RFC v1: https://lore.kernel.org/netdev/cover.1719518113.git.pabeni@redhat.com/ > >[1] https://lore.kernel.org/netdev/20240405102313.GA310894@kernel.org/ > >Paolo Abeni (7): > netlink: spec: add shaper YAML spec > net-shapers: implement NL get operation > net-shapers: implement NL set and delete operations > net-shapers: implement NL group operation > netlink: spec: add shaper introspection support > net: shaper: implement introspection support > testing: net-drv: add basic shaper test > >Sudheer Mogilappagari (2): > iavf: Add net_shaper_ops support > iavf: add support to exchange qos capabilities > >Wenjun Wu (2): > virtchnl: support queue rate limit and quanta size configuration > ice: Support VF queue rate limit and quanta size configuration > > Documentation/netlink/specs/shaper.yaml | 337 ++++++ > Documentation/networking/kapi.rst | 3 + > MAINTAINERS | 1 + > drivers/net/Kconfig | 1 + > drivers/net/ethernet/intel/Kconfig | 1 + > drivers/net/ethernet/intel/iavf/iavf.h | 13 + > drivers/net/ethernet/intel/iavf/iavf_main.c | 215 +++- > drivers/net/ethernet/intel/iavf/iavf_txrx.h | 2 + > .../net/ethernet/intel/iavf/iavf_virtchnl.c | 157 ++- > drivers/net/ethernet/intel/ice/ice.h | 2 + > drivers/net/ethernet/intel/ice/ice_base.c | 2 + > drivers/net/ethernet/intel/ice/ice_common.c | 21 + > .../net/ethernet/intel/ice/ice_hw_autogen.h | 8 + > drivers/net/ethernet/intel/ice/ice_txrx.h | 1 + > drivers/net/ethernet/intel/ice/ice_type.h | 1 + > drivers/net/ethernet/intel/ice/ice_vf_lib.h | 8 + > drivers/net/ethernet/intel/ice/ice_virtchnl.c | 333 ++++++ > drivers/net/ethernet/intel/ice/ice_virtchnl.h | 11 + > .../intel/ice/ice_virtchnl_allowlist.c | 6 + > drivers/net/netdevsim/netdev.c | 37 + > include/linux/avf/virtchnl.h | 119 +++ > include/linux/netdevice.h | 17 + > include/net/net_shaper.h | 169 +++ > include/uapi/linux/net_shaper.h | 91 ++ > net/Kconfig | 3 + > net/Makefile | 1 + > net/core/dev.c | 2 + > net/core/dev.h | 6 + > net/shaper/Makefile | 9 + > net/shaper/shaper.c | 962 ++++++++++++++++++ > net/shaper/shaper_nl_gen.c | 142 +++ > net/shaper/shaper_nl_gen.h | 30 + > tools/testing/selftests/drivers/net/Makefile | 1 + > tools/testing/selftests/drivers/net/shaper.py | 267 +++++ > .../testing/selftests/net/lib/py/__init__.py | 1 + > tools/testing/selftests/net/lib/py/ynl.py | 5 + > 36 files changed, 2983 insertions(+), 2 deletions(-) > create mode 100644 Documentation/netlink/specs/shaper.yaml > create mode 100644 include/net/net_shaper.h > create mode 100644 include/uapi/linux/net_shaper.h > create mode 100644 net/shaper/Makefile > create mode 100644 net/shaper/shaper.c > create mode 100644 net/shaper/shaper_nl_gen.c > create mode 100644 net/shaper/shaper_nl_gen.h > create mode 100755 tools/testing/selftests/drivers/net/shaper.py > >-- >2.45.2 >
On 7/29/24 08:30, Jiri Pirko wrote: > From what I understand, and please correct me if I'm wrong, this > patchset is about HW shaper configuration. Basically it provides new UAPI > to configure the HW shaper. So Why you say "offload"? I don't see > anything being offloaded here. The offload part comes from the initial, very old tentative solution. I guess we can change the title to "net: introduce TX H/W shaping API" > Also, from the previous discussions, I gained impression that the goal > of this work is to replace multiple driver apis for the shaper and > consolidate it under new one. I don't see anything like this in this > patchset. Do you plan it as a follow-up? Do you have RFC for that step > as well? The general idea is, with time, to leverage this API to replace others H/W shaping related in-kernel interfaces. At least ndo_set_tx_maxrate() should be quite straight-forward, after that the relevant device drivers have implemented (very limited) support for this API. The latter will need some effort from the drivers' owners. Thanks, Paolo
Mon, Jul 29, 2024 at 04:42:19PM CEST, pabeni@redhat.com wrote: >On 7/29/24 08:30, Jiri Pirko wrote: >> From what I understand, and please correct me if I'm wrong, this >> patchset is about HW shaper configuration. Basically it provides new UAPI >> to configure the HW shaper. So Why you say "offload"? I don't see >> anything being offloaded here. > >The offload part comes from the initial, very old tentative solution. I guess >we can change the title to "net: introduce TX H/W shaping API" > >> Also, from the previous discussions, I gained impression that the goal >> of this work is to replace multiple driver apis for the shaper and >> consolidate it under new one. I don't see anything like this in this >> patchset. Do you plan it as a follow-up? Do you have RFC for that step >> as well? > >The general idea is, with time, to leverage this API to replace others H/W >shaping related in-kernel interfaces. > >At least ndo_set_tx_maxrate() should be quite straight-forward, after that >the relevant device drivers have implemented (very limited) support for this >API. Could you try to draft at least one example per each user? I mean, this is likely to be the tricky part of this work, would be great to make that click from very beginning. > >The latter will need some effort from the drivers' owners. Let me know what you need exactly. Will try to do my best to help. Thanks! > >Thanks, > >Paolo >
On 7/29/24 17:13, Jiri Pirko wrote: > Mon, Jul 29, 2024 at 04:42:19PM CEST, pabeni@redhat.com wrote: >> The general idea is, with time, to leverage this API to replace others H/W >> shaping related in-kernel interfaces. >> >> At least ndo_set_tx_maxrate() should be quite straight-forward, after that >> the relevant device drivers have implemented (very limited) support for this >> API. > > Could you try to draft at least one example per each user? I mean, this > is likely to be the tricky part of this work, would be great to make > that click from very beginning. I think we need to clarify that we are not going to replace all the existing in-kernel interfaces that somewhat configure shapers on the network devices. Trying to achieve the above would be a (not so) nice way to block this effort forever. ndo_set_tx_maxrate() is IMHO a good, feasible example. Others could be ieee_setmaxrate() or ndo_set_vf_rate() - ignoring the “obsoleted” argument. >> The latter will need some effort from the drivers' owners. > > Let me know what you need exactly. Will try to do my best to help. Something like what is implemented in this series for the iavf driver would suffice. Thanks, Paolo
We have a plurality of shaping-related drivers API, but none flexible enough to meet existing demand from vendors[1]. This series introduces new device APIs to configure in a flexible way TX shaping H/W offload. The new functionality is exposed via a newly defined generic netlink interface and includes introspection capabilities. Some self-tests are included, on top of a dummy netdevsim implementation, and a basic implementation for the iavf driver. Some usage examples: * Configure shaping on a given queue: ./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \ --do set --json '{"ifindex":'$IFINDEX', "shaper": {"handle": {"scope": "queue", "id":'$QUEUEID' }, "bw-max": 2000000 }}' * Container B/W sharing The orchestration infrastructure wants to group the container-related queues under a RR scheduling and limit the aggregate bandwidth: ./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \ --do group --json '{"ifindex":'$IFINDEX', "inputs": [ {"handle": {"scope": "queue", "id":'$QID1' }, "weight": '$W1'}, {"handle": {"scope": "queue", "id":'$QID2' }, "weight": '$W2'}], {"handle": {"scope": "queue", "id":'$QID3' }, "weight": '$W3'}], "output": { "handle": {"scope":"netdev"}, "output": { "handle": {"scope":"netdev"}, "bw-max": 10000000}}' {'handle': {'id': 0, 'scope': 'netdev'}} * Delegation A container wants to set a B/W limit on 2 of its own queues: ./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \ --do group --json '{"ifindex":'$IFINDEX', "inputs": [ {"handle": {"scope": "queue", "id":'$QID1' }, "weight": '$W1'}, {"handle": {"scope": "queue", "id":'$QID2' }, "weight": '$W2'}], "output": { "handle": {"scope":"detached"}, "bw-max": 5000000}}' {'handle': {'id': 0, 'scope': 'detached'}} * Cleanup: Deleting a single queue shaper: ./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \ --do delete --json \ '{"ifindex":'$IFINDEX', "handle": {"scope": "queue", "id":'$QID1' }}' Deleting the last shaper under a group deletes the group, too: ./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \ --do delete --json \ '{"ifindex":'$IFINDEX', "handle": {"scope": "queue", "id":'$QID2' }}' ./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \ --do get --json '{"ifindex":'$IF', "handle": { "scope": "detached", "id": 0}}' Netlink error: Invalid argument nl_len = 80 (64) nl_flags = 0x300 nl_type = 2 error: -22 extack: {'msg': "Can't find shaper for handle 10000000"} Changes from RFC v1: - set() and delete() ops operate on a single shaper - added group() op to allow grouping and nesting - split the NL implementation into multiple patches to help reviewing RFC v1: https://lore.kernel.org/netdev/cover.1719518113.git.pabeni@redhat.com/ [1] https://lore.kernel.org/netdev/20240405102313.GA310894@kernel.org/ Paolo Abeni (7): netlink: spec: add shaper YAML spec net-shapers: implement NL get operation net-shapers: implement NL set and delete operations net-shapers: implement NL group operation netlink: spec: add shaper introspection support net: shaper: implement introspection support testing: net-drv: add basic shaper test Sudheer Mogilappagari (2): iavf: Add net_shaper_ops support iavf: add support to exchange qos capabilities Wenjun Wu (2): virtchnl: support queue rate limit and quanta size configuration ice: Support VF queue rate limit and quanta size configuration Documentation/netlink/specs/shaper.yaml | 337 ++++++ Documentation/networking/kapi.rst | 3 + MAINTAINERS | 1 + drivers/net/Kconfig | 1 + drivers/net/ethernet/intel/Kconfig | 1 + drivers/net/ethernet/intel/iavf/iavf.h | 13 + drivers/net/ethernet/intel/iavf/iavf_main.c | 215 +++- drivers/net/ethernet/intel/iavf/iavf_txrx.h | 2 + .../net/ethernet/intel/iavf/iavf_virtchnl.c | 157 ++- drivers/net/ethernet/intel/ice/ice.h | 2 + drivers/net/ethernet/intel/ice/ice_base.c | 2 + drivers/net/ethernet/intel/ice/ice_common.c | 21 + .../net/ethernet/intel/ice/ice_hw_autogen.h | 8 + drivers/net/ethernet/intel/ice/ice_txrx.h | 1 + drivers/net/ethernet/intel/ice/ice_type.h | 1 + drivers/net/ethernet/intel/ice/ice_vf_lib.h | 8 + drivers/net/ethernet/intel/ice/ice_virtchnl.c | 333 ++++++ drivers/net/ethernet/intel/ice/ice_virtchnl.h | 11 + .../intel/ice/ice_virtchnl_allowlist.c | 6 + drivers/net/netdevsim/netdev.c | 37 + include/linux/avf/virtchnl.h | 119 +++ include/linux/netdevice.h | 17 + include/net/net_shaper.h | 169 +++ include/uapi/linux/net_shaper.h | 91 ++ net/Kconfig | 3 + net/Makefile | 1 + net/core/dev.c | 2 + net/core/dev.h | 6 + net/shaper/Makefile | 9 + net/shaper/shaper.c | 962 ++++++++++++++++++ net/shaper/shaper_nl_gen.c | 142 +++ net/shaper/shaper_nl_gen.h | 30 + tools/testing/selftests/drivers/net/Makefile | 1 + tools/testing/selftests/drivers/net/shaper.py | 267 +++++ .../testing/selftests/net/lib/py/__init__.py | 1 + tools/testing/selftests/net/lib/py/ynl.py | 5 + 36 files changed, 2983 insertions(+), 2 deletions(-) create mode 100644 Documentation/netlink/specs/shaper.yaml create mode 100644 include/net/net_shaper.h create mode 100644 include/uapi/linux/net_shaper.h create mode 100644 net/shaper/Makefile create mode 100644 net/shaper/shaper.c create mode 100644 net/shaper/shaper_nl_gen.c create mode 100644 net/shaper/shaper_nl_gen.h create mode 100755 tools/testing/selftests/drivers/net/shaper.py