| Message ID | 20221022220158.74933-1-steven.hsieh@broadcom.com (mailing list archive) |
|---|---|
| State | Changes Requested |
| Delegated to: | Netdev Maintainers |
| Headers | show |
| Series | [net-next] bonding: 3ad: bonding of links with different data rate | expand |
Steven Hsieh <steven.hsieh@broadcom.com> wrote: >Current Linux Bonding driver supports IEEE802.3ad-2000. >Operation across multiple data rates— >All links in a Link Aggregation Group operate at the same data rate. > >In IEEE802.1AX-2014 >Aggregation of links of different data rates is not prohibited >nor required by this standard. The -2014 and -2020 versions change a lot of things at once; I'm not sure we can just cherry pick out one thing (or maybe we can, I'm reading through the changes). Notably, the -2020 version states, in reference to changes added at -2014, "[...] it explicitly allowed the aggregation of point-to-point links of any speed using any physical media or logical connection capable of supporting the Internal Sublayer Service specified in IEEE Std 802.1AC." whereas the -2008 standard specifies "CSMA/CD MACs" instead of the ISS from 802.1AC. I'm not yet sure if this makes any relevant difference. >This patch provides configuration option to allow aggregation of links >with different speed. Have you tested all of the edge cases? E.g., what is the behavior with and without the option enabled when an interface in an aggregator changes its speed? If you have tests, consider including test scripts in tools/testing/selftests/drivers/net/bonding/ >Enhancement is disabled by default and can be enabled thru > echo 1 > /sys/class/net/bond*/bonding/async_linkspeed New option settings like this require (a) support in iproute2 (to set/get the option like any other bonding option), and (b) updates to the documentation (Documentation/networking/bonding.rst). I'm not completely sold on the name, either, "async" doesn't really describe "differing data rates" in my mind. Perhaps an option named "ad_link_speed" with allowed values of "same" or "any"? -J >Signed-off-by: Steven Hsieh <steven.hsieh@broadcom.com> > >--- > > drivers/net/bonding/bond_3ad.c | 12 +++++++++++- > drivers/net/bonding/bond_options.c | 26 ++++++++++++++++++++++++++ > drivers/net/bonding/bond_sysfs.c | 15 +++++++++++++++ > include/net/bond_options.h | 1 + > include/net/bonding.h | 1 + > 5 files changed, 54 insertions(+), 1 deletion(-) > >diff --git a/drivers/net/bonding/bond_3ad.c b/drivers/net/bonding/bond_3ad.c >index e58a1e0cadd2..f5689dae88c3 100644 >--- a/drivers/net/bonding/bond_3ad.c >+++ b/drivers/net/bonding/bond_3ad.c >@@ -385,6 +385,13 @@ static void __ad_actor_update_port(struct port *port) > port->actor_system_priority = BOND_AD_INFO(bond).system.sys_priority; > } > >+static inline u32 __get_agg_async_linkspeed(struct port *port) >+{ >+ const struct bonding *bond = bond_get_bond_by_slave(port->slave); >+ >+ return (bond) ? bond->params.async_linkspeed : 0; >+} >+ > /* Conversions */ > > /** >@@ -2476,7 +2483,10 @@ static void ad_update_actor_keys(struct port *port, bool reset) > speed = __get_link_speed(port); > ospeed = (old_oper_key & AD_SPEED_KEY_MASKS) >> 1; > duplex = __get_duplex(port); >- port->actor_admin_port_key |= (speed << 1) | duplex; >+ if (__get_agg_async_linkspeed(port)) >+ port->actor_admin_port_key |= duplex; >+ else >+ port->actor_admin_port_key |= (speed << 1) | duplex; > } > port->actor_oper_port_key = port->actor_admin_port_key; > >diff --git a/drivers/net/bonding/bond_options.c b/drivers/net/bonding/bond_options.c >index 3498db1c1b3c..cd871075b85c 100644 >--- a/drivers/net/bonding/bond_options.c >+++ b/drivers/net/bonding/bond_options.c >@@ -84,6 +84,8 @@ static int bond_option_ad_user_port_key_set(struct bonding *bond, > const struct bond_opt_value *newval); > static int bond_option_missed_max_set(struct bonding *bond, > const struct bond_opt_value *newval); >+static int bond_option_async_linkspeed_set(struct bonding *bond, >+ const struct bond_opt_value *newval); > > > static const struct bond_opt_value bond_mode_tbl[] = { >@@ -226,6 +228,12 @@ static const struct bond_opt_value bond_missed_max_tbl[] = { > { NULL, -1, 0}, > }; > >+static const struct bond_opt_value bond_async_linkspeed_tbl[] = { >+ { "off", 0, BOND_VALFLAG_DEFAULT}, >+ { "on", 1, 0}, >+ { NULL, -1, 0}, >+}; >+ > static const struct bond_option bond_opts[BOND_OPT_LAST] = { > [BOND_OPT_MODE] = { > .id = BOND_OPT_MODE, >@@ -360,6 +368,14 @@ static const struct bond_option bond_opts[BOND_OPT_LAST] = { > .values = bond_num_peer_notif_tbl, > .set = bond_option_num_peer_notif_set > }, >+ [BOND_OPT_ASYNC_LINKSPEED] = { >+ .id = BOND_OPT_ASYNC_LINKSPEED, >+ .name = "async_linkspeed", >+ .desc = "Enable aggregation of links of different data rates", >+ .unsuppmodes = BOND_MODE_ALL_EX(BIT(BOND_MODE_8023AD)), >+ .values = bond_async_linkspeed_tbl, >+ .set = bond_option_async_linkspeed_set >+ }, > [BOND_OPT_MIIMON] = { > .id = BOND_OPT_MIIMON, > .name = "miimon", >@@ -1702,3 +1718,13 @@ static int bond_option_ad_user_port_key_set(struct bonding *bond, > bond->params.ad_user_port_key = newval->value; > return 0; > } >+ >+static int bond_option_async_linkspeed_set(struct bonding *bond, >+ const struct bond_opt_value *newval) >+{ >+ netdev_info(bond->dev, "Setting async_linkspeed to %s (%llu)\n", >+ newval->string, newval->value); >+ bond->params.async_linkspeed = newval->value; >+ >+ return 0; >+} >diff --git a/drivers/net/bonding/bond_sysfs.c b/drivers/net/bonding/bond_sysfs.c >index 8996bd0a194a..6a0b4e1098af 100644 >--- a/drivers/net/bonding/bond_sysfs.c >+++ b/drivers/net/bonding/bond_sysfs.c >@@ -753,6 +753,20 @@ static ssize_t bonding_show_ad_user_port_key(struct device *d, > static DEVICE_ATTR(ad_user_port_key, 0644, > bonding_show_ad_user_port_key, bonding_sysfs_store_option); > >+static ssize_t bonding_show_async_linkspeed(struct device *d, >+ struct device_attribute *attr, >+ char *buf) >+{ >+ struct bonding *bond = to_bond(d); >+ const struct bond_opt_value *val; >+ >+ val = bond_opt_get_val(BOND_OPT_ASYNC_LINKSPEED, bond->params.async_linkspeed); >+ >+ return sprintf(buf, "%s %d\n", val->string, bond->params.async_linkspeed); >+} >+static DEVICE_ATTR(async_linkspeed, (00400 | 00040 | 00004) | 00200, /*S_IRUGO | S_IWUSR,*/ >+ bonding_show_async_linkspeed, bonding_sysfs_store_option); >+ > static struct attribute *per_bond_attrs[] = { > &dev_attr_slaves.attr, > &dev_attr_mode.attr, >@@ -792,6 +806,7 @@ static struct attribute *per_bond_attrs[] = { > &dev_attr_ad_actor_system.attr, > &dev_attr_ad_user_port_key.attr, > &dev_attr_arp_missed_max.attr, >+ &dev_attr_async_linkspeed.attr, > NULL, > }; > >diff --git a/include/net/bond_options.h b/include/net/bond_options.h >index 69292ecc0325..5b33f8b3e1c7 100644 >--- a/include/net/bond_options.h >+++ b/include/net/bond_options.h >@@ -76,6 +76,7 @@ enum { > BOND_OPT_MISSED_MAX, > BOND_OPT_NS_TARGETS, > BOND_OPT_PRIO, >+ BOND_OPT_ASYNC_LINKSPEED, > BOND_OPT_LAST > }; > >diff --git a/include/net/bonding.h b/include/net/bonding.h >index e999f851738b..5d83daab0669 100644 >--- a/include/net/bonding.h >+++ b/include/net/bonding.h >@@ -146,6 +146,7 @@ struct bond_params { > int lp_interval; > int packets_per_slave; > int tlb_dynamic_lb; >+ int async_linkspeed; > struct reciprocal_value reciprocal_packets_per_slave; > u16 ad_actor_sys_prio; > u16 ad_user_port_key; >-- >2.34.1 --- -Jay Vosburgh, jay.vosburgh@canonical.com
Thanks Jay for reviewing patch request. We decided not to move forward. Instead of adding different speed links to one aggregator, we can assign them to multiple aggregators and use ad_select=bandwidth to pick highest bandwidth aggregator. Thanks Steven H. > From: Jay Vosburgh <jay.vosburgh@canonical.com> > To: Steven Hsieh <steven.hsieh@broadcom.com> > Cc: Andy Gospodarek <andy@greyhouse.net>, > "David S. Miller" <davem@davemloft.net>, > Eric Dumazet <edumazet@google.com>, > Jakub Kicinski <kuba@kernel.org>, Paolo Abeni <pabeni@redhat.com>, > Veaceslav Falico <vfalico@gmail.com>, > linux-kernel@vger.kernel.org, netdev@vger.kernel.org > Subject: Re: [PATCH net-next] bonding: 3ad: bonding of links with different data rate > Date: Mon, 24 Oct 2022 09:25:09 -0700 [thread overview] > Message-ID: <15633.1666628709@famine> (raw) > In-Reply-To: <20221022220158.74933-1-steven.hsieh@broadcom.com> > > Steven Hsieh <steven.hsieh@broadcom.com> wrote: > > >Current Linux Bonding driver supports IEEE802.3ad-2000. > >Operation across multiple data rates— > >All links in a Link Aggregation Group operate at the same data rate. > > > >In IEEE802.1AX-2014 > >Aggregation of links of different data rates is not prohibited > >nor required by this standard. > > The -2014 and -2020 versions change a lot of things at once; I'm > not sure we can just cherry pick out one thing (or maybe we can, I'm > reading through the changes). Notably, the -2020 version states, in > reference to changes added at -2014, > > "[...] it explicitly allowed the aggregation of point-to-point links of > any speed using any physical media or logical connection capable of > supporting the Internal Sublayer Service specified in IEEE Std > 802.1AC." > > whereas the -2008 standard specifies "CSMA/CD MACs" instead of > the ISS from 802.1AC. I'm not yet sure if this makes any relevant > difference. > >This patch provides configuration option to allow aggregation of links > >with different speed. > > Have you tested all of the edge cases? E.g., what is the > behavior with and without the option enabled when an interface in an > aggregator changes its speed? > > If you have tests, consider including test scripts in > tools/testing/selftests/drivers/net/bonding/ > In current code, when 2nd port is linked up with different speed, it will not be part of active aggregator. > >Enhancement is disabled by default and can be enabled thru > > echo 1 > /sys/class/net/bond*/bonding/async_linkspeed > > New option settings like this require (a) support in iproute2 > (to set/get the option like any other bonding option), and (b) updates > to the documentation (Documentation/networking/bonding.rst). > > I'm not completely sold on the name, either, "async" doesn't > really describe "differing data rates" in my mind. Perhaps an option > named "ad_link_speed" with allowed values of "same" or "any"?
diff --git a/drivers/net/bonding/bond_3ad.c b/drivers/net/bonding/bond_3ad.c index e58a1e0cadd2..f5689dae88c3 100644 --- a/drivers/net/bonding/bond_3ad.c +++ b/drivers/net/bonding/bond_3ad.c @@ -385,6 +385,13 @@ static void __ad_actor_update_port(struct port *port) port->actor_system_priority = BOND_AD_INFO(bond).system.sys_priority; } +static inline u32 __get_agg_async_linkspeed(struct port *port) +{ + const struct bonding *bond = bond_get_bond_by_slave(port->slave); + + return (bond) ? bond->params.async_linkspeed : 0; +} + /* Conversions */ /** @@ -2476,7 +2483,10 @@ static void ad_update_actor_keys(struct port *port, bool reset) speed = __get_link_speed(port); ospeed = (old_oper_key & AD_SPEED_KEY_MASKS) >> 1; duplex = __get_duplex(port); - port->actor_admin_port_key |= (speed << 1) | duplex; + if (__get_agg_async_linkspeed(port)) + port->actor_admin_port_key |= duplex; + else + port->actor_admin_port_key |= (speed << 1) | duplex; } port->actor_oper_port_key = port->actor_admin_port_key; diff --git a/drivers/net/bonding/bond_options.c b/drivers/net/bonding/bond_options.c index 3498db1c1b3c..cd871075b85c 100644 --- a/drivers/net/bonding/bond_options.c +++ b/drivers/net/bonding/bond_options.c @@ -84,6 +84,8 @@ static int bond_option_ad_user_port_key_set(struct bonding *bond, const struct bond_opt_value *newval); static int bond_option_missed_max_set(struct bonding *bond, const struct bond_opt_value *newval); +static int bond_option_async_linkspeed_set(struct bonding *bond, + const struct bond_opt_value *newval); static const struct bond_opt_value bond_mode_tbl[] = { @@ -226,6 +228,12 @@ static const struct bond_opt_value bond_missed_max_tbl[] = { { NULL, -1, 0}, }; +static const struct bond_opt_value bond_async_linkspeed_tbl[] = { + { "off", 0, BOND_VALFLAG_DEFAULT}, + { "on", 1, 0}, + { NULL, -1, 0}, +}; + static const struct bond_option bond_opts[BOND_OPT_LAST] = { [BOND_OPT_MODE] = { .id = BOND_OPT_MODE, @@ -360,6 +368,14 @@ static const struct bond_option bond_opts[BOND_OPT_LAST] = { .values = bond_num_peer_notif_tbl, .set = bond_option_num_peer_notif_set }, + [BOND_OPT_ASYNC_LINKSPEED] = { + .id = BOND_OPT_ASYNC_LINKSPEED, + .name = "async_linkspeed", + .desc = "Enable aggregation of links of different data rates", + .unsuppmodes = BOND_MODE_ALL_EX(BIT(BOND_MODE_8023AD)), + .values = bond_async_linkspeed_tbl, + .set = bond_option_async_linkspeed_set + }, [BOND_OPT_MIIMON] = { .id = BOND_OPT_MIIMON, .name = "miimon", @@ -1702,3 +1718,13 @@ static int bond_option_ad_user_port_key_set(struct bonding *bond, bond->params.ad_user_port_key = newval->value; return 0; } + +static int bond_option_async_linkspeed_set(struct bonding *bond, + const struct bond_opt_value *newval) +{ + netdev_info(bond->dev, "Setting async_linkspeed to %s (%llu)\n", + newval->string, newval->value); + bond->params.async_linkspeed = newval->value; + + return 0; +} diff --git a/drivers/net/bonding/bond_sysfs.c b/drivers/net/bonding/bond_sysfs.c index 8996bd0a194a..6a0b4e1098af 100644 --- a/drivers/net/bonding/bond_sysfs.c +++ b/drivers/net/bonding/bond_sysfs.c @@ -753,6 +753,20 @@ static ssize_t bonding_show_ad_user_port_key(struct device *d, static DEVICE_ATTR(ad_user_port_key, 0644, bonding_show_ad_user_port_key, bonding_sysfs_store_option); +static ssize_t bonding_show_async_linkspeed(struct device *d, + struct device_attribute *attr, + char *buf) +{ + struct bonding *bond = to_bond(d); + const struct bond_opt_value *val; + + val = bond_opt_get_val(BOND_OPT_ASYNC_LINKSPEED, bond->params.async_linkspeed); + + return sprintf(buf, "%s %d\n", val->string, bond->params.async_linkspeed); +} +static DEVICE_ATTR(async_linkspeed, (00400 | 00040 | 00004) | 00200, /*S_IRUGO | S_IWUSR,*/ + bonding_show_async_linkspeed, bonding_sysfs_store_option); + static struct attribute *per_bond_attrs[] = { &dev_attr_slaves.attr, &dev_attr_mode.attr, @@ -792,6 +806,7 @@ static struct attribute *per_bond_attrs[] = { &dev_attr_ad_actor_system.attr, &dev_attr_ad_user_port_key.attr, &dev_attr_arp_missed_max.attr, + &dev_attr_async_linkspeed.attr, NULL, }; diff --git a/include/net/bond_options.h b/include/net/bond_options.h index 69292ecc0325..5b33f8b3e1c7 100644 --- a/include/net/bond_options.h +++ b/include/net/bond_options.h @@ -76,6 +76,7 @@ enum { BOND_OPT_MISSED_MAX, BOND_OPT_NS_TARGETS, BOND_OPT_PRIO, + BOND_OPT_ASYNC_LINKSPEED, BOND_OPT_LAST }; diff --git a/include/net/bonding.h b/include/net/bonding.h index e999f851738b..5d83daab0669 100644 --- a/include/net/bonding.h +++ b/include/net/bonding.h @@ -146,6 +146,7 @@ struct bond_params { int lp_interval; int packets_per_slave; int tlb_dynamic_lb; + int async_linkspeed; struct reciprocal_value reciprocal_packets_per_slave; u16 ad_actor_sys_prio; u16 ad_user_port_key;
Current Linux Bonding driver supports IEEE802.3ad-2000. Operation across multiple data rates— All links in a Link Aggregation Group operate at the same data rate. In IEEE802.1AX-2014 Aggregation of links of different data rates is not prohibited nor required by this standard. This patch provides configuration option to allow aggregation of links with different speed. Enhancement is disabled by default and can be enabled thru echo 1 > /sys/class/net/bond*/bonding/async_linkspeed Signed-off-by: Steven Hsieh <steven.hsieh@broadcom.com> --- drivers/net/bonding/bond_3ad.c | 12 +++++++++++- drivers/net/bonding/bond_options.c | 26 ++++++++++++++++++++++++++ drivers/net/bonding/bond_sysfs.c | 15 +++++++++++++++ include/net/bond_options.h | 1 + include/net/bonding.h | 1 + 5 files changed, 54 insertions(+), 1 deletion(-)