| Message ID | 8857cb8138b33c8938782e2154a56b095d611d18.1678200041.git.lorenzo@kernel.org (mailing list archive) |
|---|---|
| State | Superseded |
| Delegated to: | Netdev Maintainers |
| Headers | show |
| Series | update xdp_features flag according to NIC re-configuration | expand |
On 07/03/2023 16:54, Lorenzo Bianconi wrote: > Take into account LRO and GRO configuration setting device xdp_features > flag. Moreover consider channel rq_wq_type enabling rx scatter-gatter > support in xdp_features flag. > > Fixes: 66c0e13ad236 ("drivers: net: turn on XDP features") > Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org> > --- > drivers/net/ethernet/mellanox/mlx5/core/en.h | 1 + > .../ethernet/mellanox/mlx5/core/en_ethtool.c | 10 ++++- > .../net/ethernet/mellanox/mlx5/core/en_main.c | 45 ++++++++++++++++--- > .../net/ethernet/mellanox/mlx5/core/en_rep.c | 3 ++ > 4 files changed, 51 insertions(+), 8 deletions(-) > > diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h > index 88460b7796e5..4276c6eb6820 100644 > --- a/drivers/net/ethernet/mellanox/mlx5/core/en.h > +++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h > @@ -1243,6 +1243,7 @@ void mlx5e_build_nic_params(struct mlx5e_priv *priv, struct mlx5e_xsk *xsk, u16 > void mlx5e_rx_dim_work(struct work_struct *work); > void mlx5e_tx_dim_work(struct work_struct *work); > > +void mlx5e_set_xdp_feature(struct net_device *netdev); > netdev_features_t mlx5e_features_check(struct sk_buff *skb, > struct net_device *netdev, > netdev_features_t features); > diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c > index 7708acc9b2ab..79fd21ecb9cb 100644 > --- a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c > +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c > @@ -1985,6 +1985,7 @@ static int set_pflag_rx_striding_rq(struct net_device *netdev, bool enable) > struct mlx5e_priv *priv = netdev_priv(netdev); > struct mlx5_core_dev *mdev = priv->mdev; > struct mlx5e_params new_params; > + int err; > > if (enable) { > /* Checking the regular RQ here; mlx5e_validate_xsk_param called > @@ -2005,7 +2006,14 @@ static int set_pflag_rx_striding_rq(struct net_device *netdev, bool enable) > MLX5E_SET_PFLAG(&new_params, MLX5E_PFLAG_RX_STRIDING_RQ, enable); > mlx5e_set_rq_type(mdev, &new_params); > > - return mlx5e_safe_switch_params(priv, &new_params, NULL, NULL, true); > + err = mlx5e_safe_switch_params(priv, &new_params, NULL, NULL, true); > + if (err) > + return err; > + > + /* update XDP supported features */ > + mlx5e_set_xdp_feature(netdev); > + > + return 0; > } > > static int set_pflag_rx_no_csum_complete(struct net_device *netdev, bool enable) > diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c > index 76a9c5194a70..1b68dd2be2c5 100644 > --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c > +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c > @@ -4004,6 +4004,30 @@ static int mlx5e_handle_feature(struct net_device *netdev, > return 0; > } > > +void mlx5e_set_xdp_feature(struct net_device *netdev) > +{ > + struct mlx5e_priv *priv = netdev_priv(netdev); > + bool ndo_xmit = test_bit(MLX5E_STATE_XDP_ACTIVE, &priv->state); Our driver doesn't require loading a dummy XDP program to have the redirect-in ability. It's always there. I actually have a bug fix under internal review with Saeed that addresses this. In addition, it cleans up the NETDEV_XDP_ACT_NDO_XMIT_SG as we do not support it yet. I have a series that's adding support and will submit it soon. Any reason you're submitting these fixes to net-next rather than net? Maybe it'd be better if we integrate the patches, here's my fix (still under review...): Author: Tariq Toukan <tariqt@nvidia.com> Date: Thu Feb 23 08:58:04 2023 +0200 net/mlx5e: Fix exposed xdp_features Always declare NETDEV_XDP_ACT_NDO_XMIT as the ndo_xdp_xmit callback is always functional per our design, and does not require loading a dummy xdp program. Although non-linear XDP buffer is supported for XDP_TX flow, do not declare NETDEV_XDP_ACT_NDO_XMIT_SG as it is yet supported for redirected-in frames. Fixes: 66c0e13ad236 ("drivers: net: turn on XDP features") Signed-off-by: Tariq Toukan <tariqt@nvidia.com> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c index 53feb0529943..9a5d3ce1fbcd 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c @@ -4741,13 +4741,6 @@ static int mlx5e_xdp_set(struct net_device *netdev, struct bpf_prog *prog) if (old_prog) bpf_prog_put(old_prog); - if (reset) { - if (prog) - xdp_features_set_redirect_target(netdev, true); - else - xdp_features_clear_redirect_target(netdev); - } - if (!test_bit(MLX5E_STATE_OPENED, &priv->state) || reset) goto unlock; @@ -5144,6 +5137,7 @@ static void mlx5e_build_nic_netdev(struct net_device *netdev) netdev->features |= NETIF_F_HW_VLAN_STAG_FILTER; netdev->xdp_features = NETDEV_XDP_ACT_BASIC | NETDEV_XDP_ACT_REDIRECT | + NETDEV_XDP_ACT_NDO_XMIT | NETDEV_XDP_ACT_XSK_ZEROCOPY | NETDEV_XDP_ACT_RX_SG; > + struct mlx5e_params *params = &priv->channels.params; > + xdp_features_t val; > + > + if (params->packet_merge.type != MLX5E_PACKET_MERGE_NONE) { > + xdp_clear_features_flag(netdev); > + return; > + } > + > + val = NETDEV_XDP_ACT_BASIC | NETDEV_XDP_ACT_REDIRECT | > + NETDEV_XDP_ACT_XSK_ZEROCOPY; > + if (ndo_xmit) > + val |= NETDEV_XDP_ACT_NDO_XMIT; > + if (params->rq_wq_type == MLX5_WQ_TYPE_CYCLIC) { > + val |= NETDEV_XDP_ACT_RX_SG; > + if (ndo_xmit) > + val |= NETDEV_XDP_ACT_NDO_XMIT_SG; This NETDEV_XDP_ACT_NDO_XMIT_SG capability is not related to the RQ type. It's still not supported at this point. BTW, I have a series completing all the missing capabilities (multibuf on Striding + multibuf redirect-in), should be submitted in this kernel. > + } > + xdp_set_features_flag(netdev, val); > +} > + > int mlx5e_set_features(struct net_device *netdev, netdev_features_t features) > { > netdev_features_t oper_features = features; > @@ -4030,6 +4054,9 @@ int mlx5e_set_features(struct net_device *netdev, netdev_features_t features) > return -EINVAL; > } > > + /* update XDP supported features */ > + mlx5e_set_xdp_feature(netdev); > + > return 0; > } > > @@ -4762,10 +4789,14 @@ static int mlx5e_xdp_set(struct net_device *netdev, struct bpf_prog *prog) > bpf_prog_put(old_prog); > > if (reset) { > - if (prog) > - xdp_features_set_redirect_target(netdev, true); > - else > + if (prog) { > + bool xmit_sg; > + > + xmit_sg = new_params.rq_wq_type == MLX5_WQ_TYPE_CYCLIC; Same, not related. Still not supported at this point. > + xdp_features_set_redirect_target(netdev, xmit_sg); > + } else { > xdp_features_clear_redirect_target(netdev); > + } > } > > if (!test_bit(MLX5E_STATE_OPENED, &priv->state) || reset) > @@ -5163,13 +5194,10 @@ static void mlx5e_build_nic_netdev(struct net_device *netdev) > netdev->features |= NETIF_F_HIGHDMA; > netdev->features |= NETIF_F_HW_VLAN_STAG_FILTER; > > - netdev->xdp_features = NETDEV_XDP_ACT_BASIC | NETDEV_XDP_ACT_REDIRECT | > - NETDEV_XDP_ACT_XSK_ZEROCOPY | > - NETDEV_XDP_ACT_RX_SG; > - > netdev->priv_flags |= IFF_UNICAST_FLT; > > netif_set_tso_max_size(netdev, GSO_MAX_SIZE); > + mlx5e_set_xdp_feature(netdev); > mlx5e_set_netdev_dev_addr(netdev); > mlx5e_macsec_build_netdev(priv); > mlx5e_ipsec_build_netdev(priv); > @@ -5241,6 +5269,9 @@ static int mlx5e_nic_init(struct mlx5_core_dev *mdev, > mlx5_core_err(mdev, "TLS initialization failed, %d\n", err); > > mlx5e_health_create_reporters(priv); > + /* update XDP supported features */ > + mlx5e_set_xdp_feature(netdev); > + > return 0; > } > > diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c > index 9b9203443085..43fd12fb87b8 100644 > --- a/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c > +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c > @@ -747,6 +747,9 @@ static void mlx5e_build_rep_params(struct net_device *netdev) > /* RQ */ > mlx5e_build_rq_params(mdev, params); > > + /* update XDP supported features */ > + mlx5e_set_xdp_feature(netdev); > + > /* CQ moderation params */ > params->rx_dim_enabled = MLX5_CAP_GEN(mdev, cq_moderation); > mlx5e_set_rx_cq_mode_params(params, cq_period_mode);
> > > On 07/03/2023 16:54, Lorenzo Bianconi wrote: > > Take into account LRO and GRO configuration setting device xdp_features > > flag. Moreover consider channel rq_wq_type enabling rx scatter-gatter > > support in xdp_features flag. > > > > Fixes: 66c0e13ad236 ("drivers: net: turn on XDP features") > > Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org> > > --- > > drivers/net/ethernet/mellanox/mlx5/core/en.h | 1 + > > .../ethernet/mellanox/mlx5/core/en_ethtool.c | 10 ++++- > > .../net/ethernet/mellanox/mlx5/core/en_main.c | 45 ++++++++++++++++--- > > .../net/ethernet/mellanox/mlx5/core/en_rep.c | 3 ++ > > 4 files changed, 51 insertions(+), 8 deletions(-) > > > > diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h > > index 88460b7796e5..4276c6eb6820 100644 > > --- a/drivers/net/ethernet/mellanox/mlx5/core/en.h > > +++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h > > @@ -1243,6 +1243,7 @@ void mlx5e_build_nic_params(struct mlx5e_priv *priv, struct mlx5e_xsk *xsk, u16 > > void mlx5e_rx_dim_work(struct work_struct *work); > > void mlx5e_tx_dim_work(struct work_struct *work); > > +void mlx5e_set_xdp_feature(struct net_device *netdev); > > netdev_features_t mlx5e_features_check(struct sk_buff *skb, > > struct net_device *netdev, > > netdev_features_t features); > > diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c > > index 7708acc9b2ab..79fd21ecb9cb 100644 > > --- a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c > > +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c > > @@ -1985,6 +1985,7 @@ static int set_pflag_rx_striding_rq(struct net_device *netdev, bool enable) > > struct mlx5e_priv *priv = netdev_priv(netdev); > > struct mlx5_core_dev *mdev = priv->mdev; > > struct mlx5e_params new_params; > > + int err; > > if (enable) { > > /* Checking the regular RQ here; mlx5e_validate_xsk_param called > > @@ -2005,7 +2006,14 @@ static int set_pflag_rx_striding_rq(struct net_device *netdev, bool enable) > > MLX5E_SET_PFLAG(&new_params, MLX5E_PFLAG_RX_STRIDING_RQ, enable); > > mlx5e_set_rq_type(mdev, &new_params); > > - return mlx5e_safe_switch_params(priv, &new_params, NULL, NULL, true); > > + err = mlx5e_safe_switch_params(priv, &new_params, NULL, NULL, true); > > + if (err) > > + return err; > > + > > + /* update XDP supported features */ > > + mlx5e_set_xdp_feature(netdev); > > + > > + return 0; > > } > > static int set_pflag_rx_no_csum_complete(struct net_device *netdev, bool enable) > > diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c > > index 76a9c5194a70..1b68dd2be2c5 100644 > > --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c > > +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c > > @@ -4004,6 +4004,30 @@ static int mlx5e_handle_feature(struct net_device *netdev, > > return 0; > > } > > +void mlx5e_set_xdp_feature(struct net_device *netdev) > > +{ > > + struct mlx5e_priv *priv = netdev_priv(netdev); > > + bool ndo_xmit = test_bit(MLX5E_STATE_XDP_ACTIVE, &priv->state); > > Our driver doesn't require loading a dummy XDP program to have the > redirect-in ability. It's always there. > > I actually have a bug fix under internal review with Saeed that addresses > this. > > In addition, it cleans up the NETDEV_XDP_ACT_NDO_XMIT_SG as we do not > support it yet. I have a series that's adding support and will submit it > soon. > > Any reason you're submitting these fixes to net-next rather than net? Hi Tariq, I am fine to repost this series for net instead of net-next. Any downsides about it? > Maybe it'd be better if we integrate the patches, here's my fix (still under > review...): > > Author: Tariq Toukan <tariqt@nvidia.com> > Date: Thu Feb 23 08:58:04 2023 +0200 > > net/mlx5e: Fix exposed xdp_features > > Always declare NETDEV_XDP_ACT_NDO_XMIT as the ndo_xdp_xmit callback > is always functional per our design, and does not require loading > a dummy xdp program. > > Although non-linear XDP buffer is supported for XDP_TX flow, do not > declare NETDEV_XDP_ACT_NDO_XMIT_SG as it is yet supported for > redirected-in frames. > > Fixes: 66c0e13ad236 ("drivers: net: turn on XDP features") > Signed-off-by: Tariq Toukan <tariqt@nvidia.com> > > diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c > b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c > index 53feb0529943..9a5d3ce1fbcd 100644 > --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c > +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c > @@ -4741,13 +4741,6 @@ static int mlx5e_xdp_set(struct net_device *netdev, > struct bpf_prog *prog) > if (old_prog) > bpf_prog_put(old_prog); > > - if (reset) { > - if (prog) > - xdp_features_set_redirect_target(netdev, true); > - else > - xdp_features_clear_redirect_target(netdev); > - } > - > if (!test_bit(MLX5E_STATE_OPENED, &priv->state) || reset) > goto unlock; > > @@ -5144,6 +5137,7 @@ static void mlx5e_build_nic_netdev(struct net_device > *netdev) > netdev->features |= NETIF_F_HW_VLAN_STAG_FILTER; > > netdev->xdp_features = NETDEV_XDP_ACT_BASIC | > NETDEV_XDP_ACT_REDIRECT | > + NETDEV_XDP_ACT_NDO_XMIT | > NETDEV_XDP_ACT_XSK_ZEROCOPY | > NETDEV_XDP_ACT_RX_SG; I am fine to drop this my patch and rely on the one you provided but it depends on the eta about the described patches because otherwise real capabilities and xdp-features will not be aligned. Any inputs on it? > > > > + struct mlx5e_params *params = &priv->channels.params; > > + xdp_features_t val; > > + > > + if (params->packet_merge.type != MLX5E_PACKET_MERGE_NONE) { > > + xdp_clear_features_flag(netdev); > > + return; > > + } > > + > > + val = NETDEV_XDP_ACT_BASIC | NETDEV_XDP_ACT_REDIRECT | > > + NETDEV_XDP_ACT_XSK_ZEROCOPY; > > + if (ndo_xmit) > > + val |= NETDEV_XDP_ACT_NDO_XMIT; > > + if (params->rq_wq_type == MLX5_WQ_TYPE_CYCLIC) { > > + val |= NETDEV_XDP_ACT_RX_SG; > > + if (ndo_xmit) > > + val |= NETDEV_XDP_ACT_NDO_XMIT_SG; > > This NETDEV_XDP_ACT_NDO_XMIT_SG capability is not related to the RQ type. > It's still not supported at this point. ack, I will fix it. > > BTW, I have a series completing all the missing capabilities (multibuf on > Striding + multibuf redirect-in), should be submitted in this kernel. cool :) Regards, Lorenzo > > > + } > > + xdp_set_features_flag(netdev, val); > > +} > > + > > int mlx5e_set_features(struct net_device *netdev, netdev_features_t features) > > { > > netdev_features_t oper_features = features; > > @@ -4030,6 +4054,9 @@ int mlx5e_set_features(struct net_device *netdev, netdev_features_t features) > > return -EINVAL; > > } > > + /* update XDP supported features */ > > + mlx5e_set_xdp_feature(netdev); > > + > > return 0; > > } > > @@ -4762,10 +4789,14 @@ static int mlx5e_xdp_set(struct net_device *netdev, struct bpf_prog *prog) > > bpf_prog_put(old_prog); > > if (reset) { > > - if (prog) > > - xdp_features_set_redirect_target(netdev, true); > > - else > > + if (prog) { > > + bool xmit_sg; > > + > > + xmit_sg = new_params.rq_wq_type == MLX5_WQ_TYPE_CYCLIC; > > Same, not related. Still not supported at this point. > > > + xdp_features_set_redirect_target(netdev, xmit_sg); > > + } else { > > xdp_features_clear_redirect_target(netdev); > > + } > > } > > if (!test_bit(MLX5E_STATE_OPENED, &priv->state) || reset) > > @@ -5163,13 +5194,10 @@ static void mlx5e_build_nic_netdev(struct net_device *netdev) > > netdev->features |= NETIF_F_HIGHDMA; > > netdev->features |= NETIF_F_HW_VLAN_STAG_FILTER; > > - netdev->xdp_features = NETDEV_XDP_ACT_BASIC | NETDEV_XDP_ACT_REDIRECT | > > - NETDEV_XDP_ACT_XSK_ZEROCOPY | > > - NETDEV_XDP_ACT_RX_SG; > > - > > netdev->priv_flags |= IFF_UNICAST_FLT; > > netif_set_tso_max_size(netdev, GSO_MAX_SIZE); > > + mlx5e_set_xdp_feature(netdev); > > mlx5e_set_netdev_dev_addr(netdev); > > mlx5e_macsec_build_netdev(priv); > > mlx5e_ipsec_build_netdev(priv); > > @@ -5241,6 +5269,9 @@ static int mlx5e_nic_init(struct mlx5_core_dev *mdev, > > mlx5_core_err(mdev, "TLS initialization failed, %d\n", err); > > mlx5e_health_create_reporters(priv); > > + /* update XDP supported features */ > > + mlx5e_set_xdp_feature(netdev); > > + > > return 0; > > } > > diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c > > index 9b9203443085..43fd12fb87b8 100644 > > --- a/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c > > +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c > > @@ -747,6 +747,9 @@ static void mlx5e_build_rep_params(struct net_device *netdev) > > /* RQ */ > > mlx5e_build_rq_params(mdev, params); > > + /* update XDP supported features */ > > + mlx5e_set_xdp_feature(netdev); > > + > > /* CQ moderation params */ > > params->rx_dim_enabled = MLX5_CAP_GEN(mdev, cq_moderation); > > mlx5e_set_rx_cq_mode_params(params, cq_period_mode); >
On 08/03/2023 17:47, Lorenzo Bianconi wrote: >> >> >> On 07/03/2023 16:54, Lorenzo Bianconi wrote: >>> Take into account LRO and GRO configuration setting device xdp_features >>> flag. Moreover consider channel rq_wq_type enabling rx scatter-gatter >>> support in xdp_features flag. >>> >>> Fixes: 66c0e13ad236 ("drivers: net: turn on XDP features") >>> Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org> >>> --- >>> drivers/net/ethernet/mellanox/mlx5/core/en.h | 1 + >>> .../ethernet/mellanox/mlx5/core/en_ethtool.c | 10 ++++- >>> .../net/ethernet/mellanox/mlx5/core/en_main.c | 45 ++++++++++++++++--- >>> .../net/ethernet/mellanox/mlx5/core/en_rep.c | 3 ++ >>> 4 files changed, 51 insertions(+), 8 deletions(-) >>> >>> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h >>> index 88460b7796e5..4276c6eb6820 100644 >>> --- a/drivers/net/ethernet/mellanox/mlx5/core/en.h >>> +++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h >>> @@ -1243,6 +1243,7 @@ void mlx5e_build_nic_params(struct mlx5e_priv *priv, struct mlx5e_xsk *xsk, u16 >>> void mlx5e_rx_dim_work(struct work_struct *work); >>> void mlx5e_tx_dim_work(struct work_struct *work); >>> +void mlx5e_set_xdp_feature(struct net_device *netdev); >>> netdev_features_t mlx5e_features_check(struct sk_buff *skb, >>> struct net_device *netdev, >>> netdev_features_t features); >>> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c >>> index 7708acc9b2ab..79fd21ecb9cb 100644 >>> --- a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c >>> +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c >>> @@ -1985,6 +1985,7 @@ static int set_pflag_rx_striding_rq(struct net_device *netdev, bool enable) >>> struct mlx5e_priv *priv = netdev_priv(netdev); >>> struct mlx5_core_dev *mdev = priv->mdev; >>> struct mlx5e_params new_params; >>> + int err; >>> if (enable) { >>> /* Checking the regular RQ here; mlx5e_validate_xsk_param called >>> @@ -2005,7 +2006,14 @@ static int set_pflag_rx_striding_rq(struct net_device *netdev, bool enable) >>> MLX5E_SET_PFLAG(&new_params, MLX5E_PFLAG_RX_STRIDING_RQ, enable); >>> mlx5e_set_rq_type(mdev, &new_params); >>> - return mlx5e_safe_switch_params(priv, &new_params, NULL, NULL, true); >>> + err = mlx5e_safe_switch_params(priv, &new_params, NULL, NULL, true); >>> + if (err) >>> + return err; >>> + >>> + /* update XDP supported features */ >>> + mlx5e_set_xdp_feature(netdev); >>> + >>> + return 0; >>> } >>> static int set_pflag_rx_no_csum_complete(struct net_device *netdev, bool enable) >>> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c >>> index 76a9c5194a70..1b68dd2be2c5 100644 >>> --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c >>> +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c >>> @@ -4004,6 +4004,30 @@ static int mlx5e_handle_feature(struct net_device *netdev, >>> return 0; >>> } >>> +void mlx5e_set_xdp_feature(struct net_device *netdev) >>> +{ >>> + struct mlx5e_priv *priv = netdev_priv(netdev); >>> + bool ndo_xmit = test_bit(MLX5E_STATE_XDP_ACTIVE, &priv->state); >> >> Our driver doesn't require loading a dummy XDP program to have the >> redirect-in ability. It's always there. >> >> I actually have a bug fix under internal review with Saeed that addresses >> this. >> >> In addition, it cleans up the NETDEV_XDP_ACT_NDO_XMIT_SG as we do not >> support it yet. I have a series that's adding support and will submit it >> soon. >> >> Any reason you're submitting these fixes to net-next rather than net? > > Hi Tariq, > > I am fine to repost this series for net instead of net-next. Any downsides about > it? Let's repost to net. It's a fixes series, and 6.3 is still in its RCs. If you don't post it to net then the xdp-features in 6.3 will be broken. > >> Maybe it'd be better if we integrate the patches, here's my fix (still under >> review...): >> >> Author: Tariq Toukan <tariqt@nvidia.com> >> Date: Thu Feb 23 08:58:04 2023 +0200 >> >> net/mlx5e: Fix exposed xdp_features >> >> Always declare NETDEV_XDP_ACT_NDO_XMIT as the ndo_xdp_xmit callback >> is always functional per our design, and does not require loading >> a dummy xdp program. >> >> Although non-linear XDP buffer is supported for XDP_TX flow, do not >> declare NETDEV_XDP_ACT_NDO_XMIT_SG as it is yet supported for >> redirected-in frames. >> >> Fixes: 66c0e13ad236 ("drivers: net: turn on XDP features") >> Signed-off-by: Tariq Toukan <tariqt@nvidia.com> >> >> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c >> b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c >> index 53feb0529943..9a5d3ce1fbcd 100644 >> --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c >> +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c >> @@ -4741,13 +4741,6 @@ static int mlx5e_xdp_set(struct net_device *netdev, >> struct bpf_prog *prog) >> if (old_prog) >> bpf_prog_put(old_prog); >> >> - if (reset) { >> - if (prog) >> - xdp_features_set_redirect_target(netdev, true); >> - else >> - xdp_features_clear_redirect_target(netdev); >> - } >> - >> if (!test_bit(MLX5E_STATE_OPENED, &priv->state) || reset) >> goto unlock; >> >> @@ -5144,6 +5137,7 @@ static void mlx5e_build_nic_netdev(struct net_device >> *netdev) >> netdev->features |= NETIF_F_HW_VLAN_STAG_FILTER; >> >> netdev->xdp_features = NETDEV_XDP_ACT_BASIC | >> NETDEV_XDP_ACT_REDIRECT | >> + NETDEV_XDP_ACT_NDO_XMIT | >> NETDEV_XDP_ACT_XSK_ZEROCOPY | >> NETDEV_XDP_ACT_RX_SG; > > I am fine to drop this my patch and rely on the one you provided but it depends > on the eta about the described patches because otherwise real capabilities and > xdp-features will not be aligned. Any inputs on it? > My patch doesn't replace yours, as it doesn't fix the missing features_update according to striding RQ and HW LRO/GRO. I think we should combine them, either take mine as-is into your series, or squash it into this patch. I'm fine with both. >> >> >>> + struct mlx5e_params *params = &priv->channels.params; >>> + xdp_features_t val; >>> + >>> + if (params->packet_merge.type != MLX5E_PACKET_MERGE_NONE) { >>> + xdp_clear_features_flag(netdev); >>> + return; >>> + } >>> + >>> + val = NETDEV_XDP_ACT_BASIC | NETDEV_XDP_ACT_REDIRECT | >>> + NETDEV_XDP_ACT_XSK_ZEROCOPY; >>> + if (ndo_xmit) >>> + val |= NETDEV_XDP_ACT_NDO_XMIT; >>> + if (params->rq_wq_type == MLX5_WQ_TYPE_CYCLIC) { >>> + val |= NETDEV_XDP_ACT_RX_SG; >>> + if (ndo_xmit) >>> + val |= NETDEV_XDP_ACT_NDO_XMIT_SG; >> >> This NETDEV_XDP_ACT_NDO_XMIT_SG capability is not related to the RQ type. >> It's still not supported at this point. > > ack, I will fix it. > >> >> BTW, I have a series completing all the missing capabilities (multibuf on >> Striding + multibuf redirect-in), should be submitted in this kernel. > > cool :) > > Regards, > Lorenzo >
On Thu, 9 Mar 2023 09:23:10 +0200 Tariq Toukan wrote: > > Hi Tariq, > > > > I am fine to repost this series for net instead of net-next. Any downsides about > > it? > > Let's repost to net. > It's a fixes series, and 6.3 is still in its RCs. > If you don't post it to net then the xdp-features in 6.3 will be broken. minor heads up - patch 2 will now apply to lib/nlspec.py I just moved the enum classes there as part of another fix but the code and the changes should be identical
> On Thu, 9 Mar 2023 09:23:10 +0200 Tariq Toukan wrote: > > > Hi Tariq, > > > > > > I am fine to repost this series for net instead of net-next. Any downsides about > > > it? > > > > Let's repost to net. > > It's a fixes series, and 6.3 is still in its RCs. > > If you don't post it to net then the xdp-features in 6.3 will be broken. > > minor heads up - patch 2 will now apply to lib/nlspec.py > I just moved the enum classes there as part of another fix > but the code and the changes should be identical > ack, fine. I will fix the conflicts rebasing on net tree. Regards, Lorenzo
> > > On 08/03/2023 17:47, Lorenzo Bianconi wrote: > > > > > > > > > On 07/03/2023 16:54, Lorenzo Bianconi wrote: > > > > Take into account LRO and GRO configuration setting device xdp_features > > > > flag. Moreover consider channel rq_wq_type enabling rx scatter-gatter > > > > support in xdp_features flag. > > > > > > > > Fixes: 66c0e13ad236 ("drivers: net: turn on XDP features") > > > > Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org> > > > > --- > > > > drivers/net/ethernet/mellanox/mlx5/core/en.h | 1 + > > > > .../ethernet/mellanox/mlx5/core/en_ethtool.c | 10 ++++- > > > > .../net/ethernet/mellanox/mlx5/core/en_main.c | 45 ++++++++++++++++--- > > > > .../net/ethernet/mellanox/mlx5/core/en_rep.c | 3 ++ > > > > 4 files changed, 51 insertions(+), 8 deletions(-) > > > > > > > > diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h > > > > index 88460b7796e5..4276c6eb6820 100644 > > > > --- a/drivers/net/ethernet/mellanox/mlx5/core/en.h > > > > +++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h > > > > @@ -1243,6 +1243,7 @@ void mlx5e_build_nic_params(struct mlx5e_priv *priv, struct mlx5e_xsk *xsk, u16 > > > > void mlx5e_rx_dim_work(struct work_struct *work); > > > > void mlx5e_tx_dim_work(struct work_struct *work); > > > > +void mlx5e_set_xdp_feature(struct net_device *netdev); > > > > netdev_features_t mlx5e_features_check(struct sk_buff *skb, > > > > struct net_device *netdev, > > > > netdev_features_t features); > > > > diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c > > > > index 7708acc9b2ab..79fd21ecb9cb 100644 > > > > --- a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c > > > > +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c > > > > @@ -1985,6 +1985,7 @@ static int set_pflag_rx_striding_rq(struct net_device *netdev, bool enable) > > > > struct mlx5e_priv *priv = netdev_priv(netdev); > > > > struct mlx5_core_dev *mdev = priv->mdev; > > > > struct mlx5e_params new_params; > > > > + int err; > > > > if (enable) { > > > > /* Checking the regular RQ here; mlx5e_validate_xsk_param called > > > > @@ -2005,7 +2006,14 @@ static int set_pflag_rx_striding_rq(struct net_device *netdev, bool enable) > > > > MLX5E_SET_PFLAG(&new_params, MLX5E_PFLAG_RX_STRIDING_RQ, enable); > > > > mlx5e_set_rq_type(mdev, &new_params); > > > > - return mlx5e_safe_switch_params(priv, &new_params, NULL, NULL, true); > > > > + err = mlx5e_safe_switch_params(priv, &new_params, NULL, NULL, true); > > > > + if (err) > > > > + return err; > > > > + > > > > + /* update XDP supported features */ > > > > + mlx5e_set_xdp_feature(netdev); > > > > + > > > > + return 0; > > > > } > > > > static int set_pflag_rx_no_csum_complete(struct net_device *netdev, bool enable) > > > > diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c > > > > index 76a9c5194a70..1b68dd2be2c5 100644 > > > > --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c > > > > +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c > > > > @@ -4004,6 +4004,30 @@ static int mlx5e_handle_feature(struct net_device *netdev, > > > > return 0; > > > > } > > > > +void mlx5e_set_xdp_feature(struct net_device *netdev) > > > > +{ > > > > + struct mlx5e_priv *priv = netdev_priv(netdev); > > > > + bool ndo_xmit = test_bit(MLX5E_STATE_XDP_ACTIVE, &priv->state); > > > > > > Our driver doesn't require loading a dummy XDP program to have the > > > redirect-in ability. It's always there. > > > > > > I actually have a bug fix under internal review with Saeed that addresses > > > this. > > > > > > In addition, it cleans up the NETDEV_XDP_ACT_NDO_XMIT_SG as we do not > > > support it yet. I have a series that's adding support and will submit it > > > soon. > > > > > > Any reason you're submitting these fixes to net-next rather than net? > > > > Hi Tariq, > > > > I am fine to repost this series for net instead of net-next. Any downsides about > > it? > > Let's repost to net. > It's a fixes series, and 6.3 is still in its RCs. > If you don't post it to net then the xdp-features in 6.3 will be broken. ack, fine. > > > > > > Maybe it'd be better if we integrate the patches, here's my fix (still under > > > review...): > > > > > > Author: Tariq Toukan <tariqt@nvidia.com> > > > Date: Thu Feb 23 08:58:04 2023 +0200 > > > > > > net/mlx5e: Fix exposed xdp_features > > > > > > Always declare NETDEV_XDP_ACT_NDO_XMIT as the ndo_xdp_xmit callback > > > is always functional per our design, and does not require loading > > > a dummy xdp program. > > > > > > Although non-linear XDP buffer is supported for XDP_TX flow, do not > > > declare NETDEV_XDP_ACT_NDO_XMIT_SG as it is yet supported for > > > redirected-in frames. > > > > > > Fixes: 66c0e13ad236 ("drivers: net: turn on XDP features") > > > Signed-off-by: Tariq Toukan <tariqt@nvidia.com> > > > > > > diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c > > > b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c > > > index 53feb0529943..9a5d3ce1fbcd 100644 > > > --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c > > > +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c > > > @@ -4741,13 +4741,6 @@ static int mlx5e_xdp_set(struct net_device *netdev, > > > struct bpf_prog *prog) > > > if (old_prog) > > > bpf_prog_put(old_prog); > > > > > > - if (reset) { > > > - if (prog) > > > - xdp_features_set_redirect_target(netdev, true); > > > - else > > > - xdp_features_clear_redirect_target(netdev); > > > - } > > > - > > > if (!test_bit(MLX5E_STATE_OPENED, &priv->state) || reset) > > > goto unlock; > > > > > > @@ -5144,6 +5137,7 @@ static void mlx5e_build_nic_netdev(struct net_device > > > *netdev) > > > netdev->features |= NETIF_F_HW_VLAN_STAG_FILTER; > > > > > > netdev->xdp_features = NETDEV_XDP_ACT_BASIC | > > > NETDEV_XDP_ACT_REDIRECT | > > > + NETDEV_XDP_ACT_NDO_XMIT | > > > NETDEV_XDP_ACT_XSK_ZEROCOPY | > > > NETDEV_XDP_ACT_RX_SG; > > > > I am fine to drop this my patch and rely on the one you provided but it depends > > on the eta about the described patches because otherwise real capabilities and > > xdp-features will not be aligned. Any inputs on it? > > > > My patch doesn't replace yours, as it doesn't fix the missing > features_update according to striding RQ and HW LRO/GRO. > > I think we should combine them, either take mine as-is into your series, or > squash it into this patch. I'm fine with both. ack fine, I will squash your changes into the patch posting a new version, thx. Regards, Lorenzo > > > > > > > > > > > + struct mlx5e_params *params = &priv->channels.params; > > > > + xdp_features_t val; > > > > + > > > > + if (params->packet_merge.type != MLX5E_PACKET_MERGE_NONE) { > > > > + xdp_clear_features_flag(netdev); > > > > + return; > > > > + } > > > > + > > > > + val = NETDEV_XDP_ACT_BASIC | NETDEV_XDP_ACT_REDIRECT | > > > > + NETDEV_XDP_ACT_XSK_ZEROCOPY; > > > > + if (ndo_xmit) > > > > + val |= NETDEV_XDP_ACT_NDO_XMIT; > > > > + if (params->rq_wq_type == MLX5_WQ_TYPE_CYCLIC) { > > > > + val |= NETDEV_XDP_ACT_RX_SG; > > > > + if (ndo_xmit) > > > > + val |= NETDEV_XDP_ACT_NDO_XMIT_SG; > > > > > > This NETDEV_XDP_ACT_NDO_XMIT_SG capability is not related to the RQ type. > > > It's still not supported at this point. > > > > ack, I will fix it. > > > > > > > > BTW, I have a series completing all the missing capabilities (multibuf on > > > Striding + multibuf redirect-in), should be submitted in this kernel. > > > > cool :) > > > > Regards, > > Lorenzo > > >
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h index 88460b7796e5..4276c6eb6820 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h @@ -1243,6 +1243,7 @@ void mlx5e_build_nic_params(struct mlx5e_priv *priv, struct mlx5e_xsk *xsk, u16 void mlx5e_rx_dim_work(struct work_struct *work); void mlx5e_tx_dim_work(struct work_struct *work); +void mlx5e_set_xdp_feature(struct net_device *netdev); netdev_features_t mlx5e_features_check(struct sk_buff *skb, struct net_device *netdev, netdev_features_t features); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c index 7708acc9b2ab..79fd21ecb9cb 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c @@ -1985,6 +1985,7 @@ static int set_pflag_rx_striding_rq(struct net_device *netdev, bool enable) struct mlx5e_priv *priv = netdev_priv(netdev); struct mlx5_core_dev *mdev = priv->mdev; struct mlx5e_params new_params; + int err; if (enable) { /* Checking the regular RQ here; mlx5e_validate_xsk_param called @@ -2005,7 +2006,14 @@ static int set_pflag_rx_striding_rq(struct net_device *netdev, bool enable) MLX5E_SET_PFLAG(&new_params, MLX5E_PFLAG_RX_STRIDING_RQ, enable); mlx5e_set_rq_type(mdev, &new_params); - return mlx5e_safe_switch_params(priv, &new_params, NULL, NULL, true); + err = mlx5e_safe_switch_params(priv, &new_params, NULL, NULL, true); + if (err) + return err; + + /* update XDP supported features */ + mlx5e_set_xdp_feature(netdev); + + return 0; } static int set_pflag_rx_no_csum_complete(struct net_device *netdev, bool enable) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c index 76a9c5194a70..1b68dd2be2c5 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c @@ -4004,6 +4004,30 @@ static int mlx5e_handle_feature(struct net_device *netdev, return 0; } +void mlx5e_set_xdp_feature(struct net_device *netdev) +{ + struct mlx5e_priv *priv = netdev_priv(netdev); + bool ndo_xmit = test_bit(MLX5E_STATE_XDP_ACTIVE, &priv->state); + struct mlx5e_params *params = &priv->channels.params; + xdp_features_t val; + + if (params->packet_merge.type != MLX5E_PACKET_MERGE_NONE) { + xdp_clear_features_flag(netdev); + return; + } + + val = NETDEV_XDP_ACT_BASIC | NETDEV_XDP_ACT_REDIRECT | + NETDEV_XDP_ACT_XSK_ZEROCOPY; + if (ndo_xmit) + val |= NETDEV_XDP_ACT_NDO_XMIT; + if (params->rq_wq_type == MLX5_WQ_TYPE_CYCLIC) { + val |= NETDEV_XDP_ACT_RX_SG; + if (ndo_xmit) + val |= NETDEV_XDP_ACT_NDO_XMIT_SG; + } + xdp_set_features_flag(netdev, val); +} + int mlx5e_set_features(struct net_device *netdev, netdev_features_t features) { netdev_features_t oper_features = features; @@ -4030,6 +4054,9 @@ int mlx5e_set_features(struct net_device *netdev, netdev_features_t features) return -EINVAL; } + /* update XDP supported features */ + mlx5e_set_xdp_feature(netdev); + return 0; } @@ -4762,10 +4789,14 @@ static int mlx5e_xdp_set(struct net_device *netdev, struct bpf_prog *prog) bpf_prog_put(old_prog); if (reset) { - if (prog) - xdp_features_set_redirect_target(netdev, true); - else + if (prog) { + bool xmit_sg; + + xmit_sg = new_params.rq_wq_type == MLX5_WQ_TYPE_CYCLIC; + xdp_features_set_redirect_target(netdev, xmit_sg); + } else { xdp_features_clear_redirect_target(netdev); + } } if (!test_bit(MLX5E_STATE_OPENED, &priv->state) || reset) @@ -5163,13 +5194,10 @@ static void mlx5e_build_nic_netdev(struct net_device *netdev) netdev->features |= NETIF_F_HIGHDMA; netdev->features |= NETIF_F_HW_VLAN_STAG_FILTER; - netdev->xdp_features = NETDEV_XDP_ACT_BASIC | NETDEV_XDP_ACT_REDIRECT | - NETDEV_XDP_ACT_XSK_ZEROCOPY | - NETDEV_XDP_ACT_RX_SG; - netdev->priv_flags |= IFF_UNICAST_FLT; netif_set_tso_max_size(netdev, GSO_MAX_SIZE); + mlx5e_set_xdp_feature(netdev); mlx5e_set_netdev_dev_addr(netdev); mlx5e_macsec_build_netdev(priv); mlx5e_ipsec_build_netdev(priv); @@ -5241,6 +5269,9 @@ static int mlx5e_nic_init(struct mlx5_core_dev *mdev, mlx5_core_err(mdev, "TLS initialization failed, %d\n", err); mlx5e_health_create_reporters(priv); + /* update XDP supported features */ + mlx5e_set_xdp_feature(netdev); + return 0; } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c index 9b9203443085..43fd12fb87b8 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c @@ -747,6 +747,9 @@ static void mlx5e_build_rep_params(struct net_device *netdev) /* RQ */ mlx5e_build_rq_params(mdev, params); + /* update XDP supported features */ + mlx5e_set_xdp_feature(netdev); + /* CQ moderation params */ params->rx_dim_enabled = MLX5_CAP_GEN(mdev, cq_moderation); mlx5e_set_rx_cq_mode_params(params, cq_period_mode);
Take into account LRO and GRO configuration setting device xdp_features flag. Moreover consider channel rq_wq_type enabling rx scatter-gatter support in xdp_features flag. Fixes: 66c0e13ad236 ("drivers: net: turn on XDP features") Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org> --- drivers/net/ethernet/mellanox/mlx5/core/en.h | 1 + .../ethernet/mellanox/mlx5/core/en_ethtool.c | 10 ++++- .../net/ethernet/mellanox/mlx5/core/en_main.c | 45 ++++++++++++++++--- .../net/ethernet/mellanox/mlx5/core/en_rep.c | 3 ++ 4 files changed, 51 insertions(+), 8 deletions(-)