mbox series

[rdma-next,v4,00/13] Optional counter statistics support

Message ID 20211008122439.166063-1-markzhang@nvidia.com (mailing list archive)
Headers show
Series Optional counter statistics support | expand

Message

Mark Zhang Oct. 8, 2021, 12:24 p.m. UTC
Change Log:
v4:
 * Add stats NULL check in rdma_free_hw_stats_struct();
 * Improve nldev_stat_set_doit() and nldev_stat_set_mode_doit();
 * Remove RDMA_NLDEV_ATTR_STAT_HWCOUNTER_DYNAMIC attribute dependence when
   set dynamic counters;
v3: https://lore.kernel.org/all/cover.1633513239.git.leonro@nvidia.com/
 * Many cosmetic changes;
 * Fixed rebase error which caused to have two same patches with
   different implementation;
 * New patch that split nldev_stat_set_doit;
v2: https://lore.kernel.org/all/cover.1632988543.git.leonro@nvidia.com
 * Add rdma_free_hw_stats_struct() helper API (with a new patch);
 * In sysfs add a WARN_ON to check if optional stats are always at the end;
 * Add a new nldev command to get the counter status;
 * Improve nldev_stat_set_counter_dynamic_doit() by creating a target state
   bitmap;
v1: https://lore.kernel.org/all/cover.1631660727.git.leonro@nvidia.com
 * Add a descriptor structure to replace name in struct rdma_hw_stats;
 * Add a bitmap in struct rdma_hw_stats to indicate the enable/disable
   status of all counters;
 * Add a "flag" field in counter descriptor and define
   IB_STAT_FLAG_OPTIONAL flag;
 * add/remove_op_stat() are replaced by modify_op_stat();
 * Use "set/unset" in command line and send full opcounters list through
   netlink, and send opcounter indexes instead of names;
 * Patches are re-ordered.
v0: https://lore.kernel.org/all/20210818112428.209111-1-markzhang@nvidia.com

----------------------------------------------------------------------
Hi,

This series from Neta and Aharon provides an extension to the rdma
statistics tool that allows to set optional counters dynamically, using
netlink.

The idea of having optional counters is to provide to the users the
ability to get statistics of counters that hurts performance.

Once an optional counter was added, its statistics will be presented
along with all the counters, using the show command.

Binding objects to the optional counters is currently not supported,
neither in auto mode nor in manual mode.

To get the list of optional counters that are supported on this device,
use "rdma statistic mode supported". To see which counters are currently
enabled, use "rdma statistic mode".

Examples:

$ rdma statistic mode supported
link rocep8s0f0/1 supported optional-counters cc_rx_ce_pkts,cc_rx_cnp_pkts,cc_tx_cnp_pkts
link rocep8s0f1/1 supported optional-counters cc_rx_ce_pkts,cc_rx_cnp_pkts,cc_tx_cnp_pkts

$ sudo rdma statistic set link rocep8s0f0/1 optional-counters cc_rx_ce_pkts,cc_rx_cnp_pkts
$ rdma statistic mode link rocep8s0f0/1
link rocep8s0f0/1 optional-counters cc_rx_ce_pkts,cc_rx_cnp_pkts

$ rdma statistic show link rocep8s0f0/1
link rocep8s0f0/1 rx_write_requests 0 rx_read_requests 0 rx_atomic_requests 0 out_of_buffer 0
out_of_sequence 0 duplicate_request 0 rnr_nak_retry_err 0 packet_seq_err 0 implied_nak_seq_err 0
local_ack_timeout_err 0 resp_local_length_error 0 resp_cqe_error 0 req_cqe_error 0
req_remote_invalid_request 0 req_remote_access_errors 0 resp_remote_access_errors 0
resp_cqe_flush_error 0 req_cqe_flush_error 0 roce_adp_retrans 0 roce_adp_retrans_to 0
roce_slow_restart 0 roce_slow_restart_cnps 0 roce_slow_restart_trans 0 rp_cnp_ignored 0
rp_cnp_handled 0 np_ecn_marked_roce_packets 0 np_cnp_sent 0 rx_icrc_encapsulated 0 cc_rx_ce_pkts 0
cc_rx_cnp_pkts 0

$ sudo rdma statistic set link rocep8s0f0/1 optional-counters cc_rx_ce_pkts
$ rdma statistic mode link rocep8s0f0/1
link rocep8s0f0/1 optional-counters cc_rx_ce_pkts

Thanks

Aharon Landau (12):
  net/mlx5: Add ifc bits to support optional counters
  net/mlx5: Add priorities for counters in RDMA namespaces
  RDMA/counter: Add a descriptor in struct rdma_hw_stats
  RDMA/counter: Add an is_disabled field in struct rdma_hw_stats
  RDMA/counter: Add optional counter support
  RDMA/nldev: Add support to get status of all counters
  RDMA/nldev: Split nldev_stat_set_mode_doit out of nldev_stat_set_doit
  RDMA/nldev: Allow optional-counter status configuration through RDMA
    netlink
  RDMA/mlx5: Support optional counters in hw_stats initialization
  RDMA/mlx5: Add steering support in optional flow counters
  RDMA/mlx5: Add modify_op_stat() support
  RDMA/mlx5: Add optional counter support in get_hw_stats callback

Mark Zhang (1):
  RDMA/core: Add a helper API rdma_free_hw_stats_struct

 drivers/infiniband/core/counters.c            |  40 ++-
 drivers/infiniband/core/device.c              |   1 +
 drivers/infiniband/core/nldev.c               | 278 ++++++++++++++---
 drivers/infiniband/core/sysfs.c               |  52 ++--
 drivers/infiniband/core/verbs.c               |  48 +++
 drivers/infiniband/hw/bnxt_re/hw_counters.c   | 137 +++++----
 drivers/infiniband/hw/cxgb4/provider.c        |  22 +-
 drivers/infiniband/hw/efa/efa_verbs.c         |  19 +-
 drivers/infiniband/hw/hfi1/verbs.c            |  53 ++--
 drivers/infiniband/hw/irdma/verbs.c           | 100 +++----
 drivers/infiniband/hw/mlx4/main.c             |  44 ++-
 drivers/infiniband/hw/mlx4/mlx4_ib.h          |   2 +-
 drivers/infiniband/hw/mlx5/counters.c         | 282 +++++++++++++++---
 drivers/infiniband/hw/mlx5/fs.c               | 187 ++++++++++++
 drivers/infiniband/hw/mlx5/mlx5_ib.h          |  28 +-
 drivers/infiniband/sw/rxe/rxe_hw_counters.c   |  42 +--
 .../net/ethernet/mellanox/mlx5/core/fs_core.c |  54 +++-
 include/linux/mlx5/device.h                   |   2 +
 include/linux/mlx5/fs.h                       |   2 +
 include/linux/mlx5/mlx5_ifc.h                 |  22 +-
 include/rdma/ib_hdrs.h                        |   1 +
 include/rdma/ib_verbs.h                       |  58 ++--
 include/rdma/rdma_counter.h                   |   2 +
 include/uapi/rdma/rdma_netlink.h              |   5 +
 24 files changed, 1127 insertions(+), 354 deletions(-)

Comments

Jason Gunthorpe Oct. 8, 2021, 6:57 p.m. UTC | #1
On Fri, Oct 08, 2021 at 03:24:26PM +0300, Mark Zhang wrote:
> 
> Aharon Landau (12):
>   net/mlx5: Add ifc bits to support optional counters
>   net/mlx5: Add priorities for counters in RDMA namespaces
>   RDMA/counter: Add a descriptor in struct rdma_hw_stats
>   RDMA/counter: Add an is_disabled field in struct rdma_hw_stats
>   RDMA/counter: Add optional counter support
>   RDMA/nldev: Add support to get status of all counters
>   RDMA/nldev: Split nldev_stat_set_mode_doit out of nldev_stat_set_doit
>   RDMA/nldev: Allow optional-counter status configuration through RDMA
>     netlink
>   RDMA/mlx5: Support optional counters in hw_stats initialization
>   RDMA/mlx5: Add steering support in optional flow counters
>   RDMA/mlx5: Add modify_op_stat() support
>   RDMA/mlx5: Add optional counter support in get_hw_stats callback
> 
> Mark Zhang (1):
>   RDMA/core: Add a helper API rdma_free_hw_stats_struct

This seems fine now, please update the shared branch

Jason
Leon Romanovsky Oct. 9, 2021, 9:04 a.m. UTC | #2
On Fri, Oct 08, 2021 at 03:57:36PM -0300, Jason Gunthorpe wrote:
> On Fri, Oct 08, 2021 at 03:24:26PM +0300, Mark Zhang wrote:
> > 
> > Aharon Landau (12):
> >   net/mlx5: Add ifc bits to support optional counters
> >   net/mlx5: Add priorities for counters in RDMA namespaces
> >   RDMA/counter: Add a descriptor in struct rdma_hw_stats
> >   RDMA/counter: Add an is_disabled field in struct rdma_hw_stats
> >   RDMA/counter: Add optional counter support
> >   RDMA/nldev: Add support to get status of all counters
> >   RDMA/nldev: Split nldev_stat_set_mode_doit out of nldev_stat_set_doit
> >   RDMA/nldev: Allow optional-counter status configuration through RDMA
> >     netlink
> >   RDMA/mlx5: Support optional counters in hw_stats initialization
> >   RDMA/mlx5: Add steering support in optional flow counters
> >   RDMA/mlx5: Add modify_op_stat() support
> >   RDMA/mlx5: Add optional counter support in get_hw_stats callback
> > 
> > Mark Zhang (1):
> >   RDMA/core: Add a helper API rdma_free_hw_stats_struct
> 
> This seems fine now, please update the shared branch

Thanks, applied

b8dfed636fc6 net/mlx5: Add priorities for counters in RDMA namespaces
8208461d3912 net/mlx5: Add ifc bits to support optional counters

> 
> Jason
Jason Gunthorpe Oct. 12, 2021, 3:53 p.m. UTC | #3
On Sat, Oct 09, 2021 at 12:04:24PM +0300, Leon Romanovsky wrote:
> On Fri, Oct 08, 2021 at 03:57:36PM -0300, Jason Gunthorpe wrote:
> > On Fri, Oct 08, 2021 at 03:24:26PM +0300, Mark Zhang wrote:
> > > 
> > > Aharon Landau (12):
> > >   net/mlx5: Add ifc bits to support optional counters
> > >   net/mlx5: Add priorities for counters in RDMA namespaces
> > >   RDMA/counter: Add a descriptor in struct rdma_hw_stats
> > >   RDMA/counter: Add an is_disabled field in struct rdma_hw_stats
> > >   RDMA/counter: Add optional counter support
> > >   RDMA/nldev: Add support to get status of all counters
> > >   RDMA/nldev: Split nldev_stat_set_mode_doit out of nldev_stat_set_doit
> > >   RDMA/nldev: Allow optional-counter status configuration through RDMA
> > >     netlink
> > >   RDMA/mlx5: Support optional counters in hw_stats initialization
> > >   RDMA/mlx5: Add steering support in optional flow counters
> > >   RDMA/mlx5: Add modify_op_stat() support
> > >   RDMA/mlx5: Add optional counter support in get_hw_stats callback
> > > 
> > > Mark Zhang (1):
> > >   RDMA/core: Add a helper API rdma_free_hw_stats_struct
> > 
> > This seems fine now, please update the shared branch
> 
> Thanks, applied
> 
> b8dfed636fc6 net/mlx5: Add priorities for counters in RDMA namespaces
> 8208461d3912 net/mlx5: Add ifc bits to support optional counters

Done

Thanks,
Jason