[00/15] Rework the locking and datastructures for mlx5 implicit ODP
mbox series

Message ID 20191009160934.3143-1-jgg@ziepe.ca
Headers show
Series
  • Rework the locking and datastructures for mlx5 implicit ODP
Related show

Message

Jason Gunthorpe Oct. 9, 2019, 4:09 p.m. UTC
From: Jason Gunthorpe <jgg@mellanox.com>

In order to hoist the interval tree code out of the drivers and into the
mmu_notifiers it is necessary for the drivers to not use the interval tree
for other things.

This series replaces the interval tree with an xarray and along the way
re-aligns all the locking to use a sensible SRCU model where the 'update'
step is done by modifying an xarray.

The result is overall much simpler and with less locking in the critical
path. Many functions were reworked for clarity and small details like
using 'imr' to refer to the implicit MR make the entire code flow here
more readable.

This also squashes at least two race bugs on its own, and quite possibily
more that haven't been identified.

Jason Gunthorpe (15):
  RDMA/mlx5: Use SRCU properly in ODP prefetch
  RDMA/mlx5: Split sig_err MR data into its own xarray
  RDMA/mlx5: Use a dedicated mkey xarray for ODP
  RDMA/mlx5: Delete struct mlx5_priv->mkey_table
  RDMA/mlx5: Rework implicit_mr_get_data
  RDMA/mlx5: Lift implicit_mr_alloc() into the two routines that call it
  RDMA/mlx5: Set the HW IOVA of the child MRs to their place in the tree
  RDMA/mlx5: Split implicit handling from pagefault_mr
  RDMA/mlx5: Use an xarray for the children of an implicit ODP
  RDMA/mlx5: Reduce locking in implicit_mr_get_data()
  RDMA/mlx5: Avoid double lookups on the pagefault path
  RDMA/mlx5: Rework implicit ODP destroy
  RDMA/mlx5: Do not store implicit children in the odp_mkeys xarray
  RDMA/mlx5: Do not race with mlx5_ib_invalidate_range during create and
    destroy
  RDMA/odp: Remove broken debugging call to invalidate_range

 drivers/infiniband/core/umem_odp.c            |  38 +-
 drivers/infiniband/hw/mlx5/cq.c               |  33 +-
 drivers/infiniband/hw/mlx5/devx.c             |   8 +-
 drivers/infiniband/hw/mlx5/main.c             |  17 +-
 drivers/infiniband/hw/mlx5/mlx5_ib.h          |  22 +-
 drivers/infiniband/hw/mlx5/mr.c               | 139 ++-
 drivers/infiniband/hw/mlx5/odp.c              | 976 +++++++++---------
 .../net/ethernet/mellanox/mlx5/core/main.c    |   4 -
 drivers/net/ethernet/mellanox/mlx5/core/mr.c  |  28 +-
 include/linux/mlx5/driver.h                   |   4 -
 include/rdma/ib_umem_odp.h                    |  18 -
 11 files changed, 627 insertions(+), 660 deletions(-)

Comments

Jason Gunthorpe Oct. 28, 2019, 7:47 p.m. UTC | #1
On Wed, Oct 09, 2019 at 01:09:20PM -0300, Jason Gunthorpe wrote:
> From: Jason Gunthorpe <jgg@mellanox.com>
> 
> In order to hoist the interval tree code out of the drivers and into the
> mmu_notifiers it is necessary for the drivers to not use the interval tree
> for other things.
> 
> This series replaces the interval tree with an xarray and along the way
> re-aligns all the locking to use a sensible SRCU model where the 'update'
> step is done by modifying an xarray.
> 
> The result is overall much simpler and with less locking in the critical
> path. Many functions were reworked for clarity and small details like
> using 'imr' to refer to the implicit MR make the entire code flow here
> more readable.
> 
> This also squashes at least two race bugs on its own, and quite possibily
> more that haven't been identified.
> 
> Jason Gunthorpe (15):
>   RDMA/mlx5: Use SRCU properly in ODP prefetch
>   RDMA/mlx5: Split sig_err MR data into its own xarray
>   RDMA/mlx5: Use a dedicated mkey xarray for ODP
>   RDMA/mlx5: Delete struct mlx5_priv->mkey_table
>   RDMA/mlx5: Rework implicit_mr_get_data
>   RDMA/mlx5: Lift implicit_mr_alloc() into the two routines that call it
>   RDMA/mlx5: Set the HW IOVA of the child MRs to their place in the tree
>   RDMA/mlx5: Split implicit handling from pagefault_mr
>   RDMA/mlx5: Use an xarray for the children of an implicit ODP
>   RDMA/mlx5: Reduce locking in implicit_mr_get_data()
>   RDMA/mlx5: Avoid double lookups on the pagefault path
>   RDMA/mlx5: Rework implicit ODP destroy
>   RDMA/mlx5: Do not store implicit children in the odp_mkeys xarray
>   RDMA/mlx5: Do not race with mlx5_ib_invalidate_range during create and
>     destroy
>   RDMA/odp: Remove broken debugging call to invalidate_range

Applied to for-next with the two noted fixes

Jason