| Message ID | 20190213041256.22437-1-jgg@ziepe.ca (mailing list archive) |
|---|---|
| Headers | show |
| Series | Revise device handling in rxe | expand |
On Tue, Feb 12, 2019 at 09:12:46PM -0700, Jason Gunthorpe wrote: > From: Jason Gunthorpe <jgg@mellanox.com> > > This creates a series of additional core entry points that meet the needs > of a driver like rxe, and other drivers, as it relates to their > register/unregister and netdev notifier paths. > > The existing driver code is generally full of races in this area, while > these new core routines sit on the new locking schemes and provide a > complete race free solution. > > rxe is used as an example conversion, but other drivers should be > converted as well. > > The approach uses a combination of the registration refcount (combined > with core level tracking and lookup) and a series of new lifecycle > callbacks that the driver can implement to reliably get the correct > destruction sequence. These work together with the new unregistration > entry points to allow the driver to have a robust lifetime model. > > To protect the typical asynchronous access via netdev notifiers from > racing with unregistration this adds ib_device_get_by_netdev(). It does > the lookup and holds the registration refcount under a proper core driven > locking scheme. > > usnic has already posted a patch to use this api, siw will need it, rxe is > converted here and other drivers should be adjusted as well. > > Jason Gunthorpe (10): > RDMA: Add and use rdma_for_each_port > RDMA/device: Consolidate ib_device per_port data into one place > RDMA/cache: Move the cache per-port data into the main ib_port_data > RDMA/device: Add ib_device_set_netdev() as an alternative to > get_netdev > RDMA/device: Add ib_device_get_by_netdev() > RDMA/rxe: Use ib_device_get_by_netdev() instead of open coding > RDMA/device: Provide APIs from the core code to help unregistration > RDMA/rxe: Use driver_unregister and new unregistration API > RDMA/rxe: Add ib_device_get_by_name() and use it in rxe > RDMA/rxe: Close a race after ib_register_device Applied to for-next Jason
From: Jason Gunthorpe <jgg@mellanox.com> This creates a series of additional core entry points that meet the needs of a driver like rxe, and other drivers, as it relates to their register/unregister and netdev notifier paths. The existing driver code is generally full of races in this area, while these new core routines sit on the new locking schemes and provide a complete race free solution. rxe is used as an example conversion, but other drivers should be converted as well. The approach uses a combination of the registration refcount (combined with core level tracking and lookup) and a series of new lifecycle callbacks that the driver can implement to reliably get the correct destruction sequence. These work together with the new unregistration entry points to allow the driver to have a robust lifetime model. To protect the typical asynchronous access via netdev notifiers from racing with unregistration this adds ib_device_get_by_netdev(). It does the lookup and holds the registration refcount under a proper core driven locking scheme. usnic has already posted a patch to use this api, siw will need it, rxe is converted here and other drivers should be adjusted as well. Jason Gunthorpe (10): RDMA: Add and use rdma_for_each_port RDMA/device: Consolidate ib_device per_port data into one place RDMA/cache: Move the cache per-port data into the main ib_port_data RDMA/device: Add ib_device_set_netdev() as an alternative to get_netdev RDMA/device: Add ib_device_get_by_netdev() RDMA/rxe: Use ib_device_get_by_netdev() instead of open coding RDMA/device: Provide APIs from the core code to help unregistration RDMA/rxe: Use driver_unregister and new unregistration API RDMA/rxe: Add ib_device_get_by_name() and use it in rxe RDMA/rxe: Close a race after ib_register_device .clang-format | 1 + drivers/infiniband/core/cache.c | 121 ++--- drivers/infiniband/core/cma.c | 7 +- drivers/infiniband/core/core_priv.h | 3 + drivers/infiniband/core/device.c | 620 ++++++++++++++++++---- drivers/infiniband/core/mad.c | 4 +- drivers/infiniband/core/nldev.c | 8 +- drivers/infiniband/core/security.c | 35 +- drivers/infiniband/core/sysfs.c | 12 +- drivers/infiniband/core/user_mad.c | 9 +- drivers/infiniband/core/verbs.c | 5 +- drivers/infiniband/sw/rxe/rxe.c | 41 +- drivers/infiniband/sw/rxe/rxe.h | 14 +- drivers/infiniband/sw/rxe/rxe_loc.h | 2 +- drivers/infiniband/sw/rxe/rxe_net.c | 84 +-- drivers/infiniband/sw/rxe/rxe_net.h | 2 +- drivers/infiniband/sw/rxe/rxe_pool.c | 14 +- drivers/infiniband/sw/rxe/rxe_sysfs.c | 36 +- drivers/infiniband/sw/rxe/rxe_verbs.c | 48 +- drivers/infiniband/sw/rxe/rxe_verbs.h | 3 - drivers/infiniband/ulp/ipoib/ipoib_main.c | 4 +- drivers/infiniband/ulp/srp/ib_srp.c | 5 +- include/rdma/ib_verbs.h | 118 ++-- 23 files changed, 765 insertions(+), 431 deletions(-)