| Message ID | 20190207054154.23806-1-jgg@ziepe.ca (mailing list archive) |
|---|---|
| Headers | show |
| Series | Rework device, client and client data locking | expand |
On Wed, Feb 06, 2019 at 10:41:46PM -0700, Jason Gunthorpe wrote: > From: Jason Gunthorpe <jgg@mellanox.com> > > The convoluted locking in this file has become a problem now that we want > to do more complicated things with the device registration life cycle. > > This replaces the scheme with simple approaching using xarray instead of > linked lists. Each of the three xarray's has its own lock. Which lock to > hold becomes very clear as they only protect one xarray, no more > multiple locks protecting the same data. > > xarray makes this possible because it has restartable iteration which > trivially solves the sticky locking problem between client data and client > registration. > > The xarray is a natural fit for these datastructures as we now have an ID > number direct lookup for the struct device and we have a direct lookup > access pattern for client data. The client list is converted as well as > that is a straightforward way to get unique per-client ID numbers for the > client data array and keeps the locking scheme symmetrical. > > This code could be improved with two functional improvement to xarray - > iteration over all allocated values (including NULL) and reverse find. For > now some work arounds are used and I can explore xarray improvements > later. > > v2: > - Change the xarray as discussed (Matthew) > - Don't fail rename (Gal) > > Jason Gunthorpe (8): > RDMA/device: Check that the rename is nop under the lock > RDMA/device: Ensure that security memory is always freed > RDMA/device: Call ib_cache_release_one() only from ib_device_release() > RDMA/device: Get rid of reg_state > RDMA/device: Use an ida instead of a free page in alloc_name > RDMA/devices: Use xarray to store the clients > RDMA/devices: Use xarray to store the client_data > RDMA/devices: Re-organize device.c locking Applied to for-next Jason
From: Jason Gunthorpe <jgg@mellanox.com> The convoluted locking in this file has become a problem now that we want to do more complicated things with the device registration life cycle. This replaces the scheme with simple approaching using xarray instead of linked lists. Each of the three xarray's has its own lock. Which lock to hold becomes very clear as they only protect one xarray, no more multiple locks protecting the same data. xarray makes this possible because it has restartable iteration which trivially solves the sticky locking problem between client data and client registration. The xarray is a natural fit for these datastructures as we now have an ID number direct lookup for the struct device and we have a direct lookup access pattern for client data. The client list is converted as well as that is a straightforward way to get unique per-client ID numbers for the client data array and keeps the locking scheme symmetrical. This code could be improved with two functional improvement to xarray - iteration over all allocated values (including NULL) and reverse find. For now some work arounds are used and I can explore xarray improvements later. v2: - Change the xarray as discussed (Matthew) - Don't fail rename (Gal) Jason Gunthorpe (8): RDMA/device: Check that the rename is nop under the lock RDMA/device: Ensure that security memory is always freed RDMA/device: Call ib_cache_release_one() only from ib_device_release() RDMA/device: Get rid of reg_state RDMA/device: Use an ida instead of a free page in alloc_name RDMA/devices: Use xarray to store the clients RDMA/devices: Use xarray to store the client_data RDMA/devices: Re-organize device.c locking drivers/infiniband/core/cache.c | 3 + drivers/infiniband/core/core_priv.h | 4 +- drivers/infiniband/core/device.c | 726 ++++++++++++++++------------ drivers/infiniband/core/security.c | 4 +- include/rdma/ib_verbs.h | 33 +- 5 files changed, 436 insertions(+), 334 deletions(-)