| Message ID | 20190502074807.26566-1-leon@kernel.org (mailing list archive) |
|---|---|
| Headers | show |
| Series | Allow RoCE GID attribute netdev detachment | expand |
On Thu, May 02, 2019 at 10:48:00AM +0300, Leon Romanovsky wrote: > From: Leon Romanovsky <leonro@mellanox.com> > > Changelog: > v1 -> v2: > * Resent > v0 -> v1: > * Fixed wrong RCU pointer access in patch "RDMA/core: Allow detaching > gid attribute netdevice for RoCE" > > ----------------------------------------------------------------------- > > >From Parav, > > This series covers following changes. > > 1. A fix in RXE to consider right reserved space of the netdev. > 2. ib_cm to avoid accessing netdev of GID attribute. > 3. Several RoCE drivers and net/smc needs to know the mac and vlan of > the GID entry. > > Instead of open coded accessing netdev fields, we introduce an API > to get such fields filled up using new API rdma_read_gid_l2_fields(). > > 4. When there is active traffic through a GID, a QP/AH holds reference > to this GID entry. GID entry holds reference to its attached netdevice. > Due to this when netdevice (such as vlan netdev) is deleted by admin user, > its refcount is not dropped. > > Therefore, use netdev under rcu lock so that netdev reference can be > dropped when netdev and associated RoCE GID entry is deleted. This is > facilitated by existing API rdma_read_gid_attr_ndev_rcu. > > Thanks > > > Parav Pandit (7): > RDMA/rxe: Consider skb reserve space based on netdev of GID > IB/cm: Reduce dependency on gid attribute ndev check > RDMA: Introduce and use GID attr helper to read RoCE L2 fields > RDMA/cma: Use rdma_read_gid_attr_ndev_rcu to access netdev > RDMA/rxe: Use rdma_read_gid_attr_ndev_rcu to access netdev > net/smc: Use rdma_read_gid_l2_fields to L2 fields > RDMA/core: Allow detaching gid attribute netdevice for RoCE > > drivers/infiniband/core/addr.c | 1 + > drivers/infiniband/core/cache.c | 117 +++++++++++++++++++-- > drivers/infiniband/core/cm.c | 5 +- > drivers/infiniband/core/cma.c | 12 ++- > drivers/infiniband/core/sysfs.c | 13 ++- > drivers/infiniband/hw/bnxt_re/ib_verbs.c | 18 ++-- > drivers/infiniband/hw/hns/hns_roce_ah.c | 14 +-- > drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 7 +- > drivers/infiniband/hw/mlx4/ah.c | 8 +- > drivers/infiniband/hw/mlx4/qp.c | 6 +- > drivers/infiniband/hw/mlx5/main.c | 42 ++------ > drivers/infiniband/hw/ocrdma/ocrdma_ah.c | 9 +- > drivers/infiniband/hw/ocrdma/ocrdma_hw.c | 7 +- > drivers/infiniband/hw/qedr/qedr_roce_cm.c | 11 +- > drivers/infiniband/hw/qedr/verbs.c | 5 +- > drivers/infiniband/sw/rxe/rxe_net.c | 18 +++- > include/rdma/ib_cache.h | 4 + > include/rdma/ib_verbs.h | 2 +- > net/smc/smc_ib.c | 16 +-- > 19 files changed, 221 insertions(+), 94 deletions(-) Now that we have the RCU pointer in the gid_attr it is really ugly that the onstack version and the pointer version are the same type, this needs a cleanup to add some kind of gid_attr_init structure instead But otherwise applied to for-next Thanks, Jason
> -----Original Message----- > From: Jason Gunthorpe > Sent: Friday, May 3, 2019 7:45 PM > To: Leon Romanovsky <leon@kernel.org> > Cc: Doug Ledford <dledford@redhat.com>; Leon Romanovsky > <leonro@mellanox.com>; RDMA mailing list <linux-rdma@vger.kernel.org>; > Huy Nguyen <huyn@mellanox.com>; Martin Wilck <mwilck@suse.com>; > Parav Pandit <parav@mellanox.com> > Subject: Re: [PATCH rdma-next v2 0/7] Allow RoCE GID attribute netdev > detachment > > On Thu, May 02, 2019 at 10:48:00AM +0300, Leon Romanovsky wrote: > > From: Leon Romanovsky <leonro@mellanox.com> > > > > Changelog: > > v1 -> v2: > > * Resent > > v0 -> v1: > > * Fixed wrong RCU pointer access in patch "RDMA/core: Allow detaching > > gid attribute netdevice for RoCE" > > > > ---------------------------------------------------------------------- > > - > > > > >From Parav, > > > > This series covers following changes. > > > > 1. A fix in RXE to consider right reserved space of the netdev. > > 2. ib_cm to avoid accessing netdev of GID attribute. > > 3. Several RoCE drivers and net/smc needs to know the mac and vlan of > > the GID entry. > > > > Instead of open coded accessing netdev fields, we introduce an API to > > get such fields filled up using new API rdma_read_gid_l2_fields(). > > > > 4. When there is active traffic through a GID, a QP/AH holds reference > > to this GID entry. GID entry holds reference to its attached netdevice. > > Due to this when netdevice (such as vlan netdev) is deleted by admin > > user, its refcount is not dropped. > > > > Therefore, use netdev under rcu lock so that netdev reference can be > > dropped when netdev and associated RoCE GID entry is deleted. This is > > facilitated by existing API rdma_read_gid_attr_ndev_rcu. > > > > Thanks > > > > > > Parav Pandit (7): > > RDMA/rxe: Consider skb reserve space based on netdev of GID > > IB/cm: Reduce dependency on gid attribute ndev check > > RDMA: Introduce and use GID attr helper to read RoCE L2 fields > > RDMA/cma: Use rdma_read_gid_attr_ndev_rcu to access netdev > > RDMA/rxe: Use rdma_read_gid_attr_ndev_rcu to access netdev > > net/smc: Use rdma_read_gid_l2_fields to L2 fields > > RDMA/core: Allow detaching gid attribute netdevice for RoCE > > > > drivers/infiniband/core/addr.c | 1 + > > drivers/infiniband/core/cache.c | 117 +++++++++++++++++++-- > > drivers/infiniband/core/cm.c | 5 +- > > drivers/infiniband/core/cma.c | 12 ++- > > drivers/infiniband/core/sysfs.c | 13 ++- > > drivers/infiniband/hw/bnxt_re/ib_verbs.c | 18 ++-- > > drivers/infiniband/hw/hns/hns_roce_ah.c | 14 +-- > > drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 7 +- > > drivers/infiniband/hw/mlx4/ah.c | 8 +- > > drivers/infiniband/hw/mlx4/qp.c | 6 +- > > drivers/infiniband/hw/mlx5/main.c | 42 ++------ > > drivers/infiniband/hw/ocrdma/ocrdma_ah.c | 9 +- > > drivers/infiniband/hw/ocrdma/ocrdma_hw.c | 7 +- > > drivers/infiniband/hw/qedr/qedr_roce_cm.c | 11 +- > > drivers/infiniband/hw/qedr/verbs.c | 5 +- > > drivers/infiniband/sw/rxe/rxe_net.c | 18 +++- > > include/rdma/ib_cache.h | 4 + > > include/rdma/ib_verbs.h | 2 +- > > net/smc/smc_ib.c | 16 +-- > > 19 files changed, 221 insertions(+), 94 deletions(-) > > Now that we have the RCU pointer in the gid_attr it is really ugly that the > onstack version and the pointer version are the same type, this needs a > cleanup to add some kind of gid_attr_init structure instead > Hmm ok. Other option I was considering is to pass ndev pointer as additional argument in add_gid code flow. But it was too much changes in roce_gid_mgmt.c. We can possibly keep the scope limited of gid_init_attr within cache.c and git_mgmt.c. > But otherwise applied to for-next > Thanks > Thanks, > Jason
From: Leon Romanovsky <leonro@mellanox.com> Changelog: v1 -> v2: * Resent v0 -> v1: * Fixed wrong RCU pointer access in patch "RDMA/core: Allow detaching gid attribute netdevice for RoCE" ----------------------------------------------------------------------- From Parav, This series covers following changes. 1. A fix in RXE to consider right reserved space of the netdev. 2. ib_cm to avoid accessing netdev of GID attribute. 3. Several RoCE drivers and net/smc needs to know the mac and vlan of the GID entry. Instead of open coded accessing netdev fields, we introduce an API to get such fields filled up using new API rdma_read_gid_l2_fields(). 4. When there is active traffic through a GID, a QP/AH holds reference to this GID entry. GID entry holds reference to its attached netdevice. Due to this when netdevice (such as vlan netdev) is deleted by admin user, its refcount is not dropped. Therefore, use netdev under rcu lock so that netdev reference can be dropped when netdev and associated RoCE GID entry is deleted. This is facilitated by existing API rdma_read_gid_attr_ndev_rcu. Thanks Parav Pandit (7): RDMA/rxe: Consider skb reserve space based on netdev of GID IB/cm: Reduce dependency on gid attribute ndev check RDMA: Introduce and use GID attr helper to read RoCE L2 fields RDMA/cma: Use rdma_read_gid_attr_ndev_rcu to access netdev RDMA/rxe: Use rdma_read_gid_attr_ndev_rcu to access netdev net/smc: Use rdma_read_gid_l2_fields to L2 fields RDMA/core: Allow detaching gid attribute netdevice for RoCE drivers/infiniband/core/addr.c | 1 + drivers/infiniband/core/cache.c | 117 +++++++++++++++++++-- drivers/infiniband/core/cm.c | 5 +- drivers/infiniband/core/cma.c | 12 ++- drivers/infiniband/core/sysfs.c | 13 ++- drivers/infiniband/hw/bnxt_re/ib_verbs.c | 18 ++-- drivers/infiniband/hw/hns/hns_roce_ah.c | 14 +-- drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 7 +- drivers/infiniband/hw/mlx4/ah.c | 8 +- drivers/infiniband/hw/mlx4/qp.c | 6 +- drivers/infiniband/hw/mlx5/main.c | 42 ++------ drivers/infiniband/hw/ocrdma/ocrdma_ah.c | 9 +- drivers/infiniband/hw/ocrdma/ocrdma_hw.c | 7 +- drivers/infiniband/hw/qedr/qedr_roce_cm.c | 11 +- drivers/infiniband/hw/qedr/verbs.c | 5 +- drivers/infiniband/sw/rxe/rxe_net.c | 18 +++- include/rdma/ib_cache.h | 4 + include/rdma/ib_verbs.h | 2 +- net/smc/smc_ib.c | 16 +-- 19 files changed, 221 insertions(+), 94 deletions(-) -- 2.20.1