| Message ID | 1544605748-20791-2-git-send-email-galpress@amazon.com (mailing list archive) |
|---|---|
| State | Accepted |
| Headers | show |
| Series | Mark create/destroy address handle calls as atomic/sleepable | expand |
On Wed, Dec 12, 2018 at 11:09:05AM +0200, Gal Pressman wrote: > Introduce a 'flags' field to create address handle callback and add a > flag that marks whether the callback is executed in an atomic context or > not. > This will allow drivers to wait for completion instead of polling for it > when it is allowed. > > Signed-off-by: Gal Pressman <galpress@amazon.com> > --- > drivers/infiniband/core/cm.c | 2 +- > drivers/infiniband/core/sa_query.c | 3 ++- > drivers/infiniband/core/verbs.c | 15 ++++++++++----- > drivers/infiniband/hw/bnxt_re/ib_verbs.c | 1 + > drivers/infiniband/hw/bnxt_re/ib_verbs.h | 1 + > drivers/infiniband/hw/hfi1/mad.c | 2 +- > drivers/infiniband/hw/hns/hns_roce_ah.c | 1 + > drivers/infiniband/hw/hns/hns_roce_device.h | 1 + > drivers/infiniband/hw/mlx4/ah.c | 4 ++-- > drivers/infiniband/hw/mlx4/mad.c | 4 ++-- > drivers/infiniband/hw/mlx4/mlx4_ib.h | 2 +- > drivers/infiniband/hw/mlx5/ah.c | 2 +- > drivers/infiniband/hw/mlx5/mlx5_ib.h | 2 +- > drivers/infiniband/hw/mthca/mthca_mad.c | 2 +- > drivers/infiniband/hw/mthca/mthca_provider.c | 1 + > drivers/infiniband/hw/ocrdma/ocrdma_ah.c | 2 +- > drivers/infiniband/hw/ocrdma/ocrdma_ah.h | 2 +- > drivers/infiniband/hw/qedr/verbs.c | 2 +- > drivers/infiniband/hw/qedr/verbs.h | 2 +- > drivers/infiniband/hw/qib/qib_verbs.c | 2 +- > drivers/infiniband/hw/usnic/usnic_ib_verbs.c | 1 + > drivers/infiniband/hw/usnic/usnic_ib_verbs.h | 1 + > drivers/infiniband/hw/vmw_pvrdma/pvrdma_verbs.c | 3 ++- > drivers/infiniband/hw/vmw_pvrdma/pvrdma_verbs.h | 2 +- > drivers/infiniband/sw/rdmavt/ah.c | 2 ++ > drivers/infiniband/sw/rdmavt/ah.h | 1 + > drivers/infiniband/sw/rxe/rxe_verbs.c | 1 + > drivers/infiniband/ulp/ipoib/ipoib_ib.c | 2 +- > drivers/infiniband/ulp/opa_vnic/opa_vnic_vema.c | 2 +- > include/rdma/ib_verbs.h | 10 +++++++++- > 30 files changed, 52 insertions(+), 26 deletions(-) Have you missed 'ib_uverbs_create_ah' ? I ask this because you created new flag between uverbs and rdma layers and it seems that you are not 100% consistent here where you pass this flag from uverbs in error path of ib_uverbs_create_ah (patch 2/4) but not in the valid path. I'd expect you to pass this flag from ib_uverbs_create_ah to rdma_create_user_ah as well. > > diff --git a/drivers/infiniband/core/cm.c b/drivers/infiniband/core/cm.c > index edb2cb758be7..cf5b3c4314bb 100644 > --- a/drivers/infiniband/core/cm.c > +++ b/drivers/infiniband/core/cm.c > @@ -343,7 +343,7 @@ static int cm_alloc_msg(struct cm_id_private *cm_id_priv, > ret = -ENODEV; > goto out; > } > - ah = rdma_create_ah(mad_agent->qp->pd, &av->ah_attr); > + ah = rdma_create_ah(mad_agent->qp->pd, &av->ah_attr, 0); > if (IS_ERR(ah)) { > ret = PTR_ERR(ah); > goto out; > diff --git a/drivers/infiniband/core/sa_query.c b/drivers/infiniband/core/sa_query.c > index be5ba5e15496..fb96d9df1fdd 100644 > --- a/drivers/infiniband/core/sa_query.c > +++ b/drivers/infiniband/core/sa_query.c > @@ -2276,7 +2276,8 @@ static void update_sm_ah(struct work_struct *work) > cpu_to_be64(IB_SA_WELL_KNOWN_GUID)); > } > > - new_ah->ah = rdma_create_ah(port->agent->qp->pd, &ah_attr); > + new_ah->ah = rdma_create_ah(port->agent->qp->pd, &ah_attr, > + RDMA_CREATE_AH_SLEEPABLE); > if (IS_ERR(new_ah->ah)) { > pr_warn("Couldn't create new SM AH\n"); > kfree(new_ah); > diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c > index 178899e3ce73..6fd8179ce82e 100644 > --- a/drivers/infiniband/core/verbs.c > +++ b/drivers/infiniband/core/verbs.c > @@ -475,14 +475,17 @@ rdma_update_sgid_attr(struct rdma_ah_attr *ah_attr, > > static struct ib_ah *_rdma_create_ah(struct ib_pd *pd, > struct rdma_ah_attr *ah_attr, > + u32 flags, > struct ib_udata *udata) > { Why you do not add the creation flags to rdma_ah_attr? Something like struct rdma_ah_attr { [...] u8 ah_flags; u32 ah_creation_flags; [...] } It is already set to 0 in ib_uverbs_create_ah. You just need to make sure the rest of the none user path (such as mad) also set this to 0. > struct ib_ah *ah; > > + might_sleep_if(flags & RDMA_CREATE_AH_SLEEPABLE); > + > if (!pd->device->create_ah) > return ERR_PTR(-EOPNOTSUPP); > > - ah = pd->device->create_ah(pd, ah_attr, udata); > + ah = pd->device->create_ah(pd, ah_attr, flags, udata); >
On 12-Dec-18 12:24, Shamir Rabinovitch wrote: > On Wed, Dec 12, 2018 at 11:09:05AM +0200, Gal Pressman wrote: >> Introduce a 'flags' field to create address handle callback and add a >> flag that marks whether the callback is executed in an atomic context or >> not. >> This will allow drivers to wait for completion instead of polling for it >> when it is allowed. >> >> Signed-off-by: Gal Pressman <galpress@amazon.com> >> --- >> drivers/infiniband/core/cm.c | 2 +- >> drivers/infiniband/core/sa_query.c | 3 ++- >> drivers/infiniband/core/verbs.c | 15 ++++++++++----- >> drivers/infiniband/hw/bnxt_re/ib_verbs.c | 1 + >> drivers/infiniband/hw/bnxt_re/ib_verbs.h | 1 + >> drivers/infiniband/hw/hfi1/mad.c | 2 +- >> drivers/infiniband/hw/hns/hns_roce_ah.c | 1 + >> drivers/infiniband/hw/hns/hns_roce_device.h | 1 + >> drivers/infiniband/hw/mlx4/ah.c | 4 ++-- >> drivers/infiniband/hw/mlx4/mad.c | 4 ++-- >> drivers/infiniband/hw/mlx4/mlx4_ib.h | 2 +- >> drivers/infiniband/hw/mlx5/ah.c | 2 +- >> drivers/infiniband/hw/mlx5/mlx5_ib.h | 2 +- >> drivers/infiniband/hw/mthca/mthca_mad.c | 2 +- >> drivers/infiniband/hw/mthca/mthca_provider.c | 1 + >> drivers/infiniband/hw/ocrdma/ocrdma_ah.c | 2 +- >> drivers/infiniband/hw/ocrdma/ocrdma_ah.h | 2 +- >> drivers/infiniband/hw/qedr/verbs.c | 2 +- >> drivers/infiniband/hw/qedr/verbs.h | 2 +- >> drivers/infiniband/hw/qib/qib_verbs.c | 2 +- >> drivers/infiniband/hw/usnic/usnic_ib_verbs.c | 1 + >> drivers/infiniband/hw/usnic/usnic_ib_verbs.h | 1 + >> drivers/infiniband/hw/vmw_pvrdma/pvrdma_verbs.c | 3 ++- >> drivers/infiniband/hw/vmw_pvrdma/pvrdma_verbs.h | 2 +- >> drivers/infiniband/sw/rdmavt/ah.c | 2 ++ >> drivers/infiniband/sw/rdmavt/ah.h | 1 + >> drivers/infiniband/sw/rxe/rxe_verbs.c | 1 + >> drivers/infiniband/ulp/ipoib/ipoib_ib.c | 2 +- >> drivers/infiniband/ulp/opa_vnic/opa_vnic_vema.c | 2 +- >> include/rdma/ib_verbs.h | 10 +++++++++- >> 30 files changed, 52 insertions(+), 26 deletions(-) > > Have you missed 'ib_uverbs_create_ah' ? > > I ask this because you created new flag between uverbs and rdma layers > and it seems that you are not 100% consistent here where you pass this > flag from uverbs in error path of ib_uverbs_create_ah (patch 2/4) but > not in the valid path. > > I'd expect you to pass this flag from ib_uverbs_create_ah to > rdma_create_user_ah as well. rdma_create_ah and rdma_destroy_ah aren't exactly symmetric, _rdma_create_ah and rdma_destroy_ah are. _rdma_create_ah is called from two functions: 1. rdma_create_ah, which has the 'flags' parameter, since it's called both from atomic and non-atomic context. 2. rdma_create_user_ah, which doesn't need the 'flags' parameter, since it's always called in non-atomic context, hence always passes RDMA_CREATE_AH_SLEEPABLE flag to _rdma_create_ah. The function you mentioned (ib_uverbs_create_ah) doesn't need the flags on create (since it calls rdma_creat_user_ah) and needs it on destroy. > >> >> diff --git a/drivers/infiniband/core/cm.c b/drivers/infiniband/core/cm.c >> index edb2cb758be7..cf5b3c4314bb 100644 >> --- a/drivers/infiniband/core/cm.c >> +++ b/drivers/infiniband/core/cm.c >> @@ -343,7 +343,7 @@ static int cm_alloc_msg(struct cm_id_private *cm_id_priv, >> ret = -ENODEV; >> goto out; >> } >> - ah = rdma_create_ah(mad_agent->qp->pd, &av->ah_attr); >> + ah = rdma_create_ah(mad_agent->qp->pd, &av->ah_attr, 0); >> if (IS_ERR(ah)) { >> ret = PTR_ERR(ah); >> goto out; >> diff --git a/drivers/infiniband/core/sa_query.c b/drivers/infiniband/core/sa_query.c >> index be5ba5e15496..fb96d9df1fdd 100644 >> --- a/drivers/infiniband/core/sa_query.c >> +++ b/drivers/infiniband/core/sa_query.c >> @@ -2276,7 +2276,8 @@ static void update_sm_ah(struct work_struct *work) >> cpu_to_be64(IB_SA_WELL_KNOWN_GUID)); >> } >> >> - new_ah->ah = rdma_create_ah(port->agent->qp->pd, &ah_attr); >> + new_ah->ah = rdma_create_ah(port->agent->qp->pd, &ah_attr, >> + RDMA_CREATE_AH_SLEEPABLE); >> if (IS_ERR(new_ah->ah)) { >> pr_warn("Couldn't create new SM AH\n"); >> kfree(new_ah); >> diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c >> index 178899e3ce73..6fd8179ce82e 100644 >> --- a/drivers/infiniband/core/verbs.c >> +++ b/drivers/infiniband/core/verbs.c >> @@ -475,14 +475,17 @@ rdma_update_sgid_attr(struct rdma_ah_attr *ah_attr, >> >> static struct ib_ah *_rdma_create_ah(struct ib_pd *pd, >> struct rdma_ah_attr *ah_attr, >> + u32 flags, >> struct ib_udata *udata) >> { > > Why you do not add the creation flags to rdma_ah_attr? > > Something like > > struct rdma_ah_attr { > [...] > u8 ah_flags; > u32 ah_creation_flags; > [...] > } > > It is already set to 0 in ib_uverbs_create_ah. > > You just need to make sure the rest of the none user path (such as mad) > also set this to 0. I considered it, but sleepable is not an attribute of the address handle itself, it's an attribute of the function call context so placing it as an extra parameter made more sense to me. > >> struct ib_ah *ah; >> >> + might_sleep_if(flags & RDMA_CREATE_AH_SLEEPABLE); >> + >> if (!pd->device->create_ah) >> return ERR_PTR(-EOPNOTSUPP); >> >> - ah = pd->device->create_ah(pd, ah_attr, udata); >> + ah = pd->device->create_ah(pd, ah_attr, flags, udata); >>
diff --git a/drivers/infiniband/core/cm.c b/drivers/infiniband/core/cm.c index edb2cb758be7..cf5b3c4314bb 100644 --- a/drivers/infiniband/core/cm.c +++ b/drivers/infiniband/core/cm.c @@ -343,7 +343,7 @@ static int cm_alloc_msg(struct cm_id_private *cm_id_priv, ret = -ENODEV; goto out; } - ah = rdma_create_ah(mad_agent->qp->pd, &av->ah_attr); + ah = rdma_create_ah(mad_agent->qp->pd, &av->ah_attr, 0); if (IS_ERR(ah)) { ret = PTR_ERR(ah); goto out; diff --git a/drivers/infiniband/core/sa_query.c b/drivers/infiniband/core/sa_query.c index be5ba5e15496..fb96d9df1fdd 100644 --- a/drivers/infiniband/core/sa_query.c +++ b/drivers/infiniband/core/sa_query.c @@ -2276,7 +2276,8 @@ static void update_sm_ah(struct work_struct *work) cpu_to_be64(IB_SA_WELL_KNOWN_GUID)); } - new_ah->ah = rdma_create_ah(port->agent->qp->pd, &ah_attr); + new_ah->ah = rdma_create_ah(port->agent->qp->pd, &ah_attr, + RDMA_CREATE_AH_SLEEPABLE); if (IS_ERR(new_ah->ah)) { pr_warn("Couldn't create new SM AH\n"); kfree(new_ah); diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c index 178899e3ce73..6fd8179ce82e 100644 --- a/drivers/infiniband/core/verbs.c +++ b/drivers/infiniband/core/verbs.c @@ -475,14 +475,17 @@ rdma_update_sgid_attr(struct rdma_ah_attr *ah_attr, static struct ib_ah *_rdma_create_ah(struct ib_pd *pd, struct rdma_ah_attr *ah_attr, + u32 flags, struct ib_udata *udata) { struct ib_ah *ah; + might_sleep_if(flags & RDMA_CREATE_AH_SLEEPABLE); + if (!pd->device->create_ah) return ERR_PTR(-EOPNOTSUPP); - ah = pd->device->create_ah(pd, ah_attr, udata); + ah = pd->device->create_ah(pd, ah_attr, flags, udata); if (!IS_ERR(ah)) { ah->device = pd->device; @@ -502,12 +505,14 @@ static struct ib_ah *_rdma_create_ah(struct ib_pd *pd, * given address vector. * @pd: The protection domain associated with the address handle. * @ah_attr: The attributes of the address vector. + * @flags: Create address handle flags (see enum rdma_create_ah_flags). * * It returns 0 on success and returns appropriate error code on error. * The address handle is used to reference a local or global destination * in all UD QP post sends. */ -struct ib_ah *rdma_create_ah(struct ib_pd *pd, struct rdma_ah_attr *ah_attr) +struct ib_ah *rdma_create_ah(struct ib_pd *pd, struct rdma_ah_attr *ah_attr, + u32 flags) { const struct ib_gid_attr *old_sgid_attr; struct ib_ah *ah; @@ -517,7 +522,7 @@ struct ib_ah *rdma_create_ah(struct ib_pd *pd, struct rdma_ah_attr *ah_attr) if (ret) return ERR_PTR(ret); - ah = _rdma_create_ah(pd, ah_attr, NULL); + ah = _rdma_create_ah(pd, ah_attr, flags, NULL); rdma_unfill_sgid_attr(ah_attr, old_sgid_attr); return ah; @@ -557,7 +562,7 @@ struct ib_ah *rdma_create_user_ah(struct ib_pd *pd, } } - ah = _rdma_create_ah(pd, ah_attr, udata); + ah = _rdma_create_ah(pd, ah_attr, RDMA_CREATE_AH_SLEEPABLE, udata); out: rdma_unfill_sgid_attr(ah_attr, old_sgid_attr); @@ -869,7 +874,7 @@ struct ib_ah *ib_create_ah_from_wc(struct ib_pd *pd, const struct ib_wc *wc, if (ret) return ERR_PTR(ret); - ah = rdma_create_ah(pd, &ah_attr); + ah = rdma_create_ah(pd, &ah_attr, RDMA_CREATE_AH_SLEEPABLE); rdma_destroy_ah_attr(&ah_attr); return ah; diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.c b/drivers/infiniband/hw/bnxt_re/ib_verbs.c index 54fdd4cf5288..d44a2551bcd5 100644 --- a/drivers/infiniband/hw/bnxt_re/ib_verbs.c +++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.c @@ -664,6 +664,7 @@ int bnxt_re_destroy_ah(struct ib_ah *ib_ah) struct ib_ah *bnxt_re_create_ah(struct ib_pd *ib_pd, struct rdma_ah_attr *ah_attr, + u32 flags, struct ib_udata *udata) { struct bnxt_re_pd *pd = container_of(ib_pd, struct bnxt_re_pd, ib_pd); diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.h b/drivers/infiniband/hw/bnxt_re/ib_verbs.h index aa33e7b82c84..6675db4b4ba4 100644 --- a/drivers/infiniband/hw/bnxt_re/ib_verbs.h +++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.h @@ -169,6 +169,7 @@ struct ib_pd *bnxt_re_alloc_pd(struct ib_device *ibdev, int bnxt_re_dealloc_pd(struct ib_pd *pd); struct ib_ah *bnxt_re_create_ah(struct ib_pd *pd, struct rdma_ah_attr *ah_attr, + u32 flags, struct ib_udata *udata); int bnxt_re_modify_ah(struct ib_ah *ah, struct rdma_ah_attr *ah_attr); int bnxt_re_query_ah(struct ib_ah *ah, struct rdma_ah_attr *ah_attr); diff --git a/drivers/infiniband/hw/hfi1/mad.c b/drivers/infiniband/hw/hfi1/mad.c index 88a0cf930136..4228393e6c4c 100644 --- a/drivers/infiniband/hw/hfi1/mad.c +++ b/drivers/infiniband/hw/hfi1/mad.c @@ -305,7 +305,7 @@ static struct ib_ah *hfi1_create_qp0_ah(struct hfi1_ibport *ibp, u32 dlid) rcu_read_lock(); qp0 = rcu_dereference(ibp->rvp.qp[0]); if (qp0) - ah = rdma_create_ah(qp0->ibqp.pd, &attr); + ah = rdma_create_ah(qp0->ibqp.pd, &attr, 0); rcu_read_unlock(); return ah; } diff --git a/drivers/infiniband/hw/hns/hns_roce_ah.c b/drivers/infiniband/hw/hns/hns_roce_ah.c index 9990dc9eb96a..dae4c54f021d 100644 --- a/drivers/infiniband/hw/hns/hns_roce_ah.c +++ b/drivers/infiniband/hw/hns/hns_roce_ah.c @@ -41,6 +41,7 @@ struct ib_ah *hns_roce_create_ah(struct ib_pd *ibpd, struct rdma_ah_attr *ah_attr, + u32 flags, struct ib_udata *udata) { struct hns_roce_dev *hr_dev = to_hr_dev(ibpd->device); diff --git a/drivers/infiniband/hw/hns/hns_roce_device.h b/drivers/infiniband/hw/hns/hns_roce_device.h index 779dd4c409cb..19a56bcb2ab0 100644 --- a/drivers/infiniband/hw/hns/hns_roce_device.h +++ b/drivers/infiniband/hw/hns/hns_roce_device.h @@ -1054,6 +1054,7 @@ void hns_roce_bitmap_free_range(struct hns_roce_bitmap *bitmap, struct ib_ah *hns_roce_create_ah(struct ib_pd *pd, struct rdma_ah_attr *ah_attr, + u32 flags, struct ib_udata *udata); int hns_roce_query_ah(struct ib_ah *ibah, struct rdma_ah_attr *ah_attr); int hns_roce_destroy_ah(struct ib_ah *ah); diff --git a/drivers/infiniband/hw/mlx4/ah.c b/drivers/infiniband/hw/mlx4/ah.c index e9e3a6f390db..6c0e6bcf1bff 100644 --- a/drivers/infiniband/hw/mlx4/ah.c +++ b/drivers/infiniband/hw/mlx4/ah.c @@ -144,7 +144,7 @@ static struct ib_ah *create_iboe_ah(struct ib_pd *pd, } struct ib_ah *mlx4_ib_create_ah(struct ib_pd *pd, struct rdma_ah_attr *ah_attr, - struct ib_udata *udata) + u32 flags, struct ib_udata *udata) { struct mlx4_ib_ah *ah; @@ -189,7 +189,7 @@ struct ib_ah *mlx4_ib_create_ah_slave(struct ib_pd *pd, slave_attr.grh.sgid_attr = NULL; slave_attr.grh.sgid_index = slave_sgid_index; - ah = mlx4_ib_create_ah(pd, &slave_attr, NULL); + ah = mlx4_ib_create_ah(pd, &slave_attr, 0, NULL); if (IS_ERR(ah)) return ah; diff --git a/drivers/infiniband/hw/mlx4/mad.c b/drivers/infiniband/hw/mlx4/mad.c index 8942f5f7f04d..b227f7c2fca2 100644 --- a/drivers/infiniband/hw/mlx4/mad.c +++ b/drivers/infiniband/hw/mlx4/mad.c @@ -202,7 +202,7 @@ static void update_sm_ah(struct mlx4_ib_dev *dev, u8 port_num, u16 lid, u8 sl) rdma_ah_set_port_num(&ah_attr, port_num); new_ah = rdma_create_ah(dev->send_agent[port_num - 1][0]->qp->pd, - &ah_attr); + &ah_attr, 0); if (IS_ERR(new_ah)) return; @@ -567,7 +567,7 @@ int mlx4_ib_send_to_slave(struct mlx4_ib_dev *dev, int slave, u8 port, return -EINVAL; rdma_ah_set_grh(&attr, &dgid, 0, 0, 0, 0); } - ah = rdma_create_ah(tun_ctx->pd, &attr); + ah = rdma_create_ah(tun_ctx->pd, &attr, 0); if (IS_ERR(ah)) return -ENOMEM; diff --git a/drivers/infiniband/hw/mlx4/mlx4_ib.h b/drivers/infiniband/hw/mlx4/mlx4_ib.h index 5cb52424912e..7b827b185d5f 100644 --- a/drivers/infiniband/hw/mlx4/mlx4_ib.h +++ b/drivers/infiniband/hw/mlx4/mlx4_ib.h @@ -754,7 +754,7 @@ void __mlx4_ib_cq_clean(struct mlx4_ib_cq *cq, u32 qpn, struct mlx4_ib_srq *srq) void mlx4_ib_cq_clean(struct mlx4_ib_cq *cq, u32 qpn, struct mlx4_ib_srq *srq); struct ib_ah *mlx4_ib_create_ah(struct ib_pd *pd, struct rdma_ah_attr *ah_attr, - struct ib_udata *udata); + u32 flags, struct ib_udata *udata); struct ib_ah *mlx4_ib_create_ah_slave(struct ib_pd *pd, struct rdma_ah_attr *ah_attr, int slave_sgid_index, u8 *s_mac, diff --git a/drivers/infiniband/hw/mlx5/ah.c b/drivers/infiniband/hw/mlx5/ah.c index ffd03bf1a71e..536c2c067b94 100644 --- a/drivers/infiniband/hw/mlx5/ah.c +++ b/drivers/infiniband/hw/mlx5/ah.c @@ -72,7 +72,7 @@ static struct ib_ah *create_ib_ah(struct mlx5_ib_dev *dev, } struct ib_ah *mlx5_ib_create_ah(struct ib_pd *pd, struct rdma_ah_attr *ah_attr, - struct ib_udata *udata) + u32 flags, struct ib_udata *udata) { struct mlx5_ib_ah *ah; diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h index 24cb2f793210..2c9e6045d11a 100644 --- a/drivers/infiniband/hw/mlx5/mlx5_ib.h +++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h @@ -1042,7 +1042,7 @@ int mlx5_MAD_IFC(struct mlx5_ib_dev *dev, int ignore_mkey, int ignore_bkey, u8 port, const struct ib_wc *in_wc, const struct ib_grh *in_grh, const void *in_mad, void *response_mad); struct ib_ah *mlx5_ib_create_ah(struct ib_pd *pd, struct rdma_ah_attr *ah_attr, - struct ib_udata *udata); + u32 flags, struct ib_udata *udata); int mlx5_ib_query_ah(struct ib_ah *ibah, struct rdma_ah_attr *ah_attr); int mlx5_ib_destroy_ah(struct ib_ah *ah); struct ib_srq *mlx5_ib_create_srq(struct ib_pd *pd, diff --git a/drivers/infiniband/hw/mthca/mthca_mad.c b/drivers/infiniband/hw/mthca/mthca_mad.c index 2e5dc0a67cfc..4a5916d8f7da 100644 --- a/drivers/infiniband/hw/mthca/mthca_mad.c +++ b/drivers/infiniband/hw/mthca/mthca_mad.c @@ -89,7 +89,7 @@ static void update_sm_ah(struct mthca_dev *dev, rdma_ah_set_port_num(&ah_attr, port_num); new_ah = rdma_create_ah(dev->send_agent[port_num - 1][0]->qp->pd, - &ah_attr); + &ah_attr, 0); if (IS_ERR(new_ah)) return; diff --git a/drivers/infiniband/hw/mthca/mthca_provider.c b/drivers/infiniband/hw/mthca/mthca_provider.c index 691c6f048938..b235e9270122 100644 --- a/drivers/infiniband/hw/mthca/mthca_provider.c +++ b/drivers/infiniband/hw/mthca/mthca_provider.c @@ -412,6 +412,7 @@ static int mthca_dealloc_pd(struct ib_pd *pd) static struct ib_ah *mthca_ah_create(struct ib_pd *pd, struct rdma_ah_attr *ah_attr, + u32 flags, struct ib_udata *udata) { diff --git a/drivers/infiniband/hw/ocrdma/ocrdma_ah.c b/drivers/infiniband/hw/ocrdma/ocrdma_ah.c index 58188fe5aed2..62c502220724 100644 --- a/drivers/infiniband/hw/ocrdma/ocrdma_ah.c +++ b/drivers/infiniband/hw/ocrdma/ocrdma_ah.c @@ -157,7 +157,7 @@ static inline int set_av_attr(struct ocrdma_dev *dev, struct ocrdma_ah *ah, } struct ib_ah *ocrdma_create_ah(struct ib_pd *ibpd, struct rdma_ah_attr *attr, - struct ib_udata *udata) + u32 flags, struct ib_udata *udata) { u32 *ahid_addr; int status; diff --git a/drivers/infiniband/hw/ocrdma/ocrdma_ah.h b/drivers/infiniband/hw/ocrdma/ocrdma_ah.h index c0c32c9b80ae..c407022d885c 100644 --- a/drivers/infiniband/hw/ocrdma/ocrdma_ah.h +++ b/drivers/infiniband/hw/ocrdma/ocrdma_ah.h @@ -52,7 +52,7 @@ enum { }; struct ib_ah *ocrdma_create_ah(struct ib_pd *pd, struct rdma_ah_attr *ah_attr, - struct ib_udata *udata); + u32 flags, struct ib_udata *udata); int ocrdma_destroy_ah(struct ib_ah *ah); int ocrdma_query_ah(struct ib_ah *ah, struct rdma_ah_attr *ah_attr); diff --git a/drivers/infiniband/hw/qedr/verbs.c b/drivers/infiniband/hw/qedr/verbs.c index 82ee4b4a7084..0116b7ccd556 100644 --- a/drivers/infiniband/hw/qedr/verbs.c +++ b/drivers/infiniband/hw/qedr/verbs.c @@ -2626,7 +2626,7 @@ int qedr_destroy_qp(struct ib_qp *ibqp) } struct ib_ah *qedr_create_ah(struct ib_pd *ibpd, struct rdma_ah_attr *attr, - struct ib_udata *udata) + u32 flags, struct ib_udata *udata) { struct qedr_ah *ah; diff --git a/drivers/infiniband/hw/qedr/verbs.h b/drivers/infiniband/hw/qedr/verbs.h index 0b7d0124b16c..80ac864dbc78 100644 --- a/drivers/infiniband/hw/qedr/verbs.h +++ b/drivers/infiniband/hw/qedr/verbs.h @@ -76,7 +76,7 @@ int qedr_destroy_srq(struct ib_srq *ibsrq); int qedr_post_srq_recv(struct ib_srq *ibsrq, const struct ib_recv_wr *wr, const struct ib_recv_wr **bad_recv_wr); struct ib_ah *qedr_create_ah(struct ib_pd *ibpd, struct rdma_ah_attr *attr, - struct ib_udata *udata); + u32 flags, struct ib_udata *udata); int qedr_destroy_ah(struct ib_ah *ibah); int qedr_dereg_mr(struct ib_mr *); diff --git a/drivers/infiniband/hw/qib/qib_verbs.c b/drivers/infiniband/hw/qib/qib_verbs.c index 8914abdd7584..f36cad6da489 100644 --- a/drivers/infiniband/hw/qib/qib_verbs.c +++ b/drivers/infiniband/hw/qib/qib_verbs.c @@ -1362,7 +1362,7 @@ struct ib_ah *qib_create_qp0_ah(struct qib_ibport *ibp, u16 dlid) rcu_read_lock(); qp0 = rcu_dereference(ibp->rvp.qp[0]); if (qp0) - ah = rdma_create_ah(qp0->ibqp.pd, &attr); + ah = rdma_create_ah(qp0->ibqp.pd, &attr, 0); rcu_read_unlock(); return ah; } diff --git a/drivers/infiniband/hw/usnic/usnic_ib_verbs.c b/drivers/infiniband/hw/usnic/usnic_ib_verbs.c index 0b91ff36768a..fa477140f548 100644 --- a/drivers/infiniband/hw/usnic/usnic_ib_verbs.c +++ b/drivers/infiniband/hw/usnic/usnic_ib_verbs.c @@ -760,6 +760,7 @@ int usnic_ib_mmap(struct ib_ucontext *context, /* In ib callbacks section - Start of stub funcs */ struct ib_ah *usnic_ib_create_ah(struct ib_pd *pd, struct rdma_ah_attr *ah_attr, + u32 flags, struct ib_udata *udata) { diff --git a/drivers/infiniband/hw/usnic/usnic_ib_verbs.h b/drivers/infiniband/hw/usnic/usnic_ib_verbs.h index 2a2c9beb715f..066f53636d0b 100644 --- a/drivers/infiniband/hw/usnic/usnic_ib_verbs.h +++ b/drivers/infiniband/hw/usnic/usnic_ib_verbs.h @@ -77,6 +77,7 @@ int usnic_ib_mmap(struct ib_ucontext *context, struct vm_area_struct *vma); struct ib_ah *usnic_ib_create_ah(struct ib_pd *pd, struct rdma_ah_attr *ah_attr, + u32 flags, struct ib_udata *udata); int usnic_ib_destroy_ah(struct ib_ah *ah); diff --git a/drivers/infiniband/hw/vmw_pvrdma/pvrdma_verbs.c b/drivers/infiniband/hw/vmw_pvrdma/pvrdma_verbs.c index b65d10b0a875..f79c010c483d 100644 --- a/drivers/infiniband/hw/vmw_pvrdma/pvrdma_verbs.c +++ b/drivers/infiniband/hw/vmw_pvrdma/pvrdma_verbs.c @@ -533,11 +533,12 @@ int pvrdma_dealloc_pd(struct ib_pd *pd) * @pd: the protection domain * @ah_attr: the attributes of the AH * @udata: user data blob + * @flags: create address handle flags (see enum rdma_create_ah_flags) * * @return: the ib_ah pointer on success, otherwise errno. */ struct ib_ah *pvrdma_create_ah(struct ib_pd *pd, struct rdma_ah_attr *ah_attr, - struct ib_udata *udata) + u32 flags, struct ib_udata *udata) { struct pvrdma_dev *dev = to_vdev(pd->device); struct pvrdma_ah *ah; diff --git a/drivers/infiniband/hw/vmw_pvrdma/pvrdma_verbs.h b/drivers/infiniband/hw/vmw_pvrdma/pvrdma_verbs.h index b2e3ab50cb08..d872e6aa159c 100644 --- a/drivers/infiniband/hw/vmw_pvrdma/pvrdma_verbs.h +++ b/drivers/infiniband/hw/vmw_pvrdma/pvrdma_verbs.h @@ -420,7 +420,7 @@ int pvrdma_destroy_cq(struct ib_cq *cq); int pvrdma_poll_cq(struct ib_cq *ibcq, int num_entries, struct ib_wc *wc); int pvrdma_req_notify_cq(struct ib_cq *cq, enum ib_cq_notify_flags flags); struct ib_ah *pvrdma_create_ah(struct ib_pd *pd, struct rdma_ah_attr *ah_attr, - struct ib_udata *udata); + u32 flags, struct ib_udata *udata); int pvrdma_destroy_ah(struct ib_ah *ah); struct ib_srq *pvrdma_create_srq(struct ib_pd *pd, diff --git a/drivers/infiniband/sw/rdmavt/ah.c b/drivers/infiniband/sw/rdmavt/ah.c index 084bb4baebb5..43e46877d300 100644 --- a/drivers/infiniband/sw/rdmavt/ah.c +++ b/drivers/infiniband/sw/rdmavt/ah.c @@ -91,6 +91,7 @@ EXPORT_SYMBOL(rvt_check_ah); * rvt_create_ah - create an address handle * @pd: the protection domain * @ah_attr: the attributes of the AH + * @create_flags: create address handle flags (see enum rdma_create_ah_flags) * @udata: pointer to user's input output buffer information. * * This may be called from interrupt context. @@ -99,6 +100,7 @@ EXPORT_SYMBOL(rvt_check_ah); */ struct ib_ah *rvt_create_ah(struct ib_pd *pd, struct rdma_ah_attr *ah_attr, + u32 create_flags, struct ib_udata *udata) { struct rvt_ah *ah; diff --git a/drivers/infiniband/sw/rdmavt/ah.h b/drivers/infiniband/sw/rdmavt/ah.h index 25271b48a683..70f8d089f740 100644 --- a/drivers/infiniband/sw/rdmavt/ah.h +++ b/drivers/infiniband/sw/rdmavt/ah.h @@ -52,6 +52,7 @@ struct ib_ah *rvt_create_ah(struct ib_pd *pd, struct rdma_ah_attr *ah_attr, + u32 create_flags, struct ib_udata *udata); int rvt_destroy_ah(struct ib_ah *ibah); int rvt_modify_ah(struct ib_ah *ibah, struct rdma_ah_attr *ah_attr); diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.c b/drivers/infiniband/sw/rxe/rxe_verbs.c index 30817c79ba96..5801556f3731 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.c +++ b/drivers/infiniband/sw/rxe/rxe_verbs.c @@ -239,6 +239,7 @@ static void rxe_init_av(struct rxe_dev *rxe, struct rdma_ah_attr *attr, static struct ib_ah *rxe_create_ah(struct ib_pd *ibpd, struct rdma_ah_attr *attr, + u32 flags, struct ib_udata *udata) { diff --git a/drivers/infiniband/ulp/ipoib/ipoib_ib.c b/drivers/infiniband/ulp/ipoib/ipoib_ib.c index 9006a13af1de..1ecf21006f72 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_ib.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_ib.c @@ -66,7 +66,7 @@ struct ipoib_ah *ipoib_create_ah(struct net_device *dev, ah->last_send = 0; kref_init(&ah->ref); - vah = rdma_create_ah(pd, attr); + vah = rdma_create_ah(pd, attr, RDMA_CREATE_AH_SLEEPABLE); if (IS_ERR(vah)) { kfree(ah); ah = (struct ipoib_ah *)vah; diff --git a/drivers/infiniband/ulp/opa_vnic/opa_vnic_vema.c b/drivers/infiniband/ulp/opa_vnic/opa_vnic_vema.c index d119d9afa845..2f4f63847c10 100644 --- a/drivers/infiniband/ulp/opa_vnic/opa_vnic_vema.c +++ b/drivers/infiniband/ulp/opa_vnic/opa_vnic_vema.c @@ -777,7 +777,7 @@ void opa_vnic_vema_send_trap(struct opa_vnic_adapter *adapter, } rdma_ah_set_dlid(&ah_attr, trap_lid); - ah = rdma_create_ah(port->mad_agent->qp->pd, &ah_attr); + ah = rdma_create_ah(port->mad_agent->qp->pd, &ah_attr, 0); if (IS_ERR(ah)) { c_err("%s:Couldn't create new AH = %p\n", __func__, ah); c_err("%s:dlid = %d, sl = %d, port = %d\n", __func__, diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index 85021451eee0..73dcadc2c49c 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -2374,6 +2374,7 @@ struct ib_device { int (*dealloc_pd)(struct ib_pd *pd); struct ib_ah * (*create_ah)(struct ib_pd *pd, struct rdma_ah_attr *ah_attr, + u32 flags, struct ib_udata *udata); int (*modify_ah)(struct ib_ah *ah, struct rdma_ah_attr *ah_attr); @@ -3176,15 +3177,22 @@ struct ib_pd *__ib_alloc_pd(struct ib_device *device, unsigned int flags, __ib_alloc_pd((device), (flags), KBUILD_MODNAME) void ib_dealloc_pd(struct ib_pd *pd); +enum rdma_create_ah_flags { + /* In a sleepable context */ + RDMA_CREATE_AH_SLEEPABLE = BIT(0), +}; + /** * rdma_create_ah - Creates an address handle for the given address vector. * @pd: The protection domain associated with the address handle. * @ah_attr: The attributes of the address vector. + * @flags: Create address handle flags (see enum rdma_create_ah_flags). * * The address handle is used to reference a local or global destination * in all UD QP post sends. */ -struct ib_ah *rdma_create_ah(struct ib_pd *pd, struct rdma_ah_attr *ah_attr); +struct ib_ah *rdma_create_ah(struct ib_pd *pd, struct rdma_ah_attr *ah_attr, + u32 flags); /** * rdma_create_user_ah - Creates an address handle for the given address vector.
Introduce a 'flags' field to create address handle callback and add a flag that marks whether the callback is executed in an atomic context or not. This will allow drivers to wait for completion instead of polling for it when it is allowed. Signed-off-by: Gal Pressman <galpress@amazon.com> --- drivers/infiniband/core/cm.c | 2 +- drivers/infiniband/core/sa_query.c | 3 ++- drivers/infiniband/core/verbs.c | 15 ++++++++++----- drivers/infiniband/hw/bnxt_re/ib_verbs.c | 1 + drivers/infiniband/hw/bnxt_re/ib_verbs.h | 1 + drivers/infiniband/hw/hfi1/mad.c | 2 +- drivers/infiniband/hw/hns/hns_roce_ah.c | 1 + drivers/infiniband/hw/hns/hns_roce_device.h | 1 + drivers/infiniband/hw/mlx4/ah.c | 4 ++-- drivers/infiniband/hw/mlx4/mad.c | 4 ++-- drivers/infiniband/hw/mlx4/mlx4_ib.h | 2 +- drivers/infiniband/hw/mlx5/ah.c | 2 +- drivers/infiniband/hw/mlx5/mlx5_ib.h | 2 +- drivers/infiniband/hw/mthca/mthca_mad.c | 2 +- drivers/infiniband/hw/mthca/mthca_provider.c | 1 + drivers/infiniband/hw/ocrdma/ocrdma_ah.c | 2 +- drivers/infiniband/hw/ocrdma/ocrdma_ah.h | 2 +- drivers/infiniband/hw/qedr/verbs.c | 2 +- drivers/infiniband/hw/qedr/verbs.h | 2 +- drivers/infiniband/hw/qib/qib_verbs.c | 2 +- drivers/infiniband/hw/usnic/usnic_ib_verbs.c | 1 + drivers/infiniband/hw/usnic/usnic_ib_verbs.h | 1 + drivers/infiniband/hw/vmw_pvrdma/pvrdma_verbs.c | 3 ++- drivers/infiniband/hw/vmw_pvrdma/pvrdma_verbs.h | 2 +- drivers/infiniband/sw/rdmavt/ah.c | 2 ++ drivers/infiniband/sw/rdmavt/ah.h | 1 + drivers/infiniband/sw/rxe/rxe_verbs.c | 1 + drivers/infiniband/ulp/ipoib/ipoib_ib.c | 2 +- drivers/infiniband/ulp/opa_vnic/opa_vnic_vema.c | 2 +- include/rdma/ib_verbs.h | 10 +++++++++- 30 files changed, 52 insertions(+), 26 deletions(-)