| Message ID | 1592379956-7043-11-git-send-email-yishaih@mellanox.com (mailing list archive) |
|---|---|
| State | Not Applicable |
| Headers | show |
| Series | verbs: Introduce import verbs for device, PD, MR | expand |
On Wed, Jun 17, 2020 at 10:45:53AM +0300, Yishai Hadas wrote: > Implement the import/unimport MR verbs based on their definition as > described in the man page. > > It uses the query MR KABI to retrieve the original MR properties based > on its given handle. > > Signed-off-by: Yishai Hadas <yishaih@mellanox.com> > libibverbs/cmd_mr.c | 35 +++++++++++++++++++++++++++++++++++ > libibverbs/driver.h | 3 +++ > libibverbs/libibverbs.map.in | 1 + > providers/mlx5/mlx5.c | 2 ++ > providers/mlx5/mlx5.h | 3 +++ > providers/mlx5/verbs.c | 24 ++++++++++++++++++++++++ > 6 files changed, 68 insertions(+) > > diff --git a/libibverbs/cmd_mr.c b/libibverbs/cmd_mr.c > index cb729b6..6984948 100644 > +++ b/libibverbs/cmd_mr.c > @@ -85,3 +85,38 @@ int ibv_cmd_dereg_mr(struct verbs_mr *vmr) > return ret; > return 0; > } > + > +int ibv_cmd_query_mr(struct ibv_pd *pd, struct verbs_mr *vmr, > + uint32_t mr_handle) > +{ > + DECLARE_FBCMD_BUFFER(cmd, UVERBS_OBJECT_MR, > + UVERBS_METHOD_QUERY_MR, > + 5, NULL); > + struct ibv_mr *mr = &vmr->ibv_mr; > + uint64_t iova; > + int ret; > + > + fill_attr_in_obj(cmd, UVERBS_ATTR_QUERY_MR_HANDLE, mr_handle); > + fill_attr_out_ptr(cmd, UVERBS_ATTR_QUERY_MR_RESP_LKEY, > + &mr->lkey); > + fill_attr_out_ptr(cmd, UVERBS_ATTR_QUERY_MR_RESP_RKEY, > + &mr->rkey); > + fill_attr_out_ptr(cmd, UVERBS_ATTR_QUERY_MR_RESP_LENGTH, > + &mr->length); > + /* The iova may be used down the road, let's have it ready from kernel */ > + fill_attr_out_ptr(cmd, UVERBS_ATTR_QUERY_MR_RESP_IOVA, > + &iova); There isn't much reason to fill the attribute here.. Jason
On 6/19/2020 3:50 PM, Jason Gunthorpe wrote: > On Wed, Jun 17, 2020 at 10:45:53AM +0300, Yishai Hadas wrote: >> Implement the import/unimport MR verbs based on their definition as >> described in the man page. >> >> It uses the query MR KABI to retrieve the original MR properties based >> on its given handle. >> >> Signed-off-by: Yishai Hadas <yishaih@mellanox.com> >> libibverbs/cmd_mr.c | 35 +++++++++++++++++++++++++++++++++++ >> libibverbs/driver.h | 3 +++ >> libibverbs/libibverbs.map.in | 1 + >> providers/mlx5/mlx5.c | 2 ++ >> providers/mlx5/mlx5.h | 3 +++ >> providers/mlx5/verbs.c | 24 ++++++++++++++++++++++++ >> 6 files changed, 68 insertions(+) >> >> diff --git a/libibverbs/cmd_mr.c b/libibverbs/cmd_mr.c >> index cb729b6..6984948 100644 >> +++ b/libibverbs/cmd_mr.c >> @@ -85,3 +85,38 @@ int ibv_cmd_dereg_mr(struct verbs_mr *vmr) >> return ret; >> return 0; >> } >> + >> +int ibv_cmd_query_mr(struct ibv_pd *pd, struct verbs_mr *vmr, >> + uint32_t mr_handle) >> +{ >> + DECLARE_FBCMD_BUFFER(cmd, UVERBS_OBJECT_MR, >> + UVERBS_METHOD_QUERY_MR, >> + 5, NULL); >> + struct ibv_mr *mr = &vmr->ibv_mr; >> + uint64_t iova; >> + int ret; >> + >> + fill_attr_in_obj(cmd, UVERBS_ATTR_QUERY_MR_HANDLE, mr_handle); >> + fill_attr_out_ptr(cmd, UVERBS_ATTR_QUERY_MR_RESP_LKEY, >> + &mr->lkey); >> + fill_attr_out_ptr(cmd, UVERBS_ATTR_QUERY_MR_RESP_RKEY, >> + &mr->rkey); >> + fill_attr_out_ptr(cmd, UVERBS_ATTR_QUERY_MR_RESP_LENGTH, >> + &mr->length); >> + /* The iova may be used down the road, let's have it ready from kernel */ >> + fill_attr_out_ptr(cmd, UVERBS_ATTR_QUERY_MR_RESP_IOVA, >> + &iova); > > There isn't much reason to fill the attribute here.. > We have defined this attribute from kernel perspective to be mandatory from day one as of other attributes above. Are you suggesting to change in kernel to let this attribute be optional and not fill it here ? other alternative ? Yishai
On Sun, Jun 21, 2020 at 11:44:52AM +0300, Yishai Hadas wrote: > On 6/19/2020 3:50 PM, Jason Gunthorpe wrote: > > On Wed, Jun 17, 2020 at 10:45:53AM +0300, Yishai Hadas wrote: > > > Implement the import/unimport MR verbs based on their definition as > > > described in the man page. > > > > > > It uses the query MR KABI to retrieve the original MR properties based > > > on its given handle. > > > > > > Signed-off-by: Yishai Hadas <yishaih@mellanox.com> > > > libibverbs/cmd_mr.c | 35 +++++++++++++++++++++++++++++++++++ > > > libibverbs/driver.h | 3 +++ > > > libibverbs/libibverbs.map.in | 1 + > > > providers/mlx5/mlx5.c | 2 ++ > > > providers/mlx5/mlx5.h | 3 +++ > > > providers/mlx5/verbs.c | 24 ++++++++++++++++++++++++ > > > 6 files changed, 68 insertions(+) > > > > > > diff --git a/libibverbs/cmd_mr.c b/libibverbs/cmd_mr.c > > > index cb729b6..6984948 100644 > > > +++ b/libibverbs/cmd_mr.c > > > @@ -85,3 +85,38 @@ int ibv_cmd_dereg_mr(struct verbs_mr *vmr) > > > return ret; > > > return 0; > > > } > > > + > > > +int ibv_cmd_query_mr(struct ibv_pd *pd, struct verbs_mr *vmr, > > > + uint32_t mr_handle) > > > +{ > > > + DECLARE_FBCMD_BUFFER(cmd, UVERBS_OBJECT_MR, > > > + UVERBS_METHOD_QUERY_MR, > > > + 5, NULL); > > > + struct ibv_mr *mr = &vmr->ibv_mr; > > > + uint64_t iova; > > > + int ret; > > > + > > > + fill_attr_in_obj(cmd, UVERBS_ATTR_QUERY_MR_HANDLE, mr_handle); > > > + fill_attr_out_ptr(cmd, UVERBS_ATTR_QUERY_MR_RESP_LKEY, > > > + &mr->lkey); > > > + fill_attr_out_ptr(cmd, UVERBS_ATTR_QUERY_MR_RESP_RKEY, > > > + &mr->rkey); > > > + fill_attr_out_ptr(cmd, UVERBS_ATTR_QUERY_MR_RESP_LENGTH, > > > + &mr->length); > > > + /* The iova may be used down the road, let's have it ready from kernel */ > > > + fill_attr_out_ptr(cmd, UVERBS_ATTR_QUERY_MR_RESP_IOVA, > > > + &iova); > > > > There isn't much reason to fill the attribute here.. > > > > We have defined this attribute from kernel perspective to be mandatory from > day one as of other attributes above. > Are you suggesting to change in kernel to let this attribute be optional and > not fill it here ? other alternative ? I'm not sure output attributes should be marked as mandatory? Jason
On 6/23/2020 8:33 PM, Jason Gunthorpe wrote: > On Sun, Jun 21, 2020 at 11:44:52AM +0300, Yishai Hadas wrote: >> On 6/19/2020 3:50 PM, Jason Gunthorpe wrote: >>> On Wed, Jun 17, 2020 at 10:45:53AM +0300, Yishai Hadas wrote: >>>> Implement the import/unimport MR verbs based on their definition as >>>> described in the man page. >>>> >>>> It uses the query MR KABI to retrieve the original MR properties based >>>> on its given handle. >>>> >>>> Signed-off-by: Yishai Hadas <yishaih@mellanox.com> >>>> libibverbs/cmd_mr.c | 35 +++++++++++++++++++++++++++++++++++ >>>> libibverbs/driver.h | 3 +++ >>>> libibverbs/libibverbs.map.in | 1 + >>>> providers/mlx5/mlx5.c | 2 ++ >>>> providers/mlx5/mlx5.h | 3 +++ >>>> providers/mlx5/verbs.c | 24 ++++++++++++++++++++++++ >>>> 6 files changed, 68 insertions(+) >>>> >>>> diff --git a/libibverbs/cmd_mr.c b/libibverbs/cmd_mr.c >>>> index cb729b6..6984948 100644 >>>> +++ b/libibverbs/cmd_mr.c >>>> @@ -85,3 +85,38 @@ int ibv_cmd_dereg_mr(struct verbs_mr *vmr) >>>> return ret; >>>> return 0; >>>> } >>>> + >>>> +int ibv_cmd_query_mr(struct ibv_pd *pd, struct verbs_mr *vmr, >>>> + uint32_t mr_handle) >>>> +{ >>>> + DECLARE_FBCMD_BUFFER(cmd, UVERBS_OBJECT_MR, >>>> + UVERBS_METHOD_QUERY_MR, >>>> + 5, NULL); >>>> + struct ibv_mr *mr = &vmr->ibv_mr; >>>> + uint64_t iova; >>>> + int ret; >>>> + >>>> + fill_attr_in_obj(cmd, UVERBS_ATTR_QUERY_MR_HANDLE, mr_handle); >>>> + fill_attr_out_ptr(cmd, UVERBS_ATTR_QUERY_MR_RESP_LKEY, >>>> + &mr->lkey); >>>> + fill_attr_out_ptr(cmd, UVERBS_ATTR_QUERY_MR_RESP_RKEY, >>>> + &mr->rkey); >>>> + fill_attr_out_ptr(cmd, UVERBS_ATTR_QUERY_MR_RESP_LENGTH, >>>> + &mr->length); >>>> + /* The iova may be used down the road, let's have it ready from kernel */ >>>> + fill_attr_out_ptr(cmd, UVERBS_ATTR_QUERY_MR_RESP_IOVA, >>>> + &iova); >>> >>> There isn't much reason to fill the attribute here.. >>> >> >> We have defined this attribute from kernel perspective to be mandatory from >> day one as of other attributes above. >> Are you suggesting to change in kernel to let this attribute be optional and >> not fill it here ? other alternative ? > > I'm not sure output attributes should be marked as mandatory? > Looking in kernel side, the semantics in most of the cases including for other MR method [1] which returns lkey & rkey as of this method is to define the output attributes as mandatory. When used, it may point that from API point of view it makes no-sense to not return those attributes back to user as them are really mandatory. (see also other examples, as of port_num for UVERBS_METHOD_QUERY_PORT, QP, SRQ, CQ output fields, DEVX usage, etc.). However, In case an attribute is not a spec mandatory (e.g. SRQN for non XRC case) or that application can work in some way without having it (e.g. 'core_support' for UVERBS_METHOD_GET_CONTEXT) it's defined an optional. As this IOVA attribute is currently not really in use, I believe that we can set it in kernel as an optional while leaving other attributes for this API (e.g. rkey, lkey, etc.) to be mandatory and cleanup here its setting. Makes sense ? if yes, I'll go by that approach and send V1 in kernel side for that change. [1] https://elixir.bootlin.com/linux/v5.3-rc7/source/drivers/infiniband/core/uverbs_std_types_mr.c#L196 Yishai
On 6/23/2020 8:33 PM, Jason Gunthorpe wrote: > On Sun, Jun 21, 2020 at 11:44:52AM +0300, Yishai Hadas wrote: >> On 6/19/2020 3:50 PM, Jason Gunthorpe wrote: >>> On Wed, Jun 17, 2020 at 10:45:53AM +0300, Yishai Hadas wrote: >>>> Implement the import/unimport MR verbs based on their definition as >>>> described in the man page. >>>> >>>> It uses the query MR KABI to retrieve the original MR properties based >>>> on its given handle. >>>> >>>> Signed-off-by: Yishai Hadas <yishaih@mellanox.com> >>>> libibverbs/cmd_mr.c | 35 +++++++++++++++++++++++++++++++++++ >>>> libibverbs/driver.h | 3 +++ >>>> libibverbs/libibverbs.map.in | 1 + >>>> providers/mlx5/mlx5.c | 2 ++ >>>> providers/mlx5/mlx5.h | 3 +++ >>>> providers/mlx5/verbs.c | 24 ++++++++++++++++++++++++ >>>> 6 files changed, 68 insertions(+) >>>> >>>> diff --git a/libibverbs/cmd_mr.c b/libibverbs/cmd_mr.c >>>> index cb729b6..6984948 100644 >>>> +++ b/libibverbs/cmd_mr.c >>>> @@ -85,3 +85,38 @@ int ibv_cmd_dereg_mr(struct verbs_mr *vmr) >>>> return ret; >>>> return 0; >>>> } >>>> + >>>> +int ibv_cmd_query_mr(struct ibv_pd *pd, struct verbs_mr *vmr, >>>> + uint32_t mr_handle) >>>> +{ >>>> + DECLARE_FBCMD_BUFFER(cmd, UVERBS_OBJECT_MR, >>>> + UVERBS_METHOD_QUERY_MR, >>>> + 5, NULL); >>>> + struct ibv_mr *mr = &vmr->ibv_mr; >>>> + uint64_t iova; >>>> + int ret; >>>> + >>>> + fill_attr_in_obj(cmd, UVERBS_ATTR_QUERY_MR_HANDLE, mr_handle); >>>> + fill_attr_out_ptr(cmd, UVERBS_ATTR_QUERY_MR_RESP_LKEY, >>>> + &mr->lkey); >>>> + fill_attr_out_ptr(cmd, UVERBS_ATTR_QUERY_MR_RESP_RKEY, >>>> + &mr->rkey); >>>> + fill_attr_out_ptr(cmd, UVERBS_ATTR_QUERY_MR_RESP_LENGTH, >>>> + &mr->length); >>>> + /* The iova may be used down the road, let's have it ready from kernel */ >>>> + fill_attr_out_ptr(cmd, UVERBS_ATTR_QUERY_MR_RESP_IOVA, >>>> + &iova); >>> >>> There isn't much reason to fill the attribute here.. >>> >> >> We have defined this attribute from kernel perspective to be mandatory from >> day one as of other attributes above. >> Are you suggesting to change in kernel to let this attribute be optional and >> not fill it here ? other alternative ? > > I'm not sure output attributes should be marked as mandatory? > OK, this attribute was changed to be optional in V1 kernel series, the PR was updated to not fill this attribute at all, thanks. Yishai
On Wed, Jul 01, 2020 at 03:28:24PM +0300, Yishai Hadas wrote: > On 6/23/2020 8:33 PM, Jason Gunthorpe wrote: > > On Sun, Jun 21, 2020 at 11:44:52AM +0300, Yishai Hadas wrote: > > > On 6/19/2020 3:50 PM, Jason Gunthorpe wrote: > > > > On Wed, Jun 17, 2020 at 10:45:53AM +0300, Yishai Hadas wrote: > > > > > Implement the import/unimport MR verbs based on their definition as > > > > > described in the man page. > > > > > > > > > > It uses the query MR KABI to retrieve the original MR properties based > > > > > on its given handle. > > > > > > > > > > Signed-off-by: Yishai Hadas <yishaih@mellanox.com> > > > > > libibverbs/cmd_mr.c | 35 +++++++++++++++++++++++++++++++++++ > > > > > libibverbs/driver.h | 3 +++ > > > > > libibverbs/libibverbs.map.in | 1 + > > > > > providers/mlx5/mlx5.c | 2 ++ > > > > > providers/mlx5/mlx5.h | 3 +++ > > > > > providers/mlx5/verbs.c | 24 ++++++++++++++++++++++++ > > > > > 6 files changed, 68 insertions(+) > > > > > > > > > > diff --git a/libibverbs/cmd_mr.c b/libibverbs/cmd_mr.c > > > > > index cb729b6..6984948 100644 > > > > > +++ b/libibverbs/cmd_mr.c > > > > > @@ -85,3 +85,38 @@ int ibv_cmd_dereg_mr(struct verbs_mr *vmr) > > > > > return ret; > > > > > return 0; > > > > > } > > > > > + > > > > > +int ibv_cmd_query_mr(struct ibv_pd *pd, struct verbs_mr *vmr, > > > > > + uint32_t mr_handle) > > > > > +{ > > > > > + DECLARE_FBCMD_BUFFER(cmd, UVERBS_OBJECT_MR, > > > > > + UVERBS_METHOD_QUERY_MR, > > > > > + 5, NULL); > > > > > + struct ibv_mr *mr = &vmr->ibv_mr; > > > > > + uint64_t iova; > > > > > + int ret; > > > > > + > > > > > + fill_attr_in_obj(cmd, UVERBS_ATTR_QUERY_MR_HANDLE, mr_handle); > > > > > + fill_attr_out_ptr(cmd, UVERBS_ATTR_QUERY_MR_RESP_LKEY, > > > > > + &mr->lkey); > > > > > + fill_attr_out_ptr(cmd, UVERBS_ATTR_QUERY_MR_RESP_RKEY, > > > > > + &mr->rkey); > > > > > + fill_attr_out_ptr(cmd, UVERBS_ATTR_QUERY_MR_RESP_LENGTH, > > > > > + &mr->length); > > > > > + /* The iova may be used down the road, let's have it ready from kernel */ > > > > > + fill_attr_out_ptr(cmd, UVERBS_ATTR_QUERY_MR_RESP_IOVA, > > > > > + &iova); > > > > > > > > There isn't much reason to fill the attribute here.. > > > > > > > > > > We have defined this attribute from kernel perspective to be mandatory from > > > day one as of other attributes above. > > > Are you suggesting to change in kernel to let this attribute be optional and > > > not fill it here ? other alternative ? > > > > I'm not sure output attributes should be marked as mandatory? > > > > OK, this attribute was changed to be optional in V1 kernel series, the PR > was updated to not fill this attribute at all, thanks. The kernel series looks OK, someone will send the v1? Jason
diff --git a/libibverbs/cmd_mr.c b/libibverbs/cmd_mr.c index cb729b6..6984948 100644 --- a/libibverbs/cmd_mr.c +++ b/libibverbs/cmd_mr.c @@ -85,3 +85,38 @@ int ibv_cmd_dereg_mr(struct verbs_mr *vmr) return ret; return 0; } + +int ibv_cmd_query_mr(struct ibv_pd *pd, struct verbs_mr *vmr, + uint32_t mr_handle) +{ + DECLARE_FBCMD_BUFFER(cmd, UVERBS_OBJECT_MR, + UVERBS_METHOD_QUERY_MR, + 5, NULL); + struct ibv_mr *mr = &vmr->ibv_mr; + uint64_t iova; + int ret; + + fill_attr_in_obj(cmd, UVERBS_ATTR_QUERY_MR_HANDLE, mr_handle); + fill_attr_out_ptr(cmd, UVERBS_ATTR_QUERY_MR_RESP_LKEY, + &mr->lkey); + fill_attr_out_ptr(cmd, UVERBS_ATTR_QUERY_MR_RESP_RKEY, + &mr->rkey); + fill_attr_out_ptr(cmd, UVERBS_ATTR_QUERY_MR_RESP_LENGTH, + &mr->length); + /* The iova may be used down the road, let's have it ready from kernel */ + fill_attr_out_ptr(cmd, UVERBS_ATTR_QUERY_MR_RESP_IOVA, + &iova); + + ret = execute_ioctl(pd->context, cmd); + if (ret) + return ret; + + mr->handle = mr_handle; + mr->context = pd->context; + mr->pd = pd; + mr->addr = NULL; + + vmr->mr_type = IBV_MR_TYPE_IMPORTED_MR; + return 0; +} + diff --git a/libibverbs/driver.h b/libibverbs/driver.h index 0edff0b..094891d 100644 --- a/libibverbs/driver.h +++ b/libibverbs/driver.h @@ -80,6 +80,7 @@ enum ibv_gid_type { enum ibv_mr_type { IBV_MR_TYPE_MR, IBV_MR_TYPE_NULL_MR, + IBV_MR_TYPE_IMPORTED_MR, }; struct verbs_mr { @@ -476,6 +477,8 @@ int ibv_cmd_rereg_mr(struct verbs_mr *vmr, uint32_t flags, void *addr, size_t cmd_sz, struct ib_uverbs_rereg_mr_resp *resp, size_t resp_sz); int ibv_cmd_dereg_mr(struct verbs_mr *vmr); +int ibv_cmd_query_mr(struct ibv_pd *pd, struct verbs_mr *vmr, + uint32_t mr_handle); int ibv_cmd_advise_mr(struct ibv_pd *pd, enum ibv_advise_mr_advice advice, uint32_t flags, diff --git a/libibverbs/libibverbs.map.in b/libibverbs/libibverbs.map.in index 5f52a9e..0d4f820 100644 --- a/libibverbs/libibverbs.map.in +++ b/libibverbs/libibverbs.map.in @@ -197,6 +197,7 @@ IBVERBS_PRIVATE_@IBVERBS_PABI_VERSION@ { ibv_cmd_query_context; ibv_cmd_query_device; ibv_cmd_query_device_ex; + ibv_cmd_query_mr; ibv_cmd_query_port; ibv_cmd_query_qp; ibv_cmd_query_srq; diff --git a/providers/mlx5/mlx5.c b/providers/mlx5/mlx5.c index 4067204..2413387 100644 --- a/providers/mlx5/mlx5.c +++ b/providers/mlx5/mlx5.c @@ -148,6 +148,7 @@ static const struct verbs_context_ops mlx5_ctx_common_ops = { .destroy_wq = mlx5_destroy_wq, .free_dm = mlx5_free_dm, .get_srq_num = mlx5_get_srq_num, + .import_mr = mlx5_import_mr, .import_pd = mlx5_import_pd, .modify_cq = mlx5_modify_cq, .modify_flow_action_esp = mlx5_modify_flow_action_esp, @@ -160,6 +161,7 @@ static const struct verbs_context_ops mlx5_ctx_common_ops = { .query_rt_values = mlx5_query_rt_values, .read_counters = mlx5_read_counters, .reg_dm_mr = mlx5_reg_dm_mr, + .unimport_mr = mlx5_unimport_mr, .unimport_pd = mlx5_unimport_pd, .alloc_null_mr = mlx5_alloc_null_mr, .free_context = mlx5_free_context, diff --git a/providers/mlx5/mlx5.h b/providers/mlx5/mlx5.h index f2344c5..654c164 100644 --- a/providers/mlx5/mlx5.h +++ b/providers/mlx5/mlx5.h @@ -1023,6 +1023,9 @@ int mlx5_advise_mr(struct ibv_pd *pd, uint32_t flags, struct ibv_sge *sg_list, uint32_t num_sges); +struct ibv_mr *mlx5_import_mr(struct ibv_pd *pd, + uint32_t mr_handle); +void mlx5_unimport_mr(struct ibv_mr *mr); struct ibv_pd *mlx5_import_pd(struct ibv_context *context, uint32_t pd_handle); void mlx5_unimport_pd(struct ibv_pd *pd); diff --git a/providers/mlx5/verbs.c b/providers/mlx5/verbs.c index b2c54ba..972b783 100644 --- a/providers/mlx5/verbs.c +++ b/providers/mlx5/verbs.c @@ -756,6 +756,30 @@ void mlx5_unimport_pd(struct ibv_pd *pd) assert(false); } +struct ibv_mr *mlx5_import_mr(struct ibv_pd *pd, + uint32_t mr_handle) +{ + struct mlx5_mr *mr; + int ret; + + mr = calloc(1, sizeof *mr); + if (!mr) + return NULL; + + ret = ibv_cmd_query_mr(pd, &mr->vmr, mr_handle); + if (ret) { + free(mr); + return NULL; + } + + return &mr->vmr.ibv_mr; +} + +void mlx5_unimport_mr(struct ibv_mr *ibmr) +{ + free(to_mmr(ibmr)); +} + struct ibv_mw *mlx5_alloc_mw(struct ibv_pd *pd, enum ibv_mw_type type) { struct ibv_mw *mw;
Implement the import/unimport MR verbs based on their definition as described in the man page. It uses the query MR KABI to retrieve the original MR properties based on its given handle. Signed-off-by: Yishai Hadas <yishaih@mellanox.com> --- libibverbs/cmd_mr.c | 35 +++++++++++++++++++++++++++++++++++ libibverbs/driver.h | 3 +++ libibverbs/libibverbs.map.in | 1 + providers/mlx5/mlx5.c | 2 ++ providers/mlx5/mlx5.h | 3 +++ providers/mlx5/verbs.c | 24 ++++++++++++++++++++++++ 6 files changed, 68 insertions(+)