From patchwork Wed Oct 10 16:56:28 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yishai Hadas X-Patchwork-Id: 10634897 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3E4BA69B5 for ; Wed, 10 Oct 2018 17:14:04 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F390429FC4 for ; Wed, 10 Oct 2018 17:14:02 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E7D2C29F92; Wed, 10 Oct 2018 17:14:02 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C25C329FC4 for ; Wed, 10 Oct 2018 17:14:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726693AbeJKAhF (ORCPT ); Wed, 10 Oct 2018 20:37:05 -0400 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:59781 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726562AbeJKAhF (ORCPT ); Wed, 10 Oct 2018 20:37:05 -0400 Received: from Internal Mail-Server by MTLPINE1 (envelope-from yishaih@mellanox.com) with ESMTPS (AES256-SHA encrypted); 10 Oct 2018 19:01:36 +0200 Received: from vnc17.mtl.labs.mlnx (vnc17.mtl.labs.mlnx [10.7.2.17]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id w9AGukU0005877; Wed, 10 Oct 2018 19:56:46 +0300 Received: from vnc17.mtl.labs.mlnx (vnc17.mtl.labs.mlnx [127.0.0.1]) by vnc17.mtl.labs.mlnx (8.13.8/8.13.8) with ESMTP id w9AGuknZ031298; Wed, 10 Oct 2018 19:56:46 +0300 Received: (from yishaih@localhost) by vnc17.mtl.labs.mlnx (8.13.8/8.13.8/Submit) id w9AGukZ9031297; Wed, 10 Oct 2018 19:56:46 +0300 From: Yishai Hadas To: linux-rdma@vger.kernel.org Cc: yishaih@mellanox.com, Alexr@mellanox.com, jgg@mellanox.com, majd@mellanox.com Subject: [PATCH rdma-core 1/3] mlx5: Introduce DEVX object and its DV APIs Date: Wed, 10 Oct 2018 19:56:28 +0300 Message-Id: <1539190590-31186-2-git-send-email-yishaih@mellanox.com> X-Mailer: git-send-email 1.8.2.3 In-Reply-To: <1539190590-31186-1-git-send-email-yishaih@mellanox.com> References: <1539190590-31186-1-git-send-email-yishaih@mellanox.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Introduce DEVX object and its DV APIs as of: create/modify/read/destroy. The DEVX general command API was added as well to enable reading CAPs directly from the firmware. Signed-off-by: Yishai Hadas --- debian/ibverbs-providers.symbols | 5 ++ providers/mlx5/libmlx5.map | 5 ++ providers/mlx5/man/CMakeLists.txt | 7 ++ providers/mlx5/man/mlx5dv_devx_obj_create.3.md | 74 ++++++++++++++++++++ providers/mlx5/mlx5.h | 5 ++ providers/mlx5/mlx5dv.h | 12 ++++ providers/mlx5/verbs.c | 95 ++++++++++++++++++++++++++ 7 files changed, 203 insertions(+) create mode 100644 providers/mlx5/man/mlx5dv_devx_obj_create.3.md diff --git a/debian/ibverbs-providers.symbols b/debian/ibverbs-providers.symbols index 23b0bbf..acd1715 100644 --- a/debian/ibverbs-providers.symbols +++ b/debian/ibverbs-providers.symbols @@ -28,3 +28,8 @@ libmlx5.so.1 ibverbs-providers #MINVER# mlx5dv_create_flow_action_modify_header@MLX5_1.7 21 mlx5dv_create_flow_action_packet_reformat@MLX5_1.7 21 mlx5dv_open_device@MLX5_1.7 21 + mlx5dv_devx_general_cmd@MLX5_1.7 21 + mlx5dv_devx_obj_create@MLX5_1.7 21 + mlx5dv_devx_obj_destroy@MLX5_1.7 21 + mlx5dv_devx_obj_query@MLX5_1.7 21 + mlx5dv_devx_obj_modify@MLX5_1.7 21 diff --git a/providers/mlx5/libmlx5.map b/providers/mlx5/libmlx5.map index a178af6..91a7a32 100644 --- a/providers/mlx5/libmlx5.map +++ b/providers/mlx5/libmlx5.map @@ -46,4 +46,9 @@ MLX5_1.7 { mlx5dv_create_flow_action_modify_header; mlx5dv_create_flow_action_packet_reformat; mlx5dv_open_device; + mlx5dv_devx_general_cmd; + mlx5dv_devx_obj_create; + mlx5dv_devx_obj_destroy; + mlx5dv_devx_obj_query; + mlx5dv_devx_obj_modify; } MLX5_1.6; diff --git a/providers/mlx5/man/CMakeLists.txt b/providers/mlx5/man/CMakeLists.txt index e8405a2..8d5d8e2 100644 --- a/providers/mlx5/man/CMakeLists.txt +++ b/providers/mlx5/man/CMakeLists.txt @@ -4,6 +4,7 @@ rdma_man_pages( mlx5dv_create_flow_action_packet_reformat.3.md mlx5dv_create_flow_matcher.3.md mlx5dv_create_qp.3.md + mlx5dv_devx_obj_create.3.md mlx5dv_flow_action_esp.3.md mlx5dv_get_clock_info.3 mlx5dv_init_obj.3 @@ -12,3 +13,9 @@ rdma_man_pages( mlx5dv_ts_to_ns.3 mlx5dv.7 ) +rdma_alias_man_pages( + mlx5dv_devx_obj_create.3 mlx5dv_devx_general_cmd.3 + mlx5dv_devx_obj_create.3 mlx5dv_devx_obj_destroy.3 + mlx5dv_devx_obj_create.3 mlx5dv_devx_obj_query.3 + mlx5dv_devx_obj_create.3 mlx5dv_devx_obj_modify.3 +) diff --git a/providers/mlx5/man/mlx5dv_devx_obj_create.3.md b/providers/mlx5/man/mlx5dv_devx_obj_create.3.md new file mode 100644 index 0000000..320404a --- /dev/null +++ b/providers/mlx5/man/mlx5dv_devx_obj_create.3.md @@ -0,0 +1,74 @@ +--- +layout: page +title: mlx5dv_devx_obj_create / destroy / modify /query / general +section: 3 +tagline: Verbs +--- + +# NAME + +mlx5dv_devx_obj_create - Creates a devx object + +mlx5dv_devx_obj_destroy - Destroys a devx object + +mlx5dv_devx_obj_modify - Modifies a devx object + +mlx5dv_devx_obj_query - Queries a devx object + +mlx5dv_devx_general_cmd - Issues a general command over the devx interface + +# SYNOPSIS + +```c +#include + +struct mlx5dv_devx_obj * +mlx5dv_devx_obj_create(struct ibv_context *context, const void *in, size_t inlen, + void *out, size_t outlen); +int mlx5dv_devx_obj_query(struct mlx5dv_devx_obj *obj, const void *in, size_t inlen, + void *out, size_t outlen); +int mlx5dv_devx_obj_modify(struct mlx5dv_devx_obj *obj, const void *in, size_t inlen, + void *out, size_t outlen); +int mlx5dv_devx_obj_destroy(struct mlx5dv_devx_obj *obj); +int mlx5dv_devx_general_cmd(struct ibv_context *context, const void *in, size_t inlen, + void *out, size_t outlen); +``` + +# DESCRIPTION + +Create / destroy / modify / query a devx object, issue a general command over the devx interface. + +# ARGUMENTS + +*context* +: RDMA device context to create the action on. + +*in* +: A buffer which contains the command's input data provided in a device specification format. + +*inlen* +: The size of *in* buffer in bytes. + +*out* +: A buffer which contains the command's output data according to the device specification format. + +*outlen* +: The size of *out* buffer in bytes. + +*obj* +: For query, modify, destroy: the devx object to work on. + +# RETURN VALUE + +Upon success *mlx5dv_devx_create_obj* will return a new *struct +mlx5dv_devx_obj* on error NULL will be returned and errno will be set. + +Upon success query, modify, destroy, general commands, 0 is returned or the value of errno on a failure. + +# SEE ALSO + +**mlx5dv_open_device** + +AUTHOR + +Yishai Hadas diff --git a/providers/mlx5/mlx5.h b/providers/mlx5/mlx5.h index c5b9e6f..91fbc50 100644 --- a/providers/mlx5/mlx5.h +++ b/providers/mlx5/mlx5.h @@ -567,6 +567,11 @@ struct mlx5dv_flow_matcher { uint32_t handle; }; +struct mlx5dv_devx_obj { + struct ibv_context *context; + uint32_t handle; +}; + static inline int mlx5_ilog2(int n) { int t; diff --git a/providers/mlx5/mlx5dv.h b/providers/mlx5/mlx5dv.h index 28d68bc..662433b 100644 --- a/providers/mlx5/mlx5dv.h +++ b/providers/mlx5/mlx5dv.h @@ -1015,6 +1015,18 @@ struct mlx5dv_context_attr { struct ibv_context * mlx5dv_open_device(struct ibv_device *device, struct mlx5dv_context_attr *attr); +struct mlx5dv_devx_obj; + +struct mlx5dv_devx_obj * +mlx5dv_devx_obj_create(struct ibv_context *context, const void *in, size_t inlen, + void *out, size_t outlen); +int mlx5dv_devx_obj_query(struct mlx5dv_devx_obj *obj, const void *in, size_t inlen, + void *out, size_t outlen); +int mlx5dv_devx_obj_modify(struct mlx5dv_devx_obj *obj, const void *in, size_t inlen, + void *out, size_t outlen); +int mlx5dv_devx_obj_destroy(struct mlx5dv_devx_obj *obj); +int mlx5dv_devx_general_cmd(struct ibv_context *context, const void *in, size_t inlen, + void *out, size_t outlen); #ifdef __cplusplus } #endif diff --git a/providers/mlx5/verbs.c b/providers/mlx5/verbs.c index 9ebd742..f476bc5 100644 --- a/providers/mlx5/verbs.c +++ b/providers/mlx5/verbs.c @@ -3842,3 +3842,98 @@ err: free(mflow); return NULL; } + +struct mlx5dv_devx_obj * +mlx5dv_devx_obj_create(struct ibv_context *context, const void *in, size_t inlen, + void *out, size_t outlen) +{ + DECLARE_COMMAND_BUFFER(cmd, + MLX5_IB_OBJECT_DEVX_OBJ, + MLX5_IB_METHOD_DEVX_OBJ_CREATE, + 3); + struct ib_uverbs_attr *handle; + struct mlx5dv_devx_obj *obj; + int ret; + + obj = calloc(1, sizeof(*obj)); + if (!obj) { + errno = ENOMEM; + return NULL; + } + + handle = fill_attr_out_obj(cmd, MLX5_IB_ATTR_DEVX_OBJ_CREATE_HANDLE); + fill_attr_in(cmd, MLX5_IB_ATTR_DEVX_OBJ_CREATE_CMD_IN, in, inlen); + fill_attr_out(cmd, MLX5_IB_ATTR_DEVX_OBJ_CREATE_CMD_OUT, out, outlen); + + ret = execute_ioctl(context, cmd); + if (ret) + goto err; + + obj->handle = read_attr_obj(MLX5_IB_ATTR_DEVX_OBJ_CREATE_HANDLE, handle); + obj->context = context; + return obj; +err: + free(obj); + return NULL; +} + +int mlx5dv_devx_obj_query(struct mlx5dv_devx_obj *obj, const void *in, size_t inlen, + void *out, size_t outlen) +{ + DECLARE_COMMAND_BUFFER(cmd, + MLX5_IB_OBJECT_DEVX_OBJ, + MLX5_IB_METHOD_DEVX_OBJ_QUERY, + 3); + + fill_attr_in_obj(cmd, MLX5_IB_ATTR_DEVX_OBJ_QUERY_HANDLE, obj->handle); + fill_attr_in(cmd, MLX5_IB_ATTR_DEVX_OBJ_QUERY_CMD_IN, in, inlen); + fill_attr_out(cmd, MLX5_IB_ATTR_DEVX_OBJ_QUERY_CMD_OUT, out, outlen); + + return execute_ioctl(obj->context, cmd); +} + +int mlx5dv_devx_obj_modify(struct mlx5dv_devx_obj *obj, const void *in, size_t inlen, + void *out, size_t outlen) +{ + DECLARE_COMMAND_BUFFER(cmd, + MLX5_IB_OBJECT_DEVX_OBJ, + MLX5_IB_METHOD_DEVX_OBJ_MODIFY, + 3); + + fill_attr_in_obj(cmd, MLX5_IB_ATTR_DEVX_OBJ_MODIFY_HANDLE, obj->handle); + fill_attr_in(cmd, MLX5_IB_ATTR_DEVX_OBJ_MODIFY_CMD_IN, in, inlen); + fill_attr_out(cmd, MLX5_IB_ATTR_DEVX_OBJ_MODIFY_CMD_OUT, out, outlen); + + return execute_ioctl(obj->context, cmd); +} + +int mlx5dv_devx_obj_destroy(struct mlx5dv_devx_obj *obj) +{ + DECLARE_COMMAND_BUFFER(cmd, + MLX5_IB_OBJECT_DEVX_OBJ, + MLX5_IB_METHOD_DEVX_OBJ_DESTROY, + 1); + int ret; + + fill_attr_in_obj(cmd, MLX5_IB_ATTR_DEVX_OBJ_DESTROY_HANDLE, obj->handle); + ret = execute_ioctl(obj->context, cmd); + + if (ret) + return ret; + free(obj); + return 0; +} + +int mlx5dv_devx_general_cmd(struct ibv_context *context, const void *in, size_t inlen, + void *out, size_t outlen) +{ + DECLARE_COMMAND_BUFFER(cmd, + MLX5_IB_OBJECT_DEVX, + MLX5_IB_METHOD_DEVX_OTHER, + 2); + + fill_attr_in(cmd, MLX5_IB_ATTR_DEVX_OTHER_CMD_IN, in, inlen); + fill_attr_out(cmd, MLX5_IB_ATTR_DEVX_OTHER_CMD_OUT, out, outlen); + + return execute_ioctl(context, cmd); +} From patchwork Wed Oct 10 16:56:29 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yishai Hadas X-Patchwork-Id: 10634899 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2A7AB69B4 for ; Wed, 10 Oct 2018 17:14:04 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0498029EC1 for ; Wed, 10 Oct 2018 17:14:04 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id ECFE429FFC; Wed, 10 Oct 2018 17:14:03 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 64C0729EC1 for ; Wed, 10 Oct 2018 17:14:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726649AbeJKAhH (ORCPT ); Wed, 10 Oct 2018 20:37:07 -0400 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:59797 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726515AbeJKAhG (ORCPT ); Wed, 10 Oct 2018 20:37:06 -0400 Received: from Internal Mail-Server by MTLPINE1 (envelope-from yishaih@mellanox.com) with ESMTPS (AES256-SHA encrypted); 10 Oct 2018 19:01:45 +0200 Received: from vnc17.mtl.labs.mlnx (vnc17.mtl.labs.mlnx [10.7.2.17]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id w9AGus9B007069; Wed, 10 Oct 2018 19:56:54 +0300 Received: from vnc17.mtl.labs.mlnx (vnc17.mtl.labs.mlnx [127.0.0.1]) by vnc17.mtl.labs.mlnx (8.13.8/8.13.8) with ESMTP id w9AGukNQ031302; Wed, 10 Oct 2018 19:56:54 +0300 Received: (from yishaih@localhost) by vnc17.mtl.labs.mlnx (8.13.8/8.13.8/Submit) id w9AGuk8x031301; Wed, 10 Oct 2018 19:56:46 +0300 From: Yishai Hadas To: linux-rdma@vger.kernel.org Cc: yishaih@mellanox.com, Alexr@mellanox.com, jgg@mellanox.com, majd@mellanox.com Subject: [PATCH rdma-core 2/3] mlx5: Add DV DEVX APIs to reg/dereg user memory Date: Wed, 10 Oct 2018 19:56:29 +0300 Message-Id: <1539190590-31186-3-git-send-email-yishaih@mellanox.com> X-Mailer: git-send-email 1.8.2.3 In-Reply-To: <1539190590-31186-1-git-send-email-yishaih@mellanox.com> References: <1539190590-31186-1-git-send-email-yishaih@mellanox.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add DV DEVX APIs to reg/dereg user memory: mlx5dv_devx_umem_reg, mlx5dv_devx_umem_dereg. The returned umem_id should be used later on as part of other device commands that require registered user memory. (e.g. create_qp, etc.). Signed-off-by: Yishai Hadas --- debian/ibverbs-providers.symbols | 2 + providers/mlx5/libmlx5.map | 2 + providers/mlx5/man/CMakeLists.txt | 2 + providers/mlx5/man/mlx5dv_devx_umem_reg.3.md | 58 ++++++++++++++++++++++++++++ providers/mlx5/mlx5.h | 6 +++ providers/mlx5/mlx5dv.h | 9 +++++ providers/mlx5/verbs.c | 57 +++++++++++++++++++++++++++ 7 files changed, 136 insertions(+) create mode 100644 providers/mlx5/man/mlx5dv_devx_umem_reg.3.md diff --git a/debian/ibverbs-providers.symbols b/debian/ibverbs-providers.symbols index acd1715..9bab750 100644 --- a/debian/ibverbs-providers.symbols +++ b/debian/ibverbs-providers.symbols @@ -33,3 +33,5 @@ libmlx5.so.1 ibverbs-providers #MINVER# mlx5dv_devx_obj_destroy@MLX5_1.7 21 mlx5dv_devx_obj_query@MLX5_1.7 21 mlx5dv_devx_obj_modify@MLX5_1.7 21 + mlx5dv_devx_umem_dereg@MLX5_1.7 21 + mlx5dv_devx_umem_reg@MLX5_1.7 21 diff --git a/providers/mlx5/libmlx5.map b/providers/mlx5/libmlx5.map index 91a7a32..d0499cd 100644 --- a/providers/mlx5/libmlx5.map +++ b/providers/mlx5/libmlx5.map @@ -51,4 +51,6 @@ MLX5_1.7 { mlx5dv_devx_obj_destroy; mlx5dv_devx_obj_query; mlx5dv_devx_obj_modify; + mlx5dv_devx_umem_dereg; + mlx5dv_devx_umem_reg; } MLX5_1.6; diff --git a/providers/mlx5/man/CMakeLists.txt b/providers/mlx5/man/CMakeLists.txt index 8d5d8e2..72a50e0 100644 --- a/providers/mlx5/man/CMakeLists.txt +++ b/providers/mlx5/man/CMakeLists.txt @@ -5,6 +5,7 @@ rdma_man_pages( mlx5dv_create_flow_matcher.3.md mlx5dv_create_qp.3.md mlx5dv_devx_obj_create.3.md + mlx5dv_devx_umem_reg.3.md mlx5dv_flow_action_esp.3.md mlx5dv_get_clock_info.3 mlx5dv_init_obj.3 @@ -18,4 +19,5 @@ rdma_alias_man_pages( mlx5dv_devx_obj_create.3 mlx5dv_devx_obj_destroy.3 mlx5dv_devx_obj_create.3 mlx5dv_devx_obj_query.3 mlx5dv_devx_obj_create.3 mlx5dv_devx_obj_modify.3 + mlx5dv_devx_umem_reg.3 mlx5dv_devx_umem_dereg.3 ) diff --git a/providers/mlx5/man/mlx5dv_devx_umem_reg.3.md b/providers/mlx5/man/mlx5dv_devx_umem_reg.3.md new file mode 100644 index 0000000..c829d54 --- /dev/null +++ b/providers/mlx5/man/mlx5dv_devx_umem_reg.3.md @@ -0,0 +1,58 @@ + +--- +layout: page +title: mlx5dv_devx_umem_reg, mlx5dv_devx_umem_dereg +section: 3 +tagline: Verbs +--- + +# NAME + +mlx5dv_devx_umem_reg - Register a user memory to be used by the devx interface + +mlx5dv_devx_umem_dereg - Deregister a devx umem object + +# SYNOPSIS + +```c +#include + +struct mlx5dv_devx_umem * +mlx5dv_devx_umem_reg(struct ibv_context *context, void *addr, size_t size, + uint32_t access) + +int mlx5dv_devx_umem_dereg(struct mlx5dv_devx_umem *dv_devx_umem) +``` + +# DESCRIPTION + +Register or deregister a user memory to be used by the devx interface. + +# ARGUMENTS +*context* +: RDMA device context to create the action on. + +*addr* +: The memory start address to register. + +*size* +: The size of *addr* buffer. + +*access* +: The desired memory protection attributes; it is either 0 or the bitwise OR of one or more of *enum ibv_access_flags*. + + +# RETURN VALUE + +Upon success *mlx5dv_devx_umem_reg* will return a new *struct +mlx5dv_devx_umem* object, on error NULL will be returned and errno will be set. + +*mlx5dv_devx_umem_dereg* returns 0 on success, or the value of errno on failure (which indicates the failure reason). + +# SEE ALSO + +*mlx5dv_open_device(3)*, *ibv_reg_mr(3)* + +#AUTHOR + +Yishai Hadas diff --git a/providers/mlx5/mlx5.h b/providers/mlx5/mlx5.h index 91fbc50..61240ed 100644 --- a/providers/mlx5/mlx5.h +++ b/providers/mlx5/mlx5.h @@ -572,6 +572,12 @@ struct mlx5dv_devx_obj { uint32_t handle; }; +struct mlx5_devx_umem { + struct mlx5dv_devx_umem dv_devx_umem; + struct ibv_context *context; + uint32_t handle; +}; + static inline int mlx5_ilog2(int n) { int t; diff --git a/providers/mlx5/mlx5dv.h b/providers/mlx5/mlx5dv.h index 662433b..0385b68 100644 --- a/providers/mlx5/mlx5dv.h +++ b/providers/mlx5/mlx5dv.h @@ -1027,6 +1027,15 @@ int mlx5dv_devx_obj_modify(struct mlx5dv_devx_obj *obj, const void *in, size_t i int mlx5dv_devx_obj_destroy(struct mlx5dv_devx_obj *obj); int mlx5dv_devx_general_cmd(struct ibv_context *context, const void *in, size_t inlen, void *out, size_t outlen); + +struct mlx5dv_devx_umem { + uint32_t umem_id; +}; + +struct mlx5dv_devx_umem * +mlx5dv_devx_umem_reg(struct ibv_context *ctx, void *addr, size_t size, uint32_t access); +int mlx5dv_devx_umem_dereg(struct mlx5dv_devx_umem *umem); + #ifdef __cplusplus } #endif diff --git a/providers/mlx5/verbs.c b/providers/mlx5/verbs.c index f476bc5..a8a7fa3 100644 --- a/providers/mlx5/verbs.c +++ b/providers/mlx5/verbs.c @@ -3843,6 +3843,63 @@ err: return NULL; } +struct mlx5dv_devx_umem * +mlx5dv_devx_umem_reg(struct ibv_context *context, void *addr, size_t size, uint32_t access) +{ + DECLARE_COMMAND_BUFFER(cmd, + MLX5_IB_OBJECT_DEVX_UMEM, + MLX5_IB_METHOD_DEVX_UMEM_REG, + 5); + struct ib_uverbs_attr *handle; + struct mlx5_devx_umem *umem; + int ret; + + umem = calloc(1, sizeof(*umem)); + if (!umem) { + errno = ENOMEM; + return NULL; + } + + fill_attr_in_uint64(cmd, MLX5_IB_ATTR_DEVX_UMEM_REG_ADDR, (intptr_t)addr); + fill_attr_in_uint64(cmd, MLX5_IB_ATTR_DEVX_UMEM_REG_LEN, size); + fill_attr_in_uint32(cmd, MLX5_IB_ATTR_DEVX_UMEM_REG_ACCESS, access); + fill_attr_out(cmd, MLX5_IB_ATTR_DEVX_UMEM_REG_OUT_ID, + &umem->dv_devx_umem.umem_id, + sizeof(umem->dv_devx_umem.umem_id)); + handle = fill_attr_out_obj(cmd, MLX5_IB_ATTR_DEVX_UMEM_REG_HANDLE); + + ret = execute_ioctl(context, cmd); + if (ret) + goto err; + + umem->handle = read_attr_obj(MLX5_IB_ATTR_DEVX_UMEM_REG_HANDLE, handle); + umem->context = context; + + return &umem->dv_devx_umem; +err: + free(umem); + return NULL; +} + +int mlx5dv_devx_umem_dereg(struct mlx5dv_devx_umem *dv_devx_umem) +{ + DECLARE_COMMAND_BUFFER(cmd, + MLX5_IB_OBJECT_DEVX_UMEM, + MLX5_IB_METHOD_DEVX_UMEM_DEREG, + 1); + int ret; + struct mlx5_devx_umem *umem = container_of(dv_devx_umem, struct mlx5_devx_umem, + dv_devx_umem); + + fill_attr_in_obj(cmd, MLX5_IB_ATTR_DEVX_UMEM_DEREG_HANDLE, umem->handle); + ret = execute_ioctl(umem->context, cmd); + if (ret) + return ret; + + free(umem); + return 0; +} + struct mlx5dv_devx_obj * mlx5dv_devx_obj_create(struct ibv_context *context, const void *in, size_t inlen, void *out, size_t outlen) From patchwork Wed Oct 10 16:56:30 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yishai Hadas X-Patchwork-Id: 10634901 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DF39317E3 for ; Wed, 10 Oct 2018 17:14:03 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1B74E29A86 for ; Wed, 10 Oct 2018 17:14:03 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0F02B29FFC; Wed, 10 Oct 2018 17:14:03 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A4B2029A86 for ; Wed, 10 Oct 2018 17:14:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726562AbeJKAhG (ORCPT ); Wed, 10 Oct 2018 20:37:06 -0400 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:59811 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726649AbeJKAhG (ORCPT ); Wed, 10 Oct 2018 20:37:06 -0400 Received: from Internal Mail-Server by MTLPINE1 (envelope-from yishaih@mellanox.com) with ESMTPS (AES256-SHA encrypted); 10 Oct 2018 19:01:45 +0200 Received: from vnc17.mtl.labs.mlnx (vnc17.mtl.labs.mlnx [10.7.2.17]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id w9AGus0b007096; Wed, 10 Oct 2018 19:56:54 +0300 Received: from vnc17.mtl.labs.mlnx (vnc17.mtl.labs.mlnx [127.0.0.1]) by vnc17.mtl.labs.mlnx (8.13.8/8.13.8) with ESMTP id w9AGusAA031326; Wed, 10 Oct 2018 19:56:54 +0300 Received: (from yishaih@localhost) by vnc17.mtl.labs.mlnx (8.13.8/8.13.8/Submit) id w9AGusk4031325; Wed, 10 Oct 2018 19:56:54 +0300 From: Yishai Hadas To: linux-rdma@vger.kernel.org Cc: yishaih@mellanox.com, Alexr@mellanox.com, jgg@mellanox.com, majd@mellanox.com Subject: [PATCH rdma-core 3/3] mlx5: Add support for DEVX object as a flow destination Date: Wed, 10 Oct 2018 19:56:30 +0300 Message-Id: <1539190590-31186-4-git-send-email-yishaih@mellanox.com> X-Mailer: git-send-email 1.8.2.3 In-Reply-To: <1539190590-31186-1-git-send-email-yishaih@mellanox.com> References: <1539190590-31186-1-git-send-email-yishaih@mellanox.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP A flow destination can be some DEVX object that was previously created, adds this option as part of mlx5dv_create_flow(). Signed-off-by: Yishai Hadas --- providers/mlx5/mlx5dv.h | 2 ++ providers/mlx5/verbs.c | 12 +++++++++++- 2 files changed, 13 insertions(+), 1 deletion(-) diff --git a/providers/mlx5/mlx5dv.h b/providers/mlx5/mlx5dv.h index 0385b68..9c39e3c 100644 --- a/providers/mlx5/mlx5dv.h +++ b/providers/mlx5/mlx5dv.h @@ -224,6 +224,7 @@ enum mlx5dv_flow_action_type { MLX5DV_FLOW_ACTION_IBV_COUNTER, MLX5DV_FLOW_ACTION_IBV_FLOW_ACTION, MLX5DV_FLOW_ACTION_TAG, + MLX5DV_FLOW_ACTION_DEST_DEVX, }; struct mlx5dv_flow_action_attr { @@ -233,6 +234,7 @@ struct mlx5dv_flow_action_attr { struct ibv_counters *counter; struct ibv_flow_action *action; uint32_t tag_value; + struct mlx5dv_devx_obj *obj; }; }; diff --git a/providers/mlx5/verbs.c b/providers/mlx5/verbs.c index a8a7fa3..13616ad 100644 --- a/providers/mlx5/verbs.c +++ b/providers/mlx5/verbs.c @@ -3775,6 +3775,7 @@ mlx5dv_create_flow(struct mlx5dv_flow_matcher *flow_matcher, int num_flow_actions = 0; struct mlx5_flow *mflow; bool have_qp = false; + bool have_dest_devx = false; int ret; int i; DECLARE_COMMAND_BUFFER(cmd, UVERBS_OBJECT_FLOW, @@ -3799,7 +3800,7 @@ mlx5dv_create_flow(struct mlx5dv_flow_matcher *flow_matcher, type = actions_attr[i].type; switch (type) { case MLX5DV_FLOW_ACTION_DEST_IBV_QP: - if (have_qp) { + if (have_qp || have_dest_devx) { errno = EOPNOTSUPP; goto err; } @@ -3820,6 +3821,15 @@ mlx5dv_create_flow(struct mlx5dv_flow_matcher *flow_matcher, flow_actions[num_flow_actions] = vaction->handle; num_flow_actions++; break; + case MLX5DV_FLOW_ACTION_DEST_DEVX: + if (have_dest_devx || have_qp) { + errno = EOPNOTSUPP; + goto err; + } + fill_attr_in_obj(cmd, MLX5_IB_ATTR_CREATE_FLOW_DEST_DEVX, + actions_attr[i].obj->handle); + have_dest_devx = true; + break; default: errno = EOPNOTSUPP; goto err;