From patchwork Wed May 6 09:41:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yishai Hadas X-Patchwork-Id: 11530851 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9EAED92A for ; Wed, 6 May 2020 09:42:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 908A120838 for ; Wed, 6 May 2020 09:42:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729210AbgEFJmH (ORCPT ); Wed, 6 May 2020 05:42:07 -0400 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:32986 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728854AbgEFJmG (ORCPT ); Wed, 6 May 2020 05:42:06 -0400 Received: from Internal Mail-Server by MTLPINE2 (envelope-from yishaih@mellanox.com) with ESMTPS (AES256-SHA encrypted); 6 May 2020 12:42:00 +0300 Received: from vnc17.mtl.labs.mlnx (vnc17.mtl.labs.mlnx [10.7.2.17]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 0469g0Kq015438; Wed, 6 May 2020 12:42:00 +0300 Received: from vnc17.mtl.labs.mlnx (vnc17.mtl.labs.mlnx [127.0.0.1]) by vnc17.mtl.labs.mlnx (8.13.8/8.13.8) with ESMTP id 0469fxiC024583; Wed, 6 May 2020 12:41:59 +0300 Received: (from yishaih@localhost) by vnc17.mtl.labs.mlnx (8.13.8/8.13.8/Submit) id 0469fx5p024582; Wed, 6 May 2020 12:41:59 +0300 From: Yishai Hadas To: linux-rdma@vger.kernel.org Cc: jgg@mellanox.com, yishaih@mellanox.com, maorg@mellanox.com Subject: [PATCH rdma-core 1/8] Update kernel headers Date: Wed, 6 May 2020 12:41:02 +0300 Message-Id: <1588758069-24464-2-git-send-email-yishaih@mellanox.com> X-Mailer: git-send-email 1.8.2.3 In-Reply-To: <1588758069-24464-1-git-send-email-yishaih@mellanox.com> References: <1588758069-24464-1-git-send-email-yishaih@mellanox.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org To commit 86a2d80e24eb ("IB/uverbs: Introduce create/destroy QP commands over ioctl") Signed-off-by: Yishai Hadas --- kernel-headers/rdma/ib_user_ioctl_cmds.h | 81 +++++++++++++++++++++++++++++++ kernel-headers/rdma/ib_user_ioctl_verbs.h | 43 ++++++++++++++++ 2 files changed, 124 insertions(+) diff --git a/kernel-headers/rdma/ib_user_ioctl_cmds.h b/kernel-headers/rdma/ib_user_ioctl_cmds.h index d4ddbe4..4961d5e 100644 --- a/kernel-headers/rdma/ib_user_ioctl_cmds.h +++ b/kernel-headers/rdma/ib_user_ioctl_cmds.h @@ -95,6 +95,7 @@ enum uverbs_attrs_create_cq_cmd_attr_ids { UVERBS_ATTR_CREATE_CQ_COMP_VECTOR, UVERBS_ATTR_CREATE_CQ_FLAGS, UVERBS_ATTR_CREATE_CQ_RESP_CQE, + UVERBS_ATTR_CREATE_CQ_EVENT_FD, }; enum uverbs_attrs_destroy_cq_cmd_attr_ids { @@ -120,11 +121,91 @@ enum uverbs_attrs_destroy_flow_action_esp { UVERBS_ATTR_DESTROY_FLOW_ACTION_HANDLE, }; +enum uverbs_attrs_create_qp_cmd_attr_ids { + UVERBS_ATTR_CREATE_QP_HANDLE, + UVERBS_ATTR_CREATE_QP_XRCD_HANDLE, + UVERBS_ATTR_CREATE_QP_PD_HANDLE, + UVERBS_ATTR_CREATE_QP_SRQ_HANDLE, + UVERBS_ATTR_CREATE_QP_SEND_CQ_HANDLE, + UVERBS_ATTR_CREATE_QP_RECV_CQ_HANDLE, + UVERBS_ATTR_CREATE_QP_IND_TABLE_HANDLE, + UVERBS_ATTR_CREATE_QP_USER_HANDLE, + UVERBS_ATTR_CREATE_QP_CAP, + UVERBS_ATTR_CREATE_QP_TYPE, + UVERBS_ATTR_CREATE_QP_FLAGS, + UVERBS_ATTR_CREATE_QP_SOURCE_QPN, + UVERBS_ATTR_CREATE_QP_EVENT_FD, + UVERBS_ATTR_CREATE_QP_RESP_CAP, + UVERBS_ATTR_CREATE_QP_RESP_QP_NUM, +}; + +enum uverbs_attrs_destroy_qp_cmd_attr_ids { + UVERBS_ATTR_DESTROY_QP_HANDLE, + UVERBS_ATTR_DESTROY_QP_RESP, +}; + +enum uverbs_methods_qp { + UVERBS_METHOD_QP_CREATE, + UVERBS_METHOD_QP_DESTROY, +}; + +enum uverbs_attrs_create_srq_cmd_attr_ids { + UVERBS_ATTR_CREATE_SRQ_HANDLE, + UVERBS_ATTR_CREATE_SRQ_PD_HANDLE, + UVERBS_ATTR_CREATE_SRQ_XRCD_HANDLE, + UVERBS_ATTR_CREATE_SRQ_CQ_HANDLE, + UVERBS_ATTR_CREATE_SRQ_USER_HANDLE, + UVERBS_ATTR_CREATE_SRQ_MAX_WR, + UVERBS_ATTR_CREATE_SRQ_MAX_SGE, + UVERBS_ATTR_CREATE_SRQ_LIMIT, + UVERBS_ATTR_CREATE_SRQ_MAX_NUM_TAGS, + UVERBS_ATTR_CREATE_SRQ_TYPE, + UVERBS_ATTR_CREATE_SRQ_EVENT_FD, + UVERBS_ATTR_CREATE_SRQ_RESP_MAX_WR, + UVERBS_ATTR_CREATE_SRQ_RESP_MAX_SGE, + UVERBS_ATTR_CREATE_SRQ_RESP_SRQ_NUM, +}; + +enum uverbs_attrs_destroy_srq_cmd_attr_ids { + UVERBS_ATTR_DESTROY_SRQ_HANDLE, + UVERBS_ATTR_DESTROY_SRQ_RESP, +}; + +enum uverbs_methods_srq { + UVERBS_METHOD_SRQ_CREATE, + UVERBS_METHOD_SRQ_DESTROY, +}; + enum uverbs_methods_cq { UVERBS_METHOD_CQ_CREATE, UVERBS_METHOD_CQ_DESTROY, }; +enum uverbs_attrs_create_wq_cmd_attr_ids { + UVERBS_ATTR_CREATE_WQ_HANDLE, + UVERBS_ATTR_CREATE_WQ_PD_HANDLE, + UVERBS_ATTR_CREATE_WQ_CQ_HANDLE, + UVERBS_ATTR_CREATE_WQ_USER_HANDLE, + UVERBS_ATTR_CREATE_WQ_TYPE, + UVERBS_ATTR_CREATE_WQ_EVENT_FD, + UVERBS_ATTR_CREATE_WQ_MAX_WR, + UVERBS_ATTR_CREATE_WQ_MAX_SGE, + UVERBS_ATTR_CREATE_WQ_FLAGS, + UVERBS_ATTR_CREATE_WQ_RESP_MAX_WR, + UVERBS_ATTR_CREATE_WQ_RESP_MAX_SGE, + UVERBS_ATTR_CREATE_WQ_RESP_WQ_NUM, +}; + +enum uverbs_attrs_destroy_wq_cmd_attr_ids { + UVERBS_ATTR_DESTROY_WQ_HANDLE, + UVERBS_ATTR_DESTROY_WQ_RESP, +}; + +enum uverbs_methods_wq { + UVERBS_METHOD_WQ_CREATE, + UVERBS_METHOD_WQ_DESTROY, +}; + enum uverbs_methods_actions_flow_action_ops { UVERBS_METHOD_FLOW_ACTION_ESP_CREATE, UVERBS_METHOD_FLOW_ACTION_DESTROY, diff --git a/kernel-headers/rdma/ib_user_ioctl_verbs.h b/kernel-headers/rdma/ib_user_ioctl_verbs.h index a640bb8..5debab4 100644 --- a/kernel-headers/rdma/ib_user_ioctl_verbs.h +++ b/kernel-headers/rdma/ib_user_ioctl_verbs.h @@ -64,6 +64,41 @@ enum ib_uverbs_access_flags { ~(IB_UVERBS_ACCESS_OPTIONAL_FIRST - 1) }; +enum ib_uverbs_srq_type { + IB_UVERBS_SRQT_BASIC, + IB_UVERBS_SRQT_XRC, + IB_UVERBS_SRQT_TM, +}; + +enum ib_uverbs_wq_type { + IB_UVERBS_WQT_RQ, +}; + +enum ib_uverbs_wq_flags { + IB_UVERBS_WQ_FLAGS_CVLAN_STRIPPING = 1 << 0, + IB_UVERBS_WQ_FLAGS_SCATTER_FCS = 1 << 1, + IB_UVERBS_WQ_FLAGS_DELAY_DROP = 1 << 2, + IB_UVERBS_WQ_FLAGS_PCI_WRITE_END_PADDING = 1 << 3, +}; + +enum ib_uverbs_qp_type { + IB_UVERBS_QPT_RC = 2, + IB_UVERBS_QPT_UC, + IB_UVERBS_QPT_UD, + IB_UVERBS_QPT_RAW_PACKET = 8, + IB_UVERBS_QPT_XRC_INI, + IB_UVERBS_QPT_XRC_TGT, + IB_UVERBS_QPT_DRIVER = 0xFF, +}; + +enum ib_uverbs_qp_create_flags { + IB_UVERBS_QP_CREATE_BLOCK_MULTICAST_LOOPBACK = 1 << 1, + IB_UVERBS_QP_CREATE_SCATTER_FCS = 1 << 8, + IB_UVERBS_QP_CREATE_CVLAN_STRIPPING = 1 << 9, + IB_UVERBS_QP_CREATE_PCI_WRITE_END_PADDING = 1 << 11, + IB_UVERBS_QP_CREATE_SQ_SIG_ALL = 1 << 12, +}; + enum ib_uverbs_query_port_cap_flags { IB_UVERBS_PCF_SM = 1 << 1, IB_UVERBS_PCF_NOTICE_SUP = 1 << 2, @@ -185,6 +220,14 @@ struct ib_uverbs_query_port_resp_ex { __u8 reserved[6]; }; +struct ib_uverbs_qp_cap { + __u32 max_send_wr; + __u32 max_recv_wr; + __u32 max_send_sge; + __u32 max_recv_sge; + __u32 max_inline_data; +}; + enum rdma_driver_id { RDMA_DRIVER_UNKNOWN, RDMA_DRIVER_MLX5, From patchwork Wed May 6 09:41:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yishai Hadas X-Patchwork-Id: 11530849 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6D56E81 for ; Wed, 6 May 2020 09:42:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5FFF22082E for ; Wed, 6 May 2020 09:42:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729084AbgEFJmD (ORCPT ); Wed, 6 May 2020 05:42:03 -0400 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:55214 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1729193AbgEFJmD (ORCPT ); Wed, 6 May 2020 05:42:03 -0400 Received: from Internal Mail-Server by MTLPINE1 (envelope-from yishaih@mellanox.com) with ESMTPS (AES256-SHA encrypted); 6 May 2020 12:42:00 +0300 Received: from vnc17.mtl.labs.mlnx (vnc17.mtl.labs.mlnx [10.7.2.17]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 0469g07D015441; Wed, 6 May 2020 12:42:00 +0300 Received: from vnc17.mtl.labs.mlnx (vnc17.mtl.labs.mlnx [127.0.0.1]) by vnc17.mtl.labs.mlnx (8.13.8/8.13.8) with ESMTP id 0469g0Gk024587; Wed, 6 May 2020 12:42:00 +0300 Received: (from yishaih@localhost) by vnc17.mtl.labs.mlnx (8.13.8/8.13.8/Submit) id 0469g07J024586; Wed, 6 May 2020 12:42:00 +0300 From: Yishai Hadas To: linux-rdma@vger.kernel.org Cc: jgg@mellanox.com, yishaih@mellanox.com, maorg@mellanox.com Subject: [PATCH rdma-core 2/8] verbs: Extend CQ KABI to get an async FD Date: Wed, 6 May 2020 12:41:03 +0300 Message-Id: <1588758069-24464-3-git-send-email-yishaih@mellanox.com> X-Mailer: git-send-email 1.8.2.3 In-Reply-To: <1588758069-24464-1-git-send-email-yishaih@mellanox.com> References: <1588758069-24464-1-git-send-email-yishaih@mellanox.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Extend CQ KABI to get an async FD for the given CQ. For now this attribute is optional, down the road this may become mandatory in few cases. (e.g. secondary process). Signed-off-by: Yishai Hadas --- libibverbs/cmd_cq.c | 7 +++++-- libibverbs/cmd_fallback.c | 2 +- 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/libibverbs/cmd_cq.c b/libibverbs/cmd_cq.c index 542daa7..65293f6 100644 --- a/libibverbs/cmd_cq.c +++ b/libibverbs/cmd_cq.c @@ -31,14 +31,14 @@ */ #include - static int ibv_icmd_create_cq(struct ibv_context *context, int cqe, struct ibv_comp_channel *channel, int comp_vector, uint32_t flags, struct ibv_cq *cq, struct ibv_command_buffer *link) { - DECLARE_FBCMD_BUFFER(cmdb, UVERBS_OBJECT_CQ, UVERBS_METHOD_CQ_CREATE, 7, link); + DECLARE_FBCMD_BUFFER(cmdb, UVERBS_OBJECT_CQ, UVERBS_METHOD_CQ_CREATE, 8, link); struct ib_uverbs_attr *handle; + struct ib_uverbs_attr *async_fd_attr; uint32_t resp_cqe; int ret; @@ -52,6 +52,9 @@ static int ibv_icmd_create_cq(struct ibv_context *context, int cqe, if (channel) fill_attr_in_fd(cmdb, UVERBS_ATTR_CREATE_CQ_COMP_CHANNEL, channel->fd); fill_attr_in_uint32(cmdb, UVERBS_ATTR_CREATE_CQ_COMP_VECTOR, comp_vector); + async_fd_attr = fill_attr_in_fd(cmdb, UVERBS_ATTR_CREATE_CQ_EVENT_FD, context->async_fd); + /* Prevent fallback to the 'write' mode if kernel doesn't support it */ + attr_optional(async_fd_attr); if (flags) { fallback_require_ex(cmdb); diff --git a/libibverbs/cmd_fallback.c b/libibverbs/cmd_fallback.c index 46c09f3..ee18217 100644 --- a/libibverbs/cmd_fallback.c +++ b/libibverbs/cmd_fallback.c @@ -104,7 +104,7 @@ enum write_fallback _execute_ioctl_fallback(struct ibv_context *ctx, if (*ret == EPROTONOSUPPORT) { /* * EPROTONOSUPPORT means we have the ioctl framework but this - * specific method is not supported + * specific method or a mandatory attribute is not supported */ bitmap_set_bit(priv->unsupported_ioctls, cmd_bit); return _check_legacy(cmdb, ret); From patchwork Wed May 6 09:41:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yishai Hadas X-Patchwork-Id: 11530857 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B955781 for ; Wed, 6 May 2020 09:42:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id ABBA62087E for ; Wed, 6 May 2020 09:42:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729209AbgEFJmM (ORCPT ); Wed, 6 May 2020 05:42:12 -0400 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:33007 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1729134AbgEFJmL (ORCPT ); Wed, 6 May 2020 05:42:11 -0400 Received: from Internal Mail-Server by MTLPINE2 (envelope-from yishaih@mellanox.com) with ESMTPS (AES256-SHA encrypted); 6 May 2020 12:42:00 +0300 Received: from vnc17.mtl.labs.mlnx (vnc17.mtl.labs.mlnx [10.7.2.17]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 0469g0Mw015448; Wed, 6 May 2020 12:42:00 +0300 Received: from vnc17.mtl.labs.mlnx (vnc17.mtl.labs.mlnx [127.0.0.1]) by vnc17.mtl.labs.mlnx (8.13.8/8.13.8) with ESMTP id 0469g0a7024591; Wed, 6 May 2020 12:42:00 +0300 Received: (from yishaih@localhost) by vnc17.mtl.labs.mlnx (8.13.8/8.13.8/Submit) id 0469g056024590; Wed, 6 May 2020 12:42:00 +0300 From: Yishai Hadas To: linux-rdma@vger.kernel.org Cc: jgg@mellanox.com, yishaih@mellanox.com, maorg@mellanox.com Subject: [PATCH rdma-core 3/8] mlx4: Delete comp_mask from verbs_srq Date: Wed, 6 May 2020 12:41:04 +0300 Message-Id: <1588758069-24464-4-git-send-email-yishaih@mellanox.com> X-Mailer: git-send-email 1.8.2.3 In-Reply-To: <1588758069-24464-1-git-send-email-yishaih@mellanox.com> References: <1588758069-24464-1-git-send-email-yishaih@mellanox.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Jason Gunthorpe This is a hold over from when the driver.h was not tied to the rdma-core version, delete it. The only reader in mlx4 was using it to tell if the SRQ was XRC or not, just test that directly. Signed-off-by: Yishai Hadas Signed-off-by: Jason Gunthorpe --- libibverbs/cmd.c | 13 +++---------- libibverbs/driver.h | 19 ------------------- providers/mlx4/mlx4.c | 2 +- providers/mlx4/mlx4.h | 1 + providers/mlx4/verbs.c | 11 +++++++++++ 5 files changed, 16 insertions(+), 30 deletions(-) diff --git a/libibverbs/cmd.c b/libibverbs/cmd.c index 728d884..03d9143 100644 --- a/libibverbs/cmd.c +++ b/libibverbs/cmd.c @@ -577,21 +577,14 @@ int ibv_cmd_create_srq_ex(struct ibv_context *context, * If it is than all the others exist as well */ if (vext_field_avail(struct verbs_srq, srq_num, vsrq_sz)) { - srq->comp_mask = IBV_SRQ_INIT_ATTR_TYPE; srq->srq_type = (attr_ex->comp_mask & IBV_SRQ_INIT_ATTR_TYPE) ? attr_ex->srq_type : IBV_SRQT_BASIC; - if (srq->srq_type == IBV_SRQT_XRC) { - srq->comp_mask |= VERBS_SRQ_NUM; + if (srq->srq_type == IBV_SRQT_XRC) srq->srq_num = resp->srqn; - } - if (attr_ex->comp_mask & IBV_SRQ_INIT_ATTR_XRCD) { - srq->comp_mask |= VERBS_SRQ_XRCD; + if (attr_ex->comp_mask & IBV_SRQ_INIT_ATTR_XRCD) srq->xrcd = vxrcd; - } - if (attr_ex->comp_mask & IBV_SRQ_INIT_ATTR_CQ) { - srq->comp_mask |= VERBS_SRQ_CQ; + if (attr_ex->comp_mask & IBV_SRQ_INIT_ATTR_CQ) srq->cq = attr_ex->cq; - } } attr_ex->attr.max_wr = resp->max_wr; diff --git a/libibverbs/driver.h b/libibverbs/driver.h index a0e6f89..9cb78a5 100644 --- a/libibverbs/driver.h +++ b/libibverbs/driver.h @@ -59,17 +59,8 @@ struct verbs_xrcd { uint32_t handle; }; -enum verbs_srq_mask { - VERBS_SRQ_TYPE = 1 << 0, - VERBS_SRQ_XRCD = 1 << 1, - VERBS_SRQ_CQ = 1 << 2, - VERBS_SRQ_NUM = 1 << 3, - VERBS_SRQ_RESERVED = 1 << 4 -}; - struct verbs_srq { struct ibv_srq srq; - uint32_t comp_mask; enum ibv_srq_type srq_type; struct verbs_xrcd *xrcd; struct ibv_cq *cq; @@ -633,16 +624,6 @@ int ibv_read_ibdev_sysfs_file(char *buf, size_t size, __attribute__((format(printf, 4, 5))); int ibv_get_fw_ver(char *value, size_t len, struct verbs_sysfs_dev *sysfs_dev); -static inline int verbs_get_srq_num(struct ibv_srq *srq, uint32_t *srq_num) -{ - struct verbs_srq *vsrq = container_of(srq, struct verbs_srq, srq); - if (vsrq->comp_mask & VERBS_SRQ_NUM) { - *srq_num = vsrq->srq_num; - return 0; - } - return EOPNOTSUPP; -} - static inline bool check_comp_mask(uint64_t input, uint64_t supported) { return (input & ~supported) == 0; diff --git a/providers/mlx4/mlx4.c b/providers/mlx4/mlx4.c index 0842ff0..f04d346 100644 --- a/providers/mlx4/mlx4.c +++ b/providers/mlx4/mlx4.c @@ -126,7 +126,7 @@ static const struct verbs_context_ops mlx4_ctx_ops = { .destroy_flow = mlx4_destroy_flow, .destroy_rwq_ind_table = mlx4_destroy_rwq_ind_table, .destroy_wq = mlx4_destroy_wq, - .get_srq_num = verbs_get_srq_num, + .get_srq_num = mlx4_get_srq_num, .modify_cq = mlx4_modify_cq, .modify_wq = mlx4_modify_wq, .open_qp = mlx4_open_qp, diff --git a/providers/mlx4/mlx4.h b/providers/mlx4/mlx4.h index 3c161e8..16ed7d6 100644 --- a/providers/mlx4/mlx4.h +++ b/providers/mlx4/mlx4.h @@ -319,6 +319,7 @@ int mlx4_free_pd(struct ibv_pd *pd); struct ibv_xrcd *mlx4_open_xrcd(struct ibv_context *context, struct ibv_xrcd_init_attr *attr); int mlx4_close_xrcd(struct ibv_xrcd *xrcd); +int mlx4_get_srq_num(struct ibv_srq *srq, uint32_t *srq_num); struct ibv_mr *mlx4_reg_mr(struct ibv_pd *pd, void *addr, size_t length, uint64_t hca_va, int access); diff --git a/providers/mlx4/verbs.c b/providers/mlx4/verbs.c index 9f39ecd..010cd6c 100644 --- a/providers/mlx4/verbs.c +++ b/providers/mlx4/verbs.c @@ -274,6 +274,17 @@ int mlx4_close_xrcd(struct ibv_xrcd *ib_xrcd) return 0; } +int mlx4_get_srq_num(struct ibv_srq *srq, uint32_t *srq_num) +{ + struct mlx4_srq *msrq = + container_of(srq, struct mlx4_srq, verbs_srq.srq); + + if (!msrq->verbs_srq.xrcd) + return EOPNOTSUPP; + *srq_num = msrq->verbs_srq.srq_num; + return 0; +} + struct ibv_mr *mlx4_reg_mr(struct ibv_pd *pd, void *addr, size_t length, uint64_t hca_va, int access) { From patchwork Wed May 6 09:41:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yishai Hadas X-Patchwork-Id: 11530847 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 57A1392A for ; Wed, 6 May 2020 09:42:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4A1982082E for ; Wed, 6 May 2020 09:42:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729205AbgEFJmD (ORCPT ); Wed, 6 May 2020 05:42:03 -0400 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:55215 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1729084AbgEFJmD (ORCPT ); Wed, 6 May 2020 05:42:03 -0400 Received: from Internal Mail-Server by MTLPINE1 (envelope-from yishaih@mellanox.com) with ESMTPS (AES256-SHA encrypted); 6 May 2020 12:42:00 +0300 Received: from vnc17.mtl.labs.mlnx (vnc17.mtl.labs.mlnx [10.7.2.17]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 0469g0oO015451; Wed, 6 May 2020 12:42:00 +0300 Received: from vnc17.mtl.labs.mlnx (vnc17.mtl.labs.mlnx [127.0.0.1]) by vnc17.mtl.labs.mlnx (8.13.8/8.13.8) with ESMTP id 0469g0O5024596; Wed, 6 May 2020 12:42:00 +0300 Received: (from yishaih@localhost) by vnc17.mtl.labs.mlnx (8.13.8/8.13.8/Submit) id 0469g0Xq024595; Wed, 6 May 2020 12:42:00 +0300 From: Yishai Hadas To: linux-rdma@vger.kernel.org Cc: jgg@mellanox.com, yishaih@mellanox.com, maorg@mellanox.com Subject: [PATCH rdma-core 4/8] verbs: Fix ibv_get_srq_num() man page Date: Wed, 6 May 2020 12:41:05 +0300 Message-Id: <1588758069-24464-5-git-send-email-yishaih@mellanox.com> X-Mailer: git-send-email 1.8.2.3 In-Reply-To: <1588758069-24464-1-git-send-email-yishaih@mellanox.com> References: <1588758069-24464-1-git-send-email-yishaih@mellanox.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Fix ibv_get_srq_num() man page to mention that it's applicable only when the given SRQ is an XRC as defined by its specification. Signed-off-by: Yishai Hadas --- libibverbs/man/ibv_get_srq_num.3.md | 2 +- providers/mlx5/verbs.c | 6 +++++- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/libibverbs/man/ibv_get_srq_num.3.md b/libibverbs/man/ibv_get_srq_num.3.md index f015b9e..9140a37 100644 --- a/libibverbs/man/ibv_get_srq_num.3.md +++ b/libibverbs/man/ibv_get_srq_num.3.md @@ -23,7 +23,7 @@ int ibv_get_srq_num(struct ibv_srq *srq, uint32_t *srq_num); # DESCRIPTION -**ibv_get_srq_num()** return srq number associated with the given shared +**ibv_get_srq_num()** return srq number associated with the given XRC shared receive queue The argument *srq* is an ibv_srq struct, as defined in . *srq_num* is an output parameter that holds the returned srq number. diff --git a/providers/mlx5/verbs.c b/providers/mlx5/verbs.c index 47e8380..1835e93 100644 --- a/providers/mlx5/verbs.c +++ b/providers/mlx5/verbs.c @@ -2898,8 +2898,12 @@ int mlx5_get_srq_num(struct ibv_srq *srq, uint32_t *srq_num) { struct mlx5_srq *msrq = to_msrq(srq); + /* May be used by DC users in addition to XRC ones, as there is no + * indication on the SRQ for DC usage we can't force the above check. + * Even DC users are encouraged to use mlx5dv_init_obj() to get + * the SRQN. + */ *srq_num = msrq->srqn; - return 0; } From patchwork Wed May 6 09:41:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yishai Hadas X-Patchwork-Id: 11530853 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C0B2592A for ; Wed, 6 May 2020 09:42:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A493220838 for ; Wed, 6 May 2020 09:42:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729206AbgEFJmI (ORCPT ); Wed, 6 May 2020 05:42:08 -0400 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:33008 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1729207AbgEFJmH (ORCPT ); Wed, 6 May 2020 05:42:07 -0400 Received: from Internal Mail-Server by MTLPINE2 (envelope-from yishaih@mellanox.com) with ESMTPS (AES256-SHA encrypted); 6 May 2020 12:42:00 +0300 Received: from vnc17.mtl.labs.mlnx (vnc17.mtl.labs.mlnx [10.7.2.17]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 0469g0ep015455; Wed, 6 May 2020 12:42:00 +0300 Received: from vnc17.mtl.labs.mlnx (vnc17.mtl.labs.mlnx [127.0.0.1]) by vnc17.mtl.labs.mlnx (8.13.8/8.13.8) with ESMTP id 0469g0cl024600; Wed, 6 May 2020 12:42:00 +0300 Received: (from yishaih@localhost) by vnc17.mtl.labs.mlnx (8.13.8/8.13.8/Submit) id 0469g0Me024599; Wed, 6 May 2020 12:42:00 +0300 From: Yishai Hadas To: linux-rdma@vger.kernel.org Cc: jgg@mellanox.com, yishaih@mellanox.com, maorg@mellanox.com Subject: [PATCH rdma-core 5/8] verbs: Move SRQ create and destroy to ioctl Date: Wed, 6 May 2020 12:41:06 +0300 Message-Id: <1588758069-24464-6-git-send-email-yishaih@mellanox.com> X-Mailer: git-send-email 1.8.2.3 In-Reply-To: <1588758069-24464-1-git-send-email-yishaih@mellanox.com> References: <1588758069-24464-1-git-send-email-yishaih@mellanox.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Introduce create/destroy SRQ commands over the ioctl interface to let it be extended to get an asynchronous event FD. Signed-off-by: Yishai Hadas --- libibverbs/CMakeLists.txt | 1 + libibverbs/cmd.c | 137 ----------------------- libibverbs/cmd_srq.c | 279 ++++++++++++++++++++++++++++++++++++++++++++++ libibverbs/driver.h | 2 +- libibverbs/kern-abi.h | 4 + providers/mlx4/srq.c | 1 - providers/mlx5/verbs.c | 2 +- 7 files changed, 286 insertions(+), 140 deletions(-) create mode 100644 libibverbs/cmd_srq.c diff --git a/libibverbs/CMakeLists.txt b/libibverbs/CMakeLists.txt index 4328548..088f20f 100644 --- a/libibverbs/CMakeLists.txt +++ b/libibverbs/CMakeLists.txt @@ -37,6 +37,7 @@ rdma_library(ibverbs "${CMAKE_CURRENT_BINARY_DIR}/libibverbs.map" cmd_mw.c cmd_pd.c cmd_rwq_ind.c + cmd_srq.c cmd_xrcd.c compat-1_0.c device.c diff --git a/libibverbs/cmd.c b/libibverbs/cmd.c index 03d9143..a733cbf 100644 --- a/libibverbs/cmd.c +++ b/libibverbs/cmd.c @@ -480,120 +480,6 @@ int ibv_cmd_resize_cq(struct ibv_cq *cq, int cqe, return 0; } -int ibv_cmd_create_srq(struct ibv_pd *pd, - struct ibv_srq *srq, struct ibv_srq_init_attr *attr, - struct ibv_create_srq *cmd, size_t cmd_size, - struct ib_uverbs_create_srq_resp *resp, size_t resp_size) -{ - int ret; - - cmd->user_handle = (uintptr_t) srq; - cmd->pd_handle = pd->handle; - cmd->max_wr = attr->attr.max_wr; - cmd->max_sge = attr->attr.max_sge; - cmd->srq_limit = attr->attr.srq_limit; - - ret = execute_cmd_write(pd->context, IB_USER_VERBS_CMD_CREATE_SRQ, cmd, - cmd_size, resp, resp_size); - if (ret) - return ret; - - srq->handle = resp->srq_handle; - srq->context = pd->context; - - if (abi_ver > 5) { - attr->attr.max_wr = resp->max_wr; - attr->attr.max_sge = resp->max_sge; - } else { - struct ibv_create_srq_resp_v5 *resp_v5 = - (struct ibv_create_srq_resp_v5 *) resp; - - memmove((void *) resp + sizeof *resp, - (void *) resp_v5 + sizeof *resp_v5, - resp_size - sizeof *resp); - } - - return 0; -} - -int ibv_cmd_create_srq_ex(struct ibv_context *context, - struct verbs_srq *srq, int vsrq_sz, - struct ibv_srq_init_attr_ex *attr_ex, - struct ibv_create_xsrq *cmd, size_t cmd_size, - struct ib_uverbs_create_srq_resp *resp, size_t resp_size) -{ - struct verbs_xrcd *vxrcd = NULL; - int ret; - - if (attr_ex->comp_mask >= IBV_SRQ_INIT_ATTR_RESERVED) - return EOPNOTSUPP; - - if (!(attr_ex->comp_mask & IBV_SRQ_INIT_ATTR_PD)) - return EINVAL; - - cmd->user_handle = (uintptr_t) srq; - cmd->pd_handle = attr_ex->pd->handle; - cmd->max_wr = attr_ex->attr.max_wr; - cmd->max_sge = attr_ex->attr.max_sge; - cmd->srq_limit = attr_ex->attr.srq_limit; - - cmd->srq_type = (attr_ex->comp_mask & IBV_SRQ_INIT_ATTR_TYPE) ? - attr_ex->srq_type : IBV_SRQT_BASIC; - if (attr_ex->comp_mask & IBV_SRQ_INIT_ATTR_XRCD) { - if (!(attr_ex->comp_mask & IBV_SRQ_INIT_ATTR_CQ)) - return EINVAL; - - vxrcd = container_of(attr_ex->xrcd, struct verbs_xrcd, xrcd); - cmd->xrcd_handle = vxrcd->handle; - cmd->cq_handle = attr_ex->cq->handle; - } else if (attr_ex->comp_mask & IBV_SRQ_INIT_ATTR_TM) { - if (cmd->srq_type != IBV_SRQT_TM) - return EINVAL; - if (!(attr_ex->comp_mask & IBV_SRQ_INIT_ATTR_CQ) || - !attr_ex->tm_cap.max_num_tags) - return EINVAL; - - cmd->cq_handle = attr_ex->cq->handle; - cmd->max_num_tags = attr_ex->tm_cap.max_num_tags; - } else if (cmd->srq_type != IBV_SRQT_BASIC) { - return EINVAL; - } - - ret = execute_cmd_write(context, IB_USER_VERBS_CMD_CREATE_XSRQ, cmd, - cmd_size, resp, resp_size); - if (ret) - return ret; - - srq->srq.handle = resp->srq_handle; - srq->srq.context = context; - srq->srq.srq_context = attr_ex->srq_context; - srq->srq.pd = attr_ex->pd; - srq->srq.events_completed = 0; - pthread_mutex_init(&srq->srq.mutex, NULL); - pthread_cond_init(&srq->srq.cond, NULL); - - /* - * check that the last field is available. - * If it is than all the others exist as well - */ - if (vext_field_avail(struct verbs_srq, srq_num, vsrq_sz)) { - srq->srq_type = (attr_ex->comp_mask & IBV_SRQ_INIT_ATTR_TYPE) ? - attr_ex->srq_type : IBV_SRQT_BASIC; - if (srq->srq_type == IBV_SRQT_XRC) - srq->srq_num = resp->srqn; - if (attr_ex->comp_mask & IBV_SRQ_INIT_ATTR_XRCD) - srq->xrcd = vxrcd; - if (attr_ex->comp_mask & IBV_SRQ_INIT_ATTR_CQ) - srq->cq = attr_ex->cq; - } - - attr_ex->attr.max_wr = resp->max_wr; - attr_ex->attr.max_sge = resp->max_sge; - - return 0; -} - - static int ibv_cmd_modify_srq_v3(struct ibv_srq *srq, struct ibv_srq_attr *srq_attr, int srq_attr_mask, @@ -657,29 +543,6 @@ int ibv_cmd_query_srq(struct ibv_srq *srq, struct ibv_srq_attr *srq_attr, return 0; } -int ibv_cmd_destroy_srq(struct ibv_srq *srq) -{ - struct ibv_destroy_srq req; - struct ib_uverbs_destroy_srq_resp resp; - int ret; - - req.core_payload = (struct ib_uverbs_destroy_srq){ - .srq_handle = srq->handle, - }; - - ret = execute_cmd_write(srq->context, IB_USER_VERBS_CMD_DESTROY_SRQ, - &req, sizeof(req), &resp, sizeof(resp)); - if (verbs_is_destroy_err(&ret)) - return ret; - - pthread_mutex_lock(&srq->mutex); - while (srq->events_completed != resp.events_reported) - pthread_cond_wait(&srq->cond, &srq->mutex); - pthread_mutex_unlock(&srq->mutex); - - return 0; -} - static int create_qp_ex_common(struct verbs_qp *qp, struct ibv_qp_init_attr_ex *qp_attr, struct verbs_xrcd *vxrcd, diff --git a/libibverbs/cmd_srq.c b/libibverbs/cmd_srq.c new file mode 100644 index 0000000..4e63046 --- /dev/null +++ b/libibverbs/cmd_srq.c @@ -0,0 +1,279 @@ +/* + * Copyright (c) 2020 Mellanox Technologies, Ltd. All rights reserved. + * + * This software is available to you under a choice of one of two + * licenses. You may choose to be licensed under the terms of the GNU + * General Public License (GPL) Version 2, available from the file + * COPYING in the main directory of this source tree, or the + * OpenIB.org BSD license below: + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +#include +#include "ibverbs.h" + +static void set_vsrq(struct verbs_srq *vsrq, + struct ibv_srq_init_attr_ex *attr_ex, + uint32_t srq_num) +{ + vsrq->srq_type = (attr_ex->comp_mask & IBV_SRQ_INIT_ATTR_TYPE) ? + attr_ex->srq_type : IBV_SRQT_BASIC; + if (vsrq->srq_type == IBV_SRQT_XRC) { + vsrq->srq_num = srq_num; + vsrq->xrcd = container_of(attr_ex->xrcd, struct verbs_xrcd, xrcd); + } + if (attr_ex->comp_mask & IBV_SRQ_INIT_ATTR_CQ) + vsrq->cq = attr_ex->cq; +} + +static int ibv_icmd_create_srq(struct ibv_pd *pd, struct verbs_srq *vsrq, + struct ibv_srq *srq_in, + struct ibv_srq_init_attr_ex *attr_ex, + struct ibv_command_buffer *link) +{ + DECLARE_FBCMD_BUFFER(cmdb, UVERBS_OBJECT_SRQ, UVERBS_METHOD_SRQ_CREATE, 13, link); + struct ib_uverbs_attr *handle; + uint32_t max_wr; + uint32_t max_sge; + uint32_t srq_num; + int ret; + struct ibv_srq *srq = vsrq ? &vsrq->srq : srq_in; + struct verbs_xrcd *vxrcd = NULL; + enum ibv_srq_type srq_type; + + srq->context = pd->context; + pthread_mutex_init(&srq->mutex, NULL); + pthread_cond_init(&srq->cond, NULL); + + srq_type = (attr_ex->comp_mask & IBV_SRQ_INIT_ATTR_TYPE) ? + attr_ex->srq_type : IBV_SRQT_BASIC; + switch (srq_type) { + case IBV_SRQT_XRC: + if (!(attr_ex->comp_mask & IBV_SRQ_INIT_ATTR_XRCD) || + !(attr_ex->comp_mask & IBV_SRQ_INIT_ATTR_CQ)) { + errno = EINVAL; + return errno; + } + + vxrcd = container_of(attr_ex->xrcd, struct verbs_xrcd, xrcd); + fill_attr_in_obj(cmdb, UVERBS_ATTR_CREATE_SRQ_XRCD_HANDLE, vxrcd->handle); + fill_attr_in_obj(cmdb, UVERBS_ATTR_CREATE_SRQ_CQ_HANDLE, attr_ex->cq->handle); + fill_attr_out_ptr(cmdb, UVERBS_ATTR_CREATE_SRQ_RESP_SRQ_NUM, &srq_num); + break; + case IBV_SRQT_TM: + if (!(attr_ex->comp_mask & IBV_SRQ_INIT_ATTR_CQ) || + !(attr_ex->comp_mask & IBV_SRQ_INIT_ATTR_TM) || + !(attr_ex->tm_cap.max_num_tags)) { + errno = EINVAL; + return errno; + } + + fill_attr_in_obj(cmdb, UVERBS_ATTR_CREATE_SRQ_CQ_HANDLE, attr_ex->cq->handle); + fill_attr_in_uint32(cmdb, UVERBS_ATTR_CREATE_SRQ_MAX_NUM_TAGS, attr_ex->tm_cap.max_num_tags); + break; + default: + break; + } + + handle = fill_attr_out_obj(cmdb, UVERBS_ATTR_CREATE_SRQ_HANDLE); + fill_attr_const_in(cmdb, UVERBS_ATTR_CREATE_SRQ_TYPE, srq_type); + fill_attr_in_uint64(cmdb, UVERBS_ATTR_CREATE_SRQ_USER_HANDLE, (uintptr_t)srq); + fill_attr_in_obj(cmdb, UVERBS_ATTR_CREATE_SRQ_PD_HANDLE, pd->handle); + fill_attr_in_uint32(cmdb, UVERBS_ATTR_CREATE_SRQ_MAX_WR, attr_ex->attr.max_wr); + fill_attr_in_uint32(cmdb, UVERBS_ATTR_CREATE_SRQ_MAX_SGE, attr_ex->attr.max_sge); + fill_attr_in_uint32(cmdb, UVERBS_ATTR_CREATE_SRQ_LIMIT, attr_ex->attr.srq_limit); + fill_attr_in_fd(cmdb, UVERBS_ATTR_CREATE_SRQ_EVENT_FD, pd->context->async_fd); + fill_attr_out_ptr(cmdb, UVERBS_ATTR_CREATE_SRQ_RESP_MAX_WR, &max_wr); + fill_attr_out_ptr(cmdb, UVERBS_ATTR_CREATE_SRQ_RESP_MAX_SGE, &max_sge); + + switch (execute_ioctl_fallback(srq->context, create_srq, cmdb, &ret)) { + case TRY_WRITE: { + if (attr_ex->srq_type == IBV_SRQT_BASIC && abi_ver > 5) { + DECLARE_LEGACY_UHW_BUFS(link, IB_USER_VERBS_CMD_CREATE_SRQ); + + *req = (struct ib_uverbs_create_srq){ + .pd_handle = pd->handle, + .user_handle = (uintptr_t)srq, + .max_wr = attr_ex->attr.max_wr, + .max_sge = attr_ex->attr.max_sge, + .srq_limit = attr_ex->attr.srq_limit, + }; + + ret = execute_write_bufs( + srq->context, IB_USER_VERBS_CMD_CREATE_SRQ, req, resp); + if (ret) + return ret; + + srq->handle = resp->srq_handle; + attr_ex->attr.max_wr = resp->max_wr; + attr_ex->attr.max_sge = resp->max_sge; + } else if (attr_ex->srq_type == IBV_SRQT_BASIC && abi_ver <= 5) { + DECLARE_LEGACY_UHW_BUFS(link, IB_USER_VERBS_CMD_CREATE_SRQ_V5); + + *req = (struct ib_uverbs_create_srq){ + .pd_handle = pd->handle, + .user_handle = (uintptr_t)srq, + .max_wr = attr_ex->attr.max_wr, + .max_sge = attr_ex->attr.max_sge, + .srq_limit = attr_ex->attr.srq_limit, + }; + + ret = execute_write_bufs( + srq->context, IB_USER_VERBS_CMD_CREATE_SRQ_V5, req, resp); + if (ret) + return ret; + + srq->handle = resp->srq_handle; + } else { + DECLARE_LEGACY_UHW_BUFS(link, IB_USER_VERBS_CMD_CREATE_XSRQ); + + *req = (struct ib_uverbs_create_xsrq){ + .pd_handle = pd->handle, + .user_handle = (uintptr_t)srq, + .max_wr = attr_ex->attr.max_wr, + .max_sge = attr_ex->attr.max_sge, + .srq_limit = attr_ex->attr.srq_limit, + .srq_type = attr_ex->srq_type, + .cq_handle = attr_ex->cq->handle, + }; + + if (attr_ex->srq_type == IBV_SRQT_TM) + req->max_num_tags = attr_ex->tm_cap.max_num_tags; + else + req->xrcd_handle = vxrcd->handle; + + ret = execute_write_bufs( + srq->context, IB_USER_VERBS_CMD_CREATE_XSRQ, req, resp); + if (ret) + return ret; + + srq->handle = resp->srq_handle; + attr_ex->attr.max_wr = resp->max_wr; + attr_ex->attr.max_sge = resp->max_sge; + set_vsrq(vsrq, attr_ex, resp->srqn); + } + + return 0; + } + + case SUCCESS: + break; + + default: + return ret; + } + + srq->handle = read_attr_obj(UVERBS_ATTR_CREATE_SRQ_HANDLE, handle); + attr_ex->attr.max_wr = max_wr; + attr_ex->attr.max_sge = max_sge; + if (vsrq) + set_vsrq(vsrq, attr_ex, srq_num); + + return 0; +} + +int ibv_cmd_create_srq(struct ibv_pd *pd, struct ibv_srq *srq, + struct ibv_srq_init_attr *attr, + struct ibv_create_srq *cmd, size_t cmd_size, + struct ib_uverbs_create_srq_resp *resp, size_t resp_size) +{ + DECLARE_CMD_BUFFER_COMPAT(cmdb, UVERBS_OBJECT_SRQ, + UVERBS_METHOD_SRQ_CREATE, cmd, cmd_size, resp, + resp_size); + + struct ibv_srq_init_attr_ex attr_ex = {}; + int ret; + + memcpy(&attr_ex, attr, sizeof(*attr)); + ret = ibv_icmd_create_srq(pd, NULL, srq, &attr_ex, cmdb); + if (!ret) { + attr->attr.max_wr = attr_ex.attr.max_wr; + attr->attr.max_sge = attr_ex.attr.max_sge; + } + + return ret; +} + +int ibv_cmd_create_srq_ex(struct ibv_context *context, + struct verbs_srq *srq, + struct ibv_srq_init_attr_ex *attr_ex, + struct ibv_create_xsrq *cmd, size_t cmd_size, + struct ib_uverbs_create_srq_resp *resp, size_t resp_size) +{ + DECLARE_CMD_BUFFER_COMPAT(cmdb, UVERBS_OBJECT_SRQ, + UVERBS_METHOD_SRQ_CREATE, cmd, cmd_size, resp, + resp_size); + + if (attr_ex->comp_mask >= IBV_SRQ_INIT_ATTR_RESERVED) { + errno = EOPNOTSUPP; + return errno; + } + + if (!(attr_ex->comp_mask & IBV_SRQ_INIT_ATTR_PD)) { + errno = EINVAL; + return errno; + } + + return ibv_icmd_create_srq(attr_ex->pd, srq, NULL, attr_ex, cmdb); +} + +int ibv_cmd_destroy_srq(struct ibv_srq *srq) +{ + DECLARE_FBCMD_BUFFER(cmdb, UVERBS_OBJECT_SRQ, UVERBS_METHOD_SRQ_DESTROY, 2, + NULL); + struct ib_uverbs_destroy_srq_resp resp; + int ret; + + fill_attr_out_ptr(cmdb, UVERBS_ATTR_DESTROY_SRQ_RESP, &resp); + fill_attr_in_obj(cmdb, UVERBS_ATTR_DESTROY_SRQ_HANDLE, srq->handle); + + switch (execute_ioctl_fallback(srq->context, destroy_srq, cmdb, &ret)) { + case TRY_WRITE: { + struct ibv_destroy_srq req; + + req.core_payload = (struct ib_uverbs_destroy_srq){ + .srq_handle = srq->handle, + }; + + ret = execute_cmd_write(srq->context, + IB_USER_VERBS_CMD_DESTROY_SRQ, &req, + sizeof(req), &resp, sizeof(resp)); + break; + } + + default: + break; + } + + if (verbs_is_destroy_err(&ret)) + return ret; + + pthread_mutex_lock(&srq->mutex); + while (srq->events_completed != resp.events_reported) + pthread_cond_wait(&srq->cond, &srq->mutex); + pthread_mutex_unlock(&srq->mutex); + + return 0; +} + diff --git a/libibverbs/driver.h b/libibverbs/driver.h index 9cb78a5..9a6a8ae 100644 --- a/libibverbs/driver.h +++ b/libibverbs/driver.h @@ -503,7 +503,7 @@ int ibv_cmd_create_srq(struct ibv_pd *pd, struct ibv_create_srq *cmd, size_t cmd_size, struct ib_uverbs_create_srq_resp *resp, size_t resp_size); int ibv_cmd_create_srq_ex(struct ibv_context *context, - struct verbs_srq *srq, int vsrq_sz, + struct verbs_srq *srq, struct ibv_srq_init_attr_ex *attr_ex, struct ibv_create_xsrq *cmd, size_t cmd_size, struct ib_uverbs_create_srq_resp *resp, size_t resp_size); diff --git a/libibverbs/kern-abi.h b/libibverbs/kern-abi.h index dc2f33d..ef9c3f6 100644 --- a/libibverbs/kern-abi.h +++ b/libibverbs/kern-abi.h @@ -308,4 +308,8 @@ struct ibv_create_srq_resp_v5 { __u32 srq_handle; }; +#define _STRUCT_ib_uverbs_create_srq_v5 +enum { IB_USER_VERBS_CMD_CREATE_SRQ_V5 = IB_USER_VERBS_CMD_CREATE_SRQ }; +DECLARE_CMDX(IB_USER_VERBS_CMD_CREATE_SRQ_V5, ibv_create_srq_v5, ib_uverbs_create_srq, ibv_create_srq_resp_v5); + #endif /* KERN_ABI_H */ diff --git a/providers/mlx4/srq.c b/providers/mlx4/srq.c index a02a932..987b7b7 100644 --- a/providers/mlx4/srq.c +++ b/providers/mlx4/srq.c @@ -267,7 +267,6 @@ struct ibv_srq *mlx4_create_xrc_srq(struct ibv_context *context, cmd.db_addr = (uintptr_t) srq->db; ret = ibv_cmd_create_srq_ex(context, &srq->verbs_srq, - sizeof(srq->verbs_srq), attr_ex, &cmd.ibv_cmd, sizeof cmd, &resp.ibv_resp, sizeof resp); diff --git a/providers/mlx5/verbs.c b/providers/mlx5/verbs.c index 1835e93..830c5b1 100644 --- a/providers/mlx5/verbs.c +++ b/providers/mlx5/verbs.c @@ -3160,7 +3160,7 @@ struct ibv_srq *mlx5_create_srq_ex(struct ibv_context *context, */ attr->attr.max_wr = msrq->max - 1; - err = ibv_cmd_create_srq_ex(context, &msrq->vsrq, sizeof(msrq->vsrq), + err = ibv_cmd_create_srq_ex(context, &msrq->vsrq, attr, &cmd.ibv_cmd, sizeof(cmd), &resp.ibv_resp, sizeof(resp)); From patchwork Wed May 6 09:41:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yishai Hadas X-Patchwork-Id: 11530855 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5DE9C81 for ; Wed, 6 May 2020 09:42:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 50D2D20882 for ; Wed, 6 May 2020 09:42:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729207AbgEFJmK (ORCPT ); Wed, 6 May 2020 05:42:10 -0400 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:33015 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1729209AbgEFJmH (ORCPT ); Wed, 6 May 2020 05:42:07 -0400 Received: from Internal Mail-Server by MTLPINE2 (envelope-from yishaih@mellanox.com) with ESMTPS (AES256-SHA encrypted); 6 May 2020 12:42:00 +0300 Received: from vnc17.mtl.labs.mlnx (vnc17.mtl.labs.mlnx [10.7.2.17]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 0469g0I3015458; Wed, 6 May 2020 12:42:00 +0300 Received: from vnc17.mtl.labs.mlnx (vnc17.mtl.labs.mlnx [127.0.0.1]) by vnc17.mtl.labs.mlnx (8.13.8/8.13.8) with ESMTP id 0469g0E1024604; Wed, 6 May 2020 12:42:00 +0300 Received: (from yishaih@localhost) by vnc17.mtl.labs.mlnx (8.13.8/8.13.8/Submit) id 0469g0HT024603; Wed, 6 May 2020 12:42:00 +0300 From: Yishai Hadas To: linux-rdma@vger.kernel.org Cc: jgg@mellanox.com, yishaih@mellanox.com, maorg@mellanox.com Subject: [PATCH rdma-core 6/8] verbs: Fix ibv_create_wq() to set wq_context Date: Wed, 6 May 2020 12:41:07 +0300 Message-Id: <1588758069-24464-7-git-send-email-yishaih@mellanox.com> X-Mailer: git-send-email 1.8.2.3 In-Reply-To: <1588758069-24464-1-git-send-email-yishaih@mellanox.com> References: <1588758069-24464-1-git-send-email-yishaih@mellanox.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Fix ibv_create_wq() to set wq_context upon a successful creation. Fixes: 2864904f82bf ("Introduce Work Queue object and its verbs") Signed-off-by: Yishai Hadas --- libibverbs/verbs.h | 1 + 1 file changed, 1 insertion(+) diff --git a/libibverbs/verbs.h b/libibverbs/verbs.h index 288985d..5e256b4 100644 --- a/libibverbs/verbs.h +++ b/libibverbs/verbs.h @@ -3073,6 +3073,7 @@ static inline struct ibv_wq *ibv_create_wq(struct ibv_context *context, wq = vctx->create_wq(context, wq_init_attr); if (wq) { + wq->wq_context = wq_init_attr->wq_context; wq->events_completed = 0; pthread_mutex_init(&wq->mutex, NULL); pthread_cond_init(&wq->cond, NULL); From patchwork Wed May 6 09:41:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yishai Hadas X-Patchwork-Id: 11530861 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1AE9881 for ; Wed, 6 May 2020 09:42:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 07DC42082E for ; Wed, 6 May 2020 09:42:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728640AbgEFJmU (ORCPT ); Wed, 6 May 2020 05:42:20 -0400 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:55221 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1729204AbgEFJmU (ORCPT ); Wed, 6 May 2020 05:42:20 -0400 Received: from Internal Mail-Server by MTLPINE1 (envelope-from yishaih@mellanox.com) with ESMTPS (AES256-SHA encrypted); 6 May 2020 12:42:00 +0300 Received: from vnc17.mtl.labs.mlnx (vnc17.mtl.labs.mlnx [10.7.2.17]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 0469g0R2015465; Wed, 6 May 2020 12:42:00 +0300 Received: from vnc17.mtl.labs.mlnx (vnc17.mtl.labs.mlnx [127.0.0.1]) by vnc17.mtl.labs.mlnx (8.13.8/8.13.8) with ESMTP id 0469g0mo024608; Wed, 6 May 2020 12:42:00 +0300 Received: (from yishaih@localhost) by vnc17.mtl.labs.mlnx (8.13.8/8.13.8/Submit) id 0469g0X1024607; Wed, 6 May 2020 12:42:00 +0300 From: Yishai Hadas To: linux-rdma@vger.kernel.org Cc: jgg@mellanox.com, yishaih@mellanox.com, maorg@mellanox.com Subject: [PATCH rdma-core 7/8] verbs: Move WQ create and destroy to ioctl Date: Wed, 6 May 2020 12:41:08 +0300 Message-Id: <1588758069-24464-8-git-send-email-yishaih@mellanox.com> X-Mailer: git-send-email 1.8.2.3 In-Reply-To: <1588758069-24464-1-git-send-email-yishaih@mellanox.com> References: <1588758069-24464-1-git-send-email-yishaih@mellanox.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Introduce create/destroy WQ commands over the ioctl interface to let it be extended to get an asynchronous event FD. Signed-off-by: Yishai Hadas --- libibverbs/CMakeLists.txt | 1 + libibverbs/cmd.c | 73 ------------------- libibverbs/cmd_wq.c | 173 ++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 174 insertions(+), 73 deletions(-) create mode 100644 libibverbs/cmd_wq.c diff --git a/libibverbs/CMakeLists.txt b/libibverbs/CMakeLists.txt index 088f20f..2b77828 100644 --- a/libibverbs/CMakeLists.txt +++ b/libibverbs/CMakeLists.txt @@ -38,6 +38,7 @@ rdma_library(ibverbs "${CMAKE_CURRENT_BINARY_DIR}/libibverbs.map" cmd_pd.c cmd_rwq_ind.c cmd_srq.c + cmd_wq.c cmd_xrcd.c compat-1_0.c device.c diff --git a/libibverbs/cmd.c b/libibverbs/cmd.c index a733cbf..d962649 100644 --- a/libibverbs/cmd.c +++ b/libibverbs/cmd.c @@ -1600,53 +1600,6 @@ int ibv_cmd_create_flow(struct ibv_qp *qp, return 0; } -int ibv_cmd_create_wq(struct ibv_context *context, - struct ibv_wq_init_attr *wq_init_attr, - struct ibv_wq *wq, - struct ibv_create_wq *cmd, - size_t cmd_size, - struct ib_uverbs_ex_create_wq_resp *resp, - size_t resp_size) -{ - int err; - - if (wq_init_attr->comp_mask >= IBV_WQ_INIT_ATTR_RESERVED) - return EINVAL; - - cmd->user_handle = (uintptr_t)wq; - cmd->pd_handle = wq_init_attr->pd->handle; - cmd->cq_handle = wq_init_attr->cq->handle; - cmd->wq_type = wq_init_attr->wq_type; - cmd->max_sge = wq_init_attr->max_sge; - cmd->max_wr = wq_init_attr->max_wr; - cmd->comp_mask = 0; - - if (wq_init_attr->comp_mask & IBV_WQ_INIT_ATTR_FLAGS) { - if (wq_init_attr->create_flags & ~(IBV_WQ_FLAGS_RESERVED - 1)) - return EOPNOTSUPP; - cmd->create_flags = wq_init_attr->create_flags; - } - - err = execute_cmd_write_ex(context, IB_USER_VERBS_EX_CMD_CREATE_WQ, - cmd, cmd_size, resp, resp_size); - if (err) - return err; - - if (resp->response_length < sizeof(*resp)) - return EINVAL; - - wq->handle = resp->wq_handle; - wq_init_attr->max_wr = resp->max_wr; - wq_init_attr->max_sge = resp->max_sge; - wq->wq_num = resp->wqn; - wq->context = context; - wq->cq = wq_init_attr->cq; - wq->pd = wq_init_attr->pd; - wq->wq_type = wq_init_attr->wq_type; - - return 0; -} - int ibv_cmd_modify_wq(struct ibv_wq *wq, struct ibv_wq_attr *attr, struct ibv_modify_wq *cmd, size_t cmd_size) { @@ -1679,32 +1632,6 @@ int ibv_cmd_modify_wq(struct ibv_wq *wq, struct ibv_wq_attr *attr, return 0; } -int ibv_cmd_destroy_wq(struct ibv_wq *wq) -{ - struct ibv_destroy_wq req; - struct ib_uverbs_ex_destroy_wq_resp resp; - int ret; - - req.core_payload = (struct ib_uverbs_ex_destroy_wq){ - .wq_handle = wq->handle, - }; - - ret = execute_cmd_write_ex(wq->context, IB_USER_VERBS_EX_CMD_DESTROY_WQ, - &req, sizeof(req), &resp, sizeof(resp)); - if (verbs_is_destroy_err(&ret)) - return ret; - - if (resp.response_length < sizeof(resp)) - return EINVAL; - - pthread_mutex_lock(&wq->mutex); - while (wq->events_completed != resp.events_reported) - pthread_cond_wait(&wq->cond, &wq->mutex); - pthread_mutex_unlock(&wq->mutex); - - return 0; -} - int ibv_cmd_create_rwq_ind_table(struct ibv_context *context, struct ibv_rwq_ind_table_init_attr *init_attr, struct ibv_rwq_ind_table *rwq_ind_table, diff --git a/libibverbs/cmd_wq.c b/libibverbs/cmd_wq.c new file mode 100644 index 0000000..d233c3a --- /dev/null +++ b/libibverbs/cmd_wq.c @@ -0,0 +1,173 @@ +/* + * Copyright (c) 2020 Mellanox Technologies, Ltd. All rights reserved. + * + * This software is available to you under a choice of one of two + * licenses. You may choose to be licensed under the terms of the GNU + * General Public License (GPL) Version 2, available from the file + * COPYING in the main directory of this source tree, or the + * OpenIB.org BSD license below: + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +#include + +static int ibv_icmd_create_wq(struct ibv_context *context, + struct ibv_wq_init_attr *wq_init_attr, + struct ibv_wq *wq, + struct ibv_command_buffer *link) +{ + DECLARE_FBCMD_BUFFER(cmdb, UVERBS_OBJECT_WQ, UVERBS_METHOD_WQ_CREATE, 13, link); + struct ib_uverbs_attr *handle; + uint32_t max_wr; + uint32_t max_sge; + uint32_t wq_num; + int ret; + + wq->context = context; + wq->cq = wq_init_attr->cq; + wq->pd = wq_init_attr->pd; + wq->wq_type = wq_init_attr->wq_type; + + handle = fill_attr_out_obj(cmdb, UVERBS_ATTR_CREATE_WQ_HANDLE); + fill_attr_in_uint64(cmdb, UVERBS_ATTR_CREATE_WQ_USER_HANDLE, (uintptr_t)wq); + fill_attr_in_obj(cmdb, UVERBS_ATTR_CREATE_WQ_PD_HANDLE, wq_init_attr->pd->handle); + fill_attr_in_obj(cmdb, UVERBS_ATTR_CREATE_WQ_CQ_HANDLE, wq_init_attr->cq->handle); + fill_attr_const_in(cmdb, UVERBS_ATTR_CREATE_WQ_TYPE, wq_init_attr->wq_type); + fill_attr_in_uint32(cmdb, UVERBS_ATTR_CREATE_WQ_MAX_WR, wq_init_attr->max_wr); + fill_attr_in_uint32(cmdb, UVERBS_ATTR_CREATE_WQ_MAX_SGE, wq_init_attr->max_sge); + fill_attr_in_fd(cmdb, UVERBS_ATTR_CREATE_WQ_EVENT_FD, wq->context->async_fd); + fill_attr_in_uint32(cmdb, UVERBS_ATTR_CREATE_WQ_FLAGS, wq_init_attr->create_flags); + fill_attr_out_ptr(cmdb, UVERBS_ATTR_CREATE_WQ_RESP_MAX_WR, &max_wr); + fill_attr_out_ptr(cmdb, UVERBS_ATTR_CREATE_WQ_RESP_MAX_SGE, &max_sge); + fill_attr_out_ptr(cmdb, UVERBS_ATTR_CREATE_WQ_RESP_WQ_NUM, &wq_num); + + fallback_require_ex(cmdb); + + switch (execute_ioctl_fallback(context, create_wq, cmdb, &ret)) { + case TRY_WRITE_EX: { + DECLARE_LEGACY_UHW_BUFS_EX(link, + IB_USER_VERBS_EX_CMD_CREATE_WQ); + + *req = (struct ib_uverbs_ex_create_wq){ + .user_handle = (uintptr_t)wq, + .pd_handle = wq_init_attr->pd->handle, + .cq_handle = wq_init_attr->cq->handle, + .max_wr = wq_init_attr->max_wr, + .max_sge = wq_init_attr->max_sge, + .wq_type = wq_init_attr->wq_type, + .create_flags = wq_init_attr->create_flags, + }; + + ret = execute_write_bufs_ex( + context, IB_USER_VERBS_EX_CMD_CREATE_WQ, req, resp); + if (ret) + return ret; + + wq->handle = resp->wq_handle; + wq_init_attr->max_wr = resp->max_wr; + wq_init_attr->max_sge = resp->max_sge; + wq->wq_num = resp->wqn; + return 0; + } + + case SUCCESS: + break; + + default: + return ret; + } + + wq->handle = read_attr_obj(UVERBS_ATTR_CREATE_WQ_HANDLE, handle); + wq->wq_num = wq_num; + wq_init_attr->max_wr = max_wr; + wq_init_attr->max_sge = max_sge; + + return 0; +} + +int ibv_cmd_create_wq(struct ibv_context *context, + struct ibv_wq_init_attr *wq_init_attr, + struct ibv_wq *wq, + struct ibv_create_wq *cmd, + size_t cmd_size, + struct ib_uverbs_ex_create_wq_resp *resp, + size_t resp_size) +{ + DECLARE_CMD_BUFFER_COMPAT(cmdb, UVERBS_OBJECT_WQ, + UVERBS_METHOD_WQ_CREATE, cmd, cmd_size, resp, + resp_size); + + if (wq_init_attr->comp_mask >= IBV_WQ_INIT_ATTR_RESERVED) { + errno = EINVAL; + return errno; + } + + if (wq_init_attr->comp_mask & IBV_WQ_INIT_ATTR_FLAGS) { + if (wq_init_attr->create_flags & ~(IBV_WQ_FLAGS_RESERVED - 1)) { + errno = EOPNOTSUPP; + return errno; + } + } + + return ibv_icmd_create_wq(context, wq_init_attr, wq, cmdb); +} + +int ibv_cmd_destroy_wq(struct ibv_wq *wq) +{ + DECLARE_FBCMD_BUFFER(cmdb, UVERBS_OBJECT_WQ, UVERBS_METHOD_WQ_DESTROY, 2, + NULL); + struct ib_uverbs_ex_destroy_wq_resp resp; + int ret; + + fill_attr_out_ptr(cmdb, UVERBS_ATTR_DESTROY_WQ_RESP, &resp.events_reported); + fill_attr_in_obj(cmdb, UVERBS_ATTR_DESTROY_WQ_HANDLE, wq->handle); + + switch (execute_ioctl_fallback(wq->context, destroy_wq, cmdb, &ret)) { + case TRY_WRITE: { + struct ibv_destroy_wq req; + + req.core_payload = (struct ib_uverbs_ex_destroy_wq){ + .wq_handle = wq->handle, + }; + + + ret = execute_cmd_write_ex(wq->context, IB_USER_VERBS_EX_CMD_DESTROY_WQ, + &req, sizeof(req), &resp, sizeof(resp)); + break; + } + + default: + break; + } + + if (verbs_is_destroy_err(&ret)) + return ret; + + pthread_mutex_lock(&wq->mutex); + while (wq->events_completed != resp.events_reported) + pthread_cond_wait(&wq->cond, &wq->mutex); + pthread_mutex_unlock(&wq->mutex); + + return 0; +} From patchwork Wed May 6 09:41:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yishai Hadas X-Patchwork-Id: 11530937 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B553181 for ; Wed, 6 May 2020 09:53:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9264520714 for ; Wed, 6 May 2020 09:53:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728640AbgEFJxL (ORCPT ); Wed, 6 May 2020 05:53:11 -0400 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:33619 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728306AbgEFJxJ (ORCPT ); Wed, 6 May 2020 05:53:09 -0400 Received: from Internal Mail-Server by MTLPINE2 (envelope-from yishaih@mellanox.com) with ESMTPS (AES256-SHA encrypted); 6 May 2020 12:42:00 +0300 Received: from vnc17.mtl.labs.mlnx (vnc17.mtl.labs.mlnx [10.7.2.17]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 0469g0Ed015468; Wed, 6 May 2020 12:42:00 +0300 Received: from vnc17.mtl.labs.mlnx (vnc17.mtl.labs.mlnx [127.0.0.1]) by vnc17.mtl.labs.mlnx (8.13.8/8.13.8) with ESMTP id 0469g0m7024612; Wed, 6 May 2020 12:42:00 +0300 Received: (from yishaih@localhost) by vnc17.mtl.labs.mlnx (8.13.8/8.13.8/Submit) id 0469g0kZ024611; Wed, 6 May 2020 12:42:00 +0300 From: Yishai Hadas To: linux-rdma@vger.kernel.org Cc: jgg@mellanox.com, yishaih@mellanox.com, maorg@mellanox.com Subject: [PATCH rdma-core 8/8] verbs: Move QP create and destroy commands to ioctl Date: Wed, 6 May 2020 12:41:09 +0300 Message-Id: <1588758069-24464-9-git-send-email-yishaih@mellanox.com> X-Mailer: git-send-email 1.8.2.3 In-Reply-To: <1588758069-24464-1-git-send-email-yishaih@mellanox.com> References: <1588758069-24464-1-git-send-email-yishaih@mellanox.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Introduce create/destroy QP commands over the ioctl interface to let it be extended to get an asynchronous event FD. Signed-off-by: Yishai Hadas --- libibverbs/CMakeLists.txt | 1 + libibverbs/cmd.c | 264 ------------------------- libibverbs/cmd_qp.c | 476 ++++++++++++++++++++++++++++++++++++++++++++++ libibverbs/driver.h | 4 +- libibverbs/kern-abi.h | 7 + libibverbs/verbs.c | 14 -- providers/efa/verbs.c | 2 +- providers/mlx4/verbs.c | 6 +- providers/mlx5/verbs.c | 8 +- 9 files changed, 494 insertions(+), 288 deletions(-) create mode 100644 libibverbs/cmd_qp.c diff --git a/libibverbs/CMakeLists.txt b/libibverbs/CMakeLists.txt index 2b77828..73380ed 100644 --- a/libibverbs/CMakeLists.txt +++ b/libibverbs/CMakeLists.txt @@ -36,6 +36,7 @@ rdma_library(ibverbs "${CMAKE_CURRENT_BINARY_DIR}/libibverbs.map" cmd_mr.c cmd_mw.c cmd_pd.c + cmd_qp.c cmd_rwq_ind.c cmd_srq.c cmd_wq.c diff --git a/libibverbs/cmd.c b/libibverbs/cmd.c index d962649..9512639 100644 --- a/libibverbs/cmd.c +++ b/libibverbs/cmd.c @@ -543,90 +543,6 @@ int ibv_cmd_query_srq(struct ibv_srq *srq, struct ibv_srq_attr *srq_attr, return 0; } -static int create_qp_ex_common(struct verbs_qp *qp, - struct ibv_qp_init_attr_ex *qp_attr, - struct verbs_xrcd *vxrcd, - struct ib_uverbs_create_qp *cmd) -{ - cmd->user_handle = (uintptr_t)qp; - - if (qp_attr->comp_mask & IBV_QP_INIT_ATTR_XRCD) { - vxrcd = container_of(qp_attr->xrcd, struct verbs_xrcd, xrcd); - cmd->pd_handle = vxrcd->handle; - } else { - if (!(qp_attr->comp_mask & IBV_QP_INIT_ATTR_PD)) - return EINVAL; - - cmd->pd_handle = qp_attr->pd->handle; - if (qp_attr->comp_mask & IBV_QP_INIT_ATTR_IND_TABLE) { - if (cmd->max_recv_wr || cmd->max_recv_sge || - cmd->recv_cq_handle || qp_attr->srq) - return EINVAL; - - /* send_cq is optinal */ - if (qp_attr->cap.max_send_wr) - cmd->send_cq_handle = qp_attr->send_cq->handle; - } else { - cmd->send_cq_handle = qp_attr->send_cq->handle; - - if (qp_attr->qp_type != IBV_QPT_XRC_SEND) { - cmd->recv_cq_handle = qp_attr->recv_cq->handle; - cmd->srq_handle = qp_attr->srq ? qp_attr->srq->handle : - 0; - } - } - } - - cmd->max_send_wr = qp_attr->cap.max_send_wr; - cmd->max_recv_wr = qp_attr->cap.max_recv_wr; - cmd->max_send_sge = qp_attr->cap.max_send_sge; - cmd->max_recv_sge = qp_attr->cap.max_recv_sge; - cmd->max_inline_data = qp_attr->cap.max_inline_data; - cmd->sq_sig_all = qp_attr->sq_sig_all; - cmd->qp_type = qp_attr->qp_type; - cmd->is_srq = !!qp_attr->srq; - cmd->reserved = 0; - - return 0; -} - -static void create_qp_handle_resp_common(struct ibv_context *context, - struct verbs_qp *qp, - struct ibv_qp_init_attr_ex *qp_attr, - struct ib_uverbs_create_qp_resp *resp, - struct verbs_xrcd *vxrcd, - int vqp_sz) -{ - if (abi_ver > 3) { - qp_attr->cap.max_recv_sge = resp->max_recv_sge; - qp_attr->cap.max_send_sge = resp->max_send_sge; - qp_attr->cap.max_recv_wr = resp->max_recv_wr; - qp_attr->cap.max_send_wr = resp->max_send_wr; - qp_attr->cap.max_inline_data = resp->max_inline_data; - } - - qp->qp.handle = resp->qp_handle; - qp->qp.qp_num = resp->qpn; - qp->qp.context = context; - qp->qp.qp_context = qp_attr->qp_context; - qp->qp.pd = qp_attr->pd; - qp->qp.send_cq = qp_attr->send_cq; - qp->qp.recv_cq = qp_attr->recv_cq; - qp->qp.srq = qp_attr->srq; - qp->qp.qp_type = qp_attr->qp_type; - qp->qp.state = IBV_QPS_RESET; - qp->qp.events_completed = 0; - pthread_mutex_init(&qp->qp.mutex, NULL); - pthread_cond_init(&qp->qp.cond, NULL); - - qp->comp_mask = 0; - if (vext_field_avail(struct verbs_qp, xrcd, vqp_sz) && - (qp_attr->comp_mask & IBV_QP_INIT_ATTR_XRCD)) { - qp->comp_mask |= VERBS_QP_XRCD; - qp->xrcd = vxrcd; - } -} - enum { CREATE_QP_EX2_SUP_CREATE_FLAGS = IBV_QP_CREATE_BLOCK_SELF_MCAST_LB | IBV_QP_CREATE_SCATTER_FCS | @@ -635,163 +551,6 @@ enum { IBV_QP_CREATE_PCI_WRITE_END_PADDING, }; -int ibv_cmd_create_qp_ex2(struct ibv_context *context, - struct verbs_qp *qp, int vqp_sz, - struct ibv_qp_init_attr_ex *qp_attr, - struct ibv_create_qp_ex *cmd, - size_t cmd_size, - struct ib_uverbs_ex_create_qp_resp *resp, - size_t resp_size) -{ - struct verbs_xrcd *vxrcd = NULL; - int err; - - if (!check_comp_mask(qp_attr->comp_mask, - IBV_QP_INIT_ATTR_PD | - IBV_QP_INIT_ATTR_XRCD | - IBV_QP_INIT_ATTR_CREATE_FLAGS | - IBV_QP_INIT_ATTR_MAX_TSO_HEADER | - IBV_QP_INIT_ATTR_IND_TABLE | - IBV_QP_INIT_ATTR_RX_HASH | - IBV_QP_INIT_ATTR_SEND_OPS_FLAGS)) - return EINVAL; - - memset(&cmd->core_payload, 0, sizeof(cmd->core_payload)); - - err = create_qp_ex_common(qp, qp_attr, vxrcd, - ibv_create_qp_ex_to_reg(cmd)); - if (err) - return err; - - if (qp_attr->comp_mask & IBV_QP_INIT_ATTR_CREATE_FLAGS) { - if (qp_attr->create_flags & ~CREATE_QP_EX2_SUP_CREATE_FLAGS) - return EINVAL; - cmd->create_flags = qp_attr->create_flags; - - if (qp_attr->create_flags & IBV_QP_CREATE_SOURCE_QPN) - cmd->source_qpn = qp_attr->source_qpn; - } - - if (qp_attr->comp_mask & IBV_QP_INIT_ATTR_IND_TABLE) { - cmd->rwq_ind_tbl_handle = qp_attr->rwq_ind_tbl->ind_tbl_handle; - cmd->comp_mask = IB_UVERBS_CREATE_QP_MASK_IND_TABLE; - } - - err = execute_cmd_write_ex(context, IB_USER_VERBS_EX_CMD_CREATE_QP, - cmd, cmd_size, resp, resp_size); - if (err) - return err; - - create_qp_handle_resp_common(context, qp, qp_attr, &resp->base, vxrcd, - vqp_sz); - - return 0; -} - -int ibv_cmd_create_qp_ex(struct ibv_context *context, - struct verbs_qp *qp, int vqp_sz, - struct ibv_qp_init_attr_ex *attr_ex, - struct ibv_create_qp *cmd, size_t cmd_size, - struct ib_uverbs_create_qp_resp *resp, size_t resp_size) -{ - struct verbs_xrcd *vxrcd = NULL; - int err; - - if (!check_comp_mask(attr_ex->comp_mask, - IBV_QP_INIT_ATTR_PD | - IBV_QP_INIT_ATTR_XRCD | - IBV_QP_INIT_ATTR_SEND_OPS_FLAGS)) - return EOPNOTSUPP; - - err = create_qp_ex_common(qp, attr_ex, vxrcd, - &cmd->core_payload); - if (err) - return err; - - err = execute_cmd_write(context, IB_USER_VERBS_CMD_CREATE_QP, cmd, - cmd_size, resp, resp_size); - if (err) - return err; - - if (abi_ver == 4) { - struct ibv_create_qp_resp_v4 *resp_v4 = - (struct ibv_create_qp_resp_v4 *)resp; - - memmove((void *)resp + sizeof *resp, - (void *)resp_v4 + sizeof *resp_v4, - resp_size - sizeof *resp); - } else if (abi_ver <= 3) { - struct ibv_create_qp_resp_v3 *resp_v3 = - (struct ibv_create_qp_resp_v3 *)resp; - - memmove((void *)resp + sizeof *resp, - (void *)resp_v3 + sizeof *resp_v3, - resp_size - sizeof *resp); - } - - create_qp_handle_resp_common(context, qp, attr_ex, resp, vxrcd, vqp_sz); - - return 0; -} - -int ibv_cmd_create_qp(struct ibv_pd *pd, - struct ibv_qp *qp, struct ibv_qp_init_attr *attr, - struct ibv_create_qp *cmd, size_t cmd_size, - struct ib_uverbs_create_qp_resp *resp, size_t resp_size) -{ - int ret; - - cmd->user_handle = (uintptr_t) qp; - cmd->pd_handle = pd->handle; - cmd->send_cq_handle = attr->send_cq->handle; - cmd->recv_cq_handle = attr->recv_cq->handle; - cmd->srq_handle = attr->srq ? attr->srq->handle : 0; - cmd->max_send_wr = attr->cap.max_send_wr; - cmd->max_recv_wr = attr->cap.max_recv_wr; - cmd->max_send_sge = attr->cap.max_send_sge; - cmd->max_recv_sge = attr->cap.max_recv_sge; - cmd->max_inline_data = attr->cap.max_inline_data; - cmd->sq_sig_all = attr->sq_sig_all; - cmd->qp_type = attr->qp_type; - cmd->is_srq = !!attr->srq; - cmd->reserved = 0; - - ret = execute_cmd_write(pd->context, IB_USER_VERBS_CMD_CREATE_QP, cmd, - cmd_size, resp, resp_size); - if (ret) - return ret; - - qp->handle = resp->qp_handle; - qp->qp_num = resp->qpn; - qp->context = pd->context; - - if (abi_ver > 3) { - attr->cap.max_recv_sge = resp->max_recv_sge; - attr->cap.max_send_sge = resp->max_send_sge; - attr->cap.max_recv_wr = resp->max_recv_wr; - attr->cap.max_send_wr = resp->max_send_wr; - attr->cap.max_inline_data = resp->max_inline_data; - } - - if (abi_ver == 4) { - struct ibv_create_qp_resp_v4 *resp_v4 = - (struct ibv_create_qp_resp_v4 *) resp; - - memmove((void *) resp + sizeof *resp, - (void *) resp_v4 + sizeof *resp_v4, - resp_size - sizeof *resp); - } else if (abi_ver <= 3) { - struct ibv_create_qp_resp_v3 *resp_v3 = - (struct ibv_create_qp_resp_v3 *) resp; - - memmove((void *) resp + sizeof *resp, - (void *) resp_v3 + sizeof *resp_v3, - resp_size - sizeof *resp); - } - - return 0; -} - int ibv_cmd_open_qp(struct ibv_context *context, struct verbs_qp *qp, int vqp_sz, struct ibv_qp_open_attr *attr, @@ -1294,29 +1053,6 @@ int ibv_cmd_create_ah(struct ibv_pd *pd, struct ibv_ah *ah, return 0; } -int ibv_cmd_destroy_qp(struct ibv_qp *qp) -{ - struct ibv_destroy_qp req; - struct ib_uverbs_destroy_qp_resp resp; - int ret; - - req.core_payload = (struct ib_uverbs_destroy_qp){ - .qp_handle = qp->handle, - }; - - ret = execute_cmd_write(qp->context, IB_USER_VERBS_CMD_DESTROY_QP, &req, - sizeof(req), &resp, sizeof(resp)); - if (verbs_is_destroy_err(&ret)) - return ret; - - pthread_mutex_lock(&qp->mutex); - while (qp->events_completed != resp.events_reported) - pthread_cond_wait(&qp->cond, &qp->mutex); - pthread_mutex_unlock(&qp->mutex); - - return 0; -} - int ibv_cmd_attach_mcast(struct ibv_qp *qp, const union ibv_gid *gid, uint16_t lid) { struct ibv_attach_mcast req; diff --git a/libibverbs/cmd_qp.c b/libibverbs/cmd_qp.c new file mode 100644 index 0000000..a11bb98 --- /dev/null +++ b/libibverbs/cmd_qp.c @@ -0,0 +1,476 @@ +/* + * Copyright (c) 2020 Mellanox Technologies, Ltd. All rights reserved. + * + * This software is available to you under a choice of one of two + * licenses. You may choose to be licensed under the terms of the GNU + * General Public License (GPL) Version 2, available from the file + * COPYING in the main directory of this source tree, or the + * OpenIB.org BSD license below: + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above + * copyright notice, this list of conditions and the following + * disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials + * provided with the distribution. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS + * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN + * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +#include +#include "ibverbs.h" + +enum { + CREATE_QP_EX_SUP_CREATE_FLAGS = IBV_QP_CREATE_BLOCK_SELF_MCAST_LB | + IBV_QP_CREATE_SCATTER_FCS | + IBV_QP_CREATE_CVLAN_STRIPPING | + IBV_QP_CREATE_SOURCE_QPN | + IBV_QP_CREATE_PCI_WRITE_END_PADDING +}; + + +static void set_qp(struct verbs_qp *vqp, + struct ibv_qp *qp_in, + struct ibv_qp_init_attr_ex *attr_ex, + struct verbs_xrcd *vxrcd) +{ + struct ibv_qp *qp = vqp ? &vqp->qp : qp_in; + + qp->qp_context = attr_ex->qp_context; + qp->pd = attr_ex->pd; + qp->send_cq = attr_ex->send_cq; + qp->recv_cq = attr_ex->recv_cq; + qp->srq = attr_ex->srq; + qp->qp_type = attr_ex->qp_type; + qp->state = IBV_QPS_RESET; + qp->events_completed = 0; + pthread_mutex_init(&qp->mutex, NULL); + pthread_cond_init(&qp->cond, NULL); + + if (vqp) { + vqp->comp_mask = 0; + if (attr_ex->comp_mask & IBV_QP_INIT_ATTR_XRCD) { + vqp->comp_mask |= VERBS_QP_XRCD; + vqp->xrcd = vxrcd; + } + } +} + +static int ibv_icmd_create_qp(struct ibv_context *context, + struct verbs_qp *vqp, + struct ibv_qp *qp_in, + struct ibv_qp_init_attr_ex *attr_ex, + struct ibv_command_buffer *link) +{ + DECLARE_FBCMD_BUFFER(cmdb, UVERBS_OBJECT_QP, UVERBS_METHOD_QP_CREATE, 15, link); + struct ib_uverbs_attr *handle; + uint32_t qp_num; + uint32_t pd_handle; + uint32_t send_cq_handle = 0; + uint32_t recv_cq_handle = 0; + int ret; + struct ibv_qp *qp = vqp ? &vqp->qp : qp_in; + struct verbs_xrcd *vxrcd = NULL; + uint32_t create_flags = 0; + + qp->context = context; + + switch (attr_ex->qp_type) { + case IBV_QPT_XRC_RECV: + if (!(attr_ex->comp_mask & IBV_QP_INIT_ATTR_XRCD)) { + errno = EINVAL; + return errno; + } + + vxrcd = container_of(attr_ex->xrcd, struct verbs_xrcd, xrcd); + fill_attr_in_obj(cmdb, UVERBS_ATTR_CREATE_QP_XRCD_HANDLE, vxrcd->handle); + pd_handle = vxrcd->handle; + break; + case IBV_QPT_RC: + case IBV_QPT_UD: + case IBV_QPT_UC: + case IBV_QPT_RAW_PACKET: + case IBV_QPT_XRC_SEND: + case IBV_QPT_DRIVER: + if (!(attr_ex->comp_mask & IBV_QP_INIT_ATTR_PD)) { + errno = EINVAL; + return errno; + } + + fill_attr_in_obj(cmdb, UVERBS_ATTR_CREATE_QP_PD_HANDLE, attr_ex->pd->handle); + pd_handle = attr_ex->pd->handle; + + if (attr_ex->comp_mask & IBV_QP_INIT_ATTR_IND_TABLE) { + if (attr_ex->cap.max_recv_wr || attr_ex->cap.max_recv_sge || + attr_ex->recv_cq || attr_ex->srq) { + errno = EINVAL; + return errno; + } + + fallback_require_ex(cmdb); + fill_attr_in_obj(cmdb, UVERBS_ATTR_CREATE_QP_IND_TABLE_HANDLE, + attr_ex->rwq_ind_tbl->ind_tbl_handle); + + /* send_cq is optinal */ + if (attr_ex->cap.max_send_wr) { + fill_attr_in_obj(cmdb, UVERBS_ATTR_CREATE_QP_SEND_CQ_HANDLE, + attr_ex->send_cq->handle); + send_cq_handle = attr_ex->send_cq->handle; + } + } else { + fill_attr_in_obj(cmdb, UVERBS_ATTR_CREATE_QP_SEND_CQ_HANDLE, + attr_ex->send_cq->handle); + send_cq_handle = attr_ex->send_cq->handle; + + if (attr_ex->qp_type != IBV_QPT_XRC_SEND) { + fill_attr_in_obj(cmdb, UVERBS_ATTR_CREATE_QP_RECV_CQ_HANDLE, + attr_ex->recv_cq->handle); + recv_cq_handle = attr_ex->recv_cq->handle; + } + } + + /* compatible with kernel code from the 'write' mode */ + if (attr_ex->qp_type == IBV_QPT_XRC_SEND) { + attr_ex->cap.max_recv_wr = 0; + attr_ex->cap.max_recv_sge = 0; + } + + break; + default: + errno = EINVAL; + return errno; + } + + handle = fill_attr_out_obj(cmdb, UVERBS_ATTR_CREATE_QP_HANDLE); + fill_attr_const_in(cmdb, UVERBS_ATTR_CREATE_QP_TYPE, attr_ex->qp_type); + fill_attr_in_uint64(cmdb, UVERBS_ATTR_CREATE_QP_USER_HANDLE, (uintptr_t)qp); + + static_assert(offsetof(struct ibv_qp_cap, max_send_wr) == + offsetof(struct ib_uverbs_qp_cap, max_send_wr), "Bad layout"); + static_assert(offsetof(struct ibv_qp_cap, max_recv_wr) == + offsetof(struct ib_uverbs_qp_cap, max_recv_wr), "Bad layout"); + static_assert(offsetof(struct ibv_qp_cap, max_send_sge) == + offsetof(struct ib_uverbs_qp_cap, max_send_sge), "Bad layout"); + static_assert(offsetof(struct ibv_qp_cap, max_recv_sge) == + offsetof(struct ib_uverbs_qp_cap, max_recv_sge), "Bad layout"); + static_assert(offsetof(struct ibv_qp_cap, max_inline_data) == + offsetof(struct ib_uverbs_qp_cap, max_inline_data), "Bad layout"); + + fill_attr_in_ptr(cmdb, UVERBS_ATTR_CREATE_QP_CAP, &attr_ex->cap); + fill_attr_in_fd(cmdb, UVERBS_ATTR_CREATE_QP_EVENT_FD, context->async_fd); + + if (attr_ex->sq_sig_all) + create_flags |= IB_UVERBS_QP_CREATE_SQ_SIG_ALL; + + if (attr_ex->comp_mask & IBV_QP_INIT_ATTR_CREATE_FLAGS) { + if (attr_ex->create_flags & ~CREATE_QP_EX_SUP_CREATE_FLAGS) { + errno = EINVAL; + return errno; + } + + fallback_require_ex(cmdb); + create_flags |= attr_ex->create_flags; + + if (attr_ex->create_flags & IBV_QP_CREATE_SOURCE_QPN) { + fill_attr_in_uint32(cmdb, UVERBS_ATTR_CREATE_QP_SOURCE_QPN, + attr_ex->source_qpn); + /* source QPN is a self attribute once moving to ioctl, + * no extra bit is supported. + */ + create_flags &= ~IBV_QP_CREATE_SOURCE_QPN; + } + } + + if (create_flags) + fill_attr_in_uint32(cmdb, UVERBS_ATTR_CREATE_QP_FLAGS, + create_flags); + + if (attr_ex->srq) + fill_attr_in_obj(cmdb, UVERBS_ATTR_CREATE_QP_SRQ_HANDLE, attr_ex->srq->handle); + + fill_attr_out_ptr(cmdb, UVERBS_ATTR_CREATE_QP_RESP_CAP, &attr_ex->cap); + fill_attr_out_ptr(cmdb, UVERBS_ATTR_CREATE_QP_RESP_QP_NUM, &qp_num); + + switch (execute_ioctl_fallback(context, create_qp, cmdb, &ret)) { + case TRY_WRITE: { + if (abi_ver > 4) { + DECLARE_LEGACY_UHW_BUFS(link, IB_USER_VERBS_CMD_CREATE_QP); + + *req = (struct ib_uverbs_create_qp){ + .pd_handle = pd_handle, + .user_handle = (uintptr_t)qp, + .max_send_wr = attr_ex->cap.max_send_wr, + .max_recv_wr = attr_ex->cap.max_recv_wr, + .max_send_sge = attr_ex->cap.max_send_sge, + .max_recv_sge = attr_ex->cap.max_recv_sge, + .max_inline_data = attr_ex->cap.max_inline_data, + .sq_sig_all = attr_ex->sq_sig_all, + .qp_type = attr_ex->qp_type, + .srq_handle = attr_ex->srq ? attr_ex->srq->handle : 0, + .is_srq = !!attr_ex->srq, + .recv_cq_handle = recv_cq_handle, + .send_cq_handle = send_cq_handle, + }; + + ret = execute_write_bufs( + context, IB_USER_VERBS_CMD_CREATE_QP, req, resp); + if (ret) + return ret; + + qp->handle = resp->qp_handle; + qp->qp_num = resp->qpn; + + attr_ex->cap.max_recv_sge = resp->max_recv_sge; + attr_ex->cap.max_send_sge = resp->max_send_sge; + attr_ex->cap.max_recv_wr = resp->max_recv_wr; + attr_ex->cap.max_send_wr = resp->max_send_wr; + attr_ex->cap.max_inline_data = resp->max_inline_data; + + } else if (abi_ver == 4) { + DECLARE_LEGACY_UHW_BUFS(link, IB_USER_VERBS_CMD_CREATE_QP_V4); + + *req = (struct ib_uverbs_create_qp){ + .pd_handle = pd_handle, + .user_handle = (uintptr_t)qp, + .max_send_wr = attr_ex->cap.max_send_wr, + .max_recv_wr = attr_ex->cap.max_recv_wr, + .max_send_sge = attr_ex->cap.max_send_sge, + .max_recv_sge = attr_ex->cap.max_recv_sge, + .max_inline_data = attr_ex->cap.max_inline_data, + .sq_sig_all = attr_ex->sq_sig_all, + .qp_type = attr_ex->qp_type, + .srq_handle = attr_ex->srq ? attr_ex->srq->handle : 0, + .is_srq = !!attr_ex->srq, + .recv_cq_handle = recv_cq_handle, + .send_cq_handle = send_cq_handle, + }; + + ret = execute_write_bufs( + context, IB_USER_VERBS_CMD_CREATE_QP_V4, req, resp); + if (ret) + return ret; + + qp->handle = resp->qp_handle; + qp->qp_num = resp->qpn; + + attr_ex->cap.max_recv_sge = resp->max_recv_sge; + attr_ex->cap.max_send_sge = resp->max_send_sge; + attr_ex->cap.max_recv_wr = resp->max_recv_wr; + attr_ex->cap.max_send_wr = resp->max_send_wr; + attr_ex->cap.max_inline_data = resp->max_inline_data; + } else { + DECLARE_LEGACY_UHW_BUFS(link, IB_USER_VERBS_CMD_CREATE_QP_V3); + + *req = (struct ib_uverbs_create_qp){ + .pd_handle = pd_handle, + .user_handle = (uintptr_t)qp, + .max_send_wr = attr_ex->cap.max_send_wr, + .max_recv_wr = attr_ex->cap.max_recv_wr, + .max_send_sge = attr_ex->cap.max_send_sge, + .max_recv_sge = attr_ex->cap.max_recv_sge, + .max_inline_data = attr_ex->cap.max_inline_data, + .sq_sig_all = attr_ex->sq_sig_all, + .qp_type = attr_ex->qp_type, + .srq_handle = attr_ex->srq ? attr_ex->srq->handle : 0, + .is_srq = !!attr_ex->srq, + .recv_cq_handle = recv_cq_handle, + .send_cq_handle = send_cq_handle, + }; + + ret = execute_write_bufs( + context, IB_USER_VERBS_CMD_CREATE_QP_V3, req, resp); + if (ret) + return ret; + + qp->handle = resp->qp_handle; + qp->qp_num = resp->qpn; + } + + set_qp(vqp, qp, attr_ex, vxrcd); + return 0; + } + + case TRY_WRITE_EX: { + DECLARE_LEGACY_UHW_BUFS_EX(link, + IB_USER_VERBS_EX_CMD_CREATE_QP); + + *req = (struct ib_uverbs_ex_create_qp){ + .pd_handle = pd_handle, + .user_handle = (uintptr_t)qp, + .max_send_wr = attr_ex->cap.max_send_wr, + .max_recv_wr = attr_ex->cap.max_recv_wr, + .max_send_sge = attr_ex->cap.max_send_sge, + .max_recv_sge = attr_ex->cap.max_recv_sge, + .max_inline_data = attr_ex->cap.max_inline_data, + .sq_sig_all = attr_ex->sq_sig_all, + .qp_type = attr_ex->qp_type, + .srq_handle = attr_ex->srq ? attr_ex->srq->handle : 0, + .is_srq = !!attr_ex->srq, + .recv_cq_handle = recv_cq_handle, + .send_cq_handle = send_cq_handle, + }; + + if (attr_ex->comp_mask & IBV_QP_INIT_ATTR_CREATE_FLAGS) { + req->create_flags = attr_ex->create_flags; + + if (attr_ex->create_flags & IBV_QP_CREATE_SOURCE_QPN) + req->source_qpn = attr_ex->source_qpn; + } + + if (attr_ex->comp_mask & IBV_QP_INIT_ATTR_IND_TABLE) { + req->rwq_ind_tbl_handle = attr_ex->rwq_ind_tbl->ind_tbl_handle; + req->comp_mask = IB_UVERBS_CREATE_QP_MASK_IND_TABLE; + } + + ret = execute_write_bufs_ex( + context, IB_USER_VERBS_EX_CMD_CREATE_QP, req, resp); + if (ret) + return ret; + + qp->handle = resp->base.qp_handle; + qp->qp_num = resp->base.qpn; + + attr_ex->cap.max_recv_sge = resp->base.max_recv_sge; + attr_ex->cap.max_send_sge = resp->base.max_send_sge; + attr_ex->cap.max_recv_wr = resp->base.max_recv_wr; + attr_ex->cap.max_send_wr = resp->base.max_send_wr; + attr_ex->cap.max_inline_data = resp->base.max_inline_data; + set_qp(vqp, qp, attr_ex, vxrcd); + return 0; + } + + case SUCCESS: + break; + + default: + return ret; + } + + qp->handle = read_attr_obj(UVERBS_ATTR_CREATE_QP_HANDLE, handle); + qp->qp_num = qp_num; + set_qp(vqp, qp, attr_ex, vxrcd); + + return 0; +} + +int ibv_cmd_create_qp(struct ibv_pd *pd, + struct ibv_qp *qp, struct ibv_qp_init_attr *attr, + struct ibv_create_qp *cmd, size_t cmd_size, + struct ib_uverbs_create_qp_resp *resp, size_t resp_size) +{ + DECLARE_CMD_BUFFER_COMPAT(cmdb, UVERBS_OBJECT_QP, + UVERBS_METHOD_QP_CREATE, cmd, cmd_size, resp, + resp_size); + + struct ibv_qp_init_attr_ex attr_ex = {}; + int ret; + + memcpy(&attr_ex, attr, sizeof(*attr)); + attr_ex.comp_mask |= IBV_QP_INIT_ATTR_PD; + attr_ex.pd = pd; + ret = ibv_icmd_create_qp(pd->context, NULL, qp, &attr_ex, cmdb); + if (!ret) + memcpy(&attr->cap, &attr_ex.cap, sizeof(attr_ex.cap)); + + return ret; +} + +int ibv_cmd_create_qp_ex(struct ibv_context *context, + struct verbs_qp *qp, + struct ibv_qp_init_attr_ex *attr_ex, + struct ibv_create_qp *cmd, size_t cmd_size, + struct ib_uverbs_create_qp_resp *resp, size_t resp_size) +{ + DECLARE_CMD_BUFFER_COMPAT(cmdb, UVERBS_OBJECT_QP, + UVERBS_METHOD_QP_CREATE, cmd, cmd_size, resp, + resp_size); + + if (!check_comp_mask(attr_ex->comp_mask, + IBV_QP_INIT_ATTR_PD | + IBV_QP_INIT_ATTR_XRCD | + IBV_QP_INIT_ATTR_SEND_OPS_FLAGS)) { + errno = EINVAL; + return errno; + } + + return ibv_icmd_create_qp(context, qp, NULL, attr_ex, cmdb); +} + +int ibv_cmd_create_qp_ex2(struct ibv_context *context, + struct verbs_qp *qp, + struct ibv_qp_init_attr_ex *attr_ex, + struct ibv_create_qp_ex *cmd, + size_t cmd_size, + struct ib_uverbs_ex_create_qp_resp *resp, + size_t resp_size) +{ + DECLARE_CMD_BUFFER_COMPAT(cmdb, UVERBS_OBJECT_QP, + UVERBS_METHOD_QP_CREATE, cmd, cmd_size, resp, + resp_size); + + if (!check_comp_mask(attr_ex->comp_mask, + IBV_QP_INIT_ATTR_PD | + IBV_QP_INIT_ATTR_XRCD | + IBV_QP_INIT_ATTR_CREATE_FLAGS | + IBV_QP_INIT_ATTR_MAX_TSO_HEADER | + IBV_QP_INIT_ATTR_IND_TABLE | + IBV_QP_INIT_ATTR_RX_HASH | + IBV_QP_INIT_ATTR_SEND_OPS_FLAGS)) { + errno = EINVAL; + return errno; + } + + return ibv_icmd_create_qp(context, qp, NULL, attr_ex, cmdb); +} + +int ibv_cmd_destroy_qp(struct ibv_qp *qp) +{ + DECLARE_FBCMD_BUFFER(cmdb, UVERBS_OBJECT_QP, UVERBS_METHOD_QP_DESTROY, 2, + NULL); + struct ib_uverbs_destroy_qp_resp resp; + int ret; + + fill_attr_out_ptr(cmdb, UVERBS_ATTR_DESTROY_QP_RESP, &resp); + fill_attr_in_obj(cmdb, UVERBS_ATTR_DESTROY_QP_HANDLE, qp->handle); + + switch (execute_ioctl_fallback(qp->context, destroy_qp, cmdb, &ret)) { + case TRY_WRITE: { + struct ibv_destroy_qp req; + + req.core_payload = (struct ib_uverbs_destroy_qp){ + .qp_handle = qp->handle, + }; + + ret = execute_cmd_write(qp->context, + IB_USER_VERBS_CMD_DESTROY_QP, &req, + sizeof(req), &resp, sizeof(resp)); + break; + } + + default: + break; + } + + if (verbs_is_destroy_err(&ret)) + return ret; + + pthread_mutex_lock(&qp->mutex); + while (qp->events_completed != resp.events_reported) + pthread_cond_wait(&qp->cond, &qp->mutex); + pthread_mutex_unlock(&qp->mutex); + + return 0; +} diff --git a/libibverbs/driver.h b/libibverbs/driver.h index 9a6a8ae..de81955 100644 --- a/libibverbs/driver.h +++ b/libibverbs/driver.h @@ -521,12 +521,12 @@ int ibv_cmd_create_qp(struct ibv_pd *pd, struct ibv_create_qp *cmd, size_t cmd_size, struct ib_uverbs_create_qp_resp *resp, size_t resp_size); int ibv_cmd_create_qp_ex(struct ibv_context *context, - struct verbs_qp *qp, int vqp_sz, + struct verbs_qp *qp, struct ibv_qp_init_attr_ex *attr_ex, struct ibv_create_qp *cmd, size_t cmd_size, struct ib_uverbs_create_qp_resp *resp, size_t resp_size); int ibv_cmd_create_qp_ex2(struct ibv_context *context, - struct verbs_qp *qp, int vqp_sz, + struct verbs_qp *qp, struct ibv_qp_init_attr_ex *qp_attr, struct ibv_create_qp_ex *cmd, size_t cmd_size, diff --git a/libibverbs/kern-abi.h b/libibverbs/kern-abi.h index ef9c3f6..1238569 100644 --- a/libibverbs/kern-abi.h +++ b/libibverbs/kern-abi.h @@ -312,4 +312,11 @@ struct ibv_create_srq_resp_v5 { enum { IB_USER_VERBS_CMD_CREATE_SRQ_V5 = IB_USER_VERBS_CMD_CREATE_SRQ }; DECLARE_CMDX(IB_USER_VERBS_CMD_CREATE_SRQ_V5, ibv_create_srq_v5, ib_uverbs_create_srq, ibv_create_srq_resp_v5); +#define _STRUCT_ib_uverbs_create_qp_v4 +enum { IB_USER_VERBS_CMD_CREATE_QP_V4 = IB_USER_VERBS_CMD_CREATE_QP }; +DECLARE_CMDX(IB_USER_VERBS_CMD_CREATE_QP_V4, ibv_create_qp_v4, ib_uverbs_create_qp, ibv_create_qp_resp_v4); + +#define _STRUCT_ib_uverbs_create_qp_v3 +enum { IB_USER_VERBS_CMD_CREATE_QP_V3 = IB_USER_VERBS_CMD_CREATE_QP }; +DECLARE_CMDX(IB_USER_VERBS_CMD_CREATE_QP_V3, ibv_create_qp_v3, ib_uverbs_create_qp, ibv_create_qp_resp_v3); #endif /* KERN_ABI_H */ diff --git a/libibverbs/verbs.c b/libibverbs/verbs.c index 629f24c..f380036 100644 --- a/libibverbs/verbs.c +++ b/libibverbs/verbs.c @@ -600,20 +600,6 @@ LATEST_SYMVER_FUNC(ibv_create_qp, 1_1, "IBVERBS_1.1", { struct ibv_qp *qp = get_ops(pd->context)->create_qp(pd, qp_init_attr); - if (qp) { - qp->context = pd->context; - qp->qp_context = qp_init_attr->qp_context; - qp->pd = pd; - qp->send_cq = qp_init_attr->send_cq; - qp->recv_cq = qp_init_attr->recv_cq; - qp->srq = qp_init_attr->srq; - qp->qp_type = qp_init_attr->qp_type; - qp->state = IBV_QPS_RESET; - qp->events_completed = 0; - pthread_mutex_init(&qp->mutex, NULL); - pthread_cond_init(&qp->cond, NULL); - } - return qp; } diff --git a/providers/efa/verbs.c b/providers/efa/verbs.c index 03b8cf9..5f8e7b8 100644 --- a/providers/efa/verbs.c +++ b/providers/efa/verbs.c @@ -826,7 +826,7 @@ static struct ibv_qp *create_qp(struct ibv_context *ibvctx, if (attr->qp_type == IBV_QPT_DRIVER) req.driver_qp_type = efa_attr->driver_qp_type; - err = ibv_cmd_create_qp_ex(ibvctx, &qp->verbs_qp, sizeof(qp->verbs_qp), + err = ibv_cmd_create_qp_ex(ibvctx, &qp->verbs_qp, attr, &req.ibv_cmd, sizeof(req), &resp.ibv_resp, sizeof(resp)); if (err) diff --git a/providers/mlx4/verbs.c b/providers/mlx4/verbs.c index 010cd6c..c68cc35 100644 --- a/providers/mlx4/verbs.c +++ b/providers/mlx4/verbs.c @@ -790,7 +790,7 @@ static int mlx4_cmd_create_qp_ex_rss(struct ibv_context *context, sizeof(cmd_ex.rx_hash_key)); ret = ibv_cmd_create_qp_ex2(context, &qp->verbs_qp, - sizeof(qp->verbs_qp), attr, &cmd_ex.ibv_cmd, + attr, &cmd_ex.ibv_cmd, sizeof(cmd_ex), &resp.ibv_resp, sizeof(resp)); return ret; @@ -849,7 +849,7 @@ static int mlx4_cmd_create_qp_ex(struct ibv_context *context, cmd_ex.drv_payload = cmd->drv_payload; ret = ibv_cmd_create_qp_ex2(context, &qp->verbs_qp, - sizeof(qp->verbs_qp), attr, &cmd_ex.ibv_cmd, + attr, &cmd_ex.ibv_cmd, sizeof(cmd_ex), &resp.ibv_resp, sizeof(resp)); return ret; @@ -981,7 +981,7 @@ static struct ibv_qp *create_qp_ex(struct ibv_context *context, ret = mlx4_cmd_create_qp_ex(context, attr, &cmd, qp); else ret = ibv_cmd_create_qp_ex(context, &qp->verbs_qp, - sizeof(qp->verbs_qp), attr, + attr, &cmd.ibv_cmd, sizeof(cmd), &resp, sizeof(resp)); if (ret) diff --git a/providers/mlx5/verbs.c b/providers/mlx5/verbs.c index 830c5b1..ddf84f7 100644 --- a/providers/mlx5/verbs.c +++ b/providers/mlx5/verbs.c @@ -1866,7 +1866,7 @@ static int mlx5_cmd_create_rss_qp(struct ibv_context *context, attr->rx_hash_conf.rx_hash_key_len); ret = ibv_cmd_create_qp_ex2(context, &qp->verbs_qp, - sizeof(qp->verbs_qp), attr, + attr, &cmd_ex_rss.ibv_cmd, sizeof(cmd_ex_rss), &resp.ibv_resp, sizeof(resp)); if (ret) @@ -1899,7 +1899,7 @@ static int mlx5_cmd_create_qp_ex(struct ibv_context *context, cmd_ex.drv_payload = cmd->drv_payload; ret = ibv_cmd_create_qp_ex2(context, &qp->verbs_qp, - sizeof(qp->verbs_qp), attr, &cmd_ex.ibv_cmd, + attr, &cmd_ex.ibv_cmd, sizeof(cmd_ex), &resp->ibv_resp, sizeof(*resp)); @@ -1990,7 +1990,7 @@ static int create_dct(struct ibv_context *context, } cmd.uidx = usr_idx; - ret = ibv_cmd_create_qp_ex(context, &qp->verbs_qp, sizeof(qp->verbs_qp), + ret = ibv_cmd_create_qp_ex(context, &qp->verbs_qp, attr, &cmd.ibv_cmd, sizeof(cmd), &resp.ibv_resp, sizeof(resp)); if (ret) { @@ -2290,7 +2290,7 @@ static struct ibv_qp *create_qp(struct ibv_context *context, if (attr->comp_mask & MLX5_CREATE_QP_EX2_COMP_MASK) ret = mlx5_cmd_create_qp_ex(context, attr, &cmd, qp, &resp_ex); else - ret = ibv_cmd_create_qp_ex(context, &qp->verbs_qp, sizeof(qp->verbs_qp), + ret = ibv_cmd_create_qp_ex(context, &qp->verbs_qp, attr, &cmd.ibv_cmd, sizeof(cmd), &resp.ibv_resp, sizeof(resp)); if (ret) {