From patchwork Thu Aug 2 16:13:32 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yishai Hadas X-Patchwork-Id: 10553943 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0EE8E13BB for ; Thu, 2 Aug 2018 16:14:40 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F12692C2CA for ; Thu, 2 Aug 2018 16:14:39 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E5C2F2C30A; Thu, 2 Aug 2018 16:14:39 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1EE172C2CA for ; Thu, 2 Aug 2018 16:14:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726547AbeHBSG1 (ORCPT ); Thu, 2 Aug 2018 14:06:27 -0400 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:48786 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726636AbeHBSG1 (ORCPT ); Thu, 2 Aug 2018 14:06:27 -0400 Received: from Internal Mail-Server by MTLPINE1 (envelope-from yishaih@mellanox.com) with ESMTPS (AES256-SHA encrypted); 2 Aug 2018 19:17:58 +0300 Received: from vnc17.mtl.labs.mlnx (vnc17.mtl.labs.mlnx [10.7.2.17]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id w72GEXRr029953; Thu, 2 Aug 2018 19:14:34 +0300 Received: from vnc17.mtl.labs.mlnx (vnc17.mtl.labs.mlnx [127.0.0.1]) by vnc17.mtl.labs.mlnx (8.13.8/8.13.8) with ESMTP id w72GEX6I019392; Thu, 2 Aug 2018 19:14:33 +0300 Received: (from yishaih@localhost) by vnc17.mtl.labs.mlnx (8.13.8/8.13.8/Submit) id w72GEXYj019389; Thu, 2 Aug 2018 19:14:33 +0300 From: Yishai Hadas To: linux-rdma@vger.kernel.org Cc: yishaih@mellanox.com, jgg@mellanox.com, majd@mellanox.com Subject: [PATCH rdma-core 2/6] verbs: Convert write path callers to use DECLARE_LEGACY_CORE_BUFS Date: Thu, 2 Aug 2018 19:13:32 +0300 Message-Id: <1533226416-19122-3-git-send-email-yishaih@mellanox.com> X-Mailer: git-send-email 1.8.2.3 In-Reply-To: <1533226416-19122-1-git-send-email-yishaih@mellanox.com> References: <1533226416-19122-1-git-send-email-yishaih@mellanox.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jason Gunthorpe This changes all the write cmd wrappers that do not use UDATA to directly use execute write and the new methods. This removes a lot of duplicated code and makes this code closer to something that can be moved into the ioctl fallback pattern. As part of this patch the ibv_cmd_dealloc_mw was cleaned-up to not get cmd and its size from its callers as there is no real usage for it in both user space and kernel. Signed-off-by: Jason Gunthorpe Signed-off-by: Yishai Hadas --- libibverbs/cmd.c | 206 ++++++++++++++++++++----------------------------- libibverbs/driver.h | 3 +- providers/mlx4/verbs.c | 3 +- providers/mlx5/verbs.c | 3 +- 4 files changed, 87 insertions(+), 128 deletions(-) diff --git a/libibverbs/cmd.c b/libibverbs/cmd.c index 0e60193..5bad9a3 100644 --- a/libibverbs/cmd.c +++ b/libibverbs/cmd.c @@ -41,6 +41,7 @@ #include #include +#include #include "ibverbs.h" #include @@ -334,15 +335,12 @@ int ibv_cmd_alloc_pd(struct ibv_context *context, struct ibv_pd *pd, int ibv_cmd_dealloc_pd(struct ibv_pd *pd) { - struct ibv_dealloc_pd cmd; + DECLARE_LEGACY_CORE_REQ(IB_USER_VERBS_CMD_DEALLOC_PD); - IBV_INIT_CMD(&cmd, sizeof cmd, DEALLOC_PD); - cmd.pd_handle = pd->handle; - - if (write(pd->context->cmd_fd, &cmd, sizeof cmd) != sizeof cmd) - return errno; - - return 0; + *req = (struct ib_uverbs_dealloc_pd) { + .pd_handle = pd->handle, + }; + return execute_write(pd->context, req, NULL); } int ibv_cmd_open_xrcd(struct ibv_context *context, struct verbs_xrcd *xrcd, @@ -379,15 +377,12 @@ int ibv_cmd_open_xrcd(struct ibv_context *context, struct verbs_xrcd *xrcd, int ibv_cmd_close_xrcd(struct verbs_xrcd *xrcd) { - struct ibv_close_xrcd cmd; - - IBV_INIT_CMD(&cmd, sizeof cmd, CLOSE_XRCD); - cmd.xrcd_handle = xrcd->handle; - - if (write(xrcd->xrcd.context->cmd_fd, &cmd, sizeof cmd) != sizeof cmd) - return errno; + DECLARE_LEGACY_CORE_REQ(IB_USER_VERBS_CMD_CLOSE_XRCD); - return 0; + *req = (struct ib_uverbs_close_xrcd){ + .xrcd_handle = xrcd->handle, + }; + return execute_write(xrcd->xrcd.context, req, NULL); } int ibv_cmd_reg_mr(struct ibv_pd *pd, void *addr, size_t length, @@ -450,16 +445,12 @@ int ibv_cmd_rereg_mr(struct verbs_mr *vmr, uint32_t flags, void *addr, int ibv_cmd_dereg_mr(struct verbs_mr *vmr) { - struct ibv_dereg_mr cmd; + DECLARE_LEGACY_CORE_REQ(IB_USER_VERBS_CMD_DEREG_MR); - IBV_INIT_CMD(&cmd, sizeof cmd, DEREG_MR); - cmd.mr_handle = vmr->ibv_mr.handle; - - if (write(vmr->ibv_mr.context->cmd_fd, &cmd, sizeof(cmd)) != - sizeof(cmd)) - return errno; - - return 0; + *req = (struct ib_uverbs_dereg_mr){ + .mr_handle = vmr->ibv_mr.handle, + }; + return execute_write(vmr->ibv_mr.context, req, NULL); } int ibv_cmd_alloc_mw(struct ibv_pd *pd, enum ibv_mw_type type, @@ -486,16 +477,14 @@ int ibv_cmd_alloc_mw(struct ibv_pd *pd, enum ibv_mw_type type, return 0; } -int ibv_cmd_dealloc_mw(struct ibv_mw *mw, - struct ibv_dealloc_mw *cmd, size_t cmd_size) +int ibv_cmd_dealloc_mw(struct ibv_mw *mw) { - IBV_INIT_CMD(cmd, cmd_size, DEALLOC_MW); - cmd->mw_handle = mw->handle; - - if (write(mw->context->cmd_fd, cmd, cmd_size) != cmd_size) - return errno; + DECLARE_LEGACY_CORE_REQ(IB_USER_VERBS_CMD_DEALLOC_MW); - return 0; + *req = (struct ib_uverbs_dealloc_mw) { + .mw_handle = mw->handle, + }; + return execute_write(mw->context, req, NULL); } int ibv_cmd_poll_cq(struct ibv_cq *ibcq, int ne, struct ibv_wc *wc) @@ -547,16 +536,13 @@ out: int ibv_cmd_req_notify_cq(struct ibv_cq *ibcq, int solicited_only) { - struct ibv_req_notify_cq cmd; - - IBV_INIT_CMD(&cmd, sizeof cmd, REQ_NOTIFY_CQ); - cmd.cq_handle = ibcq->handle; - cmd.solicited_only = !!solicited_only; + DECLARE_LEGACY_CORE_REQ(IB_USER_VERBS_CMD_REQ_NOTIFY_CQ); - if (write(ibcq->context->cmd_fd, &cmd, sizeof cmd) != sizeof cmd) - return errno; - - return 0; + *req = (struct ib_uverbs_req_notify_cq){ + .cq_handle = ibcq->handle, + .solicited_only = !!solicited_only, + }; + return execute_write(ibcq->context, req, NULL); } int ibv_cmd_resize_cq(struct ibv_cq *cq, int cqe, @@ -771,17 +757,16 @@ int ibv_cmd_query_srq(struct ibv_srq *srq, struct ibv_srq_attr *srq_attr, int ibv_cmd_destroy_srq(struct ibv_srq *srq) { - struct ibv_destroy_srq cmd; - struct ib_uverbs_destroy_qp_resp resp; - - IBV_INIT_CMD_RESP(&cmd, sizeof cmd, DESTROY_SRQ, &resp, sizeof resp); - cmd.srq_handle = srq->handle; - cmd.reserved = 0; + DECLARE_LEGACY_CORE_BUFS(IB_USER_VERBS_CMD_DESTROY_SRQ); + int ret; - if (write(srq->context->cmd_fd, &cmd, sizeof cmd) != sizeof cmd) - return errno; + *req = (struct ib_uverbs_destroy_srq){ + .srq_handle = srq->handle, + }; - (void) VALGRIND_MAKE_MEM_DEFINED(&resp, sizeof resp); + ret = execute_write(srq->context, req, &resp); + if (ret) + return ret; pthread_mutex_lock(&srq->mutex); while (srq->events_completed != resp.events_reported) @@ -1571,30 +1556,26 @@ int ibv_cmd_create_ah(struct ibv_pd *pd, struct ibv_ah *ah, int ibv_cmd_destroy_ah(struct ibv_ah *ah) { - struct ibv_destroy_ah cmd; - - IBV_INIT_CMD(&cmd, sizeof cmd, DESTROY_AH); - cmd.ah_handle = ah->handle; - - if (write(ah->context->cmd_fd, &cmd, sizeof cmd) != sizeof cmd) - return errno; + DECLARE_LEGACY_CORE_REQ(IB_USER_VERBS_CMD_DESTROY_AH); - return 0; + *req = (struct ib_uverbs_destroy_ah){ + .ah_handle = ah->handle, + }; + return execute_write(ah->context, req, NULL); } int ibv_cmd_destroy_qp(struct ibv_qp *qp) { - struct ibv_destroy_qp cmd; - struct ib_uverbs_destroy_qp_resp resp; + DECLARE_LEGACY_CORE_BUFS(IB_USER_VERBS_CMD_DESTROY_QP); + int ret; - IBV_INIT_CMD_RESP(&cmd, sizeof cmd, DESTROY_QP, &resp, sizeof resp); - cmd.qp_handle = qp->handle; - cmd.reserved = 0; + *req = (struct ib_uverbs_destroy_qp){ + .qp_handle = qp->handle, + }; - if (write(qp->context->cmd_fd, &cmd, sizeof cmd) != sizeof cmd) - return errno; - - (void) VALGRIND_MAKE_MEM_DEFINED(&resp, sizeof resp); + ret = execute_write(qp->context, req, &resp); + if (ret) + return ret; pthread_mutex_lock(&qp->mutex); while (qp->events_completed != resp.events_reported) @@ -1606,34 +1587,26 @@ int ibv_cmd_destroy_qp(struct ibv_qp *qp) int ibv_cmd_attach_mcast(struct ibv_qp *qp, const union ibv_gid *gid, uint16_t lid) { - struct ibv_attach_mcast cmd; - - IBV_INIT_CMD(&cmd, sizeof cmd, ATTACH_MCAST); - memcpy(cmd.gid, gid->raw, sizeof cmd.gid); - cmd.qp_handle = qp->handle; - cmd.mlid = lid; - cmd.reserved = 0; - - if (write(qp->context->cmd_fd, &cmd, sizeof cmd) != sizeof cmd) - return errno; - - return 0; + DECLARE_LEGACY_CORE_REQ(IB_USER_VERBS_CMD_ATTACH_MCAST); + + *req = (struct ib_uverbs_attach_mcast){ + .qp_handle = qp->handle, + .mlid = lid, + }; + memcpy(req->gid, gid->raw, sizeof(req->gid)); + return execute_write(qp->context, req, NULL); } int ibv_cmd_detach_mcast(struct ibv_qp *qp, const union ibv_gid *gid, uint16_t lid) { - struct ibv_detach_mcast cmd; - - IBV_INIT_CMD(&cmd, sizeof cmd, DETACH_MCAST); - memcpy(cmd.gid, gid->raw, sizeof cmd.gid); - cmd.qp_handle = qp->handle; - cmd.mlid = lid; - cmd.reserved = 0; - - if (write(qp->context->cmd_fd, &cmd, sizeof cmd) != sizeof cmd) - return errno; - - return 0; + DECLARE_LEGACY_CORE_REQ(IB_USER_VERBS_CMD_DETACH_MCAST); + + *req = (struct ib_uverbs_detach_mcast){ + .qp_handle = qp->handle, + .mlid = lid, + }; + memcpy(req->gid, gid->raw, sizeof(req->gid)); + return execute_write(qp->context, req, NULL); } static int buffer_is_zero(char *addr, ssize_t size) @@ -1893,16 +1866,12 @@ err: int ibv_cmd_destroy_flow(struct ibv_flow *flow_id) { - struct ibv_destroy_flow cmd; - int ret = 0; - - memset(&cmd, 0, sizeof(cmd)); - IBV_INIT_CMD_EX(&cmd, sizeof(cmd), DESTROY_FLOW); - cmd.flow_handle = flow_id->handle; + DECLARE_LEGACY_CORE_BUFS_EX(IB_USER_VERBS_EX_CMD_DESTROY_FLOW); - if (write(flow_id->context->cmd_fd, &cmd, sizeof(cmd)) != sizeof(cmd)) - ret = errno; - return ret; + *req = (struct ib_uverbs_destroy_flow){ + .flow_handle = flow_id->handle, + }; + return execute_write_ex(flow_id->context, req); } int ibv_cmd_create_wq(struct ibv_context *context, @@ -1997,18 +1966,16 @@ int ibv_cmd_modify_wq(struct ibv_wq *wq, struct ibv_wq_attr *attr, int ibv_cmd_destroy_wq(struct ibv_wq *wq) { - struct ibv_destroy_wq cmd; - struct ib_uverbs_ex_destroy_wq_resp resp; - int ret = 0; - - memset(&cmd, 0, sizeof(cmd)); - memset(&resp, 0, sizeof(resp)); + DECLARE_LEGACY_CORE_BUFS_EX(IB_USER_VERBS_EX_CMD_DESTROY_WQ); + int ret; - IBV_INIT_CMD_RESP_EX(&cmd, sizeof(cmd), DESTROY_WQ, &resp, sizeof(resp)); - cmd.wq_handle = wq->handle; + *req = (struct ib_uverbs_ex_destroy_wq){ + .wq_handle = wq->handle, + }; - if (write(wq->context->cmd_fd, &cmd, sizeof(cmd)) != sizeof(cmd)) - return errno; + ret = execute_write_ex(wq->context, req); + if (ret) + return ret; if (resp.response_length < sizeof(resp)) return EINVAL; @@ -2018,7 +1985,7 @@ int ibv_cmd_destroy_wq(struct ibv_wq *wq) pthread_cond_wait(&wq->cond, &wq->mutex); pthread_mutex_unlock(&wq->mutex); - return ret; + return 0; } int ibv_cmd_create_rwq_ind_table(struct ibv_context *context, @@ -2076,17 +2043,12 @@ int ibv_cmd_create_rwq_ind_table(struct ibv_context *context, int ibv_cmd_destroy_rwq_ind_table(struct ibv_rwq_ind_table *rwq_ind_table) { - struct ibv_destroy_rwq_ind_table cmd; - int ret = 0; - - memset(&cmd, 0, sizeof(cmd)); - IBV_INIT_CMD_EX(&cmd, sizeof(cmd), DESTROY_RWQ_IND_TBL); - cmd.ind_tbl_handle = rwq_ind_table->ind_tbl_handle; + DECLARE_LEGACY_CORE_BUFS_EX(IB_USER_VERBS_EX_CMD_DESTROY_RWQ_IND_TBL); - if (write(rwq_ind_table->context->cmd_fd, &cmd, sizeof(cmd)) != sizeof(cmd)) - ret = errno; - - return ret; + *req = (struct ib_uverbs_ex_destroy_rwq_ind_table){ + .ind_tbl_handle = rwq_ind_table->ind_tbl_handle, + }; + return execute_write_ex(rwq_ind_table->context, req); } diff --git a/libibverbs/driver.h b/libibverbs/driver.h index 64c8757..50962f8 100644 --- a/libibverbs/driver.h +++ b/libibverbs/driver.h @@ -435,8 +435,7 @@ int ibv_cmd_alloc_mw(struct ibv_pd *pd, enum ibv_mw_type type, struct ibv_mw *mw, struct ibv_alloc_mw *cmd, size_t cmd_size, struct ib_uverbs_alloc_mw_resp *resp, size_t resp_size); -int ibv_cmd_dealloc_mw(struct ibv_mw *mw, - struct ibv_dealloc_mw *cmd, size_t cmd_size); +int ibv_cmd_dealloc_mw(struct ibv_mw *mw); int ibv_cmd_create_cq(struct ibv_context *context, int cqe, struct ibv_comp_channel *channel, int comp_vector, struct ibv_cq *cq, diff --git a/providers/mlx4/verbs.c b/providers/mlx4/verbs.c index de778bb..40e94c4 100644 --- a/providers/mlx4/verbs.c +++ b/providers/mlx4/verbs.c @@ -354,9 +354,8 @@ struct ibv_mw *mlx4_alloc_mw(struct ibv_pd *pd, enum ibv_mw_type type) int mlx4_dealloc_mw(struct ibv_mw *mw) { int ret; - struct ibv_dealloc_mw cmd; - ret = ibv_cmd_dealloc_mw(mw, &cmd, sizeof(cmd)); + ret = ibv_cmd_dealloc_mw(mw); if (ret && !cleanup_on_fatal(ret)) return ret; diff --git a/providers/mlx5/verbs.c b/providers/mlx5/verbs.c index a6e0ae3..25f0d5f 100644 --- a/providers/mlx5/verbs.c +++ b/providers/mlx5/verbs.c @@ -518,9 +518,8 @@ struct ibv_mw *mlx5_alloc_mw(struct ibv_pd *pd, enum ibv_mw_type type) int mlx5_dealloc_mw(struct ibv_mw *mw) { int ret; - struct ibv_dealloc_mw cmd; - ret = ibv_cmd_dealloc_mw(mw, &cmd, sizeof(cmd)); + ret = ibv_cmd_dealloc_mw(mw); if (ret) return ret;