From patchwork Tue Oct 9 15:41:35 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yishai Hadas X-Patchwork-Id: 10632901 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 65BC415E2 for ; Tue, 9 Oct 2018 15:42:37 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6226B2902C for ; Tue, 9 Oct 2018 15:42:37 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5683D29062; Tue, 9 Oct 2018 15:42:37 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DF86A2902C for ; Tue, 9 Oct 2018 15:42:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726393AbeJIXAG (ORCPT ); Tue, 9 Oct 2018 19:00:06 -0400 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:58882 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726492AbeJIXAG (ORCPT ); Tue, 9 Oct 2018 19:00:06 -0400 Received: from Internal Mail-Server by MTLPINE1 (envelope-from yishaih@mellanox.com) with ESMTPS (AES256-SHA encrypted); 9 Oct 2018 17:46:41 +0200 Received: from vnc17.mtl.labs.mlnx (vnc17.mtl.labs.mlnx [10.7.2.17]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id w99FfqSW015353; Tue, 9 Oct 2018 18:41:52 +0300 Received: from vnc17.mtl.labs.mlnx (vnc17.mtl.labs.mlnx [127.0.0.1]) by vnc17.mtl.labs.mlnx (8.13.8/8.13.8) with ESMTP id w99FfqAh011455; Tue, 9 Oct 2018 18:41:52 +0300 Received: (from yishaih@localhost) by vnc17.mtl.labs.mlnx (8.13.8/8.13.8/Submit) id w99FfqIi011451; Tue, 9 Oct 2018 18:41:52 +0300 From: Yishai Hadas To: linux-rdma@vger.kernel.org Cc: yishaih@mellanox.com, markb@mellanox.com, jgg@mellanox.com, majd@mellanox.com Subject: [PATCH rdma-core 3/3] mlx5: Expose device handles for RAW QP via the DV API Date: Tue, 9 Oct 2018 18:41:35 +0300 Message-Id: <1539099695-11323-4-git-send-email-yishaih@mellanox.com> X-Mailer: git-send-email 1.8.2.3 In-Reply-To: <1539099695-11323-1-git-send-email-yishaih@mellanox.com> References: <1539099695-11323-1-git-send-email-yishaih@mellanox.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Expose device handles for RAW QP via the DV API, this includes, tirn, tisn, rqn, sqn. This enables using the DEVX interface with the above device handles. Signed-off-by: Yishai Hadas --- providers/mlx5/mlx5.c | 8 ++++++++ providers/mlx5/mlx5.h | 4 ++++ providers/mlx5/mlx5dv.h | 5 +++++ providers/mlx5/verbs.c | 21 +++++++++++++++++---- 4 files changed, 34 insertions(+), 4 deletions(-) diff --git a/providers/mlx5/mlx5.c b/providers/mlx5/mlx5.c index 31bf3b2..22b27fd 100644 --- a/providers/mlx5/mlx5.c +++ b/providers/mlx5/mlx5.c @@ -761,6 +761,14 @@ static int mlx5dv_get_qp(struct ibv_qp *qp_in, mask_out |= MLX5DV_QP_MASK_UAR_MMAP_OFFSET; } + if (qp_out->comp_mask & MLX5DV_QP_MASK_RAW_QP_HANDLES) { + qp_out->tirn = mqp->tirn; + qp_out->tisn = mqp->tisn; + qp_out->rqn = mqp->rqn; + qp_out->sqn = mqp->sqn; + mask_out |= MLX5DV_QP_MASK_RAW_QP_HANDLES; + } + if (mqp->bf->uuarn > 0) qp_out->bf.size = mqp->bf->buf_size; else diff --git a/providers/mlx5/mlx5.h b/providers/mlx5/mlx5.h index 9af07c8..c5b9e6f 100644 --- a/providers/mlx5/mlx5.h +++ b/providers/mlx5/mlx5.h @@ -518,6 +518,10 @@ struct mlx5_qp { int rss_qp; uint32_t flags; /* Use enum mlx5_qp_flags */ enum mlx5dv_dc_type dc_type; + uint32_t tirn; + uint32_t tisn; + uint32_t rqn; + uint32_t sqn; }; struct mlx5_ah { diff --git a/providers/mlx5/mlx5dv.h b/providers/mlx5/mlx5dv.h index d340fdd..28d68bc 100644 --- a/providers/mlx5/mlx5dv.h +++ b/providers/mlx5/mlx5dv.h @@ -288,6 +288,7 @@ int mlx5dv_query_device(struct ibv_context *ctx_in, enum mlx5dv_qp_comp_mask { MLX5DV_QP_MASK_UAR_MMAP_OFFSET = 1 << 0, + MLX5DV_QP_MASK_RAW_QP_HANDLES = 1 << 1, }; struct mlx5dv_qp { @@ -308,6 +309,10 @@ struct mlx5dv_qp { } bf; uint64_t comp_mask; off_t uar_mmap_offset; + uint32_t tirn; + uint32_t tisn; + uint32_t rqn; + uint32_t sqn; }; struct mlx5dv_cq { diff --git a/providers/mlx5/verbs.c b/providers/mlx5/verbs.c index 54d229e..9ebd742 100644 --- a/providers/mlx5/verbs.c +++ b/providers/mlx5/verbs.c @@ -1673,11 +1673,11 @@ static struct ibv_qp *create_qp(struct ibv_context *context, struct mlx5_context *ctx = to_mctx(context); struct ibv_qp *ibqp; int32_t usr_idx = 0; - uint32_t uuar_index; uint32_t mlx5_create_flags = 0; struct mlx5_bf *bf = NULL; FILE *fp = ctx->dbg_fp; struct mlx5_parent_domain *mparent_domain; + struct mlx5_ib_create_qp_resp *resp_drv; if (attr->comp_mask & ~MLX5_CREATE_QP_SUP_COMP_MASK) return NULL; @@ -1884,8 +1884,8 @@ static struct ibv_qp *create_qp(struct ibv_context *context, goto err_free_uidx; } - uuar_index = (attr->comp_mask & MLX5_CREATE_QP_EX2_COMP_MASK) ? - resp_ex.bfreg_index : resp.bfreg_index; + resp_drv = attr->comp_mask & MLX5_CREATE_QP_EX2_COMP_MASK ? + &resp_ex.drv_payload : &resp.drv_payload; if (!ctx->cqe_version) { if (qp->sq.wqe_cnt || qp->rq.wqe_cnt) { ret = mlx5_store_qp(ctx, ibqp->qp_num, qp); @@ -1898,7 +1898,7 @@ static struct ibv_qp *create_qp(struct ibv_context *context, pthread_mutex_unlock(&ctx->qp_table_mutex); } - map_uuar(context, qp, uuar_index, bf); + map_uuar(context, qp, resp_drv->bfreg_index, bf); qp->rq.max_post = qp->rq.wqe_cnt; if (attr->sq_sig_all) @@ -1916,6 +1916,19 @@ static struct ibv_qp *create_qp(struct ibv_context *context, if (mparent_domain) atomic_fetch_add(&mparent_domain->mpd.refcount, 1); + + if (resp_drv->comp_mask & MLX5_IB_CREATE_QP_RESP_MASK_TIRN) + qp->tirn = resp_drv->tirn; + + if (resp_drv->comp_mask & MLX5_IB_CREATE_QP_RESP_MASK_TISN) + qp->tisn = resp_drv->tisn; + + if (resp_drv->comp_mask & MLX5_IB_CREATE_QP_RESP_MASK_RQN) + qp->rqn = resp_drv->rqn; + + if (resp_drv->comp_mask & MLX5_IB_CREATE_QP_RESP_MASK_SQN) + qp->sqn = resp_drv->sqn; + return ibqp; err_destroy: