From patchwork Sun Nov 12 21:41:42 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yishai Hadas X-Patchwork-Id: 10054997 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id C9D9C60586 for ; Sun, 12 Nov 2017 23:40:47 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B1F7B292C7 for ; Sun, 12 Nov 2017 23:40:47 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 95193292CC; Sun, 12 Nov 2017 23:40:47 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F2318292C7 for ; Sun, 12 Nov 2017 23:40:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751405AbdKLXkn (ORCPT ); Sun, 12 Nov 2017 18:40:43 -0500 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:58394 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751418AbdKLXkk (ORCPT ); Sun, 12 Nov 2017 18:40:40 -0500 Received: from Internal Mail-Server by MTLPINE1 (envelope-from yishaih@mellanox.com) with ESMTPS (AES256-SHA encrypted); 12 Nov 2017 23:42:05 +0200 Received: from vnc17.mtl.labs.mlnx (vnc17.mtl.labs.mlnx [10.7.2.17]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id vACLg55B010480; Sun, 12 Nov 2017 23:42:05 +0200 Received: from vnc17.mtl.labs.mlnx (vnc17.mtl.labs.mlnx [127.0.0.1]) by vnc17.mtl.labs.mlnx (8.13.8/8.13.8) with ESMTP id vACLg5KS007002; Sun, 12 Nov 2017 23:42:05 +0200 Received: (from yishaih@localhost) by vnc17.mtl.labs.mlnx (8.13.8/8.13.8/Submit) id vACLg5k0007001; Sun, 12 Nov 2017 23:42:05 +0200 From: Yishai Hadas To: linux-rdma@vger.kernel.org Cc: yishaih@mellanox.com, majd@mellanox.com, Alexr@mellanox.com, dledford@redhat.com Subject: [PATCH RFC rdma-core 4/5] mlx5: Add support for ibv_parent domain and its related verbs Date: Sun, 12 Nov 2017 23:41:42 +0200 Message-Id: <1510522903-6838-5-git-send-email-yishaih@mellanox.com> X-Mailer: git-send-email 1.8.2.3 In-Reply-To: <1510522903-6838-1-git-send-email-yishaih@mellanox.com> References: <1510522903-6838-1-git-send-email-yishaih@mellanox.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This patch is the initial implementation of ibv_alloc/dealloc_parent domain verbs in the mlx5 driver. The driver uses internally mlx5_pd structure which its prefix holds the legacy ibv_pd so that the return ibv_pd can be used with any verb around that gets a protection domain. In addition, the driver saves an indication whether this ibv_pd is a parent domain and as such may hold a thread domain and a pointer to a previously allocated protection domain. In downstream patches the mlx5 driver will consider upon getting an ibv_pd which fields should be used based on the above indicator. Signed-off-by: Yishai Hadas --- providers/mlx5/mlx5.c | 2 ++ providers/mlx5/mlx5.h | 9 +++++++++ providers/mlx5/verbs.c | 23 +++++++++++++++++++++++ 3 files changed, 34 insertions(+) diff --git a/providers/mlx5/mlx5.c b/providers/mlx5/mlx5.c index 0c073e2..b160373 100644 --- a/providers/mlx5/mlx5.c +++ b/providers/mlx5/mlx5.c @@ -1011,6 +1011,8 @@ static int mlx5_init_context(struct verbs_device *vdev, verbs_set_ctx_op(v_ctx, post_srq_ops, mlx5_post_srq_ops); verbs_set_ctx_op(v_ctx, alloc_td, mlx5_alloc_td); verbs_set_ctx_op(v_ctx, dealloc_td, mlx5_dealloc_td); + verbs_set_ctx_op(v_ctx, alloc_parent_domain, mlx5_alloc_parent_domain); + verbs_set_ctx_op(v_ctx, dealloc_parent_domain, mlx5_dealloc_parent_domain); memset(&device_attr, 0, sizeof(device_attr)); if (!mlx5_query_device_ex(ctx, NULL, &device_attr, diff --git a/providers/mlx5/mlx5.h b/providers/mlx5/mlx5.h index da0aa91..f9b83ad 100644 --- a/providers/mlx5/mlx5.h +++ b/providers/mlx5/mlx5.h @@ -319,6 +319,9 @@ struct mlx5_buf { struct mlx5_pd { struct ibv_pd ibv_pd; uint32_t pdn; + int is_parent_domain; + struct ibv_td *td; + struct ibv_pd *protection_domain; }; struct mlx5_td { @@ -766,6 +769,12 @@ int mlx5_post_srq_ops(struct ibv_srq *srq, struct ibv_td *mlx5_alloc_td(struct ibv_context *context); int mlx5_dealloc_td(struct ibv_td *td); +struct ibv_pd *mlx5_alloc_parent_domain(struct ibv_context *context, + struct ibv_parent_domain_init_attr *attr); + +int mlx5_dealloc_parent_domain(struct ibv_pd *parent_domain); + + static inline void *mlx5_find_uidx(struct mlx5_context *ctx, uint32_t uidx) { int tind = uidx >> MLX5_UIDX_TABLE_SHIFT; diff --git a/providers/mlx5/verbs.c b/providers/mlx5/verbs.c index 8cf14c0..13844b3 100644 --- a/providers/mlx5/verbs.c +++ b/providers/mlx5/verbs.c @@ -196,6 +196,29 @@ int mlx5_dealloc_td(struct ibv_td *ib_td) return 0; } +struct ibv_pd *mlx5_alloc_parent_domain(struct ibv_context *context, + struct ibv_parent_domain_init_attr *attr) +{ + struct mlx5_pd *pd; + + pd = calloc(1, sizeof *pd); + if (!pd) + return NULL; + + pd->is_parent_domain = 1; + pd->td = attr->td; + pd->protection_domain= attr->pd; + + pd->ibv_pd.context = context; + return &pd->ibv_pd; +} + +int mlx5_dealloc_parent_domain(struct ibv_pd *parent_domain) +{ + free(to_mpd(parent_domain)); + return 0; +} + int mlx5_free_pd(struct ibv_pd *pd) { int ret;