From patchwork Thu Jan 10 14:23:35 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gal Pressman X-Patchwork-Id: 10755931 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C002F13B5 for ; Thu, 10 Jan 2019 14:23:52 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B104E299A4 for ; Thu, 10 Jan 2019 14:23:52 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A57E2298CF; Thu, 10 Jan 2019 14:23:52 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2D25A299BE for ; Thu, 10 Jan 2019 14:23:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728322AbfAJOXw (ORCPT ); Thu, 10 Jan 2019 09:23:52 -0500 Received: from smtp-fw-9101.amazon.com ([207.171.184.25]:40865 "EHLO smtp-fw-9101.amazon.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727723AbfAJOXv (ORCPT ); Thu, 10 Jan 2019 09:23:51 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1547130230; x=1578666230; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=gMRlj5dYM4GR+HCDIlMEcZoYEwKlipbwDvMbqrBPZD4=; b=LUQyJNtHs7oR/it8G86o4MESCudZ33gKoGYMC7NXgrWVXYn2aT+q7DvU S2FgntbCQ96ShZj1QhpRSl2q7prI2s3gdMzPgiqPRPbjwbzvzzlcmWs9b Ow4XlRv55lEyPQJqLRAq1XdtztYFJ2fwf8MGy/mYbios+o2020K1kioID k=; X-IronPort-AV: E=Sophos;i="5.56,253,1539648000"; d="scan'208";a="780782394" Received: from sea3-co-svc-lb6-vlan3.sea.amazon.com (HELO email-inbound-relay-1a-821c648d.us-east-1.amazon.com) ([10.47.22.38]) by smtp-border-fw-out-9101.sea19.amazon.com with ESMTP/TLS/DHE-RSA-AES256-SHA; 10 Jan 2019 14:23:47 +0000 Received: from EX13MTAUWC001.ant.amazon.com (iad55-ws-svc-p15-lb9-vlan2.iad.amazon.com [10.40.159.162]) by email-inbound-relay-1a-821c648d.us-east-1.amazon.com (8.14.7/8.14.7) with ESMTP id x0AENfSN050362 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=FAIL); Thu, 10 Jan 2019 14:23:45 GMT Received: from EX13D19UWC003.ant.amazon.com (10.43.162.184) by EX13MTAUWC001.ant.amazon.com (10.43.162.135) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Thu, 10 Jan 2019 14:23:44 +0000 Received: from EX13MTAUWC001.ant.amazon.com (10.43.162.135) by EX13D19UWC003.ant.amazon.com (10.43.162.184) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Thu, 10 Jan 2019 14:23:44 +0000 Received: from galpress-VirtualBox.hfa16.amazon.com (10.218.62.29) by mail-relay.amazon.com (10.43.162.232) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Thu, 10 Jan 2019 14:23:41 +0000 From: Gal Pressman To: Jason Gunthorpe , Doug Ledford CC: , Alexander Matushevsky , Yossi Leybovich , Dave Goodell , "Brian Barrett" , Leah Shalev , Sean Hefty , Sagi Grimberg , Leon Romanovsky , Gal Pressman Subject: [PATCH rdma-next v5 1/2] RDMA: Add indication for in kernel API support to IB device Date: Thu, 10 Jan 2019 16:23:35 +0200 Message-ID: <1547130216-12663-2-git-send-email-galpress@amazon.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1547130216-12663-1-git-send-email-galpress@amazon.com> References: <1547130216-12663-1-git-send-email-galpress@amazon.com> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Drivers that do not provide kernel verbs support should not be used by ib kernel clients and fail. In case a device does not implement all mandatory verbs for kverbs usage mark it as a non kverbs provider and prevent its usage for all clients except for uverbs. The device is marked as a non kverbs provider using the 'kverbs_provider' flag which should only be set by the core code. The clients can choose whether kverbs are requested for its usage using the 'no_kverbs_req' flag which is currently set for uverbs only. This patch allows drivers to remove mandatory verbs stubs and simply set the callback to NULL. The IB device will be registered as a non-kverbs provider. Signed-off-by: Gal Pressman --- drivers/infiniband/core/device.c | 35 ++++++++++++++++++++++------------- drivers/infiniband/core/uverbs_main.c | 1 + include/rdma/ib_verbs.h | 5 +++++ 3 files changed, 28 insertions(+), 13 deletions(-) diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c index 8872453e26c0..3f32d05df55d 100644 --- a/drivers/infiniband/core/device.c +++ b/drivers/infiniband/core/device.c @@ -121,13 +121,12 @@ static int ib_device_check_mandatory(struct ib_device *device) }; int i; + device->kverbs_provider = true; for (i = 0; i < ARRAY_SIZE(mandatory_table); ++i) { if (!*(void **) ((void *) &device->ops + mandatory_table[i].offset)) { - dev_warn(&device->dev, - "Device is missing mandatory function %s\n", - mandatory_table[i].name); - return -EINVAL; + device->kverbs_provider = false; + break; } } @@ -330,6 +329,9 @@ static int add_client_context(struct ib_device *device, struct ib_client *client { struct ib_client_data *context; + if (!device->kverbs_provider && !client->no_kverbs_req) + return -EOPNOTSUPP; + context = kmalloc(sizeof(*context), GFP_KERNEL); if (!context) return -ENOMEM; @@ -374,10 +376,12 @@ static int read_port_immutable(struct ib_device *device) return -ENOMEM; for (port = start_port; port <= end_port; ++port) { - ret = device->ops.get_port_immutable( - device, port, &device->port_immutable[port]); - if (ret) - return ret; + if (device->ops.get_port_immutable) { + ret = device->ops.get_port_immutable( + device, port, &device->port_immutable[port]); + if (ret) + return ret; + } if (verify_immutable(device, port)) return -EINVAL; @@ -537,11 +541,13 @@ static int setup_device(struct ib_device *device) } memset(&device->attrs, 0, sizeof(device->attrs)); - ret = device->ops.query_device(device, &device->attrs, &uhw); - if (ret) { - dev_warn(&device->dev, - "Couldn't query the device attributes\n"); - goto port_cleanup; + if (device->ops.query_device) { + ret = device->ops.query_device(device, &device->attrs, &uhw); + if (ret) { + dev_warn(&device->dev, + "Couldn't query the device attributes\n"); + goto port_cleanup; + } } ret = setup_port_pkey_list(device); @@ -920,6 +926,9 @@ int ib_query_port(struct ib_device *device, union ib_gid gid; int err; + if (!device->ops.query_port) + return -EOPNOTSUPP; + if (!rdma_is_port_valid(device, port_num)) return -EINVAL; diff --git a/drivers/infiniband/core/uverbs_main.c b/drivers/infiniband/core/uverbs_main.c index fb0007aa0c27..0eafee9a2ffc 100644 --- a/drivers/infiniband/core/uverbs_main.c +++ b/drivers/infiniband/core/uverbs_main.c @@ -1127,6 +1127,7 @@ static const struct file_operations uverbs_mmap_fops = { static struct ib_client uverbs_client = { .name = "uverbs", + .no_kverbs_req = true, .add = ib_uverbs_add_one, .remove = ib_uverbs_remove_one }; diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h index c073a4720d28..fb6074d2e394 100644 --- a/include/rdma/ib_verbs.h +++ b/include/rdma/ib_verbs.h @@ -2566,6 +2566,8 @@ struct ib_device { __be64 node_guid; u32 local_dma_lkey; u16 is_switch:1; + /* Indicates kernel verbs support, should not be used in drivers */ + u8 kverbs_provider:1; u8 node_type; u8 phys_port_cnt; struct ib_device_attr attrs; @@ -2620,6 +2622,9 @@ struct ib_client { const struct sockaddr *addr, void *client_data); struct list_head list; + + /* kverbs are not required by the client */ + u8 no_kverbs_req:1; }; struct ib_device *ib_alloc_device(size_t size); From patchwork Thu Jan 10 14:23:36 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gal Pressman X-Patchwork-Id: 10755933 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 85EF86C2 for ; Thu, 10 Jan 2019 14:24:06 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 75F18299AF for ; Thu, 10 Jan 2019 14:24:06 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 73DDE299B9; Thu, 10 Jan 2019 14:24:06 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 38DDE299CF for ; Thu, 10 Jan 2019 14:24:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728790AbfAJOYE (ORCPT ); Thu, 10 Jan 2019 09:24:04 -0500 Received: from smtp-fw-6002.amazon.com ([52.95.49.90]:39506 "EHLO smtp-fw-6002.amazon.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728355AbfAJOYE (ORCPT ); Thu, 10 Jan 2019 09:24:04 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1547130243; x=1578666243; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=zPFpjlQacErIJTiU63cUc9+In4GW97TryqbXbSOlgf4=; b=W/4MY/2H7CAs18D1yeHTJFVs2eF57siJ1ReY/bBjt6DsNEb28I15+SID PWT0Ww59S2z1dICcgXqHy+ZG9mCvWSgFZekzTkgRdUtjMMYJCMoiXZJzm Mo4CyI7x5IPq8YUVnvowTocrPy6rH90MPHIlQ71gUGEAbQT0pnVjwnIvO 4=; X-IronPort-AV: E=Sophos;i="5.56,253,1539648000"; d="scan'208";a="381076795" Received: from iad6-co-svc-p1-lb1-vlan3.amazon.com (HELO email-inbound-relay-1d-2c665b5d.us-east-1.amazon.com) ([10.124.125.6]) by smtp-border-fw-out-6002.iad6.amazon.com with ESMTP/TLS/DHE-RSA-AES256-SHA; 10 Jan 2019 14:24:02 +0000 Received: from EX13MTAUWC001.ant.amazon.com (iad55-ws-svc-p15-lb9-vlan2.iad.amazon.com [10.40.159.162]) by email-inbound-relay-1d-2c665b5d.us-east-1.amazon.com (8.14.7/8.14.7) with ESMTP id x0AENwQO077356 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=FAIL); Thu, 10 Jan 2019 14:24:00 GMT Received: from EX13D19UWC001.ant.amazon.com (10.43.162.64) by EX13MTAUWC001.ant.amazon.com (10.43.162.135) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Thu, 10 Jan 2019 14:23:48 +0000 Received: from EX13MTAUWC001.ant.amazon.com (10.43.162.135) by EX13D19UWC001.ant.amazon.com (10.43.162.64) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Thu, 10 Jan 2019 14:23:48 +0000 Received: from galpress-VirtualBox.hfa16.amazon.com (10.218.62.29) by mail-relay.amazon.com (10.43.162.232) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Thu, 10 Jan 2019 14:23:44 +0000 From: Gal Pressman To: Jason Gunthorpe , Doug Ledford CC: , Alexander Matushevsky , Yossi Leybovich , Dave Goodell , "Brian Barrett" , Leah Shalev , Sean Hefty , Sagi Grimberg , Leon Romanovsky , Gal Pressman , Christian Benvenuti , Nelson Escobar , Parvi Kaustubhi Subject: [PATCH rdma-next v5 2/2] IB/usnic: Remove stub functions Date: Thu, 10 Jan 2019 16:23:36 +0200 Message-ID: <1547130216-12663-3-git-send-email-galpress@amazon.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1547130216-12663-1-git-send-email-galpress@amazon.com> References: <1547130216-12663-1-git-send-email-galpress@amazon.com> MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Lack of mandatory verbs no longer fail device registration, the device will be marked as a non-kverbs provider. Cc: Christian Benvenuti Cc: Nelson Escobar Cc: Parvi Kaustubhi Signed-off-by: Gal Pressman --- drivers/infiniband/hw/usnic/usnic_ib_main.c | 7 ---- drivers/infiniband/hw/usnic/usnic_ib_verbs.c | 53 ---------------------------- drivers/infiniband/hw/usnic/usnic_ib_verbs.h | 15 -------- 3 files changed, 75 deletions(-) diff --git a/drivers/infiniband/hw/usnic/usnic_ib_main.c b/drivers/infiniband/hw/usnic/usnic_ib_main.c index b2323a52a0dd..cc68f3d757eb 100644 --- a/drivers/infiniband/hw/usnic/usnic_ib_main.c +++ b/drivers/infiniband/hw/usnic/usnic_ib_main.c @@ -333,32 +333,25 @@ static void usnic_get_dev_fw_str(struct ib_device *device, char *str) static const struct ib_device_ops usnic_dev_ops = { .alloc_pd = usnic_ib_alloc_pd, .alloc_ucontext = usnic_ib_alloc_ucontext, - .create_ah = usnic_ib_create_ah, .create_cq = usnic_ib_create_cq, .create_qp = usnic_ib_create_qp, .dealloc_pd = usnic_ib_dealloc_pd, .dealloc_ucontext = usnic_ib_dealloc_ucontext, .dereg_mr = usnic_ib_dereg_mr, - .destroy_ah = usnic_ib_destroy_ah, .destroy_cq = usnic_ib_destroy_cq, .destroy_qp = usnic_ib_destroy_qp, .get_dev_fw_str = usnic_get_dev_fw_str, - .get_dma_mr = usnic_ib_get_dma_mr, .get_link_layer = usnic_ib_port_link_layer, .get_netdev = usnic_get_netdev, .get_port_immutable = usnic_port_immutable, .mmap = usnic_ib_mmap, .modify_qp = usnic_ib_modify_qp, - .poll_cq = usnic_ib_poll_cq, - .post_recv = usnic_ib_post_recv, - .post_send = usnic_ib_post_send, .query_device = usnic_ib_query_device, .query_gid = usnic_ib_query_gid, .query_pkey = usnic_ib_query_pkey, .query_port = usnic_ib_query_port, .query_qp = usnic_ib_query_qp, .reg_user_mr = usnic_ib_reg_mr, - .req_notify_cq = usnic_ib_req_notify_cq, }; /* Start of PF discovery section */ diff --git a/drivers/infiniband/hw/usnic/usnic_ib_verbs.c b/drivers/infiniband/hw/usnic/usnic_ib_verbs.c index 1d4abef17e38..53c9e9b8b5c9 100644 --- a/drivers/infiniband/hw/usnic/usnic_ib_verbs.c +++ b/drivers/infiniband/hw/usnic/usnic_ib_verbs.c @@ -760,57 +760,4 @@ int usnic_ib_mmap(struct ib_ucontext *context, return -EINVAL; } -/* In ib callbacks section - Start of stub funcs */ -struct ib_ah *usnic_ib_create_ah(struct ib_pd *pd, - struct rdma_ah_attr *ah_attr, - u32 flags, - struct ib_udata *udata) - -{ - usnic_dbg("\n"); - return ERR_PTR(-EPERM); -} - -int usnic_ib_destroy_ah(struct ib_ah *ah, u32 flags) -{ - usnic_dbg("\n"); - return -EINVAL; -} - -int usnic_ib_post_send(struct ib_qp *ibqp, const struct ib_send_wr *wr, - const struct ib_send_wr **bad_wr) -{ - usnic_dbg("\n"); - return -EINVAL; -} - -int usnic_ib_post_recv(struct ib_qp *ibqp, const struct ib_recv_wr *wr, - const struct ib_recv_wr **bad_wr) -{ - usnic_dbg("\n"); - return -EINVAL; -} - -int usnic_ib_poll_cq(struct ib_cq *ibcq, int num_entries, - struct ib_wc *wc) -{ - usnic_dbg("\n"); - return -EINVAL; -} - -int usnic_ib_req_notify_cq(struct ib_cq *cq, - enum ib_cq_notify_flags flags) -{ - usnic_dbg("\n"); - return -EINVAL; -} - -struct ib_mr *usnic_ib_get_dma_mr(struct ib_pd *pd, int acc) -{ - usnic_dbg("\n"); - return ERR_PTR(-ENOMEM); -} - - -/* In ib callbacks section - End of stub funcs */ /* End of ib callbacks section */ diff --git a/drivers/infiniband/hw/usnic/usnic_ib_verbs.h b/drivers/infiniband/hw/usnic/usnic_ib_verbs.h index e33144261b9a..99a6d81c2bcd 100644 --- a/drivers/infiniband/hw/usnic/usnic_ib_verbs.h +++ b/drivers/infiniband/hw/usnic/usnic_ib_verbs.h @@ -75,19 +75,4 @@ struct ib_ucontext *usnic_ib_alloc_ucontext(struct ib_device *ibdev, int usnic_ib_dealloc_ucontext(struct ib_ucontext *ibcontext); int usnic_ib_mmap(struct ib_ucontext *context, struct vm_area_struct *vma); -struct ib_ah *usnic_ib_create_ah(struct ib_pd *pd, - struct rdma_ah_attr *ah_attr, - u32 flags, - struct ib_udata *udata); - -int usnic_ib_destroy_ah(struct ib_ah *ah, u32 flags); -int usnic_ib_post_send(struct ib_qp *ibqp, const struct ib_send_wr *wr, - const struct ib_send_wr **bad_wr); -int usnic_ib_post_recv(struct ib_qp *ibqp, const struct ib_recv_wr *wr, - const struct ib_recv_wr **bad_wr); -int usnic_ib_poll_cq(struct ib_cq *ibcq, int num_entries, - struct ib_wc *wc); -int usnic_ib_req_notify_cq(struct ib_cq *cq, - enum ib_cq_notify_flags flags); -struct ib_mr *usnic_ib_get_dma_mr(struct ib_pd *pd, int acc); #endif /* !USNIC_IB_VERBS_H */