From patchwork Fri Nov 3 23:20:53 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 10041357 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 40A58602D8 for ; Fri, 3 Nov 2017 23:21:04 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3269C283AF for ; Fri, 3 Nov 2017 23:21:04 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 271252920A; Fri, 3 Nov 2017 23:21:04 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A089A283AF for ; Fri, 3 Nov 2017 23:21:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752558AbdKCXVB (ORCPT ); Fri, 3 Nov 2017 19:21:01 -0400 Received: from esa3.hgst.iphmx.com ([216.71.153.141]:41264 "EHLO esa3.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752127AbdKCXVA (ORCPT ); Fri, 3 Nov 2017 19:21:00 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1509751261; x=1541287261; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=Zgu66HCU0f1EhTMB6bvBK9zxhcEMUGGscmcwVa21xhI=; b=cQD3tGVZkE2Uj6XjdHPB8iaV4tc+wgc9RAYm9rB86AaNs2cxQkTir1j1 VzoKneTGOp35jDJvRqfvh7YMnWK2C4SPZEP+lHY92bU+Lk1K+0u+NnR09 NIAe8oGeVonWYXycbdb4jNdnV0VX/Am6V5GllwNPZWiG9BhKPjX0x/gaN qt5ccxl348OUoy+m5xO8iyn32UP1iCo2soRf/9JHqLfcLONn4hPawGUNK qKbfk9RcwFce/cuMtlrbt0dRwEaglNZW9D/AlvjuQz2Z6Ph4Cmlc4LzvA 1vP9p3y/xgeo0ppiC8GpHt3uqPhE6WRTXpcHpwnPXnPOEz/WxyDYFE2da Q==; X-IronPort-AV: E=Sophos;i="5.44,339,1505750400"; d="scan'208";a="61365166" Received: from uls-op-cesaip01.wdc.com (HELO uls-op-cesaep01.wdc.com) ([199.255.45.14]) by ob1.hgst.iphmx.com with ESMTP; 04 Nov 2017 07:21:00 +0800 Received: from uls-op-cesaip02.wdc.com ([10.248.3.37]) by uls-op-cesaep01.wdc.com with ESMTP; 03 Nov 2017 16:19:02 -0700 Received: from unknown (HELO MILHUBIP03.sdcorp.global.sandisk.com) ([10.177.9.96]) by uls-op-cesaip02.wdc.com with ESMTP; 03 Nov 2017 16:21:00 -0700 Received: from milsmgip12.sandisk.com (10.177.8.100) by MILHUBIP03.sdcorp.global.sandisk.com (10.177.9.96) with Microsoft SMTP Server id 14.3.319.2; Fri, 3 Nov 2017 16:20:59 -0700 X-AuditID: 0ab1095a-d99e3980000146e4-bc-59fcf9db2642 Received: from thinkpad-bart.int.fusionio.com ( [10.177.9.6]) by (Symantec Messaging Gateway) with SMTP id D7.33.18148.BD9FCF95; Fri, 3 Nov 2017 16:20:59 -0700 (PDT) From: Bart Van Assche To: Doug Ledford CC: , Mike Marciniszyn , Bart Van Assche Subject: [PATCH 2/5] IB/srpt: Introduce helper functions for SRQ allocation and freeing Date: Fri, 3 Nov 2017 16:20:53 -0700 Message-ID: <20171103232056.30614-3-bart.vanassche@wdc.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20171103232056.30614-1-bart.vanassche@wdc.com> References: <20171103232056.30614-1-bart.vanassche@wdc.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprJJMWRmVeSWpSXmKPExsXCtZGTTff2zz+RBvuPSVpc+vOF0eLl+Q+s Fs8O9bJYbPjfyO7A4rF4z0smj/f7rrJ5fN4k59F+oJspgCWKyyYlNSezLLVI3y6BK2Pmph8s BR/VKtY0vGNrYPwp38XIwSEhYCJx+XxJFyMnh5DASkaJn191QWw2AT2JU/P2MYHYIgJqEpte LWIHsZkFqiWu7pnOCGILC4RL7G7eyQZiswioSux73cAMYvMKWEvcmHKWBcSWEJCXWPv2HiPI Kk4BG4nzM8UhVllLLLnZwDqBkXsBI8MqRrHczJzi3PTMAkMjveLEvJTM4my95PzcTYwQz0ft YLw+0fwQIxMHp1QDY43vxCWHrX72RJqX/5ogWnXnUkTLfGaZBsX7kscl9dKYmbkX/L6y/l1Y cGPvaZt3ketV9/D/m75V+/WF+mUuvx4vFE1541cTv3Jdr8eyB9+TOfkWH519JvZw+X92q8O3 3jtbxM5TnnmoaI/ehc89wlPuOj4onn1waUBu/3qVRPdNfpudr9/yPKXEUpyRaKjFXFScCABF x4m7rAEAAA== MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The only functional change in this patch is in the srpt_add_one() error path: if allocating the ring buffer for the SRQ fails, fall back to non-SRQ mode instead of disabling SRP target functionality. Signed-off-by: Bart Van Assche --- drivers/infiniband/ulp/srpt/ib_srpt.c | 115 +++++++++++++++++++++------------- 1 file changed, 72 insertions(+), 43 deletions(-) diff --git a/drivers/infiniband/ulp/srpt/ib_srpt.c b/drivers/infiniband/ulp/srpt/ib_srpt.c index 98b1b80e476b..a5f232e9b1f3 100644 --- a/drivers/infiniband/ulp/srpt/ib_srpt.c +++ b/drivers/infiniband/ulp/srpt/ib_srpt.c @@ -2510,6 +2510,74 @@ static struct se_wwn *srpt_lookup_wwn(const char *name) return wwn; } +static void srpt_free_srq(struct srpt_device *sdev) +{ + if (!sdev->srq) + return; + + ib_destroy_srq(sdev->srq); + srpt_free_ioctx_ring((struct srpt_ioctx **)sdev->ioctx_ring, sdev, + sdev->srq_size, srp_max_req_size, DMA_FROM_DEVICE); + sdev->srq = NULL; +} + +static int srpt_alloc_srq(struct srpt_device *sdev) +{ + struct ib_srq_init_attr srq_attr = { + .event_handler = srpt_srq_event, + .srq_context = (void *)sdev, + .attr.max_wr = sdev->srq_size, + .attr.max_sge = 1, + .srq_type = IB_SRQT_BASIC, + }; + struct ib_device *device = sdev->device; + struct ib_srq *srq; + int i; + + WARN_ON_ONCE(sdev->srq); + srq = ib_create_srq(sdev->pd, &srq_attr); + if (IS_ERR(srq)) { + pr_debug("ib_create_srq() failed: %ld\n", PTR_ERR(srq)); + return PTR_ERR(srq); + } + + pr_debug("create SRQ #wr= %d max_allow=%d dev= %s\n", sdev->srq_size, + sdev->device->attrs.max_srq_wr, device->name); + + sdev->ioctx_ring = (struct srpt_recv_ioctx **) + srpt_alloc_ioctx_ring(sdev, sdev->srq_size, + sizeof(*sdev->ioctx_ring[0]), + srp_max_req_size, DMA_FROM_DEVICE); + if (!sdev->ioctx_ring) { + ib_destroy_srq(srq); + return -ENOMEM; + } + + sdev->use_srq = true; + sdev->srq = srq; + + for (i = 0; i < sdev->srq_size; ++i) + srpt_post_recv(sdev, NULL, sdev->ioctx_ring[i]); + + return 0; +} + +static int srpt_use_srq(struct srpt_device *sdev, bool use_srq) +{ + struct ib_device *device = sdev->device; + int ret = 0; + + if (!use_srq) { + srpt_free_srq(sdev); + sdev->use_srq = false; + } else if (use_srq && !sdev->srq) { + ret = srpt_alloc_srq(sdev); + } + pr_debug("%s(%s): use_srq = %d; ret = %d\n", __func__, device->name, + sdev->use_srq, ret); + return ret; +} + /** * srpt_add_one() - Infiniband device addition callback function. */ @@ -2517,7 +2585,6 @@ static void srpt_add_one(struct ib_device *device) { struct srpt_device *sdev; struct srpt_port *sport; - struct ib_srq_init_attr srq_attr; int i; pr_debug("device = %p\n", device); @@ -2539,38 +2606,7 @@ static void srpt_add_one(struct ib_device *device) sdev->srq_size = min(srpt_srq_size, sdev->device->attrs.max_srq_wr); - srq_attr.event_handler = srpt_srq_event; - srq_attr.srq_context = (void *)sdev; - srq_attr.attr.max_wr = sdev->srq_size; - srq_attr.attr.max_sge = 1; - srq_attr.attr.srq_limit = 0; - srq_attr.srq_type = IB_SRQT_BASIC; - - sdev->srq = sdev->port[0].port_attrib.use_srq ? - ib_create_srq(sdev->pd, &srq_attr) : ERR_PTR(-ENOTSUPP); - if (IS_ERR(sdev->srq)) { - pr_debug("ib_create_srq() failed: %ld\n", PTR_ERR(sdev->srq)); - - /* SRQ not supported. */ - sdev->use_srq = false; - } else { - pr_debug("create SRQ #wr= %d max_allow=%d dev= %s\n", - sdev->srq_size, sdev->device->attrs.max_srq_wr, - device->name); - - sdev->use_srq = true; - - sdev->ioctx_ring = (struct srpt_recv_ioctx **) - srpt_alloc_ioctx_ring(sdev, sdev->srq_size, - sizeof(*sdev->ioctx_ring[0]), - srp_max_req_size, - DMA_FROM_DEVICE); - if (!sdev->ioctx_ring) - goto err_pd; - - for (i = 0; i < sdev->srq_size; ++i) - srpt_post_recv(sdev, NULL, sdev->ioctx_ring[i]); - } + srpt_use_srq(sdev, sdev->port[0].port_attrib.use_srq); if (!srpt_service_guid) srpt_service_guid = be64_to_cpu(device->node_guid); @@ -2630,12 +2666,7 @@ static void srpt_add_one(struct ib_device *device) err_cm: ib_destroy_cm_id(sdev->cm_id); err_ring: - if (sdev->use_srq) - ib_destroy_srq(sdev->srq); - srpt_free_ioctx_ring((struct srpt_ioctx **)sdev->ioctx_ring, sdev, - sdev->srq_size, srp_max_req_size, - DMA_FROM_DEVICE); -err_pd: + srpt_free_srq(sdev); ib_dealloc_pd(sdev->pd); free_dev: kfree(sdev); @@ -2678,10 +2709,8 @@ static void srpt_remove_one(struct ib_device *device, void *client_data) spin_unlock(&srpt_dev_lock); srpt_release_sdev(sdev); - if (sdev->use_srq) - ib_destroy_srq(sdev->srq); - srpt_free_ioctx_ring((struct srpt_ioctx **)sdev->ioctx_ring, sdev, - sdev->srq_size, srp_max_req_size, DMA_FROM_DEVICE); + srpt_free_srq(sdev); + ib_dealloc_pd(sdev->pd); kfree(sdev);