From patchwork Wed Nov 8 09:57:38 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sagi Grimberg X-Patchwork-Id: 10048163 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id B9A8560247 for ; Wed, 8 Nov 2017 09:58:23 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9F4512A51E for ; Wed, 8 Nov 2017 09:58:23 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9432F2A522; Wed, 8 Nov 2017 09:58:23 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.1 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI, T_DKIM_INVALID, URIBL_BLACK autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 10CD7292D6 for ; Wed, 8 Nov 2017 09:58:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751786AbdKHJ6U (ORCPT ); Wed, 8 Nov 2017 04:58:20 -0500 Received: from bombadil.infradead.org ([65.50.211.133]:53404 "EHLO bombadil.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751470AbdKHJ6T (ORCPT ); Wed, 8 Nov 2017 04:58:19 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=References:In-Reply-To:Message-Id: Date:Subject:Cc:To:From:Sender:Reply-To:MIME-Version:Content-Type: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=eGYXe6WUM1E6FIwkAaL7Z2rfWF5kMFEJFzkCgdu4jeQ=; b=BEcvFnglhSh65FC4cre+enNGY UisdEYUjM52gGzgdoWoE++ESyXE3x/8KAIlhg4Wa7+E7TgOq1ZM4ElAvX8P6FgMY1V5ZO5dnzKNZx J91VW4Vs3h3xOn9LZ7q6uXWwHuEuMMT8MdZ19DxHwLX4seir6A+MoP91FiP74SAJ0ljrOvtGh7dEP mNawWNZorXTouOFMp3hQGdbK1X9lceCgJIV/maw861Uq7+Jy5nHuGU1kx4/5IDdBlNpxf+MhPByYI HkDBww07oUVmT/CpOu3tUe3voTdqbxBgVa9uxMMYwQc55wxMBq/AXRFvxT7PvUMJrQn1fUm2SEN3c 1Vdlo4VIA==; Received: from [31.154.58.122] (helo=bombadil.infradead.org) by bombadil.infradead.org with esmtpsa (Exim 4.87 #1 (Red Hat Linux)) id 1eCN7g-0001zj-4D; Wed, 08 Nov 2017 09:58:16 +0000 From: Sagi Grimberg To: linux-rdma@vger.kernel.org Cc: linux-nvme@lists.infradead.org, Christoph Hellwig , Max Gurtuvoy Subject: [PATCH v3 5/9] svcrdma: Use RDMA core implicit CQ allocation Date: Wed, 8 Nov 2017 11:57:38 +0200 Message-Id: <20171108095742.25365-6-sagi@grimberg.me> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20171108095742.25365-1-sagi@grimberg.me> References: <20171108095742.25365-1-sagi@grimberg.me> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Get some of the wisdom of CQ completion vector spreading and CQ queue-pair chunking for free. Signed-off-by: Sagi Grimberg --- include/linux/sunrpc/svc_rdma.h | 2 -- net/sunrpc/xprtrdma/svc_rdma_transport.c | 22 ++-------------------- 2 files changed, 2 insertions(+), 22 deletions(-) diff --git a/include/linux/sunrpc/svc_rdma.h b/include/linux/sunrpc/svc_rdma.h index 995c6fe9ee90..95e0b7a1b311 100644 --- a/include/linux/sunrpc/svc_rdma.h +++ b/include/linux/sunrpc/svc_rdma.h @@ -118,8 +118,6 @@ struct svcxprt_rdma { struct list_head sc_rq_dto_q; spinlock_t sc_rq_dto_lock; struct ib_qp *sc_qp; - struct ib_cq *sc_rq_cq; - struct ib_cq *sc_sq_cq; spinlock_t sc_lock; /* transport lock */ diff --git a/net/sunrpc/xprtrdma/svc_rdma_transport.c b/net/sunrpc/xprtrdma/svc_rdma_transport.c index 5caf8e722a11..d51ead156898 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_transport.c +++ b/net/sunrpc/xprtrdma/svc_rdma_transport.c @@ -780,21 +780,11 @@ static struct svc_xprt *svc_rdma_accept(struct svc_xprt *xprt) dprintk("svcrdma: error creating PD for connect request\n"); goto errout; } - newxprt->sc_sq_cq = ib_alloc_cq(dev, newxprt, newxprt->sc_sq_depth, - 0, IB_POLL_WORKQUEUE); - if (IS_ERR(newxprt->sc_sq_cq)) { - dprintk("svcrdma: error creating SQ CQ for connect request\n"); - goto errout; - } - newxprt->sc_rq_cq = ib_alloc_cq(dev, newxprt, newxprt->sc_rq_depth, - 0, IB_POLL_WORKQUEUE); - if (IS_ERR(newxprt->sc_rq_cq)) { - dprintk("svcrdma: error creating RQ CQ for connect request\n"); - goto errout; - } memset(&qp_attr, 0, sizeof qp_attr); qp_attr.event_handler = qp_event_handler; + qp_attr.create_flags = IB_QP_CREATE_ASSIGN_CQS; + qp_attr.poll_ctx = IB_POLL_WORKQUEUE; qp_attr.qp_context = &newxprt->sc_xprt; qp_attr.port_num = newxprt->sc_port_num; qp_attr.cap.max_rdma_ctxs = ctxts; @@ -804,8 +794,6 @@ static struct svc_xprt *svc_rdma_accept(struct svc_xprt *xprt) qp_attr.cap.max_recv_sge = newxprt->sc_max_sge; qp_attr.sq_sig_type = IB_SIGNAL_REQ_WR; qp_attr.qp_type = IB_QPT_RC; - qp_attr.send_cq = newxprt->sc_sq_cq; - qp_attr.recv_cq = newxprt->sc_rq_cq; dprintk("svcrdma: newxprt->sc_cm_id=%p, newxprt->sc_pd=%p\n", newxprt->sc_cm_id, newxprt->sc_pd); dprintk(" cap.max_send_wr = %d, cap.max_recv_wr = %d\n", @@ -959,12 +947,6 @@ static void __svc_rdma_free(struct work_struct *work) if (rdma->sc_qp && !IS_ERR(rdma->sc_qp)) ib_destroy_qp(rdma->sc_qp); - if (rdma->sc_sq_cq && !IS_ERR(rdma->sc_sq_cq)) - ib_free_cq(rdma->sc_sq_cq); - - if (rdma->sc_rq_cq && !IS_ERR(rdma->sc_rq_cq)) - ib_free_cq(rdma->sc_rq_cq); - if (rdma->sc_pd && !IS_ERR(rdma->sc_pd)) ib_dealloc_pd(rdma->sc_pd);