From patchwork Mon Mar 27 13:48:19 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 9646797 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 94905601D7 for ; Mon, 27 Mar 2017 13:59:47 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5B88928449 for ; Mon, 27 Mar 2017 13:59:47 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3794728448; Mon, 27 Mar 2017 13:59:47 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.3 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI, RCVD_IN_SORBS_SPAM, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5637428418 for ; Mon, 27 Mar 2017 13:59:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752297AbdC0N5r (ORCPT ); Mon, 27 Mar 2017 09:57:47 -0400 Received: from mail-it0-f65.google.com ([209.85.214.65]:33044 "EHLO mail-it0-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751124AbdC0N5S (ORCPT ); Mon, 27 Mar 2017 09:57:18 -0400 Received: by mail-it0-f65.google.com with SMTP id 76so11525223itj.0; Mon, 27 Mar 2017 06:56:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=Sx9IjdVfe/K75zdAto/IymG4WBn83Ia8iINQi+mJOnY=; b=QRmWy4ScOL0gSBUQ1EgHahwEPHzofXMJAeAi/4HZM6HXcD0lf5f/8gKxH1h4U97Ewi 7ucDrfqspCJDzFJ3oZEyVpRtMKh+h9VNL4wrXhHQzlxYcUMfhJhz4pnIiUaGa1o5UGEn sAevqBsIDDWkVuP6nIQ9CpoyuzNOqbaljztZCBsyqzWsEzdVREdC1w6bdzUQ19l1PpuN bX2RzFOs0df90Cyckan7I8oIEbnvch/pR7Ule2NjK/aTHPq1u+7KE6iEfukwHbQZAxW1 pMmO3PkDA2Ke5xmxYNCoXzHObvRTEJ8ZUuKu4+vKQKrO+kgp3iXiv2IfANd70gQuPPFA pBiw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=Sx9IjdVfe/K75zdAto/IymG4WBn83Ia8iINQi+mJOnY=; b=iUMTrOygMBBuZT9cxKLA4w20hEpZmRzmQH7flpJUWXjgCwwJcYTNEs/b0qrHtg2QeB cTFQbZfvWFDE6JZNs4h6PMG7OOsTV/OZTlR208G77LWeV0e1FMEgcVB/kwZ/jNFBa+FF yM9A5cPqyBl4PB2uN7U5TB6CdrUAOxBNVfwQUnjXpCv0MhJ5+cMhYJap5nei7xM18PZ6 PA90z1A212dH8CJVzEPRjG2pXkW7KUIIIS4wRtanb8HeJRvnsPScNgfT50XSXr+T1dDN eoOrYTuHIRSXikOrWOE294HBz9OMsTXspMZBt5FiBwnDhlHrq00gScEFQ0uBNJNRF16g UzGg== X-Gm-Message-State: AFeK/H0vey5SPLSts6PLxN+ywy2Mfu9EcO5gc2/vW2QN2XbHdXtDzAvsZoC4MwJxrL4MLg== X-Received: by 10.107.136.96 with SMTP id k93mr20418269iod.20.1490622500162; Mon, 27 Mar 2017 06:48:20 -0700 (PDT) Received: from klimt.1015granger.net (c-68-46-169-226.hsd1.mi.comcast.net. [68.46.169.226]) by smtp.gmail.com with ESMTPSA id x26sm269067ioi.5.2017.03.27.06.48.19 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 27 Mar 2017 06:48:19 -0700 (PDT) Subject: [PATCH v2 03/13] svcrdma: Eliminate RPCRDMA_SQ_DEPTH_MULT From: Chuck Lever To: linux-rdma@vger.kernel.org, linux-nfs@vger.kernel.org Date: Mon, 27 Mar 2017 09:48:19 -0400 Message-ID: <20170327134819.5585.90071.stgit@klimt.1015granger.net> In-Reply-To: <20170327132329.5585.98465.stgit@klimt.1015granger.net> References: <20170327132329.5585.98465.stgit@klimt.1015granger.net> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The Send Queue depth is temporarily reduced to 1 SQE per credit. The new rdma_rw API does an internal computation, during QP creation, to increase the depth of the Send Queue to handle RDMA Read and Write operations. This change has to come before the NFSD code paths are updated to use the rdma_rw API. Without this patch, rdma_rw_init_qp() increases the size of the SQ too much, resulting in memory allocation failures during QP creation. Signed-off-by: Chuck Lever Reviewed-by: Christoph Hellwig --- include/linux/sunrpc/svc_rdma.h | 1 - net/sunrpc/xprtrdma/svc_rdma.c | 2 -- net/sunrpc/xprtrdma/svc_rdma_transport.c | 2 +- 3 files changed, 1 insertion(+), 4 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/include/linux/sunrpc/svc_rdma.h b/include/linux/sunrpc/svc_rdma.h index ac05495..f066349 100644 --- a/include/linux/sunrpc/svc_rdma.h +++ b/include/linux/sunrpc/svc_rdma.h @@ -182,7 +182,6 @@ struct svcxprt_rdma { /* The default ORD value is based on two outstanding full-size writes with a * page size of 4k, or 32k * 2 ops / 4k = 16 outstanding RDMA_READ. */ #define RPCRDMA_ORD (64/4) -#define RPCRDMA_SQ_DEPTH_MULT 8 #define RPCRDMA_MAX_REQUESTS 32 #define RPCRDMA_MAX_REQ_SIZE 4096 diff --git a/net/sunrpc/xprtrdma/svc_rdma.c b/net/sunrpc/xprtrdma/svc_rdma.c index c846ca9..9124441 100644 --- a/net/sunrpc/xprtrdma/svc_rdma.c +++ b/net/sunrpc/xprtrdma/svc_rdma.c @@ -247,8 +247,6 @@ int svc_rdma_init(void) dprintk("SVCRDMA Module Init, register RPC RDMA transport\n"); dprintk("\tsvcrdma_ord : %d\n", svcrdma_ord); dprintk("\tmax_requests : %u\n", svcrdma_max_requests); - dprintk("\tsq_depth : %u\n", - svcrdma_max_requests * RPCRDMA_SQ_DEPTH_MULT); dprintk("\tmax_bc_requests : %u\n", svcrdma_max_bc_requests); dprintk("\tmax_inline : %d\n", svcrdma_max_req_size); diff --git a/net/sunrpc/xprtrdma/svc_rdma_transport.c b/net/sunrpc/xprtrdma/svc_rdma_transport.c index c13a5c3..b84cd53 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_transport.c +++ b/net/sunrpc/xprtrdma/svc_rdma_transport.c @@ -1013,7 +1013,7 @@ static struct svc_xprt *svc_rdma_accept(struct svc_xprt *xprt) svcrdma_max_bc_requests); newxprt->sc_rq_depth = newxprt->sc_max_requests + newxprt->sc_max_bc_requests; - newxprt->sc_sq_depth = RPCRDMA_SQ_DEPTH_MULT * newxprt->sc_rq_depth; + newxprt->sc_sq_depth = newxprt->sc_rq_depth; atomic_set(&newxprt->sc_sq_avail, newxprt->sc_sq_depth); if (!svc_rdma_prealloc_ctxts(newxprt))