From patchwork Thu Sep 15 14:56:43 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 9333933 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id BD9C46089F for ; Thu, 15 Sep 2016 14:56:58 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B2CDB298E6 for ; Thu, 15 Sep 2016 14:56:58 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A79F0298EF; Thu, 15 Sep 2016 14:56:58 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.3 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI, RCVD_IN_SORBS_SPAM, T_DKIM_INVALID autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 31C79298E6 for ; Thu, 15 Sep 2016 14:56:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752233AbcIOO4x (ORCPT ); Thu, 15 Sep 2016 10:56:53 -0400 Received: from mail-it0-f67.google.com ([209.85.214.67]:36713 "EHLO mail-it0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752497AbcIOO4q (ORCPT ); Thu, 15 Sep 2016 10:56:46 -0400 Received: by mail-it0-f67.google.com with SMTP id n143so4725475ita.3; Thu, 15 Sep 2016 07:56:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=cQEUphyJXjGi7haragPzvcMJf6rDaqBcgZwksXYk2sk=; b=VwvxA2UEEItCBKZOY1J3367xKEsoyL8KDPq+odshgAq7g1HY45a5Qmku0CBdIKq7Te lZIZeG9tOOcqOogu/gjdxPp55MpISZ9wpyzmZsZpJflOn4YOXoGx/gk0Vkm3HPnjvC77 fOinvVwlouHJtWr+LCdaeQ+6mT22uInAoaYkRu7rcL63S3EDVffdzXfPjdprYFB/GUk6 443tWWhgQSM+MhdMVE5nEvNj0qOOxqvW4M4B+90CAB2uoqWab2wOVFocv4+szQq1o1V7 izS38LmSifQylYpxd5x/aksRp7x0Ch5nt3mMLCriOFRNFYr+Dd2UjwymBh+EHpT494xC u7GQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=cQEUphyJXjGi7haragPzvcMJf6rDaqBcgZwksXYk2sk=; b=KUpV9RDoM9yG65sAfhhf4fBIeWeHRjF25ZnZIFIMD3ALw0ycTrnGF/dlZBGJVVU7bs Y/SZPqCvwSIvYU9iKZtaX/Xbc2xar7hJ1luS4YnT3KYBbUnlmSAmu5olFRcLoylYUwBK Hhu6lzmaaPPCW95bTu/snbFC6YOamReNB7HZ9MWZ/cRdca2u7wi86SLI02jYfs3cBwn2 jrlglnX1otOLYpAzMqu5kQBLsjTxpQR5yJ9pbJPRZzfQ0Lg3ZLYfMO9vVQoSXdG5V8cD 3iaKriO7Pgo0qlj2+mPIz70Oo4rBRGGo4FySDPi4Cd/oMXvkWClwnC53k4+IqioHhLI3 MQ4g== X-Gm-Message-State: AE9vXwP2pQN9wQZkLdq6hhfxxtRBiq+vdrrYG68S6mr8ZB0OGG4jwHp31Xf3bsbd8EHYTw== X-Received: by 10.107.160.83 with SMTP id j80mr17270844ioe.27.1473951404173; Thu, 15 Sep 2016 07:56:44 -0700 (PDT) Received: from manet.1015granger.net ([2604:8800:100:81fc:ec4:7aff:fe6c:1dce]) by smtp.gmail.com with ESMTPSA id e124sm1801987ioe.15.2016.09.15.07.56.43 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 15 Sep 2016 07:56:43 -0700 (PDT) Subject: [PATCH v3 13/22] xprtrdma: Move send_wr to struct rpcrdma_req From: Chuck Lever To: anna.schumaker@netapp.com Cc: linux-rdma@vger.kernel.org, linux-nfs@vger.kernel.org Date: Thu, 15 Sep 2016 10:56:43 -0400 Message-ID: <20160915145643.11080.71281.stgit@manet.1015granger.net> In-Reply-To: <20160915143440.11080.89241.stgit@manet.1015granger.net> References: <20160915143440.11080.89241.stgit@manet.1015granger.net> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Clean up: Most of the fields in each send_wr do not vary. There is no need to initialize them before each ib_post_send(). This removes a large-ish data structure from the stack. Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/backchannel.c | 3 ++- net/sunrpc/xprtrdma/rpc_rdma.c | 5 +++-- net/sunrpc/xprtrdma/verbs.c | 36 +++++++++++++++++------------------- net/sunrpc/xprtrdma/xprt_rdma.h | 4 ++-- 4 files changed, 24 insertions(+), 24 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/net/sunrpc/xprtrdma/backchannel.c b/net/sunrpc/xprtrdma/backchannel.c index 887ef44..61a58f5 100644 --- a/net/sunrpc/xprtrdma/backchannel.c +++ b/net/sunrpc/xprtrdma/backchannel.c @@ -241,7 +241,8 @@ int rpcrdma_bc_marshal_reply(struct rpc_rqst *rqst) req->rl_send_iov[1].length = rpclen; req->rl_send_iov[1].lkey = rdmab_lkey(req->rl_sendbuf); - req->rl_niovs = 2; + req->rl_send_wr.num_sge = 2; + return 0; out_map: diff --git a/net/sunrpc/xprtrdma/rpc_rdma.c b/net/sunrpc/xprtrdma/rpc_rdma.c index 6187cee..c2906e3 100644 --- a/net/sunrpc/xprtrdma/rpc_rdma.c +++ b/net/sunrpc/xprtrdma/rpc_rdma.c @@ -687,7 +687,7 @@ rpcrdma_marshal_req(struct rpc_rqst *rqst) req->rl_send_iov[0].length = hdrlen; req->rl_send_iov[0].lkey = rdmab_lkey(req->rl_rdmabuf); - req->rl_niovs = 1; + req->rl_send_wr.num_sge = 1; if (rtype == rpcrdma_areadch) return 0; @@ -697,7 +697,8 @@ rpcrdma_marshal_req(struct rpc_rqst *rqst) req->rl_send_iov[1].length = rpclen; req->rl_send_iov[1].lkey = rdmab_lkey(req->rl_sendbuf); - req->rl_niovs = 2; + req->rl_send_wr.num_sge = 2; + return 0; out_overflow: diff --git a/net/sunrpc/xprtrdma/verbs.c b/net/sunrpc/xprtrdma/verbs.c index 7b189fe..79a6346 100644 --- a/net/sunrpc/xprtrdma/verbs.c +++ b/net/sunrpc/xprtrdma/verbs.c @@ -849,6 +849,10 @@ rpcrdma_create_req(struct rpcrdma_xprt *r_xprt) req->rl_cqe.done = rpcrdma_wc_send; req->rl_buffer = &r_xprt->rx_buf; INIT_LIST_HEAD(&req->rl_registered); + req->rl_send_wr.next = NULL; + req->rl_send_wr.wr_cqe = &req->rl_cqe; + req->rl_send_wr.sg_list = req->rl_send_iov; + req->rl_send_wr.opcode = IB_WR_SEND; return req; } @@ -1128,7 +1132,7 @@ rpcrdma_buffer_put(struct rpcrdma_req *req) struct rpcrdma_buffer *buffers = req->rl_buffer; struct rpcrdma_rep *rep = req->rl_reply; - req->rl_niovs = 0; + req->rl_send_wr.num_sge = 0; req->rl_reply = NULL; spin_lock(&buffers->rb_lock); @@ -1259,38 +1263,32 @@ rpcrdma_ep_post(struct rpcrdma_ia *ia, struct rpcrdma_req *req) { struct ib_device *device = ia->ri_device; - struct ib_send_wr send_wr, *send_wr_fail; - struct rpcrdma_rep *rep = req->rl_reply; - struct ib_sge *iov = req->rl_send_iov; + struct ib_send_wr *send_wr = &req->rl_send_wr; + struct ib_send_wr *send_wr_fail; + struct ib_sge *sge = req->rl_send_iov; int i, rc; - if (rep) { - rc = rpcrdma_ep_post_recv(ia, rep); + if (req->rl_reply) { + rc = rpcrdma_ep_post_recv(ia, req->rl_reply); if (rc) return rc; req->rl_reply = NULL; } - send_wr.next = NULL; - send_wr.wr_cqe = &req->rl_cqe; - send_wr.sg_list = iov; - send_wr.num_sge = req->rl_niovs; - send_wr.opcode = IB_WR_SEND; - - for (i = 0; i < send_wr.num_sge; i++) - ib_dma_sync_single_for_device(device, iov[i].addr, - iov[i].length, DMA_TO_DEVICE); + for (i = 0; i < send_wr->num_sge; i++) + ib_dma_sync_single_for_device(device, sge[i].addr, + sge[i].length, DMA_TO_DEVICE); dprintk("RPC: %s: posting %d s/g entries\n", - __func__, send_wr.num_sge); + __func__, send_wr->num_sge); if (DECR_CQCOUNT(ep) > 0) - send_wr.send_flags = 0; + send_wr->send_flags = 0; else { /* Provider must take a send completion every now and then */ INIT_CQCOUNT(ep); - send_wr.send_flags = IB_SEND_SIGNALED; + send_wr->send_flags = IB_SEND_SIGNALED; } - rc = ib_post_send(ia->ri_id->qp, &send_wr, &send_wr_fail); + rc = ib_post_send(ia->ri_id->qp, send_wr, &send_wr_fail); if (rc) goto out_postsend_err; return 0; diff --git a/net/sunrpc/xprtrdma/xprt_rdma.h b/net/sunrpc/xprtrdma/xprt_rdma.h index decd134..3c5a89a 100644 --- a/net/sunrpc/xprtrdma/xprt_rdma.h +++ b/net/sunrpc/xprtrdma/xprt_rdma.h @@ -284,10 +284,10 @@ struct rpcrdma_mr_seg { /* chunk descriptors */ struct rpcrdma_buffer; struct rpcrdma_req { struct list_head rl_free; - unsigned int rl_niovs; unsigned int rl_connect_cookie; struct rpcrdma_buffer *rl_buffer; - struct rpcrdma_rep *rl_reply;/* holder for reply buffer */ + struct rpcrdma_rep *rl_reply; + struct ib_send_wr rl_send_wr; struct ib_sge rl_send_iov[RPCRDMA_MAX_IOVS]; struct rpcrdma_regbuf *rl_rdmabuf; /* xprt header */ struct rpcrdma_regbuf *rl_sendbuf; /* rq_snd_buf */