From patchwork Thu Sep 15 14:55:29 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 9333889 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 4B437607FD for ; Thu, 15 Sep 2016 14:55:44 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3F2A2296B9 for ; Thu, 15 Sep 2016 14:55:44 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 33BAC298E6; Thu, 15 Sep 2016 14:55:44 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.3 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI, RCVD_IN_SORBS_SPAM, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C8FE7298F9 for ; Thu, 15 Sep 2016 14:55:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752975AbcIOOzh (ORCPT ); Thu, 15 Sep 2016 10:55:37 -0400 Received: from mail-it0-f67.google.com ([209.85.214.67]:36301 "EHLO mail-it0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753211AbcIOOzb (ORCPT ); Thu, 15 Sep 2016 10:55:31 -0400 Received: by mail-it0-f67.google.com with SMTP id n143so4722169ita.3; Thu, 15 Sep 2016 07:55:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=w5ywxOvQu1eRs2YF9oMrpCxbSsu9rVmFhjNhi3q5XwM=; b=SDNJ+DobLD8SxwKdE1PwUwLCq3f4TK4dDmViEJzkppTkeUpPFjNSZLpYU2Lbz6nNWg 7y53SxeyDiJEWCsB9fKUs+hKqg/9RRCkTmOWmrZ00OAQvPKbk+p5gG7IQ9WaPnV/ziLy LnDJRd338psf64PFoakPIa6Yl619L5Q0Q4OJ9DjWXZ7e1j1zLBKZMId5W6yVknU7jWmq l+zkzgZogwSznMftNtqMWlBbziKtE/7yJl03S8ys+CvvSi7BUCehL/kKgGcPzwumgEHz wYKVpz7sS3ddgU25qewjM917wIAvhpWhUPOUPJvf6ds6rO7rlNAOyn5fNOL5n3ID+8Y9 oRog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=w5ywxOvQu1eRs2YF9oMrpCxbSsu9rVmFhjNhi3q5XwM=; b=UhyjupYsygJPNxvKpu1DnibT+W6QUtW+cyI6AEziO/uAw38q2fo/ScTmy63EHs5h/J USWlBTcO6+UYEziX4EF7DTzClakssr+E5mxw4AONvbXLlcNAqX8xkibuDFJ2SW+vkcK1 SODWGTgUi57JrwROWYueh5lwsINQ5/c7ThgXVfTFHLD9mpiolhO6wrwUfkCI3oXnFXCd F+WAtyVStyljPf8uFioqbEGb23+NegHkkzREfTEM6MY/xgoYDvWUCff8LRoQsX2swTnS kABIvvzLVy/wXZMV0SeUs7vlKjCWwO5WdLb+fziqmOlXfntRz2X0NkudYwHC62MNNYRO 8g/Q== X-Gm-Message-State: AE9vXwPZPUNgZimqaZGN8zmUT6MZVgyp7ApJL4y+s+oKcfmAQAL8mmhwfPRclAbG3Q9zYA== X-Received: by 10.107.7.9 with SMTP id 9mr17057883ioh.96.1473951330337; Thu, 15 Sep 2016 07:55:30 -0700 (PDT) Received: from manet.1015granger.net ([2604:8800:100:81fc:ec4:7aff:fe6c:1dce]) by smtp.gmail.com with ESMTPSA id f14sm1076503ita.10.2016.09.15.07.55.29 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 15 Sep 2016 07:55:29 -0700 (PDT) Subject: [PATCH v3 04/22] SUNRPC: Generalize the RPC buffer release API From: Chuck Lever To: anna.schumaker@netapp.com Cc: linux-rdma@vger.kernel.org, linux-nfs@vger.kernel.org Date: Thu, 15 Sep 2016 10:55:29 -0400 Message-ID: <20160915145528.11080.75215.stgit@manet.1015granger.net> In-Reply-To: <20160915143440.11080.89241.stgit@manet.1015granger.net> References: <20160915143440.11080.89241.stgit@manet.1015granger.net> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP xprtrdma needs to allocate the Call and Reply buffers separately. TBH, the reliance on using a single buffer for the pair of XDR buffers is transport implementation-specific. Instead of passing just the rq_buffer into the buf_free method, pass the task structure and let buf_free take care of freeing both XDR buffers at once. There's a micro-optimization here. In the common case, both xprt_release and the transport's buf_free method were checking if rq_buffer was NULL. Now the check is done only once per RPC. Signed-off-by: Chuck Lever --- include/linux/sunrpc/sched.h | 2 +- include/linux/sunrpc/xprt.h | 2 +- net/sunrpc/sched.c | 10 ++++------ net/sunrpc/xprt.c | 2 +- net/sunrpc/xprtrdma/svc_rdma_backchannel.c | 2 +- net/sunrpc/xprtrdma/transport.c | 26 ++++++++++---------------- net/sunrpc/xprtrdma/xprt_rdma.h | 1 - net/sunrpc/xprtsock.c | 6 ++---- 8 files changed, 20 insertions(+), 31 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/include/linux/sunrpc/sched.h b/include/linux/sunrpc/sched.h index 38d4c1b..7ba040c 100644 --- a/include/linux/sunrpc/sched.h +++ b/include/linux/sunrpc/sched.h @@ -240,7 +240,7 @@ struct rpc_task *rpc_wake_up_first(struct rpc_wait_queue *, void rpc_wake_up_status(struct rpc_wait_queue *, int); void rpc_delay(struct rpc_task *, unsigned long); int rpc_malloc(struct rpc_task *); -void rpc_free(void *); +void rpc_free(struct rpc_task *); int rpciod_up(void); void rpciod_down(void); int __rpc_wait_for_completion_task(struct rpc_task *task, wait_bit_action_f *); diff --git a/include/linux/sunrpc/xprt.h b/include/linux/sunrpc/xprt.h index c01f468..72c2aeb 100644 --- a/include/linux/sunrpc/xprt.h +++ b/include/linux/sunrpc/xprt.h @@ -128,7 +128,7 @@ struct rpc_xprt_ops { void (*set_port)(struct rpc_xprt *xprt, unsigned short port); void (*connect)(struct rpc_xprt *xprt, struct rpc_task *task); int (*buf_alloc)(struct rpc_task *task); - void (*buf_free)(void *buffer); + void (*buf_free)(struct rpc_task *task); int (*send_request)(struct rpc_task *task); void (*set_retrans_timeout)(struct rpc_task *task); void (*timer)(struct rpc_xprt *xprt, struct rpc_task *task); diff --git a/net/sunrpc/sched.c b/net/sunrpc/sched.c index b964d40..6690ebc 100644 --- a/net/sunrpc/sched.c +++ b/net/sunrpc/sched.c @@ -896,18 +896,16 @@ int rpc_malloc(struct rpc_task *task) EXPORT_SYMBOL_GPL(rpc_malloc); /** - * rpc_free - free buffer allocated via rpc_malloc - * @buffer: buffer to free + * rpc_free - free RPC buffer resources allocated via rpc_malloc + * @task: RPC task * */ -void rpc_free(void *buffer) +void rpc_free(struct rpc_task *task) { + void *buffer = task->tk_rqstp->rq_buffer; size_t size; struct rpc_buffer *buf; - if (!buffer) - return; - buf = container_of(buffer, struct rpc_buffer, data); size = buf->len; diff --git a/net/sunrpc/xprt.c b/net/sunrpc/xprt.c index ea244b2..685e6d2 100644 --- a/net/sunrpc/xprt.c +++ b/net/sunrpc/xprt.c @@ -1295,7 +1295,7 @@ void xprt_release(struct rpc_task *task) xprt_schedule_autodisconnect(xprt); spin_unlock_bh(&xprt->transport_lock); if (req->rq_buffer) - xprt->ops->buf_free(req->rq_buffer); + xprt->ops->buf_free(task); xprt_inject_disconnect(xprt); if (req->rq_cred != NULL) put_rpccred(req->rq_cred); diff --git a/net/sunrpc/xprtrdma/svc_rdma_backchannel.c b/net/sunrpc/xprtrdma/svc_rdma_backchannel.c index 124688b..fa89350 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_backchannel.c +++ b/net/sunrpc/xprtrdma/svc_rdma_backchannel.c @@ -186,7 +186,7 @@ xprt_rdma_bc_allocate(struct rpc_task *task) } static void -xprt_rdma_bc_free(void *buffer) +xprt_rdma_bc_free(struct rpc_task *task) { /* No-op: ctxt and page have already been freed. */ } diff --git a/net/sunrpc/xprtrdma/transport.c b/net/sunrpc/xprtrdma/transport.c index daa7d4d..ebf14ba 100644 --- a/net/sunrpc/xprtrdma/transport.c +++ b/net/sunrpc/xprtrdma/transport.c @@ -523,7 +523,6 @@ xprt_rdma_allocate(struct rpc_task *task) out: dprintk("RPC: %s: size %zd, request 0x%p\n", __func__, size, req); req->rl_connect_cookie = 0; /* our reserved value */ - req->rl_task = task; rqst->rq_buffer = req->rl_sendbuf->rg_base; return 0; @@ -571,31 +570,26 @@ out_fail: return -ENOMEM; } -/* - * This function returns all RDMA resources to the pool. +/** + * xprt_rdma_free - release resources allocated by xprt_rdma_allocate + * @task: RPC task + * + * Caller guarantees rqst->rq_buffer is non-NULL. */ static void -xprt_rdma_free(void *buffer) +xprt_rdma_free(struct rpc_task *task) { - struct rpcrdma_req *req; - struct rpcrdma_xprt *r_xprt; - struct rpcrdma_regbuf *rb; - - if (buffer == NULL) - return; + struct rpc_rqst *rqst = task->tk_rqstp; + struct rpcrdma_xprt *r_xprt = rpcx_to_rdmax(rqst->rq_xprt); + struct rpcrdma_req *req = rpcr_to_rdmar(rqst); - rb = container_of(buffer, struct rpcrdma_regbuf, rg_base[0]); - req = rb->rg_owner; if (req->rl_backchannel) return; - r_xprt = container_of(req->rl_buffer, struct rpcrdma_xprt, rx_buf); - dprintk("RPC: %s: called on 0x%p\n", __func__, req->rl_reply); r_xprt->rx_ia.ri_ops->ro_unmap_safe(r_xprt, req, - !RPC_IS_ASYNC(req->rl_task)); - + !RPC_IS_ASYNC(task)); rpcrdma_buffer_put(req); } diff --git a/net/sunrpc/xprtrdma/xprt_rdma.h b/net/sunrpc/xprtrdma/xprt_rdma.h index 9df47c8..4838a85 100644 --- a/net/sunrpc/xprtrdma/xprt_rdma.h +++ b/net/sunrpc/xprtrdma/xprt_rdma.h @@ -283,7 +283,6 @@ struct rpcrdma_req { struct list_head rl_free; unsigned int rl_niovs; unsigned int rl_connect_cookie; - struct rpc_task *rl_task; struct rpcrdma_buffer *rl_buffer; struct rpcrdma_rep *rl_reply;/* holder for reply buffer */ struct ib_sge rl_send_iov[RPCRDMA_MAX_IOVS]; diff --git a/net/sunrpc/xprtsock.c b/net/sunrpc/xprtsock.c index bd30b4b..bde39f2 100644 --- a/net/sunrpc/xprtsock.c +++ b/net/sunrpc/xprtsock.c @@ -2560,13 +2560,11 @@ static int bc_malloc(struct rpc_task *task) /* * Free the space allocated in the bc_alloc routine */ -static void bc_free(void *buffer) +static void bc_free(struct rpc_task *task) { + void *buffer = task->tk_rqstp->rq_buffer; struct rpc_buffer *buf; - if (!buffer) - return; - buf = container_of(buffer, struct rpc_buffer, data); free_page((unsigned long)buf); }