From patchwork Mon Aug 3 17:03:58 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 6931481 Return-Path: X-Original-To: patchwork-linux-rdma@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 7A3A9C05AC for ; Mon, 3 Aug 2015 17:04:23 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 1EFDA203A0 for ; Mon, 3 Aug 2015 17:04:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 88B6C20394 for ; Mon, 3 Aug 2015 17:04:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754009AbbHCREE (ORCPT ); Mon, 3 Aug 2015 13:04:04 -0400 Received: from mail-qg0-f42.google.com ([209.85.192.42]:33614 "EHLO mail-qg0-f42.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754732AbbHCREC (ORCPT ); Mon, 3 Aug 2015 13:04:02 -0400 Received: by qged69 with SMTP id d69so92924422qge.0; Mon, 03 Aug 2015 10:04:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-type:content-transfer-encoding; bh=K1f7WcQ19BfogtSfZtKcLok9oNhcE3WsaSGu79vdGWA=; b=QGz5ODa0vFSZX8n/r+w//XAkbe6kgFlKTFFESg//Y3DtqtJN06zqqoXfN5t7/B2BQV mHIXGDQ2vYynWjwQouojBYIo/UZbuZRV9U54LGg5wREjaZ7kEWrT08yUz1a5XXZU6bnx imtmOPEbD2lPxQPrcEx74UFPZ7gTwnnsJAl1XuP3mtaFvGKvU+wU22TrlergVglMf5bI 33R1k9ABrFIdYbOS//Qitb1hf67N/0LFZAWzoc6egR41VqCuh5GLS1X9ejp7fTonHi8n E5G9xw1p0lJV4ooOqwwaUrwL6ETWpGBqwAcxl0ktWM9MzoWI0OdTZZFRJfeQ63+naC8K 7xqw== X-Received: by 10.140.235.18 with SMTP id g18mr27413323qhc.103.1438621441025; Mon, 03 Aug 2015 10:04:01 -0700 (PDT) Received: from manet.1015granger.net ([2604:8800:100:81fc:82ee:73ff:fe43:d64f]) by smtp.gmail.com with ESMTPSA id a5sm7151424qga.39.2015.08.03.10.03.59 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 03 Aug 2015 10:04:00 -0700 (PDT) Subject: [PATCH v4 09/16] xprtrdma: Always provide a write list when sending NFS READ From: Chuck Lever To: linux-rdma@vger.kernel.org, linux-nfs@vger.kernel.org Date: Mon, 03 Aug 2015 13:03:58 -0400 Message-ID: <20150803170358.9115.81548.stgit@manet.1015granger.net> In-Reply-To: <20150803165807.9115.23842.stgit@manet.1015granger.net> References: <20150803165807.9115.23842.stgit@manet.1015granger.net> User-Agent: StGit/0.17.1-3-g7d0f MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Spam-Status: No, score=-7.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,RP_MATCHES_RCVD,T_DKIM_INVALID,UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The client has been setting up a reply chunk for NFS READs that are smaller than the inline threshold. This is not efficient: both the server and client CPUs have to copy the reply's data payload into and out of the memory region that is then transferred via RDMA. Using the write list, the data payload is moved by the device and no extra data copying is necessary. Signed-off-by: Chuck Lever Reviewed-by: Devesh Sharma Reviewed-By: Sagi Grimberg Tested-by: Devesh Sharma --- net/sunrpc/xprtrdma/rpc_rdma.c | 21 ++++----------------- 1 file changed, 4 insertions(+), 17 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/net/sunrpc/xprtrdma/rpc_rdma.c b/net/sunrpc/xprtrdma/rpc_rdma.c index 950b654..e7cf976 100644 --- a/net/sunrpc/xprtrdma/rpc_rdma.c +++ b/net/sunrpc/xprtrdma/rpc_rdma.c @@ -418,28 +418,15 @@ rpcrdma_marshal_req(struct rpc_rqst *rqst) /* * Chunks needed for results? * + * o Read ops return data as write chunk(s), header as inline. * o If the expected result is under the inline threshold, all ops * return as inline (but see later). * o Large non-read ops return as a single reply chunk. - * o Large read ops return data as write chunk(s), header as inline. - * - * Note: the NFS code sending down multiple result segments implies - * the op is one of read, readdir[plus], readlink or NFSv4 getacl. - */ - - /* - * This code can handle read chunks, write chunks OR reply - * chunks -- only one type. If the request is too big to fit - * inline, then we will choose read chunks. If the request is - * a READ, then use write chunks to separate the file data - * into pages; otherwise use reply chunks. */ - if (rpcrdma_results_inline(rqst)) - wtype = rpcrdma_noch; - else if (rqst->rq_rcv_buf.page_len == 0) - wtype = rpcrdma_replych; - else if (rqst->rq_rcv_buf.flags & XDRBUF_READ) + if (rqst->rq_rcv_buf.flags & XDRBUF_READ) wtype = rpcrdma_writech; + else if (rpcrdma_results_inline(rqst)) + wtype = rpcrdma_noch; else wtype = rpcrdma_replych;