From patchwork Mon Jul 13 16:31:07 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 6780461 Return-Path: X-Original-To: patchwork-linux-rdma@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 67B4E9F38C for ; Mon, 13 Jul 2015 16:31:14 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 793092053F for ; Mon, 13 Jul 2015 16:31:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 8DE2C2052A for ; Mon, 13 Jul 2015 16:31:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751818AbbGMQbL (ORCPT ); Mon, 13 Jul 2015 12:31:11 -0400 Received: from mail-qk0-f172.google.com ([209.85.220.172]:34194 "EHLO mail-qk0-f172.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751808AbbGMQbL (ORCPT ); Mon, 13 Jul 2015 12:31:11 -0400 Received: by qkcl188 with SMTP id l188so74566324qkc.1; Mon, 13 Jul 2015 09:31:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-type:content-transfer-encoding; bh=+SmsTY/GNckdGrxhT15Mkcp2nz4sEvQGaz2K3qUdrZU=; b=LPsCWg6uYLLemBm3tk1PNOuZ5/00azGqdjNlFxDHsrLkNuFhsSpkt/cgq8ciTWuMnY jKhOwYQB+xjQTtBJye7gHq6sO7+bJDQWs1pKB6wvIRszRKeC6G1H0RnMloKVT5VMrVV8 Fe9C9M3KtvKa+GWusrnkRM3UJ2B4yLvAWGpNne7BS9ucgcOOgaMh9KfpVSSYEaMsM+QF u9LvXBKTOlfun9ZMVOQjJbqrf8DasV7GPZ7Z8Ie/MdHAZ5N2hUOCrX8QUaW3OCRwL31Q BismSTkm9qnOyzDlPSvLvXJPHQWKBsqQsO3/OXFnbNiVNkgLjfG8aH+/9fslrtWzVMuM aooQ== X-Received: by 10.55.23.151 with SMTP id 23mr7226875qkx.1.1436805070301; Mon, 13 Jul 2015 09:31:10 -0700 (PDT) Received: from manet.1015granger.net ([2604:8800:100:81fc:82ee:73ff:fe43:d64f]) by smtp.gmail.com with ESMTPSA id 47sm11211538qgt.15.2015.07.13.09.31.08 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 13 Jul 2015 09:31:09 -0700 (PDT) Subject: [PATCH v2 09/14] xprtrdma: Always provide a write list when sending NFS READ From: Chuck Lever To: linux-rdma@vger.kernel.org, linux-nfs@vger.kernel.org Date: Mon, 13 Jul 2015 12:31:07 -0400 Message-ID: <20150713163107.17630.60420.stgit@manet.1015granger.net> In-Reply-To: <20150713160617.17630.97475.stgit@manet.1015granger.net> References: <20150713160617.17630.97475.stgit@manet.1015granger.net> User-Agent: StGit/0.17.1-3-g7d0f MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Spam-Status: No, score=-8.2 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,RP_MATCHES_RCVD,T_DKIM_INVALID,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The client has been setting up a reply chunk for NFS READs that are smaller than the inline threshold. This is not efficient: both the server and client CPUs have to copy the reply's data payload into and out of the memory region that is then transferred via RDMA. Using the write list, the data payload is moved by the device and no extra data copying is necessary. Signed-off-by: Chuck Lever Reviewed-by: Devesh Sharma Reviewed-By: Sagi Grimberg --- net/sunrpc/xprtrdma/rpc_rdma.c | 21 ++++----------------- 1 file changed, 4 insertions(+), 17 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/net/sunrpc/xprtrdma/rpc_rdma.c b/net/sunrpc/xprtrdma/rpc_rdma.c index 950b654..e7cf976 100644 --- a/net/sunrpc/xprtrdma/rpc_rdma.c +++ b/net/sunrpc/xprtrdma/rpc_rdma.c @@ -418,28 +418,15 @@ rpcrdma_marshal_req(struct rpc_rqst *rqst) /* * Chunks needed for results? * + * o Read ops return data as write chunk(s), header as inline. * o If the expected result is under the inline threshold, all ops * return as inline (but see later). * o Large non-read ops return as a single reply chunk. - * o Large read ops return data as write chunk(s), header as inline. - * - * Note: the NFS code sending down multiple result segments implies - * the op is one of read, readdir[plus], readlink or NFSv4 getacl. - */ - - /* - * This code can handle read chunks, write chunks OR reply - * chunks -- only one type. If the request is too big to fit - * inline, then we will choose read chunks. If the request is - * a READ, then use write chunks to separate the file data - * into pages; otherwise use reply chunks. */ - if (rpcrdma_results_inline(rqst)) - wtype = rpcrdma_noch; - else if (rqst->rq_rcv_buf.page_len == 0) - wtype = rpcrdma_replych; - else if (rqst->rq_rcv_buf.flags & XDRBUF_READ) + if (rqst->rq_rcv_buf.flags & XDRBUF_READ) wtype = rpcrdma_writech; + else if (rpcrdma_results_inline(rqst)) + wtype = rpcrdma_noch; else wtype = rpcrdma_replych;