From patchwork Fri Jan 9 19:23:02 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 5602511 Return-Path: X-Original-To: patchwork-linux-nfs@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 3C50BC058E for ; Fri, 9 Jan 2015 19:23:10 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 526A22063B for ; Fri, 9 Jan 2015 19:23:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 61F402063A for ; Fri, 9 Jan 2015 19:23:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752824AbbAITXG (ORCPT ); Fri, 9 Jan 2015 14:23:06 -0500 Received: from mail-ig0-f171.google.com ([209.85.213.171]:35669 "EHLO mail-ig0-f171.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752465AbbAITXF (ORCPT ); Fri, 9 Jan 2015 14:23:05 -0500 Received: by mail-ig0-f171.google.com with SMTP id z20so3212576igj.4; Fri, 09 Jan 2015 11:23:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:from:subject:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-type:content-transfer-encoding; bh=0NGirQvAs+THfvEx1d6tc5IhvjFKoqUWPrGlnf6H2Ag=; b=lqkn3cwdeZFYMjsviZs7r21jWQtCgV0QLlaoEBIa/5nmIubx/XhKKelXGYdGMmQwEC RN12TRQhz8sQaRbNo1yUW6e4MVi6vnOKJXhCKiy7TRcS5CGQiuzjzOoi8+faaWSq3D/U kPuHZ+tZCL3YYf4ZhVCXkzuF05B0XSL3cGPMxPwx50zm8NWQr+ss9152CgNHdruvQzgu /SaVNpTiGrjdjzB9eLaLmcVcmSxaLh99dg0S15QRhKbbnPsU4f6FonFXHDpvifK19+/R k2dOxOzEv04d8xn5ssUGvDPta2nOC72SiDb4uCZH0ImOZZei8iXIqUKCW6NSwJx5xK0j bvgg== X-Received: by 10.42.102.148 with SMTP id i20mr14357213ico.39.1420831383741; Fri, 09 Jan 2015 11:23:03 -0800 (PST) Received: from klimt.1015granger.net ([2604:8800:100:81fc:be5f:f4ff:fed6:c3ba]) by mx.google.com with ESMTPSA id gc9sm9974062igd.8.2015.01.09.11.23.03 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 09 Jan 2015 11:23:03 -0800 (PST) From: Chuck Lever Subject: [PATCH v1 08/10] svcrdma: Support RDMA_NOMSG requests To: linux-rdma@vger.kernel.org, linux-nfs@vger.kernel.org Date: Fri, 09 Jan 2015 14:23:02 -0500 Message-ID: <20150109192302.4901.76354.stgit@klimt.1015granger.net> In-Reply-To: <20150109191910.4901.29548.stgit@klimt.1015granger.net> References: <20150109191910.4901.29548.stgit@klimt.1015granger.net> User-Agent: StGIT/0.14.3 MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org X-Spam-Status: No, score=-6.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID,T_RP_MATCHES_RCVD,UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Currently the Linux server can not decode RDMA_NOMSG type requests. The client sends the RPC/RDMA header via RDMA SEND. The header has a read chunk list that conveys the RPC and NFS headers and the NFS payload (if any). Large operations like NFSv4 CREATE(NF4LNK) will by conveyed via RDMA_NOMSG, so add this support. Signed-off-by: Chuck Lever --- include/linux/sunrpc/svc_rdma.h | 1 + net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 39 +++++++++++++++++++++++++++++-- 2 files changed, 37 insertions(+), 3 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/include/linux/sunrpc/svc_rdma.h b/include/linux/sunrpc/svc_rdma.h index f161e30..c343a94 100644 --- a/include/linux/sunrpc/svc_rdma.h +++ b/include/linux/sunrpc/svc_rdma.h @@ -79,6 +79,7 @@ struct svc_rdma_op_ctxt { enum ib_wr_opcode wr_op; enum ib_wc_status wc_status; u32 byte_len; + u32 position; struct svcxprt_rdma *xprt; unsigned long flags; enum dma_data_direction direction; diff --git a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c index a67dd1a..36cf51a 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c +++ b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c @@ -60,6 +60,7 @@ static void rdma_build_arg_xdr(struct svc_rqst *rqstp, struct svc_rdma_op_ctxt *ctxt, u32 byte_count) { + struct rpcrdma_msg *rmsgp; struct page *page; u32 bc; int sge_no; @@ -82,7 +83,14 @@ static void rdma_build_arg_xdr(struct svc_rqst *rqstp, /* If data remains, store it in the pagelist */ rqstp->rq_arg.page_len = bc; rqstp->rq_arg.page_base = 0; - rqstp->rq_arg.pages = &rqstp->rq_pages[1]; + + /* RDMA_NOMSG: RDMA READ data should land just after RDMA RECV data */ + rmsgp = (struct rpcrdma_msg *)rqstp->rq_arg.head[0].iov_base; + if (be32_to_cpu(rmsgp->rm_type) == RDMA_NOMSG) + rqstp->rq_arg.pages = &rqstp->rq_pages[0]; + else + rqstp->rq_arg.pages = &rqstp->rq_pages[1]; + sge_no = 1; while (bc && sge_no < ctxt->count) { page = ctxt->pages[sge_no]; @@ -383,7 +391,6 @@ static int rdma_read_chunks(struct svcxprt_rdma *xprt, */ head->arg.head[0] = rqstp->rq_arg.head[0]; head->arg.tail[0] = rqstp->rq_arg.tail[0]; - head->arg.pages = &head->pages[head->count]; head->hdr_count = head->count; head->arg.page_base = 0; head->arg.page_len = 0; @@ -393,9 +400,17 @@ static int rdma_read_chunks(struct svcxprt_rdma *xprt, ch = (struct rpcrdma_read_chunk *)&rmsgp->rm_body.rm_chunks[0]; position = be32_to_cpu(ch->rc_position); + /* RDMA_NOMSG: RDMA READ data should land just after RDMA RECV data */ + if (position == 0) { + head->arg.pages = &head->pages[0]; + page_offset = head->byte_len; + } else { + head->arg.pages = &head->pages[head->count]; + page_offset = 0; + } + ret = 0; page_no = 0; - page_offset = 0; for (; ch->rc_discrim != xdr_zero; ch++) { if (be32_to_cpu(ch->rc_position) != position) goto err; @@ -418,7 +433,10 @@ static int rdma_read_chunks(struct svcxprt_rdma *xprt, head->arg.buflen += ret; } } + ret = 1; + head->position = position; + err: /* Detach arg pages. svc_recv will replenish them */ for (page_no = 0; @@ -465,6 +483,21 @@ static int rdma_read_complete(struct svc_rqst *rqstp, put_page(rqstp->rq_pages[page_no]); rqstp->rq_pages[page_no] = head->pages[page_no]; } + + /* Adjustments made for RDMA_NOMSG type requests */ + if (head->position == 0) { + if (head->arg.len <= head->sge[0].length) { + head->arg.head[0].iov_len = head->arg.len - + head->byte_len; + head->arg.page_len = 0; + } else { + head->arg.head[0].iov_len = head->sge[0].length - + head->byte_len; + head->arg.page_len = head->arg.len - + head->sge[0].length; + } + } + /* Point rq_arg.pages past header */ rdma_fix_xdr_pad(&head->arg); rqstp->rq_arg.pages = &rqstp->rq_pages[head->hdr_count];