From patchwork Tue Nov 29 16:04:42 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 9452541 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id F3EB360710 for ; Tue, 29 Nov 2016 16:04:48 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E647E201BD for ; Tue, 29 Nov 2016 16:04:48 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DB3B4283FB; Tue, 29 Nov 2016 16:04:48 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.3 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI, RCVD_IN_SORBS_SPAM, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DEEFD283F6 for ; Tue, 29 Nov 2016 16:04:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S933901AbcK2QEp (ORCPT ); Tue, 29 Nov 2016 11:04:45 -0500 Received: from mail-io0-f196.google.com ([209.85.223.196]:33069 "EHLO mail-io0-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S933710AbcK2QEo (ORCPT ); Tue, 29 Nov 2016 11:04:44 -0500 Received: by mail-io0-f196.google.com with SMTP id j92so30366938ioi.0; Tue, 29 Nov 2016 08:04:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=9/SxJOoNRMxMP1agQWPHKjXbCFfU4/js4mLXISZCWvA=; b=i4pJnVWsA8qF7hMHNJcRr+MTFMxbh/odi/cqq1CkvA3OJbJKs7SbVCpGEWwIJro3t5 Eu/LrYFr+lvQOI/OuLdO9jCVjjBTc9YfS7gfETeoKPc255q6b47UD0qS1ajpdBDHSK4X lJULBROAd3agMP3mhXO3+OrP4Ut7bKeMf2zXDEmH5EsU7T8r/0RwJKTbn+fWIf5rkThx +qQYWo8EMJOATlrNQWFX3CDUkeTew4DDFBHi4MSpFrWHl9eeryO7kyXFfYrkw4V2ss85 /ggmZ11QXYu5/54t11Dhitef6GRss7ST2KMZG55e9qmwncuMNrayc/J6D3MZMjBdy+3I mrlA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=9/SxJOoNRMxMP1agQWPHKjXbCFfU4/js4mLXISZCWvA=; b=HjTEJ5X0jq/4r2VBZUzaxH7aFGn1EiAz6TGtvjBRNd9G/KEduQTAWTZooVp8DUARvB RsYIrLHaWGG7+CtNR235KIvxk0mxTU4kdjOZ3/Txl1sFGDgzRN1egMjPfJtANgwfWcCW 5hPDa1w1g1EOWRGv08nx6Z+a1jG8z/75lmy2ToQhLnW+SpGf3tNsMklMtos0voh+ydOO dCMcMZan/HP7oCkpjvwRUUcVoTkM0hWZA2HsSdNFYWAqVM544VBO0z/42MddwV1Yn4Bu YHhlgOCFNfQrsT032VHh9fp1mZ4zcWL2gVbvKrxV0ZJncdeTO8JCrSVXfddQ348OOwcr l+CA== X-Gm-Message-State: AKaTC000/Rr5MQw6Q1ahyllsqEOmS9HUVw6b1ozV70VxfSYndcyV+xMVvBjnq2/6DorMkw== X-Received: by 10.36.108.17 with SMTP id w17mr24017315itb.108.1480435483708; Tue, 29 Nov 2016 08:04:43 -0800 (PST) Received: from klimt.1015granger.net ([2604:8800:100:81fc:ec4:7aff:fe6c:6aa0]) by smtp.gmail.com with ESMTPSA id n102sm21746542ioi.38.2016.11.29.08.04.43 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 29 Nov 2016 08:04:43 -0800 (PST) Subject: [PATCH v3 03/10] svcrdma: Renovate sendto chunk list parsing From: Chuck Lever To: bfields@fieldses.org Cc: linux-rdma@vger.kernel.org, linux-nfs@vger.kernel.org Date: Tue, 29 Nov 2016 11:04:42 -0500 Message-ID: <20161129160442.4477.7758.stgit@klimt.1015granger.net> In-Reply-To: <20161129155521.4477.53561.stgit@klimt.1015granger.net> References: <20161129155521.4477.53561.stgit@klimt.1015granger.net> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The current sendto code appears to support clients that provide only one of a Read list, a Write list, or a Reply chunk. My reading of that code is that it doesn't support the following cases: - Read list + Write list - Read list + Reply chunk - Write list + Reply chunk - Read list + Write list + Reply chunk The protocol allows more than one Read or Write chunk in those lists. Some clients do send a Read list and Reply chunk simultaneously. NFSv4 WRITE uses a Read list for the data payload, and a Reply chunk because the GETATTR result in the reply can contain a large object like an ACL. Generalize one of the sendto code paths needed to support all of the above cases, and attempt to ensure that only one pass is done through the RPC Call's transport header to gather chunk list information for building the reply. Signed-off-by: Chuck Lever --- include/linux/sunrpc/svc_rdma.h | 2 - net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 14 +++++ net/sunrpc/xprtrdma/svc_rdma_sendto.c | 92 ++++++++----------------------- 3 files changed, 39 insertions(+), 69 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/include/linux/sunrpc/svc_rdma.h b/include/linux/sunrpc/svc_rdma.h index cc3ae16..6aef63b 100644 --- a/include/linux/sunrpc/svc_rdma.h +++ b/include/linux/sunrpc/svc_rdma.h @@ -236,8 +236,6 @@ extern int rdma_read_chunk_frmr(struct svcxprt_rdma *, struct svc_rqst *, extern int svc_rdma_map_xdr(struct svcxprt_rdma *, struct xdr_buf *, struct svc_rdma_req_map *, bool); extern int svc_rdma_sendto(struct svc_rqst *); -extern struct rpcrdma_read_chunk * - svc_rdma_get_read_chunk(struct rpcrdma_msg *); extern void svc_rdma_send_error(struct svcxprt_rdma *, struct rpcrdma_msg *, int); diff --git a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c index ad1df97..873c2a9 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c +++ b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c @@ -415,6 +415,20 @@ int rdma_read_chunk_frmr(struct svcxprt_rdma *xprt, return 1; } +/* Returns the address of the first read chunk or if no read chunk + * is present + */ +static struct rpcrdma_read_chunk * +svc_rdma_get_read_chunk(struct rpcrdma_msg *rmsgp) +{ + struct rpcrdma_read_chunk *ch = + (struct rpcrdma_read_chunk *)&rmsgp->rm_body.rm_chunks[0]; + + if (ch->rc_discrim == xdr_zero) + return NULL; + return ch; +} + static int rdma_read_chunks(struct svcxprt_rdma *xprt, struct rpcrdma_msg *rmsgp, struct svc_rqst *rqstp, diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c index f5a91ed..0a58d40 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c +++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c @@ -153,76 +153,35 @@ static dma_addr_t dma_map_xdr(struct svcxprt_rdma *xprt, return dma_addr; } -/* Returns the address of the first read chunk or if no read chunk - * is present +/* Parse the RPC Call's transport header. */ -struct rpcrdma_read_chunk * -svc_rdma_get_read_chunk(struct rpcrdma_msg *rmsgp) +static void svc_rdma_get_write_arrays(struct rpcrdma_msg *rmsgp, + struct rpcrdma_write_array **write, + struct rpcrdma_write_array **reply) { - struct rpcrdma_read_chunk *ch = - (struct rpcrdma_read_chunk *)&rmsgp->rm_body.rm_chunks[0]; + __be32 *p; - if (ch->rc_discrim == xdr_zero) - return NULL; - return ch; -} - -/* Returns the address of the first read write array element or - * if no write array list is present - */ -static struct rpcrdma_write_array * -svc_rdma_get_write_array(struct rpcrdma_msg *rmsgp) -{ - if (rmsgp->rm_body.rm_chunks[0] != xdr_zero || - rmsgp->rm_body.rm_chunks[1] == xdr_zero) - return NULL; - return (struct rpcrdma_write_array *)&rmsgp->rm_body.rm_chunks[1]; -} + p = (__be32 *)&rmsgp->rm_body.rm_chunks[0]; -/* Returns the address of the first reply array element or if no - * reply array is present - */ -static struct rpcrdma_write_array * -svc_rdma_get_reply_array(struct rpcrdma_msg *rmsgp, - struct rpcrdma_write_array *wr_ary) -{ - struct rpcrdma_read_chunk *rch; - struct rpcrdma_write_array *rp_ary; + /* Read list */ + while (*p++ != xdr_zero) + p += 5; - /* XXX: Need to fix when reply chunk may occur with read list - * and/or write list. - */ - if (rmsgp->rm_body.rm_chunks[0] != xdr_zero || - rmsgp->rm_body.rm_chunks[1] != xdr_zero) - return NULL; - - rch = svc_rdma_get_read_chunk(rmsgp); - if (rch) { - while (rch->rc_discrim != xdr_zero) - rch++; - - /* The reply chunk follows an empty write array located - * at 'rc_position' here. The reply array is at rc_target. - */ - rp_ary = (struct rpcrdma_write_array *)&rch->rc_target; - goto found_it; - } - - if (wr_ary) { - int chunk = be32_to_cpu(wr_ary->wc_nchunks); - - rp_ary = (struct rpcrdma_write_array *) - &wr_ary->wc_array[chunk].wc_target.rs_length; - goto found_it; + /* Write list */ + if (*p != xdr_zero) { + *write = (struct rpcrdma_write_array *)p; + while (*p++ != xdr_zero) + p += 1 + be32_to_cpu(*p) * 4; + } else { + *write = NULL; + p++; } - /* No read list, no write list */ - rp_ary = (struct rpcrdma_write_array *)&rmsgp->rm_body.rm_chunks[2]; - - found_it: - if (rp_ary->wc_discrim == xdr_zero) - return NULL; - return rp_ary; + /* Reply chunk */ + if (*p != xdr_zero) + *reply = (struct rpcrdma_write_array *)p; + else + *reply = NULL; } /* RPC-over-RDMA Version One private extension: Remote Invalidation. @@ -244,8 +203,8 @@ static u32 svc_rdma_get_inv_rkey(struct rpcrdma_msg *rdma_argp, inv_rkey = 0; - rd_ary = svc_rdma_get_read_chunk(rdma_argp); - if (rd_ary) { + rd_ary = (struct rpcrdma_read_chunk *)&rdma_argp->rm_body.rm_chunks[0]; + if (rd_ary->rc_discrim != xdr_zero) { inv_rkey = be32_to_cpu(rd_ary->rc_target.rs_handle); goto out; } @@ -622,8 +581,7 @@ int svc_rdma_sendto(struct svc_rqst *rqstp) * places this at the start of page 0. */ rdma_argp = page_address(rqstp->rq_pages[0]); - wr_ary = svc_rdma_get_write_array(rdma_argp); - rp_ary = svc_rdma_get_reply_array(rdma_argp, wr_ary); + svc_rdma_get_write_arrays(rdma_argp, &wr_ary, &rp_ary); inv_rkey = 0; if (rdma->sc_snd_w_inv)