From patchwork Fri Feb 28 00:30:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 11411237 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 767D01805 for ; Fri, 28 Feb 2020 00:30:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 5542E246A1 for ; Fri, 28 Feb 2020 00:30:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="SDQPOs+S" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730146AbgB1Aaf (ORCPT ); Thu, 27 Feb 2020 19:30:35 -0500 Received: from mail-pg1-f195.google.com ([209.85.215.195]:34243 "EHLO mail-pg1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730120AbgB1Aaf (ORCPT ); Thu, 27 Feb 2020 19:30:35 -0500 Received: by mail-pg1-f195.google.com with SMTP id t3so565012pgn.1; Thu, 27 Feb 2020 16:30:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=+B6lJUYrUXialh126JR42u2hESr6pKGaaUWrmI+Up2g=; b=SDQPOs+SSds0PzFJwr1bwUj1w0tAhnNv6T4wAPHkD41uvGpJxpkI0p+6MMwPDEYa68 FfzuVcb9Zor0ALWbOoMPywhNhlG/1DvH8NHaRatgAO2rd/QiSogQEOYwT5I+abiQA51f 67hsyTmQAPTSucpdibaAfd2c9g90/xJLfM+r3nfAUKX1ZY1kAU/s/rUmNpirl0cWfYnW KYP6QQ8m45lyzfD8YShAnyvDlPU0H2GBE8upaRjyZjHbDVLm1/dwhkjKRE5h5z7iGjDE 4VcmuskcpL7TOhXn1n0MjNLvc/wBnSeJXO4TLsMFvGApbbE2Vrh9MWMRG1sj8YP4vUyg VUBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=+B6lJUYrUXialh126JR42u2hESr6pKGaaUWrmI+Up2g=; b=EVjPg6WLUH1GaLQOji30V1LjUxQZxVuqZdbF2njAd+wD5c3L8BM0IxkQAjX6FUUZQN eFIjLNCPfvHVRb486pDm8c2gXC9UB/dYQEhqcxEVr4IcmEHEdOIYE+EHdN2UUZkMoE+T /xKJpUf11nxnUNOP+pr5oif5rU5veeY6aclA0tt+PY8uWgzDdhRmbBli2VjJN/bGdoYQ Ly1wJiN7TIa6JjZE0cDnVlDLg5paP4OSRltKnsqcpgzZU+X1WH7OgO6vhrqI3n80l40x +uOBUZdHqmkxTjPkIbpOLtrbhpOZmbP5D43jOGTaeyEhJhie3EyiIo4EXsFtkPsyb4D5 P4Ig== X-Gm-Message-State: APjAAAU8ttah4bLFcj1ZaVhxqTzIVpGkre+1T+A3EoNa0+iNdpbV22Ct VoEHYl0QBs42TLYT4coA2XMDzI4y X-Google-Smtp-Source: APXvYqxyfCHzMEN12A3Ce5dc1A7Hwswwnkvknyw7wAlQbbyYnqH15ZLQGF6HQCDZpOk/zThaBpYAUA== X-Received: by 2002:a63:5859:: with SMTP id i25mr1874527pgm.74.1582849833779; Thu, 27 Feb 2020 16:30:33 -0800 (PST) Received: from seurat29.1015granger.net (ip-184-250-247-225.sanjca.spcsdns.net. [184.250.247.225]) by smtp.gmail.com with ESMTPSA id v25sm8210030pfe.147.2020.02.27.16.30.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 Feb 2020 16:30:33 -0800 (PST) Subject: [PATCH v1 01/16] nfsd: Fix NFSv4 READ on RDMA when using readv From: Chuck Lever To: bfields@fieldses.org Cc: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Thu, 27 Feb 2020 19:30:31 -0500 Message-ID: <158284983177.38468.13271201055916310888.stgit@seurat29.1015granger.net> In-Reply-To: <158284930886.38468.17045380766660946827.stgit@seurat29.1015granger.net> References: <158284930886.38468.17045380766660946827.stgit@seurat29.1015granger.net> User-Agent: StGit/unknown-version MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org svcrdma expects that the payload falls precisely into the xdr_buf page vector. This does not seem to be the case for nfsd4_encode_readv(). This code is called only when fops->splice_read is missing or when RQ_SPLICE_OK is clear, so it's not a noticeable problem in many common cases. Add new transport method: ->xpo_read_payload so that when a READ payload does not fit exactly in rq_res's page vector, the XDR encoder can inform the RPC transport exactly where that payload is, without the payload's XDR pad. That way, when a Write chunk is present, the transport knows what byte range in the Reply message is supposed to be matched with the chunk. Note that the Linux NFS server implementation of NFS/RDMA can currently handle only one Write chunk per RPC-over-RDMA message. This simplifies the implementation of this fix. Fixes: b04209806384 ("nfsd4: allow exotic read compounds") Buglink: https://bugzilla.kernel.org/show_bug.cgi?id=198053 Signed-off-by: Chuck Lever --- fs/nfsd/nfs4xdr.c | 20 ++++++++------- include/linux/sunrpc/svc.h | 3 ++ include/linux/sunrpc/svc_rdma.h | 8 +++++- include/linux/sunrpc/svc_xprt.h | 2 ++ net/sunrpc/svc.c | 16 ++++++++++++ net/sunrpc/svcsock.c | 8 ++++++ net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 1 + net/sunrpc/xprtrdma/svc_rdma_rw.c | 30 ++++++++++++++--------- net/sunrpc/xprtrdma/svc_rdma_sendto.c | 40 +++++++++++++++++++++++++++++- net/sunrpc/xprtrdma/svc_rdma_transport.c | 1 + 10 files changed, 106 insertions(+), 23 deletions(-) diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c index 9761512674a0..60be969d8be1 100644 --- a/fs/nfsd/nfs4xdr.c +++ b/fs/nfsd/nfs4xdr.c @@ -3594,17 +3594,17 @@ static __be32 nfsd4_encode_readv(struct nfsd4_compoundres *resp, u32 zzz = 0; int pad; + /* + * svcrdma requires every READ payload to start somewhere + * in xdr->pages. + */ + if (xdr->iov == xdr->buf->head) { + xdr->iov = NULL; + xdr->end = xdr->p; + } + len = maxcount; v = 0; - - thislen = min_t(long, len, ((void *)xdr->end - (void *)xdr->p)); - p = xdr_reserve_space(xdr, (thislen+3)&~3); - WARN_ON_ONCE(!p); - resp->rqstp->rq_vec[v].iov_base = p; - resp->rqstp->rq_vec[v].iov_len = thislen; - v++; - len -= thislen; - while (len) { thislen = min_t(long, len, PAGE_SIZE); p = xdr_reserve_space(xdr, (thislen+3)&~3); @@ -3623,6 +3623,8 @@ static __be32 nfsd4_encode_readv(struct nfsd4_compoundres *resp, read->rd_length = maxcount; if (nfserr) return nfserr; + if (svc_encode_read_payload(resp->rqstp, starting_len + 8, maxcount)) + return nfserr_io; xdr_truncate_encode(xdr, starting_len + 8 + ((maxcount+3)&~3)); tmp = htonl(eof); diff --git a/include/linux/sunrpc/svc.h b/include/linux/sunrpc/svc.h index 1afe38eb33f7..82665ff360fd 100644 --- a/include/linux/sunrpc/svc.h +++ b/include/linux/sunrpc/svc.h @@ -517,6 +517,9 @@ void svc_wake_up(struct svc_serv *); void svc_reserve(struct svc_rqst *rqstp, int space); struct svc_pool * svc_pool_for_cpu(struct svc_serv *serv, int cpu); char * svc_print_addr(struct svc_rqst *, char *, size_t); +int svc_encode_read_payload(struct svc_rqst *rqstp, + unsigned int offset, + unsigned int length); unsigned int svc_fill_write_vector(struct svc_rqst *rqstp, struct page **pages, struct kvec *first, size_t total); diff --git a/include/linux/sunrpc/svc_rdma.h b/include/linux/sunrpc/svc_rdma.h index 40f65888dd38..04e4a34d1c6a 100644 --- a/include/linux/sunrpc/svc_rdma.h +++ b/include/linux/sunrpc/svc_rdma.h @@ -137,6 +137,8 @@ struct svc_rdma_recv_ctxt { unsigned int rc_page_count; unsigned int rc_hdr_count; u32 rc_inv_rkey; + unsigned int rc_read_payload_offset; + unsigned int rc_read_payload_length; struct page *rc_pages[RPCSVC_MAXPAGES]; }; @@ -170,7 +172,9 @@ extern int svc_rdma_recv_read_chunk(struct svcxprt_rdma *rdma, struct svc_rqst *rqstp, struct svc_rdma_recv_ctxt *head, __be32 *p); extern int svc_rdma_send_write_chunk(struct svcxprt_rdma *rdma, - __be32 *wr_ch, struct xdr_buf *xdr); + __be32 *wr_ch, struct xdr_buf *xdr, + unsigned int offset, + unsigned long length); extern int svc_rdma_send_reply_chunk(struct svcxprt_rdma *rdma, __be32 *rp_ch, bool writelist, struct xdr_buf *xdr); @@ -189,6 +193,8 @@ extern int svc_rdma_map_reply_msg(struct svcxprt_rdma *rdma, struct svc_rdma_send_ctxt *ctxt, struct xdr_buf *xdr, __be32 *wr_lst); extern int svc_rdma_sendto(struct svc_rqst *); +extern int svc_rdma_read_payload(struct svc_rqst *rqstp, unsigned int offset, + unsigned int length); /* svc_rdma_transport.c */ extern int svc_rdma_create_listen(struct svc_serv *, int, struct sockaddr *); diff --git a/include/linux/sunrpc/svc_xprt.h b/include/linux/sunrpc/svc_xprt.h index ea6f46be9cb7..9e1e046de176 100644 --- a/include/linux/sunrpc/svc_xprt.h +++ b/include/linux/sunrpc/svc_xprt.h @@ -21,6 +21,8 @@ struct svc_xprt_ops { int (*xpo_has_wspace)(struct svc_xprt *); int (*xpo_recvfrom)(struct svc_rqst *); int (*xpo_sendto)(struct svc_rqst *); + int (*xpo_read_payload)(struct svc_rqst *, unsigned int, + unsigned int); void (*xpo_release_rqst)(struct svc_rqst *); void (*xpo_detach)(struct svc_xprt *); void (*xpo_free)(struct svc_xprt *); diff --git a/net/sunrpc/svc.c b/net/sunrpc/svc.c index 187dd4e73d64..18676d36f490 100644 --- a/net/sunrpc/svc.c +++ b/net/sunrpc/svc.c @@ -1636,6 +1636,22 @@ u32 svc_max_payload(const struct svc_rqst *rqstp) } EXPORT_SYMBOL_GPL(svc_max_payload); +/** + * svc_encode_read_payload - mark a range of bytes as a READ payload + * @rqstp: svc_rqst to operate on + * @offset: payload's byte offset in rqstp->rq_res + * @length: size of payload, in bytes + * + * Returns zero on success, or a negative errno if a permanent + * error occurred. + */ +int svc_encode_read_payload(struct svc_rqst *rqstp, unsigned int offset, + unsigned int length) +{ + return rqstp->rq_xprt->xpt_ops->xpo_read_payload(rqstp, offset, length); +} +EXPORT_SYMBOL_GPL(svc_encode_read_payload); + /** * svc_fill_write_vector - Construct data argument for VFS write call * @rqstp: svc_rqst to operate on diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c index 2934dd711715..758ab10690de 100644 --- a/net/sunrpc/svcsock.c +++ b/net/sunrpc/svcsock.c @@ -279,6 +279,12 @@ static int svc_sendto(struct svc_rqst *rqstp, struct xdr_buf *xdr) return len; } +static int svc_sock_read_payload(struct svc_rqst *rqstp, unsigned int offset, + unsigned int length) +{ + return 0; +} + /* * Report socket names for nfsdfs */ @@ -653,6 +659,7 @@ static const struct svc_xprt_ops svc_udp_ops = { .xpo_create = svc_udp_create, .xpo_recvfrom = svc_udp_recvfrom, .xpo_sendto = svc_udp_sendto, + .xpo_read_payload = svc_sock_read_payload, .xpo_release_rqst = svc_release_udp_skb, .xpo_detach = svc_sock_detach, .xpo_free = svc_sock_free, @@ -1171,6 +1178,7 @@ static const struct svc_xprt_ops svc_tcp_ops = { .xpo_create = svc_tcp_create, .xpo_recvfrom = svc_tcp_recvfrom, .xpo_sendto = svc_tcp_sendto, + .xpo_read_payload = svc_sock_read_payload, .xpo_release_rqst = svc_release_skb, .xpo_detach = svc_tcp_sock_detach, .xpo_free = svc_sock_free, diff --git a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c index 96bccd398469..71127d898562 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c +++ b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c @@ -193,6 +193,7 @@ svc_rdma_recv_ctxt_get(struct svcxprt_rdma *rdma) out: ctxt->rc_page_count = 0; + ctxt->rc_read_payload_length = 0; return ctxt; out_empty: diff --git a/net/sunrpc/xprtrdma/svc_rdma_rw.c b/net/sunrpc/xprtrdma/svc_rdma_rw.c index 48fe3b16b0d9..b0ac535c8728 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_rw.c +++ b/net/sunrpc/xprtrdma/svc_rdma_rw.c @@ -482,18 +482,19 @@ static int svc_rdma_send_xdr_kvec(struct svc_rdma_write_info *info, vec->iov_len); } -/* Send an xdr_buf's page list by itself. A Write chunk is - * just the page list. a Reply chunk is the head, page list, - * and tail. This function is shared between the two types - * of chunk. +/* Send an xdr_buf's page list by itself. A Write chunk is just + * the page list. A Reply chunk is @xdr's head, page list, and + * tail. This function is shared between the two types of chunk. */ static int svc_rdma_send_xdr_pagelist(struct svc_rdma_write_info *info, - struct xdr_buf *xdr) + struct xdr_buf *xdr, + unsigned int offset, + unsigned long length) { info->wi_xdr = xdr; - info->wi_next_off = 0; + info->wi_next_off = offset - xdr->head[0].iov_len; return svc_rdma_build_writes(info, svc_rdma_pagelist_to_sg, - xdr->page_len); + length); } /** @@ -501,6 +502,8 @@ static int svc_rdma_send_xdr_pagelist(struct svc_rdma_write_info *info, * @rdma: controlling RDMA transport * @wr_ch: Write chunk provided by client * @xdr: xdr_buf containing the data payload + * @offset: payload's byte offset in @xdr + * @length: size of payload, in bytes * * Returns a non-negative number of bytes the chunk consumed, or * %-E2BIG if the payload was larger than the Write chunk, @@ -510,19 +513,20 @@ static int svc_rdma_send_xdr_pagelist(struct svc_rdma_write_info *info, * %-EIO if rdma_rw initialization failed (DMA mapping, etc). */ int svc_rdma_send_write_chunk(struct svcxprt_rdma *rdma, __be32 *wr_ch, - struct xdr_buf *xdr) + struct xdr_buf *xdr, + unsigned int offset, unsigned long length) { struct svc_rdma_write_info *info; int ret; - if (!xdr->page_len) + if (!length) return 0; info = svc_rdma_write_info_alloc(rdma, wr_ch); if (!info) return -ENOMEM; - ret = svc_rdma_send_xdr_pagelist(info, xdr); + ret = svc_rdma_send_xdr_pagelist(info, xdr, offset, length); if (ret < 0) goto out_err; @@ -531,7 +535,7 @@ int svc_rdma_send_write_chunk(struct svcxprt_rdma *rdma, __be32 *wr_ch, goto out_err; trace_svcrdma_encode_write(xdr->page_len); - return xdr->page_len; + return length; out_err: svc_rdma_write_info_free(info); @@ -571,7 +575,9 @@ int svc_rdma_send_reply_chunk(struct svcxprt_rdma *rdma, __be32 *rp_ch, * client did not provide Write chunks. */ if (!writelist && xdr->page_len) { - ret = svc_rdma_send_xdr_pagelist(info, xdr); + ret = svc_rdma_send_xdr_pagelist(info, xdr, + xdr->head[0].iov_len, + xdr->page_len); if (ret < 0) goto out_err; consumed += xdr->page_len; diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c index f3f108090aa4..a11983c2056f 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c +++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c @@ -858,7 +858,18 @@ int svc_rdma_sendto(struct svc_rqst *rqstp) if (wr_lst) { /* XXX: Presume the client sent only one Write chunk */ - ret = svc_rdma_send_write_chunk(rdma, wr_lst, xdr); + unsigned long offset; + unsigned int length; + + if (rctxt->rc_read_payload_length) { + offset = rctxt->rc_read_payload_offset; + length = rctxt->rc_read_payload_length; + } else { + offset = xdr->head[0].iov_len; + length = xdr->page_len; + } + ret = svc_rdma_send_write_chunk(rdma, wr_lst, xdr, offset, + length); if (ret < 0) goto err2; svc_rdma_xdr_encode_write_list(rdma_resp, wr_lst, ret); @@ -900,3 +911,30 @@ int svc_rdma_sendto(struct svc_rqst *rqstp) ret = -ENOTCONN; goto out; } + +/** + * svc_rdma_read_payload - special processing for a READ payload + * @rqstp: svc_rqst to operate on + * @offset: payload's byte offset in @xdr + * @length: size of payload, in bytes + * + * Returns zero on success. + * + * For the moment, just record the xdr_buf location of the READ + * payload. svc_rdma_sendto will use that location later when + * we actually send the payload. + */ +int svc_rdma_read_payload(struct svc_rqst *rqstp, unsigned int offset, + unsigned int length) +{ + struct svc_rdma_recv_ctxt *rctxt = rqstp->rq_xprt_ctxt; + + /* XXX: Just one READ payload slot for now, since our + * transport implementation currently supports only one + * Write chunk. + */ + rctxt->rc_read_payload_offset = offset; + rctxt->rc_read_payload_length = length; + + return 0; +} diff --git a/net/sunrpc/xprtrdma/svc_rdma_transport.c b/net/sunrpc/xprtrdma/svc_rdma_transport.c index 145a3615c319..f6aad2798063 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_transport.c +++ b/net/sunrpc/xprtrdma/svc_rdma_transport.c @@ -82,6 +82,7 @@ static const struct svc_xprt_ops svc_rdma_ops = { .xpo_create = svc_rdma_create, .xpo_recvfrom = svc_rdma_recvfrom, .xpo_sendto = svc_rdma_sendto, + .xpo_read_payload = svc_rdma_read_payload, .xpo_release_rqst = svc_rdma_release_rqst, .xpo_detach = svc_rdma_detach, .xpo_free = svc_rdma_free, From patchwork Fri Feb 28 00:30:39 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 11411241 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B6E4A1805 for ; Fri, 28 Feb 2020 00:30:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 96FCB246A0 for ; Fri, 28 Feb 2020 00:30:42 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="B6gmXhKj" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730148AbgB1Aam (ORCPT ); Thu, 27 Feb 2020 19:30:42 -0500 Received: from mail-pj1-f65.google.com ([209.85.216.65]:55259 "EHLO mail-pj1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730117AbgB1Aal (ORCPT ); Thu, 27 Feb 2020 19:30:41 -0500 Received: by mail-pj1-f65.google.com with SMTP id dw13so503609pjb.4; Thu, 27 Feb 2020 16:30:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=OArWSvRlQC6nv37i8m+YGKW30FGPFbQtp+/YdSZuH2k=; b=B6gmXhKj1W1BRHRQOEKxbUHpmQwJd60D7TlUa/fcZ2y3sTz4Bu0O1iKGe0Cdr7TtLO dcsqFkJxUL1o+oi/HJi3PAwTPIR2I0myQ7DExPp8P94f94pty7d6QpmlqwOlrsBkzcKh nanbB5dEsAgE4Mp1DpjXWNDC7sQGOE3mf7rnXcGBoz/oQwXWySoxfuYTr7NKUh9Dyg5A ux54Me3csymEMYJd9CyA/KK90l9JaJp/0lYlyyDxv0c4xOUrUNXtL+uC6nTyG7xP7Omn OEK9lbaAozWL3836cn72NEV5mnkQ2medo4nkXR14YdnL+swc94DpSkBnFPYNccrzBTeI 1/YA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=OArWSvRlQC6nv37i8m+YGKW30FGPFbQtp+/YdSZuH2k=; b=Gt58Y4uvf594Uno1AGoS7ODaTQM6EbmixmweMupqY6advYLl6qnIeZde1xqqin/hM9 CXFZdfilCCr3GgH8Yp0ugSnR8r9/BrDbKt7Bh68RSSGr19/jTu9i3fvcrg03HUrwJkjf U734xzZuofQo2kuoN4NN/2DtRDjQTvNxt/lU8hCxLYU6XD/zzWZIA9f6tdL0agIQ5xof +4zJjjYboiJL4XiZ7kC7CrM8QXarPEELjC7KYh4pTk1oK3r9CgWfIfmEsNugxCeDKs3g 78erUMHG53H69eGhtO0bVvobtaOFcBBqnO8i955HXe18w3psUv33siG4v0dp6DggJDKr e0CA== X-Gm-Message-State: APjAAAVteUxjUUfL1pUYmnGFy1zAp1w+6mhpv9E25/2Tuh2xudA8lbKG buSopW0qwg8T42Qh8da+NzQ= X-Google-Smtp-Source: APXvYqzRobcgWXOxFQVSTeFG9jTk1wtm+FxzCvLqNoiITBQVA6hbh8zUJ2uAcaIP58xc5m4MS0WE8w== X-Received: by 2002:a17:90b:315:: with SMTP id ay21mr1661647pjb.25.1582849840751; Thu, 27 Feb 2020 16:30:40 -0800 (PST) Received: from seurat29.1015granger.net (ip-184-250-247-225.sanjca.spcsdns.net. [184.250.247.225]) by smtp.gmail.com with ESMTPSA id o2sm2965295pfh.26.2020.02.27.16.30.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 Feb 2020 16:30:40 -0800 (PST) Subject: [PATCH v1 02/16] NFSD: Clean up nfsd4_encode_readv From: Chuck Lever To: bfields@fieldses.org Cc: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Thu, 27 Feb 2020 19:30:39 -0500 Message-ID: <158284983901.38468.38555112653975164.stgit@seurat29.1015granger.net> In-Reply-To: <158284930886.38468.17045380766660946827.stgit@seurat29.1015granger.net> References: <158284930886.38468.17045380766660946827.stgit@seurat29.1015granger.net> User-Agent: StGit/unknown-version MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Address some minor nits I noticed while working on this function. Signed-off-by: Chuck Lever --- fs/nfsd/nfs4xdr.c | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c index 60be969d8be1..262f9fc76e4e 100644 --- a/fs/nfsd/nfs4xdr.c +++ b/fs/nfsd/nfs4xdr.c @@ -3591,7 +3591,6 @@ static __be32 nfsd4_encode_readv(struct nfsd4_compoundres *resp, __be32 nfserr; __be32 tmp; __be32 *p; - u32 zzz = 0; int pad; /* @@ -3607,7 +3606,7 @@ static __be32 nfsd4_encode_readv(struct nfsd4_compoundres *resp, v = 0; while (len) { thislen = min_t(long, len, PAGE_SIZE); - p = xdr_reserve_space(xdr, (thislen+3)&~3); + p = xdr_reserve_space(xdr, thislen); WARN_ON_ONCE(!p); resp->rqstp->rq_vec[v].iov_base = p; resp->rqstp->rq_vec[v].iov_len = thislen; @@ -3616,7 +3615,6 @@ static __be32 nfsd4_encode_readv(struct nfsd4_compoundres *resp, } read->rd_vlen = v; - len = maxcount; nfserr = nfsd_readv(resp->rqstp, read->rd_fhp, file, read->rd_offset, resp->rqstp->rq_vec, read->rd_vlen, &maxcount, &eof); @@ -3625,16 +3623,17 @@ static __be32 nfsd4_encode_readv(struct nfsd4_compoundres *resp, return nfserr; if (svc_encode_read_payload(resp->rqstp, starting_len + 8, maxcount)) return nfserr_io; - xdr_truncate_encode(xdr, starting_len + 8 + ((maxcount+3)&~3)); + xdr_truncate_encode(xdr, starting_len + 8 + xdr_align_size(maxcount)); tmp = htonl(eof); write_bytes_to_xdr_buf(xdr->buf, starting_len , &tmp, 4); tmp = htonl(maxcount); write_bytes_to_xdr_buf(xdr->buf, starting_len + 4, &tmp, 4); + tmp = xdr_zero; pad = (maxcount&3) ? 4 - (maxcount&3) : 0; write_bytes_to_xdr_buf(xdr->buf, starting_len + 8 + maxcount, - &zzz, pad); + &tmp, pad); return 0; } From patchwork Fri Feb 28 00:30:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 11411245 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D10D71805 for ; Fri, 28 Feb 2020 00:30:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id B025724691 for ; Fri, 28 Feb 2020 00:30:49 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ptFrkJOD" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730223AbgB1Aat (ORCPT ); Thu, 27 Feb 2020 19:30:49 -0500 Received: from mail-pf1-f195.google.com ([209.85.210.195]:43080 "EHLO mail-pf1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730117AbgB1Aas (ORCPT ); Thu, 27 Feb 2020 19:30:48 -0500 Received: by mail-pf1-f195.google.com with SMTP id s1so729873pfh.10; Thu, 27 Feb 2020 16:30:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=7Tl6CT9IsO+dzMDBFGszYkcdJnK5mt31G6SnF6ZysW4=; b=ptFrkJODPhpYFuImd1f5SP/eycjHt4qP28j5fN7PCS0ILCa0gjSjU+8qq+sIyLO7NB 0pnSy9k7coEEqx6QATEKziTM/UFvgHl6ia/unFaPX277Y/micfeonyqWXiWTM6pVj9q9 E+h/ZWbyDCKe5kMRGmN6o8T1jg66CwJHtAf4J5iVz8H93GlE4APQ0HLABeKA/wjivM/M f1nN3tJ9X+dJRQGnB3ZmkMM1BU+DEwo00Nrup0wI7LWT118+ENl1JJrQgTjX4LdWxCYh nj8zmT5GO06th7hqjUOYLwfY9QlaB6sJqUPCI4l1j9NwyPwv3lm42TPq97dibZQN45su LrVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=7Tl6CT9IsO+dzMDBFGszYkcdJnK5mt31G6SnF6ZysW4=; b=s1hjZUfudZ8S0o4ZG62i2V8AE3dLzOFIghTKkIizd+t7Pt/FGthCo87hNSvU/o7c2b nlg75pi6sOCzTb8LgIxcC2RWWAIz87FBkpxppBw3gHmt2IvYkBM02mH2wp0nWV3nVK/H L3JXR3icoFPJL8QXl9CMOhbffdb+LMMUGwp28hKhvBChKHa0UIxT9EiBNjrrdafEMnQT SOQfJmFvCaWpsamaoZPaBwjseGhDakTLXVKvxalOflN0M5iVT/kUmOAn+5ds1uv7wmgQ hPDke5JIWW0W88zgx3UjXgmbugGKqnc86GzgXJx7dN9PGRZTO4SPouITpx5k9/z+ZUCX N5dQ== X-Gm-Message-State: APjAAAWSuhSlpAtEwtF1JY95wRNKqzCWNtQFhzcsxx5U3ywEekX84EkM ebfJD6h9SMG5nOgnGR7Fis8= X-Google-Smtp-Source: APXvYqw91piUdOoAWzdxkAIyWZ4DZLV+/Dmtw6DtjtCrPnUPMvlkfetMmTA4pomV/qsZ1HqFp5D2Dw== X-Received: by 2002:a63:cc4c:: with SMTP id q12mr1840352pgi.443.1582849847748; Thu, 27 Feb 2020 16:30:47 -0800 (PST) Received: from seurat29.1015granger.net (ip-184-250-247-225.sanjca.spcsdns.net. [184.250.247.225]) by smtp.gmail.com with ESMTPSA id h2sm7826833pgv.40.2020.02.27.16.30.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 Feb 2020 16:30:47 -0800 (PST) Subject: [PATCH v1 03/16] svcrdma: Fix double svc_rdma_send_ctxt_put() in an error path From: Chuck Lever To: bfields@fieldses.org Cc: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Thu, 27 Feb 2020 19:30:45 -0500 Message-ID: <158284984592.38468.654235622058128742.stgit@seurat29.1015granger.net> In-Reply-To: <158284930886.38468.17045380766660946827.stgit@seurat29.1015granger.net> References: <158284930886.38468.17045380766660946827.stgit@seurat29.1015granger.net> User-Agent: StGit/unknown-version MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org This error path is almost never executed. Found by code inspection. Fixes: 99722fe4d5a6 ("svcrdma: Persistently allocate and DMA-map Send buffers") Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/svc_rdma_sendto.c | 9 +-------- 1 file changed, 1 insertion(+), 8 deletions(-) diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c index a11983c2056f..354c5619176a 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c +++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c @@ -786,7 +786,6 @@ static int svc_rdma_send_error_msg(struct svcxprt_rdma *rdma, struct svc_rqst *rqstp) { __be32 *p; - int ret; p = ctxt->sc_xprt_buf; trace_svcrdma_err_chunk(*p); @@ -798,13 +797,7 @@ static int svc_rdma_send_error_msg(struct svcxprt_rdma *rdma, svc_rdma_save_io_pages(rqstp, ctxt); ctxt->sc_send_wr.opcode = IB_WR_SEND; - ret = svc_rdma_send(rdma, &ctxt->sc_send_wr); - if (ret) { - svc_rdma_send_ctxt_put(rdma, ctxt); - return ret; - } - - return 0; + return svc_rdma_send(rdma, &ctxt->sc_send_wr); } /** From patchwork Fri Feb 28 00:30:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 11411247 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 21809924 for ; Fri, 28 Feb 2020 00:30:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id F2EE02469D for ; Fri, 28 Feb 2020 00:30:56 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="TOqgQ9Ts" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730224AbgB1Aa4 (ORCPT ); Thu, 27 Feb 2020 19:30:56 -0500 Received: from mail-pj1-f65.google.com ([209.85.216.65]:55305 "EHLO mail-pj1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730117AbgB1Aa4 (ORCPT ); Thu, 27 Feb 2020 19:30:56 -0500 Received: by mail-pj1-f65.google.com with SMTP id a18so501410pjs.5; Thu, 27 Feb 2020 16:30:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=PxXEbyCt2Y5aTckU+MyVu/CMPVJ9yuH9or7Jnd/zP6o=; b=TOqgQ9TsNg47yc1Fw0yWdvw6lZoCxHASK5LRVWiWzgVQW+48ondnDmbD4QRwYKICZb +cyq0aRu2lfuuKcEuOMkhp4siJzv6/Z6JJ5/OdlrzQmwKGaVzzaF0tkAntblCmFpEN3z 2R2I7sT9v1s7RFf1k43LnT0Nvi4pEVph6yozmgP5nEJVAZzFa2t3OwWmGSnNCt4vsGd5 7wq0oX5CjvrWnDD6Hkneb64YaOH7Uwo1o7CePh58CwzOb2Wg+1X82tO7u+HiVOrxLRoD g4vBkS7lLToyikWqAqbi+KTirumvqRom4sRcT8RdiGgCi+wCEaMYcp9oMuFNoHhWjvh+ i3xQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=PxXEbyCt2Y5aTckU+MyVu/CMPVJ9yuH9or7Jnd/zP6o=; b=Sdf3Hlbhl2G2+sA0PFhQ3IAROjLL1olEtc0Avd8BPufSar8RbnuW0ERG9zcZqvpIaE tckF6VHptRkk0qdppNZmroKw8moFbzlYCSgcKMeJG8qc8NSnBOHQ7L6Qm/5kZraEMmPn 89Jae5RzHoZiNHwr37zAlo7X0aXpWlTfnStctrQomNqBEDagvPIrvTgqqaengAxTfA0X nPHPmeJ8vjy31BrYqW3Wh6HgSL8twkv96w/+7Xn/P+cGhhuUkod1rnJP9P9tC6wOyDHr 2e75tJLEx7119EG6VvnAqn22Bh5FJGCKYC60Ypzw/SOiaEtJ8zWc8uXsEWNeShYZVTqB KzZg== X-Gm-Message-State: APjAAAUYZpsUgwE1k5pm07QvSgYadrGZsbwrx68FlBTPMBg2f7CoNUXS KH7bP4I6RBVab6DKZM+cT3qVTEXykto= X-Google-Smtp-Source: APXvYqyjQXna0tMDUNC9GKxsMIW0LBEA1U21JcB+qx0SSuzMSMDC8RY26dkznuQMOy5WLUsoNnXgdg== X-Received: by 2002:a17:90a:2565:: with SMTP id j92mr1675159pje.117.1582849854749; Thu, 27 Feb 2020 16:30:54 -0800 (PST) Received: from seurat29.1015granger.net (ip-184-250-247-225.sanjca.spcsdns.net. [184.250.247.225]) by smtp.gmail.com with ESMTPSA id w11sm7822133pgh.5.2020.02.27.16.30.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 Feb 2020 16:30:54 -0800 (PST) Subject: [PATCH v1 04/16] SUNRPC: Add xdr_pad_size() helper From: Chuck Lever To: bfields@fieldses.org Cc: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Thu, 27 Feb 2020 19:30:53 -0500 Message-ID: <158284985308.38468.4975436193883884123.stgit@seurat29.1015granger.net> In-Reply-To: <158284930886.38468.17045380766660946827.stgit@seurat29.1015granger.net> References: <158284930886.38468.17045380766660946827.stgit@seurat29.1015granger.net> User-Agent: StGit/unknown-version MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Introduce a helper function to compute the XDR pad size of a variable-length XDR object. Clean up: Replace open-coded calculation of XDR pad sizes. I'm sure I haven't found every instance of this calculation. Signed-off-by: Chuck Lever --- include/linux/sunrpc/xdr.h | 16 ++++++++++++++++ net/sunrpc/auth_gss/auth_gss.c | 4 ++-- net/sunrpc/auth_gss/svcauth_gss.c | 4 ++-- net/sunrpc/xprtrdma/rpc_rdma.c | 3 ++- net/sunrpc/xprtrdma/svc_rdma_sendto.c | 9 ++------- 5 files changed, 24 insertions(+), 12 deletions(-) diff --git a/include/linux/sunrpc/xdr.h b/include/linux/sunrpc/xdr.h index b41f34977995..80972512edfd 100644 --- a/include/linux/sunrpc/xdr.h +++ b/include/linux/sunrpc/xdr.h @@ -300,6 +300,22 @@ xdr_align_size(size_t n) return (n + mask) & ~mask; } +/** + * xdr_pad_size - Calculate size of an object's pad + * @n: Size of an object being XDR encoded (in bytes) + * + * This implementation avoids the need for conditional + * branches or modulo division. + * + * Return value: + * Size (in bytes) of the needed XDR pad + */ +static inline size_t +xdr_pad_size(size_t n) +{ + return xdr_align_size(n) - n; +} + /** * xdr_stream_encode_u32 - Encode a 32-bit integer * @xdr: pointer to xdr_stream diff --git a/net/sunrpc/auth_gss/auth_gss.c b/net/sunrpc/auth_gss/auth_gss.c index 24ca861815b1..60a8f46973dc 100644 --- a/net/sunrpc/auth_gss/auth_gss.c +++ b/net/sunrpc/auth_gss/auth_gss.c @@ -1877,7 +1877,7 @@ static int gss_wrap_req_priv(struct rpc_cred *cred, struct gss_cl_ctx *ctx, else iov = snd_buf->head; p = iov->iov_base + iov->iov_len; - pad = 3 - ((snd_buf->len - offset - 1) & 3); + pad = xdr_pad_size(snd_buf->len - offset); memset(p, 0, pad); iov->iov_len += pad; snd_buf->len += pad; @@ -1949,7 +1949,7 @@ gss_unwrap_resp_integ(struct rpc_task *task, struct rpc_cred *cred, if (unlikely(!p)) goto unwrap_failed; integ_len = be32_to_cpup(p++); - if (integ_len & 3) + if (xdr_pad_size(integ_len)) goto unwrap_failed; data_offset = (u8 *)(p) - (u8 *)rcv_buf->head[0].iov_base; mic_offset = integ_len + data_offset; diff --git a/net/sunrpc/auth_gss/svcauth_gss.c b/net/sunrpc/auth_gss/svcauth_gss.c index 65b67b257302..dd03096922c9 100644 --- a/net/sunrpc/auth_gss/svcauth_gss.c +++ b/net/sunrpc/auth_gss/svcauth_gss.c @@ -961,7 +961,7 @@ unwrap_priv_data(struct svc_rqst *rqstp, struct xdr_buf *buf, u32 seq, struct gs /* XXX: This is very inefficient. It would be better to either do * this while we encrypt, or maybe in the receive code, if we can peak * ahead and work out the service and mechanism there. */ - offset = buf->head[0].iov_len % 4; + offset = xdr_pad_size(buf->head[0].iov_len); if (offset) { buf->buflen = RPCSVC_MAXPAYLOAD; xdr_shift_buf(buf, offset); @@ -1680,7 +1680,7 @@ svcauth_gss_wrap_resp_integ(struct svc_rqst *rqstp) goto out; integ_offset = (u8 *)(p + 1) - (u8 *)resbuf->head[0].iov_base; integ_len = resbuf->len - integ_offset; - BUG_ON(integ_len % 4); + BUG_ON(xdr_pad_size(integ_len)); *p++ = htonl(integ_len); *p++ = htonl(gc->gc_seq); if (xdr_buf_subsegment(resbuf, &integ_buf, integ_offset, integ_len)) { diff --git a/net/sunrpc/xprtrdma/rpc_rdma.c b/net/sunrpc/xprtrdma/rpc_rdma.c index 28020ec104d4..b51e92d43dfd 100644 --- a/net/sunrpc/xprtrdma/rpc_rdma.c +++ b/net/sunrpc/xprtrdma/rpc_rdma.c @@ -1305,7 +1305,8 @@ rpcrdma_decode_msg(struct rpcrdma_xprt *r_xprt, struct rpcrdma_rep *rep, base = (char *)xdr_inline_decode(xdr, 0); rpclen = xdr_stream_remaining(xdr); r_xprt->rx_stats.fixup_copy_count += - rpcrdma_inline_fixup(rqst, base, rpclen, writelist & 3); + rpcrdma_inline_fixup(rqst, base, rpclen, + xdr_pad_size(writelist)); r_xprt->rx_stats.total_rdma_reply += writelist; return rpclen + xdr_align_size(writelist); diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c index 354c5619176a..4add875277f8 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c +++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c @@ -322,11 +322,6 @@ int svc_rdma_send(struct svcxprt_rdma *rdma, struct ib_send_wr *wr) return ret; } -static u32 xdr_padsize(u32 len) -{ - return (len & 3) ? (4 - (len & 3)) : 0; -} - /* Returns length of transport header, in bytes. */ static unsigned int svc_rdma_reply_hdr_len(__be32 *rdma_resp) @@ -595,7 +590,7 @@ static int svc_rdma_pull_up_reply_msg(struct svcxprt_rdma *rdma, if (wr_lst) { u32 xdrpad; - xdrpad = xdr_padsize(xdr->page_len); + xdrpad = xdr_pad_size(xdr->page_len); if (taillen && xdrpad) { tailbase += xdrpad; taillen -= xdrpad; @@ -670,7 +665,7 @@ int svc_rdma_map_reply_msg(struct svcxprt_rdma *rdma, if (wr_lst) { base = xdr->tail[0].iov_base; len = xdr->tail[0].iov_len; - xdr_pad = xdr_padsize(xdr->page_len); + xdr_pad = xdr_pad_size(xdr->page_len); if (len && xdr_pad) { base += xdr_pad; From patchwork Fri Feb 28 00:30:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 11411253 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 96F6414B7 for ; Fri, 28 Feb 2020 00:31:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 7733F246A6 for ; Fri, 28 Feb 2020 00:31:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="cyX01OBz" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730277AbgB1AbE (ORCPT ); Thu, 27 Feb 2020 19:31:04 -0500 Received: from mail-pl1-f194.google.com ([209.85.214.194]:35586 "EHLO mail-pl1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730117AbgB1AbC (ORCPT ); Thu, 27 Feb 2020 19:31:02 -0500 Received: by mail-pl1-f194.google.com with SMTP id g6so504753plt.2; Thu, 27 Feb 2020 16:31:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=lg3kW/ZNuCeM5EyA07wYe13GW+GeWvh9tixEQ7JryzQ=; b=cyX01OBzp8GkyfjhCVJvgI8rpBlciq2ehbMyIZpScr+DIHxt/CsNwGWdahb7QtjYe7 u2IuG0GjCltPjwXTl38bMHLWBSQ9miADsWsYhx5Y1CatFvLJbYzWUzZDsD9NR+VfWb5L kqwFAmQJrRs3igdU2mXW5P3WgKqDNrMGNvRkUwmkrCyR7ummc+pqqEuN4TIzUuakcriq A+95BJnffz4PdNmiP85UzP0gKucarcwbNgjEnsi9DRXQLz5UqPj/xDtmw9/bGU6nEY4h v5x2OwVogRXp4n4ns0F5bZvfQvRk5FGvdGBOOMZHYHCF1YGgvN2RGRWHbwyaeeVdERWl jV5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=lg3kW/ZNuCeM5EyA07wYe13GW+GeWvh9tixEQ7JryzQ=; b=PY5FTwfnFoqfuAY6+Fo/AqwLLST4HF6jB71zrlFQ+uNbRjc39QdmaS5mgOKBC8p3QG AxjWo8PD1YvxGK/ebvUYLWWkfQz8VxfgGboD4oUiYmXEQfo3mpfBhCdD6/h/c0MgAPft rgb3/6WUbYoZOISw1rWZgG+b4k7YTezVySip/9Y2YJc9PwkAMWAdBWyS5NHe/bzapTBu QkCeuCLiqlgDOCO6gGPa/PK/JLYjGufXQ/2cxZPKawHRu8foAu4Dd3vhMcVLtFguAE/J HK1e9F7IIBx3+WcjtAQjuYfzdI92J8QjGYyuRrWC5b1kyD9Krpbj06ulyf3nXZ1QiM/s RbXg== X-Gm-Message-State: APjAAAUsKvT5lLCoP064qRu6b9EzgR1WHkEHC8oZyyariuOUbBjePOHF Rvdu3LYGHmbuzk4HPi0Dc8eZ+7Q8vbs= X-Google-Smtp-Source: APXvYqz9GDqrV4TEJhfKXIGs942F3u5olLg1v13Be2wO6dlKk9oW5g3Qv7hQQJJwz+slfXFXqfPxDw== X-Received: by 2002:a17:902:bc86:: with SMTP id bb6mr1397049plb.140.1582849861678; Thu, 27 Feb 2020 16:31:01 -0800 (PST) Received: from seurat29.1015granger.net (ip-184-250-247-225.sanjca.spcsdns.net. [184.250.247.225]) by smtp.gmail.com with ESMTPSA id u25sm3426994pfn.59.2020.02.27.16.31.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 Feb 2020 16:31:01 -0800 (PST) Subject: [PATCH v1 05/16] svcrdma: Create a generic tracing class for displaying xdr_buf layout From: Chuck Lever To: bfields@fieldses.org Cc: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Thu, 27 Feb 2020 19:30:59 -0500 Message-ID: <158284985994.38468.11081309957716392757.stgit@seurat29.1015granger.net> In-Reply-To: <158284930886.38468.17045380766660946827.stgit@seurat29.1015granger.net> References: <158284930886.38468.17045380766660946827.stgit@seurat29.1015granger.net> User-Agent: StGit/unknown-version MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org This class can be used to create trace points in either the RPC client or RPC server paths. It simply displays the length of each part of an xdr_buf, which is useful to determine that the transport and XDR codecs are operating correctly. Signed-off-by: Chuck Lever --- include/trace/events/sunrpc.h | 43 +++++++++++++++++++++++++++++++++++++++++ net/sunrpc/svc_xprt.c | 6 +++++- net/sunrpc/xprt.c | 4 ++-- 3 files changed, 50 insertions(+), 3 deletions(-) diff --git a/include/trace/events/sunrpc.h b/include/trace/events/sunrpc.h index ee993575d2fa..1577223add43 100644 --- a/include/trace/events/sunrpc.h +++ b/include/trace/events/sunrpc.h @@ -14,6 +14,49 @@ #include #include +DECLARE_EVENT_CLASS(xdr_buf_class, + TP_PROTO( + const struct xdr_buf *xdr + ), + + TP_ARGS(xdr), + + TP_STRUCT__entry( + __field(const void *, head_base) + __field(size_t, head_len) + __field(const void *, tail_base) + __field(size_t, tail_len) + __field(unsigned int, page_len) + __field(unsigned int, msg_len) + ), + + TP_fast_assign( + __entry->head_base = xdr->head[0].iov_base; + __entry->head_len = xdr->head[0].iov_len; + __entry->tail_base = xdr->tail[0].iov_base; + __entry->tail_len = xdr->tail[0].iov_len; + __entry->page_len = xdr->page_len; + __entry->msg_len = xdr->len; + ), + + TP_printk("head=[%p,%zu] page=%u tail=[%p,%zu] len=%u", + __entry->head_base, __entry->head_len, __entry->page_len, + __entry->tail_base, __entry->tail_len, __entry->msg_len + ) +); + +#define DEFINE_XDRBUF_EVENT(name) \ + DEFINE_EVENT(xdr_buf_class, name, \ + TP_PROTO( \ + const struct xdr_buf *xdr \ + ), \ + TP_ARGS(xdr)) + +DEFINE_XDRBUF_EVENT(xprt_sendto); +DEFINE_XDRBUF_EVENT(xprt_recvfrom); +DEFINE_XDRBUF_EVENT(svc_recvfrom); +DEFINE_XDRBUF_EVENT(svc_sendto); + TRACE_DEFINE_ENUM(RPC_AUTH_OK); TRACE_DEFINE_ENUM(RPC_AUTH_BADCRED); TRACE_DEFINE_ENUM(RPC_AUTH_REJECTEDCRED); diff --git a/net/sunrpc/svc_xprt.c b/net/sunrpc/svc_xprt.c index de3c077733a7..081564449e98 100644 --- a/net/sunrpc/svc_xprt.c +++ b/net/sunrpc/svc_xprt.c @@ -802,6 +802,8 @@ static int svc_handle_xprt(struct svc_rqst *rqstp, struct svc_xprt *xprt) len = svc_deferred_recv(rqstp); else len = xprt->xpt_ops->xpo_recvfrom(rqstp); + if (len > 0) + trace_svc_recvfrom(&rqstp->rq_arg); rqstp->rq_stime = ktime_get(); rqstp->rq_reserved = serv->sv_max_mesg; atomic_add(rqstp->rq_reserved, &xprt->xpt_reserved); @@ -912,8 +914,10 @@ int svc_send(struct svc_rqst *rqstp) if (test_bit(XPT_DEAD, &xprt->xpt_flags) || test_bit(XPT_CLOSE, &xprt->xpt_flags)) len = -ENOTCONN; - else + else { + trace_svc_sendto(xb); len = xprt->xpt_ops->xpo_sendto(rqstp); + } mutex_unlock(&xprt->xpt_mutex); trace_svc_send(rqstp, len); svc_xprt_release(rqstp); diff --git a/net/sunrpc/xprt.c b/net/sunrpc/xprt.c index 1aafe8d3f3f4..30989d9b61a0 100644 --- a/net/sunrpc/xprt.c +++ b/net/sunrpc/xprt.c @@ -1117,9 +1117,8 @@ void xprt_complete_rqst(struct rpc_task *task, int copied) struct rpc_rqst *req = task->tk_rqstp; struct rpc_xprt *xprt = req->rq_xprt; - dprintk("RPC: %5u xid %08x complete (%d bytes received)\n", - task->tk_pid, ntohl(req->rq_xid), copied); trace_xprt_complete_rqst(xprt, req->rq_xid, copied); + trace_xprt_recvfrom(&req->rq_rcv_buf); xprt->stat.recvs++; @@ -1462,6 +1461,7 @@ xprt_request_transmit(struct rpc_rqst *req, struct rpc_task *snd_task) */ req->rq_ntrans++; + trace_xprt_sendto(&req->rq_snd_buf); connect_cookie = xprt->connect_cookie; status = xprt->ops->send_request(req); if (status != 0) { From patchwork Fri Feb 28 00:31:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 11411257 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 92B091805 for ; Fri, 28 Feb 2020 00:31:10 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 72C20246A1 for ; Fri, 28 Feb 2020 00:31:10 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="h1ttOOa5" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730289AbgB1AbJ (ORCPT ); Thu, 27 Feb 2020 19:31:09 -0500 Received: from mail-pf1-f194.google.com ([209.85.210.194]:39418 "EHLO mail-pf1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730117AbgB1AbJ (ORCPT ); Thu, 27 Feb 2020 19:31:09 -0500 Received: by mail-pf1-f194.google.com with SMTP id l7so737173pff.6; Thu, 27 Feb 2020 16:31:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=PbIbcAfdrfdUvi7CUhRos30gioh103M9RwWAyygvMH8=; b=h1ttOOa5bdr2iNv+pVGTcztbLpy5V604ycxSPtlxq7OUW+ijN6FjbZTvMWvZb/vRM8 9r1tq/HqhRMYm8F4iEiKPxv9LltaQxawtIMkcMrW4KjH9ndfrFrk2Dtj/N+UVzWuK0rQ 0LsKyNCriz2QA7Jestgp7Cu5h28o8GGR8CxdQTDQcxDf6X5RDnOej3l7Mk9Ax+9QeCgw Et+5FfVqqFQ0wb8XtDZJhW9IcSP7kM3mv4DP42M1wwT3/OmSKJumv45+S2VkLJSLtgbX Et6TQvYvI8lNnHng107TZ7BlTByq9ioAWeofQyurJThg2OSodL2T9NUnBRy/v/dEer77 DsCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=PbIbcAfdrfdUvi7CUhRos30gioh103M9RwWAyygvMH8=; b=YEnHTGX7O2HLhH9hMQVHsveZc94yE6IDSgBJ0ckuWPsEAvIWiKt3k3t0wMt8A+PvXm QFjxF6NVQPYo7GSr5iuyp64quY2Yrc4fw/fi/DUuxtqFyn4mzW22PBK+qLVOypIbn3dn 1uZhVikpZW8YiGpTpGYrdpeDkEGdJmq0qBDd6OFHcuhpzHrzhQnxKdU6QH/8DVKsPCef njr9jkHmdZiBEUl4mWFTcpVNhCuNReeUF7x8kpWuL4LCNA9tikrVNqzAByScXcK73ZPs lk5KRtTm54IvZMFkIxVwQbvBnJ7IqQOq/732qwRyqGwCIztsJKft+Q2FSIlx6BGtv5LC EQEA== X-Gm-Message-State: APjAAAVvTnU6nL5kqxFyCkviVNhXtwYojbZ/FfcI/tJf9W2uSHTCHkX+ nK2cHwrsB6NPs7VeRs25v9Q= X-Google-Smtp-Source: APXvYqzm1HX9Skt6sL8kFZXoAmGRg+okWcpTtb5IzHyUQdv1t7uSUhJzjR9E1TPfv5y7a4uRuh8iZw== X-Received: by 2002:aa7:947b:: with SMTP id t27mr1750816pfq.212.1582849868538; Thu, 27 Feb 2020 16:31:08 -0800 (PST) Received: from seurat29.1015granger.net (ip-184-250-247-225.sanjca.spcsdns.net. [184.250.247.225]) by smtp.gmail.com with ESMTPSA id 188sm4018091pfa.62.2020.02.27.16.31.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 Feb 2020 16:31:08 -0800 (PST) Subject: [PATCH v1 06/16] svcrdma: Remove svcrdma_cm_event() trace point From: Chuck Lever To: bfields@fieldses.org Cc: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Thu, 27 Feb 2020 19:31:06 -0500 Message-ID: <158284986689.38468.4218009285915028152.stgit@seurat29.1015granger.net> In-Reply-To: <158284930886.38468.17045380766660946827.stgit@seurat29.1015granger.net> References: <158284930886.38468.17045380766660946827.stgit@seurat29.1015granger.net> User-Agent: StGit/unknown-version MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Clean up. This trace point is no longer needed because the RDMA/core CMA code has an equivalent trace point that was added by commit ed999f820a6c ("RDMA/cma: Add trace points in RDMA Connection Manager"). Signed-off-by: Chuck Lever --- include/trace/events/rpcrdma.h | 28 ---------------------------- net/sunrpc/xprtrdma/svc_rdma_transport.c | 7 ------- 2 files changed, 35 deletions(-) diff --git a/include/trace/events/rpcrdma.h b/include/trace/events/rpcrdma.h index c0e4c93324f5..545fe936a0cc 100644 --- a/include/trace/events/rpcrdma.h +++ b/include/trace/events/rpcrdma.h @@ -1813,34 +1813,6 @@ TRACE_EVENT(svcrdma_post_rw, DEFINE_SENDCOMP_EVENT(read); DEFINE_SENDCOMP_EVENT(write); -TRACE_EVENT(svcrdma_cm_event, - TP_PROTO( - const struct rdma_cm_event *event, - const struct sockaddr *sap - ), - - TP_ARGS(event, sap), - - TP_STRUCT__entry( - __field(unsigned int, event) - __field(int, status) - __array(__u8, addr, INET6_ADDRSTRLEN + 10) - ), - - TP_fast_assign( - __entry->event = event->event; - __entry->status = event->status; - snprintf(__entry->addr, sizeof(__entry->addr) - 1, - "%pISpc", sap); - ), - - TP_printk("addr=%s event=%s (%u/%d)", - __entry->addr, - rdma_show_cm_event(__entry->event), - __entry->event, __entry->status - ) -); - TRACE_EVENT(svcrdma_qp_error, TP_PROTO( const struct ib_event *event, diff --git a/net/sunrpc/xprtrdma/svc_rdma_transport.c b/net/sunrpc/xprtrdma/svc_rdma_transport.c index f6aad2798063..8bb99980ae85 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_transport.c +++ b/net/sunrpc/xprtrdma/svc_rdma_transport.c @@ -241,10 +241,6 @@ static void handle_connect_req(struct rdma_cm_id *new_cma_id, static int rdma_listen_handler(struct rdma_cm_id *cma_id, struct rdma_cm_event *event) { - struct sockaddr *sap = (struct sockaddr *)&cma_id->route.addr.src_addr; - - trace_svcrdma_cm_event(event, sap); - switch (event->event) { case RDMA_CM_EVENT_CONNECT_REQUEST: dprintk("svcrdma: Connect request on cma_id=%p, xprt = %p, " @@ -266,12 +262,9 @@ static int rdma_listen_handler(struct rdma_cm_id *cma_id, static int rdma_cma_handler(struct rdma_cm_id *cma_id, struct rdma_cm_event *event) { - struct sockaddr *sap = (struct sockaddr *)&cma_id->route.addr.dst_addr; struct svcxprt_rdma *rdma = cma_id->context; struct svc_xprt *xprt = &rdma->sc_xprt; - trace_svcrdma_cm_event(event, sap); - switch (event->event) { case RDMA_CM_EVENT_ESTABLISHED: /* Accept complete */ From patchwork Fri Feb 28 00:31:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 11411261 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A67951805 for ; Fri, 28 Feb 2020 00:31:20 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 86DEF246AE for ; Fri, 28 Feb 2020 00:31:20 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="RUcR/iWq" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730337AbgB1AbT (ORCPT ); Thu, 27 Feb 2020 19:31:19 -0500 Received: from mail-pl1-f195.google.com ([209.85.214.195]:43449 "EHLO mail-pl1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730117AbgB1AbS (ORCPT ); Thu, 27 Feb 2020 19:31:18 -0500 Received: by mail-pl1-f195.google.com with SMTP id p11so489252plq.10; Thu, 27 Feb 2020 16:31:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=JvPz9meT5Rq1J6C53cftMDXWbrZconUYoyP9oaQmm4g=; b=RUcR/iWqBqp+W8xwuyAfoqistRB++apA4ETj3SyXCQGNIOwunQ06J7SpyuktXQuW0m /QIkDFvioRSljLlWvTHwQJ7+Ppq7RNHcK8WKIlm0bIBaCr3m3rJlYtinakdjjOwGuGDL 7T+o8I6Br+xQrfAtoQ0/mnYRKpf051xLqI+FO8jCpfPfoS0/qcxyF4aeI8/YRHsyciUq ggujHUqOiulpFWb19cEdSZ649FfWiQeoO0yM9ZVoVQ81qzbScMBTzU+Abh8fV1AD8s0p 7J3dnyzODIjoltrL4LYtirEXyI6aOl5dtxgGOeb+N7ETr08tjkuW5TPQ6DGUMmHCmtrK jDTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=JvPz9meT5Rq1J6C53cftMDXWbrZconUYoyP9oaQmm4g=; b=LivwbDz1+1MBJ2cANwGwNKchNWfOtdke5+Izc9nZosUFWZR9Yh4XcvhZBipqbswFw2 G4ysAHiepnvmjOSKI/JrOawKun9bIoYLZHP2fMUi+/lqE0wi/9srkYTicaEA569xjnj7 kG4t7MiYggn88++W1Fa4YWFUJJYt347Y+MgFNTabLl7EVHghN8RwQcvTA6+b5oAWiQSe x8XhMyoNdjM2GogM7bjTTefIm5V/FgZv9gJR4bCI+VU6WeGKHAOfkRjG/9vlJbXo0wyI njSNxQ+GZNQP5o6wK3F1hG8dgvYZtWOlOhgibP+//a5D4u9GvraaYrQh0Zc1xYxghC7P VBSQ== X-Gm-Message-State: APjAAAUVtRaKZM4eGfBIjfJttBElo4Z5mHZHYKhvkcA8IA6b7znlPw5V 3fQQps5F+hMOrGAmUXuqaGg= X-Google-Smtp-Source: APXvYqzhF9BZE1G6fgfOfMkac5opXCuVm/heclFDOqTYxhYR1wc0RXQSiuMjIItwTwXRPO6gCcHdbw== X-Received: by 2002:a17:90a:664c:: with SMTP id f12mr251890pjm.61.1582849875409; Thu, 27 Feb 2020 16:31:15 -0800 (PST) Received: from seurat29.1015granger.net (ip-184-250-247-225.sanjca.spcsdns.net. [184.250.247.225]) by smtp.gmail.com with ESMTPSA id q12sm8337945pfh.158.2020.02.27.16.31.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 Feb 2020 16:31:14 -0800 (PST) Subject: [PATCH v1 07/16] svcrdma: Use struct xdr_stream to decode ingress transport headers From: Chuck Lever To: bfields@fieldses.org Cc: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Thu, 27 Feb 2020 19:31:13 -0500 Message-ID: <158284987376.38468.8418651624640545176.stgit@seurat29.1015granger.net> In-Reply-To: <158284930886.38468.17045380766660946827.stgit@seurat29.1015granger.net> References: <158284930886.38468.17045380766660946827.stgit@seurat29.1015granger.net> User-Agent: StGit/unknown-version MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org The logic that checks incoming network headers has to be scrupulous. De-duplicate: replace open-coded buffer overflow checks with the use of xdr_stream helpers that are used most everywhere else XDR decoding is done. One minor change to the sanity checks: instead of checking the length of individual segments, cap the length of the whole chunk to be sure it can fit in the set of pages available in rq_pages. This should be a better test of whether the server can handle the chunks in each request. Signed-off-by: Chuck Lever --- include/linux/sunrpc/rpc_rdma.h | 3 include/linux/sunrpc/svc_rdma.h | 1 include/trace/events/rpcrdma.h | 7 + net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 207 +++++++++++++++++++------------ 4 files changed, 131 insertions(+), 87 deletions(-) diff --git a/include/linux/sunrpc/rpc_rdma.h b/include/linux/sunrpc/rpc_rdma.h index 92d182fd8e3b..320c672d84de 100644 --- a/include/linux/sunrpc/rpc_rdma.h +++ b/include/linux/sunrpc/rpc_rdma.h @@ -58,7 +58,8 @@ enum { enum { rpcrdma_fixed_maxsz = 4, rpcrdma_segment_maxsz = 4, - rpcrdma_readchunk_maxsz = 2 + rpcrdma_segment_maxsz, + rpcrdma_readseg_maxsz = 1 + rpcrdma_segment_maxsz, + rpcrdma_readchunk_maxsz = 1 + rpcrdma_readseg_maxsz, }; /* diff --git a/include/linux/sunrpc/svc_rdma.h b/include/linux/sunrpc/svc_rdma.h index 04e4a34d1c6a..c790dbb0dd90 100644 --- a/include/linux/sunrpc/svc_rdma.h +++ b/include/linux/sunrpc/svc_rdma.h @@ -132,6 +132,7 @@ struct svc_rdma_recv_ctxt { struct ib_sge rc_recv_sge; void *rc_recv_buf; struct xdr_buf rc_arg; + struct xdr_stream rc_stream; bool rc_temp; u32 rc_byte_len; unsigned int rc_page_count; diff --git a/include/trace/events/rpcrdma.h b/include/trace/events/rpcrdma.h index 545fe936a0cc..814b73bd2cc7 100644 --- a/include/trace/events/rpcrdma.h +++ b/include/trace/events/rpcrdma.h @@ -1469,7 +1469,7 @@ DECLARE_EVENT_CLASS(svcrdma_segment_event, ); #define DEFINE_SEGMENT_EVENT(name) \ - DEFINE_EVENT(svcrdma_segment_event, svcrdma_encode_##name,\ + DEFINE_EVENT(svcrdma_segment_event, svcrdma_##name,\ TP_PROTO( \ u32 handle, \ u32 length, \ @@ -1477,8 +1477,9 @@ DECLARE_EVENT_CLASS(svcrdma_segment_event, ), \ TP_ARGS(handle, length, offset)) -DEFINE_SEGMENT_EVENT(rseg); -DEFINE_SEGMENT_EVENT(wseg); +DEFINE_SEGMENT_EVENT(decode_wseg); +DEFINE_SEGMENT_EVENT(encode_rseg); +DEFINE_SEGMENT_EVENT(encode_wseg); DECLARE_EVENT_CLASS(svcrdma_chunk_event, TP_PROTO( diff --git a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c index 71127d898562..7db151b7dfd3 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c +++ b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c @@ -358,15 +358,14 @@ static void svc_rdma_build_arg_xdr(struct svc_rqst *rqstp, arg->len = ctxt->rc_byte_len; } -/* This accommodates the largest possible Write chunk, - * in one segment. +/* This accommodates the largest possible Write chunk. */ -#define MAX_BYTES_WRITE_SEG ((u32)(RPCSVC_MAXPAGES << PAGE_SHIFT)) +#define MAX_BYTES_WRITE_CHUNK ((u32)(RPCSVC_MAXPAGES << PAGE_SHIFT)) /* This accommodates the largest possible Position-Zero - * Read chunk or Reply chunk, in one segment. + * Read chunk or Reply chunk. */ -#define MAX_BYTES_SPECIAL_SEG ((u32)((RPCSVC_MAXPAGES + 2) << PAGE_SHIFT)) +#define MAX_BYTES_SPECIAL_CHUNK ((u32)((RPCSVC_MAXPAGES + 2) << PAGE_SHIFT)) /* Sanity check the Read list. * @@ -374,7 +373,7 @@ static void svc_rdma_build_arg_xdr(struct svc_rqst *rqstp, * - This implementation supports only one Read chunk. * * Sanity checks: - * - Read list does not overflow buffer. + * - Read list does not overflow Receive buffer. * - Segment size limited by largest NFS data payload. * * The segment count is limited to how many segments can @@ -382,30 +381,44 @@ static void svc_rdma_build_arg_xdr(struct svc_rqst *rqstp, * buffer. That's about 40 Read segments for a 1KB inline * threshold. * - * Returns pointer to the following Write list. + * Return values: + * %true: Read list is valid. @rctxt's xdr_stream is updated + * to point to the first byte past the Read list. + * %false: Read list is corrupt. @rctxt's xdr_stream is left + * in an unknown state. */ -static __be32 *xdr_check_read_list(__be32 *p, const __be32 *end) +static bool xdr_check_read_list(struct svc_rdma_recv_ctxt *rctxt) { - u32 position; + u32 position, len; bool first; + __be32 *p; + + p = xdr_inline_decode(&rctxt->rc_stream, sizeof(*p)); + if (!p) + return false; + len = 0; first = true; - while (*p++ != xdr_zero) { + while (*p != xdr_zero) { + p = xdr_inline_decode(&rctxt->rc_stream, + rpcrdma_readseg_maxsz * sizeof(*p)); + if (!p) + return false; + if (first) { - position = be32_to_cpup(p++); + position = be32_to_cpup(p); first = false; - } else if (be32_to_cpup(p++) != position) { - return NULL; + } else if (be32_to_cpup(p) != position) { + return false; } - p++; /* handle */ - if (be32_to_cpup(p++) > MAX_BYTES_SPECIAL_SEG) - return NULL; - p += 2; /* offset */ + p += 2; + len += be32_to_cpup(p); - if (p > end) - return NULL; + p = xdr_inline_decode(&rctxt->rc_stream, sizeof(*p)); + if (!p) + return false; } - return p; + return len <= MAX_BYTES_SPECIAL_CHUNK; } /* The segment count is limited to how many segments can @@ -413,67 +426,94 @@ static __be32 *xdr_check_read_list(__be32 *p, const __be32 *end) * buffer. That's about 60 Write segments for a 1KB inline * threshold. */ -static __be32 *xdr_check_write_chunk(__be32 *p, const __be32 *end, - u32 maxlen) +static bool xdr_check_write_chunk(struct svc_rdma_recv_ctxt *rctxt, + u32 maxlen) { - u32 i, segcount; + u32 i, segcount, total; + __be32 *p; + + p = xdr_inline_decode(&rctxt->rc_stream, sizeof(*p)); + if (!p) + return false; + segcount = be32_to_cpup(p); - segcount = be32_to_cpup(p++); + total = 0; for (i = 0; i < segcount; i++) { - p++; /* handle */ - if (be32_to_cpup(p++) > maxlen) - return NULL; - p += 2; /* offset */ + u32 handle, length; + u64 offset; - if (p > end) - return NULL; - } + p = xdr_inline_decode(&rctxt->rc_stream, + rpcrdma_segment_maxsz * sizeof(*p)); + if (!p) + return false; + + handle = be32_to_cpup(p++); + length = be32_to_cpup(p++); + xdr_decode_hyper(p, &offset); + trace_svcrdma_decode_wseg(handle, length, offset); - return p; + total += length; + } + return total <= maxlen; } /* Sanity check the Write list. * * Implementation limits: - * - This implementation supports only one Write chunk. + * - This implementation currently supports only one Write chunk. * * Sanity checks: - * - Write list does not overflow buffer. - * - Segment size limited by largest NFS data payload. - * - * Returns pointer to the following Reply chunk. + * - Write list does not overflow Receive buffer. + * - Chunk size limited by largest NFS data payload. + * + * Return values: + * %true: Write list is valid. @rctxt's xdr_stream is updated + * to point to the first byte past the Write list. + * %false: Write list is corrupt. @rctxt's xdr_stream is left + * in an unknown state. */ -static __be32 *xdr_check_write_list(__be32 *p, const __be32 *end) +static bool xdr_check_write_list(struct svc_rdma_recv_ctxt *rctxt) { - u32 chcount; + u32 chcount = 0; + __be32 *p; - chcount = 0; - while (*p++ != xdr_zero) { - p = xdr_check_write_chunk(p, end, MAX_BYTES_WRITE_SEG); + p = xdr_inline_decode(&rctxt->rc_stream, sizeof(*p)); + if (!p) + return false; + while (*p != xdr_zero) { + if (!xdr_check_write_chunk(rctxt, MAX_BYTES_WRITE_CHUNK)) + return false; + ++chcount; + p = xdr_inline_decode(&rctxt->rc_stream, sizeof(*p)); if (!p) - return NULL; - if (chcount++ > 1) - return NULL; + return false; } - return p; + return chcount < 2; } /* Sanity check the Reply chunk. * * Sanity checks: - * - Reply chunk does not overflow buffer. - * - Segment size limited by largest NFS data payload. - * - * Returns pointer to the following RPC header. + * - Reply chunk does not overflow Receive buffer. + * - Chunk size limited by largest NFS data payload. + * + * Return values: + * %true: Reply chunk is valid. @rctxt's xdr_stream is updated + * to point to the first byte past the Reply chunk. + * %false: Reply chunk is corrupt. @rctxt's xdr_stream is left + * in an unknown state. */ -static __be32 *xdr_check_reply_chunk(__be32 *p, const __be32 *end) +static bool xdr_check_reply_chunk(struct svc_rdma_recv_ctxt *rctxt) { - if (*p++ != xdr_zero) { - p = xdr_check_write_chunk(p, end, MAX_BYTES_SPECIAL_SEG); - if (!p) - return NULL; - } - return p; + __be32 *p; + + p = xdr_inline_decode(&rctxt->rc_stream, sizeof(*p)); + if (!p) + return false; + if (*p != xdr_zero) + if (!xdr_check_write_chunk(rctxt, MAX_BYTES_SPECIAL_CHUNK)) + return false; + return true; } /* RPC-over-RDMA Version One private extension: Remote Invalidation. @@ -538,60 +578,61 @@ static void svc_rdma_get_inv_rkey(struct svcxprt_rdma *rdma, ctxt->rc_inv_rkey = be32_to_cpu(inv_rkey); } -/* On entry, xdr->head[0].iov_base points to first byte in the - * RPC-over-RDMA header. +/** + * svc_rdma_xdr_decode_req - Decode the transport header + * @rq_arg: xdr_buf containing ingress RPC/RDMA message + * @rctxt: state of decoding + * + * On entry, xdr->head[0].iov_base points to first byte of the + * RPC-over-RDMA transport header. * * On successful exit, head[0] points to first byte past the * RPC-over-RDMA header. For RDMA_MSG, this is the RPC message. + * * The length of the RPC-over-RDMA header is returned. * * Assumptions: * - The transport header is entirely contained in the head iovec. */ -static int svc_rdma_xdr_decode_req(struct xdr_buf *rq_arg) +static int svc_rdma_xdr_decode_req(struct xdr_buf *rq_arg, + struct svc_rdma_recv_ctxt *rctxt) { - __be32 *p, *end, *rdma_argp; + __be32 *p, *rdma_argp; unsigned int hdr_len; - /* Verify that there's enough bytes for header + something */ - if (rq_arg->len <= RPCRDMA_HDRLEN_ERR) - goto out_short; - rdma_argp = rq_arg->head[0].iov_base; - if (*(rdma_argp + 1) != rpcrdma_version) - goto out_version; + xdr_init_decode(&rctxt->rc_stream, rq_arg, rdma_argp, NULL); - switch (*(rdma_argp + 3)) { + p = xdr_inline_decode(&rctxt->rc_stream, + rpcrdma_fixed_maxsz * sizeof(*p)); + if (unlikely(!p)) + goto out_short; + p++; + if (*p != rpcrdma_version) + goto out_version; + p += 2; + switch (*p) { case rdma_msg: break; case rdma_nomsg: break; - case rdma_done: goto out_drop; - case rdma_error: goto out_drop; - default: goto out_proc; } - end = (__be32 *)((unsigned long)rdma_argp + rq_arg->len); - p = xdr_check_read_list(rdma_argp + 4, end); - if (!p) + if (!xdr_check_read_list(rctxt)) goto out_inval; - p = xdr_check_write_list(p, end); - if (!p) - goto out_inval; - p = xdr_check_reply_chunk(p, end); - if (!p) + if (!xdr_check_write_list(rctxt)) goto out_inval; - if (p > end) + if (!xdr_check_reply_chunk(rctxt)) goto out_inval; - rq_arg->head[0].iov_base = p; - hdr_len = (unsigned long)p - (unsigned long)rdma_argp; + rq_arg->head[0].iov_base = rctxt->rc_stream.p; + hdr_len = xdr_stream_pos(&rctxt->rc_stream); rq_arg->head[0].iov_len -= hdr_len; rq_arg->len -= hdr_len; trace_svcrdma_decode_rqst(rdma_argp, hdr_len); @@ -786,7 +827,7 @@ int svc_rdma_recvfrom(struct svc_rqst *rqstp) rqstp->rq_next_page = rqstp->rq_respages; p = (__be32 *)rqstp->rq_arg.head[0].iov_base; - ret = svc_rdma_xdr_decode_req(&rqstp->rq_arg); + ret = svc_rdma_xdr_decode_req(&rqstp->rq_arg, ctxt); if (ret < 0) goto out_err; if (ret == 0) From patchwork Fri Feb 28 00:31:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 11411265 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A50EF1805 for ; Fri, 28 Feb 2020 00:31:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 84E51246AA for ; Fri, 28 Feb 2020 00:31:24 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="mRVabJ4J" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730135AbgB1AbX (ORCPT ); Thu, 27 Feb 2020 19:31:23 -0500 Received: from mail-pj1-f68.google.com ([209.85.216.68]:51527 "EHLO mail-pj1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730117AbgB1AbX (ORCPT ); Thu, 27 Feb 2020 19:31:23 -0500 Received: by mail-pj1-f68.google.com with SMTP id fa20so514496pjb.1; Thu, 27 Feb 2020 16:31:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=AzkiZb8px8kNE2AdDZkFKa0fWQaErrSv90fuEowsxTQ=; b=mRVabJ4JXPv/zW57Td4ZCaBbL0pHGtSygEHoOZ1MqZPRKlqwWbobCJzuEWXdX2cDT/ 1u3G+onZTuz3y4KI/xZGDB6YdB9Fr0Ay2dVf+jfzqkgaokcyXIkTV0vq9OArV9lqv0mI STsXbcU+blAX787Ep5GiPjnPUBohvXlN20MCTQXNyMMRqFA2ti3HNQzUawiJA7c3dSdM GzMv6NI23FgG5oR+8y5QOQeSHI1IpFw7Qe1ffS3/QRELEKuK9Ion/vtxUl9Wp/zp9X4R /OVFPmkk7bumR7oVMLe462oj0i2Hfu8MEi777QBxehOTZyGc1cJTzL99E7APX4+oEf9P TaFw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=AzkiZb8px8kNE2AdDZkFKa0fWQaErrSv90fuEowsxTQ=; b=HisRkyhkALDa7pGWfkFWoRdJNkgKY6ws7orGe52S6Tz04icd+Sx90rO0sIUHn5UkDT DCA4BZS7bbYVGmFGU2/pHwtIWTcy2FgBVLzsBL8qe0Gc8FNrMDTN1cyj8kntngKaU6A4 2ekhlCEzORYVfvZxXPe6KHpEOUBdlxlIHOJHBhB5GH6WIL0pdtHDcF1pkdyskpB7osYz JLeSKGaoQVXTwwH0ntrC9J1dyRQO7Bnh1KH06AauYKvd+t9si8AvQRNKSrC57Wmbuuty 0+QfRUdYMtbktW90wdxIm4zhGSHc5UPaC525avUCOOEUJ3Rx+W8mSdBIQQP4N8+vIkv5 8iyQ== X-Gm-Message-State: APjAAAWQxzPzc8q0P21rfaEiOLAcNwB41f8TCLyYu1OxKaMiOEuUp5Ve wG2uQRbZAVBo1tGx4ZErz+Y= X-Google-Smtp-Source: APXvYqwbWaZiX9/pOpDCzcJhnDjqgXT+4oGxEDwJPVxYul7BjvNscUdeznrDFAP78IUFHcYJS/6T9w== X-Received: by 2002:a17:902:8642:: with SMTP id y2mr1387721plt.306.1582849882259; Thu, 27 Feb 2020 16:31:22 -0800 (PST) Received: from seurat29.1015granger.net (ip-184-250-247-225.sanjca.spcsdns.net. [184.250.247.225]) by smtp.gmail.com with ESMTPSA id q13sm9402296pfn.162.2020.02.27.16.31.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 Feb 2020 16:31:21 -0800 (PST) Subject: [PATCH v1 08/16] svcrdma: De-duplicate code that locates Write and Reply chunks From: Chuck Lever To: bfields@fieldses.org Cc: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Thu, 27 Feb 2020 19:31:20 -0500 Message-ID: <158284988067.38468.555434593805309782.stgit@seurat29.1015granger.net> In-Reply-To: <158284930886.38468.17045380766660946827.stgit@seurat29.1015granger.net> References: <158284930886.38468.17045380766660946827.stgit@seurat29.1015granger.net> User-Agent: StGit/unknown-version MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Cache the locations of the Requester-provided Write list and Reply chunk so that the Send path doesn't need to parse the Call header again. Signed-off-by: Chuck Lever --- include/linux/sunrpc/svc_rdma.h | 2 ++ net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 8 ++++++- net/sunrpc/xprtrdma/svc_rdma_sendto.c | 38 +++---------------------------- 3 files changed, 13 insertions(+), 35 deletions(-) diff --git a/include/linux/sunrpc/svc_rdma.h b/include/linux/sunrpc/svc_rdma.h index c790dbb0dd90..e714e4d90ac5 100644 --- a/include/linux/sunrpc/svc_rdma.h +++ b/include/linux/sunrpc/svc_rdma.h @@ -138,6 +138,8 @@ struct svc_rdma_recv_ctxt { unsigned int rc_page_count; unsigned int rc_hdr_count; u32 rc_inv_rkey; + __be32 *rc_write_list; + __be32 *rc_reply_chunk; unsigned int rc_read_payload_offset; unsigned int rc_read_payload_length; struct page *rc_pages[RPCSVC_MAXPAGES]; diff --git a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c index 7db151b7dfd3..f9a3c413491c 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c +++ b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c @@ -480,6 +480,7 @@ static bool xdr_check_write_list(struct svc_rdma_recv_ctxt *rctxt) p = xdr_inline_decode(&rctxt->rc_stream, sizeof(*p)); if (!p) return false; + rctxt->rc_write_list = p; while (*p != xdr_zero) { if (!xdr_check_write_chunk(rctxt, MAX_BYTES_WRITE_CHUNK)) return false; @@ -488,6 +489,8 @@ static bool xdr_check_write_list(struct svc_rdma_recv_ctxt *rctxt) if (!p) return false; } + if (!chcount) + rctxt->rc_write_list = NULL; return chcount < 2; } @@ -510,9 +513,12 @@ static bool xdr_check_reply_chunk(struct svc_rdma_recv_ctxt *rctxt) p = xdr_inline_decode(&rctxt->rc_stream, sizeof(*p)); if (!p) return false; - if (*p != xdr_zero) + rctxt->rc_reply_chunk = p; + if (*p != xdr_zero) { if (!xdr_check_write_chunk(rctxt, MAX_BYTES_SPECIAL_CHUNK)) return false; + } else + rctxt->rc_reply_chunk = NULL; return true; } diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c index 4add875277f8..94895635c007 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c +++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c @@ -449,36 +449,6 @@ static void svc_rdma_xdr_encode_reply_chunk(__be32 *rdma_resp, __be32 *rp_ch, xdr_encode_write_chunk(p, rp_ch, consumed); } -/* Parse the RPC Call's transport header. - */ -static void svc_rdma_get_write_arrays(__be32 *rdma_argp, - __be32 **write, __be32 **reply) -{ - __be32 *p; - - p = rdma_argp + rpcrdma_fixed_maxsz; - - /* Read list */ - while (*p++ != xdr_zero) - p += 5; - - /* Write list */ - if (*p != xdr_zero) { - *write = p; - while (*p++ != xdr_zero) - p += 1 + be32_to_cpu(*p) * 4; - } else { - *write = NULL; - p++; - } - - /* Reply chunk */ - if (*p != xdr_zero) - *reply = p; - else - *reply = NULL; -} - static int svc_rdma_dma_map_page(struct svcxprt_rdma *rdma, struct svc_rdma_send_ctxt *ctxt, struct page *page, @@ -813,14 +783,14 @@ int svc_rdma_sendto(struct svc_rqst *rqstp) struct svcxprt_rdma *rdma = container_of(xprt, struct svcxprt_rdma, sc_xprt); struct svc_rdma_recv_ctxt *rctxt = rqstp->rq_xprt_ctxt; - __be32 *p, *rdma_argp, *rdma_resp, *wr_lst, *rp_ch; + __be32 *rdma_argp = rctxt->rc_recv_buf; + __be32 *wr_lst = rctxt->rc_write_list; + __be32 *rp_ch = rctxt->rc_reply_chunk; struct xdr_buf *xdr = &rqstp->rq_res; struct svc_rdma_send_ctxt *sctxt; + __be32 *p, *rdma_resp; int ret; - rdma_argp = rctxt->rc_recv_buf; - svc_rdma_get_write_arrays(rdma_argp, &wr_lst, &rp_ch); - /* Create the RDMA response header. xprt->xpt_mutex, * acquired in svc_send(), serializes RPC replies. The * code path below that inserts the credit grant value From patchwork Fri Feb 28 00:31:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 11411269 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BDF801805 for ; Fri, 28 Feb 2020 00:31:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 9E201246A8 for ; Fri, 28 Feb 2020 00:31:32 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="BTcc7nJ9" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730341AbgB1Abc (ORCPT ); Thu, 27 Feb 2020 19:31:32 -0500 Received: from mail-pl1-f195.google.com ([209.85.214.195]:43461 "EHLO mail-pl1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730117AbgB1Abb (ORCPT ); Thu, 27 Feb 2020 19:31:31 -0500 Received: by mail-pl1-f195.google.com with SMTP id p11so489464plq.10; Thu, 27 Feb 2020 16:31:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=JlXp/UrJNcRE29eMFkaSctlpVhJd/rHOgcmwyTldv80=; b=BTcc7nJ9v9S0yxYdj2pEqfn37kuTNISQHT0QSxgTJYATWIZFDoq900KoJVnVQQ52WR YDn5dpN2Q09fRErlTsoyOpwQhcvxKEIFdJHMJREXaSpSv4720TWmg72LFX1zm6gXgO30 M399oPIHZd9TV0aC4Xrx9I6SXY91KOaO477K+5t0f8aeXaxavzoOcNRIUUMh82+db7R6 CMBke5JwVyOM2vZ/X6wpOzf43sVCvj5UZhgCsXXUvql6IwdaGKYBmqbk5GsbjoslTa1d GDxCPN3Y3hE3jBkcXz+ywuNlUBC6OUXy7SkIB4RDbrwbvck3sxWVNKE6upKNpapLvbMa OazQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=JlXp/UrJNcRE29eMFkaSctlpVhJd/rHOgcmwyTldv80=; b=uN3j1/ziz9QasAnY2HbAx/iDn3JtNv9mw0U2oSalqSEFPV2bCrqDhc3QTE/hNDa2ez Tez95MANTY7o7MzM9K+4ytGNc/SP8DspDPp8FmRABm/P8llYuwm08FHh8vKMu4eJ2QYc UmjVi+/LYGXILeKYxBweL+t8Bq+Qoi8+3MC7QQAfxTJcqyL5iQFjR49mpFJ76NuN8XhJ RfCHAQejFpXXe8jZ+5Ofv6IRFvaNKn5UQTwWpK/A9FJpoDUcgo+IltNpagxEEgpDVtmp draUBF0mBIMZJOdQjZ4jJQBXwUaTAQwfoSd9JQimq0orZHoHEed5U/8k2YkomaCY8fwN XnTw== X-Gm-Message-State: APjAAAVPCY79V2V33UiBg3Gmyc7N1llSHxul6ixVuEHGPrnXFreywJrs raxnRLbGnhk4j0Er7x05OxFC4a21670= X-Google-Smtp-Source: APXvYqyiZJnLyKkHFb83oQv298tbsfT0shCW9bG0hv0O6iqMb1O2OSyRzhNOqhwUzSsEXkJ/RTb+bg== X-Received: by 2002:a17:90a:348a:: with SMTP id p10mr1713700pjb.120.1582849889179; Thu, 27 Feb 2020 16:31:29 -0800 (PST) Received: from seurat29.1015granger.net (ip-184-250-247-225.sanjca.spcsdns.net. [184.250.247.225]) by smtp.gmail.com with ESMTPSA id m12sm7685190pjf.25.2020.02.27.16.31.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 Feb 2020 16:31:28 -0800 (PST) Subject: [PATCH v1 09/16] svcrdma: Update synopsis of svc_rdma_send_reply_chunk() From: Chuck Lever To: bfields@fieldses.org Cc: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Thu, 27 Feb 2020 19:31:27 -0500 Message-ID: <158284988744.38468.439479906974116037.stgit@seurat29.1015granger.net> In-Reply-To: <158284930886.38468.17045380766660946827.stgit@seurat29.1015granger.net> References: <158284930886.38468.17045380766660946827.stgit@seurat29.1015granger.net> User-Agent: StGit/unknown-version MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Preparing for subsequent patches, no behavior change expected. Pass the RPC Call's svc_rdma_recv_ctxt deeper into the sendto() path. This enables passing more information about Requester- provided Write and Reply chunks into the lower-level send functions. Signed-off-by: Chuck Lever --- include/linux/sunrpc/svc_rdma.h | 2 +- net/sunrpc/xprtrdma/svc_rdma_rw.c | 12 ++++++------ net/sunrpc/xprtrdma/svc_rdma_sendto.c | 2 +- 3 files changed, 8 insertions(+), 8 deletions(-) diff --git a/include/linux/sunrpc/svc_rdma.h b/include/linux/sunrpc/svc_rdma.h index e714e4d90ac5..42b68126cc60 100644 --- a/include/linux/sunrpc/svc_rdma.h +++ b/include/linux/sunrpc/svc_rdma.h @@ -179,7 +179,7 @@ extern int svc_rdma_send_write_chunk(struct svcxprt_rdma *rdma, unsigned int offset, unsigned long length); extern int svc_rdma_send_reply_chunk(struct svcxprt_rdma *rdma, - __be32 *rp_ch, bool writelist, + const struct svc_rdma_recv_ctxt *rctxt, struct xdr_buf *xdr); /* svc_rdma_sendto.c */ diff --git a/net/sunrpc/xprtrdma/svc_rdma_rw.c b/net/sunrpc/xprtrdma/svc_rdma_rw.c index b0ac535c8728..9d854755d78d 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_rw.c +++ b/net/sunrpc/xprtrdma/svc_rdma_rw.c @@ -545,8 +545,7 @@ int svc_rdma_send_write_chunk(struct svcxprt_rdma *rdma, __be32 *wr_ch, /** * svc_rdma_send_reply_chunk - Write all segments in the Reply chunk * @rdma: controlling RDMA transport - * @rp_ch: Reply chunk provided by client - * @writelist: true if client provided a Write list + * @rctxt: Write and Reply chunks from client * @xdr: xdr_buf containing an RPC Reply * * Returns a non-negative number of bytes the chunk consumed, or @@ -556,13 +555,14 @@ int svc_rdma_send_write_chunk(struct svcxprt_rdma *rdma, __be32 *wr_ch, * %-ENOTCONN if posting failed (connection is lost), * %-EIO if rdma_rw initialization failed (DMA mapping, etc). */ -int svc_rdma_send_reply_chunk(struct svcxprt_rdma *rdma, __be32 *rp_ch, - bool writelist, struct xdr_buf *xdr) +int svc_rdma_send_reply_chunk(struct svcxprt_rdma *rdma, + const struct svc_rdma_recv_ctxt *rctxt, + struct xdr_buf *xdr) { struct svc_rdma_write_info *info; int consumed, ret; - info = svc_rdma_write_info_alloc(rdma, rp_ch); + info = svc_rdma_write_info_alloc(rdma, rctxt->rc_reply_chunk); if (!info) return -ENOMEM; @@ -574,7 +574,7 @@ int svc_rdma_send_reply_chunk(struct svcxprt_rdma *rdma, __be32 *rp_ch, /* Send the page list in the Reply chunk only if the * client did not provide Write chunks. */ - if (!writelist && xdr->page_len) { + if (!rctxt->rc_write_list && xdr->page_len) { ret = svc_rdma_send_xdr_pagelist(info, xdr, xdr->head[0].iov_len, xdr->page_len); diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c index 94895635c007..0b6ff55b1ab1 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c +++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c @@ -833,7 +833,7 @@ int svc_rdma_sendto(struct svc_rqst *rqstp) svc_rdma_xdr_encode_write_list(rdma_resp, wr_lst, ret); } if (rp_ch) { - ret = svc_rdma_send_reply_chunk(rdma, rp_ch, wr_lst, xdr); + ret = svc_rdma_send_reply_chunk(rdma, rctxt, &rqstp->rq_res); if (ret < 0) goto err2; svc_rdma_xdr_encode_reply_chunk(rdma_resp, rp_ch, ret); From patchwork Fri Feb 28 00:31:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 11411273 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1B4951805 for ; Fri, 28 Feb 2020 00:31:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id EF04F246A1 for ; Fri, 28 Feb 2020 00:31:39 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="DGUOefFZ" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730346AbgB1Abj (ORCPT ); Thu, 27 Feb 2020 19:31:39 -0500 Received: from mail-pl1-f193.google.com ([209.85.214.193]:33772 "EHLO mail-pl1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730117AbgB1Abj (ORCPT ); Thu, 27 Feb 2020 19:31:39 -0500 Received: by mail-pl1-f193.google.com with SMTP id ay11so510166plb.0; Thu, 27 Feb 2020 16:31:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=HqCLHXBDi5YxU/ZH3rQjRlio90XPc518A9HBjf6DjUY=; b=DGUOefFZ7jtZGwbwbgx8KLlzw/bQOTfBuxZtP1/SE0J95P/mihODUu21rIY/HpZkF8 IPqi/pdUgt9ReFpMAJuMVHx62UgnEN8ysv/GhgZ25BUTmg8X4ZtAmWH3rIfvzbxCNuI7 ctLkwHciiA4jNyjHmbmmLFV9KFUw95IRg5P0Vd8pqIobRZnaOfAR9xU4j+EHNYsoKEgW ezE1hcRYJld7qHX8z4qMjA1vdxxuPCzucGiiwM6rDX+/F5KbYZ+8Tj0WueTUUdmZu1tc PAsbSdiBVfuqqm5EWhTpSF6ERaUiAKd5lrIr7d7XNY2XDP9jqvXEzfg7Nr71iohjueYg j1BA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=HqCLHXBDi5YxU/ZH3rQjRlio90XPc518A9HBjf6DjUY=; b=Hp5C7Hk4ROqjlVMpkZFnL8VbrpFDxVDmC6Sm6g3igmEkqoNhqxyxBza4tP3dxdeCYg Bm4k+Yx5DdIS9J2bjO3VHUn+sXC8UV/s5/BFWMDVqLUWyNvlCbMAtUYQFjKOeFzgoAVJ 5Bz2LWzRTB7o2isZ/NQ81hjdSXE5ZGkpI12w5FTo+YiwdE5PhW+GxFP6YoyK2r03AQza EmK9x8o1Nk+MMXm8GCwPTTsPHt9mON+F8eBTJfW8UZHU43iNET8064JDvIfFl/TUGfSJ d8vaEts/lt6G+dbbqGBSo7FNV3Xh18bpsZ+qi1Mch5k8nfKpwfFNKeml9rou4LO9x5nr M4EQ== X-Gm-Message-State: APjAAAVhjg62hRt37jgbXT5jJs4ZFnywTPh9JorIB6u/W5/r1PRbA/Sv FHQWXXIcqa9UP/FKJ8Uhz/MUQSASfWY= X-Google-Smtp-Source: APXvYqxus+jaaiOrufGiYBcLYd26bruKEY1QJjLEKQEhOIao55czhWYcsVSRAsMeTL4UbOdjSv5JsA== X-Received: by 2002:a17:90b:3115:: with SMTP id gc21mr1648926pjb.54.1582849896045; Thu, 27 Feb 2020 16:31:36 -0800 (PST) Received: from seurat29.1015granger.net (ip-184-250-247-225.sanjca.spcsdns.net. [184.250.247.225]) by smtp.gmail.com with ESMTPSA id 200sm8585322pfz.121.2020.02.27.16.31.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 Feb 2020 16:31:35 -0800 (PST) Subject: [PATCH v1 10/16] svcrdma: Update synopsis of svc_rdma_map_reply_msg() From: Chuck Lever To: bfields@fieldses.org Cc: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Thu, 27 Feb 2020 19:31:34 -0500 Message-ID: <158284989441.38468.8027058905721344778.stgit@seurat29.1015granger.net> In-Reply-To: <158284930886.38468.17045380766660946827.stgit@seurat29.1015granger.net> References: <158284930886.38468.17045380766660946827.stgit@seurat29.1015granger.net> User-Agent: StGit/unknown-version MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Preparing for subsequent patches, no behavior change expected. Pass the RPC Call's svc_rdma_recv_ctxt deeper into the sendto() path. This enables passing more information about Requester- provided Write and Reply chunks into those lower-level functions. Signed-off-by: Chuck Lever --- include/linux/sunrpc/svc_rdma.h | 5 +- net/sunrpc/xprtrdma/svc_rdma_backchannel.c | 2 - net/sunrpc/xprtrdma/svc_rdma_sendto.c | 80 +++++++++++++++++----------- 3 files changed, 52 insertions(+), 35 deletions(-) diff --git a/include/linux/sunrpc/svc_rdma.h b/include/linux/sunrpc/svc_rdma.h index 42b68126cc60..c506732886b3 100644 --- a/include/linux/sunrpc/svc_rdma.h +++ b/include/linux/sunrpc/svc_rdma.h @@ -193,8 +193,9 @@ extern void svc_rdma_sync_reply_hdr(struct svcxprt_rdma *rdma, struct svc_rdma_send_ctxt *ctxt, unsigned int len); extern int svc_rdma_map_reply_msg(struct svcxprt_rdma *rdma, - struct svc_rdma_send_ctxt *ctxt, - struct xdr_buf *xdr, __be32 *wr_lst); + struct svc_rdma_send_ctxt *sctxt, + const struct svc_rdma_recv_ctxt *rctxt, + struct xdr_buf *xdr); extern int svc_rdma_sendto(struct svc_rqst *); extern int svc_rdma_read_payload(struct svc_rqst *rqstp, unsigned int offset, unsigned int length); diff --git a/net/sunrpc/xprtrdma/svc_rdma_backchannel.c b/net/sunrpc/xprtrdma/svc_rdma_backchannel.c index 908e78bb87c6..ce1a7a706f36 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_backchannel.c +++ b/net/sunrpc/xprtrdma/svc_rdma_backchannel.c @@ -117,7 +117,7 @@ static int svc_rdma_bc_sendto(struct svcxprt_rdma *rdma, { int ret; - ret = svc_rdma_map_reply_msg(rdma, ctxt, &rqst->rq_snd_buf, NULL); + ret = svc_rdma_map_reply_msg(rdma, ctxt, NULL, &rqst->rq_snd_buf); if (ret < 0) return -EIO; diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c index 0b6ff55b1ab1..bc8863342878 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c +++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c @@ -502,13 +502,19 @@ void svc_rdma_sync_reply_hdr(struct svcxprt_rdma *rdma, DMA_TO_DEVICE); } -/* If the xdr_buf has more elements than the device can - * transmit in a single RDMA Send, then the reply will - * have to be copied into a bounce buffer. +/** + * svc_rdma_pull_up_needed - Determine whether to use pull-up + * @rdma: controlling transport + * @rctxt: Write and Reply chunks provided by client + * @xdr: xdr_buf containing RPC message to transmit + * + * Returns: + * %true if pull-up must be used + * %false otherwise */ static bool svc_rdma_pull_up_needed(struct svcxprt_rdma *rdma, - struct xdr_buf *xdr, - __be32 *wr_lst) + const struct svc_rdma_recv_ctxt *rctxt, + struct xdr_buf *xdr) { int elements; @@ -516,7 +522,7 @@ static bool svc_rdma_pull_up_needed(struct svcxprt_rdma *rdma, elements = 1; /* xdr->pages */ - if (!wr_lst) { + if (!rctxt || !rctxt->rc_write_list) { unsigned int remaining; unsigned long pageoff; @@ -538,26 +544,35 @@ static bool svc_rdma_pull_up_needed(struct svcxprt_rdma *rdma, return elements >= rdma->sc_max_send_sges; } -/* The device is not capable of sending the reply directly. - * Assemble the elements of @xdr into the transport header - * buffer. +/** + * svc_rdma_pull_up_reply_msg - Copy Reply into a single buffer + * @rdma: controlling transport + * @sctxt: send_ctxt for the Send WR + * @rctxt: Write and Reply chunks provided by client + * @xdr: prepared xdr_buf containing RPC message + * + * The device is not capable of sending the reply directly. + * Assemble the elements of @xdr into the transport header buffer. + * + * Returns zero on success, or a negative errno on failure. */ static int svc_rdma_pull_up_reply_msg(struct svcxprt_rdma *rdma, - struct svc_rdma_send_ctxt *ctxt, - struct xdr_buf *xdr, __be32 *wr_lst) + struct svc_rdma_send_ctxt *sctxt, + const struct svc_rdma_recv_ctxt *rctxt, + struct xdr_buf *xdr) { unsigned char *dst, *tailbase; unsigned int taillen; - dst = ctxt->sc_xprt_buf; - dst += ctxt->sc_sges[0].length; + dst = sctxt->sc_xprt_buf; + dst += sctxt->sc_sges[0].length; memcpy(dst, xdr->head[0].iov_base, xdr->head[0].iov_len); dst += xdr->head[0].iov_len; tailbase = xdr->tail[0].iov_base; taillen = xdr->tail[0].iov_len; - if (wr_lst) { + if (rctxt && rctxt->rc_write_list) { u32 xdrpad; xdrpad = xdr_pad_size(xdr->page_len); @@ -586,10 +601,10 @@ static int svc_rdma_pull_up_reply_msg(struct svcxprt_rdma *rdma, if (taillen) memcpy(dst, tailbase, taillen); - ctxt->sc_sges[0].length += xdr->len; + sctxt->sc_sges[0].length += xdr->len; ib_dma_sync_single_for_device(rdma->sc_pd->device, - ctxt->sc_sges[0].addr, - ctxt->sc_sges[0].length, + sctxt->sc_sges[0].addr, + sctxt->sc_sges[0].length, DMA_TO_DEVICE); return 0; @@ -597,9 +612,9 @@ static int svc_rdma_pull_up_reply_msg(struct svcxprt_rdma *rdma, /* svc_rdma_map_reply_msg - Map the buffer holding RPC message * @rdma: controlling transport - * @ctxt: send_ctxt for the Send WR + * @sctxt: send_ctxt for the Send WR + * @rctxt: Write and Reply chunks provided by client * @xdr: prepared xdr_buf containing RPC message - * @wr_lst: pointer to Call header's Write list, or NULL * * Load the xdr_buf into the ctxt's sge array, and DMA map each * element as it is added. @@ -607,8 +622,9 @@ static int svc_rdma_pull_up_reply_msg(struct svcxprt_rdma *rdma, * Returns zero on success, or a negative errno on failure. */ int svc_rdma_map_reply_msg(struct svcxprt_rdma *rdma, - struct svc_rdma_send_ctxt *ctxt, - struct xdr_buf *xdr, __be32 *wr_lst) + struct svc_rdma_send_ctxt *sctxt, + const struct svc_rdma_recv_ctxt *rctxt, + struct xdr_buf *xdr) { unsigned int len, remaining; unsigned long page_off; @@ -617,11 +633,11 @@ int svc_rdma_map_reply_msg(struct svcxprt_rdma *rdma, u32 xdr_pad; int ret; - if (svc_rdma_pull_up_needed(rdma, xdr, wr_lst)) - return svc_rdma_pull_up_reply_msg(rdma, ctxt, xdr, wr_lst); + if (svc_rdma_pull_up_needed(rdma, rctxt, xdr)) + return svc_rdma_pull_up_reply_msg(rdma, sctxt, rctxt, xdr); - ++ctxt->sc_cur_sge_no; - ret = svc_rdma_dma_map_buf(rdma, ctxt, + ++sctxt->sc_cur_sge_no; + ret = svc_rdma_dma_map_buf(rdma, sctxt, xdr->head[0].iov_base, xdr->head[0].iov_len); if (ret < 0) @@ -632,7 +648,7 @@ int svc_rdma_map_reply_msg(struct svcxprt_rdma *rdma, * have added XDR padding in the tail buffer, and that * should not be included inline. */ - if (wr_lst) { + if (rctxt && rctxt->rc_write_list) { base = xdr->tail[0].iov_base; len = xdr->tail[0].iov_len; xdr_pad = xdr_pad_size(xdr->page_len); @@ -651,8 +667,8 @@ int svc_rdma_map_reply_msg(struct svcxprt_rdma *rdma, while (remaining) { len = min_t(u32, PAGE_SIZE - page_off, remaining); - ++ctxt->sc_cur_sge_no; - ret = svc_rdma_dma_map_page(rdma, ctxt, *ppages++, + ++sctxt->sc_cur_sge_no; + ret = svc_rdma_dma_map_page(rdma, sctxt, *ppages++, page_off, len); if (ret < 0) return ret; @@ -665,8 +681,8 @@ int svc_rdma_map_reply_msg(struct svcxprt_rdma *rdma, len = xdr->tail[0].iov_len; tail: if (len) { - ++ctxt->sc_cur_sge_no; - ret = svc_rdma_dma_map_buf(rdma, ctxt, base, len); + ++sctxt->sc_cur_sge_no; + ret = svc_rdma_dma_map_buf(rdma, sctxt, base, len); if (ret < 0) return ret; } @@ -720,8 +736,8 @@ static int svc_rdma_send_reply_msg(struct svcxprt_rdma *rdma, int ret; if (!rp_ch) { - ret = svc_rdma_map_reply_msg(rdma, sctxt, - &rqstp->rq_res, wr_lst); + ret = svc_rdma_map_reply_msg(rdma, sctxt, rctxt, + &rqstp->rq_res); if (ret < 0) return ret; } From patchwork Fri Feb 28 00:31:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 11411277 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5C52D14B7 for ; Fri, 28 Feb 2020 00:31:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 3B2B7246A1 for ; Fri, 28 Feb 2020 00:31:46 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="C0cRiBpq" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730349AbgB1Abp (ORCPT ); Thu, 27 Feb 2020 19:31:45 -0500 Received: from mail-pj1-f68.google.com ([209.85.216.68]:53331 "EHLO mail-pj1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730117AbgB1Abp (ORCPT ); Thu, 27 Feb 2020 19:31:45 -0500 Received: by mail-pj1-f68.google.com with SMTP id i11so443819pju.3; Thu, 27 Feb 2020 16:31:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=WwqATJFNRY0o8CWDMEbkFwK645hrNctY2DMp0A7motM=; b=C0cRiBpqhONdb/MMZ8K9mptrhhXFYIQ0cBesvpz+Mifh7bYWwlokuGkFZ2VE9LdAMD Djbk1KkDxUWXaiPqIJFq+wlMQn2HZWRGWqsdFqxAmXOC+SoAhWF1Z3u78uL5ISxUUlV/ DQngqUHNupej9AC2G8rX6oC1qfGPQdxk+4n+FtvkUCXOJ7aP+xCB3jLCUwCq8FSoDQt3 SpUgE2bwwmrxjQgsU+LSo7pVUG9Dwzq2QoGkz7iqGaWJzjozAfS4RfdXJtH0vqpZxi86 3rVAbvnkuXjV3WeeEwW7hM5+J6eC0u3uAKo19Io4QnH59J3MWwCTmjgi1PVmJRGk/X/d SNvw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=WwqATJFNRY0o8CWDMEbkFwK645hrNctY2DMp0A7motM=; b=IlAS2g1+8Ks+OZS5kacoUOgAhLxDK7Lkxd9A2yCdfIWgKgO97R/t7HUF8vM0piwWL7 Su28w0cGtH85lleWKRPvmsBu72VFH8td9jc9vogRohymCdIHJ3cAfBdTDjlVQoMM3Jw4 uxwPOS5GM3R5WAIkAJA+1j4+txBARj3mDGeeHlIyh3zU5pHiWngWdl2MUmrith1jiDuM Ncxbcr7UiFRPUmHISxldjz65WIyvo33r9Gn21c4D1eOUHxiVPWRXmMLpeflYkqPNpwMt 8keWa89iyv1XNkzyRk6/g+PrZRH+q2r0VcOdK/Dv4VomrCM3Rm0LBn1BDFOIkFngMyaJ +jlQ== X-Gm-Message-State: APjAAAURGYD2LdiW3QG0VFTdZ3nLtGz7F2f7ZZ2Om8b67KkZ8F9/6lFs PzZTya5Cz6KAl3QOweFwKqI= X-Google-Smtp-Source: APXvYqzLGHLHGKTKihOAy2200HvS/YysMw35F2hSIcCLGDURi/hEqGs5Ybw0dCcijw+Wy94ZZ5+0qA== X-Received: by 2002:a17:902:ac97:: with SMTP id h23mr1418492plr.237.1582849903095; Thu, 27 Feb 2020 16:31:43 -0800 (PST) Received: from seurat29.1015granger.net (ip-184-250-247-225.sanjca.spcsdns.net. [184.250.247.225]) by smtp.gmail.com with ESMTPSA id 199sm8522333pfu.71.2020.02.27.16.31.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 Feb 2020 16:31:42 -0800 (PST) Subject: [PATCH v1 11/16] svcrdma: Update synopsis of svc_rdma_send_reply_msg() From: Chuck Lever To: bfields@fieldses.org Cc: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Thu, 27 Feb 2020 19:31:41 -0500 Message-ID: <158284990128.38468.15361957366190048084.stgit@seurat29.1015granger.net> In-Reply-To: <158284930886.38468.17045380766660946827.stgit@seurat29.1015granger.net> References: <158284930886.38468.17045380766660946827.stgit@seurat29.1015granger.net> User-Agent: StGit/unknown-version MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Preparing for subsequent patches, no behavior change expected. Pass the RPC Call's svc_rdma_recv_ctxt deeper into the sendto() path. This enables passing more information about Requester- provided Write and Reply chunks into those lower-level functions. Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/svc_rdma_sendto.c | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c index bc8863342878..1035df3a3b5c 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c +++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c @@ -729,13 +729,12 @@ static void svc_rdma_save_io_pages(struct svc_rqst *rqstp, */ static int svc_rdma_send_reply_msg(struct svcxprt_rdma *rdma, struct svc_rdma_send_ctxt *sctxt, - struct svc_rdma_recv_ctxt *rctxt, - struct svc_rqst *rqstp, - __be32 *wr_lst, __be32 *rp_ch) + const struct svc_rdma_recv_ctxt *rctxt, + struct svc_rqst *rqstp) { int ret; - if (!rp_ch) { + if (!rctxt->rc_reply_chunk) { ret = svc_rdma_map_reply_msg(rdma, sctxt, rctxt, &rqstp->rq_res); if (ret < 0) @@ -856,8 +855,7 @@ int svc_rdma_sendto(struct svc_rqst *rqstp) } svc_rdma_sync_reply_hdr(rdma, sctxt, svc_rdma_reply_hdr_len(rdma_resp)); - ret = svc_rdma_send_reply_msg(rdma, sctxt, rctxt, rqstp, - wr_lst, rp_ch); + ret = svc_rdma_send_reply_msg(rdma, sctxt, rctxt, rqstp); if (ret < 0) goto err1; ret = 0; From patchwork Fri Feb 28 00:31:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 11411281 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4E81C1805 for ; Fri, 28 Feb 2020 00:31:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 2E200246A1 for ; Fri, 28 Feb 2020 00:31:52 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="e+tkr6Ae" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730247AbgB1Abv (ORCPT ); Thu, 27 Feb 2020 19:31:51 -0500 Received: from mail-pj1-f66.google.com ([209.85.216.66]:37177 "EHLO mail-pj1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730117AbgB1Abv (ORCPT ); Thu, 27 Feb 2020 19:31:51 -0500 Received: by mail-pj1-f66.google.com with SMTP id m13so514081pjb.2; Thu, 27 Feb 2020 16:31:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=N/lInLYnOgUImPT8Z29KJToYDgSjQqFLZT3k4njOq6s=; b=e+tkr6AeyNSchz2ivIcKwIOiwlh0FO7WNBIjrowGuGBn36zX/Ui6jCz9eCOczN3cFT MiifoKujLhxv4KWrAE3CwsS2lgoktkzkgFiV8fb+WeyP4Dl2XIQxmE7k5lqxraSMaydi 9KWBd7cTfVQLwQ5+IwGrIRZiPXPgVWxaZXmBo68UdxOCMUF8rCm74BT5p696/TfB/dVd L0uBRQFjxReRAsLOzJ073S7Kdf84qQOyROFkK4dh5VJ8GLZV9ufXOqMFWgJNBMVBpj7B 643thaGEfOCEiJiiL+OUkFs5qvTua3IEjJ/AjpQNCsSe5ChhAatf+sJTt/wystyZoU1Z +Kqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=N/lInLYnOgUImPT8Z29KJToYDgSjQqFLZT3k4njOq6s=; b=aJyky/aVG5JGUPq3c3oRu2MYEJCNSSlbULNWcyBmz3GoIYY627BoudZ+jY3KyT9w9R CyjjBXos+L2n1BC81bvXlV3SkL9AazrEzudBrMW285ft6dg9kg9D24T0rR6akoZuvaND 1DpXvVx7cMp7iSbS+VUU2ZahBwfkB6s1cXyTBhfOawq3mQK/dXxJpYztg5Aoxrpkw62n Bz7u9Zk0MwsWP+K9vKuNMJ4cdPZf+YL+ydYJ8GanfW6qN9V7p4TW26u33EdVgY5msGk0 8sV7cfHACfpbmbX9zvODTQt/ceKqQolZBv1vwQG06XgbqbQu1e7EzH6Tds+JcI43y3C5 CWhQ== X-Gm-Message-State: APjAAAVDcD4Kd+QTiwk7au0Os0x7WRwDrQqWSxxCnJZ6BBA9mWG57FMg MVg93sQrKPu0+EIguijtZUnq97blo4A= X-Google-Smtp-Source: APXvYqwaRaZJeJDo2Bq5MUDzCqf1ituslF8EFxhf54VhxDPPGcdPSc4BGl96Ln5VQI89grfkUSqCcA== X-Received: by 2002:a17:902:d705:: with SMTP id w5mr1456088ply.68.1582849910016; Thu, 27 Feb 2020 16:31:50 -0800 (PST) Received: from seurat29.1015granger.net (ip-184-250-247-225.sanjca.spcsdns.net. [184.250.247.225]) by smtp.gmail.com with ESMTPSA id 64sm8412413pfd.48.2020.02.27.16.31.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 Feb 2020 16:31:49 -0800 (PST) Subject: [PATCH v1 12/16] svcrdma: Rename svcrdma_encode trace points in send routines From: Chuck Lever To: bfields@fieldses.org Cc: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Thu, 27 Feb 2020 19:31:48 -0500 Message-ID: <158284990838.38468.938564136257260647.stgit@seurat29.1015granger.net> In-Reply-To: <158284930886.38468.17045380766660946827.stgit@seurat29.1015granger.net> References: <158284930886.38468.17045380766660946827.stgit@seurat29.1015granger.net> User-Agent: StGit/unknown-version MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org These trace points are misnamed: trace_svcrdma_encode_wseg trace_svcrdma_encode_write trace_svcrdma_encode_reply trace_svcrdma_encode_rseg trace_svcrdma_encode_read trace_svcrdma_encode_pzr Because they actually trace posting on the Send Queue. Let's rename them so that I can add trace points in the chunk list encoders that actually do trace chunk list encoding events. Signed-off-by: Chuck Lever --- include/trace/events/rpcrdma.h | 14 +++++++++----- net/sunrpc/xprtrdma/svc_rdma_rw.c | 13 +++++++------ 2 files changed, 16 insertions(+), 11 deletions(-) diff --git a/include/trace/events/rpcrdma.h b/include/trace/events/rpcrdma.h index 814b73bd2cc7..74b68547eefb 100644 --- a/include/trace/events/rpcrdma.h +++ b/include/trace/events/rpcrdma.h @@ -1479,7 +1479,9 @@ DECLARE_EVENT_CLASS(svcrdma_segment_event, DEFINE_SEGMENT_EVENT(decode_wseg); DEFINE_SEGMENT_EVENT(encode_rseg); +DEFINE_SEGMENT_EVENT(send_rseg); DEFINE_SEGMENT_EVENT(encode_wseg); +DEFINE_SEGMENT_EVENT(send_wseg); DECLARE_EVENT_CLASS(svcrdma_chunk_event, TP_PROTO( @@ -1502,17 +1504,19 @@ DECLARE_EVENT_CLASS(svcrdma_chunk_event, ); #define DEFINE_CHUNK_EVENT(name) \ - DEFINE_EVENT(svcrdma_chunk_event, svcrdma_encode_##name,\ + DEFINE_EVENT(svcrdma_chunk_event, svcrdma_##name, \ TP_PROTO( \ u32 length \ ), \ TP_ARGS(length)) -DEFINE_CHUNK_EVENT(pzr); -DEFINE_CHUNK_EVENT(write); -DEFINE_CHUNK_EVENT(reply); +DEFINE_CHUNK_EVENT(send_pzr); +DEFINE_CHUNK_EVENT(encode_write_chunk); +DEFINE_CHUNK_EVENT(send_write_chunk); +DEFINE_CHUNK_EVENT(encode_read_chunk); +DEFINE_CHUNK_EVENT(send_reply_chunk); -TRACE_EVENT(svcrdma_encode_read, +TRACE_EVENT(svcrdma_send_read_chunk, TP_PROTO( u32 length, u32 position diff --git a/net/sunrpc/xprtrdma/svc_rdma_rw.c b/net/sunrpc/xprtrdma/svc_rdma_rw.c index 9d854755d78d..a0e83a94beeb 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_rw.c +++ b/net/sunrpc/xprtrdma/svc_rdma_rw.c @@ -439,7 +439,8 @@ svc_rdma_build_writes(struct svc_rdma_write_info *info, if (ret < 0) goto out_initerr; - trace_svcrdma_encode_wseg(seg_handle, write_len, seg_offset); + trace_svcrdma_send_wseg(seg_handle, write_len, seg_offset); + list_add(&ctxt->rw_list, &cc->cc_rwctxts); cc->cc_sqecount += ret; if (write_len == seg_length - info->wi_seg_off) { @@ -534,7 +535,7 @@ int svc_rdma_send_write_chunk(struct svcxprt_rdma *rdma, __be32 *wr_ch, if (ret < 0) goto out_err; - trace_svcrdma_encode_write(xdr->page_len); + trace_svcrdma_send_write_chunk(xdr->page_len); return length; out_err: @@ -594,7 +595,7 @@ int svc_rdma_send_reply_chunk(struct svcxprt_rdma *rdma, if (ret < 0) goto out_err; - trace_svcrdma_encode_reply(consumed); + trace_svcrdma_send_reply_chunk(consumed); return consumed; out_err: @@ -697,7 +698,7 @@ static int svc_rdma_build_read_chunk(struct svc_rqst *rqstp, if (ret < 0) break; - trace_svcrdma_encode_rseg(rs_handle, rs_length, rs_offset); + trace_svcrdma_send_rseg(rs_handle, rs_length, rs_offset); info->ri_chunklen += rs_length; } @@ -728,7 +729,7 @@ static int svc_rdma_build_normal_read_chunk(struct svc_rqst *rqstp, if (ret < 0) goto out; - trace_svcrdma_encode_read(info->ri_chunklen, info->ri_position); + trace_svcrdma_send_read_chunk(info->ri_chunklen, info->ri_position); head->rc_hdr_count = 0; @@ -784,7 +785,7 @@ static int svc_rdma_build_pz_read_chunk(struct svc_rqst *rqstp, if (ret < 0) goto out; - trace_svcrdma_encode_pzr(info->ri_chunklen); + trace_svcrdma_send_pzr(info->ri_chunklen); head->rc_arg.len += info->ri_chunklen; head->rc_arg.buflen += info->ri_chunklen; From patchwork Fri Feb 28 00:31:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 11411285 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9879A1805 for ; Fri, 28 Feb 2020 00:31:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 786CD2469D for ; Fri, 28 Feb 2020 00:31:59 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="RXiENqS8" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730356AbgB1Ab6 (ORCPT ); Thu, 27 Feb 2020 19:31:58 -0500 Received: from mail-pf1-f195.google.com ([209.85.210.195]:39484 "EHLO mail-pf1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730117AbgB1Ab6 (ORCPT ); Thu, 27 Feb 2020 19:31:58 -0500 Received: by mail-pf1-f195.google.com with SMTP id l7so738179pff.6; Thu, 27 Feb 2020 16:31:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=SNn9dgUZJq1qDJ3pMC89zXA5DX6KAQg/f61MzZyr0zo=; b=RXiENqS8FNFInUsHXe4bjX4hGQ9oDgmdIHanQjF0oMwtWbiyt8riBQWLFlB8XD1Xjg b5aCBOiCjt8WfoYHMbWhDwQnr1oHvXttGoF4CSmW/1YYnZ8IZXsFiIvLdke0wV3MPD5c C61MueJDOGpn2AneF3htSttNYeif4MHwBEloCoIngkdVY2fzb0dDrLCwlKKl5XuFbGj5 MQRx1mnvM3/E9I/qb3qINvbXSR3yRDAxl0N+8k6RCwF1NvO8Re4mbPwXqjwVIJBZKmzO Ty4EsEbdQniRuhmqtFlAKjwv8RbzvyJUejWyW9W1742kSxXI7kJLahIbPfaOVUGf5Vas mGTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=SNn9dgUZJq1qDJ3pMC89zXA5DX6KAQg/f61MzZyr0zo=; b=f+nKGBUljMP8tWYOQQj6yhQ8bVZ0zsT6AOJYZjqCCpxlfx5Xci7WbUrAS67m9PpS5l sJsQB1ip/93qBqjeWpiLzNVUS6zEw/hqXASkPEdOVs2ih2lawlDQORo8zYJMIcALfIrW ZEwBQbtIaHVaEpHf4GiMLCu9rmYB2a1p71LT5MvB+aodaMmtUHgDC8vo0xmRgUEhiRQ0 Tn0fOrcr1O83hhxz9W+0zERfUecNrujZfqx8PbC7jsWtwc3LIlFm98EAH02MDFuzQNWL 1C1rWNursn2uU4br7fBk4l4Wj62E70Yj2FKzp3Yr8WUgNix/vWwHIW+5vNdGvZWKOnQU 0C1g== X-Gm-Message-State: APjAAAV6f76asplz3cJaSrp1a9yQUQx+n4DylTpCNrwOK63p9IcdYcLz sbRIXIHgyPNUR+DHwt7+Wtb0+bjfKRg= X-Google-Smtp-Source: APXvYqz/NKi0xqTJHP53z8MSNzfIZAnG6VoH0BDMNJrdzvTjM8MaJ8Q88ExKrOXNCoMLTPffGQHl/w== X-Received: by 2002:aa7:957c:: with SMTP id x28mr1675922pfq.157.1582849917021; Thu, 27 Feb 2020 16:31:57 -0800 (PST) Received: from seurat29.1015granger.net (ip-184-250-247-225.sanjca.spcsdns.net. [184.250.247.225]) by smtp.gmail.com with ESMTPSA id k1sm6342250pgt.70.2020.02.27.16.31.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 Feb 2020 16:31:56 -0800 (PST) Subject: [PATCH v1 13/16] SUNRPC: Add encoders for list item discriminators From: Chuck Lever To: bfields@fieldses.org Cc: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Thu, 27 Feb 2020 19:31:55 -0500 Message-ID: <158284991530.38468.7171819678223366860.stgit@seurat29.1015granger.net> In-Reply-To: <158284930886.38468.17045380766660946827.stgit@seurat29.1015granger.net> References: <158284930886.38468.17045380766660946827.stgit@seurat29.1015granger.net> User-Agent: StGit/unknown-version MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Clean up. These are taken from the client-side RPC/RDMA transport to a more global header file so they can be used elsewhere. Signed-off-by: Chuck Lever --- include/linux/sunrpc/xdr.h | 38 ++++++++++++++++++++++++++++++++++++++ net/sunrpc/xprtrdma/rpc_rdma.c | 36 +++++------------------------------- 2 files changed, 43 insertions(+), 31 deletions(-) diff --git a/include/linux/sunrpc/xdr.h b/include/linux/sunrpc/xdr.h index 80972512edfd..cddfab4a5440 100644 --- a/include/linux/sunrpc/xdr.h +++ b/include/linux/sunrpc/xdr.h @@ -316,6 +316,44 @@ xdr_pad_size(size_t n) return xdr_align_size(n) - n; } +/** + * xdr_stream_encode_item_present - Encode a "present" list item + * @xdr: pointer to xdr_stream + * + * Return values: + * On success, returns length in bytes of XDR buffer consumed + * %-EMSGSIZE on XDR buffer overflow + */ +static inline ssize_t xdr_stream_encode_item_present(struct xdr_stream *xdr) +{ + const size_t len = sizeof(__be32); + __be32 *p = xdr_reserve_space(xdr, len); + + if (unlikely(!p)) + return -EMSGSIZE; + *p = xdr_one; + return len; +} + +/** + * xdr_stream_encode_item_absent - Encode a "not present" list item + * @xdr: pointer to xdr_stream + * + * Return values: + * On success, returns length in bytes of XDR buffer consumed + * %-EMSGSIZE on XDR buffer overflow + */ +static inline int xdr_stream_encode_item_absent(struct xdr_stream *xdr) +{ + const size_t len = sizeof(__be32); + __be32 *p = xdr_reserve_space(xdr, len); + + if (unlikely(!p)) + return -EMSGSIZE; + *p = xdr_zero; + return len; +} + /** * xdr_stream_encode_u32 - Encode a 32-bit integer * @xdr: pointer to xdr_stream diff --git a/net/sunrpc/xprtrdma/rpc_rdma.c b/net/sunrpc/xprtrdma/rpc_rdma.c index b51e92d43dfd..737a54c3ad53 100644 --- a/net/sunrpc/xprtrdma/rpc_rdma.c +++ b/net/sunrpc/xprtrdma/rpc_rdma.c @@ -275,32 +275,6 @@ rpcrdma_convert_iovs(struct rpcrdma_xprt *r_xprt, struct xdr_buf *xdrbuf, return n; } -static inline int -encode_item_present(struct xdr_stream *xdr) -{ - __be32 *p; - - p = xdr_reserve_space(xdr, sizeof(*p)); - if (unlikely(!p)) - return -EMSGSIZE; - - *p = xdr_one; - return 0; -} - -static inline int -encode_item_not_present(struct xdr_stream *xdr) -{ - __be32 *p; - - p = xdr_reserve_space(xdr, sizeof(*p)); - if (unlikely(!p)) - return -EMSGSIZE; - - *p = xdr_zero; - return 0; -} - static void xdr_encode_rdma_segment(__be32 *iptr, struct rpcrdma_mr *mr) { @@ -414,7 +388,7 @@ static int rpcrdma_encode_read_list(struct rpcrdma_xprt *r_xprt, } while (nsegs); done: - return encode_item_not_present(xdr); + return xdr_stream_encode_item_absent(xdr); } /* Register and XDR encode the Write list. Supports encoding a list @@ -453,7 +427,7 @@ static int rpcrdma_encode_write_list(struct rpcrdma_xprt *r_xprt, if (nsegs < 0) return nsegs; - if (encode_item_present(xdr) < 0) + if (xdr_stream_encode_item_present(xdr) < 0) return -EMSGSIZE; segcount = xdr_reserve_space(xdr, sizeof(*segcount)); if (unlikely(!segcount)) @@ -480,7 +454,7 @@ static int rpcrdma_encode_write_list(struct rpcrdma_xprt *r_xprt, *segcount = cpu_to_be32(nchunks); done: - return encode_item_not_present(xdr); + return xdr_stream_encode_item_absent(xdr); } /* Register and XDR encode the Reply chunk. Supports encoding an array @@ -507,14 +481,14 @@ static int rpcrdma_encode_reply_chunk(struct rpcrdma_xprt *r_xprt, __be32 *segcount; if (wtype != rpcrdma_replych) - return encode_item_not_present(xdr); + return xdr_stream_encode_item_absent(xdr); seg = req->rl_segments; nsegs = rpcrdma_convert_iovs(r_xprt, &rqst->rq_rcv_buf, 0, wtype, seg); if (nsegs < 0) return nsegs; - if (encode_item_present(xdr) < 0) + if (xdr_stream_encode_item_present(xdr) < 0) return -EMSGSIZE; segcount = xdr_reserve_space(xdr, sizeof(*segcount)); if (unlikely(!segcount)) From patchwork Fri Feb 28 00:32:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 11411289 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F1ECD1805 for ; Fri, 28 Feb 2020 00:32:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id CDA17246A1 for ; Fri, 28 Feb 2020 00:32:06 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Amu9h+PR" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730374AbgB1AcG (ORCPT ); Thu, 27 Feb 2020 19:32:06 -0500 Received: from mail-pf1-f195.google.com ([209.85.210.195]:40056 "EHLO mail-pf1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730117AbgB1AcF (ORCPT ); Thu, 27 Feb 2020 19:32:05 -0500 Received: by mail-pf1-f195.google.com with SMTP id b185so737625pfb.7; Thu, 27 Feb 2020 16:32:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=7AG58vC9RtqnN57bg63Far7VqfXiBw6/6793+Cm3LCY=; b=Amu9h+PROlVuSEmyvQ3TZ6b74kc9KVxDVSO9GIuqb5QcW4hWUEHDmdLM2ayYUvq6wo xgn/CfnC4m3K2zdqq5uMn/WslISvogU8DGDLkdZGgX/7ZReDKXkgtH/Fy+viHrj4S2j8 q9kO9Y90iIEMP4Bb0DPLOY0b0cQsiUCL/OLNL/fHoXGgYHNLn81cbOzUbzzWUJwATI5Z hy1msjsFQC+NzQfqNdQpekgaR5cceZSZ7cADCLkM20Swh29E5Nlj1eSJq5ap/pK2O25X vEGtJjNUezhqWIS8yAImFbjb03RJ6oOZIATJbyEviazv8g5+wcXRBOJhfenFt7pV1wMP o8fQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=7AG58vC9RtqnN57bg63Far7VqfXiBw6/6793+Cm3LCY=; b=OHa+xF2oQNhVaKAlM4lujGyzF/O8XSs0Nc37n9APMz3QCZq88RjN236WayiTRp6Pyo I6/wqhY+G+v+8yKbLvWcvhcTu8zivzz32BbQAhsYcLTJX+M4qDuxIN5lEqcD/X8oWPSC D79nWAiSZBgAvbPmkfrhzXT5kUYO6wLaIZwRTKz0j90d4ggZYnDFLHapOpi2rQjDNxFH wJdWEeGWTZ0q0Y91uyLNf00zp/Krw/jT5pMjk8ne50Qn5kou9yfDDESjShWljXR5tWK9 CyeWB1zItXUwDVk+RvH5YHwgL3AT9nmS3TsYvesBz6ywVCMqNlBdP8CPtCtcR7hXRbcd f2xQ== X-Gm-Message-State: APjAAAVcgdVyG6Pjdevj2JVcIVAsu2Kr4X8Jg7dY2FQXjQUIsctC+i1r OPwQh85Zsb0SMk0NX+63dBM= X-Google-Smtp-Source: APXvYqwTLuz5hZhRjzfHbZUMlxZoYQ7Du11FOhcwQto4EchsJrZ016LjgO1QcTxcE6SniDHO787aQg== X-Received: by 2002:a63:731c:: with SMTP id o28mr1851311pgc.139.1582849924135; Thu, 27 Feb 2020 16:32:04 -0800 (PST) Received: from seurat29.1015granger.net (ip-184-250-247-225.sanjca.spcsdns.net. [184.250.247.225]) by smtp.gmail.com with ESMTPSA id r3sm252684pfq.126.2020.02.27.16.32.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 Feb 2020 16:32:03 -0800 (PST) Subject: [PATCH v1 14/16] svcrdma: Refactor chunk list encoders From: Chuck Lever To: bfields@fieldses.org Cc: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Thu, 27 Feb 2020 19:32:02 -0500 Message-ID: <158284992227.38468.5375514646117410024.stgit@seurat29.1015granger.net> In-Reply-To: <158284930886.38468.17045380766660946827.stgit@seurat29.1015granger.net> References: <158284930886.38468.17045380766660946827.stgit@seurat29.1015granger.net> User-Agent: StGit/unknown-version MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Same idea as the receive-side changes I did a while back: use xdr_stream helpers rather than open-coding the XDR chunk list encoders. This builds the Reply transport header from beginning to end without backtracking. As additional clean-ups, fill in documenting comments for the XDR encoders and sprinkle some trace points in the new encoding functions. Signed-off-by: Chuck Lever --- include/linux/sunrpc/svc_rdma.h | 2 net/sunrpc/xprtrdma/svc_rdma_backchannel.c | 15 +- net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 34 +++ net/sunrpc/xprtrdma/svc_rdma_sendto.c | 279 +++++++++++++++++----------- 4 files changed, 209 insertions(+), 121 deletions(-) diff --git a/include/linux/sunrpc/svc_rdma.h b/include/linux/sunrpc/svc_rdma.h index c506732886b3..d001aac13c2f 100644 --- a/include/linux/sunrpc/svc_rdma.h +++ b/include/linux/sunrpc/svc_rdma.h @@ -149,6 +149,8 @@ struct svc_rdma_send_ctxt { struct list_head sc_list; struct ib_send_wr sc_send_wr; struct ib_cqe sc_cqe; + struct xdr_buf sc_hdrbuf; + struct xdr_stream sc_stream; void *sc_xprt_buf; int sc_page_count; int sc_cur_sge_no; diff --git a/net/sunrpc/xprtrdma/svc_rdma_backchannel.c b/net/sunrpc/xprtrdma/svc_rdma_backchannel.c index ce1a7a706f36..9830748c58d2 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_backchannel.c +++ b/net/sunrpc/xprtrdma/svc_rdma_backchannel.c @@ -181,7 +181,9 @@ rpcrdma_bc_send_request(struct svcxprt_rdma *rdma, struct rpc_rqst *rqst) if (!ctxt) goto drop_connection; - p = ctxt->sc_xprt_buf; + p = xdr_reserve_space(&ctxt->sc_stream, RPCRDMA_HDRLEN_MIN); + if (!p) + goto put_ctxt; *p++ = rqst->rq_xid; *p++ = rpcrdma_version; *p++ = cpu_to_be32(r_xprt->rx_buf.rb_bc_max_requests); @@ -189,7 +191,7 @@ rpcrdma_bc_send_request(struct svcxprt_rdma *rdma, struct rpc_rqst *rqst) *p++ = xdr_zero; *p++ = xdr_zero; *p = xdr_zero; - svc_rdma_sync_reply_hdr(rdma, ctxt, RPCRDMA_HDRLEN_MIN); + svc_rdma_sync_reply_hdr(rdma, ctxt, ctxt->sc_hdrbuf.len); #ifdef SVCRDMA_BACKCHANNEL_DEBUG pr_info("%s: %*ph\n", __func__, 64, rqst->rq_buffer); @@ -197,12 +199,13 @@ rpcrdma_bc_send_request(struct svcxprt_rdma *rdma, struct rpc_rqst *rqst) rqst->rq_xtime = ktime_get(); rc = svc_rdma_bc_sendto(rdma, rqst, ctxt); - if (rc) { - svc_rdma_send_ctxt_put(rdma, ctxt); - goto drop_connection; - } + if (rc) + goto put_ctxt; return 0; +put_ctxt: + svc_rdma_send_ctxt_put(rdma, ctxt); + drop_connection: dprintk("svcrdma: failed to send bc call\n"); return -ENOTCONN; diff --git a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c index f9a3c413491c..447060dbd6fd 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c +++ b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c @@ -698,7 +698,6 @@ static void svc_rdma_send_error(struct svcxprt_rdma *xprt, __be32 *rdma_argp, int status) { struct svc_rdma_send_ctxt *ctxt; - unsigned int length; __be32 *p; int ret; @@ -706,29 +705,48 @@ static void svc_rdma_send_error(struct svcxprt_rdma *xprt, if (!ctxt) return; - p = ctxt->sc_xprt_buf; + p = xdr_reserve_space(&ctxt->sc_stream, + rpcrdma_fixed_maxsz * sizeof(*p)); + if (!p) + goto put_ctxt; + *p++ = *rdma_argp; *p++ = *(rdma_argp + 1); *p++ = xprt->sc_fc_credits; - *p++ = rdma_error; + *p = rdma_error; + switch (status) { case -EPROTONOSUPPORT: + p = xdr_reserve_space(&ctxt->sc_stream, + 3 * sizeof(*p)); + if (!p) + goto put_ctxt; + *p++ = err_vers; *p++ = rpcrdma_version; - *p++ = rpcrdma_version; + *p = rpcrdma_version; trace_svcrdma_err_vers(*rdma_argp); break; default: - *p++ = err_chunk; + p = xdr_reserve_space(&ctxt->sc_stream, + sizeof(*p)); + if (!p) + goto put_ctxt; + + *p = err_chunk; trace_svcrdma_err_chunk(*rdma_argp); } - length = (unsigned long)p - (unsigned long)ctxt->sc_xprt_buf; - svc_rdma_sync_reply_hdr(xprt, ctxt, length); + + svc_rdma_sync_reply_hdr(xprt, ctxt, ctxt->sc_hdrbuf.len); ctxt->sc_send_wr.opcode = IB_WR_SEND; ret = svc_rdma_send(xprt, &ctxt->sc_send_wr); if (ret) - svc_rdma_send_ctxt_put(xprt, ctxt); + goto put_ctxt; + return; + +put_ctxt: + svc_rdma_send_ctxt_put(xprt, ctxt); } /* By convention, backchannel calls arrive via rdma_msg type diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c index 1035df3a3b5c..f557e1cc5694 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c +++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c @@ -151,6 +151,8 @@ svc_rdma_send_ctxt_alloc(struct svcxprt_rdma *rdma) ctxt->sc_send_wr.send_flags = IB_SEND_SIGNALED; ctxt->sc_cqe.done = svc_rdma_wc_send; ctxt->sc_xprt_buf = buffer; + xdr_buf_init(&ctxt->sc_hdrbuf, ctxt->sc_xprt_buf, + rdma->sc_max_req_size); ctxt->sc_sges[0].addr = addr; for (i = 0; i < rdma->sc_max_send_sges; i++) @@ -204,6 +206,10 @@ struct svc_rdma_send_ctxt *svc_rdma_send_ctxt_get(struct svcxprt_rdma *rdma) spin_unlock(&rdma->sc_send_lock); out: + rpcrdma_set_xdrlen(&ctxt->sc_hdrbuf, 0); + xdr_init_encode(&ctxt->sc_stream, &ctxt->sc_hdrbuf, + ctxt->sc_xprt_buf, NULL); + ctxt->sc_send_wr.num_sge = 0; ctxt->sc_cur_sge_no = 0; ctxt->sc_page_count = 0; @@ -322,131 +328,171 @@ int svc_rdma_send(struct svcxprt_rdma *rdma, struct ib_send_wr *wr) return ret; } -/* Returns length of transport header, in bytes. +/** + * svc_rdma_encode_read_list - Encode RPC Reply's Read chunk list + * @sctxt: Send context for the RPC Reply + * + * Return values: + * On success, returns length in bytes of the Reply XDR buffer + * that was consumed by the Reply Read list + * %-EMSGSIZE on XDR buffer overflow */ -static unsigned int svc_rdma_reply_hdr_len(__be32 *rdma_resp) +static ssize_t svc_rdma_encode_read_list(struct svc_rdma_send_ctxt *sctxt) { - unsigned int nsegs; - __be32 *p; - - p = rdma_resp; - - /* RPC-over-RDMA V1 replies never have a Read list. */ - p += rpcrdma_fixed_maxsz + 1; - - /* Skip Write list. */ - while (*p++ != xdr_zero) { - nsegs = be32_to_cpup(p++); - p += nsegs * rpcrdma_segment_maxsz; - } + /* RPC-over-RDMA version 1 replies never have a Read list. */ + return xdr_stream_encode_item_absent(&sctxt->sc_stream); +} - /* Skip Reply chunk. */ - if (*p++ != xdr_zero) { - nsegs = be32_to_cpup(p++); - p += nsegs * rpcrdma_segment_maxsz; +/** + * svc_rdma_encode_write_segment - Encode one Write segment + * @src: matching Write chunk in the RPC Call header + * @sctxt: Send context for the RPC Reply + * @remaining: remaining bytes of the payload left in the Write chunk + * + * Return values: + * On success, returns length in bytes of the Reply XDR buffer + * that was consumed by the Write segment + * %-EMSGSIZE on XDR buffer overflow + */ +static ssize_t svc_rdma_encode_write_segment(__be32 *src, + struct svc_rdma_send_ctxt *sctxt, + unsigned int *remaining) +{ + __be32 *p; + const size_t len = rpcrdma_segment_maxsz * sizeof(*p); + u32 handle, length; + u64 offset; + + p = xdr_reserve_space(&sctxt->sc_stream, len); + if (!p) + return -EMSGSIZE; + + handle = be32_to_cpup(src++); + length = be32_to_cpup(src++); + xdr_decode_hyper(src, &offset); + + *p++ = cpu_to_be32(handle); + if (*remaining < length) { + /* segment only partly filled */ + length = *remaining; + *remaining = 0; + } else { + /* entire segment was consumed */ + *remaining -= length; } + *p++ = cpu_to_be32(length); + xdr_encode_hyper(p, offset); - return (unsigned long)p - (unsigned long)rdma_resp; + trace_svcrdma_encode_wseg(handle, length, offset); + return len; } -/* One Write chunk is copied from Call transport header to Reply - * transport header. Each segment's length field is updated to - * reflect number of bytes consumed in the segment. - * - * Returns number of segments in this chunk. +/** + * svc_rdma_encode_write_chunk - Encode one Write chunk + * @src: matching Write chunk in the RPC Call header + * @sctxt: Send context for the RPC Reply + * @remaining: size in bytes of the payload in the Write chunk + * + * Copy a Write chunk from the Call transport header to the + * Reply transport header. Update each segment's length field + * to reflect the number of bytes written in that segment. + * + * Return values: + * On success, returns length in bytes of the Reply XDR buffer + * that was consumed by the Write chunk + * %-EMSGSIZE on XDR buffer overflow */ -static unsigned int xdr_encode_write_chunk(__be32 *dst, __be32 *src, +static ssize_t svc_rdma_encode_write_chunk(__be32 *src, + struct svc_rdma_send_ctxt *sctxt, unsigned int remaining) { unsigned int i, nsegs; - u32 seg_len; + ssize_t len, ret; - /* Write list discriminator */ - *dst++ = *src++; + len = 0; + trace_svcrdma_encode_write_chunk(remaining); - /* number of segments in this chunk */ - nsegs = be32_to_cpup(src); - *dst++ = *src++; + src++; + ret = xdr_stream_encode_item_present(&sctxt->sc_stream); + if (ret < 0) + return -EMSGSIZE; + len += ret; - for (i = nsegs; i; i--) { - /* segment's RDMA handle */ - *dst++ = *src++; - - /* bytes returned in this segment */ - seg_len = be32_to_cpu(*src); - if (remaining >= seg_len) { - /* entire segment was consumed */ - *dst = *src; - remaining -= seg_len; - } else { - /* segment only partly filled */ - *dst = cpu_to_be32(remaining); - remaining = 0; - } - dst++; src++; + nsegs = be32_to_cpup(src++); + ret = xdr_stream_encode_u32(&sctxt->sc_stream, nsegs); + if (ret < 0) + return -EMSGSIZE; + len += ret; - /* segment's RDMA offset */ - *dst++ = *src++; - *dst++ = *src++; + for (i = nsegs; i; i--) { + ret = svc_rdma_encode_write_segment(src, sctxt, &remaining); + if (ret < 0) + return -EMSGSIZE; + src += rpcrdma_segment_maxsz; + len += ret; } - return nsegs; + return len; } -/* The client provided a Write list in the Call message. Fill in - * the segments in the first Write chunk in the Reply's transport +/** + * svc_rdma_encode_write_list - Encode RPC Reply's Write chunk list + * @rctxt: Reply context with information about the RPC Call + * @sctxt: Send context for the RPC Reply + * @length: size in bytes of the payload in the first Write chunk + * + * The client provides a Write chunk list in the Call message. Fill + * in the segments in the first Write chunk in the Reply's transport * header with the number of bytes consumed in each segment. * Remaining chunks are returned unused. * * Assumptions: * - Client has provided only one Write chunk + * + * Return values: + * On success, returns length in bytes of the Reply XDR buffer + * that was consumed by the Reply's Write list + * %-EMSGSIZE on XDR buffer overflow */ -static void svc_rdma_xdr_encode_write_list(__be32 *rdma_resp, __be32 *wr_ch, - unsigned int consumed) +static ssize_t svc_rdma_encode_write_list(const struct svc_rdma_recv_ctxt *rctxt, + struct svc_rdma_send_ctxt *sctxt, + unsigned int length) { - unsigned int nsegs; - __be32 *p, *q; - - /* RPC-over-RDMA V1 replies never have a Read list. */ - p = rdma_resp + rpcrdma_fixed_maxsz + 1; - - q = wr_ch; - while (*q != xdr_zero) { - nsegs = xdr_encode_write_chunk(p, q, consumed); - q += 2 + nsegs * rpcrdma_segment_maxsz; - p += 2 + nsegs * rpcrdma_segment_maxsz; - consumed = 0; - } + ssize_t len, ret; - /* Terminate Write list */ - *p++ = xdr_zero; + ret = svc_rdma_encode_write_chunk(rctxt->rc_write_list, sctxt, length); + if (ret < 0) + return ret; + len = ret; - /* Reply chunk discriminator; may be replaced later */ - *p = xdr_zero; + /* Terminate the Write list */ + ret = xdr_stream_encode_item_absent(&sctxt->sc_stream); + if (ret < 0) + return ret; + + return len + ret; } -/* The client provided a Reply chunk in the Call message. Fill in - * the segments in the Reply chunk in the Reply message with the - * number of bytes consumed in each segment. +/** + * svc_rdma_encode_reply_chunk - Encode RPC Reply's Reply chunk + * @rctxt: Reply context with information about the RPC Call + * @sctxt: Send context for the RPC Reply + * @length: size in bytes of the payload in the Reply chunk * * Assumptions: - * - Reply can always fit in the provided Reply chunk + * - Reply can always fit in the client-provided Reply chunk + * + * Return values: + * On success, returns length in bytes of the Reply XDR buffer + * that was consumed by the Reply's Reply chunk + * %-EMSGSIZE on XDR buffer overflow */ -static void svc_rdma_xdr_encode_reply_chunk(__be32 *rdma_resp, __be32 *rp_ch, - unsigned int consumed) +static ssize_t svc_rdma_encode_reply_chunk(const struct svc_rdma_recv_ctxt *rctxt, + struct svc_rdma_send_ctxt *sctxt, + unsigned int length) { - __be32 *p; - - /* Find the Reply chunk in the Reply's xprt header. - * RPC-over-RDMA V1 replies never have a Read list. - */ - p = rdma_resp + rpcrdma_fixed_maxsz + 1; - - /* Skip past Write list */ - while (*p++ != xdr_zero) - p += 1 + be32_to_cpup(p) * rpcrdma_segment_maxsz; - - xdr_encode_write_chunk(p, rp_ch, consumed); + return svc_rdma_encode_write_chunk(rctxt->rc_reply_chunk, sctxt, + length); } static int svc_rdma_dma_map_page(struct svcxprt_rdma *rdma, @@ -765,14 +811,26 @@ static int svc_rdma_send_error_msg(struct svcxprt_rdma *rdma, struct svc_rdma_send_ctxt *ctxt, struct svc_rqst *rqstp) { + struct svc_rdma_recv_ctxt *rctxt = rqstp->rq_xprt_ctxt; + __be32 *rdma_argp = rctxt->rc_recv_buf; __be32 *p; - p = ctxt->sc_xprt_buf; - trace_svcrdma_err_chunk(*p); - p += 3; + rpcrdma_set_xdrlen(&ctxt->sc_hdrbuf, 0); + xdr_init_encode(&ctxt->sc_stream, &ctxt->sc_hdrbuf, + ctxt->sc_xprt_buf, NULL); + + p = xdr_reserve_space(&ctxt->sc_stream, RPCRDMA_HDRLEN_ERR); + if (!p) + return -ENOMSG; + + *p++ = *rdma_argp; + *p++ = *(rdma_argp + 1); + *p++ = rdma->sc_fc_credits; *p++ = rdma_error; *p = err_chunk; - svc_rdma_sync_reply_hdr(rdma, ctxt, RPCRDMA_HDRLEN_ERR); + trace_svcrdma_err_chunk(*rdma_argp); + + svc_rdma_sync_reply_hdr(rdma, ctxt, ctxt->sc_hdrbuf.len); svc_rdma_save_io_pages(rqstp, ctxt); @@ -803,7 +861,7 @@ int svc_rdma_sendto(struct svc_rqst *rqstp) __be32 *rp_ch = rctxt->rc_reply_chunk; struct xdr_buf *xdr = &rqstp->rq_res; struct svc_rdma_send_ctxt *sctxt; - __be32 *p, *rdma_resp; + __be32 *p; int ret; /* Create the RDMA response header. xprt->xpt_mutex, @@ -816,19 +874,18 @@ int svc_rdma_sendto(struct svc_rqst *rqstp) sctxt = svc_rdma_send_ctxt_get(rdma); if (!sctxt) goto err0; - rdma_resp = sctxt->sc_xprt_buf; - p = rdma_resp; + p = xdr_reserve_space(&sctxt->sc_stream, + rpcrdma_fixed_maxsz * sizeof(*p)); + if (!p) + goto err0; *p++ = *rdma_argp; *p++ = *(rdma_argp + 1); *p++ = rdma->sc_fc_credits; - *p++ = rp_ch ? rdma_nomsg : rdma_msg; - - /* Start with empty chunks */ - *p++ = xdr_zero; - *p++ = xdr_zero; - *p = xdr_zero; + *p = rp_ch ? rdma_nomsg : rdma_msg; + if (svc_rdma_encode_read_list(sctxt) < 0) + goto err0; if (wr_lst) { /* XXX: Presume the client sent only one Write chunk */ unsigned long offset; @@ -845,16 +902,24 @@ int svc_rdma_sendto(struct svc_rqst *rqstp) length); if (ret < 0) goto err2; - svc_rdma_xdr_encode_write_list(rdma_resp, wr_lst, ret); + if (svc_rdma_encode_write_list(rctxt, sctxt, length) < 0) + goto err0; + } else { + if (xdr_stream_encode_item_absent(&sctxt->sc_stream) < 0) + goto err0; } if (rp_ch) { ret = svc_rdma_send_reply_chunk(rdma, rctxt, &rqstp->rq_res); if (ret < 0) goto err2; - svc_rdma_xdr_encode_reply_chunk(rdma_resp, rp_ch, ret); + if (svc_rdma_encode_reply_chunk(rctxt, sctxt, ret) < 0) + goto err0; + } else { + if (xdr_stream_encode_item_absent(&sctxt->sc_stream) < 0) + goto err0; } - svc_rdma_sync_reply_hdr(rdma, sctxt, svc_rdma_reply_hdr_len(rdma_resp)); + svc_rdma_sync_reply_hdr(rdma, sctxt, sctxt->sc_hdrbuf.len); ret = svc_rdma_send_reply_msg(rdma, sctxt, rctxt, rqstp); if (ret < 0) goto err1; From patchwork Fri Feb 28 00:32:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 11411293 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9822514B7 for ; Fri, 28 Feb 2020 00:32:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 777F82469D for ; Fri, 28 Feb 2020 00:32:13 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="e/rew2Iv" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730266AbgB1AcM (ORCPT ); Thu, 27 Feb 2020 19:32:12 -0500 Received: from mail-pg1-f196.google.com ([209.85.215.196]:44801 "EHLO mail-pg1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730117AbgB1AcM (ORCPT ); Thu, 27 Feb 2020 19:32:12 -0500 Received: by mail-pg1-f196.google.com with SMTP id a14so537574pgb.11; Thu, 27 Feb 2020 16:32:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=HsD6XqPvYOfHqWao/oxjfSWr5+od2Da4O6WGbOcOZZE=; b=e/rew2Iv8wt4MR5uKlNxHQ+48Tnwl+boBMWgioditkkNgbTVjkW14Fz1z2Zi5xmijm 6+Ci73ODR+isRVw2cmJisEVRFqk8y66UXKAtWFAfMxMlTC4QAN9eQjh65qQt2viBE8cZ 28eRv/xKs8aVmm5trrxV1FvSB1L7z00AnqTIqAla7VHRaZtGIYpx/RjwflZ0xjYKARZg RQ9d6DgifKmLdyAeLpCrZhIDn5G1iBp7oxmiAttLKogwgljOYEjXd7Kkc5dvz/+ngGL9 OduL09B2aJyRKQDxFk+kqrsxtQGN5N1eN5/reYr7Fg8iyfQC8zeikufdI8gPXYeUbZpc NMtw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=HsD6XqPvYOfHqWao/oxjfSWr5+od2Da4O6WGbOcOZZE=; b=HSwIqLRjfjrQBxkH2ScP9B4kb3z60tWibvyjxo854ZONnwpW/IGk47nSBhTGfJEzMG SuYADE4j0V3WSM8B2wYRLlUPrzpU/wXTIKSK6XTFq9YdZHJYqOl1KIKmgQsVQW2vf7cg jgiaTehPksehV6rvzcb1yilrzymWALYAJfhiR/Q1WSXBJRkswfSmJq+q61x1e/8pjNAC mpAQJ1zcUXcp/tV2tHYF4pYMEbcmeQdofhG9yhbOOVEUiCnDxkSlKHgvujqN7HppwoAP nPdTUc8Yp5zTuzOwg/gtm+yiynoXyXQBzeDQIhnf0rxdqEYyk0HRia7twmJOYH9bapAm HaUA== X-Gm-Message-State: APjAAAV0nJP67BdTqyh8bj8XipLkFQwPUD39jTis+G6OtUY5kFtl2AbA 6CHqWRM+kVOJ6A0RA3zgLW0= X-Google-Smtp-Source: APXvYqz5uy/u2Bf3rX9pPyrNoDW1Sit3ntl26eIKI1oKaxpQDfiU03DBOUeLLwEjlijQwMICS5atwQ== X-Received: by 2002:a63:cc4c:: with SMTP id q12mr1845950pgi.443.1582849930993; Thu, 27 Feb 2020 16:32:10 -0800 (PST) Received: from seurat29.1015granger.net (ip-184-250-247-225.sanjca.spcsdns.net. [184.250.247.225]) by smtp.gmail.com with ESMTPSA id v8sm8278650pfn.172.2020.02.27.16.32.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 Feb 2020 16:32:10 -0800 (PST) Subject: [PATCH v1 15/16] svcrdma: Fix double sync of transport header buffer From: Chuck Lever To: bfields@fieldses.org Cc: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Thu, 27 Feb 2020 19:32:09 -0500 Message-ID: <158284992931.38468.12518545682966287291.stgit@seurat29.1015granger.net> In-Reply-To: <158284930886.38468.17045380766660946827.stgit@seurat29.1015granger.net> References: <158284930886.38468.17045380766660946827.stgit@seurat29.1015granger.net> User-Agent: StGit/unknown-version MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org Performance optimization: Avoid syncing the transport buffer twice when Reply buffer pull-up is necessary. Signed-off-by: Chuck Lever --- include/linux/sunrpc/svc_rdma.h | 3 - net/sunrpc/xprtrdma/svc_rdma_backchannel.c | 1 net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 4 +- net/sunrpc/xprtrdma/svc_rdma_sendto.c | 58 ++++++++++------------------ 4 files changed, 22 insertions(+), 44 deletions(-) diff --git a/include/linux/sunrpc/svc_rdma.h b/include/linux/sunrpc/svc_rdma.h index d001aac13c2f..a3fa5b4fa2e4 100644 --- a/include/linux/sunrpc/svc_rdma.h +++ b/include/linux/sunrpc/svc_rdma.h @@ -191,9 +191,6 @@ extern struct svc_rdma_send_ctxt * extern void svc_rdma_send_ctxt_put(struct svcxprt_rdma *rdma, struct svc_rdma_send_ctxt *ctxt); extern int svc_rdma_send(struct svcxprt_rdma *rdma, struct ib_send_wr *wr); -extern void svc_rdma_sync_reply_hdr(struct svcxprt_rdma *rdma, - struct svc_rdma_send_ctxt *ctxt, - unsigned int len); extern int svc_rdma_map_reply_msg(struct svcxprt_rdma *rdma, struct svc_rdma_send_ctxt *sctxt, const struct svc_rdma_recv_ctxt *rctxt, diff --git a/net/sunrpc/xprtrdma/svc_rdma_backchannel.c b/net/sunrpc/xprtrdma/svc_rdma_backchannel.c index 9830748c58d2..46b59e91d34a 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_backchannel.c +++ b/net/sunrpc/xprtrdma/svc_rdma_backchannel.c @@ -191,7 +191,6 @@ rpcrdma_bc_send_request(struct svcxprt_rdma *rdma, struct rpc_rqst *rqst) *p++ = xdr_zero; *p++ = xdr_zero; *p = xdr_zero; - svc_rdma_sync_reply_hdr(rdma, ctxt, ctxt->sc_hdrbuf.len); #ifdef SVCRDMA_BACKCHANNEL_DEBUG pr_info("%s: %*ph\n", __func__, 64, rqst->rq_buffer); diff --git a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c index 447060dbd6fd..2fb1de40613c 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c +++ b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c @@ -737,9 +737,9 @@ static void svc_rdma_send_error(struct svcxprt_rdma *xprt, trace_svcrdma_err_chunk(*rdma_argp); } - svc_rdma_sync_reply_hdr(xprt, ctxt, ctxt->sc_hdrbuf.len); - + ctxt->sc_send_wr.num_sge = 1; ctxt->sc_send_wr.opcode = IB_WR_SEND; + ctxt->sc_sges[0].length = ctxt->sc_hdrbuf.len; ret = svc_rdma_send(xprt, &ctxt->sc_send_wr); if (ret) goto put_ctxt; diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c index f557e1cc5694..9a7317bc54c9 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c +++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c @@ -301,6 +301,12 @@ int svc_rdma_send(struct svcxprt_rdma *rdma, struct ib_send_wr *wr) might_sleep(); + /* Sync the transport header buffer */ + ib_dma_sync_single_for_device(rdma->sc_pd->device, + wr->sg_list[0].addr, + wr->sg_list[0].length, + DMA_TO_DEVICE); + /* If the SQ is full, wait until an SQ entry is available */ while (1) { if ((atomic_dec_return(&rdma->sc_sq_avail) < 0)) { @@ -530,24 +536,6 @@ static int svc_rdma_dma_map_buf(struct svcxprt_rdma *rdma, offset_in_page(base), len); } -/** - * svc_rdma_sync_reply_hdr - DMA sync the transport header buffer - * @rdma: controlling transport - * @ctxt: send_ctxt for the Send WR - * @len: length of transport header - * - */ -void svc_rdma_sync_reply_hdr(struct svcxprt_rdma *rdma, - struct svc_rdma_send_ctxt *ctxt, - unsigned int len) -{ - ctxt->sc_sges[0].length = len; - ctxt->sc_send_wr.num_sge++; - ib_dma_sync_single_for_device(rdma->sc_pd->device, - ctxt->sc_sges[0].addr, len, - DMA_TO_DEVICE); -} - /** * svc_rdma_pull_up_needed - Determine whether to use pull-up * @rdma: controlling transport @@ -610,9 +598,7 @@ static int svc_rdma_pull_up_reply_msg(struct svcxprt_rdma *rdma, unsigned char *dst, *tailbase; unsigned int taillen; - dst = sctxt->sc_xprt_buf; - dst += sctxt->sc_sges[0].length; - + dst = sctxt->sc_xprt_buf + sctxt->sc_hdrbuf.len; memcpy(dst, xdr->head[0].iov_base, xdr->head[0].iov_len); dst += xdr->head[0].iov_len; @@ -648,11 +634,6 @@ static int svc_rdma_pull_up_reply_msg(struct svcxprt_rdma *rdma, memcpy(dst, tailbase, taillen); sctxt->sc_sges[0].length += xdr->len; - ib_dma_sync_single_for_device(rdma->sc_pd->device, - sctxt->sc_sges[0].addr, - sctxt->sc_sges[0].length, - DMA_TO_DEVICE); - return 0; } @@ -663,7 +644,7 @@ static int svc_rdma_pull_up_reply_msg(struct svcxprt_rdma *rdma, * @xdr: prepared xdr_buf containing RPC message * * Load the xdr_buf into the ctxt's sge array, and DMA map each - * element as it is added. + * element as it is added. The Send WR's num_sge field is set. * * Returns zero on success, or a negative errno on failure. */ @@ -679,6 +660,13 @@ int svc_rdma_map_reply_msg(struct svcxprt_rdma *rdma, u32 xdr_pad; int ret; + /* Set up the (persistently-mapped) transport header SGE */ + sctxt->sc_send_wr.num_sge = 1; + sctxt->sc_sges[0].length = sctxt->sc_hdrbuf.len; + + if (rctxt && rctxt->rc_reply_chunk) + return 0; + if (svc_rdma_pull_up_needed(rdma, rctxt, xdr)) return svc_rdma_pull_up_reply_msg(rdma, sctxt, rctxt, xdr); @@ -780,12 +768,9 @@ static int svc_rdma_send_reply_msg(struct svcxprt_rdma *rdma, { int ret; - if (!rctxt->rc_reply_chunk) { - ret = svc_rdma_map_reply_msg(rdma, sctxt, rctxt, - &rqstp->rq_res); - if (ret < 0) - return ret; - } + ret = svc_rdma_map_reply_msg(rdma, sctxt, rctxt, &rqstp->rq_res); + if (ret < 0) + return ret; svc_rdma_save_io_pages(rqstp, sctxt); @@ -795,8 +780,6 @@ static int svc_rdma_send_reply_msg(struct svcxprt_rdma *rdma, } else { sctxt->sc_send_wr.opcode = IB_WR_SEND; } - dprintk("svcrdma: posting Send WR with %u sge(s)\n", - sctxt->sc_send_wr.num_sge); return svc_rdma_send(rdma, &sctxt->sc_send_wr); } @@ -830,11 +813,11 @@ static int svc_rdma_send_error_msg(struct svcxprt_rdma *rdma, *p = err_chunk; trace_svcrdma_err_chunk(*rdma_argp); - svc_rdma_sync_reply_hdr(rdma, ctxt, ctxt->sc_hdrbuf.len); - svc_rdma_save_io_pages(rqstp, ctxt); + ctxt->sc_send_wr.num_sge = 1; ctxt->sc_send_wr.opcode = IB_WR_SEND; + ctxt->sc_sges[0].length = ctxt->sc_hdrbuf.len; return svc_rdma_send(rdma, &ctxt->sc_send_wr); } @@ -919,7 +902,6 @@ int svc_rdma_sendto(struct svc_rqst *rqstp) goto err0; } - svc_rdma_sync_reply_hdr(rdma, sctxt, sctxt->sc_hdrbuf.len); ret = svc_rdma_send_reply_msg(rdma, sctxt, rctxt, rqstp); if (ret < 0) goto err1; From patchwork Fri Feb 28 00:32:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 11411297 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1EFCE14B7 for ; Fri, 28 Feb 2020 00:32:20 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id F2BA6246A1 for ; Fri, 28 Feb 2020 00:32:19 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="JgGJRtpm" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730378AbgB1AcT (ORCPT ); Thu, 27 Feb 2020 19:32:19 -0500 Received: from mail-pl1-f194.google.com ([209.85.214.194]:43502 "EHLO mail-pl1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730253AbgB1AcT (ORCPT ); Thu, 27 Feb 2020 19:32:19 -0500 Received: by mail-pl1-f194.google.com with SMTP id p11so490220plq.10; Thu, 27 Feb 2020 16:32:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=Rh0J7J+tmI9ibvdcsm+0y9mwMYxxyAhrYZdwv/dpsVs=; b=JgGJRtpmRQYq4WDLgxCITaAj2ShvrCED1EkjwcpTiEr/6r5/0OPWOZ9k9lc31K0sBN d96QEG2Ed9x3bLuz7YP6t0ZYhWj7PPdGOL6ZU/4BWgsxTjYymn5NmXbt5eH600h0xcvo ekN8P90C7/XW//n2WG7eyjacCSCF7OIB/y9/KF4wR29ebax1TIOL7ptTFx4q51Qgk+xp TSsHPFxzBFt6mqq8b0uc11Fpy2wJq46WKBcDw5Nw16PYSlyCty+6s55vjNWDaXMnpMP0 ICCUcbKGsHWgvh1lWFxCXVtDOCR3UsFGG6FELqphsWVZ2eJUpJjFTsDIFCbci+R8wqHq o10A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:cc:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=Rh0J7J+tmI9ibvdcsm+0y9mwMYxxyAhrYZdwv/dpsVs=; b=IxaLRZHifIxwHjgx3PFZxmfLfWy/3JYcXErt1HpIpkCK9Vcsqdevb97yV28yqJsxKc pS6IqnMnF9y1rfKdeJR/dEv2FODCmlANfJzwhL6AlVhgV8xTOXYuux1rq+husbb8mlwB /JqxUYEuV1uVN4OropNVrqykHEAn3op586PlyeIQLSrGyRRmHzQeHKhuTasL3bdjjUaH G/c+kmWqdr+PCADb6YDE85rMuJSyj6H1lfmhoVsUQ+VWGMRUXkzURY2UL3pKrjfWr7Br NLDxx4NCgsFdF+4R+oCcyzrwrd0hVJAdlwulZs4FFmZeSxYzf943AMz/yKQDXO73pxLx niSQ== X-Gm-Message-State: APjAAAVSyVJSCqFDmmqVIF17pMj8yI3cB+2QIhS0jI7w1nJyYQ2q7Cu5 V2cqRd89HCv2tDmBwW/w7gLx2FqW7hA= X-Google-Smtp-Source: APXvYqzEK+YIRCXyESRiVTv9OYqSRB+MClMUSy6tM2tGMvj6K/bEJMqF8h7CEJ/2t785gOnMdIQ42w== X-Received: by 2002:a17:90a:da04:: with SMTP id e4mr1631372pjv.26.1582849937969; Thu, 27 Feb 2020 16:32:17 -0800 (PST) Received: from seurat29.1015granger.net (ip-184-250-247-225.sanjca.spcsdns.net. [184.250.247.225]) by smtp.gmail.com with ESMTPSA id z13sm7995590pge.29.2020.02.27.16.32.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 Feb 2020 16:32:17 -0800 (PST) Subject: [PATCH v1 16/16] svcrdma: Avoid DMA mapping small RPC Replies From: Chuck Lever To: bfields@fieldses.org Cc: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Date: Thu, 27 Feb 2020 19:32:16 -0500 Message-ID: <158284993622.38468.2163113637049774961.stgit@seurat29.1015granger.net> In-Reply-To: <158284930886.38468.17045380766660946827.stgit@seurat29.1015granger.net> References: <158284930886.38468.17045380766660946827.stgit@seurat29.1015granger.net> User-Agent: StGit/unknown-version MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org On some platforms, DMA mapping part of a page is more costly than copying bytes. Indeed, not involving the I/O MMU can help the RPC/RDMA transport scale better for tiny I/Os across more RDMA devices. This is because interaction with the I/O MMU is eliminated for each of these small I/Os. Without the explicit unmapping, the NIC no longer needs to do a costly internal TLB shoot down for buffers that are just a handful of bytes. Since pull-up is now a more a frequent operation, I've introduced a trace point in the pull-up path. It can be used for debugging or user-space tools that count pull-up frequency. Signed-off-by: Chuck Lever --- include/trace/events/rpcrdma.h | 18 ++++++++++++++++++ net/sunrpc/xprtrdma/svc_rdma_sendto.c | 15 ++++++++++++++- 2 files changed, 32 insertions(+), 1 deletion(-) diff --git a/include/trace/events/rpcrdma.h b/include/trace/events/rpcrdma.h index 74b68547eefb..9238d233f8cf 100644 --- a/include/trace/events/rpcrdma.h +++ b/include/trace/events/rpcrdma.h @@ -1639,6 +1639,24 @@ TRACE_EVENT(svcrdma_dma_map_rwctx, ) ); +TRACE_EVENT(svcrdma_send_pullup, + TP_PROTO( + unsigned int len + ), + + TP_ARGS(len), + + TP_STRUCT__entry( + __field(unsigned int, len) + ), + + TP_fast_assign( + __entry->len = len; + ), + + TP_printk("len=%u", __entry->len) +); + TRACE_EVENT(svcrdma_send_failed, TP_PROTO( const struct svc_rqst *rqst, diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c index 9a7317bc54c9..3669a41bf8b6 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c +++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c @@ -539,6 +539,7 @@ static int svc_rdma_dma_map_buf(struct svcxprt_rdma *rdma, /** * svc_rdma_pull_up_needed - Determine whether to use pull-up * @rdma: controlling transport + * @sctxt: send_ctxt for the Send WR * @rctxt: Write and Reply chunks provided by client * @xdr: xdr_buf containing RPC message to transmit * @@ -547,11 +548,22 @@ static int svc_rdma_dma_map_buf(struct svcxprt_rdma *rdma, * %false otherwise */ static bool svc_rdma_pull_up_needed(struct svcxprt_rdma *rdma, + struct svc_rdma_send_ctxt *sctxt, const struct svc_rdma_recv_ctxt *rctxt, struct xdr_buf *xdr) { int elements; + /* For small messages, copying bytes is cheaper than DMA + * mapping. + */ + if (sctxt->sc_hdrbuf.len + xdr->len < + RPCRDMA_V1_DEF_INLINE_SIZE >> 1) + return true; + + /* Check whether the xdr_buf has more elements than can + * fit in a single RDMA Send. + */ /* xdr->head */ elements = 1; @@ -634,6 +646,7 @@ static int svc_rdma_pull_up_reply_msg(struct svcxprt_rdma *rdma, memcpy(dst, tailbase, taillen); sctxt->sc_sges[0].length += xdr->len; + trace_svcrdma_send_pullup(sctxt->sc_sges[0].length); return 0; } @@ -667,7 +680,7 @@ int svc_rdma_map_reply_msg(struct svcxprt_rdma *rdma, if (rctxt && rctxt->rc_reply_chunk) return 0; - if (svc_rdma_pull_up_needed(rdma, rctxt, xdr)) + if (svc_rdma_pull_up_needed(rdma, sctxt, rctxt, xdr)) return svc_rdma_pull_up_reply_msg(rdma, sctxt, rctxt, xdr); ++sctxt->sc_cur_sge_no;