From patchwork Fri Jul 14 18:10:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 13314150 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A80B8EB64DC for ; Fri, 14 Jul 2023 18:11:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236418AbjGNSLk (ORCPT ); Fri, 14 Jul 2023 14:11:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58552 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236356AbjGNSLj (ORCPT ); Fri, 14 Jul 2023 14:11:39 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B9A0E3ABE for ; Fri, 14 Jul 2023 11:11:08 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 7D54961DC5 for ; Fri, 14 Jul 2023 18:10:39 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 981FAC433C8; Fri, 14 Jul 2023 18:10:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1689358238; bh=ZWGvzC9S9tYuwsOB/R/b8dF2O4JGK+mpeOrMCzr1VGE=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=Ls4UwiHVO/9xYfUc7fXrNCic89Z13w5h2SdAquf7NhMxZG9qY82x5cNNXI6FqGxqQ Y7B0gGHXBGYiJgR0ohRCGMjr/MHlNCMIK0B0W4Fl5W05nOQAoKod6T/y2X3RADzoXi QGgin3o5qeZYLsEr3LhpmN2KKlhb5Ry6sEWP2dYF9oEMOzUr/Ft3EDZI0iHwRp/SBQ 7qsUjwAimTelrvFZo7Rzoac4lIMGctY1kslB3yPTn8lzEvZA8wp/OmxX1HmblSbU1F nS7xVBpKy52s6yd1k5twefiYysp0HvVQ2u+1EBH/ctRRpCJIRCdur02XhRGG3UlphX 2wFU7N0l03l3A== Subject: [PATCH v2 1/4] SUNRPC: Convert svc_tcp_sendmsg to use bio_vecs directly From: Chuck Lever To: linux-nfs@vger.kernel.org, netdev@vger.kernel.org Cc: Chuck Lever , dhowells@redhat.com Date: Fri, 14 Jul 2023 14:10:37 -0400 Message-ID: <168935823761.1984.15760913629466718014.stgit@manet.1015granger.net> In-Reply-To: <168935791041.1984.13295336680505732841.stgit@manet.1015granger.net> References: <168935791041.1984.13295336680505732841.stgit@manet.1015granger.net> User-Agent: StGit/1.5 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Chuck Lever Add a helper to convert a whole xdr_buf directly into an array of bio_vecs, then send this array instead of iterating piecemeal over the xdr_buf containing the outbound RPC message. Note that the rules of the RPC protocol mean there can be only one outstanding send at a time on a transport socket. The kernel's SunRPC server enforces this via the transport's xpt_mutex. Thus we can use a per-transport shared array for the xdr_buf conversion rather than allocate one every time or use one that is part of struct svc_rqst. Signed-off-by: Chuck Lever --- include/linux/sunrpc/svcsock.h | 3 ++ include/linux/sunrpc/xdr.h | 2 + net/sunrpc/svcsock.c | 59 ++++++++++++++-------------------------- net/sunrpc/xdr.c | 50 ++++++++++++++++++++++++++++++++++ 4 files changed, 75 insertions(+), 39 deletions(-) diff --git a/include/linux/sunrpc/svcsock.h b/include/linux/sunrpc/svcsock.h index a7116048a4d4..a9bfeadf4cbe 100644 --- a/include/linux/sunrpc/svcsock.h +++ b/include/linux/sunrpc/svcsock.h @@ -40,6 +40,9 @@ struct svc_sock { struct completion sk_handshake_done; + struct bio_vec sk_send_bvec[RPCSVC_MAXPAGES] + ____cacheline_aligned; + struct page * sk_pages[RPCSVC_MAXPAGES]; /* received data */ }; diff --git a/include/linux/sunrpc/xdr.h b/include/linux/sunrpc/xdr.h index f89ec4b5ea16..42f9d7eb9a1a 100644 --- a/include/linux/sunrpc/xdr.h +++ b/include/linux/sunrpc/xdr.h @@ -139,6 +139,8 @@ void xdr_terminate_string(const struct xdr_buf *, const u32); size_t xdr_buf_pagecount(const struct xdr_buf *buf); int xdr_alloc_bvec(struct xdr_buf *buf, gfp_t gfp); void xdr_free_bvec(struct xdr_buf *buf); +unsigned int xdr_buf_to_bvec(struct bio_vec *bvec, unsigned int bvec_size, + const struct xdr_buf *xdr); static inline __be32 *xdr_encode_array(__be32 *p, const void *s, unsigned int len) { diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c index e43f26382411..e35e5afe4b81 100644 --- a/net/sunrpc/svcsock.c +++ b/net/sunrpc/svcsock.c @@ -36,6 +36,8 @@ #include #include #include +#include + #include #include #include @@ -1194,72 +1196,52 @@ static int svc_tcp_recvfrom(struct svc_rqst *rqstp) return 0; /* record not complete */ } -static int svc_tcp_send_kvec(struct socket *sock, const struct kvec *vec, - int flags) -{ - struct msghdr msg = { .msg_flags = MSG_SPLICE_PAGES | flags, }; - - iov_iter_kvec(&msg.msg_iter, ITER_SOURCE, vec, 1, vec->iov_len); - return sock_sendmsg(sock, &msg); -} - /* * MSG_SPLICE_PAGES is used exclusively to reduce the number of * copy operations in this path. Therefore the caller must ensure * that the pages backing @xdr are unchanging. * - * In addition, the logic assumes that * .bv_len is never larger - * than PAGE_SIZE. + * Note that the send is non-blocking. The caller has incremented + * the reference count on each page backing the RPC message, and + * the network layer will "put" these pages when transmission is + * complete. + * + * This is safe for our RPC services because the memory backing + * the head and tail components is never kmalloc'd. These always + * come from pages in the svc_rqst::rq_pages array. */ -static int svc_tcp_sendmsg(struct socket *sock, struct xdr_buf *xdr, +static int svc_tcp_sendmsg(struct svc_sock *svsk, struct xdr_buf *xdr, rpc_fraghdr marker, unsigned int *sentp) { - const struct kvec *head = xdr->head; - const struct kvec *tail = xdr->tail; struct kvec rm = { .iov_base = &marker, .iov_len = sizeof(marker), }; struct msghdr msg = { - .msg_flags = 0, + .msg_flags = MSG_MORE, }; + unsigned int count; int ret; *sentp = 0; - ret = xdr_alloc_bvec(xdr, GFP_KERNEL); - if (ret < 0) - return ret; - ret = kernel_sendmsg(sock, &msg, &rm, 1, rm.iov_len); + ret = kernel_sendmsg(svsk->sk_sock, &msg, &rm, 1, rm.iov_len); if (ret < 0) return ret; *sentp += ret; if (ret != rm.iov_len) return -EAGAIN; - ret = svc_tcp_send_kvec(sock, head, 0); - if (ret < 0) - return ret; - *sentp += ret; - if (ret != head->iov_len) - goto out; + count = xdr_buf_to_bvec(svsk->sk_send_bvec, + ARRAY_SIZE(svsk->sk_send_bvec), xdr); msg.msg_flags = MSG_SPLICE_PAGES; - iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, xdr->bvec, - xdr_buf_pagecount(xdr), xdr->page_len); - ret = sock_sendmsg(sock, &msg); + iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, svsk->sk_send_bvec, + count, xdr->len); + ret = sock_sendmsg(svsk->sk_sock, &msg); if (ret < 0) return ret; *sentp += ret; - - if (tail->iov_len) { - ret = svc_tcp_send_kvec(sock, tail, 0); - if (ret < 0) - return ret; - *sentp += ret; - } - -out: return 0; } @@ -1290,8 +1272,7 @@ static int svc_tcp_sendto(struct svc_rqst *rqstp) if (svc_xprt_is_dead(xprt)) goto out_notconn; tcp_sock_set_cork(svsk->sk_sk, true); - err = svc_tcp_sendmsg(svsk->sk_sock, xdr, marker, &sent); - xdr_free_bvec(xdr); + err = svc_tcp_sendmsg(svsk, xdr, marker, &sent); trace_svcsock_tcp_send(xprt, err < 0 ? (long)err : sent); if (err < 0 || sent != (xdr->len + sizeof(marker))) goto out_close; diff --git a/net/sunrpc/xdr.c b/net/sunrpc/xdr.c index 2a22e78af116..358e6de91775 100644 --- a/net/sunrpc/xdr.c +++ b/net/sunrpc/xdr.c @@ -164,6 +164,56 @@ xdr_free_bvec(struct xdr_buf *buf) buf->bvec = NULL; } +/** + * xdr_buf_to_bvec - Copy components of an xdr_buf into a bio_vec array + * @bvec: bio_vec array to populate + * @bvec_size: element count of @bio_vec + * @xdr: xdr_buf to be copied + * + * Returns the number of entries consumed in @bvec. + */ +unsigned int xdr_buf_to_bvec(struct bio_vec *bvec, unsigned int bvec_size, + const struct xdr_buf *xdr) +{ + const struct kvec *head = xdr->head; + const struct kvec *tail = xdr->tail; + unsigned int count = 0; + + if (head->iov_len) { + bvec_set_virt(bvec++, head->iov_base, head->iov_len); + ++count; + } + + if (xdr->page_len) { + unsigned int offset, len, remaining; + struct page **pages = xdr->pages; + + offset = offset_in_page(xdr->page_base); + remaining = xdr->page_len; + while (remaining > 0) { + len = min_t(unsigned int, remaining, + PAGE_SIZE - offset); + bvec_set_page(bvec++, *pages++, len, offset); + remaining -= len; + offset = 0; + if (unlikely(++count > bvec_size)) + goto bvec_overflow; + } + } + + if (tail->iov_len) { + bvec_set_virt(bvec, tail->iov_base, tail->iov_len); + if (unlikely(++count > bvec_size)) + goto bvec_overflow; + } + + return count; + +bvec_overflow: + pr_warn_once("%s: bio_vec array overflow\n", __func__); + return count - 1; +} + /** * xdr_inline_pages - Prepare receive buffer for a large reply * @xdr: xdr_buf into which reply will be placed From patchwork Fri Jul 14 18:10:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 13314151 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 062BCC001DC for ; Fri, 14 Jul 2023 18:11:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236202AbjGNSLm (ORCPT ); Fri, 14 Jul 2023 14:11:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58698 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236429AbjGNSLk (ORCPT ); Fri, 14 Jul 2023 14:11:40 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BB4903C02 for ; Fri, 14 Jul 2023 11:11:10 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 052AC61DB5 for ; Fri, 14 Jul 2023 18:10:46 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1FDE4C433C7; Fri, 14 Jul 2023 18:10:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1689358245; bh=byFEDI4Ms42xCBRWUSKClR4VOvCmozGhJdwzzwpFm1o=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=PlW4gMRttlkkDmvM4Ng8lkXg1as0sPRNt5FaZJaxZOBGfRVYiVEVpVvAg3AwEFDKi YaOtNYkOxfGkorHI1ahjPBzci0FoT8wAze/1BuDogpcInY1GJIIpThLE3eYZ8c/aGM zgrqF847ZfjLfZGfgqX933IYl29pYajD8U25Jn49Azjb84GCQTaOtqIwTcfSajtuAd v9vVoALQeTRT9DoBTCVMZJ3JW8zAZ0qtmVGnUw8XK+uwlpVcNuokZVB+Ng8shCvVza CEBlJPJcgFQ4jMJdnvAMPHjCw/u+P4MHPdrFWt9367/RHJKpNWlCbCDrtzgDUxsUMn EIUl5dZegNHRw== Subject: [PATCH v2 2/4] SUNRPC: Send RPC message on TCP with a single sock_sendmsg() call From: Chuck Lever To: linux-nfs@vger.kernel.org, netdev@vger.kernel.org Cc: David Howells , Chuck Lever , dhowells@redhat.com Date: Fri, 14 Jul 2023 14:10:44 -0400 Message-ID: <168935824410.1984.15912253826414043702.stgit@manet.1015granger.net> In-Reply-To: <168935791041.1984.13295336680505732841.stgit@manet.1015granger.net> References: <168935791041.1984.13295336680505732841.stgit@manet.1015granger.net> User-Agent: StGit/1.5 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Chuck Lever There is now enough infrastructure in place to combine the stream record marker into the biovec array used to send each outgoing RPC message. The whole message can be more efficiently sent with a single call to sock_sendmsg() using a bio_vec iterator. Note that this also helps with RPC-with-TLS: the TLS implementation can now clearly see where the upper layer message boundaries are. Before, it would send each component of the xdr_buf in a separate TLS record. Suggested-by: David Howells Signed-off-by: Chuck Lever --- include/linux/sunrpc/svcsock.h | 2 ++ net/sunrpc/svcsock.c | 33 ++++++++++++++++++--------------- 2 files changed, 20 insertions(+), 15 deletions(-) diff --git a/include/linux/sunrpc/svcsock.h b/include/linux/sunrpc/svcsock.h index a9bfeadf4cbe..baea012e3b04 100644 --- a/include/linux/sunrpc/svcsock.h +++ b/include/linux/sunrpc/svcsock.h @@ -38,6 +38,8 @@ struct svc_sock { /* Number of queued send requests */ atomic_t sk_sendqlen; + struct page_frag_cache sk_frag_cache; + struct completion sk_handshake_done; struct bio_vec sk_send_bvec[RPCSVC_MAXPAGES] diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c index e35e5afe4b81..bb185c0bb57c 100644 --- a/net/sunrpc/svcsock.c +++ b/net/sunrpc/svcsock.c @@ -1213,31 +1213,30 @@ static int svc_tcp_recvfrom(struct svc_rqst *rqstp) static int svc_tcp_sendmsg(struct svc_sock *svsk, struct xdr_buf *xdr, rpc_fraghdr marker, unsigned int *sentp) { - struct kvec rm = { - .iov_base = &marker, - .iov_len = sizeof(marker), - }; struct msghdr msg = { - .msg_flags = MSG_MORE, + .msg_flags = MSG_SPLICE_PAGES, }; unsigned int count; + void *tmp; int ret; *sentp = 0; - ret = kernel_sendmsg(svsk->sk_sock, &msg, &rm, 1, rm.iov_len); - if (ret < 0) - return ret; - *sentp += ret; - if (ret != rm.iov_len) - return -EAGAIN; + /* The stream record marker is copied into a temporary page + * buffer so that it can be included in sk_send_bvec. + */ + tmp = page_frag_alloc(&svsk->sk_frag_cache, sizeof(marker), + GFP_KERNEL); + if (!tmp) + return -ENOMEM; + memcpy(tmp, &marker, sizeof(marker)); + bvec_set_virt(svsk->sk_send_bvec, tmp, sizeof(marker)); - count = xdr_buf_to_bvec(svsk->sk_send_bvec, - ARRAY_SIZE(svsk->sk_send_bvec), xdr); + count = xdr_buf_to_bvec(svsk->sk_send_bvec + 1, + ARRAY_SIZE(svsk->sk_send_bvec) - 1, xdr); - msg.msg_flags = MSG_SPLICE_PAGES; iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, svsk->sk_send_bvec, - count, xdr->len); + 1 + count, sizeof(marker) + xdr->len); ret = sock_sendmsg(svsk->sk_sock, &msg); if (ret < 0) return ret; @@ -1616,6 +1615,7 @@ static void svc_tcp_sock_detach(struct svc_xprt *xprt) static void svc_sock_free(struct svc_xprt *xprt) { struct svc_sock *svsk = container_of(xprt, struct svc_sock, sk_xprt); + struct page_frag_cache *pfc = &svsk->sk_frag_cache; struct socket *sock = svsk->sk_sock; trace_svcsock_free(svsk, sock); @@ -1625,5 +1625,8 @@ static void svc_sock_free(struct svc_xprt *xprt) sockfd_put(sock); else sock_release(sock); + if (pfc->va) + __page_frag_cache_drain(virt_to_head_page(pfc->va), + pfc->pagecnt_bias); kfree(svsk); } From patchwork Fri Jul 14 18:10:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 13314152 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3A663EB64DC for ; Fri, 14 Jul 2023 18:11:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236429AbjGNSLy (ORCPT ); Fri, 14 Jul 2023 14:11:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58934 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236430AbjGNSLv (ORCPT ); Fri, 14 Jul 2023 14:11:51 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A7E3635B6 for ; Fri, 14 Jul 2023 11:11:24 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 779C761DA4 for ; Fri, 14 Jul 2023 18:10:52 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 96005C433C8; Fri, 14 Jul 2023 18:10:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1689358251; bh=nIwnDpeLNNXqIqXQ2IZyBusc1AZA0y1HhYlRmT4gqbU=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=RKQeX5dxvCCGQHXzqHhU19LZik+aQZE7pEZpcPOXKOHmH7N2TE3kGiUciP9me5S0D vZMSdHLNslH3xWr/iRDn4C84pBp78bAJcA/YTyG86e+AHVYjrQ06HD8F7h1qedu+15 QCcunZPq57zIDmewUGTIrhcJu0daitKDYJ1w5xjKDMpPl+TPFD16sH7svKHgYThFdR e31pkypO5soEC99rc4Fw0lp6UO+n6z7Jf0c8ddoj2U44LsKB8Gphufq5PRPTHGMFdi UVT1Gv+uI6OmO4I9DKO78aV9dWapHv1e9IoHNXKqBAYuzBsxet7CEuv+PmR6hQ5xjn yQ7XcwEGUTMIg== Subject: [PATCH v2 3/4] SUNRPC: Convert svc_udp_sendto() to use the per-socket bio_vec array From: Chuck Lever To: linux-nfs@vger.kernel.org, netdev@vger.kernel.org Cc: Chuck Lever , dhowells@redhat.com Date: Fri, 14 Jul 2023 14:10:50 -0400 Message-ID: <168935825062.1984.15414767603272782406.stgit@manet.1015granger.net> In-Reply-To: <168935791041.1984.13295336680505732841.stgit@manet.1015granger.net> References: <168935791041.1984.13295336680505732841.stgit@manet.1015granger.net> User-Agent: StGit/1.5 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Chuck Lever Commit da1661b93bf4 ("SUNRPC: Teach server to use xprt_sock_sendmsg for socket sends") modified svc_udp_sendto() to use xprt_sock_sendmsg() because we originally believed xprt_sock_sendmsg() would be needed for TLS support. That does not actually appear to be the case. In addition, the linkage between the client and server send code has been a bit of a maintenance headache because of the distinct ways that the client and server handle memory allocation. Going forward, eventually the XDR layer will deal with its buffers in the form of bio_vec arrays, so convert this function accordingly. Once the use of bio_vecs is ubiquitous, the xdr_buf-to-bio_vec array code can be hoisted into a path that is common for all transports. Signed-off-by: Chuck Lever --- net/sunrpc/svcsock.c | 23 ++++++++++++----------- 1 file changed, 12 insertions(+), 11 deletions(-) diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c index bb185c0bb57c..e164ea4d0e0a 100644 --- a/net/sunrpc/svcsock.c +++ b/net/sunrpc/svcsock.c @@ -695,7 +695,7 @@ static int svc_udp_sendto(struct svc_rqst *rqstp) .msg_control = cmh, .msg_controllen = sizeof(buffer), }; - unsigned int sent; + unsigned int count; int err; svc_udp_release_ctxt(xprt, rqstp->rq_xprt_ctxt); @@ -708,22 +708,23 @@ static int svc_udp_sendto(struct svc_rqst *rqstp) if (svc_xprt_is_dead(xprt)) goto out_notconn; - err = xdr_alloc_bvec(xdr, GFP_KERNEL); - if (err < 0) - goto out_unlock; + count = xdr_buf_to_bvec(svsk->sk_send_bvec, + ARRAY_SIZE(svsk->sk_send_bvec), xdr); - err = xprt_sock_sendmsg(svsk->sk_sock, &msg, xdr, 0, 0, &sent); + iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, svsk->sk_send_bvec, + count, 0); + err = sock_sendmsg(svsk->sk_sock, &msg); if (err == -ECONNREFUSED) { /* ICMP error on earlier request. */ - err = xprt_sock_sendmsg(svsk->sk_sock, &msg, xdr, 0, 0, &sent); + iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, svsk->sk_send_bvec, + count, 0); + err = sock_sendmsg(svsk->sk_sock, &msg); } - xdr_free_bvec(xdr); + trace_svcsock_udp_send(xprt, err); -out_unlock: + mutex_unlock(&xprt->xpt_mutex); - if (err < 0) - return err; - return sent; + return err; out_notconn: mutex_unlock(&xprt->xpt_mutex); From patchwork Fri Jul 14 18:10:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 13314153 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A9002C001B0 for ; Fri, 14 Jul 2023 18:11:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236523AbjGNSLy (ORCPT ); Fri, 14 Jul 2023 14:11:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58916 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236369AbjGNSLv (ORCPT ); Fri, 14 Jul 2023 14:11:51 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B1CC03C33 for ; Fri, 14 Jul 2023 11:11:24 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id EA78561D96 for ; Fri, 14 Jul 2023 18:10:58 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1272CC433C7; Fri, 14 Jul 2023 18:10:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1689358258; bh=bxTAZ0rCFm1dsqjLX5lA1j5rbtFHpQAZvI1adKlXSo0=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=ozgRLv466jxGoGME6dN6gOohalvCaZCxpcqp4/qBQfhgKuBUEM5zcgjdfbq1XQg5e NmsVM+37aQwGegDpvxIZWyMQbL7Djz6REMqNZcxi3XDPx1gVBYyVjzHi9VK91lugx0 iDzhmg/M/pm2Q1XfIz2PTGcvrbktwtbsFcicZAIx8L2wneqOgmbcPOWNBGOUuW+/KX x7++FCib52tddkhe2iRa0ZU0MFL7ITLoIpEYOiStb69B/6Vrs9jb0nawn5iCPhY3RV pe0p5iKLCTdtkbI3OeP7SwgQ8N3ZpqpJnyn1G4rIHdqmm3q3X3MpaTn8+Q3Lj0IXYn 5C4d7WZSOU+tg== Subject: [PATCH v2 4/4] SUNRPC: Use a per-transport receive bio_vec array From: Chuck Lever To: linux-nfs@vger.kernel.org, netdev@vger.kernel.org Cc: Chuck Lever , dhowells@redhat.com Date: Fri, 14 Jul 2023 14:10:57 -0400 Message-ID: <168935825709.1984.4898358403212846149.stgit@manet.1015granger.net> In-Reply-To: <168935791041.1984.13295336680505732841.stgit@manet.1015granger.net> References: <168935791041.1984.13295336680505732841.stgit@manet.1015granger.net> User-Agent: StGit/1.5 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Chuck Lever TCP receives are serialized, so we need only one bio_vec array per socket. This shrinks the size of struct svc_rqst by 4144 bytes on x86_64. Signed-off-by: Chuck Lever --- include/linux/sunrpc/svc.h | 1 - include/linux/sunrpc/svcsock.h | 2 ++ net/sunrpc/svcsock.c | 2 +- 3 files changed, 3 insertions(+), 2 deletions(-) diff --git a/include/linux/sunrpc/svc.h b/include/linux/sunrpc/svc.h index f8751118c122..36052188222d 100644 --- a/include/linux/sunrpc/svc.h +++ b/include/linux/sunrpc/svc.h @@ -224,7 +224,6 @@ struct svc_rqst { struct folio_batch rq_fbatch; struct kvec rq_vec[RPCSVC_MAXPAGES]; /* generally useful.. */ - struct bio_vec rq_bvec[RPCSVC_MAXPAGES]; __be32 rq_xid; /* transmission id */ u32 rq_prog; /* program number */ diff --git a/include/linux/sunrpc/svcsock.h b/include/linux/sunrpc/svcsock.h index baea012e3b04..55446136499f 100644 --- a/include/linux/sunrpc/svcsock.h +++ b/include/linux/sunrpc/svcsock.h @@ -42,6 +42,8 @@ struct svc_sock { struct completion sk_handshake_done; + struct bio_vec sk_recv_bvec[RPCSVC_MAXPAGES] + ____cacheline_aligned; struct bio_vec sk_send_bvec[RPCSVC_MAXPAGES] ____cacheline_aligned; diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c index e164ea4d0e0a..5cbc35e23e4f 100644 --- a/net/sunrpc/svcsock.c +++ b/net/sunrpc/svcsock.c @@ -299,7 +299,7 @@ static ssize_t svc_tcp_read_msg(struct svc_rqst *rqstp, size_t buflen, { struct svc_sock *svsk = container_of(rqstp->rq_xprt, struct svc_sock, sk_xprt); - struct bio_vec *bvec = rqstp->rq_bvec; + struct bio_vec *bvec = svsk->sk_recv_bvec; struct msghdr msg = { NULL }; unsigned int i; ssize_t len;