From patchwork Sun Jun 5 19:58:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 12869854 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D6BA7C43334 for ; Sun, 5 Jun 2022 19:58:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345154AbiFET6c (ORCPT ); Sun, 5 Jun 2022 15:58:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60606 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1343714AbiFET6b (ORCPT ); Sun, 5 Jun 2022 15:58:31 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5CFAB4A3D3 for ; Sun, 5 Jun 2022 12:58:29 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 18518B80DCA for ; Sun, 5 Jun 2022 19:58:28 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id ADFA8C34119 for ; Sun, 5 Jun 2022 19:58:26 +0000 (UTC) Subject: [PATCH v1 1/5] SUNRPC: Fix the calculation of xdr->end in xdr_get_next_encode_buffer() From: Chuck Lever To: linux-nfs@vger.kernel.org Date: Sun, 05 Jun 2022 15:58:25 -0400 Message-ID: <165445910560.1664.5852151724543272982.stgit@bazille.1015granger.net> In-Reply-To: <165445865736.1664.4497130284832282034.stgit@bazille.1015granger.net> References: <165445865736.1664.4497130284832282034.stgit@bazille.1015granger.net> User-Agent: StGit/1.5 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org I found that NFSD's new NFSv3 READDIRPLUS XDR encoder was screwing up right at the end of the page array. xdr_get_next_encode_buffer() does not compute the value of xdr->end correctly: * The check to see if we're on the final available page in xdr->buf needs to account for the space consumed by @nbytes. * The new xdr->end value needs to account for the portion of @nbytes that is to be encoded into the previous buffer. Fixes: 2825a7f90753 ("nfsd4: allow encoding across page boundaries") Signed-off-by: Chuck Lever --- net/sunrpc/xdr.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/net/sunrpc/xdr.c b/net/sunrpc/xdr.c index df194cc07035..b57cf9df4de8 100644 --- a/net/sunrpc/xdr.c +++ b/net/sunrpc/xdr.c @@ -979,7 +979,11 @@ static __be32 *xdr_get_next_encode_buffer(struct xdr_stream *xdr, */ xdr->p = (void *)p + frag2bytes; space_left = xdr->buf->buflen - xdr->buf->len; - xdr->end = (void *)p + min_t(int, space_left, PAGE_SIZE); + if (space_left - nbytes >= PAGE_SIZE) + xdr->end = (void *)p + PAGE_SIZE; + else + xdr->end = (void *)p + space_left - frag1bytes; + xdr->buf->page_len += frag2bytes; xdr->buf->len += nbytes; return p; From patchwork Sun Jun 5 19:58:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 12869855 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B1960C43334 for ; Sun, 5 Jun 2022 19:58:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344478AbiFET6k (ORCPT ); Sun, 5 Jun 2022 15:58:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32826 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1347132AbiFET6g (ORCPT ); Sun, 5 Jun 2022 15:58:36 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CEAD04D9C6 for ; Sun, 5 Jun 2022 12:58:35 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 50798B80DCA for ; Sun, 5 Jun 2022 19:58:34 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E759FC385A5 for ; Sun, 5 Jun 2022 19:58:32 +0000 (UTC) Subject: [PATCH v1 2/5] SUNRPC: Optimize xdr_reserve_space() From: Chuck Lever To: linux-nfs@vger.kernel.org Date: Sun, 05 Jun 2022 15:58:31 -0400 Message-ID: <165445911199.1664.12318094116152634589.stgit@bazille.1015granger.net> In-Reply-To: <165445865736.1664.4497130284832282034.stgit@bazille.1015granger.net> References: <165445865736.1664.4497130284832282034.stgit@bazille.1015granger.net> User-Agent: StGit/1.5 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org The xdr_get_next_encode_buffer() function is called infrequently. On a typical build/test workload, it's called about 1 time in 400 calls to xdr_reserve_space() (measured on NFSD). Force the compiler to remove it from xdr_reserve_space(), which is a hot path. This change reduces the size of xdr_reserve_space() from 10 cache lines to 4 on my test system. Signed-off-by: Chuck Lever --- net/sunrpc/xdr.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/net/sunrpc/xdr.c b/net/sunrpc/xdr.c index b57cf9df4de8..08a85375b311 100644 --- a/net/sunrpc/xdr.c +++ b/net/sunrpc/xdr.c @@ -945,8 +945,13 @@ inline void xdr_commit_encode(struct xdr_stream *xdr) } EXPORT_SYMBOL_GPL(xdr_commit_encode); -static __be32 *xdr_get_next_encode_buffer(struct xdr_stream *xdr, - size_t nbytes) +/* + * The buffer space to be reserved crosses the boundary between + * xdr->buf->head and xdr->buf->pages, or between two pages + * in xdr->buf->pages. + */ +static noinline __be32 *xdr_get_next_encode_buffer(struct xdr_stream *xdr, + size_t nbytes) { __be32 *p; int space_left; From patchwork Sun Jun 5 19:58:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 12869856 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3C9FBC433EF for ; Sun, 5 Jun 2022 19:58:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243571AbiFET6p (ORCPT ); Sun, 5 Jun 2022 15:58:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33382 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1347198AbiFET6n (ORCPT ); Sun, 5 Jun 2022 15:58:43 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 82C0E4A3D3 for ; Sun, 5 Jun 2022 12:58:40 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id CE5196118D for ; Sun, 5 Jun 2022 19:58:39 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2EA01C385A5 for ; Sun, 5 Jun 2022 19:58:39 +0000 (UTC) Subject: [PATCH v1 3/5] SUNRPC: Clean up xdr_commit_encode() From: Chuck Lever To: linux-nfs@vger.kernel.org Date: Sun, 05 Jun 2022 15:58:38 -0400 Message-ID: <165445911819.1664.8436212544546306528.stgit@bazille.1015granger.net> In-Reply-To: <165445865736.1664.4497130284832282034.stgit@bazille.1015granger.net> References: <165445865736.1664.4497130284832282034.stgit@bazille.1015granger.net> User-Agent: StGit/1.5 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Both the ::iov_len field and the third parameter of memcpy() and memmove() are size_t. There's no reason for the implicit conversion from size_t to int and back. Change the type of @shift to make the code easier to read and understand. Signed-off-by: Chuck Lever Reviewed-by: NeilBrown --- net/sunrpc/xdr.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/net/sunrpc/xdr.c b/net/sunrpc/xdr.c index 08a85375b311..de8f71468637 100644 --- a/net/sunrpc/xdr.c +++ b/net/sunrpc/xdr.c @@ -933,14 +933,16 @@ EXPORT_SYMBOL_GPL(xdr_init_encode); */ inline void xdr_commit_encode(struct xdr_stream *xdr) { - int shift = xdr->scratch.iov_len; + size_t shift = xdr->scratch.iov_len; void *page; if (shift == 0) return; + page = page_address(*xdr->page_ptr); memcpy(xdr->scratch.iov_base, page, shift); memmove(page, page + shift, (void *)xdr->p - page); + xdr_reset_scratch_buffer(xdr); } EXPORT_SYMBOL_GPL(xdr_commit_encode); From patchwork Sun Jun 5 19:58:44 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 12869857 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B7EF5C43334 for ; Sun, 5 Jun 2022 19:58:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1347198AbiFET6t (ORCPT ); Sun, 5 Jun 2022 15:58:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33834 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1347132AbiFET6s (ORCPT ); Sun, 5 Jun 2022 15:58:48 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1DD5D4A3D3 for ; Sun, 5 Jun 2022 12:58:48 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id CC9B7B80DCA for ; Sun, 5 Jun 2022 19:58:46 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7EBB4C385A5 for ; Sun, 5 Jun 2022 19:58:45 +0000 (UTC) Subject: [PATCH v1 4/5] SUNRPC: Clean up xdr_get_next_encode_buffer() From: Chuck Lever To: linux-nfs@vger.kernel.org Date: Sun, 05 Jun 2022 15:58:44 -0400 Message-ID: <165445912444.1664.7733109680322574177.stgit@bazille.1015granger.net> In-Reply-To: <165445865736.1664.4497130284832282034.stgit@bazille.1015granger.net> References: <165445865736.1664.4497130284832282034.stgit@bazille.1015granger.net> User-Agent: StGit/1.5 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org The value of @p is not used until the "location of the next item" is computed. Help human readers by moving its initial assignment to the paragraph where that value is used and by clarifying the antecedents in the documenting comment. Signed-off-by: Chuck Lever Reviewed-by: NeilBrown --- net/sunrpc/xdr.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/net/sunrpc/xdr.c b/net/sunrpc/xdr.c index de8f71468637..89cb48931a1f 100644 --- a/net/sunrpc/xdr.c +++ b/net/sunrpc/xdr.c @@ -971,6 +971,7 @@ static noinline __be32 *xdr_get_next_encode_buffer(struct xdr_stream *xdr, xdr->buf->page_len += frag1bytes; xdr->page_ptr++; xdr->iov = NULL; + /* * If the last encode didn't end exactly on a page boundary, the * next one will straddle boundaries. Encode into the next @@ -979,11 +980,12 @@ static noinline __be32 *xdr_get_next_encode_buffer(struct xdr_stream *xdr, * space at the end of the previous buffer: */ xdr_set_scratch_buffer(xdr, xdr->p, frag1bytes); - p = page_address(*xdr->page_ptr); + /* - * Note this is where the next encode will start after we've - * shifted this one back: + * xdr->p is where the next encode will start after + * xdr_commit_encode() has shifted this one back: */ + p = page_address(*xdr->page_ptr); xdr->p = (void *)p + frag2bytes; space_left = xdr->buf->buflen - xdr->buf->len; if (space_left - nbytes >= PAGE_SIZE) From patchwork Sun Jun 5 19:58:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 12869858 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 38CC2C433EF for ; Sun, 5 Jun 2022 19:58:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1347133AbiFET64 (ORCPT ); Sun, 5 Jun 2022 15:58:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34498 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1347132AbiFET6z (ORCPT ); Sun, 5 Jun 2022 15:58:55 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7D33949687 for ; Sun, 5 Jun 2022 12:58:54 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 2A4A2B80DAE for ; Sun, 5 Jun 2022 19:58:53 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id DAACDC385A5 for ; Sun, 5 Jun 2022 19:58:51 +0000 (UTC) Subject: [PATCH v1 5/5] SUNRPC: Remove pointer type casts from xdr_get_next_encode_buffer() From: Chuck Lever To: linux-nfs@vger.kernel.org Date: Sun, 05 Jun 2022 15:58:50 -0400 Message-ID: <165445913079.1664.5467024491080755855.stgit@bazille.1015granger.net> In-Reply-To: <165445865736.1664.4497130284832282034.stgit@bazille.1015granger.net> References: <165445865736.1664.4497130284832282034.stgit@bazille.1015granger.net> User-Agent: StGit/1.5 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org To make the code easier to read, remove visual clutter by changing the declared type of @p. Signed-off-by: Chuck Lever Reviewed-by: NeilBrown --- net/sunrpc/xdr.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/net/sunrpc/xdr.c b/net/sunrpc/xdr.c index 89cb48931a1f..4e4cad7aeec2 100644 --- a/net/sunrpc/xdr.c +++ b/net/sunrpc/xdr.c @@ -955,9 +955,9 @@ EXPORT_SYMBOL_GPL(xdr_commit_encode); static noinline __be32 *xdr_get_next_encode_buffer(struct xdr_stream *xdr, size_t nbytes) { - __be32 *p; int space_left; int frag1bytes, frag2bytes; + void *p; if (nbytes > PAGE_SIZE) goto out_overflow; /* Bigger buffers require special handling */ @@ -986,12 +986,12 @@ static noinline __be32 *xdr_get_next_encode_buffer(struct xdr_stream *xdr, * xdr_commit_encode() has shifted this one back: */ p = page_address(*xdr->page_ptr); - xdr->p = (void *)p + frag2bytes; + xdr->p = p + frag2bytes; space_left = xdr->buf->buflen - xdr->buf->len; if (space_left - nbytes >= PAGE_SIZE) - xdr->end = (void *)p + PAGE_SIZE; + xdr->end = p + PAGE_SIZE; else - xdr->end = (void *)p + space_left - frag1bytes; + xdr->end = p + space_left - frag1bytes; xdr->buf->page_len += frag2bytes; xdr->buf->len += nbytes;