From patchwork Fri Jan 3 01:00:01 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 13925075 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0FDF214F9F4 for ; Fri, 3 Jan 2025 01:00:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735866006; cv=none; b=cwm0sbN5Gqa1cBb/fYPRtwZMvp4W0QDBjTWJClMp2GxC3PFeWo+2TcBwuDcL3/bamcb1n3ZRPEqzWvejqL2bFEgZMtYQ7SBgRkHlAdiwcDvY5zT/dYuInutCRVlXnT7M4bfmmqhcUUdWVa3sxitl8yrS7GiymNch3qAZaV2in4M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735866006; c=relaxed/simple; bh=NigUHel74W5sekzb/UR6jksqqUz4wIYjTPl2MGxnXvw=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=rjhAZAKTgBauH2UtBGOcxHSozpQTU8fIbgED5+srNBAg2erHYKZBtXFwbFBV9bYRHkKkngf4JAfH5VUS5l7rHqoKTOBc2VTu/kZtsyBhOg5bImjysdrJVFYr/FO2ot6ApVDGbXD2p2u++XATpdVxdnNa1kwUMTs/yKCYUMXU7oE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=MOgpjnRq; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="MOgpjnRq" Received: by smtp.kernel.org (Postfix) with ESMTPSA id BEDFBC4CED0; Fri, 3 Jan 2025 01:00:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1735866005; bh=NigUHel74W5sekzb/UR6jksqqUz4wIYjTPl2MGxnXvw=; h=From:To:Cc:Subject:Date:From; b=MOgpjnRq2gUz6yGtxnTOQmsuw4wnEZtoCIEG4eHGqX98tkVdmBhUIKO8i8YwTbJOB erNvg0ktjE79CaVoq+E3Ef75JmFMtWGD9H7qPb3xjiC7aa3GNJON19OJ9R8yHRBElW C/2Uuvl4QvMFwpqHBkWIbOeRH+6A25Jp4RQqlGaZDMPCIqVK8uGQhtKtteXg0ChFYg 80AY0mrCCQ10LoBQvKksJt/bh5aYbQyl5+3ACE+fAoDyYG+U41gHk1ztpgZwgMpjMt SB0PUhaepxTjJiiAEEuC3Hp4o3+kQsd/xCup4Aig/WRJYV1ibIFdnKCK9UifP46d69 +7bQIC/USEB/g== From: cel@kernel.org To: Neil Brown , Jeff Layton , Olga Kornievskaia , Dai Ngo , Tom Talpey Cc: , Chuck Lever , Jakub Kicinski Subject: [PATCH v1 1/2] Revert "SUNRPC: Reduce thread wake-up rate when receiving large RPC messages" Date: Thu, 2 Jan 2025 20:00:01 -0500 Message-ID: <20250103010002.619062-1-cel@kernel.org> X-Mailer: git-send-email 2.47.0 Precedence: bulk X-Mailing-List: linux-nfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Chuck Lever I noticed that a handful of NFSv3 fstests were taking an unexpectedly long time to run. Troubleshooting showed that the server's TCP window closed and never re-opened, which caused the client to trigger an RPC retransmit timeout after 180 seconds. The client's recovery action was to establish a fresh connection and retransmit the timed-out requests. This worked, but it adds a long delay. I tracked the problem to the commit that attempted to reduce the rate at which the network layer delivers TCP socket data_ready callbacks. Under most circumstances this change worked as expected, but for NFSv3, which has no session or other type of throttling, it can overwhelm the receiver on occasion. I'm sure I could tweak the lowat settings, but the small benefit doesn't seem worth the bother. Just revert it. Fixes: 2b877fc53e97 ("SUNRPC: Reduce thread wake-up rate when receiving large RPC messages") Cc: Jakub Kicinski Signed-off-by: Chuck Lever Reviewed-by: Jeff Layton --- net/sunrpc/svcsock.c | 12 +----------- 1 file changed, 1 insertion(+), 11 deletions(-) diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c index 95397677673b..cb3bd12f5818 100644 --- a/net/sunrpc/svcsock.c +++ b/net/sunrpc/svcsock.c @@ -1083,9 +1083,6 @@ static void svc_tcp_fragment_received(struct svc_sock *svsk) /* If we have more data, signal svc_xprt_enqueue() to try again */ svsk->sk_tcplen = 0; svsk->sk_marker = xdr_zero; - - smp_wmb(); - tcp_set_rcvlowat(svsk->sk_sk, 1); } /** @@ -1175,17 +1172,10 @@ static int svc_tcp_recvfrom(struct svc_rqst *rqstp) goto err_delete; if (len == want) svc_tcp_fragment_received(svsk); - else { - /* Avoid more ->sk_data_ready() calls until the rest - * of the message has arrived. This reduces service - * thread wake-ups on large incoming messages. */ - tcp_set_rcvlowat(svsk->sk_sk, - svc_sock_reclen(svsk) - svsk->sk_tcplen); - + else trace_svcsock_tcp_recv_short(&svsk->sk_xprt, svc_sock_reclen(svsk), svsk->sk_tcplen - sizeof(rpc_fraghdr)); - } goto err_noclose; error: if (len != -EAGAIN) From patchwork Fri Jan 3 01:00:02 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 13925076 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C8411154BF0 for ; Fri, 3 Jan 2025 01:00:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735866006; cv=none; b=lwCWqMA7BFaNBVcCJD8jWIy1EVQE8n7VJ7I3KA1zVHo3/OpF63YY4beXVXVn5txevVvCj8LpIBYH4C7K7My+jAIwrvxOwWoXdzZSreePOV3cuvbXNl+lGpLLNJNm6SW5NKxa06uGNM2QaxOP7R7WSw/oCqSA3SvGVj4EVg6wg4U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735866006; c=relaxed/simple; bh=7corJfirsBEpEr7KZsy2vGiur1pNPia2m+YxI+yeYNc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=uURTdmKDyoLBsuyotPzt9FXLGQtuRcEumrIIJoF7Z3GKvv/M8f3wOMuAyEX2CQyBekr9/O4vabOyAEhkGomM//Qwpa3WVvu04NKjYvgWUL3zlH2W17Bo1ekc0IlfjRbbmNTCf2uH+jd1uDVvNiLw4srv83HoLLDhxwx1KMR2Ul4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=V3PvtClL; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="V3PvtClL" Received: by smtp.kernel.org (Postfix) with ESMTPSA id C2D00C4CEDE; Fri, 3 Jan 2025 01:00:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1735866006; bh=7corJfirsBEpEr7KZsy2vGiur1pNPia2m+YxI+yeYNc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=V3PvtClLWjkGSoaUaffuNJ8buRP4oBcTabzwTQZPH3IGMKahJxSiRGtk+sGwhA4eJ 3G7Jjfhwf8se8aPe8KBED7QNqiGHeVqNDxmaKhSNu2s8Ml4iXacjuLUwWH9AcKEw0q aTczvkkX95hsQ/ksfTVu98fxFAiCcwXsDZgqaMc1EAIImnNO7cYJ50s/2MVwitBDE1 0BM877J/7JaG5vQXmwFpf9rHU0Njjv46vHykgiOk484CA9o82pciMPvhOysBiOfyec 8ZIFtBf9wbnIP2VUNXzE4zWvrRHU1UHTqb2RzoNFOzdfa4CvadNw8PCARbJJpiu9AJ uZjN/xnNV+vuQ== From: cel@kernel.org To: Neil Brown , Jeff Layton , Olga Kornievskaia , Dai Ngo , Tom Talpey Cc: , Chuck Lever , Tejun Heo Subject: [PATCH v1 2/2] NFSD: Change the filecache laundrette workqueue again Date: Thu, 2 Jan 2025 20:00:02 -0500 Message-ID: <20250103010002.619062-2-cel@kernel.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250103010002.619062-1-cel@kernel.org> References: <20250103010002.619062-1-cel@kernel.org> Precedence: bulk X-Mailing-List: linux-nfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Chuck Lever Youzhong Yang noticed the workqueue subsystem complaining about how long the filecache laundrette was running. This resulted in switching from using the system_wq for the laundrette to system_unbound_wq (see commit 4b84551a35e3 ("nfsd: use system_unbound_wq for nfsd_file_gc_worker()"). However, I've seen the laundrette running for multiple milliseconds on some workloads, delaying other work. For the purpose of scheduling fairness, perhaps a better choice would be to process filecache disposal queues on the system_long_wq instead. Cc: Tejun Heo Signed-off-by: Chuck Lever Reviewed-by: Jeff Layton --- fs/nfsd/filecache.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/nfsd/filecache.c b/fs/nfsd/filecache.c index a1cdba42c4fa..91a535c2dede 100644 --- a/fs/nfsd/filecache.c +++ b/fs/nfsd/filecache.c @@ -112,7 +112,7 @@ static void nfsd_file_schedule_laundrette(void) { if (test_bit(NFSD_FILE_CACHE_UP, &nfsd_file_flags)) - queue_delayed_work(system_unbound_wq, &nfsd_filecache_laundrette, + queue_delayed_work(system_long_wq, &nfsd_filecache_laundrette, NFSD_LAUNDRETTE_DELAY); }