From patchwork Wed Jul 14 15:50:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 12377383 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B6B6DC11F68 for ; Wed, 14 Jul 2021 15:50:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9FEDF61374 for ; Wed, 14 Jul 2021 15:50:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239748AbhGNPxK (ORCPT ); Wed, 14 Jul 2021 11:53:10 -0400 Received: from mail.kernel.org ([198.145.29.99]:41088 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239674AbhGNPxK (ORCPT ); Wed, 14 Jul 2021 11:53:10 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 5D18A6128D for ; Wed, 14 Jul 2021 15:50:18 +0000 (UTC) Subject: [PATCH RFC 1/4] NFS: Unset RPC_TASK_NO_RETRANS_TIMEOUT for async lease renewal From: Chuck Lever To: linux-nfs@vger.kernel.org Date: Wed, 14 Jul 2021 11:50:17 -0400 Message-ID: <162627781762.1294.17862468684529354297.stgit@manet.1015granger.net> In-Reply-To: <162627611661.1294.9189768423517916152.stgit@manet.1015granger.net> References: <162627611661.1294.9189768423517916152.stgit@manet.1015granger.net> User-Agent: StGit/1.1 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org In some rare failure modes, the server is actually reading the transport, but then just dropping the requests on the floor. TCP_USER_TIMEOUT cannot detect that case. Prevent such a stuck server from pinning client resources indefinitely by ensuring that async lease renewal requests can time out even if the connection is still operational. Signed-off-by: Chuck Lever --- fs/nfs/nfs4proc.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c index e1214bb6b7ee..346217f6a00b 100644 --- a/fs/nfs/nfs4proc.c +++ b/fs/nfs/nfs4proc.c @@ -5612,6 +5612,12 @@ struct nfs4_renewdata { * nfs4_proc_async_renew(): This is not one of the nfs_rpc_ops; it is a special * standalone procedure for queueing an asynchronous RENEW. */ +static void nfs4_renew_prepare(struct rpc_task *task, void *calldata) +{ + task->tk_flags &= ~RPC_TASK_NO_RETRANS_TIMEOUT; + rpc_call_start(task); +} + static void nfs4_renew_release(void *calldata) { struct nfs4_renewdata *data = calldata; @@ -5650,6 +5656,7 @@ static void nfs4_renew_done(struct rpc_task *task, void *calldata) } static const struct rpc_call_ops nfs4_renew_ops = { + .rpc_call_prepare = nfs4_renew_prepare, .rpc_call_done = nfs4_renew_done, .rpc_release = nfs4_renew_release, }; @@ -9219,6 +9226,8 @@ static void nfs41_sequence_prepare(struct rpc_task *task, void *data) struct nfs4_sequence_args *args; struct nfs4_sequence_res *res; + task->tk_flags &= ~RPC_TASK_NO_RETRANS_TIMEOUT; + args = task->tk_msg.rpc_argp; res = task->tk_msg.rpc_resp; From patchwork Wed Jul 14 15:50:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 12377385 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 605D6C07E9A for ; Wed, 14 Jul 2021 15:50:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4A7EB6128D for ; Wed, 14 Jul 2021 15:50:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239674AbhGNPxQ (ORCPT ); Wed, 14 Jul 2021 11:53:16 -0400 Received: from mail.kernel.org ([198.145.29.99]:41136 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239625AbhGNPxQ (ORCPT ); Wed, 14 Jul 2021 11:53:16 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 546F9613BE for ; Wed, 14 Jul 2021 15:50:24 +0000 (UTC) Subject: [PATCH RFC 2/4] NFS: Unset RPC_TASK_NO_RETRANS_TIMEOUT for session/clientid destruction From: Chuck Lever To: linux-nfs@vger.kernel.org Date: Wed, 14 Jul 2021 11:50:23 -0400 Message-ID: <162627782362.1294.9395366920293772038.stgit@manet.1015granger.net> In-Reply-To: <162627611661.1294.9189768423517916152.stgit@manet.1015granger.net> References: <162627611661.1294.9189768423517916152.stgit@manet.1015granger.net> User-Agent: StGit/1.1 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org In some rare failure modes, the server is actually reading the transport, but then just dropping the requests on the floor. TCP_USER_TIMEOUT cannot detect that case. Prevent such a stuck server from pinning client resources indefinitely by ensuring that session and client ID clean-up can time out even if the connection is still operational. Signed-off-by: Chuck Lever --- fs/nfs/nfs4client.c | 1 + 1 file changed, 1 insertion(+) diff --git a/fs/nfs/nfs4client.c b/fs/nfs/nfs4client.c index 28431acd1230..c5032f784ac0 100644 --- a/fs/nfs/nfs4client.c +++ b/fs/nfs/nfs4client.c @@ -281,6 +281,7 @@ static void nfs4_destroy_callback(struct nfs_client *clp) static void nfs4_shutdown_client(struct nfs_client *clp) { + clp->cl_rpcclient->cl_noretranstimeo = 0; if (__test_and_clear_bit(NFS_CS_RENEWD, &clp->cl_res_state)) nfs4_kill_renewd(clp); clp->cl_mvops->shutdown_client(clp); From patchwork Wed Jul 14 15:50:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 12377387 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 415BDC07E9A for ; Wed, 14 Jul 2021 15:50:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 26C786128D for ; Wed, 14 Jul 2021 15:50:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239625AbhGNPxW (ORCPT ); Wed, 14 Jul 2021 11:53:22 -0400 Received: from mail.kernel.org ([198.145.29.99]:41174 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239755AbhGNPxW (ORCPT ); Wed, 14 Jul 2021 11:53:22 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 4BF596128D for ; Wed, 14 Jul 2021 15:50:30 +0000 (UTC) Subject: [PATCH RFC 3/4] SUNRPC: Refactor rpc_ping() From: Chuck Lever To: linux-nfs@vger.kernel.org Date: Wed, 14 Jul 2021 11:50:29 -0400 Message-ID: <162627782960.1294.3463877806680579818.stgit@manet.1015granger.net> In-Reply-To: <162627611661.1294.9189768423517916152.stgit@manet.1015granger.net> References: <162627611661.1294.9189768423517916152.stgit@manet.1015granger.net> User-Agent: StGit/1.1 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Make it use the rpc_null_call_helper() so that it can share the new rpc_call_ops structure to be introduced in the next patch. Signed-off-by: Chuck Lever --- net/sunrpc/clnt.c | 24 +++++++++++++----------- 1 file changed, 13 insertions(+), 11 deletions(-) diff --git a/net/sunrpc/clnt.c b/net/sunrpc/clnt.c index 8b4de70e8ead..ca2000d8cf64 100644 --- a/net/sunrpc/clnt.c +++ b/net/sunrpc/clnt.c @@ -2694,17 +2694,6 @@ static const struct rpc_procinfo rpcproc_null = { .p_decode = rpcproc_decode_null, }; -static int rpc_ping(struct rpc_clnt *clnt) -{ - struct rpc_message msg = { - .rpc_proc = &rpcproc_null, - }; - int err; - err = rpc_call_sync(clnt, &msg, RPC_TASK_SOFT | RPC_TASK_SOFTCONN | - RPC_TASK_NULLCREDS); - return err; -} - static struct rpc_task *rpc_call_null_helper(struct rpc_clnt *clnt, struct rpc_xprt *xprt, struct rpc_cred *cred, int flags, @@ -2733,6 +2722,19 @@ struct rpc_task *rpc_call_null(struct rpc_clnt *clnt, struct rpc_cred *cred, int } EXPORT_SYMBOL_GPL(rpc_call_null); +static int rpc_ping(struct rpc_clnt *clnt) +{ + struct rpc_task *task; + int status; + + task = rpc_call_null_helper(clnt, NULL, NULL, 0, NULL, NULL); + if (IS_ERR(task)) + return PTR_ERR(task); + status = task->tk_status; + rpc_put_task(task); + return status; +} + struct rpc_cb_add_xprt_calldata { struct rpc_xprt_switch *xps; struct rpc_xprt *xprt; From patchwork Wed Jul 14 15:50:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever III X-Patchwork-Id: 12377389 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D8699C07E9C for ; Wed, 14 Jul 2021 15:50:37 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C0F47613C8 for ; Wed, 14 Jul 2021 15:50:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239652AbhGNPx2 (ORCPT ); Wed, 14 Jul 2021 11:53:28 -0400 Received: from mail.kernel.org ([198.145.29.99]:41212 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239554AbhGNPx2 (ORCPT ); Wed, 14 Jul 2021 11:53:28 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 45662613C5 for ; Wed, 14 Jul 2021 15:50:36 +0000 (UTC) Subject: [PATCH RFC 4/4] SUNRPC: Unset RPC_TASK_NO_RETRANS_TIMEOUT for NULL RPCs From: Chuck Lever To: linux-nfs@vger.kernel.org Date: Wed, 14 Jul 2021 11:50:35 -0400 Message-ID: <162627783556.1294.4875890480589342085.stgit@manet.1015granger.net> In-Reply-To: <162627611661.1294.9189768423517916152.stgit@manet.1015granger.net> References: <162627611661.1294.9189768423517916152.stgit@manet.1015granger.net> User-Agent: StGit/1.1 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org In some rare failure modes, the server is actually reading the transport, but then just dropping the requests on the floor. TCP_USER_TIMEOUT cannot detect that case. Prevent such a stuck server from pinning client resources indefinitely by ensuring that certain idempotent requests (such as NULL) can time out even if the connection is still operational. Otherwise rpc_bind_new_program(), gss_destroy_cred(), or rpc_clnt_test_and_add_xprt() can wait forever. Signed-off-by: Chuck Lever --- net/sunrpc/clnt.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/net/sunrpc/clnt.c b/net/sunrpc/clnt.c index ca2000d8cf64..d34737a8a68a 100644 --- a/net/sunrpc/clnt.c +++ b/net/sunrpc/clnt.c @@ -2694,6 +2694,18 @@ static const struct rpc_procinfo rpcproc_null = { .p_decode = rpcproc_decode_null, }; +static void +rpc_null_call_prepare(struct rpc_task *task, void *data) +{ + task->tk_flags &= ~RPC_TASK_NO_RETRANS_TIMEOUT; + rpc_call_start(task); +} + +static const struct rpc_call_ops rpc_null_ops = { + .rpc_call_prepare = rpc_null_call_prepare, + .rpc_call_done = rpc_default_callback, +}; + static struct rpc_task *rpc_call_null_helper(struct rpc_clnt *clnt, struct rpc_xprt *xprt, struct rpc_cred *cred, int flags, @@ -2707,7 +2719,7 @@ struct rpc_task *rpc_call_null_helper(struct rpc_clnt *clnt, .rpc_xprt = xprt, .rpc_message = &msg, .rpc_op_cred = cred, - .callback_ops = (ops != NULL) ? ops : &rpc_default_ops, + .callback_ops = ops ?: &rpc_null_ops, .callback_data = data, .flags = flags | RPC_TASK_SOFT | RPC_TASK_SOFTCONN | RPC_TASK_NULLCREDS, @@ -2758,6 +2770,7 @@ static void rpc_cb_add_xprt_release(void *calldata) } static const struct rpc_call_ops rpc_cb_add_xprt_call_ops = { + .rpc_call_prepare = rpc_null_call_prepare, .rpc_call_done = rpc_cb_add_xprt_done, .rpc_release = rpc_cb_add_xprt_release, };