From patchwork Wed Feb 23 18:16:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dai Ngo X-Patchwork-Id: 12757278 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8F77AC4332F for ; Wed, 23 Feb 2022 18:17:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240419AbiBWSRP (ORCPT ); Wed, 23 Feb 2022 13:17:15 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47030 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241030AbiBWSRM (ORCPT ); Wed, 23 Feb 2022 13:17:12 -0500 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 79BC33DDC3 for ; Wed, 23 Feb 2022 10:16:43 -0800 (PST) Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 21NIDlRM029467; Wed, 23 Feb 2022 18:16:42 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2021-07-09; bh=dBINkHfPxePT7Cc/+uhijsyCoLPPtPWdjiqvHnwveTw=; b=WBEZolCzaBdBdCSvCfvwVAChY8NM9M8DII9oYBSeIo+DvyaLHTukmWqdgHOCJJ8Im1PW wqrsuw1YX756mZDE7FgISQlz5Oxa00WxMpz7CqejmYLD7MTlEdnu0h+aOlPf+S78JkjA /Gxr4BWntnl2e2ex1CJbX6G0jCZpyXmxIMrNSziYMKm1sUjFix56NcSKijJo/uvHBo8X q7u54r8TwOa+bL2AGuNAsAVPU5opOZZ7os5IKwI9NTln0U1K4o+DWUNbpdbRkkTHsRsA APEG06vdYpvpz5e8/00B9WEG1sbaIPwswj7BjTcfatCi4nTrf0fxAAFHFrBJntHk9dcb /g== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by mx0b-00069f02.pphosted.com with ESMTP id 3ectsx54un-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 23 Feb 2022 18:16:42 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 21NIAtpE155960; Wed, 23 Feb 2022 18:16:41 GMT Received: from pps.reinject (localhost [127.0.0.1]) by aserp3030.oracle.com with ESMTP id 3eapkj17pw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 23 Feb 2022 18:16:41 +0000 Received: from aserp3030.oracle.com (aserp3030.oracle.com [127.0.0.1]) by pps.reinject (8.16.0.36/8.16.0.36) with SMTP id 21NICxfS162308; Wed, 23 Feb 2022 18:16:41 GMT Received: from ca-common-hq.us.oracle.com (ca-common-hq.us.oracle.com [10.211.9.209]) by aserp3030.oracle.com with ESMTP id 3eapkj17pe-2; Wed, 23 Feb 2022 18:16:41 +0000 From: Dai Ngo To: chuck.lever@oracle.com, bfields@fieldses.org Cc: linux-nfs@vger.kernel.org Subject: [PATCH RFC v14 01/10] NFSD: Add client flags, macro and spinlock to support courteous server Date: Wed, 23 Feb 2022 10:16:28 -0800 Message-Id: <1645640197-1725-2-git-send-email-dai.ngo@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1645640197-1725-1-git-send-email-dai.ngo@oracle.com> References: <1645640197-1725-1-git-send-email-dai.ngo@oracle.com> X-Proofpoint-ORIG-GUID: Ne6D51DM3LSPqKIN-ED6OE2h_YlHZC2q X-Proofpoint-GUID: Ne6D51DM3LSPqKIN-ED6OE2h_YlHZC2q Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Add flags to track a client's lease state after the client fails to renew its lease. Add macro for courtesy client expire time. Add spinlock to synchronize access to these flags. Modify alloc_client to initialize these fields. Signed-off-by: Dai Ngo --- fs/nfsd/nfs4state.c | 2 ++ fs/nfsd/nfsd.h | 1 + fs/nfsd/state.h | 6 ++++++ 3 files changed, 9 insertions(+) diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c index f3b71fd1d134..6a10f089ef4c 100644 --- a/fs/nfsd/nfs4state.c +++ b/fs/nfsd/nfs4state.c @@ -2009,12 +2009,14 @@ static struct nfs4_client *alloc_client(struct xdr_netobj name) INIT_LIST_HEAD(&clp->cl_delegations); INIT_LIST_HEAD(&clp->cl_lru); INIT_LIST_HEAD(&clp->cl_revoked); + INIT_LIST_HEAD(&clp->cl_cs_list); #ifdef CONFIG_NFSD_PNFS INIT_LIST_HEAD(&clp->cl_lo_states); #endif INIT_LIST_HEAD(&clp->async_copies); spin_lock_init(&clp->async_lock); spin_lock_init(&clp->cl_lock); + spin_lock_init(&clp->cl_cs_lock); rpc_init_wait_queue(&clp->cl_cb_waitq, "Backchannel slot table"); return clp; err_no_hashtbl: diff --git a/fs/nfsd/nfsd.h b/fs/nfsd/nfsd.h index 4fc1fd639527..23996c6ca75e 100644 --- a/fs/nfsd/nfsd.h +++ b/fs/nfsd/nfsd.h @@ -336,6 +336,7 @@ void nfsd_lockd_shutdown(void); #define COMPOUND_ERR_SLACK_SPACE 16 /* OP_SETATTR */ #define NFSD_LAUNDROMAT_MINTIMEOUT 1 /* seconds */ +#define NFSD_COURTESY_CLIENT_TIMEOUT (24 * 60 * 60) /* seconds */ /* * The following attributes are currently not supported by the NFSv4 server: diff --git a/fs/nfsd/state.h b/fs/nfsd/state.h index 95457cfd37fc..8cbd3b69a584 100644 --- a/fs/nfsd/state.h +++ b/fs/nfsd/state.h @@ -343,6 +343,9 @@ struct nfs4_client { #define NFSD4_CLIENT_RECLAIM_COMPLETE (3) /* reclaim_complete done */ #define NFSD4_CLIENT_CONFIRMED (4) /* client is confirmed */ #define NFSD4_CLIENT_UPCALL_LOCK (5) /* upcall serialization */ +#define NFSD4_CLIENT_COURTESY (6) +#define NFSD4_CLIENT_EXPIRED (7) +#define NFSD4_CLIENT_RECONNECTED (8) #define NFSD4_CLIENT_CB_FLAG_MASK (1 << NFSD4_CLIENT_CB_UPDATE | \ 1 << NFSD4_CLIENT_CB_KILL) unsigned long cl_flags; @@ -385,6 +388,9 @@ struct nfs4_client { struct list_head async_copies; /* list of async copies */ spinlock_t async_lock; /* lock for async copies */ atomic_t cl_cb_inflight; /* Outstanding callbacks */ + + spinlock_t cl_cs_lock; + struct list_head cl_cs_list; }; /* struct nfs4_client_reset From patchwork Wed Feb 23 18:16:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dai Ngo X-Patchwork-Id: 12757279 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9FE58C43217 for ; Wed, 23 Feb 2022 18:17:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239929AbiBWSRO (ORCPT ); Wed, 23 Feb 2022 13:17:14 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47034 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234558AbiBWSRM (ORCPT ); Wed, 23 Feb 2022 13:17:12 -0500 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9BCA03EA91 for ; Wed, 23 Feb 2022 10:16:44 -0800 (PST) Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 21NIDlRN029467; Wed, 23 Feb 2022 18:16:42 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2021-07-09; bh=emwy3ls4nmCZTvvZwxTNW088hod0vShq2jaIBqTbY9Q=; b=WypIMy193eZOWJ2E+FBtXACClg4xRg3KeTE7H+LwBk3cltRSpmqsoiA0afDw88CKdV8z R5IPZdpxxD606Rab2z8rMG1m6/k1NML/LsZBmwXI/U1d7Q0OWlStnN7ttiKGnc4PaD1l varlgVKEc5G0OapSDZ2KMNdSv2f7ia5Tpi/HEJhgeM9t8mecKQo8bwf+W96NyAJtYP0w WfM1moKrVU88pOQdwU4CpqO71SZyXM64MwCV0JCAuvdB8/QOwxnRwc30tjMHjGAUcpQW qwI5jz6kRP2ImT+dTchU3WU6WUdBLpIxIXPIts6Whw4+tvVI3KxoABPYlNiWsUYyewt7 rw== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by mx0b-00069f02.pphosted.com with ESMTP id 3ectsx54up-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 23 Feb 2022 18:16:42 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 21NIAt8b156010; Wed, 23 Feb 2022 18:16:42 GMT Received: from pps.reinject (localhost [127.0.0.1]) by aserp3030.oracle.com with ESMTP id 3eapkj17q6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 23 Feb 2022 18:16:42 +0000 Received: from aserp3030.oracle.com (aserp3030.oracle.com [127.0.0.1]) by pps.reinject (8.16.0.36/8.16.0.36) with SMTP id 21NICxfU162308; Wed, 23 Feb 2022 18:16:41 GMT Received: from ca-common-hq.us.oracle.com (ca-common-hq.us.oracle.com [10.211.9.209]) by aserp3030.oracle.com with ESMTP id 3eapkj17pe-3; Wed, 23 Feb 2022 18:16:41 +0000 From: Dai Ngo To: chuck.lever@oracle.com, bfields@fieldses.org Cc: linux-nfs@vger.kernel.org Subject: [PATCH RFC v14 02/10] NFSD: Add lm_lock_expired call out Date: Wed, 23 Feb 2022 10:16:29 -0800 Message-Id: <1645640197-1725-3-git-send-email-dai.ngo@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1645640197-1725-1-git-send-email-dai.ngo@oracle.com> References: <1645640197-1725-1-git-send-email-dai.ngo@oracle.com> X-Proofpoint-ORIG-GUID: acH3gKzkfBX0X_vVQ0uKsvmBfB0eJMc- X-Proofpoint-GUID: acH3gKzkfBX0X_vVQ0uKsvmBfB0eJMc- Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Add callout function nfsd4_lm_lock_expired for lm_lock_expired. If lock request has conflict with courtesy client then expire the courtesy client and return no conflict to caller. Signed-off-by: Dai Ngo --- fs/nfsd/nfs4state.c | 37 +++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c index 6a10f089ef4c..6bca727978ea 100644 --- a/fs/nfsd/nfs4state.c +++ b/fs/nfsd/nfs4state.c @@ -6578,10 +6578,47 @@ nfsd4_lm_notify(struct file_lock *fl) } } +/** + * nfsd4_lm_lock_expired - check if lock conflict can be resolved. + * + * @fl: pointer to file_lock with a potential conflict + * Return values: + * %false: real conflict, lock conflict can not be resolved. + * %true: no conflict, lock conflict was resolved. + * + * Note that this function is called while the flc_lock is held. + */ +static bool +nfsd4_lm_lock_expired(struct file_lock *fl) +{ + struct nfs4_lockowner *lo; + struct nfs4_client *clp; + bool rc = false; + + if (!fl) + return false; + lo = (struct nfs4_lockowner *)fl->fl_owner; + clp = lo->lo_owner.so_client; + + /* need to sync with courtesy client trying to reconnect */ + spin_lock(&clp->cl_cs_lock); + if (test_bit(NFSD4_CLIENT_EXPIRED, &clp->cl_flags)) + rc = true; + else { + if (test_bit(NFSD4_CLIENT_COURTESY, &clp->cl_flags)) { + set_bit(NFSD4_CLIENT_EXPIRED, &clp->cl_flags); + rc = true; + } + } + spin_unlock(&clp->cl_cs_lock); + return rc; +} + static const struct lock_manager_operations nfsd_posix_mng_ops = { .lm_notify = nfsd4_lm_notify, .lm_get_owner = nfsd4_fl_get_owner, .lm_put_owner = nfsd4_fl_put_owner, + .lm_lock_expired = nfsd4_lm_lock_expired, }; static inline void From patchwork Wed Feb 23 18:16:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dai Ngo X-Patchwork-Id: 12757276 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 66367C433EF for ; Wed, 23 Feb 2022 18:17:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234558AbiBWSRO (ORCPT ); Wed, 23 Feb 2022 13:17:14 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47036 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240419AbiBWSRM (ORCPT ); Wed, 23 Feb 2022 13:17:12 -0500 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D49B03EAA3 for ; Wed, 23 Feb 2022 10:16:44 -0800 (PST) Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 21NIDrF5001879; Wed, 23 Feb 2022 18:16:43 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2021-07-09; bh=SmAUODvkqaf9mGb+3zygEAH0N35O0ouxz14SUhCVduc=; b=LBFf2jT5xkJ7eXlh9/tNdiaTKCDwksOxFPbZD76Ah1UDkYFB3nH7wQf5BC74+4Jz8kkp ARHb13fA/QC8ZzReYK7uJk3z3dMJKdGnyvXNcFDoiDrNLIgduivztJD63cNDx6wJZ6hL c+I0TsG5jmRUjmSg0Ko3MzUCDbCsLMUz4Thl1/JnwWHU561a1TsJhDDTY7RAaaIoEO57 CCgcAfyw9bdX9Ce0Knb/TvrMhF8w9lU22sFMsKdhPf7es9+EI/vsA+8le9Gr9HUeCU9A P7e9zQTjPf8i5uSvc8iCjctr874RRIQr5ZLBXHs/Mz5lH5uWZlHcdyclB0dPkd9581q4 uw== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by mx0b-00069f02.pphosted.com with ESMTP id 3ecv6evxpm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 23 Feb 2022 18:16:43 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 21NIAtuJ155985; Wed, 23 Feb 2022 18:16:42 GMT Received: from pps.reinject (localhost [127.0.0.1]) by aserp3030.oracle.com with ESMTP id 3eapkj17qn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 23 Feb 2022 18:16:42 +0000 Received: from aserp3030.oracle.com (aserp3030.oracle.com [127.0.0.1]) by pps.reinject (8.16.0.36/8.16.0.36) with SMTP id 21NICxfW162308; Wed, 23 Feb 2022 18:16:42 GMT Received: from ca-common-hq.us.oracle.com (ca-common-hq.us.oracle.com [10.211.9.209]) by aserp3030.oracle.com with ESMTP id 3eapkj17pe-4; Wed, 23 Feb 2022 18:16:42 +0000 From: Dai Ngo To: chuck.lever@oracle.com, bfields@fieldses.org Cc: linux-nfs@vger.kernel.org Subject: [PATCH RFC v14 03/10] NFSD: Update nfsd_breaker_owns_lease() to handle courtesy clients Date: Wed, 23 Feb 2022 10:16:30 -0800 Message-Id: <1645640197-1725-4-git-send-email-dai.ngo@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1645640197-1725-1-git-send-email-dai.ngo@oracle.com> References: <1645640197-1725-1-git-send-email-dai.ngo@oracle.com> X-Proofpoint-GUID: 2KVumqflwnuq-0hl45taFrSwSNoH1TDc X-Proofpoint-ORIG-GUID: 2KVumqflwnuq-0hl45taFrSwSNoH1TDc Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Update nfsd_breaker_owns_lease() to handle delegation conflict with courtesy clients. If conflict was caused courtesy client then discard the courtesy client by setting CLIENT_EXPIRED and return conflict resolved. Client with CLIENT_EXPIRED is expired by the laundromat. Signed-off-by: Dai Ngo --- fs/nfsd/nfs4state.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c index 6bca727978ea..542a13676c91 100644 --- a/fs/nfsd/nfs4state.c +++ b/fs/nfsd/nfs4state.c @@ -4727,6 +4727,24 @@ static bool nfsd_breaker_owns_lease(struct file_lock *fl) struct svc_rqst *rqst; struct nfs4_client *clp; + clp = dl->dl_stid.sc_client; + /* + * need to sync with courtesy client trying to reconnect using + * the cl_cs_lock, nn->client_lock can not be used since this + * function is called with the fl_lck held. + */ + spin_lock(&clp->cl_cs_lock); + if (test_bit(NFSD4_CLIENT_EXPIRED, &clp->cl_flags)) { + spin_unlock(&clp->cl_cs_lock); + return true; + } + if (test_bit(NFSD4_CLIENT_COURTESY, &clp->cl_flags)) { + set_bit(NFSD4_CLIENT_EXPIRED, &clp->cl_flags); + spin_unlock(&clp->cl_cs_lock); + return true; + } + spin_unlock(&clp->cl_cs_lock); + if (!i_am_nfsd()) return false; rqst = kthread_data(current); From patchwork Wed Feb 23 18:16:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dai Ngo X-Patchwork-Id: 12757277 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 37A1EC433F5 for ; Wed, 23 Feb 2022 18:17:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243800AbiBWSRP (ORCPT ); Wed, 23 Feb 2022 13:17:15 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47092 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241817AbiBWSRN (ORCPT ); Wed, 23 Feb 2022 13:17:13 -0500 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1BE5A3ED0A for ; Wed, 23 Feb 2022 10:16:45 -0800 (PST) Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 21NIDlCw029459; Wed, 23 Feb 2022 18:16:43 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2021-07-09; bh=B7qqXJZj8d1n11UsihJzgskiFJcdarMdYoIty2E3O4s=; b=nogTBwUriw/au9/UzGR90eoxxiLkxQYxLs2zLU+ByGJWBzyYg3sk8GjzPo1luou0wsFC m79lumE8WkQFMNH0iU6a6DquirSOE8HC9Kc8tCWkFp3obCY1FUY9vHp2tkDWjg/jzsV6 UHJQyDf1QznLPuxcX4d3NoMBJ2p8QaunlafsiQ4L2ALQKTjvWpoXQ01qclqzssuKQYm1 3Pk7/UqrlYKJatLxdNKAL5TNCcEWHAQWTNUytta2zL7KfaKvZCpHf5sHg5v0gZFIBFdT I2XHAAzCR96rPTg/LSBzkuAK6HKzMrArZsGZg/03taG1vED7d4pIDirKt7BfGi/kdJ2t wg== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by mx0b-00069f02.pphosted.com with ESMTP id 3ectsx54ur-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 23 Feb 2022 18:16:43 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 21NIAuvl156041; Wed, 23 Feb 2022 18:16:43 GMT Received: from pps.reinject (localhost [127.0.0.1]) by aserp3030.oracle.com with ESMTP id 3eapkj17r4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 23 Feb 2022 18:16:43 +0000 Received: from aserp3030.oracle.com (aserp3030.oracle.com [127.0.0.1]) by pps.reinject (8.16.0.36/8.16.0.36) with SMTP id 21NICxfY162308; Wed, 23 Feb 2022 18:16:42 GMT Received: from ca-common-hq.us.oracle.com (ca-common-hq.us.oracle.com [10.211.9.209]) by aserp3030.oracle.com with ESMTP id 3eapkj17pe-5; Wed, 23 Feb 2022 18:16:42 +0000 From: Dai Ngo To: chuck.lever@oracle.com, bfields@fieldses.org Cc: linux-nfs@vger.kernel.org Subject: [PATCH RFC v14 04/10] NFSD: Update nfs4_get_vfs_file() to handle courtesy clients Date: Wed, 23 Feb 2022 10:16:31 -0800 Message-Id: <1645640197-1725-5-git-send-email-dai.ngo@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1645640197-1725-1-git-send-email-dai.ngo@oracle.com> References: <1645640197-1725-1-git-send-email-dai.ngo@oracle.com> X-Proofpoint-ORIG-GUID: xTbmhQ0UxCNlojC6XXzNTAv3GlU4lW7h X-Proofpoint-GUID: xTbmhQ0UxCNlojC6XXzNTAv3GlU4lW7h Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Update nfs4_get_vfs_file() to handle share reservation conflict with courtesy client. If share/access check fails with share denied then check if the conflict was caused by courtesy clients. If that's the case then set CLIENT_EXPIRED flag to expire the courtesy clients and allow nfs4_get_vfs_file to continue. Client with CLIENT_EXPIRED is expired by the laundromat. Signed-off-by: Dai Ngo --- fs/nfsd/nfs4state.c | 106 ++++++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 99 insertions(+), 7 deletions(-) diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c index 542a13676c91..1ffe7bafe90b 100644 --- a/fs/nfsd/nfs4state.c +++ b/fs/nfsd/nfs4state.c @@ -4965,9 +4965,87 @@ nfsd4_truncate(struct svc_rqst *rqstp, struct svc_fh *fh, return nfsd_setattr(rqstp, fh, &iattr, 0, (time64_t)0); } +static bool +nfs4_check_access_deny_bmap(struct nfs4_ol_stateid *stp, u32 access, + bool share_access) +{ + if (share_access) { + if (!stp->st_deny_bmap) + return false; + + if ((stp->st_deny_bmap & (1 << NFS4_SHARE_DENY_BOTH)) || + (access & NFS4_SHARE_ACCESS_READ && + stp->st_deny_bmap & (1 << NFS4_SHARE_DENY_READ)) || + (access & NFS4_SHARE_ACCESS_WRITE && + stp->st_deny_bmap & (1 << NFS4_SHARE_DENY_WRITE))) { + return true; + } + return false; + } + if ((access & NFS4_SHARE_DENY_BOTH) || + (access & NFS4_SHARE_DENY_READ && + stp->st_access_bmap & (1 << NFS4_SHARE_ACCESS_READ)) || + (access & NFS4_SHARE_DENY_WRITE && + stp->st_access_bmap & (1 << NFS4_SHARE_ACCESS_WRITE))) { + return true; + } + return false; +} + +/* + * Check whether nfserr_share_denied should be returned. + * + * access: is op_share_access if share_access is true. + * Check if access mode, op_share_access, would conflict with + * the current deny mode of the file 'fp'. + * access: is op_share_deny if share_access is false. + * Check if the deny mode, op_share_deny, would conflict with + * current access of the file 'fp'. + * stp: skip checking this entry. + * new_stp: normal open, not open upgrade. + * + * Function returns: + * true - access/deny mode conflict with normal client. + * false - no conflict or conflict with courtesy client(s) is resolved. + */ +static bool +nfs4_conflict_clients(struct nfs4_file *fp, bool new_stp, + struct nfs4_ol_stateid *stp, u32 access, bool share_access) +{ + struct nfs4_ol_stateid *st; + struct nfs4_client *cl; + bool conflict = false; + + lockdep_assert_held(&fp->fi_lock); + list_for_each_entry(st, &fp->fi_stateids, st_perfile) { + if (st->st_openstp || (st == stp && new_stp) || + (!nfs4_check_access_deny_bmap(st, + access, share_access))) + continue; + + /* need to sync with courtesy client trying to reconnect */ + cl = st->st_stid.sc_client; + spin_lock(&cl->cl_cs_lock); + if (test_bit(NFSD4_CLIENT_EXPIRED, &cl->cl_flags)) { + spin_unlock(&cl->cl_cs_lock); + continue; + } + if (test_bit(NFSD4_CLIENT_COURTESY, &cl->cl_flags)) { + set_bit(NFSD4_CLIENT_EXPIRED, &cl->cl_flags); + spin_unlock(&cl->cl_cs_lock); + continue; + } + /* conflict not caused by courtesy client */ + spin_unlock(&cl->cl_cs_lock); + conflict = true; + break; + } + return conflict; +} + static __be32 nfs4_get_vfs_file(struct svc_rqst *rqstp, struct nfs4_file *fp, struct svc_fh *cur_fh, struct nfs4_ol_stateid *stp, - struct nfsd4_open *open) + struct nfsd4_open *open, bool new_stp) { struct nfsd_file *nf = NULL; __be32 status; @@ -4983,15 +5061,29 @@ static __be32 nfs4_get_vfs_file(struct svc_rqst *rqstp, struct nfs4_file *fp, */ status = nfs4_file_check_deny(fp, open->op_share_deny); if (status != nfs_ok) { - spin_unlock(&fp->fi_lock); - goto out; + if (status != nfserr_share_denied) { + spin_unlock(&fp->fi_lock); + goto out; + } + if (nfs4_conflict_clients(fp, new_stp, stp, + open->op_share_deny, false)) { + spin_unlock(&fp->fi_lock); + goto out; + } } /* set access to the file */ status = nfs4_file_get_access(fp, open->op_share_access); if (status != nfs_ok) { - spin_unlock(&fp->fi_lock); - goto out; + if (status != nfserr_share_denied) { + spin_unlock(&fp->fi_lock); + goto out; + } + if (nfs4_conflict_clients(fp, new_stp, stp, + open->op_share_access, true)) { + spin_unlock(&fp->fi_lock); + goto out; + } } /* Set access bits in stateid */ @@ -5042,7 +5134,7 @@ nfs4_upgrade_open(struct svc_rqst *rqstp, struct nfs4_file *fp, struct svc_fh *c unsigned char old_deny_bmap = stp->st_deny_bmap; if (!test_access(open->op_share_access, stp)) - return nfs4_get_vfs_file(rqstp, fp, cur_fh, stp, open); + return nfs4_get_vfs_file(rqstp, fp, cur_fh, stp, open, false); /* test and set deny mode */ spin_lock(&fp->fi_lock); @@ -5391,7 +5483,7 @@ nfsd4_process_open2(struct svc_rqst *rqstp, struct svc_fh *current_fh, struct nf goto out; } } else { - status = nfs4_get_vfs_file(rqstp, fp, current_fh, stp, open); + status = nfs4_get_vfs_file(rqstp, fp, current_fh, stp, open, true); if (status) { stp->st_stid.sc_type = NFS4_CLOSED_STID; release_open_stateid(stp); From patchwork Wed Feb 23 18:16:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dai Ngo X-Patchwork-Id: 12757280 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7E218C433FE for ; Wed, 23 Feb 2022 18:17:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241817AbiBWSRQ (ORCPT ); Wed, 23 Feb 2022 13:17:16 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47142 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242155AbiBWSRN (ORCPT ); Wed, 23 Feb 2022 13:17:13 -0500 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AF1783ED1F for ; Wed, 23 Feb 2022 10:16:45 -0800 (PST) Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 21NIDlrE029457; Wed, 23 Feb 2022 18:16:44 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2021-07-09; bh=1lXlqEw7TWDMwYFONEIfqtyIi2CZ8jjHv/6Uo2EWCow=; b=pxY2hA07eI9nhml5Nq8Ag7b2ev0d/+ueE/fAYde3oCCphmNdHCo+QsB2Z58uK1zeOPia V5CcbNitEklPGem5fXA5L4xjnoLPj2Jtf8lNn5PdjIbFz5xFQR6BgEsuMXes8IZ5S0ww ecLE2HioZBp9p9/XB7cgzK1Q3l8sr4jQYi6Bhp2pm8JcphwGugdc8YBlfTqSf2ShWPAr uyGnKe+yTJNQePjSbj0vI+afvVUMLprZX90cOR1bjN9xCSsT7pnwLrGjwPSUV2H9Z0s2 3HecaeeK8yoFx4N0PgX3qfHwz6pXsMPA/I5z1VSah9BvSbT4b34ZyIglIWukqKi00NwS hw== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by mx0b-00069f02.pphosted.com with ESMTP id 3ectsx54ut-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 23 Feb 2022 18:16:44 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 21NIAtHj155964; Wed, 23 Feb 2022 18:16:43 GMT Received: from pps.reinject (localhost [127.0.0.1]) by aserp3030.oracle.com with ESMTP id 3eapkj17rh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 23 Feb 2022 18:16:43 +0000 Received: from aserp3030.oracle.com (aserp3030.oracle.com [127.0.0.1]) by pps.reinject (8.16.0.36/8.16.0.36) with SMTP id 21NICxfa162308; Wed, 23 Feb 2022 18:16:43 GMT Received: from ca-common-hq.us.oracle.com (ca-common-hq.us.oracle.com [10.211.9.209]) by aserp3030.oracle.com with ESMTP id 3eapkj17pe-6; Wed, 23 Feb 2022 18:16:43 +0000 From: Dai Ngo To: chuck.lever@oracle.com, bfields@fieldses.org Cc: linux-nfs@vger.kernel.org Subject: [PATCH RFC v14 05/10] NFSD: Update find_clp_in_name_tree() to handle courtesy clients Date: Wed, 23 Feb 2022 10:16:32 -0800 Message-Id: <1645640197-1725-6-git-send-email-dai.ngo@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1645640197-1725-1-git-send-email-dai.ngo@oracle.com> References: <1645640197-1725-1-git-send-email-dai.ngo@oracle.com> X-Proofpoint-ORIG-GUID: zJxmJc4g9umXBGl5C5AywxC41ZFAyeis X-Proofpoint-GUID: zJxmJc4g9umXBGl5C5AywxC41ZFAyeis Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Update find_clp_in_name_tree: . skip client with CLIENT_EXPIRED flag; discarded courtesy client. . if courtesy client was found then clear CLIENT_COURTESY and set CLIENT_RECONNECTED so callers can take appropriate action. Update find_confirmed_client_by_name to discard the courtesy client; set CLIENT_EXPIRED. Update nfsd4_setclientid to expire the confirmed courtesy client to prevent multiple confirmed clients with the same name on the the conf_id_hashtbl list. Signed-off-by: Dai Ngo --- fs/nfsd/nfs4state.c | 45 ++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 42 insertions(+), 3 deletions(-) diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c index 1ffe7bafe90b..4990553180f8 100644 --- a/fs/nfsd/nfs4state.c +++ b/fs/nfsd/nfs4state.c @@ -2834,8 +2834,21 @@ find_clp_in_name_tree(struct xdr_netobj *name, struct rb_root *root) node = node->rb_left; else if (cmp < 0) node = node->rb_right; - else + else { + clear_bit(NFSD4_CLIENT_RECONNECTED, &clp->cl_flags); + /* sync with thread resolving lock/deleg conflict */ + spin_lock(&clp->cl_cs_lock); + if (test_bit(NFSD4_CLIENT_EXPIRED, &clp->cl_flags)) { + spin_unlock(&clp->cl_cs_lock); + return NULL; + } + if (test_bit(NFSD4_CLIENT_COURTESY, &clp->cl_flags)) { + clear_bit(NFSD4_CLIENT_COURTESY, &clp->cl_flags); + set_bit(NFSD4_CLIENT_RECONNECTED, &clp->cl_flags); + } + spin_unlock(&clp->cl_cs_lock); return clp; + } } return NULL; } @@ -2888,6 +2901,14 @@ find_client_in_id_table(struct list_head *tbl, clientid_t *clid, bool sessions) return NULL; } +static void +nfsd4_discard_courtesy_clnt(struct nfs4_client *clp) +{ + spin_lock(&clp->cl_cs_lock); + set_bit(NFSD4_CLIENT_EXPIRED, &clp->cl_flags); + spin_unlock(&clp->cl_cs_lock); +} + static struct nfs4_client * find_confirmed_client(clientid_t *clid, bool sessions, struct nfsd_net *nn) { @@ -2914,8 +2935,15 @@ static bool clp_used_exchangeid(struct nfs4_client *clp) static struct nfs4_client * find_confirmed_client_by_name(struct xdr_netobj *name, struct nfsd_net *nn) { + struct nfs4_client *clp; + lockdep_assert_held(&nn->client_lock); - return find_clp_in_name_tree(name, &nn->conf_name_tree); + clp = find_clp_in_name_tree(name, &nn->conf_name_tree); + if (clp && test_bit(NFSD4_CLIENT_RECONNECTED, &clp->cl_flags)) { + nfsd4_discard_courtesy_clnt(clp); + clp = NULL; + } + return clp; } static struct nfs4_client * @@ -4032,12 +4060,19 @@ nfsd4_setclientid(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, struct nfs4_client *unconf = NULL; __be32 status; struct nfsd_net *nn = net_generic(SVC_NET(rqstp), nfsd_net_id); + struct nfs4_client *cclient = NULL; new = create_client(clname, rqstp, &clverifier); if (new == NULL) return nfserr_jukebox; spin_lock(&nn->client_lock); - conf = find_confirmed_client_by_name(&clname, nn); + /* find confirmed client by name */ + conf = find_clp_in_name_tree(&clname, &nn->conf_name_tree); + if (conf && test_bit(NFSD4_CLIENT_RECONNECTED, &conf->cl_flags)) { + cclient = conf; + conf = NULL; + } + if (conf && client_has_state(conf)) { status = nfserr_clid_inuse; if (clp_used_exchangeid(conf)) @@ -4068,7 +4103,11 @@ nfsd4_setclientid(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, new = NULL; status = nfs_ok; out: + if (cclient) + unhash_client_locked(cclient); spin_unlock(&nn->client_lock); + if (cclient) + expire_client(cclient); if (new) free_client(new); if (unconf) { From patchwork Wed Feb 23 18:16:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dai Ngo X-Patchwork-Id: 12757283 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EB7C9C433EF for ; Wed, 23 Feb 2022 18:18:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242155AbiBWSRR (ORCPT ); Wed, 23 Feb 2022 13:17:17 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47152 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243616AbiBWSRO (ORCPT ); Wed, 23 Feb 2022 13:17:14 -0500 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 07CD83F32A for ; Wed, 23 Feb 2022 10:16:45 -0800 (PST) Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 21NIDlCx029459; Wed, 23 Feb 2022 18:16:45 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2021-07-09; bh=rIJPt+A+ak4PFGluCnj/+PVYjxKV1SP6JJRbkBtsgW4=; b=Kf91I2WfH8YrJCa4TFuROu3hxx0ypf1aSXN+ErDxrUJQ8g5IU8Cgx13sgkB5jascsqfi PLnt9v+LmQu/aK45zw+LhU8yh3xdfAqIQdeiQgGXUAdStLDmBl7tCklNrUKepB/Qz732 K9uFRfNaj8pC/9hcOOAjxXBZr3VmRtWgyBxlxjt/BIRF05Zi9ITmCyYAP8BhgTFZrrHr OT0lZ8Bs7gRHRRqdF7+qf2mRrT7xSEyvEA7bySlKfqY+iv0xRGIYxFZdtjs9ztQy/u8H fMYwxe5fO3i5Vuqrd8rr+ohl9iHM0nrVEfIbG4bX50yvCzt2dSVgAX2wIRkynsK905/n 4Q== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by mx0b-00069f02.pphosted.com with ESMTP id 3ectsx54ux-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 23 Feb 2022 18:16:44 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 21NIAtfh155954; Wed, 23 Feb 2022 18:16:44 GMT Received: from pps.reinject (localhost [127.0.0.1]) by aserp3030.oracle.com with ESMTP id 3eapkj17rw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 23 Feb 2022 18:16:44 +0000 Received: from aserp3030.oracle.com (aserp3030.oracle.com [127.0.0.1]) by pps.reinject (8.16.0.36/8.16.0.36) with SMTP id 21NICxfc162308; Wed, 23 Feb 2022 18:16:43 GMT Received: from ca-common-hq.us.oracle.com (ca-common-hq.us.oracle.com [10.211.9.209]) by aserp3030.oracle.com with ESMTP id 3eapkj17pe-7; Wed, 23 Feb 2022 18:16:43 +0000 From: Dai Ngo To: chuck.lever@oracle.com, bfields@fieldses.org Cc: linux-nfs@vger.kernel.org Subject: [PATCH RFC v14 06/10] NFSD: Update find_in_sessionid_hashtbl() to handle courtesy clients Date: Wed, 23 Feb 2022 10:16:33 -0800 Message-Id: <1645640197-1725-7-git-send-email-dai.ngo@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1645640197-1725-1-git-send-email-dai.ngo@oracle.com> References: <1645640197-1725-1-git-send-email-dai.ngo@oracle.com> X-Proofpoint-ORIG-GUID: -o8b47xDglb3_VyqKRxdXIKO3CJB_YpV X-Proofpoint-GUID: -o8b47xDglb3_VyqKRxdXIKO3CJB_YpV Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Update find_in_sessionid_hashtbl to: . skip client with CLIENT_EXPIRED flag; discarded courtesy client. . if courtesy client was found then clear CLIENT_COURTESY and set CLIENT_RECONNECTED so callers can take appropriate action. Update nfsd4_sequence and nfsd4_bind_conn_to_session to create client record for client with CLIENT_RECONNECTED set. Update nfsd4_destroy_session to discard client with CLIENT_RECONNECTED set. Signed-off-by: Dai Ngo --- fs/nfsd/nfs4state.c | 44 +++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 43 insertions(+), 1 deletion(-) diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c index 4990553180f8..5cef1a78cc28 100644 --- a/fs/nfsd/nfs4state.c +++ b/fs/nfsd/nfs4state.c @@ -1935,10 +1935,31 @@ find_in_sessionid_hashtbl(struct nfs4_sessionid *sessionid, struct net *net, { struct nfsd4_session *session; __be32 status = nfserr_badsession; + struct nfs4_client *clp; session = __find_in_sessionid_hashtbl(sessionid, net); if (!session) goto out; + clp = session->se_client; + if (clp) { + clear_bit(NFSD4_CLIENT_RECONNECTED, &clp->cl_flags); + /* need to sync with thread resolving lock/deleg conflict */ + spin_lock(&clp->cl_cs_lock); + if (test_bit(NFSD4_CLIENT_EXPIRED, &clp->cl_flags)) { + spin_unlock(&clp->cl_cs_lock); + session = NULL; + goto out; + } + if (test_bit(NFSD4_CLIENT_COURTESY, &clp->cl_flags)) { + /* + * clear CLIENT_COURTESY flag to prevent it from being + * destroyed by thread trying to resolve conflicts. + */ + clear_bit(NFSD4_CLIENT_COURTESY, &clp->cl_flags); + set_bit(NFSD4_CLIENT_RECONNECTED, &clp->cl_flags); + } + spin_unlock(&clp->cl_cs_lock); + } status = nfsd4_get_session_locked(session); if (status) session = NULL; @@ -3661,6 +3682,7 @@ __be32 nfsd4_bind_conn_to_session(struct svc_rqst *rqstp, struct nfsd4_session *session; struct net *net = SVC_NET(rqstp); struct nfsd_net *nn = net_generic(net, nfsd_net_id); + struct nfs4_client *clp; if (!nfsd4_last_compound_op(rqstp)) return nfserr_not_only_op; @@ -3693,6 +3715,13 @@ __be32 nfsd4_bind_conn_to_session(struct svc_rqst *rqstp, nfsd4_init_conn(rqstp, conn, session); status = nfs_ok; out: + clp = session->se_client; + if (test_bit(NFSD4_CLIENT_RECONNECTED, &clp->cl_flags)) { + if (status == nfs_ok) + nfsd4_client_record_create(clp); + else + nfsd4_discard_courtesy_clnt(clp); + } nfsd4_put_session(session); out_no_session: return status; @@ -3715,6 +3744,7 @@ nfsd4_destroy_session(struct svc_rqst *r, struct nfsd4_compound_state *cstate, int ref_held_by_me = 0; struct net *net = SVC_NET(r); struct nfsd_net *nn = net_generic(net, nfsd_net_id); + struct nfs4_client *clp; status = nfserr_not_only_op; if (nfsd4_compound_in_session(cstate, sessionid)) { @@ -3727,6 +3757,12 @@ nfsd4_destroy_session(struct svc_rqst *r, struct nfsd4_compound_state *cstate, ses = find_in_sessionid_hashtbl(sessionid, net, &status); if (!ses) goto out_client_lock; + clp = ses->se_client; + if (test_bit(NFSD4_CLIENT_RECONNECTED, &clp->cl_flags)) { + status = nfserr_badsession; + nfsd4_discard_courtesy_clnt(clp); + goto out_put_session; + } status = nfserr_wrong_cred; if (!nfsd4_mach_creds_match(ses->se_client, r)) goto out_put_session; @@ -3831,7 +3867,7 @@ nfsd4_sequence(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, struct nfsd4_compoundres *resp = rqstp->rq_resp; struct xdr_stream *xdr = resp->xdr; struct nfsd4_session *session; - struct nfs4_client *clp; + struct nfs4_client *clp = NULL; struct nfsd4_slot *slot; struct nfsd4_conn *conn; __be32 status; @@ -3941,6 +3977,12 @@ nfsd4_sequence(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate, if (conn) free_conn(conn); spin_unlock(&nn->client_lock); + if (clp && test_bit(NFSD4_CLIENT_RECONNECTED, &clp->cl_flags)) { + if (status == nfs_ok) + nfsd4_client_record_create(clp); + else + nfsd4_discard_courtesy_clnt(clp); + } return status; out_put_session: nfsd4_put_session_locked(session); From patchwork Wed Feb 23 18:16:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dai Ngo X-Patchwork-Id: 12757286 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 274E2C4332F for ; Wed, 23 Feb 2022 18:19:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243616AbiBWSRS (ORCPT ); Wed, 23 Feb 2022 13:17:18 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47224 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243680AbiBWSRO (ORCPT ); Wed, 23 Feb 2022 13:17:14 -0500 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E8E403ED1F for ; Wed, 23 Feb 2022 10:16:46 -0800 (PST) Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 21NIDsSX001902; Wed, 23 Feb 2022 18:16:45 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2021-07-09; bh=syq4AIY1zXSJPAQDPO05MJ3pl/y7Q5deB8DiBTxbbnU=; b=a30C00/b2OLDDnkmMRetOAkqzbB3r71LRDr9BNsy6Oaha3RezADU+LCD/Nhh+a7QUnzV ViOtUQQGFjSl/7P3TM6yrBouNufIt//m6WFkmOY56RZHh5ISS2XmxCyBhVZ8IqF3kgrv iUlKIMniW511+dbrDOU00FZWC9hU+ik3xvDohGUVvUPpufsEN6bDWGMZMhUvAax+YxSy GDYG47kOKc6K57bRrsI+9Mf/SjG+/i7D+Jq5pHmevwIQLbXSg1y2W1inWtLsKFtaPEVf 2L8g/3OH3tCRGYWwGy93Qs2dAKWFom2PWJU6rfQjlLszeFesOjcMRif3uJqNNmMvUFfA yg== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by mx0b-00069f02.pphosted.com with ESMTP id 3ecv6evxpr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 23 Feb 2022 18:16:45 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 21NIAtJm155927; Wed, 23 Feb 2022 18:16:45 GMT Received: from pps.reinject (localhost [127.0.0.1]) by aserp3030.oracle.com with ESMTP id 3eapkj17s8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 23 Feb 2022 18:16:45 +0000 Received: from aserp3030.oracle.com (aserp3030.oracle.com [127.0.0.1]) by pps.reinject (8.16.0.36/8.16.0.36) with SMTP id 21NICxfe162308; Wed, 23 Feb 2022 18:16:44 GMT Received: from ca-common-hq.us.oracle.com (ca-common-hq.us.oracle.com [10.211.9.209]) by aserp3030.oracle.com with ESMTP id 3eapkj17pe-8; Wed, 23 Feb 2022 18:16:44 +0000 From: Dai Ngo To: chuck.lever@oracle.com, bfields@fieldses.org Cc: linux-nfs@vger.kernel.org Subject: [PATCH RFC v14 07/10] NFSD: Update find_client_in_id_table() to handle courtesy clients Date: Wed, 23 Feb 2022 10:16:34 -0800 Message-Id: <1645640197-1725-8-git-send-email-dai.ngo@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1645640197-1725-1-git-send-email-dai.ngo@oracle.com> References: <1645640197-1725-1-git-send-email-dai.ngo@oracle.com> X-Proofpoint-GUID: kgcaJpn_62tG876OhrmWnRUpQNsL4CFc X-Proofpoint-ORIG-GUID: kgcaJpn_62tG876OhrmWnRUpQNsL4CFc Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Update find_client_in_id_table to: . skip client with CLIENT_EXPIRED; discarded courtesy client . if courtesy client was found then clear CLIENT_COURTESY and set CLIENT_RECONNECTED flag so callers can take appropriate action. Update find_confirmed_client to discard courtesy client. Update lookup_clientid to call find_client_in_id_table directly. Update set_client to create client record for courtesy client. Update find_cpntf_state to discard courtesy client. Signed-off-by: Dai Ngo --- fs/nfsd/nfs4state.c | 34 ++++++++++++++++++++++++++++++++-- 1 file changed, 32 insertions(+), 2 deletions(-) diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c index 5cef1a78cc28..6e3ca0ea4f28 100644 --- a/fs/nfsd/nfs4state.c +++ b/fs/nfsd/nfs4state.c @@ -2915,6 +2915,19 @@ find_client_in_id_table(struct list_head *tbl, clientid_t *clid, bool sessions) if (same_clid(&clp->cl_clientid, clid)) { if ((bool)clp->cl_minorversion != sessions) return NULL; + + /* need to sync with thread resolving lock/deleg conflict */ + clear_bit(NFSD4_CLIENT_RECONNECTED, &clp->cl_flags); + spin_lock(&clp->cl_cs_lock); + if (test_bit(NFSD4_CLIENT_EXPIRED, &clp->cl_flags)) { + spin_unlock(&clp->cl_cs_lock); + continue; + } + if (test_bit(NFSD4_CLIENT_COURTESY, &clp->cl_flags)) { + clear_bit(NFSD4_CLIENT_COURTESY, &clp->cl_flags); + set_bit(NFSD4_CLIENT_RECONNECTED, &clp->cl_flags); + } + spin_unlock(&clp->cl_cs_lock); renew_client_locked(clp); return clp; } @@ -2922,6 +2935,7 @@ find_client_in_id_table(struct list_head *tbl, clientid_t *clid, bool sessions) return NULL; } + static void nfsd4_discard_courtesy_clnt(struct nfs4_client *clp) { @@ -2934,9 +2948,15 @@ static struct nfs4_client * find_confirmed_client(clientid_t *clid, bool sessions, struct nfsd_net *nn) { struct list_head *tbl = nn->conf_id_hashtbl; + struct nfs4_client *clp; lockdep_assert_held(&nn->client_lock); - return find_client_in_id_table(tbl, clid, sessions); + clp = find_client_in_id_table(tbl, clid, sessions); + if (clp && test_bit(NFSD4_CLIENT_RECONNECTED, &clp->cl_flags)) { + nfsd4_discard_courtesy_clnt(clp); + clp = NULL; + } + return clp; } static struct nfs4_client * @@ -4867,9 +4887,10 @@ static struct nfs4_client *lookup_clientid(clientid_t *clid, bool sessions, struct nfsd_net *nn) { struct nfs4_client *found; + struct list_head *tbl = nn->conf_id_hashtbl; spin_lock(&nn->client_lock); - found = find_confirmed_client(clid, sessions, nn); + found = find_client_in_id_table(tbl, clid, sessions); if (found) atomic_inc(&found->cl_rpc_users); spin_unlock(&nn->client_lock); @@ -4894,6 +4915,8 @@ static __be32 set_client(clientid_t *clid, cstate->clp = lookup_clientid(clid, false, nn); if (!cstate->clp) return nfserr_expired; + if (test_bit(NFSD4_CLIENT_RECONNECTED, &cstate->clp->cl_flags)) + nfsd4_client_record_create(cstate->clp); return nfs_ok; } @@ -6222,6 +6245,13 @@ static __be32 find_cpntf_state(struct nfsd_net *nn, stateid_t *st, found = lookup_clientid(&cps->cp_p_clid, true, nn); if (!found) goto out; + if (test_bit(NFSD4_CLIENT_RECONNECTED, &found->cl_flags)) { + nfsd4_discard_courtesy_clnt(found); + if (atomic_dec_and_lock(&found->cl_rpc_users, + &nn->client_lock)) + spin_unlock(&nn->client_lock); + goto out; + } *stid = find_stateid_by_type(found, &cps->cp_p_stateid, NFS4_DELEG_STID|NFS4_OPEN_STID|NFS4_LOCK_STID); From patchwork Wed Feb 23 18:16:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dai Ngo X-Patchwork-Id: 12757284 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5359FC4321E for ; Wed, 23 Feb 2022 18:19:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243680AbiBWSRS (ORCPT ); Wed, 23 Feb 2022 13:17:18 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47270 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243730AbiBWSRP (ORCPT ); Wed, 23 Feb 2022 13:17:15 -0500 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3F3573F8A2 for ; Wed, 23 Feb 2022 10:16:47 -0800 (PST) Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 21NIDn3I020614; Wed, 23 Feb 2022 18:16:46 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2021-07-09; bh=ZjYcVpKCyxjIzJLdor0eBbh4428ealhnxBPt5hhRrnI=; b=QvqcXvYUHkWzgwnAh5xD73nSj5H1JdTLbgeFIWakUMKFIAidktFygrpUVdn5TTO3hqRB Fr/A59RQqcEluX7JfZO+BlSnS7G3Fp7BA6WQlySp2WIHhznrZ/FlVn/4MIlVori8BQDx ndWjC/jAlQ6NvVlMfTc1c1ihZx0q/blrrm3xdpu5W0wSHHqs/kBhcQ77qVPHCLkBifS+ fs3nVyQGNwyCoAdxKHAfPYZBCoGn3uPZwsDsOcmgySfPsbAxa+2OQHylomF5o6Lbx7b8 C90vYnWOFOr/uvEUaOm2hWto3LDhbbAoE72puockwoPqL2qqjcEc5j6rflYyRe+kvE/P ow== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by mx0b-00069f02.pphosted.com with ESMTP id 3ecvar4r37-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 23 Feb 2022 18:16:45 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 21NIAtua155968; Wed, 23 Feb 2022 18:16:45 GMT Received: from pps.reinject (localhost [127.0.0.1]) by aserp3030.oracle.com with ESMTP id 3eapkj17sp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 23 Feb 2022 18:16:45 +0000 Received: from aserp3030.oracle.com (aserp3030.oracle.com [127.0.0.1]) by pps.reinject (8.16.0.36/8.16.0.36) with SMTP id 21NICxfg162308; Wed, 23 Feb 2022 18:16:44 GMT Received: from ca-common-hq.us.oracle.com (ca-common-hq.us.oracle.com [10.211.9.209]) by aserp3030.oracle.com with ESMTP id 3eapkj17pe-9; Wed, 23 Feb 2022 18:16:44 +0000 From: Dai Ngo To: chuck.lever@oracle.com, bfields@fieldses.org Cc: linux-nfs@vger.kernel.org Subject: [PATCH RFC v14 08/10] NFSD: Refactor nfsd4_laundromat() Date: Wed, 23 Feb 2022 10:16:35 -0800 Message-Id: <1645640197-1725-9-git-send-email-dai.ngo@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1645640197-1725-1-git-send-email-dai.ngo@oracle.com> References: <1645640197-1725-1-git-send-email-dai.ngo@oracle.com> X-Proofpoint-GUID: SXNI-D31ftWszAyRT2eTPTo75Srs7eVj X-Proofpoint-ORIG-GUID: SXNI-D31ftWszAyRT2eTPTo75Srs7eVj Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Extract a bit of logic that is about to be expanded to handle courtesy clients. Signed-off-by: Dai Ngo --- fs/nfsd/nfs4state.c | 33 +++++++++++++++++++++------------ 1 file changed, 21 insertions(+), 12 deletions(-) diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c index 6e3ca0ea4f28..01c51adf4873 100644 --- a/fs/nfsd/nfs4state.c +++ b/fs/nfsd/nfs4state.c @@ -5821,6 +5821,26 @@ static void nfsd4_ssc_expire_umount(struct nfsd_net *nn) } #endif +static void +nfs4_get_client_reaplist(struct nfsd_net *nn, struct list_head *reaplist, + struct laundry_time *lt) +{ + struct list_head *pos, *next; + struct nfs4_client *clp; + + INIT_LIST_HEAD(reaplist); + spin_lock(&nn->client_lock); + list_for_each_safe(pos, next, &nn->client_lru) { + clp = list_entry(pos, struct nfs4_client, cl_lru); + if (!state_expired(lt, clp->cl_time)) + break; + if (mark_client_expired_locked(clp)) + continue; + list_add(&clp->cl_lru, reaplist); + } + spin_unlock(&nn->client_lock); +} + static time64_t nfs4_laundromat(struct nfsd_net *nn) { @@ -5843,7 +5863,6 @@ nfs4_laundromat(struct nfsd_net *nn) goto out; } nfsd4_end_grace(nn); - INIT_LIST_HEAD(&reaplist); spin_lock(&nn->s2s_cp_lock); idr_for_each_entry(&nn->s2s_cp_stateids, cps_t, i) { @@ -5853,17 +5872,7 @@ nfs4_laundromat(struct nfsd_net *nn) _free_cpntf_state_locked(nn, cps); } spin_unlock(&nn->s2s_cp_lock); - - spin_lock(&nn->client_lock); - list_for_each_safe(pos, next, &nn->client_lru) { - clp = list_entry(pos, struct nfs4_client, cl_lru); - if (!state_expired(<, clp->cl_time)) - break; - if (mark_client_expired_locked(clp)) - continue; - list_add(&clp->cl_lru, &reaplist); - } - spin_unlock(&nn->client_lock); + nfs4_get_client_reaplist(nn, &reaplist, <); list_for_each_safe(pos, next, &reaplist) { clp = list_entry(pos, struct nfs4_client, cl_lru); trace_nfsd_clid_purged(&clp->cl_clientid); From patchwork Wed Feb 23 18:16:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dai Ngo X-Patchwork-Id: 12757285 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3811DC43219 for ; Wed, 23 Feb 2022 18:19:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243791AbiBWSRT (ORCPT ); Wed, 23 Feb 2022 13:17:19 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47314 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241030AbiBWSRP (ORCPT ); Wed, 23 Feb 2022 13:17:15 -0500 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 945C63F32A for ; Wed, 23 Feb 2022 10:16:47 -0800 (PST) Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 21NIDn3J020614; Wed, 23 Feb 2022 18:16:46 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2021-07-09; bh=9GNq6yDqJFj1aMXmz1G3Ltq3+oz2uI8MkPCk9phihzI=; b=ynKJsSmMyq3MnKhCChTFTDD0jkvw8KvihAZFE6khN2h+bRcNA7TelE2fJGqVsP2Px/PB A+niV375xiycL0CmxvRH/NHgUQyIVG5z9aPFsgF/wD2BFFDvXasNwpJGFh66XgcAa4iE msW9LHtC+LRNIX6m9+HAeuKJ+AZxciHOM2Gv6yk+cZMSpNTi9zn85OGzIDOcNcNIhVaY GAdRJUnL+W7qdrKrD2aPHHwCk6y0jla2KioVV6fDAOvk784Dkcfa2+iz9wbWsqPHhWaA 3lms7CLRVYGyULP9FHA+IMuPuBRz9s88yl6lsVHszzhEhgAXg6vC966FWtDlsG1GEWuc Dg== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by mx0b-00069f02.pphosted.com with ESMTP id 3ecvar4r38-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 23 Feb 2022 18:16:46 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 21NIAt8e156010; Wed, 23 Feb 2022 18:16:45 GMT Received: from pps.reinject (localhost [127.0.0.1]) by aserp3030.oracle.com with ESMTP id 3eapkj17t4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 23 Feb 2022 18:16:45 +0000 Received: from aserp3030.oracle.com (aserp3030.oracle.com [127.0.0.1]) by pps.reinject (8.16.0.36/8.16.0.36) with SMTP id 21NICxfi162308; Wed, 23 Feb 2022 18:16:45 GMT Received: from ca-common-hq.us.oracle.com (ca-common-hq.us.oracle.com [10.211.9.209]) by aserp3030.oracle.com with ESMTP id 3eapkj17pe-10; Wed, 23 Feb 2022 18:16:45 +0000 From: Dai Ngo To: chuck.lever@oracle.com, bfields@fieldses.org Cc: linux-nfs@vger.kernel.org Subject: [PATCH RFC v14 09/10] NFSD: Update laundromat to handle courtesy clients Date: Wed, 23 Feb 2022 10:16:36 -0800 Message-Id: <1645640197-1725-10-git-send-email-dai.ngo@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1645640197-1725-1-git-send-email-dai.ngo@oracle.com> References: <1645640197-1725-1-git-send-email-dai.ngo@oracle.com> X-Proofpoint-GUID: hgHISUJWxfSc69unftCSYsHHZHrXBgT9 X-Proofpoint-ORIG-GUID: hgHISUJWxfSc69unftCSYsHHZHrXBgT9 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Add nfs4_anylock_blocker to check if an expired client has any blockers. Update nfs4_get_client_reaplist to: . add discarded courtesy client; client marked with CLIENT_EXPIRED, to reaplist. . detect if expired client still has state and no blockers then transit it to courtesy client by setting CLIENT_COURTESY flag and removing the client record. Signed-off-by: Dai Ngo --- fs/nfsd/nfs4state.c | 100 ++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 98 insertions(+), 2 deletions(-) diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c index 01c51adf4873..282b8f040540 100644 --- a/fs/nfsd/nfs4state.c +++ b/fs/nfsd/nfs4state.c @@ -5821,24 +5821,120 @@ static void nfsd4_ssc_expire_umount(struct nfsd_net *nn) } #endif +static bool +nfs4_anylock_blocker(struct nfs4_client *clp) +{ + int i; + struct nfs4_stateowner *so, *tmp; + struct nfs4_lockowner *lo; + struct nfs4_ol_stateid *stp; + struct nfs4_file *nf; + struct inode *ino; + struct file_lock_context *ctx; + struct file_lock *fl; + + spin_lock(&clp->cl_lock); + for (i = 0; i < OWNER_HASH_SIZE; i++) { + /* scan each lock owner */ + list_for_each_entry_safe(so, tmp, &clp->cl_ownerstr_hashtbl[i], + so_strhash) { + if (so->so_is_open_owner) + continue; + + /* scan lock states of this lock owner */ + lo = lockowner(so); + list_for_each_entry(stp, &lo->lo_owner.so_stateids, + st_perstateowner) { + nf = stp->st_stid.sc_file; + ino = nf->fi_inode; + ctx = ino->i_flctx; + if (!ctx) + continue; + /* check each lock belongs to this lock state */ + list_for_each_entry(fl, &ctx->flc_posix, fl_list) { + if (fl->fl_owner != lo) + continue; + if (!list_empty(&fl->fl_blocked_requests)) { + spin_unlock(&clp->cl_lock); + return true; + } + } + } + } + } + spin_unlock(&clp->cl_lock); + return false; +} + static void nfs4_get_client_reaplist(struct nfsd_net *nn, struct list_head *reaplist, struct laundry_time *lt) { struct list_head *pos, *next; struct nfs4_client *clp; + bool cour; + struct list_head cslist; INIT_LIST_HEAD(reaplist); + INIT_LIST_HEAD(&cslist); spin_lock(&nn->client_lock); list_for_each_safe(pos, next, &nn->client_lru) { clp = list_entry(pos, struct nfs4_client, cl_lru); if (!state_expired(lt, clp->cl_time)) break; - if (mark_client_expired_locked(clp)) + + /* client expired */ + if (!client_has_state(clp)) { + if (mark_client_expired_locked(clp)) + continue; + list_add(&clp->cl_lru, reaplist); + continue; + } + + /* expired client has state */ + if (test_bit(NFSD4_CLIENT_EXPIRED, &clp->cl_flags)) + goto exp_client; + cour = test_bit(NFSD4_CLIENT_COURTESY, &clp->cl_flags); + if (cour && ktime_get_boottime_seconds() >= + (clp->cl_time + NFSD_COURTESY_CLIENT_TIMEOUT)) + goto exp_client; + if (nfs4_anylock_blocker(clp)) { + /* expired client has state and has blocker. */ +exp_client: + if (mark_client_expired_locked(clp)) + continue; + list_add(&clp->cl_lru, reaplist); continue; - list_add(&clp->cl_lru, reaplist); + } + /* + * Client expired and has state and has no blockers. + * If there is race condition with blockers, next time + * the laundromat runs it will catch it and expires + * the client. + */ + if (!cour) { + set_bit(NFSD4_CLIENT_COURTESY, &clp->cl_flags); + list_add(&clp->cl_cs_list, &cslist); + } } spin_unlock(&nn->client_lock); + + while (!list_empty(&cslist)) { + clp = list_first_entry(&cslist, struct nfs4_client, cl_cs_list); + list_del_init(&clp->cl_cs_list); + spin_lock(&clp->cl_cs_lock); + /* + * Client might have re-connected. Make sure it's + * still in courtesy state before removing its record. + */ + if (test_bit(NFSD4_CLIENT_EXPIRED, &clp->cl_flags) || + !test_bit(NFSD4_CLIENT_COURTESY, &clp->cl_flags)) { + spin_unlock(&clp->cl_cs_lock); + continue; + } + spin_unlock(&clp->cl_cs_lock); + nfsd4_client_record_remove(clp); + } } static time64_t From patchwork Wed Feb 23 18:16:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dai Ngo X-Patchwork-Id: 12757282 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 12F36C433FE for ; Wed, 23 Feb 2022 18:19:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243730AbiBWSRT (ORCPT ); Wed, 23 Feb 2022 13:17:19 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47316 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243791AbiBWSRP (ORCPT ); Wed, 23 Feb 2022 13:17:15 -0500 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1F17F3ED1F for ; Wed, 23 Feb 2022 10:16:48 -0800 (PST) Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 21NIDlBs029458; Wed, 23 Feb 2022 18:16:47 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2021-07-09; bh=eHrSfOzLuA/l3QfZyJUNjfB216919oN9sMyCE5alM58=; b=hYFMmXWAR22UtgtmejLiCbQV4HgtJ6XBt/f8pcWtZFqsqJC82r1rkkWXy4qBhvLnCOGQ /+UcoKnRV2aEGpd2pSTOytk3s6qWp1fjUhZLtB4r7Uz0G2Eu/G1A8M7jQQGyZ66Wt0SN wId4nnio1NuniMNaf3rssea/KmF0IEZzKDeKtmX2m7RAONZy/s1GkOaJm8KMLUumL50u CnaN2ZkNh33nZRehVt5eLGIbxWVoU7U4CGqjD9YOASnn0WSKZpXrBpfa8WQft2iT5qGZ GcWn/RDZvbKLfJjBjOphLoXpOZawy+UDYm4X/g0QwqMHR+jubQ7J930J5HG3xefFPO5T Iw== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by mx0b-00069f02.pphosted.com with ESMTP id 3ectsx54v2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 23 Feb 2022 18:16:46 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 21NIAtu8155939; Wed, 23 Feb 2022 18:16:46 GMT Received: from pps.reinject (localhost [127.0.0.1]) by aserp3030.oracle.com with ESMTP id 3eapkj17te-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 23 Feb 2022 18:16:46 +0000 Received: from aserp3030.oracle.com (aserp3030.oracle.com [127.0.0.1]) by pps.reinject (8.16.0.36/8.16.0.36) with SMTP id 21NICxfk162308; Wed, 23 Feb 2022 18:16:45 GMT Received: from ca-common-hq.us.oracle.com (ca-common-hq.us.oracle.com [10.211.9.209]) by aserp3030.oracle.com with ESMTP id 3eapkj17pe-11; Wed, 23 Feb 2022 18:16:45 +0000 From: Dai Ngo To: chuck.lever@oracle.com, bfields@fieldses.org Cc: linux-nfs@vger.kernel.org Subject: [PATCH RFC v14 10/10] NFSD: Show state of courtesy clients in client info Date: Wed, 23 Feb 2022 10:16:37 -0800 Message-Id: <1645640197-1725-11-git-send-email-dai.ngo@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1645640197-1725-1-git-send-email-dai.ngo@oracle.com> References: <1645640197-1725-1-git-send-email-dai.ngo@oracle.com> X-Proofpoint-ORIG-GUID: oLSa3pCi7FuAuOO9yySD7qYYYNbQsjIc X-Proofpoint-GUID: oLSa3pCi7FuAuOO9yySD7qYYYNbQsjIc Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Update client_info_show to show state of courtesy client and number of seconds since last renew. Signed-off-by: Dai Ngo --- fs/nfsd/nfs4state.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c index 282b8f040540..b2909ae816a8 100644 --- a/fs/nfsd/nfs4state.c +++ b/fs/nfsd/nfs4state.c @@ -2435,6 +2435,10 @@ static int client_info_show(struct seq_file *m, void *v) seq_puts(m, "status: confirmed\n"); else seq_puts(m, "status: unconfirmed\n"); + seq_printf(m, "courtesy client: %s\n", + test_bit(NFSD4_CLIENT_COURTESY, &clp->cl_flags) ? "yes" : "no"); + seq_printf(m, "seconds from last renew: %lld\n", + ktime_get_boottime_seconds() - clp->cl_time); seq_printf(m, "name: "); seq_quote_mem(m, clp->cl_name.data, clp->cl_name.len); seq_printf(m, "\nminor version: %d\n", clp->cl_minorversion);