From patchwork Wed Feb 23 18:16:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dai Ngo X-Patchwork-Id: 12757285 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3811DC43219 for ; Wed, 23 Feb 2022 18:19:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243791AbiBWSRT (ORCPT ); Wed, 23 Feb 2022 13:17:19 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47314 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241030AbiBWSRP (ORCPT ); Wed, 23 Feb 2022 13:17:15 -0500 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 945C63F32A for ; Wed, 23 Feb 2022 10:16:47 -0800 (PST) Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 21NIDn3J020614; Wed, 23 Feb 2022 18:16:46 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2021-07-09; bh=9GNq6yDqJFj1aMXmz1G3Ltq3+oz2uI8MkPCk9phihzI=; b=ynKJsSmMyq3MnKhCChTFTDD0jkvw8KvihAZFE6khN2h+bRcNA7TelE2fJGqVsP2Px/PB A+niV375xiycL0CmxvRH/NHgUQyIVG5z9aPFsgF/wD2BFFDvXasNwpJGFh66XgcAa4iE msW9LHtC+LRNIX6m9+HAeuKJ+AZxciHOM2Gv6yk+cZMSpNTi9zn85OGzIDOcNcNIhVaY GAdRJUnL+W7qdrKrD2aPHHwCk6y0jla2KioVV6fDAOvk784Dkcfa2+iz9wbWsqPHhWaA 3lms7CLRVYGyULP9FHA+IMuPuBRz9s88yl6lsVHszzhEhgAXg6vC966FWtDlsG1GEWuc Dg== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by mx0b-00069f02.pphosted.com with ESMTP id 3ecvar4r38-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 23 Feb 2022 18:16:46 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.1.2/8.16.1.2) with SMTP id 21NIAt8e156010; Wed, 23 Feb 2022 18:16:45 GMT Received: from pps.reinject (localhost [127.0.0.1]) by aserp3030.oracle.com with ESMTP id 3eapkj17t4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 23 Feb 2022 18:16:45 +0000 Received: from aserp3030.oracle.com (aserp3030.oracle.com [127.0.0.1]) by pps.reinject (8.16.0.36/8.16.0.36) with SMTP id 21NICxfi162308; Wed, 23 Feb 2022 18:16:45 GMT Received: from ca-common-hq.us.oracle.com (ca-common-hq.us.oracle.com [10.211.9.209]) by aserp3030.oracle.com with ESMTP id 3eapkj17pe-10; Wed, 23 Feb 2022 18:16:45 +0000 From: Dai Ngo To: chuck.lever@oracle.com, bfields@fieldses.org Cc: linux-nfs@vger.kernel.org Subject: [PATCH RFC v14 09/10] NFSD: Update laundromat to handle courtesy clients Date: Wed, 23 Feb 2022 10:16:36 -0800 Message-Id: <1645640197-1725-10-git-send-email-dai.ngo@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1645640197-1725-1-git-send-email-dai.ngo@oracle.com> References: <1645640197-1725-1-git-send-email-dai.ngo@oracle.com> X-Proofpoint-GUID: hgHISUJWxfSc69unftCSYsHHZHrXBgT9 X-Proofpoint-ORIG-GUID: hgHISUJWxfSc69unftCSYsHHZHrXBgT9 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Add nfs4_anylock_blocker to check if an expired client has any blockers. Update nfs4_get_client_reaplist to: . add discarded courtesy client; client marked with CLIENT_EXPIRED, to reaplist. . detect if expired client still has state and no blockers then transit it to courtesy client by setting CLIENT_COURTESY flag and removing the client record. Signed-off-by: Dai Ngo --- fs/nfsd/nfs4state.c | 100 ++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 98 insertions(+), 2 deletions(-) diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c index 01c51adf4873..282b8f040540 100644 --- a/fs/nfsd/nfs4state.c +++ b/fs/nfsd/nfs4state.c @@ -5821,24 +5821,120 @@ static void nfsd4_ssc_expire_umount(struct nfsd_net *nn) } #endif +static bool +nfs4_anylock_blocker(struct nfs4_client *clp) +{ + int i; + struct nfs4_stateowner *so, *tmp; + struct nfs4_lockowner *lo; + struct nfs4_ol_stateid *stp; + struct nfs4_file *nf; + struct inode *ino; + struct file_lock_context *ctx; + struct file_lock *fl; + + spin_lock(&clp->cl_lock); + for (i = 0; i < OWNER_HASH_SIZE; i++) { + /* scan each lock owner */ + list_for_each_entry_safe(so, tmp, &clp->cl_ownerstr_hashtbl[i], + so_strhash) { + if (so->so_is_open_owner) + continue; + + /* scan lock states of this lock owner */ + lo = lockowner(so); + list_for_each_entry(stp, &lo->lo_owner.so_stateids, + st_perstateowner) { + nf = stp->st_stid.sc_file; + ino = nf->fi_inode; + ctx = ino->i_flctx; + if (!ctx) + continue; + /* check each lock belongs to this lock state */ + list_for_each_entry(fl, &ctx->flc_posix, fl_list) { + if (fl->fl_owner != lo) + continue; + if (!list_empty(&fl->fl_blocked_requests)) { + spin_unlock(&clp->cl_lock); + return true; + } + } + } + } + } + spin_unlock(&clp->cl_lock); + return false; +} + static void nfs4_get_client_reaplist(struct nfsd_net *nn, struct list_head *reaplist, struct laundry_time *lt) { struct list_head *pos, *next; struct nfs4_client *clp; + bool cour; + struct list_head cslist; INIT_LIST_HEAD(reaplist); + INIT_LIST_HEAD(&cslist); spin_lock(&nn->client_lock); list_for_each_safe(pos, next, &nn->client_lru) { clp = list_entry(pos, struct nfs4_client, cl_lru); if (!state_expired(lt, clp->cl_time)) break; - if (mark_client_expired_locked(clp)) + + /* client expired */ + if (!client_has_state(clp)) { + if (mark_client_expired_locked(clp)) + continue; + list_add(&clp->cl_lru, reaplist); + continue; + } + + /* expired client has state */ + if (test_bit(NFSD4_CLIENT_EXPIRED, &clp->cl_flags)) + goto exp_client; + cour = test_bit(NFSD4_CLIENT_COURTESY, &clp->cl_flags); + if (cour && ktime_get_boottime_seconds() >= + (clp->cl_time + NFSD_COURTESY_CLIENT_TIMEOUT)) + goto exp_client; + if (nfs4_anylock_blocker(clp)) { + /* expired client has state and has blocker. */ +exp_client: + if (mark_client_expired_locked(clp)) + continue; + list_add(&clp->cl_lru, reaplist); continue; - list_add(&clp->cl_lru, reaplist); + } + /* + * Client expired and has state and has no blockers. + * If there is race condition with blockers, next time + * the laundromat runs it will catch it and expires + * the client. + */ + if (!cour) { + set_bit(NFSD4_CLIENT_COURTESY, &clp->cl_flags); + list_add(&clp->cl_cs_list, &cslist); + } } spin_unlock(&nn->client_lock); + + while (!list_empty(&cslist)) { + clp = list_first_entry(&cslist, struct nfs4_client, cl_cs_list); + list_del_init(&clp->cl_cs_list); + spin_lock(&clp->cl_cs_lock); + /* + * Client might have re-connected. Make sure it's + * still in courtesy state before removing its record. + */ + if (test_bit(NFSD4_CLIENT_EXPIRED, &clp->cl_flags) || + !test_bit(NFSD4_CLIENT_COURTESY, &clp->cl_flags)) { + spin_unlock(&clp->cl_cs_lock); + continue; + } + spin_unlock(&clp->cl_cs_lock); + nfsd4_client_record_remove(clp); + } } static time64_t