From patchwork Thu Jan 9 14:24:38 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 13932631 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 141D321A45A for ; Thu, 9 Jan 2025 14:24:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736432682; cv=none; b=pn34ijB2FI0hByKk5Pkvxfgun3m3gM/u5bJ2TffpxlXAoKI+EotApI46DvEXXUjclTuhM963MZb/xfcZ2n+yTWeaTnARq4kW5LDsHEGkucUE+K0lKlOO7/jRBOld0A+xSSZ1GHu1UAY2umUEDqzw6CCQ0G6BsriDmNdaPRY3MDk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736432682; c=relaxed/simple; bh=IcfMc+NIRIpXO7LEmmMSS3mPx1hloPaIFnF0mLA70jY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=SQdRVJ92GESbANwf3itT7m6flS+pAb5xu/Do5rozc4FAdQXzCwX9ty4qCSeORoZQGqMhhUasrr/tRw2ihE+LS1707tex7i84rS6x9f/pnjpXmpcJDWCvc+TR/lE9z+ybzSSlZmjS9euA0v3nqodEvp0/KkeQoyYPAc7njlIf21Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=pPV21QhM; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="pPV21QhM" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1ACB7C4CED6; Thu, 9 Jan 2025 14:24:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1736432681; bh=IcfMc+NIRIpXO7LEmmMSS3mPx1hloPaIFnF0mLA70jY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=pPV21QhMi9hWcwkWLP16F1EdEYQKC8+RVeNJW3vay1wcfXAmqd81isGGmglMhjaoW Pr6/p0L9Ibt2lG4I+4YMdrnELf8fSClzJxmWtetQqtmQDJlj6WuzfDRP1WXrqDLioC h4sLI220fuejP5Y9+IpyQhFVzYVkV4WfqNr2Lg3/SbeSF5E9G1T3RLQw3hTyDxyQAB yVUOH/tfQu0KUFq/WeP18ZWqcsHGvJ0J5UKZPSzlFwtqugA8SpJ6g9/rKTeNxmKeRB MXJEQbSt+Gm4wbMkPCvQQiNin0s/rij+miBTRxNOWSoaYH61LBTgF8jI8MzVv8Bbxz Ufb9OxsYStJpA== From: cel@kernel.org To: Neil Brown , Jeff Layton , Olga Kornievskaia , Dai Ngo , Tom Talpey Cc: Subject: [PATCH v1] nfsd: drop the lock during filecache LRU scans Date: Thu, 9 Jan 2025 09:24:38 -0500 Message-ID: <20250109142438.18689-2-cel@kernel.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250109142438.18689-1-cel@kernel.org> References: <20250109142438.18689-1-cel@kernel.org> Precedence: bulk X-Mailing-List: linux-nfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: NeilBrown Under a high NFSv3 load with lots of different file being accessed, the LRU list of garbage-collectable files can become quite long. Asking list_lru_scan() to scan the whole list can result in a long period during which a spinlock is held, blocking the addition and removal of LRU items. So ask list_lru_scan() to scan only a few entries at a time, and repeat until the scan is complete. Fixes: edead3a55804 ("NFSD: Fix the filecache LRU shrinker") Signed-off-by: NeilBrown Signed-off-by: Chuck Lever --- fs/nfsd/filecache.c | 14 ++++++++++---- fs/nfsd/filecache.h | 6 ++++++ 2 files changed, 16 insertions(+), 4 deletions(-) Updated version of Neil's patch to break up filecache LRU scans. This can be backported to LTS kernels -- a Fixes tag has been proposed. Subsequent work can replace the list_lru mechanism. That would be more substantial and thus more challenging to backport. There are two concerns here: - The number of items in the LRU can now change while this loop is operating. We might need a "if (!ret) break;" or some other exit condition to prevent an infinite loop. - The list_lru_walk() always starts at the tail of the LRU, rather than picking up where it left off. So it might only visit the same handful of items on the list repeatedly, reintroducing the bug fixed by edead3a55804. diff --git a/fs/nfsd/filecache.c b/fs/nfsd/filecache.c index a1cdba42c4fa..fcd751cb7c76 100644 --- a/fs/nfsd/filecache.c +++ b/fs/nfsd/filecache.c @@ -541,13 +541,19 @@ nfsd_file_lru_cb(struct list_head *item, struct list_lru_one *lru, static void nfsd_file_gc(void) { + unsigned long remaining = list_lru_count(&nfsd_file_lru); LIST_HEAD(dispose); unsigned long ret; - ret = list_lru_walk(&nfsd_file_lru, nfsd_file_lru_cb, - &dispose, list_lru_count(&nfsd_file_lru)); - trace_nfsd_file_gc_removed(ret, list_lru_count(&nfsd_file_lru)); - nfsd_file_dispose_list_delayed(&dispose); + while (remaining > 0) { + unsigned long num_to_scan = min(remaining, NFSD_FILE_GC_BATCH); + + ret = list_lru_walk(&nfsd_file_lru, nfsd_file_lru_cb, + &dispose, num_to_scan); + trace_nfsd_file_gc_removed(ret, list_lru_count(&nfsd_file_lru)); + nfsd_file_dispose_list_delayed(&dispose); + remaining -= num_to_scan; + } } static void diff --git a/fs/nfsd/filecache.h b/fs/nfsd/filecache.h index d5db6b34ba30..463bd60b98b4 100644 --- a/fs/nfsd/filecache.h +++ b/fs/nfsd/filecache.h @@ -3,6 +3,12 @@ #include +/* + * Limit the time that the list_lru_one lock is held during + * an LRU scan. + */ +#define NFSD_FILE_GC_BATCH (32UL) + /* * This is the fsnotify_mark container that nfsd attaches to the files that it * is holding open. Note that we have a separate refcount here aside from the