From patchwork Wed Jan 22 03:54:07 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: NeilBrown X-Patchwork-Id: 13946830 X-Patchwork-Delegate: cel@kernel.org Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AF8221FAA for ; Wed, 22 Jan 2025 03:56:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737518203; cv=none; b=bCNvq9Rw/8sDx5m1gKM+29lJhxw8MBTjrdicmWX9IZRmcJy8qnSYyGB7OsOi2YG7VO9D5OJ8arGitCTbQI3RyZb+tmy9umrdpzhzzlR3wcX0Eyqu8YLSUpw8cCi+DOQDzJjEQC7Ynf9jaYGgQ3HwVWYZTympQlOi06GU2hfApr4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737518203; c=relaxed/simple; bh=2OB1K81pQOWnf/9hNKIJxwqZa+LBSAB2TQ9OBDbM8Jo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=QtDS1h91IMPmAhT832t8/egQAEKpQ+przUqLNkzub9BUgpBa+kkr51+opM3eZ3iVunMn8vW7YJ/EDL8KvUivgAKNA93qJsV6FHbLOGYvXYuXD6j1JUGo4aBu1zw8/aUBBRo5UxSDIATp2DePlE0W+qczueZ8qqWFnsmsuSaspuA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=nGvKF17F; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=ZIzsyoZX; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=nGvKF17F; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=ZIzsyoZX; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="nGvKF17F"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="ZIzsyoZX"; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="nGvKF17F"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="ZIzsyoZX" Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id AA3FB1F78B; Wed, 22 Jan 2025 03:56:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1737518198; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=B3N0XTN+VLDtnHeRjOVo6Ik84WaCNtUaFaIQ4msW+mU=; b=nGvKF17FfKQzeuyg4tNuWkM0ZAZ2259ZT7x4byk1B37x4Vni05jWA17GnrAF3FLG84HqDA +cpEmoxov8YkrGkIwZhV4aKY4ldmB1vCCc5bEtAx97p6mhuqxaGX2z1xL/CRWQ/fkuwsax h64gtwyaaJwk5DHZvbQFn7wK6nETG6c= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1737518198; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=B3N0XTN+VLDtnHeRjOVo6Ik84WaCNtUaFaIQ4msW+mU=; b=ZIzsyoZXEiyrkttRYZYawaznIxHceXpIoo3UJzDWGdQW0s42a6II0pGWBK7AHSGuPzzzeM 10+Mjk7RByTwFYDA== Authentication-Results: smtp-out2.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=nGvKF17F; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=ZIzsyoZX DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1737518198; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=B3N0XTN+VLDtnHeRjOVo6Ik84WaCNtUaFaIQ4msW+mU=; b=nGvKF17FfKQzeuyg4tNuWkM0ZAZ2259ZT7x4byk1B37x4Vni05jWA17GnrAF3FLG84HqDA +cpEmoxov8YkrGkIwZhV4aKY4ldmB1vCCc5bEtAx97p6mhuqxaGX2z1xL/CRWQ/fkuwsax h64gtwyaaJwk5DHZvbQFn7wK6nETG6c= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1737518198; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=B3N0XTN+VLDtnHeRjOVo6Ik84WaCNtUaFaIQ4msW+mU=; b=ZIzsyoZXEiyrkttRYZYawaznIxHceXpIoo3UJzDWGdQW0s42a6II0pGWBK7AHSGuPzzzeM 10+Mjk7RByTwFYDA== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 889FF136A1; Wed, 22 Jan 2025 03:56:36 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id y0RkD3RskGfIWQAAD6G6ig (envelope-from ); Wed, 22 Jan 2025 03:56:36 +0000 From: NeilBrown To: Chuck Lever , Jeff Layton Cc: linux-nfs@vger.kernel.org, Olga Kornievskaia , Dai Ngo , Tom Talpey Subject: [PATCH 1/4] nfsd: filecache: use nfsd_file_dispose_list() in nfsd_file_close_inode_sync() Date: Wed, 22 Jan 2025 14:54:07 +1100 Message-ID: <20250122035615.2893754-2-neilb@suse.de> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250122035615.2893754-1-neilb@suse.de> References: <20250122035615.2893754-1-neilb@suse.de> Precedence: bulk X-Mailing-List: linux-nfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Rspamd-Queue-Id: AA3FB1F78B X-Spam-Level: X-Spamd-Result: default: False [-3.01 / 50.00]; BAYES_HAM(-3.00)[100.00%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-1.000]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; RCVD_COUNT_TWO(0.00)[2]; ASN(0.00)[asn:25478, ipnet:::/0, country:RU]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; FUZZY_BLOCKED(0.00)[rspamd.com]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; RCPT_COUNT_FIVE(0.00)[6]; DKIM_TRACE(0.00)[suse.de:+] X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Rspamd-Action: no action X-Spam-Score: -3.01 X-Spam-Flag: NO nfsd_file_close_inode_sync() contains an exactly copy of nfsd_file_dispose_list(). This patch removes it and calls nfsd_file_dispose_list() instead. Signed-off-by: NeilBrown --- fs/nfsd/filecache.c | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/fs/nfsd/filecache.c b/fs/nfsd/filecache.c index fcd751cb7c76..d8f98e847dc0 100644 --- a/fs/nfsd/filecache.c +++ b/fs/nfsd/filecache.c @@ -682,17 +682,12 @@ nfsd_file_close_inode(struct inode *inode) void nfsd_file_close_inode_sync(struct inode *inode) { - struct nfsd_file *nf; LIST_HEAD(dispose); trace_nfsd_file_close(inode); nfsd_file_queue_for_close(inode, &dispose); - while (!list_empty(&dispose)) { - nf = list_first_entry(&dispose, struct nfsd_file, nf_gc); - list_del_init(&nf->nf_gc); - nfsd_file_free(nf); - } + nfsd_file_dispose_list(&dispose); } static int From patchwork Wed Jan 22 03:54:08 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: NeilBrown X-Patchwork-Id: 13946831 X-Patchwork-Delegate: cel@kernel.org Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4F7681FAA for ; Wed, 22 Jan 2025 03:56:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737518212; cv=none; b=JI3qpC3iOcvL1V/PAVpPzyH/fRcTKe1GdERUHPCYRM8DRw7TQjM7OBk7ll8TiThhpV/3ym03Ql2K5tfw7L65kDtT2nApZl9JADE4Han9gSrsLucJpdVgW31qkEo5VzebcGVTZdlcGVaiFo5oOP6zRd3EOgY9DUu3OdhUalnjdfg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737518212; c=relaxed/simple; bh=LQaRNwU/7xYmf1Rz71p0qpuWrOLfHi7OzMlh+u1a55k=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=OfsttM4e8Yj94Jnbo70z8nKGxZVVV5Ywrz5VqhphNj7LGNnipg/iu40Ws3QuQnKkpS+3J11w2AWJwA8XOtEDRsHgImIdYOsirSVMQpLuM5fC+2bNOQxCRfx5gTRRTutxd0OHtcDhvyLuEkjSY2HMfKzCDrUZCwihnkiuTbhQuR8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=whOUGKJw; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=1dIKc7aW; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=whOUGKJw; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=1dIKc7aW; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="whOUGKJw"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="1dIKc7aW"; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="whOUGKJw"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="1dIKc7aW" Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 5B8BD1F78B; Wed, 22 Jan 2025 03:56:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1737518208; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MylGiVV0AprP7rP6uq51w30IKiahEq26NVk9x32ZVKs=; b=whOUGKJw7LJZbJyKOgVThzTrdRH6Vc/QHwkuZQFgopRCnDP6s+iB8CMUy5NB6Qnt3PU8HN byS6OYkB+6POVsXhqiI6d82UDxA5RIdhcFbrx3W5Rd6qbpuxl9BF3jCSPkR8JujQleRYkp nuyP3HZlurH3QcmK1vuo9zx8EIhYYOw= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1737518208; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MylGiVV0AprP7rP6uq51w30IKiahEq26NVk9x32ZVKs=; b=1dIKc7aWI330WMOvLXqGlQ9i1LUjIFWSwg9aNl3JsfCrQacD2eGFzuyA62U0goQYaofWW7 uj/NA/+StarKYwBw== Authentication-Results: smtp-out2.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1737518208; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MylGiVV0AprP7rP6uq51w30IKiahEq26NVk9x32ZVKs=; b=whOUGKJw7LJZbJyKOgVThzTrdRH6Vc/QHwkuZQFgopRCnDP6s+iB8CMUy5NB6Qnt3PU8HN byS6OYkB+6POVsXhqiI6d82UDxA5RIdhcFbrx3W5Rd6qbpuxl9BF3jCSPkR8JujQleRYkp nuyP3HZlurH3QcmK1vuo9zx8EIhYYOw= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1737518208; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MylGiVV0AprP7rP6uq51w30IKiahEq26NVk9x32ZVKs=; b=1dIKc7aWI330WMOvLXqGlQ9i1LUjIFWSwg9aNl3JsfCrQacD2eGFzuyA62U0goQYaofWW7 uj/NA/+StarKYwBw== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 39468136A1; Wed, 22 Jan 2025 03:56:45 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id 64yRN31skGcAXQAAD6G6ig (envelope-from ); Wed, 22 Jan 2025 03:56:45 +0000 From: NeilBrown To: Chuck Lever , Jeff Layton Cc: linux-nfs@vger.kernel.org, Olga Kornievskaia , Dai Ngo , Tom Talpey Subject: [PATCH 2/4] nfsd: filecache: move globals nfsd_file_lru and nfsd_file_shrinker to be per-net Date: Wed, 22 Jan 2025 14:54:08 +1100 Message-ID: <20250122035615.2893754-3-neilb@suse.de> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250122035615.2893754-1-neilb@suse.de> References: <20250122035615.2893754-1-neilb@suse.de> Precedence: bulk X-Mailing-List: linux-nfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Level: X-Spamd-Result: default: False [-2.80 / 50.00]; BAYES_HAM(-3.00)[100.00%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; ARC_NA(0.00)[]; TO_DN_SOME(0.00)[]; R_RATELIMIT(0.00)[from(RLewrxuus8mos16izbn)]; RCVD_VIA_SMTP_AUTH(0.00)[]; MIME_TRACE(0.00)[0:+]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_FIVE(0.00)[6]; FROM_EQ_ENVFROM(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] X-Spam-Score: -2.80 X-Spam-Flag: NO The final freeing of nfsd files is done by per-net nfsd threads (which call nfsd_file_net_dispose()) so it makes some sense to make more of the freeing infrastructure to be per-net - in struct nfsd_fcache_disposal. This is a step towards replacing the list_lru with simple lists which each share the per-net lock in nfsd_fcache_disposal and will require less list walking. As the net is always shutdown before there is any chance that the rest of the filecache is shut down we can removed the tests on NFSD_FILE_CACHE_UP. For the filecache stats file, which assumes a global lru, we keep a separate counter which includes all files in all netns lrus. Signed-off-by: NeilBrown --- fs/nfsd/filecache.c | 125 ++++++++++++++++++++++++-------------------- 1 file changed, 68 insertions(+), 57 deletions(-) diff --git a/fs/nfsd/filecache.c b/fs/nfsd/filecache.c index d8f98e847dc0..4f39f6632b35 100644 --- a/fs/nfsd/filecache.c +++ b/fs/nfsd/filecache.c @@ -63,17 +63,19 @@ static DEFINE_PER_CPU(unsigned long, nfsd_file_evictions); struct nfsd_fcache_disposal { spinlock_t lock; + struct list_lru file_lru; struct list_head freeme; + struct delayed_work filecache_laundrette; + struct shrinker *file_shrinker; }; static struct kmem_cache *nfsd_file_slab; static struct kmem_cache *nfsd_file_mark_slab; -static struct list_lru nfsd_file_lru; static unsigned long nfsd_file_flags; static struct fsnotify_group *nfsd_file_fsnotify_group; -static struct delayed_work nfsd_filecache_laundrette; static struct rhltable nfsd_file_rhltable ____cacheline_aligned_in_smp; +static atomic_long_t nfsd_lru_total = ATOMIC_LONG_INIT(0); static bool nfsd_match_cred(const struct cred *c1, const struct cred *c2) @@ -109,11 +111,18 @@ static const struct rhashtable_params nfsd_file_rhash_params = { }; static void -nfsd_file_schedule_laundrette(void) +nfsd_file_schedule_laundrette(struct nfsd_fcache_disposal *l) { - if (test_bit(NFSD_FILE_CACHE_UP, &nfsd_file_flags)) - queue_delayed_work(system_unbound_wq, &nfsd_filecache_laundrette, - NFSD_LAUNDRETTE_DELAY); + queue_delayed_work(system_unbound_wq, &l->filecache_laundrette, + NFSD_LAUNDRETTE_DELAY); +} + +static void +nfsd_file_schedule_laundrette_net(struct net *net) +{ + struct nfsd_net *nn = net_generic(net, nfsd_net_id); + + nfsd_file_schedule_laundrette(nn->fcache_disposal); } static void @@ -318,11 +327,14 @@ nfsd_file_check_writeback(struct nfsd_file *nf) mapping_tagged(mapping, PAGECACHE_TAG_WRITEBACK); } - static bool nfsd_file_lru_add(struct nfsd_file *nf) { + struct nfsd_net *nn = net_generic(nf->nf_net, nfsd_net_id); + struct nfsd_fcache_disposal *l = nn->fcache_disposal; + set_bit(NFSD_FILE_REFERENCED, &nf->nf_flags); - if (list_lru_add_obj(&nfsd_file_lru, &nf->nf_lru)) { + if (list_lru_add_obj(&l->file_lru, &nf->nf_lru)) { + atomic_long_inc(&nfsd_lru_total); trace_nfsd_file_lru_add(nf); return true; } @@ -331,7 +343,11 @@ static bool nfsd_file_lru_add(struct nfsd_file *nf) static bool nfsd_file_lru_remove(struct nfsd_file *nf) { - if (list_lru_del_obj(&nfsd_file_lru, &nf->nf_lru)) { + struct nfsd_net *nn = net_generic(nf->nf_net, nfsd_net_id); + struct nfsd_fcache_disposal *l = nn->fcache_disposal; + + if (list_lru_del_obj(&l->file_lru, &nf->nf_lru)) { + atomic_long_dec(&nfsd_lru_total); trace_nfsd_file_lru_del(nf); return true; } @@ -373,7 +389,7 @@ nfsd_file_put(struct nfsd_file *nf) if (nfsd_file_lru_add(nf)) { /* If it's still hashed, we're done */ if (test_bit(NFSD_FILE_HASHED, &nf->nf_flags)) { - nfsd_file_schedule_laundrette(); + nfsd_file_schedule_laundrette_net(nf->nf_net); return; } @@ -539,18 +555,18 @@ nfsd_file_lru_cb(struct list_head *item, struct list_lru_one *lru, } static void -nfsd_file_gc(void) +nfsd_file_gc(struct nfsd_fcache_disposal *l) { - unsigned long remaining = list_lru_count(&nfsd_file_lru); + unsigned long remaining = list_lru_count(&l->file_lru); LIST_HEAD(dispose); unsigned long ret; while (remaining > 0) { unsigned long num_to_scan = min(remaining, NFSD_FILE_GC_BATCH); - ret = list_lru_walk(&nfsd_file_lru, nfsd_file_lru_cb, + ret = list_lru_walk(&l->file_lru, nfsd_file_lru_cb, &dispose, num_to_scan); - trace_nfsd_file_gc_removed(ret, list_lru_count(&nfsd_file_lru)); + trace_nfsd_file_gc_removed(ret, list_lru_count(&l->file_lru)); nfsd_file_dispose_list_delayed(&dispose); remaining -= num_to_scan; } @@ -559,32 +575,36 @@ nfsd_file_gc(void) static void nfsd_file_gc_worker(struct work_struct *work) { - nfsd_file_gc(); - if (list_lru_count(&nfsd_file_lru)) - nfsd_file_schedule_laundrette(); + struct nfsd_fcache_disposal *l = container_of( + work, struct nfsd_fcache_disposal, filecache_laundrette.work); + nfsd_file_gc(l); + if (list_lru_count(&l->file_lru)) + nfsd_file_schedule_laundrette(l); } static unsigned long nfsd_file_lru_count(struct shrinker *s, struct shrink_control *sc) { - return list_lru_count(&nfsd_file_lru); + struct nfsd_fcache_disposal *l = s->private_data; + + return list_lru_count(&l->file_lru); } static unsigned long nfsd_file_lru_scan(struct shrinker *s, struct shrink_control *sc) { + struct nfsd_fcache_disposal *l = s->private_data; + LIST_HEAD(dispose); unsigned long ret; - ret = list_lru_shrink_walk(&nfsd_file_lru, sc, + ret = list_lru_shrink_walk(&l->file_lru, sc, nfsd_file_lru_cb, &dispose); - trace_nfsd_file_shrinker_removed(ret, list_lru_count(&nfsd_file_lru)); + trace_nfsd_file_shrinker_removed(ret, list_lru_count(&l->file_lru)); nfsd_file_dispose_list_delayed(&dispose); return ret; } -static struct shrinker *nfsd_file_shrinker; - /** * nfsd_file_cond_queue - conditionally unhash and queue a nfsd_file * @nf: nfsd_file to attempt to queue @@ -764,29 +784,10 @@ nfsd_file_cache_init(void) goto out_err; } - ret = list_lru_init(&nfsd_file_lru); - if (ret) { - pr_err("nfsd: failed to init nfsd_file_lru: %d\n", ret); - goto out_err; - } - - nfsd_file_shrinker = shrinker_alloc(0, "nfsd-filecache"); - if (!nfsd_file_shrinker) { - ret = -ENOMEM; - pr_err("nfsd: failed to allocate nfsd_file_shrinker\n"); - goto out_lru; - } - - nfsd_file_shrinker->count_objects = nfsd_file_lru_count; - nfsd_file_shrinker->scan_objects = nfsd_file_lru_scan; - nfsd_file_shrinker->seeks = 1; - - shrinker_register(nfsd_file_shrinker); - ret = lease_register_notifier(&nfsd_file_lease_notifier); if (ret) { pr_err("nfsd: unable to register lease notifier: %d\n", ret); - goto out_shrinker; + goto out_err; } nfsd_file_fsnotify_group = fsnotify_alloc_group(&nfsd_file_fsnotify_ops, @@ -799,17 +800,12 @@ nfsd_file_cache_init(void) goto out_notifier; } - INIT_DELAYED_WORK(&nfsd_filecache_laundrette, nfsd_file_gc_worker); out: if (ret) clear_bit(NFSD_FILE_CACHE_UP, &nfsd_file_flags); return ret; out_notifier: lease_unregister_notifier(&nfsd_file_lease_notifier); -out_shrinker: - shrinker_free(nfsd_file_shrinker); -out_lru: - list_lru_destroy(&nfsd_file_lru); out_err: kmem_cache_destroy(nfsd_file_slab); nfsd_file_slab = NULL; @@ -861,13 +857,36 @@ nfsd_alloc_fcache_disposal(void) if (!l) return NULL; spin_lock_init(&l->lock); + INIT_DELAYED_WORK(&l->filecache_laundrette, nfsd_file_gc_worker); INIT_LIST_HEAD(&l->freeme); + l->file_shrinker = shrinker_alloc(0, "nfsd-filecache"); + if (!l->file_shrinker) { + pr_err("nfsd: failed to allocate nfsd_file_shrinker\n"); + kfree(l); + return NULL; + } + l->file_shrinker->count_objects = nfsd_file_lru_count; + l->file_shrinker->scan_objects = nfsd_file_lru_scan; + l->file_shrinker->seeks = 1; + l->file_shrinker->private_data = l; + + if (list_lru_init(&l->file_lru)) { + pr_err("nfsd: failed to init nfsd_file_lru\n"); + shrinker_free(l->file_shrinker); + kfree(l); + return NULL; + } + + shrinker_register(l->file_shrinker); return l; } static void nfsd_free_fcache_disposal(struct nfsd_fcache_disposal *l) { + cancel_delayed_work_sync(&l->filecache_laundrette); + shrinker_free(l->file_shrinker); + list_lru_destroy(&l->file_lru); nfsd_file_dispose_list(&l->freeme); kfree(l); } @@ -899,8 +918,7 @@ void nfsd_file_cache_purge(struct net *net) { lockdep_assert_held(&nfsd_mutex); - if (test_bit(NFSD_FILE_CACHE_UP, &nfsd_file_flags) == 1) - __nfsd_file_cache_purge(net); + __nfsd_file_cache_purge(net); } void @@ -920,14 +938,7 @@ nfsd_file_cache_shutdown(void) return; lease_unregister_notifier(&nfsd_file_lease_notifier); - shrinker_free(nfsd_file_shrinker); - /* - * make sure all callers of nfsd_file_lru_cb are done before - * calling nfsd_file_cache_purge - */ - cancel_delayed_work_sync(&nfsd_filecache_laundrette); __nfsd_file_cache_purge(NULL); - list_lru_destroy(&nfsd_file_lru); rcu_barrier(); fsnotify_put_group(nfsd_file_fsnotify_group); nfsd_file_fsnotify_group = NULL; @@ -1298,7 +1309,7 @@ int nfsd_file_cache_stats_show(struct seq_file *m, void *v) struct bucket_table *tbl; struct rhashtable *ht; - lru = list_lru_count(&nfsd_file_lru); + lru = atomic_long_read(&nfsd_lru_total); rcu_read_lock(); ht = &nfsd_file_rhltable.ht; From patchwork Wed Jan 22 03:54:09 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: NeilBrown X-Patchwork-Id: 13946832 X-Patchwork-Delegate: cel@kernel.org Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2E33A136351 for ; Wed, 22 Jan 2025 03:56:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737518218; cv=none; b=kw5/LB0EqqlYMIb5paMsyw1/QKZruKZXwznN7La/9MAX0Z1KlomungIV2rJgbKagsKh/6sxRiNbBvszznhazzOIU1+5wgLC9C7GirH2OkZnrqmlHUt5atEvqaxnMwb/l5YTAQlVnD3BijFkL4onzMNYt05rEZUVFJQxulk431Yc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737518218; c=relaxed/simple; bh=IokhloFnkI92sgyRF+U2BWQ7i67xXC+oPo5UCOsuPWI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=BFNHgtfQEJo3jy9hqW6rEV5TMEadbIVeBgNWlOm+T3g3bFoWOpHY+M9LEcXgF9bUWwx47VzSFKlpFU8C3nzanUSiStnSBQNB11uZYAjiCtl9avUjLYAwwmQQxHzBmHziDZ4aW4PmB+3Xp2x5r/Tc5uAxhj9LACCpvfeSZneQCgY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=KS4KYsbo; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=9Lz6qvud; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=KS4KYsbo; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=9Lz6qvud; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="KS4KYsbo"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="9Lz6qvud"; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="KS4KYsbo"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="9Lz6qvud" Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 4F9B51F78B; Wed, 22 Jan 2025 03:56:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1737518214; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8uVay4fGgM4OZOk5+Qe9+fWwD8YbQsLc/BQNvDw9Ie4=; b=KS4KYsbocR2G6ed7G8S0e6LfQV7knOutTK5C77/E7gjjpF5ZAf9GL6GS7Y7hSq0H6Cf6lE Apo256pJleaWsjo5NZ/pLXo5Fiz4fSNLWWipu66uClhluzqaRgAHbWL96cvMd7ZQfDNcHs 5IVwNEK7il064zwybaDJygi0K+dRnT4= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1737518214; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8uVay4fGgM4OZOk5+Qe9+fWwD8YbQsLc/BQNvDw9Ie4=; b=9Lz6qvud/y+qwFpIuezoRCWGJcpl8p1da6XxmfC4VNVRR1/oGRtxM61KBPvv6u3JAZIvzh Ral93jvBuMR3/WCQ== Authentication-Results: smtp-out2.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=KS4KYsbo; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=9Lz6qvud DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1737518214; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8uVay4fGgM4OZOk5+Qe9+fWwD8YbQsLc/BQNvDw9Ie4=; b=KS4KYsbocR2G6ed7G8S0e6LfQV7knOutTK5C77/E7gjjpF5ZAf9GL6GS7Y7hSq0H6Cf6lE Apo256pJleaWsjo5NZ/pLXo5Fiz4fSNLWWipu66uClhluzqaRgAHbWL96cvMd7ZQfDNcHs 5IVwNEK7il064zwybaDJygi0K+dRnT4= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1737518214; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8uVay4fGgM4OZOk5+Qe9+fWwD8YbQsLc/BQNvDw9Ie4=; b=9Lz6qvud/y+qwFpIuezoRCWGJcpl8p1da6XxmfC4VNVRR1/oGRtxM61KBPvv6u3JAZIvzh Ral93jvBuMR3/WCQ== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id D96B7136A1; Wed, 22 Jan 2025 03:56:51 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id yVIPI4NskGeyXgAAD6G6ig (envelope-from ); Wed, 22 Jan 2025 03:56:51 +0000 From: NeilBrown To: Chuck Lever , Jeff Layton Cc: linux-nfs@vger.kernel.org, Olga Kornievskaia , Dai Ngo , Tom Talpey Subject: [PATCH 3/4] nfsd: filecache: change garbage collection list management. Date: Wed, 22 Jan 2025 14:54:09 +1100 Message-ID: <20250122035615.2893754-4-neilb@suse.de> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250122035615.2893754-1-neilb@suse.de> References: <20250122035615.2893754-1-neilb@suse.de> Precedence: bulk X-Mailing-List: linux-nfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Rspamd-Queue-Id: 4F9B51F78B X-Spam-Score: -3.01 X-Rspamd-Action: no action X-Spamd-Result: default: False [-3.01 / 50.00]; BAYES_HAM(-3.00)[100.00%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; ASN(0.00)[asn:25478, ipnet:::/0, country:RU]; MIME_TRACE(0.00)[0:+]; R_RATELIMIT(0.00)[from(RLewrxuus8mos16izbn)]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; ARC_NA(0.00)[]; FUZZY_BLOCKED(0.00)[rspamd.com]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_TLS_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCPT_COUNT_FIVE(0.00)[6]; DKIM_TRACE(0.00)[suse.de:+] X-Rspamd-Server: rspamd1.dmz-prg2.suse.org X-Spam-Flag: NO X-Spam-Level: The nfsd filecache currently uses list_lru for tracking files recently used in NFSv3 requests which need to be "garbage collected" when they have becoming idle - unused for 2-4 seconds. I do not believe list_lru is a good tool for this. It does not allow the timeout which filecache requires so we have to add a timeout mechanism which holds the list_lru lock while the whole list is scanned looking for entries that haven't been recently accessed. When the list is largish (even a few hundred) this can block new requests which need the lock to remove a file to access it. This patch removes the list_lru and instead uses 2 simple linked lists. When a file is accessed it is removed from whichever list it is on, then added to the tail of the first list. Every 2 seconds the second list is moved to the "freeme" list and the first list is moved to the second list. This avoids any need to walk a list to find old entries. Previously a file would be unhashed before being moved to the freeme list. We don't do that any more. The freeme list is much like the other two lists (recent and older) in that they all hold a reference to the file and the file is still hashed. When the nfsd thread processes the freeme list it now uses the new nfsd_file_release_list() which uses nfsd_file_cond_queue() to unhash and drop the refcount. We no longer have a precise count of the size of the lru (recent + older) as we don't know how big "older" is when it is moved to "freeme". However the shrinker can cope with an approximation. So we keep a count of number in the lru and when "older" is moved to "freeme" we divide that count by 2. When we remove anything from the lru we decrement that counter but ensure it never goes negative. Naturally when we add to the lru we increase the counter. For the filecache stats file, which assumes a global lru, we keep a separate counter which includes all files in all netns in recent or older or freeme. We discard the nf_gc linkage in an nfsd_file and only use nf_lru. We discard NFSD_FILE_REFERENCED. This patch drops the nfsd_file_gc_removed() trace point. I couldn't think of useful information to provide. Signed-off-by: NeilBrown --- fs/nfsd/filecache.c | 215 ++++++++++++++++++++++---------------------- fs/nfsd/filecache.h | 4 +- fs/nfsd/trace.h | 4 - 3 files changed, 108 insertions(+), 115 deletions(-) diff --git a/fs/nfsd/filecache.c b/fs/nfsd/filecache.c index 4f39f6632b35..552feba94f09 100644 --- a/fs/nfsd/filecache.c +++ b/fs/nfsd/filecache.c @@ -34,7 +34,6 @@ #include #include #include -#include #include #include #include @@ -63,10 +62,13 @@ static DEFINE_PER_CPU(unsigned long, nfsd_file_evictions); struct nfsd_fcache_disposal { spinlock_t lock; - struct list_lru file_lru; - struct list_head freeme; + struct list_head recent; /* have been used in last 0-2 seconds */ + struct list_head older; /* haven't been used in last 0-2 seconds */ + struct list_head freeme; /* ready to be discarded */ + unsigned long num_gc; /* Approximate size of recent plus older */ struct delayed_work filecache_laundrette; struct shrinker *file_shrinker; + struct nfsd_net *nn; }; static struct kmem_cache *nfsd_file_slab; @@ -227,7 +229,6 @@ nfsd_file_alloc(struct net *net, struct inode *inode, unsigned char need, this_cpu_inc(nfsd_file_allocations); INIT_LIST_HEAD(&nf->nf_lru); - INIT_LIST_HEAD(&nf->nf_gc); nf->nf_birthtime = ktime_get(); nf->nf_file = NULL; nf->nf_cred = get_current_cred(); @@ -332,12 +333,16 @@ static bool nfsd_file_lru_add(struct nfsd_file *nf) struct nfsd_net *nn = net_generic(nf->nf_net, nfsd_net_id); struct nfsd_fcache_disposal *l = nn->fcache_disposal; - set_bit(NFSD_FILE_REFERENCED, &nf->nf_flags); - if (list_lru_add_obj(&l->file_lru, &nf->nf_lru)) { + spin_lock(&l->lock); + if (list_empty(&nf->nf_lru)) { + list_add_tail(&nf->nf_lru, &l->recent); + l->num_gc += 1; atomic_long_inc(&nfsd_lru_total); trace_nfsd_file_lru_add(nf); + spin_unlock(&l->lock); return true; } + spin_unlock(&l->lock); return false; } @@ -346,11 +351,17 @@ static bool nfsd_file_lru_remove(struct nfsd_file *nf) struct nfsd_net *nn = net_generic(nf->nf_net, nfsd_net_id); struct nfsd_fcache_disposal *l = nn->fcache_disposal; - if (list_lru_del_obj(&l->file_lru, &nf->nf_lru)) { + spin_lock(&l->lock); + if (!list_empty(&nf->nf_lru)) { + list_del_init(&nf->nf_lru); atomic_long_dec(&nfsd_lru_total); + if (l->num_gc > 0) + l->num_gc -= 1; trace_nfsd_file_lru_del(nf); + spin_unlock(&l->lock); return true; } + spin_unlock(&l->lock); return false; } @@ -440,12 +451,26 @@ nfsd_file_dispose_list(struct list_head *dispose) struct nfsd_file *nf; while (!list_empty(dispose)) { - nf = list_first_entry(dispose, struct nfsd_file, nf_gc); - list_del_init(&nf->nf_gc); + nf = list_first_entry(dispose, struct nfsd_file, nf_lru); + list_del_init(&nf->nf_lru); nfsd_file_free(nf); } } +static void +nfsd_file_cond_queue(struct nfsd_file *nf, struct list_head *dispose); + +static void +nfsd_file_release_list(struct list_head *dispose) +{ + LIST_HEAD(dispose2); + struct nfsd_file *nf, *nf2; + + list_for_each_entry_safe(nf, nf2, dispose, nf_lru) + nfsd_file_cond_queue(nf, &dispose2); + nfsd_file_dispose_list(&dispose2); +} + /** * nfsd_file_dispose_list_delayed - move list of dead files to net's freeme list * @dispose: list of nfsd_files to be disposed @@ -458,12 +483,12 @@ nfsd_file_dispose_list_delayed(struct list_head *dispose) { while(!list_empty(dispose)) { struct nfsd_file *nf = list_first_entry(dispose, - struct nfsd_file, nf_gc); + struct nfsd_file, nf_lru); struct nfsd_net *nn = net_generic(nf->nf_net, nfsd_net_id); struct nfsd_fcache_disposal *l = nn->fcache_disposal; spin_lock(&l->lock); - list_move_tail(&nf->nf_gc, &l->freeme); + list_move_tail(&nf->nf_lru, &l->freeme); spin_unlock(&l->lock); svc_wake_up(nn->nfsd_serv); } @@ -487,88 +512,32 @@ void nfsd_file_net_dispose(struct nfsd_net *nn) int i; spin_lock(&l->lock); - for (i = 0; i < 8 && !list_empty(&l->freeme); i++) - list_move(l->freeme.next, &dispose); + for (i = 0; i < 8 && !list_empty(&l->freeme); i++) { + struct nfsd_file *nf = list_first_entry( + &l->freeme, struct nfsd_file, nf_lru); + + /* + * Don't throw out files that are still + * undergoing I/O or that have uncleared errors + * pending. + */ + if (nfsd_file_check_writeback(nf)) { + trace_nfsd_file_gc_writeback(nf); + list_move(&nf->nf_lru, &l->recent); + l->num_gc += 1; + } else { + trace_nfsd_file_gc_disposed(nf); + list_move(&nf->nf_lru, &dispose); + this_cpu_inc(nfsd_file_evictions); + } + } spin_unlock(&l->lock); if (!list_empty(&l->freeme)) /* Wake up another thread to share the work * *before* doing any actual disposing. */ svc_wake_up(nn->nfsd_serv); - nfsd_file_dispose_list(&dispose); - } -} - -/** - * nfsd_file_lru_cb - Examine an entry on the LRU list - * @item: LRU entry to examine - * @lru: controlling LRU - * @arg: dispose list - * - * Return values: - * %LRU_REMOVED: @item was removed from the LRU - * %LRU_ROTATE: @item is to be moved to the LRU tail - * %LRU_SKIP: @item cannot be evicted - */ -static enum lru_status -nfsd_file_lru_cb(struct list_head *item, struct list_lru_one *lru, - void *arg) -{ - struct list_head *head = arg; - struct nfsd_file *nf = list_entry(item, struct nfsd_file, nf_lru); - - /* We should only be dealing with GC entries here */ - WARN_ON_ONCE(!test_bit(NFSD_FILE_GC, &nf->nf_flags)); - - /* - * Don't throw out files that are still undergoing I/O or - * that have uncleared errors pending. - */ - if (nfsd_file_check_writeback(nf)) { - trace_nfsd_file_gc_writeback(nf); - return LRU_SKIP; - } - - /* If it was recently added to the list, skip it */ - if (test_and_clear_bit(NFSD_FILE_REFERENCED, &nf->nf_flags)) { - trace_nfsd_file_gc_referenced(nf); - return LRU_ROTATE; - } - - /* - * Put the reference held on behalf of the LRU. If it wasn't the last - * one, then just remove it from the LRU and ignore it. - */ - if (!refcount_dec_and_test(&nf->nf_ref)) { - trace_nfsd_file_gc_in_use(nf); - list_lru_isolate(lru, &nf->nf_lru); - return LRU_REMOVED; - } - - /* Refcount went to zero. Unhash it and queue it to the dispose list */ - nfsd_file_unhash(nf); - list_lru_isolate(lru, &nf->nf_lru); - list_add(&nf->nf_gc, head); - this_cpu_inc(nfsd_file_evictions); - trace_nfsd_file_gc_disposed(nf); - return LRU_REMOVED; -} - -static void -nfsd_file_gc(struct nfsd_fcache_disposal *l) -{ - unsigned long remaining = list_lru_count(&l->file_lru); - LIST_HEAD(dispose); - unsigned long ret; - - while (remaining > 0) { - unsigned long num_to_scan = min(remaining, NFSD_FILE_GC_BATCH); - - ret = list_lru_walk(&l->file_lru, nfsd_file_lru_cb, - &dispose, num_to_scan); - trace_nfsd_file_gc_removed(ret, list_lru_count(&l->file_lru)); - nfsd_file_dispose_list_delayed(&dispose); - remaining -= num_to_scan; + nfsd_file_release_list(&dispose); } } @@ -577,9 +546,20 @@ nfsd_file_gc_worker(struct work_struct *work) { struct nfsd_fcache_disposal *l = container_of( work, struct nfsd_fcache_disposal, filecache_laundrette.work); - nfsd_file_gc(l); - if (list_lru_count(&l->file_lru)) + + spin_lock(&l->lock); + list_splice_init(&l->older, &l->freeme); + list_splice_init(&l->recent, &l->older); + /* We don't know how many were moved to 'freeme' and don't want + * to waste time counting - guess a half. + */ + l->num_gc /= 2; + if (!list_empty(&l->freeme)) + svc_wake_up(l->nn->nfsd_serv); + if (!list_empty(&l->older) || !list_empty(&l->recent)) nfsd_file_schedule_laundrette(l); + spin_unlock(&l->lock); + } static unsigned long @@ -587,22 +567,40 @@ nfsd_file_lru_count(struct shrinker *s, struct shrink_control *sc) { struct nfsd_fcache_disposal *l = s->private_data; - return list_lru_count(&l->file_lru); + return l->num_gc; } static unsigned long nfsd_file_lru_scan(struct shrinker *s, struct shrink_control *sc) { struct nfsd_fcache_disposal *l = s->private_data; + struct nfsd_file *nf; + int scanned = 0; + + spin_lock(&l->lock); + while (scanned < sc->nr_to_scan && + (nf = list_first_entry_or_null(&l->older, + struct nfsd_file, nf_lru)) != NULL) { + list_del_init(&nf->nf_lru); + list_add_tail(&nf->nf_lru, &l->freeme); + if (l->num_gc > 0) + l->num_gc -= 1; + scanned += 1; + } + if (scanned > 0) + svc_wake_up(l->nn->nfsd_serv); - LIST_HEAD(dispose); - unsigned long ret; + trace_nfsd_file_shrinker_removed(scanned, l->num_gc); - ret = list_lru_shrink_walk(&l->file_lru, sc, - nfsd_file_lru_cb, &dispose); - trace_nfsd_file_shrinker_removed(ret, list_lru_count(&l->file_lru)); - nfsd_file_dispose_list_delayed(&dispose); - return ret; + while (scanned < sc->nr_to_scan && + (nf = list_first_entry_or_null(&l->recent, + struct nfsd_file, nf_lru)) != NULL) { + list_del_init(&nf->nf_lru); + list_add_tail(&nf->nf_lru, &l->older); + scanned += 1; + } + spin_unlock(&l->lock); + return scanned; } /** @@ -615,7 +613,6 @@ nfsd_file_lru_scan(struct shrinker *s, struct shrink_control *sc) */ static void nfsd_file_cond_queue(struct nfsd_file *nf, struct list_head *dispose) - __must_hold(RCU) { int decrement = 1; @@ -633,7 +630,7 @@ nfsd_file_cond_queue(struct nfsd_file *nf, struct list_head *dispose) /* If refcount goes to 0, then put on the dispose list */ if (refcount_sub_and_test(decrement, &nf->nf_ref)) { - list_add(&nf->nf_gc, dispose); + list_add(&nf->nf_lru, dispose); trace_nfsd_file_closing(nf); } } @@ -858,7 +855,10 @@ nfsd_alloc_fcache_disposal(void) return NULL; spin_lock_init(&l->lock); INIT_DELAYED_WORK(&l->filecache_laundrette, nfsd_file_gc_worker); + INIT_LIST_HEAD(&l->recent); + INIT_LIST_HEAD(&l->older); INIT_LIST_HEAD(&l->freeme); + l->num_gc = 0; l->file_shrinker = shrinker_alloc(0, "nfsd-filecache"); if (!l->file_shrinker) { pr_err("nfsd: failed to allocate nfsd_file_shrinker\n"); @@ -870,13 +870,6 @@ nfsd_alloc_fcache_disposal(void) l->file_shrinker->seeks = 1; l->file_shrinker->private_data = l; - if (list_lru_init(&l->file_lru)) { - pr_err("nfsd: failed to init nfsd_file_lru\n"); - shrinker_free(l->file_shrinker); - kfree(l); - return NULL; - } - shrinker_register(l->file_shrinker); return l; } @@ -886,8 +879,12 @@ nfsd_free_fcache_disposal(struct nfsd_fcache_disposal *l) { cancel_delayed_work_sync(&l->filecache_laundrette); shrinker_free(l->file_shrinker); - list_lru_destroy(&l->file_lru); - nfsd_file_dispose_list(&l->freeme); + nfsd_file_release_list(&l->recent); + WARN_ON_ONCE(!list_empty(&l->recent)); + nfsd_file_release_list(&l->older); + WARN_ON_ONCE(!list_empty(&l->older)); + nfsd_file_release_list(&l->freeme); + WARN_ON_ONCE(!list_empty(&l->freeme)); kfree(l); } @@ -906,6 +903,8 @@ nfsd_file_cache_start_net(struct net *net) struct nfsd_net *nn = net_generic(net, nfsd_net_id); nn->fcache_disposal = nfsd_alloc_fcache_disposal(); + if (nn->fcache_disposal) + nn->fcache_disposal->nn = nn; return nn->fcache_disposal ? 0 : -ENOMEM; } diff --git a/fs/nfsd/filecache.h b/fs/nfsd/filecache.h index a09a851d510e..02059b6c5e9a 100644 --- a/fs/nfsd/filecache.h +++ b/fs/nfsd/filecache.h @@ -42,15 +42,13 @@ struct nfsd_file { struct net *nf_net; #define NFSD_FILE_HASHED (0) #define NFSD_FILE_PENDING (1) -#define NFSD_FILE_REFERENCED (2) -#define NFSD_FILE_GC (3) +#define NFSD_FILE_GC (2) unsigned long nf_flags; refcount_t nf_ref; unsigned char nf_may; struct nfsd_file_mark *nf_mark; struct list_head nf_lru; - struct list_head nf_gc; struct rcu_head nf_rcu; ktime_t nf_birthtime; }; diff --git a/fs/nfsd/trace.h b/fs/nfsd/trace.h index ad2c0c432d08..efa683541ed5 100644 --- a/fs/nfsd/trace.h +++ b/fs/nfsd/trace.h @@ -1038,7 +1038,6 @@ DEFINE_CLID_EVENT(confirmed_r); __print_flags(val, "|", \ { 1 << NFSD_FILE_HASHED, "HASHED" }, \ { 1 << NFSD_FILE_PENDING, "PENDING" }, \ - { 1 << NFSD_FILE_REFERENCED, "REFERENCED" }, \ { 1 << NFSD_FILE_GC, "GC" }) DECLARE_EVENT_CLASS(nfsd_file_class, @@ -1314,9 +1313,7 @@ DEFINE_NFSD_FILE_GC_EVENT(nfsd_file_lru_add); DEFINE_NFSD_FILE_GC_EVENT(nfsd_file_lru_add_disposed); DEFINE_NFSD_FILE_GC_EVENT(nfsd_file_lru_del); DEFINE_NFSD_FILE_GC_EVENT(nfsd_file_lru_del_disposed); -DEFINE_NFSD_FILE_GC_EVENT(nfsd_file_gc_in_use); DEFINE_NFSD_FILE_GC_EVENT(nfsd_file_gc_writeback); -DEFINE_NFSD_FILE_GC_EVENT(nfsd_file_gc_referenced); DEFINE_NFSD_FILE_GC_EVENT(nfsd_file_gc_disposed); DECLARE_EVENT_CLASS(nfsd_file_lruwalk_class, @@ -1345,7 +1342,6 @@ DEFINE_EVENT(nfsd_file_lruwalk_class, name, \ ), \ TP_ARGS(removed, remaining)) -DEFINE_NFSD_FILE_LRUWALK_EVENT(nfsd_file_gc_removed); DEFINE_NFSD_FILE_LRUWALK_EVENT(nfsd_file_shrinker_removed); TRACE_EVENT(nfsd_file_close, From patchwork Wed Jan 22 03:54:10 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: NeilBrown X-Patchwork-Id: 13946833 X-Patchwork-Delegate: cel@kernel.org Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B8BCE1FAA for ; Wed, 22 Jan 2025 03:57:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737518227; cv=none; b=J24Qi5xgbU4gwbuJMTAkTq9nzt05H9+p07l5MwSYL7RQLRh97KBq1wpQKgQNbHKTlbB19CaByzWSEMxeu5GVOi2ub4okgy1HtJHCGGYhF1R6Ax4BbDEIk3f7Ai7xx17iPZL61mjLgxJE/q9jGrt3Fp8w2M2sHPzjLh19/zjNqx8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737518227; c=relaxed/simple; bh=7mNmbzgspCfVjClHXVKvqVsSIl8DiUzd6xgJ+zmlMq8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=aGXSVcQrQdphPy3/jHm1nWTgQaSsluzsDzQj31BllrCrHkukOlBLa7sdJfZuZfNc4YSOdW5bho7fzMI54IyGVrA6OBmgpD4nu8namFvxjBOlYRrKoPFklfg/SNONCJ7CBlRjh4uAVx9GAxUWJQzbnWHIYC99yK6nu1Y9gpRX0YE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=Tvm4r8WU; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=et9G/AZY; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=Tvm4r8WU; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=et9G/AZY; arc=none smtp.client-ip=195.135.223.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="Tvm4r8WU"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="et9G/AZY"; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="Tvm4r8WU"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="et9G/AZY" Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id E1177211F1; Wed, 22 Jan 2025 03:57:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1737518223; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PXUkRC0AnQ2vHmDB4VfbeJHXYAtPL9GaER7J50GWESE=; b=Tvm4r8WULjjkfwzmFn716KKLAG6omZaUE+SxgIj8sGXqJi+46d1qlaDl3eYJTE4QEHOZ2X nfDpaF7r4MFpZAZwOYleejcXVvuktJb3NQy4cRfPPkFxcPjfXdhrc9/U4YKx9Ifx1E5lZw 4o8SlBCvR3qzJhhaYiyAWsbB9duGuMY= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1737518223; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PXUkRC0AnQ2vHmDB4VfbeJHXYAtPL9GaER7J50GWESE=; b=et9G/AZYWqs9O2+SgyGp0cqrLolMFcjYIybBHL6ilPC3cRam+maPUHHGgcNV6iwit1XfIx D3QtVySdZRqV8UCg== Authentication-Results: smtp-out1.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1737518223; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PXUkRC0AnQ2vHmDB4VfbeJHXYAtPL9GaER7J50GWESE=; b=Tvm4r8WULjjkfwzmFn716KKLAG6omZaUE+SxgIj8sGXqJi+46d1qlaDl3eYJTE4QEHOZ2X nfDpaF7r4MFpZAZwOYleejcXVvuktJb3NQy4cRfPPkFxcPjfXdhrc9/U4YKx9Ifx1E5lZw 4o8SlBCvR3qzJhhaYiyAWsbB9duGuMY= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1737518223; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PXUkRC0AnQ2vHmDB4VfbeJHXYAtPL9GaER7J50GWESE=; b=et9G/AZYWqs9O2+SgyGp0cqrLolMFcjYIybBHL6ilPC3cRam+maPUHHGgcNV6iwit1XfIx D3QtVySdZRqV8UCg== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id BFEFB136A1; Wed, 22 Jan 2025 03:57:01 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id q7PwHI1skGelYQAAD6G6ig (envelope-from ); Wed, 22 Jan 2025 03:57:01 +0000 From: NeilBrown To: Chuck Lever , Jeff Layton Cc: linux-nfs@vger.kernel.org, Olga Kornievskaia , Dai Ngo , Tom Talpey Subject: [PATCH 4/4] nfsd: filecache: change garbage collection to a timer. Date: Wed, 22 Jan 2025 14:54:10 +1100 Message-ID: <20250122035615.2893754-5-neilb@suse.de> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250122035615.2893754-1-neilb@suse.de> References: <20250122035615.2893754-1-neilb@suse.de> Precedence: bulk X-Mailing-List: linux-nfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Spam-Score: -2.80 X-Spamd-Result: default: False [-2.80 / 50.00]; BAYES_HAM(-3.00)[100.00%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; ARC_NA(0.00)[]; TO_DN_SOME(0.00)[]; R_RATELIMIT(0.00)[from(RLewrxuus8mos16izbn)]; RCVD_VIA_SMTP_AUTH(0.00)[]; MIME_TRACE(0.00)[0:+]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_FIVE(0.00)[6]; FROM_EQ_ENVFROM(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] X-Spam-Flag: NO X-Spam-Level: garbage collection never sleeps and no longer walks a list so it runs quickly only requiring a spinlock. This means we don't need to use a workqueue, we can use a simple timer instead. This means it can run in "bh" context so we need to use "_bh" locking. Signed-off-by: NeilBrown --- fs/nfsd/filecache.c | 51 +++++++++++++++++++++++++-------------------- 1 file changed, 28 insertions(+), 23 deletions(-) diff --git a/fs/nfsd/filecache.c b/fs/nfsd/filecache.c index 552feba94f09..ebde4e86cdee 100644 --- a/fs/nfsd/filecache.c +++ b/fs/nfsd/filecache.c @@ -66,7 +66,7 @@ struct nfsd_fcache_disposal { struct list_head older; /* haven't been used in last 0-2 seconds */ struct list_head freeme; /* ready to be discarded */ unsigned long num_gc; /* Approximate size of recent plus older */ - struct delayed_work filecache_laundrette; + struct timer_list timer; struct shrinker *file_shrinker; struct nfsd_net *nn; }; @@ -115,8 +115,8 @@ static const struct rhashtable_params nfsd_file_rhash_params = { static void nfsd_file_schedule_laundrette(struct nfsd_fcache_disposal *l) { - queue_delayed_work(system_unbound_wq, &l->filecache_laundrette, - NFSD_LAUNDRETTE_DELAY); + if (!timer_pending(&l->timer)) + mod_timer(&l->timer, jiffies + NFSD_LAUNDRETTE_DELAY); } static void @@ -333,16 +333,16 @@ static bool nfsd_file_lru_add(struct nfsd_file *nf) struct nfsd_net *nn = net_generic(nf->nf_net, nfsd_net_id); struct nfsd_fcache_disposal *l = nn->fcache_disposal; - spin_lock(&l->lock); + spin_lock_bh(&l->lock); if (list_empty(&nf->nf_lru)) { list_add_tail(&nf->nf_lru, &l->recent); l->num_gc += 1; atomic_long_inc(&nfsd_lru_total); trace_nfsd_file_lru_add(nf); - spin_unlock(&l->lock); + spin_unlock_bh(&l->lock); return true; } - spin_unlock(&l->lock); + spin_unlock_bh(&l->lock); return false; } @@ -351,17 +351,17 @@ static bool nfsd_file_lru_remove(struct nfsd_file *nf) struct nfsd_net *nn = net_generic(nf->nf_net, nfsd_net_id); struct nfsd_fcache_disposal *l = nn->fcache_disposal; - spin_lock(&l->lock); + spin_lock_bh(&l->lock); if (!list_empty(&nf->nf_lru)) { list_del_init(&nf->nf_lru); atomic_long_dec(&nfsd_lru_total); if (l->num_gc > 0) l->num_gc -= 1; trace_nfsd_file_lru_del(nf); - spin_unlock(&l->lock); + spin_unlock_bh(&l->lock); return true; } - spin_unlock(&l->lock); + spin_unlock_bh(&l->lock); return false; } @@ -487,9 +487,9 @@ nfsd_file_dispose_list_delayed(struct list_head *dispose) struct nfsd_net *nn = net_generic(nf->nf_net, nfsd_net_id); struct nfsd_fcache_disposal *l = nn->fcache_disposal; - spin_lock(&l->lock); + spin_lock_bh(&l->lock); list_move_tail(&nf->nf_lru, &l->freeme); - spin_unlock(&l->lock); + spin_unlock_bh(&l->lock); svc_wake_up(nn->nfsd_serv); } } @@ -511,7 +511,7 @@ void nfsd_file_net_dispose(struct nfsd_net *nn) LIST_HEAD(dispose); int i; - spin_lock(&l->lock); + spin_lock_bh(&l->lock); for (i = 0; i < 8 && !list_empty(&l->freeme); i++) { struct nfsd_file *nf = list_first_entry( &l->freeme, struct nfsd_file, nf_lru); @@ -531,7 +531,7 @@ void nfsd_file_net_dispose(struct nfsd_net *nn) this_cpu_inc(nfsd_file_evictions); } } - spin_unlock(&l->lock); + spin_unlock_bh(&l->lock); if (!list_empty(&l->freeme)) /* Wake up another thread to share the work * *before* doing any actual disposing. @@ -542,12 +542,12 @@ void nfsd_file_net_dispose(struct nfsd_net *nn) } static void -nfsd_file_gc_worker(struct work_struct *work) +nfsd_file_gc_worker(struct timer_list *t) { struct nfsd_fcache_disposal *l = container_of( - work, struct nfsd_fcache_disposal, filecache_laundrette.work); + t, struct nfsd_fcache_disposal, timer); - spin_lock(&l->lock); + spin_lock_bh(&l->lock); list_splice_init(&l->older, &l->freeme); list_splice_init(&l->recent, &l->older); /* We don't know how many were moved to 'freeme' and don't want @@ -557,9 +557,8 @@ nfsd_file_gc_worker(struct work_struct *work) if (!list_empty(&l->freeme)) svc_wake_up(l->nn->nfsd_serv); if (!list_empty(&l->older) || !list_empty(&l->recent)) - nfsd_file_schedule_laundrette(l); - spin_unlock(&l->lock); - + mod_timer(t, jiffies + NFSD_LAUNDRETTE_DELAY); + spin_unlock_bh(&l->lock); } static unsigned long @@ -577,7 +576,7 @@ nfsd_file_lru_scan(struct shrinker *s, struct shrink_control *sc) struct nfsd_file *nf; int scanned = 0; - spin_lock(&l->lock); + spin_lock_bh(&l->lock); while (scanned < sc->nr_to_scan && (nf = list_first_entry_or_null(&l->older, struct nfsd_file, nf_lru)) != NULL) { @@ -599,7 +598,9 @@ nfsd_file_lru_scan(struct shrinker *s, struct shrink_control *sc) list_add_tail(&nf->nf_lru, &l->older); scanned += 1; } - spin_unlock(&l->lock); + spin_unlock_bh(&l->lock); + + trace_nfsd_file_shrinker_removed(scanned, l->num_gc); return scanned; } @@ -854,7 +855,9 @@ nfsd_alloc_fcache_disposal(void) if (!l) return NULL; spin_lock_init(&l->lock); - INIT_DELAYED_WORK(&l->filecache_laundrette, nfsd_file_gc_worker); + timer_setup(&l->timer, nfsd_file_gc_worker, 0); + INIT_LIST_HEAD(&l->recent); + INIT_LIST_HEAD(&l->older); INIT_LIST_HEAD(&l->recent); INIT_LIST_HEAD(&l->older); INIT_LIST_HEAD(&l->freeme); @@ -871,13 +874,15 @@ nfsd_alloc_fcache_disposal(void) l->file_shrinker->private_data = l; shrinker_register(l->file_shrinker); + return l; } static void nfsd_free_fcache_disposal(struct nfsd_fcache_disposal *l) { - cancel_delayed_work_sync(&l->filecache_laundrette); + shrinker_free(l->file_shrinker); + del_timer_sync(&l->timer); shrinker_free(l->file_shrinker); nfsd_file_release_list(&l->recent); WARN_ON_ONCE(!list_empty(&l->recent));