From patchwork Fri Oct 22 19:03:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 12578647 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2E940C433F5 for ; Fri, 22 Oct 2021 19:04:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 142A161059 for ; Fri, 22 Oct 2021 19:04:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233819AbhJVTGb (ORCPT ); Fri, 22 Oct 2021 15:06:31 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:36461 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233962AbhJVTGQ (ORCPT ); Fri, 22 Oct 2021 15:06:16 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1634929438; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=IjygLTuxyMJJmBBBlc6bt4XePM8ogI+vpCzocuor/SI=; b=YxdSdM53zFeU6yG2PBx+AHAoKJWOJYLQogKGrxL0b0uMM4AaoTvYAYtuZEJ79ASXKqtxFN EFWzYq2XuBcGtOChN4/UaRC83Pyf0/hAsBnieBZtajAftAWK/FYF2WNcAS6r3Ih1XX5+Es nTbeMagqsEvHg2MEdi6O/jzf83GQRCw= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-268-eWXJTznrOIirgweI1Fongw-1; Fri, 22 Oct 2021 15:03:55 -0400 X-MC-Unique: eWXJTznrOIirgweI1Fongw-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A8E4D1006AA3; Fri, 22 Oct 2021 19:03:52 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.33.36.19]) by smtp.corp.redhat.com (Postfix) with ESMTP id CFBFD60C13; Fri, 22 Oct 2021 19:03:46 +0000 (UTC) Organization: Red Hat UK Ltd. Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SI4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 3798903 Subject: [PATCH v2 20/53] fscache: Implement cookie invalidation From: David Howells To: linux-cachefs@redhat.com Cc: dhowells@redhat.com, Trond Myklebust , Anna Schumaker , Steve French , Dominique Martinet , Jeff Layton , Matthew Wilcox , Alexander Viro , Omar Sandoval , Linus Torvalds , linux-afs@lists.infradead.org, linux-nfs@vger.kernel.org, linux-cifs@vger.kernel.org, ceph-devel@vger.kernel.org, v9fs-developer@lists.sourceforge.net, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org Date: Fri, 22 Oct 2021 20:03:45 +0100 Message-ID: <163492942592.1038219.7631386299369357960.stgit@warthog.procyon.org.uk> In-Reply-To: <163492911924.1038219.13107463173777870713.stgit@warthog.procyon.org.uk> References: <163492911924.1038219.13107463173777870713.stgit@warthog.procyon.org.uk> User-Agent: StGit/0.23 MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Add a function to invalidate the cache behind a cookie: void fscache_invalidate(struct fscache_cookie *cookie, const void *aux_data, loff_t size, unsigned int flags) If there is one, any cached data for the specified cookie is discarded. If the cookie is marked as being in use, a new cache object will be created if possible and future I/O will use that instead. In-flight I/O will be abandoned (writes) or reperformed (reads). The coherency data attached to the cookie can be updated and the cookie size should be reset. One flag is available, FSCACHE_INVAL_DIO_WRITE, which should be used to indicate invalidation due to a DIO write on a file. This will temporarily disable caching for this cookie. Signed-off-by: David Howells cc: linux-cachefs@redhat.com --- fs/fscache/cookie.c | 72 +++++++++++++++++++++++++++++++++++++++- fs/fscache/internal.h | 2 + fs/fscache/stats.c | 5 +++ include/linux/fscache-cache.h | 3 ++ include/linux/fscache.h | 33 ++++++++++++++++++ include/linux/netfs.h | 1 + include/trace/events/fscache.h | 25 ++++++++++++++ 7 files changed, 140 insertions(+), 1 deletion(-) diff --git a/fs/fscache/cookie.c b/fs/fscache/cookie.c index cf987756534c..2ad889f20430 100644 --- a/fs/fscache/cookie.c +++ b/fs/fscache/cookie.c @@ -20,6 +20,7 @@ static void fscache_cookie_lru_worker(struct work_struct *work); static void fscache_cookie_worker(struct work_struct *work); static void fscache_drop_cookie(struct fscache_cookie *cookie); static void fscache_lookup_cookie(struct fscache_cookie *cookie); +static void fscache_invalidate_cookie(struct fscache_cookie *cookie); #define fscache_cookie_hash_shift 15 static struct hlist_bl_head fscache_cookie_hash[1 << fscache_cookie_hash_shift]; @@ -29,7 +30,7 @@ static LIST_HEAD(fscache_cookie_lru); static DEFINE_SPINLOCK(fscache_cookie_lru_lock); DEFINE_TIMER(fscache_cookie_lru_timer, fscache_cookie_lru_timed_out); static DECLARE_WORK(fscache_cookie_lru_work, fscache_cookie_lru_worker); -static const char fscache_cookie_stages[FSCACHE_COOKIE_STAGE__NR] = "-LCAFMWRD"; +static const char fscache_cookie_stages[FSCACHE_COOKIE_STAGE__NR] = "-LCAIFMWRD"; unsigned int fscache_lru_cookie_timeout = 10 * HZ; void fscache_print_cookie(struct fscache_cookie *cookie, char prefix) @@ -504,6 +505,7 @@ void __fscache_use_cookie(struct fscache_cookie *cookie, bool will_modify) set_bit(FSCACHE_COOKIE_LOCAL_WRITE, &cookie->flags); break; case FSCACHE_COOKIE_STAGE_ACTIVE: + case FSCACHE_COOKIE_STAGE_INVALIDATING: if (will_modify && !test_and_set_bit(FSCACHE_COOKIE_LOCAL_WRITE, &cookie->flags)) { set_bit(FSCACHE_COOKIE_DO_PREP_TO_WRITE, &cookie->flags); @@ -598,6 +600,10 @@ static void __fscache_cookie_worker(struct fscache_cookie *cookie) cookie->debug_id, cookie->stage); break; + case FSCACHE_COOKIE_STAGE_INVALIDATING: + fscache_invalidate_cookie(cookie); + goto again; + case FSCACHE_COOKIE_STAGE_FAILED: break; @@ -723,6 +729,70 @@ static void fscache_cookie_drop_from_lru(struct fscache_cookie *cookie) } } +/* + * Ask the cache to effect invalidation of a cookie. + */ +static void fscache_invalidate_cookie(struct fscache_cookie *cookie) +{ + cookie->volume->cache->ops->invalidate_cookie(cookie); + fscache_end_cookie_access(cookie, fscache_access_invalidate_cookie_end); +} + +/* + * Invalidate an object. + */ +void __fscache_invalidate(struct fscache_cookie *cookie, + const void *aux_data, loff_t new_size, + unsigned int flags) +{ + bool is_caching; + + _enter("c=%x", cookie->debug_id); + + fscache_stat(&fscache_n_invalidates); + + if (WARN(test_bit(FSCACHE_COOKIE_RELINQUISHED, &cookie->flags), + "Trying to invalidate relinquished cookie\n")) + return; + + if ((flags & FSCACHE_INVAL_DIO_WRITE) && + test_and_set_bit(FSCACHE_COOKIE_DISABLED, &cookie->flags)) + return; + + spin_lock(&cookie->lock); + set_bit(FSCACHE_COOKIE_NO_DATA_TO_READ, &cookie->flags); + fscache_update_aux(cookie, aux_data, &new_size); + cookie->inval_counter++; + trace_fscache_invalidate(cookie, new_size); + + switch (cookie->stage) { + case FSCACHE_COOKIE_STAGE_INVALIDATING: /* is_still_valid will catch it */ + default: + spin_unlock(&cookie->lock); + _leave(" [no %u]", cookie->stage); + return; + + case FSCACHE_COOKIE_STAGE_LOOKING_UP: + case FSCACHE_COOKIE_STAGE_CREATING: + spin_unlock(&cookie->lock); + _leave(" [look %x]", cookie->inval_counter); + return; + + case FSCACHE_COOKIE_STAGE_ACTIVE: + __fscache_set_cookie_stage(cookie, FSCACHE_COOKIE_STAGE_INVALIDATING); + is_caching = fscache_begin_cookie_access( + cookie, fscache_access_invalidate_cookie); + spin_unlock(&cookie->lock); + wake_up_cookie_stage(cookie); + + if (is_caching) + fscache_queue_cookie(cookie, fscache_cookie_get_inval_work); + _leave(" [inv]"); + return; + } +} +EXPORT_SYMBOL(__fscache_invalidate); + /* * Remove a cookie from the hash table. */ diff --git a/fs/fscache/internal.h b/fs/fscache/internal.h index 5aff9c143616..4c45d2af7160 100644 --- a/fs/fscache/internal.h +++ b/fs/fscache/internal.h @@ -82,6 +82,8 @@ extern atomic_t fscache_n_acquires_no_cache; extern atomic_t fscache_n_acquires_ok; extern atomic_t fscache_n_acquires_oom; +extern atomic_t fscache_n_invalidates; + extern atomic_t fscache_n_relinquishes; extern atomic_t fscache_n_relinquishes_retire; extern atomic_t fscache_n_relinquishes_dropped; diff --git a/fs/fscache/stats.c b/fs/fscache/stats.c index 17bf57374595..45b7636c7737 100644 --- a/fs/fscache/stats.c +++ b/fs/fscache/stats.c @@ -28,6 +28,8 @@ atomic_t fscache_n_acquires_no_cache; atomic_t fscache_n_acquires_ok; atomic_t fscache_n_acquires_oom; +atomic_t fscache_n_invalidates; + atomic_t fscache_n_updates; EXPORT_SYMBOL(fscache_n_updates); @@ -63,6 +65,9 @@ int fscache_stats_show(struct seq_file *m, void *v) timer_pending(&fscache_cookie_lru_timer) ? fscache_cookie_lru_timer.expires - jiffies : 0); + seq_printf(m, "Invals : n=%u\n", + atomic_read(&fscache_n_invalidates)); + seq_printf(m, "Updates: n=%u\n", atomic_read(&fscache_n_updates)); diff --git a/include/linux/fscache-cache.h b/include/linux/fscache-cache.h index a67c29207ad5..46d77a1840cd 100644 --- a/include/linux/fscache-cache.h +++ b/include/linux/fscache-cache.h @@ -66,6 +66,9 @@ struct fscache_cache_ops { /* Withdraw an object without any cookie access counts held */ void (*withdraw_cookie)(struct fscache_cookie *cookie); + /* Invalidate an object */ + bool (*invalidate_cookie)(struct fscache_cookie *cookie); + /* Prepare to write to a live cache object */ void (*prepare_to_write)(struct fscache_cookie *cookie); }; diff --git a/include/linux/fscache.h b/include/linux/fscache.h index df985507fa5e..0b01eaaef44c 100644 --- a/include/linux/fscache.h +++ b/include/linux/fscache.h @@ -45,6 +45,8 @@ struct fscache_cookie; #define FSCACHE_ADV_WRITE_NOCACHE 0x02 /* Don't cache if written to locally */ #define FSCACHE_ADV_FALLBACK_IO 0x04 /* Going to use the fallback I/O API (dangerous) */ +#define FSCACHE_INVAL_DIO_WRITE 0x01 /* Invalidate due to DIO write */ + /* * Data object state. */ @@ -53,6 +55,7 @@ enum fscache_cookie_stage { FSCACHE_COOKIE_STAGE_LOOKING_UP, /* The cache object is being looked up */ FSCACHE_COOKIE_STAGE_CREATING, /* The cache object is being created */ FSCACHE_COOKIE_STAGE_ACTIVE, /* The cache is active, readable and writable */ + FSCACHE_COOKIE_STAGE_INVALIDATING, /* The cache is being invalidated */ FSCACHE_COOKIE_STAGE_FAILED, /* The cache failed, withdraw to clear */ FSCACHE_COOKIE_STAGE_COMMITTING, /* The cookie is being committed */ FSCACHE_COOKIE_STAGE_WITHDRAWING, /* The cookie is being withdrawn */ @@ -158,6 +161,7 @@ extern struct fscache_cookie *__fscache_acquire_cookie( extern void __fscache_use_cookie(struct fscache_cookie *, bool); extern void __fscache_unuse_cookie(struct fscache_cookie *, const void *, const loff_t *); extern void __fscache_relinquish_cookie(struct fscache_cookie *, bool); +extern void __fscache_invalidate(struct fscache_cookie *, const void *, loff_t, unsigned int); /** * fscache_acquire_volume - Register a volume as desiring caching services @@ -325,4 +329,33 @@ void __fscache_update_cookie(struct fscache_cookie *cookie, const void *aux_data set_bit(FSCACHE_COOKIE_NEEDS_UPDATE, &cookie->flags); } +/** + * fscache_invalidate - Notify cache that an object needs invalidation + * @cookie: The cookie representing the cache object + * @aux_data: The updated auxiliary data for the cookie (may be NULL) + * @size: The revised size of the object. + * @flags: Invalidation flags (FSCACHE_INVAL_*) + * + * Notify the cache that an object is needs to be invalidated and that it + * should abort any retrievals or stores it is doing on the cache. The object + * is then marked non-caching until such time as the invalidation is complete. + * + * FSCACHE_INVAL_LIGHT indicates that if the object has been invalidated and + * replaced by a temporary object, the temporary object need not be replaced + * again. This is primarily intended for use with FSCACHE_ADV_SINGLE_CHUNK. + * + * FSCACHE_INVAL_DIO_WRITE indicates that this is due to a direct I/O write and + * may cause caching to be suspended on this cookie. + * + * See Documentation/filesystems/caching/netfs-api.rst for a complete + * description. + */ +static inline +void fscache_invalidate(struct fscache_cookie *cookie, + const void *aux_data, loff_t size, unsigned int flags) +{ + if (fscache_cookie_enabled(cookie)) + __fscache_invalidate(cookie, aux_data, size, flags); +} + #endif /* _LINUX_FSCACHE_H */ diff --git a/include/linux/netfs.h b/include/linux/netfs.h index 99137486d351..3c70eef56599 100644 --- a/include/linux/netfs.h +++ b/include/linux/netfs.h @@ -103,6 +103,7 @@ struct netfs_cache_resources { void *cache_priv; void *cache_priv2; unsigned int debug_id; /* Cookie debug ID */ + unsigned int inval_counter; /* object->inval_counter at begin_op */ }; /* diff --git a/include/trace/events/fscache.h b/include/trace/events/fscache.h index 2b31ef34efa2..b705ff36bddc 100644 --- a/include/trace/events/fscache.h +++ b/include/trace/events/fscache.h @@ -50,6 +50,7 @@ enum fscache_cookie_trace { fscache_cookie_discard, fscache_cookie_get_end_access, fscache_cookie_get_hash_collision, + fscache_cookie_get_inval_work, fscache_cookie_get_lru, fscache_cookie_get_use_work, fscache_cookie_new_acquire, @@ -72,6 +73,8 @@ enum fscache_access_trace { fscache_access_acquire_volume_end, fscache_access_cache_pin, fscache_access_cache_unpin, + fscache_access_invalidate_cookie, + fscache_access_invalidate_cookie_end, fscache_access_lookup_cookie, fscache_access_lookup_cookie_end, fscache_access_relinquish_volume, @@ -113,6 +116,7 @@ enum fscache_access_trace { EM(fscache_cookie_discard, "DISCARD ") \ EM(fscache_cookie_get_hash_collision, "GET hcoll") \ EM(fscache_cookie_get_end_access, "GQ endac") \ + EM(fscache_cookie_get_inval_work, "GQ inval") \ EM(fscache_cookie_get_lru, "GET lru ") \ EM(fscache_cookie_get_use_work, "GQ use ") \ EM(fscache_cookie_new_acquire, "NEW acq ") \ @@ -134,6 +138,8 @@ enum fscache_access_trace { EM(fscache_access_acquire_volume_end, "END acq_vol") \ EM(fscache_access_cache_pin, "PIN cache ") \ EM(fscache_access_cache_unpin, "UNPIN cache ") \ + EM(fscache_access_invalidate_cookie, "BEGIN inval ") \ + EM(fscache_access_invalidate_cookie_end,"END inval ") \ EM(fscache_access_lookup_cookie, "BEGIN lookup ") \ EM(fscache_access_lookup_cookie_end, "END lookup ") \ EM(fscache_access_relinquish_volume, "BEGIN rlq_vol") \ @@ -378,6 +384,25 @@ TRACE_EVENT(fscache_relinquish, __entry->n_active, __entry->flags, __entry->retire) ); +TRACE_EVENT(fscache_invalidate, + TP_PROTO(struct fscache_cookie *cookie, loff_t new_size), + + TP_ARGS(cookie, new_size), + + TP_STRUCT__entry( + __field(unsigned int, cookie ) + __field(loff_t, new_size ) + ), + + TP_fast_assign( + __entry->cookie = cookie->debug_id; + __entry->new_size = new_size; + ), + + TP_printk("c=%08x sz=%llx", + __entry->cookie, __entry->new_size) + ); + #endif /* _TRACE_FSCACHE_H */ /* This part must be outside protection */