From patchwork Tue Dec 10 02:48:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Al Viro X-Patchwork-Id: 13900757 Received: from zeniv.linux.org.uk (zeniv.linux.org.uk [62.89.141.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1216F1A2550; Tue, 10 Dec 2024 02:48:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=62.89.141.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733798911; cv=none; b=JfA66hFFNXc1raHCxPjipF59EDm1xDHuC7vqYASst4SjezovND0uNw6jZNzRwUtuau74wfEgEeRBwXP/hunyhJT1jyAipHT1ldBTL4YObSOHe+H+i7Zyfq9DaI+uflZtxeiJ+9VoXC7OR17X9LPIfiunEbPovFga3k/mt8VXmgQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733798911; c=relaxed/simple; bh=mpFYhDYvOtBbIT9K36AARTC1rTTgN9vbLzUIZUQo2s8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=I9Ty0ndtxg9VljsL/fmXlp0iCIoPNnnnu78l2scUpQ8DAYdhgzUwo9GbsPIG2gIhAKva70nT+GtHRlQZ23PxbXb+w0v5y0XiaBpMdiLbxBy/b2AeJ4kMVz0gFTkfzFskNRl4LKr1avtrt7PB9EcH3H5Sg9eVQoBURbEjEoUWEV0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zeniv.linux.org.uk; spf=none smtp.mailfrom=ftp.linux.org.uk; dkim=pass (2048-bit key) header.d=linux.org.uk header.i=@linux.org.uk header.b=esPZk/8V; arc=none smtp.client-ip=62.89.141.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zeniv.linux.org.uk Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=ftp.linux.org.uk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linux.org.uk header.i=@linux.org.uk header.b="esPZk/8V" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=linux.org.uk; s=zeniv-20220401; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=8gymd7MXbLtZrkDxBlIIctS56ONL2jrR6GESiXTtFpM=; b=esPZk/8Ves9bTkpV8lBoIDd5kJ InX1J3bP0OQsaVTGKUKSEtVCw6j2G+Gyt7R3faYhJ8+M2wsPjVU3ZHSha7r1jgbKu1RLqLS7vD3/7 JWuNo8kro6ekGdcAAewbh3XV1q1G6JXjlM3TUvGFrUF0TIX1Omlk51/pwtR5JZN2DvA9W+3Lb7suz XfXNAXw14p7UJp/jxfomWUPGrPXBrfgmBOKYx4D0Xm0OwSzmTPJcG19ukyBYqQL25LKHn/ce22ODb 3vXvH0wXktC3kXIqg5OmADPGY2S6MO7h35TU3Mbf2mPilcgpR4/lc3bKVuvZRwFFvoFWSFtDju74M G4l2WYhw==; Received: from viro by zeniv.linux.org.uk with local (Exim 4.98 #2 (Red Hat Linux)) id 1tKqIV-00000006lRq-1DgG; Tue, 10 Dec 2024 02:48:27 +0000 From: Al Viro To: Linus Torvalds Cc: linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH 1/5] make sure that DCACHE_INLINE_LEN is a multiple of word size Date: Tue, 10 Dec 2024 02:48:23 +0000 Message-ID: <20241210024827.1612355-1-viro@zeniv.linux.org.uk> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241210024523.GD3387508@ZenIV> References: <20241210024523.GD3387508@ZenIV> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Sender: Al Viro ... calling the number of words DCACHE_INLINE_WORDS. The next step will be to have a structure to hold inline name arrays (both in dentry and in name_snapshot) and use that to alias the existing arrays of unsigned char there. That will allow both full-structure copies and convenient word-by-word accesses. Signed-off-by: Al Viro --- fs/dcache.c | 4 +--- include/linux/dcache.h | 8 +++++--- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/fs/dcache.c b/fs/dcache.c index b4d5e9e1e43d..ea0f0bea511b 100644 --- a/fs/dcache.c +++ b/fs/dcache.c @@ -2748,9 +2748,7 @@ static void swap_names(struct dentry *dentry, struct dentry *target) /* * Both are internal. */ - unsigned int i; - BUILD_BUG_ON(!IS_ALIGNED(DNAME_INLINE_LEN, sizeof(long))); - for (i = 0; i < DNAME_INLINE_LEN / sizeof(long); i++) { + for (int i = 0; i < DNAME_INLINE_WORDS; i++) { swap(((long *) &dentry->d_iname)[i], ((long *) &target->d_iname)[i]); } diff --git a/include/linux/dcache.h b/include/linux/dcache.h index bff956f7b2b9..42dd89beaf4e 100644 --- a/include/linux/dcache.h +++ b/include/linux/dcache.h @@ -68,15 +68,17 @@ extern const struct qstr dotdot_name; * large memory footprint increase). */ #ifdef CONFIG_64BIT -# define DNAME_INLINE_LEN 40 /* 192 bytes */ +# define DNAME_INLINE_WORDS 5 /* 192 bytes */ #else # ifdef CONFIG_SMP -# define DNAME_INLINE_LEN 36 /* 128 bytes */ +# define DNAME_INLINE_WORDS 9 /* 128 bytes */ # else -# define DNAME_INLINE_LEN 44 /* 128 bytes */ +# define DNAME_INLINE_WORDS 11 /* 128 bytes */ # endif #endif +#define DNAME_INLINE_LEN (DNAME_INLINE_WORDS*sizeof(unsigned long)) + #define d_lock d_lockref.lock struct dentry { From patchwork Tue Dec 10 02:48:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Al Viro X-Patchwork-Id: 13900760 Received: from zeniv.linux.org.uk (zeniv.linux.org.uk [62.89.141.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 275641A2554; Tue, 10 Dec 2024 02:48:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=62.89.141.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733798912; cv=none; b=lXQsAiESWNvMlxr9mMtx7R02ilR1/N0uUZgjHUYlmgbjKufCsEb7KqXjpVYZQQL7kruqbqAl2wO0x9gpbpsOG3fE53Zmut5Wqnmf56MTbTtQkphBOEYiDFEjPr/2iKuiH9XtuXCA5e1VrKOO/pu9792/xoIWXCFNPNEPQTUgmxs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733798912; c=relaxed/simple; bh=AWcZYexTjmt6S+cfUq+ksXQumblz26xuxvK0tmTDgaQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Qjqn2moAchMEIPL1gk6OMcC2hPnpqXZDZQ3ySaLMfSBEbrk/ZO81yyLyqfbbRItY5zCFlX1WOyrnyY7jB/aVvkpisHeojJ+eqIeSdUm/16Cm+7i03sq+yrzZecNZlj3e91P1TcH8kFhOrcP2UNbGfjRP4jf/zASVGwVs4jyQk14= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zeniv.linux.org.uk; spf=none smtp.mailfrom=ftp.linux.org.uk; dkim=pass (2048-bit key) header.d=linux.org.uk header.i=@linux.org.uk header.b=W1FtGqkb; arc=none smtp.client-ip=62.89.141.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zeniv.linux.org.uk Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=ftp.linux.org.uk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linux.org.uk header.i=@linux.org.uk header.b="W1FtGqkb" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=linux.org.uk; s=zeniv-20220401; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=zs/1MAvhqT4o0Poecm2Da1zk8cYfon5BiqshWlcSZOs=; b=W1FtGqkb0mh8s+raSTxO2d7weo buF08FLPq2LZYZkkyC8/OLdbr9wsb4zC8hz1YadEyA1mjgvNaDgKdGwMG/8a2Vrz8zazfBgG5+4b2 e0sNzk3aZ4drX+98KUOeJx3/Awt+pn8RkDctscZqVNDCuX0Oe4hrtE3IgKRWS9R5gzuzpGFV8CdHL 6KYNFMTnf/KCaDHafg0acE0LMR6C+/MXLJOdE3vK8to2lW/RLudEeTpNd3d2Eb6+TocZbtv2xFmGc k1uKYDGtsYDmYjchrqiVNPEur+aiUt5ucntSPyt2eNIc5TCWxFnGa6gEh8zCW8VC17jDXKp45Z+Ma aJYM+lhA==; Received: from viro by zeniv.linux.org.uk with local (Exim 4.98 #2 (Red Hat Linux)) id 1tKqIV-00000006lRs-2ILo; Tue, 10 Dec 2024 02:48:27 +0000 From: Al Viro To: Linus Torvalds Cc: linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH 2/5] dcache: back inline names with a struct-wrapped array of unsigned long Date: Tue, 10 Dec 2024 02:48:24 +0000 Message-ID: <20241210024827.1612355-2-viro@zeniv.linux.org.uk> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241210024827.1612355-1-viro@zeniv.linux.org.uk> References: <20241210024523.GD3387508@ZenIV> <20241210024827.1612355-1-viro@zeniv.linux.org.uk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Sender: Al Viro ... so that they can be copied with struct assignment (which generates better code) and accessed word-by-word. swap_names() used to do the latter already, using casts, etc.; now that can be done cleanly. Both dentry->d_iname and name_snapshot->inline_name got such treatment. Signed-off-by: Al Viro --- fs/dcache.c | 19 +++++++------------ include/linux/dcache.h | 14 ++++++++++++-- 2 files changed, 19 insertions(+), 14 deletions(-) diff --git a/fs/dcache.c b/fs/dcache.c index ea0f0bea511b..007e582c3e68 100644 --- a/fs/dcache.c +++ b/fs/dcache.c @@ -334,8 +334,7 @@ void take_dentry_name_snapshot(struct name_snapshot *name, struct dentry *dentry if (unlikely(dname_external(dentry))) { atomic_inc(&external_name(dentry)->u.count); } else { - memcpy(name->inline_name, dentry->d_iname, - dentry->d_name.len + 1); + name->inline_name_words = dentry->d_iname_words; name->name.name = name->inline_name; } spin_unlock(&dentry->d_lock); @@ -2729,9 +2728,8 @@ static void swap_names(struct dentry *dentry, struct dentry *target) * dentry:internal, target:external. Steal target's * storage and make target internal. */ - memcpy(target->d_iname, dentry->d_name.name, - dentry->d_name.len + 1); dentry->d_name.name = target->d_name.name; + target->d_iname_words = dentry->d_iname_words; target->d_name.name = target->d_iname; } } else { @@ -2740,18 +2738,16 @@ static void swap_names(struct dentry *dentry, struct dentry *target) * dentry:external, target:internal. Give dentry's * storage to target and make dentry internal */ - memcpy(dentry->d_iname, target->d_name.name, - target->d_name.len + 1); target->d_name.name = dentry->d_name.name; + dentry->d_iname_words = target->d_iname_words; dentry->d_name.name = dentry->d_iname; } else { /* * Both are internal. */ - for (int i = 0; i < DNAME_INLINE_WORDS; i++) { - swap(((long *) &dentry->d_iname)[i], - ((long *) &target->d_iname)[i]); - } + for (int i = 0; i < DNAME_INLINE_WORDS; i++) + swap(dentry->d_iname_words.words[i], + target->d_iname_words.words[i]); } } swap(dentry->d_name.hash_len, target->d_name.hash_len); @@ -2766,8 +2762,7 @@ static void copy_name(struct dentry *dentry, struct dentry *target) atomic_inc(&external_name(target)->u.count); dentry->d_name = target->d_name; } else { - memcpy(dentry->d_iname, target->d_name.name, - target->d_name.len + 1); + dentry->d_iname_words = target->d_iname_words; dentry->d_name.name = dentry->d_iname; dentry->d_name.hash_len = target->d_name.hash_len; } diff --git a/include/linux/dcache.h b/include/linux/dcache.h index 42dd89beaf4e..766a9156836e 100644 --- a/include/linux/dcache.h +++ b/include/linux/dcache.h @@ -79,6 +79,10 @@ extern const struct qstr dotdot_name; #define DNAME_INLINE_LEN (DNAME_INLINE_WORDS*sizeof(unsigned long)) +struct shortname_store { + unsigned long words[DNAME_INLINE_WORDS]; +}; + #define d_lock d_lockref.lock struct dentry { @@ -90,7 +94,10 @@ struct dentry { struct qstr d_name; struct inode *d_inode; /* Where the name belongs to - NULL is * negative */ - unsigned char d_iname[DNAME_INLINE_LEN]; /* small names */ + union { + unsigned char d_iname[DNAME_INLINE_LEN]; /* small names */ + struct shortname_store d_iname_words; + }; /* --- cacheline 1 boundary (64 bytes) was 32 bytes ago --- */ /* Ref lookup also touches following */ @@ -591,7 +598,10 @@ static inline struct inode *d_real_inode(const struct dentry *dentry) struct name_snapshot { struct qstr name; - unsigned char inline_name[DNAME_INLINE_LEN]; + union { + unsigned char inline_name[DNAME_INLINE_LEN]; + struct shortname_store inline_name_words; + }; }; void take_dentry_name_snapshot(struct name_snapshot *, struct dentry *); void release_dentry_name_snapshot(struct name_snapshot *); From patchwork Tue Dec 10 02:48:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Al Viro X-Patchwork-Id: 13900761 Received: from zeniv.linux.org.uk (zeniv.linux.org.uk [62.89.141.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 275C01AAA2E; Tue, 10 Dec 2024 02:48:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=62.89.141.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733798913; cv=none; b=Q2JdSDCOIpZvpurnY2wpKmkYz6WSgJpsKxdH54XVVZJSnKER+K+C8DS+C05F+FVdDicvC4TqaoWeINLyGBNXLc1CXwS44uQORZJibqHGTe7n+QHGPwPnvnLNQabvRIz9X9UiRI+WoQuwMNxf6UgvpkjriqlQPrBMlJE7T0hV+Io= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733798913; c=relaxed/simple; bh=J3kXarI6VfE3ubZPnhnT/VfrRk0OoyVolinamsq6msw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ByD8B5dOUsbO+EPQjzof5Lr94ipbKUywjrYho+Je+QHKY6L45YstsZoxm054ZvlI/+QXjK5GPCA+eSoczIPKRMOJFkoD+cL9oYjv/uLaQzSVH1N7c52F6LzNb8F8mRPGHW/VBwsTlyRVH5E+eyCQNjP33w0WwHSBUbdLyBm3M1Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zeniv.linux.org.uk; spf=none smtp.mailfrom=ftp.linux.org.uk; dkim=pass (2048-bit key) header.d=linux.org.uk header.i=@linux.org.uk header.b=DcKT+mUr; arc=none smtp.client-ip=62.89.141.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zeniv.linux.org.uk Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=ftp.linux.org.uk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linux.org.uk header.i=@linux.org.uk header.b="DcKT+mUr" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=linux.org.uk; s=zeniv-20220401; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=apmt3CdIEMCI+pB2xWtd2xgUIxQVHjMWICuGg7hFcwM=; b=DcKT+mUrmx74Sk/+1e23PZ7h5U O/H+3rgeoWx3V39kvgi0ijPF/ULz5hR4a6OTw0W+G5hQCNRvKEDAT/KLxVHMQa05kcEUig0ozHfuh 6oU7csSvaQdl7h/Jei9GQR2Yt7g33JgSLqP7OZtlqw0U5Ore5hR0qqtPg4ls7pgZg31iB008gj1fP XkQkd7SbJlAHH5Q9utAKepOQSDVkbmT1Ufwj4o9/RV4GMhCSE0eAxrza1+QdEyj0ZrmWVu/e9SoWs UmLjcunY/FJiG7QL1eOpSmUL8vkjgB5hlugxSXcbCUYzBXc4/dUetFY0+jPLjpqhYAB/V9XkeaB0+ T3+qKCnw==; Received: from viro by zeniv.linux.org.uk with local (Exim 4.98 #2 (Red Hat Linux)) id 1tKqIV-00000006lRw-2zE5; Tue, 10 Dec 2024 02:48:27 +0000 From: Al Viro To: Linus Torvalds Cc: linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH 3/5] make take_dentry_name_snapshot() lockless Date: Tue, 10 Dec 2024 02:48:25 +0000 Message-ID: <20241210024827.1612355-3-viro@zeniv.linux.org.uk> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241210024827.1612355-1-viro@zeniv.linux.org.uk> References: <20241210024523.GD3387508@ZenIV> <20241210024827.1612355-1-viro@zeniv.linux.org.uk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Sender: Al Viro Use ->d_seq instead of grabbing ->d_lock; in case of shortname dentries that avoids any stores to shared data objects and in case of long names we are down to (unavoidable) atomic_inc on the external_name refcount. Makes the thing safer as well - the areas where ->d_seq is held odd are all nested inside the areas where ->d_lock is held, and the latter are much more numerous. NOTE: we no longer can have external_name.u.count and external_name.u.head sharing space, now that we have lockless path that might try to grab a reference on already doomed instance (kudos to Linus for spotting that). For now just turn that external_name.u into a struct (instead of union) to reduce the noise in this commit; the next commit will dissolve it. Signed-off-by: Al Viro --- fs/dcache.c | 38 ++++++++++++++++++++++++++++---------- 1 file changed, 28 insertions(+), 10 deletions(-) diff --git a/fs/dcache.c b/fs/dcache.c index 007e582c3e68..ae13e89ce7d7 100644 --- a/fs/dcache.c +++ b/fs/dcache.c @@ -296,9 +296,9 @@ static inline int dentry_cmp(const struct dentry *dentry, const unsigned char *c } struct external_name { - union { - atomic_t count; - struct rcu_head head; + struct { + atomic_t count; // ->count and ->head can't be combined + struct rcu_head head; // see take_dentry_name_snapshot() } u; unsigned char name[]; }; @@ -329,15 +329,33 @@ static inline int dname_external(const struct dentry *dentry) void take_dentry_name_snapshot(struct name_snapshot *name, struct dentry *dentry) { - spin_lock(&dentry->d_lock); - name->name = dentry->d_name; - if (unlikely(dname_external(dentry))) { - atomic_inc(&external_name(dentry)->u.count); - } else { - name->inline_name_words = dentry->d_iname_words; + unsigned seq; + const unsigned char *s; + + rcu_read_lock(); +retry: + seq = read_seqcount_begin(&dentry->d_seq); + s = READ_ONCE(dentry->d_name.name); + name->name.hash_len = dentry->d_name.hash_len; + if (likely(s == dentry->d_iname)) { name->name.name = name->inline_name; + name->inline_name_words = dentry->d_iname_words; + if (read_seqcount_retry(&dentry->d_seq, seq)) + goto retry; + } else { + struct external_name *p; + p = container_of(s, struct external_name, name[0]); + name->name.name = s; + // get a valid reference + if (unlikely(!atomic_inc_not_zero(&p->u.count))) + goto retry; + if (read_seqcount_retry(&dentry->d_seq, seq)) { + if (unlikely(atomic_dec_and_test(&p->u.count))) + kfree_rcu(p, u.head); + goto retry; + } } - spin_unlock(&dentry->d_lock); + rcu_read_unlock(); } EXPORT_SYMBOL(take_dentry_name_snapshot); From patchwork Tue Dec 10 02:48:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Al Viro X-Patchwork-Id: 13900758 Received: from zeniv.linux.org.uk (zeniv.linux.org.uk [62.89.141.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 71D1E1AB6EB; Tue, 10 Dec 2024 02:48:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=62.89.141.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733798912; cv=none; b=bQiU0gDMrtRbkyyghGJ0ado77PFhvR4cm+aHZIgPqm3IlXOJWDg8iXcpFJdA3jm0RzyART3t0L0TQXHSUWm1Ygu//0n+Pca90TLv2DUpnmmQULIbnXCvEtOTV5Cyq5MswzhMU3KvVOaooywODMb+cC+mmAFI+oaDrydDJwSWBsY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733798912; c=relaxed/simple; bh=13y3cZyhTP/Hfvy1e1EVP7Q5/Ew2YMV6nCjpyOiYypY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=K3EF9ylIvo20Qoe73Oo3JOiVBVldFSRv2r4ODKG7kK9GJndYoBz+0d6eQLq7LGFOnWZ13+xnB0ZRvamuQN3TFDaTo4LTTLjDEoFMWUZdbk6jzfayv40YPDihAZ+SRSibwihLf79whU0y0/oz7nTRp4mTzCS7aGS2W+qGL7ZFdto= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zeniv.linux.org.uk; spf=none smtp.mailfrom=ftp.linux.org.uk; dkim=pass (2048-bit key) header.d=linux.org.uk header.i=@linux.org.uk header.b=taJ0EkwI; arc=none smtp.client-ip=62.89.141.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zeniv.linux.org.uk Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=ftp.linux.org.uk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linux.org.uk header.i=@linux.org.uk header.b="taJ0EkwI" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=linux.org.uk; s=zeniv-20220401; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=ZEZWZj1hEX4mTQDSnscf9Mv90kDmsF2fSkjlmj7fzhM=; b=taJ0EkwIxZ9l2Ggw9zKenszibt EMzm+PleqY6JZlblRbzW5OLOuSGXmP6j9QwJJkN0m52Yx5ldiIslv6l3JIfucGbxPZG7ILGOoVaIz T3RvBgkcCvihMPzIE5WHPY86PUsxQkDMVEzRqN4n+l94Pt/9c/B5GLkeDBgf30iVL9s7pwmoFBBAn mvSka+E3rYYgTaHLwUoYSP0utCxE+9cYMMHp9XCZeotQ/7EH2BaiuQHVkdkkHMIBjwOGsGvIlOpOR t5px8sSrP/G4QeCLAS52DxdqfDITywRZxsAc6tNoMPZvAstTHB7ySmhcZDdQ9n7K/oFv0kgj6BQT6 qs+Sziug==; Received: from viro by zeniv.linux.org.uk with local (Exim 4.98 #2 (Red Hat Linux)) id 1tKqIV-00000006lS0-3WWD; Tue, 10 Dec 2024 02:48:27 +0000 From: Al Viro To: Linus Torvalds Cc: linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH 4/5] dissolve external_name.u into separate members Date: Tue, 10 Dec 2024 02:48:26 +0000 Message-ID: <20241210024827.1612355-4-viro@zeniv.linux.org.uk> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241210024827.1612355-1-viro@zeniv.linux.org.uk> References: <20241210024523.GD3387508@ZenIV> <20241210024827.1612355-1-viro@zeniv.linux.org.uk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Sender: Al Viro kept separate from the previous commit to keep the noise separate from actual changes... Signed-off-by: Al Viro --- fs/dcache.c | 26 ++++++++++++-------------- 1 file changed, 12 insertions(+), 14 deletions(-) diff --git a/fs/dcache.c b/fs/dcache.c index ae13e89ce7d7..163bb0953b4f 100644 --- a/fs/dcache.c +++ b/fs/dcache.c @@ -296,10 +296,8 @@ static inline int dentry_cmp(const struct dentry *dentry, const unsigned char *c } struct external_name { - struct { - atomic_t count; // ->count and ->head can't be combined - struct rcu_head head; // see take_dentry_name_snapshot() - } u; + atomic_t count; // ->count and ->head can't be combined + struct rcu_head head; // see take_dentry_name_snapshot() unsigned char name[]; }; @@ -347,11 +345,11 @@ void take_dentry_name_snapshot(struct name_snapshot *name, struct dentry *dentry p = container_of(s, struct external_name, name[0]); name->name.name = s; // get a valid reference - if (unlikely(!atomic_inc_not_zero(&p->u.count))) + if (unlikely(!atomic_inc_not_zero(&p->count))) goto retry; if (read_seqcount_retry(&dentry->d_seq, seq)) { - if (unlikely(atomic_dec_and_test(&p->u.count))) - kfree_rcu(p, u.head); + if (unlikely(atomic_dec_and_test(&p->count))) + kfree_rcu(p, head); goto retry; } } @@ -364,8 +362,8 @@ void release_dentry_name_snapshot(struct name_snapshot *name) if (unlikely(name->name.name != name->inline_name)) { struct external_name *p; p = container_of(name->name.name, struct external_name, name[0]); - if (unlikely(atomic_dec_and_test(&p->u.count))) - kfree_rcu(p, u.head); + if (unlikely(atomic_dec_and_test(&p->count))) + kfree_rcu(p, head); } } EXPORT_SYMBOL(release_dentry_name_snapshot); @@ -403,7 +401,7 @@ static void dentry_free(struct dentry *dentry) WARN_ON(!hlist_unhashed(&dentry->d_u.d_alias)); if (unlikely(dname_external(dentry))) { struct external_name *p = external_name(dentry); - if (likely(atomic_dec_and_test(&p->u.count))) { + if (likely(atomic_dec_and_test(&p->count))) { call_rcu(&dentry->d_u.d_rcu, __d_free_external); return; } @@ -1684,7 +1682,7 @@ static struct dentry *__d_alloc(struct super_block *sb, const struct qstr *name) kmem_cache_free(dentry_cache, dentry); return NULL; } - atomic_set(&p->u.count, 1); + atomic_set(&p->count, 1); dname = p->name; } else { dname = dentry->d_iname; @@ -2777,15 +2775,15 @@ static void copy_name(struct dentry *dentry, struct dentry *target) if (unlikely(dname_external(dentry))) old_name = external_name(dentry); if (unlikely(dname_external(target))) { - atomic_inc(&external_name(target)->u.count); + atomic_inc(&external_name(target)->count); dentry->d_name = target->d_name; } else { dentry->d_iname_words = target->d_iname_words; dentry->d_name.name = dentry->d_iname; dentry->d_name.hash_len = target->d_name.hash_len; } - if (old_name && likely(atomic_dec_and_test(&old_name->u.count))) - kfree_rcu(old_name, u.head); + if (old_name && likely(atomic_dec_and_test(&old_name->count))) + kfree_rcu(old_name, head); } /* From patchwork Tue Dec 10 02:48:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Al Viro X-Patchwork-Id: 13900759 Received: from zeniv.linux.org.uk (zeniv.linux.org.uk [62.89.141.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 71CD21AAA2F; Tue, 10 Dec 2024 02:48:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=62.89.141.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733798912; cv=none; b=gVPgCorLbvFDUeyLpVWGeVvXzclSesawJLomSV2walUczE2z+yx18M+RDaFmrh+Tf3AYGez+bty5CnPALYJoCjykp1+6tvGKUZpjckqgardQ/E9MqNdQAFQCMlLMrS9y0515i2icbcL1af7cOhFyAqq3u20TJ4EGV3ZBqymx6K0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733798912; c=relaxed/simple; bh=0bTdncsjoAki6Q/sVnT/88MBbvQ8ijphl7e3axc+dWs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=mSR3KErjHvFORtkWI4t2IyzWOxsMyb6f4mTvx0pzobgHpZ8Z0cPzTitk0CHdR1ZW8A3j47Empx2ZyhJBra0+VPPHQ9iDg3XY2fSlft+SGbkx2i3KGa8M3ATpA2WoNScrepdS2mKKbXi5Tl8UrToL4gW+wkjtgZ4I52KEdXL1WtQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zeniv.linux.org.uk; spf=none smtp.mailfrom=ftp.linux.org.uk; dkim=pass (2048-bit key) header.d=linux.org.uk header.i=@linux.org.uk header.b=tueO/suz; arc=none smtp.client-ip=62.89.141.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zeniv.linux.org.uk Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=ftp.linux.org.uk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linux.org.uk header.i=@linux.org.uk header.b="tueO/suz" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=linux.org.uk; s=zeniv-20220401; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=OwzwWqtJRF9uqXKKqFbXfz7VMk0w6bXSQMfi/uIOrkE=; b=tueO/suzUI+HmPTrqVP3ZkOVGK Mp3+rjcadl8/tIIOF5RpRJQbs9qvlMMkoYCfCm3St2unYJqJfuSv+3bAw38PUrb5cfXpoJLfKmm0x l5wpupxiitLvCb7NLnNbwgeD9J8H/VebCYnawoTlCQ76tnCQdV2S/pPhwRvdBSnKS7LCF+Sy4oLl8 OZILac96n0d4lNRagN5XScX7/YPX5caZvA4LWavzsStE3WFX+wGaIa2u5H9za4oVq4Yx+DfVGzLoU WhHMQhPlPSkmf8qFaC5hgE2+CqIFOTpTIJ6RhGED4AEecrBXpziHsIkXiY0TB5ajfPvxRE1rBuqxw UIY2J3OQ==; Received: from viro by zeniv.linux.org.uk with local (Exim 4.98 #2 (Red Hat Linux)) id 1tKqIV-00000006lS4-3tac; Tue, 10 Dec 2024 02:48:27 +0000 From: Al Viro To: Linus Torvalds Cc: linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org Subject: [PATCH 5/5] ext4 fast_commit: make use of name_snapshot primitives Date: Tue, 10 Dec 2024 02:48:27 +0000 Message-ID: <20241210024827.1612355-5-viro@zeniv.linux.org.uk> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241210024827.1612355-1-viro@zeniv.linux.org.uk> References: <20241210024523.GD3387508@ZenIV> <20241210024827.1612355-1-viro@zeniv.linux.org.uk> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Sender: Al Viro ... rather than open-coding them (and doing pointless work with extra allocations, etc. for long names). Signed-off-by: Al Viro --- fs/ext4/fast_commit.c | 29 +++++------------------------ fs/ext4/fast_commit.h | 3 +-- 2 files changed, 6 insertions(+), 26 deletions(-) diff --git a/fs/ext4/fast_commit.c b/fs/ext4/fast_commit.c index 26c4fc37edcf..da4263a14a20 100644 --- a/fs/ext4/fast_commit.c +++ b/fs/ext4/fast_commit.c @@ -322,9 +322,7 @@ void ext4_fc_del(struct inode *inode) WARN_ON(!list_empty(&ei->i_fc_dilist)); spin_unlock(&sbi->s_fc_lock); - if (fc_dentry->fcd_name.name && - fc_dentry->fcd_name.len > DNAME_INLINE_LEN) - kfree(fc_dentry->fcd_name.name); + release_dentry_name_snapshot(&fc_dentry->fcd_name); kmem_cache_free(ext4_fc_dentry_cachep, fc_dentry); return; @@ -449,22 +447,7 @@ static int __track_dentry_update(handle_t *handle, struct inode *inode, node->fcd_op = dentry_update->op; node->fcd_parent = dir->i_ino; node->fcd_ino = inode->i_ino; - if (dentry->d_name.len > DNAME_INLINE_LEN) { - node->fcd_name.name = kmalloc(dentry->d_name.len, GFP_NOFS); - if (!node->fcd_name.name) { - kmem_cache_free(ext4_fc_dentry_cachep, node); - ext4_fc_mark_ineligible(sb, EXT4_FC_REASON_NOMEM, handle); - mutex_lock(&ei->i_fc_lock); - return -ENOMEM; - } - memcpy((u8 *)node->fcd_name.name, dentry->d_name.name, - dentry->d_name.len); - } else { - memcpy(node->fcd_iname, dentry->d_name.name, - dentry->d_name.len); - node->fcd_name.name = node->fcd_iname; - } - node->fcd_name.len = dentry->d_name.len; + take_dentry_name_snapshot(&node->fcd_name, dentry); INIT_LIST_HEAD(&node->fcd_dilist); spin_lock(&sbi->s_fc_lock); if (sbi->s_journal->j_flags & JBD2_FULL_COMMIT_ONGOING || @@ -832,7 +815,7 @@ static bool ext4_fc_add_dentry_tlv(struct super_block *sb, u32 *crc, { struct ext4_fc_dentry_info fcd; struct ext4_fc_tl tl; - int dlen = fc_dentry->fcd_name.len; + int dlen = fc_dentry->fcd_name.name.len; u8 *dst = ext4_fc_reserve_space(sb, EXT4_FC_TAG_BASE_LEN + sizeof(fcd) + dlen, crc); @@ -847,7 +830,7 @@ static bool ext4_fc_add_dentry_tlv(struct super_block *sb, u32 *crc, dst += EXT4_FC_TAG_BASE_LEN; memcpy(dst, &fcd, sizeof(fcd)); dst += sizeof(fcd); - memcpy(dst, fc_dentry->fcd_name.name, dlen); + memcpy(dst, fc_dentry->fcd_name.name.name, dlen); return true; } @@ -1328,9 +1311,7 @@ static void ext4_fc_cleanup(journal_t *journal, int full, tid_t tid) list_del_init(&fc_dentry->fcd_dilist); spin_unlock(&sbi->s_fc_lock); - if (fc_dentry->fcd_name.name && - fc_dentry->fcd_name.len > DNAME_INLINE_LEN) - kfree(fc_dentry->fcd_name.name); + release_dentry_name_snapshot(&fc_dentry->fcd_name); kmem_cache_free(ext4_fc_dentry_cachep, fc_dentry); spin_lock(&sbi->s_fc_lock); } diff --git a/fs/ext4/fast_commit.h b/fs/ext4/fast_commit.h index 2fadb2c4780c..3bd534e4dbbf 100644 --- a/fs/ext4/fast_commit.h +++ b/fs/ext4/fast_commit.h @@ -109,8 +109,7 @@ struct ext4_fc_dentry_update { int fcd_op; /* Type of update create / unlink / link */ int fcd_parent; /* Parent inode number */ int fcd_ino; /* Inode number */ - struct qstr fcd_name; /* Dirent name */ - unsigned char fcd_iname[DNAME_INLINE_LEN]; /* Dirent name string */ + struct name_snapshot fcd_name; /* Dirent name */ struct list_head fcd_list; struct list_head fcd_dilist; };