diff mbox

[v2,9/8] don't fool lockdep in freeze_super() and thaw_super() paths

Message ID 20150812131138.GA7462@redhat.com (mailing list archive)
State New, archived
Headers show

Commit Message

Oleg Nesterov Aug. 12, 2015, 1:11 p.m. UTC 
On 08/11, Oleg Nesterov wrote:
>
> The only essential change is that I dropped the lockdep improvements
> as we discussed. This means that 8/8 was changed a bit, and I decided
> to add the new documentation patch, see 3/8.

Update: The recent

	[PATCH 0/2] xfs: kill lockdep false positives from readdir

changes from Dave fixed the problems ILOCK false-positives. So we can
add the additional patch which (modulo comments) just turns v2 back into
v1.

Dave, Jan, you seem to agree with these patches. How should we route
this all?

-------------------------------------------------------------------------------
Subject: [PATCH v2 9/8] don't fool lockdep in freeze_super() and thaw_super() paths

sb_wait_write()->percpu_rwsem_release() fools lockdep to avoid the
false-positives. Now that xfs was fixed by Dave we can remove it and
change freeze_super() and thaw_super() to run with s_writers.rw_sem
locks held; we add two trivial helpers for that, sb_freeze_release()
and sb_freeze_acquire().

While at it, kill the outdated part of the comment above sb_wait_write.

Signed-off-by: Oleg Nesterov <oleg@redhat.com>
---
 fs/super.c |   41 ++++++++++++++++++++++++++---------------
 1 files changed, 26 insertions(+), 15 deletions(-)

Comments

Jan Kara Aug. 13, 2015, 11:01 a.m. UTC | #1 
On Wed 12-08-15 15:11:38, Oleg Nesterov wrote:
> On 08/11, Oleg Nesterov wrote:
> >
> > The only essential change is that I dropped the lockdep improvements
> > as we discussed. This means that 8/8 was changed a bit, and I decided
> > to add the new documentation patch, see 3/8.
> 
> Update: The recent
> 
> 	[PATCH 0/2] xfs: kill lockdep false positives from readdir
> 
> changes from Dave fixed the problems ILOCK false-positives. So we can
> add the additional patch which (modulo comments) just turns v2 back into
> v1.
> 
> Dave, Jan, you seem to agree with these patches. How should we route
> this all?

Regarding the routing, ideally Al Viro should take these as a VFS
maintainer.
 
> -------------------------------------------------------------------------------
> Subject: [PATCH v2 9/8] don't fool lockdep in freeze_super() and thaw_super() paths
> 
> sb_wait_write()->percpu_rwsem_release() fools lockdep to avoid the
> false-positives. Now that xfs was fixed by Dave we can remove it and
> change freeze_super() and thaw_super() to run with s_writers.rw_sem
> locks held; we add two trivial helpers for that, sb_freeze_release()
> and sb_freeze_acquire().
> 
> While at it, kill the outdated part of the comment above sb_wait_write.
> 
> Signed-off-by: Oleg Nesterov <oleg@redhat.com>

The patch looks good. Just one nit:

> +	for (level = SB_FREEZE_LEVELS; --level >= 0; )
> +		percpu_rwsem_release(sb->s_writers.rw_sem + level, 0, _THIS_IP_);

It is more common (and to me more readable) to have the loop written as:

for (level = SB_FREEZE_LEVELS - 1; level >= 0; level--)

I agree what you do is shorter but IMHO it's just an unnecessary
obfuscation :)

Otherwise feel free to add:

Reviewed-by: Jan Kara <jack@suse.com>

								Honza

> +}
> +
> +/*
> + * Tell lockdep we are holding these locks before we call ->unfreeze_fs(sb).
> + */
> +static void sb_freeze_acquire(struct super_block *sb)
>  {
>  	int level;
>  
>  	for (level = 0; level < SB_FREEZE_LEVELS; ++level)
>  		percpu_rwsem_acquire(sb->s_writers.rw_sem + level, 0, _THIS_IP_);
> +}
> +
> +static void sb_freeze_unlock(struct super_block *sb)
> +{
> +	int level;
>  
>  	for (level = SB_FREEZE_LEVELS; --level >= 0; )
>  		percpu_up_write(sb->s_writers.rw_sem + level);
> @@ -1329,6 +1336,7 @@ int freeze_super(struct super_block *sb)
>  	 * sees write activity when frozen is set to SB_FREEZE_COMPLETE.
>  	 */
>  	sb->s_writers.frozen = SB_FREEZE_COMPLETE;
> +	sb_freeze_release(sb);
>  	up_write(&sb->s_umount);
>  	return 0;
>  }
> @@ -1355,11 +1363,14 @@ int thaw_super(struct super_block *sb)
>  		goto out;
>  	}
>  
> +	sb_freeze_acquire(sb);
> +
>  	if (sb->s_op->unfreeze_fs) {
>  		error = sb->s_op->unfreeze_fs(sb);
>  		if (error) {
>  			printk(KERN_ERR
>  				"VFS:Filesystem thaw failed\n");
> +			sb_freeze_release(sb);
>  			up_write(&sb->s_umount);
>  			return error;
>  		}
> -- 
> 1.5.5.1
> 
>
Oleg Nesterov Aug. 13, 2015, 1:58 p.m. UTC | #2 
On 08/13, Jan Kara wrote:
>
> On Wed 12-08-15 15:11:38, Oleg Nesterov wrote:
> > On 08/11, Oleg Nesterov wrote:
> > >
> > > The only essential change is that I dropped the lockdep improvements
> > > as we discussed. This means that 8/8 was changed a bit, and I decided
> > > to add the new documentation patch, see 3/8.
> >
> > Update: The recent
> >
> > 	[PATCH 0/2] xfs: kill lockdep false positives from readdir
> >
> > changes from Dave fixed the problems ILOCK false-positives. So we can
> > add the additional patch which (modulo comments) just turns v2 back into
> > v1.
> >
> > Dave, Jan, you seem to agree with these patches. How should we route
> > this all?
>
> Regarding the routing, ideally Al Viro should take these as a VFS
> maintainer.

OK. I'll send v3.

But to remind, this particular patch depends on Dave's fixes, so I will
send it later.

And I forgot to mention that I have another patch which removes the
trylock hack from __sb_start_write() as Dave suggested, it passed the
tests. But again, I'd really like to send it separately so that it can
be reverted in (unlikely) case something else does recursive read_lock().

> > Subject: [PATCH v2 9/8] don't fool lockdep in freeze_super() and thaw_super() paths
> >
> > sb_wait_write()->percpu_rwsem_release() fools lockdep to avoid the
> > false-positives. Now that xfs was fixed by Dave we can remove it and
> > change freeze_super() and thaw_super() to run with s_writers.rw_sem
> > locks held; we add two trivial helpers for that, sb_freeze_release()
> > and sb_freeze_acquire().
> >
> > While at it, kill the outdated part of the comment above sb_wait_write.
> >
> > Signed-off-by: Oleg Nesterov <oleg@redhat.com>
>
> The patch looks good. Just one nit:
>
> > +	for (level = SB_FREEZE_LEVELS; --level >= 0; )
> > +		percpu_rwsem_release(sb->s_writers.rw_sem + level, 0, _THIS_IP_);
>
> It is more common (and to me more readable) to have the loop written as:
>
> for (level = SB_FREEZE_LEVELS - 1; level >= 0; level--)
>
> I agree what you do is shorter but IMHO it's just an unnecessary
> obfuscation :)

Agreed, will fix.

Oleg.

--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
diff mbox

Patch

diff --git a/fs/super.c b/fs/super.c
index 8762997..91c9756 100644
--- a/fs/super.c
+++ b/fs/super.c
@@ -1208,32 +1208,39 @@  EXPORT_SYMBOL(__sb_start_write);
  * @level: type of writers we wait for (normal vs page fault)
  *
  * This function waits until there are no writers of given type to given file
- * system. Caller of this function should make sure there can be no new writers
- * of type @level before calling this function. Otherwise this function can
- * livelock.
+ * system.
  */
 static void sb_wait_write(struct super_block *sb, int level)
 {
 	percpu_down_write(sb->s_writers.rw_sem + level-1);
-	/*
-	 * We are going to return to userspace and forget about this lock, the
-	 * ownership goes to the caller of thaw_super() which does unlock.
-	 *
-	 * FIXME: we should do this before return from freeze_super() after we
-	 * called sync_filesystem(sb) and s_op->freeze_fs(sb), and thaw_super()
-	 * should re-acquire these locks before s_op->unfreeze_fs(sb). However
-	 * this leads to lockdep false-positives, so currently we do the early
-	 * release right after acquire.
-	 */
-	percpu_rwsem_release(sb->s_writers.rw_sem + level-1, 0, _THIS_IP_);
 }
 
-static void sb_freeze_unlock(struct super_block *sb)
+/*
+ * We are going to return to userspace and forget about these locks, the
+ * ownership goes to the caller of thaw_super() which does unlock().
+ */
+static void sb_freeze_release(struct super_block *sb)
+{
+	int level;
+
+	for (level = SB_FREEZE_LEVELS; --level >= 0; )
+		percpu_rwsem_release(sb->s_writers.rw_sem + level, 0, _THIS_IP_);
+}
+
+/*
+ * Tell lockdep we are holding these locks before we call ->unfreeze_fs(sb).
+ */
+static void sb_freeze_acquire(struct super_block *sb)
 {
 	int level;
 
 	for (level = 0; level < SB_FREEZE_LEVELS; ++level)
 		percpu_rwsem_acquire(sb->s_writers.rw_sem + level, 0, _THIS_IP_);
+}
+
+static void sb_freeze_unlock(struct super_block *sb)
+{
+	int level;
 
 	for (level = SB_FREEZE_LEVELS; --level >= 0; )
 		percpu_up_write(sb->s_writers.rw_sem + level);
@@ -1329,6 +1336,7 @@  int freeze_super(struct super_block *sb)
 	 * sees write activity when frozen is set to SB_FREEZE_COMPLETE.
 	 */
 	sb->s_writers.frozen = SB_FREEZE_COMPLETE;
+	sb_freeze_release(sb);
 	up_write(&sb->s_umount);
 	return 0;
 }
@@ -1355,11 +1363,14 @@  int thaw_super(struct super_block *sb)
 		goto out;
 	}
 
+	sb_freeze_acquire(sb);
+
 	if (sb->s_op->unfreeze_fs) {
 		error = sb->s_op->unfreeze_fs(sb);
 		if (error) {
 			printk(KERN_ERR
 				"VFS:Filesystem thaw failed\n");
+			sb_freeze_release(sb);
 			up_write(&sb->s_umount);
 			return error;
 		}