diff mbox series

[v3] generic/707: Test moving directory while being grown

Message ID 20230131125416.16120-1-jack@suse.cz (mailing list archive)
State New, archived
Headers show
Series [v3] generic/707: Test moving directory while being grown | expand

Commit Message

Jan Kara Jan. 31, 2023, 12:54 p.m. UTC 
Test how the filesystem can handle moving a directory to a different
directory (so that parent pointer gets updated) while it is grown. Ext4
and UDF had a bug where if the directory got converted to a different
type due to growth while rename is running, the filesystem got
corrupted.

Reviewed-by: Bill O'Donnell <bodonnel@redhat.com>
Signed-off-by: Jan Kara <jack@suse.cz>
---
 tests/generic/707     | 70 +++++++++++++++++++++++++++++++++++++++++++
 tests/generic/707.out |  2 ++
 2 files changed, 72 insertions(+)
 create mode 100755 tests/generic/707
 create mode 100644 tests/generic/707.out

Comments

Zorro Lang Jan. 31, 2023, 2:25 p.m. UTC | #1 
On Tue, Jan 31, 2023 at 01:54:16PM +0100, Jan Kara wrote:
> Test how the filesystem can handle moving a directory to a different
> directory (so that parent pointer gets updated) while it is grown. Ext4
> and UDF had a bug where if the directory got converted to a different
> type due to growth while rename is running, the filesystem got
> corrupted.
> 
> Reviewed-by: Bill O'Donnell <bodonnel@redhat.com>
> Signed-off-by: Jan Kara <jack@suse.cz>
> ---

Thanks, this version looks good to me,

Reviewed-by: Zorro Lang <zlang@redhat.com>

>  tests/generic/707     | 70 +++++++++++++++++++++++++++++++++++++++++++
>  tests/generic/707.out |  2 ++
>  2 files changed, 72 insertions(+)
>  create mode 100755 tests/generic/707
>  create mode 100644 tests/generic/707.out
> 
> diff --git a/tests/generic/707 b/tests/generic/707
> new file mode 100755
> index 000000000000..ad1592a1b957
> --- /dev/null
> +++ b/tests/generic/707
> @@ -0,0 +1,70 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2023 Jan Kara, SUSE Linux.  All Rights Reserved.
> +#
> +# FS QA Test 707
> +#
> +# This is a test verifying whether the filesystem can gracefully handle
> +# modifying of a directory while it is being moved, in particular the cases
> +# where directory format changes
> +#
> +. ./common/preamble
> +_begin_fstest auto
> +
> +_supported_fs generic
> +_require_scratch
> +
> +_fixed_by_kernel_commit f950fd052913 \
> +	"udf: Protect rename against modification of moved directory"
> +_fixed_by_kernel_commit XXXXXXXXXXXX \
> +	"ext4: fix possible corruption when moving a directory"
> +
> +_scratch_mkfs >>$seqres.full 2>&1
> +_scratch_mount
> +
> +_cleanup()
> +{
> +	cd /
> +	rm -f $tmp.*
> +	if [ -n "$BGPID" ]; then
> +		# Stop background process
> +		kill -9 $BGPID &>/dev/null
> +		wait
> +	fi
> +}
> +
> +# Loop multiple times trying to hit the race
> +loops=$((100*TIME_FACTOR))
> +files=500
> +moves=500
> +
> +create_files()
> +{
> +	# We use slightly longer file name to make directory grow faster and
> +	# hopefully convert between various types
> +	for (( i = 0; i < $files; i++ )); do
> +		touch somewhatlongerfilename$i
> +	done
> +}
> +
> +for (( i = 0; i <= $moves; i++ )); do
> +	mkdir $SCRATCH_MNT/dir$i
> +done
> +
> +for (( l = 0; l < $loops; l++ )); do
> +	mkdir $SCRATCH_MNT/dir0/dir
> +	pushd $SCRATCH_MNT/dir0/dir &>/dev/null
> +	create_files &
> +	BGPID=$!
> +	popd &>/dev/null
> +	for (( i = 0; i < $moves; i++ )); do
> +		mv $SCRATCH_MNT/dir$i/dir $SCRATCH_MNT/dir$((i+1))/dir
> +	done
> +	wait
> +	unset BGPID
> +	rm -r $SCRATCH_MNT/dir$moves/dir
> +done
> +
> +echo "Silence is golden"
> +status=0
> +exit
> diff --git a/tests/generic/707.out b/tests/generic/707.out
> new file mode 100644
> index 000000000000..8e57a1d8c971
> --- /dev/null
> +++ b/tests/generic/707.out
> @@ -0,0 +1,2 @@
> +QA output created by 707
> +Silence is golden
> -- 
> 2.35.3
>
Jan Kara Oct. 12, 2023, 2:09 p.m. UTC | #2 
On Tue 31-01-23 22:25:44, Zorro Lang wrote:
> On Tue, Jan 31, 2023 at 01:54:16PM +0100, Jan Kara wrote:
> > Test how the filesystem can handle moving a directory to a different
> > directory (so that parent pointer gets updated) while it is grown. Ext4
> > and UDF had a bug where if the directory got converted to a different
> > type due to growth while rename is running, the filesystem got
> > corrupted.
> > 
> > Reviewed-by: Bill O'Donnell <bodonnel@redhat.com>
> > Signed-off-by: Jan Kara <jack@suse.cz>
> > ---
> 
> Thanks, this version looks good to me,
> 
> Reviewed-by: Zorro Lang <zlang@redhat.com>

Guys, I have just noticed this patch didn't get merged. Can you please pick
it up? Thanks!

								Honza

> 
> >  tests/generic/707     | 70 +++++++++++++++++++++++++++++++++++++++++++
> >  tests/generic/707.out |  2 ++
> >  2 files changed, 72 insertions(+)
> >  create mode 100755 tests/generic/707
> >  create mode 100644 tests/generic/707.out
> > 
> > diff --git a/tests/generic/707 b/tests/generic/707
> > new file mode 100755
> > index 000000000000..ad1592a1b957
> > --- /dev/null
> > +++ b/tests/generic/707
> > @@ -0,0 +1,70 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0
> > +# Copyright (c) 2023 Jan Kara, SUSE Linux.  All Rights Reserved.
> > +#
> > +# FS QA Test 707
> > +#
> > +# This is a test verifying whether the filesystem can gracefully handle
> > +# modifying of a directory while it is being moved, in particular the cases
> > +# where directory format changes
> > +#
> > +. ./common/preamble
> > +_begin_fstest auto
> > +
> > +_supported_fs generic
> > +_require_scratch
> > +
> > +_fixed_by_kernel_commit f950fd052913 \
> > +	"udf: Protect rename against modification of moved directory"
> > +_fixed_by_kernel_commit XXXXXXXXXXXX \
> > +	"ext4: fix possible corruption when moving a directory"
> > +
> > +_scratch_mkfs >>$seqres.full 2>&1
> > +_scratch_mount
> > +
> > +_cleanup()
> > +{
> > +	cd /
> > +	rm -f $tmp.*
> > +	if [ -n "$BGPID" ]; then
> > +		# Stop background process
> > +		kill -9 $BGPID &>/dev/null
> > +		wait
> > +	fi
> > +}
> > +
> > +# Loop multiple times trying to hit the race
> > +loops=$((100*TIME_FACTOR))
> > +files=500
> > +moves=500
> > +
> > +create_files()
> > +{
> > +	# We use slightly longer file name to make directory grow faster and
> > +	# hopefully convert between various types
> > +	for (( i = 0; i < $files; i++ )); do
> > +		touch somewhatlongerfilename$i
> > +	done
> > +}
> > +
> > +for (( i = 0; i <= $moves; i++ )); do
> > +	mkdir $SCRATCH_MNT/dir$i
> > +done
> > +
> > +for (( l = 0; l < $loops; l++ )); do
> > +	mkdir $SCRATCH_MNT/dir0/dir
> > +	pushd $SCRATCH_MNT/dir0/dir &>/dev/null
> > +	create_files &
> > +	BGPID=$!
> > +	popd &>/dev/null
> > +	for (( i = 0; i < $moves; i++ )); do
> > +		mv $SCRATCH_MNT/dir$i/dir $SCRATCH_MNT/dir$((i+1))/dir
> > +	done
> > +	wait
> > +	unset BGPID
> > +	rm -r $SCRATCH_MNT/dir$moves/dir
> > +done
> > +
> > +echo "Silence is golden"
> > +status=0
> > +exit
> > diff --git a/tests/generic/707.out b/tests/generic/707.out
> > new file mode 100644
> > index 000000000000..8e57a1d8c971
> > --- /dev/null
> > +++ b/tests/generic/707.out
> > @@ -0,0 +1,2 @@
> > +QA output created by 707
> > +Silence is golden
> > -- 
> > 2.35.3
> > 
>
Zorro Lang Oct. 18, 2023, 2:12 p.m. UTC | #3 
On Thu, Oct 12, 2023 at 04:09:03PM +0200, Jan Kara wrote:
> On Tue 31-01-23 22:25:44, Zorro Lang wrote:
> > On Tue, Jan 31, 2023 at 01:54:16PM +0100, Jan Kara wrote:
> > > Test how the filesystem can handle moving a directory to a different
> > > directory (so that parent pointer gets updated) while it is grown. Ext4
> > > and UDF had a bug where if the directory got converted to a different
> > > type due to growth while rename is running, the filesystem got
> > > corrupted.
> > > 
> > > Reviewed-by: Bill O'Donnell <bodonnel@redhat.com>
> > > Signed-off-by: Jan Kara <jack@suse.cz>
> > > ---
> > 
> > Thanks, this version looks good to me,
> > 
> > Reviewed-by: Zorro Lang <zlang@redhat.com>
> 
> Guys, I have just noticed this patch didn't get merged. Can you please pick
> it up? Thanks!

Hi Jan, I think it has been in fstests v2023.02.05 release, as:

commit da69ce70dfcdbd88d6f940d7c1ab346d5a1e5f82
Author: Jan Kara <jack@suse.cz>
Date:   Tue Jan 31 13:54:16 2023 +0100

    generic/707: Test moving directory while being grown

Isn't it same with this patch? Please check, and feel free to tell me
if anything wrong.

Thanks,
Zorro

> 
> 								Honza
> 
> > 
> > >  tests/generic/707     | 70 +++++++++++++++++++++++++++++++++++++++++++
> > >  tests/generic/707.out |  2 ++
> > >  2 files changed, 72 insertions(+)
> > >  create mode 100755 tests/generic/707
> > >  create mode 100644 tests/generic/707.out
> > > 
> > > diff --git a/tests/generic/707 b/tests/generic/707
> > > new file mode 100755
> > > index 000000000000..ad1592a1b957
> > > --- /dev/null
> > > +++ b/tests/generic/707
> > > @@ -0,0 +1,70 @@
> > > +#! /bin/bash
> > > +# SPDX-License-Identifier: GPL-2.0
> > > +# Copyright (c) 2023 Jan Kara, SUSE Linux.  All Rights Reserved.
> > > +#
> > > +# FS QA Test 707
> > > +#
> > > +# This is a test verifying whether the filesystem can gracefully handle
> > > +# modifying of a directory while it is being moved, in particular the cases
> > > +# where directory format changes
> > > +#
> > > +. ./common/preamble
> > > +_begin_fstest auto
> > > +
> > > +_supported_fs generic
> > > +_require_scratch
> > > +
> > > +_fixed_by_kernel_commit f950fd052913 \
> > > +	"udf: Protect rename against modification of moved directory"
> > > +_fixed_by_kernel_commit XXXXXXXXXXXX \
> > > +	"ext4: fix possible corruption when moving a directory"
> > > +
> > > +_scratch_mkfs >>$seqres.full 2>&1
> > > +_scratch_mount
> > > +
> > > +_cleanup()
> > > +{
> > > +	cd /
> > > +	rm -f $tmp.*
> > > +	if [ -n "$BGPID" ]; then
> > > +		# Stop background process
> > > +		kill -9 $BGPID &>/dev/null
> > > +		wait
> > > +	fi
> > > +}
> > > +
> > > +# Loop multiple times trying to hit the race
> > > +loops=$((100*TIME_FACTOR))
> > > +files=500
> > > +moves=500
> > > +
> > > +create_files()
> > > +{
> > > +	# We use slightly longer file name to make directory grow faster and
> > > +	# hopefully convert between various types
> > > +	for (( i = 0; i < $files; i++ )); do
> > > +		touch somewhatlongerfilename$i
> > > +	done
> > > +}
> > > +
> > > +for (( i = 0; i <= $moves; i++ )); do
> > > +	mkdir $SCRATCH_MNT/dir$i
> > > +done
> > > +
> > > +for (( l = 0; l < $loops; l++ )); do
> > > +	mkdir $SCRATCH_MNT/dir0/dir
> > > +	pushd $SCRATCH_MNT/dir0/dir &>/dev/null
> > > +	create_files &
> > > +	BGPID=$!
> > > +	popd &>/dev/null
> > > +	for (( i = 0; i < $moves; i++ )); do
> > > +		mv $SCRATCH_MNT/dir$i/dir $SCRATCH_MNT/dir$((i+1))/dir
> > > +	done
> > > +	wait
> > > +	unset BGPID
> > > +	rm -r $SCRATCH_MNT/dir$moves/dir
> > > +done
> > > +
> > > +echo "Silence is golden"
> > > +status=0
> > > +exit
> > > diff --git a/tests/generic/707.out b/tests/generic/707.out
> > > new file mode 100644
> > > index 000000000000..8e57a1d8c971
> > > --- /dev/null
> > > +++ b/tests/generic/707.out
> > > @@ -0,0 +1,2 @@
> > > +QA output created by 707
> > > +Silence is golden
> > > -- 
> > > 2.35.3
> > > 
> > 
> -- 
> Jan Kara <jack@suse.com>
> SUSE Labs, CR
>
Jan Kara Oct. 18, 2023, 6:55 p.m. UTC | #4 
On Wed 18-10-23 22:12:37, Zorro Lang wrote:
> On Thu, Oct 12, 2023 at 04:09:03PM +0200, Jan Kara wrote:
> > On Tue 31-01-23 22:25:44, Zorro Lang wrote:
> > > On Tue, Jan 31, 2023 at 01:54:16PM +0100, Jan Kara wrote:
> > > > Test how the filesystem can handle moving a directory to a different
> > > > directory (so that parent pointer gets updated) while it is grown. Ext4
> > > > and UDF had a bug where if the directory got converted to a different
> > > > type due to growth while rename is running, the filesystem got
> > > > corrupted.
> > > > 
> > > > Reviewed-by: Bill O'Donnell <bodonnel@redhat.com>
> > > > Signed-off-by: Jan Kara <jack@suse.cz>
> > > > ---
> > > 
> > > Thanks, this version looks good to me,
> > > 
> > > Reviewed-by: Zorro Lang <zlang@redhat.com>
> > 
> > Guys, I have just noticed this patch didn't get merged. Can you please pick
> > it up? Thanks!
> 
> Hi Jan, I think it has been in fstests v2023.02.05 release, as:
> 
> commit da69ce70dfcdbd88d6f940d7c1ab346d5a1e5f82
> Author: Jan Kara <jack@suse.cz>
> Date:   Tue Jan 31 13:54:16 2023 +0100
> 
>     generic/707: Test moving directory while being grown
> 
> Isn't it same with this patch? Please check, and feel free to tell me
> if anything wrong.

Bah, sorry. Not sure why I didn't find it when grepping :-|. I'm sorry for
the noise.

								Honza
Zorro Lang Oct. 19, 2023, 5:50 a.m. UTC | #5 
On Wed, Oct 18, 2023 at 08:55:30PM +0200, Jan Kara wrote:
> On Wed 18-10-23 22:12:37, Zorro Lang wrote:
> > On Thu, Oct 12, 2023 at 04:09:03PM +0200, Jan Kara wrote:
> > > On Tue 31-01-23 22:25:44, Zorro Lang wrote:
> > > > On Tue, Jan 31, 2023 at 01:54:16PM +0100, Jan Kara wrote:
> > > > > Test how the filesystem can handle moving a directory to a different
> > > > > directory (so that parent pointer gets updated) while it is grown. Ext4
> > > > > and UDF had a bug where if the directory got converted to a different
> > > > > type due to growth while rename is running, the filesystem got
> > > > > corrupted.
> > > > > 
> > > > > Reviewed-by: Bill O'Donnell <bodonnel@redhat.com>
> > > > > Signed-off-by: Jan Kara <jack@suse.cz>
> > > > > ---
> > > > 
> > > > Thanks, this version looks good to me,
> > > > 
> > > > Reviewed-by: Zorro Lang <zlang@redhat.com>
> > > 
> > > Guys, I have just noticed this patch didn't get merged. Can you please pick
> > > it up? Thanks!
> > 
> > Hi Jan, I think it has been in fstests v2023.02.05 release, as:
> > 
> > commit da69ce70dfcdbd88d6f940d7c1ab346d5a1e5f82
> > Author: Jan Kara <jack@suse.cz>
> > Date:   Tue Jan 31 13:54:16 2023 +0100
> > 
> >     generic/707: Test moving directory while being grown
> > 
> > Isn't it same with this patch? Please check, and feel free to tell me
> > if anything wrong.
> 
> Bah, sorry. Not sure why I didn't find it when grepping :-|. I'm sorry for
> the noise.

No problem, Feel free to ping me if any of your patches get missed :)

> 
> 								Honza
> -- 
> Jan Kara <jack@suse.com>
> SUSE Labs, CR
>
diff mbox series

Patch

diff --git a/tests/generic/707 b/tests/generic/707
new file mode 100755
index 000000000000..ad1592a1b957
--- /dev/null
+++ b/tests/generic/707
@@ -0,0 +1,70 @@ 
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2023 Jan Kara, SUSE Linux.  All Rights Reserved.
+#
+# FS QA Test 707
+#
+# This is a test verifying whether the filesystem can gracefully handle
+# modifying of a directory while it is being moved, in particular the cases
+# where directory format changes
+#
+. ./common/preamble
+_begin_fstest auto
+
+_supported_fs generic
+_require_scratch
+
+_fixed_by_kernel_commit f950fd052913 \
+	"udf: Protect rename against modification of moved directory"
+_fixed_by_kernel_commit XXXXXXXXXXXX \
+	"ext4: fix possible corruption when moving a directory"
+
+_scratch_mkfs >>$seqres.full 2>&1
+_scratch_mount
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+	if [ -n "$BGPID" ]; then
+		# Stop background process
+		kill -9 $BGPID &>/dev/null
+		wait
+	fi
+}
+
+# Loop multiple times trying to hit the race
+loops=$((100*TIME_FACTOR))
+files=500
+moves=500
+
+create_files()
+{
+	# We use slightly longer file name to make directory grow faster and
+	# hopefully convert between various types
+	for (( i = 0; i < $files; i++ )); do
+		touch somewhatlongerfilename$i
+	done
+}
+
+for (( i = 0; i <= $moves; i++ )); do
+	mkdir $SCRATCH_MNT/dir$i
+done
+
+for (( l = 0; l < $loops; l++ )); do
+	mkdir $SCRATCH_MNT/dir0/dir
+	pushd $SCRATCH_MNT/dir0/dir &>/dev/null
+	create_files &
+	BGPID=$!
+	popd &>/dev/null
+	for (( i = 0; i < $moves; i++ )); do
+		mv $SCRATCH_MNT/dir$i/dir $SCRATCH_MNT/dir$((i+1))/dir
+	done
+	wait
+	unset BGPID
+	rm -r $SCRATCH_MNT/dir$moves/dir
+done
+
+echo "Silence is golden"
+status=0
+exit
diff --git a/tests/generic/707.out b/tests/generic/707.out
new file mode 100644
index 000000000000..8e57a1d8c971
--- /dev/null
+++ b/tests/generic/707.out
@@ -0,0 +1,2 @@ 
+QA output created by 707
+Silence is golden