Message ID | 2512a38027e5286eae01d4aa36726f49a94e523d.1574799173.git.osandov@fb.com (mailing list archive) |
---|---|
State | New, archived |
Headers | show |
Series | generic: test truncating mixed written/unwritten XFS realtime extent | expand |
On Tue, Nov 26, 2019 at 12:13:56PM -0800, Omar Sandoval wrote: > From: Omar Sandoval <osandov@fb.com> > > The only XFS-specific part of this test is the setup, so we can make the > rest a generic test. It's slow, though, as it needs to write 8GB to > convert a big unwritten extent to written. > > Signed-off-by: Omar Sandoval <osandov@fb.com> > --- > tests/generic/586 | 59 +++++++++++++++++++++++++++++++++++++++++++ > tests/generic/586.out | 2 ++ > tests/generic/group | 1 + > 3 files changed, 62 insertions(+) > create mode 100755 tests/generic/586 > create mode 100644 tests/generic/586.out > > diff --git a/tests/generic/586 b/tests/generic/586 > new file mode 100755 > index 00000000..5bcad68b > --- /dev/null > +++ b/tests/generic/586 > @@ -0,0 +1,59 @@ > +#! /bin/bash > +# SPDX-License-Identifier: GPL-2.0 > +# Copyright (c) 2019 Facebook. All Rights Reserved. > +# > +# FS QA Test 586 > +# > +# Test "xfs: fix realtime file data space leak" and "xfs: don't check for AG > +# deadlock for realtime files in bunmapi". On XFS without the fix, rm will hang > +# forever. On other filesystems, this just tests writing into big fallocates. > +# > +seq=`basename $0` > +seqres=$RESULT_DIR/$seq > +echo "QA output created by $seq" > + > +here=`pwd` > +tmp=/tmp/$$ > +status=1 # failure is the default! > +trap "_cleanup; exit \$status" 0 1 2 3 15 > + > +_cleanup() > +{ > + cd / > + rm -f $tmp.* > +} > + > +. ./common/rc > +. ./common/filter > + > +rm -f $seqres.full > + > +_supported_fs generic > +_supported_os Linux > +_require_scratch > + > +maxextlen=$((0x1fffff)) /me wonders if we ought to move this to common/xfs and hoist the other place (xfs/507) where we define this. > +bs=4096 > +rextsize=4 > + > +extra_options="" > +if [[ $FSTYP = xfs && $USE_EXTERNAL = yes && -n $SCRATCH_RTDEV ]]; then > + extra_options="$extra_options -bsize=$bs" Hm. Could you rework this to _require_realtime, and if the caller didn't set SCRATCH_RTDEV, create a file on the test device so that we can always do the realtime test if the kernel supports it? That will ensure that this gets more testing that it does now... > + extra_options="$extra_options -r extsize=$((bs * rextsize))" > + extra_options="$extra_options -d agsize=$(((maxextlen + 1) * bs / 2)),rtinherit=1" ...particularly because I don't think very many people actually run fstests with rt enabled /and/ rtinherit set to stress the realtime allocator. (I did a year ago and out came a torrent of bugs such that someone could probably write themselves a nice year end bonus just fixing all that, software is terrible :() > +fi > +_scratch_mkfs $extra_options >>$seqres.full 2>&1 > +_scratch_mount > +_require_fs_space "$SCRATCH_MNT" "$(((maxextlen + 1) * bs / 1024))" > + > +fallocate -l $(((maxextlen + 1 - rextsize) * bs)) "$SCRATCH_MNT/file" > +sync $XFS_IO_PROG -c "falloc 0 <math expression>" -c fsync $SCRATCH_MNT/file (Hm ok, fallocate the first 2097148 blocks of the file...) > +fallocate -o $(((maxextlen + 1 - rextsize) * bs)) -l $((rextsize * bs)) "$SCRATCH_MNT/file" > +sync $XFS_IO_PROG -c "falloc <the -o expression> <the -l expression>" -c fsync $SCRATCH_MNT/file (now fallocate blocks 2097148 to 2097152) (Not sure why you do the last rtext separately...) > +dd if=/dev/zero of="$SCRATCH_MNT/file" bs=$bs count=$((maxextlen + 2 - rextsize)) conv=notrunc status=none > +sync $XFS_IO_PROG -c "pwrite <count= expression> $bs" -c fsync $SCRATCH_MNT/file (and finally write to block 2097149?) --D > +rm "$SCRATCH_MNT/file" > + > +echo "Silence is golden" > +status=0 > +exit > diff --git a/tests/generic/586.out b/tests/generic/586.out > new file mode 100644 > index 00000000..3d36442d > --- /dev/null > +++ b/tests/generic/586.out > @@ -0,0 +1,2 @@ > +QA output created by 586 > +Silence is golden > diff --git a/tests/generic/group b/tests/generic/group > index e5d0c1da..6ddaf640 100644 > --- a/tests/generic/group > +++ b/tests/generic/group > @@ -588,3 +588,4 @@ > 583 auto quick encrypt > 584 auto quick encrypt > 585 auto rename > +586 auto prealloc preallocrw dangerous > -- > 2.24.0 >
On Tue, Nov 26, 2019 at 05:26:46PM -0800, Darrick J. Wong wrote: > On Tue, Nov 26, 2019 at 12:13:56PM -0800, Omar Sandoval wrote: > > From: Omar Sandoval <osandov@fb.com> > > > > The only XFS-specific part of this test is the setup, so we can make the > > rest a generic test. It's slow, though, as it needs to write 8GB to > > convert a big unwritten extent to written. > > > > Signed-off-by: Omar Sandoval <osandov@fb.com> > > --- > > tests/generic/586 | 59 +++++++++++++++++++++++++++++++++++++++++++ > > tests/generic/586.out | 2 ++ > > tests/generic/group | 1 + > > 3 files changed, 62 insertions(+) > > create mode 100755 tests/generic/586 > > create mode 100644 tests/generic/586.out > > > > diff --git a/tests/generic/586 b/tests/generic/586 > > new file mode 100755 > > index 00000000..5bcad68b > > --- /dev/null > > +++ b/tests/generic/586 > > @@ -0,0 +1,59 @@ > > +#! /bin/bash > > +# SPDX-License-Identifier: GPL-2.0 > > +# Copyright (c) 2019 Facebook. All Rights Reserved. > > +# > > +# FS QA Test 586 > > +# > > +# Test "xfs: fix realtime file data space leak" and "xfs: don't check for AG > > +# deadlock for realtime files in bunmapi". On XFS without the fix, rm will hang > > +# forever. On other filesystems, this just tests writing into big fallocates. > > +# > > +seq=`basename $0` > > +seqres=$RESULT_DIR/$seq > > +echo "QA output created by $seq" > > + > > +here=`pwd` > > +tmp=/tmp/$$ > > +status=1 # failure is the default! > > +trap "_cleanup; exit \$status" 0 1 2 3 15 > > + > > +_cleanup() > > +{ > > + cd / > > + rm -f $tmp.* > > +} > > + > > +. ./common/rc > > +. ./common/filter > > + > > +rm -f $seqres.full > > + > > +_supported_fs generic > > +_supported_os Linux > > +_require_scratch > > + > > +maxextlen=$((0x1fffff)) > > /me wonders if we ought to move this to common/xfs and hoist the other > place (xfs/507) where we define this. > > > +bs=4096 > > +rextsize=4 > > + > > +extra_options="" > > +if [[ $FSTYP = xfs && $USE_EXTERNAL = yes && -n $SCRATCH_RTDEV ]]; then > > + extra_options="$extra_options -bsize=$bs" > > Hm. Could you rework this to _require_realtime, and if the caller > didn't set SCRATCH_RTDEV, create a file on the test device so that we > can always do the realtime test if the kernel supports it? That will > ensure that this gets more testing that it does now... Sorry, I don't follow. _require_realtime checks that SCRATCH_RTDEV was set. Did you mean something like this? if [[ $USE_EXTERNAL = yes && -n $SCRATCH_RTDEV ]]; then use the configured rtdev else _require_test _require_scratch set up a loop device on the test filesystem as the rtdev fi > > + extra_options="$extra_options -r extsize=$((bs * rextsize))" > > + extra_options="$extra_options -d agsize=$(((maxextlen + 1) * bs / 2)),rtinherit=1" > > ...particularly because I don't think very many people actually run > fstests with rt enabled /and/ rtinherit set to stress the realtime > allocator. > > (I did a year ago and out came a torrent of bugs such that someone could > probably write themselves a nice year end bonus just fixing all that, > software is terrible :() > > > +fi > > +_scratch_mkfs $extra_options >>$seqres.full 2>&1 > > +_scratch_mount > > +_require_fs_space "$SCRATCH_MNT" "$(((maxextlen + 1) * bs / 1024))" > > + > > +fallocate -l $(((maxextlen + 1 - rextsize) * bs)) "$SCRATCH_MNT/file" > > +sync > > $XFS_IO_PROG -c "falloc 0 <math expression>" -c fsync $SCRATCH_MNT/file Will fix. > (Hm ok, fallocate the first 2097148 blocks of the file...) Oops, I'll add some explanations for all of this stuff. Thanks for taking a look! > > +fallocate -o $(((maxextlen + 1 - rextsize) * bs)) -l $((rextsize * bs)) "$SCRATCH_MNT/file" > > +sync > > $XFS_IO_PROG -c "falloc <the -o expression> <the -l expression>" -c fsync $SCRATCH_MNT/file > > (now fallocate blocks 2097148 to 2097152) > > (Not sure why you do the last rtext separately...) > > > +dd if=/dev/zero of="$SCRATCH_MNT/file" bs=$bs count=$((maxextlen + 2 - rextsize)) conv=notrunc status=none > > +sync > > $XFS_IO_PROG -c "pwrite <count= expression> $bs" -c fsync $SCRATCH_MNT/file > > (and finally write to block 2097149?) > > --D
On Mon, Dec 02, 2019 at 11:24:24AM -0800, Omar Sandoval wrote: > On Tue, Nov 26, 2019 at 05:26:46PM -0800, Darrick J. Wong wrote: > > On Tue, Nov 26, 2019 at 12:13:56PM -0800, Omar Sandoval wrote: > > > From: Omar Sandoval <osandov@fb.com> > > > > > > The only XFS-specific part of this test is the setup, so we can make the > > > rest a generic test. It's slow, though, as it needs to write 8GB to > > > convert a big unwritten extent to written. > > > > > > Signed-off-by: Omar Sandoval <osandov@fb.com> > > > --- > > > tests/generic/586 | 59 +++++++++++++++++++++++++++++++++++++++++++ > > > tests/generic/586.out | 2 ++ > > > tests/generic/group | 1 + > > > 3 files changed, 62 insertions(+) > > > create mode 100755 tests/generic/586 > > > create mode 100644 tests/generic/586.out > > > > > > diff --git a/tests/generic/586 b/tests/generic/586 > > > new file mode 100755 > > > index 00000000..5bcad68b > > > --- /dev/null > > > +++ b/tests/generic/586 > > > @@ -0,0 +1,59 @@ > > > +#! /bin/bash > > > +# SPDX-License-Identifier: GPL-2.0 > > > +# Copyright (c) 2019 Facebook. All Rights Reserved. > > > +# > > > +# FS QA Test 586 > > > +# > > > +# Test "xfs: fix realtime file data space leak" and "xfs: don't check for AG > > > +# deadlock for realtime files in bunmapi". On XFS without the fix, rm will hang > > > +# forever. On other filesystems, this just tests writing into big fallocates. > > > +# > > > +seq=`basename $0` > > > +seqres=$RESULT_DIR/$seq > > > +echo "QA output created by $seq" > > > + > > > +here=`pwd` > > > +tmp=/tmp/$$ > > > +status=1 # failure is the default! > > > +trap "_cleanup; exit \$status" 0 1 2 3 15 > > > + > > > +_cleanup() > > > +{ > > > + cd / > > > + rm -f $tmp.* > > > +} > > > + > > > +. ./common/rc > > > +. ./common/filter > > > + > > > +rm -f $seqres.full > > > + > > > +_supported_fs generic > > > +_supported_os Linux > > > +_require_scratch > > > + > > > +maxextlen=$((0x1fffff)) > > > > /me wonders if we ought to move this to common/xfs and hoist the other > > place (xfs/507) where we define this. > > > > > +bs=4096 > > > +rextsize=4 > > > + > > > +extra_options="" > > > +if [[ $FSTYP = xfs && $USE_EXTERNAL = yes && -n $SCRATCH_RTDEV ]]; then > > > + extra_options="$extra_options -bsize=$bs" > > > > Hm. Could you rework this to _require_realtime, and if the caller > > didn't set SCRATCH_RTDEV, create a file on the test device so that we > > can always do the realtime test if the kernel supports it? That will > > ensure that this gets more testing that it does now... > > Sorry, I don't follow. _require_realtime checks that SCRATCH_RTDEV was > set. Did you mean something like this? > > if [[ $USE_EXTERNAL = yes && -n $SCRATCH_RTDEV ]]; then > use the configured rtdev > else > _require_test > _require_scratch > set up a loop device on the test filesystem as the rtdev > fi Yep, that's exactly what I meant. --D > > > + extra_options="$extra_options -r extsize=$((bs * rextsize))" > > > + extra_options="$extra_options -d agsize=$(((maxextlen + 1) * bs / 2)),rtinherit=1" > > > > ...particularly because I don't think very many people actually run > > fstests with rt enabled /and/ rtinherit set to stress the realtime > > allocator. > > > > (I did a year ago and out came a torrent of bugs such that someone could > > probably write themselves a nice year end bonus just fixing all that, > > software is terrible :() > > > > > +fi > > > +_scratch_mkfs $extra_options >>$seqres.full 2>&1 > > > +_scratch_mount > > > +_require_fs_space "$SCRATCH_MNT" "$(((maxextlen + 1) * bs / 1024))" > > > + > > > +fallocate -l $(((maxextlen + 1 - rextsize) * bs)) "$SCRATCH_MNT/file" > > > +sync > > > > $XFS_IO_PROG -c "falloc 0 <math expression>" -c fsync $SCRATCH_MNT/file > > Will fix. > > > (Hm ok, fallocate the first 2097148 blocks of the file...) > > Oops, I'll add some explanations for all of this stuff. > > Thanks for taking a look! > > > > +fallocate -o $(((maxextlen + 1 - rextsize) * bs)) -l $((rextsize * bs)) "$SCRATCH_MNT/file" > > > +sync > > > > $XFS_IO_PROG -c "falloc <the -o expression> <the -l expression>" -c fsync $SCRATCH_MNT/file > > > > (now fallocate blocks 2097148 to 2097152) > > > > (Not sure why you do the last rtext separately...) > > > > > +dd if=/dev/zero of="$SCRATCH_MNT/file" bs=$bs count=$((maxextlen + 2 - rextsize)) conv=notrunc status=none > > > +sync > > > > $XFS_IO_PROG -c "pwrite <count= expression> $bs" -c fsync $SCRATCH_MNT/file > > > > (and finally write to block 2097149?) > > > > --D
diff --git a/tests/generic/586 b/tests/generic/586 new file mode 100755 index 00000000..5bcad68b --- /dev/null +++ b/tests/generic/586 @@ -0,0 +1,59 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2019 Facebook. All Rights Reserved. +# +# FS QA Test 586 +# +# Test "xfs: fix realtime file data space leak" and "xfs: don't check for AG +# deadlock for realtime files in bunmapi". On XFS without the fix, rm will hang +# forever. On other filesystems, this just tests writing into big fallocates. +# +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +. ./common/rc +. ./common/filter + +rm -f $seqres.full + +_supported_fs generic +_supported_os Linux +_require_scratch + +maxextlen=$((0x1fffff)) +bs=4096 +rextsize=4 + +extra_options="" +if [[ $FSTYP = xfs && $USE_EXTERNAL = yes && -n $SCRATCH_RTDEV ]]; then + extra_options="$extra_options -bsize=$bs" + extra_options="$extra_options -r extsize=$((bs * rextsize))" + extra_options="$extra_options -d agsize=$(((maxextlen + 1) * bs / 2)),rtinherit=1" +fi +_scratch_mkfs $extra_options >>$seqres.full 2>&1 +_scratch_mount +_require_fs_space "$SCRATCH_MNT" "$(((maxextlen + 1) * bs / 1024))" + +fallocate -l $(((maxextlen + 1 - rextsize) * bs)) "$SCRATCH_MNT/file" +sync +fallocate -o $(((maxextlen + 1 - rextsize) * bs)) -l $((rextsize * bs)) "$SCRATCH_MNT/file" +sync +dd if=/dev/zero of="$SCRATCH_MNT/file" bs=$bs count=$((maxextlen + 2 - rextsize)) conv=notrunc status=none +sync +rm "$SCRATCH_MNT/file" + +echo "Silence is golden" +status=0 +exit diff --git a/tests/generic/586.out b/tests/generic/586.out new file mode 100644 index 00000000..3d36442d --- /dev/null +++ b/tests/generic/586.out @@ -0,0 +1,2 @@ +QA output created by 586 +Silence is golden diff --git a/tests/generic/group b/tests/generic/group index e5d0c1da..6ddaf640 100644 --- a/tests/generic/group +++ b/tests/generic/group @@ -588,3 +588,4 @@ 583 auto quick encrypt 584 auto quick encrypt 585 auto rename +586 auto prealloc preallocrw dangerous