| Message ID | 5e341bd2e900b2ba6e42109ca20e2ababc6f0873.1613508208.git.josef@toxicpanda.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | fstests: test a regression with btrfs extent reference collisions | expand |
On Tue, Feb 16, 2021 at 8:45 PM Josef Bacik <josef@toxicpanda.com> wrote: > > This is a regression test for a problem where we would flip read only if > we reflink'ed enough extents to generate key'ed references, and then got > a hash collision with those references. This is a test for the fix > > btrfs: do not error out if the extent ref hash doesn't match > > and is relatively straightforward, simply generate such a file and > watch for fireworks. > > Signed-off-by: Josef Bacik <josef@toxicpanda.com> > --- > tests/btrfs/231 | 81 +++++++++++++++++++++++++++++++++++++++++++++ > tests/btrfs/231.out | 11 ++++++ > tests/btrfs/group | 1 + > 3 files changed, 93 insertions(+) > create mode 100755 tests/btrfs/231 > create mode 100644 tests/btrfs/231.out > > diff --git a/tests/btrfs/231 b/tests/btrfs/231 > new file mode 100755 > index 00000000..b4787f9f > --- /dev/null > +++ b/tests/btrfs/231 > @@ -0,0 +1,81 @@ > +#! /bin/bash > +# SPDX-License-Identifier: GPL-2.0 > +# Copyright (c) 2021 Josef Bacik. All Rights Reserved. > +# > +# FS QA Test 231 > +# > +# This is a regression test for a problem fixed by > +# > +# btrfs: do not error out if the extent ref hash doesn't match > +# > +# Simply generate a file with a lot of extent references, then reflink in a few > +# offsets that will generate hash collisions, sync and validate we can still > +# write to the file system. > + > +seq=`basename $0` > +seqres=$RESULT_DIR/$seq > +echo "QA output created by $seq" > + > +here=`pwd` > +tmp=/tmp/$$ > +status=1 # failure is the default! > +trap "_cleanup; exit \$status" 0 1 2 3 15 > + > +_cleanup() > +{ > + cd / > + rm -f $tmp.* > +} > + > +# get standard environment, filters and checks > +. ./common/rc > +. ./common/filter > +. ./common/reflink > + > +# remove previous $seqres.full before test > +rm -f $seqres.full > + > +# real QA test starts here > +_supported_fs btrfs > +_require_test > +_require_scratch_reflink > + > +_scratch_mkfs >>$seqres.full 2>&1 > +_scratch_mount > + > +FILE=$SCRATCH_MNT/file > + > +# Create a 1m extent to reflink > +$XFS_IO_PROG -f -c "pwrite 0 1M" -c "fsync" $FILE | _filter_xfs_io > + > +# Generate a bunch of extent references so we're forced to use key'ed extent > +# references. > +offset=2 > +for i in {0..10000} 100 iterations is more than enough to reproduce the bug, and reduces the test runtime from about 50 seconds down to 3 seconds here. Even 50 iterations is enough. > +do > + $XFS_IO_PROG -c "reflink ${FILE} 0 ${offset}M 1M" $FILE \ > + > /dev/null 2>&1 We can remove 2>&1, so we end up failing if for some unexpected reason a reflink fails. Might help one day uncover some bug. > + offset=$(( offset + 2 )) > +done > + > +# Our key is > +# > +# key.objectid = bytenr > +# key.type = BTRFS_EXTENT_DATA_REF_KEY > +# key.offset = hash(tree, inode, offset) > +# > +# The tree id is 5, the inode is 257, and the reflink'ed offset is 0, the below > +# offsets generate a hash collision with that offset. We only need two to > +# collide, but if it's worth doing it's worth overdoing. > +$XFS_IO_PROG -c "reflink ${FILE} 0 17999258914816 1M" $FILE | _filter_xfs_io > +$XFS_IO_PROG -c "reflink ${FILE} 0 35998517829632 1M" $FILE | _filter_xfs_io > +$XFS_IO_PROG -c "reflink ${FILE} 0 53752752058368 1M" $FILE | _filter_xfs_io > + > +# Sync to make sure this works, it'll error out if we abort the transaction, but > +# write a file just to make sure > +$BTRFS_UTIL_PROG filesystem sync $SCRATCH_MNT > +$XFS_IO_PROG -f -c "pwrite 0 1M" $SCRATCH_MNT/write | _filter_xfs_io > + > +# success, all done > +status=0 > +exit > diff --git a/tests/btrfs/231.out b/tests/btrfs/231.out > new file mode 100644 > index 00000000..f52c71ac > --- /dev/null > +++ b/tests/btrfs/231.out > @@ -0,0 +1,11 @@ > +QA output created by 231 > +wrote 1048576/1048576 bytes at offset 0 > +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) > +linked 1048576/1048576 bytes at offset 17999258914816 > +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) > +linked 1048576/1048576 bytes at offset 35998517829632 > +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) > +linked 1048576/1048576 bytes at offset 53752752058368 > +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) > +wrote 1048576/1048576 bytes at offset 0 > +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) > diff --git a/tests/btrfs/group b/tests/btrfs/group > index a7c65983..65cedf7f 100644 > --- a/tests/btrfs/group > +++ b/tests/btrfs/group > @@ -233,3 +233,4 @@ > 228 auto quick volume > 229 auto quick send clone > 230 auto quick qgroup limit > +231 auto We can also have it in the group "clone", and with 100 or 50 iterations the 'quick' group as well. Other than that it looks good. Thanks. > -- > 2.26.2 >
diff --git a/tests/btrfs/231 b/tests/btrfs/231 new file mode 100755 index 00000000..b4787f9f --- /dev/null +++ b/tests/btrfs/231 @@ -0,0 +1,81 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2021 Josef Bacik. All Rights Reserved. +# +# FS QA Test 231 +# +# This is a regression test for a problem fixed by +# +# btrfs: do not error out if the extent ref hash doesn't match +# +# Simply generate a file with a lot of extent references, then reflink in a few +# offsets that will generate hash collisions, sync and validate we can still +# write to the file system. + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter +. ./common/reflink + +# remove previous $seqres.full before test +rm -f $seqres.full + +# real QA test starts here +_supported_fs btrfs +_require_test +_require_scratch_reflink + +_scratch_mkfs >>$seqres.full 2>&1 +_scratch_mount + +FILE=$SCRATCH_MNT/file + +# Create a 1m extent to reflink +$XFS_IO_PROG -f -c "pwrite 0 1M" -c "fsync" $FILE | _filter_xfs_io + +# Generate a bunch of extent references so we're forced to use key'ed extent +# references. +offset=2 +for i in {0..10000} +do + $XFS_IO_PROG -c "reflink ${FILE} 0 ${offset}M 1M" $FILE \ + > /dev/null 2>&1 + offset=$(( offset + 2 )) +done + +# Our key is +# +# key.objectid = bytenr +# key.type = BTRFS_EXTENT_DATA_REF_KEY +# key.offset = hash(tree, inode, offset) +# +# The tree id is 5, the inode is 257, and the reflink'ed offset is 0, the below +# offsets generate a hash collision with that offset. We only need two to +# collide, but if it's worth doing it's worth overdoing. +$XFS_IO_PROG -c "reflink ${FILE} 0 17999258914816 1M" $FILE | _filter_xfs_io +$XFS_IO_PROG -c "reflink ${FILE} 0 35998517829632 1M" $FILE | _filter_xfs_io +$XFS_IO_PROG -c "reflink ${FILE} 0 53752752058368 1M" $FILE | _filter_xfs_io + +# Sync to make sure this works, it'll error out if we abort the transaction, but +# write a file just to make sure +$BTRFS_UTIL_PROG filesystem sync $SCRATCH_MNT +$XFS_IO_PROG -f -c "pwrite 0 1M" $SCRATCH_MNT/write | _filter_xfs_io + +# success, all done +status=0 +exit diff --git a/tests/btrfs/231.out b/tests/btrfs/231.out new file mode 100644 index 00000000..f52c71ac --- /dev/null +++ b/tests/btrfs/231.out @@ -0,0 +1,11 @@ +QA output created by 231 +wrote 1048576/1048576 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +linked 1048576/1048576 bytes at offset 17999258914816 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +linked 1048576/1048576 bytes at offset 35998517829632 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +linked 1048576/1048576 bytes at offset 53752752058368 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 1048576/1048576 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) diff --git a/tests/btrfs/group b/tests/btrfs/group index a7c65983..65cedf7f 100644 --- a/tests/btrfs/group +++ b/tests/btrfs/group @@ -233,3 +233,4 @@ 228 auto quick volume 229 auto quick send clone 230 auto quick qgroup limit +231 auto
This is a regression test for a problem where we would flip read only if we reflink'ed enough extents to generate key'ed references, and then got a hash collision with those references. This is a test for the fix btrfs: do not error out if the extent ref hash doesn't match and is relatively straightforward, simply generate such a file and watch for fireworks. Signed-off-by: Josef Bacik <josef@toxicpanda.com> --- tests/btrfs/231 | 81 +++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/231.out | 11 ++++++ tests/btrfs/group | 1 + 3 files changed, 93 insertions(+) create mode 100755 tests/btrfs/231 create mode 100644 tests/btrfs/231.out