| Message ID | e8be02ae6e5495a029e1345df8b66139042a3c72.1665150613.git.fdmanana@suse.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | fstests: add a btrfs fiemap test and fiemap test group | expand |
On Fri, Oct 07, 2022 at 02:53:34PM +0100, fdmanana@kernel.org wrote: > From: Filipe Manana <fdmanana@suse.com> > > Verify that fiemap correctly reports the sharedness of extents for a file > with a very large number of extents, spanning many b+tree leaves in the fs > tree, and when the file's subvolume was snapshoted. > > Currently this passes on all kernel releases and its purpose is to prevent > and detect regressions in the future, as this actually happened during > recent development on the btrfs' fiemap related code. With this test we > now have better coverage for fiemap when a file is shared through a > snapshot. > > Signed-off-by: Filipe Manana <fdmanana@suse.com> > --- Looks good to me, Reviewed-by: Zorro Lang <zlang@redhat.com> > tests/btrfs/276 | 124 ++++++++++++++++++++++++++++++++++++++++++++ > tests/btrfs/276.out | 16 ++++++ > 2 files changed, 140 insertions(+) > create mode 100755 tests/btrfs/276 > create mode 100644 tests/btrfs/276.out > > diff --git a/tests/btrfs/276 b/tests/btrfs/276 > new file mode 100755 > index 00000000..c27e8383 > --- /dev/null > +++ b/tests/btrfs/276 > @@ -0,0 +1,124 @@ > +#! /bin/bash > +# SPDX-License-Identifier: GPL-2.0 > +# Copyright (C) 2022 SUSE Linux Products GmbH. All Rights Reserved. > +# > +# FS QA Test 276 > +# > +# Verify that fiemap correctly reports the sharedness of extents for a file with > +# a very large number of extents, spanning many b+tree leaves in the fs tree, > +# and when the file's subvolume was snapshoted. > +# > +. ./common/preamble > +_begin_fstest auto snapshot compress > + > +. ./common/filter > + > +_supported_fs btrfs > +_require_scratch > +_require_xfs_io_command "fiemap" "ranged" > + > +_scratch_mkfs >> $seqres.full 2>&1 > +# We use compression because it's a very quick way to create a file with a very > +# large number of extents (compression limits the maximum extent size to 128K) > +# and while using very little disk space. > +_scratch_mount -o compress > + > +fiemap_test_file() > +{ > + local offset=$1 > + local len=$2 > + > + # Skip the first two lines of xfs_io's fiemap output (file path and > + # header describing the output columns). > + $XFS_IO_PROG -c "fiemap -v $offset $len" $SCRATCH_MNT/foo | tail -n +3 > +} > + > +# Count the number of shared extents for the whole test file or just for a given > +# range. > +count_shared_extents() > +{ > + local offset=$1 > + local len=$2 > + > + # Column 5 (from xfs_io's "fiemap -v" command) is the flags (hex field). > + # 0x2000 is the value for the FIEMAP_EXTENT_SHARED flag. > + fiemap_test_file $offset $len | \ > + $AWK_PROG --source 'BEGIN { cnt = 0 }' \ > + --source '{ if (and(strtonum($5), 0x2000)) cnt++ }' \ > + --source 'END { print cnt }' > +} > + > +# Count the number of non shared extents for the whole test file or just for a > +# given range. > +count_not_shared_extents() > +{ > + local offset=$1 > + local len=$2 > + > + # Column 5 (from xfs_io's "fiemap -v" command) is the flags (hex field). > + # 0x2000 is the value for the FIEMAP_EXTENT_SHARED flag. > + fiemap_test_file $offset $len | \ > + $AWK_PROG --source 'BEGIN { cnt = 0 }' \ > + --source '{ if (!and(strtonum($5), 0x2000)) cnt++ }' \ > + --source 'END { print cnt }' > +} > + > +# Create a 16G file as that results in 131072 extents, all with a size of 128K > +# (due to compression), and a fs tree with a height of 3 (root node at level 2). > +# We want to verify later that fiemap correctly reports the sharedness of each > +# extent, even when it needs to switch from one leaf to the next one and from a > +# node at level 1 to the next node at level 1. > +# > +$XFS_IO_PROG -f -c "pwrite -b 8M 0 16G" $SCRATCH_MNT/foo | _filter_xfs_io > + > +# Sync to flush delalloc and commit the current transaction, so fiemap will see > +# all extents in the fs tree and extent trees and not look at delalloc. > +sync > + > +# All extents should be reported as non shared (131072 extents). > +echo "Number of non-shared extents in the whole file: $(count_not_shared_extents)" > + > +# Creating a snapshot. > +$BTRFS_UTIL_PROG subvolume snapshot $SCRATCH_MNT $SCRATCH_MNT/snap | _filter_scratch > + > +# We have a snapshot, so now all extents should be reported as shared. > +echo "Number of shared extents in the whole file: $(count_shared_extents)" > + > +# Now COW two file ranges, of 1M each, in the snapshot's file. > +# So 16 extents should become non-shared after this. > +# > +$XFS_IO_PROG -c "pwrite -b 1M 8M 1M" \ > + -c "pwrite -b 1M 12G 1M" \ > + $SCRATCH_MNT/snap/foo | _filter_xfs_io > + > +# Sync to flush delalloc and commit the current transaction, so fiemap will see > +# all extents in the fs tree and extent trees and not look at delalloc. > +sync > + > +# Now we should have 16 non-shared extents and 131056 (131072 - 16) shared > +# extents. > +echo "Number of non-shared extents in the whole file: $(count_not_shared_extents)" > +echo "Number of shared extents in the whole file: $(count_shared_extents)" > + > +# Check that the non-shared extents are indeed in the expected file ranges (each > +# with 8 extents). > +echo "Number of non-shared extents in range [8M, 9M): $(count_not_shared_extents 8M 1M)" > +echo "Number of non-shared extents in range [12G, 12G + 1M): $(count_not_shared_extents 12G 1M)" > + > +# Now delete the snapshot. > +$BTRFS_UTIL_PROG subvolume delete -c $SCRATCH_MNT/snap | _filter_scratch > + > +# We deleted the snapshot and committed the transaction used to delete it (-c), > +# but all its extents (both metadata and data) are actually only deleted in the > +# background, by the cleaner kthread. So remount, which wakes up the cleaner > +# kthread, with a commit interval of 1 second and sleep for 1.1 seconds - after > +# this we are guaranteed all extents of the snapshot were deleted. > +_scratch_remount commit=1 > +sleep 1.1 > + > +# Now all extents should be reported as not shared (131072 extents). > +echo "Number of non-shared extents in the whole file: $(count_not_shared_extents)" > + > +# success, all done > +status=0 > +exit > diff --git a/tests/btrfs/276.out b/tests/btrfs/276.out > new file mode 100644 > index 00000000..3bf5a5e6 > --- /dev/null > +++ b/tests/btrfs/276.out > @@ -0,0 +1,16 @@ > +QA output created by 276 > +wrote 17179869184/17179869184 bytes at offset 0 > +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) > +Number of non-shared extents in the whole file: 131072 > +Create a snapshot of 'SCRATCH_MNT' in 'SCRATCH_MNT/snap' > +Number of shared extents in the whole file: 131072 > +wrote 1048576/1048576 bytes at offset 8388608 > +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) > +wrote 1048576/1048576 bytes at offset 12884901888 > +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) > +Number of non-shared extents in the whole file: 16 > +Number of shared extents in the whole file: 131056 > +Number of non-shared extents in range [8M, 9M): 8 > +Number of non-shared extents in range [12G, 12G + 1M): 8 > +Delete subvolume (commit): 'SCRATCH_MNT/snap' > +Number of non-shared extents in the whole file: 131072 > -- > 2.35.1 >
diff --git a/tests/btrfs/276 b/tests/btrfs/276 new file mode 100755 index 00000000..c27e8383 --- /dev/null +++ b/tests/btrfs/276 @@ -0,0 +1,124 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) 2022 SUSE Linux Products GmbH. All Rights Reserved. +# +# FS QA Test 276 +# +# Verify that fiemap correctly reports the sharedness of extents for a file with +# a very large number of extents, spanning many b+tree leaves in the fs tree, +# and when the file's subvolume was snapshoted. +# +. ./common/preamble +_begin_fstest auto snapshot compress + +. ./common/filter + +_supported_fs btrfs +_require_scratch +_require_xfs_io_command "fiemap" "ranged" + +_scratch_mkfs >> $seqres.full 2>&1 +# We use compression because it's a very quick way to create a file with a very +# large number of extents (compression limits the maximum extent size to 128K) +# and while using very little disk space. +_scratch_mount -o compress + +fiemap_test_file() +{ + local offset=$1 + local len=$2 + + # Skip the first two lines of xfs_io's fiemap output (file path and + # header describing the output columns). + $XFS_IO_PROG -c "fiemap -v $offset $len" $SCRATCH_MNT/foo | tail -n +3 +} + +# Count the number of shared extents for the whole test file or just for a given +# range. +count_shared_extents() +{ + local offset=$1 + local len=$2 + + # Column 5 (from xfs_io's "fiemap -v" command) is the flags (hex field). + # 0x2000 is the value for the FIEMAP_EXTENT_SHARED flag. + fiemap_test_file $offset $len | \ + $AWK_PROG --source 'BEGIN { cnt = 0 }' \ + --source '{ if (and(strtonum($5), 0x2000)) cnt++ }' \ + --source 'END { print cnt }' +} + +# Count the number of non shared extents for the whole test file or just for a +# given range. +count_not_shared_extents() +{ + local offset=$1 + local len=$2 + + # Column 5 (from xfs_io's "fiemap -v" command) is the flags (hex field). + # 0x2000 is the value for the FIEMAP_EXTENT_SHARED flag. + fiemap_test_file $offset $len | \ + $AWK_PROG --source 'BEGIN { cnt = 0 }' \ + --source '{ if (!and(strtonum($5), 0x2000)) cnt++ }' \ + --source 'END { print cnt }' +} + +# Create a 16G file as that results in 131072 extents, all with a size of 128K +# (due to compression), and a fs tree with a height of 3 (root node at level 2). +# We want to verify later that fiemap correctly reports the sharedness of each +# extent, even when it needs to switch from one leaf to the next one and from a +# node at level 1 to the next node at level 1. +# +$XFS_IO_PROG -f -c "pwrite -b 8M 0 16G" $SCRATCH_MNT/foo | _filter_xfs_io + +# Sync to flush delalloc and commit the current transaction, so fiemap will see +# all extents in the fs tree and extent trees and not look at delalloc. +sync + +# All extents should be reported as non shared (131072 extents). +echo "Number of non-shared extents in the whole file: $(count_not_shared_extents)" + +# Creating a snapshot. +$BTRFS_UTIL_PROG subvolume snapshot $SCRATCH_MNT $SCRATCH_MNT/snap | _filter_scratch + +# We have a snapshot, so now all extents should be reported as shared. +echo "Number of shared extents in the whole file: $(count_shared_extents)" + +# Now COW two file ranges, of 1M each, in the snapshot's file. +# So 16 extents should become non-shared after this. +# +$XFS_IO_PROG -c "pwrite -b 1M 8M 1M" \ + -c "pwrite -b 1M 12G 1M" \ + $SCRATCH_MNT/snap/foo | _filter_xfs_io + +# Sync to flush delalloc and commit the current transaction, so fiemap will see +# all extents in the fs tree and extent trees and not look at delalloc. +sync + +# Now we should have 16 non-shared extents and 131056 (131072 - 16) shared +# extents. +echo "Number of non-shared extents in the whole file: $(count_not_shared_extents)" +echo "Number of shared extents in the whole file: $(count_shared_extents)" + +# Check that the non-shared extents are indeed in the expected file ranges (each +# with 8 extents). +echo "Number of non-shared extents in range [8M, 9M): $(count_not_shared_extents 8M 1M)" +echo "Number of non-shared extents in range [12G, 12G + 1M): $(count_not_shared_extents 12G 1M)" + +# Now delete the snapshot. +$BTRFS_UTIL_PROG subvolume delete -c $SCRATCH_MNT/snap | _filter_scratch + +# We deleted the snapshot and committed the transaction used to delete it (-c), +# but all its extents (both metadata and data) are actually only deleted in the +# background, by the cleaner kthread. So remount, which wakes up the cleaner +# kthread, with a commit interval of 1 second and sleep for 1.1 seconds - after +# this we are guaranteed all extents of the snapshot were deleted. +_scratch_remount commit=1 +sleep 1.1 + +# Now all extents should be reported as not shared (131072 extents). +echo "Number of non-shared extents in the whole file: $(count_not_shared_extents)" + +# success, all done +status=0 +exit diff --git a/tests/btrfs/276.out b/tests/btrfs/276.out new file mode 100644 index 00000000..3bf5a5e6 --- /dev/null +++ b/tests/btrfs/276.out @@ -0,0 +1,16 @@ +QA output created by 276 +wrote 17179869184/17179869184 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +Number of non-shared extents in the whole file: 131072 +Create a snapshot of 'SCRATCH_MNT' in 'SCRATCH_MNT/snap' +Number of shared extents in the whole file: 131072 +wrote 1048576/1048576 bytes at offset 8388608 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 1048576/1048576 bytes at offset 12884901888 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +Number of non-shared extents in the whole file: 16 +Number of shared extents in the whole file: 131056 +Number of non-shared extents in range [8M, 9M): 8 +Number of non-shared extents in range [12G, 12G + 1M): 8 +Delete subvolume (commit): 'SCRATCH_MNT/snap' +Number of non-shared extents in the whole file: 131072