diff mbox series

btrfs: test an incremental send scenario with cloning of unaligned extent

Message ID 76d94e63cec4cb04cbfbc0dcd0928f1fbdc27bdf.1727432832.git.fdmanana@suse.com (mailing list archive)
State New, archived
Headers show
Series btrfs: test an incremental send scenario with cloning of unaligned extent | expand

Commit Message

Filipe Manana Sept. 27, 2024, 10:28 a.m. UTC 
From: Filipe Manana <fdmanana@suse.com>

Test that doing an incremental send with a file that had its size
decreased and became the destination for a clone operation of an extent
with an unaligned end offset that matches the new file size, works
correctly.

This tests a bug fixed by the following kernel patch:

  "btrfs: send: fix invalid clone operation for file that got its size decreased"

Signed-off-by: Filipe Manana <fdmanana@suse.com>
---
 tests/btrfs/322     | 108 ++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/322.out |  24 ++++++++++
 2 files changed, 132 insertions(+)
 create mode 100755 tests/btrfs/322
 create mode 100644 tests/btrfs/322.out

Comments

Qu Wenruo Sept. 27, 2024, 10:46 a.m. UTC | #1 
在 2024/9/27 19:58, fdmanana@kernel.org 写道:
> From: Filipe Manana <fdmanana@suse.com>
>
> Test that doing an incremental send with a file that had its size
> decreased and became the destination for a clone operation of an extent
> with an unaligned end offset that matches the new file size, works
> correctly.
>
> This tests a bug fixed by the following kernel patch:
>
>    "btrfs: send: fix invalid clone operation for file that got its size decreased"
>
> Signed-off-by: Filipe Manana <fdmanana@suse.com>

Reviewed-by: Qu Wenruo <wqu@suse.com>

Just a small nitpick.

[...]
> +. ./common/filter
> +. ./common/reflink
> +. ./common/punch # for _filter_fiemap_flags
> +
> +_require_test

Initially I thought test is not necessary, but later turns out that
we're using TEST_MNT to store the two streams.

May be we can just reuse $tmp.*?

Thanks,
Qu

> +_require_scratch_reflink
> +_require_xfs_io_command "fiemap"
> +_require_odirect
> +
> +_fixed_by_kernel_commit xxxxxxxxxxxx \
> +	"btrfs: send: fix invalid clone operation for file that got its size decreased"
> +
> +check_all_extents_shared()
> +{
> +	local file=$1
> +	local fiemap_output
> +
> +	fiemap_output=$($XFS_IO_PROG -r -c "fiemap -v" $file | _filter_fiemap_flags)
> +	echo "$fiemap_output" | grep -qv 'shared'
> +	if [ $? -eq 0 ]; then
> +		echo -e "Found non-shared extents for file $file:\n"
> +		echo "$fiemap_output"
> +	fi
> +}
> +
> +send_files_dir=$TEST_DIR/btrfs-test-$seq
> +full_send_stream=$send_files_dir/full_snap.stream
> +inc_send_stream=$send_files_dir/inc_snap.stream
> +
> +rm -fr $send_files_dir
> +mkdir $send_files_dir
> +
> +_scratch_mkfs >> $seqres.full 2>&1 || _fail "first mkfs failed"
> +_scratch_mount
> +
> +# Create a file with a size of 256K + 5 bytes, having two extents, the first one
> +# with a size of 128K and the second one with a size of 128K + 5 bytes.
> +last_extent_size=$((128 * 1024 + 5))
> +$XFS_IO_PROG -f -d -c "pwrite -S 0xab -b 128K 0 128K" \
> +             -c "pwrite -S 0xcd -b $last_extent_size 128K $last_extent_size" \
> +             $SCRATCH_MNT/foo | _filter_xfs_io
> +
> +# Another file which we will later clone foo into, but initially with
> +# a larger size than foo.
> +$XFS_IO_PROG -f -c "pwrite -b 0xef 0 1M" $SCRATCH_MNT/bar | _filter_xfs_io
> +
> +echo "Creating snapshot and the full send stream for it..."
> +_btrfs subvolume snapshot -r $SCRATCH_MNT $SCRATCH_MNT/snap1
> +$BTRFS_UTIL_PROG send -f $full_send_stream $SCRATCH_MNT/snap1 >> $seqres.full 2>&1
> +
> +# Now resize bar and clone foo into it.
> +$XFS_IO_PROG -c "truncate 0" \
> +	     -c "reflink $SCRATCH_MNT/foo" $SCRATCH_MNT/bar | _filter_xfs_io
> +
> +echo "Creating another snapshot and the incremental send stream for it..."
> +_btrfs subvolume snapshot -r $SCRATCH_MNT $SCRATCH_MNT/snap2
> +$BTRFS_UTIL_PROG send -p $SCRATCH_MNT/snap1 -f $inc_send_stream \
> +		 $SCRATCH_MNT/snap2 >> $seqres.full 2>&1
> +
> +echo "File digests in the original filesystem:"
> +md5sum $SCRATCH_MNT/snap1/foo | _filter_scratch
> +md5sum $SCRATCH_MNT/snap1/bar | _filter_scratch
> +md5sum $SCRATCH_MNT/snap2/foo | _filter_scratch
> +md5sum $SCRATCH_MNT/snap2/bar | _filter_scratch
> +
> +check_all_extents_shared "$SCRATCH_MNT/snap2/bar"
> +check_all_extents_shared "$SCRATCH_MNT/snap1/foo"
> +
> +echo "Creating a new filesystem to receive the send streams..."
> +_scratch_unmount
> +_scratch_mkfs >> $seqres.full 2>&1 || _fail "second mkfs failed"
> +_scratch_mount
> +
> +$BTRFS_UTIL_PROG receive -f $full_send_stream $SCRATCH_MNT
> +$BTRFS_UTIL_PROG receive -f $inc_send_stream $SCRATCH_MNT
> +
> +echo "File digests in the new filesystem:"
> +md5sum $SCRATCH_MNT/snap1/foo | _filter_scratch
> +md5sum $SCRATCH_MNT/snap1/bar | _filter_scratch
> +md5sum $SCRATCH_MNT/snap2/foo | _filter_scratch
> +md5sum $SCRATCH_MNT/snap2/bar | _filter_scratch
> +
> +check_all_extents_shared "$SCRATCH_MNT/snap2/bar"
> +check_all_extents_shared "$SCRATCH_MNT/snap1/foo"
> +
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/btrfs/322.out b/tests/btrfs/322.out
> new file mode 100644
> index 00000000..31e1ee55
> --- /dev/null
> +++ b/tests/btrfs/322.out
> @@ -0,0 +1,24 @@
> +QA output created by 322
> +wrote 131072/131072 bytes at offset 0
> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +wrote 131077/131077 bytes at offset 131072
> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +wrote 1048576/1048576 bytes at offset 0
> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +Creating snapshot and the full send stream for it...
> +linked 0/0 bytes at offset 0
> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +Creating another snapshot and the incremental send stream for it...
> +File digests in the original filesystem:
> +c3bb068b7e21d3f009581f047be84f44  SCRATCH_MNT/snap1/foo
> +ca539970d4b1fa1f34213ba675007381  SCRATCH_MNT/snap1/bar
> +c3bb068b7e21d3f009581f047be84f44  SCRATCH_MNT/snap2/foo
> +c3bb068b7e21d3f009581f047be84f44  SCRATCH_MNT/snap2/bar
> +Creating a new filesystem to receive the send streams...
> +At subvol snap1
> +At snapshot snap2
> +File digests in the new filesystem:
> +c3bb068b7e21d3f009581f047be84f44  SCRATCH_MNT/snap1/foo
> +ca539970d4b1fa1f34213ba675007381  SCRATCH_MNT/snap1/bar
> +c3bb068b7e21d3f009581f047be84f44  SCRATCH_MNT/snap2/foo
> +c3bb068b7e21d3f009581f047be84f44  SCRATCH_MNT/snap2/bar
Filipe Manana Sept. 27, 2024, 10:54 a.m. UTC | #2 
On Fri, Sep 27, 2024 at 11:46 AM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
>
>
>
> 在 2024/9/27 19:58, fdmanana@kernel.org 写道:
> > From: Filipe Manana <fdmanana@suse.com>
> >
> > Test that doing an incremental send with a file that had its size
> > decreased and became the destination for a clone operation of an extent
> > with an unaligned end offset that matches the new file size, works
> > correctly.
> >
> > This tests a bug fixed by the following kernel patch:
> >
> >    "btrfs: send: fix invalid clone operation for file that got its size decreased"
> >
> > Signed-off-by: Filipe Manana <fdmanana@suse.com>
>
> Reviewed-by: Qu Wenruo <wqu@suse.com>
>
> Just a small nitpick.
>
> [...]
> > +. ./common/filter
> > +. ./common/reflink
> > +. ./common/punch # for _filter_fiemap_flags
> > +
> > +_require_test
>
> Initially I thought test is not necessary, but later turns out that
> we're using TEST_MNT to store the two streams.
>
> May be we can just reuse $tmp.*?

Why?
The test device is always present, or are there any setups where there's none?

We use this pattern to store in the test mount because such files
could be big (not in this case however) and /tmp is usually much
smaller.
That's the recommendation I once got many years ago.

>
> Thanks,
> Qu
>
> > +_require_scratch_reflink
> > +_require_xfs_io_command "fiemap"
> > +_require_odirect
> > +
> > +_fixed_by_kernel_commit xxxxxxxxxxxx \
> > +     "btrfs: send: fix invalid clone operation for file that got its size decreased"
> > +
> > +check_all_extents_shared()
> > +{
> > +     local file=$1
> > +     local fiemap_output
> > +
> > +     fiemap_output=$($XFS_IO_PROG -r -c "fiemap -v" $file | _filter_fiemap_flags)
> > +     echo "$fiemap_output" | grep -qv 'shared'
> > +     if [ $? -eq 0 ]; then
> > +             echo -e "Found non-shared extents for file $file:\n"
> > +             echo "$fiemap_output"
> > +     fi
> > +}
> > +
> > +send_files_dir=$TEST_DIR/btrfs-test-$seq
> > +full_send_stream=$send_files_dir/full_snap.stream
> > +inc_send_stream=$send_files_dir/inc_snap.stream
> > +
> > +rm -fr $send_files_dir
> > +mkdir $send_files_dir
> > +
> > +_scratch_mkfs >> $seqres.full 2>&1 || _fail "first mkfs failed"
> > +_scratch_mount
> > +
> > +# Create a file with a size of 256K + 5 bytes, having two extents, the first one
> > +# with a size of 128K and the second one with a size of 128K + 5 bytes.
> > +last_extent_size=$((128 * 1024 + 5))
> > +$XFS_IO_PROG -f -d -c "pwrite -S 0xab -b 128K 0 128K" \
> > +             -c "pwrite -S 0xcd -b $last_extent_size 128K $last_extent_size" \
> > +             $SCRATCH_MNT/foo | _filter_xfs_io
> > +
> > +# Another file which we will later clone foo into, but initially with
> > +# a larger size than foo.
> > +$XFS_IO_PROG -f -c "pwrite -b 0xef 0 1M" $SCRATCH_MNT/bar | _filter_xfs_io
> > +
> > +echo "Creating snapshot and the full send stream for it..."
> > +_btrfs subvolume snapshot -r $SCRATCH_MNT $SCRATCH_MNT/snap1
> > +$BTRFS_UTIL_PROG send -f $full_send_stream $SCRATCH_MNT/snap1 >> $seqres.full 2>&1
> > +
> > +# Now resize bar and clone foo into it.
> > +$XFS_IO_PROG -c "truncate 0" \
> > +          -c "reflink $SCRATCH_MNT/foo" $SCRATCH_MNT/bar | _filter_xfs_io
> > +
> > +echo "Creating another snapshot and the incremental send stream for it..."
> > +_btrfs subvolume snapshot -r $SCRATCH_MNT $SCRATCH_MNT/snap2
> > +$BTRFS_UTIL_PROG send -p $SCRATCH_MNT/snap1 -f $inc_send_stream \
> > +              $SCRATCH_MNT/snap2 >> $seqres.full 2>&1
> > +
> > +echo "File digests in the original filesystem:"
> > +md5sum $SCRATCH_MNT/snap1/foo | _filter_scratch
> > +md5sum $SCRATCH_MNT/snap1/bar | _filter_scratch
> > +md5sum $SCRATCH_MNT/snap2/foo | _filter_scratch
> > +md5sum $SCRATCH_MNT/snap2/bar | _filter_scratch
> > +
> > +check_all_extents_shared "$SCRATCH_MNT/snap2/bar"
> > +check_all_extents_shared "$SCRATCH_MNT/snap1/foo"
> > +
> > +echo "Creating a new filesystem to receive the send streams..."
> > +_scratch_unmount
> > +_scratch_mkfs >> $seqres.full 2>&1 || _fail "second mkfs failed"
> > +_scratch_mount
> > +
> > +$BTRFS_UTIL_PROG receive -f $full_send_stream $SCRATCH_MNT
> > +$BTRFS_UTIL_PROG receive -f $inc_send_stream $SCRATCH_MNT
> > +
> > +echo "File digests in the new filesystem:"
> > +md5sum $SCRATCH_MNT/snap1/foo | _filter_scratch
> > +md5sum $SCRATCH_MNT/snap1/bar | _filter_scratch
> > +md5sum $SCRATCH_MNT/snap2/foo | _filter_scratch
> > +md5sum $SCRATCH_MNT/snap2/bar | _filter_scratch
> > +
> > +check_all_extents_shared "$SCRATCH_MNT/snap2/bar"
> > +check_all_extents_shared "$SCRATCH_MNT/snap1/foo"
> > +
> > +# success, all done
> > +status=0
> > +exit
> > diff --git a/tests/btrfs/322.out b/tests/btrfs/322.out
> > new file mode 100644
> > index 00000000..31e1ee55
> > --- /dev/null
> > +++ b/tests/btrfs/322.out
> > @@ -0,0 +1,24 @@
> > +QA output created by 322
> > +wrote 131072/131072 bytes at offset 0
> > +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> > +wrote 131077/131077 bytes at offset 131072
> > +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> > +wrote 1048576/1048576 bytes at offset 0
> > +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> > +Creating snapshot and the full send stream for it...
> > +linked 0/0 bytes at offset 0
> > +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> > +Creating another snapshot and the incremental send stream for it...
> > +File digests in the original filesystem:
> > +c3bb068b7e21d3f009581f047be84f44  SCRATCH_MNT/snap1/foo
> > +ca539970d4b1fa1f34213ba675007381  SCRATCH_MNT/snap1/bar
> > +c3bb068b7e21d3f009581f047be84f44  SCRATCH_MNT/snap2/foo
> > +c3bb068b7e21d3f009581f047be84f44  SCRATCH_MNT/snap2/bar
> > +Creating a new filesystem to receive the send streams...
> > +At subvol snap1
> > +At snapshot snap2
> > +File digests in the new filesystem:
> > +c3bb068b7e21d3f009581f047be84f44  SCRATCH_MNT/snap1/foo
> > +ca539970d4b1fa1f34213ba675007381  SCRATCH_MNT/snap1/bar
> > +c3bb068b7e21d3f009581f047be84f44  SCRATCH_MNT/snap2/foo
> > +c3bb068b7e21d3f009581f047be84f44  SCRATCH_MNT/snap2/bar
>
Zorro Lang Sept. 27, 2024, 2:44 p.m. UTC | #3 
On Fri, Sep 27, 2024 at 11:28:07AM +0100, fdmanana@kernel.org wrote:
> From: Filipe Manana <fdmanana@suse.com>
> 
> Test that doing an incremental send with a file that had its size
> decreased and became the destination for a clone operation of an extent
> with an unaligned end offset that matches the new file size, works
> correctly.
> 
> This tests a bug fixed by the following kernel patch:
> 
>   "btrfs: send: fix invalid clone operation for file that got its size decreased"
> 
> Signed-off-by: Filipe Manana <fdmanana@suse.com>
> ---
>  tests/btrfs/322     | 108 ++++++++++++++++++++++++++++++++++++++++++++
>  tests/btrfs/322.out |  24 ++++++++++
>  2 files changed, 132 insertions(+)
>  create mode 100755 tests/btrfs/322
>  create mode 100644 tests/btrfs/322.out
> 
> diff --git a/tests/btrfs/322 b/tests/btrfs/322
> new file mode 100755
> index 00000000..c03f6a4c
> --- /dev/null
> +++ b/tests/btrfs/322
> @@ -0,0 +1,108 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (C) 2024 SUSE Linux Products GmbH. All Rights Reserved.
> +#
> +# FS QA Test 322
> +#
> +# Test that doing an incremental send with a file that had its size decreased
> +# and became the destination for a clone operation of an extent with an
> +# unaligned end offset that matches the new file size, works correctly.
> +#
> +. ./common/preamble
> +_begin_fstest auto quick send clone fiemap
> +
> +_cleanup()
> +{
> +	cd /
> +	rm -fr $tmp.*
> +	rm -fr $send_files_dir
> +}
> +
> +. ./common/filter
> +. ./common/reflink
> +. ./common/punch # for _filter_fiemap_flags
> +
> +_require_test
> +_require_scratch_reflink
> +_require_xfs_io_command "fiemap"
> +_require_odirect
> +
> +_fixed_by_kernel_commit xxxxxxxxxxxx \
> +	"btrfs: send: fix invalid clone operation for file that got its size decreased"
> +
> +check_all_extents_shared()
> +{
> +	local file=$1
> +	local fiemap_output
> +
> +	fiemap_output=$($XFS_IO_PROG -r -c "fiemap -v" $file | _filter_fiemap_flags)
> +	echo "$fiemap_output" | grep -qv 'shared'
> +	if [ $? -eq 0 ]; then
> +		echo -e "Found non-shared extents for file $file:\n"
> +		echo "$fiemap_output"
> +	fi
> +}
> +
> +send_files_dir=$TEST_DIR/btrfs-test-$seq
> +full_send_stream=$send_files_dir/full_snap.stream
> +inc_send_stream=$send_files_dir/inc_snap.stream
> +
> +rm -fr $send_files_dir
> +mkdir $send_files_dir
> +
> +_scratch_mkfs >> $seqres.full 2>&1 || _fail "first mkfs failed"
> +_scratch_mount
> +
> +# Create a file with a size of 256K + 5 bytes, having two extents, the first one
> +# with a size of 128K and the second one with a size of 128K + 5 bytes.
> +last_extent_size=$((128 * 1024 + 5))
> +$XFS_IO_PROG -f -d -c "pwrite -S 0xab -b 128K 0 128K" \
> +             -c "pwrite -S 0xcd -b $last_extent_size 128K $last_extent_size" \
> +             $SCRATCH_MNT/foo | _filter_xfs_io
> +
> +# Another file which we will later clone foo into, but initially with
> +# a larger size than foo.
> +$XFS_IO_PROG -f -c "pwrite -b 0xef 0 1M" $SCRATCH_MNT/bar | _filter_xfs_io
> +
> +echo "Creating snapshot and the full send stream for it..."
> +_btrfs subvolume snapshot -r $SCRATCH_MNT $SCRATCH_MNT/snap1
> +$BTRFS_UTIL_PROG send -f $full_send_stream $SCRATCH_MNT/snap1 >> $seqres.full 2>&1
> +
> +# Now resize bar and clone foo into it.
> +$XFS_IO_PROG -c "truncate 0" \
> +	     -c "reflink $SCRATCH_MNT/foo" $SCRATCH_MNT/bar | _filter_xfs_io

_require_xfs_io_command "reflink"

I'll help to add it when I merge it. Other looks good to me. Thanks!

> +
> +echo "Creating another snapshot and the incremental send stream for it..."
> +_btrfs subvolume snapshot -r $SCRATCH_MNT $SCRATCH_MNT/snap2
> +$BTRFS_UTIL_PROG send -p $SCRATCH_MNT/snap1 -f $inc_send_stream \
> +		 $SCRATCH_MNT/snap2 >> $seqres.full 2>&1
> +
> +echo "File digests in the original filesystem:"
> +md5sum $SCRATCH_MNT/snap1/foo | _filter_scratch
> +md5sum $SCRATCH_MNT/snap1/bar | _filter_scratch
> +md5sum $SCRATCH_MNT/snap2/foo | _filter_scratch
> +md5sum $SCRATCH_MNT/snap2/bar | _filter_scratch
> +
> +check_all_extents_shared "$SCRATCH_MNT/snap2/bar"
> +check_all_extents_shared "$SCRATCH_MNT/snap1/foo"
> +
> +echo "Creating a new filesystem to receive the send streams..."
> +_scratch_unmount
> +_scratch_mkfs >> $seqres.full 2>&1 || _fail "second mkfs failed"
> +_scratch_mount
> +
> +$BTRFS_UTIL_PROG receive -f $full_send_stream $SCRATCH_MNT
> +$BTRFS_UTIL_PROG receive -f $inc_send_stream $SCRATCH_MNT
> +
> +echo "File digests in the new filesystem:"
> +md5sum $SCRATCH_MNT/snap1/foo | _filter_scratch
> +md5sum $SCRATCH_MNT/snap1/bar | _filter_scratch
> +md5sum $SCRATCH_MNT/snap2/foo | _filter_scratch
> +md5sum $SCRATCH_MNT/snap2/bar | _filter_scratch
> +
> +check_all_extents_shared "$SCRATCH_MNT/snap2/bar"
> +check_all_extents_shared "$SCRATCH_MNT/snap1/foo"
> +
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/btrfs/322.out b/tests/btrfs/322.out
> new file mode 100644
> index 00000000..31e1ee55
> --- /dev/null
> +++ b/tests/btrfs/322.out
> @@ -0,0 +1,24 @@
> +QA output created by 322
> +wrote 131072/131072 bytes at offset 0
> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +wrote 131077/131077 bytes at offset 131072
> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +wrote 1048576/1048576 bytes at offset 0
> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +Creating snapshot and the full send stream for it...
> +linked 0/0 bytes at offset 0
> +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +Creating another snapshot and the incremental send stream for it...
> +File digests in the original filesystem:
> +c3bb068b7e21d3f009581f047be84f44  SCRATCH_MNT/snap1/foo
> +ca539970d4b1fa1f34213ba675007381  SCRATCH_MNT/snap1/bar
> +c3bb068b7e21d3f009581f047be84f44  SCRATCH_MNT/snap2/foo
> +c3bb068b7e21d3f009581f047be84f44  SCRATCH_MNT/snap2/bar
> +Creating a new filesystem to receive the send streams...
> +At subvol snap1
> +At snapshot snap2
> +File digests in the new filesystem:
> +c3bb068b7e21d3f009581f047be84f44  SCRATCH_MNT/snap1/foo
> +ca539970d4b1fa1f34213ba675007381  SCRATCH_MNT/snap1/bar
> +c3bb068b7e21d3f009581f047be84f44  SCRATCH_MNT/snap2/foo
> +c3bb068b7e21d3f009581f047be84f44  SCRATCH_MNT/snap2/bar
> -- 
> 2.43.0
> 
>
Filipe Manana Sept. 27, 2024, 3:30 p.m. UTC | #4 
On Fri, Sep 27, 2024 at 3:44 PM Zorro Lang <zlang@redhat.com> wrote:
>
> On Fri, Sep 27, 2024 at 11:28:07AM +0100, fdmanana@kernel.org wrote:
> > From: Filipe Manana <fdmanana@suse.com>
> >
> > Test that doing an incremental send with a file that had its size
> > decreased and became the destination for a clone operation of an extent
> > with an unaligned end offset that matches the new file size, works
> > correctly.
> >
> > This tests a bug fixed by the following kernel patch:
> >
> >   "btrfs: send: fix invalid clone operation for file that got its size decreased"
> >
> > Signed-off-by: Filipe Manana <fdmanana@suse.com>
> > ---
> >  tests/btrfs/322     | 108 ++++++++++++++++++++++++++++++++++++++++++++
> >  tests/btrfs/322.out |  24 ++++++++++
> >  2 files changed, 132 insertions(+)
> >  create mode 100755 tests/btrfs/322
> >  create mode 100644 tests/btrfs/322.out
> >
> > diff --git a/tests/btrfs/322 b/tests/btrfs/322
> > new file mode 100755
> > index 00000000..c03f6a4c
> > --- /dev/null
> > +++ b/tests/btrfs/322
> > @@ -0,0 +1,108 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0
> > +# Copyright (C) 2024 SUSE Linux Products GmbH. All Rights Reserved.
> > +#
> > +# FS QA Test 322
> > +#
> > +# Test that doing an incremental send with a file that had its size decreased
> > +# and became the destination for a clone operation of an extent with an
> > +# unaligned end offset that matches the new file size, works correctly.
> > +#
> > +. ./common/preamble
> > +_begin_fstest auto quick send clone fiemap
> > +
> > +_cleanup()
> > +{
> > +     cd /
> > +     rm -fr $tmp.*
> > +     rm -fr $send_files_dir
> > +}
> > +
> > +. ./common/filter
> > +. ./common/reflink
> > +. ./common/punch # for _filter_fiemap_flags
> > +
> > +_require_test
> > +_require_scratch_reflink
> > +_require_xfs_io_command "fiemap"
> > +_require_odirect
> > +
> > +_fixed_by_kernel_commit xxxxxxxxxxxx \
> > +     "btrfs: send: fix invalid clone operation for file that got its size decreased"
> > +
> > +check_all_extents_shared()
> > +{
> > +     local file=$1
> > +     local fiemap_output
> > +
> > +     fiemap_output=$($XFS_IO_PROG -r -c "fiemap -v" $file | _filter_fiemap_flags)
> > +     echo "$fiemap_output" | grep -qv 'shared'
> > +     if [ $? -eq 0 ]; then
> > +             echo -e "Found non-shared extents for file $file:\n"
> > +             echo "$fiemap_output"
> > +     fi
> > +}
> > +
> > +send_files_dir=$TEST_DIR/btrfs-test-$seq
> > +full_send_stream=$send_files_dir/full_snap.stream
> > +inc_send_stream=$send_files_dir/inc_snap.stream
> > +
> > +rm -fr $send_files_dir
> > +mkdir $send_files_dir
> > +
> > +_scratch_mkfs >> $seqres.full 2>&1 || _fail "first mkfs failed"
> > +_scratch_mount
> > +
> > +# Create a file with a size of 256K + 5 bytes, having two extents, the first one
> > +# with a size of 128K and the second one with a size of 128K + 5 bytes.
> > +last_extent_size=$((128 * 1024 + 5))
> > +$XFS_IO_PROG -f -d -c "pwrite -S 0xab -b 128K 0 128K" \
> > +             -c "pwrite -S 0xcd -b $last_extent_size 128K $last_extent_size" \
> > +             $SCRATCH_MNT/foo | _filter_xfs_io
> > +
> > +# Another file which we will later clone foo into, but initially with
> > +# a larger size than foo.
> > +$XFS_IO_PROG -f -c "pwrite -b 0xef 0 1M" $SCRATCH_MNT/bar | _filter_xfs_io
> > +
> > +echo "Creating snapshot and the full send stream for it..."
> > +_btrfs subvolume snapshot -r $SCRATCH_MNT $SCRATCH_MNT/snap1
> > +$BTRFS_UTIL_PROG send -f $full_send_stream $SCRATCH_MNT/snap1 >> $seqres.full 2>&1
> > +
> > +# Now resize bar and clone foo into it.
> > +$XFS_IO_PROG -c "truncate 0" \
> > +          -c "reflink $SCRATCH_MNT/foo" $SCRATCH_MNT/bar | _filter_xfs_io
>
> _require_xfs_io_command "reflink"
>
> I'll help to add it when I merge it. Other looks good to me. Thanks!

Thanks! I forgot that.

>
> > +
> > +echo "Creating another snapshot and the incremental send stream for it..."
> > +_btrfs subvolume snapshot -r $SCRATCH_MNT $SCRATCH_MNT/snap2
> > +$BTRFS_UTIL_PROG send -p $SCRATCH_MNT/snap1 -f $inc_send_stream \
> > +              $SCRATCH_MNT/snap2 >> $seqres.full 2>&1
> > +
> > +echo "File digests in the original filesystem:"
> > +md5sum $SCRATCH_MNT/snap1/foo | _filter_scratch
> > +md5sum $SCRATCH_MNT/snap1/bar | _filter_scratch
> > +md5sum $SCRATCH_MNT/snap2/foo | _filter_scratch
> > +md5sum $SCRATCH_MNT/snap2/bar | _filter_scratch
> > +
> > +check_all_extents_shared "$SCRATCH_MNT/snap2/bar"
> > +check_all_extents_shared "$SCRATCH_MNT/snap1/foo"
> > +
> > +echo "Creating a new filesystem to receive the send streams..."
> > +_scratch_unmount
> > +_scratch_mkfs >> $seqres.full 2>&1 || _fail "second mkfs failed"
> > +_scratch_mount
> > +
> > +$BTRFS_UTIL_PROG receive -f $full_send_stream $SCRATCH_MNT
> > +$BTRFS_UTIL_PROG receive -f $inc_send_stream $SCRATCH_MNT
> > +
> > +echo "File digests in the new filesystem:"
> > +md5sum $SCRATCH_MNT/snap1/foo | _filter_scratch
> > +md5sum $SCRATCH_MNT/snap1/bar | _filter_scratch
> > +md5sum $SCRATCH_MNT/snap2/foo | _filter_scratch
> > +md5sum $SCRATCH_MNT/snap2/bar | _filter_scratch
> > +
> > +check_all_extents_shared "$SCRATCH_MNT/snap2/bar"
> > +check_all_extents_shared "$SCRATCH_MNT/snap1/foo"
> > +
> > +# success, all done
> > +status=0
> > +exit
> > diff --git a/tests/btrfs/322.out b/tests/btrfs/322.out
> > new file mode 100644
> > index 00000000..31e1ee55
> > --- /dev/null
> > +++ b/tests/btrfs/322.out
> > @@ -0,0 +1,24 @@
> > +QA output created by 322
> > +wrote 131072/131072 bytes at offset 0
> > +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> > +wrote 131077/131077 bytes at offset 131072
> > +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> > +wrote 1048576/1048576 bytes at offset 0
> > +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> > +Creating snapshot and the full send stream for it...
> > +linked 0/0 bytes at offset 0
> > +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> > +Creating another snapshot and the incremental send stream for it...
> > +File digests in the original filesystem:
> > +c3bb068b7e21d3f009581f047be84f44  SCRATCH_MNT/snap1/foo
> > +ca539970d4b1fa1f34213ba675007381  SCRATCH_MNT/snap1/bar
> > +c3bb068b7e21d3f009581f047be84f44  SCRATCH_MNT/snap2/foo
> > +c3bb068b7e21d3f009581f047be84f44  SCRATCH_MNT/snap2/bar
> > +Creating a new filesystem to receive the send streams...
> > +At subvol snap1
> > +At snapshot snap2
> > +File digests in the new filesystem:
> > +c3bb068b7e21d3f009581f047be84f44  SCRATCH_MNT/snap1/foo
> > +ca539970d4b1fa1f34213ba675007381  SCRATCH_MNT/snap1/bar
> > +c3bb068b7e21d3f009581f047be84f44  SCRATCH_MNT/snap2/foo
> > +c3bb068b7e21d3f009581f047be84f44  SCRATCH_MNT/snap2/bar
> > --
> > 2.43.0
> >
> >
>
diff mbox series

Patch

diff --git a/tests/btrfs/322 b/tests/btrfs/322
new file mode 100755
index 00000000..c03f6a4c
--- /dev/null
+++ b/tests/btrfs/322
@@ -0,0 +1,108 @@ 
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2024 SUSE Linux Products GmbH. All Rights Reserved.
+#
+# FS QA Test 322
+#
+# Test that doing an incremental send with a file that had its size decreased
+# and became the destination for a clone operation of an extent with an
+# unaligned end offset that matches the new file size, works correctly.
+#
+. ./common/preamble
+_begin_fstest auto quick send clone fiemap
+
+_cleanup()
+{
+	cd /
+	rm -fr $tmp.*
+	rm -fr $send_files_dir
+}
+
+. ./common/filter
+. ./common/reflink
+. ./common/punch # for _filter_fiemap_flags
+
+_require_test
+_require_scratch_reflink
+_require_xfs_io_command "fiemap"
+_require_odirect
+
+_fixed_by_kernel_commit xxxxxxxxxxxx \
+	"btrfs: send: fix invalid clone operation for file that got its size decreased"
+
+check_all_extents_shared()
+{
+	local file=$1
+	local fiemap_output
+
+	fiemap_output=$($XFS_IO_PROG -r -c "fiemap -v" $file | _filter_fiemap_flags)
+	echo "$fiemap_output" | grep -qv 'shared'
+	if [ $? -eq 0 ]; then
+		echo -e "Found non-shared extents for file $file:\n"
+		echo "$fiemap_output"
+	fi
+}
+
+send_files_dir=$TEST_DIR/btrfs-test-$seq
+full_send_stream=$send_files_dir/full_snap.stream
+inc_send_stream=$send_files_dir/inc_snap.stream
+
+rm -fr $send_files_dir
+mkdir $send_files_dir
+
+_scratch_mkfs >> $seqres.full 2>&1 || _fail "first mkfs failed"
+_scratch_mount
+
+# Create a file with a size of 256K + 5 bytes, having two extents, the first one
+# with a size of 128K and the second one with a size of 128K + 5 bytes.
+last_extent_size=$((128 * 1024 + 5))
+$XFS_IO_PROG -f -d -c "pwrite -S 0xab -b 128K 0 128K" \
+             -c "pwrite -S 0xcd -b $last_extent_size 128K $last_extent_size" \
+             $SCRATCH_MNT/foo | _filter_xfs_io
+
+# Another file which we will later clone foo into, but initially with
+# a larger size than foo.
+$XFS_IO_PROG -f -c "pwrite -b 0xef 0 1M" $SCRATCH_MNT/bar | _filter_xfs_io
+
+echo "Creating snapshot and the full send stream for it..."
+_btrfs subvolume snapshot -r $SCRATCH_MNT $SCRATCH_MNT/snap1
+$BTRFS_UTIL_PROG send -f $full_send_stream $SCRATCH_MNT/snap1 >> $seqres.full 2>&1
+
+# Now resize bar and clone foo into it.
+$XFS_IO_PROG -c "truncate 0" \
+	     -c "reflink $SCRATCH_MNT/foo" $SCRATCH_MNT/bar | _filter_xfs_io
+
+echo "Creating another snapshot and the incremental send stream for it..."
+_btrfs subvolume snapshot -r $SCRATCH_MNT $SCRATCH_MNT/snap2
+$BTRFS_UTIL_PROG send -p $SCRATCH_MNT/snap1 -f $inc_send_stream \
+		 $SCRATCH_MNT/snap2 >> $seqres.full 2>&1
+
+echo "File digests in the original filesystem:"
+md5sum $SCRATCH_MNT/snap1/foo | _filter_scratch
+md5sum $SCRATCH_MNT/snap1/bar | _filter_scratch
+md5sum $SCRATCH_MNT/snap2/foo | _filter_scratch
+md5sum $SCRATCH_MNT/snap2/bar | _filter_scratch
+
+check_all_extents_shared "$SCRATCH_MNT/snap2/bar"
+check_all_extents_shared "$SCRATCH_MNT/snap1/foo"
+
+echo "Creating a new filesystem to receive the send streams..."
+_scratch_unmount
+_scratch_mkfs >> $seqres.full 2>&1 || _fail "second mkfs failed"
+_scratch_mount
+
+$BTRFS_UTIL_PROG receive -f $full_send_stream $SCRATCH_MNT
+$BTRFS_UTIL_PROG receive -f $inc_send_stream $SCRATCH_MNT
+
+echo "File digests in the new filesystem:"
+md5sum $SCRATCH_MNT/snap1/foo | _filter_scratch
+md5sum $SCRATCH_MNT/snap1/bar | _filter_scratch
+md5sum $SCRATCH_MNT/snap2/foo | _filter_scratch
+md5sum $SCRATCH_MNT/snap2/bar | _filter_scratch
+
+check_all_extents_shared "$SCRATCH_MNT/snap2/bar"
+check_all_extents_shared "$SCRATCH_MNT/snap1/foo"
+
+# success, all done
+status=0
+exit
diff --git a/tests/btrfs/322.out b/tests/btrfs/322.out
new file mode 100644
index 00000000..31e1ee55
--- /dev/null
+++ b/tests/btrfs/322.out
@@ -0,0 +1,24 @@ 
+QA output created by 322
+wrote 131072/131072 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 131077/131077 bytes at offset 131072
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 1048576/1048576 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+Creating snapshot and the full send stream for it...
+linked 0/0 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+Creating another snapshot and the incremental send stream for it...
+File digests in the original filesystem:
+c3bb068b7e21d3f009581f047be84f44  SCRATCH_MNT/snap1/foo
+ca539970d4b1fa1f34213ba675007381  SCRATCH_MNT/snap1/bar
+c3bb068b7e21d3f009581f047be84f44  SCRATCH_MNT/snap2/foo
+c3bb068b7e21d3f009581f047be84f44  SCRATCH_MNT/snap2/bar
+Creating a new filesystem to receive the send streams...
+At subvol snap1
+At snapshot snap2
+File digests in the new filesystem:
+c3bb068b7e21d3f009581f047be84f44  SCRATCH_MNT/snap1/foo
+ca539970d4b1fa1f34213ba675007381  SCRATCH_MNT/snap1/bar
+c3bb068b7e21d3f009581f047be84f44  SCRATCH_MNT/snap2/foo
+c3bb068b7e21d3f009581f047be84f44  SCRATCH_MNT/snap2/bar