diff mbox series

fstests: btrfs/012: fix a false alert due to socket/pipe files

Message ID 20240717072700.93025-1-wqu@suse.com (mailing list archive)
State New, archived
Headers show
Series fstests: btrfs/012: fix a false alert due to socket/pipe files | expand

Commit Message

Qu Wenruo July 17, 2024, 7:27 a.m. UTC
[BUG]
On my Archlinux VM, the test btrfs/012 always fail with the following
output diff:

     QA output created by 012
    +File /etc/pacman.d/gnupg/S.dirmngr is a socket while file /mnt/scratch/etc/pacman.d/gnupg/S.dirmngr is a socket
    +File /etc/pacman.d/gnupg/S.gpg-agent is a socket while file /mnt/scratch/etc/pacman.d/gnupg/S.gpg-agent is a socket
    +File /etc/pacman.d/gnupg/S.gpg-agent.browser is a socket while file /mnt/scratch/etc/pacman.d/gnupg/S.gpg-agent.browser is a socket
    +File /etc/pacman.d/gnupg/S.gpg-agent.extra is a socket while file /mnt/scratch/etc/pacman.d/gnupg/S.gpg-agent.extra is a socket
    +File /etc/pacman.d/gnupg/S.gpg-agent.ssh is a socket while file /mnt/scratch/etc/pacman.d/gnupg/S.gpg-agent.ssh is a socket
    +File /etc/pacman.d/gnupg/S.keyboxd is a socket while file /mnt/scratch/etc/pacman.d/gnupg/S.keyboxd is a socket
    ...

[CAUSE]
It's a false alerts.

When diff hits two files which are not directory/softlink/regular files
(aka, socket/pipe/char/block files), they are all treated as
non-comparable.
In that case, diff would just do the above message.

And with Archlinux, pacman (the package manager) maintains its gpg
directory inside "/etc/pacman.d/gnupg", and the test case uses
"/etc" as the source directory to populate the target ext4 fs.

And the socket files inside "/etc/pacman.d/gnupg" is causing the false
alerts.

[FIX]
- Use fsstress to populate the fs
  That covers all kind of operations, including creating special files.
  And fsstress is very reproducible, with the seed saved to the full
  log, it's much easier to reproduce than using the distro dependent
  "/etc/" directory.

- Use fssum to save the digest and later verify the contents
  It does not only verify the contents but also other things like
  timestamps/xattrs/uid/gid/mode/etc.
  And it's more comprehensive than the content oriented diff tool.

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 tests/btrfs/012     | 29 +++++++++++++++++------------
 tests/btrfs/012.out |  6 ++++++
 2 files changed, 23 insertions(+), 12 deletions(-)

Comments

Filipe Manana July 17, 2024, 11:25 a.m. UTC | #1
On Wed, Jul 17, 2024 at 8:27 AM Qu Wenruo <wqu@suse.com> wrote:
>
> [BUG]
> On my Archlinux VM, the test btrfs/012 always fail with the following
> output diff:
>
>      QA output created by 012
>     +File /etc/pacman.d/gnupg/S.dirmngr is a socket while file /mnt/scratch/etc/pacman.d/gnupg/S.dirmngr is a socket
>     +File /etc/pacman.d/gnupg/S.gpg-agent is a socket while file /mnt/scratch/etc/pacman.d/gnupg/S.gpg-agent is a socket
>     +File /etc/pacman.d/gnupg/S.gpg-agent.browser is a socket while file /mnt/scratch/etc/pacman.d/gnupg/S.gpg-agent.browser is a socket
>     +File /etc/pacman.d/gnupg/S.gpg-agent.extra is a socket while file /mnt/scratch/etc/pacman.d/gnupg/S.gpg-agent.extra is a socket
>     +File /etc/pacman.d/gnupg/S.gpg-agent.ssh is a socket while file /mnt/scratch/etc/pacman.d/gnupg/S.gpg-agent.ssh is a socket
>     +File /etc/pacman.d/gnupg/S.keyboxd is a socket while file /mnt/scratch/etc/pacman.d/gnupg/S.keyboxd is a socket
>     ...
>
> [CAUSE]
> It's a false alerts.
>
> When diff hits two files which are not directory/softlink/regular files
> (aka, socket/pipe/char/block files), they are all treated as
> non-comparable.
> In that case, diff would just do the above message.
>
> And with Archlinux, pacman (the package manager) maintains its gpg
> directory inside "/etc/pacman.d/gnupg", and the test case uses
> "/etc" as the source directory to populate the target ext4 fs.
>
> And the socket files inside "/etc/pacman.d/gnupg" is causing the false
> alerts.
>
> [FIX]
> - Use fsstress to populate the fs
>   That covers all kind of operations, including creating special files.
>   And fsstress is very reproducible, with the seed saved to the full
>   log, it's much easier to reproduce than using the distro dependent
>   "/etc/" directory.
>
> - Use fssum to save the digest and later verify the contents
>   It does not only verify the contents but also other things like
>   timestamps/xattrs/uid/gid/mode/etc.
>   And it's more comprehensive than the content oriented diff tool.
>
> Signed-off-by: Qu Wenruo <wqu@suse.com>
> ---
>  tests/btrfs/012     | 29 +++++++++++++++++------------
>  tests/btrfs/012.out |  6 ++++++
>  2 files changed, 23 insertions(+), 12 deletions(-)
>
> diff --git a/tests/btrfs/012 b/tests/btrfs/012
> index a96efeff..d172d93f 100755
> --- a/tests/btrfs/012
> +++ b/tests/btrfs/012
> @@ -23,13 +23,13 @@ _require_scratch_nocheck
>  _require_command "$BTRFS_CONVERT_PROG" btrfs-convert
>  _require_command "$MKFS_EXT4_PROG" mkfs.ext4
>  _require_command "$E2FSCK_PROG" e2fsck
> +_require_fssum
>  # ext4 does not support zoned block device
>  _require_non_zoned_device "${SCRATCH_DEV}"
>  _require_loop
>  _require_extra_fs ext4
>
> -SOURCE_DIR=/etc
> -BASENAME=$(basename $SOURCE_DIR)
> +BASENAME="stressdir"
>  BLOCK_SIZE=`_get_block_size $TEST_DIR`
>
>  # Create & populate an ext4 filesystem
> @@ -38,17 +38,21 @@ $MKFS_EXT4_PROG -F -b $BLOCK_SIZE $SCRATCH_DEV > $seqres.full 2>&1 || \
>  # Manual mount so we don't use -t btrfs or selinux context
>  mount -t ext4 $SCRATCH_DEV $SCRATCH_MNT
>
> -_require_fs_space $SCRATCH_MNT $(du -s $SOURCE_DIR | ${AWK_PROG} '{print $1}')
> +echo "populating the initial ext fs:" >> $seqres.full
> +mkdir "$SCRATCH_MNT/$BASENAME"
> +$FSSTRESS_PROG -w -d "$SCRATCH_MNT/$BASENAME" -n 20 -p 500 >> $seqres.full
>
> -$TIMEOUT_PROG 10 cp -aRP $SOURCE_DIR $SCRATCH_MNT
> +# Create the checksum to verify later.
> +$FSSUM_PROG -A -f -w $tmp.original "$SCRATCH_MNT/$BASENAME"
>  _scratch_unmount
>
>  # Convert it to btrfs, mount it, verify the data
>  $BTRFS_CONVERT_PROG $SCRATCH_DEV >> $seqres.full 2>&1 || \
>         _fail "btrfs-convert failed"
>  _try_scratch_mount || _fail "Could not mount new btrfs fs"
> -# (Ignore the symlinks which may be broken/nonexistent)
> -diff --no-dereference -r $SOURCE_DIR $SCRATCH_MNT/$BASENAME/ 2>&1
> +
> +echo "Checking converted btrfs against the original one:"
> +$FSSUM_PROG -r $tmp.original $SCRATCH_MNT/$BASENAME
>
>  # Old ext4 image file should exist & be consistent
>  $E2FSCK_PROG -fn $SCRATCH_MNT/ext2_saved/image >> $seqres.full 2>&1 || \
> @@ -58,13 +62,14 @@ $E2FSCK_PROG -fn $SCRATCH_MNT/ext2_saved/image >> $seqres.full 2>&1 || \
>  mkdir -p $SCRATCH_MNT/mnt
>  mount -o loop $SCRATCH_MNT/ext2_saved/image $SCRATCH_MNT/mnt || \
>         _fail "could not loop mount saved ext4 image"
> -# Ignore the symlinks which may be broken/nonexistent
> -diff --no-dereference -r $SOURCE_DIR $SCRATCH_MNT/mnt/$BASENAME/ 2>&1
> +
> +echo "Checking saved ext2 image against the original one:"
> +$FSSUM_PROG -r $tmp.original $SCRATCH_MNT/mnt/$BASENAME
>  umount $SCRATCH_MNT/mnt
>
> -# Now put some fresh data on the btrfs fs
> +echo "genereating new data on the converted btrfs" >> $seqres.full

Typo:  genereating -> generating

With that fixed:

Reviewed-by: Filipe Manana <fdmanana@suse.com>

Thanks.

>  mkdir -p $SCRATCH_MNT/new
> -$TIMEOUT_PROG 10 cp -aRP $SOURCE_DIR $SCRATCH_MNT/new
> +$FSSTRESS_PROG -w -d "$SCRATCH_MNT/new" -n 20 -p 500 >> $seqres.full
>
>  _scratch_unmount
>
> @@ -78,8 +83,8 @@ $E2FSCK_PROG -fn $SCRATCH_DEV >> $seqres.full 2>&1 || \
>
>  # Mount the un-converted ext4 device & check the contents
>  mount -t ext4 $SCRATCH_DEV $SCRATCH_MNT
> -# (Ignore the symlinks which may be broken/nonexistent)
> -diff --no-dereference -r $SOURCE_DIR $SCRATCH_MNT/$BASENAME/ 2>&1
> +echo "Checking rolled back ext2 against the original one:"
> +$FSSUM_PROG -r $tmp.original $SCRATCH_MNT/$BASENAME
>
>  _scratch_unmount
>
> diff --git a/tests/btrfs/012.out b/tests/btrfs/012.out
> index 7aa5ae94..8ea81fad 100644
> --- a/tests/btrfs/012.out
> +++ b/tests/btrfs/012.out
> @@ -1 +1,7 @@
>  QA output created by 012
> +Checking converted btrfs against the original one:
> +OK
> +Checking saved ext2 image against the original one:
> +OK
> +Checking rolled back ext2 against the original one:
> +OK
> --
> 2.45.2
>
>
diff mbox series

Patch

diff --git a/tests/btrfs/012 b/tests/btrfs/012
index a96efeff..d172d93f 100755
--- a/tests/btrfs/012
+++ b/tests/btrfs/012
@@ -23,13 +23,13 @@  _require_scratch_nocheck
 _require_command "$BTRFS_CONVERT_PROG" btrfs-convert
 _require_command "$MKFS_EXT4_PROG" mkfs.ext4
 _require_command "$E2FSCK_PROG" e2fsck
+_require_fssum
 # ext4 does not support zoned block device
 _require_non_zoned_device "${SCRATCH_DEV}"
 _require_loop
 _require_extra_fs ext4
 
-SOURCE_DIR=/etc
-BASENAME=$(basename $SOURCE_DIR)
+BASENAME="stressdir"
 BLOCK_SIZE=`_get_block_size $TEST_DIR`
 
 # Create & populate an ext4 filesystem
@@ -38,17 +38,21 @@  $MKFS_EXT4_PROG -F -b $BLOCK_SIZE $SCRATCH_DEV > $seqres.full 2>&1 || \
 # Manual mount so we don't use -t btrfs or selinux context
 mount -t ext4 $SCRATCH_DEV $SCRATCH_MNT
 
-_require_fs_space $SCRATCH_MNT $(du -s $SOURCE_DIR | ${AWK_PROG} '{print $1}')
+echo "populating the initial ext fs:" >> $seqres.full
+mkdir "$SCRATCH_MNT/$BASENAME"
+$FSSTRESS_PROG -w -d "$SCRATCH_MNT/$BASENAME" -n 20 -p 500 >> $seqres.full
 
-$TIMEOUT_PROG 10 cp -aRP $SOURCE_DIR $SCRATCH_MNT
+# Create the checksum to verify later.
+$FSSUM_PROG -A -f -w $tmp.original "$SCRATCH_MNT/$BASENAME"
 _scratch_unmount
 
 # Convert it to btrfs, mount it, verify the data
 $BTRFS_CONVERT_PROG $SCRATCH_DEV >> $seqres.full 2>&1 || \
 	_fail "btrfs-convert failed"
 _try_scratch_mount || _fail "Could not mount new btrfs fs"
-# (Ignore the symlinks which may be broken/nonexistent)
-diff --no-dereference -r $SOURCE_DIR $SCRATCH_MNT/$BASENAME/ 2>&1
+
+echo "Checking converted btrfs against the original one:"
+$FSSUM_PROG -r $tmp.original $SCRATCH_MNT/$BASENAME
 
 # Old ext4 image file should exist & be consistent
 $E2FSCK_PROG -fn $SCRATCH_MNT/ext2_saved/image >> $seqres.full 2>&1 || \
@@ -58,13 +62,14 @@  $E2FSCK_PROG -fn $SCRATCH_MNT/ext2_saved/image >> $seqres.full 2>&1 || \
 mkdir -p $SCRATCH_MNT/mnt
 mount -o loop $SCRATCH_MNT/ext2_saved/image $SCRATCH_MNT/mnt || \
 	_fail "could not loop mount saved ext4 image"
-# Ignore the symlinks which may be broken/nonexistent
-diff --no-dereference -r $SOURCE_DIR $SCRATCH_MNT/mnt/$BASENAME/ 2>&1
+
+echo "Checking saved ext2 image against the original one:"
+$FSSUM_PROG -r $tmp.original $SCRATCH_MNT/mnt/$BASENAME
 umount $SCRATCH_MNT/mnt
 
-# Now put some fresh data on the btrfs fs
+echo "genereating new data on the converted btrfs" >> $seqres.full
 mkdir -p $SCRATCH_MNT/new 
-$TIMEOUT_PROG 10 cp -aRP $SOURCE_DIR $SCRATCH_MNT/new
+$FSSTRESS_PROG -w -d "$SCRATCH_MNT/new" -n 20 -p 500 >> $seqres.full
 
 _scratch_unmount
 
@@ -78,8 +83,8 @@  $E2FSCK_PROG -fn $SCRATCH_DEV >> $seqres.full 2>&1 || \
 
 # Mount the un-converted ext4 device & check the contents
 mount -t ext4 $SCRATCH_DEV $SCRATCH_MNT
-# (Ignore the symlinks which may be broken/nonexistent)
-diff --no-dereference -r $SOURCE_DIR $SCRATCH_MNT/$BASENAME/ 2>&1
+echo "Checking rolled back ext2 against the original one:"
+$FSSUM_PROG -r $tmp.original $SCRATCH_MNT/$BASENAME
 
 _scratch_unmount
 
diff --git a/tests/btrfs/012.out b/tests/btrfs/012.out
index 7aa5ae94..8ea81fad 100644
--- a/tests/btrfs/012.out
+++ b/tests/btrfs/012.out
@@ -1 +1,7 @@ 
 QA output created by 012
+Checking converted btrfs against the original one:
+OK
+Checking saved ext2 image against the original one:
+OK
+Checking rolled back ext2 against the original one:
+OK