From patchwork Mon May 16 08:59:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Chinner X-Patchwork-Id: 12850444 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 97CAFC433EF for ; Mon, 16 May 2022 08:59:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231253AbiEPI7d (ORCPT ); Mon, 16 May 2022 04:59:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50120 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229692AbiEPI73 (ORCPT ); Mon, 16 May 2022 04:59:29 -0400 Received: from mail104.syd.optusnet.com.au (mail104.syd.optusnet.com.au [211.29.132.246]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 24A411C904 for ; Mon, 16 May 2022 01:59:26 -0700 (PDT) Received: from dread.disaster.area (pa49-181-2-147.pa.nsw.optusnet.com.au [49.181.2.147]) by mail104.syd.optusnet.com.au (Postfix) with ESMTPS id B4A845348A8 for ; Mon, 16 May 2022 18:59:25 +1000 (AEST) Received: from discord.disaster.area ([192.168.253.110]) by dread.disaster.area with esmtp (Exim 4.92.3) (envelope-from ) id 1nqWZY-00CbZP-1G for fstests@vger.kernel.org; Mon, 16 May 2022 18:59:24 +1000 Received: from dave by discord.disaster.area with local (Exim 4.95) (envelope-from ) id 1nqWZY-005U49-0P for fstests@vger.kernel.org; Mon, 16 May 2022 18:59:24 +1000 From: Dave Chinner To: fstests@vger.kernel.org Subject: [PATCH 1/4] fstests: fix group list generation for whacky test names Date: Mon, 16 May 2022 18:59:19 +1000 Message-Id: <20220516085922.1306879-2-david@fromorbit.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220516085922.1306879-1-david@fromorbit.com> References: <20220516085922.1306879-1-david@fromorbit.com> MIME-Version: 1.0 X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.4 cv=VuxAv86n c=1 sm=1 tr=0 ts=6282126e a=ivVLWpVy4j68lT4lJFbQgw==:117 a=ivVLWpVy4j68lT4lJFbQgw==:17 a=oZkIemNP1mAA:10 a=20KFwNOVAAAA:8 a=-d-PVJR42_HGuCqAdrkA:9 Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org From: Dave Chinner Darrick noticed that tests/xfs/191-input-validation didn't get generated properly. Fix the regex to handle this. $ grep -I -R "^_begin_fstest" tests/xfs | \ sed -e 's/^.*\/\([0-9]*\):_begin_fstest/\1/' |grep 191 tests/xfs/191-input-validation:_begin_fstest auto quick mkfs realtime $ $ grep -I -R "^_begin_fstest" tests/xfs | \ sed -e 's/^.*\/\([0-9]*\).*:_begin_fstest/\1/ ' |grep 191 191 auto quick mkfs realtime $ Long term, we should rename that test to '191' and rip out all that unused and unnecessary complexity for matching ascii test names because we just don't use it. Numbers for tests are still working just fine. Signed-off-by: Dave Chinner --- tools/mkgroupfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/mkgroupfile b/tools/mkgroupfile index 24435898..958d4e2f 100755 --- a/tools/mkgroupfile +++ b/tools/mkgroupfile @@ -60,7 +60,7 @@ ENDL # Aggregate the groups each test belongs to for the group file grep -I -R "^_begin_fstest" $test_dir/ | \ - sed -e 's/^.*\/\([0-9]*\):_begin_fstest/\1/' >> $new_groups + sed -e 's/^.*\/\([0-9]*\).*:_begin_fstest/\1/' >> $new_groups # Create the list of unique groups for existence checking grep -I -R "^_begin_fstest" $test_dir/ | \ From patchwork Mon May 16 08:59:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Chinner X-Patchwork-Id: 12850445 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E7376C433FE for ; Mon, 16 May 2022 08:59:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229692AbiEPI7f (ORCPT ); Mon, 16 May 2022 04:59:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50118 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230070AbiEPI73 (ORCPT ); Mon, 16 May 2022 04:59:29 -0400 Received: from mail104.syd.optusnet.com.au (mail104.syd.optusnet.com.au [211.29.132.246]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 249811C903 for ; Mon, 16 May 2022 01:59:26 -0700 (PDT) Received: from dread.disaster.area (pa49-181-2-147.pa.nsw.optusnet.com.au [49.181.2.147]) by mail104.syd.optusnet.com.au (Postfix) with ESMTPS id B64AC5348AC for ; Mon, 16 May 2022 18:59:25 +1000 (AEST) Received: from discord.disaster.area ([192.168.253.110]) by dread.disaster.area with esmtp (Exim 4.92.3) (envelope-from ) id 1nqWZY-00CbZQ-2W for fstests@vger.kernel.org; Mon, 16 May 2022 18:59:24 +1000 Received: from dave by discord.disaster.area with local (Exim 4.95) (envelope-from ) id 1nqWZY-005U4D-1C for fstests@vger.kernel.org; Mon, 16 May 2022 18:59:24 +1000 From: Dave Chinner To: fstests@vger.kernel.org Subject: [PATCH 2/4] xfs/148: make test debuggable Date: Mon, 16 May 2022 18:59:20 +1000 Message-Id: <20220516085922.1306879-3-david@fromorbit.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220516085922.1306879-1-david@fromorbit.com> References: <20220516085922.1306879-1-david@fromorbit.com> MIME-Version: 1.0 X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.4 cv=VuxAv86n c=1 sm=1 tr=0 ts=6282126e a=ivVLWpVy4j68lT4lJFbQgw==:117 a=ivVLWpVy4j68lT4lJFbQgw==:17 a=oZkIemNP1mAA:10 a=20KFwNOVAAAA:8 a=shh0ZNA0_UVB9JGXS7UA:9 Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org From: Dave Chinner Don't clean up image files - leave them laying around on the test device so that when the test fails there's a corpse left behind to post-mortem.... Signed-off-by: Dave Chinner --- tests/xfs/148 | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/tests/xfs/148 b/tests/xfs/148 index 9427aff0..8d50a642 100755 --- a/tests/xfs/148 +++ b/tests/xfs/148 @@ -16,7 +16,7 @@ _cleanup() cd / $UMOUNT_PROG $mntpt > /dev/null 2>&1 test -n "$loopdev" && _destroy_loop_device $loopdev > /dev/null 2>&1 - rm -r -f $imgfile $mntpt $tmp.* + rm -r -f $tmp.* } # Import common functions. @@ -38,6 +38,8 @@ nullstr="too_many_beans" slashstr="are_bad_for_you" test_names=("something" "$nullstr" "$slashstr" "another") +rm -f $imgfile $imgfile.old + # Format image file w/o crcs so we can sed the image file $XFS_IO_PROG -f -c 'truncate 40m' $imgfile loopdev=$(_create_loop_device $imgfile) @@ -91,7 +93,6 @@ sed -b \ -i $imgfile test "$(md5sum < $imgfile)" != "$(md5sum < $imgfile.old)" || _fail "sed failed to change the image file?" -rm -f $imgfile.old loopdev=$(_create_loop_device $imgfile) _mount $loopdev $mntpt From patchwork Mon May 16 08:59:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Chinner X-Patchwork-Id: 12850446 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 08D36C4332F for ; Mon, 16 May 2022 08:59:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229820AbiEPI7f (ORCPT ); Mon, 16 May 2022 04:59:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50124 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230353AbiEPI73 (ORCPT ); Mon, 16 May 2022 04:59:29 -0400 Received: from mail105.syd.optusnet.com.au (mail105.syd.optusnet.com.au [211.29.132.249]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 24CF21C905 for ; Mon, 16 May 2022 01:59:26 -0700 (PDT) Received: from dread.disaster.area (pa49-181-2-147.pa.nsw.optusnet.com.au [49.181.2.147]) by mail105.syd.optusnet.com.au (Postfix) with ESMTPS id 950AA10E6A77 for ; Mon, 16 May 2022 18:59:25 +1000 (AEST) Received: from discord.disaster.area ([192.168.253.110]) by dread.disaster.area with esmtp (Exim 4.92.3) (envelope-from ) id 1nqWZY-00CbZR-44 for fstests@vger.kernel.org; Mon, 16 May 2022 18:59:24 +1000 Received: from dave by discord.disaster.area with local (Exim 4.95) (envelope-from ) id 1nqWZY-005U4H-2g for fstests@vger.kernel.org; Mon, 16 May 2022 18:59:24 +1000 From: Dave Chinner To: fstests@vger.kernel.org Subject: [PATCH 3/4] xfs/148: fix failure from bad shortform size assumptions Date: Mon, 16 May 2022 18:59:21 +1000 Message-Id: <20220516085922.1306879-4-david@fromorbit.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220516085922.1306879-1-david@fromorbit.com> References: <20220516085922.1306879-1-david@fromorbit.com> MIME-Version: 1.0 X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.4 cv=deDjYVbe c=1 sm=1 tr=0 ts=6282126e a=ivVLWpVy4j68lT4lJFbQgw==:117 a=ivVLWpVy4j68lT4lJFbQgw==:17 a=oZkIemNP1mAA:10 a=20KFwNOVAAAA:8 a=mDPyBUI4OX8v4RjPpF4A:9 Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org From: Dave Chinner We replaced an attr named: slashstr="are_bad_for_you" with this substitution: cp $imgfile $imgfile.old sed -b \ -e "s/$nullstr/too_many\x00beans/g" \ -e "s/$slashstr/are_bad\/for_you/g" \ -i $imgfile We then try to retreive the attr named 'a_are_bad/for_you'. The failure is: -Attribute "a_are_bad/for_you" had a 3 byte value for TEST_DIR/mount-148/testfile: -heh +attr_get: No data available +Could not get "a_are_bad/for_you" for TEST_DIR/mount-148/testfile The error returned is ENODATA - the xattr does not exist. While the name might exist in the attr leaf block: .... nvlist[0].valuelen = 3 nvlist[0].namelen = 17 nvlist[0].name = "a_are_bad/for_you" nvlist[0].value = "heh" nvlist[1].valuelen = 3 .... xattrs are not looked up by name matches when in leaf or node form like they are in short form. They are looked up by *name hash* matches, and if the hash is not found, then the name does not exist. Only if the has match is found, then it goes and retrieves the xattr pointed to by the hash and checks the name. At this point, it should be obvious that the hash of "a_are_bad_for_you" is different to "a_a_are_bad/for_you". Hence the leaf lookup is always rejected at the hash match stage and never gets to the name compare stage. IOWs, this test can *never* pass when the xattr is in leaf/node form, only when it is in short form. The reason the attr fork is in leaf form is that we are using "-m crc=0" and so the inodes are only 256 bytes in size and can only hold ~150 bytes in the literal area. That leaves ~100 bytes maximum for shortform attr data. The test consumes ~80 bytes of shortform space, so it should fit and the test pass. However: nvlist[4].valuelen = 37 nvlist[4].namelen = 7 nvlist[4].name = "selinux" nvlist[4].value = "unconfined_u:object_r:unlabeled_t:s0\000" Yes, I run the fstests with selinux enabled on some of test machines. The selinux attr pushes the attr fork way over the size that can fit in the shortform literal area, and so it moves to leaf form as the attrs are initially added and the test fails. Fix this by forcing the test to use 512 byte inodes, so as to provide around 350 bytes of usable attr fork literal area so it's not affected by security attributes. While there, clean up the silly conditional loop device cleanup code. Signed-off-by: Dave Chinner Reviewed-by: Darrick J. Wong --- tests/xfs/148 | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/tests/xfs/148 b/tests/xfs/148 index 8d50a642..5d0a0bf4 100755 --- a/tests/xfs/148 +++ b/tests/xfs/148 @@ -15,7 +15,7 @@ _cleanup() { cd / $UMOUNT_PROG $mntpt > /dev/null 2>&1 - test -n "$loopdev" && _destroy_loop_device $loopdev > /dev/null 2>&1 + _destroy_loop_device $loopdev > /dev/null 2>&1 rm -r -f $tmp.* } @@ -41,9 +41,12 @@ test_names=("something" "$nullstr" "$slashstr" "another") rm -f $imgfile $imgfile.old # Format image file w/o crcs so we can sed the image file +# We need to use 512 byte inodes to ensure the attr forks remain in short form +# even when security xattrs are present so we are always doing name matches on +# lookup and not name hash compares as leaf/node forms will do. $XFS_IO_PROG -f -c 'truncate 40m' $imgfile loopdev=$(_create_loop_device $imgfile) -MKFS_OPTIONS="-m crc=0" _mkfs_dev $loopdev >> $seqres.full +MKFS_OPTIONS="-m crc=0 -i size=512" _mkfs_dev $loopdev >> $seqres.full # Mount image file mkdir -p $mntpt @@ -121,9 +124,6 @@ res=$? test $res -eq 1 || \ echo "repair failed to report corruption ($res)" -_destroy_loop_device $loopdev -loopdev= - # success, all done status=0 exit From patchwork Mon May 16 08:59:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Chinner X-Patchwork-Id: 12850447 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E5A40C433EF for ; Mon, 16 May 2022 08:59:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230070AbiEPI7g (ORCPT ); Mon, 16 May 2022 04:59:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50184 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231337AbiEPI7d (ORCPT ); Mon, 16 May 2022 04:59:33 -0400 Received: from mail104.syd.optusnet.com.au (mail104.syd.optusnet.com.au [211.29.132.246]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 4CBAF1C919 for ; Mon, 16 May 2022 01:59:29 -0700 (PDT) Received: from dread.disaster.area (pa49-181-2-147.pa.nsw.optusnet.com.au [49.181.2.147]) by mail104.syd.optusnet.com.au (Postfix) with ESMTPS id B46DD5348A5 for ; Mon, 16 May 2022 18:59:25 +1000 (AEST) Received: from discord.disaster.area ([192.168.253.110]) by dread.disaster.area with esmtp (Exim 4.92.3) (envelope-from ) id 1nqWZY-00CbZW-5W for fstests@vger.kernel.org; Mon, 16 May 2022 18:59:24 +1000 Received: from dave by discord.disaster.area with local (Exim 4.95) (envelope-from ) id 1nqWZY-005U4L-47 for fstests@vger.kernel.org; Mon, 16 May 2022 18:59:24 +1000 From: Dave Chinner To: fstests@vger.kernel.org Subject: [PATCH 4/4] generic/081: don't run on DAX capable devices Date: Mon, 16 May 2022 18:59:22 +1000 Message-Id: <20220516085922.1306879-5-david@fromorbit.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220516085922.1306879-1-david@fromorbit.com> References: <20220516085922.1306879-1-david@fromorbit.com> MIME-Version: 1.0 X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.4 cv=e9dl9Yl/ c=1 sm=1 tr=0 ts=6282126e a=ivVLWpVy4j68lT4lJFbQgw==:117 a=ivVLWpVy4j68lT4lJFbQgw==:17 a=oZkIemNP1mAA:10 a=20KFwNOVAAAA:8 a=ZX5orWydIAkmiaoJ8_sA:9 Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org From: Dave Chinner LVM/DM has conniptions when you try to use snapshots on a device that has DAX capability. It first sets up the underlying device as a DAX capable mapping (type 3 or DM_TYPE_DAX_BIO_BASED) but because snapshots require COW and shared mappings, it isn't supported on DAX capable devices. Hence creating the snapshot device fails because it requires a type 1 (DM_TYPE_BIO_BASED) device and DM can't change types on a loaded mapping. Hence we get this obscure error message in the log: device-mapper: ioctl: can't change device type (old=3 vs new=1) after initial table load. and these obscure, unhelpful error messages from the LVM command outputs: device-mapper: reload ioctl on (251:0) failed: Invalid argument Failed to suspend logical volume vg_081/base_081. Device vg_081-base_081-real (251:1) is used by another device. Failed to revert logical volume vg_081/base_081. Aborting. Manual intervention required. Failed to create snapshot How to turn off DAX capability is not documented in dmsetup or LVM man pages, nor is dax mentioned anywhere in Documentation/admin/device-mapper/ so I have no idea how to tell LVM/DM "don't try to enable DAX support!". As such, if the uderlying block device is dax capable, skip this test. Signed-off-by: Dave Chinner --- common/rc | 33 +++++++++++++++++++++++---------- 1 file changed, 23 insertions(+), 10 deletions(-) diff --git a/common/rc b/common/rc index 4201a059..f5ead044 100644 --- a/common/rc +++ b/common/rc @@ -2167,7 +2167,7 @@ _require_sane_bdev_flush() # 3. "dax=inode" or nothing means "use scratch dev capability" to # determine whether DAX is going to be used. # -# Returns 0 if DAX will be used, 1 if DAX is not going to be used. +# Returns 0 if the filesytem will use DAX, 1 if it won't. __scratch_uses_fsdax() { local ops=$(_normalize_mount_options "$MOUNT_OPTIONS") @@ -2175,9 +2175,19 @@ __scratch_uses_fsdax() echo $ops | egrep -qw "dax(=always| |$)" && return 0 echo $ops | grep -qw "dax=never" && return 1 + return 0 +} + +# Determine if the scratch device is DAX capable. Every if the fs is not +# using DAX, we still can't use certain device mapper targets if the block +# device is DAX capable. hence the check needs to be separat from the FS +# capability. +__scratch_dev_has_dax() +{ local sysfs="/sys/block/$(_short_dev $SCRATCH_DEV)" test -e "${sysfs}/dax" && return 0 test "$(cat "${sysfs}/queue/dax" 2>/dev/null)" = "1" && return 0 + return 1 } @@ -2194,15 +2204,18 @@ _require_dm_target() _require_sane_bdev_flush $SCRATCH_DEV _require_command "$DMSETUP_PROG" dmsetup - if __scratch_uses_fsdax; then - case $target in - stripe|linear|log-writes) - ;; - *) - _notrun "Cannot run tests with DAX on $target devices." - ;; - esac - fi + case $target in + stripe|linear|log-writes) + ;; + *) + if __scratch_uses_fsdax; then + _notrun "Cannot run tests with fsdax on $target devices." + fi + if __scratch_dev_has_dax; then + _notrun "Cannot use $target devices on DAX capable block devices." + fi + ;; + esac modprobe dm-$target >/dev/null 2>&1