From patchwork Tue Mar 9 04:40:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 12123993 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E46EFC43333 for ; Tue, 9 Mar 2021 04:40:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BA14C65275 for ; Tue, 9 Mar 2021 04:40:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230139AbhCIEkM (ORCPT ); Mon, 8 Mar 2021 23:40:12 -0500 Received: from mail.kernel.org ([198.145.29.99]:60988 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229872AbhCIEkJ (ORCPT ); Mon, 8 Mar 2021 23:40:09 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id 684B86523B; Tue, 9 Mar 2021 04:40:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1615264809; bh=won8P26HD3UWLjx3SPmMJ0Gn5xM9EYpQhFiLLUgkBGQ=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=p9e0tImGdK0aoTQgmKIq43PpNocxVXDYHp0bCWusyjLbM5fsXUWT8DR7h4B9JLUKR AwiP2wgXUCnnNTYtPdiLlsLw0gLndbCdKClFUF04UbFUTKhY6owIG3TxO3QCUPuSwW DlAfrOZA5gZKuUmCF4kEsIe4xuBGvYg56VeuqbFRnh5V9o7vuu0V6cNgAd8cKu2GZZ T0CpExWxdmrJ+UIH8ltbGc9urRN5IJmYMrPbpMTQxZ98ZjBgi/T1HpT7QRECDfAkHn 5WazVlRQfxKgk0rd8yKJ8icgHWwyW0YLVMXIfIBmkeMGmMsQ7MEODZs041oi0maFqK Kuv9MAZfqYUsg== Subject: [PATCH 01/10] xfs: test regression in xfs_bmap_validate_extent From: "Darrick J. Wong" To: djwong@kernel.org, guaneryu@gmail.com Cc: Chandan Babu R , linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me Date: Mon, 08 Mar 2021 20:40:09 -0800 Message-ID: <161526480925.1214319.17395174541667296356.stgit@magnolia> In-Reply-To: <161526480371.1214319.3263690953532787783.stgit@magnolia> References: <161526480371.1214319.3263690953532787783.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org From: Darrick J. Wong This is a regression test to make sure that we can have realtime files with xattr blocks and not trip the verifiers. Signed-off-by: Darrick J. Wong Reviewed-by: Chandan Babu R --- tests/xfs/758 | 59 +++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/758.out | 2 ++ tests/xfs/group | 1 + 3 files changed, 62 insertions(+) create mode 100755 tests/xfs/758 create mode 100644 tests/xfs/758.out diff --git a/tests/xfs/758 b/tests/xfs/758 new file mode 100755 index 00000000..a247701e --- /dev/null +++ b/tests/xfs/758 @@ -0,0 +1,59 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2021 Oracle. All Rights Reserved. +# +# FS QA Test No. 758 +# +# This is a regression test for commit d0c20d38af13 "xfs: fix +# xfs_bmap_validate_extent_raw when checking attr fork of rt files", which +# fixes the bmap record validator so that it will not check the attr fork +# extent mappings of a realtime file against the size of the realtime volume. + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter + +# real QA test starts here +_supported_fs xfs +_require_scratch +_require_realtime + +rm -f $seqres.full + +# Format filesystem with very tiny realtime volume +_scratch_mkfs -r size=256k > $seqres.full +_scratch_mount >> $seqres.full + +# Create a realtime file +$XFS_IO_PROG -f -R -c 'pwrite 0 64k' -c stat $SCRATCH_MNT/v >> $seqres.full + +# Add enough xattr data to force creation of xattr blocks at a higher address +# on the data device than the size of the realtime volume +for i in `seq 0 16`; do + $ATTR_PROG -s user.test$i $SCRATCH_MNT/v < $SCRATCH_MNT/v >> $seqres.full +done + +# Force flushing extent maps to disk to trip the verifier +_scratch_cycle_mount + +# Now let that unmount +echo Silence is golden. + +# success, all done +status=0 +exit diff --git a/tests/xfs/758.out b/tests/xfs/758.out new file mode 100644 index 00000000..6d219f8e --- /dev/null +++ b/tests/xfs/758.out @@ -0,0 +1,2 @@ +QA output created by 758 +Silence is golden. diff --git a/tests/xfs/group b/tests/xfs/group index 754fc7be..4dd9901f 100644 --- a/tests/xfs/group +++ b/tests/xfs/group @@ -500,4 +500,5 @@ 525 auto quick mkfs 526 auto quick mkfs 527 auto quick quota +758 auto quick rw attr realtime 763 auto quick rw realtime From patchwork Tue Mar 9 04:40:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 12124005 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, LOTS_OF_MONEY,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D0BAFC433E0 for ; Tue, 9 Mar 2021 04:41:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A54CF6528D for ; Tue, 9 Mar 2021 04:41:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229929AbhCIEkm (ORCPT ); Mon, 8 Mar 2021 23:40:42 -0500 Received: from mail.kernel.org ([198.145.29.99]:32860 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229872AbhCIEkP (ORCPT ); Mon, 8 Mar 2021 23:40:15 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id D15B36523B; Tue, 9 Mar 2021 04:40:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1615264814; bh=UrcceFFjmTQRogHNHnizLL7HLDHcmj2usP6zC065DlY=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=cV7C0E4k91au0OYExa9NaaogUcPqbyh7R9/D+bpa0Ogab7ZHT4hHFoBylbH99THNp 3vu/fUO2HSXjm8sUtMO8CAHIuEAtB/uFamMvohVoym82wb9+O58Sp+6EVWRkYUbwZv eHMM9mDnHYBuiDnSl4A0AZmWpVOLcGowqgtFByMkVCimDmWXJW48JNbGcuj3WU8xHt Jf6WStOoSuOqSAN+XZAhnJ68Z6HncdLF7TCR4kBAD2jx1XaEGEaDID7kgDOQ/YMjbk KKmnGWtym9LVYVIIGtPaYWCQfsjkTTRYjDNC1NMTWsLL/FqxJ1rfOZXf2Lz7apcDLy lQTKttKJFL7/g== Subject: [PATCH 02/10] generic: test reflink and copy_file_range behavior with O_SYNC and FS_XFLAG_SYNC files From: "Darrick J. Wong" To: djwong@kernel.org, guaneryu@gmail.com Cc: Chandan Babu R , linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me Date: Mon, 08 Mar 2021 20:40:14 -0800 Message-ID: <161526481473.1214319.4844099354726360669.stgit@magnolia> In-Reply-To: <161526480371.1214319.3263690953532787783.stgit@magnolia> References: <161526480371.1214319.3263690953532787783.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org From: Darrick J. Wong Add two regression tests to make sure that FICLONERANGE and the splice based copy_file_range actually flush all data and metadata to disk before the call ends. Signed-off-by: Darrick J. Wong Reviewed-by: Chandan Babu R --- tests/generic/947 | 118 +++++++++++++++++++++++++++++++++++++++++++++++++ tests/generic/947.out | 15 ++++++ tests/generic/948 | 92 ++++++++++++++++++++++++++++++++++++++ tests/generic/948.out | 9 ++++ tests/generic/group | 2 + 5 files changed, 236 insertions(+) create mode 100755 tests/generic/947 create mode 100644 tests/generic/947.out create mode 100755 tests/generic/948 create mode 100644 tests/generic/948.out diff --git a/tests/generic/947 b/tests/generic/947 new file mode 100755 index 00000000..d0edb876 --- /dev/null +++ b/tests/generic/947 @@ -0,0 +1,118 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2021 Oracle. All Rights Reserved. +# +# FS QA Test No. 947 +# +# Make sure that reflink forces the log out if we open the file with O_SYNC or +# set FS_XFLAG_SYNC on the file. We test that it actually forced the log by +# using dm-error to shut down the fs without flushing the log and then +# remounting to check file contents. This is a regression test for commit +# 5ffce3cc22a0 ("xfs: force the log after remapping a synchronous-writes file") + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* + _dmerror_unmount + _dmerror_cleanup +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter +. ./common/reflink +. ./common/dmerror + +# real QA test starts here +_supported_fs generic +_require_dm_target error +_require_scratch_reflink +_require_xfs_io_command "chattr" "s" +_require_cp_reflink + +rm -f $seqres.full + +# Format filesystem and set up quota limits +_scratch_mkfs > $seqres.full +_require_metadata_journaling $SCRATCH_DEV +_dmerror_init +_dmerror_mount + +# Test that O_SYNC actually results in file data being written even if the +# fs immediately dies +echo "test o_sync write" +$XFS_IO_PROG -x -f -s -c "pwrite -S 0x58 0 1m -b 1m" $SCRATCH_MNT/0 >> $seqres.full +_dmerror_load_error_table +_dmerror_unmount +_dmerror_load_working_table +_dmerror_mount +md5sum $SCRATCH_MNT/0 | _filter_scratch + +# Set up initial files for reflink test +$XFS_IO_PROG -f -c 'pwrite -S 0x58 0 1m -b 1m' $SCRATCH_MNT/a >> $seqres.full +$XFS_IO_PROG -f -c 'pwrite -S 0x59 0 1m -b 1m' $SCRATCH_MNT/c >> $seqres.full +_cp_reflink $SCRATCH_MNT/a $SCRATCH_MNT/e +_cp_reflink $SCRATCH_MNT/c $SCRATCH_MNT/d +touch $SCRATCH_MNT/b +sync + +# Test that reflink forces dirty data/metadata to disk when destination file +# opened with O_SYNC +echo "test reflink flag not set o_sync" +$XFS_IO_PROG -x -s -c "reflink $SCRATCH_MNT/a" $SCRATCH_MNT/b >> $seqres.full +_dmerror_load_error_table +_dmerror_unmount +_dmerror_load_working_table +_dmerror_mount +md5sum $SCRATCH_MNT/a $SCRATCH_MNT/b | _filter_scratch + +# Test that reflink to a shared file forces dirty data/metadata to disk when +# destination is opened with O_SYNC +echo "test reflink flag already set o_sync" +$XFS_IO_PROG -x -s -c "reflink $SCRATCH_MNT/a" $SCRATCH_MNT/d >> $seqres.full +_dmerror_load_error_table +_dmerror_unmount +_dmerror_load_working_table +_dmerror_mount +md5sum $SCRATCH_MNT/a $SCRATCH_MNT/d | _filter_scratch + +# Set up the two files with chattr +S +rm -f $SCRATCH_MNT/b $SCRATCH_MNT/d +_cp_reflink $SCRATCH_MNT/c $SCRATCH_MNT/d +touch $SCRATCH_MNT/b +chattr +S $SCRATCH_MNT/b $SCRATCH_MNT/d +sync + +# Test that reflink forces dirty data/metadata to disk when destination file +# has the sync iflag set +echo "test reflink flag not set iflag" +$XFS_IO_PROG -x -c "reflink $SCRATCH_MNT/a" $SCRATCH_MNT/b >> $seqres.full +_dmerror_load_error_table +_dmerror_unmount +_dmerror_load_working_table +_dmerror_mount +md5sum $SCRATCH_MNT/a $SCRATCH_MNT/b | _filter_scratch + +# Test that reflink to a shared file forces dirty data/metadata to disk when +# destination file has the sync iflag set +echo "test reflink flag already set iflag" +$XFS_IO_PROG -x -c "reflink $SCRATCH_MNT/a" $SCRATCH_MNT/d >> $seqres.full +_dmerror_load_error_table +_dmerror_unmount +_dmerror_load_working_table +_dmerror_mount +md5sum $SCRATCH_MNT/a $SCRATCH_MNT/d | _filter_scratch + +# success, all done +status=0 +exit diff --git a/tests/generic/947.out b/tests/generic/947.out new file mode 100644 index 00000000..05ba10d1 --- /dev/null +++ b/tests/generic/947.out @@ -0,0 +1,15 @@ +QA output created by 947 +test o_sync write +310f146ce52077fcd3308dcbe7632bb2 SCRATCH_MNT/0 +test reflink flag not set o_sync +310f146ce52077fcd3308dcbe7632bb2 SCRATCH_MNT/a +310f146ce52077fcd3308dcbe7632bb2 SCRATCH_MNT/b +test reflink flag already set o_sync +310f146ce52077fcd3308dcbe7632bb2 SCRATCH_MNT/a +310f146ce52077fcd3308dcbe7632bb2 SCRATCH_MNT/d +test reflink flag not set iflag +310f146ce52077fcd3308dcbe7632bb2 SCRATCH_MNT/a +310f146ce52077fcd3308dcbe7632bb2 SCRATCH_MNT/b +test reflink flag already set iflag +310f146ce52077fcd3308dcbe7632bb2 SCRATCH_MNT/a +310f146ce52077fcd3308dcbe7632bb2 SCRATCH_MNT/d diff --git a/tests/generic/948 b/tests/generic/948 new file mode 100755 index 00000000..b79cd279 --- /dev/null +++ b/tests/generic/948 @@ -0,0 +1,92 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2021 Oracle. All Rights Reserved. +# +# FS QA Test No. 948 +# +# Make sure that copy_file_range forces the log out if we open the file with +# O_SYNC or set FS_XFLAG_SYNC on the file. We test that it actually forced the +# log by using dm-error to shut down the fs without flushing the log and then +# remounting to check file contents. This is a regression test for commit +# 5ffce3cc22a0 ("xfs: force the log after remapping a synchronous-writes file") + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* + _dmerror_unmount + _dmerror_cleanup +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter +. ./common/dmerror + +# real QA test starts here +_supported_fs generic +_require_dm_target error +_require_xfs_io_command "chattr" "s" +_require_scratch + +rm -f $seqres.full + +# Format filesystem and set up quota limits +_scratch_mkfs > $seqres.full +_require_metadata_journaling $SCRATCH_DEV +_dmerror_init +_dmerror_mount + +# Test that O_SYNC actually results in file data being written even if the +# fs immediately dies +echo "test o_sync write" +$XFS_IO_PROG -x -f -s -c "pwrite -S 0x58 0 1m -b 1m" $SCRATCH_MNT/0 >> $seqres.full +_dmerror_load_error_table +_dmerror_unmount +_dmerror_load_working_table +_dmerror_mount +md5sum $SCRATCH_MNT/0 | _filter_scratch + +# Set up initial files for copy test +$XFS_IO_PROG -f -c 'pwrite -S 0x58 0 1m -b 1m' $SCRATCH_MNT/a >> $seqres.full +touch $SCRATCH_MNT/b +sync + +# Test that unaligned copy file range forces dirty data/metadata to disk when +# destination file opened with O_SYNC +echo "test unaligned copy range o_sync" +$XFS_IO_PROG -x -s -c "copy_range -s 13 -d 13 -l 1048550 $SCRATCH_MNT/a" $SCRATCH_MNT/b >> $seqres.full +_dmerror_load_error_table +_dmerror_unmount +_dmerror_load_working_table +_dmerror_mount +md5sum $SCRATCH_MNT/a $SCRATCH_MNT/b | _filter_scratch + +# Set up dest file with chattr +S +rm -f $SCRATCH_MNT/b +touch $SCRATCH_MNT/b +chattr +S $SCRATCH_MNT/b +sync + +# Test that unaligned copy file range forces dirty data/metadata to disk when +# destination file has the sync iflag set +echo "test unaligned copy range iflag" +$XFS_IO_PROG -x -c "copy_range -s 13 -d 13 -l 1048550 $SCRATCH_MNT/a" $SCRATCH_MNT/b >> $seqres.full +_dmerror_load_error_table +_dmerror_unmount +_dmerror_load_working_table +_dmerror_mount +md5sum $SCRATCH_MNT/a $SCRATCH_MNT/b | _filter_scratch + +# success, all done +status=0 +exit diff --git a/tests/generic/948.out b/tests/generic/948.out new file mode 100644 index 00000000..eec6c0dc --- /dev/null +++ b/tests/generic/948.out @@ -0,0 +1,9 @@ +QA output created by 948 +test o_sync write +310f146ce52077fcd3308dcbe7632bb2 SCRATCH_MNT/0 +test unaligned copy range o_sync +310f146ce52077fcd3308dcbe7632bb2 SCRATCH_MNT/a +2a715d2093b5aca82783a0c5943ac0b8 SCRATCH_MNT/b +test unaligned copy range iflag +310f146ce52077fcd3308dcbe7632bb2 SCRATCH_MNT/a +2a715d2093b5aca82783a0c5943ac0b8 SCRATCH_MNT/b diff --git a/tests/generic/group b/tests/generic/group index 84db3789..d5cfdd51 100644 --- a/tests/generic/group +++ b/tests/generic/group @@ -628,3 +628,5 @@ 623 auto quick shutdown 624 auto quick verity 625 auto quick verity +947 auto quick rw clone +948 auto quick rw copy_range From patchwork Tue Mar 9 04:40:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 12124013 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 87BDAC43333 for ; Tue, 9 Mar 2021 04:41:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5F47C6528A for ; Tue, 9 Mar 2021 04:41:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229872AbhCIEkm (ORCPT ); Mon, 8 Mar 2021 23:40:42 -0500 Received: from mail.kernel.org ([198.145.29.99]:32878 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230140AbhCIEkU (ORCPT ); Mon, 8 Mar 2021 23:40:20 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id 5102065275; Tue, 9 Mar 2021 04:40:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1615264820; bh=/34Q4K6Ae87lq3+BnoM2xICN4SUmvFlYJ9HGAm/kpHg=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=FYvfkGSg0abVCGbbgl6DG5Uvi3UYCR8hzrJtfonQvdeGa+Ydd5T55+G8gIfbFicWn gj+OXuXkJFqvFRACeNg1gRVqeOZzlt+xCllh7zV7QCMEYtV0Y5aosA/o1n6z2QAKAn hYEd7dqXnz4oyIKtvgI9lVjRmFNG8wkA65YyZve+afG2t4wJp26eEiIEZ0hHfcRpIt w1r0EsOY6ZBAAFMUVT5aq+aGzPQkU2RcKEVSlheX7ON5gCKf/3qIgTsgL1BWWfk17F eh4lwZqcf1M37SPBlrlaolMtxEE6SLYkToUkT/fkYFtybYkXf5gShJOf49fiDXenpC knWzlO8TlkaqQ== Subject: [PATCH 03/10] xfs: test rtalloc alignment and math errors From: "Darrick J. Wong" To: djwong@kernel.org, guaneryu@gmail.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me Date: Mon, 08 Mar 2021 20:40:20 -0800 Message-ID: <161526482015.1214319.6227125326960502859.stgit@magnolia> In-Reply-To: <161526480371.1214319.3263690953532787783.stgit@magnolia> References: <161526480371.1214319.3263690953532787783.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org From: Darrick J. Wong Add a couple of regression tests for "xfs: make sure the rt allocator doesn't run off the end" and "xfs: ensure that fpunch, fcollapse, and finsert operations are aligned to rt extent size". Signed-off-by: Darrick J. Wong Reviewed-by: Chandan Babu R --- tests/xfs/759 | 100 +++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/759.out | 2 + tests/xfs/760 | 68 ++++++++++++++++++++++++++++++++++++ tests/xfs/760.out | 9 +++++ tests/xfs/group | 2 + 5 files changed, 181 insertions(+) create mode 100755 tests/xfs/759 create mode 100644 tests/xfs/759.out create mode 100755 tests/xfs/760 create mode 100644 tests/xfs/760.out diff --git a/tests/xfs/759 b/tests/xfs/759 new file mode 100755 index 00000000..8558fe30 --- /dev/null +++ b/tests/xfs/759 @@ -0,0 +1,100 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2021 Oracle. All Rights Reserved. +# +# FS QA Test No. 759 +# +# This is a regression test for commit 2a6ca4baed62 ("xfs: make sure the rt +# allocator doesn't run off the end") which fixes an overflow error in the +# _near realtime allocator. If the rt bitmap ends exactly at the end of a +# block and the number of rt extents is large enough to allow an allocation +# request larger than the maximum extent size, it's possible that during a +# large allocation request, the allocator will fail to constrain maxlen on the +# second run through the loop, and the rt bitmap range check will run right off +# the end of the rtbitmap file. When this happens, xfs triggers a verifier +# error and returns EFSCORRUPTED. + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter + +# real QA test starts here +_supported_fs xfs +_require_scratch +_require_realtime +_require_test_program "punch-alternating" + +rm -f $seqres.full + +# Format filesystem to get the block size +_scratch_mkfs > $seqres.full +_scratch_mount >> $seqres.full + +blksz=$(_get_block_size $SCRATCH_MNT) +rextsize=$($XFS_INFO_PROG $SCRATCH_MNT | grep realtime.*extsz | sed -e 's/^.*extsz=\([0-9]*\).*$/\1/g') +rextblks=$((rextsize / blksz)) + +echo "blksz $blksz rextsize $rextsize rextblks $rextblks" >> $seqres.full + +_scratch_unmount + +# Format filesystem with a realtime volume whose size fits the following: +# 1. Longer than (XFS MAXEXTLEN * blocksize) bytes. +# 2. Exactly a multiple of (NBBY * blksz * rextsize) bytes. + +rtsize1=$((2097151 * blksz)) +rtsize2=$((8 * blksz * rextsize)) +rtsize=$(( $(blockdev --getsz $SCRATCH_RTDEV) * 512 )) + +echo "rtsize1 $rtsize1 rtsize2 $rtsize2 rtsize $rtsize" >> $seqres.full + +test $rtsize -gt $rtsize1 || \ + _notrun "scratch rt device too small, need $rtsize1 bytes" +test $rtsize -gt $rtsize2 || \ + _notrun "scratch rt device too small, need $rtsize2 bytes" + +rtsize=$((rtsize - (rtsize % rtsize2))) + +echo "rt size will be $rtsize" >> $seqres.full + +_scratch_mkfs -r size=$rtsize >> $seqres.full +_scratch_mount >> $seqres.full + +# Make sure the root directory has rtinherit set so our test file will too +$XFS_IO_PROG -c 'chattr +t' $SCRATCH_MNT + +# Allocate some stuff at the start, to force the first falloc of the ouch file +# to happen somewhere in the middle of the rt volume +$XFS_IO_PROG -f -c 'falloc 0 64m' "$SCRATCH_MNT/b" +$here/src/punch-alternating -i $((rextblks * 2)) -s $((rextblks)) "$SCRATCH_MNT/b" + +avail="$(df -P "$SCRATCH_MNT" | awk 'END {print $4}')"1 +toobig="$((avail * 2))" + +# falloc the ouch file in the middle of the rt extent to exercise the near +# allocator in the last step. +$XFS_IO_PROG -f -c 'falloc 0 1g' "$SCRATCH_MNT/ouch" + +# Try to get the near allocator to overflow on an allocation that matches +# exactly one of the rtsummary size levels. This should return ENOSPC and +# not EFSCORRUPTED. +$XFS_IO_PROG -f -c "falloc 0 ${toobig}k" "$SCRATCH_MNT/ouch" + +# success, all done +status=0 +exit diff --git a/tests/xfs/759.out b/tests/xfs/759.out new file mode 100644 index 00000000..df693d50 --- /dev/null +++ b/tests/xfs/759.out @@ -0,0 +1,2 @@ +QA output created by 759 +fallocate: No space left on device diff --git a/tests/xfs/760 b/tests/xfs/760 new file mode 100755 index 00000000..7f2b52d4 --- /dev/null +++ b/tests/xfs/760 @@ -0,0 +1,68 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2021 Oracle. All Rights Reserved. +# +# FS QA Test No. 760 +# +# Make sure we validate realtime extent size alignment for fallocate modes. +# This is a regression test for fe341eb151ec ("xfs: ensure that fpunch, +# fcollapse, and finsert operations are aligned to rt extent size") + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter + +# real QA test starts here +_supported_fs xfs +_require_scratch +_require_realtime +_require_xfs_io_command "fcollapse" +_require_xfs_io_command "finsert" +_require_xfs_io_command "funshare" +_require_xfs_io_command "fzero" +_require_xfs_io_command "falloc" + +rm -f $seqres.full + +# Format filesystem with a 256k realtime extent size +_scratch_mkfs -r extsize=256k > $seqres.full +_scratch_mount >> $seqres.full + +blksz=$(_get_block_size $SCRATCH_MNT) +rextsize=$($XFS_INFO_PROG $SCRATCH_MNT | grep realtime.*extsz | sed -e 's/^.*extsz=\([0-9]*\).*$/\1/g') +rextblks=$((rextsize / blksz)) + +echo "blksz $blksz rextsize $rextsize rextblks $rextblks" >> $seqres.full + +# Make sure the root directory has rtinherit set so our test file will too +$XFS_IO_PROG -c 'chattr +t' $SCRATCH_MNT + +sz=$((rextsize * 100)) +range="$((blksz * 3)) $blksz" + +for verb in fpunch finsert fcollapse fzero funshare falloc; do + echo "test $verb" + $XFS_IO_PROG -f -c "falloc 0 $sz" "$SCRATCH_MNT/b" + $XFS_IO_PROG -f -c "$verb $range" "$SCRATCH_MNT/b" + rm -f "$SCRATCH_MNT/b" + _scratch_cycle_mount +done + +# success, all done +status=0 +exit diff --git a/tests/xfs/760.out b/tests/xfs/760.out new file mode 100644 index 00000000..3d73c6fa --- /dev/null +++ b/tests/xfs/760.out @@ -0,0 +1,9 @@ +QA output created by 760 +test fpunch +test finsert +fallocate: Invalid argument +test fcollapse +fallocate: Invalid argument +test fzero +test funshare +test falloc diff --git a/tests/xfs/group b/tests/xfs/group index 4dd9901f..318468b5 100644 --- a/tests/xfs/group +++ b/tests/xfs/group @@ -501,4 +501,6 @@ 526 auto quick mkfs 527 auto quick quota 758 auto quick rw attr realtime +759 auto quick rw realtime +760 auto quick rw collapse punch insert zero prealloc 763 auto quick rw realtime From patchwork Tue Mar 9 04:40:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 12124011 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 48F70C4332D for ; Tue, 9 Mar 2021 04:41:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2B291652A4 for ; Tue, 9 Mar 2021 04:41:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230050AbhCIEkm (ORCPT ); Mon, 8 Mar 2021 23:40:42 -0500 Received: from mail.kernel.org ([198.145.29.99]:32896 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229772AbhCIEk0 (ORCPT ); Mon, 8 Mar 2021 23:40:26 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id C24AD65275; Tue, 9 Mar 2021 04:40:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1615264825; bh=z8gtE7eBiVPyS4gAqaZDmLXXstaBQIF2xzfaaY1mtp4=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=glibU5M9vKw/TrWhPutRohiLu5GZUaiSkUtL3j9QxLEPWWcI8gxRO2InI+31aHwdk T74gm4yvbg6ipw3iF+UyGT0XmgiNr3jj+ra21BytluNxFIkLKJ9C+QYBRDdZgL7N41 +pvXnJ3G8why0ZJaqAsCkvrpUObPmhQM5OpZTkWh6siTBWDd1XYvZw1IxFCwHixSUG WaZBKdqBE5TgEXOXrnWURPIPNM0kICcVc0ZXrnRSYO2WyQSSaTCHjGE6OFM7ONNVCG 1NPYhktiphH9HhNHSX0PudGFhXlPEG+NqSw/g2dsZtGGcTA+F3S8+LO07WXyJCYFRC 9EvbK9yoQ20EQ== Subject: [PATCH 04/10] xfs: test mkfs min log size calculation w/ rt volumes From: "Darrick J. Wong" To: djwong@kernel.org, guaneryu@gmail.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me Date: Mon, 08 Mar 2021 20:40:25 -0800 Message-ID: <161526482563.1214319.7317631500409765514.stgit@magnolia> In-Reply-To: <161526480371.1214319.3263690953532787783.stgit@magnolia> References: <161526480371.1214319.3263690953532787783.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org From: Darrick J. Wong In "mkfs: set required parts of the realtime geometry before computing log geometry" we made sure that mkfs set up enough of the fs geometry to compute the minimum xfs log size correctly when formatting the filesystem. This is the regression test for that issue. Signed-off-by: Darrick J. Wong Reviewed-by: Chandan Babu R --- tests/xfs/761 | 45 +++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/761.out | 1 + tests/xfs/group | 1 + 3 files changed, 47 insertions(+) create mode 100755 tests/xfs/761 create mode 100644 tests/xfs/761.out diff --git a/tests/xfs/761 b/tests/xfs/761 new file mode 100755 index 00000000..b9770d90 --- /dev/null +++ b/tests/xfs/761 @@ -0,0 +1,45 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2021 Oracle. All Rights Reserved. +# +# FS QA Test No. 761 +# +# Make sure mkfs sets up enough of the rt geometry that we can compute the +# correct min log size for formatting the fs. +# +# This is a regression test for the xfsprogs commit 31409f48 ("mkfs: set +# required parts of the realtime geometry before computing log geometry"). + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter + +# real QA test starts here +_supported_fs xfs +_require_scratch +_require_realtime + +rm -f $seqres.full + +# Format a tiny filesystem to force minimum log size, then see if it mounts +_scratch_mkfs -r size=1m -d size=100m > $seqres.full +_scratch_mount >> $seqres.full + +# success, all done +status=0 +exit diff --git a/tests/xfs/761.out b/tests/xfs/761.out new file mode 100644 index 00000000..8c9d9e90 --- /dev/null +++ b/tests/xfs/761.out @@ -0,0 +1 @@ +QA output created by 761 diff --git a/tests/xfs/group b/tests/xfs/group index 318468b5..87badd56 100644 --- a/tests/xfs/group +++ b/tests/xfs/group @@ -503,4 +503,5 @@ 758 auto quick rw attr realtime 759 auto quick rw realtime 760 auto quick rw collapse punch insert zero prealloc +761 auto quick realtime 763 auto quick rw realtime From patchwork Tue Mar 9 04:40:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 12124009 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2A24EC4332B for ; Tue, 9 Mar 2021 04:41:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id F20A1652AD for ; Tue, 9 Mar 2021 04:41:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230122AbhCIEkn (ORCPT ); Mon, 8 Mar 2021 23:40:43 -0500 Received: from mail.kernel.org ([198.145.29.99]:32914 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229701AbhCIEkb (ORCPT ); Mon, 8 Mar 2021 23:40:31 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id 5AB2665275; Tue, 9 Mar 2021 04:40:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1615264831; bh=uhr8wJ7bb74FE76utpAtAKmZe5Cuj55gHF4cpzpgcPQ=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=rU5Nm0M5I+QGmKD9a0FqUV2AOWd3G9lQ5PhhghxRidEL3Uxy/kT0BmufH/DL+5rSD bBRMHk+rti8dPxLFlAHXAfVFMlh9DdofU3DZkLxAd5YE7mNxRx1VhMrnmTSrC7DCm/ 4WnYS2rvURuWPhzSwVg1weR0LSGUpwDHSQNaXRen86Bl4Ep2InRfB0uYcf1XQnvwpF cOgoaMMRn76/3CBdNyoBTx4FxSfxeyhZN+x4cuheozH7dT92mAdlgF6LS2SrDK5I3k vKJELY/uSjeU8PgpWkywmi2SOwsmfvK4nIOj1OomhDOhHOG2mTQVWvEEHRFc4PzymU jqDIouusz1DPw== Subject: [PATCH 05/10] common/filter: refactor quota report filtering From: "Darrick J. Wong" To: djwong@kernel.org, guaneryu@gmail.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me Date: Mon, 08 Mar 2021 20:40:31 -0800 Message-ID: <161526483109.1214319.14555094406560973318.stgit@magnolia> In-Reply-To: <161526480371.1214319.3263690953532787783.stgit@magnolia> References: <161526480371.1214319.3263690953532787783.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org From: Darrick J. Wong xfs/299 and xfs/050 share the same function to filter quota reporting into a format suitable for the golden output. Refactor this so that we can use it in a new test. Signed-off-by: Darrick J. Wong Reviewed-by: Chandan Babu R --- common/filter | 24 ++++++++++++++++++++++++ tests/xfs/050 | 30 ++++++------------------------ tests/xfs/299 | 30 ++++++------------------------ 3 files changed, 36 insertions(+), 48 deletions(-) diff --git a/common/filter b/common/filter index 2f3277f7..2efbbd99 100644 --- a/common/filter +++ b/common/filter @@ -637,5 +637,29 @@ _filter_getcap() sed -e "s/= //" -e "s/\+/=/g" } +# Filter user/group/project id numbers out of quota reports, and standardize +# the block counts to use filesystem block size. Callers must set the id and +# bsize variables before calling this function. +_filter_quota_report() +{ + test -n "$id" || echo "id must be set" + test -n "$bsize" || echo "block size must be set" + + tr -s '[:space:]' | \ + perl -npe ' + s/^\#'$id' /[NAME] /g; + s/^\#0 \d+ /[ROOT] 0 /g; + s/6 days/7 days/g' | + perl -npe ' + $val = 0; + if ($ENV{'LARGE_SCRATCH_DEV'}) { + $val = $ENV{'NUM_SPACE_FILES'}; + } + s/(^\[ROOT\] \S+ \S+ \S+ \S+ \[--------\] )(\S+)/$1@{[$2 - $val]}/g' | + sed -e 's/ 65535 \[--------\]/ 00 \[--------\]/g' | + perl -npe ' + s|^(.*?) (\d+) (\d+) (\d+)|$1 @{[$2 * 1024 /'$bsize']} @{[$3 * 1024 /'$bsize']} @{[$4 * 1024 /'$bsize']}|' +} + # make sure this script returns success /bin/true diff --git a/tests/xfs/050 b/tests/xfs/050 index 53412a13..1df97537 100755 --- a/tests/xfs/050 +++ b/tests/xfs/050 @@ -47,24 +47,6 @@ bhard=$(( 1000 * $bsize )) isoft=4 ihard=10 -_filter_report() -{ - tr -s '[:space:]' | \ - perl -npe ' - s/^\#'$id' /[NAME] /g; - s/^\#0 \d+ /[ROOT] 0 /g; - s/6 days/7 days/g' | - perl -npe ' - $val = 0; - if ($ENV{'LARGE_SCRATCH_DEV'}) { - $val = $ENV{'NUM_SPACE_FILES'}; - } - s/(^\[ROOT\] \S+ \S+ \S+ \S+ \[--------\] )(\S+)/$1@{[$2 - $val]}/g' | - sed -e 's/ 65535 \[--------\]/ 00 \[--------\]/g' | - perl -npe ' - s|^(.*?) (\d+) (\d+) (\d+)|$1 @{[$2 * 1024 /'$bsize']} @{[$3 * 1024 /'$bsize']} @{[$4 * 1024 /'$bsize']}|' -} - # The actual point at which limit enforcement takes place for the # hard block limit is variable depending on filesystem blocksize, # and iosize. What we want to test is that the limit is enforced @@ -84,7 +66,7 @@ _filter_and_check_blks() } s/^(\#'$id'\s+)(\d+)/\1 =OK=/g; } - ' | _filter_report + ' | _filter_quota_report } _qsetup() @@ -134,7 +116,7 @@ _exercise() echo "*** report no quota settings" | tee -a $seqres.full $XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \ -c "repquota -birnN -$type" $SCRATCH_DEV | - _filter_report | LC_COLLATE=POSIX sort -ru + _filter_quota_report | LC_COLLATE=POSIX sort -ru echo echo "*** report initial settings" | tee -a $seqres.full @@ -147,7 +129,7 @@ _exercise() $SCRATCH_DEV $XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \ -c "repquota -birnN -$type" $SCRATCH_DEV | - _filter_report | LC_COLLATE=POSIX sort -ru + _filter_quota_report | LC_COLLATE=POSIX sort -ru echo echo "*** push past the soft inode limit" | tee -a $seqres.full @@ -159,7 +141,7 @@ _exercise() $XFS_QUOTA_PROG -x -c "warn -i -$type 0 $id" $SCRATCH_DEV $XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \ -c "repquota -birnN -$type" $SCRATCH_DEV | - _filter_report | LC_COLLATE=POSIX sort -ru + _filter_quota_report | LC_COLLATE=POSIX sort -ru echo echo "*** push past the soft block limit" | tee -a $seqres.full @@ -169,7 +151,7 @@ _exercise() -c "warn -b -$type 0 $id" $SCRATCH_DEV $XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \ -c "repquota -birnN -$type" $SCRATCH_DEV | - _filter_report | LC_COLLATE=POSIX sort -ru + _filter_quota_report | LC_COLLATE=POSIX sort -ru echo # Note: for quota accounting (not enforcement), EDQUOT is not expected @@ -183,7 +165,7 @@ _exercise() -c "warn -i -$type 0 $id" $SCRATCH_DEV $XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \ -c "repquota -birnN -$type" $SCRATCH_DEV | - _filter_report | LC_COLLATE=POSIX sort -ru + _filter_quota_report | LC_COLLATE=POSIX sort -ru echo # Note: for quota accounting (not enforcement), EDQUOT is not expected diff --git a/tests/xfs/299 b/tests/xfs/299 index 15e0edf6..b862e67e 100755 --- a/tests/xfs/299 +++ b/tests/xfs/299 @@ -40,24 +40,6 @@ _require_xfs_quota _require_xfs_mkfs_crc _require_xfs_crc -_filter_report() -{ - tr -s '[:space:]' | \ - perl -npe ' - s/^\#'$id' /[NAME] /g; - s/^\#0 \d+ /[ROOT] 0 /g; - s/6 days/7 days/g' | - perl -npe ' - $val = 0; - if ($ENV{'LARGE_SCRATCH_DEV'}) { - $val = $ENV{'NUM_SPACE_FILES'}; - } - s/(^\[ROOT\] \S+ \S+ \S+ \S+ \[--------\] )(\S+)/$1@{[$2 - $val]}/g' | - sed -e 's/ 65535 \[--------\]/ 00 \[--------\]/g' | - perl -npe ' - s|^(.*?) (\d+) (\d+) (\d+)|$1 @{[$2 * 1024 /'$bsize']} @{[$3 * 1024 /'$bsize']} @{[$4 * 1024 /'$bsize']}|' -} - # The actual point at which limit enforcement takes place for the # hard block limit is variable depending on filesystem blocksize, # and iosize. What we want to test is that the limit is enforced @@ -77,7 +59,7 @@ _filter_and_check_blks() } s/^(\#'$id'\s+)(\d+)/\1 =OK=/g; } - ' | _filter_report + ' | _filter_quota_report } _qsetup() @@ -120,7 +102,7 @@ _exercise() echo "*** report no quota settings" | tee -a $seqres.full $XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \ -c "repquota -birnN -$type" $SCRATCH_DEV | - _filter_report | LC_COLLATE=POSIX sort -ru + _filter_quota_report | LC_COLLATE=POSIX sort -ru echo echo "*** report initial settings" | tee -a $seqres.full @@ -133,7 +115,7 @@ _exercise() $SCRATCH_DEV $XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \ -c "repquota -birnN -$type" $SCRATCH_DEV | - _filter_report | LC_COLLATE=POSIX sort -ru + _filter_quota_report | LC_COLLATE=POSIX sort -ru echo echo "*** push past the soft inode limit" | tee -a $seqres.full @@ -145,7 +127,7 @@ _exercise() $XFS_QUOTA_PROG -x -c "warn -i -$type 0 $id" $SCRATCH_DEV $XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \ -c "repquota -birnN -$type" $SCRATCH_DEV | - _filter_report | LC_COLLATE=POSIX sort -ru + _filter_quota_report | LC_COLLATE=POSIX sort -ru echo echo "*** push past the soft block limit" | tee -a $seqres.full @@ -155,7 +137,7 @@ _exercise() -c "warn -b -$type 0 $id" $SCRATCH_DEV $XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \ -c "repquota -birnN -$type" $SCRATCH_DEV | - _filter_report | LC_COLLATE=POSIX sort -ru + _filter_quota_report | LC_COLLATE=POSIX sort -ru echo # Note: for quota accounting (not enforcement), EDQUOT is not expected @@ -169,7 +151,7 @@ _exercise() -c "warn -i -$type 0 $id" $SCRATCH_DEV $XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \ -c "repquota -birnN -$type" $SCRATCH_DEV | - _filter_report | LC_COLLATE=POSIX sort -ru + _filter_quota_report | LC_COLLATE=POSIX sort -ru echo # Note: for quota accounting (not enforcement), EDQUOT is not expected From patchwork Tue Mar 9 04:40:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 12124007 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EFDD1C43381 for ; Tue, 9 Mar 2021 04:41:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CBABC6529E for ; Tue, 9 Mar 2021 04:41:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230140AbhCIEkn (ORCPT ); Mon, 8 Mar 2021 23:40:43 -0500 Received: from mail.kernel.org ([198.145.29.99]:32936 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229813AbhCIEkh (ORCPT ); Mon, 8 Mar 2021 23:40:37 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id DE12865275; Tue, 9 Mar 2021 04:40:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1615264837; bh=kxWH7F4brECXLYtjcDpMuJt2/GCOUol5Wd8naC3DfBc=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=dCst7V+c+g4VD52VSzim5IFGf32Sa9HFnGbzzJjbiUe4DvMzdfRSk0x/344f2LlNt xreqIBroq8hhl4fZ0x/pYm1tAhbNDPUZ/BztYjdlcxTVz0QUV5Spk8d37hjis0Hzs4 E6W4YAQ0RTZs5wbJe5zVI74SCG+6vGZIe+Kbe3Lv9v+Q+R+vbaSP5z5GckNhUnqEDM ZMwnsNY2o43x4PfrQKzM3zCYqsC3Ep/r7lDQ7jVq3ACPLZkAKe97v300kUVnJJPb4G rqjuIokEA63MGD/Ta/DxcYl5F9qrlBTEta8yw/D9cIFY1V0GBJmuYkc78AFJ9e11dg yh8iOOgX0Rd5A== Subject: [PATCH 06/10] xfs: test quota softlimit warning functionality From: "Darrick J. Wong" To: djwong@kernel.org, guaneryu@gmail.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me Date: Mon, 08 Mar 2021 20:40:36 -0800 Message-ID: <161526483668.1214319.17667836667890283825.stgit@magnolia> In-Reply-To: <161526480371.1214319.3263690953532787783.stgit@magnolia> References: <161526480371.1214319.3263690953532787783.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org From: Darrick J. Wong Make sure that quota softlimits work, which is to say that one can exceed the softlimit up to warnlimit times before it starts enforcing that. Signed-off-by: Darrick J. Wong Reviewed-by: Chandan Babu R --- tests/xfs/915 | 162 +++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/915.out | 151 +++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/group | 1 3 files changed, 314 insertions(+) create mode 100755 tests/xfs/915 create mode 100644 tests/xfs/915.out diff --git a/tests/xfs/915 b/tests/xfs/915 new file mode 100755 index 00000000..a2cdbbb7 --- /dev/null +++ b/tests/xfs/915 @@ -0,0 +1,162 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2021 Oracle. All Rights Reserved. +# +# FS QA Test No. 915 +# +# Check that quota softlimit warnings work the way they should. This means +# that we can disobey the softlimit up to warnlimit times before it turns into +# hard(er) enforcement. This is a functional test for quota warnings, but +# since the functionality has been broken for decades, this is also a +# regression test for commit 4b8628d57b72 ("xfs: actually bump warning counts +# when we send warnings"). + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter +. ./common/quota + +# real QA test starts here +_supported_fs xfs +_require_xfs_quota +_require_scratch + +rm -f $seqres.full + +qsetup() +{ + opt=$1 + enforce=0 + if [ $opt = "u" -o $opt = "uno" ]; then + type=u + eval `_choose_uid` + elif [ $opt = "g" -o $opt = "gno" ]; then + type=g + eval `_choose_gid` + elif [ $opt = "p" -o $opt = "pno" ]; then + type=p + eval `_choose_prid` + fi + [ $opt = "u" -o $opt = "g" -o $opt = "p" ] && enforce=1 + + echo "Using type=$type id=$id" >> $seqres.full +} + +exercise() { + _scratch_mkfs_xfs | _filter_mkfs 2>$tmp.mkfs + cat $tmp.mkfs >>$seqres.full + + # keep the blocksize and data size for dd later + . $tmp.mkfs + + _qmount + + qsetup $1 + + echo "Using type=$type id=$id" >>$seqres.full + + echo + echo "*** report initial settings" | tee -a $seqres.full + $XFS_QUOTA_PROG -x \ + -c "limit -$type isoft=3 ihard=500000 $id" \ + -c "warn -$type -i -d 13" \ + $SCRATCH_DEV + $XFS_QUOTA_PROG -x \ + -c "state -$type" >> $seqres.full + $XFS_QUOTA_PROG -x \ + -c "repquota -birnN -$type" $SCRATCH_DEV | + _filter_quota_report | LC_COLLATE=POSIX sort -ru + + echo + echo "*** push past the soft inode limit" | tee -a $seqres.full + _file_as_id $SCRATCH_MNT/softok1 $id $type $bsize 0 + _file_as_id $SCRATCH_MNT/softok2 $id $type $bsize 0 + _file_as_id $SCRATCH_MNT/softok3 $id $type $bsize 0 + _file_as_id $SCRATCH_MNT/softwarn1 $id $type $bsize 0 + $XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \ + -c "repquota -birnN -$type" $SCRATCH_DEV | + _filter_quota_report | LC_COLLATE=POSIX sort -ru + + echo + echo "*** push further past the soft inode limit" | tee -a $seqres.full + for warn_nr in $(seq 2 5); do + _file_as_id $SCRATCH_MNT/softwarn$warn_nr $id $type $bsize 0 + done + $XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \ + -c "repquota -birnN -$type" $SCRATCH_DEV | + _filter_quota_report | LC_COLLATE=POSIX sort -ru + + echo + echo "*** push past the soft inode warning limit" | tee -a $seqres.full + for warn_nr in $(seq 6 15); do + _file_as_id $SCRATCH_MNT/softwarn$warn_nr $id $type $bsize 0 + done + $XFS_QUOTA_PROG -D $tmp.projects -P $tmp.projid -x \ + -c "repquota -birnN -$type" $SCRATCH_DEV | + _filter_quota_report | LC_COLLATE=POSIX sort -ru + + echo + echo "*** unmount" + _scratch_unmount +} + +_scratch_mkfs > $seqres.full +_scratch_mount >> $seqres.full + +chmod a+rwx $SCRATCH_MNT $seqres.full # arbitrary users will write here +bsize=$(_get_file_block_size $SCRATCH_MNT) +_scratch_unmount + +cat >$tmp.projects <$tmp.projid < X-Patchwork-Id: 12124017 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6005CC43381 for ; Tue, 9 Mar 2021 04:41:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4D2ED652B3 for ; Tue, 9 Mar 2021 04:41:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229775AbhCIElN (ORCPT ); Mon, 8 Mar 2021 23:41:13 -0500 Received: from mail.kernel.org ([198.145.29.99]:32990 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229992AbhCIEkm (ORCPT ); Mon, 8 Mar 2021 23:40:42 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id 5EF70652AB; Tue, 9 Mar 2021 04:40:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1615264842; bh=TC8sqN/K3xRmSr3J0c+aaYt48aeySfMJX7cOvq0X+1Y=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=YROBj0y2SKlsZIwGac97uHH9r9Vay6ZqJc9x4dEztoblu0UW96SptEUo8if50L7l4 1RdnRVXG1fN32d2JfQ8TpPa5V7u602mMq5cjAldFlCGsi9xzB9OZ7F5yuzd5aESSi3 o/8/aSQafwXoSUwicEJldclZjPGzeX2vFInUhmd/li9zuZhc+Ybc/LtqF6yFV7fQCy azMWPJwV1Ht6+Lgm+5lHA+H2QXoYlxyxNwxGE4ssV5tZsG8Mma6dHth7rF5aHRgB9z L9RoSrI3QWcEo8Ktt+RjSOCGLI1c/qN/IMo+F/ZDBURvi+tawIe/6wpN+XRcH8cy/y bK3tNggyLZYcQ== Subject: [PATCH 07/10] xfs/122: fix test for xfs_attr_shortform_t conversion From: "Darrick J. Wong" To: djwong@kernel.org, guaneryu@gmail.com Cc: Chandan Babu R , linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me Date: Mon, 08 Mar 2021 20:40:42 -0800 Message-ID: <161526484222.1214319.7083379928394196240.stgit@magnolia> In-Reply-To: <161526480371.1214319.3263690953532787783.stgit@magnolia> References: <161526480371.1214319.3263690953532787783.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org From: Darrick J. Wong The typedef xfs_attr_shortform_t was converted to a struct in 5.10. Update this test to pass. Signed-off-by: Darrick J. Wong Reviewed-by: Chandan Babu R Reviewed-by: Christoph Hellwig --- tests/xfs/122 | 1 - tests/xfs/122.out | 1 + 2 files changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/xfs/122 b/tests/xfs/122 index a4248031..322e1d81 100755 --- a/tests/xfs/122 +++ b/tests/xfs/122 @@ -72,7 +72,6 @@ for hdr in /usr/include/xfs/xfs*.h; do done # missing: -# xfs_attr_shortform_t # xfs_trans_header_t cat >$tmp.ignore < X-Patchwork-Id: 12124019 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2BFCCC433E6 for ; Tue, 9 Mar 2021 04:41:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0C4F965275 for ; Tue, 9 Mar 2021 04:41:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229948AbhCIElO (ORCPT ); Mon, 8 Mar 2021 23:41:14 -0500 Received: from mail.kernel.org ([198.145.29.99]:33066 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229813AbhCIEks (ORCPT ); Mon, 8 Mar 2021 23:40:48 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id DA9F56523B; Tue, 9 Mar 2021 04:40:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1615264848; bh=52XpmYjnOyESsgmNsto1erN8ehlo2ih2NhysKtNmy4g=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=tbI3DlSObX2770VE+j0kqseh9hUrPgEySJRHsFLkpn3a7SvqNxS9GJbby/Kn7XBc0 O3VsGhZg4L8/aVIwbanO7+qxcXjgUAlEVdXmQkuNVOlOQtY7gWx9LtVK3KJ/ldInxm Sjq55srLGHKrCYRcuwS1LU4yaosDabTpo5RTOBQuwzh+NuAP8yWOTJ9nFH8hFMXYd1 T8JKlT5jUo/m4PeWo3RqGAE6+9fxT3A4ZfwxSa+glq3ADyUyEaLSQLgt/7IXZDW3dL +qEEgXcDmpqcuS3EV9fbieYmlQQgM6gTprRWw5LWZyhN8izRkL9CvaTkzXensMS2dU 1W/QK0C1+Bmvw== Subject: [PATCH 08/10] generic: test file writers racing with FIDEDUPERANGE From: "Darrick J. Wong" To: djwong@kernel.org, guaneryu@gmail.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me Date: Mon, 08 Mar 2021 20:40:47 -0800 Message-ID: <161526484769.1214319.11110389021630982078.stgit@magnolia> In-Reply-To: <161526480371.1214319.3263690953532787783.stgit@magnolia> References: <161526480371.1214319.3263690953532787783.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org From: Darrick J. Wong Create a test to make sure that dedupe actually locks the file ranges correctly before starting the content comparison and keeps them locked until the operation completes. Signed-off-by: Darrick J. Wong Reviewed-by: Chandan Babu R --- src/Makefile | 2 src/deduperace.c | 370 +++++++++++++++++++++++++++++++++++++++++++++++++ tests/generic/949 | 51 +++++++ tests/generic/949.out | 2 tests/generic/group | 1 5 files changed, 425 insertions(+), 1 deletion(-) create mode 100644 src/deduperace.c create mode 100755 tests/generic/949 create mode 100644 tests/generic/949.out diff --git a/src/Makefile b/src/Makefile index 811b24e4..38ee6718 100644 --- a/src/Makefile +++ b/src/Makefile @@ -21,7 +21,7 @@ TARGETS = dirstress fill fill2 getpagesize holes lstat64 \ LINUX_TARGETS = xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader \ preallo_rw_pattern_writer ftrunc trunc fs_perms testx looptest \ - locktest unwritten_mmap bulkstat_unlink_test \ + locktest unwritten_mmap bulkstat_unlink_test deduperace \ bulkstat_unlink_test_modified t_dir_offset t_futimens t_immutable \ stale_handle pwrite_mmap_blocked t_dir_offset2 seek_sanity_test \ seek_copy_test t_readdir_1 t_readdir_2 fsync-tester nsexec cloner \ diff --git a/src/deduperace.c b/src/deduperace.c new file mode 100644 index 00000000..b252d436 --- /dev/null +++ b/src/deduperace.c @@ -0,0 +1,370 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * Copyright (c) 2021 Oracle. All Rights Reserved. + * Author: Darrick J. Wong + * + * Race pwrite/mwrite with dedupe to see if we got the locking right. + * + * File writes and mmap writes should not be able to change the src_fd's + * contents after dedupe prep has verified that the file contents are the same. + */ +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#define GOOD_BYTE 0x58 +#define BAD_BYTE 0x66 + +#ifndef FIDEDUPERANGE +/* extent-same (dedupe) ioctls; these MUST match the btrfs ioctl definitions */ +#define FILE_DEDUPE_RANGE_SAME 0 +#define FILE_DEDUPE_RANGE_DIFFERS 1 + +/* from struct btrfs_ioctl_file_extent_same_info */ +struct file_dedupe_range_info { + __s64 dest_fd; /* in - destination file */ + __u64 dest_offset; /* in - start of extent in destination */ + __u64 bytes_deduped; /* out - total # of bytes we were able + * to dedupe from this file. */ + /* status of this dedupe operation: + * < 0 for error + * == FILE_DEDUPE_RANGE_SAME if dedupe succeeds + * == FILE_DEDUPE_RANGE_DIFFERS if data differs + */ + __s32 status; /* out - see above description */ + __u32 reserved; /* must be zero */ +}; + +/* from struct btrfs_ioctl_file_extent_same_args */ +struct file_dedupe_range { + __u64 src_offset; /* in - start of extent in source */ + __u64 src_length; /* in - length of extent */ + __u16 dest_count; /* in - total elements in info array */ + __u16 reserved1; /* must be zero */ + __u32 reserved2; /* must be zero */ + struct file_dedupe_range_info info[0]; +}; +#define FIDEDUPERANGE _IOWR(0x94, 54, struct file_dedupe_range) +#endif /* FIDEDUPERANGE */ + +static int fd1, fd2; +static loff_t offset = 37; /* Nice low offset to trick the compare */ +static loff_t blksz; + +/* Continuously dirty the pagecache for the region being dupe-tested. */ +void * +mwriter( + void *data) +{ + volatile char *p; + + p = mmap(NULL, blksz, PROT_WRITE, MAP_SHARED, fd1, 0); + if (p == MAP_FAILED) { + perror("mmap"); + exit(2); + } + + while (1) { + *(p + offset) = BAD_BYTE; + *(p + offset) = GOOD_BYTE; + } +} + +/* Continuously write to the region being dupe-tested. */ +void * +pwriter( + void *data) +{ + char v; + ssize_t sz; + + while (1) { + v = BAD_BYTE; + sz = pwrite(fd1, &v, sizeof(v), offset); + if (sz != sizeof(v)) { + perror("pwrite0"); + exit(2); + } + + v = GOOD_BYTE; + sz = pwrite(fd1, &v, sizeof(v), offset); + if (sz != sizeof(v)) { + perror("pwrite1"); + exit(2); + } + } + + return NULL; +} + +static inline void +complain( + loff_t offset, + char bad) +{ + fprintf(stderr, "ASSERT: offset %llu should be 0x%x, got 0x%x!\n", + (unsigned long long)offset, GOOD_BYTE, bad); + abort(); +} + +/* Make sure the destination file pagecache never changes. */ +void * +mreader( + void *data) +{ + volatile char *p; + + p = mmap(NULL, blksz, PROT_READ, MAP_SHARED, fd2, 0); + if (p == MAP_FAILED) { + perror("mmap"); + exit(2); + } + + while (1) { + if (*(p + offset) != GOOD_BYTE) + complain(offset, *(p + offset)); + } +} + +/* Make sure the destination file never changes. */ +void * +preader( + void *data) +{ + char v; + ssize_t sz; + + while (1) { + sz = pread(fd2, &v, sizeof(v), offset); + if (sz != sizeof(v)) { + perror("pwrite0"); + exit(2); + } + + if (v != GOOD_BYTE) + complain(offset, v); + } + + return NULL; +} + +void +print_help(const char *progname) +{ + printf("Usage: %s [-b blksz] [-c dir] [-n nr_ops] [-o offset] [-r] [-w] [-v]\n", + progname); + printf("-b sets the block size (default is autoconfigured)\n"); + printf("-c chdir to this path before starting\n"); + printf("-n controls the number of dedupe ops (default 10000)\n"); + printf("-o reads and writes to this offset (default 37)\n"); + printf("-r uses pread instead of mmap read.\n"); + printf("-v prints status updates.\n"); + printf("-w uses pwrite instead of mmap write.\n"); +} + +int +main( + int argc, + char *argv[]) +{ + struct file_dedupe_range *fdr; + char *Xbuf; + void *(*reader_fn)(void *) = mreader; + void *(*writer_fn)(void *) = mwriter; + unsigned long same = 0; + unsigned long differs = 0; + unsigned long i, nr_ops = 10000; + ssize_t sz; + pthread_t reader, writer; + int verbose = 0; + int c; + int ret; + + while ((c = getopt(argc, argv, "b:c:n:o:rvw")) != -1) { + switch (c) { + case 'b': + errno = 0; + blksz = strtoul(optarg, NULL, 0); + if (errno) { + perror(optarg); + exit(1); + } + break; + case 'c': + ret = chdir(optarg); + if (ret) { + perror("chdir"); + exit(1); + } + break; + case 'n': + errno = 0; + nr_ops = strtoul(optarg, NULL, 0); + if (errno) { + perror(optarg); + exit(1); + } + break; + case 'o': + errno = 0; + offset = strtoul(optarg, NULL, 0); + if (errno) { + perror(optarg); + exit(1); + } + break; + case 'r': + reader_fn = preader; + break; + case 'v': + verbose = 1; + break; + case 'w': + writer_fn = pwriter; + break; + default: + print_help(argv[0]); + exit(1); + break; + } + } + + fdr = malloc(sizeof(struct file_dedupe_range) + + sizeof(struct file_dedupe_range_info)); + if (!fdr) { + perror("malloc"); + exit(1); + } + + /* Initialize both files. */ + fd1 = open("file1", O_RDWR | O_CREAT | O_TRUNC | O_NOATIME, 0600); + if (fd1 < 0) { + perror("file1"); + exit(1); + } + + fd2 = open("file2", O_RDWR | O_CREAT | O_TRUNC | O_NOATIME, 0600); + if (fd2 < 0) { + perror("file2"); + exit(1); + } + + if (blksz <= 0) { + struct stat statbuf; + + ret = fstat(fd1, &statbuf); + if (ret) { + perror("file1 stat"); + exit(1); + } + blksz = statbuf.st_blksize; + } + + if (offset >= blksz) { + fprintf(stderr, "offset (%llu) < blksz (%llu)?\n", + (unsigned long long)offset, + (unsigned long long)blksz); + exit(1); + } + + Xbuf = malloc(blksz); + if (!Xbuf) { + perror("malloc buffer"); + exit(1); + } + memset(Xbuf, GOOD_BYTE, blksz); + + sz = pwrite(fd1, Xbuf, blksz, 0); + if (sz != blksz) { + perror("file1 write"); + exit(1); + } + + sz = pwrite(fd2, Xbuf, blksz, 0); + if (sz != blksz) { + perror("file2 write"); + exit(1); + } + + ret = fsync(fd1); + if (ret) { + perror("file1 fsync"); + exit(1); + } + + ret = fsync(fd2); + if (ret) { + perror("file2 fsync"); + exit(1); + } + + /* Start our reader and writer threads. */ + ret = pthread_create(&reader, NULL, reader_fn, NULL); + if (ret) { + fprintf(stderr, "rthread: %s\n", strerror(ret)); + exit(1); + } + + ret = pthread_create(&writer, NULL, writer_fn, NULL); + if (ret) { + fprintf(stderr, "wthread: %s\n", strerror(ret)); + exit(1); + } + + /* + * Now start deduping. If the contents match, fd1's blocks will be + * remapped into fd2, which is why the writer thread targets fd1 and + * the reader checks fd2 to make sure that none of fd1's writes ever + * make it into fd2. + */ + for (i = 1; i <= nr_ops; i++) { + fdr->src_offset = 0; + fdr->src_length = blksz; + fdr->dest_count = 1; + fdr->reserved1 = 0; + fdr->reserved2 = 0; + fdr->info[0].dest_fd = fd2; + fdr->info[0].dest_offset = 0; + fdr->info[0].reserved = 0; + + ret = ioctl(fd1, FIDEDUPERANGE, fdr); + if (ret) { + perror("dedupe"); + exit(2); + } + + switch (fdr->info[0].status) { + case FILE_DEDUPE_RANGE_DIFFERS: + differs++; + break; + case FILE_DEDUPE_RANGE_SAME: + same++; + break; + default: + fprintf(stderr, "deduperange: %s\n", + strerror(-fdr->info[0].status)); + exit(2); + break; + } + + if (verbose && (i % 337) == 0) + printf("nr_ops: %lu; same: %lu; differs: %lu\n", + i, same, differs); + } + + if (verbose) + printf("nr_ops: %lu; same: %lu; differs: %lu\n", i - 1, same, + differs); + + /* Program termination will kill the threads and close the files. */ + return 0; +} diff --git a/tests/generic/949 b/tests/generic/949 new file mode 100755 index 00000000..3951490b --- /dev/null +++ b/tests/generic/949 @@ -0,0 +1,51 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2021 Oracle. All Rights Reserved. +# +# FS QA Test No. 949 +# +# Make sure that mmap and file writers racing with FIDEDUPERANGE cannot write +# to the file after the dedupe prep function has decided that the file contents +# are identical and we can therefore go ahead with the remapping. + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/reflink + +# real QA test starts here +_supported_fs generic +_require_scratch_dedupe + +rm -f $seqres.full + +nr_ops=$((TIME_FACTOR * 10000)) + +# Format filesystem +_scratch_mkfs > $seqres.full +_scratch_mount + +# Test once with mmap writes +$here/src/deduperace -c $SCRATCH_MNT -n $nr_ops + +# Test again with pwrites for the lulz +$here/src/deduperace -c $SCRATCH_MNT -n $nr_ops -w + +echo Silence is golden. +# success, all done +status=0 +exit diff --git a/tests/generic/949.out b/tests/generic/949.out new file mode 100644 index 00000000..2998b46c --- /dev/null +++ b/tests/generic/949.out @@ -0,0 +1,2 @@ +QA output created by 949 +Silence is golden. diff --git a/tests/generic/group b/tests/generic/group index d5cfdd51..778aa8c4 100644 --- a/tests/generic/group +++ b/tests/generic/group @@ -630,3 +630,4 @@ 625 auto quick verity 947 auto quick rw clone 948 auto quick rw copy_range +949 auto quick rw dedupe clone From patchwork Tue Mar 9 04:40:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 12124015 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 158A8C433DB for ; Tue, 9 Mar 2021 04:41:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DE6AF65290 for ; Tue, 9 Mar 2021 04:41:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229992AbhCIElO (ORCPT ); Mon, 8 Mar 2021 23:41:14 -0500 Received: from mail.kernel.org ([198.145.29.99]:33082 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229854AbhCIEkx (ORCPT ); Mon, 8 Mar 2021 23:40:53 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id 61C9365275; Tue, 9 Mar 2021 04:40:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1615264853; bh=O/8xPqrlVA31AQ13Ug1211UcQ/I3PFRPbN+ktCU8lnA=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=X5sjV4Owp9wNDaWj3wvaU+5sJP1ReNw7zHNjErAla/dyulNZsVXUy8fvIimRV4gsJ 0dGoEdovTS5K3dzc+F8NGxNGySjnrbcIlZgYbYO3TjTfl5X+zT9Fpc3sQTo5doD314 kJz2hSvU2fG8E8AaGq0mVw7B5dewvv+GBKYBPNq76/8KJi2eibGD5yaIA4Mroyn/eF U5lqA/c/ld3I3rEcuXdiCA19nyx3V8Ll3YVoHoTqmPbFuVyni1nSNdNmwrwSTD3zCK XFn/DNuvCKJ3GAlQzJY1wP/gDqcIW4XpPVGKHPS9uA9vZyzbIRshf+LwySPcnJ5MIv LtvmWeibSkVcA== Subject: [PATCH 09/10] generic: test a deadlock in xfs_rename when whiteing out files From: "Darrick J. Wong" To: djwong@kernel.org, guaneryu@gmail.com Cc: wenli xie , linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me Date: Mon, 08 Mar 2021 20:40:53 -0800 Message-ID: <161526485320.1214319.14486851135232825638.stgit@magnolia> In-Reply-To: <161526480371.1214319.3263690953532787783.stgit@magnolia> References: <161526480371.1214319.3263690953532787783.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org From: Darrick J. Wong wenli xie reported a buffer cache deadlock when an overlayfs is mounted atop xfs and overlayfs tries to replace a single-nlink file with a whiteout file. This test reproduces that deadlock. Reported-by: wenli xie Signed-off-by: Darrick J. Wong --- tests/generic/1300 | 109 ++++++++++++++++++++++++++++++++++++++++++++++++ tests/generic/1300.out | 2 + tests/generic/group | 1 3 files changed, 112 insertions(+) create mode 100755 tests/generic/1300 create mode 100644 tests/generic/1300.out diff --git a/tests/generic/1300 b/tests/generic/1300 new file mode 100755 index 00000000..10df44e3 --- /dev/null +++ b/tests/generic/1300 @@ -0,0 +1,109 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2021 Oracle. All Rights Reserved. +# +# FS QA Test No. 1300 +# +# Reproducer for a deadlock in xfs_rename reported by Wenli Xie. +# +# When overlayfs is running on top of xfs and the user unlinks a file in the +# overlay, overlayfs will create a whiteout inode and ask us to "rename" the +# whiteout file atop the one being unlinked. If the file being unlinked loses +# its one nlink, we then have to put the inode on the unlinked list. +# +# This requires us to grab the AGI buffer of the whiteout inode to take it +# off the unlinked list (which is where whiteouts are created) and to grab +# the AGI buffer of the file being deleted. If the whiteout was created in +# a higher numbered AG than the file being deleted, we'll lock the AGIs in +# the wrong order and deadlock. +# +# Note that this test doesn't do anything xfs-specific so it's a generic test. +# This is a regression test for commit 6da1b4b1ab36 ("xfs: fix an ABBA deadlock +# in xfs_rename"). + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + stop_workers + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc + +# real QA test starts here +_supported_fs generic +_require_scratch +test "$FSTYP" = "overlay" && _notrun "Test does not apply to overlayfs." + +modprobe -q overlay +grep -q overlay /proc/filesystems || _notrun "Test requires overlayfs." + +rm -f $seqres.full + +_scratch_mkfs >> $seqres.full +_scratch_mount + +mkdir $SCRATCH_MNT/lowerdir +mkdir $SCRATCH_MNT/lowerdir1 +mkdir $SCRATCH_MNT/lowerdir/etc +mkdir $SCRATCH_MNT/workers +echo salts > $SCRATCH_MNT/lowerdir/etc/access.conf +touch $SCRATCH_MNT/running + +stop_workers() { + test -e $SCRATCH_MNT/running || return + rm -f $SCRATCH_MNT/running + + while [ "$(ls $SCRATCH_MNT/workers/ | wc -l)" -gt 0 ]; do + wait + done +} + +worker() { + local tag="$1" + local mergedir="$SCRATCH_MNT/merged$tag" + local l="lowerdir=$SCRATCH_MNT/lowerdir:$SCRATCH_MNT/lowerdir1" + local u="upperdir=$SCRATCH_MNT/upperdir$tag" + local w="workdir=$SCRATCH_MNT/workdir$tag" + local i="index=off,nfs_export=off" + + touch $SCRATCH_MNT/workers/$tag + while test -e $SCRATCH_MNT/running; do + rm -rf $SCRATCH_MNT/merged$tag + rm -rf $SCRATCH_MNT/upperdir$tag + rm -rf $SCRATCH_MNT/workdir$tag + mkdir $SCRATCH_MNT/merged$tag + mkdir $SCRATCH_MNT/workdir$tag + mkdir $SCRATCH_MNT/upperdir$tag + + mount -t overlay overlay -o "$l,$u,$w,$i" $mergedir + mv $mergedir/etc/access.conf $mergedir/etc/access.conf.bak + touch $mergedir/etc/access.conf + mv $mergedir/etc/access.conf $mergedir/etc/access.conf.bak + touch $mergedir/etc/access.conf + umount $mergedir + done + rm -f $SCRATCH_MNT/workers/$tag +} + +for i in $(seq 0 $((4 + LOAD_FACTOR)) ); do + worker $i & +done + +sleep $((30 * TIME_FACTOR)) +stop_workers + +echo Silence is golden. +# success, all done +status=0 +exit diff --git a/tests/generic/1300.out b/tests/generic/1300.out new file mode 100644 index 00000000..5805d30d --- /dev/null +++ b/tests/generic/1300.out @@ -0,0 +1,2 @@ +QA output created by 1300 +Silence is golden. diff --git a/tests/generic/group b/tests/generic/group index 778aa8c4..2233a59d 100644 --- a/tests/generic/group +++ b/tests/generic/group @@ -631,3 +631,4 @@ 947 auto quick rw clone 948 auto quick rw copy_range 949 auto quick rw dedupe clone +1300 auto rw overlay From patchwork Tue Mar 9 04:40:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 12124021 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7EEA4C43331 for ; Tue, 9 Mar 2021 04:41:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6A7556528A for ; Tue, 9 Mar 2021 04:41:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230056AbhCIElO (ORCPT ); Mon, 8 Mar 2021 23:41:14 -0500 Received: from mail.kernel.org ([198.145.29.99]:33102 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229875AbhCIEk7 (ORCPT ); Mon, 8 Mar 2021 23:40:59 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id D81CA6523B; Tue, 9 Mar 2021 04:40:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1615264858; bh=BRd+NOJXW3rif+YdB76MWtwYJY6rd4IdNc/vUZxoaYM=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=HpORDagaSHsuYc/HT88bNCVXxV7T9QaqMEFu6bs+btiC6nqhynWuYPCaC6tn5hDp5 a357UwUuyrxFl7wnWPYbEc9KzGWmcO2/nf7MZhbAsYb6Y8NXyWCz5c1Mf978vGwL/g K11PFe8/HMV47Of8OhaUfjIuFYkoSjhRk80sbNr2XsAgfEfTcn8zX+6fY4yw8vpQM6 LU0SdT8xH7ZauiWOhyt/M9Bs1qaEo2eY6ZC1CnpAkooeSjyjl/R8AWiqP/OCjJHmPN UJw+LgYoVEo/ytNqsRB9e9vDBW+hxbP1keDpC31kX3G4/6cXPOgH9dQ7dwPlsMARKC 5+DE7KJ5t4kmw== Subject: [PATCH 10/10] xfs: test delalloc quota leak when chprojid fails From: "Darrick J. Wong" To: djwong@kernel.org, guaneryu@gmail.com Cc: linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me Date: Mon, 08 Mar 2021 20:40:58 -0800 Message-ID: <161526485870.1214319.7885928745714445687.stgit@magnolia> In-Reply-To: <161526480371.1214319.3263690953532787783.stgit@magnolia> References: <161526480371.1214319.3263690953532787783.stgit@magnolia> User-Agent: StGit/0.19 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org From: Darrick J. Wong This is a regression test for a bug in the XFS implementation of FSSETXATTR. When we try to change a file's project id, the quota reservation code will update the incore quota reservations for delayed allocation blocks. Unfortunately, it does this before we finish validating all the FSSETXATTR parameters, which means that if we decide to bail out, we also fail to undo the incore changes. Signed-off-by: Darrick J. Wong Reviewed-by: Chandan Babu R --- .gitignore | 1 + src/Makefile | 2 + src/chprojid_fail.c | 92 +++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/765 | 71 +++++++++++++++++++++++++++++++++++++++ tests/xfs/765.out | 4 ++ tests/xfs/group | 1 + 6 files changed, 170 insertions(+), 1 deletion(-) create mode 100644 src/chprojid_fail.c create mode 100755 tests/xfs/765 create mode 100644 tests/xfs/765.out diff --git a/.gitignore b/.gitignore index 03c03be5..3af8e207 100644 --- a/.gitignore +++ b/.gitignore @@ -58,6 +58,7 @@ /src/bulkstat_null_ocount /src/bulkstat_unlink_test /src/bulkstat_unlink_test_modified +/src/chprojid_fail /src/cloner /src/dbtest /src/devzero diff --git a/src/Makefile b/src/Makefile index 38ee6718..3d729a34 100644 --- a/src/Makefile +++ b/src/Makefile @@ -29,7 +29,7 @@ LINUX_TARGETS = xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader \ attr-list-by-handle-cursor-test listxattr dio-interleaved t_dir_type \ dio-invalidate-cache stat_test t_encrypted_d_revalidate \ attr_replace_test swapon mkswap t_attr_corruption t_open_tmpfiles \ - fscrypt-crypt-util bulkstat_null_ocount splice-test + fscrypt-crypt-util bulkstat_null_ocount splice-test chprojid_fail SUBDIRS = log-writes perf diff --git a/src/chprojid_fail.c b/src/chprojid_fail.c new file mode 100644 index 00000000..8c5b5fee --- /dev/null +++ b/src/chprojid_fail.c @@ -0,0 +1,92 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * Copyright (c) 2021 Oracle. All Rights Reserved. + * Author: Darrick J. Wong + * + * Regression test for failing to undo delalloc quota reservations when + * changing project id and we fail some other FSSETXATTR validation. + */ +#include +#include +#include +#include +#include +#include +#include +#include +#include + +static char zerobuf[65536]; + +int +main( + int argc, + char *argv[]) +{ + struct fsxattr fa; + ssize_t sz; + int fd, ret; + + if (argc < 2) { + printf("Usage: %s filename\n", argv[0]); + return 1; + } + + fd = open(argv[1], O_CREAT | O_TRUNC | O_RDWR, 0600); + if (fd < 0) { + perror(argv[1]); + return 2; + } + + /* Zero the project id and the extent size hint. */ + ret = ioctl(fd, FS_IOC_FSGETXATTR, &fa); + if (ret) { + perror("FSGETXATTR check file"); + return 2; + } + + if (fa.fsx_projid != 0 || fa.fsx_extsize != 0) { + fa.fsx_projid = 0; + fa.fsx_extsize = 0; + ret = ioctl(fd, FS_IOC_FSSETXATTR, &fa); + if (ret) { + perror("FSSETXATTR zeroing"); + return 2; + } + } + + /* Dirty a few kb of a file to create delalloc extents. */ + sz = write(fd, zerobuf, sizeof(zerobuf)); + if (sz != sizeof(zerobuf)) { + perror("delalloc write"); + return 2; + } + + /* + * The regression we're trying to test happens when the fsxattr input + * validation decides to bail out after the chown quota reservation has + * been made on a file containing delalloc extents. Extent size hints + * can't be set on non-empty files and we can't check the value until + * we've reserved resources and taken the file's ILOCK, so this is a + * perfect vector for triggering this condition. In this way we set up + * a FSSETXATTR call that will fail. + */ + ret = ioctl(fd, FS_IOC_FSGETXATTR, &fa); + if (ret) { + perror("FSGETXATTR"); + return 2; + } + + fa.fsx_projid = 23652; + fa.fsx_extsize = 2; + fa.fsx_xflags |= FS_XFLAG_EXTSIZE; + + ret = ioctl(fd, FS_IOC_FSSETXATTR, &fa); + if (ret) { + printf("FSSETXATTRR should fail: %s\n", strerror(errno)); + return 0; + } + + /* Uhoh, that FSSETXATTR call should have failed! */ + return 3; +} diff --git a/tests/xfs/765 b/tests/xfs/765 new file mode 100755 index 00000000..68b89ce3 --- /dev/null +++ b/tests/xfs/765 @@ -0,0 +1,71 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2021 Oracle. All Rights Reserved. +# +# FS QA Test No. 765 +# +# Regression test for failing to undo delalloc quota reservations when changing +# project id but we fail some other part of FSSETXATTR validation. If we fail +# the test, we trip debugging assertions in dmesg. This is a regression test +# for commit 1aecf3734a95 ("xfs: fix chown leaking delalloc quota blocks when +# fssetxattr fails"). + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/quota + +# real QA test starts here +_supported_fs xfs +_require_command "$FILEFRAG_PROG" filefrag +_require_test_program "chprojid_fail" +_require_quota +_require_scratch + +rm -f $seqres.full + +echo "Format filesystem" | tee -a $seqres.full +_scratch_mkfs > $seqres.full +_qmount_option 'prjquota' +_qmount +_require_prjquota $SCRATCH_DEV + +# Make sure that a regular buffered write produces delalloc reservations. +$XFS_IO_PROG -f -c 'pwrite 0 64k' $SCRATCH_MNT/testy &> /dev/null +$FILEFRAG_PROG -v $SCRATCH_MNT/testy 2>&1 | grep -q delalloc || \ + _notrun "test requires delayed allocation writes" +rm -f $SCRATCH_MNT/testy + +echo "Run test program" +$XFS_QUOTA_PROG -f -x -c 'report -ap' $SCRATCH_MNT >> $seqres.full +$here/src/chprojid_fail $SCRATCH_MNT/blah + +# The regression we're testing for is an accounting bug involving delalloc +# reservations. FSSETXATTR does not itself cause dirty data writeback, so we +# assume that if the file still has delalloc extents, then it must have had +# them when chprojid_fail was running, and therefore the test was set up +# correctly. There's a slight chance that background writeback can sneak in +# and flush the file, but this should be a small enough gap. +$FILEFRAG_PROG -v $SCRATCH_MNT/blah 2>&1 | grep -q delalloc || \ + echo "file didn't get delalloc extents, test invalid?" + +# Make a note of current quota status for diagnostic purposes +$XFS_QUOTA_PROG -f -x -c 'report -ap' $SCRATCH_MNT >> $seqres.full + +# success, all done +status=0 +exit diff --git a/tests/xfs/765.out b/tests/xfs/765.out new file mode 100644 index 00000000..d5f8fc11 --- /dev/null +++ b/tests/xfs/765.out @@ -0,0 +1,4 @@ +QA output created by 765 +Format filesystem +Run test program +FSSETXATTRR should fail: Invalid argument diff --git a/tests/xfs/group b/tests/xfs/group index d7aafc94..84468d10 100644 --- a/tests/xfs/group +++ b/tests/xfs/group @@ -505,4 +505,5 @@ 760 auto quick rw collapse punch insert zero prealloc 761 auto quick realtime 763 auto quick rw realtime +765 auto quick quota 915 auto quick quota