From patchwork Tue Nov 26 01:20:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13885389 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3C6D9366; Tue, 26 Nov 2024 01:20:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732584048; cv=none; b=but/IYYfJGNcEOVgl33W4R4XFwJCHcCi+2/EmWYZhSwLqGMno2+c8HPp3WFHV4VDYOtwKFaLlTARCUdNb0GjS0lvLfZwqAriVWHMM1wUDVdmCVZSQ9SuMEtrpL77pAJ2CxMj54NVUwzbdpnX+LG0Rb6LBKQ/y5Oo/OdLEUMgNao= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732584048; c=relaxed/simple; bh=118F3cLeQLECfTPe2n1rCyAfJUReJzNfz7TOatchghk=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=O7BoaF2KIYtOn6iR5IWf/x07fxBBGHu2i0/Ph0SD4nChQlmjDq0JWbivrFS4ZD1iIrKmqZTTbEMSKZCsAJV7iDL+59Vv3P/KRLdxGNdnlX/jHGiEQHnAuh4hFPb5lX+/ZCf1tfS2Dktq/19agP7X5ZgJve6c0lOECCDafyrlHPI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=E3DTKTPL; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="E3DTKTPL" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1404CC4CECE; Tue, 26 Nov 2024 01:20:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1732584048; bh=118F3cLeQLECfTPe2n1rCyAfJUReJzNfz7TOatchghk=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=E3DTKTPLk8idIiXl45llLT53UVm4akjJ7AHJrBV74aP7KNAtKAOxcnQAzZfvcp/6J tXu6qg3sTK7pwYU0QyPUJZFqqW3b4FuIN52cogkJwpbl3DZIpT8Es5L59APUGPai7E 1aAvuu5Gp/dJMHxxKyXt/BE7B6JM0A/ylITKqr6MCljQYANkdtWSp2YUK7HZ6fAoIM zMX7grPZ+XsAn7VmkO595OPAMpDF6KhdqhAFo3agsVS/gmhAUkPTSPRmSha2GaBMnS oy/l+uxieGOLqvc8fwvwgpfTfRB6uRJsV7EbWP/B5sED0REdJZ5e4KKm187ybjBKJO U5TnE5NByn1NQ== Date: Mon, 25 Nov 2024 17:20:47 -0800 Subject: [PATCH 01/16] generic/757: fix various bugs in this test From: "Darrick J. Wong" To: djwong@kernel.org, zlang@redhat.com Cc: hch@lst.de, fstests@vger.kernel.org, linux-xfs@vger.kernel.org Message-ID: <173258395086.4031902.11102441101818456621.stgit@frogsfrogsfrogs> In-Reply-To: <173258395050.4031902.8257740212723106524.stgit@frogsfrogsfrogs> References: <173258395050.4031902.8257740212723106524.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Fix this test so the check doesn't fail on XFS, and restrict runtime to 100 loops because otherwise this test takes many hours. Signed-off-by: "Darrick J. Wong" Reviewed-by: Christoph Hellwig Reviewed-by: Zorro Lang --- tests/generic/757 | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/tests/generic/757 b/tests/generic/757 index 0ff5a8ac00182b..37cf49e6bc7fd9 100755 --- a/tests/generic/757 +++ b/tests/generic/757 @@ -63,9 +63,14 @@ prev=$(_log_writes_mark_to_entry_number mkfs) cur=$(_log_writes_find_next_fua $prev) [ -z "$cur" ] && _fail "failed to locate next FUA write" -while [ ! -z "$cur" ]; do +while _soak_loop_running $((100 * TIME_FACTOR)); do _log_writes_replay_log_range $cur $SCRATCH_DEV >> $seqres.full + # xfs_repair won't run if the log is dirty + if [ $FSTYP = "xfs" ]; then + _scratch_mount + _scratch_unmount + fi _check_scratch_fs prev=$cur From patchwork Tue Nov 26 01:21:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13885390 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 544E98F40; Tue, 26 Nov 2024 01:21:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732584064; cv=none; b=Cny1Chr1WYcujHDZQ9R78y3U9kNEjMI90R0WePQ0JEsTT29crK1iHmESntucfL6KFDQ7wolw0or2q0IyoJFcOA+BBwOABivBBPc4Z+Mt+d6QsyxN2zuwMKtedPid0OTkrB4MN/11cQXkBUseN3lRCzJlMI9sW3KXa4yqCO7+TQE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732584064; c=relaxed/simple; bh=lGeUt/i/gDeyl0ELHUQ7/2iFviEl45XJmogXUkGtvJ0=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=n2t4cTr4CxJc6XY4w9AXw2P4G0GGjB6J+y15hJTuZT5siCTK/SV5vx+QD4URlTvhpSTy2Zj5NsyaFBihQCLaGX/1QNGxDswhk0oUgPR03rteMEiVem/Zjj3gVlZOg6lJAb4pmU27NsZrWKI9YhtCKTu2UqFJU7kDvSEPP7xBxvc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=HChpX60x; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="HChpX60x" Received: by smtp.kernel.org (Postfix) with ESMTPSA id B106CC4CECE; Tue, 26 Nov 2024 01:21:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1732584063; bh=lGeUt/i/gDeyl0ELHUQ7/2iFviEl45XJmogXUkGtvJ0=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=HChpX60xdVPbrdrIb32DVdKilDR5rrjjBjNoEBweHTV308QNBn+rkXzb3q2l1cDaq EvuR+hCKwrh1JgwWkMaCz9cHW+yD3xFKAgmpe8wMKN+wSWxtmSLuZ3pDGPmLyT4XV0 jvsWc+rPJx6CLNxzxZcGu0k9F9kZrnuE2/wW1Vys1Nf0h3RG3OcFAgTwmeRsYHNCiR pFg7UlZ6zMaNokAQ1AJOK0OcZPy27O3tudArC4d5nkocn29nYko4XYchX3jPDpgFou fSuV0OfcKgmxXsFHzIIv9KiZVIAF7tkVe79gzvfeZpGpV2NPE5j39aD96yNFJmMKnb FqNxYYII5MYIw== Date: Mon, 25 Nov 2024 17:21:03 -0800 Subject: [PATCH 02/16] generic/757: convert to thinp From: "Darrick J. Wong" To: djwong@kernel.org, zlang@redhat.com Cc: hch@lst.de, fstests@vger.kernel.org, linux-xfs@vger.kernel.org Message-ID: <173258395101.4031902.14954667811124439467.stgit@frogsfrogsfrogs> In-Reply-To: <173258395050.4031902.8257740212723106524.stgit@frogsfrogsfrogs> References: <173258395050.4031902.8257740212723106524.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Convert this test to use dm-thinp so that discards always zero the data. This prevents weird replay problems if the scratch device doesn't guarantee that read after discard returns zeroes. Signed-off-by: "Darrick J. Wong" Reviewed-by: Christoph Hellwig Reviewed-by: Zorro Lang --- tests/generic/757 | 23 ++++++++++++++++------- 1 file changed, 16 insertions(+), 7 deletions(-) diff --git a/tests/generic/757 b/tests/generic/757 index 37cf49e6bc7fd9..6c13c6af41c57c 100755 --- a/tests/generic/757 +++ b/tests/generic/757 @@ -8,12 +8,13 @@ # This can be seen on subpage FSes on Linux 6.4. # . ./common/preamble -_begin_fstest auto quick metadata log recoveryloop aio +_begin_fstest auto quick metadata log recoveryloop aio thin _cleanup() { cd / _log_writes_cleanup &> /dev/null + _dmthin_cleanup rm -f $tmp.* } @@ -23,11 +24,14 @@ _cleanup() fio_config=$tmp.fio +. ./common/dmthin . ./common/dmlogwrites -_require_scratch +# Use thin device as replay device, which requires $SCRATCH_DEV +_require_scratch_nocheck _require_aiodio _require_log_writes +_require_dm_target thin-pool cat >$fio_config <> $seqres.full -_log_writes_init $SCRATCH_DEV +# Use a thin device to provide deterministic discard behavior. Discards are used +# by the log replay tool for fast zeroing to prevent out-of-order replay issues. +_test_unmount +sectors=$(blockdev --getsz $SCRATCH_DEV) +sectors=$((sectors * 90 / 100)) +_dmthin_init $sectors $sectors +_log_writes_init $DMTHIN_VOL_DEV _log_writes_mkfs >> $seqres.full 2>&1 _log_writes_mark mkfs @@ -64,14 +74,13 @@ cur=$(_log_writes_find_next_fua $prev) [ -z "$cur" ] && _fail "failed to locate next FUA write" while _soak_loop_running $((100 * TIME_FACTOR)); do - _log_writes_replay_log_range $cur $SCRATCH_DEV >> $seqres.full + _log_writes_replay_log_range $cur $DMTHIN_VOL_DEV >> $seqres.full # xfs_repair won't run if the log is dirty if [ $FSTYP = "xfs" ]; then - _scratch_mount - _scratch_unmount + _dmthin_mount fi - _check_scratch_fs + _dmthin_check_fs prev=$cur cur=$(_log_writes_find_next_fua $(($cur + 1))) From patchwork Tue Nov 26 01:21:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13885391 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D9DC7366; Tue, 26 Nov 2024 01:21:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732584079; cv=none; b=JTYLXreqt9AO8fkUP7cMNlnebNIZGKjbFlnBTR6oDGcX1wyV4nFAeU5WZxzMltjr2h8M32tlHiTkuPOpq3w2MB09jBhrA85oRbvSYbTSZR5iPzS9R3/tdkoXrNz4nAjJ9TDyIwynH0gkhUBadiAxhwq5QgqNtakw5bvblFz1oIk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732584079; c=relaxed/simple; bh=MwlLblXm4yvShNCo+masInqXiO+qwCtzzMrlK5awwk4=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=jEQ6p1MeHwjbsEIsYjaoMgHKc7RWdJsmKxt9jzvrDYu39/uEYUg28SZUWMAW2Ad4ghvNJizJTUOEqYeLC28u91mQ1+RfYCfFWs0UJsmxzf+5YNf4dd2UIqBBL3evjI1piFTS/xUMnas7dELJrZtxZAoSSScds7JPgbr0mSzoq7o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=hYSJlkQo; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="hYSJlkQo" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5F45DC4CECE; Tue, 26 Nov 2024 01:21:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1732584079; bh=MwlLblXm4yvShNCo+masInqXiO+qwCtzzMrlK5awwk4=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=hYSJlkQol9PsAUxtqEGslJHKlFWGQr1MFomkOPCyaMLzNmtePorY05x5d718w9Dh9 JeO7fUakz50QwiLHBAgmfswX9RlhcL3nkEk8NfQSPSiWmkm4/2ooEe9p2KUc418hkl TOr7CEpgSfJPdQhBDrI6y724as8ZJ93K8SxflCadY//aM8PZQIrYxJM97pgH4OUHEX 6/WlTJiZhBM0bFTH1u9Ar/1tqBlyBuKQR7pv7GvRhCcC3V2mckto5wpg6zOWcJWmdw D6a48WoAluJ259hE+EDPlZrqQU0GQyXYtNhfF1TRdaf5BoomIDIVA5WtquHkxo3iZ8 ek7seAJRO4/UA== Date: Mon, 25 Nov 2024 17:21:18 -0800 Subject: [PATCH 03/16] xfs/113: fix failure to corrupt the entire directory From: "Darrick J. Wong" To: djwong@kernel.org, zlang@redhat.com Cc: fstests@vger.kernel.org, hch@lst.de, fstests@vger.kernel.org, linux-xfs@vger.kernel.org Message-ID: <173258395116.4031902.7860965991021855237.stgit@frogsfrogsfrogs> In-Reply-To: <173258395050.4031902.8257740212723106524.stgit@frogsfrogsfrogs> References: <173258395050.4031902.8257740212723106524.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong This test tries to corrupt the data blocks of a directory, but it doesn't take into account the fact that __populate_check_xfs_dir can remove enough entries to cause sparse holes in the directory. If that happens, this "file data block is unmapped" logic will cause the corruption loop to exit early. Then we can add to the directory, which causes the test to fail. Instead, create a list of mappable dir block offsets, and run 100 corruptions at a time to reduce the amount of time we spend initializing xfs_db. This fixes the regressions that I see with 32k/64k block sizes. Cc: # v2022.05.01 Fixes: c8e6dbc8812653 ("xfs: test directory metadata corruption checking and repair") Signed-off-by: "Darrick J. Wong" Reviewed-by: Christoph Hellwig --- tests/xfs/113 | 33 +++++++++++++++++++++++++++------ 1 file changed, 27 insertions(+), 6 deletions(-) diff --git a/tests/xfs/113 b/tests/xfs/113 index 094ab71f2aefec..22ac8c3fd51b80 100755 --- a/tests/xfs/113 +++ b/tests/xfs/113 @@ -52,13 +52,34 @@ _scratch_xfs_repair -n >> $seqres.full 2>&1 || _fail "xfs_repair should not fail echo "+ check dir" __populate_check_xfs_dir "${inode}" btree +dir_data_offsets() { + _scratch_xfs_db -c "inode ${inode}" -c 'bmap' | \ + awk -v leaf_lblk=$leaf_lblk \ + '{ + if ($3 >= leaf_lblk) + exit; + for (i = 0; i < $8; i++) + printf("%d\n", $3 + i); + }' +} + echo "+ corrupt dir" -loff=0 -while true; do - _scratch_xfs_db -x -c "inode ${inode}" -c "dblock ${loff}" -c "stack" | grep -q 'file data block is unmapped' && break - _scratch_xfs_db -x -c "inode ${inode}" -c "dblock ${loff}" -c "stack" -c "blocktrash -x 32 -y $((blksz * 8)) -z ${FUZZ_ARGS}" >> $seqres.full - loff="$((loff + 1))" -done +subcommands=() +while read loff; do + # run 100 commands at a time + if [ "${#subcommands[@]}" -lt 600 ]; then + subcommands+=(-c "inode ${inode}") + subcommands+=(-c "dblock ${loff}") + subcommands+=(-c "blocktrash -x 32 -y $((blksz * 8)) -z ${FUZZ_ARGS}") + continue + fi + + _scratch_xfs_db -x "${subcommands[@]}" >> $seqres.full + subcommands=() +done < <(dir_data_offsets) +if [ "${#subcommands[@]}" -gt 0 ]; then + _scratch_xfs_db -x "${subcommands[@]}" >> $seqres.full +fi echo "+ mount image && modify dir" if _try_scratch_mount >> $seqres.full 2>&1; then From patchwork Tue Nov 26 01:21:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13885392 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 602CD79DC; Tue, 26 Nov 2024 01:21:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732584095; cv=none; b=K6WZKnJKaoQViAr9MRXZtQUAgJrIDDOmHHChAGnhtzS9YGYxpm5QMKNIa7Ne68/Z4GXMvXma9XHt4A0Ux9RimKtWJdY7gl041XppdwpeNdy+FFTlSFu7lJOCDYTp4H0IieQXM/sFr6F/OZZ+zDtgING4PNN3IYGpVrPcx37KxUA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732584095; c=relaxed/simple; bh=lUHYxvGvDclHRINE9ZA+ym8oqLf7fyvuq6Cr21SiTbw=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=BoTxECJSCBNeHC4J2CAf11QEhvgaH4jMzAY8IxznVT3QE85GxqVseV3HlOwgewHtBkl6tRWqf2s5sFj7NAYgZBvHKNGPg8fhVDz4VWlqKl5jSQJmoZDF1igMZFiEjfdVCbVfLdH73tqCOJlfAHvOOorW9wXg9ZZACcPEuRagkRA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=F0C/gzvH; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="F0C/gzvH" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0AE92C4CECE; Tue, 26 Nov 2024 01:21:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1732584095; bh=lUHYxvGvDclHRINE9ZA+ym8oqLf7fyvuq6Cr21SiTbw=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=F0C/gzvHzfETkMv0mdFiyesvXVGOf81COkgkPPk49Bia5BzOJU9wUHKdjrL+v9Pvr zBi1Uqx+wfghqUNY9Iz6j9evOHPOxcDd6fbGB5rlq8pp5/mU51dBtzI7TLF7pyhRDO LbYH3/wcMAYKZaVWqPLDQ3k+cl5u4ecze9YENbXkTpUhD++x4KjLwaSDOQIVGHsm5C TUV71eKQIYg0ULgrYk8lvQlZ2zchLY7WgPzaifw/qqbssrMLk78cN8bFlKHi3mbyPy o8skKWZgfuIR0mqSDu8cNxAqcbqlFrLf/yvAGTvY5uSoLlJdFnK77Vgit9CQoyXnkt kZudMQLVibsGw== Date: Mon, 25 Nov 2024 17:21:34 -0800 Subject: [PATCH 04/16] xfs/508: fix test for 64k blocksize From: "Darrick J. Wong" To: djwong@kernel.org, zlang@redhat.com Cc: hch@lst.de, fstests@vger.kernel.org, linux-xfs@vger.kernel.org Message-ID: <173258395132.4031902.17749497021307955659.stgit@frogsfrogsfrogs> In-Reply-To: <173258395050.4031902.8257740212723106524.stgit@frogsfrogsfrogs> References: <173258395050.4031902.8257740212723106524.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong It turns out that icreate transactions will try to reserve quite a bit of space on a 64k fsblock filesystem -- enough to handle the worst case parent directory expansion, a new inode chunk, and these days a parent pointer as well. This can work out to quite a bit of space: fsblock reservation 1k 172K 4k 368K 16k 1136K 64k 3650K Unfortunately, this test sets its block quota limits at 1-2MB, so we can't even create a child file. Bump the limits up by 10x so that this test will pass even if there's more metadata size creep in the future. Fixes: f769a923f576df ("xfs: project quota ineritance flag test") Signed-off-by: "Darrick J. Wong" Reviewed-by: Christoph Hellwig --- tests/xfs/508 | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/xfs/508 b/tests/xfs/508 index ee1a0371db7d6d..1bd13e98c9f641 100755 --- a/tests/xfs/508 +++ b/tests/xfs/508 @@ -44,7 +44,7 @@ do_quota_nospc() local exp=$2 echo "Write $file, expect $exp:" | _filter_scratch - $XFS_IO_PROG -t -f -c "pwrite 0 5m" $file 2>&1 >/dev/null | \ + $XFS_IO_PROG -t -f -c "pwrite 0 50m" $file 2>&1 >/dev/null | \ _filter_xfs_io_error rm -f $file } @@ -56,7 +56,7 @@ _require_prjquota $SCRATCH_DEV mkdir $SCRATCH_MNT/dir $QUOTA_CMD -x -c 'project -s test' $SCRATCH_MNT >>$seqres.full 2>&1 -$QUOTA_CMD -x -c 'limit -p bsoft=1m bhard=2m test' $SCRATCH_MNT +$QUOTA_CMD -x -c 'limit -p bsoft=10m bhard=20m test' $SCRATCH_MNT # test the Project inheritance bit is a directory only flag, and it's set on # directory by default. Expect no complain about "project inheritance flag is From patchwork Tue Nov 26 01:21:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13885393 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2ACE4BA33; Tue, 26 Nov 2024 01:21:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732584111; cv=none; b=RwFWXQpzc92YnRQU7PiamKJ+z8Gt8vwUyXotAKZL8f+LlAzaM4n33IA4Rw2gs8+oNIZ7XzGpN6r7rkcXXHg26xPLCgNTnQSvNrRJmBlAY50WMoBcXA+sKAc+hl1AwhpK/NP/GyG/OLZZ2ZLd61AsiMNRXwaGHuzU3mmMkbzS9lY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732584111; c=relaxed/simple; bh=U5YpQvVXfme59F60JiXRVBQ66exe5J5wYQtEZW4/lyk=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=BLPmax1Sq8S2+V5/9ePPK48B0TxihvqABz0fdxHJ8wWyAG+JrDTuaNDqy2jla94fzY7y1HzMjWyzxxj/QxwYMdNJIiRN7OKLabPeE8E4eA0n9IS7tsr0Znl7Y9nnBthfwc5AqVzLzHwawu1hUZDmJwM/pA+XqRHfTKDS9u8+TeQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Es2Bs/ZN; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Es2Bs/ZN" Received: by smtp.kernel.org (Postfix) with ESMTPSA id ABDF5C4CECF; Tue, 26 Nov 2024 01:21:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1732584110; bh=U5YpQvVXfme59F60JiXRVBQ66exe5J5wYQtEZW4/lyk=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=Es2Bs/ZNjLEl2UJcljv6MdH3NZVxfG3EYCGsWaJDzkJ6ZdkS8Bsdh+ah1RZT/dgwL p/dAURMgR5cF7DeMev0jyTaNvWAgWGZSc2BwUyiPCBREhUTOwfCyN2hiqcGX75luXf i0MSCzbfBAjXFyjzlRKj/1IdPcGOJ1nKc+gKQPN9g8ML3alq/hFCX+dTazVhFlNPnP ooRsRzTmggTtnKxwRJILpdFJygZfXLSbrQDpfOGIjg786qsdgkko8JZVzXby4dc8zd lf3svwLmnXPNWp6TrY6p/FawglgT1rlUXHXOUaQ1CR30OBO+qfWTlz0Q7urPo/BIgT 2W17MxazaZQYQ== Date: Mon, 25 Nov 2024 17:21:50 -0800 Subject: [PATCH 05/16] common/rc: capture dmesg when oom kills happen From: "Darrick J. Wong" To: djwong@kernel.org, zlang@redhat.com Cc: hch@lst.de, fstests@vger.kernel.org, linux-xfs@vger.kernel.org Message-ID: <173258395147.4031902.17552037714808836810.stgit@frogsfrogsfrogs> In-Reply-To: <173258395050.4031902.8257740212723106524.stgit@frogsfrogsfrogs> References: <173258395050.4031902.8257740212723106524.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Capture the dmesg output if the OOM killer is invoked during fstests. Signed-off-by: "Darrick J. Wong" Reviewed-by: Christoph Hellwig --- common/rc | 1 + 1 file changed, 1 insertion(+) diff --git a/common/rc b/common/rc index 2ee46e5101e168..70a0f1d1c6acd9 100644 --- a/common/rc +++ b/common/rc @@ -4538,6 +4538,7 @@ _check_dmesg() -e "INFO: possible circular locking dependency detected" \ -e "general protection fault:" \ -e "BUG .* remaining" \ + -e "oom-kill" \ -e "UBSAN:" \ $seqres.dmesg if [ $? -eq 0 ]; then From patchwork Tue Nov 26 01:22:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13885394 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 78023522F; Tue, 26 Nov 2024 01:22:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732584126; cv=none; b=aBtzjG0I3NF37qzkP5+l9NAP3taiPZTlWO8KBLvLVk+GIn3OZOojVeP6vUA3hMh4Oi3fH9Sw+X6xRFRbDOmOA9yzOzcwaXvP5bXDwzFli14S+tF7KiOSCOJIlCALSOo9/gssfzz4IJ3uHFh0CFkm9OHCzrn56mm48OPzZTXemMw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732584126; c=relaxed/simple; bh=jtDg7rV60EHbv2fundEmPAeiRtCpunRCaDZWcIOnfxo=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=PHJoo2ImfT/mCWDWgMvGoqwQA9e7N6ZbqfgCRAZrDfnDtVB9PdF7UStZqJGuQgeKrH6iS9lrkK1IMQ0XFDhqj/PmG0DXZ28e2UmQumnEwzeT2BYYrBeR3niWGOe4Gz/r15VMwrXFiagBlLtTqF1U9eQcUx/z1c1LGiQhnLmPhCY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=dd2tuEV9; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="dd2tuEV9" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4D4B0C4CECE; Tue, 26 Nov 2024 01:22:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1732584126; bh=jtDg7rV60EHbv2fundEmPAeiRtCpunRCaDZWcIOnfxo=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=dd2tuEV929Or10j5T8+ow51V5kI3Qtd3YNXkeXyCuzuKLgxZMtPf+havcML2H+P4F HrmIO+5c6iYzh6FO3WthaQ8fAWnMTDAR3BMn3f/n3z9z075G781rfMjieKT+LMDzlC Nj6n1YYi6XfbuaQliutvvnYWc8rQ9fg8Ch/tmzVV4wATJP3evXTltrCpruKscwxvs0 aW0ZU9DicaO0CI63UIs92R+TbQ1Xgyo5ysMld4vf+YYzNixBuJKyak3G5dxFn+YODh 6QaApF0sa6PDL2vgE7dHltxXwG9IcrZvW5bzUnP84sZorU4hQfFy/cUwhtyTkzvwSa raRekUDpW38Rw== Date: Mon, 25 Nov 2024 17:22:05 -0800 Subject: [PATCH 06/16] generic/562: handle ENOSPC while cloning gracefully From: "Darrick J. Wong" To: djwong@kernel.org, zlang@redhat.com Cc: fstests@vger.kernel.org, linux-xfs@vger.kernel.org Message-ID: <173258395162.4031902.7701863569170725350.stgit@frogsfrogsfrogs> In-Reply-To: <173258395050.4031902.8257740212723106524.stgit@frogsfrogsfrogs> References: <173258395050.4031902.8257740212723106524.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong This test creates a couple of patterned files on a tiny filesystem, fragments the free space, clones one patterned file to the other, and checks that the entire file was cloned. However, this test doesn't work on a 64k fsblock filesystem because we've used up all the free space reservation for the rmapbt, and that causes the FICLONE to error out with ENOSPC partway through. Hence we need to detect the ENOSPC and _notrun the test. That said, it turns out that XFS has been silently dropping error codes if we managed to make some progress cloning extents. That's ok if the operation has REMAP_FILE_CAN_SHORTEN like copy_file_range does, but FICLONE/FICLONERANGE do not permit partial results, so the dropped error codes is actually an error. Therefore, this testcase now becomes a regression test for the patch to fix that. Signed-off-by: "Darrick J. Wong" Reviewed-by: Christoph Hellwig --- tests/generic/562 | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/tests/generic/562 b/tests/generic/562 index 91360c4154a6a2..36bd02911c96b8 100755 --- a/tests/generic/562 +++ b/tests/generic/562 @@ -15,6 +15,9 @@ _begin_fstest auto clone punch . ./common/filter . ./common/reflink +test "$FSTYP" = "xfs" && \ + _fixed_by_kernel_commit XXXXXXXXXX "xfs: don't drop errno values when we fail to ficlone the entire range" + _require_scratch_reflink _require_test_program "punch-alternating" _require_xfs_io_command "fpunch" @@ -48,8 +51,11 @@ while true; do done # Now clone file bar into file foo. This is supposed to succeed and not fail -# with ENOSPC for example. -_reflink $SCRATCH_MNT/bar $SCRATCH_MNT/foo >>$seqres.full +# with ENOSPC for example. However, XFS will sometimes run out of space. +_reflink $SCRATCH_MNT/bar $SCRATCH_MNT/foo >>$seqres.full 2> $tmp.err +cat $tmp.err +test "$FSTYP" = "xfs" && grep -q 'No space left on device' $tmp.err && \ + _notrun "ran out of space while cloning" # Unmount and mount the filesystem again to verify the operation was durably # persisted. From patchwork Tue Nov 26 01:22:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13885395 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2A56E7462; Tue, 26 Nov 2024 01:22:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732584142; cv=none; b=j+uCrcAlOntO1JmtkB3TAaJBMjjEpOYnxLOvt2gtehAwu2Q+9H3L6PsWnS+U2ngzOuTH7cYozHLO+EVDTWbQ4+mN7MC4gnwpboj+QM0ZCAGIXlKhGChUnNxoRMmGGxcct2rwGDO553He2P9Y+n3FkGOGIbo8jIDg3NXQNs3zXuI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732584142; c=relaxed/simple; bh=gExprrLIh6QkDIw05PgeX5U6Q/m9Rxofjk2IItTszR8=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=mQeUkNsTuRWYxWNuDs2GIcJ77HBCMDis/OFJ70OzG3zMkbDoVC5hUgTYheey4ScT1cs9a7EeWAhriaP4Pas3482eixBWwPn10K+sDjAjULZU0l4bPOLr+3ObJ3TC17pJ7vPldpvoQfEgAcrnQOBBy+XoJSj7sOEJxHoWvZrUccs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=R/A9EZEo; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="R/A9EZEo" Received: by smtp.kernel.org (Postfix) with ESMTPSA id EEE39C4CECE; Tue, 26 Nov 2024 01:22:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1732584142; bh=gExprrLIh6QkDIw05PgeX5U6Q/m9Rxofjk2IItTszR8=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=R/A9EZEoy2nDxYfOg3r8jc2dM24OWEQ/s2YbU3R5B7LFNNKoqxjBqlogvDzM4CJlS Qq31HKr8SPuagDqm1TUrBe+lvzyEBAmTY6qZq+6EsNibUPk6d5iaueklULp7o+2qpN i3T0PDz0MJJzFgCcImZ+Ib/irfB/Xdf/EaKFLjYD29lJrU6Pa+PWSsWCKVvdoQelUc uWy22Yf/SInQiA5ybNAt6ni9QRKZQmWGCJ0CNN8Pdd6+/MeEfB6KJQRyJGRgpFLQC4 N6azger0akoh9OCGzDVy0/sJvJl8eXbN2fPsC1yZvD4MA2uqIUYU3BIoYwcKvHpiiw L76chCsXtONOw== Date: Mon, 25 Nov 2024 17:22:21 -0800 Subject: [PATCH 07/16] xfs/163: skip test if we can't shrink due to enospc issues From: "Darrick J. Wong" To: djwong@kernel.org, zlang@redhat.com Cc: hch@lst.de, fstests@vger.kernel.org, linux-xfs@vger.kernel.org Message-ID: <173258395178.4031902.13169152131264583585.stgit@frogsfrogsfrogs> In-Reply-To: <173258395050.4031902.8257740212723106524.stgit@frogsfrogsfrogs> References: <173258395050.4031902.8257740212723106524.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong If this test fails due to insufficient space, skip this test. This can happen if a realtime volume is enabled on the filesystem and we cannot shrink due to the rtbitmap. Signed-off-by: "Darrick J. Wong" Reviewed-by: Christoph Hellwig --- tests/xfs/163 | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/tests/xfs/163 b/tests/xfs/163 index 2bd94060222f96..75c3113dc2fd03 100755 --- a/tests/xfs/163 +++ b/tests/xfs/163 @@ -17,13 +17,20 @@ _begin_fstest auto quick growfs shrinkfs test_shrink() { - $XFS_GROWFS_PROG -D"$1" $SCRATCH_MNT >> $seqres.full 2>&1 + $XFS_GROWFS_PROG -D"$1" $SCRATCH_MNT &> $tmp.growfs ret=$? _scratch_unmount _check_scratch_fs _scratch_mount + # If we couldn't shrink the filesystem due to lack of space, we're + # done with this test. + [ $1 -ne $dblocks ] && \ + grep -q 'No space left on device' $tmp.growfs && \ + _notrun "Could not shrink due to lack of space" + cat $tmp.growfs >> $seqres.full + $XFS_INFO_PROG $SCRATCH_MNT 2>&1 | _filter_mkfs 2>$tmp.growfs >/dev/null . $tmp.growfs [ $ret -eq 0 -a $1 -eq $dblocks ] From patchwork Tue Nov 26 01:22:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13885396 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C0F15C133; Tue, 26 Nov 2024 01:22:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732584157; cv=none; b=kF+rKiZ9BmPG/gCo8HtZodPOK1HkXFpjXClGUjj11Is02RCAwea8djl+5GRCHCDTRIj7b211cjaJ+LL3Yi/cLTxEebZ26jh3OEZ2eGAd2C0mlE1Z3e/DCiU56dtS3fJhuZTLpoD/zOJ/M6HbWYDPQhVOdnQ/Z+QcVxzAJmAlllg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732584157; c=relaxed/simple; bh=BJP7oRpeUsyY8KQ7I8nt5/4GCsG2VhbI6uP5nDt1imE=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=MgEftrVhWkhO8JbFm6YMYIhogTeuWWgY0c7UP4j9Ga7+R//40xBq92oU5XRQEIFFOp4gFWXoFCIWOdep5cnpkBxOBytA69WLD+tMyR9wF4cBWxEOJoO2+NAXVOWgaUvMLL3nwodoptr7WiYWgogWdYoAOiO+cHWpfvwofHj3ePk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Hsi/eCWV; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Hsi/eCWV" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 956F3C4CECE; Tue, 26 Nov 2024 01:22:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1732584157; bh=BJP7oRpeUsyY8KQ7I8nt5/4GCsG2VhbI6uP5nDt1imE=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=Hsi/eCWVu4E5QA54G4IsZ4WDTKBHOrLe04+87jun6MxCZL++UmJg9LFDhJONonjPL QAxQW9TyqYXJyxpC5xFc8MJdAZGmkyu6dlcSiZWQiU2W1wNJZPtcZ8aCslHKGbncsd M57805tklgMFeiWn5uLLoSg7TjIbcAjPiSLIr1ldpHFK8/EbHOX5t3vrgTvbKEsmun Pz36te/S3R/yhx/5baA8dgBckEy9yPgKhh6l7PCxXJ/iAY/PPvRgkBcfAWE6WbomYc kPKACcFOwZVxX1nmy/mDvXRgZ79Y8ZBT8XM70iJmq5d7QZJP9Mif1Chh55U669waKn DIQDtVh2I9quw== Date: Mon, 25 Nov 2024 17:22:37 -0800 Subject: [PATCH 08/16] xfs/009: allow logically contiguous preallocations From: "Darrick J. Wong" To: djwong@kernel.org, zlang@redhat.com Cc: hch@lst.de, fstests@vger.kernel.org, linux-xfs@vger.kernel.org Message-ID: <173258395193.4031902.6288465123347994448.stgit@frogsfrogsfrogs> In-Reply-To: <173258395050.4031902.8257740212723106524.stgit@frogsfrogsfrogs> References: <173258395050.4031902.8257740212723106524.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong The new rtgroups feature implements a simplistic rotor to pick the rtgroup for an initial allocation to a file. This causes test failures if the preallocations are spread across two rtgroups, which happens if there are more subtests than rtgroups. One way to fix this would be to reset the rotor then each subtest starts allocating from rtgroup 0, but the only way to do that is to cycle the scratch mount, which is a bit gross. Instead, report logically contiguous mappings as a single mapping even if the physical space is not contiguous. Unfortunately, there's not enough context in the comments to know if the test actually was checking for physical contiguity? Or if this is just an exerciser of the old preallocation calls, and it's fine as long as the file ranges are mapped (or unmapped) as desired. Messing with some awk is a lot cheaper than umount/mount cycling. Signed-off-by: "Darrick J. Wong" Reviewed-by: Christoph Hellwig --- tests/xfs/009 | 29 ++++++++++++++++++++++++++++- 1 file changed, 28 insertions(+), 1 deletion(-) diff --git a/tests/xfs/009 b/tests/xfs/009 index dde505f079f4f8..bb42ce32490df5 100755 --- a/tests/xfs/009 +++ b/tests/xfs/009 @@ -49,13 +49,26 @@ _filesize() _block_filter() { $AWK_PROG -v bsize="$bsize" ' + BEGIN { + br_pos = 0 + br_len = 0 + } + function dump_blockrange() { + if (br_len == 0) + return + printf(" [%d,%d]: BLOCKRANGE\n", br_pos, br_len) + br_pos = 0 + br_len = 0 + } /blocksize/ { + dump_blockrange() printf(" blocksize BSIZE\n") next } /CMD/ { + dump_blockrange() split($3, off, "=") offset = strtonum(off[2]) if (offset != -1) @@ -72,6 +85,7 @@ _block_filter() } /MAP/ { + dump_blockrange() split($2, off, "=") offset = strtonum(off[2]) if (offset != -1) @@ -90,6 +104,7 @@ _block_filter() } /TRUNCATE/ { + dump_blockrange() split($2, off, "=") offset = strtonum(off[2]) / bsize @@ -99,16 +114,28 @@ _block_filter() } /\[[0-9]+,[0-9]+\]:/ { - printf(" %s BLOCKRANGE\n", $1) + rangestr = gensub(/\[([0-9]+),([0-9]+)\]:/, "\\1,\\2", "g", $1); + split(rangestr, off, ",") + if (br_pos + br_len == off[1]) { + br_len += off[2]; + } else { + dump_blockrange() + br_pos = off[1]; + br_len = off[2]; + } next } { + dump_blockrange() print next } + END { + dump_blockrange() + } ' } From patchwork Tue Nov 26 01:22:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13885397 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4592ACA5A; Tue, 26 Nov 2024 01:22:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732584174; cv=none; b=jGDPEKMVmTh4opaSdMRX4IaFm7yK+kRz0NixoGUuAg/aWbKzqubVvyOle6Uf7NfkvuOCIVKTqTc8/dk/EFDsG4fxtzKfHZvs/eBshwYpLsSuRWzfvwCOim+SBvwC0Ng7nCRqWTW3xtgbuOlpg9xHuaZtc4CYnRN/lKEGay/MOHc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732584174; c=relaxed/simple; bh=y8PpZm+HcG5jNGFNt/C/yNnVzaez8vzAR40bqTAe2d0=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=H3XhjmvRsHrhIovhbHUXSFiGk6XWRSw5Y0BxAJAyDsgcLJfdwyFGKyOmSxIl7NwGi5YeTW8ELJekqzxWetiBbffl/quKjS5SOlx1frmNIKLNYEPUnZxvwewtBLU5DAN8BbVPdGceiqqsJ38OfGBDRbwGekXb48VHte9FLrUYMa8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=bW1k+MB8; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="bW1k+MB8" Received: by smtp.kernel.org (Postfix) with ESMTPSA id D7A2AC4CECE; Tue, 26 Nov 2024 01:22:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1732584173; bh=y8PpZm+HcG5jNGFNt/C/yNnVzaez8vzAR40bqTAe2d0=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=bW1k+MB8Nq9X6oNUHt1pIKQ9xGDWYtyuRxjoHDgjoP6pcJlhAGwmG6EhdB6jmO8m+ rAXJEAKtQJO5qRNydK9K7Qvf7a4YA8AaJirUcC7i4JTzveKM5j2Q3jd2kmXmMwFsYy J0okHyzF5qL0e6mBoW95piOAzG+OhadCSNrkRRY/omF66N7g2s+M+PVQojy4uxEShn LOKzACgY9R0LZlOSA4SA2RXUP3Npf3yCkgLovJ/BdsudD9qSJZyJW8rEYUimDNtm9T xy3TRyPI3lyG87/JW8pG95cfcerDdnuGAcF774QKmn4yb4CAZqWzL8AuG5gtpSh+4H MCVqZF8wjjfVg== Date: Mon, 25 Nov 2024 17:22:52 -0800 Subject: [PATCH 09/16] generic/251: use sentinel files to kill the fstrim loop From: "Darrick J. Wong" To: djwong@kernel.org, zlang@redhat.com Cc: hch@lst.de, fstests@vger.kernel.org, linux-xfs@vger.kernel.org Message-ID: <173258395208.4031902.5911808012153746451.stgit@frogsfrogsfrogs> In-Reply-To: <173258395050.4031902.8257740212723106524.stgit@frogsfrogsfrogs> References: <173258395050.4031902.8257740212723106524.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Apparently the subshell kill doesn't always take, and then the test runs for hours and hours because nothing stops it. Instead, use a sentinel file to detect when fstrim_loop should stop execing background fstrims. Signed-off-by: "Darrick J. Wong" Reviewed-by: Christoph Hellwig --- tests/generic/251 | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/tests/generic/251 b/tests/generic/251 index b432fb11937911..d59e91c3e0a33a 100755 --- a/tests/generic/251 +++ b/tests/generic/251 @@ -125,12 +125,15 @@ fstrim_loop() wait $fpid fi while [ $start -lt $fsize ] ; do + test -s $tmp.fstrim_loop || break $FSTRIM_PROG -m ${minlen}k -o ${start}k -l ${step}k $SCRATCH_MNT & fpid=$! wait $fpid start=$(( $start + $step )) done + test -s $tmp.fstrim_loop || break done + rm -f $tmp.fstrim_loop } function check_sums() { @@ -188,6 +191,7 @@ find -P . -xdev -type f -print0 | xargs -0 md5sum | sort -o $tmp/content.sums echo -n "Running the test: " pids="" +echo run > $tmp.fstrim_loop fstrim_loop & fstrim_pid=$! p=1 @@ -199,8 +203,10 @@ done echo "done." wait $pids -kill $fstrim_pid -wait $fstrim_pid +truncate -s 0 $tmp.fstrim_loop +while test -e $tmp.fstrim_loop; do + sleep 1 +done status=0 From patchwork Tue Nov 26 01:23:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13885398 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0EBFCD515; Tue, 26 Nov 2024 01:23:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732584190; cv=none; b=ZiFZ1RvGyWB2TMwfrnGWYqJlj/B3uqEltQROlIgsMjG+FT2zZjoM6Co/onIvFYJ6uY60y+DYU+1Rxc6wgOECeUvxYu1w13w5uCGESpCSdjxbWQTzOWGKqlMn3lxtAOqmxUy4zHl1pIBRWxt2gr1DpnYhCoqjwaU922D1S88E314= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732584190; c=relaxed/simple; bh=2RQv47vEcvXiBvqJDWRerC1u+CftcJabusFCx4tDSpc=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=IXrwcEtJOXIhZOmXsGgSbKBdX+y8+6SRkzBDqs92T6oxXRWawQ9TBF64JQzQIbvFtaqOuq0dvvjlU5ptKCZOvZV/uw2bckSREH6D2mA3VOGFDphfCjmcl68Jy7ioU0Ry+Ss3ue8E8njbemjeY9vVO4aA1dGwcEXybVx/PpjsCGs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=M+CEIu3n; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="M+CEIu3n" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7996DC4CECE; Tue, 26 Nov 2024 01:23:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1732584189; bh=2RQv47vEcvXiBvqJDWRerC1u+CftcJabusFCx4tDSpc=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=M+CEIu3n7+Rii6VBGuBI38UPtnybXviLCrbJfE7GnOU+jK+g5pBsdBVkd1WsWqwiY QtZjdDu67lO7qxzfa254AZBHVC1WbZzaU4KKZykq2kl8oZ6Pq6ow5NMqJCNU0Cy5aU /To4+N1+tN/LxOqrDeIfqtj6JO+1/7I7R2cBjF+cyntbQvXigm6HM9ENEyvOLLj16w SBziVcOdCUADMprDPbn22AC4XtBg+UpXddEQia7vFnZhPDEFH2XsKmVXLKSfsPDedj H1jk6xnV4bmWAGd2vCQ3qGrpwm6oIs3vmAopMw2Ssup7MPDbJFFxN/jKUmsrttzAyz UhMDRKLnNX9eQ== Date: Mon, 25 Nov 2024 17:23:09 -0800 Subject: [PATCH 10/16] generic/251: constrain runtime via time/load/soak factors From: "Darrick J. Wong" To: djwong@kernel.org, zlang@redhat.com Cc: hch@lst.de, fstests@vger.kernel.org, linux-xfs@vger.kernel.org Message-ID: <173258395223.4031902.6225785818970710804.stgit@frogsfrogsfrogs> In-Reply-To: <173258395050.4031902.8257740212723106524.stgit@frogsfrogsfrogs> References: <173258395050.4031902.8257740212723106524.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong On my test fleet, this test can run for well in excess of 20 minutes: 613 generic/251 616 generic/251 624 generic/251 630 generic/251 634 generic/251 652 generic/251 675 generic/251 749 generic/251 777 generic/251 808 generic/251 832 generic/251 946 generic/251 1082 generic/251 1221 generic/251 1241 generic/251 1254 generic/251 1305 generic/251 1366 generic/251 1646 generic/251 1936 generic/251 1952 generic/251 2358 generic/251 4359 generic/251 5325 generic/251 34046 generic/251 because it hardcodes 20 threads and 10 copies. It's not great to have a test that results in a significant fraction of the total test runtime. Fix the looping and load on this test to use LOAD and TIME_FACTOR to scale up its operations, along with the usual SOAK_DURATION override. That brings the default runtime down to less than a minute. Signed-off-by: "Darrick J. Wong" Reviewed-by: Christoph Hellwig --- tests/generic/251 | 24 ++++++++++-------------- 1 file changed, 10 insertions(+), 14 deletions(-) diff --git a/tests/generic/251 b/tests/generic/251 index d59e91c3e0a33a..b4ddda10cef403 100755 --- a/tests/generic/251 +++ b/tests/generic/251 @@ -15,7 +15,6 @@ _begin_fstest ioctl trim auto tmp=`mktemp -d` trap "_cleanup; exit \$status" 0 1 3 trap "_destroy; exit \$status" 2 15 -chpid=0 mypid=$$ # Import common functions. @@ -151,29 +150,28 @@ function check_sums() { function run_process() { local p=$1 - repeat=10 + if [ -n "$SOAK_DURATION" ]; then + local duration="$SOAK_DURATION" + else + local duration="$((30 * TIME_FACTOR))" + fi + local stopat="$(( $(date +%s) + duration))" - sleep $((5*$p))s & - export chpid=$! && wait $chpid &> /dev/null - chpid=0 - - while [ $repeat -gt 0 ]; do + sleep $((5*$p))s + while [ "$(date +%s)" -lt "$stopat" ]; do # Remove old directories. rm -rf $SCRATCH_MNT/$p - export chpid=$! && wait $chpid &> /dev/null # Copy content -> partition. mkdir $SCRATCH_MNT/$p cp -axT $content/ $SCRATCH_MNT/$p/ - export chpid=$! && wait $chpid &> /dev/null check_sums - repeat=$(( $repeat - 1 )) done } -nproc=20 +nproc=$((4 * LOAD_FACTOR)) # Copy $here to the scratch fs and make coipes of the replica. The fstests # output (and hence $seqres.full) could be in $here, so we need to snapshot @@ -194,11 +192,9 @@ pids="" echo run > $tmp.fstrim_loop fstrim_loop & fstrim_pid=$! -p=1 -while [ $p -le $nproc ]; do +for ((p = 1; p < nproc; p++)); do run_process $p & pids="$pids $!" - p=$(($p+1)) done echo "done." From patchwork Tue Nov 26 01:23:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13885399 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4F759DDA8; Tue, 26 Nov 2024 01:23:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732584205; cv=none; b=Th0inF8tGqD8KDMBTJLvU6ON/iwlvJgov/vAnTJ9pgV2wMBXOg+N+LVfHhuoK5yLBfXL18CCauTrhRSq2/hWFjRonxYHIYbNoqvV2OZ3O+uclmPoXSca7TAycIsE7mWqTcxyXo1oR3+kCG2CA15FRbR+wltjzRQ4MKMEFvAqBSg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732584205; c=relaxed/simple; bh=6CIUbx8kSjo0l3eG7u1gtvJLaK57xysVRS1XG42qpEs=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=omIhybGNtWH8J40p8B9Es0NKUHoJ/oTyhTIRal/KIVs9GrPy/EQSMslbDrE6gTZCG/2BluTDdLy4spfjBST9yLJf+/GQSq2sn/ZDpIOis1icCdMlk44a8/9pdygw8uJn116C/Jbq58Xm0eEVuPQhjyyshK8rU6nWD6Ncf6j/YzQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=iyHkTMtO; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="iyHkTMtO" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 22D6DC4CECE; Tue, 26 Nov 2024 01:23:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1732584205; bh=6CIUbx8kSjo0l3eG7u1gtvJLaK57xysVRS1XG42qpEs=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=iyHkTMtOA0E5SUgrWBnPzaqNN7NhG0Bwrcznd9fZDqb+sfsgjrVLgTh31SaJXFdH4 NKMCn443RA8kCoLM3+94+fV7cUmY6VLFlygxbrVdZv5cZ1TvSF+An79TgZc0ImX3S+ LEg0jhiTBaq7nLJLgBgofe0nelADKHjuoxj2+tChcDeHbDBttn0S5Va9L15w/KoysM yx5hAEGll/bKAr0EPqyo9gXAvyNhAoYV61uPbLyIV1zCclh/26GCal34NRTqM2hU// aGZTtZaPhkITaJNAUpClRqrl/tQIGBq6UnixIgCanZ5JxzzdiiuHlnUhuzsz4uuH6H XbkGlWiDX0X4w== Date: Mon, 25 Nov 2024 17:23:24 -0800 Subject: [PATCH 11/16] generic/251: don't copy the fsstress source code From: "Darrick J. Wong" To: djwong@kernel.org, zlang@redhat.com Cc: hch@lst.de, fstests@vger.kernel.org, linux-xfs@vger.kernel.org Message-ID: <173258395238.4031902.16373799205312238046.stgit@frogsfrogsfrogs> In-Reply-To: <173258395050.4031902.8257740212723106524.stgit@frogsfrogsfrogs> References: <173258395050.4031902.8257740212723106524.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Run fsstress for a short time to generate test data to replicate on the scratch device so that we don't blow out the test runtimes on unintentionally copying .git directories or large corefiles from the developer's systems, etc. Signed-off-by: "Darrick J. Wong" Reviewed-by: Christoph Hellwig --- tests/generic/251 | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/tests/generic/251 b/tests/generic/251 index b4ddda10cef403..ec486c277c6828 100755 --- a/tests/generic/251 +++ b/tests/generic/251 @@ -173,13 +173,11 @@ function run_process() { nproc=$((4 * LOAD_FACTOR)) -# Copy $here to the scratch fs and make coipes of the replica. The fstests -# output (and hence $seqres.full) could be in $here, so we need to snapshot -# $here before computing file checksums. +# Use fsstress to create a directory tree with some variability content=$SCRATCH_MNT/orig mkdir -p $content -cp -axT $here/ $content/ - +FSSTRESS_ARGS=$(_scale_fsstress_args -p 4 -d $content -n 1000 $FSSTRESS_AVOID) +$FSSTRESS_PROG $FSSTRESS_ARGS >> $seqres.full mkdir -p $tmp ( From patchwork Tue Nov 26 01:23:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13885400 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9804B7462; Tue, 26 Nov 2024 01:23:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732584221; cv=none; b=oosE1jtrwXO/9zft6vlWEqIUoAHhhEkusRQjxrT1Sq0jmtG7sO5Y1mYkjm2RUrNi8JCTO088o3jZ8RSmbjtQJtJLpsBl6Vl4QpWKRfYDLv+V+W7tXp2/6nKnjFEWuGDWD+S9GaB3JreRiMWHzmins/pW5orG8lodEWsTmADd2qw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732584221; c=relaxed/simple; bh=aZ92cBGikadawS733SmHE9j0KKxvkfJZgfCH1/QFwe8=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=AVJruVMvz74Jet8RcpejQs/ZPWg7QQvzVs/UcRffZVRcxuuuBSt64Zx/vSerzFhypF6Oj7b4ZRHLkTNc6LxPjLsXR7bgCwuzytrjyFC1eM0rovsMyzRF3fgoFcTuP6WNvlJUjsm23Ja4JFFrJyVrboluiqXLz6y6H8udrnxQRSE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=jk/m27qR; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="jk/m27qR" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 03706C4CECE; Tue, 26 Nov 2024 01:23:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1732584221; bh=aZ92cBGikadawS733SmHE9j0KKxvkfJZgfCH1/QFwe8=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=jk/m27qRZWDpCRoReVshzp79TQpMEiIahBRQ6iY+2udm4quCnIIyNTo1o//N0XWHl pbZfmYNe7V+1rdaNbPFJ5WNgEyR9R+61Pv3vwikRdjV5Z5viPJOrLKj8lou98IaNZU xogyj8hXJjaf4l1hLhkfe4YbZ+98TdJMxnaYeJBnqhYjX0karfNou7RLxT7YZwkR7o hhm81Eg8DyARn3xJG9qYjBA4moQ9rCOLyVwyrCmWIjSxdGBJJFrQZtf9siA55NtoeW Le/lEHZ04YpZH0Zmenjh0nEvtojloQnXSeq8wO/ZEuEG9P2VCJev1mS0buY4T/HTAr AGJO+WuicQxww== Date: Mon, 25 Nov 2024 17:23:40 -0800 Subject: [PATCH 12/16] common/rc: _scratch_mkfs_sized supports extra arguments From: "Darrick J. Wong" To: djwong@kernel.org, zlang@redhat.com Cc: zlang@kernel.org, fstests@vger.kernel.org, linux-xfs@vger.kernel.org Message-ID: <173258395253.4031902.12874006046992433886.stgit@frogsfrogsfrogs> In-Reply-To: <173258395050.4031902.8257740212723106524.stgit@frogsfrogsfrogs> References: <173258395050.4031902.8257740212723106524.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Zorro Lang To give more arguments to _scratch_mkfs_sized, we generally do as: MKFS_OPTIONS="-L oldlabel $MKFS_OPTIONS" _scratch_mkfs_sized $fs_size to give "-L oldlabel" to it. But if _scratch_mkfs_sized fails, it will get rid of the whole MKFS_OPTIONS and try to mkfs again. Likes: ** mkfs failed with extra mkfs options added to "-L oldlabel -m rmapbt=1" by test 157 ** ** attempting to mkfs using only test 157 options: -d size=524288000 -b size=4096 ** But that's not the fault of "-L oldlabel". So for keeping the mkfs options ("-L oldlabel") we need, we'd better to let the scratch_mkfs_sized to support extra arguments, rather than using global MKFS_OPTIONS. Signed-off-by: Zorro Lang Reviewed-by: "Darrick J. Wong" [djwong: fix string quoting issues] Signed-off-by: "Darrick J. Wong" --- common/rc | 34 ++++++++++++++++++---------------- 1 file changed, 18 insertions(+), 16 deletions(-) diff --git a/common/rc b/common/rc index 70a0f1d1c6acd9..6acacbe4c88eea 100644 --- a/common/rc +++ b/common/rc @@ -1026,11 +1026,13 @@ _small_fs_size_mb() } # Create fs of certain size on scratch device -# _try_scratch_mkfs_sized [optional blocksize] +# _try_scratch_mkfs_sized [optional blocksize] [other options] _try_scratch_mkfs_sized() { local fssize=$1 - local blocksize=$2 + shift + local blocksize=$1 + shift local def_blksz local blocksize_opt local rt_ops @@ -1094,10 +1096,10 @@ _try_scratch_mkfs_sized() # don't override MKFS_OPTIONS that set a block size. echo $MKFS_OPTIONS |grep -E -q "b\s*size=" if [ $? -eq 0 ]; then - _try_scratch_mkfs_xfs -d size=$fssize $rt_ops + _try_scratch_mkfs_xfs -d size=$fssize $rt_ops "$@" else _try_scratch_mkfs_xfs -d size=$fssize $rt_ops \ - -b size=$blocksize + -b size=$blocksize "$@" fi ;; ext2|ext3|ext4) @@ -1108,7 +1110,7 @@ _try_scratch_mkfs_sized() _notrun "Could not make scratch logdev" MKFS_OPTIONS="$MKFS_OPTIONS -J device=$SCRATCH_LOGDEV" fi - ${MKFS_PROG} -t $FSTYP -F $MKFS_OPTIONS -b $blocksize $SCRATCH_DEV $blocks + ${MKFS_PROG} -t $FSTYP -F $MKFS_OPTIONS -b $blocksize "$@" $SCRATCH_DEV $blocks ;; gfs2) # mkfs.gfs2 doesn't automatically shrink journal files on small @@ -1123,13 +1125,13 @@ _try_scratch_mkfs_sized() (( journal_size >= min_journal_size )) || journal_size=$min_journal_size MKFS_OPTIONS="-J $journal_size $MKFS_OPTIONS" fi - ${MKFS_PROG} -t $FSTYP $MKFS_OPTIONS -O -b $blocksize $SCRATCH_DEV $blocks + ${MKFS_PROG} -t $FSTYP $MKFS_OPTIONS -O -b $blocksize "$@" $SCRATCH_DEV $blocks ;; ocfs2) - yes | ${MKFS_PROG} -t $FSTYP -F $MKFS_OPTIONS -b $blocksize $SCRATCH_DEV $blocks + yes | ${MKFS_PROG} -t $FSTYP -F $MKFS_OPTIONS -b $blocksize "$@" $SCRATCH_DEV $blocks ;; udf) - $MKFS_UDF_PROG $MKFS_OPTIONS -b $blocksize $SCRATCH_DEV $blocks + $MKFS_UDF_PROG $MKFS_OPTIONS -b $blocksize "$@" $SCRATCH_DEV $blocks ;; btrfs) local mixed_opt= @@ -1137,33 +1139,33 @@ _try_scratch_mkfs_sized() # the device is not zoned. Ref: btrfs-progs: btrfs_min_dev_size() (( fssize < $((256 * 1024 * 1024)) )) && ! _scratch_btrfs_is_zoned && mixed_opt='--mixed' - $MKFS_BTRFS_PROG $MKFS_OPTIONS $mixed_opt -b $fssize $SCRATCH_DEV + $MKFS_BTRFS_PROG $MKFS_OPTIONS $mixed_opt -b $fssize "$@" $SCRATCH_DEV ;; jfs) - ${MKFS_PROG} -t $FSTYP $MKFS_OPTIONS $SCRATCH_DEV $blocks + ${MKFS_PROG} -t $FSTYP $MKFS_OPTIONS "$@" $SCRATCH_DEV $blocks ;; reiserfs) - ${MKFS_PROG} -t $FSTYP $MKFS_OPTIONS -b $blocksize $SCRATCH_DEV $blocks + ${MKFS_PROG} -t $FSTYP $MKFS_OPTIONS -b $blocksize "$@" $SCRATCH_DEV $blocks ;; reiser4) # mkfs.resier4 requires size in KB as input for creating filesystem - $MKFS_REISER4_PROG $MKFS_OPTIONS -y -b $blocksize $SCRATCH_DEV \ + $MKFS_REISER4_PROG $MKFS_OPTIONS -y -b $blocksize "$@" $SCRATCH_DEV \ `expr $fssize / 1024` ;; f2fs) # mkfs.f2fs requires # of sectors as an input for the size local sector_size=`blockdev --getss $SCRATCH_DEV` - $MKFS_F2FS_PROG $MKFS_OPTIONS $SCRATCH_DEV `expr $fssize / $sector_size` + $MKFS_F2FS_PROG $MKFS_OPTIONS "$@" $SCRATCH_DEV `expr $fssize / $sector_size` ;; tmpfs) local free_mem=`_free_memory_bytes` if [ "$free_mem" -lt "$fssize" ] ; then _notrun "Not enough memory ($free_mem) for tmpfs with $fssize bytes" fi - export MOUNT_OPTIONS="-o size=$fssize $TMPFS_MOUNT_OPTIONS" + export MOUNT_OPTIONS="-o size=$fssize "$@" $TMPFS_MOUNT_OPTIONS" ;; bcachefs) - $MKFS_BCACHEFS_PROG $MKFS_OPTIONS --fs_size=$fssize $blocksize_opt $SCRATCH_DEV + $MKFS_BCACHEFS_PROG $MKFS_OPTIONS --fs_size=$fssize $blocksize_opt "$@" $SCRATCH_DEV ;; *) _notrun "Filesystem $FSTYP not supported in _scratch_mkfs_sized" @@ -1173,7 +1175,7 @@ _try_scratch_mkfs_sized() _scratch_mkfs_sized() { - _try_scratch_mkfs_sized $* || _notrun "_scratch_mkfs_sized failed with ($*)" + _try_scratch_mkfs_sized "$@" || _notrun "_scratch_mkfs_sized failed with ($*)" } # Emulate an N-data-disk stripe w/ various stripe units From patchwork Tue Nov 26 01:23:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13885401 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 112D6BA27; Tue, 26 Nov 2024 01:23:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732584237; cv=none; b=a7P0xYat9Bdel4LVr1cCJOSRPWA+9B7i6TOCTCagFHVW/pJBIge73QLjBlflMDyC4ma1Hw8r9a/PgBhLHahUrSm+tvlZgoN/ITS5ryPVP2IG16w7HhLTocsOdpdQ28a9IEIZqNL6Oa7x6iv/GUUc7DdzZ1jh/glW9UiT2zMQapc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732584237; c=relaxed/simple; bh=xoZYZcbhaXPVuUUb3eqxkHGXvjFqsgU9jp/QiJ6gyyE=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=pX4YqyqdEPCynNeb4bv12HzUCqW5Gf/q+zZdTsQmsFuSxrtNEeDX7Qm7jnBWTmJa6Pq1TbEhDWBSFq/ZdTUDoxEy91PTcojnRbYVWxA+PYqD2djw7ypPTERbLiUnTojBAaKJ/rVOLFkxzVfgWC/z/kxEiFQi68JONV5E1jxSME0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=goVgyehj; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="goVgyehj" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9FE48C4CECE; Tue, 26 Nov 2024 01:23:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1732584236; bh=xoZYZcbhaXPVuUUb3eqxkHGXvjFqsgU9jp/QiJ6gyyE=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=goVgyehjdw1uO+PHFS29bkInUPteGLmfAi1/pwUiJDFtfEcMpWAbfx5B06yhnLlqT iHb360wntoRLImKPgAxJkq9l/MWcmtjsMoZv13/xuzit8G7gh83q1Q6UJGBkMEaxHE Ect4mZC16sQV+I48kRQCdVZW119I8tGBy2OJroeHiMqxltWSL44j8hevmyJNUhJLJA cAE8XmyaKjduNjpHWpgCbqyP8KkhmbN82sCAe7jCvckTsrUszIy1p+96R8io+fZfTn R8VgqKXv0ATAfT2Haz8Cp3dVK/2Z7rW9tFNwGQwcCgKN2ldTIM2h3+GLWaxqSyedHe P4GKryMj6udvw== Date: Mon, 25 Nov 2024 17:23:56 -0800 Subject: [PATCH 13/16] xfs/157: do not drop necessary mkfs options From: "Darrick J. Wong" To: djwong@kernel.org, zlang@redhat.com Cc: zlang@kernel.org, fstests@vger.kernel.org, linux-xfs@vger.kernel.org Message-ID: <173258395269.4031902.7174495000706966581.stgit@frogsfrogsfrogs> In-Reply-To: <173258395050.4031902.8257740212723106524.stgit@frogsfrogsfrogs> References: <173258395050.4031902.8257740212723106524.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Zorro Lang To give the test option "-L oldlabel" to _scratch_mkfs_sized, xfs/157 does: MKFS_OPTIONS="-L oldlabel $MKFS_OPTIONS" _scratch_mkfs_sized $fs_size but the _scratch_mkfs_sized trys to keep the $fs_size, when mkfs fails with incompatible $MKFS_OPTIONS options, likes this: ** mkfs failed with extra mkfs options added to "-L oldlabel -m rmapbt=1" by test 157 ** ** attempting to mkfs using only test 157 options: -d size=524288000 -b size=4096 ** but the "-L oldlabel" is necessary, we shouldn't drop it. To avoid that, we give the "-L oldlabel" to _scratch_mkfs_sized through function parameters, not through global MKFS_OPTIONS. Signed-off-by: Zorro Lang Reviewed-by: "Darrick J. Wong" [djwong: fix more string quoting issues] Signed-off-by: "Darrick J. Wong" --- tests/xfs/157 | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/tests/xfs/157 b/tests/xfs/157 index 9b5badbaeb3c76..e102a5a10abe4b 100755 --- a/tests/xfs/157 +++ b/tests/xfs/157 @@ -66,8 +66,7 @@ scenario() { } check_label() { - MKFS_OPTIONS="-L oldlabel $MKFS_OPTIONS" _scratch_mkfs_sized $fs_size \ - >> $seqres.full + _scratch_mkfs_sized "$fs_size" "" -L oldlabel >> $seqres.full 2>&1 _scratch_xfs_db -c label _scratch_xfs_admin -L newlabel "$@" >> $seqres.full _scratch_xfs_db -c label From patchwork Tue Nov 26 01:24:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13885421 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 61370C2C6; Tue, 26 Nov 2024 01:24:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732584252; cv=none; b=Rvb33aa7pfHskktRAIVJtWAKr0dQSyXfQbqXllfzwpgOa5OkNyHFC5OUNtIhD3PZqPwTD8xD27wgEOD03usenttJm/UtZ238UbQf+kCgbzH0eCak1AosjKogj5YC51LeW3WPlcD04nn3aUTU/n8cY2aOanu5nlEvvpCv8n41NlQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732584252; c=relaxed/simple; bh=WG7sUTwQ9t+ufnDveKwxHDjJiotO1YF+NtJBqip96lc=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=U7kZb4M6g8FhpvK4tH3GD4mdEC0UgOLaKoLBOdIPpDx7aZ6DvovDbtmw8XlqF7lAVXfG1olOIf6lilqXf3DxkdtgoS8pFJarJDCA7H25Z24r8Fa0LcfmNULlQIuxghsYVwPAhl8wTi7PIhjQdqvNGIW6OmkYyWq53W1LpoIsf0I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=lWuBh2nI; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="lWuBh2nI" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 390C4C4CECE; Tue, 26 Nov 2024 01:24:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1732584252; bh=WG7sUTwQ9t+ufnDveKwxHDjJiotO1YF+NtJBqip96lc=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=lWuBh2nIKrcYb9hfx746w9tvZ+G2y4VRzoq3pJpDeuyHkmRdNJOXD832x4KYZ0eFF rtBEHDQkEDtJYgYs54wOMN84Npw68ID4ofwmZIg/SoampbDyuIjfhJC4ps1J/VUSPO /bI8dkAKCQ7Yn+7EsFopSDKZqyyFt/se2bsPpnqnNq9iADoDph5W5T+z9SOtiP1rKT sBzzTvHzgulUTaHJJgkLNAIW8z0mZvFeU77JQg63bA2OSm16m2YWMVJbQGLebgO5wt RT+t3ucIsuUBufPaE/tBpulTnRC3ddgN1NMbRlWtNBH/FnHAmIAN7bP1LSpZ01ezBi daIjTgtvKFl3Q== Date: Mon, 25 Nov 2024 17:24:11 -0800 Subject: [PATCH 14/16] generic/366: fix directio requirements checking From: "Darrick J. Wong" To: djwong@kernel.org, zlang@redhat.com Cc: fstests@vger.kernel.org, hch@lst.de, fstests@vger.kernel.org, linux-xfs@vger.kernel.org Message-ID: <173258395284.4031902.1560490632691011803.stgit@frogsfrogsfrogs> In-Reply-To: <173258395050.4031902.8257740212723106524.stgit@frogsfrogsfrogs> References: <173258395050.4031902.8257740212723106524.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong On a system with 4k-sector storage devices, this test fails with: --- /tmp/fstests/tests/generic/366.out 2024-11-17 09:04:53.161104479 -0800 +++ /var/tmp/fstests/generic/366.out.bad 2024-11-20 21:02:30.948000000 -0800 @@ -1,2 +1,34 @@ QA output created by 366 +fio: io_u error on file /opt/file1: Invalid argument: read offset=15360, buflen=512 +fio: io_u error on file /opt/file1: Invalid argument: read offset=15360, buflen=512 The cause of this failure is that we cannot do 512byte directios to a device with 4k LBAs. Update the precondition checking to exclude this scenario. Cc: # v2024.11.17 Fixes: 4c1629ae3a3a56 ("generic: new test case to verify if certain fio load will hang the filesystem") Signed-off-by: "Darrick J. Wong" Reviewed-by: Christoph Hellwig --- tests/generic/366 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/generic/366 b/tests/generic/366 index 6e7dd7279218c4..b322bcca72fecc 100755 --- a/tests/generic/366 +++ b/tests/generic/366 @@ -20,7 +20,7 @@ _begin_fstest auto quick rw . ./common/filter _require_scratch -_require_odirect +_require_odirect 512 # see fio job1 config below _require_aio _fixed_by_kernel_commit xxxxxxxxxxxx \ From patchwork Tue Nov 26 01:24:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13885422 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 16B3DD2FF; Tue, 26 Nov 2024 01:24:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732584268; cv=none; b=qseWvHvdQOHnX38AuSEdIdO+oHCoUSXoMU2lLNpjC9Lx8KfjHWY69XinJESU8X9LulkmWgGoGkhwn8Ba4vBtiCKsu8exgP1X5E2zVPnfGFqmwqdA2FHfMLHBDljbr1yPxulg2DRgrneOtmWW3c5Hi/7RoL/P7YTG9DrlC7adHqo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732584268; c=relaxed/simple; bh=VDsGXUkhjWu2r2d/kG9g9aqDH0Qd0M7McumS6moPnVc=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=NUtxbFcGjvEEcgWryjNzBTUxfu+Bf6gkem1ANyBdR7xxls3ytOjBvcDDR4fnvMYHz/Sl65Q23mWVOSS5oxnY1CyzJgde2Qx4OwRa5zMTCppdcIIgXPcYrMDVv+L9l+JB2xLFZ4DfSwoq9M+sU0fmvxWzOLRhvo1cxICwDD0TKUs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=sEpwL6p1; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="sEpwL6p1" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E8F69C4CECE; Tue, 26 Nov 2024 01:24:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1732584268; bh=VDsGXUkhjWu2r2d/kG9g9aqDH0Qd0M7McumS6moPnVc=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=sEpwL6p1+6uslijQdcZ6YCbGXFGlohAMh7Bwt98KqVaVqEPDDcSzwD84lI2f0D0NQ EDrphKk5aei2kRItnY1GNioYwzb7I6Y0XGvJ3GtgSqpEAYz9fc8YvqzMkY+NB2OKHV mGZg2CydKwOvpzX0cPw3JKFtRpp9sFWxFRhbeui/iB054k0KabHHHbMOv4FJ8KTiAD CyR+PoxAt5dv8zSivh/ucM9P9mjKWzQz3o4ZN3zk5W7pmcZizPivc4VEqouP2Qowvd XNq3ScGqBrKbZTS2gxvMIHe8RXMvHcoAr8eGEnxAijXjP1FD6t7nTDq/YrAv7rWYru bNGrK0XX48Rwg== Date: Mon, 25 Nov 2024 17:24:27 -0800 Subject: [PATCH 15/16] generic/454: actually set attr value for llamapirate subtest From: "Darrick J. Wong" To: djwong@kernel.org, zlang@redhat.com Cc: fstests@vger.kernel.org, tytso@mit.edu, fstests@vger.kernel.org, linux-xfs@vger.kernel.org Message-ID: <173258395299.4031902.7236733593217096781.stgit@frogsfrogsfrogs> In-Reply-To: <173258395050.4031902.8257740212723106524.stgit@frogsfrogsfrogs> References: <173258395050.4031902.8257740212723106524.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Ted reported that this test fails on his setup, and I noticed that I forgot to actually set a value for the xattr. In theory filesystems support zero-byte xattrs, but we might as well set and check the values so that we can make sure nobody got confused. The actual test failure comes from attr 2.4.47 refusing to set a zero-legnth xattr, whereas 2.5 and newer will. That was changed in the attr commit 0550d2bc989d39 ("Properly set and report empty attribute values") prior to 2.4.48: https://git.savannah.nongnu.org/cgit/attr.git/commit/?id=0550d2bc989d390eb25f7004ee0fae2dbc693a0d Cc: # v2024.10.28 Fixes: 9c3762ceafd430 ("misc: amend unicode confusing name tests to check for hidden tag characters") Reported-and-tested-by: tytso@mit.edu Signed-off-by: "Darrick J. Wong" Reviewed-by: Christoph Hellwig --- tests/generic/454 | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/tests/generic/454 b/tests/generic/454 index 2cc2d81ce4cc77..aec8beb8b43ca0 100755 --- a/tests/generic/454 +++ b/tests/generic/454 @@ -121,8 +121,8 @@ setf "combmark_\xe1\x80\x9c\xe1\x80\xad\xe1\x80\xaf.txt" "combining marks" setf "combmark_\xe1\x80\x9c\xe1\x80\xaf\xe1\x80\xad.txt" "combining marks" # encoding hidden tag characters in attrnames to create confusing xattrs -setf "llamapirate\xf3\xa0\x80\x81\xf3\xa0\x81\x94\xf3\xa0\x81\xa8\xf3\xa0\x81\xa5\xf3\xa0\x80\xa0\xf3\xa0\x81\xb3\xf3\xa0\x81\xa1\xf3\xa0\x81\xac\xf3\xa0\x81\xa5\xf3\xa0\x81\xb3\xf3\xa0\x80\xa0\xf3\xa0\x81\xa6\xf3\xa0\x81\xaf\xf3\xa0\x81\xb2\xf3\xa0\x80\xa0\xf3\xa0\x81\x93\xf3\xa0\x81\xa5\xf3\xa0\x81\xa1\xf3\xa0\x81\xb4\xf3\xa0\x81\xb4\xf3\xa0\x81\xac\xf3\xa0\x81\xa5\xf3\xa0\x80\xa0\xf3\xa0\x81\xb7\xf3\xa0\x81\xa5\xf3\xa0\x81\xb2\xf3\xa0\x81\xa5\xf3\xa0\x80\xa0\xf3\xa0\x81\x95\xf3\xa0\x81\x93\xf3\xa0\x81\x84\xf3\xa0\x80\xa0\xf3\xa0\x80\xb1\xf3\xa0\x80\xb2\xf3\xa0\x80\xb0\xf3\xa0\x80\xb0\xf3\xa0\x80\xb0\xf3\xa0\x80\xb0\xf3\xa0\x81\xbf" -setf "llamapirate" +setf "llamapirate\xf3\xa0\x80\x81\xf3\xa0\x81\x94\xf3\xa0\x81\xa8\xf3\xa0\x81\xa5\xf3\xa0\x80\xa0\xf3\xa0\x81\xb3\xf3\xa0\x81\xa1\xf3\xa0\x81\xac\xf3\xa0\x81\xa5\xf3\xa0\x81\xb3\xf3\xa0\x80\xa0\xf3\xa0\x81\xa6\xf3\xa0\x81\xaf\xf3\xa0\x81\xb2\xf3\xa0\x80\xa0\xf3\xa0\x81\x93\xf3\xa0\x81\xa5\xf3\xa0\x81\xa1\xf3\xa0\x81\xb4\xf3\xa0\x81\xb4\xf3\xa0\x81\xac\xf3\xa0\x81\xa5\xf3\xa0\x80\xa0\xf3\xa0\x81\xb7\xf3\xa0\x81\xa5\xf3\xa0\x81\xb2\xf3\xa0\x81\xa5\xf3\xa0\x80\xa0\xf3\xa0\x81\x95\xf3\xa0\x81\x93\xf3\xa0\x81\x84\xf3\xa0\x80\xa0\xf3\xa0\x80\xb1\xf3\xa0\x80\xb2\xf3\xa0\x80\xb0\xf3\xa0\x80\xb0\xf3\xa0\x80\xb0\xf3\xa0\x80\xb0\xf3\xa0\x81\xbf" "secret instructions" +setf "llamapirate" "no secret instructions" _getfattr --absolute-names -d "${testfile}" >> $seqres.full @@ -171,8 +171,8 @@ testf "zerojoin_moo\xe2\x80\x8ccow.txt" "zero width joiners" testf "combmark_\xe1\x80\x9c\xe1\x80\xad\xe1\x80\xaf.txt" "combining marks" testf "combmark_\xe1\x80\x9c\xe1\x80\xaf\xe1\x80\xad.txt" "combining marks" -testf "llamapirate\xf3\xa0\x80\x81\xf3\xa0\x81\x94\xf3\xa0\x81\xa8\xf3\xa0\x81\xa5\xf3\xa0\x80\xa0\xf3\xa0\x81\xb3\xf3\xa0\x81\xa1\xf3\xa0\x81\xac\xf3\xa0\x81\xa5\xf3\xa0\x81\xb3\xf3\xa0\x80\xa0\xf3\xa0\x81\xa6\xf3\xa0\x81\xaf\xf3\xa0\x81\xb2\xf3\xa0\x80\xa0\xf3\xa0\x81\x93\xf3\xa0\x81\xa5\xf3\xa0\x81\xa1\xf3\xa0\x81\xb4\xf3\xa0\x81\xb4\xf3\xa0\x81\xac\xf3\xa0\x81\xa5\xf3\xa0\x80\xa0\xf3\xa0\x81\xb7\xf3\xa0\x81\xa5\xf3\xa0\x81\xb2\xf3\xa0\x81\xa5\xf3\xa0\x80\xa0\xf3\xa0\x81\x95\xf3\xa0\x81\x93\xf3\xa0\x81\x84\xf3\xa0\x80\xa0\xf3\xa0\x80\xb1\xf3\xa0\x80\xb2\xf3\xa0\x80\xb0\xf3\xa0\x80\xb0\xf3\xa0\x80\xb0\xf3\xa0\x80\xb0\xf3\xa0\x81\xbf" -testf "llamapirate" +testf "llamapirate\xf3\xa0\x80\x81\xf3\xa0\x81\x94\xf3\xa0\x81\xa8\xf3\xa0\x81\xa5\xf3\xa0\x80\xa0\xf3\xa0\x81\xb3\xf3\xa0\x81\xa1\xf3\xa0\x81\xac\xf3\xa0\x81\xa5\xf3\xa0\x81\xb3\xf3\xa0\x80\xa0\xf3\xa0\x81\xa6\xf3\xa0\x81\xaf\xf3\xa0\x81\xb2\xf3\xa0\x80\xa0\xf3\xa0\x81\x93\xf3\xa0\x81\xa5\xf3\xa0\x81\xa1\xf3\xa0\x81\xb4\xf3\xa0\x81\xb4\xf3\xa0\x81\xac\xf3\xa0\x81\xa5\xf3\xa0\x80\xa0\xf3\xa0\x81\xb7\xf3\xa0\x81\xa5\xf3\xa0\x81\xb2\xf3\xa0\x81\xa5\xf3\xa0\x80\xa0\xf3\xa0\x81\x95\xf3\xa0\x81\x93\xf3\xa0\x81\x84\xf3\xa0\x80\xa0\xf3\xa0\x80\xb1\xf3\xa0\x80\xb2\xf3\xa0\x80\xb0\xf3\xa0\x80\xb0\xf3\xa0\x80\xb0\xf3\xa0\x80\xb0\xf3\xa0\x81\xbf" "secret instructions" +testf "llamapirate" "no secret instructions" echo "Uniqueness of keys?" crazy_keys="$(_getfattr --absolute-names -d "${testfile}" | grep -E -c '(french_|chinese_|greek_|arabic_|urk)')" From patchwork Tue Nov 26 01:24:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13885423 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AB162DDA8; Tue, 26 Nov 2024 01:24:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732584283; cv=none; b=Q7jgTkEwJYzjh1VR5w/PfxCDHzLgMcH6UxDARaa17GqKU5uzW+bQylfyowqqM75IbLPbS+fOiXNrmblMiXx714EYPuNRxV9mxN4PwrDIvE72/Brfj/rZQpDD4junw1QN4QkUFd8kFCNMA6OPNElCQrpR6CuVb9roNek0k9IsLUQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732584283; c=relaxed/simple; bh=2G1Nfh9p/id1mM/S/o+W/pEL7/5+lwMas6aaETT186o=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=MKFc+zAFou++2N0+29/4Im3dKf2w7e2S2JQ7HdlwM+cJrlw1en87uWwIDc+ETzXNKC3l8C6qmHHMyoOEDnctSvCbO0qwhiY2wY2qavA+BfxiJJbqYTgmWjvPMlzaXY8l74RYrk7Ksitt5GeKr+Z5dH/8ydq0g/vjG5kAEVLpOkc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ZTY2qtt6; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ZTY2qtt6" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7EF3CC4CECE; Tue, 26 Nov 2024 01:24:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1732584283; bh=2G1Nfh9p/id1mM/S/o+W/pEL7/5+lwMas6aaETT186o=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=ZTY2qtt6xtLWlIjGcFnXNo4BgeypQ0JRZePCb/FokAUtd8W2wqEHQYN2j4VEFDUH7 nIGQHNRi+V593eWDZR/DZHhJfwvbCJ3Oumk4kuIdF4PjybOQtRiw/4Rfp4ywNR7U6c 2upnH3TJ6+hlLPOhg9g5bP8bBK5qHSBtwYPwrRDAnODp0IUBhnb+KSmeDdLThDq7ET +g7W11AsO1krd5rG3Va1Hw5v0vrT1DKhOl6J05dYNh2xhJoAapvr8hqQvpKh9AFvp/ rbzqPrKtmWFgO6VNGsMOWN7T+sELwglt0rrqrmCOhW2G6PXMkZ7/U9cz1/C3GYnAMM 855zy1o64pvhw== Date: Mon, 25 Nov 2024 17:24:43 -0800 Subject: [PATCH 16/16] xfs/122: add tests for commitrange structures From: "Darrick J. Wong" To: djwong@kernel.org, zlang@redhat.com Cc: fstests@vger.kernel.org, linux-xfs@vger.kernel.org Message-ID: <173258395315.4031902.9082361530245352300.stgit@frogsfrogsfrogs> In-Reply-To: <173258395050.4031902.8257740212723106524.stgit@frogsfrogsfrogs> References: <173258395050.4031902.8257740212723106524.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Update this test to check the ioctl structure for XFS_IOC_COMMIT_RANGE, which was added in 6.12. This will be the last ever addition to xfs/122, because in 6.13 we moved the ondisk structure checks to libxfs after which we'll be able to _notrun this test on newer codebases. Signed-off-by: "Darrick J. Wong" Reviewed-by: Christoph Hellwig --- tests/xfs/122.out | 1 + 1 file changed, 1 insertion(+) diff --git a/tests/xfs/122.out b/tests/xfs/122.out index 60d8294551b1c8..4dc7d7d0a3602b 100644 --- a/tests/xfs/122.out +++ b/tests/xfs/122.out @@ -76,6 +76,7 @@ sizeof(struct xfs_bulk_ireq) = 64 sizeof(struct xfs_bulkstat) = 192 sizeof(struct xfs_bulkstat_req) = 64 sizeof(struct xfs_clone_args) = 32 +sizeof(struct xfs_commit_range) = 88 sizeof(struct xfs_cud_log_format) = 16 sizeof(struct xfs_cui_log_format) = 16 sizeof(struct xfs_da3_blkinfo) = 56 From patchwork Tue Nov 26 20:27:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13886343 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B0689193419; Tue, 26 Nov 2024 20:27:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732652850; cv=none; b=XSVuJZEq9X6PWBrjrOE+mxRE8UQ6YGtLBC1X5Po+hJV8kHRB+LWY+bHP+ClEPsXBaUEw2CXsMnhvsDm6/9HAvWRqFq/sHoRDQbQuLO5W/NI2+o+U35WcP1yQblWexkT4WKAHFD/y4Tf4KT1Hjj8SnZ91t2aE2Mk9PZ8d6ClgDqM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732652850; c=relaxed/simple; bh=TYTTEcADoj0P4RwbnNGjWDRcHXFtfoVsq0hH5omAcuw=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=Doijpd0HW0b3xcGLhNJhDaxKkeh4WPe/hEHeZCvgsdQRfM4rzEQR80DBvH1XK5xDqcvBRWGcLwkUSeoNGiddGbHth3/xDY59IFwQnLMGM8cVjpUzAUkDXhOtUHFrJ4u/bd5b17Za5dwenbgQ65ild91EJKBwS4ZlFeY0gfFflMY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=k9ursXUR; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="k9ursXUR" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 446DFC4CECF; Tue, 26 Nov 2024 20:27:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1732652850; bh=TYTTEcADoj0P4RwbnNGjWDRcHXFtfoVsq0hH5omAcuw=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=k9ursXURTUEReE0n5dqp+zjj2+4ENLdNB9yv2RC4OrNDHQkiDYjAc4tR17qb+oTQk pdfwKjZPJiCLlBoOaExaJPBSb4y08VRFbrkPVtK/xiUdge/5ZkUKQeP6Rc4xSPBVV+ 2sczYkKnqTcFsxZRPhBCs61OW5W/ww5yVF/mg2MqkYZpD10RPOBWBw0fK2LkvD84+r sAkkbYuoCHQ/A7wL2xgaVC8XwptlpY9lFucA3fwZm/jCWb3SOIVoJOrvQl53xif2B/ stzS4mUktk6FqklbX+M38JekD6c94dAVL7qqnvrx0AnpM/KTtAzDfufQrEywPe76CA OxXlXv74wlRug== Date: Tue, 26 Nov 2024 12:27:29 -0800 From: "Darrick J. Wong" To: zlang@redhat.com Cc: zlang@kernel.org, fstests@vger.kernel.org, linux-xfs@vger.kernel.org Subject: [PATCH 17/16] generic/459: prevent collisions between test VMs backed by a shared disk pool Message-ID: <20241126202729.GP9438@frogsfrogsfrogs> References: <20241126011838.GI9438@frogsfrogsfrogs> <173258395050.4031902.8257740212723106524.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: fstests@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <173258395050.4031902.8257740212723106524.stgit@frogsfrogsfrogs> From: Darrick J. Wong If you happen to be running fstests on a bunch of VMs and the VMs all have access to a shared disk pool, then it's possible that two VMs could be running generic/459 at exactly the same time. In that case, it's a VERY bad thing to have two nodes trying to create an LVM volume group named "vg_459" because one node will succeed, after which the other node will see the vg_459 volume group that it didn't create: A volume group called vg_459 already exists. Logical volume pool_459 already exists in Volume group vg_459. Logical Volume "lv_459" already exists in volume group "vg_459" But then, because this is bash, we don't abort the test script and continue executing. If we're lucky this fails when /dev/vg_459/lv_459 disappears before mkfs can run: Error accessing specified device /dev/mapper/vg_459-lv_459: No such file or directory Usage: mkfs.xfs But in the bad case both nodes write filesystems to the same device and then they trample all over each other. Fix this by adding the hostname and pid to all the LVM names so that they won't collide. Fixes: 461dad511f6b91 ("generic: Test filesystem lockup on full overprovisioned dm-thin") Signed-off-by: "Darrick J. Wong" Reviewed-by: Christoph Hellwig --- tests/generic/459 | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/tests/generic/459 b/tests/generic/459 index 98177f6b5ef8fb..32ee899f929819 100755 --- a/tests/generic/459 +++ b/tests/generic/459 @@ -47,10 +47,17 @@ _require_command "$THIN_CHECK_PROG" thin_check _require_freeze _require_odirect -vgname=vg_$seq -lvname=lv_$seq -poolname=pool_$seq -snapname=snap_$seq +# Create all the LVM names with the hostname and pid so that we don't have any +# collisions between VMs running from a shared pool of disks. Hyphens become +# underscores because LVM turns those into double hyphens, which messes with +# accessing /dev/mapper/$vg-$lv (which you're not supposed to do but this test +# does anyway). +lvmsuffix="${seq}_$(hostname -s | tr '-' '_')_$$" + +vgname=vg_$lvmsuffix +lvname=lv_$lvmsuffix +poolname=pool_$lvmsuffix +snapname=snap_$lvmsuffix origpsize=200 virtsize=300 newpsize=300