From patchwork Thu Jan 16 23:25:26 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13942517 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5B6FC158520; Thu, 16 Jan 2025 23:25:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737069927; cv=none; b=eTRNOCXk7Cb346TxkRDbGlNsiFQ2y1p+7Tjk77rFqduCJTmf60Y4h/iV5bFlwFcGBkgc0wddwLy54E5VBmSvXBgkzrPijOrqJZtdAV1C6VS7J8UugjRrh5rSrD4CNboI4SlsjbY1H7MscJippn2H1WMiHz7I/Q/SNtE3qVfNRlA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737069927; c=relaxed/simple; bh=hRKBO2pRMMV0iWpFdRA83ypF1qZut8GOYRoq6m0ZLZg=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=kYEUMhJYMFt1Vg0my0mP6DaNEVw2o6khzWtuhRViWdnB545s3HmzUw0UFBee0VaqsAvQspcdr41rS8+d8au4KNBL4tzVPZRpwJibKKE3hnZKGrCnVIZwKHRrm4ZzrUP3Im1Z8KBsrsUIVPV6E2l9v2RtWf49LtNLqG6NrcmhaOA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=VltWNgmq; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="VltWNgmq" Received: by smtp.kernel.org (Postfix) with ESMTPSA id D6CD3C4CED6; Thu, 16 Jan 2025 23:25:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1737069926; bh=hRKBO2pRMMV0iWpFdRA83ypF1qZut8GOYRoq6m0ZLZg=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=VltWNgmq0zstUJjOQhvLgdPLO3fIVfJ3jWlQE8vjyZfprNYo1EtGYb2PQ2uXicRCR NyKh2X7Vxm2W2BZ5+h4dm4qYcFSGV0PXKroY7m8jhIKedX0wkmh5bb66gYzcBvFt4l d3zzC9UL45t9kDNKlpU+7bxjvMRogjYxzfoBQu0LCTDQtDfSim8i5gnSBWe4shFaWn aHzz/qZalyOhE7uEVfBzxBxypFzBL50tedgU/p0O8kha7lgibMYlJOCPvdcTFvrQVt 8KGUTRJX2+/0htLmtop3pGnXlwf6W2k0KxK8x2aZ5A7BaVcgmMEXTm3mqR7ifTbRzM kf1rsmskd+1Kg== Date: Thu, 16 Jan 2025 15:25:26 -0800 Subject: [PATCH 01/23] generic/476: fix fsstress process management From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: hch@lst.de, fstests@vger.kernel.org, linux-xfs@vger.kernel.org Message-ID: <173706974091.1927324.4002591248645301199.stgit@frogsfrogsfrogs> In-Reply-To: <173706974044.1927324.7824600141282028094.stgit@frogsfrogsfrogs> References: <173706974044.1927324.7824600141282028094.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong generic/476 never used to run fsstress in the background, but 8973af00ec212f made it do that. This is incorrect, because now 476 runs for three seconds (i.e. long enough to fall out the bottom of the test and end up in _cleanup), ignoring any SOAK_DURATION/TIME_FACTOR settings. Cc: # v2024.12.08 Fixes: 8973af00ec212f ("fstests: cleanup fsstress process management") Signed-off-by: "Darrick J. Wong" Reviewed-by: Dave Chinner --- tests/generic/476 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/generic/476 b/tests/generic/476 index 769b3745f75689..4537fcd77d2f07 100755 --- a/tests/generic/476 +++ b/tests/generic/476 @@ -22,7 +22,7 @@ nr_ops=$((25000 * nr_cpus * TIME_FACTOR)) fsstress_args=(-w -d $SCRATCH_MNT -n $nr_ops -p $nr_cpus) test -n "$SOAK_DURATION" && fsstress_args+=(--duration="$SOAK_DURATION") -_run_fsstress_bg "${fsstress_args[@]}" +_run_fsstress "${fsstress_args[@]}" # success, all done status=0 From patchwork Thu Jan 16 23:25:42 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13942518 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 14E07232438; Thu, 16 Jan 2025 23:25:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737069943; cv=none; b=PfkfhP5XUkUc8BKHHKah9h0g/tpHmfMM6eTDYaNJxOASUEkjtBteGq5uUIOv5SOHfm9clAP4x/kzuVO7YtpAynj2ao20xPSuDqRxLxGTETVtKYnS71lTgBrutW4sq+Efz3s+knVjOyIXjnOoa62cjX/cwNsFUuq/bhFcE+ydgXI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737069943; c=relaxed/simple; bh=rzffV7zKDY/h9TXj41MYP7ZD0mPufvZp7Z0ydAP9svI=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Xldydgbn6GWW1ieY8G9Oi0MnFAmSN5Eto9VMRLmaFKAcP92b1ld53YXoPGenR2Mn43SKUnxERGs30FTlqSpxpQ15eBHusiuQcQlkpqWbJ1Fi9qnPyWToiVEV3NTpOsi22iNz4hv9CPrJKTvgtG0IoNwpwe1DQ9B5bHugCvVinaY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=O3TUxHus; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="O3TUxHus" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7D094C4CED6; Thu, 16 Jan 2025 23:25:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1737069942; bh=rzffV7zKDY/h9TXj41MYP7ZD0mPufvZp7Z0ydAP9svI=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=O3TUxHusoSIPhczDqmwtIeT4lKHqfuMYsU+xGla88sSbK7wDZjKA4hkGvWWPNuNWu 9rGqxaFyek5qOESGn0VkvQ9fHpFjoEV6qBQ/VHjxoOPmvZLao2hnOi7cTPSvvQ9Hc+ 0CYWW08nqnacGwgQKWljoS5up3PRQdn1kwqFgg0ZV4TfM0Odq1GnbC/Yk+OilDkAU0 Vz8vtnmLNaoojwfnmswEdMwt/xYND0EmU9hBhOgOkA7XP1Ukifetp/+ayHwzrPa50R kwdH6vF73DXdfBJF93Tkul7VJm7nPc8g8WQ0MmvtD/KyqeeAySNLcd/Qkg8x2nvRiN YCqC0um/TA/1g== Date: Thu, 16 Jan 2025 15:25:42 -0800 Subject: [PATCH 02/23] metadump: make non-local function variables more obvious From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: hch@lst.de, fstests@vger.kernel.org, linux-xfs@vger.kernel.org Message-ID: <173706974107.1927324.17123378052376946322.stgit@frogsfrogsfrogs> In-Reply-To: <173706974044.1927324.7824600141282028094.stgit@frogsfrogsfrogs> References: <173706974044.1927324.7824600141282028094.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong In _xfs_verify_metadump_v2(), we want to set up some loop devices, record the names of those loop devices, and then leave the variables in the global namespace so that _xfs_cleanup_verify_metadump can dispose of them. Elsewhere in fstests the convention for global variables is to put them in all caps to make it obvious that they're global and not local variables, so do that here too. Cc: # v2024.12.08 Fixes: ce79de11337e38 ("fstests: clean up loop device instantiation") Signed-off-by: "Darrick J. Wong" Reviewed-by: Dave Chinner --- common/metadump | 28 +++++++++++++--------------- 1 file changed, 13 insertions(+), 15 deletions(-) diff --git a/common/metadump b/common/metadump index 3aa7adf76da4bb..493f8b6379dc0b 100644 --- a/common/metadump +++ b/common/metadump @@ -25,8 +25,8 @@ _xfs_cleanup_verify_metadump() test -n "$XFS_METADUMP_FILE" && rm -f "$XFS_METADUMP_FILE" if [ -n "$XFS_METADUMP_IMG" ]; then - [ -b "$md_data_loop_dev" ] && _destroy_loop_device $md_data_loop_dev - [ -b "$md_log_loop_dev" ] && _destroy_loop_device $md_log_loop_dev + [ -b "$METADUMP_DATA_LOOP_DEV" ] && _destroy_loop_device $METADUMP_DATA_LOOP_DEV + [ -b "$METADUMP_LOG_LOOP_DEV" ] && _destroy_loop_device $METADUMP_LOG_LOOP_DEV for img in "$XFS_METADUMP_IMG"*; do test -e "$img" && rm -f "$img" done @@ -100,9 +100,7 @@ _xfs_verify_metadump_v2() local metadump_file="$XFS_METADUMP_FILE" local version="-v 2" local data_img="$XFS_METADUMP_IMG.data" - local data_loop local log_img="" - local log_loop # Capture metadump, which creates metadump_file _scratch_xfs_metadump $metadump_file $metadump_args $version @@ -118,27 +116,27 @@ _xfs_verify_metadump_v2() _scratch_xfs_mdrestore $metadump_file # Create loopdev for data device so we can mount the fs - md_data_loop_dev=$(_create_loop_device $data_img) + METADUMP_DATA_LOOP_DEV=$(_create_loop_device $data_img) # Create loopdev for log device if we recovered anything - test -s "$log_img" && md_log_loop_dev=$(_create_loop_device $log_img) + test -s "$log_img" && METADUMP_LOG_LOOP_DEV=$(_create_loop_device $log_img) # Mount fs, run an extra test, fsck, and unmount - SCRATCH_DEV=$md_data_loop_dev SCRATCH_LOGDEV=$md_log_loop_dev _scratch_mount + SCRATCH_DEV=$METADUMP_DATA_LOOP_DEV SCRATCH_LOGDEV=$METADUMP_LOG_LOOP_DEV _scratch_mount if [ -n "$extra_test" ]; then - SCRATCH_DEV=$md_data_loop_dev SCRATCH_LOGDEV=$md_log_loop_dev $extra_test + SCRATCH_DEV=$METADUMP_DATA_LOOP_DEV SCRATCH_LOGDEV=$METADUMP_LOG_LOOP_DEV $extra_test fi - SCRATCH_DEV=$md_data_loop_dev SCRATCH_LOGDEV=$md_log_loop_dev _check_xfs_scratch_fs - _unmount $md_data_loop_dev + SCRATCH_DEV=$METADUMP_DATA_LOOP_DEV SCRATCH_LOGDEV=$METADUMP_LOG_LOOP_DEV _check_xfs_scratch_fs + _unmount $METADUMP_DATA_LOOP_DEV # Tear down what we created - if [ -b "$md_log_loop_dev" ]; then - _destroy_loop_device $md_log_loop_dev - unset md_log_loop_dev + if [ -b "$METADUMP_LOG_LOOP_DEV" ]; then + _destroy_loop_device $METADUMP_LOG_LOOP_DEV + unset METADUMP_LOG_LOOP_DEV rm -f $log_img fi - _destroy_loop_device $md_data_loop_dev - unset md_data_loop_dev + _destroy_loop_device $METADUMP_DATA_LOOP_DEV + unset METADUMP_DATA_LOOP_DEV rm -f $data_img } From patchwork Thu Jan 16 23:25:57 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13942519 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8942C158520; Thu, 16 Jan 2025 23:25:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737069958; cv=none; b=pBhNy8GCy4qe+xWfesN8q58e/qRJBA1CI83KdMSkM7gyJMmRLLAsrnJbmAOxYJ0QmofVYsPwBEJFMvcPAxIxv7Xao4QwvXtKwKZcBWJ5m3sTtIEtu3sZkNqk1dH7+Q3Jc2GGP/OxXPbv8b5ILTu5G06OM6SaIvCPpoVnJV+My9I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737069958; c=relaxed/simple; bh=3NLVBAXGcewNdwbuqWIMRnRIdlhR5mt7CikGt4TuJtg=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=irZeynaLBh+bt5YWChJP3BQ25165WsX6VIxJiJT9nlw0MF2rUe4ekKH4OsrNlxGV/BbDLNOh6GJ30b41K8i/8ogYJIzijwGmXzFioArgfQEEIStcmgPhJRCA0Wd5U0E0ppldGulPjQqepR2/yxPzooyl7tKuNoMagYvXlnZ20bU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=QaiIJFAU; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="QaiIJFAU" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 21565C4CED6; Thu, 16 Jan 2025 23:25:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1737069958; bh=3NLVBAXGcewNdwbuqWIMRnRIdlhR5mt7CikGt4TuJtg=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=QaiIJFAUkL7hj957oEVo2gKeT83V4b0zR/auKOph4WD4OqTk0XzchHuMi1Ooxnvbr p3m/w0xB7HjCf7UJcQNh3rWWt4M0zXIyVoPwfZnI1izNmjRjqw4jxJEFtqWpXpPt2p OjtGxDxLK1pt2HFlCLToPZqzNop87ozdZ6Orf0bgY2RGcg8KkRLFIQoVmLbBrU28zl 11YlUzF96Hk2Sa9AKEdQMNzqN+8THMYLKL+eZcsRzK7PxA5f5lFAs629TcqniDtBWo vpsYcmuNKWg2sfK/C8/nVsStBOa+gBBPD9PPeww5bLSyRbJ/PxHkSMHap+sxE8mLRh UcWRb8bKgZofg== Date: Thu, 16 Jan 2025 15:25:57 -0800 Subject: [PATCH 03/23] metadump: fix cleanup for v1 metadump testing From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: hch@lst.de, fstests@vger.kernel.org, linux-xfs@vger.kernel.org Message-ID: <173706974122.1927324.15053204612789959097.stgit@frogsfrogsfrogs> In-Reply-To: <173706974044.1927324.7824600141282028094.stgit@frogsfrogsfrogs> References: <173706974044.1927324.7824600141282028094.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong In commit ce79de11337e38, the metadump v2 tests were updated to leave the names of loop devices in some global variables so that the cleanup method can find them and remove the loop devices. Inexplicably, the metadump v1 test function was not upgraded. Do so now. Cc: # v2024.12.08 Fixes: ce79de11337e38 ("fstests: clean up loop device instantiation") Signed-off-by: "Darrick J. Wong" Reviewed-by: Dave Chinner --- common/metadump | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/common/metadump b/common/metadump index 493f8b6379dc0b..a4ec9a7f921acf 100644 --- a/common/metadump +++ b/common/metadump @@ -61,7 +61,6 @@ _xfs_verify_metadump_v1() local metadump_file="$XFS_METADUMP_FILE" local version="" local data_img="$XFS_METADUMP_IMG.data" - local data_loop # Force v1 if we detect v2 support if [[ $MAX_XFS_METADUMP_FORMAT > 1 ]]; then @@ -75,18 +74,19 @@ _xfs_verify_metadump_v1() SCRATCH_DEV=$data_img _scratch_xfs_mdrestore $metadump_file # Create loopdev for data device so we can mount the fs - data_loop=$(_create_loop_device $data_img) + METADUMP_DATA_LOOP_DEV=$(_create_loop_device $data_img) # Mount fs, run an extra test, fsck, and unmount - SCRATCH_DEV=$data_loop _scratch_mount + SCRATCH_DEV=$METADUMP_DATA_LOOP_DEV _scratch_mount if [ -n "$extra_test" ]; then - SCRATCH_DEV=$data_loop $extra_test + SCRATCH_DEV=$METADUMP_DATA_LOOP_DEV $extra_test fi - SCRATCH_DEV=$data_loop _check_xfs_scratch_fs - SCRATCH_DEV=$data_loop _scratch_unmount + SCRATCH_DEV=$METADUMP_DATA_LOOP_DEV _check_xfs_scratch_fs + _unmount $METADUMP_DATA_LOOP_DEV # Tear down what we created - _destroy_loop_device $data_loop + _destroy_loop_device $METADUMP_DATA_LOOP_DEV + unset METADUMP_DATA_LOOP_DEV rm -f $data_img } From patchwork Thu Jan 16 23:26:13 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13942520 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1B5771547FE; Thu, 16 Jan 2025 23:26:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737069974; cv=none; b=sqrVCu7a219h9+9JkKQOytZ0+S4N5WFGjbrxuiUs0afJwF8W1AakMCPG8DjGx3lGAEDsJL0Yk9JN+lRkT2Ri3s3261eDwR7yQnsqqa0lI6RDOFiLOysOOH/UuqvL6zIMVykQ/tEbqrfrnv3tMPmymmzO1dyh7ayLoUxFNktBgIc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737069974; c=relaxed/simple; bh=goLKoTyYnbCb5ETO+PtIizmJs6IzEztQ2cvcOEGVlro=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=q/Zv/a0sIDzygbliK2X/JYcZv7aart02RAu+1a+Ow/1HdQRYItGL3A3toBgG/0sb1OKBzVXsG/K0a0MhdtcuIfTP9EAcCU4NfsUpXXS7nANWk4He5YWMKzL6qvQUPnxR7GkynxSyusjgrby9zSccFNSDAPXHknX2Xjssy3vYFIo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=egHbzbuh; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="egHbzbuh" Received: by smtp.kernel.org (Postfix) with ESMTPSA id AB933C4CED6; Thu, 16 Jan 2025 23:26:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1737069973; bh=goLKoTyYnbCb5ETO+PtIizmJs6IzEztQ2cvcOEGVlro=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=egHbzbuhcdKnn4dwrZecHwYLsX207iYBUWYz6Urr2uHlMjFgaZo6xEinKxJEp/pdr eXJYxRry6okDjE5pl6o4rO9jpC/nm1NCJiK0/yMFl3jAkLZuzK3LUltJ6apkqCVJvv g+64dC4blDu4g+iajHQdGnovfXIcg5PRVhUI+5NefgL1u3mVrkQcny0kTyTDpj3I/I 7Myihf+EoFysuHPDxpMcNToI+DywLgvw2oNTIIQH5G5H9tCA19Uykuy0Y5oC/b008D +CE1iNOU8obD0FsTCkcwJTkwlGWgocAx98Z+PVO8AvPBHR1Vo+vVLxqovtNoVMdDzd fV/eyuKqyLHgg== Date: Thu, 16 Jan 2025 15:26:13 -0800 Subject: [PATCH 04/23] generic/482: _run_fsstress needs the test filesystem From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: hch@lst.de, fstests@vger.kernel.org, linux-xfs@vger.kernel.org Message-ID: <173706974137.1927324.11572571998972107262.stgit@frogsfrogsfrogs> In-Reply-To: <173706974044.1927324.7824600141282028094.stgit@frogsfrogsfrogs> References: <173706974044.1927324.7824600141282028094.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong The test filesystem is now a hard dependency of _run_fsstress because the latter copies the fsstress binary to a different name on the test filesystem: generic/482 - output mismatch (see /var/tmp/fstests/generic/482.out.bad) --- tests/generic/482.out 2024-02-28 16:20:24.262888854 -0800 +++ /var/tmp/fstests/generic/482.out.bad 2025-01-03 15:00:43.107625116 -0800 @@ -1,2 +1,3 @@ QA output created by 482 +cp: cannot create regular file '/mnt/482.fsstress': Read-only file system Silence is golden ... (Run 'diff -u /tmp/fstests/tests/generic/482.out /var/tmp/fstests/generic/482.out.bad' to see the entire diff) Cc: # v2024.12.08 Fixes: 8973af00ec212f ("fstests: cleanup fsstress process management") Signed-off-by: "Darrick J. Wong" --- tests/generic/482 | 1 - 1 file changed, 1 deletion(-) diff --git a/tests/generic/482 b/tests/generic/482 index 8c114ee03058c6..0efc026a160040 100755 --- a/tests/generic/482 +++ b/tests/generic/482 @@ -68,7 +68,6 @@ lowspace=$((1024*1024 / 512)) # 1m low space threshold # Use a thin device to provide deterministic discard behavior. Discards are used # by the log replay tool for fast zeroing to prevent out-of-order replay issues. -_test_unmount _dmthin_init $devsize $devsize $csize $lowspace _log_writes_init $DMTHIN_VOL_DEV _log_writes_mkfs >> $seqres.full 2>&1 From patchwork Thu Jan 16 23:26:28 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13942521 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BFB551547FE; Thu, 16 Jan 2025 23:26:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737069989; cv=none; b=ne2SfrFnFDquY1yvuIFEUNHC2bxwJn/dJdMy/M1Xt1h5btw/toS8fxv+fUx2E4zIARsugnOEh0mw4JbKfjFf7CEzp7R5EPxcu2ILDwTdeEWnxVhG92yfWj/UsBF4sG52RNANnNFwb5/wG9+MNJfE7ob2xj0xhlVQM6JhZwwwFmU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737069989; c=relaxed/simple; bh=T++MOsGoUnXx0+b9ACIWM05pta1tT/sMx5uB5k/yE6c=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=X5wZ6cvKjRutlx0Hc7R6/wxqv/I00W03cLAyb+myzwb2wAsYcMIbxo5JFOAgg0IBdpKn3aNyZ+VN2y01D137I4kYgyVjoi5n4u1SyEkw/LfGozB8ufH28o/X/LWBH4CJ3ftZ0S0uY/k58DEln3f05Kar5EmkFfpCut555qQVbIU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=c8p9CiCq; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="c8p9CiCq" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 47A70C4CED6; Thu, 16 Jan 2025 23:26:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1737069989; bh=T++MOsGoUnXx0+b9ACIWM05pta1tT/sMx5uB5k/yE6c=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=c8p9CiCq0ryCyuKryfbt0bbha1yhyhhoeGoPoWmQHIQU3Kwx2NkSfrH3fk+6+PjW4 g2N/0TaY/HIV1WPpCIyxmBwdxVBfrHIafsv89xYs5A35Ujy5CbfwVR+nrsdRE9/+b/ el4jtBVC7rOBU86EeWzTvGT2x3INR+WCHw37R87E0JW1yuYpYxSOWwTqaW4+V/1ncU 8ciyB6c6B4WfTWVp7stieWQr/C2xS4WIRu+YWcuj7WV0s4RNxmNWfFOZYH+Bkl9ECA E4rbC7aY0saWWgd/8GVFDen7GquYQYqJmRXXWHclb+QbojSRuw33zIvzQda4PbhPzT 3weO4bpLi4bmQ== Date: Thu, 16 Jan 2025 15:26:28 -0800 Subject: [PATCH 05/23] generic/019: don't fail if fio crashes while shutting down From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: hch@lst.de, fstests@vger.kernel.org, linux-xfs@vger.kernel.org Message-ID: <173706974152.1927324.14222114120134004551.stgit@frogsfrogsfrogs> In-Reply-To: <173706974044.1927324.7824600141282028094.stgit@frogsfrogsfrogs> References: <173706974044.1927324.7824600141282028094.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong My system (Debian 12) has fio 3.33. Once in a while, fio crashes while shutting down after it receives a SIGBUS on account of the filesystem going down. This causes the test to fail with: generic/019 - output mismatch (see /var/tmp/fstests/generic/019.out.bad) --- tests/generic/019.out 2024-02-28 16:20:24.130889521 -0800 +++ /var/tmp/fstests/generic/019.out.bad 2025-01-03 15:00:35.903564431 -0800 @@ -5,5 +5,6 @@ Start fio.. Force SCRATCH_DEV device failure +/tmp/fstests/tests/generic/019: line 112: 90841 Segmentation fault $FIO_PROG $fio_config >> $seqres.full 2>&1 Make SCRATCH_DEV device operable again Disallow global fail_make_request feature ... (Run 'diff -u /tmp/fstests/tests/generic/019.out /var/tmp/fstests/generic/019.out.bad' to see the entire diff) because the wait command will dutifully report fatal signals that kill the fio process. Unfortunately, a core dump shows that we blew up in some library's exit handler somewhere: (gdb) where #0 unlink_chunk (p=p@entry=0x55b31cb9a430, av=0x7f8b4475ec60 ) at ./malloc/malloc.c:1628 #1 0x00007f8b446222ff in _int_free (av=0x7f8b4475ec60 , p=0x55b31cb9a430, have_lock=, have_lock@entry=0) at ./malloc/malloc.c:4603 #2 0x00007f8b44624f1f in __GI___libc_free (mem=) at ./malloc/malloc.c:3385 #3 0x00007f8b3a71cf0e in ?? () from /lib/x86_64-linux-gnu/libtasn1.so.6 #4 0x00007f8b4426447c in ?? () from /lib/x86_64-linux-gnu/libgnutls.so.30 #5 0x00007f8b4542212a in _dl_call_fini (closure_map=closure_map@entry=0x7f8b44465620) at ./elf/dl-call_fini.c:43 #6 0x00007f8b4542581e in _dl_fini () at ./elf/dl-fini.c:114 #7 0x00007f8b445ca55d in __run_exit_handlers (status=0, listp=0x7f8b4475e820 <__exit_funcs>, run_list_atexit=run_list_atexit@entry=true, run_dtors=run_dtors@entry=true) at ./stdlib/exit.c:116 #8 0x00007f8b445ca69a in __GI_exit (status=) at ./stdlib/exit.c:146 #9 0x00007f8b445b3251 in __libc_start_call_main (main=main@entry=0x55b319278e10
, argc=argc@entry=2, argv=argv@entry=0x7ffec6f8b468) at ../sysdeps/nptl/libc_start_call_main.h:74 #10 0x00007f8b445b3305 in __libc_start_main_impl (main=0x55b319278e10
, argc=2, argv=0x7ffec6f8b468, init=, fini=, rtld_fini=, stack_end=0x7ffec6f8b458) at ../csu/libc-start.c:360 #11 0x000055b319278ed1 in _start () This isn't a filesystem failure, so mask this by shovelling the output to seqres.full. Signed-off-by: "Darrick J. Wong" Reviewed-by: Dave Chinner --- tests/generic/019 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/generic/019 b/tests/generic/019 index bed916b53f98e5..676ff27dab8062 100755 --- a/tests/generic/019 +++ b/tests/generic/019 @@ -109,7 +109,7 @@ _workout() _fail "failed: still able to perform integrity fsync on $SCRATCH_MNT" _kill_fsstress - wait $fio_pid + wait $fio_pid &>> $seqres.full # old fio can crash on EIO, ignore segfault reporting unset fio_pid # We expect that broken FS still can be umounted From patchwork Thu Jan 16 23:26:44 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13942522 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 085FB1547FE; Thu, 16 Jan 2025 23:26:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737070005; cv=none; b=szCryJ2hIAhySDFCqdz5hufYuKr41vn66aOG2ZszDqd4I5aujfljY8TN1Leh4j4Og6Go8SXP8kK9j2I9sz0KG6Vi4fDE4VQ9CzS2VXhpKiXpI3FOUkr/G6H98zMULzxWmSSDtWvvnuZhV8oIdAwAYAY7v+8G8yW9aqCJ3QNI5J8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737070005; c=relaxed/simple; bh=dm3+4pF3hh29oXOtz5V2tVKTeh6e8UX6hmXyRyzSKzY=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Y1ICqKDaFPp7cuoEOyP9FHutLQTNYgYw/4SxHVERRpsym3RemEt5+rSubL4HYd6ozhGpdLDMMWvlV3chet5DJBOS9D3AFlC8xvb7go7tLhyV96UCauKjnin80Ipql2PYIsucf2wGZtyMfGdVS2Gp6nxd6Hz7POIxOvyMPClWB9A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=pAZTH4Ue; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="pAZTH4Ue" Received: by smtp.kernel.org (Postfix) with ESMTPSA id D1860C4CEDD; Thu, 16 Jan 2025 23:26:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1737070004; bh=dm3+4pF3hh29oXOtz5V2tVKTeh6e8UX6hmXyRyzSKzY=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=pAZTH4UeNzIrQ2rsFefBL07b6l7CWkEc9EJv0NvPXEbIu0EEzFSNFuk8z/8Q6TBPh zYilv+WkOdqhSnBpDFsaNIjXjSny5jnGYkr6wwNzP80CCy0A3khM+O+yHuaMVt6Fng W0b98ceMsbIZd1BDOiz26cnSaRX1g7Vs8lQfhdhZiCnouZTfR9BenuZ+c22ZvVnVON 2zp+zTQCs6tzC2gWaxmEdC8+b7svWhi16BWHCaoo8cybRzIeafOd1opuM2pWT9GBoG hjOuLLbXREcRx+LpmXQHqHg7HDuzZaRjZKICCxhPTKCHA881jRjPlVyEgpueypPIru P6BcCHWDSMneg== Date: Thu, 16 Jan 2025 15:26:44 -0800 Subject: [PATCH 06/23] fuzzy: do not set _FSSTRESS_PID when exercising fsx From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: hch@lst.de, fstests@vger.kernel.org, linux-xfs@vger.kernel.org Message-ID: <173706974167.1927324.3074850676975765263.stgit@frogsfrogsfrogs> In-Reply-To: <173706974044.1927324.7824600141282028094.stgit@frogsfrogsfrogs> References: <173706974044.1927324.7824600141282028094.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong If we're not running fsstress as the scrub exerciser, don't set _FSSTRESS_PID because the _kill_fsstress call in the cleanup function will think that it has to wait for a nonexistant fsstress process. This fixes the problem of xfs/565 runtime increasing from 30s to 800s because it tries to kill a nonexistent "565.fsstress" process and then waits for the fsx loop control process, which hasn't been sent any signals. Cc: # v2024.12.08 Fixes: 8973af00ec212f ("fstests: cleanup fsstress process management") Signed-off-by: "Darrick J. Wong" Reviewed-by: Dave Chinner --- common/fuzzy | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/common/fuzzy b/common/fuzzy index 534e91dedbbb43..0a2d91542b561e 100644 --- a/common/fuzzy +++ b/common/fuzzy @@ -1392,7 +1392,11 @@ _scratch_xfs_stress_scrub() { "__stress_scrub_${exerciser}_loop" "$end" "$runningfile" \ "$remount_period" "$stress_tgt" & - _FSSTRESS_PID=$! + # The loop is a background process, so _FSSTRESS_PID is set in that + # child. Unfortunately, this process doesn't know about it. Therefore + # we need to set _FSSTRESS_PID ourselves so that cleanup tries to kill + # fsstress. + test "${exerciser}" = "fsstress" && _FSSTRESS_PID=$! if [ -n "$freeze" ]; then __stress_scrub_freeze_loop "$end" "$runningfile" & From patchwork Thu Jan 16 23:27:00 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13942523 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0CDC5232438; Thu, 16 Jan 2025 23:27:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737070021; cv=none; b=CiNr7VeIOXc4E4iFybyR58lrKyDzzc3rZe4aQ1Wjapl7V0svFIEtbbvOmQNofLy9n8+yTisS3ElfbgSt5qFYqMbTgFmOvcpRm+Xm3/nhA5ciWibIogFhXRtHTWSURyluR9XyOpyQ+8yGvhVdKT0kSAvAoiVciryMm5ldcdgqCTE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737070021; c=relaxed/simple; bh=P890i0ghPmfjRzrDu6JEzyD2Jwl2ng7LZFFsZ+80Vns=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=NxvvKrH6ZjJ3TTlWQ8n+XN6QW8qecrNmzfchvysNMPe716rUdSiCjIrrsDVN9pK0fifDZYoHsRrDCgSQHab3mSW/mNiMh8MbrY39lgJka8zgmZbcagRyw5Kx52GbxP5spczwBR8Gdy51m6ldq98WK3CoWP7Ik8ACEsyHq+B8H8k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=UhSjr07G; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="UhSjr07G" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6FF0FC4CED6; Thu, 16 Jan 2025 23:27:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1737070020; bh=P890i0ghPmfjRzrDu6JEzyD2Jwl2ng7LZFFsZ+80Vns=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=UhSjr07G83G69XdGkZGdN5amR+gXwJrXGGGBrBXhO5uaDSB7E9g6v2yBHMHUUJdNT O1GLUKgxnSA16MM5BLZJAI3RtfOBgQRi3v0jvizGinKFd0RGl4vxP48AyvRin6wKoi 90aOe/CR34XMuvSUJ5hZPzPDV05YLAoccHvmmWdKS1BcIVeRYZTdlmL3KE46x8HgYm IaA0OzbqBv0lepaxtxrisxeKQ/RQbYee2TjKso7PMAHC/7BQVdRruf+wXLSWpDfqfE cwoib5NYSJnzWvOn9TKlQMPJyuk3dg4ih9/2Wz48gH0TPVAz2OJfGyo6PKwjcEmgpP Qv6fdhpzLDdLA== Date: Thu, 16 Jan 2025 15:27:00 -0800 Subject: [PATCH 07/23] common/rc: create a wrapper for the su command From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: hch@lst.de, fstests@vger.kernel.org, linux-xfs@vger.kernel.org Message-ID: <173706974182.1927324.17074300255490410988.stgit@frogsfrogsfrogs> In-Reply-To: <173706974044.1927324.7824600141282028094.stgit@frogsfrogsfrogs> References: <173706974044.1927324.7824600141282028094.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Create a _su wrapper around the su command so that the next patch can fix all the pkill isolation code so that this test can only kill processes started for this test. Signed-off-by: "Darrick J. Wong" --- common/rc | 9 +++++++-- tests/generic/093 | 2 +- tests/generic/125 | 2 +- tests/generic/128 | 2 +- tests/generic/193 | 36 ++++++++++++++++++------------------ tests/generic/230 | 14 +++++++------- tests/generic/231 | 2 +- tests/generic/233 | 2 +- tests/generic/270 | 4 ++-- tests/generic/314 | 2 +- tests/generic/327 | 2 +- tests/generic/328 | 4 ++-- tests/generic/355 | 2 +- tests/generic/453 | 6 +++--- tests/generic/514 | 2 +- tests/generic/573 | 2 +- tests/generic/600 | 2 +- tests/generic/601 | 2 +- tests/generic/603 | 10 +++++----- tests/generic/673 | 2 +- tests/generic/674 | 2 +- tests/generic/675 | 2 +- tests/generic/680 | 2 +- tests/generic/681 | 2 +- tests/generic/682 | 2 +- tests/generic/683 | 2 +- tests/generic/684 | 2 +- tests/generic/685 | 2 +- tests/generic/686 | 2 +- tests/generic/687 | 2 +- tests/generic/688 | 2 +- tests/generic/691 | 8 ++++---- tests/generic/721 | 10 +++++----- tests/generic/726 | 2 +- tests/generic/727 | 2 +- tests/xfs/720 | 2 +- tests/xfs/795 | 2 +- tests/xfs/803 | 2 +- 38 files changed, 82 insertions(+), 77 deletions(-) diff --git a/common/rc b/common/rc index c5421c9454fa62..459be11c11c405 100644 --- a/common/rc +++ b/common/rc @@ -2740,6 +2740,11 @@ _require_user_exists() [ "$?" == "0" ] || _notrun "$user user not defined." } +_su() +{ + su "$@" +} + # check if a user exists and is able to execute commands. # Uses 'fsgqa' user as default. # @@ -2750,7 +2755,7 @@ _require_user() qa_user=$1 fi _require_user_exists $qa_user - echo /bin/true | su $qa_user + echo /bin/true | _su $qa_user [ "$?" == "0" ] || _notrun "$qa_user cannot execute commands." } @@ -2812,7 +2817,7 @@ s,^\s*$,,; _user_do() { - echo $1 | su -s /bin/bash $qa_user 2>&1 | _filter_user_do + echo $1 | _su -s /bin/bash $qa_user 2>&1 | _filter_user_do } _require_xfs_io_command() diff --git a/tests/generic/093 b/tests/generic/093 index c4e866da1c18eb..047cc821e0e608 100755 --- a/tests/generic/093 +++ b/tests/generic/093 @@ -62,7 +62,7 @@ chmod ugo+w $TEST_DIR # don't use $here/src/writemod, as we're running it as a regular user, and # $here may contain path component that a regular user doesn't have search # permission -su $qa_user -c "src/writemod $file" | filefilter +_su $qa_user -c "src/writemod $file" | filefilter cat $file # success, all done diff --git a/tests/generic/125 b/tests/generic/125 index e2bc5fa125da6e..011a51e8fa21c2 100755 --- a/tests/generic/125 +++ b/tests/generic/125 @@ -32,7 +32,7 @@ chmod a+rw $TESTFILE # don't use $here/src/ftrunc, as we're running it as a regular user, and $here # may contain path component that a regular user doesn't have search permission -su $qa_user -c "./src/ftrunc -f $TESTFILE" +_su $qa_user -c "./src/ftrunc -f $TESTFILE" if [ "$?" != "0" ]; then echo src/ftrunc returned non 0 status! diff --git a/tests/generic/128 b/tests/generic/128 index f931ca0639687b..5a497cdea7382b 100755 --- a/tests/generic/128 +++ b/tests/generic/128 @@ -26,7 +26,7 @@ cp "$(type -P ls)" $SCRATCH_MNT chmod 700 $SCRATCH_MNT/nosuid chmod 4755 $SCRATCH_MNT/ls -su -s/bin/bash - $qa_user -c "$SCRATCH_MNT/ls $SCRATCH_MNT/nosuid >/dev/null 2>&1" +_su -s/bin/bash - $qa_user -c "$SCRATCH_MNT/ls $SCRATCH_MNT/nosuid >/dev/null 2>&1" if [ $? -eq 0 ] ; then echo "Error: we shouldn't be able to ls the directory" fi diff --git a/tests/generic/193 b/tests/generic/193 index d251d3a5c4f8ee..ba557428f77b9c 100755 --- a/tests/generic/193 +++ b/tests/generic/193 @@ -69,17 +69,17 @@ echo _create_files echo "user: chown root owned file to qa_user (should fail)" -su ${qa_user} -c "chown ${qa_user} $test_root" 2>&1 | _filter_files +_su ${qa_user} -c "chown ${qa_user} $test_root" 2>&1 | _filter_files echo "user: chown root owned file to root (should fail)" -su ${qa_user} -c "chown root $test_root" 2>&1 | _filter_files +_su ${qa_user} -c "chown root $test_root" 2>&1 | _filter_files echo "user: chown qa_user owned file to qa_user (should succeed)" -su ${qa_user} -c "chown ${qa_user} $test_user" +_su ${qa_user} -c "chown ${qa_user} $test_user" # this would work without _POSIX_CHOWN_RESTRICTED echo "user: chown qa_user owned file to root (should fail)" -su ${qa_user} -c "chown root $test_user" 2>&1 | _filter_files +_su ${qa_user} -c "chown root $test_user" 2>&1 | _filter_files _cleanup_files @@ -93,19 +93,19 @@ echo _create_files echo "user: chgrp root owned file to root (should fail)" -su ${qa_user} -c "chgrp root $test_root" 2>&1 | _filter_files +_su ${qa_user} -c "chgrp root $test_root" 2>&1 | _filter_files echo "user: chgrp qa_user owned file to root (should fail)" -su ${qa_user} -c "chgrp root $test_user" 2>&1 | _filter_files +_su ${qa_user} -c "chgrp root $test_user" 2>&1 | _filter_files echo "user: chgrp root owned file to qa_user (should fail)" -su ${qa_user} -c "chgrp ${qa_user} $test_root" 2>&1 | _filter_files +_su ${qa_user} -c "chgrp ${qa_user} $test_root" 2>&1 | _filter_files echo "user: chgrp qa_user owned file to qa_user (should succeed)" -su ${qa_user} -c "chgrp ${qa_user} $test_user" +_su ${qa_user} -c "chgrp ${qa_user} $test_user" #echo "user: chgrp qa_user owned file to secondary group (should succeed)" -#su ${qa_user} -c "chgrp ${group2} $test_user" +#_su ${qa_user} -c "chgrp ${group2} $test_user" _cleanup_files @@ -119,10 +119,10 @@ echo _create_files echo "user: chmod a+r on qa_user owned file (should succeed)" -su ${qa_user} -c "chmod a+r $test_user" +_su ${qa_user} -c "chmod a+r $test_user" echo "user: chmod a+r on root owned file (should fail)" -su ${qa_user} -c "chmod a+r $test_root" 2>&1 | _filter_files +_su ${qa_user} -c "chmod a+r $test_root" 2>&1 | _filter_files # # Setup a file owned by the qa_user, but with a group ID that @@ -143,7 +143,7 @@ chown ${qa_user}:root $test_user chmod g+s $test_user # and let the qa_user change permission bits -su ${qa_user} -c "chmod a+w $test_user" +_su ${qa_user} -c "chmod a+w $test_user" stat -c '%A' $test_user # @@ -220,7 +220,7 @@ echo "with no exec perm" echo frobnozzle >> $test_user chmod ug+s $test_user echo -n "before: "; stat -c '%A' $test_user -su ${qa_user} -c "echo > $test_user" +_su ${qa_user} -c "echo > $test_user" echo -n "after: "; stat -c '%A' $test_user echo "with user exec perm" @@ -228,7 +228,7 @@ echo frobnozzle >> $test_user chmod ug+s $test_user chmod u+x $test_user echo -n "before: "; stat -c '%A' $test_user -su ${qa_user} -c "echo > $test_user" +_su ${qa_user} -c "echo > $test_user" echo -n "after: "; stat -c '%A' $test_user echo "with group exec perm" @@ -237,7 +237,7 @@ chmod ug+s $test_user chmod g+x $test_user chmod u-x $test_user echo -n "before: "; stat -c '%A' $test_user -su ${qa_user} -c "echo > $test_user" +_su ${qa_user} -c "echo > $test_user" echo -n "after: "; stat -c '%A' $test_user echo "with user+group exec perm" @@ -245,7 +245,7 @@ echo frobnozzle >> $test_user chmod ug+s $test_user chmod ug+x $test_user echo -n "before: "; stat -c '%A' $test_user -su ${qa_user} -c "echo > $test_user" +_su ${qa_user} -c "echo > $test_user" echo -n "after: "; stat -c '%A' $test_user # @@ -258,10 +258,10 @@ echo _create_files echo "user: touch qa_user file (should succeed)" -su ${qa_user} -c "touch $test_user" +_su ${qa_user} -c "touch $test_user" echo "user: touch root file (should fail)" -su ${qa_user} -c "touch $test_root" 2>&1 | _filter_files +_su ${qa_user} -c "touch $test_root" 2>&1 | _filter_files _cleanup_files diff --git a/tests/generic/230 b/tests/generic/230 index ba95fbe725ad28..18e20f4b2e9439 100755 --- a/tests/generic/230 +++ b/tests/generic/230 @@ -33,13 +33,13 @@ test_enforcement() echo "--- initiating IO..." >>$seqres.full # Firstly fit below block soft limit echo "Write 225 blocks..." - su $qa_user -c "$XFS_IO_PROG -c 'pwrite 0 $((225 * $BLOCK_SIZE))' -c fsync \ + _su $qa_user -c "$XFS_IO_PROG -c 'pwrite 0 $((225 * $BLOCK_SIZE))' -c fsync \ $SCRATCH_MNT/file1" 2>&1 >>$seqres.full | \ _filter_xfs_io_error | tee -a $seqres.full repquota -$type $SCRATCH_MNT | grep -v "^root" >>$seqres.full 2>&1 # Secondly overcome block soft limit echo "Rewrite 250 blocks plus 1 byte..." - su $qa_user -c "$XFS_IO_PROG -c 'pwrite 0 $((250 * $BLOCK_SIZE + 1))' -c fsync \ + _su $qa_user -c "$XFS_IO_PROG -c 'pwrite 0 $((250 * $BLOCK_SIZE + 1))' -c fsync \ $SCRATCH_MNT/file1" 2>&1 >>$seqres.full | \ _filter_xfs_io_error | tee -a $seqres.full repquota -$type $SCRATCH_MNT | grep -v "^root" >>$seqres.full 2>&1 @@ -49,14 +49,14 @@ test_enforcement() # scratch device that may leed to grace time exceed. setquota -$type $qa_user -T $grace $grace $SCRATCH_MNT 2>/dev/null echo "Write 250 blocks..." - su $qa_user -c "$XFS_IO_PROG -c 'pwrite 0 $((250 * $BLOCK_SIZE))' -c fsync \ + _su $qa_user -c "$XFS_IO_PROG -c 'pwrite 0 $((250 * $BLOCK_SIZE))' -c fsync \ $SCRATCH_MNT/file2" 2>&1 >>$seqres.full | \ _filter_xfs_io_error | tee -a $seqres.full repquota -$type $SCRATCH_MNT | grep -v "^root" >>$seqres.full 2>&1 # Now sleep for grace time and check that softlimit got enforced sleep $((grace+1)) echo "Write 1 block..." - su $qa_user -c "$XFS_IO_PROG -c 'truncate 0' -c 'pwrite 0 $BLOCK_SIZE' \ + _su $qa_user -c "$XFS_IO_PROG -c 'truncate 0' -c 'pwrite 0 $BLOCK_SIZE' \ $SCRATCH_MNT/file2" 2>&1 >>$seqres.full | \ _filter_xfs_io_error | tee -a $seqres.full repquota -$type $SCRATCH_MNT | grep -v "^root" >>$seqres.full 2>&1 @@ -65,7 +65,7 @@ test_enforcement() # space reservations on XFS setquota -$type $qa_user 0 0 3 5 $SCRATCH_MNT echo "Touch 3+4" - su $qa_user -c "touch $SCRATCH_MNT/file3 $SCRATCH_MNT/file4" \ + _su $qa_user -c "touch $SCRATCH_MNT/file3 $SCRATCH_MNT/file4" \ 2>&1 >>$seqres.full | _filter_scratch | tee -a $seqres.full repquota -$type $SCRATCH_MNT | grep -v "^root" >>$seqres.full 2>&1 # Try to exceed inode hardlimit @@ -74,14 +74,14 @@ test_enforcement() # scratch device that may leed to grace time exceed. setquota -$type $qa_user -T $grace $grace $SCRATCH_MNT 2>/dev/null echo "Touch 5+6" - su $qa_user -c "touch $SCRATCH_MNT/file5 $SCRATCH_MNT/file6" \ + _su $qa_user -c "touch $SCRATCH_MNT/file5 $SCRATCH_MNT/file6" \ 2>&1 >>$seqres.full | _filter_scratch | tee -a $seqres.full repquota -$type $SCRATCH_MNT | grep -v "^root" >>$seqres.full 2>&1 # Wait and check grace time enforcement rm -f $SCRATCH_MNT/file5 >>$seqres.full 2>&1 sleep $((grace+1)) echo "Touch 5" - su $qa_user -c "touch $SCRATCH_MNT/file5" 2>&1 >>$seqres.full | + _su $qa_user -c "touch $SCRATCH_MNT/file5" 2>&1 >>$seqres.full | _filter_scratch | tee -a $seqres.full repquota -$type $SCRATCH_MNT | grep -v "^root" >>$seqres.full 2>&1 echo "--- completed IO ($type)" >>$seqres.full diff --git a/tests/generic/231 b/tests/generic/231 index ef3ea45d45d736..8dda926d875e88 100755 --- a/tests/generic/231 +++ b/tests/generic/231 @@ -24,7 +24,7 @@ _fsx() for (( i = 1; i <= $tasks; i++ )); do SEED=$RANDOM echo "ltp/fsx $FSX_ARGS -S $SEED $SCRATCH_MNT/fsx_file$i" >>$seqres.full - su $qa_user -c "ltp/fsx $FSX_ARGS -S $SEED \ + _su $qa_user -c "ltp/fsx $FSX_ARGS -S $SEED \ $FSX_AVOID $SCRATCH_MNT/fsx_file$i" >$tmp.output$i 2>&1 & done diff --git a/tests/generic/233 b/tests/generic/233 index b4c804ff217fbb..df67b39092b7cb 100755 --- a/tests/generic/233 +++ b/tests/generic/233 @@ -43,7 +43,7 @@ _fsstress() # temporarily. ulimit -l unlimited echo "fsstress $args" >> $seqres.full - if ! su $qa_user -c "$FSSTRESS_PROG $args" | tee -a $seqres.full | _filter_num + if ! _su $qa_user -c "$FSSTRESS_PROG $args" | tee -a $seqres.full | _filter_num then echo " fsstress $args returned $?" cat $tmp.out | tee -a $seqres.full diff --git a/tests/generic/270 b/tests/generic/270 index 342ac8b5d3d963..d74971bb535239 100755 --- a/tests/generic/270 +++ b/tests/generic/270 @@ -37,14 +37,14 @@ _workout() # io_uring_queue_init fail on ENOMEM, set max locked memory to unlimited # temporarily. ulimit -l unlimited - su $qa_user -c "$_FSSTRESS_PROG $args" > /dev/null 2>&1 & + _su $qa_user -c "$_FSSTRESS_PROG $args" > /dev/null 2>&1 & _FSSTRESS_PID=$! echo "Run dd writers in parallel" for ((i=0; i < num_iterations; i++)) do # File will be opened with O_TRUNC each time - su $qa_user -c "dd if=/dev/zero \ + _su $qa_user -c "dd if=/dev/zero \ of=$SCRATCH_MNT/SPACE_CONSUMER bs=1M " \ >> $seqres.full 2>&1 sleep $enospc_time diff --git a/tests/generic/314 b/tests/generic/314 index 5fbc6424de3ab4..65f7f9d9619f2e 100755 --- a/tests/generic/314 +++ b/tests/generic/314 @@ -27,7 +27,7 @@ chown $qa_user:12345 $TEST_DIR/$seq-dir chmod 2775 $TEST_DIR/$seq-dir # Make subdir -su $qa_user -c "umask 022; mkdir $TEST_DIR/$seq-dir/subdir" +_su $qa_user -c "umask 022; mkdir $TEST_DIR/$seq-dir/subdir" # Subdir should have inherited sgid _ls_l $TEST_DIR/$seq-dir/ | grep -v total | _filter_test_dir | awk '{print $1,$NF}' diff --git a/tests/generic/327 b/tests/generic/327 index 2323e1e6a12ec8..18cfcd1f655bd7 100755 --- a/tests/generic/327 +++ b/tests/generic/327 @@ -47,7 +47,7 @@ _report_quota_blocks $SCRATCH_MNT echo "Try to reflink again" touch $testdir/file3 chown $qa_user $testdir/file3 -su $qa_user -c "cp --reflink=always -f $testdir/file1 $testdir/file3" 2>&1 | _filter_scratch +_su $qa_user -c "cp --reflink=always -f $testdir/file1 $testdir/file3" 2>&1 | _filter_scratch _report_quota_blocks $SCRATCH_MNT # success, all done diff --git a/tests/generic/328 b/tests/generic/328 index 7071ded259ddb3..fa33bdb78dba12 100755 --- a/tests/generic/328 +++ b/tests/generic/328 @@ -47,12 +47,12 @@ setquota -u $qa_user 0 1024 0 0 $SCRATCH_MNT _report_quota_blocks $SCRATCH_MNT echo "Try to dio write the whole file" -su $qa_user -c '$XFS_IO_PROG -d -c "pwrite 0 '$((sz+65536))'" '$testdir'/file1' 2>&1 >> $seqres.full | \ +_su $qa_user -c '$XFS_IO_PROG -d -c "pwrite 0 '$((sz+65536))'" '$testdir'/file1' 2>&1 >> $seqres.full | \ _filter_xfs_io_error _report_quota_blocks $SCRATCH_MNT echo "Try to write the whole file" -su $qa_user -c '$XFS_IO_PROG -c "pwrite 0 '$((sz+65536))'" '$testdir'/file1' 2>&1 >> $seqres.full | \ +_su $qa_user -c '$XFS_IO_PROG -c "pwrite 0 '$((sz+65536))'" '$testdir'/file1' 2>&1 >> $seqres.full | \ _filter_xfs_io_error _report_quota_blocks $SCRATCH_MNT diff --git a/tests/generic/355 b/tests/generic/355 index b0f8fc45d2b5f4..6b4f7ebae86405 100755 --- a/tests/generic/355 +++ b/tests/generic/355 @@ -22,7 +22,7 @@ rm -f $testfile do_io() { - su $qa_user -c "$XFS_IO_PROG -d -c 'pwrite 0 4k' $testfile" \ + _su $qa_user -c "$XFS_IO_PROG -d -c 'pwrite 0 4k' $testfile" \ >>$seqres.full } diff --git a/tests/generic/453 b/tests/generic/453 index b7e686f37100da..04945ad1085b2d 100755 --- a/tests/generic/453 +++ b/tests/generic/453 @@ -196,7 +196,7 @@ setf "job offer\xef\xb9\x92pdf" "small full stop" setf "job offer\xef\xbc\x8epdf" "fullwidth full stop" setf "job offer\xdc\x81pdf" "syriac supralinear full stop" setf "job offer\xdc\x82pdf" "syriac sublinear full stop" -setf "job offer\xea\x93\xb8pdf" "lisu letter tone mya ti" +setf "job offer\xea\x93\xb8pdf" "li_su letter tone mya ti" setf "job offer.pdf" "actual period" # encoding hidden tag characters in filenames to create confusing names @@ -270,7 +270,7 @@ testf "job offer\xef\xb9\x92pdf" "small full stop" testf "job offer\xef\xbc\x8epdf" "fullwidth full stop" testf "job offer\xdc\x81pdf" "syriac supralinear full stop" testf "job offer\xdc\x82pdf" "syriac sublinear full stop" -testf "job offer\xea\x93\xb8pdf" "lisu letter tone mya ti" +testf "job offer\xea\x93\xb8pdf" "li_su letter tone mya ti" testf "job offer.pdf" "actual period" testf "llamapirate\xf3\xa0\x80\x81\xf3\xa0\x81\x94\xf3\xa0\x81\xa8\xf3\xa0\x81\xa5\xf3\xa0\x80\xa0\xf3\xa0\x81\xb3\xf3\xa0\x81\xa1\xf3\xa0\x81\xac\xf3\xa0\x81\xa5\xf3\xa0\x81\xb3\xf3\xa0\x80\xa0\xf3\xa0\x81\xa6\xf3\xa0\x81\xaf\xf3\xa0\x81\xb2\xf3\xa0\x80\xa0\xf3\xa0\x81\x93\xf3\xa0\x81\xa5\xf3\xa0\x81\xa1\xf3\xa0\x81\xb4\xf3\xa0\x81\xb4\xf3\xa0\x81\xac\xf3\xa0\x81\xa5\xf3\xa0\x80\xa0\xf3\xa0\x81\xb7\xf3\xa0\x81\xa5\xf3\xa0\x81\xb2\xf3\xa0\x81\xa5\xf3\xa0\x80\xa0\xf3\xa0\x81\x95\xf3\xa0\x81\x93\xf3\xa0\x81\x84\xf3\xa0\x80\xa0\xf3\xa0\x80\xb1\xf3\xa0\x80\xb2\xf3\xa0\x80\xb0\xf3\xa0\x80\xb0\xf3\xa0\x80\xb0\xf3\xa0\x80\xb0\xf3\xa0\x81\xbf" @@ -307,7 +307,7 @@ if _check_xfs_scrub_does_unicode "$SCRATCH_MNT" "$SCRATCH_DEV"; then grep -q "job offer.xef.xbc.x8epdf" $tmp.scrub || echo "No complaints about fullwidth full stop?" grep -q "job offer.xdc.x81pdf" $tmp.scrub || echo "No complaints about syriac supralinear full stop?" grep -q "job offer.xdc.x82pdf" $tmp.scrub || echo "No complaints about syriac sublinear full stop?" - grep -q "job offer.xea.x93.xb8pdf" $tmp.scrub || echo "No complaints about lisu letter tone mya ti?" + grep -q "job offer.xea.x93.xb8pdf" $tmp.scrub || echo "No complaints about li_su letter tone mya ti?" grep -q "job offer.*could be confused with" $tmp.scrub || echo "No complaints about confusing job offers?" grep -q "job offer.xe2.x80.xa4.xe2.x80.x8dpdf" $tmp.scrub || echo "No complaints about one dot leader with invisible space?" grep -q "llamapirate" $tmp.scrub || echo "No complaints about hidden llm instructions in filenames?" diff --git a/tests/generic/514 b/tests/generic/514 index 7f3d9c16cb70a4..a2086a255c77c6 100755 --- a/tests/generic/514 +++ b/tests/generic/514 @@ -21,7 +21,7 @@ _scratch_mount chmod a+rwx $SCRATCH_MNT $XFS_IO_PROG -f -c "pwrite -S 0x18 0 1m" $SCRATCH_MNT/foo >>$seqres.full -su -s/bin/bash - $qa_user -c "ulimit -f 64 ; $XFS_IO_PROG -f -c \"reflink $SCRATCH_MNT/foo\" $SCRATCH_MNT/bar" >> $seqres.full 2>&1 +_su -s/bin/bash - $qa_user -c "ulimit -f 64 ; $XFS_IO_PROG -f -c \"reflink $SCRATCH_MNT/foo\" $SCRATCH_MNT/bar" >> $seqres.full 2>&1 sz="$(_get_filesize $SCRATCH_MNT/bar)" if [ "$sz" -ne 0 ] && [ "$sz" -ne 65536 ]; then diff --git a/tests/generic/573 b/tests/generic/573 index b310fccbddda56..d3f3296cb6bafa 100755 --- a/tests/generic/573 +++ b/tests/generic/573 @@ -56,7 +56,7 @@ $CHATTR_PROG -i $fsv_file _fsv_scratch_begin_subtest "FS_IOC_MEASURE_VERITY doesn't require root" _fsv_create_enable_file $fsv_file >> $seqres.full chmod 444 $fsv_file -su $qa_user -c "$FSVERITY_PROG measure $fsv_file" >> $seqres.full +_su $qa_user -c "$FSVERITY_PROG measure $fsv_file" >> $seqres.full # success, all done status=0 diff --git a/tests/generic/600 b/tests/generic/600 index 43f75376a10efc..31c832251ebb6f 100755 --- a/tests/generic/600 +++ b/tests/generic/600 @@ -33,7 +33,7 @@ setquota -t -u 0 1 $SCRATCH_MNT # Soft inode limit 1, hard limit 5 setquota -u $qa_user 0 0 1 5 $SCRATCH_MNT # Run qa user over soft limit and go over grace period -su $qa_user -c "touch $SCRATCH_MNT/file1 $SCRATCH_MNT/file2" +_su $qa_user -c "touch $SCRATCH_MNT/file1 $SCRATCH_MNT/file2" sleep 3 # Extend grace to now + 100s now=`date +%s` diff --git a/tests/generic/601 b/tests/generic/601 index 78b6a4aaa13748..320ac0c758b766 100755 --- a/tests/generic/601 +++ b/tests/generic/601 @@ -42,7 +42,7 @@ $XFS_QUOTA_PROG -x -c "timer -u -i -d 1" $SCRATCH_MNT # Soft inode limit 1, hard limit 5 $XFS_QUOTA_PROG -x -c "limit -u isoft=1 ihard=5 $qa_user" $SCRATCH_MNT # Run qa user over soft limit and go over grace period -su $qa_user -c "touch $SCRATCH_MNT/file1 $SCRATCH_MNT/file2" +_su $qa_user -c "touch $SCRATCH_MNT/file1 $SCRATCH_MNT/file2" sleep 3 # Extend grace to now + 100s now=`date +%s` diff --git a/tests/generic/603 b/tests/generic/603 index 2a75cf9e022750..b199f801a8f03f 100755 --- a/tests/generic/603 +++ b/tests/generic/603 @@ -66,13 +66,13 @@ test_grace() echo "--- Test block quota ---" # Firstly fit below block soft limit echo "Write 225 blocks..." - su $qa_user -c "$XFS_IO_PROG -c 'pwrite 0 $((225 * $BLOCK_SIZE))' \ + _su $qa_user -c "$XFS_IO_PROG -c 'pwrite 0 $((225 * $BLOCK_SIZE))' \ -c fsync $dir/file1" 2>&1 >>$seqres.full | \ _filter_xfs_io_error | tee -a $seqres.full repquota -v -$type $SCRATCH_MNT | grep -v "^root" >>$seqres.full 2>&1 # Secondly overcome block soft limit echo "Rewrite 250 blocks plus 1 byte, over the block softlimit..." - su $qa_user -c "$XFS_IO_PROG -c 'pwrite 0 $((250 * $BLOCK_SIZE + 1))' \ + _su $qa_user -c "$XFS_IO_PROG -c 'pwrite 0 $((250 * $BLOCK_SIZE + 1))' \ -c fsync $dir/file1" 2>&1 >>$seqres.full | \ _filter_xfs_io_error | tee -a $seqres.full repquota -v -$type $SCRATCH_MNT | grep -v "^root" >>$seqres.full 2>&1 @@ -81,7 +81,7 @@ test_grace() # Now sleep enough grace time and check that softlimit got enforced sleep $((bgrace + 1)) echo "Try to write 1 one more block after grace..." - su $qa_user -c "$XFS_IO_PROG -c 'truncate 0' -c 'pwrite 0 $BLOCK_SIZE' \ + _su $qa_user -c "$XFS_IO_PROG -c 'truncate 0' -c 'pwrite 0 $BLOCK_SIZE' \ $dir/file2" 2>&1 >>$seqres.full | _filter_xfs_io_error | \ filter_enospc_edquot $type | tee -a $seqres.full repquota -v -$type $SCRATCH_MNT | grep -v "^root" >>$seqres.full 2>&1 @@ -91,7 +91,7 @@ test_grace() # space reservations on XFS setquota -$type $qa_user 0 0 3 100 $SCRATCH_MNT echo "Create 2 more files, over the inode softlimit..." - su $qa_user -c "touch $dir/file3 $dir/file4" 2>&1 >>$seqres.full | \ + _su $qa_user -c "touch $dir/file3 $dir/file4" 2>&1 >>$seqres.full | \ _filter_scratch | tee -a $seqres.full repquota -v -$type $SCRATCH_MNT | grep -v "^root" >>$seqres.full 2>&1 # Reset grace time here, make below grace time test more accurate @@ -99,7 +99,7 @@ test_grace() # Wait and check grace time enforcement sleep $((igrace+1)) echo "Try to create one more inode after grace..." - su $qa_user -c "touch $dir/file5" 2>&1 >>$seqres.full | \ + _su $qa_user -c "touch $dir/file5" 2>&1 >>$seqres.full | \ filter_enospc_edquot $type | _filter_scratch | \ tee -a $seqres.full repquota -v -$type $SCRATCH_MNT | grep -v "^root" >>$seqres.full 2>&1 diff --git a/tests/generic/673 b/tests/generic/673 index 8f6def9c78881a..6c54ade81f0cec 100755 --- a/tests/generic/673 +++ b/tests/generic/673 @@ -39,7 +39,7 @@ commit_and_check() { local cmd="$XFS_IO_PROG -c 'reflink $SCRATCH_MNT/b 0 0 1m' $SCRATCH_MNT/a" if [ -n "$user" ]; then - su - "$user" -c "$cmd" >> $seqres.full + _su - "$user" -c "$cmd" >> $seqres.full else $SHELL -c "$cmd" >> $seqres.full fi diff --git a/tests/generic/674 b/tests/generic/674 index 1b711f27f39ed1..41fbdeb7d9eb17 100755 --- a/tests/generic/674 +++ b/tests/generic/674 @@ -42,7 +42,7 @@ commit_and_check() { local cmd="$XFS_IO_PROG -c 'dedupe $SCRATCH_MNT/b 0 0 1m' $SCRATCH_MNT/a" if [ -n "$user" ]; then - su - "$user" -c "$cmd" >> $seqres.full + _su - "$user" -c "$cmd" >> $seqres.full else $SHELL -c "$cmd" >> $seqres.full fi diff --git a/tests/generic/675 b/tests/generic/675 index e66de84b546a25..87dfbdfe278dd2 100755 --- a/tests/generic/675 +++ b/tests/generic/675 @@ -44,7 +44,7 @@ commit_and_check() { local cmd="$XFS_IO_PROG -c 'reflink $SCRATCH_MNT/b 0 0 1m' $SCRATCH_MNT/a" if [ -n "$user" ]; then - su - "$user" -c "$cmd" >> $seqres.full + _su - "$user" -c "$cmd" >> $seqres.full else $SHELL -c "$cmd" >> $seqres.full fi diff --git a/tests/generic/680 b/tests/generic/680 index 07048db5cc39a9..1a418fa3b61b0b 100755 --- a/tests/generic/680 +++ b/tests/generic/680 @@ -38,7 +38,7 @@ chmod 0644 $localfile cp $here/src/splice2pipe $tmp.splice2pipe # Test unprivileged user's privilege escalation echo "Test unprivileged user:" -su ${qa_user} -c "$tmp.splice2pipe $localfile 1 AAAAAAAABBBBBBBB" +_su ${qa_user} -c "$tmp.splice2pipe $localfile 1 AAAAAAAABBBBBBBB" _hexdump $localfile # success, all done diff --git a/tests/generic/681 b/tests/generic/681 index aef54205d26f3a..dc4252013fc058 100755 --- a/tests/generic/681 +++ b/tests/generic/681 @@ -55,7 +55,7 @@ ls -sld $scratchdir >> $seqres.full echo "fail quota" >> $seqres.full for ((i = 0; i < dirents; i++)); do name=$(printf "y%0254d" $i) - su - "$qa_user" -c "ln $scratchfile $scratchdir/$name" 2>&1 | \ + _su - "$qa_user" -c "ln $scratchfile $scratchdir/$name" 2>&1 | \ _filter_scratch | sed -e 's/y[0-9]*/yXXX/g' test "${PIPESTATUS[0]}" -ne 0 && break done diff --git a/tests/generic/682 b/tests/generic/682 index 3572af173cbe63..6914a549dc0975 100755 --- a/tests/generic/682 +++ b/tests/generic/682 @@ -65,7 +65,7 @@ echo "fail quota" >> $seqres.full for ((i = 0; i < dirents; i++)); do name=$(printf "y%0254d" $i) ln $scratchfile $stagedir/$name - su - "$qa_user" -c "mv $stagedir/$name $scratchdir/$name" 2>&1 | \ + _su - "$qa_user" -c "mv $stagedir/$name $scratchdir/$name" 2>&1 | \ _filter_scratch | _filter_mv_output test "${PIPESTATUS[0]}" -ne 0 && break done diff --git a/tests/generic/683 b/tests/generic/683 index cc9a9786bde4bf..883905da5f9082 100755 --- a/tests/generic/683 +++ b/tests/generic/683 @@ -49,7 +49,7 @@ commit_and_check() { local cmd="$XFS_IO_PROG -c '$command $start $end' $junk_file" if [ -n "$user" ]; then - su - "$user" -c "$cmd" >> $seqres.full + _su - "$user" -c "$cmd" >> $seqres.full else $SHELL -c "$cmd" >> $seqres.full fi diff --git a/tests/generic/684 b/tests/generic/684 index 2ca036fe518050..9cdfe4ab4f4463 100755 --- a/tests/generic/684 +++ b/tests/generic/684 @@ -49,7 +49,7 @@ commit_and_check() { local cmd="$XFS_IO_PROG -c '$command $start $end' $junk_file" if [ -n "$user" ]; then - su - "$user" -c "$cmd" >> $seqres.full + _su - "$user" -c "$cmd" >> $seqres.full else $SHELL -c "$cmd" >> $seqres.full fi diff --git a/tests/generic/685 b/tests/generic/685 index de07a798a68594..5567255032e1c7 100755 --- a/tests/generic/685 +++ b/tests/generic/685 @@ -49,7 +49,7 @@ commit_and_check() { local cmd="$XFS_IO_PROG -c '$command $start $end' $junk_file" if [ -n "$user" ]; then - su - "$user" -c "$cmd" >> $seqres.full + _su - "$user" -c "$cmd" >> $seqres.full else $SHELL -c "$cmd" >> $seqres.full fi diff --git a/tests/generic/686 b/tests/generic/686 index fc6761fe61a91e..a3fa8e060aeab6 100755 --- a/tests/generic/686 +++ b/tests/generic/686 @@ -49,7 +49,7 @@ commit_and_check() { local cmd="$XFS_IO_PROG -c '$command $start $end' $junk_file" if [ -n "$user" ]; then - su - "$user" -c "$cmd" >> $seqres.full + _su - "$user" -c "$cmd" >> $seqres.full else $SHELL -c "$cmd" >> $seqres.full fi diff --git a/tests/generic/687 b/tests/generic/687 index 82dce88b85ef6d..0c4b09d29fe5e6 100755 --- a/tests/generic/687 +++ b/tests/generic/687 @@ -49,7 +49,7 @@ commit_and_check() { local cmd="$XFS_IO_PROG -c '$command $start $end' $junk_file" if [ -n "$user" ]; then - su - "$user" -c "$cmd" >> $seqres.full + _su - "$user" -c "$cmd" >> $seqres.full else $SHELL -c "$cmd" >> $seqres.full fi diff --git a/tests/generic/688 b/tests/generic/688 index e491d5cf30af23..77db29461415b3 100755 --- a/tests/generic/688 +++ b/tests/generic/688 @@ -51,7 +51,7 @@ commit_and_check() { local cmd="$XFS_IO_PROG -c 'falloc 0 64k' $junk_file" if [ -n "$user" ]; then - su - "$user" -c "$cmd" >> $seqres.full + _su - "$user" -c "$cmd" >> $seqres.full else $SHELL -c "$cmd" >> $seqres.full fi diff --git a/tests/generic/691 b/tests/generic/691 index f33d6edf18a1d1..30ae4a1e384a05 100755 --- a/tests/generic/691 +++ b/tests/generic/691 @@ -75,11 +75,11 @@ exercise() setquota -${type} -t 86400 86400 $SCRATCH_MNT repquota -v -${type} $SCRATCH_MNT | grep -v "^root" >>$seqres.full 2>&1 # Exceed the soft quota limit a bit at first - su $qa_user -c "$XFS_IO_PROG -f -t -c 'pwrite 0 2m' -c fsync ${file}.0" >>$seqres.full + _su $qa_user -c "$XFS_IO_PROG -f -t -c 'pwrite 0 2m' -c fsync ${file}.0" >>$seqres.full # Write more data more times under soft quota limit exhausted condition, # but not reach hard limit. To make sure each write won't trigger EDQUOT. for ((i=1; i<=100; i++));do - su "$qa_user" -c "$XFS_IO_PROG -f -c 'pwrite 0 1m' -c fsync ${file}.$i" >>$seqres.full + _su "$qa_user" -c "$XFS_IO_PROG -f -c 'pwrite 0 1m' -c fsync ${file}.$i" >>$seqres.full if [ $? -ne 0 ];then echo "Unexpected error (type=$type)!" break @@ -89,9 +89,9 @@ exercise() # As we've tested soft limit, now exceed the hard limit and give it a # test in passing. - su $qa_user -c "$XFS_IO_PROG -f -t -c 'pwrite 0 100m' -c fsync ${file}.hard.0" 2>&1 >/dev/null | filter_quota $type + _su $qa_user -c "$XFS_IO_PROG -f -t -c 'pwrite 0 100m' -c fsync ${file}.hard.0" 2>&1 >/dev/null | filter_quota $type for ((i=1; i<=10; i++));do - su "$qa_user" -c "$XFS_IO_PROG -f -c 'pwrite 0 1m' -c fsync ${file}.hard.$i" 2>&1 | filter_quota $type + _su "$qa_user" -c "$XFS_IO_PROG -f -c 'pwrite 0 1m' -c fsync ${file}.hard.$i" 2>&1 | filter_quota $type done } diff --git a/tests/generic/721 b/tests/generic/721 index a9565f18917831..75d5063c2d1701 100755 --- a/tests/generic/721 +++ b/tests/generic/721 @@ -52,7 +52,7 @@ cmd="$XFS_IO_PROG \ -c 'pwrite -S 0x60 44k 55k -b 1m' \ -c 'commitupdate -q' \ \"$dir/a\"" -su -s /bin/bash -c "$cmd" $qa_user 2>&1 | _filter_xfs_io | _filter_test_dir +_su -s /bin/bash -c "$cmd" $qa_user 2>&1 | _filter_xfs_io | _filter_test_dir filesnap "after commit" $dir/a echo @@ -67,7 +67,7 @@ cmd="$XFS_IO_PROG \ -c 'pwrite -S 0x60 0 55k' \ -c 'commitupdate -q' \ \"$dir/a\"" -su -s /bin/bash -c "$cmd" $qa_user 2>&1 | _filter_xfs_io | _filter_test_dir +_su -s /bin/bash -c "$cmd" $qa_user 2>&1 | _filter_xfs_io | _filter_test_dir filesnap "after shorten commit" $dir/a echo @@ -81,7 +81,7 @@ cmd="$XFS_IO_PROG \ -c \"pwrite -S 0x60 0 $(( (blksz * nrblks) + 37373 ))\" \ -c 'commitupdate -q' \ \"$dir/a\"" -su -s /bin/bash -c "$cmd" $qa_user 2>&1 | _filter_xfs_io | _filter_test_dir +_su -s /bin/bash -c "$cmd" $qa_user 2>&1 | _filter_xfs_io | _filter_test_dir filesnap "after lengthen commit" $dir/a echo @@ -96,7 +96,7 @@ cmd="$XFS_IO_PROG \ -c 'pwrite -S 0x60 44k 55k -b 1m' \ -c 'cancelupdate' \ \"$dir/a\"" -su -s /bin/bash -c "$cmd" $qa_user 2>&1 | _filter_xfs_io | _filter_test_dir +_su -s /bin/bash -c "$cmd" $qa_user 2>&1 | _filter_xfs_io | _filter_test_dir filesnap "after cancel" $dir/a echo @@ -115,7 +115,7 @@ cmd="$XFS_IO_PROG \ -c 'pwrite -S 0x61 22k 11k -b 1m' \ -c 'commitupdate -q' \ \"$dir/a\"" -su -s /bin/bash -c "$cmd" $qa_user 2>&1 | _filter_xfs_io | _filter_test_dir +_su -s /bin/bash -c "$cmd" $qa_user 2>&1 | _filter_xfs_io | _filter_test_dir filesnap "after fail commit" $dir/a echo diff --git a/tests/generic/726 b/tests/generic/726 index 131ac5b503e1a4..d2a2a2cebe522e 100755 --- a/tests/generic/726 +++ b/tests/generic/726 @@ -46,7 +46,7 @@ commit_and_check() { local cmd="$XFS_IO_PROG -c 'startupdate' -c 'pwrite -S 0x57 0 1m' -c 'commitupdate' $SCRATCH_MNT/a" if [ -n "$user" ]; then - su - "$user" -c "$cmd" >> $seqres.full + _su - "$user" -c "$cmd" >> $seqres.full else $SHELL -c "$cmd" >> $seqres.full fi diff --git a/tests/generic/727 b/tests/generic/727 index ee7ed9760a165a..9551e47cb13b06 100755 --- a/tests/generic/727 +++ b/tests/generic/727 @@ -54,7 +54,7 @@ commit_and_check() { local cmd="$XFS_IO_PROG -c 'startupdate' -c 'pwrite -S 0x57 0 1m' -c 'commitupdate' $SCRATCH_MNT/a" if [ -n "$user" ]; then - su - "$user" -c "$cmd" >> $seqres.full + _su - "$user" -c "$cmd" >> $seqres.full else $SHELL -c "$cmd" >> $seqres.full fi diff --git a/tests/xfs/720 b/tests/xfs/720 index f928cc43d3bc54..68a6c7f6e2d584 100755 --- a/tests/xfs/720 +++ b/tests/xfs/720 @@ -61,7 +61,7 @@ $XFS_QUOTA_PROG -x -c 'report -u' $SCRATCH_MNT >> $seqres.full # Fail at appending the file as qa_user to ensure quota enforcement works echo "fail quota" >> $seqres.full -su - "$qa_user" -c "$XFS_IO_PROG -c 'pwrite 10g 1' $scratchfile" >> $seqres.full +_su - "$qa_user" -c "$XFS_IO_PROG -c 'pwrite 10g 1' $scratchfile" >> $seqres.full $XFS_QUOTA_PROG -x -c 'report -u' $SCRATCH_MNT >> $seqres.full # success, all done diff --git a/tests/xfs/795 b/tests/xfs/795 index 5a67f02ec92eca..217f96092a4c42 100755 --- a/tests/xfs/795 +++ b/tests/xfs/795 @@ -63,7 +63,7 @@ $XFS_QUOTA_PROG -x -c 'report -u' $SCRATCH_MNT >> $seqres.full echo "fail quota" >> $seqres.full for ((i = 0; i < dirents; i++)); do name=$(printf "y%0254d" $i) - su - "$qa_user" -c "ln $scratchfile $scratchdir/$name" 2>&1 | \ + _su - "$qa_user" -c "ln $scratchfile $scratchdir/$name" 2>&1 | \ _filter_scratch | sed -e 's/y[0-9]*/yXXX/g' test "${PIPESTATUS[0]}" -ne 0 && break done diff --git a/tests/xfs/803 b/tests/xfs/803 index e3277181b41af0..4e5a58c4a3328e 100755 --- a/tests/xfs/803 +++ b/tests/xfs/803 @@ -88,7 +88,7 @@ echo set too long value $XFS_IO_PROG -c "setfsprops $propname=$longpropval" $TEST_DIR echo not enough permissions -su - "$qa_user" -c "$XFS_IO_PROG -c \"setfsprops $propname=$propval\" $TEST_DIR" 2>&1 | _filter_test_dir +_su - "$qa_user" -c "$XFS_IO_PROG -c \"setfsprops $propname=$propval\" $TEST_DIR" 2>&1 | _filter_test_dir echo "*** DB TEST ***" From patchwork Thu Jan 16 23:27:15 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13942524 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 39A73232438; Thu, 16 Jan 2025 23:27:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737070036; cv=none; b=OPV7KBmo0glB1arbWxk3CMZWSuSq9X8jBZO8qhh6rR3OSdrvKVDD6+Hzy8gzTFisZc/Ox3ialdOF2ZL/jcfIx1fsbZt5edxFYp5C3ROwWFTbuVgBZ/uGqlCiXEeIy0lP6IcUbqzQKe4/rucCXQsS6evI11lhMuACZZvTB3I2q9k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737070036; c=relaxed/simple; bh=oMzGVeg4ZdgtOWYLq0T6ROEUmRySL652sqQCJMoVQ1M=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=qaXrDiAfvS+o05b73k3e6GDkTNwtoDanDZyn1AAxRVBLZ4cIlV32iBIvWEhZdUcJ3R28XWUE689xGmFMh3FyLvValZoTbsi/gJi3998ximqdVMpIk3j6zTRAmVw9GDuVHr8traCXVH1hF+rPATlcbkh7UG1kJV6TD0KH6BCqguw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=iwT1tyHX; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="iwT1tyHX" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1086BC4CED6; Thu, 16 Jan 2025 23:27:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1737070036; bh=oMzGVeg4ZdgtOWYLq0T6ROEUmRySL652sqQCJMoVQ1M=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=iwT1tyHXkxOSbJLJObSW6aNUg0PX1frwi8hAIalRvZyvzqN5tH5ur0vGvjU6KeA8x ddNhvgJtpYyYluQNv2H79THAauonYtKZTav4kWADA9KKbNOrfIKxW/LsZtDHe6WMiG I5/38GDFszh+hNWZMNXlBGbNA4ZtzkBhzCiB+tWLqxgsZFTPxqPIaPa6vMgrrQz3QK qYdjslYl3dSf9mLPqYBWjM5ewk8jf6nvdRkr6cB2PqAOqrK74aCDCXbZe96PmWS0sM Wevx8k0NVUbM9rappgL+VQwOnw9CAwCOT45bYXSamCVSRR7XqPtzZI6NH5no2jUqGh e8q6Y7L7uBYYQ== Date: Thu, 16 Jan 2025 15:27:15 -0800 Subject: [PATCH 08/23] common: fix pkill by running test program in a separate session From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: hch@lst.de, fstests@vger.kernel.org, linux-xfs@vger.kernel.org Message-ID: <173706974197.1927324.9208284704325894988.stgit@frogsfrogsfrogs> In-Reply-To: <173706974044.1927324.7824600141282028094.stgit@frogsfrogsfrogs> References: <173706974044.1927324.7824600141282028094.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Run each test program with a separate session id so that we can tell pkill to kill all processes of a given name, but only within our own session id. This /should/ suffice to run multiple fstests on the same machine without one instance shooting down processes of another instance. This fixes a general problem with using "pkill --parent" -- if the process being targeted is not a direct descendant of the bash script calling pkill, then pkill will not do anything. The scrub stress tests make use of multiple background subshells, which is how a ^C in the parent process fails to result in fsx/fsstress being killed. This is necessary to fix SOAK_DURATION runtime constraints for all the scrub stress tests. However, there is a cost -- the test program no longer runs with the same controlling tty as ./check, which means that ^Z doesn't work and SIGINT/SIGQUIT are set to SIG_IGN. IOWs, if a test wants to kill its subprocesses, it must use another signal such as SIGPIPE. Fortunately, bash doesn't whine about children dying due to fatal signals if the children run in a different session id. I also explored alternate designs, and this was the least unsatisfying: a) Setting the process group didn't work because background subshells are assigned a new group id. b) Constraining the pkill/pgrep search to a cgroup could work, but we'd have to set up a cgroup in which to run the fstest. c) Putting test subprocesses in a systemd sub-scope and telling systemd to kill the sub-scope could work because ./check can already use it to ensure that all child processes of a test are killed. However, this is an *optional* feature, which means that we'd have to require systemd. d) Constraining the pkill/pgrep search to a particular mount namespace could work, but we already have tests that set up their own mount namespaces, which means the constrained pgrep will not find all child processes of a test. e) Constraining to any other type of namespace (uts, pid, etc) might not work because those namespaces might not be enabled. f) Revert check-parallel and go back to one fstests instance per system. Zorro already chose not to revert. So. Change _run_seq to create a the ./$seq process with a new session id, update _su calls to use the same session as the parent test, update all the pkill sites to use a wrapper so that we only target processes created by *this* instance of fstests, and update SIGINT to SIGPIPE. Cc: # v2024.12.08 Fixes: 8973af00ec212f ("fstests: cleanup fsstress process management") Signed-off-by: "Darrick J. Wong" --- check | 33 ++++++++++++++++++++++++++++----- common/fuzzy | 17 ++++++++--------- common/rc | 12 ++++++++++-- tests/generic/310 | 6 +++--- tests/generic/561 | 2 +- 5 files changed, 50 insertions(+), 20 deletions(-) diff --git a/check b/check index 607d2456e6a1fe..bafe48bf12db67 100755 --- a/check +++ b/check @@ -698,18 +698,41 @@ _adjust_oom_score -500 # systemd doesn't automatically remove transient scopes that fail to terminate # when systemd tells them to terminate (e.g. programs stuck in D state when # systemd sends SIGKILL), so we use reset-failed to tear down the scope. +# +# Use setsid to run the test program with a separate session id so that we +# can pkill only the processes started by this test. _run_seq() { - local cmd=(bash -c "test -w ${OOM_SCORE_ADJ} && echo 250 > ${OOM_SCORE_ADJ}; exec ./$seq") + local cmd=(bash -c "test -w ${OOM_SCORE_ADJ} && echo 250 > ${OOM_SCORE_ADJ}; exec setsid bash ./$seq") if [ -n "${HAVE_SYSTEMD_SCOPES}" ]; then local unit="$(systemd-escape "fs$seq").scope" systemctl reset-failed "${unit}" &> /dev/null - systemd-run --quiet --unit "${unit}" --scope "${cmd[@]}" + systemd-run --quiet --unit "${unit}" --scope "${cmd[@]}" & + CHILDPID=$! + wait res=$? + unset CHILDPID systemctl stop "${unit}" &> /dev/null return "${res}" else - "${cmd[@]}" + # bash won't run the SIGINT trap handler while there are + # foreground children in a separate session, so we must run + # the test in the background and wait for it. + "${cmd[@]}" & + CHILDPID=$! + wait + unset CHILDPID + fi +} + +_kill_seq() { + if [ -n "$CHILDPID" ]; then + # SIGPIPE will kill all the children (including fsstress) + # without bash logging fatal signal termination messages to the + # console + pkill -PIPE --session "$CHILDPID" + wait + unset CHILDPID fi } @@ -718,9 +741,9 @@ _prepare_test_list fstests_start_time="$(date +"%F %T")" if $OPTIONS_HAVE_SECTIONS; then - trap "_summary; exit \$status" 0 1 2 3 15 + trap "_kill_seq; _summary; exit \$status" 0 1 2 3 15 else - trap "_wrapup; exit \$status" 0 1 2 3 15 + trap "_kill_seq; _wrapup; exit \$status" 0 1 2 3 15 fi function run_section() diff --git a/common/fuzzy b/common/fuzzy index 0a2d91542b561e..772ce7ddcff6d8 100644 --- a/common/fuzzy +++ b/common/fuzzy @@ -891,7 +891,7 @@ __stress_xfs_scrub_loop() { local runningfile="$2" local scrub_startat="$3" shift; shift; shift - local sigint_ret="$(( $(kill -l SIGINT) + 128 ))" + local signal_ret="$(( $(kill -l SIGPIPE) + 128 ))" local scrublog="$tmp.scrub" while __stress_scrub_running "$scrub_startat" "$runningfile"; do @@ -901,8 +901,8 @@ __stress_xfs_scrub_loop() { while __stress_scrub_running "$end" "$runningfile"; do _scratch_scrub "$@" &> $scrublog res=$? - if [ "$res" -eq "$sigint_ret" ]; then - # Ignore SIGINT because the cleanup function sends + if [ "$res" -eq "$signal_ret" ]; then + # Ignore SIGPIPE because the cleanup function sends # that to terminate xfs_scrub res=0 fi @@ -1173,13 +1173,11 @@ _scratch_xfs_stress_scrub_cleanup() { rm -f "$runningfile" echo "Cleaning up scrub stress run at $(date)" >> $seqres.full - # Send SIGINT so that bash won't print a 'Terminated' message that - # distorts the golden output. echo "Killing stressor processes at $(date)" >> $seqres.full - _kill_fsstress - pkill -INT --parent $$ xfs_io >> $seqres.full 2>&1 - pkill -INT --parent $$ fsx >> $seqres.full 2>&1 - pkill -INT --parent $$ xfs_scrub >> $seqres.full 2>&1 + _pkill --echo -PIPE fsstress >> $seqres.full 2>&1 + _pkill --echo -PIPE xfs_io >> $seqres.full 2>&1 + _pkill --echo -PIPE fsx >> $seqres.full 2>&1 + _pkill --echo -PIPE xfs_scrub >> $seqres.full 2>&1 # Tests are not allowed to exit with the scratch fs frozen. If we # started a fs freeze/thaw background loop, wait for that loop to exit @@ -1209,6 +1207,7 @@ _scratch_xfs_stress_scrub_cleanup() { # Wait for the remaining children to exit. echo "Waiting for children to exit at $(date)" >> $seqres.full wait + echo "Children exited as of $(date)" >> $seqres.full # Ensure the scratch fs is also writable before we exit. if [ -n "$__SCRUB_STRESS_REMOUNT_LOOP" ]; then diff --git a/common/rc b/common/rc index 459be11c11c405..d143ba36265c6c 100644 --- a/common/rc +++ b/common/rc @@ -30,6 +30,12 @@ _test_sync() _sync_fs $TEST_DIR } +# Kill only the test processes started by this test +_pkill() +{ + pkill --session 0 "$@" +} + # Common execution handling for fsstress invocation. # # We need per-test fsstress binaries because of the way fsstress forks and @@ -69,7 +75,7 @@ _kill_fsstress() if [ -n "$_FSSTRESS_PID" ]; then # use SIGPIPE to avoid "Killed" messages from bash echo "killing $_FSSTRESS_BIN" >> $seqres.full - pkill -PIPE $_FSSTRESS_BIN >> $seqres.full 2>&1 + _pkill -PIPE $_FSSTRESS_BIN >> $seqres.full 2>&1 _wait_for_fsstress return $? fi @@ -2740,9 +2746,11 @@ _require_user_exists() [ "$?" == "0" ] || _notrun "$user user not defined." } +# Run all non-root processes in the same session as the root. Believe it or +# not, passing $SHELL in this manner works both for "su" and "su -c cmd". _su() { - su "$@" + su --session-command $SHELL "$@" } # check if a user exists and is able to execute commands. diff --git a/tests/generic/310 b/tests/generic/310 index 52babfdc803a21..570cc5f3859548 100755 --- a/tests/generic/310 +++ b/tests/generic/310 @@ -29,7 +29,7 @@ _begin_fstest auto # Override the default cleanup function. _cleanup() { - pkill -9 $seq.t_readdir > /dev/null 2>&1 + _pkill -9 $seq.t_readdir > /dev/null 2>&1 wait rm -rf $TEST_DIR/tmp rm -f $tmp.* @@ -83,7 +83,7 @@ _test_read() { $TEST_DIR/$seq.t_readdir_1 $SEQ_DIR > /dev/null 2>&1 & sleep $RUN_TIME - pkill -PIPE $seq.t_readdir_1 + _pkill -PIPE $seq.t_readdir_1 wait check_kernel_bug @@ -97,7 +97,7 @@ _test_lseek() $TEST_DIR/$seq.t_readdir_2 $SEQ_DIR > /dev/null 2>&1 & readdir_pid=$! sleep $RUN_TIME - pkill -PIPE $seq.t_readdir_2 + _pkill -PIPE $seq.t_readdir_2 wait check_kernel_bug diff --git a/tests/generic/561 b/tests/generic/561 index afe727ac56cbd9..b260aaf16c9256 100755 --- a/tests/generic/561 +++ b/tests/generic/561 @@ -40,7 +40,7 @@ function end_test() # stop duperemove running if [ -e $dupe_run ]; then rm -f $dupe_run - pkill $dedup_bin >/dev/null 2>&1 + _pkill $dedup_bin >/dev/null 2>&1 wait $dedup_pids rm -f $dedup_prog fi From patchwork Thu Jan 16 23:27:31 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13942525 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D05B2158520; Thu, 16 Jan 2025 23:27:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737070051; cv=none; b=Gr+SAEP9BLsyHEW79VQ73Z12FAcbrizEqkXPLinI2zdWqYpFyhljyEhatIl0ihvEK2Jxia6yqeQCYCKsco1qADUjFZGdHVoF4A95omwAKRfkjpwuds3wn6bGtwLEJVo5n9mrjStdH7cs978EeZEBk/iRgN/r7FipOJywnx5WOTU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737070051; c=relaxed/simple; bh=4ojFym6rL0NvIG7GN6s36EqDy1wFfRBh8l1I+PrhDRg=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=rMPw0Wqkwo6Ue92iiA97BKjEMKV7fcWLbybKVMzLlzlpt5MPI8dw2tTLpRngo+lPERbioyUMSYW+VOtGedJCC5Rf4A98yjdQU0+Pvr5r8YENOxwgz763W27RhJThIDZ8eDCf0uW/8ZfmzXgFSm6n8ca+maAKqCwAE27IG65UqhI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=MMGnBj+I; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="MMGnBj+I" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A65F3C4CED6; Thu, 16 Jan 2025 23:27:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1737070051; bh=4ojFym6rL0NvIG7GN6s36EqDy1wFfRBh8l1I+PrhDRg=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=MMGnBj+IHvzBPJl6LDN0MSI2SQ3mvcnfUofRywf/mtoZx0X+fDtp5rS5yctD6gniG MS8rUDURTo873U/du2+bwaTwIfyvuOfJLZ7xlazIhoni2Y/ViaYmd+13+HiFUg5cgL d7poPV47Ar5Tf7C/y2HHxWXygpJWm5WncNkceefJslQqMWJyC1jHYicn+X7n2W9q0L qWfYUULaoOsOGePGLAzlV/kakHGcryrUXc8xGC0Uo4vPBSIW8Ywh6g9T3qZOi15TVx zB1RHw+oSSExcpqSVBET/lcmzr9a21BIlJc2FWvxuDDVnq/pYPWCW4INXJqPUf5vAG LcblLT1iYD4FQ== Date: Thu, 16 Jan 2025 15:27:31 -0800 Subject: [PATCH 09/23] unmount: resume logging of stdout and stderr for filtering From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: hch@lst.de, fstests@vger.kernel.org, linux-xfs@vger.kernel.org Message-ID: <173706974213.1927324.1385565534988725161.stgit@frogsfrogsfrogs> In-Reply-To: <173706974044.1927324.7824600141282028094.stgit@frogsfrogsfrogs> References: <173706974044.1927324.7824600141282028094.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong There's a number of places where a test program calls a variant of _unmount but then pipes the output through a _filter script or something. The new _unmount helper redirects stdout and stderr to seqres.full, which means that those error messages (some of which are encoded in the golden outputs) are suppressed. This leads to test regressions in generic/050 and other places, so let's resume logging. This also undoes all the changes that removed /dev/null redirection of unmount calls. Cc: # v2024.12.08 Fixes: 4c6bc4565105e6 ("fstests: clean up mount and unmount operations") Signed-off-by: "Darrick J. Wong" Reviewed-by: Dave Chinner --- check | 10 ++++++++-- common/quota | 2 +- common/rc | 21 +++++++++++++++++++-- tests/generic/050 | 2 +- tests/generic/085 | 2 +- tests/generic/361 | 4 ++-- tests/generic/590 | 2 +- tests/generic/746 | 2 +- tests/xfs/149 | 2 +- tests/xfs/530 | 2 +- 10 files changed, 36 insertions(+), 13 deletions(-) diff --git a/check b/check index bafe48bf12db67..126a77441d700d 100755 --- a/check +++ b/check @@ -1026,8 +1026,8 @@ function run_section() if [ $sts -ne 0 ]; then _dump_err_cont "[failed, exit status $sts]" - _test_unmount 2>> $seqres.full - _scratch_unmount 2>> $seqres.full + _test_unmount 2> /dev/null + _scratch_unmount 2> /dev/null rm -f ${RESULT_DIR}/require_test* rm -f ${RESULT_DIR}/require_scratch* # Even though we failed, there may be something interesting in @@ -1113,6 +1113,12 @@ function run_section() _stash_test_status "$seqnum" "$tc_status" done + # Reset these three variables so that unmount output doesn't get + # written to $seqres.full of the last test to run. + seq="check.$$" + check="$RESULT_BASE/check" + seqres="$check" + sect_stop=`_wallclock` interrupt=false _wrapup diff --git a/common/quota b/common/quota index 8688116c6547a9..4dad9b79a27a7f 100644 --- a/common/quota +++ b/common/quota @@ -274,7 +274,7 @@ _choose_prid() _qmount() { - _scratch_unmount + _scratch_unmount >/dev/null 2>&1 _try_scratch_mount || _fail "qmount failed" # xfs doesn't need these setups and quotacheck even fails on xfs # redirect the output to $seqres.full for debug purpose and ignore results diff --git a/common/rc b/common/rc index d143ba36265c6c..9e34c301b0deb0 100644 --- a/common/rc +++ b/common/rc @@ -480,11 +480,28 @@ _scratch_mount_idmapped() } # Unmount the filesystem based on the directory or device passed. +# Log everything that happens to seqres.full, and use BASHPID because +# background subshells have the same $$ as the parent but not the same +# $BASHPID. _unmount() { - local args="$*" + local outlog="$tmp.$BASHPID.umount" + local errlog="$tmp.$BASHPID.umount.err" - $UMOUNT_PROG $args >> $seqres.full 2>&1 + rm -f "$outlog" "$errlog" + $UMOUNT_PROG "$@" 2> "$errlog" > "$outlog" + local res="${PIPESTATUS[0]}" + + if [ -s "$outlog" ]; then + cat "$outlog" >> $seqres.full + cat "$outlog" + fi + if [ -s "$errlog" ]; then + cat "$errlog" >> $seqres.full + >&2 cat "$errlog" + fi + rm -f "$outlog" "$errlog" + return $res } _scratch_unmount() diff --git a/tests/generic/050 b/tests/generic/050 index 8e9456db279003..affb072df5969f 100755 --- a/tests/generic/050 +++ b/tests/generic/050 @@ -89,7 +89,7 @@ _try_scratch_mount 2>&1 | _filter_ro_mount | _filter_scratch # expects an error, so open code the unmount echo "unmounting read-only filesystem" -$UMOUNT_PROG $SCRATCH_DEV 2>&1 | _filter_scratch | _filter_ending_dot +_scratch_unmount 2>&1 | _filter_scratch | _filter_ending_dot # # This is the way out if the underlying device really is read-only. diff --git a/tests/generic/085 b/tests/generic/085 index 7671a36ab9524f..d3fa10be9ccace 100755 --- a/tests/generic/085 +++ b/tests/generic/085 @@ -29,7 +29,7 @@ cleanup_dmdev() fi # in case it's still suspended and/or mounted $DMSETUP_PROG resume $lvdev >> $seqres.full 2>&1 - _unmount -q $SCRATCH_MNT + _unmount -q $SCRATCH_MNT >/dev/null 2>&1 _dmsetup_remove $node } diff --git a/tests/generic/361 b/tests/generic/361 index e2b7984361e87c..b584af47540020 100755 --- a/tests/generic/361 +++ b/tests/generic/361 @@ -16,7 +16,7 @@ _begin_fstest auto quick # Override the default cleanup function. _cleanup() { - _unmount $fs_mnt + _unmount $fs_mnt &>> /dev/null [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev cd / rm -f $tmp.* @@ -54,7 +54,7 @@ $XFS_IO_PROG -fc "pwrite 0 520m" $fs_mnt/testfile >>$seqres.full 2>&1 # remount should not hang $MOUNT_PROG -o remount,ro $fs_mnt >>$seqres.full 2>&1 -_unmount $fs_mnt +_unmount $fs_mnt &>/dev/null _destroy_loop_device $loop_dev unset loop_dev diff --git a/tests/generic/590 b/tests/generic/590 index 1adeef4c2ad52c..ba1337a856f15d 100755 --- a/tests/generic/590 +++ b/tests/generic/590 @@ -15,7 +15,7 @@ _begin_fstest auto prealloc preallocrw # Override the default cleanup function. _cleanup() { - _scratch_unmount + _scratch_unmount &>/dev/null [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev cd / rm -f $tmp.* diff --git a/tests/generic/746 b/tests/generic/746 index ba8ed25e845776..6f02b1cc354782 100755 --- a/tests/generic/746 +++ b/tests/generic/746 @@ -223,7 +223,7 @@ while read line; do done < $fiemap_after echo "done." -_unmount $loop_mnt +_unmount $loop_mnt &>/dev/null _destroy_loop_device $loop_dev unset loop_dev diff --git a/tests/xfs/149 b/tests/xfs/149 index 9a96f82ede1761..28dfc7f04c1773 100755 --- a/tests/xfs/149 +++ b/tests/xfs/149 @@ -22,7 +22,7 @@ loop_symlink=$TEST_DIR/loop_symlink.$$ # Override the default cleanup function. _cleanup() { - _unmount $mntdir + _unmount $mntdir &>/dev/null [ -n "$loop_dev" ] && _destroy_loop_device $loop_dev rmdir $mntdir rm -f $loop_symlink diff --git a/tests/xfs/530 b/tests/xfs/530 index d0d0e2665070f8..95ab32f1e1f828 100755 --- a/tests/xfs/530 +++ b/tests/xfs/530 @@ -116,7 +116,7 @@ done echo "Check filesystem" _check_scratch_fs -_scratch_unmount +_scratch_unmount &> /dev/null _destroy_loop_device $rt_loop_dev unset rt_loop_dev From patchwork Thu Jan 16 23:27:46 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13942526 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A2436158520; Thu, 16 Jan 2025 23:27:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737070067; cv=none; b=krTxJPbEVUdvPnmINUbRfdP4pMcKA6aL45DN6s9CKdIJniuyVSUPJudMCs+jqsj6Fw6hgIwgKhBM0MHsTXoB+BuTgOvRuudUoWm8W8YeDQ5DZxx9YQgVXsxOstsuSnmcK9hv/a4dQ/omWqAevWXQuhTfzs4+zrGPF+tsUrLh4Vw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737070067; c=relaxed/simple; bh=z8/t7g12G3jvoVSPtvzo/jYuh8mMrxA863C0HdXkXmc=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=dzkIwBJw4BrA0kdW9vDR2ccpcBe45M1h2d5FnIJVPVFydrq15qvHMrd3P3CNGJP0gqr0OOFLFgEvDEUWFgg4EYmDfowEZhGit8K2g97zTWEuhFnsG+gCUWd96iV2WMXDMNTN7VI+gXKV8HLARkqF8wiS2iw0jPR9jm/DzdcKiZ4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=OZmwXxU6; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="OZmwXxU6" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 46CC9C4CED6; Thu, 16 Jan 2025 23:27:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1737070067; bh=z8/t7g12G3jvoVSPtvzo/jYuh8mMrxA863C0HdXkXmc=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=OZmwXxU6qXfs5S6EX2IsYVFiZEOZkRWKSs4sVSNIzRq2a8DgWPWX7q/TGUCCErEhF DDZ1eP0sSSj5u1+A62UZI6TiSIFNkEIlzez0F7btvGckbKplrNGSNV5T2USTYL4hwS YA/Lc1Rs9R84FvkLKiMfIHG4cheqVH6uU4vqoLClq3WXRnhPwyqc2FPu+QL/CycnI1 7rrEebV8efdK4+H9xHr/YyGfBg9MF7vK8bq2LUpE2aY3Ljtm5myrCtk4X45cSGe/uy W/ulBDHI/DZpgaSYHPgO0GCTEasub5yLoY1qC8nuAym2D4/jsWt3EJDiz/8uJZEqnJ 6VuVJ78FLEQiQ== Date: Thu, 16 Jan 2025 15:27:46 -0800 Subject: [PATCH 10/23] mkfs: don't hardcode log size From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: hch@lst.de, fstests@vger.kernel.org, linux-xfs@vger.kernel.org Message-ID: <173706974228.1927324.17714311358227511791.stgit@frogsfrogsfrogs> In-Reply-To: <173706974044.1927324.7824600141282028094.stgit@frogsfrogsfrogs> References: <173706974044.1927324.7824600141282028094.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Commit 000813899afb46 hardcoded a log size of 256MB into xfs/501, xfs/502, and generic/530. This seems to be an attempt to reduce test run times by increasing the log size so that more background threads can run in parallel. Unfortunately, this breaks a couple of my test configurations: - External logs smaller than 256MB - Internal logs where the AG size is less than 256MB For example, here's seqres.full from a failed xfs/501 invocation: ** mkfs failed with extra mkfs options added to " -m metadir=2,autofsck=1,uquota,gquota,pquota, -d rtinherit=1," by test 501 ** ** attempting to mkfs using only test 501 options: -l size=256m ** size 256m specified for log subvolume is too large, maximum is 32768 blocks mount -ortdev=/dev/sdb4 -ologdev=/dev/sdb2 /dev/sda4 /opt failed umount: /dev/sda4: not mounted. Note that there's some formatting error here, so we jettison the entire rt configuration to force the log size option, but then mount fails because we didn't edit out the rtdev option there too. Fortunately, mkfs.xfs already /has/ a few options to try to improve parallelism in the filesystem by avoiding contention on the log grant heads by scaling up the log size. These options are aware of log and AG size constraints so they won't conflict with other geometry options. Use them. Cc: # v2024.12.08 Fixes: 000813899afb46 ("fstests: scale some tests for high CPU count sanity") Signed-off-by: "Darrick J. Wong" --- common/rc | 27 +++++++++++++++++++++++++++ tests/generic/530 | 6 +----- tests/generic/531 | 6 +----- tests/xfs/501 | 2 +- tests/xfs/502 | 2 +- 5 files changed, 31 insertions(+), 12 deletions(-) diff --git a/common/rc b/common/rc index 9e34c301b0deb0..885669beeb5e26 100644 --- a/common/rc +++ b/common/rc @@ -689,6 +689,33 @@ _test_cycle_mount() _test_mount } +# Are there mkfs options to try to improve concurrency? +_scratch_mkfs_concurrency_options() +{ + local nr_cpus="$(( $1 * LOAD_FACTOR ))" + + case "$FSTYP" in + xfs) + # If any concurrency options are already specified, don't + # compute our own conflicting ones. + echo "$SCRATCH_OPTIONS $MKFS_OPTIONS" | \ + grep -q 'concurrency=' && + return + + local sections=(d r) + + # -l concurrency does not work with external logs + test _has_logdev || sections+=(l) + + for section in "${sections[@]}"; do + $MKFS_XFS_PROG -$section concurrency=$nr_cpus 2>&1 | \ + grep -q "unknown option -$section" || + echo "-$section concurrency=$nr_cpus " + done + ;; + esac +} + _scratch_mkfs_options() { _scratch_options mkfs diff --git a/tests/generic/530 b/tests/generic/530 index f2513156a920e8..7413840476b588 100755 --- a/tests/generic/530 +++ b/tests/generic/530 @@ -25,11 +25,7 @@ _require_test_program "t_open_tmpfiles" # For XFS, pushing 50000 unlinked inode inactivations through a small xfs log # can result in bottlenecks on the log grant heads, so try to make the log # larger to reduce runtime. -if [ "$FSTYP" = "xfs" ] && ! _has_logdev; then - _scratch_mkfs "-l size=256m" >> $seqres.full 2>&1 -else - _scratch_mkfs >> $seqres.full 2>&1 -fi +_scratch_mkfs $(_scratch_mkfs_concurrency_options 32) >> $seqres.full 2>&1 _scratch_mount # Set ULIMIT_NOFILE to min(file-max / 2, 50000 files per LOAD_FACTOR) diff --git a/tests/generic/531 b/tests/generic/531 index ed6c3f91153ecc..3ba2790c923464 100755 --- a/tests/generic/531 +++ b/tests/generic/531 @@ -23,11 +23,7 @@ _require_test_program "t_open_tmpfiles" # On high CPU count machines, this runs a -lot- of create and unlink # concurrency. Set the filesytsem up to handle this. -if [ $FSTYP = "xfs" ]; then - _scratch_mkfs "-d agcount=32" >> $seqres.full 2>&1 -else - _scratch_mkfs >> $seqres.full 2>&1 -fi +_scratch_mkfs $(_scratch_mkfs_concurrency_options 32) >> $seqres.full 2>&1 _scratch_mount # Try to load up all the CPUs, two threads per CPU. diff --git a/tests/xfs/501 b/tests/xfs/501 index 678c51b52948c5..4b29ef97d36c1a 100755 --- a/tests/xfs/501 +++ b/tests/xfs/501 @@ -33,7 +33,7 @@ _require_xfs_sysfs debug/log_recovery_delay _require_scratch _require_test_program "t_open_tmpfiles" -_scratch_mkfs "-l size=256m" >> $seqres.full 2>&1 +_scratch_mkfs $(_scratch_mkfs_concurrency_options 32) >> $seqres.full 2>&1 _scratch_mount # Set ULIMIT_NOFILE to min(file-max / 2, 30000 files per LOAD_FACTOR) diff --git a/tests/xfs/502 b/tests/xfs/502 index 10b0017f6b2eb2..df3e7bcb17872d 100755 --- a/tests/xfs/502 +++ b/tests/xfs/502 @@ -23,7 +23,7 @@ _require_xfs_io_error_injection "iunlink_fallback" _require_scratch _require_test_program "t_open_tmpfiles" -_scratch_mkfs "-l size=256m" | _filter_mkfs 2> $tmp.mkfs > /dev/null +_scratch_mkfs $(_scratch_mkfs_concurrency_options 32) | _filter_mkfs 2> $tmp.mkfs > /dev/null cat $tmp.mkfs >> $seqres.full . $tmp.mkfs From patchwork Thu Jan 16 23:28:02 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13942527 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 531BF158520; Thu, 16 Jan 2025 23:28:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737070083; cv=none; b=ILmaNMzYJLSQmJEiZGb9cmdQak/WNhyCBcWYEmrpV/1V+ySbSjthh/TPOXgqVssdNzq1HZcTpcLFDNLO9yIMOLiTqK2FH0LeEoohziMEkBwm4TGhV2uKTItnsI8zeSYCqAc9j1N9qU32t3dH/IqXAbgZWzg1kThSNlfOLv9Bx00= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737070083; c=relaxed/simple; bh=SF1mBj/m9kWKItneW0c2xDO8rz/3FsnINM96FO/7Y0I=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=sMJ5kbsiZPt0SYrp5RbRMitD06TC6cR8dc4NPKj9fa4xZnGTipRr/k/bfUWuAH3WJjAlGoxHLfuMlN7R76lcD2EmdUjO2+daih5dEkAil9zhbGnRmDNEuSfmlfr/9/mhcdKOakLQW6iaXISNW8qwd/xUfCwIW9SU+BdpVZKHT6A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=R12rnhOB; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="R12rnhOB" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E11A5C4CED6; Thu, 16 Jan 2025 23:28:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1737070082; bh=SF1mBj/m9kWKItneW0c2xDO8rz/3FsnINM96FO/7Y0I=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=R12rnhOB0kD7UxXvn1rQ22l67zOvSptQjVVDBgNEJfVRkJaJOCVZxVUtYRLmxTV6H Klz31mGJVR2IBrB4dDikkoOI32CNbnsrV8BuixOyNymRxdrXHCrnEdrm2rwavYens5 Kf0UQf5TUr1OWbZRqcw70pL+rxg/bNzc1nFEEgvjYPEg+PMQ5Xs5I7F1jxcgFeN5M2 2d+dClvYkRYCX48IZuOQrd7CFDJytY+0oG0UdiFDCtG4oXU9UWVv9tpU1cTG93Moks GMrMmSRHZ4USqVs5k5Xv+aso319b66JUtSB1g99x3ab/7J/q5KFs3p1Kx7i1+aAeq3 sD9KKnkJ6u08g== Date: Thu, 16 Jan 2025 15:28:02 -0800 Subject: [PATCH 11/23] common/xfs: find loop devices for non-blockdevs passed to _prepare_for_eio_shutdown From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: hch@lst.de, fstests@vger.kernel.org, linux-xfs@vger.kernel.org Message-ID: <173706974243.1927324.9105721327110864014.stgit@frogsfrogsfrogs> In-Reply-To: <173706974044.1927324.7824600141282028094.stgit@frogsfrogsfrogs> References: <173706974044.1927324.7824600141282028094.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong xfs/336 does this somewhat sketchy thing where it mdrestores into a regular file, and then does this to validate the restored metadata: SCRATCH_DEV=$TEST_DIR/image _scratch_mount Unfortunately, commit 1a49022fab9b4d causes the following regression: --- /tmp/fstests/tests/xfs/336.out 2024-11-12 16:17:36.733447713 -0800 +++ /var/tmp/fstests/xfs/336.out.bad 2025-01-04 19:10:39.861871114 -0800 @@ -5,4 +5,5 @@ Create big file Explode the rtrmapbt Create metadump file Restore metadump -Check restored fs +Usage: _set_fs_sysfs_attr +(see /var/tmp/fstests/xfs/336.full for details) This is due to the fact that SCRATCH_DEV is temporarily reassigned to the regular file. That path is passed straight through _scratch_mount to _xfs_prepare_for_eio_shutdown, but that helper _fails because the "dev" argument isn't actually a path to a block device. Fix this by detecting non-bdevs and finding (we hope) the loop device that was created to handle the mount. While we're at it, have the helper return the exit code from mount, not _prepare_for_eio_shutdown. Cc: # v2024.12.08 Fixes: 1a49022fab9b4d ("fstests: always use fail-at-unmount semantics for XFS") Signed-off-by: "Darrick J. Wong" --- common/rc | 8 ++++++++ common/xfs | 6 ++++++ 2 files changed, 14 insertions(+) diff --git a/common/rc b/common/rc index 885669beeb5e26..4419cfc3188374 100644 --- a/common/rc +++ b/common/rc @@ -441,6 +441,7 @@ _try_scratch_mount() [ $mount_ret -ne 0 ] && return $mount_ret _idmapped_mount $SCRATCH_DEV $SCRATCH_MNT _prepare_for_eio_shutdown $SCRATCH_DEV + return $mount_ret } # mount scratch device with given options and _fail if mount fails @@ -658,6 +659,7 @@ _test_mount() [ $mount_ret -ne 0 ] && return $mount_ret _idmapped_mount $TEST_DEV $TEST_DIR _prepare_for_eio_shutdown $TEST_DEV + return $mount_ret } _test_unmount() @@ -4469,6 +4471,12 @@ _destroy_loop_device() losetup -d $dev || _fail "Cannot destroy loop device $dev" } +# Find the loop bdev for a given file, if there is one. +_find_loop_device() +{ + losetup --list -n -O NAME -j "$1" +} + _scale_fsstress_args() { local args="" diff --git a/common/xfs b/common/xfs index 0417a40adba3e2..c68bd6d7c773ac 100644 --- a/common/xfs +++ b/common/xfs @@ -1110,6 +1110,12 @@ _xfs_prepare_for_eio_shutdown() local dev="$1" local ctlfile="error/fail_at_unmount" + # Is this a regular file? Check if there's a loop device somewhere. + # Hopefully that lines up with a mounted filesystem. + if [ ! -b "$dev" ]; then + dev=$(_find_loop_device "$1" | tail -n 1) + fi + # Once we enable IO errors, it's possible that a writer thread will # trip over EIO, cancel the transaction, and shut down the system. # This is expected behavior, so we need to remove the "Internal error" From patchwork Thu Jan 16 23:28:18 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13942528 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AD84A158520; Thu, 16 Jan 2025 23:28:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737070098; cv=none; b=NCti7dZnwXchHG3VPf7UJCQMzzD6oCPtoW4QGCOm9I+FOpgBs1tJan1u44m1IDKVq4fLSOwHHZU+VPo4ZbxJwmaD+2IUqASeH34Nff2lSbJVu2C9jNDQnPD52Fh48RxytwkbEjx9SSP6iOi7eUXfUft90grD8QgPRoVcwn7rSk0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737070098; c=relaxed/simple; bh=Wl5xP6xhB0gBa5dsBogm+FnTOOsgy8OUUTBBbUunl4Q=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=TtVOVC9RsdDqQTzj/DvkmI90c3JezN7sOlpa7Cdekrn0AtFySgwJv1fS4/MPZ8SMxbu9Ph3Go79Og6BYgDd+yx6NBSKV47T5Fjv1MoOn3BsJ4/m2LsAaTgEf4VROHZcbH8IL0ut0/YusfYkLW3NZ7bxTZgrM4pX3YNgzz/jPjic= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=GKKZJ048; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="GKKZJ048" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7772DC4CED6; Thu, 16 Jan 2025 23:28:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1737070098; bh=Wl5xP6xhB0gBa5dsBogm+FnTOOsgy8OUUTBBbUunl4Q=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=GKKZJ048gMhKlvbcP7wb6a2ddRvL9RarX3Tlq9lssLkmaaQKSH+3UMG2gmXCJNfjJ lFg9U3PN7kfj0y2zT2NkUs0DQhc2fIyHGoSWP1xiiAG1JxwETkosvFpRcVaSdlB49/ OQn1MwKCsWaiTK36l4RCKTxGEd5mbcfLaYL1k0Htgnjsq1EIhvQTqtf7toAQ+baqhj VQiGlZAgQ01YzZSGG3D9k1bpDKMPrVX9kB+UtfPzAsML8sT73nnekCNQp0bxUBG0x4 aICMdxp7lQdopwEq/ywy5lvwa/rNTMymaCBOuMa+BvgCDYHvVlR4zGU+nny9Y+f/K0 181NYd0RYowZA== Date: Thu, 16 Jan 2025 15:28:18 -0800 Subject: [PATCH 12/23] preamble: fix missing _kill_fsstress From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: hch@lst.de, fstests@vger.kernel.org, linux-xfs@vger.kernel.org Message-ID: <173706974258.1927324.7737993478703584623.stgit@frogsfrogsfrogs> In-Reply-To: <173706974044.1927324.7824600141282028094.stgit@frogsfrogsfrogs> References: <173706974044.1927324.7824600141282028094.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Commit 8973af00ec212f added a _kill_fsstress to the standard _cleanup function. However, if something breaks during test program initialization before it gets to sourcing common/rc, then you get failures that look like this: --- /tmp/fstests/tests/generic/556.out 2024-09-25 12:09:52.938797554 -0700 +++ /var/tmp/fstests/generic/556.out.bad 2025-01-04 22:34:01.268327003 -0800 @@ -1,16 +1,3 @@ QA output created by 556 -SCRATCH_MNT/basic Casefold -SCRATCH_MNT/basic -SCRATCH_MNT/casefold_flag_removal Casefold -SCRATCH_MNT/casefold_flag_removal Casefold -SCRATCH_MNT/flag_inheritance/d1/d2/d3 Casefold -SCRATCH_MNT/symlink/ind1/TARGET -mv: 'SCRATCH_MNT/rename/rename' and 'SCRATCH_MNT/rename/RENAME' are the same file -# file: SCRATCH_MNT/xattrs/x -user.foo="bar" - -# file: SCRATCH_MNT/xattrs/x/f1 -user.foo="bar" - -touch: 'SCRATCH_MNT/strict/corac'$'\314\247\303': Invalid argument -touch: 'SCRATCH_MNT/strict/cora'$'\303\247\303': Invalid argument +./tests/generic/556: 108: common/config: Syntax error: "&" unexpected +./tests/generic/556: 10: _kill_fsstress: not found It's that last line that's unnecessary. Fix this by checking for the presence of a _kill_fsstress before invoking it. Cc: # v2024.12.08 Fixes: 8973af00ec212f ("fstests: cleanup fsstress process management") Signed-off-by: "Darrick J. Wong" Reviewed-by: Dave Chinner --- common/preamble | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/common/preamble b/common/preamble index 78e45d522f482c..0c9ee2e0377dd5 100644 --- a/common/preamble +++ b/common/preamble @@ -7,7 +7,7 @@ # Standard cleanup function. Individual tests can override this. _cleanup() { - _kill_fsstress + command -v _kill_fsstress &>/dev/null && _kill_fsstress cd / rm -r -f $tmp.* } From patchwork Thu Jan 16 23:28:33 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13942529 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7E366158520; Thu, 16 Jan 2025 23:28:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737070115; cv=none; b=Utw25cBXrKC5E1CpMy+9+aQpTv40B2bzHkc6aB4q11BPiZh0eAJjysPRNO59mXCwZdLAozuTllWt7xyLsAP05gVO7MqwJsCn6q8QxGN1O5MBKzhGSZhcGqVmcOmqu73KImrr8cICqpEAAKXr34jH+eUshF7Co69PwApUPYs6bLY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737070115; c=relaxed/simple; bh=Xbmso0VYrBw0g9dqlVTyOuZX6fvejTrIpqlmeccAthc=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=KeoLISKvrssW7cQSVz8etH8hbE5+YNgTpt53njEyluBdrmZANmCMwoDDQLDmW/YrhHkPyKFeAGXAQAN9XbiFPbz/efiG+wILPGNpK8Av4QTPovi4VyAyzVHHuFw8cRLNAgdn21RktdfjGE15y7r5p2fLlbvcWcHt7DT+44am5jE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=hb75bkep; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="hb75bkep" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1394CC4CED6; Thu, 16 Jan 2025 23:28:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1737070114; bh=Xbmso0VYrBw0g9dqlVTyOuZX6fvejTrIpqlmeccAthc=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=hb75bkepRL9hJFczK6tfjJ4C5gXloySQ2pWAXU/olu//73shiGOGGAxsQa4763hke 7XAoyd1mPCd9wjPJGvyP8ov7XTfWljWJz5vLfpeYiCMHYUo+vCjKTe8JM5AHVIHe73 9ffz4BseXt0jAxGBHfjpiOcr/xBUsoiLgMqAuBjxgtWH0jOm2vEoPjLf/xC8MUoo/E bIy9NgJXIhq0q5F33xrAef5+NMamjEILv31QRQPDSNTTwEJzSRZf9Zq/qgtKFYqaR3 jPsYjzu5dLmRAJaAtZw64RsRaAxvfpkwqRfVtLjvCz7d8jXpahDYi38YRer72Cud/7 D+ZdqkZTNXFIg== Date: Thu, 16 Jan 2025 15:28:33 -0800 Subject: [PATCH 13/23] generic/650: revert SOAK DURATION changes From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: hch@lst.de, fstests@vger.kernel.org, linux-xfs@vger.kernel.org Message-ID: <173706974273.1927324.11899201065662863518.stgit@frogsfrogsfrogs> In-Reply-To: <173706974044.1927324.7824600141282028094.stgit@frogsfrogsfrogs> References: <173706974044.1927324.7824600141282028094.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Prior to commit 8973af00ec21, in the absence of an explicit SOAK_DURATION, this test would run 2500 fsstress operations each of ten times through the loop body. On the author's machines, this kept the runtime to about 30s total. Oddly, this was changed to 30s per loop body with no specific justification in the middle of an fsstress process management change. On the author's machine, this explodes the runtime from ~30s to 420s. Put things back the way they were. Cc: # v2024.12.08 Fixes: 8973af00ec212f ("fstests: cleanup fsstress process management") Signed-off-by: "Darrick J. Wong" --- tests/generic/650 | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/tests/generic/650 b/tests/generic/650 index 60f86fdf518961..d376488f2fedeb 100755 --- a/tests/generic/650 +++ b/tests/generic/650 @@ -68,11 +68,8 @@ test "$nr_cpus" -gt 1024 && nr_cpus="$nr_hotplug_cpus" fsstress_args+=(-p $nr_cpus) if [ -n "$SOAK_DURATION" ]; then test "$SOAK_DURATION" -lt 10 && SOAK_DURATION=10 -else - # run for 30s per iteration max - SOAK_DURATION=300 + fsstress_args+=(--duration="$((SOAK_DURATION / 10))") fi -fsstress_args+=(--duration="$((SOAK_DURATION / 10))") nr_ops=$((2500 * TIME_FACTOR)) fsstress_args+=(-n $nr_ops) From patchwork Thu Jan 16 23:28:49 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13942530 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 61C29158520; Thu, 16 Jan 2025 23:28:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737070130; cv=none; b=H0RKCHu0NWqfM2WUk3E2JmGLkpG5joI9Z3v4z8hFTmAwWpDlGST1dfU8LV5pgBEYr1Ks4FyudnpaKdnK1aAuU8/8nTZ9JOWPgRKDcpm1q0E6T3gX5NTkVgVvAQ5n9v0WlOF8lsfbh3sXw3F3sW27HvlHtjpAcLrG1+yOfBJIjhg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737070130; c=relaxed/simple; bh=ieWnkCipB5kPbcx22+AjHHxCnlqGHMblPx9+o2f3+e4=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=eQPgIUd1cGhCV4+vEe9R2IYGuoXh4Tn7wbFPlZQgsXRJrqwxDEm2GeSMPQbj/+T2K56gTfEwhYOkOCFrQxIPo77vyaYMxVXpB4YXBcT9vgwB6Xy/MU2rQjTPhzOTmo5n4Cd0xejucS3Bj5R5hL0ttZc0sBdnqjrJUD/529sMJ3s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=rofbTrzZ; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="rofbTrzZ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id BF74BC4CEE3; Thu, 16 Jan 2025 23:28:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1737070129; bh=ieWnkCipB5kPbcx22+AjHHxCnlqGHMblPx9+o2f3+e4=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=rofbTrzZlVpyRytyExye0iAHYIg5e4B2F0laGxJFDwjevGuUcPZV2B8VUDIyr7VJj Mw0wHvkJkzgHVPEStqJ4EhrHzGFIh3DS2slzoSS/DKjymmVw+aNmyQ9n03M99hRNNT ueZOpRMCmPPMqNDmVmm3vhEhuorznVgdzF8O4ctXvAjlo9DgheD6/IaJYDR9L3m9C3 APgQxN+QK2BEDOX3fSXLwy0nIx2a2S6uxvgaPrdT1p13iCdsp8A1U2Uaz+KEUPrlne uMBD0fFu4iLlHFbo+LEKTQp4+K9qa9el46B2zL/OSGFQgXHXZcikzykJ0M47s2oxgZ TTuT7Dzolkb6A== Date: Thu, 16 Jan 2025 15:28:49 -0800 Subject: [PATCH 14/23] generic/032: fix pinned mount failure From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: hch@lst.de, fstests@vger.kernel.org, linux-xfs@vger.kernel.org Message-ID: <173706974288.1927324.17585931341351454094.stgit@frogsfrogsfrogs> In-Reply-To: <173706974044.1927324.7824600141282028094.stgit@frogsfrogsfrogs> References: <173706974044.1927324.7824600141282028094.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong generic/032 now periodically fails with: --- /tmp/fstests/tests/generic/032.out 2025-01-05 11:42:14.427388698 -0800 +++ /var/tmp/fstests/generic/032.out.bad 2025-01-06 18:20:17.122818195 -0800 @@ -1,5 +1,7 @@ QA output created by 032 100 iterations -000000 cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd >................< -* -100000 +umount: /opt: target is busy. +mount: /opt: /dev/sda4 already mounted on /opt. + dmesg(1) may have more information after failed mount system call. +cycle mount failed +(see /var/tmp/fstests/generic/032.full for details) The root cause of this regression is the _syncloop subshell. This background process runs _scratch_sync, which is actually an xfs_io process that calls syncfs on the scratch mount. Unfortunately, while the test kills the _syncloop subshell, it doesn't actually kill the xfs_io process. If the xfs_io process is in D state running the syncfs, it won't react to the signal, but it will pin the mount. Then the _scratch_cycle_mount fails because the mount is pinned. Prior to commit 8973af00ec212f the _syncloop ran sync(1) which avoided pinning the scratch filesystem. Fix this by pgrepping for the xfs_io process and killing and waiting for it if necessary. Cc: # v2024.12.08 Fixes: 8973af00ec212f ("fstests: cleanup fsstress process management") Signed-off-by: "Darrick J. Wong" Reviewed-by: Dave Chinner --- common/rc | 6 ++++++ tests/generic/032 | 9 +++++++++ 2 files changed, 15 insertions(+) diff --git a/common/rc b/common/rc index 4419cfc3188374..d7f3c48eafe590 100644 --- a/common/rc +++ b/common/rc @@ -36,6 +36,12 @@ _pkill() pkill --session 0 "$@" } +# Find only the test processes started by this test +_pgrep() +{ + pgrep --session 0 "$@" +} + # Common execution handling for fsstress invocation. # # We need per-test fsstress binaries because of the way fsstress forks and diff --git a/tests/generic/032 b/tests/generic/032 index 30290c7225a2fa..48d594fe9315b8 100755 --- a/tests/generic/032 +++ b/tests/generic/032 @@ -81,6 +81,15 @@ echo $iters iterations kill $syncpid wait +# The xfs_io instance started by _scratch_sync could be stuck in D state when +# the subshell running _syncloop & is killed. That xfs_io process pins the +# mount so we must kill it and wait for it to die before cycling the mount. +dead_syncfs_pid=$(_pgrep xfs_io) +if [ -n "$dead_syncfs_pid" ]; then + _pkill xfs_io + wait $dead_syncfs_pid +fi + # clear page cache and dump the file _scratch_cycle_mount _hexdump $SCRATCH_MNT/file From patchwork Thu Jan 16 23:29:04 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13942531 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B01E3246A1A; Thu, 16 Jan 2025 23:29:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737070145; cv=none; b=Xhj2H8rbc1tX1IJjLTu7wVxoSfkMWSGOk3BKp8Z/k+1aJX5iipiNLLr4bimZVsAtEjwylY+HMZtEdirz/lNbPB8mxtPGfMxvVT5b9SFjRsMXuaaVif+zrsGp2QbUCmNn6gDrAIqnJi3BVPTic/8TXQP8LiOL36RzSMqUC4+2opE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737070145; c=relaxed/simple; bh=iO0iI3hdF14/3TfhI+y5wOBx1ezEVL6+28e34+1KT1o=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=NiLZc92BsvxTSJifWceDl1B/NhHOV/ZqoZ9YFvkxQqN5fvx0q+DmTj8RFWGtUGCtlKF4c6ivllKFlZcvgs6nAQod7i+gFOqQOTePj+QSBO93efBiqWXyn/qDuWNCV9njRqecJiZf+NvQSXjjG6+Zd2p014abpvT1VrPPxWvDlYA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=FlKDS+PH; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="FlKDS+PH" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6ED19C4CEE5; Thu, 16 Jan 2025 23:29:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1737070145; bh=iO0iI3hdF14/3TfhI+y5wOBx1ezEVL6+28e34+1KT1o=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=FlKDS+PHN8nNbzZas/ZbMydyzVzffdDKf0AwDxWf00Swq3LtsmdhgroEzD2HAB1sY ptiG+Bq8rSfNj8v5SOwKbN1co7rgA0Yv0y9dux3Mz3wSlmUCIPMRQ25ma6VkhxEnRU WFdUPcnhSL0VzspEBRH2B9OIM3yfnsocywYD3pOv04xIiVwR9J0PPv36ITchg5BHxQ 9rGkDJ2RMLGUlNPdt5hzbYPaC6165GvAmYoGIJFxojN4jNPmfAENap+qJ/gbNS8cee jrEb+o8tMGGaGWKrM01qPC9I1he+1vmE650EaGj+bLOzid+WsuwS3ISUp46gNQiybN xcBS1UHCY//Iw== Date: Thu, 16 Jan 2025 15:29:04 -0800 Subject: [PATCH 15/23] fuzzy: stop __stress_scrub_fsx_loop if fsx fails From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: hch@lst.de, fstests@vger.kernel.org, linux-xfs@vger.kernel.org Message-ID: <173706974303.1927324.5257281389524141050.stgit@frogsfrogsfrogs> In-Reply-To: <173706974044.1927324.7824600141282028094.stgit@frogsfrogsfrogs> References: <173706974044.1927324.7824600141282028094.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Stop the fsx scrub stress loop if fsx returns a nonzero error code. Cc: # v2023.01.15 Fixes: a2056ca8917bc8 ("fuzzy: enhance scrub stress testing to use fsx") Signed-off-by: "Darrick J. Wong" --- common/fuzzy | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/common/fuzzy b/common/fuzzy index 772ce7ddcff6d8..46771b1d117106 100644 --- a/common/fuzzy +++ b/common/fuzzy @@ -942,6 +942,7 @@ __stress_scrub_fsx_loop() { local remount_period="$3" local stress_tgt="$4" # ignored local focus=(-q -X) # quiet, validate file contents + local res # As of November 2022, 2 million fsx ops should be enough to keep # any filesystem busy for a couple of hours. @@ -993,7 +994,9 @@ __stress_scrub_fsx_loop() { # Need to recheck running conditions if we cleared anything __stress_scrub_clean_scratch && continue $here/ltp/fsx $args >> $seqres.full - echo "fsx exits with $? at $(date)" >> $seqres.full + res=$? + echo "fsx exits with $res at $(date)" >> $seqres.full + test "$res" -ne 0 && break done rm -f "$runningfile" } From patchwork Thu Jan 16 23:29:20 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13942532 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 759CC25533D; Thu, 16 Jan 2025 23:29:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737070161; cv=none; b=El/blULUJEHR8O38xh53jwYlggneyBa5iAfYpU1G0uDXtC5rtAdWAc9Niu0Y2iOc9Prq936kMqkwCWPUL3lLFvKHPHTd77+VhrQs6rVdF/LoWbUaBakFqg50DexwmpHauewivIUB6qcILMYC7UMc6R3rvka6LK4E8mX/cok9rHk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737070161; c=relaxed/simple; bh=BRT6MpGEu6+R7PprLQVjGH6LqRHDWm5ghc05XcM6aos=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=GkrI6sBL/yONC2gwOA+ETNetmZMpUpYUitMeFresKTqJ6pIoPWDqTukxcJd+613O+2AwbWqeb2IPr6/jPDuDMQFMCO5GyBNgnPFoPOTjv+oAGh0247O09Ex8ocozSEz1I+3YTJpQ7tX8D+BPGzixUmkSZbdVU+Aq81fIebejnlc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=m54PYWYb; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="m54PYWYb" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 11C53C4CEDD; Thu, 16 Jan 2025 23:29:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1737070161; bh=BRT6MpGEu6+R7PprLQVjGH6LqRHDWm5ghc05XcM6aos=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=m54PYWYbElwTSlkyOlqpJQ5MAphnZUxKZcDBdvzwXMDe3EFsbhxdcDD7vC6fS35Gy dPtQIZoFuNVObmfGxpl+F9iRrVHLV7FknMZRLDeFmWJq7DTPMS/PizkHx+olVaFZIb 7gEWxO1AyFMCbQz3AlR6kYKqdUaY4RtjyjAn3wV74TY9OLeVKAlpUJxJbREmkJbH+3 txdaCr0yL93eVs6FzF5a91miAs8rbVbfVZf4TsMVUjUK/Pcz49Ix26ScVyvZMB7A5d FBrsGglYo6g92e9qqUfjcfdh4X6+UX2c/igDjlItVcbnUkxnOzPBdQOM1enyZyFV1v GEFS9iiogrlVA== Date: Thu, 16 Jan 2025 15:29:20 -0800 Subject: [PATCH 16/23] fuzzy: don't use readarray for xfsfind output From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: hch@lst.de, fstests@vger.kernel.org, linux-xfs@vger.kernel.org Message-ID: <173706974318.1927324.15556508804317088041.stgit@frogsfrogsfrogs> In-Reply-To: <173706974044.1927324.7824600141282028094.stgit@frogsfrogsfrogs> References: <173706974044.1927324.7824600141282028094.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Some of the scrub stress tests (e.g. xfs/796) walk the directory tree to find filepaths to scrub, and load the entire list of paths into a bash array. On a large filesystem or a long-running test this is hugely wasteful of memory because we use each path exactly once. Fix __stress_one_scrub_loop to avoid this by reading lines directly from the output of the xfsfind utility. However, we play some games with fd 77 so that the processes in the loop body will use the same stdin as the test and /not/ the piped stdout of xfsfind. To avoid read(1) becoming confused by newlines in the file paths, adapt xfsfind to print nulls between pathnames, and the bash code to recognize them. This was a debugging patch while I was trying to figure out why xfs/286 and other scrub soak tests started OOMing after the v2024.12.08 changes, though in the end the OOMs were the result of memory leaks in fsstress. Signed-off-by: "Darrick J. Wong" --- common/fuzzy | 22 +++++++++++++--------- src/xfsfind.c | 14 +++++++++++--- 2 files changed, 24 insertions(+), 12 deletions(-) diff --git a/common/fuzzy b/common/fuzzy index 46771b1d117106..f97c92340ba9ae 100644 --- a/common/fuzzy +++ b/common/fuzzy @@ -861,14 +861,14 @@ __stress_one_scrub_loop() { ;; esac - local target_cmd=(echo "$scrub_tgt") + local target_cmd=(echo -en "$scrub_tgt\0") case "$scrub_tgt" in - "%file%") target_cmd=($here/src/xfsfind -q "$SCRATCH_MNT");; - "%attrfile%") target_cmd=($here/src/xfsfind -qa "$SCRATCH_MNT");; - "%datafile%") target_cmd=($here/src/xfsfind -qb "$SCRATCH_MNT");; - "%dir%") target_cmd=($here/src/xfsfind -qd "$SCRATCH_MNT");; - "%regfile%") target_cmd=($here/src/xfsfind -qr "$SCRATCH_MNT");; - "%cowfile%") target_cmd=($here/src/xfsfind -qs "$SCRATCH_MNT");; + "%file%") target_cmd=($here/src/xfsfind -0q "$SCRATCH_MNT");; + "%attrfile%") target_cmd=($here/src/xfsfind -0qa "$SCRATCH_MNT");; + "%datafile%") target_cmd=($here/src/xfsfind -0qb "$SCRATCH_MNT");; + "%dir%") target_cmd=($here/src/xfsfind -0qd "$SCRATCH_MNT");; + "%regfile%") target_cmd=($here/src/xfsfind -0qr "$SCRATCH_MNT");; + "%cowfile%") target_cmd=($here/src/xfsfind -0qs "$SCRATCH_MNT");; esac while __stress_scrub_running "$scrub_startat" "$runningfile"; do @@ -876,12 +876,16 @@ __stress_one_scrub_loop() { done while __stress_scrub_running "$end" "$runningfile"; do - readarray -t fnames < <("${target_cmd[@]}" 2>> $seqres.full) - for fname in "${fnames[@]}"; do + # Attach the stdout of xfsfind to fd 77 so that we can read + # pathnames from that file descriptor without passing the pipe + # to the loop body as stdin. + exec 77< <("${target_cmd[@]}" 2>> $seqres.full) + while read -u 77 -d '' fname; do $XFS_IO_PROG -x "${xfs_io_args[@]}" "$fname" 2>&1 | \ __stress_scrub_filter_output "${extra_filters[@]}" __stress_scrub_running "$end" "$runningfile" || break done + exec 77<&- done } diff --git a/src/xfsfind.c b/src/xfsfind.c index c81deaf64f57e9..2043d01ded3210 100644 --- a/src/xfsfind.c +++ b/src/xfsfind.c @@ -20,6 +20,7 @@ static int want_dir; static int want_regfile; static int want_sharedfile; static int report_errors = 1; +static int print0; static int check_datafile( @@ -115,6 +116,7 @@ print_help( printf("\n"); printf("Print all file paths matching any of the given predicates.\n"); printf("\n"); + printf("-0 Print nulls between paths instead of newlines.\n"); printf("-a Match files with xattrs.\n"); printf("-b Match files with data blocks.\n"); printf("-d Match directories.\n"); @@ -208,8 +210,13 @@ visit( out_fd: close(fd); out: - if (printme) - printf("%s\n", path); + if (printme) { + if (print0) + printf("%s%c", path, 0); + else + printf("%s\n", path); + fflush(stdout); + } return retval; } @@ -236,8 +243,9 @@ main( int c; int ret; - while ((c = getopt(argc, argv, "abdqrs")) >= 0) { + while ((c = getopt(argc, argv, "0abdqrs")) >= 0) { switch (c) { + case '0': print0 = 1; break; case 'a': want_attrfile = 1; break; case 'b': want_datafile = 1; break; case 'd': want_dir = 1; break; From patchwork Thu Jan 16 23:29:36 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13942533 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 25984244153; Thu, 16 Jan 2025 23:29:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737070177; cv=none; b=pPxIaQI4eqhx9DqKBueRzdyAr/pFLUjz5vc4HRzf3wcDMBDcrVACS8xgsY9mIXz+N6PGnBPgiv17O9rkuhjZ07e5aA3E/f78g93VC/b8ifNSsHd5i2V1BBFFz0IYMlYFHd+Z4ma5tm4OVPHXSxjXorrYXU69lRYzAF1H3k52NHs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737070177; c=relaxed/simple; bh=X9Ji8lr1QW5xZtOi5oMyQIkT61qXQ0IYWjXUWtQlwbA=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=QUSvQ+tqWfF4WXmeDs9P2GPwBRz3q1wlaCG1Yq/nsJkSi4Y7Vtj8SKdGH3ZqLAMpD11djMehn8tq2e8Its3eIhmE7g9ml0PexfclDYkNFi4W4U8VicExWkSuGTXX1JfqEMJWnQJtdnddGlw/XdOG1jhXpr2BEM3GHLjUbGvHPtY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=tf6uE5Ck; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="tf6uE5Ck" Received: by smtp.kernel.org (Postfix) with ESMTPSA id AEBDCC4CED6; Thu, 16 Jan 2025 23:29:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1737070176; bh=X9Ji8lr1QW5xZtOi5oMyQIkT61qXQ0IYWjXUWtQlwbA=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=tf6uE5Ckl2C1DRp6kNCh8T6cU0MQDFcqf9k1V4ihGr1Gr4rXInGc/SQBEqt3BdN+c msWjJ3EOFGDxgrD8tJ/vrN3iWT3mWNVRzx/lzRzicQ+U4qcqXeybr3uAmK3g1rHQ+3 lWUjO/hfwV/tntt3+7JlQY97smWh40aSe9AKu6IP/vpZT9JM3jK5loOeI9jiPu7u4Q SPFVTVzIBDI3wuvshzaioCYTBOvqWRWqGvepv9lP0ygJmWq1QMxM3LaVfvsK6K88xI q3lW/fbMCy5Vaxu9chBuPy/pdsE7eHfnOk5vvj1wbDJ6jjjmh1QHg6DBwoaV4wapfn asYfjJuhpjn3A== Date: Thu, 16 Jan 2025 15:29:36 -0800 Subject: [PATCH 17/23] fuzzy: always stop the scrub fsstress loop on error From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: hch@lst.de, fstests@vger.kernel.org, linux-xfs@vger.kernel.org Message-ID: <173706974333.1927324.5781121702398584131.stgit@frogsfrogsfrogs> In-Reply-To: <173706974044.1927324.7824600141282028094.stgit@frogsfrogsfrogs> References: <173706974044.1927324.7824600141282028094.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Always abort the non-remount scrub fsstress loop in __stress_scrub_fsstress_loop if fsstress returns a nonzero exit code. Cc: # v2023.01.15 Fixes: 20df87599f66d0 ("fuzzy: make scrub stress loop control more robust") Signed-off-by: "Darrick J. Wong" --- common/fuzzy | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/common/fuzzy b/common/fuzzy index f97c92340ba9ae..e9150173a5d723 100644 --- a/common/fuzzy +++ b/common/fuzzy @@ -1012,6 +1012,7 @@ __stress_scrub_fsstress_loop() { local remount_period="$3" local stress_tgt="$4" local focus=() + local res case "$stress_tgt" in "parent") @@ -1143,7 +1144,9 @@ __stress_scrub_fsstress_loop() { # Need to recheck running conditions if we cleared anything __stress_scrub_clean_scratch && continue _run_fsstress $args >> $seqres.full - echo "fsstress exits with $? at $(date)" >> $seqres.full + res=$? + echo "$mode fsstress exits with $res at $(date)" >> $seqres.full + [ "$res" -ne 0 ] && break; done rm -f "$runningfile" } From patchwork Thu Jan 16 23:29:51 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13942654 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 75C7B244120; Thu, 16 Jan 2025 23:29:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737070192; cv=none; b=IHjF+cEBMc6J8Chg+4nAv+dO3Fodc8dmm79G7+solEmOP3+pHj8vokCxbWrvTH5y0otKkwMyJstgqY5sKoJH0GX2VqOs9B/zgCpM2EkkDYmESvAtxp3rcV9LtTooiKyPZYWRzea0IbUW7fpuqaWZyJvLo99Ly0zgACYJo3mGdug= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737070192; c=relaxed/simple; bh=j6BIMpCWn3HdOSM7wSPlEkZqVHz/YluzwmaLJh02kA8=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Ws1s80GbzxBA7SvWVUCd+ljY91eaiIolgWRexraNmLa/ZaR+kZx/Abhlu+O0FgbDHJaN7ZOXENX84H2UeHnKfXHwFhklZAlWLHVI3J+RQVZuz6Huxrf/0+/WZ0GqoS6/f68V1CaekLSgxf1fWc+nVkSxISr2MVRHd5cdi8WdKrE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=q2xoAPV5; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="q2xoAPV5" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4B215C4CED6; Thu, 16 Jan 2025 23:29:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1737070192; bh=j6BIMpCWn3HdOSM7wSPlEkZqVHz/YluzwmaLJh02kA8=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=q2xoAPV5VMc5H6W621TKIakNQZZI/nAjHoB9dvQMd44QtVX9aObygk3ePpbCMcyjE hTZtM+A86sqQeNlALvJH3hgHzb/5cXnyz8GayvR5uUz+9xv6sSiuBIl+1yCTy1DFrk k8E4zGj6GHikuALK0B1tIs1tRlwcb9uyi0zYAOrKC9vt3ySTCXD4WOUN55Ipwvwf0I hJaOtuUiXAfGVuoiI+ejgoLT9YIfLDokVvEVLhxalnxO3KYXCcuoAMOj2VSV40l1kI WRBvM1hKWnhNa3ZbB0+axqsgSaSVcjrtpIO5aYBNkNIpZ5iT1eU3xZy3mMXwXvWoLe fvK+NwU8iuHRg== Date: Thu, 16 Jan 2025 15:29:51 -0800 Subject: [PATCH 18/23] fuzzy: port fsx and fsstress loop to use --duration From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: hch@lst.de, fstests@vger.kernel.org, linux-xfs@vger.kernel.org Message-ID: <173706974348.1927324.11258907441789009054.stgit@frogsfrogsfrogs> In-Reply-To: <173706974044.1927324.7824600141282028094.stgit@frogsfrogsfrogs> References: <173706974044.1927324.7824600141282028094.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Quite a while ago, I added --duration= arguments to fsx and fsstress, and apparently I forgot to update the scrub stress loops to use them. Replace the usage of timeout(1) for the remount_period versions of the loop to clean up that code; and convert the non-remount loop so that they don't run over time. Signed-off-by: "Darrick J. Wong" --- common/fuzzy | 40 ++++++++++++++++++++++++++++------------ 1 file changed, 28 insertions(+), 12 deletions(-) diff --git a/common/fuzzy b/common/fuzzy index e9150173a5d723..331bf5ad7bbafa 100644 --- a/common/fuzzy +++ b/common/fuzzy @@ -939,6 +939,22 @@ __stress_scrub_clean_scratch() { return 0 } +# Compute a --duration= interval for fsx and fsstress +___stress_scrub_duration() +{ + local end="$1" + local remount_period="$2" + local now="$(date +%s)" + local delta="$((end - now))" + + test "$delta" -lt 0 && delta=0 + + test -n "$remount_period" && test "$remount_period" -lt "$delta" && \ + delta="$remount_period" + + echo "--duration=$delta" +} + # Run fsx while we're testing online fsck. __stress_scrub_fsx_loop() { local end="$1" @@ -946,6 +962,7 @@ __stress_scrub_fsx_loop() { local remount_period="$3" local stress_tgt="$4" # ignored local focus=(-q -X) # quiet, validate file contents + local duration local res # As of November 2022, 2 million fsx ops should be enough to keep @@ -965,17 +982,12 @@ __stress_scrub_fsx_loop() { # anything. test "$mode" = "rw" && __stress_scrub_clean_scratch && continue - timeout -s TERM "$remount_period" $here/ltp/fsx \ - $args $rw_arg >> $seqres.full + duration=$(___stress_scrub_duration "$end" "$remount_period") + $here/ltp/fsx $duration $args $rw_arg >> $seqres.full res=$? echo "$mode fsx exits with $res at $(date)" >> $seqres.full - if [ "$res" -ne 0 ] && [ "$res" -ne 124 ]; then - # Stop if fsx returns error. Mask off - # the magic code 124 because that is how the - # timeout(1) program communicates that we ran - # out of time. - break; - fi + test "$res" -ne 0 && break + if [ "$mode" = "rw" ]; then mode="ro" rw_arg="-t 0 -w 0 -FHzCIJBE0" @@ -997,7 +1009,8 @@ __stress_scrub_fsx_loop() { while __stress_scrub_running "$end" "$runningfile"; do # Need to recheck running conditions if we cleared anything __stress_scrub_clean_scratch && continue - $here/ltp/fsx $args >> $seqres.full + duration=$(___stress_scrub_duration "$end" "$remount_period") + $here/ltp/fsx $duration $args >> $seqres.full res=$? echo "fsx exits with $res at $(date)" >> $seqres.full test "$res" -ne 0 && break @@ -1013,6 +1026,7 @@ __stress_scrub_fsstress_loop() { local stress_tgt="$4" local focus=() local res + local duration case "$stress_tgt" in "parent") @@ -1115,7 +1129,8 @@ __stress_scrub_fsstress_loop() { # anything. test "$mode" = "rw" && __stress_scrub_clean_scratch && continue - _run_fsstress_bg $args $rw_arg >> $seqres.full + duration=$(___stress_scrub_duration "$end" "$remount_period") + _run_fsstress_bg $duration $args $rw_arg >> $seqres.full sleep $remount_period _kill_fsstress res=$? @@ -1143,7 +1158,8 @@ __stress_scrub_fsstress_loop() { while __stress_scrub_running "$end" "$runningfile"; do # Need to recheck running conditions if we cleared anything __stress_scrub_clean_scratch && continue - _run_fsstress $args >> $seqres.full + duration=$(___stress_scrub_duration "$end" "$remount_period") + _run_fsstress $duration $args >> $seqres.full res=$? echo "$mode fsstress exits with $res at $(date)" >> $seqres.full [ "$res" -ne 0 ] && break; From patchwork Thu Jan 16 23:30:07 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13942655 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 15D00244F88; Thu, 16 Jan 2025 23:30:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737070208; cv=none; b=jwwmww1Sjv06quk3qyHPnawHy4UPFUc5tFP9R9Y6rMCJ2xjyzRyeCQJBzUcYkXXxtX2Apa/rQfgE0qdv2WhuYwg9/Z6oGskLcsDfqJHC9woc2ilxsb/cH81ubPPsqmN1OTVoIKf0PEGynyBXRj0EqDMnAi9dyRFeswj7qKgSIlI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737070208; c=relaxed/simple; bh=8W4jon+wBP5T2YfRVfvvWAa61PCluVstIBpzRPPKUtQ=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=mfX0SBUPVeeqGNWUd9acbOdUtSDs4kOudanRDUJ9OOM+TlvL1Z0UJedbB/JQna0rBD4fBAJ4Paroeu/JKNjri19CMEuZ6/rElguszgXx/yosc+sxNbKgkm65K28DyvB3blagHemZIGbKQJ7ipJrEzDlf33AbNdRtU9M+R6qzkUc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=dq509AcF; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="dq509AcF" Received: by smtp.kernel.org (Postfix) with ESMTPSA id D7F1DC4CED6; Thu, 16 Jan 2025 23:30:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1737070207; bh=8W4jon+wBP5T2YfRVfvvWAa61PCluVstIBpzRPPKUtQ=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=dq509AcFegUcW7DKuOGSKbBIr+AMDH1rRmkUys8q82YbiHcb15XQ54XnaMwKtoMaY G83RQMoFx+jCOKynjrkfalHZKtPP1k0FzS0L6SFZoSShAkNWSHMXEvKEmfVfIlFZoB z3MCnuKX9+/fJY+6B+M+JrZHwf62b9LsVFzNzyJ9/mJtHqYF5oEi7mkEx3tDWOquXm uEzwF5hYMeO9kZou5ZqLDz9JQdYCbs553HCggx7QlLOJMgeCCkAEYm7bg+/k27bXt9 uRqOfnhNjlyZNLHiWW9L1LViLab9bPvMj0+Xtphkyz6LalsV0xInlgC8NHIWBWNdDJ nmcc7QbLVNhOg== Date: Thu, 16 Jan 2025 15:30:07 -0800 Subject: [PATCH 19/23] common/rc: don't copy fsstress to $TEST_DIR From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: hch@lst.de, fstests@vger.kernel.org, linux-xfs@vger.kernel.org Message-ID: <173706974363.1927324.3221404706023084828.stgit@frogsfrogsfrogs> In-Reply-To: <173706974044.1927324.7824600141282028094.stgit@frogsfrogsfrogs> References: <173706974044.1927324.7824600141282028094.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Now that we can pkill only processes that were started by this test, we don't need to copy the fsstress binary to $TEST_DIR to avoid killing the wrong program instances. This avoids a whole slew of ETXTBSY problems with scrub stress tests that run multiple copies of fsstress in the background. Revert most of the changes to generic/270, because it wants to do something fancy with the fsstress binary, so it needs to control the process directly. Signed-off-by: "Darrick J. Wong" Reviewed-by: Dave Chinner --- common/rc | 14 +++++--------- tests/generic/270 | 10 ++++++---- tests/generic/482 | 1 + 3 files changed, 12 insertions(+), 13 deletions(-) diff --git a/common/rc b/common/rc index d7f3c48eafe590..25eb13ab2c5a48 100644 --- a/common/rc +++ b/common/rc @@ -56,11 +56,9 @@ _pgrep() # task name to kill. # # If tasks want to start fsstress themselves (e.g. under a different uid) then -# they can set up _FSSTRESS_BIN and record _FSSTRESS_PID themselves. Then if the -# test is killed then it will get cleaned up automatically. +# they can record _FSSTRESS_PID themselves. Then if the test is killed then it +# will get cleaned up automatically. -_FSSTRESS_BIN="$seq.fsstress" -_FSSTRESS_PROG="$TEST_DIR/$seq.fsstress" _FSSTRESS_PID="" _wait_for_fsstress() { @@ -71,7 +69,6 @@ _wait_for_fsstress() ret=$? unset _FSSTRESS_PID fi - rm -f $_FSSTRESS_PROG return $ret } @@ -80,8 +77,8 @@ _kill_fsstress() { if [ -n "$_FSSTRESS_PID" ]; then # use SIGPIPE to avoid "Killed" messages from bash - echo "killing $_FSSTRESS_BIN" >> $seqres.full - _pkill -PIPE $_FSSTRESS_BIN >> $seqres.full 2>&1 + echo "killing fsstress" >> $seqres.full + _pkill -PIPE fsstress >> $seqres.full 2>&1 _wait_for_fsstress return $? fi @@ -89,8 +86,7 @@ _kill_fsstress() _run_fsstress_bg() { - cp -f $FSSTRESS_PROG $_FSSTRESS_PROG - $_FSSTRESS_PROG $FSSTRESS_AVOID "$@" >> $seqres.full 2>&1 & + $FSSTRESS_PROG $FSSTRESS_AVOID "$@" >> $seqres.full 2>&1 & _FSSTRESS_PID=$! } diff --git a/tests/generic/270 b/tests/generic/270 index d74971bb535239..ce51592004fe77 100755 --- a/tests/generic/270 +++ b/tests/generic/270 @@ -28,8 +28,8 @@ _workout() args=`_scale_fsstress_args -p128 -n999999999 -f setattr=1 $FSSTRESS_AVOID -d $out` echo "fsstress $args" >> $seqres.full # Grant chown capability - cp $FSSTRESS_PROG $_FSSTRESS_PROG - $SETCAP_PROG cap_chown=epi $_FSSTRESS_PROG + cp $FSSTRESS_PROG $tmp.fsstress.bin + $SETCAP_PROG cap_chown=epi $tmp.fsstress.bin # io_uring accounts memory it needs under the rlimit memlocked option, # which can be quite low on some setups (especially 64K pagesize). root @@ -37,7 +37,7 @@ _workout() # io_uring_queue_init fail on ENOMEM, set max locked memory to unlimited # temporarily. ulimit -l unlimited - _su $qa_user -c "$_FSSTRESS_PROG $args" > /dev/null 2>&1 & + _su $qa_user -c "$tmp.fsstress.bin $args" > /dev/null 2>&1 & _FSSTRESS_PID=$! echo "Run dd writers in parallel" @@ -50,7 +50,9 @@ _workout() sleep $enospc_time done - _kill_fsstress + _pkill -PIPE -f fsstress + pidwait $_FSSTRESS_PID + return 0 } _require_quota diff --git a/tests/generic/482 b/tests/generic/482 index 0efc026a160040..8c114ee03058c6 100755 --- a/tests/generic/482 +++ b/tests/generic/482 @@ -68,6 +68,7 @@ lowspace=$((1024*1024 / 512)) # 1m low space threshold # Use a thin device to provide deterministic discard behavior. Discards are used # by the log replay tool for fast zeroing to prevent out-of-order replay issues. +_test_unmount _dmthin_init $devsize $devsize $csize $lowspace _log_writes_init $DMTHIN_VOL_DEV _log_writes_mkfs >> $seqres.full 2>&1 From patchwork Thu Jan 16 23:30:23 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13942656 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 132FC246A36; Thu, 16 Jan 2025 23:30:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737070224; cv=none; b=NwqHsltTuwtz8/lBboje2OgewKTwZw1t5bYDdH3yUYkTiCRTwIM34JCWVpT7h1f3jvv4U0OMidDTBAKbuf+db34FSZuvIEe83qW+jCU7t5UXlebhGSYTXMmvR7PwpXAU18PYI/7zujxY4pEDu9ndyUx21QoYkiCYduUAjKIqF2Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737070224; c=relaxed/simple; bh=kviDYx/tiZ6S2quEjPzgbnlcQNGrHrI04gkxA1v/nW8=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=QIXgLHRSlPSA0MSnIyjgMywHrZ55rkMR8cdf2UofgxWrllP5haEQL0j0CFtrxoTzObI5hIwOX+WEJ2tfA7Nm5szjHNxFMgOPPh1cKYPK97ijNhiJWCs5Zip4o8DMLRmJZRq6vzJRNeZqMFgmAkbG3Jpc/NrICcYeHKjK5HJwp4I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Rsw46QuA; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Rsw46QuA" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 743CCC4CED6; Thu, 16 Jan 2025 23:30:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1737070223; bh=kviDYx/tiZ6S2quEjPzgbnlcQNGrHrI04gkxA1v/nW8=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=Rsw46QuAzkGKN8sJS6s4rHQiAfxosas1nwZUBrkZyL6F4mn6EIniKAfVVAPgrrt9b kJGaziG86D9uu0kITJ7DnQ9j/SABomjReWKBADH0AFDVN9SXtTXX0U3PyXOiBlx4I/ fAimKK4qbAmeMRPIL3WR+3pCCfvlPWA5d3zBx1Qml9Roatkf+t//qBGqQg/DMDk2jy Ctr0JMOMhxpNwmB/o6UjR++S5xF+/fexWp19xZOu+ZxFbqo/CKkQ7J2EaOxdKezqR5 o8akLXeruN5wlzOJJiE2+8DB1yZ7yIFz4xls5cqVyncCvQQsX4Drr6x/OptWE49+hT 6lbS8jWTtxx4Q== Date: Thu, 16 Jan 2025 15:30:23 -0800 Subject: [PATCH 20/23] fix _require_scratch_duperemove ordering From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: hch@lst.de, fstests@vger.kernel.org, linux-xfs@vger.kernel.org Message-ID: <173706974378.1927324.3637401394689754053.stgit@frogsfrogsfrogs> In-Reply-To: <173706974044.1927324.7824600141282028094.stgit@frogsfrogsfrogs> References: <173706974044.1927324.7824600141282028094.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Zorro complained that generic/559 stopped working, and I noticed that the duperemove invocation in the _require_scratch_duperemove function would fail with: Error 2: No such file or directory while getting path to file /opt/file1. Skipping. Error 2: No such file or directory while getting path to file /opt/file2. Skipping. No dedupe candidates found. Gathering file list... The cause of this is the incorrect placement of _require_scratch_dedupe after a _scratch_mount. _require_scratch_dedupe formats, mounts, tests, and unmounts the scratch filesystem, which means that it should not come between a _scratch_mount call and code that uses $SCRATCH_MNT. Cc: # v2024.12.22 Fixes: 3b9f5fc7d7d853 ("common: call _require_scratch_dedupe from _require_scratch_duperemove") Signed-off-by: "Darrick J. Wong" --- common/reflink | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/common/reflink b/common/reflink index 9177c45e70bb37..757f06c1c69fa7 100644 --- a/common/reflink +++ b/common/reflink @@ -80,11 +80,11 @@ _require_scratch_duperemove() { _require_scratch _require_command "$DUPEREMOVE_PROG" duperemove - - _scratch_mkfs > /dev/null - _scratch_mount _require_scratch_dedupe + _scratch_mkfs > /dev/null + _scratch_mount + dd if=/dev/zero of="$SCRATCH_MNT/file1" bs=128k count=1 >& /dev/null dd if=/dev/zero of="$SCRATCH_MNT/file2" bs=128k count=1 >& /dev/null if ! "$DUPEREMOVE_PROG" -d "$SCRATCH_MNT/file1" \ From patchwork Thu Jan 16 23:30:38 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13942657 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8086B22BAB7; Thu, 16 Jan 2025 23:30:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737070239; cv=none; b=GSxZwZypPUirBVtbQYKxueb0RH3gO/wpiXi0u1ZSlS7JQKOUFqiqxqUOvWdZ9j8YV4RgPivu5TW3sxrHnU+kYxqdnaq5K2VbFyvt/kLlhfzcsiKrAziwb8iWxf4hw4Yv3G7xar0yBVKWsOk7QVznKen5wXykiu+H5gwakmmWW94= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737070239; c=relaxed/simple; bh=At3Bxfr7sSrI/qpBYHOXD7YiHX585/7ELYh3rNmWhwQ=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=BjGbr0ROeiYoGR2pAGWEtmxN4lyrB9Vh0qPHoeP+OzJeVVMlvvSDGNSB/KjJ58rlBDDdGiYhOl5ljIjbEawvnC1IAn3MbT2uybvzGOTxcH4uuN55EvXA8A7C5FYTMCqUFgN/9iw8UzqrEyONbj0ZQchDnj9c/O3jTQXvJjS+qgs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=JMTyw8u1; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="JMTyw8u1" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 154E5C4CED6; Thu, 16 Jan 2025 23:30:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1737070239; bh=At3Bxfr7sSrI/qpBYHOXD7YiHX585/7ELYh3rNmWhwQ=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=JMTyw8u1N44aq0s1eico8PH8cDpFgVp7GRj3wghlqH4IqIy28JP4dBTBQbphLZiHO wYNy3tFEn9Y4W3WIaFyFCHtR05oBMqWj1GVeVlVRpLU7HgxKkLyKTFOPvgJjv4Pfpj AR/gFI3LyxQXs7U78xFvthuot/r6t+whCD1jhfuFY1GrsT2AR5R/yKPadZheN2ZHKy I26XeDhdsIX20B2eifFbOECdSl/5CIim9CGAvb5WkOFksbRdfYxCuIrq11szaBXqXo ksWt+BXk/tRzhHsYcvq6QCOZ4tMtIbj9sJC6lftHgq2DUrzI+DMjqjagL316SeA7Nq 65WFEen5D5dZQ== Date: Thu, 16 Jan 2025 15:30:38 -0800 Subject: [PATCH 21/23] fsstress: fix a memory leak From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: hch@lst.de, fstests@vger.kernel.org, linux-xfs@vger.kernel.org Message-ID: <173706974393.1927324.13119018006451156178.stgit@frogsfrogsfrogs> In-Reply-To: <173706974044.1927324.7824600141282028094.stgit@frogsfrogsfrogs> References: <173706974044.1927324.7824600141282028094.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Someone forgot to free the iovec that readv_f allocates. Fixes: 80499d8f5f251e ("fsstress: new writev and readv operations test") Signed-off-by: "Darrick J. Wong" --- ltp/fsstress.c | 1 + 1 file changed, 1 insertion(+) diff --git a/ltp/fsstress.c b/ltp/fsstress.c index 9e8eaa6d8656da..14c29921e8b0f8 100644 --- a/ltp/fsstress.c +++ b/ltp/fsstress.c @@ -4652,6 +4652,7 @@ readv_f(opnum_t opno, long r) } e = readv(fd, iov, iovcnt) < 0 ? errno : 0; + free(iov); free(buf); if (v) printf("%d/%lld: readv %s%s [%lld,%d,%d] %d\n", From patchwork Thu Jan 16 23:30:54 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13942658 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 271192419F3; Thu, 16 Jan 2025 23:30:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737070255; cv=none; b=TeS+HefB1eVciZa5u8bcPgwWav21F2SN2mWkDWBW9ynNzEXDmkM4R+tkOQr0SOPUhU0bLxsuGHz031V40NBbxeJx6MOFNqCAOYLb7+4MTo8n/Y1l0+NiHqurAVMFDJS1yBleQg5+ozKeeyTG6StBD8PLF6RkdM7QwdU67nFfCF8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737070255; c=relaxed/simple; bh=IrM7i4LJFkArnRNgd+TIleFYE6/XmKgM8G4vbLu4IcQ=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=ElXUkNfHkDzsjv2hVy4TuysA/I0sd5Z+vZVhCPd2vinXYGUZP32z50/OZaerj14HhwTMawG/tm1czcT1cNLfgrqyAEYpF2koCPXXyE3t2xYrw9KV9Z00cBLYhO4x7/mYGaA8l/hdi4BVUrHa9HwadwjBFaHNk8skBgRwRspREwA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=kHkATOMB; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="kHkATOMB" Received: by smtp.kernel.org (Postfix) with ESMTPSA id B3737C4CED6; Thu, 16 Jan 2025 23:30:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1737070254; bh=IrM7i4LJFkArnRNgd+TIleFYE6/XmKgM8G4vbLu4IcQ=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=kHkATOMBWdZ0OonEH8sy15UjDgYskJkbxoWidnkTWbQz1V01Ub23Thcmm/TITtOuC vYlOc/h58rl31BXJt+KMt1QNJTUiqRh2K6FeLVB9f0Z1pshHuRGVOeGAPZfLGtvEUR /GpmjQp8X+fJD9iowfU9ALmEnfsPQjEfv54VaESAPdCg+cA/rhzsIs6Ocq93da8+qH Qd1TOM5Apv1vbyE8ycQIXqcBqKp3Kw0w/rdIHmzD/P2fQsCUtcf8J98PvKPeJd315d aHvObuh3p8tDO2qpL0nOxQgS8K6zQ/RJfZ8vU+JnhjcT1RfCsieBAwd79nPa7kymTv /qoCy+6yn/htQ== Date: Thu, 16 Jan 2025 15:30:54 -0800 Subject: [PATCH 22/23] fsx: fix leaked log file pointer From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: hch@lst.de, fstests@vger.kernel.org, linux-xfs@vger.kernel.org Message-ID: <173706974408.1927324.5268034153341516183.stgit@frogsfrogsfrogs> In-Reply-To: <173706974044.1927324.7824600141282028094.stgit@frogsfrogsfrogs> References: <173706974044.1927324.7824600141282028094.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Fix a resource leaks in fsx, where we fail to close the fsx logfile, because the C library could have some buffered contents that aren't flushed when the program terminates. glibc seems to do this for us, but I wouldn't be so sure about the others. Fixes: 3f742550dfed84 ("fsx: add support for recording operations to a file") Signed-off-by: "Darrick J. Wong" --- ltp/fsx.c | 1 + 1 file changed, 1 insertion(+) diff --git a/ltp/fsx.c b/ltp/fsx.c index 9efd2f5c86d11c..2c19fff880330a 100644 --- a/ltp/fsx.c +++ b/ltp/fsx.c @@ -3345,6 +3345,7 @@ main(int argc, char **argv) if (recordops) logdump(); + fclose(fsxlogf); exit(0); return 0; } From patchwork Thu Jan 16 23:31:09 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Darrick J. Wong" X-Patchwork-Id: 13942659 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E45471DE4C3; Thu, 16 Jan 2025 23:31:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737070271; cv=none; b=FQTMNQfbXxtrIT9Jq9zDBgiElS8qga7ZzbTL0+F2LudRhpqEY4+ASiM88/ZXqGYyuMf9DcUzjEFsMDM4vitHJf5NvVPGle6rPhak7ZNRq0yDFT5ngrW6ndQpgnIZKCshSWHC3w+K8US6ZAobrjWJsN6DOmZIdcw/Fo4U0nhTzpY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737070271; c=relaxed/simple; bh=O4Cyfmf1PiUsgG1Ck2sttsvWEKF9xt3iM267UnUXtB8=; h=Date:Subject:From:To:Cc:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=jRN09Sdn4+FZABx+LuxHM4fp7docvrfXMFylYB9iRpePmdL83hVC5DV73h9d/sjmv37QLx52Ycgx+F/zMDaSVJMDu82uX8OS5SW3xGbWtGMZNNz4g79bSWx0LchTi21mrqs+kI+Jh2zdsdKn581qXhKfYCDr7URrGOlPrea6zLA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Y6CgiXhU; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Y6CgiXhU" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5B57AC4CED6; Thu, 16 Jan 2025 23:31:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1737070270; bh=O4Cyfmf1PiUsgG1Ck2sttsvWEKF9xt3iM267UnUXtB8=; h=Date:Subject:From:To:Cc:In-Reply-To:References:From; b=Y6CgiXhUC40qT/J05W0ktfZrZMEUZqJXARG9O8WUAshT1It9dIKMNtvxaAWqRvpDM 0extf7HdSaC5xavgFDMTTH9e/UNoNyJDQNzZSKodW9bBBSOR4rboUPPz8Ku8wFSAOe X+j7OBxAvCg0rGH68WsNNBxOzAhdb+HOZhx32cPCGDQK1+Ayz2CCj1EHSm9eY3lJQ/ BCvdliPalNhRxlu1Bgj+mbim9oF0cdePcUg4wYzkfCGikkjaQBuFKyfYxPqy3eaIcf plH7i9da0/saFid9dIB779IyJOsGKh/NIBY/CcNgoQ3AWrcp2wRB4S6BojHtighzfn 5n39DLDoiHnkg== Date: Thu, 16 Jan 2025 15:31:09 -0800 Subject: [PATCH 23/23] build: initialize stack variables to zero by default From: "Darrick J. Wong" To: zlang@redhat.com, djwong@kernel.org Cc: hch@lst.de, fstests@vger.kernel.org, linux-xfs@vger.kernel.org Message-ID: <173706974423.1927324.17566007446431552196.stgit@frogsfrogsfrogs> In-Reply-To: <173706974044.1927324.7824600141282028094.stgit@frogsfrogsfrogs> References: <173706974044.1927324.7824600141282028094.stgit@frogsfrogsfrogs> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Darrick J. Wong Newer versions of gcc and clang can include the ability to zero stack variables by default. Let's enable it so that we (a) reduce the risk of writing stack contents to disk somewhere and (b) try to reduce unpredictable program behavior based on random stack contents. The kernel added this 6 years ago, so I think it's mature enough for fstests. Signed-off-by: "Darrick J. Wong" --- configure.ac | 1 + include/builddefs.in | 3 ++- m4/package_libcdev.m4 | 14 ++++++++++++++ 3 files changed, 17 insertions(+), 1 deletion(-) diff --git a/configure.ac b/configure.ac index c81411e735a90d..f3c8c643f0eb9e 100644 --- a/configure.ac +++ b/configure.ac @@ -72,6 +72,7 @@ AC_HAVE_NFTW AC_HAVE_RLIMIT_NOFILE AC_NEED_INTERNAL_XFS_IOC_EXCHANGE_RANGE AC_HAVE_FICLONE +AC_HAVE_TRIVIAL_AUTO_VAR_INIT AC_CHECK_FUNCS([renameat2]) AC_CHECK_FUNCS([reallocarray]) diff --git a/include/builddefs.in b/include/builddefs.in index 7274cde8d0814c..5b5864278682a4 100644 --- a/include/builddefs.in +++ b/include/builddefs.in @@ -76,6 +76,7 @@ NEED_INTERNAL_XFS_IOC_EXCHANGE_RANGE = @need_internal_xfs_ioc_exchange_range@ HAVE_FICLONE = @have_ficlone@ GCCFLAGS = -funsigned-char -fno-strict-aliasing -Wall +SANITIZER_CFLAGS += @autovar_init_cflags@ ifeq ($(PKG_PLATFORM),linux) PCFLAGS = -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 $(GCCFLAGS) @@ -90,7 +91,7 @@ GCFLAGS = $(OPTIMIZER) $(DEBUG) $(CPPFLAGS) \ -I$(TOPDIR)/include -DVERSION=\"$(PKG_VERSION)\" # Global, Platform, Local CFLAGS -CFLAGS += $(GCFLAGS) $(PCFLAGS) $(LCFLAGS) +CFLAGS += $(GCFLAGS) $(PCFLAGS) $(LCFLAGS) $(SANITIZER_CFLAGS) include $(TOPDIR)/include/buildmacros diff --git a/m4/package_libcdev.m4 b/m4/package_libcdev.m4 index a0d50f4d9b68e4..ed8fe6e32ae00a 100644 --- a/m4/package_libcdev.m4 +++ b/m4/package_libcdev.m4 @@ -72,3 +72,17 @@ AC_DEFUN([AC_HAVE_FICLONE], AC_MSG_RESULT(yes)],[AC_MSG_RESULT(no)]) AC_SUBST(have_ficlone) ]) + +# Check if we have -ftrivial-auto-var-init=zero +AC_DEFUN([AC_HAVE_TRIVIAL_AUTO_VAR_INIT], + [ AC_MSG_CHECKING([if C compiler supports zeroing automatic vars]) + OLD_CFLAGS="$CFLAGS" + TEST_CFLAGS="-ftrivial-auto-var-init=zero" + CFLAGS="$CFLAGS $TEST_CFLAGS" + AC_LINK_IFELSE([AC_LANG_PROGRAM([])], + [AC_MSG_RESULT([yes])] + [autovar_init_cflags=$TEST_CFLAGS], + [AC_MSG_RESULT([no])]) + CFLAGS="${OLD_CFLAGS}" + AC_SUBST(autovar_init_cflags) + ])