From patchwork Mon Oct 31 11:11:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Filipe Manana X-Patchwork-Id: 13025621 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C1F01FA3744 for ; Mon, 31 Oct 2022 11:11:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229477AbiJaLLf (ORCPT ); Mon, 31 Oct 2022 07:11:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53366 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230090AbiJaLLf (ORCPT ); Mon, 31 Oct 2022 07:11:35 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 59059B847; Mon, 31 Oct 2022 04:11:34 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 1634CB815E0; Mon, 31 Oct 2022 11:11:33 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id ED216C433C1; Mon, 31 Oct 2022 11:11:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1667214691; bh=Oxlp09yroYRgvBSdhCOiolxQcbXzvoPBSpbmhof2feM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=opl3udtGWTC7zekgxmYouurpf8PTePgj5ufw4VQVu0gL8h8W3cev32naEb9MaEewT WvOnZp+YZOKzseCDw4r5D0wAx+Ykio2K+CqDL45v98WlxDjRCT42I8eCwwUbGRrDmU 6FMJfY0iz55yIYLkfjW2zXjLQkVdopULSddnlMxBi/JQ1lKEPg4IYN+eqEqHt7CoRh OaIqj93lavD5Qjmq7UThi3Gz/4pCTPPkWryL0OuwOzg7w+cj9BJkOCBgeSF+/r7BTe 8qodfReH/V6QfdNdxYgN993VBeGRAX2yqsb0c/jPeW1tlftH/lUfxoMzPykbJYwoqu yohUegB8pJ4bw== From: fdmanana@kernel.org To: fstests@vger.kernel.org Cc: linux-btrfs@vger.kernel.org, Filipe Manana Subject: [PATCH 1/3] common/punch: fix flags printing for filter _filter_fiemap_flags Date: Mon, 31 Oct 2022 11:11:19 +0000 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org From: Filipe Manana In the filter _filter_fiemap_flags, if we get a flags field that only has the 'last' flag set, we end up printing the string "nonelast", which is ugly and not intuitive. For example: $XFS_IO_PROG -f -c "pwrite 0 64K" $SCRATCH_MNT/foo > /dev/null $XFS_IO_PROG -c "fiemap -v" $SCRATCH_MNT/foo | _filter_fiemap_flags Gives: 0: [0..127]: nonelast So fix this by updating the filter's awk code to reset the flags string to an empty string if we have the "last" flag set and we haven't updated the flags string before. So now the same test gives the following result: 0: [0..127]: last Signed-off-by: Filipe Manana Reviewed-by: Zorro Lang --- common/punch | 2 ++ 1 file changed, 2 insertions(+) diff --git a/common/punch b/common/punch index 47a29c92..94599c35 100644 --- a/common/punch +++ b/common/punch @@ -130,6 +130,8 @@ _filter_fiemap_flags() if (and(flags, 0x1)) { if (set) { flag_str = flag_str"|"; + } else { + flag_str = ""; } flag_str = flag_str"last"; } From patchwork Mon Oct 31 11:11:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Filipe Manana X-Patchwork-Id: 13025623 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id ECD1BFA3740 for ; Mon, 31 Oct 2022 11:11:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230133AbiJaLLi (ORCPT ); Mon, 31 Oct 2022 07:11:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53378 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230107AbiJaLLg (ORCPT ); Mon, 31 Oct 2022 07:11:36 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9BD13B847; Mon, 31 Oct 2022 04:11:35 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 54903B815E2; Mon, 31 Oct 2022 11:11:34 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2FE14C433D7; Mon, 31 Oct 2022 11:11:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1667214693; bh=KgBdBtmhw7LMT5jL6xTCRVd52JXwApMCk08xC5XsQ7E=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=nmQ1c2kw4JXFSnhAlEixf352xihZja5bkRoDQt/B+ELoL9uIqMQeL5NZj9VnRJ6Pr 3NMF0eyjlNJvS0gLIPf7xhr6hUIQ6+gpkCV/pETTqsF+IGzk81XGRfJmqM7139nYJD CutYeiHO8uAaHFY6TLN/sfKLnRb5VYPh6uIHVJHeW8vEB8lHB4v7aQR1KQqcgnQVhb mLvEE6gWaCzd8Rpo0eevJtuon8aFkYUqreyi9usXyBBLKqEpf2dRFzpSXrvWRrUd/Y 23XlPtZqhQ1o+hE00fDdjy2AQMSmiMtLdv8jZXlWBmMhevATymoEWEf5QhkuaVy8zj KXi6glO12YKUA== From: fdmanana@kernel.org To: fstests@vger.kernel.org Cc: linux-btrfs@vger.kernel.org, Filipe Manana Subject: [PATCH 2/3] btrfs: test that fiemap reports extent as not shared after deleting file Date: Mon, 31 Oct 2022 11:11:20 +0000 Message-Id: <27a0c4ab551b7a7410f4062f5235f20c88e77cfc.1667214081.git.fdmanana@suse.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org From: Filipe Manana Test that if we have two files with shared extents, after removing one of the files, if we do a fiemap against the other file, it does not report extents as shared anymore. This exercises the processing of delayed references for data extents in the backref walking code, used by fiemap to determine if an extent is shared. This used to fail until very recently and was fixed by the following kernel commit that landed in 6.1-rc2: 4fc7b5722824 (""btrfs: fix processing of delayed data refs during backref walking") Signed-off-by: Filipe Manana Reviewed-by: Zorro Lang --- tests/btrfs/279 | 82 +++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/279.out | 39 +++++++++++++++++++++ 2 files changed, 121 insertions(+) create mode 100755 tests/btrfs/279 create mode 100644 tests/btrfs/279.out diff --git a/tests/btrfs/279 b/tests/btrfs/279 new file mode 100755 index 00000000..5b5824fd --- /dev/null +++ b/tests/btrfs/279 @@ -0,0 +1,82 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) 2022 SUSE Linux Products GmbH. All Rights Reserved. +# +# FS QA Test 279 +# +# Test that if we have two files with shared extents, after removing one of the +# files, if we do a fiemap against the other file, it does not report extents as +# shared anymore. +# +# This exercises the processing of delayed references for data extents in the +# backref walking code, used by fiemap to determine if an extent is shared. +# +. ./common/preamble +_begin_fstest auto quick subvol fiemap clone + +. ./common/filter +. ./common/reflink +. ./common/punch # for _filter_fiemap_flags + +_supported_fs btrfs +_require_scratch_reflink +_require_cp_reflink +_require_xfs_io_command "fiemap" + +_fixed_by_kernel_commit 4fc7b5722824 \ + "btrfs: fix processing of delayed data refs during backref walking" + +run_test() +{ + local file_path_1=$1 + local file_path_2=$2 + local do_sync=$3 + + $XFS_IO_PROG -f -c "pwrite 0 64K" $file_path_1 | _filter_xfs_io + _cp_reflink $file_path_1 $file_path_2 + + if [ $do_sync -eq 1 ]; then + sync + fi + + echo "Fiemap of $file_path_1 before deleting $file_path_2:" | \ + _filter_scratch + $XFS_IO_PROG -c "fiemap -v" $file_path_1 | _filter_fiemap_flags + + rm -f $file_path_2 + + echo "Fiemap of $file_path_1 after deleting $file_path_2:" | \ + _filter_scratch + $XFS_IO_PROG -c "fiemap -v" $file_path_1 | _filter_fiemap_flags +} + +_scratch_mkfs >> $seqres.full 2>&1 +_scratch_mount + +# Create two test subvolumes, we'll reflink files between them. +$BTRFS_UTIL_PROG subvolume create $SCRATCH_MNT/subv1 | _filter_scratch +$BTRFS_UTIL_PROG subvolume create $SCRATCH_MNT/subv2 | _filter_scratch + +echo +echo "Testing with same subvolume and without transaction commit" +echo +run_test "$SCRATCH_MNT/subv1/f1" "$SCRATCH_MNT/subv1/f2" 0 + +echo +echo "Testing with same subvolume and with transaction commit" +echo +run_test "$SCRATCH_MNT/subv1/f3" "$SCRATCH_MNT/subv1/f4" 1 + +echo +echo "Testing with different subvolumes and without transaction commit" +echo +run_test "$SCRATCH_MNT/subv1/f5" "$SCRATCH_MNT/subv2/f6" 0 + +echo +echo "Testing with different subvolumes and with transaction commit" +echo +run_test "$SCRATCH_MNT/subv1/f7" "$SCRATCH_MNT/subv2/f8" 1 + +# success, all done +status=0 +exit diff --git a/tests/btrfs/279.out b/tests/btrfs/279.out new file mode 100644 index 00000000..a959a86d --- /dev/null +++ b/tests/btrfs/279.out @@ -0,0 +1,39 @@ +QA output created by 279 +Create subvolume 'SCRATCH_MNT/subv1' +Create subvolume 'SCRATCH_MNT/subv2' + +Testing with same subvolume and without transaction commit + +wrote 65536/65536 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +Fiemap of SCRATCH_MNT/subv1/f1 before deleting SCRATCH_MNT/subv1/f2: +0: [0..127]: shared|last +Fiemap of SCRATCH_MNT/subv1/f1 after deleting SCRATCH_MNT/subv1/f2: +0: [0..127]: last + +Testing with same subvolume and with transaction commit + +wrote 65536/65536 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +Fiemap of SCRATCH_MNT/subv1/f3 before deleting SCRATCH_MNT/subv1/f4: +0: [0..127]: shared|last +Fiemap of SCRATCH_MNT/subv1/f3 after deleting SCRATCH_MNT/subv1/f4: +0: [0..127]: last + +Testing with different subvolumes and without transaction commit + +wrote 65536/65536 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +Fiemap of SCRATCH_MNT/subv1/f5 before deleting SCRATCH_MNT/subv2/f6: +0: [0..127]: shared|last +Fiemap of SCRATCH_MNT/subv1/f5 after deleting SCRATCH_MNT/subv2/f6: +0: [0..127]: last + +Testing with different subvolumes and with transaction commit + +wrote 65536/65536 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +Fiemap of SCRATCH_MNT/subv1/f7 before deleting SCRATCH_MNT/subv2/f8: +0: [0..127]: shared|last +Fiemap of SCRATCH_MNT/subv1/f7 after deleting SCRATCH_MNT/subv2/f8: +0: [0..127]: last From patchwork Mon Oct 31 11:11:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Filipe Manana X-Patchwork-Id: 13025622 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7AE37FA3745 for ; Mon, 31 Oct 2022 11:11:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230112AbiJaLLh (ORCPT ); Mon, 31 Oct 2022 07:11:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53376 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230099AbiJaLLg (ORCPT ); Mon, 31 Oct 2022 07:11:36 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2D394BE12; Mon, 31 Oct 2022 04:11:35 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id C9BB2611B1; Mon, 31 Oct 2022 11:11:34 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6710AC433D6; Mon, 31 Oct 2022 11:11:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1667214694; bh=sOkBrdDnXcDZdgQNhP5o5ExoUYCFtnwmRn4CHam5tqE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=RdNdG6YTqP9Mi65Xfs20VoMuYjjONVVgpepdJWePkyrm2oABFNPLZQLdtDy0favcP gmHrpBWLEnagRZgSoFUo3uvTThJmqdN5hkjNHtpjTYBVUqbb4PaO2jm5w804bBCV2S 0SG/raqkcla8bc/Uw72EC78hh0h1GLzymxa9fte2bu7VM23b3eRHJ0S+Hvo0noW8EZ 5RbFWmiS+LhoRm150zZfY/qrzSM4IvyNR0YPb4WTe3wF6xCU01u8mIVRrieX9wEjc6 m/9EPTa9s02JFag2NgSZqhlSVDewmCnOd1zAj28gKokduBdF2G1EFKry8kodGLc8M8 ZThFwQH8xQeLg== From: fdmanana@kernel.org To: fstests@vger.kernel.org Cc: linux-btrfs@vger.kernel.org, Filipe Manana Subject: [PATCH 3/3] btrfs: test fiemap reports extent as not shared after COWing it in snapshot Date: Mon, 31 Oct 2022 11:11:21 +0000 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: fstests@vger.kernel.org From: Filipe Manana Test that if we have a large file, with file extent items spanning several leaves in the fs tree, and that is shared due to a snapshot, if we COW one of the extents, doing a fiemap will report the respective file range as not shared. This exercises the processing of delayed references for metadata extents in the backref walking code, used by fiemap to determine if an extent is shared. This used to fail until very recently and was fixed by the following kernel commit that landed in 6.1-rc2: 943553ef9b51 (""btrfs: fix processing of delayed tree block refs during backref walking") Signed-off-by: Filipe Manana Reviewed-by: Zorro Lang --- tests/btrfs/280 | 64 +++++++++++++++++++++++++++++++++++++++++++++ tests/btrfs/280.out | 21 +++++++++++++++ 2 files changed, 85 insertions(+) create mode 100755 tests/btrfs/280 create mode 100644 tests/btrfs/280.out diff --git a/tests/btrfs/280 b/tests/btrfs/280 new file mode 100755 index 00000000..06ef221e --- /dev/null +++ b/tests/btrfs/280 @@ -0,0 +1,64 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) 2022 SUSE Linux Products GmbH. All Rights Reserved. +# +# FS QA Test 280 +# +# Test that if we have a large file, with file extent items spanning several +# leaves in the fs tree, and that is shared due to a snapshot, if we COW one of +# the extents, doing a fiemap will report the respective file range as not +# shared. +# +# This exercises the processing of delayed references for metadata extents in +# the backref walking code, used by fiemap to determine if an extent is shared. +# +. ./common/preamble +_begin_fstest auto quick compress snapshot fiemap + +. ./common/filter +. ./common/punch # for _filter_fiemap_flags + +_supported_fs btrfs +_require_scratch +_require_xfs_io_command "fiemap" + +_fixed_by_kernel_commit 943553ef9b51 \ + "btrfs: fix processing of delayed tree block refs during backref walking" + +_scratch_mkfs >> $seqres.full 2>&1 +# We use compression because it's a very quick way to create a file with a very +# large number of extents (compression limits the maximum extent size to 128K) +# and while using very little disk space. +_scratch_mount -o compress + +# A 128M file full of compressed extents results in a fs tree with a heigth +# of 2 (root at level 1), so we'll end up with tree block references in the +# extent tree (if the root was a leaf, we would have only data references). +$XFS_IO_PROG -f -c "pwrite -b 1M 0 128M" $SCRATCH_MNT/foo | _filter_xfs_io + +# Create a RW snapshot of the default subvolume. +$BTRFS_UTIL_PROG subvolume snapshot $SCRATCH_MNT $SCRATCH_MNT/snap | _filter_scratch + +echo +echo "File foo fiemap before COWing extent:" +echo +$XFS_IO_PROG -c "fiemap -v" $SCRATCH_MNT/foo | _filter_fiemap_flags + +echo +echo "Overwriting file range [120M, 120M + 128K) in the snapshot" +echo +$XFS_IO_PROG -c "pwrite -b 128K 120M 128K" $SCRATCH_MNT/snap/foo | _filter_xfs_io +# Now fsync the file to force COWing the extent and the path from the root of +# the snapshot tree down to the leaf where the extent is at. +$XFS_IO_PROG -c "fsync" $SCRATCH_MNT/snap/foo + +echo +echo "File foo fiemap after COWing extent in the snapshot:" +echo +# Now we should have all extents marked as shared except the 128K extent in the +# file range [120M, 120M + 128K). +$XFS_IO_PROG -c "fiemap -v" $SCRATCH_MNT/foo | _filter_fiemap_flags + +# success, all done +status=0 +exit diff --git a/tests/btrfs/280.out b/tests/btrfs/280.out new file mode 100644 index 00000000..c3f82966 --- /dev/null +++ b/tests/btrfs/280.out @@ -0,0 +1,21 @@ +QA output created by 280 +wrote 134217728/134217728 bytes at offset 0 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +Create a snapshot of 'SCRATCH_MNT' in 'SCRATCH_MNT/snap' + +File foo fiemap before COWing extent: + +0: [0..261887]: shared +1: [261888..262143]: shared|last + +Overwriting file range [120M, 120M + 128K) in the snapshot + +wrote 131072/131072 bytes at offset 125829120 +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +File foo fiemap after COWing extent in the snapshot: + +0: [0..245759]: shared +1: [245760..246015]: none +2: [246016..261887]: shared +3: [261888..262143]: shared|last