diff mbox series

[v6,4/4] generic: test fs-verity EFBIG scenarios

Message ID f8189930d20888a7ac95b7fc2fbb0d522e8851fc.1644883592.git.boris@bur.io (mailing list archive)
State Superseded
Headers show
Series tests for btrfs fsverity | expand

Commit Message

Boris Burkov Feb. 15, 2022, 12:09 a.m. UTC
btrfs, ext4, and f2fs cache the Merkle tree past EOF, which restricts
the maximum file size beneath the normal maximum. Test the logic in
those filesystems against files with sizes near the maximum.

To work properly, this does require some understanding of the practical
but not standardized layout of the Merkle tree. This is a bit unpleasant
and could make the test incorrect in the future, if the implementation
changes. On the other hand, it feels quite useful to test this tricky
edge case. It could perhaps be made more generic by adding some ioctls
to let the file system communicate the maximum file size for a verity
file or some information about the storage of the Merkle tree.

Signed-off-by: Boris Burkov <boris@bur.io>
---
 common/verity         | 11 ++++++++
 tests/generic/690     | 66 +++++++++++++++++++++++++++++++++++++++++++
 tests/generic/690.out |  7 +++++
 3 files changed, 84 insertions(+)
 create mode 100755 tests/generic/690
 create mode 100644 tests/generic/690.out

Comments

Eric Biggers March 15, 2022, 2:16 a.m. UTC | #1
On Mon, Feb 14, 2022 at 04:09:58PM -0800, Boris Burkov wrote:
> diff --git a/tests/generic/690 b/tests/generic/690
> new file mode 100755
> index 00000000..77906dd8
> --- /dev/null
> +++ b/tests/generic/690
> @@ -0,0 +1,66 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2021 Facebook, Inc.  All Rights Reserved.
> +#
> +# FS QA Test 690
> +#
> +# fs-verity requires the filesystem to decide how it stores the Merkle tree,
> +# which can be quite large.
> +# It is convenient to treat the Merkle tree as past EOF, and ext4, f2fs, and
> +# btrfs do so in at least some fashion. This leads to an edge case where a
> +# large file can be under the file system file size limit, but trigger EFBIG
> +# on enabling fs-verity. Test enabling verity on some large files to exercise
> +# EFBIG logic for filesystems with fs-verity specific limits.
> +#
> +. ./common/preamble
> +_begin_fstest auto quick verity
> +
> +
> +# Import common functions.
> +. ./common/filter
> +. ./common/verity
> +
> +# real QA test starts here
> +_supported_fs generic
> +_require_test
> +_require_math
> +_require_scratch_verity
> +_require_fsverity_max_file_size_limit
> +_require_scratch_nocheck

Why is _require_scratch_nocheck() needed?  _require_scratch_verity() already
does _require_scratch(), and I don't see why skipping fsck would be needed.

> +# have to go back by 4096 from max to not hit the fsverity MAX_LEVELS check.
> +truncate -s $max_sz $fsv_file

The above comment should be removed.

- Eric
Boris Burkov March 15, 2022, 10:06 p.m. UTC | #2
On Tue, Mar 15, 2022 at 02:16:30AM +0000, Eric Biggers wrote:
> On Mon, Feb 14, 2022 at 04:09:58PM -0800, Boris Burkov wrote:
> > diff --git a/tests/generic/690 b/tests/generic/690
> > new file mode 100755
> > index 00000000..77906dd8
> > --- /dev/null
> > +++ b/tests/generic/690
> > @@ -0,0 +1,66 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0
> > +# Copyright (c) 2021 Facebook, Inc.  All Rights Reserved.
> > +#
> > +# FS QA Test 690
> > +#
> > +# fs-verity requires the filesystem to decide how it stores the Merkle tree,
> > +# which can be quite large.
> > +# It is convenient to treat the Merkle tree as past EOF, and ext4, f2fs, and
> > +# btrfs do so in at least some fashion. This leads to an edge case where a
> > +# large file can be under the file system file size limit, but trigger EFBIG
> > +# on enabling fs-verity. Test enabling verity on some large files to exercise
> > +# EFBIG logic for filesystems with fs-verity specific limits.
> > +#
> > +. ./common/preamble
> > +_begin_fstest auto quick verity
> > +
> > +
> > +# Import common functions.
> > +. ./common/filter
> > +. ./common/verity
> > +
> > +# real QA test starts here
> > +_supported_fs generic
> > +_require_test
> > +_require_math
> > +_require_scratch_verity
> > +_require_fsverity_max_file_size_limit
> > +_require_scratch_nocheck
> 
> Why is _require_scratch_nocheck() needed?  _require_scratch_verity() already
> does _require_scratch(), and I don't see why skipping fsck would be needed.
> 
> > +# have to go back by 4096 from max to not hit the fsverity MAX_LEVELS check.
> > +truncate -s $max_sz $fsv_file
> 
> The above comment should be removed.
> 

Thanks for catching these oversights, will fix them. The nocheck thing
was left over from mixing/copy-pasting with btrfs/290 during this test's
evolution. Works fine without it on ext4 and btrfs.

> - Eric
diff mbox series

Patch

diff --git a/common/verity b/common/verity
index 07d9d3fe..8be86ef7 100644
--- a/common/verity
+++ b/common/verity
@@ -345,3 +345,14 @@  _fsv_scratch_corrupt_merkle_tree()
 		;;
 	esac
 }
+
+_require_fsverity_max_file_size_limit()
+{
+	case $FSTYP in
+	btrfs|ext4|f2fs)
+		;;
+	*)
+		_notrun "$FSTYP does not store verity data past EOF; no special file size limit"
+		;;
+	esac
+}
diff --git a/tests/generic/690 b/tests/generic/690
new file mode 100755
index 00000000..77906dd8
--- /dev/null
+++ b/tests/generic/690
@@ -0,0 +1,66 @@ 
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2021 Facebook, Inc.  All Rights Reserved.
+#
+# FS QA Test 690
+#
+# fs-verity requires the filesystem to decide how it stores the Merkle tree,
+# which can be quite large.
+# It is convenient to treat the Merkle tree as past EOF, and ext4, f2fs, and
+# btrfs do so in at least some fashion. This leads to an edge case where a
+# large file can be under the file system file size limit, but trigger EFBIG
+# on enabling fs-verity. Test enabling verity on some large files to exercise
+# EFBIG logic for filesystems with fs-verity specific limits.
+#
+. ./common/preamble
+_begin_fstest auto quick verity
+
+
+# Import common functions.
+. ./common/filter
+. ./common/verity
+
+# real QA test starts here
+_supported_fs generic
+_require_test
+_require_math
+_require_scratch_verity
+_require_fsverity_max_file_size_limit
+_require_scratch_nocheck
+
+_scratch_mkfs_verity &>> $seqres.full
+_scratch_mount
+
+fsv_file=$SCRATCH_MNT/file.fsv
+
+max_sz=$(_get_max_file_size)
+_fsv_scratch_begin_subtest "way too big: fail on first merkle block"
+# have to go back by 4096 from max to not hit the fsverity MAX_LEVELS check.
+truncate -s $max_sz $fsv_file
+_fsv_enable $fsv_file |& _filter_scratch
+
+# The goal of this second test is to make a big enough file that we trip the
+# EFBIG codepath, but not so big that we hit it immediately when writing the
+# first Merkle leaf.
+#
+# The Merkle tree is stored with the leaf node level (L0) last, but it is
+# written first.  To get an interesting overflow, we need the maximum file size
+# (MAX) to be in the middle of L0 -- ideally near the beginning of L0 so that we
+# don't have to write many blocks before getting an error.
+#
+# With SHA-256 and 4K blocks, there are 128 hashes per block.  Thus, ignoring
+# padding, L0 is 1/128 of the file size while the other levels in total are
+# 1/128**2 + 1/128**3 + 1/128**4 + ... = 1/16256 of the file size.  So still
+# ignoring padding, for L0 start exactly at MAX, the file size must be s such
+# that s + s/16256 = MAX, i.e. s = MAX * (16256/16257).  Then to get a file size
+# where MAX occurs *near* the start of L0 rather than *at* the start, we can
+# just subtract an overestimate of the padding: 64K after the file contents,
+# then 4K per level, where the consideration of 8 levels is sufficient.
+sz=$(echo "scale=20; $max_sz * (16256/16257) - 65536 - 4096*8" | $BC -q | cut -d. -f1)
+_fsv_scratch_begin_subtest "still too big: fail on first invalid merkle block"
+truncate -s $sz $fsv_file
+_fsv_enable $fsv_file |& _filter_scratch
+
+# success, all done
+status=0
+exit
diff --git a/tests/generic/690.out b/tests/generic/690.out
new file mode 100644
index 00000000..a3e2b9b9
--- /dev/null
+++ b/tests/generic/690.out
@@ -0,0 +1,7 @@ 
+QA output created by 690
+
+# way too big: fail on first merkle block
+ERROR: FS_IOC_ENABLE_VERITY failed on 'SCRATCH_MNT/file.fsv': File too large
+
+# still too big: fail on first invalid merkle block
+ERROR: FS_IOC_ENABLE_VERITY failed on 'SCRATCH_MNT/file.fsv': File too large