| Message ID | 20191024055435.6059-1-cgxu519@mykernel.net (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | [v2] overlay/066: copy-up test for variant sparse files | expand |
On Thu, Oct 24, 2019 at 8:54 AM Chengguang Xu <cgxu519@mykernel.net> wrote: > > This is intensive copy-up test for sparse files, > these cases will be mainly used for regression test > of copy-up improvement for sparse files. > > Signed-off-by: Chengguang Xu <cgxu519@mykernel.net> You can add: Reviewed-by: Amir Goldstein <amir73il@gmail.com> after fixing nits below... > --- > v1->v2: > - Call _get_block_size to get fs block size. > - Add comment for test space requirement. > - Print meaningful error message when copy-up fail. > - Adjust random hole range to 1M~5M. > - Fix typo. > > tests/overlay/066 | 120 ++++++++++++++++++++++++++++++++++++++++++ > tests/overlay/066.out | 2 + > tests/overlay/group | 1 + > 3 files changed, 123 insertions(+) > create mode 100755 tests/overlay/066 > create mode 100644 tests/overlay/066.out > > diff --git a/tests/overlay/066 b/tests/overlay/066 > new file mode 100755 > index 00000000..b01fc2a4 > --- /dev/null > +++ b/tests/overlay/066 > @@ -0,0 +1,120 @@ > +#! /bin/bash > +# SPDX-License-Identifier: GPL-2.0 > +# Copyright (c) 2019 Chengguang Xu <cgxu519@mykernel.net> > +# All Rights Reserved. > +# > +# FS QA Test 066 > +# > +# Test overlayfs copy-up function for variant sparse files. > +# > +seq=`basename $0` > +seqres=$RESULT_DIR/$seq > +echo "QA output created by $seq" > + > +here=`pwd` > +tmp=/tmp/$$ > +status=1 # failure is the default! > +trap "_cleanup; exit \$status" 0 1 2 3 15 > + > +_cleanup() > +{ > + cd / > + rm -f $tmp.* > +} > + > +# get standard environment, filters and checks > +. ./common/rc > +. ./common/filter > + > +# remove previous $seqres.full before test > +rm -f $seqres.full > + > +# real QA test starts here > + > +# Modify as appropriate. > +_supported_fs generic > +_supported_os Linux > +_require_test > +_require_scratch > + > +# Remove all files from previous tests > +_scratch_mkfs > +# We have totally 14 test files in this test, > +# one file for 100M and 13 files for 10M. So we need double that amount of space if both upper and lower do not support holes. Also it is not obvious to the reader how the number 13 came to be. It is better to explicitly say 1 empty file + 2^0 .. 2^11 hole size file, so if ever the test pattern changes, it will be easier to adapt this formula. > +_require_fs_space $OVL_BASE_SCRATCH_MNT $((10*1024*13 + 100*1024*1)) > + > +lowerdir=$OVL_BASE_SCRATCH_MNT/$OVL_LOWER > +upperdir=$OVL_BASE_SCRATCH_MNT/$OVL_UPPER > +testfile="copyup_sparse_test" > +mkdir -p $lowerdir > + > +# Create a completely empty hole file. > +$XFS_IO_PROG -fc "truncate 10M" "${lowerdir}/${testfile}_empty_holefile" \ > + >>$seqres.full > + > +iosize=$(_get_block_size "${lowerdir}") > +if [ $iosize -le 1024 ]; then > + iosize=1 > +else > + iosize=`expr $iosize / 1024` > +fi > + > +# Create test files with different hole size patterns. Better be more verbose in the comment about the pattern of the files, so reader won't need to follow code to understand. Also better to have constants to make the algorithm clearer: max_iosize=2048 file_size=10240 max_pos=`expr $file_size - $max_iosize` > +while [ $iosize -le 2048 ]; do > + pos=$iosize > + $XFS_IO_PROG -fc "truncate 10M" \ > + "${lowerdir}/${testfile}_iosize${iosize}K_holefile" >>$seqres.full > + while [ $pos -lt 8192 ]; do > + $XFS_IO_PROG -fc "pwrite ${pos}K ${iosize}K" \ > + "${lowerdir}/${testfile}_iosize${iosize}K_holefile" >>$seqres.full > + pos=`expr $pos + $iosize + $iosize` > + done > + iosize=`expr $iosize + $iosize` > +done > + > +# Create test file with many random holes(1M~5M). > +$XFS_IO_PROG -fc "truncate 100M" "${lowerdir}/${testfile}_random_holefile" \ > + >>$seqres.full Same comment about using constants instead of unexplained numbers, including: min_hole=1024 max_hole=5120 > +pos=2048 > +while [ $pos -le 81920 ]; do > + iosize=`expr $RANDOM % 5120` > + if [ $iosize -lt 1024 ]; then > + iosize=`expr $iosize + 1024` > + fi So that is a weird way to express random 1MB~5MB it results in 2 times higher probability for holes in the range 1MB~2MB and I don't think that was intentional. Better use: iosize=$(($RANDOM % ($max_hole - $min_hole) + $min_hole)) Thanks, Amir.
diff --git a/tests/overlay/066 b/tests/overlay/066 new file mode 100755 index 00000000..b01fc2a4 --- /dev/null +++ b/tests/overlay/066 @@ -0,0 +1,120 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2019 Chengguang Xu <cgxu519@mykernel.net> +# All Rights Reserved. +# +# FS QA Test 066 +# +# Test overlayfs copy-up function for variant sparse files. +# +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* +} + +# get standard environment, filters and checks +. ./common/rc +. ./common/filter + +# remove previous $seqres.full before test +rm -f $seqres.full + +# real QA test starts here + +# Modify as appropriate. +_supported_fs generic +_supported_os Linux +_require_test +_require_scratch + +# Remove all files from previous tests +_scratch_mkfs +# We have totally 14 test files in this test, +# one file for 100M and 13 files for 10M. +_require_fs_space $OVL_BASE_SCRATCH_MNT $((10*1024*13 + 100*1024*1)) + +lowerdir=$OVL_BASE_SCRATCH_MNT/$OVL_LOWER +upperdir=$OVL_BASE_SCRATCH_MNT/$OVL_UPPER +testfile="copyup_sparse_test" +mkdir -p $lowerdir + +# Create a completely empty hole file. +$XFS_IO_PROG -fc "truncate 10M" "${lowerdir}/${testfile}_empty_holefile" \ + >>$seqres.full + +iosize=$(_get_block_size "${lowerdir}") +if [ $iosize -le 1024 ]; then + iosize=1 +else + iosize=`expr $iosize / 1024` +fi + +# Create test files with different hole size patterns. +while [ $iosize -le 2048 ]; do + pos=$iosize + $XFS_IO_PROG -fc "truncate 10M" \ + "${lowerdir}/${testfile}_iosize${iosize}K_holefile" >>$seqres.full + while [ $pos -lt 8192 ]; do + $XFS_IO_PROG -fc "pwrite ${pos}K ${iosize}K" \ + "${lowerdir}/${testfile}_iosize${iosize}K_holefile" >>$seqres.full + pos=`expr $pos + $iosize + $iosize` + done + iosize=`expr $iosize + $iosize` +done + +# Create test file with many random holes(1M~5M). +$XFS_IO_PROG -fc "truncate 100M" "${lowerdir}/${testfile}_random_holefile" \ + >>$seqres.full +pos=2048 +while [ $pos -le 81920 ]; do + iosize=`expr $RANDOM % 5120` + if [ $iosize -lt 1024 ]; then + iosize=`expr $iosize + 1024` + fi + $XFS_IO_PROG -fc "pwrite ${pos}K ${iosize}K" \ + "${lowerdir}/${testfile}_random_holefile" >>$seqres.full + pos=`expr $pos + $iosize + $iosize` +done + +_scratch_mount + +# Open the files should succeed, no errors are expected. +for f in $SCRATCH_MNT/*; do + $XFS_IO_PROG -c "open" $f >>$seqres.full +done + +echo "Silence is golden" + +# Check all copy-up files in upper layer. +iosize=$(_get_block_size "${lowerdir}") +if [ $iosize -le 1024 ]; then + iosize=1 +else + iosize=`expr $iosize / 1024` +fi + +while [ $iosize -le 2048 ]; do + diff "${lowerdir}/${testfile}_iosize${iosize}K_holefile" \ + "${upperdir}/${testfile}_iosize${iosize}K_holefile" >>$seqres.full ||\ + echo "${upperdir}/${testfile}_iosize${iosize}K_holefile" copy up failed! + iosize=`expr $iosize + $iosize` +done + +diff "${lowerdir}/${testfile}_empty_holefile" "${upperdir}/${testfile}_empty_holefile" \ + >>$seqres.full || echo "${upperdir}/${testfile}_empty_holefile" copy up failed! +diff "${lowerdir}/${testfile}_random_holefile" "${upperdir}/${testfile}_random_holefile" \ + >>$seqres.full || echo "${upperdir}/${testfile}_random_holefile" copy up failed! + +# success, all done +status=0 +exit diff --git a/tests/overlay/066.out b/tests/overlay/066.out new file mode 100644 index 00000000..b60cc24c --- /dev/null +++ b/tests/overlay/066.out @@ -0,0 +1,2 @@ +QA output created by 066 +Silence is golden diff --git a/tests/overlay/group b/tests/overlay/group index ef8517a1..1dec7db9 100644 --- a/tests/overlay/group +++ b/tests/overlay/group @@ -68,3 +68,4 @@ 063 auto quick whiteout 064 auto quick copyup 065 auto quick mount +066 auto quick copyup
This is intensive copy-up test for sparse files, these cases will be mainly used for regression test of copy-up improvement for sparse files. Signed-off-by: Chengguang Xu <cgxu519@mykernel.net> --- v1->v2: - Call _get_block_size to get fs block size. - Add comment for test space requirement. - Print meaningful error message when copy-up fail. - Adjust random hole range to 1M~5M. - Fix typo. tests/overlay/066 | 120 ++++++++++++++++++++++++++++++++++++++++++ tests/overlay/066.out | 2 + tests/overlay/group | 1 + 3 files changed, 123 insertions(+) create mode 100755 tests/overlay/066 create mode 100644 tests/overlay/066.out