| Message ID | 20210705091549.178335-12-vsementsov@virtuozzo.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | iotests: support zstd | expand |
On 05.07.21 11:15, Vladimir Sementsov-Ogievskiy wrote: > We want iotests pass with both the default zlib compression and with > IMGOPTS='compression_type=zstd'. > > Actually the only test that is interested in real compression type in > test output is 287 (test for qcow2 compression type), so implement > specific option for it. > > Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> > --- > tests/qemu-iotests/060.out | 2 +- > tests/qemu-iotests/061.out | 12 ++++++------ > tests/qemu-iotests/082.out | 14 +++++++------- > tests/qemu-iotests/198.out | 4 ++-- > tests/qemu-iotests/287 | 8 ++++---- > tests/qemu-iotests/common.filter | 7 +++++++ > tests/qemu-iotests/common.rc | 14 +++++++++++++- > 7 files changed, 40 insertions(+), 21 deletions(-) [...] > diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter > index 268b749e2f..78efe3e4dd 100644 > --- a/tests/qemu-iotests/common.filter > +++ b/tests/qemu-iotests/common.filter > @@ -247,6 +247,7 @@ _filter_img_info() > -e "/block_state_zero: \\(on\\|off\\)/d" \ > -e "/log_size: [0-9]\\+/d" \ > -e "s/iters: [0-9]\\+/iters: 1024/" \ > + -e 's/\(compression type: \)\(zlib\|zstd\)/\1COMPRESSION_TYPE/' \ > -e "s/uuid: [-a-f0-9]\\+/uuid: 00000000-0000-0000-0000-000000000000/" | \ > while IFS='' read -r line; do > if [[ $format_specific == 1 ]]; then > @@ -332,5 +333,11 @@ for fname in fnames: > sys.stdout.write(result)' > } > > +_filter_qcow2_compression_type_bit() > +{ > + $SED -e 's/\(incompatible_features\s\+\)\[3\(, \)\?/\1[/' \ > + -e 's/\(incompatible_features.*\), 3\]/\1]/' What about “incompatble_features [2, 3, 4]”? I’d like to propose adding some form of filtering parameter to qcow2.py which allows filtering a specific bit from the qcow2_format.Flags64 representation, but that seems rather difficult, actually... > +} > + > # make sure this script returns success > true > diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc > index ee4b9d795e..813b51ee03 100644 > --- a/tests/qemu-iotests/common.rc > +++ b/tests/qemu-iotests/common.rc > @@ -697,6 +697,7 @@ _img_info() > -e "s#$TEST_DIR#TEST_DIR#g" \ > -e "s#$SOCK_DIR/fuse-#TEST_DIR/#g" \ > -e "s#$IMGFMT#IMGFMT#g" \ > + -e 's/\(compression type: \)\(zlib\|zstd\)/\1COMPRESSION_TYPE/' \ > -e "/^disk size:/ D" \ > -e "/actual-size/ D" | \ > while IFS='' read -r line; do > @@ -996,12 +997,23 @@ _require_one_device_of() > > _qcow2_dump_header() > { > + if [[ "$1" == "--no-filter-compression" ]]; then > + local filter_compression=0 > + shift > + else > + local filter_compression=1 > + fi > + > img="$1" > if [ -z "$img" ]; then > img="$TEST_IMG" > fi > > - $PYTHON qcow2.py "$img" dump-header > + if [[ $filter_compression == 0 ]]; then > + $PYTHON qcow2.py "$img" dump-header > + else > + $PYTHON qcow2.py "$img" dump-header | _filter_qcow2_compression_type_bit > + fi > } > > # make sure this script returns success Could have been done more extensibly for the future (i.e. a loop over the parameters, and a variable to invoke all applicable filters), but, well. Not much reason to think about a future that we’re not sure will ever happen. Max
16.07.2021 16:17, Max Reitz wrote: > On 05.07.21 11:15, Vladimir Sementsov-Ogievskiy wrote: >> We want iotests pass with both the default zlib compression and with >> IMGOPTS='compression_type=zstd'. >> >> Actually the only test that is interested in real compression type in >> test output is 287 (test for qcow2 compression type), so implement >> specific option for it. >> >> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> >> --- >> tests/qemu-iotests/060.out | 2 +- >> tests/qemu-iotests/061.out | 12 ++++++------ >> tests/qemu-iotests/082.out | 14 +++++++------- >> tests/qemu-iotests/198.out | 4 ++-- >> tests/qemu-iotests/287 | 8 ++++---- >> tests/qemu-iotests/common.filter | 7 +++++++ >> tests/qemu-iotests/common.rc | 14 +++++++++++++- >> 7 files changed, 40 insertions(+), 21 deletions(-) > > [...] > >> diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter >> index 268b749e2f..78efe3e4dd 100644 >> --- a/tests/qemu-iotests/common.filter >> +++ b/tests/qemu-iotests/common.filter >> @@ -247,6 +247,7 @@ _filter_img_info() >> -e "/block_state_zero: \\(on\\|off\\)/d" \ >> -e "/log_size: [0-9]\\+/d" \ >> -e "s/iters: [0-9]\\+/iters: 1024/" \ >> + -e 's/\(compression type: \)\(zlib\|zstd\)/\1COMPRESSION_TYPE/' \ >> -e "s/uuid: [-a-f0-9]\\+/uuid: 00000000-0000-0000-0000-000000000000/" | \ >> while IFS='' read -r line; do >> if [[ $format_specific == 1 ]]; then >> @@ -332,5 +333,11 @@ for fname in fnames: >> sys.stdout.write(result)' >> } >> +_filter_qcow2_compression_type_bit() >> +{ >> + $SED -e 's/\(incompatible_features\s\+\)\[3\(, \)\?/\1[/' \ >> + -e 's/\(incompatible_features.*\), 3\]/\1]/' > > What about “incompatble_features [2, 3, 4]”? Will add. > > I’d like to propose adding some form of filtering parameter to qcow2.py which allows filtering a specific bit from the qcow2_format.Flags64 representation, but that seems rather difficult, actually... > >> +} >> + >> # make sure this script returns success >> true >> diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc >> index ee4b9d795e..813b51ee03 100644 >> --- a/tests/qemu-iotests/common.rc >> +++ b/tests/qemu-iotests/common.rc >> @@ -697,6 +697,7 @@ _img_info() >> -e "s#$TEST_DIR#TEST_DIR#g" \ >> -e "s#$SOCK_DIR/fuse-#TEST_DIR/#g" \ >> -e "s#$IMGFMT#IMGFMT#g" \ >> + -e 's/\(compression type: \)\(zlib\|zstd\)/\1COMPRESSION_TYPE/' \ >> -e "/^disk size:/ D" \ >> -e "/actual-size/ D" | \ >> while IFS='' read -r line; do >> @@ -996,12 +997,23 @@ _require_one_device_of() >> _qcow2_dump_header() >> { >> + if [[ "$1" == "--no-filter-compression" ]]; then >> + local filter_compression=0 >> + shift >> + else >> + local filter_compression=1 >> + fi >> + >> img="$1" >> if [ -z "$img" ]; then >> img="$TEST_IMG" >> fi >> - $PYTHON qcow2.py "$img" dump-header >> + if [[ $filter_compression == 0 ]]; then >> + $PYTHON qcow2.py "$img" dump-header >> + else >> + $PYTHON qcow2.py "$img" dump-header | _filter_qcow2_compression_type_bit >> + fi >> } >> # make sure this script returns success > > Could have been done more extensibly for the future (i.e. a loop over the parameters, and a variable to invoke all applicable filters), but, well. Not much reason to think about a future that we’re not sure will ever happen. > > Max >
diff --git a/tests/qemu-iotests/060.out b/tests/qemu-iotests/060.out index b74540bafb..329977d9b9 100644 --- a/tests/qemu-iotests/060.out +++ b/tests/qemu-iotests/060.out @@ -17,7 +17,7 @@ virtual size: 64 MiB (67108864 bytes) cluster_size: 65536 Format specific information: compat: 1.1 - compression type: zlib + compression type: COMPRESSION_TYPE lazy refcounts: false refcount bits: 16 corrupt: true diff --git a/tests/qemu-iotests/061.out b/tests/qemu-iotests/061.out index ee30da2665..11b6404186 100644 --- a/tests/qemu-iotests/061.out +++ b/tests/qemu-iotests/061.out @@ -524,7 +524,7 @@ virtual size: 64 MiB (67108864 bytes) cluster_size: 65536 Format specific information: compat: 1.1 - compression type: zlib + compression type: COMPRESSION_TYPE lazy refcounts: false refcount bits: 16 data file: TEST_DIR/t.IMGFMT.data @@ -551,7 +551,7 @@ virtual size: 64 MiB (67108864 bytes) cluster_size: 65536 Format specific information: compat: 1.1 - compression type: zlib + compression type: COMPRESSION_TYPE lazy refcounts: false refcount bits: 16 data file: foo @@ -566,7 +566,7 @@ virtual size: 64 MiB (67108864 bytes) cluster_size: 65536 Format specific information: compat: 1.1 - compression type: zlib + compression type: COMPRESSION_TYPE lazy refcounts: false refcount bits: 16 data file raw: false @@ -582,7 +582,7 @@ virtual size: 64 MiB (67108864 bytes) cluster_size: 65536 Format specific information: compat: 1.1 - compression type: zlib + compression type: COMPRESSION_TYPE lazy refcounts: false refcount bits: 16 data file: TEST_DIR/t.IMGFMT.data @@ -596,7 +596,7 @@ virtual size: 64 MiB (67108864 bytes) cluster_size: 65536 Format specific information: compat: 1.1 - compression type: zlib + compression type: COMPRESSION_TYPE lazy refcounts: false refcount bits: 16 data file: TEST_DIR/t.IMGFMT.data @@ -611,7 +611,7 @@ virtual size: 64 MiB (67108864 bytes) cluster_size: 65536 Format specific information: compat: 1.1 - compression type: zlib + compression type: COMPRESSION_TYPE lazy refcounts: false refcount bits: 16 data file: TEST_DIR/t.IMGFMT.data diff --git a/tests/qemu-iotests/082.out b/tests/qemu-iotests/082.out index b70c12c139..f8e2e039fc 100644 --- a/tests/qemu-iotests/082.out +++ b/tests/qemu-iotests/082.out @@ -17,7 +17,7 @@ virtual size: 128 MiB (134217728 bytes) cluster_size: 4096 Format specific information: compat: 1.1 - compression type: zlib + compression type: COMPRESSION_TYPE lazy refcounts: true refcount bits: 16 corrupt: false @@ -31,7 +31,7 @@ virtual size: 128 MiB (134217728 bytes) cluster_size: 8192 Format specific information: compat: 1.1 - compression type: zlib + compression type: COMPRESSION_TYPE lazy refcounts: true refcount bits: 16 corrupt: false @@ -329,7 +329,7 @@ virtual size: 128 MiB (134217728 bytes) cluster_size: 4096 Format specific information: compat: 1.1 - compression type: zlib + compression type: COMPRESSION_TYPE lazy refcounts: true refcount bits: 16 corrupt: false @@ -342,7 +342,7 @@ virtual size: 128 MiB (134217728 bytes) cluster_size: 8192 Format specific information: compat: 1.1 - compression type: zlib + compression type: COMPRESSION_TYPE lazy refcounts: true refcount bits: 16 corrupt: false @@ -639,7 +639,7 @@ virtual size: 128 MiB (134217728 bytes) cluster_size: 65536 Format specific information: compat: 1.1 - compression type: zlib + compression type: COMPRESSION_TYPE lazy refcounts: true refcount bits: 16 corrupt: false @@ -652,7 +652,7 @@ virtual size: 130 MiB (136314880 bytes) cluster_size: 65536 Format specific information: compat: 1.1 - compression type: zlib + compression type: COMPRESSION_TYPE lazy refcounts: false refcount bits: 16 corrupt: false @@ -665,7 +665,7 @@ virtual size: 132 MiB (138412032 bytes) cluster_size: 65536 Format specific information: compat: 1.1 - compression type: zlib + compression type: COMPRESSION_TYPE lazy refcounts: true refcount bits: 16 corrupt: false diff --git a/tests/qemu-iotests/198.out b/tests/qemu-iotests/198.out index 3952708444..805494916f 100644 --- a/tests/qemu-iotests/198.out +++ b/tests/qemu-iotests/198.out @@ -36,7 +36,7 @@ image: json:{ /* filtered */ } file format: IMGFMT virtual size: 16 MiB (16777216 bytes) Format specific information: - compression type: zlib + compression type: COMPRESSION_TYPE encrypt: ivgen alg: plain64 hash alg: sha256 @@ -81,7 +81,7 @@ virtual size: 16 MiB (16777216 bytes) backing file: TEST_DIR/t.IMGFMT.base backing file format: IMGFMT Format specific information: - compression type: zlib + compression type: COMPRESSION_TYPE encrypt: ivgen alg: plain64 hash alg: sha256 diff --git a/tests/qemu-iotests/287 b/tests/qemu-iotests/287 index 6716419da4..aab03fb973 100755 --- a/tests/qemu-iotests/287 +++ b/tests/qemu-iotests/287 @@ -61,13 +61,13 @@ echo echo "=== Testing compression type incompatible bit setting for zlib ===" echo _make_test_img -o compression_type=zlib 64M -_qcow2_dump_header | grep incompatible_features +_qcow2_dump_header --no-filter-compression | grep incompatible_features echo echo "=== Testing compression type incompatible bit setting for zstd ===" echo _make_test_img -o compression_type=zstd 64M -_qcow2_dump_header | grep incompatible_features +_qcow2_dump_header --no-filter-compression | grep incompatible_features echo echo "=== Testing zlib with incompatible bit set ===" @@ -75,7 +75,7 @@ echo _make_test_img -o compression_type=zlib 64M $PYTHON qcow2.py "$TEST_IMG" set-feature-bit incompatible 3 # to make sure the bit was actually set -_qcow2_dump_header | grep incompatible_features +_qcow2_dump_header --no-filter-compression | grep incompatible_features if $QEMU_IMG info "$TEST_IMG" >/dev/null 2>&1 ; then echo "Error: The image opened successfully. The image must not be opened." @@ -87,7 +87,7 @@ echo _make_test_img -o compression_type=zstd 64M $PYTHON qcow2.py "$TEST_IMG" set-header incompatible_features 0 # to make sure the bit was actually unset -_qcow2_dump_header | grep incompatible_features +_qcow2_dump_header --no-filter-compression | grep incompatible_features if $QEMU_IMG info "$TEST_IMG" >/dev/null 2>&1 ; then echo "Error: The image opened successfully. The image must not be opened." diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter index 268b749e2f..78efe3e4dd 100644 --- a/tests/qemu-iotests/common.filter +++ b/tests/qemu-iotests/common.filter @@ -247,6 +247,7 @@ _filter_img_info() -e "/block_state_zero: \\(on\\|off\\)/d" \ -e "/log_size: [0-9]\\+/d" \ -e "s/iters: [0-9]\\+/iters: 1024/" \ + -e 's/\(compression type: \)\(zlib\|zstd\)/\1COMPRESSION_TYPE/' \ -e "s/uuid: [-a-f0-9]\\+/uuid: 00000000-0000-0000-0000-000000000000/" | \ while IFS='' read -r line; do if [[ $format_specific == 1 ]]; then @@ -332,5 +333,11 @@ for fname in fnames: sys.stdout.write(result)' } +_filter_qcow2_compression_type_bit() +{ + $SED -e 's/\(incompatible_features\s\+\)\[3\(, \)\?/\1[/' \ + -e 's/\(incompatible_features.*\), 3\]/\1]/' +} + # make sure this script returns success true diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc index ee4b9d795e..813b51ee03 100644 --- a/tests/qemu-iotests/common.rc +++ b/tests/qemu-iotests/common.rc @@ -697,6 +697,7 @@ _img_info() -e "s#$TEST_DIR#TEST_DIR#g" \ -e "s#$SOCK_DIR/fuse-#TEST_DIR/#g" \ -e "s#$IMGFMT#IMGFMT#g" \ + -e 's/\(compression type: \)\(zlib\|zstd\)/\1COMPRESSION_TYPE/' \ -e "/^disk size:/ D" \ -e "/actual-size/ D" | \ while IFS='' read -r line; do @@ -996,12 +997,23 @@ _require_one_device_of() _qcow2_dump_header() { + if [[ "$1" == "--no-filter-compression" ]]; then + local filter_compression=0 + shift + else + local filter_compression=1 + fi + img="$1" if [ -z "$img" ]; then img="$TEST_IMG" fi - $PYTHON qcow2.py "$img" dump-header + if [[ $filter_compression == 0 ]]; then + $PYTHON qcow2.py "$img" dump-header + else + $PYTHON qcow2.py "$img" dump-header | _filter_qcow2_compression_type_bit + fi } # make sure this script returns success
We want iotests pass with both the default zlib compression and with IMGOPTS='compression_type=zstd'. Actually the only test that is interested in real compression type in test output is 287 (test for qcow2 compression type), so implement specific option for it. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> --- tests/qemu-iotests/060.out | 2 +- tests/qemu-iotests/061.out | 12 ++++++------ tests/qemu-iotests/082.out | 14 +++++++------- tests/qemu-iotests/198.out | 4 ++-- tests/qemu-iotests/287 | 8 ++++---- tests/qemu-iotests/common.filter | 7 +++++++ tests/qemu-iotests/common.rc | 14 +++++++++++++- 7 files changed, 40 insertions(+), 21 deletions(-)