Message ID | 20230711172553.234055-4-andrey.drobyshev@virtuozzo.com (mailing list archive) |
---|---|
State | New, archived |
Headers | show |
Series | block: align CoR requests to subclusters | expand |
On 11.07.23 20:25, Andrey Drobyshev via wrote: > Add testcase which checks that allocations during copy-on-read are > performed on the subcluster basis when subclusters are enabled in target > image. > > This testcase also triggers the following assert with previous commit > not being applied, so we check that as well: > > qemu-io: ../block/io.c:1236: bdrv_co_do_copy_on_readv: Assertion `skip_bytes < pnum' failed. > > Reviewed-by: Eric Blake<eblake@redhat.com> > Reviewed-by: Denis V. Lunev<den@openvz.org> > Signed-off-by: Andrey Drobyshev<andrey.drobyshev@virtuozzo.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
On 7/11/23 19:25, Andrey Drobyshev wrote: > Add testcase which checks that allocations during copy-on-read are > performed on the subcluster basis when subclusters are enabled in target > image. > > This testcase also triggers the following assert with previous commit > not being applied, so we check that as well: > > qemu-io: ../block/io.c:1236: bdrv_co_do_copy_on_readv: Assertion `skip_bytes < pnum' failed. > > Reviewed-by: Eric Blake <eblake@redhat.com> > Reviewed-by: Denis V. Lunev <den@openvz.org> > Signed-off-by: Andrey Drobyshev <andrey.drobyshev@virtuozzo.com> > --- > tests/qemu-iotests/197 | 29 +++++++++++++++++++++++++++++ > tests/qemu-iotests/197.out | 24 ++++++++++++++++++++++++ > 2 files changed, 53 insertions(+) > > diff --git a/tests/qemu-iotests/197 b/tests/qemu-iotests/197 > index a2547bc280..f07a9da136 100755 > --- a/tests/qemu-iotests/197 > +++ b/tests/qemu-iotests/197 > @@ -122,6 +122,35 @@ $QEMU_IO -f qcow2 -C -c 'read 0 1024' "$TEST_WRAP" | _filter_qemu_io > $QEMU_IO -f qcow2 -c map "$TEST_WRAP" > _check_test_img > > +echo > +echo '=== Copy-on-read with subclusters ===' > +echo > + > +# Create base and top images 64K (1 cluster) each. Make subclusters enabled > +# for the top image > +_make_test_img 64K > +IMGPROTO=file IMGFMT=qcow2 TEST_IMG_FILE="$TEST_WRAP" \ > + _make_test_img --no-opts -o extended_l2=true -F "$IMGFMT" -b "$TEST_IMG" \ > + 64K | _filter_img_create > + > +$QEMU_IO -c "write -P 0xaa 0 64k" "$TEST_IMG" | _filter_qemu_io > + > +# Allocate individual subclusters in the top image, and not the whole cluster > +$QEMU_IO -c "write -P 0xbb 28K 2K" -c "write -P 0xcc 34K 2K" "$TEST_WRAP" \ > + | _filter_qemu_io > + > +# Only 2 subclusters should be allocated in the top image at this point > +$QEMU_IMG map "$TEST_WRAP" | _filter_qemu_img_map > + > +# Actual copy-on-read operation > +$QEMU_IO -C -c "read -P 0xaa 30K 4K" "$TEST_WRAP" | _filter_qemu_io > + > +# And here we should have 4 subclusters allocated right in the middle of the > +# top image. Make sure the whole cluster remains unallocated > +$QEMU_IMG map "$TEST_WRAP" | _filter_qemu_img_map > + > +_check_test_img > + > # success, all done > echo '*** done' > status=0 > diff --git a/tests/qemu-iotests/197.out b/tests/qemu-iotests/197.out > index ad414c3b0e..8f34a30afe 100644 > --- a/tests/qemu-iotests/197.out > +++ b/tests/qemu-iotests/197.out > @@ -31,4 +31,28 @@ read 1024/1024 bytes at offset 0 > 1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) > 1 KiB (0x400) bytes allocated at offset 0 bytes (0x0) > No errors were found on the image. > + > +=== Copy-on-read with subclusters === > + > +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=65536 > +Formatting 'TEST_DIR/t.wrap.IMGFMT', fmt=IMGFMT size=65536 backing_file=TEST_DIR/t.IMGFMT backing_fmt=IMGFMT > +wrote 65536/65536 bytes at offset 0 > +64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) > +wrote 2048/2048 bytes at offset 28672 > +2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) > +wrote 2048/2048 bytes at offset 34816 > +2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) > +Offset Length File > +0 0x7000 TEST_DIR/t.IMGFMT > +0x7000 0x800 TEST_DIR/t.wrap.IMGFMT > +0x7800 0x1000 TEST_DIR/t.IMGFMT > +0x8800 0x800 TEST_DIR/t.wrap.IMGFMT > +0x9000 0x7000 TEST_DIR/t.IMGFMT > +read 4096/4096 bytes at offset 30720 > +4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) > +Offset Length File > +0 0x7000 TEST_DIR/t.IMGFMT > +0x7000 0x2000 TEST_DIR/t.wrap.IMGFMT > +0x9000 0x7000 TEST_DIR/t.IMGFMT > +No errors were found on the image. > *** done It is revealed that this patch seems to break unit tests if run for NBD format iris ~/src/qemu/build/tests/qemu-iotests $ ./check -nbd 197 QEMU -- "/home/den/src/qemu/build/tests/qemu-iotests/../../qemu-system-x86_64" -nodefaults -display none -accel qtest QEMU_IMG -- "/home/den/src/qemu/build/tests/qemu-iotests/../../qemu-img" QEMU_IO -- "/home/den/src/qemu/build/tests/qemu-iotests/../../qemu-io" --cache writeback --aio threads -f raw QEMU_NBD -- "/home/den/src/qemu/build/tests/qemu-iotests/../../qemu-nbd" IMGFMT -- raw IMGPROTO -- nbd PLATFORM -- Linux/x86_64 iris 6.2.0-31-generic TEST_DIR -- /home/den/src/qemu/build/tests/qemu-iotests/scratch SOCK_DIR -- /tmp/tmpzw5ky8d3 GDB_OPTIONS -- VALGRIND_QEMU -- PRINT_QEMU_OUTPUT -- 197 fail [11:41:26] [11:41:31] 5.7s (last: 3.8s) output mismatch (see /home/den/src/qemu/build/tests/qemu-iotests/scratch/raw-nbd-197/197.out.bad) --- /home/den/src/qemu/tests/qemu-iotests/197.out +++ /home/den/src/qemu/build/tests/qemu-iotests/scratch/raw-nbd-197/197.out.bad @@ -43,16 +43,11 @@ wrote 2048/2048 bytes at offset 34816 2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) Offset Length File -0 0x7000 TEST_DIR/t.IMGFMT -0x7000 0x800 TEST_DIR/t.wrap.IMGFMT -0x7800 0x1000 TEST_DIR/t.IMGFMT -0x8800 0x800 TEST_DIR/t.wrap.IMGFMT -0x9000 0x7000 TEST_DIR/t.IMGFMT +0 0x10000 nbd+unix://?socket=SOCK_DIR/nbd +Pattern verification failed at offset 30720, 4096 bytes read 4096/4096 bytes at offset 30720 4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) Offset Length File -0 0x7000 TEST_DIR/t.IMGFMT -0x7000 0x2000 TEST_DIR/t.wrap.IMGFMT -0x9000 0x7000 TEST_DIR/t.IMGFMT +0 0x10000 nbd+unix://?socket=SOCK_DIR/nbd No errors were found on the image. *** done Failures: 197 Failed 1 of 1 iotests iris ~/src/qemu/build/tests/qemu-iotests $ it is good for QCOW2 format. iris ~/src/qemu/build/tests/qemu-iotests $ ./check -qcow2 197 QEMU -- "/home/den/src/qemu/build/tests/qemu-iotests/../../qemu-system-x86_64" -nodefaults -display none -accel qtest QEMU_IMG -- "/home/den/src/qemu/build/tests/qemu-iotests/../../qemu-img" QEMU_IO -- "/home/den/src/qemu/build/tests/qemu-iotests/../../qemu-io" --cache writeback --aio threads -f qcow2 QEMU_NBD -- "/home/den/src/qemu/build/tests/qemu-iotests/../../qemu-nbd" IMGFMT -- qcow2 IMGPROTO -- file PLATFORM -- Linux/x86_64 iris 6.2.0-31-generic TEST_DIR -- /home/den/src/qemu/build/tests/qemu-iotests/scratch SOCK_DIR -- /tmp/tmpe1qldhs9 GDB_OPTIONS -- VALGRIND_QEMU -- PRINT_QEMU_OUTPUT -- 197 pass [11:42:14] [11:42:17] 2.8s Passed all 1 iotests iris ~/src/qemu/build/tests/qemu-iotests $ Regards, Den
On 9/6/23 12:43, Denis V. Lunev wrote: > On 7/11/23 19:25, Andrey Drobyshev wrote: >> Add testcase which checks that allocations during copy-on-read are >> performed on the subcluster basis when subclusters are enabled in target >> image. >> >> This testcase also triggers the following assert with previous commit >> not being applied, so we check that as well: >> >> qemu-io: ../block/io.c:1236: bdrv_co_do_copy_on_readv: Assertion >> `skip_bytes < pnum' failed. >> >> Reviewed-by: Eric Blake <eblake@redhat.com> >> Reviewed-by: Denis V. Lunev <den@openvz.org> >> Signed-off-by: Andrey Drobyshev <andrey.drobyshev@virtuozzo.com> >> --- >> tests/qemu-iotests/197 | 29 +++++++++++++++++++++++++++++ >> tests/qemu-iotests/197.out | 24 ++++++++++++++++++++++++ >> 2 files changed, 53 insertions(+) >> >> diff --git a/tests/qemu-iotests/197 b/tests/qemu-iotests/197 >> index a2547bc280..f07a9da136 100755 >> --- a/tests/qemu-iotests/197 >> +++ b/tests/qemu-iotests/197 >> @@ -122,6 +122,35 @@ $QEMU_IO -f qcow2 -C -c 'read 0 1024' >> "$TEST_WRAP" | _filter_qemu_io >> $QEMU_IO -f qcow2 -c map "$TEST_WRAP" >> _check_test_img >> +echo >> +echo '=== Copy-on-read with subclusters ===' >> +echo >> + >> +# Create base and top images 64K (1 cluster) each. Make subclusters >> enabled >> +# for the top image >> +_make_test_img 64K >> +IMGPROTO=file IMGFMT=qcow2 TEST_IMG_FILE="$TEST_WRAP" \ >> + _make_test_img --no-opts -o extended_l2=true -F "$IMGFMT" -b >> "$TEST_IMG" \ >> + 64K | _filter_img_create >> + >> +$QEMU_IO -c "write -P 0xaa 0 64k" "$TEST_IMG" | _filter_qemu_io >> + >> +# Allocate individual subclusters in the top image, and not the whole >> cluster >> +$QEMU_IO -c "write -P 0xbb 28K 2K" -c "write -P 0xcc 34K 2K" >> "$TEST_WRAP" \ >> + | _filter_qemu_io >> + >> +# Only 2 subclusters should be allocated in the top image at this point >> +$QEMU_IMG map "$TEST_WRAP" | _filter_qemu_img_map >> + >> +# Actual copy-on-read operation >> +$QEMU_IO -C -c "read -P 0xaa 30K 4K" "$TEST_WRAP" | _filter_qemu_io >> + >> +# And here we should have 4 subclusters allocated right in the middle >> of the >> +# top image. Make sure the whole cluster remains unallocated >> +$QEMU_IMG map "$TEST_WRAP" | _filter_qemu_img_map >> + >> +_check_test_img >> + >> # success, all done >> echo '*** done' >> status=0 >> diff --git a/tests/qemu-iotests/197.out b/tests/qemu-iotests/197.out >> index ad414c3b0e..8f34a30afe 100644 >> --- a/tests/qemu-iotests/197.out >> +++ b/tests/qemu-iotests/197.out >> @@ -31,4 +31,28 @@ read 1024/1024 bytes at offset 0 >> 1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) >> 1 KiB (0x400) bytes allocated at offset 0 bytes (0x0) >> No errors were found on the image. >> + >> +=== Copy-on-read with subclusters === >> + >> +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=65536 >> +Formatting 'TEST_DIR/t.wrap.IMGFMT', fmt=IMGFMT size=65536 >> backing_file=TEST_DIR/t.IMGFMT backing_fmt=IMGFMT >> +wrote 65536/65536 bytes at offset 0 >> +64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) >> +wrote 2048/2048 bytes at offset 28672 >> +2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) >> +wrote 2048/2048 bytes at offset 34816 >> +2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) >> +Offset Length File >> +0 0x7000 TEST_DIR/t.IMGFMT >> +0x7000 0x800 TEST_DIR/t.wrap.IMGFMT >> +0x7800 0x1000 TEST_DIR/t.IMGFMT >> +0x8800 0x800 TEST_DIR/t.wrap.IMGFMT >> +0x9000 0x7000 TEST_DIR/t.IMGFMT >> +read 4096/4096 bytes at offset 30720 >> +4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) >> +Offset Length File >> +0 0x7000 TEST_DIR/t.IMGFMT >> +0x7000 0x2000 TEST_DIR/t.wrap.IMGFMT >> +0x9000 0x7000 TEST_DIR/t.IMGFMT >> +No errors were found on the image. >> *** done > It is revealed that this patch seems to break unit tests if run for NBD > format > > iris ~/src/qemu/build/tests/qemu-iotests $ ./check -nbd 197 > QEMU -- > "/home/den/src/qemu/build/tests/qemu-iotests/../../qemu-system-x86_64" > -nodefaults -display none -accel qtest > QEMU_IMG -- > "/home/den/src/qemu/build/tests/qemu-iotests/../../qemu-img" > QEMU_IO -- > "/home/den/src/qemu/build/tests/qemu-iotests/../../qemu-io" --cache > writeback --aio threads -f raw > QEMU_NBD -- > "/home/den/src/qemu/build/tests/qemu-iotests/../../qemu-nbd" > IMGFMT -- raw > IMGPROTO -- nbd > PLATFORM -- Linux/x86_64 iris 6.2.0-31-generic > TEST_DIR -- /home/den/src/qemu/build/tests/qemu-iotests/scratch > SOCK_DIR -- /tmp/tmpzw5ky8d3 > GDB_OPTIONS -- > VALGRIND_QEMU -- > PRINT_QEMU_OUTPUT -- > > 197 fail [11:41:26] [11:41:31] 5.7s (last: 3.8s) output > mismatch (see > /home/den/src/qemu/build/tests/qemu-iotests/scratch/raw-nbd-197/197.out.bad) > --- /home/den/src/qemu/tests/qemu-iotests/197.out > +++ > /home/den/src/qemu/build/tests/qemu-iotests/scratch/raw-nbd-197/197.out.bad > @@ -43,16 +43,11 @@ > wrote 2048/2048 bytes at offset 34816 > 2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) > Offset Length File > -0 0x7000 TEST_DIR/t.IMGFMT > -0x7000 0x800 TEST_DIR/t.wrap.IMGFMT > -0x7800 0x1000 TEST_DIR/t.IMGFMT > -0x8800 0x800 TEST_DIR/t.wrap.IMGFMT > -0x9000 0x7000 TEST_DIR/t.IMGFMT > +0 0x10000 nbd+unix://?socket=SOCK_DIR/nbd > +Pattern verification failed at offset 30720, 4096 bytes > read 4096/4096 bytes at offset 30720 > 4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) > Offset Length File > -0 0x7000 TEST_DIR/t.IMGFMT > -0x7000 0x2000 TEST_DIR/t.wrap.IMGFMT > -0x9000 0x7000 TEST_DIR/t.IMGFMT > +0 0x10000 nbd+unix://?socket=SOCK_DIR/nbd > No errors were found on the image. > *** done > Failures: 197 > Failed 1 of 1 iotests > iris ~/src/qemu/build/tests/qemu-iotests $ > > it is good for QCOW2 format. > > iris ~/src/qemu/build/tests/qemu-iotests $ ./check -qcow2 197 > QEMU -- > "/home/den/src/qemu/build/tests/qemu-iotests/../../qemu-system-x86_64" > -nodefaults -display none -accel qtest > QEMU_IMG -- > "/home/den/src/qemu/build/tests/qemu-iotests/../../qemu-img" > QEMU_IO -- > "/home/den/src/qemu/build/tests/qemu-iotests/../../qemu-io" --cache > writeback --aio threads -f qcow2 > QEMU_NBD -- > "/home/den/src/qemu/build/tests/qemu-iotests/../../qemu-nbd" > IMGFMT -- qcow2 > IMGPROTO -- file > PLATFORM -- Linux/x86_64 iris 6.2.0-31-generic > TEST_DIR -- /home/den/src/qemu/build/tests/qemu-iotests/scratch > SOCK_DIR -- /tmp/tmpe1qldhs9 > GDB_OPTIONS -- > VALGRIND_QEMU -- > PRINT_QEMU_OUTPUT -- > > 197 pass [11:42:14] [11:42:17] 2.8s > Passed all 1 iotests > iris ~/src/qemu/build/tests/qemu-iotests $ > > Regards, > Den Thanks for pointing out, apparently I didn't realize that this test should accept ANY format as a backing file. I've sent a fixup as patch 4/3. Andrey
diff --git a/tests/qemu-iotests/197 b/tests/qemu-iotests/197 index a2547bc280..f07a9da136 100755 --- a/tests/qemu-iotests/197 +++ b/tests/qemu-iotests/197 @@ -122,6 +122,35 @@ $QEMU_IO -f qcow2 -C -c 'read 0 1024' "$TEST_WRAP" | _filter_qemu_io $QEMU_IO -f qcow2 -c map "$TEST_WRAP" _check_test_img +echo +echo '=== Copy-on-read with subclusters ===' +echo + +# Create base and top images 64K (1 cluster) each. Make subclusters enabled +# for the top image +_make_test_img 64K +IMGPROTO=file IMGFMT=qcow2 TEST_IMG_FILE="$TEST_WRAP" \ + _make_test_img --no-opts -o extended_l2=true -F "$IMGFMT" -b "$TEST_IMG" \ + 64K | _filter_img_create + +$QEMU_IO -c "write -P 0xaa 0 64k" "$TEST_IMG" | _filter_qemu_io + +# Allocate individual subclusters in the top image, and not the whole cluster +$QEMU_IO -c "write -P 0xbb 28K 2K" -c "write -P 0xcc 34K 2K" "$TEST_WRAP" \ + | _filter_qemu_io + +# Only 2 subclusters should be allocated in the top image at this point +$QEMU_IMG map "$TEST_WRAP" | _filter_qemu_img_map + +# Actual copy-on-read operation +$QEMU_IO -C -c "read -P 0xaa 30K 4K" "$TEST_WRAP" | _filter_qemu_io + +# And here we should have 4 subclusters allocated right in the middle of the +# top image. Make sure the whole cluster remains unallocated +$QEMU_IMG map "$TEST_WRAP" | _filter_qemu_img_map + +_check_test_img + # success, all done echo '*** done' status=0 diff --git a/tests/qemu-iotests/197.out b/tests/qemu-iotests/197.out index ad414c3b0e..8f34a30afe 100644 --- a/tests/qemu-iotests/197.out +++ b/tests/qemu-iotests/197.out @@ -31,4 +31,28 @@ read 1024/1024 bytes at offset 0 1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) 1 KiB (0x400) bytes allocated at offset 0 bytes (0x0) No errors were found on the image. + +=== Copy-on-read with subclusters === + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=65536 +Formatting 'TEST_DIR/t.wrap.IMGFMT', fmt=IMGFMT size=65536 backing_file=TEST_DIR/t.IMGFMT backing_fmt=IMGFMT +wrote 65536/65536 bytes at offset 0 +64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 2048/2048 bytes at offset 28672 +2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 2048/2048 bytes at offset 34816 +2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +Offset Length File +0 0x7000 TEST_DIR/t.IMGFMT +0x7000 0x800 TEST_DIR/t.wrap.IMGFMT +0x7800 0x1000 TEST_DIR/t.IMGFMT +0x8800 0x800 TEST_DIR/t.wrap.IMGFMT +0x9000 0x7000 TEST_DIR/t.IMGFMT +read 4096/4096 bytes at offset 30720 +4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +Offset Length File +0 0x7000 TEST_DIR/t.IMGFMT +0x7000 0x2000 TEST_DIR/t.wrap.IMGFMT +0x9000 0x7000 TEST_DIR/t.IMGFMT +No errors were found on the image. *** done