[v14,0/3] qcow2: Add list of bitmaps to ImageInfoSpecificQCow2

Message

Andrey Shinkevich Feb. 8, 2019, 3:06 p.m. UTC

v14:
The test 242 was amended to be safer against changes in the bitmap allocation
algorithm because the bitmap directory is reallocated on VM shutdown.
Unfortunately, an empty line at the end of the benchmark file 242.out persists.
The version #13 was discussed in email thread with the message ID:
<[e-mail]>

v13:
The enhancement of the Python code style for fopen() in the new test 242.
The 'unknown flag' was actually removed from the structure Qcow2BitmapInfo.

v12:
In the function block_crypto_get_specific_info_luks(),
checking the format was replaced with assertion.
The 'unknown flag' was removed from the structure Qcow2BitmapInfo.
A new case added to the test file 242 to check QEMU behavior in case of
unknown flag in a bitmap directory entry of QCOW2 image file.

v11:
An assertion was added to the get_bitmap_info_flags() to check the
completed mapping of all the reserved bitmap BME_ flags.
The heading comment of get_bitmap_info_flags() was changed to
describe the function design properly.
In qcow2_get_specific_info(), two function calls g_free() were
replaced with one call to qapi_free_ImageInfoSpecific() that does
all the cleaning work.
The version #11 was discussed in email thread with the message ID:
<[e-mail]>

v10:
The 'struct Error' parameter was added to the function prototype
bdrv_query_image_info().
The code refactoring of the function get_bitmap_info_flags().
The comments to the structures ImageInfoSpecificQCow2 and
Qcow2BitmapInfo in the file qapi/block-core.json were corrected.
The changes in the *.out files of the tests 060, 065 082, 198
and 206 were discarded. The new test 239 was enriched by adding human
readable format output and by checking the output with bitmap extra
parameters, such as  non-persistent and disabled.
The version #10 was discussed in email thread with the message ID:
<[e-mail]>

v9:
The new test 239 of the qemu-iotests set was amended to show the bitmaps
being added and to demonstrate the bitmap flag "in-use".
The version #9 was discussed with the message ID:
<[e-mail]>

v8:
The output benchmark files for the qemu-iotests, namely 060, 065 082, 198
and 206, were modified to show the bitmap extension for the qemu specific
information. A new test file 239 was added to the test set that checks the
output for the fields of the bitmap section.
The backward compatibility of the output for images of the version 2
of qcow2 was added.
The version #8 was discussed in email thread with the message ID:
<[e-mail]>

v7:
A description was added to the function qcow2_get_bitmap_info_list().
In the function qcow2_get_specific_info(), the comment was modified
so that we ignore any error in obtaining the list of bitmaps to
pass the rest of QCOW2 specific information to a caller.
The version #7 was discussed in email thread with the message ID:
<[e-mail]>

v6:
'[PATCH v6] qemu-img info lists bitmap directory entries'.
The error handling logic for the bitmaps empty list was reversed.

v5:
'[PATCH v5] qemu-img info lists bitmap directory entries'.
The error handling logic for the bitmaps empty list was fixed and documented.

v4:
'[PATCH v4] qemu-img info lists bitmap directory entries'.
Unknown flags are checked with the mask BME_RESERVED_FLAGS.
The code minor refactoring was made.

v3:
'[PATCH v3] qemu-img info lists bitmap directory entries'.
Now, qcow2_get_bitmap_info_list() is invoked under the condition of QCOW
version #3 to avoid memory leaks in case of QCOW version #2.
Furthermore, qcow2_get_bitmap_info_list() checks the number of existing bitmaps.
So, if no bitmap exists, no bitmap error message is printed in the output.
The data type of the bitmap 'granularity' parameter was left as 'uint32'
because bitmap_list_load() returns error if granularity_bits is grater than 31.

v2:
'[PATCH v2] qemu-img info lists bitmap directory entries'.
The targeted version of the release at 'Since' word of the comments to the new
structures changed to 4.0 in the file qapi/block-core.json.
A comment to the 'bitmaps' new member was supplied.
The 'unknown flags' parameter was introduced to indicate presence of QCOW2
bitmap unknown flags, if any.
The word 'dirty' was removed from the code and from the comments as we list all
the bitmaps.
The 'bitmaps' printed parameter was removed for the release versions earlier
than 3.x.
The example of the output was moved above the 'Signed-off-by' line.

The first version was '[PATCH] qemu-img info lists bitmap directory entries'.

Andrey Shinkevich (3):
  bdrv_query_image_info Error parameter added
  qcow2: Add list of bitmaps to ImageInfoSpecificQCow2
  qcow2: list of bitmaps new test 242

 block.c                    |   5 +-
 block/crypto.c             |   9 +--
 block/qapi.c               |   7 +-
 block/qcow2-bitmap.c       |  76 +++++++++++++++++++++
 block/qcow2.c              |  21 +++++-
 block/qcow2.h              |   2 +
 block/vmdk.c               |   3 +-
 include/block/block.h      |   3 +-
 include/block/block_int.h  |   3 +-
 qapi/block-core.json       |  41 ++++++++++-
 qemu-io-cmds.c             |   7 +-
 tests/qemu-iotests/242     | 103 ++++++++++++++++++++++++++++
 tests/qemu-iotests/242.out | 165 +++++++++++++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/group   |   1 +
 14 files changed, 429 insertions(+), 17 deletions(-)
 create mode 100755 tests/qemu-iotests/242
 create mode 100644 tests/qemu-iotests/242.out

Comments

Eric Blake Feb. 8, 2019, 3:25 p.m. UTC | #1

On 2/8/19 9:06 AM, Andrey Shinkevich wrote:
> v14:
> The test 242 was amended to be safer against changes in the bitmap allocation
> algorithm because the bitmap directory is reallocated on VM shutdown.
> Unfortunately, an empty line at the end of the benchmark file 242.out persists.

Easy solution: in 242, add an extra log("Test complete") or similar
after the output that produces the blank line, then 242.out will no
longer have a trailing blank line.

Andrey Shinkevich Feb. 8, 2019, 3:37 p.m. UTC | #2

On 08/02/2019 18:25, Eric Blake wrote:
> On 2/8/19 9:06 AM, Andrey Shinkevich wrote:
>> v14:
>> The test 242 was amended to be safer against changes in the bitmap allocation
>> algorithm because the bitmap directory is reallocated on VM shutdown.
>> Unfortunately, an empty line at the end of the benchmark file 242.out persists.
> 
> Easy solution: in 242, add an extra log("Test complete") or similar
> after the output that produces the blank line, then 242.out will no
> longer have a trailing blank line.
> 
Thank you, Eric. It is going to be version 15.

Vladimir Sementsov-Ogievskiy Feb. 8, 2019, 3:47 p.m. UTC | #3

08.02.2019 18:25, Eric Blake wrote:
> On 2/8/19 9:06 AM, Andrey Shinkevich wrote:
>> v14:
>> The test 242 was amended to be safer against changes in the bitmap allocation
>> algorithm because the bitmap directory is reallocated on VM shutdown.
>> Unfortunately, an empty line at the end of the benchmark file 242.out persists.
> 
> Easy solution: in 242, add an extra log("Test complete") or similar
> after the output that produces the blank line, then 242.out will no
> longer have a trailing blank line.
> 

with this fixed:
Reviewed-by: Vladimir Sementsov-Ogievskiy <[e-mail]>

Eric Blake Feb. 8, 2019, 3:53 p.m. UTC | #4

On 2/8/19 9:37 AM, Andrey Shinkevich wrote:
> 
> 
> On 08/02/2019 18:25, Eric Blake wrote:
>> On 2/8/19 9:06 AM, Andrey Shinkevich wrote:
>>> v14:
>>> The test 242 was amended to be safer against changes in the bitmap allocation
>>> algorithm because the bitmap directory is reallocated on VM shutdown.
>>> Unfortunately, an empty line at the end of the benchmark file 242.out persists.
>>
>> Easy solution: in 242, add an extra log("Test complete") or similar
>> after the output that produces the blank line, then 242.out will no
>> longer have a trailing blank line.
>>
> Thank you, Eric. It is going to be version 15.

If that's the only change, then no need to post v15; you can check what
I'm preparing to include in Monday's pull request:

https://repo.or.cz/qemu/ericb.git/shortlog/refs/heads/nbd

Andrey Shinkevich Feb. 8, 2019, 4:22 p.m. UTC | #5

On 08/02/2019 18:53, Eric Blake wrote:
> On 2/8/19 9:37 AM, Andrey Shinkevich wrote:
>>
>>
>> On 08/02/2019 18:25, Eric Blake wrote:
>>> On 2/8/19 9:06 AM, Andrey Shinkevich wrote:
>>>> v14:
>>>> The test 242 was amended to be safer against changes in the bitmap allocation
>>>> algorithm because the bitmap directory is reallocated on VM shutdown.
>>>> Unfortunately, an empty line at the end of the benchmark file 242.out persists.
>>>
>>> Easy solution: in 242, add an extra log("Test complete") or similar
>>> after the output that produces the blank line, then 242.out will no
>>> longer have a trailing blank line.
>>>
>> Thank you, Eric. It is going to be version 15.
> 
> If that's the only change, then no need to post v15; you can check what
> I'm preparing to include in Monday's pull request:
> 
> https://repo.or.cz/qemu/ericb.git/shortlog/refs/heads/nbd
> 
Thank you again.
With those minor amendments (+spaces) and a couple of the significant
"Reviewed-by", the version 15 has been prepared. May I send it
or you are OK with the version 14?

Andrey Shinkevich Feb. 8, 2019, 4:30 p.m. UTC | #6

On 08/02/2019 18:53, Eric Blake wrote:
> On 2/8/19 9:37 AM, Andrey Shinkevich wrote:
>>
>>
>> On 08/02/2019 18:25, Eric Blake wrote:
>>> On 2/8/19 9:06 AM, Andrey Shinkevich wrote:
>>>> v14:
>>>> The test 242 was amended to be safer against changes in the bitmap allocation
>>>> algorithm because the bitmap directory is reallocated on VM shutdown.
>>>> Unfortunately, an empty line at the end of the benchmark file 242.out persists.
>>>
>>> Easy solution: in 242, add an extra log("Test complete") or similar
>>> after the output that produces the blank line, then 242.out will no
>>> longer have a trailing blank line.
>>>
>> Thank you, Eric. It is going to be version 15.
> 
> If that's the only change, then no need to post v15; you can check what
> I'm preparing to include in Monday's pull request:
> 
> https://repo.or.cz/qemu/ericb.git/shortlog/refs/heads/nbd
> 
I followed the link above and saw, thank you.
The only difference with my version #15 is that I added the spaces
around '*' in the chunk assignment. That's it!