diff mbox series

[v11,2/3] qemu-img info lists bitmap directory entries

Message ID 1548942405-760115-3-git-send-email-andrey.shinkevich@virtuozzo.com (mailing list archive)
State New, archived
Headers show
Series qemu-img info lists bitmap directory entries | expand

Commit Message

Andrey Shinkevich Jan. 31, 2019, 1:46 p.m. UTC
In the 'Format specific information' section of the 'qemu-img info'
command output, the supplemental information about existing QCOW2
bitmaps will be shown, such as a bitmap name, flags and granularity:

image: /vz/vmprivate/VM1/harddisk.hdd
file format: qcow2
virtual size: 64G (68719476736 bytes)
disk size: 3.0M
cluster_size: 1048576
Format specific information:
    compat: 1.1
    lazy refcounts: true
    bitmaps:
        [0]:
            flags:
                [0]: in-use
                [1]: auto
            name: back-up1
            unknown flags: 4
            granularity: 65536
        [1]:
            flags:
                [0]: in-use
                [1]: auto
            name: back-up2
            unknown flags: 8
            granularity: 65536
    refcount bits: 16
    corrupt: false

Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
---
 block/qcow2-bitmap.c | 78 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 block/qcow2.c        | 11 +++++++-
 block/qcow2.h        |  2 ++
 qapi/block-core.json | 40 ++++++++++++++++++++++++++-
 4 files changed, 129 insertions(+), 2 deletions(-)

Comments

Vladimir Sementsov-Ogievskiy Feb. 1, 2019, 3:19 p.m. UTC | #1
31.01.2019 16:46, Andrey Shinkevich wrote:
> In the 'Format specific information' section of the 'qemu-img info'
> command output, the supplemental information about existing QCOW2
> bitmaps will be shown, such as a bitmap name, flags and granularity:
> 
> image: /vz/vmprivate/VM1/harddisk.hdd
> file format: qcow2
> virtual size: 64G (68719476736 bytes)
> disk size: 3.0M
> cluster_size: 1048576
> Format specific information:
>      compat: 1.1
>      lazy refcounts: true
>      bitmaps:
>          [0]:
>              flags:
>                  [0]: in-use
>                  [1]: auto
>              name: back-up1
>              unknown flags: 4
>              granularity: 65536
>          [1]:
>              flags:
>                  [0]: in-use
>                  [1]: auto
>              name: back-up2
>              unknown flags: 8
>              granularity: 65536
>      refcount bits: 16
>      corrupt: false
> 
> Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>

Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Eric Blake Feb. 1, 2019, 5:08 p.m. UTC | #2
On 1/31/19 7:46 AM, Andrey Shinkevich wrote:
> In the 'Format specific information' section of the 'qemu-img info'
> command output, the supplemental information about existing QCOW2
> bitmaps will be shown, such as a bitmap name, flags and granularity:
> 

> +
> +static Qcow2BitmapInfoFlagsList *get_bitmap_info_flags(uint32_t flags)
> +{
> +    Qcow2BitmapInfoFlagsList *list = NULL;
> +    Qcow2BitmapInfoFlagsList **plist = &list;
> +    int i;
> +
> +    static const struct {
> +        int bme;  /* Bitmap directory entry flags */
> +        int info; /* The flags to report to the user */
> +    } map[] = {
> +        { BME_FLAG_IN_USE, QCOW2_BITMAP_INFO_FLAGS_IN_USE },
> +        { BME_FLAG_AUTO,   QCOW2_BITMAP_INFO_FLAGS_AUTO },
> +    };
> +
> +    int map_size = sizeof(map) / sizeof(map[0]);

We have the ARRAY_SIZE() macro for this.

> +
> +    for (i = 0; i < map_size; ++i) {
> +        if (flags & map[i].bme) {
> +            Qcow2BitmapInfoFlagsList *entry =
> +                g_new0(Qcow2BitmapInfoFlagsList, 1);
> +            entry->value = map[i].info;
> +            *plist = entry;
> +            plist = &entry->next;
> +            flags &= ~map[i].bme;
> +        }
> +    }
> +    /* Check if the BME_* mapping above is complete */
> +    assert(!flags);
> +

> +        info->flags = get_bitmap_info_flags(bm->flags & ~BME_RESERVED_FLAGS);
> +        info->unknown_flags = bm->flags & BME_RESERVED_FLAGS;
> +        info->has_unknown_flags = !!info->unknown_flags;

Glad you liked my idea for better sanity checking.

If that's the only change needed, I don't mind making it as part of
staging for a pull request (of course, review comments on 3/3 may still
result in a v12 for other reasons).

Reviewed-by: Eric Blake <eblake@redhat.com>
Kevin Wolf Feb. 1, 2019, 5:19 p.m. UTC | #3
Am 31.01.2019 um 14:46 hat Andrey Shinkevich geschrieben:
> In the 'Format specific information' section of the 'qemu-img info'
> command output, the supplemental information about existing QCOW2
> bitmaps will be shown, such as a bitmap name, flags and granularity:
> 
> image: /vz/vmprivate/VM1/harddisk.hdd
> file format: qcow2
> virtual size: 64G (68719476736 bytes)
> disk size: 3.0M
> cluster_size: 1048576
> Format specific information:
>     compat: 1.1
>     lazy refcounts: true
>     bitmaps:
>         [0]:
>             flags:
>                 [0]: in-use
>                 [1]: auto
>             name: back-up1
>             unknown flags: 4
>             granularity: 65536
>         [1]:
>             flags:
>                 [0]: in-use
>                 [1]: auto
>             name: back-up2
>             unknown flags: 8
>             granularity: 65536
>     refcount bits: 16
>     corrupt: false
> 
> Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>

I haven't reviewed the patch in detail, but I suggest that you change
the subject line to make clear this is a qcow2 patch, not a qemu-img
one. Maybe:

    "qcow2: Add list of bitmaps to ImageInfoSpecificQCow2"

Kevin
Markus Armbruster Feb. 1, 2019, 6:39 p.m. UTC | #4
Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> writes:

> In the 'Format specific information' section of the 'qemu-img info'
> command output, the supplemental information about existing QCOW2
> bitmaps will be shown, such as a bitmap name, flags and granularity:
>
> image: /vz/vmprivate/VM1/harddisk.hdd
> file format: qcow2
> virtual size: 64G (68719476736 bytes)
> disk size: 3.0M
> cluster_size: 1048576
> Format specific information:
>     compat: 1.1
>     lazy refcounts: true
>     bitmaps:
>         [0]:
>             flags:
>                 [0]: in-use
>                 [1]: auto
>             name: back-up1
>             unknown flags: 4
>             granularity: 65536
>         [1]:
>             flags:
>                 [0]: in-use
>                 [1]: auto
>             name: back-up2
>             unknown flags: 8
>             granularity: 65536
>     refcount bits: 16
>     corrupt: false
>
> Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
> ---
[...]
> diff --git a/qapi/block-core.json b/qapi/block-core.json
> index 91685be..271e0df 100644
> --- a/qapi/block-core.json
> +++ b/qapi/block-core.json
> @@ -69,6 +69,8 @@
>  # @encrypt: details about encryption parameters; only set if image
>  #           is encrypted (since 2.10)
>  #
> +# @bitmaps: A list of qcow2 bitmap details (since 4.0)
> +#
>  # Since: 1.7
>  ##
>  { 'struct': 'ImageInfoSpecificQCow2',
> @@ -77,7 +79,8 @@
>        '*lazy-refcounts': 'bool',
>        '*corrupt': 'bool',
>        'refcount-bits': 'int',
> -      '*encrypt': 'ImageInfoSpecificQCow2Encryption'
> +      '*encrypt': 'ImageInfoSpecificQCow2Encryption',
> +      '*bitmaps': ['Qcow2BitmapInfo']
>    } }
>  
>  ##
> @@ -454,6 +457,41 @@
>             'status': 'DirtyBitmapStatus'} }
>  
>  ##
> +# @Qcow2BitmapInfoFlags:
> +#
> +# An enumeration of flags that a bitmap can report to the user.
> +#
> +# @in-use: The bitmap was not saved correctly and may be inconsistent.

I doubt the casual reader could guess the meaning from the name.  What
about @dirty?

> +#
> +# @auto: The bitmap must reflect all changes of the virtual disk by any
> +#        application that would write to this qcow2 file.
> +#
> +# Since: 4.0
> +##
> +{ 'enum': 'Qcow2BitmapInfoFlags',
> +  'data': ['in-use', 'auto'] }
> +
> +##
> +# @Qcow2BitmapInfo:
> +#
> +# Qcow2 bitmap information.
> +#
> +# @name: the name of the bitmap
> +#
> +# @granularity: granularity of the bitmap in bytes
> +#
> +# @flags: recognized flags of the bitmap
> +#
> +# @unknown-flags: any remaining flags not recognized by the current qemu version

Intended use cases for @unknown-flags?

> +#
> +# Since: 4.0
> +##
> +{ 'struct': 'Qcow2BitmapInfo',
> +  'data': {'name': 'str', 'granularity': 'uint32',
> +           'flags': ['Qcow2BitmapInfoFlags'],
> +           '*unknown-flags': 'uint32' } }
> +
> +##
>  # @BlockLatencyHistogramInfo:
>  #
>  # Block latency histogram.
Eric Blake Feb. 1, 2019, 7:04 p.m. UTC | #5
On 2/1/19 12:39 PM, Markus Armbruster wrote:
> Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> writes:
> 

>>  
>>  ##
>> +# @Qcow2BitmapInfoFlags:
>> +#
>> +# An enumeration of flags that a bitmap can report to the user.
>> +#
>> +# @in-use: The bitmap was not saved correctly and may be inconsistent.
> 
> I doubt the casual reader could guess the meaning from the name.  What
> about @dirty?

I like it.  The existing name was chosen to match
docs/interop/qcow2.txt, which uses in_use, but I don't see a problem in
making the UI nicer than the specs (and/or rewording the specs, as the
field name doesn't matter there, only the layout).


>> +# @unknown-flags: any remaining flags not recognized by the current qemu version
> 
> Intended use cases for @unknown-flags?

The qcow2 spec defines bit 2 extra_data_compatible; and also leaves the
door open for future extensions that may define other bits. If a new
version of qemu (or some non-qemu qcow2 creation app) creates an image
with additional feature bits set, THIS version of qemu doesn't know what
name to give those bits, but can still inform the user that those bits
are set via this field. It will be omitted for all images created by
this version of qemu.
Markus Armbruster Feb. 1, 2019, 7:23 p.m. UTC | #6
Eric Blake <eblake@redhat.com> writes:

> On 2/1/19 12:39 PM, Markus Armbruster wrote:
>> Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> writes:
>> 
>
>>>  
>>>  ##
>>> +# @Qcow2BitmapInfoFlags:
>>> +#
>>> +# An enumeration of flags that a bitmap can report to the user.
>>> +#
>>> +# @in-use: The bitmap was not saved correctly and may be inconsistent.
>> 
>> I doubt the casual reader could guess the meaning from the name.  What
>> about @dirty?
>
> I like it.  The existing name was chosen to match
> docs/interop/qcow2.txt, which uses in_use, but I don't see a problem in
> making the UI nicer than the specs (and/or rewording the specs, as the
> field name doesn't matter there, only the layout).
>
>
>>> +# @unknown-flags: any remaining flags not recognized by the current qemu version
>> 
>> Intended use cases for @unknown-flags?
>
> The qcow2 spec defines bit 2 extra_data_compatible; and also leaves the
> door open for future extensions that may define other bits. If a new
> version of qemu (or some non-qemu qcow2 creation app) creates an image
> with additional feature bits set, THIS version of qemu doesn't know what
> name to give those bits, but can still inform the user that those bits
> are set via this field. It will be omitted for all images created by
> this version of qemu.

What would QMP clients do with this information?
Eric Blake Feb. 1, 2019, 7:28 p.m. UTC | #7
On 2/1/19 1:23 PM, Markus Armbruster wrote:

>>>> +# @unknown-flags: any remaining flags not recognized by the current qemu version
>>>
>>> Intended use cases for @unknown-flags?
>>
>> The qcow2 spec defines bit 2 extra_data_compatible; and also leaves the
>> door open for future extensions that may define other bits. If a new
>> version of qemu (or some non-qemu qcow2 creation app) creates an image
>> with additional feature bits set, THIS version of qemu doesn't know what
>> name to give those bits, but can still inform the user that those bits
>> are set via this field. It will be omitted for all images created by
>> this version of qemu.
> 
> What would QMP clients do with this information?

'qemu-img info' is the intended QMP client; and it will print the
unknown-flags along with everything else. In other words, we're trying
to make qemu-img become useful for inspecting as much useful information
as possible from an image with unknown origins.  Knowing that an image
uses bitmaps with flags unknown to THIS version of qemu is a good
indication that you should be careful about using/altering the image
without first upgrading qemu, as it may destroy data that some other
product desires to utilize.
Vladimir Sementsov-Ogievskiy Feb. 4, 2019, 7:49 a.m. UTC | #8
01.02.2019 21:39, Markus Armbruster wrote:
> Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> writes:
> 
>> In the 'Format specific information' section of the 'qemu-img info'
>> command output, the supplemental information about existing QCOW2
>> bitmaps will be shown, such as a bitmap name, flags and granularity:
>>
>> image: /vz/vmprivate/VM1/harddisk.hdd
>> file format: qcow2
>> virtual size: 64G (68719476736 bytes)
>> disk size: 3.0M
>> cluster_size: 1048576
>> Format specific information:
>>      compat: 1.1
>>      lazy refcounts: true
>>      bitmaps:
>>          [0]:
>>              flags:
>>                  [0]: in-use
>>                  [1]: auto
>>              name: back-up1
>>              unknown flags: 4
>>              granularity: 65536
>>          [1]:
>>              flags:
>>                  [0]: in-use
>>                  [1]: auto
>>              name: back-up2
>>              unknown flags: 8
>>              granularity: 65536
>>      refcount bits: 16
>>      corrupt: false
>>
>> Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
>> ---
> [...]
>> diff --git a/qapi/block-core.json b/qapi/block-core.json
>> index 91685be..271e0df 100644
>> --- a/qapi/block-core.json
>> +++ b/qapi/block-core.json
>> @@ -69,6 +69,8 @@
>>   # @encrypt: details about encryption parameters; only set if image
>>   #           is encrypted (since 2.10)
>>   #
>> +# @bitmaps: A list of qcow2 bitmap details (since 4.0)
>> +#
>>   # Since: 1.7
>>   ##
>>   { 'struct': 'ImageInfoSpecificQCow2',
>> @@ -77,7 +79,8 @@
>>         '*lazy-refcounts': 'bool',
>>         '*corrupt': 'bool',
>>         'refcount-bits': 'int',
>> -      '*encrypt': 'ImageInfoSpecificQCow2Encryption'
>> +      '*encrypt': 'ImageInfoSpecificQCow2Encryption',
>> +      '*bitmaps': ['Qcow2BitmapInfo']
>>     } }
>>   
>>   ##
>> @@ -454,6 +457,41 @@
>>              'status': 'DirtyBitmapStatus'} }
>>   
>>   ##
>> +# @Qcow2BitmapInfoFlags:
>> +#
>> +# An enumeration of flags that a bitmap can report to the user.
>> +#
>> +# @in-use: The bitmap was not saved correctly and may be inconsistent.
> 
> I doubt the casual reader could guess the meaning from the name.  What
> about @dirty?

I'd prefer to keep it matching Qcow2 format specification. This Info aims
to export qcow2 specifics as is. And Qcow2 spec is not something qemu-internal,
why is it worse than QQPI, and why to have different name for the same thing
in different parts of spec?

We may instead rephrase it like

The bitmap is possibly in use by other process or was not saved correctly (and
therefore this flag was not cleared). The data of the bitmap may be inconsistent.

to reflect in-use name of the flag..


And, if we change it to @dirty, we at least should add a comment that this field
matches IN_USE flag of qcow2 format.

> 
>> +#
>> +# @auto: The bitmap must reflect all changes of the virtual disk by any
>> +#        application that would write to this qcow2 file.
>> +#
>> +# Since: 4.0
>> +##
>> +{ 'enum': 'Qcow2BitmapInfoFlags',
>> +  'data': ['in-use', 'auto'] }
>> +
>> +##
>> +# @Qcow2BitmapInfo:
>> +#
>> +# Qcow2 bitmap information.
>> +#
>> +# @name: the name of the bitmap
>> +#
>> +# @granularity: granularity of the bitmap in bytes
>> +#
>> +# @flags: recognized flags of the bitmap
>> +#
>> +# @unknown-flags: any remaining flags not recognized by the current qemu version
> 
> Intended use cases for @unknown-flags?
> 
>> +#
>> +# Since: 4.0
>> +##
>> +{ 'struct': 'Qcow2BitmapInfo',
>> +  'data': {'name': 'str', 'granularity': 'uint32',
>> +           'flags': ['Qcow2BitmapInfoFlags'],
>> +           '*unknown-flags': 'uint32' } }
>> +
>> +##
>>   # @BlockLatencyHistogramInfo:
>>   #
>>   # Block latency histogram.
Kevin Wolf Feb. 4, 2019, 9:46 a.m. UTC | #9
Am 01.02.2019 um 19:39 hat Markus Armbruster geschrieben:
> Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> writes:
> 
> > In the 'Format specific information' section of the 'qemu-img info'
> > command output, the supplemental information about existing QCOW2
> > bitmaps will be shown, such as a bitmap name, flags and granularity:
> >
> > image: /vz/vmprivate/VM1/harddisk.hdd
> > file format: qcow2
> > virtual size: 64G (68719476736 bytes)
> > disk size: 3.0M
> > cluster_size: 1048576
> > Format specific information:
> >     compat: 1.1
> >     lazy refcounts: true
> >     bitmaps:
> >         [0]:
> >             flags:
> >                 [0]: in-use
> >                 [1]: auto
> >             name: back-up1
> >             unknown flags: 4
> >             granularity: 65536
> >         [1]:
> >             flags:
> >                 [0]: in-use
> >                 [1]: auto
> >             name: back-up2
> >             unknown flags: 8
> >             granularity: 65536
> >     refcount bits: 16
> >     corrupt: false
> >
> > Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
> > ---
> [...]
> > diff --git a/qapi/block-core.json b/qapi/block-core.json
> > index 91685be..271e0df 100644
> > --- a/qapi/block-core.json
> > +++ b/qapi/block-core.json
> > @@ -69,6 +69,8 @@
> >  # @encrypt: details about encryption parameters; only set if image
> >  #           is encrypted (since 2.10)
> >  #
> > +# @bitmaps: A list of qcow2 bitmap details (since 4.0)
> > +#
> >  # Since: 1.7
> >  ##
> >  { 'struct': 'ImageInfoSpecificQCow2',
> > @@ -77,7 +79,8 @@
> >        '*lazy-refcounts': 'bool',
> >        '*corrupt': 'bool',
> >        'refcount-bits': 'int',
> > -      '*encrypt': 'ImageInfoSpecificQCow2Encryption'
> > +      '*encrypt': 'ImageInfoSpecificQCow2Encryption',
> > +      '*bitmaps': ['Qcow2BitmapInfo']
> >    } }
> >  
> >  ##
> > @@ -454,6 +457,41 @@
> >             'status': 'DirtyBitmapStatus'} }
> >  
> >  ##
> > +# @Qcow2BitmapInfoFlags:
> > +#
> > +# An enumeration of flags that a bitmap can report to the user.
> > +#
> > +# @in-use: The bitmap was not saved correctly and may be inconsistent.
> 
> I doubt the casual reader could guess the meaning from the name.  What
> about @dirty?

Feels like overloading the word "dirty" in the context of "dirty
bitmaps". This might easily lead to misinterpretation as "this bitmap
marks some clusters as dirty" rather than "this bitmap might be
inconsistent".

Kevin
Markus Armbruster Feb. 4, 2019, 1:05 p.m. UTC | #10
Eric Blake <eblake@redhat.com> writes:

> On 2/1/19 1:23 PM, Markus Armbruster wrote:
>
>>>>> +# @unknown-flags: any remaining flags not recognized by the current qemu version
>>>>
>>>> Intended use cases for @unknown-flags?
>>>
>>> The qcow2 spec defines bit 2 extra_data_compatible; and also leaves the
>>> door open for future extensions that may define other bits. If a new
>>> version of qemu (or some non-qemu qcow2 creation app) creates an image
>>> with additional feature bits set, THIS version of qemu doesn't know what
>>> name to give those bits, but can still inform the user that those bits
>>> are set via this field. It will be omitted for all images created by
>>> this version of qemu.
>> 
>> What would QMP clients do with this information?
>
> 'qemu-img info' is the intended QMP client; and it will print the
> unknown-flags along with everything else. In other words, we're trying
> to make qemu-img become useful for inspecting as much useful information
> as possible from an image with unknown origins.  Knowing that an image
> uses bitmaps with flags unknown to THIS version of qemu is a good
> indication that you should be careful about using/altering the image
> without first upgrading qemu, as it may destroy data that some other
> product desires to utilize.

Okay.  Now work that into the documentation :)
Markus Armbruster Feb. 4, 2019, 1:45 p.m. UTC | #11
Kevin Wolf <kwolf@redhat.com> writes:

> Am 01.02.2019 um 19:39 hat Markus Armbruster geschrieben:
>> Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> writes:
>> 
>> > In the 'Format specific information' section of the 'qemu-img info'
>> > command output, the supplemental information about existing QCOW2
>> > bitmaps will be shown, such as a bitmap name, flags and granularity:
>> >
>> > image: /vz/vmprivate/VM1/harddisk.hdd
>> > file format: qcow2
>> > virtual size: 64G (68719476736 bytes)
>> > disk size: 3.0M
>> > cluster_size: 1048576
>> > Format specific information:
>> >     compat: 1.1
>> >     lazy refcounts: true
>> >     bitmaps:
>> >         [0]:
>> >             flags:
>> >                 [0]: in-use
>> >                 [1]: auto
>> >             name: back-up1
>> >             unknown flags: 4
>> >             granularity: 65536
>> >         [1]:
>> >             flags:
>> >                 [0]: in-use
>> >                 [1]: auto
>> >             name: back-up2
>> >             unknown flags: 8
>> >             granularity: 65536
>> >     refcount bits: 16
>> >     corrupt: false
>> >
>> > Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
>> > ---
>> [...]
>> > diff --git a/qapi/block-core.json b/qapi/block-core.json
>> > index 91685be..271e0df 100644
>> > --- a/qapi/block-core.json
>> > +++ b/qapi/block-core.json
>> > @@ -69,6 +69,8 @@
>> >  # @encrypt: details about encryption parameters; only set if image
>> >  #           is encrypted (since 2.10)
>> >  #
>> > +# @bitmaps: A list of qcow2 bitmap details (since 4.0)
>> > +#
>> >  # Since: 1.7
>> >  ##
>> >  { 'struct': 'ImageInfoSpecificQCow2',
>> > @@ -77,7 +79,8 @@
>> >        '*lazy-refcounts': 'bool',
>> >        '*corrupt': 'bool',
>> >        'refcount-bits': 'int',
>> > -      '*encrypt': 'ImageInfoSpecificQCow2Encryption'
>> > +      '*encrypt': 'ImageInfoSpecificQCow2Encryption',
>> > +      '*bitmaps': ['Qcow2BitmapInfo']
>> >    } }
>> >  
>> >  ##
>> > @@ -454,6 +457,41 @@
>> >             'status': 'DirtyBitmapStatus'} }
>> >  
>> >  ##
>> > +# @Qcow2BitmapInfoFlags:
>> > +#
>> > +# An enumeration of flags that a bitmap can report to the user.
>> > +#
>> > +# @in-use: The bitmap was not saved correctly and may be inconsistent.
>> 
>> I doubt the casual reader could guess the meaning from the name.  What
>> about @dirty?
>
> Feels like overloading the word "dirty" in the context of "dirty
> bitmaps". This might easily lead to misinterpretation as "this bitmap
> marks some clusters as dirty" rather than "this bitmap might be
> inconsistent".

Call it @possibly-inconsistent then?
Eric Blake Feb. 4, 2019, 3:23 p.m. UTC | #12
On 2/4/19 1:49 AM, Vladimir Sementsov-Ogievskiy wrote:

>>> +++ b/qapi/block-core.json

>>>   ##
>>> +# @Qcow2BitmapInfoFlags:
>>> +#
>>> +# An enumeration of flags that a bitmap can report to the user.
>>> +#
>>> +# @in-use: The bitmap was not saved correctly and may be inconsistent.
>>
>> I doubt the casual reader could guess the meaning from the name.  What
>> about @dirty?
> 
> I'd prefer to keep it matching Qcow2 format specification. This Info aims
> to export qcow2 specifics as is. And Qcow2 spec is not something qemu-internal,

Indeed, but we CAN reword the qcow2 spec to pick more useful names, if
the existing names there are confusing.  (The name in the spec does not
matter, so much as the layout described by the name)

> why is it worse than QQPI, and why to have different name for the same thing
> in different parts of spec?
> 
> We may instead rephrase it like
> 
> The bitmap is possibly in use by other process or was not saved correctly (and
> therefore this flag was not cleared). The data of the bitmap may be inconsistent.

s/other/another/

but that rewording works for me.
Vladimir Sementsov-Ogievskiy Feb. 4, 2019, 3:36 p.m. UTC | #13
04.02.2019 16:45, Markus Armbruster wrote:
> Kevin Wolf <kwolf@redhat.com> writes:
> 
>> Am 01.02.2019 um 19:39 hat Markus Armbruster geschrieben:
>>> Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> writes:
>>>
>>>> In the 'Format specific information' section of the 'qemu-img info'
>>>> command output, the supplemental information about existing QCOW2
>>>> bitmaps will be shown, such as a bitmap name, flags and granularity:
>>>>
>>>> image: /vz/vmprivate/VM1/harddisk.hdd
>>>> file format: qcow2
>>>> virtual size: 64G (68719476736 bytes)
>>>> disk size: 3.0M
>>>> cluster_size: 1048576
>>>> Format specific information:
>>>>      compat: 1.1
>>>>      lazy refcounts: true
>>>>      bitmaps:
>>>>          [0]:
>>>>              flags:
>>>>                  [0]: in-use
>>>>                  [1]: auto
>>>>              name: back-up1
>>>>              unknown flags: 4
>>>>              granularity: 65536
>>>>          [1]:
>>>>              flags:
>>>>                  [0]: in-use
>>>>                  [1]: auto
>>>>              name: back-up2
>>>>              unknown flags: 8
>>>>              granularity: 65536
>>>>      refcount bits: 16
>>>>      corrupt: false
>>>>
>>>> Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
>>>> ---
>>> [...]
>>>> diff --git a/qapi/block-core.json b/qapi/block-core.json
>>>> index 91685be..271e0df 100644
>>>> --- a/qapi/block-core.json
>>>> +++ b/qapi/block-core.json
>>>> @@ -69,6 +69,8 @@
>>>>   # @encrypt: details about encryption parameters; only set if image
>>>>   #           is encrypted (since 2.10)
>>>>   #
>>>> +# @bitmaps: A list of qcow2 bitmap details (since 4.0)
>>>> +#
>>>>   # Since: 1.7
>>>>   ##
>>>>   { 'struct': 'ImageInfoSpecificQCow2',
>>>> @@ -77,7 +79,8 @@
>>>>         '*lazy-refcounts': 'bool',
>>>>         '*corrupt': 'bool',
>>>>         'refcount-bits': 'int',
>>>> -      '*encrypt': 'ImageInfoSpecificQCow2Encryption'
>>>> +      '*encrypt': 'ImageInfoSpecificQCow2Encryption',
>>>> +      '*bitmaps': ['Qcow2BitmapInfo']
>>>>     } }
>>>>   
>>>>   ##
>>>> @@ -454,6 +457,41 @@
>>>>              'status': 'DirtyBitmapStatus'} }
>>>>   
>>>>   ##
>>>> +# @Qcow2BitmapInfoFlags:
>>>> +#
>>>> +# An enumeration of flags that a bitmap can report to the user.
>>>> +#
>>>> +# @in-use: The bitmap was not saved correctly and may be inconsistent.
>>>
>>> I doubt the casual reader could guess the meaning from the name.  What
>>> about @dirty?
>>
>> Feels like overloading the word "dirty" in the context of "dirty
>> bitmaps". This might easily lead to misinterpretation as "this bitmap
>> marks some clusters as dirty" rather than "this bitmap might be
>> inconsistent".
> 
> Call it @possibly-inconsistent then?
> 

If you run qmp query-block while vm is running, all bitmaps in image will
be in use. And in this context, in-use seems more appropriate than
possibly-inconsistent. And in-use has less interference with the fact that
actual bitmaps now are in-RAM BdrvDirtyBitmap structures which are actually
consistent (and also shown by query-block)
Vladimir Sementsov-Ogievskiy Feb. 4, 2019, 4:03 p.m. UTC | #14
04.02.2019 16:05, Markus Armbruster wrote:
> Eric Blake <eblake@redhat.com> writes:
> 
>> On 2/1/19 1:23 PM, Markus Armbruster wrote:
>>
>>>>>> +# @unknown-flags: any remaining flags not recognized by the current qemu version
>>>>>
>>>>> Intended use cases for @unknown-flags?
>>>>
>>>> The qcow2 spec defines bit 2 extra_data_compatible; and also leaves the
>>>> door open for future extensions that may define other bits. If a new
>>>> version of qemu (or some non-qemu qcow2 creation app) creates an image
>>>> with additional feature bits set, THIS version of qemu doesn't know what
>>>> name to give those bits, but can still inform the user that those bits
>>>> are set via this field. It will be omitted for all images created by
>>>> this version of qemu.
>>>
>>> What would QMP clients do with this information?
>>
>> 'qemu-img info' is the intended QMP client; and it will print the
>> unknown-flags along with everything else. In other words, we're trying
>> to make qemu-img become useful for inspecting as much useful information
>> as possible from an image with unknown origins.  Knowing that an image
>> uses bitmaps with flags unknown to THIS version of qemu is a good
>> indication that you should be careful about using/altering the image
>> without first upgrading qemu, as it may destroy data that some other
>> product desires to utilize.
> 
> Okay.  Now work that into the documentation :)
> 

I'll try:

@unknown-flags: any flags except 'in-use' and 'auto' set for the bitmap.
                 Presence of non-zero @unknown-flags definitely shows that
                 image was created by newer version of software (which
                 supports more bitmap flags) or corrupted. Image can't be
                 used for r-w with non-zero @unknown-flags in any bitmap.
Eric Blake Feb. 4, 2019, 4:24 p.m. UTC | #15
On 2/4/19 10:03 AM, Vladimir Sementsov-Ogievskiy wrote:

>>>> What would QMP clients do with this information?
>>>
>>> 'qemu-img info' is the intended QMP client; and it will print the
>>> unknown-flags along with everything else. In other words, we're trying
>>> to make qemu-img become useful for inspecting as much useful information
>>> as possible from an image with unknown origins.  Knowing that an image
>>> uses bitmaps with flags unknown to THIS version of qemu is a good
>>> indication that you should be careful about using/altering the image
>>> without first upgrading qemu, as it may destroy data that some other
>>> product desires to utilize.
>>
>> Okay.  Now work that into the documentation :)
>>
> 
> I'll try:
> 
> @unknown-flags: any flags except 'in-use' and 'auto' set for the bitmap.
>                  Presence of non-zero @unknown-flags definitely shows that
>                  image was created by newer version of software (which
>                  supports more bitmap flags) or corrupted. Image can't be
>                  used for r-w with non-zero @unknown-flags in any bitmap.

The grammar still feels awkward.  Maybe:

@unknown-flags: If present, this is the value of additional bitmap flags
that were set in the image but not reported in @flags, implying that the
image was created with a newer version of software (that supports new
flags) or has been corrupted. Modifying the image without upgrading qemu
first will likely break the contents of the bitmap.
Vladimir Sementsov-Ogievskiy Feb. 4, 2019, 4:35 p.m. UTC | #16
04.02.2019 19:24, Eric Blake wrote:
> On 2/4/19 10:03 AM, Vladimir Sementsov-Ogievskiy wrote:
> 
>>>>> What would QMP clients do with this information?
>>>>
>>>> 'qemu-img info' is the intended QMP client; and it will print the
>>>> unknown-flags along with everything else. In other words, we're trying
>>>> to make qemu-img become useful for inspecting as much useful information
>>>> as possible from an image with unknown origins.  Knowing that an image
>>>> uses bitmaps with flags unknown to THIS version of qemu is a good
>>>> indication that you should be careful about using/altering the image
>>>> without first upgrading qemu, as it may destroy data that some other
>>>> product desires to utilize.
>>>
>>> Okay.  Now work that into the documentation :)
>>>
>>
>> I'll try:
>>
>> @unknown-flags: any flags except 'in-use' and 'auto' set for the bitmap.
>>                   Presence of non-zero @unknown-flags definitely shows that
>>                   image was created by newer version of software (which
>>                   supports more bitmap flags) or corrupted. Image can't be
>>                   used for r-w with non-zero @unknown-flags in any bitmap.
> 
> The grammar still feels awkward.  Maybe:
> 
> @unknown-flags: If present, this is the value of additional bitmap flags
> that were set in the image but not reported in @flags, implying that the
> image was created with a newer version of software (that supports new
> flags) or has been corrupted. Modifying the image without upgrading qemu
> first will likely break the contents of the bitmap.
> 

Thank you!

Oops, hm, I now doubt, how Andrey's patches work, if bitmap_list_load function
fails when unknown flags are found. So, can we really show unknown-flags,
or we'll fail if there any?

And if we fail (I hope we fail) on unknown flags, than we don't need this
field :).
Vladimir Sementsov-Ogievskiy Feb. 4, 2019, 4:46 p.m. UTC | #17
04.02.2019 19:35, Vladimir Sementsov-Ogievskiy wrote:
> 04.02.2019 19:24, Eric Blake wrote:
>> On 2/4/19 10:03 AM, Vladimir Sementsov-Ogievskiy wrote:
>>
>>>>>> What would QMP clients do with this information?
>>>>>
>>>>> 'qemu-img info' is the intended QMP client; and it will print the
>>>>> unknown-flags along with everything else. In other words, we're trying
>>>>> to make qemu-img become useful for inspecting as much useful information
>>>>> as possible from an image with unknown origins.  Knowing that an image
>>>>> uses bitmaps with flags unknown to THIS version of qemu is a good
>>>>> indication that you should be careful about using/altering the image
>>>>> without first upgrading qemu, as it may destroy data that some other
>>>>> product desires to utilize.
>>>>
>>>> Okay.  Now work that into the documentation :)
>>>>
>>>
>>> I'll try:
>>>
>>> @unknown-flags: any flags except 'in-use' and 'auto' set for the bitmap.
>>>                   Presence of non-zero @unknown-flags definitely shows that
>>>                   image was created by newer version of software (which
>>>                   supports more bitmap flags) or corrupted. Image can't be
>>>                   used for r-w with non-zero @unknown-flags in any bitmap.
>>
>> The grammar still feels awkward.  Maybe:
>>
>> @unknown-flags: If present, this is the value of additional bitmap flags
>> that were set in the image but not reported in @flags, implying that the
>> image was created with a newer version of software (that supports new
>> flags) or has been corrupted. Modifying the image without upgrading qemu
>> first will likely break the contents of the bitmap.
>>
> 
> Thank you!
> 
> Oops, hm, I now doubt, how Andrey's patches work, if bitmap_list_load function
> fails when unknown flags are found. So, can we really show unknown-flags,
> or we'll fail if there any?
> 
> And if we fail (I hope we fail) on unknown flags, than we don't need this
> field :).
> 

But we can update bitmap_list_load, to have a flag, and show bitmaps with
unknown flags in qemu-img info.. And we have to somehow not fail on open before
it. I'm very sorry that I understand it all only now :(.

So, do we really need unknown-flags field? If yes, may be, do it as a next step?
And proceed now without them?
Eric Blake Feb. 4, 2019, 5:33 p.m. UTC | #18
On 2/4/19 10:46 AM, Vladimir Sementsov-Ogievskiy wrote:

>>
>> Oops, hm, I now doubt, how Andrey's patches work, if bitmap_list_load function
>> fails when unknown flags are found. So, can we really show unknown-flags,
>> or we'll fail if there any?
>>
>> And if we fail (I hope we fail) on unknown flags, than we don't need this
>> field :).
>>
> 
> But we can update bitmap_list_load, to have a flag, and show bitmaps with
> unknown flags in qemu-img info.. And we have to somehow not fail on open before
> it. I'm very sorry that I understand it all only now :(.
> 
> So, do we really need unknown-flags field? If yes, may be, do it as a next step?
> And proceed now without them?

Good point about not even being able to open an image with unknown flags
to be able to report on it (qemu-img check might want to be able to do
it, but if qemu-img info refuses to open the image because of unknown
flags, that's acceptable).  I'm good with your idea of skipping
unknown-flags for now, and adding it in a later patch if it proves
worthwhile.
Eric Blake Feb. 4, 2019, 5:37 p.m. UTC | #19
On 2/4/19 11:33 AM, Eric Blake wrote:
> On 2/4/19 10:46 AM, Vladimir Sementsov-Ogievskiy wrote:
> 
>>>
>>> Oops, hm, I now doubt, how Andrey's patches work, if bitmap_list_load function
>>> fails when unknown flags are found. So, can we really show unknown-flags,
>>> or we'll fail if there any?
>>>
>>> And if we fail (I hope we fail) on unknown flags, than we don't need this
>>> field :).
>>>
>>
>> But we can update bitmap_list_load, to have a flag, and show bitmaps with
>> unknown flags in qemu-img info.. And we have to somehow not fail on open before
>> it. I'm very sorry that I understand it all only now :(.
>>
>> So, do we really need unknown-flags field? If yes, may be, do it as a next step?
>> And proceed now without them?
> 
> Good point about not even being able to open an image with unknown flags
> to be able to report on it (qemu-img check might want to be able to do
> it, but if qemu-img info refuses to open the image because of unknown
> flags, that's acceptable).  I'm good with your idea of skipping
> unknown-flags for now, and adding it in a later patch if it proves
> worthwhile.

It may also be worth an addition to iotests that intentionally corrupts
an image with persistent bitmaps to set an unknown bit, just to see how
qemu reacts to such an image.
Kevin Wolf Feb. 5, 2019, 10 a.m. UTC | #20
Am 04.02.2019 um 16:36 hat Vladimir Sementsov-Ogievskiy geschrieben:
> 04.02.2019 16:45, Markus Armbruster wrote:
> > Kevin Wolf <kwolf@redhat.com> writes:
> > 
> >> Am 01.02.2019 um 19:39 hat Markus Armbruster geschrieben:
> >>> Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> writes:
> >>>
> >>>> In the 'Format specific information' section of the 'qemu-img info'
> >>>> command output, the supplemental information about existing QCOW2
> >>>> bitmaps will be shown, such as a bitmap name, flags and granularity:
> >>>>
> >>>> image: /vz/vmprivate/VM1/harddisk.hdd
> >>>> file format: qcow2
> >>>> virtual size: 64G (68719476736 bytes)
> >>>> disk size: 3.0M
> >>>> cluster_size: 1048576
> >>>> Format specific information:
> >>>>      compat: 1.1
> >>>>      lazy refcounts: true
> >>>>      bitmaps:
> >>>>          [0]:
> >>>>              flags:
> >>>>                  [0]: in-use
> >>>>                  [1]: auto
> >>>>              name: back-up1
> >>>>              unknown flags: 4
> >>>>              granularity: 65536
> >>>>          [1]:
> >>>>              flags:
> >>>>                  [0]: in-use
> >>>>                  [1]: auto
> >>>>              name: back-up2
> >>>>              unknown flags: 8
> >>>>              granularity: 65536
> >>>>      refcount bits: 16
> >>>>      corrupt: false
> >>>>
> >>>> Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
> >>>> ---
> >>> [...]
> >>>> diff --git a/qapi/block-core.json b/qapi/block-core.json
> >>>> index 91685be..271e0df 100644
> >>>> --- a/qapi/block-core.json
> >>>> +++ b/qapi/block-core.json
> >>>> @@ -69,6 +69,8 @@
> >>>>   # @encrypt: details about encryption parameters; only set if image
> >>>>   #           is encrypted (since 2.10)
> >>>>   #
> >>>> +# @bitmaps: A list of qcow2 bitmap details (since 4.0)
> >>>> +#
> >>>>   # Since: 1.7
> >>>>   ##
> >>>>   { 'struct': 'ImageInfoSpecificQCow2',
> >>>> @@ -77,7 +79,8 @@
> >>>>         '*lazy-refcounts': 'bool',
> >>>>         '*corrupt': 'bool',
> >>>>         'refcount-bits': 'int',
> >>>> -      '*encrypt': 'ImageInfoSpecificQCow2Encryption'
> >>>> +      '*encrypt': 'ImageInfoSpecificQCow2Encryption',
> >>>> +      '*bitmaps': ['Qcow2BitmapInfo']
> >>>>     } }
> >>>>   
> >>>>   ##
> >>>> @@ -454,6 +457,41 @@
> >>>>              'status': 'DirtyBitmapStatus'} }
> >>>>   
> >>>>   ##
> >>>> +# @Qcow2BitmapInfoFlags:
> >>>> +#
> >>>> +# An enumeration of flags that a bitmap can report to the user.
> >>>> +#
> >>>> +# @in-use: The bitmap was not saved correctly and may be inconsistent.
> >>>
> >>> I doubt the casual reader could guess the meaning from the name.  What
> >>> about @dirty?
> >>
> >> Feels like overloading the word "dirty" in the context of "dirty
> >> bitmaps". This might easily lead to misinterpretation as "this bitmap
> >> marks some clusters as dirty" rather than "this bitmap might be
> >> inconsistent".
> > 
> > Call it @possibly-inconsistent then?
> > 
> 
> If you run qmp query-block while vm is running, all bitmaps in image will
> be in use. And in this context, in-use seems more appropriate than
> possibly-inconsistent. And in-use has less interference with the fact that
> actual bitmaps now are in-RAM BdrvDirtyBitmap structures which are actually
> consistent (and also shown by query-block)

Good point, but then this description isn't correct either:

    @in-use: The bitmap was not saved correctly and may be inconsistent.

Maybe what needs to change is the description, not the name?

Kevin
Vladimir Sementsov-Ogievskiy Feb. 5, 2019, 1:16 p.m. UTC | #21
05.02.2019 13:00, Kevin Wolf wrote:
> Am 04.02.2019 um 16:36 hat Vladimir Sementsov-Ogievskiy geschrieben:
>> 04.02.2019 16:45, Markus Armbruster wrote:
>>> Kevin Wolf <kwolf@redhat.com> writes:
>>>
>>>> Am 01.02.2019 um 19:39 hat Markus Armbruster geschrieben:
>>>>> Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> writes:
>>>>>
>>>>>> In the 'Format specific information' section of the 'qemu-img info'
>>>>>> command output, the supplemental information about existing QCOW2
>>>>>> bitmaps will be shown, such as a bitmap name, flags and granularity:
>>>>>>
>>>>>> image: /vz/vmprivate/VM1/harddisk.hdd
>>>>>> file format: qcow2
>>>>>> virtual size: 64G (68719476736 bytes)
>>>>>> disk size: 3.0M
>>>>>> cluster_size: 1048576
>>>>>> Format specific information:
>>>>>>       compat: 1.1
>>>>>>       lazy refcounts: true
>>>>>>       bitmaps:
>>>>>>           [0]:
>>>>>>               flags:
>>>>>>                   [0]: in-use
>>>>>>                   [1]: auto
>>>>>>               name: back-up1
>>>>>>               unknown flags: 4
>>>>>>               granularity: 65536
>>>>>>           [1]:
>>>>>>               flags:
>>>>>>                   [0]: in-use
>>>>>>                   [1]: auto
>>>>>>               name: back-up2
>>>>>>               unknown flags: 8
>>>>>>               granularity: 65536
>>>>>>       refcount bits: 16
>>>>>>       corrupt: false
>>>>>>
>>>>>> Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
>>>>>> ---
>>>>> [...]
>>>>>> diff --git a/qapi/block-core.json b/qapi/block-core.json
>>>>>> index 91685be..271e0df 100644
>>>>>> --- a/qapi/block-core.json
>>>>>> +++ b/qapi/block-core.json
>>>>>> @@ -69,6 +69,8 @@
>>>>>>    # @encrypt: details about encryption parameters; only set if image
>>>>>>    #           is encrypted (since 2.10)
>>>>>>    #
>>>>>> +# @bitmaps: A list of qcow2 bitmap details (since 4.0)
>>>>>> +#
>>>>>>    # Since: 1.7
>>>>>>    ##
>>>>>>    { 'struct': 'ImageInfoSpecificQCow2',
>>>>>> @@ -77,7 +79,8 @@
>>>>>>          '*lazy-refcounts': 'bool',
>>>>>>          '*corrupt': 'bool',
>>>>>>          'refcount-bits': 'int',
>>>>>> -      '*encrypt': 'ImageInfoSpecificQCow2Encryption'
>>>>>> +      '*encrypt': 'ImageInfoSpecificQCow2Encryption',
>>>>>> +      '*bitmaps': ['Qcow2BitmapInfo']
>>>>>>      } }
>>>>>>    
>>>>>>    ##
>>>>>> @@ -454,6 +457,41 @@
>>>>>>               'status': 'DirtyBitmapStatus'} }
>>>>>>    
>>>>>>    ##
>>>>>> +# @Qcow2BitmapInfoFlags:
>>>>>> +#
>>>>>> +# An enumeration of flags that a bitmap can report to the user.
>>>>>> +#
>>>>>> +# @in-use: The bitmap was not saved correctly and may be inconsistent.
>>>>>
>>>>> I doubt the casual reader could guess the meaning from the name.  What
>>>>> about @dirty?
>>>>
>>>> Feels like overloading the word "dirty" in the context of "dirty
>>>> bitmaps". This might easily lead to misinterpretation as "this bitmap
>>>> marks some clusters as dirty" rather than "this bitmap might be
>>>> inconsistent".
>>>
>>> Call it @possibly-inconsistent then?
>>>
>>
>> If you run qmp query-block while vm is running, all bitmaps in image will
>> be in use. And in this context, in-use seems more appropriate than
>> possibly-inconsistent. And in-use has less interference with the fact that
>> actual bitmaps now are in-RAM BdrvDirtyBitmap structures which are actually
>> consistent (and also shown by query-block)
> 
> Good point, but then this description isn't correct either:
> 
>      @in-use: The bitmap was not saved correctly and may be inconsistent.
> 
> Maybe what needs to change is the description, not the name?
> 
> Kevin
> 

How about the following:

@in-use: This flag is set while bitmap is in use by software and it's data in
qcow2 image may be out of date when actual bitmap data is managed by software.
Presence of this flag in offline image means that bitmap was not saved correctly
after last usage and it's data may be inconsistent.
Kevin Wolf Feb. 5, 2019, 2:28 p.m. UTC | #22
Am 05.02.2019 um 14:16 hat Vladimir Sementsov-Ogievskiy geschrieben:
> 05.02.2019 13:00, Kevin Wolf wrote:
> > Am 04.02.2019 um 16:36 hat Vladimir Sementsov-Ogievskiy geschrieben:
> >> 04.02.2019 16:45, Markus Armbruster wrote:
> >>> Kevin Wolf <kwolf@redhat.com> writes:
> >>>
> >>>> Am 01.02.2019 um 19:39 hat Markus Armbruster geschrieben:
> >>>>> Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> writes:
> >>>>>
> >>>>>> In the 'Format specific information' section of the 'qemu-img info'
> >>>>>> command output, the supplemental information about existing QCOW2
> >>>>>> bitmaps will be shown, such as a bitmap name, flags and granularity:
> >>>>>>
> >>>>>> image: /vz/vmprivate/VM1/harddisk.hdd
> >>>>>> file format: qcow2
> >>>>>> virtual size: 64G (68719476736 bytes)
> >>>>>> disk size: 3.0M
> >>>>>> cluster_size: 1048576
> >>>>>> Format specific information:
> >>>>>>       compat: 1.1
> >>>>>>       lazy refcounts: true
> >>>>>>       bitmaps:
> >>>>>>           [0]:
> >>>>>>               flags:
> >>>>>>                   [0]: in-use
> >>>>>>                   [1]: auto
> >>>>>>               name: back-up1
> >>>>>>               unknown flags: 4
> >>>>>>               granularity: 65536
> >>>>>>           [1]:
> >>>>>>               flags:
> >>>>>>                   [0]: in-use
> >>>>>>                   [1]: auto
> >>>>>>               name: back-up2
> >>>>>>               unknown flags: 8
> >>>>>>               granularity: 65536
> >>>>>>       refcount bits: 16
> >>>>>>       corrupt: false
> >>>>>>
> >>>>>> Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
> >>>>>> ---
> >>>>> [...]
> >>>>>> diff --git a/qapi/block-core.json b/qapi/block-core.json
> >>>>>> index 91685be..271e0df 100644
> >>>>>> --- a/qapi/block-core.json
> >>>>>> +++ b/qapi/block-core.json
> >>>>>> @@ -69,6 +69,8 @@
> >>>>>>    # @encrypt: details about encryption parameters; only set if image
> >>>>>>    #           is encrypted (since 2.10)
> >>>>>>    #
> >>>>>> +# @bitmaps: A list of qcow2 bitmap details (since 4.0)
> >>>>>> +#
> >>>>>>    # Since: 1.7
> >>>>>>    ##
> >>>>>>    { 'struct': 'ImageInfoSpecificQCow2',
> >>>>>> @@ -77,7 +79,8 @@
> >>>>>>          '*lazy-refcounts': 'bool',
> >>>>>>          '*corrupt': 'bool',
> >>>>>>          'refcount-bits': 'int',
> >>>>>> -      '*encrypt': 'ImageInfoSpecificQCow2Encryption'
> >>>>>> +      '*encrypt': 'ImageInfoSpecificQCow2Encryption',
> >>>>>> +      '*bitmaps': ['Qcow2BitmapInfo']
> >>>>>>      } }
> >>>>>>    
> >>>>>>    ##
> >>>>>> @@ -454,6 +457,41 @@
> >>>>>>               'status': 'DirtyBitmapStatus'} }
> >>>>>>    
> >>>>>>    ##
> >>>>>> +# @Qcow2BitmapInfoFlags:
> >>>>>> +#
> >>>>>> +# An enumeration of flags that a bitmap can report to the user.
> >>>>>> +#
> >>>>>> +# @in-use: The bitmap was not saved correctly and may be inconsistent.
> >>>>>
> >>>>> I doubt the casual reader could guess the meaning from the name.  What
> >>>>> about @dirty?
> >>>>
> >>>> Feels like overloading the word "dirty" in the context of "dirty
> >>>> bitmaps". This might easily lead to misinterpretation as "this bitmap
> >>>> marks some clusters as dirty" rather than "this bitmap might be
> >>>> inconsistent".
> >>>
> >>> Call it @possibly-inconsistent then?
> >>>
> >>
> >> If you run qmp query-block while vm is running, all bitmaps in image will
> >> be in use. And in this context, in-use seems more appropriate than
> >> possibly-inconsistent. And in-use has less interference with the fact that
> >> actual bitmaps now are in-RAM BdrvDirtyBitmap structures which are actually
> >> consistent (and also shown by query-block)
> > 
> > Good point, but then this description isn't correct either:
> > 
> >      @in-use: The bitmap was not saved correctly and may be inconsistent.
> > 
> > Maybe what needs to change is the description, not the name?
> > 
> > Kevin
> > 
> 
> How about the following:
> 
> @in-use: This flag is set while bitmap is in use by software and it's data in
> qcow2 image may be out of date when actual bitmap data is managed by software.
> Presence of this flag in offline image means that bitmap was not saved correctly
> after last usage and it's data may be inconsistent.

Yes, I think this has the information it needs to have.

It needs some spelling and grammar fixes, but before I make them, I'll
wait if Eric would phrase it in a different way that sounds more natural
to him as a native speaker.

Kevin
Eric Blake Feb. 5, 2019, 2:38 p.m. UTC | #23
On 2/5/19 8:28 AM, Kevin Wolf wrote:

>> How about the following:
>>
>> @in-use: This flag is set while bitmap is in use by software and it's data in
>> qcow2 image may be out of date when actual bitmap data is managed by software.
>> Presence of this flag in offline image means that bitmap was not saved correctly
>> after last usage and it's data may be inconsistent.
> 
> Yes, I think this has the information it needs to have.
> 
> It needs some spelling and grammar fixes, but before I make them, I'll
> wait if Eric would phrase it in a different way that sounds more natural
> to him as a native speaker.

@in-use: This flag is set by any process actively modifying the qcow2
file, and cleared when the updated bitmap is flushed to the qcow2 image.
Presence of this flag in an offline image means that the bitmap was not
saved correctly after its last usage, and may contain inconsistent data.
diff mbox series

Patch

diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c
index b946301..516994e 100644
--- a/block/qcow2-bitmap.c
+++ b/block/qcow2-bitmap.c
@@ -1006,6 +1006,84 @@  fail:
     return false;
 }
 
+
+static Qcow2BitmapInfoFlagsList *get_bitmap_info_flags(uint32_t flags)
+{
+    Qcow2BitmapInfoFlagsList *list = NULL;
+    Qcow2BitmapInfoFlagsList **plist = &list;
+    int i;
+
+    static const struct {
+        int bme;  /* Bitmap directory entry flags */
+        int info; /* The flags to report to the user */
+    } map[] = {
+        { BME_FLAG_IN_USE, QCOW2_BITMAP_INFO_FLAGS_IN_USE },
+        { BME_FLAG_AUTO,   QCOW2_BITMAP_INFO_FLAGS_AUTO },
+    };
+
+    int map_size = sizeof(map) / sizeof(map[0]);
+
+    for (i = 0; i < map_size; ++i) {
+        if (flags & map[i].bme) {
+            Qcow2BitmapInfoFlagsList *entry =
+                g_new0(Qcow2BitmapInfoFlagsList, 1);
+            entry->value = map[i].info;
+            *plist = entry;
+            plist = &entry->next;
+            flags &= ~map[i].bme;
+        }
+    }
+    /* Check if the BME_* mapping above is complete */
+    assert(!flags);
+
+    return list;
+}
+
+/*
+ * qcow2_get_bitmap_info_list()
+ * Returns a list of QCOW2 bitmap details.
+ * In case of no bitmaps, the function returns NULL and
+ * the @errp parameter is not set.
+ * When bitmap information can not be obtained, the function returns
+ * NULL and the @errp parameter is set.
+ */
+Qcow2BitmapInfoList *qcow2_get_bitmap_info_list(BlockDriverState *bs,
+                                                Error **errp)
+{
+    BDRVQcow2State *s = bs->opaque;
+    Qcow2BitmapList *bm_list;
+    Qcow2Bitmap *bm;
+    Qcow2BitmapInfoList *list = NULL;
+    Qcow2BitmapInfoList **plist = &list;
+
+    if (s->nb_bitmaps == 0) {
+        return NULL;
+    }
+
+    bm_list = bitmap_list_load(bs, s->bitmap_directory_offset,
+                               s->bitmap_directory_size, errp);
+    if (bm_list == NULL) {
+        return NULL;
+    }
+
+    QSIMPLEQ_FOREACH(bm, bm_list, entry) {
+        Qcow2BitmapInfo *info = g_new0(Qcow2BitmapInfo, 1);
+        Qcow2BitmapInfoList *obj = g_new0(Qcow2BitmapInfoList, 1);
+        info->granularity = 1U << bm->granularity_bits;
+        info->name = g_strdup(bm->name);
+        info->flags = get_bitmap_info_flags(bm->flags & ~BME_RESERVED_FLAGS);
+        info->unknown_flags = bm->flags & BME_RESERVED_FLAGS;
+        info->has_unknown_flags = !!info->unknown_flags;
+        obj->value = info;
+        *plist = obj;
+        plist = &obj->next;
+    }
+
+    bitmap_list_free(bm_list);
+
+    return list;
+}
+
 int qcow2_reopen_bitmaps_rw_hint(BlockDriverState *bs, bool *header_updated,
                                  Error **errp)
 {
diff --git a/block/qcow2.c b/block/qcow2.c
index 27e5a2c..a5607f1 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -4386,7 +4386,7 @@  static ImageInfoSpecific *qcow2_get_specific_info(BlockDriverState *bs,
     spec_info = g_new(ImageInfoSpecific, 1);
     *spec_info = (ImageInfoSpecific){
         .type  = IMAGE_INFO_SPECIFIC_KIND_QCOW2,
-        .u.qcow2.data = g_new(ImageInfoSpecificQCow2, 1),
+        .u.qcow2.data = g_new0(ImageInfoSpecificQCow2, 1),
     };
     if (s->qcow_version == 2) {
         *spec_info->u.qcow2.data = (ImageInfoSpecificQCow2){
@@ -4394,6 +4394,13 @@  static ImageInfoSpecific *qcow2_get_specific_info(BlockDriverState *bs,
             .refcount_bits      = s->refcount_bits,
         };
     } else if (s->qcow_version == 3) {
+        Qcow2BitmapInfoList *bitmaps;
+        bitmaps = qcow2_get_bitmap_info_list(bs, &local_err);
+        if (local_err) {
+            error_propagate(errp, local_err);
+            qapi_free_ImageInfoSpecific(spec_info);
+            return NULL;
+        }
         *spec_info->u.qcow2.data = (ImageInfoSpecificQCow2){
             .compat             = g_strdup("1.1"),
             .lazy_refcounts     = s->compatible_features &
@@ -4403,6 +4410,8 @@  static ImageInfoSpecific *qcow2_get_specific_info(BlockDriverState *bs,
                                   QCOW2_INCOMPAT_CORRUPT,
             .has_corrupt        = true,
             .refcount_bits      = s->refcount_bits,
+            .has_bitmaps        = !!bitmaps,
+            .bitmaps            = bitmaps,
         };
     } else {
         /* if this assertion fails, this probably means a new version was
diff --git a/block/qcow2.h b/block/qcow2.h
index 438a1de..13e8964 100644
--- a/block/qcow2.h
+++ b/block/qcow2.h
@@ -684,6 +684,8 @@  int qcow2_check_bitmaps_refcounts(BlockDriverState *bs, BdrvCheckResult *res,
                                   void **refcount_table,
                                   int64_t *refcount_table_size);
 bool qcow2_load_dirty_bitmaps(BlockDriverState *bs, Error **errp);
+Qcow2BitmapInfoList *qcow2_get_bitmap_info_list(BlockDriverState *bs,
+                                                Error **errp);
 int qcow2_reopen_bitmaps_rw_hint(BlockDriverState *bs, bool *header_updated,
                                  Error **errp);
 int qcow2_reopen_bitmaps_rw(BlockDriverState *bs, Error **errp);
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 91685be..271e0df 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -69,6 +69,8 @@ 
 # @encrypt: details about encryption parameters; only set if image
 #           is encrypted (since 2.10)
 #
+# @bitmaps: A list of qcow2 bitmap details (since 4.0)
+#
 # Since: 1.7
 ##
 { 'struct': 'ImageInfoSpecificQCow2',
@@ -77,7 +79,8 @@ 
       '*lazy-refcounts': 'bool',
       '*corrupt': 'bool',
       'refcount-bits': 'int',
-      '*encrypt': 'ImageInfoSpecificQCow2Encryption'
+      '*encrypt': 'ImageInfoSpecificQCow2Encryption',
+      '*bitmaps': ['Qcow2BitmapInfo']
   } }
 
 ##
@@ -454,6 +457,41 @@ 
            'status': 'DirtyBitmapStatus'} }
 
 ##
+# @Qcow2BitmapInfoFlags:
+#
+# An enumeration of flags that a bitmap can report to the user.
+#
+# @in-use: The bitmap was not saved correctly and may be inconsistent.
+#
+# @auto: The bitmap must reflect all changes of the virtual disk by any
+#        application that would write to this qcow2 file.
+#
+# Since: 4.0
+##
+{ 'enum': 'Qcow2BitmapInfoFlags',
+  'data': ['in-use', 'auto'] }
+
+##
+# @Qcow2BitmapInfo:
+#
+# Qcow2 bitmap information.
+#
+# @name: the name of the bitmap
+#
+# @granularity: granularity of the bitmap in bytes
+#
+# @flags: recognized flags of the bitmap
+#
+# @unknown-flags: any remaining flags not recognized by the current qemu version
+#
+# Since: 4.0
+##
+{ 'struct': 'Qcow2BitmapInfo',
+  'data': {'name': 'str', 'granularity': 'uint32',
+           'flags': ['Qcow2BitmapInfoFlags'],
+           '*unknown-flags': 'uint32' } }
+
+##
 # @BlockLatencyHistogramInfo:
 #
 # Block latency histogram.