[v2,00/33] block: publish backup-top filter

Message

Vladimir Sementsov-Ogievskiy May 20, 2021, 2:21 p.m. UTC

Hi all!

v2:
01-02: new
03: don't bother with supporting empty child: we should never have such
    at this point
05: add comment
06: keep checking conflict with global
    add realized_set_allowed to qdev_prop_drive_iothread
07: improve cbw_cbw() name
    improve commit message
10: rebased on unchanged backup_calculate_cluster_size(). keep r-b  CHECK ME
12: new
13: drop extra bdrv_unref()
18: add compress local variable
    add comment about x-deprecated-compress
19: new, replacement for "[PATCH 17/21] block/block-copy: switch to fully set bitmap by default"
22: improve qapi documentation
23-33: test: a lot of refactoring

We have image fleecing scheme to export point-in-time state of active
disk (iotest 222):


                                      backup(sync=none)
                     ┌───────────────────────────────────────┐
                     ▼                                       │
┌────────────┐     ┌────────────────┐  backing             ┌─────────────┐
│ NBD export │ ─── │ temp qcow2 img │ ───────────────────▶ │ active disk │
└────────────┘     └────────────────┘                      └─────────────┘
                                                             ▲
┌────────────┐                                               │
│ guest blk  │ ──────────────────────────────────────────────┘
└────────────┘                        


Actually, backup job inserts a backup-top filter, so in detail it looks
like:

                                      backup(sync=none)
                     ┌───────────────────────────────────────┐
                     ▼                                       │
┌────────────┐     ┌────────────────┐  backing             ┌─────────────┐
│ NBD export │ ─── │ temp qcow2 img │ ───────────────────▶ │ active disk │
└────────────┘     └────────────────┘                      └─────────────┘
                     ▲                                       ▲
                     │ target                                │
                     │                                       │
┌────────────┐     ┌────────────────┐  backing               │
│ guest blk  │ ──▶ │   backup-top   │ ───────────────────────┘
└────────────┘     └────────────────┘

And job does nothing here. In a new blockdev world user is intended to
operate on node level, and insert/remove filters by hand. Let's get rid
of job in the scheme:

┌────────────┐     ┌────────────────┐  backing             ┌─────────────┐
│ NBD export │ ─── │ temp qcow2 img │ ───────────────────▶ │ active disk │
└────────────┘     └────────────────┘                      └─────────────┘
                     ▲                                       ▲
                     │ target                                │
                     │                                       │
┌────────────┐     ┌────────────────┐  backing               │
│ guest blk  │ ──▶ │   backup-top   │ ───────────────────────┘
└────────────┘     └────────────────┘


The series prepares qom-set to make possible inserting filters above
root node (patches 03-06), rename backup-top to copy-before-write, do
other preparations for publishing the filter, and finally publish it,
add qapi interface and test new fleecing scheme in 222 (first, some
good test refactoring).

Vladimir Sementsov-Ogievskiy (33):
  block: rename bdrv_replace_child to bdrv_replace_child_tran
  block: comment graph-modifying function not updating permissions
  block: introduce bdrv_replace_child_bs()
  block: introduce blk_replace_bs
  qdev-properties: PropertyInfo: add realized_set_allowed field
  qdev: allow setting drive property for realized device
  block: rename backup-top to copy-before-write
  block/backup: drop support for copy_range
  block-copy: always set BDRV_REQ_SERIALISING flag
  block/backup: move cluster size calculation to block-copy
  block/copy-before-write: relax permission requirements when no parents
  block/copy-before-write: drop extra bdrv_unref on failure path
  block/copy-before-write: use file child instead of backing
  block/copy-before-write: bdrv_cbw_append(): replace child at last
  block/copy-before-write: introduce cbw_init()
  block/copy-before-write: cbw_init(): rename variables
  block/copy-before-write: cbw_init(): use file child after attaching
  block/copy-before-write: cbw_init(): use options
  block/copy-before-write: initialize block-copy bitmap
  block/block-copy: make setting progress optional
  block/copy-before-write: make public block driver
  qapi: publish copy-before-write filter
  python/qemu/machine.py: refactor _qemu_args()
  python/qemu/machine: QEMUMachine: improve qmp() method
  iotests.py: VM: add own __enter__ method
  iotests/222: fix pylint and mypy complains
  iotests/222: constantly use single quotes for strings
  iotests: move 222 to tests/image-fleecing
  iotests.py: hmp_qemu_io: support qdev
  iotests/image-fleecing: proper source device
  iotests/image-fleecing: rename tgt_node
  iotests/image-fleecing: prepare for adding new test-case
  iotests/image-fleecing: add test-case for copy-before-write filter

 qapi/block-core.json                        |  30 ++-
 block/{backup-top.h => copy-before-write.h} |  26 +-
 include/block/block-copy.h                  |   4 +-
 include/block/block.h                       |   2 +
 include/hw/qdev-properties.h                |   1 +
 include/sysemu/block-backend.h              |   1 +
 block.c                                     |  52 +++-
 block/backup-top.c                          | 253 ------------------
 block/backup.c                              | 115 ++-------
 block/block-backend.c                       |   8 +
 block/block-copy.c                          |  98 ++++++-
 block/copy-before-write.c                   | 268 ++++++++++++++++++++
 hw/core/qdev-properties-system.c            |  43 +++-
 hw/core/qdev-properties.c                   |   6 +-
 MAINTAINERS                                 |   4 +-
 block/meson.build                           |   2 +-
 python/qemu/machine.py                      |  30 ++-
 tests/qemu-iotests/222                      | 159 ------------
 tests/qemu-iotests/222.out                  |  67 -----
 tests/qemu-iotests/283                      |  35 ++-
 tests/qemu-iotests/283.out                  |   4 +-
 tests/qemu-iotests/297                      |   2 +-
 tests/qemu-iotests/iotests.py               |   9 +-
 tests/qemu-iotests/tests/image-fleecing     | 192 ++++++++++++++
 tests/qemu-iotests/tests/image-fleecing.out | 139 ++++++++++
 25 files changed, 890 insertions(+), 660 deletions(-)
 rename block/{backup-top.h => copy-before-write.h} (56%)
 delete mode 100644 block/backup-top.c
 create mode 100644 block/copy-before-write.c
 delete mode 100755 tests/qemu-iotests/222
 delete mode 100644 tests/qemu-iotests/222.out
 create mode 100755 tests/qemu-iotests/tests/image-fleecing
 create mode 100644 tests/qemu-iotests/tests/image-fleecing.out

Comments

Max Reitz May 31, 2021, 5:11 p.m. UTC | #1

On 20.05.21 16:21, Vladimir Sementsov-Ogievskiy wrote:
> Hi all!
>
> v2:
> 01-02: new
> 03: don't bother with supporting empty child: we should never have such
>      at this point
> 05: add comment
> 06: keep checking conflict with global
>      add realized_set_allowed to qdev_prop_drive_iothread
> 07: improve cbw_cbw() name
>      improve commit message
> 10: rebased on unchanged backup_calculate_cluster_size(). keep r-b  CHECK ME

I checked you! It’s ok. :)

(and btw, I’ll look at the rest tomorrow)

> 12: new
> 13: drop extra bdrv_unref()
> 18: add compress local variable
>      add comment about x-deprecated-compress
> 19: new, replacement for "[PATCH 17/21] block/block-copy: switch to fully set bitmap by default"
> 22: improve qapi documentation
> 23-33: test: a lot of refactoring

Vladimir Sementsov-Ogievskiy May 31, 2021, 5:35 p.m. UTC | #2

31.05.2021 20:11, Max Reitz wrote:
> On 20.05.21 16:21, Vladimir Sementsov-Ogievskiy wrote:
>> Hi all!
>>
>> v2:
>> 01-02: new
>> 03: don't bother with supporting empty child: we should never have such
>>      at this point
>> 05: add comment
>> 06: keep checking conflict with global
>>      add realized_set_allowed to qdev_prop_drive_iothread
>> 07: improve cbw_cbw() name
>>      improve commit message
>> 10: rebased on unchanged backup_calculate_cluster_size(). keep r-b  CHECK ME
> 
> I checked you! It’s ok. :)
> 
> (and btw, I’ll look at the rest tomorrow)

Thanks)

Actually, "CHECK ME" was for myself, and "ME" is a patch, not me) I forget to drop it.. Interesting, did I forget to check

> 
>> 12: new
>> 13: drop extra bdrv_unref()
>> 18: add compress local variable
>>      add comment about x-deprecated-compress
>> 19: new, replacement for "[PATCH 17/21] block/block-copy: switch to fully set bitmap by default"
>> 22: improve qapi documentation
>> 23-33: test: a lot of refactoring
>