| Message ID | 20220329204107.411011-1-v.sementsov-og@mail.ru (mailing list archive) |
|---|---|
| Headers | show |
| Series | Transactional block-graph modifying API | expand |
Outdated :(. I should rebase it on master, there are a lot of conflicts after Emanuele's series. 29.03.2022 23:40, Vladimir Sementsov-Ogievskiy wrote: > Hi all! > > That's a big series, which unites some of my previous ones, and > completes them with necessary additions to finally implement block-graph > modifying API. The series is called "v4" as it inherits > "[PATCH v3 00/11] blockdev-replace" (among other things). > > After this series, we have blockdev-add, blockdev-del and > x-blockdev-replace transaction actions, which allows to insert and > remove filters. > > Additional challenge is to avoid intermediate permission update. That's > and existing paradigm of block graph modifications: first do all the > modifications and then refresh the permissions. Now we should bring this > paradigm to block-graph modifying transactions: if several graph > modifying commands are sequential in one transaction, permission are > updated after the last of these commands. The application of this is > possibility to correct copy-before-write filter permission requirements > (see last patch). > > I now unite all these things into one series because: > - they depend on each other and I have to rebase them together when > something needs fix or refactoring > - just to resend with my new email address > If needed, parts may go in separate, and I can split them again if > necessary. > > So, what is here: > > 1. "[PATCH 00/14] block: cleanup backing and file handling" series, > unchanged: > > block: BlockDriver: add .filtered_child_is_backing field > block: introduce bdrv_open_file_child() helper > block/blklogwrites: don't care to remove bs->file child on failure > test-bdrv-graph-mod: update test_parallel_perm_update test case > tests-bdrv-drain: bdrv_replace_test driver: declare supports_backing > test-bdrv-graph-mod: fix filters to be filters > block: document connection between child roles and > bs->backing/bs->file > block/snapshot: stress that we fallback to primary child > Revert "block: Let replace_child_noperm free children" > Revert "block: Let replace_child_tran keep indirect pointer" > Revert "block: Restructure remove_file_or_backing_child()" > Revert "block: Pass BdrvChild ** to replace_child_noperm" > block: Manipulate bs->file / bs->backing pointers in .attach/.detach > block/snapshot: drop indirection around bdrv_snapshot_fallback_ptr > > 2. implement bdrv_unref_tran() - the key thing to implement blockdev-del > transaction action later. > This part inherits from "[PATCH 00/14] block: blockdev-del force=false". > Still force=false is not realized and qcow2 is untouched, as the target > now is transactional removement. > > block: refactor bdrv_remove_file_or_backing_child to bdrv_remove_child > block: drop bdrv_detach_child() > block: drop bdrv_remove_filter_or_cow_child > block: bdrv_refresh_perms(): allow external tran > block: refactor bdrv_list_refresh_perms to allow any list of nodes > block: make permission update functions public > block: add bdrv_try_set_aio_context_tran transaction action > block: implemet bdrv_unref_tran() > > 3. Move blockdev.c transactions to util/transactions.c API. > > blockdev: refactor transaction to use Transaction API > blockdev: transactions: rename some things > blockdev: qmp_transaction: refactor loop to classic for > blockdev: transaction: refactor handling transaction properties > blockdev: qmp_transaction: drop extra generic layer > > 4. add blockdev-del transaction action > > qapi: block: add blockdev-del transaction action > > 5. add blockdev-add transaction action > (inherits from "[PATCH 0/2] blockdev-add transaction") > > block: introduce BDRV_O_NOPERM flag > block: bdrv_insert_node(): use BDRV_O_NOPERM > qapi: block: add blockdev-add transaction action > iotests: add blockdev-add-transaction > > 6. add x-blockdev-replace command and transaction action > (inherits from "[PATCH v3 00/11] blockdev-replace") > > block-backend: blk_root(): drop const specifier on return type > block/export: add blk_by_export_id() > block: make bdrv_find_child() function public > block: bdrv_replace_child_bs(): move to external transaction > qapi: add x-blockdev-replace command > qapi: add x-blockdev-replace transaction action > block: bdrv_get_xdbg_block_graph(): report export ids > iotests.py: qemu_img_create: use imgfmt by default > iotests.py: introduce VM.assert_edges_list() method > iotests.py: add VM.qmp_check() helper > iotests: add filter-insertion > > 7. Correct permission scheme of copy-before-write filter, with help of > new design of graph-modifying API. > > block: bdrv_open_inherit: create BlockBackend only when necessary > block/copy-before-write: correct permission scheme > > block.c | 869 ++++++++++-------- > block/blkdebug.c | 9 +- > block/blklogwrites.c | 11 +- > block/blkreplay.c | 7 +- > block/blkverify.c | 9 +- > block/block-backend.c | 10 +- > block/bochs.c | 7 +- > block/cloop.c | 7 +- > block/commit.c | 1 + > block/copy-before-write.c | 24 +- > block/copy-on-read.c | 9 +- > block/crypto.c | 11 +- > block/dmg.c | 7 +- > block/export/export.c | 31 + > block/filter-compress.c | 6 +- > block/mirror.c | 1 + > block/monitor/block-hmp-cmds.c | 2 +- > block/parallels.c | 7 +- > block/preallocate.c | 9 +- > block/qcow.c | 6 +- > block/qcow2.c | 8 +- > block/qed.c | 8 +- > block/raw-format.c | 4 +- > block/replication.c | 8 +- > block/snapshot.c | 60 +- > block/throttle.c | 8 +- > block/vdi.c | 7 +- > block/vhdx.c | 7 +- > block/vmdk.c | 7 +- > block/vpc.c | 7 +- > blockdev.c | 818 +++++++++-------- > include/block/block.h | 72 +- > include/block/block_int.h | 41 +- > include/block/export.h | 1 + > include/sysemu/block-backend.h | 3 +- > qapi/block-core.json | 73 +- > qapi/transaction.json | 35 +- > stubs/blk-by-qdev-id.c | 9 + > stubs/blk-exp-find-by-blk.c | 9 + > stubs/meson.build | 2 + > tests/qemu-iotests/iotests.py | 23 + > .../tests/blockdev-add-transaction | 52 ++ > .../tests/blockdev-add-transaction.out | 6 + > tests/qemu-iotests/tests/filter-insertion | 253 +++++ > tests/qemu-iotests/tests/filter-insertion.out | 5 + > tests/qemu-iotests/tests/image-fleecing | 29 +- > tests/qemu-iotests/tests/image-fleecing.out | 2 - > tests/unit/test-bdrv-drain.c | 11 +- > tests/unit/test-bdrv-graph-mod.c | 94 +- > 49 files changed, 1715 insertions(+), 990 deletions(-) > create mode 100644 stubs/blk-by-qdev-id.c > create mode 100644 stubs/blk-exp-find-by-blk.c > create mode 100755 tests/qemu-iotests/tests/blockdev-add-transaction > create mode 100644 tests/qemu-iotests/tests/blockdev-add-transaction.out > create mode 100755 tests/qemu-iotests/tests/filter-insertion > create mode 100644 tests/qemu-iotests/tests/filter-insertion.out >
Hi all! That's a big series, which unites some of my previous ones, and completes them with necessary additions to finally implement block-graph modifying API. The series is called "v4" as it inherits "[PATCH v3 00/11] blockdev-replace" (among other things). After this series, we have blockdev-add, blockdev-del and x-blockdev-replace transaction actions, which allows to insert and remove filters. Additional challenge is to avoid intermediate permission update. That's and existing paradigm of block graph modifications: first do all the modifications and then refresh the permissions. Now we should bring this paradigm to block-graph modifying transactions: if several graph modifying commands are sequential in one transaction, permission are updated after the last of these commands. The application of this is possibility to correct copy-before-write filter permission requirements (see last patch). I now unite all these things into one series because: - they depend on each other and I have to rebase them together when something needs fix or refactoring - just to resend with my new email address If needed, parts may go in separate, and I can split them again if necessary. So, what is here: 1. "[PATCH 00/14] block: cleanup backing and file handling" series, unchanged: block: BlockDriver: add .filtered_child_is_backing field block: introduce bdrv_open_file_child() helper block/blklogwrites: don't care to remove bs->file child on failure test-bdrv-graph-mod: update test_parallel_perm_update test case tests-bdrv-drain: bdrv_replace_test driver: declare supports_backing test-bdrv-graph-mod: fix filters to be filters block: document connection between child roles and bs->backing/bs->file block/snapshot: stress that we fallback to primary child Revert "block: Let replace_child_noperm free children" Revert "block: Let replace_child_tran keep indirect pointer" Revert "block: Restructure remove_file_or_backing_child()" Revert "block: Pass BdrvChild ** to replace_child_noperm" block: Manipulate bs->file / bs->backing pointers in .attach/.detach block/snapshot: drop indirection around bdrv_snapshot_fallback_ptr 2. implement bdrv_unref_tran() - the key thing to implement blockdev-del transaction action later. This part inherits from "[PATCH 00/14] block: blockdev-del force=false". Still force=false is not realized and qcow2 is untouched, as the target now is transactional removement. block: refactor bdrv_remove_file_or_backing_child to bdrv_remove_child block: drop bdrv_detach_child() block: drop bdrv_remove_filter_or_cow_child block: bdrv_refresh_perms(): allow external tran block: refactor bdrv_list_refresh_perms to allow any list of nodes block: make permission update functions public block: add bdrv_try_set_aio_context_tran transaction action block: implemet bdrv_unref_tran() 3. Move blockdev.c transactions to util/transactions.c API. blockdev: refactor transaction to use Transaction API blockdev: transactions: rename some things blockdev: qmp_transaction: refactor loop to classic for blockdev: transaction: refactor handling transaction properties blockdev: qmp_transaction: drop extra generic layer 4. add blockdev-del transaction action qapi: block: add blockdev-del transaction action 5. add blockdev-add transaction action (inherits from "[PATCH 0/2] blockdev-add transaction") block: introduce BDRV_O_NOPERM flag block: bdrv_insert_node(): use BDRV_O_NOPERM qapi: block: add blockdev-add transaction action iotests: add blockdev-add-transaction 6. add x-blockdev-replace command and transaction action (inherits from "[PATCH v3 00/11] blockdev-replace") block-backend: blk_root(): drop const specifier on return type block/export: add blk_by_export_id() block: make bdrv_find_child() function public block: bdrv_replace_child_bs(): move to external transaction qapi: add x-blockdev-replace command qapi: add x-blockdev-replace transaction action block: bdrv_get_xdbg_block_graph(): report export ids iotests.py: qemu_img_create: use imgfmt by default iotests.py: introduce VM.assert_edges_list() method iotests.py: add VM.qmp_check() helper iotests: add filter-insertion 7. Correct permission scheme of copy-before-write filter, with help of new design of graph-modifying API. block: bdrv_open_inherit: create BlockBackend only when necessary block/copy-before-write: correct permission scheme block.c | 869 ++++++++++-------- block/blkdebug.c | 9 +- block/blklogwrites.c | 11 +- block/blkreplay.c | 7 +- block/blkverify.c | 9 +- block/block-backend.c | 10 +- block/bochs.c | 7 +- block/cloop.c | 7 +- block/commit.c | 1 + block/copy-before-write.c | 24 +- block/copy-on-read.c | 9 +- block/crypto.c | 11 +- block/dmg.c | 7 +- block/export/export.c | 31 + block/filter-compress.c | 6 +- block/mirror.c | 1 + block/monitor/block-hmp-cmds.c | 2 +- block/parallels.c | 7 +- block/preallocate.c | 9 +- block/qcow.c | 6 +- block/qcow2.c | 8 +- block/qed.c | 8 +- block/raw-format.c | 4 +- block/replication.c | 8 +- block/snapshot.c | 60 +- block/throttle.c | 8 +- block/vdi.c | 7 +- block/vhdx.c | 7 +- block/vmdk.c | 7 +- block/vpc.c | 7 +- blockdev.c | 818 +++++++++-------- include/block/block.h | 72 +- include/block/block_int.h | 41 +- include/block/export.h | 1 + include/sysemu/block-backend.h | 3 +- qapi/block-core.json | 73 +- qapi/transaction.json | 35 +- stubs/blk-by-qdev-id.c | 9 + stubs/blk-exp-find-by-blk.c | 9 + stubs/meson.build | 2 + tests/qemu-iotests/iotests.py | 23 + .../tests/blockdev-add-transaction | 52 ++ .../tests/blockdev-add-transaction.out | 6 + tests/qemu-iotests/tests/filter-insertion | 253 +++++ tests/qemu-iotests/tests/filter-insertion.out | 5 + tests/qemu-iotests/tests/image-fleecing | 29 +- tests/qemu-iotests/tests/image-fleecing.out | 2 - tests/unit/test-bdrv-drain.c | 11 +- tests/unit/test-bdrv-graph-mod.c | 94 +- 49 files changed, 1715 insertions(+), 990 deletions(-) create mode 100644 stubs/blk-by-qdev-id.c create mode 100644 stubs/blk-exp-find-by-blk.c create mode 100755 tests/qemu-iotests/tests/blockdev-add-transaction create mode 100644 tests/qemu-iotests/tests/blockdev-add-transaction.out create mode 100755 tests/qemu-iotests/tests/filter-insertion create mode 100644 tests/qemu-iotests/tests/filter-insertion.out