mbox series

[v2,00/31] block/export: Add infrastructure and QAPI for block exports

Message ID 20200924152717.287415-1-kwolf@redhat.com (mailing list archive)
Headers show
Series block/export: Add infrastructure and QAPI for block exports | expand

Message

Kevin Wolf Sept. 24, 2020, 3:26 p.m. UTC
We are planning to add more block export types than just NBD in the near
future (e.g. vhost-user-blk and FUSE). This series lays the ground for
this with some generic block export infrastructure and QAPI interfaces
that will allow managing all of them (for now add/remove/query).

As a side effect, qemu-storage-daemon can now map --export directly to
the block-export-add QMP command, similar to other command line options.
The built-in NBD servers also gains new options that bring it at least a
little closer to feature parity with qemu-nbd.

v2:
- Rebased on current master
- Fixed assumption of &blk_exp_nbd instead of drv in generic code [Max]
- Documented blk_exp_request_shutdown() better [Max]
- iotests 140: Fixed race between QMP return value and event [Max]
- Improved the commit message for patch 26
- Removed copy&paste error in deprecated.rst [Max]
- iotests: Take Sequence instead of positional arguments in
  qemu_tool_pipe_and_status() [Max]
- iotests: Separate patch for qemu_nbd_list_log [Max]
- iotests 307: [Max]
  * Allow more image formats
  * Use sock_dir for the socket
  * Use f-strings instead of % operator
  * Log events after deleting an export
  * Test force removing an export

Kevin Wolf (31):
  nbd: Remove unused nbd_export_get_blockdev()
  qapi: Create block-export module
  qapi: Rename BlockExport to BlockExportOptions
  block/export: Add BlockExport infrastructure and block-export-add
  qemu-storage-daemon: Use qmp_block_export_add()
  qemu-nbd: Use raw block driver for --offset
  block/export: Remove magic from block-export-add
  nbd: Add max-connections to nbd-server-start
  nbd: Add writethrough to block-export-add
  nbd: Remove NBDExport.close callback
  qemu-nbd: Use blk_exp_add() to create the export
  nbd/server: Simplify export shutdown
  block/export: Move refcount from NBDExport to BlockExport
  block/export: Move AioContext from NBDExport to BlockExport
  block/export: Add node-name to BlockExportOptions
  block/export: Allocate BlockExport in blk_exp_add()
  block/export: Add blk_exp_close_all(_type)
  block/export: Add 'id' option to block-export-add
  block/export: Move strong user reference to block_exports
  block/export: Add block-export-del
  block/export: Add BLOCK_EXPORT_DELETED event
  block/export: Move blk to BlockExport
  block/export: Create BlockBackend in blk_exp_add()
  block/export: Add query-block-exports
  block/export: Move writable to BlockExportOptions
  nbd: Merge nbd_export_new() and nbd_export_create()
  nbd: Deprecate nbd-server-add/remove
  iotests: Factor out qemu_tool_pipe_and_status()
  iotests: Introduce qemu_nbd_list_log()
  iotests: Allow supported and unsupported formats at the same time
  iotests: Test block-export-* QMP interface

 qapi/block-core.json                 | 166 --------------
 qapi/block-export.json               | 291 ++++++++++++++++++++++++
 qapi/qapi-schema.json                |   1 +
 docs/system/deprecated.rst           |   6 +
 include/block/export.h               |  89 ++++++++
 include/block/nbd.h                  |  22 +-
 block.c                              |   2 +-
 block/export/export.c                | 325 +++++++++++++++++++++++++++
 block/monitor/block-hmp-cmds.c       |  13 +-
 blockdev-nbd.c                       | 171 +++++++-------
 nbd/server.c                         | 309 +++++++++++--------------
 qemu-nbd.c                           |  67 +++---
 storage-daemon/qemu-storage-daemon.c |  27 +--
 tests/qemu-iotests/iotests.py        |  68 +++---
 block/export/meson.build             |   1 +
 block/meson.build                    |   2 +
 meson.build                          |   2 +-
 qapi/meson.build                     |   4 +-
 storage-daemon/qapi/qapi-schema.json |   1 +
 tests/qemu-iotests/140               |   9 +-
 tests/qemu-iotests/140.out           |   2 +-
 tests/qemu-iotests/223.out           |   8 +-
 tests/qemu-iotests/307               | 132 +++++++++++
 tests/qemu-iotests/307.out           | 124 ++++++++++
 tests/qemu-iotests/group             |   1 +
 25 files changed, 1311 insertions(+), 532 deletions(-)
 create mode 100644 qapi/block-export.json
 create mode 100644 include/block/export.h
 create mode 100644 block/export/export.c
 create mode 100644 block/export/meson.build
 create mode 100755 tests/qemu-iotests/307
 create mode 100644 tests/qemu-iotests/307.out

Comments

no-reply@patchew.org Sept. 24, 2020, 5:21 p.m. UTC | #1
Patchew URL: https://patchew.org/QEMU/20200924152717.287415-1-kwolf@redhat.com/



Hi,

This series seems to have some coding style problems. See output below for
more information:

Type: series
Message-id: 20200924152717.287415-1-kwolf@redhat.com
Subject: [PATCH v2 00/31] block/export: Add infrastructure and QAPI for block exports

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

From https://github.com/patchew-project/qemu
   c122bca..f7f1d91  master     -> master
From https://github.com/patchew-project/qemu
 * [new tag]         patchew/20200924152717.287415-1-kwolf@redhat.com -> patchew/20200924152717.287415-1-kwolf@redhat.com
Switched to a new branch 'test'
772bb89 iotests: Test block-export-* QMP interface
a89a8b7 iotests: Allow supported and unsupported formats at the same time
fd15df6 iotests: Introduce qemu_nbd_list_log()
7fe6af5 iotests: Factor out qemu_tool_pipe_and_status()
67144d8 nbd: Deprecate nbd-server-add/remove
81170eb nbd: Merge nbd_export_new() and nbd_export_create()
2c8869b block/export: Move writable to BlockExportOptions
63951c4 block/export: Add query-block-exports
7f1940a block/export: Create BlockBackend in blk_exp_add()
c20f5f0 block/export: Move blk to BlockExport
46c45ed block/export: Add BLOCK_EXPORT_DELETED event
549525b block/export: Add block-export-del
deb98e0 block/export: Move strong user reference to block_exports
ad75b22 block/export: Add 'id' option to block-export-add
ffadf38 block/export: Add blk_exp_close_all(_type)
1da2124 block/export: Allocate BlockExport in blk_exp_add()
602df28 block/export: Add node-name to BlockExportOptions
b1fc3ff block/export: Move AioContext from NBDExport to BlockExport
a5d275f block/export: Move refcount from NBDExport to BlockExport
650f2eb nbd/server: Simplify export shutdown
634dc2f qemu-nbd: Use blk_exp_add() to create the export
3e1f7cf nbd: Remove NBDExport.close callback
3a41880 nbd: Add writethrough to block-export-add
f990833 nbd: Add max-connections to nbd-server-start
de1c397 block/export: Remove magic from block-export-add
9bc13b8 qemu-nbd: Use raw block driver for --offset
8eb933e qemu-storage-daemon: Use qmp_block_export_add()
38a4b01 block/export: Add BlockExport infrastructure and block-export-add
66aff14 qapi: Rename BlockExport to BlockExportOptions
026ac5d qapi: Create block-export module
b1715f6 nbd: Remove unused nbd_export_get_blockdev()

=== OUTPUT BEGIN ===
1/31 Checking commit b1715f628199 (nbd: Remove unused nbd_export_get_blockdev())
2/31 Checking commit 026ac5dda678 (qapi: Create block-export module)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#231: 
new file mode 100644

total: 0 errors, 1 warnings, 410 lines checked

Patch 2/31 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
3/31 Checking commit 66aff14308cb (qapi: Rename BlockExport to BlockExportOptions)
4/31 Checking commit 38a4b017ec72 (block/export: Add BlockExport infrastructure and block-export-add)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#21: 
new file mode 100644

ERROR: "(foo*)" should be "(foo *)"
#156: FILE: blockdev-nbd.c:224:
+    return (BlockExport*) exp;

total: 1 errors, 1 warnings, 247 lines checked

Patch 4/31 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

5/31 Checking commit 8eb933e1e3f7 (qemu-storage-daemon: Use qmp_block_export_add())
6/31 Checking commit 9bc13b8dba46 (qemu-nbd: Use raw block driver for --offset)
7/31 Checking commit de1c39724d9e (block/export: Remove magic from block-export-add)
8/31 Checking commit f99083319892 (nbd: Add max-connections to nbd-server-start)
9/31 Checking commit 3a418807fc14 (nbd: Add writethrough to block-export-add)
10/31 Checking commit 3e1f7cf80488 (nbd: Remove NBDExport.close callback)
11/31 Checking commit 634dc2f6f2ec (qemu-nbd: Use blk_exp_add() to create the export)
12/31 Checking commit 650f2eba0db8 (nbd/server: Simplify export shutdown)
13/31 Checking commit a5d275f4c538 (block/export: Move refcount from NBDExport to BlockExport)
ERROR: "(foo*)" should be "(foo *)"
#49: FILE: blockdev-nbd.c:239:
+    blk_exp_unref((BlockExport*) exp);

total: 1 errors, 0 warnings, 214 lines checked

Patch 13/31 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

14/31 Checking commit b1fc3ffc0ccd (block/export: Move AioContext from NBDExport to BlockExport)
15/31 Checking commit 602df286881c (block/export: Add node-name to BlockExportOptions)
16/31 Checking commit 1da2124e164e (block/export: Allocate BlockExport in blk_exp_add())
17/31 Checking commit ffadf38dd6c3 (block/export: Add blk_exp_close_all(_type))
18/31 Checking commit ad75b22688f5 (block/export: Add 'id' option to block-export-add)
19/31 Checking commit deb98e00621d (block/export: Move strong user reference to block_exports)
20/31 Checking commit 549525bb9497 (block/export: Add block-export-del)
21/31 Checking commit 46c45ed62a8d (block/export: Add BLOCK_EXPORT_DELETED event)
22/31 Checking commit c20f5f0cba91 (block/export: Move blk to BlockExport)
23/31 Checking commit 7f1940ac70c5 (block/export: Create BlockBackend in blk_exp_add())
24/31 Checking commit 63951c4d2fbd (block/export: Add query-block-exports)
25/31 Checking commit 2c8869bd86fb (block/export: Move writable to BlockExportOptions)
26/31 Checking commit 81170eb030be (nbd: Merge nbd_export_new() and nbd_export_create())
27/31 Checking commit 67144d8ca22d (nbd: Deprecate nbd-server-add/remove)
28/31 Checking commit 7fe6af5d135c (iotests: Factor out qemu_tool_pipe_and_status())
29/31 Checking commit fd15df623a84 (iotests: Introduce qemu_nbd_list_log())
30/31 Checking commit a89a8b774b81 (iotests: Allow supported and unsupported formats at the same time)
31/31 Checking commit 772bb8914d39 (iotests: Test block-export-* QMP interface)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#11: 
new file mode 100755

total: 0 errors, 1 warnings, 260 lines checked

Patch 31/31 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
=== OUTPUT END ===

Test command exited with code: 1


The full log is available at
http://patchew.org/logs/20200924152717.287415-1-kwolf@redhat.com/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
Stefan Hajnoczi Sept. 25, 2020, 8:45 a.m. UTC | #2
On Thu, Sep 24, 2020 at 10:21:56AM -0700, no-reply@patchew.org wrote:
> ERROR: "(foo*)" should be "(foo *)"
> #156: FILE: blockdev-nbd.c:224:
> +    return (BlockExport*) exp;
[...]
> ERROR: "(foo*)" should be "(foo *)"
> #49: FILE: blockdev-nbd.c:239:
> +    blk_exp_unref((BlockExport*) exp);

These can be fixed when merging the patches.

Stefan
Stefan Hajnoczi Sept. 25, 2020, 8:51 a.m. UTC | #3
On Thu, Sep 24, 2020 at 05:26:46PM +0200, Kevin Wolf wrote:
> We are planning to add more block export types than just NBD in the near
> future (e.g. vhost-user-blk and FUSE). This series lays the ground for
> this with some generic block export infrastructure and QAPI interfaces
> that will allow managing all of them (for now add/remove/query).
> 
> As a side effect, qemu-storage-daemon can now map --export directly to
> the block-export-add QMP command, similar to other command line options.
> The built-in NBD servers also gains new options that bring it at least a
> little closer to feature parity with qemu-nbd.
> 
> v2:
> - Rebased on current master
> - Fixed assumption of &blk_exp_nbd instead of drv in generic code [Max]
> - Documented blk_exp_request_shutdown() better [Max]
> - iotests 140: Fixed race between QMP return value and event [Max]
> - Improved the commit message for patch 26
> - Removed copy&paste error in deprecated.rst [Max]
> - iotests: Take Sequence instead of positional arguments in
>   qemu_tool_pipe_and_status() [Max]
> - iotests: Separate patch for qemu_nbd_list_log [Max]
> - iotests 307: [Max]
>   * Allow more image formats
>   * Use sock_dir for the socket
>   * Use f-strings instead of % operator
>   * Log events after deleting an export
>   * Test force removing an export

I rebased and retested the vhost-user-blk-server block exports series on
top of this:

Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Kevin Wolf Sept. 28, 2020, 3:29 p.m. UTC | #4
Am 24.09.2020 um 17:26 hat Kevin Wolf geschrieben:
> We are planning to add more block export types than just NBD in the near
> future (e.g. vhost-user-blk and FUSE). This series lays the ground for
> this with some generic block export infrastructure and QAPI interfaces
> that will allow managing all of them (for now add/remove/query).
> 
> As a side effect, qemu-storage-daemon can now map --export directly to
> the block-export-add QMP command, similar to other command line options.
> The built-in NBD servers also gains new options that bring it at least a
> little closer to feature parity with qemu-nbd.

Thanks for the review, applied to the block branch.

Kevin