mbox series

[PULL,v2] Block layer patches

Message ID 20200908115322.325832-1-kwolf@redhat.com
State New
Headers show
Series [PULL,v2] Block layer patches | expand

Pull-request

git://repo.or.cz/qemu/kevin.git tags/for-upstream

Message

Kevin Wolf Sept. 8, 2020, 11:53 a.m. UTC
The following changes since commit 7c37270b3fbe3d034ba80e488761461676e21eb4:

  Merge remote-tracking branch 'remotes/kraxel/tags/ui-20200904-pull-request' into staging (2020-09-06 16:23:55 +0100)

are available in the Git repository at:

  git://repo.or.cz/qemu/kevin.git tags/for-upstream

for you to fetch changes up to c984095a47c30e0952d34e77decf9f4c0f8d5a19:

  block/nvme: Pair doorbell registers (2020-09-08 13:40:53 +0200)

----------------------------------------------------------------
Block layer patches:

- qemu-img create: Fail gracefully when backing file is an empty string
- Fixes related to filter block nodes ("Deal with filters" series)
- block/nvme: Various cleanups required to use multiple queues
- block/nvme: Use NvmeBar structure from "block/nvme.h"
- file-win32: Fix "locking" option
- iotests: Allow running from different directory

----------------------------------------------------------------
Connor Kuehl (1):
      block: Raise an error when backing file parameter is an empty string

Kevin Wolf (2):
      iotests: Allow running from different directory
      file-win32: Fix "locking" option

Max Reitz (43):
      block: Add child access functions
      block: Add chain helper functions
      block: bdrv_cow_child() for bdrv_has_zero_init()
      block: bdrv_set_backing_hd() is about bs->backing
      block: Include filters when freezing backing chain
      block: Drop bdrv_is_encrypted()
      block: Add bdrv_supports_compressed_writes()
      throttle: Support compressed writes
      copy-on-read: Support compressed writes
      block: Use bdrv_filter_(bs|child) where obvious
      block: Use CAFs in block status functions
      stream: Deal with filters
      block: Use CAFs when working with backing chains
      block: Use bdrv_cow_child() in bdrv_co_truncate()
      block: Re-evaluate backing file handling in reopen
      block: Flush all children in generic code
      vmdk: Drop vmdk_co_flush()
      block: Iterate over children in refresh_limits
      block: Use CAFs in bdrv_refresh_filename()
      block: Use CAF in bdrv_co_rw_vmstate()
      block/snapshot: Fix fallback
      block: Use CAFs for debug breakpoints
      block: Improve get_allocated_file_size's default
      block/null: Implement bdrv_get_allocated_file_size
      blockdev: Use CAF in external_snapshot_prepare()
      block: Report data child for query-blockstats
      block: Use child access functions for QAPI queries
      block-copy: Use CAF to find sync=top base
      mirror: Deal with filters
      backup: Deal with filters
      commit: Deal with filters
      nbd: Use CAF when looking for dirty bitmap
      qemu-img: Use child access functions
      block: Drop backing_bs()
      blockdev: Fix active commit choice
      block: Inline bdrv_co_block_status_from_*()
      block: Leave BDS.backing_{file,format} constant
      iotests: Test that qcow2's data-file is flushed
      iotests: Let complete_and_wait() work with commit
      iotests: Add filter commit test cases
      iotests: Add filter mirror test cases
      iotests: Add test for commit in sub directory
      iotests: Test committing to overridden backing

Philippe Mathieu-Daudé (18):
      block/nvme: Replace magic value by SCALE_MS definition
      block/nvme: Avoid further processing if trace event not enabled
      block/nvme: Let nvme_create_queue_pair() fail gracefully
      block/nvme: Define INDEX macros to ease code review
      block/nvme: Improve error message when IO queue creation failed
      block/nvme: Use common error path in nvme_add_io_queue()
      block/nvme: Rename local variable
      block/nvme: Use union of NvmeIdCtrl / NvmeIdNs structures
      block/nvme: Replace qemu_try_blockalign0 by qemu_try_blockalign/memset
      block/nvme: Replace qemu_try_blockalign(bs) by qemu_try_memalign(pg_sz)
      block/nvme: Simplify nvme_init_queue() arguments
      block/nvme: Replace BDRV_POLL_WHILE by AIO_WAIT_WHILE
      block/nvme: Simplify nvme_create_queue_pair() arguments
      block/nvme: Extract nvme_poll_queue()
      block/nvme: Use an array of EventNotifier
      block/nvme: Group controller registers in NVMeRegs structure
      block/nvme: Use generic NvmeBar structure
      block/nvme: Pair doorbell registers

 qapi/block-core.json           |  58 +++--
 include/block/block.h          |   2 +-
 include/block/block_int.h      |  95 +++++---
 block.c                        | 492 +++++++++++++++++++++++++++++++++--------
 block/backup-top.c             |   4 +-
 block/backup.c                 |   9 +-
 block/blkdebug.c               |   7 +-
 block/blklogwrites.c           |   1 -
 block/block-backend.c          |   7 +-
 block/block-copy.c             |   4 +-
 block/commit.c                 |  95 +++++---
 block/copy-on-read.c           |  13 +-
 block/file-win32.c             |  22 +-
 block/filter-compress.c        |   2 -
 block/io.c                     | 142 ++++++------
 block/mirror.c                 | 119 +++++++---
 block/monitor/block-hmp-cmds.c |   2 +-
 block/null.c                   |   7 +
 block/nvme.c                   | 248 +++++++++++----------
 block/qapi.c                   |  74 +++++--
 block/snapshot.c               | 104 +++++++--
 block/stream.c                 |  63 ++++--
 block/throttle.c               |  11 +-
 block/vmdk.c                   |  16 --
 blockdev.c                     | 101 +++++++--
 migration/block-dirty-bitmap.c |   8 +-
 nbd/server.c                   |   6 +-
 qemu-img.c                     |  43 ++--
 tests/qemu-iotests/iotests.py  |  10 +-
 tests/qemu-iotests/020         |  44 ++++
 tests/qemu-iotests/020.out     |  10 +
 tests/qemu-iotests/040         | 238 ++++++++++++++++++++
 tests/qemu-iotests/040.out     |   4 +-
 tests/qemu-iotests/041         | 146 +++++++++++-
 tests/qemu-iotests/041.out     |   4 +-
 tests/qemu-iotests/049         |   4 +
 tests/qemu-iotests/049.out     |   5 +
 tests/qemu-iotests/153.out     |   2 +-
 tests/qemu-iotests/184         |   3 +-
 tests/qemu-iotests/184.out     |  14 +-
 tests/qemu-iotests/204.out     |   1 +
 tests/qemu-iotests/228         |   6 +-
 tests/qemu-iotests/228.out     |   6 +-
 tests/qemu-iotests/244         |  49 ++++
 tests/qemu-iotests/244.out     |   7 +
 tests/qemu-iotests/245         |   4 +-
 tests/qemu-iotests/273.out     |   4 +-
 tests/qemu-iotests/check       |   2 +-
 48 files changed, 1763 insertions(+), 555 deletions(-)

Comments

Peter Maydell Sept. 9, 2020, 9:55 p.m. UTC | #1
On Tue, 8 Sep 2020 at 12:53, Kevin Wolf <kwolf@redhat.com> wrote:
>
> The following changes since commit 7c37270b3fbe3d034ba80e488761461676e21eb4:
>
>   Merge remote-tracking branch 'remotes/kraxel/tags/ui-20200904-pull-request' into staging (2020-09-06 16:23:55 +0100)
>
> are available in the Git repository at:
>
>   git://repo.or.cz/qemu/kevin.git tags/for-upstream
>
> for you to fetch changes up to c984095a47c30e0952d34e77decf9f4c0f8d5a19:
>
>   block/nvme: Pair doorbell registers (2020-09-08 13:40:53 +0200)
>
> ----------------------------------------------------------------
> Block layer patches:
>
> - qemu-img create: Fail gracefully when backing file is an empty string
> - Fixes related to filter block nodes ("Deal with filters" series)
> - block/nvme: Various cleanups required to use multiple queues
> - block/nvme: Use NvmeBar structure from "block/nvme.h"
> - file-win32: Fix "locking" option
> - iotests: Allow running from different directory

This fails 'make check' on NetBSD and OpenBSD:

./check: line 47: realpath: command not found
./check: line 60: /common.env: No such file or directory
check: failed to source common.env (make sure the qemu-iotests are run
from tests/qemu-iotests in the build tree)
gmake: *** [/home/qemu/qemu-test.vcb7nz/src/tests/Makefile.include:144:
check-block] Error 1

thanks
-- PMM
Eric Blake Sept. 9, 2020, 10:09 p.m. UTC | #2
On 9/9/20 4:55 PM, Peter Maydell wrote:

> 
> This fails 'make check' on NetBSD and OpenBSD:
> 
> ./check: line 47: realpath: command not found
> ./check: line 60: /common.env: No such file or directory
> check: failed to source common.env (make sure the qemu-iotests are run
> from tests/qemu-iotests in the build tree)
> gmake: *** [/home/qemu/qemu-test.vcb7nz/src/tests/Makefile.include:144:
> check-block] Error 1

BSD has 'readlink -f' (and so does coreutils on Linux), which does the 
same thing as the Linux-only realpath.
Kevin Wolf Sept. 10, 2020, 9:13 a.m. UTC | #3
Am 10.09.2020 um 00:09 hat Eric Blake geschrieben:
> On 9/9/20 4:55 PM, Peter Maydell wrote:
> 
> > 
> > This fails 'make check' on NetBSD and OpenBSD:
> > 
> > ./check: line 47: realpath: command not found
> > ./check: line 60: /common.env: No such file or directory
> > check: failed to source common.env (make sure the qemu-iotests are run
> > from tests/qemu-iotests in the build tree)
> > gmake: *** [/home/qemu/qemu-test.vcb7nz/src/tests/Makefile.include:144:
> > check-block] Error 1
> 
> BSD has 'readlink -f' (and so does coreutils on Linux), which does the same
> thing as the Linux-only realpath.

Thanks, I'll use that in v3.

Max, if v3 doesn't go through either, please take over the pull request
as I will be on PTO until the 22th.

Kevin
Max Reitz Sept. 10, 2020, 11:54 a.m. UTC | #4
On 10.09.20 11:13, Kevin Wolf wrote:
> Am 10.09.2020 um 00:09 hat Eric Blake geschrieben:
>> On 9/9/20 4:55 PM, Peter Maydell wrote:
>>
>>>
>>> This fails 'make check' on NetBSD and OpenBSD:
>>>
>>> ./check: line 47: realpath: command not found
>>> ./check: line 60: /common.env: No such file or directory
>>> check: failed to source common.env (make sure the qemu-iotests are run
>>> from tests/qemu-iotests in the build tree)
>>> gmake: *** [/home/qemu/qemu-test.vcb7nz/src/tests/Makefile.include:144:
>>> check-block] Error 1
>>
>> BSD has 'readlink -f' (and so does coreutils on Linux), which does the same
>> thing as the Linux-only realpath.
> 
> Thanks, I'll use that in v3.
> 
> Max, if v3 doesn't go through either, please take over the pull request
> as I will be on PTO until the 22th.

OK, sure.

(Enjoy your time off. :))

Max
Thomas Huth Sept. 12, 2020, 12:27 p.m. UTC | #5
On 10/09/2020 00.09, Eric Blake wrote:
> On 9/9/20 4:55 PM, Peter Maydell wrote:
> 
>>
>> This fails 'make check' on NetBSD and OpenBSD:
>>
>> ./check: line 47: realpath: command not found
>> ./check: line 60: /common.env: No such file or directory
>> check: failed to source common.env (make sure the qemu-iotests are run
>> from tests/qemu-iotests in the build tree)
>> gmake: *** [/home/qemu/qemu-test.vcb7nz/src/tests/Makefile.include:144:
>> check-block] Error 1
> 
> BSD has 'readlink -f' (and so does coreutils on Linux), which does the
> same thing as the Linux-only realpath.

Seems like readlink -f does not work on macOS:

 https://cirrus-ci.com/task/5735398972325888?command=main#L7038

Any ideas what to use instead?

Peter, why did this slip through your merge tests, do you still skip the
iotests there?

 Thomas
Peter Maydell Sept. 12, 2020, 6:38 p.m. UTC | #6
On Sat, 12 Sep 2020 at 13:27, Thomas Huth <thuth@redhat.com> wrote:
> Peter, why did this slip through your merge tests, do you still skip the
> iotests there?

I forget what the reason for them being skipped is, maybe
it's because they demand a gnu sed ?

thanks
-- PMM
Thomas Huth Sept. 14, 2020, 5:36 a.m. UTC | #7
On 12/09/2020 20.38, Peter Maydell wrote:
> On Sat, 12 Sep 2020 at 13:27, Thomas Huth <thuth@redhat.com> wrote:
>> Peter, why did this slip through your merge tests, do you still skip the
>> iotests there?
> 
> I forget what the reason for them being skipped is, maybe
> it's because they demand a gnu sed ?

The tests/check-block.sh script should tell you when you run "make
check-block" ... but yes, they need gnu sed - which should be available
via homebrew.

 Thomas