mbox series

[PULL,v2] Block layer patches

Message ID 20200709121713.283092-1-kwolf@redhat.com
State New, archived
Headers show
Series [PULL,v2] Block layer patches | expand

Pull-request

git://repo.or.cz/qemu/kevin.git tags/for-upstream

Message

Kevin Wolf July 9, 2020, 12:17 p.m. UTC
The following changes since commit 8796c64ecdfd34be394ea277aaaaa53df0c76996:

  Merge remote-tracking branch 'remotes/kraxel/tags/audio-20200706-pull-request' into staging (2020-07-08 16:33:59 +0100)

are available in the Git repository at:

  git://repo.or.cz/qemu/kevin.git tags/for-upstream

for you to fetch changes up to d1c824e681e423a6224c6831a493e9175fa02dc1:

  qemu-img: Deprecate use of -b without -F (2020-07-09 14:14:55 +0200)

----------------------------------------------------------------
Block layer patches:

- file-posix: Mitigate file fragmentation with extent size hints
- Tighten qemu-img rules on missing backing format
- qemu-img map: Don't limit block status request size

----------------------------------------------------------------
Eric Blake (10):
      qemu-img: Flush stdout before before potential stderr messages
      block: Finish deprecation of 'qemu-img convert -n -o'
      sheepdog: Add trivial backing_fmt support
      vmdk: Add trivial backing_fmt support
      qcow: Tolerate backing_fmt=
      block: Error if backing file fails during creation without -u
      qcow2: Deprecate use of qemu-img amend to change backing file
      iotests: Specify explicit backing format where sensible
      block: Add support to warn on backing file change without format
      qemu-img: Deprecate use of -b without -F

Kevin Wolf (2):
      qemu-img map: Don't limit block status request size
      file-posix: Mitigate file fragmentation with extent size hints

Max Reitz (1):
      iotests: Simplify _filter_img_create() a bit

 qapi/block-core.json             | 11 +++--
 docs/system/deprecated.rst       | 58 ++++++++++++++++++++++----
 docs/tools/qemu-img.rst          |  4 ++
 include/block/block.h            |  4 +-
 include/block/block_int.h        |  1 +
 block.c                          | 53 +++++++++++++++++-------
 block/file-posix.c               | 42 +++++++++++++++++++
 block/qcow.c                     | 20 ++++++++-
 block/qcow2.c                    |  7 +++-
 block/sheepdog.c                 | 18 +++++++-
 block/stream.c                   |  2 +-
 block/vmdk.c                     | 14 +++++++
 blockdev.c                       |  3 +-
 qemu-img.c                       | 20 +++++----
 tests/qemu-iotests/017           |  2 +-
 tests/qemu-iotests/017.out       |  2 +-
 tests/qemu-iotests/018           |  2 +-
 tests/qemu-iotests/018.out       |  2 +-
 tests/qemu-iotests/019           |  5 ++-
 tests/qemu-iotests/019.out       |  2 +-
 tests/qemu-iotests/020           | 31 ++++++++------
 tests/qemu-iotests/020.out       | 15 +------
 tests/qemu-iotests/024           |  8 ++--
 tests/qemu-iotests/024.out       |  5 ++-
 tests/qemu-iotests/028           |  4 +-
 tests/qemu-iotests/028.out       |  2 +-
 tests/qemu-iotests/030           | 26 ++++++++----
 tests/qemu-iotests/034           |  2 +-
 tests/qemu-iotests/034.out       |  2 +-
 tests/qemu-iotests/037           |  2 +-
 tests/qemu-iotests/037.out       |  2 +-
 tests/qemu-iotests/038           |  2 +-
 tests/qemu-iotests/038.out       |  2 +-
 tests/qemu-iotests/039           |  3 +-
 tests/qemu-iotests/039.out       |  2 +-
 tests/qemu-iotests/040           | 47 +++++++++++++++------
 tests/qemu-iotests/041           | 37 +++++++++++------
 tests/qemu-iotests/042           |  4 +-
 tests/qemu-iotests/043           | 18 ++++----
 tests/qemu-iotests/043.out       | 16 +++++---
 tests/qemu-iotests/046           |  2 +-
 tests/qemu-iotests/046.out       |  2 +-
 tests/qemu-iotests/049.out       |  8 ++--
 tests/qemu-iotests/050           |  4 +-
 tests/qemu-iotests/050.out       |  2 +-
 tests/qemu-iotests/051           |  2 +-
 tests/qemu-iotests/051.out       |  2 +-
 tests/qemu-iotests/051.pc.out    |  2 +-
 tests/qemu-iotests/054.out       |  2 +-
 tests/qemu-iotests/056           |  3 +-
 tests/qemu-iotests/060           |  2 +-
 tests/qemu-iotests/060.out       |  2 +-
 tests/qemu-iotests/061           | 10 ++---
 tests/qemu-iotests/061.out       | 11 ++---
 tests/qemu-iotests/069           |  2 +-
 tests/qemu-iotests/069.out       |  2 +-
 tests/qemu-iotests/073           |  2 +-
 tests/qemu-iotests/073.out       |  2 +-
 tests/qemu-iotests/079.out       |  2 +-
 tests/qemu-iotests/082           | 10 +++--
 tests/qemu-iotests/082.out       | 30 +++++++++++---
 tests/qemu-iotests/085           |  4 +-
 tests/qemu-iotests/085.out       |  6 +--
 tests/qemu-iotests/089           |  2 +-
 tests/qemu-iotests/089.out       |  2 +-
 tests/qemu-iotests/095           |  4 +-
 tests/qemu-iotests/095.out       |  4 +-
 tests/qemu-iotests/097           |  4 +-
 tests/qemu-iotests/097.out       | 16 ++++----
 tests/qemu-iotests/098           |  2 +-
 tests/qemu-iotests/098.out       |  8 ++--
 tests/qemu-iotests/110           |  4 +-
 tests/qemu-iotests/110.out       |  4 +-
 tests/qemu-iotests/111.out       |  2 +-
 tests/qemu-iotests/112.out       |  4 +-
 tests/qemu-iotests/114           | 12 ++++++
 tests/qemu-iotests/114.out       |  9 ++++
 tests/qemu-iotests/122           | 34 +++++++++++-----
 tests/qemu-iotests/122.out       | 12 ++++--
 tests/qemu-iotests/126           |  4 +-
 tests/qemu-iotests/126.out       |  4 +-
 tests/qemu-iotests/127           |  4 +-
 tests/qemu-iotests/127.out       |  4 +-
 tests/qemu-iotests/129           |  3 +-
 tests/qemu-iotests/133           |  2 +-
 tests/qemu-iotests/133.out       |  2 +-
 tests/qemu-iotests/139           |  2 +-
 tests/qemu-iotests/141           |  6 +--
 tests/qemu-iotests/141.out       |  4 +-
 tests/qemu-iotests/142           |  2 +-
 tests/qemu-iotests/142.out       |  2 +-
 tests/qemu-iotests/153           | 14 +++----
 tests/qemu-iotests/153.out       | 35 ++++++++--------
 tests/qemu-iotests/154           | 42 +++++++++----------
 tests/qemu-iotests/154.out       | 42 +++++++++----------
 tests/qemu-iotests/155           | 12 ++++--
 tests/qemu-iotests/156           |  9 ++--
 tests/qemu-iotests/156.out       |  6 +--
 tests/qemu-iotests/158           |  2 +-
 tests/qemu-iotests/158.out       |  2 +-
 tests/qemu-iotests/161           |  8 ++--
 tests/qemu-iotests/161.out       |  8 ++--
 tests/qemu-iotests/176           |  4 +-
 tests/qemu-iotests/176.out       | 32 +++++++--------
 tests/qemu-iotests/177           |  2 +-
 tests/qemu-iotests/177.out       |  2 +-
 tests/qemu-iotests/179           |  2 +-
 tests/qemu-iotests/179.out       |  2 +-
 tests/qemu-iotests/189           |  2 +-
 tests/qemu-iotests/189.out       |  2 +-
 tests/qemu-iotests/191           | 12 +++---
 tests/qemu-iotests/191.out       | 12 +++---
 tests/qemu-iotests/195           |  6 +--
 tests/qemu-iotests/195.out       |  6 +--
 tests/qemu-iotests/198           |  2 +-
 tests/qemu-iotests/198.out       |  3 +-
 tests/qemu-iotests/204           |  2 +-
 tests/qemu-iotests/204.out       |  2 +-
 tests/qemu-iotests/216           |  2 +-
 tests/qemu-iotests/224           |  4 +-
 tests/qemu-iotests/225           |  2 +-
 tests/qemu-iotests/225.out       |  2 +-
 tests/qemu-iotests/228           |  5 ++-
 tests/qemu-iotests/243           |  7 ++--
 tests/qemu-iotests/245           |  3 +-
 tests/qemu-iotests/249           |  4 +-
 tests/qemu-iotests/249.out       |  4 +-
 tests/qemu-iotests/252           |  2 +-
 tests/qemu-iotests/257           |  3 +-
 tests/qemu-iotests/259.out       |  2 +-
 tests/qemu-iotests/267           |  4 +-
 tests/qemu-iotests/267.out       |  6 +--
 tests/qemu-iotests/270           |  2 +-
 tests/qemu-iotests/270.out       |  2 +-
 tests/qemu-iotests/273           |  4 +-
 tests/qemu-iotests/273.out       |  4 +-
 tests/qemu-iotests/274           | 12 +++---
 tests/qemu-iotests/274.out       | 29 +++++++------
 tests/qemu-iotests/279           |  4 +-
 tests/qemu-iotests/279.out       |  4 +-
 tests/qemu-iotests/290           |  2 +-
 tests/qemu-iotests/290.out       |  4 +-
 tests/qemu-iotests/301           | 88 ++++++++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/301.out       | 59 +++++++++++++++++++++++++++
 tests/qemu-iotests/common.filter | 62 ++++++++++++----------------
 tests/qemu-iotests/group         |  1 +
 146 files changed, 906 insertions(+), 472 deletions(-)
 create mode 100755 tests/qemu-iotests/301
 create mode 100644 tests/qemu-iotests/301.out

Comments

Peter Maydell July 10, 2020, 3:41 p.m. UTC | #1
On Thu, 9 Jul 2020 at 13:17, Kevin Wolf <kwolf@redhat.com> wrote:
>
> The following changes since commit 8796c64ecdfd34be394ea277aaaaa53df0c76996:
>
>   Merge remote-tracking branch 'remotes/kraxel/tags/audio-20200706-pull-request' into staging (2020-07-08 16:33:59 +0100)
>
> are available in the Git repository at:
>
>   git://repo.or.cz/qemu/kevin.git tags/for-upstream
>
> for you to fetch changes up to d1c824e681e423a6224c6831a493e9175fa02dc1:
>
>   qemu-img: Deprecate use of -b without -F (2020-07-09 14:14:55 +0200)
>
> ----------------------------------------------------------------
> Block layer patches:
>
> - file-posix: Mitigate file fragmentation with extent size hints
> - Tighten qemu-img rules on missing backing format
> - qemu-img map: Don't limit block status request size
>

iotest 114 fails on FreeBSD and OpenBSD (and probably NetBSD
but that build hasn't reported back yet): looks like a
non-portable use of 'truncate' ?

  TEST    iotest-qcow2: 114 [fail]
QEMU          --
"/home/qemu/qemu-test.FOdDM2/build/tests/qemu-iotests/../../aarch64-softmmu/qemu-system-aarch64"
-nodefaults -disp
lay none -machine virt -accel qtest
QEMU_IMG      --
"/home/qemu/qemu-test.FOdDM2/build/tests/qemu-iotests/../../qemu-img"
QEMU_IO       --
"/home/qemu/qemu-test.FOdDM2/build/tests/qemu-iotests/../../qemu-io"
--cache writeback --aio threads -f qcow2
QEMU_NBD      --
"/home/qemu/qemu-test.FOdDM2/build/tests/qemu-iotests/../../qemu-nbd"
IMGFMT        -- qcow2 (compat=1.1)
IMGPROTO      -- file
PLATFORM      -- FreeBSD/amd64 freebsd 12.1-RELEASE
TEST_DIR      -- /home/qemu/qemu-test.FOdDM2/build/tests/qemu-iotests/scratch
SOCK_DIR      -- /tmp/tmp.RQQ2LyuG
SOCKET_SCM_HELPER --

--- /home/qemu/qemu-test.FOdDM2/src/tests/qemu-iotests/114.out
2020-07-10 14:51:21.000000000 +0000
+++ /home/qemu/qemu-test.FOdDM2/build/tests/qemu-iotests/114.out.bad
 2020-07-10 15:15:14.446278000 +0000
@@ -1,8 +1,11 @@
 QA output created by 114
-qemu-img: warning: Deprecated use of backing file without explicit
backing format (detected format of raw)
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
backing_file=TEST_DIR/t.IMGFMT.orig
+truncate: illegal option -- -
+usage: truncate [-c] -s [+|-|%|/]size[K|k|M|m|G|g|T|t] file ...
+       truncate [-c] -r rfile file ...
+qemu-img: TEST_DIR/t.IMGFMT: Could not open 'TEST_DIR/t.IMGFMT.orig':
No such file or directory
+Could not open backing image.
 Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=67108864
-qemu-img: warning: Deprecated use of backing file without explicit
backing format
+qemu-img: Could not open
'/home/qemu/qemu-test.FOdDM2/build/tests/qemu-iotests/scratch/t.qcow2.orig':
Could not open '/home/qemu/qemu-test.FOdDM2/build/tests/qemu-iotests/scratch/t.qcow2.orig':
No such file or directory
 qemu-img: warning: Deprecated use of backing file without explicit
backing format (detected format of IMGFMT)
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=IMGFMT
 qemu-img: warning: Deprecated use of unopened backing file without
explicit backing format, use of this image requires potentially unsafe
format probing

thanks
-- PMM
Eric Blake July 10, 2020, 4:05 p.m. UTC | #2
On 7/10/20 10:41 AM, Peter Maydell wrote:

>>    qemu-img: Deprecate use of -b without -F (2020-07-09 14:14:55 +0200)
>>
>> ----------------------------------------------------------------
>> Block layer patches:
>>
>> - file-posix: Mitigate file fragmentation with extent size hints
>> - Tighten qemu-img rules on missing backing format
>> - qemu-img map: Don't limit block status request size
>>
> 
> iotest 114 fails on FreeBSD and OpenBSD (and probably NetBSD
> but that build hasn't reported back yet): looks like a
> non-portable use of 'truncate' ?
> 

> @@ -1,8 +1,11 @@
>   QA output created by 114
> -qemu-img: warning: Deprecated use of backing file without explicit
> backing format (detected format of raw)
> -Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
> backing_file=TEST_DIR/t.IMGFMT.orig
> +truncate: illegal option -- -
> +usage: truncate [-c] -s [+|-|%|/]size[K|k|M|m|G|g|T|t] file ...
> +       truncate [-c] -r rfile file ...
> +qemu-img: TEST_DIR/t.IMGFMT: Could not open 'TEST_DIR/t.IMGFMT.orig':

Yep, 'truncate' is not POSIX, but we've been lucky that most of our uses 
have been under '_supported_os Linux'; test 272 runs on BSD with 
'truncate -s' instead of 'truncate --size'.  The fix is obvious; I can 
post the followup to squash in if that will help.
Eric Blake July 10, 2020, 4:14 p.m. UTC | #3
On 7/9/20 7:17 AM, Kevin Wolf wrote:
> The following changes since commit 8796c64ecdfd34be394ea277aaaaa53df0c76996:
> 
>    Merge remote-tracking branch 'remotes/kraxel/tags/audio-20200706-pull-request' into staging (2020-07-08 16:33:59 +0100)
> 
> are available in the Git repository at:
> 
>    git://repo.or.cz/qemu/kevin.git tags/for-upstream
> 
> for you to fetch changes up to d1c824e681e423a6224c6831a493e9175fa02dc1:
> 
>    qemu-img: Deprecate use of -b without -F (2020-07-09 14:14:55 +0200)
> 
> ----------------------------------------------------------------
> Block layer patches:
> 
> - file-posix: Mitigate file fragmentation with extent size hints
> - Tighten qemu-img rules on missing backing format
> - qemu-img map: Don't limit block status request size
> 
> ----------------------------------------------------------------
> Eric Blake (10):
>        qemu-img: Flush stdout before before potential stderr messages
>        block: Finish deprecation of 'qemu-img convert -n -o'
>        sheepdog: Add trivial backing_fmt support
>        vmdk: Add trivial backing_fmt support
>        qcow: Tolerate backing_fmt=

I see you renumbered my test 293 to 301 for conflict resolution, but did 
not update the commit message to call out the updated number ;)

Since we may have to do a v3 because of the 'truncate -s' issue in 114, 
you could touch that up too.


>   tests/qemu-iotests/301           | 88 ++++++++++++++++++++++++++++++++++++++++
>   tests/qemu-iotests/301.out       | 59 +++++++++++++++++++++++++++
>   tests/qemu-iotests/common.filter | 62 ++++++++++++----------------
>   tests/qemu-iotests/group         |  1 +
>   146 files changed, 906 insertions(+), 472 deletions(-)
>   create mode 100755 tests/qemu-iotests/301
>   create mode 100644 tests/qemu-iotests/301.out
> 
>