mbox

[PULL,00/26] Misc patches for 2021-05-20

Message ID 20210520082257.187061-1-pbonzini@redhat.com (mailing list archive)
State New, archived
Headers show

Pull-request

https://gitlab.com/bonzini/qemu.git tags/tags/for-upstream-v2

Message

Paolo Bonzini May 20, 2021, 8:22 a.m. UTC
The following changes since commit 15e147b3c778f9f4c08c79b99747b848b6e2117b:

  Merge remote-tracking branch 'remotes/nvme/tags/nvme-next-pull-request' into staging (2021-05-18 12:22:16 +0100)

are available in the Git repository at:

  https://gitlab.com/bonzini/qemu.git tags/tags/for-upstream-v2

for you to fetch changes up to 976551ba504a3284f9ab9862d9434b422c6841a1:

  scsi-generic: pass max_segments via max_iov field in BlockLimits (2021-05-18 17:51:15 +0200)

----------------------------------------------------------------
* submodule cleanups (Philippe, myself)
* tiny step towards a usable preconfig mode (myself)
* bump Meson submodule (myself)
* Kconfig and LOCK_GUARD cleanups (philippe)
* new x86 CPUID feature (Yang Zhong)
* "-object qtest" support (myself)
* Dirty ring support for KVM (Peter)
* Fixes for 6.0 command line parsing breakage (myself)
* Fix for macOS 11.3 SDK (Katsuhiro)
* Fix for scsi-generic handling (myself)

----------------------------------------------------------------
Katsuhiro Ueno (1):
      meson: Set implicit_include_directories to false

Paolo Bonzini (11):
      configure: check for submodules if --with-git-submodules=ignore
      configure: simplify assignment to GIT_SUBMODULES
      meson: bump submodule to 0.57.2
      object: add more commands to preconfig mode
      qtest: add a QOM object for qtest
      KVM: do not allow setting properties at runtime
      remove qemu-options* from root directory
      replication: move include out of root directory
      vl: allow not specifying size in -m when using -M memory-backend
      qemu-config: load modules when instantiating option groups
      scsi-generic: pass max_segments via max_iov field in BlockLimits

Peter Xu (10):
      memory: Introduce log_sync_global() to memory listener
      KVM: Use a big lock to replace per-kml slots_lock
      KVM: Create the KVMSlot dirty bitmap on flag changes
      KVM: Provide helper to get kvm dirty log
      KVM: Provide helper to sync dirty bitmap from slot to ramblock
      KVM: Simplify dirty log sync in kvm_set_phys_mem
      KVM: Cache kvm slot dirty bitmap size
      KVM: Add dirty-ring-size property
      KVM: Disable manual dirty log when dirty ring enabled
      KVM: Dirty ring support

Philippe Mathieu-Daudé (3):
      configure: Only clone softfloat-3 repositories if TCG is enabled
      hw/mem/nvdimm: Use Kconfig 'imply' instead of 'depends on'
      tests/qtest/fuzz: Fix build failure

Yang Zhong (1):
      i386/cpu: Expose AVX_VNNI instruction to guest

 Makefile                                      |   2 +
 accel/kvm/kvm-all.c                           | 615 ++++++++++++++++++++++----
 accel/kvm/trace-events                        |   7 +
 block/file-posix.c                            |   3 +-
 block/replication.c                           |   2 +-
 configure                                     |  67 ++-
 default-configs/devices/ppc64-softmmu.mak     |   1 -
 hmp-commands.hx                               |   2 +
 hw/arm/Kconfig                                |   1 +
 hw/i386/Kconfig                               |   1 +
 hw/mem/Kconfig                                |   2 -
 hw/ppc/Kconfig                                |   1 +
 hw/scsi/scsi-generic.c                        |   6 +-
 replication.h => include/block/replication.h  |   4 +-
 include/exec/memory.h                         |  12 +
 include/hw/core/cpu.h                         |   7 +
 include/qemu/config-file.h                    |   2 +-
 qemu-options.h => include/qemu/qemu-options.h |   9 +-
 include/sysemu/kvm_int.h                      |   7 +-
 meson                                         |   2 +-
 meson.build                                   |   1 +
 migration/colo.c                              |   2 +-
 os-posix.c                                    |   2 +-
 os-win32.c                                    |   1 -
 qapi/qom.json                                 |  23 +-
 qemu-options-wrapper.h                        |  40 --
 qemu-options.hx                               |  16 +
 replication.c                                 |   2 +-
 softmmu/memory.c                              |  33 +-
 softmmu/qtest.c                               | 185 +++++++-
 softmmu/vl.c                                  |  52 ++-
 stubs/meson.build                             |   1 +
 stubs/module-opts.c                           |   6 +
 target/i386/cpu.c                             |   4 +-
 target/i386/cpu.h                             |   2 +
 tests/qtest/fuzz/fuzz.c                       |   1 +
 tests/unit/test-replication.c                 |   2 +-
 util/qemu-config.c                            |   1 +
 38 files changed, 908 insertions(+), 219 deletions(-)
 rename replication.h => include/block/replication.h (98%)
 rename qemu-options.h => include/qemu/qemu-options.h (88%)
 delete mode 100644 qemu-options-wrapper.h
 create mode 100644 stubs/module-opts.c

Comments

no-reply@patchew.org May 20, 2021, 8:54 a.m. UTC | #1
Patchew URL: https://patchew.org/QEMU/20210520082257.187061-1-pbonzini@redhat.com/



Hi,

This series seems to have some coding style problems. See output below for
more information:

Type: series
Message-id: 20210520082257.187061-1-pbonzini@redhat.com
Subject: [PULL 00/26] Misc patches for 2021-05-20

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu
 - [tag update]      patchew/20210519163448.2154339-1-philmd@redhat.com -> patchew/20210519163448.2154339-1-philmd@redhat.com
 * [new tag]         patchew/20210520082257.187061-1-pbonzini@redhat.com -> patchew/20210520082257.187061-1-pbonzini@redhat.com
Switched to a new branch 'test'
fd40ea5 scsi-generic: pass max_segments via max_iov field in BlockLimits
37a6979 qemu-config: load modules when instantiating option groups
32b94da vl: allow not specifying size in -m when using -M memory-backend
675997a replication: move include out of root directory
2e3973b remove qemu-options* from root directory
5d724e9 meson: Set implicit_include_directories to false
ecc742f tests/qtest/fuzz: Fix build failure
3e04412 KVM: Dirty ring support
a954451 KVM: Disable manual dirty log when dirty ring enabled
24a98e2 KVM: Add dirty-ring-size property
589af7b KVM: Cache kvm slot dirty bitmap size
73442ac KVM: Simplify dirty log sync in kvm_set_phys_mem
53278ea KVM: Provide helper to sync dirty bitmap from slot to ramblock
ea3b6f9 KVM: Provide helper to get kvm dirty log
51d5550 KVM: Create the KVMSlot dirty bitmap on flag changes
c834893 KVM: Use a big lock to replace per-kml slots_lock
1f2cf76 memory: Introduce log_sync_global() to memory listener
d92fe4b KVM: do not allow setting properties at runtime
75d0f51 qtest: add a QOM object for qtest
50a63a3 object: add more commands to preconfig mode
5681fd8 meson: bump submodule to 0.57.2
3838574 i386/cpu: Expose AVX_VNNI instruction to guest
a531870 hw/mem/nvdimm: Use Kconfig 'imply' instead of 'depends on'
8431b7a configure: simplify assignment to GIT_SUBMODULES
f35bc5c configure: check for submodules if --with-git-submodules=ignore
dad7063 configure: Only clone softfloat-3 repositories if TCG is enabled

=== OUTPUT BEGIN ===
1/26 Checking commit dad70637e0bc (configure: Only clone softfloat-3 repositories if TCG is enabled)
2/26 Checking commit f35bc5c7fde4 (configure: check for submodules if --with-git-submodules=ignore)
3/26 Checking commit 8431b7a87054 (configure: simplify assignment to GIT_SUBMODULES)
4/26 Checking commit a531870aa3c8 (hw/mem/nvdimm: Use Kconfig 'imply' instead of 'depends on')
5/26 Checking commit 3838574cd38d (i386/cpu: Expose AVX_VNNI instruction to guest)
6/26 Checking commit 5681fd804f8f (meson: bump submodule to 0.57.2)
7/26 Checking commit 50a63a3492af (object: add more commands to preconfig mode)
8/26 Checking commit 75d0f5167f5b (qtest: add a QOM object for qtest)
9/26 Checking commit d92fe4b72972 (KVM: do not allow setting properties at runtime)
ERROR: line over 90 characters
#33: FILE: accel/kvm/kvm-all.c:3115:
+        error_setg(errp, "Cannot set properties after the accelerator has been initialized");

ERROR: line over 90 characters
#45: FILE: accel/kvm/kvm-all.c:3134:
+        error_setg(errp, "Cannot set properties after the accelerator has been initialized");

total: 2 errors, 0 warnings, 37 lines checked

Patch 9/26 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

10/26 Checking commit 1f2cf76b0ea0 (memory: Introduce log_sync_global() to memory listener)
11/26 Checking commit c8348939ca12 (KVM: Use a big lock to replace per-kml slots_lock)
12/26 Checking commit 51d55505d416 (KVM: Create the KVMSlot dirty bitmap on flag changes)
13/26 Checking commit ea3b6f9db553 (KVM: Provide helper to get kvm dirty log)
14/26 Checking commit 53278ea4368f (KVM: Provide helper to sync dirty bitmap from slot to ramblock)
15/26 Checking commit 73442acd2e96 (KVM: Simplify dirty log sync in kvm_set_phys_mem)
16/26 Checking commit 589af7b254d2 (KVM: Cache kvm slot dirty bitmap size)
17/26 Checking commit 24a98e28b284 (KVM: Add dirty-ring-size property)
ERROR: line over 90 characters
#59: FILE: accel/kvm/kvm-all.c:3206:
+        error_setg(errp, "Cannot set properties after the accelerator has been initialized");

total: 1 errors, 0 warnings, 94 lines checked

Patch 17/26 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

18/26 Checking commit a954451955ef (KVM: Disable manual dirty log when dirty ring enabled)
19/26 Checking commit 3e04412f7670 (KVM: Dirty ring support)
WARNING: line over 80 characters
#451: FILE: accel/kvm/kvm-all.c:2466:
+                             "Suggested mininum value is 1024.", strerror(-ret));

total: 0 errors, 1 warnings, 498 lines checked

Patch 19/26 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
20/26 Checking commit ecc742f3f5f3 (tests/qtest/fuzz: Fix build failure)
21/26 Checking commit 5d724e9361f4 (meson: Set implicit_include_directories to false)
22/26 Checking commit 2e3973bf8de9 (remove qemu-options* from root directory)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#26: 
rename from qemu-options.h

ERROR: Macros with complex values should be enclosed in parenthesis
#38: FILE: include/qemu/qemu-options.h:33:
+#define DEF(option, opt_arg, opt_enum, opt_help, arch_mask)     \
+    opt_enum,

total: 1 errors, 1 warnings, 79 lines checked

Patch 22/26 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

23/26 Checking commit 675997a53ff0 (replication: move include out of root directory)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#35: 
rename from replication.h

total: 0 errors, 1 warnings, 48 lines checked

Patch 23/26 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
24/26 Checking commit 32b94da773e7 (vl: allow not specifying size in -m when using -M memory-backend)
25/26 Checking commit 37a69797e76c (qemu-config: load modules when instantiating option groups)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#93: 
new file mode 100644

total: 0 errors, 1 warnings, 61 lines checked

Patch 25/26 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
26/26 Checking commit fd40ea59615b (scsi-generic: pass max_segments via max_iov field in BlockLimits)
WARNING: line over 80 characters
#51: FILE: hw/scsi/scsi-generic.c:186:
+            max_transfer = MIN_NON_ZERO(max_transfer, max_iov * qemu_real_host_page_size)

total: 0 errors, 1 warnings, 23 lines checked

Patch 26/26 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
=== OUTPUT END ===

Test command exited with code: 1


The full log is available at
http://patchew.org/logs/20210520082257.187061-1-pbonzini@redhat.com/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
Peter Maydell May 20, 2021, 1:26 p.m. UTC | #2
On Thu, 20 May 2021 at 09:31, Paolo Bonzini <pbonzini@redhat.com> wrote:
>
> The following changes since commit 15e147b3c778f9f4c08c79b99747b848b6e2117b:
>
>   Merge remote-tracking branch 'remotes/nvme/tags/nvme-next-pull-request' into staging (2021-05-18 12:22:16 +0100)
>
> are available in the Git repository at:
>
>   https://gitlab.com/bonzini/qemu.git tags/tags/for-upstream-v2
>
> for you to fetch changes up to 976551ba504a3284f9ab9862d9434b422c6841a1:
>
>   scsi-generic: pass max_segments via max_iov field in BlockLimits (2021-05-18 17:51:15 +0200)
>
> ----------------------------------------------------------------
> * submodule cleanups (Philippe, myself)
> * tiny step towards a usable preconfig mode (myself)
> * bump Meson submodule (myself)
> * Kconfig and LOCK_GUARD cleanups (philippe)
> * new x86 CPUID feature (Yang Zhong)
> * "-object qtest" support (myself)
> * Dirty ring support for KVM (Peter)
> * Fixes for 6.0 command line parsing breakage (myself)
> * Fix for macOS 11.3 SDK (Katsuhiro)
> * Fix for scsi-generic handling (myself)
>


Fails to build on PPC64:

WARNING: Project specifies a minimum meson_version '>=0.55.0' but uses
features which were added in newer versions:
 * 0.57.0: {'test() timeout <= 0'}


ERROR: Could not detect Ninja v1.8.2 or newer


(This system has ninja-build 1.7.2.)


thanks
-- PMM
Peter Maydell May 20, 2021, 2:20 p.m. UTC | #3
On Thu, 20 May 2021 at 14:26, Peter Maydell <peter.maydell@linaro.org> wrote:
>
> On Thu, 20 May 2021 at 09:31, Paolo Bonzini <pbonzini@redhat.com> wrote:
> >
> > The following changes since commit 15e147b3c778f9f4c08c79b99747b848b6e2117b:
> >
> >   Merge remote-tracking branch 'remotes/nvme/tags/nvme-next-pull-request' into staging (2021-05-18 12:22:16 +0100)
> >
> > are available in the Git repository at:
> >
> >   https://gitlab.com/bonzini/qemu.git tags/tags/for-upstream-v2
> >
> > for you to fetch changes up to 976551ba504a3284f9ab9862d9434b422c6841a1:
> >
> >   scsi-generic: pass max_segments via max_iov field in BlockLimits (2021-05-18 17:51:15 +0200)
> >
> > ----------------------------------------------------------------
> > * submodule cleanups (Philippe, myself)
> > * tiny step towards a usable preconfig mode (myself)
> > * bump Meson submodule (myself)
> > * Kconfig and LOCK_GUARD cleanups (philippe)
> > * new x86 CPUID feature (Yang Zhong)
> > * "-object qtest" support (myself)
> > * Dirty ring support for KVM (Peter)
> > * Fixes for 6.0 command line parsing breakage (myself)
> > * Fix for macOS 11.3 SDK (Katsuhiro)
> > * Fix for scsi-generic handling (myself)
> >
>
>
> Fails to build on PPC64:
>
> WARNING: Project specifies a minimum meson_version '>=0.55.0' but uses
> features which were added in newer versions:
>  * 0.57.0: {'test() timeout <= 0'}
>
>
> ERROR: Could not detect Ninja v1.8.2 or newer

Also, my OSX build which I configure with
../..configure --extra-cflags='-fdiagnostics-color=never -Werror
-Wno-error=deprecated-declarations'

now triggers a meson.build warning:

../../meson.build:118: WARNING: Consider using the built-in werror
option instead of using "-Werror".
../../meson.build:120: WARNING: Consider using the built-in werror
option instead of using "-Werror".
../../meson.build:122: WARNING: Consider using the built-in werror
option instead of using "-Werror".

thanks
-- PMM
Peter Maydell May 20, 2021, 2:25 p.m. UTC | #4
On Thu, 20 May 2021 at 15:20, Peter Maydell <peter.maydell@linaro.org> wrote:
>
> On Thu, 20 May 2021 at 14:26, Peter Maydell <peter.maydell@linaro.org> wrote:
> >
> > On Thu, 20 May 2021 at 09:31, Paolo Bonzini <pbonzini@redhat.com> wrote:

> > > ----------------------------------------------------------------
> > > * submodule cleanups (Philippe, myself)
> > > * tiny step towards a usable preconfig mode (myself)
> > > * bump Meson submodule (myself)
> > > * Kconfig and LOCK_GUARD cleanups (philippe)
> > > * new x86 CPUID feature (Yang Zhong)
> > > * "-object qtest" support (myself)
> > > * Dirty ring support for KVM (Peter)
> > > * Fixes for 6.0 command line parsing breakage (myself)
> > > * Fix for macOS 11.3 SDK (Katsuhiro)
> > > * Fix for scsi-generic handling (myself)
> > >
> >
> >
> > Fails to build on PPC64:
> >
> > WARNING: Project specifies a minimum meson_version '>=0.55.0' but uses
> > features which were added in newer versions:
> >  * 0.57.0: {'test() timeout <= 0'}
> >
> >
> > ERROR: Could not detect Ninja v1.8.2 or newer
>
> Also, my OSX build which I configure with
> ../..configure --extra-cflags='-fdiagnostics-color=never -Werror
> -Wno-error=deprecated-declarations'
>
> now triggers a meson.build warning:

Also also, dropping the merge commit reveals that the pullreq somehow
broke the build trees such that trying to do a build on master no
longer works:

make: Entering directory '/home/pm/qemu/build/all'
config-host.mak is out-of-date, running configure
  GIT     ui/keycodemapdb meson tests/fp/berkeley-testfloat-3
tests/fp/berkeley-softfloat-3 dtc capstone slirp
/usr/bin/ninja  build.ninja && touch build.ninja.stamp
[0/1] Regenerating build files.
Traceback (most recent call last):
  File "/home/pm/qemu/meson/mesonbuild/mesonmain.py", line 140, in run
    return options.run_func(options)
  File "/home/pm/qemu/meson/mesonbuild/msetup.py", line 245, in run
    app.generate()
  File "/home/pm/qemu/meson/mesonbuild/msetup.py", line 154, in generate
    env = environment.Environment(self.source_dir, self.build_dir, self.options)
  File "/home/pm/qemu/meson/mesonbuild/environment.py", line 523, in __init__
    self.coredata = coredata.load(self.get_build_dir())
  File "/home/pm/qemu/meson/mesonbuild/coredata.py", line 1016, in load
    obj = pickle.load(f)
ModuleNotFoundError: No module named 'mesonbuild.mesonlib.universal';
'mesonbuild.mesonlib' is not a package
FAILED: build.ninja


This seems like a meson bug -- if the cached data isn't valid for
this version of meson it should just blow it away, not die with
a backtrace.

thanks
-- PMM
Paolo Bonzini May 21, 2021, 1:27 p.m. UTC | #5
On 20/05/21 15:26, Peter Maydell wrote:
> WARNING: Project specifies a minimum meson_version '>=0.55.0' but uses
> features which were added in newer versions:
>   * 0.57.0: {'test() timeout <= 0'}
> 
> ERROR: Could not detect Ninja v1.8.2 or newer
> 
> (This system has ninja-build 1.7.2.)

What OS is it, and is there any chance of updating it?  ninja 1.7 was 
released in 2017, and I checked that all the supported OSes for QEMU 
have 1.8 before sending the pull request.

> Also, my OSX build which I configure with
> ../..configure --extra-cflags='-fdiagnostics-color=never -Werror
> -Wno-error=deprecated-declarations'
> 
> now triggers a meson.build warning:
> 
> ../../meson.build:118: WARNING: Consider using the built-in werror
> option instead of using "-Werror".
> ../../meson.build:120: WARNING: Consider using the built-in werror
> option instead of using "-Werror".
> ../../meson.build:122: WARNING: Consider using the built-in werror
> option instead of using "-Werror".

Which makes sense since we have --enable/--disable-werror.

> Also also, dropping the merge commit reveals that the pullreq somehow
> broke the build trees such that trying to do a build on master no
> longer works:  [...] This seems like a meson bug

It is and has been fixed since in Meson, but the older version obviously 
doesn't have the fix.

Paolo
Peter Maydell May 21, 2021, 1:44 p.m. UTC | #6
On Fri, 21 May 2021 at 14:27, Paolo Bonzini <pbonzini@redhat.com> wrote:
>
> On 20/05/21 15:26, Peter Maydell wrote:
> > WARNING: Project specifies a minimum meson_version '>=0.55.0' but uses
> > features which were added in newer versions:
> >   * 0.57.0: {'test() timeout <= 0'}
> >
> > ERROR: Could not detect Ninja v1.8.2 or newer
> >
> > (This system has ninja-build 1.7.2.)
>
> What OS is it, and is there any chance of updating it?  ninja 1.7 was
> released in 2017, and I checked that all the supported OSes for QEMU
> have 1.8 before sending the pull request.

It's the gcc compile farm's ppc64 box, whose /etc/redhat-release
says it's running "CentOS Linux release 7.9.2009 (AltArch)".
I don't have control over system upgrades for it.

I do notice that there's a newer ppc64 box in the farm running
Debian 11 (bullseye), so maybe we could just switch to using that.

> > Also, my OSX build which I configure with
> > ../..configure --extra-cflags='-fdiagnostics-color=never -Werror
> > -Wno-error=deprecated-declarations'
> >
> > now triggers a meson.build warning:
> >
> > ../../meson.build:118: WARNING: Consider using the built-in werror
> > option instead of using "-Werror".
> > ../../meson.build:120: WARNING: Consider using the built-in werror
> > option instead of using "-Werror".
> > ../../meson.build:122: WARNING: Consider using the built-in werror
> > option instead of using "-Werror".
>
> Which makes sense since we have --enable/--disable-werror.

Mmm. I think I started using cflags for that back when configure
never enabled Werror for anything except Linux. I'll check that
there isn't some subtle issue with -Werror and -Wno-error=foo
needing to be in a particular order in the command line, but otherwise
I'll just update the configure rune for that build box.

thanks
-- PMM
Paolo Bonzini May 24, 2021, 9:42 a.m. UTC | #7
On 21/05/21 15:44, Peter Maydell wrote:
>> What OS is it, and is there any chance of updating it?  ninja 1.7 was
>> released in 2017, and I checked that all the supported OSes for QEMU
>> have 1.8 before sending the pull request.
>
> It's the gcc compile farm's ppc64 box, whose /etc/redhat-release
> says it's running "CentOS Linux release 7.9.2009 (AltArch)".
> I don't have control over system upgrades for it.

I see...  Too bad, there is an updated ninja for that distro.  But 
anyway, CentOS 7 is going to be dropped soon from the list of supported 
distros 
(https://patchew.org/QEMU/20210514120415.1368922-1-berrange@redhat.com/) 
so switching to bullseys is probably a good idea.

Thanks,

Paolo

> I do notice that there's a newer ppc64 box in the farm running
> Debian 11 (bullseye), so maybe we could just switch to using that.