mbox

[PULL,00/17] Python patches

Message ID 20220122000931.536322-1-jsnow@redhat.com (mailing list archive)
State New, archived
Headers show

Pull-request

https://gitlab.com/jsnow/qemu.git tags/python-pull-request

Message

John Snow Jan. 22, 2022, 12:09 a.m. UTC
The following changes since commit 5e9d14f2bea6df89c0675df953f9c839560d2266:

  Merge remote-tracking branch 'remotes/alistair/tags/pull-riscv-to-apply-20220121-1' into staging (2022-01-21 10:31:25 +0000)

are available in the Git repository at:

  https://gitlab.com/jsnow/qemu.git tags/python-pull-request

for you to fetch changes up to 05908602429cf9d6fce9b60704b8395f6d295441:

  scripts/render-block-graph: switch to AQMP (2022-01-21 16:01:31 -0500)

----------------------------------------------------------------
Python patches

A few fixes to the Python CI tests, a few fixes to the (async) QMP
library, and a set of patches that begin to shift us towards using the
new qmp lib.

----------------------------------------------------------------

John Snow (17):
  python: pin setuptools below v60.0.0
  python: use avocado's "new" runner
  python/aqmp: fix docstring typo
  python/aqmp: add __del__ method to legacy interface
  python/aqmp: handle asyncio.TimeoutError on execute()
  python/aqmp: copy type definitions from qmp
  python/aqmp: add SocketAddrT to package root
  python/aqmp: rename AQMPError to QMPError
  python/qemu-ga-client: don't use deprecated CLI syntax in usage
    comment
  python/qmp: switch qemu-ga-client to AQMP
  python/qmp: switch qom tools to AQMP
  python/qmp: switch qmp-shell to AQMP
  python: move qmp utilities to python/qemu/utils
  python: move qmp-shell under the AQMP package
  scripts/cpu-x86-uarch-abi: fix CLI parsing
  scripts/cpu-x86-uarch-abi: switch to AQMP
  scripts/render-block-graph: switch to AQMP

 python/README.rst                            |  2 +-
 python/Makefile                              |  2 +
 python/avocado.cfg                           |  2 +-
 python/qemu/aqmp/__init__.py                 | 16 ++++++--
 python/qemu/aqmp/error.py                    | 12 +++---
 python/qemu/aqmp/events.py                   |  4 +-
 python/qemu/aqmp/legacy.py                   | 41 +++++++++++++++++++-
 python/qemu/aqmp/protocol.py                 | 24 +++++++-----
 python/qemu/aqmp/qmp_client.py               | 16 +++++---
 python/qemu/{qmp => aqmp}/qmp_shell.py       | 31 ++++++++-------
 python/qemu/{qmp => utils}/qemu_ga_client.py | 24 ++++++------
 python/qemu/{qmp => utils}/qom.py            |  5 ++-
 python/qemu/{qmp => utils}/qom_common.py     |  3 +-
 python/qemu/{qmp => utils}/qom_fuse.py       | 11 +++---
 python/setup.cfg                             | 19 ++++-----
 scripts/cpu-x86-uarch-abi.py                 |  7 ++--
 scripts/qmp/qemu-ga-client                   |  2 +-
 scripts/qmp/qmp-shell                        |  2 +-
 scripts/qmp/qom-fuse                         |  2 +-
 scripts/qmp/qom-get                          |  2 +-
 scripts/qmp/qom-list                         |  2 +-
 scripts/qmp/qom-set                          |  2 +-
 scripts/qmp/qom-tree                         |  2 +-
 scripts/render_block_graph.py                |  8 ++--
 24 files changed, 151 insertions(+), 90 deletions(-)
 rename python/qemu/{qmp => aqmp}/qmp_shell.py (96%)
 rename python/qemu/{qmp => utils}/qemu_ga_client.py (94%)
 rename python/qemu/{qmp => utils}/qom.py (98%)
 rename python/qemu/{qmp => utils}/qom_common.py (98%)
 rename python/qemu/{qmp => utils}/qom_fuse.py (97%)

Comments

Peter Maydell Jan. 22, 2022, 1:57 p.m. UTC | #1
On Sat, 22 Jan 2022 at 00:09, John Snow <jsnow@redhat.com> wrote:
>
> The following changes since commit 5e9d14f2bea6df89c0675df953f9c839560d2266:
>
>   Merge remote-tracking branch 'remotes/alistair/tags/pull-riscv-to-apply-20220121-1' into staging (2022-01-21 10:31:25 +0000)
>
> are available in the Git repository at:
>
>   https://gitlab.com/jsnow/qemu.git tags/python-pull-request
>
> for you to fetch changes up to 05908602429cf9d6fce9b60704b8395f6d295441:
>
>   scripts/render-block-graph: switch to AQMP (2022-01-21 16:01:31 -0500)
>
> ----------------------------------------------------------------
> Python patches
>
> A few fixes to the Python CI tests, a few fixes to the (async) QMP
> library, and a set of patches that begin to shift us towards using the
> new qmp lib.
>
> ----------------------------------------------------------------

Was this set of patches supposed to fix the NetBSD VM intermittents?

I still see that failure, though it looks a little different now:

  TEST   iotest-qcow2: 040 [fail]
QEMU          --
"/home/qemu/qemu-test.ArLruP/build/tests/qemu-iotests/../../qemu-system-aarch64"
-nodefaults -display none -accel qtest -machine virt
QEMU_IMG      --
"/home/qemu/qemu-test.ArLruP/build/tests/qemu-iotests/../../qemu-img"
QEMU_IO       --
"/home/qemu/qemu-test.ArLruP/build/tests/qemu-iotests/../../qemu-io"
--cache writeback --aio threads -f qcow2
QEMU_NBD      --
"/home/qemu/qemu-test.ArLruP/build/tests/qemu-iotests/../../qemu-nbd"
IMGFMT        -- qcow2
IMGPROTO      -- file
PLATFORM      -- NetBSD/amd64 localhost 9.2
TEST_DIR      -- /home/qemu/qemu-test.ArLruP/build/tests/qemu-iotests/scratch
SOCK_DIR      -- /tmp/tmpb3k9h89o
GDB_OPTIONS   --
VALGRIND_QEMU --
PRINT_QEMU_OUTPUT --

--- /home/qemu/qemu-test.ArLruP/src/tests/qemu-iotests/040.out
+++ 040.out.bad
@@ -1,5 +1,30 @@
-.................................................................
+............ERROR:qemu.aqmp.qmp_client.qemu-11749:Failed to establish
connection: concurrent.futures._base.CancelledError
+E....................................................
+======================================================================
+ERROR: test_top_invalid (__main__.TestActiveZeroLengthImage)
+----------------------------------------------------------------------
+Traceback (most recent call last):
+  File "/home/qemu/qemu-test.ArLruP/src/tests/qemu-iotests/040", line
94, in setUp
+    self.vm.launch()
+  File "/home/qemu/qemu-test.ArLruP/src/python/qemu/machine/machine.py",
line 399, in launch
+    self._launch()
+  File "/home/qemu/qemu-test.ArLruP/src/python/qemu/machine/machine.py",
line 434, in _launch
+    self._post_launch()
+  File "/home/qemu/qemu-test.ArLruP/src/python/qemu/machine/qtest.py",
line 147, in _post_launch
+    super()._post_launch()
+  File "/home/qemu/qemu-test.ArLruP/src/python/qemu/machine/machine.py",
line 340, in _post_launch
+    self._qmp.accept(self._qmp_timer)
+  File "/home/qemu/qemu-test.ArLruP/src/python/qemu/aqmp/legacy.py",
line 92, in accept
+    timeout
+  File "/home/qemu/qemu-test.ArLruP/src/python/qemu/aqmp/legacy.py",
line 65, in _sync
+    asyncio.wait_for(future, timeout=timeout)
+  File "/usr/pkg/lib/python3.7/asyncio/base_events.py", line 587, in
run_until_complete
+    return future.result()
+  File "/usr/pkg/lib/python3.7/asyncio/tasks.py", line 449, in wait_for
+    raise futures.TimeoutError()
+concurrent.futures._base.TimeoutError
+
 ----------------------------------------------------------------------
 Ran 65 tests

-OK
+FAILED (errors=1)

thanks
-- PMM
John Snow Jan. 22, 2022, 5:06 p.m. UTC | #2
On Sat, Jan 22, 2022, 8:58 AM Peter Maydell <peter.maydell@linaro.org>
wrote:

> On Sat, 22 Jan 2022 at 00:09, John Snow <jsnow@redhat.com> wrote:
> >
> > The following changes since commit
> 5e9d14f2bea6df89c0675df953f9c839560d2266:
> >
> >   Merge remote-tracking branch
> 'remotes/alistair/tags/pull-riscv-to-apply-20220121-1' into staging
> (2022-01-21 10:31:25 +0000)
> >
> > are available in the Git repository at:
> >
> >   https://gitlab.com/jsnow/qemu.git tags/python-pull-request
> >
> > for you to fetch changes up to 05908602429cf9d6fce9b60704b8395f6d295441:
> >
> >   scripts/render-block-graph: switch to AQMP (2022-01-21 16:01:31 -0500)
> >
> > ----------------------------------------------------------------
> > Python patches
> >
> > A few fixes to the Python CI tests, a few fixes to the (async) QMP
> > library, and a set of patches that begin to shift us towards using the
> > new qmp lib.
> >
> > ----------------------------------------------------------------
>
> Was this set of patches supposed to fix the NetBSD VM intermittents?
>

No, nobody reviewed or tested that series yet, so I didn't pull it.

(Is that too conservative ...?)

If you'd like, I can include them anyway in a v2 pull here and you can test
it as part of your merge.


> I still see that failure, though it looks a little different now:
>
>   TEST   iotest-qcow2: 040 [fail]
> QEMU          --
>
> "/home/qemu/qemu-test.ArLruP/build/tests/qemu-iotests/../../qemu-system-aarch64"
> -nodefaults -display none -accel qtest -machine virt
> QEMU_IMG      --
> "/home/qemu/qemu-test.ArLruP/build/tests/qemu-iotests/../../qemu-img"
> QEMU_IO       --
> "/home/qemu/qemu-test.ArLruP/build/tests/qemu-iotests/../../qemu-io"
> --cache writeback --aio threads -f qcow2
> QEMU_NBD      --
> "/home/qemu/qemu-test.ArLruP/build/tests/qemu-iotests/../../qemu-nbd"
> IMGFMT        -- qcow2
> IMGPROTO      -- file
> PLATFORM      -- NetBSD/amd64 localhost 9.2
> TEST_DIR      --
> /home/qemu/qemu-test.ArLruP/build/tests/qemu-iotests/scratch
> SOCK_DIR      -- /tmp/tmpb3k9h89o
> GDB_OPTIONS   --
> VALGRIND_QEMU --
> PRINT_QEMU_OUTPUT --
>
> --- /home/qemu/qemu-test.ArLruP/src/tests/qemu-iotests/040.out
> +++ 040.out.bad
> @@ -1,5 +1,30 @@
> -.................................................................
> +............ERROR:qemu.aqmp.qmp_client.qemu-11749:Failed to establish
> connection: concurrent.futures._base.CancelledError
> +E....................................................
> +======================================================================
> +ERROR: test_top_invalid (__main__.TestActiveZeroLengthImage)
> +----------------------------------------------------------------------
> +Traceback (most recent call last):
> +  File "/home/qemu/qemu-test.ArLruP/src/tests/qemu-iotests/040", line
> 94, in setUp
> +    self.vm.launch()
> +  File "/home/qemu/qemu-test.ArLruP/src/python/qemu/machine/machine.py",
> line 399, in launch
> +    self._launch()
> +  File "/home/qemu/qemu-test.ArLruP/src/python/qemu/machine/machine.py",
> line 434, in _launch
> +    self._post_launch()
> +  File "/home/qemu/qemu-test.ArLruP/src/python/qemu/machine/qtest.py",
> line 147, in _post_launch
> +    super()._post_launch()
> +  File "/home/qemu/qemu-test.ArLruP/src/python/qemu/machine/machine.py",
> line 340, in _post_launch
> +    self._qmp.accept(self._qmp_timer)
> +  File "/home/qemu/qemu-test.ArLruP/src/python/qemu/aqmp/legacy.py",
> line 92, in accept
> +    timeout
> +  File "/home/qemu/qemu-test.ArLruP/src/python/qemu/aqmp/legacy.py",
> line 65, in _sync
> +    asyncio.wait_for(future, timeout=timeout)
> +  File "/usr/pkg/lib/python3.7/asyncio/base_events.py", line 587, in
> run_until_complete
> +    return future.result()
> +  File "/usr/pkg/lib/python3.7/asyncio/tasks.py", line 449, in wait_for
> +    raise futures.TimeoutError()
> +concurrent.futures._base.TimeoutError
> +
>  ----------------------------------------------------------------------
>  Ran 65 tests
>
> -OK
> +FAILED (errors=1)
>
> thanks
> -- PMM
>
>
Peter Maydell Jan. 22, 2022, 5:56 p.m. UTC | #3
On Sat, 22 Jan 2022 at 17:06, John Snow <jsnow@redhat.com> wrote:
>
>
>
> On Sat, Jan 22, 2022, 8:58 AM Peter Maydell <peter.maydell@linaro.org> wrote:
>>
>> On Sat, 22 Jan 2022 at 00:09, John Snow <jsnow@redhat.com> wrote:
>> >
>> > The following changes since commit 5e9d14f2bea6df89c0675df953f9c839560d2266:
>> >
>> >   Merge remote-tracking branch 'remotes/alistair/tags/pull-riscv-to-apply-20220121-1' into staging (2022-01-21 10:31:25 +0000)
>> >
>> > are available in the Git repository at:
>> >
>> >   https://gitlab.com/jsnow/qemu.git tags/python-pull-request
>> >
>> > for you to fetch changes up to 05908602429cf9d6fce9b60704b8395f6d295441:
>> >
>> >   scripts/render-block-graph: switch to AQMP (2022-01-21 16:01:31 -0500)
>> >
>> > ----------------------------------------------------------------
>> > Python patches
>> >
>> > A few fixes to the Python CI tests, a few fixes to the (async) QMP
>> > library, and a set of patches that begin to shift us towards using the
>> > new qmp lib.
>> >
>> > ----------------------------------------------------------------
>>
>> Was this set of patches supposed to fix the NetBSD VM intermittents?
>
>
> No, nobody reviewed or tested that series yet, so I didn't pull it.
>
> (Is that too conservative ...?)
>
> If you'd like, I can include them anyway in a v2 pull here and you can test it as part of your merge.

No, that's fine -- I just hadn't been keeping track of which patches
were intended as the fix.


Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/7.0
for any user-visible changes.

-- PMM