mbox

[PULL,00/52] testing, gdbstub and cputlb fixes

Message ID 20190607090552.12434-1-alex.bennee@linaro.org (mailing list archive)
State New, archived
Headers show

Pull-request

https://github.com/stsquad/qemu.git tags/pull-testing-gdbstub-cputlb-070619-1

Message

Alex Bennée June 7, 2019, 9:04 a.m. UTC
The following changes since commit 0d74f3b4277a7ecb0ccb80c865797d11f8e466f5:

  Merge remote-tracking branch 'remotes/vivier2/tags/trivial-branch-pull-request' into staging (2019-06-06 14:09:14 +0100)

are available in the Git repository at:

  https://github.com/stsquad/qemu.git tags/pull-testing-gdbstub-cputlb-070619-1

for you to fetch changes up to 611a3db380ef9f670c5c82837c4889f7e19ded10:

  gdbstub: Implement qemu physical memory mode (2019-06-07 08:39:03 +0100)

----------------------------------------------------------------
Various fixes and updates:

  - editor config tweak for shell scripts
  - iotest updates (still not default for make check)
  - various docker updates
  - gcc/ubsan updates for travis
  - clean-ups for tests/vm including autoinstall
  - semihosting fix for Coverity
  - fixes for cputlb in 64-on-32 cases
  - gdbstub re-factor + maintainership update

----------------------------------------------------------------
Alex Bennée (13):
      editorconfig: add setting for shell scripts
      qemu-io-cmds: use clock_gettime for benchmarking
      tests/docker: Update the Fedora cross compile images to 30
      tests/docker: Update the Ubuntu image to 19.04
      .travis.yml: bump gcc sanitiser job to gcc-9
      .travis.yml: add clang ubsan job
      semihosting: split console_out into string and char versions
      cputlb: use uint64_t for interim values for unaligned load
      tests/tcg: better detect truncated reads
      tests/tcg: clean-up VPATH/TESTS for i386
      tests/tcg/x86_64: add a PVH crt.o for x86_64 system tests
      MAINTAINERS: put myself forward for gdbstub
      cputlb: cast size_t to target_ulong before using for address masks

Gerd Hoffmann (14):
      scripts: use git archive in archive-source
      tests/vm: python3 fixes
      tests/vm: send proxy environment variables over ssh
      tests/vm: use ssh with pty unconditionally
      tests/vm: run test builds on snapshot
      tests/vm: proper guest shutdown
      tests/vm: add vm-boot-{ssh,serial}-<guest> targets
      tests/vm: add DEBUG=1 to help text
      tests/vm: serial console support helpers
      tests/vm: openbsd autoinstall, using serial console
      tests/vm: freebsd autoinstall, using serial console
      tests/vm: netbsd autoinstall, using serial console
      tests/vm: fedora autoinstall, using serial console
      tests/vm: ubuntu.i386: apt proxy setup

Jon Doron (20):
      gdbstub: Add infrastructure to parse cmd packets
      gdbstub: Implement deatch (D pkt) with new infra
      gdbstub: Implement thread_alive (T pkt) with new infra
      gdbstub: Implement continue (c pkt) with new infra
      gdbstub: Implement continue with signal (C pkt) with new infra
      gdbstub: Implement set_thread (H pkt) with new infra
      gdbstub: Implement breakpoint commands (Z/z pkt) with new infra
      gdbstub: Implement set register (P pkt) with new infra
      gdbstub: Implement get register (p pkt) with new infra
      gdbstub: Implement write memory (M pkt) with new infra
      gdbstub: Implement read memory (m pkt) with new infra
      gdbstub: Implement write all registers (G pkt) with new infra
      gdbstub: Implement read all registers (g pkt) with new infra
      gdbstub: Implement file io (F pkt) with new infra
      gdbstub: Implement step (s pkt) with new infra
      gdbstub: Implement v commands with new infra
      gdbstub: Implement generic set/query (Q/q pkt) with new infra
      gdbstub: Implement target halted (? pkt) with new infra
      gdbstub: Clear unused variables in gdb_handle_packet
      gdbstub: Implement qemu physical memory mode

Philippe Mathieu-Daudé (1):
      tests/docker: Update the Fedora image to Fedora 30

Wainer dos Santos Moschetta (4):
      tests/vm: Use python configured on build
      tests/vm: Port basevm to Python 3
      tests/vm: Fix build-centos docker-based tests run
      tests/vm: Add missing variables on help

 .editorconfig                                     |    4 +
 .travis.yml                                       |   17 +-
 MAINTAINERS                                       |    4 +-
 accel/tcg/cputlb.c                                |    4 +-
 gdbstub.c                                         | 1761 +++++++++++++++------
 hw/semihosting/console.c                          |   34 +-
 include/hw/semihosting/console.h                  |   25 +-
 linux-user/arm/semihost.c                         |   31 +-
 qemu-io-cmds.c                                    |   77 +-
 scripts/archive-source.sh                         |   72 +-
 target/arm/arm-semi.c                             |    4 +-
 tests/docker/dockerfiles/fedora-cris-cross.docker |    2 +-
 tests/docker/dockerfiles/fedora-i386-cross.docker |    2 +-
 tests/docker/dockerfiles/fedora.docker            |    2 +-
 tests/docker/dockerfiles/ubuntu.docker            |   19 +-
 tests/tcg/i386/Makefile.softmmu-target            |   10 +-
 tests/tcg/multiarch/system/memory.c               |   36 +-
 tests/tcg/x86_64/system/boot.S                    |  277 ++++
 tests/tcg/x86_64/system/kernel.ld                 |   33 +
 tests/vm/Makefile.include                         |   37 +-
 tests/vm/basevm.py                                |  147 +-
 tests/vm/centos                                   |    6 +-
 tests/vm/fedora                                   |  187 +++
 tests/vm/freebsd                                  |  179 ++-
 tests/vm/netbsd                                   |  187 ++-
 tests/vm/openbsd                                  |  158 +-
 tests/vm/ubuntu.i386                              |    4 +
 27 files changed, 2635 insertions(+), 684 deletions(-)
 create mode 100644 tests/tcg/x86_64/system/boot.S
 create mode 100644 tests/tcg/x86_64/system/kernel.ld
 create mode 100755 tests/vm/fedora

Comments

Peter Maydell June 7, 2019, 9:50 a.m. UTC | #1
On Fri, 7 Jun 2019 at 10:05, Alex Bennée <alex.bennee@linaro.org> wrote:
>
> The following changes since commit 0d74f3b4277a7ecb0ccb80c865797d11f8e466f5:
>
>   Merge remote-tracking branch 'remotes/vivier2/tags/trivial-branch-pull-request' into staging (2019-06-06 14:09:14 +0100)
>
> are available in the Git repository at:
>
>   https://github.com/stsquad/qemu.git tags/pull-testing-gdbstub-cputlb-070619-1
>
> for you to fetch changes up to 611a3db380ef9f670c5c82837c4889f7e19ded10:
>
>   gdbstub: Implement qemu physical memory mode (2019-06-07 08:39:03 +0100)
>
> ----------------------------------------------------------------
> Various fixes and updates:
>
>   - editor config tweak for shell scripts
>   - iotest updates (still not default for make check)
>   - various docker updates
>   - gcc/ubsan updates for travis
>   - clean-ups for tests/vm including autoinstall
>   - semihosting fix for Coverity
>   - fixes for cputlb in 64-on-32 cases
>   - gdbstub re-factor + maintainership update
>

This seems to break the tests/vm testing I do. Eg openbsd:

hackbox mergebuild: seq openbsd
make: Entering directory '/home/peter.maydell/qemu-openbsd/build'
python3 -B /home/peter.maydell/qemu-openbsd/tests/vm/openbsd  --debug
--image "/home/peter.maydell/.cache/qemu-vm/images/openbsd.img"
--force --build-image
/home/peter.maydell/.cache/qemu-vm/images/openbsd.img
### Downloading install iso ...
### Preparing iso and disk image ...
Formatting '/home/peter.maydell/.cache/qemu-vm/images/openbsd.img.tmp',
fmt=qcow2 size=21474836480 cluster_size=65536 lazy_refcounts=off
refcount_bits=16
### Booting installer ...
DEBUG:root:QEMU args: -nodefaults -m 4G -cpu max -netdev
user,id=vnet,hostfwd=:127.0.0.1:0-:22 -device
virtio-net-pci,netdev=vnet -vnc 127.0.0.1:0,to=20 -smp 18 -enable-kvm
-device VGA -drive
file=/home/peter.maydell/.cache/qemu-vm/images/openbsd.img.tmp,if=none,id=drive0,cache=writeback
-device virtio-blk,drive=drive0,bootindex=0 -machine graphics=off
-cdrom /home/peter.maydell/.cache/qemu-vm/images/openbsd.img.install.iso
DEBUG:qemu:VM launch command: 'qemu-system-x86_64 -chardev
socket,id=mon,path=/var/tmp/tmpcex0zicz/qemu-6023-monitor.sock -mon
chardev=mon,mode=control -display none -vga none -machine pc -chardev
socket,id=console,path=/var/tmp/tmpcex0zicz/qemu-6023-console.sock,server,nowait
-serial chardev:console -nodefaults -m 4G -cpu max -netdev
user,id=vnet,hostfwd=:127.0.0.1:0-:22 -device
virtio-net-pci,netdev=vnet -vnc 127.0.0.1:0,to=20 -smp 18 -enable-kvm
-device VGA -drive
file=/home/peter.maydell/.cache/qemu-vm/images/openbsd.img.tmp,if=none,id=drive0,cache=writeback
-device virtio-blk,drive=drive0,bootindex=0 -machine graphics=off
-cdrom /home/peter.maydell/.cache/qemu-vm/images/openbsd.img.install.iso'
DEBUG:QMP:>>> {'execute': 'qmp_capabilities'}
DEBUG:QMP:<<< {'return': {}}
DEBUG:QMP:>>> {'execute': 'human-monitor-command', 'arguments':
{'command-line': 'info usernet'}}
DEBUG:QMP:<<< {'return': 'VLAN -1 (vnet):\r\n  Protocol[State]    FD
Source Address  Port   Dest. Address  Port RecvQ SendQ\r\n
TCP[HOST_FORWARD]  13       127.0.0.1 37043       10.0.2.15    22
0     0\r\n'}
console: *** read timeout ***
console: waiting for: 'boot>'
console: line buffer:


Failed to prepare guest environment
Traceback (most recent call last):
  File "/home/peter.maydell/qemu-openbsd/tests/vm/basevm.py", line 351, in main
    return vm.build_image(args.image)
  File "/home/peter.maydell/qemu-openbsd/tests/vm/openbsd", line 85,
in build_image
    self.console_wait_send("boot>", "set tty com0\n")
  File "/home/peter.maydell/qemu-openbsd/tests/vm/basevm.py", line
251, in console_wait_send
    self.console_wait(wait)
  File "/home/peter.maydell/qemu-openbsd/tests/vm/basevm.py", line
213, in console_wait
    chars = vm.console_socket.recv(1024)
socket.timeout: timed out
DEBUG:QMP:>>> {'execute': 'quit'}
DEBUG:QMP:<<< {'return': {}}
/home/peter.maydell/qemu-openbsd/tests/vm/Makefile.include:47: recipe
for target '/home/peter.maydell/.cache/qemu-vm/images/openbsd.img'
failed
make: *** [/home/peter.maydell/.cache/qemu-vm/images/openbsd.img] Error 2
make: Leaving directory '/home/peter.maydell/qemu-openbsd/build'

---
NetBSD:

peter.maydell@hackbox2.linaro.org:~$ mergebuild netbsd
hackbox mergebuild: seq netbsd
make: Entering directory '/home/peter.maydell/qemu-netbsd/build'
python3 -B /home/peter.maydell/qemu-netbsd/tests/vm/netbsd  --debug
--image "/home/peter.maydell/.cache/qemu-vm/images/netbsd.img" --force
--build-image /home/peter.maydell/.cache/qemu-vm/images/netbsd.img
### Preparing iso and disk image ...
Formatting '/home/peter.maydell/.cache/qemu-vm/images/netbsd.img.tmp',
fmt=qcow2 size=21474836480 cluster_size=65536 lazy_refcounts=off
refcount_bits=16
### Booting installer ...
DEBUG:root:QEMU args: -nodefaults -m 4G -cpu max -netdev
user,id=vnet,hostfwd=:127.0.0.1:0-:22 -device
virtio-net-pci,netdev=vnet -vnc 127.0.0.1:0,to=20 -smp 18 -enable-kvm
-device VGA -drive
file=/home/peter.maydell/.cache/qemu-vm/images/netbsd.img.tmp,if=none,id=drive0,cache=writeback
-device virtio-blk,drive=drive0,bootindex=0 -device VGA -machine
graphics=off -cdrom
/home/peter.maydell/.cache/qemu-vm/images/netbsd.img.install.iso
DEBUG:qemu:VM launch command: 'qemu-system-x86_64 -chardev
socket,id=mon,path=/var/tmp/tmpp_t16skr/qemu-13312-monitor.sock -mon
chardev=mon,mode=control -display none -vga none -machine pc -chardev
socket,id=console,path=/var/tmp/tmpp_t16skr/qemu-13312-console.sock,server,nowait
-serial chardev:console -nodefaults -m 4G -cpu max -netdev
user,id=vnet,hostfwd=:127.0.0.1:0-:22 -device
virtio-net-pci,netdev=vnet -vnc 127.0.0.1:0,to=20 -smp 18 -enable-kvm
-device VGA -drive
file=/home/peter.maydell/.cache/qemu-vm/images/netbsd.img.tmp,if=none,id=drive0,cache=writeback
-device virtio-blk,drive=drive0,bootindex=0 -device VGA -machine
graphics=off -cdrom
/home/peter.maydell/.cache/qemu-vm/images/netbsd.img.install.iso'
DEBUG:QMP:>>> {'execute': 'qmp_capabilities'}
DEBUG:QMP:>>> {'execute': 'quit'}
WARNING:qemu:qemu received signal 6: qemu-system-x86_64 -chardev
socket,id=mon,path=/var/tmp/tmpp_t16skr/qemu-13312-monitor.sock -mon
chardev=mon,mode=control -display none -vga none -machine pc -chardev
socket,id=console,path=/var/tmp/tmpp_t16skr/qemu-13312-console.sock,server,nowait
-serial chardev:console -nodefaults -m 4G -cpu max -netdev
user,id=vnet,hostfwd=:127.0.0.1:0-:22 -device
virtio-net-pci,netdev=vnet -vnc 127.0.0.1:0,to=20 -smp 18 -enable-kvm
-device VGA -drive
file=/home/peter.maydell/.cache/qemu-vm/images/netbsd.img.tmp,if=none,id=drive0,cache=writeback
-device virtio-blk,drive=drive0,bootindex=0 -device VGA -machine
graphics=off -cdrom
/home/peter.maydell/.cache/qemu-vm/images/netbsd.img.install.iso
DEBUG:qemu:Error launching VM
DEBUG:qemu:Command: 'qemu-system-x86_64 -chardev
socket,id=mon,path=/var/tmp/tmpp_t16skr/qemu-13312-monitor.sock -mon
chardev=mon,mode=control -display none -vga none -machine pc -chardev
socket,id=console,path=/var/tmp/tmpp_t16skr/qemu-13312-console.sock,server,nowait
-serial chardev:console -nodefaults -m 4G -cpu max -netdev
user,id=vnet,hostfwd=:127.0.0.1:0-:22 -device
virtio-net-pci,netdev=vnet -vnc 127.0.0.1:0,to=20 -smp 18 -enable-kvm
-device VGA -drive
file=/home/peter.maydell/.cache/qemu-vm/images/netbsd.img.tmp,if=none,id=drive0,cache=writeback
-device virtio-blk,drive=drive0,bootindex=0 -device VGA -machine
graphics=off -cdrom
/home/peter.maydell/.cache/qemu-vm/images/netbsd.img.install.iso'
DEBUG:qemu:Output: 'RAMBlock "vga.vram" already registered, abort!\n'
ERROR:root:Failed to launch QEMU, command line:
ERROR:root:qemu-system-x86_64 -nodefaults -m 4G -cpu max -netdev
user,id=vnet,hostfwd=:127.0.0.1:0-:22 -device
virtio-net-pci,netdev=vnet -vnc 127.0.0.1:0,to=20 -smp 18 -enable-kvm
-device VGA -drive
file=/home/peter.maydell/.cache/qemu-vm/images/netbsd.img.tmp,if=none,id=drive0,cache=writeback
-device virtio-blk,drive=drive0,bootindex=0 -device VGA -machine
graphics=off -cdrom
/home/peter.maydell/.cache/qemu-vm/images/netbsd.img.install.iso
ERROR:root:Log:
ERROR:root:RAMBlock "vga.vram" already registered, abort!

ERROR:root:QEMU version >= 2.10 is required
Failed to prepare guest environment
Traceback (most recent call last):
  File "/home/peter.maydell/qemu-netbsd/tests/vm/basevm.py", line 351, in main
    return vm.build_image(args.image)
  File "/home/peter.maydell/qemu-netbsd/tests/vm/netbsd", line 79, in
build_image
    "-cdrom", iso
  File "/home/peter.maydell/qemu-netbsd/tests/vm/basevm.py", line 170, in boot
    guest.launch()
  File "/home/peter.maydell/qemu-netbsd/tests/vm/../../python/qemu/__init__.py",
line 297, in launch
    self._launch()
  File "/home/peter.maydell/qemu-netbsd/tests/vm/../../python/qemu/__init__.py",
line 324, in _launch
    self._post_launch()
  File "/home/peter.maydell/qemu-netbsd/tests/vm/../../python/qemu/__init__.py",
line 268, in _post_launch
    self._qmp.accept()
  File "/home/peter.maydell/qemu-netbsd/tests/vm/../../python/qemu/qmp.py",
line 157, in accept
    return self.__negotiate_capabilities()
  File "/home/peter.maydell/qemu-netbsd/tests/vm/../../python/qemu/qmp.py",
line 75, in __negotiate_capabilities
    resp = self.cmd('qmp_capabilities')
  File "/home/peter.maydell/qemu-netbsd/tests/vm/../../python/qemu/qmp.py",
line 191, in cmd
    return self.cmd_obj(qmp_cmd)
  File "/home/peter.maydell/qemu-netbsd/tests/vm/../../python/qemu/qmp.py",
line 174, in cmd_obj
    resp = self.__json_read()
  File "/home/peter.maydell/qemu-netbsd/tests/vm/../../python/qemu/qmp.py",
line 82, in __json_read
    data = self.__sockfile.readline()
  File "/usr/lib/python3.6/socket.py", line 586, in readinto
    return self._sock.recv_into(b)
ConnectionResetError: [Errno 104] Connection reset by peer
/home/peter.maydell/qemu-netbsd/tests/vm/Makefile.include:47: recipe
for target '/home/peter.maydell/.cache/qemu-vm/images/netbsd.img'
failed
make: *** [/home/peter.maydell/.cache/qemu-vm/images/netbsd.img] Error 2
make: Leaving directory '/home/peter.maydell/qemu-netbsd/build'


FreeBSD:

peter.maydell@hackbox2.linaro.org:~$ mergebuild freebsd
hackbox mergebuild: seq freebsd
make: Entering directory '/home/peter.maydell/qemu-freebsd/build'
python3 -B /home/peter.maydell/qemu-freebsd/tests/vm/freebsd  --debug
--image "/home/peter.maydell/.cache/qemu-vm/images/freebsd.img"
--force --build-image
/home/peter.maydell/.cache/qemu-vm/images/freebsd.img
### Downloading install iso ...
### Preparing iso and disk image ...
/home/peter.maydell/.cache/qemu-vm/images/freebsd.img.install.iso.xz (1/1)
  100 %       595.0 MiB / 851.1 MiB = 0.699   117 MiB/s       0:07
Formatting '/home/peter.maydell/.cache/qemu-vm/images/freebsd.img.tmp',
fmt=qcow2 size=21474836480 cluster_size=65536 lazy_refcounts=off
refcount_bits=16
### Booting installer ...
DEBUG:root:QEMU args: -nodefaults -m 4G -cpu max -netdev
user,id=vnet,hostfwd=:127.0.0.1:0-:22 -device
virtio-net-pci,netdev=vnet -vnc 127.0.0.1:0,to=20 -smp 18 -enable-kvm
-device VGA -drive
file=/home/peter.maydell/.cache/qemu-vm/images/freebsd.img.tmp,if=none,id=drive0,cache=writeback
-device virtio-blk,drive=drive0,bootindex=0 -machine graphics=off
-cdrom /home/peter.maydell/.cache/qemu-vm/images/freebsd.img.install.iso
DEBUG:qemu:VM launch command: 'qemu-system-x86_64 -chardev
socket,id=mon,path=/var/tmp/tmpc1sqg5ms/qemu-23074-monitor.sock -mon
chardev=mon,mode=control -display none -vga none -machine pc -chardev
socket,id=console,path=/var/tmp/tmpc1sqg5ms/qemu-23074-console.sock,server,nowait
-serial chardev:console -nodefaults -m 4G -cpu max -netdev
user,id=vnet,hostfwd=:127.0.0.1:0-:22 -device
virtio-net-pci,netdev=vnet -vnc 127.0.0.1:0,to=20 -smp 18 -enable-kvm
-device VGA -drive
file=/home/peter.maydell/.cache/qemu-vm/images/freebsd.img.tmp,if=none,id=drive0,cache=writeback
-device virtio-blk,drive=drive0,bootindex=0 -machine graphics=off
-cdrom /home/peter.maydell/.cache/qemu-vm/images/freebsd.img.install.iso'
DEBUG:QMP:>>> {'execute': 'qmp_capabilities'}
DEBUG:QMP:<<< {'return': {}}
DEBUG:QMP:>>> {'execute': 'human-monitor-command', 'arguments':
{'command-line': 'info usernet'}}
DEBUG:QMP:<<< {'return': 'VLAN -1 (vnet):\r\n  Protocol[State]    FD
Source Address  Port   Dest. Address  Port RecvQ SendQ\r\n
TCP[HOST_FORWARD]  13       127.0.0.1 37719       10.0.2.15    22
0     0\r\n'}
console: *** read timeout ***
console: waiting for: 'Autoboot'
console: line buffer:


Failed to prepare guest environment
Traceback (most recent call last):
  File "/home/peter.maydell/qemu-freebsd/tests/vm/basevm.py", line 351, in main
    return vm.build_image(args.image)
  File "/home/peter.maydell/qemu-freebsd/tests/vm/freebsd", line 94,
in build_image
    self.console_boot_serial()
  File "/home/peter.maydell/qemu-freebsd/tests/vm/freebsd", line 71,
in console_boot_serial
    self.console_wait_send("Autoboot", "3")
  File "/home/peter.maydell/qemu-freebsd/tests/vm/basevm.py", line
251, in console_wait_send
    self.console_wait(wait)
  File "/home/peter.maydell/qemu-freebsd/tests/vm/basevm.py", line
213, in console_wait
    chars = vm.console_socket.recv(1024)
socket.timeout: timed out
DEBUG:QMP:>>> {'execute': 'quit'}
DEBUG:QMP:<<< {'timestamp': {'seconds': 1559900897, 'microseconds':
206270}, 'event': 'NIC_RX_FILTER_CHANGED', 'data': {'path':
'/machine/peripheral-anon/device[0]/virtio-backend'}}
DEBUG:QMP:<<< {'return': {}}
/home/peter.maydell/qemu-freebsd/tests/vm/Makefile.include:47: recipe
for target '/home/peter.maydell/.cache/qemu-vm/images/freebsd.img'
failed
make: *** [/home/peter.maydell/.cache/qemu-vm/images/freebsd.img] Error 2
make: Leaving directory '/home/peter.maydell/qemu-freebsd/build'



thanks
-- PMM
Alex Bennée June 7, 2019, 1:25 p.m. UTC | #2
Peter Maydell <peter.maydell@linaro.org> writes:

> On Fri, 7 Jun 2019 at 10:05, Alex Bennée <alex.bennee@linaro.org> wrote:
>>
>> The following changes since commit 0d74f3b4277a7ecb0ccb80c865797d11f8e466f5:
>>
>>   Merge remote-tracking branch 'remotes/vivier2/tags/trivial-branch-pull-request' into staging (2019-06-06 14:09:14 +0100)
>>
>> are available in the Git repository at:
>>
>>   https://github.com/stsquad/qemu.git tags/pull-testing-gdbstub-cputlb-070619-1
>>
>> for you to fetch changes up to 611a3db380ef9f670c5c82837c4889f7e19ded10:
>>
>>   gdbstub: Implement qemu physical memory mode (2019-06-07 08:39:03 +0100)
>>
>> ----------------------------------------------------------------
>> Various fixes and updates:
>>
>>   - editor config tweak for shell scripts
>>   - iotest updates (still not default for make check)
>>   - various docker updates
>>   - gcc/ubsan updates for travis
>>   - clean-ups for tests/vm including autoinstall
>>   - semihosting fix for Coverity
>>   - fixes for cputlb in 64-on-32 cases
>>   - gdbstub re-factor + maintainership update
>>
>
> This seems to break the tests/vm testing I do. Eg openbsd:
>
> hackbox mergebuild: seq openbsd
> make: Entering directory '/home/peter.maydell/qemu-openbsd/build'
> python3 -B /home/peter.maydell/qemu-openbsd/tests/vm/openbsd  --debug
> --image "/home/peter.maydell/.cache/qemu-vm/images/openbsd.img"
> --force --build-image
> /home/peter.maydell/.cache/qemu-vm/images/openbsd.img
> ### Downloading install iso ...
> ### Preparing iso and disk image ...
> Formatting '/home/peter.maydell/.cache/qemu-vm/images/openbsd.img.tmp',
> fmt=qcow2 size=21474836480 cluster_size=65536 lazy_refcounts=off
> refcount_bits=16
> ### Booting installer ...
> DEBUG:root:QEMU args: -nodefaults -m 4G -cpu max -netdev
> user,id=vnet,hostfwd=:127.0.0.1:0-:22 -device
> virtio-net-pci,netdev=vnet -vnc 127.0.0.1:0,to=20 -smp 18 -enable-kvm
> -device VGA -drive
> file=/home/peter.maydell/.cache/qemu-vm/images/openbsd.img.tmp,if=none,id=drive0,cache=writeback
> -device virtio-blk,drive=drive0,bootindex=0 -machine graphics=off
> -cdrom /home/peter.maydell/.cache/qemu-vm/images/openbsd.img.install.iso
> DEBUG:qemu:VM launch command: 'qemu-system-x86_64 -chardev
> socket,id=mon,path=/var/tmp/tmpcex0zicz/qemu-6023-monitor.sock -mon
> chardev=mon,mode=control -display none -vga none -machine pc -chardev
> socket,id=console,path=/var/tmp/tmpcex0zicz/qemu-6023-console.sock,server,nowait
> -serial chardev:console -nodefaults -m 4G -cpu max -netdev
> user,id=vnet,hostfwd=:127.0.0.1:0-:22 -device
> virtio-net-pci,netdev=vnet -vnc 127.0.0.1:0,to=20 -smp 18 -enable-kvm
> -device VGA -drive
> file=/home/peter.maydell/.cache/qemu-vm/images/openbsd.img.tmp,if=none,id=drive0,cache=writeback
> -device virtio-blk,drive=drive0,bootindex=0 -machine graphics=off
> -cdrom /home/peter.maydell/.cache/qemu-vm/images/openbsd.img.install.iso'
> DEBUG:QMP:>>> {'execute': 'qmp_capabilities'}
> DEBUG:QMP:<<< {'return': {}}
> DEBUG:QMP:>>> {'execute': 'human-monitor-command', 'arguments':
> {'command-line': 'info usernet'}}
> DEBUG:QMP:<<< {'return': 'VLAN -1 (vnet):\r\n  Protocol[State]    FD
> Source Address  Port   Dest. Address  Port RecvQ SendQ\r\n
> TCP[HOST_FORWARD]  13       127.0.0.1 37043       10.0.2.15    22
> 0     0\r\n'}
> console: *** read timeout ***

This seems to be a bug with the QEMU's installed on hackbox. If I do:

  set -x QEMU /home/alex.bennee/lsrc/qemu.git/x86_64-softmmu/qemu-system-x86_64

(in this case current PR)

And run the vm-test targets they work (or at least the ones I've tried
so far, I'm just kicking off a full build).

--
Alex Bennée
Alex Bennée June 11, 2019, 2:18 p.m. UTC | #3
Peter Maydell <peter.maydell@linaro.org> writes:

> On Fri, 7 Jun 2019 at 10:05, Alex Bennée <alex.bennee@linaro.org> wrote:
>>
>> The following changes since commit 0d74f3b4277a7ecb0ccb80c865797d11f8e466f5:
>>
>>   Merge remote-tracking branch 'remotes/vivier2/tags/trivial-branch-pull-request' into staging (2019-06-06 14:09:14 +0100)
>>
>> are available in the Git repository at:
>>
>>   https://github.com/stsquad/qemu.git tags/pull-testing-gdbstub-cputlb-070619-1
>>
>> for you to fetch changes up to 611a3db380ef9f670c5c82837c4889f7e19ded10:
>>
>>   gdbstub: Implement qemu physical memory mode (2019-06-07 08:39:03 +0100)
>>
>> ----------------------------------------------------------------
>> Various fixes and updates:
>>
>>   - editor config tweak for shell scripts
>>   - iotest updates (still not default for make check)
>>   - various docker updates
>>   - gcc/ubsan updates for travis
>>   - clean-ups for tests/vm including autoinstall
>>   - semihosting fix for Coverity
>>   - fixes for cputlb in 64-on-32 cases
>>   - gdbstub re-factor + maintainership update
>>
>
> This seems to break the tests/vm testing I do. Eg openbsd:
>
> hackbox mergebuild: seq openbsd
> make: Entering directory '/home/peter.maydell/qemu-openbsd/build'
> python3 -B /home/peter.maydell/qemu-openbsd/tests/vm/openbsd  --debug
> --image "/home/peter.maydell/.cache/qemu-vm/images/openbsd.img"
> --force --build-image
> /home/peter.maydell/.cache/qemu-vm/images/openbsd.img
> ### Downloading install iso ...
> ### Preparing iso and disk image ...
> Formatting '/home/peter.maydell/.cache/qemu-vm/images/openbsd.img.tmp',
> fmt=qcow2 size=21474836480 cluster_size=65536 lazy_refcounts=off
> refcount_bits=16
> ### Booting installer ...
> DEBUG:root:QEMU args: -nodefaults -m 4G -cpu max -netdev
> user,id=vnet,hostfwd=:127.0.0.1:0-:22 -device
> virtio-net-pci,netdev=vnet -vnc 127.0.0.1:0,to=20 -smp 18 -enable-kvm
> -device VGA -drive
> file=/home/peter.maydell/.cache/qemu-vm/images/openbsd.img.tmp,if=none,id=drive0,cache=writeback
> -device virtio-blk,drive=drive0,bootindex=0 -machine graphics=off
> -cdrom /home/peter.maydell/.cache/qemu-vm/images/openbsd.img.install.iso
> DEBUG:qemu:VM launch command: 'qemu-system-x86_64 -chardev
> socket,id=mon,path=/var/tmp/tmpcex0zicz/qemu-6023-monitor.sock -mon
> chardev=mon,mode=control -display none -vga none -machine pc -chardev
> socket,id=console,path=/var/tmp/tmpcex0zicz/qemu-6023-console.sock,server,nowait
> -serial chardev:console -nodefaults -m 4G -cpu max -netdev
> user,id=vnet,hostfwd=:127.0.0.1:0-:22 -device
> virtio-net-pci,netdev=vnet -vnc 127.0.0.1:0,to=20 -smp 18 -enable-kvm
> -device VGA -drive
> file=/home/peter.maydell/.cache/qemu-vm/images/openbsd.img.tmp,if=none,id=drive0,cache=writeback
> -device virtio-blk,drive=drive0,bootindex=0 -machine graphics=off
> -cdrom /home/peter.maydell/.cache/qemu-vm/images/openbsd.img.install.iso'
> DEBUG:QMP:>>> {'execute': 'qmp_capabilities'}
> DEBUG:QMP:<<< {'return': {}}
> DEBUG:QMP:>>> {'execute': 'human-monitor-command', 'arguments':
> {'command-line': 'info usernet'}}
> DEBUG:QMP:<<< {'return': 'VLAN -1 (vnet):\r\n  Protocol[State]    FD
> Source Address  Port   Dest. Address  Port RecvQ SendQ\r\n
> TCP[HOST_FORWARD]  13       127.0.0.1 37043       10.0.2.15    22
> 0     0\r\n'}
> console: *** read timeout ***
> console: waiting for: 'boot>'
> console: line buffer:
>
>
> Failed to prepare guest environment
> Traceback (most recent call last):
>   File "/home/peter.maydell/qemu-openbsd/tests/vm/basevm.py", line 351, in main
>     return vm.build_image(args.image)
>   File "/home/peter.maydell/qemu-openbsd/tests/vm/openbsd", line 85,
> in build_image
>     self.console_wait_send("boot>", "set tty com0\n")
>   File "/home/peter.maydell/qemu-openbsd/tests/vm/basevm.py", line
> 251, in console_wait_send
>     self.console_wait(wait)
>   File "/home/peter.maydell/qemu-openbsd/tests/vm/basevm.py", line
> 213, in console_wait
>     chars = vm.console_socket.recv(1024)
> socket.timeout: timed out
> DEBUG:QMP:>>> {'execute': 'quit'}
> DEBUG:QMP:<<< {'return': {}}
> /home/peter.maydell/qemu-openbsd/tests/vm/Makefile.include:47: recipe
> for target '/home/peter.maydell/.cache/qemu-vm/images/openbsd.img'
> failed
> make: *** [/home/peter.maydell/.cache/qemu-vm/images/openbsd.img] Error 2
> make: Leaving directory '/home/peter.maydell/qemu-openbsd/build'
>
> ---
> NetBSD:
>
> peter.maydell@hackbox2.linaro.org:~$ mergebuild netbsd
> hackbox mergebuild: seq netbsd
> make: Entering directory '/home/peter.maydell/qemu-netbsd/build'
> python3 -B /home/peter.maydell/qemu-netbsd/tests/vm/netbsd  --debug
> --image "/home/peter.maydell/.cache/qemu-vm/images/netbsd.img" --force
> --build-image /home/peter.maydell/.cache/qemu-vm/images/netbsd.img
> ### Preparing iso and disk image ...
> Formatting '/home/peter.maydell/.cache/qemu-vm/images/netbsd.img.tmp',
> fmt=qcow2 size=21474836480 cluster_size=65536 lazy_refcounts=off
> refcount_bits=16
> ### Booting installer ...
> DEBUG:root:QEMU args: -nodefaults -m 4G -cpu max -netdev
> user,id=vnet,hostfwd=:127.0.0.1:0-:22 -device
> virtio-net-pci,netdev=vnet -vnc 127.0.0.1:0,to=20 -smp 18 -enable-kvm
> -device VGA -drive
> file=/home/peter.maydell/.cache/qemu-vm/images/netbsd.img.tmp,if=none,id=drive0,cache=writeback
> -device virtio-blk,drive=drive0,bootindex=0 -device VGA -machine
> graphics=off -cdrom
> /home/peter.maydell/.cache/qemu-vm/images/netbsd.img.install.iso
> DEBUG:qemu:VM launch command: 'qemu-system-x86_64 -chardev
> socket,id=mon,path=/var/tmp/tmpp_t16skr/qemu-13312-monitor.sock -mon
> chardev=mon,mode=control -display none -vga none -machine pc -chardev
> socket,id=console,path=/var/tmp/tmpp_t16skr/qemu-13312-console.sock,server,nowait
> -serial chardev:console -nodefaults -m 4G -cpu max -netdev
> user,id=vnet,hostfwd=:127.0.0.1:0-:22 -device
> virtio-net-pci,netdev=vnet -vnc 127.0.0.1:0,to=20 -smp 18 -enable-kvm
> -device VGA -drive
> file=/home/peter.maydell/.cache/qemu-vm/images/netbsd.img.tmp,if=none,id=drive0,cache=writeback
> -device virtio-blk,drive=drive0,bootindex=0 -device VGA -machine
> graphics=off -cdrom
> /home/peter.maydell/.cache/qemu-vm/images/netbsd.img.install.iso'
> DEBUG:QMP:>>> {'execute': 'qmp_capabilities'}
> DEBUG:QMP:>>> {'execute': 'quit'}
> WARNING:qemu:qemu received signal 6: qemu-system-x86_64 -chardev
> socket,id=mon,path=/var/tmp/tmpp_t16skr/qemu-13312-monitor.sock -mon
> chardev=mon,mode=control -display none -vga none -machine pc -chardev
> socket,id=console,path=/var/tmp/tmpp_t16skr/qemu-13312-console.sock,server,nowait
> -serial chardev:console -nodefaults -m 4G -cpu max -netdev
> user,id=vnet,hostfwd=:127.0.0.1:0-:22 -device
> virtio-net-pci,netdev=vnet -vnc 127.0.0.1:0,to=20 -smp 18 -enable-kvm
> -device VGA -drive
> file=/home/peter.maydell/.cache/qemu-vm/images/netbsd.img.tmp,if=none,id=drive0,cache=writeback
> -device virtio-blk,drive=drive0,bootindex=0 -device VGA -machine
> graphics=off -cdrom
> /home/peter.maydell/.cache/qemu-vm/images/netbsd.img.install.iso
> DEBUG:qemu:Error launching VM
> DEBUG:qemu:Command: 'qemu-system-x86_64 -chardev
> socket,id=mon,path=/var/tmp/tmpp_t16skr/qemu-13312-monitor.sock -mon
> chardev=mon,mode=control -display none -vga none -machine pc -chardev
> socket,id=console,path=/var/tmp/tmpp_t16skr/qemu-13312-console.sock,server,nowait
> -serial chardev:console -nodefaults -m 4G -cpu max -netdev
> user,id=vnet,hostfwd=:127.0.0.1:0-:22 -device
> virtio-net-pci,netdev=vnet -vnc 127.0.0.1:0,to=20 -smp 18 -enable-kvm
> -device VGA -drive
> file=/home/peter.maydell/.cache/qemu-vm/images/netbsd.img.tmp,if=none,id=drive0,cache=writeback
> -device virtio-blk,drive=drive0,bootindex=0 -device VGA -machine
> graphics=off -cdrom
> /home/peter.maydell/.cache/qemu-vm/images/netbsd.img.install.iso'
> DEBUG:qemu:Output: 'RAMBlock "vga.vram" already registered, abort!\n'
> ERROR:root:Failed to launch QEMU, command line:
> ERROR:root:qemu-system-x86_64 -nodefaults -m 4G -cpu max -netdev
> user,id=vnet,hostfwd=:127.0.0.1:0-:22 -device
> virtio-net-pci,netdev=vnet -vnc 127.0.0.1:0,to=20 -smp 18 -enable-kvm
> -device VGA -drive
> file=/home/peter.maydell/.cache/qemu-vm/images/netbsd.img.tmp,if=none,id=drive0,cache=writeback
> -device virtio-blk,drive=drive0,bootindex=0 -device VGA -machine
> graphics=off -cdrom
> /home/peter.maydell/.cache/qemu-vm/images/netbsd.img.install.iso
> ERROR:root:Log:
> ERROR:root:RAMBlock "vga.vram" already registered, abort!
>
> ERROR:root:QEMU version >= 2.10 is required
> Failed to prepare guest environment
> Traceback (most recent call last):
>   File "/home/peter.maydell/qemu-netbsd/tests/vm/basevm.py", line 351, in main
>     return vm.build_image(args.image)
>   File "/home/peter.maydell/qemu-netbsd/tests/vm/netbsd", line 79, in
> build_image
>     "-cdrom", iso
>   File "/home/peter.maydell/qemu-netbsd/tests/vm/basevm.py", line 170, in boot
>     guest.launch()
>   File "/home/peter.maydell/qemu-netbsd/tests/vm/../../python/qemu/__init__.py",
> line 297, in launch
>     self._launch()
>   File "/home/peter.maydell/qemu-netbsd/tests/vm/../../python/qemu/__init__.py",
> line 324, in _launch
>     self._post_launch()
>   File "/home/peter.maydell/qemu-netbsd/tests/vm/../../python/qemu/__init__.py",
> line 268, in _post_launch
>     self._qmp.accept()
>   File "/home/peter.maydell/qemu-netbsd/tests/vm/../../python/qemu/qmp.py",
> line 157, in accept
>     return self.__negotiate_capabilities()
>   File "/home/peter.maydell/qemu-netbsd/tests/vm/../../python/qemu/qmp.py",
> line 75, in __negotiate_capabilities
>     resp = self.cmd('qmp_capabilities')
>   File "/home/peter.maydell/qemu-netbsd/tests/vm/../../python/qemu/qmp.py",
> line 191, in cmd
>     return self.cmd_obj(qmp_cmd)
>   File "/home/peter.maydell/qemu-netbsd/tests/vm/../../python/qemu/qmp.py",
> line 174, in cmd_obj
>     resp = self.__json_read()
>   File "/home/peter.maydell/qemu-netbsd/tests/vm/../../python/qemu/qmp.py",
> line 82, in __json_read
>     data = self.__sockfile.readline()
>   File "/usr/lib/python3.6/socket.py", line 586, in readinto
>     return self._sock.recv_into(b)
> ConnectionResetError: [Errno 104] Connection reset by peer
> /home/peter.maydell/qemu-netbsd/tests/vm/Makefile.include:47: recipe
> for target '/home/peter.maydell/.cache/qemu-vm/images/netbsd.img'
> failed
> make: *** [/home/peter.maydell/.cache/qemu-vm/images/netbsd.img] Error 2
> make: Leaving directory '/home/peter.maydell/qemu-netbsd/build'
>
>
> FreeBSD:
>
> peter.maydell@hackbox2.linaro.org:~$ mergebuild freebsd
> hackbox mergebuild: seq freebsd
> make: Entering directory '/home/peter.maydell/qemu-freebsd/build'
> python3 -B /home/peter.maydell/qemu-freebsd/tests/vm/freebsd  --debug
> --image "/home/peter.maydell/.cache/qemu-vm/images/freebsd.img"
> --force --build-image
> /home/peter.maydell/.cache/qemu-vm/images/freebsd.img
> ### Downloading install iso ...
> ### Preparing iso and disk image ...
> /home/peter.maydell/.cache/qemu-vm/images/freebsd.img.install.iso.xz (1/1)
>   100 %       595.0 MiB / 851.1 MiB = 0.699   117 MiB/s       0:07
> Formatting '/home/peter.maydell/.cache/qemu-vm/images/freebsd.img.tmp',
> fmt=qcow2 size=21474836480 cluster_size=65536 lazy_refcounts=off
> refcount_bits=16
> ### Booting installer ...
> DEBUG:root:QEMU args: -nodefaults -m 4G -cpu max -netdev
> user,id=vnet,hostfwd=:127.0.0.1:0-:22 -device
> virtio-net-pci,netdev=vnet -vnc 127.0.0.1:0,to=20 -smp 18 -enable-kvm
> -device VGA -drive
> file=/home/peter.maydell/.cache/qemu-vm/images/freebsd.img.tmp,if=none,id=drive0,cache=writeback
> -device virtio-blk,drive=drive0,bootindex=0 -machine graphics=off
> -cdrom /home/peter.maydell/.cache/qemu-vm/images/freebsd.img.install.iso
> DEBUG:qemu:VM launch command: 'qemu-system-x86_64 -chardev
> socket,id=mon,path=/var/tmp/tmpc1sqg5ms/qemu-23074-monitor.sock -mon
> chardev=mon,mode=control -display none -vga none -machine pc -chardev
> socket,id=console,path=/var/tmp/tmpc1sqg5ms/qemu-23074-console.sock,server,nowait
> -serial chardev:console -nodefaults -m 4G -cpu max -netdev
> user,id=vnet,hostfwd=:127.0.0.1:0-:22 -device
> virtio-net-pci,netdev=vnet -vnc 127.0.0.1:0,to=20 -smp 18 -enable-kvm
> -device VGA -drive
> file=/home/peter.maydell/.cache/qemu-vm/images/freebsd.img.tmp,if=none,id=drive0,cache=writeback
> -device virtio-blk,drive=drive0,bootindex=0 -machine graphics=off
> -cdrom /home/peter.maydell/.cache/qemu-vm/images/freebsd.img.install.iso'
> DEBUG:QMP:>>> {'execute': 'qmp_capabilities'}
> DEBUG:QMP:<<< {'return': {}}
> DEBUG:QMP:>>> {'execute': 'human-monitor-command', 'arguments':
> {'command-line': 'info usernet'}}
> DEBUG:QMP:<<< {'return': 'VLAN -1 (vnet):\r\n  Protocol[State]    FD
> Source Address  Port   Dest. Address  Port RecvQ SendQ\r\n
> TCP[HOST_FORWARD]  13       127.0.0.1 37719       10.0.2.15    22
> 0     0\r\n'}
> console: *** read timeout ***
> console: waiting for: 'Autoboot'
> console: line buffer:
>
>
> Failed to prepare guest environment
> Traceback (most recent call last):
>   File "/home/peter.maydell/qemu-freebsd/tests/vm/basevm.py", line 351, in main
>     return vm.build_image(args.image)
>   File "/home/peter.maydell/qemu-freebsd/tests/vm/freebsd", line 94,
> in build_image
>     self.console_boot_serial()
>   File "/home/peter.maydell/qemu-freebsd/tests/vm/freebsd", line 71,
> in console_boot_serial
>     self.console_wait_send("Autoboot", "3")
>   File "/home/peter.maydell/qemu-freebsd/tests/vm/basevm.py", line
> 251, in console_wait_send
>     self.console_wait(wait)
>   File "/home/peter.maydell/qemu-freebsd/tests/vm/basevm.py", line
> 213, in console_wait
>     chars = vm.console_socket.recv(1024)
> socket.timeout: timed out
> DEBUG:QMP:>>> {'execute': 'quit'}
> DEBUG:QMP:<<< {'timestamp': {'seconds': 1559900897, 'microseconds':
> 206270}, 'event': 'NIC_RX_FILTER_CHANGED', 'data': {'path':
> '/machine/peripheral-anon/device[0]/virtio-backend'}}
> DEBUG:QMP:<<< {'return': {}}
> /home/peter.maydell/qemu-freebsd/tests/vm/Makefile.include:47: recipe
> for target '/home/peter.maydell/.cache/qemu-vm/images/freebsd.img'
> failed
> make: *** [/home/peter.maydell/.cache/qemu-vm/images/freebsd.img] Error 2
> make: Leaving directory '/home/peter.maydell/qemu-freebsd/build'

So I have a branch that works:

  https://github.com/stsquad/qemu/tree/testing/pull-testing-gdbstub-cputlb-070619-1

Which has the following additional patch:

  Subject: [PATCH] tests/vm: favour the locally built QEMU for bootstrapping
  Date: Fri,  7 Jun 2019 19:53:37 +0100
  Message-Id: <20190607185337.14524-1-alex.bennee@linaro.org>

And reverts the patch:

  tests/vm: netbsd autoinstall, using serial console

Which seems to have problems running on hackbox (serial works but we
don't step through the install menus, lots of qqqqqqqqqqqqqqqqqqqqqqq's
in the logs).

Should I respin the PR with 1 new and 1 dropped?

--
Alex Bennée