[kvm-unit-tests,RFC,0/5] arm: Add kvmtool to the runner script

Message

Alexandru Elisei July 2, 2021, 4:31 p.m. UTC

kvm-unit-tests has support for running arm64 tests under kvmtool since
2019. However, each test had to be run manually since the runner script was
blissfully unware of other VMMs beside qemu.

This series aims to support running all the tests automatically with
kvmtool, like it has always been possible to do with qemu by invoking:

$ ./run_tests.sh

The runner script will choose kvmtool as the VMM if the tests has been
configured with the kvmtool target.

I see several advantages to being able to use kvmtool alongside qemu:

- kvmtool is smaller and a lot easier to hack than qemu, which means it may
  be possible for developers to prefer it over qemu when adding new
  features to KVM. Being able to run all the tests reliably and
  automatically is useful in the development process.

- kvmtool runs all the tests faster than qemu (roughly 3 times faster on
  4xA53s). I think this is another nice feature for development.

- kvmtool does things differently than qemu: different memory layout,
  different uart, PMU emulation is optional, etc. This makes it a good
  testing vehicule for kvm-unit-tests.

This series is an RFC for various reasons:

- The migration tests work under kvmtool because when kvm-unit-tests
  writes something to the UART (like "Now migrate the VM, then press a key
  a key to continue...\n"), a read will return the last character that was
  written (newline, in this case). I don't know if this is a feature or a
  bug with the kvm-unit-tests UART mini-driver or with kvmtool, and I'm
  investigating it.

- I've tried to keep the changes as small as possible, but I would like
  some feedback about my approach, as I am not very familiar with bash
  scripting.

- The series needs more testing. I've only tested the patches on a
  rockpro64 with qemu and kvmtool, and on my Ryzen amd64
  machine (qemu only, --target=kvmtool is available only for arm/arm64).

TODO:

- More testing, especially on powerpc and s390x which are touched in patch #2.
- README changes to reflect kvmtool support for the runner script.
- Figure out how to handle migration tests under kvmtool.

Comments welcome and much appreciated.

Alexandru Elisei (5):
  lib: arm: Print test exit status on exit if chr-testdev is not
    available
  scripts: Rename run_qemu_status -> run_test_status
  run_tests.sh: Add kvmtool support
  scripts: Generate kvmtool standalone tests
  configure: Ignore --erratatxt when --target=kvmtool

 scripts/arch-run.bash   |  50 ++++++++++++++++--
 scripts/runtime.bash    |  94 ++++++++++++++++++++++++++++------
 scripts/mkstandalone.sh |   9 +++-
 arm/run                 | 110 ++++++++++++++++++++++++----------------
 powerpc/run             |   2 +-
 s390x/run               |   2 +-
 run_tests.sh            |  11 +++-
 configure               |  26 +++++++---
 lib/chr-testdev.h       |   1 +
 lib/arm/io.c            |  10 +++-
 lib/chr-testdev.c       |   5 ++
 11 files changed, 243 insertions(+), 77 deletions(-)