| Message ID | 20241121154218.1423005-1-berrange@redhat.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | test/functional: improve functional test debugging & fix tuxrun | expand |
Daniel P. Berrangé <berrange@redhat.com> writes: > This started out as a series to get rid of the many GBs of temp > files the functional tests leave behind. Then it expanded into > improving the functional test debugging by ensuring we preserve > the QEMU stdout/stderr log file created by the QEMUMachine class. > In the course of doing that I encountered some other minor points > worth fixing, and then got side tracked into looking at the tuxrun > hangs with aarch64be. Investigating the latter exposed some further > holes in the debugging story prompting yet more patches, as well as > a final solution for tuxrun. So this series does: > > * Purge all scratch files created by tests > * Preserve the stdout/stderr log file > * Capture debug log messages on QEMUMachine > * Provide a QMP backdoor for debugging stuck QEMUs > * Enhance console handling for partial line matches > * Fix the tuxrun tests by eliminating sleeps > > There's quite alot of code here, but at the same time it feels like > the kind of stuff that'll be valuable either in the 9.2 release, or > in the soon to exist 9.2 stable branch. > > NB, with this series applied Thomas' tuxrun conversion to functional > testing survives 200 iterations on my machine, whereas it would > reliably hang in < 20, and often in < 10, before. Queued to testing/next, thanks. I'll combine with plugins/next and some other misc fixes and post a pre-PR for next week. > > Changed in v2: > > - Changed console interaction to forbid 'failure_message' > without 'success_message' > - Reword console interaction log messages > - Avoid stack trace when seeing early failure > - Rewrote comment in acpi bits test > - Avoid duplicate os.environ access > > Daniel P. Berrangé (18): > tests/functional: fix mips64el test to honour workdir > tests/functional: automatically clean up scratch files after tests > tests/functional: remove "AVOCADO" from env variable name > tests/functional: remove todo wrt avocado.utils.wait_for > tests/functional: remove leftover :avocado: tags > tests/functional: remove obsolete reference to avocado bug > tests/functional: remove comments talking about avocado > tests/functional: honour self.workdir in ACPI bits tests > tests/functional: put QEMUMachine logs in testcase log directory > tests/functional: honour requested test VM name in QEMUMachine > tests/functional: enable debug logging for QEMUMachine > tests/functional: logs details of console interaction operations > tests/functional: don't try to wait for the empty string > tests/functional: require non-NULL success_message for console wait > tests/functional: rewrite console handling to be bytewise > tests/functional: remove time.sleep usage from tuxrun tests > tests/functional: add a QMP backdoor for debugging stalled tests > tests/functional: avoid accessing log_filename on earlier failures > > docs/devel/testing/functional.rst | 16 +++++ > tests/functional/qemu_test/cmd.py | 89 +++++++++++++++++++----- > tests/functional/qemu_test/testcase.py | 43 +++++++++--- > tests/functional/qemu_test/tuxruntest.py | 17 ++--- > tests/functional/test_acpi_bits.py | 57 +++++---------- > tests/functional/test_arm_bpim2u.py | 20 ------ > tests/functional/test_arm_orangepi.py | 27 ------- > tests/functional/test_m68k_nextcube.py | 3 +- > tests/functional/test_mips64el_malta.py | 4 +- > tests/functional/test_virtio_gpu.py | 3 +- > 10 files changed, 150 insertions(+), 129 deletions(-)