| Message ID | 20220303160330.2979753-7-berrange@redhat.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | nbd: enable use of TLS on non-TCP transports and other TLS improvements | expand |
On Thu, Mar 03, 2022 at 04:03:24PM +0000, Daniel P. Berrangé wrote: > When developing an I/O test it is typical to add some logic to the > test script, run it to view the output diff, and then apply the > output diff to the reference file. This can be drastically simplified > by letting the test runner update the reference file in place. > > By setting 'QEMU_IOTESTS_REGEN=1', the test runner will report the > failure and show the diff, but at the same time update the reference > file. So next time the I/O test is run it will succeed. > > Continuing to display the diff when updating the reference gives the > developer a chance to review what was changed. > > Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> > --- > tests/qemu-iotests/testrunner.py | 6 ++++++ > 1 file changed, 6 insertions(+) I like it - I've wished for similar when writing a test. Reviewed-by: Eric Blake <eblake@redhat.com>
diff --git a/tests/qemu-iotests/testrunner.py b/tests/qemu-iotests/testrunner.py index 9a94273975..8a82696a6b 100644 --- a/tests/qemu-iotests/testrunner.py +++ b/tests/qemu-iotests/testrunner.py @@ -25,6 +25,7 @@ import contextlib import json import termios +import shutil import sys from multiprocessing import Pool from contextlib import contextmanager @@ -320,6 +321,11 @@ def do_run_test(self, test: str, mp: bool) -> TestResult: diff = file_diff(str(f_reference), str(f_bad)) if diff: + if os.environ.get("QEMU_IOTESTS_REGEN", None) is not None: + shutil.copyfile(str(f_bad), str(f_reference)) + print("########################################") + print("##### REFERENCE FILE UPDATED #####") + print("########################################") return TestResult(status='fail', elapsed=elapsed, description=f'output mismatch (see {f_bad})', diff=diff, casenotrun=casenotrun)
When developing an I/O test it is typical to add some logic to the test script, run it to view the output diff, and then apply the output diff to the reference file. This can be drastically simplified by letting the test runner update the reference file in place. By setting 'QEMU_IOTESTS_REGEN=1', the test runner will report the failure and show the diff, but at the same time update the reference file. So next time the I/O test is run it will succeed. Continuing to display the diff when updating the reference gives the developer a chance to review what was changed. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- tests/qemu-iotests/testrunner.py | 6 ++++++ 1 file changed, 6 insertions(+)