| Message ID | 20230717182859.707658-8-berrange@redhat.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | tests: enable meson test timeouts to improve debuggability | expand |
On 17/07/2023 20.28, Daniel P. Berrangé wrote: > The prom-env-test takes about 1 + 1/2 minutes in a --enable-debug > build. Bumping to 3 minutes will give more headroom. > > Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> > --- > tests/qtest/meson.build | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/tests/qtest/meson.build b/tests/qtest/meson.build > index c6da428dc5..095c98820e 100644 > --- a/tests/qtest/meson.build > +++ b/tests/qtest/meson.build > @@ -5,6 +5,7 @@ slow_qtests = { > 'qom-test' : 900, > 'test-hmp' : 240, > 'pxe-test': 180, > + 'prom-env-test': 180, > } > > qtests_generic = [ I tested your patches, and initially, everything looked good now. But this prom-env-test them reminded me that we run some additional tests in the "SPEED=slow" mode ... I guess we have to take these into consideration, too? I now did a "configure --enable-debug" build and then ran: make -j$(nproc) check-qtest-ppc64 SPEED=slow and indeed, this prom-env-test is now timing out there. Also the device-introspect-test was timing out in SPEED=slow mode. Should we bump the timeout for those, or could this maybe be handled via the TIMEOUT_MULTIPLIER in the final patch? Thomas
On 19/07/2023 15.22, Thomas Huth wrote: > On 17/07/2023 20.28, Daniel P. Berrangé wrote: >> The prom-env-test takes about 1 + 1/2 minutes in a --enable-debug >> build. Bumping to 3 minutes will give more headroom. >> >> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> >> --- >> tests/qtest/meson.build | 1 + >> 1 file changed, 1 insertion(+) >> >> diff --git a/tests/qtest/meson.build b/tests/qtest/meson.build >> index c6da428dc5..095c98820e 100644 >> --- a/tests/qtest/meson.build >> +++ b/tests/qtest/meson.build >> @@ -5,6 +5,7 @@ slow_qtests = { >> 'qom-test' : 900, >> 'test-hmp' : 240, >> 'pxe-test': 180, >> + 'prom-env-test': 180, >> } >> qtests_generic = [ > > I tested your patches, and initially, everything looked good now. But this > prom-env-test them reminded me that we run some additional tests in the > "SPEED=slow" mode ... I guess we have to take these into consideration, too? > > I now did a "configure --enable-debug" build and then ran: > > make -j$(nproc) check-qtest-ppc64 SPEED=slow > > and indeed, this prom-env-test is now timing out there. Also the > device-introspect-test was timing out in SPEED=slow mode. Should we bump the > timeout for those, or could this maybe be handled via the TIMEOUT_MULTIPLIER > in the final patch? I've now done a full check-qtest run with SPEED=slow and --enable-debug, and these tests are timing out: 14/131 qemu:qtest+qtest-aarch64 / qtest-aarch64/migration-test TIMEOUT 300.02s killed by signal 15 SIGTERM 89/131 qemu:qtest+qtest-s390x / qtest-s390x/device-introspect-test TIMEOUT 60.01s killed by signal 15 SIGTERM 103/131 qemu:qtest+qtest-ppc64 / qtest-ppc64/device-introspect-test TIMEOUT 60.02s killed by signal 15 SIGTERM 108/131 qemu:qtest+qtest-ppc64 / qtest-ppc64/pxe-test TIMEOUT 180.02s killed by signal 15 SIGTERM 129/131 qemu:qtest+qtest-x86_64 / qtest-x86_64/device-introspect-test TIMEOUT 60.05s killed by signal 15 SIGTERM 130/131 qemu:qtest+qtest-aarch64 / qtest-aarch64/device-introspect-test TIMEOUT 60.01s killed by signal 15 SIGTERM Thomas
On Wed, Jul 19, 2023 at 03:22:30PM +0200, Thomas Huth wrote: > On 17/07/2023 20.28, Daniel P. Berrangé wrote: > > The prom-env-test takes about 1 + 1/2 minutes in a --enable-debug > > build. Bumping to 3 minutes will give more headroom. > > > > Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> > > --- > > tests/qtest/meson.build | 1 + > > 1 file changed, 1 insertion(+) > > > > diff --git a/tests/qtest/meson.build b/tests/qtest/meson.build > > index c6da428dc5..095c98820e 100644 > > --- a/tests/qtest/meson.build > > +++ b/tests/qtest/meson.build > > @@ -5,6 +5,7 @@ slow_qtests = { > > 'qom-test' : 900, > > 'test-hmp' : 240, > > 'pxe-test': 180, > > + 'prom-env-test': 180, > > } > > qtests_generic = [ > > I tested your patches, and initially, everything looked good now. But this > prom-env-test them reminded me that we run some additional tests in the > "SPEED=slow" mode ... I guess we have to take these into consideration, too? > > I now did a "configure --enable-debug" build and then ran: > > make -j$(nproc) check-qtest-ppc64 SPEED=slow > > and indeed, this prom-env-test is now timing out there. Also the > device-introspect-test was timing out in SPEED=slow mode. Should we bump the > timeout for those, or could this maybe be handled via the TIMEOUT_MULTIPLIER > in the final patch? My goal was "no regressions" wrt current status for people working on common development hardware. If people are using old, or modern low performance hardware than a timeout multiplier would be expected. IOW, for SPEED=slow, I think we need to "do the right thing" with the default timeouts. With regards, Daniel
diff --git a/tests/qtest/meson.build b/tests/qtest/meson.build index c6da428dc5..095c98820e 100644 --- a/tests/qtest/meson.build +++ b/tests/qtest/meson.build @@ -5,6 +5,7 @@ slow_qtests = { 'qom-test' : 900, 'test-hmp' : 240, 'pxe-test': 180, + 'prom-env-test': 180, } qtests_generic = [
The prom-env-test takes about 1 + 1/2 minutes in a --enable-debug build. Bumping to 3 minutes will give more headroom. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- tests/qtest/meson.build | 1 + 1 file changed, 1 insertion(+)