| Message ID | 20221213183242.1908249-1-nfraprado@collabora.com (mailing list archive) |
|---|---|
| State | Superseded |
| Headers | show |
| Series | kselftest/alsa: Increase kselftest timeout | expand |
On Tue, Dec 13, 2022 at 03:32:42PM -0300, Nícolas F. R. A. Prado wrote: > The default timeout for kselftests is 45 seconds, but pcm-test can take > longer than that to run depending on the number of PCMs present on a > device. Reviewed-by: Mark Brown <broonie@kernel.org> This is also useful for mixer-test with slower control buses and fancier CODECs.
On 12/13/22 11:32, Nícolas F. R. A. Prado wrote: > The default timeout for kselftests is 45 seconds, but pcm-test can take > longer than that to run depending on the number of PCMs present on a > device. > > As a data point, running pcm-test on mt8192-asurada-spherion takes about > 1m15s. > > Set the timeout to 10 minutes, which should give enough slack to run the > test even on devices with many PCMs. > 10 minutes is way too long. > Signed-off-by: Nícolas F. R. A. Prado <nfraprado@collabora.com> > > --- > > tools/testing/selftests/alsa/settings | 1 + > 1 file changed, 1 insertion(+) > create mode 100644 tools/testing/selftests/alsa/settings > > diff --git a/tools/testing/selftests/alsa/settings b/tools/testing/selftests/alsa/settings > new file mode 100644 > index 000000000000..a62d2fa1275c > --- /dev/null > +++ b/tools/testing/selftests/alsa/settings > @@ -0,0 +1 @@ > +timeout=600 Adding timeouts like this especially 10 minutes will increase the time it takes to run tests. We run the risk of people not wanting to run tests anymore. thanks, -- Shuah
On Tue, Dec 13, 2022 at 11:37:56AM -0700, Shuah Khan wrote: > On 12/13/22 11:32, Nícolas F. R. A. Prado wrote: > > The default timeout for kselftests is 45 seconds, but pcm-test can take > > longer than that to run depending on the number of PCMs present on a > > device. > > > > As a data point, running pcm-test on mt8192-asurada-spherion takes about > > 1m15s. > > > > Set the timeout to 10 minutes, which should give enough slack to run the > > test even on devices with many PCMs. > > > > 10 minutes is way too long. > > > Signed-off-by: Nícolas F. R. A. Prado <nfraprado@collabora.com> > > > > --- > > > > tools/testing/selftests/alsa/settings | 1 + > > 1 file changed, 1 insertion(+) > > create mode 100644 tools/testing/selftests/alsa/settings > > > > diff --git a/tools/testing/selftests/alsa/settings b/tools/testing/selftests/alsa/settings > > new file mode 100644 > > index 000000000000..a62d2fa1275c > > --- /dev/null > > +++ b/tools/testing/selftests/alsa/settings > > @@ -0,0 +1 @@ > > +timeout=600 > > Adding timeouts like this especially 10 minutes will increase the time > it takes to run tests. We run the risk of people not wanting to run tests > anymore. I set it high because I suspect the time it takes to run pcm-test might be a lot higher in some machines (even on mt8192-asurada-spherion, it takes 1m15s, but only ~10% of the tests are actually running, since many of the PCMs are disabled). But I can lower it to, say, 3 minutes, and we can increase the timeout as needed. Thanks, Nícolas
Hi Shuah, Bumping an old thread, this is still an issue for this test [1] and it could end up affecting many others. On mar 13-12-2022 11:37:56, Shuah Khan wrote: > On 12/13/22 11:32, Nícolas F. R. A. Prado wrote: > > The default timeout for kselftests is 45 seconds, but pcm-test can take > > longer than that to run depending on the number of PCMs present on a > > device. > > > > As a data point, running pcm-test on mt8192-asurada-spherion takes about > > 1m15s. > > > > Set the timeout to 10 minutes, which should give enough slack to run the > > test even on devices with many PCMs. > > > > 10 minutes is way too long. Is there a downside to that? There'll be some tests that take more time, I don't think that's a problem with the tests or a reason to let them timeout. IMO it's the test framework which should adapt to the needs of different types of tests, and the solution provided by this patch is good enough, it addresses the problem for this particular test suite. If the solution is still unacceptable, do you have an alternative proposal in mind that we can try to implement? Thanks, Ricardo [1] https://linux.kernelci.org/test/case/id/646127cf62438996ba2e8648/
Hi Shuah, Gentle ping for this On lun, may 15 2023 at 11:43:10, Ricardo Cañuelo <ricardo.canuelo@collabora.com> wrote: > Is there a downside to that? There'll be some tests that take more time, > I don't think that's a problem with the tests or a reason to let them > timeout. IMO it's the test framework which should adapt to the needs of > different types of tests, and the solution provided by this patch is > good enough, it addresses the problem for this particular test suite. > > If the solution is still unacceptable, do you have an alternative > proposal in mind that we can try to implement? There are some tests failing because of this and we're trying to address these problems to clean up the regression results. Thanks, Ricardo
diff --git a/tools/testing/selftests/alsa/settings b/tools/testing/selftests/alsa/settings new file mode 100644 index 000000000000..a62d2fa1275c --- /dev/null +++ b/tools/testing/selftests/alsa/settings @@ -0,0 +1 @@ +timeout=600
The default timeout for kselftests is 45 seconds, but pcm-test can take longer than that to run depending on the number of PCMs present on a device. As a data point, running pcm-test on mt8192-asurada-spherion takes about 1m15s. Set the timeout to 10 minutes, which should give enough slack to run the test even on devices with many PCMs. Signed-off-by: Nícolas F. R. A. Prado <nfraprado@collabora.com> --- tools/testing/selftests/alsa/settings | 1 + 1 file changed, 1 insertion(+) create mode 100644 tools/testing/selftests/alsa/settings