| Message ID | 20230116173233.2948746-1-alex.bennee@linaro.org (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | [RFC] tests/tcg: skip the vma-pthread test on CI | expand |
On 1/16/23 07:32, Alex Bennée wrote: > diff --git a/tests/tcg/multiarch/Makefile.target b/tests/tcg/multiarch/Makefile.target > index e7213af492..ae8b3d7268 100644 > --- a/tests/tcg/multiarch/Makefile.target > +++ b/tests/tcg/multiarch/Makefile.target > @@ -42,6 +42,15 @@ munmap-pthread: LDFLAGS+=-pthread > vma-pthread: CFLAGS+=-pthread > vma-pthread: LDFLAGS+=-pthread > > +# The vma-pthread seems very sensitive on gitlab and we currently > +# don't know if its exposing a real bug or the test is flaky. > +ifneq ($(GITLAB_CI),) > +run-vma-pthread: vma-pthread > + $(call skip-test, $<, "flaky on CI?") > +run-plugin-vma-pthread-with-%: vma-pthread > + $(call skip-test, $<, "flaky on CI?") > +endif > + Ok I guess. I'd have thought the ifdef around the entire mention of the test would be better -- no point in even building it. But, Acked-by: Richard Henderson <richard.henderson@linaro.org> r~
I did consider it but it would involve messing about with filter to remove the test from the wildcards. This way we don't forget about it when looking through the logs. I've not been able to get this to fail on any other machine though. It's been rock solid over several thousand runs. On Mon, 16 Jan 2023, 19:25 Richard Henderson, <richard.henderson@linaro.org> wrote: > On 1/16/23 07:32, Alex Bennée wrote: > > diff --git a/tests/tcg/multiarch/Makefile.target > b/tests/tcg/multiarch/Makefile.target > > index e7213af492..ae8b3d7268 100644 > > --- a/tests/tcg/multiarch/Makefile.target > > +++ b/tests/tcg/multiarch/Makefile.target > > @@ -42,6 +42,15 @@ munmap-pthread: LDFLAGS+=-pthread > > vma-pthread: CFLAGS+=-pthread > > vma-pthread: LDFLAGS+=-pthread > > > > +# The vma-pthread seems very sensitive on gitlab and we currently > > +# don't know if its exposing a real bug or the test is flaky. > > +ifneq ($(GITLAB_CI),) > > +run-vma-pthread: vma-pthread > > + $(call skip-test, $<, "flaky on CI?") > > +run-plugin-vma-pthread-with-%: vma-pthread > > + $(call skip-test, $<, "flaky on CI?") > > +endif > > + > > Ok I guess. I'd have thought the ifdef around the entire mention of the > test would be > better -- no point in even building it. But, > > Acked-by: Richard Henderson <richard.henderson@linaro.org> > > > r~ >
diff --git a/tests/tcg/multiarch/Makefile.target b/tests/tcg/multiarch/Makefile.target index e7213af492..ae8b3d7268 100644 --- a/tests/tcg/multiarch/Makefile.target +++ b/tests/tcg/multiarch/Makefile.target @@ -42,6 +42,15 @@ munmap-pthread: LDFLAGS+=-pthread vma-pthread: CFLAGS+=-pthread vma-pthread: LDFLAGS+=-pthread +# The vma-pthread seems very sensitive on gitlab and we currently +# don't know if its exposing a real bug or the test is flaky. +ifneq ($(GITLAB_CI),) +run-vma-pthread: vma-pthread + $(call skip-test, $<, "flaky on CI?") +run-plugin-vma-pthread-with-%: vma-pthread + $(call skip-test, $<, "flaky on CI?") +endif + # We define the runner for test-mmap after the individual # architectures have defined their supported pages sizes. If no # additional page sizes are defined we only run the default test.
We are getting a lot of failures that are not related to changes so this could be a flaky test. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Cc: Richard Henderson <richard.henderson@linaro.org> --- tests/tcg/multiarch/Makefile.target | 9 +++++++++ 1 file changed, 9 insertions(+)