diff mbox series

[RFC,1/2] tests/test-qht-par: test gets stuck intermittently on OSX

Message ID 20181109153002.597-2-crosa@redhat.com (mailing list archive)
State New, archived
Headers show
Series Address OS X Travis failures | expand

Commit Message

Cleber Rosa Nov. 9, 2018, 3:30 p.m. UTC
To be fully honest, this may not be a OSX (alone) condition, but may
be a situation that only happens with OSX on Travis-CI, were resources
are quite limited.

I have personal experience with tests that exercise parallelism or
depend on timing to fail on Travis.  Because I'm not 100% certain that
this is a situation that only happens with OSX on Travis-CI, and
because I'm not certain that we should be skipping tests because
they're running on Travis-CI, let's disable them on OSX as a whole.

A small note: this type of change makes me believe that there should
be a list of testing related caveats or TODO list tracked on the
documentation.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
---
 tests/Makefile.include | 3 +++
 1 file changed, 3 insertions(+)

Comments

Peter Maydell Nov. 9, 2018, 3:34 p.m. UTC | #1
On 9 November 2018 at 15:30, Cleber Rosa <crosa@redhat.com> wrote:
> To be fully honest, this may not be a OSX (alone) condition, but may
> be a situation that only happens with OSX on Travis-CI, were resources
> are quite limited.
>
> I have personal experience with tests that exercise parallelism or
> depend on timing to fail on Travis.  Because I'm not 100% certain that
> this is a situation that only happens with OSX on Travis-CI, and
> because I'm not certain that we should be skipping tests because
> they're running on Travis-CI, let's disable them on OSX as a whole.

Does this test still get stuck in builds after a458774ad711bceabefb ?
That fixed an OSX-specific problem that caused hangs (notably
in test-bdrv-drain, but it could I guess also affect other tests).

thanks
-- PMM
Emilio Cota Nov. 9, 2018, 3:56 p.m. UTC | #2
On Fri, Nov 09, 2018 at 10:30:01 -0500, Cleber Rosa wrote:
> To be fully honest, this may not be a OSX (alone) condition, but may
> be a situation that only happens with OSX on Travis-CI, were resources
> are quite limited.
> 
> I have personal experience with tests that exercise parallelism or
> depend on timing to fail on Travis.  Because I'm not 100% certain that
> this is a situation that only happens with OSX on Travis-CI, and
> because I'm not certain that we should be skipping tests because
> they're running on Travis-CI, let's disable them on OSX as a whole.
> 
> A small note: this type of change makes me believe that there should
> be a list of testing related caveats or TODO list tracked on the
> documentation.
> 
> Signed-off-by: Cleber Rosa <crosa@redhat.com>

Is there a way to know where it hangs?

Also, are these the g_test_quick tests, or the other ones? It it's
just a matter of the program taking too long to run, we could just
reduce the length of the !quick tests (currently at 5s vs. 1s for
the quick ones).

Thanks,

		Emilio
Cleber Rosa Nov. 9, 2018, 5:06 p.m. UTC | #3
On 11/9/18 10:34 AM, Peter Maydell wrote:
> On 9 November 2018 at 15:30, Cleber Rosa <crosa@redhat.com> wrote:
>> To be fully honest, this may not be a OSX (alone) condition, but may
>> be a situation that only happens with OSX on Travis-CI, were resources
>> are quite limited.
>>
>> I have personal experience with tests that exercise parallelism or
>> depend on timing to fail on Travis.  Because I'm not 100% certain that
>> this is a situation that only happens with OSX on Travis-CI, and
>> because I'm not certain that we should be skipping tests because
>> they're running on Travis-CI, let's disable them on OSX as a whole.
> 
> Does this test still get stuck in builds after a458774ad711bceabefb ?
> That fixed an OSX-specific problem that caused hangs (notably
> in test-bdrv-drain, but it could I guess also affect other tests).
> 

It didn't get stuck here:

https://travis-ci.org/clebergnu/qemu/jobs/452663111

So I believe that was the common issue indeed.  Thanks for pointing that
out.  Also, the other patch in this series also seems to be unnecessary now.

- Cleber.

> thanks
> -- PMM
>
Cleber Rosa Nov. 9, 2018, 5:08 p.m. UTC | #4
On 11/9/18 10:56 AM, Emilio G. Cota wrote:
> On Fri, Nov 09, 2018 at 10:30:01 -0500, Cleber Rosa wrote:
>> To be fully honest, this may not be a OSX (alone) condition, but may
>> be a situation that only happens with OSX on Travis-CI, were resources
>> are quite limited.
>>
>> I have personal experience with tests that exercise parallelism or
>> depend on timing to fail on Travis.  Because I'm not 100% certain that
>> this is a situation that only happens with OSX on Travis-CI, and
>> because I'm not certain that we should be skipping tests because
>> they're running on Travis-CI, let's disable them on OSX as a whole.
>>
>> A small note: this type of change makes me believe that there should
>> be a list of testing related caveats or TODO list tracked on the
>> documentation.
>>
>> Signed-off-by: Cleber Rosa <crosa@redhat.com>
> 
> Is there a way to know where it hangs?
> 
> Also, are these the g_test_quick tests, or the other ones? It it's
> just a matter of the program taking too long to run, we could just
> reduce the length of the !quick tests (currently at 5s vs. 1s for
> the quick ones).
> 
> Thanks,
> 
> 		Emilio
> 

Looks like the common cause was fixed a458774ad711bceabefb, as pointed
out by Peter.

- Cleber.
diff mbox series

Patch

diff --git a/tests/Makefile.include b/tests/Makefile.include
index 074eece558..c821b01467 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -87,7 +87,10 @@  check-unit-y += tests/test-rcu-simpleq$(EXESUF)
 check-unit-y += tests/test-rcu-tailq$(EXESUF)
 check-unit-y += tests/test-qdist$(EXESUF)
 check-unit-y += tests/test-qht$(EXESUF)
+# test-qht-par gets stuck quite often on OSX
+ifneq ($(CONFIG_DARWIN),y)
 check-unit-y += tests/test-qht-par$(EXESUF)
+endif
 check-unit-y += tests/test-bitops$(EXESUF)
 check-unit-y += tests/test-bitcnt$(EXESUF)
 check-unit-y += tests/test-qdev-global-props$(EXESUF)