diff mbox series

[3/4] tests/docker: Add test-acceptance runner

Message ID 20190818231827.27573-4-philmd@redhat.com (mailing list archive)
State New, archived
Headers show
Series docker: Update Travis-CI image to run acceptance tests | expand

Commit Message

Philippe Mathieu-Daudé Aug. 18, 2019, 11:18 p.m. UTC
Add a runner script to be able to run acceptance tests within
Docker images. We can now reproduce Travis CI builds locally (and
debug  them!).

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 tests/docker/test-acceptance | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)
 create mode 100755 tests/docker/test-acceptance

Comments

Cleber Rosa Aug. 20, 2019, 1:31 p.m. UTC | #1
On Mon, Aug 19, 2019 at 01:18:26AM +0200, Philippe Mathieu-Daudé wrote:
> Add a runner script to be able to run acceptance tests within
> Docker images. We can now reproduce Travis CI builds locally (and
> debug  them!).
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
>  tests/docker/test-acceptance | 21 +++++++++++++++++++++
>  1 file changed, 21 insertions(+)
>  create mode 100755 tests/docker/test-acceptance
> 
> diff --git a/tests/docker/test-acceptance b/tests/docker/test-acceptance
> new file mode 100755
> index 0000000000..84edaa676c
> --- /dev/null
> +++ b/tests/docker/test-acceptance
> @@ -0,0 +1,21 @@
> +#!/bin/bash -e
> +#
> +# Compile default Travis-CI target and run Avocado acceptance tests
> +#
> +# Copyright (c) 2019 Red Hat Inc.
> +#
> +# Authors:
> +#  Philippe Mathieu-Daudé <philmd@redhat.com>
> +#
> +# This work is licensed under the terms of the GNU GPL, version 2
> +# or (at your option) any later version. See the COPYING file in
> +# the top-level directory.
> +
> +. common.rc
> +
> +cd "$BUILD_DIR"
> +
> +DEF_TARGET_LIST="x86_64-softmmu,mips-softmmu,mips64el-softmmu,aarch64-softmmu,arm-softmmu,s390x-softmmu,alpha-softmmu"
> +TARGET_LIST=${TARGET_LIST:-$DEF_TARGET_LIST} \
> +build_qemu
> +check_qemu check-acceptance
> -- 
> 2.20.1
> 
> 

I'm currently seeing test errors when running in a container:

  MKDIR   /tmp/qemu-test/build/tests/results
  AVOCADO tests/acceptance
JOB ID     : fe56cc0b2d1adbc0b5bb5828902e113d596edccf
JOB LOG    : /tmp/qemu-test/build/tests/results/job-2019-08-19T22.13-fe56cc0/job.log
 (01/27) /tmp/qemu-test/src/tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_x86_64_pc:  ERROR: join() argument must be str or bytes, not 'NoneType' (0.05 s)
Interrupting job (failfast).
RESULTS    : PASS 0 | ERROR 1 | FAIL 0 | SKIP 26 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB TIME   : 0.26 s
/tmp/qemu-test/src/tests/Makefile.include:1158: recipe for target 'check-acceptance' failed
make: *** [check-acceptance] Error 9

That being said, I'm not running it under docker, but under podman,
although I'm not convinced yet that is the defining issue.  I'll try
to identify what's going here.

Cheers,
- Cleber.
Alex Bennée Dec. 3, 2019, 1:41 p.m. UTC | #2
Cleber Rosa <crosa@redhat.com> writes:

> On Mon, Aug 19, 2019 at 01:18:26AM +0200, Philippe Mathieu-Daudé wrote:
>> Add a runner script to be able to run acceptance tests within
>> Docker images. We can now reproduce Travis CI builds locally (and
>> debug  them!).
>> 
>> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
>> ---
>>  tests/docker/test-acceptance | 21 +++++++++++++++++++++
>>  1 file changed, 21 insertions(+)
>>  create mode 100755 tests/docker/test-acceptance
>> 
>> diff --git a/tests/docker/test-acceptance b/tests/docker/test-acceptance
>> new file mode 100755
>> index 0000000000..84edaa676c
>> --- /dev/null
>> +++ b/tests/docker/test-acceptance
>> @@ -0,0 +1,21 @@
>> +#!/bin/bash -e
>> +#
>> +# Compile default Travis-CI target and run Avocado acceptance tests
>> +#
>> +# Copyright (c) 2019 Red Hat Inc.
>> +#
>> +# Authors:
>> +#  Philippe Mathieu-Daudé <philmd@redhat.com>
>> +#
>> +# This work is licensed under the terms of the GNU GPL, version 2
>> +# or (at your option) any later version. See the COPYING file in
>> +# the top-level directory.
>> +
>> +. common.rc
>> +
>> +cd "$BUILD_DIR"
>> +
>> +DEF_TARGET_LIST="x86_64-softmmu,mips-softmmu,mips64el-softmmu,aarch64-softmmu,arm-softmmu,s390x-softmmu,alpha-softmmu"
>> +TARGET_LIST=${TARGET_LIST:-$DEF_TARGET_LIST} \
>> +build_qemu
>> +check_qemu check-acceptance
>> -- 
>> 2.20.1
>> 
>> 
>
> I'm currently seeing test errors when running in a container:
>
>   MKDIR   /tmp/qemu-test/build/tests/results
>   AVOCADO tests/acceptance
> JOB ID     : fe56cc0b2d1adbc0b5bb5828902e113d596edccf
> JOB LOG    : /tmp/qemu-test/build/tests/results/job-2019-08-19T22.13-fe56cc0/job.log
>  (01/27) /tmp/qemu-test/src/tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_x86_64_pc:  ERROR: join() argument must be str or bytes, not 'NoneType' (0.05 s)
> Interrupting job (failfast).
> RESULTS    : PASS 0 | ERROR 1 | FAIL 0 | SKIP 26 | WARN 0 | INTERRUPT 0 | CANCEL 0
> JOB TIME   : 0.26 s
> /tmp/qemu-test/src/tests/Makefile.include:1158: recipe for target 'check-acceptance' failed
> make: *** [check-acceptance] Error 9
>
> That being said, I'm not running it under docker, but under podman,
> although I'm not convinced yet that is the defining issue.  I'll try
> to identify what's going here.

Was there a conclusion to the discussion about this series?
Philippe Mathieu-Daudé Feb. 24, 2020, 12:23 p.m. UTC | #3
On 12/3/19 2:41 PM, Alex Bennée wrote:
> Cleber Rosa <crosa@redhat.com> writes:
>> On Mon, Aug 19, 2019 at 01:18:26AM +0200, Philippe Mathieu-Daudé wrote:
>>> Add a runner script to be able to run acceptance tests within
>>> Docker images. We can now reproduce Travis CI builds locally (and
>>> debug  them!).
>>>
>>> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
>>> ---
>>>   tests/docker/test-acceptance | 21 +++++++++++++++++++++
>>>   1 file changed, 21 insertions(+)
>>>   create mode 100755 tests/docker/test-acceptance
>>>
>>> diff --git a/tests/docker/test-acceptance b/tests/docker/test-acceptance
>>> new file mode 100755
>>> index 0000000000..84edaa676c
>>> --- /dev/null
>>> +++ b/tests/docker/test-acceptance
>>> @@ -0,0 +1,21 @@
>>> +#!/bin/bash -e
>>> +#
>>> +# Compile default Travis-CI target and run Avocado acceptance tests
>>> +#
>>> +# Copyright (c) 2019 Red Hat Inc.
>>> +#
>>> +# Authors:
>>> +#  Philippe Mathieu-Daudé <philmd@redhat.com>
>>> +#
>>> +# This work is licensed under the terms of the GNU GPL, version 2
>>> +# or (at your option) any later version. See the COPYING file in
>>> +# the top-level directory.
>>> +
>>> +. common.rc
>>> +
>>> +cd "$BUILD_DIR"
>>> +
>>> +DEF_TARGET_LIST="x86_64-softmmu,mips-softmmu,mips64el-softmmu,aarch64-softmmu,arm-softmmu,s390x-softmmu,alpha-softmmu"
>>> +TARGET_LIST=${TARGET_LIST:-$DEF_TARGET_LIST} \
>>> +build_qemu
>>> +check_qemu check-acceptance
>>> -- 
>>> 2.20.1
>>>
>>>
>>
>> I'm currently seeing test errors when running in a container:
>>
>>    MKDIR   /tmp/qemu-test/build/tests/results
>>    AVOCADO tests/acceptance
>> JOB ID     : fe56cc0b2d1adbc0b5bb5828902e113d596edccf
>> JOB LOG    : /tmp/qemu-test/build/tests/results/job-2019-08-19T22.13-fe56cc0/job.log
>>   (01/27) /tmp/qemu-test/src/tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_x86_64_pc:  ERROR: join() argument must be str or bytes, not 'NoneType' (0.05 s)
>> Interrupting job (failfast).
>> RESULTS    : PASS 0 | ERROR 1 | FAIL 0 | SKIP 26 | WARN 0 | INTERRUPT 0 | CANCEL 0
>> JOB TIME   : 0.26 s
>> /tmp/qemu-test/src/tests/Makefile.include:1158: recipe for target 'check-acceptance' failed
>> make: *** [check-acceptance] Error 9
>>
>> That being said, I'm not running it under docker, but under podman,
>> although I'm not convinced yet that is the defining issue.  I'll try
>> to identify what's going here.
> 
> Was there a conclusion to the discussion about this series?

No sure if there was a conclusion...

tests/docker/ is listed as 'Maintained' but the listed maintainers only 
use Docker. If Podman is supported, then maybe we need another 
maintainer for Podman? We are not testing it so far.
diff mbox series

Patch

diff --git a/tests/docker/test-acceptance b/tests/docker/test-acceptance
new file mode 100755
index 0000000000..84edaa676c
--- /dev/null
+++ b/tests/docker/test-acceptance
@@ -0,0 +1,21 @@ 
+#!/bin/bash -e
+#
+# Compile default Travis-CI target and run Avocado acceptance tests
+#
+# Copyright (c) 2019 Red Hat Inc.
+#
+# Authors:
+#  Philippe Mathieu-Daudé <philmd@redhat.com>
+#
+# This work is licensed under the terms of the GNU GPL, version 2
+# or (at your option) any later version. See the COPYING file in
+# the top-level directory.
+
+. common.rc
+
+cd "$BUILD_DIR"
+
+DEF_TARGET_LIST="x86_64-softmmu,mips-softmmu,mips64el-softmmu,aarch64-softmmu,arm-softmmu,s390x-softmmu,alpha-softmmu"
+TARGET_LIST=${TARGET_LIST:-$DEF_TARGET_LIST} \
+build_qemu
+check_qemu check-acceptance