diff mbox series

[PATCH-for-5.2,v3] tests/acceptance: Disable Spartan-3A DSP 1800A test

Message ID 20201109091719.2449141-1-f4bug@amsat.org (mailing list archive)
State New, archived
Headers show
Series [PATCH-for-5.2,v3] tests/acceptance: Disable Spartan-3A DSP 1800A test | expand

Commit Message

Philippe Mathieu-Daudé Nov. 9, 2020, 9:17 a.m. UTC
This test is regularly failing on CI:

   (05/34) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_microblaze_s3adsp1800:
  Linux version 4.11.3 (thuth@thuth.remote.csb) (gcc version 6.4.0 (Buildroot 2018.05.2) ) #5 Tue Dec 11 11:56:23 CET 2018
  ...
  Freeing unused kernel memory: 1444K
  This architecture does not have kernel memory protection.
  [nothing happens here]
  Runner error occurred: Timeout reached (90.91 s)

This is a regression. Until someone figure out the problem,
disable the test to keep CI pipeline useful.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
v3: Simply disable with @skip
---
 tests/acceptance/boot_linux_console.py | 2 ++
 tests/acceptance/replay_kernel.py      | 2 ++
 2 files changed, 4 insertions(+)

Comments

Thomas Huth Nov. 10, 2020, 12:20 p.m. UTC | #1
On 09/11/2020 10.17, Philippe Mathieu-Daudé wrote:
> This test is regularly failing on CI:
> 
>    (05/34) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_microblaze_s3adsp1800:
>   Linux version 4.11.3 (thuth@thuth.remote.csb) (gcc version 6.4.0 (Buildroot 2018.05.2) ) #5 Tue Dec 11 11:56:23 CET 2018
>   ...
>   Freeing unused kernel memory: 1444K
>   This architecture does not have kernel memory protection.
>   [nothing happens here]
>   Runner error occurred: Timeout reached (90.91 s)
> 
> This is a regression. Until someone figure out the problem,
> disable the test to keep CI pipeline useful.
> 
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
> v3: Simply disable with @skip
> ---
>  tests/acceptance/boot_linux_console.py | 2 ++
>  tests/acceptance/replay_kernel.py      | 2 ++
>  2 files changed, 4 insertions(+)
> 
> diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py
> index 8f433a67f84..cc6ec0f8c15 100644
> --- a/tests/acceptance/boot_linux_console.py
> +++ b/tests/acceptance/boot_linux_console.py
> @@ -13,6 +13,7 @@
>  import gzip
>  import shutil
>  
> +from avocado import skip
>  from avocado import skipUnless
>  from avocado_qemu import Test
>  from avocado_qemu import exec_command_and_wait_for_pattern
> @@ -1025,6 +1026,7 @@ def test_m68k_mcf5208evb(self):
>          tar_hash = 'ac688fd00561a2b6ce1359f9ff6aa2b98c9a570c'
>          self.do_test_advcal_2018('07', tar_hash, 'sanity-clause.elf')
>  
> +    @skip("Test currently broken") # Console stuck as of 5.2-rc1
>      def test_microblaze_s3adsp1800(self):
>          """
>          :avocado: tags=arch:microblaze
> diff --git a/tests/acceptance/replay_kernel.py b/tests/acceptance/replay_kernel.py
> index 00c228382bd..772633b01da 100644
> --- a/tests/acceptance/replay_kernel.py
> +++ b/tests/acceptance/replay_kernel.py
> @@ -14,6 +14,7 @@
>  import logging
>  import time
>  
> +from avocado import skip
>  from avocado import skipIf
>  from avocado import skipUnless
>  from avocado_qemu import wait_for_console_pattern
> @@ -280,6 +281,7 @@ def test_m68k_mcf5208evb(self):
>          file_path = self.fetch_asset(tar_url, asset_hash=tar_hash)
>          self.do_test_advcal_2018(file_path, 'sanity-clause.elf')
>  
> +    @skip("Test currently broken") # Console stuck as of 5.2-rc1
>      def test_microblaze_s3adsp1800(self):
>          """
>          :avocado: tags=arch:microblaze
> 

Acked-by: Thomas Huth <thuth@redhat.com>

FWIW, I tried to bisect this issue today to see where it broke, but if I run
the test often enough, I even got failures with QEMU v5.0 already, so it's
likely not a new issue after all:

make check-venv
for ((i=0;i<500;i++)); do echo $i ; \
 AVOCADO_ALLOW_UNTRUSTED_CODE=1 tests/venv/bin/avocado run \
 tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_microblaze_s3adsp1800 \
 || break ; sleep 0.1 ; \
done
Alex Bennée Nov. 10, 2020, 5:21 p.m. UTC | #2
Philippe Mathieu-Daudé <f4bug@amsat.org> writes:

> This test is regularly failing on CI:
>
>    (05/34) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_microblaze_s3adsp1800:
>   Linux version 4.11.3 (thuth@thuth.remote.csb) (gcc version 6.4.0 (Buildroot 2018.05.2) ) #5 Tue Dec 11 11:56:23 CET 2018
>   ...
>   Freeing unused kernel memory: 1444K
>   This architecture does not have kernel memory protection.
>   [nothing happens here]
>   Runner error occurred: Timeout reached (90.91 s)
>
> This is a regression. Until someone figure out the problem,
> disable the test to keep CI pipeline useful.
>
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

Queued to for-5.2/various-fixes, thanks.

> ---
> v3: Simply disable with @skip
> ---
>  tests/acceptance/boot_linux_console.py | 2 ++
>  tests/acceptance/replay_kernel.py      | 2 ++
>  2 files changed, 4 insertions(+)
>
> diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py
> index 8f433a67f84..cc6ec0f8c15 100644
> --- a/tests/acceptance/boot_linux_console.py
> +++ b/tests/acceptance/boot_linux_console.py
> @@ -13,6 +13,7 @@
>  import gzip
>  import shutil
>  
> +from avocado import skip
>  from avocado import skipUnless
>  from avocado_qemu import Test
>  from avocado_qemu import exec_command_and_wait_for_pattern
> @@ -1025,6 +1026,7 @@ def test_m68k_mcf5208evb(self):
>          tar_hash = 'ac688fd00561a2b6ce1359f9ff6aa2b98c9a570c'
>          self.do_test_advcal_2018('07', tar_hash, 'sanity-clause.elf')
>  
> +    @skip("Test currently broken") # Console stuck as of 5.2-rc1
>      def test_microblaze_s3adsp1800(self):
>          """
>          :avocado: tags=arch:microblaze
> diff --git a/tests/acceptance/replay_kernel.py b/tests/acceptance/replay_kernel.py
> index 00c228382bd..772633b01da 100644
> --- a/tests/acceptance/replay_kernel.py
> +++ b/tests/acceptance/replay_kernel.py
> @@ -14,6 +14,7 @@
>  import logging
>  import time
>  
> +from avocado import skip
>  from avocado import skipIf
>  from avocado import skipUnless
>  from avocado_qemu import wait_for_console_pattern
> @@ -280,6 +281,7 @@ def test_m68k_mcf5208evb(self):
>          file_path = self.fetch_asset(tar_url, asset_hash=tar_hash)
>          self.do_test_advcal_2018(file_path, 'sanity-clause.elf')
>  
> +    @skip("Test currently broken") # Console stuck as of 5.2-rc1
>      def test_microblaze_s3adsp1800(self):
>          """
>          :avocado: tags=arch:microblaze
diff mbox series

Patch

diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py
index 8f433a67f84..cc6ec0f8c15 100644
--- a/tests/acceptance/boot_linux_console.py
+++ b/tests/acceptance/boot_linux_console.py
@@ -13,6 +13,7 @@ 
 import gzip
 import shutil
 
+from avocado import skip
 from avocado import skipUnless
 from avocado_qemu import Test
 from avocado_qemu import exec_command_and_wait_for_pattern
@@ -1025,6 +1026,7 @@  def test_m68k_mcf5208evb(self):
         tar_hash = 'ac688fd00561a2b6ce1359f9ff6aa2b98c9a570c'
         self.do_test_advcal_2018('07', tar_hash, 'sanity-clause.elf')
 
+    @skip("Test currently broken") # Console stuck as of 5.2-rc1
     def test_microblaze_s3adsp1800(self):
         """
         :avocado: tags=arch:microblaze
diff --git a/tests/acceptance/replay_kernel.py b/tests/acceptance/replay_kernel.py
index 00c228382bd..772633b01da 100644
--- a/tests/acceptance/replay_kernel.py
+++ b/tests/acceptance/replay_kernel.py
@@ -14,6 +14,7 @@ 
 import logging
 import time
 
+from avocado import skip
 from avocado import skipIf
 from avocado import skipUnless
 from avocado_qemu import wait_for_console_pattern
@@ -280,6 +281,7 @@  def test_m68k_mcf5208evb(self):
         file_path = self.fetch_asset(tar_url, asset_hash=tar_hash)
         self.do_test_advcal_2018(file_path, 'sanity-clause.elf')
 
+    @skip("Test currently broken") # Console stuck as of 5.2-rc1
     def test_microblaze_s3adsp1800(self):
         """
         :avocado: tags=arch:microblaze