diff mbox series

[PULL,5/5] tests/acceptance: Count multiple Tux logos displayed on framebuffer

Message ID 1582810056-22646-6-git-send-email-aleksandar.markovic@rt-rk.com (mailing list archive)
State New, archived
Headers show
Series [PULL,1/5] MAINTAINERS: Orphan MIPS KVM CPUs | expand

Commit Message

Aleksandar Markovic Feb. 27, 2020, 1:27 p.m. UTC
From: Philippe Mathieu-Daudé <f4bug@amsat.org>

Add a test that verifies that each core properly displays the Tux
logo on the framebuffer device.

We simply follow the OpenCV "Template Matching with Multiple Objects"
tutorial, replacing Lionel Messi by Tux:
https://docs.opencv.org/4.2.0/d4/dc6/tutorial_py_template_matching.html

When OpenCV and NumPy are installed, this test can be run using:

  $ avocado --show=app,framebuffer \
    run -t cpu:i6400 \
    tests/acceptance/machine_mips_malta.py
  JOB ID     : 54f3d8efd8674f289b8aa01a87f5d70c5814544c
  JOB LOG    : avocado/job-results/job-2020-02-01T20.52-54f3d8e/job.log
   (1/3) tests/acceptance/machine_mips_malta.py:MaltaMachineFramebuffer.test_mips_malta_i6400_framebuffer_logo_1core:
  framebuffer: found Tux at position (x, y) = (0, 0)
  PASS (3.37 s)
   (2/3) tests/acceptance/machine_mips_malta.py:MaltaMachineFramebuffer.test_mips_malta_i6400_framebuffer_logo_7cores:
  framebuffer: found Tux at position (x, y) = (0, 0)
  framebuffer: found Tux at position (x, y) = (88, 0)
  framebuffer: found Tux at position (x, y) = (176, 0)
  framebuffer: found Tux at position (x, y) = (264, 0)
  framebuffer: found Tux at position (x, y) = (352, 0)
  framebuffer: found Tux at position (x, y) = (440, 0)
  framebuffer: found Tux at position (x, y) = (528, 0)
  PASS (5.80 s)
   (3/3) tests/acceptance/machine_mips_malta.py:MaltaMachineFramebuffer.test_mips_malta_i6400_framebuffer_logo_8cores:
  framebuffer: found Tux at position (x, y) = (0, 0)
  framebuffer: found Tux at position (x, y) = (88, 0)
  framebuffer: found Tux at position (x, y) = (176, 0)
  framebuffer: found Tux at position (x, y) = (264, 0)
  framebuffer: found Tux at position (x, y) = (352, 0)
  framebuffer: found Tux at position (x, y) = (440, 0)
  framebuffer: found Tux at position (x, y) = (528, 0)
  framebuffer: found Tux at position (x, y) = (616, 0)
  PASS (6.67 s)
  RESULTS    : PASS 3 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
  JOB TIME   : 16.79 s

If the AVOCADO_CV2_SCREENDUMP_PNG_PATH environment variable is set, the
test will save the screenshot with matched squares to it.

Test inspired by the following post:
https://www.mips.com/blog/how-to-run-smp-linux-in-qemu-on-a-mips64-release-6-cpu/
Kernel built with the following Docker file:
https://github.com/philmd/qemu-testing-blob/blob/malta_i6400/mips/malta/mips64el/Dockerfile

Reviewed-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Signed-off-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200201204751.17810-1-f4bug@amsat.org>
---
 MAINTAINERS                            |   1 +
 tests/acceptance/machine_mips_malta.py | 118 +++++++++++++++++++++++++++++++++
 2 files changed, 119 insertions(+)
 create mode 100644 tests/acceptance/machine_mips_malta.py

Comments

Peter Maydell March 2, 2020, 8:26 p.m. UTC | #1
On Thu, 27 Feb 2020 at 13:28, Aleksandar Markovic
<aleksandar.markovic@rt-rk.com> wrote:
>
> From: Philippe Mathieu-Daudé <f4bug@amsat.org>
>
> Add a test that verifies that each core properly displays the Tux
> logo on the framebuffer device.
>
> We simply follow the OpenCV "Template Matching with Multiple Objects"
> tutorial, replacing Lionel Messi by Tux:
> https://docs.opencv.org/4.2.0/d4/dc6/tutorial_py_template_matching.html

Hi -- this doesn't seem to work on my system:

 (38/74) tests/acceptance/machine_arm_integratorcp.py:IntegratorMachine.test_integratorcp_console:
SKIP: untrusted code
 (39/74) tests/acceptance/machine_arm_integratorcp.py:IntegratorMachine.test_framebuffer_tux_logo:
SKIP: Python NumPy not installed
 (40/74) tests/acceptance/machine_arm_n8x0.py:N8x0Machine.test_n800:
SKIP: untrusted code
 (41/74) tests/acceptance/machine_arm_n8x0.py:N8x0Machine.test_n810:
SKIP: untrusted code
 (42/74) tests/acceptance/machine_m68k_nextcube.py:NextCubeMachine.test_bootrom_framebuffer_size:
PASS (3.47 s)
 (43/74) tests/acceptance/machine_m68k_nextcube.py:NextCubeMachine.test_bootrom_framebuffer_ocr_with_tesseract_v3:
SKIP: tesseract v3 OCR tool not available
 (44/74) tests/acceptance/machine_m68k_nextcube.py:NextCubeMachine.test_bootrom_framebuffer_ocr_with_tesseract_v4:
SKIP: tesseract v4 OCR tool not available
 (45/74) tests/acceptance/machine_mips_malta.py:MaltaMachineFramebuffer.test_mips_malta_i6400_framebuffer_logo_1core:
ERROR: name 'cv2' is not defined (5.43 s)

It looks like the @skipUnless directive on the class didn't
have any effect. Moving it to the individual test functions seems
to cause them to be skipped as intended (as the integratorcp
framebuffer test is):

diff --git a/tests/acceptance/machine_mips_malta.py
b/tests/acceptance/machine_mips_malta.py
index 92b4f28a112..98463f77573 100644
--- a/tests/acceptance/machine_mips_malta.py
+++ b/tests/acceptance/machine_mips_malta.py
@@ -30,8 +30,6 @@ except ImportError:
     CV2_AVAILABLE = False


-@skipUnless(NUMPY_AVAILABLE, 'Python NumPy not installed')
-@skipUnless(CV2_AVAILABLE, 'Python OpenCV not installed')
 class MaltaMachineFramebuffer(Test):

     timeout = 30
@@ -91,6 +89,8 @@ class MaltaMachineFramebuffer(Test):
             cv2.imwrite(debug_png, screendump_bgr)
         self.assertGreaterEqual(tuxlogo_count, cpu_cores_count)

+    @skipUnless(NUMPY_AVAILABLE, 'Python NumPy not installed')
+    @skipUnless(CV2_AVAILABLE, 'Python OpenCV not installed')
     def test_mips_malta_i6400_framebuffer_logo_1core(self):
         """
         :avocado: tags=arch:mips64el
@@ -99,6 +99,8 @@ class MaltaMachineFramebuffer(Test):
         """
         self.do_test_i6400_framebuffer_logo(1)

+    @skipUnless(NUMPY_AVAILABLE, 'Python NumPy not installed')
+    @skipUnless(CV2_AVAILABLE, 'Python OpenCV not installed')
     def test_mips_malta_i6400_framebuffer_logo_7cores(self):
         """
         :avocado: tags=arch:mips64el
@@ -108,6 +110,8 @@ class MaltaMachineFramebuffer(Test):
         """
         self.do_test_i6400_framebuffer_logo(7)

+    @skipUnless(NUMPY_AVAILABLE, 'Python NumPy not installed')
+    @skipUnless(CV2_AVAILABLE, 'Python OpenCV not installed')
     def test_mips_malta_i6400_framebuffer_logo_8cores(self):
         """
         :avocado: tags=arch:mips64el

thanks
-- PMM
Aleksandar Markovic March 2, 2020, 8:58 p.m. UTC | #2
> From: Peter Maydell <peter.maydell@linaro.org>
> Sent: Monday, March 2, 2020 9:26 PM
> To: Aleksandar Markovic
> Cc: QEMU Developers; Aleksandar Markovic; Philippe Mathieu-Daudé
> Subject: [EXTERNAL]Re: [PULL 5/5] tests/acceptance: Count multiple Tux logos displayed on framebuffer
> 
> On Thu, 27 Feb 2020 at 13:28, Aleksandar Markovic
> <aleksandar.markovic@rt-rk.com> wrote:
> >
> > From: Philippe Mathieu-Daudé <f4bug@amsat.org>
> >
> > Add a test that verifies that each core properly displays the Tux
> > logo on the framebuffer device.
> >
> > We simply follow the OpenCV "Template Matching with Multiple Objects"
> > tutorial, replacing Lionel Messi by Tux:
> > https://docs.opencv.org/4.2.0/d4/dc6/tutorial_py_template_matching.html
> 
> Hi -- this doesn't seem to work on my system:
> 

Thank you for bringing this to our attention.

It looks like an honest mistake to me. Too bad
none of us tested the case of environment like
yours, even though it looked reasonable to test it.

Philippe, Peter's solution looks good to me. What
do you think?

Peter, would you like me to submit a patch with your
proposal, authored by you, and than soon send it as
a part of a small pull request?

Thanks again,
Aleksandar

>  (38/74) tests/acceptance/machine_arm_integratorcp.py:IntegratorMachine.test_integratorcp_console:
> SKIP: untrusted code
>  (39/74) tests/acceptance/machine_arm_integratorcp.py:IntegratorMachine.test_framebuffer_tux_logo:
> SKIP: Python NumPy not installed
>  (40/74) tests/acceptance/machine_arm_n8x0.py:N8x0Machine.test_n800:
> SKIP: untrusted code
>  (41/74) tests/acceptance/machine_arm_n8x0.py:N8x0Machine.test_n810:
> SKIP: untrusted code
>  (42/74) tests/acceptance/machine_m68k_nextcube.py:NextCubeMachine.test_bootrom_framebuffer_size:
> PASS (3.47 s)
>  (43/74) tests/acceptance/machine_m68k_nextcube.py:NextCubeMachine.test_bootrom_framebuffer_ocr_with_tesseract_v3:
> SKIP: tesseract v3 OCR tool not available
>  (44/74) tests/acceptance/machine_m68k_nextcube.py:NextCubeMachine.test_bootrom_framebuffer_ocr_with_tesseract_v4:
> SKIP: tesseract v4 OCR tool not available
>  (45/74) tests/acceptance/machine_mips_malta.py:MaltaMachineFramebuffer.test_mips_malta_i6400_framebuffer_logo_1core:
> ERROR: name 'cv2' is not defined (5.43 s)
> 
> It looks like the @skipUnless directive on the class didn't
> have any effect. Moving it to the individual test functions seems
> to cause them to be skipped as intended (as the integratorcp
> framebuffer test is):
> 
> diff --git a/tests/acceptance/machine_mips_malta.py
> b/tests/acceptance/machine_mips_malta.py
> index 92b4f28a112..98463f77573 100644
> --- a/tests/acceptance/machine_mips_malta.py
> +++ b/tests/acceptance/machine_mips_malta.py
> @@ -30,8 +30,6 @@ except ImportError:
>      CV2_AVAILABLE = False
> 
> 
> -@skipUnless(NUMPY_AVAILABLE, 'Python NumPy not installed')
> -@skipUnless(CV2_AVAILABLE, 'Python OpenCV not installed')
>  class MaltaMachineFramebuffer(Test):
> 
>      timeout = 30
> @@ -91,6 +89,8 @@ class MaltaMachineFramebuffer(Test):
>              cv2.imwrite(debug_png, screendump_bgr)
>          self.assertGreaterEqual(tuxlogo_count, cpu_cores_count)
> 
> +    @skipUnless(NUMPY_AVAILABLE, 'Python NumPy not installed')
> +    @skipUnless(CV2_AVAILABLE, 'Python OpenCV not installed')
>      def test_mips_malta_i6400_framebuffer_logo_1core(self):
>          """
>          :avocado: tags=arch:mips64el
> @@ -99,6 +99,8 @@ class MaltaMachineFramebuffer(Test):
>          """
>          self.do_test_i6400_framebuffer_logo(1)
> 
> +    @skipUnless(NUMPY_AVAILABLE, 'Python NumPy not installed')
> +    @skipUnless(CV2_AVAILABLE, 'Python OpenCV not installed')
>      def test_mips_malta_i6400_framebuffer_logo_7cores(self):
>          """
>          :avocado: tags=arch:mips64el
> @@ -108,6 +110,8 @@ class MaltaMachineFramebuffer(Test):
>          """
>          self.do_test_i6400_framebuffer_logo(7)
> 
> +    @skipUnless(NUMPY_AVAILABLE, 'Python NumPy not installed')
> +    @skipUnless(CV2_AVAILABLE, 'Python OpenCV not installed')
>      def test_mips_malta_i6400_framebuffer_logo_8cores(self):
>          """
>          :avocado: tags=arch:mips64el
> 
> thanks
> -- PMM
Philippe Mathieu-Daudé March 2, 2020, 11:40 p.m. UTC | #3
On 3/2/20 9:58 PM, Aleksandar Markovic wrote:
>> From: Peter Maydell <peter.maydell@linaro.org>
>> Sent: Monday, March 2, 2020 9:26 PM
>> To: Aleksandar Markovic
>> Cc: QEMU Developers; Aleksandar Markovic; Philippe Mathieu-Daudé
>> Subject: [EXTERNAL]Re: [PULL 5/5] tests/acceptance: Count multiple Tux logos displayed on framebuffer
>>
>> On Thu, 27 Feb 2020 at 13:28, Aleksandar Markovic
>> <aleksandar.markovic@rt-rk.com> wrote:
>>>
>>> From: Philippe Mathieu-Daudé <f4bug@amsat.org>
>>>
>>> Add a test that verifies that each core properly displays the Tux
>>> logo on the framebuffer device.
>>>
>>> We simply follow the OpenCV "Template Matching with Multiple Objects"
>>> tutorial, replacing Lionel Messi by Tux:
>>> https://docs.opencv.org/4.2.0/d4/dc6/tutorial_py_template_matching.html
>>
>> Hi -- this doesn't seem to work on my system:
>>
> 
> Thank you for bringing this to our attention.
> 
> It looks like an honest mistake to me. Too bad
> none of us tested the case of environment like
> yours, even though it looked reasonable to test it.

I'm sorry I missed that, I'm pretty sure I tested it as we noticed this 
"bug" in Avocado more than 1 year ago already (bug = class decorator not 
working). I suppose I forgot to test that on Travis-CI and all my 
systems have opencv2 installed.

> 
> Philippe, Peter's solution looks good to me. What
> do you think?

Yes this is correct. Alex suggested it earlier and I resend his patch 
fixed which is exactly the diff Peter pasted (I am just noticing it now):
https://lists.gnu.org/archive/html/qemu-devel/2020-03/msg00389.html

> 
> Peter, would you like me to submit a patch with your
> proposal, authored by you, and than soon send it as
> a part of a small pull request?
> 
> Thanks again,
> Aleksandar
> 
>>   (38/74) tests/acceptance/machine_arm_integratorcp.py:IntegratorMachine.test_integratorcp_console:
>> SKIP: untrusted code
>>   (39/74) tests/acceptance/machine_arm_integratorcp.py:IntegratorMachine.test_framebuffer_tux_logo:
>> SKIP: Python NumPy not installed
>>   (40/74) tests/acceptance/machine_arm_n8x0.py:N8x0Machine.test_n800:
>> SKIP: untrusted code
>>   (41/74) tests/acceptance/machine_arm_n8x0.py:N8x0Machine.test_n810:
>> SKIP: untrusted code
>>   (42/74) tests/acceptance/machine_m68k_nextcube.py:NextCubeMachine.test_bootrom_framebuffer_size:
>> PASS (3.47 s)
>>   (43/74) tests/acceptance/machine_m68k_nextcube.py:NextCubeMachine.test_bootrom_framebuffer_ocr_with_tesseract_v3:
>> SKIP: tesseract v3 OCR tool not available
>>   (44/74) tests/acceptance/machine_m68k_nextcube.py:NextCubeMachine.test_bootrom_framebuffer_ocr_with_tesseract_v4:
>> SKIP: tesseract v4 OCR tool not available
>>   (45/74) tests/acceptance/machine_mips_malta.py:MaltaMachineFramebuffer.test_mips_malta_i6400_framebuffer_logo_1core:
>> ERROR: name 'cv2' is not defined (5.43 s)
>>
>> It looks like the @skipUnless directive on the class didn't
>> have any effect. Moving it to the individual test functions seems
>> to cause them to be skipped as intended (as the integratorcp
>> framebuffer test is):
>>
>> diff --git a/tests/acceptance/machine_mips_malta.py
>> b/tests/acceptance/machine_mips_malta.py
>> index 92b4f28a112..98463f77573 100644
>> --- a/tests/acceptance/machine_mips_malta.py
>> +++ b/tests/acceptance/machine_mips_malta.py
>> @@ -30,8 +30,6 @@ except ImportError:
>>       CV2_AVAILABLE = False
>>
>>
>> -@skipUnless(NUMPY_AVAILABLE, 'Python NumPy not installed')
>> -@skipUnless(CV2_AVAILABLE, 'Python OpenCV not installed')
>>   class MaltaMachineFramebuffer(Test):
>>
>>       timeout = 30
>> @@ -91,6 +89,8 @@ class MaltaMachineFramebuffer(Test):
>>               cv2.imwrite(debug_png, screendump_bgr)
>>           self.assertGreaterEqual(tuxlogo_count, cpu_cores_count)
>>
>> +    @skipUnless(NUMPY_AVAILABLE, 'Python NumPy not installed')
>> +    @skipUnless(CV2_AVAILABLE, 'Python OpenCV not installed')
>>       def test_mips_malta_i6400_framebuffer_logo_1core(self):
>>           """
>>           :avocado: tags=arch:mips64el
>> @@ -99,6 +99,8 @@ class MaltaMachineFramebuffer(Test):
>>           """
>>           self.do_test_i6400_framebuffer_logo(1)
>>
>> +    @skipUnless(NUMPY_AVAILABLE, 'Python NumPy not installed')
>> +    @skipUnless(CV2_AVAILABLE, 'Python OpenCV not installed')
>>       def test_mips_malta_i6400_framebuffer_logo_7cores(self):
>>           """
>>           :avocado: tags=arch:mips64el
>> @@ -108,6 +110,8 @@ class MaltaMachineFramebuffer(Test):
>>           """
>>           self.do_test_i6400_framebuffer_logo(7)
>>
>> +    @skipUnless(NUMPY_AVAILABLE, 'Python NumPy not installed')
>> +    @skipUnless(CV2_AVAILABLE, 'Python OpenCV not installed')
>>       def test_mips_malta_i6400_framebuffer_logo_8cores(self):
>>           """
>>           :avocado: tags=arch:mips64el
>>
>> thanks
>> -- PMM
> 
>
Philippe Mathieu-Daudé March 3, 2020, 12:55 p.m. UTC | #4
Hi Cleber,

 On Tue, Mar 3, 2020 at 12:40 AM Philippe Mathieu-Daudé
<philmd@redhat.com> wrote:
> On 3/2/20 9:58 PM, Aleksandar Markovic wrote:
> >> From: Peter Maydell <peter.maydell@linaro.org>
> >> Sent: Monday, March 2, 2020 9:26 PM
> >> To: Aleksandar Markovic
> >> Cc: QEMU Developers; Aleksandar Markovic; Philippe Mathieu-Daudé
> >> Subject: [EXTERNAL]Re: [PULL 5/5] tests/acceptance: Count multiple Tux logos displayed on framebuffer
> >>
> >> On Thu, 27 Feb 2020 at 13:28, Aleksandar Markovic
> >> <aleksandar.markovic@rt-rk.com> wrote:
> >>>
> >>> From: Philippe Mathieu-Daudé <f4bug@amsat.org>
> >>>
> >>> Add a test that verifies that each core properly displays the Tux
> >>> logo on the framebuffer device.
> >>>
> >>> We simply follow the OpenCV "Template Matching with Multiple Objects"
> >>> tutorial, replacing Lionel Messi by Tux:
> >>> https://docs.opencv.org/4.2.0/d4/dc6/tutorial_py_template_matching.html
> >>
> >> Hi -- this doesn't seem to work on my system:
> >>
> >
> > Thank you for bringing this to our attention.
> >
> > It looks like an honest mistake to me. Too bad
> > none of us tested the case of environment like
> > yours, even though it looked reasonable to test it.
>
> I'm sorry I missed that, I'm pretty sure I tested it as we noticed this
> "bug" in Avocado more than 1 year ago already (bug = class decorator not
> working).

I see in commit 2fe6f4d9ba you moved the @skip decorators to the setUp() method.
Is this a kind of kludge to not use it on all the test_*() methods?
(since we can not use it on an AvocadoTest class)

> I suppose I forgot to test that on Travis-CI and all my
> systems have opencv2 installed.
>
> >
> > Philippe, Peter's solution looks good to me. What
> > do you think?
>
> Yes this is correct. Alex suggested it earlier and I resend his patch
> fixed which is exactly the diff Peter pasted (I am just noticing it now):
> https://lists.gnu.org/archive/html/qemu-devel/2020-03/msg00389.html
>
> >
> > Peter, would you like me to submit a patch with your
> > proposal, authored by you, and than soon send it as
> > a part of a small pull request?
> >
> > Thanks again,
> > Aleksandar
> >
> >>   (38/74) tests/acceptance/machine_arm_integratorcp.py:IntegratorMachine.test_integratorcp_console:
> >> SKIP: untrusted code
> >>   (39/74) tests/acceptance/machine_arm_integratorcp.py:IntegratorMachine.test_framebuffer_tux_logo:
> >> SKIP: Python NumPy not installed
> >>   (40/74) tests/acceptance/machine_arm_n8x0.py:N8x0Machine.test_n800:
> >> SKIP: untrusted code
> >>   (41/74) tests/acceptance/machine_arm_n8x0.py:N8x0Machine.test_n810:
> >> SKIP: untrusted code
> >>   (42/74) tests/acceptance/machine_m68k_nextcube.py:NextCubeMachine.test_bootrom_framebuffer_size:
> >> PASS (3.47 s)
> >>   (43/74) tests/acceptance/machine_m68k_nextcube.py:NextCubeMachine.test_bootrom_framebuffer_ocr_with_tesseract_v3:
> >> SKIP: tesseract v3 OCR tool not available
> >>   (44/74) tests/acceptance/machine_m68k_nextcube.py:NextCubeMachine.test_bootrom_framebuffer_ocr_with_tesseract_v4:
> >> SKIP: tesseract v4 OCR tool not available
> >>   (45/74) tests/acceptance/machine_mips_malta.py:MaltaMachineFramebuffer.test_mips_malta_i6400_framebuffer_logo_1core:
> >> ERROR: name 'cv2' is not defined (5.43 s)
> >>
> >> It looks like the @skipUnless directive on the class didn't
> >> have any effect. Moving it to the individual test functions seems
> >> to cause them to be skipped as intended (as the integratorcp
> >> framebuffer test is):
> >>
> >> diff --git a/tests/acceptance/machine_mips_malta.py
> >> b/tests/acceptance/machine_mips_malta.py
> >> index 92b4f28a112..98463f77573 100644
> >> --- a/tests/acceptance/machine_mips_malta.py
> >> +++ b/tests/acceptance/machine_mips_malta.py
> >> @@ -30,8 +30,6 @@ except ImportError:
> >>       CV2_AVAILABLE = False
> >>
> >>
> >> -@skipUnless(NUMPY_AVAILABLE, 'Python NumPy not installed')
> >> -@skipUnless(CV2_AVAILABLE, 'Python OpenCV not installed')
> >>   class MaltaMachineFramebuffer(Test):
> >>
> >>       timeout = 30
> >> @@ -91,6 +89,8 @@ class MaltaMachineFramebuffer(Test):
> >>               cv2.imwrite(debug_png, screendump_bgr)
> >>           self.assertGreaterEqual(tuxlogo_count, cpu_cores_count)
> >>
> >> +    @skipUnless(NUMPY_AVAILABLE, 'Python NumPy not installed')
> >> +    @skipUnless(CV2_AVAILABLE, 'Python OpenCV not installed')
> >>       def test_mips_malta_i6400_framebuffer_logo_1core(self):
> >>           """
> >>           :avocado: tags=arch:mips64el
> >> @@ -99,6 +99,8 @@ class MaltaMachineFramebuffer(Test):
> >>           """
> >>           self.do_test_i6400_framebuffer_logo(1)
> >>
> >> +    @skipUnless(NUMPY_AVAILABLE, 'Python NumPy not installed')
> >> +    @skipUnless(CV2_AVAILABLE, 'Python OpenCV not installed')
> >>       def test_mips_malta_i6400_framebuffer_logo_7cores(self):
> >>           """
> >>           :avocado: tags=arch:mips64el
> >> @@ -108,6 +110,8 @@ class MaltaMachineFramebuffer(Test):
> >>           """
> >>           self.do_test_i6400_framebuffer_logo(7)
> >>
> >> +    @skipUnless(NUMPY_AVAILABLE, 'Python NumPy not installed')
> >> +    @skipUnless(CV2_AVAILABLE, 'Python OpenCV not installed')
> >>       def test_mips_malta_i6400_framebuffer_logo_8cores(self):
> >>           """
> >>           :avocado: tags=arch:mips64el
> >>
> >> thanks
> >> -- PMM
> >
> >
Cleber Rosa March 3, 2020, 2:29 p.m. UTC | #5
On Tue, Mar 03, 2020 at 01:55:35PM +0100, Philippe Mathieu-Daudé wrote:
> Hi Cleber,
>

Hi Phil!

>  On Tue, Mar 3, 2020 at 12:40 AM Philippe Mathieu-Daudé
> <philmd@redhat.com> wrote:
> > On 3/2/20 9:58 PM, Aleksandar Markovic wrote:
> > >> From: Peter Maydell <peter.maydell@linaro.org>
> > >> Sent: Monday, March 2, 2020 9:26 PM
> > >> To: Aleksandar Markovic
> > >> Cc: QEMU Developers; Aleksandar Markovic; Philippe Mathieu-Daudé
> > >> Subject: [EXTERNAL]Re: [PULL 5/5] tests/acceptance: Count multiple Tux logos displayed on framebuffer
> > >>
> > >> On Thu, 27 Feb 2020 at 13:28, Aleksandar Markovic
> > >> <aleksandar.markovic@rt-rk.com> wrote:
> > >>>
> > >>> From: Philippe Mathieu-Daudé <f4bug@amsat.org>
> > >>>
> > >>> Add a test that verifies that each core properly displays the Tux
> > >>> logo on the framebuffer device.
> > >>>
> > >>> We simply follow the OpenCV "Template Matching with Multiple Objects"
> > >>> tutorial, replacing Lionel Messi by Tux:
> > >>> https://docs.opencv.org/4.2.0/d4/dc6/tutorial_py_template_matching.html
> > >>
> > >> Hi -- this doesn't seem to work on my system:
> > >>
> > >
> > > Thank you for bringing this to our attention.
> > >
> > > It looks like an honest mistake to me. Too bad
> > > none of us tested the case of environment like
> > > yours, even though it looked reasonable to test it.
> >
> > I'm sorry I missed that, I'm pretty sure I tested it as we noticed this
> > "bug" in Avocado more than 1 year ago already (bug = class decorator not
> > working).
> 
> I see in commit 2fe6f4d9ba you moved the @skip decorators to the setUp() method.
> Is this a kind of kludge to not use it on all the test_*() methods?
> (since we can not use it on an AvocadoTest class)
>

Yes, that was a workaround for a previous Avocado limitation.  As I said
on another thread, on Avocado >= 76.0 it's possible to just decorate the
entire class.

- Cleber.
diff mbox series

Patch

diff --git a/MAINTAINERS b/MAINTAINERS
index b084201..880da9c 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1005,6 +1005,7 @@  F: hw/mips/mips_malta.c
 F: hw/mips/gt64xxx_pci.c
 F: include/hw/southbridge/piix.h
 F: tests/acceptance/linux_ssh_mips_malta.py
+F: tests/acceptance/machine_mips_malta.py
 
 Mipssim
 M: Aleksandar Markovic <amarkovic@wavecomp.com>
diff --git a/tests/acceptance/machine_mips_malta.py b/tests/acceptance/machine_mips_malta.py
new file mode 100644
index 0000000..92b4f28
--- /dev/null
+++ b/tests/acceptance/machine_mips_malta.py
@@ -0,0 +1,118 @@ 
+# Functional tests for the MIPS Malta board
+#
+# Copyright (c) Philippe Mathieu-Daudé <f4bug@amsat.org>
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or later.
+# See the COPYING file in the top-level directory.
+#
+# SPDX-License-Identifier: GPL-2.0-or-later
+
+import os
+import gzip
+import logging
+
+from avocado import skipUnless
+from avocado_qemu import Test
+from avocado_qemu import wait_for_console_pattern
+from avocado.utils import archive
+
+
+NUMPY_AVAILABLE = True
+try:
+    import numpy as np
+except ImportError:
+    NUMPY_AVAILABLE = False
+
+CV2_AVAILABLE = True
+try:
+    import cv2
+except ImportError:
+    CV2_AVAILABLE = False
+
+
+@skipUnless(NUMPY_AVAILABLE, 'Python NumPy not installed')
+@skipUnless(CV2_AVAILABLE, 'Python OpenCV not installed')
+class MaltaMachineFramebuffer(Test):
+
+    timeout = 30
+
+    KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 '
+
+    def do_test_i6400_framebuffer_logo(self, cpu_cores_count):
+        """
+        Boot Linux kernel and check Tux logo is displayed on the framebuffer.
+        """
+        screendump_path = os.path.join(self.workdir, 'screendump.pbm')
+
+        kernel_url = ('https://github.com/philmd/qemu-testing-blob/raw/'
+                      'a5966ca4b5/mips/malta/mips64el/'
+                      'vmlinux-4.7.0-rc1.I6400.gz')
+        kernel_hash = '096f50c377ec5072e6a366943324622c312045f6'
+        kernel_path_gz = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
+        kernel_path = self.workdir + "vmlinux"
+        archive.gzip_uncompress(kernel_path_gz, kernel_path)
+
+        tuxlogo_url = ('https://github.com/torvalds/linux/raw/v2.6.12/'
+                       'drivers/video/logo/logo_linux_vga16.ppm')
+        tuxlogo_hash = '3991c2ddbd1ddaecda7601f8aafbcf5b02dc86af'
+        tuxlogo_path = self.fetch_asset(tuxlogo_url, asset_hash=tuxlogo_hash)
+
+        self.vm.set_console()
+        kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
+                               'clocksource=GIC console=tty0 console=ttyS0')
+        self.vm.add_args('-kernel', kernel_path,
+                         '-cpu', 'I6400',
+                         '-smp', '%u' % cpu_cores_count,
+                         '-vga', 'std',
+                         '-append', kernel_command_line)
+        self.vm.launch()
+        framebuffer_ready = 'Console: switching to colour frame buffer device'
+        wait_for_console_pattern(self, framebuffer_ready,
+                                 failure_message='Kernel panic - not syncing')
+        self.vm.command('human-monitor-command', command_line='stop')
+        self.vm.command('human-monitor-command',
+                        command_line='screendump %s' % screendump_path)
+        logger = logging.getLogger('framebuffer')
+
+        match_threshold = 0.95
+        screendump_bgr = cv2.imread(screendump_path, cv2.IMREAD_COLOR)
+        tuxlogo_bgr = cv2.imread(tuxlogo_path, cv2.IMREAD_COLOR)
+        result = cv2.matchTemplate(screendump_bgr, tuxlogo_bgr,
+                                   cv2.TM_CCOEFF_NORMED)
+        loc = np.where(result >= match_threshold)
+        tuxlogo_count = 0
+        h, w = tuxlogo_bgr.shape[:2]
+        debug_png = os.getenv('AVOCADO_CV2_SCREENDUMP_PNG_PATH')
+        for tuxlogo_count, pt in enumerate(zip(*loc[::-1]), start=1):
+            logger.debug('found Tux at position (x, y) = %s', pt)
+            cv2.rectangle(screendump_bgr, pt,
+                          (pt[0] + w, pt[1] + h), (0, 0, 255), 2)
+        if debug_png:
+            cv2.imwrite(debug_png, screendump_bgr)
+        self.assertGreaterEqual(tuxlogo_count, cpu_cores_count)
+
+    def test_mips_malta_i6400_framebuffer_logo_1core(self):
+        """
+        :avocado: tags=arch:mips64el
+        :avocado: tags=machine:malta
+        :avocado: tags=cpu:i6400
+        """
+        self.do_test_i6400_framebuffer_logo(1)
+
+    def test_mips_malta_i6400_framebuffer_logo_7cores(self):
+        """
+        :avocado: tags=arch:mips64el
+        :avocado: tags=machine:malta
+        :avocado: tags=cpu:i6400
+        :avocado: tags=mips:smp
+        """
+        self.do_test_i6400_framebuffer_logo(7)
+
+    def test_mips_malta_i6400_framebuffer_logo_8cores(self):
+        """
+        :avocado: tags=arch:mips64el
+        :avocado: tags=machine:malta
+        :avocado: tags=cpu:i6400
+        :avocado: tags=mips:smp
+        """
+        self.do_test_i6400_framebuffer_logo(8)