diff mbox series

[v2,3/3] Travis support for the acceptance tests

Message ID 20181009041826.19462-4-crosa@redhat.com (mailing list archive)
State New, archived
Headers show
Series Bootstrap Python venv and acceptance/functional tests | expand

Commit Message

Cleber Rosa Oct. 9, 2018, 4:18 a.m. UTC
This enables the execution of the acceptance tests on Travis.

Because the Travis environment is based on Ubuntu Trusty, it requires
the python3-pip.

Note: while another supposedely required component on newer versions
(such as on Bionic) split the Python 3 installation further on the
python3-venv package.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
---
 .travis.yml | 6 ++++++
 1 file changed, 6 insertions(+)

Comments

Philippe Mathieu-Daudé Oct. 9, 2018, 1:46 p.m. UTC | #1
Hi Cleber,

On 09/10/2018 06:18, Cleber Rosa wrote:
> This enables the execution of the acceptance tests on Travis.

Did you test this? =)

> 
> Because the Travis environment is based on Ubuntu Trusty, it requires
> the python3-pip.
> 
> Note: while another supposedely required component on newer versions
> (such as on Bionic) split the Python 3 installation further on the
> python3-venv package.
> 
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
> ---
>  .travis.yml | 6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/.travis.yml b/.travis.yml
> index 95be6ec59f..db1a31ea51 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -36,6 +36,7 @@ addons:
>        - liburcu-dev
>        - libusb-1.0-0-dev
>        - libvte-2.90-dev
> +      - python3-pip
>        - sparse
>        - uuid-dev
>        - gcovr
> @@ -117,6 +118,11 @@ matrix:
>      - env: CONFIG="--target-list=x86_64-softmmu"
>        python:
>          - "3.6"
> +    # Acceptance (Functional) tests
> +    - env: CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu"
> +           TEST_CMD="make check-acceptance"
> +      python:
> +        - "3.6"
>      # Using newer GCC with sanitizers
>      - addons:
>          apt:
> 

Using the following patch:

-- >8 --
diff --git a/.travis.yml b/.travis.yml
index 95be6ec59f..87e0c9a13f 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -118,4 +118,15 @@ matrix:
       python:
         - "3.6"
+    # Acceptance (Functional) tests
+    - env: CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu"
+           TEST_CMD="make check-acceptance"
+           # sudo rm /usr/local/bin/pip*
+      python:
+        - "3.6"
+      addons:
+        apt:
+          packages:
+            - python3-pip
+            - python3.4-venv
     # Using newer GCC with sanitizers
     - addons:
---

I got some improvements until:

  VENV    /home/travis/build/philmd/qemu/tests/venv
  MKDIR   /home/travis/build/philmd/qemu/tests/results
  PIP     /home/travis/build/philmd/qemu/tests/venv-requirements.txt
Exception:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pip/basecommand.py", line 122, in
main
    status = self.run(options, args)
  File "/usr/lib/python3/dist-packages/pip/commands/install.py", line
283, in run
    requirement_set.install(install_options, global_options,
root=options.root_path)
  File "/usr/lib/python3/dist-packages/pip/req.py", line 1436, in install
    requirement.install(install_options, global_options, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/pip/req.py", line 672, in install
    self.move_wheel_files(self.source_dir, root=root)
  File "/usr/lib/python3/dist-packages/pip/req.py", line 902, in
move_wheel_files
    pycompile=self.pycompile,
  File "/usr/lib/python3/dist-packages/pip/wheel.py", line 206, in
move_wheel_files
    clobber(source, lib_dir, True)
  File "/usr/lib/python3/dist-packages/pip/wheel.py", line 193, in clobber
    os.makedirs(destsubdir)
  File "/usr/lib/python3.4/os.py", line 237, in makedirs
    mkdir(name, mode)
PermissionError: [Errno 13] Permission denied:
'/usr/local/lib/python3.4/dist-packages/avocado'

See: https://travis-ci.org/philmd/qemu/jobs/439138706
Alex Bennée Oct. 9, 2018, 2:15 p.m. UTC | #2
Philippe Mathieu-Daudé <philmd@redhat.com> writes:

> Hi Cleber,
>
> On 09/10/2018 06:18, Cleber Rosa wrote:
>> This enables the execution of the acceptance tests on Travis.
>
> Did you test this? =)
>
>>
>> Because the Travis environment is based on Ubuntu Trusty, it requires
>> the python3-pip.
>>
>> Note: while another supposedely required component on newer versions
>> (such as on Bionic) split the Python 3 installation further on the
>> python3-venv package.
>>
>> Signed-off-by: Cleber Rosa <crosa@redhat.com>
>> ---
>>  .travis.yml | 6 ++++++
>>  1 file changed, 6 insertions(+)
>>
>> diff --git a/.travis.yml b/.travis.yml
>> index 95be6ec59f..db1a31ea51 100644
>> --- a/.travis.yml
>> +++ b/.travis.yml
>> @@ -36,6 +36,7 @@ addons:
>>        - liburcu-dev
>>        - libusb-1.0-0-dev
>>        - libvte-2.90-dev
>> +      - python3-pip
>>        - sparse
>>        - uuid-dev
>>        - gcovr
>> @@ -117,6 +118,11 @@ matrix:
>>      - env: CONFIG="--target-list=x86_64-softmmu"
>>        python:
>>          - "3.6"
>> +    # Acceptance (Functional) tests
>> +    - env: CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu"
>> +           TEST_CMD="make check-acceptance"
>> +      python:
>> +        - "3.6"
>>      # Using newer GCC with sanitizers
>>      - addons:
>>          apt:
>>
>
> Using the following patch:
>
> -- >8 --
> diff --git a/.travis.yml b/.travis.yml
> index 95be6ec59f..87e0c9a13f 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -118,4 +118,15 @@ matrix:
>        python:
>          - "3.6"
> +    # Acceptance (Functional) tests
> +    - env: CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu"
> +           TEST_CMD="make check-acceptance"
> +           # sudo rm /usr/local/bin/pip*

That snuck in ^

> +      python:
> +        - "3.6"
> +      addons:
> +        apt:
> +          packages:
> +            - python3-pip
> +            - python3.4-venv
>      # Using newer GCC with sanitizers
>      - addons:
> ---
>
> I got some improvements until:
>
>   VENV    /home/travis/build/philmd/qemu/tests/venv
>   MKDIR   /home/travis/build/philmd/qemu/tests/results
>   PIP     /home/travis/build/philmd/qemu/tests/venv-requirements.txt
> Exception:
> Traceback (most recent call last):
>   File "/usr/lib/python3/dist-packages/pip/basecommand.py", line 122, in
> main
>     status = self.run(options, args)
>   File "/usr/lib/python3/dist-packages/pip/commands/install.py", line
> 283, in run
>     requirement_set.install(install_options, global_options,
> root=options.root_path)
>   File "/usr/lib/python3/dist-packages/pip/req.py", line 1436, in install
>     requirement.install(install_options, global_options, *args, **kwargs)
>   File "/usr/lib/python3/dist-packages/pip/req.py", line 672, in install
>     self.move_wheel_files(self.source_dir, root=root)
>   File "/usr/lib/python3/dist-packages/pip/req.py", line 902, in
> move_wheel_files
>     pycompile=self.pycompile,
>   File "/usr/lib/python3/dist-packages/pip/wheel.py", line 206, in
> move_wheel_files
>     clobber(source, lib_dir, True)
>   File "/usr/lib/python3/dist-packages/pip/wheel.py", line 193, in clobber
>     os.makedirs(destsubdir)
>   File "/usr/lib/python3.4/os.py", line 237, in makedirs
>     mkdir(name, mode)
> PermissionError: [Errno 13] Permission denied:
> '/usr/local/lib/python3.4/dist-packages/avocado'
>
> See: https://travis-ci.org/philmd/qemu/jobs/439138706

Don't we need to do a pip install or is avocado already included?

--
Alex Bennée
Philippe Mathieu-Daudé Oct. 9, 2018, 2:23 p.m. UTC | #3
On 09/10/2018 16:15, Alex Bennée wrote:
> 
> Philippe Mathieu-Daudé <philmd@redhat.com> writes:
> 
>> Hi Cleber,
>>
>> On 09/10/2018 06:18, Cleber Rosa wrote:
>>> This enables the execution of the acceptance tests on Travis.
>>
>> Did you test this? =)
>>
>>>
>>> Because the Travis environment is based on Ubuntu Trusty, it requires
>>> the python3-pip.
>>>
>>> Note: while another supposedely required component on newer versions
>>> (such as on Bionic) split the Python 3 installation further on the
>>> python3-venv package.
>>>
>>> Signed-off-by: Cleber Rosa <crosa@redhat.com>
>>> ---
>>>  .travis.yml | 6 ++++++
>>>  1 file changed, 6 insertions(+)
>>>
>>> diff --git a/.travis.yml b/.travis.yml
>>> index 95be6ec59f..db1a31ea51 100644
>>> --- a/.travis.yml
>>> +++ b/.travis.yml
>>> @@ -36,6 +36,7 @@ addons:
>>>        - liburcu-dev
>>>        - libusb-1.0-0-dev
>>>        - libvte-2.90-dev
>>> +      - python3-pip
>>>        - sparse
>>>        - uuid-dev
>>>        - gcovr
>>> @@ -117,6 +118,11 @@ matrix:
>>>      - env: CONFIG="--target-list=x86_64-softmmu"
>>>        python:
>>>          - "3.6"
>>> +    # Acceptance (Functional) tests
>>> +    - env: CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu"
>>> +           TEST_CMD="make check-acceptance"
>>> +      python:
>>> +        - "3.6"
>>>      # Using newer GCC with sanitizers
>>>      - addons:
>>>          apt:
>>>
>>
>> Using the following patch:
>>
>> -- >8 --
>> diff --git a/.travis.yml b/.travis.yml
>> index 95be6ec59f..87e0c9a13f 100644
>> --- a/.travis.yml
>> +++ b/.travis.yml
>> @@ -118,4 +118,15 @@ matrix:
>>        python:
>>          - "3.6"
>> +    # Acceptance (Functional) tests
>> +    - env: CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu"
>> +           TEST_CMD="make check-acceptance"
>> +           # sudo rm /usr/local/bin/pip*
> 
> That snuck in ^

=)

Travis-ci image comes with this version which is Python2.

>> +      python:
>> +        - "3.6"
>> +      addons:
>> +        apt:
>> +          packages:
>> +            - python3-pip
>> +            - python3.4-venv
>>      # Using newer GCC with sanitizers
>>      - addons:
>> ---
>>
>> I got some improvements until:
>>
>>   VENV    /home/travis/build/philmd/qemu/tests/venv
>>   MKDIR   /home/travis/build/philmd/qemu/tests/results
>>   PIP     /home/travis/build/philmd/qemu/tests/venv-requirements.txt
>> Exception:
>> Traceback (most recent call last):
>>   File "/usr/lib/python3/dist-packages/pip/basecommand.py", line 122, in
>> main
>>     status = self.run(options, args)
>>   File "/usr/lib/python3/dist-packages/pip/commands/install.py", line
>> 283, in run
>>     requirement_set.install(install_options, global_options,
>> root=options.root_path)
>>   File "/usr/lib/python3/dist-packages/pip/req.py", line 1436, in install
>>     requirement.install(install_options, global_options, *args, **kwargs)
>>   File "/usr/lib/python3/dist-packages/pip/req.py", line 672, in install
>>     self.move_wheel_files(self.source_dir, root=root)
>>   File "/usr/lib/python3/dist-packages/pip/req.py", line 902, in
>> move_wheel_files
>>     pycompile=self.pycompile,
>>   File "/usr/lib/python3/dist-packages/pip/wheel.py", line 206, in
>> move_wheel_files
>>     clobber(source, lib_dir, True)
>>   File "/usr/lib/python3/dist-packages/pip/wheel.py", line 193, in clobber
>>     os.makedirs(destsubdir)
>>   File "/usr/lib/python3.4/os.py", line 237, in makedirs
>>     mkdir(name, mode)
>> PermissionError: [Errno 13] Permission denied:
>> '/usr/local/lib/python3.4/dist-packages/avocado'
>>
>> See: https://travis-ci.org/philmd/qemu/jobs/439138706
> 
> Don't we need to do a pip install or is avocado already included?

Yes I think we do. This is why I asked Cleber on patch #1 of this series
why he choose to use "venv ... --system-site-packages".
Cleber Rosa Oct. 9, 2018, 4:48 p.m. UTC | #4
On 10/9/18 9:46 AM, Philippe Mathieu-Daudé wrote:
> Hi Cleber,
> 
> On 09/10/2018 06:18, Cleber Rosa wrote:
>> This enables the execution of the acceptance tests on Travis.
> 
> Did you test this? =)
> 

I did have some jobs on Travis that looked promising.  But yeah, later I
found some issues. :/

>>
>> Because the Travis environment is based on Ubuntu Trusty, it requires
>> the python3-pip.
>>
>> Note: while another supposedely required component on newer versions
>> (such as on Bionic) split the Python 3 installation further on the
>> python3-venv package.
>>
>> Signed-off-by: Cleber Rosa <crosa@redhat.com>
>> ---
>>  .travis.yml | 6 ++++++
>>  1 file changed, 6 insertions(+)
>>
>> diff --git a/.travis.yml b/.travis.yml
>> index 95be6ec59f..db1a31ea51 100644
>> --- a/.travis.yml
>> +++ b/.travis.yml
>> @@ -36,6 +36,7 @@ addons:
>>        - liburcu-dev
>>        - libusb-1.0-0-dev
>>        - libvte-2.90-dev
>> +      - python3-pip
>>        - sparse
>>        - uuid-dev
>>        - gcovr
>> @@ -117,6 +118,11 @@ matrix:
>>      - env: CONFIG="--target-list=x86_64-softmmu"
>>        python:
>>          - "3.6"
>> +    # Acceptance (Functional) tests
>> +    - env: CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu"
>> +           TEST_CMD="make check-acceptance"
>> +      python:
>> +        - "3.6"
>>      # Using newer GCC with sanitizers
>>      - addons:
>>          apt:
>>
> 
> Using the following patch:
> 
> -- >8 --
> diff --git a/.travis.yml b/.travis.yml
> index 95be6ec59f..87e0c9a13f 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -118,4 +118,15 @@ matrix:
>        python:
>          - "3.6"
> +    # Acceptance (Functional) tests
> +    - env: CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu"
> +           TEST_CMD="make check-acceptance"
> +           # sudo rm /usr/local/bin/pip*
> +      python:
> +        - "3.6"
> +      addons:
> +        apt:
> +          packages:
> +            - python3-pip
> +            - python3.4-venv

This patch revealed to me that, even though we're asking Travis for a
Python 3.6 stack, we're manually pointing to the primary Python (3.4)
installation.  This is in itself a source of problems.

Read on...

>      # Using newer GCC with sanitizers
>      - addons:
> ---
> 
> I got some improvements until:
> 
>   VENV    /home/travis/build/philmd/qemu/tests/venv
>   MKDIR   /home/travis/build/philmd/qemu/tests/results
>   PIP     /home/travis/build/philmd/qemu/tests/venv-requirements.txt
> Exception:
> Traceback (most recent call last):
>   File "/usr/lib/python3/dist-packages/pip/basecommand.py", line 122, in
> main
>     status = self.run(options, args)
>   File "/usr/lib/python3/dist-packages/pip/commands/install.py", line
> 283, in run
>     requirement_set.install(install_options, global_options,
> root=options.root_path)
>   File "/usr/lib/python3/dist-packages/pip/req.py", line 1436, in install
>     requirement.install(install_options, global_options, *args, **kwargs)
>   File "/usr/lib/python3/dist-packages/pip/req.py", line 672, in install
>     self.move_wheel_files(self.source_dir, root=root)
>   File "/usr/lib/python3/dist-packages/pip/req.py", line 902, in
> move_wheel_files
>     pycompile=self.pycompile,
>   File "/usr/lib/python3/dist-packages/pip/wheel.py", line 206, in
> move_wheel_files
>     clobber(source, lib_dir, True)
>   File "/usr/lib/python3/dist-packages/pip/wheel.py", line 193, in clobber
>     os.makedirs(destsubdir)
>   File "/usr/lib/python3.4/os.py", line 237, in makedirs
>     mkdir(name, mode)
> PermissionError: [Errno 13] Permission denied:
> '/usr/local/lib/python3.4/dist-packages/avocado'
> 

This looks to me like it's caused by the use of the system wide Python
3.4 installation, as mentioned above.  That, and possibly the
"--system-wide-packages" flag to the venv.

So, in order to understand if, in those environments, we should let
"configure" pick up the right Python, I did the following experiment:

 $ python3 -m venv /tmp/py3-build
 $ . /tmp/py3-build/bin/activate
 $ which python
 /tmp/py3-build/bin/python
 $ /tmp/py3-build/bin/python --version
 Python 3.6.6

This shows that when a venv is activated, the "right" Python should be
available in the $PATH.  So, building QEMU without pointint to a
specific Python binary (when inside a venv) gives you:

 $ mkdir -p /tmp/qemu-build
 $ cd /tmp/qemu-build
 $ ~/src/qemu/configure --target-list="x86_64-softmmu"
 $ make print-PYTHON
 PYTHON=python -B
 $ which python
 /tmp/py3-build/bin/python

The whole question now is how/where Travis puts the requested Python
version (python: - "3.6").  Is it on a venv?  Is it on a specific
location? Is that reliable enough?

I'm going to look for those answers now.

- Cleber.

> See: https://travis-ci.org/philmd/qemu/jobs/439138706
>
Alex Bennée Oct. 9, 2018, 6:16 p.m. UTC | #5
Cleber Rosa <crosa@redhat.com> writes:

> On 10/9/18 9:46 AM, Philippe Mathieu-Daudé wrote:
>> Hi Cleber,
>>
>> On 09/10/2018 06:18, Cleber Rosa wrote:
>>> This enables the execution of the acceptance tests on Travis.
>>
>> Did you test this? =)
>>
>
> I did have some jobs on Travis that looked promising.  But yeah, later I
> found some issues. :/
>
>>>
>>> Because the Travis environment is based on Ubuntu Trusty, it requires
>>> the python3-pip.
>>>
>>> Note: while another supposedely required component on newer versions
>>> (such as on Bionic) split the Python 3 installation further on the
>>> python3-venv package.
>>>
>>> Signed-off-by: Cleber Rosa <crosa@redhat.com>
>>> ---
>>>  .travis.yml | 6 ++++++
>>>  1 file changed, 6 insertions(+)
>>>
>>> diff --git a/.travis.yml b/.travis.yml
>>> index 95be6ec59f..db1a31ea51 100644
>>> --- a/.travis.yml
>>> +++ b/.travis.yml
>>> @@ -36,6 +36,7 @@ addons:
>>>        - liburcu-dev
>>>        - libusb-1.0-0-dev
>>>        - libvte-2.90-dev
>>> +      - python3-pip
>>>        - sparse
>>>        - uuid-dev
>>>        - gcovr
>>> @@ -117,6 +118,11 @@ matrix:
>>>      - env: CONFIG="--target-list=x86_64-softmmu"
>>>        python:
>>>          - "3.6"
>>> +    # Acceptance (Functional) tests
>>> +    - env: CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu"
>>> +           TEST_CMD="make check-acceptance"
>>> +      python:
>>> +        - "3.6"
>>>      # Using newer GCC with sanitizers
>>>      - addons:
>>>          apt:
>>>
>>
>> Using the following patch:
>>
>> -- >8 --
>> diff --git a/.travis.yml b/.travis.yml
>> index 95be6ec59f..87e0c9a13f 100644
>> --- a/.travis.yml
>> +++ b/.travis.yml
>> @@ -118,4 +118,15 @@ matrix:
>>        python:
>>          - "3.6"
>> +    # Acceptance (Functional) tests
>> +    - env: CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu"
>> +           TEST_CMD="make check-acceptance"
>> +           # sudo rm /usr/local/bin/pip*
>> +      python:
>> +        - "3.6"
>> +      addons:
>> +        apt:
>> +          packages:
>> +            - python3-pip
>> +            - python3.4-venv
>
> This patch revealed to me that, even though we're asking Travis for a
> Python 3.6 stack, we're manually pointing to the primary Python (3.4)
> installation.  This is in itself a source of problems.
>
> Read on...
>
>>      # Using newer GCC with sanitizers
>>      - addons:
>> ---
>>
>> I got some improvements until:
>>
>>   VENV    /home/travis/build/philmd/qemu/tests/venv
>>   MKDIR   /home/travis/build/philmd/qemu/tests/results
>>   PIP     /home/travis/build/philmd/qemu/tests/venv-requirements.txt
>> Exception:
>> Traceback (most recent call last):
>>   File "/usr/lib/python3/dist-packages/pip/basecommand.py", line 122, in
>> main
>>     status = self.run(options, args)
>>   File "/usr/lib/python3/dist-packages/pip/commands/install.py", line
>> 283, in run
>>     requirement_set.install(install_options, global_options,
>> root=options.root_path)
>>   File "/usr/lib/python3/dist-packages/pip/req.py", line 1436, in install
>>     requirement.install(install_options, global_options, *args, **kwargs)
>>   File "/usr/lib/python3/dist-packages/pip/req.py", line 672, in install
>>     self.move_wheel_files(self.source_dir, root=root)
>>   File "/usr/lib/python3/dist-packages/pip/req.py", line 902, in
>> move_wheel_files
>>     pycompile=self.pycompile,
>>   File "/usr/lib/python3/dist-packages/pip/wheel.py", line 206, in
>> move_wheel_files
>>     clobber(source, lib_dir, True)
>>   File "/usr/lib/python3/dist-packages/pip/wheel.py", line 193, in clobber
>>     os.makedirs(destsubdir)
>>   File "/usr/lib/python3.4/os.py", line 237, in makedirs
>>     mkdir(name, mode)
>> PermissionError: [Errno 13] Permission denied:
>> '/usr/local/lib/python3.4/dist-packages/avocado'
>>
>
> This looks to me like it's caused by the use of the system wide Python
> 3.4 installation, as mentioned above.  That, and possibly the
> "--system-wide-packages" flag to the venv.
>
> So, in order to understand if, in those environments, we should let
> "configure" pick up the right Python, I did the following experiment:
>
>  $ python3 -m venv /tmp/py3-build
>  $ . /tmp/py3-build/bin/activate
>  $ which python
>  /tmp/py3-build/bin/python
>  $ /tmp/py3-build/bin/python --version
>  Python 3.6.6
>
> This shows that when a venv is activated, the "right" Python should be
> available in the $PATH.  So, building QEMU without pointint to a
> specific Python binary (when inside a venv) gives you:
>
>  $ mkdir -p /tmp/qemu-build
>  $ cd /tmp/qemu-build
>  $ ~/src/qemu/configure --target-list="x86_64-softmmu"
>  $ make print-PYTHON
>  PYTHON=python -B
>  $ which python
>  /tmp/py3-build/bin/python
>
> The whole question now is how/where Travis puts the requested Python
> version (python: - "3.6").  Is it on a venv?  Is it on a specific
> location? Is that reliable enough?
>
> I'm going to look for those answers now.

You can experiment with the Travis environment on your own system using:

  make docker-test-build@travis DEBUG=1

>
> - Cleber.
>
>> See: https://travis-ci.org/philmd/qemu/jobs/439138706
>>


--
Alex Bennée
diff mbox series

Patch

diff --git a/.travis.yml b/.travis.yml
index 95be6ec59f..db1a31ea51 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -36,6 +36,7 @@  addons:
       - liburcu-dev
       - libusb-1.0-0-dev
       - libvte-2.90-dev
+      - python3-pip
       - sparse
       - uuid-dev
       - gcovr
@@ -117,6 +118,11 @@  matrix:
     - env: CONFIG="--target-list=x86_64-softmmu"
       python:
         - "3.6"
+    # Acceptance (Functional) tests
+    - env: CONFIG="--python=/usr/bin/python3 --target-list=x86_64-softmmu"
+           TEST_CMD="make check-acceptance"
+      python:
+        - "3.6"
     # Using newer GCC with sanitizers
     - addons:
         apt: