| Message ID | 20210526002454.124728-32-jsnow@redhat.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | python: create installable package | expand |
26.05.2021 03:24, John Snow wrote: > Add a python container that contains just enough juice for us to run the python > code quality analysis tools. > > Base this container on fedora, because fedora has very convenient > packaging for testing multiple python versions. > > Add two tests: > > check-python-pipenv uses pipenv to test a frozen, very explicit set of > packages against our minimum supported python version, Python 3.6. This > test is not allowed to fail. > > check-python-tox uses tox to install the latest versions of required > python dependencies against a wide array of Python versions from 3.6 to > 3.9, even including the yet-to-be-released Python 3.10. This test is > allowed to fail with a warning. > > Signed-off-by: John Snow <jsnow@redhat.com> > --- > .gitlab-ci.d/containers.yml | 5 +++++ > .gitlab-ci.yml | 26 ++++++++++++++++++++++++++ > tests/docker/dockerfiles/python.docker | 18 ++++++++++++++++++ > 3 files changed, 49 insertions(+) > create mode 100644 tests/docker/dockerfiles/python.docker > > diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml > index 765408ae274..05ebd4dc11d 100644 > --- a/.gitlab-ci.d/containers.yml > +++ b/.gitlab-ci.d/containers.yml > @@ -242,3 +242,8 @@ amd64-opensuse-leap-container: > extends: .container_job_template > variables: > NAME: opensuse-leap > + > +python-container: > + extends: .container_job_template > + variables: > + NAME: python > diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml > index f718b61fa78..cc2a3935c62 100644 > --- a/.gitlab-ci.yml > +++ b/.gitlab-ci.yml > @@ -789,6 +789,32 @@ check-patch: > GIT_DEPTH: 1000 > allow_failure: true > > + > +check-python-pipenv: > + stage: test > + image: $CI_REGISTRY_IMAGE/qemu/python:latest > + script: > + - cd python > + - make venv-check > + variables: > + GIT_DEPTH: 1000 Hmm, interesting, why we need depth = 1000? gitlab recommends to keep that number "small like 10" https://docs.gitlab.com/ee/ci/large_repositories/ > + needs: > + job: python-container > + > + > +check-python-tox: > + stage: test > + image: $CI_REGISTRY_IMAGE/qemu/python:latest > + script: > + - cd python > + - make check-tox > + variables: > + GIT_DEPTH: 1000 > + needs: > + job: python-container > + allow_failure: true > + > + > check-dco: > stage: build > image: $CI_REGISTRY_IMAGE/qemu/centos8:latest > diff --git a/tests/docker/dockerfiles/python.docker b/tests/docker/dockerfiles/python.docker > new file mode 100644 > index 00000000000..56d88417df4 > --- /dev/null > +++ b/tests/docker/dockerfiles/python.docker > @@ -0,0 +1,18 @@ > +# Python library testing environment > + > +FROM fedora:latest > +MAINTAINER John Snow <jsnow@redhat.com> > + > +# Please keep this list sorted alphabetically > +ENV PACKAGES \ > + gcc \ hmm, interesting, why you need gcc to run python linters? > + make \ > + pipenv \ > + python3 \ > + python3-pip \ > + python3-tox \ > + python3-virtualenv \ > + python3.10 > + > +RUN dnf install -y $PACKAGES > +RUN rpm -q $PACKAGES | sort > /packages.txt > weak, as I'm far from understanding the details, I can only check that it looks similar with nearby files and entities: Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
John Snow <jsnow@redhat.com> writes: > Add a python container that contains just enough juice for us to run the python > code quality analysis tools. > > Base this container on fedora, because fedora has very convenient > packaging for testing multiple python versions. > > Add two tests: > > check-python-pipenv uses pipenv to test a frozen, very explicit set of > packages against our minimum supported python version, Python 3.6. This > test is not allowed to fail. > > check-python-tox uses tox to install the latest versions of required > python dependencies against a wide array of Python versions from 3.6 to > 3.9, even including the yet-to-be-released Python 3.10. This test is > allowed to fail with a warning. > > Signed-off-by: John Snow <jsnow@redhat.com> > --- > .gitlab-ci.d/containers.yml | 5 +++++ > .gitlab-ci.yml | 26 ++++++++++++++++++++++++++ > tests/docker/dockerfiles/python.docker | 18 ++++++++++++++++++ > 3 files changed, 49 insertions(+) > create mode 100644 tests/docker/dockerfiles/python.docker > > diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml > index 765408ae274..05ebd4dc11d 100644 > --- a/.gitlab-ci.d/containers.yml > +++ b/.gitlab-ci.d/containers.yml > @@ -242,3 +242,8 @@ amd64-opensuse-leap-container: > extends: .container_job_template > variables: > NAME: opensuse-leap > + > +python-container: > + extends: .container_job_template > + variables: > + NAME: python > diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml > index f718b61fa78..cc2a3935c62 100644 > --- a/.gitlab-ci.yml > +++ b/.gitlab-ci.yml > @@ -789,6 +789,32 @@ check-patch: > GIT_DEPTH: 1000 > allow_failure: true > > + > +check-python-pipenv: > + stage: test > + image: $CI_REGISTRY_IMAGE/qemu/python:latest > + script: > + - cd python > + - make venv-check > + variables: > + GIT_DEPTH: 1000 A GIT_DEPTH of 1000 seems rather arbitrary - why not 1 (or 3)? Do the tools actually care about diff history? > + needs: > + job: python-container > + > + > +check-python-tox: > + stage: test > + image: $CI_REGISTRY_IMAGE/qemu/python:latest > + script: > + - cd python > + - make check-tox > + variables: > + GIT_DEPTH: 1000 > + needs: > + job: python-container > + allow_failure: true > + > + > check-dco: > stage: build > image: $CI_REGISTRY_IMAGE/qemu/centos8:latest > diff --git a/tests/docker/dockerfiles/python.docker b/tests/docker/dockerfiles/python.docker > new file mode 100644 > index 00000000000..56d88417df4 > --- /dev/null > +++ b/tests/docker/dockerfiles/python.docker > @@ -0,0 +1,18 @@ > +# Python library testing environment > + > +FROM fedora:latest > +MAINTAINER John Snow <jsnow@redhat.com> > + > +# Please keep this list sorted alphabetically > +ENV PACKAGES \ > + gcc \ > + make \ > + pipenv \ > + python3 \ > + python3-pip \ > + python3-tox \ > + python3-virtualenv \ > + python3.10 > + > +RUN dnf install -y $PACKAGES > +RUN rpm -q $PACKAGES | sort > /packages.txt Otherwise seems OK to me: Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
On 5/26/21 11:32 AM, Alex Bennée wrote: > > John Snow <jsnow@redhat.com> writes: > >> Add a python container that contains just enough juice for us to run the python >> code quality analysis tools. >> >> Base this container on fedora, because fedora has very convenient >> packaging for testing multiple python versions. >> >> Add two tests: >> >> check-python-pipenv uses pipenv to test a frozen, very explicit set of >> packages against our minimum supported python version, Python 3.6. This >> test is not allowed to fail. >> >> check-python-tox uses tox to install the latest versions of required >> python dependencies against a wide array of Python versions from 3.6 to >> 3.9, even including the yet-to-be-released Python 3.10. This test is >> allowed to fail with a warning. >> >> Signed-off-by: John Snow <jsnow@redhat.com> >> --- >> .gitlab-ci.d/containers.yml | 5 +++++ >> .gitlab-ci.yml | 26 ++++++++++++++++++++++++++ >> tests/docker/dockerfiles/python.docker | 18 ++++++++++++++++++ >> 3 files changed, 49 insertions(+) >> create mode 100644 tests/docker/dockerfiles/python.docker >> >> diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml >> index 765408ae274..05ebd4dc11d 100644 >> --- a/.gitlab-ci.d/containers.yml >> +++ b/.gitlab-ci.d/containers.yml >> @@ -242,3 +242,8 @@ amd64-opensuse-leap-container: >> extends: .container_job_template >> variables: >> NAME: opensuse-leap >> + >> +python-container: >> + extends: .container_job_template >> + variables: >> + NAME: python >> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml >> index f718b61fa78..cc2a3935c62 100644 >> --- a/.gitlab-ci.yml >> +++ b/.gitlab-ci.yml >> @@ -789,6 +789,32 @@ check-patch: >> GIT_DEPTH: 1000 >> allow_failure: true >> >> + >> +check-python-pipenv: >> + stage: test >> + image: $CI_REGISTRY_IMAGE/qemu/python:latest >> + script: >> + - cd python >> + - make venv-check >> + variables: >> + GIT_DEPTH: 1000 > > A GIT_DEPTH of 1000 seems rather arbitrary - why not 1 (or 3)? Do the > tools actually care about diff history? > Just copy-pasted from other check jobs nearby. I'll set it to 1. >> + needs: >> + job: python-container >> + >> + >> +check-python-tox: >> + stage: test >> + image: $CI_REGISTRY_IMAGE/qemu/python:latest >> + script: >> + - cd python >> + - make check-tox >> + variables: >> + GIT_DEPTH: 1000 >> + needs: >> + job: python-container >> + allow_failure: true >> + >> + >> check-dco: >> stage: build >> image: $CI_REGISTRY_IMAGE/qemu/centos8:latest >> diff --git a/tests/docker/dockerfiles/python.docker b/tests/docker/dockerfiles/python.docker >> new file mode 100644 >> index 00000000000..56d88417df4 >> --- /dev/null >> +++ b/tests/docker/dockerfiles/python.docker >> @@ -0,0 +1,18 @@ >> +# Python library testing environment >> + >> +FROM fedora:latest >> +MAINTAINER John Snow <jsnow@redhat.com> >> + >> +# Please keep this list sorted alphabetically >> +ENV PACKAGES \ >> + gcc \ >> + make \ >> + pipenv \ >> + python3 \ >> + python3-pip \ >> + python3-tox \ >> + python3-virtualenv \ >> + python3.10 >> + >> +RUN dnf install -y $PACKAGES >> +RUN rpm -q $PACKAGES | sort > /packages.txt > > > Otherwise seems OK to me: > > Reviewed-by: Alex Bennée <alex.bennee@linaro.org> > Thanks :)
On 5/26/21 5:33 AM, Vladimir Sementsov-Ogievskiy wrote: > 26.05.2021 03:24, John Snow wrote: >> Add a python container that contains just enough juice for us to run >> the python >> code quality analysis tools. >> >> Base this container on fedora, because fedora has very convenient >> packaging for testing multiple python versions. >> >> Add two tests: >> >> check-python-pipenv uses pipenv to test a frozen, very explicit set of >> packages against our minimum supported python version, Python 3.6. This >> test is not allowed to fail. >> >> check-python-tox uses tox to install the latest versions of required >> python dependencies against a wide array of Python versions from 3.6 to >> 3.9, even including the yet-to-be-released Python 3.10. This test is >> allowed to fail with a warning. >> >> Signed-off-by: John Snow <jsnow@redhat.com> >> --- >> .gitlab-ci.d/containers.yml | 5 +++++ >> .gitlab-ci.yml | 26 ++++++++++++++++++++++++++ >> tests/docker/dockerfiles/python.docker | 18 ++++++++++++++++++ >> 3 files changed, 49 insertions(+) >> create mode 100644 tests/docker/dockerfiles/python.docker >> >> diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml >> index 765408ae274..05ebd4dc11d 100644 >> --- a/.gitlab-ci.d/containers.yml >> +++ b/.gitlab-ci.d/containers.yml >> @@ -242,3 +242,8 @@ amd64-opensuse-leap-container: >> extends: .container_job_template >> variables: >> NAME: opensuse-leap >> + >> +python-container: >> + extends: .container_job_template >> + variables: >> + NAME: python >> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml >> index f718b61fa78..cc2a3935c62 100644 >> --- a/.gitlab-ci.yml >> +++ b/.gitlab-ci.yml >> @@ -789,6 +789,32 @@ check-patch: >> GIT_DEPTH: 1000 >> allow_failure: true >> + >> +check-python-pipenv: >> + stage: test >> + image: $CI_REGISTRY_IMAGE/qemu/python:latest >> + script: >> + - cd python >> + - make venv-check >> + variables: >> + GIT_DEPTH: 1000 > > Hmm, interesting, why we need depth = 1000? gitlab recommends to keep > that number "small like 10" > https://docs.gitlab.com/ee/ci/large_repositories/ > Yeah, I don't. Just copy-pasted and didn't consider it. I can set it to "1". The default is apparently 50 and I don't need that either. >> + needs: >> + job: python-container >> + >> + >> +check-python-tox: >> + stage: test >> + image: $CI_REGISTRY_IMAGE/qemu/python:latest >> + script: >> + - cd python >> + - make check-tox >> + variables: >> + GIT_DEPTH: 1000 >> + needs: >> + job: python-container >> + allow_failure: true >> + >> + >> check-dco: >> stage: build >> image: $CI_REGISTRY_IMAGE/qemu/centos8:latest >> diff --git a/tests/docker/dockerfiles/python.docker >> b/tests/docker/dockerfiles/python.docker >> new file mode 100644 >> index 00000000000..56d88417df4 >> --- /dev/null >> +++ b/tests/docker/dockerfiles/python.docker >> @@ -0,0 +1,18 @@ >> +# Python library testing environment >> + >> +FROM fedora:latest >> +MAINTAINER John Snow <jsnow@redhat.com> >> + >> +# Please keep this list sorted alphabetically >> +ENV PACKAGES \ >> + gcc \ > > hmm, interesting, why you need gcc to run python linters? > build requisite for PyPI packages in the event that PyPI only has a sdist and not a bdist for a given dependency during installation. Found that out the hard way. >> + make \ >> + pipenv \ >> + python3 \ >> + python3-pip \ >> + python3-tox \ >> + python3-virtualenv \ >> + python3.10 >> + >> +RUN dnf install -y $PACKAGES >> +RUN rpm -q $PACKAGES | sort > /packages.txt >> > > > weak, as I'm far from understanding the details, I can only check that > it looks similar with nearby files and entities: > > Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> > Thanks!
26.05.2021 21:24, John Snow wrote: > On 5/26/21 5:33 AM, Vladimir Sementsov-Ogievskiy wrote: >> 26.05.2021 03:24, John Snow wrote: >>> Add a python container that contains just enough juice for us to run the python >>> code quality analysis tools. >>> >>> Base this container on fedora, because fedora has very convenient >>> packaging for testing multiple python versions. >>> >>> Add two tests: >>> >>> check-python-pipenv uses pipenv to test a frozen, very explicit set of >>> packages against our minimum supported python version, Python 3.6. This >>> test is not allowed to fail. >>> >>> check-python-tox uses tox to install the latest versions of required >>> python dependencies against a wide array of Python versions from 3.6 to >>> 3.9, even including the yet-to-be-released Python 3.10. This test is >>> allowed to fail with a warning. >>> >>> Signed-off-by: John Snow <jsnow@redhat.com> >>> --- >>> .gitlab-ci.d/containers.yml | 5 +++++ >>> .gitlab-ci.yml | 26 ++++++++++++++++++++++++++ >>> tests/docker/dockerfiles/python.docker | 18 ++++++++++++++++++ >>> 3 files changed, 49 insertions(+) >>> create mode 100644 tests/docker/dockerfiles/python.docker >>> >>> diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml >>> index 765408ae274..05ebd4dc11d 100644 >>> --- a/.gitlab-ci.d/containers.yml >>> +++ b/.gitlab-ci.d/containers.yml >>> @@ -242,3 +242,8 @@ amd64-opensuse-leap-container: >>> extends: .container_job_template >>> variables: >>> NAME: opensuse-leap >>> + >>> +python-container: >>> + extends: .container_job_template >>> + variables: >>> + NAME: python >>> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml >>> index f718b61fa78..cc2a3935c62 100644 >>> --- a/.gitlab-ci.yml >>> +++ b/.gitlab-ci.yml >>> @@ -789,6 +789,32 @@ check-patch: >>> GIT_DEPTH: 1000 >>> allow_failure: true >>> + >>> +check-python-pipenv: >>> + stage: test >>> + image: $CI_REGISTRY_IMAGE/qemu/python:latest >>> + script: >>> + - cd python >>> + - make venv-check >>> + variables: >>> + GIT_DEPTH: 1000 >> >> Hmm, interesting, why we need depth = 1000? gitlab recommends to keep that number "small like 10" https://docs.gitlab.com/ee/ci/large_repositories/ >> > > Yeah, I don't. Just copy-pasted and didn't consider it. I can set it to "1". The default is apparently 50 and I don't need that either. > >>> + needs: >>> + job: python-container >>> + >>> + >>> +check-python-tox: >>> + stage: test >>> + image: $CI_REGISTRY_IMAGE/qemu/python:latest >>> + script: >>> + - cd python >>> + - make check-tox >>> + variables: >>> + GIT_DEPTH: 1000 >>> + needs: >>> + job: python-container >>> + allow_failure: true >>> + >>> + >>> check-dco: >>> stage: build >>> image: $CI_REGISTRY_IMAGE/qemu/centos8:latest >>> diff --git a/tests/docker/dockerfiles/python.docker b/tests/docker/dockerfiles/python.docker >>> new file mode 100644 >>> index 00000000000..56d88417df4 >>> --- /dev/null >>> +++ b/tests/docker/dockerfiles/python.docker >>> @@ -0,0 +1,18 @@ >>> +# Python library testing environment >>> + >>> +FROM fedora:latest >>> +MAINTAINER John Snow <jsnow@redhat.com> >>> + >>> +# Please keep this list sorted alphabetically >>> +ENV PACKAGES \ >>> + gcc \ >> >> hmm, interesting, why you need gcc to run python linters? >> > > build requisite for PyPI packages in the event that PyPI only has a sdist and not a bdist for a given dependency during installation. i.e. some packages are compiled during installation? > > Found that out the hard way. Worth leaving the comment somewhere? (not worth any kind of resending of course) > >>> + make \ >>> + pipenv \ >>> + python3 \ >>> + python3-pip \ >>> + python3-tox \ >>> + python3-virtualenv \ >>> + python3.10 >>> + >>> +RUN dnf install -y $PACKAGES >>> +RUN rpm -q $PACKAGES | sort > /packages.txt >>> >> >> >> weak, as I'm far from understanding the details, I can only check that it looks similar with nearby files and entities: >> >> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> >> > > Thanks! >
On 5/26/21 2:47 PM, Vladimir Sementsov-Ogievskiy wrote: > Worth leaving the comment somewhere? (not worth any kind of resending of > course) Yeah, I'll explain the package selection in the commit message at least. --js
On 5/26/21 2:47 PM, Vladimir Sementsov-Ogievskiy wrote: >> build requisite for PyPI packages in the event that PyPI only has a >> sdist and not a bdist for a given dependency during installation. > > i.e. some packages are compiled during installation? Realized I didn't answer this directly. Yes, sometimes, depending on your platform or your python version or how new the python package is, it may not have a binary distribution available and will require compilation. This comes up for Python 3.10 dependencies right now in particular. They do not have binary distributions because (I assume) 3.10 isn't finalized yet, so they haven't done a re-build. Or something like that. --js
On Wed, May 26, 2021 at 03:56:31PM -0400, John Snow wrote: > On 5/26/21 2:47 PM, Vladimir Sementsov-Ogievskiy wrote: > > > build requisite for PyPI packages in the event that PyPI only has a > > > sdist and not a bdist for a given dependency during installation. > > > > i.e. some packages are compiled during installation? > > Realized I didn't answer this directly. Yes, sometimes, depending on your > platform or your python version or how new the python package is, it may not > have a binary distribution available and will require compilation. > But shouldn't this be known at this time, given that you're putting the depedencies for one specific platform? I'd very very much like to know which packages, for this specific platform, is triggering a Python package build that has C-based extensions. And it would be even weired if such a package does *not* have C-based extensions, and it's still requiring gcc. I would judge it as a major setuptools design issue. > This comes up for Python 3.10 dependencies right now in particular. They do > not have binary distributions because (I assume) 3.10 isn't finalized yet, > so they haven't done a re-build. Or something like that. > > --js OK... but can you share which package available only in source is requiring gcc? I'm not going to get a good night of sleep without knowing that! :) Thanks, - Cleber.
On Thu, May 27, 2021 at 12:17:36PM -0400, Cleber Rosa wrote: > On Wed, May 26, 2021 at 03:56:31PM -0400, John Snow wrote: > > On 5/26/21 2:47 PM, Vladimir Sementsov-Ogievskiy wrote: > > > > build requisite for PyPI packages in the event that PyPI only has a > > > > sdist and not a bdist for a given dependency during installation. > > > > > > i.e. some packages are compiled during installation? > > > > Realized I didn't answer this directly. Yes, sometimes, depending on your > > platform or your python version or how new the python package is, it may not > > have a binary distribution available and will require compilation. > > > > But shouldn't this be known at this time, given that you're putting > the depedencies for one specific platform? I'd very very much like > to know which packages, for this specific platform, is triggering > a Python package build that has C-based extensions. > > And it would be even weired if such a package does *not* have C-based > extensions, and it's still requiring gcc. I would judge it as a > major setuptools design issue. > > > This comes up for Python 3.10 dependencies right now in particular. They do > > not have binary distributions because (I assume) 3.10 isn't finalized yet, > > so they haven't done a re-build. Or something like that. > > > > --js > > OK... but can you share which package available only in source is > requiring gcc? I'm not going to get a good night of sleep without > knowing that! :) > > Thanks, > - Cleber. OK, so typed-ast is the culprit, and we can attest its requirement for a compiler here: https://github.com/python/typed_ast/blob/8eed936014f81a55a3e17310629c40c0203327c3/setup.py#L123 Now I can sleep in peace. :) - Cleber.
On Tue, May 25, 2021 at 08:24:54PM -0400, John Snow wrote: > Add a python container that contains just enough juice for us to run the python > code quality analysis tools. > > Base this container on fedora, because fedora has very convenient > packaging for testing multiple python versions. > > Add two tests: > > check-python-pipenv uses pipenv to test a frozen, very explicit set of > packages against our minimum supported python version, Python 3.6. This > test is not allowed to fail. > > check-python-tox uses tox to install the latest versions of required > python dependencies against a wide array of Python versions from 3.6 to > 3.9, even including the yet-to-be-released Python 3.10. This test is > allowed to fail with a warning. > > Signed-off-by: John Snow <jsnow@redhat.com> > --- > .gitlab-ci.d/containers.yml | 5 +++++ > .gitlab-ci.yml | 26 ++++++++++++++++++++++++++ > tests/docker/dockerfiles/python.docker | 18 ++++++++++++++++++ > 3 files changed, 49 insertions(+) > create mode 100644 tests/docker/dockerfiles/python.docker > > diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml > index 765408ae274..05ebd4dc11d 100644 > --- a/.gitlab-ci.d/containers.yml > +++ b/.gitlab-ci.d/containers.yml > @@ -242,3 +242,8 @@ amd64-opensuse-leap-container: > extends: .container_job_template > variables: > NAME: opensuse-leap > + > +python-container: > + extends: .container_job_template > + variables: > + NAME: python > diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml > index f718b61fa78..cc2a3935c62 100644 > --- a/.gitlab-ci.yml > +++ b/.gitlab-ci.yml > @@ -789,6 +789,32 @@ check-patch: > GIT_DEPTH: 1000 As others have pointed out, this can use a tweak. > allow_failure: true > > + > +check-python-pipenv: > + stage: test > + image: $CI_REGISTRY_IMAGE/qemu/python:latest > + script: > + - cd python > + - make venv-check Nipick: - make -C python venv-check > + variables: > + GIT_DEPTH: 1000 > + needs: > + job: python-container > + > + > +check-python-tox: > + stage: test > + image: $CI_REGISTRY_IMAGE/qemu/python:latest > + script: > + - cd python > + - make check-tox > + variables: > + GIT_DEPTH: 1000 Same here. > + needs: > + job: python-container > + allow_failure: true > + > + > check-dco: > stage: build > image: $CI_REGISTRY_IMAGE/qemu/centos8:latest > diff --git a/tests/docker/dockerfiles/python.docker b/tests/docker/dockerfiles/python.docker > new file mode 100644 > index 00000000000..56d88417df4 > --- /dev/null > +++ b/tests/docker/dockerfiles/python.docker > @@ -0,0 +1,18 @@ > +# Python library testing environment > + > +FROM fedora:latest > +MAINTAINER John Snow <jsnow@redhat.com> > + > +# Please keep this list sorted alphabetically > +ENV PACKAGES \ > + gcc \ Now with this question answered, with or without the suggestion above: Reviewed-by: Cleber Rosa <crosa@redhat.com>
On 5/27/21 1:02 PM, Cleber Rosa wrote: > Nipick: > > - make -C python venv-check ACK, made this change.
diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml index 765408ae274..05ebd4dc11d 100644 --- a/.gitlab-ci.d/containers.yml +++ b/.gitlab-ci.d/containers.yml @@ -242,3 +242,8 @@ amd64-opensuse-leap-container: extends: .container_job_template variables: NAME: opensuse-leap + +python-container: + extends: .container_job_template + variables: + NAME: python diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index f718b61fa78..cc2a3935c62 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -789,6 +789,32 @@ check-patch: GIT_DEPTH: 1000 allow_failure: true + +check-python-pipenv: + stage: test + image: $CI_REGISTRY_IMAGE/qemu/python:latest + script: + - cd python + - make venv-check + variables: + GIT_DEPTH: 1000 + needs: + job: python-container + + +check-python-tox: + stage: test + image: $CI_REGISTRY_IMAGE/qemu/python:latest + script: + - cd python + - make check-tox + variables: + GIT_DEPTH: 1000 + needs: + job: python-container + allow_failure: true + + check-dco: stage: build image: $CI_REGISTRY_IMAGE/qemu/centos8:latest diff --git a/tests/docker/dockerfiles/python.docker b/tests/docker/dockerfiles/python.docker new file mode 100644 index 00000000000..56d88417df4 --- /dev/null +++ b/tests/docker/dockerfiles/python.docker @@ -0,0 +1,18 @@ +# Python library testing environment + +FROM fedora:latest +MAINTAINER John Snow <jsnow@redhat.com> + +# Please keep this list sorted alphabetically +ENV PACKAGES \ + gcc \ + make \ + pipenv \ + python3 \ + python3-pip \ + python3-tox \ + python3-virtualenv \ + python3.10 + +RUN dnf install -y $PACKAGES +RUN rpm -q $PACKAGES | sort > /packages.txt
Add a python container that contains just enough juice for us to run the python code quality analysis tools. Base this container on fedora, because fedora has very convenient packaging for testing multiple python versions. Add two tests: check-python-pipenv uses pipenv to test a frozen, very explicit set of packages against our minimum supported python version, Python 3.6. This test is not allowed to fail. check-python-tox uses tox to install the latest versions of required python dependencies against a wide array of Python versions from 3.6 to 3.9, even including the yet-to-be-released Python 3.10. This test is allowed to fail with a warning. Signed-off-by: John Snow <jsnow@redhat.com> --- .gitlab-ci.d/containers.yml | 5 +++++ .gitlab-ci.yml | 26 ++++++++++++++++++++++++++ tests/docker/dockerfiles/python.docker | 18 ++++++++++++++++++ 3 files changed, 49 insertions(+) create mode 100644 tests/docker/dockerfiles/python.docker