| Message ID | 4f14ea4aeb44c234d9930578614a35234769fa41.1694450145.git.javi.merino@cloud.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | python: Use setuptools instead of the deprecated distutils | expand |
On Mon, 11 Sep 2023, Javi Merino wrote: > In preparation of dropping python distutils and moving to setuptools, > add the python3 setuptools module to the containers that need it. > > The centos7 container was building using python2. Change it to build > python scripts using python3. > > Debian Stretch is no longer debian oldstable, so move to the archive > repositories. > > Signed-off-by: Javi Merino <javi.merino@cloud.com> > --- > automation/build/alpine/3.18.dockerfile | 1 + > automation/build/archlinux/current.dockerfile | 1 + > automation/build/centos/7.dockerfile | 3 ++- > automation/build/debian/bookworm.dockerfile | 1 + > automation/build/debian/stretch.dockerfile | 11 ++++++++++- > automation/build/suse/opensuse-leap.dockerfile | 1 + > automation/build/ubuntu/bionic.dockerfile | 1 + > automation/build/ubuntu/focal.dockerfile | 1 + > automation/build/ubuntu/trusty.dockerfile | 1 + > automation/build/ubuntu/xenial.dockerfile | 1 + We are missing: automation/build/alpine/3.18-arm64v8.dockerfile automation/build/suse/opensuse-tumbleweed.dockerfile automation/build/suse/opensuse-leap.dockerfile automation/build/debian/jessie-i386.dockerfile automation/build/debian/bookworm-i386.dockerfile automation/build/debian/stretch-i386.dockerfile automation/build/debian/jessie.dockerfile automation/build/debian/bookworm-arm64v8.dockerfile automation/build/fedora/29.dockerfile +Bertrand because I am not sure if we need any changes to the Yocto containers > 10 files changed, 20 insertions(+), 2 deletions(-) > > diff --git a/automation/build/alpine/3.18.dockerfile b/automation/build/alpine/3.18.dockerfile > index ca5756955e..5d2a69a060 100644 > --- a/automation/build/alpine/3.18.dockerfile > +++ b/automation/build/alpine/3.18.dockerfile > @@ -34,6 +34,7 @@ RUN apk --no-cache add \ > ocaml-findlib \ > patch \ > python3-dev \ > + py3-setuptools \ > texinfo \ > util-linux-dev \ > xz-dev \ > diff --git a/automation/build/archlinux/current.dockerfile b/automation/build/archlinux/current.dockerfile > index 13fb472d9e..47e79637a4 100644 > --- a/automation/build/archlinux/current.dockerfile > +++ b/automation/build/archlinux/current.dockerfile > @@ -34,6 +34,7 @@ RUN pacman -S --refresh --sysupgrade --noconfirm --noprogressbar --needed \ > pixman \ > pkgconfig \ > python \ > + python-setuptools \ > sdl \ > sdl2 \ > spice \ > diff --git a/automation/build/centos/7.dockerfile b/automation/build/centos/7.dockerfile > index 69dcefb2f0..f14f70014a 100644 > --- a/automation/build/centos/7.dockerfile > +++ b/automation/build/centos/7.dockerfile > @@ -22,7 +22,8 @@ RUN yum -y update \ > ncurses-devel \ > zlib-devel \ > openssl-devel \ > - python-devel \ > + python3-devel \ > + python3-setuptools \ > libuuid-devel \ > pkgconfig \ > flex \ > diff --git a/automation/build/debian/bookworm.dockerfile b/automation/build/debian/bookworm.dockerfile > index f8415425e6..ae008c8d46 100644 > --- a/automation/build/debian/bookworm.dockerfile > +++ b/automation/build/debian/bookworm.dockerfile > @@ -16,6 +16,7 @@ RUN apt-get update && \ > libncurses5-dev \ > libssl-dev \ > python3-dev \ > + python3-setuptools \ > xorg-dev \ > uuid-dev \ > libyajl-dev \ > diff --git a/automation/build/debian/stretch.dockerfile b/automation/build/debian/stretch.dockerfile > index 1af6c691f8..9f4b91a9e3 100644 > --- a/automation/build/debian/stretch.dockerfile > +++ b/automation/build/debian/stretch.dockerfile > @@ -1,4 +1,4 @@ > -FROM debian:stretch > +FROM debian/eol:stretch > LABEL maintainer.name="The Xen Project" \ > maintainer.email="xen-devel@lists.xenproject.org" > > @@ -8,6 +8,14 @@ ENV USER root > RUN mkdir /build > WORKDIR /build > > +# Debian stretch is now archived. Fetch packages from > +# archive.debian.org and skip the stretch-updates repository, as it is > +# not longer valid. > +RUN sed -i \ > + -e 's/deb.debian.org/archive.debian.org/' \ > + -e '/stretch-updates/d' \ > + /etc/apt/sources.list For the records I would prefer to get rid of Stretch altogether, but I don't want to scope-creep this patch series, so this is OK for now. > # build depends > RUN apt-get update && \ > apt-get --quiet --yes install \ > @@ -17,6 +25,7 @@ RUN apt-get update && \ > libssl-dev \ > python-dev \ > python3-dev \ > + python3-setuptools \ > xorg-dev \ > uuid-dev \ > libyajl-dev \ > diff --git a/automation/build/suse/opensuse-leap.dockerfile b/automation/build/suse/opensuse-leap.dockerfile > index 98ee42970d..7010b71aca 100644 > --- a/automation/build/suse/opensuse-leap.dockerfile > +++ b/automation/build/suse/opensuse-leap.dockerfile > @@ -60,6 +60,7 @@ RUN zypper install -y --no-recommends \ > 'pkgconfig(sdl)' \ > 'pkgconfig(sdl2)' \ > python3-devel \ > + python3-setuptools \ > systemd-devel \ > tar \ > transfig \ > diff --git a/automation/build/ubuntu/bionic.dockerfile b/automation/build/ubuntu/bionic.dockerfile > index e15f54431e..9cceb11ef9 100644 > --- a/automation/build/ubuntu/bionic.dockerfile > +++ b/automation/build/ubuntu/bionic.dockerfile > @@ -17,6 +17,7 @@ RUN apt-get update && \ > libssl-dev \ > python-dev \ > python3-dev \ > + python3-setuptools \ > xorg-dev \ > uuid-dev \ > libyajl-dev \ > diff --git a/automation/build/ubuntu/focal.dockerfile b/automation/build/ubuntu/focal.dockerfile > index 9c851c0620..e3d11194ee 100644 > --- a/automation/build/ubuntu/focal.dockerfile > +++ b/automation/build/ubuntu/focal.dockerfile > @@ -16,6 +16,7 @@ RUN apt-get update && \ > libncurses5-dev \ > libssl-dev \ > python3-dev \ > + python3-setuptools \ > xorg-dev \ > uuid-dev \ > libyajl-dev \ > diff --git a/automation/build/ubuntu/trusty.dockerfile b/automation/build/ubuntu/trusty.dockerfile > index 22e294c20c..7e819c8e4e 100644 > --- a/automation/build/ubuntu/trusty.dockerfile > +++ b/automation/build/ubuntu/trusty.dockerfile > @@ -17,6 +17,7 @@ RUN apt-get update && \ > libssl-dev \ > python-dev \ > python3-dev \ > + python3-setuptools \ > xorg-dev \ > uuid-dev \ > libyajl-dev \ > diff --git a/automation/build/ubuntu/xenial.dockerfile b/automation/build/ubuntu/xenial.dockerfile > index 2d2ba3e31f..19464eead8 100644 > --- a/automation/build/ubuntu/xenial.dockerfile > +++ b/automation/build/ubuntu/xenial.dockerfile > @@ -17,6 +17,7 @@ RUN apt-get update && \ > libssl-dev \ > python-dev \ > python3-dev \ > + python3-setuptools \ > xorg-dev \ > uuid-dev \ > libyajl-dev \ > -- > 2.41.0 >
On 11/09/2023 5:51 pm, Javi Merino wrote: > In preparation of dropping python distutils and moving to setuptools, > add the python3 setuptools module to the containers that need it. > > The centos7 container was building using python2. Change it to build > python scripts using python3. > > Debian Stretch is no longer debian oldstable, so move to the archive > repositories. > > Signed-off-by: Javi Merino <javi.merino@cloud.com> We are not dropping distutils. We're moving to support both distutils and setuptools, because setuptools doesn't support the minimum version of python that Xen supports. Therefore, it's important to keep some of the containers on distutils rather than switching all to setuptools. CenOS can stay as is, as can Stretch and probably Bionic/Focal. Any containers with Py3.10 or later definitely need to move, seeing as distuils is formally deprecated there It's sadly a little too early to make a Py3.12 container, which will lack distutils, but we can come back to that in 4.19. As Stefano points out, you should refresh at least some of the arm64 containers too. RISC-V and PPC aren't set up for tools builds set, so they're fine to leave. ~Andrew
On Mon, Sep 11, 2023 at 06:15:03PM -0700, Stefano Stabellini wrote: > On Mon, 11 Sep 2023, Javi Merino wrote: > > In preparation of dropping python distutils and moving to setuptools, > > add the python3 setuptools module to the containers that need it. > > > > The centos7 container was building using python2. Change it to build > > python scripts using python3. > > > > Debian Stretch is no longer debian oldstable, so move to the archive > > repositories. > > > > Signed-off-by: Javi Merino <javi.merino@cloud.com> > > --- > > automation/build/alpine/3.18.dockerfile | 1 + > > automation/build/archlinux/current.dockerfile | 1 + > > automation/build/centos/7.dockerfile | 3 ++- > > automation/build/debian/bookworm.dockerfile | 1 + > > automation/build/debian/stretch.dockerfile | 11 ++++++++++- > > automation/build/suse/opensuse-leap.dockerfile | 1 + > > automation/build/ubuntu/bionic.dockerfile | 1 + > > automation/build/ubuntu/focal.dockerfile | 1 + > > automation/build/ubuntu/trusty.dockerfile | 1 + > > automation/build/ubuntu/xenial.dockerfile | 1 + > > We are missing: > automation/build/alpine/3.18-arm64v8.dockerfile > automation/build/suse/opensuse-tumbleweed.dockerfile > automation/build/debian/bookworm-i386.dockerfile > automation/build/debian/bookworm-arm64v8.dockerfile > automation/build/debian/stretch-i386.dockerfile Of course! I wasn't able to test it using CI and I missed a bunch of failed tests. I have now added it to all of these. > automation/build/suse/opensuse-leap.dockerfile Leap is already in the list. > automation/build/fedora/29.dockerfile Why? It already has setuptools. > automation/build/debian/jessie-i386.dockerfile > automation/build/debian/jessie.dockerfile Why? The jessie container is not run in automation. Arguably, the docker files should be deleted instead. Cheers, Javi
On Tue, Sep 12, 2023 at 11:18:42AM +0100, Andrew Cooper wrote: > On 11/09/2023 5:51 pm, Javi Merino wrote: > > In preparation of dropping python distutils and moving to setuptools, > > add the python3 setuptools module to the containers that need it. > > > > The centos7 container was building using python2. Change it to build > > python scripts using python3. > > > > Debian Stretch is no longer debian oldstable, so move to the archive > > repositories. > > > > Signed-off-by: Javi Merino <javi.merino@cloud.com> > > We are not dropping distutils. We're moving to support both distutils > and setuptools, because setuptools doesn't support the minimum version > of python that Xen supports. Indeed. I wrote this when the series was about dropping distutils and forgot to update the commit message when I change it to support both. > Therefore, it's important to keep some of the containers on distutils > rather than switching all to setuptools. > > CenOS can stay as is, as can Stretch and probably Bionic/Focal. I agree for CentOS and Debian. For Ubuntu we have the following containers: * 14.04 (trusty) * 16.04 (xenial) * 18.04 (bionic) * 20.04 (focal) Following the logic of only moving the new ones, I will leave as is the old ones (trusty, xenial and bionic) and only move focal to setuptools. > Any containers with Py3.10 or later definitely need to move, seeing as > distuils is formally deprecated there > > It's sadly a little too early to make a Py3.12 container, which will > lack distutils, but we can come back to that in 4.19. Yes, even python's own 3.12 release candidate still includes it. $ podman run --rm -it docker.io/python:3.12-rc-bookworm Python 3.12.0rc2 (main, Sep 8 2023, 03:00:59) [GCC 12.2.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import distutils >>> > As Stefano points out, you should refresh at least some of the arm64 > containers too. RISC-V and PPC aren't set up for tools builds set, so > they're fine to leave. I have refreshed the arm64 containers. Cheers, Javi
diff --git a/automation/build/alpine/3.18.dockerfile b/automation/build/alpine/3.18.dockerfile index ca5756955e..5d2a69a060 100644 --- a/automation/build/alpine/3.18.dockerfile +++ b/automation/build/alpine/3.18.dockerfile @@ -34,6 +34,7 @@ RUN apk --no-cache add \ ocaml-findlib \ patch \ python3-dev \ + py3-setuptools \ texinfo \ util-linux-dev \ xz-dev \ diff --git a/automation/build/archlinux/current.dockerfile b/automation/build/archlinux/current.dockerfile index 13fb472d9e..47e79637a4 100644 --- a/automation/build/archlinux/current.dockerfile +++ b/automation/build/archlinux/current.dockerfile @@ -34,6 +34,7 @@ RUN pacman -S --refresh --sysupgrade --noconfirm --noprogressbar --needed \ pixman \ pkgconfig \ python \ + python-setuptools \ sdl \ sdl2 \ spice \ diff --git a/automation/build/centos/7.dockerfile b/automation/build/centos/7.dockerfile index 69dcefb2f0..f14f70014a 100644 --- a/automation/build/centos/7.dockerfile +++ b/automation/build/centos/7.dockerfile @@ -22,7 +22,8 @@ RUN yum -y update \ ncurses-devel \ zlib-devel \ openssl-devel \ - python-devel \ + python3-devel \ + python3-setuptools \ libuuid-devel \ pkgconfig \ flex \ diff --git a/automation/build/debian/bookworm.dockerfile b/automation/build/debian/bookworm.dockerfile index f8415425e6..ae008c8d46 100644 --- a/automation/build/debian/bookworm.dockerfile +++ b/automation/build/debian/bookworm.dockerfile @@ -16,6 +16,7 @@ RUN apt-get update && \ libncurses5-dev \ libssl-dev \ python3-dev \ + python3-setuptools \ xorg-dev \ uuid-dev \ libyajl-dev \ diff --git a/automation/build/debian/stretch.dockerfile b/automation/build/debian/stretch.dockerfile index 1af6c691f8..9f4b91a9e3 100644 --- a/automation/build/debian/stretch.dockerfile +++ b/automation/build/debian/stretch.dockerfile @@ -1,4 +1,4 @@ -FROM debian:stretch +FROM debian/eol:stretch LABEL maintainer.name="The Xen Project" \ maintainer.email="xen-devel@lists.xenproject.org" @@ -8,6 +8,14 @@ ENV USER root RUN mkdir /build WORKDIR /build +# Debian stretch is now archived. Fetch packages from +# archive.debian.org and skip the stretch-updates repository, as it is +# not longer valid. +RUN sed -i \ + -e 's/deb.debian.org/archive.debian.org/' \ + -e '/stretch-updates/d' \ + /etc/apt/sources.list + # build depends RUN apt-get update && \ apt-get --quiet --yes install \ @@ -17,6 +25,7 @@ RUN apt-get update && \ libssl-dev \ python-dev \ python3-dev \ + python3-setuptools \ xorg-dev \ uuid-dev \ libyajl-dev \ diff --git a/automation/build/suse/opensuse-leap.dockerfile b/automation/build/suse/opensuse-leap.dockerfile index 98ee42970d..7010b71aca 100644 --- a/automation/build/suse/opensuse-leap.dockerfile +++ b/automation/build/suse/opensuse-leap.dockerfile @@ -60,6 +60,7 @@ RUN zypper install -y --no-recommends \ 'pkgconfig(sdl)' \ 'pkgconfig(sdl2)' \ python3-devel \ + python3-setuptools \ systemd-devel \ tar \ transfig \ diff --git a/automation/build/ubuntu/bionic.dockerfile b/automation/build/ubuntu/bionic.dockerfile index e15f54431e..9cceb11ef9 100644 --- a/automation/build/ubuntu/bionic.dockerfile +++ b/automation/build/ubuntu/bionic.dockerfile @@ -17,6 +17,7 @@ RUN apt-get update && \ libssl-dev \ python-dev \ python3-dev \ + python3-setuptools \ xorg-dev \ uuid-dev \ libyajl-dev \ diff --git a/automation/build/ubuntu/focal.dockerfile b/automation/build/ubuntu/focal.dockerfile index 9c851c0620..e3d11194ee 100644 --- a/automation/build/ubuntu/focal.dockerfile +++ b/automation/build/ubuntu/focal.dockerfile @@ -16,6 +16,7 @@ RUN apt-get update && \ libncurses5-dev \ libssl-dev \ python3-dev \ + python3-setuptools \ xorg-dev \ uuid-dev \ libyajl-dev \ diff --git a/automation/build/ubuntu/trusty.dockerfile b/automation/build/ubuntu/trusty.dockerfile index 22e294c20c..7e819c8e4e 100644 --- a/automation/build/ubuntu/trusty.dockerfile +++ b/automation/build/ubuntu/trusty.dockerfile @@ -17,6 +17,7 @@ RUN apt-get update && \ libssl-dev \ python-dev \ python3-dev \ + python3-setuptools \ xorg-dev \ uuid-dev \ libyajl-dev \ diff --git a/automation/build/ubuntu/xenial.dockerfile b/automation/build/ubuntu/xenial.dockerfile index 2d2ba3e31f..19464eead8 100644 --- a/automation/build/ubuntu/xenial.dockerfile +++ b/automation/build/ubuntu/xenial.dockerfile @@ -17,6 +17,7 @@ RUN apt-get update && \ libssl-dev \ python-dev \ python3-dev \ + python3-setuptools \ xorg-dev \ uuid-dev \ libyajl-dev \
In preparation of dropping python distutils and moving to setuptools, add the python3 setuptools module to the containers that need it. The centos7 container was building using python2. Change it to build python scripts using python3. Debian Stretch is no longer debian oldstable, so move to the archive repositories. Signed-off-by: Javi Merino <javi.merino@cloud.com> --- automation/build/alpine/3.18.dockerfile | 1 + automation/build/archlinux/current.dockerfile | 1 + automation/build/centos/7.dockerfile | 3 ++- automation/build/debian/bookworm.dockerfile | 1 + automation/build/debian/stretch.dockerfile | 11 ++++++++++- automation/build/suse/opensuse-leap.dockerfile | 1 + automation/build/ubuntu/bionic.dockerfile | 1 + automation/build/ubuntu/focal.dockerfile | 1 + automation/build/ubuntu/trusty.dockerfile | 1 + automation/build/ubuntu/xenial.dockerfile | 1 + 10 files changed, 20 insertions(+), 2 deletions(-)