| Message ID | 20210121174451.658924-1-thuth@redhat.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | gitlab-ci.yml: Avoid some submodules to speed up the CI a little bit | expand |
Hi Thomas, On 1/21/21 6:44 PM, Thomas Huth wrote: > Since the meson build system rework, the configure script prefers the > git submodules over the system libraries. So we are testing compilation > with capstone, fdt and libslirp as a submodule all over the place, > burning CPU cycles by recompiling these third party modules and wasting > some network bandwidth in the CI by cloning the submodules each time. > Let's stop doing that in at least a couple of jobs and use the system > libraries instead. > > While we're at it, also install meson in the Fedora container, since > it is new enough already, so we do not need to check out the meson > submodule here. > > Signed-off-by: Thomas Huth <thuth@redhat.com> > --- > .gitlab-ci.yml | 6 +++++- > tests/docker/dockerfiles/centos8.docker | 1 + > tests/docker/dockerfiles/debian-amd64.docker | 1 + > tests/docker/dockerfiles/fedora.docker | 3 +++ > tests/docker/dockerfiles/ubuntu2004.docker | 1 + > 5 files changed, 11 insertions(+), 1 deletion(-) So now which are the jobs left building from git submodule clone?
On 26/01/2021 13.08, Philippe Mathieu-Daudé wrote: > Hi Thomas, > > On 1/21/21 6:44 PM, Thomas Huth wrote: >> Since the meson build system rework, the configure script prefers the >> git submodules over the system libraries. So we are testing compilation >> with capstone, fdt and libslirp as a submodule all over the place, >> burning CPU cycles by recompiling these third party modules and wasting >> some network bandwidth in the CI by cloning the submodules each time. >> Let's stop doing that in at least a couple of jobs and use the system >> libraries instead. >> >> While we're at it, also install meson in the Fedora container, since >> it is new enough already, so we do not need to check out the meson >> submodule here. >> >> Signed-off-by: Thomas Huth <thuth@redhat.com> >> --- >> .gitlab-ci.yml | 6 +++++- >> tests/docker/dockerfiles/centos8.docker | 1 + >> tests/docker/dockerfiles/debian-amd64.docker | 1 + >> tests/docker/dockerfiles/fedora.docker | 3 +++ >> tests/docker/dockerfiles/ubuntu2004.docker | 1 + >> 5 files changed, 11 insertions(+), 1 deletion(-) > > So now which are the jobs left building from git submodule clone? All the jobs which don't have a depending test stage jobs should still use the submodules, e.g. build-oss-fuzz, build-tci, etc. Also the centos7-based jobs still use all submodules, e.g. build-crypto-old-nettle etc. Thomas
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index de3a3d25b5..e2f9c99e27 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -109,6 +109,7 @@ build-system-ubuntu: <<: *native_build_job_definition variables: IMAGE: ubuntu2004 + CONFIGURE_ARGS: --enable-fdt=system --enable-slirp=system TARGETS: aarch64-softmmu alpha-softmmu cris-softmmu hppa-softmmu moxie-softmmu microblazeel-softmmu mips64el-softmmu MAKE_CHECK_ARGS: check-build @@ -140,6 +141,7 @@ build-system-debian: <<: *native_build_job_definition variables: IMAGE: debian-amd64 + CONFIGURE_ARGS: --enable-fdt=system TARGETS: arm-softmmu avr-softmmu i386-softmmu mipsel-softmmu riscv64-softmmu sh4eb-softmmu sparc-softmmu xtensaeb-softmmu MAKE_CHECK_ARGS: check-build @@ -185,6 +187,7 @@ build-system-fedora: variables: IMAGE: fedora CONFIGURE_ARGS: --disable-gcrypt --enable-nettle --enable-docs + --enable-fdt=system --enable-slirp=system --enable-capstone=system TARGETS: tricore-softmmu microblaze-softmmu mips-softmmu xtensa-softmmu m68k-softmmu riscv32-softmmu ppc-softmmu sparc64-softmmu MAKE_CHECK_ARGS: check-build @@ -216,7 +219,7 @@ build-system-centos: <<: *native_build_job_definition variables: IMAGE: centos8 - CONFIGURE_ARGS: --disable-nettle --enable-gcrypt + CONFIGURE_ARGS: --disable-nettle --enable-gcrypt --enable-fdt=system TARGETS: ppc64-softmmu or1k-softmmu s390x-softmmu x86_64-softmmu rx-softmmu sh4-softmmu nios2-softmmu MAKE_CHECK_ARGS: check-build @@ -248,6 +251,7 @@ build-system-opensuse: <<: *native_build_job_definition variables: IMAGE: opensuse-leap + CONFIGURE_ARGS: --enable-fdt=system TARGETS: s390x-softmmu x86_64-softmmu aarch64-softmmu MAKE_CHECK_ARGS: check-build artifacts: diff --git a/tests/docker/dockerfiles/centos8.docker b/tests/docker/dockerfiles/centos8.docker index 64cb7a6eda..a763d55730 100644 --- a/tests/docker/dockerfiles/centos8.docker +++ b/tests/docker/dockerfiles/centos8.docker @@ -15,6 +15,7 @@ ENV PACKAGES \ glib2-devel \ libaio-devel \ libepoxy-devel \ + libfdt-devel \ libgcrypt-devel \ lzo-devel \ make \ diff --git a/tests/docker/dockerfiles/debian-amd64.docker b/tests/docker/dockerfiles/debian-amd64.docker index a98314757d..ed546edcd6 100644 --- a/tests/docker/dockerfiles/debian-amd64.docker +++ b/tests/docker/dockerfiles/debian-amd64.docker @@ -21,6 +21,7 @@ RUN apt update && \ libbz2-dev \ liblzo2-dev \ libgcrypt20-dev \ + libfdt-dev \ librdmacm-dev \ libsasl2-dev \ libsnappy-dev \ diff --git a/tests/docker/dockerfiles/fedora.docker b/tests/docker/dockerfiles/fedora.docker index 0b5053f2d0..0d7602abbe 100644 --- a/tests/docker/dockerfiles/fedora.docker +++ b/tests/docker/dockerfiles/fedora.docker @@ -6,6 +6,7 @@ ENV PACKAGES \ brlapi-devel \ bzip2 \ bzip2-devel \ + capstone-devel \ ccache \ clang \ cyrus-sasl-devel \ @@ -37,6 +38,7 @@ ENV PACKAGES \ libpng-devel \ librbd-devel \ libseccomp-devel \ + libslirp-devel \ libssh-devel \ libubsan \ libudev-devel \ @@ -46,6 +48,7 @@ ENV PACKAGES \ llvm \ lzo-devel \ make \ + meson \ mingw32-bzip2 \ mingw32-curl \ mingw32-glib2 \ diff --git a/tests/docker/dockerfiles/ubuntu2004.docker b/tests/docker/dockerfiles/ubuntu2004.docker index ae889d8482..8519584d2b 100644 --- a/tests/docker/dockerfiles/ubuntu2004.docker +++ b/tests/docker/dockerfiles/ubuntu2004.docker @@ -37,6 +37,7 @@ ENV PACKAGES flex bison \ libsasl2-dev \ libsdl2-dev \ libseccomp-dev \ + libslirp-dev \ libsnappy-dev \ libspice-protocol-dev \ libspice-server-dev \
Since the meson build system rework, the configure script prefers the git submodules over the system libraries. So we are testing compilation with capstone, fdt and libslirp as a submodule all over the place, burning CPU cycles by recompiling these third party modules and wasting some network bandwidth in the CI by cloning the submodules each time. Let's stop doing that in at least a couple of jobs and use the system libraries instead. While we're at it, also install meson in the Fedora container, since it is new enough already, so we do not need to check out the meson submodule here. Signed-off-by: Thomas Huth <thuth@redhat.com> --- .gitlab-ci.yml | 6 +++++- tests/docker/dockerfiles/centos8.docker | 1 + tests/docker/dockerfiles/debian-amd64.docker | 1 + tests/docker/dockerfiles/fedora.docker | 3 +++ tests/docker/dockerfiles/ubuntu2004.docker | 1 + 5 files changed, 11 insertions(+), 1 deletion(-)