[XEN,v2,3/6] CI: Refresh the Debian 12 arm64 container

Commit Message

Javi Merino Nov. 6, 2024, 1:05 p.m. UTC

Rework the container to use heredocs for readability and use
apt-get --no-install-recommends to keep the size down.  Rename the job
to debian-12-arm64-* to follow the naming scheme of all the other CI
jobs.

This reduces the size of the debian:12-arm64v8 from 2.25GB down to 1.62GB.

The container is left running the builds and tests as root to avoid
breaking the xilinx runners.

Signed-off-by: Javi Merino <[e-mail]>
---
 automation/build/debian/12-arm64v8.dockerfile | 68 +++++++++++++++++++
 .../build/debian/bookworm-arm64v8.dockerfile  | 55 ---------------
 automation/gitlab-ci/build.yaml               |  8 +--
 automation/gitlab-ci/test.yaml                |  4 +-
 automation/scripts/containerize               |  2 +-
 5 files changed, 75 insertions(+), 62 deletions(-)
 create mode 100644 automation/build/debian/12-arm64v8.dockerfile
 delete mode 100644 automation/build/debian/bookworm-arm64v8.dockerfile

Comments

Stefano Stabellini Nov. 6, 2024, 4:23 p.m. UTC | #1

On Wed, 6 Nov 2024, Javi Merino wrote:
> Rework the container to use heredocs for readability and use
> apt-get --no-install-recommends to keep the size down.  Rename the job
> to debian-12-arm64-* to follow the naming scheme of all the other CI
> jobs.
> 
> This reduces the size of the debian:12-arm64v8 from 2.25GB down to 1.62GB.
> 
> The container is left running the builds and tests as root to avoid
> breaking the xilinx runners.
> 
> Signed-off-by: Javi Merino <[e-mail]>


Hi Javi,

I generated a new dependency diff, this time it should be correct, see
the diff appended below. Most of the differences are not concerning,
but I have a couple of questions.


> --- /tmp/1a	2024-11-06 16:12:09.235734221 +0000
> +++ /tmp/2a	2024-11-06 16:12:15.203659831 +0000
> @@ -1,36 +1,32 @@
>  acpica-tools
> -bcc
> -bin86
>  bison
>  build-essential
>  busybox-static
> +ca-certificates
>  checkpolicy
> -clang
>  cpio
>  curl
>  device-tree-compiler
>  expect
> +file
>  flex
> -git
> -libaio-dev

I think this one might be needed to build QEMU but given that we don't
have any build jobs building QEMU on ARM today anymore, it could be
removed. We are going to add a new build job to build QEMU on ARM at
some point soon but we can add any relevant dependency back at that
time.


> +git-core
> +golang-go
> +libbz2-dev
> +libext2fs-dev
>  libfdt-dev
> -libfindlib-ocaml-dev
>  libglib2.0-dev
>  liblzma-dev
> -libncurses5-dev
> -libnl-3-dev

I think it is the same for these two


> +liblzo2-dev
>  libpixman-1-dev
>  libyajl-dev
> -markdown
> -nasm
> +libzstd-dev
> +ocaml-findlib
>  ocaml-nox
> -pandoc

The removal of pandoc and markdown both concerns me. Wouldn't this cause
the documents under docs/ not to be built anymore?


>  pkg-config
>  python3-dev
>  python3-setuptools
> -transfig
>  u-boot-qemu
>  u-boot-tools
>  uuid-dev
>  wget
> -zlib1g-dev

Patch

diff --git a/automation/build/debian/12-arm64v8.dockerfile b/automation/build/debian/12-arm64v8.dockerfile
new file mode 100644
index 000000000000..fa53eb174862
--- /dev/null
+++ b/automation/build/debian/12-arm64v8.dockerfile
@@ -0,0 +1,68 @@ 
+# syntax=docker/dockerfile:1
+FROM --platform=linux/arm64/v8 debian:bookworm
+LABEL maintainer.name="The Xen Project"
+LABEL maintainer.email="[e-mail]"
+
+ENV DEBIAN_FRONTEND=noninteractive
+
+# build depends
+RUN <<EOF
+#!/bin/bash
+    set -eu
+
+    apt-get update
+    DEPS=(
+        # Xen
+        bison
+        build-essential
+        flex
+        # Flask
+        checkpolicy
+
+        # Tools (general)
+        git-core
+        libext2fs-dev
+        libfdt-dev
+        libglib2.0-dev
+        libpixman-1-dev
+        pkg-config
+        wget
+        # libxenguest dombuilder
+        libbz2-dev
+        liblzma-dev
+        liblzo2-dev
+        libzstd-dev
+        # libacpi
+        acpica-tools
+        # libxl
+        uuid-dev
+        libyajl-dev
+        # Python bindings
+        python3-dev
+        python3-setuptools
+        # Golang bindings
+        golang-go
+        # Ocaml bindings/oxenstored
+        ocaml-nox
+        ocaml-findlib
+
+        # for test phase, qemu-smoke-* jobs
+        busybox-static
+        ca-certificates
+        cpio
+        curl
+        device-tree-compiler
+        expect
+        u-boot-qemu
+        # for imagebuilder
+        file
+        u-boot-tools
+    )
+
+    apt-get -y --no-install-recommends install "${DEPS[@]}"
+
+    rm -rf /var/lib/apt/lists*
+EOF
+
+USER root
+WORKDIR /build
diff --git a/automation/build/debian/bookworm-arm64v8.dockerfile b/automation/build/debian/bookworm-arm64v8.dockerfile
deleted file mode 100644
index da8ae7512589..000000000000
--- a/automation/build/debian/bookworm-arm64v8.dockerfile
+++ /dev/null
@@ -1,55 +0,0 @@ 
-# syntax=docker/dockerfile:1
-FROM --platform=linux/arm64/v8 debian:bookworm
-LABEL maintainer.name="The Xen Project" \
-      maintainer.email="[e-mail]"
-
-ENV DEBIAN_FRONTEND=noninteractive
-ENV USER root
-
-RUN mkdir /build
-WORKDIR /build
-
-# build depends
-RUN apt-get update && \
-    apt-get --quiet --yes install \
-        build-essential \
-        zlib1g-dev \
-        libncurses5-dev \
-        python3-dev \
-        python3-setuptools \
-        uuid-dev \
-        libyajl-dev \
-        libaio-dev \
-        libglib2.0-dev \
-        clang \
-        libpixman-1-dev \
-        pkg-config \
-        flex \
-        bison \
-        acpica-tools \
-        libfdt-dev \
-        bin86 \
-        bcc \
-        liblzma-dev \
-        libnl-3-dev \
-        ocaml-nox \
-        libfindlib-ocaml-dev \
-        markdown \
-        transfig \
-        pandoc \
-        checkpolicy \
-        wget \
-        git \
-        nasm \
-        # for test phase, qemu-smoke-* jobs
-        u-boot-qemu \
-        u-boot-tools \
-        device-tree-compiler \
-        curl \
-        cpio \
-        busybox-static \
-        expect \
-        && \
-        apt-get autoremove -y && \
-        apt-get clean && \
-        rm -rf /var/lib/apt/lists* /tmp/* /var/tmp/*
diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index 871beb70e157..8ee1049d857c 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -411,15 +411,15 @@  debian-bookworm-gcc-arm32-debug-earlyprintk:
 
 # Arm builds
 
-debian-bookworm-gcc-arm64:
+debian-12-arm64-gcc:
   extends: .gcc-arm64-build
   variables:
-    CONTAINER: debian:bookworm-arm64v8
+    CONTAINER: debian:12-arm64v8
 
-debian-bookworm-gcc-debug-arm64:
+debian-12-arm64-gcc-debug:
   extends: .gcc-arm64-build-debug
   variables:
-    CONTAINER: debian:bookworm-arm64v8
+    CONTAINER: debian:12-arm64v8
 
 alpine-3.18-gcc-arm64:
   extends: .gcc-arm64-build
diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml
index 5b89cfa33cf8..3a4d0eb7e15d 100644
--- a/automation/gitlab-ci/test.yaml
+++ b/automation/gitlab-ci/test.yaml
@@ -17,7 +17,7 @@ 
 .qemu-arm64:
   extends: .test-jobs-common
   variables:
-    CONTAINER: debian:bookworm-arm64v8
+    CONTAINER: debian:12-arm64v8
     LOGFILE: qemu-smoke-arm64.log
   artifacts:
     paths:
@@ -30,7 +30,7 @@ 
 .qemu-arm32:
   extends: .test-jobs-common
   variables:
-    CONTAINER: debian:bookworm-arm64v8
+    CONTAINER: debian:12-arm64v8
     LOGFILE: qemu-smoke-arm32.log
   artifacts:
     paths:
diff --git a/automation/scripts/containerize b/automation/scripts/containerize
index d72c22c103ff..1b75c8d93724 100755
--- a/automation/scripts/containerize
+++ b/automation/scripts/containerize
@@ -37,7 +37,7 @@  case "_${CONTAINER}" in
     _bookworm|_bookworm-x86_64|_) CONTAINER="${BASE}/debian:12-x86_64" ;;
     _bookworm-i386|_bookworm-x86_32) CONTAINER="${BASE}/debian:12-x86_32" ;;
     _bookworm-arm64v8-arm32-gcc) CONTAINER="${BASE}/debian:bookworm-arm64v8-arm32-gcc" ;;
-    _bookworm-arm64v8) CONTAINER="${BASE}/debian:bookworm-arm64v8" ;;
+    _bookworm-arm64v8) CONTAINER="${BASE}/debian:12-arm64v8" ;;
     _bookworm-cppcheck) CONTAINER="${BASE}/debian:bookworm-cppcheck" ;;
     _opensuse-leap|_leap) CONTAINER="${BASE}/opensuse:leap-15.6-x86_64" ;;
     _opensuse-tumbleweed|_tumbleweed) CONTAINER="${BASE}/opensuse:tumbleweed-x86_64" ;;