| Message ID | 20240912094537.GB589828@coredump.intra.peff.net (mailing list archive) |
|---|---|
| State | Accepted |
| Commit | e24a7bc7f028bc9b9a54167276c0f15bbc773631 |
| Headers | show |
| Series | make linux32 ci job work with recent actions | expand |
On Thu, Sep 12, 2024 at 05:45:37AM -0400, Jeff King wrote: > diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh > index 4781cd20bb..00cb7df67a 100755 > --- a/ci/install-dependencies.sh > +++ b/ci/install-dependencies.sh > @@ -33,7 +33,7 @@ fedora-*) > dnf -yq update >/dev/null && > dnf -yq install make gcc findutils diffutils perl python3 gettext zlib-devel expat-devel openssl-devel curl-devel pcre2-devel >/dev/null > ;; > -ubuntu-*) > +ubuntu-*|ubuntu32-*) > # Required so that apt doesn't wait for user input on certain packages. > export DEBIAN_FRONTEND=noninteractive > > @@ -45,25 +45,23 @@ ubuntu-*) > libemail-valid-perl libio-pty-perl libio-socket-ssl-perl libnet-smtp-ssl-perl libdbd-sqlite3-perl libcgi-pm-perl \ > ${CC_PACKAGE:-${CC:-gcc}} $PYTHON_PACKAGE > > - mkdir --parents "$CUSTOM_PATH" > - wget --quiet --directory-prefix="$CUSTOM_PATH" \ > - "$P4WHENCE/bin.linux26x86_64/p4d" "$P4WHENCE/bin.linux26x86_64/p4" > - chmod a+x "$CUSTOM_PATH/p4d" "$CUSTOM_PATH/p4" > + case "$distro" in > + ubuntu-*) > + mkdir --parents "$CUSTOM_PATH" > > - wget --quiet "$LFSWHENCE/git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" > - tar -xzf "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" \ > - -C "$CUSTOM_PATH" --strip-components=1 "git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs" > - rm "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" > + wget --quiet --directory-prefix="$CUSTOM_PATH" \ > + "$P4WHENCE/bin.linux26x86_64/p4d" "$P4WHENCE/bin.linux26x86_64/p4" > + chmod a+x "$CUSTOM_PATH/p4d" "$CUSTOM_PATH/p4" > > - wget --quiet "$JGITWHENCE" --output-document="$CUSTOM_PATH/jgit" > - chmod a+x "$CUSTOM_PATH/jgit" > - ;; > -ubuntu32-*) > - sudo linux32 --32bit i386 sh -c ' > - apt update >/dev/null && > - apt install -y build-essential libcurl4-openssl-dev \ > - libssl-dev libexpat-dev gettext python >/dev/null > - ' > + wget --quiet "$LFSWHENCE/git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" > + tar -xzf "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" \ > + -C "$CUSTOM_PATH" --strip-components=1 "git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs" > + rm "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" > + > + wget --quiet "$JGITWHENCE" --output-document="$CUSTOM_PATH/jgit" > + chmod a+x "$CUSTOM_PATH/jgit" > + ;; > + esac > ;; Okay. All of the build dependencies retrieved via apt are the same across two platforms now, but the dependencies we fetch and install manually are 64-bit specific. So overall we get more coverage with less duplication. Nice. Patrick
diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh index 4781cd20bb..00cb7df67a 100755 --- a/ci/install-dependencies.sh +++ b/ci/install-dependencies.sh @@ -33,7 +33,7 @@ fedora-*) dnf -yq update >/dev/null && dnf -yq install make gcc findutils diffutils perl python3 gettext zlib-devel expat-devel openssl-devel curl-devel pcre2-devel >/dev/null ;; -ubuntu-*) +ubuntu-*|ubuntu32-*) # Required so that apt doesn't wait for user input on certain packages. export DEBIAN_FRONTEND=noninteractive @@ -45,25 +45,23 @@ ubuntu-*) libemail-valid-perl libio-pty-perl libio-socket-ssl-perl libnet-smtp-ssl-perl libdbd-sqlite3-perl libcgi-pm-perl \ ${CC_PACKAGE:-${CC:-gcc}} $PYTHON_PACKAGE - mkdir --parents "$CUSTOM_PATH" - wget --quiet --directory-prefix="$CUSTOM_PATH" \ - "$P4WHENCE/bin.linux26x86_64/p4d" "$P4WHENCE/bin.linux26x86_64/p4" - chmod a+x "$CUSTOM_PATH/p4d" "$CUSTOM_PATH/p4" + case "$distro" in + ubuntu-*) + mkdir --parents "$CUSTOM_PATH" - wget --quiet "$LFSWHENCE/git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" - tar -xzf "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" \ - -C "$CUSTOM_PATH" --strip-components=1 "git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs" - rm "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" + wget --quiet --directory-prefix="$CUSTOM_PATH" \ + "$P4WHENCE/bin.linux26x86_64/p4d" "$P4WHENCE/bin.linux26x86_64/p4" + chmod a+x "$CUSTOM_PATH/p4d" "$CUSTOM_PATH/p4" - wget --quiet "$JGITWHENCE" --output-document="$CUSTOM_PATH/jgit" - chmod a+x "$CUSTOM_PATH/jgit" - ;; -ubuntu32-*) - sudo linux32 --32bit i386 sh -c ' - apt update >/dev/null && - apt install -y build-essential libcurl4-openssl-dev \ - libssl-dev libexpat-dev gettext python >/dev/null - ' + wget --quiet "$LFSWHENCE/git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" + tar -xzf "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" \ + -C "$CUSTOM_PATH" --strip-components=1 "git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs" + rm "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" + + wget --quiet "$JGITWHENCE" --output-document="$CUSTOM_PATH/jgit" + chmod a+x "$CUSTOM_PATH/jgit" + ;; + esac ;; macos-*) export HOMEBREW_NO_AUTO_UPDATE=1 HOMEBREW_NO_INSTALL_CLEANUP=1
The script to install dependencies has two separate entries for 32-bit and 64-bit Ubuntu systems. This increases the maintenance burden since both should need roughly the same packages. That hasn't been too bad so far because we've stayed on the same 32-bit image since 2017. Trying to move to a newer image revealed several problems with the linux32 job: - newer images complain about using "linux32 --32bit i386", due to seccomp restrictions. We can loosen these with a docker option, but I don't think running it is even doing anything. We use it only for pretending to "apt" that we're on a 32-bit machine, but inside the container image apt is already configured as a 32-bit system (even though the kernel outside the container is obviously 64-bit). Using the same apt invocation for both architectures just gets rid of this call entirely. - we set DEBIAN_FRONTEND to avoid hanging on packages that ask the user questions. This wasn't a problem on the old image, but it is on newer ones. The 64-bit stanza handles this already. As a bonus, the 64-bit stanza uses "apt -q" instead of redirecting output to /dev/null. This would have saved me a lot of debugging time trying to figure out why it was hanging. :) - the old image seems to have zlib-dev installed by default, but newer ones do not. In addition, there were probably many tests being skipped on the 32-bit build because we didn't have support packages installed (e.g., gpg). Now we'll run them. We do need to keep some parts split off just for 64-bit systems: our p4 and lfs installs reference x86_64/amd64 binaries. The downloaded jgit should work in theory, since it's just a jar file embedded in a shell script that relies on the system java. But the system java in our image is too old, so I've left it as 64-bit only for now. Signed-off-by: Jeff King <peff@peff.net> --- Patch is easier to read with "-w". ci/install-dependencies.sh | 34 ++++++++++++++++------------------ 1 file changed, 16 insertions(+), 18 deletions(-)