From patchwork Thu Sep 12 09:43:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff King X-Patchwork-Id: 13801820 Received: from cloud.peff.net (cloud.peff.net [104.130.231.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4A5E3195980 for ; Thu, 12 Sep 2024 09:43:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=104.130.231.41 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726134220; cv=none; b=O38We27lwkcg0d+iuRo1DKkuwHrlyhuo8SyiT4FntNl+EDuqtkzlgRrNqw0Kv60QgzngBai+XPHzkg6uiC+aSUrQrH+4YUWwiOGU+ipY+GEeHiMS/Lstm/kYAPj00EMFrmGwm1fcj8b28O6gV2RVK2pvzwgXca30EANp5uUQ5G4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726134220; c=relaxed/simple; bh=/2RlBHwLlobZMJ51m4fnc071Fvv/0DpZwaBTxyb8Wvc=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=b36uma2FA7IjwTdqjNxx/czrCJC4RZr8Ubrb/5X5G/xPiSsOM3oWTp9qCTedE/kFT/+DVD+sFxZsTTM6rgCLDd6T2iWG/A4VTmBVFK3db7kPf/F6EoDooMicP0y3DkuW94JHlMWJ8txyGp4cwdNba6YvTrTvkPBzMMZmwJQrGUM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=peff.net; spf=pass smtp.mailfrom=peff.net; arc=none smtp.client-ip=104.130.231.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=peff.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=peff.net Received: (qmail 17299 invoked by uid 109); 12 Sep 2024 09:43:37 -0000 Received: from Unknown (HELO peff.net) (10.0.1.2) by cloud.peff.net (qpsmtpd/0.94) with ESMTP; Thu, 12 Sep 2024 09:43:37 +0000 Authentication-Results: cloud.peff.net; auth=none Received: (qmail 23131 invoked by uid 111); 12 Sep 2024 09:43:37 -0000 Received: from coredump.intra.peff.net (HELO coredump.intra.peff.net) (10.0.0.2) by peff.net (qpsmtpd/0.94) with (TLS_AES_256_GCM_SHA384 encrypted) ESMTPS; Thu, 12 Sep 2024 05:43:37 -0400 Authentication-Results: peff.net; auth=none Date: Thu, 12 Sep 2024 05:43:36 -0400 From: Jeff King To: Junio C Hamano Cc: git@vger.kernel.org Subject: [PATCH 1/4] ci: drop run-docker scripts Message-ID: <20240912094336.GA589828@coredump.intra.peff.net> References: <20240912094238.GA589050@coredump.intra.peff.net> Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20240912094238.GA589050@coredump.intra.peff.net> We haven't used these scripts since 4a6e4b9602 (CI: remove Travis CI support, 2021-11-23), as the GitHub Actions config has support for directly running jobs within docker containers. It's possible we might want to resurrect something like this in order to be more agnostic to the CI platform. But it's not clear exactly what it would look like. And in the meantime, it's just a maintenance burden as we make changes to CI config, and is subject to bitrot. In fact it's already broken; it references ci/install-docker-dependencies.sh, which went away in 9cdeb34b96 (ci: merge scripts which install dependencies, 2024-04-12). Signed-off-by: Jeff King --- I think this is worth doing even if we don't take the rest of the patches. But obviously it gets even more bit-rotted as the other patches change the CI config file. ci/run-docker-build.sh | 66 ------------------------------------------ ci/run-docker.sh | 47 ------------------------------ 2 files changed, 113 deletions(-) delete mode 100755 ci/run-docker-build.sh delete mode 100755 ci/run-docker.sh diff --git a/ci/run-docker-build.sh b/ci/run-docker-build.sh deleted file mode 100755 index 6cd832efb9..0000000000 --- a/ci/run-docker-build.sh +++ /dev/null @@ -1,66 +0,0 @@ -#!/bin/sh -# -# Build and test Git inside container -# -# Usage: -# run-docker-build.sh -# - -set -ex - -if test $# -ne 1 || test -z "$1" -then - echo >&2 "usage: run-docker-build.sh " - exit 1 -fi - -case "$jobname" in -linux32) - switch_cmd="linux32 --32bit i386" - ;; -linux-musl) - switch_cmd= - useradd () { adduser -D "$@"; } - ;; -*) - exit 1 - ;; -esac - -"${0%/*}/install-docker-dependencies.sh" - -# If this script runs inside a docker container, then all commands are -# usually executed as root. Consequently, the host user might not be -# able to access the test output files. -# If a non 0 host user id is given, then create a user "ci" with that -# user id to make everything accessible to the host user. -HOST_UID=$1 -if test $HOST_UID -eq 0 -then - # Just in case someone does want to run the test suite as root. - CI_USER=root -else - CI_USER=ci - if test "$(id -u $CI_USER 2>/dev/null)" = $HOST_UID - then - echo "user '$CI_USER' already exists with the requested ID $HOST_UID" - else - useradd -u $HOST_UID $CI_USER - fi -fi - -# Build and test -command $switch_cmd su -m -l $CI_USER -c " - set -ex - export DEVELOPER='$DEVELOPER' - export DEFAULT_TEST_TARGET='$DEFAULT_TEST_TARGET' - export GIT_PROVE_OPTS='$GIT_PROVE_OPTS' - export GIT_TEST_OPTS='$GIT_TEST_OPTS' - export GIT_TEST_CLONE_2GB='$GIT_TEST_CLONE_2GB' - export MAKEFLAGS='$MAKEFLAGS' - export cache_dir='$cache_dir' - cd /usr/src/git - test -n '$cache_dir' && ln -s '$cache_dir/.prove' t/.prove - make - make test -" diff --git a/ci/run-docker.sh b/ci/run-docker.sh deleted file mode 100755 index af89d1624a..0000000000 --- a/ci/run-docker.sh +++ /dev/null @@ -1,47 +0,0 @@ -#!/bin/sh -# -# Download and run Docker image to build and test Git -# - -. ${0%/*}/lib.sh - -case "$jobname" in -linux32) - CI_CONTAINER="daald/ubuntu32:xenial" - ;; -linux-musl) - CI_CONTAINER=alpine - ;; -*) - exit 1 - ;; -esac - -docker pull "$CI_CONTAINER" - -# Use the following command to debug the docker build locally: -# must be 0 if podman is used as drop-in replacement for docker -# $ docker run -itv "${PWD}:/usr/src/git" --entrypoint /bin/sh "$CI_CONTAINER" -# root@container:/# export jobname= -# root@container:/# /usr/src/git/ci/run-docker-build.sh - -container_cache_dir=/tmp/container-cache - -docker run \ - --interactive \ - --env DEVELOPER \ - --env DEFAULT_TEST_TARGET \ - --env GIT_PROVE_OPTS \ - --env GIT_TEST_OPTS \ - --env GIT_TEST_CLONE_2GB \ - --env MAKEFLAGS \ - --env jobname \ - --env cache_dir="$container_cache_dir" \ - --volume "${PWD}:/usr/src/git" \ - --volume "$cache_dir:$container_cache_dir" \ - "$CI_CONTAINER" \ - /usr/src/git/ci/run-docker-build.sh $(id -u $USER) - -check_unignored_build_artifacts - -save_good_tree From patchwork Thu Sep 12 09:45:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff King X-Patchwork-Id: 13801821 Received: from cloud.peff.net (cloud.peff.net [104.130.231.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4958F18EFFB for ; Thu, 12 Sep 2024 09:45:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=104.130.231.41 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726134341; cv=none; b=QTSErI5nnN0safthJjGoo263jZj94rxgoutPJBU/kvL1lmMnPhuGJHZNZglSDNqKUuAv/9qML0UKXSSezG3NvuzPc/VZN2/9ZOzEHAdT2yhoQc06pqO6VOvHW65VmXjab5A6lPBSR6jdXbLiuOyKdK71VFf2xch2wT+muMb0a4s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726134341; c=relaxed/simple; bh=Lmn/arMWC8Hgx39gu4T7pfyRIT08168VuWd53DoZ5xo=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=XIoFVHAYfhGBeoZmk5GrB8h124Bro6kkOVCKveue5eFykU8n9sAhvPvo7zucN6w/Y5xIiW74k1+/QtxIDPNnYNcGewXIOd1QTzdFVxMmX1ZI6cx4J/i3n0xn3KTYOcDTKfXEiYoSyiohr6NggiCDqsAnknSjqKtQndKl7J3DL4I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=peff.net; spf=pass smtp.mailfrom=peff.net; arc=none smtp.client-ip=104.130.231.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=peff.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=peff.net Received: (qmail 17308 invoked by uid 109); 12 Sep 2024 09:45:38 -0000 Received: from Unknown (HELO peff.net) (10.0.1.2) by cloud.peff.net (qpsmtpd/0.94) with ESMTP; Thu, 12 Sep 2024 09:45:38 +0000 Authentication-Results: cloud.peff.net; auth=none Received: (qmail 23170 invoked by uid 111); 12 Sep 2024 09:45:37 -0000 Received: from coredump.intra.peff.net (HELO coredump.intra.peff.net) (10.0.0.2) by peff.net (qpsmtpd/0.94) with (TLS_AES_256_GCM_SHA384 encrypted) ESMTPS; Thu, 12 Sep 2024 05:45:37 -0400 Authentication-Results: peff.net; auth=none Date: Thu, 12 Sep 2024 05:45:37 -0400 From: Jeff King To: Junio C Hamano Cc: git@vger.kernel.org Subject: [PATCH 2/4] ci: unify ubuntu and ubuntu32 dependencies Message-ID: <20240912094537.GB589828@coredump.intra.peff.net> References: <20240912094238.GA589050@coredump.intra.peff.net> Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20240912094238.GA589050@coredump.intra.peff.net> The script to install dependencies has two separate entries for 32-bit and 64-bit Ubuntu systems. This increases the maintenance burden since both should need roughly the same packages. That hasn't been too bad so far because we've stayed on the same 32-bit image since 2017. Trying to move to a newer image revealed several problems with the linux32 job: - newer images complain about using "linux32 --32bit i386", due to seccomp restrictions. We can loosen these with a docker option, but I don't think running it is even doing anything. We use it only for pretending to "apt" that we're on a 32-bit machine, but inside the container image apt is already configured as a 32-bit system (even though the kernel outside the container is obviously 64-bit). Using the same apt invocation for both architectures just gets rid of this call entirely. - we set DEBIAN_FRONTEND to avoid hanging on packages that ask the user questions. This wasn't a problem on the old image, but it is on newer ones. The 64-bit stanza handles this already. As a bonus, the 64-bit stanza uses "apt -q" instead of redirecting output to /dev/null. This would have saved me a lot of debugging time trying to figure out why it was hanging. :) - the old image seems to have zlib-dev installed by default, but newer ones do not. In addition, there were probably many tests being skipped on the 32-bit build because we didn't have support packages installed (e.g., gpg). Now we'll run them. We do need to keep some parts split off just for 64-bit systems: our p4 and lfs installs reference x86_64/amd64 binaries. The downloaded jgit should work in theory, since it's just a jar file embedded in a shell script that relies on the system java. But the system java in our image is too old, so I've left it as 64-bit only for now. Signed-off-by: Jeff King --- Patch is easier to read with "-w". ci/install-dependencies.sh | 34 ++++++++++++++++------------------ 1 file changed, 16 insertions(+), 18 deletions(-) diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh index 4781cd20bb..00cb7df67a 100755 --- a/ci/install-dependencies.sh +++ b/ci/install-dependencies.sh @@ -33,7 +33,7 @@ fedora-*) dnf -yq update >/dev/null && dnf -yq install make gcc findutils diffutils perl python3 gettext zlib-devel expat-devel openssl-devel curl-devel pcre2-devel >/dev/null ;; -ubuntu-*) +ubuntu-*|ubuntu32-*) # Required so that apt doesn't wait for user input on certain packages. export DEBIAN_FRONTEND=noninteractive @@ -45,25 +45,23 @@ ubuntu-*) libemail-valid-perl libio-pty-perl libio-socket-ssl-perl libnet-smtp-ssl-perl libdbd-sqlite3-perl libcgi-pm-perl \ ${CC_PACKAGE:-${CC:-gcc}} $PYTHON_PACKAGE - mkdir --parents "$CUSTOM_PATH" - wget --quiet --directory-prefix="$CUSTOM_PATH" \ - "$P4WHENCE/bin.linux26x86_64/p4d" "$P4WHENCE/bin.linux26x86_64/p4" - chmod a+x "$CUSTOM_PATH/p4d" "$CUSTOM_PATH/p4" + case "$distro" in + ubuntu-*) + mkdir --parents "$CUSTOM_PATH" - wget --quiet "$LFSWHENCE/git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" - tar -xzf "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" \ - -C "$CUSTOM_PATH" --strip-components=1 "git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs" - rm "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" + wget --quiet --directory-prefix="$CUSTOM_PATH" \ + "$P4WHENCE/bin.linux26x86_64/p4d" "$P4WHENCE/bin.linux26x86_64/p4" + chmod a+x "$CUSTOM_PATH/p4d" "$CUSTOM_PATH/p4" - wget --quiet "$JGITWHENCE" --output-document="$CUSTOM_PATH/jgit" - chmod a+x "$CUSTOM_PATH/jgit" - ;; -ubuntu32-*) - sudo linux32 --32bit i386 sh -c ' - apt update >/dev/null && - apt install -y build-essential libcurl4-openssl-dev \ - libssl-dev libexpat-dev gettext python >/dev/null - ' + wget --quiet "$LFSWHENCE/git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" + tar -xzf "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" \ + -C "$CUSTOM_PATH" --strip-components=1 "git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs" + rm "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" + + wget --quiet "$JGITWHENCE" --output-document="$CUSTOM_PATH/jgit" + chmod a+x "$CUSTOM_PATH/jgit" + ;; + esac ;; macos-*) export HOMEBREW_NO_AUTO_UPDATE=1 HOMEBREW_NO_INSTALL_CLEANUP=1 From patchwork Thu Sep 12 09:47:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff King X-Patchwork-Id: 13801822 Received: from cloud.peff.net (cloud.peff.net [104.130.231.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D9EFB19F117 for ; Thu, 12 Sep 2024 09:47:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=104.130.231.41 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726134453; cv=none; b=jLt75R6bJk97AefR+64jN/no4koFbWONLcEmjlkIa0wdHudN1ojaR2S8kPwOLZWLEJEAOokccuOhA3CJGHey/y2xGI5u34scwvCrmnLlJdDFDYHtH6+VTIwFtY5hqk8Ek0Im+PRKTS4cW5cjNQ0f+Ct2vedh7dq2e8UAZDFRMJk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726134453; c=relaxed/simple; bh=RJOHpm/WcLqlNi1wKFQs2REbMSCHENxgsNCnOsu9CEc=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=YN58XLqfxYKauPmE4BhU83ASjaDI03hitdKZ14IfIdHeHZSox11zCjTw25Z7NyFgadtH1YIWWLsR01Ee8yjme0Y7VzVf4wYJR/DPP1HtyMAdDctQ6EIw4aze4zWrWGX7P1Bgyd2i8ckbc8o/lploDrhSDvW1/FrA2POdi7PTqDw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=peff.net; spf=pass smtp.mailfrom=peff.net; arc=none smtp.client-ip=104.130.231.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=peff.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=peff.net Received: (qmail 17317 invoked by uid 109); 12 Sep 2024 09:47:30 -0000 Received: from Unknown (HELO peff.net) (10.0.1.2) by cloud.peff.net (qpsmtpd/0.94) with ESMTP; Thu, 12 Sep 2024 09:47:30 +0000 Authentication-Results: cloud.peff.net; auth=none Received: (qmail 23174 invoked by uid 111); 12 Sep 2024 09:47:30 -0000 Received: from coredump.intra.peff.net (HELO coredump.intra.peff.net) (10.0.0.2) by peff.net (qpsmtpd/0.94) with (TLS_AES_256_GCM_SHA384 encrypted) ESMTPS; Thu, 12 Sep 2024 05:47:30 -0400 Authentication-Results: peff.net; auth=none Date: Thu, 12 Sep 2024 05:47:30 -0400 From: Jeff King To: Junio C Hamano Cc: git@vger.kernel.org Subject: [PATCH 3/4] ci: use more recent linux32 image Message-ID: <20240912094730.GC589828@coredump.intra.peff.net> References: <20240912094238.GA589050@coredump.intra.peff.net> Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20240912094238.GA589050@coredump.intra.peff.net> The Xenial image we're using was released more than 8 years ago. This is a problem for using some recent GitHub Actions scripts, as they require Node.js 20, and all of the binaries they ship need glibc 2.28 or later. We're not using them yet, but moving forward prepares us for a future patch which will. Xenial was actually the last official 32-bit Ubuntu release, but you can still find i386 images for more recent releases. This patch uses Focal, which was released in 2020 (and is the oldest one with glibc 2.28). There are two small downsides here: - while Xenial is pretty old, it is still in LTS support until April 2026. So there's probably some value in testing with such an old system, and we're losing that. - there are no i386 subversion packages in the Focal repository. So we won't be able to test that (OTOH, we had never tested it until the previous patch which unified the 32/64-bit dependency code). Signed-off-by: Jeff King --- .github/workflows/main.yml | 4 ++-- ci/install-dependencies.sh | 11 ++++++++++- 2 files changed, 12 insertions(+), 3 deletions(-) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 97f9b06310..db8e8f75a4 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -339,8 +339,8 @@ jobs: image: alpine distro: alpine-latest - jobname: linux32 - image: daald/ubuntu32:xenial - distro: ubuntu32-16.04 + image: i386/ubuntu:focal + distro: ubuntu32-20.04 - jobname: pedantic image: fedora distro: fedora-latest diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh index 00cb7df67a..735ee6f463 100755 --- a/ci/install-dependencies.sh +++ b/ci/install-dependencies.sh @@ -37,9 +37,18 @@ ubuntu-*|ubuntu32-*) # Required so that apt doesn't wait for user input on certain packages. export DEBIAN_FRONTEND=noninteractive + case "$distro" in + ubuntu-*) + SVN='libsvn-perl subversion' + ;; + *) + SVN= + ;; + esac + sudo apt-get -q update sudo apt-get -q -y install \ - language-pack-is libsvn-perl apache2 cvs cvsps git gnupg subversion \ + language-pack-is apache2 cvs cvsps git gnupg $SVN \ make libssl-dev libcurl4-openssl-dev libexpat-dev wget sudo default-jre \ tcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl \ libemail-valid-perl libio-pty-perl libio-socket-ssl-perl libnet-smtp-ssl-perl libdbd-sqlite3-perl libcgi-pm-perl \ From patchwork Thu Sep 12 09:48:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff King X-Patchwork-Id: 13801823 Received: from cloud.peff.net (cloud.peff.net [104.130.231.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6A95919C57E for ; Thu, 12 Sep 2024 09:48:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=104.130.231.41 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726134525; cv=none; b=bpsaJExBXIlp5v86BM/gtiV4vO2esKESeoeoiPc3HXEgeCt2hmSsHDfk6O7lb610yO7MaKrl22OSXaZSU7p3llGyGtDsBBzGQHkCYp5mWk/w/fyFpi3DeaVH4yXltunDj3iPqTQH56XCAPSBvIeG23X3Iyn6qTbrj2mBsp4MKMA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726134525; c=relaxed/simple; bh=0bXaUXVbuI71wy+G/9fNYyPO9PL6CMtaTdX5o0jELRA=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=GqxVHa50miTry72iCKoz5xOHgZxFQgBIdi9sn6eGg/A+X+0CR/6yz+D9EUQLcbu+glWcVwDeaFyTjqjcbzZE4bcH48EMEcZDwgZe1Z0ROqKFUeijDiR2hD67MerHN4msMbSKRcDbOYTilPqHBQAEza4LDCPKaCRK79ISW0bO+44= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=peff.net; spf=pass smtp.mailfrom=peff.net; arc=none smtp.client-ip=104.130.231.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=peff.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=peff.net Received: (qmail 17326 invoked by uid 109); 12 Sep 2024 09:48:42 -0000 Received: from Unknown (HELO peff.net) (10.0.1.2) by cloud.peff.net (qpsmtpd/0.94) with ESMTP; Thu, 12 Sep 2024 09:48:42 +0000 Authentication-Results: cloud.peff.net; auth=none Received: (qmail 23179 invoked by uid 111); 12 Sep 2024 09:48:42 -0000 Received: from coredump.intra.peff.net (HELO coredump.intra.peff.net) (10.0.0.2) by peff.net (qpsmtpd/0.94) with (TLS_AES_256_GCM_SHA384 encrypted) ESMTPS; Thu, 12 Sep 2024 05:48:42 -0400 Authentication-Results: peff.net; auth=none Date: Thu, 12 Sep 2024 05:48:41 -0400 From: Jeff King To: Junio C Hamano Cc: git@vger.kernel.org Subject: [PATCH 4/4] ci: use regular action versions for linux32 job Message-ID: <20240912094841.GD589828@coredump.intra.peff.net> References: <20240912094238.GA589050@coredump.intra.peff.net> Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20240912094238.GA589050@coredump.intra.peff.net> The linux32 job runs inside a docker container with a 32-bit libc, etc. This breaks any GitHub Actions scripts that are implemented in javascript, because they ship with their own 64-bit version of Node.js that's dynamically linked. They'll fail with a message like: exec /__e/node20/bin/node: no such file or directory because they can't find the runtime linker. This hasn't been a problem until recently because we special-case older, non-javascript versions of these actions for the linux32 job. But it recently became an issue when our old version of actions/upload-artifact was deprecated, causing the job to fail. We worked around that in 90f2c7240c (ci: remove 'Upload failed tests' directories' step from linux32 jobs, 2024-09-09), but it meant a loss of functionality for that job. And we may eventually run into the same deprecation problem with actions/checkout, which can't just be removed. We can solve the linking issue by installing the 64-bit libc and stdc++ packages before doing anything else. Coupled with the switch to a more recent image in the previous patch, that lets us remove the special-casing of the action scripts entirely. Signed-off-by: Jeff King --- .github/workflows/main.yml | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index db8e8f75a4..916a64b673 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -350,17 +350,17 @@ jobs: runs-on: ubuntu-latest container: ${{matrix.vector.image}} steps: - - uses: actions/checkout@v4 - if: matrix.vector.jobname != 'linux32' - - uses: actions/checkout@v1 # cannot be upgraded because Node.js Actions aren't supported in this container + - name: prepare libc6 for actions if: matrix.vector.jobname == 'linux32' + run: apt -q update && apt -q -y install libc6-amd64 lib64stdc++6 + - uses: actions/checkout@v4 - run: ci/install-dependencies.sh - run: ci/run-build-and-tests.sh - name: print test failures if: failure() && env.FAILED_TEST_ARTIFACTS != '' run: ci/print-test-failures.sh - name: Upload failed tests' directories - if: failure() && env.FAILED_TEST_ARTIFACTS != '' && matrix.vector.jobname != 'linux32' + if: failure() && env.FAILED_TEST_ARTIFACTS != '' uses: actions/upload-artifact@v4 with: name: failed-tests-${{matrix.vector.jobname}} From patchwork Fri Sep 13 05:52:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13803002 Received: from fout5-smtp.messagingengine.com (fout5-smtp.messagingengine.com [103.168.172.148]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6874A323D for ; Fri, 13 Sep 2024 05:52:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.148 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726206777; cv=none; b=iMbfHnqeTgscNpCkkCC7kjUp6b2r3k3NTr0xKohj/wUi/o+7LCxK8MfA9NnqhDkzPgfUXzqT8GBusaca16ZFWejfol8le/rkg2m3Ne9BQJIGaLDuRY7mmPBWVL7qHjEGoXSkUlnj5HIhFdhUpfzEYOQPWPDBtjpnlU8rlXXQdsU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726206777; c=relaxed/simple; bh=2mQpO0ZboyXjJjn74F2xbwnMZ6EeEP7PoWSluTqc3O4=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=abtLbid8sCTixLqzVU32lDXFJgSXzMAQb1X9XcpQeUrWc2A02RE6Zs/yFKXPXXVV+YQpyt4ErGpamk+sOZvijV2s7egwaD9lb15puR1yaV4I5G+TsHlCfgnCAaZ3MtFposQJqVGLZArnGxPz+HkBE3xyKMzoJSCZ+Q++2i+0KeY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=MeRpR0PL; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=AOzXLFPA; arc=none smtp.client-ip=103.168.172.148 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="MeRpR0PL"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="AOzXLFPA" Received: from phl-compute-07.internal (phl-compute-07.phl.internal [10.202.2.47]) by mailfout.phl.internal (Postfix) with ESMTP id 7B325138025F; Fri, 13 Sep 2024 01:52:54 -0400 (EDT) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-07.internal (MEProxy); Fri, 13 Sep 2024 01:52:54 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm1; t=1726206774; x=1726293174; bh=mOx/cuDRPc dLERWWu0DuiblHxxgv8CWCaFxuWTvcOpU=; b=MeRpR0PLsA4AIXhZ4+9hnq/sSV xv/u+9x6qqcp/x+vZcOBTWBkxwG34BiLQU1yxqErX3BNslMp9PqpARsdAnkcHHfH WkbIflYh7EZXDK+H7tfDAIRodFpJWQFeizM1kBOfe1UuSlMzrgZ4pXwT4m0MePoe vencu8kEx8faV7gbAW47HAuN1s2D4zUku6TIis2nYKehOE0vajpyv/kY1emUrG0s pCC62K7DpelSchAgxhv3VYJU+oPOO2xAFU54ZDzEglPWbytUfuL2Jcdmhp2+mOKP b2mx+acd1KEZqKHUWLYRdpB076ILIMYnBKxUNCsLUIeRCesWV045vcRUXqmA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1726206774; x=1726293174; bh=mOx/cuDRPcdLERWWu0DuiblHxxgv 8CWCaFxuWTvcOpU=; b=AOzXLFPAdIuss7iWvwmUZUVPGIojCtZBnVaZfuPObKa4 UeIHTveaX5g87QdhJ7EYOlkXg7EVHOO5VYaJqEWCJD3pF98xSwrp3/icZAI0Tiqw m9wgOCw+e34GnJbU1hl6wMkrHW17pI+doCkAyUPt234Nr377OBzBMyenBvddc92f Y4hex4IdzyJDDFV8dNAkGWyBBw8Chz6Lq2YbKnRRTSALeuV7zg4/MNlmlO5uafib L7xyqvUWx6MukNb9d5mOE9Wy/EaUF080rKX6Jfa3698T9ipnf7LEyS18YSPC6kCb uk1yi81zFJL8X1QHccCSCuGf4lGAzgNx1/DugFjyYQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeftddrudejiedgvdduucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdggtfgfnhhsuhgsshgtrhhisggvpdfu rfetoffkrfgpnffqhgenuceurghilhhouhhtmecufedttdenucenucfjughrpeffhffvve fukfhfgggtuggjsehttdertddttddvnecuhfhrohhmpefrrghtrhhitghkucfuthgvihhn hhgrrhguthcuoehpshesphhkshdrihhmqeenucggtffrrghtthgvrhhnpeevhfeffedtte eluefgtdeugfdtkeeugfelkedtheegieekudduueevgeehudekffenucffohhmrghinhep ghhithhlrggsrdgtohhmnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrg hilhhfrhhomhepphhssehpkhhsrdhimhdpnhgspghrtghpthhtohepfedpmhhouggvpehs mhhtphhouhhtpdhrtghpthhtohepphgvfhhfsehpvghffhdrnhgvthdprhgtphhtthhope hgihhtsehvghgvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtohepghhithhsthgvrhes phhosghogidrtghomh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 13 Sep 2024 01:52:53 -0400 (EDT) Received: by vm-mail (OpenSMTPD) with ESMTPSA id aed3448e (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Fri, 13 Sep 2024 05:52:41 +0000 (UTC) Date: Fri, 13 Sep 2024 07:52:51 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Jeff King , Junio C Hamano Subject: [PATCH 5/4] ci: add Ubuntu 16.04 job to GitLab CI Message-ID: <00a9fe6b7d77c16c9fd6dfe746aacf9068a76942.1726206484.git.ps@pks.im> References: <20240912094238.GA589050@coredump.intra.peff.net> Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20240912094238.GA589050@coredump.intra.peff.net> In the preceding commits we had to convert the linux32 job to be based on Ubuntu 20.04 instead of Ubuntu 16.04 due to a limitation in GitHub Workflows. This was the only job left that still tested against this old but supported Ubuntu version, and we have no other jobs that test with a comparatively old Linux distribution. Add a new job to GitLab CI that tests with Ubuntu 16.04 to cover the resulting test gap. GitLab doesn't modify Docker images in the same way GitHub does and thus doesn't fall prey to the same issue. There are two compatibility issues uncovered by this: - Ubuntu 16.04 does not support HTTP/2 in Apache. We thus cannot set `GIT_TEST_HTTPD=true`, which would otherwise cause us to fail when Apache fails to start. - Ubuntu 16.04 cannot use recent JGit versions as they depend on a more recent Java runtime than we have available. We thus disable installing any kind of optional dependencies that do not come from the package manager. These two restrictions are fine though, as we only really care about whether Git compiles and runs on such old distributions in the first place. Signed-off-by: Patrick Steinhardt --- Test runs of this can be found at [1]. Note that the Alpine Linux jobs fail, due to an upstream regression in libcurl [2]. [1]: https://gitlab.com/gitlab-org/git/-/merge_requests/210 [2]: .gitlab-ci.yml | 3 +++ ci/install-dependencies.sh | 5 +++++ ci/lib.sh | 9 ++++++++- 3 files changed, 16 insertions(+), 1 deletion(-) base-commit: f33406dbcc16e699be71aa4982133c325838ae1b diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 2589098eff7..80b1668ebeb 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -25,6 +25,9 @@ test:linux: fi parallel: matrix: + - jobname: linux-old + image: ubuntu:16.04 + CC: gcc - jobname: linux-sha256 image: ubuntu:latest CC: clang diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh index 735ee6f4639..08656a15308 100755 --- a/ci/install-dependencies.sh +++ b/ci/install-dependencies.sh @@ -55,6 +55,11 @@ ubuntu-*|ubuntu32-*) ${CC_PACKAGE:-${CC:-gcc}} $PYTHON_PACKAGE case "$distro" in + ubuntu-16.04) + # Does not support JGit, but we also don't really care about + # the others. We rather care whether Git still compiles and + # runs fine overall. + ;; ubuntu-*) mkdir --parents "$CUSTOM_PATH" diff --git a/ci/lib.sh b/ci/lib.sh index 51f8f59a296..74b430be238 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -336,7 +336,14 @@ ubuntu-*) fi MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=/usr/bin/$PYTHON_PACKAGE" - export GIT_TEST_HTTPD=true + case "$distro" in + ubuntu-16.04) + # Apache is too old for HTTP/2. + ;; + *) + export GIT_TEST_HTTPD=true + ;; + esac # The Linux build installs the defined dependency versions below. # The OS X build installs much more recent versions, whichever