| Message ID | 20221124153934.12470-2-worldhello.net@gmail.com (mailing list archive) |
|---|---|
| State | Superseded |
| Headers | show |
| Series | Fix broken CI on newer github-actions runner image | expand |
On Thu, Nov 24 2022, Jiang Xin wrote: > From: Jiang Xin <zhiyou.jx@alibaba-inc.com> > > GitHub starts to upgrade its runner image "ubuntu-latest" from version > "ubuntu-20.04" to version "ubuntu-22.04". It will fail to find and > install "gcc-8" package on the new runner image. > > Change the runner images from "ubuntu-latest" to "ubuntu-20.04" in order > to run with "gcc-8" as a dependency. > > Instead of use the environment "$runs_on_pool" as below: > > case "$runs_on_pool" in > ubuntu-20.04 | ubuntu-latest) > ;; > > we can reuse the os field in the matrix, and use a new environment > "$runs_on_os" as below: > > case "$runs_on_os" in > ubuntu) > ;; > > In this way, we can change the "ubuntu-latest" runner image to any > version such as "ubuntu-22.04" to test CI behavior in advance. > > Reviewed-by: Johannes Schindelin <johannes.schindelin@gmx.de> > Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com> > --- > .github/workflows/main.yml | 16 ++++++++++++---- > ci/install-dependencies.sh | 6 +++--- > ci/lib.sh | 6 +++--- > 3 files changed, 18 insertions(+), 10 deletions(-) > > diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml > index 592f9193a8..da0d8ab0bf 100644 > --- a/.github/workflows/main.yml > +++ b/.github/workflows/main.yml > @@ -224,6 +224,7 @@ jobs: > vector: > - jobname: linux-clang > cc: clang > + os: ubuntu > pool: ubuntu-latest > - jobname: linux-sha256 > cc: clang > @@ -232,36 +233,43 @@ jobs: > - jobname: linux-gcc > cc: gcc > cc_package: gcc-8 > - pool: ubuntu-latest > + os: ubuntu > + pool: ubuntu-20.04 > - jobname: linux-TEST-vars > cc: gcc > - os: ubuntu > cc_package: gcc-8 > - pool: ubuntu-latest > + os: ubuntu > + pool: ubuntu-20.04 > - jobname: osx-clang > cc: clang > + os: macos > pool: macos-latest > - jobname: osx-gcc > cc: gcc > cc_package: gcc-9 > + os: macos > pool: macos-latest > - jobname: linux-gcc-default > cc: gcc > + os: ubuntu > pool: ubuntu-latest > - jobname: linux-leaks > cc: gcc > + os: ubuntu > pool: ubuntu-latest > - jobname: linux-asan > cc: gcc > + os: ubuntu > pool: ubuntu-latest > - jobname: linux-ubsan > cc: gcc > + os: ubuntu > pool: ubuntu-latest > env: > CC: ${{matrix.vector.cc}} > CC_PACKAGE: ${{matrix.vector.cc_package}} > jobname: ${{matrix.vector.jobname}} > - runs_on_pool: ${{matrix.vector.pool}} > + runs_on_os: ${{matrix.vector.os}} > runs-on: ${{matrix.vector.pool}} > steps: > - uses: actions/checkout@v2 > diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh > index 107757a1fe..f639263a62 100755 > --- a/ci/install-dependencies.sh > +++ b/ci/install-dependencies.sh > @@ -11,8 +11,8 @@ UBUNTU_COMMON_PKGS="make libssl-dev libcurl4-openssl-dev libexpat-dev > tcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl > libemail-valid-perl libio-socket-ssl-perl libnet-smtp-ssl-perl" > > -case "$runs_on_pool" in > -ubuntu-latest) > +case "$runs_on_os" in > +ubuntu) > sudo apt-get -q update > sudo apt-get -q -y install language-pack-is libsvn-perl apache2 \ > $UBUNTU_COMMON_PKGS $CC_PACKAGE > @@ -30,7 +30,7 @@ ubuntu-latest) > cp git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs . > popd > ;; > -macos-latest) > +macos) > export HOMEBREW_NO_AUTO_UPDATE=1 HOMEBREW_NO_INSTALL_CLEANUP=1 > # Uncomment this if you want to run perf tests: > # brew install gnu-time > diff --git a/ci/lib.sh b/ci/lib.sh > index 24d20a5d64..0c0767d354 100755 > --- a/ci/lib.sh > +++ b/ci/lib.sh > @@ -225,8 +225,8 @@ export DEFAULT_TEST_TARGET=prove > export GIT_TEST_CLONE_2GB=true > export SKIP_DASHED_BUILT_INS=YesPlease > > -case "$runs_on_pool" in > -ubuntu-latest) > +case "$runs_on_os" in > +ubuntu) > if test "$jobname" = "linux-gcc-default" > then > break > @@ -253,7 +253,7 @@ ubuntu-latest) > GIT_LFS_PATH="$HOME/custom/git-lfs" > export PATH="$GIT_LFS_PATH:$P4_PATH:$PATH" > ;; > -macos-latest) > +macos) > if [ "$jobname" = osx-gcc ] > then > MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)" At the end of [1] I suggested splitting up the refactoring part of this. I still think that would be a good thing to do, but just to comment on the *actual* change (which I was hoping a split-out would make clearer). This is proposing to refactor "runs-on-pool={ubuntu-latest,macos-latest}" to "os={ubuntu,macos}". I think it's worth cleaning up how we match main.yml and ci/*.sh, but think it's a bit of a digression in this otherwise narrowly focused series. Now, this seems to somewhat come back to being all my fault. AFAICT I added "os" in c08bb260105 (CI: rename the "Linux32" job to lower-case "linux32", 2021-11-23) as part of some WIP (or from an earlier version?) that ende dup doing nothing. But looking at it again, I don't see how an "os" here makes much sense. For "macos-latest" if we're going to refactor it let's just stop selecting stuff based on this "pool name", and instead fold it into the adjacent "jobname" selection: diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh index 107757a1fea..b5a373060c1 100755 --- a/ci/install-dependencies.sh +++ b/ci/install-dependencies.sh @@ -30,7 +30,9 @@ ubuntu-latest) cp git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs . popd ;; -macos-latest) +esac +case "$jobname" in +osx-*) export HOMEBREW_NO_AUTO_UPDATE=1 HOMEBREW_NO_INSTALL_CLEANUP=1 # Uncomment this if you want to run perf tests: # brew install gnu-time diff --git a/ci/lib.sh b/ci/lib.sh index 24d20a5d648..fef60a507e9 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -253,19 +253,17 @@ ubuntu-latest) GIT_LFS_PATH="$HOME/custom/git-lfs" export PATH="$GIT_LFS_PATH:$P4_PATH:$PATH" ;; -macos-latest) - if [ "$jobname" = osx-gcc ] - then - MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)" - else - MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python2)" - MAKEFLAGS="$MAKEFLAGS NO_APPLE_COMMON_CRYPTO=NoThanks" - MAKEFLAGS="$MAKEFLAGS NO_OPENSSL=NoThanks" - fi - ;; esac case "$jobname" in +osx-gcc) + MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)" + ;; +osx-clang) + MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python2)" + MAKEFLAGS="$MAKEFLAGS NO_APPLE_COMMON_CRYPTO=NoThanks" + MAKEFLAGS="$MAKEFLAGS NO_OPENSSL=NoThanks" + ;; linux32) CC=gcc ;; That nicely sidesteps the need to refactor it here, and IMO makes it more future-proof. For the "ubuntu" case I think it's even worse, because it now *looks* like a case that should be equivalent to just: test "$(lsb_release -si 2>&1 | grep -x Debian)" = "Debian" But it's not, because we're not setting this "os=ubuntu" in all the places where the OS is Ubuntu. It really just means "the stuff that used to set env.runs_on_pool=ubuntu-latest, which e.g. doesn't cover the "documentation" job, which runs on "ubuntu-latest". So I think *if* we're going to refactor this while-at-it, let's not make it "os=ubuntu", but e.g. "extra_ubuntu_packages: 1", which is really what it means. I.e. we're intercepting it in ci/lib.sh to set the "p4" variables etc., which we only need if we're doing the optional "apt install" on the "ubuntu-latest". 1. https://lore.kernel.org/git/221124.86mt8gpqra.gmgdl@evledraar.gmail.com/
diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 592f9193a8..da0d8ab0bf 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -224,6 +224,7 @@ jobs: vector: - jobname: linux-clang cc: clang + os: ubuntu pool: ubuntu-latest - jobname: linux-sha256 cc: clang @@ -232,36 +233,43 @@ jobs: - jobname: linux-gcc cc: gcc cc_package: gcc-8 - pool: ubuntu-latest + os: ubuntu + pool: ubuntu-20.04 - jobname: linux-TEST-vars cc: gcc - os: ubuntu cc_package: gcc-8 - pool: ubuntu-latest + os: ubuntu + pool: ubuntu-20.04 - jobname: osx-clang cc: clang + os: macos pool: macos-latest - jobname: osx-gcc cc: gcc cc_package: gcc-9 + os: macos pool: macos-latest - jobname: linux-gcc-default cc: gcc + os: ubuntu pool: ubuntu-latest - jobname: linux-leaks cc: gcc + os: ubuntu pool: ubuntu-latest - jobname: linux-asan cc: gcc + os: ubuntu pool: ubuntu-latest - jobname: linux-ubsan cc: gcc + os: ubuntu pool: ubuntu-latest env: CC: ${{matrix.vector.cc}} CC_PACKAGE: ${{matrix.vector.cc_package}} jobname: ${{matrix.vector.jobname}} - runs_on_pool: ${{matrix.vector.pool}} + runs_on_os: ${{matrix.vector.os}} runs-on: ${{matrix.vector.pool}} steps: - uses: actions/checkout@v2 diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh index 107757a1fe..f639263a62 100755 --- a/ci/install-dependencies.sh +++ b/ci/install-dependencies.sh @@ -11,8 +11,8 @@ UBUNTU_COMMON_PKGS="make libssl-dev libcurl4-openssl-dev libexpat-dev tcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl libemail-valid-perl libio-socket-ssl-perl libnet-smtp-ssl-perl" -case "$runs_on_pool" in -ubuntu-latest) +case "$runs_on_os" in +ubuntu) sudo apt-get -q update sudo apt-get -q -y install language-pack-is libsvn-perl apache2 \ $UBUNTU_COMMON_PKGS $CC_PACKAGE @@ -30,7 +30,7 @@ ubuntu-latest) cp git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs . popd ;; -macos-latest) +macos) export HOMEBREW_NO_AUTO_UPDATE=1 HOMEBREW_NO_INSTALL_CLEANUP=1 # Uncomment this if you want to run perf tests: # brew install gnu-time diff --git a/ci/lib.sh b/ci/lib.sh index 24d20a5d64..0c0767d354 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -225,8 +225,8 @@ export DEFAULT_TEST_TARGET=prove export GIT_TEST_CLONE_2GB=true export SKIP_DASHED_BUILT_INS=YesPlease -case "$runs_on_pool" in -ubuntu-latest) +case "$runs_on_os" in +ubuntu) if test "$jobname" = "linux-gcc-default" then break @@ -253,7 +253,7 @@ ubuntu-latest) GIT_LFS_PATH="$HOME/custom/git-lfs" export PATH="$GIT_LFS_PATH:$P4_PATH:$PATH" ;; -macos-latest) +macos) if [ "$jobname" = osx-gcc ] then MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)"