| Message ID | 20221124090545.4790-1-worldhello.net@gmail.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | Fix broken CI on newer github-actions runner image | expand |
Hi, On Thu, 24 Nov 2022, Jiang Xin wrote: > From: Jiang Xin <zhiyou.jx@alibaba-inc.com> > > GitHub CI runner image "ubuntu-latest" will upgrade to "ubuntu-22.04" > soon, and the CI runner image of my private host repository has > already been upgraded. > > Our CI will break on new runner image because there is not "gcc-8" > package in ubuntu-22.04. See log of CI#2: > > * https://github.com/jiangxin/git-ci-test/actions/runs/3537628107/jobs/5937769957 > > This issue is fixed in patch 1/3 by downgrade the version of the runner > image for jobs which need "gcc-8". But there are still some CI errors. > This is because p4/p4d version 16.2 cannot run on ubuntu-22.04. See this > log of CI#3: > > * https://github.com/jiangxin/git-ci-test/actions/runs/3537650146/jobs/5937813922 > > This issue is fixed in patch 2/3 by upgrade p4 and p4d. But all p4 > related test cases failed becasue python was missing on ubuntu-22.04. > See the log of CI#4: > > * https://github.com/jiangxin/git-ci-test/actions/runs/3537695959/jobs/5937929695 > > This issue is fixed in patch 3/3 by install python2/python3 on ubutnu. > > If install p4 version 22.2, will break several p4 related test cases, > see the log of CI#7: > > * https://github.com/jiangxin/git-ci-test/actions/runs/3538795233/jobs/5939989823 > > So we choose p4 21.2, and the final successful log of CI#8 is below: > > * https://github.com/jiangxin/git-ci-test/actions/runs/3538946849 ACK! I verified that this patch series does the job, by applying it on top of `microsoft/git`'s tentative rebase to v2.39.0-rc0: https://github.com/dscho/git/actions/runs/3539338333/jobs/5941122554 This run was restricted to `linux-clang` and the p4 tests. I care a lot about not using more resources than one's fair share, therefore I wanted to avoid using build minutes unnecessarily during my debuging. To build even more confidence in the patch series, I will now start a full run (which will take *a lot* of build minutes, unfortunately). Thank you so much! Dscho
Hi, On Thu, 24 Nov 2022, Johannes Schindelin wrote: > To build even more confidence in the patch series, I will now start a full > run (which will take *a lot* of build minutes, unfortunately). And it passed: https://github.com/dscho/git/actions/runs/3539451056 I also had a look at the range-diff between v1 and v2: -- snip -- 1: ef80c39de1e5 ! 1: 6d4607a4ee46 github-actions: run gcc-8 on ubuntu-20.04 image @@ Commit message ubuntu) ;; + In this way, we can change the "ubuntu-latest" runner image to any + version such as "ubuntu-22.04" to test CI behavior in advance. + Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com> 2: 1d0903c8b2f9 ! 2: eba96648368a ci: upgrade version of p4 @@ Metadata Author: Jiang Xin <zhiyou.jx@alibaba-inc.com> ## Commit message ## - ci: upgrade version of p4 + ci: upgrade version of p4 to 21.2 There would be a segmentation fault when running p4 v16.2 on ubuntu 22.04 which is the latest version of ubuntu runner image for github - actions. Upgrade p4 from version 16.2 to 19.2 will fix this issue. + actions. - Also add some instructions to show errors of command "p4 -V", so we can - see why the output doesn't match. + By checking each version from [1], p4d version 21.1 and above can work + properly on ubuntu 22.04. But version 22.x will break some p4 test + cases. So p4 version 21.x is exactly the version we can use. + + In addition to upgrade p4 from version 16.2 to 21.2, also add some + instructions to show errors of command "p4 -V", so we can see why the + command output doesn't match. + + [1]: https://cdist2.perforce.com/perforce/ Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com> @@ ci/lib.sh: ubuntu) # image. # Keep that in mind when you encounter a broken OS X build! - export LINUX_P4_VERSION="16.2" -+ export LINUX_P4_VERSION="19.2" ++ export LINUX_P4_VERSION="21.2" export LINUX_GIT_LFS_VERSION="1.5.2" P4_PATH="$HOME/custom/p4" -: ------------ > 3: 8e432f13bef8 ci: install python on ubuntu -- snap -- The changes look good! One alternative I considered about 8e432f13bef8 (ci: install python on ubuntu, 2022-11-24) was to drop testing Python v2.x (it's years past end of life after all, see https://endoflife.date/python). However, as we see frequently demonstrated on the Git mailing list: losing focus, trying to force patch series to address more than a single concern, gets us nowhere, and not even fast. So I agree that the best idea in this patch series is the stop-gap solution to install `python2` on `ubuntu-22.04`, and deal with deprecating Python v2.x support separately, later, or never, whichever comes first ;-) With this, here goes my (probably final, because I think this patch series is ready): Reviewed-by: Johannes Schindelin <johannes.schindelin@gmx.de> Thank you so much for working on this, it was a thoroughly enjoyable experience for me! Ciao, Dscho
On Thu, Nov 24, 2022 at 6:48 PM Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote: > > Hi, > > On Thu, 24 Nov 2022, Johannes Schindelin wrote: > > > To build even more confidence in the patch series, I will now start a full > > run (which will take *a lot* of build minutes, unfortunately). > > And it passed: https://github.com/dscho/git/actions/runs/3539451056 > > I also had a look at the range-diff between v1 and v2: > > -- snip -- > 1: ef80c39de1e5 ! 1: 6d4607a4ee46 github-actions: run gcc-8 on ubuntu-20.04 image > @@ Commit message > ubuntu) > ;; > > + In this way, we can change the "ubuntu-latest" runner image to any > + version such as "ubuntu-22.04" to test CI behavior in advance. > + > Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com> > > 2: 1d0903c8b2f9 ! 2: eba96648368a ci: upgrade version of p4 > @@ Metadata > Author: Jiang Xin <zhiyou.jx@alibaba-inc.com> > > ## Commit message ## > - ci: upgrade version of p4 > + ci: upgrade version of p4 to 21.2 > > There would be a segmentation fault when running p4 v16.2 on ubuntu > 22.04 which is the latest version of ubuntu runner image for github > - actions. Upgrade p4 from version 16.2 to 19.2 will fix this issue. > + actions. > > - Also add some instructions to show errors of command "p4 -V", so we can > - see why the output doesn't match. > + By checking each version from [1], p4d version 21.1 and above can work > + properly on ubuntu 22.04. But version 22.x will break some p4 test > + cases. So p4 version 21.x is exactly the version we can use. > + > + In addition to upgrade p4 from version 16.2 to 21.2, also add some > + instructions to show errors of command "p4 -V", so we can see why the > + command output doesn't match. > + > + [1]: https://cdist2.perforce.com/perforce/ > > Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com> > @@ ci/lib.sh: ubuntu) > # image. > # Keep that in mind when you encounter a broken OS X build! > - export LINUX_P4_VERSION="16.2" > -+ export LINUX_P4_VERSION="19.2" > ++ export LINUX_P4_VERSION="21.2" > export LINUX_GIT_LFS_VERSION="1.5.2" > > P4_PATH="$HOME/custom/p4" > -: ------------ > 3: 8e432f13bef8 ci: install python on ubuntu > -- snap -- > > The changes look good! Thank you for providing this range-diff. -- Jiang Xin
On Thu, Nov 24 2022, Johannes Schindelin wrote: > On Thu, 24 Nov 2022, Johannes Schindelin wrote: > [...] > The changes look good! > > One alternative I considered about 8e432f13bef8 (ci: install python on > ubuntu, 2022-11-24) was to drop testing Python v2.x (it's years past end > of life after all, see https://endoflife.date/python). Just 2 years? :) We're still pinning "perl" to a supported 5.8, which is so out-of-life that I couldn't find a good reference to when exactly it went EOL. My guess based on [1] and "perldoc perlhist" is sometime around 2009-2010. > So I agree that the best idea in this patch series is the stop-gap > solution to install `python2` on `ubuntu-22.04`, and deal with deprecating > Python v2.x support separately, later, or never, whichever comes first ;-) Yes, let's address that later. We had a recent discussion relating to EOL-ing it in-tree. See the ML discussion around[2]. I would like to note that you seem to be assuming that upstream's EOL for something like this should match our EOL for supporting the software. I think one could argue that, but that's not at all the stance we've taken in the past, as the "perl" example shows. I've personally wanted to bump the "perl" dependency more aggressively for purely selfish reasons in the past (being able to use some newer feature), but the reality is that people "back-port" newer git versions onto various older platforms. But in terms of the cost-benefit of the disruption that would incur I also don't think it's worth it (although a bit past 5.8 is probably justifiable at this point). Someone using "perl" on an older system for git's tests and git-svn etc. really doesn't need to worry much about the full security surface that "perl" might provide, which includes e.g. all the CPAN libraries it ships with, I expect that the same would go for "python". The one potential security issue I can think of that we've ever had because of it is that you could trick "gitweb" and the like into ever-growing memory use if you had a perl version older than the "hash randomization" security/DoS fix. But other than that we're not exposing perl, python etc. directly over the Internet, so I think we don't need to be too paranoid about it. 1. https://endoflife.date/perl 2. f7b5ff607fa (git-p4: improve encoding handling to support inconsistent encodings, 2022-04-30)
Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes: >> So I agree that the best idea in this patch series is the stop-gap >> solution to install `python2` on `ubuntu-22.04`, and deal with deprecating >> Python v2.x support separately, later, or never, whichever comes first ;-) > > Yes, let's address that later. We had a recent discussion relating to > EOL-ing it in-tree. See the ML discussion around[2]. > ... > 2. f7b5ff607fa (git-p4: improve encoding handling to support > inconsistent encodings, 2022-04-30) I agree with two of you that a good first step is the posted patch. I do not seem to find any message in the thread that led to the commit about dropping Python 2 support, though. Thanks.
From: Jiang Xin <zhiyou.jx@alibaba-inc.com> GitHub CI runner image "ubuntu-latest" will upgrade to "ubuntu-22.04" soon, and the CI runner image of my private host repository has already been upgraded. Our CI will break on new runner image because there is not "gcc-8" package in ubuntu-22.04. See log of CI#2: * https://github.com/jiangxin/git-ci-test/actions/runs/3537628107/jobs/5937769957 This issue is fixed in patch 1/3 by downgrade the version of the runner image for jobs which need "gcc-8". But there are still some CI errors. This is because p4/p4d version 16.2 cannot run on ubuntu-22.04. See this log of CI#3: * https://github.com/jiangxin/git-ci-test/actions/runs/3537650146/jobs/5937813922 This issue is fixed in patch 2/3 by upgrade p4 and p4d. But all p4 related test cases failed becasue python was missing on ubuntu-22.04. See the log of CI#4: * https://github.com/jiangxin/git-ci-test/actions/runs/3537695959/jobs/5937929695 This issue is fixed in patch 3/3 by install python2/python3 on ubutnu. If install p4 version 22.2, will break several p4 related test cases, see the log of CI#7: * https://github.com/jiangxin/git-ci-test/actions/runs/3538795233/jobs/5939989823 So we choose p4 21.2, and the final successful log of CI#8 is below: * https://github.com/jiangxin/git-ci-test/actions/runs/3538946849 -- Jiang Xin (3): github-actions: run gcc-8 on ubuntu-20.04 image ci: upgrade version of p4 to 21.2 ci: install python on ubuntu .github/workflows/main.yml | 16 ++++++++++++---- ci/install-dependencies.sh | 12 ++++++------ ci/lib.sh | 10 ++++++---- 3 files changed, 24 insertions(+), 14 deletions(-)