Message ID | 20190906102711.6401-1-szeder.dev@gmail.com (mailing list archive) |
---|---|
State | New, archived |
Headers | show |
Series | ci: install P4 from package to fix build error | expand |
On Fri, Sep 06, 2019 at 12:27:11PM +0200, SZEDER Gábor wrote: > To test 'git-p4' in the Linux Clang and GCC build jobs we used to > install the 'p4' and 'p4d' binaries by directly downloading those > binaries from a Perforce filehost. This has worked just fine ever > since we started using Travis CI [1], but during the last day or so > that filehost appeared to be gone: while its hostname still resolves, > the host doesn't seem to reply to any download request, it doesn't > even refuse the connection, and eventually our build jobs time out > [2]. > > Now, this might be just a temporary glitch, but I'm afraid that it > isn't. Well, now would you believe it, while I was testing this patch (I even made a gitgitgadget PR to run it on Azure Pipelines! :) and touching up its log message the good old Perforce filehost sprang back to life, and the CI build jobs now succeed again even without this patch. > Let's install P4 from the package repository, because this approach > seems to be simpler and more future proof. > > Note that we used to install an old P4 version (2016.2) in the Linux > build jobs, but with this change we'll install the most recent version > available in the Perforce package repository (currently 2019.1). So I'm not quite sure whether we really want this patch. It depends on how important it is to test 'git-p4' with an old P4 version, but I don't really have an opinion on that.
Hi, On Fri, 6 Sep 2019, SZEDER Gábor wrote: > On Fri, Sep 06, 2019 at 12:27:11PM +0200, SZEDER Gábor wrote: > > To test 'git-p4' in the Linux Clang and GCC build jobs we used to > > install the 'p4' and 'p4d' binaries by directly downloading those > > binaries from a Perforce filehost. This has worked just fine ever > > since we started using Travis CI [1], but during the last day or so > > that filehost appeared to be gone: while its hostname still resolves, > > the host doesn't seem to reply to any download request, it doesn't > > even refuse the connection, and eventually our build jobs time out > > [2]. > > > > Now, this might be just a temporary glitch, but I'm afraid that it > > isn't. > > Well, now would you believe it, while I was testing this patch (I even > made a gitgitgadget PR to run it on Azure Pipelines! :) and touching > up its log message the good old Perforce filehost sprang back to life, > and the CI build jobs now succeed again even without this patch. Sorry for being so slow with granting you access to GitGitGadget. FWIW _anybody_ who already was granted access can issue `/allow` commands, it is not just me. > > Let's install P4 from the package repository, because this approach > > seems to be simpler and more future proof. > > > > Note that we used to install an old P4 version (2016.2) in the Linux > > build jobs, but with this change we'll install the most recent version > > available in the Perforce package repository (currently 2019.1). > > So I'm not quite sure whether we really want this patch. It depends > on how important it is to test 'git-p4' with an old P4 version, but I > don't really have an opinion on that. I'd rather have that patch. It seems to be a much better idea to use the package management system than to rely on one host, especially when said host already displayed hiccups. Ciao, Dscho
On Tue, Sep 10, 2019 at 12:51:01AM +0200, Johannes Schindelin wrote: > On Fri, 6 Sep 2019, SZEDER Gábor wrote: > > > On Fri, Sep 06, 2019 at 12:27:11PM +0200, SZEDER Gábor wrote: > > > To test 'git-p4' in the Linux Clang and GCC build jobs we used to > > > install the 'p4' and 'p4d' binaries by directly downloading those > > > binaries from a Perforce filehost. This has worked just fine ever > > > since we started using Travis CI [1], but during the last day or so > > > that filehost appeared to be gone: while its hostname still resolves, > > > the host doesn't seem to reply to any download request, it doesn't > > > even refuse the connection, and eventually our build jobs time out > > > [2]. > > > > > > Now, this might be just a temporary glitch, but I'm afraid that it > > > isn't. > > > > Well, now would you believe it, while I was testing this patch (I even > > made a gitgitgadget PR to run it on Azure Pipelines! :) and touching > > up its log message the good old Perforce filehost sprang back to life, > > and the CI build jobs now succeed again even without this patch. > > Sorry for being so slow with granting you access to GitGitGadget. FWIW > _anybody_ who already was granted access can issue `/allow` commands, it > is not just me. No problem; I was only interested in the results of the Azure Pipelines build, and that seemed to go well even without access. > > > Let's install P4 from the package repository, because this approach > > > seems to be simpler and more future proof. > > > > > > Note that we used to install an old P4 version (2016.2) in the Linux > > > build jobs, but with this change we'll install the most recent version > > > available in the Perforce package repository (currently 2019.1). > > > > So I'm not quite sure whether we really want this patch. It depends > > on how important it is to test 'git-p4' with an old P4 version, but I > > don't really have an opinion on that. > > I'd rather have that patch. It seems to be a much better idea to use the > package management system than to rely on one host, especially when said > host already displayed hiccups. Well, I'm not so sure. As far as I remember this was the first time that this Perforce filehost was inaccessible and a simple "Restart job" could not rectify the situation, because it was inaccessible for about a day or more. OTOH, transient errors or timeouts from 'apt-get update' or 'install' from the official Ubuntu package repositories are not uncommon (at least on Travis CI), although in those cases it's usually enough to just restart the errored job.
Hi, On Tue, 10 Sep 2019, SZEDER Gábor wrote: > On Tue, Sep 10, 2019 at 12:51:01AM +0200, Johannes Schindelin wrote: > > On Fri, 6 Sep 2019, SZEDER Gábor wrote: > > > > > On Fri, Sep 06, 2019 at 12:27:11PM +0200, SZEDER Gábor wrote: > > > > > Let's install P4 from the package repository, because this > > > > approach seems to be simpler and more future proof. > > > > > > > > Note that we used to install an old P4 version (2016.2) in the > > > > Linux build jobs, but with this change we'll install the most > > > > recent version available in the Perforce package repository > > > > (currently 2019.1). > > > > > > So I'm not quite sure whether we really want this patch. It > > > depends on how important it is to test 'git-p4' with an old P4 > > > version, but I don't really have an opinion on that. > > > > I'd rather have that patch. It seems to be a much better idea to use > > the package management system than to rely on one host, especially > > when said host already displayed hiccups. > > Well, I'm not so sure. As far as I remember this was the first time > that this Perforce filehost was inaccessible and a simple "Restart > job" could not rectify the situation, because it was inaccessible for > about a day or more. Right. > OTOH, transient errors or timeouts from 'apt-get update' or 'install' > from the official Ubuntu package repositories are not uncommon (at > least on Travis CI), although in those cases it's usually enough to > just restart the errored job. My impression precisely. I trust the Ubuntu package servers to have transient, _short-lived_ problems ;-) Ciao, Dscho
On Wed, Sep 11, 2019 at 05:12:31PM +0200, Johannes Schindelin wrote: > On Tue, 10 Sep 2019, SZEDER Gábor wrote: > > > On Tue, Sep 10, 2019 at 12:51:01AM +0200, Johannes Schindelin wrote: > > > On Fri, 6 Sep 2019, SZEDER Gábor wrote: > > > > > > > On Fri, Sep 06, 2019 at 12:27:11PM +0200, SZEDER Gábor wrote: > > > > > > > Let's install P4 from the package repository, because this > > > > > approach seems to be simpler and more future proof. > > > > > > > > > > Note that we used to install an old P4 version (2016.2) in the > > > > > Linux build jobs, but with this change we'll install the most > > > > > recent version available in the Perforce package repository > > > > > (currently 2019.1). > > > > > > > > So I'm not quite sure whether we really want this patch. It > > > > depends on how important it is to test 'git-p4' with an old P4 > > > > version, but I don't really have an opinion on that. > > > > > > I'd rather have that patch. It seems to be a much better idea to use > > > the package management system than to rely on one host, especially > > > when said host already displayed hiccups. > > > > Well, I'm not so sure. As far as I remember this was the first time > > that this Perforce filehost was inaccessible and a simple "Restart > > job" could not rectify the situation, because it was inaccessible for > > about a day or more. > > Right. > > > OTOH, transient errors or timeouts from 'apt-get update' or 'install' > > from the official Ubuntu package repositories are not uncommon (at > > least on Travis CI), although in those cases it's usually enough to > > just restart the errored job. > > My impression precisely. I trust the Ubuntu package servers to have > transient, _short-lived_ problems ;-) Heh, that's true, but note that we won't install P4 from the Ubuntu package servers, but from the Perforce package repository, so I doubt we can expect any improvements in this regard. OTOH, instead of issuing only two HTTP requests to download those two binaries we'll issue a total of 9 requests (1 pubkey, 2 package lists, 6 packages) that could go wrong. I'm undecided, and at the very least the proposed commit message should be updated.
diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh index 8cc72503cb..0df48365dc 100755 --- a/ci/install-dependencies.sh +++ b/ci/install-dependencies.sh @@ -5,27 +5,23 @@ . ${0%/*}/lib.sh -P4WHENCE=http://filehost.perforce.com/perforce/r$LINUX_P4_VERSION LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION case "$jobname" in linux-clang|linux-gcc) + wget -qO - https://package.perforce.com/perforce.pubkey | sudo apt-key add - + echo "deb http://package.perforce.com/apt/ubuntu xenial release" >perforce.list + sudo mv perforce.list /etc/apt/sources.list.d/ sudo apt-add-repository -y "ppa:ubuntu-toolchain-r/test" sudo apt-get -q update - sudo apt-get -q -y install language-pack-is libsvn-perl apache2 + sudo apt-get -q -y install language-pack-is libsvn-perl apache2 \ + helix-p4d case "$jobname" in linux-gcc) sudo apt-get -q -y install gcc-8 ;; esac - mkdir --parents "$P4_PATH" - pushd "$P4_PATH" - wget --quiet "$P4WHENCE/bin.linux26x86_64/p4d" - wget --quiet "$P4WHENCE/bin.linux26x86_64/p4" - chmod u+x p4d - chmod u+x p4 - popd mkdir --parents "$GIT_LFS_PATH" pushd "$GIT_LFS_PATH" wget --quiet "$LFSWHENCE/git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" diff --git a/ci/lib.sh b/ci/lib.sh index 44db2d5cbb..efcccfee6f 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -162,17 +162,15 @@ linux-clang|linux-gcc) export GIT_TEST_HTTPD=YesPlease - # The Linux build installs the defined dependency versions below. - # The OS X build installs much more recent versions, whichever + # The Linux build installs the defined dependency version below. + # The OS X build installs much more recent version, whichever # were recorded in the Homebrew database upon creating the OS X # image. # Keep that in mind when you encounter a broken OS X build! - export LINUX_P4_VERSION="16.2" export LINUX_GIT_LFS_VERSION="1.5.2" - P4_PATH="$HOME/custom/p4" GIT_LFS_PATH="$HOME/custom/git-lfs" - export PATH="$GIT_LFS_PATH:$P4_PATH:$PATH" + export PATH="$GIT_LFS_PATH:$PATH" ;; osx-clang|osx-gcc) if [ "$jobname" = osx-gcc ]
To test 'git-p4' in the Linux Clang and GCC build jobs we used to install the 'p4' and 'p4d' binaries by directly downloading those binaries from a Perforce filehost. This has worked just fine ever since we started using Travis CI [1], but during the last day or so that filehost appeared to be gone: while its hostname still resolves, the host doesn't seem to reply to any download request, it doesn't even refuse the connection, and eventually our build jobs time out [2]. Now, this might be just a temporary glitch, but I'm afraid that it isn't. The "Helix Core Server Administrator Guide" [3] describes two ways to install these binaries on Linux, and none of them mentions the filehost that we've been downloading from in the past: - non-package installation: open the website's download page in your web browser, select OS and platform, click on the download link, and eventually you get a .tar.gz archive containing, among other things, the necessary 'p4' and 'p4d' binaries. Although we could use the URL of this archive to download it in our CI scripts with 'wget', nobody said that that URL remains stable, and we would still need to extract the archive and copy the binaries to $PATH. - package installation for various distros, including Ubuntu 16.04 (i.e. the Ubuntu version used both in our Travis CI and Azure Pipelines builds): add a package repository and its pubkey, 'apt-get update && apt-get install', and ready to go. Let's install P4 from the package repository, because this approach seems to be simpler and more future proof. Note that we used to install an old P4 version (2016.2) in the Linux build jobs, but with this change we'll install the most recent version available in the Perforce package repository (currently 2019.1). [1] 522354d70f (Add Travis CI support, 2015-11-27). [2] https://travis-ci.org/git/git/jobs/581429927#L422 [3] https://www.perforce.com/manuals/p4sag/Content/P4SAG/chapter.install.html Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> --- ci/install-dependencies.sh | 14 +++++--------- ci/lib.sh | 8 +++----- 2 files changed, 8 insertions(+), 14 deletions(-)