[1/2] ci: don't update Homebrew
diff mbox series

Message ID 20190703104748.32533-1-szeder.dev@gmail.com
State New
Headers show
Series
  • [1/2] ci: don't update Homebrew
Related show

Commit Message

SZEDER Gábor July 3, 2019, 10:47 a.m. UTC
Lately our GCC macOS build job on Travis CI has been erroring out
while installing dependencies with:

  +brew link gcc@8
  Error: No such keg: /usr/local/Cellar/gcc@8
  The command "ci/install-dependencies.sh" failed and exited with 1 during .

Now, while gcc@8 is still pre-installed (but not linked) and would be
perfectly usable in the Travis CI macOS image we use [1], it's at
version 8.2.  However, when installing dependencies we first
explicitly run 'brew update', which spends over two minutes to update
itself and information about the available packages, and it learns
about GCC 8.3.  After that point gcc@8 exclusively refers to v8.3,
and, unfortunately, 'brew' is just too dumb to be able to do anything
with the still installed 8.2 package, and the subsequent 'brew link
gcc@8' fails.  (Even 'brew uninstall gcc@8' fails with the same
error!)

Don't run 'brew update' to keep the already installed GCC 8.2 'brew
link'-able.  Note that in addition we have to 'export
HOMEBREW_NO_AUTO_UPDATE=1' first, because 'brew' is so very helpful
that it would implicitly run update for us on the next 'brew install
<pkg>' otherwise.

Disabling 'brew update' has additional benefits:

  - It shaves off 2-3mins from the ~4mins currently spent on
    installing dependencies, and the macOS build jobs have always been
    prone to exceeding the time limit on Travis CI.

  - Our builds won't suddenly break because of the occasional Homebrew
    breakages [2].

The drawback is that we'll be stuck with slightly older versions of
the packages that we install via Homebrew (Git-LFS 2.5.2 and Perforce
2018.1; they are currently at 2.7.2 and 2019.1, respectively).  We
might want to reconsider this decision as time goes on and/or switch
to a more recent macOS image as they become available.

[1] 2000ac9fbf (travis-ci: switch to Xcode 10.1 macOS image,
    2019-01-17)

[2] See e.g. a1ccaedd62 (travis-ci: make the OSX build jobs' 'brew
    update' more quiet, 2019-02-02) or

    https://public-inbox.org/git/20180907032002.23366-1-szeder.dev@gmail.com/T/#+u

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
---
 ci/install-dependencies.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Comments

Taylor Blau July 9, 2019, 4:32 p.m. UTC | #1
Hi Gábor,

On Wed, Jul 03, 2019 at 12:47:47PM +0200, SZEDER Gábor wrote:
> Lately our GCC macOS build job on Travis CI has been erroring out
> while installing dependencies with:
>
>   +brew link gcc@8
>   Error: No such keg: /usr/local/Cellar/gcc@8
>   The command "ci/install-dependencies.sh" failed and exited with 1 during .

Thanks for working on this. I think that the patch below is ultimately a
better approach than what we had discussed in a previous thread [1].

> Now, while gcc@8 is still pre-installed (but not linked) and would be
> perfectly usable in the Travis CI macOS image we use [1], it's at
> version 8.2.  However, when installing dependencies we first
> explicitly run 'brew update', which spends over two minutes to update
> itself and information about the available packages, and it learns
> about GCC 8.3.  After that point gcc@8 exclusively refers to v8.3,
> and, unfortunately, 'brew' is just too dumb to be able to do anything
> with the still installed 8.2 package, and the subsequent 'brew link
> gcc@8' fails.  (Even 'brew uninstall gcc@8' fails with the same
> error!)
>
> Don't run 'brew update' to keep the already installed GCC 8.2 'brew
> link'-able.  Note that in addition we have to 'export
> HOMEBREW_NO_AUTO_UPDATE=1' first, because 'brew' is so very helpful
> that it would implicitly run update for us on the next 'brew install
> <pkg>' otherwise.
>
> Disabling 'brew update' has additional benefits:
>
>   - It shaves off 2-3mins from the ~4mins currently spent on
>     installing dependencies, and the macOS build jobs have always been
>     prone to exceeding the time limit on Travis CI.
>
>   - Our builds won't suddenly break because of the occasional Homebrew
>     breakages [2].

This is sensible. We'll rely on the most-recent version of the Homebrew
packages that are known by the macOS image _from Travis_, ignoring any
changes that have happened in Homebrew core since the image snapshot.

Thanks also for paying attention to setting 'HOMEBREW_NO_AUTO_UPDATE' in
the environment. This will harden any other existing or future uses of
'brew' against accidentally spending a bunch of time trying to update
itself (and perhaps breaking a subsequent 'brew link', if it should
happen before that).

> The drawback is that we'll be stuck with slightly older versions of
> the packages that we install via Homebrew (Git-LFS 2.5.2 and Perforce
> 2018.1; they are currently at 2.7.2 and 2019.1, respectively).  We
> might want to reconsider this decision as time goes on and/or switch
> to a more recent macOS image as they become available.

I think that this may in fact be better than what we have now. If we
find ourselves wanting a newer version of, say, Git LFS, then I think
we'd be benefited by upgrading the image itself. Perhaps we should look
into that shortly, although I'm not aware of any urgency to do so (at
least as it pertains to Git LFS).

> [1] 2000ac9fbf (travis-ci: switch to Xcode 10.1 macOS image,
>     2019-01-17)
>
> [2] See e.g. a1ccaedd62 (travis-ci: make the OSX build jobs' 'brew
>     update' more quiet, 2019-02-02) or
>
>     https://public-inbox.org/git/20180907032002.23366-1-szeder.dev@gmail.com/T/#+u
>
> Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
> ---
>  ci/install-dependencies.sh | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
> index 7f6acdd803..7f546c8552 100755
> --- a/ci/install-dependencies.sh
> +++ b/ci/install-dependencies.sh
> @@ -34,7 +34,7 @@ linux-clang|linux-gcc)
>  	popd
>  	;;
>  osx-clang|osx-gcc)
> -	brew update >/dev/null
> +	export HOMEBREW_NO_AUTO_UPDATE=1
>  	# Uncomment this if you want to run perf tests:
>  	# brew install gnu-time
>  	test -z "$BREW_INSTALL_PACKAGES" ||
> --
> 2.22.0.621.ge52941b842.dirty

Thanks,
Taylor

[1]: https://public-inbox.org/git/20190602112239.GO951@szeder.dev
Junio C Hamano July 10, 2019, 10:33 p.m. UTC | #2
Taylor Blau <me@ttaylorr.com> writes:

> Hi Gábor,
>
> On Wed, Jul 03, 2019 at 12:47:47PM +0200, SZEDER Gábor wrote:
>> Lately our GCC macOS build job on Travis CI has been erroring out
>> while installing dependencies with:
>>
>>   +brew link gcc@8
>>   Error: No such keg: /usr/local/Cellar/gcc@8
>>   The command "ci/install-dependencies.sh" failed and exited with 1 during .
>
> Thanks for working on this. I think that the patch below is ultimately a
> better approach than what we had discussed in a previous thread [1].
> ...
> I think that this may in fact be better than what we have now. If we
> find ourselves wanting a newer version of, say, Git LFS, then I think
> we'd be benefited by upgrading the image itself. Perhaps we should look
> into that shortly, although I'm not aware of any urgency to do so (at
> least as it pertains to Git LFS).

Thanks all.  Let's fast-track this fix down to 'maint'.  We left the
macOS build broken for too long.

Patch
diff mbox series

diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
index 7f6acdd803..7f546c8552 100755
--- a/ci/install-dependencies.sh
+++ b/ci/install-dependencies.sh
@@ -34,7 +34,7 @@  linux-clang|linux-gcc)
 	popd
 	;;
 osx-clang|osx-gcc)
-	brew update >/dev/null
+	export HOMEBREW_NO_AUTO_UPDATE=1
 	# Uncomment this if you want to run perf tests:
 	# brew install gnu-time
 	test -z "$BREW_INSTALL_PACKAGES" ||