[v2,5/5] travis-ci: build with the right compiler
diff mbox series

Message ID 20190117012913.30562-6-szeder.dev@gmail.com
State New
Headers show
Series
  • travis-ci: build with the right compiler
Related show

Commit Message

SZEDER Gábor Jan. 17, 2019, 1:29 a.m. UTC
Our 'Makefile' hardcodes the compiler to build Git as 'CC = cc'.  This
CC variable can be overridden from the command line, i.e. 'make
CC=gcc-X.Y' will build with that particular GCC version, but not from
the environment, i.e. 'CC=gcc-X.Y make' will still build with whatever
'cc' happens to be on the platform.

Our build jobs on Travis CI are badly affected by this.  In the build
matrix we have dedicated build jobs to build Git with GCC and Clang
both on Linux and macOS from the very beginning (522354d70f (Add
Travis CI support, 2015-11-27)).  Alas, this never really worked as
supposed to, because Travis CI specifies the compiler for those build
jobs as 'export CC=gcc' and 'export CC=clang' (which works fine for
projects built with './configure && make').  Consequently, our
'linux-clang' build job has always used GCC, because that's where 'cc'
points at in Travis CI's Linux images, while the 'osx-gcc' build job
has always used Clang.  Furthermore, 37fa4b3c78 (travis-ci: run gcc-8
on linux-gcc jobs, 2018-05-19) added an 'export CC=gcc-8' in an
attempt to build with a more modern compiler, but to no avail.

Set MAKEFLAGS with CC based on the $CC environment variable, so 'make'
will run the "right" compiler.  The Xcode 10.1 macOS image on Travis
CI already contains the gcc@8 package from Homebrew, but we have to
'brew link' it first to be able to use it.

So with this patch our build jobs will build Git with the following
compiler versions:

  linux-clang: clang version 5.0.0 (tags/RELEASE_500/final)
  linux-gcc:   gcc-8 (Ubuntu 8.1.0-5ubuntu1~14.04) 8.1.0

  osx-clang: Apple LLVM version 10.0.0 (clang-1000.11.45.5)
  osx-gcc:   gcc-8 (Homebrew GCC 8.2.0) 8.2.0

  GETTEXT_POISON: gcc (Ubuntu 4.8.4-2ubuntu1~14.04.3) 4.8.4

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
---
 ci/install-dependencies.sh |  5 +++++
 ci/lib-travisci.sh         | 15 ++++++++++++---
 2 files changed, 17 insertions(+), 3 deletions(-)

Comments

Johannes Schindelin Jan. 17, 2019, 1:44 p.m. UTC | #1
Hi Gábor,

On Thu, 17 Jan 2019, SZEDER Gábor wrote:

> Our 'Makefile' hardcodes the compiler to build Git as 'CC = cc'.  This
> CC variable can be overridden from the command line, i.e. 'make
> CC=gcc-X.Y' will build with that particular GCC version, but not from
> the environment, i.e. 'CC=gcc-X.Y make' will still build with whatever
> 'cc' happens to be on the platform.
> 
> Our build jobs on Travis CI are badly affected by this.  In the build
> matrix we have dedicated build jobs to build Git with GCC and Clang
> both on Linux and macOS from the very beginning (522354d70f (Add
> Travis CI support, 2015-11-27)).  Alas, this never really worked as
> supposed to, because Travis CI specifies the compiler for those build
> jobs as 'export CC=gcc' and 'export CC=clang' (which works fine for
> projects built with './configure && make').  Consequently, our
> 'linux-clang' build job has always used GCC, because that's where 'cc'
> points at in Travis CI's Linux images, while the 'osx-gcc' build job
> has always used Clang.  Furthermore, 37fa4b3c78 (travis-ci: run gcc-8
> on linux-gcc jobs, 2018-05-19) added an 'export CC=gcc-8' in an
> attempt to build with a more modern compiler, but to no avail.
> 
> Set MAKEFLAGS with CC based on the $CC environment variable, so 'make'
> will run the "right" compiler.  The Xcode 10.1 macOS image on Travis
> CI already contains the gcc@8 package from Homebrew, but we have to
> 'brew link' it first to be able to use it.
> 
> So with this patch our build jobs will build Git with the following
> compiler versions:
> 
>   linux-clang: clang version 5.0.0 (tags/RELEASE_500/final)
>   linux-gcc:   gcc-8 (Ubuntu 8.1.0-5ubuntu1~14.04) 8.1.0
> 
>   osx-clang: Apple LLVM version 10.0.0 (clang-1000.11.45.5)
>   osx-gcc:   gcc-8 (Homebrew GCC 8.2.0) 8.2.0
> 
>   GETTEXT_POISON: gcc (Ubuntu 4.8.4-2ubuntu1~14.04.3) 4.8.4
> 
> Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>

Nicely explained and implemented. Thank you!

Apart from the single typo I stumbled over (quite honestly, I do not
really care about typos, I just point them out when I see them to beat
Eric to it), I have just one more wish: I would *love* to see a Travis
run.

Surely you opened a PR at https://github.com/git/git. Oh wait, I can look
for it! But no, there does not seem to be one. So probably in your fork,
https://github.com/szeder/git. No PRs. Ah, but there are branches! 60 of
them. And yes, I guess I found a Travis build:

	https://travis-ci.org/szeder/git/builds/480654256

But no, the associated branch does not look like it reflects this patch
series...

In any case, this series is:

	Reviewed-by: Johannes Schindelin <johannes.schindelin@gmx.de>

Ciao,
Dscho


> ---
>  ci/install-dependencies.sh |  5 +++++
>  ci/lib-travisci.sh         | 15 ++++++++++++---
>  2 files changed, 17 insertions(+), 3 deletions(-)
> 
> diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
> index 06c3546e1e..dc719876bb 100755
> --- a/ci/install-dependencies.sh
> +++ b/ci/install-dependencies.sh
> @@ -40,6 +40,11 @@ osx-clang|osx-gcc)
>  	brew install git-lfs gettext
>  	brew link --force gettext
>  	brew install caskroom/cask/perforce
> +	case "$jobname" in
> +	osx-gcc)
> +		brew link gcc@8
> +		;;
> +	esac
>  	;;
>  StaticAnalysis)
>  	sudo apt-get -q update
> diff --git a/ci/lib-travisci.sh b/ci/lib-travisci.sh
> index 69dff4d1ec..a479613a57 100755
> --- a/ci/lib-travisci.sh
> +++ b/ci/lib-travisci.sh
> @@ -99,12 +99,14 @@ export DEFAULT_TEST_TARGET=prove
>  export GIT_PROVE_OPTS="--timer --jobs 3 --state=failed,slow,save"
>  export GIT_TEST_OPTS="--verbose-log -x --immediate"
>  export GIT_TEST_CLONE_2GB=YesPlease
> -if [ "$jobname" = linux-gcc ]; then
> -	export CC=gcc-8
> -fi
>  
>  case "$jobname" in
>  linux-clang|linux-gcc)
> +	if [ "$jobname" = linux-gcc ]
> +	then
> +		export CC=gcc-8
> +	fi
> +
>  	export GIT_TEST_HTTPD=YesPlease
>  
>  	# The Linux build installs the defined dependency versions below.
> @@ -118,6 +120,11 @@ linux-clang|linux-gcc)
>  	export PATH="$GIT_LFS_PATH:$P4_PATH:$PATH"
>  	;;
>  osx-clang|osx-gcc)
> +	if [ "$jobname" = osx-gcc ]
> +	then
> +		export CC=gcc-8
> +	fi
> +
>  	# t9810 occasionally fails on Travis CI OS X
>  	# t9816 occasionally fails with "TAP out of sequence errors" on
>  	# Travis CI OS X
> @@ -127,3 +134,5 @@ GIT_TEST_GETTEXT_POISON)
>  	export GIT_TEST_GETTEXT_POISON=YesPlease
>  	;;
>  esac
> +
> +export MAKEFLAGS="CC=${CC:-cc}"
> -- 
> 2.20.1.499.gf60de1223c
> 
>
SZEDER Gábor Jan. 17, 2019, 2:56 p.m. UTC | #2
On Thu, Jan 17, 2019 at 02:44:59PM +0100, Johannes Schindelin wrote:
> Hi Gábor,
> 
> On Thu, 17 Jan 2019, SZEDER Gábor wrote:
> 
> > Our 'Makefile' hardcodes the compiler to build Git as 'CC = cc'.  This
> > CC variable can be overridden from the command line, i.e. 'make
> > CC=gcc-X.Y' will build with that particular GCC version, but not from
> > the environment, i.e. 'CC=gcc-X.Y make' will still build with whatever
> > 'cc' happens to be on the platform.
> > 
> > Our build jobs on Travis CI are badly affected by this.  In the build
> > matrix we have dedicated build jobs to build Git with GCC and Clang
> > both on Linux and macOS from the very beginning (522354d70f (Add
> > Travis CI support, 2015-11-27)).  Alas, this never really worked as
> > supposed to, because Travis CI specifies the compiler for those build
> > jobs as 'export CC=gcc' and 'export CC=clang' (which works fine for
> > projects built with './configure && make').  Consequently, our
> > 'linux-clang' build job has always used GCC, because that's where 'cc'
> > points at in Travis CI's Linux images, while the 'osx-gcc' build job
> > has always used Clang.  Furthermore, 37fa4b3c78 (travis-ci: run gcc-8
> > on linux-gcc jobs, 2018-05-19) added an 'export CC=gcc-8' in an
> > attempt to build with a more modern compiler, but to no avail.
> > 
> > Set MAKEFLAGS with CC based on the $CC environment variable, so 'make'
> > will run the "right" compiler.  The Xcode 10.1 macOS image on Travis
> > CI already contains the gcc@8 package from Homebrew, but we have to
> > 'brew link' it first to be able to use it.
> > 
> > So with this patch our build jobs will build Git with the following
> > compiler versions:
> > 
> >   linux-clang: clang version 5.0.0 (tags/RELEASE_500/final)
> >   linux-gcc:   gcc-8 (Ubuntu 8.1.0-5ubuntu1~14.04) 8.1.0
> > 
> >   osx-clang: Apple LLVM version 10.0.0 (clang-1000.11.45.5)
> >   osx-gcc:   gcc-8 (Homebrew GCC 8.2.0) 8.2.0
> > 
> >   GETTEXT_POISON: gcc (Ubuntu 4.8.4-2ubuntu1~14.04.3) 4.8.4
> > 
> > Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
> 
> Nicely explained and implemented. Thank you!
> 
> Apart from the single typo I stumbled over (quite honestly, I do not
> really care about typos, I just point them out when I see them to beat
> Eric to it)

Heh ;)

> I have just one more wish: I would *love* to see a Travis
> run.
> 
> Surely you opened a PR at https://github.com/git/git. Oh wait, I can look
> for it! But no, there does not seem to be one.

Nope, "all pull requests are ignored", so I don't bother.

> So probably in your fork,
> https://github.com/szeder/git. No PRs. Ah, but there are branches! 60 of
> them.

Oh, wow, 60, that really needs to be cleaned up and all the already
upstreamed, obsolete, or test branches removed.  And, more
importantly, my WIP branches finished and finally upstreamed...
Sigh, maybe someday...

> And yes, I guess I found a Travis build:
> 
> 	https://travis-ci.org/szeder/git/builds/480654256

That would be the right build, but it was the result of a rebase
squashing a fixup commit and editing commit messages while leaving the
resulting tree intact, and then 9cc2c76f5e (travis-ci: record and skip
successfully built trees, 2017-12-31) kicked in and optimized away the
build.

Anyway, I deleted the branch's cache and restarted the build, it will
most likely be finished by the time I send this email.

> But no, the associated branch does not look like it reflects this patch
> series...

That's the right branch, though, except that it contains a debug patch
on top (that I didn't submitted) that shows the used compiler and its
version from within the Makefile, to make sure that the build jobs do
indeed use the compilers that I think they should be using.

The other patches on that branch are the same that were submitted in
this patch series.


Here are the most interesting bits of the problematic build jobs, i.e.
the compiler versions from that debug patch:

  linux-clang:  https://travis-ci.org/szeder/git/jobs/480654257#L824
  osx-gcc:      https://travis-ci.org/szeder/git/jobs/480654264#L824


> In any case, this series is:
> 
> 	Reviewed-by: Johannes Schindelin <johannes.schindelin@gmx.de>

Thanks.
Johannes Schindelin Jan. 18, 2019, 8:40 a.m. UTC | #3
Hi Gábor,

On Thu, 17 Jan 2019, SZEDER Gábor wrote:

> Here are the most interesting bits of the problematic build jobs, i.e.
> the compiler versions from that debug patch:
> 
>   linux-clang:  https://travis-ci.org/szeder/git/jobs/480654257#L824
>   osx-gcc:      https://travis-ci.org/szeder/git/jobs/480654264#L824

Thanks!
Dscho

Patch
diff mbox series

diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
index 06c3546e1e..dc719876bb 100755
--- a/ci/install-dependencies.sh
+++ b/ci/install-dependencies.sh
@@ -40,6 +40,11 @@  osx-clang|osx-gcc)
 	brew install git-lfs gettext
 	brew link --force gettext
 	brew install caskroom/cask/perforce
+	case "$jobname" in
+	osx-gcc)
+		brew link gcc@8
+		;;
+	esac
 	;;
 StaticAnalysis)
 	sudo apt-get -q update
diff --git a/ci/lib-travisci.sh b/ci/lib-travisci.sh
index 69dff4d1ec..a479613a57 100755
--- a/ci/lib-travisci.sh
+++ b/ci/lib-travisci.sh
@@ -99,12 +99,14 @@  export DEFAULT_TEST_TARGET=prove
 export GIT_PROVE_OPTS="--timer --jobs 3 --state=failed,slow,save"
 export GIT_TEST_OPTS="--verbose-log -x --immediate"
 export GIT_TEST_CLONE_2GB=YesPlease
-if [ "$jobname" = linux-gcc ]; then
-	export CC=gcc-8
-fi
 
 case "$jobname" in
 linux-clang|linux-gcc)
+	if [ "$jobname" = linux-gcc ]
+	then
+		export CC=gcc-8
+	fi
+
 	export GIT_TEST_HTTPD=YesPlease
 
 	# The Linux build installs the defined dependency versions below.
@@ -118,6 +120,11 @@  linux-clang|linux-gcc)
 	export PATH="$GIT_LFS_PATH:$P4_PATH:$PATH"
 	;;
 osx-clang|osx-gcc)
+	if [ "$jobname" = osx-gcc ]
+	then
+		export CC=gcc-8
+	fi
+
 	# t9810 occasionally fails on Travis CI OS X
 	# t9816 occasionally fails with "TAP out of sequence errors" on
 	# Travis CI OS X
@@ -127,3 +134,5 @@  GIT_TEST_GETTEXT_POISON)
 	export GIT_TEST_GETTEXT_POISON=YesPlease
 	;;
 esac
+
+export MAKEFLAGS="CC=${CC:-cc}"