mbox series

[v2,00/13] Offer to run CI/PR builds in Azure Pipelines

Message ID pull.31.v2.git.gitgitgadget@gmail.com (mailing list archive)
Headers show
Series Offer to run CI/PR builds in Azure Pipelines | expand

Message

Johannes Schindelin via GitGitGadget Oct. 15, 2018, 10:11 a.m. UTC
For a long time already, we have Git's source code continuously tested via
Travis CI, see e.g. https://travis-ci.org/git/git/builds/421738884. It has
served us well, and more and more developers actually pay attention and
benefit from the testing this gives us.

It is also an invaluable tool for contributors who can validate their code
contributions via PRs on GitHub, e.g. to verify that their tests do actually
run on macOS (i.e. with the BSD family of Unix tools instead of the GNU
one).

The one sad part about this is the Windows support. Travis lacks it, and we
work around that by using Azure Pipelines (the CI part of Azure DevOps,
formerly known as Visual Studio Team Services) indirectly: one phase in
Travis would trigger a build, wait for its log, and then paste that log.

As Git's Windows builds (and tests!) take quite a bit of time, Travis often
timed out, or somehow the trigger did not work, and for security reasons
(the Windows builds are performed in a private pool of containers), the
Windows builds are completely disabled for Pull Requests on GitHub.

One might ask why we did not use Azure Pipelines directly. There were a
couple of reasons for that:

 * most notably, Azure Pipelines' build logs could not be viewed
   anonymously,
 * while Azure Pipelines had Linux and Windows agents, it lacked macOS
   agents,
 * etc

The main two reasons no longer apply: macOS agents are available now
[https://docs.microsoft.com/en-us/azure/devops/release-notes/2018/jul-10-vsts]
, and there is a limited preview of "public projects"
[https://blogs.msdn.microsoft.com/devops/2018/04/27/vsts-public-projects-limited-preview/]
, i.e. it is possible to configure a Azure Pipelines project so that anybody
can view the logs.

I had secured such a public project for Git for Windows already, and I
recently also got one for Git. For now, the latter is hooked up with my
personal git.git fork on GitHub, but it is my hope that I convince y'all
that these Azure Pipelines builds are a good idea, and then hook it up with 
https://github.com/git/git.

As a special treat, this patch series adds the ability to present the
outcome of Git's test suite as JUnit-style .xml files. This allows the Azure
Pipelines build to present fun diagrams, trends, and makes it a lot easier
to drill down to test failures than before. See for example 
https://dev.azure.com/git/git/_build/results?buildId=113&view=ms.vss-test-web.test-result-details
[https://dev.azure.com/git/git/_build/results?buildId=113&view=ms.vss-test-web.test-result-details] 
(you can click on the label of the failed test, and then see the detailed
output in the right pane).

This patch series took way more time than I had originally planned, but I
think that in particular the advanced display of the test results was worth
it. Please let me know what you think about this.

Changes since v1:

 * Removed a superfluous eval.
 * Added the commit that fixes the Travis PR builds targeting master that 
   just happens to be tagged (see 
   https://travis-ci.org/git/git/jobs/424276413 for an incorrectly-skipped
   build).
 * The commit messages and the cover letter now reflect the name change from
   Visual Studio Team Services to Azure DevOps (and in particular, Azure
   Pipelines for the automated builds).
 * Now we're using test_atexit (which we introduced for that purpose)
   instead of hard-coding kill_p4d and stop_git_daemon.
 * The build should now also succeed for Pull Requests (where secret
   variables are not available, for security reasons, and as a consequence
   the file share cannot be mounted).
 * The shell scripted parts now use proper && chains.

Johannes Schindelin (13):
  ci: rename the library of common functions
  ci/lib.sh: encapsulate Travis-specific things
  test-date: add a subcommand to measure times in shell scripts
  tests: optionally write results as JUnit-style .xml
  ci/lib.sh: add support for Azure Pipelines
  Add a build definition for Azure DevOps
  tests: introduce `test_atexit`
  git-daemon: use `test_atexit` in the tests
  git-p4: use `test_atexit` to kill the daemon
  tests: include detailed trace logs with --write-junit-xml upon failure
  tests: record more stderr with --write-junit-xml in case of failure
  README: add a build badge (status of the Azure Pipelines build)
  travis: fix skipping tagged releases

 README.md                                  |   2 +
 azure-pipelines.yml                        | 319 +++++++++++++++++++++
 ci/install-dependencies.sh                 |   5 +-
 ci/{lib-travisci.sh => lib.sh}             |  67 ++++-
 ci/mount-fileshare.sh                      |  26 ++
 ci/print-test-failures.sh                  |   4 +-
 ci/run-build-and-tests.sh                  |   2 +-
 ci/run-linux32-docker.sh                   |   2 +-
 ci/run-static-analysis.sh                  |   2 +-
 ci/run-windows-build.sh                    |   2 +-
 ci/test-documentation.sh                   |   3 +-
 t/.gitignore                               |   1 +
 t/helper/test-date.c                       |  12 +
 t/interop/i5500-git-daemon.sh              |   1 -
 t/lib-git-daemon.sh                        |   3 +-
 t/lib-git-p4.sh                            |  10 +-
 t/t0000-basic.sh                           |  20 ++
 t/t5570-git-daemon.sh                      |   1 -
 t/t9800-git-p4-basic.sh                    |   4 -
 t/t9801-git-p4-branch.sh                   |   4 -
 t/t9802-git-p4-filetype.sh                 |   4 -
 t/t9803-git-p4-shell-metachars.sh          |   4 -
 t/t9804-git-p4-label.sh                    |   4 -
 t/t9805-git-p4-skip-submit-edit.sh         |   4 -
 t/t9806-git-p4-options.sh                  |   5 -
 t/t9807-git-p4-submit.sh                   |   4 -
 t/t9808-git-p4-chdir.sh                    |   4 -
 t/t9809-git-p4-client-view.sh              |   4 -
 t/t9810-git-p4-rcs.sh                      |   4 -
 t/t9811-git-p4-label-import.sh             |   5 -
 t/t9812-git-p4-wildcards.sh                |   4 -
 t/t9813-git-p4-preserve-users.sh           |   4 -
 t/t9814-git-p4-rename.sh                   |   4 -
 t/t9815-git-p4-submit-fail.sh              |   4 -
 t/t9816-git-p4-locked.sh                   |   4 -
 t/t9817-git-p4-exclude.sh                  |   4 -
 t/t9818-git-p4-block.sh                    |   4 -
 t/t9819-git-p4-case-folding.sh             |   4 -
 t/t9820-git-p4-editor-handling.sh          |   4 -
 t/t9821-git-p4-path-variations.sh          |   4 -
 t/t9822-git-p4-path-encoding.sh            |   4 -
 t/t9823-git-p4-mock-lfs.sh                 |   4 -
 t/t9824-git-p4-git-lfs.sh                  |   4 -
 t/t9825-git-p4-handle-utf16-without-bom.sh |   4 -
 t/t9826-git-p4-keep-empty-commits.sh       |   4 -
 t/t9827-git-p4-change-filetype.sh          |   4 -
 t/t9828-git-p4-map-user.sh                 |   4 -
 t/t9829-git-p4-jobs.sh                     |   4 -
 t/t9830-git-p4-symlink-dir.sh              |   4 -
 t/t9831-git-p4-triggers.sh                 |   4 -
 t/t9832-unshelve.sh                        |   4 -
 t/t9833-errors.sh                          |   5 -
 t/test-lib-functions.sh                    |  29 ++
 t/test-lib.sh                              | 140 +++++++++
 54 files changed, 616 insertions(+), 174 deletions(-)
 create mode 100644 azure-pipelines.yml
 rename ci/{lib-travisci.sh => lib.sh} (61%)
 create mode 100755 ci/mount-fileshare.sh


base-commit: 5a0cc8aca797dbd7d2be3b67458ff880ed45cddf
Published-As: https://github.com/gitgitgadget/git/releases/tags/pr-31%2Fdscho%2Fvsts-ci-v2
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-31/dscho/vsts-ci-v2
Pull-Request: https://github.com/gitgitgadget/git/pull/31

Range-diff vs v1:

  1:  858b80bfcf !  1:  c963184510 ci: rename the library of common functions
     @@ -5,8 +5,8 @@
          The name is hard-coded to reflect that we use Travis CI for continuous
          testing.
      
     -    In the next commits, we will extend this to be able use Visual Studio
     -    Team Services, too.
     +    In the next commits, we will extend this to be able use Azure DevOps,
     +    too.
      
          So let's adjust the name to make it more generic.
      
  2:  18e6beec5f !  2:  815152e0f5 ci/lib.sh: encapsulate Travis-specific things
     @@ -2,8 +2,9 @@
      
          ci/lib.sh: encapsulate Travis-specific things
      
     -    The upcoming patches will allow building git.git via VSTS CI, where
     -    variable names and URLs look a bit different than in Travis CI.
     +    The upcoming patches will allow building git.git via Azure Pipelines
     +    (i.e. Azure DevOps' Continuous Integration), where variable names and
     +    URLs look a bit different than in Travis CI.
      
          Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
      
     @@ -16,7 +17,7 @@
       	# brew install gnu-time
      -	brew install git-lfs gettext
      +	test -z "$BREW_INSTALL_PACKAGES" ||
     -+	eval brew install $BREW_INSTALL_PACKAGES
     ++	brew install $BREW_INSTALL_PACKAGES
       	brew link --force gettext
       	brew install caskroom/cask/perforce
       	;;
     @@ -27,7 +28,7 @@
      @@
       # Library of functions shared by all CI scripts
       
     -+if test -n "$TRAVIS_COMMIT"
     ++if test true = "$TRAVIS"
      +then
      +	# We are running within Travis CI
      +	CI_BRANCH="$TRAVIS_BRANCH"
  3:  064db77669 =  3:  52337f1875 test-date: add a subcommand to measure times in shell scripts
  4:  2b5d678594 !  4:  cf4c5ae470 tests: optionally write results as JUnit-style .xml
     @@ -3,7 +3,7 @@
          tests: optionally write results as JUnit-style .xml
      
          This will come in handy when publishing the results of Git's test suite
     -    during an automated VSTS CI run.
     +    during an automated Azure DevOps run.
      
          Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
      
  5:  127dfcfb09 !  5:  486d1d2518 ci/lib.sh: add support for VSTS CI
     @@ -1,6 +1,6 @@
      Author: Johannes Schindelin <johannes.schindelin@gmx.de>
      
     -    ci/lib.sh: add support for VSTS CI
     +    ci/lib.sh: add support for Azure Pipelines
      
          This patch introduces a conditional arm that defines some environment
          variables and a function that displays the URL given the job id (to
     @@ -17,7 +17,7 @@
       	export GIT_TEST_OPTS="--verbose-log -x --immediate"
      +elif test -n "$SYSTEM_TASKDEFINITIONSURI"
      +then
     -+	# We are running in VSTS CI
     ++	# We are running in Azure Pipelines
      +	CI_BRANCH="$BUILD_SOURCEBRANCH"
      +	CI_COMMIT="$BUILD_SOURCEVERSION"
      +	CI_JOB_ID="$BUILD_BUILDID"
     @@ -29,7 +29,7 @@
      +
      +	# use a subdirectory of the cache dir (because the file share is shared
      +	# among *all* phases)
     -+	cache_dir="$HOME/vsts-cache/$SYSTEM_PHASENAME"
     ++	cache_dir="$HOME/test-cache/$SYSTEM_PHASENAME"
      +
      +	url_for_job_id () {
      +		echo "$SYSTEM_TASKDEFINITIONSURI$SYSTEM_TEAMPROJECT/_build/results?buildId=$1"
  6:  ab66060c57 !  6:  1a22efe849 Add a build definition for VSTS
     @@ -1,8 +1,8 @@
      Author: Johannes Schindelin <johannes.schindelin@gmx.de>
      
     -    Add a build definition for VSTS
     +    Add a build definition for Azure DevOps
      
     -    This commit adds a .vsts-ci.yml which is Visual Studio Team Services'
     +    This commit adds an azure-pipelines.yml file which is Azure DevOps'
          equivalent to Travis CI's .travis.yml.
      
          To make things a bit easier to understand, we refrain from using the
     @@ -19,10 +19,10 @@
      
          Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
      
     -diff --git a/.vsts-ci.yml b/.vsts-ci.yml
     +diff --git a/azure-pipelines.yml b/azure-pipelines.yml
      new file mode 100644
      --- /dev/null
     -+++ b/.vsts-ci.yml
     ++++ b/azure-pipelines.yml
      @@
      +resources:
      +- repo: self
     @@ -36,12 +36,12 @@
      +    name: Hosted Ubuntu 1604
      +  steps:
      +  - bash: |
     -+       ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/vsts-cache gitfileshare "$(gitfileshare.pwd)" "$HOME/vsts-cache" || exit 1
     ++       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
      +
     -+       sudo apt-get update
     -+       sudo apt-get -y install git gcc make libssl-dev libcurl4-openssl-dev libexpat-dev tcl tk gettext git-email zlib1g-dev apache2-bin
     ++       sudo apt-get update &&
     ++       sudo apt-get -y install git gcc make libssl-dev libcurl4-openssl-dev libexpat-dev tcl tk gettext git-email zlib1g-dev apache2-bin &&
      +
     -+       export CC=clang
     ++       export CC=clang || exit 1
      +
      +       ci/install-dependencies.sh
      +       ci/run-build-and-tests.sh || {
     @@ -49,8 +49,10 @@
      +           exit 1
      +       }
      +
     -+       sudo umount "$HOME/vsts-cache" || exit 1
     ++       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || exit 1
      +    displayName: 'ci/run-build-and-tests.sh'
     ++    env:
     ++      GITFILESHAREPWD: $(gitfileshare.pwd)
      +  - task: PublishTestResults@2
      +    displayName: 'Publish Test Results **/TEST-*.xml'
      +    inputs:
     @@ -67,10 +69,10 @@
      +    name: Hosted Ubuntu 1604
      +  steps:
      +  - bash: |
     -+       ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/vsts-cache gitfileshare "$(gitfileshare.pwd)" "$HOME/vsts-cache" || exit 1
     ++       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
      +
     -+       sudo apt-get update
     -+       sudo apt-get -y install git gcc make libssl-dev libcurl4-openssl-dev libexpat-dev tcl tk gettext git-email zlib1g-dev apache2-bin
     ++       sudo apt-get update &&
     ++       sudo apt-get -y install git gcc make libssl-dev libcurl4-openssl-dev libexpat-dev tcl tk gettext git-email zlib1g-dev apache2-bin || exit 1
      +
      +       ci/install-dependencies.sh
      +       ci/run-build-and-tests.sh || {
     @@ -78,8 +80,10 @@
      +           exit 1
      +       }
      +
     -+       sudo umount "$HOME/vsts-cache" || exit 1
     ++       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || exit 1
      +    displayName: 'ci/run-build-and-tests.sh'
     ++    env:
     ++      GITFILESHAREPWD: $(gitfileshare.pwd)
      +  - task: PublishTestResults@2
      +    displayName: 'Publish Test Results **/TEST-*.xml'
      +    inputs:
     @@ -96,7 +100,7 @@
      +    name: Hosted macOS
      +  steps:
      +  - bash: |
     -+       ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/vsts-cache gitfileshare "$(gitfileshare.pwd)" "$HOME/vsts-cache" || exit 1
     ++       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
      +
      +       export CC=clang
      +
     @@ -106,8 +110,10 @@
      +           exit 1
      +       }
      +
     -+       umount "$HOME/vsts-cache" || exit 1
     ++       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || umount "$HOME/test-cache" || exit 1
      +    displayName: 'ci/run-build-and-tests.sh'
     ++    env:
     ++      GITFILESHAREPWD: $(gitfileshare.pwd)
      +  - task: PublishTestResults@2
      +    displayName: 'Publish Test Results **/TEST-*.xml'
      +    inputs:
     @@ -124,7 +130,7 @@
      +    name: Hosted macOS
      +  steps:
      +  - bash: |
     -+       ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/vsts-cache gitfileshare "$(gitfileshare.pwd)" "$HOME/vsts-cache" || exit 1
     ++       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
      +
      +       ci/install-dependencies.sh
      +       ci/run-build-and-tests.sh || {
     @@ -132,8 +138,10 @@
      +           exit 1
      +       }
      +
     -+       umount "$HOME/vsts-cache" || exit 1
     ++       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || umount "$HOME/test-cache" || exit 1
      +    displayName: 'ci/run-build-and-tests.sh'
     ++    env:
     ++      GITFILESHAREPWD: $(gitfileshare.pwd)
      +  - task: PublishTestResults@2
      +    displayName: 'Publish Test Results **/TEST-*.xml'
      +    inputs:
     @@ -150,20 +158,22 @@
      +    name: Hosted Ubuntu 1604
      +  steps:
      +  - bash: |
     -+       ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/vsts-cache gitfileshare "$(gitfileshare.pwd)" "$HOME/vsts-cache" || exit 1
     ++       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
      +
     -+       sudo apt-get update
     -+       sudo apt-get -y install git gcc make libssl-dev libcurl4-openssl-dev libexpat-dev tcl tk gettext git-email zlib1g-dev
     ++       sudo apt-get update &&
     ++       sudo apt-get -y install git gcc make libssl-dev libcurl4-openssl-dev libexpat-dev tcl tk gettext git-email zlib1g-dev &&
      +
     -+       export jobname=GETTEXT_POISON
     ++       export jobname=GETTEXT_POISON || exit 1
      +
      +       ci/run-build-and-tests.sh || {
      +           ci/print-test-failures.sh
      +           exit 1
      +       }
      +
     -+       sudo umount "$HOME/vsts-cache" || exit 1
     ++       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || exit 1
      +    displayName: 'ci/run-build-and-tests.sh'
     ++    env:
     ++      GITFILESHAREPWD: $(gitfileshare.pwd)
      +  - task: PublishTestResults@2
      +    displayName: 'Publish Test Results **/TEST-*.xml'
      +    inputs:
     @@ -184,8 +194,10 @@
      +       # Helper to check the error level of the latest command (exit with error when appropriate)
      +       function c() { if (!$?) { exit(1) } }
      +
     -+       net use s: \\gitfileshare.file.core.windows.net\vsts-cache "$(gitfileshare.pwd)" /user:AZURE\gitfileshare /persistent:no; c
     -+       cmd /c mklink /d "$(Build.SourcesDirectory)\vsts-cache" S:\; c
     ++       if ("$GITFILESHAREPWD" -ne "" -and "$GITFILESHAREPWD" -ne "`$`(gitfileshare.pwd)") {
     ++         net use s: \\gitfileshare.file.core.windows.net\test-cache "$GITFILESHAREPWD" /user:AZURE\gitfileshare /persistent:no; c
     ++         cmd /c mklink /d "$(Build.SourcesDirectory)\test-cache" S:\; c
     ++       }
      +
      +       # Add build agent's MinGit to PATH
      +       $env:PATH = $env:AGENT_HOMEDIRECTORY +"\externals\\git\cmd;" +$env:PATH
     @@ -215,7 +227,7 @@
      +
      +       # Initialize Git for Windows' SDK
      +       $sdk_path = "$(Build.SourcesDirectory)\git-sdk-64"
     -+       init "$sdk_path" "https://git-for-windows.visualstudio.com/_git/git-sdk-64" 0
     ++       init "$sdk_path" "https://dev.azure.com/git-for-windows/git-sdk-64/_git/git-sdk-64" 0
      +       init usr\src\build-extra https://github.com/git-for-windows/build-extra 1
      +
      +       cd "$(Build.SourcesDirectory)"; c
     @@ -227,14 +239,19 @@
      +
      +         make -j10 DEVELOPER=1 NO_PERL=1 || exit 1
      +         NO_PERL=1 NO_SVN_TESTS=1 GIT_TEST_OPTS=\"--quiet --write-junit-xml\" time make -j15 -k DEVELOPER=1 test || {
     -+           NO_PERL=1 NO_SVN_TESTS=1 GIT_TEST_OPTS=\"-i -v -x\" make -k failed; exit 1
     ++           NO_PERL=1 NO_SVN_TESTS=1 GIT_TEST_OPTS=\"-i -v -x\" make -k -C t failed; exit 1
      +         }
      +
      +         save_good_tree
      +       "@
     ++       c
      +
     -+       cmd /c rmdir "$(Build.SourcesDirectory)\vsts-cache"
     ++       if ("$GITFILESHAREPWD" -ne "" -and "$GITFILESHAREPWD" -ne "`$`(gitfileshare.pwd)") {
     ++         cmd /c rmdir "$(Build.SourcesDirectory)\test-cache"
     ++       }
      +    displayName: 'build & test'
     ++    env:
     ++      GITFILESHAREPWD: $(gitfileshare.pwd)
      +  - task: PublishTestResults@2
      +    displayName: 'Publish Test Results **/TEST-*.xml'
      +    inputs:
     @@ -251,28 +268,30 @@
      +    name: Hosted Ubuntu 1604
      +  steps:
      +  - bash: |
     -+       ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/vsts-cache gitfileshare "$(gitfileshare.pwd)" "$HOME/vsts-cache" || exit 1
     ++       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
      +
     -+       sudo apt-get update
     ++       sudo apt-get update &&
      +       sudo apt-get -y install \
      +           apt-transport-https \
      +           ca-certificates \
      +           curl \
     -+           software-properties-common
     -+       curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
     ++           software-properties-common &&
     ++       curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add - &&
      +       sudo add-apt-repository \
      +          "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
      +          $(lsb_release -cs) \
     -+          stable"
     -+       sudo apt-get update
     -+       sudo apt-get -y install docker-ce
     ++          stable" &&
     ++       sudo apt-get update &&
     ++       sudo apt-get -y install docker-ce &&
      +
      +       sudo AGENT_OS="$AGENT_OS" BUILD_BUILDNUMBER="$BUILD_BUILDNUMBER" BUILD_REPOSITORY_URI="$BUILD_REPOSITORY_URI" BUILD_SOURCEBRANCH="$BUILD_SOURCEBRANCH" BUILD_SOURCEVERSION="$BUILD_SOURCEVERSION" SYSTEM_PHASENAME="$SYSTEM_PHASENAME" SYSTEM_TASKDEFINITIONSURI="$SYSTEM_TASKDEFINITIONSURI" SYSTEM_TEAMPROJECT="$SYSTEM_TEAMPROJECT" CC=$CC MAKEFLAGS=-j3 bash -lxc ci/run-linux32-docker.sh || exit 1
      +
      +       sudo chmod a+r t/out/TEST-*.xml
      +
     -+       sudo umount "$HOME/vsts-cache" || exit 1
     ++       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || exit 1
      +    displayName: 'ci/run-linux32-docker.sh'
     ++    env:
     ++      GITFILESHAREPWD: $(gitfileshare.pwd)
      +  - task: PublishTestResults@2
      +    displayName: 'Publish Test Results **/TEST-*.xml'
      +    inputs:
     @@ -289,17 +308,19 @@
      +    name: Hosted Ubuntu 1604
      +  steps:
      +  - bash: |
     -+       ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/vsts-cache gitfileshare "$(gitfileshare.pwd)" "$HOME/vsts-cache" || exit 1
     ++       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
      +
     -+       sudo apt-get update
     -+       sudo apt-get install -y coccinelle
     ++       sudo apt-get update &&
     ++       sudo apt-get install -y coccinelle &&
      +
     -+       export jobname=StaticAnalysis
     ++       export jobname=StaticAnalysis &&
      +
      +       ci/run-static-analysis.sh || exit 1
      +
     -+       sudo umount "$HOME/vsts-cache" || exit 1
     ++       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || exit 1
      +    displayName: 'ci/run-static-analysis.sh'
     ++    env:
     ++      GITFILESHAREPWD: $(gitfileshare.pwd)
      +
      +- phase: documentation
      +  displayName: Documentation
     @@ -308,18 +329,20 @@
      +    name: Hosted Ubuntu 1604
      +  steps:
      +  - bash: |
     -+       ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/vsts-cache gitfileshare "$(gitfileshare.pwd)" "$HOME/vsts-cache" || exit 1
     ++       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
      +
     -+       sudo apt-get update
     -+       sudo apt-get install -y asciidoc xmlto asciidoctor
     ++       sudo apt-get update &&
     ++       sudo apt-get install -y asciidoc xmlto asciidoctor &&
      +
     -+       export ALREADY_HAVE_ASCIIDOCTOR=yes.
     -+       export jobname=Documentation
     ++       export ALREADY_HAVE_ASCIIDOCTOR=yes. &&
     ++       export jobname=Documentation &&
      +
      +       ci/test-documentation.sh || exit 1
      +
     -+       sudo umount "$HOME/vsts-cache" || exit 1
     ++       test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || exit 1
      +    displayName: 'ci/test-documentation.sh'
     ++    env:
     ++      GITFILESHAREPWD: $(gitfileshare.pwd)
      
      diff --git a/ci/mount-fileshare.sh b/ci/mount-fileshare.sh
      new file mode 100755
  -:  ---------- >  7:  12d6137f8d tests: introduce `test_atexit`
  -:  ---------- >  8:  3bb226b79b git-daemon: use `test_atexit` in the tests
  -:  ---------- >  9:  3e2193a73d git-p4: use `test_atexit` to kill the daemon
  7:  942bf423a4 ! 10:  ae3c42519a tests: include detailed trace logs with --write-junit-xml upon failure
     @@ -14,9 +14,9 @@
          almost all of those cases: only when a test fails do we have a way to
          include the trace.
      
     -    So let's do something different in VSTS: let's run all the tests with
     -    `--quiet` first, and only if a failure is encountered, try to trace the
     -    commands as they are executed.
     +    So let's do something different when using Azure DevOps: let's run all
     +    the tests with `--quiet` first, and only if a failure is encountered,
     +    try to trace the commands as they are executed.
      
          Of course, we cannot turn on `--verbose-log` after the fact. So let's
          just re-run the test with all the same options, adding `--verbose-log`.
     @@ -25,15 +25,10 @@
          Note: there is an off chance that re-running the test in verbose mode
          "fixes" the failures (and this does happen from time to time!). That is
          a possibility we should be able to live with. Ideally, we would label
     -    this as "Passed upon rerun", and that outcome even exists on VSTS, but
     -    it is not available when using the JUnit XML format for now:
     -    https://github.com/Microsoft/vsts-agent/blob/master/src/Agent.Worker/TestResults/JunitResultReader.cs
     -
     -    This patch contains a slightly inelegant workaround for the p4 and
     -    git-daemon tests: when we try to re-run the p4/git-daemon tests after
     -    the daemon has been started already, we need to kill said daemon. We do
     -    this by detecting the presence of the `kill_p4d` and `stop_git_daemon`
     -    shell functions and calling them if available.
     +    this as "Passed upon rerun", and Azure Pipelines even know about that
     +    outcome, but it is not available when using the JUnit XML format for
     +    now:
     +    https://github.com/Microsoft/azure-pipelines-agent/blob/master/src/Agent.Worker/TestResults/JunitResultReader.cs
      
          Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
      
     @@ -60,14 +55,8 @@
       	then
      +		if test -z "$GIT_TEST_TEE_OUTPUT_FILE"
      +		then
     -+			case "$(type kill_p4d 2>/dev/null | head -n 1)" in
     -+			*function*) kill_p4d;;
     -+			esac
     -+
     -+			case "$(type stop_git_daemon 2>/dev/null |
     -+				head -n 1)" in
     -+			*function*) stop_git_daemon;;
     -+			esac
     ++			# clean up
     ++			test_atexit_handler
      +
      +			# re-run with --verbose-log
      +			echo "# Re-running: $junit_rerun_options_sq" >&2
  8:  3e83c64090 = 11:  2466a48aa3 tests: record more stderr with --write-junit-xml in case of failure
  9:  dc1d890d71 ! 12:  d112b3fe86 README: add a build badge (status of the VSTS build)
     @@ -1,6 +1,6 @@
      Author: Johannes Schindelin <johannes.schindelin@gmx.de>
      
     -    README: add a build badge (status of the VSTS build)
     +    README: add a build badge (status of the Azure Pipelines build)
      
          Just like so many other OSS projects, we now also have a build badge.
      
     @@ -10,7 +10,7 @@
      --- a/README.md
      +++ b/README.md
      @@
     -+[![Build Status](https://git.visualstudio.com/git/_apis/build/status/test-git.git)](https://git.visualstudio.com/git/_build/latest?definitionId=2)
     ++[![Build Status](https:/dev.azure.com/git/git/_apis/build/status/test-git.git)](https://dev.azure.com/git/git/_build/latest?definitionId=2)
      +
       Git - fast, scalable, distributed revision control system
       =========================================================
  -:  ---------- > 13:  0a53f37135 travis: fix skipping tagged releases

Comments

Taylor Blau Oct. 15, 2018, 2:22 p.m. UTC | #1
Hi Johannes,

Thanks for putting this together, and offering to build Git on Azure
Pipelines. I haven't followed v1 of this series very closely, so please
excuse me if my comments have already been addressed, and I missed them
in a skim of the last revision.

On Mon, Oct 15, 2018 at 03:11:57AM -0700, Johannes Schindelin via GitGitGadget wrote:
> It is also an invaluable tool for contributors who can validate their code
> contributions via PRs on GitHub, e.g. to verify that their tests do actually
> run on macOS (i.e. with the BSD family of Unix tools instead of the GNU
> one).

Agree.

> The one sad part about this is the Windows support. Travis lacks it, and we
> work around that by using Azure Pipelines (the CI part of Azure DevOps,
> formerly known as Visual Studio Team Services) indirectly: one phase in
> Travis would trigger a build, wait for its log, and then paste that log.

I wonder if Travis' recent announcement [1] affects this at all. To
summarize [1], Travis is offering an early version of adding Windows to
their list of supported builder operations systems. This brings the list
to macOS, Linux, and Windows, which I think satisfies what we would
like to regularly build git.git on.

Would we like to abandon Travis as our main CI service for upstream
git.git, and build on Azure Pipelines only? If so, I think that this is
an OK way to go, but I think that I would be opposed to having more than
one build system. (FWIW, we tend to _three_ for Git LFS, and it can be a
hassle at times).

I see some benefit to sticking with Travis, since we already have a
build configuration that works there. But, you've done the work to
"port" that build configuration over to Azure, so perhaps the point is
moot.

> As Git's Windows builds (and tests!) take quite a bit of time, Travis often
> timed out, or somehow the trigger did not work, and for security reasons
> (the Windows builds are performed in a private pool of containers), the
> Windows builds are completely disabled for Pull Requests on GitHub.

This would be a concession of [1], in my mind: is it possible to run the
tests on Windows in a time such that Travis will not time out?

> As a special treat, this patch series adds the ability to present the
> outcome of Git's test suite as JUnit-style .xml files. This allows the Azure
> Pipelines build to present fun diagrams, trends, and makes it a lot easier
> to drill down to test failures than before. See for example
> https://dev.azure.com/git/git/_build/results?buildId=113&view=ms.vss-test-web.test-result-details
> [https://dev.azure.com/git/git/_build/results?buildId=113&view=ms.vss-test-web.test-result-details]
> (you can click on the label of the failed test, and then see the detailed
> output in the right pane).

That's pretty cool. Travis doesn't support this (to the best of my
knowledge).

Thanks,
Taylor

[1]: https://blog.travis-ci.com/2018-10-11-windows-early-release
Johannes Schindelin Oct. 15, 2018, 2:55 p.m. UTC | #2
Hi Taylor,

On Mon, 15 Oct 2018, Taylor Blau wrote:

> Thanks for putting this together, and offering to build Git on Azure
> Pipelines. I haven't followed v1 of this series very closely, so please
> excuse me if my comments have already been addressed, and I missed them
> in a skim of the last revision.
> 
> On Mon, Oct 15, 2018 at 03:11:57AM -0700, Johannes Schindelin via GitGitGadget wrote:
> > It is also an invaluable tool for contributors who can validate their code
> > contributions via PRs on GitHub, e.g. to verify that their tests do actually
> > run on macOS (i.e. with the BSD family of Unix tools instead of the GNU
> > one).
> 
> Agree.
> 
> > The one sad part about this is the Windows support. Travis lacks it, and we
> > work around that by using Azure Pipelines (the CI part of Azure DevOps,
> > formerly known as Visual Studio Team Services) indirectly: one phase in
> > Travis would trigger a build, wait for its log, and then paste that log.
> 
> I wonder if Travis' recent announcement [1] affects this at all.

:-)

It did not escape my notice that after years and years of trying to get
*anybody* at Travis to listen to my offers to help get this started, the
announcement of Azure Pipelines for OSS seemed to finally do the trick
(they still don't bother to talk to me, of course).

And to answer your question without a question mark: I do not really think
that the Travis announcement affects this here patch series: I have a ton
of good experience with Azure Pipelines, use it in Git for Windows for
ages, and I am finally able to have it in core Git, too. So I want it, and
I spent a lot of time getting there, and I think it probably won't hurt
core Git at all (besides, it seems that at least some of the phases are a
bit faster on Azure Pipelines than Travis).

Another really good reason for me to do this is that I can prod the Azure
Pipelines team directly. And I even get an answer, usually within minutes.
Which is a lot faster than the Travis team answers my questions, which
is... not yet? (I tried to get in contact with them in late 2015 or early
2016, and I tried again a year later, and then a couple of months later,
and I have yet to hear back.)

Also, I am not quite sure about the timeouts on Travis, but at least
AppVeyor had rather short timeouts: the Windows build (due to our
extensive use of Unix shell scripting in our test suite) takes 1h40m
currently, and AppVeyor times out after 20 or 30 minutes. I could imagine
that Travis times out after the same time, or maybe 60 minutes, which
would still be too short. On Azure Pipelines, the maximum timeout (which
can be configured via the azure-pipelines.yml file) is four hours IIRC.
Plenty enough even for our test suite on Windows.

> To summarize [1], Travis is offering an early version of adding Windows
> to their list of supported builder operations systems. This brings the
> list to macOS, Linux, and Windows, which I think satisfies what we would
> like to regularly build git.git on.

Honestly, I would love to have also FreeBSD and other platforms being
tested. And with Azure Pipelines, I can make that happen (eventually), by
adding another pool of VMs (given that I have a free $150/month Azure
subscription, I'd use Azure VMs, of course). As long as a platform can run
.NET Core software, it can run Azure Pipelines agents.

With Travis, I don't think I can add private agent pools.

> Would we like to abandon Travis as our main CI service for upstream
> git.git, and build on Azure Pipelines only? If so, I think that this is
> an OK way to go, but I think that I would be opposed to having more than
> one build system. (FWIW, we tend to _three_ for Git LFS, and it can be a
> hassle at times).

This question of abandoning Travis in favor of Azure Pipelines is a bit of
a hornets' nest, as I really, really only want to bring the goodness of
Azure Pipelines to git.git, and I am *clearly* biased, as I work at
Microsoft.

Which is the reason why I did not even hint at it in the cover letter, let
alone included a patch to make it so.

My patch series is purely about adding support for running CI/PR builds of
https://github.com/git/git via Azure Pipelines.

> I see some benefit to sticking with Travis, since we already have a
> build configuration that works there. But, you've done the work to
> "port" that build configuration over to Azure, so perhaps the point is
> moot.

It is not so much a port, as an attempt to generalize our ci/* files.

> > As Git's Windows builds (and tests!) take quite a bit of time, Travis often
> > timed out, or somehow the trigger did not work, and for security reasons
> > (the Windows builds are performed in a private pool of containers), the
> > Windows builds are completely disabled for Pull Requests on GitHub.
> 
> This would be a concession of [1], in my mind: is it possible to run the
> tests on Windows in a time such that Travis will not time out?

To be honest, I spent such a lot of time to get things to work on Azure
Pipelines, *and* we get a nice view on the test failures there, too (which
Travis will probably also offer soon, in response to what Azure Pipelines
offer ;-)), I cannot really justify spending time on trying to make things
work on Travis' Windows VMs, too. Especially when I have to expect to run
into timeout issues anyway.

> > As a special treat, this patch series adds the ability to present the
> > outcome of Git's test suite as JUnit-style .xml files. This allows the Azure
> > Pipelines build to present fun diagrams, trends, and makes it a lot easier
> > to drill down to test failures than before. See for example
> > https://dev.azure.com/git/git/_build/results?buildId=113&view=ms.vss-test-web.test-result-details
> > [https://dev.azure.com/git/git/_build/results?buildId=113&view=ms.vss-test-web.test-result-details]
> > (you can click on the label of the failed test, and then see the detailed
> > output in the right pane).
> 
> That's pretty cool. Travis doesn't support this (to the best of my
> knowledge).

Exactly.

Plus, if things don't work in Azure Pipelines, I (or one of the other
Microsoft employees among the core Git developers) can easily take a
shortcut to the team and get things fixed. In my mind, that counts for a
lot, too, especially given my own, frustrating personal experience with
Travis.

Ciao,
Dscho

> [1]: https://blog.travis-ci.com/2018-10-11-windows-early-release
Taylor Blau Oct. 16, 2018, 2:53 a.m. UTC | #3
On Mon, Oct 15, 2018 at 04:55:25PM +0200, Johannes Schindelin wrote:
> Hi Taylor,
>
> On Mon, 15 Oct 2018, Taylor Blau wrote:
>
> > Thanks for putting this together, and offering to build Git on Azure
> > Pipelines. I haven't followed v1 of this series very closely, so please
> > excuse me if my comments have already been addressed, and I missed them
> > in a skim of the last revision.
> >
> > On Mon, Oct 15, 2018 at 03:11:57AM -0700, Johannes Schindelin via GitGitGadget wrote:
> > > It is also an invaluable tool for contributors who can validate their code
> > > contributions via PRs on GitHub, e.g. to verify that their tests do actually
> > > run on macOS (i.e. with the BSD family of Unix tools instead of the GNU
> > > one).
> >
> > Agree.
> >
> > > The one sad part about this is the Windows support. Travis lacks it, and we
> > > work around that by using Azure Pipelines (the CI part of Azure DevOps,
> > > formerly known as Visual Studio Team Services) indirectly: one phase in
> > > Travis would trigger a build, wait for its log, and then paste that log.
> >
> > I wonder if Travis' recent announcement [1] affects this at all.
>
> :-)
>
> It did not escape my notice that after years and years of trying to get
> *anybody* at Travis to listen to my offers to help get this started, the
> announcement of Azure Pipelines for OSS seemed to finally do the trick
> (they still don't bother to talk to me, of course).
>
> And to answer your question without a question mark: I do not really think
> that the Travis announcement affects this here patch series: I have a ton
> of good experience with Azure Pipelines, use it in Git for Windows for
> ages, and I am finally able to have it in core Git, too. So I want it, and
> I spent a lot of time getting there, and I think it probably won't hurt
> core Git at all (besides, it seems that at least some of the phases are a
> bit faster on Azure Pipelines than Travis).

I think that there are fair reasons to prefer Azure Pipelines over
Travis. In particular, I am encouraged by the fact that we (1) know that
we won't timeout, and (2) can have a standardized CI interface on Git
and Git for Windows. Certainly, the Windows support on Azure Pipelines
is more developed than that of Travis', so that's another point for
Azure.

> Another really good reason for me to do this is that I can prod the Azure
> Pipelines team directly. And I even get an answer, usually within minutes.
> Which is a lot faster than the Travis team answers my questions, which
> is... not yet? (I tried to get in contact with them in late 2015 or early
> 2016, and I tried again a year later, and then a couple of months later,
> and I have yet to hear back.)

Certainly a good reason. To be clear/fair, I've sent in a number of
support tickets to Travis CI over the years, and have always been
responded to in a short amount of time with helpful answers. I think
that we would really be fine in either case, TBH.

> Also, I am not quite sure about the timeouts on Travis, but at least
> AppVeyor had rather short timeouts: the Windows build (due to our
> extensive use of Unix shell scripting in our test suite) takes 1h40m
> currently, and AppVeyor times out after 20 or 30 minutes. I could imagine
> that Travis times out after the same time, or maybe 60 minutes, which
> would still be too short. On Azure Pipelines, the maximum timeout (which
> can be configured via the azure-pipelines.yml file) is four hours IIRC.
> Plenty enough even for our test suite on Windows.
>
> > To summarize [1], Travis is offering an early version of adding Windows
> > to their list of supported builder operations systems. This brings the
> > list to macOS, Linux, and Windows, which I think satisfies what we would
> > like to regularly build git.git on.
>
> Honestly, I would love to have also FreeBSD and other platforms being
> tested. And with Azure Pipelines, I can make that happen (eventually), by
> adding another pool of VMs (given that I have a free $150/month Azure
> subscription, I'd use Azure VMs, of course). As long as a platform can run
> .NET Core software, it can run Azure Pipelines agents.
>
> With Travis, I don't think I can add private agent pools.

I think that's right.

> > Would we like to abandon Travis as our main CI service for upstream
> > git.git, and build on Azure Pipelines only? If so, I think that this is
> > an OK way to go, but I think that I would be opposed to having more than
> > one build system. (FWIW, we tend to _three_ for Git LFS, and it can be a
> > hassle at times).
>
> This question of abandoning Travis in favor of Azure Pipelines is a bit of
> a hornets' nest, as I really, really only want to bring the goodness of
> Azure Pipelines to git.git, and I am *clearly* biased, as I work at
> Microsoft.
>
> Which is the reason why I did not even hint at it in the cover letter, let
> alone included a patch to make it so.
>
> My patch series is purely about adding support for running CI/PR builds of
> https://github.com/git/git via Azure Pipelines.

I think that adding support for one CI system does carry the weight of
removing the other, for the sake of having few CI systems running at any
given time. In other words, even if you don't intend to start a
discussion about removing Travis, those that want run the smallest
number of services (including myself) will ask you about it ;-).

But I don't see a benefit to dragging this series through the mud by
spending too much time talking about Travis' future. I think that your
cover letter does a fine job to address the point, and we can revisit it
in the future if more people than just myself are opposed to running >1
CI service.

> > I see some benefit to sticking with Travis, since we already have a
> > build configuration that works there. But, you've done the work to
> > "port" that build configuration over to Azure, so perhaps the point is
> > moot.
>
> It is not so much a port, as an attempt to generalize our ci/* files.
>
> > > As Git's Windows builds (and tests!) take quite a bit of time, Travis often
> > > timed out, or somehow the trigger did not work, and for security reasons
> > > (the Windows builds are performed in a private pool of containers), the
> > > Windows builds are completely disabled for Pull Requests on GitHub.
> >
> > This would be a concession of [1], in my mind: is it possible to run the
> > tests on Windows in a time such that Travis will not time out?
>
> To be honest, I spent such a lot of time to get things to work on Azure
> Pipelines, *and* we get a nice view on the test failures there, too (which
> Travis will probably also offer soon, in response to what Azure Pipelines
> offer ;-)), I cannot really justify spending time on trying to make things
> work on Travis' Windows VMs, too. Especially when I have to expect to run
> into timeout issues anyway.

Agreed.

> > > As a special treat, this patch series adds the ability to present the
> > > outcome of Git's test suite as JUnit-style .xml files. This allows the Azure
> > > Pipelines build to present fun diagrams, trends, and makes it a lot easier
> > > to drill down to test failures than before. See for example
> > > https://dev.azure.com/git/git/_build/results?buildId=113&view=ms.vss-test-web.test-result-details
> > > [https://dev.azure.com/git/git/_build/results?buildId=113&view=ms.vss-test-web.test-result-details]
> > > (you can click on the label of the failed test, and then see the detailed
> > > output in the right pane).
> >
> > That's pretty cool. Travis doesn't support this (to the best of my
> > knowledge).
>
> Exactly.
>
> Plus, if things don't work in Azure Pipelines, I (or one of the other
> Microsoft employees among the core Git developers) can easily take a
> shortcut to the team and get things fixed. In my mind, that counts for a
> lot, too, especially given my own, frustrating personal experience with
> Travis.

Agreed, and thanks for your response.

> Ciao,
> Dscho
>
> > [1]: https://blog.travis-ci.com/2018-10-11-windows-early-release

Thanks,
Taylor
SZEDER Gábor Oct. 16, 2018, 10:30 a.m. UTC | #4
On Mon, Oct 15, 2018 at 07:22:15AM -0700, Taylor Blau wrote:
> Would we like to abandon Travis as our main CI service for upstream
> git.git, and build on Azure Pipelines only?

It's not only about "upstream git.git", but also about contributors,
who might have enabled Travis CI integration on their forks on GitHub.
Having a '.travis.yml' and associated 'ci/*' scripts in git.git makes
it possible for them to easily build and test their branches on their
own.
Ævar Arnfjörð Bjarmason Oct. 16, 2018, 11:55 a.m. UTC | #5
On Tue, Oct 16, 2018 at 4:55 AM Taylor Blau <me@ttaylorr.com> wrote:
>
> On Mon, Oct 15, 2018 at 04:55:25PM +0200, Johannes Schindelin wrote:
> > Another really good reason for me to do this is that I can prod the Azure
> > Pipelines team directly. And I even get an answer, usually within minutes.
> > Which is a lot faster than the Travis team answers my questions, which
> > is... not yet? (I tried to get in contact with them in late 2015 or early
> > 2016, and I tried again a year later, and then a couple of months later,
> > and I have yet to hear back.)
>
> Certainly a good reason. To be clear/fair, I've sent in a number of
> support tickets to Travis CI over the years, and have always been
> responded to in a short amount of time with helpful answers. I think
> that we would really be fine in either case, TBH.

Was this in the context of "I'm this random dude using Travis" or "as
you guys know I work for GitHub, your biggest? customer..." ? :)
Junio C Hamano Oct. 25, 2018, 8:17 a.m. UTC | #6
"Johannes Schindelin via GitGitGadget" <gitgitgadget@gmail.com>
writes:

> For a long time already, we have Git's source code continuously tested via
> Travis CI, see e.g. https://travis-ci.org/git/git/builds/421738884. It has
> served us well, and more and more developers actually pay attention and
> benefit from the testing this gives us.

What's the current status of this topic?  Has the "p4 daemon gets
left behind" one resolved to everybody's satisfaction?  I think that
one was the only large discussion on the series (aside from "do we
want to keep Travis?" subthread, which does not make this series
undesirable), modulo your "oy oy oy that is leftover debugging I
need to remove in a reroll".

The topic was marked as "On hold, monitoring discussion" and it
seems that discussion has quieted down, so the next step is to see
an updated series?

Thanks.
Johannes Schindelin Oct. 26, 2018, 8:37 a.m. UTC | #7
Hi Junio,

On Thu, 25 Oct 2018, Junio C Hamano wrote:

> "Johannes Schindelin via GitGitGadget" <gitgitgadget@gmail.com>
> writes:
> 
> > For a long time already, we have Git's source code continuously tested via
> > Travis CI, see e.g. https://travis-ci.org/git/git/builds/421738884. It has
> > served us well, and more and more developers actually pay attention and
> > benefit from the testing this gives us.
> 
> What's the current status of this topic?  Has the "p4 daemon gets
> left behind" one resolved to everybody's satisfaction?

No. I was kind of waiting for Luke's answer, and in the alternative I
hoped to find some time to work on trying to reproduce his issues on my
system (but I failed to find said time so far).

> I think that one was the only large discussion on the series (aside from
> "do we want to keep Travis?" subthread, which does not make this series
> undesirable), modulo your "oy oy oy that is leftover debugging I need to
> remove in a reroll".
> 
> The topic was marked as "On hold, monitoring discussion" and it
> seems that discussion has quieted down, so the next step is to see
> an updated series?

I really think that I have to figure out what causes those p4d issues
before I can give you that updated. I *am* interested, to be sure, it's
just that other things seem to get in my way all the time.

One thing that keeps getting in my way, for example, is the performance
issue identified in the chain linter. And I do think that I have to take
this into consideration for another update to this here patch series, too,
by adding `--no-chain-lint` to the Windows phase. There is a similar thing
with `--with-dashes`, too.

Will keep you updated,
Dscho

> 
> Thanks.
> 
> 
>
Johannes Schindelin Jan. 16, 2019, 2:04 p.m. UTC | #8
Hi Junio,

On Thu, 25 Oct 2018, Junio C Hamano wrote:

> "Johannes Schindelin via GitGitGadget" <gitgitgadget@gmail.com>
> writes:
> 
> > For a long time already, we have Git's source code continuously tested via
> > Travis CI, see e.g. https://travis-ci.org/git/git/builds/421738884. It has
> > served us well, and more and more developers actually pay attention and
> > benefit from the testing this gives us.
> 
> What's the current status of this topic?
> [...]
> 
> The topic was marked as "On hold, monitoring discussion" and it seems
> that discussion has quieted down, so the next step is to see an updated
> series?

See the updated series:
https://public-inbox.org/git/pull.31.v3.git.gitgitgadget@gmail.com/

Thanks,
Dscho
Junio C Hamano Jan. 17, 2019, 8:59 p.m. UTC | #9
Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> ...
> See the updated series:
> https://public-inbox.org/git/pull.31.v3.git.gitgitgadget@gmail.com/

Thanks.

I see that you are already planning for v4, but I'll find time to
take a look at what is posted sometime this week anyway.
Johannes Schindelin Jan. 18, 2019, 11:49 a.m. UTC | #10
Hi Junio,

On Thu, 17 Jan 2019, Junio C Hamano wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> > ...
> > See the updated series:
> > https://public-inbox.org/git/pull.31.v3.git.gitgitgadget@gmail.com/
> 
> Thanks.
> 
> I see that you are already planning for v4, but I'll find time to
> take a look at what is posted sometime this week anyway.

Yes, v4 will ideally have only cosmetic changes apart from the
part that fixes the traces of the published failed tests:

-- snipsnap --
15:  f678b105f81e ! 15:  7b74987d72a6 tests: include detailed trace logs with --write-junit-xml upon failure
    @@ -32,6 +32,38 @@
     
         Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
     
    + diff --git a/t/helper/test-path-utils.c b/t/helper/test-path-utils.c
    + --- a/t/helper/test-path-utils.c
    + +++ b/t/helper/test-path-utils.c
    +@@
    + 		return !!res;
    + 	}
    + 
    ++	if (argc == 4 && !strcmp(argv[1], "skip-n-bytes")) {
    ++		int fd = open(argv[2], O_RDONLY), offset = atoi(argv[3]);
    ++		char buffer[65536];
    ++
    ++		if (fd < 0)
    ++			die_errno("could not open '%s'", argv[2]);
    ++		if (lseek(fd, offset, SEEK_SET) < 0)
    ++			die_errno("could not skip %d bytes", offset);
    ++		for (;;) {
    ++			ssize_t count = read(fd, buffer, sizeof(buffer));
    ++			if (count < 0)
    ++				die_errno("could not read '%s'", argv[2]);
    ++			if (!count)
    ++				break;
    ++			if (write(1, buffer, count) < 0)
    ++				die_errno("could not write to stdout");
    ++		}
    ++		close(fd);
    ++		return 0;
    ++	}
    ++
    + 	fprintf(stderr, "%s: unknown function name: %s\n", argv[0],
    + 		argv[1] ? argv[1] : "(there was none)");
    + 	return 1;
    +
      diff --git a/t/test-lib.sh b/t/test-lib.sh
      --- a/t/test-lib.sh
      +++ b/t/test-lib.sh
    @@ -42,7 +74,8 @@
     -			"$(printf '%s\n' "$@" | sed 1d)")"
     +			"$(if test -n "$GIT_TEST_TEE_OUTPUT_FILE"
     +			   then
    -+				cut -c "$GIT_TEST_TEE_OFFSET-" <"$GIT_TEST_TEE_OUTPUT_FILE"
    ++				test-tool path-utils skip-n-bytes \
    ++					"$GIT_TEST_TEE_OUTPUT_FILE" $GIT_TEST_TEE_OFFSET
     +			   else
     +				printf '%s\n' "$@" | sed 1d
     +			   fi)")"
    @@ -56,17 +89,17 @@
      	fi
      	test_failure=$(($test_failure + 1))
     @@
    - 	write_junit_xml "$(printf '%s\n' \
    - 		"    <testcase $junit_attrs>" "$@" "    </testcase>")"
    - 	junit_have_testcase=t
    -+	if test -n "$GIT_TEST_TEE_OUTPUT_FILE"
    + 	echo >&3 ""
    + 	maybe_teardown_valgrind
    + 	maybe_teardown_verbose
    ++	if test -n "$GIT_TEST_TEE_OFFSET"
     +	then
     +		GIT_TEST_TEE_OFFSET=$(test-tool path-utils file-size \
     +			"$GIT_TEST_TEE_OUTPUT_FILE")
     +	fi
      }
      
    - test_done () {
    + test_skip () {
     @@
      		date +%Y-%m-%dT%H:%M:%S)\""
      	write_junit_xml --truncate "<testsuites>" "  <testsuite $junit_attrs>"
    @@ -74,7 +107,6 @@
     +	if test -n "$GIT_TEST_TEE_OUTPUT_FILE"
     +	then
     +		GIT_TEST_TEE_OFFSET=0
    -+		GIT_TEST_TEE_ERR_OFFSET=0
     +	fi
      fi