From patchwork Sun Jan 27 23:26:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee via GitGitGadget X-Patchwork-Id: 10783061 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CD2A5159A for ; Sun, 27 Jan 2019 23:26:55 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BBEC6298E1 for ; Sun, 27 Jan 2019 23:26:55 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9D49C298EC; Sun, 27 Jan 2019 23:26:55 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 200AD298E1 for ; Sun, 27 Jan 2019 23:26:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726630AbfA0X0x (ORCPT ); Sun, 27 Jan 2019 18:26:53 -0500 Received: from mail-ed1-f67.google.com ([209.85.208.67]:38148 "EHLO mail-ed1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726511AbfA0X0x (ORCPT ); Sun, 27 Jan 2019 18:26:53 -0500 Received: by mail-ed1-f67.google.com with SMTP id h50so11588683ede.5 for ; Sun, 27 Jan 2019 15:26:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=96M+SI3VKYDk3uCf1BpYpa5//cuUAVpfWWWcGr5kIog=; b=Fsxdfz4JXstOAoJ3gjFIvuqexGLx7A1U4ONjbqkPZdsdXxi9B4vL4iaRj/EYhJXivi FP7sXZWabDbNtAQhYuw//agLl+wxW9kjuPzO6Dr5YDxV9w2OlNK+yikcF89M1yTs9lNV 61mWxkW9Nbw7L7hOrOLeorm7VSbFJPh2M5nFRc6QC/4rNtLNXovhU2jbVQN8JRr1iCHx XVCsN45u03bmg0f/gtUEBtpzFGogje65d27jQCGuZvyLYsX1Kc6kejuddZi1gVASSxBT rRyvsXYNQv2BGd6R6Ic6TeksAUiLR88XcxCP/4TK1Okl4gjCR50eUFq5+kz18Q+mMpNq TBgg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=96M+SI3VKYDk3uCf1BpYpa5//cuUAVpfWWWcGr5kIog=; b=GZRx4qknOHGHMylWtdWbVnRhC4K8z8chHYkHuPlEOp7fDFwAyZZlv932U9QNkxyTVE nQR6DWq1wqfVeG0f02a6/Lf6ZobweFes2y2olJd4xLBAk4cCYnc/DltG8Xja3hEI9W3W 3BAkYvfEeI/LAORJpguQ7xzG38YraVQdZ79b2k/u3JEf9QTXSBLT9VT7Uen6IshYoCy0 /v12/2PtG0stcJhMnVCisn0TfLHO8seVV+4lt4gsZUlOnR1BYx1YdV1J94Z0sVWVxIgU sYoPnESfNBXfr1Lp5ZXIK08vDsCENsYx6ZhtIe+YcvyawnGiCeG1YJravbTTmLsFozj3 Pucw== X-Gm-Message-State: AJcUukdrPKDbLpNCUeMnuD2GY7oulpr2O6OukvJkhi52DbbSor43xI3/ gqg9qobJAbMUqzchauzOpTw5yscd X-Google-Smtp-Source: ALg8bN51Fg53oCdriw5Jbw8qHvSXkxxDXXr/o23tv7Q1RYKh6dhTOgTXe/o9hGwZ4Cz7w+Z1puhZRg== X-Received: by 2002:a17:906:288d:: with SMTP id o13-v6mr16727302ejd.53.1548631610939; Sun, 27 Jan 2019 15:26:50 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id a11sm14022650edc.28.2019.01.27.15.26.50 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 27 Jan 2019 15:26:50 -0800 (PST) Date: Sun, 27 Jan 2019 15:26:50 -0800 (PST) X-Google-Original-Date: Sun, 27 Jan 2019 23:26:28 GMT Message-Id: In-Reply-To: References: From: "Johannes Schindelin via GitGitGadget" Subject: [PATCH v5 02/21] ci: rename the library of common functions Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Johannes Schindelin Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Johannes Schindelin The name is hard-coded to reflect that we use Travis CI for continuous testing. In the next commits, we will extend this to be able use Azure DevOps, too. So let's adjust the name to make it more generic. Signed-off-by: Johannes Schindelin --- ci/install-dependencies.sh | 2 +- ci/{lib-travisci.sh => lib.sh} | 0 ci/print-test-failures.sh | 2 +- ci/run-build-and-tests.sh | 2 +- ci/run-linux32-docker.sh | 2 +- ci/run-static-analysis.sh | 2 +- ci/run-windows-build.sh | 2 +- ci/test-documentation.sh | 2 +- 8 files changed, 7 insertions(+), 7 deletions(-) rename ci/{lib-travisci.sh => lib.sh} (100%) diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh index 06c3546e1e..fe65144152 100755 --- a/ci/install-dependencies.sh +++ b/ci/install-dependencies.sh @@ -3,7 +3,7 @@ # Install dependencies required to build and test Git on Linux and macOS # -. ${0%/*}/lib-travisci.sh +. ${0%/*}/lib.sh P4WHENCE=http://filehost.perforce.com/perforce/r$LINUX_P4_VERSION LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION diff --git a/ci/lib-travisci.sh b/ci/lib.sh similarity index 100% rename from ci/lib-travisci.sh rename to ci/lib.sh diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh index d55460a212..7aef39a2fd 100755 --- a/ci/print-test-failures.sh +++ b/ci/print-test-failures.sh @@ -3,7 +3,7 @@ # Print output of failing tests # -. ${0%/*}/lib-travisci.sh +. ${0%/*}/lib.sh # Tracing executed commands would produce too much noise in the loop below. set +x diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh index cda170d5c2..db342bb6a8 100755 --- a/ci/run-build-and-tests.sh +++ b/ci/run-build-and-tests.sh @@ -3,7 +3,7 @@ # Build and test Git # -. ${0%/*}/lib-travisci.sh +. ${0%/*}/lib.sh ln -s "$cache_dir/.prove" t/.prove diff --git a/ci/run-linux32-docker.sh b/ci/run-linux32-docker.sh index 21637903ce..751acfcf8a 100755 --- a/ci/run-linux32-docker.sh +++ b/ci/run-linux32-docker.sh @@ -3,7 +3,7 @@ # Download and run Docker image to build and test 32-bit Git # -. ${0%/*}/lib-travisci.sh +. ${0%/*}/lib.sh docker pull daald/ubuntu32:xenial diff --git a/ci/run-static-analysis.sh b/ci/run-static-analysis.sh index 5688f261d0..dc189c7456 100755 --- a/ci/run-static-analysis.sh +++ b/ci/run-static-analysis.sh @@ -3,7 +3,7 @@ # Perform various static code analysis checks # -. ${0%/*}/lib-travisci.sh +. ${0%/*}/lib.sh make --jobs=2 coccicheck diff --git a/ci/run-windows-build.sh b/ci/run-windows-build.sh index d99a180e52..a73a4eca0a 100755 --- a/ci/run-windows-build.sh +++ b/ci/run-windows-build.sh @@ -6,7 +6,7 @@ # supported) and a commit hash. # -. ${0%/*}/lib-travisci.sh +. ${0%/*}/lib.sh test $# -ne 2 && echo "Unexpected number of parameters" && exit 1 test -z "$GFW_CI_TOKEN" && echo "GFW_CI_TOKEN not defined" && exit diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh index a20de9ca12..d3cdbac73f 100755 --- a/ci/test-documentation.sh +++ b/ci/test-documentation.sh @@ -3,7 +3,7 @@ # Perform sanity checks on documentation and build it. # -. ${0%/*}/lib-travisci.sh +. ${0%/*}/lib.sh gem install asciidoctor From patchwork Sun Jan 27 23:26:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee via GitGitGadget X-Patchwork-Id: 10783067 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D056A13B5 for ; Sun, 27 Jan 2019 23:27:01 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BDDB7298AD for ; Sun, 27 Jan 2019 23:27:01 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B270D298F2; Sun, 27 Jan 2019 23:27:01 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8FA1B298AD for ; Sun, 27 Jan 2019 23:27:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726878AbfA0X05 (ORCPT ); Sun, 27 Jan 2019 18:26:57 -0500 Received: from mail-ed1-f67.google.com ([209.85.208.67]:39138 "EHLO mail-ed1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726302AbfA0X0y (ORCPT ); Sun, 27 Jan 2019 18:26:54 -0500 Received: by mail-ed1-f67.google.com with SMTP id b14so11593559edt.6 for ; Sun, 27 Jan 2019 15:26:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=RELDQxxtpPecLmfvF8MO2ybLT3oALQRfzq+V9+mgang=; b=HGvt2y2/fe7UZE/U2hABDe/wh4pPEOuDu74JbKiKOfIjP4VXRY0w2IjO7amA4ibyfg kJMHGPDkPHRSGYTwvCYKVGUg+BLnwrx2tuavgZ6FRfLPNmKC7qBCsWdWwkIQVlwHbBh/ WLA50t7G+IbGz0El/EZMLALQu7boEXQDikX8J+oYnCU9x0s78LbbDU+P/0evXZKAYfpl jaxojEbcJOygrKD1goDz/2HcuntO1hICYYwrfktf/4jKKADlPHUBcx7/QJwIZ+ePpm9M ujQTHPMW08DB02LLCyRFMySRmiPzoHqTkStnp3qlyG9mQeloRPrYzhx46KT4vZUYL3I4 99QQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=RELDQxxtpPecLmfvF8MO2ybLT3oALQRfzq+V9+mgang=; b=Uxtw+DlByawLxt7sFcDV4PFcvymgTTaFgBg9zY461VVT4sP1uerPKVFHNsqA3uHdEO eAgvEqH7Lc3nhwRObc3N/o7CGCjC9pX/H7ceaDN0IHAszwj9HRZa22zK1lG+nXMiB8h9 ZoFj4CUa8ERxqGs+tM7mQx8sCoHUNqyD8sG0HsDyl1c8HDQ1nf4TIYjYxtLB3b0dUwhh 8uMin+sajJjkUcacuzeIb3bO0esg2C0eadd73gBfGrqrUKITRsr8gKhpnGalF4trZKEn HzKWGVZzSnkmbaLoXVHhXxGcfQo9PxiLrofeTmHmGZ5xIQrgSW/3A01hdB+Hth03hhA9 Vjfg== X-Gm-Message-State: AJcUukd6vSk36chpQTFvpx929czjxGs+i4MM8PoDPEWRbRiI+k41okXS 7ou8SPLXq4QPFhcvu/01xxUAtFxo X-Google-Smtp-Source: ALg8bN7z6dZO/+3/baX9iBsQ3qhR9oTIxEgduKL3HA5iSw1+bLb5bQp9TS6apPtynxnsoDLqMP1hrA== X-Received: by 2002:a50:875d:: with SMTP id 29mr19838798edv.100.1548631611844; Sun, 27 Jan 2019 15:26:51 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id 97sm13503904edq.45.2019.01.27.15.26.51 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 27 Jan 2019 15:26:51 -0800 (PST) Date: Sun, 27 Jan 2019 15:26:51 -0800 (PST) X-Google-Original-Date: Sun, 27 Jan 2019 23:26:29 GMT Message-Id: <7c16d31b46258ad00768c6125d0c6ad64d341c15.1548631607.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Johannes Schindelin via GitGitGadget" Subject: [PATCH v5 03/21] ci/lib.sh: encapsulate Travis-specific things Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Johannes Schindelin Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Johannes Schindelin The upcoming patches will allow building git.git via Azure Pipelines (i.e. Azure DevOps' Continuous Integration), where variable names and URLs look a bit different than in Travis CI. Also, the configurations of the available agents are different. For example, Travis' and Azure Pipelines' macOS agents are set up differently, so that on Travis, we have to install the git-lfs and gettext Homebrew packages, and on Azure Pipelines we do not need to. Likewise, Azure Pipelines' Ubuntu agents already have asciidoctor installed. Finally, on Azure Pipelines the natural way is not to base64-encode tar files of the trash directories of failed tests, but to publish build artifacts instead. Therefore, that code to log those base64-encoded tar files is guarded to be Travis-specific. Signed-off-by: Johannes Schindelin --- ci/install-dependencies.sh | 3 ++- ci/lib.sh | 45 +++++++++++++++++++++++++++----------- ci/print-test-failures.sh | 8 +++++++ ci/test-documentation.sh | 1 + 4 files changed, 43 insertions(+), 14 deletions(-) diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh index fe65144152..bcdcc71592 100755 --- a/ci/install-dependencies.sh +++ b/ci/install-dependencies.sh @@ -37,7 +37,8 @@ osx-clang|osx-gcc) brew update --quiet # Uncomment this if you want to run perf tests: # brew install gnu-time - brew install git-lfs gettext + test -z "$BREW_INSTALL_PACKAGES" || + brew install $BREW_INSTALL_PACKAGES brew link --force gettext brew install caskroom/cask/perforce ;; diff --git a/ci/lib.sh b/ci/lib.sh index 9c6ddeb374..3f286d86a6 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -24,7 +24,7 @@ skip_branch_tip_with_tag () { # job if we encounter the same tree again and can provide a useful info # message. save_good_tree () { - echo "$(git rev-parse $TRAVIS_COMMIT^{tree}) $TRAVIS_COMMIT $TRAVIS_JOB_NUMBER $TRAVIS_JOB_ID" >>"$good_trees_file" + echo "$(git rev-parse $CI_COMMIT^{tree}) $CI_COMMIT $CI_JOB_NUMBER $CI_JOB_ID" >>"$good_trees_file" # limit the file size tail -1000 "$good_trees_file" >"$good_trees_file".tmp mv "$good_trees_file".tmp "$good_trees_file" @@ -34,7 +34,7 @@ save_good_tree () { # successfully before (e.g. because the branch got rebased, changing only # the commit messages). skip_good_tree () { - if ! good_tree_info="$(grep "^$(git rev-parse $TRAVIS_COMMIT^{tree}) " "$good_trees_file")" + if ! good_tree_info="$(grep "^$(git rev-parse $CI_COMMIT^{tree}) " "$good_trees_file")" then # Haven't seen this tree yet, or no cached good trees file yet. # Continue the build job. @@ -44,18 +44,18 @@ skip_good_tree () { echo "$good_tree_info" | { read tree prev_good_commit prev_good_job_number prev_good_job_id - if test "$TRAVIS_JOB_ID" = "$prev_good_job_id" + if test "$CI_JOB_ID" = "$prev_good_job_id" then cat <<-EOF - $(tput setaf 2)Skipping build job for commit $TRAVIS_COMMIT.$(tput sgr0) + $(tput setaf 2)Skipping build job for commit $CI_COMMIT.$(tput sgr0) This commit has already been built and tested successfully by this build job. To force a re-build delete the branch's cache and then hit 'Restart job'. EOF else cat <<-EOF - $(tput setaf 2)Skipping build job for commit $TRAVIS_COMMIT.$(tput sgr0) + $(tput setaf 2)Skipping build job for commit $CI_COMMIT.$(tput sgr0) This commit's tree has already been built and tested successfully in build job $prev_good_job_number for commit $prev_good_commit. - The log of that build job is available at https://travis-ci.org/$TRAVIS_REPO_SLUG/jobs/$prev_good_job_id + The log of that build job is available at $(url_for_job_id $prev_good_job_id) To force a re-build delete the branch's cache and then hit 'Restart job'. EOF fi @@ -80,11 +80,32 @@ check_unignored_build_artifacts () # and installing dependencies. set -ex -# When building a PR, TRAVIS_BRANCH refers to the *target* branch. Not what we -# want here. We want the source branch instead. -CI_BRANCH="${TRAVIS_PULL_REQUEST_BRANCH:-$TRAVIS_BRANCH}" +if test true = "$TRAVIS" +then + CI_TYPE=travis + # When building a PR, TRAVIS_BRANCH refers to the *target* branch. Not + # what we want here. We want the source branch instead. + CI_BRANCH="${TRAVIS_PULL_REQUEST_BRANCH:-$TRAVIS_BRANCH}" + CI_COMMIT="$TRAVIS_COMMIT" + CI_JOB_ID="$TRAVIS_JOB_ID" + CI_JOB_NUMBER="$TRAVIS_JOB_NUMBER" + CI_OS_NAME="$TRAVIS_OS_NAME" + CI_REPO_SLUG="$TRAVIS_REPO_SLUG" + + cache_dir="$HOME/travis-cache" + + url_for_job_id () { + echo "https://travis-ci.org/$CI_REPO_SLUG/jobs/$1" + } + + BREW_INSTALL_PACKAGES="git-lfs gettext" + export GIT_PROVE_OPTS="--timer --jobs 3 --state=failed,slow,save" + export GIT_TEST_OPTS="--verbose-log -x --immediate" +else + echo "Could not identify CI type" >&2 + exit 1 +fi -cache_dir="$HOME/travis-cache" good_trees_file="$cache_dir/good-trees" mkdir -p "$cache_dir" @@ -94,13 +115,11 @@ skip_good_tree if test -z "$jobname" then - jobname="$TRAVIS_OS_NAME-$CC" + jobname="$CI_OS_NAME-$CC" fi export DEVELOPER=1 export DEFAULT_TEST_TARGET=prove -export GIT_PROVE_OPTS="--timer --jobs 3 --state=failed,slow,save" -export GIT_TEST_OPTS="--verbose-log -x --immediate" export GIT_TEST_CLONE_2GB=YesPlease if [ "$jobname" = linux-gcc ]; then export CC=gcc-8 diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh index 7aef39a2fd..cf321b474d 100755 --- a/ci/print-test-failures.sh +++ b/ci/print-test-failures.sh @@ -38,6 +38,14 @@ do test_name="${TEST_EXIT%.exit}" test_name="${test_name##*/}" trash_dir="trash directory.$test_name" + case "$CI_TYPE" in + travis) + ;; + *) + echo "Unhandled CI type: $CI_TYPE" >&2 + exit 1 + ;; + esac trash_tgz_b64="trash.$test_name.base64" if [ -d "$trash_dir" ] then diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh index d3cdbac73f..7d0beb2832 100755 --- a/ci/test-documentation.sh +++ b/ci/test-documentation.sh @@ -5,6 +5,7 @@ . ${0%/*}/lib.sh +test -n "$ALREADY_HAVE_ASCIIDOCTOR" || gem install asciidoctor make check-builtins From patchwork Sun Jan 27 23:26:52 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee via GitGitGadget X-Patchwork-Id: 10783063 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AC73A159A for ; Sun, 27 Jan 2019 23:27:01 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 97AA3298F2 for ; Sun, 27 Jan 2019 23:27:01 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8AE2A29915; Sun, 27 Jan 2019 23:27:01 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6C857298F2 for ; Sun, 27 Jan 2019 23:27:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726864AbfA0X04 (ORCPT ); Sun, 27 Jan 2019 18:26:56 -0500 Received: from mail-ed1-f46.google.com ([209.85.208.46]:40361 "EHLO mail-ed1-f46.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726511AbfA0X0z (ORCPT ); Sun, 27 Jan 2019 18:26:55 -0500 Received: by mail-ed1-f46.google.com with SMTP id g22so11590881edr.7 for ; Sun, 27 Jan 2019 15:26:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=QDfwBZMkgyvZGlij6f9DAjFec83Ya7Y4uw5FE6ySdt4=; b=Ki+B5YWhw9Qd3UZBBZsAWVlg3EDsCR3zkemQdUD9ZCbb5JRsy6TDjiTHlcAyKxLjxc KQlPawDVh/QAksWYZf4SibRONsyMitkYyVZn4hS4U+BbUtu3bK4EB0iqS4SUE5JNs9Yp CvzNa9M3neQMGQpEncJL4/RRlapkYr7aZy8MvmJ/xeNIODy/XDJkSaFMfTlFdmYHjOEv g7hmuhp8NVz3Sl5iLIQq85BAIijV1k9ty6zNPhHFTydWz66KpJFlxnjaz+Y3nxqXhgCW B+096THn7bdnaUAqA6l6e0cDsxv46KgwjgTqWby7s4rYFDqnMBKdo4C+vdJ84s2/8mZg 4Rlg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=QDfwBZMkgyvZGlij6f9DAjFec83Ya7Y4uw5FE6ySdt4=; b=npe2RBePdkmlZn3PNfHD05pQymEmwieyFAdYzC6APuPNHiZQOSsUqmlrAx2INN0k1I F8xFSMMpZGS86TT3nOxdfhLP8cL8xWEWLbxXcLpvyvk+QIwF5lm3Xw1ltsYxx9sPd9fQ gwDBS06x8rpieb775whWC9Boc8f2rhBrY6+O/jqns/cGhlFfP38fjqomAPRvEgza62wf IzX+wE+F4BPk+BqxWJhuO2R6enaJ2jtd1x96EndVsZ/4ltWC5JLGEP7kYdd198M7cT/3 s44tC6wVoX53EI5wmXjMtI2vp2fs0FurvYMsnn++4exTGQALnhoIxBtv1cfcFEbZC5PM FZVw== X-Gm-Message-State: AJcUukfMKpnQKTBC79x4kkg9aDNJs9fsUVRdO/bzhwDA7T1BQmNPCYHI 5sXzWKCbTyQg4R8YLhcuHEJNgaPl X-Google-Smtp-Source: ALg8bN68qaUc9wyQx7uRs5GpYBXCv2J3U7CF5nfH7thngh3lxotic6bgBs/Yh18W4eJlg8Ih9+kV2w== X-Received: by 2002:a17:906:6b0f:: with SMTP id q15mr3015399ejr.139.1548631612906; Sun, 27 Jan 2019 15:26:52 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id p21sm12894109edr.67.2019.01.27.15.26.52 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 27 Jan 2019 15:26:52 -0800 (PST) Date: Sun, 27 Jan 2019 15:26:52 -0800 (PST) X-Google-Original-Date: Sun, 27 Jan 2019 23:26:30 GMT Message-Id: In-Reply-To: References: From: "Johannes Schindelin via GitGitGadget" Subject: [PATCH v5 04/21] ci: inherit --jobs via MAKEFLAGS in run-build-and-tests Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Johannes Schindelin Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Johannes Schindelin Let's not decide in the generic ci/ part how many jobs to run in parallel; different CI configurations would favor a different number of parallel jobs, and it is easy enough to hand that information down via the `MAKEFLAGS` variable. Signed-off-by: Johannes Schindelin --- ci/lib.sh | 1 + ci/run-build-and-tests.sh | 2 +- ci/run-linux32-build.sh | 2 +- ci/run-static-analysis.sh | 2 +- ci/test-documentation.sh | 4 ++-- 5 files changed, 6 insertions(+), 5 deletions(-) diff --git a/ci/lib.sh b/ci/lib.sh index 3f286d86a6..32a28fd209 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -101,6 +101,7 @@ then BREW_INSTALL_PACKAGES="git-lfs gettext" export GIT_PROVE_OPTS="--timer --jobs 3 --state=failed,slow,save" export GIT_TEST_OPTS="--verbose-log -x --immediate" + export MAKEFLAGS="--jobs=2" else echo "Could not identify CI type" >&2 exit 1 diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh index db342bb6a8..80d72d120f 100755 --- a/ci/run-build-and-tests.sh +++ b/ci/run-build-and-tests.sh @@ -7,7 +7,7 @@ ln -s "$cache_dir/.prove" t/.prove -make --jobs=2 +make make --quiet test if test "$jobname" = "linux-gcc" then diff --git a/ci/run-linux32-build.sh b/ci/run-linux32-build.sh index 2c60d2e70a..09e9276e12 100755 --- a/ci/run-linux32-build.sh +++ b/ci/run-linux32-build.sh @@ -55,6 +55,6 @@ linux32 --32bit i386 su -m -l $CI_USER -c ' set -ex cd /usr/src/git test -n "$cache_dir" && ln -s "$cache_dir/.prove" t/.prove - make --jobs=2 + make make --quiet test ' diff --git a/ci/run-static-analysis.sh b/ci/run-static-analysis.sh index dc189c7456..a19aa7ebbc 100755 --- a/ci/run-static-analysis.sh +++ b/ci/run-static-analysis.sh @@ -5,7 +5,7 @@ . ${0%/*}/lib.sh -make --jobs=2 coccicheck +make coccicheck set +x diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh index 7d0beb2832..be3b7d376a 100755 --- a/ci/test-documentation.sh +++ b/ci/test-documentation.sh @@ -12,7 +12,7 @@ make check-builtins make check-docs # Build docs with AsciiDoc -make --jobs=2 doc > >(tee stdout.log) 2> >(tee stderr.log >&2) +make doc > >(tee stdout.log) 2> >(tee stderr.log >&2) ! test -s stderr.log test -s Documentation/git.html test -s Documentation/git.xml @@ -24,7 +24,7 @@ check_unignored_build_artifacts # Build docs with AsciiDoctor make clean -make --jobs=2 USE_ASCIIDOCTOR=1 doc > >(tee stdout.log) 2> >(tee stderr.log >&2) +make USE_ASCIIDOCTOR=1 doc > >(tee stdout.log) 2> >(tee stderr.log >&2) sed '/^GIT_VERSION = / d' stderr.log ! test -s stderr.log test -s Documentation/git.html From patchwork Sun Jan 27 23:26:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee via GitGitGadget X-Patchwork-Id: 10783075 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D424B6C2 for ; Sun, 27 Jan 2019 23:27:05 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C299C286BF for ; Sun, 27 Jan 2019 23:27:05 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B716B298EC; Sun, 27 Jan 2019 23:27:05 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 67C62286BF for ; Sun, 27 Jan 2019 23:27:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726859AbfA0X04 (ORCPT ); Sun, 27 Jan 2019 18:26:56 -0500 Received: from mail-ed1-f65.google.com ([209.85.208.65]:42193 "EHLO mail-ed1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726775AbfA0X0z (ORCPT ); Sun, 27 Jan 2019 18:26:55 -0500 Received: by mail-ed1-f65.google.com with SMTP id y20so11587454edw.9 for ; Sun, 27 Jan 2019 15:26:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=lCqeYJ5kfLqoi2oIKKtYTdsgfR0s8LiDnVTc/mVzstc=; b=B8QIYEwIufQVnROqJNIFItY759s7R42Cu77TeoYOn9zN77VtE2XZQVe+IGvq17Gybu aCLQmEywRkR3y201xGLegQ8C4x45toqVn+6lo7GZd6njn0eE6mdnwAI58vqWqQyYPPPD FaeR3CWHlINzyS+i8cLvcDVQNbxhVM610VvOhXZvMjQBJjkLBcB+NfK7UZH1A/TFi7cy I+TQIqHNpGiINrNNfrMyFyudCmNcrjqoDHG2TzCCEwvpyVmvTb+0cPD0R0uxg7HPkVeW yvXNdJkBFZiKxYYQbaylkAHe5Oni724XKs+zUhgHJhgAKrOtm1lShNQ85Kxk1sJkR7eY vnag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=lCqeYJ5kfLqoi2oIKKtYTdsgfR0s8LiDnVTc/mVzstc=; b=V17R0veiAdZWPJgyOmwbnXBF7/lV1bvuuGZqmv0u76PVe3WqATxujqXddhRmTw5a6J 2OFDun1tdvcaQy1YTlo8xP4p6at8n5t1GF5wZApqEKZG3D7UudQ6hBba3FQHoTvzvQCZ 5HKoOvbJQGHzcn3HR2qcU5B/QTPzbIaGUqZ7WfypK9za+LujcbcnCBiqRlY0cPa3cEI5 A5H6ZnqUVvlwbEha7f1yklTqoZSi+5ADNhnialXbwO0uzUlgbH66ddkgAsVJ/4zkbACe f14JSbPMAmWzTUwVI3dAhX1/6cR1PUpjTrpVw8fEvX2O2BxX86gE6IDnNNDh3iPOCyd6 8+6A== X-Gm-Message-State: AJcUukc4MkPJ8bZvmUiFCSfzyAh2WAwECXk6ar6iMUhiV4ZofGayN6Pz uVPn3w6rZ7Ry5N3I1lKDkM3VdFSA X-Google-Smtp-Source: ALg8bN7dFT10ggn2MTCPZlOrG2ULImHq8ytmBQYysl4ZM2OjV3mN/oqlgcEXTtl6dCkztljKisedsg== X-Received: by 2002:a50:c408:: with SMTP id v8mr19277725edf.144.1548631613743; Sun, 27 Jan 2019 15:26:53 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id g21sm13637807edc.63.2019.01.27.15.26.53 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 27 Jan 2019 15:26:53 -0800 (PST) Date: Sun, 27 Jan 2019 15:26:53 -0800 (PST) X-Google-Original-Date: Sun, 27 Jan 2019 23:26:31 GMT Message-Id: <681f8e65b4d3cf82a5920840aa8dc88bde812964.1548631607.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Johannes Schindelin via GitGitGadget" Subject: [PATCH v5 05/21] ci: use a junction on Windows instead of a symlink Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Johannes Schindelin Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Johannes Schindelin Symbolic links are still not quite as easy to use on Windows as on Linux (for example, on versions older than Windows 10, only administrators can create symlinks, and on Windows 10 you still need to be in developer mode for regular users to have permission), but NTFS junctions can give us a way out. Signed-off-by: Johannes Schindelin --- ci/run-build-and-tests.sh | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh index 80d72d120f..74d838ea01 100755 --- a/ci/run-build-and-tests.sh +++ b/ci/run-build-and-tests.sh @@ -5,7 +5,10 @@ . ${0%/*}/lib.sh -ln -s "$cache_dir/.prove" t/.prove +case "$CI_OS_NAME" in +windows*) cmd //c mklink //j t\\.prove "$(cygpath -aw "$cache_dir/.prove")";; +*) ln -s "$cache_dir/.prove" t/.prove;; +esac make make --quiet test From patchwork Sun Jan 27 23:26:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee via GitGitGadget X-Patchwork-Id: 10783073 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B8083159A for ; Sun, 27 Jan 2019 23:27:04 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A4929286BF for ; Sun, 27 Jan 2019 23:27:04 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 989CC298EC; Sun, 27 Jan 2019 23:27:04 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 84DF1286BF for ; Sun, 27 Jan 2019 23:27:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726915AbfA0X1B (ORCPT ); Sun, 27 Jan 2019 18:27:01 -0500 Received: from mail-ed1-f52.google.com ([209.85.208.52]:44160 "EHLO mail-ed1-f52.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726403AbfA0X04 (ORCPT ); Sun, 27 Jan 2019 18:26:56 -0500 Received: by mail-ed1-f52.google.com with SMTP id y56so11584316edd.11 for ; Sun, 27 Jan 2019 15:26:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=oT7R2wDGQ27bqNVq7NMA4Ofuj3PjG2VZGG2HORDkQtU=; b=C8FV+CGKUT7XUUOwSPnrsKtGBUQ8xZbytkbKNF/BS8XRFo94+V8NVWorpysuEwFmQa 5KC3uUf0/qC/QdHnvBNRwB15W6ZFAoTsf31c08gsqh79kZCP0aTXRQFwOuWphs9/KrTj jLwrOF2TfAQ6AqcohPb7wnP8cgvexVSOvvN3pget3GhRynwThhWG9BclRHSxBNVOzLib GdEsMIuyr8emhlxTEWEXGdzha8pXX1Zj56YS4uHvKy4SiNGInkBzm0ZiQZllQfJaCAly enc0RRaq3YfPhKS01qjEKAGMlOV5YNKLm4/CBRlCDaIi/AfD8IfQiMi3SjJ0Q81IbgYO 7UDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=oT7R2wDGQ27bqNVq7NMA4Ofuj3PjG2VZGG2HORDkQtU=; b=rf37oioWdDzhKe/sws3Ch6rp2yo8duhQPMo92ns4YrAvv5F7x4yvGd9DgCmqmIt7oX 00DkWOhyHPHTxhujzxOLv2PzwrtOYtE/BJQ20t1Iv3atoaJ2HSFqQ3t1RdAynN+WlAdf Yq7r6FUNrM0MC4kb+Xxf3QdhiN9Q+isE9u+pFK4y3p5eGrgvVVx+dKP3i9FdHmbY1Wzu kzymRJLWEqRMtrk3vOvRyDZ4x1tMRLzq4YZZXDRYkO8MS1Cdj6xr8KMn4uH79rR/TEZP kubPw6gBpcbL4aFEq6SQIUaXObAdYil7XE+iFrJMzzFr7GS+rpgCuAjoc6EblbdqLPE0 l7Zw== X-Gm-Message-State: AJcUukcagQlPpLPQllpK0yQcQdCfBPfALKBN8rkozzZpUa3pry4If+El +8AD1xbTyodDqu60IBOfDLpY2o9b X-Google-Smtp-Source: ALg8bN4yQi0l6yKGGM7tEdzN5PjAzzgg5jb3Q8owGiQHyvPRGfKb55t43J/0D/sacUVWapxeTi7zaQ== X-Received: by 2002:a50:84a9:: with SMTP id 38mr19279117edq.185.1548631614475; Sun, 27 Jan 2019 15:26:54 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id k12sm647405eja.60.2019.01.27.15.26.53 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 27 Jan 2019 15:26:54 -0800 (PST) Date: Sun, 27 Jan 2019 15:26:54 -0800 (PST) X-Google-Original-Date: Sun, 27 Jan 2019 23:26:32 GMT Message-Id: In-Reply-To: References: From: "Johannes Schindelin via GitGitGadget" Subject: [PATCH v5 06/21] test-date: add a subcommand to measure times in shell scripts Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Johannes Schindelin Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Johannes Schindelin In the next commit, we want to teach Git's test suite to optionally output test results in JUnit-style .xml files. These files contain information about the time spent. So we need a way to measure time. While we could use `date +%s` for that, this will give us only seconds, i.e. very coarse-grained timings. GNU `date` supports `date +%s.%N` (i.e. nanosecond-precision output), but there is no equivalent in BSD `date` (read: on macOS, we would not be able to obtain precise timings). So let's introduce `test-tool date getnanos`, with an optional start time, that outputs preciser values. Note that this might not actually give us nanosecond precision on some platforms, but it will give us as precise information as possible, without the portability issues of shell commands. Granted, it is a bit pointless to try measuring times accurately in shell scripts, certainly to nanosecond precision. But it is better than second-granularity. Signed-off-by: Johannes Schindelin --- t/helper/test-date.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/t/helper/test-date.c b/t/helper/test-date.c index a0837371ab..792a805374 100644 --- a/t/helper/test-date.c +++ b/t/helper/test-date.c @@ -7,6 +7,7 @@ static const char *usage_msg = "\n" " test-tool date parse [date]...\n" " test-tool date approxidate [date]...\n" " test-tool date timestamp [date]...\n" +" test-tool date getnanos [start-nanos]\n" " test-tool date is64bit\n" " test-tool date time_t-is64bit\n"; @@ -82,6 +83,15 @@ static void parse_approx_timestamp(const char **argv, struct timeval *now) } } +static void getnanos(const char **argv, struct timeval *now) +{ + double seconds = getnanotime() / 1.0e9; + + if (*argv) + seconds -= strtod(*argv, NULL); + printf("%lf\n", seconds); +} + int cmd__date(int argc, const char **argv) { struct timeval now; @@ -108,6 +118,8 @@ int cmd__date(int argc, const char **argv) parse_approxidate(argv+1, &now); else if (!strcmp(*argv, "timestamp")) parse_approx_timestamp(argv+1, &now); + else if (!strcmp(*argv, "getnanos")) + getnanos(argv+1, &now); else if (!strcmp(*argv, "is64bit")) return sizeof(timestamp_t) == 8 ? 0 : 1; else if (!strcmp(*argv, "time_t-is64bit")) From patchwork Sun Jan 27 23:26:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee via GitGitGadget X-Patchwork-Id: 10783069 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 436E5188D for ; Sun, 27 Jan 2019 23:27:02 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 320BD298E1 for ; Sun, 27 Jan 2019 23:27:02 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 26A3129915; Sun, 27 Jan 2019 23:27:02 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B0322298E1 for ; Sun, 27 Jan 2019 23:27:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726884AbfA0X07 (ORCPT ); Sun, 27 Jan 2019 18:26:59 -0500 Received: from mail-ed1-f49.google.com ([209.85.208.49]:45775 "EHLO mail-ed1-f49.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726861AbfA0X05 (ORCPT ); Sun, 27 Jan 2019 18:26:57 -0500 Received: by mail-ed1-f49.google.com with SMTP id d39so11569391edb.12 for ; Sun, 27 Jan 2019 15:26:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=+n7i2tQJa0DlH1QtOlg+5zTg3n07wnZ+0QGC/00TeMc=; b=Jd+caI2zNb0G//pQJypwjBsdkjfJYciQQ/rufjXGt8qHLM2fwULPg3zRo6+1s+PCWi Y9YTz5f73Qr+h08JqSeUKi0GJXfknF+0nthVVRlX4vh8ml6Z9vXmzzXl3D3FIS+5HX9N h/QmQdfXmdNXxE/R5Ukdao6Ddp6O8ljNNTNXMkHN49az+14aVcK1bfy5P0+XZZ63lntJ wV06gRnBAhKn3ymHFK5Pw6KePqjjRLcrFGz3VV5hHz9XVyRs1Aj59dXRvIGkkvup2VO4 Ec7TUPg7szdMHd3OpXCnoaiFRBx+DaN4P2LQnrhjESHj5Tn5Q5yEgiM8WZBY7KpWgJPe +QGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=+n7i2tQJa0DlH1QtOlg+5zTg3n07wnZ+0QGC/00TeMc=; b=Y3zsDyzZ43fQNJ1ZS2HwdSlfjGMqsrTQWsn4MPwS9OOizTxboVuZytbOXjqudeH6Ta pi0fOy/iGZlqlcdXNCrixcCtUb8V+i2f5yMgFKXx39gfXUxTYH6nUcghpm30fhddojLo K8dVyN8Zz3zcaMLXuyyDkZjAifDvYiBpQEyLqzT6ANGPPJ6Qvndr7JZ62VVn6y+IlJEj hykRYG7nTjKgAdFzCOly98Lj2aOTEiyv2AdQmsRPxvOxftsGnVIVrGSZ3UeuyJK9A/rw dCVe82v+SfTaO4t2i/m1Yz0yhW8NLqxc2qD3C2R/+sh375LMFs6BXYo/Z9jUpLfxTOV6 rV/Q== X-Gm-Message-State: AJcUukeDi+6rRKj9+V6ZoVVu/SoxmETBWjjxSR9+wOdtO0CiBIcRY/B0 dBjLwvFfjtVqSwRRoTwtQK2hRwW4 X-Google-Smtp-Source: ALg8bN6JRmmNCc8Q9NEXO7Ukckzvjzi/5GYcIAUmT0O8LoXBRlkKQqVmZq2UZGuaKX2B0MsSckY7yg== X-Received: by 2002:a17:906:4bd9:: with SMTP id x25mr6075895ejv.171.1548631615313; Sun, 27 Jan 2019 15:26:55 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id p25sm373189ejn.6.2019.01.27.15.26.54 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 27 Jan 2019 15:26:54 -0800 (PST) Date: Sun, 27 Jan 2019 15:26:54 -0800 (PST) X-Google-Original-Date: Sun, 27 Jan 2019 23:26:33 GMT Message-Id: <31fbe114256688710216ad268c0c34206f71badd.1548631607.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Johannes Schindelin via GitGitGadget" Subject: [PATCH v5 07/21] tests: optionally write results as JUnit-style .xml Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Johannes Schindelin Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Johannes Schindelin This will come in handy when publishing the results of Git's test suite during an automated Azure DevOps run. Note: we need to make extra sure that invalid UTF-8 encoding is turned into valid UTF-8 (using the Replacement Character, \uFFFD) because t9902's trace contains such invalid byte sequences, and the task in the Azure Pipeline that uploads the test results would refuse to do anything if it was asked to parse an .xml file with invalid UTF-8 in it. Signed-off-by: Johannes Schindelin --- Makefile | 1 + t/.gitignore | 1 + t/helper/test-tool.c | 1 + t/helper/test-tool.h | 1 + t/helper/test-xml-encode.c | 80 +++++++++++++++++++++++++++++++++ t/test-lib.sh | 91 ++++++++++++++++++++++++++++++++++++++ 6 files changed, 175 insertions(+) create mode 100644 t/helper/test-xml-encode.c diff --git a/Makefile b/Makefile index 1a44c811aa..044b4f77bd 100644 --- a/Makefile +++ b/Makefile @@ -754,6 +754,7 @@ TEST_BUILTINS_OBJS += test-submodule-config.o TEST_BUILTINS_OBJS += test-submodule-nested-repo-config.o TEST_BUILTINS_OBJS += test-subprocess.o TEST_BUILTINS_OBJS += test-urlmatch-normalization.o +TEST_BUILTINS_OBJS += test-xml-encode.o TEST_BUILTINS_OBJS += test-wildmatch.o TEST_BUILTINS_OBJS += test-windows-named-pipe.o TEST_BUILTINS_OBJS += test-write-cache.o diff --git a/t/.gitignore b/t/.gitignore index 348715f0e4..91cf5772fe 100644 --- a/t/.gitignore +++ b/t/.gitignore @@ -2,3 +2,4 @@ /test-results /.prove /chainlinttmp +/out/ diff --git a/t/helper/test-tool.c b/t/helper/test-tool.c index bfb195b1a8..4b4b397d93 100644 --- a/t/helper/test-tool.c +++ b/t/helper/test-tool.c @@ -49,6 +49,7 @@ static struct test_cmd cmds[] = { { "submodule-nested-repo-config", cmd__submodule_nested_repo_config }, { "subprocess", cmd__subprocess }, { "urlmatch-normalization", cmd__urlmatch_normalization }, + { "xml-encode", cmd__xml_encode }, { "wildmatch", cmd__wildmatch }, #ifdef GIT_WINDOWS_NATIVE { "windows-named-pipe", cmd__windows_named_pipe }, diff --git a/t/helper/test-tool.h b/t/helper/test-tool.h index 042f12464b..c0ab65e370 100644 --- a/t/helper/test-tool.h +++ b/t/helper/test-tool.h @@ -45,6 +45,7 @@ int cmd__submodule_config(int argc, const char **argv); int cmd__submodule_nested_repo_config(int argc, const char **argv); int cmd__subprocess(int argc, const char **argv); int cmd__urlmatch_normalization(int argc, const char **argv); +int cmd__xml_encode(int argc, const char **argv); int cmd__wildmatch(int argc, const char **argv); #ifdef GIT_WINDOWS_NATIVE int cmd__windows_named_pipe(int argc, const char **argv); diff --git a/t/helper/test-xml-encode.c b/t/helper/test-xml-encode.c new file mode 100644 index 0000000000..367c4875e6 --- /dev/null +++ b/t/helper/test-xml-encode.c @@ -0,0 +1,80 @@ +#include "test-tool.h" + +static const char *utf8_replace_character = "�"; + +/* + * Encodes (possibly incorrect) UTF-8 on to , to be embedded + * in an XML file. + */ +int cmd__xml_encode(int argc, const char **argv) +{ + unsigned char buf[1024], tmp[4], *tmp2 = NULL; + ssize_t cur = 0, len = 1, remaining = 0; + unsigned char ch; + + for (;;) { + if (++cur == len) { + len = xread(0, buf, sizeof(buf)); + if (!len) + return 0; + if (len < 0) + die_errno("Could not read "); + cur = 0; + } + ch = buf[cur]; + + if (tmp2) { + if ((ch & 0xc0) != 0x80) { + fputs(utf8_replace_character, stdout); + tmp2 = 0; + cur--; + continue; + } + *tmp2 = ch; + tmp2++; + if (--remaining == 0) { + fwrite(tmp, tmp2 - tmp, 1, stdout); + tmp2 = 0; + } + continue; + } + + if (!(ch & 0x80)) { + /* 0xxxxxxx */ + if (ch == '&') + fputs("&", stdout); + else if (ch == '\'') + fputs("'", stdout); + else if (ch == '"') + fputs(""", stdout); + else if (ch == '<') + fputs("<", stdout); + else if (ch == '>') + fputs(">", stdout); + else if (ch >= 0x20) + fputc(ch, stdout); + else if (ch == 0x09 || ch == 0x0a || ch == 0x0d) + fprintf(stdout, "&#x%02x;", ch); + else + fputs(utf8_replace_character, stdout); + } else if ((ch & 0xe0) == 0xc0) { + /* 110XXXXx 10xxxxxx */ + tmp[0] = ch; + remaining = 1; + tmp2 = tmp + 1; + } else if ((ch & 0xf0) == 0xe0) { + /* 1110XXXX 10Xxxxxx 10xxxxxx */ + tmp[0] = ch; + remaining = 2; + tmp2 = tmp + 1; + } else if ((ch & 0xf8) == 0xf0) { + /* 11110XXX 10XXxxxx 10xxxxxx 10xxxxxx */ + tmp[0] = ch; + remaining = 3; + tmp2 = tmp + 1; + } else + fputs(utf8_replace_character, stdout); + } + + return 0; +} diff --git a/t/test-lib.sh b/t/test-lib.sh index a1abb1177a..a3b2166cb5 100644 --- a/t/test-lib.sh +++ b/t/test-lib.sh @@ -139,6 +139,9 @@ do verbose_log=t tee=t ;; + --write-junit-xml) + write_junit_xml=t + ;; --stress) stress=t ;; --stress=*) @@ -622,11 +625,24 @@ trap 'exit $?' INT TERM HUP # the test_expect_* functions instead. test_ok_ () { + if test -n "$write_junit_xml" + then + write_junit_xml_testcase "$*" + fi test_success=$(($test_success + 1)) say_color "" "ok $test_count - $@" } test_failure_ () { + if test -n "$write_junit_xml" + then + junit_insert="" + junit_insert="$junit_insert $(xml_attr_encode \ + "$(printf '%s\n' "$@" | sed 1d)")" + junit_insert="$junit_insert" + write_junit_xml_testcase "$1" " $junit_insert" + fi test_failure=$(($test_failure + 1)) say_color error "not ok $test_count - $1" shift @@ -635,11 +651,19 @@ test_failure_ () { } test_known_broken_ok_ () { + if test -n "$write_junit_xml" + then + write_junit_xml_testcase "$* (breakage fixed)" + fi test_fixed=$(($test_fixed+1)) say_color error "ok $test_count - $@ # TODO known breakage vanished" } test_known_broken_failure_ () { + if test -n "$write_junit_xml" + then + write_junit_xml_testcase "$* (known breakage)" + fi test_broken=$(($test_broken+1)) say_color warn "not ok $test_count - $@ # TODO known breakage" } @@ -897,6 +921,10 @@ test_start_ () { test_count=$(($test_count+1)) maybe_setup_verbose maybe_setup_valgrind + if test -n "$write_junit_xml" + then + junit_start=$(test-tool date getnanos) + fi } test_finish_ () { @@ -934,6 +962,13 @@ test_skip () { case "$to_skip" in t) + if test -n "$write_junit_xml" + then + message="$(xml_attr_encode "$skipped_reason")" + write_junit_xml_testcase "$1" \ + " " + fi + say_color skip >&3 "skipping test: $@" say_color skip "ok $test_count # skip $1 ($skipped_reason)" : true @@ -949,9 +984,51 @@ test_at_end_hook_ () { : } +write_junit_xml () { + case "$1" in + --truncate) + >"$junit_xml_path" + junit_have_testcase= + shift + ;; + esac + printf '%s\n' "$@" >>"$junit_xml_path" +} + +xml_attr_encode () { + printf '%s\n' "$@" | test-tool xml-encode +} + +write_junit_xml_testcase () { + junit_attrs="name=\"$(xml_attr_encode "$this_test.$test_count $1")\"" + shift + junit_attrs="$junit_attrs classname=\"$this_test\"" + junit_attrs="$junit_attrs time=\"$(test-tool \ + date getnanos $junit_start)\"" + write_junit_xml "$(printf '%s\n' \ + " " "$@" " ")" + junit_have_testcase=t +} + test_done () { GIT_EXIT_OK=t + if test -n "$write_junit_xml" && test -n "$junit_xml_path" + then + test -n "$junit_have_testcase" || { + junit_start=$(test-tool date getnanos) + write_junit_xml_testcase "all tests skipped" + } + + # adjust the overall time + junit_time=$(test-tool date getnanos $junit_suite_start) + sed "s/]*/& time=\"$junit_time\"/" \ + <"$junit_xml_path" >"$junit_xml_path.new" + mv "$junit_xml_path.new" "$junit_xml_path" + + write_junit_xml " " "" + fi + if test -z "$HARNESS_ACTIVE" then mkdir -p "$TEST_RESULTS_DIR" @@ -1178,6 +1255,7 @@ then else mkdir -p "$TRASH_DIRECTORY" fi + # Use -P to resolve symlinks in our working directory so that the cwd # in subprocesses like git equals our $PWD (for pathname comparisons). cd -P "$TRASH_DIRECTORY" || exit 1 @@ -1191,6 +1269,19 @@ then test_done fi +if test -n "$write_junit_xml" +then + junit_xml_dir="$TEST_OUTPUT_DIRECTORY/out" + mkdir -p "$junit_xml_dir" + junit_xml_base=${0##*/} + junit_xml_path="$junit_xml_dir/TEST-${junit_xml_base%.sh}.xml" + junit_attrs="name=\"${junit_xml_base%.sh}\"" + junit_attrs="$junit_attrs timestamp=\"$(TZ=UTC \ + date +%Y-%m-%dT%H:%M:%S)\"" + write_junit_xml --truncate "" " " + junit_suite_start=$(test-tool date getnanos) +fi + # Provide an implementation of the 'yes' utility yes () { if test $# = 0 From patchwork Sun Jan 27 23:26:55 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee via GitGitGadget X-Patchwork-Id: 10783071 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 95A0C13B5 for ; Sun, 27 Jan 2019 23:27:04 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 84D04298EC for ; Sun, 27 Jan 2019 23:27:04 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 792B4298F2; Sun, 27 Jan 2019 23:27:04 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F0962298AD for ; Sun, 27 Jan 2019 23:27:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726904AbfA0X1A (ORCPT ); Sun, 27 Jan 2019 18:27:00 -0500 Received: from mail-ed1-f48.google.com ([209.85.208.48]:34656 "EHLO mail-ed1-f48.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726511AbfA0X06 (ORCPT ); Sun, 27 Jan 2019 18:26:58 -0500 Received: by mail-ed1-f48.google.com with SMTP id b3so11653464ede.1 for ; Sun, 27 Jan 2019 15:26:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=RJiBb8F0ZIhkEzjeNE5PJ80Os/lj+Z9pFBzCKAm8HF8=; b=cpddnDVuJECZm7SqQttSe9vANpntyz1GbFEtkj0ZGKlMOLL7z40RF/dITdEme5sWed LTfe9CeSKuORAwNoN+nLURQghVb/HDtztJIGHabgwVAbhLQbGFexxuLWP3+lj80W37VL 6N4dRR0KFbwWlRkMEvxD8BwIIYYbYEdH+i+Jbly4Z0+IaNDfQT6Gi6gy0irxQGJ13wml ps6BkorPPLe52w1Nv5TeUQ5OTup70Yui3NmmnjkB9yBeiAHzDSQ8PhURDjqW+iRtdsna vzOGP0uA8+FATjWjCXVu/ONkpellOV2WLqLmu+G92LHPToW0S0KnyAEFPjGk0uTE2HVF uaRg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=RJiBb8F0ZIhkEzjeNE5PJ80Os/lj+Z9pFBzCKAm8HF8=; b=tbSZNDAR/YrBO7sp9Sg88I1G4nUlkZ60yG6UaPKDqZLpl49n1phxnesywuMMztvu+o UCsTR3WRE/61DibhZO2pH5njF7lq1Kg2fklvMiOBYkCs7BnwSfXBdUj1eJVmfy18+EW5 Ek24I9FOxSmmaGOxazCID/Pzot9nq5M7OVeDdsaywPorhMAbpGqhsusaXd1tXsbddz0r evUlG5TBFvoTn28PoNWf+mqmCvOzuXPaMGK+sdZhrM/PfnBhNGXGSsEZYMVATt97Ab1G WcTonr7dIyUZynfAZ9vOVI6wdylNJ2XKm6FZzQqs8OdfoqrfNZkHdP5IK860+qZHv88q N1Bw== X-Gm-Message-State: AJcUukcvX1brwehSO8ciX87evUkKVZs/GLOXJaJYys/dt2QNdArS7wrj 1ugvduQlr4mlcBK/8SSx9IiIo3oe X-Google-Smtp-Source: ALg8bN7kNXT/Nom0lcDHtKqxmtOyJccqzJTUUu7xxCgPuViPGEB+QkED/I5RJq4z2aB4utWdbdXEoQ== X-Received: by 2002:aa7:d487:: with SMTP id b7mr18758395edr.256.1548631615922; Sun, 27 Jan 2019 15:26:55 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id f4sm13269290edb.21.2019.01.27.15.26.55 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 27 Jan 2019 15:26:55 -0800 (PST) Date: Sun, 27 Jan 2019 15:26:55 -0800 (PST) X-Google-Original-Date: Sun, 27 Jan 2019 23:26:34 GMT Message-Id: <47fe8aa84aa11b8da9c01c5e8e88b68909dcfdbd.1548631607.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Johannes Schindelin via GitGitGadget" Subject: [PATCH v5 08/21] ci/lib.sh: add support for Azure Pipelines Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Johannes Schindelin Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Johannes Schindelin This patch introduces a conditional arm that defines some environment variables and a function that displays the URL given the job id (to identify previous runs for known-good trees). Because Azure Pipeline's macOS agents already have git-lfs and gettext installed, we can leave `BREW_INSTALL_PACKAGES` empty (unlike in Travis' case). Note: this patch does not introduce an Azure Pipelines definition yet; That is left for the next patch. Signed-off-by: Johannes Schindelin --- ci/lib.sh | 25 +++++++++++++++++++++++++ ci/print-test-failures.sh | 5 +++++ 2 files changed, 30 insertions(+) diff --git a/ci/lib.sh b/ci/lib.sh index 32a28fd209..5505776876 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -102,6 +102,31 @@ then export GIT_PROVE_OPTS="--timer --jobs 3 --state=failed,slow,save" export GIT_TEST_OPTS="--verbose-log -x --immediate" export MAKEFLAGS="--jobs=2" +elif test -n "$SYSTEM_COLLECTIONURI" || test -n "$SYSTEM_TASKDEFINITIONSURI" +then + CI_TYPE=azure-pipelines + # We are running in Azure Pipelines + CI_BRANCH="$BUILD_SOURCEBRANCH" + CI_COMMIT="$BUILD_SOURCEVERSION" + CI_JOB_ID="$BUILD_BUILDID" + CI_JOB_NUMBER="$BUILD_BUILDNUMBER" + CI_OS_NAME="$(echo "$AGENT_OS" | tr A-Z a-z)" + test darwin != "$CI_OS_NAME" || CI_OS_NAME=osx + CI_REPO_SLUG="$(expr "$BUILD_REPOSITORY_URI" : '.*/\([^/]*/[^/]*\)$')" + CC="${CC:-gcc}" + + # use a subdirectory of the cache dir (because the file share is shared + # among *all* phases) + cache_dir="$HOME/test-cache/$SYSTEM_PHASENAME" + + url_for_job_id () { + echo "$SYSTEM_TASKDEFINITIONSURI$SYSTEM_TEAMPROJECT/_build/results?buildId=$1" + } + + BREW_INSTALL_PACKAGES= + export GIT_PROVE_OPTS="--timer --jobs 10 --state=failed,slow,save" + export GIT_TEST_OPTS="--verbose-log -x --write-junit-xml" + export MAKEFLAGS="--jobs=10" else echo "Could not identify CI type" >&2 exit 1 diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh index cf321b474d..e688a26f0d 100755 --- a/ci/print-test-failures.sh +++ b/ci/print-test-failures.sh @@ -41,6 +41,11 @@ do case "$CI_TYPE" in travis) ;; + azure-pipelines) + mkdir -p failed-test-artifacts + mv "$trash_dir" failed-test-artifacts + continue + ;; *) echo "Unhandled CI type: $CI_TYPE" >&2 exit 1 From patchwork Sun Jan 27 23:26:56 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee via GitGitGadget X-Patchwork-Id: 10783093 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 79CB113B5 for ; Sun, 27 Jan 2019 23:27:20 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 67C3B286BF for ; Sun, 27 Jan 2019 23:27:20 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5C1E7298EC; Sun, 27 Jan 2019 23:27:20 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 67F78286BF for ; Sun, 27 Jan 2019 23:27:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727035AbfA0X1S (ORCPT ); Sun, 27 Jan 2019 18:27:18 -0500 Received: from mail-ed1-f45.google.com ([209.85.208.45]:35785 "EHLO mail-ed1-f45.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726879AbfA0X1A (ORCPT ); Sun, 27 Jan 2019 18:27:00 -0500 Received: by mail-ed1-f45.google.com with SMTP id x30so11634151edx.2 for ; Sun, 27 Jan 2019 15:26:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=pFbplBFQUOIV1Nh4Mnv6OTr2Rj2NGgBQhiUPNzuNSoI=; b=TXoCGnSLs6JoiuM6Ahgw7wvlGsvXsrl9d1uvwbwWWvgXc/i9U0FoTQJnaq+KtXrdzs Xu+/WjfR59Y78OV5TIPV/wGASEjOkv6AU5gM3Pa0fWF0a05fhND+qXYwK7DYWvi4Mxbn 7leRLKXfTmDqLuIv9B9Dqy7hnSGkB7aN8Dr6747RHmNuKLlTtFKY/TJVd/EsR5yw/UJ9 iXCmPEXwEB5S1ZkTuSVUPtq0AQ0z/+fRymWeRRmDVPSork7VfjW1XlArsn1i97Jsejp6 gW3tBFa0KPSoOSH4SovObPVUy1kz6Wz1zShvnUu9yHWhShYK9EkvEQIfL3xrvKyUXya+ 0XTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=pFbplBFQUOIV1Nh4Mnv6OTr2Rj2NGgBQhiUPNzuNSoI=; b=RVBiEltFN5+OZ6803osUQAEbV7IANytQNLk7i/EBogGsyTR1CEmnHm9+Chz5yPA2TE 58aS1WPOWAwxA46EVmKxmvZS94X6ezvGsNjiQN2Su0Y0uu3jDElbYjcxS+IHcUMIqJHf lqktJ/9OpMkWJYvwJt+Ia0Er2R+w5uxYStkZv1Rqxj4JaWXiQDMmhExno6D4vIXoKg3D RVkU2m7ON5bOFRRD7DVKiw95mhigFrh81rFv4u7cqzH5fjbyfgsRoOPoUPbf3XYyNWim 7pmJW7PsEL7As/W00B0aDnOucCsEYk4rtoyGozXCLjKOdEfJBR0byUAuCHRbz9R2Ox0N RD2g== X-Gm-Message-State: AJcUukeFZEIC5P8G4q/MoVY0tGVZn8ckIeme84TSVN+y+B0eA6qk7bBU kIofwd9IQeNkF1lRI9iyo0vWBgnk X-Google-Smtp-Source: ALg8bN57z1U5rQA18tyu8tmzuBBVmpWaygUYlb5fvIKr/Qi0mbtUNTdYLTZ0d5dCeVugpgr+Pp5IxQ== X-Received: by 2002:a50:8c04:: with SMTP id p4mr19332404edp.274.1548631616698; Sun, 27 Jan 2019 15:26:56 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id r51sm13635859eda.64.2019.01.27.15.26.56 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 27 Jan 2019 15:26:56 -0800 (PST) Date: Sun, 27 Jan 2019 15:26:56 -0800 (PST) X-Google-Original-Date: Sun, 27 Jan 2019 23:26:35 GMT Message-Id: <6044b919b0c186aa1f20b56b4235696a248983cf.1548631607.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Johannes Schindelin via GitGitGadget" Subject: [PATCH v5 09/21] Add a build definition for Azure DevOps Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Johannes Schindelin Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Johannes Schindelin This commit adds an azure-pipelines.yml file which is Azure DevOps' equivalent to Travis CI's .travis.yml. The main idea is to replicate the Travis configuration as faithfully as possible, to make it easy to compare the Azure Pipeline builds to the Travis ones (spoiler: some parts, especially the macOS jobs, are way faster in Azure Pileines). Meaning: the number and the order of the jobs added in this commit faithfully replicates what we have in .travis.yml. Note: Our .travis.yml configuration has a Windows part that is *not* replicated in the Azure Pipelines definition. The reason is easy to see: As Travis cannot support our Windws needs (even with the preliminary Windows support that was recently added to Travis after waiting for *years* for that feature, our test suite would simply hit Travis' timeout every single time). To make things a bit easier to understand, we refrain from using the `matrix` feature here because (while it is powerful) it can be a bit confusing to users who are not familiar with CI setups. Therefore, we use a separate phase even for similar configurations (such as GCC vs Clang on Linux, GCC vs Clang on macOS). Also, we make use of the shiny new feature we just introduced where the test suite can output JUnit-style .xml files. This information is made available in a nice UI that allows the viewer to filter by phase and/or test number, and to see trends such as: number of (failing) tests, time spent running the test suite, etc. (While this seemingly contradicts the intention to replicate the Travis configuration as faithfully as possible, it is just too nice to show off that capability here already.) Signed-off-by: Johannes Schindelin --- azure-pipelines.yml | 259 ++++++++++++++++++++++++++++++++++++++++++ ci/mount-fileshare.sh | 25 ++++ 2 files changed, 284 insertions(+) create mode 100644 azure-pipelines.yml create mode 100755 ci/mount-fileshare.sh diff --git a/azure-pipelines.yml b/azure-pipelines.yml new file mode 100644 index 0000000000..8cdef105c6 --- /dev/null +++ b/azure-pipelines.yml @@ -0,0 +1,259 @@ +resources: +- repo: self + fetchDepth: 1 + +jobs: +- job: linux_clang + displayName: linux-clang + condition: succeeded() + pool: Hosted Ubuntu 1604 + steps: + - bash: | + test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1 + + sudo apt-get update && + sudo apt-get -y install git gcc make libssl-dev libcurl4-openssl-dev libexpat-dev tcl tk gettext git-email zlib1g-dev apache2-bin && + + export CC=clang || exit 1 + + ci/install-dependencies.sh || exit 1 + ci/run-build-and-tests.sh || { + ci/print-test-failures.sh + exit 1 + } + + test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || exit 1 + displayName: 'ci/run-build-and-tests.sh' + env: + GITFILESHAREPWD: $(gitfileshare.pwd) + - task: PublishTestResults@2 + displayName: 'Publish Test Results **/TEST-*.xml' + inputs: + mergeTestResults: true + testRunTitle: 'linux-clang' + platform: Linux + publishRunAttachments: false + condition: succeededOrFailed() + - task: PublishBuildArtifacts@1 + displayName: 'Publish trash directories of failed tests' + condition: failed() + inputs: + PathtoPublish: t/failed-test-artifacts + ArtifactName: failed-test-artifacts + +- job: linux_gcc + displayName: linux-gcc + condition: succeeded() + pool: Hosted Ubuntu 1604 + steps: + - bash: | + test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1 + + sudo add-apt-repository ppa:ubuntu-toolchain-r/test && + sudo apt-get update && + sudo apt-get -y install git gcc make libssl-dev libcurl4-openssl-dev libexpat-dev tcl tk gettext git-email zlib1g-dev apache2 language-pack-is git-svn gcc-8 || exit 1 + + ci/install-dependencies.sh || exit 1 + ci/run-build-and-tests.sh || { + ci/print-test-failures.sh + exit 1 + } + + test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || exit 1 + displayName: 'ci/run-build-and-tests.sh' + env: + GITFILESHAREPWD: $(gitfileshare.pwd) + - task: PublishTestResults@2 + displayName: 'Publish Test Results **/TEST-*.xml' + inputs: + mergeTestResults: true + testRunTitle: 'linux-gcc' + platform: Linux + publishRunAttachments: false + condition: succeededOrFailed() + - task: PublishBuildArtifacts@1 + displayName: 'Publish trash directories of failed tests' + condition: failed() + inputs: + PathtoPublish: t/failed-test-artifacts + ArtifactName: failed-test-artifacts + +- job: osx_clang + displayName: osx-clang + condition: succeeded() + pool: Hosted macOS + steps: + - bash: | + test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1 + + export CC=clang + + ci/install-dependencies.sh || exit 1 + ci/run-build-and-tests.sh || { + ci/print-test-failures.sh + exit 1 + } + + test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || umount "$HOME/test-cache" || exit 1 + displayName: 'ci/run-build-and-tests.sh' + env: + GITFILESHAREPWD: $(gitfileshare.pwd) + - task: PublishTestResults@2 + displayName: 'Publish Test Results **/TEST-*.xml' + inputs: + mergeTestResults: true + testRunTitle: 'osx-clang' + platform: macOS + publishRunAttachments: false + condition: succeededOrFailed() + - task: PublishBuildArtifacts@1 + displayName: 'Publish trash directories of failed tests' + condition: failed() + inputs: + PathtoPublish: t/failed-test-artifacts + ArtifactName: failed-test-artifacts + +- job: osx_gcc + displayName: osx-gcc + condition: succeeded() + pool: Hosted macOS + steps: + - bash: | + test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1 + + ci/install-dependencies.sh || exit 1 + ci/run-build-and-tests.sh || { + ci/print-test-failures.sh + exit 1 + } + + test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || umount "$HOME/test-cache" || exit 1 + displayName: 'ci/run-build-and-tests.sh' + env: + GITFILESHAREPWD: $(gitfileshare.pwd) + - task: PublishTestResults@2 + displayName: 'Publish Test Results **/TEST-*.xml' + inputs: + mergeTestResults: true + testRunTitle: 'osx-gcc' + platform: macOS + publishRunAttachments: false + condition: succeededOrFailed() + - task: PublishBuildArtifacts@1 + displayName: 'Publish trash directories of failed tests' + condition: failed() + inputs: + PathtoPublish: t/failed-test-artifacts + ArtifactName: failed-test-artifacts + +- job: gettext_poison + displayName: GETTEXT_POISON + condition: succeeded() + pool: Hosted Ubuntu 1604 + steps: + - bash: | + test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1 + + sudo apt-get update && + sudo apt-get -y install git gcc make libssl-dev libcurl4-openssl-dev libexpat-dev tcl tk gettext git-email zlib1g-dev && + + export jobname=GETTEXT_POISON || exit 1 + + ci/run-build-and-tests.sh || { + ci/print-test-failures.sh + exit 1 + } + + test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || exit 1 + displayName: 'ci/run-build-and-tests.sh' + env: + GITFILESHAREPWD: $(gitfileshare.pwd) + - task: PublishTestResults@2 + displayName: 'Publish Test Results **/TEST-*.xml' + inputs: + mergeTestResults: true + testRunTitle: 'gettext-poison' + platform: Linux + publishRunAttachments: false + condition: succeededOrFailed() + - task: PublishBuildArtifacts@1 + displayName: 'Publish trash directories of failed tests' + condition: failed() + inputs: + PathtoPublish: t/failed-test-artifacts + ArtifactName: failed-test-artifacts + +- job: linux32 + displayName: Linux32 + condition: succeeded() + pool: Hosted Ubuntu 1604 + steps: + - bash: | + test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1 + + res=0 + sudo AGENT_OS="$AGENT_OS" BUILD_BUILDNUMBER="$BUILD_BUILDNUMBER" BUILD_REPOSITORY_URI="$BUILD_REPOSITORY_URI" BUILD_SOURCEBRANCH="$BUILD_SOURCEBRANCH" BUILD_SOURCEVERSION="$BUILD_SOURCEVERSION" SYSTEM_PHASENAME="$SYSTEM_PHASENAME" SYSTEM_TASKDEFINITIONSURI="$SYSTEM_TASKDEFINITIONSURI" SYSTEM_TEAMPROJECT="$SYSTEM_TEAMPROJECT" CC=$CC MAKEFLAGS="$MAKEFLAGS" bash -lxc ci/run-linux32-docker.sh || res=1 + + sudo chmod a+r t/out/TEST-*.xml + test ! -d t/failed-test-artifacts || sudo chmod a+r t/failed-test-artifacts + + test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || res=1 + exit $res + displayName: 'ci/run-linux32-docker.sh' + env: + GITFILESHAREPWD: $(gitfileshare.pwd) + - task: PublishTestResults@2 + displayName: 'Publish Test Results **/TEST-*.xml' + inputs: + mergeTestResults: true + testRunTitle: 'linux32' + platform: Linux + publishRunAttachments: false + condition: succeededOrFailed() + - task: PublishBuildArtifacts@1 + displayName: 'Publish trash directories of failed tests' + condition: failed() + inputs: + PathtoPublish: t/failed-test-artifacts + ArtifactName: failed-test-artifacts + +- job: static_analysis + displayName: StaticAnalysis + condition: succeeded() + pool: Hosted Ubuntu 1604 + steps: + - bash: | + test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1 + + sudo apt-get update && + sudo apt-get install -y coccinelle && + + export jobname=StaticAnalysis && + + ci/run-static-analysis.sh || exit 1 + + test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || exit 1 + displayName: 'ci/run-static-analysis.sh' + env: + GITFILESHAREPWD: $(gitfileshare.pwd) + +- job: documentation + displayName: Documentation + condition: succeeded() + pool: Hosted Ubuntu 1604 + steps: + - bash: | + test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1 + + sudo apt-get update && + sudo apt-get install -y asciidoc xmlto asciidoctor && + + export ALREADY_HAVE_ASCIIDOCTOR=yes. && + export jobname=Documentation && + + ci/test-documentation.sh || exit 1 + + test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || exit 1 + displayName: 'ci/test-documentation.sh' + env: + GITFILESHAREPWD: $(gitfileshare.pwd) diff --git a/ci/mount-fileshare.sh b/ci/mount-fileshare.sh new file mode 100755 index 0000000000..26b58a8096 --- /dev/null +++ b/ci/mount-fileshare.sh @@ -0,0 +1,25 @@ +#!/bin/sh + +die () { + echo "$*" >&2 + exit 1 +} + +test $# = 4 || +die "Usage: $0 " + +mkdir -p "$4" || die "Could not create $4" + +case "$(uname -s)" in +Linux) + sudo mount -t cifs -o vers=3.0,username="$2",password="$3",dir_mode=0777,file_mode=0777,serverino "$1" "$4" + ;; +Darwin) + pass="$(echo "$3" | sed -e 's/\//%2F/g' -e 's/+/%2B/g')" && + mount -t smbfs,soft "smb://$2:$pass@${1#//}" "$4" + ;; +*) + die "No support for $(uname -s)" + ;; +esac || +die "Could not mount $4" From patchwork Sun Jan 27 23:26:57 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee via GitGitGadget X-Patchwork-Id: 10783101 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8753E13B5 for ; Sun, 27 Jan 2019 23:27:27 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 75F01286BF for ; Sun, 27 Jan 2019 23:27:27 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6A703298EC; Sun, 27 Jan 2019 23:27:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E2580286BF for ; Sun, 27 Jan 2019 23:27:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727044AbfA0X1Z (ORCPT ); Sun, 27 Jan 2019 18:27:25 -0500 Received: from mail-ed1-f53.google.com ([209.85.208.53]:44163 "EHLO mail-ed1-f53.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726302AbfA0X1A (ORCPT ); Sun, 27 Jan 2019 18:27:00 -0500 Received: by mail-ed1-f53.google.com with SMTP id y56so11584363edd.11 for ; Sun, 27 Jan 2019 15:26:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=6QKYRFfu4o2JPn3Gwqn28G35+BXfLMeTFboEI1dxAj4=; b=pPtXoN18NSTrLf+34+ECumO1LGk4gR3E1ZsgNR/MF93W/oyD3dr7v6aXNVXN6vQ0WM Mx+EqPLutz7C0xG7UpobbvQKloF19va4xwqUCQEri0xSJxoKDiY5scjbH9wB7VWRJHyK y3Qh0UJjL9uskp+CDUlxbiext+jVutn84THTq4Tc+UDPS2F2yMLvVT751nVAW572z7vb LGeBG+GCGVmUc5twRxtXUrfpYRYvClbIB9X4u48cT71mlUjGHR+0Fwc21OxaPOvlJ2nD z9MZkvd5uJCEXcdMhMLhR+6qmV5NuP9J4PGduAl7471IeX28JRCW8ZwMV89JropYAtw6 sIVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=6QKYRFfu4o2JPn3Gwqn28G35+BXfLMeTFboEI1dxAj4=; b=qWhO8tUwTbshLJ1zCNRYMa5/Sjs+WjfJr5WGFobT8YOpD5eGlmhyVB0n6ZZqjyKZau pi8HRVXZKJAP3j7esl1Sr4BMh8/HHgvHAB/WSx2oUpirG+1x16oZizTc6+XDvytvWfDY cOc0aiD+/r/9l29uxFIktfzuUOpj17lsg+Un2JRR758ID2QeCNhhDLvS3xeNa8Qyi2t8 m7Mn9URvwe8JUy4UyCR1h+A7Mk4FgEoqDTe/UADbBO+kjvBJDfQ1lCKLNP2kn16ywxuz uk7lrTJDdxD4tufT/ZalV5P+S/Hmt3clEIm1dnYmjyTzGglgSUXjM+vjbPo1WAzVgjFh pHAg== X-Gm-Message-State: AJcUukc0ssyU0bHEuCeTpgfveDBqA98F8+pVFi3WFzMJSxa2drHc8wRK vzbYBWIUhslUPq2QsGP741ju5Qrn X-Google-Smtp-Source: ALg8bN7FF+g0gJxqvOPMU+nlDywlSAhDSJjGhm4ZFAuXc+aV/dsnebcn80CIv8XhcZrqee2xlbBt3g== X-Received: by 2002:a50:d5dc:: with SMTP id g28mr10000104edj.226.1548631617584; Sun, 27 Jan 2019 15:26:57 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id j31sm13612830eda.46.2019.01.27.15.26.56 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 27 Jan 2019 15:26:57 -0800 (PST) Date: Sun, 27 Jan 2019 15:26:57 -0800 (PST) X-Google-Original-Date: Sun, 27 Jan 2019 23:26:36 GMT Message-Id: <3a2993c826904a92f5a3198e8a5764991a7d430c.1548631607.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Johannes Schindelin via GitGitGadget" Subject: [PATCH v5 10/21] ci: add a Windows job to the Azure Pipelines definition Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Johannes Schindelin Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Johannes Schindelin Previously, we did not have robust support for Windows in our CI definition, simply because Travis cannot accommodate our needs (even after Travis added experimental Windows support very recently, it takes longer than Travis' 50 minute timeout to build Git and run the test suite on Windows). Instead, we used a hack that started a dedicated Azure Pipeline from Travis and waited for the output, often timing out (which is quite fragile, as we found out). With this commit, we finally have first-class support for Windows in our CI definition (in the Azure Pipelines one, that is). Due to our reliance on Unix shell scripting in the test suite, combined with the challenges on executing such scripts on Windows, the Windows job currently takes a whopping ~1h20m to complete. Which is *far* longer than the next-longest job takes (linux-gcc, ~35m). Now, Azure Pipelines's free tier for open source projects (such as Git) offers up to 10 concurrent jobs for free, meaning that the overall run time will be dominated by the slowest job(s). Therefore, it makes sense to start the Windows job first, to minimize the time the entire build takes from start to end (which is now pretty safely the run time of the Windows job). Signed-off-by: Johannes Schindelin --- azure-pipelines.yml | 91 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 91 insertions(+) diff --git a/azure-pipelines.yml b/azure-pipelines.yml index 8cdef105c6..0f7b2125a1 100644 --- a/azure-pipelines.yml +++ b/azure-pipelines.yml @@ -3,6 +3,97 @@ resources: fetchDepth: 1 jobs: +- job: windows + displayName: Windows + condition: succeeded() + pool: Hosted + timeoutInMinutes: 240 + steps: + - powershell: | + if ("$GITFILESHAREPWD" -ne "" -and "$GITFILESHAREPWD" -ne "`$`(gitfileshare.pwd)") { + net use s: \\gitfileshare.file.core.windows.net\test-cache "$GITFILESHAREPWD" /user:AZURE\gitfileshare /persistent:no + cmd /c mklink /d "$(Build.SourcesDirectory)\test-cache" S:\ + } + displayName: 'Mount test-cache' + env: + GITFILESHAREPWD: $(gitfileshare.pwd) + - powershell: | + # Helper to check the error level of the latest command (exit with error when appropriate) + function c() { if (!$?) { exit(1) } } + + # Add build agent's MinGit to PATH + $env:PATH = $env:AGENT_HOMEDIRECTORY +"\externals\\git\cmd;" +$env:PATH + + # Helper to initialize (or update) a Git worktree + function init ($path, $url, $set_origin) { + if (Test-Path $path) { + cd $path; c + if (Test-Path .git) { + & git init; c + } else { + & git status + } + } else { + & git init $path; c + cd $path; c + } + & git config core.autocrlf false; c + & git config core.untrackedCache true; c + if (($set_origin -ne 0) -and !(git config remote.origin.url)) { + & git remote add origin $url; c + } + & git fetch --depth=1 $url master; c + & git reset --hard FETCH_HEAD; c + & git clean -df; c + } + + # Initialize Git for Windows' SDK + $sdk_path = "$(Build.SourcesDirectory)\git-sdk-64" + init "$sdk_path" "https://dev.azure.com/git-for-windows/git-sdk-64/_git/git-sdk-64" 0 + + # Let Git ignore the SDK and the test-cache + "/git-sdk-64/`n/test-cache/`n" | Out-File -NoNewLine -Encoding ascii -Append "$(Build.SourcesDirectory)\.git\info\exclude" + displayName: 'Initialize the Git for Windows SDK' + - powershell: | + & "git-sdk-64\git-cmd.exe" --command=usr\\bin\\bash.exe -lc @" + export DEVELOPER=1 + export NO_PERL=1 + export NO_SVN_TESTS=1 + export GIT_TEST_SKIP_REBASE_P=1 + + ci/run-build-and-tests.sh || { + ci/print-test-failures.sh + exit 1 + } + "@ + if (!$?) { exit(1) } + displayName: 'Build & Test' + env: + HOME: $(Build.SourcesDirectory) + MSYSTEM: MINGW64 + - powershell: | + if ("$GITFILESHAREPWD" -ne "" -and "$GITFILESHAREPWD" -ne "`$`(gitfileshare.pwd)") { + cmd /c rmdir "$(Build.SourcesDirectory)\test-cache" + } + displayName: 'Unmount test-cache' + condition: true + env: + GITFILESHAREPWD: $(gitfileshare.pwd) + - task: PublishTestResults@2 + displayName: 'Publish Test Results **/TEST-*.xml' + inputs: + mergeTestResults: true + testRunTitle: 'windows' + platform: Windows + publishRunAttachments: false + condition: succeededOrFailed() + - task: PublishBuildArtifacts@1 + displayName: 'Publish trash directories of failed tests' + condition: failed() + inputs: + PathtoPublish: t/failed-test-artifacts + ArtifactName: failed-test-artifacts + - job: linux_clang displayName: linux-clang condition: succeeded() From patchwork Sun Jan 27 23:26:57 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee via GitGitGadget X-Patchwork-Id: 10783091 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6085313B5 for ; Sun, 27 Jan 2019 23:27:19 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4DDA3286BF for ; Sun, 27 Jan 2019 23:27:19 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 40FC8298EC; Sun, 27 Jan 2019 23:27:19 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BFC62286BF for ; Sun, 27 Jan 2019 23:27:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727029AbfA0X1Q (ORCPT ); Sun, 27 Jan 2019 18:27:16 -0500 Received: from mail-ed1-f46.google.com ([209.85.208.46]:36223 "EHLO mail-ed1-f46.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726888AbfA0X1B (ORCPT ); Sun, 27 Jan 2019 18:27:01 -0500 Received: by mail-ed1-f46.google.com with SMTP id f23so11603865edb.3 for ; Sun, 27 Jan 2019 15:26:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=78IrL+5/ULRbiBWonyqqOoF9pTaUuqJK4U/mjC0vwos=; b=EYyVZ8JUrMpgx0x8445+5kZiyD1w0ao38QS0fny295lCmYXOjkJKyaEq5nXiCe+Sek bcIPqP/VJicptWBHd4H45Z/SO3JY5/d1C5zfRSrveq1SEKLcxfvktxdYViEBop02EPA0 vKpUYFk2MakbYc0K9ZCJnkstkjm1Pu8Mb93kNJgb6C1kpSiwOiuMBDGioZyHMAcagDOn rKuM5RfTlrhRp3iv7eHyTvZElVhpIiWQbV4qLDLjFHSscYdU5lo+SU3mkaB7PUTDbiVG HNiHbXJjh9su2vD2cu19GHnt1Z241htmk5TFyHxkJmhiMgo9gSbexBBea0kR5rmvkr2f Z9ZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=78IrL+5/ULRbiBWonyqqOoF9pTaUuqJK4U/mjC0vwos=; b=uaxGIa3N0iKLESminJ/SfufwkXYmiBPKuGCOxsNE2im66GNLrhxfH+laapBxpagyDR wmu3DKFftYVOZjJQTyx4bOed3lzVmMBbEZcmenydGnGH/8tGu87Z89dde8ksI3ePAdhp snTZ7Ms2atbXWEeEdGT67rznAQXaLaJ95U8va10foBuvvfLEIQ8AYrcV3SqJAgLrtU7Q pgqhTR2BweXaXHasjZp75RdXyxcKWDPUjTC4qwb0e66GqxByIFPDFYj1X9bBX5VfaH1r 2fEg4TyXHR5DOtRErArS+De2HTw18pruyafw/NJU1Xig/01P2ra3kS7S8Gjiywd68HIS kzWw== X-Gm-Message-State: AJcUukdBtcU8XATTPxBFEWt5ULcRNwK3rcti8hssyusij/5W8VBKDDJc 02i0uEiKmiDfQiF0hzmctBnRYyyE X-Google-Smtp-Source: ALg8bN6ziNNoh2PO9a3vWni1u190bxr404KEpflnLHyWwCQwWyrNGIry9yWGnPT3xuqbbBaX3ESt5A== X-Received: by 2002:a17:906:118c:: with SMTP id n12mr12687163eja.228.1548631618476; Sun, 27 Jan 2019 15:26:58 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id w31sm13394557edw.82.2019.01.27.15.26.57 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 27 Jan 2019 15:26:57 -0800 (PST) Date: Sun, 27 Jan 2019 15:26:57 -0800 (PST) X-Google-Original-Date: Sun, 27 Jan 2019 23:26:37 GMT Message-Id: <52d63018c9d4ef0374fe0dca58fc1f8a856736c9.1548631607.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Johannes Schindelin via GitGitGadget" Subject: [PATCH v5 11/21] ci: use git-sdk-64-minimal build artifact Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Johannes Schindelin Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Johannes Schindelin Instead of a shallow fetch followed by a sparse checkout, we are better off by using a separate, dedicated Pipeline that bundles the SDK as a build artifact, and then consuming that build artifact here. In fact, since this artifact will be used a lot, we spent substantial time on figuring out a minimal subset of the Git for Windows SDK, just enough to build and test Git. The result is a size reduction from around 1GB (compressed) to around 55MB (compressed). This also comes with the change where we now call `usr\bin\bash.exe` directly, as `git-cmd.exe` is not included in the minimal SDK. That reduces the time to initialize Git for Windows' SDK from anywhere between 2m30s-7m to a little over 1m. Note: in theory, we could also use the DownloadBuildArtifacts@0 task here. However, restricted permissions that are in effect when building from forks would let this fail for PR builds, defeating the whole purpose of the Azure Pipelines support for git.git. Signed-off-by: Johannes Schindelin --- azure-pipelines.yml | 44 +++++++++----------------------------------- 1 file changed, 9 insertions(+), 35 deletions(-) diff --git a/azure-pipelines.yml b/azure-pipelines.yml index 0f7b2125a1..480e841a85 100644 --- a/azure-pipelines.yml +++ b/azure-pipelines.yml @@ -18,44 +18,18 @@ jobs: env: GITFILESHAREPWD: $(gitfileshare.pwd) - powershell: | - # Helper to check the error level of the latest command (exit with error when appropriate) - function c() { if (!$?) { exit(1) } } - - # Add build agent's MinGit to PATH - $env:PATH = $env:AGENT_HOMEDIRECTORY +"\externals\\git\cmd;" +$env:PATH - - # Helper to initialize (or update) a Git worktree - function init ($path, $url, $set_origin) { - if (Test-Path $path) { - cd $path; c - if (Test-Path .git) { - & git init; c - } else { - & git status - } - } else { - & git init $path; c - cd $path; c - } - & git config core.autocrlf false; c - & git config core.untrackedCache true; c - if (($set_origin -ne 0) -and !(git config remote.origin.url)) { - & git remote add origin $url; c - } - & git fetch --depth=1 $url master; c - & git reset --hard FETCH_HEAD; c - & git clean -df; c - } - - # Initialize Git for Windows' SDK - $sdk_path = "$(Build.SourcesDirectory)\git-sdk-64" - init "$sdk_path" "https://dev.azure.com/git-for-windows/git-sdk-64/_git/git-sdk-64" 0 + $urlbase = "https://dev.azure.com/git-for-windows/git/_apis/build/builds" + $id = ((Invoke-WebRequest -UseBasicParsing "${urlbase}?definitions=22&statusFilter=completed&resultFilter=succeeded&`$top=1").content | ConvertFrom-JSON).value[0].id + $downloadUrl = ((Invoke-WebRequest -UseBasicParsing "${urlbase}/$id/artifacts").content | ConvertFrom-JSON).value[1].resource.downloadUrl + (New-Object Net.WebClient).DownloadFile($downloadUrl,"git-sdk-64-minimal.zip") + Expand-Archive git-sdk-64-minimal.zip -DestinationPath . -Force + Remove-Item git-sdk-64-minimal.zip # Let Git ignore the SDK and the test-cache - "/git-sdk-64/`n/test-cache/`n" | Out-File -NoNewLine -Encoding ascii -Append "$(Build.SourcesDirectory)\.git\info\exclude" - displayName: 'Initialize the Git for Windows SDK' + "/git-sdk-64-minimal/`n/test-cache/`n" | Out-File -NoNewLine -Encoding ascii -Append "$(Build.SourcesDirectory)\.git\info\exclude" + displayName: 'Download git-sdk-64-minimal' - powershell: | - & "git-sdk-64\git-cmd.exe" --command=usr\\bin\\bash.exe -lc @" + & git-sdk-64-minimal\usr\bin\bash.exe -lc @" export DEVELOPER=1 export NO_PERL=1 export NO_SVN_TESTS=1 From patchwork Sun Jan 27 23:26:58 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee via GitGitGadget X-Patchwork-Id: 10783089 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C463D6C2 for ; Sun, 27 Jan 2019 23:27:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B433F286BF for ; Sun, 27 Jan 2019 23:27:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A8F8B298EC; Sun, 27 Jan 2019 23:27:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 580DC286BF for ; Sun, 27 Jan 2019 23:27:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727009AbfA0X1P (ORCPT ); Sun, 27 Jan 2019 18:27:15 -0500 Received: from mail-ed1-f41.google.com ([209.85.208.41]:42124 "EHLO mail-ed1-f41.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726895AbfA0X1B (ORCPT ); Sun, 27 Jan 2019 18:27:01 -0500 Received: by mail-ed1-f41.google.com with SMTP id y20so11587538edw.9 for ; Sun, 27 Jan 2019 15:27:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=Nx9l6UGI/Mo+cXJbFVbhoaNDqQugjJei4UPiehgiQbs=; b=kZQPc80Z9BTecGNg59oKnUah9ba5RVMMANIFoMzcJLx2Zz8Zfw6buDzcGGr9oRMS6/ jmw/a+q2HPK9XJGt0dmypqvH94q7p7SzF17DkUE8unk8lwBlFqIw67UgKBfShlCBgfkX fAofePIbW95BZ887uhiLshdYOiMW1aqs6HUewsTaH2mA5/0GAD3+heeZQ3AEE/NXohx6 lNrMw8Szp8Ma0H6aDBHGkR68yDYx9WTFAGO12Tjz+X1x/MKP6snEOXasr+ci5O81FPnJ aeZRJBzmhi0aVfBRnnEX+usgUzVwEPqkYz5RPOAFie/4Eo5HJuv2MliRSGQ44XeOwtXn CKzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=Nx9l6UGI/Mo+cXJbFVbhoaNDqQugjJei4UPiehgiQbs=; b=XewwkexRbhVwWbiW7gEo1qqdhsqPlhK95BR0LA2OwgeZAoyLuN16NHBh9FpCaRmNz6 ChPPe5mECObjxvKX0knZoBPmgz6bTaVhlBen44+RUhK0NSRTZ3aYKy+jdRDugaepB8Dv 2NA/2JRguHdwagO8m4Kc38aj801KU3chlQ8IxTAE8wUF1inOpi0cfktYnCmvsI73ir8m k024bMLyMlCpPw8MjdtS9P6VeAwJvG+MSkDNTgYEoVyW6tW1z6rn9prJ+Cq6iaGa9pwX Ah0yoadNDpWt4NiDlwdneLh+k5WBdu1PlVmLCMXWOxTOodxDrwb6HtVwY0TqkY5DUwL2 3EJQ== X-Gm-Message-State: AJcUukfhU2GgSJuL3E+TrX3uGRPxxB9CCD2xSbmCxQxvHlmtj2A7kYMc 7rAFTt8QunXNTTrzZJmoOhsxH8jD X-Google-Smtp-Source: ALg8bN4ICoPzsUJ1uGg5WUfp+Px8ixFyu6ZxSfMoTroSZThIn06w0RHY9nwtmydaP1H/bf6u/WxFyQ== X-Received: by 2002:a17:906:d191:: with SMTP id c17-v6mr17068232ejz.27.1548631619336; Sun, 27 Jan 2019 15:26:59 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id c30sm13447633edc.70.2019.01.27.15.26.58 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 27 Jan 2019 15:26:58 -0800 (PST) Date: Sun, 27 Jan 2019 15:26:58 -0800 (PST) X-Google-Original-Date: Sun, 27 Jan 2019 23:26:38 GMT Message-Id: <1cef14e4c04e27751445397538bc824154a9d42e.1548631608.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Johannes Schindelin via GitGitGadget" Subject: [PATCH v5 12/21] mingw: be more generous when wrapping up the setitimer() emulation Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Johannes Schindelin Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Johannes Schindelin Every once in a while, the Azure Pipeline fails with some semi-random error: timer thread did not terminate timely This error message means that the thread that is used to emulate the setitimer() function did not terminate within 1,000 milliseconds. The most likely explanation (and therefore the one we should assume to be true, according to Occam's Razor) is that the timeout of one second is simply not enough because we try to run so many tasks in parallel. So let's give it ten seconds instead of only one. That should be enough. Signed-off-by: Johannes Schindelin --- compat/mingw.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/compat/mingw.c b/compat/mingw.c index b459e1a291..e0dfe8844d 100644 --- a/compat/mingw.c +++ b/compat/mingw.c @@ -2100,7 +2100,7 @@ static void stop_timer_thread(void) if (timer_event) SetEvent(timer_event); /* tell thread to terminate */ if (timer_thread) { - int rc = WaitForSingleObject(timer_thread, 1000); + int rc = WaitForSingleObject(timer_thread, 10000); if (rc == WAIT_TIMEOUT) error("timer thread did not terminate timely"); else if (rc != WAIT_OBJECT_0) From patchwork Sun Jan 27 23:26:59 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee via GitGitGadget X-Patchwork-Id: 10783087 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 07FDD13B5 for ; Sun, 27 Jan 2019 23:27:16 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EC519286BF for ; Sun, 27 Jan 2019 23:27:15 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E0E7D298EC; Sun, 27 Jan 2019 23:27:15 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 80382286BF for ; Sun, 27 Jan 2019 23:27:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726998AbfA0X1O (ORCPT ); Sun, 27 Jan 2019 18:27:14 -0500 Received: from mail-ed1-f44.google.com ([209.85.208.44]:44156 "EHLO mail-ed1-f44.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726912AbfA0X1C (ORCPT ); Sun, 27 Jan 2019 18:27:02 -0500 Received: by mail-ed1-f44.google.com with SMTP id y56so11584397edd.11 for ; Sun, 27 Jan 2019 15:27:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=yIFmvH39JP873kVuTr99FvXDnugqBCZqKax9pCZMHbE=; b=kTSv1COUear/65ZncuMu4tj439uyVxbx2gw4JkZmtufr7o6huXGs9PxeVX6ciYy0IH pY1J/rDiAPK9cz4wAzRzTodUvaZF/7/qTW4R/+fqbZKyhQMe3gJOjxciuklcFqIlNAVx slu+Me2E79C6KcyxalSjyC+bcyVkwnheT38dRGou3Lz+rOohQlM+r2G9xBEJitQjzKDr FfsUPsVvZJKSEPrIiRQLlPZH96wDIlWtxYlPjG1X9Xso4RktYuO6PEe4Z623fQ9U7G1D QsTBtNc59HvjRISJgNxnPSV7tYQYB2qi48ih58yjdqubGutxfJu9HkFVLzjokKSqDsRi JKhQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=yIFmvH39JP873kVuTr99FvXDnugqBCZqKax9pCZMHbE=; b=tL94/+1PEOasrhvCgswyDi8fb9FzUKEx9GoJEsGhT3A4AnqMEAxKhDnyI8QXrnHmhn tDoCIm/c4z+lmxymRnUjoeEZ4DJ4EkO4AuSe1z9C/o6qozSCmEnbi16zfYywgwjsXmx9 uGYNpvLmigjFVI951W/8JYuV1WmnoaHmBJ4EQdsMckOpDiBwReP0ScqoLIKt4ngErMgm YT7i5Y1TMJWJI3Hg5tT0E8Tnqgo1Q8CWZa0fCCVczW03U6/8abAxMhcvzQ7ZSnMaUNhu T7Tb8vf69Rc2LI7iIj6EcrpRmFN+HjZn0wIuxPZ81O6ShVZfrAx05Ul17P/1tEv9c6/U Suaw== X-Gm-Message-State: AJcUuketH63OKXkLgM8bXL6GGrsxpOCHaAXaxbwlTjJn0oEECAbGCP3B oeNm4AX5ywiuObdk8Etb0pax7Nmu X-Google-Smtp-Source: ALg8bN7QHzPC1rWFioDCkxxufH3OLMOP2vBHpJcLIWMmKtOoE5xm1cKLrZNO+LUUfnxpCcwAoexQdw== X-Received: by 2002:a17:906:a3c8:: with SMTP id ca8-v6mr17434026ejb.88.1548631620140; Sun, 27 Jan 2019 15:27:00 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id c53sm14352818ede.26.2019.01.27.15.26.59 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 27 Jan 2019 15:26:59 -0800 (PST) Date: Sun, 27 Jan 2019 15:26:59 -0800 (PST) X-Google-Original-Date: Sun, 27 Jan 2019 23:26:39 GMT Message-Id: In-Reply-To: References: From: "Johannes Schindelin via GitGitGadget" Subject: [PATCH v5 13/21] README: add a build badge (status of the Azure Pipelines build) Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Johannes Schindelin Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Johannes Schindelin Just like so many other OSS projects, we now also have a build badge. Signed-off-by: Johannes Schindelin --- README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.md b/README.md index f920a42fad..764c480c66 100644 --- a/README.md +++ b/README.md @@ -1,3 +1,5 @@ +[![Build Status](https://dev.azure.com/git/git/_apis/build/status/test-git.git)](https://dev.azure.com/git/git/_build/latest?definitionId=2) + Git - fast, scalable, distributed revision control system ========================================================= From patchwork Sun Jan 27 23:27:00 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee via GitGitGadget X-Patchwork-Id: 10783085 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 51F2113B5 for ; Sun, 27 Jan 2019 23:27:15 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 41F00286BF for ; Sun, 27 Jan 2019 23:27:15 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 36419298EC; Sun, 27 Jan 2019 23:27:15 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BCAAD286BF for ; Sun, 27 Jan 2019 23:27:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726997AbfA0X1N (ORCPT ); Sun, 27 Jan 2019 18:27:13 -0500 Received: from mail-ed1-f68.google.com ([209.85.208.68]:41672 "EHLO mail-ed1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726403AbfA0X1D (ORCPT ); Sun, 27 Jan 2019 18:27:03 -0500 Received: by mail-ed1-f68.google.com with SMTP id a20so11582198edc.8 for ; Sun, 27 Jan 2019 15:27:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=qFU9SlpnDsxWdLk7t1RD4T2tVhhrbl/R3qfgIkC/7lI=; b=bRNJMImM1ImOwOyBfAtb5XNAY9IAIU2y7JPIfgnub/Zdrao64eFTkd5RIN3G/33S5O F8mbDY5xiUhLXzxnpUiyQZdtanMaToG3VfcO5XVSNzEwo3KVwrcKxhkHWrm5Xft7qWTA p3zCIllr8NpiLYtXWhYnlVJPQZEfwxWbhSF+e9rQj8njfghR2YKJ1EcC4W/E3KP7ZKfu L/wU3xmHcVFbumRxzgp4ca5Qwjdt3Xf1FSQL7hBF9SwKKrRNSwz/q4KOQWveBemFmcFG DFs+zoIUo1BiaF48a+dsrGfi6PeXfhsfyuwUI0MLxkUOOcgzJDEL2gA6xABYyaOBSJn3 0ITw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=qFU9SlpnDsxWdLk7t1RD4T2tVhhrbl/R3qfgIkC/7lI=; b=MIpmesY2kcxya7cuu5EoG/wR2IQ41DxVskxK0ZedHRkB42iYkfcpfeZNFweNjpiCQE XmRiIjhhHOjv4BeXHPhFXXsB6aYMgLUV+ywdmFEC+3byBGeZ6vs+lvdyMWugdE+PAIZm ZDL+9PJQrHyeNzvAbnMUVqbxOQD+7tgYkXQJptG0+t8ibHJoB6ej2fDQ7Pn45QoRemm2 tmu5/XXJrxf8PgqvCV4ndrWXj2wlisKVFrQZUQn8NiZXZW8U2jxQWCDkSnWAw8L/6ilR b5Gc3ywn5DltB5CC6gIqjCC/xgXzF7qFKzciKLRliD+SiyWKd/L7EXbjMGy9O/9QTMcs UStg== X-Gm-Message-State: AJcUukcrH6MhImJoeQTQW1vEpB650YD0GyuKiEu2zc9GBCyYFobGrtjq fPPrwsd/FODcYygBjiuGsC8nd82W X-Google-Smtp-Source: AHgI3IZ0sTV9VoHCevLlVtyGjk4GZ8e0HjXQBmj4OwhHuejKAtUPYdsMnNE3pZP6OWB5tGf4XNIFEw== X-Received: by 2002:a17:906:b3c9:: with SMTP id cg9mr1075629ejb.159.1548631621351; Sun, 27 Jan 2019 15:27:01 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id z40sm13397797edz.86.2019.01.27.15.27.00 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 27 Jan 2019 15:27:00 -0800 (PST) Date: Sun, 27 Jan 2019 15:27:00 -0800 (PST) X-Google-Original-Date: Sun, 27 Jan 2019 23:26:40 GMT Message-Id: <4303bbd4e816a110c18ed29e8ec371ae50afaeba.1548631608.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Johannes Schindelin via GitGitGadget" Subject: [PATCH v5 14/21] tests: avoid calling Perl just to determine file sizes Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Johannes Schindelin Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Johannes Schindelin It is a bit ridiculous to spin up a full-blown Perl instance (especially on Windows, where that means spinning up a full POSIX emulation layer, AKA the MSYS2 runtime) just to tell how large a given file is. So let's just use the test-tool to do that job instead. This command will also be used over the next commits, to allow for cutting out individual test cases' verbose log from the file generated via --verbose-log. Signed-off-by: Johannes Schindelin --- t/helper/test-path-utils.c | 12 ++++++++++++ t/t0021-conversion.sh | 2 +- t/t1050-large.sh | 2 +- t/t5315-pack-objects-compression.sh | 2 +- t/t9303-fast-import-compression.sh | 2 +- 5 files changed, 16 insertions(+), 4 deletions(-) diff --git a/t/helper/test-path-utils.c b/t/helper/test-path-utils.c index ae091d9b3e..30211d6d64 100644 --- a/t/helper/test-path-utils.c +++ b/t/helper/test-path-utils.c @@ -291,6 +291,18 @@ int cmd__path_utils(int argc, const char **argv) return !!res; } + if (argc > 2 && !strcmp(argv[1], "file-size")) { + int res = 0, i; + struct stat st; + + for (i = 2; i < argc; i++) + if (stat(argv[i], &st)) + res = error_errno("Cannot stat '%s'", argv[i]); + else + printf("%"PRIuMAX"\n", (uintmax_t)st.st_size); + return !!res; + } + fprintf(stderr, "%s: unknown function name: %s\n", argv[0], argv[1] ? argv[1] : "(there was none)"); return 1; diff --git a/t/t0021-conversion.sh b/t/t0021-conversion.sh index fd5f1ac649..e10f5f787f 100755 --- a/t/t0021-conversion.sh +++ b/t/t0021-conversion.sh @@ -24,7 +24,7 @@ generate_random_characters () { } file_size () { - perl -e 'print -s $ARGV[0]' "$1" + test-tool path-utils file-size "$1" } filter_git () { diff --git a/t/t1050-large.sh b/t/t1050-large.sh index 1a9b21b293..dcb4dbba67 100755 --- a/t/t1050-large.sh +++ b/t/t1050-large.sh @@ -8,7 +8,7 @@ test_description='adding and checking out large blobs' # This should be moved to test-lib.sh together with the # copy in t0021 after both topics have graduated to 'master'. file_size () { - perl -e 'print -s $ARGV[0]' "$1" + test-tool path-utils file-size "$1" } test_expect_success setup ' diff --git a/t/t5315-pack-objects-compression.sh b/t/t5315-pack-objects-compression.sh index 34c47dae09..df970d7584 100755 --- a/t/t5315-pack-objects-compression.sh +++ b/t/t5315-pack-objects-compression.sh @@ -7,7 +7,7 @@ test_description='pack-object compression configuration' # This should be moved to test-lib.sh together with the # copy in t0021 after both topics have graduated to 'master'. file_size () { - perl -e 'print -s $ARGV[0]' "$1" + test-tool path-utils file-size "$1" } test_expect_success setup ' diff --git a/t/t9303-fast-import-compression.sh b/t/t9303-fast-import-compression.sh index 856219f46a..5045f02a53 100755 --- a/t/t9303-fast-import-compression.sh +++ b/t/t9303-fast-import-compression.sh @@ -6,7 +6,7 @@ test_description='compression setting of fast-import utility' # This should be moved to test-lib.sh together with the # copy in t0021 after both topics have graduated to 'master'. file_size () { - perl -e 'print -s $ARGV[0]' "$1" + test-tool path-utils file-size "$1" } import_large () { From patchwork Sun Jan 27 23:27:01 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee via GitGitGadget X-Patchwork-Id: 10783083 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D06406C2 for ; Sun, 27 Jan 2019 23:27:13 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BF29A286BF for ; Sun, 27 Jan 2019 23:27:13 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B37CB298EC; Sun, 27 Jan 2019 23:27:13 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 41A28286BF for ; Sun, 27 Jan 2019 23:27:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726993AbfA0X1L (ORCPT ); Sun, 27 Jan 2019 18:27:11 -0500 Received: from mail-ed1-f52.google.com ([209.85.208.52]:40372 "EHLO mail-ed1-f52.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726920AbfA0X1E (ORCPT ); Sun, 27 Jan 2019 18:27:04 -0500 Received: by mail-ed1-f52.google.com with SMTP id g22so11591045edr.7 for ; Sun, 27 Jan 2019 15:27:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=mErgkvF3RwT1uEOg4N3negTT4Om6xVcCHQyQL+1J7Sw=; b=ia6WC85chyVZJFMn+VHX72+HsU4uTzIx5wRYvjZDUHiT6XxyCGavj7To/8YTplr7Ah AyMUKxtdOPWVTV3HgNx+XN2y/Egio/CdzEnCbKOvIq+zFHUymKbWlO9+jW+kDYA9JzpL bFcRyADK/Us5aLaUNFSelUvdoC4p0z6bPbut4OZX/ZLncZRj5BVBoEcmpUy+AzVxhoBe fuag7WTTp/TA1Nt3JAOZOK4d/P0L4do992KYGcT0QJhLMrKEBSw/NDD9HV9aHaLSXgcP C6Bw88PF+grRUBFMda9FRplLRdFutO4QfuHcVLCEKW6hSqtqW2lyuFC2QLJxMh/S3eD6 7E7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=mErgkvF3RwT1uEOg4N3negTT4Om6xVcCHQyQL+1J7Sw=; b=BRY6/pxdOaxhR5b7jIE/BcQqNm+Ghr4JiTxXz+7hhSBKUTyKazpYjVT9pj2oySe/eV QAyHA6BdwWhbJ616HWNmbYEE+HEEdER+ajG2JWgQCtkSuxdfxQHHAMcyUBUM7Tl06LzD YbFF1OUS1ePeGCrX9cbsveb5G5c3YsQvCryBIuglJqL3CQKoWEQx4czSNynxFDY3sxHl GJAsIFTV5ICUH4mQB7LeNaWLPKT45qj24Nx45CceBHJ3DxcRPcOP5o1WbxQJHBva/82d llUAf4xQOdfmx5iNDCrjhR47Iq7QW3uwmLj2JIDC6D3RhyDhm4n76OL33IomsSymn5nY VXAA== X-Gm-Message-State: AJcUukeuhFqPtIG0fOSOe6QDgkucCLa9yjJKE8r4UUJ3SvMC4v3sDETu sY2OYgWzqzz39GfmXbSqCFalOm+B X-Google-Smtp-Source: ALg8bN5YWm0NRLdhTrvUisiFCpTQ+fuwh/d2klu83Gfmd+4C7kM3FB+ynwdTS81X7/zggtCbwnLFow== X-Received: by 2002:a17:906:258f:: with SMTP id m15-v6mr17505266ejb.191.1548631622161; Sun, 27 Jan 2019 15:27:02 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id r42sm13689214edd.23.2019.01.27.15.27.01 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 27 Jan 2019 15:27:01 -0800 (PST) Date: Sun, 27 Jan 2019 15:27:01 -0800 (PST) X-Google-Original-Date: Sun, 27 Jan 2019 23:26:41 GMT Message-Id: In-Reply-To: References: From: "Johannes Schindelin via GitGitGadget" Subject: [PATCH v5 15/21] tests: include detailed trace logs with --write-junit-xml upon failure Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Johannes Schindelin Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Johannes Schindelin The JUnit XML format lends itself to be presented in a powerful UI, where you can drill down to the information you are interested in very quickly. For test failures, this usually means that you want to see the detailed trace of the failing tests. With Travis CI, we passed the `--verbose-log` option to get those traces. However, that seems excessive, as we do not need/use the logs in almost all of those cases: only when a test fails do we have a way to include the trace. So let's do something different when using Azure DevOps: let's run all the tests with `--quiet` first, and only if a failure is encountered, try to trace the commands as they are executed. Of course, we cannot turn on `--verbose-log` after the fact. So let's just re-run the test with all the same options, adding `--verbose-log`. And then munging the output file into the JUnit XML on the fly. Note: there is an off chance that re-running the test in verbose mode "fixes" the failures (and this does happen from time to time!). That is a possibility we should be able to live with. Ideally, we would label this as "Passed upon rerun", and Azure Pipelines even know about that outcome, but it is not available when using the JUnit XML format for now: https://github.com/Microsoft/azure-pipelines-agent/blob/master/src/Agent.Worker/TestResults/JunitResultReader.cs Signed-off-by: Johannes Schindelin --- t/helper/test-path-utils.c | 21 +++++++++++++++++++++ t/test-lib.sh | 22 +++++++++++++++++++++- 2 files changed, 42 insertions(+), 1 deletion(-) diff --git a/t/helper/test-path-utils.c b/t/helper/test-path-utils.c index 30211d6d64..6efde6f5ba 100644 --- a/t/helper/test-path-utils.c +++ b/t/helper/test-path-utils.c @@ -303,6 +303,27 @@ int cmd__path_utils(int argc, const char **argv) return !!res; } + if (argc == 4 && !strcmp(argv[1], "skip-n-bytes")) { + int fd = open(argv[2], O_RDONLY), offset = atoi(argv[3]); + char buffer[65536]; + + if (fd < 0) + die_errno("could not open '%s'", argv[2]); + if (lseek(fd, offset, SEEK_SET) < 0) + die_errno("could not skip %d bytes", offset); + for (;;) { + ssize_t count = read(fd, buffer, sizeof(buffer)); + if (count < 0) + die_errno("could not read '%s'", argv[2]); + if (!count) + break; + if (write(1, buffer, count) < 0) + die_errno("could not write to stdout"); + } + close(fd); + return 0; + } + fprintf(stderr, "%s: unknown function name: %s\n", argv[0], argv[1] ? argv[1] : "(there was none)"); return 1; diff --git a/t/test-lib.sh b/t/test-lib.sh index a3b2166cb5..f31a1c8f79 100644 --- a/t/test-lib.sh +++ b/t/test-lib.sh @@ -639,8 +639,19 @@ test_failure_ () { junit_insert="" junit_insert="$junit_insert $(xml_attr_encode \ - "$(printf '%s\n' "$@" | sed 1d)")" + "$(if test -n "$GIT_TEST_TEE_OUTPUT_FILE" + then + test-tool path-utils skip-n-bytes \ + "$GIT_TEST_TEE_OUTPUT_FILE" $GIT_TEST_TEE_OFFSET + else + printf '%s\n' "$@" | sed 1d + fi)")" junit_insert="$junit_insert" + if test -n "$GIT_TEST_TEE_OUTPUT_FILE" + then + junit_insert="$junit_insert$(xml_attr_encode \ + "$(cat "$GIT_TEST_TEE_OUTPUT_FILE")")" + fi write_junit_xml_testcase "$1" " $junit_insert" fi test_failure=$(($test_failure + 1)) @@ -931,6 +942,11 @@ test_finish_ () { echo >&3 "" maybe_teardown_valgrind maybe_teardown_verbose + if test -n "$GIT_TEST_TEE_OFFSET" + then + GIT_TEST_TEE_OFFSET=$(test-tool path-utils file-size \ + "$GIT_TEST_TEE_OUTPUT_FILE") + fi } test_skip () { @@ -1280,6 +1296,10 @@ then date +%Y-%m-%dT%H:%M:%S)\"" write_junit_xml --truncate "" " " junit_suite_start=$(test-tool date getnanos) + if test -n "$GIT_TEST_TEE_OUTPUT_FILE" + then + GIT_TEST_TEE_OFFSET=0 + fi fi # Provide an implementation of the 'yes' utility From patchwork Sun Jan 27 23:27:02 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee via GitGitGadget X-Patchwork-Id: 10783081 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CB17B13B5 for ; Sun, 27 Jan 2019 23:27:12 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BAB06286BF for ; Sun, 27 Jan 2019 23:27:12 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AF55D298EC; Sun, 27 Jan 2019 23:27:12 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5A437286BF for ; Sun, 27 Jan 2019 23:27:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726984AbfA0X1K (ORCPT ); Sun, 27 Jan 2019 18:27:10 -0500 Received: from mail-ed1-f46.google.com ([209.85.208.46]:41898 "EHLO mail-ed1-f46.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726926AbfA0X1E (ORCPT ); Sun, 27 Jan 2019 18:27:04 -0500 Received: by mail-ed1-f46.google.com with SMTP id a20so11582213edc.8 for ; Sun, 27 Jan 2019 15:27:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=tz7VD9jZtEnv8yooHJRr2Q/rjZIeYRglb+VYrbixkYA=; b=SkljJnS0roBWhd4KGIqMrxsH4wUTuCyfrkL7ML0w3jEKg++xgzasF+4aGl/xJdXPMw u6Amx6LQUbD7Hhq8wN/0O0VAJwAA2084PZ7HjKWw1AuPwzRrKuYiqBUA8qWN/ZmWGJLn HnXvBwwI07su7EFoZyC0TAl6ZSjR+reyUjFKi/psfQCp035p+lCikaD5hm6D5sr/Cjor emdMZSCYepjPxXNQ5auRXCUN4Z8qFBTcwxGo5j+ux/TILGz2AvrSLPsWuKLl2kKnVJCp izqueroejHf4O1L72eiQ9eMwg0yKUpp10MJdny9JaelXB9hz1o0XmP2Vood1UmxzTQqc QZIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=tz7VD9jZtEnv8yooHJRr2Q/rjZIeYRglb+VYrbixkYA=; b=gR0u+XWRg5fMqCuP1CJ/dfBw7DeRcKvz/4xCBH0e7GZ0iOssLBIzwDChIo5WL2x+v6 Ac4djVGjUKCRJOpGXEKSddCtldKcTTi2mdNJSRcNIvHU6T5rtvgkAMDq9kbVxha7k+3C zbmk/geO8nIT48fZbd8CT17FwXQLUE9Y6/rQwrFu6+M7O5qVTTLRE4DiknVLHbLhld4Z 7AykWPHK+0VT+B7CiLuG4XiH6H1rgaAW2lZZggc96xcCEGrL6ijfVt2jN07lyoOWcULA A/U29C5FB5/luRB52yceNYPwZyi1CmPFaECroe9SJjX3cRVCk7LvoevcVRLkpoC/7sDI Pqxg== X-Gm-Message-State: AJcUukekmCyb4CSln62G9PaZAUT4wN662cSB/fZgypsBGo2IeVNrdlG4 RUrdIgDj6Vr87ZTMu8w3ua1AJ4kL X-Google-Smtp-Source: ALg8bN7EDwCRUoAWUFqyI5PbB0fNV7zQE2uYE007ZOyuLfRbaucXnT5NnW2VuPc5KA+EEYQilijosg== X-Received: by 2002:a17:906:4e19:: with SMTP id z25mr12560278eju.77.1548631622752; Sun, 27 Jan 2019 15:27:02 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id o19sm2401794eja.64.2019.01.27.15.27.02 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 27 Jan 2019 15:27:02 -0800 (PST) Date: Sun, 27 Jan 2019 15:27:02 -0800 (PST) X-Google-Original-Date: Sun, 27 Jan 2019 23:26:42 GMT Message-Id: <9984c6d17bc09fc96c627a7c7fe5ecf83070dffe.1548631608.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Johannes Schindelin via GitGitGadget" Subject: [PATCH v5 16/21] mingw: try to work around issues with the test cleanup Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Johannes Schindelin Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Johannes Schindelin It seems that every once in a while in the Git for Windows SDK, there are some transient file locking issues preventing the test clean up to delete the trash directory. Let's be gentle and try again five seconds later, and only error out if it still fails the second time. This change helps Windows, and does not hurt any other platform (normally, it is highly unlikely that said deletion fails, and if it does, normally it will fail again even 5 seconds later). Signed-off-by: Johannes Schindelin --- t/test-lib.sh | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/t/test-lib.sh b/t/test-lib.sh index f31a1c8f79..9c0ca5effb 100644 --- a/t/test-lib.sh +++ b/t/test-lib.sh @@ -1104,7 +1104,11 @@ test_done () { error "Tests passed but trash directory already removed before test cleanup; aborting" cd "$TRASH_DIRECTORY/.." && - rm -fr "$TRASH_DIRECTORY" || + rm -fr "$TRASH_DIRECTORY" || { + # try again in a bit + sleep 5; + rm -fr "$TRASH_DIRECTORY" + } || error "Tests passed but test cleanup failed; aborting" fi test_at_end_hook_ From patchwork Sun Jan 27 23:27:03 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee via GitGitGadget X-Patchwork-Id: 10783097 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id ED2EF159A for ; Sun, 27 Jan 2019 23:27:23 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DCF51286BF for ; Sun, 27 Jan 2019 23:27:23 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D11D0298EC; Sun, 27 Jan 2019 23:27:23 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 876AF298AD for ; Sun, 27 Jan 2019 23:27:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726968AbfA0X1J (ORCPT ); Sun, 27 Jan 2019 18:27:09 -0500 Received: from mail-ed1-f66.google.com ([209.85.208.66]:35015 "EHLO mail-ed1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726930AbfA0X1F (ORCPT ); Sun, 27 Jan 2019 18:27:05 -0500 Received: by mail-ed1-f66.google.com with SMTP id x30so11634276edx.2 for ; Sun, 27 Jan 2019 15:27:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=6wZF5XyFnvK4tOVLEGObR7qCxdrcyoiAaAwrz8JEDdo=; b=tUKZQODyN+d2cJfU/8r7HN0MCDNYJkXdKmZZ/BFP65FNbeF7MgHXXytyOanZ7tbEgd nl9T6FHSTOA/B59eeyDdJjXxmLWxp0M7CAT+yhW0WPaCtnVPWDQEqUHZD387y6PMbX4z hQpIEOh4piSf/EO3275/rk5OTLr8kRS4Ci5icDr4OLGfKp4qo10rF8YuBjoriQWlxcEy 7OLkw1ayq0x0/ea73PvljTjidOaG9+AGKsys8kDojRoJianM3T5tM3XY8KazUqOStK25 98p3kXK7e2CY/YGtogtaF2L2nJoD174T7xj20WtSXge5T1OulCDYatvebX6B2l0Cd5zQ ZXCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=6wZF5XyFnvK4tOVLEGObR7qCxdrcyoiAaAwrz8JEDdo=; b=hSD5opHVQyo39LQqj5VK57vwD8sAoyhPs1/v9yyt9v1NSXzlTd30TxQ77oYqzaW3Zd R+T9S8ayFlj9/pKFkOE4dG2veUcm3XSx9fvqr/LNuH1lis7sj8iuvkLFCmMjQZ/9KEgA 71efUMpT/qpDQtWF/2hJz/2xz/gDa2rN5vUUtcC32Mm3aTAkI3Ro1ixDqbYD0sIvSFr2 M66Cl/xY2Wujtn8LBs993dg2gzmeGybN6/XHEFPt8ST/lUbyqU67S6IGfj+lZ8/WyV63 53c6u4etA28ZKbqGe3O5UUvTVDdGpYcsHQpcqFXTkk69pnt1/T8znn87qbcVrasg4b4S pr3g== X-Gm-Message-State: AJcUukfURv2Q3IW7dBlJJITtwpQKc9lWlPpUcRXoAMpl/GCSi9isWYUE rcQA4sEmAZky74J8ko1pS7korXin X-Google-Smtp-Source: ALg8bN59dEn4Dms1jK1ttCBmajWsZdYmLAVrCSLW1uR5hwCNBLBnWhXFniYcDLj5mteHj3EFloX5hw== X-Received: by 2002:a17:906:1c86:: with SMTP id g6-v6mr17478580ejh.195.1548631623513; Sun, 27 Jan 2019 15:27:03 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id gf12sm755708ejb.26.2019.01.27.15.27.02 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 27 Jan 2019 15:27:03 -0800 (PST) Date: Sun, 27 Jan 2019 15:27:03 -0800 (PST) X-Google-Original-Date: Sun, 27 Jan 2019 23:26:43 GMT Message-Id: In-Reply-To: References: From: "Johannes Schindelin via GitGitGadget" Subject: [PATCH v5 17/21] tests: add t/helper/ to the PATH with --with-dashes Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Johannes Schindelin Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Johannes Schindelin We really need to be able to find the test helpers... Really. This change was forgotten when we moved the test helpers into t/helper/ Signed-off-by: Johannes Schindelin --- t/test-lib.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/t/test-lib.sh b/t/test-lib.sh index 9c0ca5effb..c790e98fd2 100644 --- a/t/test-lib.sh +++ b/t/test-lib.sh @@ -1227,7 +1227,7 @@ else # normal case, use ../bin-wrappers only unless $with_dashes: GIT_EXEC_PATH=$GIT_BUILD_DIR if test -n "$with_dashes" then - PATH="$GIT_BUILD_DIR:$PATH" + PATH="$GIT_BUILD_DIR:$GIT_BUILD_DIR/t/helper:$PATH" fi fi GIT_TEMPLATE_DIR="$GIT_BUILD_DIR"/templates/blt From patchwork Sun Jan 27 23:27:03 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee via GitGitGadget X-Patchwork-Id: 10783079 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4F69313B5 for ; Sun, 27 Jan 2019 23:27:11 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3F770298AD for ; Sun, 27 Jan 2019 23:27:11 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 33CEF298EC; Sun, 27 Jan 2019 23:27:11 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C6B8C298AD for ; Sun, 27 Jan 2019 23:27:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726979AbfA0X1J (ORCPT ); Sun, 27 Jan 2019 18:27:09 -0500 Received: from mail-ed1-f65.google.com ([209.85.208.65]:39154 "EHLO mail-ed1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726934AbfA0X1G (ORCPT ); Sun, 27 Jan 2019 18:27:06 -0500 Received: by mail-ed1-f65.google.com with SMTP id b14so11593771edt.6 for ; Sun, 27 Jan 2019 15:27:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=dxzqfZAxZ+TPwgPiHydUMW7hZp+IB66oCyS61ZHth2s=; b=ZPf/7CM5qjB4x6/9SZJ1oXIAzNMiXy7PBbdJ8wAV755CwCb6Gq2aeNDbQcTSNArQwi QfcRnYoKB9KqYx4ez7I4ZNzW+O94jZnvoP35FyRZyltg4gDBUwmgJyPT6pXFSxtkD4wQ Y5nkVCZy1gdEzoWBnhwPhKP9Lhe3+kTreIsLslPNrYNUY/x4EyWjNrPgQvlKqtLPZMZs EYMFEKpTNJ4ol9o5a0/axE34sxBAhl8M/G9Rz47zz0aOhYdTf15gmfEhrKpmaW/44QqZ b3Bxnhuotd17raua5ecR0dcGOBwS944iD30y738tkBuhvYK1xyv8MBLRKBVyYwa5UaHZ SiPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=dxzqfZAxZ+TPwgPiHydUMW7hZp+IB66oCyS61ZHth2s=; b=g4YRH8PNqgGjCQaIb6IIlNnkSSRPdSkTxX4+pf4KRjAt8ra7EtkrpUDy4wNmdP0rJe /qjdVnmwxR4d0DCcHHnLoANw2FEJVTDipfS/tjZEgzjelBx/qoArlohRe6WZr7i4BVe2 C933//OenV2FTVAD4UAGMOW4scnEItwY8zmf5BjNJt6xWv1E0ZWeVd0stO/vumBBggN+ fsSFpgU8n1dLpcODixNsOJWnFvMJMkt1SrongXuXx4bh+/WCd2OxmaKE+a5vOo8a9kF3 mPB2yJoZznZ+aVDqaWEvRAY87lloc1cDHKm/9lDnbYoPDtYbMyqkhiQDdarkyyokYw2M pskA== X-Gm-Message-State: AJcUukdjsZHiLt3cstQxJkwTus8kWOq54M+5oGW5V1rMQwv7h5lNNvEO RbaY7pzVXreu5/sDEDjXN1bHsJ6E X-Google-Smtp-Source: ALg8bN5/IdBCuloGLRZnImkf1EeJvr6M6fSiQMknpJ+rVvu+liie+2SKit1/+RJKqehzmwxsYOWDpA== X-Received: by 2002:aa7:d749:: with SMTP id a9mr18848627eds.223.1548631624400; Sun, 27 Jan 2019 15:27:04 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id v25sm2320962ejj.8.2019.01.27.15.27.03 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 27 Jan 2019 15:27:03 -0800 (PST) Date: Sun, 27 Jan 2019 15:27:03 -0800 (PST) X-Google-Original-Date: Sun, 27 Jan 2019 23:26:44 GMT Message-Id: <7518266a19e4167fbe6459ff54f452573176c296.1548631608.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Johannes Schindelin via GitGitGadget" Subject: [PATCH v5 18/21] t0061: fix with --with-dashes and RUNTIME_PREFIX Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Johannes Schindelin Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Johannes Schindelin When building Git with RUNTIME_PREFIX and starting a test helper from t/helper/, it fails to detect the system prefix correctly. This is the reason that the warning RUNTIME_PREFIX requested, but prefix computation failed. [...] to be printed. In t0061, we did not expect that to happen, and it actually did not happen in the normal case, because bin-wrappers/test-tool specifically sets GIT_TEXTDOMAINDIR (and as a consequence, nothing in test-tool wants to know about the runtime prefix). However, with --with-dashes, bin-wrappers/test-tool is no longer called, but t/helper/test-tool is called directly. So let's just ignore the RUNTIME_PREFIX warning. Signed-off-by: Johannes Schindelin --- t/t0061-run-command.sh | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/t/t0061-run-command.sh b/t/t0061-run-command.sh index 99a614bc7c..5a2d087bf0 100755 --- a/t/t0061-run-command.sh +++ b/t/t0061-run-command.sh @@ -166,7 +166,8 @@ test_trace () { expect="$1" shift GIT_TRACE=1 test-tool run-command "$@" run-command true 2>&1 >/dev/null | \ - sed -e 's/.* run_command: //' -e '/trace: .*/d' >actual && + sed -e 's/.* run_command: //' -e '/trace: .*/d' \ + -e '/RUNTIME_PREFIX requested/d' >actual && echo "$expect true" >expect && test_cmp expect actual } From patchwork Sun Jan 27 23:27:04 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee via GitGitGadget X-Patchwork-Id: 10783077 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 954146C2 for ; Sun, 27 Jan 2019 23:27:10 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 84191286BF for ; Sun, 27 Jan 2019 23:27:10 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 787C8298EC; Sun, 27 Jan 2019 23:27:10 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 17017286BF for ; Sun, 27 Jan 2019 23:27:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726976AbfA0X1J (ORCPT ); Sun, 27 Jan 2019 18:27:09 -0500 Received: from mail-ed1-f66.google.com ([209.85.208.66]:35016 "EHLO mail-ed1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726946AbfA0X1G (ORCPT ); Sun, 27 Jan 2019 18:27:06 -0500 Received: by mail-ed1-f66.google.com with SMTP id x30so11634308edx.2 for ; Sun, 27 Jan 2019 15:27:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=7gtFKAG898gIOKd93T1TNRDKFMtd3jqrEAjEXx25Pgw=; b=Y8lp5jB43rH4f2IwTAcVRXuJzwBpJGaAAl1zIuINrk7Z41ajr7S7Hf2m1EyPIhii/d cQqtmMQ0uhMLfhPViF8xoESp/Q4HLUtuCBFJYrBSK/2uIx5r0sDvV6urD+O/qggvtYJO 19VEt/y8PqzicMUHPWYo1M6+oHJbGmS1BD+oJdJq47dA1NYI54n27kmQAc24O2BMkl3a XXYzWAwOX7hYgnEXWdduQotK7zwvkrDqZEjuGcEjZxqVtDjfu4UfwO+2LALpU+XDTaEa VdNm7KQ3n7ZTHMqV1aH8PwjvsenryILFUvUQbj8AYxTx9/kd1Ctr49WmTGPZAhgkNZYz jGyQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=7gtFKAG898gIOKd93T1TNRDKFMtd3jqrEAjEXx25Pgw=; b=Ef1lwAlCv8oxI2LJS+fzHrWhN/jdrmYmxfMZKQRiMAfWz3y5JBn+6RQPKfMICIMGpC 29hLz0UUekbEbnl3NFgqEsi5T7Z5Wrc2ipsKliocYlW1uT2RPXA4AL425SfwfNK3U6ed KeVTOrM1G9YesqtiJCCubS8TN0SmQH6eAS+Vsf8QGt0H23pWRTkp3bx9sfPGDRit/l5t 1LDBXkPlcMWvmm1cQdYeJoLzDpOx2DvF/fTWk4jhU3lTE0GhCHvr9jRjyDS6Dmev5evA RUGVygEKomwoaug+yZLL3fzh98ieLTNA04iJFmnyx7aoVBEj3vSPxlcU88TDZMWetQfA wpmA== X-Gm-Message-State: AJcUukesrubFpfQD9qTpn5sWd+u2uTcgeKNRRCqOplqvrwPj+oiwvKfx Yt8nAhqvX9UsftY9vK90rlqHBFo5 X-Google-Smtp-Source: ALg8bN4ra0e8LlsoqZWiMcCdTH17He5JvIwxgg9BKpbEjsPsUWC2Nc55Z1eFqC14AivPRvvw6fuzQg== X-Received: by 2002:a50:8a45:: with SMTP id i63mr19754149edi.262.1548631625165; Sun, 27 Jan 2019 15:27:05 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id r51sm13635907eda.64.2019.01.27.15.27.04 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 27 Jan 2019 15:27:04 -0800 (PST) Date: Sun, 27 Jan 2019 15:27:04 -0800 (PST) X-Google-Original-Date: Sun, 27 Jan 2019 23:26:45 GMT Message-Id: <6a8adbe8baedd166f13d4fa74bce712165168f1b.1548631608.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Johannes Schindelin via GitGitGadget" Subject: [PATCH v5 19/21] tests: optionally skip bin-wrappers/ Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Johannes Schindelin Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Johannes Schindelin This speeds up the tests by a bit on Windows, where running Unix shell scripts (and spawning processes) is not exactly a cheap operation. Signed-off-by: Johannes Schindelin --- t/README | 9 +++++++++ t/test-lib.sh | 19 +++++++++++++------ 2 files changed, 22 insertions(+), 6 deletions(-) diff --git a/t/README b/t/README index 11ce7675e3..063530234f 100644 --- a/t/README +++ b/t/README @@ -170,6 +170,15 @@ appropriately before running "make". implied by other options like --valgrind and GIT_TEST_INSTALLED. +--no-bin-wrappers:: + By default, the test suite uses the wrappers in + `../bin-wrappers/` to execute `git` and friends. With this option, + `../git` and friends are run directly. This is not recommended + in general, as the wrappers contain safeguards to ensure that no + files from an installed Git are used, but can speed up test runs + especially on platforms where running shell scripts is expensive + (most notably, Windows). + --root=:: Create "trash" directories used to store all temporary data during testing under , instead of the t/ directory. diff --git a/t/test-lib.sh b/t/test-lib.sh index c790e98fd2..25e649c997 100644 --- a/t/test-lib.sh +++ b/t/test-lib.sh @@ -111,6 +111,8 @@ do test -z "$HARNESS_ACTIVE" && quiet=t ;; --with-dashes) with_dashes=t ;; + --no-bin-wrappers) + no_bin_wrappers=t ;; --no-color) color= ;; --va|--val|--valg|--valgr|--valgri|--valgrin|--valgrind) @@ -1214,16 +1216,21 @@ then PATH=$GIT_TEST_INSTALLED:$GIT_BUILD_DIR/t/helper:$PATH GIT_EXEC_PATH=${GIT_TEST_EXEC_PATH:-$GIT_EXEC_PATH} else # normal case, use ../bin-wrappers only unless $with_dashes: - git_bin_dir="$GIT_BUILD_DIR/bin-wrappers" - if ! test -x "$git_bin_dir/git" + if test -n "$no_bin_wrappers" then - if test -z "$with_dashes" + with_dashes=t + else + git_bin_dir="$GIT_BUILD_DIR/bin-wrappers" + if ! test -x "$git_bin_dir/git" then - say "$git_bin_dir/git is not executable; using GIT_EXEC_PATH" + if test -z "$with_dashes" + then + say "$git_bin_dir/git is not executable; using GIT_EXEC_PATH" + fi + with_dashes=t fi - with_dashes=t + PATH="$git_bin_dir:$PATH" fi - PATH="$git_bin_dir:$PATH" GIT_EXEC_PATH=$GIT_BUILD_DIR if test -n "$with_dashes" then From patchwork Sun Jan 27 23:27:05 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee via GitGitGadget X-Patchwork-Id: 10783099 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A6A0213B5 for ; Sun, 27 Jan 2019 23:27:25 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 96E5A286BF for ; Sun, 27 Jan 2019 23:27:25 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8B32C298EC; Sun, 27 Jan 2019 23:27:25 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 25226286BF for ; Sun, 27 Jan 2019 23:27:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726960AbfA0X1I (ORCPT ); Sun, 27 Jan 2019 18:27:08 -0500 Received: from mail-ed1-f43.google.com ([209.85.208.43]:32846 "EHLO mail-ed1-f43.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726948AbfA0X1H (ORCPT ); Sun, 27 Jan 2019 18:27:07 -0500 Received: by mail-ed1-f43.google.com with SMTP id p6so11652777eds.0 for ; Sun, 27 Jan 2019 15:27:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=8Smsl1dhl4ohZgrvGoo0qeTJuxC6Y18tYIdYty2ZmJE=; b=EK0VpLoiqVh5djnpRBDPIph7ykXwmXVS4Ei8E1Z53yTDyC+5QesdNzCoKzm6er9NRu ExhMOW7YRIEihMenE9fgWRA5h9KBWSve7OLKCMq5ti3RhsD1whutUIng7U1q8FpHYedJ rMdQbh9gtHIiX5XBU67IcUAcIGsjRfwoqFkP7D7igpnZTVLmf5weHIwYAPfkqZVisGUN y3XxVk5iW5pUcljmOKmwbggQjllFsN/QYUSWJhLlRXGqKq5fQoZFLBdphP1+JAmO/RNk cL19oa2vjNMPYPupiMqK3RcrTtivHx4nLzsfySfj/CX0ERST//BuVsv2EY5p9Y1aWgxN Aafw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=8Smsl1dhl4ohZgrvGoo0qeTJuxC6Y18tYIdYty2ZmJE=; b=mwqTkrgOnYhQ31W1JrB7cSOrf27/SOPXARAPtRqT8d/F5ipadGlfQXJRXNPIxoAJjy noK11I/FWwl8m+yjydnDKca0gr2p8j1OQELurP+M2GKIH5H5zArxBMdB77vUfOEyitY/ 2mxnuxriIDfQg5dVAqpGaDqIWLKIIUIQe5utHT9kajpdSNMkyx927HENtUaPpWApTW4j Y/6f++OuC5RTHqEgWehO6eqHmDs6Ep1FYFQRnAvvdoRezlAr8pveSM+ZksfYxN3z17Wr hsF2gvwyRBO8Q2wXTSCb4okb4BtJroNubjD5+7lyhv8MGqy6YAglI4JArA9dB9TaChij nisA== X-Gm-Message-State: AJcUukedKtVtBufJ/lvmtlvIhhtwSUd4CvQYBKZqTBGzL59LU4OIpfgC /QaJ/+gXvFArBpPStr+12ZLC0ko5 X-Google-Smtp-Source: ALg8bN4sF8ewNIwtbr5eG6fY0aeWhjKQ0fRinkDN3OCRVGXX4mZOKeJXhxgJ+DwxKRK/pElcPv4mdw== X-Received: by 2002:a05:6402:1347:: with SMTP id y7mr19308361edw.114.1548631625914; Sun, 27 Jan 2019 15:27:05 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id a11sm14022759edc.28.2019.01.27.15.27.05 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 27 Jan 2019 15:27:05 -0800 (PST) Date: Sun, 27 Jan 2019 15:27:05 -0800 (PST) X-Google-Original-Date: Sun, 27 Jan 2019 23:26:46 GMT Message-Id: In-Reply-To: References: From: "Johannes Schindelin via GitGitGadget" Subject: [PATCH v5 20/21] ci: speed up Windows phase Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Johannes Schindelin Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Johannes Schindelin As Unix shell scripting comes at a hefty price on Windows, we have to see where we can save some time to run the test suite. Let's skip the chain linting and the bin-wrappers/ redirection on Windows; this seems to shave of anywhere between 10-30% from the overall runtime. Signed-off-by: Johannes Schindelin --- ci/lib.sh | 2 ++ 1 file changed, 2 insertions(+) diff --git a/ci/lib.sh b/ci/lib.sh index 5505776876..c2bc6c68b9 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -127,6 +127,8 @@ then export GIT_PROVE_OPTS="--timer --jobs 10 --state=failed,slow,save" export GIT_TEST_OPTS="--verbose-log -x --write-junit-xml" export MAKEFLAGS="--jobs=10" + test windows_nt != "$CI_OS_NAME" || + GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS" else echo "Could not identify CI type" >&2 exit 1 From patchwork Sun Jan 27 23:27:06 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee via GitGitGadget X-Patchwork-Id: 10783095 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C3D8E6C2 for ; Sun, 27 Jan 2019 23:27:23 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B238D286BF for ; Sun, 27 Jan 2019 23:27:23 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A6384298F2; Sun, 27 Jan 2019 23:27:23 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DAD4B286BF for ; Sun, 27 Jan 2019 23:27:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727041AbfA0X1W (ORCPT ); Sun, 27 Jan 2019 18:27:22 -0500 Received: from mail-ed1-f65.google.com ([209.85.208.65]:38168 "EHLO mail-ed1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726953AbfA0X1J (ORCPT ); Sun, 27 Jan 2019 18:27:09 -0500 Received: by mail-ed1-f65.google.com with SMTP id h50so11588936ede.5 for ; Sun, 27 Jan 2019 15:27:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=b7cW3NQWvBmpQlrQPIMNuGX0tGz65l91wYYGw5modhg=; b=BHQnXyBoMOuSkWC5CcsWzwthXjAYHua6N66XfJmbZV3R35fDXTfGszih70rDzD9QC6 WZ0vI+vEv1g6j1VAtCgGhORk5OY05YseYTcWP4yywSckcaprvzfCZcMMQqvO+MIVQdcu bNEyy46cLMxrE4/dwnvLQNYDkYKGtIH30WPqUhvJyMO/kQAyOncakxGGEdK+SwYk0rMu ir+Y606+4UJtFvebm/hwdHWmk5/cFVgclMb/j64Xp7BiAPPC3H4fZDbP59I0kTVRe8QO bj+3sORE8qZqQf0au6Yg6lX4+6DYacuMizu3K6Uo9OEsgwKNywuq26Jo/QKzcHjlUVZa KMEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=b7cW3NQWvBmpQlrQPIMNuGX0tGz65l91wYYGw5modhg=; b=WAVZ0maAFcJO6PxP1yi1diozrgsPF8XEh9dzTu1EDCrF3SpZ4c/kNZLS+S56XG/sYc Fcjw+1vsd/aqTWyRQbywKovBqEDDpwCmxDJxECDy+l3IfUEkZsJKe2qbmsqntXkScjfx XonnWzOjkiiH0GRH0WZwdyazxRuvkZP1+9UzEJ92pSDtjrm3RQOrH6B/ZpW/QnJWngO1 Y3/7jCfgYiJVzCdql+e8nZ9pnR/WYEcyCIwdDJwu/fNF5LPtjrwNYL6zw94TcffA9VEe V3rl8sHcBrgri9f86UuZIRdjRB2Oqtui9UGQpdfYnTD20ZiQsebzL8lKpA7D9vL4sR5M J/8A== X-Gm-Message-State: AJcUukfPQwEjyjG5k3eaE6chpHD43yTSyEMclGt0C6MLeHTpTFIyQbD3 OTscOm/KgL+3HvLyvJ9Ks5tvgK1j X-Google-Smtp-Source: ALg8bN4ANTkWm1UOpqA7KFpLqL3ftAJUsmv1hb+ajgOBZ5xJQrgZxQX7kkaHS/WjUsDjaZPMszCP0w== X-Received: by 2002:a50:9315:: with SMTP id m21mr19019001eda.58.1548631626664; Sun, 27 Jan 2019 15:27:06 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id k20sm13208109eda.71.2019.01.27.15.27.06 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 27 Jan 2019 15:27:06 -0800 (PST) Date: Sun, 27 Jan 2019 15:27:06 -0800 (PST) X-Google-Original-Date: Sun, 27 Jan 2019 23:26:47 GMT Message-Id: <6ff87d17729db3dca73e74162cd8f148ce63efdd.1548631608.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Johannes Schindelin via GitGitGadget" Subject: [PATCH v5 21/21] ci: parallelize testing on Windows Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Johannes Schindelin Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Johannes Schindelin The fact that Git's test suite is implemented in Unix shell script that is as portable as we can muster, combined with the fact that Unix shell scripting is foreign to Windows (and therefore has to be emulated), results in pretty abysmal speed of the test suite on that platform, for pretty much no other reason than that language choice. For comparison: while the Linux build & test is typically done within about 8 minutes, the Windows build & test typically lasts about 80 minutes in Azure Pipelines. To help with that, let's use the Azure Pipeline feature where you can parallelize jobs, make jobs depend on each other, and pass artifacts between them. The tests are distributed using the following heuristic: listing all test scripts ordered by size in descending order (as a cheap way to estimate the overall run time), every Nth script is run (where N is the total number of parallel jobs), starting at the index corresponding to the parallel job. This slicing is performed by a new function that is added to the `test-tool`. To optimize the overall runtime of the entire Pipeline, we need to move the Windows jobs to the beginning (otherwise there would be a very decent chance for the Pipeline to be run only the Windows build, while all the parallel Windows test jobs wait for this single one). We use Azure Pipelines Artifacts for both the minimal Git for Windows SDK as well as the built executables, as deduplication and caching close to the agents makes that really fast. For comparison: while downloading and unpacking the minimal Git for Windows SDK via PowerShell takes only one minute (down from anywhere between 2.5 to 7 when using a shallow clone), uploading it as Pipeline Artifact takes less than 30s and downloading and unpacking less than 20s (sometimes even as little as only twelve seconds). Signed-off-by: Johannes Schindelin --- Makefile | 10 +++++ azure-pipelines.yml | 79 ++++++++++++++++++++++++++++++++++---- ci/make-test-artifacts.sh | 12 ++++++ ci/run-test-slice.sh | 17 ++++++++ t/helper/test-path-utils.c | 31 +++++++++++++++ 5 files changed, 141 insertions(+), 8 deletions(-) create mode 100755 ci/make-test-artifacts.sh create mode 100755 ci/run-test-slice.sh diff --git a/Makefile b/Makefile index 044b4f77bd..daa318fe17 100644 --- a/Makefile +++ b/Makefile @@ -2927,6 +2927,16 @@ rpm:: @false .PHONY: rpm +artifacts-tar:: $(ALL_PROGRAMS) $(SCRIPT_LIB) $(BUILT_INS) $(OTHER_PROGRAMS) \ + GIT-BUILD-OPTIONS $(TEST_PROGRAMS) $(test_bindir_programs) \ + $(NO_INSTALL) $(MOFILES) + $(QUIET_SUBDIR0)templates $(QUIET_SUBDIR1) \ + SHELL_PATH='$(SHELL_PATH_SQ)' PERL_PATH='$(PERL_PATH_SQ)' + test -n "$(ARTIFACTS_DIRECTORY)" + mkdir -p "$(ARTIFACTS_DIRECTORY)" + $(TAR) czf "$(ARTIFACTS_DIRECTORY)/artifacts.tar.gz" $^ templates/blt/ +.PHONY: artifacts-tar + htmldocs = git-htmldocs-$(GIT_VERSION) manpages = git-manpages-$(GIT_VERSION) .PHONY: dist-doc distclean diff --git a/azure-pipelines.yml b/azure-pipelines.yml index 480e841a85..c329b7218b 100644 --- a/azure-pipelines.yml +++ b/azure-pipelines.yml @@ -3,8 +3,8 @@ resources: fetchDepth: 1 jobs: -- job: windows - displayName: Windows +- job: windows_build + displayName: Windows Build condition: succeeded() pool: Hosted timeoutInMinutes: 240 @@ -30,21 +30,84 @@ jobs: displayName: 'Download git-sdk-64-minimal' - powershell: | & git-sdk-64-minimal\usr\bin\bash.exe -lc @" - export DEVELOPER=1 - export NO_PERL=1 - export NO_SVN_TESTS=1 - export GIT_TEST_SKIP_REBASE_P=1 + ci/make-test-artifacts.sh artifacts + "@ + if (!$?) { exit(1) } + displayName: Build + env: + HOME: $(Build.SourcesDirectory) + MSYSTEM: MINGW64 + DEVELOPER: 1 + NO_PERL: 1 + - task: PublishPipelineArtifact@0 + displayName: 'Publish Pipeline Artifact: test artifacts' + inputs: + artifactName: 'windows-artifacts' + targetPath: '$(Build.SourcesDirectory)\artifacts' + - task: PublishPipelineArtifact@0 + displayName: 'Publish Pipeline Artifact: git-sdk-64-minimal' + inputs: + artifactName: 'git-sdk-64-minimal' + targetPath: '$(Build.SourcesDirectory)\git-sdk-64-minimal' + - powershell: | + if ("$GITFILESHAREPWD" -ne "" -and "$GITFILESHAREPWD" -ne "`$`(gitfileshare.pwd)") { + cmd /c rmdir "$(Build.SourcesDirectory)\test-cache" + } + displayName: 'Unmount test-cache' + condition: true + env: + GITFILESHAREPWD: $(gitfileshare.pwd) + +- job: windows_test + displayName: Windows Test + dependsOn: windows_build + condition: succeeded() + pool: Hosted + timeoutInMinutes: 240 + strategy: + parallel: 10 + steps: + - powershell: | + if ("$GITFILESHAREPWD" -ne "" -and "$GITFILESHAREPWD" -ne "`$`(gitfileshare.pwd)") { + net use s: \\gitfileshare.file.core.windows.net\test-cache "$GITFILESHAREPWD" /user:AZURE\gitfileshare /persistent:no + cmd /c mklink /d "$(Build.SourcesDirectory)\test-cache" S:\ + } + displayName: 'Mount test-cache' + env: + GITFILESHAREPWD: $(gitfileshare.pwd) + - task: DownloadPipelineArtifact@0 + displayName: 'Download Pipeline Artifact: test artifacts' + inputs: + artifactName: 'windows-artifacts' + targetPath: '$(Build.SourcesDirectory)' + - task: DownloadPipelineArtifact@0 + displayName: 'Download Pipeline Artifact: git-sdk-64-minimal' + inputs: + artifactName: 'git-sdk-64-minimal' + targetPath: '$(Build.SourcesDirectory)\git-sdk-64-minimal' + - powershell: | + & git-sdk-64-minimal\usr\bin\bash.exe -lc @" + test -f artifacts.tar.gz || { + echo No test artifacts found\; skipping >&2 + exit 0 + } + tar xf artifacts.tar.gz || exit 1 + + # Let Git ignore the SDK and the test-cache + printf '%s\n' /git-sdk-64-minimal/ /test-cache/ >>.git/info/exclude - ci/run-build-and-tests.sh || { + ci/run-test-slice.sh `$SYSTEM_JOBPOSITIONINPHASE `$SYSTEM_TOTALJOBSINPHASE || { ci/print-test-failures.sh exit 1 } "@ if (!$?) { exit(1) } - displayName: 'Build & Test' + displayName: 'Test (parallel)' env: HOME: $(Build.SourcesDirectory) MSYSTEM: MINGW64 + NO_SVN_TESTS: 1 + GIT_TEST_SKIP_REBASE_P: 1 - powershell: | if ("$GITFILESHAREPWD" -ne "" -and "$GITFILESHAREPWD" -ne "`$`(gitfileshare.pwd)") { cmd /c rmdir "$(Build.SourcesDirectory)\test-cache" diff --git a/ci/make-test-artifacts.sh b/ci/make-test-artifacts.sh new file mode 100755 index 0000000000..646967481f --- /dev/null +++ b/ci/make-test-artifacts.sh @@ -0,0 +1,12 @@ +#!/bin/sh +# +# Build Git and store artifacts for testing +# + +mkdir -p "$1" # in case ci/lib.sh decides to quit early + +. ${0%/*}/lib.sh + +make artifacts-tar ARTIFACTS_DIRECTORY="$1" + +check_unignored_build_artifacts diff --git a/ci/run-test-slice.sh b/ci/run-test-slice.sh new file mode 100755 index 0000000000..f8c2c3106a --- /dev/null +++ b/ci/run-test-slice.sh @@ -0,0 +1,17 @@ +#!/bin/sh +# +# Test Git in parallel +# + +. ${0%/*}/lib.sh + +case "$CI_OS_NAME" in +windows*) cmd //c mklink //j t\\.prove "$(cygpath -aw "$cache_dir/.prove")";; +*) ln -s "$cache_dir/.prove" t/.prove;; +esac + +make --quiet -C t T="$(cd t && + ./helper/test-tool path-utils slice-tests "$1" "$2" t[0-9]*.sh | + tr '\n' ' ')" + +check_unignored_build_artifacts diff --git a/t/helper/test-path-utils.c b/t/helper/test-path-utils.c index 6efde6f5ba..5d543ad21f 100644 --- a/t/helper/test-path-utils.c +++ b/t/helper/test-path-utils.c @@ -177,6 +177,14 @@ static int is_dotgitmodules(const char *path) return is_hfs_dotgitmodules(path) || is_ntfs_dotgitmodules(path); } +static int cmp_by_st_size(const void *a, const void *b) +{ + intptr_t x = (intptr_t)((struct string_list_item *)a)->util; + intptr_t y = (intptr_t)((struct string_list_item *)b)->util; + + return x > y ? -1 : (x < y ? +1 : 0); +} + int cmd__path_utils(int argc, const char **argv) { if (argc == 3 && !strcmp(argv[1], "normalize_path_copy")) { @@ -324,6 +332,29 @@ int cmd__path_utils(int argc, const char **argv) return 0; } + if (argc > 5 && !strcmp(argv[1], "slice-tests")) { + int res = 0; + long offset, stride, i; + struct string_list list = STRING_LIST_INIT_NODUP; + struct stat st; + + offset = strtol(argv[2], NULL, 10); + stride = strtol(argv[3], NULL, 10); + if (stride < 1) + stride = 1; + for (i = 4; i < argc; i++) + if (stat(argv[i], &st)) + res = error_errno("Cannot stat '%s'", argv[i]); + else + string_list_append(&list, argv[i])->util = + (void *)(intptr_t)st.st_size; + QSORT(list.items, list.nr, cmp_by_st_size); + for (i = offset; i < list.nr; i+= stride) + printf("%s\n", list.items[i].string); + + return !!res; + } + fprintf(stderr, "%s: unknown function name: %s\n", argv[0], argv[1] ? argv[1] : "(there was none)"); return 1;