From patchwork Wed Jan 16 13:36:11 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Passaro via GitGitGadget X-Patchwork-Id: 10765933 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8B73691E for ; Wed, 16 Jan 2019 13:36:16 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 79A422BD36 for ; Wed, 16 Jan 2019 13:36:16 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6D41E2DD52; Wed, 16 Jan 2019 13:36:16 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 12BDE2BD36 for ; Wed, 16 Jan 2019 13:36:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2393203AbfAPNgO (ORCPT ); Wed, 16 Jan 2019 08:36:14 -0500 Received: from mail-ed1-f65.google.com ([209.85.208.65]:41798 "EHLO mail-ed1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390490AbfAPNgO (ORCPT ); Wed, 16 Jan 2019 08:36:14 -0500 Received: by mail-ed1-f65.google.com with SMTP id a20so5404174edc.8 for ; Wed, 16 Jan 2019 05:36:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=HXiejwwfCeMnioxjbQbemqGN0nu3EIP7X0SejTQQ79I=; b=otUVaGVHFowMgfVgcGUlOwGB8S8J8rVUcatzMdyrLsWAcI+08WyG7c7nxtttoABtS+ aa7lj7J4/6CPCGHn5ip0D+1hiPTjVTqwU3d4mvqwjg1eo+Wspd9PvQQ+saQuyUDnc59M ICBAfNhKCHm+AjpkKA9dmZ9uHu+l4GB9TAgB0qZuqklPphiIcuwnChRyMRzA4gIVwbHR 2/imuHvkQw5XVU5wwo0rXss9rxyv/CNOggi8TdAA2M/dVeG5C8mg/OqPgnWQyOp6wUcL vsRlTTsRmIJz0umPGEElCJESRMhD9/OhE0pSaLMW+op0t60/1sAupKos+fgjmBOdLRhL jDOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=HXiejwwfCeMnioxjbQbemqGN0nu3EIP7X0SejTQQ79I=; b=eNQ5zSI1w1BLBVzOi/w7AISGG6C4eBTId7Igxcv1/lfof83jfmuoOFihbfUOTGHDq+ 9qpE/vnIyUwHyhZV+vMoHLvX8rahBx3cTq3ffhLc94YUYKPp9UVfqEzil2hm4COCql1o s+4l9qSunWi/UcVHNoE1vLaeAq6CTsZfJDWEdbwHc7EXmMPZM4iRRssg7zMxBlPxiONx hnFbdzvQDsck5hDzA3/7Dzu88UQUp3xl8BICPQ92wLXfBzTJfBccx/ACzSC3tKUYcOoy DCsxZM7woLicXiuZ1yYaNrGM2kc/vGjeKJJ+ellFFaSpMGyRiAPCrhLjDQ7u6s/ErdaL QmyA== X-Gm-Message-State: AJcUukegeytXmdzq3Ppnb2p9IeoGywUtuAXLyKNu/7m3r9y+FYBQ3S1F PgwBK+xvtEepnpHAohQV+N5gcnli X-Google-Smtp-Source: ALg8bN4f7xWWbrf1vVtLTLxOqKzzIVGTcB+IJwGUJeU2Ql2Nn5Y4PcvPL/4x08klZgGn4iw75dHvKQ== X-Received: by 2002:a50:a517:: with SMTP id y23mr7621955edb.219.1547645772458; Wed, 16 Jan 2019 05:36:12 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id p36sm5690199edc.78.2019.01.16.05.36.11 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 16 Jan 2019 05:36:11 -0800 (PST) Date: Wed, 16 Jan 2019 05:36:11 -0800 (PST) X-Google-Original-Date: Wed, 16 Jan 2019 13:35:49 GMT Message-Id: <75ec97b3921f3ed346e9ab119ebff2546f03fade.1547645770.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Johannes Schindelin via GitGitGadget" Subject: [PATCH v3 01/21] travis: fix skipping tagged releases Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Johannes Schindelin Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Johannes Schindelin When building a PR, TRAVIS_BRANCH refers to the *target branch*. Therefore, if a PR targets `master`, and `master` happened to be tagged, we skipped the build by mistake. Fix this by using TRAVIS_PULL_REQUEST_BRANCH (i.e. the *source branch*) when available, falling back to TRAVIS_BRANCH (i.e. for CI builds, also known as "push builds"). Signed-off-by: Johannes Schindelin --- ci/lib-travisci.sh | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/ci/lib-travisci.sh b/ci/lib-travisci.sh index 69dff4d1ec..d9d4f1a9d7 100755 --- a/ci/lib-travisci.sh +++ b/ci/lib-travisci.sh @@ -1,5 +1,9 @@ # Library of functions shared by all CI scripts +# When building a PR, TRAVIS_BRANCH refers to the *target* branch. Not what we +# want here. We want the source branch instead. +TRAVIS_BRANCH="${TRAVIS_PULL_REQUEST_BRANCH:-$TRAVIS_BRANCH}" + skip_branch_tip_with_tag () { # Sometimes, a branch is pushed at the same time the tag that points # at the same commit as the tip of the branch is pushed, and building From patchwork Wed Jan 16 13:36:12 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Passaro via GitGitGadget X-Patchwork-Id: 10765935 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 61FA091E for ; Wed, 16 Jan 2019 13:36:19 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 524122D172 for ; Wed, 16 Jan 2019 13:36:19 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 46CC22DE14; Wed, 16 Jan 2019 13:36:19 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C2CEA2D172 for ; Wed, 16 Jan 2019 13:36:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2393207AbfAPNgR (ORCPT ); Wed, 16 Jan 2019 08:36:17 -0500 Received: from mail-ed1-f65.google.com ([209.85.208.65]:41800 "EHLO mail-ed1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390472AbfAPNgQ (ORCPT ); Wed, 16 Jan 2019 08:36:16 -0500 Received: by mail-ed1-f65.google.com with SMTP id a20so5404229edc.8 for ; Wed, 16 Jan 2019 05:36:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=96M+SI3VKYDk3uCf1BpYpa5//cuUAVpfWWWcGr5kIog=; b=OPgAP1ewTojsAdaG2LqLaQABYazEhjnrBWNjSKpQX9DfNviSFKb3LAmrsJ8fos7mdj 88Vv+lf7WK5Dl/Rmw4ILMpZkYFSkG3Ew2NNeY6gNqQ9xPfGeFM13R1M4rMEFmSK8n+1y OfZxP2ta55U1jP8xrDNNp2UiYQOR24qRuxy1M0aY07bsgpKqLhjWsfDebphiaCllFunx 3xiqh+jXwSIscuAWrEo0MWxkmD0PqtNotLQe1hrspP4ymglwwFRZNuQ6RUcsYoofUsLI jBl+g7FZpNCU9Zx7u1zv3qCSClISeJRrI/Bey8rY12F30FKi53NJnXEE/4TKasADfbMM 9Ihw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=96M+SI3VKYDk3uCf1BpYpa5//cuUAVpfWWWcGr5kIog=; b=q8LvAAFua9unyuxOusAU3e2vWDxy5tXfYOMKyZJHE19VTNxh/hWOtJN3O4CgaMDhHA w/kRvKeXpNDOOB1G1xNxZ/pcIjunoTe+ETVASZMwUFZWI2ntYG6EhOZcgRgHxn0QatOX PjmJ6+eUQRG9SRMHNcS3I7iLtOtCFQe/bNJX7WE0VUrBeIfBof1MyvKcPcuW+f/zk8G1 LdhN5nsjN77LCi3c/WZRwIaL+J4a5QoM+wIXPjFRsofeiESAQwC0y58vTkgCiaKY5Jvl ewuuXVpIw7orBr0NaYhMgYUdKGk5A3cSvGhEilDduU+ZmwCRuZF6Wr6kBGP0QZblKYj/ mTXg== X-Gm-Message-State: AJcUukcffuthouVPkmmLOBLvvYvVimSELUj46T60ZOnTESbm5w4JS82o /srrlqSXAmlzVUAyhMkKZq2cQitl X-Google-Smtp-Source: ALg8bN7zit2FmWxj/NfgwwV4OSSMVXV1ef9SoRKG/mU6xRIB0g4nZ9lPypP45l3xT/cdVEYyzWXTDw== X-Received: by 2002:a17:906:1489:: with SMTP id x9-v6mr6670465ejc.222.1547645773387; Wed, 16 Jan 2019 05:36:13 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id v11sm5458667edy.49.2019.01.16.05.36.12 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 16 Jan 2019 05:36:12 -0800 (PST) Date: Wed, 16 Jan 2019 05:36:12 -0800 (PST) X-Google-Original-Date: Wed, 16 Jan 2019 13:35:50 GMT Message-Id: In-Reply-To: References: From: "Johannes Schindelin via GitGitGadget" Subject: [PATCH v3 02/21] ci: rename the library of common functions Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Johannes Schindelin Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Johannes Schindelin The name is hard-coded to reflect that we use Travis CI for continuous testing. In the next commits, we will extend this to be able use Azure DevOps, too. So let's adjust the name to make it more generic. Signed-off-by: Johannes Schindelin --- ci/install-dependencies.sh | 2 +- ci/{lib-travisci.sh => lib.sh} | 0 ci/print-test-failures.sh | 2 +- ci/run-build-and-tests.sh | 2 +- ci/run-linux32-docker.sh | 2 +- ci/run-static-analysis.sh | 2 +- ci/run-windows-build.sh | 2 +- ci/test-documentation.sh | 2 +- 8 files changed, 7 insertions(+), 7 deletions(-) rename ci/{lib-travisci.sh => lib.sh} (100%) diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh index 06c3546e1e..fe65144152 100755 --- a/ci/install-dependencies.sh +++ b/ci/install-dependencies.sh @@ -3,7 +3,7 @@ # Install dependencies required to build and test Git on Linux and macOS # -. ${0%/*}/lib-travisci.sh +. ${0%/*}/lib.sh P4WHENCE=http://filehost.perforce.com/perforce/r$LINUX_P4_VERSION LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION diff --git a/ci/lib-travisci.sh b/ci/lib.sh similarity index 100% rename from ci/lib-travisci.sh rename to ci/lib.sh diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh index d55460a212..7aef39a2fd 100755 --- a/ci/print-test-failures.sh +++ b/ci/print-test-failures.sh @@ -3,7 +3,7 @@ # Print output of failing tests # -. ${0%/*}/lib-travisci.sh +. ${0%/*}/lib.sh # Tracing executed commands would produce too much noise in the loop below. set +x diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh index cda170d5c2..db342bb6a8 100755 --- a/ci/run-build-and-tests.sh +++ b/ci/run-build-and-tests.sh @@ -3,7 +3,7 @@ # Build and test Git # -. ${0%/*}/lib-travisci.sh +. ${0%/*}/lib.sh ln -s "$cache_dir/.prove" t/.prove diff --git a/ci/run-linux32-docker.sh b/ci/run-linux32-docker.sh index 21637903ce..751acfcf8a 100755 --- a/ci/run-linux32-docker.sh +++ b/ci/run-linux32-docker.sh @@ -3,7 +3,7 @@ # Download and run Docker image to build and test 32-bit Git # -. ${0%/*}/lib-travisci.sh +. ${0%/*}/lib.sh docker pull daald/ubuntu32:xenial diff --git a/ci/run-static-analysis.sh b/ci/run-static-analysis.sh index 5688f261d0..dc189c7456 100755 --- a/ci/run-static-analysis.sh +++ b/ci/run-static-analysis.sh @@ -3,7 +3,7 @@ # Perform various static code analysis checks # -. ${0%/*}/lib-travisci.sh +. ${0%/*}/lib.sh make --jobs=2 coccicheck diff --git a/ci/run-windows-build.sh b/ci/run-windows-build.sh index d99a180e52..a73a4eca0a 100755 --- a/ci/run-windows-build.sh +++ b/ci/run-windows-build.sh @@ -6,7 +6,7 @@ # supported) and a commit hash. # -. ${0%/*}/lib-travisci.sh +. ${0%/*}/lib.sh test $# -ne 2 && echo "Unexpected number of parameters" && exit 1 test -z "$GFW_CI_TOKEN" && echo "GFW_CI_TOKEN not defined" && exit diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh index a20de9ca12..d3cdbac73f 100755 --- a/ci/test-documentation.sh +++ b/ci/test-documentation.sh @@ -3,7 +3,7 @@ # Perform sanity checks on documentation and build it. # -. ${0%/*}/lib-travisci.sh +. ${0%/*}/lib.sh gem install asciidoctor From patchwork Wed Jan 16 13:36:13 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Passaro via GitGitGadget X-Patchwork-Id: 10765937 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 561011580 for ; Wed, 16 Jan 2019 13:36:20 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 44DA72BD36 for ; Wed, 16 Jan 2019 13:36:20 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 38D612DD52; Wed, 16 Jan 2019 13:36:20 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 906942D172 for ; Wed, 16 Jan 2019 13:36:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2393209AbfAPNgS (ORCPT ); Wed, 16 Jan 2019 08:36:18 -0500 Received: from mail-ed1-f65.google.com ([209.85.208.65]:34910 "EHLO mail-ed1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390475AbfAPNgQ (ORCPT ); Wed, 16 Jan 2019 08:36:16 -0500 Received: by mail-ed1-f65.google.com with SMTP id x30so5448242edx.2 for ; Wed, 16 Jan 2019 05:36:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=z0RGshkWSxccEvuS6Pf5xHIjQIPy7RZBuis9Ougj0l0=; b=GjD0Gnl3yprnoFdd0kHNE8I6FTWgho7dv0eI7fcVgT/g8vEIa7aBlE3JyC8RCp+GAZ FnToYsM2bwT/PkuOj4+9IK4VT34s6FUIsIFhGcVqWlgxnV3PCGM8IrZJ3+D+tFEE2U5q csBmsisp3qiC2D0UJ10wTcvLeUIS0yMF7paO+yLMMICiawXkENJ5MH6Ufqq44FiqR961 /Y7OcH+uJ1tNWDm+YZ8YSRYIskfCNuAAnt3HfEFw/CR7hD0nT4jORHv84FsoiGIUAiQo a5yoZ12PJmiXHmVjOmWKV6K2E7YHcwO4ZJmefIJ4WwiqSME7qM6zY5cnacsVRD752FMp xDuw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=z0RGshkWSxccEvuS6Pf5xHIjQIPy7RZBuis9Ougj0l0=; b=OLK4Dm+YxnNnQXaqLq+aLNvgI8g0ZvuvkoxCfE0lXOnWK5oMQ2WLexZ4iEcvFNKRzt 9kOC0T0ggb6aHAQDuyzDraWBaPbvDdV29gDKxzpo2hEeSbmXUf5nLxdBSD76le5WHojb x4+rLBT6L+bBmmfzEsHShXRV32/hVv2NXs4hKf5EblH3NoPgQtf6wUXhPTaK40y4BExe hfa38+DkhjYBR/Mwh0A4xMZkJnSJXnyB2kU/5HTifO27sxyRPLguKCklkwui9Ilu+uQA BmAw9OfX+R+WTz1tMlaRZAKQFNMA6R+aYwpcjQijkyDqfez5VqKI03srOUYrqATNSd/R QX1Q== X-Gm-Message-State: AJcUukdkxe51TQtH3hFe66M9DCbNeFueqaGN4v40pKeCcvej8HrbtUF2 kH0XgM7phN8Db0a0jbFWTGBslRtG X-Google-Smtp-Source: ALg8bN5q0KBg3QJ/w5YHHGT0HWy6q3UAW73QRDvHtjvOUR4Zu0iwUoRufyCzLkZhIYr4n6G+ue6rTw== X-Received: by 2002:a50:b0a4:: with SMTP id j33mr7091915edd.267.1547645774242; Wed, 16 Jan 2019 05:36:14 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id v12-v6sm3028009ejk.58.2019.01.16.05.36.13 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 16 Jan 2019 05:36:13 -0800 (PST) Date: Wed, 16 Jan 2019 05:36:13 -0800 (PST) X-Google-Original-Date: Wed, 16 Jan 2019 13:35:51 GMT Message-Id: <06fa564386307146762d4b30259d8636fc0ac268.1547645770.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Johannes Schindelin via GitGitGadget" Subject: [PATCH v3 03/21] ci/lib.sh: encapsulate Travis-specific things Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Johannes Schindelin Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Johannes Schindelin The upcoming patches will allow building git.git via Azure Pipelines (i.e. Azure DevOps' Continuous Integration), where variable names and URLs look a bit different than in Travis CI. Signed-off-by: Johannes Schindelin --- ci/install-dependencies.sh | 3 ++- ci/lib.sh | 47 +++++++++++++++++++++++++------------- ci/print-test-failures.sh | 2 +- ci/test-documentation.sh | 1 + 4 files changed, 35 insertions(+), 18 deletions(-) diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh index fe65144152..bcdcc71592 100755 --- a/ci/install-dependencies.sh +++ b/ci/install-dependencies.sh @@ -37,7 +37,8 @@ osx-clang|osx-gcc) brew update --quiet # Uncomment this if you want to run perf tests: # brew install gnu-time - brew install git-lfs gettext + test -z "$BREW_INSTALL_PACKAGES" || + brew install $BREW_INSTALL_PACKAGES brew link --force gettext brew install caskroom/cask/perforce ;; diff --git a/ci/lib.sh b/ci/lib.sh index d9d4f1a9d7..bfc406a63b 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -1,8 +1,26 @@ # Library of functions shared by all CI scripts -# When building a PR, TRAVIS_BRANCH refers to the *target* branch. Not what we -# want here. We want the source branch instead. -TRAVIS_BRANCH="${TRAVIS_PULL_REQUEST_BRANCH:-$TRAVIS_BRANCH}" +if test true = "$TRAVIS" +then + # When building a PR, TRAVIS_BRANCH refers to the *target* branch. Not + # what we want here. We want the source branch instead. + CI_BRANCH="${TRAVIS_PULL_REQUEST_BRANCH:-$TRAVIS_BRANCH}" + CI_COMMIT="$TRAVIS_COMMIT" + CI_JOB_ID="$TRAVIS_JOB_ID" + CI_JOB_NUMBER="$TRAVIS_JOB_NUMBER" + CI_OS_NAME="$TRAVIS_OS_NAME" + CI_REPO_SLUG="$TRAVIS_REPO_SLUG" + + cache_dir="$HOME/travis-cache" + + url_for_job_id () { + echo "https://travis-ci.org/$CI_REPO_SLUG/jobs/$1" + } + + BREW_INSTALL_PACKAGES="git-lfs gettext" + export GIT_PROVE_OPTS="--timer --jobs 3 --state=failed,slow,save" + export GIT_TEST_OPTS="--verbose-log -x --immediate" +fi skip_branch_tip_with_tag () { # Sometimes, a branch is pushed at the same time the tag that points @@ -17,10 +35,10 @@ skip_branch_tip_with_tag () { # we can skip the build because we won't be skipping a build # of a tag. - if TAG=$(git describe --exact-match "$TRAVIS_BRANCH" 2>/dev/null) && - test "$TAG" != "$TRAVIS_BRANCH" + if TAG=$(git describe --exact-match "$CI_BRANCH" 2>/dev/null) && + test "$TAG" != "$CI_BRANCH" then - echo "$(tput setaf 2)Tip of $TRAVIS_BRANCH is exactly at $TAG$(tput sgr0)" + echo "$(tput setaf 2)Tip of $CI_BRANCH is exactly at $TAG$(tput sgr0)" exit 0 fi } @@ -29,7 +47,7 @@ skip_branch_tip_with_tag () { # job if we encounter the same tree again and can provide a useful info # message. save_good_tree () { - echo "$(git rev-parse $TRAVIS_COMMIT^{tree}) $TRAVIS_COMMIT $TRAVIS_JOB_NUMBER $TRAVIS_JOB_ID" >>"$good_trees_file" + echo "$(git rev-parse $CI_COMMIT^{tree}) $CI_COMMIT $CI_JOB_NUMBER $CI_JOB_ID" >>"$good_trees_file" # limit the file size tail -1000 "$good_trees_file" >"$good_trees_file".tmp mv "$good_trees_file".tmp "$good_trees_file" @@ -39,7 +57,7 @@ save_good_tree () { # successfully before (e.g. because the branch got rebased, changing only # the commit messages). skip_good_tree () { - if ! good_tree_info="$(grep "^$(git rev-parse $TRAVIS_COMMIT^{tree}) " "$good_trees_file")" + if ! good_tree_info="$(grep "^$(git rev-parse $CI_COMMIT^{tree}) " "$good_trees_file")" then # Haven't seen this tree yet, or no cached good trees file yet. # Continue the build job. @@ -49,18 +67,18 @@ skip_good_tree () { echo "$good_tree_info" | { read tree prev_good_commit prev_good_job_number prev_good_job_id - if test "$TRAVIS_JOB_ID" = "$prev_good_job_id" + if test "$CI_JOB_ID" = "$prev_good_job_id" then cat <<-EOF - $(tput setaf 2)Skipping build job for commit $TRAVIS_COMMIT.$(tput sgr0) + $(tput setaf 2)Skipping build job for commit $CI_COMMIT.$(tput sgr0) This commit has already been built and tested successfully by this build job. To force a re-build delete the branch's cache and then hit 'Restart job'. EOF else cat <<-EOF - $(tput setaf 2)Skipping build job for commit $TRAVIS_COMMIT.$(tput sgr0) + $(tput setaf 2)Skipping build job for commit $CI_COMMIT.$(tput sgr0) This commit's tree has already been built and tested successfully in build job $prev_good_job_number for commit $prev_good_commit. - The log of that build job is available at https://travis-ci.org/$TRAVIS_REPO_SLUG/jobs/$prev_good_job_id + The log of that build job is available at $(url_for_job_id $prev_good_job_id) To force a re-build delete the branch's cache and then hit 'Restart job'. EOF fi @@ -85,7 +103,6 @@ check_unignored_build_artifacts () # and installing dependencies. set -ex -cache_dir="$HOME/travis-cache" good_trees_file="$cache_dir/good-trees" mkdir -p "$cache_dir" @@ -95,13 +112,11 @@ skip_good_tree if test -z "$jobname" then - jobname="$TRAVIS_OS_NAME-$CC" + jobname="$CI_OS_NAME-$CC" fi export DEVELOPER=1 export DEFAULT_TEST_TARGET=prove -export GIT_PROVE_OPTS="--timer --jobs 3 --state=failed,slow,save" -export GIT_TEST_OPTS="--verbose-log -x --immediate" export GIT_TEST_CLONE_2GB=YesPlease if [ "$jobname" = linux-gcc ]; then export CC=gcc-8 diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh index 7aef39a2fd..d2045b63a6 100755 --- a/ci/print-test-failures.sh +++ b/ci/print-test-failures.sh @@ -69,7 +69,7 @@ do fi done -if [ $combined_trash_size -gt 0 ] +if [ -n "$TRAVIS_JOB_ID" -a $combined_trash_size -gt 0 ] then echo "------------------------------------------------------------------------" echo "Trash directories embedded in this log can be extracted by running:" diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh index d3cdbac73f..7d0beb2832 100755 --- a/ci/test-documentation.sh +++ b/ci/test-documentation.sh @@ -5,6 +5,7 @@ . ${0%/*}/lib.sh +test -n "$ALREADY_HAVE_ASCIIDOCTOR" || gem install asciidoctor make check-builtins From patchwork Wed Jan 16 13:36:14 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Passaro via GitGitGadget X-Patchwork-Id: 10765939 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E0B5C186E for ; Wed, 16 Jan 2019 13:36:20 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D0C662BD36 for ; Wed, 16 Jan 2019 13:36:20 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C4D302DD52; Wed, 16 Jan 2019 13:36:20 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7C4282BD36 for ; Wed, 16 Jan 2019 13:36:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2393211AbfAPNgS (ORCPT ); Wed, 16 Jan 2019 08:36:18 -0500 Received: from mail-ed1-f66.google.com ([209.85.208.66]:34912 "EHLO mail-ed1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390514AbfAPNgR (ORCPT ); Wed, 16 Jan 2019 08:36:17 -0500 Received: by mail-ed1-f66.google.com with SMTP id x30so5448284edx.2 for ; Wed, 16 Jan 2019 05:36:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=yjmVd1mMTh5O9pgiFT5EpWKGV+TAK5GJK5Kpo9RboIc=; b=UmpDX8ffZBxCxFnUrvObPE7CUSM8cbBiCDexCfNn20eUqONdyEJeQarDC41NuMChRa DTJxtlZQTjEySWtUQMHobuuRZ+KMnw/3Hz8hxLOyuG4EZQPopFwD5mF2ZLb/+PqvAx1N QErfOTRc5QYe24Za9HZBogKCcdk4Z5XDyJSQI5y4YYzsQkIJcPlu+Q/ja7HYjNCQBzpx d6zuTxT0xtKk+tlXV8Y9M15xF8tFj7QasnflOm8eXbfQTg0mugV1woeVORws2R5cpWzM x38vhvOiSCccVsIdn6M8QbNj68JjGe/9HSfAWeQu/VXzHuj7Kx55cDt09Gq9FLrviD4I f8Bw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=yjmVd1mMTh5O9pgiFT5EpWKGV+TAK5GJK5Kpo9RboIc=; b=aR4MLnlpPpjQg/mDSukkG227N4tNKcSmWcPmbu9nn29WIpXfNtZS+HyVLr1+BmaQ5Q RJ+4xuwbSMyhoTRgu+apJhYs+gKLUcAYLDYBWXe3mMCIQcwaQ1xOwuWn0yEtRVZfx5qb dHZ9ecoyVftEBssne+dASMkBCAPup+mmGt/W1Cn4pkNQ38kHHNZxDqkg84+1QofX+4qX zBnqRf6/ctsrvRToxcJcWn+Mdd73RMBZr7rb/PHb8i0AWr+7CTN0NeveN59SMK8cFNaQ 3szGtgmmn/edJigQ5sZi/qohlD5uUKjCe+we1nD8I32wNH75RbdII8Xl59DLQpzM9NPg qAVA== X-Gm-Message-State: AJcUukchR6R09hKH9U8wynuzol5PFSv1YcXqnlBHyhfMKpqqBgW+RuOB jVXHgC5Z5Pc5ZDpm4oXorAOp5Ayp X-Google-Smtp-Source: ALg8bN6q52ReVkltzlA8UTTPWtGeLY4qwXIRPbkHXS3y9BkoyEoWjA/hUB9CkQHD/0IJGegBHkBk2A== X-Received: by 2002:a17:906:1859:: with SMTP id w25-v6mr6894146eje.3.1547645775127; Wed, 16 Jan 2019 05:36:15 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id e14sm5541257edb.79.2019.01.16.05.36.14 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 16 Jan 2019 05:36:14 -0800 (PST) Date: Wed, 16 Jan 2019 05:36:14 -0800 (PST) X-Google-Original-Date: Wed, 16 Jan 2019 13:35:52 GMT Message-Id: <52fb8e72fb09a5b02e19de172ee28684d35af0a6.1547645770.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Johannes Schindelin via GitGitGadget" Subject: [PATCH v3 04/21] ci: inherit --jobs via MAKEFLAGS in run-build-and-tests Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Johannes Schindelin Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Johannes Schindelin Let's not decide in the generic ci/ script how many jobs to run in parallel; it is easy enough to hand that information down via the `MAKEFLAGS`. Signed-off-by: Johannes Schindelin --- ci/run-build-and-tests.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh index db342bb6a8..80d72d120f 100755 --- a/ci/run-build-and-tests.sh +++ b/ci/run-build-and-tests.sh @@ -7,7 +7,7 @@ ln -s "$cache_dir/.prove" t/.prove -make --jobs=2 +make make --quiet test if test "$jobname" = "linux-gcc" then From patchwork Wed Jan 16 13:36:15 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Passaro via GitGitGadget X-Patchwork-Id: 10765975 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 61A5313BF for ; Wed, 16 Jan 2019 13:36:48 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 50D402D172 for ; Wed, 16 Jan 2019 13:36:48 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 452E22DE2E; Wed, 16 Jan 2019 13:36:48 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E9C482D172 for ; Wed, 16 Jan 2019 13:36:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2393248AbfAPNgr (ORCPT ); Wed, 16 Jan 2019 08:36:47 -0500 Received: from mail-ed1-f67.google.com ([209.85.208.67]:43450 "EHLO mail-ed1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390489AbfAPNgS (ORCPT ); Wed, 16 Jan 2019 08:36:18 -0500 Received: by mail-ed1-f67.google.com with SMTP id f9so5395350eds.10 for ; Wed, 16 Jan 2019 05:36:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=lCqeYJ5kfLqoi2oIKKtYTdsgfR0s8LiDnVTc/mVzstc=; b=tawbc5JknGk1rrz5XREzH99okPiMB/82XTuhdu+TabsmK7oloCJCnfiuZhaINMYPGa E5LBQGm+kFluEdpmUmm8ZensEZw8j8nY6GWdkWypi3d9tE6WF0DU4sieL3JHWSId2BQm JFWNOEPvWcsKsMC6c3avisCBrXw1fZG/CSJIPuWJO9YdjvS5xZ5j0kSOTS2A2XoO/hoR J6vmOFNOFdsG2j6uiPtv9oO1ziBq9Z82UIonRIwwjGyMH7g9wITHqPBuT59/P3z7bqH7 ed098PieH6fK8MZkBkPIFZ2zwpdWGIjiFrdb/Dh5cluapRXFQIoLMnRfuyb4P/DM7dV7 fzaQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=lCqeYJ5kfLqoi2oIKKtYTdsgfR0s8LiDnVTc/mVzstc=; b=AQp8i7vKW+X01spm9Sjl3STsHJhlBXpaTW9z8rZkR0Od8ohkkUpnK8VhOHxjLNu6M1 wuT1dUyJvTBG9oRSIlFfW3aLrbvJRW8cg5EhmpUY2n5TGcSOjFt81Fk18lf0nQn8wCa+ zMoldyoWnsnyEdGMI8RaUGvYyunwONz5f4auhuJuP+QUYVQ7fvsUxjkfPbvRfI7+l+MA x4YMIpD6j/rY/9vMPivfPP3ppy0EecqVHthA5iU3QJYFMmkW2kEVyDSG22bSVwnRfuSC 3V1kyIN/Mg791TNw56DuEKC7Tx0vxVpbxzPrFw4PiUWqibGTkYKzt2QIsTwCJ4VQxCBb lpjA== X-Gm-Message-State: AJcUukffBtDrBAsVEeU0cAQhiwhJaNR2ZSfaIHz7w1mR+z7Wrc4DWzbs 7MF5EJFoKgRyj+A7FStKrgSWtMVO X-Google-Smtp-Source: ALg8bN6r4pzdurv2NytyLW8I6fHd8ZbZWgczUo6u8QeW2uwn8tqav+6tZ02LEVLKfMjbzkQnuCB5Lg== X-Received: by 2002:aa7:d88c:: with SMTP id u12mr7158149edq.237.1547645776129; Wed, 16 Jan 2019 05:36:16 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id g40sm5504564edg.39.2019.01.16.05.36.15 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 16 Jan 2019 05:36:15 -0800 (PST) Date: Wed, 16 Jan 2019 05:36:15 -0800 (PST) X-Google-Original-Date: Wed, 16 Jan 2019 13:35:53 GMT Message-Id: In-Reply-To: References: From: "Johannes Schindelin via GitGitGadget" Subject: [PATCH v3 05/21] ci: use a junction on Windows instead of a symlink Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Johannes Schindelin Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Johannes Schindelin Symbolic links are still not quite as easy to use on Windows as on Linux (for example, on versions older than Windows 10, only administrators can create symlinks, and on Windows 10 you still need to be in developer mode for regular users to have permission), but NTFS junctions can give us a way out. Signed-off-by: Johannes Schindelin --- ci/run-build-and-tests.sh | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh index 80d72d120f..74d838ea01 100755 --- a/ci/run-build-and-tests.sh +++ b/ci/run-build-and-tests.sh @@ -5,7 +5,10 @@ . ${0%/*}/lib.sh -ln -s "$cache_dir/.prove" t/.prove +case "$CI_OS_NAME" in +windows*) cmd //c mklink //j t\\.prove "$(cygpath -aw "$cache_dir/.prove")";; +*) ln -s "$cache_dir/.prove" t/.prove;; +esac make make --quiet test From patchwork Wed Jan 16 13:36:16 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Passaro via GitGitGadget X-Patchwork-Id: 10765943 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1651991E for ; Wed, 16 Jan 2019 13:36:25 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 056402BD36 for ; Wed, 16 Jan 2019 13:36:25 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id EDBF92DD52; Wed, 16 Jan 2019 13:36:24 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8FC1F2BD36 for ; Wed, 16 Jan 2019 13:36:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2393220AbfAPNgX (ORCPT ); Wed, 16 Jan 2019 08:36:23 -0500 Received: from mail-ed1-f45.google.com ([209.85.208.45]:44285 "EHLO mail-ed1-f45.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390475AbfAPNgT (ORCPT ); Wed, 16 Jan 2019 08:36:19 -0500 Received: by mail-ed1-f45.google.com with SMTP id y56so5390874edd.11 for ; Wed, 16 Jan 2019 05:36:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=bnpLIlyyLtIXH9ZdKeZynSDp+QyusEe60+Hv8ehTygc=; b=DfXSC78k5s0W8VnSqsztTDJU3818wG8JVltN2xnn48M6N00mmbHf6QQWXwC0OAENWZ c5+sL2953edljsOrZRm+oolKFRYq0aqKyLjHCl4H72U2NSy3AaYm+4zZ9R2evfh4yMU+ oyl+Va/jIsxOqXkEDM4b/zu7tgMd1wxtG3FoRwZc197GP+okEMMF56ig8D36q8dFuIYF zW6VmvkHbtO+Z8k8vQm5kjNLdp/9VmuybDgd7WIwY5PVnjqaniQqUgOJKDP/wDkyH70R Zl8UHd4AAPVanbl5HGlATcufvEBCVsBCUK+lo5a7i8Rkor7S16gcj80h4MVcBDuyZPA7 yJ9A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=bnpLIlyyLtIXH9ZdKeZynSDp+QyusEe60+Hv8ehTygc=; b=mh2p/h8+JV0oEGayBYtpEGHnRRS80y1YphomSLbIXk5jFmYZyVHce1Aaw+uXyCsSL2 cYNxLvcRj1pUHPRwJEEnsfftyDMZPwsQyGsyRofEQV/Eknl6RMv5QrK1I0yCDQTHTSQa cN/cMctf6uViz9CaEzHdANdrPVzYdXCPXxzs9zgRljLfq2oPwJ42kYdotcjWwPeLQK1f FW0MbScXxLoDxPYFGj1SoclGxM3+Dks+QzYyKiMpLNSeGA81avlhm7n2ZcnbJMlC6DN1 x1xjPIEaXbMKt0L/gR3s+FBK7U1jDpoyhOYzUWKneg1FUTNcCEIAR1Wv3eDNxES8ev2k nbJg== X-Gm-Message-State: AJcUukcDrjzXDc9o/bynkAWrUtO6pB+t7Kom7rwx6P/JnSCFJ2jIio45 4LzGbyis9JQ5UBi1yOIxYQRUOQIh X-Google-Smtp-Source: ALg8bN6XM8bSHWG4rv17ZrVTPofzGuETd73pFxXm2QV2dHPIWOH2VEDNTCz9DS3XskhZEkUIR1f2Jg== X-Received: by 2002:a50:bdc8:: with SMTP id z8mr7450698edh.46.1547645777176; Wed, 16 Jan 2019 05:36:17 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id s36sm5834959edb.43.2019.01.16.05.36.16 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 16 Jan 2019 05:36:16 -0800 (PST) Date: Wed, 16 Jan 2019 05:36:16 -0800 (PST) X-Google-Original-Date: Wed, 16 Jan 2019 13:35:54 GMT Message-Id: <16090ff67caeffe59bdc2294a93af634eb23e33e.1547645770.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Johannes Schindelin via GitGitGadget" Subject: [PATCH v3 06/21] test-date: add a subcommand to measure times in shell scripts Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Johannes Schindelin Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Johannes Schindelin In the next commit, we want to teach Git's test suite to optionally output test results in JUnit-style .xml files. These files contain information about the time spent. So we need a way to measure time. While we could use `date +%s` for that, this will give us only seconds, i.e. very coarse-grained timings. GNU `date` supports `date +%s.%N` (i.e. nanosecond-precision output), but there is no equivalent in BSD `date` (read: on macOS, we would not be able to obtain precise timings). So let's introduce `test-tool date getnanos`, with an optional start time, that outputs preciser values. Granted, it is a bit pointless to try measuring times accurately in shell scripts, certainly to nanosecond precision. But it is better than second-granularity. Signed-off-by: Johannes Schindelin --- t/helper/test-date.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/t/helper/test-date.c b/t/helper/test-date.c index a0837371ab..792a805374 100644 --- a/t/helper/test-date.c +++ b/t/helper/test-date.c @@ -7,6 +7,7 @@ static const char *usage_msg = "\n" " test-tool date parse [date]...\n" " test-tool date approxidate [date]...\n" " test-tool date timestamp [date]...\n" +" test-tool date getnanos [start-nanos]\n" " test-tool date is64bit\n" " test-tool date time_t-is64bit\n"; @@ -82,6 +83,15 @@ static void parse_approx_timestamp(const char **argv, struct timeval *now) } } +static void getnanos(const char **argv, struct timeval *now) +{ + double seconds = getnanotime() / 1.0e9; + + if (*argv) + seconds -= strtod(*argv, NULL); + printf("%lf\n", seconds); +} + int cmd__date(int argc, const char **argv) { struct timeval now; @@ -108,6 +118,8 @@ int cmd__date(int argc, const char **argv) parse_approxidate(argv+1, &now); else if (!strcmp(*argv, "timestamp")) parse_approx_timestamp(argv+1, &now); + else if (!strcmp(*argv, "getnanos")) + getnanos(argv+1, &now); else if (!strcmp(*argv, "is64bit")) return sizeof(timestamp_t) == 8 ? 0 : 1; else if (!strcmp(*argv, "time_t-is64bit")) From patchwork Wed Jan 16 13:36:17 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Passaro via GitGitGadget X-Patchwork-Id: 10765965 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8F52D91E for ; Wed, 16 Jan 2019 13:36:42 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7D8642D172 for ; Wed, 16 Jan 2019 13:36:42 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6EE562DE2E; Wed, 16 Jan 2019 13:36:42 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AAE402D172 for ; Wed, 16 Jan 2019 13:36:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2393218AbfAPNgW (ORCPT ); Wed, 16 Jan 2019 08:36:22 -0500 Received: from mail-ed1-f66.google.com ([209.85.208.66]:34917 "EHLO mail-ed1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2393212AbfAPNgV (ORCPT ); Wed, 16 Jan 2019 08:36:21 -0500 Received: by mail-ed1-f66.google.com with SMTP id x30so5448422edx.2 for ; Wed, 16 Jan 2019 05:36:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=3DGxN8Mavl1Nm4GqVulu8vxoERsc8p3SpIFDxpXgZDk=; b=jvNrMPgF6MIP5nQyh4fYsKRMeNP2whNtcqCCQkqLP27XvvCWk6LzjgW1apiiuT58Xf nUOeOyT+JiDjzJvVstNcJo058Nmuc5R0oztl/xlCblGSwa18Hs4SoJmDlDqh0iijgZuF WAyYIIxmjQ0WO4OBFDAAtNDGewMshkssR7F5u4v3H1EK+r38cxxmZCyIsIeMZeaz7HZh rkQQCJwtBFnWs4mO3crP3Dj2Mpe1uTqAJWkZKu+C9YI0E3lqzn4IXdBKH1/g8MUaLjh2 vP5JfRV4Z23/Puh5k7YKRTc64jtI7G+6FfnDv6Joy8afsWrPzIKFGXIlaWbIt4QzUl8a GzVg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=3DGxN8Mavl1Nm4GqVulu8vxoERsc8p3SpIFDxpXgZDk=; b=Ap/PrG/X6kstcRjsDP81gcIN9bIpj0o+CWTCFBEzwD95WF2qZBilP8MdQ9G80EFPFU 6cDwSxC93dlKJy3uI531Jo+uy3XO0coZ4Rn02PYlFIWcZ366I9mpbxmkzbKvL9H6TadR cFaI0gh7UI3xF7fXA3oZP2LUsJY9FIpX9gN0Ij7HjrS6k3xf/3dx8SfguHzQA6VfoFzT /JcTvAPUhPsZ3Q9nkNEOer4DgdhUGtwKSYdYimI5xRyDSuLbDPf1X0DB59w9d2XudfjA 4oMQF4wwx7WtDwbwlptUOW9dWLQfzvVsgMvTdj81WT1d7lJBiJGNsFqfHRCCa3tSh3uZ u12w== X-Gm-Message-State: AJcUukcfNuZEdPhTP6gygSe73Jz8SRkIfOvpelwEt0/isepiJ/yx0bqf QHT6x9EDQ3eqrO0Gw7bLHs9GLUCm X-Google-Smtp-Source: ALg8bN4NstBG+gGicLhj3Sg0TkYK02X9KZAw0NFRTlSB+YX4Nf1PdhE/17vUFkyWDKMYoAQldFxwtA== X-Received: by 2002:a50:d753:: with SMTP id i19mr7580055edj.75.1547645778021; Wed, 16 Jan 2019 05:36:18 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id m5-v6sm3094670ejq.21.2019.01.16.05.36.17 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 16 Jan 2019 05:36:17 -0800 (PST) Date: Wed, 16 Jan 2019 05:36:17 -0800 (PST) X-Google-Original-Date: Wed, 16 Jan 2019 13:35:55 GMT Message-Id: <272c0c044617977c838dcf97d60f9315f8acb36f.1547645770.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Johannes Schindelin via GitGitGadget" Subject: [PATCH v3 07/21] tests: optionally write results as JUnit-style .xml Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Johannes Schindelin Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Johannes Schindelin This will come in handy when publishing the results of Git's test suite during an automated Azure DevOps run. Note: we need to make extra sure that invalid UTF-8 encoding is turned into valid UTF-8 (using the Replacement Character, \uFFFD) because t9902's trace contains such invalid byte sequences, and the task in the Azure Pipeline that uploads the test results would refuse to do anything if it was asked to parse an .xml file with invalid UTF-8 in it. Signed-off-by: Johannes Schindelin --- Makefile | 1 + t/.gitignore | 1 + t/helper/test-tool.c | 1 + t/helper/test-tool.h | 1 + t/helper/test-xml-encode.c | 80 +++++++++++++++++++++++++++++++++ t/test-lib.sh | 91 ++++++++++++++++++++++++++++++++++++++ 6 files changed, 175 insertions(+) create mode 100644 t/helper/test-xml-encode.c diff --git a/Makefile b/Makefile index 1a44c811aa..044b4f77bd 100644 --- a/Makefile +++ b/Makefile @@ -754,6 +754,7 @@ TEST_BUILTINS_OBJS += test-submodule-config.o TEST_BUILTINS_OBJS += test-submodule-nested-repo-config.o TEST_BUILTINS_OBJS += test-subprocess.o TEST_BUILTINS_OBJS += test-urlmatch-normalization.o +TEST_BUILTINS_OBJS += test-xml-encode.o TEST_BUILTINS_OBJS += test-wildmatch.o TEST_BUILTINS_OBJS += test-windows-named-pipe.o TEST_BUILTINS_OBJS += test-write-cache.o diff --git a/t/.gitignore b/t/.gitignore index 348715f0e4..91cf5772fe 100644 --- a/t/.gitignore +++ b/t/.gitignore @@ -2,3 +2,4 @@ /test-results /.prove /chainlinttmp +/out/ diff --git a/t/helper/test-tool.c b/t/helper/test-tool.c index bfb195b1a8..4b4b397d93 100644 --- a/t/helper/test-tool.c +++ b/t/helper/test-tool.c @@ -49,6 +49,7 @@ static struct test_cmd cmds[] = { { "submodule-nested-repo-config", cmd__submodule_nested_repo_config }, { "subprocess", cmd__subprocess }, { "urlmatch-normalization", cmd__urlmatch_normalization }, + { "xml-encode", cmd__xml_encode }, { "wildmatch", cmd__wildmatch }, #ifdef GIT_WINDOWS_NATIVE { "windows-named-pipe", cmd__windows_named_pipe }, diff --git a/t/helper/test-tool.h b/t/helper/test-tool.h index 042f12464b..c0ab65e370 100644 --- a/t/helper/test-tool.h +++ b/t/helper/test-tool.h @@ -45,6 +45,7 @@ int cmd__submodule_config(int argc, const char **argv); int cmd__submodule_nested_repo_config(int argc, const char **argv); int cmd__subprocess(int argc, const char **argv); int cmd__urlmatch_normalization(int argc, const char **argv); +int cmd__xml_encode(int argc, const char **argv); int cmd__wildmatch(int argc, const char **argv); #ifdef GIT_WINDOWS_NATIVE int cmd__windows_named_pipe(int argc, const char **argv); diff --git a/t/helper/test-xml-encode.c b/t/helper/test-xml-encode.c new file mode 100644 index 0000000000..367c4875e6 --- /dev/null +++ b/t/helper/test-xml-encode.c @@ -0,0 +1,80 @@ +#include "test-tool.h" + +static const char *utf8_replace_character = "�"; + +/* + * Encodes (possibly incorrect) UTF-8 on to , to be embedded + * in an XML file. + */ +int cmd__xml_encode(int argc, const char **argv) +{ + unsigned char buf[1024], tmp[4], *tmp2 = NULL; + ssize_t cur = 0, len = 1, remaining = 0; + unsigned char ch; + + for (;;) { + if (++cur == len) { + len = xread(0, buf, sizeof(buf)); + if (!len) + return 0; + if (len < 0) + die_errno("Could not read "); + cur = 0; + } + ch = buf[cur]; + + if (tmp2) { + if ((ch & 0xc0) != 0x80) { + fputs(utf8_replace_character, stdout); + tmp2 = 0; + cur--; + continue; + } + *tmp2 = ch; + tmp2++; + if (--remaining == 0) { + fwrite(tmp, tmp2 - tmp, 1, stdout); + tmp2 = 0; + } + continue; + } + + if (!(ch & 0x80)) { + /* 0xxxxxxx */ + if (ch == '&') + fputs("&", stdout); + else if (ch == '\'') + fputs("'", stdout); + else if (ch == '"') + fputs(""", stdout); + else if (ch == '<') + fputs("<", stdout); + else if (ch == '>') + fputs(">", stdout); + else if (ch >= 0x20) + fputc(ch, stdout); + else if (ch == 0x09 || ch == 0x0a || ch == 0x0d) + fprintf(stdout, "&#x%02x;", ch); + else + fputs(utf8_replace_character, stdout); + } else if ((ch & 0xe0) == 0xc0) { + /* 110XXXXx 10xxxxxx */ + tmp[0] = ch; + remaining = 1; + tmp2 = tmp + 1; + } else if ((ch & 0xf0) == 0xe0) { + /* 1110XXXX 10Xxxxxx 10xxxxxx */ + tmp[0] = ch; + remaining = 2; + tmp2 = tmp + 1; + } else if ((ch & 0xf8) == 0xf0) { + /* 11110XXX 10XXxxxx 10xxxxxx 10xxxxxx */ + tmp[0] = ch; + remaining = 3; + tmp2 = tmp + 1; + } else + fputs(utf8_replace_character, stdout); + } + + return 0; +} diff --git a/t/test-lib.sh b/t/test-lib.sh index 0f1faa24b2..e9782b6b32 100644 --- a/t/test-lib.sh +++ b/t/test-lib.sh @@ -339,6 +339,9 @@ do -V|--verbose-log) verbose_log=t shift ;; + --write-junit-xml) + write_junit_xml=t + shift ;; *) echo "error: unknown test option '$1'" >&2; exit 1 ;; esac @@ -486,11 +489,24 @@ trap 'exit $?' INT # the test_expect_* functions instead. test_ok_ () { + if test -n "$write_junit_xml" + then + write_junit_xml_testcase "$*" + fi test_success=$(($test_success + 1)) say_color "" "ok $test_count - $@" } test_failure_ () { + if test -n "$write_junit_xml" + then + junit_insert="" + junit_insert="$junit_insert $(xml_attr_encode \ + "$(printf '%s\n' "$@" | sed 1d)")" + junit_insert="$junit_insert" + write_junit_xml_testcase "$1" " $junit_insert" + fi test_failure=$(($test_failure + 1)) say_color error "not ok $test_count - $1" shift @@ -499,11 +515,19 @@ test_failure_ () { } test_known_broken_ok_ () { + if test -n "$write_junit_xml" + then + write_junit_xml_testcase "$* (breakage fixed)" + fi test_fixed=$(($test_fixed+1)) say_color error "ok $test_count - $@ # TODO known breakage vanished" } test_known_broken_failure_ () { + if test -n "$write_junit_xml" + then + write_junit_xml_testcase "$* (known breakage)" + fi test_broken=$(($test_broken+1)) say_color warn "not ok $test_count - $@ # TODO known breakage" } @@ -761,6 +785,10 @@ test_start_ () { test_count=$(($test_count+1)) maybe_setup_verbose maybe_setup_valgrind + if test -n "$write_junit_xml" + then + junit_start=$(test-tool date getnanos) + fi } test_finish_ () { @@ -798,6 +826,13 @@ test_skip () { case "$to_skip" in t) + if test -n "$write_junit_xml" + then + message="$(xml_attr_encode "$skipped_reason")" + write_junit_xml_testcase "$1" \ + " " + fi + say_color skip >&3 "skipping test: $@" say_color skip "ok $test_count # skip $1 ($skipped_reason)" : true @@ -813,9 +848,51 @@ test_at_end_hook_ () { : } +write_junit_xml () { + case "$1" in + --truncate) + >"$junit_xml_path" + junit_have_testcase= + shift + ;; + esac + printf '%s\n' "$@" >>"$junit_xml_path" +} + +xml_attr_encode () { + printf '%s\n' "$@" | test-tool xml-encode +} + +write_junit_xml_testcase () { + junit_attrs="name=\"$(xml_attr_encode "$this_test.$test_count $1")\"" + shift + junit_attrs="$junit_attrs classname=\"$this_test\"" + junit_attrs="$junit_attrs time=\"$(test-tool \ + date getnanos $junit_start)\"" + write_junit_xml "$(printf '%s\n' \ + " " "$@" " ")" + junit_have_testcase=t +} + test_done () { GIT_EXIT_OK=t + if test -n "$write_junit_xml" && test -n "$junit_xml_path" + then + test -n "$junit_have_testcase" || { + junit_start=$(test-tool date getnanos) + write_junit_xml_testcase "all tests skipped" + } + + # adjust the overall time + junit_time=$(test-tool date getnanos $junit_suite_start) + sed "s/]*/& time=\"$junit_time\"/" \ + <"$junit_xml_path" >"$junit_xml_path.new" + mv "$junit_xml_path.new" "$junit_xml_path" + + write_junit_xml " " "" + fi + if test -z "$HARNESS_ACTIVE" then test_results_dir="$TEST_OUTPUT_DIRECTORY/test-results" @@ -1051,6 +1128,7 @@ then else mkdir -p "$TRASH_DIRECTORY" fi + # Use -P to resolve symlinks in our working directory so that the cwd # in subprocesses like git equals our $PWD (for pathname comparisons). cd -P "$TRASH_DIRECTORY" || exit 1 @@ -1064,6 +1142,19 @@ then test_done fi +if test -n "$write_junit_xml" +then + junit_xml_dir="$TEST_OUTPUT_DIRECTORY/out" + mkdir -p "$junit_xml_dir" + junit_xml_base=${0##*/} + junit_xml_path="$junit_xml_dir/TEST-${junit_xml_base%.sh}.xml" + junit_attrs="name=\"${junit_xml_base%.sh}\"" + junit_attrs="$junit_attrs timestamp=\"$(TZ=UTC \ + date +%Y-%m-%dT%H:%M:%S)\"" + write_junit_xml --truncate "" " " + junit_suite_start=$(test-tool date getnanos) +fi + # Provide an implementation of the 'yes' utility yes () { if test $# = 0 From patchwork Wed Jan 16 13:36:18 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Passaro via GitGitGadget X-Patchwork-Id: 10765973 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D26AC91E for ; Wed, 16 Jan 2019 13:36:46 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C21702D172 for ; Wed, 16 Jan 2019 13:36:46 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B63DA2DE2E; Wed, 16 Jan 2019 13:36:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5B2032D172 for ; Wed, 16 Jan 2019 13:36:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2393216AbfAPNgW (ORCPT ); Wed, 16 Jan 2019 08:36:22 -0500 Received: from mail-ed1-f47.google.com ([209.85.208.47]:43518 "EHLO mail-ed1-f47.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390514AbfAPNgV (ORCPT ); Wed, 16 Jan 2019 08:36:21 -0500 Received: by mail-ed1-f47.google.com with SMTP id f9so5395474eds.10 for ; Wed, 16 Jan 2019 05:36:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=d8mn0sQqmpMCyJw+G3k+5UZvKwi59kYYKD0XNAi7wSI=; b=GpCkGpRmWchQwcYjbXmvkF8db0mfNw91XwLc1xGCLN6uTgsuhahkgN9WVh//UOjCPX 9nLkKCOQpGqTEkBYcXEFMWtJmYR+D5yYeQN2mOY7bfn4DfjyClnWR7YhxC07slSf62HR Jr1nSibA1Vpc+iybfq9h8180LSmS5Xb2C32KTG+dUdVqNNSRc4O8gt1vyFQw6+tvOFzj 8jSb24ITa3tdSIDKZxdSsJKl06dMXO2jMx5jtiULMDPsU57BZY4C1Ddh3yGwMu0xLWZG cmi8fytDFohMGPR/ghOkoifCSkh+kVkVUNvOQtoBFpH9PuxjQ3L2yzQaLpePtOadtQnN iS6A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=d8mn0sQqmpMCyJw+G3k+5UZvKwi59kYYKD0XNAi7wSI=; b=PqjXAZ8BC0LE9g/eJHCkAUG+3StkwDlgygY13BUQlCWD6EjG3PxZ4Ee82J0Csf0F1r D1coVD2sKw7iz1gM+FWJnIA0HgnAP4evS0FL7qufk5oQ40IUzhDs1Hfm3RYngO/zCiYo dmhRDW8oLdv2XjZrL7pquzb6PKxdZhboZ/jWk7TI1rUfGNtzoghAHcMmsaXJNNEILvKe JodG/m6tX2b6z2/mDMDtt59pjB9srfogiiPQFAtpdKVXHQK52OKbfcItjtjEpS08xVSl 3VKyS6bhMk7IuUO0PRM2rIKNdb7pE5/ChoICbBQUaDCv8jBzPpg87RfA+pcKf1EuRjue 90hw== X-Gm-Message-State: AJcUukfBK0v/Li7GOAFH/fX1CbnQhUELV9LpQyZFpiR0zQ0tuuCJcS9y TdbPDOgCuxYVAfKEI34sqbv7ljGg X-Google-Smtp-Source: ALg8bN7hlqNb0tj3yKZI+j61/2flaOwDeK4pTGQgdeviBu49Z25/HA87Yx1QhcwbCfcPT6Sx0khYhg== X-Received: by 2002:a50:ae64:: with SMTP id c91mr7481700edd.222.1547645779014; Wed, 16 Jan 2019 05:36:19 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id g40sm5504591edg.39.2019.01.16.05.36.18 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 16 Jan 2019 05:36:18 -0800 (PST) Date: Wed, 16 Jan 2019 05:36:18 -0800 (PST) X-Google-Original-Date: Wed, 16 Jan 2019 13:35:56 GMT Message-Id: In-Reply-To: References: From: "Johannes Schindelin via GitGitGadget" Subject: [PATCH v3 08/21] ci/lib.sh: add support for Azure Pipelines Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Johannes Schindelin Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Johannes Schindelin This patch introduces a conditional arm that defines some environment variables and a function that displays the URL given the job id (to identify previous runs for known-good trees). For example, we do not have to install the git-lfs and gettext packages on Azure Pipelines' macOS agents: they are already installed, and trying to install them again would result in an error. Signed-off-by: Johannes Schindelin --- ci/lib.sh | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/ci/lib.sh b/ci/lib.sh index bfc406a63b..e560fae3aa 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -20,6 +20,29 @@ then BREW_INSTALL_PACKAGES="git-lfs gettext" export GIT_PROVE_OPTS="--timer --jobs 3 --state=failed,slow,save" export GIT_TEST_OPTS="--verbose-log -x --immediate" +elif test -n "$SYSTEM_COLLECTIONURI" || test -n "$SYSTEM_TASKDEFINITIONSURI" +then + # We are running in Azure Pipelines + CI_BRANCH="$BUILD_SOURCEBRANCH" + CI_COMMIT="$BUILD_SOURCEVERSION" + CI_JOB_ID="$BUILD_BUILDID" + CI_JOB_NUMBER="$BUILD_BUILDNUMBER" + CI_OS_NAME="$(echo "$AGENT_OS" | tr A-Z a-z)" + test darwin != "$CI_OS_NAME" || CI_OS_NAME=osx + CI_REPO_SLUG="$(expr "$BUILD_REPOSITORY_URI" : '.*/\([^/]*/[^/]*\)$')" + CC="${CC:-gcc}" + + # use a subdirectory of the cache dir (because the file share is shared + # among *all* phases) + cache_dir="$HOME/test-cache/$SYSTEM_PHASENAME" + + url_for_job_id () { + echo "$SYSTEM_TASKDEFINITIONSURI$SYSTEM_TEAMPROJECT/_build/results?buildId=$1" + } + + BREW_INSTALL_PACKAGES= + export GIT_PROVE_OPTS="--timer --jobs 10 --state=failed,slow,save" + export GIT_TEST_OPTS="--quiet --write-junit-xml" fi skip_branch_tip_with_tag () { From patchwork Wed Jan 16 13:36:19 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Passaro via GitGitGadget X-Patchwork-Id: 10765947 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 955D191E for ; Wed, 16 Jan 2019 13:36:27 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 831B42D172 for ; Wed, 16 Jan 2019 13:36:27 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 778352DE14; Wed, 16 Jan 2019 13:36:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 799792DD52 for ; Wed, 16 Jan 2019 13:36:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2393223AbfAPNgX (ORCPT ); Wed, 16 Jan 2019 08:36:23 -0500 Received: from mail-ed1-f52.google.com ([209.85.208.52]:34800 "EHLO mail-ed1-f52.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2393213AbfAPNgX (ORCPT ); Wed, 16 Jan 2019 08:36:23 -0500 Received: by mail-ed1-f52.google.com with SMTP id b3so5465754ede.1 for ; Wed, 16 Jan 2019 05:36:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=eEsPtggeKU1T16mYrxdfOnWERMc44Fk2vwvx8Q5ak/Q=; b=O3GeaF2j8/tIGTxMYh9beCgXVXz7dieNMAADaePzOnc97t7jg2XBNzo4rCx6DVKMbB Aga+sOR5qpMyd5MBwWLPagU7AaYYydwbEWkFtZATHDBIsx7Rk7115wJuihZo9RQyjWi4 nYbidlia9PyKVFMy/8zSanxdX2cvNiUi3zaAfDwDg2Epx7wkDMF5DUJRQEUnEksAqyDO UIntMu3o79i4OBYalT9gUwmDY+nXMpiUwGn0N8CfR9jz/9NjJlVH8QNZYORWllCh8iWM pwAmNSmTgswDiRvFXiwS7JajxCWzklEItuALb+fBb6bU/g3oreNeH/zZEgfz60j2TRup o+bQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=eEsPtggeKU1T16mYrxdfOnWERMc44Fk2vwvx8Q5ak/Q=; b=reQUN9u4Kks4ZjbwMY8ZPQEz5Ptp7J5oyObvrAXQvOnq1lqwGHEkajn15D5vPVxTJ4 lekprsTeEX1vJU4yEPpvQyhHW7jfkbrIcXJTjb2R2UNsXoZYzxhZkQPzs1MoWFjfvjiZ HIVDMDmOSn7uwhIlGZkJBNteKwhBRYeD/CFNNk4j15oSo4qn/ttmv+EHhbUV9GTGyfRh Qvm0Lf2LrGMdhAjE2t0bgElw1tAAsn2dJ6c8X9xTOk/n74A2B8SLQfM3+PLMF53dB4QN No0xGOsO1YlsUr+sBCHZzvdtb/d3ZN/fucgp+iu8OMvMIKZQNBrXVFmHCUCsKkalJ6Wg dPqA== X-Gm-Message-State: AJcUukf+3sxvRiT3xAqUngYzc237hQfisxhEzmYVIftydZ1R0zBgqFmf kXCZJOee9TqCH9K5HbQve2LEpskc X-Google-Smtp-Source: ALg8bN7kICgPVX/78HyW8SaK/tdbdV9++Xu575NbordSJ/PIE7OgnG7ecDYZ7JsQGg12bnRy2KXoQQ== X-Received: by 2002:a50:8863:: with SMTP id c32mr7232102edc.156.1547645779882; Wed, 16 Jan 2019 05:36:19 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id s46sm5552302edd.9.2019.01.16.05.36.19 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 16 Jan 2019 05:36:19 -0800 (PST) Date: Wed, 16 Jan 2019 05:36:19 -0800 (PST) X-Google-Original-Date: Wed, 16 Jan 2019 13:35:57 GMT Message-Id: In-Reply-To: References: From: "Johannes Schindelin via GitGitGadget" Subject: [PATCH v3 09/21] Add a build definition for Azure DevOps Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Johannes Schindelin Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Johannes Schindelin This commit adds an azure-pipelines.yml file which is Azure DevOps' equivalent to Travis CI's .travis.yml. To make things a bit easier to understand, we refrain from using the `matrix` feature here because (while it is powerful) it can be a bit confusing to users who are not familiar with CI setups. Therefore, we use a separate phase even for similar configurations (such as GCC vs Clang on Linux, GCC vs Clang on macOS). Also, we make use of the shiny new feature we just introduced where the test suite can output JUnit-style .xml files. This information is made available in a nice UI that allows the viewer to filter by phase and/or test number, and to see trends such as: number of (failing) tests, time spent running the test suite, etc. Signed-off-by: Johannes Schindelin --- azure-pipelines.yml | 306 ++++++++++++++++++++++++++++++++++++++++++ ci/lib.sh | 2 +- ci/mount-fileshare.sh | 25 ++++ 3 files changed, 332 insertions(+), 1 deletion(-) create mode 100644 azure-pipelines.yml create mode 100755 ci/mount-fileshare.sh diff --git a/azure-pipelines.yml b/azure-pipelines.yml new file mode 100644 index 0000000000..f3cabb0dd0 --- /dev/null +++ b/azure-pipelines.yml @@ -0,0 +1,306 @@ +resources: +- repo: self + fetchDepth: 1 + +jobs: +- job: linux_clang + displayName: linux-clang + condition: succeeded() + pool: Hosted Ubuntu 1604 + steps: + - bash: | + test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1 + + sudo apt-get update && + sudo apt-get -y install git gcc make libssl-dev libcurl4-openssl-dev libexpat-dev tcl tk gettext git-email zlib1g-dev apache2-bin && + + export CC=clang || exit 1 + + ci/install-dependencies.sh || exit 1 + ci/run-build-and-tests.sh || { + ci/print-test-failures.sh + exit 1 + } + + test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || exit 1 + displayName: 'ci/run-build-and-tests.sh' + env: + GITFILESHAREPWD: $(gitfileshare.pwd) + - task: PublishTestResults@2 + displayName: 'Publish Test Results **/TEST-*.xml' + inputs: + mergeTestResults: true + testRunTitle: 'linux-clang' + platform: Linux + publishRunAttachments: false + condition: succeededOrFailed() + +- job: linux_gcc + displayName: linux-gcc + condition: succeeded() + pool: Hosted Ubuntu 1604 + steps: + - bash: | + test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1 + + sudo add-apt-repository ppa:ubuntu-toolchain-r/test && + sudo apt-get update && + sudo apt-get -y install git gcc make libssl-dev libcurl4-openssl-dev libexpat-dev tcl tk gettext git-email zlib1g-dev apache2 language-pack-is git-svn gcc-8 || exit 1 + + ci/install-dependencies.sh || exit 1 + ci/run-build-and-tests.sh || { + ci/print-test-failures.sh + exit 1 + } + + test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || exit 1 + displayName: 'ci/run-build-and-tests.sh' + env: + GITFILESHAREPWD: $(gitfileshare.pwd) + - task: PublishTestResults@2 + displayName: 'Publish Test Results **/TEST-*.xml' + inputs: + mergeTestResults: true + testRunTitle: 'linux-gcc' + platform: Linux + publishRunAttachments: false + condition: succeededOrFailed() + +- job: osx_clang + displayName: osx-clang + condition: succeeded() + pool: Hosted macOS + steps: + - bash: | + test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1 + + export CC=clang + + ci/install-dependencies.sh || exit 1 + ci/run-build-and-tests.sh || { + ci/print-test-failures.sh + exit 1 + } + + test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || umount "$HOME/test-cache" || exit 1 + displayName: 'ci/run-build-and-tests.sh' + env: + GITFILESHAREPWD: $(gitfileshare.pwd) + - task: PublishTestResults@2 + displayName: 'Publish Test Results **/TEST-*.xml' + inputs: + mergeTestResults: true + testRunTitle: 'osx-clang' + platform: macOS + publishRunAttachments: false + condition: succeededOrFailed() + +- job: osx_gcc + displayName: osx-gcc + condition: succeeded() + pool: Hosted macOS + steps: + - bash: | + test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1 + + ci/install-dependencies.sh || exit 1 + ci/run-build-and-tests.sh || { + ci/print-test-failures.sh + exit 1 + } + + test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || umount "$HOME/test-cache" || exit 1 + displayName: 'ci/run-build-and-tests.sh' + env: + GITFILESHAREPWD: $(gitfileshare.pwd) + - task: PublishTestResults@2 + displayName: 'Publish Test Results **/TEST-*.xml' + inputs: + mergeTestResults: true + testRunTitle: 'osx-gcc' + platform: macOS + publishRunAttachments: false + condition: succeededOrFailed() + +- job: gettext_poison + displayName: GETTEXT_POISON + condition: succeeded() + pool: Hosted Ubuntu 1604 + steps: + - bash: | + test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1 + + sudo apt-get update && + sudo apt-get -y install git gcc make libssl-dev libcurl4-openssl-dev libexpat-dev tcl tk gettext git-email zlib1g-dev && + + export jobname=GETTEXT_POISON || exit 1 + + ci/run-build-and-tests.sh || { + ci/print-test-failures.sh + exit 1 + } + + test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || exit 1 + displayName: 'ci/run-build-and-tests.sh' + env: + GITFILESHAREPWD: $(gitfileshare.pwd) + - task: PublishTestResults@2 + displayName: 'Publish Test Results **/TEST-*.xml' + inputs: + mergeTestResults: true + testRunTitle: 'gettext-poison' + platform: Linux + publishRunAttachments: false + condition: succeededOrFailed() + +- job: windows + displayName: Windows + condition: succeeded() + pool: Hosted + timeoutInMinutes: 240 + steps: + - powershell: | + if ("$GITFILESHAREPWD" -ne "" -and "$GITFILESHAREPWD" -ne "`$`(gitfileshare.pwd)") { + net use s: \\gitfileshare.file.core.windows.net\test-cache "$GITFILESHAREPWD" /user:AZURE\gitfileshare /persistent:no + cmd /c mklink /d "$(Build.SourcesDirectory)\test-cache" S:\ + } + displayName: 'Mount test-cache' + env: + GITFILESHAREPWD: $(gitfileshare.pwd) + - powershell: | + # Helper to check the error level of the latest command (exit with error when appropriate) + function c() { if (!$?) { exit(1) } } + + # Add build agent's MinGit to PATH + $env:PATH = $env:AGENT_HOMEDIRECTORY +"\externals\\git\cmd;" +$env:PATH + + # Helper to initialize (or update) a Git worktree + function init ($path, $url, $set_origin) { + if (Test-Path $path) { + cd $path; c + if (Test-Path .git) { + & git init; c + } else { + & git status + } + } else { + & git init $path; c + cd $path; c + } + & git config core.autocrlf false; c + & git config core.untrackedCache true; c + if (($set_origin -ne 0) -and !(git config remote.origin.url)) { + & git remote add origin $url; c + } + & git fetch --depth=1 $url master; c + & git reset --hard FETCH_HEAD; c + & git clean -df; c + } + + # Initialize Git for Windows' SDK + $sdk_path = "$(Build.SourcesDirectory)\git-sdk-64" + init "$sdk_path" "https://dev.azure.com/git-for-windows/git-sdk-64/_git/git-sdk-64" 0 + + # Let Git ignore the SDK and the test-cache + "/git-sdk-64/`n/test-cache/`n" | Out-File -NoNewLine -Encoding ascii -Append "$(Build.SourcesDirectory)\.git\info\exclude" + displayName: 'Initialize the Git for Windows SDK' + - powershell: | + & "git-sdk-64\git-cmd.exe" --command=usr\\bin\\bash.exe -lc @" + export MAKEFLAGS=-j10 + export DEVELOPER=1 + export NO_PERL=1 + export NO_SVN_TESTS=1 + export GIT_TEST_SKIP_REBASE_P=1 + + ci/run-build-and-tests.sh || { + ci/print-test-failures.sh + exit 1 + } + "@ + if (!$?) { exit(1) } + displayName: 'Build & Test' + env: + HOME: $(Build.SourcesDirectory) + MSYSTEM: MINGW64 + - powershell: | + if ("$GITFILESHAREPWD" -ne "" -and "$GITFILESHAREPWD" -ne "`$`(gitfileshare.pwd)") { + cmd /c rmdir "$(Build.SourcesDirectory)\test-cache" + } + displayName: 'Unmount test-cache' + condition: true + env: + GITFILESHAREPWD: $(gitfileshare.pwd) + - task: PublishTestResults@2 + displayName: 'Publish Test Results **/TEST-*.xml' + inputs: + mergeTestResults: true + testRunTitle: 'windows' + platform: Windows + publishRunAttachments: false + condition: succeededOrFailed() + +- job: linux32 + displayName: Linux32 + condition: succeeded() + pool: Hosted Ubuntu 1604 + steps: + - bash: | + test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1 + + sudo AGENT_OS="$AGENT_OS" BUILD_BUILDNUMBER="$BUILD_BUILDNUMBER" BUILD_REPOSITORY_URI="$BUILD_REPOSITORY_URI" BUILD_SOURCEBRANCH="$BUILD_SOURCEBRANCH" BUILD_SOURCEVERSION="$BUILD_SOURCEVERSION" SYSTEM_PHASENAME="$SYSTEM_PHASENAME" SYSTEM_TASKDEFINITIONSURI="$SYSTEM_TASKDEFINITIONSURI" SYSTEM_TEAMPROJECT="$SYSTEM_TEAMPROJECT" CC=$CC MAKEFLAGS=-j3 bash -lxc ci/run-linux32-docker.sh || exit 1 + + sudo chmod a+r t/out/TEST-*.xml + + test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || exit 1 + displayName: 'ci/run-linux32-docker.sh' + env: + GITFILESHAREPWD: $(gitfileshare.pwd) + - task: PublishTestResults@2 + displayName: 'Publish Test Results **/TEST-*.xml' + inputs: + mergeTestResults: true + testRunTitle: 'linux32' + platform: Linux + publishRunAttachments: false + condition: succeededOrFailed() + +- job: static_analysis + displayName: StaticAnalysis + condition: succeeded() + pool: Hosted Ubuntu 1604 + steps: + - bash: | + test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1 + + sudo apt-get update && + sudo apt-get install -y coccinelle && + + export jobname=StaticAnalysis && + + ci/run-static-analysis.sh || exit 1 + + test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || exit 1 + displayName: 'ci/run-static-analysis.sh' + env: + GITFILESHAREPWD: $(gitfileshare.pwd) + +- job: documentation + displayName: Documentation + condition: succeeded() + pool: Hosted Ubuntu 1604 + steps: + - bash: | + test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1 + + sudo apt-get update && + sudo apt-get install -y asciidoc xmlto asciidoctor && + + export ALREADY_HAVE_ASCIIDOCTOR=yes. && + export jobname=Documentation && + + ci/test-documentation.sh || exit 1 + + test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || exit 1 + displayName: 'ci/test-documentation.sh' + env: + GITFILESHAREPWD: $(gitfileshare.pwd) diff --git a/ci/lib.sh b/ci/lib.sh index e560fae3aa..c5228c1547 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -42,7 +42,7 @@ then BREW_INSTALL_PACKAGES= export GIT_PROVE_OPTS="--timer --jobs 10 --state=failed,slow,save" - export GIT_TEST_OPTS="--quiet --write-junit-xml" + export GIT_TEST_OPTS="--verbose-log -x --write-junit-xml" fi skip_branch_tip_with_tag () { diff --git a/ci/mount-fileshare.sh b/ci/mount-fileshare.sh new file mode 100755 index 0000000000..26b58a8096 --- /dev/null +++ b/ci/mount-fileshare.sh @@ -0,0 +1,25 @@ +#!/bin/sh + +die () { + echo "$*" >&2 + exit 1 +} + +test $# = 4 || +die "Usage: $0 " + +mkdir -p "$4" || die "Could not create $4" + +case "$(uname -s)" in +Linux) + sudo mount -t cifs -o vers=3.0,username="$2",password="$3",dir_mode=0777,file_mode=0777,serverino "$1" "$4" + ;; +Darwin) + pass="$(echo "$3" | sed -e 's/\//%2F/g' -e 's/+/%2B/g')" && + mount -t smbfs,soft "smb://$2:$pass@${1#//}" "$4" + ;; +*) + die "No support for $(uname -s)" + ;; +esac || +die "Could not mount $4" From patchwork Wed Jan 16 13:36:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Passaro via GitGitGadget X-Patchwork-Id: 10765945 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CB60D91E for ; Wed, 16 Jan 2019 13:36:26 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B78B12D172 for ; Wed, 16 Jan 2019 13:36:26 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A893F2DE2E; Wed, 16 Jan 2019 13:36:26 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 07F8C2D172 for ; Wed, 16 Jan 2019 13:36:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2393226AbfAPNgY (ORCPT ); Wed, 16 Jan 2019 08:36:24 -0500 Received: from mail-ed1-f65.google.com ([209.85.208.65]:36920 "EHLO mail-ed1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2393214AbfAPNgX (ORCPT ); Wed, 16 Jan 2019 08:36:23 -0500 Received: by mail-ed1-f65.google.com with SMTP id h15so5437508edb.4 for ; Wed, 16 Jan 2019 05:36:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=sWRntq7x9k3MAHX6XyhqaSg3cYHmaN4xC2+w1hNZ+fo=; b=ZGQSOZYxgRscH5VVJ9QF/a/dgBYnyE6HLZnxHtwyp3yIjC4vAiQxUiRvLJEXtJu6jp EPHfA/lg2y4Qey21R+KnHhCfyD85YOeSdkvHl9JdgwnNM6a6/UEatgJiH1zr/xhrbzRh r0211hjpH9GYeN8LtIyLosoqemuTWZC6k8JB6dW5jQtz6mG9Mou1X2JeKTV1s+9JF6w0 E9BL9wvfVJh/4/V+caeUZwjiFhM+jKJ6YLhlYXdf5rzwGBfFk2kd90ICJrUSDtZ3X/N4 +lzlu7R5egfqE8qM44QNdZoRk2k6OLRgGRgqo+CoApxB4WV6IAcq3ryKwt2vzTOap8+x YtlQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=sWRntq7x9k3MAHX6XyhqaSg3cYHmaN4xC2+w1hNZ+fo=; b=dygyyVFmlfsNqfkgNq1QnYTNSXrXHMAg2V92ZPdk7rtGELR1kF9aPjqai9SRBaN7jM fdHU+ffxGhgIk4VyumbSSizYnphJYKhytNbd/ufbvpmO4tN2GFaqAT0daJr6MRs8tZjo GSgJBmn+OzP/pbAdmsWTlqojUus4/fvDox18DSVXKryjJ/oL9BaR7jMcrgK0AQ666Adw 10WZvdFqJ2Un9SHVshOXak+yTeoADBK1HkefULpApVE9gmyTguLfYk8SyAvKEj9XIVXy 9jtr0HN4LdYIzKJiydBns08AsFQKM6+4Tg2r7uTMu4VJjPhueI+71OvV/ZCZe/jOiCBG WDIQ== X-Gm-Message-State: AJcUukehPiwysDrb2nwPUjzKVLFmjdcnOLPRrAjlLHUp9Wb8hGAEuNFD GAhLC1I0SoI9OYzWzpQkl7rMbD/t X-Google-Smtp-Source: ALg8bN5dIILwSeoT9C1REWEtOevAoZw+D781l2XGeGgyFXBLNGOup6htX0ELTXrBbCN1XY4OcxfU9w== X-Received: by 2002:a50:b3c9:: with SMTP id t9mr7598093edd.270.1547645780667; Wed, 16 Jan 2019 05:36:20 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id v14sm5260082edq.74.2019.01.16.05.36.20 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 16 Jan 2019 05:36:20 -0800 (PST) Date: Wed, 16 Jan 2019 05:36:20 -0800 (PST) X-Google-Original-Date: Wed, 16 Jan 2019 13:35:58 GMT Message-Id: <58bb8d0469a4a9f7bd422b7ca60df4ab70ef5a29.1547645770.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Johannes Schindelin via GitGitGadget" Subject: [PATCH v3 10/21] ci: move the Windows job to the top Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Johannes Schindelin Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Johannes Schindelin The Windows job currently takes a whopping ~1h20m to complete. Which is *far* longer than the next-longest job takes (linux-gcc, ~35m). As such, it makes sense to start the Windows job first, to minimize the overall run time (which is now pretty safely the run time of the Windows job). This affects only the Azure Pipelines configuration, not the Travis one, of course, as Travis cannot run our Windows job: 1h20m is distinctly longer than the 50 minute timeout of Travis' free tier. This commit is best viewed with `--color-moved`. Signed-off-by: Johannes Schindelin --- azure-pipelines.yml | 172 ++++++++++++++++++++++---------------------- 1 file changed, 86 insertions(+), 86 deletions(-) diff --git a/azure-pipelines.yml b/azure-pipelines.yml index f3cabb0dd0..e44d2733a4 100644 --- a/azure-pipelines.yml +++ b/azure-pipelines.yml @@ -3,6 +3,92 @@ resources: fetchDepth: 1 jobs: +- job: windows + displayName: Windows + condition: succeeded() + pool: Hosted + timeoutInMinutes: 240 + steps: + - powershell: | + if ("$GITFILESHAREPWD" -ne "" -and "$GITFILESHAREPWD" -ne "`$`(gitfileshare.pwd)") { + net use s: \\gitfileshare.file.core.windows.net\test-cache "$GITFILESHAREPWD" /user:AZURE\gitfileshare /persistent:no + cmd /c mklink /d "$(Build.SourcesDirectory)\test-cache" S:\ + } + displayName: 'Mount test-cache' + env: + GITFILESHAREPWD: $(gitfileshare.pwd) + - powershell: | + # Helper to check the error level of the latest command (exit with error when appropriate) + function c() { if (!$?) { exit(1) } } + + # Add build agent's MinGit to PATH + $env:PATH = $env:AGENT_HOMEDIRECTORY +"\externals\\git\cmd;" +$env:PATH + + # Helper to initialize (or update) a Git worktree + function init ($path, $url, $set_origin) { + if (Test-Path $path) { + cd $path; c + if (Test-Path .git) { + & git init; c + } else { + & git status + } + } else { + & git init $path; c + cd $path; c + } + & git config core.autocrlf false; c + & git config core.untrackedCache true; c + if (($set_origin -ne 0) -and !(git config remote.origin.url)) { + & git remote add origin $url; c + } + & git fetch --depth=1 $url master; c + & git reset --hard FETCH_HEAD; c + & git clean -df; c + } + + # Initialize Git for Windows' SDK + $sdk_path = "$(Build.SourcesDirectory)\git-sdk-64" + init "$sdk_path" "https://dev.azure.com/git-for-windows/git-sdk-64/_git/git-sdk-64" 0 + + # Let Git ignore the SDK and the test-cache + "/git-sdk-64/`n/test-cache/`n" | Out-File -NoNewLine -Encoding ascii -Append "$(Build.SourcesDirectory)\.git\info\exclude" + displayName: 'Initialize the Git for Windows SDK' + - powershell: | + & "git-sdk-64\git-cmd.exe" --command=usr\\bin\\bash.exe -lc @" + export MAKEFLAGS=-j10 + export DEVELOPER=1 + export NO_PERL=1 + export NO_SVN_TESTS=1 + export GIT_TEST_SKIP_REBASE_P=1 + + ci/run-build-and-tests.sh || { + ci/print-test-failures.sh + exit 1 + } + "@ + if (!$?) { exit(1) } + displayName: 'Build & Test' + env: + HOME: $(Build.SourcesDirectory) + MSYSTEM: MINGW64 + - powershell: | + if ("$GITFILESHAREPWD" -ne "" -and "$GITFILESHAREPWD" -ne "`$`(gitfileshare.pwd)") { + cmd /c rmdir "$(Build.SourcesDirectory)\test-cache" + } + displayName: 'Unmount test-cache' + condition: true + env: + GITFILESHAREPWD: $(gitfileshare.pwd) + - task: PublishTestResults@2 + displayName: 'Publish Test Results **/TEST-*.xml' + inputs: + mergeTestResults: true + testRunTitle: 'windows' + platform: Windows + publishRunAttachments: false + condition: succeededOrFailed() + - job: linux_clang displayName: linux-clang condition: succeeded() @@ -153,92 +239,6 @@ jobs: publishRunAttachments: false condition: succeededOrFailed() -- job: windows - displayName: Windows - condition: succeeded() - pool: Hosted - timeoutInMinutes: 240 - steps: - - powershell: | - if ("$GITFILESHAREPWD" -ne "" -and "$GITFILESHAREPWD" -ne "`$`(gitfileshare.pwd)") { - net use s: \\gitfileshare.file.core.windows.net\test-cache "$GITFILESHAREPWD" /user:AZURE\gitfileshare /persistent:no - cmd /c mklink /d "$(Build.SourcesDirectory)\test-cache" S:\ - } - displayName: 'Mount test-cache' - env: - GITFILESHAREPWD: $(gitfileshare.pwd) - - powershell: | - # Helper to check the error level of the latest command (exit with error when appropriate) - function c() { if (!$?) { exit(1) } } - - # Add build agent's MinGit to PATH - $env:PATH = $env:AGENT_HOMEDIRECTORY +"\externals\\git\cmd;" +$env:PATH - - # Helper to initialize (or update) a Git worktree - function init ($path, $url, $set_origin) { - if (Test-Path $path) { - cd $path; c - if (Test-Path .git) { - & git init; c - } else { - & git status - } - } else { - & git init $path; c - cd $path; c - } - & git config core.autocrlf false; c - & git config core.untrackedCache true; c - if (($set_origin -ne 0) -and !(git config remote.origin.url)) { - & git remote add origin $url; c - } - & git fetch --depth=1 $url master; c - & git reset --hard FETCH_HEAD; c - & git clean -df; c - } - - # Initialize Git for Windows' SDK - $sdk_path = "$(Build.SourcesDirectory)\git-sdk-64" - init "$sdk_path" "https://dev.azure.com/git-for-windows/git-sdk-64/_git/git-sdk-64" 0 - - # Let Git ignore the SDK and the test-cache - "/git-sdk-64/`n/test-cache/`n" | Out-File -NoNewLine -Encoding ascii -Append "$(Build.SourcesDirectory)\.git\info\exclude" - displayName: 'Initialize the Git for Windows SDK' - - powershell: | - & "git-sdk-64\git-cmd.exe" --command=usr\\bin\\bash.exe -lc @" - export MAKEFLAGS=-j10 - export DEVELOPER=1 - export NO_PERL=1 - export NO_SVN_TESTS=1 - export GIT_TEST_SKIP_REBASE_P=1 - - ci/run-build-and-tests.sh || { - ci/print-test-failures.sh - exit 1 - } - "@ - if (!$?) { exit(1) } - displayName: 'Build & Test' - env: - HOME: $(Build.SourcesDirectory) - MSYSTEM: MINGW64 - - powershell: | - if ("$GITFILESHAREPWD" -ne "" -and "$GITFILESHAREPWD" -ne "`$`(gitfileshare.pwd)") { - cmd /c rmdir "$(Build.SourcesDirectory)\test-cache" - } - displayName: 'Unmount test-cache' - condition: true - env: - GITFILESHAREPWD: $(gitfileshare.pwd) - - task: PublishTestResults@2 - displayName: 'Publish Test Results **/TEST-*.xml' - inputs: - mergeTestResults: true - testRunTitle: 'windows' - platform: Windows - publishRunAttachments: false - condition: succeededOrFailed() - - job: linux32 displayName: Linux32 condition: succeeded() From patchwork Wed Jan 16 13:36:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Passaro via GitGitGadget X-Patchwork-Id: 10765949 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 107FC13BF for ; Wed, 16 Jan 2019 13:36:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F26E52D172 for ; Wed, 16 Jan 2019 13:36:28 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E706D2DE14; Wed, 16 Jan 2019 13:36:28 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 687662D172 for ; Wed, 16 Jan 2019 13:36:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390502AbfAPNg0 (ORCPT ); Wed, 16 Jan 2019 08:36:26 -0500 Received: from mail-ed1-f51.google.com ([209.85.208.51]:44299 "EHLO mail-ed1-f51.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390514AbfAPNgY (ORCPT ); Wed, 16 Jan 2019 08:36:24 -0500 Received: by mail-ed1-f51.google.com with SMTP id y56so5391094edd.11 for ; Wed, 16 Jan 2019 05:36:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=sI0J8sK8qLVY5A7TwFi50tscdkZxAfA3T+dfrtp7SBQ=; b=SZ6V8qxATvZfGBGdzm7yzt0+TowNpjy1GVD1pNk+TSfFS56J1fAGCylYlCDNX7hLTV nR0zRsapdgd3cdm1QgSO9mnqIrAdovPhDINb29CQd0chrmGEaPQAPAOEUWpPqCMJrAwT 3CpzbiwLvV66bvImQiEswHwgeuTbQ+veZSywRdnfwAfZ64KY1Vy2pQ6xckkMtAkGOc8e Ti/0/OKEFWgHQx5HhClrGnQH66hIk5C09ttMbJw8UaiYeKJZ46zy3rd5SeBJS+Ngw/8r jSsY48gBfZ9eSFwk3z9cTPFdhOI41PUlnrZtmkmKYcNC/8ECuuXiF/fgBbKNKvjZ7Hda VbLQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=sI0J8sK8qLVY5A7TwFi50tscdkZxAfA3T+dfrtp7SBQ=; b=OEl5Syedk/695huyVszqql7gzdnYW8TEuBLvTasZS+ipVCtgH02UUMmaLd523RgW3/ jeT17ipBLqYadaq1U895i/F/KsvkAsJfWb6tV3aPgJ/O37PC8UHWRk3FjHVwi0A7KiMB OcYV2WLMlK1/vMKlf1rH5R1noH5UBPZreorDEKuU1vCsNU6w4Zbs41fOqNaK9hk5l+6a HvKdOqejuitKG1zFlknxuU70hVVBJH3clDWaMV5pdEotRcZNLVAZOZunO0V/Qpcggnb9 D+TX2ggLH0lKRUCtYER7zAZOTjzqcUW3ntbh6j5RPJIWa99oo5P0JS6AGhGynrIFqRWW RhbA== X-Gm-Message-State: AJcUukd3+TV4rGXrN5igNJVoi09fpZH9gUECfq68BrRA6RU5t3iNcukw yuo4CtSg64GaVzz5sd/YAf99UN+0 X-Google-Smtp-Source: ALg8bN7iGlOvYodnvcJaxq5qgoJFQ2T4DCMBw81sJS6b+L5lLoQGGylcns5Oo0AEcbJicYQnGEHR1Q== X-Received: by 2002:a17:906:1d01:: with SMTP id n1-v6mr6878643ejh.61.1547645781562; Wed, 16 Jan 2019 05:36:21 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id n23-v6sm2980934ejx.57.2019.01.16.05.36.20 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 16 Jan 2019 05:36:20 -0800 (PST) Date: Wed, 16 Jan 2019 05:36:20 -0800 (PST) X-Google-Original-Date: Wed, 16 Jan 2019 13:35:59 GMT Message-Id: In-Reply-To: References: From: "Johannes Schindelin via GitGitGadget" Subject: [PATCH v3 11/21] ci: use git-sdk-64-minimal build artifact Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Johannes Schindelin Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Johannes Schindelin Instead of a shallow fetch followed by a sparse checkout, we are better off by using a separate, dedicated Pipeline that bundles the SDK as a build artifact, and then consuming that build artifact here. In fact, since this artifact will be used a lot, we spent substantial time on figuring out a minimal subset of the Git for Windows SDK, just enough to build and test Git. The result is a size reduction from around 1GB (compressed) to around 55MB (compressed). This also comes with the change where we now call `usr\bin\bash.exe` directly, as `git-cmd.exe` is not included in the minimal SDK. That reduces the time to initialize Git for Windows' SDK from anywhere between 2m30s-7m to a little over 1m. Note: in theory, we could also use the DownloadBuildArtifacts@0 task here. However, restricted permissions that are in effect when building from forks would let this fail for PR builds, defeating the whole purpose of the Azure Pipelines support for git.git. Signed-off-by: Johannes Schindelin --- azure-pipelines.yml | 44 +++++++++----------------------------------- 1 file changed, 9 insertions(+), 35 deletions(-) diff --git a/azure-pipelines.yml b/azure-pipelines.yml index e44d2733a4..3085ff6ace 100644 --- a/azure-pipelines.yml +++ b/azure-pipelines.yml @@ -18,44 +18,18 @@ jobs: env: GITFILESHAREPWD: $(gitfileshare.pwd) - powershell: | - # Helper to check the error level of the latest command (exit with error when appropriate) - function c() { if (!$?) { exit(1) } } - - # Add build agent's MinGit to PATH - $env:PATH = $env:AGENT_HOMEDIRECTORY +"\externals\\git\cmd;" +$env:PATH - - # Helper to initialize (or update) a Git worktree - function init ($path, $url, $set_origin) { - if (Test-Path $path) { - cd $path; c - if (Test-Path .git) { - & git init; c - } else { - & git status - } - } else { - & git init $path; c - cd $path; c - } - & git config core.autocrlf false; c - & git config core.untrackedCache true; c - if (($set_origin -ne 0) -and !(git config remote.origin.url)) { - & git remote add origin $url; c - } - & git fetch --depth=1 $url master; c - & git reset --hard FETCH_HEAD; c - & git clean -df; c - } - - # Initialize Git for Windows' SDK - $sdk_path = "$(Build.SourcesDirectory)\git-sdk-64" - init "$sdk_path" "https://dev.azure.com/git-for-windows/git-sdk-64/_git/git-sdk-64" 0 + $urlbase = "https://dev.azure.com/git-for-windows/git/_apis/build/builds" + $id = ((Invoke-WebRequest -UseBasicParsing "${urlbase}?definitions=22&statusFilter=completed&resultFilter=succeeded&`$top=1").content | ConvertFrom-JSON).value[0].id + $downloadUrl = ((Invoke-WebRequest -UseBasicParsing "${urlbase}/$id/artifacts").content | ConvertFrom-JSON).value[1].resource.downloadUrl + (New-Object Net.WebClient).DownloadFile($downloadUrl,"git-sdk-64-minimal.zip") + Expand-Archive git-sdk-64-minimal.zip -DestinationPath . -Force + Remove-Item git-sdk-64-minimal.zip # Let Git ignore the SDK and the test-cache - "/git-sdk-64/`n/test-cache/`n" | Out-File -NoNewLine -Encoding ascii -Append "$(Build.SourcesDirectory)\.git\info\exclude" - displayName: 'Initialize the Git for Windows SDK' + "/git-sdk-64-minimal/`n/test-cache/`n" | Out-File -NoNewLine -Encoding ascii -Append "$(Build.SourcesDirectory)\.git\info\exclude" + displayName: 'Download git-sdk-64-minimal' - powershell: | - & "git-sdk-64\git-cmd.exe" --command=usr\\bin\\bash.exe -lc @" + & git-sdk-64-minimal\usr\bin\bash.exe -lc @" export MAKEFLAGS=-j10 export DEVELOPER=1 export NO_PERL=1 From patchwork Wed Jan 16 13:36:22 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Passaro via GitGitGadget X-Patchwork-Id: 10765953 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id ADDA291E for ; Wed, 16 Jan 2019 13:36:31 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9D87A2DE33 for ; Wed, 16 Jan 2019 13:36:31 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 91CE72DE6A; Wed, 16 Jan 2019 13:36:31 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 29C0B2DE4C for ; Wed, 16 Jan 2019 13:36:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390552AbfAPNga (ORCPT ); Wed, 16 Jan 2019 08:36:30 -0500 Received: from mail-ed1-f50.google.com ([209.85.208.50]:42237 "EHLO mail-ed1-f50.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2393221AbfAPNgY (ORCPT ); Wed, 16 Jan 2019 08:36:24 -0500 Received: by mail-ed1-f50.google.com with SMTP id y20so5397531edw.9 for ; Wed, 16 Jan 2019 05:36:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=Nx9l6UGI/Mo+cXJbFVbhoaNDqQugjJei4UPiehgiQbs=; b=Ee0piCGNHxJBy5Wb5GIvcKAGLQvLmHg70PUz0DAz10D8P6PCnzzmXC5J/c1y0TVmpK VUBcesPB6V3JDSir9lm2y2C0K6JJDXQvAMvVAxbmQnmTy601kLYHdIVJfKVpzWw+NO1d zTZx+3l9/YYhaKkjuDP/pC/69gRKThawnmZva0XLDf6SgvzNwpWn4nwhh0QTxtUpjJRk oSaud/mzYP5aIRH+Cn11kT8Kp1uLJ8FKqoljDgQb6MkmzibepH9KOixRhZS6XBnRnM9d NOHNUkafB8sFQ6iwq/vLyn89uyU663X8rgqwtS2RTVA8F5M4YeVApajgxTs2EjtTKKvJ mB0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=Nx9l6UGI/Mo+cXJbFVbhoaNDqQugjJei4UPiehgiQbs=; b=IxBv0Wtsr69Uj/A6u9zkyxabo37DJwVV0v9W+5odjPuCzSDx3kD1hPqvsIbIDuVhWj kTxLqlALbMUqk1AeWShxZJAFEVETqXb1xfkVsE3t7iOKokbJjOkuhn6owYNcvW/mmYu3 kX3N9H5mYHom72+9h5Nl5nOSjW/Lx1ynKesOXCSBYn2ybbB3rzWMrPm0Hbv3x0p+9sUT red2Fr3b/miTuVX4nDHX5sbLLaBlZFcoeTPtwj1PlwGni2t1SgFyP4bcsHtpUZBiTYZq jibqJJH2moP/5GhOV0DpzA/0fu26gAKVW5y8qjzHZDweLfh541BK0uNLfZ5puVP6GHME ukfQ== X-Gm-Message-State: AJcUukdH99cHNVJuI0lLoPigLVxhFBH8wLdLjovoTmcSvZjxuty2UIdd Wj4X20i8ve2aGgOan1FFVySHAYnX X-Google-Smtp-Source: ALg8bN5SCO1zsQLv7S6AeAVf9lsV93sEQeDQlrfQQar1AsbOBxocGQUl5vT8x74l5FsFDs/UwBIbhQ== X-Received: by 2002:a17:906:a3c8:: with SMTP id ca8-v6mr6901822ejb.88.1547645782642; Wed, 16 Jan 2019 05:36:22 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id l20-v6sm3106404eja.18.2019.01.16.05.36.21 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 16 Jan 2019 05:36:22 -0800 (PST) Date: Wed, 16 Jan 2019 05:36:22 -0800 (PST) X-Google-Original-Date: Wed, 16 Jan 2019 13:36:00 GMT Message-Id: <2b6ee78f021a47c5892d96b5d63d19bdedf25e2d.1547645770.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Johannes Schindelin via GitGitGadget" Subject: [PATCH v3 12/21] mingw: be more generous when wrapping up the setitimer() emulation Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Johannes Schindelin Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Johannes Schindelin Every once in a while, the Azure Pipeline fails with some semi-random error: timer thread did not terminate timely This error message means that the thread that is used to emulate the setitimer() function did not terminate within 1,000 milliseconds. The most likely explanation (and therefore the one we should assume to be true, according to Occam's Razor) is that the timeout of one second is simply not enough because we try to run so many tasks in parallel. So let's give it ten seconds instead of only one. That should be enough. Signed-off-by: Johannes Schindelin --- compat/mingw.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/compat/mingw.c b/compat/mingw.c index b459e1a291..e0dfe8844d 100644 --- a/compat/mingw.c +++ b/compat/mingw.c @@ -2100,7 +2100,7 @@ static void stop_timer_thread(void) if (timer_event) SetEvent(timer_event); /* tell thread to terminate */ if (timer_thread) { - int rc = WaitForSingleObject(timer_thread, 1000); + int rc = WaitForSingleObject(timer_thread, 10000); if (rc == WAIT_TIMEOUT) error("timer thread did not terminate timely"); else if (rc != WAIT_OBJECT_0) From patchwork Wed Jan 16 13:36:23 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Passaro via GitGitGadget X-Patchwork-Id: 10765951 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CFAE113BF for ; Wed, 16 Jan 2019 13:36:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BFAAD2D172 for ; Wed, 16 Jan 2019 13:36:29 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B3C0D2DE14; Wed, 16 Jan 2019 13:36:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5BC6A2D172 for ; Wed, 16 Jan 2019 13:36:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2393230AbfAPNg1 (ORCPT ); Wed, 16 Jan 2019 08:36:27 -0500 Received: from mail-ed1-f44.google.com ([209.85.208.44]:34798 "EHLO mail-ed1-f44.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2393225AbfAPNgZ (ORCPT ); Wed, 16 Jan 2019 08:36:25 -0500 Received: by mail-ed1-f44.google.com with SMTP id b3so5465888ede.1 for ; Wed, 16 Jan 2019 05:36:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=yIFmvH39JP873kVuTr99FvXDnugqBCZqKax9pCZMHbE=; b=UhWs255EXZkjxkH4rgnOJcm9p55An8s5kc4nMHx2s5UFvuuCUbNBN+t9Kcc9poj8ze SpX4niUzbM68pOCw36Tl7qLq7llPcQkwfspoCSBDowxaTKsMLPCvgk9ILsNp1xrAn2lH JIk+IYDeTeODYZCRNSWRH7h+uOEWdTsXpjwhHGjUl0bVVnHVGmvUDhL6D+Hc8CpnFbOu 78qoI/HZuYYY4n7h7pjqfWheEDRpGdqfrvzNGA7ZD4j1gITFGpwsWwTeLBpsPheiAF2k wlOa0smfWhEAWlA2kSUS9VKY1S/9dSyMxMY01qfSQaUVZb4SZrU+gm57RKLe0IMeLjwK 1GqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=yIFmvH39JP873kVuTr99FvXDnugqBCZqKax9pCZMHbE=; b=Bl9yxXKLmV0NI2XQOl9y9GIMtMko+4WDSZe+BOtBUGwobQaskyCNWkHRoLlWNrvgJ3 1XMRPBPF1rwHuFcZbEKz67ExZ9925wShRRnF1arxz1SZ4jAVmJcJD0KCtH4Oy30ZkC/h YMkXXtIvkXmFSLjMAibL6bksj5BhJFaM/IR2OlZtIrUiE73HGYyzMUwyGBndUjk2hp7d 19NMR2QQ6v4qe1WwOPmDFKAZyL2qXNAuQrwseCcbG1TWoGmBded09xBlviYdpWbYSzUK 6pN2CiM61sEBtEpL4n7YQiU4f1qFoN/6FPSI+X2RT9dT+gTM9rUnS0LwoYPpNjte5Ome bwrA== X-Gm-Message-State: AJcUukfF6j+DJKblbREHrh4w9xw8yPBUORyaNWgbRgGqBUJDP3SFSLgm vWRNRj+ClvzhSh5TnsjmAc0/4y0b X-Google-Smtp-Source: ALg8bN5xrYhRo5T+8vlHZs8MsmqdmucpDv64UMXFU04/lQUJdiszR7M+k9QspZrJNtNpTNzBR0cHXg== X-Received: by 2002:a50:a974:: with SMTP id m49mr7740921edc.220.1547645783527; Wed, 16 Jan 2019 05:36:23 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id d27-v6sm3028218eja.20.2019.01.16.05.36.22 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 16 Jan 2019 05:36:23 -0800 (PST) Date: Wed, 16 Jan 2019 05:36:23 -0800 (PST) X-Google-Original-Date: Wed, 16 Jan 2019 13:36:01 GMT Message-Id: In-Reply-To: References: From: "Johannes Schindelin via GitGitGadget" Subject: [PATCH v3 13/21] README: add a build badge (status of the Azure Pipelines build) Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Johannes Schindelin Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Johannes Schindelin Just like so many other OSS projects, we now also have a build badge. Signed-off-by: Johannes Schindelin --- README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.md b/README.md index f920a42fad..764c480c66 100644 --- a/README.md +++ b/README.md @@ -1,3 +1,5 @@ +[![Build Status](https://dev.azure.com/git/git/_apis/build/status/test-git.git)](https://dev.azure.com/git/git/_build/latest?definitionId=2) + Git - fast, scalable, distributed revision control system ========================================================= From patchwork Wed Jan 16 13:36:23 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Passaro via GitGitGadget X-Patchwork-Id: 10765967 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 502C691E for ; Wed, 16 Jan 2019 13:36:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3FF652D172 for ; Wed, 16 Jan 2019 13:36:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 316DB2DE33; Wed, 16 Jan 2019 13:36:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AD5DE2D172 for ; Wed, 16 Jan 2019 13:36:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2393263AbfAPNgl (ORCPT ); Wed, 16 Jan 2019 08:36:41 -0500 Received: from mail-ed1-f47.google.com ([209.85.208.47]:37420 "EHLO mail-ed1-f47.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2393214AbfAPNg0 (ORCPT ); Wed, 16 Jan 2019 08:36:26 -0500 Received: by mail-ed1-f47.google.com with SMTP id h15so5437647edb.4 for ; Wed, 16 Jan 2019 05:36:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=qFU9SlpnDsxWdLk7t1RD4T2tVhhrbl/R3qfgIkC/7lI=; b=vdNFFLaQzUFGrzKBUY8Qmwne3VEtm9VJsu/PJDfsh/JUgXJ/x9uMSPh+AOA7489SFC L4m3gNYZhqkyBY4UGMEudNOOid2M9fDczWJ/q9/T3Tcm9b9eftt+NLqs3qB/TH9jonBi SLEjc5hKuSzjtQktB82THtEB/kMZ2AFWcxo0b8SZutqhJyrsmRyKLa1TAw6NZ2S+tuIg adimhZ7nVE7GJBVq6bsZJplPLQVFMdwZsKcFOkyTs7PRXckLSBhpALFQs1pbiEhgNzM2 dag2wdS9BgdoPVADJAwbGbERJjxfj42u2CM4ErGslBhND6oc3IhjIROxHtDSGDBL24bh OPsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=qFU9SlpnDsxWdLk7t1RD4T2tVhhrbl/R3qfgIkC/7lI=; b=lX3g9cZCSpO0cszfKUZCjt9ZhtoiaXYTseuwvIU6pbp7Hz2m1CNyvhjWkr6m3nLjxS eiT2G4ffSIskAIrUlBej7daMClUVq15vh/6bKE1nfJhGrUogsTMZsPxUCrIX/TQ0lJ5S 8zbpJUfqxGBqJyKwEDMgI13Y6OF3HYjW7sLI2cl2VZeEKWEROxKpb9znJbe0xF2QYFeI qyiOY5BQmZ85/gXYzHXekZ8zgMZCly6u65JPocNIJwcuDOhYI+U3wPmz7M05wv4fQR5y My140Y5SBXpB9Da1nrfiyGadfhH8+aXqDFhxG+P1ILeSeL2UR3PhhNRrFquZBnewb0y1 eTEw== X-Gm-Message-State: AJcUukfJWVT3WTpgk1mxaNVCMyH2lKCVAPAFwN9kwv/s02RCEeH52vKL a8ccMg8uuAQGRi4JbgW4+7Ij/ErT X-Google-Smtp-Source: ALg8bN6nT5xkmvE0ZyoOMlsgtzjA/yRaq+LdcLX7c7iXwSVxLcgJynvhIn0KRXf8g9sZfxXnmpvvGg== X-Received: by 2002:a17:906:7b97:: with SMTP id s23-v6mr6950656ejo.57.1547645784368; Wed, 16 Jan 2019 05:36:24 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id r7-v6sm3063523ejs.36.2019.01.16.05.36.23 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 16 Jan 2019 05:36:23 -0800 (PST) Date: Wed, 16 Jan 2019 05:36:23 -0800 (PST) X-Google-Original-Date: Wed, 16 Jan 2019 13:36:02 GMT Message-Id: <0d547db8f75fa38223712355b6f7152acd3254a4.1547645770.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Johannes Schindelin via GitGitGadget" Subject: [PATCH v3 14/21] tests: avoid calling Perl just to determine file sizes Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Johannes Schindelin Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Johannes Schindelin It is a bit ridiculous to spin up a full-blown Perl instance (especially on Windows, where that means spinning up a full POSIX emulation layer, AKA the MSYS2 runtime) just to tell how large a given file is. So let's just use the test-tool to do that job instead. This command will also be used over the next commits, to allow for cutting out individual test cases' verbose log from the file generated via --verbose-log. Signed-off-by: Johannes Schindelin --- t/helper/test-path-utils.c | 12 ++++++++++++ t/t0021-conversion.sh | 2 +- t/t1050-large.sh | 2 +- t/t5315-pack-objects-compression.sh | 2 +- t/t9303-fast-import-compression.sh | 2 +- 5 files changed, 16 insertions(+), 4 deletions(-) diff --git a/t/helper/test-path-utils.c b/t/helper/test-path-utils.c index ae091d9b3e..30211d6d64 100644 --- a/t/helper/test-path-utils.c +++ b/t/helper/test-path-utils.c @@ -291,6 +291,18 @@ int cmd__path_utils(int argc, const char **argv) return !!res; } + if (argc > 2 && !strcmp(argv[1], "file-size")) { + int res = 0, i; + struct stat st; + + for (i = 2; i < argc; i++) + if (stat(argv[i], &st)) + res = error_errno("Cannot stat '%s'", argv[i]); + else + printf("%"PRIuMAX"\n", (uintmax_t)st.st_size); + return !!res; + } + fprintf(stderr, "%s: unknown function name: %s\n", argv[0], argv[1] ? argv[1] : "(there was none)"); return 1; diff --git a/t/t0021-conversion.sh b/t/t0021-conversion.sh index fd5f1ac649..e10f5f787f 100755 --- a/t/t0021-conversion.sh +++ b/t/t0021-conversion.sh @@ -24,7 +24,7 @@ generate_random_characters () { } file_size () { - perl -e 'print -s $ARGV[0]' "$1" + test-tool path-utils file-size "$1" } filter_git () { diff --git a/t/t1050-large.sh b/t/t1050-large.sh index 1a9b21b293..dcb4dbba67 100755 --- a/t/t1050-large.sh +++ b/t/t1050-large.sh @@ -8,7 +8,7 @@ test_description='adding and checking out large blobs' # This should be moved to test-lib.sh together with the # copy in t0021 after both topics have graduated to 'master'. file_size () { - perl -e 'print -s $ARGV[0]' "$1" + test-tool path-utils file-size "$1" } test_expect_success setup ' diff --git a/t/t5315-pack-objects-compression.sh b/t/t5315-pack-objects-compression.sh index 34c47dae09..df970d7584 100755 --- a/t/t5315-pack-objects-compression.sh +++ b/t/t5315-pack-objects-compression.sh @@ -7,7 +7,7 @@ test_description='pack-object compression configuration' # This should be moved to test-lib.sh together with the # copy in t0021 after both topics have graduated to 'master'. file_size () { - perl -e 'print -s $ARGV[0]' "$1" + test-tool path-utils file-size "$1" } test_expect_success setup ' diff --git a/t/t9303-fast-import-compression.sh b/t/t9303-fast-import-compression.sh index 856219f46a..5045f02a53 100755 --- a/t/t9303-fast-import-compression.sh +++ b/t/t9303-fast-import-compression.sh @@ -6,7 +6,7 @@ test_description='compression setting of fast-import utility' # This should be moved to test-lib.sh together with the # copy in t0021 after both topics have graduated to 'master'. file_size () { - perl -e 'print -s $ARGV[0]' "$1" + test-tool path-utils file-size "$1" } import_large () { From patchwork Wed Jan 16 13:36:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Passaro via GitGitGadget X-Patchwork-Id: 10765969 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 07D9313BF for ; Wed, 16 Jan 2019 13:36:44 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EA68F2D172 for ; Wed, 16 Jan 2019 13:36:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DF1192DE2E; Wed, 16 Jan 2019 13:36:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 715472D172 for ; Wed, 16 Jan 2019 13:36:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2393244AbfAPNgm (ORCPT ); Wed, 16 Jan 2019 08:36:42 -0500 Received: from mail-ed1-f42.google.com ([209.85.208.42]:35321 "EHLO mail-ed1-f42.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2393227AbfAPNg1 (ORCPT ); Wed, 16 Jan 2019 08:36:27 -0500 Received: by mail-ed1-f42.google.com with SMTP id x30so5448724edx.2 for ; Wed, 16 Jan 2019 05:36:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=YWWVkd3zsvRM1u9QD3+kA6vezpKuExOYkyGwHgrrNPA=; b=sPUAz/FAqh+0HrBokgMoPnOs3mOSCKyHtZUxHG69Eoitto01E5AijzGApVsV/oQk8I OAtZVbTNXvToPAMhvOmL+GjKDzTvG76HXQ5+RVlqB+jZsKVL1ACJuOgf9wUsk9sFlFkd ra/SBIo14s0JeWicF8DltD3isQ+1GdRIWGz2e6oc1rq6Svy1dfm02Viv6isSKA9BTmkt meSQF3JX/2PwfzZiQKKd9DemG6Zj8+yqG1zrXgqdNa3MEcEJTuFDifgroztHBEi7oqlo K4dqfi+/DWdsLkBODRwqOcdTG1GsYc+/9U9uuSPVzy/kypjn776R1d/eITLhzQsIdyiD Eh/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=YWWVkd3zsvRM1u9QD3+kA6vezpKuExOYkyGwHgrrNPA=; b=SEk9pwFjUhTJIZSzdA1qOZrjtLGJ/g2kEcF2D17GsO5zamXA58TaAa4Qy34kWPj3vz rNziegPULv2b1/PS8VJHgecPdiSjRwj4XcwampHTVo9CA8+75tnBdZUzJDmSS8eZT20s W9d6f1wxtbIobPaxlrEavkZrvDoxS4XrFyj14VpxFriQpMKHfQoqwjjII+cd6FnVFxhU 0lSjk6EaBMAKhfL+6QzZ2qhDekmGyYyWtBBW1uFXYs6dAZg8I8kJwCJ6iGC84X2j7qqC Qt10VJvH+XxvZSa67zsiEuZqwoPtRNXoNeC17q7JmDSPdEoF/suTypVhyZ5xWR/snCgA e7Pw== X-Gm-Message-State: AJcUuke7clDyhNDGgTZmfRUTm8Zpj+rvNcod67SKf8Ymv7eRZkUFvEbU YrgF14qW+TNPbid90xMwFzL+NUek X-Google-Smtp-Source: ALg8bN7ebWyZRXQf4j6ZijiWjbnz2m8M6ig1/o9r5z1lQalY/ZYqVaKpj/IeqS3KEUY88AWagNA24Q== X-Received: by 2002:a17:906:9413:: with SMTP id q19-v6mr6800396ejx.208.1547645785214; Wed, 16 Jan 2019 05:36:25 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id k13-v6sm3031521ejk.50.2019.01.16.05.36.24 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 16 Jan 2019 05:36:24 -0800 (PST) Date: Wed, 16 Jan 2019 05:36:24 -0800 (PST) X-Google-Original-Date: Wed, 16 Jan 2019 13:36:03 GMT Message-Id: In-Reply-To: References: From: "Johannes Schindelin via GitGitGadget" Subject: [PATCH v3 15/21] tests: include detailed trace logs with --write-junit-xml upon failure Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Johannes Schindelin Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Johannes Schindelin The JUnit XML format lends itself to be presented in a powerful UI, where you can drill down to the information you are interested in very quickly. For test failures, this usually means that you want to see the detailed trace of the failing tests. With Travis CI, we passed the `--verbose-log` option to get those traces. However, that seems excessive, as we do not need/use the logs in almost all of those cases: only when a test fails do we have a way to include the trace. So let's do something different when using Azure DevOps: let's run all the tests with `--quiet` first, and only if a failure is encountered, try to trace the commands as they are executed. Of course, we cannot turn on `--verbose-log` after the fact. So let's just re-run the test with all the same options, adding `--verbose-log`. And then munging the output file into the JUnit XML on the fly. Note: there is an off chance that re-running the test in verbose mode "fixes" the failures (and this does happen from time to time!). That is a possibility we should be able to live with. Ideally, we would label this as "Passed upon rerun", and Azure Pipelines even know about that outcome, but it is not available when using the JUnit XML format for now: https://github.com/Microsoft/azure-pipelines-agent/blob/master/src/Agent.Worker/TestResults/JunitResultReader.cs Signed-off-by: Johannes Schindelin --- t/test-lib.sh | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/t/test-lib.sh b/t/test-lib.sh index e9782b6b32..f5371f505a 100644 --- a/t/test-lib.sh +++ b/t/test-lib.sh @@ -503,8 +503,18 @@ test_failure_ () { junit_insert="" junit_insert="$junit_insert $(xml_attr_encode \ - "$(printf '%s\n' "$@" | sed 1d)")" + "$(if test -n "$GIT_TEST_TEE_OUTPUT_FILE" + then + cut -c "$GIT_TEST_TEE_OFFSET-" <"$GIT_TEST_TEE_OUTPUT_FILE" + else + printf '%s\n' "$@" | sed 1d + fi)")" junit_insert="$junit_insert" + if test -n "$GIT_TEST_TEE_OUTPUT_FILE" + then + junit_insert="$junit_insert$(xml_attr_encode \ + "$(cat "$GIT_TEST_TEE_OUTPUT_FILE")")" + fi write_junit_xml_testcase "$1" " $junit_insert" fi test_failure=$(($test_failure + 1)) @@ -872,6 +882,11 @@ write_junit_xml_testcase () { write_junit_xml "$(printf '%s\n' \ " " "$@" " ")" junit_have_testcase=t + if test -n "$GIT_TEST_TEE_OUTPUT_FILE" + then + GIT_TEST_TEE_OFFSET=$(test-tool path-utils file-size \ + "$GIT_TEST_TEE_OUTPUT_FILE") + fi } test_done () { @@ -1153,6 +1168,11 @@ then date +%Y-%m-%dT%H:%M:%S)\"" write_junit_xml --truncate "" " " junit_suite_start=$(test-tool date getnanos) + if test -n "$GIT_TEST_TEE_OUTPUT_FILE" + then + GIT_TEST_TEE_OFFSET=0 + GIT_TEST_TEE_ERR_OFFSET=0 + fi fi # Provide an implementation of the 'yes' utility From patchwork Wed Jan 16 13:36:25 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Passaro via GitGitGadget X-Patchwork-Id: 10765971 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 625AF13BF for ; Wed, 16 Jan 2019 13:36:45 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 525562D172 for ; Wed, 16 Jan 2019 13:36:45 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 469472DE2E; Wed, 16 Jan 2019 13:36:45 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EB9552D172 for ; Wed, 16 Jan 2019 13:36:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2393261AbfAPNgl (ORCPT ); Wed, 16 Jan 2019 08:36:41 -0500 Received: from mail-ed1-f65.google.com ([209.85.208.65]:37926 "EHLO mail-ed1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2393229AbfAPNg2 (ORCPT ); Wed, 16 Jan 2019 08:36:28 -0500 Received: by mail-ed1-f65.google.com with SMTP id h50so5429305ede.5 for ; Wed, 16 Jan 2019 05:36:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=yoiCJ8uETqg+M/HRT251FaAYX3Ug22kI1sfozM0TKsQ=; b=mVLTR/2MmONbQcfc0+l1XFR4FPec7cAf1utAOwiD7qfPwmYkjMOrUrzauLsaFBXFyz 9a6cR2qrnfmGMUN08x4RmEnWaoGc1r7FicRJZNpwq/aXZCcGL7q5nhI1QaO9mCebfwbb qbu1NPd+jQjPJsJVc36N45kPGRyRAl2CR6zp23HFljmS9+KzivxRBDgXtpK5LLJvYd7N jvTO8gqUirGbo4KAspqsLITD+lbDIsUVN2wB0PCwVg6IAhVidk4soOfpt1V21CEp3bo2 ytVA0I61X3gSxvxFJoKurhUzZehGdvvQjaSSnAw1m6OHvTOsKz/lVDESYnAWLBmsnEBg dWzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=yoiCJ8uETqg+M/HRT251FaAYX3Ug22kI1sfozM0TKsQ=; b=gOhbsVE2QYXtLh4mebwx03Z+HE7lJfKrh59jtHULS6cHW8GO8X7W4yk4EYOb7dqVsr 2TR8KkFX/B7Q5KwgfQ4piNB1lXcdsEO3mSD6k5+njyJBUjI3QWyl55OmREW5tZWLOy7h sONCN0NgPJbvqCZSnYRZBGQzS9kpkL54k4o3ahOY9bzYovVgruPu2fDoFhML1TS8+MPo Xws92GznDaC12nKyUgvuF0mgbjdAyWkCgdw049G/Oxkhluko5Qt2VpOxHmIt+6tVOl6S Iav2OVx1OWfp7ZUlpnpYwgWgAr0pmrBOvNontuQ4FIaPPgUcAmlTZx6XXWyouRXVEeda D2IQ== X-Gm-Message-State: AJcUukcYg4zErankcbANLVMJ64bPut4903pXBOXWDV7YF4i+BqfmeYby Di6nN1BuLZ1r/+Ywo/gLSi+cfeSw X-Google-Smtp-Source: ALg8bN55z3/8jArMbrIKiAA/vtMWvXc1UATlBCF4GT8fv427YEyXB11eIcezUCeGBn8e6bgH/WqO1A== X-Received: by 2002:a50:cd5c:: with SMTP id d28mr7189540edj.176.1547645786088; Wed, 16 Jan 2019 05:36:26 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id n23-v6sm2980964ejx.57.2019.01.16.05.36.25 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 16 Jan 2019 05:36:25 -0800 (PST) Date: Wed, 16 Jan 2019 05:36:25 -0800 (PST) X-Google-Original-Date: Wed, 16 Jan 2019 13:36:04 GMT Message-Id: <8ef674a2364a624eeb0b6f279a088d4c4bc78d44.1547645770.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Johannes Schindelin via GitGitGadget" Subject: [PATCH v3 16/21] mingw: try to work around issues with the test cleanup Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Johannes Schindelin Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Johannes Schindelin It seems that every once in a while in the Git for Windows SDK, there are some transient file locking issues preventing the test clean up to delete the trash directory. Let's be gentle and try again five seconds later, and only error out if it still fails the second time. This change helps Windows, and does not hurt any other platform (normally, it is highly unlikely that said deletion fails, and if it does, normally it will fail again even 5 seconds later). Signed-off-by: Johannes Schindelin --- t/test-lib.sh | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/t/test-lib.sh b/t/test-lib.sh index f5371f505a..531170011e 100644 --- a/t/test-lib.sh +++ b/t/test-lib.sh @@ -970,7 +970,11 @@ test_done () { error "Tests passed but trash directory already removed before test cleanup; aborting" cd "$TRASH_DIRECTORY/.." && - rm -fr "$TRASH_DIRECTORY" || + rm -fr "$TRASH_DIRECTORY" || { + # try again in a bit + sleep 5; + rm -fr "$TRASH_DIRECTORY" + } || error "Tests passed but test cleanup failed; aborting" fi test_at_end_hook_ From patchwork Wed Jan 16 13:36:26 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Passaro via GitGitGadget X-Patchwork-Id: 10765959 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4C20791E for ; Wed, 16 Jan 2019 13:36:36 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 392682D172 for ; Wed, 16 Jan 2019 13:36:36 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2DACE2DE2E; Wed, 16 Jan 2019 13:36:36 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DA0182D172 for ; Wed, 16 Jan 2019 13:36:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2393239AbfAPNgd (ORCPT ); Wed, 16 Jan 2019 08:36:33 -0500 Received: from mail-ed1-f66.google.com ([209.85.208.66]:44064 "EHLO mail-ed1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2393231AbfAPNg3 (ORCPT ); Wed, 16 Jan 2019 08:36:29 -0500 Received: by mail-ed1-f66.google.com with SMTP id y56so5391319edd.11 for ; Wed, 16 Jan 2019 05:36:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=UIFTqxguWXt+P+iMJANnsfm86ufXvnIAqssuXHg5Cak=; b=cRhDx0+sF+kpbqR/gdIPdlzFi7GYMCf00LCjvZ8eoSv+Z1fpb2CDY+XhH7vdYfd3gV gy2PBqFpeB6GV5MvWhaqfAbe/vgmLViLfPDXNnBTOAJwDLCmx66D7tMNmbdIgFTGqa9j r3UnA1r9GsicOj2C+p5BLx96O4S+6j8DYdjrKbRuq+dLcUq+VRHluzmg90e5xcpCs4Cl wPD6/+aQaCQzk0Y3hHaEWF5c0XNs4umun/a3sxOnLtNUCh+XSxuoJ02+qr3432dvB08P Ek+i6TjqzI1v/gzo9IeduXRK0rrf9FeoV1Z1drfp7pe+Uy/QIt1JJau4rLU8yl4XEsM+ LICA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=UIFTqxguWXt+P+iMJANnsfm86ufXvnIAqssuXHg5Cak=; b=j+Vkf9v2yKCovFaK37akKqfYhnIzjLuJrNZCHSKH6BMdJCQ8lQa76UzgjiWjLYZbh5 p0kR/7uXeuHTCK4/Jhpv5RtpM4vNfUXnnxO2+gnbe42I/gHDPrHVnFTkW6YnL/Fn3eLG UtitS4mcEfSYtdzZQKuC76OvNN+UbNYjIWwieDFTQesl0xKksdr0p5X/EZ+FuBWLIwIF FzdjhRV31IaJaVBzOdf00qNN/dOrMMpMXR2rcb5O/Kp0FGZU1WlJy4mU4cUNvWqShA9B T1UTQTOwd7iXRvCV6xk2cckL/teS5BjTQRtSEPXJncsTsCs3qIBh09RSu9bYx1pkAOCt aySw== X-Gm-Message-State: AJcUukfnglyjPfoa60XsHyV55msvC/OJB6b+N4cxVemQvRSl7HMRlzKc +V3DJmR0c5CDlPJ6L4sBV80LErhc X-Google-Smtp-Source: ALg8bN6ZhZC2ReiqfhkZV58Wzhqhw+NiDp9Cz+1YI8cWYvI27wUijWAFrJvwAjYcOvL8AkM9U8gbMA== X-Received: by 2002:a17:906:23f1:: with SMTP id j17-v6mr6851561ejg.155.1547645786996; Wed, 16 Jan 2019 05:36:26 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id b46sm5583410edd.94.2019.01.16.05.36.26 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 16 Jan 2019 05:36:26 -0800 (PST) Date: Wed, 16 Jan 2019 05:36:26 -0800 (PST) X-Google-Original-Date: Wed, 16 Jan 2019 13:36:05 GMT Message-Id: In-Reply-To: References: From: "Johannes Schindelin via GitGitGadget" Subject: [PATCH v3 17/21] tests: add t/helper/ to the PATH with --with-dashes Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Johannes Schindelin Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Johannes Schindelin We really need to be able to find the test helpers... Really. This change was forgotten when we moved the test helpers into t/helper/ Signed-off-by: Johannes Schindelin --- t/test-lib.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/t/test-lib.sh b/t/test-lib.sh index 531170011e..9113ad86b4 100644 --- a/t/test-lib.sh +++ b/t/test-lib.sh @@ -1093,7 +1093,7 @@ else # normal case, use ../bin-wrappers only unless $with_dashes: GIT_EXEC_PATH=$GIT_BUILD_DIR if test -n "$with_dashes" then - PATH="$GIT_BUILD_DIR:$PATH" + PATH="$GIT_BUILD_DIR:$GIT_BUILD_DIR/t/helper:$PATH" fi fi GIT_TEMPLATE_DIR="$GIT_BUILD_DIR"/templates/blt From patchwork Wed Jan 16 13:36:27 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Passaro via GitGitGadget X-Patchwork-Id: 10765963 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3A4E613BF for ; Wed, 16 Jan 2019 13:36:40 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 290B32D172 for ; Wed, 16 Jan 2019 13:36:40 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1D86D2DE2E; Wed, 16 Jan 2019 13:36:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AD2CC2D172 for ; Wed, 16 Jan 2019 13:36:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2393236AbfAPNgc (ORCPT ); Wed, 16 Jan 2019 08:36:32 -0500 Received: from mail-ed1-f65.google.com ([209.85.208.65]:35927 "EHLO mail-ed1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2393232AbfAPNga (ORCPT ); Wed, 16 Jan 2019 08:36:30 -0500 Received: by mail-ed1-f65.google.com with SMTP id f23so5440170edb.3 for ; Wed, 16 Jan 2019 05:36:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=dxzqfZAxZ+TPwgPiHydUMW7hZp+IB66oCyS61ZHth2s=; b=IM0PGUEdnIwVLe+o4XXZ77beaNnETk3dyYsNXPDvX0zMXaP95yUgJzfFNg5vO+yA9o VhofXi7T8/mi8SjFOtB8ZxHOiBWn629ATpR0Y8e4hRR+HI7Ao1MqpJIyFs9hDw6X03cz k0RyCK3LBAz8bJggpqbgPbG00WMwyfZsq3Fluv+EOEPFpM2HP8pwDeUwTHE4MOmY1uzq bsbfVB5LVOJuO/y93RO73RTRFCqqZNQY2qi+V3a3wsT/nIlnhnS9D7BpP/VtOeHsEx7y UHejTE6wY68q4i8jfalw/pSEDQTbLdOWV9SyJeUSyrKBX0nVT2kKxMA08OyZPp1cHEG8 Fhng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=dxzqfZAxZ+TPwgPiHydUMW7hZp+IB66oCyS61ZHth2s=; b=Ayaz9MQMIPOcnH0bvzl61J9RbbNbBGUdVvTQGoaU/sDC1B+af0G12EBj/wV5VdJEfR Be3p7T1W55DL6InDHgPA1GUjeZ0qp2OE5ZwNAAFXrlw06jDMIysJARw/GraNXFaTlIov tG2O08I/k85vGRWjDn0q+/h/ppcpXXz4U2WeH+MPL940kZwbT1afitgCdFNyYLEdFFbE 2ehm5ctsF/7NTIRySX1eW7gGJk3F9XKsOEDWsGnr41EqiIPnm0nUpry9rXBHdA/lDhrA vQzuSn+ckNarAm+rJ74w2/kI3ve0Fw1z1385Nd4Q1ys6V6N1uje3em8iWyO8FWGqzsS+ A9CA== X-Gm-Message-State: AJcUuke0RoM7//SntxbOkAJtCrWYkdmWfVNmvjXAit3Sl7teRcgTFYPe VGzd8KrSqVuQC+i7QNU3hjwju6rf X-Google-Smtp-Source: ALg8bN4gjoq+U2XgSTNpYP+28F+USu8K7tEFRjgwKQmfcYED9YIccmT69aT2pc0tpdacorONfLfGkA== X-Received: by 2002:a17:906:304b:: with SMTP id d11-v6mr6702743ejd.4.1547645787771; Wed, 16 Jan 2019 05:36:27 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id q50sm5581240edd.66.2019.01.16.05.36.27 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 16 Jan 2019 05:36:27 -0800 (PST) Date: Wed, 16 Jan 2019 05:36:27 -0800 (PST) X-Google-Original-Date: Wed, 16 Jan 2019 13:36:06 GMT Message-Id: <713910e1dc634e3c09b374759a3bd282946eca96.1547645770.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Johannes Schindelin via GitGitGadget" Subject: [PATCH v3 18/21] t0061: fix with --with-dashes and RUNTIME_PREFIX Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Johannes Schindelin Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Johannes Schindelin When building Git with RUNTIME_PREFIX and starting a test helper from t/helper/, it fails to detect the system prefix correctly. This is the reason that the warning RUNTIME_PREFIX requested, but prefix computation failed. [...] to be printed. In t0061, we did not expect that to happen, and it actually did not happen in the normal case, because bin-wrappers/test-tool specifically sets GIT_TEXTDOMAINDIR (and as a consequence, nothing in test-tool wants to know about the runtime prefix). However, with --with-dashes, bin-wrappers/test-tool is no longer called, but t/helper/test-tool is called directly. So let's just ignore the RUNTIME_PREFIX warning. Signed-off-by: Johannes Schindelin --- t/t0061-run-command.sh | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/t/t0061-run-command.sh b/t/t0061-run-command.sh index 99a614bc7c..5a2d087bf0 100755 --- a/t/t0061-run-command.sh +++ b/t/t0061-run-command.sh @@ -166,7 +166,8 @@ test_trace () { expect="$1" shift GIT_TRACE=1 test-tool run-command "$@" run-command true 2>&1 >/dev/null | \ - sed -e 's/.* run_command: //' -e '/trace: .*/d' >actual && + sed -e 's/.* run_command: //' -e '/trace: .*/d' \ + -e '/RUNTIME_PREFIX requested/d' >actual && echo "$expect true" >expect && test_cmp expect actual } From patchwork Wed Jan 16 13:36:28 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Passaro via GitGitGadget X-Patchwork-Id: 10765955 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 083E913BF for ; Wed, 16 Jan 2019 13:36:35 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id ECD522D172 for ; Wed, 16 Jan 2019 13:36:34 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E1A762DE2E; Wed, 16 Jan 2019 13:36:34 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BADF12DD52 for ; Wed, 16 Jan 2019 13:36:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2393238AbfAPNgc (ORCPT ); Wed, 16 Jan 2019 08:36:32 -0500 Received: from mail-ed1-f66.google.com ([209.85.208.66]:39969 "EHLO mail-ed1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390473AbfAPNga (ORCPT ); Wed, 16 Jan 2019 08:36:30 -0500 Received: by mail-ed1-f66.google.com with SMTP id g22so5413083edr.7 for ; Wed, 16 Jan 2019 05:36:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=f78M31hStk/N5JvwPhMSjs9jTx9EW8TkkVA6LB1wmqY=; b=Dhf+aNg/9Mz1h/v+uX3NcvrDbo1byjaHZU/QB7lCtIixVyJc4LSCEe+nFAh/nYK9hs hCaH9JTwXxfZDxXEvCm6yjw6p8/9tnYAzEK0077OMBUyY4os+DF0LJUlFTRyQ+68RXz+ Qxgq8q/GNsFlMdTBnfZ8zGO6+AYdyIPQ7o3bssYHHlIlclI5FnYAmtHFv+poWDwYxId4 fre80rRZondtLDmea7KinNO8SRNwM9Qj0zBKBBgMsxNiZmOFtvVkYbmkZIfVOUlM7FtK g6LHT3Ig2QSTCVQHVNzDpdCO5t3+6NsGgW/PZkzTDcK5p+RQxDNx4gTFhfhZRtXUYxkL 8SwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=f78M31hStk/N5JvwPhMSjs9jTx9EW8TkkVA6LB1wmqY=; b=iuhgV0ioio2OeV7XjSRnRPWJfpT8EqdQRZ4nTQZWlpgRM3nv/T0gU4cKYYIceSgsfP fRElCVhU91ciobVB/E99jBXlIyh/PrUkphrY3s7Af5qqCP3R7sSjAQHXmE2VwAGFz9tm f85r/XmLTvikhNdeiSZwUz4H7YSts51rYtfCW0taqnELYbmUQk7BXFv7Ka876OnXKuO8 m/1Ju3UzhBqcuh8k8lN2Cpbxb4UR2z+0h8ebgNDNzgHDGOyfgnuchuedNMiQi9L+SCMy X+kFcuNOdHfcW+KMUyL5pS1v/iBh8jLv0N6fd22/pET5QYW6C/RJNFeJ3JZdpYgfyY3G KT1A== X-Gm-Message-State: AJcUukfiwhRatldLXMAZrWhffeaVdZZfxBSDBfRbNbpsb/mi9IS3K0Gc FCJuhsjRY0NToichk8PNg5KlSTma X-Google-Smtp-Source: ALg8bN5i/ueJVG+v1dMsrNXuliLG9fTmh4Zy4x6yJPze3DVZTs++kBV11+xt1KH81qWbDP1PeNaa1w== X-Received: by 2002:a50:999b:: with SMTP id m27mr7596370edb.10.1547645788684; Wed, 16 Jan 2019 05:36:28 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id f35sm5877344edd.80.2019.01.16.05.36.27 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 16 Jan 2019 05:36:28 -0800 (PST) Date: Wed, 16 Jan 2019 05:36:28 -0800 (PST) X-Google-Original-Date: Wed, 16 Jan 2019 13:36:07 GMT Message-Id: <3a77eafb44117092a97121afabac5106bf58fcdd.1547645770.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Johannes Schindelin via GitGitGadget" Subject: [PATCH v3 19/21] tests: optionally skip bin-wrappers/ Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Johannes Schindelin Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Johannes Schindelin This speeds up the tests by a bit on Windows, where running Unix shell scripts (and spawning processes) is not exactly a cheap operation. Signed-off-by: Johannes Schindelin --- t/README | 9 +++++++++ t/test-lib.sh | 19 +++++++++++++------ 2 files changed, 22 insertions(+), 6 deletions(-) diff --git a/t/README b/t/README index 28711cc508..3f645b083a 100644 --- a/t/README +++ b/t/README @@ -170,6 +170,15 @@ appropriately before running "make". implied by other options like --valgrind and GIT_TEST_INSTALLED. +--no-bin-wrappers:: + By default, the test suite uses the wrappers in + `../bin-wrappers/` to execute `git` and friends. With this option, + `../git` and friends are run directly. This is not recommended + in general, as the wrappers contain safeguards to ensure that no + files from an installed Git are used, but can speed up test runs + especially on platforms where running shell scripts is expensive + (most notably, Windows). + --root=:: Create "trash" directories used to store all temporary data during testing under , instead of the t/ directory. diff --git a/t/test-lib.sh b/t/test-lib.sh index 9113ad86b4..38a0f5d96f 100644 --- a/t/test-lib.sh +++ b/t/test-lib.sh @@ -294,6 +294,8 @@ do test -z "$HARNESS_ACTIVE" && quiet=t; shift ;; --with-dashes) with_dashes=t; shift ;; + --no-bin-wrappers) + no_bin_wrappers=t; shift ;; --no-color) color=; shift ;; --va|--val|--valg|--valgr|--valgri|--valgrin|--valgrind) @@ -1080,16 +1082,21 @@ then PATH=$GIT_TEST_INSTALLED:$GIT_BUILD_DIR/t/helper:$PATH GIT_EXEC_PATH=${GIT_TEST_EXEC_PATH:-$GIT_EXEC_PATH} else # normal case, use ../bin-wrappers only unless $with_dashes: - git_bin_dir="$GIT_BUILD_DIR/bin-wrappers" - if ! test -x "$git_bin_dir/git" + if test -n "$no_bin_wrappers" then - if test -z "$with_dashes" + with_dashes=t + else + git_bin_dir="$GIT_BUILD_DIR/bin-wrappers" + if ! test -x "$git_bin_dir/git" then - say "$git_bin_dir/git is not executable; using GIT_EXEC_PATH" + if test -z "$with_dashes" + then + say "$git_bin_dir/git is not executable; using GIT_EXEC_PATH" + fi + with_dashes=t fi - with_dashes=t + PATH="$git_bin_dir:$PATH" fi - PATH="$git_bin_dir:$PATH" GIT_EXEC_PATH=$GIT_BUILD_DIR if test -n "$with_dashes" then From patchwork Wed Jan 16 13:36:28 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Passaro via GitGitGadget X-Patchwork-Id: 10765957 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EC7AF13BF for ; Wed, 16 Jan 2019 13:36:35 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DB7722DD52 for ; Wed, 16 Jan 2019 13:36:35 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D04782DE2E; Wed, 16 Jan 2019 13:36:35 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 800A12D172 for ; Wed, 16 Jan 2019 13:36:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2393241AbfAPNgd (ORCPT ); Wed, 16 Jan 2019 08:36:33 -0500 Received: from mail-ed1-f66.google.com ([209.85.208.66]:35930 "EHLO mail-ed1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2393221AbfAPNgb (ORCPT ); Wed, 16 Jan 2019 08:36:31 -0500 Received: by mail-ed1-f66.google.com with SMTP id f23so5440263edb.3 for ; Wed, 16 Jan 2019 05:36:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=ziOnt1d5ptto+qoVg/U05MvTYTU5KnFXSFZ0TfHiJK0=; b=k6whV3eC3YFMSB/rN2BqM0/Pjr9NZGPn10Ey6G1oMx08e1qxR3CpTS45IQhnBvaVc1 AZlQiwjg/wPsZR5w60l2cB42Fljryum/JHq4W+aZyDMlppLAafLmnMgjHBb4XN35hnli BMhmXykGzneSmm2yY+nqkiEsjvRxnDHuMcMxcr7XrZ9TKJz8fY7Ie0tR43iafsCCqbqY RYV7j0PxUakducuWa4bRPD39XvPmrTcvU9Yyd93FztivFI9Jz6fFwC4AoYZWHts8TlZD /o0X+2s8C0UAhLZCnZ7fjfjS/Tj1ksNjrZ244QSwEkYI9GOnWy0t+TZ7dH7aQRfN2Y1c NKzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=ziOnt1d5ptto+qoVg/U05MvTYTU5KnFXSFZ0TfHiJK0=; b=YMhrbKEgQLkdeGEr883EZ4ReNPn2/yW1tTAS7iGwS4UESzCN7RheemwUXBu4KlP7hX jaTx3/Hm7f5oFD6VmvWYYpChXvIEPJ1aC2vckw+tM60Y+rI+GlYcQCxtl/APB0clvTVN 0WIcWLWesXe21L5dldoZ/+lxnvzZu4aunvyGyyBeql2mqBURvGB1hQSF7By42GvhdrM0 9S+2SExgFt1zVpwZETWAZVH6ATizwwXVYWnVpHvwf9aKCozk1HK3D+EymPTsvlF4YkNg ElobYatDKfMr31yGQf1JqOaOzVn7q+bzdPPKuWu8RsYPQoCsFXnVVcYlhxUOPtfSZqtI 8KCQ== X-Gm-Message-State: AJcUukfAJyu32egHnP8U1rYqiDOg1TfkANVcJQiBgT2gKyqzuUwqNlL2 dSeHL73dNrBFChD3ZbkPDEFzBiu9 X-Google-Smtp-Source: ALg8bN7g/3zK231dA0gW7exUFU053y2To+D9qgMRNP0LEYfOKC3K/dQ3757BT7T6sSbUiVOXYdpdyQ== X-Received: by 2002:a17:906:2452:: with SMTP id a18-v6mr6926069ejb.17.1547645789779; Wed, 16 Jan 2019 05:36:29 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id l20-v6sm3106434eja.18.2019.01.16.05.36.28 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 16 Jan 2019 05:36:28 -0800 (PST) Date: Wed, 16 Jan 2019 05:36:28 -0800 (PST) X-Google-Original-Date: Wed, 16 Jan 2019 13:36:08 GMT Message-Id: In-Reply-To: References: From: "Johannes Schindelin via GitGitGadget" Subject: [PATCH v3 20/21] ci: speed up Windows phase Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Johannes Schindelin Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Johannes Schindelin As Unix shell scripting comes at a hefty price on Windows, we have to see where we can save some time to run the test suite. Let's skip the chain linting and the bin-wrappers/ redirection on Windows; this seems to shave of anywhere between 10-30% from the overall runtime. Signed-off-by: Johannes Schindelin --- ci/lib.sh | 2 ++ 1 file changed, 2 insertions(+) diff --git a/ci/lib.sh b/ci/lib.sh index c5228c1547..bf5f1c9c0f 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -43,6 +43,8 @@ then BREW_INSTALL_PACKAGES= export GIT_PROVE_OPTS="--timer --jobs 10 --state=failed,slow,save" export GIT_TEST_OPTS="--verbose-log -x --write-junit-xml" + test windows_nt != "$CI_OS_NAME" || + GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS" fi skip_branch_tip_with_tag () { From patchwork Wed Jan 16 13:36:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Passaro via GitGitGadget X-Patchwork-Id: 10765961 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2439F13BF for ; Wed, 16 Jan 2019 13:36:38 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 10A4C2D172 for ; Wed, 16 Jan 2019 13:36:38 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0421D2DE2E; Wed, 16 Jan 2019 13:36:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 372DB2D172 for ; Wed, 16 Jan 2019 13:36:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2393243AbfAPNgf (ORCPT ); Wed, 16 Jan 2019 08:36:35 -0500 Received: from mail-ed1-f68.google.com ([209.85.208.68]:39406 "EHLO mail-ed1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2393234AbfAPNge (ORCPT ); Wed, 16 Jan 2019 08:36:34 -0500 Received: by mail-ed1-f68.google.com with SMTP id b14so5419525edt.6 for ; Wed, 16 Jan 2019 05:36:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:message-id:in-reply-to:references:from:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=FHfxuYhCfxnIhaTnsh5Jsfqur4kPq5sAb4sj7zyrQk8=; b=XHXwohN/X9R35WuZsDwHzwz69qrxuw7DFg+0ab4SnZ021sdsXCXbbNeRRfSPpbTkUu DetVQ/qwT2lCLLNo6202Qf54h0Sv0bOWnENdqG502mVWOKDig1cUwKx3U878CFSl8pxg eiqllgQ1OylhShJf8lPQoWWJNQKFDcMDwtZ8Ki2jnqSWe/0hKqZWlIdFpRwEtnauwgw5 Oycj2+FaYH/OA/KqBns4XE7B8joVkOP0+MMPi4e1JA8eqhLCq5ggJzmZEij5ax6NvddR uGbQzXYemVhn8r/MK7QmBRAvyalxAdQBC5Htn0ThkN23pDgtwN9qLmNNJj+W/UHMkPRH O6gQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:in-reply-to:references:from :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=FHfxuYhCfxnIhaTnsh5Jsfqur4kPq5sAb4sj7zyrQk8=; b=d3Je9NfpOXdRRqQGT81Wz3Rsm14qETKydasTQDhCaKIKBZ3ytHEwd/V1mCmnHrh1t+ Tp/CjBf+K5BbbkRXHLe2amQ2nGmgcI1wEZhvld3Su90AIUUHyVwO92/1N+3KYf+nAIlv KgSDxoHPm8BjpfK1ojt6rF909Q/YKTlTtYQOX5hyNBmC020rO6MJEByzFApDBfcALfmS 9ApmClPDPidPEKa7+QtKbSl/rie9ZDYb0mfExZSLIlCTa9B9WFZJH4S7zhVUdE41JMse udo1VNMIauiW3GchoEHSmofxmyUkloHhAstb0YyGdr3ddItHLrxV+yPeBvpZLzuPg8y6 ZzJg== X-Gm-Message-State: AJcUukdym7iv0t25feYKwnbcPm6xtpSfCkfeC/TuhcVIdffFasYu/ytl b/Et/YqP53HsRD78TU0Ma92MLo/+ X-Google-Smtp-Source: ALg8bN5SuEfl8FZ+FmJRP65oLQlZPyDcysFIrJXqBwUrZFml03vQ858TVRbYuv/5hCkuADKfSYBZkg== X-Received: by 2002:a05:6402:121a:: with SMTP id c26mr7327580edw.104.1547645790812; Wed, 16 Jan 2019 05:36:30 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id g2sm5635024edh.15.2019.01.16.05.36.29 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 16 Jan 2019 05:36:30 -0800 (PST) Date: Wed, 16 Jan 2019 05:36:30 -0800 (PST) X-Google-Original-Date: Wed, 16 Jan 2019 13:36:09 GMT Message-Id: In-Reply-To: References: From: "Johannes Schindelin via GitGitGadget" Subject: [PATCH v3 21/21] ci: parallelize testing on Windows Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Johannes Schindelin Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Johannes Schindelin The fact that Git's test suite is implemented in Unix shell script that is as portable as we can muster, combined with the fact that Unix shell scripting is foreign to Windows (and therefore has to be emulated), results in pretty abysmal speed of the test suite on that platform, for pretty much no other reason than that language choice. For comparison: while the Linux build & test is typically done within about 8 minutes, the Windows build & test typically lasts about 80 minutes in Azure Pipelines. To help with that, let's use the Azure Pipeline feature where you can parallelize jobs, make jobs depend on each other, and pass artifacts between them. The tests are distributed using the following heuristic: listing all test scripts ordered by size in descending order (as a cheap way to estimate the overall run time), every Nth script is run (where N is the total number of parallel jobs), starting at the index corresponding to the parallel job. This slicing is performed by a new function that is added to the `test-tool`. To optimize the overall runtime of the entire Pipeline, we need to move the Windows jobs to the beginning (otherwise there would be a very decent chance for the Pipeline to be run only the Windows build, while all the parallel Windows test jobs wait for this single one). We use Azure Pipelines Artifacts for both the minimal Git for Windows SDK as well as the built executables, as deduplication and caching close to the agents makes that really fast. For comparison: while downloading and unpacking the minimal Git for Windows SDK via PowerShell takes only one minute (down from anywhere between 2.5 to 7 when using a shallow clone), uploading it as Pipeline Artifact takes less than 30s and downloading and unpacking less than 20s (sometimes even as little as only twelve seconds). Signed-off-by: Johannes Schindelin --- Makefile | 10 +++++ azure-pipelines.yml | 82 +++++++++++++++++++++++++++++++++----- ci/make-test-artifacts.sh | 12 ++++++ ci/run-test-slice.sh | 17 ++++++++ t/helper/test-path-utils.c | 31 ++++++++++++++ 5 files changed, 143 insertions(+), 9 deletions(-) create mode 100755 ci/make-test-artifacts.sh create mode 100755 ci/run-test-slice.sh diff --git a/Makefile b/Makefile index 044b4f77bd..daa318fe17 100644 --- a/Makefile +++ b/Makefile @@ -2927,6 +2927,16 @@ rpm:: @false .PHONY: rpm +artifacts-tar:: $(ALL_PROGRAMS) $(SCRIPT_LIB) $(BUILT_INS) $(OTHER_PROGRAMS) \ + GIT-BUILD-OPTIONS $(TEST_PROGRAMS) $(test_bindir_programs) \ + $(NO_INSTALL) $(MOFILES) + $(QUIET_SUBDIR0)templates $(QUIET_SUBDIR1) \ + SHELL_PATH='$(SHELL_PATH_SQ)' PERL_PATH='$(PERL_PATH_SQ)' + test -n "$(ARTIFACTS_DIRECTORY)" + mkdir -p "$(ARTIFACTS_DIRECTORY)" + $(TAR) czf "$(ARTIFACTS_DIRECTORY)/artifacts.tar.gz" $^ templates/blt/ +.PHONY: artifacts-tar + htmldocs = git-htmldocs-$(GIT_VERSION) manpages = git-manpages-$(GIT_VERSION) .PHONY: dist-doc distclean diff --git a/azure-pipelines.yml b/azure-pipelines.yml index 3085ff6ace..6cd27b3483 100644 --- a/azure-pipelines.yml +++ b/azure-pipelines.yml @@ -3,8 +3,8 @@ resources: fetchDepth: 1 jobs: -- job: windows - displayName: Windows +- job: windows_build + displayName: Windows Build condition: succeeded() pool: Hosted timeoutInMinutes: 240 @@ -30,22 +30,86 @@ jobs: displayName: 'Download git-sdk-64-minimal' - powershell: | & git-sdk-64-minimal\usr\bin\bash.exe -lc @" - export MAKEFLAGS=-j10 - export DEVELOPER=1 - export NO_PERL=1 - export NO_SVN_TESTS=1 - export GIT_TEST_SKIP_REBASE_P=1 + ci/make-test-artifacts.sh artifacts + "@ + if (!$?) { exit(1) } + displayName: Build + env: + HOME: $(Build.SourcesDirectory) + MSYSTEM: MINGW64 + MAKEFLAGS: -j10 + DEVELOPER: 1 + NO_PERL: 1 + - task: PublishPipelineArtifact@0 + displayName: 'Publish Pipeline Artifact: test artifacts' + inputs: + artifactName: 'windows-artifacts' + targetPath: '$(Build.SourcesDirectory)\artifacts' + - task: PublishPipelineArtifact@0 + displayName: 'Publish Pipeline Artifact: git-sdk-64-minimal' + inputs: + artifactName: 'git-sdk-64-minimal' + targetPath: '$(Build.SourcesDirectory)\git-sdk-64-minimal' + - powershell: | + if ("$GITFILESHAREPWD" -ne "" -and "$GITFILESHAREPWD" -ne "`$`(gitfileshare.pwd)") { + cmd /c rmdir "$(Build.SourcesDirectory)\test-cache" + } + displayName: 'Unmount test-cache' + condition: true + env: + GITFILESHAREPWD: $(gitfileshare.pwd) + +- job: windows_test + displayName: Windows Test + dependsOn: windows_build + condition: succeeded() + pool: Hosted + timeoutInMinutes: 240 + strategy: + parallel: 10 + steps: + - powershell: | + if ("$GITFILESHAREPWD" -ne "" -and "$GITFILESHAREPWD" -ne "`$`(gitfileshare.pwd)") { + net use s: \\gitfileshare.file.core.windows.net\test-cache "$GITFILESHAREPWD" /user:AZURE\gitfileshare /persistent:no + cmd /c mklink /d "$(Build.SourcesDirectory)\test-cache" S:\ + } + displayName: 'Mount test-cache' + env: + GITFILESHAREPWD: $(gitfileshare.pwd) + - task: DownloadPipelineArtifact@0 + displayName: 'Download Pipeline Artifact: test artifacts' + inputs: + artifactName: 'windows-artifacts' + targetPath: '$(Build.SourcesDirectory)' + - task: DownloadPipelineArtifact@0 + displayName: 'Download Pipeline Artifact: git-sdk-64-minimal' + inputs: + artifactName: 'git-sdk-64-minimal' + targetPath: '$(Build.SourcesDirectory)\git-sdk-64-minimal' + - powershell: | + & git-sdk-64-minimal\usr\bin\bash.exe -lc @" + test -f artifacts.tar.gz || { + echo No test artifacts found\; skipping >&2 + exit 0 + } + tar xf artifacts.tar.gz || exit 1 + + # Let Git ignore the SDK and the test-cache + printf '%s\n' /git-sdk-64-minimal/ /test-cache/ >>.git/info/exclude - ci/run-build-and-tests.sh || { + ci/run-test-slice.sh `$SYSTEM_JOBPOSITIONINPHASE `$SYSTEM_TOTALJOBSINPHASE || { ci/print-test-failures.sh exit 1 } "@ if (!$?) { exit(1) } - displayName: 'Build & Test' + displayName: 'Test (parallel)' env: HOME: $(Build.SourcesDirectory) MSYSTEM: MINGW64 + MAKEFLAGS: -j10 + NO_SVN_TESTS: 1 + GIT_TEST_SKIP_REBASE_P: 1 - powershell: | if ("$GITFILESHAREPWD" -ne "" -and "$GITFILESHAREPWD" -ne "`$`(gitfileshare.pwd)") { cmd /c rmdir "$(Build.SourcesDirectory)\test-cache" diff --git a/ci/make-test-artifacts.sh b/ci/make-test-artifacts.sh new file mode 100755 index 0000000000..646967481f --- /dev/null +++ b/ci/make-test-artifacts.sh @@ -0,0 +1,12 @@ +#!/bin/sh +# +# Build Git and store artifacts for testing +# + +mkdir -p "$1" # in case ci/lib.sh decides to quit early + +. ${0%/*}/lib.sh + +make artifacts-tar ARTIFACTS_DIRECTORY="$1" + +check_unignored_build_artifacts diff --git a/ci/run-test-slice.sh b/ci/run-test-slice.sh new file mode 100755 index 0000000000..f8c2c3106a --- /dev/null +++ b/ci/run-test-slice.sh @@ -0,0 +1,17 @@ +#!/bin/sh +# +# Test Git in parallel +# + +. ${0%/*}/lib.sh + +case "$CI_OS_NAME" in +windows*) cmd //c mklink //j t\\.prove "$(cygpath -aw "$cache_dir/.prove")";; +*) ln -s "$cache_dir/.prove" t/.prove;; +esac + +make --quiet -C t T="$(cd t && + ./helper/test-tool path-utils slice-tests "$1" "$2" t[0-9]*.sh | + tr '\n' ' ')" + +check_unignored_build_artifacts diff --git a/t/helper/test-path-utils.c b/t/helper/test-path-utils.c index 30211d6d64..f4c5775090 100644 --- a/t/helper/test-path-utils.c +++ b/t/helper/test-path-utils.c @@ -177,6 +177,14 @@ static int is_dotgitmodules(const char *path) return is_hfs_dotgitmodules(path) || is_ntfs_dotgitmodules(path); } +static int cmp_by_st_size(const void *a, const void *b) +{ + intptr_t x = (intptr_t)((struct string_list_item *)a)->util; + intptr_t y = (intptr_t)((struct string_list_item *)b)->util; + + return x > y ? -1 : (x < y ? +1 : 0); +} + int cmd__path_utils(int argc, const char **argv) { if (argc == 3 && !strcmp(argv[1], "normalize_path_copy")) { @@ -303,6 +311,29 @@ int cmd__path_utils(int argc, const char **argv) return !!res; } + if (argc > 5 && !strcmp(argv[1], "slice-tests")) { + int res = 0; + long offset, stride, i; + struct string_list list = STRING_LIST_INIT_NODUP; + struct stat st; + + offset = strtol(argv[2], NULL, 10); + stride = strtol(argv[3], NULL, 10); + if (stride < 1) + stride = 1; + for (i = 4; i < argc; i++) + if (stat(argv[i], &st)) + res = error_errno("Cannot stat '%s'", argv[i]); + else + string_list_append(&list, argv[i])->util = + (void *)(intptr_t)st.st_size; + QSORT(list.items, list.nr, cmp_by_st_size); + for (i = offset; i < list.nr; i+= stride) + printf("%s\n", list.items[i].string); + + return !!res; + } + fprintf(stderr, "%s: unknown function name: %s\n", argv[0], argv[1] ? argv[1] : "(there was none)"); return 1;