From patchwork Thu Apr 21 18:22:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12822172 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 916FAC433F5 for ; Thu, 21 Apr 2022 18:23:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1391304AbiDUS0l (ORCPT ); Thu, 21 Apr 2022 14:26:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47070 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1391277AbiDUS0f (ORCPT ); Thu, 21 Apr 2022 14:26:35 -0400 Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com [IPv6:2a00:1450:4864:20::335]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F23DF4AE2C for ; Thu, 21 Apr 2022 11:23:36 -0700 (PDT) Received: by mail-wm1-x335.google.com with SMTP id l62-20020a1c2541000000b0038e4570af2fso3967446wml.5 for ; Thu, 21 Apr 2022 11:23:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=blemg6QF5VP0OGiyXKjlZppwmz1GulH6TiTTNO90amg=; b=o7EEBNvQlf8GUQ1JZEsSlVzXO5jbd7TGb5OF3ciHkf+KygRSagZ/d5Trv9oU2qKlw5 XLa2LRomWfLwWFfx3Flnai/rP/BtIwyIu/YvwN8Af89RkwiBvICNS5Jh6KPZx+uCPTK8 DLtWrAD+zin5VOrRQpv3+XKejZQHeSu/iuo20IoJKbhaA/y3B6X+OCOzvPvMllCenryG hZqChjgaxif++6LHCluLLiNZM2rDGz65WMqVJrNadEWxNpfmAUtNahwS6ltOq8+lqyLX XkyLM9+V3A+t6x9S11tZHq0AytUgLO0DS4LGMSuPbc+E8UbgMVQBbpo1S5GrjCATIFOJ Ykcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=blemg6QF5VP0OGiyXKjlZppwmz1GulH6TiTTNO90amg=; b=5sv9V+iqV5yx8uFwXtdASpDTtAJkz4eRcFP91BtplLY7HTdPrCTXxpwzQsHeti4ag4 AOP/mmNE/2chR18E5AIzzoen64W+7nue9huW4AYskTVsbd0Z4qUtkl18NosInCYnU1Vb XVeCXw01V+yGgTNxBAkTMJfCwcqGhHOVtgCHLCmaHnQgH36WVtkE1JAVUNnj6UXjvOPk 58myVC7q4mBfam1dk0+9RL1Hv78CogXMYRogXfnGnIoJ5YusAJfVoB9No8wArB/wt1aX RpLwJ/aMHNXyvGcAIufR/SnPQBp7tXz1hZ+TgN+iuCuq6mCKwJxOvQBK/tH9OWqP9C5M hv9Q== X-Gm-Message-State: AOAM533QxoKaTExRdYWdrWT4mRa38cusXg8ewaU3n677jTRLN4FVKa3a tV/ump7lrEuH1krvrztjlp8aUCk2MVmAdw== X-Google-Smtp-Source: ABdhPJxrv8xNvYJKmxFwCxV9X+3ml7zDC5Muq1hdOHydo2RTw3AGrctOVoir7xtS83djIAHUocm2Tg== X-Received: by 2002:a05:600c:1e15:b0:38f:f556:9349 with SMTP id ay21-20020a05600c1e1500b0038ff5569349mr565233wmb.3.1650565415359; Thu, 21 Apr 2022 11:23:35 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k11-20020a5d6d4b000000b0020599079f68sm2909090wri.106.2022.04.21.11.23.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Apr 2022 11:23:34 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , Eric Sunshine , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBC?= =?utf-8?b?amFybWFzb24=?= Subject: [PATCH v5 01/29] CI: run "set -ex" early in ci/lib.sh Date: Thu, 21 Apr 2022 20:22:56 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.879.g3659959fcca In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Change ci/lib.sh to run "set -ex" before it does anything else, before this we were omitting a few "export" commands in our trace output that happened before we enabled tracing. Signed-off-by: Ævar Arnfjörð Bjarmason --- ci/lib.sh | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/ci/lib.sh b/ci/lib.sh index cbc2f8f1caa..57141d38a85 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -1,5 +1,11 @@ # Library of functions shared by all CI scripts +# Set 'exit on error' for all CI scripts to let the caller know that +# something went wrong. +# Set tracing executed commands, primarily setting environment variables +# and installing dependencies. +set -ex + skip_branch_tip_with_tag () { # Sometimes, a branch is pushed at the same time the tag that points # at the same commit as the tip of the branch is pushed, and building @@ -85,12 +91,6 @@ export TERM=${TERM:-dumb} # Clear MAKEFLAGS that may come from the outside world. export MAKEFLAGS= -# Set 'exit on error' for all CI scripts to let the caller know that -# something went wrong. -# Set tracing executed commands, primarily setting environment variables -# and installing dependencies. -set -ex - if test -n "$SYSTEM_COLLECTIONURI" || test -n "$SYSTEM_TASKDEFINITIONSURI" then CI_TYPE=azure-pipelines From patchwork Thu Apr 21 18:22:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12822173 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0820DC433FE for ; Thu, 21 Apr 2022 18:23:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1391316AbiDUS0o (ORCPT ); Thu, 21 Apr 2022 14:26:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47076 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1391284AbiDUS0f (ORCPT ); Thu, 21 Apr 2022 14:26:35 -0400 Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com [IPv6:2a00:1450:4864:20::335]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 24F7B4AE38 for ; Thu, 21 Apr 2022 11:23:38 -0700 (PDT) Received: by mail-wm1-x335.google.com with SMTP id u17-20020a05600c211100b0038eaf4cdaaeso6432991wml.1 for ; Thu, 21 Apr 2022 11:23:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=dGt5pX+/GfUy7JCUft/YvumTDhqV2Fqma6AGQu5mwYw=; b=EtT8kdFTqTe5wp2If+oYJ60Lb5/91QI/BSlDoz/NAm2vGILFjlhwfLL9/PWO4By8qR I3gad2pRm8VF6hW/NaitHbb9fo15WSjAF2T+vpt+I+alhidbfKRTjNoqAQYvHZjF0Y3j /KF9d6zy5Di5U3dbdlRAi+nS8GV/A2jRNYSxi9RmhGbg0h4gUWHqNuxCDwymirgwPanz fdikAKPVrmpUeW962BcHGA+XhWKzABODPGuZuepIekRl9IyXX3UWRTM5gSd4cxdLYAw1 gFRTkFVCMWdehWEq2C9GvgY8piB5W2wk6YiNjyuNVS5LweIyyD2XpP/5c/LIBFIYkdWl g23Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=dGt5pX+/GfUy7JCUft/YvumTDhqV2Fqma6AGQu5mwYw=; b=kpJ4spOARmW7GK2gQyNfnx+J6eKpWRZT8wDLNnzkEJ651Bz+ZjXM/56gbAkG1NVDmt yCubi1iulvFpkexnt+JaScnF0HsajvppA5YBJvSmMci4arGywqPCmxHmR7H89SioI0Ha y4EAwFOEO4OnSdFnEyvKA8GA2tYH4y0aG3UsCvRR2t/Yi0oXhuy8mW7BZvaF/kMl5Jnh Zq1a4podbr9oXw6GJ6kmdRlb//GCCI65BxrBbXm+Ab7z2Q7CbY7pDUdWXUujMRMqCymu LBKOjgWNX2IDSCuUi0HeYFEp8WMBxiZSXC9emoHX9r2VNWhBNnITacmFvo8+RfxjTx2v gjdA== X-Gm-Message-State: AOAM53030BDu0FZNBvdSTxFLiyYly+CHCAqLo+oXoduVtyYagI8dBStU k2PN7QIGufYoSOEQJxGSykewZmAgQrgo+w== X-Google-Smtp-Source: ABdhPJxFHMHC0ga77I1ktLwWTTnbiQ2t6r7GlRl36Fc4MUFZBOFGSuSeyyNFhtxmlmwqSgf15TNQkg== X-Received: by 2002:a05:600c:1c15:b0:392:8c46:4fd4 with SMTP id j21-20020a05600c1c1500b003928c464fd4mr9520523wms.205.1650565416371; Thu, 21 Apr 2022 11:23:36 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k11-20020a5d6d4b000000b0020599079f68sm2909090wri.106.2022.04.21.11.23.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Apr 2022 11:23:35 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , Eric Sunshine , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBC?= =?utf-8?b?amFybWFzb24=?= Subject: [PATCH v5 02/29] CI: make "$jobname" explicit, remove fallback Date: Thu, 21 Apr 2022 20:22:57 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.879.g3659959fcca In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Remove the $jobname fallback last adjusted in b011fabd6e0 (ci/lib.sh: encapsulate Travis-specific things, 2019-01-27). This makes all of our CI jobs have an explicit $jobname, and removes the special-case of the Windows jobs implicitly setting one (that they don't currently use). In subsequent commits we'll use this new $jobname explicitly. This is a follow-up to my 25715419bf4 (CI: don't run "make test" twice in one job, 2021-11-23), since then all the jobs except the Windows jobs have had an explicit $jobname. In the case of the Windows jobs we were using a jobname of windows-gcc for both, due to picking "gcc" as a default fallback for "$CC" in ci/lib.sh. Since we'll need this to run ci/lib.sh it makes sense to move the check to the top of the file. Subsequent commits will add more such assertions to it. Signed-off-by: Ævar Arnfjörð Bjarmason --- .github/workflows/main.yml | 8 ++++++++ ci/lib.sh | 12 +++++++----- 2 files changed, 15 insertions(+), 5 deletions(-) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index c35200defb9..309b9141249 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -79,6 +79,8 @@ jobs: windows-build: name: win build + env: + jobname: windows-build needs: ci-config if: needs.ci-config.outputs.enabled == 'yes' runs-on: windows-latest @@ -100,6 +102,8 @@ jobs: path: artifacts windows-test: name: win test + env: + jobname: windows-test runs-on: windows-latest needs: [windows-build] strategy: @@ -131,6 +135,8 @@ jobs: path: ${{env.FAILED_TEST_ARTIFACTS}} vs-build: name: win+VS build + env: + jobname: vs-build needs: ci-config if: needs.ci-config.outputs.enabled == 'yes' env: @@ -183,6 +189,8 @@ jobs: path: artifacts vs-test: name: win+VS test + env: + jobname: vs-test runs-on: windows-latest needs: vs-build strategy: diff --git a/ci/lib.sh b/ci/lib.sh index 57141d38a85..6d76c2d406b 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -6,6 +6,13 @@ # and installing dependencies. set -ex +# Starting assertions +if test -z "$jobname" +then + echo "error: must set a CI jobname in the environment" >&2 + exit 1 +fi + skip_branch_tip_with_tag () { # Sometimes, a branch is pushed at the same time the tag that points # at the same commit as the tip of the branch is pushed, and building @@ -146,11 +153,6 @@ test -n "${DONT_SKIP_TAGS-}" || skip_branch_tip_with_tag skip_good_tree -if test -z "$jobname" -then - jobname="$CI_OS_NAME-$CC" -fi - export DEVELOPER=1 export DEFAULT_TEST_TARGET=prove export GIT_TEST_CLONE_2GB=true From patchwork Thu Apr 21 18:22:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12822171 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 37F20C433EF for ; Thu, 21 Apr 2022 18:23:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1391308AbiDUS0j (ORCPT ); Thu, 21 Apr 2022 14:26:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47078 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1391237AbiDUS0f (ORCPT ); Thu, 21 Apr 2022 14:26:35 -0400 Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com [IPv6:2a00:1450:4864:20::329]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4485E4AE14 for ; Thu, 21 Apr 2022 11:23:39 -0700 (PDT) Received: by mail-wm1-x329.google.com with SMTP id v64-20020a1cac43000000b0038cfd1b3a6dso6413611wme.5 for ; Thu, 21 Apr 2022 11:23:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=hk0TzKoDmEfpghLk5LLxx7ep6UrW7ymLhwMtag80aMI=; b=KgXqN4No/ziIshyb4jc0cB+IlLh9OsJT0YbVap0nxGCGb2B+RjjGP/ECo+kir1Zr64 uCkqCWVOE0UIyCp5DzY/SHyluAFThuuFZvEN11eETeWZOOIHkk77tOhAh5hK+GuGxdiZ Pn+5GOIQMQbp3l7RMUpO+Onboh1ge86HHoXKxiOyYxQWDDKsKQDv69PCHwDwgcsc2T0p REZ4845SnWlgcsnW+OQp8466Qk9xB8hc4GoMo+vW2RLYWzNuhvmgJDFRs2nCJtgIFv1P PACtK47oPUitwGxNI9Z7NnFjLhGootYuB3rT5eBcsj2EYYE4gBg+tThr2PGmv1ca6E/N 0uRQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hk0TzKoDmEfpghLk5LLxx7ep6UrW7ymLhwMtag80aMI=; b=Fnul+8qTRTXNVV+V/gYs/HXRjr2xCCZMQzlIdbP03PWH5hy3P8GtRi42nnLqG9/EQF 7KcZxbUixZ9NO9HaonvR9aG5WqrPiBAmfanpiHFHjSQ3xbpt72GGr0LRwCI2yOXtqH3c GfzPjMhBge6WJHPA00WZr8OYkMG+kPhVug6lsIv0w5YL7j9IfYlRQIwAolR2rakz9lil w+MKl5/tQRs20zOgBX7RrXIkSbDU40jz7Ygn1aWt9Er99Xl8hn5nJWwtMTjVrSITVIVd 8LhLc7MsJykKFoDNRFltxDNwAgT7q26UwbmdQ3CDoGZMZ/Wcb6yrWGKy4wc0OgOi03+Z yHtg== X-Gm-Message-State: AOAM532LV6QxUWPxsYHCx1/DnDIDVf8rfrRMU84y3vWVQkHTGDC2sjdq HVTb3+1/DPJo9LT8OqfpQTUBv+1EI9HMfw== X-Google-Smtp-Source: ABdhPJyyFEtDZhxxswv1gEbf2Apf6u0dd2MjwQrQIuING3syk5gigXJGp4gZzbFIZqq8pybGEjGNrA== X-Received: by 2002:a7b:ce04:0:b0:38c:6c34:9aac with SMTP id m4-20020a7bce04000000b0038c6c349aacmr583964wmc.142.1650565417380; Thu, 21 Apr 2022 11:23:37 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k11-20020a5d6d4b000000b0020599079f68sm2909090wri.106.2022.04.21.11.23.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Apr 2022 11:23:36 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , Eric Sunshine , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBC?= =?utf-8?b?amFybWFzb24=?= Subject: [PATCH v5 03/29] CI: remove more dead Travis CI support Date: Thu, 21 Apr 2022 20:22:58 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.879.g3659959fcca In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Remove code that I missed in 4a6e4b96026 (CI: remove Travis CI support, 2021-11-23). This code was only called from or used by the now-removed .travis.yml, or needed by the Travis CI environment. For the symlinking in ci/run-build-and-tests.sh: Back when 3c93b829205 (travis-ci: build Git during the 'script' phase, 2018-01-08) and 4b060a4d973 (ci: use a junction on Windows instead of a symlink, 2019-01-27) adjusted this "ln" command, the Windows build would use ci/run-build-and-tests.sh. As seen in 889cacb6897 (ci: configure GitHub Actions for CI/PR, 2020-04-11) the current windows build uses a different entry point under the GitHub CI, which doesn't use this .prove caching. Namely "ci/run-test-slice.sh". We can be certain that it's never used in "ci/run-test-slice.sh" because to have a ".prove" file we'd need to use "--state=save", which we only do in the dead Azure codepath in ci/lib.sh. If it were used it would do the wrong thing, because the different test slices would each try to clobber the same "t/.prove" file. If a subsequent run then used the "--state=failed,slow,save" it would defeat the purpose of "ci/run-test-slice.sh", since all slices would then run all tests. I.e. behavior of prove's "--state" options is to select tests to run from the provided "--state" file, in addition to those specified on the command-line. For ci/run-docker{,-build}.sh: It was likewise last referenced in the .travis.yml removed in my 4a6e4b96026. The current "dockerized" run in ".github/workflows/main.yml" calls the same entry points as the main "regular" job. For "ci/print-test-failures.sh" the creation of the tarball added in aea8879a6ac (travis-ci: include the trash directories of failed tests in the trace log, 2018-08-01) has been dead since my 4a6e4b96026, which removed the fall-through case from the "case/esac" statement. Signed-off-by: Ævar Arnfjörð Bjarmason --- ci/lib.sh | 10 ++---- ci/print-test-failures.sh | 39 --------------------- ci/run-build-and-tests.sh | 5 --- ci/run-docker-build.sh | 66 ----------------------------------- ci/run-docker.sh | 47 ------------------------- ci/run-test-slice.sh | 5 --- ci/util/extract-trash-dirs.sh | 50 -------------------------- 7 files changed, 2 insertions(+), 220 deletions(-) delete mode 100755 ci/run-docker-build.sh delete mode 100755 ci/run-docker.sh delete mode 100755 ci/util/extract-trash-dirs.sh diff --git a/ci/lib.sh b/ci/lib.sh index 6d76c2d406b..648940da877 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -106,9 +106,6 @@ then CI_COMMIT="$BUILD_SOURCEVERSION" CI_JOB_ID="$BUILD_BUILDID" CI_JOB_NUMBER="$BUILD_BUILDNUMBER" - CI_OS_NAME="$(echo "$AGENT_OS" | tr A-Z a-z)" - test darwin != "$CI_OS_NAME" || CI_OS_NAME=osx - CI_REPO_SLUG="$(expr "$BUILD_REPOSITORY_URI" : '.*/\([^/]*/[^/]*\)$')" CC="${CC:-gcc}" # use a subdirectory of the cache dir (because the file share is shared @@ -118,16 +115,13 @@ then export GIT_PROVE_OPTS="--timer --jobs 10 --state=failed,slow,save" export GIT_TEST_OPTS="--verbose-log -x --write-junit-xml" MAKEFLAGS="$MAKEFLAGS --jobs=10" - test windows_nt != "$CI_OS_NAME" || + test Windows_NT != "$AGENT_OS" || GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS" elif test true = "$GITHUB_ACTIONS" then CI_TYPE=github-actions CI_BRANCH="$GITHUB_REF" CI_COMMIT="$GITHUB_SHA" - CI_OS_NAME="$(echo "$RUNNER_OS" | tr A-Z a-z)" - test macos != "$CI_OS_NAME" || CI_OS_NAME=osx - CI_REPO_SLUG="$GITHUB_REPOSITORY" CI_JOB_ID="$GITHUB_RUN_ID" CC="${CC:-gcc}" DONT_SKIP_TAGS=t @@ -137,7 +131,7 @@ then export GIT_PROVE_OPTS="--timer --jobs 10" export GIT_TEST_OPTS="--verbose-log -x" MAKEFLAGS="$MAKEFLAGS --jobs=10" - test windows != "$CI_OS_NAME" || + test Windows != "$RUNNER_OS" || GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS" else echo "Could not identify CI type" >&2 diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh index 57277eefcd0..740ebb7c529 100755 --- a/ci/print-test-failures.sh +++ b/ci/print-test-failures.sh @@ -16,15 +16,6 @@ then exit fi -case "$jobname" in -osx-clang|osx-gcc) - # base64 in OSX doesn't wrap its output at 76 columns by - # default, but prints a single, very long line. - base64_opts="-b 76" - ;; -esac - -combined_trash_size=0 for TEST_EXIT in test-results/*.exit do if [ "$(cat "$TEST_EXIT")" != "0" ] @@ -42,47 +33,17 @@ do azure-pipelines) mkdir -p failed-test-artifacts mv "$trash_dir" failed-test-artifacts - continue ;; github-actions) mkdir -p failed-test-artifacts echo "FAILED_TEST_ARTIFACTS=t/failed-test-artifacts" >>$GITHUB_ENV cp "${TEST_EXIT%.exit}.out" failed-test-artifacts/ tar czf failed-test-artifacts/"$test_name".trash.tar.gz "$trash_dir" - continue ;; *) echo "Unhandled CI type: $CI_TYPE" >&2 exit 1 ;; esac - trash_tgz_b64="trash.$test_name.base64" - if [ -d "$trash_dir" ] - then - tar czp "$trash_dir" |base64 $base64_opts >"$trash_tgz_b64" - - trash_size=$(wc -c <"$trash_tgz_b64") - if [ $trash_size -gt 1048576 ] - then - # larger than 1MB - echo "$(tput setaf 1)Didn't include the trash directory of '$test_name' in the trace log, it's too big$(tput sgr0)" - continue - fi - - new_combined_trash_size=$(($combined_trash_size + $trash_size)) - if [ $new_combined_trash_size -gt 1048576 ] - then - echo "$(tput setaf 1)Didn't include the trash directory of '$test_name' in the trace log, there is plenty of trash in there already.$(tput sgr0)" - continue - fi - combined_trash_size=$new_combined_trash_size - - # DO NOT modify these two 'echo'-ed strings below - # without updating 'ci/util/extract-trash-dirs.sh' - # as well. - echo "$(tput setaf 1)Start of trash directory of '$test_name':$(tput sgr0)" - cat "$trash_tgz_b64" - echo "$(tput setaf 1)End of trash directory of '$test_name'$(tput sgr0)" - fi fi done diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh index 280dda7d285..0a1ec8c2bae 100755 --- a/ci/run-build-and-tests.sh +++ b/ci/run-build-and-tests.sh @@ -5,11 +5,6 @@ . ${0%/*}/lib.sh -case "$CI_OS_NAME" in -windows*) cmd //c mklink //j t\\.prove "$(cygpath -aw "$cache_dir/.prove")";; -*) ln -s "$cache_dir/.prove" t/.prove;; -esac - export MAKE_TARGETS="all test" case "$jobname" in diff --git a/ci/run-docker-build.sh b/ci/run-docker-build.sh deleted file mode 100755 index 6cd832efb9c..00000000000 --- a/ci/run-docker-build.sh +++ /dev/null @@ -1,66 +0,0 @@ -#!/bin/sh -# -# Build and test Git inside container -# -# Usage: -# run-docker-build.sh -# - -set -ex - -if test $# -ne 1 || test -z "$1" -then - echo >&2 "usage: run-docker-build.sh " - exit 1 -fi - -case "$jobname" in -linux32) - switch_cmd="linux32 --32bit i386" - ;; -linux-musl) - switch_cmd= - useradd () { adduser -D "$@"; } - ;; -*) - exit 1 - ;; -esac - -"${0%/*}/install-docker-dependencies.sh" - -# If this script runs inside a docker container, then all commands are -# usually executed as root. Consequently, the host user might not be -# able to access the test output files. -# If a non 0 host user id is given, then create a user "ci" with that -# user id to make everything accessible to the host user. -HOST_UID=$1 -if test $HOST_UID -eq 0 -then - # Just in case someone does want to run the test suite as root. - CI_USER=root -else - CI_USER=ci - if test "$(id -u $CI_USER 2>/dev/null)" = $HOST_UID - then - echo "user '$CI_USER' already exists with the requested ID $HOST_UID" - else - useradd -u $HOST_UID $CI_USER - fi -fi - -# Build and test -command $switch_cmd su -m -l $CI_USER -c " - set -ex - export DEVELOPER='$DEVELOPER' - export DEFAULT_TEST_TARGET='$DEFAULT_TEST_TARGET' - export GIT_PROVE_OPTS='$GIT_PROVE_OPTS' - export GIT_TEST_OPTS='$GIT_TEST_OPTS' - export GIT_TEST_CLONE_2GB='$GIT_TEST_CLONE_2GB' - export MAKEFLAGS='$MAKEFLAGS' - export cache_dir='$cache_dir' - cd /usr/src/git - test -n '$cache_dir' && ln -s '$cache_dir/.prove' t/.prove - make - make test -" diff --git a/ci/run-docker.sh b/ci/run-docker.sh deleted file mode 100755 index af89d1624a4..00000000000 --- a/ci/run-docker.sh +++ /dev/null @@ -1,47 +0,0 @@ -#!/bin/sh -# -# Download and run Docker image to build and test Git -# - -. ${0%/*}/lib.sh - -case "$jobname" in -linux32) - CI_CONTAINER="daald/ubuntu32:xenial" - ;; -linux-musl) - CI_CONTAINER=alpine - ;; -*) - exit 1 - ;; -esac - -docker pull "$CI_CONTAINER" - -# Use the following command to debug the docker build locally: -# must be 0 if podman is used as drop-in replacement for docker -# $ docker run -itv "${PWD}:/usr/src/git" --entrypoint /bin/sh "$CI_CONTAINER" -# root@container:/# export jobname= -# root@container:/# /usr/src/git/ci/run-docker-build.sh - -container_cache_dir=/tmp/container-cache - -docker run \ - --interactive \ - --env DEVELOPER \ - --env DEFAULT_TEST_TARGET \ - --env GIT_PROVE_OPTS \ - --env GIT_TEST_OPTS \ - --env GIT_TEST_CLONE_2GB \ - --env MAKEFLAGS \ - --env jobname \ - --env cache_dir="$container_cache_dir" \ - --volume "${PWD}:/usr/src/git" \ - --volume "$cache_dir:$container_cache_dir" \ - "$CI_CONTAINER" \ - /usr/src/git/ci/run-docker-build.sh $(id -u $USER) - -check_unignored_build_artifacts - -save_good_tree diff --git a/ci/run-test-slice.sh b/ci/run-test-slice.sh index f8c2c3106a2..b9a682b4bcd 100755 --- a/ci/run-test-slice.sh +++ b/ci/run-test-slice.sh @@ -5,11 +5,6 @@ . ${0%/*}/lib.sh -case "$CI_OS_NAME" in -windows*) cmd //c mklink //j t\\.prove "$(cygpath -aw "$cache_dir/.prove")";; -*) ln -s "$cache_dir/.prove" t/.prove;; -esac - make --quiet -C t T="$(cd t && ./helper/test-tool path-utils slice-tests "$1" "$2" t[0-9]*.sh | tr '\n' ' ')" diff --git a/ci/util/extract-trash-dirs.sh b/ci/util/extract-trash-dirs.sh deleted file mode 100755 index 8e67bec21a2..00000000000 --- a/ci/util/extract-trash-dirs.sh +++ /dev/null @@ -1,50 +0,0 @@ -#!/bin/sh - -error () { - echo >&2 "error: $@" - exit 1 -} - -find_embedded_trash () { - while read -r line - do - case "$line" in - *Start\ of\ trash\ directory\ of\ \'t[0-9][0-9][0-9][0-9]-*\':*) - test_name="${line#*\'}" - test_name="${test_name%\'*}" - - return 0 - esac - done - - return 1 -} - -extract_embedded_trash () { - while read -r line - do - case "$line" in - *End\ of\ trash\ directory\ of\ \'$test_name\'*) - return - ;; - *) - printf '%s\n' "$line" - ;; - esac - done - - error "unexpected end of input" -} - -# Raw logs from Linux build jobs have CRLF line endings, while OSX -# build jobs mostly have CRCRLF, except an odd line every now and -# then that has CRCRCRLF. 'base64 -d' from 'coreutils' doesn't like -# CRs and complains about "invalid input", so remove all CRs at the -# end of lines. -sed -e 's/\r*$//' | \ -while find_embedded_trash -do - echo "Extracting trash directory of '$test_name'" - - extract_embedded_trash |base64 -d |tar xzp -done From patchwork Thu Apr 21 18:22:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12822174 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F2ECAC433F5 for ; Thu, 21 Apr 2022 18:23:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1391329AbiDUS0p (ORCPT ); Thu, 21 Apr 2022 14:26:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47086 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1391288AbiDUS0f (ORCPT ); Thu, 21 Apr 2022 14:26:35 -0400 Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com [IPv6:2a00:1450:4864:20::330]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2A8C04B1DD for ; Thu, 21 Apr 2022 11:23:40 -0700 (PDT) Received: by mail-wm1-x330.google.com with SMTP id n126-20020a1c2784000000b0038e8af3e788so3977813wmn.1 for ; Thu, 21 Apr 2022 11:23:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=RLfBFLJg1sBPte7neCUJOU8fsLLyNGw27jDkYlSFctU=; b=j8mh7777IiQkWP9w5wJS6Rn8CVo7MyQ2kgQ3FeFkM3tOFzXqC71Cu0pfqwvttZDD4V HWUeQl4pn1SvwGqsZNHu3GTQHzIZPQZHuTQPsCc8/GJ2+WLJ7//+KuQL4/GYDLeQVWYl 0dq9+9HVXEKYsPCbXSCq1Mg20z43aZWxaGpomz6ZW7dAKw6oUCsrox3DyOvgjbKZGgJK xLcMvuQK1jIfxCqlbLQMnHarOQ5zyk3DxA7feY4NiDGJSGnWgSQkbyyvRjFmAPMdiZIT qsoIN5JnseumdjBb2AEt/DzqJGj/rV5nvLxzW1Xty78WVvPpXyY8Wh/skRv7YUVyOJ9f C2DA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=RLfBFLJg1sBPte7neCUJOU8fsLLyNGw27jDkYlSFctU=; b=qj3QJ/NxKU765Cm9KGuILbS0vBL27tMTCJXZRU3NEP8dtExk7ttYybS2jT33rJ4MTg VLv8T30jYIS3yLvQtXqEeGbS7LRy1lr73X5Zk/5eBNdtBIby3sFYmwTa53kcTHscxq2+ IY0nWiEaMlDISJYfsLOHwT8/PjO6x6JIOPfPIexJl+ZjvNzDJu5TgqcU2hXWS/3Zxo/0 H7s5oa5apD6YYUz7AzIErDCIinOL8af4+MuKwjVch/wTBB+oejZyDOkpbOAhd0W6kVCb C0Wm+S0d/Swjm8sRq+Ub3sSLLU7FkkdVns2+FwRzo0BiKHnosoUMhv+0VccXUgahXqKn WnSQ== X-Gm-Message-State: AOAM530985nhdb4qQovaPBouidlpAaf4Y0BxwHP0ZwTO0dDVJY3Zof2J Fyc31VnMTbcIwQr72hrKwX/csnAAEY8nOw== X-Google-Smtp-Source: ABdhPJwul2iTsq6YttNmYYLJYEI2zlle9KENlukXwvK1c73wK2p9WoL3kT26NS35zdlL0XsRU2OaZw== X-Received: by 2002:a05:600c:1d29:b0:38e:c4d1:a41b with SMTP id l41-20020a05600c1d2900b0038ec4d1a41bmr9712834wms.70.1650565418492; Thu, 21 Apr 2022 11:23:38 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k11-20020a5d6d4b000000b0020599079f68sm2909090wri.106.2022.04.21.11.23.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Apr 2022 11:23:37 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , Eric Sunshine , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBC?= =?utf-8?b?amFybWFzb24=?= Subject: [PATCH v5 04/29] CI: remove dead "tree skipping" code Date: Thu, 21 Apr 2022 20:22:59 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.879.g3659959fcca In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Remove the code related to skipping CI runs if the tree was successfully tested before. With the GitHub CI this is done with the "ci-config" step, see e76eec35540 (ci: allow per-branch config for GitHub Actions, 2020-05-07) and 7d78d5fc1a9 (ci: skip GitHub workflow runs for already-tested commits/trees, 2020-10-08). This code hasn't been used since 4a6e4b96026 (CI: remove Travis CI support, 2021-11-23), and before that for the retired Azure pipeline support removed in 6081d3898fe (ci: retire the Azure Pipelines definition, 2020-04-11). This change is needed because this and subsequent commits are turning "ci/lib.sh" into a dumber library that'll only be tasked with setting variables for CI jobs, or for specific steps within those jobs. Now we don't need to worry about it potentially skipping the run on its own. This change also removes a subtle potential logic error introduced in 0e7696c64db (ci: disallow directional formatting, 2021-11-04). The "ci/check-directional-formatting.bash" script would have been made to run after the "save_good_tree" in invoked in "ci/run-static-analysis.sh". So if e.g. the "make coccicheck" failed we'd still mark the tree as good. I.e. the addition of "ci/check-directional-formatting.bash" didn't take into account that the various users of the "save_good_tree" function in ci/*.sh made a hard assumption that they're the only ci/*.sh script that's being run (and they use "set -e"). This is entirely academic since we weren't actually running this code, but is something to be careful of if anyone ever needs to resurrect parts of this. Finally, the cache_dir="$HOME/none" added in a3f2eec862b (ci/lib: allow running in GitHub Actions, 2020-04-08) for the GitHub CI can be removed. It appears to have been added to appease the subsequent 'mkdir -p "$cache_dir"', which wasn't needed in GitHub CI. 1. https://lore.kernel.org/git/patch-1.1-eec0a8c3164-20211217T000418Z-avarab@gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason --- ci/lib.sh | 91 --------------------------------------- ci/run-build-and-tests.sh | 2 - ci/run-static-analysis.sh | 2 - ci/test-documentation.sh | 2 - 4 files changed, 97 deletions(-) diff --git a/ci/lib.sh b/ci/lib.sh index 648940da877..a30ab35d9c2 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -13,75 +13,6 @@ then exit 1 fi -skip_branch_tip_with_tag () { - # Sometimes, a branch is pushed at the same time the tag that points - # at the same commit as the tip of the branch is pushed, and building - # both at the same time is a waste. - # - # When the build is triggered by a push to a tag, $CI_BRANCH will - # have that tagname, e.g. v2.14.0. Let's see if $CI_BRANCH is - # exactly at a tag, and if so, if it is different from $CI_BRANCH. - # That way, we can tell if we are building the tip of a branch that - # is tagged and we can skip the build because we won't be skipping a - # build of a tag. - - if TAG=$(git describe --exact-match "$CI_BRANCH" 2>/dev/null) && - test "$TAG" != "$CI_BRANCH" - then - echo "$(tput setaf 2)Tip of $CI_BRANCH is exactly at $TAG$(tput sgr0)" - exit 0 - fi -} - -# Save some info about the current commit's tree, so we can skip the build -# job if we encounter the same tree again and can provide a useful info -# message. -save_good_tree () { - echo "$(git rev-parse $CI_COMMIT^{tree}) $CI_COMMIT $CI_JOB_NUMBER $CI_JOB_ID" >>"$good_trees_file" - # limit the file size - tail -1000 "$good_trees_file" >"$good_trees_file".tmp - mv "$good_trees_file".tmp "$good_trees_file" -} - -# Skip the build job if the same tree has already been built and tested -# successfully before (e.g. because the branch got rebased, changing only -# the commit messages). -skip_good_tree () { - if test true = "$GITHUB_ACTIONS" - then - return - fi - - if ! good_tree_info="$(grep "^$(git rev-parse $CI_COMMIT^{tree}) " "$good_trees_file")" - then - # Haven't seen this tree yet, or no cached good trees file yet. - # Continue the build job. - return - fi - - echo "$good_tree_info" | { - read tree prev_good_commit prev_good_job_number prev_good_job_id - - if test "$CI_JOB_ID" = "$prev_good_job_id" - then - cat <<-EOF - $(tput setaf 2)Skipping build job for commit $CI_COMMIT.$(tput sgr0) - This commit has already been built and tested successfully by this build job. - To force a re-build delete the branch's cache and then hit 'Restart job'. - EOF - else - cat <<-EOF - $(tput setaf 2)Skipping build job for commit $CI_COMMIT.$(tput sgr0) - This commit's tree has already been built and tested successfully in build job $prev_good_job_number for commit $prev_good_commit. - The log of that build job is available at $SYSTEM_TASKDEFINITIONSURI$SYSTEM_TEAMPROJECT/_build/results?buildId=$prev_good_job_id - To force a re-build delete the branch's cache and then hit 'Restart job'. - EOF - fi - } - - exit 0 -} - check_unignored_build_artifacts () { ! git ls-files --other --exclude-standard --error-unmatch \ @@ -102,16 +33,8 @@ if test -n "$SYSTEM_COLLECTIONURI" || test -n "$SYSTEM_TASKDEFINITIONSURI" then CI_TYPE=azure-pipelines # We are running in Azure Pipelines - CI_BRANCH="$BUILD_SOURCEBRANCH" - CI_COMMIT="$BUILD_SOURCEVERSION" - CI_JOB_ID="$BUILD_BUILDID" - CI_JOB_NUMBER="$BUILD_BUILDNUMBER" CC="${CC:-gcc}" - # use a subdirectory of the cache dir (because the file share is shared - # among *all* phases) - cache_dir="$HOME/test-cache/$SYSTEM_PHASENAME" - export GIT_PROVE_OPTS="--timer --jobs 10 --state=failed,slow,save" export GIT_TEST_OPTS="--verbose-log -x --write-junit-xml" MAKEFLAGS="$MAKEFLAGS --jobs=10" @@ -120,13 +43,7 @@ then elif test true = "$GITHUB_ACTIONS" then CI_TYPE=github-actions - CI_BRANCH="$GITHUB_REF" - CI_COMMIT="$GITHUB_SHA" - CI_JOB_ID="$GITHUB_RUN_ID" CC="${CC:-gcc}" - DONT_SKIP_TAGS=t - - cache_dir="$HOME/none" export GIT_PROVE_OPTS="--timer --jobs 10" export GIT_TEST_OPTS="--verbose-log -x" @@ -139,14 +56,6 @@ else exit 1 fi -good_trees_file="$cache_dir/good-trees" - -mkdir -p "$cache_dir" - -test -n "${DONT_SKIP_TAGS-}" || -skip_branch_tip_with_tag -skip_good_tree - export DEVELOPER=1 export DEFAULT_TEST_TARGET=prove export GIT_TEST_CLONE_2GB=true diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh index 0a1ec8c2bae..e5a21985b44 100755 --- a/ci/run-build-and-tests.sh +++ b/ci/run-build-and-tests.sh @@ -45,5 +45,3 @@ esac # start running tests. make $MAKE_TARGETS check_unignored_build_artifacts - -save_good_tree diff --git a/ci/run-static-analysis.sh b/ci/run-static-analysis.sh index 65bcebda41a..5a87b1cac96 100755 --- a/ci/run-static-analysis.sh +++ b/ci/run-static-analysis.sh @@ -28,5 +28,3 @@ fi make hdr-check || exit 1 - -save_good_tree diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh index de41888430a..9e0652c30dd 100755 --- a/ci/test-documentation.sh +++ b/ci/test-documentation.sh @@ -41,5 +41,3 @@ grep ' X-Patchwork-Id: 12822181 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D5641C433EF for ; Thu, 21 Apr 2022 18:24:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1391342AbiDUS1F (ORCPT ); Thu, 21 Apr 2022 14:27:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47070 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1391292AbiDUS0f (ORCPT ); Thu, 21 Apr 2022 14:26:35 -0400 Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com [IPv6:2a00:1450:4864:20::335]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 193124B1FB for ; Thu, 21 Apr 2022 11:23:41 -0700 (PDT) Received: by mail-wm1-x335.google.com with SMTP id bg25so3002923wmb.4 for ; Thu, 21 Apr 2022 11:23:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=7Lapla7bdj1i+B+MklpJ7L5IL7ja01Drd/6puN0vfq4=; b=QHqXBEu5ESK9O4n4eZXVzrrnQRJuyYqKWPtngNHgk7pIh332SWc0XCW/pr80jf5e+F 9d8uiXVz3+sVcxjR8butmQ0Ww159BhsEVIIhojnqFYNP+Mc1SB8jBP0y7yc+oW5S8tyg GZwUkKGohd5Kz5k/GyVDuRj/WjNRegtM8Ai8HZT0CYGx3gQprDKR7+D05bxA4InJbZKM fDfZNoTLlJZoFZroLiO6zGZwnTzrzidXVwpRz5ZFFVVYdj73GCJkXMEEcCKNE6o9HdgU hKVCwKuh+SQfVGf3eDSwcLeo7y+K3m3Bf97pv/CcPnBRG1dudOQOCmYmR+FStSsln24D R5xQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=7Lapla7bdj1i+B+MklpJ7L5IL7ja01Drd/6puN0vfq4=; b=ThFotZ1yET04oOtjxHmEXK3PNM/U3uepnBEJJDguK123vXPtikUA7noSJuvNREa467 tZIlzlKAGBPyq+NIQfCVgvNfY/pTFmHgdst++XyQT1L6RRhgH1jfUeIui0GTyQdJkWBy exeinj0eauIKdfPLO6s3gBrI+Md4DIy9dcrVfL8Dre+MMk1Uf1sY6NJbK86AAQBQFv5f 7iEs8+RS15HYxjpZl94QR0pYme1WopClJ7F9Cp7hiUjYxXuzUhqw4NDOHuhTIAc8fKYB mCj+WR1Z0TO6N+szcDFEA0RhRmJHN8ik+wC1/7vVshJAiLQcaPvCxW7iZ+929TvBjqrI f1hQ== X-Gm-Message-State: AOAM532Jgr6NfZmgxVxQd2BJgfKlfOd551XND6BAPChJfhhs0feszHxz /Pil5AaMgQLJX1MtNHOtpkr0iOHjr5O5ig== X-Google-Smtp-Source: ABdhPJwuY3IrNZyZzwpcGVxQeFI9Bjj98TCB540O7+2KWtKIR+hYyiVWwtBuM43X3966+6xasuurag== X-Received: by 2002:a05:600c:20c7:b0:38e:bf41:edef with SMTP id y7-20020a05600c20c700b0038ebf41edefmr9791893wmm.173.1650565419449; Thu, 21 Apr 2022 11:23:39 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k11-20020a5d6d4b000000b0020599079f68sm2909090wri.106.2022.04.21.11.23.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Apr 2022 11:23:38 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , Eric Sunshine , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBC?= =?utf-8?b?amFybWFzb24=?= Subject: [PATCH v5 05/29] CI: remove unused Azure ci/* code Date: Thu, 21 Apr 2022 20:23:00 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.879.g3659959fcca In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Remove Azure-specific code that's been unused since 6081d3898fe (ci: retire the Azure Pipelines definition, 2020-04-11). As noted in a larger removal of all of the Azure-supporting code in [1] (although that missed some of this) there is more of it in-tree, but let's focus on only the ci/* code for now. This is needed because in subsequent commits this unused code would either need to be changed to accommodate changes in ci/*, or be removed. As we'll see in those subsequent commits the end-state we're heading towards will actually make it easier to add new CI types in the future, even though the only one we're left with now is the GitHub CI. I.e. the "ci/*" framework will be made to do less, not more. We'll be offloading more of what it does to our generic build and test system. While I'm at it (since the line needs to be touched anyway) change an odd 'if test true == "$GITHUB_ACTIONS"' to the more usual style used in other places of 'if test "$GITHUB_ACTIONS" = "true"'. 1. https://lore.kernel.org/git/patch-1.1-eec0a8c3164-20211217T000418Z-avarab@gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason --- ci/install-dependencies.sh | 3 --- ci/lib.sh | 13 +------------ ci/mount-fileshare.sh | 25 ------------------------- ci/print-test-failures.sh | 4 ---- 4 files changed, 1 insertion(+), 44 deletions(-) delete mode 100755 ci/mount-fileshare.sh diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh index dbcebad2fb2..e7ea8799411 100755 --- a/ci/install-dependencies.sh +++ b/ci/install-dependencies.sh @@ -34,8 +34,6 @@ macos-latest) export HOMEBREW_NO_AUTO_UPDATE=1 HOMEBREW_NO_INSTALL_CLEANUP=1 # Uncomment this if you want to run perf tests: # brew install gnu-time - test -z "$BREW_INSTALL_PACKAGES" || - brew install $BREW_INSTALL_PACKAGES brew link --force gettext brew install --cask --no-quarantine perforce || { # Update the definitions and try again @@ -69,7 +67,6 @@ Documentation) sudo apt-get -q update sudo apt-get -q -y install asciidoc xmlto docbook-xsl-ns make - test -n "$ALREADY_HAVE_ASCIIDOCTOR" || sudo gem install --version 1.5.8 asciidoctor ;; linux-gcc-default) diff --git a/ci/lib.sh b/ci/lib.sh index a30ab35d9c2..a6cdcfa014b 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -29,18 +29,7 @@ export TERM=${TERM:-dumb} # Clear MAKEFLAGS that may come from the outside world. export MAKEFLAGS= -if test -n "$SYSTEM_COLLECTIONURI" || test -n "$SYSTEM_TASKDEFINITIONSURI" -then - CI_TYPE=azure-pipelines - # We are running in Azure Pipelines - CC="${CC:-gcc}" - - export GIT_PROVE_OPTS="--timer --jobs 10 --state=failed,slow,save" - export GIT_TEST_OPTS="--verbose-log -x --write-junit-xml" - MAKEFLAGS="$MAKEFLAGS --jobs=10" - test Windows_NT != "$AGENT_OS" || - GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS" -elif test true = "$GITHUB_ACTIONS" +if test "$GITHUB_ACTIONS" = "true" then CI_TYPE=github-actions CC="${CC:-gcc}" diff --git a/ci/mount-fileshare.sh b/ci/mount-fileshare.sh deleted file mode 100755 index 26b58a80960..00000000000 --- a/ci/mount-fileshare.sh +++ /dev/null @@ -1,25 +0,0 @@ -#!/bin/sh - -die () { - echo "$*" >&2 - exit 1 -} - -test $# = 4 || -die "Usage: $0 " - -mkdir -p "$4" || die "Could not create $4" - -case "$(uname -s)" in -Linux) - sudo mount -t cifs -o vers=3.0,username="$2",password="$3",dir_mode=0777,file_mode=0777,serverino "$1" "$4" - ;; -Darwin) - pass="$(echo "$3" | sed -e 's/\//%2F/g' -e 's/+/%2B/g')" && - mount -t smbfs,soft "smb://$2:$pass@${1#//}" "$4" - ;; -*) - die "No support for $(uname -s)" - ;; -esac || -die "Could not mount $4" diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh index 740ebb7c529..47e2ea1ad10 100755 --- a/ci/print-test-failures.sh +++ b/ci/print-test-failures.sh @@ -30,10 +30,6 @@ do test_name="${test_name##*/}" trash_dir="trash directory.$test_name" case "$CI_TYPE" in - azure-pipelines) - mkdir -p failed-test-artifacts - mv "$trash_dir" failed-test-artifacts - ;; github-actions) mkdir -p failed-test-artifacts echo "FAILED_TEST_ARTIFACTS=t/failed-test-artifacts" >>$GITHUB_ENV From patchwork Thu Apr 21 18:23:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12822182 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A33A4C433EF for ; Thu, 21 Apr 2022 18:24:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1391344AbiDUS1H (ORCPT ); Thu, 21 Apr 2022 14:27:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47076 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1391296AbiDUS0f (ORCPT ); Thu, 21 Apr 2022 14:26:35 -0400 Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com [IPv6:2a00:1450:4864:20::330]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 062EC4B1FF for ; Thu, 21 Apr 2022 11:23:42 -0700 (PDT) Received: by mail-wm1-x330.google.com with SMTP id r4-20020a05600c35c400b0039295dc1fc3so3975005wmq.3 for ; Thu, 21 Apr 2022 11:23:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=1UZG2+FDX/SsTVNX2KhVttTMLEQMgCP4/6H+sdKR6IE=; b=YnqfQUsNVMVabBGK6GZFiHI4mQFeRXmJ+PJLnVbTT7AN7m9n1tC9rEanX7z33B+x3G GYgSrz4BlFuSL79ibNZ+hYIOPFpJGstCfeWhpcQceoFJUuZovOMwr+BkJ17sET1UWpbx SxURpj+ZBTE2TMcgxamXRjl9boIglvkEK5mcWbGkRiU8qXgzS+nOd/6V03ToOeTuTNQ6 EZ9BoiQzsfKauftxYNzYKhaq4DtUz6ux5SsBhzfGBagBsaTVo5tIEHsa2y1cy58RuvLY esF3y1fq48laN1CnuWBYdwVtYNS4+1KgI2vktLsd2lWliT1/bgDZhPdQlbgEDXRdJ1fB aAmw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=1UZG2+FDX/SsTVNX2KhVttTMLEQMgCP4/6H+sdKR6IE=; b=1k9OxNOey3Yn29Xgq+qvo0lPkxdZtZow4nbbmbNCSsYABxr3DV8/naITv2KOWLJSdb yLLFOUTpcaSgxP1qr8Xi4LENB2Jekzpzc49luA9rDXM983DB5usHiWnX78MkXGlLwsWI cB2feUoTpBeC+CaMlu4FpbUUCxxN4efmF84Hll3+nI7l+RZ6FcTNl1aojiUMtPeEsdIR LDRMtzDqVLQXusV0s19otL4bPqBppg7xZNNa5F5KpRNeGIMyrAeEuaUbI1MykUALLus7 vwWUlvnj5I2Bb8JUoj6UTbsNDk6Ojb6HGq3R4/VpZaLwe3QC5ixwIuK07SDJSHJOifg7 eHtw== X-Gm-Message-State: AOAM5330BkLR3jhJtFqHpSvWstcssNHQvpUo2BAiWFyrWVeeCn2Gs0OB sIXGjof7w0qm8ZAM+83sviwpcpHqGWhuNA== X-Google-Smtp-Source: ABdhPJzZwm0g5za1GGFcyGEe9ks2R7mya9CDhVcmmAmRIdYap6ReVqr225NkminwyXffvpbE78WTzQ== X-Received: by 2002:a05:600c:1d12:b0:391:3cf6:243e with SMTP id l18-20020a05600c1d1200b003913cf6243emr565593wms.137.1650565420336; Thu, 21 Apr 2022 11:23:40 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k11-20020a5d6d4b000000b0020599079f68sm2909090wri.106.2022.04.21.11.23.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Apr 2022 11:23:39 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , Eric Sunshine , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBC?= =?utf-8?b?amFybWFzb24=?= Subject: [PATCH v5 06/29] CI/lib.sh: stop adding leading whitespace to $MAKEFLAGS Date: Thu, 21 Apr 2022 20:23:01 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.879.g3659959fcca In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In a8c51f77d16 (ci: clear and mark MAKEFLAGS exported just once, 2019-02-07) the assignment of MAKEFLAGS was done up-front to future-proof this code for the bug fixed in 406f93ae488 (ci: make sure we build Git parallel, 2019-02-07). But since 4a6e4b96026 (CI: remove Travis CI support, 2021-11-23) the --jobs=10 has been an invariant here, so let's assign --jobs=$NPROC to the MAKEFLAGS right away, this gets rid of the leading whitespace from the $MAKEFLAGS variable, as it would previously be empty on the first assignment. Signed-off-by: Ævar Arnfjörð Bjarmason --- ci/lib.sh | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/ci/lib.sh b/ci/lib.sh index a6cdcfa014b..edf26984aeb 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -26,17 +26,19 @@ check_unignored_build_artifacts () # GitHub Action doesn't set TERM, which is required by tput export TERM=${TERM:-dumb} +# How many jobs to run in parallel? +NPROC=10 + # Clear MAKEFLAGS that may come from the outside world. -export MAKEFLAGS= +export MAKEFLAGS=--jobs=$NPROC if test "$GITHUB_ACTIONS" = "true" then CI_TYPE=github-actions CC="${CC:-gcc}" - export GIT_PROVE_OPTS="--timer --jobs 10" + export GIT_PROVE_OPTS="--timer --jobs $NPROC" export GIT_TEST_OPTS="--verbose-log -x" - MAKEFLAGS="$MAKEFLAGS --jobs=10" test Windows != "$RUNNER_OS" || GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS" else From patchwork Thu Apr 21 18:23:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12822180 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0BB72C433F5 for ; Thu, 21 Apr 2022 18:24:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1391340AbiDUS1D (ORCPT ); Thu, 21 Apr 2022 14:27:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47082 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1391271AbiDUS0g (ORCPT ); Thu, 21 Apr 2022 14:26:36 -0400 Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com [IPv6:2a00:1450:4864:20::329]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F0ACE4B42F for ; Thu, 21 Apr 2022 11:23:42 -0700 (PDT) Received: by mail-wm1-x329.google.com with SMTP id r19so3721936wmq.0 for ; Thu, 21 Apr 2022 11:23:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=0YLmAHuYAq0v4BJ6eaPK9P1vsJBoYJl0pC8kgGRrJyM=; b=GoAXAyDGC0UyM/IglPiNJwmOmcLptRC4cd1WefBAye743Z9rzSsbgVfRmYVDUmrRci U09OVvfMduhrG7fWzSOrHcWB1ewEXhLNoQV+vL9Ue4U0cRw8oBmwOT2IPr87no2jYBmG Dl/ARCb3PgaX2AhzT0KJPde0QYc11+5E2vjk2I+h6D+qnH5sMSxBL+pkZjTSjtCENLys m6Ks+U9ctCzQmxQdZ9Runt/17L+AgMq2h6ivM1RHdGMeZAmBDmNxwSBLjOUP0C9j2Qk4 xeJliXsvqamfYAIUsIQJAFnBGrTPlYh+xe4IPnbYGcb6IvjoRNXjtvlX7Etklrdq9j34 pFrg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0YLmAHuYAq0v4BJ6eaPK9P1vsJBoYJl0pC8kgGRrJyM=; b=gY4Gpumy9wKPkFGcHJ9caP+cPrZ95NOqGuD2enJGtplJ8M//D7EL6d8v7Oiuu3si6J Mkmg3zuH1a9crIdYwau0VhKQ62jjY9PROeUayMvClKEEpoa/MBgsQ3G3Ngcugqfx7yeP D+Wz2GjFFpiLYFjttVpbQrho59rjoUOMqo3xuIzQw5h1eByU9efIUL9M0T/F9+IWHHYZ 2XLrSHP44hdiZvnLtJwjFnCYtleY2D5FFB792uLT6296u4H7PiUx7lmq23zGIWP5Z73m ezibJy/rPzYRe696r1avLXLGtgN1yW3C9t9/vipaQtntaNHEpn9esQEPRb6j0+E9pMKi OayA== X-Gm-Message-State: AOAM5316OQFK9ENxI1NH5IMqlJIXb10QNu3pBy1EGxxRO0hscRTn1asp +LwpZkp+kpUGjCFzSCk5LkGEb9s9TBxpSA== X-Google-Smtp-Source: ABdhPJxT3Xa3t0KXj9YwBHdTLR2RH4oz1PZIuA0qi2Ab/AGSAw7DrCCF+WQDHMzf6C9Pa+Ua5MfKNA== X-Received: by 2002:a7b:c310:0:b0:38c:f07a:e10d with SMTP id k16-20020a7bc310000000b0038cf07ae10dmr610006wmj.110.1650565421241; Thu, 21 Apr 2022 11:23:41 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k11-20020a5d6d4b000000b0020599079f68sm2909090wri.106.2022.04.21.11.23.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Apr 2022 11:23:40 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , Eric Sunshine , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBC?= =?utf-8?b?amFybWFzb24=?= Subject: [PATCH v5 07/29] CI: don't have "git grep" invoke a pager in tree content check Date: Thu, 21 Apr 2022 20:23:02 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.879.g3659959fcca In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Fix the omission of not adding "-P" (a.k.a. "--no-pager") to the "git" invocation that spaws "git grep", added in 0e7696c64db (ci: disallow directional formatting, 2021-11-04). This didn't matter for the GitHub CI, but does when running the check locally. A subsequent commit will start invoking it via a new check from the top-level Makefile, and without this we'd visibly spawn a pager when doing so. Signed-off-by: Ævar Arnfjörð Bjarmason --- ci/check-directional-formatting.bash | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ci/check-directional-formatting.bash b/ci/check-directional-formatting.bash index e6211b141aa..de8e6e486f0 100755 --- a/ci/check-directional-formatting.bash +++ b/ci/check-directional-formatting.bash @@ -23,5 +23,5 @@ # U+2066..U+2069: LRI, RLI, FSI and PDI regex='(\u202a|\u202b|\u202c|\u202d|\u202e|\u2066|\u2067|\u2068|\u2069)' -! LC_CTYPE=C git grep -El "$(LC_CTYPE=C.UTF-8 printf "$regex")" \ +! LC_CTYPE=C git -P grep -El "$(LC_CTYPE=C.UTF-8 printf "$regex")" \ -- ':(exclude,attr:binary)' ':(exclude)*.po' From patchwork Thu Apr 21 18:23:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12822175 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B07C1C433F5 for ; Thu, 21 Apr 2022 18:23:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1391330AbiDUS0r (ORCPT ); Thu, 21 Apr 2022 14:26:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47128 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1391303AbiDUS0g (ORCPT ); Thu, 21 Apr 2022 14:26:36 -0400 Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com [IPv6:2a00:1450:4864:20::32a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 17CE34B436 for ; Thu, 21 Apr 2022 11:23:43 -0700 (PDT) Received: by mail-wm1-x32a.google.com with SMTP id bg25so3002996wmb.4 for ; Thu, 21 Apr 2022 11:23:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=jDvU982G2/CYVsnX0HBg4rxjdT9bVcnyrD7SxrTWg7E=; b=L4OGngu5xoe3K0ySPz7Jy3vlbLooZpe1I3jKAALCo9ke96UHjXRs6NnALfKS5r+lPG c189aE2kjMhZpIGI0n+4wMwycXkwCcGCbyhan556OBg8uts7sSuOqC+yUNZkgjbl0ews gD1LORdQwQygmVm9ADtd/xCZSA9gE56IXtg+9Mk/sbIDGs8+mLyNtaJFYLhWpQYij5qf 15s77Rw26wKKsHyNe9gAoYir32vfnvHq0E09e3b/Hq95rkF7O+gYjGrqLXL/S4yyxlgD 5g1ZB4Iw7+l7G2f/HkGli5x0dYeqTnSQmzFH6HKercf6DiL15YJXs9JV0z3rgAKIHr1E iAEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=jDvU982G2/CYVsnX0HBg4rxjdT9bVcnyrD7SxrTWg7E=; b=P5z8gB0V/hGyuy0z7UvTUwhx6zlsXOh0OheGgx1Mw2TjLzAJIM8WW/gIZrp7fIbAW5 AB74pZJTR2D/wSAJlFi3zuuYy8FGyVDK9JIi6/t5ezVmpRRfht4bbvkIFxpuuA5ImFPB l5GsIkSCDRIUdFivPbgByrtHUyWRgf3/F9YgYCXzUn1CWMX4wIgKBY/twisSf2OKS0SF O0Kg6nNd7QFYTxK8JlW9SI0QFLnIOmh7L+/Dos1lYp7zBCNaXNc/LB1I/uwIeMbg2oKj 4wpJC0L5Zul23LY5FMlI+rE/WWddEqjcMHxhHKmraYtvom4ecQjrwEOmeFFyN8efNTuj IUfg== X-Gm-Message-State: AOAM531JJtAE4Uhz0Vrx2QHNcX2t1qfxJYs5aC+3UaYz0h+hU88kx8ag ReJBjZ7yAD597ZwWWyAfKlprbNPwYYc9ww== X-Google-Smtp-Source: ABdhPJwGjwO9M8wrfHWbgOHdmhEuoLTp6Z3mvCIzcPpoPTVTu2ElrFdVXe4A8Z+BYdzmqkiuEF2ZfA== X-Received: by 2002:a1c:721a:0:b0:38e:c24d:7b8d with SMTP id n26-20020a1c721a000000b0038ec24d7b8dmr9976718wmc.83.1650565422298; Thu, 21 Apr 2022 11:23:42 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k11-20020a5d6d4b000000b0020599079f68sm2909090wri.106.2022.04.21.11.23.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Apr 2022 11:23:41 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , Eric Sunshine , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBC?= =?utf-8?b?amFybWFzb24=?= Subject: [PATCH v5 08/29] CI: have "static-analysis" run a "make ci-static-analysis" target Date: Thu, 21 Apr 2022 20:23:03 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.879.g3659959fcca In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Make it easier to run the CI logic locally by creating "ci-static-analysis" target. Now the "check-coccicheck" logic added in 0860a7641ba (travis-ci: fail if Coccinelle static analysis found something to transform, 2018-07-23) can be easily tested outside of CI, and likewise the sanity check added in 0e7696c64db (ci: disallow directional formatting, 2021-11-04). These targets could be improved, the "hdr-check" target will re-do all of its work every time it's run, and could with a change similar to my c234e8a0ecf (Makefile: make the "sparse" target non-.PHONY, 2021-09-23). The same goes for the new "ci-check-directional-formatting" target. But without those improvements being able to easily run a 1=1 equivalent of the checks we do in CI during a local build is already an improvement. Subsequent commits will expand on this ability. This change will also make the CI check faster, since we can take advantage of parallelism. The "make coccicheck" takes a very long time, and at the tail end we might only have one job pegging 100% of one CPU core for a long time. Now any extra cores will be free to run the rest of the targets under "ci-static-analysis". Signed-off-by: Ævar Arnfjörð Bjarmason --- .github/workflows/main.yml | 3 +-- Makefile | 28 ++++++++++++++++++++++++++++ ci/run-static-analysis.sh | 30 ------------------------------ shared.mak | 1 + 4 files changed, 30 insertions(+), 32 deletions(-) delete mode 100755 ci/run-static-analysis.sh diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 309b9141249..ad752010102 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -317,8 +317,7 @@ jobs: steps: - uses: actions/checkout@v2 - run: ci/install-dependencies.sh - - run: ci/run-static-analysis.sh - - run: ci/check-directional-formatting.bash + - run: make ci-static-analysis sparse: needs: ci-config if: needs.ci-config.outputs.enabled == 'yes' diff --git a/Makefile b/Makefile index f8bccfab5e9..8717ea7e745 100644 --- a/Makefile +++ b/Makefile @@ -3031,6 +3031,20 @@ coccicheck: $(addsuffix .patch,$(filter-out %.pending.cocci,$(wildcard contrib/c # See contrib/coccinelle/README coccicheck-pending: $(addsuffix .patch,$(wildcard contrib/coccinelle/*.pending.cocci)) +.PHONY: check-coccicheck +check-coccicheck: coccicheck + $(QUIET_CHECK)for cocci_patch in contrib/coccinelle/*.patch; do \ + if test -s "$$cocci_patch"; then \ + echo "Coccinelle suggests the following changes in '$$cocci_patch':"; \ + cat "$$cocci_patch"; \ + fail=UnfortunatelyYes; \ + fi \ + done; \ + if test -n "$$fail"; then \ + echo "error: Coccinelle suggested some changes"; \ + exit 1; \ + fi + .PHONY: coccicheck coccicheck-pending ### Installation rules @@ -3466,3 +3480,17 @@ $(FUZZ_PROGRAMS): all $(XDIFF_OBJS) $(EXTLIBS) git.o $@.o $(LIB_FUZZING_ENGINE) -o $@ fuzz-all: $(FUZZ_PROGRAMS) + +### CI "check" targets +# +# These targets are run from the CI, see .github/workflows/main.yml, +# but can also be run manually to run the same assertions locally. + +.PHONY: ci-check-directional-formatting +ci-check-directional-formatting: + $(QUIET_CHECK)ci/check-directional-formatting.bash + +.PHONY: ci-static-analysis +ci-static-analysis: ci-check-directional-formatting +ci-static-analysis: check-coccicheck +ci-static-analysis: hdr-check diff --git a/ci/run-static-analysis.sh b/ci/run-static-analysis.sh deleted file mode 100755 index 5a87b1cac96..00000000000 --- a/ci/run-static-analysis.sh +++ /dev/null @@ -1,30 +0,0 @@ -#!/bin/sh -# -# Perform various static code analysis checks -# - -. ${0%/*}/lib.sh - -make coccicheck - -set +x - -fail= -for cocci_patch in contrib/coccinelle/*.patch -do - if test -s "$cocci_patch" - then - echo "$(tput setaf 1)Coccinelle suggests the following changes in '$cocci_patch':$(tput sgr0)" - cat "$cocci_patch" - fail=UnfortunatelyYes - fi -done - -if test -n "$fail" -then - echo "$(tput setaf 1)error: Coccinelle suggested some changes$(tput sgr0)" - exit 1 -fi - -make hdr-check || -exit 1 diff --git a/shared.mak b/shared.mak index 50d4596f0d6..83e0aa16e7d 100644 --- a/shared.mak +++ b/shared.mak @@ -68,6 +68,7 @@ ifndef V QUIET_HDR = @echo ' ' HDR $(<:hcc=h); QUIET_RC = @echo ' ' RC $@; QUIET_SPATCH = @echo ' ' SPATCH $<; + QUIET_CHECK = @echo ' ' CHECK $@; ## Used in "Documentation/Makefile" QUIET_ASCIIDOC = @echo ' ' ASCIIDOC $@; From patchwork Thu Apr 21 18:23:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12822178 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2F04EC433F5 for ; Thu, 21 Apr 2022 18:24:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1391336AbiDUS07 (ORCPT ); Thu, 21 Apr 2022 14:26:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47168 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1391289AbiDUS0i (ORCPT ); Thu, 21 Apr 2022 14:26:38 -0400 Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 17F814B43E for ; Thu, 21 Apr 2022 11:23:45 -0700 (PDT) Received: by mail-wm1-x32b.google.com with SMTP id r187-20020a1c44c4000000b0038ccb70e239so6418340wma.3 for ; Thu, 21 Apr 2022 11:23:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=yDi1cqrfLPkuxHUYiccz2SbD/EgDesc5OV8tTagblpY=; b=lnVewjLSsFC4bL5xm8XptUmpFguVSabtrgJpnGax/UJyKj8lsvOI+LczfpiaIshw+B Ybzb0S0QYc7gtfrahN2upDWBFAZd8xwYbK/stp3R4UtBw4WmlNAo3tVwaf0/FF6xUWHW tett/kE+MTXfJVFZtO6ffyGnpOqSDuEDPPGQrEvC22Z5sRbOrtPYXy8/Bz8Y/O2b1NtI y8jSq/HPvSKP6lAl5kryKUt3Xzv2QJuYBtQ1Re8pnNUlyTpzO2TsNoxo+Xq8iDKoQpaG GVSaD12eASeGaK/nOxwj6t/w4mLnw8razlkvLWhSmYNR98kcXxQgGNaJ58xSMZgdkDdx HZjA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=yDi1cqrfLPkuxHUYiccz2SbD/EgDesc5OV8tTagblpY=; b=iEg0wf1/TVJPkF7/pGZszQns3PPTZP41Ch/p+P18Zj+SGbVaYwaYTgegHfJKS6iLkr H0s74mwLm7t6AOpPpCi3jkkMUx5VXXt81LgVOS6awlw2RoKHf95DZCmptGjd5+85tLzB D8dhwBBytK34XWl2iqicK/46PAXLMkCuWJHu4D9qdVcHwkW9s9ytvbxpn8rL69j2Z+Ph SWPo16CbOGysWMnahNPdWyO+bQCiHgtSgLBz9V8Rpdsr13WwGnbLKSyca3ncDbfQoa9I 6MyF49UtwYB7nsRi0O9F5PnUz9dctRUFMUis8NJ4bMsI8uZAS8vfpzQD+L7a2STLPIwq WoRw== X-Gm-Message-State: AOAM532r11Q50JagMnWSkFNx+aeSjmdWlhzOhgvGZZwMNJu8UZY/8u85 wZ+aXeWGZUhpbMrthbOebLrcrOqniwv28Q== X-Google-Smtp-Source: ABdhPJxYRGK2MDZykmfEBWIr0eUpUhuUJuVjM7XV6RBl7qslfFQx80Kr2m92MQq2R+VESaVHb95gfw== X-Received: by 2002:a7b:c119:0:b0:38e:d23c:1743 with SMTP id w25-20020a7bc119000000b0038ed23c1743mr613326wmi.28.1650565423412; Thu, 21 Apr 2022 11:23:43 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k11-20020a5d6d4b000000b0020599079f68sm2909090wri.106.2022.04.21.11.23.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Apr 2022 11:23:42 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , Eric Sunshine , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBC?= =?utf-8?b?amFybWFzb24=?= Subject: [PATCH v5 09/29] CI: have "static-analysis" run "check-builtins", not "documentation" Date: Thu, 21 Apr 2022 20:23:04 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.879.g3659959fcca In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Move the "make check-builtins" check from the "documentation" job to the "static-analysis" job. The "check-builtins" target added in c74390e4a1d (cherry is built-in, do not ship git-cherry.sh, 2006-11-05) is unrelated to the documentation, so it's odd that b98712b9aa9 (travis-ci: build documentation, 2016-05-04) added it to the "documentation" job. Let's just move it to the "ci-static-analysis" target, and while we're at it improve the output with $(QUIET_CHECK). Signed-off-by: Ævar Arnfjörð Bjarmason --- Makefile | 3 ++- ci/test-documentation.sh | 1 - 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Makefile b/Makefile index 8717ea7e745..2e1ae6c6067 100644 --- a/Makefile +++ b/Makefile @@ -3402,7 +3402,7 @@ check-docs:: ### Make sure built-ins do not have dups and listed in git.c # check-builtins:: - ./check-builtins.sh + $(QUIET_CHECK)./check-builtins.sh ### Test suite coverage testing # @@ -3492,5 +3492,6 @@ ci-check-directional-formatting: .PHONY: ci-static-analysis ci-static-analysis: ci-check-directional-formatting +ci-static-analysis: check-builtins ci-static-analysis: check-coccicheck ci-static-analysis: hdr-check diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh index 9e0652c30dd..41e2b126311 100755 --- a/ci/test-documentation.sh +++ b/ci/test-documentation.sh @@ -14,7 +14,6 @@ filter_log () { "$1" } -make check-builtins make check-docs # Build docs with AsciiDoc From patchwork Thu Apr 21 18:23:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12822176 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BFDC5C433F5 for ; Thu, 21 Apr 2022 18:24:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1391332AbiDUS0t (ORCPT ); Thu, 21 Apr 2022 14:26:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47128 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1391312AbiDUS0i (ORCPT ); Thu, 21 Apr 2022 14:26:38 -0400 Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com [IPv6:2a00:1450:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 161EC4AE3F for ; Thu, 21 Apr 2022 11:23:46 -0700 (PDT) Received: by mail-wr1-x42d.google.com with SMTP id s25so2954503wrb.8 for ; Thu, 21 Apr 2022 11:23:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=6LDBL5d9Z4CT6LF1/mindoM3/poAAlg7z5IziHQyOjw=; b=TU41YEdCoFd9DSCmQycsNrGu7R1+ivoRW0zakvZvxMi9jgnfB2j7J83yYXh+e5+fO+ gTMV3wKC1VfY9ClDW2RXrk/ldJcRxX1gnPYFgcsK+Dx4sVrttT9ttFwaZw6ujiSOoBNa bmMuQl05kh95Y8ovRYfS+SiUxhddgL4TSjm4jI2BqiiVbdtzvUYjUMtdpl28VSb2dOaj D+12f+bd7vHJ+KlDMpKBrXzIYEqYL08zsr+flwVxN5SfNvXWkG2/YV2mS6GyimOXesQ0 9NGLADcftpTX40j4FPFKzzrwCV85zgda6+/DVmvFz5CAm024lNOReKBUqJwAGy921gD4 jDQg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=6LDBL5d9Z4CT6LF1/mindoM3/poAAlg7z5IziHQyOjw=; b=PqmdIwOcS5e5meDrMZjeCr25agJ0BdsZMbjNbdzUGoSjKqdgdySdA1rkKCG89XIpFZ +U6Ud1ejC6NtIEjGmj6hFm69bdbXlUSHBq1x5wBGc+dB5Kp7T3LHEdT7N3m75TzAziwH cprlq8LXy3MPPIi2KtiDSArM1DQIg4RQ3HZPn5/AM58ZwHhaOKC/NdfBtM/CpKRcmE59 kKxUbuyaqu5BdiZJOc9++vT8OksMEsmNb99fS79nirltzJ2EOjJwAz2RfgqjMJj9+0QS 3jzaKwdCgN+pyrWksLvQV7wkcFDzWpgOTVeV4NhawcFISw23+v+VVpeuwlCVf6QihlXK 1Zig== X-Gm-Message-State: AOAM531RBGecQpRsUWjjdwIrAFnp0TsJQXvKVkQIx0K3H4+lgcAuHTEl s04/6jEMZZGyM9Lg3kVHUjcoEvw+S7Ln9A== X-Google-Smtp-Source: ABdhPJwzFIG0yTOo3YpgkPID5JDe2tz+YTxOHgWrD2dMyYvpZCC80RKnnVnLdv3mTgtbag/DTqyN3g== X-Received: by 2002:a5d:5707:0:b0:20a:c768:bc8 with SMTP id a7-20020a5d5707000000b0020ac7680bc8mr319564wrv.565.1650565424367; Thu, 21 Apr 2022 11:23:44 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k11-20020a5d6d4b000000b0020599079f68sm2909090wri.106.2022.04.21.11.23.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Apr 2022 11:23:43 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , Eric Sunshine , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBC?= =?utf-8?b?amFybWFzb24=?= Subject: [PATCH v5 10/29] CI: move p4 and git-lfs variables to ci/install-dependencies.sh Date: Thu, 21 Apr 2022 20:23:05 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.879.g3659959fcca In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Move the declaration of variables that are only used by the "ubuntu-latest" block in "ci/install-dependencies.sh" there from "ci/lib.sh". This makes the code easier to follow, and changes "ci/lib.sh" to a library that only exports CI variables for general use, we didn't need to export these $P4_PATH and $GIT_LFS_PATH variables. Signed-off-by: Ævar Arnfjörð Bjarmason --- ci/install-dependencies.sh | 17 +++++++++++++++-- ci/lib.sh | 12 ------------ 2 files changed, 15 insertions(+), 14 deletions(-) diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh index e7ea8799411..34b4400f38b 100755 --- a/ci/install-dependencies.sh +++ b/ci/install-dependencies.sh @@ -5,14 +5,27 @@ . ${0%/*}/lib.sh -P4WHENCE=http://filehost.perforce.com/perforce/r$LINUX_P4_VERSION -LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION UBUNTU_COMMON_PKGS="make libssl-dev libcurl4-openssl-dev libexpat-dev tcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl libemail-valid-perl libio-socket-ssl-perl libnet-smtp-ssl-perl" case "$runs_on_pool" in ubuntu-latest) + # The Linux build installs the defined dependency versions below. + # The OS X build installs much more recent versions, whichever + # were recorded in the Homebrew database upon creating the OS X + # image. + # Keep that in mind when you encounter a broken OS X build! + LINUX_P4_VERSION="16.2" + LINUX_GIT_LFS_VERSION="1.5.2" + + P4_PATH="$HOME/custom/p4" + GIT_LFS_PATH="$HOME/custom/git-lfs" + export PATH="$GIT_LFS_PATH:$P4_PATH:$PATH" + + P4WHENCE=http://filehost.perforce.com/perforce/r$LINUX_P4_VERSION + LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION + sudo apt-get -q update sudo apt-get -q -y install language-pack-is libsvn-perl apache2 \ $UBUNTU_COMMON_PKGS $CC_PACKAGE diff --git a/ci/lib.sh b/ci/lib.sh index edf26984aeb..3ceb36c7058 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -67,18 +67,6 @@ ubuntu-latest) fi export GIT_TEST_HTTPD=true - - # The Linux build installs the defined dependency versions below. - # The OS X build installs much more recent versions, whichever - # were recorded in the Homebrew database upon creating the OS X - # image. - # Keep that in mind when you encounter a broken OS X build! - export LINUX_P4_VERSION="16.2" - export LINUX_GIT_LFS_VERSION="1.5.2" - - P4_PATH="$HOME/custom/p4" - GIT_LFS_PATH="$HOME/custom/git-lfs" - export PATH="$GIT_LFS_PATH:$P4_PATH:$PATH" ;; macos-latest) if [ "$jobname" = osx-gcc ] From patchwork Thu Apr 21 18:23:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12822177 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 99C8EC433F5 for ; Thu, 21 Apr 2022 18:24:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1391334AbiDUS04 (ORCPT ); Thu, 21 Apr 2022 14:26:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47094 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1391313AbiDUS0i (ORCPT ); Thu, 21 Apr 2022 14:26:38 -0400 Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com [IPv6:2a00:1450:4864:20::430]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0CA1D4AE38 for ; Thu, 21 Apr 2022 11:23:47 -0700 (PDT) Received: by mail-wr1-x430.google.com with SMTP id u3so7857617wrg.3 for ; Thu, 21 Apr 2022 11:23:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=p94WjbMGHOc2dL0HHWYI6XlNlZZtXEyEIXxT07L8HJs=; b=VAR37CQ5yqCiTl9KpS6ym+a1LaVQHDeovV9/FuOpnOXiICES9ack/Cqvznao+ROUeK 1lEhUoNyT1kirxGEsFlRaqGJlxVypP2rnCu8vTvQn/uZrxj1Tx+51wZg0wmcc5wsdlfx saYEQM2PAtgRjQqOkPpx+ua3dbjVFAkhn9m6N8WX9T8Ibm68vaineqtS1dX/R6zDrGBR 9em6rOylQVdGSGNXZawo7tFiAEfxp1ZcB6Sf9RZfzyE1zanTAnxjvHeX0GVi3b2JAuhn DMAqAp0v3/fwXWFnhEcRi2JPBl4jnIeDhf71+rbyyasRfZYXTo0RuGIIXOjmnZmDFh2p o1AA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=p94WjbMGHOc2dL0HHWYI6XlNlZZtXEyEIXxT07L8HJs=; b=ER6FBRfhO+wCJ6KpDuK7S06VZAJjqIFfwzazIyPo44lAtgUnkkSmFHPzMYRGKoF1ax IShhNz4RmqgCvIIRtXttupC/potuu5MpoF2RbYAH9E35qRt4RSXbVAi2h7ZxkhRPbcuY 4nVbKTVM+uxuKMjo+sStfk3Fc/k8sV1LIm1F7BJEyrYvCHHWufFH2SwSyxK66vr7lo1l 9YH9M3SV3QLawbAIZnFpHrX5y3nKjMUw528qegiczPf33Hvovhq1x2y7inXtF1A4ad7r TnhLXM4RsLdz2dLl2zTiWwCYZRHfETrxeGB8ix0euRG0fGx2lH6tF24ORl+cYYjoIr1k Vvbw== X-Gm-Message-State: AOAM532HthK4Chbw206wX6S3cy0dePCsrPFETpYbt/QlU2O7+1X0zHez y257I5oajWqk56uPFTwdrRpuCmiHt4u4IA== X-Google-Smtp-Source: ABdhPJxmjx0eJMxFHV4fC/XcNTnIjz4cs+FxPzWRgfYwaEusXusFKcJErVqRoVv+S90ppqI7mBnMlQ== X-Received: by 2002:a5d:6da9:0:b0:20a:8d08:9977 with SMTP id u9-20020a5d6da9000000b0020a8d089977mr747545wrs.24.1650565425342; Thu, 21 Apr 2022 11:23:45 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k11-20020a5d6d4b000000b0020599079f68sm2909090wri.106.2022.04.21.11.23.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Apr 2022 11:23:44 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , Eric Sunshine , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBC?= =?utf-8?b?amFybWFzb24=?= Subject: [PATCH v5 11/29] CI: consistently use "export" in ci/lib.sh Date: Thu, 21 Apr 2022 20:23:06 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.879.g3659959fcca In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Change the "ci/lib.sh" script co consistently use "export", for e.g. MAKEFLAGS we were exporting it, and then assigning to it, let's do it the other way around. Right now this doesn't matter, since we in e.g. "ci/install-dependencies.sh" source this file, and don't use something like "env(1)" to retrieve these variables. But in a subsequent commit we'll "export" these variables through a wrapper (to additionally write them to a GitHub CI-specific $GITHUB_ENV file). This change makes that subsequent change easier to read, as it won't need to do any control flow refactoring. Signed-off-by: Ævar Arnfjörð Bjarmason --- ci/lib.sh | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/ci/lib.sh b/ci/lib.sh index 3ceb36c7058..2e425cb787c 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -30,7 +30,7 @@ export TERM=${TERM:-dumb} NPROC=10 # Clear MAKEFLAGS that may come from the outside world. -export MAKEFLAGS=--jobs=$NPROC +MAKEFLAGS=--jobs=$NPROC if test "$GITHUB_ACTIONS" = "true" then @@ -38,9 +38,10 @@ then CC="${CC:-gcc}" export GIT_PROVE_OPTS="--timer --jobs $NPROC" - export GIT_TEST_OPTS="--verbose-log -x" + GIT_TEST_OPTS="--verbose-log -x" test Windows != "$RUNNER_OS" || GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS" + export GIT_TEST_OPTS else echo "Could not identify CI type" >&2 env >&2 @@ -94,4 +95,4 @@ linux-leaks) ;; esac -MAKEFLAGS="$MAKEFLAGS CC=${CC:-cc}" +export MAKEFLAGS="$MAKEFLAGS CC=${CC:-cc}" From patchwork Thu Apr 21 18:23:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12822179 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E06B9C433FE for ; Thu, 21 Apr 2022 18:24:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1391338AbiDUS1B (ORCPT ); Thu, 21 Apr 2022 14:27:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47084 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1391253AbiDUS0i (ORCPT ); Thu, 21 Apr 2022 14:26:38 -0400 Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 16BA94B1D9 for ; Thu, 21 Apr 2022 11:23:48 -0700 (PDT) Received: by mail-wr1-x431.google.com with SMTP id w4so7804901wrg.12 for ; Thu, 21 Apr 2022 11:23:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=DswBgY57DvonvV+IlvDXZDkdUDSKRitcg2pnXHgFQQg=; b=FUyR3c4L8dc5NoTjxbfSdmNZGe9xelCEftyapVp1ATc7odeualdArywZK6uAwgP8hV dRv+Vj4dgjwb1Hpix5oor4rZl4x10UvZ4cWlXdMDc+MhBx4h92r5MmSo1rfwXkvYOKPW 2QRvR6f7NTieFrOcddAkxa8uzhx7dujwyGLTMtPwUp2aAAtuEjRQuQauFptJ5+2QknNS T/qBtX/H1GR3bCJOahkygcTYOkxN4xk5hH6L3BOOJOWuPqITFK5A+ghw5iUScoOZCoJM o446xc1X+asMvkfF/cC96Kqk5NFNHSFR36+r771AtnEDvAcaa+i+q/LH7EyhYoXgWtof a3ug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=DswBgY57DvonvV+IlvDXZDkdUDSKRitcg2pnXHgFQQg=; b=V1QbRM/FjtBVVgDWnGEeZ4bvI7uhY4t0U/RoZ+QFKCu2h+oXZ1ZzgYD8yiaYlzORzr AQKP/fodQGd/z+SEe98cBLRfSPSXiDLMjXs1qqhIE8pZewp+B7bR4fqEmYKn+EZoz2lO V9zJvuXPZ372Bz0fduH937/ajw5sf44s6UBJzQcCJ4vfOaquDz+Cj5hctezLuhOoWh0h BTRTWc6oH27BQQbcrkTqNhmV6HgKUy/cyISHDMZ0hDtFWH01456FocIFqskMTYaSdiYq bbJfW3nbfVt5pr3l+Vh/JUTQySsx47iKrIiXjz8UppddaCpDnDw3PZm8GYIQp2kA45Jy Rl0w== X-Gm-Message-State: AOAM533/v0ZliZI1hnzL3RY80OS9esY0Xbk7BikNNXu2mZsJr/kx4xr5 Eh0I+5oiBUU0bx3Q7azzzv7w+CRzfzVokQ== X-Google-Smtp-Source: ABdhPJzYk05UuX2mUxLitZHL+oaLrMbkpbElKmf+hY917Fi0FApKzWsILaQ0uoI2Tcp18G2DB/+cQg== X-Received: by 2002:a05:6000:2c9:b0:20a:8f9f:4a08 with SMTP id o9-20020a05600002c900b0020a8f9f4a08mr719848wry.297.1650565426331; Thu, 21 Apr 2022 11:23:46 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k11-20020a5d6d4b000000b0020599079f68sm2909090wri.106.2022.04.21.11.23.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Apr 2022 11:23:45 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , Eric Sunshine , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBC?= =?utf-8?b?amFybWFzb24=?= Subject: [PATCH v5 12/29] CI: export variables via a wrapper Date: Thu, 21 Apr 2022 20:23:07 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.879.g3659959fcca In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Add and use a "setenv" wrapper to export the variables set in ci/lib.sh. For now this merely adds debugging information about what variables are set, but in a subsequent commit we'll expand this to set variables in $GITHUB_ENV instead of the environment, and thus carry variable settings between steps in a job. We'll also eventually be interested in what type of variable we're setting, and when it needs to be used, because we'll prominently expose what variables we set in what CI "step", so having only the ones we actually use will improve readability. To that end call the "setenv" function with optional "--build", "--test" and "--all" parameters to indicate whether they're needed for building, testing or both. We currently ignore the "varname" parameter, but will make use of it in a subsequent commit. Adding it here reduces the needless churn, as we won't need to change all these lines again for that subsequent change. Signed-off-by: Ævar Arnfjörð Bjarmason --- ci/lib.sh | 51 ++++++++++++++++++++++++++++++--------- ci/run-build-and-tests.sh | 34 +++++++++++++------------- 2 files changed, 57 insertions(+), 28 deletions(-) diff --git a/ci/lib.sh b/ci/lib.sh index 2e425cb787c..e36e350e74c 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -13,6 +13,35 @@ then exit 1 fi +# Helper functions +setenv () { + while test $# != 0 + do + case "$1" in + --build) + ;; + --test) + ;; + --all) + ;; + -*) + echo "BUG: bad setenv() option '$1'" >&2 + exit 1 + ;; + *) + break + ;; + esac + shift + done + + key=$1 + val=$2 + shift 2 + + eval "export $key=\"$val\"" +} + check_unignored_build_artifacts () { ! git ls-files --other --exclude-standard --error-unmatch \ @@ -24,7 +53,7 @@ check_unignored_build_artifacts () } # GitHub Action doesn't set TERM, which is required by tput -export TERM=${TERM:-dumb} +setenv TERM ${TERM:-dumb} # How many jobs to run in parallel? NPROC=10 @@ -37,21 +66,21 @@ then CI_TYPE=github-actions CC="${CC:-gcc}" - export GIT_PROVE_OPTS="--timer --jobs $NPROC" + setenv --test GIT_PROVE_OPTS "--timer --jobs $NPROC" GIT_TEST_OPTS="--verbose-log -x" test Windows != "$RUNNER_OS" || GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS" - export GIT_TEST_OPTS + setenv --test GIT_TEST_OPTS "$GIT_TEST_OPTS" else echo "Could not identify CI type" >&2 env >&2 exit 1 fi -export DEVELOPER=1 -export DEFAULT_TEST_TARGET=prove -export GIT_TEST_CLONE_2GB=true -export SKIP_DASHED_BUILT_INS=YesPlease +setenv --build DEVELOPER 1 +setenv --test DEFAULT_TEST_TARGET prove +setenv --test GIT_TEST_CLONE_2GB true +setenv --build SKIP_DASHED_BUILT_INS YesPlease case "$runs_on_pool" in ubuntu-latest) @@ -67,7 +96,7 @@ ubuntu-latest) MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=/usr/bin/python2" fi - export GIT_TEST_HTTPD=true + setenv --test GIT_TEST_HTTPD true ;; macos-latest) if [ "$jobname" = osx-gcc ] @@ -90,9 +119,9 @@ linux-musl) MAKEFLAGS="$MAKEFLAGS GIT_TEST_UTF8_LOCALE=C.UTF-8" ;; linux-leaks) - export SANITIZE=leak - export GIT_TEST_PASSING_SANITIZE_LEAK=true + setenv --build SANITIZE leak + setenv --test GIT_TEST_PASSING_SANITIZE_LEAK true ;; esac -export MAKEFLAGS="$MAKEFLAGS CC=${CC:-cc}" +setenv --all MAKEFLAGS "$MAKEFLAGS CC=${CC:-cc}" diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh index e5a21985b44..35d45a9373e 100755 --- a/ci/run-build-and-tests.sh +++ b/ci/run-build-and-tests.sh @@ -9,33 +9,33 @@ export MAKE_TARGETS="all test" case "$jobname" in linux-gcc) - export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main + setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main ;; linux-TEST-vars) - export GIT_TEST_SPLIT_INDEX=yes - export GIT_TEST_MERGE_ALGORITHM=recursive - export GIT_TEST_FULL_IN_PACK_ARRAY=true - export GIT_TEST_OE_SIZE=10 - export GIT_TEST_OE_DELTA_SIZE=5 - export GIT_TEST_COMMIT_GRAPH=1 - export GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS=1 - export GIT_TEST_MULTI_PACK_INDEX=1 - export GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP=1 - export GIT_TEST_ADD_I_USE_BUILTIN=1 - export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=master - export GIT_TEST_WRITE_REV_INDEX=1 - export GIT_TEST_CHECKOUT_WORKERS=2 + setenv --test GIT_TEST_SPLIT_INDEX yes + setenv --test GIT_TEST_MERGE_ALGORITHM recursive + setenv --test GIT_TEST_FULL_IN_PACK_ARRAY true + setenv --test GIT_TEST_OE_SIZE 10 + setenv --test GIT_TEST_OE_DELTA_SIZE 5 + setenv --test GIT_TEST_COMMIT_GRAPH 1 + setenv --test GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS 1 + setenv --test GIT_TEST_MULTI_PACK_INDEX 1 + setenv --test GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP 1 + setenv --test GIT_TEST_ADD_I_USE_BUILTIN 1 + setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME master + setenv --test GIT_TEST_WRITE_REV_INDEX 1 + setenv --test GIT_TEST_CHECKOUT_WORKERS 2 ;; linux-clang) - export GIT_TEST_DEFAULT_HASH=sha1 + setenv --test GIT_TEST_DEFAULT_HASH sha1 ;; linux-sha256) - export GIT_TEST_DEFAULT_HASH=sha256 + setenv --test GIT_TEST_DEFAULT_HASH sha256 ;; pedantic) # Don't run the tests; we only care about whether Git can be # built. - export DEVOPTS=pedantic + setenv --build DEVOPTS pedantic export MAKE_TARGETS=all ;; esac From patchwork Thu Apr 21 18:23:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12822183 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 733E3C433F5 for ; Thu, 21 Apr 2022 18:24:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1391326AbiDUS1T (ORCPT ); Thu, 21 Apr 2022 14:27:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47084 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1391298AbiDUS0l (ORCPT ); Thu, 21 Apr 2022 14:26:41 -0400 Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9897E4AE14 for ; Thu, 21 Apr 2022 11:23:49 -0700 (PDT) Received: by mail-wr1-x432.google.com with SMTP id e2so1563039wrh.7 for ; Thu, 21 Apr 2022 11:23:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=AcAjaHTUSDRHJ5iMmjQz4+KkbYnqbeoIjeiRBAbmf48=; b=UxI0AHLvY22P+aZ9ACLM0+X3m58rV3FiTgh2QjDbHtavWCq+v1wbCyOWckbCFMQ5zc jaxuyU2FmWtdmgKGif7sRUbd5QAY2t0GKcXPv/cFRaiEmyDMfBFX6SqeC9OxeG2mXzK/ lQxRcTdq0vJ1y2WgBDGahGcxjglELhcnuAAS0eLQxHyL2ATqpsXtBehzsXbn5ehOid98 +gTopPjJOFKcNrj4RV2Go3W0/kH5oOWNW3xjYCesKkG0zyv4EDqV+2zByQtOL6XECngS RuUQDVlsBd2lv8iIiKZ5umBxs8uIKgoyiQBoDodm42Hn5OnH/XfDC38JnNcnZtiz7uAr aXSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=AcAjaHTUSDRHJ5iMmjQz4+KkbYnqbeoIjeiRBAbmf48=; b=kz0Fx6N0q+Qgdicx1Ts0t2YlXaClOx/Auo2QZ9lqNdzQFQ/mwtKDo6DiueVSQV7SlL 4hqLvJ+rNmsGyc8vjbEShLx6j8PjlROy3rD4xYtcrDNhq/qSfqKWpQ4HcrOTXKOPnOx+ 4aarAW5kF6Ig28Zb7SJfoECpL8um6lj+SFGY0PJh4UWDbZiwZpoIMeMYhO+uv+De1/We A1Xk99GGkhXfsyJVK3xkzXOFo4eIeBDcSdoqLHsITFwYHV5oH45q49/FQ86Wi2pjdSr4 odQAOnmscvIRoNvLwKQEXfCK8qb6wSwdE6y3GeP1t2xu/MZevcgOOFE4UsKbTPzO5jLQ M84Q== X-Gm-Message-State: AOAM5305QDjFNbwk5BCAYDIP7X77Fcz/0OGQWOWfvaW9oA23gFBTpIh/ QzMcPoXWMkfL0x0qnqAwAhcQfqtDJhbU0A== X-Google-Smtp-Source: ABdhPJxpbH/FZUzgw9AUF9ijTCOfGR0cJMiQYXO5IIJ7PUjTTgJOOY3jPsybQmAHfUxboLzrSiyRwg== X-Received: by 2002:a5d:6208:0:b0:203:dde4:c76e with SMTP id y8-20020a5d6208000000b00203dde4c76emr726854wru.273.1650565427420; Thu, 21 Apr 2022 11:23:47 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k11-20020a5d6d4b000000b0020599079f68sm2909090wri.106.2022.04.21.11.23.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Apr 2022 11:23:46 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , Eric Sunshine , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBC?= =?utf-8?b?amFybWFzb24=?= Subject: [PATCH v5 13/29] CI: remove "run-build-and-tests.sh", run "make [test]" directly Date: Thu, 21 Apr 2022 20:23:08 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.879.g3659959fcca In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Remove the already thin "ci/run-build-and-tests.sh" wrapper and instead make the CI run "make" or "make test" directly. By doing this we'll be able to easily see at a glance whether our failure was in the compilation or testing, whether that's via human eyes or improve machine readability. We also need to run our new "ci/check-unignored-build-artifacts.sh" on success() in the CI now, just like we already had a step conditional on failure() running ci/print-test-failures.sh. The reason we used a "ci/run-build-and-tests.sh" wrapper in the first place had to do with Travis CI-specific constraints that no longer apply to us, as the Travis CI support has been removed. Instead we can configure the CI in an earlier step by running "ci/lib.sh", which under GitHub CI will write the environment variables we need to the "$GITHUB_ENV" file. We'll then have access to them in subsequent steps, and crucially those variables will be prominently visible at the start of each step via an expandable drop-down in the UI.drop-do. I.e. this changes the CI run from a top-down flow like (pseudocode): - job: - step1: - use ci/lib.sh to set env vars - run a script like ci/run-build-and-tests.sh - step2: - if: failure() - use ci/lib.sh to set env vars - run ci/print-test-failures.sh To: - job: - step1: - set variables in $GITHUB_ENV using ci/lib.sh - step2: - make - step3: - make test - step4: - if: failure() - run ci/print-test-failures.sh - step5: - if: success() - run ci/check-unignored-build-artifacts.sh There is a proposal[2] to get some of the benefits of this approach by not re-arranging our variable setup in this way, but to instead use the GitHub CI grouping syntax to focus on the relevant parts of "make" or "make test" when we have failures. Doing it this way makes for better looking GitHub CI UI, and lays much better ground work for our CI going forward. Because: * The CI logic will be more portable to a future CI system, since a common feature of them is to run various commands in sequence, but a future system won't necessarily support the GitHub-specifics syntax of "grouping" output within a "step". Even if those systems don't support a "$GITHUB_ENV" emulating will be much easier than to deal with some CI-specific grouping syntax. * At the start of every step the GitHub CI presents an expandable list of environment variables from "$GITHUB_ENV". We'll now see exactly what variables affected that step (although we currently overshoot that a bit, and always define all variables). * CI failures will be easier to reproduce locally, as this makes the relevant ci/* scripts something that sets up our environment, but leaves "make" and "make test" working as they do locally. To reproduce a run the user only needs to set the variables discussed in the drop-down above, either manually or by running "ci/lib.sh". * The output will be less verbose. The "ci/lib.sh" script uses "set -x", and before this e.g. "ci/static-analysis.sh" would start with 40 lines of trace output, culminating in using "export" to export the relevant environment variables. Now that verbosity is in the earlier "ci/lib.sh" step, and not in any subsequent one. The "make" targets then start out with the relevant non-trace output right away. * If we do want to use the grouping syntax within a "step" it'll now be easier to do so. It doesn't support nesting, so we'd have to make a choice between using it for e.g. "make" v.s. "make test", or individual test failures. See "sadly" in [3]. 1. https://lore.kernel.org/git/211120.86k0h30zuw.gmgdl@evledraar.gmail.com/ 2. https://lore.kernel.org/git/pull.1117.git.1643050574.gitgitgadget@gmail.com/ 3. https://lore.kernel.org/git/9333ba781b8240f704e739b00d274f8c3d887e39.1643050574.git.gitgitgadget@gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason --- .github/workflows/main.yml | 14 ++++++-- ci/check-unignored-build-artifacts.sh | 8 +++++ ci/install-dependencies.sh | 4 +++ ci/lib.sh | 36 +++++++++++++++++++- ci/make-test-artifacts.sh | 2 -- ci/run-build-and-tests.sh | 47 --------------------------- ci/run-test-slice.sh | 2 -- ci/test-documentation.sh | 1 - 8 files changed, 58 insertions(+), 56 deletions(-) create mode 100755 ci/check-unignored-build-artifacts.sh delete mode 100755 ci/run-build-and-tests.sh diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index ad752010102..57b2db06923 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -268,7 +268,10 @@ jobs: steps: - uses: actions/checkout@v2 - run: ci/install-dependencies.sh - - run: ci/run-build-and-tests.sh + - run: ci/lib.sh + - run: make + - run: make test + if: success() - run: ci/print-test-failures.sh if: failure() - name: Upload failed tests' directories @@ -292,6 +295,7 @@ jobs: image: daald/ubuntu32:xenial - jobname: pedantic image: fedora + skip-tests: yes env: jobname: ${{matrix.vector.jobname}} runs-on: ubuntu-latest @@ -299,9 +303,12 @@ jobs: steps: - uses: actions/checkout@v1 - run: ci/install-docker-dependencies.sh - - run: ci/run-build-and-tests.sh + - run: ci/lib.sh + - run: make + - run: make test + if: success() && matrix.vector.skip-tests != 'yes' - run: ci/print-test-failures.sh - if: failure() + if: failure() && matrix.vector.skip-tests != 'yes' - name: Upload failed tests' directories if: failure() && env.FAILED_TEST_ARTIFACTS != '' uses: actions/upload-artifact@v1 @@ -317,6 +324,7 @@ jobs: steps: - uses: actions/checkout@v2 - run: ci/install-dependencies.sh + - run: ci/lib.sh - run: make ci-static-analysis sparse: needs: ci-config diff --git a/ci/check-unignored-build-artifacts.sh b/ci/check-unignored-build-artifacts.sh new file mode 100755 index 00000000000..56d04b0db9a --- /dev/null +++ b/ci/check-unignored-build-artifacts.sh @@ -0,0 +1,8 @@ +#!/bin/sh +# +# Check whether the build created anything not in our .gitignore +# + +. ${0%/*}/lib.sh + +check_unignored_build_artifacts diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh index 34b4400f38b..01006379fd0 100755 --- a/ci/install-dependencies.sh +++ b/ci/install-dependencies.sh @@ -22,6 +22,10 @@ ubuntu-latest) P4_PATH="$HOME/custom/p4" GIT_LFS_PATH="$HOME/custom/git-lfs" export PATH="$GIT_LFS_PATH:$P4_PATH:$PATH" + if test -n "$GITHUB_PATH" + then + echo "$PATH" >>"$GITHUB_PATH" + fi P4WHENCE=http://filehost.perforce.com/perforce/r$LINUX_P4_VERSION LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION diff --git a/ci/lib.sh b/ci/lib.sh index e36e350e74c..43231862aa5 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -1,3 +1,5 @@ +#!/bin/sh + # Library of functions shared by all CI scripts # Set 'exit on error' for all CI scripts to let the caller know that @@ -39,7 +41,10 @@ setenv () { val=$2 shift 2 - eval "export $key=\"$val\"" + if test -n "$GITHUB_ENV" + then + echo "$key=$val" >>"$GITHUB_ENV" + fi } check_unignored_build_artifacts () @@ -109,6 +114,35 @@ macos-latest) esac case "$jobname" in +linux-gcc) + setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main + ;; +linux-TEST-vars) + setenv --test GIT_TEST_SPLIT_INDEX yes + setenv --test GIT_TEST_MERGE_ALGORITHM recursive + setenv --test GIT_TEST_FULL_IN_PACK_ARRAY true + setenv --test GIT_TEST_OE_SIZE 10 + setenv --test GIT_TEST_OE_DELTA_SIZE 5 + setenv --test GIT_TEST_COMMIT_GRAPH 1 + setenv --test GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS 1 + setenv --test GIT_TEST_MULTI_PACK_INDEX 1 + setenv --test GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP 1 + setenv --test GIT_TEST_ADD_I_USE_BUILTIN 1 + setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME master + setenv --test GIT_TEST_WRITE_REV_INDEX 1 + setenv --test GIT_TEST_CHECKOUT_WORKERS 2 + ;; +linux-clang) + setenv --test GIT_TEST_DEFAULT_HASH sha1 + ;; +linux-sha256) + setenv --test GIT_TEST_DEFAULT_HASH sha256 + ;; +pedantic) + # Don't run the tests; we only care about whether Git can be + # built. + setenv --build DEVOPTS pedantic + ;; linux32) CC=gcc ;; diff --git a/ci/make-test-artifacts.sh b/ci/make-test-artifacts.sh index 646967481f6..45298562982 100755 --- a/ci/make-test-artifacts.sh +++ b/ci/make-test-artifacts.sh @@ -8,5 +8,3 @@ mkdir -p "$1" # in case ci/lib.sh decides to quit early . ${0%/*}/lib.sh make artifacts-tar ARTIFACTS_DIRECTORY="$1" - -check_unignored_build_artifacts diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh deleted file mode 100755 index 35d45a9373e..00000000000 --- a/ci/run-build-and-tests.sh +++ /dev/null @@ -1,47 +0,0 @@ -#!/bin/sh -# -# Build and test Git -# - -. ${0%/*}/lib.sh - -export MAKE_TARGETS="all test" - -case "$jobname" in -linux-gcc) - setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main - ;; -linux-TEST-vars) - setenv --test GIT_TEST_SPLIT_INDEX yes - setenv --test GIT_TEST_MERGE_ALGORITHM recursive - setenv --test GIT_TEST_FULL_IN_PACK_ARRAY true - setenv --test GIT_TEST_OE_SIZE 10 - setenv --test GIT_TEST_OE_DELTA_SIZE 5 - setenv --test GIT_TEST_COMMIT_GRAPH 1 - setenv --test GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS 1 - setenv --test GIT_TEST_MULTI_PACK_INDEX 1 - setenv --test GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP 1 - setenv --test GIT_TEST_ADD_I_USE_BUILTIN 1 - setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME master - setenv --test GIT_TEST_WRITE_REV_INDEX 1 - setenv --test GIT_TEST_CHECKOUT_WORKERS 2 - ;; -linux-clang) - setenv --test GIT_TEST_DEFAULT_HASH sha1 - ;; -linux-sha256) - setenv --test GIT_TEST_DEFAULT_HASH sha256 - ;; -pedantic) - # Don't run the tests; we only care about whether Git can be - # built. - setenv --build DEVOPTS pedantic - export MAKE_TARGETS=all - ;; -esac - -# Any new "test" targets should not go after this "make", but should -# adjust $MAKE_TARGETS. Otherwise compilation-only targets above will -# start running tests. -make $MAKE_TARGETS -check_unignored_build_artifacts diff --git a/ci/run-test-slice.sh b/ci/run-test-slice.sh index b9a682b4bcd..70326961454 100755 --- a/ci/run-test-slice.sh +++ b/ci/run-test-slice.sh @@ -8,5 +8,3 @@ make --quiet -C t T="$(cd t && ./helper/test-tool path-utils slice-tests "$1" "$2" t[0-9]*.sh | tr '\n' ' ')" - -check_unignored_build_artifacts diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh index 41e2b126311..6b5cce03bd7 100755 --- a/ci/test-documentation.sh +++ b/ci/test-documentation.sh @@ -39,4 +39,3 @@ test -s Documentation/git.html grep ' X-Patchwork-Id: 12822184 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C487FC433F5 for ; Thu, 21 Apr 2022 18:24:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1391223AbiDUS1W (ORCPT ); Thu, 21 Apr 2022 14:27:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47262 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1377142AbiDUS0l (ORCPT ); Thu, 21 Apr 2022 14:26:41 -0400 Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com [IPv6:2a00:1450:4864:20::42a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 37A2F4AE3F for ; Thu, 21 Apr 2022 11:23:50 -0700 (PDT) Received: by mail-wr1-x42a.google.com with SMTP id w4so7805008wrg.12 for ; Thu, 21 Apr 2022 11:23:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=38g2rMJXcnBmbijAYyYrd1bW4p/cJtwMwJk7Rt695jk=; b=gRzXx27Y1Cy0aUp7m2myEOpxcK9WBhZxwpzQdGQN/RAHKhDw6a9uZFG51RikJpDyFo rCOSjeJ8a713eqhK8BBfidS3WhvMkmDsnfCCGHhzHmDJsM5RrGELwQALPGCTZIC4vyYU qtIiu4S+92mqfYwZdHUispghUeNkhw4RLK2r54Ekng1wEYx2Gyx91fWkZ9aXdnZ0zT+8 q2bVSR00DoJji4ixFKovw9YHXaSFx/w17H/syc0Y5PIibpiB7YU2ag7auyTpVL3tLWnl T2v0wuEJo4I7dCPdASu3bUZBlP5GrbDMrKDZQcwD+Q9ULJbvZyZ+Th/zmOP7mYJwxHnk RGZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=38g2rMJXcnBmbijAYyYrd1bW4p/cJtwMwJk7Rt695jk=; b=BnBzGwF0CxTsXMg8TOi7q2wO9OMfa0N6k2E+1Vpsu0mvJT4c2mV5tgTXC9gat0ojp+ iny37CeHoKGqyLT4FKQnXkpLSurxT7XCEj3eQggay2HvCB6GB/uvSx6mDPEgrAgIK/Se Z0kfAhdtvof09Athm8pTxBxAmxrr7lhg4y6atNHsQpJPOKCnrIygriskkmJ8bUVAqmhO QmOekBn+vSwywR0VVnJp0RSMaD4SW3xYUuSXUAks+G2oPFWVVaGisWirU5UG0ynhuKiw 9uZs29gJ6A10Vq63zWc7Bq4qHrFMeXFq+iTq9L8iM4Kl73alYfjfrPRF7L2jEFaW0nwU UXYw== X-Gm-Message-State: AOAM533d5bHcqD0QG+fQlQeLrlkdCL1+bhc5el8nuz3H5s54KI2jieHr mFmHeysgy5FtMZKsVxYfpx6U4pN/rywHWw== X-Google-Smtp-Source: ABdhPJx8FqaqngYmij96F32vtEu0Iyucf7ii9A/ugx7NVoFj9E4+CakrZr2dshcXe6nvJYK8FGsTrw== X-Received: by 2002:a05:6000:15c8:b0:20a:92f1:c054 with SMTP id y8-20020a05600015c800b0020a92f1c054mr779994wry.146.1650565428468; Thu, 21 Apr 2022 11:23:48 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k11-20020a5d6d4b000000b0020599079f68sm2909090wri.106.2022.04.21.11.23.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Apr 2022 11:23:47 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , Eric Sunshine , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBC?= =?utf-8?b?amFybWFzb24=?= Subject: [PATCH v5 14/29] CI: make ci/{lib,install-dependencies}.sh POSIX-compatible Date: Thu, 21 Apr 2022 20:23:09 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.879.g3659959fcca In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Change the unportable constructs in the "ci/install-dependencies.sh" scripts to use POSIX replacements, and have it run under "/bin/sh" instead of "/usr/bin/env bash". We needed to: * Replace pushd/popd with a sub-shell. * Change "mkdir --parents" to "mkdir -p". * Use POSIX "export" syntax. This is in preparation for combining this script and "ci/install-docker-dependencies.sh", which doesn't have access to "bash". While we're at it change "if [" to "if test" in ci/lib.sh. This isn't strictly speaking a case of POSIX-incompatibility, but that syntax was being used there because this code was using idiomatic bash constructs. Let's have it use our usual style of "if test" instead. Signed-off-by: Ævar Arnfjörð Bjarmason --- ci/install-dependencies.sh | 27 +++++++++++++++++---------- ci/lib.sh | 4 ++-- 2 files changed, 19 insertions(+), 12 deletions(-) diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh index 01006379fd0..1ad4dd7db39 100755 --- a/ci/install-dependencies.sh +++ b/ci/install-dependencies.sh @@ -1,4 +1,4 @@ -#!/usr/bin/env bash +#!/bin/sh # # Install dependencies required to build and test Git on Linux and macOS # @@ -21,7 +21,8 @@ ubuntu-latest) P4_PATH="$HOME/custom/p4" GIT_LFS_PATH="$HOME/custom/git-lfs" - export PATH="$GIT_LFS_PATH:$P4_PATH:$PATH" + PATH="$GIT_LFS_PATH:$P4_PATH:$PATH" + export PATH if test -n "$GITHUB_PATH" then echo "$PATH" >>"$GITHUB_PATH" @@ -33,22 +34,28 @@ ubuntu-latest) sudo apt-get -q update sudo apt-get -q -y install language-pack-is libsvn-perl apache2 \ $UBUNTU_COMMON_PKGS $CC_PACKAGE - mkdir --parents "$P4_PATH" - pushd "$P4_PATH" + mkdir -p "$P4_PATH" + ( + cd "$P4_PATH" wget --quiet "$P4WHENCE/bin.linux26x86_64/p4d" wget --quiet "$P4WHENCE/bin.linux26x86_64/p4" chmod u+x p4d chmod u+x p4 - popd - mkdir --parents "$GIT_LFS_PATH" - pushd "$GIT_LFS_PATH" + ) + mkdir -p "$GIT_LFS_PATH" + ( + cd "$GIT_LFS_PATH" wget --quiet "$LFSWHENCE/git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" tar --extract --gunzip --file "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" cp git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs . - popd + ) ;; macos-latest) - export HOMEBREW_NO_AUTO_UPDATE=1 HOMEBREW_NO_INSTALL_CLEANUP=1 + HOMEBREW_NO_AUTO_UPDATE=1 + export HOMEBREW_NO_AUTO_UPDATE + HOMEBREW_NO_INSTALL_CLEANUP=1 + export HOMEBREW_NO_INSTALL_CLEANUP + # Uncomment this if you want to run perf tests: # brew install gnu-time brew link --force gettext @@ -62,7 +69,7 @@ macos-latest) if test -n "$CC_PACKAGE" then - BREW_PACKAGE=${CC_PACKAGE/-/@} + BREW_PACKAGE=$(echo $CC_PACKAGE | tr '-' '@') brew install "$BREW_PACKAGE" brew link "$BREW_PACKAGE" fi diff --git a/ci/lib.sh b/ci/lib.sh index 43231862aa5..2fd022c94bc 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -94,7 +94,7 @@ ubuntu-latest) break fi - if [ "$jobname" = linux-gcc ] + if test "$jobname" = linux-gcc then MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=/usr/bin/python3" else @@ -104,7 +104,7 @@ ubuntu-latest) setenv --test GIT_TEST_HTTPD true ;; macos-latest) - if [ "$jobname" = osx-gcc ] + if test "$jobname" = osx-gcc then MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)" else From patchwork Thu Apr 21 18:23:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12822196 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DFB74C433EF for ; Thu, 21 Apr 2022 18:25:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1391393AbiDUS1c (ORCPT ); Thu, 21 Apr 2022 14:27:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47282 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1391238AbiDUS0l (ORCPT ); Thu, 21 Apr 2022 14:26:41 -0400 Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com [IPv6:2a00:1450:4864:20::435]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 29ABD4B1D9 for ; Thu, 21 Apr 2022 11:23:51 -0700 (PDT) Received: by mail-wr1-x435.google.com with SMTP id c10so7864083wrb.1 for ; Thu, 21 Apr 2022 11:23:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=wXDZk42LlS9I1d/3NLMgr7fvX5h+CpA70Pot+6x9Foo=; b=g3Wq2qzBb1Pz7RS7J1gSZNKNYbPjQFkHc+er1elVDgLbIYxSHOQLo5C7cO9XMmN7pL +kGJi55AenraME0r/DdInmdcJL1Shlfk5SxVHpDUszuyMw2R++DtUOeuBjCYrWXQ92sM rtb6zkl8zIYp467MoJn9mk5XxU85SORMFqAufCjlrz11sO+9bgA+TFUOVnIBwmY3Vp8b ML/LBl5cfUltZuQwEPeGjNwhnztQ7Xpa3oFnc8Gk2AYCjrzw+KYsVC2WKPVtJg93YxyC OzjSQeD6NuUbANMgdO53CGg5jAPvM1hxAbaEiRiJBaT+f9QmBoPeen7rkUfM2CHEGo5K eaUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=wXDZk42LlS9I1d/3NLMgr7fvX5h+CpA70Pot+6x9Foo=; b=wMy89ewZV58OeIG8QUeGjRh3RvwBVk0ZM+yiCPXBPefd7XAVTpbGnxeno/SP+wIatF rzXHQuKz4m/ui00i//mA/MBLEvgzTeJbwgXCSViBTB1+uOS0tc6fgfU/U8lgKM7v/Mbm uinKX5MVpgDjDJJeFJznOQI5dc4TjpxX6Id/Jql+vVtPKy9GeOQOrl29+s5Sngq1cDI5 6YEMo2Jeh5FNjOZESLe8t72C3yGI8QQHJLsv+WHEq5pstwHYXtfcp70TpUDxsXVUrqLN rdwNL7H+ECgC9Cf0Ry2ZAtWhMzmwoPN6Bb9dJdgotZUwxBv/teX9Edb4WJZxC3ZyzUB4 oUmA== X-Gm-Message-State: AOAM530IRy5V3pZMwKSBfZ5V2lEnPX1G9n+WI5gVvgjL0ioPKObcGt30 tJPXmOiBFz23p3Xzf3qozzMzAe7ehwoCOw== X-Google-Smtp-Source: ABdhPJxukkOsvtb7rs+/4GqPSM3WWNXVqAZ5VLIMOlYGvgiHsHEuJHKIjt0bHJtZpzIXnb6adnYjiQ== X-Received: by 2002:adf:f943:0:b0:203:e832:129 with SMTP id q3-20020adff943000000b00203e8320129mr738688wrr.626.1650565429473; Thu, 21 Apr 2022 11:23:49 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k11-20020a5d6d4b000000b0020599079f68sm2909090wri.106.2022.04.21.11.23.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Apr 2022 11:23:49 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , Eric Sunshine , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBC?= =?utf-8?b?amFybWFzb24=?= Subject: [PATCH v5 15/29] CI: check ignored unignored build artifacts in "win[+VS] build" too Date: Thu, 21 Apr 2022 20:23:10 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.879.g3659959fcca In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Use the newly created "ci/check-unignored-build-artifacts.sh" script in the "win build" and "win+VS build" steps too. These steps weren't doing this sanity check of untracked files against .gitignore before, but as the check passes for them let's keep doing that extra sanity check. Signed-off-by: Ævar Arnfjörð Bjarmason --- .github/workflows/main.yml | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 57b2db06923..c97fbd0f0b3 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -93,6 +93,9 @@ jobs: HOME: ${{runner.workspace}} NO_PERL: 1 run: . /etc/profile && ci/make-test-artifacts.sh artifacts + - run: ci/check-unignored-build-artifacts.sh + if: success() + shell: bash - name: zip up tracked files run: git archive -o artifacts/tracked.tar.gz HEAD - name: upload tracked files and build artifacts @@ -180,6 +183,9 @@ jobs: run: | mkdir -p artifacts && eval "$(make -n artifacts-tar INCLUDE_DLLS_IN_ARTIFACTS=YesPlease ARTIFACTS_DIRECTORY=artifacts NO_GETTEXT=YesPlease 2>&1 | grep ^tar)" + - run: ci/check-unignored-build-artifacts.sh + if: success() + shell: bash - name: zip up tracked files run: git archive -o artifacts/tracked.tar.gz HEAD - name: upload tracked files and build artifacts From patchwork Thu Apr 21 18:23:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12822185 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 488E6C433FE for ; Thu, 21 Apr 2022 18:24:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1391295AbiDUS1Y (ORCPT ); Thu, 21 Apr 2022 14:27:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47366 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1391305AbiDUS0n (ORCPT ); Thu, 21 Apr 2022 14:26:43 -0400 Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6532B4B423 for ; Thu, 21 Apr 2022 11:23:52 -0700 (PDT) Received: by mail-wr1-x431.google.com with SMTP id s25so2954813wrb.8 for ; Thu, 21 Apr 2022 11:23:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=kIwTifU4UCoJ7n13IFuUdN3lXAptchaCwfqeBAxUG7s=; b=KiksIpd5jkSWi3plEcWuwLbRsYzSbYgm9+GarAW5KCsnMdSOcZKavbTmYUZCBp73zM tLL/NL1zfK3CLPBOqxx69sAOAvcEQ+EL53YvvMU9INPoLt0fI2Io2O14CHcx97GocZE9 jSWmIGFgHBsEoBwyqYIWcjw6rm/fueI5etHNLs44VOcTil5E3+yWvQYbePFVJ7oKPQUU /LSW0TjX5n0TRan0Z6FzjST9+Ghqz+ljt9S3gQoOB9iqfu+oo9VZ3NiNQBORbY8KbV/l rBqaUKDs3E5G8HEB6tzZMFDV7zJOvjJhQVTFUbXnXDlpGWV/hpmYaJv6NPj3v3M+WTst KkRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=kIwTifU4UCoJ7n13IFuUdN3lXAptchaCwfqeBAxUG7s=; b=D43FBVrGcg3DDnSVO9OMRMfIa7MO7NDWR2jYyOJwLvjb+AP6IcNucxF0Yj/W0Ip/5D 6JGWNJ81551+3VByfO9QhJ9gzrFWTcuIIU1klj68KC9gHqQj1STTjDyynE/Aupxe02Tg wMX9ozEJ8D2RiLjb5pjfaM2OWEzgsbHmyiTINQMfPGUoKwISIeRZrvlsCHmmQ6khTSXq hZZriHdXT7Clc0y5lx8Z70XUdCtcjgKMnQvo0x9f5ahEIfOhW+SIAZsHJoLUQbip7rou KMd7FRQ3UgEoJ9ZTzEa0cY2CBkRSEDpTo3U5sFeFwwCFEmt/3lXCZDiSnSfWHtx3DIcW eKkg== X-Gm-Message-State: AOAM530BPbi80MP9ytPkIoVeiJEWYVFuqTgwapGDv5XO3u3dYICX3B20 YnpfUprXBO3tNR3fSkrvEA1h9eA6SwqKfg== X-Google-Smtp-Source: ABdhPJwng+LdrMXHkVwBbrI3ht2Lhds8zy/p8+lI4xW3bGNLhEzSpwNIknZJNgpJXAivUnYATguMYQ== X-Received: by 2002:a5d:4cc8:0:b0:207:b601:3137 with SMTP id c8-20020a5d4cc8000000b00207b6013137mr735378wrt.652.1650565430473; Thu, 21 Apr 2022 11:23:50 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k11-20020a5d6d4b000000b0020599079f68sm2909090wri.106.2022.04.21.11.23.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Apr 2022 11:23:49 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , Eric Sunshine , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBC?= =?utf-8?b?amFybWFzb24=?= Subject: [PATCH v5 16/29] CI: invoke "make artifacts-tar" directly in windows-build Date: Thu, 21 Apr 2022 20:23:11 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.879.g3659959fcca In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Change the windows-build job to invoke the "make artifacts-tar" step directly, instead of calling a "ci/make-test-artifacts.sh" script. The script was needed because "ci/lib.sh" would set up various environment variables for us, but now we can instead use the "ci/lib.sh" in its script mode. The "mkdir -p" added in b819f1d2cec (ci: parallelize testing on Windows, 2019-01-29) isn't needed, the same commit added that "mkdir -p" to the "artifacts-tar" rule itself, so we can have "make" create the directory for us. This also has the benefit of making the "build" step less chatty, since it won't start with the verbose "set -x" output, that's now contained in the "ci/lib.sh" step. Signed-off-by: Ævar Arnfjörð Bjarmason --- .github/workflows/main.yml | 5 ++++- ci/make-test-artifacts.sh | 10 ---------- 2 files changed, 4 insertions(+), 11 deletions(-) delete mode 100755 ci/make-test-artifacts.sh diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index c97fbd0f0b3..ca993afd098 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -87,12 +87,15 @@ jobs: steps: - uses: actions/checkout@v2 - uses: git-for-windows/setup-git-for-windows-sdk@v1 + - run: ci/lib.sh + shell: bash - name: build shell: bash env: HOME: ${{runner.workspace}} NO_PERL: 1 - run: . /etc/profile && ci/make-test-artifacts.sh artifacts + run: . /etc/profile && make artifacts-tar ARTIFACTS_DIRECTORY=artifacts + if: success() - run: ci/check-unignored-build-artifacts.sh if: success() shell: bash diff --git a/ci/make-test-artifacts.sh b/ci/make-test-artifacts.sh deleted file mode 100755 index 45298562982..00000000000 --- a/ci/make-test-artifacts.sh +++ /dev/null @@ -1,10 +0,0 @@ -#!/bin/sh -# -# Build Git and store artifacts for testing -# - -mkdir -p "$1" # in case ci/lib.sh decides to quit early - -. ${0%/*}/lib.sh - -make artifacts-tar ARTIFACTS_DIRECTORY="$1" From patchwork Thu Apr 21 18:23:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12822186 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7F901C433F5 for ; Thu, 21 Apr 2022 18:24:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1391310AbiDUS1Z (ORCPT ); Thu, 21 Apr 2022 14:27:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47438 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1391319AbiDUS0o (ORCPT ); Thu, 21 Apr 2022 14:26:44 -0400 Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4439C4B432 for ; Thu, 21 Apr 2022 11:23:53 -0700 (PDT) Received: by mail-wr1-x431.google.com with SMTP id q7so207666wrm.5 for ; Thu, 21 Apr 2022 11:23:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=oo+ICyxdZE46CpbXr8+aj6uyPRH2+M40XExzUK+f9gA=; b=AfDkq3hLQEWnL8eAnjBQos5fVQ1qbKDwv2+v6RSyBAG8rdSMNmLOpdhIolRnWpSxzO rxmC6Qo77TkeTJusQaNG/WRCywF4K7Ywzw9tL5uCK7yxHBeoVmbkztIf9+l2KhdDVh+l s9UNzVEt72v3ZY1svvb7unJUAtwL/qo1yUnrZ090x7t/uAUh2LxnW629h8nZAs2BWyhg h7H5cxpcEoMcgs2o7xYia3vZuORM0HuPeBgFMnpwfgxT4eeUUMwd5fKugXuqJhIWjvFC z9etd0K0FrDKrVLyp0hJ+9D59O1g9XGdbGsam4Uf+2BRylP0HHYgh6n8DkzHQvSlDAgl Utxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=oo+ICyxdZE46CpbXr8+aj6uyPRH2+M40XExzUK+f9gA=; b=CnbHcqBxO2S6ZhR0yJ3MgjWzZoBT+Zi7NvXmYx9CAiA0jFNO1IwFcEPMzuc4zJTrGq LRK62WLIkPCNamgYN5IcJ7C0tKiJ4gUYhqaNJnVRa1HQAYEvP6KMX4HGWNwoy/Po3Qty JOnCHcujqTNk5V9Ee+Vqsz+v/g/CgDuPCHInCFesRNQUtGVZfKdg4y3jZ3r0CBmBvy+x Sz5ieFk0HJasZuy7pX7n1Ku5RtseeVVxS/qc5JDZFZVnarp+MJcAu8jVdXpVAzQbOaNZ AIxwQXgeuxgdAdlLYzHLSSJ6SQ7ZFEjNqBTmf8LFAi3Id9c9CwHpbOmnYK/Df5ZNet4f etjA== X-Gm-Message-State: AOAM532m4+JOdhy6q0CnS2ooI6nAoXJombZ0a9Ebt+arlNHHjSZ5DPjG AHj4KbyCzW/oy0kZYNZgeae7K7imgoIHGA== X-Google-Smtp-Source: ABdhPJyXINqzI5GSkYhzIxHwwmuckZ7dpPrj+gsnDMpiRXufJ03eM7NNQnR4OgTGe2wwNRHgPUClug== X-Received: by 2002:a05:6000:1c0b:b0:20a:855e:1272 with SMTP id ba11-20020a0560001c0b00b0020a855e1272mr744289wrb.261.1650565431457; Thu, 21 Apr 2022 11:23:51 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k11-20020a5d6d4b000000b0020599079f68sm2909090wri.106.2022.04.21.11.23.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Apr 2022 11:23:50 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , Eric Sunshine , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBC?= =?utf-8?b?amFybWFzb24=?= Subject: [PATCH v5 17/29] CI: split up and reduce "ci/test-documentation.sh" Date: Thu, 21 Apr 2022 20:23:12 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.879.g3659959fcca In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Change the "ci/test-documentation.sh" script to run the bash-specific parts in as one command in the CI job itself, and to run the two "make doc" commands at the top-level. It'll now be obvious from the title of the step if we failed in the asciidoc or asciidoctor step. Since the "check_unignored_build_artifacts()" function is now only used in "ci/check-unignored-build-artifacts.sh" move that function there. The recipe for the job in ".github/workflows/main.yml" is now a bit verbose because it's effectively the same job twice, with a "make clean" in-between. It would be better for the verbosity to run it via a matrix as done in the alternate approach in [1] does, but then we'd sacrifice overall CPU time for the brevity. It might still be worth doing, but let's go for this simpler approach for now. 1. https://lore.kernel.org/git/patch-v2-6.6-7c423c8283d-20211120T030848Z-avarab@gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason --- .github/workflows/main.yml | 16 +++++++++++++- ci/check-unignored-build-artifacts.sh | 10 +++++++++ ci/lib.sh | 10 --------- ci/test-documentation.sh | 31 +++++++++------------------ 4 files changed, 35 insertions(+), 32 deletions(-) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index ca993afd098..88eb2ef2dad 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -365,4 +365,18 @@ jobs: steps: - uses: actions/checkout@v2 - run: ci/install-dependencies.sh - - run: ci/test-documentation.sh + - run: ci/lib.sh + - run: make check-docs + - run: "make doc > >(tee stdout.log) 2> >(tee stderr.raw >&2)" + shell: bash + - run: ci/test-documentation.sh AsciiDoc + if: success() + - run: ci/check-unignored-build-artifacts.sh + if: success() + - run: make clean + - run: "make USE_ASCIIDOCTOR=1 doc > >(tee stdout.log) 2> >(tee stderr.raw >&2)" + shell: bash + - run: ci/test-documentation.sh Asciidoctor + if: success() + - run: ci/check-unignored-build-artifacts.sh + if: success() diff --git a/ci/check-unignored-build-artifacts.sh b/ci/check-unignored-build-artifacts.sh index 56d04b0db9a..0bc04f32804 100755 --- a/ci/check-unignored-build-artifacts.sh +++ b/ci/check-unignored-build-artifacts.sh @@ -5,4 +5,14 @@ . ${0%/*}/lib.sh +check_unignored_build_artifacts () +{ + ! git ls-files --other --exclude-standard --error-unmatch \ + -- ':/*' 2>/dev/null || + { + echo "$(tput setaf 1)error: found unignored build artifacts$(tput sgr0)" + false + } +} + check_unignored_build_artifacts diff --git a/ci/lib.sh b/ci/lib.sh index 2fd022c94bc..63c14f504b3 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -47,16 +47,6 @@ setenv () { fi } -check_unignored_build_artifacts () -{ - ! git ls-files --other --exclude-standard --error-unmatch \ - -- ':/*' 2>/dev/null || - { - echo "$(tput setaf 1)error: found unignored build artifacts$(tput sgr0)" - false - } -} - # GitHub Action doesn't set TERM, which is required by tput setenv TERM ${TERM:-dumb} diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh index 6b5cce03bd7..b8a6a6f664e 100755 --- a/ci/test-documentation.sh +++ b/ci/test-documentation.sh @@ -1,10 +1,12 @@ -#!/usr/bin/env bash +#!/bin/sh # -# Perform sanity checks on documentation and build it. +# Perform sanity checks on "make doc" output and built documentation # . ${0%/*}/lib.sh +generator=$1 + filter_log () { sed -e '/^GIT_VERSION = /d' \ -e "/constant Gem::ConfigMap is deprecated/d" \ @@ -14,28 +16,15 @@ filter_log () { "$1" } -make check-docs - -# Build docs with AsciiDoc -make doc > >(tee stdout.log) 2> >(tee stderr.raw >&2) -cat stderr.raw -filter_log stderr.raw >stderr.log -test ! -s stderr.log -test -s Documentation/git.html -test -s Documentation/git.xml -test -s Documentation/git.1 -grep ' X-Patchwork-Id: 12822187 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AF307C433EF for ; Thu, 21 Apr 2022 18:25:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1391317AbiDUS13 (ORCPT ); Thu, 21 Apr 2022 14:27:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47484 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1391324AbiDUS0p (ORCPT ); Thu, 21 Apr 2022 14:26:45 -0400 Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com [IPv6:2a00:1450:4864:20::430]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 60E054B1C4 for ; Thu, 21 Apr 2022 11:23:54 -0700 (PDT) Received: by mail-wr1-x430.google.com with SMTP id c10so7864233wrb.1 for ; Thu, 21 Apr 2022 11:23:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=TNUVJxC79CfszbM9CXqmRo7F85wzo5iIoKEaX8LvQfU=; b=W/4ijIaqoDOiYiZRMpwp5f3T5ZJFJsEOkOxL/VOwM5PiEzMpL8yo5cr8mkGIDMqOuO gPnUag/WvGtlGJ9DNTzfIwkB/SsgDdEQDjybXTZ+AmKTfmYCrtPCiD/NFySDrNoQhN73 3LqcdHpRm8bQLVaZ4e3GlwQCmFhJo674lMbUV/eCoGhokaO0a1WIIqGFymP7rMN42m3K frd6jIrmYTXlT1wHrq0zx0l5EOG6ZwDT0Pa4BTjlRECRlPWK3a3vhso+8yde1JGQC4MA tXlZW10zFI2OI309A4NL9QGFNwhmi/uR4QXyN4S3OvfScNPKpEJe2NS27FKLDHJHnrUx HGQQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=TNUVJxC79CfszbM9CXqmRo7F85wzo5iIoKEaX8LvQfU=; b=EX+maJ4oWOOaNu6h/HS7fn7MBx9SxXdPX5+3E0pZzL5df4Bd2YZDc337w5A3+lWBNB Mf0RdQ8GWiZW0VjjVGXdzvxmOWHPu6pMbCwCn+RFWL+kXEaKXWPHgQbTFaPnUdeGIYRE Et+VD0dzt+rqMHugFBTO7BlnLSWnbSrBEnYz0aLIm1t6Jh+BXt7XV5fkv7N1XLa17Lvk +zYlkvWZJGNk+jr8tHjCMULtRWm+i8YAoeH2FMO68g7js/w/n656YnOa4BQiMvPRExbE g8aJuBwIVvH85qVVSt1c2lkUJKV4MXtPmP4Og6AzA2yz95ZKBuoGRXOWNig69didB0Cc iT2Q== X-Gm-Message-State: AOAM531eP93LWKv3buXzfH49oFwRjVS172Z2aOI/DlFn7wzY90wXAteo Y+VEXunvCIHkFynqRgkZZTSDBFXN91IeZw== X-Google-Smtp-Source: ABdhPJzF/gascFQmuOXJ1QZxrPE3s0J6gUw3A+L6hQFTw+XMmbTPj7QAQMjhkHZPl435iJ4pXOLChA== X-Received: by 2002:a5d:6981:0:b0:205:e04a:e20f with SMTP id g1-20020a5d6981000000b00205e04ae20fmr759757wru.6.1650565432370; Thu, 21 Apr 2022 11:23:52 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k11-20020a5d6d4b000000b0020599079f68sm2909090wri.106.2022.04.21.11.23.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Apr 2022 11:23:51 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , Eric Sunshine , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBC?= =?utf-8?b?amFybWFzb24=?= Subject: [PATCH v5 18/29] CI: combine ci/install{,-docker}-dependencies.sh Date: Thu, 21 Apr 2022 20:23:13 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.879.g3659959fcca In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Combine the "ci/install-dependencies.sh" and "ci/install-docker-dependencies.sh" into one script. The "case" statement in the latter only cared about "$jobname", and can be folded into the same "case" statement in the former. The reason they split up is historical, until a preceding commit "ci/lib.sh" required "bash", which might not have been available in "docker". This also fixes issue in "ci/install-docker-dependencies.sh" where we'd hide errors due to not using "set -e". Now that we include "ci/lib.sh" we won't have that potential issue. Signed-off-by: Ævar Arnfjörð Bjarmason --- .github/workflows/main.yml | 2 +- ci/install-dependencies.sh | 15 +++++++++++++++ ci/install-docker-dependencies.sh | 22 ---------------------- 3 files changed, 16 insertions(+), 23 deletions(-) delete mode 100755 ci/install-docker-dependencies.sh diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 88eb2ef2dad..42944020436 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -311,7 +311,7 @@ jobs: container: ${{matrix.vector.image}} steps: - uses: actions/checkout@v1 - - run: ci/install-docker-dependencies.sh + - run: ci/install-dependencies.sh - run: ci/lib.sh - run: make - run: make test diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh index 1ad4dd7db39..adece989491 100755 --- a/ci/install-dependencies.sh +++ b/ci/install-dependencies.sh @@ -97,6 +97,21 @@ linux-gcc-default) sudo apt-get -q update sudo apt-get -q -y install $UBUNTU_COMMON_PKGS ;; +linux32) + linux32 --32bit i386 sh -c ' + apt update >/dev/null && + apt install -y build-essential libcurl4-openssl-dev \ + libssl-dev libexpat-dev gettext python >/dev/null + ' + ;; +linux-musl) + apk add --update build-base curl-dev openssl-dev expat-dev gettext \ + pcre2-dev python3 musl-libintl perl-utils ncurses >/dev/null + ;; +pedantic) + dnf -yq update >/dev/null && + dnf -yq install make gcc findutils diffutils perl python3 gettext zlib-devel expat-devel openssl-devel curl-devel pcre2-devel >/dev/null + ;; esac if type p4d >/dev/null && type p4 >/dev/null diff --git a/ci/install-docker-dependencies.sh b/ci/install-docker-dependencies.sh deleted file mode 100755 index 78b7e326da6..00000000000 --- a/ci/install-docker-dependencies.sh +++ /dev/null @@ -1,22 +0,0 @@ -#!/bin/sh -# -# Install dependencies required to build and test Git inside container -# - -case "$jobname" in -linux32) - linux32 --32bit i386 sh -c ' - apt update >/dev/null && - apt install -y build-essential libcurl4-openssl-dev \ - libssl-dev libexpat-dev gettext python >/dev/null - ' - ;; -linux-musl) - apk add --update build-base curl-dev openssl-dev expat-dev gettext \ - pcre2-dev python3 musl-libintl perl-utils ncurses >/dev/null - ;; -pedantic) - dnf -yq update >/dev/null && - dnf -yq install make gcc findutils diffutils perl python3 gettext zlib-devel expat-devel openssl-devel curl-devel pcre2-devel >/dev/null - ;; -esac From patchwork Thu Apr 21 18:23:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12822188 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D315DC433FE for ; Thu, 21 Apr 2022 18:25:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1391296AbiDUS16 (ORCPT ); Thu, 21 Apr 2022 14:27:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47138 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1391287AbiDUS0q (ORCPT ); Thu, 21 Apr 2022 14:26:46 -0400 Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com [IPv6:2a00:1450:4864:20::42e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 12FD14B43E for ; Thu, 21 Apr 2022 11:23:55 -0700 (PDT) Received: by mail-wr1-x42e.google.com with SMTP id v12so1042601wrv.10 for ; Thu, 21 Apr 2022 11:23:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=YG71xcNttSkhpSpeukLj4pYrcQVDkMqSFmcPCQCL7rY=; b=WCVB5eL4w0jAo88l1RNtvBSKaJ4w80/k3NMe75tl8VsAv5cDp2G2Gd5YJo4uj+9M9Q ByKZ04BCrTerBGCruLp/lKiRB0wUZEdRknxcTDuW0NXCscyzYDEfuoO1fWaOn27tzZ4l qdpbEOPFCI+hHwasWgWuBecKUPb/lYtlXezEjLBuZOnG8jbgWOrwev9VsdIum2T/xTou 3EY1ThhZSXHbW9wm5AmVRsbHy2zJlJTPfEZypsA6GC4n+/tVEsGT2zK83OctFm9ep0KW YzbHzE4BnWIRaBlK/o91yQBWPcjJCjbgvtnKNcaXHZFJBAe5GMWa241yVazn63RohEWz rdgg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=YG71xcNttSkhpSpeukLj4pYrcQVDkMqSFmcPCQCL7rY=; b=UGCwsDBmFa5AIW8Ks+/l9twmDlcRTQ8O9aMGRoUY/Yxcb17KJD4LzoQWH4bUcjcTPK ZCaS8LUnuE/vDqyb5PvhRL0qNQHAzc9mdwb7wMkht6+iyly2oGrhF1rh3Xf3woixOWa9 5vWkac1kMcfO9/+4QFEviv9CGdQtKhYKYOV6RtvUWHejCoKHya2Io4JdSOL0imgPIz+5 /C+w4MthWycvGZtv+uHoAd+TBhwqBszIW3frj1jtNcRdiBDFf4PVmIGX0E9MkRNyPZ3N a0f8ylpoMGu9rYfBtbcGhED5WT5W2Z5ZaLL36rdmAxn3uqHWsaKz9aRmYM2oK2LeB94+ 24lg== X-Gm-Message-State: AOAM5316RIOvQPTCnDzBUEGBf03w0guJj32qBs7j1bQ7Yrur07hfgmrA xVsEIjCD9/uFNYgr4ycEmN7wLR+U7jH5ag== X-Google-Smtp-Source: ABdhPJyKaL5gfdySnup60fj1mgmw3k/hQNpXc3KB6JFzzlf2IVjtTztTQtdMRSDM60KoKre1PeWOmA== X-Received: by 2002:a5d:50ca:0:b0:207:bc3d:1026 with SMTP id f10-20020a5d50ca000000b00207bc3d1026mr777565wrt.154.1650565433321; Thu, 21 Apr 2022 11:23:53 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k11-20020a5d6d4b000000b0020599079f68sm2909090wri.106.2022.04.21.11.23.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Apr 2022 11:23:52 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , Eric Sunshine , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBC?= =?utf-8?b?amFybWFzb24=?= Subject: [PATCH v5 19/29] CI: move "env" definitions into ci/lib.sh Date: Thu, 21 Apr 2022 20:23:14 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.879.g3659959fcca In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Have the GitHub CI jobs use the "ci/lib".sh as a source of truth for environment variables that affect their builds and tests. This brings them in line with the rest of the jobs, and make it easier to see the entire CI configuration per jobname at a glance. To do this we need to add a ci/lib.sh "step" to the remaining jobs that didn't have it added in preceding commits. The Makefile parameters are luckily all accepted via the environment, so we can export these instead of adding them to MAKEFLAGS. Let's also use the documented GIT_CONIFG_* mechanism for setting config instead of the internal GIT_CONFIG_PARAMETERS variable. This adjusts code added in 889cacb6897 (ci: configure GitHub Actions for CI/PR, 2020-04-11), we could probably use the GIT_AUTHOR_NAME etc, but let's keep setting this via config, just with the documented mechanism. By setting "ARTIFACTS_DIRECTORY=artifacts" once we don't need to repeat it in various places, let's get it from the environment instead. Finally, the "DEVELOPER: 1" was only needed by the "sparse" job, which wasn't using "ci/lib.sh" (and which sets DEVELOPER=1). Let's instead have the "sparse" job use the "ci/lib.sh" and remove DEVELOPER=1 from ".github/workflows/main.yml". This substantially speeds up the "sparse" job, since it'll now pick up the "--jobs" setting in MAKEFLAGS that we use everywhere else. Before it ran in around 4m30s, now in around 2m30s. Signed-off-by: Ævar Arnfjörð Bjarmason --- .github/workflows/main.yml | 29 +++++++++++------------------ ci/lib.sh | 21 +++++++++++++++++++++ 2 files changed, 32 insertions(+), 18 deletions(-) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 42944020436..5f167ac5be6 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -2,9 +2,6 @@ name: CI on: [push, pull_request] -env: - DEVELOPER: 1 - jobs: ci-config: name: config @@ -93,8 +90,7 @@ jobs: shell: bash env: HOME: ${{runner.workspace}} - NO_PERL: 1 - run: . /etc/profile && make artifacts-tar ARTIFACTS_DIRECTORY=artifacts + run: . /etc/profile && make artifacts-tar if: success() - run: ci/check-unignored-build-artifacts.sh if: success() @@ -105,7 +101,7 @@ jobs: uses: actions/upload-artifact@v2 with: name: windows-artifacts - path: artifacts + path: ${{env.ARTIFACTS_DIRECTORY}} windows-test: name: win test env: @@ -126,6 +122,8 @@ jobs: shell: bash run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz - uses: git-for-windows/setup-git-for-windows-sdk@v1 + - run: ci/lib.sh + shell: bash - name: test shell: bash run: . /etc/profile && ci/run-test-slice.sh ${{matrix.nr}} 10 @@ -145,9 +143,6 @@ jobs: jobname: vs-build needs: ci-config if: needs.ci-config.outputs.enabled == 'yes' - env: - NO_PERL: 1 - GIT_CONFIG_PARAMETERS: "'user.name=CI' 'user.email=ci@git'" runs-on: windows-latest steps: - uses: actions/checkout@v2 @@ -171,6 +166,8 @@ jobs: - name: copy dlls to root shell: cmd run: compat\vcbuild\vcpkg_copy_dlls.bat release + - run: ci/lib.sh + shell: bash - name: generate Visual Studio solution shell: bash run: | @@ -181,21 +178,16 @@ jobs: - name: bundle artifact tar shell: bash env: - MSVC: 1 VCPKG_ROOT: ${{github.workspace}}\compat\vcbuild\vcpkg run: | - mkdir -p artifacts && - eval "$(make -n artifacts-tar INCLUDE_DLLS_IN_ARTIFACTS=YesPlease ARTIFACTS_DIRECTORY=artifacts NO_GETTEXT=YesPlease 2>&1 | grep ^tar)" - - run: ci/check-unignored-build-artifacts.sh - if: success() - shell: bash + eval "$(make -n artifacts-tar 2>&1 | grep -e ^mkdir -e ^tar)" - name: zip up tracked files run: git archive -o artifacts/tracked.tar.gz HEAD - name: upload tracked files and build artifacts uses: actions/upload-artifact@v2 with: name: vs-artifacts - path: artifacts + path: ${{env.ARTIFACTS_DIRECTORY}} vs-test: name: win+VS test env: @@ -216,10 +208,10 @@ jobs: - name: extract tracked files and build artifacts shell: bash run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz + - run: ci/lib.sh + shell: bash - name: test shell: bash - env: - NO_SVN_TESTS: 1 run: . /etc/profile && ci/run-test-slice.sh ${{matrix.nr}} 10 - name: ci/print-test-failures.sh if: failure() @@ -354,6 +346,7 @@ jobs: - uses: actions/checkout@v2 - name: Install other dependencies run: ci/install-dependencies.sh + - run: ci/lib.sh - run: make sparse documentation: name: documentation diff --git a/ci/lib.sh b/ci/lib.sh index 63c14f504b3..cae324e2fd7 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -104,6 +104,27 @@ macos-latest) esac case "$jobname" in +windows-build) + setenv --build NO_PERL NoThanks + setenv --build ARTIFACTS_DIRECTORY artifacts + ;; +vs-build) + setenv --build NO_PERL NoThanks + setenv --build NO_GETTEXT NoThanks + setenv --build ARTIFACTS_DIRECTORY artifacts + setenv --build INCLUDE_DLLS_IN_ARTIFACTS YesPlease + setenv --build MSVC YesPlease + + setenv --build GIT_CONFIG_COUNT 2 + setenv --build GIT_CONFIG_KEY_0 user.name + setenv --build GIT_CONFIG_VALUE_0 CI + setenv --build GIT_CONFIG_KEY_1 user.emailname + setenv --build GIT_CONFIG_VALUE_1 ci@git + setenv --build GIT_CONFIG_VALUE_1 ci@git + ;; +vs-test) + setenv --test NO_SVN_TESTS YesPlease + ;; linux-gcc) setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main ;; From patchwork Thu Apr 21 18:23:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12822189 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DA7A8C433F5 for ; Thu, 21 Apr 2022 18:25:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1391320AbiDUS2C (ORCPT ); Thu, 21 Apr 2022 14:28:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48562 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1391345AbiDUS1Q (ORCPT ); Thu, 21 Apr 2022 14:27:16 -0400 Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com [IPv6:2a00:1450:4864:20::329]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 09E3D4B846 for ; Thu, 21 Apr 2022 11:23:56 -0700 (PDT) Received: by mail-wm1-x329.google.com with SMTP id bg25so3003299wmb.4 for ; Thu, 21 Apr 2022 11:23:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=bOJciQByR0PpEVFPL6yjCiXMxNdJ38bge9UgYcT26Ec=; b=LycUbstx1XgjfEkcqcZmXw1XHtzmHyF3q6yEx/L8P9v2ZxnYnQhzmOBQVNxY9b+dhx lzxC5YldAy+seYzWgr68wOidMtBBb/dxRKh3GvsYfHxfap/ckmqQltHYrn2R5PLi0o/D DUd/SIwZskVeyMS/fKX+da2Y3V5kOXg5uFJYy3D2p0vWJyDSVbiV2fD68G9npPsvAL/6 jGNYw+wqQljOIG481jxUpJYqUgKf9Tx0ad4PXGyhTtIVy+ORGYvyyPtYwQCq4egjbqLL YQLP1bq4kwHtC5dzoyOGL7wzNRxX+p+P+1Qgdz9U+rvdW0R4xNzv0Sm5nk49KMrYEfn9 GAsw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=bOJciQByR0PpEVFPL6yjCiXMxNdJ38bge9UgYcT26Ec=; b=QQ7lWgsW8tinax2qNOUOyZgy4IUyNay+H8L2npr/JYY2DjOLgZUnnNTBIPhVSM+j9I 6oYMea0bE5T+an0e3zbTRmuAcCLoe83Cj8xm6heHdm+hQwhVHggnJvywWsXifNMhC4Nw uxm6XrsdnqIZsxi9+sWNbEJa1m0Uxj0fNXwBwypXP8S+yB1h9DtSt8Tkd/d8gAftFeuB THy83hZntE3rrWgLX4HFCTdXJaVahGx+cej0YUJ5gRESUd9PmabKdkBtCL+IDJQ36FoD 3KSm+ifr/Dz+TrZesAnfXZeCH5Q6YzSiT2wSaH659k8FjAf6BJOY8EdZ/ZrYoNNa/83R JIbQ== X-Gm-Message-State: AOAM532lkWynFz0HTq2aYK1+5498YPxrc/0r+Ptjg+eC0Fiiby/gZw9w 8ptm0UaXLaH8vphfsn/8jjQdnn2DXzLf+A== X-Google-Smtp-Source: ABdhPJymJEHig+OleuB5ra8reOZjtRdUuHkerG9tbWVGjKFvjJqVssKhcJKh4Q1LStd8NhF7Yg7mcA== X-Received: by 2002:a05:600c:410c:b0:38e:d2b5:ef2a with SMTP id j12-20020a05600c410c00b0038ed2b5ef2amr560098wmi.91.1650565434374; Thu, 21 Apr 2022 11:23:54 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k11-20020a5d6d4b000000b0020599079f68sm2909090wri.106.2022.04.21.11.23.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Apr 2022 11:23:53 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , Eric Sunshine , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBC?= =?utf-8?b?amFybWFzb24=?= Subject: [PATCH v5 20/29] ci/run-test-slice.sh: replace shelling out with "echo" Date: Thu, 21 Apr 2022 20:23:15 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.879.g3659959fcca In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Amend this code added in b819f1d2cec (ci: parallelize testing on Windows, 2019-01-29) to stop shelling out to "tr", all we're trying to do here is to turn '\n'-delimited output into a ' '-delimited list. To do that we can just use "echo". None of these files will contain the odd characters using "echo" might choke on in the general case (i.e. '\'-escapes and the like). This has the added benefit of not adding a redundant ' ' to the end of the T-parameter, which didn't matter in practice, but as we'll see in a subsequent commit makes for slightly nicer output, as we'll show this variable using a GitHub CI-native feature. Signed-off-by: Ævar Arnfjörð Bjarmason --- ci/run-test-slice.sh | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/ci/run-test-slice.sh b/ci/run-test-slice.sh index 70326961454..1de6a18ca47 100755 --- a/ci/run-test-slice.sh +++ b/ci/run-test-slice.sh @@ -5,6 +5,6 @@ . ${0%/*}/lib.sh -make --quiet -C t T="$(cd t && - ./helper/test-tool path-utils slice-tests "$1" "$2" t[0-9]*.sh | - tr '\n' ' ')" +tests=$(echo $(cd t && ./helper/test-tool path-utils slice-tests "$1" "$2" \ + t[0-9]*.sh)) +make --quiet -C t T="$tests" From patchwork Thu Apr 21 18:23:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12822197 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BAFDFC433EF for ; Thu, 21 Apr 2022 18:25:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1391376AbiDUS2a (ORCPT ); Thu, 21 Apr 2022 14:28:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48566 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1391348AbiDUS1Q (ORCPT ); Thu, 21 Apr 2022 14:27:16 -0400 Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com [IPv6:2a00:1450:4864:20::32f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E64C34B437 for ; Thu, 21 Apr 2022 11:23:56 -0700 (PDT) Received: by mail-wm1-x32f.google.com with SMTP id v64-20020a1cac43000000b0038cfd1b3a6dso6414011wme.5 for ; Thu, 21 Apr 2022 11:23:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=UXHpyYHpqYmeqD3dRDb1GfL1xBghdzRo2jWukUDkt34=; b=lp2rEV3685ANEG+VsaUfnM5pCojL40mXwLhiVmDpglFKSzEqPv5Ibj7SHVKqbaVRri mJkcs+6hnePb1h0fTvSNpjpUuKF4/IpJA/LB1xsqBNdSA8wkWVHKDoLFgArRu8mSdecV kJ5N8NWcmI8k7sG0bxl7TprNPNsm8QFLtDnF88AfCHG8hJ5RAEu66PVx8AsWXyJIMAIL hTYnZB86Yah6Ii5VyGP9lhtFB990kq/g4RZkOh+MZOT0wjzXbNxlE4vrkGAoLYByJfVv TpDn5rOJjyJ36QYnB+xOa9Wv8MbQmRF2LXTbj3qv6KgUwdgrv8+0PQwiRYZzdpft2Grw X1HA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=UXHpyYHpqYmeqD3dRDb1GfL1xBghdzRo2jWukUDkt34=; b=Qofxxg+AVlQz8zTd4Z5/EDrEAUWPzLxD9KAAKHXe6HQVmaaNPyt8bY/PC/30GeH9w8 j/3drpY1VmOTMJ38iHhVQ5ykSVbn8qKccLDFPAqARe9TvPy/5LsOBexATB6YAxHdvFtF BpZN/5sxy3viu8NIzNiM0B545IR5LYIX86859lCF6Gr3YmKtpFKbt7xdcs0ychPI7ody NSoLbrPWC7CArJZ5EQ0eAIrba8PKr2TcSFgFyjbhfzrMONx7GPK8oY4ACI/enPDqdTbH 27FD3gxwTxZJquoOi/CvrQS0JOq/E/XazMgYdGdTuj7+r/LtsEKKT/B7HQNl4pJtibYz x+XA== X-Gm-Message-State: AOAM533V2q2Sn2Xf+CRiCrJLmV+qtXjwQNyR5Qj2hgwnekGXW0gl3EcA ver2KOBPAlbkQIqg8i5j6BZc25dlKncVcA== X-Google-Smtp-Source: ABdhPJz+sFkV4tVak6BBYzomJY8GDUl23cZSUyIJzFN1efrVUzyXZycDUakqmUi08eZxJERjW1tc5Q== X-Received: by 2002:a1c:cc01:0:b0:37b:dcc8:7dfd with SMTP id h1-20020a1ccc01000000b0037bdcc87dfdmr600893wmb.134.1650565435265; Thu, 21 Apr 2022 11:23:55 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k11-20020a5d6d4b000000b0020599079f68sm2909090wri.106.2022.04.21.11.23.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Apr 2022 11:23:54 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , Eric Sunshine , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBC?= =?utf-8?b?amFybWFzb24=?= Subject: [PATCH v5 21/29] CI: pre-select test slice in Windows & VS tests Date: Thu, 21 Apr 2022 20:23:16 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.879.g3659959fcca In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In preceding commits the tests have been changed to do their setup via $GITHUB_ENV in one step, and to then have subsequent steps that re-use that environment. Let's change the "test slice" tests added in b819f1d2cec (ci: parallelize testing on Windows, 2019-01-29) to do the same. These tests select 10% of the tests to run in 10 "test slices". Now we'll select those in a step that immediately precedes the testing step, and then simply invoke "make -C t -e". This has the advantage that the tests to be run are now listed in the standard "Run" drop-down at the start of the "test" step. Since the "T" variable in "t/Makefile" doesn't normally accept overrides from the environment we need to invoke "make" with the "-e" option (a.k.a. "--environment-overrides"). We could also make $(T) in t/Makefile be a "?=" assigned variable, but this way works, and is arguably clearer as it's more obvious that we're injecting a special list of tests that override the normal behavior of that Makefile. Note that we cannot run the top-level "make test" here, because of how the Windows CI builds git, i.e. either via CMake or some option that would cause "make test" to recompile git itself. Instead we run "make -C t [...]". Signed-off-by: Ævar Arnfjörð Bjarmason --- .github/workflows/main.yml | 10 ++++++++-- ci/{run-test-slice.sh => select-test-slice.sh} | 7 +++++-- 2 files changed, 13 insertions(+), 4 deletions(-) rename ci/{run-test-slice.sh => select-test-slice.sh} (50%) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 5f167ac5be6..83e0aa1f469 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -124,9 +124,12 @@ jobs: - uses: git-for-windows/setup-git-for-windows-sdk@v1 - run: ci/lib.sh shell: bash + - name: select tests + run: . /etc/profile && ci/select-test-slice.sh ${{matrix.nr}} 10 + shell: bash - name: test shell: bash - run: . /etc/profile && ci/run-test-slice.sh ${{matrix.nr}} 10 + run: . /etc/profile && make -C t -e - name: ci/print-test-failures.sh if: failure() shell: bash @@ -210,9 +213,12 @@ jobs: run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz - run: ci/lib.sh shell: bash + - name: select tests + run: . /etc/profile && ci/select-test-slice.sh ${{matrix.nr}} 10 + shell: bash - name: test shell: bash - run: . /etc/profile && ci/run-test-slice.sh ${{matrix.nr}} 10 + run: . /etc/profile && make -C t -e - name: ci/print-test-failures.sh if: failure() shell: bash diff --git a/ci/run-test-slice.sh b/ci/select-test-slice.sh similarity index 50% rename from ci/run-test-slice.sh rename to ci/select-test-slice.sh index 1de6a18ca47..f59d8cadda5 100755 --- a/ci/run-test-slice.sh +++ b/ci/select-test-slice.sh @@ -1,10 +1,13 @@ #!/bin/sh # -# Test Git in parallel +# Select a portion of the tests for testing Git in parallel # . ${0%/*}/lib.sh tests=$(echo $(cd t && ./helper/test-tool path-utils slice-tests "$1" "$2" \ t[0-9]*.sh)) -make --quiet -C t T="$tests" +if test -n "$GITHUB_ENV" +then + echo T="$tests" >>$GITHUB_ENV +fi From patchwork Thu Apr 21 18:23:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12822190 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7564FC433F5 for ; Thu, 21 Apr 2022 18:25:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1391400AbiDUS2H (ORCPT ); Thu, 21 Apr 2022 14:28:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48578 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1391351AbiDUS1R (ORCPT ); Thu, 21 Apr 2022 14:27:17 -0400 Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com [IPv6:2a00:1450:4864:20::32a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 48E2F4B1E2 for ; Thu, 21 Apr 2022 11:23:58 -0700 (PDT) Received: by mail-wm1-x32a.google.com with SMTP id q20so3714935wmq.1 for ; Thu, 21 Apr 2022 11:23:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=jPxt9c67GJ21FrfGuSr0O8Wq7lwQtm3f1W/ThUDAe/g=; b=PjhUa2YV7vGVntIMgnDB07Xd6KoEVaq5glpHRIR+jbl+aH+zTiAU0jPMLWvq3TQnuJ d4fgdf0qrQmPDmT5+6jwvSZhlOIQrJRvc6xhkGE4EpRgfUoH1DlAifb26szp3MdXymyq R7iyTYLnqgdCz6ySIZ55FhRDlR5/zkG2gkLXb1IzKd76cjg8LrSs7d+bh2fGD0hLOL5k HOPv6jWAKnHkeM/ZemwM2trcOm+U62a25OdcxawpIUQG+xoOZ1UuMoTV+bsQqvxJb2Ld zOsk+Sbsd6m3+klVi9+mJEBsY+ekbEG48vdnu21lHYwgC7EQ6QeW34MBOPw6IT29xPwm WdJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=jPxt9c67GJ21FrfGuSr0O8Wq7lwQtm3f1W/ThUDAe/g=; b=LEfZEWR1DhI3YP1YxeVNr7wyZkyz56S0HhvYap16zEUF1iomXtoTRABIjOl/uiA4Lt rnqPN9oZRJYi9QgrMcFquSlnJUbscOLjKK43KhgEHZf/GGeiqUNMifAbnQZ/f5u7tpoY wLI1iau8LI+ZD5JgIIe/7kMvpZINin/q0pSt+RGx2PlK1IbrFvGPUCIU8DpIpFVS4+AS TuHGv4C90FsFEV2E0t1dSQ4tfjrKDtzq/ntu9UF18da5Gz34RgVIjVMT1XiTARDNWPVp zJ4yFGdCTzw89xkzkWa/UCFMM8Cu5QtYN9o3rz+KThwm0xLytYK1JtFTbny7dJMMIwmG nWlQ== X-Gm-Message-State: AOAM530Nb3KFE14lO6objhwsr6APugjGcd+lgOMI+cNkqQEvmeuFmk4p 3d0KSoLhN15s1zDsic1cfE8Fp3qSJKs1aQ== X-Google-Smtp-Source: ABdhPJyZo4JaVCjHH68FM1zSC9P1VaGmQ1f6N5J/t/d0SwALJsdpOYGsxNgdSCs4rgo5L+TNAGeSOQ== X-Received: by 2002:a05:600c:4f91:b0:38f:fba2:3ccb with SMTP id n17-20020a05600c4f9100b0038ffba23ccbmr567783wmq.125.1650565436423; Thu, 21 Apr 2022 11:23:56 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k11-20020a5d6d4b000000b0020599079f68sm2909090wri.106.2022.04.21.11.23.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Apr 2022 11:23:55 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , Eric Sunshine , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBC?= =?utf-8?b?amFybWFzb24=?= Subject: [PATCH v5 22/29] CI: only invoke ci/lib.sh as "steps" in main.yml Date: Thu, 21 Apr 2022 20:23:17 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.879.g3659959fcca In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Change the scripts in ci/ to stop using ci/lib.sh as a library, now that the only thing it did for them was to "set -ex" and possibly set TERM=dumb. Let's create a ci/lib-tput.sh for those that need to use "tput" instead, and have these scripts invoke "set -ex" themselves. This makes their invocation a lot less verbose, since they'll be relying on an earlier step in the CI job to have set the variables in $GITHUB_ENV, and won't be spewing their own trace output to set those variables again. Let's also create a ci/lib-ci-type.sh, and have ci/lib.sh and ci/print-test-failures.sh share the logic to discover the CI type. We could have set the CI_TYPE in the environment with "setenv", but let's avoid that verbosity for this purely internal variable. The "ci/lib.sh" is now no longer a "Library of functions shared by all CI scripts", so let's remove that commentary, and the misleading comment about "set -ex" being for "installing dependencies", we're now no longer using it in "ci/install-dependencies.sh" (but it does its own "set -ex"). Signed-off-by: Ævar Arnfjörð Bjarmason --- ci/check-unignored-build-artifacts.sh | 4 +++- ci/install-dependencies.sh | 2 +- ci/lib-ci-type.sh | 9 +++++++++ ci/lib-tput.sh | 5 +++++ ci/lib.sh | 27 ++++++++++----------------- ci/print-test-failures.sh | 6 +++--- ci/select-test-slice.sh | 2 +- ci/test-documentation.sh | 2 +- 8 files changed, 33 insertions(+), 24 deletions(-) create mode 100644 ci/lib-ci-type.sh create mode 100644 ci/lib-tput.sh diff --git a/ci/check-unignored-build-artifacts.sh b/ci/check-unignored-build-artifacts.sh index 0bc04f32804..c27d6a97f45 100755 --- a/ci/check-unignored-build-artifacts.sh +++ b/ci/check-unignored-build-artifacts.sh @@ -3,7 +3,9 @@ # Check whether the build created anything not in our .gitignore # -. ${0%/*}/lib.sh +set -ex + +. ${0%/*}/lib-tput.sh check_unignored_build_artifacts () { diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh index adece989491..4b4cdbe26e0 100755 --- a/ci/install-dependencies.sh +++ b/ci/install-dependencies.sh @@ -3,7 +3,7 @@ # Install dependencies required to build and test Git on Linux and macOS # -. ${0%/*}/lib.sh +set -ex UBUNTU_COMMON_PKGS="make libssl-dev libcurl4-openssl-dev libexpat-dev tcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl diff --git a/ci/lib-ci-type.sh b/ci/lib-ci-type.sh new file mode 100644 index 00000000000..6f01fd9e5d9 --- /dev/null +++ b/ci/lib-ci-type.sh @@ -0,0 +1,9 @@ +#!/bin/sh + +if test "$GITHUB_ACTIONS" = "true" +then + CI_TYPE=github-actions +else + echo "Could not identify CI type" >&2 + exit 1 +fi diff --git a/ci/lib-tput.sh b/ci/lib-tput.sh new file mode 100644 index 00000000000..b62ef13c44c --- /dev/null +++ b/ci/lib-tput.sh @@ -0,0 +1,5 @@ +#!/bin/sh + +# GitHub Action doesn't set TERM, which is required by tput +TERM=${TERM:-dumb} +export TERM diff --git a/ci/lib.sh b/ci/lib.sh index cae324e2fd7..fea45a1a8d3 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -1,13 +1,9 @@ #!/bin/sh - -# Library of functions shared by all CI scripts - -# Set 'exit on error' for all CI scripts to let the caller know that -# something went wrong. -# Set tracing executed commands, primarily setting environment variables -# and installing dependencies. set -ex +# Helper libraries +. ${0%/*}/lib-ci-type.sh + # Starting assertions if test -z "$jobname" then @@ -47,18 +43,14 @@ setenv () { fi } -# GitHub Action doesn't set TERM, which is required by tput -setenv TERM ${TERM:-dumb} - # How many jobs to run in parallel? NPROC=10 # Clear MAKEFLAGS that may come from the outside world. MAKEFLAGS=--jobs=$NPROC -if test "$GITHUB_ACTIONS" = "true" -then - CI_TYPE=github-actions +case "$CI_TYPE" in +github-actions) CC="${CC:-gcc}" setenv --test GIT_PROVE_OPTS "--timer --jobs $NPROC" @@ -66,11 +58,12 @@ then test Windows != "$RUNNER_OS" || GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS" setenv --test GIT_TEST_OPTS "$GIT_TEST_OPTS" -else - echo "Could not identify CI type" >&2 - env >&2 + ;; +*) + echo "Unhandled CI type: $CI_TYPE" >&2 exit 1 -fi + ;; +esac setenv --build DEVELOPER 1 setenv --test DEFAULT_TEST_TARGET prove diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh index 47e2ea1ad10..29f8c332eca 100755 --- a/ci/print-test-failures.sh +++ b/ci/print-test-failures.sh @@ -3,10 +3,10 @@ # Print output of failing tests # -. ${0%/*}/lib.sh +set -e -# Tracing executed commands would produce too much noise in the loop below. -set +x +. ${0%/*}/lib-ci-type.sh +. ${0%/*}/lib-tput.sh cd t/ diff --git a/ci/select-test-slice.sh b/ci/select-test-slice.sh index f59d8cadda5..eccf9a48104 100755 --- a/ci/select-test-slice.sh +++ b/ci/select-test-slice.sh @@ -3,7 +3,7 @@ # Select a portion of the tests for testing Git in parallel # -. ${0%/*}/lib.sh +set -ex tests=$(echo $(cd t && ./helper/test-tool path-utils slice-tests "$1" "$2" \ t[0-9]*.sh)) diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh index b8a6a6f664e..64ff212cdaa 100755 --- a/ci/test-documentation.sh +++ b/ci/test-documentation.sh @@ -3,7 +3,7 @@ # Perform sanity checks on "make doc" output and built documentation # -. ${0%/*}/lib.sh +set -ex generator=$1 From patchwork Thu Apr 21 18:23:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12822191 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0B5EAC433EF for ; Thu, 21 Apr 2022 18:25:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1391409AbiDUS2I (ORCPT ); Thu, 21 Apr 2022 14:28:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47288 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1391357AbiDUS1R (ORCPT ); Thu, 21 Apr 2022 14:27:17 -0400 Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com [IPv6:2a00:1450:4864:20::32d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 587344B85C for ; Thu, 21 Apr 2022 11:23:59 -0700 (PDT) Received: by mail-wm1-x32d.google.com with SMTP id u17-20020a05600c211100b0038eaf4cdaaeso6433464wml.1 for ; Thu, 21 Apr 2022 11:23:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=FYU1ZRLvlGMdp0h88uObE0FPjs8lVSXFGsK4mEstnxg=; b=jIuaiRwHUAFHxmUYFwXY6gvZRvlAGTZaQEsFMVBCj+U/XYXKVbT5GXIFU1EoJI8tqb LXc7wVvpw0n1+MZRt4q2gWiUnDM1dclparostzWlf5ELlTIdtlRkB455YMyYOlGCScO3 12i+drTZQawWn+yO2g1eUOUUpUQrXaJJdI7cmLEsQ1tDNPq61Ca2bTKjaiK/6PMUmD+U /zniK3QiWp/8oGZfgpz722ISQ1tTfAF9bRKbF13N7DS8iQqs/rMrkQBhPPfev5kSHhKf ae82nrYhBYhOMl2dLdiXAGydNsMz2FiV7t9jPLLwYUcclV3DtgNKySDzdjqFmEIM+jOw nIGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FYU1ZRLvlGMdp0h88uObE0FPjs8lVSXFGsK4mEstnxg=; b=3/QZ4ucbbXrs0Golt5hVCPWOTp/4QH+zgIheu2TPMomxSLaBq/osyrveowlh8bvLC9 TR4OwQEaigdLcHcdNfF28K7erztUyw8FOQWdnzr7ktqEyAhWJ4D1A8M/Vmr1ykxYj8vU gLyYOSno00NfLyBN4ssqX96sOkOuRZzAyLle18Gd1xrHDDM1xk6qpmysk23iNwLzudwt b20XcaHQkoa8rAQoY0Dcnd2bdoULQVyc85swxL83D/BiaWivIUOaZEZCZJiHTIdnqMY5 JlZJrzc5+flEmgcbQaR2Kr8MNO16wPcCqC5vjZAltAN9m25IL8ETwfFguZUhXjt6xNRi DdCA== X-Gm-Message-State: AOAM533NqaivTxvRdK9k7alxFGigrOuql960WhwodzT/VYk2hqZhuLNU XiTt3hXAc20aJ1/t2fG1ApCV8ke3xkX+Cw== X-Google-Smtp-Source: ABdhPJwh471Xudkqhnc53GaCSL5nb3yQtlcMxNoFtbtsD2YAixSo1jhmvnmEoTYtEpbJQoxvDITr5Q== X-Received: by 2002:a05:600c:590:b0:38c:804e:4197 with SMTP id o16-20020a05600c059000b0038c804e4197mr630064wmd.22.1650565437426; Thu, 21 Apr 2022 11:23:57 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k11-20020a5d6d4b000000b0020599079f68sm2909090wri.106.2022.04.21.11.23.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Apr 2022 11:23:56 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , Eric Sunshine , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBC?= =?utf-8?b?amFybWFzb24=?= Subject: [PATCH v5 23/29] CI: narrow down variable definitions in --build and --test Date: Thu, 21 Apr 2022 20:23:18 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.879.g3659959fcca In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In a preceding step the "setvar" function was made to take a "--build", "--test" or "--all" argument to indicate where the variables it sets were used. Let's make use of that by having the relevant parts of ".github/workflows/main.yml" invoke "ci/lib.sh" with those options. By doing this the set of variables shown in build-only steps will be fewer, which makes diagnosing anything going on there easier, as we won't have to look at a deluge of e.g. GIT_TEST_* variables. For the "pedantic" job (which has no test phase) we won't run the "ci/lib.sh --test" step, which will be clearly visible as a skipped step in the UX. Since we'll now always run "--build" for "make" and "--test" for "make test" we can stop setting the "fat" MAKEFLAGS entirely on the "test" steps, i.e. the one with PYTHON_PATH, CC etc. These will all be carried over from the earlier --build step. That won't be true in the case of the "windows-test" and "vs-test" jobs, since they run separately from the corresponding "build" step. So we'll need to make sure that we have the --jobs=N argument for those. This doesn't matter that much, as we'll still have --jobs=N in GIT_PROVE_OPTS. So the only thing we'll use it for is parallelism in the t/Makefile before we get to running "prove". Still, it's good to be consistent for good measure, and to run the t/Makefile itself in parallel. Signed-off-by: Ævar Arnfjörð Bjarmason --- .github/workflows/main.yml | 21 ++++++++------- ci/lib.sh | 52 +++++++++++++++++++++++++++++++++++--- 2 files changed, 60 insertions(+), 13 deletions(-) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 83e0aa1f469..bb62b4ff725 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -84,7 +84,7 @@ jobs: steps: - uses: actions/checkout@v2 - uses: git-for-windows/setup-git-for-windows-sdk@v1 - - run: ci/lib.sh + - run: ci/lib.sh --build shell: bash - name: build shell: bash @@ -122,7 +122,7 @@ jobs: shell: bash run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz - uses: git-for-windows/setup-git-for-windows-sdk@v1 - - run: ci/lib.sh + - run: ci/lib.sh --test shell: bash - name: select tests run: . /etc/profile && ci/select-test-slice.sh ${{matrix.nr}} 10 @@ -169,7 +169,7 @@ jobs: - name: copy dlls to root shell: cmd run: compat\vcbuild\vcpkg_copy_dlls.bat release - - run: ci/lib.sh + - run: ci/lib.sh --build shell: bash - name: generate Visual Studio solution shell: bash @@ -211,7 +211,7 @@ jobs: - name: extract tracked files and build artifacts shell: bash run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz - - run: ci/lib.sh + - run: ci/lib.sh --test shell: bash - name: select tests run: . /etc/profile && ci/select-test-slice.sh ${{matrix.nr}} 10 @@ -275,8 +275,9 @@ jobs: steps: - uses: actions/checkout@v2 - run: ci/install-dependencies.sh - - run: ci/lib.sh + - run: ci/lib.sh --build - run: make + - run: ci/lib.sh --test - run: make test if: success() - run: ci/print-test-failures.sh @@ -310,8 +311,10 @@ jobs: steps: - uses: actions/checkout@v1 - run: ci/install-dependencies.sh - - run: ci/lib.sh + - run: ci/lib.sh --build - run: make + - run: ci/lib.sh --test + if: success() && matrix.vector.skip-tests != 'yes' - run: make test if: success() && matrix.vector.skip-tests != 'yes' - run: ci/print-test-failures.sh @@ -331,7 +334,7 @@ jobs: steps: - uses: actions/checkout@v2 - run: ci/install-dependencies.sh - - run: ci/lib.sh + - run: ci/lib.sh --build - run: make ci-static-analysis sparse: needs: ci-config @@ -352,7 +355,7 @@ jobs: - uses: actions/checkout@v2 - name: Install other dependencies run: ci/install-dependencies.sh - - run: ci/lib.sh + - run: ci/lib.sh --build - run: make sparse documentation: name: documentation @@ -364,7 +367,7 @@ jobs: steps: - uses: actions/checkout@v2 - run: ci/install-dependencies.sh - - run: ci/lib.sh + - run: ci/lib.sh --build - run: make check-docs - run: "make doc > >(tee stdout.log) 2> >(tee stderr.raw >&2)" shell: bash diff --git a/ci/lib.sh b/ci/lib.sh index fea45a1a8d3..c875208817f 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -4,6 +4,30 @@ set -ex # Helper libraries . ${0%/*}/lib-ci-type.sh +# Parse options +mode_build= +mode_test= +while test $# != 0 +do + case "$1" in + --build) + mode_build=t + ;; + --test) + mode_test=t + ;; + -*) + echo "error: invalid option: $1" >&2 + exit 1 + ;; + *) + echo "error: invalid argument: $1" >&2 + exit 1 + ;; + esac + shift +done + # Starting assertions if test -z "$jobname" then @@ -11,16 +35,28 @@ then exit 1 fi +if test "$mode_test$mode_build" != "t" +then + echo "error: need one mode, e.g. --build or --test" >&2 + exit 1 +fi + # Helper functions setenv () { while test $# != 0 do case "$1" in --build) + if test -z "$mode_build" + then + return 0 + fi ;; --test) - ;; - --all) + if test -z "$mode_test" + then + return 0 + fi ;; -*) echo "BUG: bad setenv() option '$1'" >&2 @@ -46,8 +82,12 @@ setenv () { # How many jobs to run in parallel? NPROC=10 +# For "--test" we carry the MAKEFLAGS over from earlier steps, except +# in stand-alone jobs which will use $COMMON_MAKEFLAGS. +COMMON_MAKEFLAGS=--jobs=$NPROC + # Clear MAKEFLAGS that may come from the outside world. -MAKEFLAGS=--jobs=$NPROC +MAKEFLAGS=$COMMON_MAKEFLAGS case "$CI_TYPE" in github-actions) @@ -101,6 +141,9 @@ windows-build) setenv --build NO_PERL NoThanks setenv --build ARTIFACTS_DIRECTORY artifacts ;; +windows-test) + setenv --test MAKEFLAGS "$COMMON_MAKEFLAGS" + ;; vs-build) setenv --build NO_PERL NoThanks setenv --build NO_GETTEXT NoThanks @@ -117,6 +160,7 @@ vs-build) ;; vs-test) setenv --test NO_SVN_TESTS YesPlease + setenv --test MAKEFLAGS "$COMMON_MAKEFLAGS" ;; linux-gcc) setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main @@ -162,4 +206,4 @@ linux-leaks) ;; esac -setenv --all MAKEFLAGS "$MAKEFLAGS CC=${CC:-cc}" +setenv --build MAKEFLAGS "$MAKEFLAGS CC=${CC:-cc}" From patchwork Thu Apr 21 18:23:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12822192 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 300EAC433EF for ; Thu, 21 Apr 2022 18:25:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1391434AbiDUS2L (ORCPT ); Thu, 21 Apr 2022 14:28:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47282 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1391358AbiDUS1R (ORCPT ); Thu, 21 Apr 2022 14:27:17 -0400 Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com [IPv6:2a00:1450:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2DEF04B861 for ; Thu, 21 Apr 2022 11:24:00 -0700 (PDT) Received: by mail-wr1-x433.google.com with SMTP id t6so4408058wra.4 for ; Thu, 21 Apr 2022 11:24:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ON6D6lRakahV8Zg3e4cgHJbiblHbaUon+oceq5fedTU=; b=dDAWrQ4MqujZXc4igqBgdcfuHauRcP06IYW2veHHOqFJNUpJtSmqNjMai89t3NbiNT Lh1hzuo5KpKCs1I/7AEgGhc1LSPZ6uHOK/K+E3r+hCGic1W8n4I4WRP7no4uCP69QVR0 ZuuNEnjUkrhKNrvNEB0QYgyBhlcEbPjsT5phs03iP24Uy+6ZpA9T7UltBwKvTwKxPwFy 2OKWwyd5GO5DdDJk4NLDF3Ej88VYMRcAfaewfLptOyHtJnuGlfPh2ioLOh/u+TAx2WSe m8yWmqOH3nJh2aCiHK+HZz4KnmvQ/mfqYnm5ONV4T0eaWW6pNj+z5k9+7HWxwYOSNI5Z VcUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ON6D6lRakahV8Zg3e4cgHJbiblHbaUon+oceq5fedTU=; b=i5sEqoAoi4+GV68reJZ/b8sNr4gFVWPAZ5p3aINgR05YE3KN68ruI21ei4m7fv59nN L1DwuZ1uX3gq5tUdm1dcGNoC+cv7G6rw9kDkKkjAD45XoRhbWn3tm//TLXJfCNlgm5Em wIWyzJvxZEIgGsVi7zzfykT8h5E1vMAKYH5xqJHbA7m7MRxj0jd+NvEsdBDbk1DsQGf1 7f3ggJA+gfFj/2o7DrijqO5ooCoBGwyPl1qalZpOeQK2AM64MOR59gkHxQlsE0vMLVza tmmluygxpvqwjpnfg4gewI5m6RstjSCr4Lsz+9AmFINq7cl3cMSzkQItkbOQYNUFiqVJ t+Eg== X-Gm-Message-State: AOAM533bV5Un8UwTDCGP5gQdmaxCWB7q/2Ppjz+7rs+nVUKk1DFmHMDV Oa076sJQ0Ai7hH/rbrghlz/RIykElPS0Dw== X-Google-Smtp-Source: ABdhPJw5VtAEmFxolSFGluAlfj1SeqLAHC/6vXO0xM8hV4RJpscVSpGHFkWOJ6fCTfWyTqeC1t7ODg== X-Received: by 2002:a5d:49c1:0:b0:20a:a65b:4648 with SMTP id t1-20020a5d49c1000000b0020aa65b4648mr715746wrs.707.1650565438374; Thu, 21 Apr 2022 11:23:58 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k11-20020a5d6d4b000000b0020599079f68sm2909090wri.106.2022.04.21.11.23.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Apr 2022 11:23:57 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , Eric Sunshine , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBC?= =?utf-8?b?amFybWFzb24=?= Subject: [PATCH v5 24/29] CI: add more variables to MAKEFLAGS, except under vs-build Date: Thu, 21 Apr 2022 20:23:19 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.879.g3659959fcca In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org It's clearer that "DEVELOPER" is a flag that affects the Makefile itself in particular if it's put into "MAKEFLAGS than" if it generically sits in the environment. Let's move both it and "SKIP_DASHED_BUILT_INS" to "MAKEFLAGS". We can't do this under vs-build, since that invokes cmake. Let's have only that job set these in the environment. Signed-off-by: Ævar Arnfjörð Bjarmason --- ci/lib.sh | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/ci/lib.sh b/ci/lib.sh index c875208817f..c8cc37c50b4 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -89,6 +89,13 @@ COMMON_MAKEFLAGS=--jobs=$NPROC # Clear MAKEFLAGS that may come from the outside world. MAKEFLAGS=$COMMON_MAKEFLAGS +# Use common options for "make" (cmake in "vs-build" below uses the +# intermediate variables directly) +DEVELOPER=1 +MAKEFLAGS="$MAKEFLAGS DEVELOPER=$DEVELOPER" +SKIP_DASHED_BUILT_INS=YesPlease +MAKEFLAGS="$MAKEFLAGS SKIP_DASHED_BUILT_INS=$SKIP_DASHED_BUILT_INS" + case "$CI_TYPE" in github-actions) CC="${CC:-gcc}" @@ -105,10 +112,8 @@ github-actions) ;; esac -setenv --build DEVELOPER 1 setenv --test DEFAULT_TEST_TARGET prove setenv --test GIT_TEST_CLONE_2GB true -setenv --build SKIP_DASHED_BUILT_INS YesPlease case "$runs_on_pool" in ubuntu-latest) @@ -145,6 +150,9 @@ windows-test) setenv --test MAKEFLAGS "$COMMON_MAKEFLAGS" ;; vs-build) + setenv --build DEVELOPER $DEVELOPER + setenv --build SKIP_DASHED_BUILT_INS $SKIP_DASHED_BUILT_INS + setenv --build NO_PERL NoThanks setenv --build NO_GETTEXT NoThanks setenv --build ARTIFACTS_DIRECTORY artifacts From patchwork Thu Apr 21 18:23:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12822194 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F1D2FC433FE for ; Thu, 21 Apr 2022 18:25:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1391458AbiDUS2O (ORCPT ); Thu, 21 Apr 2022 14:28:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48600 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1391361AbiDUS1S (ORCPT ); Thu, 21 Apr 2022 14:27:18 -0400 Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com [IPv6:2a00:1450:4864:20::42a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 084094B86A for ; Thu, 21 Apr 2022 11:24:00 -0700 (PDT) Received: by mail-wr1-x42a.google.com with SMTP id s25so2955242wrb.8 for ; Thu, 21 Apr 2022 11:24:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=bmhBKyKyPV6fUgUNIiudhurcU8D5YRUsUDfT8mYYhK0=; b=OTwbZKuqXTNKIKqVDNqrQRufQXQwIXHiZsEf+04mBsmbTKgRreRi7c5E1OeRLZLCvm q3WixNNoYC7g4jrO/pfJlK00AeO7tTUjADvhx3SrMNJXUK3SfDhzWl+9y6Xq4W1PrJoG CICH6BTiPZ9hhP1JJIA5amUhYCz1O36gp6kGxMo0mq/tdxJWRLtIt6Xrry8yet5xFtDN k4iQh6poCNIn7UvxaLgnxDqNG+1RHRvB2KUVEpyRTufczuh16tCjqIKa4cg/sBTiFpj+ i+2OUiC7i3K56X9Dxxg+eoUVPva9M3d/S03/3Yg09jx42fdNsBvjavwGqs/9PrpTQt3c tytA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=bmhBKyKyPV6fUgUNIiudhurcU8D5YRUsUDfT8mYYhK0=; b=PoZ4tgRnUE0j6oa5CFdYmB3vOQ6dcMsgUadBgoBxyFtA5Z3WXeDIpZW7NR28x+z8ge yGjYRNxoHRlldLUvPvJqC1JtdwxppE2lFSz5tURfV1CaZZ/9GqRN2e9QWLmnwC7oRGbg SGyW+/42H2/UI8fz9odl6ACri5e08GlXWd3lZFTf5E68AGkOXDG66uHn+/w2qGzOEEfE 792dwf57HxbEHo6nb0XONyycGlptrMRAxLlbBsO4KLAvgiFkmR56YpIm/1QvZQIv/mFK iLReEwmQwQ8E+9UQpEIuUzMFL+FCB/AWwqwQW0eg8/hjMue+M+krYPux59Ay520iUE3X xQLQ== X-Gm-Message-State: AOAM5316TFgG8AR7Ulm/4Jy0nzlbRRRk3l/3HwMvfyjQOlh4yx46VcyA SZ7GcHfua57+Pr8Ow2BlHJkQTi/G0k5YYA== X-Google-Smtp-Source: ABdhPJx5QuQVIe6hkkUdZoXT3b9jnYQCf/yQ2fjfscVG9On1v/h6T7S7NEPmwFhDw2jvy80rCaB5uA== X-Received: by 2002:adf:e944:0:b0:207:af9e:a4e7 with SMTP id m4-20020adfe944000000b00207af9ea4e7mr742792wrn.296.1650565439285; Thu, 21 Apr 2022 11:23:59 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k11-20020a5d6d4b000000b0020599079f68sm2909090wri.106.2022.04.21.11.23.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Apr 2022 11:23:58 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , Eric Sunshine , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBC?= =?utf-8?b?amFybWFzb24=?= Subject: [PATCH v5 25/29] CI: set CC in MAKEFLAGS directly, don't add it to the environment Date: Thu, 21 Apr 2022 20:23:20 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.879.g3659959fcca In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Rather than pass a "$CC" in the environment to be picked up in ci/lib.sh let's instead have ci/lib.sh itself add it directly to MAKEFLAGS. For "$CC_PACKAGE" its setting and use can stay within ci/install-dependencies.sh. Setting CC=gcc by default made for confusing trace output, and since a preceding change to carry it and others over across "steps" in the GitHub CI it's been even more misleading. E.g. the "win+VS build" job confusingly has CC=gcc set, even though it builds with MSVC. Let's instead rely on the Makefile default of CC=cc, and only override it for those jobs where it's needed. This does mean that we'll need to set it for the "pedantic" job, which previously relied on the default CC=gcc in case "clang" become the default on that platform. This partially reverts my 707d2f2fe86 (CI: use "$runs_on_pool", not "$jobname" to select packages & config, 2021-11-23), i.e. we're now aiming to only set those variables specific jobs need. Since we don't need to make this generic enough to handle "gcc-9" turning into "gcc@9" for "brew install" let's remove what was originally a bash-specific replacement in 707d2f2fe86 (CI: use "$runs_on_pool", not "$jobname" to select packages & config, 2021-11-23), and which a preceding commit changed to a "tr" invocation. Signed-off-by: Ævar Arnfjörð Bjarmason --- .github/workflows/main.yml | 13 ------------- ci/install-dependencies.sh | 18 ++++++++++++++---- ci/lib.sh | 23 ++++++++++++++++++++--- 3 files changed, 34 insertions(+), 20 deletions(-) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index bb62b4ff725..6835e942280 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -238,37 +238,24 @@ jobs: matrix: vector: - jobname: linux-clang - cc: clang pool: ubuntu-latest - jobname: linux-sha256 - cc: clang os: ubuntu pool: ubuntu-latest - jobname: linux-gcc - cc: gcc - cc_package: gcc-8 pool: ubuntu-latest - jobname: linux-TEST-vars - cc: gcc os: ubuntu - cc_package: gcc-8 pool: ubuntu-latest - jobname: osx-clang - cc: clang pool: macos-latest - jobname: osx-gcc - cc: gcc - cc_package: gcc-9 pool: macos-latest - jobname: linux-gcc-default - cc: gcc pool: ubuntu-latest - jobname: linux-leaks - cc: gcc pool: ubuntu-latest env: - CC: ${{matrix.vector.cc}} - CC_PACKAGE: ${{matrix.vector.cc_package}} jobname: ${{matrix.vector.jobname}} runs_on_pool: ${{matrix.vector.pool}} runs-on: ${{matrix.vector.pool}} diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh index 4b4cdbe26e0..65548bf0b25 100755 --- a/ci/install-dependencies.sh +++ b/ci/install-dependencies.sh @@ -9,6 +9,17 @@ UBUNTU_COMMON_PKGS="make libssl-dev libcurl4-openssl-dev libexpat-dev tcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl libemail-valid-perl libio-socket-ssl-perl libnet-smtp-ssl-perl" +CC_PACKAGE= +BREW_CC_PACKAGE= +case "$jobname" in +linux-gcc | linux-TEST-vars) + CC_PACKAGE=gcc-8 + ;; +osx-gcc) + BREW_CC_PACKAGE=gcc@9 + ;; +esac + case "$runs_on_pool" in ubuntu-latest) # The Linux build installs the defined dependency versions below. @@ -67,11 +78,10 @@ macos-latest) } || brew install homebrew/cask/perforce - if test -n "$CC_PACKAGE" + if test -n "$BREW_CC_PACKAGE" then - BREW_PACKAGE=$(echo $CC_PACKAGE | tr '-' '@') - brew install "$BREW_PACKAGE" - brew link "$BREW_PACKAGE" + brew install "$BREW_CC_PACKAGE" + brew link "$BREW_CC_PACKAGE" fi ;; esac diff --git a/ci/lib.sh b/ci/lib.sh index c8cc37c50b4..8d19ca5ced8 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -79,6 +79,9 @@ setenv () { fi } +# Clear variables that may come from the outside world. +CC= + # How many jobs to run in parallel? NPROC=10 @@ -98,8 +101,6 @@ MAKEFLAGS="$MAKEFLAGS SKIP_DASHED_BUILT_INS=$SKIP_DASHED_BUILT_INS" case "$CI_TYPE" in github-actions) - CC="${CC:-gcc}" - setenv --test GIT_PROVE_OPTS "--timer --jobs $NPROC" GIT_TEST_OPTS="--verbose-log -x" test Windows != "$RUNNER_OS" || @@ -171,9 +172,14 @@ vs-test) setenv --test MAKEFLAGS "$COMMON_MAKEFLAGS" ;; linux-gcc) + CC=gcc setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main ;; +linux-gcc-default) + CC=gcc + ;; linux-TEST-vars) + CC=gcc setenv --test GIT_TEST_SPLIT_INDEX yes setenv --test GIT_TEST_MERGE_ALGORITHM recursive setenv --test GIT_TEST_FULL_IN_PACK_ARRAY true @@ -188,13 +194,22 @@ linux-TEST-vars) setenv --test GIT_TEST_WRITE_REV_INDEX 1 setenv --test GIT_TEST_CHECKOUT_WORKERS 2 ;; +osx-gcc) + CC=gcc + ;; +osx-clang) + CC=clang + ;; linux-clang) + CC=clang setenv --test GIT_TEST_DEFAULT_HASH sha1 ;; linux-sha256) + CC=clang setenv --test GIT_TEST_DEFAULT_HASH sha256 ;; pedantic) + CC=gcc # Don't run the tests; we only care about whether Git can be # built. setenv --build DEVOPTS pedantic @@ -209,9 +224,11 @@ linux-musl) MAKEFLAGS="$MAKEFLAGS GIT_TEST_UTF8_LOCALE=C.UTF-8" ;; linux-leaks) + CC=gcc setenv --build SANITIZE leak setenv --test GIT_TEST_PASSING_SANITIZE_LEAK true ;; esac -setenv --build MAKEFLAGS "$MAKEFLAGS CC=${CC:-cc}" +MAKEFLAGS="$MAKEFLAGS${CC:+ CC=$CC}" +setenv --build MAKEFLAGS "$MAKEFLAGS" From patchwork Thu Apr 21 18:23:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12822193 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9E2CEC433F5 for ; Thu, 21 Apr 2022 18:25:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1391444AbiDUS2M (ORCPT ); Thu, 21 Apr 2022 14:28:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47470 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1391370AbiDUS1S (ORCPT ); Thu, 21 Apr 2022 14:27:18 -0400 Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com [IPv6:2a00:1450:4864:20::330]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 10F614B878 for ; Thu, 21 Apr 2022 11:24:02 -0700 (PDT) Received: by mail-wm1-x330.google.com with SMTP id q20so3715033wmq.1 for ; Thu, 21 Apr 2022 11:24:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=mJ/p4NFke3CntaRD5Ptns/VbtFyum0knDUI2LOiVivM=; b=Ty28r8ohRSiOEvG5AYmubGXhWDt6WEEtKfkHRqchKo4GpBRleuRjkXWZ8yAtMil1yG M33qz6uPTFcr/D2QX80FUmww6A2supLKwuuUagwmtv10GuwxbZk8AHXpsI/VGupinozA tAHzmDwaF4TnMPPXv4zENDAxPG5h12mYy+cOwbbNDLxCH1hwULgy9l2JGnSfs0UeKm0s 54T8R4hvC6asg4lfAspbzzZAd0Fi9TYDV9QD0EM+qqmV8o9mzSnz7agj6/Lt5iWHtbmj VOVP6wtemrkubfXB1CWz67h5X9U8xI9rvifg/2WWz5djIVGexojXGAmW2pPgOLdNqrog SWmQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=mJ/p4NFke3CntaRD5Ptns/VbtFyum0knDUI2LOiVivM=; b=EhrAaYZ3QOZlu1BE2uCX6WowH9oj081XOgisX99XW9XXpl/Trr0fiMCM671cnJC2rJ c1EQY1p/IgNnCThsV22STC7y5fmzJxHCNWkIShtrWN79SALAVmY8M/kvpHnUnlc8pl/P Ws1ndXAmGGrJNPZ6D5HxS89RVnC5qgiq4YFge/mNav663E3CFdQnTHlaZWlejBSPXDLL Baa2MqB33H10NeKBGF5sZTntTIo0652kBdn27VN1HAYEByAgQSGsbNUqHJUF4evr17il 2GQ4Kp0GNdoT/pmMLCFHiPgYggNkr68O+rOvIrQr0yvFnAmFtCE5pJ74FBdsVAElN0Xm ardw== X-Gm-Message-State: AOAM533m+1F01eyhgY34YYOWs6HPqDSuUK7ocwqN6548pkLOD1o0c1ND OyiaqrL75a8ww5CLx0m7fvBR86QP3x/ewA== X-Google-Smtp-Source: ABdhPJydDXpwGNtcYIYsFbLYMiWU5Sv09BkiDyDJ63WTdcizEZwJSMKkL4vl7hJNVop8CR/821jOtw== X-Received: by 2002:a1c:3587:0:b0:381:50ff:cbd with SMTP id c129-20020a1c3587000000b0038150ff0cbdmr9933131wma.140.1650565440186; Thu, 21 Apr 2022 11:24:00 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k11-20020a5d6d4b000000b0020599079f68sm2909090wri.106.2022.04.21.11.23.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Apr 2022 11:23:59 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , Eric Sunshine , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBC?= =?utf-8?b?amFybWFzb24=?= Subject: [PATCH v5 26/29] CI: set SANITIZE=leak in MAKEFLAGS directly Date: Thu, 21 Apr 2022 20:23:21 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.879.g3659959fcca In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org As in the preceding commit's change to do likewise for CC, let's not set SANITIZE in the environment. When I added the "linux-leaks" job in in 956d2e4639b (tests: add a test mode for SANITIZE=leak, run it in CI, 2021-09-23) I missed that I could do that. Signed-off-by: Ævar Arnfjörð Bjarmason --- ci/lib.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ci/lib.sh b/ci/lib.sh index 8d19ca5ced8..b6e7a6647bb 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -225,7 +225,7 @@ linux-musl) ;; linux-leaks) CC=gcc - setenv --build SANITIZE leak + MAKEFLAGS="$MAKEFLAGS SANITIZE=leak" setenv --test GIT_TEST_PASSING_SANITIZE_LEAK true ;; esac From patchwork Thu Apr 21 18:23:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12822198 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F05F8C433F5 for ; Thu, 21 Apr 2022 18:25:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1391378AbiDUS2b (ORCPT ); Thu, 21 Apr 2022 14:28:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47108 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1391380AbiDUS1S (ORCPT ); Thu, 21 Apr 2022 14:27:18 -0400 Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com [IPv6:2a00:1450:4864:20::435]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 074FF4BB84 for ; Thu, 21 Apr 2022 11:24:03 -0700 (PDT) Received: by mail-wr1-x435.google.com with SMTP id u3so7858447wrg.3 for ; Thu, 21 Apr 2022 11:24:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=dzKGPpsweXetdh8icuSxBnF8XbGz6Oc+HLv24lUro1c=; b=EhOSSe4IStKkXoBVUP+gm3/Weg+pXVR0i9X0XY9gX6D0IzUBaGlJz/znGxJrxtBkFx N4PCoSgzcjF2sMtTwODDokhTiMLXAhEVZKdTJ0NCtbDTBWrWTSLg6WUnTPm8ekqvrCx9 L+NsXxhVnsxFi5BWFZ1eoQWWLZ8DTUmwl5ZMrQ+jm7nAJhw5e3mQgGlSoTGiY4mmP3gq BeuYDJvxBzF5iLeHNXiemc11OBeFV2RzmcfWMprkljkMQR553mFhqUV0SRoXNHhT+XJE VT2Y+rUsFZdkVWXYWvU3QL1j6FE+rb4x4qSjST0rihpA09vtpI6CWHx4JTyPJPQ7ZfX3 7NKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=dzKGPpsweXetdh8icuSxBnF8XbGz6Oc+HLv24lUro1c=; b=D8lKspBkPt0SsyvthEwtDMfvoYd1ENBO+JQg9P4AHKK20jyIMfVfnYQ/rDVE3yDI1R lz2m2tmm8lKV2royzedDC/S9WCHNSCxZCB+OKFrp5TKQOpmwpbYasd3kXpCdxGhKU46x R2Rut/kHn9AGkVkznSqBw08BxLI6c0Zfj5q7oWlh2GGpoVQ3e1hPGRfKnXA3Js4ZevMR uHtl3T/b/x/Q/KtVoArag6+3TwfOGx/e7vCZQ7KMLD/4I1zHjSXHunL79nDX38bagX6+ 3zYTFjVF5kIsaQK1epQdhK7zhllVMihDSwFUlvxqFBtaKKPxUchI0CgEb0PugZIYxcdE l0eg== X-Gm-Message-State: AOAM532kgm1j6/FEvTonlkeE2br/KToYdfTvTK1vz12bEpNuMaKwnfOs 5/zaLM8aRE/J9rMZFj7JBcZWLR9v3h9raA== X-Google-Smtp-Source: ABdhPJxINUEVvy57XxZ3vdO0sjq9E0FhsKPXMhpYBIh8oHM6eooJ/dlj/YWvd5/t0BSOZ19Jbf2G/g== X-Received: by 2002:a5d:6488:0:b0:203:b628:70d2 with SMTP id o8-20020a5d6488000000b00203b62870d2mr731970wri.83.1650565441224; Thu, 21 Apr 2022 11:24:01 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k11-20020a5d6d4b000000b0020599079f68sm2909090wri.106.2022.04.21.11.24.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Apr 2022 11:24:00 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , Eric Sunshine , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBC?= =?utf-8?b?amFybWFzb24=?= Subject: [PATCH v5 27/29] CI: set PYTHON_PATH setting for osx-{clang,gcc} into "$jobname" case Date: Thu, 21 Apr 2022 20:23:22 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.879.g3659959fcca In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Move the setting of "PYTHON_PATH=[...]" to the "$jobname" case statement. This partially backs out of my 707d2f2fe86 (CI: use "$runs_on_pool", not "$jobname" to select packages & config, 2021-11-23), now that we have a "osx-{clang,gcc}" anyway for setting "$CC" we might as well do away with this part of the "$runs_on_pool" case. Signed-off-by: Ævar Arnfjörð Bjarmason --- ci/lib.sh | 10 ++-------- 1 file changed, 2 insertions(+), 8 deletions(-) diff --git a/ci/lib.sh b/ci/lib.sh index b6e7a6647bb..f2f75901a9c 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -132,14 +132,6 @@ ubuntu-latest) setenv --test GIT_TEST_HTTPD true ;; -macos-latest) - if test "$jobname" = osx-gcc - then - MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)" - else - MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python2)" - fi - ;; esac case "$jobname" in @@ -195,9 +187,11 @@ linux-TEST-vars) setenv --test GIT_TEST_CHECKOUT_WORKERS 2 ;; osx-gcc) + MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)" CC=gcc ;; osx-clang) + MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python2)" CC=clang ;; linux-clang) From patchwork Thu Apr 21 18:23:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12822195 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 10DD2C433F5 for ; Thu, 21 Apr 2022 18:25:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1391353AbiDUS2Q (ORCPT ); Thu, 21 Apr 2022 14:28:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47124 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1391381AbiDUS1S (ORCPT ); Thu, 21 Apr 2022 14:27:18 -0400 Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com [IPv6:2a00:1450:4864:20::331]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 04F594BB8C for ; Thu, 21 Apr 2022 11:24:04 -0700 (PDT) Received: by mail-wm1-x331.google.com with SMTP id y21so3716324wmi.2 for ; Thu, 21 Apr 2022 11:24:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=fzZfrtxis8Ant9G/3l523Eo/Io6+LmmRdUb5KEOYm3U=; b=b53wub/vu+pDpp+hxXmae0tyfnUTIRRDHznXdF9zVLEflyiQ195Blmf8o6VUQUdVI4 EVbXe0+QyqMBleotbFhEMtJR6gU/ToCVISSDWAskAgiu7xLxAyofFgia268ck3j2wsg1 jZN12ASbHTGR1PdIKK/cn9dXbWcoZiyOG2KSyWcExQqJAzt7FCq+cgMYbZtszpo/O/yb ExkLTeC58zbBeMmBjIkTodw95HvMDKk8oqe/qoPzBPMk4Nx/AFYduDRmsH3HUBis+w1Y tX8/zp9c91XVJ7GQvmDTKpcaSRwqPYUMoApmsUmHKZ+OWWZWFiRX3QwI087e9ZCUvIKm fOGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fzZfrtxis8Ant9G/3l523Eo/Io6+LmmRdUb5KEOYm3U=; b=un/CIG11ioLX1xm365IYwy+u7THVoHjljzC1rW6aW2XzE9Q56q+vH6O6BuVwLGMGJE 9VgnrvPuBBsOQMxDpEplDQXHa2H+QANWpr2/pGDrM/X/jJM1hFI0FT8IHluq4SYquG0C qkDrphdyFaVXTQUrC/aqHjOu4YgE/p6Fd29gGgXZWdmVFM8i/MEVRhP3dWDmpBQqXpOg qPU2oofwoY8Fx99CyCC5ZV4FG3UjJc0kD8FK3r2CeExca5pL3WFqJVGk0NRYvfTapu84 Pdp0oq+fprza9XZea6fStHNKe6sVtztN3Q56+s/FUMYsIBjsyUb4uIdk64W5E4j/eZ1v yhgA== X-Gm-Message-State: AOAM5307Fjsivezsu62VAeuRL0rVHY7jijgh0Gli2qByMwdvhra6rW6k D81IvQCyMR5c9xvBzyEvJL/kqPu4G3tKwQ== X-Google-Smtp-Source: ABdhPJyGYnt+BKEIWnqjBl87onruYjaNvbYgrOdJ0ySHnfFQmRzSc1hoIfbXXDUlfxGc7CC/ZXAhEQ== X-Received: by 2002:a05:600c:6020:b0:38e:c006:f55f with SMTP id az32-20020a05600c602000b0038ec006f55fmr552007wmb.123.1650565442294; Thu, 21 Apr 2022 11:24:02 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k11-20020a5d6d4b000000b0020599079f68sm2909090wri.106.2022.04.21.11.24.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Apr 2022 11:24:01 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , Eric Sunshine , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBC?= =?utf-8?b?amFybWFzb24=?= Subject: [PATCH v5 28/29] CI: don't use "set -x" in "ci/lib.sh" output Date: Thu, 21 Apr 2022 20:23:23 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.879.g3659959fcca In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Remove the "set -x" from the "ci/lib.sh" output. Before preceding commits the logic in that file was much more complex, and likely to fail in some scenarios. Now we only task "ci/lib.sh" with setting various variables for subsequent steps in our jobs, so we can start emitting more tailored debugging output, which makes what it's doing easier to read. This change also changes the output of the "ci/print-test-failures.sh" script, since it's the only other user of "ci/lib-ci-type.sh". In that case it's also helpful to know what "$CI_TYPE" we're using, as that script doesn't "set -x" and will act differently depending on the $CI_TYPE. It's useful when developing this script to see what variables it *doesn't* set, but such output isn't useful to users, so let's hide the "SKIP" output behind a --debug option. Signed-off-by: Ævar Arnfjörð Bjarmason --- ci/lib.sh | 35 ++++++++++++++++++++++++++++++++--- 1 file changed, 32 insertions(+), 3 deletions(-) diff --git a/ci/lib.sh b/ci/lib.sh index f2f75901a9c..9e38c893890 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -1,5 +1,5 @@ #!/bin/sh -set -ex +set -e # Helper libraries . ${0%/*}/lib-ci-type.sh @@ -7,15 +7,22 @@ set -ex # Parse options mode_build= mode_test= +mode_debug= while test $# != 0 do case "$1" in --build) + echo "MODE: $1" >&2 mode_build=t ;; --test) + echo "MODE: $1" >&2 mode_test=t ;; + --debug) + echo "DEBUG: $1" >&2 + mode_debug=t + ;; -*) echo "error: invalid option: $1" >&2 exit 1 @@ -41,21 +48,32 @@ then exit 1 fi +# Show our configuration +echo "CONFIG: CI_TYPE=$CI_TYPE" >&2 +echo "CONFIG: jobname=$jobname" >&2 +echo "CONFIG: runs_on_pool=$runs_on_pool" >&2 +if test -n "$GITHUB_ENV" +then + echo "CONFIG: GITHUB_ENV=$GITHUB_ENV" >&2 +fi +echo >&2 + # Helper functions setenv () { + skip= while test $# != 0 do case "$1" in --build) if test -z "$mode_build" then - return 0 + skip=t fi ;; --test) if test -z "$mode_test" then - return 0 + skip=t fi ;; -*) @@ -73,10 +91,21 @@ setenv () { val=$2 shift 2 + if test -n "$skip" + then + if test -n "$mode_debug" + then + echo "SKIP: '$key=$val'" >&2 + fi + return 0 + fi + if test -n "$GITHUB_ENV" then echo "$key=$val" >>"$GITHUB_ENV" fi + + echo "SET: '$key=$val'" >&2 } # Clear variables that may come from the outside world. From patchwork Thu Apr 21 18:23:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12822199 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 82727C433FE for ; Thu, 21 Apr 2022 18:25:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1391384AbiDUS2d (ORCPT ); Thu, 21 Apr 2022 14:28:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48982 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1391319AbiDUS1c (ORCPT ); Thu, 21 Apr 2022 14:27:32 -0400 Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com [IPv6:2a00:1450:4864:20::42a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F21494BB95 for ; Thu, 21 Apr 2022 11:24:04 -0700 (PDT) Received: by mail-wr1-x42a.google.com with SMTP id e2so1563879wrh.7 for ; Thu, 21 Apr 2022 11:24:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=oz0mdEw13vDLO+FgHBYEq0ULOnS9kZiQpvR8Lco819s=; b=UA6s/c0Gixgd2/jmc1v4FTpNHzWSG1RuiWRs5ia9jV7JEoOQ7u4Vv5i5/J5flHi0my 6Za376qaBvlOIREfeVtMT6TFEh9F+F7NSwbyu06c+zSwde7kQpDBVd7sB3BNzeWkH6Ux lJkVqWygSTR+i5Jvxhh4sM87vprmzotb4/wnuOxYaHUvOdIEhgJuWeFwywzeu0X6ofsQ /O1ND0aIN3mIhs0BnTSpsjTw0qEl72LvtneXjy58+p7219/cDXJ4Ese6ZeJ0qinx3wza T9Tg90gI53UGf5jvfzf0DxqqzKFkou3IjwGmeNYvDbHGa28D9F5faPwCsMG3gp74xOHt yEyw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=oz0mdEw13vDLO+FgHBYEq0ULOnS9kZiQpvR8Lco819s=; b=z6BCTu1ZKiVYVOaA7M4cJVKr9IuYSI6PDla+jo0a3T6EBNWvFpWR8Zax85Rs8J457H UbL9vBFb0FxNOSgwbYTUzAFsH5qrjYhsWd84Y2UvjgP0893kotnTsRliqlQft089wwcO 2AyIhQEqiMezQ0KpsspCgTL3f/6bjXL6pUTWbLfUojsijvUO+K5HRyIuNcZ7iDUIehGu ugTcPlACx7oTZk2nmxP5ZNVJuf9EcZTRHGcDW6ylvDw9zHHodmTZYmDI8C0AHiwhC2+6 6luxjNvdnKnKeyMCCMcytFWCH/XKrlIyfhnUGNvnQnFDKH5XMBCDpENTNNhol/RbMmp2 4ANg== X-Gm-Message-State: AOAM530KcGbUbpIICHD0PtOiHYcSSeGubfQ+ubBXmrNlrTGVFHxs8lkb URcJUIFQwjfj3L0f4B/NjqIaULyMEyv8ZQ== X-Google-Smtp-Source: ABdhPJx5mfhOP0EzFDt/1TiQFgqewhVZCgOncJVlDrQ1Qtf7L8/YL/lmWXUx97ktJECiQIrnaBx5wA== X-Received: by 2002:a5d:64c4:0:b0:20a:8f02:1fe6 with SMTP id f4-20020a5d64c4000000b0020a8f021fe6mr750195wri.15.1650565443279; Thu, 21 Apr 2022 11:24:03 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k11-20020a5d6d4b000000b0020599079f68sm2909090wri.106.2022.04.21.11.24.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Apr 2022 11:24:02 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , Eric Sunshine , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBC?= =?utf-8?b?amFybWFzb24=?= Subject: [PATCH v5 29/29] CI: make it easy to use ci/*.sh outside of CI Date: Thu, 21 Apr 2022 20:23:24 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.879.g3659959fcca In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In preceding commits the ci/.sh scripts have lost most of their CI-specific assumptions. Let's go even further and explicitly support running ci/lib.sh outside of CI. This was possible before by faking up enough CI-specific variables, but as shown in the new "help" output being added here using the ci/lib.sh to provide "CI-like" has now become trivial. The ci/print-test-failures.sh scripts can now be used outside of CI as well, the only GitHub CI-specific part is now guarded by a check that we'll pass if outside of GitHub CI. There's also a special-case here to not clobber $MAKEFLAGS in the environment if we're outside of CI, in case the user has e.g. "jN" or other flags to "make" that they'd prefer configured already. Using "ci/lib.sh" as a stand-alone script is much more useful if it doesn't hardcode NPROC=10, let's provide a poor shellscript replacement for the online_cpus() we have in thread-utils.c to cover the most common OS's. Helped-by: Eric Sunshine Signed-off-by: Ævar Arnfjörð Bjarmason --- ci/lib-ci-type.sh | 3 -- ci/lib-online_cpus.sh | 26 +++++++++++++ ci/lib.sh | 80 ++++++++++++++++++++++++++++++++++----- ci/print-test-failures.sh | 4 -- 4 files changed, 96 insertions(+), 17 deletions(-) create mode 100644 ci/lib-online_cpus.sh diff --git a/ci/lib-ci-type.sh b/ci/lib-ci-type.sh index 6f01fd9e5d9..09acab7aaec 100644 --- a/ci/lib-ci-type.sh +++ b/ci/lib-ci-type.sh @@ -3,7 +3,4 @@ if test "$GITHUB_ACTIONS" = "true" then CI_TYPE=github-actions -else - echo "Could not identify CI type" >&2 - exit 1 fi diff --git a/ci/lib-online_cpus.sh b/ci/lib-online_cpus.sh new file mode 100644 index 00000000000..7cf0cd042fa --- /dev/null +++ b/ci/lib-online_cpus.sh @@ -0,0 +1,26 @@ +#!/bin/sh + +# TODO: Ideally we'd compile t/helper/test-online-cpus.c early, but +# that currently presents a chicken & egg problem. We need this before +# we build (much of) anything. +online_cpus() { + NPROC= + + if command -v nproc >/dev/null + then + # GNU coreutils + NPROC=$(nproc) + elif command -v sysctl >/dev/null + then + # BSD & Mac OS X + NPROC=$(sysctl -n hw.ncpu) + elif test -n "$NUMBER_OF_PROCESSORS" + then + # Windows + NPROC="$NUMBER_OF_PROCESSORS" + else + NPROC=1 + fi + + echo $NPROC +} diff --git a/ci/lib.sh b/ci/lib.sh index 9e38c893890..80e89f89b7f 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -1,16 +1,55 @@ #!/bin/sh set -e +# Usage +CI_TYPE_HELP_COMMANDS=' + # run "make all test" like the "linux-leaks" job + (eval $(jobname=linux-leaks ci/lib.sh --all) && make test) + + # run "make all test" like the "linux-musl" job + (eval $(jobname=linux-musl ci/lib.sh --all) && make test) + + # run "make test" like the "linux-TEST-vars" job (uses various GIT_TEST_* modes) + make && (eval $(jobname=linux-TEST-vars ci/lib.sh --test) && make test) + + # run "make test" like the "linux-sha256" job + make && (eval $(jobname=linux-sha256 ci/lib.sh --test) && make test) +' + +CI_TYPE_HELP=" +Running $0 outside of CI? You can use ci/lib.sh to set up your +environment like a given CI job. E.g.: +$CI_TYPE_HELP_COMMANDS +Note that some of these (e.g. the linux-musl one) may not work as +expected due to the CI job configuring a platform that may not match +yours." + +usage() { + echo "usage: jobname= [runs_on_pool=] $0 (--build | --test)" >&2 + if test -z "$CI_TYPE" + then + echo "$CI_TYPE_HELP" >&2 + fi + exit 1 +} + # Helper libraries . ${0%/*}/lib-ci-type.sh # Parse options +mode_all= mode_build= mode_test= mode_debug= while test $# != 0 do case "$1" in + --all) + echo "MODE: $1" >&2 + mode_all=t + mode_build=t + mode_test=t + ;; --build) echo "MODE: $1" >&2 mode_build=t @@ -25,11 +64,11 @@ do ;; -*) echo "error: invalid option: $1" >&2 - exit 1 + usage ;; *) echo "error: invalid argument: $1" >&2 - exit 1 + usage ;; esac shift @@ -39,13 +78,13 @@ done if test -z "$jobname" then echo "error: must set a CI jobname in the environment" >&2 - exit 1 + usage fi -if test "$mode_test$mode_build" != "t" +if test "$mode_test$mode_build" != "t" && test -z "$mode_all" then echo "error: need one mode, e.g. --build or --test" >&2 - exit 1 + usage fi # Show our configuration @@ -65,13 +104,13 @@ setenv () { do case "$1" in --build) - if test -z "$mode_build" + if test -z "$mode_build$mode_all" then skip=t fi ;; --test) - if test -z "$mode_test" + if test -z "$mode_test$mode_all" then skip=t fi @@ -103,6 +142,10 @@ setenv () { if test -n "$GITHUB_ENV" then echo "$key=$val" >>"$GITHUB_ENV" + elif test -z "$CI_TYPE" + then + echo "$key=\"$val\"" + echo "export $key" fi echo "SET: '$key=$val'" >&2 @@ -113,10 +156,25 @@ CC= # How many jobs to run in parallel? NPROC=10 +case "$CI_TYPE" in +'') + . ${0%/*}/lib-online_cpus.sh + NPROC=$(online_cpus) + + if test -n "$MAKEFLAGS" + then + COMMON_MAKEFLAGS="$MAKEFLAGS" + else + COMMON_MAKEFLAGS=--jobs=$NPROC + fi + ;; +*) + # For "--test" we carry the MAKEFLAGS over from earlier steps, except + # in stand-alone jobs which will use $COMMON_MAKEFLAGS. + COMMON_MAKEFLAGS=--jobs=$NPROC + ;; +esac -# For "--test" we carry the MAKEFLAGS over from earlier steps, except -# in stand-alone jobs which will use $COMMON_MAKEFLAGS. -COMMON_MAKEFLAGS=--jobs=$NPROC # Clear MAKEFLAGS that may come from the outside world. MAKEFLAGS=$COMMON_MAKEFLAGS @@ -136,6 +194,8 @@ github-actions) GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS" setenv --test GIT_TEST_OPTS "$GIT_TEST_OPTS" ;; +'') + ;; *) echo "Unhandled CI type: $CI_TYPE" >&2 exit 1 diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh index 29f8c332eca..ec395c79ccd 100755 --- a/ci/print-test-failures.sh +++ b/ci/print-test-failures.sh @@ -36,10 +36,6 @@ do cp "${TEST_EXIT%.exit}.out" failed-test-artifacts/ tar czf failed-test-artifacts/"$test_name".trash.tar.gz "$trash_dir" ;; - *) - echo "Unhandled CI type: $CI_TYPE" >&2 - exit 1 - ;; esac fi done