From patchwork Wed Apr 13 19:51:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12812552 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EC9F2C433FE for ; Wed, 13 Apr 2022 19:52:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237731AbiDMTyW (ORCPT ); Wed, 13 Apr 2022 15:54:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58040 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232808AbiDMTyS (ORCPT ); Wed, 13 Apr 2022 15:54:18 -0400 Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com [IPv6:2a00:1450:4864:20::336]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0C75D2FFED for ; Wed, 13 Apr 2022 12:51:56 -0700 (PDT) Received: by mail-wm1-x336.google.com with SMTP id q8so1527041wmc.0 for ; Wed, 13 Apr 2022 12:51:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=4AT93aekwZkrFPALjFFzKF49BVsnvhcFhM5GfPgohGI=; b=es3uANGuCpeXLxiivy9CPOW6K4IY+g8/n67cYQmrfy0qMyPnp7MbmSFL4oNZgE0UxW vMLDlMPuiQ5UoSG4Nh0hnOBrzd3oZv5nSAoGnsep0CkZu10jDzg9GD2hMhJF6Q8qKFcA U7Uvb57tGUke3aJn9zsECte4+h6rmZ5KLAKQGOr40uJmQCZIGzkFQPbjOBLGYMxqt2tw JIJZhmIYSM9cA9+IvaI9Dt58WDrd0qvyk/ivo3HJ7YGgxGF/rsLeIWe0R7BcdgWM1kI/ PM97KNqMOTcsHuVf5ugzG1LsSV/yy8/55PuiqbePxgzwDqPEX/BMTLCgj3gmWu6TVpAc g0ng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=4AT93aekwZkrFPALjFFzKF49BVsnvhcFhM5GfPgohGI=; b=o1PAp255VHvAW7VU5WczKYrU+LNHN/7H8xwq2rdeXhMGxlYnoT/B3gcxUqVcSSV9ii e7K+DUgglTW92hF1wwDgjuXVocnYAHNs0ogZUNMT3IjCD+Nf3dEXX3UowrwD5Yx7VX4Y CJhAReo22VaJMZ9dNztq6Gzh/2nGLOIm1XbsxZ6EzEXlzTtQnTg/OBehPwI5/Yyxr6ps T9IP5vBokLl/N3hT0bklW7n/p4LjTCDvD3nwJC+djdXitlblSdzRLZZoCBNmAcDIzU/2 BIoKHBzvXKk47mW/3qH/7HDMfcQnZZOkuWzSR7l0jF478Q9Jwa3dtKyPw6eae7ObMBy4 aYhA== X-Gm-Message-State: AOAM532XHL8nWfaAWCjOs2SipwaRTOsK8RaT4Y5pV+fzs80I8xlxGiG9 eUt42PPLDVGJS5y+ccBAri6aS7C+Qr1lCw== X-Google-Smtp-Source: ABdhPJy7cC6kYFYRjZ4vftrpkTwHKB/FmnBJQDc/s2wPhNJ4R9hiJtdDVMJyA4HC77uGe7fsFYu/rQ== X-Received: by 2002:a05:600c:5122:b0:38e:bf28:58d8 with SMTP id o34-20020a05600c512200b0038ebf2858d8mr258541wms.107.1649879514396; Wed, 13 Apr 2022 12:51:54 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k20-20020a05600c1c9400b0038ecd1ccc17sm3432484wms.35.2022.04.13.12.51.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Apr 2022 12:51:53 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v3 01/29] CI: run "set -ex" early in ci/lib.sh Date: Wed, 13 Apr 2022 21:51:14 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.rc2.843.g193535c2aa7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Change ci/lib.sh to run "set -ex" before it does anything else, before this we were omitting a few "export" commands in our trace output that happened before we enabled tracing. Signed-off-by: Ævar Arnfjörð Bjarmason --- ci/lib.sh | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/ci/lib.sh b/ci/lib.sh index cbc2f8f1caa..57141d38a85 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -1,5 +1,11 @@ # Library of functions shared by all CI scripts +# Set 'exit on error' for all CI scripts to let the caller know that +# something went wrong. +# Set tracing executed commands, primarily setting environment variables +# and installing dependencies. +set -ex + skip_branch_tip_with_tag () { # Sometimes, a branch is pushed at the same time the tag that points # at the same commit as the tip of the branch is pushed, and building @@ -85,12 +91,6 @@ export TERM=${TERM:-dumb} # Clear MAKEFLAGS that may come from the outside world. export MAKEFLAGS= -# Set 'exit on error' for all CI scripts to let the caller know that -# something went wrong. -# Set tracing executed commands, primarily setting environment variables -# and installing dependencies. -set -ex - if test -n "$SYSTEM_COLLECTIONURI" || test -n "$SYSTEM_TASKDEFINITIONSURI" then CI_TYPE=azure-pipelines From patchwork Wed Apr 13 19:51:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12812554 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9B918C433F5 for ; Wed, 13 Apr 2022 19:52:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238469AbiDMTyX (ORCPT ); Wed, 13 Apr 2022 15:54:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58046 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237039AbiDMTyT (ORCPT ); Wed, 13 Apr 2022 15:54:19 -0400 Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3134A31DCC for ; Wed, 13 Apr 2022 12:51:57 -0700 (PDT) Received: by mail-wr1-x434.google.com with SMTP id t1so4140941wra.4 for ; Wed, 13 Apr 2022 12:51:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=0GLiRbG6gjj0P6dug5ZgN0vQ/XREwjkyqnctAJTBsl4=; b=ZM/dJnxmolvRNvfmAS/V97ZqM/4W0C62Aaa2B6JYZRcx/Ux0dFVgiB+lz5zq4LdGjG Gj4YSv/Jb4v5YwwXQkx9xqf9P6L+6FSTk8yS26O6nNsDJIJFpUCAiSf0KeZhX7gf1boV mOzC226PGyeeTmzH5MSpxyeXw23xkaSlPYBKDdBPokLKz4fhqiknO37MrZtiogX2Wo+a 8H8i4f2ul/F7Ld8opynaB/akRrEviKgKAwEZECLHXFlfgq2MO+dpphIkqFdQGRk1ePfH akYaOtgsBYGBzDViM3n4myrqAW5Fd4OhvRHCym4/WBz53tOZEhx2RSouGrnVyiYP8VFt 541Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0GLiRbG6gjj0P6dug5ZgN0vQ/XREwjkyqnctAJTBsl4=; b=bpgryrJ7Fx8VG70YNzCltXfZM69dLHG/5jp75hi0o2GXARHO5czOelyh/np9aMcO3/ oHo9cq5Rus/dS7udDmLn2KBaXOpwm7X84MqL5XPAmI6XFIgMqVGTGHX0btCPUdqDTvMW PHmoCrtu6yQE5Yfr+m2A6NZ1qqPUxpcwSijbJvDffPAdtf4oF+WS7/6bN4KPfrthVRuo fASPF6sNsRSVoQnBXN5pUsxGqRWQG6SSWYh9vroTQpftDBGWqECGF8pMn0gTfrDt76Yi kDj/P3zRWNlQjSH1RxQhfwFgLzZEag35QQKyqjSwxiU2ycePgDt6CUI5OePvG9aXcrC2 E4bA== X-Gm-Message-State: AOAM531ynWg15ianecSMd4U/lV1pFCwMnO2wz60lMWO0QJtDSvJFKnLE mpyzMMkHY6IPLC/INAWTPRyG+KaRPZb5Lg== X-Google-Smtp-Source: ABdhPJwALPoPuqzSPsRhX9svHNP0ZBeq4FNx/5Giwax7OnzSG1WAjdcovY/PFOojpFZRM+K41vsLeg== X-Received: by 2002:a5d:5144:0:b0:207:9a13:ca73 with SMTP id u4-20020a5d5144000000b002079a13ca73mr348545wrt.268.1649879515366; Wed, 13 Apr 2022 12:51:55 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k20-20020a05600c1c9400b0038ecd1ccc17sm3432484wms.35.2022.04.13.12.51.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Apr 2022 12:51:54 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v3 02/29] CI: make "$jobname" explicit, remove fallback Date: Wed, 13 Apr 2022 21:51:15 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.rc2.843.g193535c2aa7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Remove the $jobname fallback last adjusted in b011fabd6e0 (ci/lib.sh: encapsulate Travis-specific things, 2019-01-27). This makes all of our CI jobs have an explicit $jobname, and removes the special-case of the Windows jobs implicitly setting one (that they don't currently use). In subsequent commits we'll use this new $jobname explicitly. This is a follow-up to my 25715419bf4 (CI: don't run "make test" twice in one job, 2021-11-23), since then all the jobs except the Windows jobs have had an explicit $jobname. In the case of the Windows jobs we were using a jobname of windows-gcc for both, due to picking "gcc" as a default fallback for "$CC" in ci/lib.sh. Since we'll need this to run ci/lib.sh it makes sense to move the check to the top of the file. Subsequent commits will add more such assertions to it. Signed-off-by: Ævar Arnfjörð Bjarmason --- .github/workflows/main.yml | 8 ++++++++ ci/lib.sh | 12 +++++++----- 2 files changed, 15 insertions(+), 5 deletions(-) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index c35200defb9..309b9141249 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -79,6 +79,8 @@ jobs: windows-build: name: win build + env: + jobname: windows-build needs: ci-config if: needs.ci-config.outputs.enabled == 'yes' runs-on: windows-latest @@ -100,6 +102,8 @@ jobs: path: artifacts windows-test: name: win test + env: + jobname: windows-test runs-on: windows-latest needs: [windows-build] strategy: @@ -131,6 +135,8 @@ jobs: path: ${{env.FAILED_TEST_ARTIFACTS}} vs-build: name: win+VS build + env: + jobname: vs-build needs: ci-config if: needs.ci-config.outputs.enabled == 'yes' env: @@ -183,6 +189,8 @@ jobs: path: artifacts vs-test: name: win+VS test + env: + jobname: vs-test runs-on: windows-latest needs: vs-build strategy: diff --git a/ci/lib.sh b/ci/lib.sh index 57141d38a85..5fda166ad47 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -6,6 +6,13 @@ # and installing dependencies. set -ex +# Starting assertions +if test -z "$jobname" +then + echo "must set a CI jobname in the environment" >&2 + exit 1 +fi + skip_branch_tip_with_tag () { # Sometimes, a branch is pushed at the same time the tag that points # at the same commit as the tip of the branch is pushed, and building @@ -146,11 +153,6 @@ test -n "${DONT_SKIP_TAGS-}" || skip_branch_tip_with_tag skip_good_tree -if test -z "$jobname" -then - jobname="$CI_OS_NAME-$CC" -fi - export DEVELOPER=1 export DEFAULT_TEST_TARGET=prove export GIT_TEST_CLONE_2GB=true From patchwork Wed Apr 13 19:51:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12812555 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D482FC433FE for ; Wed, 13 Apr 2022 19:52:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238481AbiDMTyY (ORCPT ); Wed, 13 Apr 2022 15:54:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58064 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238403AbiDMTyV (ORCPT ); Wed, 13 Apr 2022 15:54:21 -0400 Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com [IPv6:2a00:1450:4864:20::330]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3B93E2C116 for ; Wed, 13 Apr 2022 12:51:58 -0700 (PDT) Received: by mail-wm1-x330.google.com with SMTP id o20-20020a05600c511400b0038ebbbb2ad8so1856608wms.0 for ; Wed, 13 Apr 2022 12:51:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=b9olFUC0AWUbDYJyxX9MLBskz31rkDJErfp4kOIteAw=; b=b0Nx2p6Qq8KD8FVWKbuLd61PAi6WP9gsiM6r/wtF/fSQtjhWjtC00vy+Y1NFWdXSea n90mz6I0ZN4evJIqJyLpZaN8iEQo3L/lhLujAm8SmngYC/2bL99FDLCAzgimMU7ztLO8 BXTPvUby0bm6BvsvLTzDExJSuqB4voTE9AIl1Z95jZ0FNZgTBGvE5sXoxGBI3YImNUPn Aab/F4KGaGms6r42sIhtfEW2Zf606RgzWxnJI/Zz+XYh0k8N6mdfYlAlH5loQStcrw5Z 0L7nD8eK8GBP8HKbTzfyUnS1fzKVfu2MjFe5+OiXvEPuZ6ANW1jzqCOQK/vhRbWAHLNz U7Hg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=b9olFUC0AWUbDYJyxX9MLBskz31rkDJErfp4kOIteAw=; b=fyprSq0Ov4NTZNN7lua4GuN8VmNDJ7jx+LCrMp5hJxAEy0/bM5886MlyE42ADh74Xs bTHBS1GzKiYXkKgEt1KcvyiYPB7cCl4/pzxFhrupdcLih1hwXCO67wAorbUI0/548flb 3VnPHKIxGtRqqku57hb0zyZEQ3afj3SR/1BgUeAthjIadcgmDQc8LBEq5nNtkPaxaI3A T+4WZ4hzjAFdS19io8oMBvEzFVUaL2PGnrtIh9vTwA/a9lUEul6baVTWsEUBXRit7rsE pIfRTkctbJ0diwINv/OuD3kj8EHF9/k3rv4tqbry5StXK76u2zFvIWxz3b/2Sb40J+8P vU2A== X-Gm-Message-State: AOAM533iOLcJR+01rjTYoFN+6VV/9lYeMiXvkSdtuHI89bQb7TZLBJqY 6dlmu6XE/wp4ivHegJkfg/cMYRv+QoADww== X-Google-Smtp-Source: ABdhPJwGN+LwjnRQsoeNXL+0eDk9i2Yr6AO977pQXgfk58IV6Duqb1qWxHpMIF9rdGlKDdaRq6ds5g== X-Received: by 2002:a05:600c:35d4:b0:38f:6799:a386 with SMTP id r20-20020a05600c35d400b0038f6799a386mr284071wmq.196.1649879516417; Wed, 13 Apr 2022 12:51:56 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k20-20020a05600c1c9400b0038ecd1ccc17sm3432484wms.35.2022.04.13.12.51.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Apr 2022 12:51:55 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v3 03/29] CI: remove more dead Travis CI support Date: Wed, 13 Apr 2022 21:51:16 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.rc2.843.g193535c2aa7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Remove code that I missed in 4a6e4b96026 (CI: remove Travis CI support, 2021-11-23). This code was only called from or used by the now-removed .travis.yml, or needed by the Travis CI environment. For the symlinking in ci/run-build-and-tests.sh: Back when 3c93b829205 (travis-ci: build Git during the 'script' phase, 2018-01-08) and 4b060a4d973 (ci: use a junction on Windows instead of a symlink, 2019-01-27) adjusted this "ln" command, the Windows build would use ci/run-build-and-tests.sh. As seen in 889cacb6897 (ci: configure GitHub Actions for CI/PR, 2020-04-11) the current windows build uses a different entry point under the GitHub CI, which doesn't use this .prove caching. Namely "ci/run-test-slice.sh". We can be certain that it's never used in "ci/run-test-slice.sh" because to have a ".prove" file we'd need to use "--state=save", which we only do in the dead Azure codepath in ci/lib.sh. If it were used it would do the wrong thing, because the different test slices would each try to clobber the same "t/.prove" file. If a subsequent run then used the -"-state=failed,slow,save" it would defeat the purpose of "ci/run-test-slice.sh", since all slices would then run all tests. I.e. behavior of prove's "--state" options is to select tests to run from the provided "--state" file, in addition to those specified on the command-line. For ci/run-docker{,-build}.sh: It was likewise last referenced in the .travis.yml removed in my 4a6e4b96026. The current "dockerized" run in ".github/workflows/main.yml" calls the same entry points as the main "regular" job. Signed-off-by: Ævar Arnfjörð Bjarmason --- ci/lib.sh | 10 ++---- ci/print-test-failures.sh | 3 -- ci/run-build-and-tests.sh | 5 --- ci/run-docker-build.sh | 66 ----------------------------------- ci/run-docker.sh | 47 ------------------------- ci/run-test-slice.sh | 5 --- ci/util/extract-trash-dirs.sh | 50 -------------------------- 7 files changed, 2 insertions(+), 184 deletions(-) delete mode 100755 ci/run-docker-build.sh delete mode 100755 ci/run-docker.sh delete mode 100755 ci/util/extract-trash-dirs.sh diff --git a/ci/lib.sh b/ci/lib.sh index 5fda166ad47..510b20726ab 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -106,9 +106,6 @@ then CI_COMMIT="$BUILD_SOURCEVERSION" CI_JOB_ID="$BUILD_BUILDID" CI_JOB_NUMBER="$BUILD_BUILDNUMBER" - CI_OS_NAME="$(echo "$AGENT_OS" | tr A-Z a-z)" - test darwin != "$CI_OS_NAME" || CI_OS_NAME=osx - CI_REPO_SLUG="$(expr "$BUILD_REPOSITORY_URI" : '.*/\([^/]*/[^/]*\)$')" CC="${CC:-gcc}" # use a subdirectory of the cache dir (because the file share is shared @@ -118,16 +115,13 @@ then export GIT_PROVE_OPTS="--timer --jobs 10 --state=failed,slow,save" export GIT_TEST_OPTS="--verbose-log -x --write-junit-xml" MAKEFLAGS="$MAKEFLAGS --jobs=10" - test windows_nt != "$CI_OS_NAME" || + test Windows_NT != "$AGENT_OS" || GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS" elif test true = "$GITHUB_ACTIONS" then CI_TYPE=github-actions CI_BRANCH="$GITHUB_REF" CI_COMMIT="$GITHUB_SHA" - CI_OS_NAME="$(echo "$RUNNER_OS" | tr A-Z a-z)" - test macos != "$CI_OS_NAME" || CI_OS_NAME=osx - CI_REPO_SLUG="$GITHUB_REPOSITORY" CI_JOB_ID="$GITHUB_RUN_ID" CC="${CC:-gcc}" DONT_SKIP_TAGS=t @@ -137,7 +131,7 @@ then export GIT_PROVE_OPTS="--timer --jobs 10" export GIT_TEST_OPTS="--verbose-log -x" MAKEFLAGS="$MAKEFLAGS --jobs=10" - test windows != "$CI_OS_NAME" || + test Windows != "$RUNNER_OS" || GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS" else echo "Could not identify CI type" >&2 diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh index 57277eefcd0..00fcf27b1b2 100755 --- a/ci/print-test-failures.sh +++ b/ci/print-test-failures.sh @@ -77,9 +77,6 @@ do fi combined_trash_size=$new_combined_trash_size - # DO NOT modify these two 'echo'-ed strings below - # without updating 'ci/util/extract-trash-dirs.sh' - # as well. echo "$(tput setaf 1)Start of trash directory of '$test_name':$(tput sgr0)" cat "$trash_tgz_b64" echo "$(tput setaf 1)End of trash directory of '$test_name'$(tput sgr0)" diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh index 280dda7d285..0a1ec8c2bae 100755 --- a/ci/run-build-and-tests.sh +++ b/ci/run-build-and-tests.sh @@ -5,11 +5,6 @@ . ${0%/*}/lib.sh -case "$CI_OS_NAME" in -windows*) cmd //c mklink //j t\\.prove "$(cygpath -aw "$cache_dir/.prove")";; -*) ln -s "$cache_dir/.prove" t/.prove;; -esac - export MAKE_TARGETS="all test" case "$jobname" in diff --git a/ci/run-docker-build.sh b/ci/run-docker-build.sh deleted file mode 100755 index 6cd832efb9c..00000000000 --- a/ci/run-docker-build.sh +++ /dev/null @@ -1,66 +0,0 @@ -#!/bin/sh -# -# Build and test Git inside container -# -# Usage: -# run-docker-build.sh -# - -set -ex - -if test $# -ne 1 || test -z "$1" -then - echo >&2 "usage: run-docker-build.sh " - exit 1 -fi - -case "$jobname" in -linux32) - switch_cmd="linux32 --32bit i386" - ;; -linux-musl) - switch_cmd= - useradd () { adduser -D "$@"; } - ;; -*) - exit 1 - ;; -esac - -"${0%/*}/install-docker-dependencies.sh" - -# If this script runs inside a docker container, then all commands are -# usually executed as root. Consequently, the host user might not be -# able to access the test output files. -# If a non 0 host user id is given, then create a user "ci" with that -# user id to make everything accessible to the host user. -HOST_UID=$1 -if test $HOST_UID -eq 0 -then - # Just in case someone does want to run the test suite as root. - CI_USER=root -else - CI_USER=ci - if test "$(id -u $CI_USER 2>/dev/null)" = $HOST_UID - then - echo "user '$CI_USER' already exists with the requested ID $HOST_UID" - else - useradd -u $HOST_UID $CI_USER - fi -fi - -# Build and test -command $switch_cmd su -m -l $CI_USER -c " - set -ex - export DEVELOPER='$DEVELOPER' - export DEFAULT_TEST_TARGET='$DEFAULT_TEST_TARGET' - export GIT_PROVE_OPTS='$GIT_PROVE_OPTS' - export GIT_TEST_OPTS='$GIT_TEST_OPTS' - export GIT_TEST_CLONE_2GB='$GIT_TEST_CLONE_2GB' - export MAKEFLAGS='$MAKEFLAGS' - export cache_dir='$cache_dir' - cd /usr/src/git - test -n '$cache_dir' && ln -s '$cache_dir/.prove' t/.prove - make - make test -" diff --git a/ci/run-docker.sh b/ci/run-docker.sh deleted file mode 100755 index af89d1624a4..00000000000 --- a/ci/run-docker.sh +++ /dev/null @@ -1,47 +0,0 @@ -#!/bin/sh -# -# Download and run Docker image to build and test Git -# - -. ${0%/*}/lib.sh - -case "$jobname" in -linux32) - CI_CONTAINER="daald/ubuntu32:xenial" - ;; -linux-musl) - CI_CONTAINER=alpine - ;; -*) - exit 1 - ;; -esac - -docker pull "$CI_CONTAINER" - -# Use the following command to debug the docker build locally: -# must be 0 if podman is used as drop-in replacement for docker -# $ docker run -itv "${PWD}:/usr/src/git" --entrypoint /bin/sh "$CI_CONTAINER" -# root@container:/# export jobname= -# root@container:/# /usr/src/git/ci/run-docker-build.sh - -container_cache_dir=/tmp/container-cache - -docker run \ - --interactive \ - --env DEVELOPER \ - --env DEFAULT_TEST_TARGET \ - --env GIT_PROVE_OPTS \ - --env GIT_TEST_OPTS \ - --env GIT_TEST_CLONE_2GB \ - --env MAKEFLAGS \ - --env jobname \ - --env cache_dir="$container_cache_dir" \ - --volume "${PWD}:/usr/src/git" \ - --volume "$cache_dir:$container_cache_dir" \ - "$CI_CONTAINER" \ - /usr/src/git/ci/run-docker-build.sh $(id -u $USER) - -check_unignored_build_artifacts - -save_good_tree diff --git a/ci/run-test-slice.sh b/ci/run-test-slice.sh index f8c2c3106a2..b9a682b4bcd 100755 --- a/ci/run-test-slice.sh +++ b/ci/run-test-slice.sh @@ -5,11 +5,6 @@ . ${0%/*}/lib.sh -case "$CI_OS_NAME" in -windows*) cmd //c mklink //j t\\.prove "$(cygpath -aw "$cache_dir/.prove")";; -*) ln -s "$cache_dir/.prove" t/.prove;; -esac - make --quiet -C t T="$(cd t && ./helper/test-tool path-utils slice-tests "$1" "$2" t[0-9]*.sh | tr '\n' ' ')" diff --git a/ci/util/extract-trash-dirs.sh b/ci/util/extract-trash-dirs.sh deleted file mode 100755 index 8e67bec21a2..00000000000 --- a/ci/util/extract-trash-dirs.sh +++ /dev/null @@ -1,50 +0,0 @@ -#!/bin/sh - -error () { - echo >&2 "error: $@" - exit 1 -} - -find_embedded_trash () { - while read -r line - do - case "$line" in - *Start\ of\ trash\ directory\ of\ \'t[0-9][0-9][0-9][0-9]-*\':*) - test_name="${line#*\'}" - test_name="${test_name%\'*}" - - return 0 - esac - done - - return 1 -} - -extract_embedded_trash () { - while read -r line - do - case "$line" in - *End\ of\ trash\ directory\ of\ \'$test_name\'*) - return - ;; - *) - printf '%s\n' "$line" - ;; - esac - done - - error "unexpected end of input" -} - -# Raw logs from Linux build jobs have CRLF line endings, while OSX -# build jobs mostly have CRCRLF, except an odd line every now and -# then that has CRCRCRLF. 'base64 -d' from 'coreutils' doesn't like -# CRs and complains about "invalid input", so remove all CRs at the -# end of lines. -sed -e 's/\r*$//' | \ -while find_embedded_trash -do - echo "Extracting trash directory of '$test_name'" - - extract_embedded_trash |base64 -d |tar xzp -done From patchwork Wed Apr 13 19:51:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12812556 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 970F2C433EF for ; Wed, 13 Apr 2022 19:52:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238488AbiDMTy1 (ORCPT ); Wed, 13 Apr 2022 15:54:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58080 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238409AbiDMTyW (ORCPT ); Wed, 13 Apr 2022 15:54:22 -0400 Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com [IPv6:2a00:1450:4864:20::32d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3EFF72FFED for ; Wed, 13 Apr 2022 12:51:59 -0700 (PDT) Received: by mail-wm1-x32d.google.com with SMTP id q8so1527106wmc.0 for ; Wed, 13 Apr 2022 12:51:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=/s5v66zOZ+5QGI07R+eOf0ZGz76UFQUzVwEZcUsMTFw=; b=U8I5V/AmKNPsHE3025HBxxY3maPAfRk61fnRSGMP+KRIqeMCasDzrAwnrHUU6tekLF Flu9DbdvFPEtQHltZn0Omb+VAz/FT7TnJPe3QliYN2AZvryWaTD7nWdxGNG7cRYZvYmQ nX2iGq2aK6hngh306Nb9lmt6BjTtu67s9R3hnq5FRymUEyS32c4Uw9Q56mqi/7HxNM5J 0uwGtd71dZLhrP8qh4IgPM8zCYl0vaQezOckFUYyn/42ivD4gELb61Rg3/FbITRjRQPf tbnJ7sMU1ganc62HSpW9O+Zl3ZxecNEwUUjizOPcTrFI8qUyOAGZUYanafIlVRXU1NrS KRlw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=/s5v66zOZ+5QGI07R+eOf0ZGz76UFQUzVwEZcUsMTFw=; b=5sP1ha7N/9Pj4QDuf5p1pQULwSdFrYIhHRZfbTQj2aY/wH4iHk/bqAWLeOeNpGmFwN YQwTvF7MfDiUTAgGPiq9BpbEPYcmqb8mLSaVtvm+cGnjKNanx0KV27a0P3KwLfAAMWiw RQ7jMxI+KeWygYDW0z7kJlwjKriG93raCIlYZWw2oz0LXnsuEBvL5Zv/TPNM5rxGPhpT mC53eGj3HuwD7iovY0uzBASr1VLmvuSD4dD5I2m50z1tMj9Aym6bSjJuKcFxnVrcOyTz It1HClXMcllpp64vJV78OK9oKtcM8guhiMqvQztd+yHjk5mHQfy0A6eR/50foJxUb0P+ 5xwQ== X-Gm-Message-State: AOAM533Pc3Q4gHzxVAU4CAVAWQSsxXX4+aSVfjUyyRlJXNJggZCjDzqL H126PVOdpkb2OYGGcGRTy4ztGdHD7XUmSQ== X-Google-Smtp-Source: ABdhPJy/pFNNfjFFU8XW7+52InOS25uQgoRC2v+ipOM4mYmfBjQakstKpDiq5Un3ipNxAH5tM64hlw== X-Received: by 2002:a1c:f413:0:b0:37b:d1de:5762 with SMTP id z19-20020a1cf413000000b0037bd1de5762mr293258wma.108.1649879517492; Wed, 13 Apr 2022 12:51:57 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k20-20020a05600c1c9400b0038ecd1ccc17sm3432484wms.35.2022.04.13.12.51.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Apr 2022 12:51:56 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v3 04/29] CI: remove dead "tree skipping" code Date: Wed, 13 Apr 2022 21:51:17 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.rc2.843.g193535c2aa7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Remove the code related to skipping CI runs if the tree was successfully tested before. With the GitHub CI this is done with the "ci-config" step, see e76eec35540 (ci: allow per-branch config for GitHub Actions, 2020-05-07) and 7d78d5fc1a9 (ci: skip GitHub workflow runs for already-tested commits/trees, 2020-10-08). This code hasn't been used since 4a6e4b96026 (CI: remove Travis CI support, 2021-11-23), and before that for the retired Azure pipeline support removed in 6081d3898fe (ci: retire the Azure Pipelines definition, 2020-04-11). This change is needed because this and subsequent commits are turning "ci/lib.sh" into a dumber library that'll only be tasked with setting variables for CI jobs, or for specific steps within those jobs. Now we don't need to worry about it potentially skipping the run on its own. This change also removes a subtle potential logic error introduced in 0e7696c64db (ci: disallow directional formatting, 2021-11-04). The "ci/check-directional-formatting.bash" script would have been made to run after the "save_good_tree" in invoked in "ci/run-static-analysis.sh". So if e.g. the "make coccicheck" failed we'd still mark the tree as good. I.e. the addition of "ci/check-directional-formatting.bash" didn't take into account that the various users of the "save_good_tree" function in ci/*.sh made a hard assumption that they're the only ci/*.sh script that's being run (and they use "set -e"). This is entirely academic since we weren't actually running this code, but is something to be careful of if anyone ever needs to resurrect parts of this. Finally, the cache_dir="$HOME/none" added in a3f2eec862b (ci/lib: allow running in GitHub Actions, 2020-04-08) for the GitHub CI can be removed. It appears to have been added to appease the subsequent 'mkdir -p "$cache_dir"', which wasn't needed in GitHub CI. 1. https://lore.kernel.org/git/patch-1.1-eec0a8c3164-20211217T000418Z-avarab@gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason --- ci/lib.sh | 91 --------------------------------------- ci/run-build-and-tests.sh | 2 - ci/run-static-analysis.sh | 2 - ci/test-documentation.sh | 2 - 4 files changed, 97 deletions(-) diff --git a/ci/lib.sh b/ci/lib.sh index 510b20726ab..d12eaf338f9 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -13,75 +13,6 @@ then exit 1 fi -skip_branch_tip_with_tag () { - # Sometimes, a branch is pushed at the same time the tag that points - # at the same commit as the tip of the branch is pushed, and building - # both at the same time is a waste. - # - # When the build is triggered by a push to a tag, $CI_BRANCH will - # have that tagname, e.g. v2.14.0. Let's see if $CI_BRANCH is - # exactly at a tag, and if so, if it is different from $CI_BRANCH. - # That way, we can tell if we are building the tip of a branch that - # is tagged and we can skip the build because we won't be skipping a - # build of a tag. - - if TAG=$(git describe --exact-match "$CI_BRANCH" 2>/dev/null) && - test "$TAG" != "$CI_BRANCH" - then - echo "$(tput setaf 2)Tip of $CI_BRANCH is exactly at $TAG$(tput sgr0)" - exit 0 - fi -} - -# Save some info about the current commit's tree, so we can skip the build -# job if we encounter the same tree again and can provide a useful info -# message. -save_good_tree () { - echo "$(git rev-parse $CI_COMMIT^{tree}) $CI_COMMIT $CI_JOB_NUMBER $CI_JOB_ID" >>"$good_trees_file" - # limit the file size - tail -1000 "$good_trees_file" >"$good_trees_file".tmp - mv "$good_trees_file".tmp "$good_trees_file" -} - -# Skip the build job if the same tree has already been built and tested -# successfully before (e.g. because the branch got rebased, changing only -# the commit messages). -skip_good_tree () { - if test true = "$GITHUB_ACTIONS" - then - return - fi - - if ! good_tree_info="$(grep "^$(git rev-parse $CI_COMMIT^{tree}) " "$good_trees_file")" - then - # Haven't seen this tree yet, or no cached good trees file yet. - # Continue the build job. - return - fi - - echo "$good_tree_info" | { - read tree prev_good_commit prev_good_job_number prev_good_job_id - - if test "$CI_JOB_ID" = "$prev_good_job_id" - then - cat <<-EOF - $(tput setaf 2)Skipping build job for commit $CI_COMMIT.$(tput sgr0) - This commit has already been built and tested successfully by this build job. - To force a re-build delete the branch's cache and then hit 'Restart job'. - EOF - else - cat <<-EOF - $(tput setaf 2)Skipping build job for commit $CI_COMMIT.$(tput sgr0) - This commit's tree has already been built and tested successfully in build job $prev_good_job_number for commit $prev_good_commit. - The log of that build job is available at $SYSTEM_TASKDEFINITIONSURI$SYSTEM_TEAMPROJECT/_build/results?buildId=$prev_good_job_id - To force a re-build delete the branch's cache and then hit 'Restart job'. - EOF - fi - } - - exit 0 -} - check_unignored_build_artifacts () { ! git ls-files --other --exclude-standard --error-unmatch \ @@ -102,16 +33,8 @@ if test -n "$SYSTEM_COLLECTIONURI" || test -n "$SYSTEM_TASKDEFINITIONSURI" then CI_TYPE=azure-pipelines # We are running in Azure Pipelines - CI_BRANCH="$BUILD_SOURCEBRANCH" - CI_COMMIT="$BUILD_SOURCEVERSION" - CI_JOB_ID="$BUILD_BUILDID" - CI_JOB_NUMBER="$BUILD_BUILDNUMBER" CC="${CC:-gcc}" - # use a subdirectory of the cache dir (because the file share is shared - # among *all* phases) - cache_dir="$HOME/test-cache/$SYSTEM_PHASENAME" - export GIT_PROVE_OPTS="--timer --jobs 10 --state=failed,slow,save" export GIT_TEST_OPTS="--verbose-log -x --write-junit-xml" MAKEFLAGS="$MAKEFLAGS --jobs=10" @@ -120,13 +43,7 @@ then elif test true = "$GITHUB_ACTIONS" then CI_TYPE=github-actions - CI_BRANCH="$GITHUB_REF" - CI_COMMIT="$GITHUB_SHA" - CI_JOB_ID="$GITHUB_RUN_ID" CC="${CC:-gcc}" - DONT_SKIP_TAGS=t - - cache_dir="$HOME/none" export GIT_PROVE_OPTS="--timer --jobs 10" export GIT_TEST_OPTS="--verbose-log -x" @@ -139,14 +56,6 @@ else exit 1 fi -good_trees_file="$cache_dir/good-trees" - -mkdir -p "$cache_dir" - -test -n "${DONT_SKIP_TAGS-}" || -skip_branch_tip_with_tag -skip_good_tree - export DEVELOPER=1 export DEFAULT_TEST_TARGET=prove export GIT_TEST_CLONE_2GB=true diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh index 0a1ec8c2bae..e5a21985b44 100755 --- a/ci/run-build-and-tests.sh +++ b/ci/run-build-and-tests.sh @@ -45,5 +45,3 @@ esac # start running tests. make $MAKE_TARGETS check_unignored_build_artifacts - -save_good_tree diff --git a/ci/run-static-analysis.sh b/ci/run-static-analysis.sh index 65bcebda41a..5a87b1cac96 100755 --- a/ci/run-static-analysis.sh +++ b/ci/run-static-analysis.sh @@ -28,5 +28,3 @@ fi make hdr-check || exit 1 - -save_good_tree diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh index de41888430a..9e0652c30dd 100755 --- a/ci/test-documentation.sh +++ b/ci/test-documentation.sh @@ -41,5 +41,3 @@ grep ' X-Patchwork-Id: 12812557 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 606ACC433EF for ; Wed, 13 Apr 2022 19:52:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238491AbiDMTyb (ORCPT ); Wed, 13 Apr 2022 15:54:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58092 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232808AbiDMTyW (ORCPT ); Wed, 13 Apr 2022 15:54:22 -0400 Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com [IPv6:2a00:1450:4864:20::430]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 66BB631DCC for ; Wed, 13 Apr 2022 12:52:00 -0700 (PDT) Received: by mail-wr1-x430.google.com with SMTP id q3so3498985wrj.7 for ; Wed, 13 Apr 2022 12:52:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=KDNZ571zlXAYf5vSBIaRatC5rxkgtLsjLynW+wqdIjU=; b=YOiPIHNKmFCgiT49u3Fn6vVW8sc1TzBYqO9r4v7Olf3scJe1CfhDd82ZSyJ/e5x3SF yMrkADINa/QEvfeSlPrRW0kPL3fyl3LLnBO9A8+NsWhmLoGZN6A5VYC3eNHc1irn+4jU y+yOf/42EqQCseq0LVZnQRemriTDEiSntEPoW8p411uJzA5wxwz2Ztfge/y/fzdOWc4U K3uxyUo+NJ6jp2/6Vz6zOyS3ymSDEZmc3QDeRU75JjR+Ldx6qDIune+FLVnoWOwNz5pz lJwVLkcfOtZUBAKseCOgAtwyZ/MZnMPoshNK+rT7gwiUpQ/t4TgHVuKefIcig/6q3Sl9 lFKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KDNZ571zlXAYf5vSBIaRatC5rxkgtLsjLynW+wqdIjU=; b=rsWJlKEZZitNlUCW9+tguou5/gyGi5LLt74cAvbskY1Ov9BeJ4P6IRA/0yJdNdt/CF zG7sRdmLvPHpS0m4rNo1mW02pVGzz1iQfkANhBwTy9oI/kZ/Q5s1MC7sdNmz57357jYZ lUewneklVXJoaMPkL5UmA1ZuMC9Q4w1dOq8seLBJvxlnfHZ4b8vUDpOUS1t0C+VJdqsJ omOoN2tx3A3k614gxieFdRzBCmFGFJR38Zd+opBGCO5tsbUoxuj7HndYghmQXXiigQcF a3KNA2K7LhYhczmK0AiW2dDKTdCoKGFtzh9mIe7o83pf2RZ9owHJ10l8hPeZtdsEffNZ XeIw== X-Gm-Message-State: AOAM532/ub/4QDU/t57RE2XiYr/mqnieyhdEb4gWBbJiiuWcX16k8E6T +pTOcx4VZYcj5TYlcB76PpqLsZJewan9rQ== X-Google-Smtp-Source: ABdhPJxFz3vfD0qS90E4AMFnoyoFRLP8hyAAd3oMYu/GelMVjOzGX391Zy9vLKbHrudG1KWWncbCzg== X-Received: by 2002:a5d:4cce:0:b0:207:a1f1:6f50 with SMTP id c14-20020a5d4cce000000b00207a1f16f50mr321568wrt.139.1649879518755; Wed, 13 Apr 2022 12:51:58 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k20-20020a05600c1c9400b0038ecd1ccc17sm3432484wms.35.2022.04.13.12.51.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Apr 2022 12:51:57 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v3 05/29] CI: remove unused Azure ci/* code Date: Wed, 13 Apr 2022 21:51:18 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.rc2.843.g193535c2aa7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Remove Azure-specific code that's been unused since 6081d3898fe (ci: retire the Azure Pipelines definition, 2020-04-11). As noted in a larger removal of all of the Azure-supporting code in [1] (although that missed some of this) there is more of it in-tree, but let's focus on only the ci/* code for now. This is needed because in subsequent commits this unused code would either need to be changed to accommodate changes in ci/*, or be removed. As we'll see in those subsequent commits the end-state we're heading towards will actually make it easier to add new CI types in the future, even though the only one we're left with now is the GitHub CI. I.e. the "ci/*" framework will be made to do less, not more. We'll be offloading more of what it does to our generic build and test system. While I'm at it (since the line needs to be touched anyway) change an odd 'if test true == "$GITHUB_ACTIONS"' to the more usual style used in other places of 'if test "$GITHUB_ACTIONS" = "true"'. 1. https://lore.kernel.org/git/patch-1.1-eec0a8c3164-20211217T000418Z-avarab@gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason --- ci/install-dependencies.sh | 3 --- ci/lib.sh | 13 +------------ ci/mount-fileshare.sh | 25 ------------------------- ci/print-test-failures.sh | 5 ----- 4 files changed, 1 insertion(+), 45 deletions(-) delete mode 100755 ci/mount-fileshare.sh diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh index dbcebad2fb2..e7ea8799411 100755 --- a/ci/install-dependencies.sh +++ b/ci/install-dependencies.sh @@ -34,8 +34,6 @@ macos-latest) export HOMEBREW_NO_AUTO_UPDATE=1 HOMEBREW_NO_INSTALL_CLEANUP=1 # Uncomment this if you want to run perf tests: # brew install gnu-time - test -z "$BREW_INSTALL_PACKAGES" || - brew install $BREW_INSTALL_PACKAGES brew link --force gettext brew install --cask --no-quarantine perforce || { # Update the definitions and try again @@ -69,7 +67,6 @@ Documentation) sudo apt-get -q update sudo apt-get -q -y install asciidoc xmlto docbook-xsl-ns make - test -n "$ALREADY_HAVE_ASCIIDOCTOR" || sudo gem install --version 1.5.8 asciidoctor ;; linux-gcc-default) diff --git a/ci/lib.sh b/ci/lib.sh index d12eaf338f9..c4b829c1eed 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -29,18 +29,7 @@ export TERM=${TERM:-dumb} # Clear MAKEFLAGS that may come from the outside world. export MAKEFLAGS= -if test -n "$SYSTEM_COLLECTIONURI" || test -n "$SYSTEM_TASKDEFINITIONSURI" -then - CI_TYPE=azure-pipelines - # We are running in Azure Pipelines - CC="${CC:-gcc}" - - export GIT_PROVE_OPTS="--timer --jobs 10 --state=failed,slow,save" - export GIT_TEST_OPTS="--verbose-log -x --write-junit-xml" - MAKEFLAGS="$MAKEFLAGS --jobs=10" - test Windows_NT != "$AGENT_OS" || - GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS" -elif test true = "$GITHUB_ACTIONS" +if test "$GITHUB_ACTIONS" = "true" then CI_TYPE=github-actions CC="${CC:-gcc}" diff --git a/ci/mount-fileshare.sh b/ci/mount-fileshare.sh deleted file mode 100755 index 26b58a80960..00000000000 --- a/ci/mount-fileshare.sh +++ /dev/null @@ -1,25 +0,0 @@ -#!/bin/sh - -die () { - echo "$*" >&2 - exit 1 -} - -test $# = 4 || -die "Usage: $0 " - -mkdir -p "$4" || die "Could not create $4" - -case "$(uname -s)" in -Linux) - sudo mount -t cifs -o vers=3.0,username="$2",password="$3",dir_mode=0777,file_mode=0777,serverino "$1" "$4" - ;; -Darwin) - pass="$(echo "$3" | sed -e 's/\//%2F/g' -e 's/+/%2B/g')" && - mount -t smbfs,soft "smb://$2:$pass@${1#//}" "$4" - ;; -*) - die "No support for $(uname -s)" - ;; -esac || -die "Could not mount $4" diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh index 00fcf27b1b2..a878cca67e4 100755 --- a/ci/print-test-failures.sh +++ b/ci/print-test-failures.sh @@ -39,11 +39,6 @@ do test_name="${test_name##*/}" trash_dir="trash directory.$test_name" case "$CI_TYPE" in - azure-pipelines) - mkdir -p failed-test-artifacts - mv "$trash_dir" failed-test-artifacts - continue - ;; github-actions) mkdir -p failed-test-artifacts echo "FAILED_TEST_ARTIFACTS=t/failed-test-artifacts" >>$GITHUB_ENV From patchwork Wed Apr 13 19:51:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12812558 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 46405C433FE for ; Wed, 13 Apr 2022 19:52:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232808AbiDMTyc (ORCPT ); Wed, 13 Apr 2022 15:54:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58110 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238470AbiDMTyX (ORCPT ); Wed, 13 Apr 2022 15:54:23 -0400 Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com [IPv6:2a00:1450:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4CB892FFED for ; Wed, 13 Apr 2022 12:52:01 -0700 (PDT) Received: by mail-wr1-x42d.google.com with SMTP id i20so4096212wrb.13 for ; Wed, 13 Apr 2022 12:52:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=/yGc/HrvdckN1BUvUV+axxJi8hTlsjJOnGca9LoC2ck=; b=Ap6btNONvszB0zRzMUidDHMZ0qeLO9urqFUpG6knMt3aRsKrr7gBUUQoQF/zK4+mbz XCdCjRRh5chOLsf49Oy13aYJnfqOYXsTXzutozQ2Vp5KgYLqxKJIEaMCywQGq3e59Ga9 DiClBvwc7o/1yGzrehn8KMsHKKzVI4milbpmGZ+5CePMgPvbJatiEmaBGWwYvoXGyPuY Aiw20W9tUfOJmsdJ6YqRZ7z01UnpVTyUopwN3QthdpC7UyzcpcTn+/6D3xWZbsxPTcAP shtAoa8IrCT+gtIItf6Meln3nWB4hmd7jlE7Or7DGq8wRwaHBmeQQodfEnnSdHylBkwZ YTFw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=/yGc/HrvdckN1BUvUV+axxJi8hTlsjJOnGca9LoC2ck=; b=n0az3aeciWE2pwskrUBsrM9rjjjQ+xa6B3LcLiRxWgXm6T47NtQNkTmlI0H5e9BS1c JBrrtgnmPJ5qF/sTfClfGqCUv4TjKOso4KfySrGzPeVFqB6qTdMrrZllsTSEFBkT+jav ikj1UbKeHPfJXANmUQzjjlAMdKZ8nOa+ptHWSpL2jOcXsquXtiJ3WEVKqh6iIOYE24TF AaL3jJ3aRxCQgGIET9EL6Vk+tgMsK+CANIDib7O5HJLjplnzxDmK3iKK3dIydBgwdhwU /8SN/MLIUMVuInmFsbunbWl2QeU3+uYB6zIssNB9zXo6v349OiF8LFP/SDW42lrbV7/v zfTQ== X-Gm-Message-State: AOAM5302hvm4vn8xbLgqQuBl0sbIcAwwzGLFD+ZOJMkSWhHXtKNgPiCP L2MvX52N0QZ5SgLqV8mp9gljyvYsDTAdhw== X-Google-Smtp-Source: ABdhPJz7amvG2x2NIwVQM8mVRCXuhy56bf3VDQE75K3cSCxNRBuq9a5fmmzYyHBCI8jnYNaUD1WhSg== X-Received: by 2002:adf:9bd5:0:b0:207:a2a6:636a with SMTP id e21-20020adf9bd5000000b00207a2a6636amr343384wrc.480.1649879519627; Wed, 13 Apr 2022 12:51:59 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k20-20020a05600c1c9400b0038ecd1ccc17sm3432484wms.35.2022.04.13.12.51.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Apr 2022 12:51:59 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v3 06/29] CI/lib.sh: stop adding leading whitespace to $MAKEFLAGS Date: Wed, 13 Apr 2022 21:51:19 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.rc2.843.g193535c2aa7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In a8c51f77d16 (ci: clear and mark MAKEFLAGS exported just once, 2019-02-07) the assignment of MAKEFLAGS was done up-front to future-proof this code for the bug fixed in 406f93ae488 (ci: make sure we build Git parallel, 2019-02-07). But since 4a6e4b96026 (CI: remove Travis CI support, 2021-11-23) the --jobs=10 has been an invariant here, so let's assign --jobs=$NPROC to the MAKEFLAGS right away, this gets rid of the leading whitespace from the $MAKEFLAGS variable, as it would previously be empty on the first assignment. Signed-off-by: Ævar Arnfjörð Bjarmason --- ci/lib.sh | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/ci/lib.sh b/ci/lib.sh index c4b829c1eed..21c009757f4 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -26,17 +26,19 @@ check_unignored_build_artifacts () # GitHub Action doesn't set TERM, which is required by tput export TERM=${TERM:-dumb} +# How many jobs to run in parallel? +NPROC=10 + # Clear MAKEFLAGS that may come from the outside world. -export MAKEFLAGS= +export MAKEFLAGS=--jobs=$NPROC if test "$GITHUB_ACTIONS" = "true" then CI_TYPE=github-actions CC="${CC:-gcc}" - export GIT_PROVE_OPTS="--timer --jobs 10" + export GIT_PROVE_OPTS="--timer --jobs $NPROC" export GIT_TEST_OPTS="--verbose-log -x" - MAKEFLAGS="$MAKEFLAGS --jobs=10" test Windows != "$RUNNER_OS" || GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS" else From patchwork Wed Apr 13 19:51:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12812559 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7FD71C433EF for ; Wed, 13 Apr 2022 19:52:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238505AbiDMTye (ORCPT ); Wed, 13 Apr 2022 15:54:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58154 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238480AbiDMTyY (ORCPT ); Wed, 13 Apr 2022 15:54:24 -0400 Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 64EFD424A7 for ; Wed, 13 Apr 2022 12:52:02 -0700 (PDT) Received: by mail-wr1-x431.google.com with SMTP id k22so4153199wrd.2 for ; Wed, 13 Apr 2022 12:52:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=rrxN0Vogdsk0Cdc8ZI8CjlHpPUqJD2QXpwX7utfRL+o=; b=TGF1P8sxbPMOqmpzX0UMNE1yuWP/ZzY1mtXYOloJ+HPyz75zSIQDB7pxHgnCjnqsyk tb6nsmOJ/mgKR0wgckhkd6YSJfd7pI8GfYdnWgYSVps57SwSGl5AOwxLLLr52dxEfg8m s1faOnZJCTGGGd/DpL17AlOFdnh9nnDbmlgZyeKMKGerNVf2/L3CKDf1kRcBzt4xLX5O SYn5Oh/jesP4zJdo0bP5NkIL5wNy/ZfSWWeOlE8bxHnS2R/TY9Q9TvQo0JpEOGLrpPJP TV4wsgli0ZseT9uv+yTWxc4BoPpVLMaBXtKGWKrSfeAMV9RIiwP8B3nfflRq/yo+hdl7 82tA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=rrxN0Vogdsk0Cdc8ZI8CjlHpPUqJD2QXpwX7utfRL+o=; b=XXbpAT49xryV/dHd1CInnDWtl8hPdBIVK0Y7g3J1ri/1Jdlzdj/tiC+s5+AfU7dBe8 srKZW5gqjm6gNftV/bxNKUYswpCPH5enhDS0UWkvfgukGzjFbeCyS4W8l8W4pUBlT5dH VfezsXZXcYA4ij0YXveOhq/XTQyqYHJRHQZCC598SPlLXVMaOwpjoR6KJJI4jWg7OTuR aOkfDCBFij2Y1wrr3Bq7F6Uh3UCThqROSUILlkE2uwFRpGoPfu90KitkiqP1eP+p8oqf lBda7lnQucRo2Kuix0lJq8f0bu+D51wQfVnQLjoUtbiWepvpbnEOstcsp4cAQDY3hB+2 Se9g== X-Gm-Message-State: AOAM532oHv564TjPYb5Nx2RTJrxXvGfsLxEgmSmXN7rI/t+yI/gb13gC 2N3A3qOptJ5gwLOuZuNrxHPwVuKH+mu9Zw== X-Google-Smtp-Source: ABdhPJzA/xFoGnsl1c/34qGvReWz9o3yEict1bW2kwyNw+qAlkUHlZR7H+DWoIrzY7f4MxBbKxNMxw== X-Received: by 2002:adf:c14d:0:b0:207:a28f:f5dd with SMTP id w13-20020adfc14d000000b00207a28ff5ddmr307002wre.679.1649879520538; Wed, 13 Apr 2022 12:52:00 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k20-20020a05600c1c9400b0038ecd1ccc17sm3432484wms.35.2022.04.13.12.51.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Apr 2022 12:51:59 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v3 07/29] CI: don't have "git grep" invoke a pager in tree content check Date: Wed, 13 Apr 2022 21:51:20 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.rc2.843.g193535c2aa7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Fix the omission of not adding "-P" (a.k.a. "--no-pager") to the "git" invocation that spaws "git grep", added in 0e7696c64db (ci: disallow directional formatting, 2021-11-04). This didn't matter for the GitHub CI, but does when running the check locally. A subsequent commit will start invoking it via a new check from the top-level Makefile, and without this we'd visibly spawn a pager when doing so. Signed-off-by: Ævar Arnfjörð Bjarmason --- ci/check-directional-formatting.bash | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ci/check-directional-formatting.bash b/ci/check-directional-formatting.bash index e6211b141aa..de8e6e486f0 100755 --- a/ci/check-directional-formatting.bash +++ b/ci/check-directional-formatting.bash @@ -23,5 +23,5 @@ # U+2066..U+2069: LRI, RLI, FSI and PDI regex='(\u202a|\u202b|\u202c|\u202d|\u202e|\u2066|\u2067|\u2068|\u2069)' -! LC_CTYPE=C git grep -El "$(LC_CTYPE=C.UTF-8 printf "$regex")" \ +! LC_CTYPE=C git -P grep -El "$(LC_CTYPE=C.UTF-8 printf "$regex")" \ -- ':(exclude,attr:binary)' ':(exclude)*.po' From patchwork Wed Apr 13 19:51:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12812560 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DEED5C433F5 for ; Wed, 13 Apr 2022 19:52:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238510AbiDMTyg (ORCPT ); Wed, 13 Apr 2022 15:54:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58208 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238485AbiDMTyZ (ORCPT ); Wed, 13 Apr 2022 15:54:25 -0400 Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6298847380 for ; Wed, 13 Apr 2022 12:52:03 -0700 (PDT) Received: by mail-wr1-x432.google.com with SMTP id w4so4102515wrg.12 for ; Wed, 13 Apr 2022 12:52:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=b0QuDd9djWVzYs6J48mRmtc8VbI/bdwGiSwuPClbQB0=; b=NTwzCdzh9uaSXmUt3BUtvka9z90dr3lfq0dzrXzx73f05LL4MFNsURO3Z4f+CJMK0J sACRP3Wew0795nblgsOG0nnLkDmb52KGFUH/y0bjQqxo8oRhnceDb0sDzdq7r+ENQ539 1a+tVMf8G3vRI3jUOJAB28rhezMbTkQ1ytfygDfNsJ+y06lT76WFOtvWubgX1WyxrbZM vxDhMZo1j1GeyHxDb3JjiXJ0du4GF9iklWzdMc9hZvBDstMkFVdlKI/8JNv/1ps54+19 hofB/g80yerXQSQJg2twhNcTNQ6g/wgUwD8FiqbPYqoXv5psKvtQvWJVBtYPW5lSqWNK Va/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=b0QuDd9djWVzYs6J48mRmtc8VbI/bdwGiSwuPClbQB0=; b=zWkQ0m3lXmgEKwHRxZFvBJqZNJgNplZ9E2DlnShnVz9w6PjwYp8ZJkfJ5bwf2D3zm3 jihZwYzNxENt1gEpVntAXpJ0F/eAjEsymxtBowuSC8H15JbaAqrOsxaPYgKnT5xDEGFt rBMYLXw1ejmDZOpT03JHbDBcK+4bEg3RTfTOS6Ope91LhP/m2Bf6bV/EZBfJ+Xm+3uFG PzyWEW8RgSJWFSNl1tZD8DVZWnSGPfFEJBowf0uBhqrnNRUiCJ1R0jHLe6BuaHLxzhO7 CLL930zU9a3FNUgRQ/U+NpztNn+y2r7tL+eu8jEqSItj4Nj0vj9AFUNVTMIggt5hIait s6lA== X-Gm-Message-State: AOAM532EG4jC9NiI+k3lQofpRL7HLLbjEMT0J+VokiSLPlyBBVTCcv7j pfYkwMrgAenHN9mHTpSmJJ0q0omfXyxxpg== X-Google-Smtp-Source: ABdhPJyzCL2N0sNjrW39z2zv6QnY1Y+RRRlwPsXSiJNeaMUNXuvv5q+a8tu0umEL4rI5A32pxZS8dw== X-Received: by 2002:a05:6000:18d:b0:208:4b82:6b90 with SMTP id p13-20020a056000018d00b002084b826b90mr320217wrx.267.1649879521657; Wed, 13 Apr 2022 12:52:01 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k20-20020a05600c1c9400b0038ecd1ccc17sm3432484wms.35.2022.04.13.12.52.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Apr 2022 12:52:00 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v3 08/29] CI: have "static-analysis" run a "make ci-static-analysis" target Date: Wed, 13 Apr 2022 21:51:21 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.rc2.843.g193535c2aa7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Make it easier to run the CI logic locally by creating "ci-static-analysis" target. Now the "check-coccicheck" logic added in 0860a7641ba (travis-ci: fail if Coccinelle static analysis found something to transform, 2018-07-23) can be easily tested outside of CI, and likewise the sanity check added in 0e7696c64db (ci: disallow directional formatting, 2021-11-04). These targets could be improved, the "hdr-check" target will re-do all of its work every time it's run, and could with a change similar to my c234e8a0ecf (Makefile: make the "sparse" target non-.PHONY, 2021-09-23). The same goes for the new "ci-check-directional-formatting" target. But without those improvements being able to easily run a 1=1 equivalent of the checks we do in CI during a local build is already an improvement. Subsequent commits will expand on this ability. This change will also make the CI check faster, since we can take advantage of parallelism. The "make coccicheck" takes a very long time, and at the tail end we might only have one job pegging 100% of one CPU core for a long time. Now any extra cores will be free to run the rest of the targets under "ci-static-analysis". Signed-off-by: Ævar Arnfjörð Bjarmason --- .github/workflows/main.yml | 3 +-- Makefile | 28 ++++++++++++++++++++++++++++ ci/run-static-analysis.sh | 30 ------------------------------ shared.mak | 1 + 4 files changed, 30 insertions(+), 32 deletions(-) delete mode 100755 ci/run-static-analysis.sh diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 309b9141249..ad752010102 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -317,8 +317,7 @@ jobs: steps: - uses: actions/checkout@v2 - run: ci/install-dependencies.sh - - run: ci/run-static-analysis.sh - - run: ci/check-directional-formatting.bash + - run: make ci-static-analysis sparse: needs: ci-config if: needs.ci-config.outputs.enabled == 'yes' diff --git a/Makefile b/Makefile index f8bccfab5e9..8717ea7e745 100644 --- a/Makefile +++ b/Makefile @@ -3031,6 +3031,20 @@ coccicheck: $(addsuffix .patch,$(filter-out %.pending.cocci,$(wildcard contrib/c # See contrib/coccinelle/README coccicheck-pending: $(addsuffix .patch,$(wildcard contrib/coccinelle/*.pending.cocci)) +.PHONY: check-coccicheck +check-coccicheck: coccicheck + $(QUIET_CHECK)for cocci_patch in contrib/coccinelle/*.patch; do \ + if test -s "$$cocci_patch"; then \ + echo "Coccinelle suggests the following changes in '$$cocci_patch':"; \ + cat "$$cocci_patch"; \ + fail=UnfortunatelyYes; \ + fi \ + done; \ + if test -n "$$fail"; then \ + echo "error: Coccinelle suggested some changes"; \ + exit 1; \ + fi + .PHONY: coccicheck coccicheck-pending ### Installation rules @@ -3466,3 +3480,17 @@ $(FUZZ_PROGRAMS): all $(XDIFF_OBJS) $(EXTLIBS) git.o $@.o $(LIB_FUZZING_ENGINE) -o $@ fuzz-all: $(FUZZ_PROGRAMS) + +### CI "check" targets +# +# These targets are run from the CI, see .github/workflows/main.yml, +# but can also be run manually to run the same assertions locally. + +.PHONY: ci-check-directional-formatting +ci-check-directional-formatting: + $(QUIET_CHECK)ci/check-directional-formatting.bash + +.PHONY: ci-static-analysis +ci-static-analysis: ci-check-directional-formatting +ci-static-analysis: check-coccicheck +ci-static-analysis: hdr-check diff --git a/ci/run-static-analysis.sh b/ci/run-static-analysis.sh deleted file mode 100755 index 5a87b1cac96..00000000000 --- a/ci/run-static-analysis.sh +++ /dev/null @@ -1,30 +0,0 @@ -#!/bin/sh -# -# Perform various static code analysis checks -# - -. ${0%/*}/lib.sh - -make coccicheck - -set +x - -fail= -for cocci_patch in contrib/coccinelle/*.patch -do - if test -s "$cocci_patch" - then - echo "$(tput setaf 1)Coccinelle suggests the following changes in '$cocci_patch':$(tput sgr0)" - cat "$cocci_patch" - fail=UnfortunatelyYes - fi -done - -if test -n "$fail" -then - echo "$(tput setaf 1)error: Coccinelle suggested some changes$(tput sgr0)" - exit 1 -fi - -make hdr-check || -exit 1 diff --git a/shared.mak b/shared.mak index 50d4596f0d6..83e0aa16e7d 100644 --- a/shared.mak +++ b/shared.mak @@ -68,6 +68,7 @@ ifndef V QUIET_HDR = @echo ' ' HDR $(<:hcc=h); QUIET_RC = @echo ' ' RC $@; QUIET_SPATCH = @echo ' ' SPATCH $<; + QUIET_CHECK = @echo ' ' CHECK $@; ## Used in "Documentation/Makefile" QUIET_ASCIIDOC = @echo ' ' ASCIIDOC $@; From patchwork Wed Apr 13 19:51:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12812563 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id ACFE7C433EF for ; Wed, 13 Apr 2022 19:52:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238516AbiDMTyn (ORCPT ); Wed, 13 Apr 2022 15:54:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58328 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238494AbiDMTy3 (ORCPT ); Wed, 13 Apr 2022 15:54:29 -0400 Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 64D3656220 for ; Wed, 13 Apr 2022 12:52:04 -0700 (PDT) Received: by mail-wr1-x432.google.com with SMTP id e21so4121259wrc.8 for ; Wed, 13 Apr 2022 12:52:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=VS0qqJ+MyHkAenOD1VJx9zPTY3FrInp30v7m+6spciM=; b=YpSwndtl8jVansfCgpxd7RB1qCBFXoo4CeCfu48UGi4R9ASfUL7/Juzy9ISc/fGvJj r1y7zOu6JASHSKMzklQLslVDZJ26BAbtF9XUO6m2otkZJf6WI7GkMHqlIG4arvLcgJph 8dc7xjzfG0xOyHKT6uPZRi73Qm0i28yww0ec8hpD3G9GEBHHU1rz1VXGWGETj4hMG4Du PmI6RLqFl0rbXCjk7x8H++0jDZMYuZv9WfgPGEkRP9dXIqMP5VRs+GokxHg12YigFSC2 WbdkPbm7HJsxiwxW73o/vPXRVXmF9HKwTC/F2U7oX05MJ9MS+vhXekLR/bEpMYgDrIXY wiEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=VS0qqJ+MyHkAenOD1VJx9zPTY3FrInp30v7m+6spciM=; b=SwM//t+zG7Rs5DpRgsXQQG6T37R7jexC0k4bpz9FbsqMjy7qeHSZIuQzqfSDXTEoSq t3ZOh9vnV244h5FU7zQ8+gBFxmF7i408p+vlXPJmHFYK30896a5amJqei0fe6i2Nuz40 AV4jT4alzweddUa4KY1kfqtLNEfbp14V8smsgeA+9x7iLoKtbs3T4H1fttWmnabHqAWL e/1phzJsnvl2fEtVJg5FQ0jh85nHpmk92K2pfLV/smvAZSLNLkIHH43XQuJURYPD5x2U 99rCMap7rCPvJmpYAw/QnETC+ivBqxxuEFdbzBi2eQH0Xg2n3Wh70TV5bKuXICm9KdtN 97qw== X-Gm-Message-State: AOAM531+BfUvW79vke7Hz3KBaRFJ2HdMuNlQKdca9abmmFnVBNqFzrex BmvpZJAcLg4fTar96dcWTPPzlSB0tEnClw== X-Google-Smtp-Source: ABdhPJwW6VUYtGm+bO8+EJa5CYZNm6zkhBjr60+coURJO5SuUMIXuhcFbj8fHXTCd5SVO0yS0qX7+g== X-Received: by 2002:adf:fb4d:0:b0:207:a376:7585 with SMTP id c13-20020adffb4d000000b00207a3767585mr309946wrs.489.1649879522641; Wed, 13 Apr 2022 12:52:02 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k20-20020a05600c1c9400b0038ecd1ccc17sm3432484wms.35.2022.04.13.12.52.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Apr 2022 12:52:02 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v3 09/29] CI: have "static-analysis" run "check-builtins", not "documentation" Date: Wed, 13 Apr 2022 21:51:22 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.rc2.843.g193535c2aa7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Move the "make check-builtins" check from the "documentation" job to the "static-analysis" job. The "check-builtins" target added in c74390e4a1d (cherry is built-in, do not ship git-cherry.sh, 2006-11-05) is unrelated to the documentation, so it's odd that b98712b9aa9 (travis-ci: build documentation, 2016-05-04) added it to the "documentation" job. Let's just move it to the "ci-static-analysis" target, and while we're at it improve the output with $(QUIET_CHECK). Signed-off-by: Ævar Arnfjörð Bjarmason --- Makefile | 3 ++- ci/test-documentation.sh | 1 - 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Makefile b/Makefile index 8717ea7e745..2e1ae6c6067 100644 --- a/Makefile +++ b/Makefile @@ -3402,7 +3402,7 @@ check-docs:: ### Make sure built-ins do not have dups and listed in git.c # check-builtins:: - ./check-builtins.sh + $(QUIET_CHECK)./check-builtins.sh ### Test suite coverage testing # @@ -3492,5 +3492,6 @@ ci-check-directional-formatting: .PHONY: ci-static-analysis ci-static-analysis: ci-check-directional-formatting +ci-static-analysis: check-builtins ci-static-analysis: check-coccicheck ci-static-analysis: hdr-check diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh index 9e0652c30dd..41e2b126311 100755 --- a/ci/test-documentation.sh +++ b/ci/test-documentation.sh @@ -14,7 +14,6 @@ filter_log () { "$1" } -make check-builtins make check-docs # Build docs with AsciiDoc From patchwork Wed Apr 13 19:51:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12812561 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 78A90C433EF for ; Wed, 13 Apr 2022 19:52:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238485AbiDMTyi (ORCPT ); Wed, 13 Apr 2022 15:54:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58326 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238496AbiDMTy3 (ORCPT ); Wed, 13 Apr 2022 15:54:29 -0400 Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A50C05F4D0 for ; Wed, 13 Apr 2022 12:52:05 -0700 (PDT) Received: by mail-wr1-x434.google.com with SMTP id c10so4156761wrb.1 for ; Wed, 13 Apr 2022 12:52:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=kYZ5DqbSSL4w0iww5+QZypYcPXBnFvdTFkv3iMHxxeg=; b=aHPwY4i1p/eSnkPRmwczdWSApVYSfRpM1JCW48dBneY3lLduVsE6Ha/lHOBEKlhvbh 4xwmZ+OP9iNDDtgcHsEg4KnaytPy82gmLUgNwQMh5WAMFQ2gYe8JJHkVM0Nd8dhvsHLI TrGMr2d618oAtztichPufdy91tmOvWzPC49tJ4N3Jy8fYDHiMXX1//kbtHiYa7NM9PBv 01RWOBCQdBYgZYRVfZ0OAOMh/ywsk4weEp3ng9SLz1LrkU03dh6lIgAJJAG4s4cOqMCS d2CaqYbf2+e7N2ipYV7US/2nAmpV+fR5SS+HxShD59uOBAlCU5SnXM4ajp2rmquRPtPU QYXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=kYZ5DqbSSL4w0iww5+QZypYcPXBnFvdTFkv3iMHxxeg=; b=kcm5xgMQAMceng/4z39mn435MgazknIC7KbHlem4t5apxUkjWKVyv+K7HEZJpxuonc qLE/SKP1NkSOAM19SRLt5maMvwjkYQQ6bwY32kGI4BoBvXaQD0dGjlM4O5zt6UvXZBZ2 iHf66UapiORWhGY74hgOjJ+6dZ12m1tpqY4kxF2EqcoWTLCIuesT5cOuVD5XNog8Ad94 qvG9RDQGk8QbzQqILMxQWpqt1SCOy0VBdYs3NZ+iDDFqFr3LKRfp6sTWD5XTQIypDV72 H0wSBIBg3IpOX9lVOQK0vsIhqgf/T7kz8tlHiDSZQJ33blTFt6hNjY1+8VkjsVF3IwnR 8rCg== X-Gm-Message-State: AOAM530R73yLMYMkSNQoC53poWuq3QjSsTFXhdeBvcryao21S756Iwe3 C20nxcGYz2XVIE98ihyPFeEmAu2R3RtVjw== X-Google-Smtp-Source: ABdhPJysRhKR7kt9dXb1MIkwcvAYSYwTi7qhyb6i/VYGUbJNwkvBDN+2J8CpZz6WzToehJYfM4/GFQ== X-Received: by 2002:adf:9581:0:b0:1ed:c341:4ed1 with SMTP id p1-20020adf9581000000b001edc3414ed1mr310646wrp.299.1649879523679; Wed, 13 Apr 2022 12:52:03 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k20-20020a05600c1c9400b0038ecd1ccc17sm3432484wms.35.2022.04.13.12.52.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Apr 2022 12:52:03 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v3 10/29] CI: move p4 and git-lfs variables to ci/install-dependencies.sh Date: Wed, 13 Apr 2022 21:51:23 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.rc2.843.g193535c2aa7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Move the declaration of variables that are only used by the "ubuntu-latest" block in "ci/install-dependencies.sh" there from "ci/lib.sh". This makes the code easier to follow, and changes "ci/lib.sh" to a library that only exports CI variables for general use, we didn't need to export these $P4_PATH and $GIT_LFS_PATH variables. Signed-off-by: Ævar Arnfjörð Bjarmason --- ci/install-dependencies.sh | 17 +++++++++++++++-- ci/lib.sh | 12 ------------ 2 files changed, 15 insertions(+), 14 deletions(-) diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh index e7ea8799411..34b4400f38b 100755 --- a/ci/install-dependencies.sh +++ b/ci/install-dependencies.sh @@ -5,14 +5,27 @@ . ${0%/*}/lib.sh -P4WHENCE=http://filehost.perforce.com/perforce/r$LINUX_P4_VERSION -LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION UBUNTU_COMMON_PKGS="make libssl-dev libcurl4-openssl-dev libexpat-dev tcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl libemail-valid-perl libio-socket-ssl-perl libnet-smtp-ssl-perl" case "$runs_on_pool" in ubuntu-latest) + # The Linux build installs the defined dependency versions below. + # The OS X build installs much more recent versions, whichever + # were recorded in the Homebrew database upon creating the OS X + # image. + # Keep that in mind when you encounter a broken OS X build! + LINUX_P4_VERSION="16.2" + LINUX_GIT_LFS_VERSION="1.5.2" + + P4_PATH="$HOME/custom/p4" + GIT_LFS_PATH="$HOME/custom/git-lfs" + export PATH="$GIT_LFS_PATH:$P4_PATH:$PATH" + + P4WHENCE=http://filehost.perforce.com/perforce/r$LINUX_P4_VERSION + LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION + sudo apt-get -q update sudo apt-get -q -y install language-pack-is libsvn-perl apache2 \ $UBUNTU_COMMON_PKGS $CC_PACKAGE diff --git a/ci/lib.sh b/ci/lib.sh index 21c009757f4..5736a4d2e04 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -67,18 +67,6 @@ ubuntu-latest) fi export GIT_TEST_HTTPD=true - - # The Linux build installs the defined dependency versions below. - # The OS X build installs much more recent versions, whichever - # were recorded in the Homebrew database upon creating the OS X - # image. - # Keep that in mind when you encounter a broken OS X build! - export LINUX_P4_VERSION="16.2" - export LINUX_GIT_LFS_VERSION="1.5.2" - - P4_PATH="$HOME/custom/p4" - GIT_LFS_PATH="$HOME/custom/git-lfs" - export PATH="$GIT_LFS_PATH:$P4_PATH:$PATH" ;; macos-latest) if [ "$jobname" = osx-gcc ] From patchwork Wed Apr 13 19:51:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12812562 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B9F51C433EF for ; Wed, 13 Apr 2022 19:52:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238502AbiDMTyk (ORCPT ); Wed, 13 Apr 2022 15:54:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58306 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238501AbiDMTy3 (ORCPT ); Wed, 13 Apr 2022 15:54:29 -0400 Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com [IPv6:2a00:1450:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 525DA5F4F3 for ; Wed, 13 Apr 2022 12:52:06 -0700 (PDT) Received: by mail-wr1-x42d.google.com with SMTP id g18so4115374wrb.10 for ; Wed, 13 Apr 2022 12:52:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=95G+ydbQyhJHHHkhIbeovRa95gH5GpdV/tUGZfCYYDg=; b=Rh7tP1Si/3h73NrgJzGOiHiOFLgrGyiWnAvy3LfqYxyzD3QbMYLmNurxfyNIRzvOnK VofGLuBPcdD4+rQFsFFU8yT5W/k+OSaV7QwDQl1Vmd+1xx9YT9XqLtXx+Ricrd/bYPqS NaxpYh/9ZdUL9Tw7QWM+tEW9aJ81HYNkjFCunJ8xRI77Df6zGEOJbcbiMpwYDhUvRRKj SwNNWBsBIy997fvzQHk3A5qAKFMWTQ1baUzU3F7E8ad5H5Pd8KLJopa1GY9sCGZVn+em qi22JAJXJsUuReoEkoPtoLev0I0UANWugDFfHCVFjsBnH3ItgWbiaYOjZB7Vsqrcdd1G xQPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=95G+ydbQyhJHHHkhIbeovRa95gH5GpdV/tUGZfCYYDg=; b=2ZOpMrEusi9bCRWqzAtxXDzsmT9aqKjJu6DAQVJsqcXJEcP2Bw9QqoEFFVWViZZ8LK Qi16UWwHuarxoRLmXG7u4bTMV/VGrPP9W52LSMA4NEPKKGiVe9KtN7mahTZ6TpHa621l O3o3AahQpT9Mug+PvfjLHGOkw1O3Sq4YhVSjodHJDLcdREcrZIdMTYqPN74uIFi1XSL1 NnQX7ieqsN2HGmqMYl3M4kvdItAA0dVGDfNSkLVIowFGwU6+NvX6mq0jPjEjidaj3aqE vQkMu8bTVoI8RTfSH1OOKjOPIQwVDjIk10qqgo+w7j5uOTI9ccUUeXaPnMOHSFcG8Onk jXZw== X-Gm-Message-State: AOAM530rcbO/VFZRom+ZwuEh+ccXDuBx6YUvF4EtUPPhCorwp+S9pUL9 x1k/ZFVlMqhzM5y/nxVtcjE8PgeCuC0wIA== X-Google-Smtp-Source: ABdhPJyQ34KJ5l6ssIIhOi5CPWMK964K8FG5pxQOTBIHBM0o1A69J/3SOOQuOcyflxy7y3DAU4i9CA== X-Received: by 2002:a05:6000:15c1:b0:204:7e75:1eb1 with SMTP id y1-20020a05600015c100b002047e751eb1mr347462wry.482.1649879524689; Wed, 13 Apr 2022 12:52:04 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k20-20020a05600c1c9400b0038ecd1ccc17sm3432484wms.35.2022.04.13.12.52.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Apr 2022 12:52:03 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v3 11/29] CI: consistently use "export" in ci/lib.sh Date: Wed, 13 Apr 2022 21:51:24 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.rc2.843.g193535c2aa7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Change the "ci/lib.sh" script co consistently use "export", for e.g. MAKEFLAGS we were exporting it, and then assigning to it, let's do it the other way around. Right now this doesn't matter, since we in e.g. "ci/install-dependencies.sh" source this file, and don't use something like "env(1)" to retrieve these variables. But in a subsequent commit we'll "export" these variables through a wrapper (to additionally write them to a GitHub CI-specific $GITHUB_ENV file). This change makes that subsequent change easier to read, as it won't need to do any control flow refactoring. Signed-off-by: Ævar Arnfjörð Bjarmason --- ci/lib.sh | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/ci/lib.sh b/ci/lib.sh index 5736a4d2e04..5783eed1b20 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -30,7 +30,7 @@ export TERM=${TERM:-dumb} NPROC=10 # Clear MAKEFLAGS that may come from the outside world. -export MAKEFLAGS=--jobs=$NPROC +MAKEFLAGS=--jobs=$NPROC if test "$GITHUB_ACTIONS" = "true" then @@ -38,9 +38,10 @@ then CC="${CC:-gcc}" export GIT_PROVE_OPTS="--timer --jobs $NPROC" - export GIT_TEST_OPTS="--verbose-log -x" + GIT_TEST_OPTS="--verbose-log -x" test Windows != "$RUNNER_OS" || GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS" + export GIT_TEST_OPTS else echo "Could not identify CI type" >&2 env >&2 @@ -94,4 +95,4 @@ linux-leaks) ;; esac -MAKEFLAGS="$MAKEFLAGS CC=${CC:-cc}" +export MAKEFLAGS="$MAKEFLAGS CC=${CC:-cc}" From patchwork Wed Apr 13 19:51:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12812564 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A73D1C433EF for ; Wed, 13 Apr 2022 19:52:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238522AbiDMTyw (ORCPT ); Wed, 13 Apr 2022 15:54:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58398 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238508AbiDMTya (ORCPT ); Wed, 13 Apr 2022 15:54:30 -0400 Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com [IPv6:2a00:1450:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 81D56716FD for ; Wed, 13 Apr 2022 12:52:07 -0700 (PDT) Received: by mail-wr1-x42d.google.com with SMTP id b19so4108537wrh.11 for ; Wed, 13 Apr 2022 12:52:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=65wWbrrQGYhG3qKb95NMDH0xy7DyeMltzZbqs4tYmqw=; b=Y5qmw3GRgY2E6m4TSNkdN5K4sje329+VGN43SuUOdya2SGjm+fvXgx5T+t9r4ZdU1f m1Z7aXnipYL/FmPkVJOjsCGfVQ1tfsbM4AczMNUqZttlcIs+r1z7Cem676gfQ7zI/JCz tsNAgG4gGbuJQDh+Q8WgIrYAa9O84JJznu/0Vq5/9Cuz5EoYVosibqR+aqFeErZt9vwd MfzSi//pqtOoKApa3WY8NHw1AlP9T4DEJmS+Z+Qoo7fgjLErPcBepsAAg1U/SU3HoBXd kQLAn+mhFfeuADF5CMoUuADvCNLHGLSr0VbUsgN/4mhy9lAk08EyIhOgWdSYu2lYZGW8 i6gQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=65wWbrrQGYhG3qKb95NMDH0xy7DyeMltzZbqs4tYmqw=; b=lVs/eFgd3qCD1wwKJo60zyliHpj6RPUQRH5e+ErApMRy+megCddeIgl/MDcP6cpPks Ragag/rb0WmickY52UN4dQWY9A4Bjf5uxi8N9EuO9hkWPwNboUsR0ChaQ4NQhVHs5LE6 mzMRG22tbEGEQRRDxpE75nSgHZCr67eTQLDwrrU/50S6N4xLVsgTOG/arRLwFf9g16QZ LQ1ZbTdnK3rMx8aSoOIsrGzuJCAUdr8K9uxQcMOW2pyGD75cM4GOo62d73OdsSZoaSXS UA4pR9it1zzsxBttJoUB2YaY7C0xUdzL7yUypZwR2uEmM+dDSy4vMs8HgdbBFP83jlaz xXhg== X-Gm-Message-State: AOAM531Gs/JVCaLzF+HmQ22bKe1fTY09Ii/9X/6tf6elopFnYQ4/s61g PwMqJ3DhKM1yls/AR+BGkK/yKFY1/eVzFg== X-Google-Smtp-Source: ABdhPJwae1Qip+TC1+rX5760wW+rw0BXIe+5k9mkIAtcEjZmARM7jw3NU6NAkWkR/rrM2dovhEuz9w== X-Received: by 2002:a05:6000:1887:b0:204:2917:ad3e with SMTP id a7-20020a056000188700b002042917ad3emr364831wri.138.1649879525721; Wed, 13 Apr 2022 12:52:05 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k20-20020a05600c1c9400b0038ecd1ccc17sm3432484wms.35.2022.04.13.12.52.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Apr 2022 12:52:05 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v3 12/29] CI: export variables via a wrapper Date: Wed, 13 Apr 2022 21:51:25 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.rc2.843.g193535c2aa7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Add and use a "setenv" wrapper to export the variables set in ci/lib.sh. For now this merely adds debugging information about what variables are set, but in a subsequent commit we'll expand this to set variables in $GITHUB_ENV instead of the environment, and thus carry variable settings between steps in a job. We'll also eventually be interested in what type of variable we're setting, and when it needs to be used, because we'll prominently expose what variables we set in what CI "step", so having only the ones we actually use will improve readability. To that end call the "setenv" function with optional "--build", "--test" and "--all" parameters to indicate whether they're needed for building, testing or both. We currently ignore the "varname" parameter, but will make use of it in a subsequent commit. Adding it here reduces the needless churn, as we won't need to change all these lines again for that subsequent change. Signed-off-by: Ævar Arnfjörð Bjarmason --- ci/lib.sh | 51 ++++++++++++++++++++++++++++++--------- ci/run-build-and-tests.sh | 34 +++++++++++++------------- 2 files changed, 57 insertions(+), 28 deletions(-) diff --git a/ci/lib.sh b/ci/lib.sh index 5783eed1b20..3727e6ec2f9 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -13,6 +13,35 @@ then exit 1 fi +# Helper functions +setenv () { + while test $# != 0 + do + case "$1" in + --build) + ;; + --test) + ;; + --all) + ;; + -*) + echo "BUG: bad setenv() option '$1'" >&2 + exit 1 + ;; + *) + break + ;; + esac + shift + done + + key=$1 + val=$2 + shift 2 + + eval "export $key=\"$val\"" +} + check_unignored_build_artifacts () { ! git ls-files --other --exclude-standard --error-unmatch \ @@ -24,7 +53,7 @@ check_unignored_build_artifacts () } # GitHub Action doesn't set TERM, which is required by tput -export TERM=${TERM:-dumb} +setenv TERM ${TERM:-dumb} # How many jobs to run in parallel? NPROC=10 @@ -37,21 +66,21 @@ then CI_TYPE=github-actions CC="${CC:-gcc}" - export GIT_PROVE_OPTS="--timer --jobs $NPROC" + setenv --test GIT_PROVE_OPTS "--timer --jobs $NPROC" GIT_TEST_OPTS="--verbose-log -x" test Windows != "$RUNNER_OS" || GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS" - export GIT_TEST_OPTS + setenv --test GIT_TEST_OPTS "$GIT_TEST_OPTS" else echo "Could not identify CI type" >&2 env >&2 exit 1 fi -export DEVELOPER=1 -export DEFAULT_TEST_TARGET=prove -export GIT_TEST_CLONE_2GB=true -export SKIP_DASHED_BUILT_INS=YesPlease +setenv --build DEVELOPER 1 +setenv --test DEFAULT_TEST_TARGET prove +setenv --test GIT_TEST_CLONE_2GB true +setenv --build SKIP_DASHED_BUILT_INS YesPlease case "$runs_on_pool" in ubuntu-latest) @@ -67,7 +96,7 @@ ubuntu-latest) MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=/usr/bin/python2" fi - export GIT_TEST_HTTPD=true + setenv --test GIT_TEST_HTTPD true ;; macos-latest) if [ "$jobname" = osx-gcc ] @@ -90,9 +119,9 @@ linux-musl) MAKEFLAGS="$MAKEFLAGS GIT_TEST_UTF8_LOCALE=C.UTF-8" ;; linux-leaks) - export SANITIZE=leak - export GIT_TEST_PASSING_SANITIZE_LEAK=true + setenv --build SANITIZE leak + setenv --test GIT_TEST_PASSING_SANITIZE_LEAK true ;; esac -export MAKEFLAGS="$MAKEFLAGS CC=${CC:-cc}" +setenv --all MAKEFLAGS "$MAKEFLAGS CC=${CC:-cc}" diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh index e5a21985b44..35d45a9373e 100755 --- a/ci/run-build-and-tests.sh +++ b/ci/run-build-and-tests.sh @@ -9,33 +9,33 @@ export MAKE_TARGETS="all test" case "$jobname" in linux-gcc) - export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main + setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main ;; linux-TEST-vars) - export GIT_TEST_SPLIT_INDEX=yes - export GIT_TEST_MERGE_ALGORITHM=recursive - export GIT_TEST_FULL_IN_PACK_ARRAY=true - export GIT_TEST_OE_SIZE=10 - export GIT_TEST_OE_DELTA_SIZE=5 - export GIT_TEST_COMMIT_GRAPH=1 - export GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS=1 - export GIT_TEST_MULTI_PACK_INDEX=1 - export GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP=1 - export GIT_TEST_ADD_I_USE_BUILTIN=1 - export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=master - export GIT_TEST_WRITE_REV_INDEX=1 - export GIT_TEST_CHECKOUT_WORKERS=2 + setenv --test GIT_TEST_SPLIT_INDEX yes + setenv --test GIT_TEST_MERGE_ALGORITHM recursive + setenv --test GIT_TEST_FULL_IN_PACK_ARRAY true + setenv --test GIT_TEST_OE_SIZE 10 + setenv --test GIT_TEST_OE_DELTA_SIZE 5 + setenv --test GIT_TEST_COMMIT_GRAPH 1 + setenv --test GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS 1 + setenv --test GIT_TEST_MULTI_PACK_INDEX 1 + setenv --test GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP 1 + setenv --test GIT_TEST_ADD_I_USE_BUILTIN 1 + setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME master + setenv --test GIT_TEST_WRITE_REV_INDEX 1 + setenv --test GIT_TEST_CHECKOUT_WORKERS 2 ;; linux-clang) - export GIT_TEST_DEFAULT_HASH=sha1 + setenv --test GIT_TEST_DEFAULT_HASH sha1 ;; linux-sha256) - export GIT_TEST_DEFAULT_HASH=sha256 + setenv --test GIT_TEST_DEFAULT_HASH sha256 ;; pedantic) # Don't run the tests; we only care about whether Git can be # built. - export DEVOPTS=pedantic + setenv --build DEVOPTS pedantic export MAKE_TARGETS=all ;; esac From patchwork Wed Apr 13 19:51:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12812568 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 12725C433EF for ; Wed, 13 Apr 2022 19:52:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238527AbiDMTy6 (ORCPT ); Wed, 13 Apr 2022 15:54:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58452 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238499AbiDMTyb (ORCPT ); Wed, 13 Apr 2022 15:54:31 -0400 Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com [IPv6:2a00:1450:4864:20::42a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C5C994EF42 for ; Wed, 13 Apr 2022 12:52:08 -0700 (PDT) Received: by mail-wr1-x42a.google.com with SMTP id c10so4156931wrb.1 for ; Wed, 13 Apr 2022 12:52:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=/Dc+hK8vz5gzAn5XbB26784E/5X3cIurpe4OoZLIjvs=; b=XTrQdR29v9gHHjgfV93D8O9cWXKYdK5QiqtO3fs1sx/YMXdeJBAnQgekgUD8RExxqb gsjJLUTyuMnTVbx3UxIYBPR4e06UODmMdp4Pme2bED0pK7u0RNsEfy9PMOCrGluxnb9h 2y6sJ5H8bOHo4Lp9UtYapIUMcRXQitlXKuo35edHAcT1/WgsIEaP1aJ4g4C3hj13Jxkb O80jOviMgDy19J8hUuiwNqgDzdogoi5ngTqYvmnJlS5spa0j3m1gYDTGcSgG7VVryMJT wgMDSTTnkaBhiy+JSaCMeG76RkPLQ/57HbB2hjjejY0aKV6am805XTvM3CrimPm+gv71 KoXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=/Dc+hK8vz5gzAn5XbB26784E/5X3cIurpe4OoZLIjvs=; b=sSldstrOhJWM9UpHQn1h2XP7ys8Z7x8fwwTRQGQadUa2dVZw3sc4CR2S8yW8BtFU2q uBWbB+hmcgv/4wqff4aDu7UulznJfFNCU22wwINu6HqVRRGECiFzjxSczqiUWSWD8e/V bJMKMPaSUXsV0lmKT/nDY+/CN+XVwy8BjanmPXx3IbUgBQl2wFLwg1hx7IY4gFjMJjjX CQC1qcohXESapvB+9PMVoPOEmzLqOpfWW3lYX397mPKCgtef6O2k1/zLsF7aGyiK4uMv HkIa3W8DvRCRDASq/jZziJO4AZYhpuevmzK0uVUnWAJ+vnzZtMYj0lj6m+Gg5qM0RLsc qQYA== X-Gm-Message-State: AOAM533JUs8tk4ypk/EhjBVfm4iIgBbMEaU2JFW8nFswRys44Tb7EIL9 H0dFgTX8djeREQ8tvs9fQ9tQlYSkkZNjQg== X-Google-Smtp-Source: ABdhPJyaWttVRJHvSFD2DiNsdOpe/aHaG5nB4nn1RV3IzJrrCgO0tudZhfuQtDTWUTL5OT52pImN0g== X-Received: by 2002:adf:e2cc:0:b0:203:e8ba:c709 with SMTP id d12-20020adfe2cc000000b00203e8bac709mr311894wrj.713.1649879526788; Wed, 13 Apr 2022 12:52:06 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k20-20020a05600c1c9400b0038ecd1ccc17sm3432484wms.35.2022.04.13.12.52.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Apr 2022 12:52:06 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v3 13/29] CI: remove "run-build-and-tests.sh", run "make [test]" directly Date: Wed, 13 Apr 2022 21:51:26 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.rc2.843.g193535c2aa7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Remove the already thin "ci/run-build-and-tests.sh" wrapper and instead make the CI run "make" or "make test" directly. By doing this we'll be able to easily see at a glance whether our failure was in the compilation or testing, whether that's via human eyes or improve machine readability. We also need to run our new "ci/check-unignored-build-artifacts.sh" on success() in the CI now, just like we already had a step conditional on failure() running ci/print-test-failures.sh. The reason we used a "ci/run-build-and-tests.sh" wrapper in the first place had to do with Travis CI-specific constraints that no longer apply to us, as the Travis CI support has been removed. Instead we can configure the CI in an earlier step by running "ci/lib.sh", which under GitHub CI will write the environment variables we need to the "$GITHUB_ENV" file. We'll then have access to them in subsequent steps, and crucially those variables will be prominently visible at the start of each step via an expandable drop-down in the UI.drop-do. I.e. this changes the CI run from a top-down flow like (pseudocode): - job: - step1: - use ci/lib.sh to set env vars - run a script like ci/run-build-and-tests.sh - step2: - if: failure() - use ci/lib.sh to set env vars - run ci/print-test-failures.sh To: - job: - step1: - set variables in $GITHUB_ENV using ci/lib.sh - step2: - make - step3: - make test - step4: - if: failure() - run ci/print-test-failures.sh - step5: - if: success() - run ci/check-unignored-build-artifacts.sh There is a proposal[2] to get some of the benefits of this approach by not re-arranging our variable setup in this way, but to instead use the GitHub CI grouping syntax to focus on the relevant parts of "make" or "make test" when we have failures. Doing it this way makes for better looking GitHub CI UI, and lays much better ground work for our CI going forward. Because: * The CI logic will be more portable to a future CI system, since a common feature of them is to run various commands in sequence, but a future system won't necessarily support the GitHub-specifics syntax of "grouping" output within a "step". Even if those systems don't support a "$GITHUB_ENV" emulating will be much easier than to deal with some CI-specific grouping syntax. * At the start of every step the GitHub CI presents an expandable list of environment variables from "$GITHUB_ENV". We'll now see exactly what variables affected that step (although we currently overshoot that a bit, and always define all variables). * CI failures will be easier to reproduce locally, as this makes the relevant ci/* scripts something that sets up our environment, but leaves "make" and "make test" working as they do locally. To reproduce a run the user only needs to set the variables discussed in the drop-down above, either manually or by running "ci/lib.sh". * The output will be less verbose. The "ci/lib.sh" script uses "set -x", and before this e.g. "ci/static-analysis.sh" would start with 40 lines of trace output, culminating in using "export" to export the relevant environment variables. Now that verbosity is in the earlier "ci/lib.sh" step, and not in any subsequent one. The "make" targets then start out with the relevant output non-trace output right away. * If we do want to use the grouping syntax within a "step" it'll now be easier to do so. It doesn't support nesting, so we'd have to make a choice between using it for e.g. "make" v.s. "make test", or individual test failures. See "sadly" in [3]. 1. https://lore.kernel.org/git/211120.86k0h30zuw.gmgdl@evledraar.gmail.com/ 2. https://lore.kernel.org/git/pull.1117.git.1643050574.gitgitgadget@gmail.com/ 3. https://lore.kernel.org/git/9333ba781b8240f704e739b00d274f8c3d887e39.1643050574.git.gitgitgadget@gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason --- .github/workflows/main.yml | 14 ++++++-- ci/check-unignored-build-artifacts.sh | 8 +++++ ci/install-dependencies.sh | 4 +++ ci/lib.sh | 36 +++++++++++++++++++- ci/make-test-artifacts.sh | 2 -- ci/run-build-and-tests.sh | 47 --------------------------- ci/run-test-slice.sh | 2 -- ci/test-documentation.sh | 1 - 8 files changed, 58 insertions(+), 56 deletions(-) create mode 100755 ci/check-unignored-build-artifacts.sh delete mode 100755 ci/run-build-and-tests.sh diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index ad752010102..c83f6a21ca3 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -268,7 +268,10 @@ jobs: steps: - uses: actions/checkout@v2 - run: ci/install-dependencies.sh - - run: ci/run-build-and-tests.sh + - run: ci/lib.sh + - run: make + - run: make test + if: success() - run: ci/print-test-failures.sh if: failure() - name: Upload failed tests' directories @@ -292,6 +295,7 @@ jobs: image: daald/ubuntu32:xenial - jobname: pedantic image: fedora + skip-tests: no env: jobname: ${{matrix.vector.jobname}} runs-on: ubuntu-latest @@ -299,9 +303,12 @@ jobs: steps: - uses: actions/checkout@v1 - run: ci/install-docker-dependencies.sh - - run: ci/run-build-and-tests.sh + - run: ci/lib.sh + - run: make + - run: make test + if: success() && matrix.vector.skip-tests != 'no' - run: ci/print-test-failures.sh - if: failure() + if: failure() && matrix.vector.skip-tests != 'no' - name: Upload failed tests' directories if: failure() && env.FAILED_TEST_ARTIFACTS != '' uses: actions/upload-artifact@v1 @@ -317,6 +324,7 @@ jobs: steps: - uses: actions/checkout@v2 - run: ci/install-dependencies.sh + - run: ci/lib.sh - run: make ci-static-analysis sparse: needs: ci-config diff --git a/ci/check-unignored-build-artifacts.sh b/ci/check-unignored-build-artifacts.sh new file mode 100755 index 00000000000..56d04b0db9a --- /dev/null +++ b/ci/check-unignored-build-artifacts.sh @@ -0,0 +1,8 @@ +#!/bin/sh +# +# Check whether the build created anything not in our .gitignore +# + +. ${0%/*}/lib.sh + +check_unignored_build_artifacts diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh index 34b4400f38b..01006379fd0 100755 --- a/ci/install-dependencies.sh +++ b/ci/install-dependencies.sh @@ -22,6 +22,10 @@ ubuntu-latest) P4_PATH="$HOME/custom/p4" GIT_LFS_PATH="$HOME/custom/git-lfs" export PATH="$GIT_LFS_PATH:$P4_PATH:$PATH" + if test -n "$GITHUB_PATH" + then + echo "$PATH" >>"$GITHUB_PATH" + fi P4WHENCE=http://filehost.perforce.com/perforce/r$LINUX_P4_VERSION LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION diff --git a/ci/lib.sh b/ci/lib.sh index 3727e6ec2f9..cc9a368be42 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -1,3 +1,5 @@ +#!/bin/sh + # Library of functions shared by all CI scripts # Set 'exit on error' for all CI scripts to let the caller know that @@ -39,7 +41,10 @@ setenv () { val=$2 shift 2 - eval "export $key=\"$val\"" + if test -n "$GITHUB_ENV" + then + echo "$key=$val" >>"$GITHUB_ENV" + fi } check_unignored_build_artifacts () @@ -109,6 +114,35 @@ macos-latest) esac case "$jobname" in +linux-gcc) + setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main + ;; +linux-TEST-vars) + setenv --test GIT_TEST_SPLIT_INDEX yes + setenv --test GIT_TEST_MERGE_ALGORITHM recursive + setenv --test GIT_TEST_FULL_IN_PACK_ARRAY true + setenv --test GIT_TEST_OE_SIZE 10 + setenv --test GIT_TEST_OE_DELTA_SIZE 5 + setenv --test GIT_TEST_COMMIT_GRAPH 1 + setenv --test GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS 1 + setenv --test GIT_TEST_MULTI_PACK_INDEX 1 + setenv --test GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP 1 + setenv --test GIT_TEST_ADD_I_USE_BUILTIN 1 + setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME master + setenv --test GIT_TEST_WRITE_REV_INDEX 1 + setenv --test GIT_TEST_CHECKOUT_WORKERS 2 + ;; +linux-clang) + setenv --test GIT_TEST_DEFAULT_HASH sha1 + ;; +linux-sha256) + setenv --test GIT_TEST_DEFAULT_HASH sha256 + ;; +pedantic) + # Don't run the tests; we only care about whether Git can be + # built. + setenv --build DEVOPTS pedantic + ;; linux32) CC=gcc ;; diff --git a/ci/make-test-artifacts.sh b/ci/make-test-artifacts.sh index 646967481f6..45298562982 100755 --- a/ci/make-test-artifacts.sh +++ b/ci/make-test-artifacts.sh @@ -8,5 +8,3 @@ mkdir -p "$1" # in case ci/lib.sh decides to quit early . ${0%/*}/lib.sh make artifacts-tar ARTIFACTS_DIRECTORY="$1" - -check_unignored_build_artifacts diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh deleted file mode 100755 index 35d45a9373e..00000000000 --- a/ci/run-build-and-tests.sh +++ /dev/null @@ -1,47 +0,0 @@ -#!/bin/sh -# -# Build and test Git -# - -. ${0%/*}/lib.sh - -export MAKE_TARGETS="all test" - -case "$jobname" in -linux-gcc) - setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main - ;; -linux-TEST-vars) - setenv --test GIT_TEST_SPLIT_INDEX yes - setenv --test GIT_TEST_MERGE_ALGORITHM recursive - setenv --test GIT_TEST_FULL_IN_PACK_ARRAY true - setenv --test GIT_TEST_OE_SIZE 10 - setenv --test GIT_TEST_OE_DELTA_SIZE 5 - setenv --test GIT_TEST_COMMIT_GRAPH 1 - setenv --test GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS 1 - setenv --test GIT_TEST_MULTI_PACK_INDEX 1 - setenv --test GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP 1 - setenv --test GIT_TEST_ADD_I_USE_BUILTIN 1 - setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME master - setenv --test GIT_TEST_WRITE_REV_INDEX 1 - setenv --test GIT_TEST_CHECKOUT_WORKERS 2 - ;; -linux-clang) - setenv --test GIT_TEST_DEFAULT_HASH sha1 - ;; -linux-sha256) - setenv --test GIT_TEST_DEFAULT_HASH sha256 - ;; -pedantic) - # Don't run the tests; we only care about whether Git can be - # built. - setenv --build DEVOPTS pedantic - export MAKE_TARGETS=all - ;; -esac - -# Any new "test" targets should not go after this "make", but should -# adjust $MAKE_TARGETS. Otherwise compilation-only targets above will -# start running tests. -make $MAKE_TARGETS -check_unignored_build_artifacts diff --git a/ci/run-test-slice.sh b/ci/run-test-slice.sh index b9a682b4bcd..70326961454 100755 --- a/ci/run-test-slice.sh +++ b/ci/run-test-slice.sh @@ -8,5 +8,3 @@ make --quiet -C t T="$(cd t && ./helper/test-tool path-utils slice-tests "$1" "$2" t[0-9]*.sh | tr '\n' ' ')" - -check_unignored_build_artifacts diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh index 41e2b126311..6b5cce03bd7 100755 --- a/ci/test-documentation.sh +++ b/ci/test-documentation.sh @@ -39,4 +39,3 @@ test -s Documentation/git.html grep ' X-Patchwork-Id: 12812565 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6FFE1C433EF for ; Wed, 13 Apr 2022 19:52:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238524AbiDMTyy (ORCPT ); Wed, 13 Apr 2022 15:54:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58442 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238498AbiDMTyb (ORCPT ); Wed, 13 Apr 2022 15:54:31 -0400 Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9E95245050 for ; Wed, 13 Apr 2022 12:52:09 -0700 (PDT) Received: by mail-wr1-x42f.google.com with SMTP id t1so4141646wra.4 for ; Wed, 13 Apr 2022 12:52:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=7K6v0a1KKYnl6d5SIBnL2OAl1nvW3QnnjHEmdvfZVsY=; b=M4KjzsYkIAv7oeYuDtT/8n1usxicYRLFEgo0F+Iqb225n75jNpdr2kPxBJJes9Wz4c BqV7D/YGLFhuOJ4p8rJg9X1RtTuiqNeEcJ6T/AVb5Non9GAgf2onOX2PgTSQVh7qMjt6 lP//r0U880RvKPM8JuRJj0UZTYZea1ydub+EJxSfJifyMhB3979gg5w8DwHiurmUd/Et zNRl2XharqF8BpLlJxnhq2udspv9KMH9/K3MPKTL+O4vXDPQw5y3LwxJ3FUPXqPqbdYh +L7NnJUCUY8pDSWEh4ewYxxipeTdpBvfBz1czpxfxs0h3P6JLTP6IcZ3GyO6mFZmky6A h2yA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=7K6v0a1KKYnl6d5SIBnL2OAl1nvW3QnnjHEmdvfZVsY=; b=3uJpu0ZgeWimI5g0+GY9N3yv/jXZMltzPCD+Mdmy83WcMfTXAbCgEthS6fCWEp1ZRT tRLrr52kQlJef8xHpM/XNUbpiuO16Wmo4fPS/dUM0xfqaJtbWJutFvNKqVCmTdKeoDvM engoX4l+kJVazjzOdCjrHdSnlWokf0MxqU7DLx2QJZJi2RKnxvVanHoBM7//Jl3Ttf94 A+dZsuhi0pWJR13BiBGl66DEfZMhcUHUjT+AbbtB9pnRt/3ka9U5OuNfeVZmFMNHBIpA i8NHqv4IhCwEt+cYNj4+dzXa7EcIHhJ425wKPHEXe25embTAfxuXhGiaYbws+G3eshnk nCSQ== X-Gm-Message-State: AOAM5335jwjpHaWMIJbj3mVPPmiDWfTRj0vaQ4SCCg6B4HtMuf8cNvbp v30Tn20WXhPzP9oyQlvmoXND5O1OxZjmJQ== X-Google-Smtp-Source: ABdhPJz9Riq3I/tPCv6ok46kK2iVmP0IBnzFX4l75D06YwxyQLl5/gSzK/tci+PKhFiuxz9JZKjS8g== X-Received: by 2002:adf:d1e8:0:b0:207:a5e5:ef9b with SMTP id g8-20020adfd1e8000000b00207a5e5ef9bmr357175wrd.200.1649879527731; Wed, 13 Apr 2022 12:52:07 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k20-20020a05600c1c9400b0038ecd1ccc17sm3432484wms.35.2022.04.13.12.52.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Apr 2022 12:52:07 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v3 14/29] ci/lib.sh: use "test" instead of "[" Date: Wed, 13 Apr 2022 21:51:27 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.rc2.843.g193535c2aa7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Change the ci/lib.sh script to use "if test" rather than "if [". Until the preceding commit this script relied on the non-POSIX "export X=Y" syntax, but now that it's POSIX portable let's use our usual style in it, and not the bash-specific style that was used in it historically. Signed-off-by: Ævar Arnfjörð Bjarmason --- ci/lib.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/ci/lib.sh b/ci/lib.sh index cc9a368be42..ffc0e524aaa 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -94,7 +94,7 @@ ubuntu-latest) break fi - if [ "$jobname" = linux-gcc ] + if test "$jobname" = linux-gcc then MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=/usr/bin/python3" else @@ -104,7 +104,7 @@ ubuntu-latest) setenv --test GIT_TEST_HTTPD true ;; macos-latest) - if [ "$jobname" = osx-gcc ] + if test "$jobname" = osx-gcc then MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)" else From patchwork Wed Apr 13 19:51:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12812566 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C599DC433F5 for ; Wed, 13 Apr 2022 19:52:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238526AbiDMTy4 (ORCPT ); Wed, 13 Apr 2022 15:54:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58326 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238507AbiDMTyc (ORCPT ); Wed, 13 Apr 2022 15:54:32 -0400 Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com [IPv6:2a00:1450:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 47E285DA2F for ; Wed, 13 Apr 2022 12:52:10 -0700 (PDT) Received: by mail-wr1-x433.google.com with SMTP id m14so4139632wrb.6 for ; Wed, 13 Apr 2022 12:52:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=60nuax5qO5z26fsccBFDrD10fZ/skSnVsCvFVqk18V0=; b=dpNmaWMK4SXgA6Vq1ia4Mtl9B61XJFGaAoww3w4CpOyHk2Fn8mQqNVcjfqk/kXMp7d zhwO0ufChPzKim5RHb0tZa5ld6RF8b4PnlGRJdJ4h5OkUoSAzcX1lZoIeDH1ueZN/Loq 3wAkrzmyR7i+JV0fvLZ88NfbwWxGQurRqMHwyZZ8YwhNCutFhTN+FZpmBZIlp0f51MZv rTSMUvwfYtB2Y93wRwqi6RJOXgZpVB5JweOoZO7Hp7PNcIuCgIWZTAVD6NSISRzghAh8 71J7p5aKJX13AJRDpZHZ4G0Uu4ameSSyqd4Kk+WkQFcVecHebi6owGntJ1ko5JSrZsU4 GRhA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=60nuax5qO5z26fsccBFDrD10fZ/skSnVsCvFVqk18V0=; b=nsmkpVQ72AoV8kQK9z3eU4S8VPT3QS52Woa8CNoFcRpHlaTgMmzdc6PP7N7Iw7CWBX WTxhbwUW+qQapqDQlVuOmuLUARPlw7h2n9DZnMRgJAAFZ/KXqkW73++srVcAxpOkFxPQ +ICFYSMpqQ72tI4cY81X5/G7W1sFhcPW5Efom09eoNRKpoVmD0iOrSOlxiz0o0G0sd9F kCRrB1FZSUZACYjwWubyZzfDdIJ/EA9htowKGQr4B9fyAb4I979G7pfrXY+EMCJ6oK/y w8SeH/rIcFKrDcobz3bwWcbHtIKEGRf9OpYy52CSYKJh83PdHFMfrAWtSBFifg2l9eMp 2BCQ== X-Gm-Message-State: AOAM530wb42fW7UBq4oGw4yVGVPRYmxTtetEzbwF2/bo3y691pHoUXz2 b0t0UphmH0aMITVG37r7aB8CE7diOfu9DA== X-Google-Smtp-Source: ABdhPJw2WPQt0uixF5hFFzuCHEbhOsT6HcbKhnRsDY4tOQEZp5ELMASH/Q6/IRIlpnRMbWpfj59Bnw== X-Received: by 2002:a5d:5144:0:b0:207:9a13:ca73 with SMTP id u4-20020a5d5144000000b002079a13ca73mr348951wrt.268.1649879528592; Wed, 13 Apr 2022 12:52:08 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k20-20020a05600c1c9400b0038ecd1ccc17sm3432484wms.35.2022.04.13.12.52.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Apr 2022 12:52:08 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v3 15/29] CI: check ignored unignored build artifacts in "win[+VS] build" too Date: Wed, 13 Apr 2022 21:51:28 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.rc2.843.g193535c2aa7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Use the newly created "ci/check-unignored-build-artifacts.sh" script in the "win build" and "win+VS build" steps too. These steps weren't doing this sanity check of untracked files against .gitignore before, but as the check passes for them let's keep doing that extra sanity check. Signed-off-by: Ævar Arnfjörð Bjarmason --- .github/workflows/main.yml | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index c83f6a21ca3..05c5771e7f6 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -93,6 +93,9 @@ jobs: HOME: ${{runner.workspace}} NO_PERL: 1 run: . /etc/profile && ci/make-test-artifacts.sh artifacts + - run: ci/check-unignored-build-artifacts.sh + if: success() + shell: bash - name: zip up tracked files run: git archive -o artifacts/tracked.tar.gz HEAD - name: upload tracked files and build artifacts @@ -180,6 +183,9 @@ jobs: run: | mkdir -p artifacts && eval "$(make -n artifacts-tar INCLUDE_DLLS_IN_ARTIFACTS=YesPlease ARTIFACTS_DIRECTORY=artifacts NO_GETTEXT=YesPlease 2>&1 | grep ^tar)" + - run: ci/check-unignored-build-artifacts.sh + if: success() + shell: bash - name: zip up tracked files run: git archive -o artifacts/tracked.tar.gz HEAD - name: upload tracked files and build artifacts From patchwork Wed Apr 13 19:51:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12812567 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 81255C4332F for ; Wed, 13 Apr 2022 19:52:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238529AbiDMTy7 (ORCPT ); Wed, 13 Apr 2022 15:54:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58556 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238504AbiDMTye (ORCPT ); Wed, 13 Apr 2022 15:54:34 -0400 Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com [IPv6:2a00:1450:4864:20::330]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3CE0549CAC for ; Wed, 13 Apr 2022 12:52:11 -0700 (PDT) Received: by mail-wm1-x330.google.com with SMTP id q8so1527368wmc.0 for ; Wed, 13 Apr 2022 12:52:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=EBhnjJaPqhCQBjPju8SCWcqr1rrxxCJwBCmPDyZy8J4=; b=mfuClIy2L+SlHqAUvEi/SrvejCbjYoIK5VNWEl1NdvbM6tlWcU/lmJb+kIC49lzsLb wgm7l3aJ6d4jlF3Vw5AE+W0wwVbVevksExSKGt4Hdz2eyPxybhnHabzhqvFaUisRh7ZS g09SpzYbkwLHACo4XjyT0ZHrlfid0pWCai/J+vlcrfNuZ5/76/2gcCHLEu/7CaEJZXYO +yfNVjfKGsznGdR1cCebz1PZk164swNzEw/hD0uj45UjiLp/R44StrbEmijRH4Jer0S9 7gjSuhSslOsuRY+xAf5EGRYYUmX2KHxW98lqnbvyP5lUY2VpzA1KJRnNv1DnqC6clDP2 nFew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=EBhnjJaPqhCQBjPju8SCWcqr1rrxxCJwBCmPDyZy8J4=; b=FiUYx5Svwr8m2KhavOX25ewq//c+3B2KBR4TSTB4O7s0gKXBe40bVT1WZjwJDe1rpJ S3DlLasBlKf6jmzv/2euuKrycpjbWZYseSdMNGhDTCytEuIRNFQIcjlhKkZMD5tdRIN7 /SrvH9pmwE2j8VMV1W+p9MK6Gw+fC3CK/cb7MA8tiU9aCLwkhfmCt/A4Ltapd+3KvUCL rI1dsGhwAsMdhc36ti8AcmJv97AaJYVOx5rWGIIN5uJ2fiwc4Dm/n5ls2Lltnc19qD1G m7lwuzveSM5uFB1ShWlqRM2fSYH/xQF5RfO46fQH6CceNeRDrLsgv614nVHCGfZCVm4Z BGkA== X-Gm-Message-State: AOAM533SzPse5UDLcLVJ/9U0NaKq50xEj0ZbwkL6N1Xpf3qBSngrZzmE O9V29qfLiQ+yn6X6TacHZ0zr4ogTab3qrg== X-Google-Smtp-Source: ABdhPJzkZKGWU36g6kieAu6lK/eTPrLvZlvNUgWIVSEb3WCQQtGV6b00KmIaHb+nyqhwxpldDG7KPQ== X-Received: by 2002:a05:600c:281:b0:38e:bc6e:5971 with SMTP id 1-20020a05600c028100b0038ebc6e5971mr264752wmk.111.1649879529548; Wed, 13 Apr 2022 12:52:09 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k20-20020a05600c1c9400b0038ecd1ccc17sm3432484wms.35.2022.04.13.12.52.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Apr 2022 12:52:08 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v3 16/29] CI: invoke "make artifacts-tar" directly in windows-build Date: Wed, 13 Apr 2022 21:51:29 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.rc2.843.g193535c2aa7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Change the windows-build job to invoke the "make artifacts-tar" step directly, instead of calling a "ci/make-test-artifacts.sh" script. The script was needed because "ci/lib.sh" would set up various environment variables for us, but now we can instead use the "ci/lib.sh" in its script mode. The "mkdir -p" added in b819f1d2cec (ci: parallelize testing on Windows, 2019-01-29) isn't needed, the same commit added that "mkdir -p" to the "artifacts-tar" rule itself, so we can have "make" create the directory for us. This also has the benefit of making the "build" step less chatty, since it won't start with the verbose "set -x" output, that's now contained in the "ci/lib.sh" step. Signed-off-by: Ævar Arnfjörð Bjarmason --- .github/workflows/main.yml | 5 ++++- ci/make-test-artifacts.sh | 10 ---------- 2 files changed, 4 insertions(+), 11 deletions(-) delete mode 100755 ci/make-test-artifacts.sh diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 05c5771e7f6..2c23a19eac2 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -87,12 +87,15 @@ jobs: steps: - uses: actions/checkout@v2 - uses: git-for-windows/setup-git-for-windows-sdk@v1 + - run: ci/lib.sh + shell: bash - name: build shell: bash env: HOME: ${{runner.workspace}} NO_PERL: 1 - run: . /etc/profile && ci/make-test-artifacts.sh artifacts + run: . /etc/profile && make artifacts-tar ARTIFACTS_DIRECTORY=artifacts + if: success() - run: ci/check-unignored-build-artifacts.sh if: success() shell: bash diff --git a/ci/make-test-artifacts.sh b/ci/make-test-artifacts.sh deleted file mode 100755 index 45298562982..00000000000 --- a/ci/make-test-artifacts.sh +++ /dev/null @@ -1,10 +0,0 @@ -#!/bin/sh -# -# Build Git and store artifacts for testing -# - -mkdir -p "$1" # in case ci/lib.sh decides to quit early - -. ${0%/*}/lib.sh - -make artifacts-tar ARTIFACTS_DIRECTORY="$1" From patchwork Wed Apr 13 19:51:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12812569 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9CE78C433EF for ; Wed, 13 Apr 2022 19:52:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238531AbiDMTzB (ORCPT ); Wed, 13 Apr 2022 15:55:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58594 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238489AbiDMTyf (ORCPT ); Wed, 13 Apr 2022 15:54:35 -0400 Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com [IPv6:2a00:1450:4864:20::32d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6A2975F4D0 for ; Wed, 13 Apr 2022 12:52:12 -0700 (PDT) Received: by mail-wm1-x32d.google.com with SMTP id 123-20020a1c1981000000b0038b3616a71aso1830083wmz.4 for ; Wed, 13 Apr 2022 12:52:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=VxPCloD++6LtuowtMXehLXgj0gHRGotIIMkXvAnTFc8=; b=hclXciiqn1BZtTiQ2K9EhR1cxx4KMeYWAbIapBMYzWaJXRrLsayJhoMQjCLtrKRdqk QjA+yrMCm6FWxct8GhlNUdK/rmsn6gQmcaxF3AWyCt8m0w+njYzJ3e900VTolYJIzfiB ChKG1ugqqh+kp4z/2tnGDdGVRfHcuph906Itvz3pCxBdBqmOKYr7mtJQq0q0+7Jd7oWY PVbL0Oxszn/8UN7H/6j+eg8rmiyMprPwdxiEo/uoY6N+ISIrCo+BOnS6WfzayaDjs0v2 52ZwJ9XcWbfXWJdsUNZxUCTVW/v5YpFSTm5YQBoomYhFuPLWD5t/9XDGM2+OqRKO5CuL dtyQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=VxPCloD++6LtuowtMXehLXgj0gHRGotIIMkXvAnTFc8=; b=UVgA2vSGTaNfuTOAQaWJwvHPCS924+APxryZANEF7vHAgdBxEBYDDBWUkJAIOLT5qz s2auaqM5Ur7PclZrCGCN2fQbug2WcUaz2ijI/kVB7dTemNSFgCiK1fz4L5zx7VbJJ3cW I4/+5EcUr5AwcD7UQYeP4EWP0f8fQc4vYPaEcgSunjA9HkWQYN4CXZawgLRtILrvBCsk 0dZPLTzH6YmbbKzQ17UjTz7sjqGdJYMglwimubZLTxXsIgh/Gbg12PDInFTsZ4Xps5kf 7ep7cBQ5jSgGt/Uefd4bjTashqJLMlHlB2Z74/xsqk8i6NUhJOlGoOcltVOXXj9Y3lPr 6K6g== X-Gm-Message-State: AOAM530oNdQKRJmS6rhbvoDHTmKlSaAapRrjqy388yMoaIEmgVmqWiGa AFjhTfQG37v96KeUzuoMss8dfIJ1MOkW5w== X-Google-Smtp-Source: ABdhPJz6bQJERRG2g3qzU2X8K7+DnarNNTm07SJgRQJeuKn+0WdubNjKqublcH5ZGVNyzSK7iB9JJw== X-Received: by 2002:a05:600c:6013:b0:38e:ad43:24d6 with SMTP id az19-20020a05600c601300b0038ead4324d6mr289982wmb.78.1649879530523; Wed, 13 Apr 2022 12:52:10 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k20-20020a05600c1c9400b0038ecd1ccc17sm3432484wms.35.2022.04.13.12.52.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Apr 2022 12:52:10 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v3 17/29] CI: split up and reduce "ci/test-documentation.sh" Date: Wed, 13 Apr 2022 21:51:30 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.rc2.843.g193535c2aa7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Change the "ci/test-documentation.sh" script to run the bash-specific parts in as one command in the CI job itself, and to run the two "make doc" commands at the top-level. It'll now be obvious from the title of the step if if we failed in the asciidoc or asciidoctor step. Since the "check_unignored_build_artifacts()" function is now only used in "ci/check-unignored-build-artifacts.sh" move that function there. The recipe for the job in ".github/workflows/main.yml" is now a bit verbose because it's effectively the same job twice, with a "make clean" in-between. It would be better for the verbosity to run it via a matrix as done in the alternate approach in [1] does, but then we'd sacrifice overall CPU time for the brevity. It might still be worth doing, but let's go for this simpler approach for now. 1. https://lore.kernel.org/git/patch-v2-6.6-7c423c8283d-20211120T030848Z-avarab@gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason --- .github/workflows/main.yml | 16 +++++++++++++- ci/check-unignored-build-artifacts.sh | 10 +++++++++ ci/lib.sh | 10 --------- ci/test-documentation.sh | 31 +++++++++------------------ 4 files changed, 35 insertions(+), 32 deletions(-) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 2c23a19eac2..92b914f16fd 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -365,4 +365,18 @@ jobs: steps: - uses: actions/checkout@v2 - run: ci/install-dependencies.sh - - run: ci/test-documentation.sh + - run: ci/lib.sh + - run: make check-docs + - run: "make doc > >(tee stdout.log) 2> >(tee stderr.raw >&2)" + shell: bash + - run: ci/test-documentation.sh AsciiDoc + if: success() + - run: ci/check-unignored-build-artifacts.sh + if: success() + - run: make clean + - run: "make USE_ASCIIDOCTOR=1 doc > >(tee stdout.log) 2> >(tee stderr.raw >&2)" + shell: bash + - run: ci/test-documentation.sh Asciidoctor + if: success() + - run: ci/check-unignored-build-artifacts.sh + if: success() diff --git a/ci/check-unignored-build-artifacts.sh b/ci/check-unignored-build-artifacts.sh index 56d04b0db9a..0bc04f32804 100755 --- a/ci/check-unignored-build-artifacts.sh +++ b/ci/check-unignored-build-artifacts.sh @@ -5,4 +5,14 @@ . ${0%/*}/lib.sh +check_unignored_build_artifacts () +{ + ! git ls-files --other --exclude-standard --error-unmatch \ + -- ':/*' 2>/dev/null || + { + echo "$(tput setaf 1)error: found unignored build artifacts$(tput sgr0)" + false + } +} + check_unignored_build_artifacts diff --git a/ci/lib.sh b/ci/lib.sh index ffc0e524aaa..35a972fa3f5 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -47,16 +47,6 @@ setenv () { fi } -check_unignored_build_artifacts () -{ - ! git ls-files --other --exclude-standard --error-unmatch \ - -- ':/*' 2>/dev/null || - { - echo "$(tput setaf 1)error: found unignored build artifacts$(tput sgr0)" - false - } -} - # GitHub Action doesn't set TERM, which is required by tput setenv TERM ${TERM:-dumb} diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh index 6b5cce03bd7..b8a6a6f664e 100755 --- a/ci/test-documentation.sh +++ b/ci/test-documentation.sh @@ -1,10 +1,12 @@ -#!/usr/bin/env bash +#!/bin/sh # -# Perform sanity checks on documentation and build it. +# Perform sanity checks on "make doc" output and built documentation # . ${0%/*}/lib.sh +generator=$1 + filter_log () { sed -e '/^GIT_VERSION = /d' \ -e "/constant Gem::ConfigMap is deprecated/d" \ @@ -14,28 +16,15 @@ filter_log () { "$1" } -make check-docs - -# Build docs with AsciiDoc -make doc > >(tee stdout.log) 2> >(tee stderr.raw >&2) -cat stderr.raw -filter_log stderr.raw >stderr.log -test ! -s stderr.log -test -s Documentation/git.html -test -s Documentation/git.xml -test -s Documentation/git.1 -grep ' X-Patchwork-Id: 12812570 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E98CDC433EF for ; Wed, 13 Apr 2022 19:52:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237427AbiDMTzE (ORCPT ); Wed, 13 Apr 2022 15:55:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58548 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238409AbiDMTyh (ORCPT ); Wed, 13 Apr 2022 15:54:37 -0400 Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com [IPv6:2a00:1450:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AE78B7628B for ; Wed, 13 Apr 2022 12:52:13 -0700 (PDT) Received: by mail-wr1-x433.google.com with SMTP id q3so3499643wrj.7 for ; Wed, 13 Apr 2022 12:52:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=fphnWiFu3yAnTqygU3drfwoWeRkHow1VfNgzONBkwyk=; b=Ki9wVCWrk/uclrldHu+xWcZzlHrsup5lJVu8qSngtYnx2CTsncnQBQsdCpcwC4tkB4 B6TqswIYpjHuH7btLvsfLz1AT5AaHKXuUXGmF6YGnYPI6RSKZxSjtCu0qL+l8ewtIJoP yhI8/VS1hAChK9prDb8O6O9cTu+XMnr88m7R7hQuniaBMIAtg62Q5fZ1YdfKrKZznai9 MPLChll4kFpprZc0W43c1e/w+aFbuuT7CvGhNdOTNks9F0s0aklso/52E8feFkbadV6X 5XBfJGGl6gWOwyJrvPC+nwVJEyGqRDnz4ddwYT2TWqnU/rx5Alquf6mBnHRmNR/fCELC TG4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fphnWiFu3yAnTqygU3drfwoWeRkHow1VfNgzONBkwyk=; b=ylFrgtFDHQyHoN2eXDeuOo5WABx9Xck7Tlzt+R3sk9frsAgELRO04H/5JGk3fJzDFc 3NFonPhh+50MGFwCJ3NYPTlyannkiD3BXtHDL8SzKJeTAcJ/bNLBaxWX/sGsUbKhRh9I 8SMm9uuiPpdQBustVQ3zP5euXcVrTjKEYwQfJmg0s44y+U+u+ZlFeCP43uY1Z7Bp0bwW k93WYFjFCTxU8dZeUlq1xD6F7q3SpDc61PjJ9zkPZWpbqKu3PBpWy9BoSuIvdKiF0W9E x5cvCin6ljcFwy+4FasV5Ck/4ZiQW2LsA+2QYPPKhn82t0pLErxJrnCH5f6GHgH7QYzZ p4lw== X-Gm-Message-State: AOAM531ER1VqoR0FAt47B396hxeIlPS2uUflY4GKExOM19/zWB/a+VUX jOmPDib/4JJocEhwcg4LE1RInKn36FA0sQ== X-Google-Smtp-Source: ABdhPJynZ0+ahaQOyZifnBNBSuzt1YhOGXDh5Bw8CFKGwm3/XfwWzkInIZW9iJNgR66mTgbbMeK1KA== X-Received: by 2002:adf:9bd5:0:b0:207:a2a6:636a with SMTP id e21-20020adf9bd5000000b00207a2a6636amr343770wrc.480.1649879531713; Wed, 13 Apr 2022 12:52:11 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k20-20020a05600c1c9400b0038ecd1ccc17sm3432484wms.35.2022.04.13.12.52.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Apr 2022 12:52:11 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v3 18/29] CI: combine ci/install{,-docker}-dependencies.sh Date: Wed, 13 Apr 2022 21:51:31 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.rc2.843.g193535c2aa7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Combine the "ci/install-dependencies.sh" and "ci/install-docker-dependencies.sh" into one script. The "case" statement in the latter only cared about "$jobname", and can be folded into the same "case" statement in the former. The reason they split up is historical, and because the "ci/install-dependencies.sh" used "ci/lib.sh", which requires "bash". At least one of the docker containers doesn't have "bash". To make the existing code POSIX-compatible we need to replace pushd/popd with a sub-shell, but no other changes were needed. This also fixes issue in "ci/install-docker-dependencies.sh" where we'd hide errors due to not using "set -e". Now that we include "ci/lib.sh" we won't have that potential issue. Signed-off-by: Ævar Arnfjörð Bjarmason --- .github/workflows/main.yml | 2 +- ci/install-dependencies.sh | 27 ++++++++++++++++++++++----- ci/install-docker-dependencies.sh | 22 ---------------------- 3 files changed, 23 insertions(+), 28 deletions(-) delete mode 100755 ci/install-docker-dependencies.sh diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 92b914f16fd..3fd3e4f873c 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -311,7 +311,7 @@ jobs: container: ${{matrix.vector.image}} steps: - uses: actions/checkout@v1 - - run: ci/install-docker-dependencies.sh + - run: ci/install-dependencies.sh - run: ci/lib.sh - run: make - run: make test diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh index 01006379fd0..9388289d0ed 100755 --- a/ci/install-dependencies.sh +++ b/ci/install-dependencies.sh @@ -1,4 +1,4 @@ -#!/usr/bin/env bash +#!/bin/sh # # Install dependencies required to build and test Git on Linux and macOS # @@ -34,18 +34,20 @@ ubuntu-latest) sudo apt-get -q -y install language-pack-is libsvn-perl apache2 \ $UBUNTU_COMMON_PKGS $CC_PACKAGE mkdir --parents "$P4_PATH" - pushd "$P4_PATH" + ( + cd "$P4_PATH" wget --quiet "$P4WHENCE/bin.linux26x86_64/p4d" wget --quiet "$P4WHENCE/bin.linux26x86_64/p4" chmod u+x p4d chmod u+x p4 - popd + ) mkdir --parents "$GIT_LFS_PATH" - pushd "$GIT_LFS_PATH" + ( + cd "$GIT_LFS_PATH" wget --quiet "$LFSWHENCE/git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" tar --extract --gunzip --file "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" cp git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs . - popd + ) ;; macos-latest) export HOMEBREW_NO_AUTO_UPDATE=1 HOMEBREW_NO_INSTALL_CLEANUP=1 @@ -90,6 +92,21 @@ linux-gcc-default) sudo apt-get -q update sudo apt-get -q -y install $UBUNTU_COMMON_PKGS ;; +linux32) + linux32 --32bit i386 sh -c ' + apt update >/dev/null && + apt install -y build-essential libcurl4-openssl-dev \ + libssl-dev libexpat-dev gettext python >/dev/null + ' + ;; +linux-musl) + apk add --update build-base curl-dev openssl-dev expat-dev gettext \ + pcre2-dev python3 musl-libintl perl-utils ncurses >/dev/null + ;; +pedantic) + dnf -yq update >/dev/null && + dnf -yq install make gcc findutils diffutils perl python3 gettext zlib-devel expat-devel openssl-devel curl-devel pcre2-devel >/dev/null + ;; esac if type p4d >/dev/null && type p4 >/dev/null diff --git a/ci/install-docker-dependencies.sh b/ci/install-docker-dependencies.sh deleted file mode 100755 index 78b7e326da6..00000000000 --- a/ci/install-docker-dependencies.sh +++ /dev/null @@ -1,22 +0,0 @@ -#!/bin/sh -# -# Install dependencies required to build and test Git inside container -# - -case "$jobname" in -linux32) - linux32 --32bit i386 sh -c ' - apt update >/dev/null && - apt install -y build-essential libcurl4-openssl-dev \ - libssl-dev libexpat-dev gettext python >/dev/null - ' - ;; -linux-musl) - apk add --update build-base curl-dev openssl-dev expat-dev gettext \ - pcre2-dev python3 musl-libintl perl-utils ncurses >/dev/null - ;; -pedantic) - dnf -yq update >/dev/null && - dnf -yq install make gcc findutils diffutils perl python3 gettext zlib-devel expat-devel openssl-devel curl-devel pcre2-devel >/dev/null - ;; -esac From patchwork Wed Apr 13 19:51:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12812571 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B133DC433F5 for ; Wed, 13 Apr 2022 19:52:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238534AbiDMTzF (ORCPT ); Wed, 13 Apr 2022 15:55:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58554 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238519AbiDMTyi (ORCPT ); Wed, 13 Apr 2022 15:54:38 -0400 Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C128976299 for ; Wed, 13 Apr 2022 12:52:14 -0700 (PDT) Received: by mail-wm1-x32c.google.com with SMTP id x3so1745490wmj.5 for ; Wed, 13 Apr 2022 12:52:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=e6Xr85s3oTXiyDfFmVol9APEkxHmQBUqETlBj+rzYRc=; b=echgYLPv3m+S7jCigML1W+ymMywtrmDdwFNjAXL4cde+CxVpuWbYQhdV1+BcIpofPL Lja6eppW/TTbNwhAFa5TBZjIdo7jIXOEpZyAMlnQsiMj0hT7nXMPJh01rvMlwr2KrOV2 K7xuxubYy5hwk1J2zhSelEehmmmF6or7xQX8MU/p9Hk+bxr5SWZamVrj0M2U8tcd8qmI zHdM/0kDKbnjebTDqsKWXLC7thetaiCcGjA7TIOPCbXBXQWbfbLA0cCseKHWlahJAF8E k/5MpHntje6RuWs9ySZp+MhWRWvlqdYgodQ6KZnjbewi3p4moyi6TtXCSDSd1sHD9Rbq qwDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=e6Xr85s3oTXiyDfFmVol9APEkxHmQBUqETlBj+rzYRc=; b=2wDUffV4rbrqWyJyLhfCbDVbQ+/IAuBfuzQtnL9o9AzHKXfEW0ekunYhkUu9dR4mR+ GyYMxuPgFM/DpFwUOnnMKtkQuUNncGxt8GZo/xnIIVMXGVcrFfQ1JsloNIKzAM5l5NIt XbGu3u+tiql9jfHxTc+OOD6qQ5H5RMa0uTg1u0bxr7GTUQZKU7czqGY5lmD8p0gd/P2m dSRQPn3X8eSj+C4kGjIeVLJawPiJG0gY8oE2I5S1GJHX3aAg1WDEquCZOowthhKmQpJH uvZCknc6Gin9CFLOjqcogBLVe6FLAyuhLm8PfzjgBETwKWIxZdgnkit92Xn57PGpd9U7 LATQ== X-Gm-Message-State: AOAM5302NhCQ1oVp5CMzDrcL82hP5pVs/THCUae5LlxlgXhLwrcCp3xN yGrS8TjP92S2XB41NcMaOejP34T6smi7aA== X-Google-Smtp-Source: ABdhPJxnNPapUNtMQD5b/VKTovoCw5Vuwr+TdK3pE4l+LZUHz4mpYtRM3evWVa2UATg1Y7Vp0VRSdA== X-Received: by 2002:a05:600c:3789:b0:38c:bd93:77d6 with SMTP id o9-20020a05600c378900b0038cbd9377d6mr307691wmr.12.1649879532843; Wed, 13 Apr 2022 12:52:12 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k20-20020a05600c1c9400b0038ecd1ccc17sm3432484wms.35.2022.04.13.12.52.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Apr 2022 12:52:12 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v3 19/29] CI: move "env" definitions into ci/lib.sh Date: Wed, 13 Apr 2022 21:51:32 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.rc2.843.g193535c2aa7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Have the GitHub CI jobs use the "ci/lib".sh as a source of truth for environment variables that affect their builds and tests. This brings them in line with the rest of the jobs, and make it easier to see the entire CI configuration per jobname at a glance. To do this we need to add a ci/lib.sh "step" to the remaining jobs that didn't have it added in preceding commits. The Makefile parameters are luckily all accepted via the environment, so we can export these instead of adding them to MAKEFLAGS. Let's also use the documented GIT_CONIFG_* mechanism for setting config instead of the internal GIT_CONFIG_PARAMETERS variable. This adjusts code added in 889cacb6897 (ci: configure GitHub Actions for CI/PR, 2020-04-11), we could probably use the GIT_AUTHOR_NAME etc, but let's keep setting this via config, just with the documented mechanism. By setting "ARTIFACTS_DIRECTORY=artifacts" once we don't need to repeat it in various places, let's get it from the environment instead. Finally, the "DEVELOPER: 1" was only needed by the "sparse" job, which wasn't using "ci/lib.sh" (and which sets DEVELOPER=1). Let's instead have the "sparse" job use the "ci/lib.sh" and remove DEVELOPER=1 from ".github/workflows/main.yml". This substantially speeds up the "sparse" job, since it'll now pick up the "--jobs" setting in MAKEFLAGS that we use everywhere else. Before it ran in around 4m30s, now in around 2m30s. Signed-off-by: Ævar Arnfjörð Bjarmason --- .github/workflows/main.yml | 29 +++++++++++------------------ ci/lib.sh | 21 +++++++++++++++++++++ 2 files changed, 32 insertions(+), 18 deletions(-) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 3fd3e4f873c..fe37e0f1b36 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -2,9 +2,6 @@ name: CI on: [push, pull_request] -env: - DEVELOPER: 1 - jobs: ci-config: name: config @@ -93,8 +90,7 @@ jobs: shell: bash env: HOME: ${{runner.workspace}} - NO_PERL: 1 - run: . /etc/profile && make artifacts-tar ARTIFACTS_DIRECTORY=artifacts + run: . /etc/profile && make artifacts-tar if: success() - run: ci/check-unignored-build-artifacts.sh if: success() @@ -105,7 +101,7 @@ jobs: uses: actions/upload-artifact@v2 with: name: windows-artifacts - path: artifacts + path: ${{env.ARTIFACTS_DIRECTORY}} windows-test: name: win test env: @@ -126,6 +122,8 @@ jobs: shell: bash run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz - uses: git-for-windows/setup-git-for-windows-sdk@v1 + - run: ci/lib.sh + shell: bash - name: test shell: bash run: . /etc/profile && ci/run-test-slice.sh ${{matrix.nr}} 10 @@ -145,9 +143,6 @@ jobs: jobname: vs-build needs: ci-config if: needs.ci-config.outputs.enabled == 'yes' - env: - NO_PERL: 1 - GIT_CONFIG_PARAMETERS: "'user.name=CI' 'user.email=ci@git'" runs-on: windows-latest steps: - uses: actions/checkout@v2 @@ -171,6 +166,8 @@ jobs: - name: copy dlls to root shell: cmd run: compat\vcbuild\vcpkg_copy_dlls.bat release + - run: ci/lib.sh + shell: bash - name: generate Visual Studio solution shell: bash run: | @@ -181,21 +178,16 @@ jobs: - name: bundle artifact tar shell: bash env: - MSVC: 1 VCPKG_ROOT: ${{github.workspace}}\compat\vcbuild\vcpkg run: | - mkdir -p artifacts && - eval "$(make -n artifacts-tar INCLUDE_DLLS_IN_ARTIFACTS=YesPlease ARTIFACTS_DIRECTORY=artifacts NO_GETTEXT=YesPlease 2>&1 | grep ^tar)" - - run: ci/check-unignored-build-artifacts.sh - if: success() - shell: bash + eval "$(make -n artifacts-tar 2>&1 | grep -e ^mkdir -e ^tar)" - name: zip up tracked files run: git archive -o artifacts/tracked.tar.gz HEAD - name: upload tracked files and build artifacts uses: actions/upload-artifact@v2 with: name: vs-artifacts - path: artifacts + path: ${{env.ARTIFACTS_DIRECTORY}} vs-test: name: win+VS test env: @@ -216,10 +208,10 @@ jobs: - name: extract tracked files and build artifacts shell: bash run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz + - run: ci/lib.sh + shell: bash - name: test shell: bash - env: - NO_SVN_TESTS: 1 run: . /etc/profile && ci/run-test-slice.sh ${{matrix.nr}} 10 - name: ci/print-test-failures.sh if: failure() @@ -354,6 +346,7 @@ jobs: - uses: actions/checkout@v2 - name: Install other dependencies run: ci/install-dependencies.sh + - run: ci/lib.sh - run: make sparse documentation: name: documentation diff --git a/ci/lib.sh b/ci/lib.sh index 35a972fa3f5..8c574b21a67 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -104,6 +104,27 @@ macos-latest) esac case "$jobname" in +windows-build) + setenv --build NO_PERL NoThanks + setenv --build ARTIFACTS_DIRECTORY artifacts + ;; +vs-build) + setenv --build NO_PERL NoThanks + setenv --build NO_GETTEXT NoThanks + setenv --build ARTIFACTS_DIRECTORY artifacts + setenv --build INCLUDE_DLLS_IN_ARTIFACTS YesPlease + setenv --build MSVC YesPlease + + setenv --build GIT_CONFIG_COUNT 2 + setenv --build GIT_CONFIG_KEY_0 user.name + setenv --build GIT_CONFIG_VALUE_0 CI + setenv --build GIT_CONFIG_KEY_1 user.emailname + setenv --build GIT_CONFIG_VALUE_1 ci@git + setenv --build GIT_CONFIG_VALUE_1 ci@git + ;; +vs-test) + setenv --test NO_SVN_TESTS YesPlease + ;; linux-gcc) setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main ;; From patchwork Wed Apr 13 19:51:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12812572 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B0155C433FE for ; Wed, 13 Apr 2022 19:52:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237413AbiDMTzH (ORCPT ); Wed, 13 Apr 2022 15:55:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58548 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238512AbiDMTyj (ORCPT ); Wed, 13 Apr 2022 15:54:39 -0400 Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 95CBE78912 for ; Wed, 13 Apr 2022 12:52:15 -0700 (PDT) Received: by mail-wm1-x334.google.com with SMTP id l9-20020a05600c4f0900b0038ccd1b8642so3517655wmq.0 for ; Wed, 13 Apr 2022 12:52:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=1IwY17bPgBCm5VQCcU1j6qM5TL4VUWFJdyOP9C2eNEk=; b=OV5KhclmXaeYf1Ypdp6V0MAZ/FCOeHWx9GXVltqR1pPx4vHAXCuzwvH0tfYnDNtnUB DOVjaTOB5z4YCoEu61GkKQbQK7xP6Vop0qHV3WkEH9iDwNXgkM0qnHxRBu37dGzMrRvm YpPqbS6h3smLywdsBq/c7J7a0frTtjwOONV9O2PdZMjmSMP+qyqtAGDitB4os60aNl/p biczNO3FZk7EPPA/GWFX2CH5jxpjqaAequKKBqnrFsN5yQRqEOiwhJoY/rmG981/oR9J TfttMji/WfFobc8Y/lXUPl5ZOU+lC+ojVjX8cgdmBarhc2YuMJJPpQ+4oOQ45ZatyKvF Pakw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=1IwY17bPgBCm5VQCcU1j6qM5TL4VUWFJdyOP9C2eNEk=; b=eeqly11kCUkAhfbwYBF4z4sQeSSM+QVcQTDUjJxJu0mejZeLtePiloQ/9XcV7fKzXC JNuf4IQ+TOpWE7CYrAeDEtsSkWyf7V966BYCOZscvd5hm9Tg7YZh4hRxJ6oOfQpOgfGS gMtlSU3WRDWH2V+vocKAMl4FCPxlqoH3WGRRPl9hq56k08OUHWLKbJq1CpS1/G8O7Q41 iPiH57vSvqR3dQ5e6KeBHQQ6e9ISh9Q3zgu6ykOPnuD50/8WAqKRY2HCvAn0hAnIZim8 HtIBUX2Nzw3IwZ8CTULi8I3WbdxLQTt+70501Z1QwWxzLCtRtvRYyE8llVoCURcsPC4k gmPQ== X-Gm-Message-State: AOAM5315vLmGCmOF/DPwjU/6b0AxdZ60rCi1vGIRITPcCKSNlH3xO1Cj o8EX5aQsVvs9MmIkQSSpXZH7lyfxTkOYZw== X-Google-Smtp-Source: ABdhPJwlDJWnPoXw4OpXSF67bFRU4Hoo49dy8YM4tBWLlYT3UHo/npqZVkBEEz1bizqF2Z4UL/whZA== X-Received: by 2002:a1c:7416:0:b0:38e:b8b7:e271 with SMTP id p22-20020a1c7416000000b0038eb8b7e271mr271753wmc.7.1649879533874; Wed, 13 Apr 2022 12:52:13 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k20-20020a05600c1c9400b0038ecd1ccc17sm3432484wms.35.2022.04.13.12.52.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Apr 2022 12:52:13 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v3 20/29] ci/run-test-slice.sh: replace shelling out with "echo" Date: Wed, 13 Apr 2022 21:51:33 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.rc2.843.g193535c2aa7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Amend this code added in b819f1d2cec (ci: parallelize testing on Windows, 2019-01-29) to stop shelling out to "tr", all we're trying to do here is to turn '\n'-delimited output into a ' '-delimited list. To do that we can just use "echo". None of these files will contain the odd characters using "echo" might choke on in the general case (i.e. '\'-escapes and the like). This has the added benefit of not adding a redundant ' ' to the end of the T-parameter, which didn't matter in practice, but as we'll see in a subsequent commit makes for slightly nicer output, as we'll show this variable using a GitHub CI-native feature. Signed-off-by: Ævar Arnfjörð Bjarmason --- ci/run-test-slice.sh | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/ci/run-test-slice.sh b/ci/run-test-slice.sh index 70326961454..1de6a18ca47 100755 --- a/ci/run-test-slice.sh +++ b/ci/run-test-slice.sh @@ -5,6 +5,6 @@ . ${0%/*}/lib.sh -make --quiet -C t T="$(cd t && - ./helper/test-tool path-utils slice-tests "$1" "$2" t[0-9]*.sh | - tr '\n' ' ')" +tests=$(echo $(cd t && ./helper/test-tool path-utils slice-tests "$1" "$2" \ + t[0-9]*.sh)) +make --quiet -C t T="$tests" From patchwork Wed Apr 13 19:51:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12812573 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B71EAC433F5 for ; Wed, 13 Apr 2022 19:52:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238536AbiDMTzP (ORCPT ); Wed, 13 Apr 2022 15:55:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58804 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238515AbiDMTyk (ORCPT ); Wed, 13 Apr 2022 15:54:40 -0400 Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DC3A350E1A for ; Wed, 13 Apr 2022 12:52:16 -0700 (PDT) Received: by mail-wr1-x42c.google.com with SMTP id c10so4157371wrb.1 for ; Wed, 13 Apr 2022 12:52:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=GGxnsA1f0mYbf6/nSP5mRjqeCTrc3gj+EPjqTur0rVQ=; b=dA3NIcogDLsZ1kL60oNrK7juuwk6/MYpS6fyMHQmIkc8fAwhSnp5ofNl05C32kjbbg nJ88qbznxJVRYBS7FqVajORINcT6vefhHHtgqgNYEwlwUjbArH2Bq5ohfyluzZUUB8lN /f2IKaRjy/i85dT9sB8reWCgar9y03GOGvZun21ECjZckVivA8I7Q6ZIYo8ObJ+BPfxz KLrtz/kAl3u9D5oPLQpaGBRHmorvUEfaNhxJ+LUMBK23F73bcCBXQrSlpLmkp/LExYgY 9anPbrqxd2pQgEnXciSqKxTeZgblFeELgastXEzaM1GAemSWoTEley3DbSszM1dO5+rD utHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=GGxnsA1f0mYbf6/nSP5mRjqeCTrc3gj+EPjqTur0rVQ=; b=kuYpdlXvS+QPyAhyVENFH3FJ07v7MJMzWp4tw833YTXX6Pv7tFeKwEJdMXKX9S4Thk 4PC0WuR9EP9zUpkEo0n2xZADpXV168vVx6zzFEg2atmzE1QJfSAlIjAN+lVSSxe8RfaS N5069SCmle1Ww16IEsgzwFwzjFU8A1b20HnLP1KKizY6WkSQmJ1vR7uQznBK2W4+CPMw Mjs5gysT89pfyJJcPUbK3QWVmDkvo/1b3ticzWcF0n8ptFZiZWKel21VkEmwIbNhMSI9 N/VhCHZ0ZjQryeXbeUuydsEbRL366YRor8n1+TINNJo+cMAkWfclWXL0X/RI5f6d7Rvi w/vw== X-Gm-Message-State: AOAM532VWC+A7DO6K7mfE3s7J+/5qhl0XDDVbm76dC3gGw+VXV//xlUY S2Kc6r6WSqdW1BJvBflOCcyXlTrqsSm/cQ== X-Google-Smtp-Source: ABdhPJzqDyvEdjP375xVEE7DZlbAXN1K8mylIxc+lDOLDbQF2M0owk4Wst3ZzSZWYNReS2CE4dvJlQ== X-Received: by 2002:adf:cd87:0:b0:207:b0ad:6d8 with SMTP id q7-20020adfcd87000000b00207b0ad06d8mr331905wrj.111.1649879534847; Wed, 13 Apr 2022 12:52:14 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k20-20020a05600c1c9400b0038ecd1ccc17sm3432484wms.35.2022.04.13.12.52.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Apr 2022 12:52:14 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v3 21/29] CI: pre-select test slice in Windows & VS tests Date: Wed, 13 Apr 2022 21:51:34 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.rc2.843.g193535c2aa7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In preceding commits the tests have been changed to do their setup via $GITHUB_ENV in one step, and to then have subsequent steps that re-use that environment. Let's change the "test slice" tests added in b819f1d2cec (ci: parallelize testing on Windows, 2019-01-29) to do the same. These tests select 10% of the tests to run in 10 "test slices". Now we'll select those in a step that immediately precedes the testing step, and then simply invoke "make -C t -e". This has the advantage that the tests to be run are now listed in the standard "Run" drop-down at the start of the "test" step. Since the "T" variable in "t/Makefile" doesn't normally accept overrides from the environment we need to invoke "make" with the "-e" option (a.k.a. "--environment-overrides"). We could also make $(T) in t/Makefile be a "?=" assigned variable, but this way works, and is arguably clearer as it's more obvious that we're injecting a special list of tests that override the normal behavior of that Makefile. Note that we cannot run the top-level "make test" here, because of how the Windows CI builds git, i.e. either via CMake or some option that would cause "make test" to recompile git itself. Instead we run "make -C t [...]". Signed-off-by: Ævar Arnfjörð Bjarmason --- .github/workflows/main.yml | 10 ++++++++-- ci/{run-test-slice.sh => select-test-slice.sh} | 4 ++-- 2 files changed, 10 insertions(+), 4 deletions(-) rename ci/{run-test-slice.sh => select-test-slice.sh} (57%) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index fe37e0f1b36..1e13718b989 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -124,9 +124,12 @@ jobs: - uses: git-for-windows/setup-git-for-windows-sdk@v1 - run: ci/lib.sh shell: bash + - name: select tests + run: . /etc/profile && ci/select-test-slice.sh ${{matrix.nr}} 10 + shell: bash - name: test shell: bash - run: . /etc/profile && ci/run-test-slice.sh ${{matrix.nr}} 10 + run: . /etc/profile && make -C t -e - name: ci/print-test-failures.sh if: failure() shell: bash @@ -210,9 +213,12 @@ jobs: run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz - run: ci/lib.sh shell: bash + - name: select tests + run: . /etc/profile && ci/select-test-slice.sh ${{matrix.nr}} 10 + shell: bash - name: test shell: bash - run: . /etc/profile && ci/run-test-slice.sh ${{matrix.nr}} 10 + run: . /etc/profile && make -C t -e - name: ci/print-test-failures.sh if: failure() shell: bash diff --git a/ci/run-test-slice.sh b/ci/select-test-slice.sh similarity index 57% rename from ci/run-test-slice.sh rename to ci/select-test-slice.sh index 1de6a18ca47..ec602f8a053 100755 --- a/ci/run-test-slice.sh +++ b/ci/select-test-slice.sh @@ -1,10 +1,10 @@ #!/bin/sh # -# Test Git in parallel +# Select a portion of the tests for testing Git in parallel # . ${0%/*}/lib.sh tests=$(echo $(cd t && ./helper/test-tool path-utils slice-tests "$1" "$2" \ t[0-9]*.sh)) -make --quiet -C t T="$tests" +echo T="$tests" >>$GITHUB_ENV From patchwork Wed Apr 13 19:51:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12812574 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AF6C1C433EF for ; Wed, 13 Apr 2022 19:52:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238587AbiDMTzQ (ORCPT ); Wed, 13 Apr 2022 15:55:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59986 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238477AbiDMTzO (ORCPT ); Wed, 13 Apr 2022 15:55:14 -0400 Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com [IPv6:2a00:1450:4864:20::32a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 96C2E68330 for ; Wed, 13 Apr 2022 12:52:17 -0700 (PDT) Received: by mail-wm1-x32a.google.com with SMTP id y21so274100wmi.2 for ; Wed, 13 Apr 2022 12:52:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Uw05yWeJO3a9e/mmdRQ6J67KpoPLuci6H9mjTcaZxy0=; b=DgqRKxi/Gn4DefT6Mf1ak9Hh+0L+7ridu0ObCtiCOwxWDZeQ5DOu6z54OXx5ra2Qfi CehewOAwqsNNcY36HdQbsZNheENpLfOdFsVVrvcDjz1VUFprHrGp3Le7/XqIg+ucmkVm RVa/5nHNorS+vKLr2jNqoopcVDJwzXfwEfg4cRs67iyD5u+nTr2bciqLlLiheX4WiXwZ ZEKJPqF1BaCyp77Fqa3t7mWDA6ZCY+1YCWTxcncjbMAhKsZctYLKGi7+Y23F7oJ1wmEH b1Ul3XGz264sEFCzSgVCeVJ45pkgUKZbseX7nd++++LcUocG1roNP2ck1e3y2pCTOUAW irIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Uw05yWeJO3a9e/mmdRQ6J67KpoPLuci6H9mjTcaZxy0=; b=knDPq6g9B/Ot+1m8E8QINTfut/n0GXQXR3gI7wbrW4y7LstkAjbZZthlx5YyM0YlM0 taCm8/9K+Vs5rNlF/fNAqk6XkjmfsI+Tmmon+ur0+OlkDEhrB31wG7lOx44IG9woO76v WCEz4l3VH82XM/wgoLL1Nn2Ec6lWC2nFDEZUkeB9DyMpRumcxVs9Govs3HQrErg00LJU ok3nmJruI2TSFgNkAmgy2OuOfiFAZ7HDQRzMpnCM0IwMOVbIrrj/kVAJ7+2RGhHpOIeF /D6qzdaSnZssythJcpsN9F5ddef3xP2pNZCYjnBw4+hBrQMd8IdPgzUJSodZUKVSqBWm obQQ== X-Gm-Message-State: AOAM530jah7Stk2doKTmwdjj/7un34moiz9mM3K9YLp8IdXr33nM1Mui A3v/61/Vo5z/D65rsa/HH6k6yFNmrW7XGg== X-Google-Smtp-Source: ABdhPJxnekn0oSFHfy5o/F98pa3FY3TL5DxEzJIZzDUwFTtYGwSXuXRBN84PLhfPQYNBXj1+NLTqpA== X-Received: by 2002:a7b:c041:0:b0:38c:7c21:8ade with SMTP id u1-20020a7bc041000000b0038c7c218ademr263504wmc.163.1649879535750; Wed, 13 Apr 2022 12:52:15 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k20-20020a05600c1c9400b0038ecd1ccc17sm3432484wms.35.2022.04.13.12.52.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Apr 2022 12:52:15 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v3 22/29] CI: only invoke ci/lib.sh as "steps" in main.yml Date: Wed, 13 Apr 2022 21:51:35 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.rc2.843.g193535c2aa7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Change the scripts in ci/ to stop using ci/lib.sh as a library, now that the only thing it did for them was to "set -ex" and possibly set TERM=dumb. Let's create a ci/lib-tput.sh for those that need to use "tput" instead, and have these scripts invoke "set -ex" themselves. This makes their invocation a lot less verbose, since they'll be relying on an earlier step in the CI job to have set the variables in $GITHUB_ENV, and won't be spewing their own trace output to set those variables again. Let's also create a ci/lib-ci-type.sh, and have ci/lib.sh and ci/print-test-failures.sh share the logic to discover the CI type. We could have set the CI_TYPE in the environment with "setenv", but let's avoid that verbosity for this purely internal variable. The "ci/lib.sh" is now no longer a "Library of functions shared by all CI scripts", so let's remove that commentary, and the misleading comment about "set -ex" being for "installing dependencies", we're now no longer using it in "ci/install-dependencies.sh" (but it does its own "set -ex"). Signed-off-by: Ævar Arnfjörð Bjarmason --- ci/check-unignored-build-artifacts.sh | 4 +++- ci/install-dependencies.sh | 2 +- ci/lib-ci-type.sh | 9 +++++++++ ci/lib-tput.sh | 5 +++++ ci/lib.sh | 27 ++++++++++----------------- ci/print-test-failures.sh | 6 +++--- ci/select-test-slice.sh | 2 +- ci/test-documentation.sh | 2 +- 8 files changed, 33 insertions(+), 24 deletions(-) create mode 100644 ci/lib-ci-type.sh create mode 100644 ci/lib-tput.sh diff --git a/ci/check-unignored-build-artifacts.sh b/ci/check-unignored-build-artifacts.sh index 0bc04f32804..c27d6a97f45 100755 --- a/ci/check-unignored-build-artifacts.sh +++ b/ci/check-unignored-build-artifacts.sh @@ -3,7 +3,9 @@ # Check whether the build created anything not in our .gitignore # -. ${0%/*}/lib.sh +set -ex + +. ${0%/*}/lib-tput.sh check_unignored_build_artifacts () { diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh index 9388289d0ed..ee9af62fc92 100755 --- a/ci/install-dependencies.sh +++ b/ci/install-dependencies.sh @@ -3,7 +3,7 @@ # Install dependencies required to build and test Git on Linux and macOS # -. ${0%/*}/lib.sh +set -ex UBUNTU_COMMON_PKGS="make libssl-dev libcurl4-openssl-dev libexpat-dev tcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl diff --git a/ci/lib-ci-type.sh b/ci/lib-ci-type.sh new file mode 100644 index 00000000000..6f01fd9e5d9 --- /dev/null +++ b/ci/lib-ci-type.sh @@ -0,0 +1,9 @@ +#!/bin/sh + +if test "$GITHUB_ACTIONS" = "true" +then + CI_TYPE=github-actions +else + echo "Could not identify CI type" >&2 + exit 1 +fi diff --git a/ci/lib-tput.sh b/ci/lib-tput.sh new file mode 100644 index 00000000000..b62ef13c44c --- /dev/null +++ b/ci/lib-tput.sh @@ -0,0 +1,5 @@ +#!/bin/sh + +# GitHub Action doesn't set TERM, which is required by tput +TERM=${TERM:-dumb} +export TERM diff --git a/ci/lib.sh b/ci/lib.sh index 8c574b21a67..8f7e178f3ff 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -1,13 +1,9 @@ #!/bin/sh - -# Library of functions shared by all CI scripts - -# Set 'exit on error' for all CI scripts to let the caller know that -# something went wrong. -# Set tracing executed commands, primarily setting environment variables -# and installing dependencies. set -ex +# Helper libraries +. ${0%/*}/lib-ci-type.sh + # Starting assertions if test -z "$jobname" then @@ -47,18 +43,14 @@ setenv () { fi } -# GitHub Action doesn't set TERM, which is required by tput -setenv TERM ${TERM:-dumb} - # How many jobs to run in parallel? NPROC=10 # Clear MAKEFLAGS that may come from the outside world. MAKEFLAGS=--jobs=$NPROC -if test "$GITHUB_ACTIONS" = "true" -then - CI_TYPE=github-actions +case "$CI_TYPE" in +github-actions) CC="${CC:-gcc}" setenv --test GIT_PROVE_OPTS "--timer --jobs $NPROC" @@ -66,11 +58,12 @@ then test Windows != "$RUNNER_OS" || GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS" setenv --test GIT_TEST_OPTS "$GIT_TEST_OPTS" -else - echo "Could not identify CI type" >&2 - env >&2 + ;; +*) + echo "Unhandled CI type: $CI_TYPE" >&2 exit 1 -fi + ;; +esac setenv --build DEVELOPER 1 setenv --test DEFAULT_TEST_TARGET prove diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh index a878cca67e4..2405f65650a 100755 --- a/ci/print-test-failures.sh +++ b/ci/print-test-failures.sh @@ -3,10 +3,10 @@ # Print output of failing tests # -. ${0%/*}/lib.sh +set -e -# Tracing executed commands would produce too much noise in the loop below. -set +x +. ${0%/*}/lib-ci-type.sh +. ${0%/*}/lib-tput.sh cd t/ diff --git a/ci/select-test-slice.sh b/ci/select-test-slice.sh index ec602f8a053..a0332095bb3 100755 --- a/ci/select-test-slice.sh +++ b/ci/select-test-slice.sh @@ -3,7 +3,7 @@ # Select a portion of the tests for testing Git in parallel # -. ${0%/*}/lib.sh +set -ex tests=$(echo $(cd t && ./helper/test-tool path-utils slice-tests "$1" "$2" \ t[0-9]*.sh)) diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh index b8a6a6f664e..64ff212cdaa 100755 --- a/ci/test-documentation.sh +++ b/ci/test-documentation.sh @@ -3,7 +3,7 @@ # Perform sanity checks on "make doc" output and built documentation # -. ${0%/*}/lib.sh +set -ex generator=$1 From patchwork Wed Apr 13 19:51:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12812575 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4BB23C433FE for ; Wed, 13 Apr 2022 19:52:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238597AbiDMTzR (ORCPT ); Wed, 13 Apr 2022 15:55:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60006 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238541AbiDMTzO (ORCPT ); Wed, 13 Apr 2022 15:55:14 -0400 Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com [IPv6:2a00:1450:4864:20::436]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1C04B78917 for ; Wed, 13 Apr 2022 12:52:18 -0700 (PDT) Received: by mail-wr1-x436.google.com with SMTP id r13so4119089wrr.9 for ; Wed, 13 Apr 2022 12:52:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=1bRnEfBrn6ZL38hGLbyN7ReuRiuXEP9h79QAJEf8/Ik=; b=XgCpTl3txT2OTf82zAgGJwboL1CoIGKQt5gVy/9PkDpfLgxs27uXh901hV59ly8F43 HhBM0YCTc38cFDqgT16sdqiazdPrDMPq3Jfun9MB0FODSe1FtkFqpGNGBmaILJ5+6x17 p4VSkMPeUbYcnTm250JR1/+t59OZ3mbRGnHh+OLjvUq22+zPYb9eFUS0dzlnl9BRUy4B NOxjt6tspm48j80dapnUJNXwVGZmlMOl+kqjarNCTh1JRwnWQOtQfQ6uu7sYHiA9owSG de6s4H1CMT/X00A/PzUzM9qP5P9UHxv3dW3uSJc3I7aO6eJmu4S7FI5ACId/jWr9G5Qs 9Rbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=1bRnEfBrn6ZL38hGLbyN7ReuRiuXEP9h79QAJEf8/Ik=; b=yzqMAxIbmX5YYYvujowLSglFxOJGnCWLS2Zuydy9lNtSgFqU1c8NxsZy49ItvbYRSK 9uoJIFka4nb7Tq/9zA5eZzloqP09jCSUVeUTXC45jo6bnPOQFZ0O3XlA1b70G49Hpw4j 7Ha2uLXTFg/yOz8U9Vgl09ZQL+qBUt/UIZIu22zjjvO1q7vDze/vPcbvzB/q5M+sLC1O s1PcpHvZmKSza/xMIJO3RDu+Y5SWvON4i8VRAgHUIdJfQm7+nLIUlYbc/n7jMT7p16FF sukpeo5Psj8nW83wAm5yRfcNsUjr/2rzxqA3PdFaQHErptuQ/6GpK7gxWoOzOW8zrZuN yftQ== X-Gm-Message-State: AOAM532dhwrD75kE8/GwYVsHnCel03NAcGYZ0GuGSTLC0fJpqTusicpW cwtyapwT9yYaoLTQq0auKi5KkxLGrAlxVA== X-Google-Smtp-Source: ABdhPJykF4ZNm2hSN0GWEHgdMAN3s/T6ssh/g2uCiU2MPNrD2f6M0YdCuIIjpyADMjBP1fz+EdiUmw== X-Received: by 2002:a05:6000:186f:b0:205:857d:dee8 with SMTP id d15-20020a056000186f00b00205857ddee8mr339559wri.532.1649879536718; Wed, 13 Apr 2022 12:52:16 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k20-20020a05600c1c9400b0038ecd1ccc17sm3432484wms.35.2022.04.13.12.52.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Apr 2022 12:52:16 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v3 23/29] CI: narrow down variable definitions in --build and --test Date: Wed, 13 Apr 2022 21:51:36 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.rc2.843.g193535c2aa7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In a preceding step the "setvar" function was made to take a "--build", "--test" or "--all" argument to indicate where the variables it sets were used. Let's make use of that by having the relevant parts of ".github/workflows/main.yml" invoke "ci/lib.sh" with those options. By doing this the set of variables shown in build-only steps will be fewer, which makes diagnosing anything going on there easier, as we won't have to look at a deluge of e.g. GIT_TEST_* variables. Since we'll now always run "--build" for "make" and "--test" for "make test" we can stop setting the "fat" MAKEFLAGS entirely on the "test" steps, i.e. the one with PYTHON_PATH, CC etc. These will all be carried over from the earlier --build step. That won't be true in the case of the "windows-test" and "vs-test" jobs, since they run separately from the corresponding "build" step. So we'll need to make sure that we have the --jobs=N argument for those. This doesn't matter that much, as we'll still have --jobs=N in GIT_PROVE_OPTS. So the only thing we'll use it for is parallelism in the t/Makefile before we get to running "prove". Still, it's good to be consistent for good measure, and to run the t/Makefile itself in parallel. Signed-off-by: Ævar Arnfjörð Bjarmason --- .github/workflows/main.yml | 20 +++++++++++--------- ci/lib.sh | 30 +++++++++++++++++++++++------- 2 files changed, 34 insertions(+), 16 deletions(-) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 1e13718b989..0787cadc76b 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -84,7 +84,7 @@ jobs: steps: - uses: actions/checkout@v2 - uses: git-for-windows/setup-git-for-windows-sdk@v1 - - run: ci/lib.sh + - run: ci/lib.sh --build shell: bash - name: build shell: bash @@ -122,7 +122,7 @@ jobs: shell: bash run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz - uses: git-for-windows/setup-git-for-windows-sdk@v1 - - run: ci/lib.sh + - run: ci/lib.sh --test shell: bash - name: select tests run: . /etc/profile && ci/select-test-slice.sh ${{matrix.nr}} 10 @@ -169,7 +169,7 @@ jobs: - name: copy dlls to root shell: cmd run: compat\vcbuild\vcpkg_copy_dlls.bat release - - run: ci/lib.sh + - run: ci/lib.sh --build shell: bash - name: generate Visual Studio solution shell: bash @@ -211,7 +211,7 @@ jobs: - name: extract tracked files and build artifacts shell: bash run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz - - run: ci/lib.sh + - run: ci/lib.sh --test shell: bash - name: select tests run: . /etc/profile && ci/select-test-slice.sh ${{matrix.nr}} 10 @@ -275,8 +275,9 @@ jobs: steps: - uses: actions/checkout@v2 - run: ci/install-dependencies.sh - - run: ci/lib.sh + - run: ci/lib.sh --build - run: make + - run: ci/lib.sh --test - run: make test if: success() - run: ci/print-test-failures.sh @@ -310,8 +311,9 @@ jobs: steps: - uses: actions/checkout@v1 - run: ci/install-dependencies.sh - - run: ci/lib.sh + - run: ci/lib.sh --build - run: make + - run: ci/lib.sh --test - run: make test if: success() && matrix.vector.skip-tests != 'no' - run: ci/print-test-failures.sh @@ -331,7 +333,7 @@ jobs: steps: - uses: actions/checkout@v2 - run: ci/install-dependencies.sh - - run: ci/lib.sh + - run: ci/lib.sh --build - run: make ci-static-analysis sparse: needs: ci-config @@ -352,7 +354,7 @@ jobs: - uses: actions/checkout@v2 - name: Install other dependencies run: ci/install-dependencies.sh - - run: ci/lib.sh + - run: ci/lib.sh --build - run: make sparse documentation: name: documentation @@ -364,7 +366,7 @@ jobs: steps: - uses: actions/checkout@v2 - run: ci/install-dependencies.sh - - run: ci/lib.sh + - run: ci/lib.sh --build - run: make check-docs - run: "make doc > >(tee stdout.log) 2> >(tee stderr.raw >&2)" shell: bash diff --git a/ci/lib.sh b/ci/lib.sh index 8f7e178f3ff..50bc766254b 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -5,6 +5,13 @@ set -ex . ${0%/*}/lib-ci-type.sh # Starting assertions +mode=$1 +if test -z "$mode" +then + echo "need a $0 mode, e.g. --build or --test" >&2 + exit 1 +fi + if test -z "$jobname" then echo "must set a CI jobname in the environment" >&2 @@ -16,11 +23,12 @@ setenv () { while test $# != 0 do case "$1" in - --build) - ;; - --test) - ;; - --all) + --build | --test) + if test "$1" != "$mode" + then + return 0 + fi + ;; -*) echo "BUG: bad setenv() option '$1'" >&2 @@ -46,8 +54,12 @@ setenv () { # How many jobs to run in parallel? NPROC=10 +# For "--test" we carry the MAKEFLAGS over from earlier steps, except +# in stand-alone jobs which will use $COMMON_MAKEFLAGS. +COMMON_MAKEFLAGS=--jobs=$NPROC + # Clear MAKEFLAGS that may come from the outside world. -MAKEFLAGS=--jobs=$NPROC +MAKEFLAGS=$COMMON_MAKEFLAGS case "$CI_TYPE" in github-actions) @@ -101,6 +113,9 @@ windows-build) setenv --build NO_PERL NoThanks setenv --build ARTIFACTS_DIRECTORY artifacts ;; +windows-test) + setenv --test MAKEFLAGS "$COMMON_MAKEFLAGS" + ;; vs-build) setenv --build NO_PERL NoThanks setenv --build NO_GETTEXT NoThanks @@ -117,6 +132,7 @@ vs-build) ;; vs-test) setenv --test NO_SVN_TESTS YesPlease + setenv --test MAKEFLAGS "$COMMON_MAKEFLAGS" ;; linux-gcc) setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main @@ -162,4 +178,4 @@ linux-leaks) ;; esac -setenv --all MAKEFLAGS "$MAKEFLAGS CC=${CC:-cc}" +setenv --build MAKEFLAGS "$MAKEFLAGS CC=${CC:-cc}" From patchwork Wed Apr 13 19:51:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12812576 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2E69CC433EF for ; Wed, 13 Apr 2022 19:52:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238513AbiDMTzS (ORCPT ); Wed, 13 Apr 2022 15:55:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60010 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238543AbiDMTzO (ORCPT ); Wed, 13 Apr 2022 15:55:14 -0400 Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7E6BD49CAC for ; Wed, 13 Apr 2022 12:52:19 -0700 (PDT) Received: by mail-wr1-x432.google.com with SMTP id w4so4103334wrg.12 for ; Wed, 13 Apr 2022 12:52:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=I4xjNP7hxNNURYwDe2DFOL3ZTfaQYI1lJlLATyWzdT4=; b=hX93PSWLKv0DPZbswVVXoYpHRRHY+/ED+U3xMouoco6dgCHR10X0w3n2oQVzl9gmNY dU2x560cqCLTxU7kumvlIKaFP1Ue5rdnnjwryMA4BhUPtN09tAcTfjmYz9cq5WdQNX4F 3dnPxUrHF3a6wj2aI6Aog0XafmnYTSVYQsmnXMcVUfIF/m+StoSTNBmQEMDFqv1MgJbo 65d+yWVnC+/5utEvqzOsaP51y8QsyfExe1I/tT0eJvk1zfZrkK1tYOwYfRJCwExnobow UfddJvVDyApPWkrGGsP+OHPXfT8mYsC/qZ28wqHEWGBxVG7JHVdsW9qr8EzDcGnKDwTY gclA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=I4xjNP7hxNNURYwDe2DFOL3ZTfaQYI1lJlLATyWzdT4=; b=g0crWpnAqSSrlZcp7Fi1s0cdc+uNzuAVvpO0Zzn0dR0Uf8HKTnMBSEUMfpRqPq2gs8 njzqAevkTtugNYNwiXRN74+HUQOMRgX6YBZWhNu6dEjl/mDD+KSS6vQ03JlNSxMPwlT0 FR7H26pJH2pgPpfwGA1eclLFpqmv8AzjpExTpWfXnft5Ht4SMyUWRol6d4HEbN34JApr 3icaOciQpAzg1kLKsON4zm0HUVz20hoPQfZnpx5pnFvmr+E/GR/DJAOC5D8GQNEjwziT 3VFkzt3qxOhmuEM7K3hkBCLYjCiPXuf9C3lNwP/1N+l3oDOtvzpDJmKoRl8PWLvhMx2o ntNA== X-Gm-Message-State: AOAM5322sOoJcJ0LFDbyV9+lMQgZSOrXqPiDEjfP4ISg4WdJ63hTcgWV de1QfTCrWXZxKTqLthIZHaS5IYQnhka+ag== X-Google-Smtp-Source: ABdhPJycpih7E21dYUYRcwxhPDbjBqbN4YamH9B1DU+Xp2PbLzLp/V4Gtl4nNS9tGmlZZ3ZO/LSeZg== X-Received: by 2002:adf:9581:0:b0:1ed:c341:4ed1 with SMTP id p1-20020adf9581000000b001edc3414ed1mr311086wrp.299.1649879537621; Wed, 13 Apr 2022 12:52:17 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k20-20020a05600c1c9400b0038ecd1ccc17sm3432484wms.35.2022.04.13.12.52.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Apr 2022 12:52:17 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v3 24/29] CI: add more variables to MAKEFLAGS, except under vs-build Date: Wed, 13 Apr 2022 21:51:37 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.rc2.843.g193535c2aa7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org It's clearer that "DEVELOPER" is a flag that affects the Makefile itself in particular if it's put into "MAKEFLAGS than" if it generically sits in the environment. Let's move both it and "SKIP_DASHED_BUILT_INS" to "MAKEFLAGS". We can't do this under vs-build, since that invokes cmake. Let's have only that job set these in the environment. Signed-off-by: Ævar Arnfjörð Bjarmason --- ci/lib.sh | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/ci/lib.sh b/ci/lib.sh index 50bc766254b..8fb0bfd43e1 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -61,6 +61,13 @@ COMMON_MAKEFLAGS=--jobs=$NPROC # Clear MAKEFLAGS that may come from the outside world. MAKEFLAGS=$COMMON_MAKEFLAGS +# Use common options for "make" (cmake in "vs-build" below uses the +# intermediate variables directly) +DEVELOPER=1 +MAKEFLAGS="$MAKEFLAGS DEVELOPER=$DEVELOPER" +SKIP_DASHED_BUILT_INS=YesPlease +MAKEFLAGS="$MAKEFLAGS SKIP_DASHED_BUILT_INS=$SKIP_DASHED_BUILT_INS" + case "$CI_TYPE" in github-actions) CC="${CC:-gcc}" @@ -77,10 +84,8 @@ github-actions) ;; esac -setenv --build DEVELOPER 1 setenv --test DEFAULT_TEST_TARGET prove setenv --test GIT_TEST_CLONE_2GB true -setenv --build SKIP_DASHED_BUILT_INS YesPlease case "$runs_on_pool" in ubuntu-latest) @@ -117,6 +122,9 @@ windows-test) setenv --test MAKEFLAGS "$COMMON_MAKEFLAGS" ;; vs-build) + setenv --build DEVELOPER $DEVELOPER + setenv --build SKIP_DASHED_BUILT_INS $SKIP_DASHED_BUILT_INS + setenv --build NO_PERL NoThanks setenv --build NO_GETTEXT NoThanks setenv --build ARTIFACTS_DIRECTORY artifacts From patchwork Wed Apr 13 19:51:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12812578 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1397FC433EF for ; Wed, 13 Apr 2022 19:53:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238547AbiDMTzX (ORCPT ); Wed, 13 Apr 2022 15:55:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59996 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238553AbiDMTzO (ORCPT ); Wed, 13 Apr 2022 15:55:14 -0400 Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com [IPv6:2a00:1450:4864:20::335]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 82FDB78FD4 for ; Wed, 13 Apr 2022 12:52:20 -0700 (PDT) Received: by mail-wm1-x335.google.com with SMTP id 123-20020a1c1981000000b0038b3616a71aso1830252wmz.4 for ; Wed, 13 Apr 2022 12:52:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=z6hN6yhxTyL4HTmIabilzROseTzBQj4dl12uB2yOnbg=; b=b43QXCOuNlQS1GjRUkUdi+zgxU50/u+yu4sKKsL/+qtfxD5prSG7zQ/EpIz1kApiZM vyLbjw2rPU/TAYJcXkAqvdit5R+T6wpbrjNdqgXMfaRrNE1+wHsjtTYaWKySYwwomQkl QVpw1rUEA5VL/JjZSPlsuvQc/nPeruWJPeQApqpcMvq5nYKJHHR6ntXhBejTLx3Efkom 1G4oRqvc/tSP6oA78WiVptfD+A0odbabMIt7rsjMJjYBzoUr1+q7yzwL6N47FzjBb9pP PF0RyQ030ep/oCndBalhIU+v8HWWMHv5GSsdAiT4uSBXTtaQdoMyweLyzjJXfqaHMtcj LMQQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=z6hN6yhxTyL4HTmIabilzROseTzBQj4dl12uB2yOnbg=; b=npJSEFl8V0t86lvf05ZqyBW+HKehOrD3aXvy0wGN7Z/XeM+onGo4Ia5rHw6PpL1opS CHpeygSV3VnzUryDjo8nfhLBUsW6UE8fVk+XK4MenPT1z1D7KsA/F4l/JnMr1pdZZVj3 lCGcuVl8mKOhnp3TTlPfuOZWmv/TAwvQT8+nn2/VaRuyw8BVienfOW7Umh2MIP0YDWps 616zJqkjHR16SHgq4HpjEfQEIBvCtL0RWh5HQzt6ZuVWsJts9ifUQhNtOfrcopW1Dwiq yKmYSRtrRs2A1sWuixE/A70BjSBJ3zZuR+1ZOuLrzos8UKVOtNv6+2TyrX1O+WfARG05 Oo8Q== X-Gm-Message-State: AOAM530HbLIxVuZTQp7PJL9C87awSCMCpIOiJUwhZU77EcPhpweL3VHe Vro+2YLpjSZmIV+7k1HYfyyTW0RQLb1d2Q== X-Google-Smtp-Source: ABdhPJyJAI9x9qnl/W4VMiviVSrYRatrlLzhLa+KCSb2Cj5QPFj/1k4X+3qNI+iqvR2irNDd0Oy1oQ== X-Received: by 2002:a05:600c:1e17:b0:38e:ba41:2465 with SMTP id ay23-20020a05600c1e1700b0038eba412465mr284859wmb.132.1649879538637; Wed, 13 Apr 2022 12:52:18 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k20-20020a05600c1c9400b0038ecd1ccc17sm3432484wms.35.2022.04.13.12.52.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Apr 2022 12:52:18 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v3 25/29] CI: set CC in MAKEFLAGS directly, don't add it to the environment Date: Wed, 13 Apr 2022 21:51:38 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.rc2.843.g193535c2aa7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Rather than pass a "$CC" and "$CC_PACKAGE" in the environment to be picked up in ci/lib.sh let's instead have ci/lib.sh itself add it directly to MAKEFLAGS. Setting CC=gcc by default made for confusing trace output, and since a preceding change to carry it and others over across "steps" in the GitHub CI it's been even more misleading. E.g. the "win+VS build" job confusingly has CC=gcc set, even though it builds with MSVC. Let's instead reply on the Makefile default of CC=cc, and only override it for those jobs where it's needed. This does mean that we'll need to set it for the "pedantic" job, which previously relied on the default CC=gcc in case "clang" become the default on that platform. This partially reverts my 707d2f2fe86 (CI: use "$runs_on_pool", not "$jobname" to select packages & config, 2021-11-23), i.e. we're now aiming to only set those variables specific jobs need. Signed-off-by: Ævar Arnfjörð Bjarmason --- .github/workflows/main.yml | 13 ------------- ci/lib.sh | 27 ++++++++++++++++++++++++--- 2 files changed, 24 insertions(+), 16 deletions(-) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 0787cadc76b..6d25ec4ae3b 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -238,37 +238,24 @@ jobs: matrix: vector: - jobname: linux-clang - cc: clang pool: ubuntu-latest - jobname: linux-sha256 - cc: clang os: ubuntu pool: ubuntu-latest - jobname: linux-gcc - cc: gcc - cc_package: gcc-8 pool: ubuntu-latest - jobname: linux-TEST-vars - cc: gcc os: ubuntu - cc_package: gcc-8 pool: ubuntu-latest - jobname: osx-clang - cc: clang pool: macos-latest - jobname: osx-gcc - cc: gcc - cc_package: gcc-9 pool: macos-latest - jobname: linux-gcc-default - cc: gcc pool: ubuntu-latest - jobname: linux-leaks - cc: gcc pool: ubuntu-latest env: - CC: ${{matrix.vector.cc}} - CC_PACKAGE: ${{matrix.vector.cc_package}} jobname: ${{matrix.vector.jobname}} runs_on_pool: ${{matrix.vector.pool}} runs-on: ${{matrix.vector.pool}} diff --git a/ci/lib.sh b/ci/lib.sh index 8fb0bfd43e1..c73b107d9c7 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -51,6 +51,10 @@ setenv () { fi } +# Clear variables that may come from the outside world. +CC= +CC_PACKAGE= + # How many jobs to run in parallel? NPROC=10 @@ -70,8 +74,6 @@ MAKEFLAGS="$MAKEFLAGS SKIP_DASHED_BUILT_INS=$SKIP_DASHED_BUILT_INS" case "$CI_TYPE" in github-actions) - CC="${CC:-gcc}" - setenv --test GIT_PROVE_OPTS "--timer --jobs $NPROC" GIT_TEST_OPTS="--verbose-log -x" test Windows != "$RUNNER_OS" || @@ -143,9 +145,16 @@ vs-test) setenv --test MAKEFLAGS "$COMMON_MAKEFLAGS" ;; linux-gcc) + CC=gcc + CC_PACKAGE=gcc-8 setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main ;; +linux-gcc-default) + CC=gcc + ;; linux-TEST-vars) + CC=gcc + CC_PACKAGE=gcc-8 setenv --test GIT_TEST_SPLIT_INDEX yes setenv --test GIT_TEST_MERGE_ALGORITHM recursive setenv --test GIT_TEST_FULL_IN_PACK_ARRAY true @@ -160,13 +169,23 @@ linux-TEST-vars) setenv --test GIT_TEST_WRITE_REV_INDEX 1 setenv --test GIT_TEST_CHECKOUT_WORKERS 2 ;; +osx-gcc) + CC=gcc + CC_PACKAGE=gcc-9 + ;; +osx-clang) + CC=clang + ;; linux-clang) + CC=clang setenv --test GIT_TEST_DEFAULT_HASH sha1 ;; linux-sha256) + CC=clang setenv --test GIT_TEST_DEFAULT_HASH sha256 ;; pedantic) + CC=gcc # Don't run the tests; we only care about whether Git can be # built. setenv --build DEVOPTS pedantic @@ -181,9 +200,11 @@ linux-musl) MAKEFLAGS="$MAKEFLAGS GIT_TEST_UTF8_LOCALE=C.UTF-8" ;; linux-leaks) + CC=gcc setenv --build SANITIZE leak setenv --test GIT_TEST_PASSING_SANITIZE_LEAK true ;; esac -setenv --build MAKEFLAGS "$MAKEFLAGS CC=${CC:-cc}" +MAKEFLAGS="$MAKEFLAGS${CC:+ CC=$CC}" +setenv --build MAKEFLAGS "$MAKEFLAGS" From patchwork Wed Apr 13 19:51:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12812577 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 13F90C433FE for ; Wed, 13 Apr 2022 19:53:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238611AbiDMTzV (ORCPT ); Wed, 13 Apr 2022 15:55:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60000 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238552AbiDMTzO (ORCPT ); Wed, 13 Apr 2022 15:55:14 -0400 Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com [IPv6:2a00:1450:4864:20::32e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8B8527938D for ; Wed, 13 Apr 2022 12:52:21 -0700 (PDT) Received: by mail-wm1-x32e.google.com with SMTP id m33-20020a05600c3b2100b0038ec0218103so1830335wms.3 for ; Wed, 13 Apr 2022 12:52:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=pRa04VMpehokdK0nTkHU8KFYBRhmU7wVmus9ul3lobQ=; b=IqgRIc6GvOVOVpvnwMU7231UvNioKoZ26N7cKcZzyYjPTD8yQ88XwMPH0SfbMpz4N8 mvLNWHZTFFgSqkzSvw8501mGnH+uLxuBFsHlAIvtsM35toTj21mQucAr/kQcfispyhp2 KBKELOjlfn87Vcey8CnAwHWxt8LecK4elFWBw4CQF+yw2sPH2cQAVvvhxe0co9fuig75 9WCTZdjsWn5y1zf5sP8AvYTj6ZffdWHhzLxPXpLG6x9E86cPxioH8LcQiVThZIAWmfTa RHyZbaDyW8HMzGaveMNOMNM0cNtRd49W6xTf5eu6RhHYzX1zp8uCeiyQMW5PGKg6ZFrv eQsw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pRa04VMpehokdK0nTkHU8KFYBRhmU7wVmus9ul3lobQ=; b=b8AULyTJOaiNOpVV6UsrRj+Pb1hJVKN2h5pC09DxPaDBmb1yQ+6Y1Y09lJtbY09eZ3 jrNHh/emEdAOfIlRgbnYt/76zZYVPof9n8R74aUNY6qtJZ47yXFY2bOk44zTAChe+0gB sqC3rqWLY9XnI0qRVakCoLkTOSZ4PJZ04qdOs5YK3k51CO94Vn7z14j/soIlI07FzfXN O9OPzIOapeXU3wMMlF5Y7kIHB0GcIeqzPJkl5TZ0VPySGvRU2a0t2rJs6SZ8rkkb3T66 VzlVGJ7jUhzvrYGg2WTquKbdkVbD6L0ZgVS/GWLTvll02S6mA492C7e1CWThyKtdvJvI b3BQ== X-Gm-Message-State: AOAM53333qrXnF7SKqNolO6q+7eHQcDV9rBe4Dqj/lC64/Zay5aE+jru eHb+V1lyk2ewg7E5zDsTNxZvnycBwZLJzw== X-Google-Smtp-Source: ABdhPJwajpE7vVAwvRWeOE/ETllQLo1xfuYpWiKH2sYLpE9HOymnhh7eeuL4wj++fVQ6qpxfnPibvw== X-Received: by 2002:a1c:f607:0:b0:381:1db:d767 with SMTP id w7-20020a1cf607000000b0038101dbd767mr252853wmc.165.1649879539523; Wed, 13 Apr 2022 12:52:19 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k20-20020a05600c1c9400b0038ecd1ccc17sm3432484wms.35.2022.04.13.12.52.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Apr 2022 12:52:18 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v3 26/29] CI: set SANITIZE=leak in MAKEFLAGS directly Date: Wed, 13 Apr 2022 21:51:39 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.rc2.843.g193535c2aa7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org As in the preceding commit's change to do likewise for CC, let's not set SANITIZE in the environment. When I added the "linux-leaks" job in in 956d2e4639b (tests: add a test mode for SANITIZE=leak, run it in CI, 2021-09-23) I missed that I could do that. Signed-off-by: Ævar Arnfjörð Bjarmason --- ci/lib.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ci/lib.sh b/ci/lib.sh index c73b107d9c7..b439c1a4f82 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -201,7 +201,7 @@ linux-musl) ;; linux-leaks) CC=gcc - setenv --build SANITIZE leak + MAKEFLAGS="$MAKEFLAGS SANITIZE=leak" setenv --test GIT_TEST_PASSING_SANITIZE_LEAK true ;; esac From patchwork Wed Apr 13 19:51:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12812580 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 46DE9C433EF for ; Wed, 13 Apr 2022 19:53:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238613AbiDMTz1 (ORCPT ); Wed, 13 Apr 2022 15:55:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60030 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238556AbiDMTzO (ORCPT ); Wed, 13 Apr 2022 15:55:14 -0400 Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com [IPv6:2a00:1450:4864:20::32e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8F27D7B107 for ; Wed, 13 Apr 2022 12:52:22 -0700 (PDT) Received: by mail-wm1-x32e.google.com with SMTP id q20so1757925wmq.1 for ; Wed, 13 Apr 2022 12:52:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=tXDlyatyKqavNyZNG9G6Gr1854moB/y55bDXwPw41KQ=; b=eCF6H0PcvfgyYL8Al2Loptb9SlQbcAlUWNQbjJRf2yuAY+MTmnYWiKKCOxEMgyJHMV qzas4TomXg69YEvsyG8oI+3CMu2x74Jph/uLthe8W2l21UEXcy+xDSd06Nxm/kWilWPi 1oC6jdqcvr0Z1m2mH+utlKP3oASMBjOjpBcxA/Y2xS0pb+LkIwLvWP+l7p4lgGyhgfdF ocurAzJnMxeqjWDNgt1/+QiRVpCQ7OtqPZwPbtPDdZLd3Mo2cVltBGY8d50wKHqUIbq7 b6or58Lk5h/knn4nDvdn8mDrXF9fXWn++oJDu16mskHQQeL9gXF6db1xVnBFNiyUno4S vV7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=tXDlyatyKqavNyZNG9G6Gr1854moB/y55bDXwPw41KQ=; b=1Olle0c1gkoeZr1LSQudMnbaAtOibEWYg0P2e4EKrSe74C3EwojGqWgSm1o470GfyF UjzQD62MygHFpr6Of1rzLsGeQ/RCpmFQamrFzZXA1qZJd0Q82xnPc7LuM0FwVVUq1WsT l36oDelFN/RroDgaiR8Na2Tc70q32+ZlLMDxEETKsa3E49SHA2QpSx85PFRu6FLtSd6M pfCE/leXAIIDZbL0x/c1CdM2OyZ7IRhR1pwrs93o5zaXvhJOgzi79P6HKei/2bgKbiDP NrleKl6gPY+MkS+B5z1PM2oPYK+si9jeEFesGVPKVJEqhnTQtrm99Qyw1kr8K/0xNbdK pyXA== X-Gm-Message-State: AOAM5331legjg+VKVhMUFs7xPgYtOrNwLOki4h+R++YEmgTsxZMrIXQH 0rz7PwyPRF64tUcmrgiAzUhtcOnhxZ74jw== X-Google-Smtp-Source: ABdhPJwK1n83pBhsliuHZ9+m9JiqfvCi8FqgwflpZWT4w+slZk6A3bC5o3hcsDzUIVxCfWp/RrLliQ== X-Received: by 2002:a1c:e916:0:b0:37c:f44f:573 with SMTP id q22-20020a1ce916000000b0037cf44f0573mr259034wmc.179.1649879540479; Wed, 13 Apr 2022 12:52:20 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k20-20020a05600c1c9400b0038ecd1ccc17sm3432484wms.35.2022.04.13.12.52.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Apr 2022 12:52:19 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v3 27/29] CI: set PYTHON_PATH setting for osx-{clang,gcc} into "$jobname" case Date: Wed, 13 Apr 2022 21:51:40 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.rc2.843.g193535c2aa7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Move the setting of "PYTHON_PATH=[...]" to the "$jobname" case statement. This partially backs out of my 707d2f2fe86 (CI: use "$runs_on_pool", not "$jobname" to select packages & config, 2021-11-23), now that we have a "osx-{clang,gcc}" anyway for setting "$CC" we might as well do away with this part of the "$runs_on_pool" case. Signed-off-by: Ævar Arnfjörð Bjarmason --- ci/lib.sh | 10 ++-------- 1 file changed, 2 insertions(+), 8 deletions(-) diff --git a/ci/lib.sh b/ci/lib.sh index b439c1a4f82..d62cd757ef4 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -105,14 +105,6 @@ ubuntu-latest) setenv --test GIT_TEST_HTTPD true ;; -macos-latest) - if test "$jobname" = osx-gcc - then - MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)" - else - MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python2)" - fi - ;; esac case "$jobname" in @@ -170,10 +162,12 @@ linux-TEST-vars) setenv --test GIT_TEST_CHECKOUT_WORKERS 2 ;; osx-gcc) + MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)" CC=gcc CC_PACKAGE=gcc-9 ;; osx-clang) + MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python2)" CC=clang ;; linux-clang) From patchwork Wed Apr 13 19:51:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12812581 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B8FC3C433EF for ; Wed, 13 Apr 2022 19:53:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238614AbiDMTz2 (ORCPT ); Wed, 13 Apr 2022 15:55:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60058 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238563AbiDMTzP (ORCPT ); Wed, 13 Apr 2022 15:55:15 -0400 Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com [IPv6:2a00:1450:4864:20::330]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 795587B12F for ; Wed, 13 Apr 2022 12:52:23 -0700 (PDT) Received: by mail-wm1-x330.google.com with SMTP id q20so1757961wmq.1 for ; Wed, 13 Apr 2022 12:52:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=uUOquCtDFxghbCgPJPLdp/jLF8/rM7kXCG1dL4CcXUs=; b=idkXoN7uMqxXA121f5cuF/Tw7ThRGJE8k8vp9FjRdtpPgq4MF3cvoihZ6uBZMv9ho0 4UD3C3tUJ2pH80PskhFW/16EGXWzPScXRigfU3ZLvEKWEyC76Qo8aQ6CJc3pxlxEFhDP KKIZANCtNOgGqOYBS+XpqyGHNBO2cyQrMs0KRSQLOOQklcAPP9uIew3zewT32Ci194Pg X1S0Eu8xW9BP/bq3JusfgbSfoI8dbJ4hZT0U65PvnFC4XssU64jTAUZcn2wezHAmVt0d 9Odi4HQ+y3Cz7ob7+0jesZb3tl0e/0ulK+5XeuTdXj3ktjFFUk4CEPJupp3+0N5Xyzoh mwvg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=uUOquCtDFxghbCgPJPLdp/jLF8/rM7kXCG1dL4CcXUs=; b=BEJllewv7UY/H+6T5Zs1mei1CAXE0oXAR3XmBiKoC5o0Gjio3FRecBQoRk8kG5W8fF CztdVeW1Sro+Iu5VZIv4BGhnS5ec3dhuawPLcZkWWI5R7RDpMV0XoIWEwvj16IctVYkM wIy9QDxHY5FhavjzHiTHGU27SySTISMhdMvQCh9/VJJ5cxEMn6+CZaFb7He6qf8aS/Az 6cO646kDiiZPhctxvKAbkSvQiXSTIRBpjVel6R4DV4IbX2rXHEcu+2uufty9x7fpnnGt TNewSiI37LGuVBi2bFnEOpNPeSC4MxAr2i94XujpBsIkIR7Js/YDqlISyf/9qyNDIQQp 7+Vg== X-Gm-Message-State: AOAM532AQqnkm3B2W0CzQSn5B3TM1UGxOYtTxt6Jbk8fnv5UIhAzIcyR xK2ApDnhB5a7QWyW1WDjYO39XM2AItgDwA== X-Google-Smtp-Source: ABdhPJwxIDiQza1qB8mIfs2KoAb88akG6mhsJ8cjHaDupkiUkCwGoAKNFIVcFH93iJW1Ihg0LGEipw== X-Received: by 2002:a05:600c:35ca:b0:38e:c3cd:9692 with SMTP id r10-20020a05600c35ca00b0038ec3cd9692mr293534wmq.102.1649879541404; Wed, 13 Apr 2022 12:52:21 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k20-20020a05600c1c9400b0038ecd1ccc17sm3432484wms.35.2022.04.13.12.52.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Apr 2022 12:52:20 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v3 28/29] CI: don't use "set -x" in "ci/lib.sh" output Date: Wed, 13 Apr 2022 21:51:41 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.rc2.843.g193535c2aa7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Remove the "set -x" from the "ci/lib.sh" output. Before preceding commits the logic in that file was much more complex, and likely to fail in some scenarios. Now we only task "ci/lib.sh" with setting various variables for subsequent steps in our jobs, so we can start emitting more tailored debugging output, which makes what it's doing easier to read. This change also changes the output of the "ci/print-test-failures.sh" script, since it's the only other user of "ci/lib-ci-type.sh". In that case it's also helpful to know what "$CI_TYPE" we're using, as that script doesn't "set -x" and will act differently depending on the $CI_TYPE. Signed-off-by: Ævar Arnfjörð Bjarmason --- ci/lib.sh | 23 +++++++++++++++++++++-- 1 file changed, 21 insertions(+), 2 deletions(-) diff --git a/ci/lib.sh b/ci/lib.sh index d62cd757ef4..94f83069884 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -1,5 +1,5 @@ #!/bin/sh -set -ex +set -e # Helper libraries . ${0%/*}/lib-ci-type.sh @@ -18,15 +18,26 @@ then exit 1 fi +# Show our configuration +echo "CONFIG: CI_TYPE=$CI_TYPE" >&2 +echo "CONFIG: jobname=$jobname" >&2 +echo "CONFIG: runs_on_pool=$runs_on_pool" >&2 +echo "CONFIG: GITHUB_ENV=$GITHUB_ENV" >&2 +if test -n "$GITHUB_ENV" +then + echo "CONFIG: GITHUB_ENV=$GITHUB_ENV" >&2 +fi + # Helper functions setenv () { + local skip= while test $# != 0 do case "$1" in --build | --test) if test "$1" != "$mode" then - return 0 + skip=t fi ;; @@ -45,10 +56,18 @@ setenv () { val=$2 shift 2 + if test -n "$skip" + then + echo "SKIP '$key=$val'" >&2 + return 0 + fi + if test -n "$GITHUB_ENV" then echo "$key=$val" >>"$GITHUB_ENV" fi + + echo "SET: '$key=$val'" >&2 } # Clear variables that may come from the outside world. From patchwork Wed Apr 13 19:51:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12812582 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CB54FC433EF for ; Wed, 13 Apr 2022 19:53:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231805AbiDMTzd (ORCPT ); Wed, 13 Apr 2022 15:55:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60070 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238567AbiDMTzP (ORCPT ); Wed, 13 Apr 2022 15:55:15 -0400 Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com [IPv6:2a00:1450:4864:20::332]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B05157B545 for ; Wed, 13 Apr 2022 12:52:24 -0700 (PDT) Received: by mail-wm1-x332.google.com with SMTP id x3so1745729wmj.5 for ; Wed, 13 Apr 2022 12:52:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=IumLz8zjxTQlcfRdQHq8uFmPHRTJczvPdPUKXWE7Nyo=; b=dn1QVVHvOEukJmmOZ9KlW14gntWtVqF2o7dFRrwT9Iq87qdVh0QduHREV87PEJJvfY dBPYlNLZK7lAZ/pMsJEXaeh2MEnr8k/u8nqMv7CKHPmD5JeL7fyUZMA39vvFYyw+OY3P 5/ZXX4bUVWdbm2MV/Q+W+jx7eJgrr6FkMmP9XFg5zBNwhvNscG0bggGy9o/Npn1u/rHB JOVn3TbU28hi84RyGi6t4WtS5fDOMjiURqI9bsy5Uwh550fbWAlcFB1USD6RnEW4dmTt XfBOpZIJsMwIIwPIedRFcZFFY3Ehz3XsBVvnyedi3+N828INhUAlpiGSIu1ujZqCPHsC 6BIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=IumLz8zjxTQlcfRdQHq8uFmPHRTJczvPdPUKXWE7Nyo=; b=VU70AYrsNEe+r9sGD8POYLyq4A4vADlSDVs3hr4ViIgy+wbM/sugQF1jdMv4FZJZSi HWZ0NZNq5EQWKfGTWwwzW2Zq2mAKCvFAB0jj3t3W5qWieOUq4hS+ddWYJte51aMGPyk7 lzXopXd5BSYPy4jLmI0AT4Be857YXyLuQVXiGLU5Rf5QmzYuA2/Uiz/Qn/PTIShxLSm8 G0apGk72At+clz9ZhaTsrad0fWr5EsgKpJZTiAgrJt89mbb66e2Asa+bBiu1E2nbtu5x 7kbuxofNKes4qJ3ehZPvlZhK6hFeLNMWJp1jbuv+8z6OV+A/m4YlHd2Khz1Ofk8XeUPq Qd6w== X-Gm-Message-State: AOAM532BVhbh7Krwf1EUoz1LgL4M2DkBKARemCHGS4IV2U4Ur+GRII/Z 5kEyAxHy2/dL96QH4MTrmJsdFjjnO1rqhw== X-Google-Smtp-Source: ABdhPJy27FRXpU1n2VCyaFOkW6FHv4dfHdh0s2c8N35g9Icf0nNPw1q+4C1X5l4cF5BE2RmqAdAQzw== X-Received: by 2002:a05:600c:3547:b0:38c:ac1c:53e9 with SMTP id i7-20020a05600c354700b0038cac1c53e9mr256766wmq.159.1649879542272; Wed, 13 Apr 2022 12:52:22 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id k20-20020a05600c1c9400b0038ecd1ccc17sm3432484wms.35.2022.04.13.12.52.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Apr 2022 12:52:21 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v3 29/29] CI: make it easy to use ci/*.sh outside of CI Date: Wed, 13 Apr 2022 21:51:42 +0200 Message-Id: X-Mailer: git-send-email 2.36.0.rc2.843.g193535c2aa7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In preceding commits the ci/.sh scripts have lost most of their CI-specific assumptions. Let's go even further and explicitly support running ci/lib.sh outside of CI. This was possible before by faking up enough CI-specific variables, but as shown in the new "help" output being added here using the ci/lib.sh to provide "CI-like" has now become trivial. The ci/print-test-failures.sh scripts can now be used outside of CI as well, the only GitHub CI-specific part is now guarded by a check that we'll pass if outside of GitHub CI. There's also a special-case here to not clobber $MAKEFLAGS in the environment if we're outside of CI, in case the user has e.g. "jN" or other flags to "make" that they'd prefer configured already. Signed-off-by: Ævar Arnfjörð Bjarmason --- ci/lib-ci-type.sh | 3 -- ci/lib.sh | 61 ++++++++++++++++++++++++++++++++++++--- ci/print-test-failures.sh | 11 ++----- 3 files changed, 60 insertions(+), 15 deletions(-) diff --git a/ci/lib-ci-type.sh b/ci/lib-ci-type.sh index 6f01fd9e5d9..09acab7aaec 100644 --- a/ci/lib-ci-type.sh +++ b/ci/lib-ci-type.sh @@ -3,7 +3,4 @@ if test "$GITHUB_ACTIONS" = "true" then CI_TYPE=github-actions -else - echo "Could not identify CI type" >&2 - exit 1 fi diff --git a/ci/lib.sh b/ci/lib.sh index 94f83069884..3fd5291bc80 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -1,6 +1,30 @@ #!/bin/sh set -e +# Usage +CI_TYPE_HELP_COMMANDS=' + # run "make all test" like the "linux-leaks" job + (eval $(jobname=linux-leaks ci/lib.sh --all) && make test) + + # run "make all test" like the "linux-musl" job + (eval $(jobname=linux-musl ci/lib.sh --all) && make test) + + # run "make test" like the "linux-TEST-vars" job (uses various GIT_TEST_* modes) + make && (eval $(jobname=linux-TEST-vars ci/lib.sh --test) && make test) + + # run "make test" like the "linux-sha256" job + make && (eval $(jobname=linux-sha256 ci/lib.sh --test) && make test) +' + +CI_TYPE_HELP=" +running $0 outside of CI? You can use ci/lib.sh to set up your +environment like a given CI job. E.g.: +$CI_TYPE_HELP_COMMANDS + +note that some of these (e.g. the linux-musl one) may not work as +expected due to the CI job configuring a platform that may not match +yours." + # Helper libraries . ${0%/*}/lib-ci-type.sh @@ -9,6 +33,10 @@ mode=$1 if test -z "$mode" then echo "need a $0 mode, e.g. --build or --test" >&2 + if test -z "$CI_TYPE" + then + echo "$CI_TYPE_HELP" >&2 + fi exit 1 fi @@ -35,7 +63,7 @@ setenv () { do case "$1" in --build | --test) - if test "$1" != "$mode" + if test "$1" != "$mode" && test "$mode" != "--all" then skip=t fi @@ -65,6 +93,10 @@ setenv () { if test -n "$GITHUB_ENV" then echo "$key=$val" >>"$GITHUB_ENV" + elif test -z "$CI_TYPE" + then + echo "$key=\"$val\"" + echo "export $key" fi echo "SET: '$key=$val'" >&2 @@ -76,10 +108,29 @@ CC_PACKAGE= # How many jobs to run in parallel? NPROC=10 +case "$CI_TYPE" in +'') + if command -v nproc >/dev/null + then + NPROC=$(nproc) + else + NPROC=1 + fi + + if test -n "$MAKEFLAGS" + then + COMMON_MAKEFLAGS="$MAKEFLAGS" + else + COMMON_MAKEFLAGS=--jobs=$NPROC + fi + ;; +*) + # For "--test" we carry the MAKEFLAGS over from earlier steps, except + # in stand-alone jobs which will use $COMMON_MAKEFLAGS. + COMMON_MAKEFLAGS=--jobs=$NPROC + ;; +esac -# For "--test" we carry the MAKEFLAGS over from earlier steps, except -# in stand-alone jobs which will use $COMMON_MAKEFLAGS. -COMMON_MAKEFLAGS=--jobs=$NPROC # Clear MAKEFLAGS that may come from the outside world. MAKEFLAGS=$COMMON_MAKEFLAGS @@ -99,6 +150,8 @@ github-actions) GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS" setenv --test GIT_TEST_OPTS "$GIT_TEST_OPTS" ;; +'') + ;; *) echo "Unhandled CI type: $CI_TYPE" >&2 exit 1 diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh index 2405f65650a..ba8428ad484 100755 --- a/ci/print-test-failures.sh +++ b/ci/print-test-failures.sh @@ -38,19 +38,14 @@ do test_name="${TEST_EXIT%.exit}" test_name="${test_name##*/}" trash_dir="trash directory.$test_name" - case "$CI_TYPE" in - github-actions) + if test "$CI_TYPE" = "github-actions" + then mkdir -p failed-test-artifacts echo "FAILED_TEST_ARTIFACTS=t/failed-test-artifacts" >>$GITHUB_ENV cp "${TEST_EXIT%.exit}.out" failed-test-artifacts/ tar czf failed-test-artifacts/"$test_name".trash.tar.gz "$trash_dir" continue - ;; - *) - echo "Unhandled CI type: $CI_TYPE" >&2 - exit 1 - ;; - esac + fi trash_tgz_b64="trash.$test_name.base64" if [ -d "$trash_dir" ] then