From patchwork Mon Feb 21 14:46:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12753706 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 56D63C433EF for ; Mon, 21 Feb 2022 14:46:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378294AbiBUOrP (ORCPT ); Mon, 21 Feb 2022 09:47:15 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:59298 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1378288AbiBUOrO (ORCPT ); Mon, 21 Feb 2022 09:47:14 -0500 Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com [IPv6:2a00:1450:4864:20::332]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D073538A5 for ; Mon, 21 Feb 2022 06:46:49 -0800 (PST) Received: by mail-wm1-x332.google.com with SMTP id p14-20020a05600c1d8e00b0037f881182a8so1529542wms.2 for ; Mon, 21 Feb 2022 06:46:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=+u1fGve9Kxhe6eFVjZJn/mvFAwvpubdq0fiKHmt+o+w=; b=FQ3Vsj1QIbpvH0fMOi91Zfv2JHvjCE+XwgDuStFKL1gdPB5ycJT8ygnDhLzFN2U4tU em2qXw0htwjPRJjqR84CVQb+297PC1gxt0VjtdZCrn88UF3iKeLG0tiBXrVcfTwy5kyK +55hBL/kZfs8MAFaA71vZAxoAEQGSgbBcNFmkhLt0wAMm74TyuC07g7yrRYmiZdUPc6c wGelJP6u3Fyb2mD1ecL0iE8+tSYqs46laQnmV/Kfwz1XaL64jnniF43r6jwsZULFW9E1 FgIAjBjSx+yl5urqxd/tof5Ythod29r6Q1tAcRAzTkbwAOoaV4FkDtpAjLVRouEsPFP1 +KUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+u1fGve9Kxhe6eFVjZJn/mvFAwvpubdq0fiKHmt+o+w=; b=FyAzmW1x37Wz1FZPzraXbUKwGL+OJgl1tKYgrl634lrHd13yXY59euKMzVN5qaR2lN jQQWB73H8N04MzlW5wJyDxlPH/8EH3sqSz3eQyK5ufMKWa2mSk59p74Iwx5Dl9soanJw m94xM3g5vY7ZUJEzBMS9RS7O+DsJ9IoV4lMxU3lWHovXUd1qCji2aq8EJoZzH5OyTtbs I+FF8EgqZOFF9Y5DP9QN3ZQU2olYsREB+pjmDc6OmkvpW9MekRMdmHXxANszFpg1AOJ7 R6I1N+pb744neah7puPxC+cCJDytug3VGbpsIEm68YOJX5V1DRbGroYRmuSfLDRjk2rZ 1U4g== X-Gm-Message-State: AOAM531wSMS9B7ZAoqHkppO7BzUDQJFlyU4RQ9ge8yO9LWCim2B1xrnb 0FD3/8/oPXLbOY5x0M0zij52wjFydpE0Qg== X-Google-Smtp-Source: ABdhPJx1lppag57f6S0GJyqId14rKtw1Z1KGuHlLLkYXyOB6PrYnV2riNORKkqBQJ9M9OnefubfOcw== X-Received: by 2002:a05:600c:4c27:b0:37b:fa17:899a with SMTP id d39-20020a05600c4c2700b0037bfa17899amr17907951wmp.126.1645454808171; Mon, 21 Feb 2022 06:46:48 -0800 (PST) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id r14sm25071094wrz.84.2022.02.21.06.46.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Feb 2022 06:46:47 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH 01/25] CI: run "set -ex" early in ci/lib.sh Date: Mon, 21 Feb 2022 15:46:13 +0100 Message-Id: X-Mailer: git-send-email 2.35.1.1132.ga1fe46f8690 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Change ci/lib.sh to run "set -ex" before it does anything else, before this we were omitting a few "export" commands in our trace output that happened before we enabled tracing. Signed-off-by: Ævar Arnfjörð Bjarmason --- ci/lib.sh | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/ci/lib.sh b/ci/lib.sh index cbc2f8f1caa..57141d38a85 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -1,5 +1,11 @@ # Library of functions shared by all CI scripts +# Set 'exit on error' for all CI scripts to let the caller know that +# something went wrong. +# Set tracing executed commands, primarily setting environment variables +# and installing dependencies. +set -ex + skip_branch_tip_with_tag () { # Sometimes, a branch is pushed at the same time the tag that points # at the same commit as the tip of the branch is pushed, and building @@ -85,12 +91,6 @@ export TERM=${TERM:-dumb} # Clear MAKEFLAGS that may come from the outside world. export MAKEFLAGS= -# Set 'exit on error' for all CI scripts to let the caller know that -# something went wrong. -# Set tracing executed commands, primarily setting environment variables -# and installing dependencies. -set -ex - if test -n "$SYSTEM_COLLECTIONURI" || test -n "$SYSTEM_TASKDEFINITIONSURI" then CI_TYPE=azure-pipelines From patchwork Mon Feb 21 14:46:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12753708 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 641CDC433EF for ; Mon, 21 Feb 2022 14:46:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378302AbiBUOrS (ORCPT ); Mon, 21 Feb 2022 09:47:18 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:59352 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1378292AbiBUOrO (ORCPT ); Mon, 21 Feb 2022 09:47:14 -0500 Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D851D5596 for ; Mon, 21 Feb 2022 06:46:50 -0800 (PST) Received: by mail-wr1-x42f.google.com with SMTP id m27so15711421wrb.4 for ; Mon, 21 Feb 2022 06:46:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=sV2MR96BaXSEm8tHvq8YfW66x01Lto0/DfOwR/D0QSE=; b=kxsaZ8nB919JsmmfhJhKgWMe52F3mdDLk2UOSIJOCE84pApYbFd37l4Wzg8LGnTPgC L+8BSrO4GUuXcybEtnti3S6x/U9gCX9DSRUNEXPVVsgRLldZkDKG3VTFsEKFbGZjMtBp QGaBjfZIl1PHw9CwVWhZlRsF8vZ5aad+y4IHv4nVWe4uQ2SwmYmYv8DYkOKnYQwzoyyi Wnfl/cbwvU8vkX7eTheCZBO4SGLMEQoZRFAJjvb0voZSF2PIjkmA77g9MGHBu4Pxbq1j I3eCQhU7B6K7Mnde14KoG2oH9z8nVsDquCcg/t8H/qjRnPUagAGY70d89uzZvxdd87LK mEFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=sV2MR96BaXSEm8tHvq8YfW66x01Lto0/DfOwR/D0QSE=; b=ijXgV1jcsSRrRAz2TAZKTuP16uUBIsrjZYcfj0Ts/xGLtZSDB96P1RcF62OQovl+Sm P2gE0pC0QjqTDtfG1exNkzPFZSMKjJ4EQ4+ESZnTsmVwSbFnct4EnnYuzLNkFoZJOjcs KcZpUgIUfw99DsjXnVSfo78ZL+bn0ViR3fgCTIvBE20k7sHt+389Za6Ih39282zuyzGc PLEAmcfrXD0PURbiRGg6uJzNzLNTjPWyGHW8yt80YXIuDgGVbwYDD7xdEo9z+vWZSVn7 PfsTSZRV+SSdBe0tuSi3Y7YgeJcHwPNaH5cUbsDnFf9WCbjFXXDv/5RnHYsO+IqYqR3T Xprw== X-Gm-Message-State: AOAM532SQKgAavEpTWE8XToDUFkLgzTLcjDHQcInALbjg6oEzZ63wQpq GMwyG3nlmAugCLEi0qjJePyrMwyRPtMMyA== X-Google-Smtp-Source: ABdhPJzr6KWmhJ+uToVgv49+YHAXVq9Nh5LJ1TcZ8KXtgSImgx/sROBwy3f7W7OC17udZzJDll7/Ig== X-Received: by 2002:a5d:4292:0:b0:1e4:b7fd:eb84 with SMTP id k18-20020a5d4292000000b001e4b7fdeb84mr16216328wrq.657.1645454809161; Mon, 21 Feb 2022 06:46:49 -0800 (PST) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id r14sm25071094wrz.84.2022.02.21.06.46.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Feb 2022 06:46:48 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH 02/25] CI: make "$jobname" explicit, remove fallback Date: Mon, 21 Feb 2022 15:46:14 +0100 Message-Id: X-Mailer: git-send-email 2.35.1.1132.ga1fe46f8690 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Remove the $jobname fallback last adjusted in b011fabd6e0 (ci/lib.sh: encapsulate Travis-specific things, 2019-01-27). This makes all of our CI jobs have an explicit $jobname, and removes the special-case of the Windows jobs implicitly setting one (that they don't currently use). In subsequent commits we'll use this new $jobname explicitly. This is a follow-up to my 25715419bf4 (CI: don't run "make test" twice in one job, 2021-11-23), since then all the jobs except the Windows jobs have had an explicit $jobname. In the case of the Windows jobs we were using a jobname of windows-gcc for both, due to picking "gcc" as a default fallback for "$CC" in ci/lib.sh. Since we'll need this to run ci/lib.sh it makes sense to move the check to the top of the file. Subsequent commits will add more such assertions to it. Signed-off-by: Ævar Arnfjörð Bjarmason --- .github/workflows/main.yml | 8 ++++++++ ci/lib.sh | 12 +++++++----- 2 files changed, 15 insertions(+), 5 deletions(-) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index c35200defb9..309b9141249 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -79,6 +79,8 @@ jobs: windows-build: name: win build + env: + jobname: windows-build needs: ci-config if: needs.ci-config.outputs.enabled == 'yes' runs-on: windows-latest @@ -100,6 +102,8 @@ jobs: path: artifacts windows-test: name: win test + env: + jobname: windows-test runs-on: windows-latest needs: [windows-build] strategy: @@ -131,6 +135,8 @@ jobs: path: ${{env.FAILED_TEST_ARTIFACTS}} vs-build: name: win+VS build + env: + jobname: vs-build needs: ci-config if: needs.ci-config.outputs.enabled == 'yes' env: @@ -183,6 +189,8 @@ jobs: path: artifacts vs-test: name: win+VS test + env: + jobname: vs-test runs-on: windows-latest needs: vs-build strategy: diff --git a/ci/lib.sh b/ci/lib.sh index 57141d38a85..d2a7c33f536 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -6,6 +6,13 @@ # and installing dependencies. set -ex +# Starting assertions +if test -z "$jobname" +then + echo "must set a CI jobname" >&2 + exit 1 +fi + skip_branch_tip_with_tag () { # Sometimes, a branch is pushed at the same time the tag that points # at the same commit as the tip of the branch is pushed, and building @@ -146,11 +153,6 @@ test -n "${DONT_SKIP_TAGS-}" || skip_branch_tip_with_tag skip_good_tree -if test -z "$jobname" -then - jobname="$CI_OS_NAME-$CC" -fi - export DEVELOPER=1 export DEFAULT_TEST_TARGET=prove export GIT_TEST_CLONE_2GB=true From patchwork Mon Feb 21 14:46:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12753709 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B5EC5C433F5 for ; Mon, 21 Feb 2022 14:47:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378304AbiBUOrb (ORCPT ); Mon, 21 Feb 2022 09:47:31 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:59416 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1378296AbiBUOrP (ORCPT ); Mon, 21 Feb 2022 09:47:15 -0500 Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com [IPv6:2a00:1450:4864:20::32f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CC1D95F80 for ; Mon, 21 Feb 2022 06:46:51 -0800 (PST) Received: by mail-wm1-x32f.google.com with SMTP id j9-20020a05600c190900b0037bff8a24ebso13660040wmq.4 for ; Mon, 21 Feb 2022 06:46:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Q7UOlMxPKIhbrfK8TBnf/mwywbUgoLtWvkCygeNatzI=; b=du5QoCZ+Y7zXc0Lk4wPTOAWnWZXhZ4Cd5mddi/mcxtS/p2k7/PniWRfO/6A8sNpvJl rxm9ITWVUZDEjp36lDLWezNOOJa0DZQAMYz6QSvc3O0/iVcxoEH6vJhjQ/Vouuf489cn zX64PIUJuwEkPVIuSnzu+KchOtpdlzYLXRRHl5mpd7R82mbEpTqW7xV0xbUiQ1hrbwSu jEER9CgfjkvXwNudX8UvanSt6U0QDMKyznmcn3mcUZXpa17kuhOB8U8p5fR2MUrCwADZ 4aIYpUqlLLiJM/q8EMOaX37gTGT/n5eglzBc8/YEBXMSjnC/kpAd/wm47Wzvfk79pU8K LdIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Q7UOlMxPKIhbrfK8TBnf/mwywbUgoLtWvkCygeNatzI=; b=BHriZN9hW+lUjtXEkE39gUThFvREDTWfBWAmoM3hBT4H1A8HPUeSl6OTQxhs3i5CV/ +HLRnmrPiu2kYwRzUDR7Gp3q+4f3jsBoWnxKCP/1Il6NybSbv6MVDpsz47/38m8M5zzw goASJs5xPTfAcyu0waPvf0X74Xp/vBphwQLF1juy5XJ97jNavtrTlGv/3xJhhyhV/CNd RbddNhqlq+0X4T+QcffnWgWdNahtBApMqSXptB8T2zika/RbeyQLhWuNSSlrT2IzawIE 7RXQcP3o93u1yStBUyG1XxfxmbP07Soa+iKGGQEKdMgYRzKiOnWhFw5WnYoKsr8WSHAT Qzfw== X-Gm-Message-State: AOAM533G9ZkpStnELAx9IkNSz6sD3fxkCu1GN2tUgqBlajivd1x4aiVw j6BqDYgYpxkvj8bIB9g4GS1oJ6iMaNCVMg== X-Google-Smtp-Source: ABdhPJz1+BBUXG0cCBD9RHl/ZNU1mKZl1M8CVUrgELFXKJxaxVjLs9WXIF+6yOssVZvugHpmvaHvWw== X-Received: by 2002:a05:600c:348a:b0:37b:ecb3:a249 with SMTP id a10-20020a05600c348a00b0037becb3a249mr21770160wmq.33.1645454810087; Mon, 21 Feb 2022 06:46:50 -0800 (PST) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id r14sm25071094wrz.84.2022.02.21.06.46.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Feb 2022 06:46:49 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH 03/25] CI: remove more dead Travis CI support Date: Mon, 21 Feb 2022 15:46:15 +0100 Message-Id: X-Mailer: git-send-email 2.35.1.1132.ga1fe46f8690 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Remove code that I missed in 4a6e4b96026 (CI: remove Travis CI support, 2021-11-23). This code was only called from or used by the now-removed .travis.yml, or needed by the Travis CI environment. For the symlinking in ci/run-build-and-tests.sh: Back when 3c93b829205 (travis-ci: build Git during the 'script' phase, 2018-01-08) and 4b060a4d973 (ci: use a junction on Windows instead of a symlink, 2019-01-27) adjusted this "ln" command, the Windows build would use ci/run-build-and-tests.sh. As seen in 889cacb6897 (ci: configure GitHub Actions for CI/PR, 2020-04-11) the current windows build uses a different entry point under the GitHub CI, which doesn't use this .prove caching. Namely "ci/run-test-slice.sh". We can be certain that it's never used in "ci/run-test-slice.sh" because to have a ".prove" file we'd need to use "--state=save", which we only do in the dead Azure codepath in ci/lib.sh. If it were used it would do the wrong thing, because the different test slices would each try to clobber the same "t/.prove" file. If a subsequent run then used the -"-state=failed,slow,save" it would defeat the purpose of "ci/run-test-slice.sh", since all slices would then run all tests. I.e. behavior of prove's "--state" options is to select tests to run from the provided "--state" file, in addition to those specified on the command-line. For ci/run-docker{,-build}.sh: It was likewise last referenced in the .travis.yml removed in my 4a6e4b96026. The current "dockerized" run in ".github/workflows/main.yml" calls the same entry points as the main "regular" job. Signed-off-by: Ævar Arnfjörð Bjarmason --- ci/lib.sh | 10 ++---- ci/print-test-failures.sh | 3 -- ci/run-build-and-tests.sh | 5 --- ci/run-docker-build.sh | 66 ----------------------------------- ci/run-docker.sh | 47 ------------------------- ci/run-test-slice.sh | 5 --- ci/util/extract-trash-dirs.sh | 50 -------------------------- 7 files changed, 2 insertions(+), 184 deletions(-) delete mode 100755 ci/run-docker-build.sh delete mode 100755 ci/run-docker.sh delete mode 100755 ci/util/extract-trash-dirs.sh diff --git a/ci/lib.sh b/ci/lib.sh index d2a7c33f536..c3c06d66862 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -106,9 +106,6 @@ then CI_COMMIT="$BUILD_SOURCEVERSION" CI_JOB_ID="$BUILD_BUILDID" CI_JOB_NUMBER="$BUILD_BUILDNUMBER" - CI_OS_NAME="$(echo "$AGENT_OS" | tr A-Z a-z)" - test darwin != "$CI_OS_NAME" || CI_OS_NAME=osx - CI_REPO_SLUG="$(expr "$BUILD_REPOSITORY_URI" : '.*/\([^/]*/[^/]*\)$')" CC="${CC:-gcc}" # use a subdirectory of the cache dir (because the file share is shared @@ -118,16 +115,13 @@ then export GIT_PROVE_OPTS="--timer --jobs 10 --state=failed,slow,save" export GIT_TEST_OPTS="--verbose-log -x --write-junit-xml" MAKEFLAGS="$MAKEFLAGS --jobs=10" - test windows_nt != "$CI_OS_NAME" || + test Windows_NT != "$AGENT_OS" || GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS" elif test true = "$GITHUB_ACTIONS" then CI_TYPE=github-actions CI_BRANCH="$GITHUB_REF" CI_COMMIT="$GITHUB_SHA" - CI_OS_NAME="$(echo "$RUNNER_OS" | tr A-Z a-z)" - test macos != "$CI_OS_NAME" || CI_OS_NAME=osx - CI_REPO_SLUG="$GITHUB_REPOSITORY" CI_JOB_ID="$GITHUB_RUN_ID" CC="${CC:-gcc}" DONT_SKIP_TAGS=t @@ -137,7 +131,7 @@ then export GIT_PROVE_OPTS="--timer --jobs 10" export GIT_TEST_OPTS="--verbose-log -x" MAKEFLAGS="$MAKEFLAGS --jobs=10" - test windows != "$CI_OS_NAME" || + test Windows != "$RUNNER_OS" || GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS" else echo "Could not identify CI type" >&2 diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh index 57277eefcd0..00fcf27b1b2 100755 --- a/ci/print-test-failures.sh +++ b/ci/print-test-failures.sh @@ -77,9 +77,6 @@ do fi combined_trash_size=$new_combined_trash_size - # DO NOT modify these two 'echo'-ed strings below - # without updating 'ci/util/extract-trash-dirs.sh' - # as well. echo "$(tput setaf 1)Start of trash directory of '$test_name':$(tput sgr0)" cat "$trash_tgz_b64" echo "$(tput setaf 1)End of trash directory of '$test_name'$(tput sgr0)" diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh index 280dda7d285..0a1ec8c2bae 100755 --- a/ci/run-build-and-tests.sh +++ b/ci/run-build-and-tests.sh @@ -5,11 +5,6 @@ . ${0%/*}/lib.sh -case "$CI_OS_NAME" in -windows*) cmd //c mklink //j t\\.prove "$(cygpath -aw "$cache_dir/.prove")";; -*) ln -s "$cache_dir/.prove" t/.prove;; -esac - export MAKE_TARGETS="all test" case "$jobname" in diff --git a/ci/run-docker-build.sh b/ci/run-docker-build.sh deleted file mode 100755 index 6cd832efb9c..00000000000 --- a/ci/run-docker-build.sh +++ /dev/null @@ -1,66 +0,0 @@ -#!/bin/sh -# -# Build and test Git inside container -# -# Usage: -# run-docker-build.sh -# - -set -ex - -if test $# -ne 1 || test -z "$1" -then - echo >&2 "usage: run-docker-build.sh " - exit 1 -fi - -case "$jobname" in -linux32) - switch_cmd="linux32 --32bit i386" - ;; -linux-musl) - switch_cmd= - useradd () { adduser -D "$@"; } - ;; -*) - exit 1 - ;; -esac - -"${0%/*}/install-docker-dependencies.sh" - -# If this script runs inside a docker container, then all commands are -# usually executed as root. Consequently, the host user might not be -# able to access the test output files. -# If a non 0 host user id is given, then create a user "ci" with that -# user id to make everything accessible to the host user. -HOST_UID=$1 -if test $HOST_UID -eq 0 -then - # Just in case someone does want to run the test suite as root. - CI_USER=root -else - CI_USER=ci - if test "$(id -u $CI_USER 2>/dev/null)" = $HOST_UID - then - echo "user '$CI_USER' already exists with the requested ID $HOST_UID" - else - useradd -u $HOST_UID $CI_USER - fi -fi - -# Build and test -command $switch_cmd su -m -l $CI_USER -c " - set -ex - export DEVELOPER='$DEVELOPER' - export DEFAULT_TEST_TARGET='$DEFAULT_TEST_TARGET' - export GIT_PROVE_OPTS='$GIT_PROVE_OPTS' - export GIT_TEST_OPTS='$GIT_TEST_OPTS' - export GIT_TEST_CLONE_2GB='$GIT_TEST_CLONE_2GB' - export MAKEFLAGS='$MAKEFLAGS' - export cache_dir='$cache_dir' - cd /usr/src/git - test -n '$cache_dir' && ln -s '$cache_dir/.prove' t/.prove - make - make test -" diff --git a/ci/run-docker.sh b/ci/run-docker.sh deleted file mode 100755 index af89d1624a4..00000000000 --- a/ci/run-docker.sh +++ /dev/null @@ -1,47 +0,0 @@ -#!/bin/sh -# -# Download and run Docker image to build and test Git -# - -. ${0%/*}/lib.sh - -case "$jobname" in -linux32) - CI_CONTAINER="daald/ubuntu32:xenial" - ;; -linux-musl) - CI_CONTAINER=alpine - ;; -*) - exit 1 - ;; -esac - -docker pull "$CI_CONTAINER" - -# Use the following command to debug the docker build locally: -# must be 0 if podman is used as drop-in replacement for docker -# $ docker run -itv "${PWD}:/usr/src/git" --entrypoint /bin/sh "$CI_CONTAINER" -# root@container:/# export jobname= -# root@container:/# /usr/src/git/ci/run-docker-build.sh - -container_cache_dir=/tmp/container-cache - -docker run \ - --interactive \ - --env DEVELOPER \ - --env DEFAULT_TEST_TARGET \ - --env GIT_PROVE_OPTS \ - --env GIT_TEST_OPTS \ - --env GIT_TEST_CLONE_2GB \ - --env MAKEFLAGS \ - --env jobname \ - --env cache_dir="$container_cache_dir" \ - --volume "${PWD}:/usr/src/git" \ - --volume "$cache_dir:$container_cache_dir" \ - "$CI_CONTAINER" \ - /usr/src/git/ci/run-docker-build.sh $(id -u $USER) - -check_unignored_build_artifacts - -save_good_tree diff --git a/ci/run-test-slice.sh b/ci/run-test-slice.sh index f8c2c3106a2..b9a682b4bcd 100755 --- a/ci/run-test-slice.sh +++ b/ci/run-test-slice.sh @@ -5,11 +5,6 @@ . ${0%/*}/lib.sh -case "$CI_OS_NAME" in -windows*) cmd //c mklink //j t\\.prove "$(cygpath -aw "$cache_dir/.prove")";; -*) ln -s "$cache_dir/.prove" t/.prove;; -esac - make --quiet -C t T="$(cd t && ./helper/test-tool path-utils slice-tests "$1" "$2" t[0-9]*.sh | tr '\n' ' ')" diff --git a/ci/util/extract-trash-dirs.sh b/ci/util/extract-trash-dirs.sh deleted file mode 100755 index 8e67bec21a2..00000000000 --- a/ci/util/extract-trash-dirs.sh +++ /dev/null @@ -1,50 +0,0 @@ -#!/bin/sh - -error () { - echo >&2 "error: $@" - exit 1 -} - -find_embedded_trash () { - while read -r line - do - case "$line" in - *Start\ of\ trash\ directory\ of\ \'t[0-9][0-9][0-9][0-9]-*\':*) - test_name="${line#*\'}" - test_name="${test_name%\'*}" - - return 0 - esac - done - - return 1 -} - -extract_embedded_trash () { - while read -r line - do - case "$line" in - *End\ of\ trash\ directory\ of\ \'$test_name\'*) - return - ;; - *) - printf '%s\n' "$line" - ;; - esac - done - - error "unexpected end of input" -} - -# Raw logs from Linux build jobs have CRLF line endings, while OSX -# build jobs mostly have CRCRLF, except an odd line every now and -# then that has CRCRCRLF. 'base64 -d' from 'coreutils' doesn't like -# CRs and complains about "invalid input", so remove all CRs at the -# end of lines. -sed -e 's/\r*$//' | \ -while find_embedded_trash -do - echo "Extracting trash directory of '$test_name'" - - extract_embedded_trash |base64 -d |tar xzp -done From patchwork Mon Feb 21 14:46:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12753710 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 07CB5C433F5 for ; Mon, 21 Feb 2022 14:47:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378323AbiBUOre (ORCPT ); Mon, 21 Feb 2022 09:47:34 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:59474 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1378298AbiBUOrQ (ORCPT ); Mon, 21 Feb 2022 09:47:16 -0500 Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com [IPv6:2a00:1450:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D0DBD1122 for ; Mon, 21 Feb 2022 06:46:52 -0800 (PST) Received: by mail-wr1-x42d.google.com with SMTP id m27so15711633wrb.4 for ; Mon, 21 Feb 2022 06:46:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=kGnWKoJdyrnKKTEOcUKcWHEHzPOLi+v3luyaskEh4T0=; b=LBHNWUEN1SMY6QOQUkcrmYoOj3/wihCV/EbUllY08JfSHGrcvL/VqRpmh8NGAqj/F9 mMEJtFyMSqXNuOJLQwr/w6Y8Z7+egUne2MinKajW1QgofUSu30efbr+4aciJtqcQWe9a A0j+G3+06pfoj3fqKnoiiihV5yL1UbpBEO9DV4QHMQG/fqn65yG2ZFxMqFe6U3bTa+Ek +Fhjs4WPHinpKJdp//naUfY8a915x0ADND12ZiwymtDF4ZKjxUpEA7u+cryHijSjoX7L tcndonrYxgUSOA8pmR8giZiDg4stSNqxZlmLNC2GOEZ66OGAKMYH9yX5lG7/kguvjJvt 1Ufw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=kGnWKoJdyrnKKTEOcUKcWHEHzPOLi+v3luyaskEh4T0=; b=i8cseG1LxRBhVsYxaLJQzP6ISCA7ugHtZ0jvD/ypmbZceseQh6NThZdKXgeIjLB8dR +cDv2Ih8H4CtQ97eP4IWYegLCf1ybl6z0fnfFxV11ZM7T84jdXd7RNABuMmzqKfv70+w qShUc/WjCjqa0L+bfKzzAfb0gES2GDotnj7i6WA8XukJ4b3XCNZCYD4lHvbBTo+dggH7 hOiSw7AmmXmuUPeOamc7hihhXnGTW5UeuNsiJpIzFLhVCPYGWArGQdcTJWZCX0hzOZW+ XvnUkekubxqm9KqRnXk+u3Iv3GtcN1J/ebPk1mGCOUb4vpXc9k5bj5/kiwIYkKL8Rjh7 4gxw== X-Gm-Message-State: AOAM531O5K6sC+t508Wqpc1hm5vt7kd6S88HEZgpiemlZ8IxR8xQ8EO6 CKolT6mwk4Oacc9jYOjN1fR/bhoyUJVE9w== X-Google-Smtp-Source: ABdhPJysji3K/k7oG+8lAZaTqZEio/2yYmM7k9/Lmtt1ruLAtITUUYE0pYZVukQasqAcyr5B1ZW6Xg== X-Received: by 2002:a5d:5045:0:b0:1e3:3d02:84b3 with SMTP id h5-20020a5d5045000000b001e33d0284b3mr15509359wrt.384.1645454811080; Mon, 21 Feb 2022 06:46:51 -0800 (PST) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id r14sm25071094wrz.84.2022.02.21.06.46.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Feb 2022 06:46:50 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH 04/25] CI: remove dead "tree skipping" code Date: Mon, 21 Feb 2022 15:46:16 +0100 Message-Id: X-Mailer: git-send-email 2.35.1.1132.ga1fe46f8690 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Remove the code related to skipping CI runs if the tree was successfully tested before. With the GitHub CI this is done with the "ci-config" step, see e76eec35540 (ci: allow per-branch config for GitHub Actions, 2020-05-07) and 7d78d5fc1a9 (ci: skip GitHub workflow runs for already-tested commits/trees, 2020-10-08). This code hasn't been used since 4a6e4b96026 (CI: remove Travis CI support, 2021-11-23), and before that for the retired Azure pipeline support removed in 6081d3898fe (ci: retire the Azure Pipelines definition, 2020-04-11). This change is needed because this and subsequent commits are turning "ci/lib.sh" into a dumber library that'll only be tasked with setting variables for CI jobs, or for specific steps within those jobs. Now we don't need to worry about it potentially skipping the run on its own. This change also removes a subtle potential logic error introduced in 0e7696c64db (ci: disallow directional formatting, 2021-11-04). The "ci/check-directional-formatting.bash" script would have been made to run after the "save_good_tree" in invoked in "ci/run-static-analysis.sh". So if e.g. the "make coccicheck" failed we'd still mark the tree as good. I.e. the addition of "ci/check-directional-formatting.bash" didn't take into account that the various users of the "save_good_tree" function in ci/*.sh made a hard assumption that they're the only ci/*.sh script that's being run (and they use "set -e"). This is entirely academic since we weren't actually running this code, but is something to be careful of if anyone ever needs to resurrect parts of this. Finally, the cache_dir="$HOME/none" added in a3f2eec862b (ci/lib: allow running in GitHub Actions, 2020-04-08) for the GitHub CI can be removed. It appears to have been added to appease the subsequent 'mkdir -p "$cache_dir"', which wasn't needed in GitHub CI. 1. https://lore.kernel.org/git/patch-1.1-eec0a8c3164-20211217T000418Z-avarab@gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason --- ci/lib.sh | 91 --------------------------------------- ci/run-build-and-tests.sh | 2 - ci/run-static-analysis.sh | 2 - ci/test-documentation.sh | 2 - 4 files changed, 97 deletions(-) diff --git a/ci/lib.sh b/ci/lib.sh index c3c06d66862..5a8938d4b3c 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -13,75 +13,6 @@ then exit 1 fi -skip_branch_tip_with_tag () { - # Sometimes, a branch is pushed at the same time the tag that points - # at the same commit as the tip of the branch is pushed, and building - # both at the same time is a waste. - # - # When the build is triggered by a push to a tag, $CI_BRANCH will - # have that tagname, e.g. v2.14.0. Let's see if $CI_BRANCH is - # exactly at a tag, and if so, if it is different from $CI_BRANCH. - # That way, we can tell if we are building the tip of a branch that - # is tagged and we can skip the build because we won't be skipping a - # build of a tag. - - if TAG=$(git describe --exact-match "$CI_BRANCH" 2>/dev/null) && - test "$TAG" != "$CI_BRANCH" - then - echo "$(tput setaf 2)Tip of $CI_BRANCH is exactly at $TAG$(tput sgr0)" - exit 0 - fi -} - -# Save some info about the current commit's tree, so we can skip the build -# job if we encounter the same tree again and can provide a useful info -# message. -save_good_tree () { - echo "$(git rev-parse $CI_COMMIT^{tree}) $CI_COMMIT $CI_JOB_NUMBER $CI_JOB_ID" >>"$good_trees_file" - # limit the file size - tail -1000 "$good_trees_file" >"$good_trees_file".tmp - mv "$good_trees_file".tmp "$good_trees_file" -} - -# Skip the build job if the same tree has already been built and tested -# successfully before (e.g. because the branch got rebased, changing only -# the commit messages). -skip_good_tree () { - if test true = "$GITHUB_ACTIONS" - then - return - fi - - if ! good_tree_info="$(grep "^$(git rev-parse $CI_COMMIT^{tree}) " "$good_trees_file")" - then - # Haven't seen this tree yet, or no cached good trees file yet. - # Continue the build job. - return - fi - - echo "$good_tree_info" | { - read tree prev_good_commit prev_good_job_number prev_good_job_id - - if test "$CI_JOB_ID" = "$prev_good_job_id" - then - cat <<-EOF - $(tput setaf 2)Skipping build job for commit $CI_COMMIT.$(tput sgr0) - This commit has already been built and tested successfully by this build job. - To force a re-build delete the branch's cache and then hit 'Restart job'. - EOF - else - cat <<-EOF - $(tput setaf 2)Skipping build job for commit $CI_COMMIT.$(tput sgr0) - This commit's tree has already been built and tested successfully in build job $prev_good_job_number for commit $prev_good_commit. - The log of that build job is available at $SYSTEM_TASKDEFINITIONSURI$SYSTEM_TEAMPROJECT/_build/results?buildId=$prev_good_job_id - To force a re-build delete the branch's cache and then hit 'Restart job'. - EOF - fi - } - - exit 0 -} - check_unignored_build_artifacts () { ! git ls-files --other --exclude-standard --error-unmatch \ @@ -102,16 +33,8 @@ if test -n "$SYSTEM_COLLECTIONURI" || test -n "$SYSTEM_TASKDEFINITIONSURI" then CI_TYPE=azure-pipelines # We are running in Azure Pipelines - CI_BRANCH="$BUILD_SOURCEBRANCH" - CI_COMMIT="$BUILD_SOURCEVERSION" - CI_JOB_ID="$BUILD_BUILDID" - CI_JOB_NUMBER="$BUILD_BUILDNUMBER" CC="${CC:-gcc}" - # use a subdirectory of the cache dir (because the file share is shared - # among *all* phases) - cache_dir="$HOME/test-cache/$SYSTEM_PHASENAME" - export GIT_PROVE_OPTS="--timer --jobs 10 --state=failed,slow,save" export GIT_TEST_OPTS="--verbose-log -x --write-junit-xml" MAKEFLAGS="$MAKEFLAGS --jobs=10" @@ -120,13 +43,7 @@ then elif test true = "$GITHUB_ACTIONS" then CI_TYPE=github-actions - CI_BRANCH="$GITHUB_REF" - CI_COMMIT="$GITHUB_SHA" - CI_JOB_ID="$GITHUB_RUN_ID" CC="${CC:-gcc}" - DONT_SKIP_TAGS=t - - cache_dir="$HOME/none" export GIT_PROVE_OPTS="--timer --jobs 10" export GIT_TEST_OPTS="--verbose-log -x" @@ -139,14 +56,6 @@ else exit 1 fi -good_trees_file="$cache_dir/good-trees" - -mkdir -p "$cache_dir" - -test -n "${DONT_SKIP_TAGS-}" || -skip_branch_tip_with_tag -skip_good_tree - export DEVELOPER=1 export DEFAULT_TEST_TARGET=prove export GIT_TEST_CLONE_2GB=true diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh index 0a1ec8c2bae..e5a21985b44 100755 --- a/ci/run-build-and-tests.sh +++ b/ci/run-build-and-tests.sh @@ -45,5 +45,3 @@ esac # start running tests. make $MAKE_TARGETS check_unignored_build_artifacts - -save_good_tree diff --git a/ci/run-static-analysis.sh b/ci/run-static-analysis.sh index 65bcebda41a..5a87b1cac96 100755 --- a/ci/run-static-analysis.sh +++ b/ci/run-static-analysis.sh @@ -28,5 +28,3 @@ fi make hdr-check || exit 1 - -save_good_tree diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh index de41888430a..9e0652c30dd 100755 --- a/ci/test-documentation.sh +++ b/ci/test-documentation.sh @@ -41,5 +41,3 @@ grep ' X-Patchwork-Id: 12753714 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 34E79C433FE for ; Mon, 21 Feb 2022 14:47:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378332AbiBUOrt (ORCPT ); Mon, 21 Feb 2022 09:47:49 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:59520 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1378299AbiBUOrR (ORCPT ); Mon, 21 Feb 2022 09:47:17 -0500 Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E950838A5 for ; Mon, 21 Feb 2022 06:46:53 -0800 (PST) Received: by mail-wm1-x334.google.com with SMTP id j9-20020a05600c190900b0037bff8a24ebso13660131wmq.4 for ; Mon, 21 Feb 2022 06:46:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=f49KGz3e37xjRKjzAr5CLd2/qJwdIq3VCIRWvRn/2wU=; b=Q5zdopKMihAxle3hCZL1FinGprHPBvvW+wiWQKwpmqUYbfOEynIjBF96MimCTetYXC 4t6ZsPfx4qzjVSY+O88OULdYlYl+olQN30HFgJugozowcI/bdqwHnewqZNoUhxgcpzjB NXJe+Pn2S91Trcph1uK6PwIbBGOSF+I8kop3CgNXSeJMXsp5ZsjWGeKZkXOqXqi6FHT9 RRs/X60VwJ4TMW99EtzBhuxPXn8dAv3onFvHkzh29ZVl2eWB6q1HzsVlSECd/Mjbd+JN 9SCUaw1rlUH+WZeF3TCazGK4f6KNjjlSEeqnIp5egZP/8/qhfYr+/OaEzN/qcSAIpY1y wm6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=f49KGz3e37xjRKjzAr5CLd2/qJwdIq3VCIRWvRn/2wU=; b=0GxVXJ6jcwUq5RsVExXGRSHG0TrcZA5hLJaHXziPiDQk1qnvv9rRPHT1ve2nrPJqY8 7Y/CwjjPMytu1Qm8xFmwXg9KaIbArO6VXWP5Z/D2Um40pUeTA39kiTBGnqongRg+wnbg 9bgYwKSaFnGkKkcWEUCF8u7wRhzxReME81ompUo7y1AfqPzCnZKodkRzcmwAWPYhxT89 24H2PZg/NOGVjpJuNT9GKJ0oxwUZkJn2khduo6PvU+gZ3uIxiDvvKHn/qwWIDTSlClai Veu2XAj6Mxu0jQ2ThiFwsQe0K5ocTRK0tLKufHIkVZNt4YmRsmFJqwBc+EzNVinXdyZB /9EQ== X-Gm-Message-State: AOAM532eMi0IP78RAxzF4qlrkQDHWOnrNHlAj85lq8rZ1xvvVhCqKpaF NqQRDUZZtc7IjonwkffIbAKBNz/eDwD4ug== X-Google-Smtp-Source: ABdhPJxejbn7Hpm3cuLKFbJSEr/z+Bk8oGSV/SLkYCuaq/l+XZH0OQfNugYEenZxj9vVjd1PU+JTnw== X-Received: by 2002:a05:600c:230d:b0:37d:5882:ec9b with SMTP id 13-20020a05600c230d00b0037d5882ec9bmr21426038wmo.162.1645454812114; Mon, 21 Feb 2022 06:46:52 -0800 (PST) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id r14sm25071094wrz.84.2022.02.21.06.46.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Feb 2022 06:46:51 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH 05/25] CI: remove unused Azure ci/* code Date: Mon, 21 Feb 2022 15:46:17 +0100 Message-Id: X-Mailer: git-send-email 2.35.1.1132.ga1fe46f8690 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Remove Azure-specific code that's been unused since 6081d3898fe (ci: retire the Azure Pipelines definition, 2020-04-11). As noted in a larger removal of all of the Azure-supporting code in [1] (although that missed some of this) there is more of it in-tree, but let's focus on only the ci/* code for now. This is needed because in subsequent commits this unused code would either need to be changed to accommodate changes in ci/*, or be removed. As we'll see in those subsequent commits the end-state we're heading towards will actually make it easier to add new CI types in the future, even though the only one we're left with now is the GitHub CI. I.e. the "ci/*" framework will be made to do less, not more. We'll be offloading more of what it does to our generic build and test system. While I'm at it (since the line needs to be touched anyway) change an odd 'if test true == "$GITHUB_ACTIONS"' to the more usual style used in other places of 'if test "$GITHUB_ACTIONS" = "true"'. 1. https://lore.kernel.org/git/patch-1.1-eec0a8c3164-20211217T000418Z-avarab@gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason --- ci/install-dependencies.sh | 3 --- ci/lib.sh | 13 +------------ ci/mount-fileshare.sh | 25 ------------------------- ci/print-test-failures.sh | 7 +------ 4 files changed, 2 insertions(+), 46 deletions(-) delete mode 100755 ci/mount-fileshare.sh diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh index dbcebad2fb2..e7ea8799411 100755 --- a/ci/install-dependencies.sh +++ b/ci/install-dependencies.sh @@ -34,8 +34,6 @@ macos-latest) export HOMEBREW_NO_AUTO_UPDATE=1 HOMEBREW_NO_INSTALL_CLEANUP=1 # Uncomment this if you want to run perf tests: # brew install gnu-time - test -z "$BREW_INSTALL_PACKAGES" || - brew install $BREW_INSTALL_PACKAGES brew link --force gettext brew install --cask --no-quarantine perforce || { # Update the definitions and try again @@ -69,7 +67,6 @@ Documentation) sudo apt-get -q update sudo apt-get -q -y install asciidoc xmlto docbook-xsl-ns make - test -n "$ALREADY_HAVE_ASCIIDOCTOR" || sudo gem install --version 1.5.8 asciidoctor ;; linux-gcc-default) diff --git a/ci/lib.sh b/ci/lib.sh index 5a8938d4b3c..ad1aa59a384 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -29,18 +29,7 @@ export TERM=${TERM:-dumb} # Clear MAKEFLAGS that may come from the outside world. export MAKEFLAGS= -if test -n "$SYSTEM_COLLECTIONURI" || test -n "$SYSTEM_TASKDEFINITIONSURI" -then - CI_TYPE=azure-pipelines - # We are running in Azure Pipelines - CC="${CC:-gcc}" - - export GIT_PROVE_OPTS="--timer --jobs 10 --state=failed,slow,save" - export GIT_TEST_OPTS="--verbose-log -x --write-junit-xml" - MAKEFLAGS="$MAKEFLAGS --jobs=10" - test Windows_NT != "$AGENT_OS" || - GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS" -elif test true = "$GITHUB_ACTIONS" +if test "$GITHUB_ACTIONS" = "true" then CI_TYPE=github-actions CC="${CC:-gcc}" diff --git a/ci/mount-fileshare.sh b/ci/mount-fileshare.sh deleted file mode 100755 index 26b58a80960..00000000000 --- a/ci/mount-fileshare.sh +++ /dev/null @@ -1,25 +0,0 @@ -#!/bin/sh - -die () { - echo "$*" >&2 - exit 1 -} - -test $# = 4 || -die "Usage: $0 " - -mkdir -p "$4" || die "Could not create $4" - -case "$(uname -s)" in -Linux) - sudo mount -t cifs -o vers=3.0,username="$2",password="$3",dir_mode=0777,file_mode=0777,serverino "$1" "$4" - ;; -Darwin) - pass="$(echo "$3" | sed -e 's/\//%2F/g' -e 's/+/%2B/g')" && - mount -t smbfs,soft "smb://$2:$pass@${1#//}" "$4" - ;; -*) - die "No support for $(uname -s)" - ;; -esac || -die "Could not mount $4" diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh index 00fcf27b1b2..0c63b6f7962 100755 --- a/ci/print-test-failures.sh +++ b/ci/print-test-failures.sh @@ -38,14 +38,9 @@ do test_name="${TEST_EXIT%.exit}" test_name="${test_name##*/}" trash_dir="trash directory.$test_name" + mkdir -p failed-test-artifacts case "$CI_TYPE" in - azure-pipelines) - mkdir -p failed-test-artifacts - mv "$trash_dir" failed-test-artifacts - continue - ;; github-actions) - mkdir -p failed-test-artifacts echo "FAILED_TEST_ARTIFACTS=t/failed-test-artifacts" >>$GITHUB_ENV cp "${TEST_EXIT%.exit}.out" failed-test-artifacts/ tar czf failed-test-artifacts/"$test_name".trash.tar.gz "$trash_dir" From patchwork Mon Feb 21 14:46:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12753712 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC9B6C433F5 for ; Mon, 21 Feb 2022 14:47:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378327AbiBUOrk (ORCPT ); Mon, 21 Feb 2022 09:47:40 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:59550 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1378300AbiBUOrR (ORCPT ); Mon, 21 Feb 2022 09:47:17 -0500 Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A692F5596 for ; Mon, 21 Feb 2022 06:46:54 -0800 (PST) Received: by mail-wr1-x432.google.com with SMTP id d3so11853219wrf.1 for ; Mon, 21 Feb 2022 06:46:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=74RcGAtf/M5wljt+DWF/2V0kyvyrPbZpV5I45HW//EQ=; b=nKlVhZvYpZczDpyefRrWy10r0gzJ4GPwu8fEv1dtyav58Vie3gyr9KATb+/YPEgnDg JG7slw7zBzlGu75dkRJNA7aV+6akhyZEp0Yuupwnu+1OS5FmaY5zav7WJubwdzx0zjcR +b6CFk2eKYiYUiIHy9aSkEqd1RhgOru53X7zxkR6ITvfRALQACycU91U4zOf42ZhlzlG peIaZ13sIMj5Nd7aO1ot33XBp0IfFsc2RtC897RaYIRwBMP5zsbmfxYJhPfpHa/z3T67 Ryfg36ir+XdDfZ/tnam2D5jlZm4I3nde5+xWXqb7wvKTrAIPWKVdVmHXLcC2cDf3GC/I 2mug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=74RcGAtf/M5wljt+DWF/2V0kyvyrPbZpV5I45HW//EQ=; b=aRRGiEp101e551PkcP2pCJUlM8B7xxbqUjxRvz44u5iiJAoE1E5vTY6hWVAQ+JQKQG uAbsMQTBScBvEHTAIv94TXTaMooBEnTC73f/fDDU1ICVDWiTe1aX+r+BpPyVR5Yv74Ui D76saEJn5BG9PMws7pnK/P7HFFv2ASUrX/2iXbrXv1AvgmnJdm+PQz4DRN8vg+n8kPjJ mFYh9IE3XzxHqgZLU9vFJSj78c0nC8yqNOYKeEVjMfKK5XamEtIsrTdS1L8UH00Dbbn0 +KRyqo/umyx2xd2gdkTUq7bOd7LI3SF38Sn7f7aTyUdD//h3lVZHmNfpVQvBbJMEJNe7 REKw== X-Gm-Message-State: AOAM5337jNs8hfGKytZo2tJZBvToEoYdeXtb5ZCPFxcrHLX+2869owxL hCW63CuiWBNtAxcQcV0TDioFO7rEQrMLQg== X-Google-Smtp-Source: ABdhPJxZL0uiHEwrH2fu93Do+D1Zpw4o0hHA8UvkriFT1cz1BkeuC4L2KLAsHsQlrOzVsHwpvpJ8iw== X-Received: by 2002:adf:fa0d:0:b0:1e3:f7b0:1037 with SMTP id m13-20020adffa0d000000b001e3f7b01037mr16323813wrr.188.1645454812990; Mon, 21 Feb 2022 06:46:52 -0800 (PST) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id r14sm25071094wrz.84.2022.02.21.06.46.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Feb 2022 06:46:52 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH 06/25] CI: don't have "git grep" invoke a pager in tree content check Date: Mon, 21 Feb 2022 15:46:18 +0100 Message-Id: X-Mailer: git-send-email 2.35.1.1132.ga1fe46f8690 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Fix the omission of not adding "-P" (a.k.a. "--no-pager") to the "git" invocation that spaws "git grep", added in 0e7696c64db (ci: disallow directional formatting, 2021-11-04). This didn't matter for the GitHub CI, but does when running the check locally. A subsequent commit will start invoking it via a new check from the top-level Makefile, and without this we'd visibly spawn a pager when doing so. Signed-off-by: Ævar Arnfjörð Bjarmason --- ci/check-directional-formatting.bash | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ci/check-directional-formatting.bash b/ci/check-directional-formatting.bash index e6211b141aa..de8e6e486f0 100755 --- a/ci/check-directional-formatting.bash +++ b/ci/check-directional-formatting.bash @@ -23,5 +23,5 @@ # U+2066..U+2069: LRI, RLI, FSI and PDI regex='(\u202a|\u202b|\u202c|\u202d|\u202e|\u2066|\u2067|\u2068|\u2069)' -! LC_CTYPE=C git grep -El "$(LC_CTYPE=C.UTF-8 printf "$regex")" \ +! LC_CTYPE=C git -P grep -El "$(LC_CTYPE=C.UTF-8 printf "$regex")" \ -- ':(exclude,attr:binary)' ':(exclude)*.po' From patchwork Mon Feb 21 14:46:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12753713 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 86EF1C433EF for ; Mon, 21 Feb 2022 14:47:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378329AbiBUOro (ORCPT ); Mon, 21 Feb 2022 09:47:44 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:60418 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242285AbiBUOr3 (ORCPT ); Mon, 21 Feb 2022 09:47:29 -0500 Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com [IPv6:2a00:1450:4864:20::42b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9C7795FA8 for ; Mon, 21 Feb 2022 06:46:55 -0800 (PST) Received: by mail-wr1-x42b.google.com with SMTP id j17so818240wrc.0 for ; Mon, 21 Feb 2022 06:46:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=lqYChkBLuiLY4LNO1CIj1mE6XKgBlp82KYad2s48vsg=; b=plMYYHeuuv6BhqR9uvTcig1mFMQP9WfhJOpDxXGOwTc+keVd/kaX05639ZEanJt/OB yIePGr6qMZR+0BAaCb3pNu/7yiRNO34q63p0i2LRGJi2HIcDz4/VtyzaQyJrJ2yLU3T7 ffzOHIlXLTXNR7N5m0BE1Ij1i985qykOV+jjtb4n5WIra+ORt+vMnCvKBL4XyR8gNW84 9dgUTcVpspWl8ILtcsEpPuSpTsEP4+iQijxExRR0wA7v07QPqcCiqFZ970288/49CERP OxcgYYITBwbgtecO/TzaQJx1yZCW/8ZVaJyxAlIrbbG98v8Im0jrgD8TVYed58DlkEKH TGyw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=lqYChkBLuiLY4LNO1CIj1mE6XKgBlp82KYad2s48vsg=; b=xvtyu7vuivhd9F0oRVMyd+5mPuLrvjDxjKNzrmXGQkkUJcFAnCqf9JO0f9+EaQU+Hb 7zVZKRjixB48/GP0bOvDG282NTMq8NBfIyEPEDtBrbBxe7eGxlUyAwd/xmFzqJ3tSXJV NqHWpBiYijUd81Qn0njXpwYcw+gaQddtLEoq3g6LSFO5xUaayCe+Brz2S8+a3BXXIxWU RyNxCG7veNSPBmGMV79EQVVRz0iZ3kPRs3E9dDpoxIbg7wunFnmv/du+pyKGj1v2dOni wVEB26bvoXxSvU3mgi7lLOqMn2pf4py5HdLcs3MUHEMavrvaMqStzkRY2LZHV3V8FahZ huUQ== X-Gm-Message-State: AOAM531bXInKnSU0VeCyB+6bbSIOdO9dcVNdBgLkbi3u2nZY7mcuwyNV KJ2DDb9uaEUvKUdZhgVTb0vFo+Vi2o3HgQ== X-Google-Smtp-Source: ABdhPJxRPvRL07JcqICqMCcAHoNx8o6MtR99wtNgb1s4ugsF1ZtIjXEbBKd6vxFZ+ut+i2Z94gTXgA== X-Received: by 2002:adf:dc90:0:b0:1e4:a5fe:9b42 with SMTP id r16-20020adfdc90000000b001e4a5fe9b42mr15977352wrj.98.1645454813885; Mon, 21 Feb 2022 06:46:53 -0800 (PST) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id r14sm25071094wrz.84.2022.02.21.06.46.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Feb 2022 06:46:53 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH 07/25] CI: have "static-analysis" run a "make ci-static-analysis" target Date: Mon, 21 Feb 2022 15:46:19 +0100 Message-Id: X-Mailer: git-send-email 2.35.1.1132.ga1fe46f8690 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Make it easier to run the CI logic locally by creating "ci-static-analysis" target. Now the "check-coccicheck" logic added in 0860a7641ba (travis-ci: fail if Coccinelle static analysis found something to transform, 2018-07-23) can be easily tested outside of CI, and likewise the sanity check added in 0e7696c64db (ci: disallow directional formatting, 2021-11-04). These targets could be improved, the "hdr-check" target will re-do all of its work every time it's run, and could with a change similar to my c234e8a0ecf (Makefile: make the "sparse" target non-.PHONY, 2021-09-23). The same goes for the new "ci-check-directional-formatting" target. But without those improvements being able to easily run a 1=1 equivalent of the checks we do in CI during a local build is already an improvement. Subsequent commits will expand on this ability. This change will also make the CI check faster, since we can take advantage of parallelism. The "make coccicheck" takes a very long time, and at the tail end we might only have one job pegging 100% of one CPU core for a long time. Now any extra cores will be free to run the rest of the targets under "ci-static-analysis". Signed-off-by: Ævar Arnfjörð Bjarmason --- .github/workflows/main.yml | 3 +-- Makefile | 29 +++++++++++++++++++++++++++++ ci/run-static-analysis.sh | 30 ------------------------------ 3 files changed, 30 insertions(+), 32 deletions(-) delete mode 100755 ci/run-static-analysis.sh diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 309b9141249..ad752010102 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -317,8 +317,7 @@ jobs: steps: - uses: actions/checkout@v2 - run: ci/install-dependencies.sh - - run: ci/run-static-analysis.sh - - run: ci/check-directional-formatting.bash + - run: make ci-static-analysis sparse: needs: ci-config if: needs.ci-config.outputs.enabled == 'yes' diff --git a/Makefile b/Makefile index 6f0b4b775fe..02cd563e216 100644 --- a/Makefile +++ b/Makefile @@ -2005,6 +2005,7 @@ ifndef V QUIET_HDR = @echo ' ' HDR $(<:hcc=h); QUIET_RC = @echo ' ' RC $@; QUIET_SPATCH = @echo ' ' SPATCH $<; + QUIET_CHECK = @echo ' ' CHECK $@; QUIET_SUBDIR0 = +@subdir= QUIET_SUBDIR1 = ;$(NO_SUBDIR) echo ' ' SUBDIR $$subdir; \ $(MAKE) $(PRINT_DIR) -C $$subdir @@ -3052,6 +3053,20 @@ coccicheck: $(addsuffix .patch,$(filter-out %.pending.cocci,$(wildcard contrib/c # See contrib/coccinelle/README coccicheck-pending: $(addsuffix .patch,$(wildcard contrib/coccinelle/*.pending.cocci)) +.PHONY: check-coccicheck +check-coccicheck: coccicheck + $(QUIET_CHECK)for cocci_patch in contrib/coccinelle/*.patch; do \ + if test -s "$$cocci_patch"; then \ + echo "Coccinelle suggests the following changes in '$$cocci_patch':"; \ + cat "$$cocci_patch"; \ + fail=UnfortunatelyYes; \ + fi \ + done; \ + if test -n "$$fail"; then \ + echo "error: Coccinelle suggested some changes"; \ + exit 1; \ + fi + .PHONY: coccicheck coccicheck-pending ### Installation rules @@ -3487,3 +3502,17 @@ $(FUZZ_PROGRAMS): all $(XDIFF_OBJS) $(EXTLIBS) git.o $@.o $(LIB_FUZZING_ENGINE) -o $@ fuzz-all: $(FUZZ_PROGRAMS) + +### CI "check" targets +# +# These targets are run from the CI, see .github/workflows/main.yml, +# but can also be run manually to run the same assertions locally. + +.PHONY: ci-check-directional-formatting +ci-check-directional-formatting: + $(QUIET_CHECK)ci/check-directional-formatting.bash + +.PHONY: ci-static-analysis +ci-static-analysis: ci-check-directional-formatting +ci-static-analysis: check-coccicheck +ci-static-analysis: hdr-check diff --git a/ci/run-static-analysis.sh b/ci/run-static-analysis.sh deleted file mode 100755 index 5a87b1cac96..00000000000 --- a/ci/run-static-analysis.sh +++ /dev/null @@ -1,30 +0,0 @@ -#!/bin/sh -# -# Perform various static code analysis checks -# - -. ${0%/*}/lib.sh - -make coccicheck - -set +x - -fail= -for cocci_patch in contrib/coccinelle/*.patch -do - if test -s "$cocci_patch" - then - echo "$(tput setaf 1)Coccinelle suggests the following changes in '$cocci_patch':$(tput sgr0)" - cat "$cocci_patch" - fail=UnfortunatelyYes - fi -done - -if test -n "$fail" -then - echo "$(tput setaf 1)error: Coccinelle suggested some changes$(tput sgr0)" - exit 1 -fi - -make hdr-check || -exit 1 From patchwork Mon Feb 21 14:46:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12753711 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0B21EC433F5 for ; Mon, 21 Feb 2022 14:47:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378325AbiBUOri (ORCPT ); Mon, 21 Feb 2022 09:47:38 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:60420 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1378292AbiBUOr3 (ORCPT ); Mon, 21 Feb 2022 09:47:29 -0500 Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8A2A75FC8 for ; Mon, 21 Feb 2022 06:46:56 -0800 (PST) Received: by mail-wr1-x431.google.com with SMTP id u1so27475428wrg.11 for ; Mon, 21 Feb 2022 06:46:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=tD+tWBHdrxVz3ZpPKSudMeRDUxjZwZOPIB8MPEw0wuw=; b=SRIBjh0dLJdXLZbFfYchCK/UmEjxUqn4SKD7gwajMJ/111mGqnSs8FOs7HicD2Kde/ VmeboNoslU+WjS23HKnwfKtwphkgzhcqiPuY7JjYOJKQMZMotrkoLH9DptKGWfX8VVO5 gTAN2mMTi/PdLzIePZHLFRSmLM48s9n917sHl+eqMwVMryovJ+rIfT1uMkkXAq4lBOrB NfdSRP6P7jB96jew6s9mSvdZNeYUaPAe++pOxI8DZZSDQeH7ao581qGtvgHInCdLq7XK avuhlc3oX7XO5fIEnau/WFcfHqa9XuEkicpEH/TnNovC/crJ1i2GKxiducXRJYmPeujh 092w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=tD+tWBHdrxVz3ZpPKSudMeRDUxjZwZOPIB8MPEw0wuw=; b=rEBkWvx4uERU7J+N9QSPgJeXoe3Yxk9vPX1+Mp052n29FTQUCbVnRqJUaWJctyyx+9 KMCryUb+Z9l0V3EC21bCfm+Qdy6vHSZ/Tum7cXP0HzDK6zUhlt7VIByKy6U883oCgRgS OySX0IigV1xO9k+hX6v7nIBrlQ9refizKi1mhTMz8Yvn25xxBw+m3kX7cLFzBRtUQpzR jbJVYsCY5toCsv+/bdnTprjstOBrEir9vPPWAhmFkEqQjiKnEbpymqwMypcRF/8ryxuh 5FyvQZskQUt/s+dGPaGNksSHDE4zm40hu+Ag2ZoOKtzsl4if2SWroVPYgnhUtF/cU9y8 1cvQ== X-Gm-Message-State: AOAM531HTqC6Z34S+Cz9l6gXFg+Wfp4JNdOH1/CS2bmYXg0b5WcuIFD1 qnyDCkpmzUhj2aQj0FSdAVEsueWz0Fggjg== X-Google-Smtp-Source: ABdhPJyD69KzfW7syaZvlmTYSoPMu1+ywvYeb9SxLMIgvRCTRRRt2c3WvE1au9h0mqTIu6wVAHWiKw== X-Received: by 2002:a5d:6145:0:b0:1e3:169c:197c with SMTP id y5-20020a5d6145000000b001e3169c197cmr15787064wrt.611.1645454814915; Mon, 21 Feb 2022 06:46:54 -0800 (PST) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id r14sm25071094wrz.84.2022.02.21.06.46.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Feb 2022 06:46:54 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH 08/25] CI: have "static-analysis" run "check-builtins", not "documentation" Date: Mon, 21 Feb 2022 15:46:20 +0100 Message-Id: X-Mailer: git-send-email 2.35.1.1132.ga1fe46f8690 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Move the "make check-builtins" check from the "documentation" job to the "static-analysis" job. The "check-builtins" target added in c74390e4a1d (cherry is built-in, do not ship git-cherry.sh, 2006-11-05) is unrelated to the documentation, so it's odd that b98712b9aa9 (travis-ci: build documentation, 2016-05-04) added it to the "documentation" job. Let's just move it to the "ci-static-analysis" target, and while we're at it improve the output with $(QUIET_CHECK). Signed-off-by: Ævar Arnfjörð Bjarmason --- Makefile | 3 ++- ci/test-documentation.sh | 1 - 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Makefile b/Makefile index 02cd563e216..0c117d32f48 100644 --- a/Makefile +++ b/Makefile @@ -3424,7 +3424,7 @@ check-docs:: ### Make sure built-ins do not have dups and listed in git.c # check-builtins:: - ./check-builtins.sh + $(QUIET_CHECK)./check-builtins.sh ### Test suite coverage testing # @@ -3514,5 +3514,6 @@ ci-check-directional-formatting: .PHONY: ci-static-analysis ci-static-analysis: ci-check-directional-formatting +ci-static-analysis: check-builtins ci-static-analysis: check-coccicheck ci-static-analysis: hdr-check diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh index 9e0652c30dd..41e2b126311 100755 --- a/ci/test-documentation.sh +++ b/ci/test-documentation.sh @@ -14,7 +14,6 @@ filter_log () { "$1" } -make check-builtins make check-docs # Build docs with AsciiDoc From patchwork Mon Feb 21 14:46:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12753716 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7842BC433FE for ; Mon, 21 Feb 2022 14:47:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378317AbiBUOr5 (ORCPT ); Mon, 21 Feb 2022 09:47:57 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:60432 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1378305AbiBUOr3 (ORCPT ); Mon, 21 Feb 2022 09:47:29 -0500 Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8186D5FCF for ; Mon, 21 Feb 2022 06:46:57 -0800 (PST) Received: by mail-wr1-x42f.google.com with SMTP id v12so27529544wrv.2 for ; Mon, 21 Feb 2022 06:46:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=EhDGSve0vZuK/v7hn9A3BTJOjFaVHhTjk6bB5p9fNgM=; b=StRGXmDNf5OR2C+taFvvMpexKpSco0pr0yj/NXulRPM3M6NnL+mXitKFeXjMZ609ej KjqwITB+JqATaZNa9SBSPPlV1jRdxhhqCrir/wLK1qLhrb+TmaYxCmP+7sSgP22rqT1e hQ5UTczMz8iOy6dn9UkSsHFsW2fFrRyZWkQFmz9lrh1GxB1DLdBTTrcUO6u2io0bb7kI wTnQ8sowbV8qRuVd5PwhKt0VFl4OaPmKNgh4QEsWOiTDF/qIgQoNk4QEjgIaSJGL2W+y 5J2+MIp/tBLEiZUt4vBB1MOLJIbC2+KWKPhJbt9+CNZV45/dm8WXucVdQVP7AkFk8sx1 5J8A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=EhDGSve0vZuK/v7hn9A3BTJOjFaVHhTjk6bB5p9fNgM=; b=vf0ROgs0HeHpP0p1dwW7JImXaDT1jTEyI9nFTZw49O5K6uChQBRAYSP1AeC4N8NUdu pIi72m/UIngqsJpPjJBVwFF6n9VKodU24P6j78sz/92i5gMPIZIE4GgAigEzAcm0cH8X wMXfTMDY3slk6pJytcwxlA3jTXtSXUKkkXa//sG3rlALL8o/U5KbcOaQfsfwaFDj2AKE EPPjixKzuvUfxOIt74gh0neUI4wCunGb1R35G3AUSRiM7wemWSMsZxDZ9r1UDFFZhOfA cPM7eaie38XGqEFSAQ/RAyy9zqcxBH/TmJEw1bW58ZI/NbAuMibfI+jGC6qhpIt88Vd+ zczA== X-Gm-Message-State: AOAM5326cs1pJOKl4FXu1/bGXMULVL8k9OVMqrHw1ECJtGDBWpjM6LV/ qY07xpHFPszzZ6bD//bYPpr+VhisdEjRKg== X-Google-Smtp-Source: ABdhPJyqktaP5bzzaUTerr63XoarTDPD8lgy68onv0eMhKq2bqq8/zJ74UjFR+Xi4vlydHh12onqGg== X-Received: by 2002:adf:e5d1:0:b0:1e6:1109:5a11 with SMTP id a17-20020adfe5d1000000b001e611095a11mr16065192wrn.641.1645454815876; Mon, 21 Feb 2022 06:46:55 -0800 (PST) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id r14sm25071094wrz.84.2022.02.21.06.46.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Feb 2022 06:46:55 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH 09/25] CI: move p4 and git-lfs variables to ci/install-dependencies.sh Date: Mon, 21 Feb 2022 15:46:21 +0100 Message-Id: X-Mailer: git-send-email 2.35.1.1132.ga1fe46f8690 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Move the declaration of variables that are only used by the "ubuntu-latest" block in "ci/install-dependencies.sh" there from "ci/lib.sh". This makes the code easier to follow, and changes "ci/lib.sh" to a library that only exports CI variables for general use, we didn't need to export these $P4_PATH and $GIT_LFS_PATH variables. Signed-off-by: Ævar Arnfjörð Bjarmason --- ci/install-dependencies.sh | 17 +++++++++++++++-- ci/lib.sh | 12 ------------ 2 files changed, 15 insertions(+), 14 deletions(-) diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh index e7ea8799411..34b4400f38b 100755 --- a/ci/install-dependencies.sh +++ b/ci/install-dependencies.sh @@ -5,14 +5,27 @@ . ${0%/*}/lib.sh -P4WHENCE=http://filehost.perforce.com/perforce/r$LINUX_P4_VERSION -LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION UBUNTU_COMMON_PKGS="make libssl-dev libcurl4-openssl-dev libexpat-dev tcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl libemail-valid-perl libio-socket-ssl-perl libnet-smtp-ssl-perl" case "$runs_on_pool" in ubuntu-latest) + # The Linux build installs the defined dependency versions below. + # The OS X build installs much more recent versions, whichever + # were recorded in the Homebrew database upon creating the OS X + # image. + # Keep that in mind when you encounter a broken OS X build! + LINUX_P4_VERSION="16.2" + LINUX_GIT_LFS_VERSION="1.5.2" + + P4_PATH="$HOME/custom/p4" + GIT_LFS_PATH="$HOME/custom/git-lfs" + export PATH="$GIT_LFS_PATH:$P4_PATH:$PATH" + + P4WHENCE=http://filehost.perforce.com/perforce/r$LINUX_P4_VERSION + LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION + sudo apt-get -q update sudo apt-get -q -y install language-pack-is libsvn-perl apache2 \ $UBUNTU_COMMON_PKGS $CC_PACKAGE diff --git a/ci/lib.sh b/ci/lib.sh index ad1aa59a384..6b37b10d750 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -65,18 +65,6 @@ ubuntu-latest) fi export GIT_TEST_HTTPD=true - - # The Linux build installs the defined dependency versions below. - # The OS X build installs much more recent versions, whichever - # were recorded in the Homebrew database upon creating the OS X - # image. - # Keep that in mind when you encounter a broken OS X build! - export LINUX_P4_VERSION="16.2" - export LINUX_GIT_LFS_VERSION="1.5.2" - - P4_PATH="$HOME/custom/p4" - GIT_LFS_PATH="$HOME/custom/git-lfs" - export PATH="$GIT_LFS_PATH:$P4_PATH:$PATH" ;; macos-latest) if [ "$jobname" = osx-gcc ] From patchwork Mon Feb 21 14:46:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12753715 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CC18FC433EF for ; Mon, 21 Feb 2022 14:47:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241176AbiBUOrx (ORCPT ); Mon, 21 Feb 2022 09:47:53 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:60420 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1378306AbiBUOr3 (ORCPT ); Mon, 21 Feb 2022 09:47:29 -0500 Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5E82560C7 for ; Mon, 21 Feb 2022 06:46:58 -0800 (PST) Received: by mail-wr1-x42f.google.com with SMTP id d27so27512687wrc.6 for ; Mon, 21 Feb 2022 06:46:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=6W6pzwo2pcJV0O/hQWzO0q9jWSzFQTXekGfcJvx/GT4=; b=SCpOon71yKRDS0lM6e5XxvF00PS2OkrVkhH97hkIsdxIXMe5cE/VG9qAq/apvSkwOC ugriwbdIydexFrLcYSliQDGQBgjboOFO3Mp5lWqfC01EUK+Vi5vNPQ8IdQTojACo274d onSuynwVvnygvbLbprJgN90d+Oc7cZ1/KHPGyoH+S1UsCM03rG+TZX4Mxh/hiRtRDVuQ vtPSNvCKggkXhgGCuO2r9IZ7XnuzJmunyT846YdZO3I+bd98eSTN1hAp7j8Xj1tAIOgb SN8lOj/OSLIUYoUAABcPme7CCmBLlUeBFtZva4VSG+BJ79+aLyfzmYms20cZUBC6dGlf XZsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=6W6pzwo2pcJV0O/hQWzO0q9jWSzFQTXekGfcJvx/GT4=; b=dkLRk3RumWcf49eJkBHLn3JT/p6jLmlPzElVQYrqh2YRa/xMuz9P5LlGU2atDbNGVZ LsEdYQ0r187BPXpJnmrGVxoo1YJC41QikTZQv8bKgORybrYB+EXlhTZcA1iK2dhKhIhH tKJKwrKSAq7nhtWPjexwd762ATyKkdIGRjwsvQ5zFrjjUCfVDFURmR6+m/2FwvUod6ol oiuxMGlOVUZf4oLsUblLjLGAiRDGr6//Sg5zOzgDQkBZFS8mmUg+dDNtQnlB5I/Ws4jI mnozc/4rLMd5MkrC9oOhiIy3O2VX68M4G+VnEAFCWufjHgn9oqKxNN+7KYXi6khYT2Lh kBeA== X-Gm-Message-State: AOAM5330e3GFW7ty4QzzLESA3gDjqVXve4wtDSu13XBiL6zxVNAgChAs 0DaoB7ZV9cwT7bKHuJRjErItsiqFK5Tarw== X-Google-Smtp-Source: ABdhPJyzF6V7J45URC30jxl2Ub6s2/mYzFlItySbeQaH3EMhWimhFv/B8U+gWm2NhhjKuDVlqIoOrQ== X-Received: by 2002:adf:b318:0:b0:1e7:7143:d92d with SMTP id j24-20020adfb318000000b001e77143d92dmr16047439wrd.137.1645454816703; Mon, 21 Feb 2022 06:46:56 -0800 (PST) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id r14sm25071094wrz.84.2022.02.21.06.46.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Feb 2022 06:46:56 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH 10/25] CI: consistently use "export" in ci/lib.sh Date: Mon, 21 Feb 2022 15:46:22 +0100 Message-Id: X-Mailer: git-send-email 2.35.1.1132.ga1fe46f8690 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Change the "ci/lib.sh" script co consistently use "export", for e.g. MAKEFLAGS we were exporting it, and then assigning to it, let's do it the other way around. Right now this doesn't matter, since we in e.g. "ci/install-dependencies.sh" source this file, and don't use something like "env(1)" to retrieve these variables. But in a subsequent commit we'll "export" these variables through a wrapper (to additionally write them to a GitHub CI-specific $GITHUB_ENV file). This change makes that subsequent change easier to read, as it won't need to do any control flow refactoring. Signed-off-by: Ævar Arnfjörð Bjarmason --- ci/lib.sh | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/ci/lib.sh b/ci/lib.sh index 6b37b10d750..31fe3a476a9 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -27,7 +27,7 @@ check_unignored_build_artifacts () export TERM=${TERM:-dumb} # Clear MAKEFLAGS that may come from the outside world. -export MAKEFLAGS= +MAKEFLAGS= if test "$GITHUB_ACTIONS" = "true" then @@ -35,10 +35,12 @@ then CC="${CC:-gcc}" export GIT_PROVE_OPTS="--timer --jobs 10" - export GIT_TEST_OPTS="--verbose-log -x" + GIT_TEST_OPTS="--verbose-log -x" MAKEFLAGS="$MAKEFLAGS --jobs=10" test Windows != "$RUNNER_OS" || GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS" + + export GIT_TEST_OPTS else echo "Could not identify CI type" >&2 env >&2 @@ -92,4 +94,4 @@ linux-leaks) ;; esac -MAKEFLAGS="$MAKEFLAGS CC=${CC:-cc}" +export MAKEFLAGS="$MAKEFLAGS CC=${CC:-cc}" From patchwork Mon Feb 21 14:46:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12753717 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B815AC433EF for ; Mon, 21 Feb 2022 14:47:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378318AbiBUOr7 (ORCPT ); Mon, 21 Feb 2022 09:47:59 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:60440 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1378309AbiBUOr3 (ORCPT ); Mon, 21 Feb 2022 09:47:29 -0500 Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 674C660E0 for ; Mon, 21 Feb 2022 06:46:59 -0800 (PST) Received: by mail-wr1-x431.google.com with SMTP id s1so1070992wrg.10 for ; Mon, 21 Feb 2022 06:46:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=WIDVvDW7dORamPdWUOqvN7ITwcMNQ/B2UzfOVfACKJc=; b=I57Ysoqmspfx/Mmhjuea0bx7eA20r8yvjdCgxAC9Dzf4K2ZqJG+BLyylb1BE6PHDKX 3LWDwM/hjQJKv/TYg6BPYGGS8T35dqKXZLfaFm6SAFFur6yQOWJ1HfJnhVdXWeD9M7CY aN5ClK+6EyhWxBvluYaWIBXOUbq8pOcjqoxxYAOIEraQV3hX6haoqzoSTcObCmdbdd2M UOAB1FcMV/WMU0ko5AMQb4zAEdr/TE8+3cqo5HQehcAUbmzDXEBo9q3wCes+XjGQUfSI FBOR6Go1vWE85WokiRBw2DL/GxtLafHROogKe1NJG2oi0N5WAh8W38ROavIZMJAyaUjt cB9w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=WIDVvDW7dORamPdWUOqvN7ITwcMNQ/B2UzfOVfACKJc=; b=0/IwtoQ3c7KdOv0L89UIbVlxhw9nXGycziKkVTxJoJnI2cDXyFk/MAcCjHFgC7ul/H 9X22vtU9bHmYCim+E9FbwdSotibiuxuqXGUmBulGZPK4Xk0OmYTS6USSGzSnRnSsaPV8 74pYmdwTUBS34ufVfSNENEtR5OQsRJbEcMjFY/UWdAM6EJN8Ffd/AXXsA8PR10MWPGUa o7TE+aI9CsEgsxtPqBu1qfHSpsBQpTXHTWMZmnk09qgWcosBYRqjG6tD28gVJrDDcLGk WRLAPHsIkGVi5LxDw9K/mdXe9eqaGJPXT6gy62we57Bik03w6CVIkBPtl88TkpuBAnW2 hB2Q== X-Gm-Message-State: AOAM533yAZhW8bcPtq8xOZV5k8CQULR5ZZuml+P7qCjc11UQHvuc1wNc j/5U9+2dmK7ViHzSJY8gUupJJhmEr6kfZQ== X-Google-Smtp-Source: ABdhPJyNzPkGwHNwdai6RjEYkObtkO2DMSe9AD+1tm9VAKxEwRuGJo73XOuZuVTcR2ViOaO9J5KXoQ== X-Received: by 2002:a05:6000:1181:b0:1e5:a6af:7b8a with SMTP id g1-20020a056000118100b001e5a6af7b8amr15851549wrx.416.1645454817751; Mon, 21 Feb 2022 06:46:57 -0800 (PST) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id r14sm25071094wrz.84.2022.02.21.06.46.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Feb 2022 06:46:57 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH 11/25] CI: export variables via a wrapper Date: Mon, 21 Feb 2022 15:46:23 +0100 Message-Id: X-Mailer: git-send-email 2.35.1.1132.ga1fe46f8690 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Add and use a "setenv" wrapper to export the variables set in ci/lib.sh. For now this merely adds debugging information about what variables are set, but in a subsequent commit we'll expand this to set variables in $GITHUB_ENV instead of the environment, and thus carry variable settings between steps in a job. We'll also eventually be interested in what type of variable we're setting, and when it needs to be used, because we'll prominently expose what variables we set in what CI "step", so having only the ones we actually use will improve readability. To that end call the "setenv" function with optional "--build", "--test" and "--all" parameters to indicate whether they're needed for building, testing or both. We currently ignore this parameter, but will make use of it in a subsequent commit. Adding it here reduces the needless churn, as we won't need to change all these lines again for that subsequent change. Signed-off-by: Ævar Arnfjörð Bjarmason --- ci/lib.sh | 39 ++++++++++++++++++++++++++++----------- ci/run-build-and-tests.sh | 34 +++++++++++++++++----------------- 2 files changed, 45 insertions(+), 28 deletions(-) diff --git a/ci/lib.sh b/ci/lib.sh index 31fe3a476a9..1a1fa0e4bb2 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -13,6 +13,23 @@ then exit 1 fi +# Helper functions +setenv () { + varmode= + case "$1" in + --*) + varmode=$1 + shift + ;; + esac + + key=$1 + val=$2 + shift 2 + + eval "export $key=\"$val\"" +} + check_unignored_build_artifacts () { ! git ls-files --other --exclude-standard --error-unmatch \ @@ -24,7 +41,7 @@ check_unignored_build_artifacts () } # GitHub Action doesn't set TERM, which is required by tput -export TERM=${TERM:-dumb} +setenv TERM ${TERM:-dumb} # Clear MAKEFLAGS that may come from the outside world. MAKEFLAGS= @@ -34,23 +51,23 @@ then CI_TYPE=github-actions CC="${CC:-gcc}" - export GIT_PROVE_OPTS="--timer --jobs 10" + setenv --test GIT_PROVE_OPTS "--timer --jobs 10" GIT_TEST_OPTS="--verbose-log -x" MAKEFLAGS="$MAKEFLAGS --jobs=10" test Windows != "$RUNNER_OS" || GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS" - export GIT_TEST_OPTS + setenv --test GIT_TEST_OPTS "$GIT_TEST_OPTS" else echo "Could not identify CI type" >&2 env >&2 exit 1 fi -export DEVELOPER=1 -export DEFAULT_TEST_TARGET=prove -export GIT_TEST_CLONE_2GB=true -export SKIP_DASHED_BUILT_INS=YesPlease +setenv --build DEVELOPER 1 +setenv --test DEFAULT_TEST_TARGET prove +setenv --test GIT_TEST_CLONE_2GB true +setenv --build SKIP_DASHED_BUILT_INS YesPlease case "$runs_on_pool" in ubuntu-latest) @@ -66,7 +83,7 @@ ubuntu-latest) MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=/usr/bin/python2" fi - export GIT_TEST_HTTPD=true + setenv --test GIT_TEST_HTTPD true ;; macos-latest) if [ "$jobname" = osx-gcc ] @@ -89,9 +106,9 @@ linux-musl) MAKEFLAGS="$MAKEFLAGS GIT_TEST_UTF8_LOCALE=C.UTF-8" ;; linux-leaks) - export SANITIZE=leak - export GIT_TEST_PASSING_SANITIZE_LEAK=true + setenv --build SANITIZE leak + setenv --test GIT_TEST_PASSING_SANITIZE_LEAK true ;; esac -export MAKEFLAGS="$MAKEFLAGS CC=${CC:-cc}" +setenv --all MAKEFLAGS "$MAKEFLAGS CC=${CC:-cc}" diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh index e5a21985b44..35d45a9373e 100755 --- a/ci/run-build-and-tests.sh +++ b/ci/run-build-and-tests.sh @@ -9,33 +9,33 @@ export MAKE_TARGETS="all test" case "$jobname" in linux-gcc) - export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main + setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main ;; linux-TEST-vars) - export GIT_TEST_SPLIT_INDEX=yes - export GIT_TEST_MERGE_ALGORITHM=recursive - export GIT_TEST_FULL_IN_PACK_ARRAY=true - export GIT_TEST_OE_SIZE=10 - export GIT_TEST_OE_DELTA_SIZE=5 - export GIT_TEST_COMMIT_GRAPH=1 - export GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS=1 - export GIT_TEST_MULTI_PACK_INDEX=1 - export GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP=1 - export GIT_TEST_ADD_I_USE_BUILTIN=1 - export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=master - export GIT_TEST_WRITE_REV_INDEX=1 - export GIT_TEST_CHECKOUT_WORKERS=2 + setenv --test GIT_TEST_SPLIT_INDEX yes + setenv --test GIT_TEST_MERGE_ALGORITHM recursive + setenv --test GIT_TEST_FULL_IN_PACK_ARRAY true + setenv --test GIT_TEST_OE_SIZE 10 + setenv --test GIT_TEST_OE_DELTA_SIZE 5 + setenv --test GIT_TEST_COMMIT_GRAPH 1 + setenv --test GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS 1 + setenv --test GIT_TEST_MULTI_PACK_INDEX 1 + setenv --test GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP 1 + setenv --test GIT_TEST_ADD_I_USE_BUILTIN 1 + setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME master + setenv --test GIT_TEST_WRITE_REV_INDEX 1 + setenv --test GIT_TEST_CHECKOUT_WORKERS 2 ;; linux-clang) - export GIT_TEST_DEFAULT_HASH=sha1 + setenv --test GIT_TEST_DEFAULT_HASH sha1 ;; linux-sha256) - export GIT_TEST_DEFAULT_HASH=sha256 + setenv --test GIT_TEST_DEFAULT_HASH sha256 ;; pedantic) # Don't run the tests; we only care about whether Git can be # built. - export DEVOPTS=pedantic + setenv --build DEVOPTS pedantic export MAKE_TARGETS=all ;; esac From patchwork Mon Feb 21 14:46:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12753724 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A824FC433F5 for ; Mon, 21 Feb 2022 14:48:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378357AbiBUOsW (ORCPT ); Mon, 21 Feb 2022 09:48:22 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:60444 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1378310AbiBUOr3 (ORCPT ); Mon, 21 Feb 2022 09:47:29 -0500 Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com [IPv6:2a00:1450:4864:20::331]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8198760EB for ; Mon, 21 Feb 2022 06:47:00 -0800 (PST) Received: by mail-wm1-x331.google.com with SMTP id o34so9674985wms.1 for ; Mon, 21 Feb 2022 06:47:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=E5aGqlBNzN9rQLni39Ueone2+t6aZiFoX9UD4DvtGyA=; b=L+McUR9WhzDG7dRRkHvmhUwcZpIRYauSKWMUz23HsXuGiyVfJ0LAuZ5a6UQzmUJ70g adksJR5nYjPVPlFdnmPNfENvJKqhXBlnFg5biADxSGTU4/l5U98W3It6Ul82qWpEIosz O5eJYQBiD6/KX7xcpShffzKEs/TSU+QzJBu+f6oOmeyEGnJdZwvQF8T5NG4hqIUxutzL SRMNHZxTq8gLg/VdoaFOWPlkblHxRnrOg8oJbR7qLvlZu3jpIkeChIoBzkvkpoWjh4kj K8B3ehqBJjfvCzRA9lwxKDeai8EUOUphuc0lBwWO2TBSn7xwwLyizNsgbXoFd5wg/X82 JNaw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=E5aGqlBNzN9rQLni39Ueone2+t6aZiFoX9UD4DvtGyA=; b=ctVBnKY3ts/XaaO8dkIVEcMCxoIDHbJwU5/8Kbz5AX3eTnUIwrHFHHL3W0Fy7mROxe CfvhA3jC9yNgbqvmyO1GEPSoydt8SIX1MYKps2Ib5YS+qCurOgJIN7JMyqV9nh5Vc3dm utf26EgPFb5WIQBYN0m3N735nXksTSicggRH1rRNsSRai80rPrbGC/z9bI6uOZu2P68m X1e+bd2Pysp7fRYjDtX5RjjWBBx0lQfc9aoZ/Yjyygw+ZhqtF3nz3SLmzcRFvl3TI/Zv b1Ndjxk2I9iy1/pCJn0jchvfm6ZMq3KYqACYVocemTdUrXktQMp4j6KJOwb6L/8HpB98 KGpg== X-Gm-Message-State: AOAM5319sZgnqIX3uGLRbH+VYb3hSXqpXJvxnw6Nq9gfyV9AX8ypTvdQ HrwZ5HKppdxhAJU0/zdAPvjq7ZixDfzYsg== X-Google-Smtp-Source: ABdhPJwV530BVLFUTyi4S+QkueBUltOXqR0YDzDPkzZ2CyFIvNUXOofHQ/ni7/GCVQKcbZwuO6f09g== X-Received: by 2002:a05:600c:1c9a:b0:37b:c3ec:e41a with SMTP id k26-20020a05600c1c9a00b0037bc3ece41amr18249976wms.162.1645454818722; Mon, 21 Feb 2022 06:46:58 -0800 (PST) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id r14sm25071094wrz.84.2022.02.21.06.46.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Feb 2022 06:46:58 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH 12/25] CI: remove "run-build-and-tests.sh", run "make [test]" directly Date: Mon, 21 Feb 2022 15:46:24 +0100 Message-Id: X-Mailer: git-send-email 2.35.1.1132.ga1fe46f8690 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Remove the already thin "ci/run-build-and-tests.sh" wrapper and instead make the CI run "make" or "make test" directly. By doing this we'll be able to easily see at a glance whether our failure was in the compilation or testing, whether that's via human eyes or improve machine readability. We also need to run our new "ci/check-unignored-build-artifacts.sh" on success() in the CI now, just like we already had a step conditional on failure() running ci/print-test-failures.sh. The reason we used a "ci/run-build-and-tests.sh" wrapper in the first place had to do with Travis CI-specific constraints that no longer apply to us, as the Travis CI support has been removed. Instead we can configure the CI in an earlier step by running "ci/lib.sh", which under GitHub CI will write the environment variables we need to the "$GITHUB_ENV" file. We'll then have access to them in subsequent steps, and crucially those variables will be prominently visible at the start of each step via an expandable drop-down in the UI.drop-do. I.e. this changes the CI run from a top-down flow like (pseudocode): - job: - step1: - use ci/lib.sh to set env vars - run a script like ci/run-build-and-tests.sh - step2: - if: failure() - use ci/lib.sh to set env vars - run ci/print-test-failures.sh To: - job: - step1: - set variables in $GITHUB_ENV using ci/lib.sh - step2: - make - step3: - make test - step4: - if: failure() - run ci/print-test-failures.sh - step5: - if: success() - run ci/check-unignored-build-artifacts.sh There is a proposal[2] to get some of the benefits of this approach by not re-arranging our variable setup in this way, but to instead use the GitHub CI grouping syntax to focus on the relevant parts of "make" or "make test" when we have failures. Doing it this way makes for better looking GitHub CI UI, and lays much better ground work for our CI going forward. Because: * The CI logic will be more portable to a future CI system, since a common feature of them is to run various commands in sequence, but a future system won't necessarily support the GitHub-specifics syntax of "grouping" output within a "step". Even if those systems don't support a "$GITHUB_ENV" emulating will be much easier than to deal with some CI-specific grouping syntax. * At the start of every step the GitHub CI presents an expandable list of environment variables from "$GITHUB_ENV". We'll now see exactly what variables affected that step (although we currently overshoot that a bit, and always define all variables). * CI failures will be easier to reproduce locally, as this makes the relevant ci/* scripts something that sets up our environment, but leaves "make" and "make test" working as they do locally. To reproduce a run the user only needs to set the variables discussed in the drop-down above, either manually or by running "ci/lib.sh". * The output will be less verbose. The "ci/lib.sh" script uses "set -x", and before this e.g. "ci/static-analysis.sh" would start with 40 lines of trace output, culminating in using "export" to export the relevant environment variables. Now that verbosity is in the earlier "ci/lib.sh" step, and not in any subsequent one. The "make" targets then start out with the relevant output non-trace output right away. * If we do want to use the grouping syntax within a "step" it'll now be easier to do so. It doesn't support nesting, so we'd have to make a choice between using it for e.g. "make" v.s. "make test", or individual test failures. See "sadly" in [3]. 1. https://lore.kernel.org/git/211120.86k0h30zuw.gmgdl@evledraar.gmail.com/ 2. https://lore.kernel.org/git/pull.1117.git.1643050574.gitgitgadget@gmail.com/ 3. https://lore.kernel.org/git/9333ba781b8240f704e739b00d274f8c3d887e39.1643050574.git.gitgitgadget@gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason --- .github/workflows/main.yml | 14 ++++++-- ci/check-unignored-build-artifacts.sh | 8 +++++ ci/install-dependencies.sh | 4 +++ ci/lib.sh | 40 ++++++++++++++++++++++- ci/make-test-artifacts.sh | 2 -- ci/run-build-and-tests.sh | 47 --------------------------- ci/run-test-slice.sh | 2 -- ci/test-documentation.sh | 1 - 8 files changed, 62 insertions(+), 56 deletions(-) create mode 100755 ci/check-unignored-build-artifacts.sh delete mode 100755 ci/run-build-and-tests.sh diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index ad752010102..c83f6a21ca3 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -268,7 +268,10 @@ jobs: steps: - uses: actions/checkout@v2 - run: ci/install-dependencies.sh - - run: ci/run-build-and-tests.sh + - run: ci/lib.sh + - run: make + - run: make test + if: success() - run: ci/print-test-failures.sh if: failure() - name: Upload failed tests' directories @@ -292,6 +295,7 @@ jobs: image: daald/ubuntu32:xenial - jobname: pedantic image: fedora + skip-tests: no env: jobname: ${{matrix.vector.jobname}} runs-on: ubuntu-latest @@ -299,9 +303,12 @@ jobs: steps: - uses: actions/checkout@v1 - run: ci/install-docker-dependencies.sh - - run: ci/run-build-and-tests.sh + - run: ci/lib.sh + - run: make + - run: make test + if: success() && matrix.vector.skip-tests != 'no' - run: ci/print-test-failures.sh - if: failure() + if: failure() && matrix.vector.skip-tests != 'no' - name: Upload failed tests' directories if: failure() && env.FAILED_TEST_ARTIFACTS != '' uses: actions/upload-artifact@v1 @@ -317,6 +324,7 @@ jobs: steps: - uses: actions/checkout@v2 - run: ci/install-dependencies.sh + - run: ci/lib.sh - run: make ci-static-analysis sparse: needs: ci-config diff --git a/ci/check-unignored-build-artifacts.sh b/ci/check-unignored-build-artifacts.sh new file mode 100755 index 00000000000..56d04b0db9a --- /dev/null +++ b/ci/check-unignored-build-artifacts.sh @@ -0,0 +1,8 @@ +#!/bin/sh +# +# Check whether the build created anything not in our .gitignore +# + +. ${0%/*}/lib.sh + +check_unignored_build_artifacts diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh index 34b4400f38b..01006379fd0 100755 --- a/ci/install-dependencies.sh +++ b/ci/install-dependencies.sh @@ -22,6 +22,10 @@ ubuntu-latest) P4_PATH="$HOME/custom/p4" GIT_LFS_PATH="$HOME/custom/git-lfs" export PATH="$GIT_LFS_PATH:$P4_PATH:$PATH" + if test -n "$GITHUB_PATH" + then + echo "$PATH" >>"$GITHUB_PATH" + fi P4WHENCE=http://filehost.perforce.com/perforce/r$LINUX_P4_VERSION LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION diff --git a/ci/lib.sh b/ci/lib.sh index 1a1fa0e4bb2..05f3dd15e27 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -1,3 +1,5 @@ +#!/bin/sh + # Library of functions shared by all CI scripts # Set 'exit on error' for all CI scripts to let the caller know that @@ -27,7 +29,14 @@ setenv () { val=$2 shift 2 - eval "export $key=\"$val\"" + if test -n "$GITHUB_ENV" + then + echo "$key=$val" >>"$GITHUB_ENV" + else + # For local debugging. Not used by the GitHub CI + # itself. + eval "export $key=\"$val\"" + fi } check_unignored_build_artifacts () @@ -96,6 +105,35 @@ macos-latest) esac case "$jobname" in +linux-gcc) + setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main + ;; +linux-TEST-vars) + setenv --test GIT_TEST_SPLIT_INDEX yes + setenv --test GIT_TEST_MERGE_ALGORITHM recursive + setenv --test GIT_TEST_FULL_IN_PACK_ARRAY true + setenv --test GIT_TEST_OE_SIZE 10 + setenv --test GIT_TEST_OE_DELTA_SIZE 5 + setenv --test GIT_TEST_COMMIT_GRAPH 1 + setenv --test GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS 1 + setenv --test GIT_TEST_MULTI_PACK_INDEX 1 + setenv --test GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP 1 + setenv --test GIT_TEST_ADD_I_USE_BUILTIN 1 + setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME master + setenv --test GIT_TEST_WRITE_REV_INDEX 1 + setenv --test GIT_TEST_CHECKOUT_WORKERS 2 + ;; +linux-clang) + setenv --test GIT_TEST_DEFAULT_HASH sha1 + ;; +linux-sha256) + setenv --test GIT_TEST_DEFAULT_HASH sha256 + ;; +pedantic) + # Don't run the tests; we only care about whether Git can be + # built. + setenv --build DEVOPTS pedantic + ;; linux32) CC=gcc ;; diff --git a/ci/make-test-artifacts.sh b/ci/make-test-artifacts.sh index 646967481f6..45298562982 100755 --- a/ci/make-test-artifacts.sh +++ b/ci/make-test-artifacts.sh @@ -8,5 +8,3 @@ mkdir -p "$1" # in case ci/lib.sh decides to quit early . ${0%/*}/lib.sh make artifacts-tar ARTIFACTS_DIRECTORY="$1" - -check_unignored_build_artifacts diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh deleted file mode 100755 index 35d45a9373e..00000000000 --- a/ci/run-build-and-tests.sh +++ /dev/null @@ -1,47 +0,0 @@ -#!/bin/sh -# -# Build and test Git -# - -. ${0%/*}/lib.sh - -export MAKE_TARGETS="all test" - -case "$jobname" in -linux-gcc) - setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main - ;; -linux-TEST-vars) - setenv --test GIT_TEST_SPLIT_INDEX yes - setenv --test GIT_TEST_MERGE_ALGORITHM recursive - setenv --test GIT_TEST_FULL_IN_PACK_ARRAY true - setenv --test GIT_TEST_OE_SIZE 10 - setenv --test GIT_TEST_OE_DELTA_SIZE 5 - setenv --test GIT_TEST_COMMIT_GRAPH 1 - setenv --test GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS 1 - setenv --test GIT_TEST_MULTI_PACK_INDEX 1 - setenv --test GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP 1 - setenv --test GIT_TEST_ADD_I_USE_BUILTIN 1 - setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME master - setenv --test GIT_TEST_WRITE_REV_INDEX 1 - setenv --test GIT_TEST_CHECKOUT_WORKERS 2 - ;; -linux-clang) - setenv --test GIT_TEST_DEFAULT_HASH sha1 - ;; -linux-sha256) - setenv --test GIT_TEST_DEFAULT_HASH sha256 - ;; -pedantic) - # Don't run the tests; we only care about whether Git can be - # built. - setenv --build DEVOPTS pedantic - export MAKE_TARGETS=all - ;; -esac - -# Any new "test" targets should not go after this "make", but should -# adjust $MAKE_TARGETS. Otherwise compilation-only targets above will -# start running tests. -make $MAKE_TARGETS -check_unignored_build_artifacts diff --git a/ci/run-test-slice.sh b/ci/run-test-slice.sh index b9a682b4bcd..70326961454 100755 --- a/ci/run-test-slice.sh +++ b/ci/run-test-slice.sh @@ -8,5 +8,3 @@ make --quiet -C t T="$(cd t && ./helper/test-tool path-utils slice-tests "$1" "$2" t[0-9]*.sh | tr '\n' ' ')" - -check_unignored_build_artifacts diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh index 41e2b126311..6b5cce03bd7 100755 --- a/ci/test-documentation.sh +++ b/ci/test-documentation.sh @@ -39,4 +39,3 @@ test -s Documentation/git.html grep ' X-Patchwork-Id: 12753723 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 651ADC433EF for ; Mon, 21 Feb 2022 14:48:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378355AbiBUOsV (ORCPT ); Mon, 21 Feb 2022 09:48:21 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:60446 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1378311AbiBUOr3 (ORCPT ); Mon, 21 Feb 2022 09:47:29 -0500 Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 75264624E for ; Mon, 21 Feb 2022 06:47:01 -0800 (PST) Received: by mail-wr1-x42c.google.com with SMTP id h6so27480953wrb.9 for ; Mon, 21 Feb 2022 06:47:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=9Paw+vcgMDCBuDqlliib9ljjvXwbuiNwm5TpeoWveeQ=; b=GQU4czUBmdO9iLIaWtUYWXG6iC2MoZbrvnBgv+MZVc7njx3HHYUSTKSmU4Z/9yUyYn tc43X+eZxCgi9wmCCQrHfaH8EH2HsgfWxcNXtcErQUIRFp3prw5NpZtgAKf+YmSB9iSE hoY07eVoyoKFI/FxQTksAAm//Nt/J/b+pm0QPRu3VBta0xzNQ9XEwFLJnIoFqyd91yFs NC5UJXSPJpLlLASXi8g4rYeg7uTKGMk+BQ5LgycdQjvlH8+ujzQY8WxMqTn1Age6H1fq ZjqcY+aLcrD7AtTyQL/+rh0vVbUdk2jo9T/jhXkD6qonpx7z7J/04jqolzOQXKYfldHE kwHQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9Paw+vcgMDCBuDqlliib9ljjvXwbuiNwm5TpeoWveeQ=; b=NipI3Tc9BwkxTWoFU1Wdf+F0UgsEwN9HmMWsbqaJZ2mNIbTwWsC+VtWSadlncoKXWd N3Ljvbu7KEKXOtdrZFkQcs07QMI0p8jIQJReiw9sjA2nVa+QO3w7+WSLyGAW5mHc0ECc 5C/SC3QaXcBU+ufjTOC6S9r/kQXDsT2qwd6r+QSZB13V7xHLSPgj+MvcAQdG4+3oweCJ thO7WHePExZnlD26yBxRjdR0OprRE3U+LbAvmTH6TU63Mb/HunGnLYIh9kcwws26KlX7 foxhd47ARJbtNRGe2PIc7i6AZdVwsmRmCdxoo+v7uhAy+xQ9pGXzwriQr4orD2jVYWWn XAGg== X-Gm-Message-State: AOAM532WdIRlwr9KJARvSzvrmNmUghUJMetytf2vKdG/LF25Tam/33Bo 6eeALoWBWjmk0+QkXUlsGokBwIwNXXaiGw== X-Google-Smtp-Source: ABdhPJxoJG/Cirwy+Jinl7E7d93tpDTglBTF9hogap4fRB00iB6exVC1eQT+6qfb2sUWQ8wkvd1RWw== X-Received: by 2002:adf:df8f:0:b0:1e6:8a5c:4b07 with SMTP id z15-20020adfdf8f000000b001e68a5c4b07mr16355732wrl.523.1645454819747; Mon, 21 Feb 2022 06:46:59 -0800 (PST) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id r14sm25071094wrz.84.2022.02.21.06.46.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Feb 2022 06:46:59 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH 13/25] CI: check ignored unignored build artifacts in "win[+VS] build" too Date: Mon, 21 Feb 2022 15:46:25 +0100 Message-Id: X-Mailer: git-send-email 2.35.1.1132.ga1fe46f8690 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Use the newly created "ci/check-unignored-build-artifacts.sh" script in the "win build" and "win+VS build" steps too. These steps weren't doing this sanity check of untracked files against .gitignore before, but as the check passes for them let's keep doing that extra sanity check. Signed-off-by: Ævar Arnfjörð Bjarmason --- .github/workflows/main.yml | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index c83f6a21ca3..05c5771e7f6 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -93,6 +93,9 @@ jobs: HOME: ${{runner.workspace}} NO_PERL: 1 run: . /etc/profile && ci/make-test-artifacts.sh artifacts + - run: ci/check-unignored-build-artifacts.sh + if: success() + shell: bash - name: zip up tracked files run: git archive -o artifacts/tracked.tar.gz HEAD - name: upload tracked files and build artifacts @@ -180,6 +183,9 @@ jobs: run: | mkdir -p artifacts && eval "$(make -n artifacts-tar INCLUDE_DLLS_IN_ARTIFACTS=YesPlease ARTIFACTS_DIRECTORY=artifacts NO_GETTEXT=YesPlease 2>&1 | grep ^tar)" + - run: ci/check-unignored-build-artifacts.sh + if: success() + shell: bash - name: zip up tracked files run: git archive -o artifacts/tracked.tar.gz HEAD - name: upload tracked files and build artifacts From patchwork Mon Feb 21 14:46:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12753722 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6CEE9C433EF for ; Mon, 21 Feb 2022 14:47:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378350AbiBUOsP (ORCPT ); Mon, 21 Feb 2022 09:48:15 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:60418 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1378312AbiBUOr3 (ORCPT ); Mon, 21 Feb 2022 09:47:29 -0500 Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com [IPv6:2a00:1450:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6D02B6266 for ; Mon, 21 Feb 2022 06:47:02 -0800 (PST) Received: by mail-wr1-x42d.google.com with SMTP id k1so27491878wrd.8 for ; Mon, 21 Feb 2022 06:47:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=jR8UhyEfKlABU1DWsZl4GRIBnrDbe+8EI49UjDUamhk=; b=Y/CBMch0nzOBuDMXJeEfUxBXP3DMVL2I51q+mC3aMAk6rrPOmQ+fmvEp1fvlFA6LyU ATYMosTtFPBBQLHYb/M6fFimp7+klO3W7S2Z+vJECa3E152oz0aVsl/CgKyBO46PRVrs t5kDxxzZkVF8hSZU1mPJZEIj0Rl1SLUzii4QgdCI4TmIKS+3E04UgNngNXQKduvBMN8c Ecgubei/nycRKhbZs8IhRE1YeYUU/OQL9jKqlUV1TZ2dzra/tC36F3MHeLZPE8GPKQea UJchSAyF+tQrNN064rRZ8ehWIPjn3HNa3rZSiYYHi3Jatrw16ZQ4fILJev6fAh08HA+4 88TA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=jR8UhyEfKlABU1DWsZl4GRIBnrDbe+8EI49UjDUamhk=; b=TUhydAKQG9v4NcIoRRXlTT5nH737qiRl4H0f//nrQmnyNEuSVnnI0Usnd0gN/aiFRr XnPvbMRZkQu3AyXw9SKarQZ8FI2BWPILqJHa1K2OquRwUz6LUjf05eUitF4zsLDAe94A k4fKyTE2vg8lmXn//gdiPhOG8SBdZr6DYKBrxUi2O2zCuXIG3Hmq91uxgAZHjYcukz5X I4psUFAdJQq4nvM+ABU5NWfIwxErsQCU2+H88PTveG/FaPvHkNmUoh1hKO+OUyOIbqxy 7Oq+rc4eeYiDmCzCyyN8yfPBBbHwQ5h5D1zO4+mEGF/0UfHnXbVwSQCX+f5R5itq1Q7X D/+g== X-Gm-Message-State: AOAM531FPoZWNjgLlDlrC0CM45UJXWeSO8G6qgi8M+0SSYLfEYhTO8lU yZFSFxRf1ciQh+8y7wpqBeeuk/LjEcAD6g== X-Google-Smtp-Source: ABdhPJxIZxIGhICSXBwG4DXmvqygtTxYkkmJQzthS/bynx/Q6OO+q354THczr3pmCBlTfGXFMvYmbw== X-Received: by 2002:a5d:6b06:0:b0:1e3:3e52:8a6 with SMTP id v6-20020a5d6b06000000b001e33e5208a6mr15741937wrw.148.1645454820770; Mon, 21 Feb 2022 06:47:00 -0800 (PST) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id r14sm25071094wrz.84.2022.02.21.06.46.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Feb 2022 06:47:00 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH 14/25] CI: invoke "make artifacts-tar" directly in windows-build Date: Mon, 21 Feb 2022 15:46:26 +0100 Message-Id: X-Mailer: git-send-email 2.35.1.1132.ga1fe46f8690 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Change the windows-build job to invoke the "make artifacts-tar" step directly, instead of calling a "ci/make-test-artifacts.sh" script. The script was needed because "ci/lib.sh" would set up various environment variables for us, but now we can instead use the "ci/lib.sh" in its script mode. The "mkdir -p" added in b819f1d2cec (ci: parallelize testing on Windows, 2019-01-29) isn't needed, the same commit added that "mkdir -p" to the "artifacts-tar" rule itself, so we can have "make" create the directory for us. This also has the benefit of making the "build" step less chatty, since it won't start with the verbose "set -x" output, that's now contained in the "ci/lib.sh" step. Signed-off-by: Ævar Arnfjörð Bjarmason --- .github/workflows/main.yml | 5 ++++- ci/make-test-artifacts.sh | 10 ---------- 2 files changed, 4 insertions(+), 11 deletions(-) delete mode 100755 ci/make-test-artifacts.sh diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 05c5771e7f6..2c23a19eac2 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -87,12 +87,15 @@ jobs: steps: - uses: actions/checkout@v2 - uses: git-for-windows/setup-git-for-windows-sdk@v1 + - run: ci/lib.sh + shell: bash - name: build shell: bash env: HOME: ${{runner.workspace}} NO_PERL: 1 - run: . /etc/profile && ci/make-test-artifacts.sh artifacts + run: . /etc/profile && make artifacts-tar ARTIFACTS_DIRECTORY=artifacts + if: success() - run: ci/check-unignored-build-artifacts.sh if: success() shell: bash diff --git a/ci/make-test-artifacts.sh b/ci/make-test-artifacts.sh deleted file mode 100755 index 45298562982..00000000000 --- a/ci/make-test-artifacts.sh +++ /dev/null @@ -1,10 +0,0 @@ -#!/bin/sh -# -# Build Git and store artifacts for testing -# - -mkdir -p "$1" # in case ci/lib.sh decides to quit early - -. ${0%/*}/lib.sh - -make artifacts-tar ARTIFACTS_DIRECTORY="$1" From patchwork Mon Feb 21 14:46:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12753725 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 646B4C433EF for ; Mon, 21 Feb 2022 14:48:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378310AbiBUOsZ (ORCPT ); Mon, 21 Feb 2022 09:48:25 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:60450 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1378314AbiBUOr3 (ORCPT ); Mon, 21 Feb 2022 09:47:29 -0500 Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com [IPv6:2a00:1450:4864:20::331]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5643B6304 for ; Mon, 21 Feb 2022 06:47:03 -0800 (PST) Received: by mail-wm1-x331.google.com with SMTP id w13so9187962wmi.2 for ; Mon, 21 Feb 2022 06:47:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=EKrlFz9E89SyDmXih67+BR+4Z2h2uP3Yp3sCtHeWhvA=; b=XyXWRB/2D4Ygyv8HrLGR/NYyOGfrSHYjWdEfeSHUq2vJkzUUQhhJoTEh221Enzv4ZW zDE8l/P4/Rh2xK0oIuSBIhg5VsYOvQJbbV8j2caZ0m+LsIe0H77nA3/RmQ7GgXfCLkpF GiTTifMbCnrDR0YH24JnkXiPgMbBTYSTLfrEpz9y+0wuB0WMcg3sc8WfbXSjUY9n3dFv sB+VWMQcMhJIkqA0pnu6SXx6r9YpdyOJbjPtXOaPq8RlTovJYEFCr8alGT+tcSu6WLvb bpjDAJmjp5Sy7Ctggs+dRAYkKlTWhO+hwnP22+wzo2LYKxwOwoD3DoJI+wz4k/t9Sicy qztQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=EKrlFz9E89SyDmXih67+BR+4Z2h2uP3Yp3sCtHeWhvA=; b=awugKn2PQ/M4NkCFzOXcU/iYivVw7Jtz+U9OBlTszVvJA6pXGrsYyShakoklkpCinl Nr2OOKdPmme0ZNv/S/FBLf1gcAmZdXP4WadqD8AawVP2VDBeFh00TJkQ1oGN7RN2lUKz L/AguQhP0DhgFLCYPqex//Fdil0xmRM/knqXQnTxKxb4hR8d+48NaP/EAHvGoAPmVWRi 8RxD02s05wytWtE7Pzo4fsjOxZqFVJyrLEDvb/uOYu1aDMzdcGvb03oyKqaeQIjxz9yc sjlQOZIwBsMMrJhGkIxKPBr7jgQC9A3XVk/PxpW9Z4YK49p4PGSIkg2/NplFgYuBKkIj 9PLg== X-Gm-Message-State: AOAM531rcNEFVygaDm6x3c26j49pfpJlbSngUIrX/11aABHEFalMEwMr TMvlAPuV7gC4g99ofdyJfFxvQYdgPg7tqg== X-Google-Smtp-Source: ABdhPJzEPCd4n6lhlvywy15TeN1XaH4DrD51G4UAwyCEpjis1qPsE68KBQaQWJcgRJhhyYQ2lJZ5IA== X-Received: by 2002:a1c:7216:0:b0:37b:f84d:d56e with SMTP id n22-20020a1c7216000000b0037bf84dd56emr21398999wmc.177.1645454821678; Mon, 21 Feb 2022 06:47:01 -0800 (PST) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id r14sm25071094wrz.84.2022.02.21.06.47.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Feb 2022 06:47:01 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH 15/25] CI: split up and reduce "ci/test-documentation.sh" Date: Mon, 21 Feb 2022 15:46:27 +0100 Message-Id: X-Mailer: git-send-email 2.35.1.1132.ga1fe46f8690 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Change the "ci/test-documentation.sh" script to run the bash-specific parts in as one command in the CI job itself, and to run the two "make doc" commands at the top-level. It'll now be obvious from the title of the step if if we failed in the asciidoc or asciidoctor step. Since the "check_unignored_build_artifacts()" function is now only used in "ci/check-unignored-build-artifacts.sh" move that function there. The recipe for the job in ".github/workflows/main.yml" is now a bit verbose because it's effectively the same job twice, with a "make clean" in-between. It would be better for the verbosity to run it via a matrix as done in the alternate approach in [1] does, but then we'd sacrifice overall CPU time for the brevity. It might still be worth doing, but let's go for this simpler approach for now. 1. https://lore.kernel.org/git/patch-v2-6.6-7c423c8283d-20211120T030848Z-avarab@gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason --- .github/workflows/main.yml | 16 +++++++++++++- ci/check-unignored-build-artifacts.sh | 10 +++++++++ ci/lib.sh | 10 --------- ci/test-documentation.sh | 31 +++++++++------------------ 4 files changed, 35 insertions(+), 32 deletions(-) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 2c23a19eac2..92b914f16fd 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -365,4 +365,18 @@ jobs: steps: - uses: actions/checkout@v2 - run: ci/install-dependencies.sh - - run: ci/test-documentation.sh + - run: ci/lib.sh + - run: make check-docs + - run: "make doc > >(tee stdout.log) 2> >(tee stderr.raw >&2)" + shell: bash + - run: ci/test-documentation.sh AsciiDoc + if: success() + - run: ci/check-unignored-build-artifacts.sh + if: success() + - run: make clean + - run: "make USE_ASCIIDOCTOR=1 doc > >(tee stdout.log) 2> >(tee stderr.raw >&2)" + shell: bash + - run: ci/test-documentation.sh Asciidoctor + if: success() + - run: ci/check-unignored-build-artifacts.sh + if: success() diff --git a/ci/check-unignored-build-artifacts.sh b/ci/check-unignored-build-artifacts.sh index 56d04b0db9a..0bc04f32804 100755 --- a/ci/check-unignored-build-artifacts.sh +++ b/ci/check-unignored-build-artifacts.sh @@ -5,4 +5,14 @@ . ${0%/*}/lib.sh +check_unignored_build_artifacts () +{ + ! git ls-files --other --exclude-standard --error-unmatch \ + -- ':/*' 2>/dev/null || + { + echo "$(tput setaf 1)error: found unignored build artifacts$(tput sgr0)" + false + } +} + check_unignored_build_artifacts diff --git a/ci/lib.sh b/ci/lib.sh index 05f3dd15e27..fc6ce4d3e04 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -39,16 +39,6 @@ setenv () { fi } -check_unignored_build_artifacts () -{ - ! git ls-files --other --exclude-standard --error-unmatch \ - -- ':/*' 2>/dev/null || - { - echo "$(tput setaf 1)error: found unignored build artifacts$(tput sgr0)" - false - } -} - # GitHub Action doesn't set TERM, which is required by tput setenv TERM ${TERM:-dumb} diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh index 6b5cce03bd7..b8a6a6f664e 100755 --- a/ci/test-documentation.sh +++ b/ci/test-documentation.sh @@ -1,10 +1,12 @@ -#!/usr/bin/env bash +#!/bin/sh # -# Perform sanity checks on documentation and build it. +# Perform sanity checks on "make doc" output and built documentation # . ${0%/*}/lib.sh +generator=$1 + filter_log () { sed -e '/^GIT_VERSION = /d' \ -e "/constant Gem::ConfigMap is deprecated/d" \ @@ -14,28 +16,15 @@ filter_log () { "$1" } -make check-docs - -# Build docs with AsciiDoc -make doc > >(tee stdout.log) 2> >(tee stderr.raw >&2) -cat stderr.raw -filter_log stderr.raw >stderr.log -test ! -s stderr.log -test -s Documentation/git.html -test -s Documentation/git.xml -test -s Documentation/git.1 -grep ' X-Patchwork-Id: 12753719 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 13D04C433EF for ; Mon, 21 Feb 2022 14:47:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378346AbiBUOsE (ORCPT ); Mon, 21 Feb 2022 09:48:04 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:60452 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1378315AbiBUOr3 (ORCPT ); Mon, 21 Feb 2022 09:47:29 -0500 Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 47EF6630B for ; Mon, 21 Feb 2022 06:47:04 -0800 (PST) Received: by mail-wr1-x432.google.com with SMTP id k1so27492054wrd.8 for ; Mon, 21 Feb 2022 06:47:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=PsmC87namBfpmpFeLpdn5ujuPdebbGw+521whNKLqg8=; b=cW0HbkIoUvBzGO62DgN/buQ8QEbAbE2E91DGRoWt3mShiRwNm1KDx2+A2D0QmWeVW4 vfYxxs5fu8FlURzsSgJMiKmLrAhirIYArWL64oxamIQ/POe27ckukQKZhp6iorqSt18Z VYfQPIZ4YOh8yF7Lm23d0EniorptwLpuHv+iUQtGjyzvZDM1O/r5sYS13+fLaHR5HTHE vZsL/7FByGnIR/IryCK3RzEyefgHGkYpRDZFJUKgpbXAJ/z3KUAXzn5/swv9DeuUyJlh cGeJWNfPwysyp051LOLeULIg+rO70Qolfd85m6cmgvJohWD6xW4wVGvetganFG0nnF7z NqBQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=PsmC87namBfpmpFeLpdn5ujuPdebbGw+521whNKLqg8=; b=RbVrnHOWjyhLfG41j3r0VAhVkBiT0yr4OmgMFpqDd+4qZxgLQmJNb3CsbHSEuDVVeA akHAALPEmtKqpnAHRJQPKtYSvgsg8kepFHHQ4Kmc8twzr0v8lXhCcgck8MdGDYMJHM9a 8Yn+M4iXwQmXIWk04Bn3hK75LyxlMOrHuZlpL9tG1jwW1vAzr/Qp0Tp8HNX3jYqtRobz 7jxuxmZBcBwh2CuqJMs1vjr1XlxadRMYD0xRTZVCy2rSXWTYbPKkzC72xE/cECyytZXI 6yvC/PpF7CXZkDdOzDOEyJp9sL1/rlBNWkumqt4ZWQP0Z1LuKFpgZ0thlMMs8vRJQUJH wW/A== X-Gm-Message-State: AOAM532Up5tY+PYp8Oa7VqiTwEQ4z/ar+e83c8PwbOVn7mB9Wm/x1B3v E8jdNSU2fTj6yaoDs/Lg1ULL660OyhBwJA== X-Google-Smtp-Source: ABdhPJxVy82SHVip1MaWSE3Ir7m6jFZ1q3MwQfqUFqdRNWGgE2nZ8bi6IosfEe25dyqPi+Q79Nxcqw== X-Received: by 2002:adf:fa0d:0:b0:1e3:f7b0:1037 with SMTP id m13-20020adffa0d000000b001e3f7b01037mr16324269wrr.188.1645454822576; Mon, 21 Feb 2022 06:47:02 -0800 (PST) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id r14sm25071094wrz.84.2022.02.21.06.47.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Feb 2022 06:47:02 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH 16/25] CI: combine ci/install{,-docker}-dependencies.sh Date: Mon, 21 Feb 2022 15:46:28 +0100 Message-Id: X-Mailer: git-send-email 2.35.1.1132.ga1fe46f8690 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Combine the "ci/install-dependencies.sh" and "ci/install-docker-dependencies.sh" into one script. The "case" statement in the latter only cared about "$jobname", and can be folded into the same "case" statement in the former. The reason they split up is historical, and because the "ci/install-dependencies.sh" used "ci/lib.sh", which requires "bash". At least one of the docker containers doesn't have "bash". To make the existing code POSIX-compatible we need to replace pushd/popd with a sub-shell, but no other changes were needed. This also fixes issue in "ci/install-docker-dependencies.sh" where we'd hide errors due to not using "set -e". Now that we include "ci/lib.sh" we won't have that potential issue. Signed-off-by: Ævar Arnfjörð Bjarmason --- .github/workflows/main.yml | 2 +- ci/install-dependencies.sh | 27 ++++++++++++++++++++++----- ci/install-docker-dependencies.sh | 22 ---------------------- 3 files changed, 23 insertions(+), 28 deletions(-) delete mode 100755 ci/install-docker-dependencies.sh diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 92b914f16fd..3fd3e4f873c 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -311,7 +311,7 @@ jobs: container: ${{matrix.vector.image}} steps: - uses: actions/checkout@v1 - - run: ci/install-docker-dependencies.sh + - run: ci/install-dependencies.sh - run: ci/lib.sh - run: make - run: make test diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh index 01006379fd0..9388289d0ed 100755 --- a/ci/install-dependencies.sh +++ b/ci/install-dependencies.sh @@ -1,4 +1,4 @@ -#!/usr/bin/env bash +#!/bin/sh # # Install dependencies required to build and test Git on Linux and macOS # @@ -34,18 +34,20 @@ ubuntu-latest) sudo apt-get -q -y install language-pack-is libsvn-perl apache2 \ $UBUNTU_COMMON_PKGS $CC_PACKAGE mkdir --parents "$P4_PATH" - pushd "$P4_PATH" + ( + cd "$P4_PATH" wget --quiet "$P4WHENCE/bin.linux26x86_64/p4d" wget --quiet "$P4WHENCE/bin.linux26x86_64/p4" chmod u+x p4d chmod u+x p4 - popd + ) mkdir --parents "$GIT_LFS_PATH" - pushd "$GIT_LFS_PATH" + ( + cd "$GIT_LFS_PATH" wget --quiet "$LFSWHENCE/git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" tar --extract --gunzip --file "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz" cp git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs . - popd + ) ;; macos-latest) export HOMEBREW_NO_AUTO_UPDATE=1 HOMEBREW_NO_INSTALL_CLEANUP=1 @@ -90,6 +92,21 @@ linux-gcc-default) sudo apt-get -q update sudo apt-get -q -y install $UBUNTU_COMMON_PKGS ;; +linux32) + linux32 --32bit i386 sh -c ' + apt update >/dev/null && + apt install -y build-essential libcurl4-openssl-dev \ + libssl-dev libexpat-dev gettext python >/dev/null + ' + ;; +linux-musl) + apk add --update build-base curl-dev openssl-dev expat-dev gettext \ + pcre2-dev python3 musl-libintl perl-utils ncurses >/dev/null + ;; +pedantic) + dnf -yq update >/dev/null && + dnf -yq install make gcc findutils diffutils perl python3 gettext zlib-devel expat-devel openssl-devel curl-devel pcre2-devel >/dev/null + ;; esac if type p4d >/dev/null && type p4 >/dev/null diff --git a/ci/install-docker-dependencies.sh b/ci/install-docker-dependencies.sh deleted file mode 100755 index 78b7e326da6..00000000000 --- a/ci/install-docker-dependencies.sh +++ /dev/null @@ -1,22 +0,0 @@ -#!/bin/sh -# -# Install dependencies required to build and test Git inside container -# - -case "$jobname" in -linux32) - linux32 --32bit i386 sh -c ' - apt update >/dev/null && - apt install -y build-essential libcurl4-openssl-dev \ - libssl-dev libexpat-dev gettext python >/dev/null - ' - ;; -linux-musl) - apk add --update build-base curl-dev openssl-dev expat-dev gettext \ - pcre2-dev python3 musl-libintl perl-utils ncurses >/dev/null - ;; -pedantic) - dnf -yq update >/dev/null && - dnf -yq install make gcc findutils diffutils perl python3 gettext zlib-devel expat-devel openssl-devel curl-devel pcre2-devel >/dev/null - ;; -esac From patchwork Mon Feb 21 14:46:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12753721 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8323BC433EF for ; Mon, 21 Feb 2022 14:47:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378349AbiBUOsL (ORCPT ); Mon, 21 Feb 2022 09:48:11 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:60434 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1378303AbiBUOra (ORCPT ); Mon, 21 Feb 2022 09:47:30 -0500 Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 46E4A6312 for ; Mon, 21 Feb 2022 06:47:05 -0800 (PST) Received: by mail-wr1-x42c.google.com with SMTP id m27so15712661wrb.4 for ; Mon, 21 Feb 2022 06:47:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=8iuq08/hgZpKBlsobR/vQylO/sPOU3RAd1XaudbFbSk=; b=Nr7dLjCSEwwYki2a8GNaLDl9kWMLZv52fFYpl/2t2eqb/RzNSMLZahvZzv6tdZDQze n/4vvo0ua+xxUA7Ok2DxKGNXOauxIxL+q1pBaW8EahEccCMMDAgEBKI0yZjdrIH2VeN7 0TftW+XpmZjLfdGB6Jy2pxBgG5PYtNWMSonb1X1clkOR/I4Cb8Rmuu9s2cSiv0CmWIEk gmC+54/eltO+ahzCnmDkobUC+EXq+jBISHWH2k857EVfkB4sHgmcig7WvdEsv+Xip8lx KW96rkSG9Krbv9dXkwtZo6GtAC4b++1NVw7lo3/Qzs61S5VDwRrVTAoRZ1OlVNoNSOlX OBRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=8iuq08/hgZpKBlsobR/vQylO/sPOU3RAd1XaudbFbSk=; b=Lc3QGfVqQWReqls8RKL4Rt1BLvd8tNk/kjql6gcG90SrEgKO+8Fa6/SQH8ri6+9YkC /aeawispq49in1Ao+4GHmK5DU7b0Vh8K+Jwy56WdJqv8CwXBVk9EFb+19SH2XRnh73kI YPuCU+uXJwK+qYERor1gCB1JcKTPsaIMF9YJCsnjwShS74T2P0rbMhjVW+hceNTUzyj3 +VEXBCJJrBwIl56ewiLfSsR7w7g6NZwMPBaDE1NFiQP0QGBPOfvj7/ZGbqLfU9MqC0DR A2M9mXZGlJoX4PwxmcaYXbYj0OPoLcuO9vqqdLaMQq1QAKk11MK9nUNQ0HOub1C+Mo6W HInw== X-Gm-Message-State: AOAM530o8CpvNYhKeqBNtbhoRJRpbxuNvnJyomHZw8n5c9NS9aTiUng+ 2rtl+lArfSgnE61Kmzja8bJPt3pu7PDRnw== X-Google-Smtp-Source: ABdhPJy1iQ8nmacHT8ys2WiLY5QGh61fFFO4Ul3a648lObgHnUkDsyWvq2zclRZ08ekJqlngqDKiDA== X-Received: by 2002:adf:dc90:0:b0:1e4:a5fe:9b42 with SMTP id r16-20020adfdc90000000b001e4a5fe9b42mr15977832wrj.98.1645454823588; Mon, 21 Feb 2022 06:47:03 -0800 (PST) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id r14sm25071094wrz.84.2022.02.21.06.47.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Feb 2022 06:47:03 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH 17/25] CI: move "env" definitions into ci/lib.sh Date: Mon, 21 Feb 2022 15:46:29 +0100 Message-Id: X-Mailer: git-send-email 2.35.1.1132.ga1fe46f8690 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Have the GitHub CI jobs use the "ci/lib".sh as a source of truth for environment variables that affect their builds and tests. This brings them in line with the rest of the jobs, and make it easier to see the entire CI configuration per jobname at a glance. To do this we need to add a ci/lib.sh "step" to the remaining jobs that didn't have it added in preceding commits. The Makefile parameters are luckily all accepted via the environment, so we can export these instead of adding them to MAKEFLAGS. Let's also use the documented GIT_CONIFG_* mechanism for setting config instead of the internal GIT_CONFIG_PARAMETERS variable. This adjusts code added in 889cacb6897 (ci: configure GitHub Actions for CI/PR, 2020-04-11), we could probably use the GIT_AUTHOR_NAME etc, but let's keep setting this via config, just with the documented mechanism. By setting "ARTIFACTS_DIRECTORY=artifacts" once we don't need to repeat it in various places, let's get it from the environment instead. Finally, the "DEVELOPER: 1" was only needed by the "sparse" job, which wasn't using "ci/lib.sh" (and which sets DEVELOPER=1). Let's instead have the "sparse" job use the "ci/lib.sh" and remove DEVELOPER=1 from ".github/workflows/main.yml". This substantially speeds up the "sparse" job, since it'll now pick up the "--jobs" setting in MAKEFLAGS that we use everywhere else. Before it ran in around 4m30s, now in around 2m30s. Signed-off-by: Ævar Arnfjörð Bjarmason --- .github/workflows/main.yml | 29 +++++++++++------------------ ci/lib.sh | 21 +++++++++++++++++++++ 2 files changed, 32 insertions(+), 18 deletions(-) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 3fd3e4f873c..fe37e0f1b36 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -2,9 +2,6 @@ name: CI on: [push, pull_request] -env: - DEVELOPER: 1 - jobs: ci-config: name: config @@ -93,8 +90,7 @@ jobs: shell: bash env: HOME: ${{runner.workspace}} - NO_PERL: 1 - run: . /etc/profile && make artifacts-tar ARTIFACTS_DIRECTORY=artifacts + run: . /etc/profile && make artifacts-tar if: success() - run: ci/check-unignored-build-artifacts.sh if: success() @@ -105,7 +101,7 @@ jobs: uses: actions/upload-artifact@v2 with: name: windows-artifacts - path: artifacts + path: ${{env.ARTIFACTS_DIRECTORY}} windows-test: name: win test env: @@ -126,6 +122,8 @@ jobs: shell: bash run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz - uses: git-for-windows/setup-git-for-windows-sdk@v1 + - run: ci/lib.sh + shell: bash - name: test shell: bash run: . /etc/profile && ci/run-test-slice.sh ${{matrix.nr}} 10 @@ -145,9 +143,6 @@ jobs: jobname: vs-build needs: ci-config if: needs.ci-config.outputs.enabled == 'yes' - env: - NO_PERL: 1 - GIT_CONFIG_PARAMETERS: "'user.name=CI' 'user.email=ci@git'" runs-on: windows-latest steps: - uses: actions/checkout@v2 @@ -171,6 +166,8 @@ jobs: - name: copy dlls to root shell: cmd run: compat\vcbuild\vcpkg_copy_dlls.bat release + - run: ci/lib.sh + shell: bash - name: generate Visual Studio solution shell: bash run: | @@ -181,21 +178,16 @@ jobs: - name: bundle artifact tar shell: bash env: - MSVC: 1 VCPKG_ROOT: ${{github.workspace}}\compat\vcbuild\vcpkg run: | - mkdir -p artifacts && - eval "$(make -n artifacts-tar INCLUDE_DLLS_IN_ARTIFACTS=YesPlease ARTIFACTS_DIRECTORY=artifacts NO_GETTEXT=YesPlease 2>&1 | grep ^tar)" - - run: ci/check-unignored-build-artifacts.sh - if: success() - shell: bash + eval "$(make -n artifacts-tar 2>&1 | grep -e ^mkdir -e ^tar)" - name: zip up tracked files run: git archive -o artifacts/tracked.tar.gz HEAD - name: upload tracked files and build artifacts uses: actions/upload-artifact@v2 with: name: vs-artifacts - path: artifacts + path: ${{env.ARTIFACTS_DIRECTORY}} vs-test: name: win+VS test env: @@ -216,10 +208,10 @@ jobs: - name: extract tracked files and build artifacts shell: bash run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz + - run: ci/lib.sh + shell: bash - name: test shell: bash - env: - NO_SVN_TESTS: 1 run: . /etc/profile && ci/run-test-slice.sh ${{matrix.nr}} 10 - name: ci/print-test-failures.sh if: failure() @@ -354,6 +346,7 @@ jobs: - uses: actions/checkout@v2 - name: Install other dependencies run: ci/install-dependencies.sh + - run: ci/lib.sh - run: make sparse documentation: name: documentation diff --git a/ci/lib.sh b/ci/lib.sh index fc6ce4d3e04..b882849ed5d 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -95,6 +95,27 @@ macos-latest) esac case "$jobname" in +windows-build) + setenv --build NO_PERL NoThanks + setenv --build ARTIFACTS_DIRECTORY artifacts + ;; +vs-build) + setenv --build NO_PERL NoThanks + setenv --build NO_GETTEXT NoThanks + setenv --build ARTIFACTS_DIRECTORY artifacts + setenv --build INCLUDE_DLLS_IN_ARTIFACTS YesPlease + setenv --build MSVC YesPlease + + setenv --build GIT_CONFIG_COUNT 2 + setenv --build GIT_CONFIG_KEY_0 user.name + setenv --build GIT_CONFIG_VALUE_0 CI + setenv --build GIT_CONFIG_KEY_1 user.emailname + setenv --build GIT_CONFIG_VALUE_1 ci@git + setenv --build GIT_CONFIG_VALUE_1 ci@git + ;; +vs-test) + setenv --test NO_SVN_TESTS YesPlease + ;; linux-gcc) setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main ;; From patchwork Mon Feb 21 14:46:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12753720 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 69A9BC433EF for ; Mon, 21 Feb 2022 14:47:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378347AbiBUOsG (ORCPT ); Mon, 21 Feb 2022 09:48:06 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:60536 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1378319AbiBUOra (ORCPT ); Mon, 21 Feb 2022 09:47:30 -0500 Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3C2456315 for ; Mon, 21 Feb 2022 06:47:06 -0800 (PST) Received: by mail-wm1-x32b.google.com with SMTP id p14-20020a05600c1d8e00b0037f881182a8so1530280wms.2 for ; Mon, 21 Feb 2022 06:47:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=RH3YHdyRn6SczNP3Jk6YG7loMe+6cfAyl+F50M5kPPQ=; b=m5EKPW/wYpsPrh5ZFrzBXB0KVToNJIy8lh2jLJoUcOrebZHKO93xmKv25hxX9Y5UOQ X+ipRkXKPjk+LDgNw61fduTLW3N3V6gaQ1uRW4DOy/giY+cPOKQztW4eIAs5v4oe2tL7 b/qfpyYieoMWNmjWBo64cKYnNg1FvhV/GZVipxAheXW5VAz/9uAIWDdf+7EuEIn/JN79 QPmXy0KuFYIjn/xkdocW+OQ3YdN3Nz/gYGJu6KPVtB8MSZIcORMCF6oOowRFV/HrErLV TL2FarGDOz7smu9QhfERl3OySJ77GVfzMUJHO4Cms8p5KX2//q7eLeLKtwEYQJCR2Rh1 K85A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=RH3YHdyRn6SczNP3Jk6YG7loMe+6cfAyl+F50M5kPPQ=; b=qbpuvDaP7bqbpqcs1w9UBiABP87B3kB8tHT3rq6I3nVW7yfRKs92vudW2D2ldAWF7N v0lOS8apnW+DOtvnb/xxVq3kiiTWbXjCnZ0w4bVaDjQ0jeJhgkWK6TrIPINBkNk+j5QP E04/EaAzqILk/aeLhM3+OZr80nz0WjyjUpMwZ72oFNZKqnk2b7msFVXEvSf6XvUJ4Oqg 8kNDThPf6xIk22PcSZ06bR79a0XelS78X3gh3Vn5dTNaR54/SiPNAw7WmihPKEHX7IFy 1+AXzJqJSmu/0SGByNvqHzNKhdPQ3qFHG8ugoM1XsVA7ohTjkpUMXjXjUdzFbkPMXo/A quWQ== X-Gm-Message-State: AOAM533xxKzeAije0WRTwji9oMUW4INb+AFkK/XNag73eZAzmbpOwI4x j2pnrgEQsuO4YHvFEy6ZOiEB6rF7p0uePA== X-Google-Smtp-Source: ABdhPJy6rLRuJelwNEYgzcqKIn2uVXlbKgNCnVbKjdTEvKygot6AO8WUFtSp0a7zHbn5NED7HI0VkA== X-Received: by 2002:a1c:ed1a:0:b0:37f:a539:5d5e with SMTP id l26-20020a1ced1a000000b0037fa5395d5emr1871033wmh.37.1645454824513; Mon, 21 Feb 2022 06:47:04 -0800 (PST) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id r14sm25071094wrz.84.2022.02.21.06.47.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Feb 2022 06:47:03 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH 18/25] ci/run-test-slice.sh: replace shelling out with "echo" Date: Mon, 21 Feb 2022 15:46:30 +0100 Message-Id: X-Mailer: git-send-email 2.35.1.1132.ga1fe46f8690 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Amend this code added in b819f1d2cec (ci: parallelize testing on Windows, 2019-01-29) to stop shelling out to "tr", all we're trying to do here is to turn '\n'-delimited output into a ' '-delimited list. To do that we can just use "echo". None of these files will contain the odd characters using "echo" might choke on in the general case (i.e. '\'-escapes and the like). This has the added benefit of not adding a redundant ' ' to the end of the T-parameter, which didn't matter in practice, but as we'll see in a subsequent commit makes for slightly nicer output, as we'll show this variable using a GitHub CI-native feature. Signed-off-by: Ævar Arnfjörð Bjarmason --- ci/run-test-slice.sh | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/ci/run-test-slice.sh b/ci/run-test-slice.sh index 70326961454..1de6a18ca47 100755 --- a/ci/run-test-slice.sh +++ b/ci/run-test-slice.sh @@ -5,6 +5,6 @@ . ${0%/*}/lib.sh -make --quiet -C t T="$(cd t && - ./helper/test-tool path-utils slice-tests "$1" "$2" t[0-9]*.sh | - tr '\n' ' ')" +tests=$(echo $(cd t && ./helper/test-tool path-utils slice-tests "$1" "$2" \ + t[0-9]*.sh)) +make --quiet -C t T="$tests" From patchwork Mon Feb 21 14:46:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12753718 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 15A3CC433EF for ; Mon, 21 Feb 2022 14:47:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378333AbiBUOsA (ORCPT ); Mon, 21 Feb 2022 09:48:00 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:60440 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1378321AbiBUOrb (ORCPT ); Mon, 21 Feb 2022 09:47:31 -0500 Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 34FB45F80 for ; Mon, 21 Feb 2022 06:47:07 -0800 (PST) Received: by mail-wr1-x429.google.com with SMTP id v12so27530417wrv.2 for ; Mon, 21 Feb 2022 06:47:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=XiNkijEQsLOHcXv/fQRjmaFqvIzmZ+P0xzLPc9nIq7Q=; b=CqTaO2gtwKJ0erOd2u+ps3LRjAPXhlw2mwR00TgMhFXTvI0uHU8sNERN6PmXPoSO4T UoKmvh00yAe8jXFZJAr7z3LNoYNOepyt1tQpEozbS52jhz6+gmdf8n926zPHOoH9I4Fj xm5Nj1O5DTeody/KiU2+feY5q4T6crKQFKuXXnWpizGNiDzIsyDg1ucapnETk6I5dRrr r3f/Ct8bWE+a06KkppTLhvIXB2lWuCdIu0tTnCD1kydYKEFQvM/6t5LpXquZ3j2BcO46 AB8VomWrKb+hPlBYLRqmcl314frD9uG/9CaxOPp1QiGXBqoXhYokeIcB6UFEuQlymCTL hjRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XiNkijEQsLOHcXv/fQRjmaFqvIzmZ+P0xzLPc9nIq7Q=; b=oZMAdOiWU+7H+y23BwjtpKzWOAzhtUrrTuFpyyCO5BdU+Gq0R5pd9rUiv+tDfN2kDI swXghu2ozmrb0ZP8dVsDiNrUY5Grb68cyWQTEIk43PvZ3uhHT6GsQ+oQgO8qfnWAU6rs jbAm7u4k0vqxqUXInMYsQbx+10H8xSbktjzL6q9KpAz3NlpEc84k+8XHp8O4Mkpp1Msf T1nk/7dOqfCSahW3PpwM48bueVsaoQ8EpQoAzg+De4O5X9pshcKnADt13cDM7sFoTQjj Wd5iJrYXcpC2reih3kYG+kJu0O0dO7zvnLieF6PCN8TA87sLNJnmNi2PvpfdfnVYzGsI GBtA== X-Gm-Message-State: AOAM5315XgCvQXJ020ws/XAO7mPJM3EAWtGXJrerGKrnvCc5MqaQucGh Mpub870V/MmQKwlCa7CzhEQigIIduo+vUg== X-Google-Smtp-Source: ABdhPJwqo86IwrFyGgv33KoZ6XcVWvKrlfZta9NybuQiXhldXLFzJQWAjz1LuXMPvwwA/rWaYslq3w== X-Received: by 2002:a05:6000:184d:b0:1ea:78a5:f9c8 with SMTP id c13-20020a056000184d00b001ea78a5f9c8mr1037252wri.439.1645454825482; Mon, 21 Feb 2022 06:47:05 -0800 (PST) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id r14sm25071094wrz.84.2022.02.21.06.47.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Feb 2022 06:47:04 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH 19/25] CI: pre-select test slice in Windows & VS tests Date: Mon, 21 Feb 2022 15:46:31 +0100 Message-Id: X-Mailer: git-send-email 2.35.1.1132.ga1fe46f8690 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In preceding commits the tests have been changed to do their setup via $GITHUB_ENV in one step, and to then have subsequent steps that re-use that environment. Let's change the "test slice" tests added in b819f1d2cec (ci: parallelize testing on Windows, 2019-01-29) to do the same. These tests select 10% of the tests to run in 10 "test slices". Now we'll select those in a step that immediately precedes the testing step, and then simply invoke "make -C t -e". This has the advantage that the tests to be run are now listed in the standard "Run" drop-down at the start of the "test" step. Since the "T" variable in "t/Makefile" doesn't normally accept overrides from the environment we need to invoke "make" with the "-e" option (a.k.a. "--environment-overrides"). We could also make $(T) in t/Makefile be a "?=" assigned variable, but this way works, and is arguably clearer as it's more obvious that we're injecting a special list of tests that override the normal behavior of that Makefile. Note that we cannot run the top-level "make test" here, because of how the Windows CI builds git, i.e. either via CMake or some option that would cause "make test" to recompile git itself. Instead we run "make -C t [...]". Signed-off-by: Ævar Arnfjörð Bjarmason --- .github/workflows/main.yml | 10 ++++++++-- ci/{run-test-slice.sh => select-test-slice.sh} | 4 ++-- 2 files changed, 10 insertions(+), 4 deletions(-) rename ci/{run-test-slice.sh => select-test-slice.sh} (57%) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index fe37e0f1b36..1e13718b989 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -124,9 +124,12 @@ jobs: - uses: git-for-windows/setup-git-for-windows-sdk@v1 - run: ci/lib.sh shell: bash + - name: select tests + run: . /etc/profile && ci/select-test-slice.sh ${{matrix.nr}} 10 + shell: bash - name: test shell: bash - run: . /etc/profile && ci/run-test-slice.sh ${{matrix.nr}} 10 + run: . /etc/profile && make -C t -e - name: ci/print-test-failures.sh if: failure() shell: bash @@ -210,9 +213,12 @@ jobs: run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz - run: ci/lib.sh shell: bash + - name: select tests + run: . /etc/profile && ci/select-test-slice.sh ${{matrix.nr}} 10 + shell: bash - name: test shell: bash - run: . /etc/profile && ci/run-test-slice.sh ${{matrix.nr}} 10 + run: . /etc/profile && make -C t -e - name: ci/print-test-failures.sh if: failure() shell: bash diff --git a/ci/run-test-slice.sh b/ci/select-test-slice.sh similarity index 57% rename from ci/run-test-slice.sh rename to ci/select-test-slice.sh index 1de6a18ca47..ec602f8a053 100755 --- a/ci/run-test-slice.sh +++ b/ci/select-test-slice.sh @@ -1,10 +1,10 @@ #!/bin/sh # -# Test Git in parallel +# Select a portion of the tests for testing Git in parallel # . ${0%/*}/lib.sh tests=$(echo $(cd t && ./helper/test-tool path-utils slice-tests "$1" "$2" \ t[0-9]*.sh)) -make --quiet -C t T="$tests" +echo T="$tests" >>$GITHUB_ENV From patchwork Mon Feb 21 14:46:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12753726 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3CCEFC433FE for ; Mon, 21 Feb 2022 14:48:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378358AbiBUOs2 (ORCPT ); Mon, 21 Feb 2022 09:48:28 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:60610 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1378307AbiBUOrb (ORCPT ); Mon, 21 Feb 2022 09:47:31 -0500 Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2C60138A5 for ; Mon, 21 Feb 2022 06:47:08 -0800 (PST) Received: by mail-wr1-x42f.google.com with SMTP id d27so27513611wrc.6 for ; Mon, 21 Feb 2022 06:47:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=3m1ruZuIYSdJ29PuDQY0iyz4mGK4VGkNcg2pJ9x2hn0=; b=jQVlFQEf4Gvi3FApB/hYlNJzox0vo7yiEVr+OJAdE2jX4PJ3dKfNkntzWyltNBuw6V pw5kj5xpprRjshFy95gSeVo9mWvhmsLSFs1Ch+IAPSyxNdb76HiEQNgARd4zq71dbzIM mixYTLeffC+AnNXqlZmFJFG28EP7dDKgUpviQdy8gBiNoxDJ0tLQK58QNQlTs9+EeM3x K/QYgnn2knlqNte9KeTu4/rA3NK2E7pPMjXdEqwXCjnR/HUfWmiZp9XK8iKc9On91tCk rrorjXGmRo+9VefhXJciwmCJWoM5ZIExNfJXKzIGEE7co/MlJBJGNTeESzVABHSIXTt4 pF6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3m1ruZuIYSdJ29PuDQY0iyz4mGK4VGkNcg2pJ9x2hn0=; b=S5aaetcV00X0qgeB8MtOJ7LPZViv4Y7eX0sTLHrV+NPwA93VwrLwU0GE9zHI57TXUU jShAf/zjUSgkkTsO1yHqYOu1NPz8mi2FYam9AX8wdK/wN38bOpjRvp1mNBwujRJrXbnl ULwRFTZawTU5IUw5TXSmYC1myi7wHNXMmW/ygBcMkv8jc7M2CyTMAGpiWDnHJiJnoIt8 EQcexnexyFOi6GbWnkmDXTSog4yi7XTm9wNHXC2YTix2+O5vQ6YcSdwBCbkw9gmktRao S2jJBlO8L5g+fvmIJaRXhKz+/F7fXbqHLVQFw56e3kiYT0Hkd7fOgrVHmITewKjpjKOU CAGA== X-Gm-Message-State: AOAM531BGJeDnpaWpj9/ZAjZ6JYRIpbo4m1eIFJBmBNjoSoPIRQQhRJC 7RETywsW6ZMT4fO4x2Uj3dBzAK6Qwh1AUQ== X-Google-Smtp-Source: ABdhPJxOlO/skdVr4LPTqWT5SrUJqS2rw8joc1O4fuxqWxaLzGcJij7VqLX2G09wS0Tf6sWxT4VgIQ== X-Received: by 2002:adf:e5d1:0:b0:1e6:1109:5a11 with SMTP id a17-20020adfe5d1000000b001e611095a11mr16065730wrn.641.1645454826480; Mon, 21 Feb 2022 06:47:06 -0800 (PST) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id r14sm25071094wrz.84.2022.02.21.06.47.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Feb 2022 06:47:05 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH 20/25] CI: only invoke ci/lib.sh as "steps" in main.yml Date: Mon, 21 Feb 2022 15:46:32 +0100 Message-Id: X-Mailer: git-send-email 2.35.1.1132.ga1fe46f8690 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Change the scripts in ci/ to stop using ci/lib.sh as a library, now that the only thing it did for them was to "set -ex" and possibly set TERM=dumb. Let's create a ci/lib-tput.sh for those that need to use "tput" instead, and have these scripts invoke "set -ex" themselves. This makes their invocation a lot less verbose, since they'll be relying on an earlier step in the CI job to have set the variables in $GITHUB_ENV, and won't be spewing their own trace output to set those variables again. Let's also create a ci/lib-ci-type.sh, and have ci/lib.sh and ci/print-test-failures.sh share the logic to discover the CI type. We could have set the CI_TYPE in the environment with "setenv", but let's avoid that verbosity for this purely internal variable. The "ci/lib.sh" is now no longer a "Library of functions shared by all CI scripts", so let's remove that commentary, and the misleading comment about "set -ex" being for "installing dependencies", we're now no longer using it in "ci/install-dependencies.sh" (but it does its own "set -ex"). Signed-off-by: Ævar Arnfjörð Bjarmason --- ci/check-unignored-build-artifacts.sh | 4 +++- ci/install-dependencies.sh | 2 +- ci/lib-ci-type.sh | 8 ++++++++ ci/lib-tput.sh | 2 ++ ci/lib.sh | 27 ++++++++++----------------- ci/print-test-failures.sh | 6 +++--- ci/select-test-slice.sh | 2 +- ci/test-documentation.sh | 2 +- 8 files changed, 29 insertions(+), 24 deletions(-) create mode 100644 ci/lib-ci-type.sh create mode 100644 ci/lib-tput.sh diff --git a/ci/check-unignored-build-artifacts.sh b/ci/check-unignored-build-artifacts.sh index 0bc04f32804..c27d6a97f45 100755 --- a/ci/check-unignored-build-artifacts.sh +++ b/ci/check-unignored-build-artifacts.sh @@ -3,7 +3,9 @@ # Check whether the build created anything not in our .gitignore # -. ${0%/*}/lib.sh +set -ex + +. ${0%/*}/lib-tput.sh check_unignored_build_artifacts () { diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh index 9388289d0ed..ee9af62fc92 100755 --- a/ci/install-dependencies.sh +++ b/ci/install-dependencies.sh @@ -3,7 +3,7 @@ # Install dependencies required to build and test Git on Linux and macOS # -. ${0%/*}/lib.sh +set -ex UBUNTU_COMMON_PKGS="make libssl-dev libcurl4-openssl-dev libexpat-dev tcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl diff --git a/ci/lib-ci-type.sh b/ci/lib-ci-type.sh new file mode 100644 index 00000000000..bd6e093c8f4 --- /dev/null +++ b/ci/lib-ci-type.sh @@ -0,0 +1,8 @@ +if test "$GITHUB_ACTIONS" = "true" +then + CI_TYPE=github-actions +else + echo "Could not identify CI type" >&2 + env >&2 + exit 1 +fi diff --git a/ci/lib-tput.sh b/ci/lib-tput.sh new file mode 100644 index 00000000000..baed1892f69 --- /dev/null +++ b/ci/lib-tput.sh @@ -0,0 +1,2 @@ +# GitHub Action doesn't set TERM, which is required by tput +export TERM=${TERM:-dumb} diff --git a/ci/lib.sh b/ci/lib.sh index b882849ed5d..5eadc96de29 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -1,13 +1,9 @@ #!/bin/sh - -# Library of functions shared by all CI scripts - -# Set 'exit on error' for all CI scripts to let the caller know that -# something went wrong. -# Set tracing executed commands, primarily setting environment variables -# and installing dependencies. set -ex +# Helper libraries +. ${0%/*}/lib-ci-type.sh + # Starting assertions if test -z "$jobname" then @@ -39,15 +35,11 @@ setenv () { fi } -# GitHub Action doesn't set TERM, which is required by tput -setenv TERM ${TERM:-dumb} - # Clear MAKEFLAGS that may come from the outside world. MAKEFLAGS= -if test "$GITHUB_ACTIONS" = "true" -then - CI_TYPE=github-actions +case "$CI_TYPE" in +github-actions) CC="${CC:-gcc}" setenv --test GIT_PROVE_OPTS "--timer --jobs 10" @@ -57,11 +49,12 @@ then GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS" setenv --test GIT_TEST_OPTS "$GIT_TEST_OPTS" -else - echo "Could not identify CI type" >&2 - env >&2 + ;; +*) + echo "Unhandled CI type: $CI_TYPE" >&2 exit 1 -fi + ;; +esac setenv --build DEVELOPER 1 setenv --test DEFAULT_TEST_TARGET prove diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh index 0c63b6f7962..452aff35d74 100755 --- a/ci/print-test-failures.sh +++ b/ci/print-test-failures.sh @@ -3,10 +3,10 @@ # Print output of failing tests # -. ${0%/*}/lib.sh +set -e -# Tracing executed commands would produce too much noise in the loop below. -set +x +. ${0%/*}/lib-ci-type.sh +. ${0%/*}/lib-tput.sh cd t/ diff --git a/ci/select-test-slice.sh b/ci/select-test-slice.sh index ec602f8a053..a0332095bb3 100755 --- a/ci/select-test-slice.sh +++ b/ci/select-test-slice.sh @@ -3,7 +3,7 @@ # Select a portion of the tests for testing Git in parallel # -. ${0%/*}/lib.sh +set -ex tests=$(echo $(cd t && ./helper/test-tool path-utils slice-tests "$1" "$2" \ t[0-9]*.sh)) diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh index b8a6a6f664e..64ff212cdaa 100755 --- a/ci/test-documentation.sh +++ b/ci/test-documentation.sh @@ -3,7 +3,7 @@ # Perform sanity checks on "make doc" output and built documentation # -. ${0%/*}/lib.sh +set -ex generator=$1 From patchwork Mon Feb 21 14:46:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12753727 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EB0E3C433F5 for ; Mon, 21 Feb 2022 14:48:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378359AbiBUOsb (ORCPT ); Mon, 21 Feb 2022 09:48:31 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:60688 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1378308AbiBUOrc (ORCPT ); Mon, 21 Feb 2022 09:47:32 -0500 Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 162665FA8 for ; Mon, 21 Feb 2022 06:47:09 -0800 (PST) Received: by mail-wr1-x431.google.com with SMTP id v12so27530552wrv.2 for ; Mon, 21 Feb 2022 06:47:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=9t4ZR/XM6WuDOTGpvUba9iAvgSF5eJXl8U9AFZQJ59w=; b=YNb+SC86CrvxOhw08ZNBloJ9pOUEblIVphJnUIvCN3QTXJYAtplkNq+GAdD4/61THO nhvafzuIEgDGizHZt/QGFFhAGtgi2llztAXgJ7WTgF8CUfSEAHl2A7kjOV7R0UT36UAb 1JbMiTcGQxOFt0NmkRJ73g04biXCMEHrSIM+FDOw5MxnpU3HmUFm4Nclnnt8CAobAmcN qyc00Xq4opR60VbnfYU2RNyqlSH+IwJU4xy/rE4K1pVXFTqt29K4j5DI9R4IimplHSlY F6mF0j7fPzAXIqhCHt9Rel4VVQWAsUv7gE2VrbhlAMliuHoWeokuMvOQr2rIBBdpCPKI FDdg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9t4ZR/XM6WuDOTGpvUba9iAvgSF5eJXl8U9AFZQJ59w=; b=Anag4ArM+wIn067wSn3qb6QEaNXlg4y2YqBqiRk0VsWwOFbcVBgnliisgw1EVt4Ftd nhQnp9KaaVkSCaXAGOnYr8A8svtrc/kU4PKqDo5owGL3iqVg4NioMkjiQBTSK1k/9okH D5BzISYZy9+12DvYJcCbTpDzpXdsWP94cGOxzxoWMp133GeYUqvabmfsHYwEu3E3yVgD 4OJMVacTsxZGMlPNMayQ+pQtDnk/vWffc/l64dUHbd1AzNNl+vyNTj2P/PL9/5/3e/p8 dlJfbsa9J00ZMMVlurq7ml1J6GAxzxUByMfm3vquKLp99HjYOh89qLQwaZQt3Rev/tG/ 3n/A== X-Gm-Message-State: AOAM532LsBGmQkk5DjyqTZ8Yo2oWllQdiKGQETC5/5bxnb+GjW/nQ8h/ IabxwaSvtA32HZPBJiqVRRRumQMX2z0apQ== X-Google-Smtp-Source: ABdhPJyigpZio2J5IReMh5R8iGVxfSR/SlQ4ypg5omUSIr0Suzey9dD1Y4caqRLt/u+0+aCZDftPaw== X-Received: by 2002:adf:9141:0:b0:1e3:1379:4cc6 with SMTP id j59-20020adf9141000000b001e313794cc6mr16210512wrj.249.1645454827410; Mon, 21 Feb 2022 06:47:07 -0800 (PST) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id r14sm25071094wrz.84.2022.02.21.06.47.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Feb 2022 06:47:06 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH 21/25] CI: narrow down variable definitions in --build and --test Date: Mon, 21 Feb 2022 15:46:33 +0100 Message-Id: X-Mailer: git-send-email 2.35.1.1132.ga1fe46f8690 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In a preceding step the "setvar" function was made to take a "--build", "--test" or "--all" argument to indicate where the variables it sets were used. Let's make use of that by having the relevant parts of ".github/workflows/main.yml" invoke "ci/lib.sh" with those options. By doing this the set of variables shown in build-only steps will be fewer, which makes diagnosing anything going on there easier, as we won't have to look at a deluge of e.g. GIT_TEST_* variables. Signed-off-by: Ævar Arnfjörð Bjarmason --- .github/workflows/main.yml | 20 +++++++++++--------- ci/lib.sh | 17 +++++++++++++++++ 2 files changed, 28 insertions(+), 9 deletions(-) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 1e13718b989..0787cadc76b 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -84,7 +84,7 @@ jobs: steps: - uses: actions/checkout@v2 - uses: git-for-windows/setup-git-for-windows-sdk@v1 - - run: ci/lib.sh + - run: ci/lib.sh --build shell: bash - name: build shell: bash @@ -122,7 +122,7 @@ jobs: shell: bash run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz - uses: git-for-windows/setup-git-for-windows-sdk@v1 - - run: ci/lib.sh + - run: ci/lib.sh --test shell: bash - name: select tests run: . /etc/profile && ci/select-test-slice.sh ${{matrix.nr}} 10 @@ -169,7 +169,7 @@ jobs: - name: copy dlls to root shell: cmd run: compat\vcbuild\vcpkg_copy_dlls.bat release - - run: ci/lib.sh + - run: ci/lib.sh --build shell: bash - name: generate Visual Studio solution shell: bash @@ -211,7 +211,7 @@ jobs: - name: extract tracked files and build artifacts shell: bash run: tar xf artifacts.tar.gz && tar xf tracked.tar.gz - - run: ci/lib.sh + - run: ci/lib.sh --test shell: bash - name: select tests run: . /etc/profile && ci/select-test-slice.sh ${{matrix.nr}} 10 @@ -275,8 +275,9 @@ jobs: steps: - uses: actions/checkout@v2 - run: ci/install-dependencies.sh - - run: ci/lib.sh + - run: ci/lib.sh --build - run: make + - run: ci/lib.sh --test - run: make test if: success() - run: ci/print-test-failures.sh @@ -310,8 +311,9 @@ jobs: steps: - uses: actions/checkout@v1 - run: ci/install-dependencies.sh - - run: ci/lib.sh + - run: ci/lib.sh --build - run: make + - run: ci/lib.sh --test - run: make test if: success() && matrix.vector.skip-tests != 'no' - run: ci/print-test-failures.sh @@ -331,7 +333,7 @@ jobs: steps: - uses: actions/checkout@v2 - run: ci/install-dependencies.sh - - run: ci/lib.sh + - run: ci/lib.sh --build - run: make ci-static-analysis sparse: needs: ci-config @@ -352,7 +354,7 @@ jobs: - uses: actions/checkout@v2 - name: Install other dependencies run: ci/install-dependencies.sh - - run: ci/lib.sh + - run: ci/lib.sh --build - run: make sparse documentation: name: documentation @@ -364,7 +366,7 @@ jobs: steps: - uses: actions/checkout@v2 - run: ci/install-dependencies.sh - - run: ci/lib.sh + - run: ci/lib.sh --build - run: make check-docs - run: "make doc > >(tee stdout.log) 2> >(tee stderr.raw >&2)" shell: bash diff --git a/ci/lib.sh b/ci/lib.sh index 5eadc96de29..367d1ee05d5 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -5,6 +5,13 @@ set -ex . ${0%/*}/lib-ci-type.sh # Starting assertions +mode=$1 +if test -z "$mode" +then + echo "need a $0 mode, e.g. --build or --test" + exit 1 +fi + if test -z "$jobname" then echo "must set a CI jobname" >&2 @@ -13,9 +20,14 @@ fi # Helper functions setenv () { + skip= varmode= case "$1" in --*) + if test "$1" != "$mode" && test "$1" != "--all" + then + skip=t + fi varmode=$1 shift ;; @@ -25,6 +37,11 @@ setenv () { val=$2 shift 2 + if test -n "$skip" + then + return 0 + fi + if test -n "$GITHUB_ENV" then echo "$key=$val" >>"$GITHUB_ENV" From patchwork Mon Feb 21 14:46:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12753728 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 160C1C433EF for ; Mon, 21 Feb 2022 14:48:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378361AbiBUOsg (ORCPT ); Mon, 21 Feb 2022 09:48:36 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:60808 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1378322AbiBUOre (ORCPT ); Mon, 21 Feb 2022 09:47:34 -0500 Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com [IPv6:2a00:1450:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 45E805FAD for ; Mon, 21 Feb 2022 06:47:09 -0800 (PST) Received: by mail-wr1-x433.google.com with SMTP id v12so27530619wrv.2 for ; Mon, 21 Feb 2022 06:47:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=btrXNaHGOrikC+owGmGo2S3bcWt3gzKHX9AgoRPRXj8=; b=nGpzRkqNhLx0c8b26eOLOjSWv5VPPtHB5Ecc3DbV6e0mHM2RPbcZbq+MMdqxkPhP5E huG28IHI18J9WmlC3lxpLwA+aVeWoZmUfmCmTruVPGGcOiMx01KLAUKMPqmTteIXjU59 odjC2uHhfwQOWcX1VjAmQddLawHq91V0n1oCuGlHyDUngP93Uas8YLngdDxBE2nJQgY7 dMLxjP6Xn0ZP+HKCGb9z+NSJ5+D/qbu0YB4dTFQfoYHQM0wktGsKWzgF/WJKnyIzYA9G 4GCm3tHmuyuyVO5vp7fPNkjekRw0y7zgCJDpFaQ49OUtxbKS4aaETCmiswama9DT3oK7 H99A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=btrXNaHGOrikC+owGmGo2S3bcWt3gzKHX9AgoRPRXj8=; b=FdSRCDbmOXW/naqjjzJZJt82ByI4+W0Hs47+RMtQnGgrVLOXtIA6PoGiONk6nF0v0h b6tW+uKEnPdMTCVKGh6v5YSSX/zbym2+X40quWgc+OBcxJkXffV8vbfceHeu1L5h75O3 Q/t0DOqx/j1L3Agnc5/lNScQ+m+7XCJ8l5BgJnk4oFJMMYuIxTHGsQkqEfp+rOkDS1vp QJw9jdK3qChIUxE+x0ypGbDwFYhZ1cRRWhiq2rEEghysshlW3vUVCgVZyqJlne0QC5F+ 8U7PBV3nw7L/kgCJptrv264X7Z/UduDrqhP8Y1Q+JctMoIknyg8oC81EgirCriR0TEeh YG3A== X-Gm-Message-State: AOAM530eX0iXzFJeYe+Nl2mnAwxPEMh6bpSMab/2BNxDR3RCoBSYMwAT uaRjnDTS0XmCa5FHQPEFXOGFSym3J2mlRg== X-Google-Smtp-Source: ABdhPJyDtQd1yegqjWFVquwp91VVf0QxNAtC87rqaOPCOEUf3SSQpB2wiDubbQMzxMvDywFfJoZYTA== X-Received: by 2002:a05:6000:178d:b0:1e8:310a:5a51 with SMTP id e13-20020a056000178d00b001e8310a5a51mr15920102wrg.642.1645454828307; Mon, 21 Feb 2022 06:47:08 -0800 (PST) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id r14sm25071094wrz.84.2022.02.21.06.47.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Feb 2022 06:47:07 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH 22/25] CI: add more variables to MAKEFLAGS, except under vs-build Date: Mon, 21 Feb 2022 15:46:34 +0100 Message-Id: X-Mailer: git-send-email 2.35.1.1132.ga1fe46f8690 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org It's clearer that "DEVELOPER" is a flag that affects the Makefile itself in particular if it's put into "MAKEFLAGS than" if it generically sits in the environment. Let's move both it and "SKIP_DASHED_BUILT_INS" to "MAKEFLAGS". We can't do this under vs-build, since that invokes cmake. Let's have only that job set these in the environment. Signed-off-by: Ævar Arnfjörð Bjarmason --- ci/lib.sh | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/ci/lib.sh b/ci/lib.sh index 367d1ee05d5..475e9f63a74 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -55,6 +55,13 @@ setenv () { # Clear MAKEFLAGS that may come from the outside world. MAKEFLAGS= +# Common make and cmake build options +DEVELOPER=1 +SKIP_DASHED_BUILT_INS=YesPlease + +# Use common options for "make" (cmake in "vs-build" below) +MAKEFLAGS="DEVELOPER=$DEVELOPER SKIP_DASHED_BUILT_INS=$SKIP_DASHED_BUILT_INS" + case "$CI_TYPE" in github-actions) CC="${CC:-gcc}" @@ -73,10 +80,8 @@ github-actions) ;; esac -setenv --build DEVELOPER 1 setenv --test DEFAULT_TEST_TARGET prove setenv --test GIT_TEST_CLONE_2GB true -setenv --build SKIP_DASHED_BUILT_INS YesPlease case "$runs_on_pool" in ubuntu-latest) @@ -110,6 +115,9 @@ windows-build) setenv --build ARTIFACTS_DIRECTORY artifacts ;; vs-build) + setenv --build DEVELOPER $DEVELOPER + setenv --build SKIP_DASHED_BUILT_INS $SKIP_DASHED_BUILT_INS + setenv --build NO_PERL NoThanks setenv --build NO_GETTEXT NoThanks setenv --build ARTIFACTS_DIRECTORY artifacts From patchwork Mon Feb 21 14:46:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12753730 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3445C433EF for ; Mon, 21 Feb 2022 14:48:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378335AbiBUOsq (ORCPT ); Mon, 21 Feb 2022 09:48:46 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:33332 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1378342AbiBUOrt (ORCPT ); Mon, 21 Feb 2022 09:47:49 -0500 Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com [IPv6:2a00:1450:4864:20::32d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 124335FC8 for ; Mon, 21 Feb 2022 06:47:11 -0800 (PST) Received: by mail-wm1-x32d.google.com with SMTP id y6-20020a7bc186000000b0037bdc5a531eso29940wmi.0 for ; Mon, 21 Feb 2022 06:47:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=uXMPVqJ67+pdj8eonhYCu2kEa++EZMGukHAHBTxKv24=; b=NynUV5gYnc/x/o4aMENurDEmsm84bZLWT91/j5SbRuP7jf/AHUNxiiH+oXON7vxFB5 8fh1sHZ1t4dESLohvJNieClHIAxTPjj8kq4mSavyCmcO/Vmvk7htS3qL2kfy/Lqm/ny9 2XBtuimDqvIdI/J8shokQByhWM9dHD1s1QHtXU/gMZDR02Vja+qDJ47+S5NWisCPitmu n+rZr6HtWoR7bnsl8jJxOAQ95HUFG6msBdbVUUhq/2/r9l7qXKpCfWH0CBf6ClGmMe7N 6zxU5/MUw25UbTFvGY44ltYgQKtZf3De3OcGhno61ppLCD0HUTU7N7pqH3diSmxdXjwt r9+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=uXMPVqJ67+pdj8eonhYCu2kEa++EZMGukHAHBTxKv24=; b=KrvrKNn7y1uwbxC0X5kNoEH1k28JD8r0FlDDIwZpTXlIsXNxVWBiT2vEFoYBa1sx4w h7pbPQ7JcOr6kBFK9JW3GIAJK1k63wYKe/ipb45+yziM28K4QZIjeiHvH3KdLENcapyP lF9/9nEJ4/ZXw5nb1G7eNCn04kWVteAFFfaFfDc1yYxjsKhfOLFNgLYUTJhwih7PbhyN b1IveuAtzRuqxDWjsFj1c7+An5ppW/TfzQ3GLbEY708XCZb+QzadBFTPO/DWeHlBqIDX gGTGrH0jOPkC8xOMwsC+jqymQ1NPmxeYPsv9coZi/j3Vln44NH9nd1Lo8XRdE8cDhqIa atPg== X-Gm-Message-State: AOAM532l62LMfvIR7RjhEbG9cGO2vcPqvfH0Jk8uox3fBCU/4t4AX4kG NG2mrxRpjNzgrvwIZDES63ugru/kGebolw== X-Google-Smtp-Source: ABdhPJz65L3GRqSY5jNQie4QLQoHkfzrKU5sAS2lUaXIEbYkaeH5Y0lqV1Ijf75xODwUJkLkKQgDMg== X-Received: by 2002:a05:600c:4c27:b0:37b:fa17:899a with SMTP id d39-20020a05600c4c2700b0037bfa17899amr17909083wmp.126.1645454829320; Mon, 21 Feb 2022 06:47:09 -0800 (PST) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id r14sm25071094wrz.84.2022.02.21.06.47.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Feb 2022 06:47:08 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH 23/25] CI: stop over-setting the $CC variable Date: Mon, 21 Feb 2022 15:46:35 +0100 Message-Id: X-Mailer: git-send-email 2.35.1.1132.ga1fe46f8690 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org As detailed in 2c8921db2b8 (travis-ci: build with the right compiler, 2019-01-17) the reason we started using $CC in $MAKEFLAGS as opposed to setting it in the environment was due to Travis CI clobbering $CC in the environment. We don't need to set it unconditionally to accomplish that, but rather just have it set for those jobs that need them. E.g. the "win+VS build" job confusingly has CC=gcc set, even though it builds with MSVC. This partially reverts my 707d2f2fe86 (CI: use "$runs_on_pool", not "$jobname" to select packages & config, 2021-11-23), i.e. we're now aiming to only set those variables specific jobs need. Signed-off-by: Ævar Arnfjörð Bjarmason --- .github/workflows/main.yml | 13 ------------- ci/lib.sh | 26 ++++++++++++++++++++++---- 2 files changed, 22 insertions(+), 17 deletions(-) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 0787cadc76b..6d25ec4ae3b 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -238,37 +238,24 @@ jobs: matrix: vector: - jobname: linux-clang - cc: clang pool: ubuntu-latest - jobname: linux-sha256 - cc: clang os: ubuntu pool: ubuntu-latest - jobname: linux-gcc - cc: gcc - cc_package: gcc-8 pool: ubuntu-latest - jobname: linux-TEST-vars - cc: gcc os: ubuntu - cc_package: gcc-8 pool: ubuntu-latest - jobname: osx-clang - cc: clang pool: macos-latest - jobname: osx-gcc - cc: gcc - cc_package: gcc-9 pool: macos-latest - jobname: linux-gcc-default - cc: gcc pool: ubuntu-latest - jobname: linux-leaks - cc: gcc pool: ubuntu-latest env: - CC: ${{matrix.vector.cc}} - CC_PACKAGE: ${{matrix.vector.cc_package}} jobname: ${{matrix.vector.jobname}} runs_on_pool: ${{matrix.vector.pool}} runs-on: ${{matrix.vector.pool}} diff --git a/ci/lib.sh b/ci/lib.sh index 475e9f63a74..7064a17cfeb 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -52,7 +52,9 @@ setenv () { fi } -# Clear MAKEFLAGS that may come from the outside world. +# Clear variables that may come from the outside world. +CC= +CC_PACKAGE= MAKEFLAGS= # Common make and cmake build options @@ -64,8 +66,6 @@ MAKEFLAGS="DEVELOPER=$DEVELOPER SKIP_DASHED_BUILT_INS=$SKIP_DASHED_BUILT_INS" case "$CI_TYPE" in github-actions) - CC="${CC:-gcc}" - setenv --test GIT_PROVE_OPTS "--timer --jobs 10" GIT_TEST_OPTS="--verbose-log -x" MAKEFLAGS="$MAKEFLAGS --jobs=10" @@ -135,9 +135,16 @@ vs-test) setenv --test NO_SVN_TESTS YesPlease ;; linux-gcc) + CC=gcc + CC_PACKAGE=gcc-8 setenv --test GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME main ;; +linux-gcc-default) + CC=gcc + ;; linux-TEST-vars) + CC=gcc + CC_PACKAGE=gcc-8 setenv --test GIT_TEST_SPLIT_INDEX yes setenv --test GIT_TEST_MERGE_ALGORITHM recursive setenv --test GIT_TEST_FULL_IN_PACK_ARRAY true @@ -152,10 +159,19 @@ linux-TEST-vars) setenv --test GIT_TEST_WRITE_REV_INDEX 1 setenv --test GIT_TEST_CHECKOUT_WORKERS 2 ;; +osx-gcc) + CC=gcc + CC_PACKAGE=gcc-9 + ;; +osx-clang) + CC=clang + ;; linux-clang) + CC=clang setenv --test GIT_TEST_DEFAULT_HASH sha1 ;; linux-sha256) + CC=clang setenv --test GIT_TEST_DEFAULT_HASH sha256 ;; pedantic) @@ -173,9 +189,11 @@ linux-musl) MAKEFLAGS="$MAKEFLAGS GIT_TEST_UTF8_LOCALE=C.UTF-8" ;; linux-leaks) + CC=gcc setenv --build SANITIZE leak setenv --test GIT_TEST_PASSING_SANITIZE_LEAK true ;; esac -setenv --all MAKEFLAGS "$MAKEFLAGS CC=${CC:-cc}" +MAKEFLAGS="$MAKEFLAGS${CC:+ CC=$CC}" +setenv --all MAKEFLAGS "$MAKEFLAGS" From patchwork Mon Feb 21 14:46:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12753731 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 88FBEC433F5 for ; Mon, 21 Feb 2022 14:48:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378334AbiBUOsr (ORCPT ); Mon, 21 Feb 2022 09:48:47 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:33370 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1378340AbiBUOrt (ORCPT ); Mon, 21 Feb 2022 09:47:49 -0500 Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 243AF5FCF for ; Mon, 21 Feb 2022 06:47:12 -0800 (PST) Received: by mail-wm1-x32c.google.com with SMTP id o34so9675276wms.1 for ; Mon, 21 Feb 2022 06:47:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=8sLgltdoLSDmb9EA6S3lYN1W6z0r8071szxt0gXE1Lw=; b=A678aGf0BkJTmnjOXBUx0VMYyECZbfGM4MFqFbQNYtOrO7PB4wjsTVObgGOqBrvHfM Q4G0D6L/D0DvYupYlCvoDqA0xMLgkDrIlTdXoHdujjU/gLn0Bzx8mCHf7yflomlX5WUA 9wgGd39vDO/HTCYOOeGwMvy68BnjLtxqhEmmh/uA6cpbksi3i38EpyEGNeidaW+IdiUq iK8l3dOW2lx/uZY3Azqexfkn2qjoXOQ3Ql9gPMiKhjX6VQRUz8uDzB2p8D32iXqDDe/Y RojZCc/56q7j07Qkq2waJOwrSg6Bd28LpeD9lHLt+QwgOgWOnD1OqW3FYD37HbNt7X+0 +ijQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=8sLgltdoLSDmb9EA6S3lYN1W6z0r8071szxt0gXE1Lw=; b=uRXOYa8hvUxXwdNx6NbszI9udHZG1x0DhMy1C7zrLoVmWtBXSu2MekbuWZS39RtKnM Kzj0dCvNOb7FB12+BLosDs6QimiJbwtZ9JQHqb6XwHRf3OsYJwbiBWxgx6JmHdtqw8of 2Q5Emege64/EFBK44prctemFRfauxr36v2Cex3Nf8mmSsSWah4+eJXWnEmLi59ogHYMD ot0TSfsxO0aLxcH1pja0hZD36GtXUpglZftAczD9LTb6wfA9Rsnz/GHg5dA8q4uC4j88 1cZtrND71K3am1UUJc0mtYkwHNRoDTJOznJF6Rt3BTptqXfEWEwx1zx8OuJhJpRL1XAg zhNQ== X-Gm-Message-State: AOAM5322swIF/eBUtfV5YJjXpJahHbKykkXc0obglSeuBf2KzfWt+bRN C3xcHn3qkfmwH6QViwMh02virrVp8tm+iw== X-Google-Smtp-Source: ABdhPJyMAHQ5fbL2fKQe5TD3hBOD5W6L0EK65MZUV3w8bn3nrRUv1gL8tZxPJefRmlEsdd40LgD6Qw== X-Received: by 2002:a1c:a1c2:0:b0:37b:aa06:9c51 with SMTP id k185-20020a1ca1c2000000b0037baa069c51mr22037007wme.114.1645454830418; Mon, 21 Feb 2022 06:47:10 -0800 (PST) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id r14sm25071094wrz.84.2022.02.21.06.47.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Feb 2022 06:47:09 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH 24/25] CI: set PYTHON_PATH setting for osx-{clang,gcc} into "$jobname" case Date: Mon, 21 Feb 2022 15:46:36 +0100 Message-Id: X-Mailer: git-send-email 2.35.1.1132.ga1fe46f8690 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Move the setting of "PYTHON_PATH=[...]" to the "$jobname" case statement. This partially backs out of my 707d2f2fe86 (CI: use "$runs_on_pool", not "$jobname" to select packages & config, 2021-11-23), now that we have a "osx-{clang,gcc}" anyway for setting "$CC" we might as well do away with this part of the "$runs_on_pool" case. Signed-off-by: Ævar Arnfjörð Bjarmason --- ci/lib.sh | 10 ++-------- 1 file changed, 2 insertions(+), 8 deletions(-) diff --git a/ci/lib.sh b/ci/lib.sh index 7064a17cfeb..36f79eb6a65 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -99,14 +99,6 @@ ubuntu-latest) setenv --test GIT_TEST_HTTPD true ;; -macos-latest) - if [ "$jobname" = osx-gcc ] - then - MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)" - else - MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python2)" - fi - ;; esac case "$jobname" in @@ -160,10 +152,12 @@ linux-TEST-vars) setenv --test GIT_TEST_CHECKOUT_WORKERS 2 ;; osx-gcc) + MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)" CC=gcc CC_PACKAGE=gcc-9 ;; osx-clang) + MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python2)" CC=clang ;; linux-clang) From patchwork Mon Feb 21 14:46:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12753729 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9D677C433EF for ; Mon, 21 Feb 2022 14:48:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1378392AbiBUOsm (ORCPT ); Mon, 21 Feb 2022 09:48:42 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:33372 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1378341AbiBUOrt (ORCPT ); Mon, 21 Feb 2022 09:47:49 -0500 Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B490C5FD3 for ; Mon, 21 Feb 2022 06:47:13 -0800 (PST) Received: by mail-wr1-x431.google.com with SMTP id f17so2625980wrh.7 for ; Mon, 21 Feb 2022 06:47:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=96GQVmOc+Lp4rHTNgKypfRY7NVX6k42s+v36CMB1BUU=; b=Mp26aO4irXEDyG0CxGwORgpLYMfJu+PYHL+IwJHJbOzpC/uAXDeBIo1nYfRhu0gg6w wrG4WPoUbkmKEhqFWg8XWPwB7z36AHHXMAZE4NTOQSzRw3sc9DUejYbi2gAEVefPihbl 4YvBs+Z58Sm+Eko6jqj2DeiY75BTc9P+Zmqk0YArtNIfjjSawS7YryQHTNaehvLcXEL/ 01elDVp82p2H35PUMwhVURc2SjAm1ZIpUuBOrlLEyqrKmV+AXXQHcZxfI5hPsA92LqEJ AjfSILz6t6/+FLyfeDaXMjj0Pnj/OGYkiwBzpsl8O4Xuyl5xtEoEYClUoP91s0Ez2cwR BFLw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=96GQVmOc+Lp4rHTNgKypfRY7NVX6k42s+v36CMB1BUU=; b=BoHUFUXsmNIebLpfMEe7lXiKi2egOJerIQrmXVCuDqFZnTXhJgdszwfwSOG2IO8im9 4qBhlnndK8c3vs9PoSUid7VO90EknkL2oC5dwRwdmgGvTgtqrcvnTxP8P4yrS5VOyVGB 2y+KEHiwBVjuV168d6zsx3xxtBp9tEkXW9rsVxZw2NypgfK9MNWTg4TdIueiPox0rAQm 7ejGKWx9iQ3LAPcMmItQoseUXg0WVgCJIw8rLwL+zVP4MLLhlzh+f584P88RDZ13Wfnw vKr8BxsL1ykiCfbjmrYOroBi2ExFnkgle9qaoRKd4T6V+lX6zsOd7hhuDFh0qTzsJvFv eMmQ== X-Gm-Message-State: AOAM530vZO0F46bsrXf0YgPu/LSx9gJ2P2/nc/pZtzYPrhKHI3DXPIR6 GQhHv3Uu+4Tf1rg0rEqQcgwbp+b+ywdGmg== X-Google-Smtp-Source: ABdhPJzpyIFozljz+0zgHrbbO/z0jOmBUG2cbn1BSt31GpWS56v3lSTPMCH3Hy+az8hI9BdzvITaFw== X-Received: by 2002:a5d:4292:0:b0:1e4:b7fd:eb84 with SMTP id k18-20020a5d4292000000b001e4b7fdeb84mr16217466wrq.657.1645454831571; Mon, 21 Feb 2022 06:47:11 -0800 (PST) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id r14sm25071094wrz.84.2022.02.21.06.47.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Feb 2022 06:47:10 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Taylor Blau , =?utf-8?q?SZEDER_G=C3=A1bor?= , =?utf-8?b?xJBvw6Bu?= =?utf-8?b?IFRy4bqnbiBDw7RuZyBEYW5o?= , =?utf-8?q?Car?= =?utf-8?q?lo_Marcelo_Arenas_Bel=C3=B3n?= , Johannes Schindelin , Victoria Dye , Matheus Tavares , Lars Schneider , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH 25/25] CI: don't use "set -x" in "ci/lib.sh" output Date: Mon, 21 Feb 2022 15:46:37 +0100 Message-Id: X-Mailer: git-send-email 2.35.1.1132.ga1fe46f8690 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Remove the "set -x" from the "ci/lib.sh" output. Before preceding commits the logic in that file was much more complex, and likely to fail in some scenarios. Now we only task "ci/lib.sh" with setting various variables for subsequent steps in our jobs, so we can start emitting more tailored debugging output, which makes what it's doing easier to read. This change also changes the output of the "ci/print-test-failures.sh" script, since it's the only other user of "ci/lib-ci-type.sh". In that case it's also helpful to know what "$CI_TYPE" we're using, as that script doesn't "set -x" and will act differently depending on the $CI_TYPE. Signed-off-by: Ævar Arnfjörð Bjarmason --- ci/lib-ci-type.sh | 2 ++ ci/lib.sh | 9 ++++++++- 2 files changed, 10 insertions(+), 1 deletion(-) diff --git a/ci/lib-ci-type.sh b/ci/lib-ci-type.sh index bd6e093c8f4..6cfe58596d2 100644 --- a/ci/lib-ci-type.sh +++ b/ci/lib-ci-type.sh @@ -6,3 +6,5 @@ else env >&2 exit 1 fi + +echo "CONFIG: CI_TYPE=$CI_TYPE" >&2 diff --git a/ci/lib.sh b/ci/lib.sh index 36f79eb6a65..f4323a3dd36 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -1,5 +1,5 @@ #!/bin/sh -set -ex +set -e # Helper libraries . ${0%/*}/lib-ci-type.sh @@ -11,12 +11,16 @@ then echo "need a $0 mode, e.g. --build or --test" exit 1 fi +echo "CONFIG: mode=$mode" >&2 if test -z "$jobname" then echo "must set a CI jobname" >&2 exit 1 fi +echo "CONFIG: jobname=$jobname" >&2 +echo "CONFIG: runs_on_pool=$runs_on_pool" >&2 +echo "CONFIG: GITHUB_ENV=$GITHUB_ENV" >&2 # Helper functions setenv () { @@ -39,6 +43,7 @@ setenv () { if test -n "$skip" then + echo "SKIP '$key=$val'" >&2 return 0 fi @@ -50,6 +55,8 @@ setenv () { # itself. eval "export $key=\"$val\"" fi + + echo "SET: '$key=$val'" >&2 } # Clear variables that may come from the outside world.