From patchwork Wed Mar 25 05:41:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee via GitGitGadget X-Patchwork-Id: 11456939 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0F89214B4 for ; Wed, 25 Mar 2020 05:41:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id DED1420789 for ; Wed, 25 Mar 2020 05:41:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="LFDjkraQ" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726072AbgCYFl0 (ORCPT ); Wed, 25 Mar 2020 01:41:26 -0400 Received: from mail-wr1-f49.google.com ([209.85.221.49]:38928 "EHLO mail-wr1-f49.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725781AbgCYFlZ (ORCPT ); Wed, 25 Mar 2020 01:41:25 -0400 Received: by mail-wr1-f49.google.com with SMTP id p10so1345657wrt.6 for ; Tue, 24 Mar 2020 22:41:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=d+CdihV/MaXlOsmdA1HIYg2dPk7X4+olTFP2vNVSVCU=; b=LFDjkraQBPA1YYsh3VQaLaH09ueX9ljAFloJmfT/+ktKh/fDdcAXJoU3nI8pCztd1v xuC+ITHUnEeZq6Fun2nMD7vxAgwerUk1CRRuKmYgPJHijdInc6K9GDDm8uI3k6yXc6dC XpKtvFZctXEOINKEtXIcn7tQOG5OD3L13S7/3ShmcCWwTwDU/wIkNcmCHbH2sRciyBXH LmSM6TP8sedg/znEPnGBl8Bkq9hhwchfph3aGrFuU0SgWXt3lX+kSGZHxNZgJ9CEbVOp RX44UeDBAfzY/+yk5XWLspgkkG91k1zHnajZ2+eLNdZaZ+SF5m0OlGJ0bhNZI0KvOO7v 4U7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=d+CdihV/MaXlOsmdA1HIYg2dPk7X4+olTFP2vNVSVCU=; b=KhwF85hy1GzqMpOrJiiTGVWZEdoD5YLLlv9SAckMatTTey6IFt4xuDNttnni2+D5zC 8Qm4lJFDYvU2MVJ7BXJq9iunSQZ/JLlbqekhSr3YPO+cPWEolKvIKbVYWvTf5nr4CZxL QAGAtwVPBPL7mUlodRR5To/DBSsvb/UYUlt9iMJqLFDVNVf9d88dopOuBYoakHaycgm0 hFu4iFPshTBAFl5F8+DJKrglzIjTrnYBFDOUBsoStl2qwiIqTNjHpekBzSCdkiBndzdw B1n/OOrQc9w3/bXP84oFNmXzjLM3MmqDF+nxjbxcDje3wKhxdnr912sXZltpnajoeE2y 43Rg== X-Gm-Message-State: ANhLgQ0PXchFL/vKU7J0DyXu4xbe1MYLlH+/9Zm8g1PpVP7AX1xigD1i IMzQ1Jc9s62tUNcEQ1gbmcmiGYLi X-Google-Smtp-Source: ADFU+vuT6zHbqiCoymbL5h3gnLaAWHBZSJCFZRfi7sOKAX3iBtIRNMWBaXsDWgVjSI/sjf1CBMT3ZQ== X-Received: by 2002:adf:e289:: with SMTP id v9mr1391726wri.361.1585114883394; Tue, 24 Mar 2020 22:41:23 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id a192sm7987270wme.5.2020.03.24.22.41.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 24 Mar 2020 22:41:23 -0700 (PDT) Message-Id: <287a21f1033b2a74420029c529ad4db956051a85.1585114881.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Johannes Schindelin via GitGitGadget" Date: Wed, 25 Mar 2020 05:41:17 +0000 Subject: [PATCH v2 1/5] tests(gpg): allow the gpg-agent to start on Windows Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Jeff King , Junio C Hamano , Johannes Schindelin , Johannes Schindelin Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Johannes Schindelin In Git for Windows' SDK, we use the MSYS2 version of OpenSSH, meaning that the `gpg-agent` will fail horribly when being passed a `--homedir` that contains colons. Previously, we did pass the Windows version of the absolute path, though, which starts in the drive letter followed by, you guessed it, a colon. Let's use the same trick found elsewhere in our test suite where `$PWD` is used to refer to the pseudo-Unix path (which works only within the MSYS2 Bash/OpenSSH/Perl/etc, as opposed to `$(pwd)` which refers to the Windows path that `git.exe` understands, too). Signed-off-by: Johannes Schindelin --- t/lib-gpg.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/t/lib-gpg.sh b/t/lib-gpg.sh index 8d28652b729..11b83b8c24a 100755 --- a/t/lib-gpg.sh +++ b/t/lib-gpg.sh @@ -29,7 +29,7 @@ then # > lib-gpg/ownertrust mkdir ./gpghome && chmod 0700 ./gpghome && - GNUPGHOME="$(pwd)/gpghome" && + GNUPGHOME="$PWD/gpghome" && export GNUPGHOME && (gpgconf --kill gpg-agent >/dev/null 2>&1 || : ) && gpg --homedir "${GNUPGHOME}" 2>/dev/null --import \ From patchwork Thu Mar 26 15:35:25 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee via GitGitGadget X-Patchwork-Id: 11460543 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 73D0015AB for ; Thu, 26 Mar 2020 15:35:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 54EB520775 for ; Thu, 26 Mar 2020 15:35:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="iQsecFI9" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728516AbgCZPff (ORCPT ); Thu, 26 Mar 2020 11:35:35 -0400 Received: from mail-ed1-f68.google.com ([209.85.208.68]:37319 "EHLO mail-ed1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728088AbgCZPfd (ORCPT ); Thu, 26 Mar 2020 11:35:33 -0400 Received: by mail-ed1-f68.google.com with SMTP id de14so7322795edb.4 for ; Thu, 26 Mar 2020 08:35:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=foBY/mH6k2D/bsKWKIhwbwseCoDDelfr5UKKVRqstfA=; b=iQsecFI9CQ62XH+xye8cjKIy5OXB6pUHqaPfr0Bvrwcfbk+oCtMd27v17Vky5ZyQqQ fB+5QNI9SoOVzj3WjzJQxnNb9CPvsRh0XzwsK0ACH1cy27YwjSq0VPrbzvgE2eu0klqc /v1GYFpJ2kJ0fWiXgP5HXLJHkHx2FcN7x3JY2umISnQ4HMQrv8IeozcJDO5DGdRc+DEd KA6sZNCUctQ57k2vDfkPLqqnoiZeV13Ufnq7CvZTLd5RfqDzUGmuqhObwJhGrdNNDL2k 2TRXZYDVUdLlokBoKlVJ3ZehKQnVS9tpjcXeKJOSbolRtX+XgRIWACNXF7snQBjHmT+y N3Aw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=foBY/mH6k2D/bsKWKIhwbwseCoDDelfr5UKKVRqstfA=; b=Evs4r214HyE8o94/FFiOAFKRrptMs3LqTLr+AjBgUl2BhwZqRMCR1jvucs+VTq7KpL giOTzOx+Y1EP8hpbaKXloaCbsW4UHTvmo0XzRKQn5IW11IoQymPTiSlnmGx9AheOejhL OTR3gIuX5LHXaB9cG5T/kITpk5ob6J9GY77HXq0m+rVj+AVBdHZGVUUeveRrq60oAfku y8X4YQJoQLGjV3sApPKytVhAiWInJqo1891JRztOmsjmlcEtC+CwHKkiwniTV6xFucI9 mSnifN8RnZEQeZZ5f9lQCk54u+k7YMBVMKZ/3x+CLfcq/iv+m1BmDlN2ID9ujuZA7REk twlw== X-Gm-Message-State: ANhLgQ1wmWtyLYDh+eNzdF+eN1qcGm0qU72D+28LZep42/mEZulN3kaX rIWtgNnAglivzCN8F0vSpW5Vx+/H X-Google-Smtp-Source: ADFU+vt++25otBwH/bYo+nNWlSz/yrxKFZL4SM1Lnk2kRYDiYsmnsvbQOdKMdHJuqXGvb/aSpJkPgA== X-Received: by 2002:a17:906:1603:: with SMTP id m3mr8496187ejd.205.1585236931886; Thu, 26 Mar 2020 08:35:31 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id cm11sm386730edb.1.2020.03.26.08.35.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Mar 2020 08:35:31 -0700 (PDT) Message-Id: In-Reply-To: References: From: "Johannes Schindelin via GitGitGadget" Date: Thu, 26 Mar 2020 15:35:25 +0000 Subject: [PATCH v3 2/5] t/lib-gpg.sh: stop pretending to be a stand-alone script Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Jeff King , Junio C Hamano , Johannes Schindelin , Johannes Schindelin Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Johannes Schindelin It makes no sense to call `./lib-gpg.sh`. Therefore the hash-bang line is unnecessary. There are other similar instances in `t/`, but they are too far from the context of the enclosing patch series, so they will be addressed separately. Signed-off-by: Johannes Schindelin --- t/lib-gpg.sh | 2 -- 1 file changed, 2 deletions(-) diff --git a/t/lib-gpg.sh b/t/lib-gpg.sh index 11b83b8c24a..4ead1268351 100755 --- a/t/lib-gpg.sh +++ b/t/lib-gpg.sh @@ -1,5 +1,3 @@ -#!/bin/sh - gpg_version=$(gpg --version 2>&1) if test $? != 127 then From patchwork Wed Mar 25 05:41:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee via GitGitGadget X-Patchwork-Id: 11456943 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9F4B817EF for ; Wed, 25 Mar 2020 05:41:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 76D2020789 for ; Wed, 25 Mar 2020 05:41:28 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="MAcES8Ku" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726109AbgCYFl1 (ORCPT ); Wed, 25 Mar 2020 01:41:27 -0400 Received: from mail-wr1-f48.google.com ([209.85.221.48]:38928 "EHLO mail-wr1-f48.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725909AbgCYFl1 (ORCPT ); Wed, 25 Mar 2020 01:41:27 -0400 Received: by mail-wr1-f48.google.com with SMTP id p10so1345733wrt.6 for ; Tue, 24 Mar 2020 22:41:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=Zr7j7KDO6DqFDe33EpDYB1oLRw9+sRniw+RwvncNmuU=; b=MAcES8KulSutiAIVkGX6fYMIYpltaw20tvlJ4RjNzeQkf/W9gs4txHpWWcQmMOttxx C+Ui6h0fnZk+JiZndYlCMlkJc47B3G/2CyqzS2IyrUlPtZAOdAX3w+knf9STqbgJI5PL dFgh9LsUgfLTStqwb7GTNh5aj+LufMiQMX5Clw71tgR/rWXxmruzfNhTnZp3EDHWpe9d KAOs1BP4b/BSO34gBpVbGpVgnwb+okOSxeIw/8PWbXZ2JPaDTYakkP0Ac+Lzlq1nfN5y c3/kh/AsiXbeDF+dHfWbPhCcysIn4pc5o/RvyCNpsqxEaIBm5n15YnCADQt+k1pBNasp pLGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=Zr7j7KDO6DqFDe33EpDYB1oLRw9+sRniw+RwvncNmuU=; b=hVxrdenRrvBt660SQ+XF0/ZqZKJ6nZk+9PTttfBTPlkq9hP7wZ2rUnL4MZ+B3p320X BcQ/k+xSgllaAD/8/s0JBJzhCkgNM4pvaiRzrRKOxCp2K155IlInWOnLsXE/dJgS4R3W hivtAUQoJ/xUzaGVGbNg3dpA7kygRfxBwBDt9VOjC1PB8ZB9q5Y7poUvSdiXUHFewimM a7FrzN9V134pApBMha9WU0Njs1muimQJNT9VSsM5QTI0qF4+ki4Gp2pXLSki+Z7zOkGj SjjeBP33JDzEDLI1xgBO3zn/OsYzbfo6Lfc3QSm6Qm68uxq4nXk0Fmxi3IwAmV0ASYdR cfAA== X-Gm-Message-State: ANhLgQ0noVd7SmhtTBaJrJeOaPsRoCAqL3akCs/xZS1yysg263gNYDAu Mr+STDf0It1fwiqpYlyBDgKKn8o0 X-Google-Smtp-Source: ADFU+vvIDM74x7VdFNSgMNiaT/ljIrW0Vw+kXG60XJtKPsUttd+WHS311BPz3u1feduq/yQaKwzAdw== X-Received: by 2002:adf:e948:: with SMTP id m8mr1393416wrn.193.1585114884690; Tue, 24 Mar 2020 22:41:24 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id v7sm7801437wml.18.2020.03.24.22.41.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 24 Mar 2020 22:41:24 -0700 (PDT) Message-Id: <85457a7b61874e8e9f3af9c231451df0aba7a7b5.1585114881.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Johannes Schindelin via GitGitGadget" Date: Wed, 25 Mar 2020 05:41:19 +0000 Subject: [PATCH v2 3/5] tests: turn GPG, GPGSM and RFC1991 into lazy prereqs Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Jeff King , Junio C Hamano , Johannes Schindelin , Johannes Schindelin Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Johannes Schindelin The code to set those prereqs is executed completely outside of any `test_eval_` block. As a consequence, its output had to be suppressed so that it does not clutter the output of a regular test script run. Unfortunately, the output *stays* suppressed even when the `--verbose` option is in effect. This hid important output when debugging why the GPG prereq was not enabled in the Windows part of our CI builds. In preparation for fixing that, let's move all of this code into lazy prereqs. The only slightly tricky part is the global environment variable `GNUPGHOME`. Originally, it was configured only when we verified that there is a `gpg` in the `PATH` that we can use. This is now no longer possible, as lazy prereqs are evaluated in a subshell that changes the working directory to a temporary one. Therefore, we simply _always_ set that environment variable: it does not hurt anything because it does not indicate the presence of a working GPG. Side note: it was quite tempting to use a hack that is possible because we do not validate what is passed to `test_lazy_prereq` (and it is therefore possible to "break out" of the lazy_prereq subshell: test_lazy_prereq GPG '...) && GNUPGHOME=... && (...' However, this is rather tricksy hobbitses code, and the current patch is _much_ easier to understand. Signed-off-by: Johannes Schindelin Signed-off-by: Junio C Hamano --- t/lib-gpg.sh | 102 ++++++++++++++++++++++++++++----------------------- 1 file changed, 57 insertions(+), 45 deletions(-) diff --git a/t/lib-gpg.sh b/t/lib-gpg.sh index 4ead1268351..7a78c562e8d 100755 --- a/t/lib-gpg.sh +++ b/t/lib-gpg.sh @@ -1,12 +1,25 @@ -gpg_version=$(gpg --version 2>&1) -if test $? != 127 -then +# We always set GNUPGHOME, even if no usable GPG was found, as +# +# - It does not hurt, and +# +# - we cannot set global environment variables in lazy prereqs because they are +# executed in an eval'ed subshell that changes the working directory to a +# temporary one. + +GNUPGHOME="$PWD/gpghome" +export GNUPGHOME + +test_lazy_prereq GPG ' + gpg_version=$(gpg --version 2>&1) + test $? != 127 || exit 1 + # As said here: http://www.gnupg.org/documentation/faqs.html#q6.19 - # the gpg version 1.0.6 didn't parse trust packets correctly, so for + # the gpg version 1.0.6 did not parse trust packets correctly, so for # that version, creation of signed tags using the generated key fails. case "$gpg_version" in - 'gpg (GnuPG) 1.0.6'*) + "gpg (GnuPG) 1.0.6"*) say "Your version of gpg (1.0.6) is too buggy for testing" + exit 1 ;; *) # Available key info: @@ -25,55 +38,54 @@ then # To export ownertrust: # gpg --homedir /tmp/gpghome --export-ownertrust \ # > lib-gpg/ownertrust - mkdir ./gpghome && - chmod 0700 ./gpghome && - GNUPGHOME="$PWD/gpghome" && - export GNUPGHOME && + mkdir "$GNUPGHOME" && + chmod 0700 "$GNUPGHOME" && (gpgconf --kill gpg-agent >/dev/null 2>&1 || : ) && gpg --homedir "${GNUPGHOME}" 2>/dev/null --import \ "$TEST_DIRECTORY"/lib-gpg/keyring.gpg && gpg --homedir "${GNUPGHOME}" 2>/dev/null --import-ownertrust \ "$TEST_DIRECTORY"/lib-gpg/ownertrust && gpg --homedir "${GNUPGHOME}" /dev/null 2>&1 \ - --sign -u committer@example.com && - test_set_prereq GPG && - # Available key info: - # * see t/lib-gpg/gpgsm-gen-key.in - # To generate new certificate: - # * no passphrase - # gpgsm --homedir /tmp/gpghome/ \ - # -o /tmp/gpgsm.crt.user \ - # --generate-key \ - # --batch t/lib-gpg/gpgsm-gen-key.in - # To import certificate: - # gpgsm --homedir /tmp/gpghome/ \ - # --import /tmp/gpgsm.crt.user - # To export into a .p12 we can later import: - # gpgsm --homedir /tmp/gpghome/ \ - # -o t/lib-gpg/gpgsm_cert.p12 \ - # --export-secret-key-p12 "committer@example.com" - echo | gpgsm --homedir "${GNUPGHOME}" 2>/dev/null \ - --passphrase-fd 0 --pinentry-mode loopback \ - --import "$TEST_DIRECTORY"/lib-gpg/gpgsm_cert.p12 && - - gpgsm --homedir "${GNUPGHOME}" 2>/dev/null -K | - grep fingerprint: | - cut -d" " -f4 | - tr -d '\n' >"${GNUPGHOME}/trustlist.txt" && - - echo " S relax" >>"${GNUPGHOME}/trustlist.txt" && - echo hello | gpgsm --homedir "${GNUPGHOME}" >/dev/null \ - -u committer@example.com -o /dev/null --sign - 2>&1 && - test_set_prereq GPGSM + --sign -u committer@example.com ;; esac -fi +' + +test_lazy_prereq GPGSM ' + test_have_prereq GPG && + # Available key info: + # * see t/lib-gpg/gpgsm-gen-key.in + # To generate new certificate: + # * no passphrase + # gpgsm --homedir /tmp/gpghome/ \ + # -o /tmp/gpgsm.crt.user \ + # --generate-key \ + # --batch t/lib-gpg/gpgsm-gen-key.in + # To import certificate: + # gpgsm --homedir /tmp/gpghome/ \ + # --import /tmp/gpgsm.crt.user + # To export into a .p12 we can later import: + # gpgsm --homedir /tmp/gpghome/ \ + # -o t/lib-gpg/gpgsm_cert.p12 \ + # --export-secret-key-p12 "committer@example.com" + echo | gpgsm --homedir "${GNUPGHOME}" 2>/dev/null \ + --passphrase-fd 0 --pinentry-mode loopback \ + --import "$TEST_DIRECTORY"/lib-gpg/gpgsm_cert.p12 && + + gpgsm --homedir "${GNUPGHOME}" 2>/dev/null -K | + grep fingerprint: | + cut -d" " -f4 | + tr -d "\\n" >"${GNUPGHOME}/trustlist.txt" && + + echo " S relax" >>"${GNUPGHOME}/trustlist.txt" && + echo hello | gpgsm --homedir "${GNUPGHOME}" >/dev/null \ + -u committer@example.com -o /dev/null --sign - 2>&1 +' -if test_have_prereq GPG && - echo | gpg --homedir "${GNUPGHOME}" -b --rfc1991 >/dev/null 2>&1 -then - test_set_prereq RFC1991 -fi +test_lazy_prereq RFC1991 ' + test_have_prereq GPG && + echo | gpg --homedir "${GNUPGHOME}" -b --rfc1991 >/dev/null 2>&1 +' sanitize_pgp() { perl -ne ' From patchwork Wed Mar 25 05:41:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee via GitGitGadget X-Patchwork-Id: 11456945 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 812A91668 for ; Wed, 25 Mar 2020 05:41:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 6171020789 for ; Wed, 25 Mar 2020 05:41:32 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="vBOv3rvb" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726239AbgCYFlb (ORCPT ); Wed, 25 Mar 2020 01:41:31 -0400 Received: from mail-wr1-f47.google.com ([209.85.221.47]:42800 "EHLO mail-wr1-f47.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725832AbgCYFl1 (ORCPT ); Wed, 25 Mar 2020 01:41:27 -0400 Received: by mail-wr1-f47.google.com with SMTP id h15so1317910wrx.9 for ; Tue, 24 Mar 2020 22:41:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=L7kQUzKOBxExsTka+a+C/tA1DcKmOKX5+TTECXmy1JY=; b=vBOv3rvbnxf9fB8X+XDyQzylfw32qjOj1PTmDRJEbaNOuLGsM6LRrrOem1iuVFW8M1 cDcGpDh1QKlda/Rzn3hn4ZZk8Tjz8tvvXqJb4a4h0oNuJv5cYcPvKbyqw/3Ix+CP6h+g /l1JWBPIVnHmFRJiPT+39aq/uo9rBa/uP1pwTB+bO9t0+tMc2BBrNsVOtTR+hyNLTkTy 00u01Q5SDY+vlPwO4Hou64lbHCLGAyAtXMhCznSCRPBvzhSwJtdiq/QSgL+nFu4hWy6G XgyH7GDQshjSXtSfwytszjw8RNA08GII2cO8RcEuSSN3r5DDb0ZiPIl6k+EJ7jtUxpMo eoVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=L7kQUzKOBxExsTka+a+C/tA1DcKmOKX5+TTECXmy1JY=; b=cPNcD6ymvOjxJzY29+hQFEQQ0eFJuCRbBDuj8s8SNjgclkWfc+KO12mBhcF6KCcfA+ gms75A//rQvVZxiN4YwycSZqg53L3kg4xA5oAzwl3Ke9JG7stxDZTWNwqbwoOyyJBJSv IEmaeRoQpZ+CCjDcbdxkahqW6do+eUDBW1mdoMWR2l92W8RuVG7xVwdvIRiHetGNrCXt SI2MQfwgZP0Ggd03i6YLA/Q0lrs08MlBFcn8OtwrajS4i9Koxog09Yi6T1E7rHF3U8AN LZ4JzT2S3jWSU2l8dY62+H59vTth4lrjgM9iLsDGYXHQ9DbI8/6kR8hLgUM6VrlfPTm6 zHrw== X-Gm-Message-State: ANhLgQ35u8hx2IOYBHQ9mCdvJiC231vpQPxgqSIiFmbJXXYvBjT7d37e gGhHHsr/vWmy3VhpvUmEZ1yM/RTE X-Google-Smtp-Source: ADFU+vtMCOshQiKp7wapoSjJ8d6kbiLIXaFDRQcdY40pBvLCpMs5ETl7KhyBm8Ffv03uU4Tqxiyy+Q== X-Received: by 2002:adf:ff82:: with SMTP id j2mr1397723wrr.48.1585114885509; Tue, 24 Mar 2020 22:41:25 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id q17sm888991wrp.11.2020.03.24.22.41.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 24 Mar 2020 22:41:25 -0700 (PDT) Message-Id: <0767c8b77c820cfc03bbc617da4dc9f20ba4a46a.1585114881.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Johannes Schindelin via GitGitGadget" Date: Wed, 25 Mar 2020 05:41:20 +0000 Subject: [PATCH v2 4/5] tests: do not let lazy prereqs inside `test_expect_*` turn off tracing Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Jeff King , Junio C Hamano , Johannes Schindelin , Johannes Schindelin Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Johannes Schindelin The `test_expect_*` functions use `test_eval_` and so does `test_run_lazy_prereq_`. If tracing is enabled via the `-x` option, `test_eval_` turns on tracing while evaluating the code block, and turns it off directly after it. This is unwanted for nested invocations. One somewhat surprising example of this is when running a test that calls `test_i18ngrep`: that function requires the `C_LOCALE_OUTPUT` prereq, and that prereq is a lazy one, so it is evaluated via `test_eval_`, the command tracing is turned off, and the test case continues to run _without tracing the commands_. Another somewhat surprising example is when one lazy prereq depends on another lazy prereq: the former will call `test_have_prereq` with the latter one, which in turn calls `test_eval_` and -- you guessed it -- tracing (if enabled) will be turned off _before_ returning to evaluating the other lazy prereq. As we will introduce just such a scenario with the GPG, GPGSM and RFC1991 prereqs, let's fix that by introducing a variable that keeps track of the current trace level: nested `test_eval_` calls will increment and then decrement the level, and only when it reaches 0, the tracing will _actually_ be turned off. Signed-off-by: Johannes Schindelin --- t/t0000-basic.sh | 13 +++++++++++++ t/test-lib.sh | 6 ++++-- 2 files changed, 17 insertions(+), 2 deletions(-) diff --git a/t/t0000-basic.sh b/t/t0000-basic.sh index 3e440c078d5..b8597216200 100755 --- a/t/t0000-basic.sh +++ b/t/t0000-basic.sh @@ -833,6 +833,19 @@ then exit 1 fi +test_expect_success 'lazy prereqs do not turn off tracing' " + run_sub_test_lib_test lazy-prereq-and-tracing \ + 'lazy prereqs and -x' -v -x <<-\\EOF && + test_lazy_prereq LAZY true + + test_expect_success lazy 'test_have_prereq LAZY && echo trace' + + test_done + EOF + + grep 'echo trace' lazy-prereq-and-tracing/err +" + test_expect_success 'tests clean up even on failures' " run_sub_test_lib_test_err \ failing-cleanup 'Failing tests with cleanup commands' <<-\\EOF && diff --git a/t/test-lib.sh b/t/test-lib.sh index 0ea1e5a05ed..dbf25348106 100644 --- a/t/test-lib.sh +++ b/t/test-lib.sh @@ -882,6 +882,7 @@ maybe_setup_valgrind () { fi } +_trace_level=0 want_trace () { test "$trace" = t && { test "$verbose" = t || test "$verbose_log" = t @@ -895,7 +896,7 @@ want_trace () { test_eval_inner_ () { # Do not add anything extra (including LF) after '$*' eval " - want_trace && set -x + want_trace && _trace_level=$(($_trace_level+1)) && set -x $*" } @@ -926,7 +927,8 @@ test_eval_ () { test_eval_ret_=$? if want_trace then - set +x + test 1 = $_trace_level && set +x + _trace_level=$(($_trace_level-1)) fi } 2>/dev/null 4>&2 From patchwork Wed Mar 25 05:41:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee via GitGitGadget X-Patchwork-Id: 11456947 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2071B17EF for ; Wed, 25 Mar 2020 05:41:33 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 01A2D20788 for ; Wed, 25 Mar 2020 05:41:33 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="qCDnX/Im" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726206AbgCYFlb (ORCPT ); Wed, 25 Mar 2020 01:41:31 -0400 Received: from mail-wr1-f67.google.com ([209.85.221.67]:37940 "EHLO mail-wr1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726102AbgCYFl2 (ORCPT ); Wed, 25 Mar 2020 01:41:28 -0400 Received: by mail-wr1-f67.google.com with SMTP id s1so1352160wrv.5 for ; Tue, 24 Mar 2020 22:41:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=ahQq2NZanyTLE/D0ebhWiNuIpzx43XZ+EEpggX/gI1k=; b=qCDnX/ImLC+zOcRqFmzCdPM2eZxc0QZAekIcH+xazM9PV+H1/lagZe42k+6M61YXI0 DH77tnSVn+qsVUgZ4YYhjw8T8oC53cTdbKbKRXJxQx0CFwg6tZtY8TZ5fxCBLYfFMC5P uJhu3SN3bbXzVBIsHVAUFNIOp5jIPXR/vNNT+o/+dYujLuR5xgE9O2LQNssfav1G0vRi PwYxVj2NfC/ahdNN7xANeUTKc2mMEQTmiV5ewa2vriZ8FbSbfve/ykwF51zxuroFsXJ6 9E/n1sd9Kt1v1WC4I3exVugva0DGQoUubWdcAaNjkibAoh1JHbAgF5nTJgqBM9Yb3DZv /NTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=ahQq2NZanyTLE/D0ebhWiNuIpzx43XZ+EEpggX/gI1k=; b=i2VoBtIOkuJg+ZRJ5JVc9nTvMIoA7hCNHHGVosSSIJo0nBrpzEYs9qhfSG1h64jEtF bDOzXmNgqKT59AW3eyLND/OMXdoy0xCVwn981e0BGjt0LejsPD1wdd2WPgPRL0GqqKDC /74db5io1xajflMC7ofkzxoXxZfmzt3lJdZSB4QULDAjLGGDmK5byiozdSMAmspLJTHs L1k7VWlnFiMxL2oy6/4dQKyegJkuI+KcNnPZu91TZy0/21VAUOBy5EhIB9d63OFwBtya z/Qk+s0hhfAHBEHEj1DvIpeorPHnWeZqG957gtV/ofOHTZ6aBIH+Rl7obG3WeEmnSZgh 379Q== X-Gm-Message-State: ANhLgQ0qFAGsl7PbXnjK2SAyQoHi1FP6pn+ULCrsulprgNOdYbkch1r4 c6Bpuh8t27udvsamKB5xSsRhHLag X-Google-Smtp-Source: ADFU+vvwrU+Ht9dsEEdzkXRvlPyWVP1GTYhUQoL36/8Dg8ejeR4UGR+e1hCJRXo7Gf6Q1Hy+FM4kXA== X-Received: by 2002:adf:82a6:: with SMTP id 35mr1406161wrc.307.1585114886275; Tue, 24 Mar 2020 22:41:26 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id h26sm7544883wmb.19.2020.03.24.22.41.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 24 Mar 2020 22:41:25 -0700 (PDT) Message-Id: <5e89b512513af0e2e16dc93c86ae3d1145061a82.1585114881.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Johannes Schindelin via GitGitGadget" Date: Wed, 25 Mar 2020 05:41:21 +0000 Subject: [PATCH v2 5/5] tests: increase the verbosity of the GPG-related prereqs Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Jeff King , Junio C Hamano , Johannes Schindelin , Johannes Schindelin Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Johannes Schindelin Especially when debugging a test failure that can only be reproduced in the CI build (e.g. when the developer has no access to a macOS machine other than running the tests on a macOS build agent), output should not be suppressed. In the instance of `hi/gpg-prefer-check-signature`, where one GPG-related test failed for no apparent reason, the entire output of `gpg` and `gpgsm` was suppressed, even in verbose mode, leaving interested readers no clue what was going wrong. Let's fix this by no longer redirecting the output not to `/dev/null`. This is now possible because the affected prereqs were turned into lazy ones (and are therefore evaluated via `test_eval_` which respects the `--verbose` option). Note that we _still_ redirect `stdout` to `/dev/null` for those commands that sign their `stdin`, as the output would be binary (and useless anyway, because the reader would not have anything against which to compare the output). Signed-off-by: Johannes Schindelin --- t/lib-gpg.sh | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/t/lib-gpg.sh b/t/lib-gpg.sh index 7a78c562e8d..9fc5241228e 100755 --- a/t/lib-gpg.sh +++ b/t/lib-gpg.sh @@ -40,12 +40,12 @@ test_lazy_prereq GPG ' # > lib-gpg/ownertrust mkdir "$GNUPGHOME" && chmod 0700 "$GNUPGHOME" && - (gpgconf --kill gpg-agent >/dev/null 2>&1 || : ) && - gpg --homedir "${GNUPGHOME}" 2>/dev/null --import \ + (gpgconf --kill gpg-agent || : ) && + gpg --homedir "${GNUPGHOME}" --import \ "$TEST_DIRECTORY"/lib-gpg/keyring.gpg && - gpg --homedir "${GNUPGHOME}" 2>/dev/null --import-ownertrust \ + gpg --homedir "${GNUPGHOME}" --import-ownertrust \ "$TEST_DIRECTORY"/lib-gpg/ownertrust && - gpg --homedir "${GNUPGHOME}" /dev/null 2>&1 \ + gpg --homedir "${GNUPGHOME}" /dev/null \ --sign -u committer@example.com ;; esac @@ -68,23 +68,23 @@ test_lazy_prereq GPGSM ' # gpgsm --homedir /tmp/gpghome/ \ # -o t/lib-gpg/gpgsm_cert.p12 \ # --export-secret-key-p12 "committer@example.com" - echo | gpgsm --homedir "${GNUPGHOME}" 2>/dev/null \ - --passphrase-fd 0 --pinentry-mode loopback \ - --import "$TEST_DIRECTORY"/lib-gpg/gpgsm_cert.p12 && + echo | gpgsm --homedir "${GNUPGHOME}" \ + --passphrase-fd 0 --pinentry-mode loopback \ + --import "$TEST_DIRECTORY"/lib-gpg/gpgsm_cert.p12 && - gpgsm --homedir "${GNUPGHOME}" 2>/dev/null -K | - grep fingerprint: | - cut -d" " -f4 | + gpgsm --homedir "${GNUPGHOME}" -K | + grep fingerprint: | + cut -d" " -f4 | tr -d "\\n" >"${GNUPGHOME}/trustlist.txt" && - echo " S relax" >>"${GNUPGHOME}/trustlist.txt" && - echo hello | gpgsm --homedir "${GNUPGHOME}" >/dev/null \ - -u committer@example.com -o /dev/null --sign - 2>&1 + echo " S relax" >>"${GNUPGHOME}/trustlist.txt" && + echo hello | gpgsm --homedir "${GNUPGHOME}" >/dev/null \ + -u committer@example.com -o /dev/null --sign - ' test_lazy_prereq RFC1991 ' test_have_prereq GPG && - echo | gpg --homedir "${GNUPGHOME}" -b --rfc1991 >/dev/null 2>&1 + echo | gpg --homedir "${GNUPGHOME}" -b --rfc1991 >/dev/null ' sanitize_pgp() {