From patchwork Wed Jul 27 23:13:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12930863 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id ECFF8C04A68 for ; Wed, 27 Jul 2022 23:13:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233037AbiG0XN4 (ORCPT ); Wed, 27 Jul 2022 19:13:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41692 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230347AbiG0XNx (ORCPT ); Wed, 27 Jul 2022 19:13:53 -0400 Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com [IPv6:2a00:1450:4864:20::32f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 522B84D17D for ; Wed, 27 Jul 2022 16:13:51 -0700 (PDT) Received: by mail-wm1-x32f.google.com with SMTP id b21-20020a05600c4e1500b003a32bc8612fso208036wmq.3 for ; Wed, 27 Jul 2022 16:13:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=xStBtPE5M0Au360I4uK5eYJJdLP0mEUY5gniDHaTijw=; b=m8gaOyEL+q/cPz/a7PSDbS5OmEiK/2jc/yTNusCgn1DFK6uBDIbbC44QvIT9A12FOD g1i5F0Ddpfh1kDxFFB+2apw+b31c2RLox5Vcl6ErJMWPF79DeUzaaHa6iEDQnRNqMn5z GgrMigFc5Gzy89vyQEcZwAzrMBf7OJztAhRPW1/QuBBVHs3DNrW33o2yz9P2KrpwdM2K DLv8yqjdePxXu2vOUThjSirpuEEblrhmJ3cTK6J/7huqheyOJfIgbUvdh+xQs29fochB 5xezDT5Nz1suX252qUeFQhxUsnan0UqYM2zmuQsocSrJeIplW0uv1gb+5M7Gl6Z/HX3u PG9A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=xStBtPE5M0Au360I4uK5eYJJdLP0mEUY5gniDHaTijw=; b=DTx4crNbdTXj59TxxuMLHJIK9vJV+5PHwDRYR4P6XUURPlc6U18c8K+8GygJbzehLv CTN6VzotVFN+igogBV4NeB2yTJUueJRKPHpCQAuvzujWGJb6tYMoNkxhjTASb8Nnhtwd F7P/pe3SJoxmdNqJPo0Ho0bixWO5czoeW7SvJutuU5FmKN+5st7bnJnYHSMkLhSriWfU wV/fpI9usk6l78RlCldwLNW/Na+OKdLfp3kHryRcel/QcwtI9UipNO8z4d0ykegEpDvR LOeN+fyeLAsViPQ703oHqH+Opt5FMcLTiwGbeQJP329BhzzRpXP/8lxURSlWx7+ZmavZ eMaQ== X-Gm-Message-State: AJIora9djrAMYS0kCFQu8tVIlOh0pwkP3qLRpH9XchHXVbVK0AiwRrA/ ycC6ulpcW6jwsFrGulTXHoTwpG6nLviWVw== X-Google-Smtp-Source: AGRyM1vfVwPhiCasrtRqJfYyRgkE4zUnN5ahrFJvLg5oMwO2R9R+DrHGEe5rTuWuGWAXSkwdQf8fEw== X-Received: by 2002:a1c:7c0d:0:b0:3a3:5e3f:497e with SMTP id x13-20020a1c7c0d000000b003a35e3f497emr4646647wmc.135.1658963629379; Wed, 27 Jul 2022 16:13:49 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id z3-20020adfec83000000b0021dd08ad8d7sm1692806wrn.46.2022.07.27.16.13.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Jul 2022 16:13:48 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v3 01/15] test-lib: use $1, not $@ in test_known_broken_{ok,failure}_ Date: Thu, 28 Jul 2022 01:13:29 +0200 Message-Id: X-Mailer: git-send-email 2.37.1.1127.g4ecf1c08f67 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Clarify that these two functions never take N arguments, they'll only ever receive one. They've needlessly used $@ over $1 since 41ac414ea2b (Sane use of test_expect_failure, 2008-02-01). In the future we might want to pass the test source to these, but now that's not the case. This preparatory change helps to clarify a follow-up change. Signed-off-by: Ævar Arnfjörð Bjarmason --- t/test-lib.sh | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/t/test-lib.sh b/t/test-lib.sh index 7726d1da88a..3f11ce35112 100644 --- a/t/test-lib.sh +++ b/t/test-lib.sh @@ -804,14 +804,14 @@ test_failure_ () { test_known_broken_ok_ () { test_fixed=$(($test_fixed+1)) - say_color error "ok $test_count - $@ # TODO known breakage vanished" - finalize_test_case_output fixed "$@" + say_color error "ok $test_count - $1 # TODO known breakage vanished" + finalize_test_case_output fixed "$1" } test_known_broken_failure_ () { test_broken=$(($test_broken+1)) - say_color warn "not ok $test_count - $@ # TODO known breakage" - finalize_test_case_output broken "$@" + say_color warn "not ok $test_count - $1 # TODO known breakage" + finalize_test_case_output broken "$1" } test_debug () { From patchwork Wed Jul 27 23:13:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12930865 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E6EAEC3F6B0 for ; Wed, 27 Jul 2022 23:14:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233241AbiG0XN6 (ORCPT ); Wed, 27 Jul 2022 19:13:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41698 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231495AbiG0XNy (ORCPT ); Wed, 27 Jul 2022 19:13:54 -0400 Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com [IPv6:2a00:1450:4864:20::32a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5EE774D4C5 for ; Wed, 27 Jul 2022 16:13:52 -0700 (PDT) Received: by mail-wm1-x32a.google.com with SMTP id v67-20020a1cac46000000b003a1888b9d36so1692780wme.0 for ; Wed, 27 Jul 2022 16:13:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=nnwUuz/jM4EM0NrDNxFRIEvdzFM+Qfdq5gD/PCOeC18=; b=dHVzDPMl9SP2pCyVVSFjRlrkPkG2tvTYcn2tkn0bXZ+FiSnURR9qZ02v5nr+S/v2VM 6WnNg+kl5IZMUrDanJgfzxoR0rW9vj8MYPYqVp6iZpXM2eyIm6QC9CIqUhA0wxPqzDTH fZAahl+l61fqwgtzNjAR1NYlZ5A0v5NUxEZ/yarAP5P43aDz2prvgjeQhe6ezgi11CJB RwzyN4eMUpQWjVISMS1RkmAYF8qne801BsH+5X8dten2obZ+PLIrChRshEVw1QRj4fAV /FOHNQ/3qA/pS9tfLPCP4c5m+I2CsrxhTM8J/xLcHYgIHzVFqxi52j6D27Nowmjwyy/k TUow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=nnwUuz/jM4EM0NrDNxFRIEvdzFM+Qfdq5gD/PCOeC18=; b=XIBr/k0BthG+UhMFGoV64WkDU4g9ku4vwFBmekvEIzAINpZfsuJ0v7gW3lsf6tUYZv bcz0cODFEYi0ihvbfHndLD2t6tTHUh9Fmf4L2xCqvr7pAvZaqLLsOoYbP0gA7EdaQj4l VCSJKPaaaF1XIyBgFkkGdtC3HGTgNUD6C0CkjNiiImA2pyGXwzabWs1qhd1vi5K0XkNU Qhyoek0XdZTI6NtRkYV98BDtsxYws2MBiJ/FFp/go+Ok3YvQQ2z6vbxt/bwII09Kl0rT U5N3FgQl+l23cJApl6O2JQynHREmgzpxb3d5tSbkgqC6pTCM0bDNAeHBdcepd6ZNTuZk Z6YQ== X-Gm-Message-State: AJIora97chOFhZT8rt4pTFdb06ebvy+bXJa161zwyBnqWbuGaeYOt/IW aoBleMSuSwoK0xcsrUq5ZUDg3gvetQkE9A== X-Google-Smtp-Source: AGRyM1sSUYWyTyEm2wAUTo2bnJtjE2PCP6QB38ScwK/MQb8PlgPLeLKE3U5B3AGbN2Wa0+gMI983jA== X-Received: by 2002:a05:600c:1d06:b0:3a3:19ee:6963 with SMTP id l6-20020a05600c1d0600b003a319ee6963mr4740685wms.89.1658963630614; Wed, 27 Jul 2022 16:13:50 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id z3-20020adfec83000000b0021dd08ad8d7sm1692806wrn.46.2022.07.27.16.13.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Jul 2022 16:13:49 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v3 02/15] test-lib: don't set GIT_EXIT_OK before calling test_atexit_handler Date: Thu, 28 Jul 2022 01:13:30 +0200 Message-Id: X-Mailer: git-send-email 2.37.1.1127.g4ecf1c08f67 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Change the control flow in test_done so that we'll set GIT_EXIT_OK=t after we call test_atexit_handler(). This seems to have been a mistake in 900721e15c4 (test-lib: introduce 'test_atexit', 2019-03-13). It doesn't make sense to allow our "atexit" handling to call "exit" without us emitting the errors we'll emit without GIT_EXIT_OK=t being set. Signed-off-by: Ævar Arnfjörð Bjarmason --- t/test-lib.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/t/test-lib.sh b/t/test-lib.sh index 3f11ce35112..c8c84ef9b14 100644 --- a/t/test-lib.sh +++ b/t/test-lib.sh @@ -1169,12 +1169,12 @@ test_atexit_handler () { } test_done () { - GIT_EXIT_OK=t - # Run the atexit commands _before_ the trash directory is # removed, so the commands can access pidfiles and socket files. test_atexit_handler + GIT_EXIT_OK=t + finalize_test_output if test -z "$HARNESS_ACTIVE" From patchwork Wed Jul 27 23:13:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12930866 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D53C2C04A68 for ; Wed, 27 Jul 2022 23:14:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234046AbiG0XOA (ORCPT ); Wed, 27 Jul 2022 19:14:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41710 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232724AbiG0XNz (ORCPT ); Wed, 27 Jul 2022 19:13:55 -0400 Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com [IPv6:2a00:1450:4864:20::42a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5B9F64D4C6 for ; Wed, 27 Jul 2022 16:13:53 -0700 (PDT) Received: by mail-wr1-x42a.google.com with SMTP id l4so82696wrm.13 for ; Wed, 27 Jul 2022 16:13:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=MxrZwK2IO/lfHzmu3Mgjck9SJ+jxnQ7/NsPXIxvVSqk=; b=SBHYzozUJ+StD5RGsouW5/WdHmNvgwnmcn9+26UXSDllFjrOV3kXmyzGzQXZy7ckFN 8HuZrAp+IkLFEnqQ+Y5+LCU4AHJhpeHu3oh5waGzVurTDbJGvdLDNd3KhCLr7i9I/jD6 TVw7SgafVQpvUPP9MrO4mLjO57qrWByj5MUdpa1d6nkHl/Y0506taQosr/anvWyXYzmD cc3Mda5tPpOkk79BKtJVXo9+ElK2uDfYvnwhrr5FWItM6Kyl1QPKQWbZSXejY/1ISWn1 C1yizNud0oV5MudhY0WfEoh9VnwPxgSJyA44RPpEWUGQUnOJPjjDW7WnnRG5R3fusYa2 JOjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=MxrZwK2IO/lfHzmu3Mgjck9SJ+jxnQ7/NsPXIxvVSqk=; b=L5UyOsHgdOHg5X7TDMvjwJdMPcKvMMJ0s3W8IP/vi76hNVUsbzg4RQaYdd64AS/g1C SKFgIlsvNN2LRfxXjhJsCXJph3NHCfV5ZGtzEoQVkpIGF7PqKbrrg4crD79KO3DH4O0O bQmyyzFyw8+NyV/cJ6GuVAly0NT5hDv2i15S9V1JSZNAuqscq4QwsmQOmzc/r8LTidPz 5KYfVZLGIAK0tBg2VEW4JH3k0rA9NE7obAJslSrLmyYJ5Kipo2D1FgMaq1zEpYjxt3Rg CzMuYar0wNo9p1Qxey0QlZ4lyvzAJKkGuV+Z/qeHhmRJB/1u5JLPyvL1Q1PVJUWMbC0m fIUA== X-Gm-Message-State: AJIora9cV48I6wbozihp1BxGQWeNF+gdjZ5FC02p54Kyjetrm302X2zo I6KWP2JRwS09pM5IkzWRm0qDw1mTBj0fVQ== X-Google-Smtp-Source: AGRyM1uR9Asz1q85W8Ld93hQJmYoCr26aJkBnI7B1u3DOFNLubIvQ/Ck7e6IBEu9SsLM2Tzg4AB8KA== X-Received: by 2002:a5d:4345:0:b0:21a:3b82:ad57 with SMTP id u5-20020a5d4345000000b0021a3b82ad57mr16105483wrr.176.1658963631544; Wed, 27 Jul 2022 16:13:51 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id z3-20020adfec83000000b0021dd08ad8d7sm1692806wrn.46.2022.07.27.16.13.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Jul 2022 16:13:50 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v3 03/15] test-lib: fix GIT_EXIT_OK logic errors, use BAIL_OUT Date: Thu, 28 Jul 2022 01:13:31 +0200 Message-Id: X-Mailer: git-send-email 2.37.1.1127.g4ecf1c08f67 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Change various "exit 1" checks that happened after our "die" handler had been set up to use BAIL_OUT instead. See 234383cd401 (test-lib.sh: use "Bail out!" syntax on bad SANITIZE=leak use, 2021-10-14) for the benefits of the BAIL_OUT function. The previous use of "error" here was not a logic error, but the "exit" without "GIT_EXIT_OK" would emit the "FATAL: Unexpected exit with code $code" message on top of the error we wanted to emit. Since we'd also like to stop "prove" in its tracks here, the right thing to do is to emit a "Bail out!" message. Let's also move the "GIT_EXIT_OK=t" assignments to just above the "exit [01]" in "test_done". It's not OK if we exit in e.g. finalize_test_output. Signed-off-by: Ævar Arnfjörð Bjarmason --- t/test-lib.sh | 16 ++++++---------- 1 file changed, 6 insertions(+), 10 deletions(-) diff --git a/t/test-lib.sh b/t/test-lib.sh index c8c84ef9b14..118720493bb 100644 --- a/t/test-lib.sh +++ b/t/test-lib.sh @@ -1173,8 +1173,6 @@ test_done () { # removed, so the commands can access pidfiles and socket files. test_atexit_handler - GIT_EXIT_OK=t - finalize_test_output if test -z "$HARNESS_ACTIVE" @@ -1246,6 +1244,7 @@ test_done () { fi test_at_end_hook_ + GIT_EXIT_OK=t exit 0 ;; *) @@ -1255,6 +1254,7 @@ test_done () { say "1..$test_count" fi + GIT_EXIT_OK=t exit 1 ;; esac @@ -1387,14 +1387,12 @@ fi GITPERLLIB="$GIT_BUILD_DIR"/perl/build/lib export GITPERLLIB test -d "$GIT_BUILD_DIR"/templates/blt || { - error "You haven't built things yet, have you?" + BAIL_OUT "You haven't built things yet, have you?" } if ! test -x "$GIT_BUILD_DIR"/t/helper/test-tool$X then - echo >&2 'You need to build test-tool:' - echo >&2 'Run "make t/helper/test-tool" in the source (toplevel) directory' - exit 1 + BAIL_OUT 'You need to build test-tool; Run "make t/helper/test-tool" in the source (toplevel) directory' fi # Are we running this test at all? @@ -1448,9 +1446,7 @@ remove_trash_directory () { # Test repository remove_trash_directory "$TRASH_DIRECTORY" || { - GIT_EXIT_OK=t - echo >&5 "FATAL: Cannot prepare test area" - exit 1 + BAIL_OUT 'cannot prepare test area' } remove_trash=t @@ -1466,7 +1462,7 @@ fi # Use -P to resolve symlinks in our working directory so that the cwd # in subprocesses like git equals our $PWD (for pathname comparisons). -cd -P "$TRASH_DIRECTORY" || exit 1 +cd -P "$TRASH_DIRECTORY" || BAIL_OUT "cannot cd -P to \"$TRASH_DIRECTORY\"" start_test_output "$0" From patchwork Wed Jul 27 23:13:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12930868 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BA3F8C04A68 for ; Wed, 27 Jul 2022 23:14:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234184AbiG0XOD (ORCPT ); Wed, 27 Jul 2022 19:14:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41712 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229532AbiG0XNz (ORCPT ); Wed, 27 Jul 2022 19:13:55 -0400 Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8CCEF4C63A for ; Wed, 27 Jul 2022 16:13:54 -0700 (PDT) Received: by mail-wr1-x429.google.com with SMTP id l22so106164wrz.7 for ; Wed, 27 Jul 2022 16:13:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=R+BrETSMlNBsU78LOI9BAZy4FNrlsZono2VQHwJpy7c=; b=Yxv6KSgAsiPACldMb+PdxjTkEr1fD0SqZLn7Q5o+osYOlwcA+1h1Vn3fEu1TyFObbG nwqpjjLh10T1de6f0fVxgFojNgpjKKAlQyiYsUP8lc5o2tlHTeo91HFnIhyQa19+qh+8 ZRLp1gRyVtGxXx71b58k3sUsz4l0ADLmidVqnvRCBEzUsd4SkUFwolK0BodgoSiQZiO3 BIiBEhD9nY1Ef3QDwmDIhIO2wQQ88cuM8IxksWAoryPfAUJSisbBVix9jtPkN7yUPI9x /Sa2mLn8IvlAaVEyKEEEGqy2wmA1h3mE0AtsS/txS6gfWEPIvnJVqi12ZIXHr56Im+Np ejrA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=R+BrETSMlNBsU78LOI9BAZy4FNrlsZono2VQHwJpy7c=; b=QxL4hPEuxo8PCyih9CUoZIiNxzrlKp6BJpQJfAI+vMbxshDJ3o4eaOroCSdqsGYTg0 vxVqmrWddV/l+VLCQ7hbwhkgaCeATmS6tbhhp+svSujQxD6DLykz6al5d2Bx4SKns8NG n3xDiXstxZ/NB8zq+bLFlM7E4a6Il7cJQoEREuQR7uh9K2njSvB2EUkEEOysvLWw4bv0 xI+cpq+6jiQdy7YSAUdCBHY8XMvcv00ta7KbpY8V0isGBU9cM1TilqiGeYPuh8R5N3Zs 8D+LlzUPSJVyCMFTC0ehMpJfPE7myRJIhXJXcwwr1igeGKFdGRkxRzEAziXx5PEuF3W7 lenA== X-Gm-Message-State: AJIora/cqbVAScU5YIqw+LsXsQ36T0xyqO8lorfIgrxFAykfEBLDGi4k Ga6faWeN0uLvx7T+NtPBilghByPMV33AEA== X-Google-Smtp-Source: AGRyM1v5inzjZalCzcob4em6NGFcaDXyztfpexqUN7aaBsPDG+K9NJIezIqSGoB50CWjXaiLG8yKUA== X-Received: by 2002:a5d:47c1:0:b0:21d:cfe2:fb00 with SMTP id o1-20020a5d47c1000000b0021dcfe2fb00mr15342193wrc.495.1658963632750; Wed, 27 Jul 2022 16:13:52 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id z3-20020adfec83000000b0021dd08ad8d7sm1692806wrn.46.2022.07.27.16.13.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Jul 2022 16:13:51 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v3 04/15] test-lib: add a --invert-exit-code switch Date: Thu, 28 Jul 2022 01:13:32 +0200 Message-Id: X-Mailer: git-send-email 2.37.1.1127.g4ecf1c08f67 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Add the ability to have those tests that fail return 0, and those tests that succeed return 1. This is useful e.g. to run "--stress" tests on tests that fail 99% of the time on some setup, i.e. to smoke out the flaky run which yielded success. In a subsequent commit a new SANITIZE=leak mode will make use of this. Signed-off-by: Ævar Arnfjörð Bjarmason --- t/t0000-basic.sh | 72 ++++++++++++++++++++++++++++++++++++++++++++++++ t/test-lib.sh | 45 ++++++++++++++++++++++++++++-- 2 files changed, 115 insertions(+), 2 deletions(-) diff --git a/t/t0000-basic.sh b/t/t0000-basic.sh index 17a268ccd1b..502b4bcf9ea 100755 --- a/t/t0000-basic.sh +++ b/t/t0000-basic.sh @@ -578,6 +578,78 @@ test_expect_success 'subtest: --run invalid range end' ' EOF_ERR ' +test_expect_success 'subtest: --invert-exit-code without --immediate' ' + run_sub_test_lib_test_err full-pass \ + --invert-exit-code && + check_sub_test_lib_test_err full-pass \ + <<-\EOF_OUT 3<<-EOF_ERR + ok 1 - passing test #1 + ok 2 - passing test #2 + ok 3 - passing test #3 + # passed all 3 test(s) + 1..3 + # faking up non-zero exit with --invert-exit-code + EOF_OUT + EOF_ERR +' + +test_expect_success 'subtest: --invert-exit-code with --immediate: all passed' ' + run_sub_test_lib_test_err full-pass \ + --invert-exit-code --immediate && + check_sub_test_lib_test_err full-pass \ + <<-\EOF_OUT 3<<-EOF_ERR + ok 1 - passing test #1 + ok 2 - passing test #2 + ok 3 - passing test #3 + # passed all 3 test(s) + 1..3 + # faking up non-zero exit with --invert-exit-code + EOF_OUT + EOF_ERR +' + +test_expect_success 'subtest: --invert-exit-code without --immediate: partial pass' ' + run_sub_test_lib_test partial-pass \ + --invert-exit-code && + check_sub_test_lib_test partial-pass <<-\EOF + ok 1 - passing test #1 + not ok 2 - # TODO induced breakage (--invert-exit-code): failing test #2 + # false + ok 3 - passing test #3 + # failed 1 among 3 test(s) + 1..3 + # faked up failures as TODO & now exiting with 0 due to --invert-exit-code + EOF +' + +test_expect_success 'subtest: --invert-exit-code with --immediate: partial pass' ' + run_sub_test_lib_test partial-pass \ + --invert-exit-code --immediate && + check_sub_test_lib_test partial-pass \ + <<-\EOF_OUT 3<<-EOF_ERR + ok 1 - passing test #1 + not ok 2 - # TODO induced breakage (--invert-exit-code): failing test #2 + # false + 1..2 + # faked up failures as TODO & now exiting with 0 due to --invert-exit-code + EOF_OUT + EOF_ERR +' + +test_expect_success 'subtest: --invert-exit-code --immediate: got a failure' ' + run_sub_test_lib_test partial-pass \ + --invert-exit-code --immediate && + check_sub_test_lib_test_err partial-pass \ + <<-\EOF_OUT 3<<-EOF_ERR + ok 1 - passing test #1 + not ok 2 - # TODO induced breakage (--invert-exit-code): failing test #2 + # false + 1..2 + # faked up failures as TODO & now exiting with 0 due to --invert-exit-code + EOF_OUT + EOF_ERR +' + test_expect_success 'subtest: tests respect prerequisites' ' write_and_run_sub_test_lib_test prereqs <<-\EOF && diff --git a/t/test-lib.sh b/t/test-lib.sh index 118720493bb..31213b5f95b 100644 --- a/t/test-lib.sh +++ b/t/test-lib.sh @@ -238,6 +238,9 @@ parse_option () { ;; esac ;; + --invert-exit-code) + invert_exit_code=t + ;; *) echo "error: unknown test option '$opt'" >&2; exit 1 ;; esac @@ -788,15 +791,31 @@ test_ok_ () { finalize_test_case_output ok "$@" } +_invert_exit_code_failure_end_blurb () { + say_color warn "# faked up failures as TODO & now exiting with 0 due to --invert-exit-code" +} + test_failure_ () { failure_label=$1 test_failure=$(($test_failure + 1)) - say_color error "not ok $test_count - $1" + local pfx="" + if test -n "$invert_exit_code" # && test -n "$HARNESS_ACTIVE" + then + pfx="# TODO induced breakage (--invert-exit-code):" + fi + say_color error "not ok $test_count - ${pfx:+$pfx }$1" shift printf '%s\n' "$*" | sed -e 's/^/# /' if test -n "$immediate" then say_color error "1..$test_count" + if test -n "$invert_exit_code" + then + finalize_test_output + _invert_exit_code_failure_end_blurb + GIT_EXIT_OK=t + exit 0 + fi _error_exit fi finalize_test_case_output failure "$failure_label" "$@" @@ -1229,7 +1248,14 @@ test_done () { esac fi - if test -z "$debug" && test -n "$remove_trash" + if test -n "$stress" && test -n "$invert_exit_code" + then + # We're about to move our "$TRASH_DIRECTORY" + # to "$TRASH_DIRECTORY.stress-failed" if + # --stress is combined with + # --invert-exit-code. + say "with --stress and --invert-exit-code we're not removing '$TRASH_DIRECTORY'" + elif test -z "$debug" && test -n "$remove_trash" then test -d "$TRASH_DIRECTORY" || error "Tests passed but trash directory already removed before test cleanup; aborting" @@ -1242,6 +1268,14 @@ test_done () { } || error "Tests passed but test cleanup failed; aborting" fi + + if test -z "$skip_all" && test -n "$invert_exit_code" + then + say_color warn "# faking up non-zero exit with --invert-exit-code" + GIT_EXIT_OK=t + exit 1 + fi + test_at_end_hook_ GIT_EXIT_OK=t @@ -1254,6 +1288,13 @@ test_done () { say "1..$test_count" fi + if test -n "$invert_exit_code" + then + _invert_exit_code_failure_end_blurb + GIT_EXIT_OK=t + exit 0 + fi + GIT_EXIT_OK=t exit 1 ;; From patchwork Wed Jul 27 23:13:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12930867 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DA315C19F29 for ; Wed, 27 Jul 2022 23:14:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234936AbiG0XOF (ORCPT ); Wed, 27 Jul 2022 19:14:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41744 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233151AbiG0XN5 (ORCPT ); Wed, 27 Jul 2022 19:13:57 -0400 Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 44E294D4C6 for ; Wed, 27 Jul 2022 16:13:56 -0700 (PDT) Received: by mail-wr1-x431.google.com with SMTP id bn9so98780wrb.9 for ; Wed, 27 Jul 2022 16:13:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=+ydImvf6cfVwO1RsBPo7CKzgDBylTtd8okEswqW+BgI=; b=JgysCvpOaiUEjeyow4YHTbiZKUWqYALIKbi5EK6Udop/lBcEwYPj9AUZ49gj/5L/nl OwG4YbfiywPISH+NKM9ksVOTm0aBiOXA6XsoGy0pW+vdEjqMUXC+t8nmLQKCWN5Djqg6 vG31qkecicikMG/GkeYWRlG0PL3TBneIgHJQEJHYKG/saNs0kQC7BIaOTaoGDSbmwWDh VfBfE9YFQo7ePXO2WSpI/nxnWZxz/HaVFWZDT5TBm+r1sM3TqUYIQMPAwIieTkhCyVXl 5VfyKIFUToCmGUKqn5neRSJVcNoGZJVytooMCRoOYhHThHwgqrNkTC3jMPdWgAyo6Auv QuLw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=+ydImvf6cfVwO1RsBPo7CKzgDBylTtd8okEswqW+BgI=; b=IJjBVmai12RA/81/LcfiH2/gNqOJ0h4jB3uKDXL5uVkKY9O8P7Ag/N/OJHiYziWLI8 Ie9nQfagnkgMFwQfKkSY6gQlsSvfjhM1M8OFUaXPPyXyeWdSSg++xY2PW1lK64UlSrlQ 3A9M43gzTx7I3KUhlkF8RCVApDzhtABJ1vMSZEF5yShxJtBkNVdXjrbbia/W+EBowbb3 CDoYreD1fgsit6w92Q90poXy72o4LyGOnaBxcDL7M9RxnTHQCmiqNWy2G6VvNLO2yZNp IUBiYXe4VhdcCWOMRnBiazAmdWVQ6ynKWQGpwkrnCZJxZnFOjQSvIRca21GdWK/0nSNP DJfQ== X-Gm-Message-State: AJIora/n9bRtVfnv4ODE2CpiiC3qXwyQwu6+xTznCgVy2ro281EbBctb 4uPi3XIKU5sjQ5labhI+7fEr7XOgybNvcg== X-Google-Smtp-Source: AGRyM1uLKSB95sdw7uHg8Ty5BrrZ2YaBTq3jl6Y9JUgiAlNiDJmU8xJQSVjdcFpUzw2tyt49CkBlhg== X-Received: by 2002:a5d:47ab:0:b0:21d:b2b1:59b1 with SMTP id 11-20020a5d47ab000000b0021db2b159b1mr15148931wrb.99.1658963634425; Wed, 27 Jul 2022 16:13:54 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id z3-20020adfec83000000b0021dd08ad8d7sm1692806wrn.46.2022.07.27.16.13.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Jul 2022 16:13:53 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v3 05/15] t/README: reword the "GIT_TEST_PASSING_SANITIZE_LEAK" description Date: Thu, 28 Jul 2022 01:13:33 +0200 Message-Id: X-Mailer: git-send-email 2.37.1.1127.g4ecf1c08f67 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Reword the documentation added in 956d2e4639b (tests: add a test mode for SANITIZE=leak, run it in CI, 2021-09-23) for brevity. The comment added in the same commit was also misleading: We skip certain tests if SANITIZE=leak and GIT_TEST_PASSING_SANITIZE_LEAK=true, not if we're compiled with SANITIZE=leak. Let's just remove the comment, the control flow here is obvious enough that the code can speak for itself. Signed-off-by: Ævar Arnfjörð Bjarmason --- t/README | 10 ++++------ t/test-lib.sh | 1 - 2 files changed, 4 insertions(+), 7 deletions(-) diff --git a/t/README b/t/README index 4f9981cf5e3..7f409392537 100644 --- a/t/README +++ b/t/README @@ -366,12 +366,10 @@ excluded as so much relies on it, but this might change in the future. GIT_TEST_SPLIT_INDEX= forces split-index mode on the whole test suite. Accept any boolean values that are accepted by git-config. -GIT_TEST_PASSING_SANITIZE_LEAK= when compiled with -SANITIZE=leak will run only those tests that have whitelisted -themselves as passing with no memory leaks. Tests can be whitelisted -by setting "TEST_PASSES_SANITIZE_LEAK=true" before sourcing -"test-lib.sh" itself at the top of the test script. This test mode is -used by the "linux-leaks" CI target. +GIT_TEST_PASSING_SANITIZE_LEAK=true skips those tests that haven't +declared themselves as leak-free by setting +"TEST_PASSES_SANITIZE_LEAK=true" before sourcing "test-lib.sh". This +test mode is used by the "linux-leaks" CI target. GIT_TEST_PROTOCOL_VERSION=, when set, makes 'protocol.version' default to n. diff --git a/t/test-lib.sh b/t/test-lib.sh index 31213b5f95b..f8adb92f027 100644 --- a/t/test-lib.sh +++ b/t/test-lib.sh @@ -1447,7 +1447,6 @@ then test_done fi -# skip non-whitelisted tests when compiled with SANITIZE=leak if test -n "$SANITIZE_LEAK" then if test_bool_env GIT_TEST_PASSING_SANITIZE_LEAK false From patchwork Wed Jul 27 23:13:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12930869 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B3138C19F21 for ; Wed, 27 Jul 2022 23:14:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233637AbiG0XOH (ORCPT ); Wed, 27 Jul 2022 19:14:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41764 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233576AbiG0XN6 (ORCPT ); Wed, 27 Jul 2022 19:13:58 -0400 Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com [IPv6:2a00:1450:4864:20::32a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4A70A4D4C9 for ; Wed, 27 Jul 2022 16:13:57 -0700 (PDT) Received: by mail-wm1-x32a.google.com with SMTP id b6so69690wmq.5 for ; Wed, 27 Jul 2022 16:13:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=89J4/LsuXFq/+4absQtheu32r0lGk3u6TrBlHx8EMZM=; b=KLV9muENCV+8UnKk/QM04IJNICCYbwisnoLorJvqDO7wzvBTbGuMswXTEC8bpWIB+P KtdBUiNTIEAQtIDD0ug9h67blLNFD58AX4Mb++TZnoZSzPn4nWkEXFLuqRoLr2EV0Yky paZk41TC1rEMPDLcz8veskUN9/3IWSU13P6N6CLssgV/IJBPTKW2csDsScMMRfg1qdA6 SahWrlldLbuNEfAobnttA8pj4onsmFJ/4DmP89uzWJ9wje44xDDRcfxB+3Bi04uTfrmt Ec+cy1C2WP9PkuSydge2HlAzPvz5Ky1QuRSgliaEy1K9E9lYG7QeVLFp9kGkCJILuW3y qHtQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=89J4/LsuXFq/+4absQtheu32r0lGk3u6TrBlHx8EMZM=; b=jjio0adaj8Y+Gh7PROn0felWvEx0RR72jfNMi+Tp+gOz2EJy32pws3IWIXv7TvW77k bu4JL9YVJeUrK2nkfmXt2ucFvrzmjrYMWVU7FQEyod2An3iwPIr8aLUEX1JQ5v2Vm8St 34J3N6asWQ61TC5giinwZhb1nLMP9k0YsgsvDkPG4Uh7e4mOedPV3uGqCApObdyTQqvu yo8ZApHdyldNJwpKo9RtsusYSOBq7w2ocr0CY3fOaquqMd4WO5Y/QH5fA46LGM2cUFhv goY6pclLF+bGcsF3HhPgVJXDW+Y4DcpozB4jAVxgbcPbA3M12fFK4xYuxsJyZZnVkVFj 2g7g== X-Gm-Message-State: AJIora9wRWdabQC1HId9QDh4sKjgizIvWMdHKkkH7gCZ7a9T8a62jFDh 9kWGtDWwUp4mcZvRwXzQ/iwK220WrsX/fg== X-Google-Smtp-Source: AGRyM1vxpylstO9+dCNj8i0yWSgX2BAexvuhPDzTtw6gUx8wXjn17KABFKIUDkWY7s/kqTPw/Cv/ug== X-Received: by 2002:a05:600c:364b:b0:3a3:7e97:8f10 with SMTP id y11-20020a05600c364b00b003a37e978f10mr2983330wmq.152.1658963635480; Wed, 27 Jul 2022 16:13:55 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id z3-20020adfec83000000b0021dd08ad8d7sm1692806wrn.46.2022.07.27.16.13.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Jul 2022 16:13:54 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v3 06/15] test-lib: add a SANITIZE=leak logging mode Date: Thu, 28 Jul 2022 01:13:34 +0200 Message-Id: X-Mailer: git-send-email 2.37.1.1127.g4ecf1c08f67 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Add the ability to run the test suite under a new "GIT_TEST_SANITIZE_LEAK_LOG=true" mode, when true we'll log the leaks we find an a new "test-results/.leak" directory. That new path is consistent with the existing "test-results/." results, except that those are all files, not directories. We also set "log_exe_name=1" to include the name of the executable in the filename. This gives us files like "trace.git." instead of the default of "trace.". I.e. we'll be able to distinguish "git" leaks from "test-tool", "git-daemon" etc. We then set "dedup_token_length" to non-zero ("0" is the default) to succinctly log a token we can de-duplicate these stacktraces on. The string is simply a one-line stack-trace with only function names up to N frames, which we limit at "9999" as a shorthand for "infinite" (there appears to be no way to say "no limit"). With these combined we can now easily get e.g. the top 10 leaks in the test suite grouped by full stacktrace: grep -o -P -h '(?<=DEDUP_TOKEN: ).*' test-results/*.leak/trace.git.* | sort | uniq -c | sort -nr | head -n 10 Or add "grep -E -o '[^-]+'" to that to group by functions instead of stack traces: grep -o -P -h '(?<=DEDUP_TOKEN: ).*' test-results/*.leak/trace.git.* | grep -E -o '[^-]+' | sort | uniq -c | sort -nr | head -n 20 This new mode requires git to be compiled with SANITIZE=leak, rather than explaining that in the documentation let's make it self-documenting by bailing out if the user asks for this without git having been compiled with SANITIZE=leak, as we do with GIT_TEST_PASSING_SANITIZE_LEAK=true. Signed-off-by: Ævar Arnfjörð Bjarmason --- t/README | 5 +++++ t/test-lib.sh | 30 +++++++++++++++++++++++++++++- 2 files changed, 34 insertions(+), 1 deletion(-) diff --git a/t/README b/t/README index 7f409392537..0664aee7ed9 100644 --- a/t/README +++ b/t/README @@ -371,6 +371,11 @@ declared themselves as leak-free by setting "TEST_PASSES_SANITIZE_LEAK=true" before sourcing "test-lib.sh". This test mode is used by the "linux-leaks" CI target. +GIT_TEST_SANITIZE_LEAK_LOG=true will log memory leaks to +"test-results/$TEST_NAME.leak/trace.*" files. The logs include a +"dedup_token" (see +"ASAN_OPTIONS=help=1 ./git") and other options to +make logs +machine-readable. + GIT_TEST_PROTOCOL_VERSION=, when set, makes 'protocol.version' default to n. diff --git a/t/test-lib.sh b/t/test-lib.sh index f8adb92f027..557f77c9717 100644 --- a/t/test-lib.sh +++ b/t/test-lib.sh @@ -305,6 +305,10 @@ TEST_NUMBER="${TEST_NAME%%-*}" TEST_NUMBER="${TEST_NUMBER#t}" TEST_RESULTS_DIR="$TEST_OUTPUT_DIRECTORY/test-results" TEST_RESULTS_BASE="$TEST_RESULTS_DIR/$TEST_NAME$TEST_STRESS_JOB_SFX" +TEST_RESULTS_SAN_FILE_PFX=trace +TEST_RESULTS_SAN_DIR_SFX=leak +TEST_RESULTS_SAN_FILE= +TEST_RESULTS_SAN_DIR="$TEST_RESULTS_DIR/$TEST_NAME.$TEST_RESULTS_SAN_DIR_SFX" TRASH_DIRECTORY="trash directory.$TEST_NAME$TEST_STRESS_JOB_SFX" test -n "$root" && TRASH_DIRECTORY="$root/$TRASH_DIRECTORY" case "$TRASH_DIRECTORY" in @@ -1447,6 +1451,10 @@ then test_done fi +BAIL_OUT_ENV_NEEDS_SANITIZE_LEAK () { + BAIL_OUT "$1 has no effect except when compiled with SANITIZE=leak" +} + if test -n "$SANITIZE_LEAK" then if test_bool_env GIT_TEST_PASSING_SANITIZE_LEAK false @@ -1461,9 +1469,29 @@ then test_done fi fi + + if test_bool_env GIT_TEST_SANITIZE_LEAK_LOG false + then + if ! mkdir -p "$TEST_RESULTS_SAN_DIR" + then + BAIL_OUT "cannot create $TEST_RESULTS_SAN_DIR" + fi && + TEST_RESULTS_SAN_FILE="$TEST_RESULTS_SAN_DIR/$TEST_RESULTS_SAN_FILE_PFX" + + # Don't litter *.leak dirs if there was nothing to report + test_atexit "rmdir \"$TEST_RESULTS_SAN_DIR\" 2>/dev/null || :" + + prepend_var LSAN_OPTIONS : dedup_token_length=9999 + prepend_var LSAN_OPTIONS : log_exe_name=1 + prepend_var LSAN_OPTIONS : log_path=\"$TEST_RESULTS_SAN_FILE\" + export LSAN_OPTIONS + fi elif test_bool_env GIT_TEST_PASSING_SANITIZE_LEAK false then - BAIL_OUT "GIT_TEST_PASSING_SANITIZE_LEAK=true has no effect except when compiled with SANITIZE=leak" + BAIL_OUT_ENV_NEEDS_SANITIZE_LEAK "GIT_TEST_PASSING_SANITIZE_LEAK=true" +elif test_bool_env GIT_TEST_SANITIZE_LEAK_LOG false +then + BAIL_OUT_ENV_NEEDS_SANITIZE_LEAK "GIT_TEST_SANITIZE_LEAK_LOG=true" fi # Last-minute variable setup From patchwork Wed Jul 27 23:13:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12930870 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F3A6EC04A68 for ; Wed, 27 Jul 2022 23:14:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231495AbiG0XOI (ORCPT ); Wed, 27 Jul 2022 19:14:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41836 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234028AbiG0XOA (ORCPT ); Wed, 27 Jul 2022 19:14:00 -0400 Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 266214D4DF for ; Wed, 27 Jul 2022 16:13:59 -0700 (PDT) Received: by mail-wm1-x32c.google.com with SMTP id i205-20020a1c3bd6000000b003a2fa488efdso1660854wma.4 for ; Wed, 27 Jul 2022 16:13:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=r5jnN7/O+TAVw3sF6mVenDS/A42VH7+SHRQ5vcEMHRo=; b=T5Eh2xpiM8U125UZTcs1BNK3+FBNk2H9HBZlLb1oqbxnKtVpEtCYLa1aotqYE9L5pw y0uAIj6I/kleeWnblb4hf3V6p74ItQqwr6rJvPHEqwTjknf9numYslt0tJ5dibksDB3y yt8CdadQxRqBZEhGQWaxYzhMKSjfjfMUKHJUgw6iUCrQzNXH0g+nD+OPkQ+1gQcjVqwm CJQoFLlfKT+veHx3QUdjnrxZi1TtXFvg/6Qagv77Is8SxMdjV/1XoO6XdVhBjkInGbT9 +iA7bNnvyrgT0jqqP+IjQK0kse44gvoUTk7qSmeSQF+Rg25lFEZ6R5xwC+Kdi1tiygxd ZXBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=r5jnN7/O+TAVw3sF6mVenDS/A42VH7+SHRQ5vcEMHRo=; b=sbaTXxG7NymqtzrnVXPcIKH1+QZ8rNn1rbOwpSC7+c9nHeU+Lb2qTztV16m3+u81Om yyTqucizTgx+ziwXl8ADbZrOjc4vzrUmIsQLpzIwz80/nKD/AgOQxNZAfhguavNaukII nEAd6zQSOQpG8f/2DJpZddB8txaVNEvxBHt86zG3XqDrf5TTagHztnXWI5hxSVYGlOb0 aqR/cvO3d0OnBrAprn3MvRhqb/7oPBc3vFDozGOKoBFxdrjo1wjeDstPBiA5Vz6MUzqk FR9VIg2dwTS6e5+Pp+N+a33Kojt8ohcBbKJBMO3XOd42BsJx0NFwYjxrRoJCMKG+BENW Eekg== X-Gm-Message-State: AJIora97mjM123tYa8+su7lskwGDXszCkWLyxPmu7StPOZlfirWtHYAv pkNqi7da24J01FLW9UQCSOpNWpp8cMoM2g== X-Google-Smtp-Source: AGRyM1sNT1Wm/nobNGxkzmYOqZxKd7fV4eNwFhZE5LzqpOmLp3MCb9PIhgESpGYcrrvWjxkT4jtuYQ== X-Received: by 2002:a05:600c:6006:b0:3a3:7245:7be4 with SMTP id az6-20020a05600c600600b003a372457be4mr4603638wmb.112.1658963637184; Wed, 27 Jul 2022 16:13:57 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id z3-20020adfec83000000b0021dd08ad8d7sm1692806wrn.46.2022.07.27.16.13.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Jul 2022 16:13:56 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v3 07/15] t/Makefile: don't remove test-results in "clean-except-prove-cache" Date: Thu, 28 Jul 2022 01:13:35 +0200 Message-Id: X-Mailer: git-send-email 2.37.1.1127.g4ecf1c08f67 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When "make test" is run with the default of "DEFAULT_TEST_TARGET=test" we'll leave the "test-results" directory in-place, but don't do so for the "prove" target. The reason for this is that when 28d836c8158 (test: allow running the tests under "prove", 2010-10-14) allowed for running the tests under "prove" there was no point in leaving the "test-results" in place. The "prove" target provides its own summary, so we don't need to run "aggregate-results", which is the reason we have "test-results" in the first place. See 2d84e9fb6d2 (Modify test-lib.sh to output stats to t/test-results/*, 2008-06-08). But in a subsequent commit test-lib.sh will start emitting reports of memory leaks in test-results/*, and it will be useful to analyze these after the fact. This wouldn't be a problem as failing tests will halt the removal of the files (we'll never reach "clean-except-prove-cache" from the "prove" target), but will be subsequently as we'll want to report a successful run, but might still have e.g. logs of known memory leaks in test-results/*. So let's stop removing this, it's sufficient that "make clean" removes it, and that "pre-clean" (which both "test" and "prove" depend on) will remove it, i.e. we'll never have a stale "test-results" because of this change. Signed-off-by: Ævar Arnfjörð Bjarmason --- contrib/scalar/t/Makefile | 2 +- contrib/subtree/t/Makefile | 2 +- t/Makefile | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/contrib/scalar/t/Makefile b/contrib/scalar/t/Makefile index 01e82e56d15..1ed174a8cf3 100644 --- a/contrib/scalar/t/Makefile +++ b/contrib/scalar/t/Makefile @@ -42,7 +42,7 @@ $(T): @echo "*** $@ ***"; GIT_CONFIG=.git/config '$(SHELL_PATH_SQ)' $@ $(GIT_TEST_OPTS) clean-except-prove-cache: - $(RM) -r 'trash directory'.* '$(TEST_RESULTS_DIRECTORY_SQ)' + $(RM) -r 'trash directory'.* $(RM) -r valgrind/bin clean: clean-except-prove-cache diff --git a/contrib/subtree/t/Makefile b/contrib/subtree/t/Makefile index 276898eb6bd..3d278bb0edb 100644 --- a/contrib/subtree/t/Makefile +++ b/contrib/subtree/t/Makefile @@ -47,7 +47,7 @@ pre-clean: $(RM) -r '$(TEST_RESULTS_DIRECTORY_SQ)' clean-except-prove-cache: - $(RM) -r 'trash directory'.* '$(TEST_RESULTS_DIRECTORY_SQ)' + $(RM) -r 'trash directory'.* $(RM) -r valgrind/bin clean: clean-except-prove-cache diff --git a/t/Makefile b/t/Makefile index 7f56e52f767..1c80c0c79a0 100644 --- a/t/Makefile +++ b/t/Makefile @@ -62,7 +62,7 @@ pre-clean: $(RM) -r '$(TEST_RESULTS_DIRECTORY_SQ)' clean-except-prove-cache: clean-chainlint - $(RM) -r 'trash directory'.* '$(TEST_RESULTS_DIRECTORY_SQ)' + $(RM) -r 'trash directory'.* $(RM) -r valgrind/bin clean: clean-except-prove-cache From patchwork Wed Jul 27 23:13:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12930871 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2C70DC3F6B0 for ; Wed, 27 Jul 2022 23:14:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237286AbiG0XOL (ORCPT ); Wed, 27 Jul 2022 19:14:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41888 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234161AbiG0XOB (ORCPT ); Wed, 27 Jul 2022 19:14:01 -0400 Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7FC864D810 for ; Wed, 27 Jul 2022 16:14:00 -0700 (PDT) Received: by mail-wm1-x32c.google.com with SMTP id id17so82090wmb.1 for ; Wed, 27 Jul 2022 16:14:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=EfXI4BqFk7qqSdXMOoW1YUSBAaPHIUHmDrndxXs/XGM=; b=cXeA1srZE9MEQUgiTl8XxU9LzLdOj536wWJbUeb60fUD7x57dPpA78qXBDfN0vGTfE CyRB0iyHvYmFtT/taZbz2nIzSASP3Wp5W3DSVS5c2E5UCiHxCKQLE+m+R+aVGtD9PoFa XLZAYaNK8ps0ApUc7hG9WhBPltxA1haOzn0oHn5IUcJE9x7xyEbUYv05MMLAGJyRSLfx /Sd4wKjlxmnfmU9v8qph7gEhZKwYqr8mqHK0sctWNnAJOIOd14DdodjJEIwmsKyY96wJ Zat96Y+Le6Uzvspn7NXzKdpieZ1NRELKjR9kMtU0kAInGDy6nG6ikEXBtsyrsefuq2G6 51tQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=EfXI4BqFk7qqSdXMOoW1YUSBAaPHIUHmDrndxXs/XGM=; b=CW2p3q12dIiHc9GJbUZQrBHi6Z1L1lHh7LgGn6ZeKaBhxe2ge9oX7uksewILHO6Mgj nn6Sz9Z0zF00tEvQbJHLxojS8g/RpLWkNlXFBTv3ZDkA+1361cAC6Cqknh+tVdIOryIA GWfIOaN1MT0m+KPXEEllj24PYWXgQh25JHEa6RYgP6ZT0+MK0jXIVBN1Gu6/DS0IRCVo D3SIHsDdlykvrhKk8lAR6wSACpbO1mA8iZZqUCWInYOewTTLeL98x3dgQjtxNT+wJF2q 3xpoQuv8Y8l+kin6mUQ+IODTJ8wZHZsLQiTc/tRNKBIbRx1xdUrMHC4MGBGZzWYxW1nw B9sA== X-Gm-Message-State: AJIora+F+TOjaXcd6etskromZEUswN7+upJl5YRIlvMV2tbggtSrpPfP zKxUjuxjhLV3D4kdmhNTUX+6G8t2AJXIng== X-Google-Smtp-Source: AGRyM1shsLvLXNHxHsLl+aViHy1sEmR1hTULI3cIL5KAKmIXko4ZBFLQ7EUoGFgxZuiMoRWqD63NxQ== X-Received: by 2002:a7b:c8ce:0:b0:3a3:2cf8:edb with SMTP id f14-20020a7bc8ce000000b003a32cf80edbmr4626256wml.70.1658963638550; Wed, 27 Jul 2022 16:13:58 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id z3-20020adfec83000000b0021dd08ad8d7sm1692806wrn.46.2022.07.27.16.13.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Jul 2022 16:13:57 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v3 08/15] tests: move copy/pasted PERL + Test::More checks to a lib-perl.sh Date: Thu, 28 Jul 2022 01:13:36 +0200 Message-Id: X-Mailer: git-send-email 2.37.1.1127.g4ecf1c08f67 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Since the original "perl -MTest::More" prerequisite check was added in [1] it's been copy/pasted in [2], [3] and [4]. As we'll be changing these codepaths in a subsequent commit let's consolidate these. While we're at it let's move these to a lazy prereq, and make them conform to our usual coding style (e.g. "\nthen", not "; then"). 1. e46f9c8161a (t9700: skip when Test::More is not available, 2008-06-29) 2. 5e9637c6297 (i18n: add infrastructure for translating Git with gettext, 2011-11-18) 3. 8d314d7afec (send-email: reduce dependencies impact on parse_address_line, 2015-07-07) 4. f07eeed123b (git-credential-netrc: adapt to test framework for git, 2018-05-12) Signed-off-by: Ævar Arnfjörð Bjarmason --- .../netrc/t-git-credential-netrc.sh | 11 ++--------- t/lib-perl.sh | 19 +++++++++++++++++++ t/t0202-gettext-perl.sh | 12 ++---------- t/t9700-perl-git.sh | 11 ++--------- 4 files changed, 25 insertions(+), 28 deletions(-) create mode 100644 t/lib-perl.sh diff --git a/contrib/credential/netrc/t-git-credential-netrc.sh b/contrib/credential/netrc/t-git-credential-netrc.sh index 07227d02287..ff17a9460cd 100755 --- a/contrib/credential/netrc/t-git-credential-netrc.sh +++ b/contrib/credential/netrc/t-git-credential-netrc.sh @@ -3,16 +3,9 @@ cd ../../../t test_description='git-credential-netrc' . ./test-lib.sh + . "$TEST_DIRECTORY"/lib-perl.sh - if ! test_have_prereq PERL; then - skip_all='skipping perl interface tests, perl not available' - test_done - fi - - perl -MTest::More -e 0 2>/dev/null || { - skip_all="Perl Test::More unavailable, skipping test" - test_done - } + skip_all_if_no_Test_More # set up test repository diff --git a/t/lib-perl.sh b/t/lib-perl.sh new file mode 100644 index 00000000000..d0bf509a167 --- /dev/null +++ b/t/lib-perl.sh @@ -0,0 +1,19 @@ +# Copyright (c) 2022 Ævar Arnfjörð Bjarmason + +test_lazy_prereq PERL_TEST_MORE ' + perl -MTest::More -e 0 +' + +skip_all_if_no_Test_More () { + if ! test_have_prereq PERL + then + skip_all='skipping perl interface tests, perl not available' + test_done + fi + + if ! test_have_prereq PERL_TEST_MORE + then + skip_all="Perl Test::More unavailable, skipping test" + test_done + fi +} diff --git a/t/t0202-gettext-perl.sh b/t/t0202-gettext-perl.sh index df2ea34932b..043b190626c 100755 --- a/t/t0202-gettext-perl.sh +++ b/t/t0202-gettext-perl.sh @@ -7,16 +7,8 @@ test_description='Perl gettext interface (Git::I18N)' TEST_PASSES_SANITIZE_LEAK=true . ./lib-gettext.sh - -if ! test_have_prereq PERL; then - skip_all='skipping perl interface tests, perl not available' - test_done -fi - -perl -MTest::More -e 0 2>/dev/null || { - skip_all="Perl Test::More unavailable, skipping test" - test_done -} +. "$TEST_DIRECTORY"/lib-perl.sh +skip_all_if_no_Test_More # The external test will outputs its own plan test_external_has_tap=1 diff --git a/t/t9700-perl-git.sh b/t/t9700-perl-git.sh index 102c133112c..17fc43f6e57 100755 --- a/t/t9700-perl-git.sh +++ b/t/t9700-perl-git.sh @@ -5,16 +5,9 @@ test_description='perl interface (Git.pm)' . ./test-lib.sh +. "$TEST_DIRECTORY"/lib-perl.sh -if ! test_have_prereq PERL; then - skip_all='skipping perl interface tests, perl not available' - test_done -fi - -perl -MTest::More -e 0 2>/dev/null || { - skip_all="Perl Test::More unavailable, skipping test" - test_done -} +skip_all_if_no_Test_More # set up test repository From patchwork Wed Jul 27 23:13:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 12930872 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4CD6FC04A68 for ; Wed, 27 Jul 2022 23:14:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234079AbiG0XON (ORCPT ); Wed, 27 Jul 2022 19:14:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41968 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234189AbiG0XOE (ORCPT ); Wed, 27 Jul 2022 19:14:04 -0400 Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com [IPv6:2a00:1450:4864:20::332]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D76764E61E for ; Wed, 27 Jul 2022 16:14:01 -0700 (PDT) Received: by mail-wm1-x332.google.com with SMTP id c22so80224wmr.2 for ; Wed, 27 Jul 2022 16:14:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=EOFI7IwXVuc67tBRMNAtCNPzjqwlgtg95pRvNx+NBGM=; b=AIO4D4/zpeIt5Pswhy79MV6oRGXFgrg7Wpu+6nElB7RHQA+sobPld00avsw+t3sU90 sDi1jpjMyl0kduQDChsKPH3lnarKTMZHA03QAEAJ1U1XxJOE/G5PB6rYqLus5/ryzXOA ZYsigbMbZCQbULvltIQgfaBUqfpctsv7itDi353tyBZpqUTVxq4aVZ2vViNkC6LODs3C HvdqmXLwncgr2OZar6RnNJtO1XxXgMgwCqzJ92jacRt1MzTSjq0t2BXtQqpZWsRLjews TITN7fl5krdlcsoRUCTj89uM1U7eQWagptfsNPBTy8uldR6PygsPG7ZXHC+z7Ut5AydN jDMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=EOFI7IwXVuc67tBRMNAtCNPzjqwlgtg95pRvNx+NBGM=; b=S8S8Sxdf7kI3MrA5yKGPg8xfAzsNAnr3Djtvq8mOFH5u78frb5oe4X8AXJ61Vk9Bi7 qXl0QqOwAWA3O4ALe7/IyuN1xBycN+6MYd/+ESfhXoPa3HU2UbvDEUE39xhh6xzrTxFS 8U99vE/02lAWzEGHfQA9bxAU4DoLR2sItTTGL2W2CAp03JRzv/129Kdw+nW/1AEyoP/M 3rMBjsZpLaxFcnWE5nXhMXChTrFBNZy/mrAPgtbfRGq3k1nziT5H62JA+ktiKim7siaG 5pdLgDUgEKZWaRfDyE0JM2jOEPymKIvwuO0YPl94ZO93KvIWsceV+Qr2elmYzJQMHfCc LXIQ== X-Gm-Message-State: AJIora81u3RfaKKvhZaYsUYx1NG6YstoHJAXkr0s6/i0VZ4mnDX/jyoL w9B9z0golB66N3qb/sOwM6obs7oxGcVIWw== X-Google-Smtp-Source: AGRyM1txdJX+/64yZ84lObcX3r11J2QjbNm1XDbrDlP0IXz4BZ7LThhjS/wQrcV3HgMlvmdt3p/S4w== X-Received: by 2002:a1c:3886:0:b0:3a3:7ad9:b203 with SMTP id f128-20020a1c3886000000b003a37ad9b203mr3912669wma.189.1658963639524; Wed, 27 Jul 2022 16:13:59 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id z3-20020adfec83000000b0021dd08ad8d7sm1692806wrn.46.2022.07.27.16.13.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Jul 2022 16:13:58 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Derrick Stolee , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= Subject: [PATCH v3 09/15] test-lib: simplify by removing test_external Date: Thu, 28 Jul 2022 01:13:37 +0200 Message-Id: X-Mailer: git-send-email 2.37.1.1127.g4ecf1c08f67 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Remove the "test_external" function added in [1]. This arguably makes the output of t9700-perl-git.sh and friends worse. But as we'll argue below the trade-off is worth it, since "chaining" to another TAP emitter in test-lib.sh is more trouble than it's worth. The new output of t9700-perl-git.sh is now: $ ./t9700-perl-git.sh ok 1 - set up test repository ok 2 - use t9700/test.pl to test Git.pm # passed all 2 test(s) 1..2 Whereas before this change it would be: $ ./t9700-perl-git.sh ok 1 - set up test repository # run 1: Perl API (perl /home/avar/g/git/t/t9700/test.pl) ok 2 - use Git; [... omitting tests 3..46 from t/t9700/test.pl ...] ok 47 - unquote escape sequences 1..47 # test_external test Perl API was ok # test_external_without_stderr test no stderr: Perl API was ok At the time of its addition supporting "test_external" was easy, but when test-lib.sh itself started to emit TAP in [2] we needed to make everything surrounding the emission of the plan consider "test_external". I added that support in [2] so that we could run: prove ./t9700-perl-git.sh :: -v But since then in [3] the door has been closed on combining $HARNESS_ACTIVE and -v, we'll now just die: $ prove ./t9700-perl-git.sh :: -v Bailout called. Further testing stopped: verbose mode forbidden under TAP harness; try --verbose-log FAILED--Further testing stopped: verbose mode forbidden under TAP harness; try --verbose-log So the only use of this has been that *if* we had failure in one of these tests we could e.g. in CI see which test failed based on the test number. Now we'll need to look at the full verbose logs to get that same information. I think this trade-off is acceptable given the reduction in complexity, and it brings these tests in line with other similar tests, e.g. the reftable tests added in [4] will be condensed down to just one test, which invokes the C helper: $ ./t0032-reftable-unittest.sh ok 1 - unittests # passed all 1 test(s) 1..1 It would still be nice to have that ":: -v" form work again, it never *really* worked, but even though we've had edge cases test output screwing up the TAP it mostly worked between d998bd4ab67 and [3], so we may have been overzealous in forbidding it outright. I have local patches which I'm planning to submit sooner than later that get us to that goal, and in a way that isn't buggy. In the meantime getting rid of this special case makes hacking on this area of test-lib.sh easier, as we'll do in subsequent commits. The switch from "perl" to "$PERL_PATH" here is because "perl" is defined as a shell function in the test suite, see a5bf824f3b4 (t: prevent '-x' tracing from interfering with test helpers' stderr, 2018-02-25). On e.g. the OSX CI the "command perl"... will be part of the emitted stderr. 1. fb32c410087 (t/test-lib.sh: add test_external and test_external_without_stderr, 2008-06-19) 2. d998bd4ab67 (test-lib: Make the test_external_* functions TAP-aware, 2010-06-24) 3. 614fe015212 (test-lib: bail out when "-v" used under "prove", 2016-10-22) 4. ef8a6c62687 (reftable: utility functions, 2021-10-07) Signed-off-by: Ævar Arnfjörð Bjarmason --- .../netrc/t-git-credential-netrc.sh | 7 +- t/README | 26 ------ t/t0202-gettext-perl.sh | 10 +-- t/t9700-perl-git.sh | 10 +-- t/test-lib-functions.sh | 89 +------------------ t/test-lib.sh | 40 ++++----- 6 files changed, 28 insertions(+), 154 deletions(-) diff --git a/contrib/credential/netrc/t-git-credential-netrc.sh b/contrib/credential/netrc/t-git-credential-netrc.sh index ff17a9460cd..bf2777308a5 100755 --- a/contrib/credential/netrc/t-git-credential-netrc.sh +++ b/contrib/credential/netrc/t-git-credential-netrc.sh @@ -13,13 +13,10 @@ 'set up test repository' \ 'git config --add gpg.program test.git-config-gpg' - # The external test will outputs its own plan - test_external_has_tap=1 - export PERL5LIB="$GITPERLLIB" - test_external \ - 'git-credential-netrc' \ + test_expect_success 'git-credential-netrc' ' perl "$GIT_BUILD_DIR"/contrib/credential/netrc/test.pl + ' test_done ) diff --git a/t/README b/t/README index 0664aee7ed9..98f69ed13d2 100644 --- a/t/README +++ b/t/README @@ -938,32 +938,6 @@ see test-lib-functions.sh for the full list and their options. test_done fi - - test_external []