From patchwork Fri Feb 23 23:33:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Steadmon X-Patchwork-Id: 13570137 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8D74B143C63 for ; Fri, 23 Feb 2024 23:34:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708731243; cv=none; b=UIQDbeMvHCgO57w5P5krRGgMJOzpvc9e0rP8eozy3oq9iQW/L0pzCbJbi6/KlaIotUIeP2goYqd8XoGuU0af2IYxMmGU9/LM+TDFZlY/E9gAhMzo2mlass84FnaNspyRUlLle2+S0HOxUKiB0SMRU7atkXUlmSAZzguTnEtZ8Rc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708731243; c=relaxed/simple; bh=DDH7wFYR6nVZ1zmBGZgR/8UW+hIVhJ3OsOCtUx2RZP4=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=OcojqG0qost8dQ9nUvXz3x58EYxiyVxVyOHAhWo8RCTfP0vzRI81WDvuf7M17D14WhX7hk4hb+j+ppklKoZCBxh2LYek66rtT/wfjFQjSeAJ0m3ZWGrbWKa3VfXjVsGbdf/4hE6eMgsCAt7ozdwsvo+zOQzxDgf4jdD9+DOVu5Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=DQ/xvLrh; arc=none smtp.client-ip=209.85.219.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="DQ/xvLrh" Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-dcbfe1a42a4so2540527276.2 for ; Fri, 23 Feb 2024 15:34:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1708731240; x=1709336040; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=J518rWlaP09F54TLZsERPa+sRbw7YYooqKtklLkyZrE=; b=DQ/xvLrhHDVCp7daIllBttUW5KSZsfPwenGvq3yY53QzkOFpcNZYfwQ93uOxKCsjxN DBEbeLVpZCm+ba8SgoAUGol5iY2W027RE5T9oulfgwx0GGywPsmeyPyWR76xTkKwegRG DHaKpnbBNNQ6pbu1zgIX09SQ4mxSPOuhwOiNfjX/uhPWXHa7in938irzzvsdJBxstM5b jONLLrurC9ms7VfAKb2+VXAiRuhWzNo1C97O/WaNHpinm+mJKJeamQ26Tog0P6bBHf7T egGBbgsfWm8lM6fBBNZWJUf5SE+M0QfYY1LPwr19A5I22T1pNHT8d2wNaz5RUZcy/zbI BtEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708731240; x=1709336040; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=J518rWlaP09F54TLZsERPa+sRbw7YYooqKtklLkyZrE=; b=f+/0cLx7HT1p956bxMdlGBFY99a4cxCfFu3k01HiEtPsIWJSPLpQ+fLwhEh5rx0ebD lDciD25YbKW4yMfzqz+WVDJ9y1REBMM/YyuRwa52syq9s7GfZTsAS6cOUmv5VNQac9xE MlLAMijqgSgBz0knloD+KMxHzG3oRvuO1AWr2AOCD9uhGzWGvdFkc7cCDIXiX9dIdw/T x2GwhOarSiyZ0WtPJT/D18FHjrbAO/KjLjBifcwjAlZyovClZRwcRhT0nf8x+x4pPSo1 bcrd78S7zg46/Mr8J3Z3X7e71TIaDLzQvfpMFfNiqQGovIbNTSWMEBQvNTFGTP5YI0IJ caNA== X-Gm-Message-State: AOJu0Yz+hYKbVmCuuhg7YdyXm3jYKjNOsSYyeJPrAoRlWqMgXh25bfnR dSUWu7ONxL6XtlMyOx51zYPsRXCGPqd6ACsbRFtOktIFozQYWAvOaPsc41j3dv8Q+qHuWwjs38K YSWFcktHtzaQ33+L6rPVujA+zlQ7wTNgKCEpjs6hYGC+/Zg7bfATPwJDUQTjEpbnbwWnn9VU2d9 s1bsc/bM+Du9Qugj3YYR7AMvMYtCa9BrhhyiW3YdY= X-Google-Smtp-Source: AGHT+IFffw8Mb8VIZ+ysa/e00XBinDw96KfkwSuizCutnkbGB++XsWtU1cjml5Ii3TBPDEytXBoUAGfHB7qpFg== X-Received: from lunarfall.svl.corp.google.com ([2620:15c:2d3:204:2fd9:97d1:a490:26b8]) (user=steadmon job=sendgmr) by 2002:a05:6902:154f:b0:dc6:d678:371d with SMTP id r15-20020a056902154f00b00dc6d678371dmr344419ybu.3.1708731240303; Fri, 23 Feb 2024 15:34:00 -0800 (PST) Date: Fri, 23 Feb 2024 15:33:50 -0800 In-Reply-To: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.44.0.rc0.258.g7320e95886-goog Message-ID: <677745110078501e611faf3d1facd1518e0453a2.1708728717.git.steadmon@google.com> Subject: [PATCH v3 1/7] t0080: turn t-basic unit test into a helper From: Josh Steadmon To: git@vger.kernel.org Cc: johannes.schindelin@gmx.de, peff@peff.net, phillip.wood@dunelm.org.uk, gitster@pobox.com While t/unit-tests/t-basic.c uses the unit-test framework added in e137fe3b29 (unit tests: add TAP unit test framework, 2023-11-09), it is not a true unit test in that it intentionally fails in order to exercise various codepaths in the unit-test framework. Thus, we intentionally exclude it when running unit tests through the various t/Makefile targets. Instead, it is executed by t0080-unit-test-output.sh, which verifies its output follows the TAP format expected for the various pass, skip, or fail cases. As such, it makes more sense for t-basic to be a helper item for t0080-unit-test-output.sh, so let's move it to t/helper/test-example-tap.c and adjust Makefiles as necessary. Signed-off-by: Josh Steadmon --- Makefile | 4 ++-- t/Makefile | 2 +- .../t-basic.c => helper/test-example-tap.c} | 5 ++-- t/helper/test-tool.c | 1 + t/helper/test-tool.h | 1 + t/t0080-unit-test-output.sh | 24 +++++++++---------- 6 files changed, 20 insertions(+), 17 deletions(-) rename t/{unit-tests/t-basic.c => helper/test-example-tap.c} (95%) diff --git a/Makefile b/Makefile index 23723367b8..ba55d817ee 100644 --- a/Makefile +++ b/Makefile @@ -802,6 +802,7 @@ TEST_BUILTINS_OBJS += test-dump-split-index.o TEST_BUILTINS_OBJS += test-dump-untracked-cache.o TEST_BUILTINS_OBJS += test-env-helper.o TEST_BUILTINS_OBJS += test-example-decorate.o +TEST_BUILTINS_OBJS += test-example-tap.o TEST_BUILTINS_OBJS += test-find-pack.o TEST_BUILTINS_OBJS += test-fsmonitor-client.o TEST_BUILTINS_OBJS += test-genrandom.o @@ -1338,7 +1339,6 @@ THIRD_PARTY_SOURCES += compat/regex/% THIRD_PARTY_SOURCES += sha1collisiondetection/% THIRD_PARTY_SOURCES += sha1dc/% -UNIT_TEST_PROGRAMS += t-basic UNIT_TEST_PROGRAMS += t-mem-pool UNIT_TEST_PROGRAMS += t-strbuf UNIT_TEST_PROGRAMS += t-ctype @@ -3218,7 +3218,7 @@ perf: all .PRECIOUS: $(TEST_OBJS) -t/helper/test-tool$X: $(patsubst %,t/helper/%,$(TEST_BUILTINS_OBJS)) +t/helper/test-tool$X: $(patsubst %,t/helper/%,$(TEST_BUILTINS_OBJS)) $(UNIT_TEST_DIR)/test-lib.o t/helper/test-%$X: t/helper/test-%.o GIT-LDFLAGS $(GITLIBS) $(REFTABLE_TEST_LIB) $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(filter %.a,$^) $(LIBS) diff --git a/t/Makefile b/t/Makefile index 2d95046f26..4861edafe6 100644 --- a/t/Makefile +++ b/t/Makefile @@ -48,7 +48,7 @@ CHAINLINTTESTS = $(sort $(patsubst chainlint/%.test,%,$(wildcard chainlint/*.tes CHAINLINT = '$(PERL_PATH_SQ)' chainlint.pl UNIT_TEST_SOURCES = $(wildcard unit-tests/t-*.c) UNIT_TEST_PROGRAMS = $(patsubst unit-tests/%.c,unit-tests/bin/%$(X),$(UNIT_TEST_SOURCES)) -UNIT_TESTS = $(sort $(filter-out unit-tests/bin/t-basic%,$(UNIT_TEST_PROGRAMS))) +UNIT_TESTS = $(sort $(UNIT_TEST_PROGRAMS)) # `test-chainlint` (which is a dependency of `test-lint`, `test` and `prove`) # checks all tests in all scripts via a single invocation, so tell individual diff --git a/t/unit-tests/t-basic.c b/t/helper/test-example-tap.c similarity index 95% rename from t/unit-tests/t-basic.c rename to t/helper/test-example-tap.c index fda1ae59a6..d072ad559f 100644 --- a/t/unit-tests/t-basic.c +++ b/t/helper/test-example-tap.c @@ -1,4 +1,5 @@ -#include "test-lib.h" +#include "test-tool.h" +#include "t/unit-tests/test-lib.h" /* * The purpose of this "unit test" is to verify a few invariants of the unit @@ -69,7 +70,7 @@ static void t_empty(void) ; /* empty */ } -int cmd_main(int argc, const char **argv) +int cmd__example_tap(int argc, const char **argv) { test_res = TEST(check_res = check_int(1, ==, 1), "passing test"); TEST(t_res(1), "passing test and assertion return 1"); diff --git a/t/helper/test-tool.c b/t/helper/test-tool.c index 33b9501c21..bb5c04c9c0 100644 --- a/t/helper/test-tool.c +++ b/t/helper/test-tool.c @@ -29,6 +29,7 @@ static struct test_cmd cmds[] = { { "dump-untracked-cache", cmd__dump_untracked_cache }, { "env-helper", cmd__env_helper }, { "example-decorate", cmd__example_decorate }, + { "example-tap", cmd__example_tap }, { "find-pack", cmd__find_pack }, { "fsmonitor-client", cmd__fsmonitor_client }, { "genrandom", cmd__genrandom }, diff --git a/t/helper/test-tool.h b/t/helper/test-tool.h index b72f07ded9..38001bd1c6 100644 --- a/t/helper/test-tool.h +++ b/t/helper/test-tool.h @@ -23,6 +23,7 @@ int cmd__dump_untracked_cache(int argc, const char **argv); int cmd__dump_reftable(int argc, const char **argv); int cmd__env_helper(int argc, const char **argv); int cmd__example_decorate(int argc, const char **argv); +int cmd__example_tap(int argc, const char **argv); int cmd__find_pack(int argc, const char **argv); int cmd__fsmonitor_client(int argc, const char **argv); int cmd__genrandom(int argc, const char **argv); diff --git a/t/t0080-unit-test-output.sh b/t/t0080-unit-test-output.sh index 961b54b06c..83b1e3b7f5 100755 --- a/t/t0080-unit-test-output.sh +++ b/t/t0080-unit-test-output.sh @@ -8,50 +8,50 @@ test_expect_success 'TAP output from unit tests' ' cat >expect <<-EOF && ok 1 - passing test ok 2 - passing test and assertion return 1 - # check "1 == 2" failed at t/unit-tests/t-basic.c:76 + # check "1 == 2" failed at t/helper/test-example-tap.c:77 # left: 1 # right: 2 not ok 3 - failing test ok 4 - failing test and assertion return 0 not ok 5 - passing TEST_TODO() # TODO ok 6 - passing TEST_TODO() returns 1 - # todo check ${SQ}check(x)${SQ} succeeded at t/unit-tests/t-basic.c:25 + # todo check ${SQ}check(x)${SQ} succeeded at t/helper/test-example-tap.c:26 not ok 7 - failing TEST_TODO() ok 8 - failing TEST_TODO() returns 0 - # check "0" failed at t/unit-tests/t-basic.c:30 + # check "0" failed at t/helper/test-example-tap.c:31 # skipping test - missing prerequisite - # skipping check ${SQ}1${SQ} at t/unit-tests/t-basic.c:32 + # skipping check ${SQ}1${SQ} at t/helper/test-example-tap.c:33 ok 9 - test_skip() # SKIP ok 10 - skipped test returns 1 # skipping test - missing prerequisite ok 11 - test_skip() inside TEST_TODO() # SKIP ok 12 - test_skip() inside TEST_TODO() returns 1 - # check "0" failed at t/unit-tests/t-basic.c:48 + # check "0" failed at t/helper/test-example-tap.c:49 not ok 13 - TEST_TODO() after failing check ok 14 - TEST_TODO() after failing check returns 0 - # check "0" failed at t/unit-tests/t-basic.c:56 + # check "0" failed at t/helper/test-example-tap.c:57 not ok 15 - failing check after TEST_TODO() ok 16 - failing check after TEST_TODO() returns 0 - # check "!strcmp("\thello\\\\", "there\"\n")" failed at t/unit-tests/t-basic.c:61 + # check "!strcmp("\thello\\\\", "there\"\n")" failed at t/helper/test-example-tap.c:62 # left: "\011hello\\\\" # right: "there\"\012" - # check "!strcmp("NULL", NULL)" failed at t/unit-tests/t-basic.c:62 + # check "!strcmp("NULL", NULL)" failed at t/helper/test-example-tap.c:63 # left: "NULL" # right: NULL - # check "${SQ}a${SQ} == ${SQ}\n${SQ}" failed at t/unit-tests/t-basic.c:63 + # check "${SQ}a${SQ} == ${SQ}\n${SQ}" failed at t/helper/test-example-tap.c:64 # left: ${SQ}a${SQ} # right: ${SQ}\012${SQ} - # check "${SQ}\\\\${SQ} == ${SQ}\\${SQ}${SQ}" failed at t/unit-tests/t-basic.c:64 + # check "${SQ}\\\\${SQ} == ${SQ}\\${SQ}${SQ}" failed at t/helper/test-example-tap.c:65 # left: ${SQ}\\\\${SQ} # right: ${SQ}\\${SQ}${SQ} not ok 17 - messages from failing string and char comparison - # BUG: test has no checks at t/unit-tests/t-basic.c:91 + # BUG: test has no checks at t/helper/test-example-tap.c:92 not ok 18 - test with no checks ok 19 - test with no checks returns 0 1..19 EOF - ! "$GIT_BUILD_DIR"/t/unit-tests/bin/t-basic >actual && + ! test-tool example-tap >actual && test_cmp expect actual ' From patchwork Fri Feb 23 23:33:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Steadmon X-Patchwork-Id: 13570138 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1EE8614DFD1 for ; Fri, 23 Feb 2024 23:34:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708731244; cv=none; b=hVhwS9VYbHCJIZSPM3/ijq+aSKnUI5WMJ9LQ7tLh0RsSb590+9OdYtvsMBM34rB+BzBgG/QE/Ns7CNxqlLvE4ob8kR3iYGHWNoRRQqLXQkOgUKDMa10KoxH5md9e3OQryKWfukiuQPEpHUr9+th0QySokyYZRZcyTQEuet9/xn8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708731244; c=relaxed/simple; bh=xbyv/p4g5RVs8l8BGqjKzWm3YMd1NpBnFMGMGjCauyU=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=uEhT4lS0sGuQqnogUtNZOEjh5El7cPC9zzfO8i0ALm9cdQNau9PoiaqCF/7m08LyZbkGqlUlM9I+1TZaSxcQstSOHv1D+J2iE64JPDFnCyhwQcSlhkt9kCBHFIAPVk6QGnqVEvsgi8mEKQotyd2ilBlGncmOlmv5EIh7Oy4wHEs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=BTVKvgmz; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="BTVKvgmz" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-608b648044fso24104217b3.3 for ; Fri, 23 Feb 2024 15:34:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1708731242; x=1709336042; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=6j1LMcw6vsmLYWz1zCuGgzvqIqQyphfQxgvW3vUB92c=; b=BTVKvgmzIoK5Up6wNfavXInX2iYW5EgXoH0urAolmQtDpOL0QLj/8QHeQy3mFegnYJ uSQZTCl1HpgDJeu/ZiI8kNM1kc9+v0LAPeekWrzZXarTnVNw9Qgho4ouQg0tkli0pb2Y Bdp1oaDh7QideIcTm12s0xxWUrvrqrrTI0ixPvKV1iAUNWKPh5EjPRnUcg5Y70INLaLI GTMtgBafkUvuTdEHdMkqz1mNGzunAuFOiLxbV1b3FbPhkdZsvbaQSlH4sInKtYWU/XLv pF95A5ympoNcAnJhP4rKPxYI60LBQBoXEPdZMKeTq4cFKp/cPIDArSHTzhg6r1w9CwQs C+Hg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708731242; x=1709336042; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=6j1LMcw6vsmLYWz1zCuGgzvqIqQyphfQxgvW3vUB92c=; b=VQ3ZsFhupisnwF91RzFYlEjwdblnPrtfgRypSOB0HXqPJVDkQQSWtD+LGA1ZCljvO5 670zfKceXwmbB95cDyYGlLT1mmtOPHaIOXkxqu+dqyqQzNYt3Nt0ZpO05ZSsCUAirXZE vGq8YMq/YjOUEdXIx5EwrfUcw5akzIqCNQb304EyU5AIJxDt9NhN4kKUk6+u38CeCVuB CtC0UyLwaRgKocGVdIUe0mfNTDTTw7tJOqao8lJgkqhryB2nRDWzTx0ugy+nagOYvqGS EzmfqMIFhTy1ou/Uw/zkgk1w99rlOha392YyTomDdTL8GOYXsaIor83hWIBDRpUE88kn 7Mhw== X-Gm-Message-State: AOJu0YxgQ300jgD1j0NMjX4NxRNu7GOASxFlKS1BYB2yYsqR4D2KC0tD SlbSW3cGT8mOUN/VfRB7HQWGUrql+2gmf+RmrLtRUHqkMg2mR1QdKtYc4HbJZnPGG6uX70iBWX/ mlE9JW6SrbBf0NsM4tl1XmWVkeKkMvYaMegsFObpXBUaKb5Din4czTFQlOsg4pcRCskDNLf1Wtr mHRiRICwyLaE0YVVzXyIQi96BOne/hfy21ZGgIEuk= X-Google-Smtp-Source: AGHT+IHbVYUb3BOH3QLFwmFEiUyFsHw+52vddjlb+zIQJBEsigSFFNrX1ZGo80NO8YLdatjCbfVYbP83263nWQ== X-Received: from lunarfall.svl.corp.google.com ([2620:15c:2d3:204:2fd9:97d1:a490:26b8]) (user=steadmon job=sendgmr) by 2002:a0d:d456:0:b0:608:5367:a508 with SMTP id w83-20020a0dd456000000b006085367a508mr300990ywd.2.1708731241971; Fri, 23 Feb 2024 15:34:01 -0800 (PST) Date: Fri, 23 Feb 2024 15:33:51 -0800 In-Reply-To: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.44.0.rc0.258.g7320e95886-goog Message-ID: <24f47f8fc7e931c2e15fa985635e3036ec1123ba.1708728717.git.steadmon@google.com> Subject: [PATCH v3 2/7] test-tool run-command testsuite: get shell from env From: Josh Steadmon To: git@vger.kernel.org Cc: johannes.schindelin@gmx.de, peff@peff.net, phillip.wood@dunelm.org.uk, gitster@pobox.com When running tests through `test-tool run-command testsuite`, we currently hardcode `sh` as the command interpreter. As discussed in [1], this is incorrect, and we should be using the shell set in TEST_SHELL_PATH instead. Add a shell_path field in struct testsuite so that we can pass this to the task runner callback. If this is non-null, we'll use it as the argv[0] of the subprocess. Otherwise, we'll just execute the test program directly. We will use this feature in a later commit to enable running binary executable unit tests. However, for now when setting up the struct testsuite in testsuite(), use the value of TEST_SHELL_PATH if it's set, otherwise keep the original behavior by defaulting to `sh`. [1] https://lore.kernel.org/git/20240123005913.GB835964@coredump.intra.peff.net/ Signed-off-by: Josh Steadmon --- t/helper/test-run-command.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/t/helper/test-run-command.c b/t/helper/test-run-command.c index c0ed8722c8..a41a54d9cb 100644 --- a/t/helper/test-run-command.c +++ b/t/helper/test-run-command.c @@ -65,6 +65,7 @@ struct testsuite { struct string_list tests, failed; int next; int quiet, immediate, verbose, verbose_log, trace, write_junit_xml; + const char *shell_path; }; #define TESTSUITE_INIT { \ .tests = STRING_LIST_INIT_DUP, \ @@ -80,7 +81,9 @@ static int next_test(struct child_process *cp, struct strbuf *err, void *cb, return 0; test = suite->tests.items[suite->next++].string; - strvec_pushl(&cp->args, "sh", test, NULL); + if (suite->shell_path) + strvec_push(&cp->args, suite->shell_path); + strvec_push(&cp->args, test); if (suite->quiet) strvec_push(&cp->args, "--quiet"); if (suite->immediate) @@ -162,6 +165,10 @@ static int testsuite(int argc, const char **argv) if (max_jobs <= 0) max_jobs = online_cpus(); + suite.shell_path = getenv("TEST_SHELL_PATH"); + if (!suite.shell_path) + suite.shell_path = "sh"; + dir = opendir("."); if (!dir) die("Could not open the current directory"); From patchwork Fri Feb 23 23:33:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Steadmon X-Patchwork-Id: 13570139 Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 134C714DFF5 for ; Fri, 23 Feb 2024 23:34:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708731246; cv=none; b=ZkX0ZthSd9VUmQAYGCWICnWTDWsjWjMRLnVWzSi130w2PopwgLnCdwR021H2HcX+P8Hk4yByhKL7NlsHAvxypHSD9Ebri5qYugzKBEbJ8UU2P0L138OQk0AiyOKsi7e9QlmjvitxQkpGC1WKIVe2+sSkpaRLEVU+5b8BEzCCiYs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708731246; c=relaxed/simple; bh=Bdv7ic5XRrwtS4nmoj4Gq5WpG+vEQxsUv37S/uXy1gQ=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=cFBEpe03PcEM05gb2yZoi0d+eUg84ko74QQqWZklZCmWN7bfOLotA4ze7V5XaV3wJx9eMHqH1/4IiJyzk5hwNn3ZgR02NXX06ODoHka1JCZNKNx4k7FG016I393HlaQufrDB6U0CJMIS+jhUOGWFMTHrgPo0j5DHfET6K8TdeTk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=sHN0ovEn; arc=none smtp.client-ip=209.85.219.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="sHN0ovEn" Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-dcc05887ee9so2186778276.1 for ; Fri, 23 Feb 2024 15:34:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1708731244; x=1709336044; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=iF5pWFdcom/O/sPsVNJm3DzfcOMbthIOUDSkLmNhSaQ=; b=sHN0ovEnqYodVM42QPUnsWfxPUL8uEJcicQMzqMORzZiIAuKjUcE1l+3j3QINH//zq 5PjohQ73V3EQwFiv7FGzyZL3TLTN+FqdiybAN4ajVn2q6UqmrbVmGXLMl5+fSvYMjs97 UZoHqBx2olWKkebNdm3lqzgedUise/dR/LoXIit7talOUci46/dYng2sq8FTAAlxeOi2 JUug3OyHYXwsMjquamcHl5WXZ9H+wQq2hnJ16OXKyEYwvKBOCnVXyuK73u6kMM7tortf GdiUGCJK7T10ocCE/OFL8HIjI1qzIreUYiZ/EmIIWBQkBIQuJ5pRSWlRs0MT9xGC8KGp 5pZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708731244; x=1709336044; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=iF5pWFdcom/O/sPsVNJm3DzfcOMbthIOUDSkLmNhSaQ=; b=PPuwFYky9GGXo9mIxdvJJhYFKuvqW+h8ORlcOM2mqG7K8T7yp1KjA8E6HRqxWwiKSz W4U7x/GL54COkzAYAhz0dyeSFRq0IMqPPoo2SD25ijULTeoCthI9LnD3FFPP1uFZs9Ep 20BHLetz39OQpvwt1danIpnHjjZYz+dHmNBv2bI2lINngtpPI/ug8dJtCBqy5CI9bIOA e7ZidvlKa8sm0AfU4qultJ2Vv4N+Z+INJAeaH/OX4FSlic895EH+5ETOgVsknYTHGaro Gx53q9TPQGTsfGOQ8S4e90mvzOZ0DZZDgNPczmwyyH/Y9FDSEisOqrZhOj7BXFVLp9qd soAQ== X-Gm-Message-State: AOJu0YyQ1Bg7BCKy51FwZ2wgPGU2za363CSzVtMRqLh0LvoLJV6pOwYF FvCu1SVW8ND1Mc54GIFjKmqGyXkhDwBRbOX6cOiiJmSu62u28oZp49rZocyMYs2EfyqAXIG5h4J wloSu25d4FFCV1GPbgSN1kNIQz+8KOL67TjHekKMsrTrhniedzFarBpGr4mXdMkFw/U0y+B0A9h LB7SSapLbGrViV0JIfm6souIws40dQom8r8+QI+oo= X-Google-Smtp-Source: AGHT+IF2+aTphCPNUyfVhEGnrmgh9SAjK0nVsRakX8Wr1TgAJXjs4w7UkEVAHSN4qx6abpzpJywXsC9MeJ+wjg== X-Received: from lunarfall.svl.corp.google.com ([2620:15c:2d3:204:2fd9:97d1:a490:26b8]) (user=steadmon job=sendgmr) by 2002:a25:8a11:0:b0:dc6:b813:5813 with SMTP id g17-20020a258a11000000b00dc6b8135813mr39443ybl.9.1708731243802; Fri, 23 Feb 2024 15:34:03 -0800 (PST) Date: Fri, 23 Feb 2024 15:33:52 -0800 In-Reply-To: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.44.0.rc0.258.g7320e95886-goog Message-ID: <4a16a3ec241d83b3de424ce835db24eee6e46f0f.1708728717.git.steadmon@google.com> Subject: [PATCH v3 3/7] test-tool run-command testsuite: remove hardcoded filter From: Josh Steadmon To: git@vger.kernel.org Cc: johannes.schindelin@gmx.de, peff@peff.net, phillip.wood@dunelm.org.uk, gitster@pobox.com `test-tool run-command testsuite` currently assumes that it will only be running the shell test suite, and therefore filters out anything that does not match a hardcoded pattern of "t[0-9][0-9][0-9][0-9]-*.sh". Later in this series, we'll adapt `test-tool run-command testsuite` to also support unit tests, which do not follow the same naming conventions as the shell tests, so this hardcoded pattern is inconvenient. Since `testsuite` also allows specifying patterns on the command-line, let's just remove this pattern. As noted in [1], there are no longer any uses of `testsuite` in our codebase, it should be OK to break backwards compatibility in this case. We also add a new filter to avoid trying to execute "." and "..", so that users who wish to execute every test in a directory can do so without specifying a pattern. [1] https://lore.kernel.org/git/850ea42c-f103-68d5-896b-9120e2628686@gmx.de/ Signed-off-by: Josh Steadmon --- t/helper/test-run-command.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/t/helper/test-run-command.c b/t/helper/test-run-command.c index a41a54d9cb..e6bd792274 100644 --- a/t/helper/test-run-command.c +++ b/t/helper/test-run-command.c @@ -175,9 +175,7 @@ static int testsuite(int argc, const char **argv) while ((d = readdir(dir))) { const char *p = d->d_name; - if (*p != 't' || !isdigit(p[1]) || !isdigit(p[2]) || - !isdigit(p[3]) || !isdigit(p[4]) || p[5] != '-' || - !ends_with(p, ".sh")) + if (!strcmp(p, ".") || !strcmp(p, "..")) continue; /* No pattern: match all */ From patchwork Fri Feb 23 23:33:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Steadmon X-Patchwork-Id: 13570140 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2B53C14E2E3 for ; Fri, 23 Feb 2024 23:34:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708731248; cv=none; b=cwMp9cK+kVudnFDSCPVRWJtdNvjqFa4tRdKjz0dI9F4pDmBX548ZreCvCuMkt9Vn+pkgUlXJB3PiAZPKdXuGY/k/cmfXJBui/bT/Ycp0+8VbDn2DEJt8ucHMFXAvSfKmH6PIhDhJbul4fH0v6JJ/r3Zbk+q+iZhl5J3zaHQco/g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708731248; c=relaxed/simple; bh=8+Ur2wDE3s5LH4SggzLj3O7YuVC/e+cLYOdrRxMnCS0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=FH6ij7vi6FCP7UDanbJEegQ4+6pJB6P5r4fMtQBkiANgIkFUsIcbLwlhUJH47qNz24yt3opSG3GUzp+H+SQPl5eo3lLbQN2DP6NkrjVMnE8Ci2aya2mWCg6Pb5XKIHdJeyms7jG48b9DOU9oJIk/dDY3gfsn8f6RneoUrNn7BoA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=yY18nXQJ; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="yY18nXQJ" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-608ad239f8fso25246467b3.0 for ; Fri, 23 Feb 2024 15:34:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1708731246; x=1709336046; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=YS5IgGzKz2ph9H6jWzxteSZ9Z//flJwLlak8BMLty8M=; b=yY18nXQJGOdU6IQdy0xEq8nQ30t5vC+fH8XZ1W2LYvOYY/A+xDnCoGGcRQEeUoN06a kylDR//vPk2FJg6YopxOWVq27CXcmUZF3Wnn7fWXyXW7aQKz8VYaWU16zYzPA9qpMqIE P7mC54DcrcyDq9FEs9axpVqnZpIHleI/fKqYcBPgv1dvMPydQxeTdTaBkm279RxxQbZd 8xRuKeL4fyGkIOyIzadDH6tn4SQCcL+j3Q/0Ic8Cli026D2yB26j1S3FpitvkpQ6XNin 6p6Lbs49sp3CYW4CS3yKEYGPvCZLl1FdB1g50tP3+1BGjFZuPJgES1QMou/GmNRCHScs l/Iw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708731246; x=1709336046; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=YS5IgGzKz2ph9H6jWzxteSZ9Z//flJwLlak8BMLty8M=; b=f21GmDxe4xpcWFPeibauQ+j5V2IyPppfDmiEm4y76Pt3viogD6oK4I1wXLXEWstjRa aoX4K16a4NagoHJ1ZOQgSjkYQsZJWdRk6t/QP/4DkvQX5QKf1FaLrXPg5P/sLD2gbuLi wrieMpqJPdVa7vyoNPSVVtjTHNwqGzazCoB7hQQ+dH/Ot1RU8jbNszd8L1fHHbzHIEyv yy0zEhXJUYlcjAPrHNAsPhWrXJb8LK/E+RoGhGd3/9MxD1HsTuP7n/UeiUmuQyPO5dm3 FuKx4rvRFSUaM3B0LB07b+BLP2H4fski62CGsvD9ow4A7qqK5oIUsKjXDeCkURQTI45R F4sw== X-Gm-Message-State: AOJu0YyedgyLkk3vrcjkzuUH3zElCas7cVP/50BS5Nccg6l4IIBf4vWd eizAGSNNoKpiIg/Tioq5IgF4b17DdweTBun5DUm2Cp6dEWhPZriyPn5J3KP/f1d7xY4HSCVi9qN uQiYTOQ8WtjmiePoBYvBcEOI9yI/R6/JS7L5P0VuQh8+uuosmWH2F5IeeNWk2ir6pg+77nC1wns zR1xmKLZrYRF+F6GNEyYZ63F5mdptGZkWpMr9kEPw= X-Google-Smtp-Source: AGHT+IEUiyVpL+OhBTINpiAon7M6NqiJNoLbypTBRHcswxDw8Cs3nSBWfU35mjymZIDmDXnor/01U4U3gGqDlg== X-Received: from lunarfall.svl.corp.google.com ([2620:15c:2d3:204:2fd9:97d1:a490:26b8]) (user=steadmon job=sendgmr) by 2002:a0d:d44d:0:b0:607:d016:d85c with SMTP id w74-20020a0dd44d000000b00607d016d85cmr317901ywd.9.1708731245886; Fri, 23 Feb 2024 15:34:05 -0800 (PST) Date: Fri, 23 Feb 2024 15:33:53 -0800 In-Reply-To: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.44.0.rc0.258.g7320e95886-goog Message-ID: Subject: [PATCH v3 4/7] test-tool run-command testsuite: support unit tests From: Josh Steadmon To: git@vger.kernel.org Cc: johannes.schindelin@gmx.de, peff@peff.net, phillip.wood@dunelm.org.uk, gitster@pobox.com Teach the testsuite runner in `test-tool run-command testsuite` how to run unit tests: if TEST_SHELL_PATH is not set, run the programs directly from CWD, rather than defaulting to "sh" as an interpreter. With this change, you can now use test-tool to run the unit tests: $ make $ cd t/unit-tests/bin $ ../../helper/test-tool run-command testsuite This should be helpful on Windows to allow running tests without requiring Perl (for `prove`), as discussed in [1] and [2]. This again breaks backwards compatibility, as it is now required to set TEST_SHELL_PATH properly for executing shell scripts, but again, as noted in [2], there are no longer any such invocations in our codebase. [1] https://lore.kernel.org/git/nycvar.QRO.7.76.6.2109091323150.59@tvgsbejvaqbjf.bet/ [2] https://lore.kernel.org/git/850ea42c-f103-68d5-896b-9120e2628686@gmx.de/ Signed-off-by: Josh Steadmon --- t/helper/test-run-command.c | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/t/helper/test-run-command.c b/t/helper/test-run-command.c index e6bd792274..61eb1175fe 100644 --- a/t/helper/test-run-command.c +++ b/t/helper/test-run-command.c @@ -158,6 +158,8 @@ static int testsuite(int argc, const char **argv) .task_finished = test_finished, .data = &suite, }; + struct strbuf progpath = STRBUF_INIT; + size_t path_prefix_len; argc = parse_options(argc, argv, NULL, options, testsuite_usage, PARSE_OPT_STOP_AT_NON_OPTION); @@ -165,9 +167,13 @@ static int testsuite(int argc, const char **argv) if (max_jobs <= 0) max_jobs = online_cpus(); + /* + * If we run without a shell, execute the programs directly from CWD. + */ suite.shell_path = getenv("TEST_SHELL_PATH"); if (!suite.shell_path) - suite.shell_path = "sh"; + strbuf_addstr(&progpath, "./"); + path_prefix_len = progpath.len; dir = opendir("."); if (!dir) @@ -180,13 +186,17 @@ static int testsuite(int argc, const char **argv) /* No pattern: match all */ if (!argc) { - string_list_append(&suite.tests, p); + strbuf_setlen(&progpath, path_prefix_len); + strbuf_addstr(&progpath, p); + string_list_append(&suite.tests, progpath.buf); continue; } for (i = 0; i < argc; i++) if (!wildmatch(argv[i], p, 0)) { - string_list_append(&suite.tests, p); + strbuf_setlen(&progpath, path_prefix_len); + strbuf_addstr(&progpath, p); + string_list_append(&suite.tests, progpath.buf); break; } } @@ -213,6 +223,7 @@ static int testsuite(int argc, const char **argv) string_list_clear(&suite.tests, 0); string_list_clear(&suite.failed, 0); + strbuf_release(&progpath); return ret; } From patchwork Fri Feb 23 23:33:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Steadmon X-Patchwork-Id: 13570141 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2A9C114EFC5 for ; Fri, 23 Feb 2024 23:34:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708731250; cv=none; b=FbxvsbRiM1FM++056GlJHDkl5LjvTt4BmMR10Lzgs2hjd9AQPq01lpoFVJmLwKnWFdurg8pixRDUXSPWOpx8zl5L5NnkO3R29RSBy3qAyc5zOp5sKdQx7JVaoeY5UkLjtJkt1/zp46IyGgbetFbvzyIX/2iF5l6EIouidzs+W8k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708731250; c=relaxed/simple; bh=SiQqByYeEQzhpPBR3mV88GDAMy83ayejYJCW2Jy+nWk=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=oi8VbcrDOKgkNMT37LkOxIS8/G75mTGUUN3J9Yb3tJBOQ/NF5rOsWeJex6IqbyDNJk/jUQtAzLYnduRLIM+JlKlsehg6wfDMKAg/mkkt9NlxgcGP/KgFnxp1QV7nYsdJJ4k0EaguYshnMMDNAsZq2gqUo9rz1DxZy0sxFMPqxr0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=FI3ZcS5Q; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="FI3ZcS5Q" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-608575317f8so22543247b3.3 for ; Fri, 23 Feb 2024 15:34:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1708731248; x=1709336048; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=tsP8u0s+63+2WSgJOtZp/ggjt+17TUlD4Z0p+56oLUc=; b=FI3ZcS5QtQw25HgHnCwGJHZdRnEvJv7PDZ0vG9hGTCNk7ILXnDTlTAFtnPvUC2sGQn kgP1jyQV5hwSSsvQ6UFl2gOlj5xxXMgF8iQcZngqeaQnxqp6EMqBK+7JTEoyenDA+HQy 3gSuP+Yg/JSS89Pt/pr2mlPcv6Gwf651sbsngQDqe62KwymdULmpQhBr/VJFVaxrlr1Y EDh5jCTnR469/+nQRKU5O4ocA1m9y5VJ0I0pUIUzombKZFLcXb4huczhHSy9FaleFnZb cnNet9KS0Op70Gu2pOWb7ezLnMLB7vcPzT/WJE6p6XyATavGhUnCMQHvb5pqFCaTVpmn pdQQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708731248; x=1709336048; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=tsP8u0s+63+2WSgJOtZp/ggjt+17TUlD4Z0p+56oLUc=; b=cVNTJnTfW7aj08h3ecENUqI3mSz/D0zQWjVN1gnivRStG8aqKq+kh5SKxWuVV3CeF8 fLxmWPJAv83L3QFPu0QlFTwoFuOHVCJ7//WmSHFpUwQRK/xWNrrTIlVdJ+CYLSo6ZPLy B/7OK6Oc4wmMkuIACzj5httDU7YBgQE8QCo3AERrda/C8kqWrjTyCdbCVyD9Mkatn16A thPMYCQIdpN9TgyzpVJR+7vJGPoQhxuOnMOGKbIb9ooe7ciLO6K6/+990nlVgE4wMETe c3AzD4MuEKuPR3NGDBHdFldM/FWEizlzggLUHJYDtXX8ylhMGPIZxoJzMg+8ORY8xGfw m9bQ== X-Gm-Message-State: AOJu0YzmRJ+1OuDtaBf/Y9mBMmkJxpNvAfpRHbqXp4337s75gyNAqx/O 3JW7bXV1RnD7XokswjB7WBRwGshnP4oqy1v2l/2Imh55Yk0Tn6LCLGq+6oIVEUGQHR9BIxFS4US ftQHtf/NeaXuM8Ql+UUhPryPzVfceiasxd7GB2v0TB1XocUQ3PI2T01IOFMZcyMZwYc8bYvG8Nt lM39R7KAW5K1y4QEuFAdrNkRyP26H488el3+r3XO8= X-Google-Smtp-Source: AGHT+IH6iJ5iAGRdXE2nXSG7GKky0UUGrZLkrn8wSmSQmTJMnIFyJDnTDrx0RJImB54ijtcOHlwwAyjV/s/CDg== X-Received: from lunarfall.svl.corp.google.com ([2620:15c:2d3:204:2fd9:97d1:a490:26b8]) (user=steadmon job=sendgmr) by 2002:a0d:cb53:0:b0:608:7d49:85de with SMTP id n80-20020a0dcb53000000b006087d4985demr254796ywd.9.1708731248076; Fri, 23 Feb 2024 15:34:08 -0800 (PST) Date: Fri, 23 Feb 2024 15:33:54 -0800 In-Reply-To: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.44.0.rc0.258.g7320e95886-goog Message-ID: Subject: [PATCH v3 5/7] unit tests: add rule for running with test-tool From: Josh Steadmon To: git@vger.kernel.org Cc: johannes.schindelin@gmx.de, peff@peff.net, phillip.wood@dunelm.org.uk, gitster@pobox.com In the previous commit, we added support in test-tool for running collections of unit tests. Now, add rules in t/Makefile for running in this way. This new rule can be executed from the top-level Makefile via `make DEFAULT_UNIT_TEST_TARGET=unit-tests-test-tool unit-tests`, or by setting DEFAULT_UNIT_TEST_TARGET in config.mak. Signed-off-by: Josh Steadmon --- Makefile | 2 +- t/Makefile | 10 +++++++++- 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/Makefile b/Makefile index ba55d817ee..b0d1f04b4d 100644 --- a/Makefile +++ b/Makefile @@ -3870,5 +3870,5 @@ $(UNIT_TEST_PROGS): $(UNIT_TEST_BIN)/%$X: $(UNIT_TEST_DIR)/%.o $(UNIT_TEST_DIR)/ .PHONY: build-unit-tests unit-tests build-unit-tests: $(UNIT_TEST_PROGS) -unit-tests: $(UNIT_TEST_PROGS) +unit-tests: $(UNIT_TEST_PROGS) t/helper/test-tool$X $(MAKE) -C t/ unit-tests diff --git a/t/Makefile b/t/Makefile index 4861edafe6..0ae04f1e42 100644 --- a/t/Makefile +++ b/t/Makefile @@ -49,6 +49,7 @@ CHAINLINT = '$(PERL_PATH_SQ)' chainlint.pl UNIT_TEST_SOURCES = $(wildcard unit-tests/t-*.c) UNIT_TEST_PROGRAMS = $(patsubst unit-tests/%.c,unit-tests/bin/%$(X),$(UNIT_TEST_SOURCES)) UNIT_TESTS = $(sort $(UNIT_TEST_PROGRAMS)) +UNIT_TESTS_NO_DIR = $(notdir $(UNIT_TESTS)) # `test-chainlint` (which is a dependency of `test-lint`, `test` and `prove`) # checks all tests in all scripts via a single invocation, so tell individual @@ -76,7 +77,7 @@ $(T): $(UNIT_TESTS): @echo "*** $@ ***"; $@ -.PHONY: unit-tests unit-tests-raw unit-tests-prove +.PHONY: unit-tests unit-tests-raw unit-tests-prove unit-tests-test-tool unit-tests: $(DEFAULT_UNIT_TEST_TARGET) unit-tests-raw: $(UNIT_TESTS) @@ -84,6 +85,13 @@ unit-tests-raw: $(UNIT_TESTS) unit-tests-prove: @echo "*** prove - unit tests ***"; $(PROVE) $(GIT_PROVE_OPTS) $(UNIT_TESTS) +unit-tests-test-tool: + @echo "*** test-tool - unit tests **" + ( \ + cd unit-tests/bin && \ + ../../helper/test-tool$X run-command testsuite $(UNIT_TESTS_NO_DIR)\ + ) + pre-clean: $(RM) -r '$(TEST_RESULTS_DIRECTORY_SQ)' From patchwork Fri Feb 23 23:33:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Steadmon X-Patchwork-Id: 13570142 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D973114E2E3 for ; Fri, 23 Feb 2024 23:34:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708731252; cv=none; b=j0jKohC2OBLj7BBHxDrSOi+KB9ndrMw8L91tAGAdlEVPKTXStBgVPgr3swFOGCd3uC61flNC4qALKuGS1/JfN2gyX4p2oNWlULwosmj6WeVimbqJMaOoJzOPRi5BCoGmqsHLLIljmPmiTzcZ3vTCPyu2fLwVrYw6jH+imFzUfto= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708731252; c=relaxed/simple; bh=tiJVQbgbTgTXpBcwOAw7AokirBNUNgOM0EsaC6DQ9So=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=TdiY+kDvq8R75fp2l65UenvbpZcEyLLs1zBQWsyrPS4uFbSLS7j5JhbItI1CwSNfXkex1XAp7paDEk7xjh/SS9tHLuMv+P0HF5iCbaT+fYy36SuVYH3pEcPFuqPniJejlDlFRLQe3cyAGq7tilLo3DNji7sx0uLbawJDYwjKjy0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=OOX504jB; arc=none smtp.client-ip=209.85.219.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="OOX504jB" Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-dcc0bcf9256so2020352276.3 for ; Fri, 23 Feb 2024 15:34:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1708731250; x=1709336050; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=V7nNYjRr2WPTi1Fwl0q7+3DWsrU9EN5PibjP4kQhviU=; b=OOX504jBkn7SVEW9wSe7jmlULk5YoHl2V9X0zzwePl5tRNi1dASRLBvsl1atmkF6MZ 9duxW4EOqWot+KKkzn8914QE2WgX31ylVTYgzRiCBOsH7xRgbBwhqZquOZT6hbW/1RMN hGBNGcqhiQnxErDWCi9cBsUeCmyxfu2Nhh+Pnezic4K9DO/XlORldiGwKLzKjtJg+X5L 3nRdYpPE2C1en0TdwfrHm361eA0pvMsgH+51MDu5qK7BQ/MOuuqFddv12z5uui5ko9U4 SywelN4sXZr3P6lbamWddpnTtLm+z8/XSrP/LpLMD77R7t/clRzA4Qu2M7emBUcBzxYr CdCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708731250; x=1709336050; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=V7nNYjRr2WPTi1Fwl0q7+3DWsrU9EN5PibjP4kQhviU=; b=foXXV01kqp8Hft3ZLHdF5qgN9lXXOHrWia0Eghsl2sEG1w7nWz9H32CPC8ORrVYHbu eGgtkPIKFMcI8Z20MvoobcN6IIClL5tO7jDePMWBWvIa7KPFiKzuI9utLnzpQU4xpMXB VHGCr/4iSCVGHyScXl9WHTRx/N8A+59IwH05gpPRZskGXLjVV93GbHbG8HvxWZrz+B1s Ss28M0RuW5hCQjlvH3s/kWKzJ4EjNxM5WzNQrSUfBCmUIe5kAIDOeNx1ypiVSasBHYFL aQFU5tSEYvTxUgoriCy0sicWfgNiM8oZWPuDB9se8456Vm6Z20sQj2Wr1+Av+hapvZrZ jjVg== X-Gm-Message-State: AOJu0YztayxUsdGkks4vnVDbs+gz41ZLdSf3Omu4uObhtOKfg3LOu8uy kPd5jPQYF/heKfyqenNR9ImlaUxjz19pfd714n43ROXywDtMRbjqaKUOBgyrBvHbDL5G270FDfC bkaZx9RU/AjWh3YtkQtf+CGkqOkFPuFs1qhuRe0Ejp7ZDDocAuhllCYqB3Ndf3D8hxpUQI3BLeM SnpzE6wwwb9pJX8/zebO+nqFjnGpHjXmp+9AFMMN8= X-Google-Smtp-Source: AGHT+IEVfGUt8FdVbB3/t5NAP4sedc61MWnlVzpwOMfXDfnOAgdwXvmI6cpBEbxrI0OG4xRaqaDfd2fieRtXEg== X-Received: from lunarfall.svl.corp.google.com ([2620:15c:2d3:204:2fd9:97d1:a490:26b8]) (user=steadmon job=sendgmr) by 2002:a25:ac67:0:b0:dcc:53c6:1133 with SMTP id r39-20020a25ac67000000b00dcc53c61133mr39590ybd.13.1708731249814; Fri, 23 Feb 2024 15:34:09 -0800 (PST) Date: Fri, 23 Feb 2024 15:33:55 -0800 In-Reply-To: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.44.0.rc0.258.g7320e95886-goog Message-ID: Subject: [PATCH v3 6/7] t/Makefile: run unit tests alongside shell tests From: Josh Steadmon To: git@vger.kernel.org Cc: johannes.schindelin@gmx.de, peff@peff.net, phillip.wood@dunelm.org.uk, gitster@pobox.com From: Jeff King Add a wrapper script to allow `prove` to run both shell tests and unit tests from a single invocation. This avoids issues around running prove twice in CI, as discussed in [1]. Additionally, this moves the unit tests into the main dev workflow, so that errors can be spotted more quickly. Accordingly, we remove the separate unit tests step for Linux CI. (We leave the Windows CI unit-test step as-is, because the sharding scheme there involves selecting specific test files rather than running `make test`.) [1] https://lore.kernel.org/git/pull.1613.git.1699894837844.gitgitgadget@gmail.com/ Signed-off-by: Jeff King Signed-off-by: Josh Steadmon --- ci/run-build-and-tests.sh | 2 -- t/Makefile | 2 +- t/run-test.sh | 13 +++++++++++++ 3 files changed, 14 insertions(+), 3 deletions(-) create mode 100755 t/run-test.sh diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh index 7a1466b868..2528f25e31 100755 --- a/ci/run-build-and-tests.sh +++ b/ci/run-build-and-tests.sh @@ -50,8 +50,6 @@ if test -n "$run_tests" then group "Run tests" make test || handle_failed_tests - group "Run unit tests" \ - make DEFAULT_UNIT_TEST_TARGET=unit-tests-prove unit-tests fi check_unignored_build_artifacts diff --git a/t/Makefile b/t/Makefile index 0ae04f1e42..0c739754d8 100644 --- a/t/Makefile +++ b/t/Makefile @@ -68,7 +68,7 @@ failed: test -z "$$failed" || $(MAKE) $$failed prove: pre-clean check-chainlint $(TEST_LINT) - @echo "*** prove ***"; $(CHAINLINTSUPPRESS) $(PROVE) --exec '$(TEST_SHELL_PATH_SQ)' $(GIT_PROVE_OPTS) $(T) :: $(GIT_TEST_OPTS) + @echo "*** prove (shell & unit tests) ***"; $(CHAINLINTSUPPRESS) $(PROVE) --exec ./run-test.sh $(GIT_PROVE_OPTS) $(T) $(UNIT_TESTS) :: $(GIT_TEST_OPTS) $(MAKE) clean-except-prove-cache $(T): diff --git a/t/run-test.sh b/t/run-test.sh new file mode 100755 index 0000000000..c29fef48dc --- /dev/null +++ b/t/run-test.sh @@ -0,0 +1,13 @@ +#!/bin/sh + +# A simple wrapper to run shell tests via TEST_SHELL_PATH, +# or exec unit tests directly. + +case "$1" in +*.sh) + exec ${TEST_SHELL_PATH:-/bin/sh} "$@" + ;; +*) + exec "$@" + ;; +esac From patchwork Fri Feb 23 23:33:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Steadmon X-Patchwork-Id: 13570143 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7736E14EFC5 for ; Fri, 23 Feb 2024 23:34:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708731253; cv=none; b=Pc8DP1AyFeKDSZLPbqdagDpCNMI/xzigOGCSu2pGi13ucnr/QgZX3nskJMnwr6M3PM3FFVn/o7bmt6OB3poOqNX0YWsurJWlPRBWkdqxSRwhn2tC/yW66ytGBAYUwNZfDRFe1LKHxX0di+oxFNya9Fp3Q+bAzCYy5ruCDhGTNXo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708731253; c=relaxed/simple; bh=f1ffTXZP5UdQksMtO3LZRF/EeihQuS9aNnziH+nzVF4=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=NHi76q5hK71BDwi/wjup6hF1BGXSXUFxs/3GkU5uxlQSaHFTVS/rz236bznIr3YMT7IBoa4VnN0ogAHXXN6Nq6Ik1zp8PPdMHAp4MYQ4b0+ooBBHEopSrZ74VJbbgGLai0tZC/7kRjiMzPqEUxxyPvTZoVCvYlrnAII02hBFw1c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=D4KfJ2LL; arc=none smtp.client-ip=209.85.219.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="D4KfJ2LL" Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-dcc4563611cso2379217276.3 for ; Fri, 23 Feb 2024 15:34:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1708731251; x=1709336051; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=K30ED04T/ZQAwsWwVKBues736gTQaJeYTdmH0g3c1rw=; b=D4KfJ2LL20YALk4K8sc69za+6mDB3Fuis/36tQ71cNmUbw8yQDiSjRc77YRcStJrtL IbmJZmwr5pTucbJl6LXHdmJo7gzPETycgZ/95hw7GN8iWDiBq1w2tihxObt8C4yrmMD4 CyV8fPVPbTXiDpWpTz8++91ZoWlKuA6VFvpapHtqLiE8wb0FeFW9hUymBZMxSwVlHPMp cyy8Gqg1Da3OxmS4irGBwaQq7LB0wIpV9zMblR7tHB63m9PcOtNFR5vwoqXQK5YMhidP yaQgIC1ksnxE/Dd3/28DgPBoJuIR3MPVDZGPJJcRpk8RtpvzeD9hgN0EzVIKNv1/cFYA STVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708731251; x=1709336051; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=K30ED04T/ZQAwsWwVKBues736gTQaJeYTdmH0g3c1rw=; b=ObxkeK9Qk2WqhE/PGTixU3kRfB5sE9yEBawDdFjElWE3N5sykKgFjDHV9p5K26Mz3p DoEDxN0ZhwDya+3YBJQkLB0FUwVpezr8s631WSM/ByqQjIE9NTWk89dLW+cxoCIMibCt AWBK/HiecROFdr9+GgrpsbsFL5wDCbsvepFCiR6jTVKiiCgghEXqyrqeXxzea3qo3Ekz x/qfGTC0u867jsoinIzY2n4gbBcTOGLdBXrXqElKpJeCFEE09hBBZ/bRTr7Tsu3MvDD8 tlw+p6SONe+uV304vKQi+He7ng3MiQUq/ZYMra05ml9Nwl7gIp7PvBD2jpQU+tjjaNhd CKHw== X-Gm-Message-State: AOJu0Yyr5dDbqwrKZHOmgYgi2KBu5AJe0JtDzohulK6FlV6uYasK6P4W Oey+jodDRo3lITa4fjh/hpyVbWtryVI8/QOwvMT2JUwSgsFywycRHtGgvIULzry1lPZv9vFRWZv w7E5rkmdeZ/8uO1T/KiZxKrcAcBoPQheE80EIPLdNrd7IPB6Anuepi94Gpk4UGKdYcuUtHjUwxs EE6L+gK5FMF++pr+372eIyLehvz9lX+stsD5Q+tb8= X-Google-Smtp-Source: AGHT+IEMWaKMclekxZmjG6sY688DdPT5gTHUF2djqJdT7XXa9xf3ySNkwX+18ZS3wMRAmFO9NlvJ70+vsiU3VQ== X-Received: from lunarfall.svl.corp.google.com ([2620:15c:2d3:204:2fd9:97d1:a490:26b8]) (user=steadmon job=sendgmr) by 2002:a05:6902:1009:b0:dc7:5aad:8965 with SMTP id w9-20020a056902100900b00dc75aad8965mr346693ybt.0.1708731251439; Fri, 23 Feb 2024 15:34:11 -0800 (PST) Date: Fri, 23 Feb 2024 15:33:56 -0800 In-Reply-To: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.44.0.rc0.258.g7320e95886-goog Message-ID: Subject: [PATCH v3 7/7] ci: use test-tool as unit test runner on Windows From: Josh Steadmon To: git@vger.kernel.org Cc: johannes.schindelin@gmx.de, peff@peff.net, phillip.wood@dunelm.org.uk, gitster@pobox.com Although the previous commit changed t/Makefile to run unit tests alongside shell tests, the Windows CI still needs a separate unit-tests step due to how the test sharding works. We want to avoid using `prove` as a test running on Windows due to performance issues [1], so use the new test-tool runner instead. [1] https://lore.kernel.org/git/850ea42c-f103-68d5-896b-9120e2628686@gmx.de/ Signed-off-by: Josh Steadmon --- ci/run-test-slice.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ci/run-test-slice.sh b/ci/run-test-slice.sh index ae8094382f..e167e646f7 100755 --- a/ci/run-test-slice.sh +++ b/ci/run-test-slice.sh @@ -17,7 +17,7 @@ handle_failed_tests # We only have one unit test at the moment, so run it in the first slice if [ "$1" == "0" ] ; then - group "Run unit tests" make --quiet -C t unit-tests-prove + group "Run unit tests" make --quiet -C t unit-tests-test-tool fi check_unignored_build_artifacts