From patchwork Sat Feb 3 00:50:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Steadmon X-Patchwork-Id: 13543654 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4DF7E80B for ; Sat, 3 Feb 2024 00:50:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706921439; cv=none; b=Lt2+QWQ00IdIbHpKdNhJM51PUiR3V+2kb9f39i8hE6tEDwdZjMnSzbcOpPLPPe6FX+OdpgK/+/buMh1CKpV//r2GieOyS6nAxCCUW1U2womTgoidOSQDT2jzvvhoi/UHlI5G7nZ6qmQIQN0TYZLw5kRXJe3w+xrZ2hF0hGlzwfI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706921439; c=relaxed/simple; bh=PXLH0pY3oA0Yi6rs6XgHa8cM5sjwAoNNTh4VytQrCNs=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=eS8s1mVn/wxKK45iASJ0C40leNixqilbZB/DETtFs4PcLE+fpdw7uWKLUMlVur9+3zXV65lXIN1UnYt80GsXduX2kAr64+K6QNuvz0DQTuTmMDokgIKtQwcah0udPfAF3qhqwG3f+AUABVRBm4I8PreTUyREg99ywZhXzNvdmYA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Gi1MeWP+; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Gi1MeWP+" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-60420dcb127so17001647b3.1 for ; Fri, 02 Feb 2024 16:50:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1706921436; x=1707526236; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=OB9yBM/Qv2VlOAI02pRTqgA5ieBc/fQLzo/91PcTd0E=; b=Gi1MeWP+qnG875wjyWge7yJtRBrH+/OdQKvEy63pb4ooqkYjqaYIDSwKH82joi4te6 VsrHyBFiDxoONJnDG13q4S+9m1vS9kS+cy0jrPGp/230rBAfi3C/oaZg3pkOf8339Z8Y sixWGnhUABKypckrjPxzkDrbkCeGPF1x6qENfc1uoEQR9bXp7HyKn0UGCGvrf5NDwv+W ic0Fh7WwsyfVImCnwpneTM07SQoevlJn0NTKbhaElOhSe+ImjKXJ/qcUHMzHEJpUocEi W1KdsdR7/Gm5z3Co6NwraOfI9x/2swMYNLXTzFiHjEEmkZyb3rXqKvk3LToUVBjbMVdT ystg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706921436; x=1707526236; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=OB9yBM/Qv2VlOAI02pRTqgA5ieBc/fQLzo/91PcTd0E=; b=r6D6RYSM4jOoCzd36P/SYeMAyBEFDgYYNM/FvOQaV3datPTd+dom/km7Jnjgc3+stK 7ZTt7BbRuXM9eJSx1seE2z+vMLGazrM+Kja/5rNzr3stb6FNPXJ4bX2cViyyG9x36VIs 6+1QcH+HMQgrAzgaKfRIyr1zbue7I1v4kG8F/FEoyVOFc2zL7NuAR3S+2ObmwM5VTkcM C44/LE7WMw7EfO47FtSi3R7yHLL8eSlYXNhb7uIH1ZSX+hRVBz0a6aQXTLjbu9j6Rymz 2L6LJgqb/pFLRFQAGi+lmwI6nUXzpQnafem/m5eckZvbiTT3m18DFlUyB+QIs4YvhOYW dPHg== X-Gm-Message-State: AOJu0Yx57c9G0XiwA0rUdprs30UxIV39KRDgZTUE/DEyqDJu1E6Ue1/t av3OfIPkWdSGesEi0gk/qco48FhDYlRjPACMlnfoyWkcnDpuneOcfonowzY6lPOP3PCNhB1OjZw ioGxt+8EDO7WJlyusn/4fBUru3L/V7JXQAndZvXd72JIVbmRArkeXOYxAnCP66H/Ji7xlAE1qaQ vOi1aiUv3sTghdtIdeMUJRg8Di6tw4Er0xCu/l40Y= X-Google-Smtp-Source: AGHT+IFDEc+wulD8wlqlFOtchbycnj8m+B4WH4BaLKrYQnbiPf98cVPVE7nye0AIpHfAl3bi8vhEt1jDJoF+2Q== X-Received: from lunarfall.svl.corp.google.com ([2620:15c:2d3:204:705c:77af:6ef2:8f42]) (user=steadmon job=sendgmr) by 2002:a05:690c:ece:b0:5ff:6e82:ea31 with SMTP id cs14-20020a05690c0ece00b005ff6e82ea31mr44916ywb.3.1706921436138; Fri, 02 Feb 2024 16:50:36 -0800 (PST) Date: Fri, 2 Feb 2024 16:50:26 -0800 In-Reply-To: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.43.0.594.gd9cf4e227d-goog Message-ID: Subject: [RFC PATCH v2 1/6] t0080: turn t-basic unit test into a helper From: Josh Steadmon To: git@vger.kernel.org Cc: johannes.schindelin@gmx.de, peff@peff.net, phillip.wood@dunelm.org.uk, gitster@pobox.com While t/unit-tests/t-basic.c uses the unit-test framework added in e137fe3b29 (unit tests: add TAP unit test framework, 2023-11-09), it is not a true unit test in that it intentionally fails in order to exercise various codepaths in the unit-test framework. Thus, we intentionally exclude it when running unit tests through the various t/Makefile targets. Instead, it is executed by t0080-unit-test-output.sh, which verifies its output follows the TAP format expected for the various pass, skip, or fail cases. As such, it makes more sense for t-basic to be a helper item for t0080-unit-test-output.sh, so let's move it to t/helper/test-example-tap.c and adjust Makefiles as necessary. This has the additional benefit that test harnesses seeking to run all unit tests can find them with a simple glob of "t/unit-tests/bin/t-*", with no exceptions needed. This will be important in a later patch where we add support for running the unit tests via a test-tool subcommand. Signed-off-by: Josh Steadmon --- Makefile | 4 ++-- t/Makefile | 3 +-- .../t-basic.c => helper/test-example-tap.c} | 5 ++-- t/helper/test-tool.c | 1 + t/helper/test-tool.h | 1 + t/t0080-unit-test-output.sh | 24 +++++++++---------- 6 files changed, 20 insertions(+), 18 deletions(-) rename t/{unit-tests/t-basic.c => helper/test-example-tap.c} (95%) diff --git a/Makefile b/Makefile index 23723367b8..ba55d817ee 100644 --- a/Makefile +++ b/Makefile @@ -802,6 +802,7 @@ TEST_BUILTINS_OBJS += test-dump-split-index.o TEST_BUILTINS_OBJS += test-dump-untracked-cache.o TEST_BUILTINS_OBJS += test-env-helper.o TEST_BUILTINS_OBJS += test-example-decorate.o +TEST_BUILTINS_OBJS += test-example-tap.o TEST_BUILTINS_OBJS += test-find-pack.o TEST_BUILTINS_OBJS += test-fsmonitor-client.o TEST_BUILTINS_OBJS += test-genrandom.o @@ -1338,7 +1339,6 @@ THIRD_PARTY_SOURCES += compat/regex/% THIRD_PARTY_SOURCES += sha1collisiondetection/% THIRD_PARTY_SOURCES += sha1dc/% -UNIT_TEST_PROGRAMS += t-basic UNIT_TEST_PROGRAMS += t-mem-pool UNIT_TEST_PROGRAMS += t-strbuf UNIT_TEST_PROGRAMS += t-ctype @@ -3218,7 +3218,7 @@ perf: all .PRECIOUS: $(TEST_OBJS) -t/helper/test-tool$X: $(patsubst %,t/helper/%,$(TEST_BUILTINS_OBJS)) +t/helper/test-tool$X: $(patsubst %,t/helper/%,$(TEST_BUILTINS_OBJS)) $(UNIT_TEST_DIR)/test-lib.o t/helper/test-%$X: t/helper/test-%.o GIT-LDFLAGS $(GITLIBS) $(REFTABLE_TEST_LIB) $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(filter %.a,$^) $(LIBS) diff --git a/t/Makefile b/t/Makefile index 281f4c3534..1283c90c10 100644 --- a/t/Makefile +++ b/t/Makefile @@ -44,8 +44,7 @@ TINTEROP = $(sort $(wildcard interop/i[0-9][0-9][0-9][0-9]-*.sh)) CHAINLINTTESTS = $(sort $(patsubst chainlint/%.test,%,$(wildcard chainlint/*.test))) CHAINLINT = '$(PERL_PATH_SQ)' chainlint.pl UNIT_TEST_SOURCES = $(wildcard unit-tests/t-*.c) -UNIT_TEST_PROGRAMS = $(patsubst unit-tests/%.c,unit-tests/bin/%$(X),$(UNIT_TEST_SOURCES)) -UNIT_TESTS = $(sort $(filter-out unit-tests/bin/t-basic%,$(UNIT_TEST_PROGRAMS))) +UNIT_TESTS = $(patsubst unit-tests/%.c,unit-tests/bin/%$(X),$(UNIT_TEST_SOURCES)) # `test-chainlint` (which is a dependency of `test-lint`, `test` and `prove`) # checks all tests in all scripts via a single invocation, so tell individual diff --git a/t/unit-tests/t-basic.c b/t/helper/test-example-tap.c similarity index 95% rename from t/unit-tests/t-basic.c rename to t/helper/test-example-tap.c index fda1ae59a6..21e4848e78 100644 --- a/t/unit-tests/t-basic.c +++ b/t/helper/test-example-tap.c @@ -1,4 +1,5 @@ -#include "test-lib.h" +#include "t/unit-tests/test-lib.h" +#include "test-tool.h" /* * The purpose of this "unit test" is to verify a few invariants of the unit @@ -69,7 +70,7 @@ static void t_empty(void) ; /* empty */ } -int cmd_main(int argc, const char **argv) +int cmd__example_tap(int argc, const char **argv) { test_res = TEST(check_res = check_int(1, ==, 1), "passing test"); TEST(t_res(1), "passing test and assertion return 1"); diff --git a/t/helper/test-tool.c b/t/helper/test-tool.c index 33b9501c21..bb5c04c9c0 100644 --- a/t/helper/test-tool.c +++ b/t/helper/test-tool.c @@ -29,6 +29,7 @@ static struct test_cmd cmds[] = { { "dump-untracked-cache", cmd__dump_untracked_cache }, { "env-helper", cmd__env_helper }, { "example-decorate", cmd__example_decorate }, + { "example-tap", cmd__example_tap }, { "find-pack", cmd__find_pack }, { "fsmonitor-client", cmd__fsmonitor_client }, { "genrandom", cmd__genrandom }, diff --git a/t/helper/test-tool.h b/t/helper/test-tool.h index b72f07ded9..38001bd1c6 100644 --- a/t/helper/test-tool.h +++ b/t/helper/test-tool.h @@ -23,6 +23,7 @@ int cmd__dump_untracked_cache(int argc, const char **argv); int cmd__dump_reftable(int argc, const char **argv); int cmd__env_helper(int argc, const char **argv); int cmd__example_decorate(int argc, const char **argv); +int cmd__example_tap(int argc, const char **argv); int cmd__find_pack(int argc, const char **argv); int cmd__fsmonitor_client(int argc, const char **argv); int cmd__genrandom(int argc, const char **argv); diff --git a/t/t0080-unit-test-output.sh b/t/t0080-unit-test-output.sh index 961b54b06c..83b1e3b7f5 100755 --- a/t/t0080-unit-test-output.sh +++ b/t/t0080-unit-test-output.sh @@ -8,50 +8,50 @@ test_expect_success 'TAP output from unit tests' ' cat >expect <<-EOF && ok 1 - passing test ok 2 - passing test and assertion return 1 - # check "1 == 2" failed at t/unit-tests/t-basic.c:76 + # check "1 == 2" failed at t/helper/test-example-tap.c:77 # left: 1 # right: 2 not ok 3 - failing test ok 4 - failing test and assertion return 0 not ok 5 - passing TEST_TODO() # TODO ok 6 - passing TEST_TODO() returns 1 - # todo check ${SQ}check(x)${SQ} succeeded at t/unit-tests/t-basic.c:25 + # todo check ${SQ}check(x)${SQ} succeeded at t/helper/test-example-tap.c:26 not ok 7 - failing TEST_TODO() ok 8 - failing TEST_TODO() returns 0 - # check "0" failed at t/unit-tests/t-basic.c:30 + # check "0" failed at t/helper/test-example-tap.c:31 # skipping test - missing prerequisite - # skipping check ${SQ}1${SQ} at t/unit-tests/t-basic.c:32 + # skipping check ${SQ}1${SQ} at t/helper/test-example-tap.c:33 ok 9 - test_skip() # SKIP ok 10 - skipped test returns 1 # skipping test - missing prerequisite ok 11 - test_skip() inside TEST_TODO() # SKIP ok 12 - test_skip() inside TEST_TODO() returns 1 - # check "0" failed at t/unit-tests/t-basic.c:48 + # check "0" failed at t/helper/test-example-tap.c:49 not ok 13 - TEST_TODO() after failing check ok 14 - TEST_TODO() after failing check returns 0 - # check "0" failed at t/unit-tests/t-basic.c:56 + # check "0" failed at t/helper/test-example-tap.c:57 not ok 15 - failing check after TEST_TODO() ok 16 - failing check after TEST_TODO() returns 0 - # check "!strcmp("\thello\\\\", "there\"\n")" failed at t/unit-tests/t-basic.c:61 + # check "!strcmp("\thello\\\\", "there\"\n")" failed at t/helper/test-example-tap.c:62 # left: "\011hello\\\\" # right: "there\"\012" - # check "!strcmp("NULL", NULL)" failed at t/unit-tests/t-basic.c:62 + # check "!strcmp("NULL", NULL)" failed at t/helper/test-example-tap.c:63 # left: "NULL" # right: NULL - # check "${SQ}a${SQ} == ${SQ}\n${SQ}" failed at t/unit-tests/t-basic.c:63 + # check "${SQ}a${SQ} == ${SQ}\n${SQ}" failed at t/helper/test-example-tap.c:64 # left: ${SQ}a${SQ} # right: ${SQ}\012${SQ} - # check "${SQ}\\\\${SQ} == ${SQ}\\${SQ}${SQ}" failed at t/unit-tests/t-basic.c:64 + # check "${SQ}\\\\${SQ} == ${SQ}\\${SQ}${SQ}" failed at t/helper/test-example-tap.c:65 # left: ${SQ}\\\\${SQ} # right: ${SQ}\\${SQ}${SQ} not ok 17 - messages from failing string and char comparison - # BUG: test has no checks at t/unit-tests/t-basic.c:91 + # BUG: test has no checks at t/helper/test-example-tap.c:92 not ok 18 - test with no checks ok 19 - test with no checks returns 0 1..19 EOF - ! "$GIT_BUILD_DIR"/t/unit-tests/bin/t-basic >actual && + ! test-tool example-tap >actual && test_cmp expect actual ' From patchwork Sat Feb 3 00:50:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Steadmon X-Patchwork-Id: 13543655 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 45BBA1854 for ; Sat, 3 Feb 2024 00:50:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706921440; cv=none; b=eSyRDm4p+dJ/dFlqJwhxmuPgU1DfHBPIm8AQDyf6YGi3zJwtA1zVRqD3oXk5Luzjh8GOrrxRj256t9Efyrf6YleHu4BHBYrmgFGJJIY/NbkIc7SYrePeTl5VgJtW9Vtz+lIl+bHQq5h0ZG2x8EU10i+0d6U+BB9KWpyhsIDJPcg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706921440; c=relaxed/simple; bh=rbf3Q3sNdOZczot6g4kqk9VpWyzg06ZPSbRL9xj5N04=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=TOV2xEZMdsvQfHI9Snn3kr830all0HXgJZ0+dCJwTyDQ1dvYAZfyc9X0fxrR9Q1HhlcOJdAi+T8SP/a1fXAw/bsOZD5GmqZAaD2JvVlostE6hVRhB/vFY+6cpfcypb96NRFUbdj6CpACxPOPPVGPw1D1PWVKxjI0jH28ROGe9QE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=BIHVkfHs; arc=none smtp.client-ip=209.85.219.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="BIHVkfHs" Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-d9a541b720aso4499829276.0 for ; Fri, 02 Feb 2024 16:50:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1706921438; x=1707526238; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=nX+KD/gXJwakeZOwzCXpgz5ZUUqqwLVPBHC9dSmqy6s=; b=BIHVkfHsBl6JKY8J2iOJlECLFj3slbHQWYNS+EGAR7sIBg5A+vKoKzp477FI67NkQ/ 7ISkIE1f8o6JfMqI84LPLPpqJpiB4ZswF32HhrdnsN9B4gObS+J3OYZ66z4DJVTOqKFd XsYwdGgtmE4yfWCZX1CDzPEAdi9tgYQ0KeOyaZ41Js15Ke3UtLPsqh0RW8f02DBGAA94 JK7NxpQSNtcA9G2gkAIY43hRBPtBxBe1zlVLD367MwpsRYDz+nMFbRjQZ6jADMkjtcKV hgOBCIWswRU2G3u0wxhop1GSU1T0KWljHDLO7v+uJpo7uW5zzjuZxJ3bU5KnJPttZnT6 2FYg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706921438; x=1707526238; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=nX+KD/gXJwakeZOwzCXpgz5ZUUqqwLVPBHC9dSmqy6s=; b=JyqsyJ3eJNG9HMhcXdYVOPBpN6sZQXsiI1F46U7cXwgKLS7T8KM/5XgC2Pwk3JBOB3 AgaidMVeWK4nrquW9fC5614PmBLu12DRUDOQsSIbPSLcqzZyBPcl0lMF4iT63458eK9s ZSJUW47XR98saGJ1eDb4OnUCBoneaXcEqzWPRr1Hc7TwKJN4xOia/KbT5shrt5wTC6EU ToXc78zfwUTSgrN+D5f1ugrfVeij7YQpjJBdN/8/7BZgiijjXLZLTYnPdBitCRY7kUDJ BP6Nhw6J3T53oFz8i/gtVXvtk1C4nvZkMLO6sal5mMoq4jyFbrcuW8KaRmq9WAJ1hWoK ajug== X-Gm-Message-State: AOJu0YyRhTY/CDHarFu40BY/1tH9nFjarbY3RNfTfylZzUUn8L0BIlUG XbWK9mj6UL+nsTH6Bsn4plh2wqdgdrxrvyIgW9K3Nw1ZSbTZSFMmKjAM8YeYt7P027+qNzGlDPM ApT6OTKkE/aZPn/WY+sc3JAbnmmA2gEMRYTtgYjpJVDhxVhBHtvjWFYrFnt30G2C2C4QsNZPWy1 EgI5GWKf54y6M5raA6y9diEYNR0ShUqI5cJJ4Y+LM= X-Google-Smtp-Source: AGHT+IGpcUl+FlLzBmXJXRIwsKv9Knbj6Q5AkfLwTI0HmZ+HAzHniv6fovZ/WAptuaR81i7JI/+3U4vDowDVpw== X-Received: from lunarfall.svl.corp.google.com ([2620:15c:2d3:204:705c:77af:6ef2:8f42]) (user=steadmon job=sendgmr) by 2002:a05:6902:1748:b0:dc6:f21f:64ac with SMTP id bz8-20020a056902174800b00dc6f21f64acmr656937ybb.12.1706921438119; Fri, 02 Feb 2024 16:50:38 -0800 (PST) Date: Fri, 2 Feb 2024 16:50:27 -0800 In-Reply-To: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.43.0.594.gd9cf4e227d-goog Message-ID: Subject: [RFC PATCH v2 2/6] test-tool run-command testsuite: get shell from env From: Josh Steadmon To: git@vger.kernel.org Cc: johannes.schindelin@gmx.de, peff@peff.net, phillip.wood@dunelm.org.uk, gitster@pobox.com When running tests through `test-tool run-command testsuite`, we currently hardcode `sh` as the command interpreter. As discussed in [1], this is incorrect, and we should be using the shell set in TEST_SHELL_PATH instead. Add a shell_path field in struct testsuite so that we can pass this to the task runner callback. If this is non-null, we'll use it as the argv[0] of the subprocess. Otherwise, we'll just execute the test program directly. When setting up the struct testsuite in testsuite(), use the value of TEST_SHELL_PATH if it's set, otherwise default to `sh`. [1] https://lore.kernel.org/git/20240123005913.GB835964@coredump.intra.peff.net/ Signed-off-by: Josh Steadmon --- t/helper/test-run-command.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/t/helper/test-run-command.c b/t/helper/test-run-command.c index c0ed8722c8..a41a54d9cb 100644 --- a/t/helper/test-run-command.c +++ b/t/helper/test-run-command.c @@ -65,6 +65,7 @@ struct testsuite { struct string_list tests, failed; int next; int quiet, immediate, verbose, verbose_log, trace, write_junit_xml; + const char *shell_path; }; #define TESTSUITE_INIT { \ .tests = STRING_LIST_INIT_DUP, \ @@ -80,7 +81,9 @@ static int next_test(struct child_process *cp, struct strbuf *err, void *cb, return 0; test = suite->tests.items[suite->next++].string; - strvec_pushl(&cp->args, "sh", test, NULL); + if (suite->shell_path) + strvec_push(&cp->args, suite->shell_path); + strvec_push(&cp->args, test); if (suite->quiet) strvec_push(&cp->args, "--quiet"); if (suite->immediate) @@ -162,6 +165,10 @@ static int testsuite(int argc, const char **argv) if (max_jobs <= 0) max_jobs = online_cpus(); + suite.shell_path = getenv("TEST_SHELL_PATH"); + if (!suite.shell_path) + suite.shell_path = "sh"; + dir = opendir("."); if (!dir) die("Could not open the current directory"); From patchwork Sat Feb 3 00:50:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Steadmon X-Patchwork-Id: 13543656 Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 27000257A for ; Sat, 3 Feb 2024 00:50:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706921442; cv=none; b=iGDIIlDTjNrsduifWHp894wTrcbjM03aQnATd76j9qLsLe9WzACf/+5HKeHdqj/3RssSqjkjw+xwsZt18GI4vkq/1DvUOHOpv6eS1dt5MhjTyDwU4Ok4yANMoVq4FlS/WN7hzvZyucLkJPMJGOQjbpUvjugEr3nVicXlzH+Lkis= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706921442; c=relaxed/simple; bh=AtMiQtE8aAJVCZGBMVK3ZcGu3LOeGL5Z4Oq0ih+F0G8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Z5mN1vScp9/IdfiA+wYPGbornroKan8L5JJNszVM4fUCyNVf0VggTUcG7JIjRAn10+SFENxRQu9e+pho8g548Z93v8C5qN1blcaswyBE7gMQlL+gt094+rw3fb736fwoPhxf1/FqTQOrR0CJPS/PNFcqSAnCsrPkRaW3Z3EMWWQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Gp28FXrU; arc=none smtp.client-ip=209.85.219.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Gp28FXrU" Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-dc6b269686aso3898040276.1 for ; Fri, 02 Feb 2024 16:50:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1706921440; x=1707526240; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=cVROZgRCYYjvX4iQzXA0HgvWzMdib79l9N9wUFJRqjw=; b=Gp28FXrUahZX7oRPqgIJQIobr4f/GL2q53rEhTggk0LiflKuwt/WQid82PLnmCRLkJ SbaPPWwPo3ukHE7p4yZMOC18IViIXPM8kE54x16Cn+MnqacLIOyEY59oA/QSL5EuOLwh oyBUzHx0TXPaV941XhP/W38J2cYDmszxQZk3Y3dKfeECoAfkoJAVhOoijQzL/ltZKV0a bbPyoStcixKCbLUQ2UE8Omu1tyeSIb4SY6veJPe7hhEayRFzt3czx9n7TJGaxU3NY9KI zh+AbnS7mYDqBt+qyPT3f2/qurkgBCGy2gKRnUVb/TSZdedPXndShdtsoGGCuiTX3pe5 bwgg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706921440; x=1707526240; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=cVROZgRCYYjvX4iQzXA0HgvWzMdib79l9N9wUFJRqjw=; b=sQjDH7edVzf+yyi6sreDwBLP5bn96h5ohmGViom2BF2uGNSRw0DtWQkY5N0UBvXAMt GoyQ9kaP1uT3r9LGnN6EJ93cKcGd+h80XYYvSAvYIQEVJGuRzuhrNkWmPhnLwTN1ddXX xzCSuyIOPJcto9aRlssp/iao7fZzZQPDrNT5mSYPg3Q2f192X17xbAtu0I1M0GmOzA2O 4E36jSLaDOgmGXEyoIgqSh9YP8Ka6bJpb6UOGzP7yI9+iGBsy4iyyraZ2hSgmApx9w1P 0k04TsHB0Mjc8NrbmA5P+eobnZoUQ9HwHkWzlniOeurSqFstLOfFtJyhp/9kYhpah6we +5mg== X-Gm-Message-State: AOJu0Yw/LGicNoaTjal8MsRPcBB6X7YzzzH2tqwE1LZWPdp5+ZWvdHm4 4Gz1JAo2wLRJqBd5yw+or9Fn+uL+FvfsoWRjJCK8196GKq85PQFik0HGkl//4/p0Oy+5iWThrgu uScUNknX1+r7IERZn7JB0KBU2Go5ckMICGBZ4GLeH7ubrx76r8krLI44eGqW9AOzUYGQaiI27lr 5P12Hj3fh+wzJwk9w0ZAncWsnUd4Z6FMOJ28buPxI= X-Google-Smtp-Source: AGHT+IGByA6gRo//9wLnu3lRV7t91EiOzK7cC8w0wWZhazosw/zAQ99LUOu0UrNfs/RMV7decKKFyFGFi8iGwQ== X-Received: from lunarfall.svl.corp.google.com ([2620:15c:2d3:204:705c:77af:6ef2:8f42]) (user=steadmon job=sendgmr) by 2002:a05:6902:2012:b0:dc6:ece2:634c with SMTP id dh18-20020a056902201200b00dc6ece2634cmr138335ybb.11.1706921440124; Fri, 02 Feb 2024 16:50:40 -0800 (PST) Date: Fri, 2 Feb 2024 16:50:28 -0800 In-Reply-To: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.43.0.594.gd9cf4e227d-goog Message-ID: Subject: [RFC PATCH v2 3/6] test-tool run-command testsuite: remove hardcoded filter From: Josh Steadmon To: git@vger.kernel.org Cc: johannes.schindelin@gmx.de, peff@peff.net, phillip.wood@dunelm.org.uk, gitster@pobox.com `test-tool run-command testsuite` currently assumes that it will only be running the shell test suite, and therefore filters out anything that does not match a hardcoded pattern of "t[0-9][0-9][0-9][0-9]-*.sh". Later in this series, we'll adapt `test-tool run-command testsuite` to also support unit tests, which do not follow the same naming conventions as the shell tests, so this hardcoded pattern is inconvenient. Since `testsuite` also allows specifying patterns on the command-line, let's just remove this pattern. As noted in [1], there are no longer any uses of `testsuite` in our codebase, it should be OK to break backwards compatibility in this case. We also add a new filter to avoid trying to execute "." and "..", so that users who wish to execute every test in a directory can do so without specifying a pattern. [1] https://lore.kernel.org/git/850ea42c-f103-68d5-896b-9120e2628686@gmx.de/ Signed-off-by: Josh Steadmon --- t/helper/test-run-command.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/t/helper/test-run-command.c b/t/helper/test-run-command.c index a41a54d9cb..e6bd792274 100644 --- a/t/helper/test-run-command.c +++ b/t/helper/test-run-command.c @@ -175,9 +175,7 @@ static int testsuite(int argc, const char **argv) while ((d = readdir(dir))) { const char *p = d->d_name; - if (*p != 't' || !isdigit(p[1]) || !isdigit(p[2]) || - !isdigit(p[3]) || !isdigit(p[4]) || p[5] != '-' || - !ends_with(p, ".sh")) + if (!strcmp(p, ".") || !strcmp(p, "..")) continue; /* No pattern: match all */ From patchwork Sat Feb 3 00:50:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Steadmon X-Patchwork-Id: 13543657 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 10C1B5396 for ; Sat, 3 Feb 2024 00:50:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706921444; cv=none; b=Xe4BI5Og+vGre/eFzUyAY1I0+JCXdblbNhnGByQGOQnvQgtWlSzyEweSnKCUt6eX73XBPQ8+A8H+7QcbKZHvMx5SfluKaTBkuYhDtA8Y49kNtm2SNOMkUIKJKC9/pAXGtGGMCERl6+5qhOwHVK/TySxlHh7EzGNkDhdqWRnnDnM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706921444; c=relaxed/simple; bh=BFkovf9z3h2O/Zj7FehocK22dRLk7/+hP9pwgz2IbGU=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=KUYj4wFZ94wYSC3c8RW7B6fHLfrguoyymHRMGoo21X2nNltdZhxCNoVePn823SoMklpZ48c1VFkidK1yC0QpJiYgTKyMsETki5591IWMtEgwmITz+bKZrt2yutGm8Vsc8Yquml70ai33lpE/5jA58rX+c9ZQbHMX5Bnk6kLEyTI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=gywh0mWb; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="gywh0mWb" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-60420dcb127so17003077b3.1 for ; Fri, 02 Feb 2024 16:50:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1706921442; x=1707526242; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Ik1LLyHE1sLsNq5BQyDgfQO92otwf9v4YaUoiA61F6M=; b=gywh0mWbGk+PMFuLHoBcnlZ9NTGH6dYX7o+Tq5QEz5Pkofvhb1HuMu3xmbBjgTf/bq FM88fpeOiKVmT/CaZONitUu5cvon//C1E1WQ077YYwZH/dTdJYRUkMpWaPkMUeHq3b68 96d1qXjyI0JtqFrmw2PipPdJryOeDuvwdAJtU0iw+mO+mU7hHkf7hFf8OVudttBMekES JQ5Nicp6M20ostVfrTbQxpIcU6xAAZy6qIQLFSW8TNnJf68BNTIjI8+/zqPiHM59zInK 3QxoIpwVzjwO+XSvF/SVAaHdce3KxGyLces9DsmsSkszmWJVAd+lj+AKSZhWsV8M5uHN ZPNA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706921442; x=1707526242; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Ik1LLyHE1sLsNq5BQyDgfQO92otwf9v4YaUoiA61F6M=; b=MJ3/Ju9MM+BO1ZQ4cabgvDpX6KnuaxqL+1rzHyWxKeDo6aQwbu0DqYY+RPwFcEVwRI jY2kPyOCbllwOoE0xuZHKHESh52d8tcoBttyvemNjo8+pzTGkxwbPVfh7J2ayBv3ixHe QbBqF5Iibdbtct7hHn/y0bdpF4Cnu5zI0SN3o1YJe3oP4rOb04JdAm8Dhj8+hty2bAVf BmLfVmpMYFavXjBWVwrUOJUIIe5DZ5QVF6900G8PI3WHp3aUM5aZ0j6orN/2c5we1sqI jPvD0SYyYsqSt8iWwx8fOYOBzncDUEokZZtM6F3S/q7D+cTFCyBBD42IT1x+ZodmpZr1 XPeg== X-Gm-Message-State: AOJu0YwncuUXiKQQOod7NSDyjpdPNojikjSjA5HclPRv0iuzoVJhcyHK EaRxSbBe+ZfEPx4g8RaZlE3xqEkRq2gnL569OYr6fVkZ3C5VsMsg3fqA+h97UNZJvYlyI+FElW9 m82MwDOMNXKseP8sjBBu0kM/vIS0c9QGnxVxTPD9QsfSXWEv2knnm4ncFEL81GM4ie7q9tgWLWk ktfxzFpthY2A3mhPLMo+5M62BuUh+8TgcJd7pXavU= X-Google-Smtp-Source: AGHT+IHpEzFQ+VUfC6AeECewDKcRpFM3xRmGbvtCvTIDLc1TjnKzfYK3cbNr0oXgtOOuYZGfF57CM6kYpj7jqA== X-Received: from lunarfall.svl.corp.google.com ([2620:15c:2d3:204:705c:77af:6ef2:8f42]) (user=steadmon job=sendgmr) by 2002:a05:690c:ece:b0:5ff:6e82:ea31 with SMTP id cs14-20020a05690c0ece00b005ff6e82ea31mr44935ywb.3.1706921442018; Fri, 02 Feb 2024 16:50:42 -0800 (PST) Date: Fri, 2 Feb 2024 16:50:29 -0800 In-Reply-To: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.43.0.594.gd9cf4e227d-goog Message-ID: Subject: [RFC PATCH v2 4/6] test-tool run-command testsuite: support unit tests From: Josh Steadmon To: git@vger.kernel.org Cc: johannes.schindelin@gmx.de, peff@peff.net, phillip.wood@dunelm.org.uk, gitster@pobox.com Teach the testsuite runner in `test-tool run-command testsuite` how to run unit tests: if TEST_SHELL_PATH is not set, assume that we're running the programs directly from CWD, rather than defaulting to "sh" as an interpreter. With this change, you can now use test-tool to run the unit tests: $ make $ cd t/unit-tests/bin $ ../../helper/test-tool run-command testsuite This should be helpful on Windows to allow running tests without requiring Perl (for `prove`), as discussed in [1] and [2]. This again breaks backwards compatibility, as it is now required to set TEST_SHELL_PATH properly for executing shell scripts, but again, as noted in [2], there are no longer any such invocations in our codebase. [1] https://lore.kernel.org/git/nycvar.QRO.7.76.6.2109091323150.59@tvgsbejvaqbjf.bet/ [2] https://lore.kernel.org/git/850ea42c-f103-68d5-896b-9120e2628686@gmx.de/ Signed-off-by: Josh Steadmon --- t/helper/test-run-command.c | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/t/helper/test-run-command.c b/t/helper/test-run-command.c index e6bd792274..a0b8dc6fd7 100644 --- a/t/helper/test-run-command.c +++ b/t/helper/test-run-command.c @@ -158,6 +158,8 @@ static int testsuite(int argc, const char **argv) .task_finished = test_finished, .data = &suite, }; + struct strbuf progpath = STRBUF_INIT; + size_t path_prefix_len; argc = parse_options(argc, argv, NULL, options, testsuite_usage, PARSE_OPT_STOP_AT_NON_OPTION); @@ -165,9 +167,14 @@ static int testsuite(int argc, const char **argv) if (max_jobs <= 0) max_jobs = online_cpus(); + /* + * If we run without a shell, we have to provide the relative path to + * the executables. + */ suite.shell_path = getenv("TEST_SHELL_PATH"); if (!suite.shell_path) - suite.shell_path = "sh"; + strbuf_addstr(&progpath, "./"); + path_prefix_len = progpath.len; dir = opendir("."); if (!dir) @@ -180,13 +187,17 @@ static int testsuite(int argc, const char **argv) /* No pattern: match all */ if (!argc) { - string_list_append(&suite.tests, p); + strbuf_setlen(&progpath, path_prefix_len); + strbuf_addstr(&progpath, p); + string_list_append(&suite.tests, progpath.buf); continue; } for (i = 0; i < argc; i++) if (!wildmatch(argv[i], p, 0)) { - string_list_append(&suite.tests, p); + strbuf_setlen(&progpath, path_prefix_len); + strbuf_addstr(&progpath, p); + string_list_append(&suite.tests, progpath.buf); break; } } @@ -213,6 +224,7 @@ static int testsuite(int argc, const char **argv) string_list_clear(&suite.tests, 0); string_list_clear(&suite.failed, 0); + strbuf_release(&progpath); return ret; } From patchwork Sat Feb 3 00:50:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Steadmon X-Patchwork-Id: 13543658 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ECDC863CB for ; Sat, 3 Feb 2024 00:50:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706921446; cv=none; b=ev7RziiuKhisEitZgkdUs1slvMlEL3w/rZLlIixbD84zVBFam/nDTjhblIBi8fog9s3HXqOGtsIfLGaCfcs32tn3VLLYiuEuxgwZiH3kcCM22zOJbA6i3glruG+OPBAyRTOodxN+hZZ3KpFDUfPdwunKa9u14hqGaSwj6QHegzM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706921446; c=relaxed/simple; bh=EWOirqDuXBrPcUm4D2QXoU1aTxTanl5L9E1qLu9aaGc=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=ZTDxTQQ1LRwdy0Q/UBPAnMFRgZhpTI+rEFoXfHrw8es++3QMTOsQ8Py885tH1QfDKycI3jGMzO7ibw/h5dQdQYVz9xmLTomz6Ygbw+d0z39LPedES4Fl8umq/fQQERwZNKoTkAVgyL7xj3YwQW/qTbtiJ5P9Keco2SQPTo7kRlc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=wqghtr/F; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="wqghtr/F" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-60404b12af2so43158767b3.2 for ; Fri, 02 Feb 2024 16:50:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1706921444; x=1707526244; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=KmSIMf2vwCAOfQzAupBVRVGEAt+IBP4x+xK6jYJMRds=; b=wqghtr/FKxuFYZlEYEdbBYSSZay4z/Po15aiqB+YngdKn7OdjOs3J7p8jXgBDlt8L1 SJtJtAdamW128XBk00P/+frw9FjTOaVHRMGK9l97JcteyqBfOle4nOQZujLXeMbb4x5a jAmK7LpDTSLXvFmVYz1x8LFwGcwd3xLc8CmcMmxZKyrhyghOXY970KjSj0SV9l6/U+Es ZgU6AOYeT+cQn1NtB5ZUVWgwuDRiEZaPloRRBQ2hxXj+mwzcFy4VmhWRUxltpT1p4VK1 y+j6q68icqcwenchNQBWoWynfk9p9FGa8uIHBAeIfRAQisMx87FLgUvCNm+dHE8TvgkQ J2Og== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706921444; x=1707526244; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=KmSIMf2vwCAOfQzAupBVRVGEAt+IBP4x+xK6jYJMRds=; b=S1m9qRRlIftiUUvqPQxR7HG9asT64NyQXEndfEbSYafGlevNyWjb7AfFtCA+n8XQwR ghypb1yGHTcMNjKV72aAY3OL5fYB1khYKmUBppuyOBFyRP7M/6VdpwubYKe41vAAc724 lrj/teddq0M5mNufqPUp6lQYCSAr0wUbcgKNUd6Dzgm0a3i9HZ6cGv5c0KLCC9RbOaZp 7qOmC01POPz7YiIHGUyfhGbiwv0GS9RNDL0++xjhaVICZVnrtaeWlExnMXVhQJVa2kiE hZl/+EGRL3XeT6ICdfa3JjZyAvo7bHGJnIqLwDHygI2KaXxlfLrBj7QaOsLucn0l8iBg v78w== X-Gm-Message-State: AOJu0Yz21Me2rCt2Vh+VUXHLgFpYpSLiyDVqivOD10ETuHOy5v/gi9sK nKsOITzBhcDWhm3BiRiwpXvD9FNT2uSTbHIHQFVAsAP0t37Fp8Una4H/zo3YUaMeFD0jvYarzuO nVkmoqTfTk6KixXWZ9otgDjtMGA7dPeO47yFDa3ecVUjtI98tUPdZ1PRnYR8Tklf1J43PZOCulf I9iMFTFmlSvJxdcdPy2qscV6G+xXZvUrosaGhp29w= X-Google-Smtp-Source: AGHT+IF89TCuUZll+8u9YSiIRVUOwr/yYqBHeVDxtSMu9srr8OW4ioYTkqmjqtxVN/GCUs8kOw6mbTlMywe3fA== X-Received: from lunarfall.svl.corp.google.com ([2620:15c:2d3:204:705c:77af:6ef2:8f42]) (user=steadmon job=sendgmr) by 2002:a05:6902:138f:b0:dc6:dd74:de68 with SMTP id x15-20020a056902138f00b00dc6dd74de68mr210023ybu.12.1706921443881; Fri, 02 Feb 2024 16:50:43 -0800 (PST) Date: Fri, 2 Feb 2024 16:50:30 -0800 In-Reply-To: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.43.0.594.gd9cf4e227d-goog Message-ID: Subject: [RFC PATCH v2 5/6] unit tests: add rule for running with test-tool From: Josh Steadmon To: git@vger.kernel.org Cc: johannes.schindelin@gmx.de, peff@peff.net, phillip.wood@dunelm.org.uk, gitster@pobox.com In the previous commit, we added support in test-tool for running collections of unit tests. Now, add rules in t/Makefile for running in this way. This new rule can be executed from the top-level Makefile via `make DEFAULT_UNIT_TEST_TARGET=unit-tests-test-tool unit-tests`, or by setting DEFAULT_UNIT_TEST_TARGET in config.mak. Signed-off-by: Josh Steadmon --- Makefile | 2 +- t/Makefile | 10 +++++++++- 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/Makefile b/Makefile index ba55d817ee..b0d1f04b4d 100644 --- a/Makefile +++ b/Makefile @@ -3870,5 +3870,5 @@ $(UNIT_TEST_PROGS): $(UNIT_TEST_BIN)/%$X: $(UNIT_TEST_DIR)/%.o $(UNIT_TEST_DIR)/ .PHONY: build-unit-tests unit-tests build-unit-tests: $(UNIT_TEST_PROGS) -unit-tests: $(UNIT_TEST_PROGS) +unit-tests: $(UNIT_TEST_PROGS) t/helper/test-tool$X $(MAKE) -C t/ unit-tests diff --git a/t/Makefile b/t/Makefile index 1283c90c10..6e6316c29b 100644 --- a/t/Makefile +++ b/t/Makefile @@ -45,6 +45,7 @@ CHAINLINTTESTS = $(sort $(patsubst chainlint/%.test,%,$(wildcard chainlint/*.tes CHAINLINT = '$(PERL_PATH_SQ)' chainlint.pl UNIT_TEST_SOURCES = $(wildcard unit-tests/t-*.c) UNIT_TESTS = $(patsubst unit-tests/%.c,unit-tests/bin/%$(X),$(UNIT_TEST_SOURCES)) +UNIT_TESTS_NO_DIR = $(notdir $(UNIT_TESTS)) # `test-chainlint` (which is a dependency of `test-lint`, `test` and `prove`) # checks all tests in all scripts via a single invocation, so tell individual @@ -72,7 +73,7 @@ $(T): $(UNIT_TESTS): @echo "*** $@ ***"; $@ -.PHONY: unit-tests unit-tests-raw unit-tests-prove +.PHONY: unit-tests unit-tests-raw unit-tests-prove unit-tests-test-tool unit-tests: $(DEFAULT_UNIT_TEST_TARGET) unit-tests-raw: $(UNIT_TESTS) @@ -80,6 +81,13 @@ unit-tests-raw: $(UNIT_TESTS) unit-tests-prove: @echo "*** prove - unit tests ***"; $(PROVE) $(GIT_PROVE_OPTS) $(UNIT_TESTS) +unit-tests-test-tool: + @echo "*** test-tool - unit tests **" + ( \ + cd unit-tests/bin && \ + ../../helper/test-tool$X run-command testsuite $(UNIT_TESTS_NO_DIR)\ + ) + pre-clean: $(RM) -r '$(TEST_RESULTS_DIRECTORY_SQ)' From patchwork Sat Feb 3 00:50:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Steadmon X-Patchwork-Id: 13543659 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AD7DF6FC8 for ; Sat, 3 Feb 2024 00:50:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706921448; cv=none; b=JPcUOuFUD/amd+Wa8bxQ4bxLQEDYecl813Ta6b1wRrHlpmc/wFGdnQR6rMo3JbPKnZUk1eGBDyWOuAO93aJF0J8H9cOcVtyT7S0wWQ9kezIVNJpmMXihxgwT9TsA/yupvGW286cKM1D6TXFTaBDYBbS7QY1qmyy6Aem8s9/At7s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706921448; c=relaxed/simple; bh=ta7jSsFsUvbqYfsj5MRTByPYVNQZY3T4CU1P1Gmxmi0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=IGtkJaSy6Jwml+wna2wq9PMoFBfP6qurM/96mA1zFKDS31E9MedGNUjJXAoMEeM5AZLbDuSpiIF6AgMLCX3rKJcDzGOvt8VKW/tBIE0SwAu8LGddKeJ/8cbxCvrqIoVEDxxeQrb7h0Rl0DqFdNP2G1jFdUS6Te6KKKBkgZPPaE4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=dcP3Myj0; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="dcP3Myj0" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-5ee22efe5eeso45717487b3.3 for ; Fri, 02 Feb 2024 16:50:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1706921445; x=1707526245; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=m1c+35MaGAYPyhmHMrHlmAOh6RNGkcWUZiwUw6lLKE8=; b=dcP3Myj0mF193QpuOsO6fmmAtJ1VlDvNi2AIg8bAP4uC8jqGLn9GvQJvDGMCrmZ6DY 2kGqDhCSnz8sGtgt368ZdB4Qk+LtzOiU96uVO7uKrIMUvrSbR4kZvki8YZKJyvbxRGeK NS2TLCuNon7Kni+0FV1Gbsg1QyW0A4M9mO5LkwnhWapW885CBF4nWd1J7IfrSbj0Tfwa ZBUWeOlgRt/nv0xYIjf4R0tkNyJYhTQbN59hSVCYmxyFxArJtWLv5iVi+hyi7rZx7f0X jpkRs/Jd76S0Ots3xP4k0mWjWl8c60eW5m8b0Z69qXJtFdaCkjQPtHZRlliDQU/1ES3E +cTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706921445; x=1707526245; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=m1c+35MaGAYPyhmHMrHlmAOh6RNGkcWUZiwUw6lLKE8=; b=sBeinp6L3I7Knmk3CeSobtPSnRRmwLTbncYx7MBBCgEHdJngfqoA8LHuWX1S7f0NmC RxGqMeWo4MPXyQCtKiWXpz8Akr2YPEvzqLPFszgN6wn7Ax/wjBt2Yx6I9NVkd65arUix 8J6tzeZwLI2SQeZ0ykx5/VXjEoskQWKk2uDFahXQEIRTwqEGsvgf2/VFxySdhKVYR/ZF bjQbZnyG2DGWZE96kjNbRJ0N32Lnv08Du314/uXSlbdgEBHueySDrvqtDwX+DCLRbZIy GEexJ2xwpC6rph1PKeK/3YuxabBjvkp2pS7PttCBO8amgI9iIBStWONZm9pwOYkrk43V qbcw== X-Gm-Message-State: AOJu0YxqNK4Nj0xEfi5pXebvkzLHmQWXUp6Vk2J+xwmaMGXilcgmyR2k uwYcYWe0REyrKM7jiVrHFmXBGLLPLM5EYAtxw1klUWsGs2rz9gG3AVwji7fcOprHc2c5rR40AeN t9nMaOpk5Xn5E+O8a+/wOI81ddAE9+n3EKp19Go3cgnm3ATAJveCHChO2eVEEpmJQnBuZexPe4+ tQL/i8uP32DuctluuzExg5a7QXIWldVW7WnLAeiIc= X-Google-Smtp-Source: AGHT+IHnYbh1I6u9Sd5z8tfSeFgQaalt7+D+gtu97c1zyt7OHq++KfEQX5aAchM0MpQAHb+c9e6pCHwg59lJbQ== X-Received: from lunarfall.svl.corp.google.com ([2620:15c:2d3:204:705c:77af:6ef2:8f42]) (user=steadmon job=sendgmr) by 2002:a81:4c14:0:b0:5ff:a885:65b with SMTP id z20-20020a814c14000000b005ffa885065bmr1173375ywa.10.1706921445608; Fri, 02 Feb 2024 16:50:45 -0800 (PST) Date: Fri, 2 Feb 2024 16:50:31 -0800 In-Reply-To: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.43.0.594.gd9cf4e227d-goog Message-ID: Subject: [RFC PATCH v2 6/6] t/Makefile: run unit tests alongside shell tests From: Josh Steadmon To: git@vger.kernel.org Cc: johannes.schindelin@gmx.de, peff@peff.net, phillip.wood@dunelm.org.uk, gitster@pobox.com From: Jeff King Add a wrapper script to allow `prove` to run both shell tests and unit tests from a single invocation. This avoids issues around running prove twice in CI, as discussed in [1]. Additionally, this moves the unit tests into the main dev workflow, so that errors can be spotted more quickly. NEEDS WORK: as discussed in previous commits in this series, there's a desire to avoid `prove` specifically and (IIUC) unnecessary fork()/exec()ing in general on Windows. This change adds an extra exec() for each shell and unit test execution, will that be a problem for Windows? [1] https://lore.kernel.org/git/pull.1613.git.1699894837844.gitgitgadget@gmail.com/ Signed-off-by: Jeff King Signed-off-by: Josh Steadmon --- t/Makefile | 2 +- t/run-test.sh | 13 +++++++++++++ 2 files changed, 14 insertions(+), 1 deletion(-) create mode 100755 t/run-test.sh diff --git a/t/Makefile b/t/Makefile index 6e6316c29b..6a67fc22d7 100644 --- a/t/Makefile +++ b/t/Makefile @@ -64,7 +64,7 @@ failed: test -z "$$failed" || $(MAKE) $$failed prove: pre-clean check-chainlint $(TEST_LINT) - @echo "*** prove ***"; $(CHAINLINTSUPPRESS) $(PROVE) --exec '$(TEST_SHELL_PATH_SQ)' $(GIT_PROVE_OPTS) $(T) :: $(GIT_TEST_OPTS) + @echo "*** prove (shell & unit tests) ***"; $(CHAINLINTSUPPRESS) $(PROVE) --exec ./run-test.sh $(GIT_PROVE_OPTS) $(T) $(UNIT_TESTS) :: $(GIT_TEST_OPTS) $(MAKE) clean-except-prove-cache $(T): diff --git a/t/run-test.sh b/t/run-test.sh new file mode 100755 index 0000000000..c29fef48dc --- /dev/null +++ b/t/run-test.sh @@ -0,0 +1,13 @@ +#!/bin/sh + +# A simple wrapper to run shell tests via TEST_SHELL_PATH, +# or exec unit tests directly. + +case "$1" in +*.sh) + exec ${TEST_SHELL_PATH:-/bin/sh} "$@" + ;; +*) + exec "$@" + ;; +esac