From patchwork Tue Apr 30 19:55:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Steadmon X-Patchwork-Id: 13649941 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DB876199E80 for ; Tue, 30 Apr 2024 19:56:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714506964; cv=none; b=gFX7K9e7SXu7whcnP2mjAJMC7PoFg9L+AtjGP7YCcJHADJJqN52f0DQ7oj+CvCLQy0V7OpF5NxfLwuMuOmuccicvKIpKiZLtAIO+RVLiX0QAa6nNhQZ698Ip5wBZX9DZ/YNLzRfmxpqaC3A5p15DFTPmRahMPafCky9EcTlsvU0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714506964; c=relaxed/simple; bh=IKJ/bmKc7i5EHiKTOEsOHP7zFCIzXavEiNpEOSGaDBw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=g5V/diOC4AtddRr/u0XeljAONcwdv4O1AUuvJQiXKfyeGJ2a0MXyrKXREOF98Fu5rGzZ8eWegL3aezF0mqmZqI4ii8xOgCOhRnrsQaL1uOoZv83uu9ZRNA6U5p4fEqYgqIAGANNqxAS0F1JvugWLPHsp88n9lsCynoyV3PufVzI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=ba5qCg6f; arc=none smtp.client-ip=209.85.219.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="ba5qCg6f" Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-dc64f63d768so12129531276.2 for ; Tue, 30 Apr 2024 12:56:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1714506962; x=1715111762; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=p2I2pmf6nAyTKwCSpel+0sQL1qh691vfCVOF0XiOvOg=; b=ba5qCg6fMbBT4BNLjMpnGbjJjvyUnuuDjZ2uefaZdSvMqYSfBGzXkNlnrLtiCMeCj1 NiOtf7l9sdA3eU+CnIHqZJpLwGj5ohtIvA0IOVtFFF5OS3yucUntSWsbgzH+vZlyGpHt fVk22Ona2qWacJR4xop//8xZEqeBqGM3WBhW7nFNF3cpCibtd17WrDjcgsWbtRAFytWw aAFIlXkKztQQ6ZotpS2KDQtNKV85xWagbni3KaCrutc/sY5pEUp5iaBzn+Vw0GKilyu5 Cc4RGUBq9dUN86mU6mV4/ShvmLOCzbWUMQy59diZnxMFl46ucp12nFSCp608wAxP2l/A ZTbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714506962; x=1715111762; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=p2I2pmf6nAyTKwCSpel+0sQL1qh691vfCVOF0XiOvOg=; b=AKVkhtqeWevq0ETv2GZJcKZ4Lw/N6InYtKq+np+3GxGq7hRCMS1ITIUeFGGv+FQ/e3 R1ckGIDtP64YQi2RweLNpfew98nENeGBjDxhjPR6Or5HR1ILbbT2XdaYoSJCnr9CpADT vZiXAl7qlq0MPE5FakGPanzFHSXYAHqVa6A74q3AZS+AOygJumLpKddj7Jtqd1O5TKIp 9611hiPXPIiKZbOuIUqa3bQd+9m/YTW7chxSm22lk8t1G/+6qJHqYQLWjk4mW0jg5Q/e DJbCydSUTbeXBjZ1x3qSngnZRTSpI5qGgUvKiXc16y/3G3wb2brqjb+VWeGFIyRTds8Y wY6w== X-Gm-Message-State: AOJu0YxK1kIseopbeq9dCEXSphkRueIoMLJQPRhejpoYd33zUQUGMiXs 8xIM1aUZ6l3Z+htTIcItvhL/rHc3fDqgpKJCkSqQcfTy4zT8W3dnu4BrpDtpV0352S5TiAJejw6 O6ec1AHijzl3erxSGcWctfxJ8u/GdUTWl6mwhCXkVO0eY8VvrGYubNyn677wuNKInAvG7YeMpbA WdcMGi4ZpZ1XuKA30ZDr1TJY+PyMjZZAgw9f3eytY= X-Google-Smtp-Source: AGHT+IHTbW8tXAQxE158SpNMgntJe0FQ2nuR1H6svsEYkZcOIBLnyghuAJT0YEOfbOikQbMpf+RpiPSwZhQehw== X-Received: from lunarfall.svl.corp.google.com ([2620:15c:2d3:204:3e48:fe18:8aa:8a0e]) (user=steadmon job=sendgmr) by 2002:a05:6902:120c:b0:dcb:b9d7:2760 with SMTP id s12-20020a056902120c00b00dcbb9d72760mr153345ybu.13.1714506961749; Tue, 30 Apr 2024 12:56:01 -0700 (PDT) Date: Tue, 30 Apr 2024 12:55:51 -0700 In-Reply-To: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.45.0.rc0.197.gbae5840b3b-goog Message-ID: <677745110078501e611faf3d1facd1518e0453a2.1714506612.git.steadmon@google.com> Subject: [PATCH v5 1/7] t0080: turn t-basic unit test into a helper From: Josh Steadmon To: git@vger.kernel.org Cc: gitster@pobox.com, peff@peff.net While t/unit-tests/t-basic.c uses the unit-test framework added in e137fe3b29 (unit tests: add TAP unit test framework, 2023-11-09), it is not a true unit test in that it intentionally fails in order to exercise various codepaths in the unit-test framework. Thus, we intentionally exclude it when running unit tests through the various t/Makefile targets. Instead, it is executed by t0080-unit-test-output.sh, which verifies its output follows the TAP format expected for the various pass, skip, or fail cases. As such, it makes more sense for t-basic to be a helper item for t0080-unit-test-output.sh, so let's move it to t/helper/test-example-tap.c and adjust Makefiles as necessary. Signed-off-by: Josh Steadmon --- Makefile | 4 ++-- t/Makefile | 2 +- .../t-basic.c => helper/test-example-tap.c} | 5 ++-- t/helper/test-tool.c | 1 + t/helper/test-tool.h | 1 + t/t0080-unit-test-output.sh | 24 +++++++++---------- 6 files changed, 20 insertions(+), 17 deletions(-) rename t/{unit-tests/t-basic.c => helper/test-example-tap.c} (95%) diff --git a/Makefile b/Makefile index 23723367b8..ba55d817ee 100644 --- a/Makefile +++ b/Makefile @@ -802,6 +802,7 @@ TEST_BUILTINS_OBJS += test-dump-split-index.o TEST_BUILTINS_OBJS += test-dump-untracked-cache.o TEST_BUILTINS_OBJS += test-env-helper.o TEST_BUILTINS_OBJS += test-example-decorate.o +TEST_BUILTINS_OBJS += test-example-tap.o TEST_BUILTINS_OBJS += test-find-pack.o TEST_BUILTINS_OBJS += test-fsmonitor-client.o TEST_BUILTINS_OBJS += test-genrandom.o @@ -1338,7 +1339,6 @@ THIRD_PARTY_SOURCES += compat/regex/% THIRD_PARTY_SOURCES += sha1collisiondetection/% THIRD_PARTY_SOURCES += sha1dc/% -UNIT_TEST_PROGRAMS += t-basic UNIT_TEST_PROGRAMS += t-mem-pool UNIT_TEST_PROGRAMS += t-strbuf UNIT_TEST_PROGRAMS += t-ctype @@ -3218,7 +3218,7 @@ perf: all .PRECIOUS: $(TEST_OBJS) -t/helper/test-tool$X: $(patsubst %,t/helper/%,$(TEST_BUILTINS_OBJS)) +t/helper/test-tool$X: $(patsubst %,t/helper/%,$(TEST_BUILTINS_OBJS)) $(UNIT_TEST_DIR)/test-lib.o t/helper/test-%$X: t/helper/test-%.o GIT-LDFLAGS $(GITLIBS) $(REFTABLE_TEST_LIB) $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(filter %.a,$^) $(LIBS) diff --git a/t/Makefile b/t/Makefile index 2d95046f26..4861edafe6 100644 --- a/t/Makefile +++ b/t/Makefile @@ -48,7 +48,7 @@ CHAINLINTTESTS = $(sort $(patsubst chainlint/%.test,%,$(wildcard chainlint/*.tes CHAINLINT = '$(PERL_PATH_SQ)' chainlint.pl UNIT_TEST_SOURCES = $(wildcard unit-tests/t-*.c) UNIT_TEST_PROGRAMS = $(patsubst unit-tests/%.c,unit-tests/bin/%$(X),$(UNIT_TEST_SOURCES)) -UNIT_TESTS = $(sort $(filter-out unit-tests/bin/t-basic%,$(UNIT_TEST_PROGRAMS))) +UNIT_TESTS = $(sort $(UNIT_TEST_PROGRAMS)) # `test-chainlint` (which is a dependency of `test-lint`, `test` and `prove`) # checks all tests in all scripts via a single invocation, so tell individual diff --git a/t/unit-tests/t-basic.c b/t/helper/test-example-tap.c similarity index 95% rename from t/unit-tests/t-basic.c rename to t/helper/test-example-tap.c index fda1ae59a6..d072ad559f 100644 --- a/t/unit-tests/t-basic.c +++ b/t/helper/test-example-tap.c @@ -1,4 +1,5 @@ -#include "test-lib.h" +#include "test-tool.h" +#include "t/unit-tests/test-lib.h" /* * The purpose of this "unit test" is to verify a few invariants of the unit @@ -69,7 +70,7 @@ static void t_empty(void) ; /* empty */ } -int cmd_main(int argc, const char **argv) +int cmd__example_tap(int argc, const char **argv) { test_res = TEST(check_res = check_int(1, ==, 1), "passing test"); TEST(t_res(1), "passing test and assertion return 1"); diff --git a/t/helper/test-tool.c b/t/helper/test-tool.c index 33b9501c21..bb5c04c9c0 100644 --- a/t/helper/test-tool.c +++ b/t/helper/test-tool.c @@ -29,6 +29,7 @@ static struct test_cmd cmds[] = { { "dump-untracked-cache", cmd__dump_untracked_cache }, { "env-helper", cmd__env_helper }, { "example-decorate", cmd__example_decorate }, + { "example-tap", cmd__example_tap }, { "find-pack", cmd__find_pack }, { "fsmonitor-client", cmd__fsmonitor_client }, { "genrandom", cmd__genrandom }, diff --git a/t/helper/test-tool.h b/t/helper/test-tool.h index b72f07ded9..38001bd1c6 100644 --- a/t/helper/test-tool.h +++ b/t/helper/test-tool.h @@ -23,6 +23,7 @@ int cmd__dump_untracked_cache(int argc, const char **argv); int cmd__dump_reftable(int argc, const char **argv); int cmd__env_helper(int argc, const char **argv); int cmd__example_decorate(int argc, const char **argv); +int cmd__example_tap(int argc, const char **argv); int cmd__find_pack(int argc, const char **argv); int cmd__fsmonitor_client(int argc, const char **argv); int cmd__genrandom(int argc, const char **argv); diff --git a/t/t0080-unit-test-output.sh b/t/t0080-unit-test-output.sh index 961b54b06c..83b1e3b7f5 100755 --- a/t/t0080-unit-test-output.sh +++ b/t/t0080-unit-test-output.sh @@ -8,50 +8,50 @@ test_expect_success 'TAP output from unit tests' ' cat >expect <<-EOF && ok 1 - passing test ok 2 - passing test and assertion return 1 - # check "1 == 2" failed at t/unit-tests/t-basic.c:76 + # check "1 == 2" failed at t/helper/test-example-tap.c:77 # left: 1 # right: 2 not ok 3 - failing test ok 4 - failing test and assertion return 0 not ok 5 - passing TEST_TODO() # TODO ok 6 - passing TEST_TODO() returns 1 - # todo check ${SQ}check(x)${SQ} succeeded at t/unit-tests/t-basic.c:25 + # todo check ${SQ}check(x)${SQ} succeeded at t/helper/test-example-tap.c:26 not ok 7 - failing TEST_TODO() ok 8 - failing TEST_TODO() returns 0 - # check "0" failed at t/unit-tests/t-basic.c:30 + # check "0" failed at t/helper/test-example-tap.c:31 # skipping test - missing prerequisite - # skipping check ${SQ}1${SQ} at t/unit-tests/t-basic.c:32 + # skipping check ${SQ}1${SQ} at t/helper/test-example-tap.c:33 ok 9 - test_skip() # SKIP ok 10 - skipped test returns 1 # skipping test - missing prerequisite ok 11 - test_skip() inside TEST_TODO() # SKIP ok 12 - test_skip() inside TEST_TODO() returns 1 - # check "0" failed at t/unit-tests/t-basic.c:48 + # check "0" failed at t/helper/test-example-tap.c:49 not ok 13 - TEST_TODO() after failing check ok 14 - TEST_TODO() after failing check returns 0 - # check "0" failed at t/unit-tests/t-basic.c:56 + # check "0" failed at t/helper/test-example-tap.c:57 not ok 15 - failing check after TEST_TODO() ok 16 - failing check after TEST_TODO() returns 0 - # check "!strcmp("\thello\\\\", "there\"\n")" failed at t/unit-tests/t-basic.c:61 + # check "!strcmp("\thello\\\\", "there\"\n")" failed at t/helper/test-example-tap.c:62 # left: "\011hello\\\\" # right: "there\"\012" - # check "!strcmp("NULL", NULL)" failed at t/unit-tests/t-basic.c:62 + # check "!strcmp("NULL", NULL)" failed at t/helper/test-example-tap.c:63 # left: "NULL" # right: NULL - # check "${SQ}a${SQ} == ${SQ}\n${SQ}" failed at t/unit-tests/t-basic.c:63 + # check "${SQ}a${SQ} == ${SQ}\n${SQ}" failed at t/helper/test-example-tap.c:64 # left: ${SQ}a${SQ} # right: ${SQ}\012${SQ} - # check "${SQ}\\\\${SQ} == ${SQ}\\${SQ}${SQ}" failed at t/unit-tests/t-basic.c:64 + # check "${SQ}\\\\${SQ} == ${SQ}\\${SQ}${SQ}" failed at t/helper/test-example-tap.c:65 # left: ${SQ}\\\\${SQ} # right: ${SQ}\\${SQ}${SQ} not ok 17 - messages from failing string and char comparison - # BUG: test has no checks at t/unit-tests/t-basic.c:91 + # BUG: test has no checks at t/helper/test-example-tap.c:92 not ok 18 - test with no checks ok 19 - test with no checks returns 0 1..19 EOF - ! "$GIT_BUILD_DIR"/t/unit-tests/bin/t-basic >actual && + ! test-tool example-tap >actual && test_cmp expect actual ' From patchwork Tue Apr 30 19:55:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Steadmon X-Patchwork-Id: 13649942 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E5845199E9B for ; Tue, 30 Apr 2024 19:56:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714506966; cv=none; b=OypSvX3bJkJwFLKa40ikImn4PftLedkBJq4RU/+F4dhSzk4Vs6/XY4/mASSWCg+yFpX4M4TMm/GEGNPqhHihL94F9LFSOQnX+W2rpLwtZw9v65Tm/fNX3z+BasyKmNwdq8nyPcFv332DWE/yK7UyKmg8nD7UxnPunOqsOMy9Tv8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714506966; c=relaxed/simple; bh=kO/wmtigDcTbr38Vw3NacoxQON8NT+9T4h/D/Dlt2p0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=PGRo9BCBgznm8yyBvT6VxygIKbVQlLxYArGGeg0EBVaKUk9zfqFgm1c7k/90qmt+VZvVcnECvEaJYhDUG7DetFoslPf9MqtHgQI1zEaIXwwx5OS2UJ541J+JXVV2bjPS3bS0DHVsR03bIHZtbYXYlCDnZYEiB2OT4D5sJ/gQw8Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=0d6r5Trn; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="0d6r5Trn" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-61bea0c36bbso24071617b3.2 for ; Tue, 30 Apr 2024 12:56:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1714506964; x=1715111764; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=aiAxp2tsvVxK6B16seVg/SlNAzZ76SYFu2z5qJAzluM=; b=0d6r5Trn3lpy2L+wAB4i4tvsRkhDRKKtOZdm7V9sheaXAKCvBQpf5qd18v5mIQ4Dg2 BsNdlA6LKYsCiQL/EC2XhAD9w2blxZ88vzu4hA3gkVVitOBQadiEFBGFr+VFCy4dhhxl FyABxnz2Qrv/EuvpUUQ3aiGowo6VtXlQdxndOTuK4APUWhaEI6KixvR7FDJ9DGets1ii 9qfxcIQvEO27oE9iQQjs4LGo21qFkr+kTynjaXtFcmsSYvySn81aQMtsqjA3aJN6uhfU vJL9gDDqIoSHNvP6GD7ofziREuvm3wJzJggiZLqV7H+7rexbRSnVnPGWGoY5ex/mkei4 hzTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714506964; x=1715111764; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=aiAxp2tsvVxK6B16seVg/SlNAzZ76SYFu2z5qJAzluM=; b=kizV5T9yrIqNSsMUEZgfNGiDStsygdMfju05Pixl0VlpKGtguTHQkBF7KLDeOZAkVD k38iq8rSXe5yI/2ayFZIBONgfq2IF1L5WS8EtQs+zn5FJXYL4Z2acLLenzEgLUErDJwK hRkWgau+9DZ8waEZ0nNT76OI9n2vvkCmLxnflVOyG3fMxZ6dwlTX/yi2mOX0eHz34q31 KTDdvfICiboSGyk0NQTX9lnvfSCtxsRX6iTvZA8TNAsHZoRglxlbUpcXq9t/Qdg/1Oq2 ZK4xw9ARYpgc4ExZykc7VTWWUGpWqyOaQQsnlIK1Fz/AomY7Yae2gwzhMoBV4E+8UbUj tk6A== X-Gm-Message-State: AOJu0Yyh/VV8s8hzszCKQ1tseqOSG7E75p38HJS9FEDUucGgZX0arbyx uXub4GcWpcpkFc9VRbJLbPELwM6SmzBkm/LOm98iG/uNTVvUm1N1AHGLWVul2ZJN1hPWOY7u3ha mYB1whMpE6+d4tjR97OTWxIoAe1nmGcLyJ3KBlVt3uoFX3qW4Uv/AZTHYUHAEg0JfCzts1chGQv jzXJdXqDee84RM0igDvg6EsVTFsR6OSmCwYEvXeA4= X-Google-Smtp-Source: AGHT+IGDQ7Ipldhbq46Isxmiz/TkLyJbt2CfPSgYTygc99lSNd17b9FOBRPylDgHDcKdL29iOPuDUCCB5ModBQ== X-Received: from lunarfall.svl.corp.google.com ([2620:15c:2d3:204:3e48:fe18:8aa:8a0e]) (user=steadmon job=sendgmr) by 2002:a0d:d881:0:b0:61b:46e:62da with SMTP id a123-20020a0dd881000000b0061b046e62damr155348ywe.4.1714506963758; Tue, 30 Apr 2024 12:56:03 -0700 (PDT) Date: Tue, 30 Apr 2024 12:55:52 -0700 In-Reply-To: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.45.0.rc0.197.gbae5840b3b-goog Message-ID: <24f47f8fc7e931c2e15fa985635e3036ec1123ba.1714506612.git.steadmon@google.com> Subject: [PATCH v5 2/7] test-tool run-command testsuite: get shell from env From: Josh Steadmon To: git@vger.kernel.org Cc: gitster@pobox.com, peff@peff.net When running tests through `test-tool run-command testsuite`, we currently hardcode `sh` as the command interpreter. As discussed in [1], this is incorrect, and we should be using the shell set in TEST_SHELL_PATH instead. Add a shell_path field in struct testsuite so that we can pass this to the task runner callback. If this is non-null, we'll use it as the argv[0] of the subprocess. Otherwise, we'll just execute the test program directly. We will use this feature in a later commit to enable running binary executable unit tests. However, for now when setting up the struct testsuite in testsuite(), use the value of TEST_SHELL_PATH if it's set, otherwise keep the original behavior by defaulting to `sh`. [1] https://lore.kernel.org/git/20240123005913.GB835964@coredump.intra.peff.net/ Signed-off-by: Josh Steadmon --- t/helper/test-run-command.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/t/helper/test-run-command.c b/t/helper/test-run-command.c index c0ed8722c8..a41a54d9cb 100644 --- a/t/helper/test-run-command.c +++ b/t/helper/test-run-command.c @@ -65,6 +65,7 @@ struct testsuite { struct string_list tests, failed; int next; int quiet, immediate, verbose, verbose_log, trace, write_junit_xml; + const char *shell_path; }; #define TESTSUITE_INIT { \ .tests = STRING_LIST_INIT_DUP, \ @@ -80,7 +81,9 @@ static int next_test(struct child_process *cp, struct strbuf *err, void *cb, return 0; test = suite->tests.items[suite->next++].string; - strvec_pushl(&cp->args, "sh", test, NULL); + if (suite->shell_path) + strvec_push(&cp->args, suite->shell_path); + strvec_push(&cp->args, test); if (suite->quiet) strvec_push(&cp->args, "--quiet"); if (suite->immediate) @@ -162,6 +165,10 @@ static int testsuite(int argc, const char **argv) if (max_jobs <= 0) max_jobs = online_cpus(); + suite.shell_path = getenv("TEST_SHELL_PATH"); + if (!suite.shell_path) + suite.shell_path = "sh"; + dir = opendir("."); if (!dir) die("Could not open the current directory"); From patchwork Tue Apr 30 19:55:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Steadmon X-Patchwork-Id: 13649943 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D8E1C199E80 for ; Tue, 30 Apr 2024 19:56:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714506968; cv=none; b=Vft+QRO20QHn/QFcUfx75khHfjyaHOGOda6/m8DkkLdMgJky99t2vnLM5b1Fhh7tX+l453dGJqawsq/zxzVzsT8rrXpMK9Y+21CiMGCVc6fyQAYOVYfCZyjKdd3F6Z/dlsodS+XKfyRkp9DKICnA+t4iMBSakHX3OUG16lhwMbo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714506968; c=relaxed/simple; bh=Dkayb9ZP5cIqLevP7ASo1806VxgNJgzYjFEwFinrpM0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=hwSs0YbOlPdq5tFVl9OOOHf9JNm1AhzDYvPIWOxWd9pE8f2X1sjZXrKq7w4JDDhZkMuCweaMsjORdHyqmXWii+K6XQu+gz65Kq1U0NkLQxnw53bbCmcDE5ABJi7Fq5Wvbn34UAQzZ60bL9PHyZDrZz7LyazIoOIHnXVOZIprQDk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=H01RTWva; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="H01RTWva" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-61c9e368833so14423337b3.3 for ; Tue, 30 Apr 2024 12:56:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1714506966; x=1715111766; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=2Uh5H7S1W0wXkp3VWpeqZdyqR8uJl36OVVEcPSNaUr8=; b=H01RTWvahWyMQMY8SDhNK6JOg4xYnmE+L7IChZjooCCjjsTc/x5hwEHWMZk7eoTMGO a9RezYf8bMJxuLFTUF+r8ReT7Thd3KEC0MhEXBYHXbWdNa9QgQt4Fg2gIT9W2nlWDwUb E3sPDXu2+wV/Stv5HIxu3dDxjHDsQdpEUPUUP+O+/TDXCod+Ip8thk7fi89CSOP+amFG jODOmi4ppmDswa4C9j5A9YpSfqHwZsMi0uNtlP4Ll2djWeXbMUYnSZKSrg56YrGwmlvd RLNtCtHKgQC00RJg7SeQqITJpmpnJrF5Gj8ONgRuu8U1phBKJt8ZzQqlTciU8xrg+oo8 +hog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714506966; x=1715111766; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=2Uh5H7S1W0wXkp3VWpeqZdyqR8uJl36OVVEcPSNaUr8=; b=HIGhWTPK5pPBg0eTDlkPFFVk9venSP7FxOQzUDRywB77gh2xppHEZUju9ed4dtf70Y JC9pSm6HX+FsTzlCo5ZC3lpiCRg/LQ3YYgGecDccuhZWs7i2+VrsBAePJ3EmBPJR6Fmu FpGHuNth84eofPRYvvo7Zs0s9eSojPgPhSOzeHJ4lnYQ5DSHhEYPwYvr2osmNuzXi4Sy u+zS9bzupiNCBrn0DzMGeO9GkwvMixQ/jqpPZswFoO01ANZz/QjG6zXD21Vf9EGPvaFB JdW6f2d3Y4Sq1V3QhNL+B1Ohm2Vl4zb9XmQRgC2e3vNAy04erW9L2k5yRIOU1E624BX1 wXdw== X-Gm-Message-State: AOJu0YxKttgqUmtqxIC6UtKVRgn6t8d3pgwCl0sRBk+dv52uc0fjqWg+ /xZZUY2tmKKpABZM2vktow9cLJtNpHE/cT7o0b67Jaj0Z5PF/OlPQITvLy1iUMbwAC1HRo3eeXg R66GZEmTK3Qp25XYUzRatru8H0dCtmAcEooe0ZzGjbAx6h/NCsS69ydkU2+Wx6B4iuqjDuZP/8P jZxT5DZOB6QY6z3fmI9+atkcqcHE1pBpkI051l7KY= X-Google-Smtp-Source: AGHT+IGjihMafKgh7LdqnE2QEgNv0iGemcYHoXR7sHO/QT7MDzKxZB6Rho+g+r7gHlc6/D/Jo3izXOCzXhUb7w== X-Received: from lunarfall.svl.corp.google.com ([2620:15c:2d3:204:3e48:fe18:8aa:8a0e]) (user=steadmon job=sendgmr) by 2002:a0d:d614:0:b0:61c:89a4:dd5f with SMTP id y20-20020a0dd614000000b0061c89a4dd5fmr147573ywd.0.1714506965647; Tue, 30 Apr 2024 12:56:05 -0700 (PDT) Date: Tue, 30 Apr 2024 12:55:53 -0700 In-Reply-To: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.45.0.rc0.197.gbae5840b3b-goog Message-ID: <4a16a3ec241d83b3de424ce835db24eee6e46f0f.1714506612.git.steadmon@google.com> Subject: [PATCH v5 3/7] test-tool run-command testsuite: remove hardcoded filter From: Josh Steadmon To: git@vger.kernel.org Cc: gitster@pobox.com, peff@peff.net `test-tool run-command testsuite` currently assumes that it will only be running the shell test suite, and therefore filters out anything that does not match a hardcoded pattern of "t[0-9][0-9][0-9][0-9]-*.sh". Later in this series, we'll adapt `test-tool run-command testsuite` to also support unit tests, which do not follow the same naming conventions as the shell tests, so this hardcoded pattern is inconvenient. Since `testsuite` also allows specifying patterns on the command-line, let's just remove this pattern. As noted in [1], there are no longer any uses of `testsuite` in our codebase, it should be OK to break backwards compatibility in this case. We also add a new filter to avoid trying to execute "." and "..", so that users who wish to execute every test in a directory can do so without specifying a pattern. [1] https://lore.kernel.org/git/850ea42c-f103-68d5-896b-9120e2628686@gmx.de/ Signed-off-by: Josh Steadmon --- t/helper/test-run-command.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/t/helper/test-run-command.c b/t/helper/test-run-command.c index a41a54d9cb..e6bd792274 100644 --- a/t/helper/test-run-command.c +++ b/t/helper/test-run-command.c @@ -175,9 +175,7 @@ static int testsuite(int argc, const char **argv) while ((d = readdir(dir))) { const char *p = d->d_name; - if (*p != 't' || !isdigit(p[1]) || !isdigit(p[2]) || - !isdigit(p[3]) || !isdigit(p[4]) || p[5] != '-' || - !ends_with(p, ".sh")) + if (!strcmp(p, ".") || !strcmp(p, "..")) continue; /* No pattern: match all */ From patchwork Tue Apr 30 19:55:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Steadmon X-Patchwork-Id: 13649944 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BEB801A0AE0 for ; Tue, 30 Apr 2024 19:56:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714506970; cv=none; b=hcOwGw4lg6fD0GbKDZIQrn+AZUe2O80cKChE9hTlELiJK7siYree0zKXP9gN24RPv+z47s3Hbo4JbaaPq9clwFhkESafheQePIBmj7HLxF4cYRJgZxQLvHRvwgM9HEMmU3nNYfjGUUVdr6Hd/l8BhOpb9YPFS1Zw7sa8eYRZ4lA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714506970; c=relaxed/simple; bh=zW1F8Yn3LK7yPQYCg7Z67QZ8ZJwL5y07M6h8BUyP21g=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Xy/agtx9cWNT/C7gzJBm7PrYoBdcDJAQYShdE5IfAF8Z2V9e7uSzWcWKZco8ZJSX8jUaY6YzN9/jfDd4QeTSnwdQTJ2fdvLjrXNiaTUVKxa23ZAZieQKsxpJSZHVP/V6584DIB7+vih6dzPrBEPgzwHdiHR2Q61X5qfvpgnZIjM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=i8Cng3Hw; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="i8Cng3Hw" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-61be75e21fdso28624157b3.0 for ; Tue, 30 Apr 2024 12:56:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1714506968; x=1715111768; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=sVdZFFCh3rUFSbBBF11GDVhRhs+IoapsU8gQaQ02ono=; b=i8Cng3Hw582CNRWYMb67D5w/cgNCcy36qgE3dH7zCrRJEGOSFEmQAvcQYS1jgRgsl6 DjakLibCyzex0XC/vP2oRDXvSrXKY0lz6lp4Va1ZfDOh1V6rzBdahwA7aiSN0xCt3wEi YuOWj6pjKrJylnUTEqm9stZ8ABoAdo9zB8Z6pE/GiX2tJBvg/3Ybp+nO6PS7O3Ngj9Ec 6+7jTAKuqrzE3ZyEywFVQwlkR5F0J0y0nUVObzg05L/5JeUidQ7PgGNeINJevBEf9/hc f4TisGexOtSv77UfZxNe7DRff5fTUW09Bxt5ZiHao3PqX0D29Cu5buTiNedJx488ndWc 1uNg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714506968; x=1715111768; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=sVdZFFCh3rUFSbBBF11GDVhRhs+IoapsU8gQaQ02ono=; b=wSH8EL2giwE5B6OLdafh3D6pf62E97Gll/wRtCOXdEsgz+ZVClAs06EVbGNl473oYo JbB3eqnsYX42WCMdDXYX8qXQW7QPUFO5U0YnsmBNy2H0rapkViIo2YQh8nf+HFEjFzyH JFI/sMLsgS9v9oT6/v9TyEXajeHyXeDr31PXaMc1TBRXwDR68yDtxy7ZuSel+3pP+Mnt hZtHCPcTy6VTp4kOMcqet6PJNwIbA6L4MKPqDGqzPDeGihaRvnV+yTvWQ1RStzKF895N ogxUH5eEcAaUrCwPEiF5zzvSE4Qi940dSRnKG5MO6ftJufuAABiuv4u7ipmeaidjXk1m 6UoA== X-Gm-Message-State: AOJu0YzMBMDAxgSyb4BMc3qlHbt3kx8++zOrRxFFdC9Eopu8ijfqIZAK IzxAi03xEdLz9+YqYQqo7WKiXu/wILB+dsR1x62ga4hW7Gu6WqTupcedLyboMBKEJLx2GTP0M8T i9XH/px/Zm0fgdSwLpi6P/5DGMagm+JhGMLrvQLizJeTdlkgamKiIv3eoeTpmG5jqX72nhXdfO8 TuRZivXQW2YDVk1JKltxV6bPYJ+O+OU+NHsy6hFOg= X-Google-Smtp-Source: AGHT+IESQt8uP4YHFvlA7BZPuv0/iuKHIiHRhpSCnSk79YnbVThyyjOv+9j7yF0Ke3nLjc/w5nHWexzI2R69gQ== X-Received: from lunarfall.svl.corp.google.com ([2620:15c:2d3:204:3e48:fe18:8aa:8a0e]) (user=steadmon job=sendgmr) by 2002:a81:6d15:0:b0:617:d650:11e2 with SMTP id i21-20020a816d15000000b00617d65011e2mr150454ywc.3.1714506967621; Tue, 30 Apr 2024 12:56:07 -0700 (PDT) Date: Tue, 30 Apr 2024 12:55:54 -0700 In-Reply-To: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.45.0.rc0.197.gbae5840b3b-goog Message-ID: Subject: [PATCH v5 4/7] test-tool run-command testsuite: support unit tests From: Josh Steadmon To: git@vger.kernel.org Cc: gitster@pobox.com, peff@peff.net Teach the testsuite runner in `test-tool run-command testsuite` how to run unit tests: if TEST_SHELL_PATH is not set, run the programs directly from CWD, rather than defaulting to "sh" as an interpreter. With this change, you can now use test-tool to run the unit tests: $ make $ cd t/unit-tests/bin $ ../../helper/test-tool run-command testsuite This should be helpful on Windows to allow running tests without requiring Perl (for `prove`), as discussed in [1] and [2]. This again breaks backwards compatibility, as it is now required to set TEST_SHELL_PATH properly for executing shell scripts, but again, as noted in [2], there are no longer any such invocations in our codebase. [1] https://lore.kernel.org/git/nycvar.QRO.7.76.6.2109091323150.59@tvgsbejvaqbjf.bet/ [2] https://lore.kernel.org/git/850ea42c-f103-68d5-896b-9120e2628686@gmx.de/ Signed-off-by: Josh Steadmon --- t/helper/test-run-command.c | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/t/helper/test-run-command.c b/t/helper/test-run-command.c index e6bd792274..61eb1175fe 100644 --- a/t/helper/test-run-command.c +++ b/t/helper/test-run-command.c @@ -158,6 +158,8 @@ static int testsuite(int argc, const char **argv) .task_finished = test_finished, .data = &suite, }; + struct strbuf progpath = STRBUF_INIT; + size_t path_prefix_len; argc = parse_options(argc, argv, NULL, options, testsuite_usage, PARSE_OPT_STOP_AT_NON_OPTION); @@ -165,9 +167,13 @@ static int testsuite(int argc, const char **argv) if (max_jobs <= 0) max_jobs = online_cpus(); + /* + * If we run without a shell, execute the programs directly from CWD. + */ suite.shell_path = getenv("TEST_SHELL_PATH"); if (!suite.shell_path) - suite.shell_path = "sh"; + strbuf_addstr(&progpath, "./"); + path_prefix_len = progpath.len; dir = opendir("."); if (!dir) @@ -180,13 +186,17 @@ static int testsuite(int argc, const char **argv) /* No pattern: match all */ if (!argc) { - string_list_append(&suite.tests, p); + strbuf_setlen(&progpath, path_prefix_len); + strbuf_addstr(&progpath, p); + string_list_append(&suite.tests, progpath.buf); continue; } for (i = 0; i < argc; i++) if (!wildmatch(argv[i], p, 0)) { - string_list_append(&suite.tests, p); + strbuf_setlen(&progpath, path_prefix_len); + strbuf_addstr(&progpath, p); + string_list_append(&suite.tests, progpath.buf); break; } } @@ -213,6 +223,7 @@ static int testsuite(int argc, const char **argv) string_list_clear(&suite.tests, 0); string_list_clear(&suite.failed, 0); + strbuf_release(&progpath); return ret; } From patchwork Tue Apr 30 19:55:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Steadmon X-Patchwork-Id: 13649945 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A43711A0AF0 for ; Tue, 30 Apr 2024 19:56:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714506972; cv=none; b=u9KEG3vUukhk001wZf1dqANbO1cF1lc+DjVrdl3/2dueyGe7GvO6XqggPf2iYquUKPSnnNHDYaGm2ZNMv4eyklhTIJgLsYVH40REra8teOmYJR7SuQjnmkImJU/Glfepi33cpDf/ATyx6hYwcgH3OFHNdD5Hx2dxT73Kh8H9Keg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714506972; c=relaxed/simple; bh=Ly5MCCx0ouMmnG8mdHlwZ0G/gMHgbS0moyNwG8CmISc=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=FcdIJ9kkSyzsSGfjiq3hX21zzoPEP9ZJEUpnqZPTRJ9Xrr6hZ1OUg20tQONWfYVHCXF2sw3kg1aT5nOduWvzIAURSonPF4TzxDM3c/6bsd6QimuCM8vDzvTCANTg7Mm5N1+3qZXxqU9ghmr0xBdMs7aXMCjEGLnV8ROIipIlVys= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=M3eNlOmb; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="M3eNlOmb" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-61be452c62bso30410447b3.2 for ; Tue, 30 Apr 2024 12:56:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1714506969; x=1715111769; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=6kHhk9VTDawcXJXAQI7oPMaxgm0kzd4qGSjA+hqQk1s=; b=M3eNlOmbsuyITPTxKlbfUSJepIx19AoGBeOquPvz8QLUHDSw7eTFCY+JtSIuox4r+i 4KFu/3yPzHQEwb5OJrQjm7OkdHVLfb7t8wiRi/nGn25qUjVW2wFltF/ixBW1yKYE8oa9 37IX3gqpz7pScsxGxHmhgmKUFbvYk4hNbmd3q96Zu9Vtf3Ks5P24ucugEr6LpjHrOx9E pXvEKFDuju6wtPVfCp4UgXdxyLaPbNbOhm1xpPHCMNPZJe2sZAFWxGo4zDZRbqsYp6PN e+zjm8F1UcJVkPJXl9f8M+uWPNaCPI+FH6po0LX2D1IwXxb28eRq/wY589C25v10BnwO uk9w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714506969; x=1715111769; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=6kHhk9VTDawcXJXAQI7oPMaxgm0kzd4qGSjA+hqQk1s=; b=rBXR7hwel+6yAMg/jPW7+4Uzj5TPRupIf4oZ5CIZEHO31rq13erXtoVffiiI7eicMa byrUlDID4D5Hou25dk7+58V62p0aa8NG0TbVMY4JbRcek48NvfMnxfZy3JBKqMGxIif+ mOuw9QhUzcFZZS3S6gmgCQFMpRnBTrqgTrx3QxZpWKmlplJGGsPriekuxOLsc5IGblHZ oeYnEa46RJDRSF3cVRpx3eRspT2wAnLSBWvsm4ua914ZobWQGTNbyPmoldKjL+q+M+dB QMFEaMWmPFz/5FVfxu7so/bEoMwjp0xJ/I3H9lhEZgtuDgH/N3TPBl02ApALhdblUlr8 RalA== X-Gm-Message-State: AOJu0Yx24mFostSD8jOrRGBiUlKcvtKk47FhEtUQ9Uwmak6IN4qRD9Oy s8mQaeH6PlReR9a3CgK/W1lLMIj5Jz7cGZaA0IF1deurggr+cUaebs/e10ZRzej0fvG7IznC5FR aPisydD5pQX+0zJmXSE/Jw64/GltkFHS0X7MLxI6bWDi7yrE8EFUG/0VSmGEEIL+RI2Tspy2zDL Q3jbXBNLVfEvGw/QzLJt/Gn778aHP/dvZG8LPgsa0= X-Google-Smtp-Source: AGHT+IGHRjrgCe7A0ev49zbiJ4NHfj5mUe+Hb4X0Vj70H2MzCgsriyQknh82/8AfKWA+dH2fU/Io8NTMiQDlvQ== X-Received: from lunarfall.svl.corp.google.com ([2620:15c:2d3:204:3e48:fe18:8aa:8a0e]) (user=steadmon job=sendgmr) by 2002:a05:6902:1109:b0:dda:c566:dadd with SMTP id o9-20020a056902110900b00ddac566daddmr58652ybu.4.1714506969451; Tue, 30 Apr 2024 12:56:09 -0700 (PDT) Date: Tue, 30 Apr 2024 12:55:55 -0700 In-Reply-To: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.45.0.rc0.197.gbae5840b3b-goog Message-ID: Subject: [PATCH v5 5/7] unit tests: add rule for running with test-tool From: Josh Steadmon To: git@vger.kernel.org Cc: gitster@pobox.com, peff@peff.net In the previous commit, we added support in test-tool for running collections of unit tests. Now, add rules in t/Makefile for running in this way. This new rule can be executed from the top-level Makefile via `make DEFAULT_UNIT_TEST_TARGET=unit-tests-test-tool unit-tests`, or by setting DEFAULT_UNIT_TEST_TARGET in config.mak. Signed-off-by: Josh Steadmon --- Makefile | 2 +- t/Makefile | 10 +++++++++- 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/Makefile b/Makefile index ba55d817ee..b0d1f04b4d 100644 --- a/Makefile +++ b/Makefile @@ -3870,5 +3870,5 @@ $(UNIT_TEST_PROGS): $(UNIT_TEST_BIN)/%$X: $(UNIT_TEST_DIR)/%.o $(UNIT_TEST_DIR)/ .PHONY: build-unit-tests unit-tests build-unit-tests: $(UNIT_TEST_PROGS) -unit-tests: $(UNIT_TEST_PROGS) +unit-tests: $(UNIT_TEST_PROGS) t/helper/test-tool$X $(MAKE) -C t/ unit-tests diff --git a/t/Makefile b/t/Makefile index 4861edafe6..0ae04f1e42 100644 --- a/t/Makefile +++ b/t/Makefile @@ -49,6 +49,7 @@ CHAINLINT = '$(PERL_PATH_SQ)' chainlint.pl UNIT_TEST_SOURCES = $(wildcard unit-tests/t-*.c) UNIT_TEST_PROGRAMS = $(patsubst unit-tests/%.c,unit-tests/bin/%$(X),$(UNIT_TEST_SOURCES)) UNIT_TESTS = $(sort $(UNIT_TEST_PROGRAMS)) +UNIT_TESTS_NO_DIR = $(notdir $(UNIT_TESTS)) # `test-chainlint` (which is a dependency of `test-lint`, `test` and `prove`) # checks all tests in all scripts via a single invocation, so tell individual @@ -76,7 +77,7 @@ $(T): $(UNIT_TESTS): @echo "*** $@ ***"; $@ -.PHONY: unit-tests unit-tests-raw unit-tests-prove +.PHONY: unit-tests unit-tests-raw unit-tests-prove unit-tests-test-tool unit-tests: $(DEFAULT_UNIT_TEST_TARGET) unit-tests-raw: $(UNIT_TESTS) @@ -84,6 +85,13 @@ unit-tests-raw: $(UNIT_TESTS) unit-tests-prove: @echo "*** prove - unit tests ***"; $(PROVE) $(GIT_PROVE_OPTS) $(UNIT_TESTS) +unit-tests-test-tool: + @echo "*** test-tool - unit tests **" + ( \ + cd unit-tests/bin && \ + ../../helper/test-tool$X run-command testsuite $(UNIT_TESTS_NO_DIR)\ + ) + pre-clean: $(RM) -r '$(TEST_RESULTS_DIRECTORY_SQ)' From patchwork Tue Apr 30 19:55:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Steadmon X-Patchwork-Id: 13649946 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 861451A0AE0 for ; Tue, 30 Apr 2024 19:56:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714506974; cv=none; b=iL04IL/Y/NsqKSiqilkGv7Crt3eb0FZWl6HhQtlo8Jp5CDpo0ZIDFzrGCfbu+cDvT2hXQPTVJP5XXAK5O8ZuoL3bczmxfbdvvnXUO/pzWvZjRiVlOW8M204il9leO5hO16T3ny2lAJXik7n/lxzY93EOBSEbB5Q+bTtwY7J455s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714506974; c=relaxed/simple; bh=P7rqg+j7YmGKYDVvzRftuaqgay3cr3G/uSkeijPYLl0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=HXXdtOTGymXlUwPk2WpUehUqVjVtRFR19PUbWpVZ6YWEsoAh2p6c9ePbVI9zzqu+D+UAVHIQ1znFyaA3dL+jLK/waCzlzZqzc7ZSJ0LXpBWxxZ/4V8h1Pc6lbsxmjvWSRREJh8i21FidwUjwso5IasKeK901nDg6ANdP9d+imNc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=UqTRmXBo; arc=none smtp.client-ip=209.85.219.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="UqTRmXBo" Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-de604ccb373so3385186276.2 for ; Tue, 30 Apr 2024 12:56:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1714506971; x=1715111771; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=rceCE5CEDtiQThJgtF1K8/2u3J5Q/t1b5qFYCDmoU3s=; b=UqTRmXBo77MFU6/GLuXPnHs7VC87eBG/wbzEuxENvaHZTvp/BLw8DZ7rqiw4R9vWbv mrMRMjVSmLkPXvPAgF5fOW3eiGVIjmx3SHNZKfL4iAJZCVb8wrNOEo1FIl9ep9ULr7MO xYkc+/M+8a2AdOXvnkalynYzna4j4g34b305cdghiw3yWKGTsVuY+ScC/gpipfHJ/noE OIOADhSWZK7fNWVeFvxWWEqQN8IpCcVpRw43D8hW6psHOBlC2bszCkpzRzG6IKrDrROt jEDMXDbMphNrwGyZJtkkFP5i3EYwR0oLeXQ1KCbsSHaeU2BRfOWVeoZEV//yVfRIoM1o nQuA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714506971; x=1715111771; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=rceCE5CEDtiQThJgtF1K8/2u3J5Q/t1b5qFYCDmoU3s=; b=hS/fYeSVFAg9XpLfvCZxAE6yr3WvA7pP9RWLQIE46kdMEs8EJP4Eluim9i4NiY39K6 olVjwKp58jzQE/vEO+US9x9eVbqewGxdhGjYrSftQzILbANy3rIU4iqkvEMdsjpnHVCJ KK8oINUeHWGCtrlVYYZxitoafvTXsCxyEI9YASsUH6WQ6Vt2mFcylGwUkCV2BVmrlH1g qq1SscVlwYvrfx/08B0d+nOGVEeZ9v/Fkbt8IER0iI7M0KCpPPlrDV2VsDcC0i2Ru7A7 bTHC68EWR7uGlCdzjkFkbFmh6NfJpuNhVxZmNncMTTrl4vUWsLxYZhGN1vJzSD7Jl5dy jWQQ== X-Gm-Message-State: AOJu0YyUc0U3M31oGGw1N4ETAPi4L0SyTU2bFF2MA5A1WjPqFkmNOicX NlA5gUGPxjbJcQetHjDAMXGoNgKkJ0WyvoCoL1cg6UX4ZxT1YDCLe8fTS4BHy1rFDLHkFTXmhwA Qn6l0buKB5MO/70Ii2dJq8ORYl8S/fd0qZsvONxGdwHw224DcLwwkvWDqLJ8GuvEA43YrG1yfEZ OJQJWcOf2nZf07M8/HR5wATQLtpjt+E42x724BzZ0= X-Google-Smtp-Source: AGHT+IGFjE/rqv+tM1gAZgWuH4hSWya5fyB6JpaYtMGeTLVvEZgLh4jYuXKD9AcUyuwHnGOpM0o+xdgPlO3Pjg== X-Received: from lunarfall.svl.corp.google.com ([2620:15c:2d3:204:3e48:fe18:8aa:8a0e]) (user=steadmon job=sendgmr) by 2002:a25:aa90:0:b0:de4:654f:9ad0 with SMTP id t16-20020a25aa90000000b00de4654f9ad0mr90110ybi.6.1714506971419; Tue, 30 Apr 2024 12:56:11 -0700 (PDT) Date: Tue, 30 Apr 2024 12:55:56 -0700 In-Reply-To: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.45.0.rc0.197.gbae5840b3b-goog Message-ID: Subject: [PATCH v5 6/7] t/Makefile: run unit tests alongside shell tests From: Josh Steadmon To: git@vger.kernel.org Cc: gitster@pobox.com, peff@peff.net From: Jeff King Add a wrapper script to allow `prove` to run both shell tests and unit tests from a single invocation. This avoids issues around running prove twice in CI, as discussed in [1]. Additionally, this moves the unit tests into the main dev workflow, so that errors can be spotted more quickly. Accordingly, we remove the separate unit tests step for Linux CI. (We leave the Windows CI unit-test step as-is, because the sharding scheme there involves selecting specific test files rather than running `make test`.) [1] https://lore.kernel.org/git/pull.1613.git.1699894837844.gitgitgadget@gmail.com/ Signed-off-by: Jeff King Signed-off-by: Josh Steadmon --- ci/run-build-and-tests.sh | 2 -- t/Makefile | 2 +- t/run-test.sh | 18 ++++++++++++++++++ 3 files changed, 19 insertions(+), 3 deletions(-) create mode 100755 t/run-test.sh diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh index 7a1466b868..2528f25e31 100755 --- a/ci/run-build-and-tests.sh +++ b/ci/run-build-and-tests.sh @@ -50,8 +50,6 @@ if test -n "$run_tests" then group "Run tests" make test || handle_failed_tests - group "Run unit tests" \ - make DEFAULT_UNIT_TEST_TARGET=unit-tests-prove unit-tests fi check_unignored_build_artifacts diff --git a/t/Makefile b/t/Makefile index 0ae04f1e42..b2eb9f770b 100644 --- a/t/Makefile +++ b/t/Makefile @@ -68,7 +68,7 @@ failed: test -z "$$failed" || $(MAKE) $$failed prove: pre-clean check-chainlint $(TEST_LINT) - @echo "*** prove ***"; $(CHAINLINTSUPPRESS) $(PROVE) --exec '$(TEST_SHELL_PATH_SQ)' $(GIT_PROVE_OPTS) $(T) :: $(GIT_TEST_OPTS) + @echo "*** prove (shell & unit tests) ***"; $(CHAINLINTSUPPRESS) TEST_SHELL_PATH='$(TEST_SHELL_PATH_SQ)' $(PROVE) --exec ./run-test.sh $(GIT_PROVE_OPTS) $(T) $(UNIT_TESTS) :: $(GIT_TEST_OPTS) $(MAKE) clean-except-prove-cache $(T): diff --git a/t/run-test.sh b/t/run-test.sh new file mode 100755 index 0000000000..0eabf42d69 --- /dev/null +++ b/t/run-test.sh @@ -0,0 +1,18 @@ +#!/bin/sh + +# A simple wrapper to run shell tests via TEST_SHELL_PATH, +# or exec unit tests directly. + +case "$1" in +*.sh) + if test -z "${TEST_SHELL_PATH+set}" + then + echo >&2 "ERROR: TEST_SHELL_PATH is empty or not set" + exit 1 + fi + exec ${TEST_SHELL_PATH} "$@" + ;; +*) + exec "$@" + ;; +esac From patchwork Tue Apr 30 19:55:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Steadmon X-Patchwork-Id: 13649947 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 01556194C8D for ; Tue, 30 Apr 2024 19:56:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714506975; cv=none; b=cc/eEBuaARkAWZhAuAqnEEKR3KfTmZzQL33UebN7tSZiPdHQ0Qkt1sn8IUMJscXl+uyduHvGadPojmzTxvdFFdR9iZFm5YFeEc9wDxw78Y4aNwxfhsp6n079iS7HO8cNqQHNeIVh40OqQ4J019vdQWiEHKxzyI+ELOMn3MgjCGU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714506975; c=relaxed/simple; bh=4Wy+IXy/+spIZ5IiE0Dk97UBTU48bZKYO1RHsU1r11A=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=cFYrhm93+f+FDpwgb9zWatuSHpDMYRS267GOkSDbP6OmOemVobI5Atg53Sk3/BsawaoTWcSWiiDE6sajAv6wpMvp+GobUVCkPANInu+cFzp4k/2dPORC6y8+3PPpxwFpsN/UbOMvq75yUBw9T1ehO5l6PtkxFaUT2b/kjaETWgU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=hj6INLVN; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="hj6INLVN" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-61dfa4090c1so6645797b3.3 for ; Tue, 30 Apr 2024 12:56:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1714506973; x=1715111773; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=dy9kW74Osm5SsZZodkyT0+b5Ssxdgg0C36+JUhh7mJY=; b=hj6INLVNjMGumMDyJsMzU9wlLS3GRpQcHZxcYXBNKSxDxBUGGD8OPzKQ8261xdnqQ+ nnBz5IY7G27Hywyl41Co9iW0MQyj/yOUHaHEPT0jB5agx3TEdrp4QJsfJImSScxB1Z1n 12atwk28YDEBRTT3F5lQ/Qj91VFBzVOq8xlIvBrRuyXOsJV64KpJb1W20In8MKeg5FY8 Aj03+WMwyxA1bwicUML7c9tRX1XmVtRm40OvvcuBuuxqZx0oRMXyVJnVD5WSYyEL2LIp zrZrChQQ1cVVhQEYZIXlB0dP/nK/WMxFPsYRvxBIjHtGK6us+HTJUVlwcoAg4n9fy3wR KvRQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714506973; x=1715111773; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=dy9kW74Osm5SsZZodkyT0+b5Ssxdgg0C36+JUhh7mJY=; b=BnuJW0hsVwpq8ael+yqaJaYlTm6nDKtXFexBoZ8g1/Rk0eY0JvD/CBJQrr+FLoILhK n8GlVbl8MnnRjxw+DVUjL6bOcjiZ53Zz6JQlnDeuhJp9v1L7xrJaJxLS6qDy6wJYJNXn kvILFKf4wuZegbAtVSRv+ScW8IRkwfkowp6E7sp9+PFvYuZ0eTmqN0IbhQtUDuJO/AWv a+A1BE0p69U/Zz2pAAn+l4zMsF4yHDb3s2lpckmD0hSnrglDuYyOhrYBcXgHczOYZ/t5 8nqAUX5KrfPH+x2i1lbfiVrzo+p1PgOS9upgRwgP8mkjGC6G8sYZJbw3r1hHFpzPw0iv DtyA== X-Gm-Message-State: AOJu0YzBHdypkXyggAI2TEADTm+CK8DhMc5j1y9wYoulgE3ByTEsZj8Q vHoblsDLvLSBOmjiveb1oXtH56pJWIUZ6+unGQtcyMAHGk565E6Gpt9mm7EOLk0rEr0xMUWUlhX ZhO5XraxZHVgEKP/WsZqAynWicV4JioOQ/ifo3cAmv0oHs6rbrCwoN473tWTozPxbn6C7jANkNN jiUvaK1oDVHDQBfXK6old9PpSiUFu1vuNDG8O5AIM= X-Google-Smtp-Source: AGHT+IEfFCSbNfEcZ3KXWHXeBbzo2fePZeWBATV/rAGBS46MfKp0skUsdorNCTl/QILLLuGQF8iECvykX4AnZA== X-Received: from lunarfall.svl.corp.google.com ([2620:15c:2d3:204:3e48:fe18:8aa:8a0e]) (user=steadmon job=sendgmr) by 2002:a0d:ccd7:0:b0:61b:ec24:a014 with SMTP id o206-20020a0dccd7000000b0061bec24a014mr125674ywd.0.1714506973060; Tue, 30 Apr 2024 12:56:13 -0700 (PDT) Date: Tue, 30 Apr 2024 12:55:57 -0700 In-Reply-To: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.45.0.rc0.197.gbae5840b3b-goog Message-ID: <4a92131ab94b06958db2d2172344d427f0494b83.1714506612.git.steadmon@google.com> Subject: [PATCH v5 7/7] ci: use test-tool as unit test runner on Windows From: Josh Steadmon To: git@vger.kernel.org Cc: gitster@pobox.com, peff@peff.net Although the previous commit changed t/Makefile to run unit tests alongside shell tests, the Windows CI still needs a separate unit-tests step due to how the test sharding works. We want to avoid using `prove` as a test running on Windows due to performance issues [1], so use the new test-tool runner instead. [1] https://lore.kernel.org/git/850ea42c-f103-68d5-896b-9120e2628686@gmx.de/ Signed-off-by: Josh Steadmon --- ci/run-test-slice.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ci/run-test-slice.sh b/ci/run-test-slice.sh index ae8094382f..e167e646f7 100755 --- a/ci/run-test-slice.sh +++ b/ci/run-test-slice.sh @@ -17,7 +17,7 @@ handle_failed_tests # We only have one unit test at the moment, so run it in the first slice if [ "$1" == "0" ] ; then - group "Run unit tests" make --quiet -C t unit-tests-prove + group "Run unit tests" make --quiet -C t unit-tests-test-tool fi check_unignored_build_artifacts