| Message ID | cover.1714506612.git.steadmon@google.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | test-tool: add unit test suite runner | expand |
Josh Steadmon <steadmon@google.com> writes: > For various reasons (see discussion at [1]) we would like an alternative > to `prove` for running test suites (including the unit tests) on > Windows. > > [1] https://lore.kernel.org/git/pull.1613.git.1699894837844.gitgitgadget@gmail.com/ > > This series extends the existing `test-tool run-command testsuite` to > support running unit tests. In addition, it includes some small > cleanups: > * move t-basic out of the unit-tests directory > * don't hardcode the shell for running tests in `test-tool ... testsuite` > * don't hardcode a test name filter in `test-tool ... testsuite` > * add a test wrapper script to allow unit tests and the shell test suite > to run in a single `prove` process I am OK to see it outside the scope of this series, but we would need unit tests supported by the GIT_SKIP_TESTS mechanism (or an alternative mechanism written), given that I hear "migrate tests to unit-test framework" every once in a while, which means we would accumlate more and more tests that ignore GIT_SKIP_TESTS mechansim. I did spot one potential problem (rather, "we'd want to fix it as we are changing it with this reroll anyway"), but other than that I did not see anything wrong in the other patches. Thanks. Will queue.
On 2024.04.30 14:15, Junio C Hamano wrote: > Josh Steadmon <steadmon@google.com> writes: > > > For various reasons (see discussion at [1]) we would like an alternative > > to `prove` for running test suites (including the unit tests) on > > Windows. > > > > [1] https://lore.kernel.org/git/pull.1613.git.1699894837844.gitgitgadget@gmail.com/ > > > > This series extends the existing `test-tool run-command testsuite` to > > support running unit tests. In addition, it includes some small > > cleanups: > > * move t-basic out of the unit-tests directory > > * don't hardcode the shell for running tests in `test-tool ... testsuite` > > * don't hardcode a test name filter in `test-tool ... testsuite` > > * add a test wrapper script to allow unit tests and the shell test suite > > to run in a single `prove` process > > I am OK to see it outside the scope of this series, but we would > need unit tests supported by the GIT_SKIP_TESTS mechanism (or an > alternative mechanism written), given that I hear "migrate tests to > unit-test framework" every once in a while, which means we would > accumlate more and more tests that ignore GIT_SKIP_TESTS mechansim. > > I did spot one potential problem (rather, "we'd want to fix it as we > are changing it with this reroll anyway"), but other than that I did > not see anything wrong in the other patches. > > Thanks. Will queue. Ack. I think the current GIT_SKIP_TESTS system should be fine. I'll put it on my TODO but can't promise that it will be a priority yet.
For various reasons (see discussion at [1]) we would like an alternative to `prove` for running test suites (including the unit tests) on Windows. [1] https://lore.kernel.org/git/pull.1613.git.1699894837844.gitgitgadget@gmail.com/ This series extends the existing `test-tool run-command testsuite` to support running unit tests. In addition, it includes some small cleanups: * move t-basic out of the unit-tests directory * don't hardcode the shell for running tests in `test-tool ... testsuite` * don't hardcode a test name filter in `test-tool ... testsuite` * add a test wrapper script to allow unit tests and the shell test suite to run in a single `prove` process Changes in V5: * Style fix in t/run-test.sh Changes in V4: * Set TEST_SHELL_PATH when running the `prove` test target, and error out of run-test.sh if TEST_SHELL_PATH is not set. Changes in V3: * Added new patch (#7) to use the new test-tool runner for unit tests in Windows CI. * Restored the explicit sort call in t/Makefile, for backwards compatibility with older GNU Make versions. * Rebased the series on the latest jk/unit-tests-buildfix branch. * Fixed header include order in patch #1. * Removed a paragraph in patch #1's commit message that is obsolete now that we're building the list of test files from the sources rather than by globbing. * Added a note in patch #2 that setting a NULL suite.shell_path will be used in a later commit. * Clarified up some sloppy wording in commit messages and comments in t/helper/test-run-command.c. Changes in V2: * Rebased the series on the latest jk/unit-tests-buildfix branch. * Patch 1: move t-basic to a test-tool subcommand rather than a new executable under t/t0080/ * New patch 2: get the shell path from TEST_SHELL_PATH in `test-tool run-command testsuite` * New patch 3: remove the hardcoded filename filter in `test-tool run-command testsuite` * Patch 4 (previously 2): simplified now that we no longer need to add any command-line flags to support unit tests * Patch 5 (previously 3): avoid trying to run cmake *.pdb files by using the unit test list built in the makefile in jk/unit-tests-buildfix. Jeff King (1): t/Makefile: run unit tests alongside shell tests Josh Steadmon (6): t0080: turn t-basic unit test into a helper test-tool run-command testsuite: get shell from env test-tool run-command testsuite: remove hardcoded filter test-tool run-command testsuite: support unit tests unit tests: add rule for running with test-tool ci: use test-tool as unit test runner on Windows Makefile | 6 ++-- ci/run-build-and-tests.sh | 2 -- ci/run-test-slice.sh | 2 +- t/Makefile | 14 ++++++++-- .../t-basic.c => helper/test-example-tap.c} | 5 ++-- t/helper/test-run-command.c | 28 +++++++++++++++---- t/helper/test-tool.c | 1 + t/helper/test-tool.h | 1 + t/run-test.sh | 18 ++++++++++++ t/t0080-unit-test-output.sh | 24 ++++++++-------- 10 files changed, 72 insertions(+), 29 deletions(-) rename t/{unit-tests/t-basic.c => helper/test-example-tap.c} (95%) create mode 100755 t/run-test.sh Range-diff against v4: 1: 6777451100 = 1: 6777451100 t0080: turn t-basic unit test into a helper 2: 24f47f8fc7 = 2: 24f47f8fc7 test-tool run-command testsuite: get shell from env 3: 4a16a3ec24 = 3: 4a16a3ec24 test-tool run-command testsuite: remove hardcoded filter 4: abc9a7afe8 = 4: abc9a7afe8 test-tool run-command testsuite: support unit tests 5: a8bbff2c6b = 5: a8bbff2c6b unit tests: add rule for running with test-tool 6: 0e32de1afe ! 6: c6606446c4 t/Makefile: run unit tests alongside shell tests @@ t/run-test.sh (new) + +case "$1" in +*.sh) -+ if test -z "${TEST_SHELL_PATH+set}" ; then -+ echo "ERROR: TEST_SHELL_PATH is not set" >&2 ++ if test -z "${TEST_SHELL_PATH+set}" ++ then ++ echo >&2 "ERROR: TEST_SHELL_PATH is empty or not set" + exit 1 + fi + exec ${TEST_SHELL_PATH} "$@" 7: c562515293 = 7: 4a92131ab9 ci: use test-tool as unit test runner on Windows base-commit: 4904a4d08cc085716df12ce713ae7ee3d5ecb75a