| Message ID | 8b08b64a-aa5a-422d-9815-e67911a703e0@ramsayjones.plus.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | t/meson.build: fix up tests for 'seen' branch | expand |
On Sun, Oct 20, 2024 at 05:00:44PM +0100, Ramsay Jones wrote: Thanks, I'll pick these up. > Hmm, this is going to be a PITA as far as maintenance is concerned! :( > If I am reading it correctly, the cmake solution uses file globbing > to get the list of test files to run - could meson do the same? In theory we can, yes. But there's a big problem with it, both in Meson and in CMake: the instructions to deduce source files only get executed at configure time. Consequently, when new files get added, the build instructions do not get updated accordingly and are thus broken. So CMake does get around this, but not in a way that is feasible for use as our main build system, and the same would be true for Meson. For our integration-style tests I'd be okay with not listing the files individually, such that we instead use e.g. prove(1) to run all tests via a single test target. It would be a regression in functionality as we now cannot easily run e.g. "meson test t0000*", but at least we would not have to maintain the list of test scripts anymore. But for our source files I don't really see an alternative to listing them out explicitly. You don't want things like git-rebase(1) or git-bisect(1) to be broken just because one of the commits happens to add or remove a file. And this is true for whichever build system we want to adopt as additional official buildsystem, whether that is CMake or Meson. I guess we have less churn here anyway, and we would notice breakage more readily because things would stop compiling. So this is likely less of a problem compared to our tests. Patrick
On Mon, Oct 21, 2024 at 08:04:16AM +0200, Patrick Steinhardt wrote: > On Sun, Oct 20, 2024 at 05:00:44PM +0100, Ramsay Jones wrote: > > Thanks, I'll pick these up. Thanks. Please make sure to let me know which topic(s) that the new version of ps/build depends on as a result so I can queue it properly. > > Hmm, this is going to be a PITA as far as maintenance is concerned! :( > > If I am reading it correctly, the cmake solution uses file globbing > > to get the list of test files to run - could meson do the same? > > In theory we can, yes. But there's a big problem with it, both in Meson > and in CMake: the instructions to deduce source files only get executed > at configure time. Consequently, when new files get added, the build > instructions do not get updated accordingly and are thus broken. So > CMake does get around this, but not in a way that is feasible for use as > our main build system, and the same would be true for Meson. > > For our integration-style tests I'd be okay with not listing the files > individually, such that we instead use e.g. prove(1) to run all tests > via a single test target. It would be a regression in functionality as > we now cannot easily run e.g. "meson test t0000*", but at least we would > not have to maintain the list of test scripts anymore. I quite like the ability to run tNNNN-*.sh individually when spelled like that (i.e. including the test number, but not the full script name). I find that for the areas that I have worked on most over the years that I have the relevant test numbers committed by heart, but often don't know the full script name. So it is nice to be able to refer to them by number only. Am I reading it write that this suggests that Meson would lose that ability? Thanks, Taylor
On 10/21/24 5:14 PM, Taylor Blau wrote: > On Mon, Oct 21, 2024 at 08:04:16AM +0200, Patrick Steinhardt wrote: >> For our integration-style tests I'd be okay with not listing the files >> individually, such that we instead use e.g. prove(1) to run all tests >> via a single test target. It would be a regression in functionality as >> we now cannot easily run e.g. "meson test t0000*", but at least we would >> not have to maintain the list of test scripts anymore. > > I quite like the ability to run tNNNN-*.sh individually when spelled > like that (i.e. including the test number, but not the full script > name). > > I find that for the areas that I have worked on most over the years that > I have the relevant test numbers committed by heart, but often don't > know the full script name. So it is nice to be able to refer to them by > number only. > > Am I reading it write that this suggests that Meson would lose that > ability? If "meson test" simply ran the prove(1) utility then it would still be possible to do: ``` meson test --test-args "--args --to --prove" ``` This would not actually do what you want, since prove(1) doesn't implement "t0000*" style globbing, does it? So your reading is correct -- meson's own globbing support relies on meson knowing the list of all test names, which means you do have to maintain that list somehow (and have that list update when you do "git pull" and new tests materialize).
diff --git a/t/meson.build b/t/meson.build index 152fd7dc13..a956e6db75 100644 --- a/t/meson.build +++ b/t/meson.build @@ -43,6 +43,7 @@ unit_test_programs = [ 'unit-tests/t-reftable-block.c', 'unit-tests/t-reftable-merged.c', 'unit-tests/t-reftable-pq.c', + 'unit-tests/t-reftable-reader.c', 'unit-tests/t-reftable-readwrite.c', 'unit-tests/t-reftable-record.c', 'unit-tests/t-reftable-stack.c', @@ -166,6 +167,7 @@ integration_tests = [ 't1014-read-tree-confusing.sh', 't1015-read-index-unmerged.sh', 't1016-compatObjectFormat.sh', + 't1017-cat-file-remote-object-info.sh', 't1020-subdirectory.sh', 't1021-rerere-in-workdir.sh', 't1022-read-tree-partial-clone.sh', @@ -295,6 +297,7 @@ integration_tests = [ 't2405-worktree-submodule.sh', 't2406-worktree-repair.sh', 't2407-worktree-heads.sh', + 't2408-worktree-relative.sh', 't2500-untracked-overwriting.sh', 't2501-cwd-empty.sh', 't3000-ls-files-others.sh', @@ -716,6 +719,7 @@ integration_tests = [ 't5703-upload-pack-ref-in-want.sh', 't5704-protocol-violations.sh', 't5705-session-id-in-capabilities.sh', + 't5710-promisor-remote-capability.sh', 't5730-protocol-v2-bundle-uri-file.sh', 't5731-protocol-v2-bundle-uri-git.sh', 't5732-protocol-v2-bundle-uri-http.sh', @@ -818,6 +822,7 @@ integration_tests = [ 't6500-gc.sh', 't6501-freshen-objects.sh', 't6600-test-reach.sh', + 't6601-path-walk.sh', 't6700-tree-depth.sh', 't7001-mv.sh', 't7002-mv-sparse-checkout.sh', @@ -944,6 +949,8 @@ integration_tests = [ 't8012-blame-colors.sh', 't8013-blame-ignore-revs.sh', 't8014-blame-ignore-fuzzy.sh', + 't8015-blame-default-ignore-revs.sh', + 't8016-blame-override-ignore-revs.sh', 't9001-send-email.sh', 't9002-column.sh', 't9003-help-autocorrect.sh',
Signed-off-by: Ramsay Jones <ramsay@ramsayjones.plus.com> --- Hi Patrick, So, having got the build to work, I noticed that 'ninja test' was running less tests than 'make test': 1028 v. 1035. Having a fixed list of tests to run has 'issues'. :) This patch fixes up the list for the 'seen' branch. The 'new' tests in 'seen' are: t1017-cat-file-remote-object-info.sh branch: 'ej/cat-file-remote-object-info' commit: ef30c4557c ("cat-file: add remote-object-info to batch-command", 2024-09-25) t5710-promisor-remote-capability.sh branch: 'cc/promisor-remote-capability' commit: 3cb155a011 ("Add 'promisor-remote' capability to protocol v2", 2024-09-10) t6601-path-walk.sh branch: 'ds/path-walk' commit: d7e16a4ca9 ("t6601: add helper for testing path-walk API", 2024-10-08) t8015-blame-default-ignore-revs.sh branch: 'am/git-blame-ignore-revs-by-default' commit: eca5438740 ("blame: respect .git-blame-ignore-revs automatically", 2024-10-12) t8016-blame-override-ignore-revs.sh branch: 'am/git-blame-ignore-revs-by-default' commit: 0386bb36aa ("blame: introduce --override-ignore-revs to bypass ignore revisions list", 2024-10-12) A 'new' test in 'next': t2408-worktree-relative.sh branch: 'cw/worktree-relative' commit: 717af916cd ("worktree: link worktrees with relative paths", 2024-10-07) A missing test which (as you know ;) ) is in maint: unit-tests/t-reftable-reader.c commit: 0a148a8eda ("reftable/reader: make table iterator reseekable", 2024-09-16) Hmm, this is going to be a PITA as far as maintenance is concerned! :( If I am reading it correctly, the cmake solution uses file globbing to get the list of test files to run - could meson do the same? ATB, Ramsay Jones t/meson.build | 7 +++++++ 1 file changed, 7 insertions(+)