| Message ID | 7e8d435d58eea19d2aae0be366720f5956d29a5d.1712075189.git.me@ttaylorr.com (mailing list archive) |
|---|---|
| State | Accepted |
| Commit | b494b1ce39725d875e6a18e65fe3376dac67db19 |
| Headers | show |
| Series | t/t7700-repack.sh: fix test breakages with `GIT_TEST_MULTI_PACK_INDEX=1` | expand |
Taylor Blau <me@ttaylorr.com> writes: > There are a handful of related test breakages which are found when > running t/t7700-repack.sh with GIT_TEST_MULTI_PACK_INDEX set to "1" in > your environment. > > Both test failures are the result of something like: > > git repack --write-midx --write-bitmap-index [...] && > > test_path_is_file $midx && > test_path_is_file $midx-$(midx_checksum $objdir).bitmap > > , where we repack instructing Git to write a new MIDX and corresponding > MIDX bitamp. > > The error occurs when GIT_TEST_MULTI_PACK_INDEX=1 is found in the > enviornment. This causes Git to write out a second MIDX (after > processing the builtin's `--write-midx` argument) which is identical to > the first, but does not request a bitmap (since we did not set the > GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP variable in the environment). Doesn't it sound more like a bug, though? If a command line option requests something, should we still be honoring a contradicting instruction given by environment variable(s)? But anyway. > diff --git a/t/t7700-repack.sh b/t/t7700-repack.sh > index 94f9f4a1da..127efe99f8 100755 > --- a/t/t7700-repack.sh > +++ b/t/t7700-repack.sh > @@ -629,6 +629,7 @@ test_expect_success '--write-midx with preferred bitmap tips' ' > git log --format="create refs/tags/%s/%s %H" HEAD >refs && > git update-ref --stdin <refs && > > + GIT_TEST_MULTI_PACK_INDEX=0 \ > git repack --write-midx --write-bitmap-index && > test_path_is_file $midx && > test_path_is_file $midx-$(midx_checksum $objdir).bitmap && Is it a viable alternative approach to skip this check (and the other one) when GIT_TEST_MULTI_PACK_INDEX is set (i.e., lazy prereq). It will give us a better documentation value, e.g., test_lazy_prereq FORCED_MIDX ' # Features that are broken when GIT_TEST_* forces it # to enable are protexted with this prerequisite. test "$GIT_TEST_MULTI_PACK_INDEX" = 1; ' test_expect_success !FORCED_MIDX '--write-midx with ...' ' ... ' With a single comment, we can annotate any future tests that relies on features working correctly even with GIT_TEST_MULTI_PACK_INDEX.
On Tue, Apr 02, 2024 at 11:37:10AM -0700, Junio C Hamano wrote: > Taylor Blau <me@ttaylorr.com> writes: > > > There are a handful of related test breakages which are found when > > running t/t7700-repack.sh with GIT_TEST_MULTI_PACK_INDEX set to "1" in > > your environment. > > > > Both test failures are the result of something like: > > > > git repack --write-midx --write-bitmap-index [...] && > > > > test_path_is_file $midx && > > test_path_is_file $midx-$(midx_checksum $objdir).bitmap > > > > , where we repack instructing Git to write a new MIDX and corresponding > > MIDX bitamp. > > > > The error occurs when GIT_TEST_MULTI_PACK_INDEX=1 is found in the > > enviornment. This causes Git to write out a second MIDX (after > > processing the builtin's `--write-midx` argument) which is identical to > > the first, but does not request a bitmap (since we did not set the > > GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP variable in the environment). > > Doesn't it sound more like a bug, though? If a command line option > requests something, should we still be honoring a contradicting > instruction given by environment variable(s)? > > But anyway. I have generally considered the `--write-midx` and `GIT_TEST_MULTI_PACK_INDEX` options to be orthogonal to each other. The latter is a developer-oriented option that forces Git to write a MIDX post-repack regardless of the command-line option. It predates the `--write-midx` option by a number of years, and IIUC was introduced to enhance test coverage while the MIDX was being originally developed. I would argue that GIT_TEST_MULTI_PACK_INDEX should be on the list of GIT_TEST_-variables to get rid of as it has served its purpose. > > diff --git a/t/t7700-repack.sh b/t/t7700-repack.sh > > index 94f9f4a1da..127efe99f8 100755 > > --- a/t/t7700-repack.sh > > +++ b/t/t7700-repack.sh > > @@ -629,6 +629,7 @@ test_expect_success '--write-midx with preferred bitmap tips' ' > > git log --format="create refs/tags/%s/%s %H" HEAD >refs && > > git update-ref --stdin <refs && > > > > + GIT_TEST_MULTI_PACK_INDEX=0 \ > > git repack --write-midx --write-bitmap-index && > > test_path_is_file $midx && > > test_path_is_file $midx-$(midx_checksum $objdir).bitmap && > > Is it a viable alternative approach to skip this check (and the > other one) when GIT_TEST_MULTI_PACK_INDEX is set (i.e., lazy > prereq). It will give us a better documentation value, e.g., > > test_lazy_prereq FORCED_MIDX ' > # Features that are broken when GIT_TEST_* forces it > # to enable are protexted with this prerequisite. > test "$GIT_TEST_MULTI_PACK_INDEX" = 1; > ' > > test_expect_success !FORCED_MIDX '--write-midx with ...' ' > ... > ' > > With a single comment, we can annotate any future tests that relies > on features working correctly even with GIT_TEST_MULTI_PACK_INDEX. It's possible, but not something that I have seen done elsewhere for this or any of the other GIT_TEST- variables. Like I said, I'd like to get rid of this (and many other) GIT_TEST-related variables, but that is a larger effort than this single patch. Thanks, Taylor
Taylor Blau <me@ttaylorr.com> writes: > I would argue that GIT_TEST_MULTI_PACK_INDEX should be on the list of > GIT_TEST_-variables to get rid of as it has served its purpose. > > Like I said, I'd like to get rid of this (and many other) > GIT_TEST-related variables, but that is a larger effort than this single > patch. Yup, that sounds like a good longer-term goals. While it does not smell like it is consistent with that goal to add more instances of use of it to the test, it may inevitably be a "few steps backward in preparation to jump big forward", perhaps.
On Tue, Apr 02, 2024 at 12:55:39PM -0700, Junio C Hamano wrote: > Taylor Blau <me@ttaylorr.com> writes: > > > I would argue that GIT_TEST_MULTI_PACK_INDEX should be on the list of > > GIT_TEST_-variables to get rid of as it has served its purpose. > > > > Like I said, I'd like to get rid of this (and many other) > > GIT_TEST-related variables, but that is a larger effort than this single > > patch. > > Yup, that sounds like a good longer-term goals. While it does not > smell like it is consistent with that goal to add more instances of > use of it to the test, it may inevitably be a "few steps backward in > preparation to jump big forward", perhaps. Yep, exactly. Thanks, Taylor
On Tue, Apr 02, 2024 at 02:45:28PM -0400, Taylor Blau wrote: > I have generally considered the `--write-midx` and > `GIT_TEST_MULTI_PACK_INDEX` options to be orthogonal to each other. The > latter is a developer-oriented option that forces Git to write a MIDX > post-repack regardless of the command-line option. > > It predates the `--write-midx` option by a number of years, and IIUC was > introduced to enhance test coverage while the MIDX was being originally > developed. > > I would argue that GIT_TEST_MULTI_PACK_INDEX should be on the list of > GIT_TEST_-variables to get rid of as it has served its purpose. Hmm. Obviously it is of little value in this explicit --write-midx test, but I thought the main value was just exercising all of the _other_ tests with a midx in place. Doesn't that potentially have value (just like testing with SPLIT_INDEX, etc, gets more coverage)? If it is worth keeping (and I do not really have a strong opinion there), the real issue seems to me that it does not behave like --write-midx. That is the source of the problem here, but also makes test runs with it unrealistic, since the command-line option is how real-world users would trigger it. I.e., I would have expected something like this, so that the variables takes precedence over config but under command-line options: diff --git a/builtin/repack.c b/builtin/repack.c index 15e4cccc45..4b02d9cb77 100644 --- a/builtin/repack.c +++ b/builtin/repack.c @@ -1197,6 +1197,11 @@ int cmd_repack(int argc, const char **argv, const char *prefix) git_config(repack_config, &cruft_po_args); + write_midx = git_env_bool(GIT_TEST_MULTI_PACK_INDEX, write_midx); + if (write_midx) + write_bitmaps = git_env_bool(GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP, + write_bitmaps); + argc = parse_options(argc, argv, prefix, builtin_repack_options, git_repack_usage, 0); @@ -1214,10 +1219,6 @@ int cmd_repack(int argc, const char **argv, const char *prefix) if (!write_midx && (!(pack_everything & ALL_INTO_ONE) || !is_bare_repository())) write_bitmaps = 0; - } else if (write_bitmaps && - git_env_bool(GIT_TEST_MULTI_PACK_INDEX, 0) && - git_env_bool(GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP, 0)) { - write_bitmaps = 0; } if (pack_kept_objects < 0) pack_kept_objects = write_bitmaps > 0 && !write_midx; @@ -1515,13 +1516,6 @@ int cmd_repack(int argc, const char **argv, const char *prefix) if (run_update_server_info) update_server_info(0); - if (git_env_bool(GIT_TEST_MULTI_PACK_INDEX, 0)) { - unsigned flags = 0; - if (git_env_bool(GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP, 0)) - flags |= MIDX_WRITE_BITMAP | MIDX_WRITE_REV_INDEX; - write_midx_file(get_object_directory(), NULL, NULL, flags); - } - cleanup: string_list_clear(&names, 1); existing_packs_release(&existing); But it gets weird because some tests (like t7700.2) explicitly ask for bitmaps on the command line and want pack bitmaps but _not_ midx bitmaps. So I dunno. Maybe this is a can of worms that is not worth falling into. After all, these are not "real" environment variables that we expect users to use. I just wonder if the ci runs with them are buying us anything for all of the tests outside of t7700. -Peff
Jeff King <peff@peff.net> writes: > So I dunno. Maybe this is a can of worms that is not worth falling into. > After all, these are not "real" environment variables that we expect > users to use. I just wonder if the ci runs with them are buying us > anything for all of the tests outside of t7700. Yeah, I agree with the general direction of reducing the role GIT_TEST_* environment variables take.
diff --git a/t/t7700-repack.sh b/t/t7700-repack.sh index 94f9f4a1da..127efe99f8 100755 --- a/t/t7700-repack.sh +++ b/t/t7700-repack.sh @@ -629,6 +629,7 @@ test_expect_success '--write-midx with preferred bitmap tips' ' git log --format="create refs/tags/%s/%s %H" HEAD >refs && git update-ref --stdin <refs && + GIT_TEST_MULTI_PACK_INDEX=0 \ git repack --write-midx --write-bitmap-index && test_path_is_file $midx && test_path_is_file $midx-$(midx_checksum $objdir).bitmap && @@ -749,6 +750,7 @@ test_expect_success '--write-midx with --pack-kept-objects' ' keep="$objdir/pack/pack-$one.keep" && touch "$keep" && + GIT_TEST_MULTI_PACK_INDEX=0 \ git repack --write-midx --write-bitmap-index --geometric=2 -d \ --pack-kept-objects &&
There are a handful of related test breakages which are found when running t/t7700-repack.sh with GIT_TEST_MULTI_PACK_INDEX set to "1" in your environment. Both test failures are the result of something like: git repack --write-midx --write-bitmap-index [...] && test_path_is_file $midx && test_path_is_file $midx-$(midx_checksum $objdir).bitmap , where we repack instructing Git to write a new MIDX and corresponding MIDX bitamp. The error occurs when GIT_TEST_MULTI_PACK_INDEX=1 is found in the enviornment. This causes Git to write out a second MIDX (after processing the builtin's `--write-midx` argument) which is identical to the first, but does not request a bitmap (since we did not set the GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP variable in the environment). Since c528e179662 (pack-bitmap: write multi-pack bitmaps, 2021-08-31), the MIDX machinery will drop an existing MIDX bitmap when rewriting an identical MIDX which does not itself request a corresponding bitmap, which is similar to the way repack itself behaves in the pack-bitmap case. Correct these issues (which date back to [1] and [2], respectively) by explicitly setting GIT_TEST_MULTI_PACK_INDEX to zero before running each command. In the future, we should consider removing GIT_TEST_MULTI_PACK_INDEX, and in general clean up unused GIT_TEST_-variables. But that is a larger effort, and this ensures that we can cleanly run: $ GIT_TEST_MULTI_PACK_INDEX=1 make test in the meantime. [1]: 324efc90d1b (builtin/repack.c: pass `--refs-snapshot` when writing bitmaps, 2021-10-01) [2]: 197443e80ab (repack: don't remove .keep packs with `--pack-kept-objects`, 2022-10-17). Signed-off-by: Taylor Blau <me@ttaylorr.com> --- t/t7700-repack.sh | 2 ++ 1 file changed, 2 insertions(+)