diff mbox series

[v2,4/4] commit-graph: fix generation number v2 overflow values

Message ID 193217c71e0aaf3f56a02d9abec6753bd19aba71.1646056423.git.gitgitgadget@gmail.com (mailing list archive)
State Superseded
Headers show
Series Commit-graph: Generation Number v2 Fixes | expand

Commit Message

Derrick Stolee Feb. 28, 2022, 1:53 p.m. UTC 
From: Derrick Stolee <derrickstolee@github.com>

The Generation Data Chunk was implemented and tested in e8b63005c
(commit-graph: implement generation data chunk, 2021-01-16), but the
test was carefully constructed to work on systems with 32-bit dates.
Since the corrected commit date offsets still required more than 31
bits, this triggered writing the generation_data_overflow chunk.

However, upon closer look, the
write_graph_chunk_generation_data_overflow() method writes the offsets
to the chunk (as dictated by the format) but fill_commit_graph_info()
treats the value in the chunk as if it is the full corrected commit date
(not an offset). For some reason, this does not cause an issue when
using the FUTURE_DATE specified in t5318-commit-graph.sh, but it does
show up as a failure in 'git commit-graph verify' if we increase that
FUTURE_DATE to be above four billion.

Fix this error and update the test to require 64-bit dates so we can
safely use this large value in our test.

Signed-off-by: Derrick Stolee <derrickstolee@github.com>
---
 commit-graph.c          |  2 +-
 t/t5318-commit-graph.sh | 17 +++++++++++++++--
 2 files changed, 16 insertions(+), 3 deletions(-)

Comments

Ævar Arnfjörð Bjarmason Feb. 28, 2022, 3:40 p.m. UTC | #1 
On Mon, Feb 28 2022, Derrick Stolee via GitGitGadget wrote:

> From: Derrick Stolee <derrickstolee@github.com>
>
> The Generation Data Chunk was implemented and tested in e8b63005c
> (commit-graph: implement generation data chunk, 2021-01-16), but the
> test was carefully constructed to work on systems with 32-bit dates.
> Since the corrected commit date offsets still required more than 31
> bits, this triggered writing the generation_data_overflow chunk.
>
> However, upon closer look, the
> write_graph_chunk_generation_data_overflow() method writes the offsets
> to the chunk (as dictated by the format) but fill_commit_graph_info()
> treats the value in the chunk as if it is the full corrected commit date
> (not an offset). For some reason, this does not cause an issue when
> using the FUTURE_DATE specified in t5318-commit-graph.sh, but it does
> show up as a failure in 'git commit-graph verify' if we increase that
> FUTURE_DATE to be above four billion.
>
> Fix this error and update the test to require 64-bit dates so we can
> safely use this large value in our test.

Hrm, so perhaps re my comment on 2/4 @2147483646 was never used? I'm not
sure I understand this.

> diff --git a/commit-graph.c b/commit-graph.c
> index 8e52bb09552..b86a6a634fe 100644
> --- a/commit-graph.c
> +++ b/commit-graph.c
> @@ -806,7 +806,7 @@ static void fill_commit_graph_info(struct commit *item, struct commit_graph *g,
>  				die(_("commit-graph requires overflow generation data but has none"));
>  
>  			offset_pos = offset ^ CORRECTED_COMMIT_DATE_OFFSET_OVERFLOW;
> -			graph_data->generation = get_be64(g->chunk_generation_data_overflow + 8 * offset_pos);
> +			graph_data->generation = item->date + get_be64(g->chunk_generation_data_overflow + 8 * offset_pos);
>  		} else
>  			graph_data->generation = item->date + offset;
>  	} else
> diff --git a/t/t5318-commit-graph.sh b/t/t5318-commit-graph.sh
> index 1afee1c2705..f4ffaad661d 100755
> --- a/t/t5318-commit-graph.sh
> +++ b/t/t5318-commit-graph.sh
> @@ -815,6 +815,15 @@ test_expect_success 'corrupt commit-graph write (missing tree)' '
>  	)
>  '

This goes back to my comment on 3/4 but:

> +# The remaining tests check timestamps that flow over
> +# 32-bits. The graph_git_behavior checks can't take a
> +# prereq, so just stop here if we are on a 32-bit machine.
> +
> +if ! test_have_prereq TIME_IS_64BIT || ! test_have_prereq TIME_T_IS_64BIT
> +then
> +	test_done
> +fi
> +

This...(continued)...

>  # We test the overflow-related code with the following repo history:
>  #
>  #               4:F - 5:N - 6:U
> @@ -832,10 +841,10 @@ test_expect_success 'corrupt commit-graph write (missing tree)' '
>  # The largest offset observed is 2 ^ 31, just large enough to overflow.
>  #
>  
> -test_expect_success 'set up and verify repo with generation data overflow chunk' '
> +test_expect_success TIME_IS_64BIT,TIME_T_IS_64BIT 'set up and verify repo with generation data overflow chunk' '
>  	objdir=".git/objects" &&
>  	UNIX_EPOCH_ZERO="@0 +0000" &&
> -	FUTURE_DATE="@2147483646 +0000" &&
> +	FUTURE_DATE="@4000000000 +0000" &&

Hrm, again this may be over my head, but @4147483646 instead of
@2147483646 in the other test, but @4000000000 instead of the same here?


>  	test_oid_cache <<-EOF &&
>  	oid_version sha1:1
>  	oid_version sha256:2
> @@ -867,4 +876,8 @@ test_expect_success 'set up and verify repo with generation data overflow chunk'
>  
>  graph_git_behavior 'generation data overflow chunk repo' repo left right
>  
> +# Do not add tests at the end of this file, unless they require 64-bit
> +# timestamps, since this portion of the script is only executed when
> +# time data types have 64 bits.
> +
>  test_done

...and this would really be much nicer if we split this test up into its
own file, which would be obviously named to note the
issue. tXXXX-commit-graph-64bit-timestamp.sh or something.

As shown in my recent 0a2bfccb9c8 (t0051: use "skip_all" under !MINGW in
single-test file, 2022-02-04) you'll also get much nicer output from
"prove" in that case.
diff mbox series

Patch

diff --git a/commit-graph.c b/commit-graph.c
index 8e52bb09552..b86a6a634fe 100644
--- a/commit-graph.c
+++ b/commit-graph.c
@@ -806,7 +806,7 @@  static void fill_commit_graph_info(struct commit *item, struct commit_graph *g,
 				die(_("commit-graph requires overflow generation data but has none"));
 
 			offset_pos = offset ^ CORRECTED_COMMIT_DATE_OFFSET_OVERFLOW;
-			graph_data->generation = get_be64(g->chunk_generation_data_overflow + 8 * offset_pos);
+			graph_data->generation = item->date + get_be64(g->chunk_generation_data_overflow + 8 * offset_pos);
 		} else
 			graph_data->generation = item->date + offset;
 	} else
diff --git a/t/t5318-commit-graph.sh b/t/t5318-commit-graph.sh
index 1afee1c2705..f4ffaad661d 100755
--- a/t/t5318-commit-graph.sh
+++ b/t/t5318-commit-graph.sh
@@ -815,6 +815,15 @@  test_expect_success 'corrupt commit-graph write (missing tree)' '
 	)
 '
 
+# The remaining tests check timestamps that flow over
+# 32-bits. The graph_git_behavior checks can't take a
+# prereq, so just stop here if we are on a 32-bit machine.
+
+if ! test_have_prereq TIME_IS_64BIT || ! test_have_prereq TIME_T_IS_64BIT
+then
+	test_done
+fi
+
 # We test the overflow-related code with the following repo history:
 #
 #               4:F - 5:N - 6:U
@@ -832,10 +841,10 @@  test_expect_success 'corrupt commit-graph write (missing tree)' '
 # The largest offset observed is 2 ^ 31, just large enough to overflow.
 #
 
-test_expect_success 'set up and verify repo with generation data overflow chunk' '
+test_expect_success TIME_IS_64BIT,TIME_T_IS_64BIT 'set up and verify repo with generation data overflow chunk' '
 	objdir=".git/objects" &&
 	UNIX_EPOCH_ZERO="@0 +0000" &&
-	FUTURE_DATE="@2147483646 +0000" &&
+	FUTURE_DATE="@4000000000 +0000" &&
 	test_oid_cache <<-EOF &&
 	oid_version sha1:1
 	oid_version sha256:2
@@ -867,4 +876,8 @@  test_expect_success 'set up and verify repo with generation data overflow chunk'
 
 graph_git_behavior 'generation data overflow chunk repo' repo left right
 
+# Do not add tests at the end of this file, unless they require 64-bit
+# timestamps, since this portion of the script is only executed when
+# time data types have 64 bits.
+
 test_done