From patchwork Tue Mar 1 19:48:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12765123 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EFA99C433EF for ; Tue, 1 Mar 2022 19:50:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237530AbiCATuu (ORCPT ); Tue, 1 Mar 2022 14:50:50 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59926 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237514AbiCATuT (ORCPT ); Tue, 1 Mar 2022 14:50:19 -0500 Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com [IPv6:2a00:1450:4864:20::32e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6E3E66D94C for ; Tue, 1 Mar 2022 11:49:01 -0800 (PST) Received: by mail-wm1-x32e.google.com with SMTP id bg31-20020a05600c3c9f00b00381590dbb33so2044507wmb.3 for ; Tue, 01 Mar 2022 11:49:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=D/L9C6JWe72f9TjEtYZgbN/FMUA9CuRmpC7e95qlm4o=; b=lkgUG4ewOuOaTvnpfSt/lKyPzprCeCpbwKF8/fW3J5OOfm9H0WuOLTisohuNolFkrF EX7GOvCRtEXs2JfOV87z/OnY+FU4l8eB5cuR0x0LEDYCFUczf/SzUAR3oSq4mZhYRKmX gSSsQbkZwgXN3w1sDTFe47OOPdBRtEyPcbMxZeMz1m5qjvXR6Z1vb1/F0m8Knw04mPoV jUIIoddmpOotb9ZyKQTzEkZk1sS0SzJMgR15hekIVC8MixXBW01N++379pyIb2O2fBiL L499NR5W9+kNfwP8l/JUhS4bDXqimaBhM0/gbMFXLiukuSg7lLwDU9D3On0J7ii+dGZ9 hqTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=D/L9C6JWe72f9TjEtYZgbN/FMUA9CuRmpC7e95qlm4o=; b=Sovbo/S6grkdmnBc10J2hUas4CE7MGoXWaJd8QFfXTkDeg4Ysyn3wFUfDZJJm5YQeg 3C+DhxFNb5BiS+wQ/gAkkBHShjZOQibsQm3//hr+LKd9+pDqEOSqY/YnPeYoU2Df6xzC 7SLptelzxeGypwmmAHxp1k/NQk/kXPor7+swBazwMP2HlCPbdEDKjk74WhLtE/k6/4gQ 4xMChz7q3v53I4USh6Db8OU4957Ke5ylTNfiqk1XefGn8zXV5kYpmLASMtgIKPbaFiHY msJegbo2VcEO4IAWqaFxIdFzTxAh+E2E3JrSuDf9e0o8gb4iL7cpEZHv4sNebeRXS9eh gKcQ== X-Gm-Message-State: AOAM532XufMKWb+5DH28pa7j5bFcPgOtgklVsXxYgK5yKnISAsEJAZC/ dUVu/jMNzwpQUw0zTCBxE+d+7nlCPNE= X-Google-Smtp-Source: ABdhPJzgIvRY8hQXdyudxLdGD9g8Uh22CW0Xy7ncJw+C2kw9Dd+/DOPx0w44g3SM207dfWO3sasU+Q== X-Received: by 2002:a05:600c:3217:b0:380:e2a3:18cf with SMTP id r23-20020a05600c321700b00380e2a318cfmr17925144wmp.129.1646164114511; Tue, 01 Mar 2022 11:48:34 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id x3-20020adfdd83000000b001e58c8de11bsm14690159wrl.39.2022.03.01.11.48.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 01 Mar 2022 11:48:33 -0800 (PST) Message-Id: <4eca8028c971e32fdf34720de4674daf7004a294.1646164112.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 01 Mar 2022 19:48:28 +0000 Subject: [PATCH v3 1/5] test-read-graph: include extra post-parse info Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: me@ttaylorr.com, gitster@pobox.com, abhishekkumar8222@gmail.com, avarab@gmail.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee It can be helpful to verify that the 'struct commit_graph' that results from parsing a commit-graph is correctly structured. The existence of different chunks is not enough to verify that all of the optional features are correctly enabled. Update 'test-tool read-graph' to output an "options:" line that includes information for different parts of the struct commit_graph. In particular, this change demonstrates that the read_generation_data option is never being enabled, which will be fixed in a later change. Signed-off-by: Derrick Stolee --- t/helper/test-read-graph.c | 13 +++++++++++++ t/t4216-log-bloom.sh | 1 + t/t5318-commit-graph.sh | 1 + t/t5324-split-commit-graph.sh | 5 +++++ 4 files changed, 20 insertions(+) diff --git a/t/helper/test-read-graph.c b/t/helper/test-read-graph.c index 75927b2c81d..98b73bb8f25 100644 --- a/t/helper/test-read-graph.c +++ b/t/helper/test-read-graph.c @@ -3,6 +3,7 @@ #include "commit-graph.h" #include "repository.h" #include "object-store.h" +#include "bloom.h" int cmd__read_graph(int argc, const char **argv) { @@ -45,6 +46,18 @@ int cmd__read_graph(int argc, const char **argv) printf(" bloom_data"); printf("\n"); + printf("options:"); + if (graph->bloom_filter_settings) + printf(" bloom(%"PRIu32",%"PRIu32",%"PRIu32")", + graph->bloom_filter_settings->hash_version, + graph->bloom_filter_settings->bits_per_entry, + graph->bloom_filter_settings->num_hashes); + if (graph->read_generation_data) + printf(" read_generation_data"); + if (graph->topo_levels) + printf(" topo_levels"); + printf("\n"); + UNLEAK(graph); return 0; diff --git a/t/t4216-log-bloom.sh b/t/t4216-log-bloom.sh index cc3cebf6722..5ed6d2a21c1 100755 --- a/t/t4216-log-bloom.sh +++ b/t/t4216-log-bloom.sh @@ -48,6 +48,7 @@ graph_read_expect () { header: 43475048 1 $(test_oid oid_version) $NUM_CHUNKS 0 num_commits: $1 chunks: oid_fanout oid_lookup commit_metadata generation_data bloom_indexes bloom_data + options: bloom(1,10,7) EOF test-tool read-graph >actual && test_cmp expect actual diff --git a/t/t5318-commit-graph.sh b/t/t5318-commit-graph.sh index edb728f77c3..2b05026cf6d 100755 --- a/t/t5318-commit-graph.sh +++ b/t/t5318-commit-graph.sh @@ -104,6 +104,7 @@ graph_read_expect() { header: 43475048 1 $(test_oid oid_version) $NUM_CHUNKS 0 num_commits: $1 chunks: oid_fanout oid_lookup commit_metadata$OPTIONAL + options: EOF test-tool read-graph >output && test_cmp expect output diff --git a/t/t5324-split-commit-graph.sh b/t/t5324-split-commit-graph.sh index 847b8097109..778fa418de2 100755 --- a/t/t5324-split-commit-graph.sh +++ b/t/t5324-split-commit-graph.sh @@ -34,6 +34,7 @@ graph_read_expect() { header: 43475048 1 $(test_oid oid_version) 4 $NUM_BASE num_commits: $1 chunks: oid_fanout oid_lookup commit_metadata generation_data + options: EOF test-tool read-graph >output && test_cmp expect output @@ -508,6 +509,7 @@ test_expect_success 'setup repo for mixed generation commit-graph-chain' ' header: 43475048 1 $(test_oid oid_version) 4 1 num_commits: $NUM_SECOND_LAYER_COMMITS chunks: oid_fanout oid_lookup commit_metadata + options: EOF test_cmp expect output && git commit-graph verify && @@ -540,6 +542,7 @@ test_expect_success 'do not write generation data chunk if not present on existi header: 43475048 1 $(test_oid oid_version) 4 2 num_commits: $NUM_THIRD_LAYER_COMMITS chunks: oid_fanout oid_lookup commit_metadata + options: EOF test_cmp expect output && git commit-graph verify @@ -581,6 +584,7 @@ test_expect_success 'do not write generation data chunk if the topmost remaining header: 43475048 1 $(test_oid oid_version) 4 2 num_commits: $(($NUM_THIRD_LAYER_COMMITS + $NUM_FOURTH_LAYER_COMMITS)) chunks: oid_fanout oid_lookup commit_metadata + options: EOF test_cmp expect output && git commit-graph verify @@ -620,6 +624,7 @@ test_expect_success 'write generation data chunk if topmost remaining layer has header: 43475048 1 $(test_oid oid_version) 5 1 num_commits: $(($NUM_SECOND_LAYER_COMMITS + $NUM_THIRD_LAYER_COMMITS + $NUM_FOURTH_LAYER_COMMITS + $NUM_FIFTH_LAYER_COMMITS)) chunks: oid_fanout oid_lookup commit_metadata generation_data + options: EOF test_cmp expect output ) From patchwork Tue Mar 1 19:48:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12765124 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BC497C433EF for ; Tue, 1 Mar 2022 19:50:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237535AbiCATuv (ORCPT ); Tue, 1 Mar 2022 14:50:51 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59976 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237520AbiCATuT (ORCPT ); Tue, 1 Mar 2022 14:50:19 -0500 Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com [IPv6:2a00:1450:4864:20::329]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E36456E55A for ; Tue, 1 Mar 2022 11:49:05 -0800 (PST) Received: by mail-wm1-x329.google.com with SMTP id o18-20020a05600c4fd200b003826701f847so1289480wmq.4 for ; Tue, 01 Mar 2022 11:49:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=1mFg2bjzdPEzCMnLZhFdBC7+b24mMKLFGnbYGnBDYJM=; b=kCUU0e13a5f2xQ9WykSg//3W+yIvMFd2h1mEVTEPH/L5VbvJ6bz3tEZ4FJjbdTdfGi V31WyNffMykDrGfXx2yiK91P9G4fnYZYLuez+wO/EbnlkDOGgav163ZF0DgU0uoEf31v 4fGvp5G2EnzE1a7ivB6b7uzb/78XQPdcxBeo+YSBv7q0/v5LG2fA9OrChimF2HTHuRph OZ9zNSb4rDQl+fnW6xvP3CHa/KWnrVvgvhEqV8sMeyLJFw9AmN0tBFe27duHqKaeKTZF ykdaXNOSuck+9pCVeKbIGxPqJpH3bwLLqxlUMNj/yRhVJk6VG26UUeLpeEkFKjnNIjwB vjmg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=1mFg2bjzdPEzCMnLZhFdBC7+b24mMKLFGnbYGnBDYJM=; b=aUjIH0RYQdmFwHySWHm4pF6KT9iTOJifuze3U+q5sYxClSwgsphFN1L9IaTba0TbLG IJ5OFRwR3vXZy34cxg8gFlo2hO7TGdqvDZ+QpEIkIyLi34QkS3cq+hBGv2uq8tHz32fj dvYOm7ts9tR2ZzhvZUlw6mW+1YSnsbN8aOL4HLvsVA8pUq+a76g4TO2T7KDcJXGOQlxz esEQwKgE4CsAcHzN7rx+DDbwnM+kRBwouhiP3Y3QFhQ6vwoB6K7b2Q1rJ4fUTEecHFQU UJyDJBYdW9Clonlu8PDO3TIK1PSOaJG30BAz7MCycwkY65LQF6XJDfDSLzlYg8/LwcOk ePVA== X-Gm-Message-State: AOAM530gTb6NjImwxwjNEBOIsIF9zNQ216kKiJgyirhgiI03HvBlW05r KB3WujV1ZgUA00VV04NSAnn1Jx6vqqA= X-Google-Smtp-Source: ABdhPJzscEKwkUYsJFerj/8MtxaiMFkoNkjinHvvcLGXuTNH9vQLahYG24rDOkfsfc6Y12sXtfzyuA== X-Received: by 2002:a1c:29c3:0:b0:350:9797:b38f with SMTP id p186-20020a1c29c3000000b003509797b38fmr18265874wmp.22.1646164115416; Tue, 01 Mar 2022 11:48:35 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id f8-20020a05600c4e8800b00380ee4a78fdsm4339797wmq.4.2022.03.01.11.48.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 01 Mar 2022 11:48:34 -0800 (PST) Message-Id: In-Reply-To: References: Date: Tue, 01 Mar 2022 19:48:29 +0000 Subject: [PATCH v3 2/5] t5318: extract helpers to lib-commit-graph.sh Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: me@ttaylorr.com, gitster@pobox.com, abhishekkumar8222@gmail.com, avarab@gmail.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee The graph_git_behavior helper is useful for testing that certain Git commands behave the same when using the commit-graph and when not using the commit-graph. Extract it to a new lib-commit-graph.sh file for use in new test scripts that will split out from t5318. While doing this extraction, also extract graph_read_expect and the logic for priming the test_oid_cache. Signed-off-by: Derrick Stolee --- t/lib-commit-graph.sh | 48 +++++++++++++++++++++++++++++++++++++++ t/t5318-commit-graph.sh | 50 ++--------------------------------------- 2 files changed, 50 insertions(+), 48 deletions(-) create mode 100755 t/lib-commit-graph.sh diff --git a/t/lib-commit-graph.sh b/t/lib-commit-graph.sh new file mode 100755 index 00000000000..07e12b9d2fe --- /dev/null +++ b/t/lib-commit-graph.sh @@ -0,0 +1,48 @@ +#!/bin/sh + +# Helper functions for testing commit-graphs. + +# Initialize OID cache with oid_version +test_oid_cache <<-EOF +oid_version sha1:1 +oid_version sha256:2 +EOF + +graph_git_two_modes() { + git -c core.commitGraph=true $1 >output && + git -c core.commitGraph=false $1 >expect && + test_cmp expect output +} + +graph_git_behavior() { + MSG=$1 + DIR=$2 + BRANCH=$3 + COMPARE=$4 + test_expect_success "check normal git operations: $MSG" ' + cd "$TRASH_DIRECTORY/$DIR" && + graph_git_two_modes "log --oneline $BRANCH" && + graph_git_two_modes "log --topo-order $BRANCH" && + graph_git_two_modes "log --graph $COMPARE..$BRANCH" && + graph_git_two_modes "branch -vv" && + graph_git_two_modes "merge-base -a $BRANCH $COMPARE" + ' +} + +graph_read_expect() { + OPTIONAL="" + NUM_CHUNKS=3 + if test -n "$2" + then + OPTIONAL=" $2" + NUM_CHUNKS=$((3 + $(echo "$2" | wc -w))) + fi + cat >expect <<- EOF + header: 43475048 1 $(test_oid oid_version) $NUM_CHUNKS 0 + num_commits: $1 + chunks: oid_fanout oid_lookup commit_metadata$OPTIONAL + options: + EOF + test-tool read-graph >output && + test_cmp expect output +} diff --git a/t/t5318-commit-graph.sh b/t/t5318-commit-graph.sh index 2b05026cf6d..9e2b5884dae 100755 --- a/t/t5318-commit-graph.sh +++ b/t/t5318-commit-graph.sh @@ -29,12 +29,7 @@ test_expect_success 'setup full repo' ' cd "$TRASH_DIRECTORY/full" && git init && git config core.commitGraph true && - objdir=".git/objects" && - - test_oid_cache <<-EOF - oid_version sha1:1 - oid_version sha256:2 - EOF + objdir=".git/objects" ' test_expect_success POSIXPERM 'tweak umask for modebit tests' ' @@ -69,47 +64,10 @@ test_expect_success 'create commits and repack' ' git repack ' -graph_git_two_modes() { - git -c core.commitGraph=true $1 >output && - git -c core.commitGraph=false $1 >expect && - test_cmp expect output -} - -graph_git_behavior() { - MSG=$1 - DIR=$2 - BRANCH=$3 - COMPARE=$4 - test_expect_success "check normal git operations: $MSG" ' - cd "$TRASH_DIRECTORY/$DIR" && - graph_git_two_modes "log --oneline $BRANCH" && - graph_git_two_modes "log --topo-order $BRANCH" && - graph_git_two_modes "log --graph $COMPARE..$BRANCH" && - graph_git_two_modes "branch -vv" && - graph_git_two_modes "merge-base -a $BRANCH $COMPARE" - ' -} +. "$TEST_DIRECTORY"/lib-commit-graph.sh graph_git_behavior 'no graph' full commits/3 commits/1 -graph_read_expect() { - OPTIONAL="" - NUM_CHUNKS=3 - if test ! -z "$2" - then - OPTIONAL=" $2" - NUM_CHUNKS=$((3 + $(echo "$2" | wc -w))) - fi - cat >expect <<- EOF - header: 43475048 1 $(test_oid oid_version) $NUM_CHUNKS 0 - num_commits: $1 - chunks: oid_fanout oid_lookup commit_metadata$OPTIONAL - options: - EOF - test-tool read-graph >output && - test_cmp expect output -} - test_expect_success 'exit with correct error on bad input to --stdin-commits' ' cd "$TRASH_DIRECTORY/full" && # invalid, non-hex OID @@ -826,10 +784,6 @@ test_expect_success 'set up and verify repo with generation data overflow chunk' objdir=".git/objects" && UNIX_EPOCH_ZERO="@0 +0000" && FUTURE_DATE="@2147483646 +0000" && - test_oid_cache <<-EOF && - oid_version sha1:1 - oid_version sha256:2 - EOF cd "$TRASH_DIRECTORY" && mkdir repo && cd repo && From patchwork Tue Mar 1 19:48:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12765120 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AA944C433EF for ; Tue, 1 Mar 2022 19:49:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237468AbiCATue (ORCPT ); Tue, 1 Mar 2022 14:50:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57142 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237531AbiCATuU (ORCPT ); Tue, 1 Mar 2022 14:50:20 -0500 Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6421F47383 for ; Tue, 1 Mar 2022 11:49:08 -0800 (PST) Received: by mail-wm1-x32b.google.com with SMTP id c192so8122510wma.4 for ; Tue, 01 Mar 2022 11:49:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:mime-version :content-transfer-encoding:fcc:to:cc; bh=T5VYTv7sYDSw8wi0GKaTWAzxizpRswtApNTiRs5+WQs=; b=HqSUlxodt1r3Gep3SASDTJPXML3nJQcCY4FonAoE8nQMn16+0QIdru5frQQkk9G4Lg GWmPS4/1UBbp8oNgx8rygNPLt9cNbN4cu7OsfRBLPBH1Iw7FGdixYreqtq2OhbNT8fmW 9gu9hELpjC7VzB/omZgoDbhD/qcqp8iggVF261qpWR6PUii7H3Nh6LZ+IOYrup2JP/Yi UvD1kOnXgbROfGduoCVtraMeSqsMYINWP0W/otCTIhugtQIxT3hg0HBRJebhwu7qdw8R CSRwHCKz8k7N4WK755GwNkioVHBU35uPLI3gXq4bnQJgC5ED+Cfk/IjZluLVNQtYJCXr zxbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:mime-version:content-transfer-encoding:fcc:to:cc; bh=T5VYTv7sYDSw8wi0GKaTWAzxizpRswtApNTiRs5+WQs=; b=tS50nvQy4etA7mod/SZ4zCp7PE78Z9zbddvCyIUBh838pFqRLGmQhyOFGMU9VJuVVI MqNmfNsAl+or4C5/AMHZZK0y816SXWnsAxoUAADHuIno5f6ITMhgk3DRPGxNwGd5Ybgx UVc4QVDxTpUnpxLtBOBOMvH/VOQrqRnF8TMohJLoW/CHGhcn+g/XIwb2d9tYEYSmMffj 1UcH/qwH3cJ1z0FUxAzf5YLPuCMVzMeKliVtgdT/+ChEHJHItdHWNomcS6OMEuVuKqzy iWQo5DkaTvbZIzoF5++6mF32TGPAh/+aU/wf/rAm1x5hKpkJUiGKIWgSyy6ZS9jMnISS i20Q== X-Gm-Message-State: AOAM532nvqWQL5DTi+VtLNNK+B623rRGnL4sQuZ+hRI80CshXwvl79qQ eX8hPSQfb1/Kh2CbO/Y2FDdunetyRK0= X-Google-Smtp-Source: ABdhPJz7IP3889bvIr3wKSt/rjhy2NPpcTppeaKq9eLuBG8PBjW+FhUFksabWml270N+phMHRd9Rag== X-Received: by 2002:a05:600c:1d08:b0:381:6eda:67d1 with SMTP id l8-20020a05600c1d0800b003816eda67d1mr7801993wms.88.1646164116281; Tue, 01 Mar 2022 11:48:36 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id i20-20020a05600c355400b0038164ef5418sm3741647wmq.32.2022.03.01.11.48.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 01 Mar 2022 11:48:35 -0800 (PST) Message-Id: <562341b76b30ff3cdc01cea4cda550a7b8165bdd.1646164112.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 01 Mar 2022 19:48:30 +0000 Subject: [PATCH v3 3/5] commit-graph: fix ordering bug in generation numbers MIME-Version: 1.0 Fcc: Sent To: git@vger.kernel.org Cc: me@ttaylorr.com, gitster@pobox.com, abhishekkumar8222@gmail.com, avarab@gmail.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee When computing the generation numbers for a commit-graph, we compute the corrected commit dates and then check if their offsets from the actual dates is too large to fit in the 32-bit Generation Data chunk. However, there is a problem with this approach: if we have parsed the generation data from the previous commit-graph, then we continue the loop because the corrected commit date is already computed. This causes an under-count in the number of overflow values. It is incorrect to add an increment to num_generation_data_overflows next to this 'continue' statement, because we might start double-counting commits that are computed because of the depth-first search walk from a commit with an earlier OID. Instead, iterate over the full commit list at the end, checking the offsets to see how many grow beyond the maximum value. Create a new t5328-commit-graph-64-bit-time.sh test script to handle special cases of testing 64-bit timestampes. This helps demonstrate this bug in more cases. It still won't hit all potential cases until the next change, which reenables reading generation numbers. Use the skip_all trick from 0a2bfccb9c8 (t0051: use "skip_all" under !MINGW in single-test file, 2022-02-04) to make the output clean when run on a 32-bit system. Hepled-by: Ævar Arnfjörð Bjarmason Signed-off-by: Derrick Stolee --- commit-graph.c | 10 +++++--- t/t5318-commit-graph.sh | 4 +-- t/t5328-commit-graph-64bit-time.sh | 39 ++++++++++++++++++++++++++++++ 3 files changed, 48 insertions(+), 5 deletions(-) create mode 100755 t/t5328-commit-graph-64bit-time.sh diff --git a/commit-graph.c b/commit-graph.c index 265c010122e..a19bd96c2ee 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -1556,12 +1556,16 @@ static void compute_generation_numbers(struct write_commit_graph_context *ctx) if (current->date && current->date > max_corrected_commit_date) max_corrected_commit_date = current->date - 1; commit_graph_data_at(current)->generation = max_corrected_commit_date + 1; - - if (commit_graph_data_at(current)->generation - current->date > GENERATION_NUMBER_V2_OFFSET_MAX) - ctx->num_generation_data_overflows++; } } } + + for (i = 0; i < ctx->commits.nr; i++) { + struct commit *c = ctx->commits.list[i]; + timestamp_t offset = commit_graph_data_at(c)->generation - c->date; + if (offset > GENERATION_NUMBER_V2_OFFSET_MAX) + ctx->num_generation_data_overflows++; + } stop_progress(&ctx->progress); } diff --git a/t/t5318-commit-graph.sh b/t/t5318-commit-graph.sh index 9e2b5884dae..0ed7e9de8e6 100755 --- a/t/t5318-commit-graph.sh +++ b/t/t5318-commit-graph.sh @@ -425,10 +425,10 @@ test_expect_success 'warn on improper hash version' ' ) ' -test_expect_success 'lower layers have overflow chunk' ' +test_expect_success TIME_IS_64BIT,TIME_T_IS_64BIT 'lower layers have overflow chunk' ' cd "$TRASH_DIRECTORY/full" && UNIX_EPOCH_ZERO="@0 +0000" && - FUTURE_DATE="@2147483646 +0000" && + FUTURE_DATE="@4147483646 +0000" && rm -f .git/objects/info/commit-graph && test_commit --date "$FUTURE_DATE" future-1 && test_commit --date "$UNIX_EPOCH_ZERO" old-1 && diff --git a/t/t5328-commit-graph-64bit-time.sh b/t/t5328-commit-graph-64bit-time.sh new file mode 100755 index 00000000000..28114bcaf47 --- /dev/null +++ b/t/t5328-commit-graph-64bit-time.sh @@ -0,0 +1,39 @@ +#!/bin/sh + +test_description='commit graph with 64-bit timestamps' +. ./test-lib.sh + +if ! test_have_prereq TIME_IS_64BIT || ! test_have_prereq TIME_T_IS_64BIT +then + skip_all='skipping 64-bit timestamp tests' + test_done +fi + +. "$TEST_DIRECTORY"/lib-commit-graph.sh + +UNIX_EPOCH_ZERO="@0 +0000" +FUTURE_DATE="@4147483646 +0000" + +GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS=0 + +test_expect_success 'lower layers have overflow chunk' ' + rm -f .git/objects/info/commit-graph && + test_commit --date "$FUTURE_DATE" future-1 && + test_commit --date "$UNIX_EPOCH_ZERO" old-1 && + git commit-graph write --reachable && + test_commit --date "$FUTURE_DATE" future-2 && + test_commit --date "$UNIX_EPOCH_ZERO" old-2 && + git commit-graph write --reachable --split=no-merge && + test_commit extra && + git commit-graph write --reachable --split=no-merge && + git commit-graph write --reachable && + graph_read_expect 5 "generation_data generation_data_overflow" && + mv .git/objects/info/commit-graph commit-graph-upgraded && + git commit-graph write --reachable && + graph_read_expect 5 "generation_data generation_data_overflow" && + test_cmp .git/objects/info/commit-graph commit-graph-upgraded +' + +graph_git_behavior 'overflow' '' HEAD~2 HEAD + +test_done From patchwork Tue Mar 1 19:48:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12765122 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5DE66C4332F for ; Tue, 1 Mar 2022 19:50:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237508AbiCATur (ORCPT ); Tue, 1 Mar 2022 14:50:47 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57146 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237530AbiCATuU (ORCPT ); Tue, 1 Mar 2022 14:50:20 -0500 Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C205C554B0 for ; Tue, 1 Mar 2022 11:49:11 -0800 (PST) Received: by mail-wr1-x431.google.com with SMTP id b5so22234855wrr.2 for ; Tue, 01 Mar 2022 11:49:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:mime-version :content-transfer-encoding:fcc:to:cc; bh=351duXGBBzHl172YvPTgEEx/kdUTzFDdFxEhP4jeM00=; b=CHc6HBx0DzZYxEcMv7TNJYBTtMhN7bohVXdFCcVxsE+aM07HIVO89g5zmjQNQASnJC 8VvOT2pNo/4c+AFemxNBJV2EkCAnuNJKq0HZTIQso0EQ3wW1O5QBrOyaobc9+b6ILE7E o5f1PHRJ8Ogxn8ByVks1gNhS9/9Ogzics0l15Crgpi+XWX69jZDWS8L01t1sJmUzYMKQ 6QUR8lK/1zg5GtFsZbR6v8Oaa4+GNyaDLIEDu/TSLPPCjS0Ns22zAQN+TNLwS1Mq6J58 WtXZiCLAl37AAzK5VkqKzyw2DBB0GIm03kZuV3+iBZDSwHEDXncZsCHyM+VRRuet+muL F/eA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:mime-version:content-transfer-encoding:fcc:to:cc; bh=351duXGBBzHl172YvPTgEEx/kdUTzFDdFxEhP4jeM00=; b=64bmwgnJ0rQnLYQsmTRqshTanGOKbU7obU5hVSORJZ6kOEYFWK+PP7UF0dWfkUK7jT g4npC1HtuwcmaMUVQ16acF9l6thc3Ti4od4gvMNc7NmpjgUToYMsBae3QKi3UGCH4jSN TRUN+10zTFgf83B1WwdlC9NFDrna9XIZDplioie4cXdv6p5LbWCZhcK/gBv+5N0yOi8+ xTYpg6lgboktOuDgTmdowvN7PDRpkE0PQDpB88g4yOrKeSmbzATNglXFFr724BzyLsz2 BvNHq7lGqNPKsIDXGro/1/rhAq4XeR4UHy0dXKmhjntpVkvlkEFYURZBD5qK6FVwC3Eu kCHg== X-Gm-Message-State: AOAM530iN32s6PvUrcy4EVd5ohPaNpQfDoD5PS7AUowslGWoBScLuxPM gbocLbQX/yPYVTtaozAgzIVy0Qm5RzU= X-Google-Smtp-Source: ABdhPJzYr2vvk3BThlFA50FUB/uD2F96ua9Zlg6f+i18QZaWFUvFFViXVhNa5z+u/Qe0SntgYXzd2Q== X-Received: by 2002:a5d:6405:0:b0:1ef:7df1:5c04 with SMTP id z5-20020a5d6405000000b001ef7df15c04mr13866086wru.178.1646164117182; Tue, 01 Mar 2022 11:48:37 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id s11-20020a5d69cb000000b001ef5b49d68esm14785022wrw.37.2022.03.01.11.48.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 01 Mar 2022 11:48:36 -0800 (PST) Message-Id: <041d96bf1d7fecbae946ad71cbe2dc0b3492fd65.1646164112.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 01 Mar 2022 19:48:31 +0000 Subject: [PATCH v3 4/5] commit-graph: start parsing generation v2 (again) MIME-Version: 1.0 Fcc: Sent To: git@vger.kernel.org Cc: me@ttaylorr.com, gitster@pobox.com, abhishekkumar8222@gmail.com, avarab@gmail.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee The 'read_generation_data' member of 'struct commit_graph' was introduced by 1fdc383c5 (commit-graph: use generation v2 only if entire chain does, 2021-01-16). The intention was to avoid using corrected commit dates if not all layers of a commit-graph had that data stored. The logic in validate_mixed_generation_chain() at that point incorrectly initialized read_generation_data to 1 if and only if the tip commit-graph contained the Corrected Commit Date chunk. This was "fixed" in 448a39e65 (commit-graph: validate layers for generation data, 2021-02-02) to validate that read_generation_data was either non-zero for all layers, or it would set read_generation_data to zero for all layers. The problem here is that read_generation_data is not initialized to be non-zero anywhere! This change initializes read_generation_data immediately after the chunk is parsed, so each layer will have its value present as soon as possible. The read_generation_data member is used in fill_commit_graph_info() to determine if we should use the corrected commit date or the topological levels stored in the Commit Data chunk. Due to this bug, all previous versions of Git were defaulting to topological levels in all cases! This can be measured with some performance tests. Using the Linux kernel as a testbed, I generated a complete commit-graph containing corrected commit dates and tested the 'new' version against the previous, 'old' version. First, rev-list with --topo-order demonstrates a 26% improvement using corrected commit dates: hyperfine \ -n "old" "$OLD_GIT rev-list --topo-order -1000 v3.6" \ -n "new" "$NEW_GIT rev-list --topo-order -1000 v3.6" \ --warmup=10 Benchmark 1: old Time (mean ± σ): 57.1 ms ± 3.1 ms Range (min … max): 52.9 ms … 62.0 ms 55 runs Benchmark 2: new Time (mean ± σ): 45.5 ms ± 3.3 ms Range (min … max): 39.9 ms … 51.7 ms 59 runs Summary 'new' ran 1.26 ± 0.11 times faster than 'old' These performance improvements are due to the algorithmic improvements given by walking fewer commits due to the higher cutoffs from corrected commit dates. However, this comes at a cost. The additional I/O cost of parsing the corrected commit dates is visible in case of merge-base commands that do not reduce the overall number of walked commits. hyperfine \ -n "old" "$OLD_GIT merge-base v4.8 v4.9" \ -n "new" "$NEW_GIT merge-base v4.8 v4.9" \ --warmup=10 Benchmark 1: old Time (mean ± σ): 110.4 ms ± 6.4 ms Range (min … max): 96.0 ms … 118.3 ms 25 runs Benchmark 2: new Time (mean ± σ): 150.7 ms ± 1.1 ms Range (min … max): 149.3 ms … 153.4 ms 19 runs Summary 'old' ran 1.36 ± 0.08 times faster than 'new' Performance issues like this are what motivated 702110aac (commit-graph: use config to specify generation type, 2021-02-25). In the future, we could fix this performance problem by inserting the corrected commit date offsets into the Commit Date chunk instead of having that data in an extra chunk. Signed-off-by: Derrick Stolee --- commit-graph.c | 3 +++ t/lib-commit-graph.sh | 12 +++++++++++- t/t4216-log-bloom.sh | 2 +- t/t5318-commit-graph.sh | 2 +- t/t5324-split-commit-graph.sh | 9 +++++++-- 5 files changed, 23 insertions(+), 5 deletions(-) diff --git a/commit-graph.c b/commit-graph.c index a19bd96c2ee..8e52bb09552 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -407,6 +407,9 @@ struct commit_graph *parse_commit_graph(struct repository *r, &graph->chunk_generation_data); pair_chunk(cf, GRAPH_CHUNKID_GENERATION_DATA_OVERFLOW, &graph->chunk_generation_data_overflow); + + if (graph->chunk_generation_data) + graph->read_generation_data = 1; } if (r->settings.commit_graph_read_changed_paths) { diff --git a/t/lib-commit-graph.sh b/t/lib-commit-graph.sh index 07e12b9d2fe..5d79e1a4e96 100755 --- a/t/lib-commit-graph.sh +++ b/t/lib-commit-graph.sh @@ -37,11 +37,21 @@ graph_read_expect() { OPTIONAL=" $2" NUM_CHUNKS=$((3 + $(echo "$2" | wc -w))) fi + GENERATION_VERSION=2 + if test -n "$3" + then + GENERATION_VERSION=$3 + fi + OPTIONS= + if test $GENERATION_VERSION -gt 1 + then + OPTIONS=" read_generation_data" + fi cat >expect <<- EOF header: 43475048 1 $(test_oid oid_version) $NUM_CHUNKS 0 num_commits: $1 chunks: oid_fanout oid_lookup commit_metadata$OPTIONAL - options: + options:$OPTIONS EOF test-tool read-graph >output && test_cmp expect output diff --git a/t/t4216-log-bloom.sh b/t/t4216-log-bloom.sh index 5ed6d2a21c1..fa9d32facfb 100755 --- a/t/t4216-log-bloom.sh +++ b/t/t4216-log-bloom.sh @@ -48,7 +48,7 @@ graph_read_expect () { header: 43475048 1 $(test_oid oid_version) $NUM_CHUNKS 0 num_commits: $1 chunks: oid_fanout oid_lookup commit_metadata generation_data bloom_indexes bloom_data - options: bloom(1,10,7) + options: bloom(1,10,7) read_generation_data EOF test-tool read-graph >actual && test_cmp expect actual diff --git a/t/t5318-commit-graph.sh b/t/t5318-commit-graph.sh index 0ed7e9de8e6..fbf0d64578c 100755 --- a/t/t5318-commit-graph.sh +++ b/t/t5318-commit-graph.sh @@ -456,7 +456,7 @@ test_expect_success 'git commit-graph verify' ' cd "$TRASH_DIRECTORY/full" && git rev-parse commits/8 | git -c commitGraph.generationVersion=1 commit-graph write --stdin-commits && git commit-graph verify >output && - graph_read_expect 9 extra_edges + graph_read_expect 9 extra_edges 1 ' NUM_COMMITS=9 diff --git a/t/t5324-split-commit-graph.sh b/t/t5324-split-commit-graph.sh index 778fa418de2..669ddc645fa 100755 --- a/t/t5324-split-commit-graph.sh +++ b/t/t5324-split-commit-graph.sh @@ -30,11 +30,16 @@ graph_read_expect() { then NUM_BASE=$2 fi + OPTIONS= + if test -z "$3" + then + OPTIONS=" read_generation_data" + fi cat >expect <<- EOF header: 43475048 1 $(test_oid oid_version) 4 $NUM_BASE num_commits: $1 chunks: oid_fanout oid_lookup commit_metadata generation_data - options: + options:$OPTIONS EOF test-tool read-graph >output && test_cmp expect output @@ -624,7 +629,7 @@ test_expect_success 'write generation data chunk if topmost remaining layer has header: 43475048 1 $(test_oid oid_version) 5 1 num_commits: $(($NUM_SECOND_LAYER_COMMITS + $NUM_THIRD_LAYER_COMMITS + $NUM_FOURTH_LAYER_COMMITS + $NUM_FIFTH_LAYER_COMMITS)) chunks: oid_fanout oid_lookup commit_metadata generation_data - options: + options: read_generation_data EOF test_cmp expect output ) From patchwork Tue Mar 1 19:48:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 12765121 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F133FC433FE for ; Tue, 1 Mar 2022 19:50:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237461AbiCATuo (ORCPT ); Tue, 1 Mar 2022 14:50:44 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59990 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237536AbiCATuU (ORCPT ); Tue, 1 Mar 2022 14:50:20 -0500 Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com [IPv6:2a00:1450:4864:20::335]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ED7A858E6C for ; Tue, 1 Mar 2022 11:49:12 -0800 (PST) Received: by mail-wm1-x335.google.com with SMTP id q7-20020a7bce87000000b00382255f4ca9so1363472wmj.2 for ; Tue, 01 Mar 2022 11:49:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=61m4XDoLYC6T0oD6FkGBetkfkc/04h98VYKs/QcSYyY=; b=lynnvt/FqF13/1mSAj32xyifmDyjbOY5OiUBTYWv5vulaxdOjQ1xtLDPQfS+QVtjUa lSO4UNW0G9F/WYKyId3dDmp7BSlN70coPmVQ5EL+RPmZRPmsKzSRQIuCKcFbA5YGOf2H jl3kovS0mNlt+ocHMkbTH4ZFq5wTe38B7x/qQvGVbV4MZjOHHrFBjssM6fadYQQ48lXw 8p6PdmRHVaEmNK62GnuKYQB7bi0crrFyiND+tyfC0jBkEi7N/6eUR8OP2i75JO8qD95V C9bmx4RmeNAMLiYWQjHsOG9t/pYho+ulPc1dZJyqwO0xeqnt43zk3wSJOEYI4pvSk23F Px6A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=61m4XDoLYC6T0oD6FkGBetkfkc/04h98VYKs/QcSYyY=; b=3LeyzSFBpweoJuJN7+GH0clkWOzgaL1BTrvAaO8g1XD22pP9ZlH5ZOeOoMD4qrwjIP l8rfiiuBUdfeehSCBneKAvGq+IywRVTI5z5hkOj3SPQxcn8nfPIpW2nTE8i3tkBQTEqQ fLRmlMCgmxIqC0aMr3dvyuP9L/S0FH39oBZpqTsiykeQ997dpXKFuzVrqSgwY9VINxSr 80NFCmu1RHnxeIsoCA2d8b9fWfN1HSmMKZUoS5f1hjonRlEfnESVDojfvozhX0Zf9crZ pI8PyWwJoqPahlvxl3F6RHUJdxFaFhjsluXKj0RKqAhLBisU3VWR2UfMI9aokQG148p7 UlIA== X-Gm-Message-State: AOAM533yPHtZyCiboT9ctslDw2d7dwUBpJBR1pit/GEPdqtcLaWFS3pf deqvMN41lLHB0d/SEt/xeEx4m9ItJx8= X-Google-Smtp-Source: ABdhPJwBy71IqhmWCwhXHGJRbgglRFVG8EwX3mw0maBN88NXkSZIRBvELOnJVp3LMNNoA8PzFvy4YQ== X-Received: by 2002:a7b:c455:0:b0:380:a646:eb0e with SMTP id l21-20020a7bc455000000b00380a646eb0emr17905468wmi.170.1646164118070; Tue, 01 Mar 2022 11:48:38 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id m3-20020a5d6243000000b001e33760776fsm14533911wrv.10.2022.03.01.11.48.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 01 Mar 2022 11:48:37 -0800 (PST) Message-Id: In-Reply-To: References: Date: Tue, 01 Mar 2022 19:48:32 +0000 Subject: [PATCH v3 5/5] commit-graph: fix generation number v2 overflow values Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: me@ttaylorr.com, gitster@pobox.com, abhishekkumar8222@gmail.com, avarab@gmail.com, Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee The Generation Data Chunk was implemented and tested in e8b63005c (commit-graph: implement generation data chunk, 2021-01-16), but the test was carefully constructed to work on systems with 32-bit dates. Since the corrected commit date offsets still required more than 31 bits, this triggered writing the generation_data_overflow chunk. However, upon closer look, the write_graph_chunk_generation_data_overflow() method writes the offsets to the chunk (as dictated by the format) but fill_commit_graph_info() treats the value in the chunk as if it is the full corrected commit date (not an offset). For some reason, this does not cause an issue when using the FUTURE_DATE specified in t5318-commit-graph.sh, but it does show up as a failure in 'git commit-graph verify' if we increase that FUTURE_DATE to be above four billion. Fix this error and create a 64-bit timestamp version of the test so we can test these larger values. Signed-off-by: Derrick Stolee --- commit-graph.c | 2 +- t/t5328-commit-graph-64bit-time.sh | 27 +++++++++++++++++++++++++++ 2 files changed, 28 insertions(+), 1 deletion(-) diff --git a/commit-graph.c b/commit-graph.c index 8e52bb09552..b86a6a634fe 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -806,7 +806,7 @@ static void fill_commit_graph_info(struct commit *item, struct commit_graph *g, die(_("commit-graph requires overflow generation data but has none")); offset_pos = offset ^ CORRECTED_COMMIT_DATE_OFFSET_OVERFLOW; - graph_data->generation = get_be64(g->chunk_generation_data_overflow + 8 * offset_pos); + graph_data->generation = item->date + get_be64(g->chunk_generation_data_overflow + 8 * offset_pos); } else graph_data->generation = item->date + offset; } else diff --git a/t/t5328-commit-graph-64bit-time.sh b/t/t5328-commit-graph-64bit-time.sh index 28114bcaf47..093f0c067af 100755 --- a/t/t5328-commit-graph-64bit-time.sh +++ b/t/t5328-commit-graph-64bit-time.sh @@ -36,4 +36,31 @@ test_expect_success 'lower layers have overflow chunk' ' graph_git_behavior 'overflow' '' HEAD~2 HEAD +test_expect_success 'set up and verify repo with generation data overflow chunk' ' + mkdir repo && + cd repo && + git init && + test_commit --date "$UNIX_EPOCH_ZERO" 1 && + test_commit 2 && + test_commit --date "$UNIX_EPOCH_ZERO" 3 && + git commit-graph write --reachable && + graph_read_expect 3 generation_data && + test_commit --date "$FUTURE_DATE" 4 && + test_commit 5 && + test_commit --date "$UNIX_EPOCH_ZERO" 6 && + git branch left && + git reset --hard 3 && + test_commit 7 && + test_commit --date "$FUTURE_DATE" 8 && + test_commit 9 && + git branch right && + git reset --hard 3 && + test_merge M left right && + git commit-graph write --reachable && + graph_read_expect 10 "generation_data generation_data_overflow" && + git commit-graph verify +' + +graph_git_behavior 'overflow 2' repo left right + test_done