From patchwork Wed Mar 15 17:45:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 13176431 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D140FC6FD1D for ; Wed, 15 Mar 2023 17:46:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232602AbjCORqf (ORCPT ); Wed, 15 Mar 2023 13:46:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59240 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232516AbjCORqZ (ORCPT ); Wed, 15 Mar 2023 13:46:25 -0400 Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B2D5551F9C for ; Wed, 15 Mar 2023 10:45:53 -0700 (PDT) Received: by mail-wr1-x429.google.com with SMTP id m2so5287442wrh.6 for ; Wed, 15 Mar 2023 10:45:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1678902345; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=me8GxrmOEisUVf0+uqtjfq9QHX6r3PDzxZZGj2gQv8c=; b=faUwF1VhIiwzSsO2W94HDwe3afR96/VPk3BrRm0fIEgWh9syY3L/kANR+TlLm5WGTL v+BEvH2F2P8ZhA9e2Dd/KZVpVWLTz+zyY+6qVYsFy1GVBjPqbFKk1eiik6dcEzr5BkX2 hYJdQTZPEyxJU2L4FIWIIwlgkkOnUsJ02+OYsIFkQW/MpGDF3w+K5tbXDUHE2ZboGeuK dXqm/nw+3WN+3TUOD3BNRA7erDOXt6uH7lhUeTP0FOw/H/g0d30XpmXl1dHS+DU3jXp7 e5WklQ8E6YLywt28jHAFpwVJwAPPkpztILfXzwdVJ1G+5ORGlyC6/PmcDHZnyTnFPugt DVjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678902345; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=me8GxrmOEisUVf0+uqtjfq9QHX6r3PDzxZZGj2gQv8c=; b=b9/ZOx42bNL389fIsT573r2LIc2VRJ0z+dycbPlzMgEiaq//5KAM785DTqWh7AiNlV DmduSQ3YLGyzrsRKgOs5Y/m5HUGIyJTinC3rLCHc4OoPudnaRmJYY19oH3pySVigP3ij 36mOzcUtNOisnGppTLj25L5Sous6LSd2TBnOKV7EJGUHFYkfUN1Kftz6NoaII9/caJJh Hu+u9/qzHD8rauGO12bG75pKu0Hw35hUDOsT34WQ+wWcnLhwzFe04is2EELeIC/4Lx9o XRXXZeNsk9CsSZLPGFuq5mfbYX+BxFEBlxeSczNZKRjhtyAamGD3V7Fgko+f0AZFO5fQ OaWA== X-Gm-Message-State: AO0yUKW1YgE2EhezsJbiwPbODv+w7BcuXaBg8byEzs6ivuBsUEAk7Z6N ETHPIm9Ndj4VHl2ca80oIMnJjStSykM= X-Google-Smtp-Source: AK7set8WGuxobaSBfa/6fYHZS7cyMH5iW8V5K36ZmzKtHA8Gp65NSNYbPQqDacrIMP6q/Uus9PZqVg== X-Received: by 2002:adf:de10:0:b0:2cf:ea38:ef29 with SMTP id b16-20020adfde10000000b002cfea38ef29mr2681911wrm.44.1678902345619; Wed, 15 Mar 2023 10:45:45 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id a8-20020a5d5088000000b002c7107ce17fsm5265193wrt.3.2023.03.15.10.45.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Mar 2023 10:45:45 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Wed, 15 Mar 2023 17:45:36 +0000 Subject: [PATCH v3 1/8] for-each-ref: add --stdin option Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, me@ttaylorr.com, vdye@github.com, Jeff King , Phillip Wood , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee When a user wishes to input a large list of patterns to 'git for-each-ref' (likely a long list of exact refs) there are frequently system limits on the number of command-line arguments. Add a new --stdin option to instead read the patterns from standard input. Add tests that check that any unrecognized arguments are considered an error when --stdin is provided. Also, an empty pattern list is interpreted as the complete ref set. When reading from stdin, we populate the filter.name_patterns array dynamically as opposed to pointing to the 'argv' array directly. This is simple when using a strvec, as it is NULL-terminated in the same way. We then free the memory directly from the strvec. Helped-by: Phillip Wood Signed-off-by: Derrick Stolee --- Documentation/git-for-each-ref.txt | 7 +++++- builtin/for-each-ref.c | 21 ++++++++++++++++- t/t6300-for-each-ref.sh | 37 ++++++++++++++++++++++++++++++ 3 files changed, 63 insertions(+), 2 deletions(-) diff --git a/Documentation/git-for-each-ref.txt b/Documentation/git-for-each-ref.txt index 6da899c6296..ccdc2911bb9 100644 --- a/Documentation/git-for-each-ref.txt +++ b/Documentation/git-for-each-ref.txt @@ -9,7 +9,8 @@ SYNOPSIS -------- [verse] 'git for-each-ref' [--count=] [--shell|--perl|--python|--tcl] - [(--sort=)...] [--format=] [...] + [(--sort=)...] [--format=] + [ --stdin | ... ] [--points-at=] [--merged[=]] [--no-merged[=]] [--contains[=]] [--no-contains[=]] @@ -32,6 +33,10 @@ OPTIONS literally, in the latter case matching completely or from the beginning up to a slash. +--stdin:: + If `--stdin` is supplied, then the list of patterns is read from + standard input instead of from the argument list. + --count=:: By default the command shows all refs that match ``. This option makes it stop after showing diff --git a/builtin/for-each-ref.c b/builtin/for-each-ref.c index 6f62f40d126..4c5f2324793 100644 --- a/builtin/for-each-ref.c +++ b/builtin/for-each-ref.c @@ -5,6 +5,7 @@ #include "object.h" #include "parse-options.h" #include "ref-filter.h" +#include "strvec.h" static char const * const for_each_ref_usage[] = { N_("git for-each-ref [] []"), @@ -25,6 +26,8 @@ int cmd_for_each_ref(int argc, const char **argv, const char *prefix) struct ref_format format = REF_FORMAT_INIT; struct strbuf output = STRBUF_INIT; struct strbuf err = STRBUF_INIT; + int from_stdin = 0; + struct strvec vec = STRVEC_INIT; struct option opts[] = { OPT_BIT('s', "shell", &format.quote_style, @@ -49,6 +52,7 @@ int cmd_for_each_ref(int argc, const char **argv, const char *prefix) OPT_CONTAINS(&filter.with_commit, N_("print only refs which contain the commit")), OPT_NO_CONTAINS(&filter.no_commit, N_("print only refs which don't contain the commit")), OPT_BOOL(0, "ignore-case", &icase, N_("sorting and filtering are case insensitive")), + OPT_BOOL(0, "stdin", &from_stdin, N_("read reference patterns from stdin")), OPT_END(), }; @@ -75,7 +79,21 @@ int cmd_for_each_ref(int argc, const char **argv, const char *prefix) ref_sorting_set_sort_flags_all(sorting, REF_SORTING_ICASE, icase); filter.ignore_case = icase; - filter.name_patterns = argv; + if (from_stdin) { + struct strbuf line = STRBUF_INIT; + + if (argv[0]) + die(_("unknown arguments supplied with --stdin")); + + while (strbuf_getline(&line, stdin) != EOF) + strvec_push(&vec, line.buf); + + /* vec.v is NULL-terminated, just like 'argv'. */ + filter.name_patterns = vec.v; + } else { + filter.name_patterns = argv; + } + filter.match_as_path = 1; filter_refs(&array, &filter, FILTER_REFS_ALL); ref_array_sort(sorting, &array); @@ -97,5 +115,6 @@ int cmd_for_each_ref(int argc, const char **argv, const char *prefix) free_commit_list(filter.with_commit); free_commit_list(filter.no_commit); ref_sorting_release(sorting); + strvec_clear(&vec); return 0; } diff --git a/t/t6300-for-each-ref.sh b/t/t6300-for-each-ref.sh index c466fd989f1..a58053a54c5 100755 --- a/t/t6300-for-each-ref.sh +++ b/t/t6300-for-each-ref.sh @@ -1464,4 +1464,41 @@ sig_crlf="$(printf "%s" "$sig" | append_cr; echo dummy)" sig_crlf=${sig_crlf%dummy} test_atom refs/tags/fake-sig-crlf contents:signature "$sig_crlf" +test_expect_success 'git for-each-ref --stdin: empty' ' + >in && + git for-each-ref --format="%(refname)" --stdin actual && + git for-each-ref --format="%(refname)" >expect && + test_cmp expect actual +' + +test_expect_success 'git for-each-ref --stdin: fails if extra args' ' + >in && + test_must_fail git for-each-ref --format="%(refname)" \ + --stdin refs/heads/extra err && + grep "unknown arguments supplied with --stdin" err +' + +test_expect_success 'git for-each-ref --stdin: matches' ' + cat >in <<-EOF && + refs/tags/multi* + refs/heads/amb* + EOF + + cat >expect <<-EOF && + refs/heads/ambiguous + refs/tags/multi-ref1-100000-user1 + refs/tags/multi-ref1-100000-user2 + refs/tags/multi-ref1-200000-user1 + refs/tags/multi-ref1-200000-user2 + refs/tags/multi-ref2-100000-user1 + refs/tags/multi-ref2-100000-user2 + refs/tags/multi-ref2-200000-user1 + refs/tags/multi-ref2-200000-user2 + refs/tags/multiline + EOF + + git for-each-ref --format="%(refname)" --stdin actual && + test_cmp expect actual +' + test_done From patchwork Wed Mar 15 17:45:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 13176432 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BCC80C61DA4 for ; Wed, 15 Mar 2023 17:46:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232642AbjCORql (ORCPT ); Wed, 15 Mar 2023 13:46:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59242 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232517AbjCORqZ (ORCPT ); Wed, 15 Mar 2023 13:46:25 -0400 Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E8CDA637E4 for ; Wed, 15 Mar 2023 10:45:54 -0700 (PDT) Received: by mail-wm1-x334.google.com with SMTP id ay8so7861487wmb.1 for ; Wed, 15 Mar 2023 10:45:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1678902346; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=rKehZwzMxp2IDxyfm57q9YaxorHBAAhc4ddH2zrnLQA=; b=Uwynq+hVJKjUgNZdefpjJwbsYmczG+tkVjk/ck7DvWG3QeePtpmkTa20+gTJ1EpY+e +zGQcj6LXfP94notivVvhHMi0qW7MfeYfY8R5KE8J6LbSr4ZcJZBuCx7xz9enI1tmDt/ 4uLagbFxhLhHWo0l7mWOFUxa4cmq2SB/fbl8IsNSfrZaLNz7+IpIoOGzeqtPHVWfPuvJ /JYo4z6JmJYXvkUsKrYKt7ZHIKxryCNWVXuOPOgRmrFJzJV1vC7pHssKN0R/60vEI14k QXL5a0JeMFZukYLnjGLJmC4R6VXZLfgKw9jUtaERly0y/se0wfFj7AzvcGIpTWpg8X1X TGxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678902346; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=rKehZwzMxp2IDxyfm57q9YaxorHBAAhc4ddH2zrnLQA=; b=yvg2krqF0lJlDCgoCN0chS3JMO3bwCHYAdWQnpkQIR7z4927WY1S2XY214wK/vdUu7 CwhjVyL4l/egiwnI1iuRpdhXpdllDR9BPjfoTOWqOc7pa2D7F4h/QxUD/iYGDwwYZbGb pKdw77we6+1GN7donI1oHFDZaNt9+ftgHuKqP1pZtDStOl/uQ+6c1sniZkoPxAluNJch oNuQb3EuoxH7mt+FmniMCS06Jiqn0IJIzgUxQqrqq7s5h3TkL2Uv7Y5KW0fvTs+Zl+6/ oaZfq0o3ALiqGQyRQj2HFwUQ96QG73I0jeLMHl8vNu+pvu2ufX2+S9eV1xrCXrU1WZQF 3ODA== X-Gm-Message-State: AO0yUKVyGQZgq0y9fbJFZ2NTBzhwok/uvxbPmHyexEVfO/wl7jWmKoBe GqiY4Hs2Pk6S0aLooMJHlB0kalCnOEA= X-Google-Smtp-Source: AK7set9uClLOUqphoqIrovCbThSr0+/CJY6CDMk8QFdAfHXRuJsATIRG1J5JZEetPJ/8ADnGcu2GEA== X-Received: by 2002:a05:600c:34c1:b0:3eb:383c:1876 with SMTP id d1-20020a05600c34c100b003eb383c1876mr18691133wmq.6.1678902346262; Wed, 15 Mar 2023 10:45:46 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id b10-20020a5d550a000000b002c706c754fesm5077929wrv.32.2023.03.15.10.45.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Mar 2023 10:45:46 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Wed, 15 Mar 2023 17:45:37 +0000 Subject: [PATCH v3 2/8] for-each-ref: explicitly test no matches Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, me@ttaylorr.com, vdye@github.com, Jeff King , Phillip Wood , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee The for-each-ref builtin can take a list of ref patterns, but if none match, it still succeeds (but with no output). Add an explicit test that demonstrates that behavior. Signed-off-by: Derrick Stolee --- t/t6300-for-each-ref.sh | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/t/t6300-for-each-ref.sh b/t/t6300-for-each-ref.sh index a58053a54c5..6614469d2d6 100755 --- a/t/t6300-for-each-ref.sh +++ b/t/t6300-for-each-ref.sh @@ -1501,4 +1501,17 @@ test_expect_success 'git for-each-ref --stdin: matches' ' test_cmp expect actual ' +test_expect_success 'git for-each-ref with non-existing refs' ' + cat >in <<-EOF && + refs/heads/this-ref-does-not-exist + refs/tags/bogus + EOF + + git for-each-ref --format="%(refname)" --stdin actual && + test_must_be_empty actual && + + xargs git for-each-ref --format="%(refname)" actual && + test_must_be_empty actual +' + test_done From patchwork Wed Mar 15 17:45:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 13176435 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 695C2C7618D for ; Wed, 15 Mar 2023 17:46:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232555AbjCORqr (ORCPT ); Wed, 15 Mar 2023 13:46:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59258 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232115AbjCORqZ (ORCPT ); Wed, 15 Mar 2023 13:46:25 -0400 Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com [IPv6:2a00:1450:4864:20::32e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EAA3565468 for ; Wed, 15 Mar 2023 10:45:54 -0700 (PDT) Received: by mail-wm1-x32e.google.com with SMTP id p16so13113321wmq.5 for ; Wed, 15 Mar 2023 10:45:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1678902347; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=fV0fO6kStkffZHpmPplRcpbqu+IzTW2g86XUKU3MSkw=; b=nbksY50vvwHevRr1QI+2+IuEXMH5/8BVHtNHaWRI6qDYSUso2BkAOqcybqm+Co9UaL bJ6ZdVvrb2QDqZVR+6VUEzrqJY7885QTtM2I+qJ7f1o5jNmS8Klo3ICt6/m+AqYMCKjf kwxE0r9Ag3rAYuyWppqL4kgMsxl5hR6uvGgImPAg+Iv98ucrE4F8xXyClmEYUgGJOhfr JMVKl4rkdek1j8W1u0TTNRuVdH8Qhhy6gV+LlvFoN0ZCnStaORDapPUX37C6qLmWZVyk 9wGJV+gGAxUlwTpP2Kw4QMtaTdgKHkbrl2D/BprddlyJdbov/enbsK5OlDaKjBb3wU9M nz9g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678902347; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=fV0fO6kStkffZHpmPplRcpbqu+IzTW2g86XUKU3MSkw=; b=N9vnV82CvZ51nWnKgN70WeuzUBQPjeowuIftXs8ieXdfLEUrTNqrRgVjEhjT7NHSiT EcYdAW5yP7xVGfpLqdaOmAdufqsGyUW91lGO7XXDPbnOs+1e3NInXoJaKmOXsRklfcoK iQLo+nRX+KljEIWYj+r3JQRz9jcKdx9x8DtU4gO6pnB/NCjZn8S8bxxS1FuJnqT0gIKY iNRQtGx2BMR0pYmDLawpnRvUMDIGAVKw2Uq4sFBaUB3jEGGsYrYdIyG51u29lfXJKtCk F3I13brl5XKsfhqCuj+raF/UgMY7C/gIYqBx61/nXR17uXTOxeTsFwKfT5eNr6+HLzPN sjiA== X-Gm-Message-State: AO0yUKW+ODLgwxHW14DIITdnBn3KKQ2GtQE08QrkqDQRzr+gN49Zf0Gh brKkwvYT9B+JrcgZkOnn1G9MzjEruug= X-Google-Smtp-Source: AK7set/qbyFhBUWmFhJIbEfFMr+5lsAd74TTOYXW4Z5XMXzsNKcTTvhalVRkN3QPfYcYw2jsTYVghw== X-Received: by 2002:a05:600c:1d0b:b0:3ed:24f7:2b48 with SMTP id l11-20020a05600c1d0b00b003ed24f72b48mr5271761wms.8.1678902346867; Wed, 15 Mar 2023 10:45:46 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id j8-20020a05600c1c0800b003ebf73acf9asm7584995wms.3.2023.03.15.10.45.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Mar 2023 10:45:46 -0700 (PDT) Message-Id: <3b15e9df770a118331a1b25f51de8ce97c1b7cab.1678902343.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Wed, 15 Mar 2023 17:45:38 +0000 Subject: [PATCH v3 3/8] commit-graph: combine generation computations Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, me@ttaylorr.com, vdye@github.com, Jeff King , Phillip Wood , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee This patch extracts the common code used to compute topological levels and corrected committer dates into a common routine, compute_reachable_generation_numbers_1(). This new routine dispatches to call the necessary functions to get and set the generation number for a given commit through a vtable (the compute_generation_info struct). Computing the generation number itself is done in compute_generation_from_max(), which dispatches its implementation based on the generation version requested, or issuing a BUG() for unrecognized generation versions. This patch cleans up the two places that currently compute topological levels and corrected commit dates by reducing the amount of duplicated code. It also makes it possible to introduce a function which dynamically computes those values for commits that aren't stored in a commit-graph, which will be required for the forthcoming ahead-behind rewrite. Signed-off-by: Taylor Blau Signed-off-by: Derrick Stolee --- commit-graph.c | 171 +++++++++++++++++++++++++++++++------------------ 1 file changed, 107 insertions(+), 64 deletions(-) diff --git a/commit-graph.c b/commit-graph.c index c11b59f28b3..deccf984a0d 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -1446,24 +1446,53 @@ static void close_reachable(struct write_commit_graph_context *ctx) stop_progress(&ctx->progress); } -static void compute_topological_levels(struct write_commit_graph_context *ctx) +struct compute_generation_info { + struct repository *r; + struct packed_commit_list *commits; + struct progress *progress; + int progress_cnt; + + timestamp_t (*get_generation)(struct commit *c, void *data); + void (*set_generation)(struct commit *c, timestamp_t gen, void *data); + void *data; +}; + +static timestamp_t compute_generation_from_max(struct commit *c, + timestamp_t max_gen, + int generation_version) +{ + switch (generation_version) { + case 1: /* topological levels */ + if (max_gen > GENERATION_NUMBER_V1_MAX - 1) + max_gen = GENERATION_NUMBER_V1_MAX - 1; + return max_gen + 1; + + case 2: /* corrected commit date */ + if (c->date && c->date > max_gen) + max_gen = c->date - 1; + return max_gen + 1; + + default: + BUG("attempting unimplemented version"); + } +} + +static void compute_reachable_generation_numbers_1( + struct compute_generation_info *info, + int generation_version) { int i; struct commit_list *list = NULL; - if (ctx->report_progress) - ctx->progress = start_delayed_progress( - _("Computing commit graph topological levels"), - ctx->commits.nr); - for (i = 0; i < ctx->commits.nr; i++) { - struct commit *c = ctx->commits.list[i]; - uint32_t level; + for (i = 0; i < info->commits->nr; i++) { + struct commit *c = info->commits->list[i]; + timestamp_t gen; + repo_parse_commit(info->r, c); + gen = info->get_generation(c, info->data); - repo_parse_commit(ctx->r, c); - level = *topo_level_slab_at(ctx->topo_levels, c); + display_progress(info->progress, info->progress_cnt + 1); - display_progress(ctx->progress, i + 1); - if (level != GENERATION_NUMBER_ZERO) + if (gen != GENERATION_NUMBER_ZERO && gen != GENERATION_NUMBER_INFINITY) continue; commit_list_insert(c, &list); @@ -1471,38 +1500,91 @@ static void compute_topological_levels(struct write_commit_graph_context *ctx) struct commit *current = list->item; struct commit_list *parent; int all_parents_computed = 1; - uint32_t max_level = 0; + uint32_t max_gen = 0; for (parent = current->parents; parent; parent = parent->next) { - repo_parse_commit(ctx->r, parent->item); - level = *topo_level_slab_at(ctx->topo_levels, parent->item); + repo_parse_commit(info->r, parent->item); + gen = info->get_generation(parent->item, info->data); - if (level == GENERATION_NUMBER_ZERO) { + if (gen == GENERATION_NUMBER_ZERO) { all_parents_computed = 0; commit_list_insert(parent->item, &list); break; } - if (level > max_level) - max_level = level; + if (gen > max_gen) + max_gen = gen; } if (all_parents_computed) { pop_commit(&list); - - if (max_level > GENERATION_NUMBER_V1_MAX - 1) - max_level = GENERATION_NUMBER_V1_MAX - 1; - *topo_level_slab_at(ctx->topo_levels, current) = max_level + 1; + gen = compute_generation_from_max( + current, max_gen, + generation_version); + info->set_generation(current, gen, info->data); } } } +} + +static timestamp_t get_topo_level(struct commit *c, void *data) +{ + struct write_commit_graph_context *ctx = data; + return *topo_level_slab_at(ctx->topo_levels, c); +} + +static void set_topo_level(struct commit *c, timestamp_t t, void *data) +{ + struct write_commit_graph_context *ctx = data; + *topo_level_slab_at(ctx->topo_levels, c) = (uint32_t)t; + display_progress(ctx->progress, ctx->progress_cnt + 1); +} + +static void compute_topological_levels(struct write_commit_graph_context *ctx) +{ + struct compute_generation_info info = { + .r = ctx->r, + .progress = ctx->progress, + .commits = &ctx->commits, + .get_generation = get_topo_level, + .set_generation = set_topo_level, + .data = ctx, + }; + + if (ctx->report_progress) + ctx->progress = start_delayed_progress( + _("Computing commit graph topological levels"), + ctx->commits.nr); + + compute_reachable_generation_numbers_1(&info, 1); + stop_progress(&ctx->progress); } +static timestamp_t get_generation_from_graph_data(struct commit *c, void *data) +{ + return commit_graph_data_at(c)->generation; +} + +static void set_generation_v2(struct commit *c, timestamp_t t, void *data) +{ + struct write_commit_graph_context *ctx = data; + struct commit_graph_data *g = commit_graph_data_at(c); + g->generation = (uint32_t)t; + display_progress(ctx->progress, ctx->progress_cnt + 1); +} + static void compute_generation_numbers(struct write_commit_graph_context *ctx) { int i; - struct commit_list *list = NULL; + struct compute_generation_info info = { + .r = ctx->r, + .progress = ctx->progress, + .commits = &ctx->commits, + .get_generation = get_generation_from_graph_data, + .set_generation = set_generation_v2, + .data = ctx, + }; if (ctx->report_progress) ctx->progress = start_delayed_progress( @@ -1517,47 +1599,7 @@ static void compute_generation_numbers(struct write_commit_graph_context *ctx) } } - for (i = 0; i < ctx->commits.nr; i++) { - struct commit *c = ctx->commits.list[i]; - timestamp_t corrected_commit_date; - - repo_parse_commit(ctx->r, c); - corrected_commit_date = commit_graph_data_at(c)->generation; - - display_progress(ctx->progress, i + 1); - if (corrected_commit_date != GENERATION_NUMBER_ZERO) - continue; - - commit_list_insert(c, &list); - while (list) { - struct commit *current = list->item; - struct commit_list *parent; - int all_parents_computed = 1; - timestamp_t max_corrected_commit_date = 0; - - for (parent = current->parents; parent; parent = parent->next) { - repo_parse_commit(ctx->r, parent->item); - corrected_commit_date = commit_graph_data_at(parent->item)->generation; - - if (corrected_commit_date == GENERATION_NUMBER_ZERO) { - all_parents_computed = 0; - commit_list_insert(parent->item, &list); - break; - } - - if (corrected_commit_date > max_corrected_commit_date) - max_corrected_commit_date = corrected_commit_date; - } - - if (all_parents_computed) { - pop_commit(&list); - - if (current->date && current->date > max_corrected_commit_date) - max_corrected_commit_date = current->date - 1; - commit_graph_data_at(current)->generation = max_corrected_commit_date + 1; - } - } - } + compute_reachable_generation_numbers_1(&info, 2); for (i = 0; i < ctx->commits.nr; i++) { struct commit *c = ctx->commits.list[i]; @@ -1565,6 +1607,7 @@ static void compute_generation_numbers(struct write_commit_graph_context *ctx) if (offset > GENERATION_NUMBER_V2_OFFSET_MAX) ctx->num_generation_data_overflows++; } + stop_progress(&ctx->progress); } From patchwork Wed Mar 15 17:45:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 13176437 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8F131C6FD1D for ; Wed, 15 Mar 2023 17:46:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232679AbjCORqv (ORCPT ); Wed, 15 Mar 2023 13:46:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58814 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232237AbjCORqZ (ORCPT ); Wed, 15 Mar 2023 13:46:25 -0400 Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CCF2F113C1 for ; Wed, 15 Mar 2023 10:45:55 -0700 (PDT) Received: by mail-wm1-x32b.google.com with SMTP id j19-20020a05600c191300b003eb3e1eb0caso1584674wmq.1 for ; Wed, 15 Mar 2023 10:45:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1678902347; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=tP+3SWSR2sc0gnbxgUHoUFsEM9juLjWHmzj1WESClb4=; b=Y0e0/5t7U9K3yOXssPx5GXSzVBYxYhO4KlGMax7u9wXlgBrHFpH94CxOR1Re6LbDOe eHHL6qz6/ZsBVYaq1iEMuu7Gqp0PRfvrwZ8kCCZdghjfvVZyPtctx+d49ot+U8DGY9Y2 8OU+7/nqP+JGJQPXHtQRbiftYo8CqZ4XUXRx/UrmBK7ng+i8Vl3m7saiIuU0dda0H/P+ fAMOqNLQaeIHmKkQIpH5XNWGhIOL3cpjzFpOZD1nQ+f0AG3GqRMknd6+dyg0bZJ3JK63 fSV/IeG9pkn9Atcz9OHU6xQ5vq3pCJQGtEMcNsvG9rqmasRT1bsGDRUTkjOK6Jlgv3y+ jvUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678902347; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=tP+3SWSR2sc0gnbxgUHoUFsEM9juLjWHmzj1WESClb4=; b=TdtRrVoi1Up2B2c1WxeBOiTVRRi8m/8K3zBvI5NrZfjteQkEfm88huPj9y6tFussLO iafCpd0yt/UOwPdoQGVb2L8FvE+zUbMO7nuKKFXgqjG1QS3ztJpk2cSK9WDqd1I58E+t 3GH0rktRha5et0Cu42qh/s3YVTvpb7T0QMvMdI1j+KS6liRg8CHZw/ErbBWQbo8JZ0JQ 6EyQh8i9fxhI93TPsgOnjxNb5IG0b+Wo0OpQhSAvJMjW/fBaIBe3rd6tY1BNBDsQ1JA7 HWJPACOp6/l/vnhey6waoxnY6uxHcpGn6MyGGRYIWC/T05SHQN0Fii9AWEtpaC0UNBIg KrJQ== X-Gm-Message-State: AO0yUKUmC981stB/b9ieGV0OX9iVw1Gdho+2mnHl1rYqohw8EsSj4Wqy AwKCwfApSkzGgEvV8Jp1BNgNEcqvlTU= X-Google-Smtp-Source: AK7set8rMLnLORMfeo9VFMkcFOhZv3T/e+IWmOEa0TjteCspk6bzGthfJW7c3x/AfQTgfOE26LoukQ== X-Received: by 2002:a05:600c:470b:b0:3ed:38e7:af59 with SMTP id v11-20020a05600c470b00b003ed38e7af59mr1792363wmo.39.1678902347432; Wed, 15 Mar 2023 10:45:47 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id e4-20020a05600c4e4400b003e8f0334db8sm2643224wmq.5.2023.03.15.10.45.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Mar 2023 10:45:47 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Wed, 15 Mar 2023 17:45:39 +0000 Subject: [PATCH v3 4/8] commit-graph: return generation from memory Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, me@ttaylorr.com, vdye@github.com, Jeff King , Phillip Wood , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee The commit_graph_generation() method used to report a value of GENERATION_NUMBER_INFINITY if the commit_graph_data_slab had an instance for the given commit but the graph_pos indicated the commit was not in the commit-graph file. Instead, trust the 'generation' member if the commit has a value in the slab _and_ the 'generation' member is non-zero. Otherwise, treat it as GENERATION_NUMBER_INFINITY. This only makes a difference for a very old case for the commit-graph: the very first Git release to write commit-graph files wrote zeroes in the topological level positions. If we are parsing a commit-graph with all zeroes, those commits will now appear to have GENERATION_NUMBER_INFINITY (as if they were not parsed from the commit-graph). I attempted several variations to work around the need for providing an uninitialized 'generation' member, but this was the best one I found. It does require a change to a verification test in t5318 because it reports a different error than the one about non-zero generation numbers. Signed-off-by: Derrick Stolee --- commit-graph.c | 8 +++----- t/t5318-commit-graph.sh | 2 +- 2 files changed, 4 insertions(+), 6 deletions(-) diff --git a/commit-graph.c b/commit-graph.c index deccf984a0d..b4da4e05067 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -116,12 +116,10 @@ timestamp_t commit_graph_generation(const struct commit *c) struct commit_graph_data *data = commit_graph_data_slab_peek(&commit_graph_data_slab, c); - if (!data) - return GENERATION_NUMBER_INFINITY; - else if (data->graph_pos == COMMIT_NOT_FROM_GRAPH) - return GENERATION_NUMBER_INFINITY; + if (data && data->generation) + return data->generation; - return data->generation; + return GENERATION_NUMBER_INFINITY; } static struct commit_graph_data *commit_graph_data_at(const struct commit *c) diff --git a/t/t5318-commit-graph.sh b/t/t5318-commit-graph.sh index 049c5fc8ead..b6e12115786 100755 --- a/t/t5318-commit-graph.sh +++ b/t/t5318-commit-graph.sh @@ -630,7 +630,7 @@ test_expect_success 'detect incorrect generation number' ' test_expect_success 'detect incorrect generation number' ' corrupt_graph_and_verify $GRAPH_BYTE_COMMIT_GENERATION "\01" \ - "non-zero generation number" + "commit-graph generation for commit" ' test_expect_success 'detect incorrect commit date' ' From patchwork Wed Mar 15 17:45:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13176434 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AE123C74A5B for ; Wed, 15 Mar 2023 17:46:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232664AbjCORqp (ORCPT ); Wed, 15 Mar 2023 13:46:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58828 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231879AbjCORqZ (ORCPT ); Wed, 15 Mar 2023 13:46:25 -0400 Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CFDC684F54 for ; Wed, 15 Mar 2023 10:45:55 -0700 (PDT) Received: by mail-wm1-x334.google.com with SMTP id p23-20020a05600c1d9700b003ead4835046so2377719wms.0 for ; Wed, 15 Mar 2023 10:45:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1678902348; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=ofjKpPkVz1iVkizfG/EM1C0RNrO41vW8RqRU6hvCL/A=; b=ejc5w76BByLAvWh5zE4KIKko6u/B7hNkqdwngoIBiCKQqiiLLRPg9GFJdK/bxC54+7 hkcrMCOiWDFaAhYT4UAhst5j923jIW/SoHVxhFbrvVgFRySrHOZ1FF95f4yPL3rEpknt u8LYxfXUKWAmWRyA7wA1KtJx+Hutl4zsG4Vzxx49riPN2Ql+TAhrgNZQvYg3yoQHYkSj YPlWqLJpNi2XadpzVq8Bv9UmtaMkI45AXvku9cV7+oItb+Q6iOsQJMfRkWF0yaTxbOIP m6Gkxs7Owqyyaaj4Gu9Tip3u3fA5lt39Nl0wlkA+HqLDX2Ti12wXq3nzMiOLGzic+S++ 15Rg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678902348; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ofjKpPkVz1iVkizfG/EM1C0RNrO41vW8RqRU6hvCL/A=; b=iXsk0DHJCXo5cZCj+F8VwEc4BHAiiY091KryExrgHq64X4JxZVbL3JnuOfSQU8qbaf MddZ0ITuDYahyRy70cw4PIpksbKw6ISiYDpRzGTvULYv06zKjE1ACFjGhwi7foCHKmkY eLdkzgoVejpfwk1VnvVrj3DXd0NoPN11pq+zOBh0e9O+RkTrirHTEJoclvWpSn77u2ao g5Ea0hXo0eX/EqkM6urDHidPRcuko1JDA1uYYRPehu/PEqQnSRhlIfjI7RNLwFHyfZcc z/cYt9IRvxA7S/WbSX/kB4aMuIZHRuQ5t6iKU8XT6DtUxMbTu0VtrWyIHHEuA/G118WM CEbw== X-Gm-Message-State: AO0yUKW8acmD7bY3LOfMXJ/YYe5w1jEujsJLubKx3UAwP08C3dawCxT6 nQzq5On9AxrmYVJgGKLkpXKNLisUFOs= X-Google-Smtp-Source: AK7set8PeVCsrW6JA+A/eVU23/kMMPE7LVsQLXRywhVjPW2YnmGzqsfSWlQpVJUubGYX/lnXQ4w7mQ== X-Received: by 2002:a05:600c:3b87:b0:3e2:66b:e90e with SMTP id n7-20020a05600c3b8700b003e2066be90emr18330619wms.10.1678902348004; Wed, 15 Mar 2023 10:45:48 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id z4-20020adff744000000b002c55de1c72bsm5289413wrp.62.2023.03.15.10.45.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Mar 2023 10:45:47 -0700 (PDT) Message-Id: In-Reply-To: References: Date: Wed, 15 Mar 2023 17:45:40 +0000 Subject: [PATCH v3 5/8] commit-graph: introduce `ensure_generations_valid()` Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, me@ttaylorr.com, vdye@github.com, Jeff King , Phillip Wood , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Derrick Stolee , Taylor Blau Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Taylor Blau From: Taylor Blau Use the just-introduced compute_reachable_generation_numbers_1() to implement a function which dynamically computes topological levels (or corrected commit dates) for out-of-graph commits. This will be useful for the ahead-behind algorithm we are about to introduce, which needs accurate topological levels on _all_ commits reachable from the tips in order to avoid over-counting. Co-authored-by: Derrick Stolee Signed-off-by: Taylor Blau Signed-off-by: Derrick Stolee --- commit-graph.c | 29 +++++++++++++++++++++++++++++ commit-graph.h | 8 ++++++++ 2 files changed, 37 insertions(+) diff --git a/commit-graph.c b/commit-graph.c index b4da4e05067..0df8d27afc8 100644 --- a/commit-graph.c +++ b/commit-graph.c @@ -1609,6 +1609,35 @@ static void compute_generation_numbers(struct write_commit_graph_context *ctx) stop_progress(&ctx->progress); } +static void set_generation_in_graph_data(struct commit *c, timestamp_t t, + void *data) +{ + commit_graph_data_at(c)->generation = t; +} + +/* + * After this method, all commits reachable from those in the given + * list will have non-zero, non-infinite generation numbers. + */ +void ensure_generations_valid(struct repository *r, + struct commit **commits, size_t nr) +{ + int generation_version = get_configured_generation_version(r); + struct packed_commit_list list = { + .list = commits, + .alloc = nr, + .nr = nr, + }; + struct compute_generation_info info = { + .r = r, + .commits = &list, + .get_generation = get_generation_from_graph_data, + .set_generation = set_generation_in_graph_data, + }; + + compute_reachable_generation_numbers_1(&info, generation_version); +} + static void trace2_bloom_filter_write_statistics(struct write_commit_graph_context *ctx) { trace2_data_intmax("commit-graph", ctx->r, "filter-computed", diff --git a/commit-graph.h b/commit-graph.h index 37faee6b66d..73e182ab2d0 100644 --- a/commit-graph.h +++ b/commit-graph.h @@ -190,4 +190,12 @@ struct commit_graph_data { */ timestamp_t commit_graph_generation(const struct commit *); uint32_t commit_graph_position(const struct commit *); + +/* + * After this method, all commits reachable from those in the given + * list will have non-zero, non-infinite generation numbers. + */ +void ensure_generations_valid(struct repository *r, + struct commit **commits, size_t nr); + #endif From patchwork Wed Mar 15 17:45:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 13176436 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 790AAC61DA4 for ; Wed, 15 Mar 2023 17:46:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232675AbjCORqs (ORCPT ); Wed, 15 Mar 2023 13:46:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59060 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232254AbjCORqZ (ORCPT ); Wed, 15 Mar 2023 13:46:25 -0400 Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 95A9F49DE for ; Wed, 15 Mar 2023 10:45:55 -0700 (PDT) Received: by mail-wm1-x32c.google.com with SMTP id x22so7653269wmj.3 for ; Wed, 15 Mar 2023 10:45:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1678902349; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=5B2aLMen/68CQm3mC7JjUjW8bb5o+wsXYP2E2aSLyoE=; b=CxfydGbRHxtSomoqa45T3nMqMYz1DzTGX5Cf5H0f8FwrZfsCW0V8fUfplnoj+oLPLo c6Pm9WoKqd0U4A0z68ifmGax5kCJnaGK8lji0h4vPG3ksEVLmb9VMhnszE2fb+jAHOj5 RdlL/0ICKlphe9zmG1bd9DUD+lErVjQP/nFbp84jOx9pReYb2eSaEHb5FnHkjvGS8RXB Lm2WhozFfh/PUMGfWXGSzVEf3Oa4ZOWmSzb5wUyHJgOEnyGjlhH7H5zzJdoSMVSGaEvS 1B+dCL+AQ/W07qGID94y3z7e2ywlcbly9QWxtHfO3SJ/6FPTjb44fri9d0OS44eZLK0/ ubYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678902349; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=5B2aLMen/68CQm3mC7JjUjW8bb5o+wsXYP2E2aSLyoE=; b=NH975o0VIq6fsr1kz9+ODrbBsj37SZcZk8b/BKNQc/1v5yngZm2kJHEeNI3xh/JfWL L+OZceUK8qbQN5MPqpFLe4WbrxMRPRqtpkW62AAt1LG7brDi0YqkvCoKIRHA+/jmHRNK BZjJyC8oXjV6A3OHbMmG4tU/DQ8tVHkaY3G7SnJmvUxYF7slnZgVSE00DMFnwuYqjTIO vgoLaeBgVFQQ3lCnDuaVsoTOI7bZAIS3LXMcx0+y2WPPoAeKv9J6vQA9SYQL8H/K69la GWT/95G4KPPJ7CvemFzkqwtjeyJ/h/eyPUKSlP8S4l1Kd0gC/w26hYthBoPrPo08m9Bk Nr7g== X-Gm-Message-State: AO0yUKVohtaABWRhH8JGPiJNukNMBLxieRdG6O1Eujj/yuOXmiOUT2Sz usM0df66Uucp6NDKJFzfBGlyLYFXUAs= X-Google-Smtp-Source: AK7set/AwGnQc/PnLi7JocPi9xJe9zp+bIRh6iZGtwG3E+tw7Q2bodRHki75RCXaE2fa1q3ipmCxnw== X-Received: by 2002:a05:600c:cc6:b0:3ed:2949:9847 with SMTP id fk6-20020a05600c0cc600b003ed29499847mr8741758wmb.10.1678902348693; Wed, 15 Mar 2023 10:45:48 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id g6-20020a05600c288600b003ed2357f7f1sm2495009wmd.37.2023.03.15.10.45.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Mar 2023 10:45:48 -0700 (PDT) Message-Id: <0fb3913810b7d4731707a0129df7f5d57c8ab39f.1678902343.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Wed, 15 Mar 2023 17:45:41 +0000 Subject: [PATCH v3 6/8] commit-reach: implement ahead_behind() logic Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, me@ttaylorr.com, vdye@github.com, Jeff King , Phillip Wood , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Fully implement the commit-counting logic required to determine ahead/behind counts for a batch of commit pairs. This is a new library method within commit-reach.h. This method will be linked to the for-each-ref builtin in the next change. The interface for ahead_behind() uses two arrays. The first array of commits contains the list of all starting points for the walk. This includes all tip commits _and_ base commits. The second array, using the new ahead_behind_count struct, indicates which commits from that initial array form the base/tip pair for the ahead/behind count it will store. This implementation of ahead_behind() allows multiple bases, if desired. Even with multiple bases, there is only one commit walk used for counting the ahead/behind values, saving time when the base/tip ranges overlap significantly. This interface for ahead_behind() also makes it very easy to call ensure_generations_valid() on the entire array of bases and tips. This call is necessary because it is critical that the walk that counts ahead/behind values never walks a commit more than once. Without generation numbers on every commit, there is a possibility that a commit date skew could cause the walk to revisit a commit and then double-count it. For this reason, it is strongly recommended that 'git ahead-behind' is only run in a repository with a commit-graph file that covers most of the reachable commits, storing precomputed generation numbers. If no commit-graph exists, this walk will be much slower as it must walk all reachable commits in ensure_generations_valid() before performing the counting logic. It is possible to detect if generation numbers are available at run time and redirect the implementation to another algorithm that does not require this property. However, that implementation requires a commit walk per base/tip pair _and_ can be slower due to the commit date heuristics required. Such an implementation could be considered in the future if there is a reason to include it, but most Git hosts should already be generating a commit-graph file as part of repository maintenance. Most Git clients should also be generating commit-graph files as part of background maintenance or automatic GCs. Now, let's discuss the ahead/behind counting algorithm. Each commit in the input commit list is associated with a bit position indicating "the ith commit can reach this commit". Each of these commits is associated with a bitmap with its position flipped on and then placed in a queue for walking commit history. We walk commits by popping the commit with maximum generation number out of the queue, guaranteeing that we will never walk a child of that commit in any future steps. As we walk, we load the bitmap for the current commit and perform two main steps. The _second_ step examines each parent of the current commit and adds the current commit's bitmap bits to each parent's bitmap. (We create a new bitmap for the parent if this is our first time seeing that parent.) After adding the bits to the parent's bitmap, the parent is added to the walk queue. Due to this passing of bits to parents, the current commit has a guarantee that the ith bit is enabled on its bitmap if and only if the ith commit can reach the current commit. The first step of the walk is to examine the bitmask on the current commit and decide which ranges the commit is in or not. Due to the "bit pushing" in the second step, we have a guarantee that the ith bit of the current commit's bitmap is on if and only if the ith starting commit can reach it. For each ahead_behind_count struct, check the base_index and tip_index to see if those bits are enabled on the current bitmap. If exactly one bit is enabled, then increment the corresponding 'ahead' or 'behind' count. This increment is the reason we _absolutely need_ to walk commits at most once. The only subtle thing to do with this walk is to check to see if a parent has all bits on in its bitmap, in which case it becomes "stale" and is marked with the STALE bit. This allows queue_has_nonstale() to be the terminating condition of the walk, which greatly reduces the number of commits walked if all of the commits are nearby in history. It avoids walking a large number of common commits when there is a deep history. We also use the helper method insert_no_dup() to add commits to the priority queue without adding them multiple times. This uses the PARENT2 flag. Thus, we must clear both the STALE and PARENT2 bits of all commits, in case ahead_behind() is called multiple times in the same process. Co-authored-by: Taylor Blau Signed-off-by: Taylor Blau Signed-off-by: Derrick Stolee --- commit-reach.c | 96 ++++++++++++++++++++++++++++++++++++++++++++++++++ commit-reach.h | 31 ++++++++++++++++ 2 files changed, 127 insertions(+) diff --git a/commit-reach.c b/commit-reach.c index 2e33c599a82..1e5a1c37fb7 100644 --- a/commit-reach.c +++ b/commit-reach.c @@ -8,6 +8,7 @@ #include "revision.h" #include "tag.h" #include "commit-reach.h" +#include "ewah/ewok.h" /* Remember to update object flag allocation in object.h */ #define PARENT1 (1u<<16) @@ -941,3 +942,98 @@ struct commit_list *get_reachable_subset(struct commit **from, int nr_from, return found_commits; } + +define_commit_slab(bit_arrays, struct bitmap *); +static struct bit_arrays bit_arrays; + +static void insert_no_dup(struct prio_queue *queue, struct commit *c) +{ + if (c->object.flags & PARENT2) + return; + prio_queue_put(queue, c); + c->object.flags |= PARENT2; +} + +static struct bitmap *init_bit_array(struct commit *c, int width) +{ + struct bitmap **bitmap = bit_arrays_at(&bit_arrays, c); + if (!*bitmap) + *bitmap = bitmap_word_alloc(width); + return *bitmap; +} + +static void free_bit_array(struct commit *c) +{ + struct bitmap **bitmap = bit_arrays_at(&bit_arrays, c); + if (!*bitmap) + return; + bitmap_free(*bitmap); + *bitmap = NULL; +} + +void ahead_behind(struct repository *r, + struct commit **commits, size_t commits_nr, + struct ahead_behind_count *counts, size_t counts_nr) +{ + struct prio_queue queue = { .compare = compare_commits_by_gen_then_commit_date }; + size_t width = (commits_nr + BITS_IN_EWORD - 1) / BITS_IN_EWORD; + + if (!commits_nr || !counts_nr) + return; + + for (size_t i = 0; i < counts_nr; i++) { + counts[i].ahead = 0; + counts[i].behind = 0; + } + + ensure_generations_valid(r, commits, commits_nr); + + init_bit_arrays(&bit_arrays); + + for (size_t i = 0; i < commits_nr; i++) { + struct commit *c = commits[i]; + struct bitmap *bitmap = init_bit_array(c, width); + + bitmap_set(bitmap, i); + insert_no_dup(&queue, c); + } + + while (queue_has_nonstale(&queue)) { + struct commit *c = prio_queue_get(&queue); + struct commit_list *p; + struct bitmap *bitmap_c = init_bit_array(c, width); + + for (size_t i = 0; i < counts_nr; i++) { + int reach_from_tip = !!bitmap_get(bitmap_c, counts[i].tip_index); + int reach_from_base = !!bitmap_get(bitmap_c, counts[i].base_index); + + if (reach_from_tip ^ reach_from_base) { + if (reach_from_base) + counts[i].behind++; + else + counts[i].ahead++; + } + } + + for (p = c->parents; p; p = p->next) { + struct bitmap *bitmap_p; + + repo_parse_commit(r, p->item); + + bitmap_p = init_bit_array(p->item, width); + bitmap_or(bitmap_p, bitmap_c); + + if (bitmap_popcount(bitmap_p) == commits_nr) + p->item->object.flags |= STALE; + + insert_no_dup(&queue, p->item); + } + + free_bit_array(c); + } + + /* STALE is used here, PARENT2 is used by insert_no_dup(). */ + repo_clear_commit_marks(r, PARENT2 | STALE); + clear_bit_arrays(&bit_arrays); + clear_prio_queue(&queue); +} diff --git a/commit-reach.h b/commit-reach.h index 148b56fea50..f708c46e523 100644 --- a/commit-reach.h +++ b/commit-reach.h @@ -104,4 +104,35 @@ struct commit_list *get_reachable_subset(struct commit **from, int nr_from, struct commit **to, int nr_to, unsigned int reachable_flag); +struct ahead_behind_count { + /** + * As input, the *_index members indicate which positions in + * the 'tips' array correspond to the tip and base of this + * comparison. + */ + size_t tip_index; + size_t base_index; + + /** + * These values store the computed counts for each side of the + * symmetric difference: + * + * 'ahead' stores the number of commits reachable from the tip + * and not reachable from the base. + * + * 'behind' stores the number of commits reachable from the base + * and not reachable from the tip. + */ + unsigned int ahead; + unsigned int behind; +}; + +/* + * Given an array of commits and an array of ahead_behind_count pairs, + * compute the ahead/behind counts for each pair. + */ +void ahead_behind(struct repository *r, + struct commit **commits, size_t commits_nr, + struct ahead_behind_count *counts, size_t counts_nr); + #endif From patchwork Wed Mar 15 17:45:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 13176439 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC7FEC6FD1D for ; Wed, 15 Mar 2023 17:46:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232574AbjCORqz (ORCPT ); Wed, 15 Mar 2023 13:46:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59064 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232547AbjCORq0 (ORCPT ); Wed, 15 Mar 2023 13:46:26 -0400 Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com [IPv6:2a00:1450:4864:20::32a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CFF27898F9 for ; Wed, 15 Mar 2023 10:45:55 -0700 (PDT) Received: by mail-wm1-x32a.google.com with SMTP id p13-20020a05600c358d00b003ed346d4522so1593269wmq.2 for ; Wed, 15 Mar 2023 10:45:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1678902349; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=wIe7Q+jk2M3rYQGQXxNGrGEcNgOOjOjjPFk9GRl86fg=; b=ZGrtL413h6ksNhrtv67t6Nz6vfaH+RbZw2KIKqCJfGjmr99FO+6LQoxeDysjgr/nMc tkmFs3B6KdCp8V4n7CQocsGXrf0/QAztK2CCWM+Yp8QTqs91lMLt+pyLfHW8uOagumYp 65Vfgy6dXiVVagTBb7EKeNM7EtY+RhbYwYFVWxhzAVXD0XDV5vDJm1iJ2aEP14r0ToeV d2mi1yHfJ8cEp1dqczLzGoQbcuVtL4B/8wTKWEWARnXVVTg8JJDdBCW/lwO80bcQvQhL AsgbwGZiiUCEBgiPZMGa4zi8IMr/pUxpdzrKSDno1h6B0YK/+X4CDclD+VnmsGarOhBr 8JMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678902349; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wIe7Q+jk2M3rYQGQXxNGrGEcNgOOjOjjPFk9GRl86fg=; b=K/yBB57etpvi2SRnFisFUMquvO/7v3I6V7/Er6O8/Ik4icjp7o8+piah8LXyRmEbxz 9QbQNnpfbzDA4GxHzsVW+HxbhMK/lbU5x+XkRsWXQuTJQVNFpk1IRU7pbatjjWlxJ6Yw quRMp1hTyi3iPqy4Kt+g/0G6NE6qZhugeUaWSp2igkdYLdcAulDY80cQstXhhCddhYKD djLNkxzKmQiZJYkz53+GwyDFIyIANZKiduR6YRzNGG9zSBtaPazTx/J7pbj68PNFJqu+ uu5bPKr6QJXxaXnRPTInDf3+XgAG9MKzW98RcIBjyQSf6sWHlq5fSHM85qz+vjwpPYZe NZbA== X-Gm-Message-State: AO0yUKWrkw2OibrWTyHfqje6ne7zjo2BcTSZkf89gJwfuS/1x8YdCnUg qcsXouRTLz0mL/UXRkwyYY2P32LJN+M= X-Google-Smtp-Source: AK7set+lCa51ka4I4aU04hfC2q4bfRYK1OCzYvj/RTC4irB08yylmiBoI4xYp6VovT7w6RKV/VociQ== X-Received: by 2002:a05:600c:3549:b0:3ea:ea3e:2d36 with SMTP id i9-20020a05600c354900b003eaea3e2d36mr18622458wmq.24.1678902349334; Wed, 15 Mar 2023 10:45:49 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id i26-20020a1c541a000000b003ed246f76a2sm2768396wmb.1.2023.03.15.10.45.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Mar 2023 10:45:49 -0700 (PDT) Message-Id: <59cf6759e60cca9dd67a105e65df46f5acbae2fc.1678902343.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Wed, 15 Mar 2023 17:45:42 +0000 Subject: [PATCH v3 7/8] for-each-ref: add ahead-behind format atom Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, me@ttaylorr.com, vdye@github.com, Jeff King , Phillip Wood , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee The previous change implemented the ahead_behind() method, including an algorithm to compute the ahead/behind values for a number of commit tips relative to a number of commit bases. Now, integrate that algorithm as part of 'git for-each-ref' hidden behind a new format atom, ahead-behind. This naturally extends to 'git branch' and 'git tag' builtins, as well. This format allows specifying multiple bases, if so desired, and all matching references are compared against all of those bases. For this reason, failing to read a reference provided from these atoms results in an error. In order to translate the ahead_behind() method information to the format output code in ref-filter.c, we must populate arrays of ahead_behind_count structs. In struct ref_array, we store the full array that will be passed to ahead_behind(). In struct ref_array_item, we store an array of pointers that point to the relvant items within the full array. In this way, we can pull all relevant ahead/behind values directly when formatting output for a specific item. It also ensures the lifetime of the ahead_behind_count structs matches the time that the array is being used. Add specific tests of the ahead/behind counts in t6600-test-reach.sh, as it has an interesting repository shape. In particular, its merging strategy and its use of different commit-graphs would demonstrate over- counting if the ahead_behind() method did not already account for that possibility. Also add tests for the specific for-each-ref, branch, and tag builtins. In the case of 'git tag', there are intersting cases that happen when some of the selected tips are not commits. This requires careful logic around commits_nr in the second loop of filter_ahead_behind(). Also, the test in t7004 is carefully located to avoid being dependent on the GPG prereq. It also avoids using the test_commit helper, as that will add ticks to the time and disrupt the expected timestamps in later tag tests. Also add performance tests in a new p1300-graph-walks.sh script. This will be useful for more uses in the future, but for now compare the ahead-behind counting algorithm in 'git for-each-ref' to the naive implementation by running 'git rev-list --count' processes for each input. For the Git source code repository, the improvement is already obvious: Test this tree --------------------------------------------------------------- 1500.2: ahead-behind counts: git for-each-ref 0.07(0.07+0.00) 1500.3: ahead-behind counts: git branch 0.07(0.06+0.00) 1500.4: ahead-behind counts: git tag 0.07(0.06+0.00) 1500.5: ahead-behind counts: git rev-list 1.32(1.04+0.27) But the standard performance benchmark is the Linux kernel repository, which demosntrates a significant improvement: Test this tree --------------------------------------------------------------- 1500.2: ahead-behind counts: git for-each-ref 0.27(0.24+0.02) 1500.3: ahead-behind counts: git branch 0.27(0.24+0.03) 1500.4: ahead-behind counts: git tag 0.28(0.27+0.01) 1500.5: ahead-behind counts: git rev-list 4.57(4.03+0.54) The 'git rev-list' test exists in this change as a demonstration, but it will be removed in the next change to avoid wasting time on this comparison. Signed-off-by: Derrick Stolee --- Documentation/git-for-each-ref.txt | 5 ++ builtin/branch.c | 1 + builtin/for-each-ref.c | 3 ++ builtin/tag.c | 1 + ref-filter.c | 71 ++++++++++++++++++++++++ ref-filter.h | 26 ++++++++- t/perf/p1500-graph-walks.sh | 45 ++++++++++++++++ t/t3203-branch-output.sh | 14 +++++ t/t6301-for-each-ref-errors.sh | 14 +++++ t/t6600-test-reach.sh | 86 ++++++++++++++++++++++++++++++ t/t7004-tag.sh | 28 ++++++++++ 11 files changed, 293 insertions(+), 1 deletion(-) create mode 100755 t/perf/p1500-graph-walks.sh diff --git a/Documentation/git-for-each-ref.txt b/Documentation/git-for-each-ref.txt index ccdc2911bb9..0713e49b499 100644 --- a/Documentation/git-for-each-ref.txt +++ b/Documentation/git-for-each-ref.txt @@ -222,6 +222,11 @@ worktreepath:: out, if it is checked out in any linked worktree. Empty string otherwise. +ahead-behind::: + Two integers, separated by a space, demonstrating the number of + commits ahead and behind, respectively, when comparing the output + ref to the `` specified in the format. + In addition to the above, for commit and tag objects, the header field names (`tree`, `parent`, `object`, `type`, and `tag`) can be used to specify the value in the header field. diff --git a/builtin/branch.c b/builtin/branch.c index f63fd45edb9..0554d7cebb3 100644 --- a/builtin/branch.c +++ b/builtin/branch.c @@ -448,6 +448,7 @@ static void print_ref_list(struct ref_filter *filter, struct ref_sorting *sortin if (verify_ref_format(format)) die(_("unable to parse format string")); + filter_ahead_behind(the_repository, format, &array); ref_array_sort(sorting, &array); for (i = 0; i < array.nr; i++) { diff --git a/builtin/for-each-ref.c b/builtin/for-each-ref.c index 4c5f2324793..bb559c343ed 100644 --- a/builtin/for-each-ref.c +++ b/builtin/for-each-ref.c @@ -6,6 +6,7 @@ #include "parse-options.h" #include "ref-filter.h" #include "strvec.h" +#include "commit-reach.h" static char const * const for_each_ref_usage[] = { N_("git for-each-ref [] []"), @@ -96,6 +97,8 @@ int cmd_for_each_ref(int argc, const char **argv, const char *prefix) filter.match_as_path = 1; filter_refs(&array, &filter, FILTER_REFS_ALL); + filter_ahead_behind(the_repository, &format, &array); + ref_array_sort(sorting, &array); if (!maxcount || array.nr < maxcount) diff --git a/builtin/tag.c b/builtin/tag.c index d428c45dc8d..1b3f49d7b4c 100644 --- a/builtin/tag.c +++ b/builtin/tag.c @@ -66,6 +66,7 @@ static int list_tags(struct ref_filter *filter, struct ref_sorting *sorting, die(_("unable to parse format string")); filter->with_commit_tag_algo = 1; filter_refs(&array, filter, FILTER_REFS_TAGS); + filter_ahead_behind(the_repository, format, &array); ref_array_sort(sorting, &array); for (i = 0; i < array.nr; i++) { diff --git a/ref-filter.c b/ref-filter.c index f8203c6b052..5a94fea7981 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -158,6 +158,7 @@ enum atom_type { ATOM_THEN, ATOM_ELSE, ATOM_REST, + ATOM_AHEADBEHIND, }; /* @@ -586,6 +587,16 @@ static int rest_atom_parser(struct ref_format *format, struct used_atom *atom, return 0; } +static int ahead_behind_atom_parser(struct ref_format *format, struct used_atom *atom, + const char *arg, struct strbuf *err) +{ + if (!arg) + return strbuf_addf_ret(err, -1, _("expected format: %%(ahead-behind:)")); + + string_list_append(&format->bases, arg); + return 0; +} + static int head_atom_parser(struct ref_format *format, struct used_atom *atom, const char *arg, struct strbuf *err) { @@ -645,6 +656,7 @@ static struct { [ATOM_THEN] = { "then", SOURCE_NONE }, [ATOM_ELSE] = { "else", SOURCE_NONE }, [ATOM_REST] = { "rest", SOURCE_NONE, FIELD_STR, rest_atom_parser }, + [ATOM_AHEADBEHIND] = { "ahead-behind", SOURCE_OTHER, FIELD_STR, ahead_behind_atom_parser }, /* * Please update $__git_ref_fieldlist in git-completion.bash * when you add new atoms @@ -1848,6 +1860,7 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err) struct object *obj; int i; struct object_info empty = OBJECT_INFO_INIT; + int ahead_behind_atoms = 0; CALLOC_ARRAY(ref->value, used_atom_cnt); @@ -1978,6 +1991,16 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err) else v->s = xstrdup(""); continue; + } else if (atom_type == ATOM_AHEADBEHIND) { + if (ref->counts) { + const struct ahead_behind_count *count; + count = ref->counts[ahead_behind_atoms++]; + v->s = xstrfmt("%d %d", count->ahead, count->behind); + } else { + /* Not a commit. */ + v->s = xstrdup(""); + } + continue; } else continue; @@ -2328,6 +2351,7 @@ static void free_array_item(struct ref_array_item *item) free((char *)item->value[i].s); free(item->value); } + free(item->counts); free(item); } @@ -2356,6 +2380,8 @@ void ref_array_clear(struct ref_array *array) free_worktrees(ref_to_worktree_map.worktrees); ref_to_worktree_map.worktrees = NULL; } + + FREE_AND_NULL(array->counts); } #define EXCLUDE_REACHED 0 @@ -2418,6 +2444,51 @@ static void reach_filter(struct ref_array *array, free(to_clear); } +void filter_ahead_behind(struct repository *r, + struct ref_format *format, + struct ref_array *array) +{ + struct commit **commits; + size_t commits_nr = format->bases.nr + array->nr; + + if (!format->bases.nr || !array->nr) + return; + + ALLOC_ARRAY(commits, commits_nr); + for (size_t i = 0; i < format->bases.nr; i++) { + const char *name = format->bases.items[i].string; + commits[i] = lookup_commit_reference_by_name(name); + if (!commits[i]) + die("failed to find '%s'", name); + } + + ALLOC_ARRAY(array->counts, st_mult(format->bases.nr, array->nr)); + + commits_nr = format->bases.nr; + array->counts_nr = 0; + for (size_t i = 0; i < array->nr; i++) { + const char *name = array->items[i]->refname; + commits[commits_nr] = lookup_commit_reference_by_name(name); + + if (!commits[commits_nr]) + continue; + + CALLOC_ARRAY(array->items[i]->counts, format->bases.nr); + for (size_t j = 0; j < format->bases.nr; j++) { + struct ahead_behind_count *count; + count = &array->counts[array->counts_nr++]; + count->tip_index = commits_nr; + count->base_index = j; + + array->items[i]->counts[j] = count; + } + commits_nr++; + } + + ahead_behind(r, commits, commits_nr, array->counts, array->counts_nr); + free(commits); +} + /* * API for filtering a set of refs. Based on the type of refs the user * has requested, we iterate through those refs and apply filters diff --git a/ref-filter.h b/ref-filter.h index aa0eea4ecf5..c9a11495177 100644 --- a/ref-filter.h +++ b/ref-filter.h @@ -5,6 +5,7 @@ #include "refs.h" #include "commit.h" #include "parse-options.h" +#include "string-list.h" /* Quoting styles */ #define QUOTE_NONE 0 @@ -24,6 +25,7 @@ struct atom_value; struct ref_sorting; +struct ahead_behind_count; enum ref_sorting_order { REF_SORTING_REVERSE = 1<<0, @@ -40,6 +42,8 @@ struct ref_array_item { const char *symref; struct commit *commit; struct atom_value *value; + struct ahead_behind_count **counts; + char refname[FLEX_ARRAY]; }; @@ -47,6 +51,9 @@ struct ref_array { int nr, alloc; struct ref_array_item **items; struct rev_info *revs; + + struct ahead_behind_count *counts; + size_t counts_nr; }; struct ref_filter { @@ -80,9 +87,15 @@ struct ref_format { /* Internal state to ref-filter */ int need_color_reset_at_eol; + + /* List of bases for ahead-behind counts. */ + struct string_list bases; }; -#define REF_FORMAT_INIT { .use_color = -1 } +#define REF_FORMAT_INIT { \ + .use_color = -1, \ + .bases = STRING_LIST_INIT_DUP, \ +} /* Macros for checking --merged and --no-merged options */ #define _OPT_MERGED_NO_MERGED(option, filter, h) \ @@ -143,4 +156,15 @@ struct ref_array_item *ref_array_push(struct ref_array *array, const char *refname, const struct object_id *oid); +/* + * If the provided format includes ahead-behind atoms, then compute the + * ahead-behind values for the array of filtered references. Must be + * called after filter_refs() but before outputting the formatted refs. + * + * If this is not called, then any ahead-behind atoms will be blank. + */ +void filter_ahead_behind(struct repository *r, + struct ref_format *format, + struct ref_array *array); + #endif /* REF_FILTER_H */ diff --git a/t/perf/p1500-graph-walks.sh b/t/perf/p1500-graph-walks.sh new file mode 100755 index 00000000000..439a448c2e6 --- /dev/null +++ b/t/perf/p1500-graph-walks.sh @@ -0,0 +1,45 @@ +#!/bin/sh + +test_description='Commit walk performance tests' +. ./perf-lib.sh + +test_perf_large_repo + +test_expect_success 'setup' ' + git for-each-ref --format="%(refname)" "refs/heads/*" "refs/tags/*" >allrefs && + sort -r allrefs | head -n 50 >refs && + for ref in $(cat refs) + do + git branch -f ref-$ref $ref && + echo ref-$ref || + return 1 + done >branches && + for ref in $(cat refs) + do + git tag -f tag-$ref $ref && + echo tag-$ref || + return 1 + done >tags && + git commit-graph write --reachable +' + +test_perf 'ahead-behind counts: git for-each-ref' ' + git for-each-ref --format="%(ahead-behind:HEAD)" --stdin expect <<-\EOF && + (HEAD detached from fromtag) 0 0 + refs/heads/ambiguous 0 0 + refs/heads/branch-one 1 0 + refs/heads/branch-two 0 0 + refs/heads/main 1 0 + refs/heads/ref-to-branch 1 0 + refs/heads/ref-to-remote 1 0 + EOF + git branch --format="%(refname) %(ahead-behind:HEAD)" >actual && + test_cmp expect actual +' + test_expect_success 'git branch with --format=%(rest) must fail' ' test_must_fail git branch --format="%(rest)" >actual ' diff --git a/t/t6301-for-each-ref-errors.sh b/t/t6301-for-each-ref-errors.sh index bfda1f46ad2..47c2c183c30 100755 --- a/t/t6301-for-each-ref-errors.sh +++ b/t/t6301-for-each-ref-errors.sh @@ -54,4 +54,18 @@ test_expect_success 'Missing objects are reported correctly' ' test_must_be_empty brief-err ' +test_expect_success 'ahead-behind requires an argument' ' + test_must_fail git for-each-ref \ + --format="%(ahead-behind)" 2>err && + echo "fatal: expected format: %(ahead-behind:)" >expect && + test_cmp expect err +' + +test_expect_success 'missing ahead-behind base' ' + test_must_fail git for-each-ref \ + --format="%(ahead-behind:refs/heads/missing)" 2>err && + echo "fatal: failed to find '\''refs/heads/missing'\''" >expect && + test_cmp expect err +' + test_done diff --git a/t/t6600-test-reach.sh b/t/t6600-test-reach.sh index 338a9c46a24..0cb50797ef7 100755 --- a/t/t6600-test-reach.sh +++ b/t/t6600-test-reach.sh @@ -443,4 +443,90 @@ test_expect_success 'get_reachable_subset:none' ' test_all_modes get_reachable_subset ' +test_expect_success 'for-each-ref ahead-behind:linear' ' + cat >input <<-\EOF && + refs/heads/commit-1-1 + refs/heads/commit-1-3 + refs/heads/commit-1-5 + refs/heads/commit-1-8 + EOF + cat >expect <<-\EOF && + refs/heads/commit-1-1 0 8 + refs/heads/commit-1-3 0 6 + refs/heads/commit-1-5 0 4 + refs/heads/commit-1-8 0 1 + EOF + run_all_modes git for-each-ref \ + --format="%(refname) %(ahead-behind:commit-1-9)" --stdin +' + +test_expect_success 'for-each-ref ahead-behind:all' ' + cat >input <<-\EOF && + refs/heads/commit-1-1 + refs/heads/commit-2-4 + refs/heads/commit-4-2 + refs/heads/commit-4-4 + EOF + cat >expect <<-\EOF && + refs/heads/commit-1-1 0 24 + refs/heads/commit-2-4 0 17 + refs/heads/commit-4-2 0 17 + refs/heads/commit-4-4 0 9 + EOF + run_all_modes git for-each-ref \ + --format="%(refname) %(ahead-behind:commit-5-5)" --stdin +' + +test_expect_success 'for-each-ref ahead-behind:some' ' + cat >input <<-\EOF && + refs/heads/commit-1-1 + refs/heads/commit-5-3 + refs/heads/commit-4-8 + refs/heads/commit-9-9 + EOF + cat >expect <<-\EOF && + refs/heads/commit-1-1 0 53 + refs/heads/commit-4-8 8 30 + refs/heads/commit-5-3 0 39 + refs/heads/commit-9-9 27 0 + EOF + run_all_modes git for-each-ref \ + --format="%(refname) %(ahead-behind:commit-9-6)" --stdin +' + +test_expect_success 'for-each-ref ahead-behind:some, multibase' ' + cat >input <<-\EOF && + refs/heads/commit-1-1 + refs/heads/commit-5-3 + refs/heads/commit-7-8 + refs/heads/commit-4-8 + refs/heads/commit-9-9 + EOF + cat >expect <<-\EOF && + refs/heads/commit-1-1 0 53 0 53 + refs/heads/commit-4-8 8 30 0 22 + refs/heads/commit-5-3 0 39 0 39 + refs/heads/commit-7-8 14 12 8 6 + refs/heads/commit-9-9 27 0 27 0 + EOF + run_all_modes git for-each-ref \ + --format="%(refname) %(ahead-behind:commit-9-6) %(ahead-behind:commit-6-9)" \ + --stdin +' + +test_expect_success 'for-each-ref ahead-behind:none' ' + cat >input <<-\EOF && + refs/heads/commit-7-5 + refs/heads/commit-4-8 + refs/heads/commit-9-9 + EOF + cat >expect <<-\EOF && + refs/heads/commit-4-8 16 16 + refs/heads/commit-7-5 7 4 + refs/heads/commit-9-9 49 0 + EOF + run_all_modes git for-each-ref \ + --format="%(refname) %(ahead-behind:commit-8-4)" --stdin +' + test_done diff --git a/t/t7004-tag.sh b/t/t7004-tag.sh index 9aa1660651b..04a4b44183d 100755 --- a/t/t7004-tag.sh +++ b/t/t7004-tag.sh @@ -792,6 +792,34 @@ test_expect_success 'annotations for blobs are empty' ' test_cmp expect actual ' +# Run this before doing any signing, so the test has the same results +# regardless of the GPG prereq. +test_expect_success 'git tag --format with ahead-behind' ' + test_when_finished git reset --hard tag-one-line && + git commit --allow-empty -m "left" && + git tag -a -m left tag-left && + git reset --hard HEAD~1 && + git commit --allow-empty -m "right" && + git tag -a -m left tag-right && + + # Use " !" at the end to demonstrate whitespace + # around empty ahead-behind token for tag-blob. + cat >expect <<-EOF && + refs/tags/tag-blob ! + refs/tags/tag-left 1 1 ! + refs/tags/tag-lines 0 1 ! + refs/tags/tag-one-line 0 1 ! + refs/tags/tag-right 0 0 ! + refs/tags/tag-zero-lines 0 1 ! + EOF + git tag -l --format="%(refname) %(ahead-behind:HEAD) !" >actual 2>err && + grep "refs/tags/tag" actual >actual.focus && + test_cmp expect actual.focus && + + # Error reported for tags that point to non-commits. + grep "error: object [0-9a-f]* is a blob, not a commit" err +' + # trying to verify annotated non-signed tags: test_expect_success GPG \ From patchwork Wed Mar 15 17:45:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 13176438 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BDA23C7618D for ; Wed, 15 Mar 2023 17:46:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232685AbjCORqx (ORCPT ); Wed, 15 Mar 2023 13:46:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59284 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232535AbjCORq0 (ORCPT ); Wed, 15 Mar 2023 13:46:26 -0400 Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CD96058C38 for ; Wed, 15 Mar 2023 10:45:55 -0700 (PDT) Received: by mail-wr1-x432.google.com with SMTP id v16so18120012wrn.0 for ; Wed, 15 Mar 2023 10:45:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1678902350; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:from:to:cc:subject:date :message-id:reply-to; bh=Xlrcu1N4bydcXBcNewXNl/4TeMF8Y9hJr5m7xfBsBvs=; b=ZYoJ58VW5J5ot2Q/WglYhAnKIzlbrfhvh0LGMY1rM2XwjUCF5U9QuMGTVkMiOtjMTG VelyCESqvdblA0A0kEPkj5eKJkf1J77mYFC92VHhdIrE7SF+Ezfh5dwE/i9q6Pz1XSAa eLeduzdd3b3TuqiMc7sC6rdkhyUYgTRAuzItnEhQ49tBwITVyErGwuRGXDOXJbUin0PK 5M/El0mqPSOBm/Di1PyPkwaP9SCdeXKMoogdoT8HYM6z54wMlnuRzeHcQj62H3an6uQ6 eI1leJOgROiUmT3DIj8aTXgOufxx25sBkFp2MTysvius5zfF5XJ7CKgMvS0Kwbo0+PL1 8NiA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678902350; h=cc:to:mime-version:content-transfer-encoding:fcc:subject:date:from :references:in-reply-to:message-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Xlrcu1N4bydcXBcNewXNl/4TeMF8Y9hJr5m7xfBsBvs=; b=n30UNv34jlJI02g94XiRwLNpNYE0euY3hXe7NJaBptYeMisxn6vEmE4encBWCM13s1 fLfjpZM5BkVuqBu5ivljZ8AhjuAYc6Zu5a7Ggvfg4JdLLA3m5azEviI3uaHNq8CoYts7 BlU+JkUGFY3WBBItSXuNXCr2TLBTNzdfa+4i+OSfaIwiD3stxZxWA4RjXSkng2r5f1j5 5UCO/Chmvo+kkqSzn1OfGuBcv3NeVjy9A7VtS5uyqsJJy1sfKIAhP5Ugzv2HEKfstJbU 7yqpaJVdq8PiSVqqw/UQzFVrbypbPggxHHWrVWTzm1O5GEodOnlEXsbpTsRufJmIqSfR +WNQ== X-Gm-Message-State: AO0yUKXFWPs0t/n2nn+udElPVjDSdmG7pX1n0OqhdKZYSZPZMX4LG6Bn wbvPukPx5Ol1kjKvCD3RbIiOsHYDjNI= X-Google-Smtp-Source: AK7set9Hg1b2F4GLMqa8WLEVismJ3JkPomFhzT85Y3QPDz09j6Wir0I2CtmFN8i4/8QA7LziYxH6tQ== X-Received: by 2002:a5d:4405:0:b0:2c7:e5f:e0e0 with SMTP id z5-20020a5d4405000000b002c70e5fe0e0mr2629599wrq.65.1678902349888; Wed, 15 Mar 2023 10:45:49 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id b10-20020a5d550a000000b002c706c754fesm5078016wrv.32.2023.03.15.10.45.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Mar 2023 10:45:49 -0700 (PDT) Message-Id: <7476a39331eb1df33c62e8d62a059473d681fa3c.1678902343.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Wed, 15 Mar 2023 17:45:43 +0000 Subject: [PATCH v3 8/8] commit-reach: add tips_reachable_from_bases() Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: gitster@pobox.com, me@ttaylorr.com, vdye@github.com, Jeff King , Phillip Wood , =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Both 'git for-each-ref --merged=' and 'git branch --merged=' use the ref-filter machinery to select references or branches (respectively) that are reachable from a set of commits presented by one or more --merged arguments. This happens within reach_filter(), which uses the revision-walk machinery to walk history in a standard way. However, the commit-reach.c file is full of custom searches that are more efficient, especially for reachability queries that can terminate early when reachability is discovered. Add a new tips_reachable_from_bases() method to commit-reach.c and call it from within reach_filter() in ref-filter.c. This affects both 'git branch' and 'git for-each-ref' as tested in p1500-graph-walks.sh. For the Linux kernel repository, we take an already-fast algorithm and make it even faster: Test HEAD~1 HEAD ------------------------------------------------------------------- 1500.5: contains: git for-each-ref --merged 0.13 0.02 -84.6% 1500.6: contains: git branch --merged 0.14 0.02 -85.7% 1500.7: contains: git tag --merged 0.15 0.03 -80.0% (Note that we remove the iterative 'git rev-list' test from p1500 because it no longer makes sense as a comparison to 'git for-each-ref' and would just waste time running it for these comparisons.) The algorithm is implemented in commit-reach.c in the method tips_reachable_from_base(). This method takes a string_list of tips and assigns the 'util' for each item with the value 1 if the base commit can reach those tips. Like other reachability queries in commit-reach.c, the fastest way to search for "can A reach B?" is to do a depth-first search up to the generation number of B, preferring to explore first parents before later parents. While we must walk all reachable commits up to that generation number when the answer is "no", the depth-first search can answer "yes" much faster than other approaches in most cases. This search becomes trickier when there are multiple targets for the depth-first search. The commits with lower generation number are more likely to be within the history of the start commit, but we don't want to waste time searching commits of low generation number if the commit target with lowest generation number has already been found. The trick here is to take the input commits and sort them by generation number in ascending order. Track the index within this order as min_generation_index. When we find a commit, if its index in the list is equal to min_generation_index, then we can increase the generation number boundary of our search to the next-lowest value in the list. With this mechanism, the number of commits to search is minimized with respect to the depth-first search heuristic. We will walk all commits up to the minimum generation number of a commit that is _not_ reachable from the start, but we will walk only the necessary portion of the depth-first search for the reachable commits of lower generation. Add extra tests for this behavior in t6600-test-reach.sh as the interesting data shape of that repository can sometimes demonstrate corner case bugs. Signed-off-by: Derrick Stolee --- commit-reach.c | 113 ++++++++++++++++++++++++++++++++++++ commit-reach.h | 9 +++ ref-filter.c | 20 ++----- t/perf/p1500-graph-walks.sh | 15 +++-- t/t6600-test-reach.sh | 83 ++++++++++++++++++++++++++ 5 files changed, 219 insertions(+), 21 deletions(-) diff --git a/commit-reach.c b/commit-reach.c index 1e5a1c37fb7..463e9e8fd0e 100644 --- a/commit-reach.c +++ b/commit-reach.c @@ -1037,3 +1037,116 @@ void ahead_behind(struct repository *r, clear_bit_arrays(&bit_arrays); clear_prio_queue(&queue); } + +struct commit_and_index { + struct commit *commit; + unsigned int index; + timestamp_t generation; +}; + +static int compare_commit_and_index_by_generation(const void *va, const void *vb) +{ + const struct commit_and_index *a = (const struct commit_and_index *)va; + const struct commit_and_index *b = (const struct commit_and_index *)vb; + + if (a->generation > b->generation) + return 1; + if (a->generation < b->generation) + return -1; + return 0; +} + +void tips_reachable_from_bases(struct repository *r, + struct commit_list *bases, + struct commit **tips, size_t tips_nr, + int mark) +{ + struct commit_and_index *commits; + size_t min_generation_index = 0; + timestamp_t min_generation; + struct commit_list *stack = NULL; + + if (!bases || !tips || !tips_nr) + return; + + /* + * Do a depth-first search starting at 'bases' to search for the + * tips. Stop at the lowest (un-found) generation number. When + * finding the lowest commit, increase the minimum generation + * number to the next lowest (un-found) generation number. + */ + + CALLOC_ARRAY(commits, tips_nr); + + for (size_t i = 0; i < tips_nr; i++) { + commits[i].commit = tips[i]; + commits[i].index = i; + commits[i].generation = commit_graph_generation(tips[i]); + } + + /* Sort with generation number ascending. */ + QSORT(commits, tips_nr, compare_commit_and_index_by_generation); + min_generation = commits[0].generation; + + while (bases) { + repo_parse_commit(r, bases->item); + commit_list_insert(bases->item, &stack); + bases = bases->next; + } + + while (stack) { + int explored_all_parents = 1; + struct commit_list *p; + struct commit *c = stack->item; + timestamp_t c_gen = commit_graph_generation(c); + + /* Does it match any of our tips? */ + for (size_t j = min_generation_index; j < tips_nr; j++) { + if (c_gen < commits[j].generation) + break; + + if (commits[j].commit == c) { + tips[commits[j].index]->object.flags |= mark; + + if (j == min_generation_index) { + unsigned int k = j + 1; + while (k < tips_nr && + (tips[commits[k].index]->object.flags & mark)) + k++; + + /* Terminate early if all found. */ + if (k >= tips_nr) + goto done; + + min_generation_index = k; + min_generation = commits[k].generation; + } + } + } + + for (p = c->parents; p; p = p->next) { + repo_parse_commit(r, p->item); + + /* Have we already explored this parent? */ + if (p->item->object.flags & SEEN) + continue; + + /* Is it below the current minimum generation? */ + if (commit_graph_generation(p->item) < min_generation) + continue; + + /* Ok, we will explore from here on. */ + p->item->object.flags |= SEEN; + explored_all_parents = 0; + commit_list_insert(p->item, &stack); + break; + } + + if (explored_all_parents) + pop_commit(&stack); + } + +done: + free(commits); + repo_clear_commit_marks(r, SEEN); +} diff --git a/commit-reach.h b/commit-reach.h index f708c46e523..d6321ae700e 100644 --- a/commit-reach.h +++ b/commit-reach.h @@ -135,4 +135,13 @@ void ahead_behind(struct repository *r, struct commit **commits, size_t commits_nr, struct ahead_behind_count *counts, size_t counts_nr); +/* + * For all tip commits, add 'mark' to their flags if and only if they + * are reachable from one of the commits in 'bases'. + */ +void tips_reachable_from_bases(struct repository *r, + struct commit_list *bases, + struct commit **tips, size_t tips_nr, + int mark); + #endif diff --git a/ref-filter.c b/ref-filter.c index 5a94fea7981..c20010a6e94 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -2390,33 +2390,22 @@ static void reach_filter(struct ref_array *array, struct commit_list *check_reachable, int include_reached) { - struct rev_info revs; int i, old_nr; struct commit **to_clear; - struct commit_list *cr; if (!check_reachable) return; CALLOC_ARRAY(to_clear, array->nr); - - repo_init_revisions(the_repository, &revs, NULL); - for (i = 0; i < array->nr; i++) { struct ref_array_item *item = array->items[i]; - add_pending_object(&revs, &item->commit->object, item->refname); to_clear[i] = item->commit; } - for (cr = check_reachable; cr; cr = cr->next) { - struct commit *merge_commit = cr->item; - merge_commit->object.flags |= UNINTERESTING; - add_pending_object(&revs, &merge_commit->object, ""); - } - - revs.limited = 1; - if (prepare_revision_walk(&revs)) - die(_("revision walk setup failed")); + tips_reachable_from_bases(the_repository, + check_reachable, + to_clear, array->nr, + UNINTERESTING); old_nr = array->nr; array->nr = 0; @@ -2440,7 +2429,6 @@ static void reach_filter(struct ref_array *array, clear_commit_marks(merge_commit, ALL_REV_FLAGS); } - release_revisions(&revs); free(to_clear); } diff --git a/t/perf/p1500-graph-walks.sh b/t/perf/p1500-graph-walks.sh index 439a448c2e6..e14e7620cce 100755 --- a/t/perf/p1500-graph-walks.sh +++ b/t/perf/p1500-graph-walks.sh @@ -35,11 +35,16 @@ test_perf 'ahead-behind counts: git tag' ' xargs git tag -l --format="%(ahead-behind:HEAD)" input <<-\EOF && + refs/heads/commit-1-1 + refs/heads/commit-1-3 + refs/heads/commit-1-5 + refs/heads/commit-1-8 + refs/heads/commit-2-1 + refs/heads/commit-5-1 + refs/heads/commit-9-1 + EOF + cat >expect <<-\EOF && + refs/heads/commit-1-1 + refs/heads/commit-1-3 + refs/heads/commit-1-5 + refs/heads/commit-1-8 + EOF + run_all_modes git for-each-ref --merged=commit-1-9 \ + --format="%(refname)" --stdin +' + +test_expect_success 'for-each-ref merged:all' ' + cat >input <<-\EOF && + refs/heads/commit-1-1 + refs/heads/commit-2-4 + refs/heads/commit-4-2 + refs/heads/commit-4-4 + EOF + cat >expect <<-\EOF && + refs/heads/commit-1-1 + refs/heads/commit-2-4 + refs/heads/commit-4-2 + refs/heads/commit-4-4 + EOF + run_all_modes git for-each-ref --merged=commit-5-5 \ + --format="%(refname)" --stdin +' + +test_expect_success 'for-each-ref ahead-behind:some' ' + cat >input <<-\EOF && + refs/heads/commit-1-1 + refs/heads/commit-5-3 + refs/heads/commit-4-8 + refs/heads/commit-9-9 + EOF + cat >expect <<-\EOF && + refs/heads/commit-1-1 + refs/heads/commit-5-3 + EOF + run_all_modes git for-each-ref --merged=commit-9-6 \ + --format="%(refname)" --stdin +' + +test_expect_success 'for-each-ref merged:some, multibase' ' + cat >input <<-\EOF && + refs/heads/commit-1-1 + refs/heads/commit-5-3 + refs/heads/commit-7-8 + refs/heads/commit-4-8 + refs/heads/commit-9-9 + EOF + cat >expect <<-\EOF && + refs/heads/commit-1-1 + refs/heads/commit-4-8 + refs/heads/commit-5-3 + EOF + run_all_modes git for-each-ref \ + --merged=commit-5-8 \ + --merged=commit-8-5 \ + --format="%(refname)" \ + --stdin +' + +test_expect_success 'for-each-ref merged:none' ' + cat >input <<-\EOF && + refs/heads/commit-7-5 + refs/heads/commit-4-8 + refs/heads/commit-9-9 + EOF + >expect && + run_all_modes git for-each-ref --merged=commit-8-4 \ + --format="%(refname)" --stdin +' + test_done