From patchwork Mon Jul 10 21:12:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13307659 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7E7A4EB64DC for ; Mon, 10 Jul 2023 21:12:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231448AbjGJVMK (ORCPT ); Mon, 10 Jul 2023 17:12:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37064 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231464AbjGJVMI (ORCPT ); Mon, 10 Jul 2023 17:12:08 -0400 Received: from mail-yw1-x112a.google.com (mail-yw1-x112a.google.com [IPv6:2607:f8b0:4864:20::112a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2F23DCE for ; Mon, 10 Jul 2023 14:12:07 -0700 (PDT) Received: by mail-yw1-x112a.google.com with SMTP id 00721157ae682-57a551ce7e9so56119897b3.3 for ; Mon, 10 Jul 2023 14:12:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1689023526; x=1691615526; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=gqHIkzCAZCvGA5l56boxPOHKvA3YoAZGs8rTZ0txMjI=; b=LotRoI2NU23Kd04KjKiEamyvHmqKR3mfebVN3BNJ4sme/t5zluM8bPywrOj/mkCrDL 7enPeINmAszV9nI+z4LfKIfhl5//O4e5iR+hfVElO4J2+ytcZwB/rqPcG8T0fhe7kR0U alhZOfn3EERQCIOjrFJGIR03aKQlvfeahopPp3A2jukQsuH9//dBtj035FJZG+TdIdHX UEc8xYZdFqm3qov/shrJYAhKaLMvzJRmXItF8zdmEwYvNP6RtnzWapEQCltVX23qlTDy DuxBGda+8wxIeD65AhpnNrnHngTnhgWuDEZFO9fPoz+wjve8CCs5NoEoWQPKOB79XqkF A3tw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689023526; x=1691615526; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=gqHIkzCAZCvGA5l56boxPOHKvA3YoAZGs8rTZ0txMjI=; b=RN7Fto2WAGbJyWke6zYZd1Wc5GMI1Ih2GXckd3TqJHK+HKGQOBSwpxu6jEWnUqLq2z yNK8dLX4q6wvH+o2BQCdPQKxo5MB3xucBMDJ8DFBXQTBkR8v5cm72WqTwFcWQJX30nA/ Dnza/CVCIrQ9sZw78dY+YRHDOTpXorBwS5UwA+ylMmYasl34cCEWQX8sSM5oOLlII1vQ fFaQ/PTLicmmrqiB8BM5EgekuWIIeYE9jqUQ9HOkONkxqgFeBRUecHHMw97JCn46O072 3P5Qn1N4oAJi99H5GHb/4+ZkvODG3VROUFDZxwJs0qgEogjuOLztzy2i1LvH2NnAHk9l f5BA== X-Gm-Message-State: ABy/qLZcd8aceD3QVkc5/Eo9fZSWfE+t+cg3xMvVBki2lYtz132UwBtC goxNNLOq77zaeDmpWL6RVd2k3MZ4xVHTiE5uE3Yy2A== X-Google-Smtp-Source: APBJJlEtRYa4e/tb7ryOPX0OAn80fiKQLzLQMiUrMpoo1AhoLCq/3LBzOZPuVlNtGHzbTZt6etKOKw== X-Received: by 2002:a81:4642:0:b0:561:d1b0:3f7e with SMTP id t63-20020a814642000000b00561d1b03f7emr12684924ywa.44.1689023526216; Mon, 10 Jul 2023 14:12:06 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id v145-20020a814897000000b0057a8de72338sm184496ywa.68.2023.07.10.14.12.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Jul 2023 14:12:05 -0700 (PDT) Date: Mon, 10 Jul 2023 17:12:05 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Chris Torek , Derrick Stolee , Jeff King , Junio C Hamano , Patrick Steinhardt Subject: [PATCH v5 01/16] refs.c: rename `ref_filter` Message-ID: <5b5ccc40d6b87f0011fc1a88b99a7dda03b36886.1689023520.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Jeff King The refs machinery has its own implementation of a `ref_filter` (used by `for-each-ref`), which is distinct from the `ref-filter.h` API (also used by `for-each-ref`, among other things). Rename the one within refs.c to more clearly indicate its purpose. Signed-off-by: Jeff King Signed-off-by: Taylor Blau --- refs.c | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/refs.c b/refs.c index c029f64982f..876df4931f6 100644 --- a/refs.c +++ b/refs.c @@ -377,8 +377,8 @@ char *resolve_refdup(const char *refname, int resolve_flags, oid, flags); } -/* The argument to filter_refs */ -struct ref_filter { +/* The argument to for_each_filter_refs */ +struct for_each_ref_filter { const char *pattern; const char *prefix; each_ref_fn *fn; @@ -411,10 +411,11 @@ int ref_exists(const char *refname) return refs_ref_exists(get_main_ref_store(the_repository), refname); } -static int filter_refs(const char *refname, const struct object_id *oid, - int flags, void *data) +static int for_each_filter_refs(const char *refname, + const struct object_id *oid, + int flags, void *data) { - struct ref_filter *filter = (struct ref_filter *)data; + struct for_each_ref_filter *filter = data; if (wildmatch(filter->pattern, refname, 0)) return 0; @@ -571,7 +572,7 @@ int for_each_glob_ref_in(each_ref_fn fn, const char *pattern, const char *prefix, void *cb_data) { struct strbuf real_pattern = STRBUF_INIT; - struct ref_filter filter; + struct for_each_ref_filter filter; int ret; if (!prefix && !starts_with(pattern, "refs/")) @@ -591,7 +592,7 @@ int for_each_glob_ref_in(each_ref_fn fn, const char *pattern, filter.prefix = prefix; filter.fn = fn; filter.cb_data = cb_data; - ret = for_each_ref(filter_refs, &filter); + ret = for_each_ref(for_each_filter_refs, &filter); strbuf_release(&real_pattern); return ret; From patchwork Mon Jul 10 21:12:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13307660 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id ED8B2EB64D9 for ; Mon, 10 Jul 2023 21:12:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231450AbjGJVMT (ORCPT ); Mon, 10 Jul 2023 17:12:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37242 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231615AbjGJVMQ (ORCPT ); Mon, 10 Jul 2023 17:12:16 -0400 Received: from mail-yb1-xb29.google.com (mail-yb1-xb29.google.com [IPv6:2607:f8b0:4864:20::b29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4538512E for ; Mon, 10 Jul 2023 14:12:10 -0700 (PDT) Received: by mail-yb1-xb29.google.com with SMTP id 3f1490d57ef6-c84fd44593aso1818419276.0 for ; Mon, 10 Jul 2023 14:12:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1689023529; x=1691615529; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=HJMJHiJIIkXonyBcT4gAkQdNc5kLwjaeH/bH+PGxOec=; b=mInqehien8xJOGCeL0UBE9B/XiUgeCE+b4p3jArHaNzNOAmVm3dCoE5WueSd6i69qd iY0w0XZURg/syo25KqimWlLRG/3z+7TW3QTsLGe+mEU8OSRa+GGkvayp04ZDX8vd/xdP CeC6VC+ewEQoucfjig5e0oJkdcnCiKwsKToiQRgE/6ERPfUAUHWR0sRC5Ub0DgioVO5t D5gDCv6QHFyyWf32BcfgITQxy2e7AevgluB60es9TBW8wzSxTvhO15yhXHc/Mrc3Ta/8 4YjT+PbYRJMRw3uQuwNMTzn/gvOQe7vAKYwD/ouv2bRUS/OdvR2w8txAXO0wnsjjGes/ ot1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689023529; x=1691615529; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=HJMJHiJIIkXonyBcT4gAkQdNc5kLwjaeH/bH+PGxOec=; b=Y2U/kcRtSfjynuMA4/Slp8c3BScrwxuSds92wL/2kj0IfrSKv5zNiz8MIPgTSFHL71 z+CawWxGEMlTuePvjONo19LJfxPUI7BocK2yYnYZZiybbsibpHvhGD5M98Uhak/EGdDh /Tj6jgLzwVAwx4SgkZSuasivjlnaGFr8eBOSNaxUX4brGikJOS9I5LYImVpEEDAklKu3 fsxgUL2NIoioj7SIY55WJSJYhn7bqwNVm8j0KzbN9/rLbqJ2GJfPQLDxOiha4YdfgknC fH/quo18TWFL6ijCuJ+bWZzSFJNkAZXiPv5gFJHa+vRUpjdkA2K6oZC6NsCZ1jBzKO4F 7lVw== X-Gm-Message-State: ABy/qLaJXJG+4HSAOJRRkao1YvwXXBdO49clN4cQ7e8dZ5rpCfmbW/Ej vUbxZ1x7BmSZVJyIQ1bh9hc2co+imcCD6p+dY1U1PA== X-Google-Smtp-Source: APBJJlGTG81sGRMIe8CamL4KH5ooQjaXOLWQD1nHZLQsrulHMLyPPlrVrWh++1tX1xWNmijEM8Ngvw== X-Received: by 2002:a0d:ebc8:0:b0:579:f43e:45d2 with SMTP id u191-20020a0debc8000000b00579f43e45d2mr14325233ywe.37.1689023529161; Mon, 10 Jul 2023 14:12:09 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id y80-20020a0dd653000000b0055a21492192sm181866ywd.115.2023.07.10.14.12.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Jul 2023 14:12:08 -0700 (PDT) Date: Mon, 10 Jul 2023 17:12:07 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Chris Torek , Derrick Stolee , Jeff King , Junio C Hamano , Patrick Steinhardt Subject: [PATCH v5 02/16] ref-filter.h: provide `REF_FILTER_INIT` Message-ID: <0c4e995f1d3bb918ea0d4c0e6aed34aa5e9fe9da.1689023520.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Jeff King Provide a sane initialization value for `struct ref_filter`, which in a subsequent patch will be used to initialize a new field. In the meantime, ensure that the `ref_filter` struct used in the test-helper's `cmd__reach()` is zero-initialized. The lack of initialization is OK, since `commit_contains()` only looks at the single `with_commit_tag_algo` field that *is* initialized directly above. So this does not fix a bug, but rather prevents one from biting us in the future. Signed-off-by: Jeff King Signed-off-by: Taylor Blau --- builtin/branch.c | 3 +-- builtin/for-each-ref.c | 3 +-- builtin/tag.c | 3 +-- ref-filter.h | 3 +++ t/helper/test-reach.c | 2 +- 5 files changed, 7 insertions(+), 7 deletions(-) diff --git a/builtin/branch.c b/builtin/branch.c index e8ff3ecc072..322646f38f5 100644 --- a/builtin/branch.c +++ b/builtin/branch.c @@ -702,7 +702,7 @@ int cmd_branch(int argc, const char **argv, const char *prefix) int reflog = 0, quiet = 0, icase = 0, force = 0, recurse_submodules_explicit = 0; enum branch_track track; - struct ref_filter filter; + struct ref_filter filter = REF_FILTER_INIT; static struct ref_sorting *sorting; struct string_list sorting_options = STRING_LIST_INIT_DUP; struct ref_format format = REF_FORMAT_INIT; @@ -760,7 +760,6 @@ int cmd_branch(int argc, const char **argv, const char *prefix) setup_ref_filter_porcelain_msg(); - memset(&filter, 0, sizeof(filter)); filter.kind = FILTER_REFS_BRANCHES; filter.abbrev = -1; diff --git a/builtin/for-each-ref.c b/builtin/for-each-ref.c index 15409337f83..6b5e313123f 100644 --- a/builtin/for-each-ref.c +++ b/builtin/for-each-ref.c @@ -24,7 +24,7 @@ int cmd_for_each_ref(int argc, const char **argv, const char *prefix) struct string_list sorting_options = STRING_LIST_INIT_DUP; int maxcount = 0, icase = 0, omit_empty = 0; struct ref_array array; - struct ref_filter filter; + struct ref_filter filter = REF_FILTER_INIT; struct ref_format format = REF_FORMAT_INIT; struct strbuf output = STRBUF_INIT; struct strbuf err = STRBUF_INIT; @@ -61,7 +61,6 @@ int cmd_for_each_ref(int argc, const char **argv, const char *prefix) }; memset(&array, 0, sizeof(array)); - memset(&filter, 0, sizeof(filter)); format.format = "%(objectname) %(objecttype)\t%(refname)"; diff --git a/builtin/tag.c b/builtin/tag.c index 7d34af416c7..a624185d105 100644 --- a/builtin/tag.c +++ b/builtin/tag.c @@ -445,7 +445,7 @@ int cmd_tag(int argc, const char **argv, const char *prefix) struct msg_arg msg = { .buf = STRBUF_INIT }; struct ref_transaction *transaction; struct strbuf err = STRBUF_INIT; - struct ref_filter filter; + struct ref_filter filter = REF_FILTER_INIT; struct ref_sorting *sorting; struct string_list sorting_options = STRING_LIST_INIT_DUP; struct ref_format format = REF_FORMAT_INIT; @@ -504,7 +504,6 @@ int cmd_tag(int argc, const char **argv, const char *prefix) git_config(git_tag_config, &sorting_options); memset(&opt, 0, sizeof(opt)); - memset(&filter, 0, sizeof(filter)); filter.lines = -1; opt.sign = -1; diff --git a/ref-filter.h b/ref-filter.h index 430701cfb76..a920f73b297 100644 --- a/ref-filter.h +++ b/ref-filter.h @@ -92,6 +92,9 @@ struct ref_format { struct string_list bases; }; +#define REF_FILTER_INIT { \ + .points_at = OID_ARRAY_INIT, \ +} #define REF_FORMAT_INIT { \ .use_color = -1, \ .bases = STRING_LIST_INIT_DUP, \ diff --git a/t/helper/test-reach.c b/t/helper/test-reach.c index 5b6f2174418..ef58f10c2d6 100644 --- a/t/helper/test-reach.c +++ b/t/helper/test-reach.c @@ -139,7 +139,7 @@ int cmd__reach(int ac, const char **av) printf("%s(X,_,_,0,0):%d\n", av[1], can_all_from_reach_with_flag(&X_obj, 2, 4, 0, 0)); } else if (!strcmp(av[1], "commit_contains")) { - struct ref_filter filter; + struct ref_filter filter = REF_FILTER_INIT; struct contains_cache cache; init_contains_cache(&cache); From patchwork Mon Jul 10 21:12:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13307661 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 54E0CEB64DC for ; Mon, 10 Jul 2023 21:12:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231685AbjGJVMU (ORCPT ); Mon, 10 Jul 2023 17:12:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37240 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231679AbjGJVMR (ORCPT ); Mon, 10 Jul 2023 17:12:17 -0400 Received: from mail-yw1-x1133.google.com (mail-yw1-x1133.google.com [IPv6:2607:f8b0:4864:20::1133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D42F91B8 for ; Mon, 10 Jul 2023 14:12:12 -0700 (PDT) Received: by mail-yw1-x1133.google.com with SMTP id 00721157ae682-5768a7e3adbso89409547b3.0 for ; Mon, 10 Jul 2023 14:12:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1689023532; x=1691615532; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=jMZg/cSyUKlkSuKoinFxU/Ai6RGGHgmeW3kjx1o9cL4=; b=5Aqb7smiKCqiFs1xW5dlW1O75s4NDxQGwuZFyXWX+gtsY6WTpt9NAAoUW+koiJxW9v cbeAiievkseWffyDneM57coKAVPshW9G2eLTlyk6U8PfGDuGpaM8gv+qe91T42CXkvXk iNq0Z/RlFhVNNgG2G1HbqjllcTYlioeRNxWl4+rEeYTA3vN5fQERDhUoODUrI+Zlce/C YcSebZh+OujHwWS7jQkA1DdcyZ/OUY+ewrAlzGkAgmIZczjO249BIPi17ZAy78neAggk 8ycQDr2jAhryH1dKWSXCzPiRDTB/vqApXbGI9jiP1w69NCcsU0uALe/BbKAtsBHlJ4VK DBOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689023532; x=1691615532; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=jMZg/cSyUKlkSuKoinFxU/Ai6RGGHgmeW3kjx1o9cL4=; b=R1BSwFcsXtr2lsB6t3bFDMhopllDzPtdM0qEcPlBkmRJ410WX+aRe8sKWHjdR4ll+q CRP6Cdjv15lV5Jrvjhr3qBWMAsSN+EnhWEFleHs0WYk3psM65ALnaF3RouBNWnIh825t 9pfGnE3Zw08ho5khkgo2mlHH9bn1xu4tQBhVZteW5R3QIhoiMRk+HjBAubvVumiyvBzQ WpPGRWW13iGEjEcRFybHrCQDfjfhTIQbmPwF1orA4MHYs/86R+hUpWVrYriG1VcJTHZG +OW1a4mfqCDFSZdd8AhKbntVj4w5yKBCuUy3IzImAMDr/eYG6jfBVA1ydxvcWK9sy9rc XWSQ== X-Gm-Message-State: ABy/qLbn2SqzbOq35X5+mG9m2uo/mugjjBrGK1EMaEU5Ic6lKDftW6gC n/wnpMg0eKeRJunGZMlCMz1sJkx0hNT/ujRklMJPSg== X-Google-Smtp-Source: APBJJlEMQrdTy49cd2YdLNt5B/nxF7Vo9ZlpiDxKfPMvle403anBv05PViTsBGbKpy2a7+DzDGrHWA== X-Received: by 2002:a0d:dd85:0:b0:56c:e371:aad with SMTP id g127-20020a0ddd85000000b0056ce3710aadmr10848360ywe.14.1689023531934; Mon, 10 Jul 2023 14:12:11 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id o8-20020a0dcc08000000b0054c0f3fd3ddsm193307ywd.30.2023.07.10.14.12.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Jul 2023 14:12:11 -0700 (PDT) Date: Mon, 10 Jul 2023 17:12:10 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Chris Torek , Derrick Stolee , Jeff King , Junio C Hamano , Patrick Steinhardt Subject: [PATCH v5 03/16] ref-filter: clear reachable list pointers after freeing Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Jeff King In `reach_filter()`, we pop all commits from the reachable lists, leaving them empty. But because we're operating on a list pointer that was passed by value, the original `filter.reachable_from` and `filter.unreachable_from` pointers are left dangling. As is the case with the previous commit, nobody touches either of these fields after calling `reach_filter()`, so leaving them dangling is OK. But this future proofs against dangerous situations. Signed-off-by: Jeff King Signed-off-by: Taylor Blau --- ref-filter.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/ref-filter.c b/ref-filter.c index e0d03a9f8e9..4dd6575400c 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -2418,13 +2418,13 @@ void ref_array_clear(struct ref_array *array) #define EXCLUDE_REACHED 0 #define INCLUDE_REACHED 1 static void reach_filter(struct ref_array *array, - struct commit_list *check_reachable, + struct commit_list **check_reachable, int include_reached) { int i, old_nr; struct commit **to_clear; - if (!check_reachable) + if (!*check_reachable) return; CALLOC_ARRAY(to_clear, array->nr); @@ -2434,7 +2434,7 @@ static void reach_filter(struct ref_array *array, } tips_reachable_from_bases(the_repository, - check_reachable, + *check_reachable, to_clear, array->nr, UNINTERESTING); @@ -2455,8 +2455,8 @@ static void reach_filter(struct ref_array *array, clear_commit_marks_many(old_nr, to_clear, ALL_REV_FLAGS); - while (check_reachable) { - struct commit *merge_commit = pop_commit(&check_reachable); + while (*check_reachable) { + struct commit *merge_commit = pop_commit(check_reachable); clear_commit_marks(merge_commit, ALL_REV_FLAGS); } @@ -2553,8 +2553,8 @@ int filter_refs(struct ref_array *array, struct ref_filter *filter, unsigned int clear_contains_cache(&ref_cbdata.no_contains_cache); /* Filters that need revision walking */ - reach_filter(array, filter->reachable_from, INCLUDE_REACHED); - reach_filter(array, filter->unreachable_from, EXCLUDE_REACHED); + reach_filter(array, &filter->reachable_from, INCLUDE_REACHED); + reach_filter(array, &filter->unreachable_from, EXCLUDE_REACHED); save_commit_buffer = save_commit_buffer_orig; return ret; From patchwork Mon Jul 10 21:12:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13307662 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9C1DEEB64D9 for ; Mon, 10 Jul 2023 21:12:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231668AbjGJVMa (ORCPT ); Mon, 10 Jul 2023 17:12:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37254 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231491AbjGJVMU (ORCPT ); Mon, 10 Jul 2023 17:12:20 -0400 Received: from mail-yw1-x1134.google.com (mail-yw1-x1134.google.com [IPv6:2607:f8b0:4864:20::1134]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 232C6E58 for ; Mon, 10 Jul 2023 14:12:16 -0700 (PDT) Received: by mail-yw1-x1134.google.com with SMTP id 00721157ae682-579e212668fso72203267b3.1 for ; Mon, 10 Jul 2023 14:12:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1689023535; x=1691615535; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=X9dGpDeDNIt0aOY5wWYLp6BsVyYhTCd6Io6uYGoTqjE=; b=4UxPXlCKrXhk153cnVTSxGIUgZfdX9fj8aWazMfaijJ2Rv8nl4VyqF/s69BZW2B7tx jnJpbm48Mf2SxMG9MRgFU9oTogXDotZtUMxsOCjQVzmwfm0/6jRUZX3yzVZrDCR4+q+C 39+RWjgkW4e0ZBtUvHOq0Y1hSUKFMB5NopuD/RPlB4rw0d8D7zWzr5GIOsl1Vy/uZylC vkOaMUAfJOc7p0g2eTv5Tk7r/w3NwC2+qC8AUXz58WYlWsbfMqSZGzzpd+6pO7xHI4QG 0XqRRdI+xWS6JpzlpdpsyxtXSCG4+vKpS9wYOcXpN7xhpBKwlVvi1qnWurVfjBLcdFkf HkkA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689023535; x=1691615535; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=X9dGpDeDNIt0aOY5wWYLp6BsVyYhTCd6Io6uYGoTqjE=; b=CAyKhtD2lZZcNlab6ptrkply0uSDLQWHk737VInTTxhJ3MrrnSpmP/po47tHwc/1Fe hce4Zdumctt1u3YJDtIE/jX6nym5q45zyuuNXvVZNBXhlNGcq5eX7HF/O+DI0FURf3J7 RDDJu2ITllyHBeGJ6ZJwIut0saCiQwVq4B5Hk9CIUCyueYe6/AteGJ4swx4urmu9ZJ/y C0w6fiQmdFUHUIGiRvoA6Zn+wEFzgHxeZ8o3ha1+azBjvsc6BES/K7Prv8Kqfa66MPzD Bz4gxBokznjRr31tzVIsoGROPBt1obV0OtGjKoQp9K17KDpmDhNzOKkvdGSutu0+Jk7W 0Z+Q== X-Gm-Message-State: ABy/qLawgcuCftgArE58/l4Bkl8p4KkD164xpi+hyqHwC2envU2sMFcF a4qM07nZRu3OZP6ITYrNW3zarKyqe3oIJZLgx+3lMg== X-Google-Smtp-Source: APBJJlGWcmeUAXot1Wix0//evfJYXYcozGZfK+xNPsBSMtKpGBQH38EXNddVq/VvyYI6WFUuN+o5Nw== X-Received: by 2002:a81:4913:0:b0:579:dfc5:20cc with SMTP id w19-20020a814913000000b00579dfc520ccmr13320772ywa.17.1689023535158; Mon, 10 Jul 2023 14:12:15 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id q63-20020a817542000000b0057a68b6c2f4sm179810ywc.133.2023.07.10.14.12.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Jul 2023 14:12:14 -0700 (PDT) Date: Mon, 10 Jul 2023 17:12:13 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Chris Torek , Derrick Stolee , Jeff King , Junio C Hamano , Patrick Steinhardt Subject: [PATCH v5 04/16] ref-filter: add `ref_filter_clear()` Message-ID: <00a4532d54d81953c931bdd948e708f90c0df15a.1689023520.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Jeff King We did not bother to clean up at all in `git branch` or `git tag`, and `git for-each-ref` only cleans up a couple of members. Add and call `ref_filter_clear()` when cleaning up a `struct ref_filter`. Running this patch (without any test changes) indicates a couple of now leak-free tests. This was found by running: $ make SANITIZE=leak $ make -C t GIT_TEST_PASSING_SANITIZE_LEAK=check GIT_TEST_OPTS=--immediate (Note that the `reachable_from` and `unreachable_from` lists should be cleaned as they are used. So this is just covering any case where we might bail before running the reachability check.) Signed-off-by: Jeff King Signed-off-by: Taylor Blau --- builtin/branch.c | 1 + builtin/for-each-ref.c | 3 +-- builtin/tag.c | 1 + ref-filter.c | 16 ++++++++++++++++ ref-filter.h | 3 +++ t/t0041-usage.sh | 1 + t/t3402-rebase-merge.sh | 1 + 7 files changed, 24 insertions(+), 2 deletions(-) diff --git a/builtin/branch.c b/builtin/branch.c index 322646f38f5..f06df4be7a5 100644 --- a/builtin/branch.c +++ b/builtin/branch.c @@ -855,6 +855,7 @@ int cmd_branch(int argc, const char **argv, const char *prefix) print_columns(&output, colopts, NULL); string_list_clear(&output, 0); ref_sorting_release(sorting); + ref_filter_clear(&filter); return 0; } else if (edit_description) { const char *branch_name; diff --git a/builtin/for-each-ref.c b/builtin/for-each-ref.c index 6b5e313123f..ccceba54aa1 100644 --- a/builtin/for-each-ref.c +++ b/builtin/for-each-ref.c @@ -120,8 +120,7 @@ int cmd_for_each_ref(int argc, const char **argv, const char *prefix) strbuf_release(&err); strbuf_release(&output); ref_array_clear(&array); - free_commit_list(filter.with_commit); - free_commit_list(filter.no_commit); + ref_filter_clear(&filter); ref_sorting_release(sorting); strvec_clear(&vec); return 0; diff --git a/builtin/tag.c b/builtin/tag.c index a624185d105..f33600c0506 100644 --- a/builtin/tag.c +++ b/builtin/tag.c @@ -659,6 +659,7 @@ int cmd_tag(int argc, const char **argv, const char *prefix) cleanup: ref_sorting_release(sorting); + ref_filter_clear(&filter); strbuf_release(&buf); strbuf_release(&ref); strbuf_release(&reflog_msg); diff --git a/ref-filter.c b/ref-filter.c index 4dd6575400c..84cd3921307 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -2866,3 +2866,19 @@ int parse_opt_merge_filter(const struct option *opt, const char *arg, int unset) return 0; } + +void ref_filter_init(struct ref_filter *filter) +{ + struct ref_filter blank = REF_FILTER_INIT; + memcpy(filter, &blank, sizeof(blank)); +} + +void ref_filter_clear(struct ref_filter *filter) +{ + oid_array_clear(&filter->points_at); + free_commit_list(filter->with_commit); + free_commit_list(filter->no_commit); + free_commit_list(filter->reachable_from); + free_commit_list(filter->unreachable_from); + ref_filter_init(filter); +} diff --git a/ref-filter.h b/ref-filter.h index a920f73b297..160b8072245 100644 --- a/ref-filter.h +++ b/ref-filter.h @@ -170,4 +170,7 @@ void filter_ahead_behind(struct repository *r, struct ref_format *format, struct ref_array *array); +void ref_filter_init(struct ref_filter *filter); +void ref_filter_clear(struct ref_filter *filter); + #endif /* REF_FILTER_H */ diff --git a/t/t0041-usage.sh b/t/t0041-usage.sh index c4fc34eb18e..9ea974b0c6c 100755 --- a/t/t0041-usage.sh +++ b/t/t0041-usage.sh @@ -5,6 +5,7 @@ test_description='Test commands behavior when given invalid argument value' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh test_expect_success 'setup ' ' diff --git a/t/t3402-rebase-merge.sh b/t/t3402-rebase-merge.sh index 79b0640c004..e9e03ca4b5e 100755 --- a/t/t3402-rebase-merge.sh +++ b/t/t3402-rebase-merge.sh @@ -8,6 +8,7 @@ test_description='git rebase --merge test' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh T="A quick brown fox From patchwork Mon Jul 10 21:12:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13307663 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C8654EB64DA for ; Mon, 10 Jul 2023 21:12:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231708AbjGJVMb (ORCPT ); Mon, 10 Jul 2023 17:12:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37388 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231787AbjGJVMX (ORCPT ); Mon, 10 Jul 2023 17:12:23 -0400 Received: from mail-yw1-x112d.google.com (mail-yw1-x112d.google.com [IPv6:2607:f8b0:4864:20::112d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2A51F18D for ; Mon, 10 Jul 2023 14:12:19 -0700 (PDT) Received: by mail-yw1-x112d.google.com with SMTP id 00721157ae682-5700b15c12fso60369867b3.1 for ; Mon, 10 Jul 2023 14:12:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1689023538; x=1691615538; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=cUVzHTQWi61QjEMUzZyauEUKckXze3BriO7vJyiP3WM=; b=QT8Vdp1XYQ5PiqQ7Hmzm6kxuB0XaWHXE0P0Lz0rqOlwrEkkynMTA/v4UGI8VAwomG2 vbeX0w/YiILqMCYa9Z9n8a6vDrKUfBAix4DzieDMGuHhoV4HXwgVzNTVtYXOvhdJxAI2 FxWInzl0eePHSw61k3KWd0NiUgic/9815PWFJrT9zy5YQbeaPp7gLFNJecZXXRu5l/xq PlKRgWHvleBA+W7HKpuJoqhbXsj1ct22IgiqggPEG+UTQQ9+pHB2ChCPUOBoLK6ztYDa 31CkhRGRiiP6gQdSTy8vKKX6+l+7jvK4fRSIOBQJnIR3jiEJqWv7kypHyEhkyyD5YhLz lGAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689023538; x=1691615538; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=cUVzHTQWi61QjEMUzZyauEUKckXze3BriO7vJyiP3WM=; b=X8RtVRdqGv+qTU7QRmE40jnLCd4j5ptvmZfeg4EJLiy4qUclR+IaFAZWGiFbOl1I8p 2JL3CxQ0iONJ1n1K5VvQrIs6AiphWfV/OqCwwT+UUNd+Uqulw54ysJILVC8vvrcv67b9 +B5yTGxJ8IRp97YPYgm0VYPfNiFFgEc7h2GQ9zH0c3j6b858Sdxjil/d2u0IkCAZltP6 rLsIa0liLrERiQ+me024fUsi6UYi6Vq0fhw2POwNxB/YyS/pmrboCimJcjmrVi+QoCJQ 3ELgkIg5e3YxRNmvl+rlrvZSAOwR5cOMWIe6w3D65OlcyIf3PfateRmYFlZNdwSLpLeE mDWg== X-Gm-Message-State: ABy/qLZkQ9kn4cilJ+ySLKDs6eWSmHcpNFOqP3vVHOhyJUfFqLTZvhrt vkX56VYvSVUUAdtthyxQ/s0XDLZG4+UXj9fllv/wrA== X-Google-Smtp-Source: APBJJlF9iWyzFM6oFgHLFRBvZYJ2LGLGzB0immVxzBrzpEPF6SH6ftvDlgf9l20RP6Cymcp7H8xy7Q== X-Received: by 2002:a81:49c9:0:b0:56d:4d7:4a45 with SMTP id w192-20020a8149c9000000b0056d04d74a45mr11497276ywa.52.1689023537995; Mon, 10 Jul 2023 14:12:17 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id v186-20020a8161c3000000b005612fc707bfsm179830ywb.120.2023.07.10.14.12.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Jul 2023 14:12:17 -0700 (PDT) Date: Mon, 10 Jul 2023 17:12:16 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Chris Torek , Derrick Stolee , Jeff King , Junio C Hamano , Patrick Steinhardt Subject: [PATCH v5 05/16] ref-filter.c: parameterize match functions over patterns Message-ID: <3ab03ac20dfd83acd5e00e027cf0da362529b7e1.1689023520.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Jeff King `match_pattern()` and `match_name_as_path()` both take a `struct ref_filter *`, and then store a stack variable `patterns` pointing at `filter->patterns`. The subsequent patch will add a new array of patterns to match over (the excluded patterns, via a new `git for-each-ref --exclude` option), treating the return value of these functions differently depending on which patterns are being used to match. Tweak `match_pattern()` and `match_name_as_path()` to take an array of patterns to prepare for passing either in. Once we start passing either in, `match_pattern()` will have little to do with a particular `struct ref_filter *` instance. To clarify this, drop it from the argument list, and replace it with the only bit of the `ref_filter` that we care about (`filter->ignore_case`). Co-authored-by: Taylor Blau Signed-off-by: Jeff King Signed-off-by: Taylor Blau --- ref-filter.c | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/ref-filter.c b/ref-filter.c index 84cd3921307..c70a430adf3 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -2104,12 +2104,12 @@ static int get_ref_atom_value(struct ref_array_item *ref, int atom, * matches a pattern "refs/heads/mas") or a wildcard (e.g. the same ref * matches "refs/heads/mas*", too). */ -static int match_pattern(const struct ref_filter *filter, const char *refname) +static int match_pattern(const char **patterns, const char *refname, + int ignore_case) { - const char **patterns = filter->name_patterns; unsigned flags = 0; - if (filter->ignore_case) + if (ignore_case) flags |= WM_CASEFOLD; /* @@ -2134,13 +2134,13 @@ static int match_pattern(const struct ref_filter *filter, const char *refname) * matches a pattern "refs/heads/" but not "refs/heads/m") or a * wildcard (e.g. the same ref matches "refs/heads/m*", too). */ -static int match_name_as_path(const struct ref_filter *filter, const char *refname) +static int match_name_as_path(const char **pattern, const char *refname, + int ignore_case) { - const char **pattern = filter->name_patterns; int namelen = strlen(refname); unsigned flags = WM_PATHNAME; - if (filter->ignore_case) + if (ignore_case) flags |= WM_CASEFOLD; for (; *pattern; pattern++) { @@ -2165,8 +2165,10 @@ static int filter_pattern_match(struct ref_filter *filter, const char *refname) if (!*filter->name_patterns) return 1; /* No pattern always matches */ if (filter->match_as_path) - return match_name_as_path(filter, refname); - return match_pattern(filter, refname); + return match_name_as_path(filter->name_patterns, refname, + filter->ignore_case); + return match_pattern(filter->name_patterns, refname, + filter->ignore_case); } /* From patchwork Mon Jul 10 21:12:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13307664 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7E7A3EB64DA for ; Mon, 10 Jul 2023 21:12:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231477AbjGJVMk (ORCPT ); Mon, 10 Jul 2023 17:12:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37232 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231639AbjGJVM2 (ORCPT ); Mon, 10 Jul 2023 17:12:28 -0400 Received: from mail-yb1-xb34.google.com (mail-yb1-xb34.google.com [IPv6:2607:f8b0:4864:20::b34]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F32F81BB for ; Mon, 10 Jul 2023 14:12:21 -0700 (PDT) Received: by mail-yb1-xb34.google.com with SMTP id 3f1490d57ef6-c4dd86f5d78so7239292276.0 for ; Mon, 10 Jul 2023 14:12:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1689023541; x=1691615541; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=O8CaQjEoLZ5MtfgNHHaoNqgpBmPZyLa4AGXk38fo4rE=; b=CA9X5TdjQKok22piDpSj2zocC61rZ6iAtbYXv/lH9u57rsMU+688UWJ4XobGl2U0bT WjQ85Y1VWvZXg8IHSbVBa/PzdeS56WYkdXbyQvhq4pKS1v6wigNBUbhPj4dnW9L1RNNT yeuNdqVVjgGC9FefrU/jFNLRHIMjkSrxtfCEowoIaO4vw4n/NM9lQd4NPY0tc6kFpbgb D5EZsIoPBZ1ScXFk8etPkG0z5L5Ivfp6IVoWGEFZ/t5rqPDz+KEBxyZF7Ti4t4HuxJX0 g3MYDQFugiPdgRgmtKtR5HE/vdNYb+H6s2AddUVKvzkSDNGd4mxB7jT4x/yl3xW4rH3Y Ky3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689023541; x=1691615541; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=O8CaQjEoLZ5MtfgNHHaoNqgpBmPZyLa4AGXk38fo4rE=; b=goLAb2tKNPmhoiJFs+nfqa9V7bMYk3Jgz+0HaiCxkMGplyeekPYPSqNFKrzwYUcvXs G+WPhlRH12mUF/KLJXI9bEjCPdeXULktZTjTdVzZY+0JIPA/zW8RkwMyKxJ5PrYAQrd1 kaW0JxVSecp+EWiIaGDFZXCGbL8xz1CZOc9audYYwwdgForMCiNAYQG8JsDGz1Eyzfox c9OpxL1Ha88BNm57dnek/lLhghLiCKXYWjIdSIN+yKOR7hDD8T50Vof1gOlBs9LmL0ro 5s7ipQe6XtdiawmTwt1XOuiH8bMSGqnaWZHeGQ5uaXNQlAFEl/UbS5LYVK5NKKC3jqi1 aOlA== X-Gm-Message-State: ABy/qLbnHKmncyp56fOVTn32ipNeRtJW75/WyN+jq5jdAEQoDJY52zB6 gEbpHSs1qqzx7Eq7bGNzalXfewsgZ/g5Lo08/zwI1Q== X-Google-Smtp-Source: APBJJlFV0n95DXDQwxqmJdXH2MPxAOMyMlaxmkkTBI/+SqHkZTwUf2+71H9ekaPlfopOqd8GSWQMsw== X-Received: by 2002:a25:ce85:0:b0:c1b:6de8:5395 with SMTP id x127-20020a25ce85000000b00c1b6de85395mr14761758ybe.16.1689023540991; Mon, 10 Jul 2023 14:12:20 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id e199-20020a2550d0000000b00bf44703efd3sm150014ybb.6.2023.07.10.14.12.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Jul 2023 14:12:20 -0700 (PDT) Date: Mon, 10 Jul 2023 17:12:19 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Chris Torek , Derrick Stolee , Jeff King , Junio C Hamano , Patrick Steinhardt Subject: [PATCH v5 06/16] builtin/for-each-ref.c: add `--exclude` option Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When using `for-each-ref`, it is sometimes convenient for the caller to be able to exclude certain parts of the references. For example, if there are many `refs/__hidden__/*` references, the caller may want to emit all references *except* the hidden ones. Currently, the only way to do this is to post-process the output, like: $ git for-each-ref --format='%(refname)' | grep -v '^refs/hidden/' Which is do-able, but requires processing a potentially large quantity of references. Teach `git for-each-ref` a new `--exclude=` option, which excludes references from the results if they match one or more excluded patterns. This patch provides a naive implementation where the `ref_filter` still sees all references (including ones that it will discard) and is left to check whether each reference matches any excluded pattern(s) before emitting them. By culling out references we know the caller doesn't care about, we can avoid allocating memory for their storage, as well as spending time sorting the output (among other things). Even the naive implementation provides a significant speed-up on a modified copy of linux.git (that has a hidden ref pointing at each commit): $ hyperfine \ 'git.compile for-each-ref --format="%(objectname) %(refname)" | grep -vE "[0-9a-f]{40} refs/pull/"' \ 'git.compile for-each-ref --format="%(objectname) %(refname)" --exclude refs/pull/' Benchmark 1: git.compile for-each-ref --format="%(objectname) %(refname)" | grep -vE "[0-9a-f]{40} refs/pull/" Time (mean ± σ): 820.1 ms ± 2.0 ms [User: 703.7 ms, System: 152.0 ms] Range (min … max): 817.7 ms … 823.3 ms 10 runs Benchmark 2: git.compile for-each-ref --format="%(objectname) %(refname)" --exclude refs/pull/ Time (mean ± σ): 106.6 ms ± 1.1 ms [User: 99.4 ms, System: 7.1 ms] Range (min … max): 104.7 ms … 109.1 ms 27 runs Summary 'git.compile for-each-ref --format="%(objectname) %(refname)" --exclude refs/pull/' ran 7.69 ± 0.08 times faster than 'git.compile for-each-ref --format="%(objectname) %(refname)" | grep -vE "[0-9a-f]{40} refs/pull/"' Subsequent patches will improve on this by avoiding visiting excluded sections of the `packed-refs` file in certain cases. Co-authored-by: Jeff King Signed-off-by: Jeff King Signed-off-by: Taylor Blau --- Documentation/git-for-each-ref.txt | 6 +++++ builtin/for-each-ref.c | 1 + ref-filter.c | 14 ++++++++++++ ref-filter.h | 6 +++++ t/t6300-for-each-ref.sh | 35 ++++++++++++++++++++++++++++++ 5 files changed, 62 insertions(+) diff --git a/Documentation/git-for-each-ref.txt b/Documentation/git-for-each-ref.txt index 1e215d4e734..5743eb5def4 100644 --- a/Documentation/git-for-each-ref.txt +++ b/Documentation/git-for-each-ref.txt @@ -14,6 +14,7 @@ SYNOPSIS [--points-at=] [--merged[=]] [--no-merged[=]] [--contains[=]] [--no-contains[=]] + [--exclude= ...] DESCRIPTION ----------- @@ -102,6 +103,11 @@ OPTIONS Do not print a newline after formatted refs where the format expands to the empty string. +--exclude=:: + If one or more patterns are given, only refs which do not match + any excluded pattern(s) are shown. Matching is done using the + same rules as `` above. + FIELD NAMES ----------- diff --git a/builtin/for-each-ref.c b/builtin/for-each-ref.c index ccceba54aa1..350bfa6e811 100644 --- a/builtin/for-each-ref.c +++ b/builtin/for-each-ref.c @@ -47,6 +47,7 @@ int cmd_for_each_ref(int argc, const char **argv, const char *prefix) OPT_INTEGER( 0 , "count", &maxcount, N_("show only matched refs")), OPT_STRING( 0 , "format", &format.format, N_("format"), N_("format to use for the output")), OPT__COLOR(&format.use_color, N_("respect format colors")), + OPT_REF_FILTER_EXCLUDE(&filter), OPT_REF_SORT(&sorting_options), OPT_CALLBACK(0, "points-at", &filter.points_at, N_("object"), N_("print only refs which points at the given object"), diff --git a/ref-filter.c b/ref-filter.c index c70a430adf3..845173a904a 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -2171,6 +2171,16 @@ static int filter_pattern_match(struct ref_filter *filter, const char *refname) filter->ignore_case); } +static int filter_exclude_match(struct ref_filter *filter, const char *refname) +{ + if (!filter->exclude.nr) + return 0; + if (filter->match_as_path) + return match_name_as_path(filter->exclude.v, refname, + filter->ignore_case); + return match_pattern(filter->exclude.v, refname, filter->ignore_case); +} + /* * This is the same as for_each_fullref_in(), but it tries to iterate * only over the patterns we'll care about. Note that it _doesn't_ do a full @@ -2338,6 +2348,9 @@ static int ref_filter_handler(const char *refname, const struct object_id *oid, if (!filter_pattern_match(filter, refname)) return 0; + if (filter_exclude_match(filter, refname)) + return 0; + if (filter->points_at.nr && !match_points_at(&filter->points_at, oid, refname)) return 0; @@ -2877,6 +2890,7 @@ void ref_filter_init(struct ref_filter *filter) void ref_filter_clear(struct ref_filter *filter) { + strvec_clear(&filter->exclude); oid_array_clear(&filter->points_at); free_commit_list(filter->with_commit); free_commit_list(filter->no_commit); diff --git a/ref-filter.h b/ref-filter.h index 160b8072245..1524bc463a5 100644 --- a/ref-filter.h +++ b/ref-filter.h @@ -6,6 +6,7 @@ #include "refs.h" #include "commit.h" #include "string-list.h" +#include "strvec.h" /* Quoting styles */ #define QUOTE_NONE 0 @@ -59,6 +60,7 @@ struct ref_array { struct ref_filter { const char **name_patterns; + struct strvec exclude; struct oid_array points_at; struct commit_list *with_commit; struct commit_list *no_commit; @@ -94,6 +96,7 @@ struct ref_format { #define REF_FILTER_INIT { \ .points_at = OID_ARRAY_INIT, \ + .exclude = STRVEC_INIT, \ } #define REF_FORMAT_INIT { \ .use_color = -1, \ @@ -112,6 +115,9 @@ struct ref_format { #define OPT_REF_SORT(var) \ OPT_STRING_LIST(0, "sort", (var), \ N_("key"), N_("field name to sort on")) +#define OPT_REF_FILTER_EXCLUDE(var) \ + OPT_STRVEC(0, "exclude", &(var)->exclude, \ + N_("pattern"), N_("exclude refs which match pattern")) /* * API for filtering a set of refs. Based on the type of refs the user diff --git a/t/t6300-for-each-ref.sh b/t/t6300-for-each-ref.sh index 5c00607608a..7e8d5785223 100755 --- a/t/t6300-for-each-ref.sh +++ b/t/t6300-for-each-ref.sh @@ -447,6 +447,41 @@ test_expect_success 'exercise glob patterns with prefixes' ' test_cmp expected actual ' +cat >expected <<\EOF +refs/tags/bar +refs/tags/baz +refs/tags/testtag +EOF + +test_expect_success 'exercise patterns with prefix exclusions' ' + for tag in foo/one foo/two foo/three bar baz + do + git tag "$tag" || return 1 + done && + test_when_finished "git tag -d foo/one foo/two foo/three bar baz" && + git for-each-ref --format="%(refname)" \ + refs/tags/ --exclude=refs/tags/foo >actual && + test_cmp expected actual +' + +cat >expected <<\EOF +refs/tags/bar +refs/tags/baz +refs/tags/foo/one +refs/tags/testtag +EOF + +test_expect_success 'exercise patterns with pattern exclusions' ' + for tag in foo/one foo/two foo/three bar baz + do + git tag "$tag" || return 1 + done && + test_when_finished "git tag -d foo/one foo/two foo/three bar baz" && + git for-each-ref --format="%(refname)" \ + refs/tags/ --exclude="refs/tags/foo/t*" >actual && + test_cmp expected actual +' + cat >expected <<\EOF 'refs/heads/main' 'refs/remotes/origin/main' From patchwork Mon Jul 10 21:12:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13307665 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 521AEEB64D9 for ; Mon, 10 Jul 2023 21:12:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231207AbjGJVMl (ORCPT ); Mon, 10 Jul 2023 17:12:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37250 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231908AbjGJVM3 (ORCPT ); Mon, 10 Jul 2023 17:12:29 -0400 Received: from mail-yw1-x1134.google.com (mail-yw1-x1134.google.com [IPv6:2607:f8b0:4864:20::1134]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0D9A3E6D for ; Mon, 10 Jul 2023 14:12:25 -0700 (PDT) Received: by mail-yw1-x1134.google.com with SMTP id 00721157ae682-576a9507a9bso89210367b3.1 for ; Mon, 10 Jul 2023 14:12:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1689023544; x=1691615544; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=yjrkiGfUCzQPf2L9v0euYLFsTSJJ7KAZPw4ucV5ZStE=; b=CIOEWLykKiZMq4hfVTuW9DF04DpgbKXtwo7e2CXSCi1HPeQrDaXz7n608KnFfZLNIc g0EiouJCEKN4FuS80vpW9zCudatd64mujqrIHDdX07d79ZcwPOFQG38h4cnOFK4XgJsP 0TFHQRCd+dvfn8vTaY7CnvMAqd0YPzCQgSJLh7YCx+1UGagvzvBLKNzN5gVaWAwD3c2P KAsZm0HAft544bvwo8W8vua/L5+t1AcNilk6iTIdlkeUN9HPeluxYPNYbgYGQw7WOfVN MEKp6rcDr8GUkMBIKqV9+4khzT4UHE2NlSy0Qwh8YWWfC+WFKBCpFQCrQTvFunofa5lC Zrbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689023544; x=1691615544; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=yjrkiGfUCzQPf2L9v0euYLFsTSJJ7KAZPw4ucV5ZStE=; b=fpfXU5lvk7QaPkPR+LrwO6/cGRiHTIw02/oJK7Xu4ejPlvennTGs+1ZF4GoEOotS/y OojgaGJnV1DwQdM3WnO0r5XsFK3I+PW7DokWdrNpj86oAey0I+2jrcR6trjoE8cVjr5l kOYax9At4eHqlb7ROz/MCcen/wXoYXLg8inknlh621ki+9O2XnKLpYv5NSBTFkfakGLQ x+kr6BQzIvrckUjzeXl+6RIQ6qlDiuH8YPDClCrOJGbElaLd7Z44IaevGwE6rvO5V9UY Nu3hZvZtZMWElt1tBvhLTWxg5X8jWIXOkAeHlmYtur8GuZZl1VuRTdmq9s5skY4MRrHJ 00bA== X-Gm-Message-State: ABy/qLY56iLdWNm2I+4VULkejmLjim66ppNYXvJ6j128xVP+OyxruNx1 dFCS9A4189qGZjHAIZg28PMS1+JfCspcteN5WCUI1g== X-Google-Smtp-Source: APBJJlFDNStRVntz5xj4Xa4JTqZiTy+AaBHEo0rm7uRFHh4lkjF0UMITtgmDeuBkvXN/CF7GczyhWg== X-Received: by 2002:a0d:d855:0:b0:56c:f547:e058 with SMTP id a82-20020a0dd855000000b0056cf547e058mr13155252ywe.18.1689023543839; Mon, 10 Jul 2023 14:12:23 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id b126-20020a0dc084000000b0056d443372f0sm181692ywd.119.2023.07.10.14.12.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Jul 2023 14:12:23 -0700 (PDT) Date: Mon, 10 Jul 2023 17:12:22 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Chris Torek , Derrick Stolee , Jeff King , Junio C Hamano , Patrick Steinhardt Subject: [PATCH v5 07/16] refs: plumb `exclude_patterns` argument throughout Message-ID: <81e223de0c82dfc213ea7d7041e66c9505c1599d.1689023520.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org The subsequent patch will want to access an optional `excluded_patterns` array within `refs/packed-backend.c` that will cull out certain references matching any of the given patterns on a best-effort basis. To do so, the refs subsystem needs to be updated to pass this value across a number of different locations. Prepare for a future patch by introducing this plumbing now, passing NULLs at top-level APIs in order to make that patch less noisy and more easily readable. Signed-off-by: Taylor Blau --- ls-refs.c | 2 +- ref-filter.c | 2 +- refs.c | 32 +++++++++++++++++++------------- refs.h | 8 +++++++- refs/debug.c | 5 +++-- refs/files-backend.c | 5 +++-- refs/packed-backend.c | 5 +++-- refs/refs-internal.h | 7 ++++--- revision.c | 2 +- 9 files changed, 42 insertions(+), 26 deletions(-) diff --git a/ls-refs.c b/ls-refs.c index a29c2364a59..65d1fa9029e 100644 --- a/ls-refs.c +++ b/ls-refs.c @@ -194,7 +194,7 @@ int ls_refs(struct repository *r, struct packet_reader *request) strvec_push(&data.prefixes, ""); refs_for_each_fullref_in_prefixes(get_main_ref_store(r), get_git_namespace(), data.prefixes.v, - send_ref, &data); + NULL, send_ref, &data); packet_fflush(stdout); strvec_clear(&data.prefixes); strbuf_release(&data.buf); diff --git a/ref-filter.c b/ref-filter.c index 845173a904a..925d77b7b79 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -2215,7 +2215,7 @@ static int for_each_fullref_in_pattern(struct ref_filter *filter, return refs_for_each_fullref_in_prefixes(get_main_ref_store(the_repository), NULL, filter->name_patterns, - cb, cb_data); + NULL, cb, cb_data); } /* diff --git a/refs.c b/refs.c index 876df4931f6..7e5cd84bf28 100644 --- a/refs.c +++ b/refs.c @@ -1528,7 +1528,9 @@ int head_ref(each_ref_fn fn, void *cb_data) struct ref_iterator *refs_ref_iterator_begin( struct ref_store *refs, - const char *prefix, int trim, + const char *prefix, + const char **exclude_patterns, + int trim, enum do_for_each_ref_flags flags) { struct ref_iterator *iter; @@ -1544,8 +1546,7 @@ struct ref_iterator *refs_ref_iterator_begin( } } - iter = refs->be->iterator_begin(refs, prefix, flags); - + iter = refs->be->iterator_begin(refs, prefix, exclude_patterns, flags); /* * `iterator_begin()` already takes care of prefix, but we * might need to do some trimming: @@ -1579,7 +1580,7 @@ static int do_for_each_repo_ref(struct repository *r, const char *prefix, if (!refs) return 0; - iter = refs_ref_iterator_begin(refs, prefix, trim, flags); + iter = refs_ref_iterator_begin(refs, prefix, NULL, trim, flags); return do_for_each_repo_ref_iterator(r, iter, fn, cb_data); } @@ -1601,6 +1602,7 @@ static int do_for_each_ref_helper(struct repository *r, } static int do_for_each_ref(struct ref_store *refs, const char *prefix, + const char **exclude_patterns, each_ref_fn fn, int trim, enum do_for_each_ref_flags flags, void *cb_data) { @@ -1610,7 +1612,8 @@ static int do_for_each_ref(struct ref_store *refs, const char *prefix, if (!refs) return 0; - iter = refs_ref_iterator_begin(refs, prefix, trim, flags); + iter = refs_ref_iterator_begin(refs, prefix, exclude_patterns, trim, + flags); return do_for_each_repo_ref_iterator(the_repository, iter, do_for_each_ref_helper, &hp); @@ -1618,7 +1621,7 @@ static int do_for_each_ref(struct ref_store *refs, const char *prefix, int refs_for_each_ref(struct ref_store *refs, each_ref_fn fn, void *cb_data) { - return do_for_each_ref(refs, "", fn, 0, 0, cb_data); + return do_for_each_ref(refs, "", NULL, fn, 0, 0, cb_data); } int for_each_ref(each_ref_fn fn, void *cb_data) @@ -1629,7 +1632,7 @@ int for_each_ref(each_ref_fn fn, void *cb_data) int refs_for_each_ref_in(struct ref_store *refs, const char *prefix, each_ref_fn fn, void *cb_data) { - return do_for_each_ref(refs, prefix, fn, strlen(prefix), 0, cb_data); + return do_for_each_ref(refs, prefix, NULL, fn, strlen(prefix), 0, cb_data); } int for_each_ref_in(const char *prefix, each_ref_fn fn, void *cb_data) @@ -1640,13 +1643,14 @@ int for_each_ref_in(const char *prefix, each_ref_fn fn, void *cb_data) int for_each_fullref_in(const char *prefix, each_ref_fn fn, void *cb_data) { return do_for_each_ref(get_main_ref_store(the_repository), - prefix, fn, 0, 0, cb_data); + prefix, NULL, fn, 0, 0, cb_data); } int refs_for_each_fullref_in(struct ref_store *refs, const char *prefix, + const char **exclude_patterns, each_ref_fn fn, void *cb_data) { - return do_for_each_ref(refs, prefix, fn, 0, 0, cb_data); + return do_for_each_ref(refs, prefix, exclude_patterns, fn, 0, 0, cb_data); } int for_each_replace_ref(struct repository *r, each_repo_ref_fn fn, void *cb_data) @@ -1663,14 +1667,14 @@ int for_each_namespaced_ref(each_ref_fn fn, void *cb_data) int ret; strbuf_addf(&buf, "%srefs/", get_git_namespace()); ret = do_for_each_ref(get_main_ref_store(the_repository), - buf.buf, fn, 0, 0, cb_data); + buf.buf, NULL, fn, 0, 0, cb_data); strbuf_release(&buf); return ret; } int refs_for_each_rawref(struct ref_store *refs, each_ref_fn fn, void *cb_data) { - return do_for_each_ref(refs, "", fn, 0, + return do_for_each_ref(refs, "", NULL, fn, 0, DO_FOR_EACH_INCLUDE_BROKEN, cb_data); } @@ -1740,6 +1744,7 @@ static void find_longest_prefixes(struct string_list *out, int refs_for_each_fullref_in_prefixes(struct ref_store *ref_store, const char *namespace, const char **patterns, + const char **exclude_patterns, each_ref_fn fn, void *cb_data) { struct string_list prefixes = STRING_LIST_INIT_DUP; @@ -1755,7 +1760,8 @@ int refs_for_each_fullref_in_prefixes(struct ref_store *ref_store, for_each_string_list_item(prefix, &prefixes) { strbuf_addstr(&buf, prefix->string); - ret = refs_for_each_fullref_in(ref_store, buf.buf, fn, cb_data); + ret = refs_for_each_fullref_in(ref_store, buf.buf, + exclude_patterns, fn, cb_data); if (ret) break; strbuf_setlen(&buf, namespace_len); @@ -2410,7 +2416,7 @@ int refs_verify_refname_available(struct ref_store *refs, strbuf_addstr(&dirname, refname + dirname.len); strbuf_addch(&dirname, '/'); - iter = refs_ref_iterator_begin(refs, dirname.buf, 0, + iter = refs_ref_iterator_begin(refs, dirname.buf, NULL, 0, DO_FOR_EACH_INCLUDE_BROKEN); while ((ok = ref_iterator_advance(iter)) == ITER_OK) { if (skip && diff --git a/refs.h b/refs.h index 933fdebe584..3c03b035a3f 100644 --- a/refs.h +++ b/refs.h @@ -344,6 +344,7 @@ int for_each_ref(each_ref_fn fn, void *cb_data); int for_each_ref_in(const char *prefix, each_ref_fn fn, void *cb_data); int refs_for_each_fullref_in(struct ref_store *refs, const char *prefix, + const char **exclude_patterns, each_ref_fn fn, void *cb_data); int for_each_fullref_in(const char *prefix, each_ref_fn fn, void *cb_data); @@ -351,10 +352,15 @@ int for_each_fullref_in(const char *prefix, each_ref_fn fn, void *cb_data); * iterate all refs in "patterns" by partitioning patterns into disjoint sets * and iterating the longest-common prefix of each set. * + * references matching any pattern in "exclude_patterns" are omitted from the + * result set on a best-effort basis. + * * callers should be prepared to ignore references that they did not ask for. */ int refs_for_each_fullref_in_prefixes(struct ref_store *refs, - const char *namespace, const char **patterns, + const char *namespace, + const char **patterns, + const char **exclude_patterns, each_ref_fn fn, void *cb_data); /** diff --git a/refs/debug.c b/refs/debug.c index c0fa707a1da..b7ffc4ce67e 100644 --- a/refs/debug.c +++ b/refs/debug.c @@ -229,11 +229,12 @@ static struct ref_iterator_vtable debug_ref_iterator_vtable = { static struct ref_iterator * debug_ref_iterator_begin(struct ref_store *ref_store, const char *prefix, - unsigned int flags) + const char **exclude_patterns, unsigned int flags) { struct debug_ref_store *drefs = (struct debug_ref_store *)ref_store; struct ref_iterator *res = - drefs->refs->be->iterator_begin(drefs->refs, prefix, flags); + drefs->refs->be->iterator_begin(drefs->refs, prefix, + exclude_patterns, flags); struct debug_ref_iterator *diter = xcalloc(1, sizeof(*diter)); base_ref_iterator_init(&diter->base, &debug_ref_iterator_vtable, 1); diter->iter = res; diff --git a/refs/files-backend.c b/refs/files-backend.c index 09b4954f21c..341354182bb 100644 --- a/refs/files-backend.c +++ b/refs/files-backend.c @@ -832,7 +832,8 @@ static struct ref_iterator_vtable files_ref_iterator_vtable = { static struct ref_iterator *files_ref_iterator_begin( struct ref_store *ref_store, - const char *prefix, unsigned int flags) + const char *prefix, const char **exclude_patterns, + unsigned int flags) { struct files_ref_store *refs; struct ref_iterator *loose_iter, *packed_iter, *overlay_iter; @@ -877,7 +878,7 @@ static struct ref_iterator *files_ref_iterator_begin( * the packed and loose references. */ packed_iter = refs_ref_iterator_begin( - refs->packed_ref_store, prefix, 0, + refs->packed_ref_store, prefix, exclude_patterns, 0, DO_FOR_EACH_INCLUDE_BROKEN); overlay_iter = overlay_ref_iterator_begin(loose_iter, packed_iter); diff --git a/refs/packed-backend.c b/refs/packed-backend.c index 27bd6339ff3..dec15dee31e 100644 --- a/refs/packed-backend.c +++ b/refs/packed-backend.c @@ -925,7 +925,8 @@ static struct ref_iterator_vtable packed_ref_iterator_vtable = { static struct ref_iterator *packed_ref_iterator_begin( struct ref_store *ref_store, - const char *prefix, unsigned int flags) + const char *prefix, const char **exclude_patterns, + unsigned int flags) { struct packed_ref_store *refs; struct snapshot *snapshot; @@ -1150,7 +1151,7 @@ static int write_with_updates(struct packed_ref_store *refs, * list of refs is exhausted, set iter to NULL. When the list * of updates is exhausted, leave i set to updates->nr. */ - iter = packed_ref_iterator_begin(&refs->base, "", + iter = packed_ref_iterator_begin(&refs->base, "", NULL, DO_FOR_EACH_INCLUDE_BROKEN); if ((ok = ref_iterator_advance(iter)) != ITER_OK) iter = NULL; diff --git a/refs/refs-internal.h b/refs/refs-internal.h index f72b7be8941..9db8aec4da8 100644 --- a/refs/refs-internal.h +++ b/refs/refs-internal.h @@ -367,8 +367,8 @@ int is_empty_ref_iterator(struct ref_iterator *ref_iterator); */ struct ref_iterator *refs_ref_iterator_begin( struct ref_store *refs, - const char *prefix, int trim, - enum do_for_each_ref_flags flags); + const char *prefix, const char **exclude_patterns, + int trim, enum do_for_each_ref_flags flags); /* * A callback function used to instruct merge_ref_iterator how to @@ -571,7 +571,8 @@ typedef int copy_ref_fn(struct ref_store *ref_store, */ typedef struct ref_iterator *ref_iterator_begin_fn( struct ref_store *ref_store, - const char *prefix, unsigned int flags); + const char *prefix, const char **exclude_patterns, + unsigned int flags); /* reflog functions */ diff --git a/revision.c b/revision.c index d66857ecc09..af11d599d35 100644 --- a/revision.c +++ b/revision.c @@ -2642,7 +2642,7 @@ static int for_each_bisect_ref(struct ref_store *refs, each_ref_fn fn, struct strbuf bisect_refs = STRBUF_INIT; int status; strbuf_addf(&bisect_refs, "refs/bisect/%s", term); - status = refs_for_each_fullref_in(refs, bisect_refs.buf, fn, cb_data); + status = refs_for_each_fullref_in(refs, bisect_refs.buf, NULL, fn, cb_data); strbuf_release(&bisect_refs); return status; } From patchwork Mon Jul 10 21:12:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13307666 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A3B51EB64D9 for ; Mon, 10 Jul 2023 21:12:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231983AbjGJVMt (ORCPT ); Mon, 10 Jul 2023 17:12:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37404 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231985AbjGJVMg (ORCPT ); Mon, 10 Jul 2023 17:12:36 -0400 Received: from mail-yb1-xb2a.google.com (mail-yb1-xb2a.google.com [IPv6:2607:f8b0:4864:20::b2a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A7A71E77 for ; Mon, 10 Jul 2023 14:12:27 -0700 (PDT) Received: by mail-yb1-xb2a.google.com with SMTP id 3f1490d57ef6-c1aad78d5b2so8901004276.0 for ; Mon, 10 Jul 2023 14:12:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1689023546; x=1691615546; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=R+z07fKh9BBAq18u3BttAlDWhemyUYrdbgJIH0yVj9k=; b=0Sc9uUG/QUjFpXG1o8bL/5KBqCdeJewvXSi9FVQzafrIaFXTpjFDwaA7ocyXQt3hyG f2CNRpfJpT91ZQRcEOWSo3s2sYrdYouquxASMhqzwn27Fokr5QnrOa72c32/uoWJvlnz yD5YVSXVY2PUb4p8hsbP/89iyN1l8hUHVtzh+U9kVYuQSouYmb2h+ZSaeBUTjPMYUImZ jIABGy89QF4nuie+b4dYQlhiVpEh2mIibBRKvfnxoO0Go9dJsdMPQiilK0jcZI4w46ei N8aTxWN7geiC8V/SyH8pvA3P5NIHznPrygrADWVuZh11qyENfDQrN7o+vc1AXexIZUKK bpuw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689023546; x=1691615546; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=R+z07fKh9BBAq18u3BttAlDWhemyUYrdbgJIH0yVj9k=; b=lWhTNVEYQECA31CdkDJOf/Q1wcnTKF7x1F06XcfT0sXmQ3LynFdxMUAG7V+dFEl/qS gIEbBjJzTTUh9v9JasxIWOxsZAPyMbBHJbjPpPXGJNK90EuWalLr9l+ywEqJcPSePn35 V5lJ7eUq5Gqu7IonoCZu7vpkkErxK6huCyxA3TMnhmFWDnUqr3He6nLxbNqIbKcmDF37 BT4ujJ6L+RL87NZCS4iRF0GeTBLdM5TtYzTtT/ESc3aysKVnYOfNxkmxmyNGuVvXHXdA HE/6i+JmX8J2dTtTp85dL85nEEGIUoKul/xA1r6UcW2ZzUiKCU2Lm5mop8/rt1b/uo2f F0aw== X-Gm-Message-State: ABy/qLYfH4ZoI5BGj2nr0myf1p2pKQMeM2cNFUTSD0DlTsODVBZ6clcZ kPvZV7x5uxFvFeLEKKHF0XFUmhAi6Ilh1LFZkHov2A== X-Google-Smtp-Source: APBJJlE9suFWBC443GGIxzO7fhbxXPJ33A3EKEpzrKPwd1pzZrX0RTgnqDI++dKjYY7PN6u3j3ARvA== X-Received: by 2002:a0d:df8a:0:b0:577:3225:818d with SMTP id i132-20020a0ddf8a000000b005773225818dmr15407651ywe.11.1689023546621; Mon, 10 Jul 2023 14:12:26 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id j9-20020a816e09000000b0054f50f71834sm180489ywc.124.2023.07.10.14.12.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Jul 2023 14:12:26 -0700 (PDT) Date: Mon, 10 Jul 2023 17:12:25 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Chris Torek , Derrick Stolee , Jeff King , Junio C Hamano , Patrick Steinhardt Subject: [PATCH v5 08/16] refs/packed-backend.c: refactor `find_reference_location()` Message-ID: <25c099a528c87651aeea3dfbd667fb51770dd24a.1689023520.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org The function `find_reference_location()` is used to perform a binary search-like function over the contents of a repository's `$GIT_DIR/packed-refs` file. The search it implements is unlike a standard binary search in that the records it searches over are not of a fixed width, so the comparison must locate the end of a record before comparing it. Extract the core routine of `find_reference_location()` in order to implement a function in the following patch which will find the first location in the `packed-refs` file that *doesn't* match the given pattern. The behavior of `find_reference_location()` is unchanged. Co-authored-by: Jeff King Signed-off-by: Jeff King Signed-off-by: Taylor Blau --- refs/packed-backend.c | 38 ++++++++++++++++++++++---------------- 1 file changed, 22 insertions(+), 16 deletions(-) diff --git a/refs/packed-backend.c b/refs/packed-backend.c index dec15dee31e..e36334e3320 100644 --- a/refs/packed-backend.c +++ b/refs/packed-backend.c @@ -528,22 +528,8 @@ static int load_contents(struct snapshot *snapshot) return 1; } -/* - * Find the place in `snapshot->buf` where the start of the record for - * `refname` starts. If `mustexist` is true and the reference doesn't - * exist, then return NULL. If `mustexist` is false and the reference - * doesn't exist, then return the point where that reference would be - * inserted, or `snapshot->eof` (which might be NULL) if it would be - * inserted at the end of the file. In the latter mode, `refname` - * doesn't have to be a proper reference name; for example, one could - * search for "refs/replace/" to find the start of any replace - * references. - * - * The record is sought using a binary search, so `snapshot->buf` must - * be sorted. - */ -static const char *find_reference_location(struct snapshot *snapshot, - const char *refname, int mustexist) +static const char *find_reference_location_1(struct snapshot *snapshot, + const char *refname, int mustexist) { /* * This is not *quite* a garden-variety binary search, because @@ -589,6 +575,26 @@ static const char *find_reference_location(struct snapshot *snapshot, return lo; } +/* + * Find the place in `snapshot->buf` where the start of the record for + * `refname` starts. If `mustexist` is true and the reference doesn't + * exist, then return NULL. If `mustexist` is false and the reference + * doesn't exist, then return the point where that reference would be + * inserted, or `snapshot->eof` (which might be NULL) if it would be + * inserted at the end of the file. In the latter mode, `refname` + * doesn't have to be a proper reference name; for example, one could + * search for "refs/replace/" to find the start of any replace + * references. + * + * The record is sought using a binary search, so `snapshot->buf` must + * be sorted. + */ +static const char *find_reference_location(struct snapshot *snapshot, + const char *refname, int mustexist) +{ + return find_reference_location_1(snapshot, refname, mustexist); +} + /* * Create a newly-allocated `snapshot` of the `packed-refs` file in * its current state and return it. The return value will already have From patchwork Mon Jul 10 21:12:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13307667 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9FE13EB64DA for ; Mon, 10 Jul 2023 21:13:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232390AbjGJVMv (ORCPT ); Mon, 10 Jul 2023 17:12:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37472 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232101AbjGJVMh (ORCPT ); Mon, 10 Jul 2023 17:12:37 -0400 Received: from mail-oa1-x33.google.com (mail-oa1-x33.google.com [IPv6:2001:4860:4864:20::33]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CA935CC for ; Mon, 10 Jul 2023 14:12:31 -0700 (PDT) Received: by mail-oa1-x33.google.com with SMTP id 586e51a60fabf-1b056276889so4017517fac.2 for ; Mon, 10 Jul 2023 14:12:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1689023550; x=1691615550; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=iJHvOsEpdDZ2fR305ElvUW96aa/QM51DtCjx3cYcCCA=; b=wauKNzx/WhHAKH77HMi9YWWVcGuztdOzsbArNZA1iB4CiHl3+VBqNv7CnMf/MidbST pnxcWkyaBQs8dC+IY8F5mVsIqRhGR+CQMxN/UXYvifj04hG+SYbM5OkH603NjcIp0rXv SomRhapHB+oZHVB7gTrIMKfV4/5pwoYPn6xh6uqIZKq9Zbgpq376PdhuhPxkr4gTFqEU OnJCWfnqDW2E8lwpsriU1gMG6sc4oeCaPNWEQlofvqjoPBlY56c/95O+kzxCLkJw5wYm gsZiz1b5u09JsQw2Pswi7/F4r/xPGwFV9eia+5KKLvYI1IAQqB97NK7Xc5gx/eRkqcDC F1+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689023550; x=1691615550; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=iJHvOsEpdDZ2fR305ElvUW96aa/QM51DtCjx3cYcCCA=; b=jginrVMC/BV4bMrB4+BVoVM1mW9euDcRbRhxVmFRcVYj2MPsKsATYYLmezYLZX8KBR dghge4XoLJh19uet0IUP2IsRnqcnhKSslC98GYehGOAmD+y+TWluA20SPiwVvhBKx2vt ADoR3mqwZZUh9TW7lWiz8l6FCDK1MAgK1eHIZrUSoM9zNxT4/r9sQgP58LtS2jFErNOe DAHuPvp+kaQ/6739HSHLps/ORYrfeM3C1OmRFIEiy0bA7HPj9UJMomREXLeOithZwcNH +jZ0nWzS52WzRCWwFzd4dLNIIcmTpD2E0S6djFL7cTTazGtY3n+ckwRX7O1el6rYml0G u6JA== X-Gm-Message-State: ABy/qLbwax89kDLjND4WdcXMhiiqenIyeYAHRJHgHMQ7SisGzLiTk+9v MTn4E8/OWr19ja4MDWw51Sb/4X4tfx9bYS8WNdRPAg== X-Google-Smtp-Source: APBJJlEF2ee7XdTlmWMRDqGDvgLMRx+u7TTVz7TOhwiXKLYEApjRZWiBZ2DdFivSBz5va+MmLz+m4g== X-Received: by 2002:a05:6870:2046:b0:1b0:58fc:c5ca with SMTP id l6-20020a056870204600b001b058fcc5camr15553428oad.1.1689023549496; Mon, 10 Jul 2023 14:12:29 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id h126-20020a0df784000000b00559f1cb8444sm186603ywf.70.2023.07.10.14.12.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Jul 2023 14:12:29 -0700 (PDT) Date: Mon, 10 Jul 2023 17:12:28 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Chris Torek , Derrick Stolee , Jeff King , Junio C Hamano , Patrick Steinhardt Subject: [PATCH v5 09/16] refs/packed-backend.c: implement jump lists to avoid excluded pattern(s) Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When iterating through the `packed-refs` file in order to answer a query like: $ git for-each-ref --exclude=refs/__hidden__ it would be useful to avoid walking over all of the entries in `refs/__hidden__/*` when possible, since we know that the ref-filter code is going to throw them away anyways. In certain circumstances, doing so is possible. The algorithm for doing so is as follows: - For each excluded pattern, find the first record that matches it, and the first record that *doesn't* match it (i.e. the location you'd next want to consider when excluding that pattern). - Sort the set of excluded regions from the previous step in ascending order of the first location within the `packed-refs` file that matches. - Clean up the results from the previous step: discard empty regions, and combine adjacent regions. The set of regions which remains is referred to as the "jump list", and never contains any references which should be included in the result set. Then when iterating through the `packed-refs` file, if `iter->pos` is ever contained in one of the regions from the previous steps, advance `iter->pos` past the end of that region, and continue enumeration. Note that we only perform this optimization when none of the excluded pattern(s) have special meta-characters in them. For a pattern like "refs/foo[ac]", the excluded regions ("refs/fooa", "refs/fooc", and everything underneath them) are not connected. A future implementation that handles this case may split the character class (pretending as if two patterns were excluded: "refs/fooa", and "refs/fooc"). There are a few other gotchas worth considering. First, note that the jump list is sorted, so once we jump past a region, we can avoid considering it (or any regions preceding it) again. The member `jump_pos` is used to track the first next-possible region to jump through. Second, note that the jump list is best-effort, since we do not handle loose references, and because of the meta-character issue above. The jump list may not skip past all references which won't appear in the results, but will never skip over a reference which does appear in the result set. In repositories with a large number of hidden references, the speed-up can be significant. Tests here are done with a copy of linux.git with a reference "refs/pull/N" pointing at every commit, as in: $ git rev-list HEAD | awk '{ print "create refs/pull/" NR " " $0 }' | git update-ref --stdin $ git pack-refs --all , it is significantly faster to have `for-each-ref` jump over the excluded references, as opposed to filtering them out after the fact: $ hyperfine \ 'git for-each-ref --format="%(objectname) %(refname)" | grep -vE "^[0-9a-f]{40} refs/pull/"' \ 'git.prev for-each-ref --format="%(objectname) %(refname)" --exclude="refs/pull"' \ 'git.compile for-each-ref --format="%(objectname) %(refname)" --exclude="refs/pull"' Benchmark 1: git for-each-ref --format="%(objectname) %(refname)" | grep -vE "^[0-9a-f]{40} refs/pull/" Time (mean ± σ): 798.1 ms ± 3.3 ms [User: 687.6 ms, System: 146.4 ms] Range (min … max): 794.5 ms … 805.5 ms 10 runs Benchmark 2: git.prev for-each-ref --format="%(objectname) %(refname)" --exclude="refs/pull" Time (mean ± σ): 98.9 ms ± 1.4 ms [User: 93.1 ms, System: 5.7 ms] Range (min … max): 97.0 ms … 104.0 ms 29 runs Benchmark 3: git.compile for-each-ref --format="%(objectname) %(refname)" --exclude="refs/pull" Time (mean ± σ): 4.5 ms ± 0.2 ms [User: 0.7 ms, System: 3.8 ms] Range (min … max): 4.1 ms … 5.8 ms 524 runs Summary 'git.compile for-each-ref --format="%(objectname) %(refname)" --exclude="refs/pull"' ran 21.87 ± 1.05 times faster than 'git.prev for-each-ref --format="%(objectname) %(refname)" --exclude="refs/pull"' 176.52 ± 8.19 times faster than 'git for-each-ref --format="%(objectname) %(refname)" | grep -vE "^[0-9a-f]{40} refs/pull/"' (Comparing stock git and this patch isn't quite fair, since an earlier commit in this series adds a naive implementation of the `--exclude` option. `git.prev` is built from the previous commit and includes this naive implementation). Using the jump list is fairly straightforward (see the changes to `refs/packed-backend.c::next_record()`), but constructing the list is not. To ensure that the construction is correct, add a new suite of tests in t1419 covering various corner cases (overlapping regions, partially overlapping regions, adjacent regions, etc.). Co-authored-by: Jeff King Signed-off-by: Jeff King Signed-off-by: Taylor Blau --- ref-filter.c | 6 +- refs.h | 4 + refs/packed-backend.c | 163 ++++++++++++++++++++++++++++++++++++-- t/helper/test-ref-store.c | 10 +++ t/t1419-exclude-refs.sh | 101 +++++++++++++++++++++++ 5 files changed, 276 insertions(+), 8 deletions(-) create mode 100755 t/t1419-exclude-refs.sh diff --git a/ref-filter.c b/ref-filter.c index 925d77b7b79..07b65affa9a 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -2210,12 +2210,14 @@ static int for_each_fullref_in_pattern(struct ref_filter *filter, if (!filter->name_patterns[0]) { /* no patterns; we have to look at everything */ - return for_each_fullref_in("", cb, cb_data); + return refs_for_each_fullref_in(get_main_ref_store(the_repository), + "", filter->exclude.v, cb, cb_data); } return refs_for_each_fullref_in_prefixes(get_main_ref_store(the_repository), NULL, filter->name_patterns, - NULL, cb, cb_data); + filter->exclude.v, + cb, cb_data); } /* diff --git a/refs.h b/refs.h index 3c03b035a3f..1ca1cdb9da5 100644 --- a/refs.h +++ b/refs.h @@ -343,6 +343,10 @@ int for_each_ref(each_ref_fn fn, void *cb_data); */ int for_each_ref_in(const char *prefix, each_ref_fn fn, void *cb_data); +/* + * references matching any pattern in "exclude_patterns" are omitted from the + * result set on a best-effort basis. + */ int refs_for_each_fullref_in(struct ref_store *refs, const char *prefix, const char **exclude_patterns, each_ref_fn fn, void *cb_data); diff --git a/refs/packed-backend.c b/refs/packed-backend.c index e36334e3320..d9e008819b5 100644 --- a/refs/packed-backend.c +++ b/refs/packed-backend.c @@ -304,7 +304,8 @@ static int cmp_packed_ref_records(const void *v1, const void *v2) * Compare a snapshot record at `rec` to the specified NUL-terminated * refname. */ -static int cmp_record_to_refname(const char *rec, const char *refname) +static int cmp_record_to_refname(const char *rec, const char *refname, + int start) { const char *r1 = rec + the_hash_algo->hexsz + 1; const char *r2 = refname; @@ -313,7 +314,7 @@ static int cmp_record_to_refname(const char *rec, const char *refname) if (*r1 == '\n') return *r2 ? -1 : 0; if (!*r2) - return 1; + return start ? 1 : -1; if (*r1 != *r2) return (unsigned char)*r1 < (unsigned char)*r2 ? -1 : +1; r1++; @@ -529,7 +530,8 @@ static int load_contents(struct snapshot *snapshot) } static const char *find_reference_location_1(struct snapshot *snapshot, - const char *refname, int mustexist) + const char *refname, int mustexist, + int start) { /* * This is not *quite* a garden-variety binary search, because @@ -559,7 +561,7 @@ static const char *find_reference_location_1(struct snapshot *snapshot, mid = lo + (hi - lo) / 2; rec = find_start_of_record(lo, mid); - cmp = cmp_record_to_refname(rec, refname); + cmp = cmp_record_to_refname(rec, refname, start); if (cmp < 0) { lo = find_end_of_record(mid, hi); } else if (cmp > 0) { @@ -592,7 +594,22 @@ static const char *find_reference_location_1(struct snapshot *snapshot, static const char *find_reference_location(struct snapshot *snapshot, const char *refname, int mustexist) { - return find_reference_location_1(snapshot, refname, mustexist); + return find_reference_location_1(snapshot, refname, mustexist, 1); +} + +/* + * Find the place in `snapshot->buf` after the end of the record for + * `refname`. In other words, find the location of first thing *after* + * `refname`. + * + * Other semantics are identical to the ones in + * `find_reference_location()`. + */ +static const char *find_reference_location_end(struct snapshot *snapshot, + const char *refname, + int mustexist) +{ + return find_reference_location_1(snapshot, refname, mustexist, 0); } /* @@ -786,6 +803,13 @@ struct packed_ref_iterator { /* The end of the part of the buffer that will be iterated over: */ const char *eof; + struct jump_list_entry { + const char *start; + const char *end; + } *jump; + size_t jump_nr, jump_alloc; + size_t jump_cur; + /* Scratch space for current values: */ struct object_id oid, peeled; struct strbuf refname_buf; @@ -803,14 +827,35 @@ struct packed_ref_iterator { */ static int next_record(struct packed_ref_iterator *iter) { - const char *p = iter->pos, *eol; + const char *p, *eol; strbuf_reset(&iter->refname_buf); + /* + * If iter->pos is contained within a skipped region, jump past + * it. + * + * Note that each skipped region is considered at most once, + * since they are ordered based on their starting position. + */ + while (iter->jump_cur < iter->jump_nr) { + struct jump_list_entry *curr = &iter->jump[iter->jump_cur]; + if (iter->pos < curr->start) + break; /* not to the next jump yet */ + + iter->jump_cur++; + if (iter->pos < curr->end) { + iter->pos = curr->end; + /* jumps are coalesced, so only one jump is necessary */ + break; + } + } + if (iter->pos == iter->eof) return ITER_DONE; iter->base.flags = REF_ISPACKED; + p = iter->pos; if (iter->eof - p < the_hash_algo->hexsz + 2 || parse_oid_hex(p, &iter->oid, &p) || @@ -918,6 +963,7 @@ static int packed_ref_iterator_abort(struct ref_iterator *ref_iterator) int ok = ITER_DONE; strbuf_release(&iter->refname_buf); + free(iter->jump); release_snapshot(iter->snapshot); base_ref_iterator_free(ref_iterator); return ok; @@ -929,6 +975,108 @@ static struct ref_iterator_vtable packed_ref_iterator_vtable = { .abort = packed_ref_iterator_abort }; +static int jump_list_entry_cmp(const void *va, const void *vb) +{ + const struct jump_list_entry *a = va; + const struct jump_list_entry *b = vb; + + if (a->start < b->start) + return -1; + if (a->start > b->start) + return 1; + return 0; +} + +static int has_glob_special(const char *str) +{ + const char *p; + for (p = str; *p; p++) { + if (is_glob_special(*p)) + return 1; + } + return 0; +} + +static void populate_excluded_jump_list(struct packed_ref_iterator *iter, + struct snapshot *snapshot, + const char **excluded_patterns) +{ + size_t i, j; + const char **pattern; + struct jump_list_entry *last_disjoint; + + if (!excluded_patterns) + return; + + for (pattern = excluded_patterns; *pattern; pattern++) { + struct jump_list_entry *e; + const char *start, *end; + + /* + * We can't feed any excludes with globs in them to the + * refs machinery. It only understands prefix matching. + * We likewise can't even feed the string leading up to + * the first meta-character, as something like "foo[a]" + * should not exclude "foobar" (but the prefix "foo" + * would match that and mark it for exclusion). + */ + if (has_glob_special(*pattern)) + continue; + + start = find_reference_location(snapshot, *pattern, 0); + end = find_reference_location_end(snapshot, *pattern, 0); + + if (start == end) + continue; /* nothing to jump over */ + + ALLOC_GROW(iter->jump, iter->jump_nr + 1, iter->jump_alloc); + + e = &iter->jump[iter->jump_nr++]; + e->start = start; + e->end = end; + } + + if (!iter->jump_nr) { + /* + * Every entry in exclude_patterns has a meta-character, + * nothing to do here. + */ + return; + } + + QSORT(iter->jump, iter->jump_nr, jump_list_entry_cmp); + + /* + * As an optimization, merge adjacent entries in the jump list + * to jump forwards as far as possible when entering a skipped + * region. + * + * For example, if we have two skipped regions: + * + * [[A, B], [B, C]] + * + * we want to combine that into a single entry jumping from A to + * C. + */ + last_disjoint = iter->jump; + + for (i = 1, j = 1; i < iter->jump_nr; i++) { + struct jump_list_entry *ours = &iter->jump[i]; + if (ours->start <= last_disjoint->end) { + /* overlapping regions extend the previous one */ + last_disjoint->end = last_disjoint->end > ours->end + ? last_disjoint->end : ours->end; + } else { + /* otherwise, insert a new region */ + iter->jump[j++] = *ours; + last_disjoint = ours; + } + } + + iter->jump_nr = j; + iter->jump_cur = 0; +} + static struct ref_iterator *packed_ref_iterator_begin( struct ref_store *ref_store, const char *prefix, const char **exclude_patterns, @@ -964,6 +1112,9 @@ static struct ref_iterator *packed_ref_iterator_begin( ref_iterator = &iter->base; base_ref_iterator_init(ref_iterator, &packed_ref_iterator_vtable, 1); + if (exclude_patterns) + populate_excluded_jump_list(iter, snapshot, exclude_patterns); + iter->snapshot = snapshot; acquire_snapshot(snapshot); diff --git a/t/helper/test-ref-store.c b/t/helper/test-ref-store.c index fb18831ec2f..13865425dd1 100644 --- a/t/helper/test-ref-store.c +++ b/t/helper/test-ref-store.c @@ -186,6 +186,15 @@ static int cmd_for_each_ref(struct ref_store *refs, const char **argv) return refs_for_each_ref_in(refs, prefix, each_ref, NULL); } +static int cmd_for_each_ref__exclude(struct ref_store *refs, const char **argv) +{ + const char *prefix = notnull(*argv++, "prefix"); + const char **exclude_patterns = argv; + + return refs_for_each_fullref_in(refs, prefix, exclude_patterns, each_ref, + NULL); +} + static int cmd_resolve_ref(struct ref_store *refs, const char **argv) { struct object_id oid = *null_oid(); @@ -318,6 +327,7 @@ static struct command commands[] = { { "delete-refs", cmd_delete_refs }, { "rename-ref", cmd_rename_ref }, { "for-each-ref", cmd_for_each_ref }, + { "for-each-ref--exclude", cmd_for_each_ref__exclude }, { "resolve-ref", cmd_resolve_ref }, { "verify-ref", cmd_verify_ref }, { "for-each-reflog", cmd_for_each_reflog }, diff --git a/t/t1419-exclude-refs.sh b/t/t1419-exclude-refs.sh new file mode 100755 index 00000000000..bc534c8ea18 --- /dev/null +++ b/t/t1419-exclude-refs.sh @@ -0,0 +1,101 @@ +#!/bin/sh + +test_description='test exclude_patterns functionality in main ref store' + +GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main +export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME + +TEST_PASSES_SANITIZE_LEAK=true +. ./test-lib.sh + +for_each_ref__exclude () { + test-tool ref-store main for-each-ref--exclude "$@" >actual.raw + cut -d ' ' -f 2 actual.raw +} + +for_each_ref () { + git for-each-ref --format='%(refname)' "$@" +} + +test_expect_success 'setup' ' + test_commit --no-tag base && + base="$(git rev-parse HEAD)" && + + for name in foo bar baz quux + do + for i in 1 2 3 + do + echo "create refs/heads/$name/$i $base" || return 1 + done || return 1 + done >in && + echo "delete refs/heads/main" >>in && + + git update-ref --stdin actual && + for_each_ref refs/heads/bar refs/heads/baz refs/heads/quux >expect && + + test_cmp expect actual +' + +test_expect_success 'excluded region at beginning' ' + for_each_ref__exclude refs/heads refs/heads/bar >actual && + for_each_ref refs/heads/baz refs/heads/foo refs/heads/quux >expect && + + test_cmp expect actual +' + +test_expect_success 'excluded region at end' ' + for_each_ref__exclude refs/heads refs/heads/quux >actual && + for_each_ref refs/heads/foo refs/heads/bar refs/heads/baz >expect && + + test_cmp expect actual +' + +test_expect_success 'disjoint excluded regions' ' + for_each_ref__exclude refs/heads refs/heads/bar refs/heads/quux >actual && + for_each_ref refs/heads/baz refs/heads/foo >expect && + + test_cmp expect actual +' + +test_expect_success 'adjacent, non-overlapping excluded regions' ' + for_each_ref__exclude refs/heads refs/heads/bar refs/heads/baz >actual && + for_each_ref refs/heads/foo refs/heads/quux >expect && + + test_cmp expect actual +' + +test_expect_success 'overlapping excluded regions' ' + for_each_ref__exclude refs/heads refs/heads/ba refs/heads/baz >actual && + for_each_ref refs/heads/foo refs/heads/quux >expect && + + test_cmp expect actual +' + +test_expect_success 'several overlapping excluded regions' ' + for_each_ref__exclude refs/heads \ + refs/heads/bar refs/heads/baz refs/heads/foo >actual && + for_each_ref refs/heads/quux >expect && + + test_cmp expect actual +' + +test_expect_success 'non-matching excluded section' ' + for_each_ref__exclude refs/heads refs/heads/does/not/exist >actual && + for_each_ref >expect && + + test_cmp expect actual +' + +test_expect_success 'meta-characters are discarded' ' + for_each_ref__exclude refs/heads "refs/heads/ba*" >actual && + for_each_ref >expect && + + test_cmp expect actual +' + +test_done From patchwork Mon Jul 10 21:12:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13307668 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0AD48EB64D9 for ; Mon, 10 Jul 2023 21:13:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232875AbjGJVNB (ORCPT ); Mon, 10 Jul 2023 17:13:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37696 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231593AbjGJVMi (ORCPT ); Mon, 10 Jul 2023 17:12:38 -0400 Received: from mail-yw1-x1136.google.com (mail-yw1-x1136.google.com [IPv6:2607:f8b0:4864:20::1136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4D25DE51 for ; Mon, 10 Jul 2023 14:12:33 -0700 (PDT) Received: by mail-yw1-x1136.google.com with SMTP id 00721157ae682-5774335bb2aso56615497b3.0 for ; Mon, 10 Jul 2023 14:12:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1689023552; x=1691615552; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=S4SRKp02/Vp9FLRQJwOFgjkNGFbphg2TUJVzDCQhIWQ=; b=mc9GR6HskVEsU+8K/lO8aXzBh+JU44Shut+r+bVR267aZf2V1FBlmOyA6gbiB1D0AN +pb5NxC7rTX3h1jZjPrhwaGzsO1JlyNRNmyg7wnmr9bI7EquAUaeqaZgUj/y0gxv3X4b YyZdPnwjotfya476fw55akQoHgld83PTwVYrYELlAmDXs/0J49Vo5fZ+8qwcUuszFlTk PhYZ8nQ7/FDijmNyIdv1mH54XPefT7JBSPuvmLgPXBDP0Aohalt6sFtaOd9n2PkJ9f2w yJiFLTvot+SPSorDe98nBG2wcuSvBr6WC7u4IjRb9j0XX6fVVawiDQwQKtWQBw5DFKII h7BQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689023552; x=1691615552; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=S4SRKp02/Vp9FLRQJwOFgjkNGFbphg2TUJVzDCQhIWQ=; b=k3youL7NS0GkWLzxDnSASEMTQeOejtmj4yyiBhmNsovVnkenPsKZxPwwBAomDECCUC GjNKWl4S2DyCc32b8uRtBWm8JMCkM43SN8VrXPILCGBQJW/4O/nhrNUKE0HNzwLrb8/C g9LvBILWcPEaxwLmn40zwK/2VJzsISWkAQvkzoytI5SR/4ewQ8lnoywZRmkN0PdfK1u1 sqW/voPjG5IrhLJ1hNymTCn0mnlkSiWqDTLvTGJUrm+jWGCgmZz4CmQ83nthXTRe1YzG gQXptdhR2SFvO5xWDlXll5IZ4HxMwYed0Wt+GstkaEZO1apdARqAYh0NcRP9wDkyZCdw giBA== X-Gm-Message-State: ABy/qLYwVWavOWZPzfEaDGfINKC4IaYmZpBLThqLlalAjmK4ovuFnx0E KVDpeW/gXLjAx0Vq4TBOKcwIPHjbIYZk0eQCn+71qg== X-Google-Smtp-Source: APBJJlE6XHkxsa1KrdbVuM8rQBlWQplzjosZCptIMlyIBwkyqluKMWtWtB0fja/HogmyH7iVWcgSxw== X-Received: by 2002:a0d:d94f:0:b0:573:1db2:7f7d with SMTP id b76-20020a0dd94f000000b005731db27f7dmr11886303ywe.2.1689023552363; Mon, 10 Jul 2023 14:12:32 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id b62-20020a816741000000b0054f56baf3f2sm179886ywc.122.2023.07.10.14.12.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Jul 2023 14:12:32 -0700 (PDT) Date: Mon, 10 Jul 2023 17:12:31 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Chris Torek , Derrick Stolee , Jeff King , Junio C Hamano , Patrick Steinhardt Subject: [PATCH v5 10/16] refs/packed-backend.c: add trace2 counters for jump list Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org The previous commit added low-level tests to ensure that the packed-refs iterator did not enumerate excluded sections of the refspace. However, there was no guarantee that these sections weren't being visited, only that they were being suppressed from the output. To harden these tests, add a trace2 counter which tracks the number of regions skipped by the packed-refs iterator, and assert on its value. Suggested-by: Derrick Stolee Signed-off-by: Taylor Blau --- refs/packed-backend.c | 2 ++ t/t1419-exclude-refs.sh | 59 ++++++++++++++++++++++++++++------------- trace2.h | 2 ++ trace2/tr2_ctr.c | 5 ++++ 4 files changed, 49 insertions(+), 19 deletions(-) diff --git a/refs/packed-backend.c b/refs/packed-backend.c index d9e008819b5..59c78d7941f 100644 --- a/refs/packed-backend.c +++ b/refs/packed-backend.c @@ -13,6 +13,7 @@ #include "../statinfo.h" #include "../wrapper.h" #include "../write-or-die.h" +#include "../trace2.h" enum mmap_strategy { /* @@ -846,6 +847,7 @@ static int next_record(struct packed_ref_iterator *iter) iter->jump_cur++; if (iter->pos < curr->end) { iter->pos = curr->end; + trace2_counter_add(TRACE2_COUNTER_ID_PACKED_REFS_JUMPS, 1); /* jumps are coalesced, so only one jump is necessary */ break; } diff --git a/t/t1419-exclude-refs.sh b/t/t1419-exclude-refs.sh index bc534c8ea18..5d8c86b6573 100755 --- a/t/t1419-exclude-refs.sh +++ b/t/t1419-exclude-refs.sh @@ -9,7 +9,8 @@ TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh for_each_ref__exclude () { - test-tool ref-store main for-each-ref--exclude "$@" >actual.raw + GIT_TRACE2_PERF=1 test-tool ref-store main \ + for-each-ref--exclude "$@" >actual.raw cut -d ' ' -f 2 actual.raw } @@ -17,6 +18,17 @@ for_each_ref () { git for-each-ref --format='%(refname)' "$@" } +assert_jumps () { + local nr="$1" + local trace="$2" + + grep -q "name:jumps_made value:$nr$" $trace +} + +assert_no_jumps () { + ! assert_jumps ".*" "$1" +} + test_expect_success 'setup' ' test_commit --no-tag base && base="$(git rev-parse HEAD)" && @@ -35,67 +47,76 @@ test_expect_success 'setup' ' ' test_expect_success 'excluded region in middle' ' - for_each_ref__exclude refs/heads refs/heads/foo >actual && + for_each_ref__exclude refs/heads refs/heads/foo >actual 2>perf && for_each_ref refs/heads/bar refs/heads/baz refs/heads/quux >expect && - test_cmp expect actual + test_cmp expect actual && + assert_jumps 1 perf ' test_expect_success 'excluded region at beginning' ' - for_each_ref__exclude refs/heads refs/heads/bar >actual && + for_each_ref__exclude refs/heads refs/heads/bar >actual 2>perf && for_each_ref refs/heads/baz refs/heads/foo refs/heads/quux >expect && - test_cmp expect actual + test_cmp expect actual && + assert_jumps 1 perf ' test_expect_success 'excluded region at end' ' - for_each_ref__exclude refs/heads refs/heads/quux >actual && + for_each_ref__exclude refs/heads refs/heads/quux >actual 2>perf && for_each_ref refs/heads/foo refs/heads/bar refs/heads/baz >expect && - test_cmp expect actual + test_cmp expect actual && + assert_jumps 1 perf ' test_expect_success 'disjoint excluded regions' ' - for_each_ref__exclude refs/heads refs/heads/bar refs/heads/quux >actual && + for_each_ref__exclude refs/heads refs/heads/bar refs/heads/quux >actual 2>perf && for_each_ref refs/heads/baz refs/heads/foo >expect && - test_cmp expect actual + test_cmp expect actual && + assert_jumps 2 perf ' test_expect_success 'adjacent, non-overlapping excluded regions' ' - for_each_ref__exclude refs/heads refs/heads/bar refs/heads/baz >actual && + for_each_ref__exclude refs/heads refs/heads/bar refs/heads/baz >actual 2>perf && for_each_ref refs/heads/foo refs/heads/quux >expect && - test_cmp expect actual + test_cmp expect actual && + assert_jumps 1 perf ' test_expect_success 'overlapping excluded regions' ' - for_each_ref__exclude refs/heads refs/heads/ba refs/heads/baz >actual && + for_each_ref__exclude refs/heads refs/heads/ba refs/heads/baz >actual 2>perf && for_each_ref refs/heads/foo refs/heads/quux >expect && - test_cmp expect actual + test_cmp expect actual && + assert_jumps 1 perf ' test_expect_success 'several overlapping excluded regions' ' for_each_ref__exclude refs/heads \ - refs/heads/bar refs/heads/baz refs/heads/foo >actual && + refs/heads/bar refs/heads/baz refs/heads/foo >actual 2>perf && for_each_ref refs/heads/quux >expect && - test_cmp expect actual + test_cmp expect actual && + assert_jumps 1 perf ' test_expect_success 'non-matching excluded section' ' - for_each_ref__exclude refs/heads refs/heads/does/not/exist >actual && + for_each_ref__exclude refs/heads refs/heads/does/not/exist >actual 2>perf && for_each_ref >expect && - test_cmp expect actual + test_cmp expect actual && + assert_no_jumps perf ' test_expect_success 'meta-characters are discarded' ' - for_each_ref__exclude refs/heads "refs/heads/ba*" >actual && + for_each_ref__exclude refs/heads "refs/heads/ba*" >actual 2>perf && for_each_ref >expect && - test_cmp expect actual + test_cmp expect actual && + assert_no_jumps perf ' test_done diff --git a/trace2.h b/trace2.h index f5c5a9e6bac..850056128fc 100644 --- a/trace2.h +++ b/trace2.h @@ -552,6 +552,8 @@ enum trace2_counter_id { TRACE2_COUNTER_ID_TEST1 = 0, /* emits summary event only */ TRACE2_COUNTER_ID_TEST2, /* emits summary and thread events */ + TRACE2_COUNTER_ID_PACKED_REFS_JUMPS, /* counts number of jumps */ + /* Add additional counter definitions before here. */ TRACE2_NUMBER_OF_COUNTERS }; diff --git a/trace2/tr2_ctr.c b/trace2/tr2_ctr.c index b342d3b1a3c..50570d01653 100644 --- a/trace2/tr2_ctr.c +++ b/trace2/tr2_ctr.c @@ -27,6 +27,11 @@ static struct tr2_counter_metadata tr2_counter_metadata[TRACE2_NUMBER_OF_COUNTER .name = "test2", .want_per_thread_events = 1, }, + [TRACE2_COUNTER_ID_PACKED_REFS_JUMPS] = { + .category = "packed-refs", + .name = "jumps_made", + .want_per_thread_events = 0, + }, /* Add additional metadata before here. */ }; From patchwork Mon Jul 10 21:12:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13307669 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 21887EB64DA for ; Mon, 10 Jul 2023 21:13:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231639AbjGJVNE (ORCPT ); Mon, 10 Jul 2023 17:13:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37574 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231787AbjGJVMm (ORCPT ); Mon, 10 Jul 2023 17:12:42 -0400 Received: from mail-yb1-xb2f.google.com (mail-yb1-xb2f.google.com [IPv6:2607:f8b0:4864:20::b2f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 73BC8C4 for ; Mon, 10 Jul 2023 14:12:36 -0700 (PDT) Received: by mail-yb1-xb2f.google.com with SMTP id 3f1490d57ef6-c4dd86f5d78so7239651276.0 for ; Mon, 10 Jul 2023 14:12:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1689023555; x=1691615555; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=nziK9Irby7MFLeFsDzsjjg/lY2fCbTtP1sERtDetdhw=; b=I+kV6PH51ipTjpSTEWmsOd/sR7RJ2G6vdhrxN9orTbLxQO83m8PQ2clHENWHu0NL5b WwSgE2z08xSdEsJuVAe+4qr/2ezgNA+a5bsz6QQPp8fFwrJf+StzrgWSZ/GjfHwmn04t RVSy/t9+ohlReDoIUILki3NQps7pTDnBS7hIueVbqyffvJlRBSbS7hh4R43iAL3wguER /sqp2/c5a/LdXvb586qR/xccWQnJ2usMQsBVTxQvgjMcNShtViN29ZDwWmi72Swi8wD8 7wFwHFcPzW+LVLQLMhmr6/bWrMk47vDxveSNFDPKCZYNIqXX2CCzf09m2oUKjd1S0QHl AQ2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689023555; x=1691615555; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=nziK9Irby7MFLeFsDzsjjg/lY2fCbTtP1sERtDetdhw=; b=KIQHagk8Q5NPyOemB7+Rd7kpMpKezNklAPvoqVSs7LX/VhcSwzRl7JI/Itq4uYAIx4 r9n+jEO0V2LPv/weURxNxE4zbl0HieW2s0ZaCnaFTsypO+YV7jL+cxtOUg/N5XJsQMc3 yelxqJmo3cuIdsXjpUEAQ5QEuFnsa8Y5nIeD+mg387Oj061H4y528/JclfDeVueAXMPY W87ziIinAdXw7yq+WERwa458JP/XUA59Bh3/a7gcWeguhyZdoL+z8cQI6ZZUQORr+td8 B9U3ffcaGteJXvoGBXuYu1j5L3e4phdNnfiD5GN3qZx5f69BffW1LpbRs8DW3PCc2dV6 RaOg== X-Gm-Message-State: ABy/qLa/Zymnm3b/tojKgu85yqu8ySutz/6jlgDzIZiHlJ+c28Np6ndA Debv2U6mwd5bF4iXm7ohg5JORBMXc6T62JkugeX5cQ== X-Google-Smtp-Source: APBJJlHUfTaH0PAzZ7swnbHMKeh4nxw05BnfT+leUnstvyco+bzN22LSVuuEOR4ha+/XuqAu0g7CbA== X-Received: by 2002:a25:7495:0:b0:c42:53eb:1599 with SMTP id p143-20020a257495000000b00c4253eb1599mr11960405ybc.16.1689023555262; Mon, 10 Jul 2023 14:12:35 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id w18-20020a25ac12000000b00c4d8c48d7c0sm148426ybi.7.2023.07.10.14.12.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Jul 2023 14:12:35 -0700 (PDT) Date: Mon, 10 Jul 2023 17:12:33 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Chris Torek , Derrick Stolee , Jeff King , Junio C Hamano , Patrick Steinhardt Subject: [PATCH v5 11/16] revision.h: store hidden refs in a `strvec` Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In subsequent commits, it will be convenient to have a 'const char **' of hidden refs (matching `transfer.hiderefs`, `uploadpack.hideRefs`, etc.), instead of a `string_list`. Convert spots throughout the tree that store the list of hidden refs from a `string_list` to a `strvec`. Note that in `parse_hide_refs_config()` there is an ugly const-cast used to avoid an extra copy of each value before trimming any trailing slash characters. This could instead be written as: ref = xstrdup(value); len = strlen(ref); while (len && ref[len - 1] == '/') ref[--len] = '\0'; strvec_push(hide_refs, ref); free(ref); but the double-copy (once when calling `xstrdup()`, and another via `strvec_push()`) is wasteful. Signed-off-by: Taylor Blau --- builtin/receive-pack.c | 4 ++-- ls-refs.c | 6 +++--- refs.c | 11 ++++++----- refs.h | 4 ++-- revision.c | 2 +- revision.h | 5 +++-- upload-pack.c | 10 +++++----- 7 files changed, 22 insertions(+), 20 deletions(-) diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c index faa8f84c5a1..56e8b4e6af4 100644 --- a/builtin/receive-pack.c +++ b/builtin/receive-pack.c @@ -91,7 +91,7 @@ static struct object_id push_cert_oid; static struct signature_check sigcheck; static const char *push_cert_nonce; static const char *cert_nonce_seed; -static struct string_list hidden_refs = STRING_LIST_INIT_DUP; +static struct strvec hidden_refs = STRVEC_INIT; static const char *NONCE_UNSOLICITED = "UNSOLICITED"; static const char *NONCE_BAD = "BAD"; @@ -2621,7 +2621,7 @@ int cmd_receive_pack(int argc, const char **argv, const char *prefix) packet_flush(1); oid_array_clear(&shallow); oid_array_clear(&ref); - string_list_clear(&hidden_refs, 0); + strvec_clear(&hidden_refs); free((void *)push_cert_nonce); return 0; } diff --git a/ls-refs.c b/ls-refs.c index 65d1fa9029e..9291ddfdf8c 100644 --- a/ls-refs.c +++ b/ls-refs.c @@ -72,7 +72,7 @@ struct ls_refs_data { unsigned symrefs; struct strvec prefixes; struct strbuf buf; - struct string_list hidden_refs; + struct strvec hidden_refs; unsigned unborn : 1; }; @@ -156,7 +156,7 @@ int ls_refs(struct repository *r, struct packet_reader *request) memset(&data, 0, sizeof(data)); strvec_init(&data.prefixes); strbuf_init(&data.buf, 0); - string_list_init_dup(&data.hidden_refs); + strvec_init(&data.hidden_refs); git_config(ls_refs_config, &data); @@ -198,7 +198,7 @@ int ls_refs(struct repository *r, struct packet_reader *request) packet_fflush(stdout); strvec_clear(&data.prefixes); strbuf_release(&data.buf); - string_list_clear(&data.hidden_refs, 0); + strvec_clear(&data.hidden_refs); return 0; } diff --git a/refs.c b/refs.c index 7e5cd84bf28..64dabc40cae 100644 --- a/refs.c +++ b/refs.c @@ -1429,7 +1429,7 @@ char *shorten_unambiguous_ref(const char *refname, int strict) } int parse_hide_refs_config(const char *var, const char *value, const char *section, - struct string_list *hide_refs) + struct strvec *hide_refs) { const char *key; if (!strcmp("transfer.hiderefs", var) || @@ -1440,22 +1440,23 @@ int parse_hide_refs_config(const char *var, const char *value, const char *secti if (!value) return config_error_nonbool(var); - ref = xstrdup(value); + + /* drop const to remove trailing '/' characters */ + ref = (char *)strvec_push(hide_refs, value); len = strlen(ref); while (len && ref[len - 1] == '/') ref[--len] = '\0'; - string_list_append_nodup(hide_refs, ref); } return 0; } int ref_is_hidden(const char *refname, const char *refname_full, - const struct string_list *hide_refs) + const struct strvec *hide_refs) { int i; for (i = hide_refs->nr - 1; i >= 0; i--) { - const char *match = hide_refs->items[i].string; + const char *match = hide_refs->v[i]; const char *subject; int neg = 0; const char *p; diff --git a/refs.h b/refs.h index 1ca1cdb9da5..fa3ce4b0bc1 100644 --- a/refs.h +++ b/refs.h @@ -820,7 +820,7 @@ int update_ref(const char *msg, const char *refname, unsigned int flags, enum action_on_err onerr); int parse_hide_refs_config(const char *var, const char *value, const char *, - struct string_list *); + struct strvec *); /* * Check whether a ref is hidden. If no namespace is set, both the first and @@ -830,7 +830,7 @@ int parse_hide_refs_config(const char *var, const char *value, const char *, * the ref is outside that namespace, the first parameter is NULL. The second * parameter always points to the full ref name. */ -int ref_is_hidden(const char *, const char *, const struct string_list *); +int ref_is_hidden(const char *, const char *, const struct strvec *); /* Is this a per-worktree ref living in the refs/ namespace? */ int is_per_worktree_ref(const char *refname); diff --git a/revision.c b/revision.c index af11d599d35..a15b18fc8e6 100644 --- a/revision.c +++ b/revision.c @@ -1561,7 +1561,7 @@ void init_ref_exclusions(struct ref_exclusions *exclusions) void clear_ref_exclusions(struct ref_exclusions *exclusions) { string_list_clear(&exclusions->excluded_refs, 0); - string_list_clear(&exclusions->hidden_refs, 0); + strvec_clear(&exclusions->hidden_refs); exclusions->hidden_refs_configured = 0; } diff --git a/revision.h b/revision.h index 25776af3815..7f219cde62f 100644 --- a/revision.h +++ b/revision.h @@ -10,6 +10,7 @@ #include "decorate.h" #include "ident.h" #include "list-objects-filter-options.h" +#include "strvec.h" /** * The revision walking API offers functions to build a list of revisions @@ -95,7 +96,7 @@ struct ref_exclusions { * Hidden refs is a list of patterns that is to be hidden via * `ref_is_hidden()`. */ - struct string_list hidden_refs; + struct strvec hidden_refs; /* * Indicates whether hidden refs have been configured. This is to @@ -110,7 +111,7 @@ struct ref_exclusions { */ #define REF_EXCLUSIONS_INIT { \ .excluded_refs = STRING_LIST_INIT_DUP, \ - .hidden_refs = STRING_LIST_INIT_DUP, \ + .hidden_refs = STRVEC_INIT, \ } struct oidset; diff --git a/upload-pack.c b/upload-pack.c index 946074920a8..cfb61ccbb5c 100644 --- a/upload-pack.c +++ b/upload-pack.c @@ -69,7 +69,7 @@ struct upload_pack_data { struct object_array have_obj; struct oid_array haves; /* v2 only */ struct string_list wanted_refs; /* v2 only */ - struct string_list hidden_refs; + struct strvec hidden_refs; struct object_array shallows; struct string_list deepen_not; @@ -127,7 +127,7 @@ static void upload_pack_data_init(struct upload_pack_data *data) { struct string_list symref = STRING_LIST_INIT_DUP; struct string_list wanted_refs = STRING_LIST_INIT_DUP; - struct string_list hidden_refs = STRING_LIST_INIT_DUP; + struct strvec hidden_refs = STRVEC_INIT; struct object_array want_obj = OBJECT_ARRAY_INIT; struct object_array have_obj = OBJECT_ARRAY_INIT; struct oid_array haves = OID_ARRAY_INIT; @@ -162,7 +162,7 @@ static void upload_pack_data_clear(struct upload_pack_data *data) { string_list_clear(&data->symref, 1); string_list_clear(&data->wanted_refs, 1); - string_list_clear(&data->hidden_refs, 0); + strvec_clear(&data->hidden_refs); object_array_clear(&data->want_obj); object_array_clear(&data->have_obj); oid_array_clear(&data->haves); @@ -1170,7 +1170,7 @@ static void receive_needs(struct upload_pack_data *data, /* return non-zero if the ref is hidden, otherwise 0 */ static int mark_our_ref(const char *refname, const char *refname_full, - const struct object_id *oid, const struct string_list *hidden_refs) + const struct object_id *oid, const struct strvec *hidden_refs) { struct object *o = lookup_unknown_object(the_repository, oid); @@ -1471,7 +1471,7 @@ static int parse_want(struct packet_writer *writer, const char *line, static int parse_want_ref(struct packet_writer *writer, const char *line, struct string_list *wanted_refs, - struct string_list *hidden_refs, + struct strvec *hidden_refs, struct object_array *want_obj) { const char *refname_nons; From patchwork Mon Jul 10 21:12:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13307670 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 17D9FEB64D9 for ; Mon, 10 Jul 2023 21:13:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231733AbjGJVNH (ORCPT ); Mon, 10 Jul 2023 17:13:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37766 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231847AbjGJVMn (ORCPT ); Mon, 10 Jul 2023 17:12:43 -0400 Received: from mail-ot1-x32a.google.com (mail-ot1-x32a.google.com [IPv6:2607:f8b0:4864:20::32a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0A5D01A8 for ; Mon, 10 Jul 2023 14:12:39 -0700 (PDT) Received: by mail-ot1-x32a.google.com with SMTP id 46e09a7af769-6b7206f106cso4038653a34.1 for ; Mon, 10 Jul 2023 14:12:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1689023558; x=1691615558; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=MAOE0b48L7RYO+dNddsG8MKfks+ILkcQuQu91zv6YCs=; b=10r5ayQrCqfwXjFUlAlkwKgHWrmWHO8xxO93ObnjYgPSH735EdBq5/IQDOkzFyXl/4 jPESXLV0g4zhjFb+G025Urpt81H5aYvYBIBO/86ISXjwwnvWjdrDRMty27lmgqFZuWIz ylx0HaXoPRD2fh7Ctac1rOH0T6BilzVwb3/DNdmqvqPB9JEgfIZKzDODsy/ihOVgpreS 9FekrJ2MRRuQnzu1V4JYQbJRwMgML9WXhkG0MGR1Fdu6qCbypAykM3kei5HPO1cGw/Bz 4GMCItmBwHo+e1ibUItow3LaTYSVlSnrOhJV2TvKIjOdH5i1DZLknep5dvDw/7LOHXgd b5+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689023558; x=1691615558; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=MAOE0b48L7RYO+dNddsG8MKfks+ILkcQuQu91zv6YCs=; b=OZGKQRWmjyRNoXmTFzUvFk/nsLcTZqhKyZyuHj0mi8672j1SJvuCVdeG9MtmTvKTl3 Wj9tEj6P+SBCuUBPMyuCwm06wA7ATx6z6pp3OW770hAe9QJm60jriZ3DIl9w9Aam0GMr CMUDfMLoB+hTSz9hBpM9yjX5ysksnraI7+HQ6/hSdoj0vg45sgiZ/vFxdnaVqqju1iHZ KLeePmGuAKkTs72moG9Tev8OgvptII9Y7uNV+7hfn8dztqN526tBmyILhOljARUSypi+ O8j8x9Y0RzonYyUYUvgTkQfUZhiBJeFGAdYPi00yWn/RqFjw5yXxRKEQYzVDk8XJTfMX nQ/A== X-Gm-Message-State: ABy/qLZWvdGJTuDHIbtxRSyDnkMR9D88GsSz3FcyX34hNVzE26JaQIHS U/PWoJI5fhrPa5g3bSjE6LmJVXZzVdPwqp4ec8R+iA== X-Google-Smtp-Source: APBJJlGlVEvQj+E/Bbyb/m7KAXoUoekG8y6psxC5jD8XbXmhSm8qbfQG0ZHOwrbzNIIk9Gz8841oDQ== X-Received: by 2002:a05:6830:2001:b0:6b9:6287:618b with SMTP id e1-20020a056830200100b006b96287618bmr3008107otp.24.1689023558094; Mon, 10 Jul 2023 14:12:38 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id j63-20020a25d242000000b00bcf788d09e3sm141931ybg.34.2023.07.10.14.12.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Jul 2023 14:12:37 -0700 (PDT) Date: Mon, 10 Jul 2023 17:12:36 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Chris Torek , Derrick Stolee , Jeff King , Junio C Hamano , Patrick Steinhardt Subject: [PATCH v5 12/16] refs.h: let `for_each_namespaced_ref()` take excluded patterns Message-ID: <7e72c23c8a4e579d324446910e8b680cca560d8a.1689023520.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org A future commit will want to call `for_each_namespaced_ref()` with a list of excluded patterns. We could introduce a variant of that function, say, `for_each_namespaced_ref_exclude()` which takes the extra parameter, and reimplement the original function in terms of that. But all but one caller (in `http-backend.c`) will supply the new parameter, so add the new parameter to `for_each_namespaced_ref()` itself instead of introducing a new function. For now, supply NULL for the list of excluded patterns at all callers to avoid changing behavior, which we will do in a future change. Signed-off-by: Taylor Blau --- http-backend.c | 2 +- refs.c | 5 +++-- refs.h | 7 ++++++- upload-pack.c | 6 +++--- 4 files changed, 13 insertions(+), 7 deletions(-) diff --git a/http-backend.c b/http-backend.c index e1969c05dc6..895fac35f79 100644 --- a/http-backend.c +++ b/http-backend.c @@ -560,7 +560,7 @@ static void get_info_refs(struct strbuf *hdr, char *arg UNUSED) } else { select_getanyfile(hdr); - for_each_namespaced_ref(show_text_ref, &buf); + for_each_namespaced_ref(NULL, show_text_ref, &buf); send_strbuf(hdr, "text/plain", &buf); } strbuf_release(&buf); diff --git a/refs.c b/refs.c index 64dabc40cae..3065e514fd0 100644 --- a/refs.c +++ b/refs.c @@ -1662,13 +1662,14 @@ int for_each_replace_ref(struct repository *r, each_repo_ref_fn fn, void *cb_dat DO_FOR_EACH_INCLUDE_BROKEN, cb_data); } -int for_each_namespaced_ref(each_ref_fn fn, void *cb_data) +int for_each_namespaced_ref(const char **exclude_patterns, + each_ref_fn fn, void *cb_data) { struct strbuf buf = STRBUF_INIT; int ret; strbuf_addf(&buf, "%srefs/", get_git_namespace()); ret = do_for_each_ref(get_main_ref_store(the_repository), - buf.buf, NULL, fn, 0, 0, cb_data); + buf.buf, exclude_patterns, fn, 0, 0, cb_data); strbuf_release(&buf); return ret; } diff --git a/refs.h b/refs.h index fa3ce4b0bc1..7871faed632 100644 --- a/refs.h +++ b/refs.h @@ -382,7 +382,12 @@ int for_each_glob_ref_in(each_ref_fn fn, const char *pattern, const char *prefix, void *cb_data); int head_ref_namespaced(each_ref_fn fn, void *cb_data); -int for_each_namespaced_ref(each_ref_fn fn, void *cb_data); +/* + * references matching any pattern in "exclude_patterns" are omitted from the + * result set on a best-effort basis. + */ +int for_each_namespaced_ref(const char **exclude_patterns, + each_ref_fn fn, void *cb_data); /* can be used to learn about broken ref and symref */ int refs_for_each_rawref(struct ref_store *refs, each_ref_fn fn, void *cb_data); diff --git a/upload-pack.c b/upload-pack.c index cfb61ccbb5c..da4f17f64ac 100644 --- a/upload-pack.c +++ b/upload-pack.c @@ -855,7 +855,7 @@ static void deepen(struct upload_pack_data *data, int depth) * marked with OUR_REF. */ head_ref_namespaced(check_ref, data); - for_each_namespaced_ref(check_ref, data); + for_each_namespaced_ref(NULL, check_ref, data); get_reachable_list(data, &reachable_shallows); result = get_shallow_commits(&reachable_shallows, @@ -1392,7 +1392,7 @@ void upload_pack(const int advertise_refs, const int stateless_rpc, if (advertise_refs) data.no_done = 1; head_ref_namespaced(send_ref, &data); - for_each_namespaced_ref(send_ref, &data); + for_each_namespaced_ref(NULL, send_ref, &data); if (!data.sent_capabilities) { const char *refname = "capabilities^{}"; write_v0_ref(&data, refname, refname, null_oid()); @@ -1406,7 +1406,7 @@ void upload_pack(const int advertise_refs, const int stateless_rpc, packet_flush(1); } else { head_ref_namespaced(check_ref, &data); - for_each_namespaced_ref(check_ref, &data); + for_each_namespaced_ref(NULL, check_ref, &data); } if (!advertise_refs) { From patchwork Mon Jul 10 21:12:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13307672 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C4FC3EB64DA for ; Mon, 10 Jul 2023 21:13:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232032AbjGJVNK (ORCPT ); Mon, 10 Jul 2023 17:13:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37792 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231908AbjGJVMo (ORCPT ); Mon, 10 Jul 2023 17:12:44 -0400 Received: from mail-yb1-xb35.google.com (mail-yb1-xb35.google.com [IPv6:2607:f8b0:4864:20::b35]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DBAA7FB for ; Mon, 10 Jul 2023 14:12:41 -0700 (PDT) Received: by mail-yb1-xb35.google.com with SMTP id 3f1490d57ef6-c4dd86f5d78so7239786276.0 for ; Mon, 10 Jul 2023 14:12:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1689023561; x=1691615561; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=x4fGxHFlcIAZYT2CBLKurvB6zt4YEHFgrdipdCyPcek=; b=rJ6onLwZ/2JUV5gx5iNqNhIOI+rzcu2pAp473mWkQusJ82/w3/TjvhQDowMJeA+Oc9 4e+WzchDCEUswkTN5aSK3Iwf5ZGMcigB6uRRPOX/n2yzqyPsFwGMfmTnGEqDDa5NuaJh Z2B2N+v7DHjOJOfv2EE+vsu8w5DvDWg9ro5Y7Y7QpzfOKtwHNj+VAD5jNxLtuVPevSAk BabZphffY4P2qerSZy/xSTW9GosDRuzALZT3dPdMebnMn6B5k7I57mkmX96LbDN+YWxo lJKGn4XJopgi+4lGL983sTFtDowfKpoqECs1kQeugN2D+hrRjeqBL4SjarwiGNtWZjE7 jHPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689023561; x=1691615561; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=x4fGxHFlcIAZYT2CBLKurvB6zt4YEHFgrdipdCyPcek=; b=j/kbsa6WnfsZr/b1QNbI0suBXl9Uiev/WS4JIfOrClX15epDVx35IJU6GnSYJXt0A5 57AJBYf/81/8R4vNKsX1xjAFmv+HguRJKTM0HKtMorzecAdQUdb+UetjrkwoqBcL7zhu AgVqBQky856Bg0fiBEo6LHeQAKBs0qd71L5CsO2vQWz9a6qkFlmE2c/L+/KlVXgy0LB5 SfyJcgMufQttYlaFbukFQivjcerojjJ5cyOjZ0LixuQtGwOVBfSAlROamT3WO7hkOJUD 6ZYmLNHfuKelxKlM4TpL4y7TdjUTG1/DYZj24aABO8UHPDNUk2XoPsrbxE1F2znKOaPh znMQ== X-Gm-Message-State: ABy/qLbeFlmoP2uV1g+bp2wj5QRyihUnZtEiz2zH8rAUR/ZQ6pWFmSsM u+uJO7xuFSUlaTgNNitFCyHd47159aacQUv2xWSJYQ== X-Google-Smtp-Source: APBJJlGrAQeDcwwWTzUmTymCBj9yIyOEiRYugYpP5zlwAQ3qD3EAq1QDjCh5o7Djw2cv0oJn18pkmA== X-Received: by 2002:a25:b30d:0:b0:c47:3430:bcb0 with SMTP id l13-20020a25b30d000000b00c473430bcb0mr13448058ybj.26.1689023560961; Mon, 10 Jul 2023 14:12:40 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id z1-20020a25ad81000000b00c581e8d0ae4sm136541ybi.56.2023.07.10.14.12.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Jul 2023 14:12:40 -0700 (PDT) Date: Mon, 10 Jul 2023 17:12:39 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Chris Torek , Derrick Stolee , Jeff King , Junio C Hamano , Patrick Steinhardt Subject: [PATCH v5 13/16] refs.h: implement `hidden_refs_to_excludes()` Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In subsequent commits, we'll teach `receive-pack` and `upload-pack` to use the new jump list feature in the packed-refs iterator by ignoring references which are mentioned via its respective hideRefs lists. However, the packed-ref jump lists cannot handle un-hiding rules (that begin with '!'), or namespace comparisons (that begin with '^'). Add a convenience function to the refs.h API to detect when either of these conditions are met, and returns an appropriate value to pass as excluded patterns. Suggested-by: Jeff King Signed-off-by: Taylor Blau --- refs.c | 24 ++++++++++++++++++++++++ refs.h | 6 ++++++ 2 files changed, 30 insertions(+) diff --git a/refs.c b/refs.c index 3065e514fd0..0916a6ba582 100644 --- a/refs.c +++ b/refs.c @@ -1482,6 +1482,30 @@ int ref_is_hidden(const char *refname, const char *refname_full, return 0; } +const char **hidden_refs_to_excludes(const struct strvec *hide_refs) +{ + const char **pattern; + for (pattern = hide_refs->v; *pattern; pattern++) { + /* + * We can't feed any excludes from hidden refs config + * sections, since later rules may override previous + * ones. For example, with rules "refs/foo" and + * "!refs/foo/bar", we should show "refs/foo/bar" (and + * everything underneath it), but the earlier exclusion + * would cause us to skip all of "refs/foo". We + * likewise don't implement the namespace stripping + * required for '^' rules. + * + * Both are possible to do, but complicated, so avoid + * populating the jump list at all if we see either of + * these patterns. + */ + if (**pattern == '!' || **pattern == '^') + return NULL; + } + return hide_refs->v; +} + const char *find_descendant_ref(const char *dirname, const struct string_list *extras, const struct string_list *skip) diff --git a/refs.h b/refs.h index 7871faed632..23211a5ea1c 100644 --- a/refs.h +++ b/refs.h @@ -837,6 +837,12 @@ int parse_hide_refs_config(const char *var, const char *value, const char *, */ int ref_is_hidden(const char *, const char *, const struct strvec *); +/* + * Returns an array of patterns to use as excluded_patterns, if none of the + * hidden references use the token '!' or '^'. + */ +const char **hidden_refs_to_excludes(const struct strvec *hide_refs); + /* Is this a per-worktree ref living in the refs/ namespace? */ int is_per_worktree_ref(const char *refname); From patchwork Mon Jul 10 21:12:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13307671 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C5D6BC0015E for ; Mon, 10 Jul 2023 21:13:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232001AbjGJVNJ (ORCPT ); Mon, 10 Jul 2023 17:13:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37872 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230107AbjGJVMs (ORCPT ); Mon, 10 Jul 2023 17:12:48 -0400 Received: from mail-yw1-x1132.google.com (mail-yw1-x1132.google.com [IPv6:2607:f8b0:4864:20::1132]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 90733E49 for ; Mon, 10 Jul 2023 14:12:44 -0700 (PDT) Received: by mail-yw1-x1132.google.com with SMTP id 00721157ae682-57688a146ecso61915677b3.2 for ; Mon, 10 Jul 2023 14:12:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1689023563; x=1691615563; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=Yky4hiMB8h603Qn4mYJPfEoeCxbIkWslKnjo8LiP6xs=; b=KqZ4nXMaXG5cHxTO5xLic8FXUi2LMJZ1Tu9JB3qNopKUOufMhUhgIC2t7gG8pf98DO 0Fvr9pyU6I1ldJ7mbvxUxg81ZTTaEsaBiGF9Qtqy1rZ57asEAxyJ/4Q+lmwk4npCgT6q bSx6n7GQ5eIfb8JF2+U8grYOdickOrHGmBbhcvlgqrxI4llW+DyDkGV4UjNMdWgLuWwy BqTVrr5MbYem23Kdnuly7tfilnwqlB4E435I4glJVWCeWOc7NP6oU/T1Hl/412w9p3AB 8jB2tXjfv4RMgkDJzOL8LLyqJzbn43C3H2HRX8oDi2u2No7tHGnZrt/fGbt9T6XeMaRL gnhw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689023563; x=1691615563; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Yky4hiMB8h603Qn4mYJPfEoeCxbIkWslKnjo8LiP6xs=; b=HUIcqwJiAjweb8/nCG9JLCi8OIimCLgPCjR0A99iivP/6RyU+WE5PoRRYhhzd5Ef1B V1+0+n3xu9g7v+3ukQ0SYC9Al4IuHVZvH4em514aggHMKoxglr/kqD5eNdhyM0QsZTdQ lwJhqvUkROHbBQXwMeTKCSUxWqKhQN2RGiM80eXPKAxohRxvt2b+KAup44qdXxPdkMdL IfLJ1uHhlVSpKlJT2MnJ9/JDCkVNcHSJwmQWUuFzvsGWca/m0Bj1yB7PeGi4aBOEooxx /Uo0V+pg/hwvGXwURLCd3M+gj9t+q3GOfJktm2tOnODZswaMnVSBzIiFJhHIXea7OfxG R4kA== X-Gm-Message-State: ABy/qLZQFBB0onzYJruvkFFhI1t5jmmktQMotgFmlUsvuEZ+NMfuhWnv kQZmfU8vUvIwj7CGR4vWUtlsBCQhsOJFA9BAgyd2mQ== X-Google-Smtp-Source: APBJJlFuIFjRSmTOy+pzpQiJaYNgmTQUTsFNxAkXoJ0rIBaLVBakHhwts91n3TIFcCMtj7GUZowR2w== X-Received: by 2002:a0d:e208:0:b0:579:16e0:b1bf with SMTP id l8-20020a0de208000000b0057916e0b1bfmr12107665ywe.12.1689023563687; Mon, 10 Jul 2023 14:12:43 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id x125-20020a0dd583000000b00579df9098e3sm190954ywd.38.2023.07.10.14.12.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Jul 2023 14:12:43 -0700 (PDT) Date: Mon, 10 Jul 2023 17:12:42 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Chris Torek , Derrick Stolee , Jeff King , Junio C Hamano , Patrick Steinhardt Subject: [PATCH v5 14/16] builtin/receive-pack.c: avoid enumerating hidden references Message-ID: <96aada36f9b7322ea408d1a49f94e051265596f0.1689023520.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Now that `refs_for_each_fullref_in()` has the ability to avoid enumerating references matching certain pattern(s), use that to avoid visiting hidden refs when constructing the ref advertisement via receive-pack. Note that since this exclusion is best-effort, we still need `show_ref_cb()` to check whether or not each reference is hidden or not before including it in the advertisement. As was the case when applying this same optimization to `upload-pack`, `receive-pack`'s reference advertisement phase can proceed much quicker by avoiding enumerating references that will not be part of the advertisement. (Below, we're still using linux.git with one hidden refs/pull/N ref per commit): $ hyperfine -L v ,.compile 'git{v} -c transfer.hideRefs=refs/pull receive-pack --advertise-refs .git' Benchmark 1: git -c transfer.hideRefs=refs/pull receive-pack --advertise-refs .git Time (mean ± σ): 89.1 ms ± 1.7 ms [User: 82.0 ms, System: 7.0 ms] Range (min … max): 87.7 ms … 95.5 ms 31 runs Benchmark 2: git.compile -c transfer.hideRefs=refs/pull receive-pack --advertise-refs .git Time (mean ± σ): 4.5 ms ± 0.2 ms [User: 0.5 ms, System: 3.9 ms] Range (min … max): 4.1 ms … 5.6 ms 508 runs Summary 'git.compile -c transfer.hideRefs=refs/pull receive-pack --advertise-refs .git' ran 20.00 ± 1.05 times faster than 'git -c transfer.hideRefs=refs/pull receive-pack --advertise-refs .git' Signed-off-by: Taylor Blau --- builtin/receive-pack.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c index 56e8b4e6af4..90aedb97cc8 100644 --- a/builtin/receive-pack.c +++ b/builtin/receive-pack.c @@ -339,7 +339,9 @@ static void write_head_info(void) { static struct oidset seen = OIDSET_INIT; - for_each_ref(show_ref_cb, &seen); + refs_for_each_fullref_in(get_main_ref_store(the_repository), "", + hidden_refs_to_excludes(&hidden_refs), + show_ref_cb, &seen); for_each_alternate_ref(show_one_alternate_ref, &seen); oidset_clear(&seen); if (!sent_capabilities) From patchwork Mon Jul 10 21:12:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13307673 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 013FDEB64D9 for ; Mon, 10 Jul 2023 21:13:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232855AbjGJVNV (ORCPT ); Mon, 10 Jul 2023 17:13:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37656 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232302AbjGJVMt (ORCPT ); Mon, 10 Jul 2023 17:12:49 -0400 Received: from mail-ot1-x32e.google.com (mail-ot1-x32e.google.com [IPv6:2607:f8b0:4864:20::32e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 291B7E6B for ; Mon, 10 Jul 2023 14:12:47 -0700 (PDT) Received: by mail-ot1-x32e.google.com with SMTP id 46e09a7af769-6b71cdb47e1so4454991a34.2 for ; Mon, 10 Jul 2023 14:12:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1689023566; x=1691615566; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=QRhKRU6IaQRy/A9sdRYUIiKuR+/G44O9TxNQUPacLsQ=; b=3iuDb4WPfvZ877/54aVBTPnZiWk3pub9Khfo4wE786mK2PxXB2R1rblULiqTba0qGs CJ7CXp2GmHp8l3w/LN7k12vtgUf0PXb9jS4/No3/N5XwPjAfyRSsOw1Gml/jewHX6zQI vc+iP+qw2MoBLvZwEOE09OjeI+vyBUIbs3JcoUu5urv0e1bgr1/1QzwndkAnxYnhh79C Makm8GpyqU6TdgdFbgjVysE73s2BlOQeURqDx5UHRmCKMAHd1KjcI9M3is/YD+FKLS+j XqpUGKGWNq+4xSvZmQUrCxvHECPTcnAHtbMJMa7SPYB8/iogz9OYkPQcCC3v9xezkIM5 x9aQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689023566; x=1691615566; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=QRhKRU6IaQRy/A9sdRYUIiKuR+/G44O9TxNQUPacLsQ=; b=dVOnvDviErJenYgiW4D0DNlm1iG7Xtjf7r+mxXYq4i3TcJvJQuvL7jA2RWt5+DeSsl Lr82b8er7mMzCiEKDltJI/1ktCRgBW7recnk5x0h/Xe3bkq0JtEYHLQNVCyDSt8BeCYe gSn0Xa+2Lht1sWegQuOraCI0xl7kxW56PsY5ILNmomE5HiGPVcWXQZvLK4kmCpfqFpMw +poVwYMBQMmYcENPP8vdmq46wHROdCxlhdenXR/8G8R7h/H6oy8f3MIm5wsZxlpHOsFl l/JS/7QBf75geHOgPMV+yKPZIHWnK0jKfdFlVSp3CDq9B2JVGOq16X2nseko9/Jh2yMf W6hg== X-Gm-Message-State: ABy/qLYt9QNBIKNdeH2Crf/LRdBdnl21ZnRt50tk3mfdblLq1tTXhCc0 ii4lJ5GamXY0Q7cIvkg/hZyJXeuvo87G52vz84r5+w== X-Google-Smtp-Source: APBJJlHzf/pgZYQtA0eWNVFHi+PBPmC3yja98GecB7tthvyDZwMnaMgEwL+VZ8fJmqkKp6ocixU1/Q== X-Received: by 2002:a9d:6d82:0:b0:6b2:dc79:5870 with SMTP id x2-20020a9d6d82000000b006b2dc795870mr13765525otp.36.1689023566355; Mon, 10 Jul 2023 14:12:46 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id w8-20020a253008000000b00be4f34d419asm141058ybw.37.2023.07.10.14.12.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Jul 2023 14:12:46 -0700 (PDT) Date: Mon, 10 Jul 2023 17:12:45 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Chris Torek , Derrick Stolee , Jeff King , Junio C Hamano , Patrick Steinhardt Subject: [PATCH v5 15/16] upload-pack.c: avoid enumerating hidden refs where possible Message-ID: <8544a647798de68eb46f9f5c5b269d03ed3500a5.1689023520.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In a similar fashion as a previous commit, teach `upload-pack` to avoid enumerating hidden references where possible. Note, however, that there are certain cases where cannot avoid enumerating even hidden references, in particular when either of: - `uploadpack.allowTipSHA1InWant`, or - `uploadpack.allowReachableSHA1InWant` are set, corresponding to `ALLOW_TIP_SHA1` and `ALLOW_REACHABLE_SHA1`, respectively. When either of these bits are set, upload-pack's `is_our_ref()` function needs to consider the `HIDDEN_REF` bit of the referent's object flags. So we must visit all references, including the hidden ones, in order to mark their referents with the `HIDDEN_REF` bit. When neither `ALLOW_TIP_SHA1` nor `ALLOW_REACHABLE_SHA1` are set, the `is_our_ref()` function considers only the `OUR_REF` bit, and not the `HIDDEN_REF` one. `OUR_REF` is applied via `mark_our_ref()`, and only to objects at the tips of non-hidden references, so we do not need to visit hidden references in this case. When neither of those bits are set, `upload-pack` can potentially avoid enumerating a large number of references. In the same example as a previous commit (linux.git with one hidden reference per commit, "refs/pull/N"): $ printf 0000 >in $ hyperfine --warmup=1 \ 'git -c transfer.hideRefs=refs/pull upload-pack . --- upload-pack.c | 37 +++++++++++++++++++++++++++++++------ 1 file changed, 31 insertions(+), 6 deletions(-) diff --git a/upload-pack.c b/upload-pack.c index da4f17f64ac..db4709adb77 100644 --- a/upload-pack.c +++ b/upload-pack.c @@ -602,11 +602,36 @@ static int get_common_commits(struct upload_pack_data *data, } } +static int allow_hidden_refs(enum allow_uor allow_uor) +{ + if ((allow_uor & ALLOW_ANY_SHA1) == ALLOW_ANY_SHA1) + return 1; + return !(allow_uor & (ALLOW_TIP_SHA1 | ALLOW_REACHABLE_SHA1)); +} + +static void for_each_namespaced_ref_1(each_ref_fn fn, + struct upload_pack_data *data) +{ + const char **excludes = NULL; + /* + * If `data->allow_uor` allows fetching hidden refs, we need to + * mark all references (including hidden ones), to check in + * `is_our_ref()` below. + * + * Otherwise, we only care about whether each reference's object + * has the OUR_REF bit set or not, so do not need to visit + * hidden references. + */ + if (allow_hidden_refs(data->allow_uor)) + excludes = hidden_refs_to_excludes(&data->hidden_refs); + + for_each_namespaced_ref(excludes, fn, data); +} + + static int is_our_ref(struct object *o, enum allow_uor allow_uor) { - int allow_hidden_ref = (allow_uor & - (ALLOW_TIP_SHA1 | ALLOW_REACHABLE_SHA1)); - return o->flags & ((allow_hidden_ref ? HIDDEN_REF : 0) | OUR_REF); + return o->flags & ((allow_hidden_refs(allow_uor) ? 0 : HIDDEN_REF) | OUR_REF); } /* @@ -855,7 +880,7 @@ static void deepen(struct upload_pack_data *data, int depth) * marked with OUR_REF. */ head_ref_namespaced(check_ref, data); - for_each_namespaced_ref(NULL, check_ref, data); + for_each_namespaced_ref_1(check_ref, data); get_reachable_list(data, &reachable_shallows); result = get_shallow_commits(&reachable_shallows, @@ -1392,7 +1417,7 @@ void upload_pack(const int advertise_refs, const int stateless_rpc, if (advertise_refs) data.no_done = 1; head_ref_namespaced(send_ref, &data); - for_each_namespaced_ref(NULL, send_ref, &data); + for_each_namespaced_ref_1(send_ref, &data); if (!data.sent_capabilities) { const char *refname = "capabilities^{}"; write_v0_ref(&data, refname, refname, null_oid()); @@ -1406,7 +1431,7 @@ void upload_pack(const int advertise_refs, const int stateless_rpc, packet_flush(1); } else { head_ref_namespaced(check_ref, &data); - for_each_namespaced_ref(NULL, check_ref, &data); + for_each_namespaced_ref_1(check_ref, &data); } if (!advertise_refs) { From patchwork Mon Jul 10 21:12:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13307674 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 81B20EB64DA for ; Mon, 10 Jul 2023 21:13:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231815AbjGJVN0 (ORCPT ); Mon, 10 Jul 2023 17:13:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37682 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232331AbjGJVMu (ORCPT ); Mon, 10 Jul 2023 17:12:50 -0400 Received: from mail-oo1-xc2d.google.com (mail-oo1-xc2d.google.com [IPv6:2607:f8b0:4864:20::c2d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 03DD4BF for ; Mon, 10 Jul 2023 14:12:50 -0700 (PDT) Received: by mail-oo1-xc2d.google.com with SMTP id 006d021491bc7-5634db21a58so3889109eaf.0 for ; Mon, 10 Jul 2023 14:12:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1689023569; x=1691615569; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=8jfTE4UGW/QR3/B1v/IqptkS5aPv7JtbMC+crI2nYtc=; b=mPXnm7HDbJAWQWIJvO4TsNE/I/8fSrSUthJZC21pKZH8kORTjIh20m29N7006st5Nh YseZGmNkNck8vHjceMNTFHhT/3v2lBCyejQmR4UO+zdOHpsg9L0+xAG8YT2201QuS839 mGiLyRgIxHHWxwxw338Dh5GoVbqu5UFEcV0x/5tyCHe5Q4BqCSwbNdsLsf2VvhPYjHDQ QHYGR6BjvnlN7Tjc5oiw8CuPj3xbdukcE8BLktgkiJML6p0LrnK9lfF7iuhiMKg0Ka+X j+loRC8sVhWFJ+TeC6hDssIS5H8+qRYUYfTbeIVO6dDDXm8T8rLiX0EXPdD9W/t0HoeJ +i7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689023569; x=1691615569; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=8jfTE4UGW/QR3/B1v/IqptkS5aPv7JtbMC+crI2nYtc=; b=Znfkilt9rH4Ip6htYYQBNSWFYVEdiAf/tK/9sRtWTzGAFtN3p6f6vLGuuhyLuwBHQc c9loF2bYxkmqkh7UBtBcqq6duXo8q+RDqLrHL37NXvDtk66N3gteKPaCN+mFQ2SygKPx y0ns4a4V4jyp68EEa/A2AZuo7PVygxqBvWJnI+qISw9uhpJRzr07bsF2Ckl+/scf4HFy VRfK3xuWuYAkvewE5PzyKOZO/ikcOy+pDXZuORAZthu/cnXSFcb3YmFohXizPJQxyH3j K8Ql+DI4SWP7mjxWAVhHSOgriklkIiKKenG0UT96uziily7o3HSpigYCwr419GZyYrif LehA== X-Gm-Message-State: ABy/qLZBeG/hfNPieKNk4vwJZzMJy5KxYNc1WMNeJfXbXEZfJQylhFWa YuDTb2y3cU4MpKlxoSbpbEwYGFS2vIPMwk7udjod2w== X-Google-Smtp-Source: APBJJlFsTmDdsKIRp7tb0UsIgt6KhybSAANqwyfe0cRRdiNs4oD0aI/KGK/HNa9/N1maOxq2dfVa9w== X-Received: by 2002:a05:6808:1244:b0:3a3:61df:da with SMTP id o4-20020a056808124400b003a361df00damr16336989oiv.53.1689023569207; Mon, 10 Jul 2023 14:12:49 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id y80-20020a0dd653000000b0055a21492192sm182181ywd.115.2023.07.10.14.12.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Jul 2023 14:12:48 -0700 (PDT) Date: Mon, 10 Jul 2023 17:12:48 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Chris Torek , Derrick Stolee , Jeff King , Junio C Hamano , Patrick Steinhardt Subject: [PATCH v5 16/16] ls-refs.c: avoid enumerating hidden refs where possible Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In a similar fashion as in previous commits, teach `ls-refs` to avoid enumerating hidden references where possible. As before, this is linux.git with one hidden reference per commit. $ hyperfine -L v ,.compile 'git{v} -c protocol.version=2 ls-remote .' Benchmark 1: git -c protocol.version=2 ls-remote . Time (mean ± σ): 89.8 ms ± 0.6 ms [User: 84.3 ms, System: 5.7 ms] Range (min … max): 88.8 ms … 91.3 ms 32 runs Benchmark 2: git.compile -c protocol.version=2 ls-remote . Time (mean ± σ): 6.5 ms ± 0.1 ms [User: 2.4 ms, System: 4.3 ms] Range (min … max): 6.2 ms … 8.3 ms 397 runs Summary 'git.compile -c protocol.version=2 ls-remote .' ran 13.85 ± 0.33 times faster than 'git -c protocol.version=2 ls-remote .' Signed-off-by: Taylor Blau --- ls-refs.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/ls-refs.c b/ls-refs.c index 9291ddfdf8c..0e49b932c30 100644 --- a/ls-refs.c +++ b/ls-refs.c @@ -194,7 +194,8 @@ int ls_refs(struct repository *r, struct packet_reader *request) strvec_push(&data.prefixes, ""); refs_for_each_fullref_in_prefixes(get_main_ref_store(r), get_git_namespace(), data.prefixes.v, - NULL, send_ref, &data); + hidden_refs_to_excludes(&data.hidden_refs), + send_ref, &data); packet_fflush(stdout); strvec_clear(&data.prefixes); strbuf_release(&data.buf);