From patchwork Mon May 15 19:23:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13242099 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 007E8C7EE25 for ; Mon, 15 May 2023 19:23:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245007AbjEOTXU (ORCPT ); Mon, 15 May 2023 15:23:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38846 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244979AbjEOTXS (ORCPT ); Mon, 15 May 2023 15:23:18 -0400 Received: from mail-yw1-x1133.google.com (mail-yw1-x1133.google.com [IPv6:2607:f8b0:4864:20::1133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 37C2A15244 for ; Mon, 15 May 2023 12:23:13 -0700 (PDT) Received: by mail-yw1-x1133.google.com with SMTP id 00721157ae682-55a79671a4dso194820687b3.2 for ; Mon, 15 May 2023 12:23:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1684178592; x=1686770592; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=GVc/88wfQhiMMJ+DSmT8cvyXoa0RTgflr/uznf2LK2c=; b=sypgIjiPvLscnm2ZU3noJspc9z2VcDMwY6Qmu4cWIM1rwx7nFcF2rbnv4Wh5wqOmJ6 j5M278FbOf4+YJpFARO53pdIWTGQxAvTxH2ha7/XVJyxm/OvMwtXfnpVHiLf9HZCXBJr xXmvTHcNYpIpqXo/52UwO/ZmFF4zmL6itgOKuBsTSWZOQxU3oYaSnABhKrEA+909ytBA YcdSSzvDQWnwNb2GR3QgBqxIqR2oQ3rPg2O14omESbGkMDz/Zr7wVMVtA5Mw1O5FawNI gZgP4jChJL9NL0Ww/HRT1l2D5knhuFE97kfUekUjgO3mCjUCH0lHXOPYSwkMNc5H+PzJ LhXQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684178592; x=1686770592; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=GVc/88wfQhiMMJ+DSmT8cvyXoa0RTgflr/uznf2LK2c=; b=QtHN8y8emX1tgrSl6uh/hiDteChUsThJtDQQD1xVvDcJIv/i3wqD1CM/c+NpmBQ6/Q j1LrR0tIj40b9YUe1OYQwW6UdOxSr6xh7MPc8XZe3r0HbfqMc+zq2f4G2w7BhCRNg9JG YMI2LI6a4mBX0BMVST8P5emBF5KvAQN55m2/Vgib+8UeEcUdsAad/2FmRgQu+NxKcnDz 36kq9bZysedIES9zhpdFZUryLy/t0KIT6NVuJrDUlbKM0sM+GezgxjAizIvntaA6ow58 uUWBn9SfK0Jb6S1YPVhuBPmal9PKCJdYi30WqogCWB6T+DSQaU1q8QNMlElZu8VIRhkv LSLg== X-Gm-Message-State: AC+VfDyxqE9jC+4vFNuoZJkyEXH3wts2wkZO2fqzIjDiVGfGVLvK7s4l gs98Li4c+/1QUI52RMslHGMUkCoegE5xaQNHUg9fEQ== X-Google-Smtp-Source: ACHHUZ6Lxklz0M4k6xUw+G5dNu1bT3A0CA8C+FXfzm2zx9aBbN2JrCFmYagV/ZZsq1oy2GmcF707Tg== X-Received: by 2002:a0d:c641:0:b0:54f:23a:4c64 with SMTP id i62-20020a0dc641000000b0054f023a4c64mr32744630ywd.2.1684178592140; Mon, 15 May 2023 12:23:12 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id n12-20020a819c4c000000b00556d4aac066sm6744ywa.133.2023.05.15.12.23.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 May 2023 12:23:11 -0700 (PDT) Date: Mon, 15 May 2023 15:23:10 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Chris Torek , Derrick Stolee , Jeff King , Junio C Hamano , Patrick Steinhardt Subject: [PATCH v2 01/16] refs.c: rename `ref_filter` Message-ID: <6cac42e70e9c1901761bd3068a9e22e60fe4deae.1684178576.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Jeff King The refs machinery has its own implementation of a `ref_filter` (used by `for-each-ref`), which is distinct from the `ref-filler.h` API (also used by `for-each-ref`, among other things). Rename the one within refs.c to more clearly indicate its purpose. Signed-off-by: Jeff King Signed-off-by: Taylor Blau --- refs.c | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/refs.c b/refs.c index d2a98e1c21..b9b77d2eff 100644 --- a/refs.c +++ b/refs.c @@ -375,8 +375,8 @@ char *resolve_refdup(const char *refname, int resolve_flags, oid, flags); } -/* The argument to filter_refs */ -struct ref_filter { +/* The argument to for_each_filter_refs */ +struct for_each_ref_filter { const char *pattern; const char *prefix; each_ref_fn *fn; @@ -409,10 +409,11 @@ int ref_exists(const char *refname) return refs_ref_exists(get_main_ref_store(the_repository), refname); } -static int filter_refs(const char *refname, const struct object_id *oid, - int flags, void *data) +static int for_each_filter_refs(const char *refname, + const struct object_id *oid, + int flags, void *data) { - struct ref_filter *filter = (struct ref_filter *)data; + struct for_each_ref_filter *filter = data; if (wildmatch(filter->pattern, refname, 0)) return 0; @@ -569,7 +570,7 @@ int for_each_glob_ref_in(each_ref_fn fn, const char *pattern, const char *prefix, void *cb_data) { struct strbuf real_pattern = STRBUF_INIT; - struct ref_filter filter; + struct for_each_ref_filter filter; int ret; if (!prefix && !starts_with(pattern, "refs/")) @@ -589,7 +590,7 @@ int for_each_glob_ref_in(each_ref_fn fn, const char *pattern, filter.prefix = prefix; filter.fn = fn; filter.cb_data = cb_data; - ret = for_each_ref(filter_refs, &filter); + ret = for_each_ref(for_each_filter_refs, &filter); strbuf_release(&real_pattern); return ret; From patchwork Mon May 15 19:23:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13242100 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E8379C7EE25 for ; Mon, 15 May 2023 19:23:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245039AbjEOTX3 (ORCPT ); Mon, 15 May 2023 15:23:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39082 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243497AbjEOTX1 (ORCPT ); Mon, 15 May 2023 15:23:27 -0400 Received: from mail-yw1-x112c.google.com (mail-yw1-x112c.google.com [IPv6:2607:f8b0:4864:20::112c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 40C6510A18 for ; Mon, 15 May 2023 12:23:16 -0700 (PDT) Received: by mail-yw1-x112c.google.com with SMTP id 00721157ae682-55a2691637bso190823417b3.0 for ; Mon, 15 May 2023 12:23:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1684178595; x=1686770595; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=oECcKFleKClFDx6LPgTiSCkStj9Y48tPM2y6FoncgkM=; b=Im4SgIRUt9zyaKeacxCYPvJXkPgzpNZP4/VHCo8Z/5RF+njX7MsRCYWj20NxiT3QV6 5bd7dKZTRpuIfVblq2RJywEAPrR8xhy3kWumxCAb+2/RWDRxRxiFDE7nBasU1UXElPOj wGQH6q9m7yAtvH3/xA1mcEoftwB283VtY6Xj9FzVL/NiX3yzcL9hoBi93vHMKkQaNiSR 3pTjtObpaRqOWkVmbJF5bsna6AzH3GlzhdDMbAkHzsLjVg04zQ3iGRSIMq7kG4ita1os is8BuLfZ/09YDjWT+P8k19972ifschCVlX6CpePEd9WKoeuDwdM5TmUG2qllDHwEZGGj juuQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684178595; x=1686770595; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=oECcKFleKClFDx6LPgTiSCkStj9Y48tPM2y6FoncgkM=; b=IPv0/+o8CKmEohR2jFFoU4j6PQNgQIBfvxt0iXhk8AKIjQxQZMGu209+ZptCq7jGBS vdYC3aPeCxKQuPuB+jyJrz2GY7hdExjbkmhV1jm+Lw5Ap5kY+YdR2FeOUxONM4+CpMwy GX58bgYBwTMkq4ezM7UaCmCXcBrP54h5u4OJxfL8xCggWdZSIQW9wom+4yWo2lKYPte0 C0+oKOSOe/VsAXNE2TtDoobVVAtIJV3LiM/z1PkbFVn6kGLDmGQ9ATItaZPom+hl7uRw q09LZKkwG9WWZeW312teBpCCjw4mxjK+mXo5iTg9JnEs3X47TidxTYVv/tN//ILmQQSk vJwA== X-Gm-Message-State: AC+VfDzBu1aVwE83jyXmC8VCRE9iGx2N1YRYDfECe3nABSxDfm3+qVcb pxe9OC/ReD42muxZ8gupaY/U3207WaachqP8c1ESpQ== X-Google-Smtp-Source: ACHHUZ4xqN5W6wQGdxR7Qe/yCxBHnVEgOwpv65SF901FDC2ZaIo09vEAPBIogTe55XDhUeiEvOKnZw== X-Received: by 2002:a81:53c5:0:b0:55d:c2c3:fbb8 with SMTP id h188-20020a8153c5000000b0055dc2c3fbb8mr29458255ywb.40.1684178595142; Mon, 15 May 2023 12:23:15 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id d133-20020a0ddb8b000000b0056183cdb2d9sm21697ywe.60.2023.05.15.12.23.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 May 2023 12:23:14 -0700 (PDT) Date: Mon, 15 May 2023 15:23:13 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Chris Torek , Derrick Stolee , Jeff King , Junio C Hamano , Patrick Steinhardt Subject: [PATCH v2 02/16] ref-filter.h: provide `REF_FILTER_INIT` Message-ID: <8dda7db73816c2e97ed6e195ac8ffb5321f649b7.1684178576.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Jeff King Provide a sane initialization value for `struct ref_filter`, which in a subsequent patch will be used to initialize a new field. In the meantime, fix a case in test-reach.c where its `ref_filter` is not even zero-initialized. Signed-off-by: Jeff King Signed-off-by: Taylor Blau --- builtin/branch.c | 3 +-- builtin/for-each-ref.c | 3 +-- builtin/tag.c | 3 +-- ref-filter.h | 3 +++ t/helper/test-reach.c | 2 +- 5 files changed, 7 insertions(+), 7 deletions(-) diff --git a/builtin/branch.c b/builtin/branch.c index 501c47657c..03bb8e414c 100644 --- a/builtin/branch.c +++ b/builtin/branch.c @@ -662,7 +662,7 @@ int cmd_branch(int argc, const char **argv, const char *prefix) int reflog = 0, quiet = 0, icase = 0, force = 0, recurse_submodules_explicit = 0; enum branch_track track; - struct ref_filter filter; + struct ref_filter filter = REF_FILTER_INIT; static struct ref_sorting *sorting; struct string_list sorting_options = STRING_LIST_INIT_DUP; struct ref_format format = REF_FORMAT_INIT; @@ -720,7 +720,6 @@ int cmd_branch(int argc, const char **argv, const char *prefix) setup_ref_filter_porcelain_msg(); - memset(&filter, 0, sizeof(filter)); filter.kind = FILTER_REFS_BRANCHES; filter.abbrev = -1; diff --git a/builtin/for-each-ref.c b/builtin/for-each-ref.c index 695fc8f4a5..99ccb73518 100644 --- a/builtin/for-each-ref.c +++ b/builtin/for-each-ref.c @@ -24,7 +24,7 @@ int cmd_for_each_ref(int argc, const char **argv, const char *prefix) struct string_list sorting_options = STRING_LIST_INIT_DUP; int maxcount = 0, icase = 0, omit_empty = 0; struct ref_array array; - struct ref_filter filter; + struct ref_filter filter = REF_FILTER_INIT; struct ref_format format = REF_FORMAT_INIT; struct strbuf output = STRBUF_INIT; struct strbuf err = STRBUF_INIT; @@ -61,7 +61,6 @@ int cmd_for_each_ref(int argc, const char **argv, const char *prefix) }; memset(&array, 0, sizeof(array)); - memset(&filter, 0, sizeof(filter)); format.format = "%(objectname) %(objecttype)\t%(refname)"; diff --git a/builtin/tag.c b/builtin/tag.c index 1850a6a6fd..6b41bb7374 100644 --- a/builtin/tag.c +++ b/builtin/tag.c @@ -443,7 +443,7 @@ int cmd_tag(int argc, const char **argv, const char *prefix) struct msg_arg msg = { .buf = STRBUF_INIT }; struct ref_transaction *transaction; struct strbuf err = STRBUF_INIT; - struct ref_filter filter; + struct ref_filter filter = REF_FILTER_INIT; struct ref_sorting *sorting; struct string_list sorting_options = STRING_LIST_INIT_DUP; struct ref_format format = REF_FORMAT_INIT; @@ -501,7 +501,6 @@ int cmd_tag(int argc, const char **argv, const char *prefix) git_config(git_tag_config, &sorting_options); memset(&opt, 0, sizeof(opt)); - memset(&filter, 0, sizeof(filter)); filter.lines = -1; opt.sign = -1; diff --git a/ref-filter.h b/ref-filter.h index 430701cfb7..a920f73b29 100644 --- a/ref-filter.h +++ b/ref-filter.h @@ -92,6 +92,9 @@ struct ref_format { struct string_list bases; }; +#define REF_FILTER_INIT { \ + .points_at = OID_ARRAY_INIT, \ +} #define REF_FORMAT_INIT { \ .use_color = -1, \ .bases = STRING_LIST_INIT_DUP, \ diff --git a/t/helper/test-reach.c b/t/helper/test-reach.c index 5b6f217441..ef58f10c2d 100644 --- a/t/helper/test-reach.c +++ b/t/helper/test-reach.c @@ -139,7 +139,7 @@ int cmd__reach(int ac, const char **av) printf("%s(X,_,_,0,0):%d\n", av[1], can_all_from_reach_with_flag(&X_obj, 2, 4, 0, 0)); } else if (!strcmp(av[1], "commit_contains")) { - struct ref_filter filter; + struct ref_filter filter = REF_FILTER_INIT; struct contains_cache cache; init_contains_cache(&cache); From patchwork Mon May 15 19:23:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13242101 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 70D51C7EE26 for ; Mon, 15 May 2023 19:23:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245022AbjEOTXc (ORCPT ); Mon, 15 May 2023 15:23:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39100 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244998AbjEOTX2 (ORCPT ); Mon, 15 May 2023 15:23:28 -0400 Received: from mail-yw1-x1132.google.com (mail-yw1-x1132.google.com [IPv6:2607:f8b0:4864:20::1132]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0C662100DE for ; Mon, 15 May 2023 12:23:19 -0700 (PDT) Received: by mail-yw1-x1132.google.com with SMTP id 00721157ae682-55d2e87048cso190604887b3.1 for ; Mon, 15 May 2023 12:23:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1684178598; x=1686770598; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=B/guUdyNpi75FMSjjUWiFCSDmxbsAUlqZEhvtlsyOCI=; b=pycF5CB5nOD5HTTj/3lDANb14wvUIfhZwpQLPt0dY6GOotayKbG9Cr/eY5/HoBhc0x cZO8+bK2oU6vD6spXn8wPqTlZPhyEdosfI4zEjxq8+BulhnkdlaKogdhXJnCwhmbmaQG khAd9p6UbEQ4Ht2IxduwvrjdvhTVil1lrBsY/ri7d92ZRBUwxV+QHtbY85cKm824aACY sy0IgUZr+6JxG1GTZnkRAgwx1AtKebR0ssMkPYjOpkxaEVyPFT3PKh0irk5LS+lYE1/8 kecMIpwglcVtt4RMtTJRo674/st7fzNSdio8YT/00DTqexRtcMwxgmJ0Ea1ubZ1Lo/jU 2yTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684178598; x=1686770598; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=B/guUdyNpi75FMSjjUWiFCSDmxbsAUlqZEhvtlsyOCI=; b=GXBhs8mhIGAkmqASTDv7E+TS8yOTnDfVIOi7ylD4I3ZnH5HIxb1vxEFUWspqaez02z EokymCPydZLd9XzPAPJRALxYbfdcNU/8Kt9MovtVCSPScwTdEvsGb3Exxwn6rcJnL6VY PSgF+sEY9Bqgm34u8U+G3jbIagBw8rvklYbZNiK4HFe61C9yg2uU/D7hKGJh5MKKixpZ GSLHQBfrUqXVWRd3CGQxS7sKQuO0GIhoHyPPcUrWIwodCJxXNlkKq5KoFdQOECKpx+f4 kgioTaGvNOpHegyeJyK0rw7iMe1/wx0rwdYHG9Ehrs7B7kz0PaVPWSEhqWTyCg2t9na8 kWOg== X-Gm-Message-State: AC+VfDyAWvWhgHqiy9zwkwrdvMd6m/j7phqG9/oNeFO9egeTp4DUq7oJ vtWqhVmQ1WqlY68s0/p5CHWgfXswQLRln2VOk4xk2g== X-Google-Smtp-Source: ACHHUZ7uVO5o+NThIdgFsjVHJqpRe4q1kcaQEF27uWALM99Ww5P2tQzdD8U6XM2V84nbMfzSJMZQ0A== X-Received: by 2002:a81:4e83:0:b0:559:e50b:fa80 with SMTP id c125-20020a814e83000000b00559e50bfa80mr29826350ywb.44.1684178598025; Mon, 15 May 2023 12:23:18 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id p64-20020a0dcd43000000b00559f1cb8444sm20879ywd.70.2023.05.15.12.23.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 May 2023 12:23:17 -0700 (PDT) Date: Mon, 15 May 2023 15:23:16 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Chris Torek , Derrick Stolee , Jeff King , Junio C Hamano , Patrick Steinhardt Subject: [PATCH v2 03/16] ref-filter: clear reachable list pointers after freeing Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Jeff King In reach_filter(), we pop all commits from the reachable lists, leaving them empty. But because we're operating on a list pointer that was passed by value, the original filter.reachable_from pointer is left dangling. Signed-off-by: Jeff King Signed-off-by: Taylor Blau --- ref-filter.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/ref-filter.c b/ref-filter.c index 4991cd4f7a..048d277cbf 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -2418,13 +2418,13 @@ void ref_array_clear(struct ref_array *array) #define EXCLUDE_REACHED 0 #define INCLUDE_REACHED 1 static void reach_filter(struct ref_array *array, - struct commit_list *check_reachable, + struct commit_list **check_reachable, int include_reached) { int i, old_nr; struct commit **to_clear; - if (!check_reachable) + if (!*check_reachable) return; CALLOC_ARRAY(to_clear, array->nr); @@ -2434,7 +2434,7 @@ static void reach_filter(struct ref_array *array, } tips_reachable_from_bases(the_repository, - check_reachable, + *check_reachable, to_clear, array->nr, UNINTERESTING); @@ -2455,8 +2455,8 @@ static void reach_filter(struct ref_array *array, clear_commit_marks_many(old_nr, to_clear, ALL_REV_FLAGS); - while (check_reachable) { - struct commit *merge_commit = pop_commit(&check_reachable); + while (*check_reachable) { + struct commit *merge_commit = pop_commit(check_reachable); clear_commit_marks(merge_commit, ALL_REV_FLAGS); } @@ -2553,8 +2553,8 @@ int filter_refs(struct ref_array *array, struct ref_filter *filter, unsigned int clear_contains_cache(&ref_cbdata.no_contains_cache); /* Filters that need revision walking */ - reach_filter(array, filter->reachable_from, INCLUDE_REACHED); - reach_filter(array, filter->unreachable_from, EXCLUDE_REACHED); + reach_filter(array, &filter->reachable_from, INCLUDE_REACHED); + reach_filter(array, &filter->unreachable_from, EXCLUDE_REACHED); save_commit_buffer = save_commit_buffer_orig; return ret; From patchwork Mon May 15 19:23:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13242102 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 73AC7C77B7D for ; Mon, 15 May 2023 19:23:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245083AbjEOTXd (ORCPT ); Mon, 15 May 2023 15:23:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39102 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245015AbjEOTX2 (ORCPT ); Mon, 15 May 2023 15:23:28 -0400 Received: from mail-yb1-xb2d.google.com (mail-yb1-xb2d.google.com [IPv6:2607:f8b0:4864:20::b2d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EE77914E70 for ; Mon, 15 May 2023 12:23:21 -0700 (PDT) Received: by mail-yb1-xb2d.google.com with SMTP id 3f1490d57ef6-ba7ed900ac4so922551276.0 for ; Mon, 15 May 2023 12:23:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1684178601; x=1686770601; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=yDzw/Zl0GNt1WZwh5Kd8MvJSroAqJ6M4H+sxaNmDvxg=; b=A4PwbgbYSdhJi+gLZalKkInJHkfpWZJrRT1iYbcHbCmm0Abh1VXQx8RoiMGdbl1ZRi lCX34PD70T80aOqYnvYOP3dVpDzKi0vZTKHldLpZ61cPAElgX7g99JDI778+Hgj3+EQu 9xD2bGZ7/lWGm6Dy1n2mZRZf6Do0FMUnKoV0dGT8HCQS6zN/2ynHl0n839+kmKeOh2NM WqjhH5s3nY+WIDi+jJmq3djPJSj8EXPQO4e4BOGhE6LMpsb/3kllabfNWp/dojkYBfVX 1W05RfJ1ErYCang3l2zkwiSaKGqie+gbjXrrz9qYtkitapStJpqztT4ZQGYee5G43snD qAjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684178601; x=1686770601; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=yDzw/Zl0GNt1WZwh5Kd8MvJSroAqJ6M4H+sxaNmDvxg=; b=fJjmRRDKxkVmUeiBPbxCJkBWPTGcjSZ1GWTC4BKy7TDlo77L1VaAiQhZRuqGMVsjPB ETA4pN4ibW+hvb2eplRtBK4pPjDC3xur7e3mUzvhtQs6uyx5Ksa2ZM/nFPSW6nwwHmEx hUhwmWWcb00eHD6OfuE/ZpTNJ12Ud95V30DLHNGOL4xMjSmpIQRPapuXLNbzt3ubNvhR 6CDfDaMEyKGiZz1nDnNBRjYwgy72s1iO1sFXm9P/sjjqHow4BLuI9Tjo7at9SeMQrJcs VB8s73GIglVBZgSmQiuMa+W7tojISRnYHGs1+BA9kEMOsGo1FMxhe03yArqDjnI1kKBM Zl7g== X-Gm-Message-State: AC+VfDwZnBDnjZhZLL7KT1L+nBTCE9PAC3BKj8TzuqGO3z+hs5gtM/3U ZuBUs+Gxaic8b0lOjZQ3AEUPcZ91/HC4AQPQwltiEg== X-Google-Smtp-Source: ACHHUZ58DDN9HZMjEDtqCrrGgeU14VP0HeRM75xVzOly/224BQ6F1G6klG379sDM+QDcnEaxBr6swg== X-Received: by 2002:a25:b28f:0:b0:ba6:bc78:a3fd with SMTP id k15-20020a25b28f000000b00ba6bc78a3fdmr11155503ybj.20.1684178600991; Mon, 15 May 2023 12:23:20 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id j10-20020a5b030a000000b00b9d8612a8bbsm27428ybp.16.2023.05.15.12.23.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 May 2023 12:23:20 -0700 (PDT) Date: Mon, 15 May 2023 15:23:19 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Chris Torek , Derrick Stolee , Jeff King , Junio C Hamano , Patrick Steinhardt Subject: [PATCH v2 04/16] ref-filter: add `ref_filter_clear()` Message-ID: <85ecb70957d27911adc6d7a36dd08ba435684528.1684178576.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Jeff King We did not bother to clean up at all in `git branch` or `git tag`, and `git for-each-ref` only cleans up a couple of members. Add and call `ref_filter_clear()` when cleaning up a `struct ref_filter`. Running this patch (without any test changes) indicates a couple of now leak-free tests. This was found by running: $ make SANITIZE=leak $ make -C t GIT_TEST_PASSING_SANITIZE_LEAK=check GIT_TEST_OPTS=--immediate (Note that the `reachable_from` and `unreachable_from` lists should be cleaned as they are used. So this is just covering any case where we might bail before running the reachability check.) Signed-off-by: Jeff King Signed-off-by: Taylor Blau --- builtin/branch.c | 1 + builtin/for-each-ref.c | 3 +-- builtin/tag.c | 1 + ref-filter.c | 16 ++++++++++++++++ ref-filter.h | 3 +++ t/t0041-usage.sh | 1 + t/t3402-rebase-merge.sh | 1 + 7 files changed, 24 insertions(+), 2 deletions(-) diff --git a/builtin/branch.c b/builtin/branch.c index 03bb8e414c..c201f0cb0b 100644 --- a/builtin/branch.c +++ b/builtin/branch.c @@ -813,6 +813,7 @@ int cmd_branch(int argc, const char **argv, const char *prefix) print_columns(&output, colopts, NULL); string_list_clear(&output, 0); ref_sorting_release(sorting); + ref_filter_clear(&filter); return 0; } else if (edit_description) { const char *branch_name; diff --git a/builtin/for-each-ref.c b/builtin/for-each-ref.c index 99ccb73518..c01fa6fefe 100644 --- a/builtin/for-each-ref.c +++ b/builtin/for-each-ref.c @@ -120,8 +120,7 @@ int cmd_for_each_ref(int argc, const char **argv, const char *prefix) strbuf_release(&err); strbuf_release(&output); ref_array_clear(&array); - free_commit_list(filter.with_commit); - free_commit_list(filter.no_commit); + ref_filter_clear(&filter); ref_sorting_release(sorting); strvec_clear(&vec); return 0; diff --git a/builtin/tag.c b/builtin/tag.c index 6b41bb7374..aab5e693fe 100644 --- a/builtin/tag.c +++ b/builtin/tag.c @@ -645,6 +645,7 @@ int cmd_tag(int argc, const char **argv, const char *prefix) cleanup: ref_sorting_release(sorting); + ref_filter_clear(&filter); strbuf_release(&buf); strbuf_release(&ref); strbuf_release(&reflog_msg); diff --git a/ref-filter.c b/ref-filter.c index 048d277cbf..d32f426898 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -2866,3 +2866,19 @@ int parse_opt_merge_filter(const struct option *opt, const char *arg, int unset) return 0; } + +void ref_filter_init(struct ref_filter *filter) +{ + struct ref_filter blank = REF_FILTER_INIT; + memcpy(filter, &blank, sizeof(blank)); +} + +void ref_filter_clear(struct ref_filter *filter) +{ + oid_array_clear(&filter->points_at); + free_commit_list(filter->with_commit); + free_commit_list(filter->no_commit); + free_commit_list(filter->reachable_from); + free_commit_list(filter->unreachable_from); + ref_filter_init(filter); +} diff --git a/ref-filter.h b/ref-filter.h index a920f73b29..160b807224 100644 --- a/ref-filter.h +++ b/ref-filter.h @@ -170,4 +170,7 @@ void filter_ahead_behind(struct repository *r, struct ref_format *format, struct ref_array *array); +void ref_filter_init(struct ref_filter *filter); +void ref_filter_clear(struct ref_filter *filter); + #endif /* REF_FILTER_H */ diff --git a/t/t0041-usage.sh b/t/t0041-usage.sh index c4fc34eb18..9ea974b0c6 100755 --- a/t/t0041-usage.sh +++ b/t/t0041-usage.sh @@ -5,6 +5,7 @@ test_description='Test commands behavior when given invalid argument value' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh test_expect_success 'setup ' ' diff --git a/t/t3402-rebase-merge.sh b/t/t3402-rebase-merge.sh index 79b0640c00..e9e03ca4b5 100755 --- a/t/t3402-rebase-merge.sh +++ b/t/t3402-rebase-merge.sh @@ -8,6 +8,7 @@ test_description='git rebase --merge test' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh T="A quick brown fox From patchwork Mon May 15 19:23:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13242103 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B0E7AC7EE25 for ; Mon, 15 May 2023 19:23:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244979AbjEOTXp (ORCPT ); Mon, 15 May 2023 15:23:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39102 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245062AbjEOTXa (ORCPT ); Mon, 15 May 2023 15:23:30 -0400 Received: from mail-yb1-xb35.google.com (mail-yb1-xb35.google.com [IPv6:2607:f8b0:4864:20::b35]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C09341155E for ; Mon, 15 May 2023 12:23:24 -0700 (PDT) Received: by mail-yb1-xb35.google.com with SMTP id 3f1490d57ef6-ba76528fe31so57816276.1 for ; Mon, 15 May 2023 12:23:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1684178604; x=1686770604; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=p0VwrK4QtQjt4LdWZCL8Ln21wC0ftrl3HZBTsJM5ENs=; b=c9kFsVuqM8TIaRBIyqfal0Sz8Rk9HAmiE1GJqSnDMGVVhGna8Twn0/nx4lZ4juWw06 4cWHvEODjNzGd/Pom8iAWc6VvX4n9tB8hlOcOcTri4HOCaM64DUGhUFFAkeh73cxoo6r fsjpuHicXVRF485eM1ZnaletCzBodGiQsdFbCwtkwtpd9R/j1+whe72PMDXAIdDALHPV gmuawFbx7Rb/ViyX0crazH7vpo7MHiNqwoEmFLmLdpY7TqJsaLeazfPd4rxAVvLJJ71Y W2cSZOsil2nBPd41n6Q0RPNJ09Ewt4JxuqsVtQO4T4Lr6yivCCBzREompyc1/e4Iy+/i H6kA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684178604; x=1686770604; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=p0VwrK4QtQjt4LdWZCL8Ln21wC0ftrl3HZBTsJM5ENs=; b=c8z3BEXXCwdH0GC13dPjhunapk/HzkmXzW/uy37/TNOLyLByQAs91+tKilqoIFwqET eeef10D5iJWh/XlP4O6yI84HpbyukY1wCbFAPIlECuKk2d3ertzcPeIen7pOEQ9u1gQ/ uzf7Do9F5XN/xm6an/REh8cqbmr3MC8SoVsPiF5IceuOi9LjBh72WcR8ySVO/Ub3zyl3 Bz5uo5bRUjV0oy2MdcB9miY4RxRW/4WoI8f9GO7q/RncYE8S9vyi5MqQDotAgBcqYY9X 0jeaMFvk+28PtVPw+A17NGJbBqXgZw+tRQBowiECMv/GOc0iqrBTDWiMWmfbs/0DfAfO Ob4Q== X-Gm-Message-State: AC+VfDzTWCNPlPpFY2nvCLtPoZKLygTXOBpjhO38zJMmFDyTEL10ozJz Q+CO0aYbgiqsgKyQcbErlP0gmTf7DNruBe+afoLfow== X-Google-Smtp-Source: ACHHUZ5JZwDOyJVg/iiHVcAUrU7B1av/hF6wGu1jtnc2YZ8Niyb6S9erUgyPlBbawhjJBOvEISALzQ== X-Received: by 2002:a81:bd7:0:b0:561:1127:30e0 with SMTP id 206-20020a810bd7000000b00561112730e0mr10073257ywl.3.1684178603785; Mon, 15 May 2023 12:23:23 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id i2-20020a816d02000000b0055a3420b25asm24522ywc.50.2023.05.15.12.23.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 May 2023 12:23:23 -0700 (PDT) Date: Mon, 15 May 2023 15:23:22 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Chris Torek , Derrick Stolee , Jeff King , Junio C Hamano , Patrick Steinhardt Subject: [PATCH v2 05/16] ref-filter.c: parameterize match functions over patterns Message-ID: <385890b45960556cdffe5c09211baf4f8d3df0ad.1684178576.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Jeff King `match_pattern()` and `match_name_as_path()` both take a `struct ref_filter *`, and then store a stack variable `patterns` pointing at `filter->patterns`. The subsequent patch will add a new array of patterns to match over (the excluded patterns, via a new `git for-each-ref --exclude` option), treating the return value of these functions differently depending on which patterns are being used to match. Tweak `match_pattern()` and `match_name_as_path()` to take an array of patterns to prepare for passing either in. Once we start passing either in, `match_pattern()` will have little to do with a particular `struct ref_filter *` instance. To clarify this, drop it from the argument list, and replace it with the only bit of the `ref_filter` that we care about (`filter->ignore_case`). Co-authored-by: Taylor Blau Signed-off-by: Jeff King Signed-off-by: Taylor Blau --- ref-filter.c | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/ref-filter.c b/ref-filter.c index d32f426898..6d91c7cb0d 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -2104,12 +2104,12 @@ static int get_ref_atom_value(struct ref_array_item *ref, int atom, * matches a pattern "refs/heads/mas") or a wildcard (e.g. the same ref * matches "refs/heads/mas*", too). */ -static int match_pattern(const struct ref_filter *filter, const char *refname) +static int match_pattern(const char **patterns, const char *refname, + const int ignore_case) { - const char **patterns = filter->name_patterns; unsigned flags = 0; - if (filter->ignore_case) + if (ignore_case) flags |= WM_CASEFOLD; /* @@ -2134,9 +2134,10 @@ static int match_pattern(const struct ref_filter *filter, const char *refname) * matches a pattern "refs/heads/" but not "refs/heads/m") or a * wildcard (e.g. the same ref matches "refs/heads/m*", too). */ -static int match_name_as_path(const struct ref_filter *filter, const char *refname) +static int match_name_as_path(const struct ref_filter *filter, + const char **pattern, + const char *refname) { - const char **pattern = filter->name_patterns; int namelen = strlen(refname); unsigned flags = WM_PATHNAME; @@ -2165,8 +2166,9 @@ static int filter_pattern_match(struct ref_filter *filter, const char *refname) if (!*filter->name_patterns) return 1; /* No pattern always matches */ if (filter->match_as_path) - return match_name_as_path(filter, refname); - return match_pattern(filter, refname); + return match_name_as_path(filter, filter->name_patterns, refname); + return match_pattern(filter->name_patterns, refname, + filter->ignore_case); } /* From patchwork Mon May 15 19:23:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13242104 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 759BEC7EE24 for ; Mon, 15 May 2023 19:23:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245015AbjEOTXx (ORCPT ); Mon, 15 May 2023 15:23:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39114 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245080AbjEOTXb (ORCPT ); Mon, 15 May 2023 15:23:31 -0400 Received: from mail-yw1-x1132.google.com (mail-yw1-x1132.google.com [IPv6:2607:f8b0:4864:20::1132]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B382312EA2 for ; Mon, 15 May 2023 12:23:27 -0700 (PDT) Received: by mail-yw1-x1132.google.com with SMTP id 00721157ae682-55af4277904so194846617b3.1 for ; Mon, 15 May 2023 12:23:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1684178607; x=1686770607; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=+dWxzIPgbFoHSCOULBFXWPtGTeSCjHYBNL5SJqJ9fNM=; b=UsgYkkc2R7merPzOOohXL37F6lsa5boBZZDeWIEirZuw2eZtkc71KIfH3v7Bt7S7Sd dk3PoB9bytFOmQLOvWz8iVwyd2v3f2f1b+7dFfbPd3YA7XDftAtSbqeCW97AY50WL9Xx 7cPE7M/SzCy2mScmJRL6D45mXxdUg7/gCaVUUHqKYyTwrUGOobaUlTrrQDpYyqT/r1kO HG4op8M1Glb9hxTEritNAf4Uxg8JTu2MqGRv6ZwqqAtl617ixHwSIf9HTBZVe+vQOd7d c7SGbAgMNZsYs2RPMxlVDzz2iKN2RVTjpA11ZHsHd4AUk8lWNCuwdSt44ZWP7UNCCy66 AZHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684178607; x=1686770607; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=+dWxzIPgbFoHSCOULBFXWPtGTeSCjHYBNL5SJqJ9fNM=; b=OfOZOfPaqSBL/oP6wjq6YRbmiyV9WJDpun22KQ2X0MjUA31UD1ovBvrVSlTGOgzGwI O+pdV45L3987SoPTjEYcpfhe1YAUmEyv5+C7XZ5ANSNI9V4NZpaQPgqu+N0GT5piiDXM NHPREbtncmsmU88XzdZgAnq2c+p7iicAQ8iTPbLv5rCok7KVlb3QQqe95qA3eFSax/s+ K4CfFH/bJBKr5hNGrhi0mPg1a5dsIW96a0O/+87gycFnxkNGP5DpmF6OspsaC7KwT8Ag qklnRZXtzTart0O/G4OsKkK1g4f4Cw9JbrkiL++ZY8Hq+driDsQlGoJaMElGQIkT8enC E7VA== X-Gm-Message-State: AC+VfDygks2Ck03s64FL3cFogMNbmACubcK5TH+x0OX82VEZN1R6STjm wSlirKUN9vEsCayC9N3y846RC+o19KiqjN/0hAjPCA== X-Google-Smtp-Source: ACHHUZ4VJOCYjdSK5x78/b7wrNBVjSNxulSYR4ephDEiELWQLMQRENE69W0xZndJwaLvMO7g1xBBOQ== X-Received: by 2002:a0d:db47:0:b0:55a:1cb0:b255 with SMTP id d68-20020a0ddb47000000b0055a1cb0b255mr31938939ywe.37.1684178606756; Mon, 15 May 2023 12:23:26 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id v125-20020a816183000000b0055a72f6a462sm32231ywb.19.2023.05.15.12.23.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 May 2023 12:23:26 -0700 (PDT) Date: Mon, 15 May 2023 15:23:25 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Chris Torek , Derrick Stolee , Jeff King , Junio C Hamano , Patrick Steinhardt Subject: [PATCH v2 06/16] builtin/for-each-ref.c: add `--exclude` option Message-ID: <1a3371a0a7511a7b47cffdf7094c51b027dd005b.1684178576.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When using `for-each-ref`, it is sometimes convenient for the caller to be able to exclude certain parts of the references. For example, if there are many `refs/__hidden__/*` references, the caller may want to emit all references *except* the hidden ones. Currently, the only way to do this is to post-process the output, like: $ git for-each-ref --format='%(refname)' | grep -v '^refs/hidden/' Which is do-able, but requires processing a potentially large quantity of references. Teach `git for-each-ref` a new `--exclude=` option, which excludes references from the results if they match one or more excluded patterns. This patch provides a naive implementation where the `ref_filter` still sees all references (including ones that it will discard) and is left to check whether each reference matches any excluded pattern(s) before emitting them. By culling out references we know the caller doesn't care about, we can avoid allocating memory for their storage, as well as spending time sorting the output (among other things). Even the naive implementation provides a significant speed-up on a modified copy of linux.git (that has a hidden ref pointing at each commit): $ hyperfine \ 'git.compile for-each-ref --format="%(objectname) %(refname)" | grep -vE "[0-9a-f]{40} refs/pull/"' \ 'git.compile for-each-ref --format="%(objectname) %(refname)" --exclude refs/pull/' Benchmark 1: git.compile for-each-ref --format="%(objectname) %(refname)" | grep -vE "[0-9a-f]{40} refs/pull/" Time (mean ± σ): 820.1 ms ± 2.0 ms [User: 703.7 ms, System: 152.0 ms] Range (min … max): 817.7 ms … 823.3 ms 10 runs Benchmark 2: git.compile for-each-ref --format="%(objectname) %(refname)" --exclude refs/pull/ Time (mean ± σ): 106.6 ms ± 1.1 ms [User: 99.4 ms, System: 7.1 ms] Range (min … max): 104.7 ms … 109.1 ms 27 runs Summary 'git.compile for-each-ref --format="%(objectname) %(refname)" --exclude refs/pull/' ran 7.69 ± 0.08 times faster than 'git.compile for-each-ref --format="%(objectname) %(refname)" | grep -vE "[0-9a-f]{40} refs/pull/"' Subsequent patches will improve on this by avoiding visiting excluded sections of the `packed-refs` file in certain cases. Co-authored-by: Jeff King Signed-off-by: Jeff King Signed-off-by: Taylor Blau --- Documentation/git-for-each-ref.txt | 6 +++++ builtin/for-each-ref.c | 1 + ref-filter.c | 13 +++++++++++ ref-filter.h | 6 +++++ t/t6300-for-each-ref.sh | 35 ++++++++++++++++++++++++++++++ 5 files changed, 61 insertions(+) diff --git a/Documentation/git-for-each-ref.txt b/Documentation/git-for-each-ref.txt index 1e215d4e73..5743eb5def 100644 --- a/Documentation/git-for-each-ref.txt +++ b/Documentation/git-for-each-ref.txt @@ -14,6 +14,7 @@ SYNOPSIS [--points-at=] [--merged[=]] [--no-merged[=]] [--contains[=]] [--no-contains[=]] + [--exclude= ...] DESCRIPTION ----------- @@ -102,6 +103,11 @@ OPTIONS Do not print a newline after formatted refs where the format expands to the empty string. +--exclude=:: + If one or more patterns are given, only refs which do not match + any excluded pattern(s) are shown. Matching is done using the + same rules as `` above. + FIELD NAMES ----------- diff --git a/builtin/for-each-ref.c b/builtin/for-each-ref.c index c01fa6fefe..3384987428 100644 --- a/builtin/for-each-ref.c +++ b/builtin/for-each-ref.c @@ -47,6 +47,7 @@ int cmd_for_each_ref(int argc, const char **argv, const char *prefix) OPT_INTEGER( 0 , "count", &maxcount, N_("show only matched refs")), OPT_STRING( 0 , "format", &format.format, N_("format"), N_("format to use for the output")), OPT__COLOR(&format.use_color, N_("respect format colors")), + OPT_REF_FILTER_EXCLUDE(&filter), OPT_REF_SORT(&sorting_options), OPT_CALLBACK(0, "points-at", &filter.points_at, N_("object"), N_("print only refs which points at the given object"), diff --git a/ref-filter.c b/ref-filter.c index 6d91c7cb0d..d44418efb7 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -2171,6 +2171,15 @@ static int filter_pattern_match(struct ref_filter *filter, const char *refname) filter->ignore_case); } +static int filter_exclude_match(struct ref_filter *filter, const char *refname) +{ + if (!filter->exclude.nr) + return 0; + if (filter->match_as_path) + return match_name_as_path(filter, filter->exclude.v, refname); + return match_pattern(filter->exclude.v, refname, filter->ignore_case); +} + /* * This is the same as for_each_fullref_in(), but it tries to iterate * only over the patterns we'll care about. Note that it _doesn't_ do a full @@ -2338,6 +2347,9 @@ static int ref_filter_handler(const char *refname, const struct object_id *oid, if (!filter_pattern_match(filter, refname)) return 0; + if (filter_exclude_match(filter, refname)) + return 0; + if (filter->points_at.nr && !match_points_at(&filter->points_at, oid, refname)) return 0; @@ -2877,6 +2889,7 @@ void ref_filter_init(struct ref_filter *filter) void ref_filter_clear(struct ref_filter *filter) { + strvec_clear(&filter->exclude); oid_array_clear(&filter->points_at); free_commit_list(filter->with_commit); free_commit_list(filter->no_commit); diff --git a/ref-filter.h b/ref-filter.h index 160b807224..1524bc463a 100644 --- a/ref-filter.h +++ b/ref-filter.h @@ -6,6 +6,7 @@ #include "refs.h" #include "commit.h" #include "string-list.h" +#include "strvec.h" /* Quoting styles */ #define QUOTE_NONE 0 @@ -59,6 +60,7 @@ struct ref_array { struct ref_filter { const char **name_patterns; + struct strvec exclude; struct oid_array points_at; struct commit_list *with_commit; struct commit_list *no_commit; @@ -94,6 +96,7 @@ struct ref_format { #define REF_FILTER_INIT { \ .points_at = OID_ARRAY_INIT, \ + .exclude = STRVEC_INIT, \ } #define REF_FORMAT_INIT { \ .use_color = -1, \ @@ -112,6 +115,9 @@ struct ref_format { #define OPT_REF_SORT(var) \ OPT_STRING_LIST(0, "sort", (var), \ N_("key"), N_("field name to sort on")) +#define OPT_REF_FILTER_EXCLUDE(var) \ + OPT_STRVEC(0, "exclude", &(var)->exclude, \ + N_("pattern"), N_("exclude refs which match pattern")) /* * API for filtering a set of refs. Based on the type of refs the user diff --git a/t/t6300-for-each-ref.sh b/t/t6300-for-each-ref.sh index 5c00607608..7e8d578522 100755 --- a/t/t6300-for-each-ref.sh +++ b/t/t6300-for-each-ref.sh @@ -447,6 +447,41 @@ test_expect_success 'exercise glob patterns with prefixes' ' test_cmp expected actual ' +cat >expected <<\EOF +refs/tags/bar +refs/tags/baz +refs/tags/testtag +EOF + +test_expect_success 'exercise patterns with prefix exclusions' ' + for tag in foo/one foo/two foo/three bar baz + do + git tag "$tag" || return 1 + done && + test_when_finished "git tag -d foo/one foo/two foo/three bar baz" && + git for-each-ref --format="%(refname)" \ + refs/tags/ --exclude=refs/tags/foo >actual && + test_cmp expected actual +' + +cat >expected <<\EOF +refs/tags/bar +refs/tags/baz +refs/tags/foo/one +refs/tags/testtag +EOF + +test_expect_success 'exercise patterns with pattern exclusions' ' + for tag in foo/one foo/two foo/three bar baz + do + git tag "$tag" || return 1 + done && + test_when_finished "git tag -d foo/one foo/two foo/three bar baz" && + git for-each-ref --format="%(refname)" \ + refs/tags/ --exclude="refs/tags/foo/t*" >actual && + test_cmp expected actual +' + cat >expected <<\EOF 'refs/heads/main' 'refs/remotes/origin/main' From patchwork Mon May 15 19:23:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13242105 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4D993C7EE24 for ; Mon, 15 May 2023 19:24:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245172AbjEOTYI (ORCPT ); Mon, 15 May 2023 15:24:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39122 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245090AbjEOTXk (ORCPT ); Mon, 15 May 2023 15:23:40 -0400 Received: from mail-yw1-x1132.google.com (mail-yw1-x1132.google.com [IPv6:2607:f8b0:4864:20::1132]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8B54A13C13 for ; Mon, 15 May 2023 12:23:30 -0700 (PDT) Received: by mail-yw1-x1132.google.com with SMTP id 00721157ae682-55a00da4e53so229279327b3.0 for ; Mon, 15 May 2023 12:23:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1684178609; x=1686770609; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=wS48wvbbRfXB939krCs+yQJyZfXgo+SlEjUh5RmMaz0=; b=fIDO6DZTwCt4s9sXJAGKjT61uMj8SOop82I8ukpk7a0fgQ7Cs6oX9fVS/QQ9htWJmH oTg+mDRHPLkrfwF04L5ETLn2ihDhW7NPXIJCdElyLiRQB6kZajVly2l3ahdIIEQiL4mE PWOq/j7P6sYMB/IIoSxYaMv/Zaami2PBa2z7pWnoCmBgA6vU1/m4y7QwfJnZ1Wi0rYFU CWKHshDCaW+ZHBA6bUUL4+2HmFKfth00WfM8CP4kEPVuZsGzkSaNWG8LyEa4XLOeAhJx rDy4uMpFYfWmU/DXrYYQpch2AHYbtqV//v7nmdYdmJ6tPbfj9KcZOMqxqakG8iPzX91R fBoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684178609; x=1686770609; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=wS48wvbbRfXB939krCs+yQJyZfXgo+SlEjUh5RmMaz0=; b=Cx+4whG8qxN5HocFW98RqQ/4Igh9AEmXS4eYLFwJSdPjnNMygc7TnKQxTwuF5Lm2sS LrZ4gvgzIk8b4FttIgl26BZampCza4+7E2/WvkoKssoocD/17AQHP2ffTHUozUANlx1n c/QiJwNpOB7A0W3lsUJ0pv8NS6hvp1khXy1/G1nFFhyj6t+CoI4RejwdkYFCNfS+woKz VhOAgGBLNdK1AK9HUCREClvQaNkTaREzqapIAU1idRB4shwDyx5MFPsPV/t+c8K1ZGjA py9EnpqI2MLsJtwgVe3fS24ADTDu8dOYLvW/T2vu6cnGeW6/lPsxwzdu1A3fIabaAnmZ IrTw== X-Gm-Message-State: AC+VfDw+4w625QNnYwH6wIWhqt5qiFfVctXnurQSEHvW5biegdg7nqDV YV5LGHH5Qs5c8ALp3zgTJzZEmpbVsNRJbxrhKIeZ4g== X-Google-Smtp-Source: ACHHUZ4FhCAK3x6WIHoORDiSLQwJ4UxrWIU99hqKayaBVO5cOZKFoWg2OMRR956rwujzP4kk+CvG2Q== X-Received: by 2002:a81:df12:0:b0:561:4bc2:1587 with SMTP id c18-20020a81df12000000b005614bc21587mr6162020ywn.39.1684178609555; Mon, 15 May 2023 12:23:29 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id i22-20020a0ddf16000000b0055aad7d3f34sm4600ywe.142.2023.05.15.12.23.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 May 2023 12:23:29 -0700 (PDT) Date: Mon, 15 May 2023 15:23:28 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Chris Torek , Derrick Stolee , Jeff King , Junio C Hamano , Patrick Steinhardt Subject: [PATCH v2 07/16] refs: plumb `exclude_patterns` argument throughout Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org The subsequent patch will want to access an optional `excluded_patterns` array within `refs/packed-backend.c` that will cull out certain references matching any of the given patterns on a best-effort basis. To do so, the refs subsystem needs to be updated to pass this value across a number of different locations. Prepare for a future patch by introducing this plumbing now, passing NULLs at top-level APIs in order to make that patch less noisy and more easily readable. Signed-off-by: Taylor Blau --- ls-refs.c | 2 +- ref-filter.c | 5 +++-- refs.c | 32 +++++++++++++++++++------------- refs.h | 8 +++++++- refs/debug.c | 5 +++-- refs/files-backend.c | 5 +++-- refs/packed-backend.c | 5 +++-- refs/refs-internal.h | 7 ++++--- revision.c | 2 +- 9 files changed, 44 insertions(+), 27 deletions(-) diff --git a/ls-refs.c b/ls-refs.c index f385938b64..6f490b2d9c 100644 --- a/ls-refs.c +++ b/ls-refs.c @@ -193,7 +193,7 @@ int ls_refs(struct repository *r, struct packet_reader *request) strvec_push(&data.prefixes, ""); refs_for_each_fullref_in_prefixes(get_main_ref_store(r), get_git_namespace(), data.prefixes.v, - send_ref, &data); + NULL, send_ref, &data); packet_fflush(stdout); strvec_clear(&data.prefixes); strbuf_release(&data.buf); diff --git a/ref-filter.c b/ref-filter.c index d44418efb7..717c3c4bcf 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -2209,12 +2209,13 @@ static int for_each_fullref_in_pattern(struct ref_filter *filter, if (!filter->name_patterns[0]) { /* no patterns; we have to look at everything */ - return for_each_fullref_in("", cb, cb_data); + return refs_for_each_fullref_in(get_main_ref_store(the_repository), + "", NULL, cb, cb_data); } return refs_for_each_fullref_in_prefixes(get_main_ref_store(the_repository), NULL, filter->name_patterns, - cb, cb_data); + NULL, cb, cb_data); } /* diff --git a/refs.c b/refs.c index b9b77d2eff..538bde644e 100644 --- a/refs.c +++ b/refs.c @@ -1526,7 +1526,9 @@ int head_ref(each_ref_fn fn, void *cb_data) struct ref_iterator *refs_ref_iterator_begin( struct ref_store *refs, - const char *prefix, int trim, + const char *prefix, + const char **exclude_patterns, + int trim, enum do_for_each_ref_flags flags) { struct ref_iterator *iter; @@ -1542,8 +1544,7 @@ struct ref_iterator *refs_ref_iterator_begin( } } - iter = refs->be->iterator_begin(refs, prefix, flags); - + iter = refs->be->iterator_begin(refs, prefix, exclude_patterns, flags); /* * `iterator_begin()` already takes care of prefix, but we * might need to do some trimming: @@ -1577,7 +1578,7 @@ static int do_for_each_repo_ref(struct repository *r, const char *prefix, if (!refs) return 0; - iter = refs_ref_iterator_begin(refs, prefix, trim, flags); + iter = refs_ref_iterator_begin(refs, prefix, NULL, trim, flags); return do_for_each_repo_ref_iterator(r, iter, fn, cb_data); } @@ -1599,6 +1600,7 @@ static int do_for_each_ref_helper(struct repository *r, } static int do_for_each_ref(struct ref_store *refs, const char *prefix, + const char **exclude_patterns, each_ref_fn fn, int trim, enum do_for_each_ref_flags flags, void *cb_data) { @@ -1608,7 +1610,8 @@ static int do_for_each_ref(struct ref_store *refs, const char *prefix, if (!refs) return 0; - iter = refs_ref_iterator_begin(refs, prefix, trim, flags); + iter = refs_ref_iterator_begin(refs, prefix, exclude_patterns, trim, + flags); return do_for_each_repo_ref_iterator(the_repository, iter, do_for_each_ref_helper, &hp); @@ -1616,7 +1619,7 @@ static int do_for_each_ref(struct ref_store *refs, const char *prefix, int refs_for_each_ref(struct ref_store *refs, each_ref_fn fn, void *cb_data) { - return do_for_each_ref(refs, "", fn, 0, 0, cb_data); + return do_for_each_ref(refs, "", NULL, fn, 0, 0, cb_data); } int for_each_ref(each_ref_fn fn, void *cb_data) @@ -1627,7 +1630,7 @@ int for_each_ref(each_ref_fn fn, void *cb_data) int refs_for_each_ref_in(struct ref_store *refs, const char *prefix, each_ref_fn fn, void *cb_data) { - return do_for_each_ref(refs, prefix, fn, strlen(prefix), 0, cb_data); + return do_for_each_ref(refs, prefix, NULL, fn, strlen(prefix), 0, cb_data); } int for_each_ref_in(const char *prefix, each_ref_fn fn, void *cb_data) @@ -1638,13 +1641,14 @@ int for_each_ref_in(const char *prefix, each_ref_fn fn, void *cb_data) int for_each_fullref_in(const char *prefix, each_ref_fn fn, void *cb_data) { return do_for_each_ref(get_main_ref_store(the_repository), - prefix, fn, 0, 0, cb_data); + prefix, NULL, fn, 0, 0, cb_data); } int refs_for_each_fullref_in(struct ref_store *refs, const char *prefix, + const char **exclude_patterns, each_ref_fn fn, void *cb_data) { - return do_for_each_ref(refs, prefix, fn, 0, 0, cb_data); + return do_for_each_ref(refs, prefix, exclude_patterns, fn, 0, 0, cb_data); } int for_each_replace_ref(struct repository *r, each_repo_ref_fn fn, void *cb_data) @@ -1661,14 +1665,14 @@ int for_each_namespaced_ref(each_ref_fn fn, void *cb_data) int ret; strbuf_addf(&buf, "%srefs/", get_git_namespace()); ret = do_for_each_ref(get_main_ref_store(the_repository), - buf.buf, fn, 0, 0, cb_data); + buf.buf, NULL, fn, 0, 0, cb_data); strbuf_release(&buf); return ret; } int refs_for_each_rawref(struct ref_store *refs, each_ref_fn fn, void *cb_data) { - return do_for_each_ref(refs, "", fn, 0, + return do_for_each_ref(refs, "", NULL, fn, 0, DO_FOR_EACH_INCLUDE_BROKEN, cb_data); } @@ -1738,6 +1742,7 @@ static void find_longest_prefixes(struct string_list *out, int refs_for_each_fullref_in_prefixes(struct ref_store *ref_store, const char *namespace, const char **patterns, + const char **exclude_patterns, each_ref_fn fn, void *cb_data) { struct string_list prefixes = STRING_LIST_INIT_DUP; @@ -1753,7 +1758,8 @@ int refs_for_each_fullref_in_prefixes(struct ref_store *ref_store, for_each_string_list_item(prefix, &prefixes) { strbuf_addstr(&buf, prefix->string); - ret = refs_for_each_fullref_in(ref_store, buf.buf, fn, cb_data); + ret = refs_for_each_fullref_in(ref_store, buf.buf, + exclude_patterns, fn, cb_data); if (ret) break; strbuf_setlen(&buf, namespace_len); @@ -2408,7 +2414,7 @@ int refs_verify_refname_available(struct ref_store *refs, strbuf_addstr(&dirname, refname + dirname.len); strbuf_addch(&dirname, '/'); - iter = refs_ref_iterator_begin(refs, dirname.buf, 0, + iter = refs_ref_iterator_begin(refs, dirname.buf, NULL, 0, DO_FOR_EACH_INCLUDE_BROKEN); while ((ok = ref_iterator_advance(iter)) == ITER_OK) { if (skip && diff --git a/refs.h b/refs.h index 123cfa4424..d672d636cf 100644 --- a/refs.h +++ b/refs.h @@ -338,6 +338,7 @@ int for_each_ref(each_ref_fn fn, void *cb_data); int for_each_ref_in(const char *prefix, each_ref_fn fn, void *cb_data); int refs_for_each_fullref_in(struct ref_store *refs, const char *prefix, + const char **exclude_patterns, each_ref_fn fn, void *cb_data); int for_each_fullref_in(const char *prefix, each_ref_fn fn, void *cb_data); @@ -345,10 +346,15 @@ int for_each_fullref_in(const char *prefix, each_ref_fn fn, void *cb_data); * iterate all refs in "patterns" by partitioning patterns into disjoint sets * and iterating the longest-common prefix of each set. * + * references matching any pattern in "exclude_patterns" are omitted from the + * result set on a best-effort basis. + * * callers should be prepared to ignore references that they did not ask for. */ int refs_for_each_fullref_in_prefixes(struct ref_store *refs, - const char *namespace, const char **patterns, + const char *namespace, + const char **patterns, + const char **exclude_patterns, each_ref_fn fn, void *cb_data); /** diff --git a/refs/debug.c b/refs/debug.c index 6f11e6de46..328f894177 100644 --- a/refs/debug.c +++ b/refs/debug.c @@ -229,11 +229,12 @@ static struct ref_iterator_vtable debug_ref_iterator_vtable = { static struct ref_iterator * debug_ref_iterator_begin(struct ref_store *ref_store, const char *prefix, - unsigned int flags) + const char **exclude_patterns, unsigned int flags) { struct debug_ref_store *drefs = (struct debug_ref_store *)ref_store; struct ref_iterator *res = - drefs->refs->be->iterator_begin(drefs->refs, prefix, flags); + drefs->refs->be->iterator_begin(drefs->refs, prefix, + exclude_patterns, flags); struct debug_ref_iterator *diter = xcalloc(1, sizeof(*diter)); base_ref_iterator_init(&diter->base, &debug_ref_iterator_vtable, 1); diter->iter = res; diff --git a/refs/files-backend.c b/refs/files-backend.c index bca7b851c5..3bc3c57c05 100644 --- a/refs/files-backend.c +++ b/refs/files-backend.c @@ -829,7 +829,8 @@ static struct ref_iterator_vtable files_ref_iterator_vtable = { static struct ref_iterator *files_ref_iterator_begin( struct ref_store *ref_store, - const char *prefix, unsigned int flags) + const char *prefix, const char **exclude_patterns, + unsigned int flags) { struct files_ref_store *refs; struct ref_iterator *loose_iter, *packed_iter, *overlay_iter; @@ -874,7 +875,7 @@ static struct ref_iterator *files_ref_iterator_begin( * the packed and loose references. */ packed_iter = refs_ref_iterator_begin( - refs->packed_ref_store, prefix, 0, + refs->packed_ref_store, prefix, exclude_patterns, 0, DO_FOR_EACH_INCLUDE_BROKEN); overlay_iter = overlay_ref_iterator_begin(loose_iter, packed_iter); diff --git a/refs/packed-backend.c b/refs/packed-backend.c index 5b412a133b..176bd3905b 100644 --- a/refs/packed-backend.c +++ b/refs/packed-backend.c @@ -924,7 +924,8 @@ static struct ref_iterator_vtable packed_ref_iterator_vtable = { static struct ref_iterator *packed_ref_iterator_begin( struct ref_store *ref_store, - const char *prefix, unsigned int flags) + const char *prefix, const char **exclude_patterns, + unsigned int flags) { struct packed_ref_store *refs; struct snapshot *snapshot; @@ -1149,7 +1150,7 @@ static int write_with_updates(struct packed_ref_store *refs, * list of refs is exhausted, set iter to NULL. When the list * of updates is exhausted, leave i set to updates->nr. */ - iter = packed_ref_iterator_begin(&refs->base, "", + iter = packed_ref_iterator_begin(&refs->base, "", NULL, DO_FOR_EACH_INCLUDE_BROKEN); if ((ok = ref_iterator_advance(iter)) != ITER_OK) iter = NULL; diff --git a/refs/refs-internal.h b/refs/refs-internal.h index a85d113123..28a11b9d61 100644 --- a/refs/refs-internal.h +++ b/refs/refs-internal.h @@ -367,8 +367,8 @@ int is_empty_ref_iterator(struct ref_iterator *ref_iterator); */ struct ref_iterator *refs_ref_iterator_begin( struct ref_store *refs, - const char *prefix, int trim, - enum do_for_each_ref_flags flags); + const char *prefix, const char **exclude_patterns, + int trim, enum do_for_each_ref_flags flags); /* * A callback function used to instruct merge_ref_iterator how to @@ -570,7 +570,8 @@ typedef int copy_ref_fn(struct ref_store *ref_store, */ typedef struct ref_iterator *ref_iterator_begin_fn( struct ref_store *ref_store, - const char *prefix, unsigned int flags); + const char *prefix, const char **exclude_patterns, + unsigned int flags); /* reflog functions */ diff --git a/revision.c b/revision.c index b33cc1d106..89953592f9 100644 --- a/revision.c +++ b/revision.c @@ -2670,7 +2670,7 @@ static int for_each_bisect_ref(struct ref_store *refs, each_ref_fn fn, struct strbuf bisect_refs = STRBUF_INIT; int status; strbuf_addf(&bisect_refs, "refs/bisect/%s", term); - status = refs_for_each_fullref_in(refs, bisect_refs.buf, fn, cb_data); + status = refs_for_each_fullref_in(refs, bisect_refs.buf, NULL, fn, cb_data); strbuf_release(&bisect_refs); return status; } From patchwork Mon May 15 19:23:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13242106 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 74E89C77B7D for ; Mon, 15 May 2023 19:24:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245207AbjEOTYR (ORCPT ); Mon, 15 May 2023 15:24:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39494 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245100AbjEOTXk (ORCPT ); Mon, 15 May 2023 15:23:40 -0400 Received: from mail-yw1-x1133.google.com (mail-yw1-x1133.google.com [IPv6:2607:f8b0:4864:20::1133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4316613282 for ; Mon, 15 May 2023 12:23:33 -0700 (PDT) Received: by mail-yw1-x1133.google.com with SMTP id 00721157ae682-559e53d1195so187972877b3.2 for ; Mon, 15 May 2023 12:23:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1684178612; x=1686770612; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=l2eMPZSz3knvl7ffREsnuCcw1LESbq6TfN+DVqj8seY=; b=wrpBYicBrTMlo5Qu7MBpNVMqH+b4J5BKlJHlK5lnalpVFFtTHxZ2zfSWT3jIt82Nle RYdJEZCgwanjEz7bGDM05vilo2rSM8kzKo5FLsA0bZYQhMBRAYtveq5g3pPObJrOO1r8 eeeulyj0vPSOT8kxXbWZfBaFkvgxtoUFzWQou8OHseZBtUe7bywusYE3l7+rr2R5Sxzi k+D0Qjazs6GYSUt7kp8sNZsgncoGC9tOkouupbaKgkW/0jJoCxLMWcBQ3lcJON2dRaJO 5hvIRs+TFlZG5Q0JoP/jwgG+X2xLyRxBHcmy/nuKw3kg2DKIyB5wEVfQr8PcGYEfaEHW 5MwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684178612; x=1686770612; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=l2eMPZSz3knvl7ffREsnuCcw1LESbq6TfN+DVqj8seY=; b=fM+PGBbhRWt+9wPjf8H+o2ygCSPGfvIELH3zf+Btn6VazWCOPmQgd4KqYiaa0OT2XM Pc+w9DuZaFTIHwdRwf+0ARFnkCaAg3GL0fshWpud+1CVgl6EPS2EAaIDh4nZYeG3oX4N 0FsBFA8Tag021y/vI1NQOL1R23Bc4TFS59eZkZ8TtwiCK6TE4GbrxAUvcrWjHaKmri5+ Csjm16PcCGKBbZK/Tj9vaWTXnodF2z4i9DMmDsgq5N50zCKbAPN2wi3HrEJYlthMyleH NmrX2NGFqGrxxLH32UK8BOh3rA03d7cLtiRM/jnLcWAvPNbSMGqTP5HEMVfL8KMpPKeg 0khQ== X-Gm-Message-State: AC+VfDw1KIxz4THQtJWyg/MoXcyZhlQrnIqBGHzUcwtJO3/jDPL85gbp Nbz1qCtHIHbvuhZbVtfWCeM0qvSzlwtuvIyf6mXTZg== X-Google-Smtp-Source: ACHHUZ6wwTRV/zoWb1pKV6L0uubx867xEkt/N1SE57rbOnM6eKopOKc/9DJaBw67GYOvrBKs2qKYxA== X-Received: by 2002:a81:920c:0:b0:55a:a635:7e1e with SMTP id j12-20020a81920c000000b0055aa6357e1emr36164488ywg.31.1684178612287; Mon, 15 May 2023 12:23:32 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id w191-20020a0dd4c8000000b0054f50f71834sm9506ywd.124.2023.05.15.12.23.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 May 2023 12:23:32 -0700 (PDT) Date: Mon, 15 May 2023 15:23:31 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Chris Torek , Derrick Stolee , Jeff King , Junio C Hamano , Patrick Steinhardt Subject: [PATCH v2 08/16] refs/packed-backend.c: refactor `find_reference_location()` Message-ID: <6002c568b54a3ea2b87f4eed13d342f6e63ec420.1684178576.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org The function `find_reference_location()` is used to perform a binary search-like function over the contents of a repository's `$GIT_DIR/packed-refs` file. The search it implements is unlike a standard binary search in that the records it searches over are not of a fixed width, so the comparison must locate the end of a record before comparing it. Extract the core routine of `find_reference_location()` in order to implement a function in the following patch which will find the first location in the `packed-refs` file that *doesn't* match the given pattern. The behavior of `find_reference_location()` is unchanged. Co-authored-by: Jeff King Signed-off-by: Jeff King Signed-off-by: Taylor Blau --- refs/packed-backend.c | 38 ++++++++++++++++++++++---------------- 1 file changed, 22 insertions(+), 16 deletions(-) diff --git a/refs/packed-backend.c b/refs/packed-backend.c index 176bd3905b..33639f73e1 100644 --- a/refs/packed-backend.c +++ b/refs/packed-backend.c @@ -527,22 +527,8 @@ static int load_contents(struct snapshot *snapshot) return 1; } -/* - * Find the place in `snapshot->buf` where the start of the record for - * `refname` starts. If `mustexist` is true and the reference doesn't - * exist, then return NULL. If `mustexist` is false and the reference - * doesn't exist, then return the point where that reference would be - * inserted, or `snapshot->eof` (which might be NULL) if it would be - * inserted at the end of the file. In the latter mode, `refname` - * doesn't have to be a proper reference name; for example, one could - * search for "refs/replace/" to find the start of any replace - * references. - * - * The record is sought using a binary search, so `snapshot->buf` must - * be sorted. - */ -static const char *find_reference_location(struct snapshot *snapshot, - const char *refname, int mustexist) +static const char *find_reference_location_1(struct snapshot *snapshot, + const char *refname, int mustexist) { /* * This is not *quite* a garden-variety binary search, because @@ -588,6 +574,26 @@ static const char *find_reference_location(struct snapshot *snapshot, return lo; } +/* + * Find the place in `snapshot->buf` where the start of the record for + * `refname` starts. If `mustexist` is true and the reference doesn't + * exist, then return NULL. If `mustexist` is false and the reference + * doesn't exist, then return the point where that reference would be + * inserted, or `snapshot->eof` (which might be NULL) if it would be + * inserted at the end of the file. In the latter mode, `refname` + * doesn't have to be a proper reference name; for example, one could + * search for "refs/replace/" to find the start of any replace + * references. + * + * The record is sought using a binary search, so `snapshot->buf` must + * be sorted. + */ +static const char *find_reference_location(struct snapshot *snapshot, + const char *refname, int mustexist) +{ + return find_reference_location_1(snapshot, refname, mustexist); +} + /* * Create a newly-allocated `snapshot` of the `packed-refs` file in * its current state and return it. The return value will already have From patchwork Mon May 15 19:23:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13242107 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 390EFC7EE24 for ; Mon, 15 May 2023 19:24:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245146AbjEOTYW (ORCPT ); Mon, 15 May 2023 15:24:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39530 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245142AbjEOTXm (ORCPT ); Mon, 15 May 2023 15:23:42 -0400 Received: from mail-yb1-xb29.google.com (mail-yb1-xb29.google.com [IPv6:2607:f8b0:4864:20::b29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4CB3B160B9 for ; Mon, 15 May 2023 12:23:36 -0700 (PDT) Received: by mail-yb1-xb29.google.com with SMTP id 3f1490d57ef6-b9246a5f3feso20314689276.1 for ; Mon, 15 May 2023 12:23:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1684178615; x=1686770615; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=Ib54/wLVPoR3SBJAGaEIu9jp6ZDEcj5FIOf+c/oCSss=; b=4UnohqVlKe9xkvaOZ9QAU1T6ZUGpO9f9XJinuAvIQwP7Gt2EEbyzZOJlCFv1T77uOX 4gib6hBASJHEwps/WZCPQb8MkHBAQWWjD7h1RSg3IEO/CIxwZCDgdJg6ZFW3kK1l2hvx V79KPfqNBg6gLNNRBObdjlAvzNSIeJGHLlnY5hcAoUrpb7c9X/uOSxj8gb4Y6snQBgaS A7cZJO87mJEubUVbnW0Faqo63KItWT2H4fHk2YhORNfibGxkrBwaFsCSNJJ+U2vc54O2 pmPDLZe+LCpIL3hgUClNVnkPqDFLtnDfwnBgyTCxguYK8Fz/BSbvax21JJeUG5QjxNuq qnow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684178615; x=1686770615; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Ib54/wLVPoR3SBJAGaEIu9jp6ZDEcj5FIOf+c/oCSss=; b=fb+1AHRKvWq4WXtVQP5XAlKZ+gCrklk18B0svqJqVV9vvdKjPPWXenutik7T1GwqzV WYMiCH4POKXBJZhvjOOIUwzwyW5zfVlix+t2vzGIuqzTh0++pDb40jQ+Jncfsxc/Q5HF DqhU0PBcdXvvThDOtdNkUnG1w7EYVj1K7pp7TSjYSZd/4vgfBFFlVkiE1XAPSlhybIdr wzV4/xiMs43EVss9eB2cyIYmFGgJF+84w+n2Sq6Flq7EtCLQh7WthgPH6PbhgNyjCb+I ztlufzOY4+U9Ng4X4nAJhJETXKbT1Wl4ty9wMPvuUUuCP17V32gX4BZlQHC+/bEtzAzD UJ1A== X-Gm-Message-State: AC+VfDx2G6D19WGXlsc3D8Omerm8XwTqlPWRnRXdsvRx9yGuii4CP2jM snzU1A8BNMm5PokA1oKZbK5W/fl0uIpVpL7YAWgphg== X-Google-Smtp-Source: ACHHUZ4Vur8MKSsqANyGBKt+qW2QT0S8H4Ee0sBMLupdJ5tbYoNx7GBgxAzpsEvxyBf2ZXfgt5JlUw== X-Received: by 2002:a25:db0d:0:b0:ba7:7c14:590e with SMTP id g13-20020a25db0d000000b00ba77c14590emr5985430ybf.0.1684178615074; Mon, 15 May 2023 12:23:35 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id g6-20020a25ef06000000b00b9e9286f1e6sm8168ybd.55.2023.05.15.12.23.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 May 2023 12:23:34 -0700 (PDT) Date: Mon, 15 May 2023 15:23:33 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Chris Torek , Derrick Stolee , Jeff King , Junio C Hamano , Patrick Steinhardt Subject: [PATCH v2 09/16] refs/packed-backend.c: implement jump lists to avoid excluded pattern(s) Message-ID: <8c78f49a8def4148c92cdd7efe4daa63fb47f641.1684178576.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When iterating through the `packed-refs` file in order to answer a query like: $ git for-each-ref --exclude=refs/__hidden__ it would be useful to avoid walking over all of the entries in `refs/__hidden__/*` when possible, since we know that the ref-filter code is going to throw them away anyways. In certain circumstances, doing so is possible. The algorithm for doing so is as follows: - For each excluded pattern, find the first record that matches it, and the first record that *doesn't* match it (i.e. the location you'd next want to consider when excluding that pattern). - Sort the set of excluded regions from the previous step in ascending order of the first location within the `packed-refs` file that matches. - Clean up the results from the previous step: discard empty regions, and combine adjacent regions. Then when iterating through the `packed-refs` file, if `iter->pos` is ever contained in one of the regions from the previous steps, advance `iter->pos` past the end of that region, and continue enumeration. Note that we only perform this optimization when none of the excluded pattern(s) have special meta-characters in them. For a pattern like "refs/foo[ac]", the excluded regions ("refs/fooa", "refs/fooc", and everything underneath them) are not connected. A future implementation that handles this case may split the character class (pretending as if two patterns were excluded: "refs/fooa", and "refs/fooc"). There are a few other gotchas worth considering. First, note that the jump list is sorted, so once we jump past a region, we can avoid considering it (or any regions preceding it) again. The member `jump_pos` is used to track the first next-possible region to jump through. Second, note that the exclusion list is best-effort, since we do not handle loose references, and because of the meta-character issue above. In repositories with a large number of hidden references, the speed-up can be significant. Tests here are done with a copy of linux.git with a reference "refs/pull/N" pointing at every commit, as in: $ git rev-list HEAD | awk '{ print "create refs/pull/" NR " " $0 }' | git update-ref --stdin $ git pack-refs --all , it is significantly faster to have `for-each-ref` jump over the excluded references, as opposed to filtering them out after the fact: $ hyperfine \ 'git for-each-ref --format="%(objectname) %(refname)" | grep -vE "^[0-9a-f]{40} refs/pull/"' \ 'git.compile for-each-ref --format="%(objectname) %(refname)" --exclude="refs/pull"' Benchmark 1: git for-each-ref --format="%(objectname) %(refname)" | grep -vE "^[0-9a-f]{40} refs/pull/" Time (mean ± σ): 802.7 ms ± 2.1 ms [User: 691.6 ms, System: 147.0 ms] Range (min … max): 800.0 ms … 807.7 ms 10 runs Benchmark 2: git.compile for-each-ref --format="%(objectname) %(refname)" --exclude="refs/pull" Time (mean ± σ): 4.7 ms ± 0.3 ms [User: 0.7 ms, System: 4.0 ms] Range (min … max): 4.3 ms … 6.7 ms 422 runs Summary 'git.compile for-each-ref --format="%(objectname) %(refname)" --exclude="refs/pull"' ran 172.03 ± 9.60 times faster than 'git for-each-ref --format="%(objectname) %(refname)" | grep -vE "^[0-9a-f]{40} refs/pull/"' Using the jump list is fairly straightforward (see the changes to `refs/packed-backend.c::next_record()`), but constructing the list is not. To ensure that the construction is correct, add a new suite of tests in t1419 covering various corner cases (overlapping regions, partially overlapping regions, adjacent regions, etc.). Co-authored-by: Jeff King Signed-off-by: Jeff King Signed-off-by: Taylor Blau --- ref-filter.c | 5 +- refs/packed-backend.c | 166 ++++++++++++++++++++++++++++++++++++-- t/helper/test-ref-store.c | 10 +++ t/t1419-exclude-refs.sh | 101 +++++++++++++++++++++++ 4 files changed, 274 insertions(+), 8 deletions(-) create mode 100755 t/t1419-exclude-refs.sh diff --git a/ref-filter.c b/ref-filter.c index 717c3c4bcf..ddc7f5204f 100644 --- a/ref-filter.c +++ b/ref-filter.c @@ -2210,12 +2210,13 @@ static int for_each_fullref_in_pattern(struct ref_filter *filter, if (!filter->name_patterns[0]) { /* no patterns; we have to look at everything */ return refs_for_each_fullref_in(get_main_ref_store(the_repository), - "", NULL, cb, cb_data); + "", filter->exclude.v, cb, cb_data); } return refs_for_each_fullref_in_prefixes(get_main_ref_store(the_repository), NULL, filter->name_patterns, - NULL, cb, cb_data); + filter->exclude.v, + cb, cb_data); } /* diff --git a/refs/packed-backend.c b/refs/packed-backend.c index 33639f73e1..67327e579c 100644 --- a/refs/packed-backend.c +++ b/refs/packed-backend.c @@ -303,7 +303,8 @@ static int cmp_packed_ref_records(const void *v1, const void *v2) * Compare a snapshot record at `rec` to the specified NUL-terminated * refname. */ -static int cmp_record_to_refname(const char *rec, const char *refname) +static int cmp_record_to_refname(const char *rec, const char *refname, + int start) { const char *r1 = rec + the_hash_algo->hexsz + 1; const char *r2 = refname; @@ -312,7 +313,7 @@ static int cmp_record_to_refname(const char *rec, const char *refname) if (*r1 == '\n') return *r2 ? -1 : 0; if (!*r2) - return 1; + return start ? 1 : -1; if (*r1 != *r2) return (unsigned char)*r1 < (unsigned char)*r2 ? -1 : +1; r1++; @@ -528,7 +529,8 @@ static int load_contents(struct snapshot *snapshot) } static const char *find_reference_location_1(struct snapshot *snapshot, - const char *refname, int mustexist) + const char *refname, int mustexist, + int start) { /* * This is not *quite* a garden-variety binary search, because @@ -558,7 +560,7 @@ static const char *find_reference_location_1(struct snapshot *snapshot, mid = lo + (hi - lo) / 2; rec = find_start_of_record(lo, mid); - cmp = cmp_record_to_refname(rec, refname); + cmp = cmp_record_to_refname(rec, refname, start); if (cmp < 0) { lo = find_end_of_record(mid, hi); } else if (cmp > 0) { @@ -591,7 +593,22 @@ static const char *find_reference_location_1(struct snapshot *snapshot, static const char *find_reference_location(struct snapshot *snapshot, const char *refname, int mustexist) { - return find_reference_location_1(snapshot, refname, mustexist); + return find_reference_location_1(snapshot, refname, mustexist, 1); +} + +/* + * Find the place in `snapshot->buf` after the end of the record for + * `refname`. In other words, find the location of first thing *after* + * `refname`. + * + * Other semantics are identical to the ones in + * `find_reference_location()`. + */ +static const char *find_reference_location_end(struct snapshot *snapshot, + const char *refname, + int mustexist) +{ + return find_reference_location_1(snapshot, refname, mustexist, 0); } /* @@ -785,6 +802,13 @@ struct packed_ref_iterator { /* The end of the part of the buffer that will be iterated over: */ const char *eof; + struct jump_list_entry { + const char *start; + const char *end; + } *jump; + size_t jump_nr, jump_alloc; + size_t jump_pos; + /* Scratch space for current values: */ struct object_id oid, peeled; struct strbuf refname_buf; @@ -802,14 +826,34 @@ struct packed_ref_iterator { */ static int next_record(struct packed_ref_iterator *iter) { - const char *p = iter->pos, *eol; + const char *p, *eol; strbuf_reset(&iter->refname_buf); + /* + * If iter->pos is contained within a skipped region, jump past + * it. + * + * Note that each skipped region is considered at most once, + * since they are ordered based on their starting position. + */ + while (iter->jump_pos < iter->jump_nr) { + struct jump_list_entry *curr = &iter->jump[iter->jump_pos]; + if (iter->pos < curr->start) + break; /* not to the next jump yet */ + + iter->jump_pos++; + if (iter->pos < curr->end) { + iter->pos = curr->end; + break; + } + } + if (iter->pos == iter->eof) return ITER_DONE; iter->base.flags = REF_ISPACKED; + p = iter->pos; if (iter->eof - p < the_hash_algo->hexsz + 2 || parse_oid_hex(p, &iter->oid, &p) || @@ -917,6 +961,7 @@ static int packed_ref_iterator_abort(struct ref_iterator *ref_iterator) int ok = ITER_DONE; strbuf_release(&iter->refname_buf); + free(iter->jump); release_snapshot(iter->snapshot); base_ref_iterator_free(ref_iterator); return ok; @@ -928,6 +973,112 @@ static struct ref_iterator_vtable packed_ref_iterator_vtable = { .abort = packed_ref_iterator_abort }; +static int jump_list_entry_cmp(const void *va, const void *vb) +{ + const struct jump_list_entry *a = va; + const struct jump_list_entry *b = vb; + + if (a->start < b->start) + return -1; + if (a->start > b->start) + return 1; + return 0; +} + +static int has_glob_special(const char *str) +{ + const char *p; + for (p = str; *p; p++) { + if (is_glob_special(*p)) + return 1; + } + return 0; +} + +static const char *ptr_max(const char *x, const char *y) +{ + if (x > y) + return x; + return y; +} + +static void populate_excluded_jump_list(struct packed_ref_iterator *iter, + struct snapshot *snapshot, + const char **excluded_patterns) +{ + size_t i, j; + const char **pattern; + struct jump_list_entry *last_disjoint; + + if (!excluded_patterns) + return; + + for (pattern = excluded_patterns; *pattern; pattern++) { + struct jump_list_entry *e; + + /* + * We can't feed any excludes with globs in them to the + * refs machinery. It only understands prefix matching. + * We likewise can't even feed the string leading up to + * the first meta-character, as something like "foo[a]" + * should not exclude "foobar" (but the prefix "foo" + * would match that and mark it for exclusion). + */ + if (has_glob_special(*pattern)) + continue; + + ALLOC_GROW(iter->jump, iter->jump_nr + 1, iter->jump_alloc); + + e = &iter->jump[iter->jump_nr++]; + e->start = find_reference_location(snapshot, *pattern, 0); + e->end = find_reference_location_end(snapshot, *pattern, 0); + } + + if (!iter->jump_nr) { + /* + * Every entry in exclude_patterns has a meta-character, + * nothing to do here. + */ + return; + } + + QSORT(iter->jump, iter->jump_nr, jump_list_entry_cmp); + + /* + * As an optimization, merge adjacent entries in the jump list + * to jump forwards as far as possible when entering a skipped + * region. + * + * For example, if we have two skipped regions: + * + * [[A, B], [B, C]] + * + * we want to combine that into a single entry jumping from A to + * C. + */ + last_disjoint = iter->jump; + + for (i = 1, j = 1; i < iter->jump_nr; i++) { + struct jump_list_entry *ours = &iter->jump[i]; + + if (ours->start == ours->end) { + /* ignore empty regions (no matching entries) */ + continue; + } else if (ours->start <= last_disjoint->end) { + /* overlapping regions extend the previous one */ + last_disjoint->end = ptr_max(last_disjoint->end, ours->end); + } else { + /* otherwise, insert a new region */ + iter->jump[j++] = *ours; + last_disjoint = ours; + + } + } + + iter->jump_nr = j; + iter->jump_pos = 0; +} + static struct ref_iterator *packed_ref_iterator_begin( struct ref_store *ref_store, const char *prefix, const char **exclude_patterns, @@ -963,6 +1114,9 @@ static struct ref_iterator *packed_ref_iterator_begin( ref_iterator = &iter->base; base_ref_iterator_init(ref_iterator, &packed_ref_iterator_vtable, 1); + if (exclude_patterns) + populate_excluded_jump_list(iter, snapshot, exclude_patterns); + iter->snapshot = snapshot; acquire_snapshot(snapshot); diff --git a/t/helper/test-ref-store.c b/t/helper/test-ref-store.c index 6d8f844e9c..2bff003f7c 100644 --- a/t/helper/test-ref-store.c +++ b/t/helper/test-ref-store.c @@ -175,6 +175,15 @@ static int cmd_for_each_ref(struct ref_store *refs, const char **argv) return refs_for_each_ref_in(refs, prefix, each_ref, NULL); } +static int cmd_for_each_ref__exclude(struct ref_store *refs, const char **argv) +{ + const char *prefix = notnull(*argv++, "prefix"); + const char **exclude_patterns = argv; + + return refs_for_each_fullref_in(refs, prefix, exclude_patterns, each_ref, + NULL); +} + static int cmd_resolve_ref(struct ref_store *refs, const char **argv) { struct object_id oid = *null_oid(); @@ -307,6 +316,7 @@ static struct command commands[] = { { "delete-refs", cmd_delete_refs }, { "rename-ref", cmd_rename_ref }, { "for-each-ref", cmd_for_each_ref }, + { "for-each-ref--exclude", cmd_for_each_ref__exclude }, { "resolve-ref", cmd_resolve_ref }, { "verify-ref", cmd_verify_ref }, { "for-each-reflog", cmd_for_each_reflog }, diff --git a/t/t1419-exclude-refs.sh b/t/t1419-exclude-refs.sh new file mode 100755 index 0000000000..bc534c8ea1 --- /dev/null +++ b/t/t1419-exclude-refs.sh @@ -0,0 +1,101 @@ +#!/bin/sh + +test_description='test exclude_patterns functionality in main ref store' + +GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main +export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME + +TEST_PASSES_SANITIZE_LEAK=true +. ./test-lib.sh + +for_each_ref__exclude () { + test-tool ref-store main for-each-ref--exclude "$@" >actual.raw + cut -d ' ' -f 2 actual.raw +} + +for_each_ref () { + git for-each-ref --format='%(refname)' "$@" +} + +test_expect_success 'setup' ' + test_commit --no-tag base && + base="$(git rev-parse HEAD)" && + + for name in foo bar baz quux + do + for i in 1 2 3 + do + echo "create refs/heads/$name/$i $base" || return 1 + done || return 1 + done >in && + echo "delete refs/heads/main" >>in && + + git update-ref --stdin actual && + for_each_ref refs/heads/bar refs/heads/baz refs/heads/quux >expect && + + test_cmp expect actual +' + +test_expect_success 'excluded region at beginning' ' + for_each_ref__exclude refs/heads refs/heads/bar >actual && + for_each_ref refs/heads/baz refs/heads/foo refs/heads/quux >expect && + + test_cmp expect actual +' + +test_expect_success 'excluded region at end' ' + for_each_ref__exclude refs/heads refs/heads/quux >actual && + for_each_ref refs/heads/foo refs/heads/bar refs/heads/baz >expect && + + test_cmp expect actual +' + +test_expect_success 'disjoint excluded regions' ' + for_each_ref__exclude refs/heads refs/heads/bar refs/heads/quux >actual && + for_each_ref refs/heads/baz refs/heads/foo >expect && + + test_cmp expect actual +' + +test_expect_success 'adjacent, non-overlapping excluded regions' ' + for_each_ref__exclude refs/heads refs/heads/bar refs/heads/baz >actual && + for_each_ref refs/heads/foo refs/heads/quux >expect && + + test_cmp expect actual +' + +test_expect_success 'overlapping excluded regions' ' + for_each_ref__exclude refs/heads refs/heads/ba refs/heads/baz >actual && + for_each_ref refs/heads/foo refs/heads/quux >expect && + + test_cmp expect actual +' + +test_expect_success 'several overlapping excluded regions' ' + for_each_ref__exclude refs/heads \ + refs/heads/bar refs/heads/baz refs/heads/foo >actual && + for_each_ref refs/heads/quux >expect && + + test_cmp expect actual +' + +test_expect_success 'non-matching excluded section' ' + for_each_ref__exclude refs/heads refs/heads/does/not/exist >actual && + for_each_ref >expect && + + test_cmp expect actual +' + +test_expect_success 'meta-characters are discarded' ' + for_each_ref__exclude refs/heads "refs/heads/ba*" >actual && + for_each_ref >expect && + + test_cmp expect actual +' + +test_done From patchwork Mon May 15 19:23:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13242108 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 30C8CC7EE24 for ; Mon, 15 May 2023 19:24:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245147AbjEOTYY (ORCPT ); Mon, 15 May 2023 15:24:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39164 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245073AbjEOTXq (ORCPT ); Mon, 15 May 2023 15:23:46 -0400 Received: from mail-yw1-x112b.google.com (mail-yw1-x112b.google.com [IPv6:2607:f8b0:4864:20::112b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D598916083 for ; Mon, 15 May 2023 12:23:38 -0700 (PDT) Received: by mail-yw1-x112b.google.com with SMTP id 00721157ae682-55a6efe95c9so187775437b3.1 for ; Mon, 15 May 2023 12:23:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1684178618; x=1686770618; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=XBI7CbX1D1uTZmQxEGtIXC6FqAeaYOFo23OZECFFXrk=; b=tzrVuDh8SsTPBB7M4RuGZAYkdYKSz9rvr1IUMeTUkvzD7LHuZTMCPzMqsLtr6kanyi 3emS2XNbrk9QXsFVoz8X/TI+w/HfwlBSOkXpSDs1X0GHWRDJ/oH9zJU0oaPWNBdP9PAi +eAKqrvvAHS9bheGbaADtcC7SI9HYcj5lfPACSrKYgtpZufokGhxT4fWVTQqpy8xcnZF CsqKGh6AvwfO9qPhkK5SbvJ7vRIHk8Zw5qL7xMeef4CNJZ2/vzV/7BIKrKOXLfabqdW7 rx5vdHIyywn+2kAXmnHA5xADf4YbTUvBFSGYc+Je/Ydqgmti/IZuTIM02p7MTmWJxdhF coNw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684178618; x=1686770618; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=XBI7CbX1D1uTZmQxEGtIXC6FqAeaYOFo23OZECFFXrk=; b=g6rm/fZ4tJ8EB5SyyIFa0fTV2olVhq2LCQCg7ctw8auKeIISYbXSlghIcTZ9dHau1m m3sdEOoQ4XV72FnzsgG56TL+j/+Efx3KTFLow4pg70ZncNkoSKTjKWjXNGQLLR0nNjTO BULOK3vUOcyQFd6b9kzkkYWGrlBeisPrW54SyH9IMSijF8AfPB+qddk4jkVkWLCVUHzx 2zB0J2mR6DRYfflhkk7uSo+Ia4utjqr+ednvvyDWT5GewqW4PrrtR0Zlc1m80qwYyH/N MJks49IZOYXQ1kMBcIUXF04irSjm6xNHJxkFGiEotPjrXI1p0XVkQO2XcP4TMca8l4P3 FpHQ== X-Gm-Message-State: AC+VfDzObnkSJlgCWjjigcHcniuv5gTDvHntqZkqsa8Kf4v5BhFUyQnR otgTAG3mDChVmvJGqzB8NVXbuG7+jD6nBh0TCNqA6g== X-Google-Smtp-Source: ACHHUZ6DzDH7Ixs2ewjc4RL3FL7XLk0ziFXysXtPGfrpElAdSnPQ3TcL7AVm8pr6EwEyFRAfee+5mg== X-Received: by 2002:a81:84ce:0:b0:55a:3b80:c00d with SMTP id u197-20020a8184ce000000b0055a3b80c00dmr30356624ywf.25.1684178617820; Mon, 15 May 2023 12:23:37 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id u7-20020a814707000000b0055dfbb44d56sm4455ywa.144.2023.05.15.12.23.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 May 2023 12:23:37 -0700 (PDT) Date: Mon, 15 May 2023 15:23:36 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Chris Torek , Derrick Stolee , Jeff King , Junio C Hamano , Patrick Steinhardt Subject: [PATCH v2 10/16] refs/packed-backend.c: add trace2 counters for jump list Message-ID: <5059f5dd421c94164ce5aee19b8f920417aee27d.1684178576.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org The previous commit added low-level tests to ensure that the packed-refs iterator did not enumerate excluded sections of the refspace. However, there was no guarantee that these sections weren't being visited, only that they were being suppressed from the output. To harden these tests, add a trace2 counter which tracks the number of regions skipped by the packed-refs iterator, and assert on its value. Suggested-by: Derrick Stolee Signed-off-by: Taylor Blau --- refs/packed-backend.c | 2 ++ t/t1419-exclude-refs.sh | 59 ++++++++++++++++++++++++++++------------- trace2.h | 2 ++ trace2/tr2_ctr.c | 5 ++++ 4 files changed, 49 insertions(+), 19 deletions(-) diff --git a/refs/packed-backend.c b/refs/packed-backend.c index 67327e579c..7ba9fa2bb8 100644 --- a/refs/packed-backend.c +++ b/refs/packed-backend.c @@ -12,6 +12,7 @@ #include "../chdir-notify.h" #include "../wrapper.h" #include "../write-or-die.h" +#include "../trace2.h" enum mmap_strategy { /* @@ -845,6 +846,7 @@ static int next_record(struct packed_ref_iterator *iter) iter->jump_pos++; if (iter->pos < curr->end) { iter->pos = curr->end; + trace2_counter_add(TRACE2_COUNTER_ID_PACKED_REFS_JUMPS, 1); break; } } diff --git a/t/t1419-exclude-refs.sh b/t/t1419-exclude-refs.sh index bc534c8ea1..350a7d2587 100755 --- a/t/t1419-exclude-refs.sh +++ b/t/t1419-exclude-refs.sh @@ -9,7 +9,8 @@ TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh for_each_ref__exclude () { - test-tool ref-store main for-each-ref--exclude "$@" >actual.raw + GIT_TRACE2_PERF=1 test-tool ref-store main \ + for-each-ref--exclude "$@" >actual.raw cut -d ' ' -f 2 actual.raw } @@ -17,6 +18,17 @@ for_each_ref () { git for-each-ref --format='%(refname)' "$@" } +assert_jumps () { + local nr="$1" + local trace="$2" + + grep -q "name:jumps_made value:$nr" $trace +} + +assert_no_jumps () { + ! assert_jumps ".*" "$1" +} + test_expect_success 'setup' ' test_commit --no-tag base && base="$(git rev-parse HEAD)" && @@ -35,67 +47,76 @@ test_expect_success 'setup' ' ' test_expect_success 'excluded region in middle' ' - for_each_ref__exclude refs/heads refs/heads/foo >actual && + for_each_ref__exclude refs/heads refs/heads/foo >actual 2>perf && for_each_ref refs/heads/bar refs/heads/baz refs/heads/quux >expect && - test_cmp expect actual + test_cmp expect actual && + assert_jumps 1 perf ' test_expect_success 'excluded region at beginning' ' - for_each_ref__exclude refs/heads refs/heads/bar >actual && + for_each_ref__exclude refs/heads refs/heads/bar >actual 2>perf && for_each_ref refs/heads/baz refs/heads/foo refs/heads/quux >expect && - test_cmp expect actual + test_cmp expect actual && + assert_jumps 1 perf ' test_expect_success 'excluded region at end' ' - for_each_ref__exclude refs/heads refs/heads/quux >actual && + for_each_ref__exclude refs/heads refs/heads/quux >actual 2>perf && for_each_ref refs/heads/foo refs/heads/bar refs/heads/baz >expect && - test_cmp expect actual + test_cmp expect actual && + assert_jumps 1 perf ' test_expect_success 'disjoint excluded regions' ' - for_each_ref__exclude refs/heads refs/heads/bar refs/heads/quux >actual && + for_each_ref__exclude refs/heads refs/heads/bar refs/heads/quux >actual 2>perf && for_each_ref refs/heads/baz refs/heads/foo >expect && - test_cmp expect actual + test_cmp expect actual && + assert_jumps 2 perf ' test_expect_success 'adjacent, non-overlapping excluded regions' ' - for_each_ref__exclude refs/heads refs/heads/bar refs/heads/baz >actual && + for_each_ref__exclude refs/heads refs/heads/bar refs/heads/baz >actual 2>perf && for_each_ref refs/heads/foo refs/heads/quux >expect && - test_cmp expect actual + test_cmp expect actual && + assert_jumps 1 perf ' test_expect_success 'overlapping excluded regions' ' - for_each_ref__exclude refs/heads refs/heads/ba refs/heads/baz >actual && + for_each_ref__exclude refs/heads refs/heads/ba refs/heads/baz >actual 2>perf && for_each_ref refs/heads/foo refs/heads/quux >expect && - test_cmp expect actual + test_cmp expect actual && + assert_jumps 1 perf ' test_expect_success 'several overlapping excluded regions' ' for_each_ref__exclude refs/heads \ - refs/heads/bar refs/heads/baz refs/heads/foo >actual && + refs/heads/bar refs/heads/baz refs/heads/foo >actual 2>perf && for_each_ref refs/heads/quux >expect && - test_cmp expect actual + test_cmp expect actual && + assert_jumps 1 perf ' test_expect_success 'non-matching excluded section' ' - for_each_ref__exclude refs/heads refs/heads/does/not/exist >actual && + for_each_ref__exclude refs/heads refs/heads/does/not/exist >actual 2>perf && for_each_ref >expect && - test_cmp expect actual + test_cmp expect actual && + assert_no_jumps ' test_expect_success 'meta-characters are discarded' ' - for_each_ref__exclude refs/heads "refs/heads/ba*" >actual && + for_each_ref__exclude refs/heads "refs/heads/ba*" >actual 2>perf && for_each_ref >expect && - test_cmp expect actual + test_cmp expect actual && + assert_no_jumps ' test_done diff --git a/trace2.h b/trace2.h index 4ced30c0db..9452e291f5 100644 --- a/trace2.h +++ b/trace2.h @@ -551,6 +551,8 @@ enum trace2_counter_id { TRACE2_COUNTER_ID_TEST1 = 0, /* emits summary event only */ TRACE2_COUNTER_ID_TEST2, /* emits summary and thread events */ + TRACE2_COUNTER_ID_PACKED_REFS_JUMPS, /* counts number of jumps */ + /* Add additional counter definitions before here. */ TRACE2_NUMBER_OF_COUNTERS }; diff --git a/trace2/tr2_ctr.c b/trace2/tr2_ctr.c index b342d3b1a3..50570d0165 100644 --- a/trace2/tr2_ctr.c +++ b/trace2/tr2_ctr.c @@ -27,6 +27,11 @@ static struct tr2_counter_metadata tr2_counter_metadata[TRACE2_NUMBER_OF_COUNTER .name = "test2", .want_per_thread_events = 1, }, + [TRACE2_COUNTER_ID_PACKED_REFS_JUMPS] = { + .category = "packed-refs", + .name = "jumps_made", + .want_per_thread_events = 0, + }, /* Add additional metadata before here. */ }; From patchwork Mon May 15 19:23:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13242125 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 07992C77B7D for ; Mon, 15 May 2023 19:24:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244803AbjEOTYe (ORCPT ); Mon, 15 May 2023 15:24:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39472 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245170AbjEOTXx (ORCPT ); Mon, 15 May 2023 15:23:53 -0400 Received: from mail-yw1-x112e.google.com (mail-yw1-x112e.google.com [IPv6:2607:f8b0:4864:20::112e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8F7C312088 for ; Mon, 15 May 2023 12:23:41 -0700 (PDT) Received: by mail-yw1-x112e.google.com with SMTP id 00721157ae682-55a14807e4cso229262787b3.1 for ; Mon, 15 May 2023 12:23:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1684178620; x=1686770620; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=G+LbZ/jFbK4nuI485lF4RMAcFPTufAXjwWYPJPFW8qE=; b=lFWdSSPvH48TPqhuUt+a6D0MfBUOWOittyYpD8SOOBxiyOlHJ3nffUpln2wSAhuP7X qoQK3GGsg/O62nXG6my0dBhr59dxmLXIGZMMLCfvWAPtkrpDt+80eO4jVzHN89Rs4hFY HstRH9mWnBKoam0TLqggW7kHm0OJ4JJ7yRF+GbL/hC+f2N2EE29XCm5/6mG7kvHyj938 cL8IBcGFNpKu+QpW1QHlKNDMo9R3cjYUtVx92HoVMWSefL7Rr+NwpDKxxdImMbUm3jhf M6qHH7g7qnPL02Lgfc+B820LLAcDYr7enyqjgvdZH5axXrkPvQlz+50+IdqMGRHKUrJE q4sA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684178620; x=1686770620; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=G+LbZ/jFbK4nuI485lF4RMAcFPTufAXjwWYPJPFW8qE=; b=I5ormjHyW1zhV6enHmIOOcAkvR68DKzabDZU7EsSXqsoB+WxO7gl8LJnIjUVVoRz57 22sKj8I8cjkv9EPRYJsz9JOlJhROXo3UEfWQnKrCOauTqm0lQ7lRHZ2wob6ezQi4+tPn XcrlDwwI8r0r4gFdkbXxO5q9AiVLlySX139sxPtcwCkafV9wXo5y/KLP9eiV/EaTEARw VjxIkSPNXUuTRUXZCI5fBzuwTP5KfkS7ECPjnfNRKmav+lCjHyMKOiJ7hACHOOJe2nHg MkYfW5VYU4H2Dfw4Mgr5esmjG1/fW/exhIAcCw0AKB1WsuB7fs1wKFPy5fl/43f90o0a YzOw== X-Gm-Message-State: AC+VfDx98dzkT84e1ofcRrEMCdXXhbtiFicj+X62cuu7Nlv90kZlY3tp 3y8NE+lvO8Ilj4/1n4saIBgUuLULB0M1hVJfMlag8A== X-Google-Smtp-Source: ACHHUZ6Glscd97NMqMbvSQGtURJHVJHE/i086s3HLq6IfrGa49BVXMDxqCjfiaA12kVMggRJpDivPQ== X-Received: by 2002:a81:4995:0:b0:55a:a1e5:9e1c with SMTP id w143-20020a814995000000b0055aa1e59e1cmr34506611ywa.2.1684178620601; Mon, 15 May 2023 12:23:40 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id d62-20020a0df441000000b00545a08184d0sm15009ywf.96.2023.05.15.12.23.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 May 2023 12:23:40 -0700 (PDT) Date: Mon, 15 May 2023 15:23:39 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Chris Torek , Derrick Stolee , Jeff King , Junio C Hamano , Patrick Steinhardt Subject: [PATCH v2 11/16] revision.h: store hidden refs in a `strvec` Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In subsequent commits, it will be convenient to have a 'const char **' of hidden refs (matching `transfer.hiderefs`, `uploadpack.hideRefs`, etc.), instead of a `string_list`. Convert spots throughout the tree that store the list of hidden refs from a `string_list` to a `strvec`. Note that in `parse_hide_refs_config()` there is an ugly const-cast used to avoid an extra copy of each value before trimming any trailing slash characters. This could instead be written as: ref = xstrdup(value); len = strlen(ref); while (len && ref[len - 1] == '/') ref[--len] = '\0'; strvec_push(hide_refs, ref); free(ref); but the double-copy (once when calling `xstrdup()`, and another via `strvec_push()`) is wasteful. Signed-off-by: Taylor Blau --- builtin/receive-pack.c | 4 ++-- ls-refs.c | 6 +++--- refs.c | 11 ++++++----- refs.h | 4 ++-- revision.c | 2 +- revision.h | 5 +++-- upload-pack.c | 10 +++++----- 7 files changed, 22 insertions(+), 20 deletions(-) diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c index 1a31a58367..1a8472eddc 100644 --- a/builtin/receive-pack.c +++ b/builtin/receive-pack.c @@ -90,7 +90,7 @@ static struct object_id push_cert_oid; static struct signature_check sigcheck; static const char *push_cert_nonce; static const char *cert_nonce_seed; -static struct string_list hidden_refs = STRING_LIST_INIT_DUP; +static struct strvec hidden_refs = STRVEC_INIT; static const char *NONCE_UNSOLICITED = "UNSOLICITED"; static const char *NONCE_BAD = "BAD"; @@ -2619,7 +2619,7 @@ int cmd_receive_pack(int argc, const char **argv, const char *prefix) packet_flush(1); oid_array_clear(&shallow); oid_array_clear(&ref); - string_list_clear(&hidden_refs, 0); + strvec_clear(&hidden_refs); free((void *)push_cert_nonce); return 0; } diff --git a/ls-refs.c b/ls-refs.c index 6f490b2d9c..8c3181d051 100644 --- a/ls-refs.c +++ b/ls-refs.c @@ -72,7 +72,7 @@ struct ls_refs_data { unsigned symrefs; struct strvec prefixes; struct strbuf buf; - struct string_list hidden_refs; + struct strvec hidden_refs; unsigned unborn : 1; }; @@ -155,7 +155,7 @@ int ls_refs(struct repository *r, struct packet_reader *request) memset(&data, 0, sizeof(data)); strvec_init(&data.prefixes); strbuf_init(&data.buf, 0); - string_list_init_dup(&data.hidden_refs); + strvec_init(&data.hidden_refs); git_config(ls_refs_config, &data); @@ -197,7 +197,7 @@ int ls_refs(struct repository *r, struct packet_reader *request) packet_fflush(stdout); strvec_clear(&data.prefixes); strbuf_release(&data.buf); - string_list_clear(&data.hidden_refs, 0); + strvec_clear(&data.hidden_refs); return 0; } diff --git a/refs.c b/refs.c index 538bde644e..ec4d5b9101 100644 --- a/refs.c +++ b/refs.c @@ -1427,7 +1427,7 @@ char *shorten_unambiguous_ref(const char *refname, int strict) } int parse_hide_refs_config(const char *var, const char *value, const char *section, - struct string_list *hide_refs) + struct strvec *hide_refs) { const char *key; if (!strcmp("transfer.hiderefs", var) || @@ -1438,22 +1438,23 @@ int parse_hide_refs_config(const char *var, const char *value, const char *secti if (!value) return config_error_nonbool(var); - ref = xstrdup(value); + + /* drop const to remove trailing '/' characters */ + ref = (char *)strvec_push(hide_refs, value); len = strlen(ref); while (len && ref[len - 1] == '/') ref[--len] = '\0'; - string_list_append_nodup(hide_refs, ref); } return 0; } int ref_is_hidden(const char *refname, const char *refname_full, - const struct string_list *hide_refs) + const struct strvec *hide_refs) { int i; for (i = hide_refs->nr - 1; i >= 0; i--) { - const char *match = hide_refs->items[i].string; + const char *match = hide_refs->v[i]; const char *subject; int neg = 0; const char *p; diff --git a/refs.h b/refs.h index d672d636cf..a7751a1fc9 100644 --- a/refs.h +++ b/refs.h @@ -810,7 +810,7 @@ int update_ref(const char *msg, const char *refname, unsigned int flags, enum action_on_err onerr); int parse_hide_refs_config(const char *var, const char *value, const char *, - struct string_list *); + struct strvec *); /* * Check whether a ref is hidden. If no namespace is set, both the first and @@ -820,7 +820,7 @@ int parse_hide_refs_config(const char *var, const char *value, const char *, * the ref is outside that namespace, the first parameter is NULL. The second * parameter always points to the full ref name. */ -int ref_is_hidden(const char *, const char *, const struct string_list *); +int ref_is_hidden(const char *, const char *, const struct strvec *); /* Is this a per-worktree ref living in the refs/ namespace? */ int is_per_worktree_ref(const char *refname); diff --git a/revision.c b/revision.c index 89953592f9..7c9367a266 100644 --- a/revision.c +++ b/revision.c @@ -1558,7 +1558,7 @@ void init_ref_exclusions(struct ref_exclusions *exclusions) void clear_ref_exclusions(struct ref_exclusions *exclusions) { string_list_clear(&exclusions->excluded_refs, 0); - string_list_clear(&exclusions->hidden_refs, 0); + strvec_clear(&exclusions->hidden_refs); exclusions->hidden_refs_configured = 0; } diff --git a/revision.h b/revision.h index 31828748dc..94f035fa22 100644 --- a/revision.h +++ b/revision.h @@ -10,6 +10,7 @@ #include "decorate.h" #include "ident.h" #include "list-objects-filter-options.h" +#include "strvec.h" /** * The revision walking API offers functions to build a list of revisions @@ -95,7 +96,7 @@ struct ref_exclusions { * Hidden refs is a list of patterns that is to be hidden via * `ref_is_hidden()`. */ - struct string_list hidden_refs; + struct strvec hidden_refs; /* * Indicates whether hidden refs have been configured. This is to @@ -110,7 +111,7 @@ struct ref_exclusions { */ #define REF_EXCLUSIONS_INIT { \ .excluded_refs = STRING_LIST_INIT_DUP, \ - .hidden_refs = STRING_LIST_INIT_DUP, \ + .hidden_refs = STRVEC_INIT, \ } struct oidset; diff --git a/upload-pack.c b/upload-pack.c index 08633dc121..d77d58bdde 100644 --- a/upload-pack.c +++ b/upload-pack.c @@ -69,7 +69,7 @@ struct upload_pack_data { struct object_array have_obj; struct oid_array haves; /* v2 only */ struct string_list wanted_refs; /* v2 only */ - struct string_list hidden_refs; + struct strvec hidden_refs; struct object_array shallows; struct string_list deepen_not; @@ -126,7 +126,7 @@ static void upload_pack_data_init(struct upload_pack_data *data) { struct string_list symref = STRING_LIST_INIT_DUP; struct string_list wanted_refs = STRING_LIST_INIT_DUP; - struct string_list hidden_refs = STRING_LIST_INIT_DUP; + struct strvec hidden_refs = STRVEC_INIT; struct object_array want_obj = OBJECT_ARRAY_INIT; struct object_array have_obj = OBJECT_ARRAY_INIT; struct oid_array haves = OID_ARRAY_INIT; @@ -161,7 +161,7 @@ static void upload_pack_data_clear(struct upload_pack_data *data) { string_list_clear(&data->symref, 1); string_list_clear(&data->wanted_refs, 1); - string_list_clear(&data->hidden_refs, 0); + strvec_clear(&data->hidden_refs); object_array_clear(&data->want_obj); object_array_clear(&data->have_obj); oid_array_clear(&data->haves); @@ -1169,7 +1169,7 @@ static void receive_needs(struct upload_pack_data *data, /* return non-zero if the ref is hidden, otherwise 0 */ static int mark_our_ref(const char *refname, const char *refname_full, - const struct object_id *oid, const struct string_list *hidden_refs) + const struct object_id *oid, const struct strvec *hidden_refs) { struct object *o = lookup_unknown_object(the_repository, oid); @@ -1453,7 +1453,7 @@ static int parse_want(struct packet_writer *writer, const char *line, static int parse_want_ref(struct packet_writer *writer, const char *line, struct string_list *wanted_refs, - struct string_list *hidden_refs, + struct strvec *hidden_refs, struct object_array *want_obj) { const char *refname_nons; From patchwork Mon May 15 19:23:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13242126 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 04656C7EE25 for ; Mon, 15 May 2023 19:24:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245235AbjEOTYf (ORCPT ); Mon, 15 May 2023 15:24:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39122 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245080AbjEOTYG (ORCPT ); Mon, 15 May 2023 15:24:06 -0400 Received: from mail-yw1-x112f.google.com (mail-yw1-x112f.google.com [IPv6:2607:f8b0:4864:20::112f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 664BF147E8 for ; Mon, 15 May 2023 12:23:44 -0700 (PDT) Received: by mail-yw1-x112f.google.com with SMTP id 00721157ae682-559f1819c5dso193450397b3.0 for ; Mon, 15 May 2023 12:23:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1684178623; x=1686770623; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=NNGi53S/3qn1N28jogNqbAYIVtJ4jXnS2kp8JsIHbE8=; b=egBp3mRzQVDtAOY+2U05aIzmpRJqiGrLdgakld9/3g+p648c9QVUXG4PqNuAGTyXfB gV6RVq1WqBdp8pgw/+NGLonHNGkuukx14bu6xe+ie/XIX6abFBk0fUprZqSc4mdX1B0+ HxqHf0lWtyCw/h7aExnO61RNWH3T9+XxmT4HoTK3hIUazby0Owy2rpth5LZ4Tubqi9AY 7QlIds1Zg9m2IRa8MlU0pKZC3m7nEJShU4cN3GcztBr0sTY7GQtZBLy0araT5rUGLMtr vefqCelKOLC/0/bzecxWR5jSYQqOML6tgCpvHC1bWeuh1RVxYRCG6xXuAB+G3Inuk0Eb J8Xg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684178623; x=1686770623; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=NNGi53S/3qn1N28jogNqbAYIVtJ4jXnS2kp8JsIHbE8=; b=NCBvxNxS2gVwcMB9koF3LhMY0DekjpVbWd3t+tehe9BMlI3Kvp7h77SUTwN+3N+whd rbyW75qtf5j5rl6BuBUqAZvvTHdi/Qmomwo0VAQRoRuCvtPOAH5aua373m/POKsDkH6j qaeGDHzsdEyoVMQ3oKKGpIu3qIKHN8Tf/nVaUDByjJd+YFhrhCECXft9aXiiDpXl3Mqv zp9KjmRCXybBn33CIhuVsUPo9fa+5sJi+TuFO4M81mhe1GgHRhHUqsi9USGoJxpc6pVx DyAEfo66cnlgz10D0EjB/zJVWH6F8mNhjIE6CDHlVLzxcW5gmO46kZbg8oD26fxoWprS H1WA== X-Gm-Message-State: AC+VfDxaMY1H5GLVOLrccSadWDkEuxmqBGCmgl0z1bDYxWbBgdxb6IXP 60Sqhj4bY+cnORnKPDwz14tJR4Zk7jTJx8VvsIRbyw== X-Google-Smtp-Source: ACHHUZ7ycX6lt1Pz5c/DNbMPdGghKLcpuV2No7V2rOuZpOKBsv8yYrSlUh7n0ZbSJwq99D4KCA3uoA== X-Received: by 2002:a0d:cf42:0:b0:55a:72f6:a451 with SMTP id r63-20020a0dcf42000000b0055a72f6a451mr32080100ywd.41.1684178623394; Mon, 15 May 2023 12:23:43 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id b64-20020a0dc043000000b0055aaccfa2c7sm16538ywd.91.2023.05.15.12.23.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 May 2023 12:23:43 -0700 (PDT) Date: Mon, 15 May 2023 15:23:42 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Chris Torek , Derrick Stolee , Jeff King , Junio C Hamano , Patrick Steinhardt Subject: [PATCH v2 12/16] refs/packed-backend.c: ignore complicated hidden refs rules Message-ID: <254bcc436124e70ae477d2668c22f2274e43e513.1684178576.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In subsequent commits, we'll teach `receive-pack` and `upload-pack` to use the new skip-list feature in the packed-refs iterator by ignoring references which are mentioned via its respective hideRefs lists. However, the packed-ref skip lists cannot handle un-hiding rules (that begin with '!'), or namespace comparisons (that begin with '^'). Detect and avoid these cases by falling back to the normal enumeration without a skip list when such patterns exist. Signed-off-by: Taylor Blau --- refs/packed-backend.c | 19 +++++++++++++++++++ t/t1419-exclude-refs.sh | 9 +++++++++ 2 files changed, 28 insertions(+) diff --git a/refs/packed-backend.c b/refs/packed-backend.c index 7ba9fa2bb8..9ea6c07866 100644 --- a/refs/packed-backend.c +++ b/refs/packed-backend.c @@ -1015,6 +1015,25 @@ static void populate_excluded_jump_list(struct packed_ref_iterator *iter, if (!excluded_patterns) return; + for (pattern = excluded_patterns; *pattern; pattern++) { + /* + * We also can't feed any excludes from hidden refs + * config sections, since later rules may override + * previous ones. For example, with rules "refs/foo" and + * "!refs/foo/bar", we should show "refs/foo/bar" (and + * everything underneath it), but the earlier exclusion + * would cause us to skip all of "refs/foo". We likewise + * don't implement the namespace stripping required for + * '^' rules. + * + * Both are possible to do, but complicated, so avoid + * populating the jump list at all if we see either of + * these patterns. + */ + if (**pattern == '!' || **pattern == '^') + return; + } + for (pattern = excluded_patterns; *pattern; pattern++) { struct jump_list_entry *e; diff --git a/t/t1419-exclude-refs.sh b/t/t1419-exclude-refs.sh index 350a7d2587..0e91e2f399 100755 --- a/t/t1419-exclude-refs.sh +++ b/t/t1419-exclude-refs.sh @@ -119,4 +119,13 @@ test_expect_success 'meta-characters are discarded' ' assert_no_jumps ' +test_expect_success 'complex hidden ref rules are discarded' ' + for_each_ref__exclude refs/heads refs/heads/foo "!refs/heads/foo/1" \ + >actual 2>perf && + for_each_ref >expect && + + test_cmp expect actual && + assert_no_jumps +' + test_done From patchwork Mon May 15 19:23:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13242127 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C0FA6C77B7D for ; Mon, 15 May 2023 19:24:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245091AbjEOTYh (ORCPT ); Mon, 15 May 2023 15:24:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39486 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245175AbjEOTYO (ORCPT ); Mon, 15 May 2023 15:24:14 -0400 Received: from mail-yw1-x1131.google.com (mail-yw1-x1131.google.com [IPv6:2607:f8b0:4864:20::1131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6131B16405 for ; Mon, 15 May 2023 12:23:47 -0700 (PDT) Received: by mail-yw1-x1131.google.com with SMTP id 00721157ae682-56186035b61so842557b3.3 for ; Mon, 15 May 2023 12:23:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1684178626; x=1686770626; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=ffUDsnLL4vECbx2kK2WEeblCLG55QfI/7WjKVedyswM=; b=m5uqAwR0BBKhSd0LXNnh0Onl4AdKVegx+/uPPQ0v5ATu0MdUKPn/PalmgRsq9A/WRS 3jGxrvHuYYTcWKVPci41CXBfWDsAv0z+S3ipe1xP52vEe02+Dgk9g26p0eY+9d1fONh3 wDJdZ4098izb029h+Pa5RaMtabPdfRpJuQzwkPhBTelgqMVRFNJIaAJuaxRy0e92itPQ LKcH364NQFMCw6V4gDr53WhTLocMDCSBTRfjUvvRs0dN0+WLi4FXd+ZvuPnSjxuIt8oP zlTde9g/LUT5+kqmV+lnIg25nUqPCCGaa9Ll9N1855q4HSjQNNwSQzJsym8UpJ6iLMDV zQLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684178626; x=1686770626; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=ffUDsnLL4vECbx2kK2WEeblCLG55QfI/7WjKVedyswM=; b=FoyOahIWs1K23TZTjROUkHR8Sj9JwuO8v+QBupMVnFTB5GUK3DfGe4m4tMLTBNUZI6 aUEnCejeTwQ4Pu2zMaUw0dTnVxivl1HYlZoc7UAV9kTBk9VEmktd57TOg7AMxPMNP6yw 3tfVZK6qU/+iKa48dgT7E1nt2IMyq4tOCT0NUvwntmP0dIImNhYy9GHHFiDG58/WdwdG 5BU8Nc2rtknHw2yn9/evmhRkxTi1SdZdk6Y+/IEHfmgAPt+pgn6nnro4fDWrbz3qFvoB 1o5r9yIZCprHOJdZxNocrxaKq/5WYOrtpUUwuJWbXx4slku/OtzoLVFzCSKcdmLOnXAr GfKA== X-Gm-Message-State: AC+VfDzloP+IE7OGo/yRrWd7WO2CevarslwV2CFzUGAX/QxWSTet++GY PlDOKOsZEu18GFzQUENuCuIxDB/7CneyPseSNAj6tA== X-Google-Smtp-Source: ACHHUZ7xFpcSrfv3q/5H9HQUolcgloqJhsuSt+KcYRPhrmDKdqxNnDTpnjsljWP0SYdkO8dtN5H64Q== X-Received: by 2002:a0d:d9c7:0:b0:55b:4942:c86b with SMTP id b190-20020a0dd9c7000000b0055b4942c86bmr31746215ywe.42.1684178626416; Mon, 15 May 2023 12:23:46 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id p205-20020a8174d6000000b0055ddea0db57sm3460ywc.146.2023.05.15.12.23.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 May 2023 12:23:46 -0700 (PDT) Date: Mon, 15 May 2023 15:23:45 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Chris Torek , Derrick Stolee , Jeff King , Junio C Hamano , Patrick Steinhardt Subject: [PATCH v2 13/16] refs.h: let `for_each_namespaced_ref()` take excluded patterns Message-ID: <50e7df7dc0932a53682664a6334791d8f1ff31bb.1684178576.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org The following commit will want to call `for_each_namespaced_ref()` with a list of excluded patterns. We could introduce a variant of that function, say, `for_each_namespaced_ref_exclude()` which takes the extra parameter, and reimplement the original function in terms of that. But all but one caller (in `http-backend.c`) will supply the new parameter, so add the new parameter to `for_each_namespaced_ref()` itself instead of introducing a new function. For now, supply NULL for the list of excluded patterns at all callers to avoid changing behavior, which we will do in the subsequent commit. Signed-off-by: Taylor Blau --- http-backend.c | 2 +- refs.c | 5 +++-- refs.h | 3 ++- upload-pack.c | 6 +++--- 4 files changed, 9 insertions(+), 7 deletions(-) diff --git a/http-backend.c b/http-backend.c index ac146d85c5..ad500683c8 100644 --- a/http-backend.c +++ b/http-backend.c @@ -559,7 +559,7 @@ static void get_info_refs(struct strbuf *hdr, char *arg UNUSED) } else { select_getanyfile(hdr); - for_each_namespaced_ref(show_text_ref, &buf); + for_each_namespaced_ref(NULL, show_text_ref, &buf); send_strbuf(hdr, "text/plain", &buf); } strbuf_release(&buf); diff --git a/refs.c b/refs.c index ec4d5b9101..95a7db9563 100644 --- a/refs.c +++ b/refs.c @@ -1660,13 +1660,14 @@ int for_each_replace_ref(struct repository *r, each_repo_ref_fn fn, void *cb_dat DO_FOR_EACH_INCLUDE_BROKEN, cb_data); } -int for_each_namespaced_ref(each_ref_fn fn, void *cb_data) +int for_each_namespaced_ref(const char **exclude_patterns, + each_ref_fn fn, void *cb_data) { struct strbuf buf = STRBUF_INIT; int ret; strbuf_addf(&buf, "%srefs/", get_git_namespace()); ret = do_for_each_ref(get_main_ref_store(the_repository), - buf.buf, NULL, fn, 0, 0, cb_data); + buf.buf, exclude_patterns, fn, 0, 0, cb_data); strbuf_release(&buf); return ret; } diff --git a/refs.h b/refs.h index a7751a1fc9..f23626beca 100644 --- a/refs.h +++ b/refs.h @@ -372,7 +372,8 @@ int for_each_glob_ref_in(each_ref_fn fn, const char *pattern, const char *prefix, void *cb_data); int head_ref_namespaced(each_ref_fn fn, void *cb_data); -int for_each_namespaced_ref(each_ref_fn fn, void *cb_data); +int for_each_namespaced_ref(const char **exclude_patterns, + each_ref_fn fn, void *cb_data); /* can be used to learn about broken ref and symref */ int refs_for_each_rawref(struct ref_store *refs, each_ref_fn fn, void *cb_data); diff --git a/upload-pack.c b/upload-pack.c index d77d58bdde..7c646ea5bd 100644 --- a/upload-pack.c +++ b/upload-pack.c @@ -854,7 +854,7 @@ static void deepen(struct upload_pack_data *data, int depth) * marked with OUR_REF. */ head_ref_namespaced(check_ref, data); - for_each_namespaced_ref(check_ref, data); + for_each_namespaced_ref(NULL, check_ref, data); get_reachable_list(data, &reachable_shallows); result = get_shallow_commits(&reachable_shallows, @@ -1378,7 +1378,7 @@ void upload_pack(const int advertise_refs, const int stateless_rpc, if (advertise_refs) data.no_done = 1; head_ref_namespaced(send_ref, &data); - for_each_namespaced_ref(send_ref, &data); + for_each_namespaced_ref(NULL, send_ref, &data); /* * fflush stdout before calling advertise_shallow_grafts because send_ref * uses stdio. @@ -1388,7 +1388,7 @@ void upload_pack(const int advertise_refs, const int stateless_rpc, packet_flush(1); } else { head_ref_namespaced(check_ref, &data); - for_each_namespaced_ref(check_ref, &data); + for_each_namespaced_ref(NULL, check_ref, &data); } if (!advertise_refs) { From patchwork Mon May 15 19:23:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13242128 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9F661C7EE24 for ; Mon, 15 May 2023 19:24:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245239AbjEOTYm (ORCPT ); Mon, 15 May 2023 15:24:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40008 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245188AbjEOTYP (ORCPT ); Mon, 15 May 2023 15:24:15 -0400 Received: from mail-yw1-x1135.google.com (mail-yw1-x1135.google.com [IPv6:2607:f8b0:4864:20::1135]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2A92816086 for ; Mon, 15 May 2023 12:23:50 -0700 (PDT) Received: by mail-yw1-x1135.google.com with SMTP id 00721157ae682-55a010774a5so194794117b3.3 for ; Mon, 15 May 2023 12:23:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1684178629; x=1686770629; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=Pt0W8TK070UZbAvvyy4ks588uT7F0hnS1aN3mrWip98=; b=ZLguvgXRNrpD6kR+DJZbbc4277bt/vfhDai4OyZujrNqd/szIVyLLtSl9nw6Y9LfxG fFic1G7NGuyC95nPVyNtDX2uJm5JNx/CL6bQpY5vr19VtbPBvynKRQmWo7Fs/fUV89eP +CTsLSxttuvjtPednVpIJ1LuyqRYSZorPIS4SAFs0Qxbp8D09hnNs3OlodRhjdRKBO3U xU7MZu+XaA1BpFGCPRHsqVhksSLTtoTv4pC8xlFVFmAM/+Hm9vPztNmtUPnhIRHj19UL M0vShhDEiTQ7exaE+LU+8QaCW4OQKzI74z4xeUArqfDmDNTxa8TRMz7zPjSuQR+hrOqi Z+0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684178629; x=1686770629; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Pt0W8TK070UZbAvvyy4ks588uT7F0hnS1aN3mrWip98=; b=SuR3cMPdPVORTXlqFRbU/XtTia79BjiE1O1XVzR8riBFR/kEuY9Lj4KqUKqoK3TwG/ cidWeeLciHgzRXTROpjorFtKDdmEFgbyKK8vn8/WCIxvn8OSX1uqH7UCXMhQnzxvg+we 1jXcIYDaNuzvOKDgce+I9YckJdOeLWZFV3cu9i7XrY7QiH1eNMGtdWjCLbIYNwI0JPwd Ghd/KwjpUXunDdrxqc3xnipqlh+RKJd3uo8iEVD+5cUsWtbfqgVW9++xYy5NLQsaiKgq 6PeDCnO9KgHXIswyiQwYLYqtGVUDEZZzkDn/DtARfufQ4H6xMhHZBLwaj3jNQ/08/ZbJ 0pvw== X-Gm-Message-State: AC+VfDxXaqDkSF6zteZmM1E499Rv+nAtk0yC0rN+cXq6m2SVregkTNj3 YgzvDFO3ptfDvF3FiFAPwH/H3xe+UCALRV8/Nx8NEw== X-Google-Smtp-Source: ACHHUZ6Oo8d0hITLGOXU29l2o5FSyJMAYU0laelFpjRxadkat/Vcfpzi18dgB7hWQS7I5AOcRxuJXw== X-Received: by 2002:a0d:ca88:0:b0:55a:9255:9267 with SMTP id m130-20020a0dca88000000b0055a92559267mr31024439ywd.36.1684178629229; Mon, 15 May 2023 12:23:49 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id u7-20020a814707000000b0055dfbb44d56sm4557ywa.144.2023.05.15.12.23.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 May 2023 12:23:49 -0700 (PDT) Date: Mon, 15 May 2023 15:23:48 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Chris Torek , Derrick Stolee , Jeff King , Junio C Hamano , Patrick Steinhardt Subject: [PATCH v2 14/16] builtin/receive-pack.c: avoid enumerating hidden references Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Now that `refs_for_each_fullref_in()` has the ability to avoid enumerating references matching certain pattern(s), use that to avoid visiting hidden refs when constructing the ref advertisement via receive-pack. Note that since this exclusion is best-effort, we still need `show_ref_cb()` to check whether or not each reference is hidden or not before including it in the advertisement. As was the case when applying this same optimization to `upload-pack`, `receive-pack`'s reference advertisement phase can proceed much quicker by avoiding enumerating references that will not be part of the advertisement. (Below, we're still using linux.git with one hidden refs/pull/N ref per commit): $ hyperfine -L v ,.compile 'git{v} -c transfer.hideRefs=refs/pull receive-pack --advertise-refs .git' Benchmark 1: git -c transfer.hideRefs=refs/pull receive-pack --advertise-refs .git Time (mean ± σ): 89.1 ms ± 1.7 ms [User: 82.0 ms, System: 7.0 ms] Range (min … max): 87.7 ms … 95.5 ms 31 runs Benchmark 2: git.compile -c transfer.hideRefs=refs/pull receive-pack --advertise-refs .git Time (mean ± σ): 4.5 ms ± 0.2 ms [User: 0.5 ms, System: 3.9 ms] Range (min … max): 4.1 ms … 5.6 ms 508 runs Summary 'git.compile -c transfer.hideRefs=refs/pull receive-pack --advertise-refs .git' ran 20.00 ± 1.05 times faster than 'git -c transfer.hideRefs=refs/pull receive-pack --advertise-refs .git' Signed-off-by: Taylor Blau --- builtin/receive-pack.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c index 1a8472eddc..bd5bcc375f 100644 --- a/builtin/receive-pack.c +++ b/builtin/receive-pack.c @@ -337,7 +337,8 @@ static void write_head_info(void) { static struct oidset seen = OIDSET_INIT; - for_each_ref(show_ref_cb, &seen); + refs_for_each_fullref_in(get_main_ref_store(the_repository), "", + hidden_refs.v, show_ref_cb, &seen); for_each_alternate_ref(show_one_alternate_ref, &seen); oidset_clear(&seen); if (!sent_capabilities) From patchwork Mon May 15 19:23:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13242129 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 99CCEC7EE24 for ; Mon, 15 May 2023 19:24:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245242AbjEOTYn (ORCPT ); Mon, 15 May 2023 15:24:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40014 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245193AbjEOTYQ (ORCPT ); Mon, 15 May 2023 15:24:16 -0400 Received: from mail-yb1-xb35.google.com (mail-yb1-xb35.google.com [IPv6:2607:f8b0:4864:20::b35]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 30A8C1493C for ; Mon, 15 May 2023 12:23:53 -0700 (PDT) Received: by mail-yb1-xb35.google.com with SMTP id 3f1490d57ef6-ba1815e12efso11344589276.3 for ; Mon, 15 May 2023 12:23:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1684178632; x=1686770632; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=HtU7LeqJ0h33agazXgkUWegoqSRW4Mp/thJTpGp+iiw=; b=XUZGnt/lstqU5vwSn5cDnO56IpiylZkxiqotaQGUlq2X30zJ9uNA2Gem0fakLR8ujM NTjydJ/i9UgCPRyXd2Mmk0utSR66hEWpHOGI7WLlPvRvyMuoByyhkLpO70mkQgdGZQ9X 0JGP8LLumQdRD9nIJqc1g5tdA2qUIZOQHb6eWPHxozScx0BgbfD7f9Ka0IFKhbtUFbZA bl/rw+t9Jcurik1DHrjtxFtpN++wLs+yXzp3gEs7AEFczVCIB8Wxtut3mFHvbAKstuZ4 xebgvWQlf3ElmKM68Ag+zC2a7vg0EFTsecmoZcj7Q87RgDHb5RwzhyMWkmy73E6LG8Mq Sl1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684178632; x=1686770632; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=HtU7LeqJ0h33agazXgkUWegoqSRW4Mp/thJTpGp+iiw=; b=fEXQwXzWcw+qecmEThd2goASo+Au8rGwyeV3ScfUWNIgbXjTMoVkqtbwmFjnvUxFwh u5ke4u8LAT0sR9o4t86WqE4iEI3aU1P7AFhCMwebTHJrgS5EMzX/laMDq69geuDyh26S e2V4zcu8kQu3FFWfJgn2C/ZKrRdJziAIExb2iCKB5usaOdpGtAMUFYohuenaxms8RrNg bOhn1l9bVwtUxI6ZkMSoE1wjYXcV6r4XIYPgoQxBRBX8OenNyaOu1FowaeLGXx0Edufj mbN2YXYkQkCIFSVwQWhx4dlFE6mg4RbpDChRQJUzgQcadjJ8whb+tmksL+D0d3hieDrG Ky1g== X-Gm-Message-State: AC+VfDzacaVACVed575gMF06YKcoDmmgyXNeoTUmdbd+8m/IAF6/nGaG psamFmBb5lWUc3CvJgX3JfH6Oi5ivye0uztkHEiBQg== X-Google-Smtp-Source: ACHHUZ58IFsWur8qhzhcYE+sAwoIJnOBX33bYFdZo+d0WECH0JvCO1rRlXWaFZYf2EMfaIKPpTEn6Q== X-Received: by 2002:a25:26cc:0:b0:ba1:e7f2:8369 with SMTP id m195-20020a2526cc000000b00ba1e7f28369mr32175974ybm.40.1684178632147; Mon, 15 May 2023 12:23:52 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id z1-20020a056902054100b00b9de731247csm8743ybs.54.2023.05.15.12.23.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 May 2023 12:23:51 -0700 (PDT) Date: Mon, 15 May 2023 15:23:51 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Chris Torek , Derrick Stolee , Jeff King , Junio C Hamano , Patrick Steinhardt Subject: [PATCH v2 15/16] upload-pack.c: avoid enumerating hidden refs where possible Message-ID: <2331fa7a4d0ef0ed2445b25064e2b3270bed5d2b.1684178576.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In a similar fashion as a previous commit, teach `upload-pack` to avoid enumerating hidden references where possible. Note, however, that there are certain cases where cannot avoid enumerating even hidden references, in particular when either of: - `uploadpack.allowTipSHA1InWant`, or - `uploadpack.allowReachableSHA1InWant` are set, corresponding to `ALLOW_TIP_SHA1` and `ALLOW_REACHABLE_SHA1`, respectively. When either of these bits are set, upload-pack's `is_our_ref()` function needs to consider the `HIDDEN_REF` bit of the referent's object flags. So we must visit all references, including the hidden ones, in order to mark their referents with the `HIDDEN_REF` bit. When neither `ALLOW_TIP_SHA1` nor `ALLOW_REACHABLE_SHA1` are set, the `is_our_ref()` function considers only the `OUR_REF` bit, and not the `HIDDEN_REF` one. `OUR_REF` is applied via `mark_our_ref()`, and only to objects at the tips of non-hidden references, so we do not need to visit hidden references in this case. When neither of those bits are set, `upload-pack` can potentially avoid enumerating a large number of references. In the same example as a previous commit (linux.git with one hidden reference per commit, "refs/pull/N"): $ printf 0000 >in $ hyperfine --warmup=1 \ 'git -c transfer.hideRefs=refs/pull upload-pack . --- upload-pack.c | 33 +++++++++++++++++++++++++++------ 1 file changed, 27 insertions(+), 6 deletions(-) diff --git a/upload-pack.c b/upload-pack.c index 7c646ea5bd..6fa667bf25 100644 --- a/upload-pack.c +++ b/upload-pack.c @@ -601,11 +601,32 @@ static int get_common_commits(struct upload_pack_data *data, } } +static int allow_hidden_refs(enum allow_uor allow_uor) +{ + return allow_uor & (ALLOW_TIP_SHA1 | ALLOW_REACHABLE_SHA1); +} + +static void for_each_namespaced_ref_1(each_ref_fn fn, + struct upload_pack_data *data) +{ + /* + * If `data->allow_uor` allows fetching hidden refs, we need to + * mark all references (including hidden ones), to check in + * `is_our_ref()` below. + * + * Otherwise, we only care about whether each reference's object + * has the OUR_REF bit set or not, so do not need to visit + * hidden references. + */ + if (allow_hidden_refs(data->allow_uor)) + for_each_namespaced_ref(NULL, fn, data); + else + for_each_namespaced_ref(data->hidden_refs.v, fn, data); +} + static int is_our_ref(struct object *o, enum allow_uor allow_uor) { - int allow_hidden_ref = (allow_uor & - (ALLOW_TIP_SHA1 | ALLOW_REACHABLE_SHA1)); - return o->flags & ((allow_hidden_ref ? HIDDEN_REF : 0) | OUR_REF); + return o->flags & ((allow_hidden_refs(allow_uor) ? HIDDEN_REF : 0) | OUR_REF); } /* @@ -854,7 +875,7 @@ static void deepen(struct upload_pack_data *data, int depth) * marked with OUR_REF. */ head_ref_namespaced(check_ref, data); - for_each_namespaced_ref(NULL, check_ref, data); + for_each_namespaced_ref_1(check_ref, data); get_reachable_list(data, &reachable_shallows); result = get_shallow_commits(&reachable_shallows, @@ -1378,7 +1399,7 @@ void upload_pack(const int advertise_refs, const int stateless_rpc, if (advertise_refs) data.no_done = 1; head_ref_namespaced(send_ref, &data); - for_each_namespaced_ref(NULL, send_ref, &data); + for_each_namespaced_ref_1(send_ref, &data); /* * fflush stdout before calling advertise_shallow_grafts because send_ref * uses stdio. @@ -1388,7 +1409,7 @@ void upload_pack(const int advertise_refs, const int stateless_rpc, packet_flush(1); } else { head_ref_namespaced(check_ref, &data); - for_each_namespaced_ref(NULL, check_ref, &data); + for_each_namespaced_ref_1(check_ref, &data); } if (!advertise_refs) { From patchwork Mon May 15 19:23:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Taylor Blau X-Patchwork-Id: 13242130 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DC7C1C7EE26 for ; Mon, 15 May 2023 19:24:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245120AbjEOTYv (ORCPT ); Mon, 15 May 2023 15:24:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40026 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245203AbjEOTYR (ORCPT ); Mon, 15 May 2023 15:24:17 -0400 Received: from mail-yb1-xb30.google.com (mail-yb1-xb30.google.com [IPv6:2607:f8b0:4864:20::b30]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 29686160BA for ; Mon, 15 May 2023 12:23:56 -0700 (PDT) Received: by mail-yb1-xb30.google.com with SMTP id 3f1490d57ef6-ba729a64ba8so3354765276.2 for ; Mon, 15 May 2023 12:23:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ttaylorr-com.20221208.gappssmtp.com; s=20221208; t=1684178635; x=1686770635; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=ZaLbohQ+PKZiejMjH0QzkbAVObghkl6P97WFXVQ+OhI=; b=T5JcUAGUKE+YUeiInqMEEmFtHd1u/E5//YY8L90/stspex3iQmKL/mSzVseITQIO9n 82nAZLs1b5TlNTaFbuSfdQtwyZKLeKMfn4eXPJqpkwWoVxVxaNlsT6tkzlbLzLzA89So M5hb3czSCpDzV5nU2QAXS0to1XunWmZ1pKbNINT707sgS5hJARIUE+Oq3iduyvN42i/U OWTBafJJIttqTdK2Earuv2yX6DDM8SM3aUOPtRme/igvrFgj6HPV3mRnq3ivKhwrqvlv eak4YIkwr90xT/a0RqFsNuy8WAsqmAdtSarlZXObwymgo9gjAJvP/nNNqplq25CyD6Fr wpYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684178635; x=1686770635; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=ZaLbohQ+PKZiejMjH0QzkbAVObghkl6P97WFXVQ+OhI=; b=GSDKDXDRjUblH6NMBwjcDemNd7r8UEZzih1AH2Xz5O6yWZzQo75Yoe95T3ifNne9xx 2zsNljSPrMa9hQ5i03komf7o+bWc9A4aCkPyj0Z8M9M+PgFKMGwVcPK9UlHYPVQE5Iqq Dp9NV210ANvYbhj18xR0ZkVG1IZCQPTl+o7oUkUm8K9zQLtmd76LnvjdCM+0t/MvSoCE W2dQPGSBFiyRcpKbEt5fADTinUyCbFK/Ucp2U/Vjjrw7sbzLGbG6LH3Lr9MvXIR8zYXL 68QFHHnFjppMWdVF4gA493+oeL/4wSubIJA2VPn0G9D9Cr8VBijmtAdzgTUmGGw5a9dQ 92zA== X-Gm-Message-State: AC+VfDziKcbCos0L6Npkg80UwBmSS0z227tPMBfyNoOUfCEpLz8YXlGH HMZ9Mt5sD2ocDTH8IG0Thf44WlGQRqhCZeJ3weNyPg== X-Google-Smtp-Source: ACHHUZ4QNJ1WIzXUhN1c7086uCwurZF4ULBgnxN1s2IJdIHh9tF313QxBayXD1fynXKgNPi/GlZKeg== X-Received: by 2002:a25:6912:0:b0:b3b:a48c:b241 with SMTP id e18-20020a256912000000b00b3ba48cb241mr30248931ybc.31.1684178634861; Mon, 15 May 2023 12:23:54 -0700 (PDT) Received: from localhost (104-178-186-189.lightspeed.milwwi.sbcglobal.net. [104.178.186.189]) by smtp.gmail.com with ESMTPSA id w14-20020a25ef4e000000b00ba7fd64e113sm17144ybm.36.2023.05.15.12.23.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 May 2023 12:23:54 -0700 (PDT) Date: Mon, 15 May 2023 15:23:53 -0400 From: Taylor Blau To: git@vger.kernel.org Cc: Chris Torek , Derrick Stolee , Jeff King , Junio C Hamano , Patrick Steinhardt Subject: [PATCH v2 16/16] ls-refs.c: avoid enumerating hidden refs where possible Message-ID: <2c6b89d64a164bdeec34db13017f3228c6a73184.1684178576.git.me@ttaylorr.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In a similar fashion as in previous commits, teach `ls-refs` to avoid enumerating hidden references where possible. As before, this is linux.git with one hidden reference per commit. $ hyperfine -L v ,.compile 'git{v} -c protocol.version=2 ls-remote .' Benchmark 1: git -c protocol.version=2 ls-remote . Time (mean ± σ): 89.8 ms ± 0.6 ms [User: 84.3 ms, System: 5.7 ms] Range (min … max): 88.8 ms … 91.3 ms 32 runs Benchmark 2: git.compile -c protocol.version=2 ls-remote . Time (mean ± σ): 6.5 ms ± 0.1 ms [User: 2.4 ms, System: 4.3 ms] Range (min … max): 6.2 ms … 8.3 ms 397 runs Summary 'git.compile -c protocol.version=2 ls-remote .' ran 13.85 ± 0.33 times faster than 'git -c protocol.version=2 ls-remote .' Signed-off-by: Taylor Blau --- ls-refs.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ls-refs.c b/ls-refs.c index 8c3181d051..c9a723ba89 100644 --- a/ls-refs.c +++ b/ls-refs.c @@ -193,7 +193,7 @@ int ls_refs(struct repository *r, struct packet_reader *request) strvec_push(&data.prefixes, ""); refs_for_each_fullref_in_prefixes(get_main_ref_store(r), get_git_namespace(), data.prefixes.v, - NULL, send_ref, &data); + data.hidden_refs.v, send_ref, &data); packet_fflush(stdout); strvec_clear(&data.prefixes); strbuf_release(&data.buf);