From patchwork Tue Nov 8 10:03:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13036088 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 860C2C433FE for ; Tue, 8 Nov 2022 10:04:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233839AbiKHKES (ORCPT ); Tue, 8 Nov 2022 05:04:18 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39066 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233907AbiKHKDn (ORCPT ); Tue, 8 Nov 2022 05:03:43 -0500 Received: from out4-smtp.messagingengine.com (out4-smtp.messagingengine.com [66.111.4.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 85879114 for ; Tue, 8 Nov 2022 02:03:42 -0800 (PST) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id EB8385C0135; Tue, 8 Nov 2022 05:03:41 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute1.internal (MEProxy); Tue, 08 Nov 2022 05:03:41 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:date:date:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to; s=fm1; t=1667901821; x=1667988221; bh=10amdenMwb Jgt2Il8mg7b7Tr2TgwW6mhrDZrbO6RIf4=; b=SxbQ1Va2jsH3fk6RrQM1aGn1dZ MS+6/+NxfHvDQbunl0vJsseSxU00QOE8kH6zZP4FQw9/CNVhx3qiaZRweAcQ/Q7m 8GjmGcJVFFl39d9qS24f8LDyR2LjQDbWigMu0p1K/lfwh5Z225TTPHNZFypQZ2Ra yGDWl9RFYxSPnh1Gfv9q742A5KOZNd/GVHCC2pQk2+5btrd2/BHb9K+BK2Cmuoen rlO+CnxJSkVJJqUIyFzWTwl+5VS2WBVjqlBCqWOfOoAA5E4lQYfCEe+U6igHQAzo jcCJ8KOZSrC+lj99sGN4B9O7YwMk+9EyoKR5BAQ54sd66ZJzYfgydKjQ6ZEg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:date:date:feedback-id :feedback-id:from:from:in-reply-to:in-reply-to:message-id :mime-version:references:reply-to:sender:subject:subject:to:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1667901821; x=1667988221; bh=10amdenMwbJgt2Il8mg7b7Tr2Tgw W6mhrDZrbO6RIf4=; b=cg1lLE8G38HAnXjgCukUmWNfAYBUOy5+s/hv23hpqBlA QR4Rin48DeoWrhcehuZNwADxGC8gnK7qZI7ecld2i0EGun/ecMnFQ7fAFxVH/TyF WRVFYEA7Tbfem0G8D/IRCXLyK97Z61MrcV8icc/mcpa9BYnBinleJ6DCPMlDDPSw YK0QCMFl/9W0BonPuTTTPlSWXZiIIkx/vEQgwWY6ir9EX15ErmHvFQH7KSzyCpWU n6k1fWb4a4AEpOVXRw1aLMNZNaJNCFs/ANk2UAvCl737Cha55QZ0h+CR7glLD1yS CWbK3QPqcxiRpV/6Y5ePOJTkOiMfB+p2fMLWGvJUgQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvgedrfedtgdduudcutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhepfffhvfevuffkfhggtggujgesghdtre ertddtvdenucfhrhhomheprfgrthhrihgtkhcuufhtvghinhhhrghrughtuceophhssehp khhsrdhimheqnecuggftrfgrthhtvghrnhepueektdevtdffveeljeetgfehheeigeekle duvdeffeeghefgledttdehjeelffetnecuvehluhhsthgvrhfuihiivgeptdenucfrrghr rghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 8 Nov 2022 05:03:40 -0500 (EST) Received: by pks.im (OpenSMTPD) with ESMTPSA id 578cd701 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Tue, 8 Nov 2022 10:03:36 +0000 (UTC) Date: Tue, 8 Nov 2022 11:03:39 +0100 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Junio C Hamano , =?iso-8859-1?q?=C6var_Arnfj=F6r=F0?= Bjarmason , Taylor Blau , Jeff King Subject: [PATCH v4 1/6] refs: get rid of global list of hidden refs Message-ID: <34afe30d60e4dbc7e50a590541440c20c27507a6.1667901510.git.ps@pks.im> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org We're about to add a new argument to git-rev-list(1) that allows it to add all references that are visible when taking `transfer.hideRefs` et al into account. This will require us to potentially parse multiple sets of hidden refs, which is not easily possible right now as there is only a single, global instance of the list of parsed hidden refs. Refactor `parse_hide_refs_config()` and `ref_is_hidden()` so that both take the list of hidden references as input and adjust callers to keep a local list, instead. This allows us to easily use multiple hidden-ref lists. Furthermore, it allows us to properly free this list before we exit. Signed-off-by: Patrick Steinhardt --- builtin/receive-pack.c | 8 +++++--- ls-refs.c | 13 +++++++++---- refs.c | 16 +++++----------- refs.h | 5 +++-- upload-pack.c | 30 ++++++++++++++++++------------ 5 files changed, 40 insertions(+), 32 deletions(-) diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c index 44bcea3a5b..1f3efc58fb 100644 --- a/builtin/receive-pack.c +++ b/builtin/receive-pack.c @@ -80,6 +80,7 @@ static struct object_id push_cert_oid; static struct signature_check sigcheck; static const char *push_cert_nonce; static const char *cert_nonce_seed; +static struct string_list hidden_refs = STRING_LIST_INIT_DUP; static const char *NONCE_UNSOLICITED = "UNSOLICITED"; static const char *NONCE_BAD = "BAD"; @@ -130,7 +131,7 @@ static enum deny_action parse_deny_action(const char *var, const char *value) static int receive_pack_config(const char *var, const char *value, void *cb) { - int status = parse_hide_refs_config(var, value, "receive"); + int status = parse_hide_refs_config(var, value, "receive", &hidden_refs); if (status) return status; @@ -296,7 +297,7 @@ static int show_ref_cb(const char *path_full, const struct object_id *oid, struct oidset *seen = data; const char *path = strip_namespace(path_full); - if (ref_is_hidden(path, path_full)) + if (ref_is_hidden(path, path_full, &hidden_refs)) return 0; /* @@ -1794,7 +1795,7 @@ static void reject_updates_to_hidden(struct command *commands) strbuf_setlen(&refname_full, prefix_len); strbuf_addstr(&refname_full, cmd->ref_name); - if (!ref_is_hidden(cmd->ref_name, refname_full.buf)) + if (!ref_is_hidden(cmd->ref_name, refname_full.buf, &hidden_refs)) continue; if (is_null_oid(&cmd->new_oid)) cmd->error_string = "deny deleting a hidden ref"; @@ -2591,6 +2592,7 @@ int cmd_receive_pack(int argc, const char **argv, const char *prefix) packet_flush(1); oid_array_clear(&shallow); oid_array_clear(&ref); + string_list_clear(&hidden_refs, 1); free((void *)push_cert_nonce); return 0; } diff --git a/ls-refs.c b/ls-refs.c index fa0d01b47c..ae89f850e9 100644 --- a/ls-refs.c +++ b/ls-refs.c @@ -6,6 +6,7 @@ #include "ls-refs.h" #include "pkt-line.h" #include "config.h" +#include "string-list.h" static int config_read; static int advertise_unborn; @@ -73,6 +74,7 @@ struct ls_refs_data { unsigned symrefs; struct strvec prefixes; struct strbuf buf; + struct string_list hidden_refs; unsigned unborn : 1; }; @@ -84,7 +86,7 @@ static int send_ref(const char *refname, const struct object_id *oid, strbuf_reset(&data->buf); - if (ref_is_hidden(refname_nons, refname)) + if (ref_is_hidden(refname_nons, refname, &data->hidden_refs)) return 0; if (!ref_match(&data->prefixes, refname_nons)) @@ -137,14 +139,15 @@ static void send_possibly_unborn_head(struct ls_refs_data *data) } static int ls_refs_config(const char *var, const char *value, - void *data UNUSED) + void *cb_data) { + struct ls_refs_data *data = cb_data; /* * We only serve fetches over v2 for now, so respect only "uploadpack" * config. This may need to eventually be expanded to "receive", but we * don't yet know how that information will be passed to ls-refs. */ - return parse_hide_refs_config(var, value, "uploadpack"); + return parse_hide_refs_config(var, value, "uploadpack", &data->hidden_refs); } int ls_refs(struct repository *r, struct packet_reader *request) @@ -154,9 +157,10 @@ int ls_refs(struct repository *r, struct packet_reader *request) memset(&data, 0, sizeof(data)); strvec_init(&data.prefixes); strbuf_init(&data.buf, 0); + string_list_init_dup(&data.hidden_refs); ensure_config_read(); - git_config(ls_refs_config, NULL); + git_config(ls_refs_config, &data); while (packet_reader_read(request) == PACKET_READ_NORMAL) { const char *arg = request->line; @@ -195,6 +199,7 @@ int ls_refs(struct repository *r, struct packet_reader *request) packet_fflush(stdout); strvec_clear(&data.prefixes); strbuf_release(&data.buf); + string_list_clear(&data.hidden_refs, 1); return 0; } diff --git a/refs.c b/refs.c index 1491ae937e..2c7e88b190 100644 --- a/refs.c +++ b/refs.c @@ -1414,9 +1414,8 @@ char *shorten_unambiguous_ref(const char *refname, int strict) refname, strict); } -static struct string_list *hide_refs; - -int parse_hide_refs_config(const char *var, const char *value, const char *section) +int parse_hide_refs_config(const char *var, const char *value, const char *section, + struct string_list *hide_refs) { const char *key; if (!strcmp("transfer.hiderefs", var) || @@ -1431,21 +1430,16 @@ int parse_hide_refs_config(const char *var, const char *value, const char *secti len = strlen(ref); while (len && ref[len - 1] == '/') ref[--len] = '\0'; - if (!hide_refs) { - CALLOC_ARRAY(hide_refs, 1); - hide_refs->strdup_strings = 1; - } - string_list_append(hide_refs, ref); + string_list_append_nodup(hide_refs, ref); } return 0; } -int ref_is_hidden(const char *refname, const char *refname_full) +int ref_is_hidden(const char *refname, const char *refname_full, + const struct string_list *hide_refs) { int i; - if (!hide_refs) - return 0; for (i = hide_refs->nr - 1; i >= 0; i--) { const char *match = hide_refs->items[i].string; const char *subject; diff --git a/refs.h b/refs.h index 8958717a17..3266fd8f57 100644 --- a/refs.h +++ b/refs.h @@ -808,7 +808,8 @@ int update_ref(const char *msg, const char *refname, const struct object_id *new_oid, const struct object_id *old_oid, unsigned int flags, enum action_on_err onerr); -int parse_hide_refs_config(const char *var, const char *value, const char *); +int parse_hide_refs_config(const char *var, const char *value, const char *, + struct string_list *); /* * Check whether a ref is hidden. If no namespace is set, both the first and @@ -818,7 +819,7 @@ int parse_hide_refs_config(const char *var, const char *value, const char *); * the ref is outside that namespace, the first parameter is NULL. The second * parameter always points to the full ref name. */ -int ref_is_hidden(const char *, const char *); +int ref_is_hidden(const char *, const char *, const struct string_list *); /* Is this a per-worktree ref living in the refs/ namespace? */ int is_per_worktree_ref(const char *refname); diff --git a/upload-pack.c b/upload-pack.c index 0b8311bd68..9db17f8787 100644 --- a/upload-pack.c +++ b/upload-pack.c @@ -62,6 +62,7 @@ struct upload_pack_data { struct object_array have_obj; struct oid_array haves; /* v2 only */ struct string_list wanted_refs; /* v2 only */ + struct string_list hidden_refs; struct object_array shallows; struct string_list deepen_not; @@ -118,6 +119,7 @@ static void upload_pack_data_init(struct upload_pack_data *data) { struct string_list symref = STRING_LIST_INIT_DUP; struct string_list wanted_refs = STRING_LIST_INIT_DUP; + struct string_list hidden_refs = STRING_LIST_INIT_DUP; struct object_array want_obj = OBJECT_ARRAY_INIT; struct object_array have_obj = OBJECT_ARRAY_INIT; struct oid_array haves = OID_ARRAY_INIT; @@ -130,6 +132,7 @@ static void upload_pack_data_init(struct upload_pack_data *data) memset(data, 0, sizeof(*data)); data->symref = symref; data->wanted_refs = wanted_refs; + data->hidden_refs = hidden_refs; data->want_obj = want_obj; data->have_obj = have_obj; data->haves = haves; @@ -151,6 +154,7 @@ static void upload_pack_data_clear(struct upload_pack_data *data) { string_list_clear(&data->symref, 1); string_list_clear(&data->wanted_refs, 1); + string_list_clear(&data->hidden_refs, 1); object_array_clear(&data->want_obj); object_array_clear(&data->have_obj); oid_array_clear(&data->haves); @@ -842,8 +846,8 @@ static void deepen(struct upload_pack_data *data, int depth) * Checking for reachable shallows requires that our refs be * marked with OUR_REF. */ - head_ref_namespaced(check_ref, NULL); - for_each_namespaced_ref(check_ref, NULL); + head_ref_namespaced(check_ref, data); + for_each_namespaced_ref(check_ref, data); get_reachable_list(data, &reachable_shallows); result = get_shallow_commits(&reachable_shallows, @@ -1158,11 +1162,11 @@ static void receive_needs(struct upload_pack_data *data, /* return non-zero if the ref is hidden, otherwise 0 */ static int mark_our_ref(const char *refname, const char *refname_full, - const struct object_id *oid) + const struct object_id *oid, const struct string_list *hidden_refs) { struct object *o = lookup_unknown_object(the_repository, oid); - if (ref_is_hidden(refname, refname_full)) { + if (ref_is_hidden(refname, refname_full, hidden_refs)) { o->flags |= HIDDEN_REF; return 1; } @@ -1171,11 +1175,12 @@ static int mark_our_ref(const char *refname, const char *refname_full, } static int check_ref(const char *refname_full, const struct object_id *oid, - int flag UNUSED, void *cb_data UNUSED) + int flag UNUSED, void *cb_data) { const char *refname = strip_namespace(refname_full); + struct upload_pack_data *data = cb_data; - mark_our_ref(refname, refname_full, oid); + mark_our_ref(refname, refname_full, oid, &data->hidden_refs); return 0; } @@ -1204,7 +1209,7 @@ static int send_ref(const char *refname, const struct object_id *oid, struct object_id peeled; struct upload_pack_data *data = cb_data; - if (mark_our_ref(refname_nons, refname, oid)) + if (mark_our_ref(refname_nons, refname, oid, &data->hidden_refs)) return 0; if (capabilities) { @@ -1327,7 +1332,7 @@ static int upload_pack_config(const char *var, const char *value, void *cb_data) if (parse_object_filter_config(var, value, data) < 0) return -1; - return parse_hide_refs_config(var, value, "uploadpack"); + return parse_hide_refs_config(var, value, "uploadpack", &data->hidden_refs); } static int upload_pack_protected_config(const char *var, const char *value, void *cb_data) @@ -1375,8 +1380,8 @@ void upload_pack(const int advertise_refs, const int stateless_rpc, advertise_shallow_grafts(1); packet_flush(1); } else { - head_ref_namespaced(check_ref, NULL); - for_each_namespaced_ref(check_ref, NULL); + head_ref_namespaced(check_ref, &data); + for_each_namespaced_ref(check_ref, &data); } if (!advertise_refs) { @@ -1441,6 +1446,7 @@ static int parse_want(struct packet_writer *writer, const char *line, static int parse_want_ref(struct packet_writer *writer, const char *line, struct string_list *wanted_refs, + struct string_list *hidden_refs, struct object_array *want_obj) { const char *refname_nons; @@ -1451,7 +1457,7 @@ static int parse_want_ref(struct packet_writer *writer, const char *line, struct strbuf refname = STRBUF_INIT; strbuf_addf(&refname, "%s%s", get_git_namespace(), refname_nons); - if (ref_is_hidden(refname_nons, refname.buf) || + if (ref_is_hidden(refname_nons, refname.buf, hidden_refs) || read_ref(refname.buf, &oid)) { packet_writer_error(writer, "unknown ref %s", refname_nons); die("unknown ref %s", refname_nons); @@ -1508,7 +1514,7 @@ static void process_args(struct packet_reader *request, continue; if (data->allow_ref_in_want && parse_want_ref(&data->writer, arg, &data->wanted_refs, - &data->want_obj)) + &data->hidden_refs, &data->want_obj)) continue; /* process have line */ if (parse_have(arg, &data->haves)) From patchwork Tue Nov 8 10:03:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13036091 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DACE2C43219 for ; Tue, 8 Nov 2022 10:04:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233849AbiKHKEV (ORCPT ); Tue, 8 Nov 2022 05:04:21 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39080 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233832AbiKHKDt (ORCPT ); Tue, 8 Nov 2022 05:03:49 -0500 Received: from out4-smtp.messagingengine.com (out4-smtp.messagingengine.com [66.111.4.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A44B2EE37 for ; Tue, 8 Nov 2022 02:03:47 -0800 (PST) Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailout.nyi.internal (Postfix) with ESMTP id 15D0D5C004E; Tue, 8 Nov 2022 05:03:47 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute2.internal (MEProxy); Tue, 08 Nov 2022 05:03:47 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:date:date:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to; s=fm1; t=1667901827; x=1667988227; bh=MfWHNTQvab 13DOt9cL/Ji73gT0H1+wBgpeNrG1eeZiU=; b=Bvy74rRoR4TCMSHj0X1+jR+sup /jb/s2uvJQLjZ5w3VIurlpbPawtmjdxUunqLiOECANSLOx08zoxNqi77warddTuU NdQ4npXS5WkroxdP2B5+eNDE6KS7B0ExQZezGlWhLNkcFRE3i7cxdAvhx4nfLwAz FtIrKX3xyBGsUOB+UOXvSE/LNajnL7pKSwMbNU0crd/0hq6SL4NQzkhSu4lqMRN4 SCyoGqex3y0/bCJO6W8YeI30rpEvsULB6vwiT+h+DHotpfo6vC8HZ2FPJ7orI7JC BnCQw9Nbx8Sz/t2ha5Ca07rpOAMS/3YpXG4SyWylE8uMrWPT+lSr9r74b5oQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:date:date:feedback-id :feedback-id:from:from:in-reply-to:in-reply-to:message-id :mime-version:references:reply-to:sender:subject:subject:to:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1667901827; x=1667988227; bh=MfWHNTQvab13DOt9cL/Ji73gT0H1 +wBgpeNrG1eeZiU=; b=uyhJqXLJmyOpV+Ltm8/J1hdKOPe+Q9z9izR+qdzXpcWo xxcx5vb8P1ia3iaWMfMZM9Ywz2/z2aEpxxwOF+JPXGxG+tDybQMNWzgV3sUa3Sbw WIKL7oJdXPSLD8NwKxUMCH3szyo12/dzAjTgZ+rHd1Rpx/Q8j8kUrCPn6SZXzPDO 0Q7aIWqaEZsh3bjCB1xOQEzmg0qAUo5jOZgMDQOwOl85n7+5nVlfuhFkRdzi2zBG J6+oftCreg3qjb0I9nmsOs8K9U47FSCzkuQ1xTWL7Ek9SroTwQN/QpECxLfDlQNh nVycqialzrp+VZ7aYmueGIdsPgnHUpNLt3qIaZRzSQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvgedrfedtgdduvdcutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhepfffhvfevuffkfhggtggujgesghdtre ertddtvdenucfhrhhomheprfgrthhrihgtkhcuufhtvghinhhhrghrughtuceophhssehp khhsrdhimheqnecuggftrfgrthhtvghrnhepueektdevtdffveeljeetgfehheeigeekle duvdeffeeghefgledttdehjeelffetnecuvehluhhsthgvrhfuihiivgeptdenucfrrghr rghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 8 Nov 2022 05:03:45 -0500 (EST) Received: by pks.im (OpenSMTPD) with ESMTPSA id 94ff17ae (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Tue, 8 Nov 2022 10:03:40 +0000 (UTC) Date: Tue, 8 Nov 2022 11:03:43 +0100 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Junio C Hamano , =?iso-8859-1?q?=C6var_Arnfj=F6r=F0?= Bjarmason , Taylor Blau , Jeff King Subject: [PATCH v4 2/6] revision: move together exclusion-related functions Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Move together the definitions of functions that handle exclusions of refs so that related functionality sits in a single place, only. Signed-off-by: Patrick Steinhardt --- revision.c | 52 ++++++++++++++++++++++++++-------------------------- 1 file changed, 26 insertions(+), 26 deletions(-) diff --git a/revision.c b/revision.c index 0760e78936..be755670e2 100644 --- a/revision.c +++ b/revision.c @@ -1517,14 +1517,6 @@ static void add_rev_cmdline_list(struct rev_info *revs, } } -struct all_refs_cb { - int all_flags; - int warned_bad_reflog; - struct rev_info *all_revs; - const char *name_for_errormsg; - struct worktree *wt; -}; - int ref_excluded(struct string_list *ref_excludes, const char *path) { struct string_list_item *item; @@ -1538,6 +1530,32 @@ int ref_excluded(struct string_list *ref_excludes, const char *path) return 0; } +void clear_ref_exclusion(struct string_list **ref_excludes_p) +{ + if (*ref_excludes_p) { + string_list_clear(*ref_excludes_p, 0); + free(*ref_excludes_p); + } + *ref_excludes_p = NULL; +} + +void add_ref_exclusion(struct string_list **ref_excludes_p, const char *exclude) +{ + if (!*ref_excludes_p) { + CALLOC_ARRAY(*ref_excludes_p, 1); + (*ref_excludes_p)->strdup_strings = 1; + } + string_list_append(*ref_excludes_p, exclude); +} + +struct all_refs_cb { + int all_flags; + int warned_bad_reflog; + struct rev_info *all_revs; + const char *name_for_errormsg; + struct worktree *wt; +}; + static int handle_one_ref(const char *path, const struct object_id *oid, int flag UNUSED, void *cb_data) @@ -1563,24 +1581,6 @@ static void init_all_refs_cb(struct all_refs_cb *cb, struct rev_info *revs, cb->wt = NULL; } -void clear_ref_exclusion(struct string_list **ref_excludes_p) -{ - if (*ref_excludes_p) { - string_list_clear(*ref_excludes_p, 0); - free(*ref_excludes_p); - } - *ref_excludes_p = NULL; -} - -void add_ref_exclusion(struct string_list **ref_excludes_p, const char *exclude) -{ - if (!*ref_excludes_p) { - CALLOC_ARRAY(*ref_excludes_p, 1); - (*ref_excludes_p)->strdup_strings = 1; - } - string_list_append(*ref_excludes_p, exclude); -} - static void handle_refs(struct ref_store *refs, struct rev_info *revs, unsigned flags, int (*for_each)(struct ref_store *, each_ref_fn, void *)) From patchwork Tue Nov 8 10:03:47 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13036090 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 12D8AC43217 for ; Tue, 8 Nov 2022 10:04:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233851AbiKHKEX (ORCPT ); Tue, 8 Nov 2022 05:04:23 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39086 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233835AbiKHKDw (ORCPT ); Tue, 8 Nov 2022 05:03:52 -0500 Received: from out4-smtp.messagingengine.com (out4-smtp.messagingengine.com [66.111.4.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 48C78A8 for ; Tue, 8 Nov 2022 02:03:51 -0800 (PST) Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailout.nyi.internal (Postfix) with ESMTP id AA35A5C018F; Tue, 8 Nov 2022 05:03:50 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute2.internal (MEProxy); Tue, 08 Nov 2022 05:03:50 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:date:date:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to; s=fm1; t=1667901830; x=1667988230; bh=N8n5aoTuzg oG9iiPjZve5lCQaOlAX3sB3BPsMAbYhcE=; b=WjY9QS4f/3UNoSI/PQq18snqrc 1MuJXJF4vYFpn09nRAT0buqjSlNtaRhoI+HjYbGN9dhO8XqOrYV6s7uPyaCxI6R8 YXkowBXB6iXhYDD+PVbuvuiWGBHVBW5hoaKtMONUOAm4zgbK6niS9PDG/5nOI0sG XLKlYDbMOoiFjDc9pcYGNUE+XSjOy83tsko5YMKsQgE5Dp9nzvgjs694AOMZtTFR lpw82iSXijVowy+sQiQJ055upRtT5MqvArC+e8MCHS2BbmG+6vJAWABULSFzoBlu verdCdlBZ2supr7hqUcVeyoGtG5XY4os9YIQ0xPMW67V5DpeU3KvRbklAnUw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:date:date:feedback-id :feedback-id:from:from:in-reply-to:in-reply-to:message-id :mime-version:references:reply-to:sender:subject:subject:to:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1667901830; x=1667988230; bh=N8n5aoTuzgoG9iiPjZve5lCQaOlA X3sB3BPsMAbYhcE=; b=gaNHnv4jeDw6+s9C4T2M+m/stX+z65TZ2a/oDOX/WfWC tCPt5vu6CG9Ex6BjK8yCGqCnHXEOG1XB1RQVQ37B0CIPyQtVVgMF1pkimmPuXEDS KlL713kxZvqXL5Ca7BELWqg9JGwNgnxnKIdLvKFPW/BBYFvCeIkeArHCSMDF+waE vtjYJleHi267PvQdqVntm0Wbaf75CN/k25j/YyJFrXDxr/knBBgH+d3onzo8TmDk X+jtkV4c9+nOU1rUSXGaOdBQmYWqKQIfMHNvV3jmGrAlMUkq9/mWlLnqNdToQG6n HfeqlxAl9bds79Gu6+f1NtDEMKmrXr0CmqaBTtHIGg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvgedrfedtgdduvdcutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhepfffhvfevuffkfhggtggujgesghdtre ertddtvdenucfhrhhomheprfgrthhrihgtkhcuufhtvghinhhhrghrughtuceophhssehp khhsrdhimheqnecuggftrfgrthhtvghrnhepueektdevtdffveeljeetgfehheeigeekle duvdeffeeghefgledttdehjeelffetnecuvehluhhsthgvrhfuihiivgepudenucfrrghr rghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 8 Nov 2022 05:03:49 -0500 (EST) Received: by pks.im (OpenSMTPD) with ESMTPSA id 813c824a (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Tue, 8 Nov 2022 10:03:44 +0000 (UTC) Date: Tue, 8 Nov 2022 11:03:47 +0100 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Junio C Hamano , =?iso-8859-1?q?=C6var_Arnfj=F6r=F0?= Bjarmason , Taylor Blau , Jeff King Subject: [PATCH v4 3/6] revision: introduce struct to handle exclusions Message-ID: <265b292ed5c2de19b7118dfe046d3d9d932e2e89.1667901510.git.ps@pks.im> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org The functions that handle exclusion of refs work on a single string list. We're about to add a second mechanism for excluding refs though, and it makes sense to reuse much of the same architecture for both kinds of exclusion. Introduce a new `struct ref_exclusions` that encapsulates all the logic related to excluding refs and move the `struct string_list` that holds all wildmatch patterns of excluded refs into it. Rename functions that operate on this struct to match its name. Signed-off-by: Patrick Steinhardt --- builtin/rev-parse.c | 8 ++++---- revision.c | 48 +++++++++++++++++++++------------------------ revision.h | 27 +++++++++++++++++++------ 3 files changed, 47 insertions(+), 36 deletions(-) diff --git a/builtin/rev-parse.c b/builtin/rev-parse.c index 8f61050bde..7fa5b6991b 100644 --- a/builtin/rev-parse.c +++ b/builtin/rev-parse.c @@ -39,7 +39,7 @@ static int abbrev_ref_strict; static int output_sq; static int stuck_long; -static struct string_list *ref_excludes; +static struct ref_exclusions ref_excludes = REF_EXCLUSIONS_INIT; /* * Some arguments are relevant "revision" arguments, @@ -198,7 +198,7 @@ static int show_default(void) static int show_reference(const char *refname, const struct object_id *oid, int flag UNUSED, void *cb_data UNUSED) { - if (ref_excluded(ref_excludes, refname)) + if (ref_excluded(&ref_excludes, refname)) return 0; show_rev(NORMAL, oid, refname); return 0; @@ -585,7 +585,7 @@ static void handle_ref_opt(const char *pattern, const char *prefix) for_each_glob_ref_in(show_reference, pattern, prefix, NULL); else for_each_ref_in(prefix, show_reference, NULL); - clear_ref_exclusion(&ref_excludes); + clear_ref_exclusions(&ref_excludes); } enum format_type { @@ -863,7 +863,7 @@ int cmd_rev_parse(int argc, const char **argv, const char *prefix) } if (!strcmp(arg, "--all")) { for_each_ref(show_reference, NULL); - clear_ref_exclusion(&ref_excludes); + clear_ref_exclusions(&ref_excludes); continue; } if (skip_prefix(arg, "--disambiguate=", &arg)) { diff --git a/revision.c b/revision.c index be755670e2..fe3ec98f46 100644 --- a/revision.c +++ b/revision.c @@ -1517,35 +1517,30 @@ static void add_rev_cmdline_list(struct rev_info *revs, } } -int ref_excluded(struct string_list *ref_excludes, const char *path) +int ref_excluded(const struct ref_exclusions *exclusions, const char *path) { struct string_list_item *item; - - if (!ref_excludes) - return 0; - for_each_string_list_item(item, ref_excludes) { + for_each_string_list_item(item, &exclusions->excluded_refs) { if (!wildmatch(item->string, path, 0)) return 1; } return 0; } -void clear_ref_exclusion(struct string_list **ref_excludes_p) +void init_ref_exclusions(struct ref_exclusions *exclusions) { - if (*ref_excludes_p) { - string_list_clear(*ref_excludes_p, 0); - free(*ref_excludes_p); - } - *ref_excludes_p = NULL; + struct ref_exclusions blank = REF_EXCLUSIONS_INIT; + memcpy(exclusions, &blank, sizeof(*exclusions)); } -void add_ref_exclusion(struct string_list **ref_excludes_p, const char *exclude) +void clear_ref_exclusions(struct ref_exclusions *exclusions) { - if (!*ref_excludes_p) { - CALLOC_ARRAY(*ref_excludes_p, 1); - (*ref_excludes_p)->strdup_strings = 1; - } - string_list_append(*ref_excludes_p, exclude); + string_list_clear(&exclusions->excluded_refs, 0); +} + +void add_ref_exclusion(struct ref_exclusions *exclusions, const char *exclude) +{ + string_list_append(&exclusions->excluded_refs, exclude); } struct all_refs_cb { @@ -1563,7 +1558,7 @@ static int handle_one_ref(const char *path, const struct object_id *oid, struct all_refs_cb *cb = cb_data; struct object *object; - if (ref_excluded(cb->all_revs->ref_excludes, path)) + if (ref_excluded(&cb->all_revs->ref_excludes, path)) return 0; object = get_reference(cb->all_revs, path, oid, cb->all_flags); @@ -1901,6 +1896,7 @@ void repo_init_revisions(struct repository *r, init_display_notes(&revs->notes_opt); list_objects_filter_init(&revs->filter); + init_ref_exclusions(&revs->ref_excludes); } static void add_pending_commit_list(struct rev_info *revs, @@ -2689,10 +2685,10 @@ static int handle_revision_pseudo_opt(struct rev_info *revs, init_all_refs_cb(&cb, revs, *flags); other_head_refs(handle_one_ref, &cb); } - clear_ref_exclusion(&revs->ref_excludes); + clear_ref_exclusions(&revs->ref_excludes); } else if (!strcmp(arg, "--branches")) { handle_refs(refs, revs, *flags, refs_for_each_branch_ref); - clear_ref_exclusion(&revs->ref_excludes); + clear_ref_exclusions(&revs->ref_excludes); } else if (!strcmp(arg, "--bisect")) { read_bisect_terms(&term_bad, &term_good); handle_refs(refs, revs, *flags, for_each_bad_bisect_ref); @@ -2701,15 +2697,15 @@ static int handle_revision_pseudo_opt(struct rev_info *revs, revs->bisect = 1; } else if (!strcmp(arg, "--tags")) { handle_refs(refs, revs, *flags, refs_for_each_tag_ref); - clear_ref_exclusion(&revs->ref_excludes); + clear_ref_exclusions(&revs->ref_excludes); } else if (!strcmp(arg, "--remotes")) { handle_refs(refs, revs, *flags, refs_for_each_remote_ref); - clear_ref_exclusion(&revs->ref_excludes); + clear_ref_exclusions(&revs->ref_excludes); } else if ((argcount = parse_long_opt("glob", argv, &optarg))) { struct all_refs_cb cb; init_all_refs_cb(&cb, revs, *flags); for_each_glob_ref(handle_one_ref, optarg, &cb); - clear_ref_exclusion(&revs->ref_excludes); + clear_ref_exclusions(&revs->ref_excludes); return argcount; } else if ((argcount = parse_long_opt("exclude", argv, &optarg))) { add_ref_exclusion(&revs->ref_excludes, optarg); @@ -2718,17 +2714,17 @@ static int handle_revision_pseudo_opt(struct rev_info *revs, struct all_refs_cb cb; init_all_refs_cb(&cb, revs, *flags); for_each_glob_ref_in(handle_one_ref, optarg, "refs/heads/", &cb); - clear_ref_exclusion(&revs->ref_excludes); + clear_ref_exclusions(&revs->ref_excludes); } else if (skip_prefix(arg, "--tags=", &optarg)) { struct all_refs_cb cb; init_all_refs_cb(&cb, revs, *flags); for_each_glob_ref_in(handle_one_ref, optarg, "refs/tags/", &cb); - clear_ref_exclusion(&revs->ref_excludes); + clear_ref_exclusions(&revs->ref_excludes); } else if (skip_prefix(arg, "--remotes=", &optarg)) { struct all_refs_cb cb; init_all_refs_cb(&cb, revs, *flags); for_each_glob_ref_in(handle_one_ref, optarg, "refs/remotes/", &cb); - clear_ref_exclusion(&revs->ref_excludes); + clear_ref_exclusions(&revs->ref_excludes); } else if (!strcmp(arg, "--reflog")) { add_reflogs_to_pending(revs, *flags); } else if (!strcmp(arg, "--indexed-objects")) { diff --git a/revision.h b/revision.h index afe1b77985..5c8ab16047 100644 --- a/revision.h +++ b/revision.h @@ -81,6 +81,21 @@ struct rev_cmdline_info { } *rev; }; +struct ref_exclusions { + /* + * Excluded refs is a list of wildmatch patterns. If any of the + * patterns matches, the reference will be excluded. + */ + struct string_list excluded_refs; +}; + +/** + * Initialize a `struct ref_exclusions` with a macro. + */ +#define REF_EXCLUSIONS_INIT { \ + .excluded_refs = STRING_LIST_INIT_DUP, \ +} + struct oidset; struct topo_walk_info; @@ -103,7 +118,7 @@ struct rev_info { struct list_objects_filter_options filter; /* excluding from --branches, --refs, etc. expansion */ - struct string_list *ref_excludes; + struct ref_exclusions ref_excludes; /* Basic information */ const char *prefix; @@ -439,12 +454,12 @@ void mark_trees_uninteresting_sparse(struct repository *r, struct oidset *trees) void show_object_with_name(FILE *, struct object *, const char *); /** - * Helpers to check if a "struct string_list" item matches with - * wildmatch(). + * Helpers to check if a reference should be excluded. */ -int ref_excluded(struct string_list *, const char *path); -void clear_ref_exclusion(struct string_list **); -void add_ref_exclusion(struct string_list **, const char *exclude); +int ref_excluded(const struct ref_exclusions *exclusions, const char *path); +void init_ref_exclusions(struct ref_exclusions *); +void clear_ref_exclusions(struct ref_exclusions *); +void add_ref_exclusion(struct ref_exclusions *, const char *exclude); /** * This function can be used if you want to add commit objects as revision From patchwork Tue Nov 8 10:03:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13036092 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 23562C4167B for ; Tue, 8 Nov 2022 10:04:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233854AbiKHKEZ (ORCPT ); Tue, 8 Nov 2022 05:04:25 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39110 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233838AbiKHKD4 (ORCPT ); Tue, 8 Nov 2022 05:03:56 -0500 Received: from out4-smtp.messagingengine.com (out4-smtp.messagingengine.com [66.111.4.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A7FD817AB8 for ; Tue, 8 Nov 2022 02:03:55 -0800 (PST) Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailout.nyi.internal (Postfix) with ESMTP id 1873A5C0135; Tue, 8 Nov 2022 05:03:55 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute2.internal (MEProxy); Tue, 08 Nov 2022 05:03:55 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:date:date:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to; s=fm1; t=1667901835; x=1667988235; bh=SIXamLHN18 Z2W0UoAZn8tvn3XoSZIzXUcVGjZVXKAew=; b=C/h3hI5ZD0lZCudjhZnVbwnmV3 llVXGfb/dP3RzYvLaCnilIJysAWJ1BVzYAWzRxnKbL6hBMJ5F3MAxzKtsWoB6lPn LDRZAgWocDPOm8/qKf2ECpxdWl5uvRCrvlzofmHZvbdJdNxW5QjceZF4KCsAhz4y CPw42qdJOrpwNQ4WtPKSDuG7K7tg/lBDtd7WgxkRU2RBWws+QaxU4PtPUioxKQDn e7F/D0O/8jDWRlJXoUWun8vWBfp82xkaUgcmkXAPotO0Lm2XtumE/o/5KZsuAXkc XNCGy7fgjH5OWqsFE9D3DKcw4soyHlzA0JfKwbNYnWacfeAvf71heyIMVLyg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:date:date:feedback-id :feedback-id:from:from:in-reply-to:in-reply-to:message-id :mime-version:references:reply-to:sender:subject:subject:to:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1667901835; x=1667988235; bh=SIXamLHN18Z2W0UoAZn8tvn3XoSZ IzXUcVGjZVXKAew=; b=F+aFFP4bn48+WxtE0cVmqeDwsRhh/FbdBDL4FoSxX5vV 5c6NSSOhIXMcf/Z33bI0cNVMav/vXuOHVXEdsKdysHx6OY9Q18bmNQ1suzG/PfFC wFaJbIAFTCO2fNu5Ws3vyNBG4LApShUpy34OVCgeADkkdq7Ml0U7VKf+xDmWZYH/ Fog7JUHLz44/P1XAZ9B+Aa5KK1wjN9TOhxEujR7nX3DZessfhf0CILJaOdK5+kEo 3k+/jy/TBxBvedEBWA0ByPsrCc6dXDpnLkEaOnrhYMPHkVmJx4caYs07UfnWzZkn fG6XKmBlJFdrHRs7d1k1OYSmNQEb1AkuIgxjPxiQow== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvgedrfedtgdduvdcutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhepfffhvfevuffkfhggtggujgesghdtre ertddtvdenucfhrhhomheprfgrthhrihgtkhcuufhtvghinhhhrghrughtuceophhssehp khhsrdhimheqnecuggftrfgrthhtvghrnhepueektdevtdffveeljeetgfehheeigeekle duvdeffeeghefgledttdehjeelffetnecuvehluhhsthgvrhfuihiivgepvdenucfrrghr rghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 8 Nov 2022 05:03:53 -0500 (EST) Received: by pks.im (OpenSMTPD) with ESMTPSA id 02d3bc36 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Tue, 8 Nov 2022 10:03:48 +0000 (UTC) Date: Tue, 8 Nov 2022 11:03:51 +0100 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Junio C Hamano , =?iso-8859-1?q?=C6var_Arnfj=F6r=F0?= Bjarmason , Taylor Blau , Jeff King Subject: [PATCH v4 4/6] revision: add new parameter to exclude hidden refs Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Users can optionally hide refs from remote users in git-upload-pack(1), git-receive-pack(1) and others via the `transfer.hideRefs`, but there is not an easy way to obtain the list of all visible or hidden refs right now. We'll require just that though for a performance improvement in our connectivity check. Add a new option `--exclude-hidden=` that excludes any hidden refs from the next pseudo-ref like `--all` or `--branches`. Signed-off-by: Patrick Steinhardt --- Documentation/rev-list-options.txt | 7 ++ builtin/rev-list.c | 1 + revision.c | 41 ++++++++- revision.h | 9 ++ t/t6021-rev-list-exclude-hidden.sh | 137 +++++++++++++++++++++++++++++ 5 files changed, 194 insertions(+), 1 deletion(-) create mode 100755 t/t6021-rev-list-exclude-hidden.sh diff --git a/Documentation/rev-list-options.txt b/Documentation/rev-list-options.txt index 1837509566..5b46781b35 100644 --- a/Documentation/rev-list-options.txt +++ b/Documentation/rev-list-options.txt @@ -195,6 +195,13 @@ respectively, and they must begin with `refs/` when applied to `--glob` or `--all`. If a trailing '/{asterisk}' is intended, it must be given explicitly. +--exclude-hidden=[receive|uploadpack]:: + Do not include refs that have been hidden via either one of + `receive.hideRefs` or `uploadpack.hideRefs` (see linkgit:git-config[1]) + that the next `--all`, `--branches`, `--tags`, `--remotes` or `--glob` + would otherwise consider. This option is cleared when seeing one of + these pseudo-refs. + --reflog:: Pretend as if all objects mentioned by reflogs are listed on the command line as ``. diff --git a/builtin/rev-list.c b/builtin/rev-list.c index 3acd93f71e..d42db0b0cc 100644 --- a/builtin/rev-list.c +++ b/builtin/rev-list.c @@ -38,6 +38,7 @@ static const char rev_list_usage[] = " --tags\n" " --remotes\n" " --stdin\n" +" --exclude-hidden=[receive|uploadpack]\n" " --quiet\n" " ordering output:\n" " --topo-order\n" diff --git a/revision.c b/revision.c index fe3ec98f46..b726cfd255 100644 --- a/revision.c +++ b/revision.c @@ -1,4 +1,5 @@ #include "cache.h" +#include "config.h" #include "object-store.h" #include "tag.h" #include "blob.h" @@ -1519,11 +1520,17 @@ static void add_rev_cmdline_list(struct rev_info *revs, int ref_excluded(const struct ref_exclusions *exclusions, const char *path) { + const char *stripped_path = strip_namespace(path); struct string_list_item *item; + for_each_string_list_item(item, &exclusions->excluded_refs) { if (!wildmatch(item->string, path, 0)) return 1; } + + if (ref_is_hidden(stripped_path, path, &exclusions->hidden_refs)) + return 1; + return 0; } @@ -1536,6 +1543,7 @@ void init_ref_exclusions(struct ref_exclusions *exclusions) void clear_ref_exclusions(struct ref_exclusions *exclusions) { string_list_clear(&exclusions->excluded_refs, 0); + string_list_clear(&exclusions->hidden_refs, 0); } void add_ref_exclusion(struct ref_exclusions *exclusions, const char *exclude) @@ -1543,6 +1551,34 @@ void add_ref_exclusion(struct ref_exclusions *exclusions, const char *exclude) string_list_append(&exclusions->excluded_refs, exclude); } +struct exclude_hidden_refs_cb { + struct ref_exclusions *exclusions; + const char *section; +}; + +static int hide_refs_config(const char *var, const char *value, void *cb_data) +{ + struct exclude_hidden_refs_cb *cb = cb_data; + return parse_hide_refs_config(var, value, cb->section, + &cb->exclusions->hidden_refs); +} + +void exclude_hidden_refs(struct ref_exclusions *exclusions, const char *section) +{ + struct exclude_hidden_refs_cb cb; + + if (strcmp(section, "receive") && strcmp(section, "uploadpack")) + die(_("unsupported section for hidden refs: %s"), section); + + if (exclusions->hidden_refs.nr) + die(_("--exclude-hidden= passed more than once")); + + cb.exclusions = exclusions; + cb.section = section; + + git_config(hide_refs_config, &cb); +} + struct all_refs_cb { int all_flags; int warned_bad_reflog; @@ -2221,7 +2257,7 @@ static int handle_revision_opt(struct rev_info *revs, int argc, const char **arg !strcmp(arg, "--bisect") || starts_with(arg, "--glob=") || !strcmp(arg, "--indexed-objects") || !strcmp(arg, "--alternate-refs") || - starts_with(arg, "--exclude=") || + starts_with(arg, "--exclude=") || starts_with(arg, "--exclude-hidden=") || starts_with(arg, "--branches=") || starts_with(arg, "--tags=") || starts_with(arg, "--remotes=") || starts_with(arg, "--no-walk=")) { @@ -2710,6 +2746,9 @@ static int handle_revision_pseudo_opt(struct rev_info *revs, } else if ((argcount = parse_long_opt("exclude", argv, &optarg))) { add_ref_exclusion(&revs->ref_excludes, optarg); return argcount; + } else if ((argcount = parse_long_opt("exclude-hidden", argv, &optarg))) { + exclude_hidden_refs(&revs->ref_excludes, optarg); + return argcount; } else if (skip_prefix(arg, "--branches=", &optarg)) { struct all_refs_cb cb; init_all_refs_cb(&cb, revs, *flags); diff --git a/revision.h b/revision.h index 5c8ab16047..a96fefebf1 100644 --- a/revision.h +++ b/revision.h @@ -87,6 +87,12 @@ struct ref_exclusions { * patterns matches, the reference will be excluded. */ struct string_list excluded_refs; + + /* + * Hidden refs is a list of patterns that is to be hidden via + * `ref_is_hidden()`. + */ + struct string_list hidden_refs; }; /** @@ -94,6 +100,7 @@ struct ref_exclusions { */ #define REF_EXCLUSIONS_INIT { \ .excluded_refs = STRING_LIST_INIT_DUP, \ + .hidden_refs = STRING_LIST_INIT_DUP, \ } struct oidset; @@ -456,10 +463,12 @@ void show_object_with_name(FILE *, struct object *, const char *); /** * Helpers to check if a reference should be excluded. */ + int ref_excluded(const struct ref_exclusions *exclusions, const char *path); void init_ref_exclusions(struct ref_exclusions *); void clear_ref_exclusions(struct ref_exclusions *); void add_ref_exclusion(struct ref_exclusions *, const char *exclude); +void exclude_hidden_refs(struct ref_exclusions *, const char *section); /** * This function can be used if you want to add commit objects as revision diff --git a/t/t6021-rev-list-exclude-hidden.sh b/t/t6021-rev-list-exclude-hidden.sh new file mode 100755 index 0000000000..4ab50e7f4f --- /dev/null +++ b/t/t6021-rev-list-exclude-hidden.sh @@ -0,0 +1,137 @@ +#!/bin/sh + +test_description='git rev-list --exclude-hidden test' + +. ./test-lib.sh + +test_expect_success 'setup' ' + test_commit_bulk --id=commit --ref=refs/heads/main 1 && + COMMIT=$(git rev-parse refs/heads/main) && + test_commit_bulk --id=tag --ref=refs/tags/lightweight 1 && + TAG=$(git rev-parse refs/tags/lightweight) && + test_commit_bulk --id=hidden --ref=refs/hidden/commit 1 && + HIDDEN=$(git rev-parse refs/hidden/commit) && + test_commit_bulk --id=namespace --ref=refs/namespaces/namespace/refs/namespaced/commit 1 && + NAMESPACE=$(git rev-parse refs/namespaces/namespace/refs/namespaced/commit) +' + +test_expect_success 'invalid section' ' + echo "fatal: unsupported section for hidden refs: unsupported" >expected && + test_must_fail git rev-list --exclude-hidden=unsupported 2>err && + test_cmp expected err +' + +for section in receive uploadpack +do + test_expect_success "$section: passed multiple times" ' + echo "fatal: --exclude-hidden= passed more than once" >expected && + test_must_fail git -c transfer.hideRefs=refs/hidden/ rev-list --exclude-hidden=$section --exclude-hidden=$section 2>err && + test_cmp expected err + ' + + test_expect_success "$section: without hiddenRefs" ' + git rev-list --exclude-hidden=$section --all >out && + cat >expected <<-EOF && + $NAMESPACE + $HIDDEN + $TAG + $COMMIT + EOF + test_cmp expected out + ' + + test_expect_success "$section: hidden via transfer.hideRefs" ' + git -c transfer.hideRefs=refs/hidden/ rev-list --exclude-hidden=$section --all >out && + cat >expected <<-EOF && + $NAMESPACE + $TAG + $COMMIT + EOF + test_cmp expected out + ' + + test_expect_success "$section: hidden via $section.hideRefs" ' + git -c $section.hideRefs=refs/hidden/ rev-list --exclude-hidden=$section --all >out && + cat >expected <<-EOF && + $NAMESPACE + $TAG + $COMMIT + EOF + test_cmp expected out + ' + + test_expect_success "$section: respects both transfer.hideRefs and $section.hideRefs" ' + git -c transfer.hideRefs=refs/tags/ -c $section.hideRefs=refs/hidden/ rev-list --exclude-hidden=$section --all >out && + cat >expected <<-EOF && + $NAMESPACE + $COMMIT + EOF + test_cmp expected out + ' + + test_expect_success "$section: negation without hidden refs marks everything as uninteresting" ' + git rev-list --all --exclude-hidden=$section --not --all >out && + test_must_be_empty out + ' + + test_expect_success "$section: negation with hidden refs marks them as interesting" ' + git -c transfer.hideRefs=refs/hidden/ rev-list --all --exclude-hidden=$section --not --all >out && + cat >expected <<-EOF && + $HIDDEN + EOF + test_cmp expected out + ' + + test_expect_success "$section: hidden refs and excludes work together" ' + git -c transfer.hideRefs=refs/hidden/ rev-list --exclude=refs/tags/* --exclude-hidden=$section --all >out && + cat >expected <<-EOF && + $NAMESPACE + $COMMIT + EOF + test_cmp expected out + ' + + test_expect_success "$section: excluded hidden refs get reset" ' + git -c transfer.hideRefs=refs/ rev-list --exclude-hidden=$section --all --all >out && + cat >expected <<-EOF && + $NAMESPACE + $HIDDEN + $TAG + $COMMIT + EOF + test_cmp expected out + ' + + test_expect_success "$section: operates on stripped refs by default" ' + GIT_NAMESPACE=namespace git -c transfer.hideRefs=refs/namespaced/ rev-list --exclude-hidden=$section --all >out && + cat >expected <<-EOF && + $HIDDEN + $TAG + $COMMIT + EOF + test_cmp expected out + ' + + test_expect_success "$section: does not hide namespace by default" ' + GIT_NAMESPACE=namespace git -c transfer.hideRefs=refs/namespaces/namespace/ rev-list --exclude-hidden=$section --all >out && + cat >expected <<-EOF && + $NAMESPACE + $HIDDEN + $TAG + $COMMIT + EOF + test_cmp expected out + ' + + test_expect_success "$section: can operate on unstripped refs" ' + GIT_NAMESPACE=namespace git -c transfer.hideRefs=^refs/namespaces/namespace/ rev-list --exclude-hidden=$section --all >out && + cat >expected <<-EOF && + $HIDDEN + $TAG + $COMMIT + EOF + test_cmp expected out + ' +done + +test_done From patchwork Tue Nov 8 10:03:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13036089 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 35A93C4321E for ; Tue, 8 Nov 2022 10:04:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233881AbiKHKE1 (ORCPT ); Tue, 8 Nov 2022 05:04:27 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39118 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233840AbiKHKEC (ORCPT ); Tue, 8 Nov 2022 05:04:02 -0500 Received: from out4-smtp.messagingengine.com (out4-smtp.messagingengine.com [66.111.4.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2DE57A8 for ; Tue, 8 Nov 2022 02:04:02 -0800 (PST) Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.nyi.internal (Postfix) with ESMTP id 919B05C01FC; Tue, 8 Nov 2022 05:04:01 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute3.internal (MEProxy); Tue, 08 Nov 2022 05:04:01 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:date:date:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to; s=fm1; t=1667901841; x=1667988241; bh=cL8Bu8U7IB G0eSW2AzVBwq/WUhaJGIA0pr3e2EVZkdw=; b=bijzGknkUtwlXpM9NcCsU4JsKc MvYFzov++SaiGWQ7fYMrhkyUozzLLbUkA19Rx9LlFwczLzDgbiytylQhJdveri9S FzwndtekGSPsPm8DLSe9yEnouP0jXfKTVJCSe6wVWpfR8C3WeSDdn0AghBdFncVk +ME1PrVlJGQ4D07xdpWVcn6L/tGfU+GQ5nTAjDUKbNL8Q1Znq254HR1roF8cSI9I ZSLwv40UjovYC97JlpHS/KWxaRrmlFSdTyFmras6zH2wsY/lnlZeg42j7nALieYi 5F6DV0FUpPxKxShaqwsf2OHkXZxyWsZEr+bdSECVBVvS76lXLJTQecqlcfCg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:date:date:feedback-id :feedback-id:from:from:in-reply-to:in-reply-to:message-id :mime-version:references:reply-to:sender:subject:subject:to:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1667901841; x=1667988241; bh=cL8Bu8U7IBG0eSW2AzVBwq/WUhaJ GIA0pr3e2EVZkdw=; b=dM3ZqD4PvTqCTMW8eO1EVGvcm3mmFlutaw5/3oOETAsy 5rwOGuU3vjOWa1hs8yTr4NALqKZsVmCLzCEr0KCgYVAX414In/2/Wl2yxAAKl6Hb ttle111uj1aJ7GkXNdO2HLGLIGniG6Rx1YNhhyPCatrK0LIv24AGnr8quC5UCZGD ouGDUpucsqAZOQb0CcTVOfL3Vbs4YrBQvDDxAQKDXClfxmSR6KC4zrXsWJ00R6yZ TjW2FOWfah6Hn6U6v7SVdpKXMxxjXEgcqkTKd9bkgJsnYuBqdIy6o9BXfkwyFZui 6gEE5boOPPb4vTKnc6hLkZ0PnqgCyUJUsbt3pIHbfA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvgedrfedtgdduudcutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhepfffhvfevuffkfhggtggujgesghdtre ertddtvdenucfhrhhomheprfgrthhrihgtkhcuufhtvghinhhhrghrughtuceophhssehp khhsrdhimheqnecuggftrfgrthhtvghrnhepueektdevtdffveeljeetgfehheeigeekle duvdeffeeghefgledttdehjeelffetnecuvehluhhsthgvrhfuihiivgeptdenucfrrghr rghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 8 Nov 2022 05:04:00 -0500 (EST) Received: by pks.im (OpenSMTPD) with ESMTPSA id 6253fbed (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Tue, 8 Nov 2022 10:03:55 +0000 (UTC) Date: Tue, 8 Nov 2022 11:03:58 +0100 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Junio C Hamano , =?iso-8859-1?q?=C6var_Arnfj=F6r=F0?= Bjarmason , Taylor Blau , Jeff King Subject: [PATCH v4 5/6] rev-parse: add `--exclude-hidden=` option Message-ID: <79c5c64a803b67ccfecc57328e565a2fa9563507.1667901510.git.ps@pks.im> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Add a new `--exclude-hidden=` option that is similar to the one we just added to git-rev-list(1). Given a seciton name `uploadpack` or `receive` as argument, it causes us to exclude all references that would be hidden by the respective `$section.hideRefs` configuration. Signed-off-by: Patrick Steinhardt --- Documentation/git-rev-parse.txt | 7 +++++++ builtin/rev-parse.c | 4 ++++ t/t6018-rev-list-glob.sh | 11 +++++++++++ 3 files changed, 22 insertions(+) diff --git a/Documentation/git-rev-parse.txt b/Documentation/git-rev-parse.txt index 6b8ca085aa..393aa6e982 100644 --- a/Documentation/git-rev-parse.txt +++ b/Documentation/git-rev-parse.txt @@ -197,6 +197,13 @@ respectively, and they must begin with `refs/` when applied to `--glob` or `--all`. If a trailing '/{asterisk}' is intended, it must be given explicitly. +--exclude-hidden=[receive|uploadpack]:: + Do not include refs that have been hidden via either one of + `receive.hideRefs` or `uploadpack.hideRefs` (see linkgit:git-config[1]) + that the next `--all`, `--branches`, `--tags`, `--remotes` or `--glob` + would otherwise consider. This option is cleared when seeing one of + these pseudo-refs. + --disambiguate=:: Show every object whose name begins with the given prefix. The must be at least 4 hexadecimal digits long to diff --git a/builtin/rev-parse.c b/builtin/rev-parse.c index 7fa5b6991b..49730c7a23 100644 --- a/builtin/rev-parse.c +++ b/builtin/rev-parse.c @@ -895,6 +895,10 @@ int cmd_rev_parse(int argc, const char **argv, const char *prefix) add_ref_exclusion(&ref_excludes, arg); continue; } + if (skip_prefix(arg, "--exclude-hidden=", &arg)) { + exclude_hidden_refs(&ref_excludes, arg); + continue; + } if (!strcmp(arg, "--show-toplevel")) { const char *work_tree = get_git_work_tree(); if (work_tree) diff --git a/t/t6018-rev-list-glob.sh b/t/t6018-rev-list-glob.sh index e1abc5c2b3..af0c55cbe7 100755 --- a/t/t6018-rev-list-glob.sh +++ b/t/t6018-rev-list-glob.sh @@ -187,6 +187,17 @@ test_expect_success 'rev-parse --exclude=ref with --remotes=glob' ' compare rev-parse "--exclude=upstream/x --remotes=upstream/*" "upstream/one upstream/two" ' +for section in receive uploadpack +do + test_expect_success "rev-parse --exclude-hidden=$section with --all" ' + compare "-c transfer.hideRefs=refs/remotes/ rev-parse" "--exclude-hidden=$section --all" "--branches --tags" + ' + + test_expect_success "rev-parse --exclude-hidden=$section with --all" ' + compare "-c transfer.hideRefs=refs/heads/subspace/ rev-parse" "--exclude-hidden=$section --all" "--exclude=refs/heads/subspace/* --all" + ' +done + test_expect_success 'rev-list --exclude=glob with --branches=glob' ' compare rev-list "--exclude=subspace-* --branches=sub*" "subspace/one subspace/two" ' From patchwork Tue Nov 8 10:04:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13036093 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 46A79C4167E for ; Tue, 8 Nov 2022 10:04:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233892AbiKHKE2 (ORCPT ); Tue, 8 Nov 2022 05:04:28 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39128 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233841AbiKHKEH (ORCPT ); Tue, 8 Nov 2022 05:04:07 -0500 Received: from out4-smtp.messagingengine.com (out4-smtp.messagingengine.com [66.111.4.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 93626FC1 for ; Tue, 8 Nov 2022 02:04:06 -0800 (PST) Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id DC2695C012C; Tue, 8 Nov 2022 05:04:05 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Tue, 08 Nov 2022 05:04:05 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:date:date:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to; s=fm1; t=1667901845; x=1667988245; bh=afgLqkpVyJ Unex8zNJFP9kSC/im9/2FFgJKthaDLQKM=; b=ihoIDpEeTW1pswfIA2e7HYVTfZ ApJG47X/mhWrnyAk8rDRe8mN8xM85tLiSC4TgAjqNnPUZoOPoB3M9yFkPntETnBG //1NDNvWR6vfJLM3tJ7pwpks6bKyEIl2Wf7kerwG00h83JdSaDc81UKDjwckSOr4 2CHtuFRBkRUqieTLUX9v4wDSpDMFkTY2GXi4QjLIvZIMvxgAmgIxKqdwuBXXMeKv yjZi0Vs5aWKTRSn6xal1j3tWx1jwi/kuiZEmanoVNB/BrgV71sLrti6xOYHxduGl YQJRPnnGwpT6O56dcVpWI5v3B2McfVkt1a/lm9gNEmd0lMs4ExrJe6ErSEUw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:date:date:feedback-id :feedback-id:from:from:in-reply-to:in-reply-to:message-id :mime-version:references:reply-to:sender:subject:subject:to:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1667901845; x=1667988245; bh=afgLqkpVyJUnex8zNJFP9kSC/im9 /2FFgJKthaDLQKM=; b=ebHSNDqVQozuCJb9ikNtYi/r+Xct5+/IUvtyhRBnZ3FK 0crMAwUAuF95DoTw70fCCJPKTpVngU+J+Zj4zsIa+/DTAhLghxh2YOppzOuKIgV6 Y8ODGYeAyXHDVVkQKumVEDARS2ex/OTgNAF7autVwDxSacocgr91Q5rIiMZL7ab6 z3sKtY22Cab5k5q/+g8790icPKiUSpRawhUeXaZ9k5MGzobr8LW1cQ5nQ2OXjwhn GK+QN+1/wiLqSbu6oO0/qVhyLEVzkml54Ul5cHJ3neEifxWVBsJV9lIqCU00ta42 bYWtkNYeYyo0n4W7jV+kyWdMEmX/zvYYBlTttT6AOQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvgedrfedtgdduudcutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhepfffhvfevuffkfhggtggujgesghdtre ertddtjeenucfhrhhomheprfgrthhrihgtkhcuufhtvghinhhhrghrughtuceophhssehp khhsrdhimheqnecuggftrfgrthhtvghrnhepteeuvefhhfdufedvgeeiueeileegtdfhge eftdeuveejjedtgfejhedujeeutddunecuvehluhhsthgvrhfuihiivgeptdenucfrrghr rghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 8 Nov 2022 05:04:04 -0500 (EST) Received: by pks.im (OpenSMTPD) with ESMTPSA id 74e065b9 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Tue, 8 Nov 2022 10:03:59 +0000 (UTC) Date: Tue, 8 Nov 2022 11:04:02 +0100 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Junio C Hamano , =?iso-8859-1?q?=C6var_Arnfj=F6r=F0?= Bjarmason , Taylor Blau , Jeff King Subject: [PATCH v4 6/6] receive-pack: only use visible refs for connectivity check Message-ID: <39b4741734f9b968430a57a9d7168fb5df398f73.1667901510.git.ps@pks.im> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When serving a push, git-receive-pack(1) needs to verify that the packfile sent by the client contains all objects that are required by the updated references. This connectivity check works by marking all preexisting references as uninteresting and using the new reference tips as starting point for a graph walk. Marking all preexisting references as uninteresting can be a problem when it comes to performance. Git forges tend to do internal bookkeeping to keep alive sets of objects for internal use or make them easy to find via certain references. These references are typically hidden away from the user so that they are neither advertised nor writeable. At GitLab, we have one particular repository that contains a total of 7 million references, of which 6.8 million are indeed internal references. With the current connectivity check we are forced to load all these references in order to mark them as uninteresting, and this alone takes around 15 seconds to compute. We can optimize this by only taking into account the set of visible refs when marking objects as uninteresting. This means that we may now walk more objects until we hit any object that is marked as uninteresting. But it is rather unlikely that clients send objects that make large parts of objects reachable that have previously only ever been hidden, whereas the common case is to push incremental changes that build on top of the visible object graph. This provides a huge boost to performance in the mentioned repository, where the vast majority of its refs hidden. Pushing a new commit into this repo with `transfer.hideRefs` set up to hide 6.8 million of 7 refs as it is configured in Gitaly leads to a 4.5-fold speedup: Benchmark 1: main Time (mean ± σ): 30.977 s ± 0.157 s [User: 30.226 s, System: 1.083 s] Range (min … max): 30.796 s … 31.071 s 3 runs Benchmark 2: pks-connectivity-check-hide-refs Time (mean ± σ): 6.799 s ± 0.063 s [User: 6.803 s, System: 0.354 s] Range (min … max): 6.729 s … 6.850 s 3 runs Summary 'pks-connectivity-check-hide-refs' ran 4.56 ± 0.05 times faster than 'main' As we mostly go through the same codepaths even in the case where there are no hidden refs at all compared to the code before there is no change in performance when no refs are hidden: Benchmark 1: main Time (mean ± σ): 48.188 s ± 0.432 s [User: 49.326 s, System: 5.009 s] Range (min … max): 47.706 s … 48.539 s 3 runs Benchmark 2: pks-connectivity-check-hide-refs Time (mean ± σ): 48.027 s ± 0.500 s [User: 48.934 s, System: 5.025 s] Range (min … max): 47.504 s … 48.500 s 3 runs Summary 'pks-connectivity-check-hide-refs' ran 1.00 ± 0.01 times faster than 'main' Signed-off-by: Patrick Steinhardt --- builtin/receive-pack.c | 2 ++ connected.c | 3 +++ connected.h | 7 +++++++ 3 files changed, 12 insertions(+) diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c index 1f3efc58fb..77ab40f123 100644 --- a/builtin/receive-pack.c +++ b/builtin/receive-pack.c @@ -1929,6 +1929,8 @@ static void execute_commands(struct command *commands, opt.err_fd = err_fd; opt.progress = err_fd && !quiet; opt.env = tmp_objdir_env(tmp_objdir); + opt.exclude_hidden_refs_section = "receive"; + if (check_connected(iterate_receive_command_list, &data, &opt)) set_connectivity_errors(commands, si); diff --git a/connected.c b/connected.c index 74a20cb32e..4f6388eed7 100644 --- a/connected.c +++ b/connected.c @@ -100,6 +100,9 @@ int check_connected(oid_iterate_fn fn, void *cb_data, strvec_push(&rev_list.args, "--exclude-promisor-objects"); if (!opt->is_deepening_fetch) { strvec_push(&rev_list.args, "--not"); + if (opt->exclude_hidden_refs_section) + strvec_pushf(&rev_list.args, "--exclude-hidden=%s", + opt->exclude_hidden_refs_section); strvec_push(&rev_list.args, "--all"); } strvec_push(&rev_list.args, "--quiet"); diff --git a/connected.h b/connected.h index 6e59c92aa3..16b2c84f2e 100644 --- a/connected.h +++ b/connected.h @@ -46,6 +46,13 @@ struct check_connected_options { * during a fetch. */ unsigned is_deepening_fetch : 1; + + /* + * If not NULL, use `--exclude-hidden=$section` to exclude all refs + * hidden via the `$section.hideRefs` config from the set of + * already-reachable refs. + */ + const char *exclude_hidden_refs_section; }; #define CHECK_CONNECTED_INIT { 0 }