From patchwork Mon Nov 7 12:16:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13034296 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D7DA3C43217 for ; Mon, 7 Nov 2022 12:17:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232027AbiKGMRc (ORCPT ); Mon, 7 Nov 2022 07:17:32 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44926 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231976AbiKGMRW (ORCPT ); Mon, 7 Nov 2022 07:17:22 -0500 Received: from wout3-smtp.messagingengine.com (wout3-smtp.messagingengine.com [64.147.123.19]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7CD3618C for ; Mon, 7 Nov 2022 04:17:20 -0800 (PST) Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.west.internal (Postfix) with ESMTP id 16E1C320099C; Mon, 7 Nov 2022 07:16:28 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute5.internal (MEProxy); Mon, 07 Nov 2022 07:16:28 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:date:date:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to; s=fm1; t=1667823387; x=1667909787; bh=1yZOoyBaIa b0/ozJhwc7BP+Ss8ZgJx3xNrWcoLY5fmI=; b=GBv9TolJJz5PMT6ISoMwwLKMiL UBKKLMfZr1ZgDfNY6GkbMrpYEbv32AhaMi8iMSeZnDRShVClAgAOvd9hN2fSDPiV rYuob24PDMTi14+qgTBmw6ZR5Kd3G2kuDCQ+4qJqbO+Q3SOjzxtEdILCyxBG3eTK 09WncbfJDtl/E6X18i4WkHxd48ZGbNAgHRWVW00PIljsQk35RQecrGomHP3vAkUx +kDf82IPzpktYrma0caBf0scFDsgHwMTT1mc7A7rXCF99lIz3awaxqADwaHsGAsL QM/EhOFmS6ML2NKQrK3srafdTj9wWnthJNKnRM9eOPJMg6jZ35KqK9xMiiTA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:date:date:feedback-id :feedback-id:from:from:in-reply-to:in-reply-to:message-id :mime-version:references:reply-to:sender:subject:subject:to:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1667823387; x=1667909787; bh=1yZOoyBaIab0/ozJhwc7BP+Ss8Zg Jx3xNrWcoLY5fmI=; b=cjWwRPY4Bxo8QaTiBbwXhCrBqsHa1i3gGIIg5FbSMbUr sM3Zd9BbNSgAXkKiHIrIPuBn0upx3r+dWKRvcc3nc4WXln1TpqIMFUrCMGmA96ka ntdiK+rH0cSQxwJW3FAW1rckLDE8evaSUjobozm0QlDc+woWYC+N2fh1wqBU/dwi z6j4d7zRHLSm8FPK+MSQ5gGp72jXBWsZvtqsVG/C1A1hg3au4qvftxmYhYQPfOZt 72zJPLeX4QqPFDPXWvjaiRnpJTPsBlZvFhYYI+zV9YtZI4OvRzbFftMBI3adiecb KPsR5lUANiCfipBp1Wh7ROHsDpsDrAW9PSvKKK+Qkw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvgedrvdekgdefiecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhepfffhvfevuffkfhggtggujgesghdtre ertddtvdenucfhrhhomheprfgrthhrihgtkhcuufhtvghinhhhrghrughtuceophhssehp khhsrdhimheqnecuggftrfgrthhtvghrnhepueektdevtdffveeljeetgfehheeigeekle duvdeffeeghefgledttdehjeelffetnecuvehluhhsthgvrhfuihiivgeptdenucfrrghr rghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 7 Nov 2022 07:16:26 -0500 (EST) Received: by pks.im (OpenSMTPD) with ESMTPSA id 2ea46772 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 7 Nov 2022 12:16:22 +0000 (UTC) Date: Mon, 7 Nov 2022 13:16:23 +0100 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Junio C Hamano , =?iso-8859-1?q?=C6var_Arnfj=F6r=F0?= Bjarmason , Taylor Blau , Jeff King Subject: [PATCH v3 1/6] refs: get rid of global list of hidden refs Message-ID: <3741f0a389ec28e683722547b2e898db08763424.1667823042.git.ps@pks.im> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org We're about to add a new argument to git-rev-list(1) that allows it to add all references that are visible when taking `transfer.hideRefs` et al into account. This will require us to potentially parse multiple sets of hidden refs, which is not easily possible right now as there is only a single, global instance of the list of parsed hidden refs. Refactor `parse_hide_refs_config()` and `ref_is_hidden()` so that both take the list of hidden references as input and adjust callers to keep a local list, instead. This allows us to easily use multiple hidden-ref lists. Furthermore, it allows us to properly free this list before we exit. Signed-off-by: Patrick Steinhardt --- builtin/receive-pack.c | 8 +++++--- ls-refs.c | 13 +++++++++---- refs.c | 14 ++++---------- refs.h | 5 +++-- upload-pack.c | 30 ++++++++++++++++++------------ 5 files changed, 39 insertions(+), 31 deletions(-) diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c index 44bcea3a5b..1f3efc58fb 100644 --- a/builtin/receive-pack.c +++ b/builtin/receive-pack.c @@ -80,6 +80,7 @@ static struct object_id push_cert_oid; static struct signature_check sigcheck; static const char *push_cert_nonce; static const char *cert_nonce_seed; +static struct string_list hidden_refs = STRING_LIST_INIT_DUP; static const char *NONCE_UNSOLICITED = "UNSOLICITED"; static const char *NONCE_BAD = "BAD"; @@ -130,7 +131,7 @@ static enum deny_action parse_deny_action(const char *var, const char *value) static int receive_pack_config(const char *var, const char *value, void *cb) { - int status = parse_hide_refs_config(var, value, "receive"); + int status = parse_hide_refs_config(var, value, "receive", &hidden_refs); if (status) return status; @@ -296,7 +297,7 @@ static int show_ref_cb(const char *path_full, const struct object_id *oid, struct oidset *seen = data; const char *path = strip_namespace(path_full); - if (ref_is_hidden(path, path_full)) + if (ref_is_hidden(path, path_full, &hidden_refs)) return 0; /* @@ -1794,7 +1795,7 @@ static void reject_updates_to_hidden(struct command *commands) strbuf_setlen(&refname_full, prefix_len); strbuf_addstr(&refname_full, cmd->ref_name); - if (!ref_is_hidden(cmd->ref_name, refname_full.buf)) + if (!ref_is_hidden(cmd->ref_name, refname_full.buf, &hidden_refs)) continue; if (is_null_oid(&cmd->new_oid)) cmd->error_string = "deny deleting a hidden ref"; @@ -2591,6 +2592,7 @@ int cmd_receive_pack(int argc, const char **argv, const char *prefix) packet_flush(1); oid_array_clear(&shallow); oid_array_clear(&ref); + string_list_clear(&hidden_refs, 1); free((void *)push_cert_nonce); return 0; } diff --git a/ls-refs.c b/ls-refs.c index fa0d01b47c..ae89f850e9 100644 --- a/ls-refs.c +++ b/ls-refs.c @@ -6,6 +6,7 @@ #include "ls-refs.h" #include "pkt-line.h" #include "config.h" +#include "string-list.h" static int config_read; static int advertise_unborn; @@ -73,6 +74,7 @@ struct ls_refs_data { unsigned symrefs; struct strvec prefixes; struct strbuf buf; + struct string_list hidden_refs; unsigned unborn : 1; }; @@ -84,7 +86,7 @@ static int send_ref(const char *refname, const struct object_id *oid, strbuf_reset(&data->buf); - if (ref_is_hidden(refname_nons, refname)) + if (ref_is_hidden(refname_nons, refname, &data->hidden_refs)) return 0; if (!ref_match(&data->prefixes, refname_nons)) @@ -137,14 +139,15 @@ static void send_possibly_unborn_head(struct ls_refs_data *data) } static int ls_refs_config(const char *var, const char *value, - void *data UNUSED) + void *cb_data) { + struct ls_refs_data *data = cb_data; /* * We only serve fetches over v2 for now, so respect only "uploadpack" * config. This may need to eventually be expanded to "receive", but we * don't yet know how that information will be passed to ls-refs. */ - return parse_hide_refs_config(var, value, "uploadpack"); + return parse_hide_refs_config(var, value, "uploadpack", &data->hidden_refs); } int ls_refs(struct repository *r, struct packet_reader *request) @@ -154,9 +157,10 @@ int ls_refs(struct repository *r, struct packet_reader *request) memset(&data, 0, sizeof(data)); strvec_init(&data.prefixes); strbuf_init(&data.buf, 0); + string_list_init_dup(&data.hidden_refs); ensure_config_read(); - git_config(ls_refs_config, NULL); + git_config(ls_refs_config, &data); while (packet_reader_read(request) == PACKET_READ_NORMAL) { const char *arg = request->line; @@ -195,6 +199,7 @@ int ls_refs(struct repository *r, struct packet_reader *request) packet_fflush(stdout); strvec_clear(&data.prefixes); strbuf_release(&data.buf); + string_list_clear(&data.hidden_refs, 1); return 0; } diff --git a/refs.c b/refs.c index 1491ae937e..f1711e2e9f 100644 --- a/refs.c +++ b/refs.c @@ -1414,9 +1414,8 @@ char *shorten_unambiguous_ref(const char *refname, int strict) refname, strict); } -static struct string_list *hide_refs; - -int parse_hide_refs_config(const char *var, const char *value, const char *section) +int parse_hide_refs_config(const char *var, const char *value, const char *section, + struct string_list *hide_refs) { const char *key; if (!strcmp("transfer.hiderefs", var) || @@ -1431,21 +1430,16 @@ int parse_hide_refs_config(const char *var, const char *value, const char *secti len = strlen(ref); while (len && ref[len - 1] == '/') ref[--len] = '\0'; - if (!hide_refs) { - CALLOC_ARRAY(hide_refs, 1); - hide_refs->strdup_strings = 1; - } string_list_append(hide_refs, ref); } return 0; } -int ref_is_hidden(const char *refname, const char *refname_full) +int ref_is_hidden(const char *refname, const char *refname_full, + const struct string_list *hide_refs) { int i; - if (!hide_refs) - return 0; for (i = hide_refs->nr - 1; i >= 0; i--) { const char *match = hide_refs->items[i].string; const char *subject; diff --git a/refs.h b/refs.h index 8958717a17..3266fd8f57 100644 --- a/refs.h +++ b/refs.h @@ -808,7 +808,8 @@ int update_ref(const char *msg, const char *refname, const struct object_id *new_oid, const struct object_id *old_oid, unsigned int flags, enum action_on_err onerr); -int parse_hide_refs_config(const char *var, const char *value, const char *); +int parse_hide_refs_config(const char *var, const char *value, const char *, + struct string_list *); /* * Check whether a ref is hidden. If no namespace is set, both the first and @@ -818,7 +819,7 @@ int parse_hide_refs_config(const char *var, const char *value, const char *); * the ref is outside that namespace, the first parameter is NULL. The second * parameter always points to the full ref name. */ -int ref_is_hidden(const char *, const char *); +int ref_is_hidden(const char *, const char *, const struct string_list *); /* Is this a per-worktree ref living in the refs/ namespace? */ int is_per_worktree_ref(const char *refname); diff --git a/upload-pack.c b/upload-pack.c index 0b8311bd68..9db17f8787 100644 --- a/upload-pack.c +++ b/upload-pack.c @@ -62,6 +62,7 @@ struct upload_pack_data { struct object_array have_obj; struct oid_array haves; /* v2 only */ struct string_list wanted_refs; /* v2 only */ + struct string_list hidden_refs; struct object_array shallows; struct string_list deepen_not; @@ -118,6 +119,7 @@ static void upload_pack_data_init(struct upload_pack_data *data) { struct string_list symref = STRING_LIST_INIT_DUP; struct string_list wanted_refs = STRING_LIST_INIT_DUP; + struct string_list hidden_refs = STRING_LIST_INIT_DUP; struct object_array want_obj = OBJECT_ARRAY_INIT; struct object_array have_obj = OBJECT_ARRAY_INIT; struct oid_array haves = OID_ARRAY_INIT; @@ -130,6 +132,7 @@ static void upload_pack_data_init(struct upload_pack_data *data) memset(data, 0, sizeof(*data)); data->symref = symref; data->wanted_refs = wanted_refs; + data->hidden_refs = hidden_refs; data->want_obj = want_obj; data->have_obj = have_obj; data->haves = haves; @@ -151,6 +154,7 @@ static void upload_pack_data_clear(struct upload_pack_data *data) { string_list_clear(&data->symref, 1); string_list_clear(&data->wanted_refs, 1); + string_list_clear(&data->hidden_refs, 1); object_array_clear(&data->want_obj); object_array_clear(&data->have_obj); oid_array_clear(&data->haves); @@ -842,8 +846,8 @@ static void deepen(struct upload_pack_data *data, int depth) * Checking for reachable shallows requires that our refs be * marked with OUR_REF. */ - head_ref_namespaced(check_ref, NULL); - for_each_namespaced_ref(check_ref, NULL); + head_ref_namespaced(check_ref, data); + for_each_namespaced_ref(check_ref, data); get_reachable_list(data, &reachable_shallows); result = get_shallow_commits(&reachable_shallows, @@ -1158,11 +1162,11 @@ static void receive_needs(struct upload_pack_data *data, /* return non-zero if the ref is hidden, otherwise 0 */ static int mark_our_ref(const char *refname, const char *refname_full, - const struct object_id *oid) + const struct object_id *oid, const struct string_list *hidden_refs) { struct object *o = lookup_unknown_object(the_repository, oid); - if (ref_is_hidden(refname, refname_full)) { + if (ref_is_hidden(refname, refname_full, hidden_refs)) { o->flags |= HIDDEN_REF; return 1; } @@ -1171,11 +1175,12 @@ static int mark_our_ref(const char *refname, const char *refname_full, } static int check_ref(const char *refname_full, const struct object_id *oid, - int flag UNUSED, void *cb_data UNUSED) + int flag UNUSED, void *cb_data) { const char *refname = strip_namespace(refname_full); + struct upload_pack_data *data = cb_data; - mark_our_ref(refname, refname_full, oid); + mark_our_ref(refname, refname_full, oid, &data->hidden_refs); return 0; } @@ -1204,7 +1209,7 @@ static int send_ref(const char *refname, const struct object_id *oid, struct object_id peeled; struct upload_pack_data *data = cb_data; - if (mark_our_ref(refname_nons, refname, oid)) + if (mark_our_ref(refname_nons, refname, oid, &data->hidden_refs)) return 0; if (capabilities) { @@ -1327,7 +1332,7 @@ static int upload_pack_config(const char *var, const char *value, void *cb_data) if (parse_object_filter_config(var, value, data) < 0) return -1; - return parse_hide_refs_config(var, value, "uploadpack"); + return parse_hide_refs_config(var, value, "uploadpack", &data->hidden_refs); } static int upload_pack_protected_config(const char *var, const char *value, void *cb_data) @@ -1375,8 +1380,8 @@ void upload_pack(const int advertise_refs, const int stateless_rpc, advertise_shallow_grafts(1); packet_flush(1); } else { - head_ref_namespaced(check_ref, NULL); - for_each_namespaced_ref(check_ref, NULL); + head_ref_namespaced(check_ref, &data); + for_each_namespaced_ref(check_ref, &data); } if (!advertise_refs) { @@ -1441,6 +1446,7 @@ static int parse_want(struct packet_writer *writer, const char *line, static int parse_want_ref(struct packet_writer *writer, const char *line, struct string_list *wanted_refs, + struct string_list *hidden_refs, struct object_array *want_obj) { const char *refname_nons; @@ -1451,7 +1457,7 @@ static int parse_want_ref(struct packet_writer *writer, const char *line, struct strbuf refname = STRBUF_INIT; strbuf_addf(&refname, "%s%s", get_git_namespace(), refname_nons); - if (ref_is_hidden(refname_nons, refname.buf) || + if (ref_is_hidden(refname_nons, refname.buf, hidden_refs) || read_ref(refname.buf, &oid)) { packet_writer_error(writer, "unknown ref %s", refname_nons); die("unknown ref %s", refname_nons); @@ -1508,7 +1514,7 @@ static void process_args(struct packet_reader *request, continue; if (data->allow_ref_in_want && parse_want_ref(&data->writer, arg, &data->wanted_refs, - &data->want_obj)) + &data->hidden_refs, &data->want_obj)) continue; /* process have line */ if (parse_have(arg, &data->haves)) From patchwork Mon Nov 7 12:16:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13034294 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2F773C4332F for ; Mon, 7 Nov 2022 12:17:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232043AbiKGMRb (ORCPT ); Mon, 7 Nov 2022 07:17:31 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44876 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231961AbiKGMRV (ORCPT ); Mon, 7 Nov 2022 07:17:21 -0500 Received: from wout3-smtp.messagingengine.com (wout3-smtp.messagingengine.com [64.147.123.19]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BB8E426FD for ; Mon, 7 Nov 2022 04:17:20 -0800 (PST) Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailout.west.internal (Postfix) with ESMTP id 7D24F3200201; Mon, 7 Nov 2022 07:16:31 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute2.internal (MEProxy); Mon, 07 Nov 2022 07:16:31 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:date:date:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to; s=fm1; t=1667823391; x=1667909791; bh=KV0NaAJoqK +AmAFuducucextndXPUIkpx8Aluo6mGwU=; b=U7pHRgYmpZ4Tj4B0sNTcK5vP4w SDlJ/y1iUVkHmT/AkVDzxz2nldgZIF74BPaeZleuPIVHUZXJKAH1w7Z5K9hMG5Bm u5wb2OLEmTRJtFfCuI34RnvQbvD8/+VX9PIOltiKdLreQ/na/AYu6YjU/nbR4N7Z rikFg7TIZ/ELzxl4rYKlsElVaz7In5KKNGR1j8bVnVcU2CRG1CMtsdgzQDNHAEF8 oQO7jxUj8m4vaQfVtVzBOOkHwIMtWnl0CcdpOOSTXaeephG2fsIQ5zDK4tXySw42 MNgVkFHj7nk5N280aRfINFPJH9UVBi4v8TADkTcEOz0eD+jcHovyzmdRqwgQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:date:date:feedback-id :feedback-id:from:from:in-reply-to:in-reply-to:message-id :mime-version:references:reply-to:sender:subject:subject:to:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1667823391; x=1667909791; bh=KV0NaAJoqK+AmAFuducucextndXP UIkpx8Aluo6mGwU=; b=t9zjs2D+yVWBEulhKG34Q7E9UDwmm0jOwkxUx/u6WBnp t2JhfZTvWrk9mftur2dfroTBhP32D7ijgWU0es6+1rpqtlw+v0rlEg4fOpIMPVpG VU4/0PLPGYwuDJlhFX6bQKEoJlIx+OVuScYjXAE1MLpVoNTJ9tLcZQjq9KJeB7gT T88YjlzpA7hdUTVoqRPT79hkHQ66llRb5OwdYUaEWsbVMsp+o4PvBflme+zo/WQ7 HV+YGo7Tt4dwKboasPr14E/2vaSjgfWPRj4Oi5vyIObr/MB7/udyLJY8pJ/NCv9b gBfGJ/TlnSpJ0bnDeqUTOuzGSfm/aSveNcwtUM6WSw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvgedrvdekgdefjecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhepfffhvfevuffkfhggtggujgesghdtre ertddtvdenucfhrhhomheprfgrthhrihgtkhcuufhtvghinhhhrghrughtuceophhssehp khhsrdhimheqnecuggftrfgrthhtvghrnhepueektdevtdffveeljeetgfehheeigeekle duvdeffeeghefgledttdehjeelffetnecuvehluhhsthgvrhfuihiivgeptdenucfrrghr rghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 7 Nov 2022 07:16:29 -0500 (EST) Received: by pks.im (OpenSMTPD) with ESMTPSA id 18cea5e4 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 7 Nov 2022 12:16:26 +0000 (UTC) Date: Mon, 7 Nov 2022 13:16:27 +0100 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Junio C Hamano , =?iso-8859-1?q?=C6var_Arnfj=F6r=F0?= Bjarmason , Taylor Blau , Jeff King Subject: [PATCH v3 2/6] revision: move together exclusion-related functions Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Move together the definitions of functions that handle exclusions of refs so that related functionality sits in a single place, only. Signed-off-by: Patrick Steinhardt --- revision.c | 52 ++++++++++++++++++++++++++-------------------------- 1 file changed, 26 insertions(+), 26 deletions(-) diff --git a/revision.c b/revision.c index 0760e78936..be755670e2 100644 --- a/revision.c +++ b/revision.c @@ -1517,14 +1517,6 @@ static void add_rev_cmdline_list(struct rev_info *revs, } } -struct all_refs_cb { - int all_flags; - int warned_bad_reflog; - struct rev_info *all_revs; - const char *name_for_errormsg; - struct worktree *wt; -}; - int ref_excluded(struct string_list *ref_excludes, const char *path) { struct string_list_item *item; @@ -1538,6 +1530,32 @@ int ref_excluded(struct string_list *ref_excludes, const char *path) return 0; } +void clear_ref_exclusion(struct string_list **ref_excludes_p) +{ + if (*ref_excludes_p) { + string_list_clear(*ref_excludes_p, 0); + free(*ref_excludes_p); + } + *ref_excludes_p = NULL; +} + +void add_ref_exclusion(struct string_list **ref_excludes_p, const char *exclude) +{ + if (!*ref_excludes_p) { + CALLOC_ARRAY(*ref_excludes_p, 1); + (*ref_excludes_p)->strdup_strings = 1; + } + string_list_append(*ref_excludes_p, exclude); +} + +struct all_refs_cb { + int all_flags; + int warned_bad_reflog; + struct rev_info *all_revs; + const char *name_for_errormsg; + struct worktree *wt; +}; + static int handle_one_ref(const char *path, const struct object_id *oid, int flag UNUSED, void *cb_data) @@ -1563,24 +1581,6 @@ static void init_all_refs_cb(struct all_refs_cb *cb, struct rev_info *revs, cb->wt = NULL; } -void clear_ref_exclusion(struct string_list **ref_excludes_p) -{ - if (*ref_excludes_p) { - string_list_clear(*ref_excludes_p, 0); - free(*ref_excludes_p); - } - *ref_excludes_p = NULL; -} - -void add_ref_exclusion(struct string_list **ref_excludes_p, const char *exclude) -{ - if (!*ref_excludes_p) { - CALLOC_ARRAY(*ref_excludes_p, 1); - (*ref_excludes_p)->strdup_strings = 1; - } - string_list_append(*ref_excludes_p, exclude); -} - static void handle_refs(struct ref_store *refs, struct rev_info *revs, unsigned flags, int (*for_each)(struct ref_store *, each_ref_fn, void *)) From patchwork Mon Nov 7 12:16:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13034297 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 57D7CC4321E for ; Mon, 7 Nov 2022 12:17:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232039AbiKGMRf (ORCPT ); Mon, 7 Nov 2022 07:17:35 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44938 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231986AbiKGMRW (ORCPT ); Mon, 7 Nov 2022 07:17:22 -0500 Received: from wout3-smtp.messagingengine.com (wout3-smtp.messagingengine.com [64.147.123.19]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C61382BE9 for ; Mon, 7 Nov 2022 04:17:20 -0800 (PST) Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.west.internal (Postfix) with ESMTP id C5A313200922; Mon, 7 Nov 2022 07:16:34 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute3.internal (MEProxy); Mon, 07 Nov 2022 07:16:35 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:date:date:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to; s=fm1; t=1667823394; x=1667909794; bh=DenOB/Iprp jo+CqHNzVaLx9E48iNOHyB/i8jydz+6MY=; b=qeKJO2ppEL3nWzJKGwlZ85x8Ht XAL0r4vS0pYWmgrEbHFfbTDMOqdZkyzvZp6cXw8OeVXHa9Y3UMY2nWJ0kPGUanuP X5+i9q+RCCw4qkQeFQKu1t/mvs0opLNnQ89upcwQDehAdtHZc7qZfnp5BJhx6cYq UTPmLqFMJxBKdxOsLyppesc/6AokxFv7Il1oUjrrrwLN+9DnJFtRMxlW7X75nV5P aAfA/oUgbxLaubWlxWKkF+aAHicj1Byy9SILtW7ZPhdSDYGVPFmuM/yD6ovRN7n2 Ln1hcT7UfKAjvW/0mj5P1blifQq70YebMuM9oHKXgqUxBjc+xiauejCsmSkQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:date:date:feedback-id :feedback-id:from:from:in-reply-to:in-reply-to:message-id :mime-version:references:reply-to:sender:subject:subject:to:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1667823394; x=1667909794; bh=DenOB/Iprpjo+CqHNzVaLx9E48iN OHyB/i8jydz+6MY=; b=ssZSsXm3GGPuJDgCRISCuaHz6sBuGVHXgJ4zmwew6T3q 7pbST/UQnF0M5zUtGBXdkfGCrI5z+ohrvzGbiL8994QKOmRAyXMBdRqPNdmsAIB7 c/2OtjoXtSmKOe6gwJpM1oVz5lCtQYD16iuX87x+tIZMags1Z4HgknXEmpd+QmiT RK4N6oMZEfxp0OpQb3q8yGwV5pHb07ndLSIFdef3qv28M6RB7sLt/kg4/YyqtWyi WphxzcuDhVrFopuQ2LPY5RCpCgLGCFolhh03bPWLq+xZOieYWs2Ps0YinMSJyC3H 3TY8frBN6aMSNkKs+kfB9O+CHwBFBRsF5TXB5Hpvnw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvgedrvdekgdefjecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhepfffhvfevuffkfhggtggujgesghdtre ertddtvdenucfhrhhomheprfgrthhrihgtkhcuufhtvghinhhhrghrughtuceophhssehp khhsrdhimheqnecuggftrfgrthhtvghrnhepueektdevtdffveeljeetgfehheeigeekle duvdeffeeghefgledttdehjeelffetnecuvehluhhsthgvrhfuihiivgeptdenucfrrghr rghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 7 Nov 2022 07:16:33 -0500 (EST) Received: by pks.im (OpenSMTPD) with ESMTPSA id 76abf4db (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 7 Nov 2022 12:16:30 +0000 (UTC) Date: Mon, 7 Nov 2022 13:16:31 +0100 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Junio C Hamano , =?iso-8859-1?q?=C6var_Arnfj=F6r=F0?= Bjarmason , Taylor Blau , Jeff King Subject: [PATCH v3 3/6] revision: introduce struct to handle exclusions Message-ID: <2a6a67df1d470bf790025d55095c237ddc6a6bd6.1667823042.git.ps@pks.im> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org The functions that handle exclusion of refs work on a single string list. We're about to add a second mechanism for excluding refs though, and it makes sense to reuse much of the same architecture for both kinds of exclusion. Introduce a new `struct ref_exclusions` that encapsulates all the logic related to excluding refs. Signed-off-by: Patrick Steinhardt --- builtin/rev-parse.c | 8 ++++---- revision.c | 47 ++++++++++++++++++++------------------------- revision.h | 22 +++++++++++++++------ 3 files changed, 41 insertions(+), 36 deletions(-) diff --git a/builtin/rev-parse.c b/builtin/rev-parse.c index 8f61050bde..7fa5b6991b 100644 --- a/builtin/rev-parse.c +++ b/builtin/rev-parse.c @@ -39,7 +39,7 @@ static int abbrev_ref_strict; static int output_sq; static int stuck_long; -static struct string_list *ref_excludes; +static struct ref_exclusions ref_excludes = REF_EXCLUSIONS_INIT; /* * Some arguments are relevant "revision" arguments, @@ -198,7 +198,7 @@ static int show_default(void) static int show_reference(const char *refname, const struct object_id *oid, int flag UNUSED, void *cb_data UNUSED) { - if (ref_excluded(ref_excludes, refname)) + if (ref_excluded(&ref_excludes, refname)) return 0; show_rev(NORMAL, oid, refname); return 0; @@ -585,7 +585,7 @@ static void handle_ref_opt(const char *pattern, const char *prefix) for_each_glob_ref_in(show_reference, pattern, prefix, NULL); else for_each_ref_in(prefix, show_reference, NULL); - clear_ref_exclusion(&ref_excludes); + clear_ref_exclusions(&ref_excludes); } enum format_type { @@ -863,7 +863,7 @@ int cmd_rev_parse(int argc, const char **argv, const char *prefix) } if (!strcmp(arg, "--all")) { for_each_ref(show_reference, NULL); - clear_ref_exclusion(&ref_excludes); + clear_ref_exclusions(&ref_excludes); continue; } if (skip_prefix(arg, "--disambiguate=", &arg)) { diff --git a/revision.c b/revision.c index be755670e2..e5eaaa24ba 100644 --- a/revision.c +++ b/revision.c @@ -1517,35 +1517,29 @@ static void add_rev_cmdline_list(struct rev_info *revs, } } -int ref_excluded(struct string_list *ref_excludes, const char *path) +int ref_excluded(const struct ref_exclusions *exclusions, const char *path) { struct string_list_item *item; - - if (!ref_excludes) - return 0; - for_each_string_list_item(item, ref_excludes) { + for_each_string_list_item(item, &exclusions->excluded_refs) { if (!wildmatch(item->string, path, 0)) return 1; } return 0; } -void clear_ref_exclusion(struct string_list **ref_excludes_p) +void init_ref_exclusions(struct ref_exclusions *exclusions) { - if (*ref_excludes_p) { - string_list_clear(*ref_excludes_p, 0); - free(*ref_excludes_p); - } - *ref_excludes_p = NULL; + string_list_init_dup(&exclusions->excluded_refs); } -void add_ref_exclusion(struct string_list **ref_excludes_p, const char *exclude) +void clear_ref_exclusions(struct ref_exclusions *exclusions) { - if (!*ref_excludes_p) { - CALLOC_ARRAY(*ref_excludes_p, 1); - (*ref_excludes_p)->strdup_strings = 1; - } - string_list_append(*ref_excludes_p, exclude); + string_list_clear(&exclusions->excluded_refs, 0); +} + +void add_ref_exclusion(struct ref_exclusions *exclusions, const char *exclude) +{ + string_list_append(&exclusions->excluded_refs, exclude); } struct all_refs_cb { @@ -1563,7 +1557,7 @@ static int handle_one_ref(const char *path, const struct object_id *oid, struct all_refs_cb *cb = cb_data; struct object *object; - if (ref_excluded(cb->all_revs->ref_excludes, path)) + if (ref_excluded(&cb->all_revs->ref_excludes, path)) return 0; object = get_reference(cb->all_revs, path, oid, cb->all_flags); @@ -1901,6 +1895,7 @@ void repo_init_revisions(struct repository *r, init_display_notes(&revs->notes_opt); list_objects_filter_init(&revs->filter); + init_ref_exclusions(&revs->ref_excludes); } static void add_pending_commit_list(struct rev_info *revs, @@ -2689,10 +2684,10 @@ static int handle_revision_pseudo_opt(struct rev_info *revs, init_all_refs_cb(&cb, revs, *flags); other_head_refs(handle_one_ref, &cb); } - clear_ref_exclusion(&revs->ref_excludes); + clear_ref_exclusions(&revs->ref_excludes); } else if (!strcmp(arg, "--branches")) { handle_refs(refs, revs, *flags, refs_for_each_branch_ref); - clear_ref_exclusion(&revs->ref_excludes); + clear_ref_exclusions(&revs->ref_excludes); } else if (!strcmp(arg, "--bisect")) { read_bisect_terms(&term_bad, &term_good); handle_refs(refs, revs, *flags, for_each_bad_bisect_ref); @@ -2701,15 +2696,15 @@ static int handle_revision_pseudo_opt(struct rev_info *revs, revs->bisect = 1; } else if (!strcmp(arg, "--tags")) { handle_refs(refs, revs, *flags, refs_for_each_tag_ref); - clear_ref_exclusion(&revs->ref_excludes); + clear_ref_exclusions(&revs->ref_excludes); } else if (!strcmp(arg, "--remotes")) { handle_refs(refs, revs, *flags, refs_for_each_remote_ref); - clear_ref_exclusion(&revs->ref_excludes); + clear_ref_exclusions(&revs->ref_excludes); } else if ((argcount = parse_long_opt("glob", argv, &optarg))) { struct all_refs_cb cb; init_all_refs_cb(&cb, revs, *flags); for_each_glob_ref(handle_one_ref, optarg, &cb); - clear_ref_exclusion(&revs->ref_excludes); + clear_ref_exclusions(&revs->ref_excludes); return argcount; } else if ((argcount = parse_long_opt("exclude", argv, &optarg))) { add_ref_exclusion(&revs->ref_excludes, optarg); @@ -2718,17 +2713,17 @@ static int handle_revision_pseudo_opt(struct rev_info *revs, struct all_refs_cb cb; init_all_refs_cb(&cb, revs, *flags); for_each_glob_ref_in(handle_one_ref, optarg, "refs/heads/", &cb); - clear_ref_exclusion(&revs->ref_excludes); + clear_ref_exclusions(&revs->ref_excludes); } else if (skip_prefix(arg, "--tags=", &optarg)) { struct all_refs_cb cb; init_all_refs_cb(&cb, revs, *flags); for_each_glob_ref_in(handle_one_ref, optarg, "refs/tags/", &cb); - clear_ref_exclusion(&revs->ref_excludes); + clear_ref_exclusions(&revs->ref_excludes); } else if (skip_prefix(arg, "--remotes=", &optarg)) { struct all_refs_cb cb; init_all_refs_cb(&cb, revs, *flags); for_each_glob_ref_in(handle_one_ref, optarg, "refs/remotes/", &cb); - clear_ref_exclusion(&revs->ref_excludes); + clear_ref_exclusions(&revs->ref_excludes); } else if (!strcmp(arg, "--reflog")) { add_reflogs_to_pending(revs, *flags); } else if (!strcmp(arg, "--indexed-objects")) { diff --git a/revision.h b/revision.h index afe1b77985..87d6824c55 100644 --- a/revision.h +++ b/revision.h @@ -81,6 +81,14 @@ struct rev_cmdline_info { } *rev; }; +struct ref_exclusions { + /* + * Excluded refs is a list of wildmatch patterns. If any of the + * patterns matches, the reference will be excluded. + */ + struct string_list excluded_refs; +}; + struct oidset; struct topo_walk_info; @@ -103,7 +111,7 @@ struct rev_info { struct list_objects_filter_options filter; /* excluding from --branches, --refs, etc. expansion */ - struct string_list *ref_excludes; + struct ref_exclusions ref_excludes; /* Basic information */ const char *prefix; @@ -439,12 +447,14 @@ void mark_trees_uninteresting_sparse(struct repository *r, struct oidset *trees) void show_object_with_name(FILE *, struct object *, const char *); /** - * Helpers to check if a "struct string_list" item matches with - * wildmatch(). + * Helpers to check if a reference should be excluded. */ -int ref_excluded(struct string_list *, const char *path); -void clear_ref_exclusion(struct string_list **); -void add_ref_exclusion(struct string_list **, const char *exclude); +#define REF_EXCLUSIONS_INIT { .excluded_refs = STRING_LIST_INIT_DUP } + +int ref_excluded(const struct ref_exclusions *exclusions, const char *path); +void init_ref_exclusions(struct ref_exclusions *); +void clear_ref_exclusions(struct ref_exclusions *); +void add_ref_exclusion(struct ref_exclusions *, const char *exclude); /** * This function can be used if you want to add commit objects as revision From patchwork Mon Nov 7 12:16:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13034299 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F3892C4332F for ; Mon, 7 Nov 2022 12:17:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232088AbiKGMRi (ORCPT ); Mon, 7 Nov 2022 07:17:38 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45170 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232022AbiKGMR3 (ORCPT ); Mon, 7 Nov 2022 07:17:29 -0500 Received: from wout3-smtp.messagingengine.com (wout3-smtp.messagingengine.com [64.147.123.19]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D21F82DFC for ; Mon, 7 Nov 2022 04:17:20 -0800 (PST) Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailout.west.internal (Postfix) with ESMTP id 35F41320093C; Mon, 7 Nov 2022 07:16:39 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute2.internal (MEProxy); Mon, 07 Nov 2022 07:16:39 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:date:date:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to; s=fm1; t=1667823398; x=1667909798; bh=0A6D7hEVAG +8j9RpLk+ViMLgYnswG2EeoIXxkwbMO5I=; b=J201j7mi+QWEz7g7N+0/r32T75 ga8B9Wrg09C15w3IpJPbp8Ss1kdoLXS9zZbupPQgWdjj3R0PrWA//ByYAVf5OCni aFlTL73nUTweDRm/8u1xnogg6w/n6CyA++ZxQL7UK9ZR1ijvaGIUMae3RLbsJcd3 wnNDWpmfyFdezC5v/ponPFPBUoIy0wURc0PgrBRZ6Ohw1pqvDZHG/0SFBi2RJG2s 25wT3ErDEswsb6TVP+vKykEtQisnHIxgIVqf4auWbsEBqnj4gBmsv9XgiKjC5xXI wY4nn/7Dq13Zq9amTPmxSRS/PpslCL/yd12mTrdOT0PIHsGFft1I0JyecJdA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:date:date:feedback-id :feedback-id:from:from:in-reply-to:in-reply-to:message-id :mime-version:references:reply-to:sender:subject:subject:to:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1667823398; x=1667909798; bh=0A6D7hEVAG+8j9RpLk+ViMLgYnsw G2EeoIXxkwbMO5I=; b=r2lbFOTcdekBE/k7V+wOV9yTV0hGz2/rsrGU3sUPwBJb JuqiSoV1z4ObogCi8zk5AAUqX7tXVvIj16EnBN3/3ZLI1RMwFqVMYrgZJ7YDlXtu vBA2+Schr8DFRZTmYn6pACAaFeO3A+kBDsbqtdhTY73yAT+uuKGcIaUnIBOimWaE jqcunxZBW1LjJ2LVoUq4zdB/gTKAerhQrxZiDQIIgWRZfNNv2taHEDMIfcMI46u8 rlVlZ3x5h0BTPfjsezSUa+PzvG1LvgOk034oPNy85jbRTfCZ1fQMHcbP1muDVj0y VNSWIQDE60oujsORDCvkcPSW1N9zcmbw4Nvvn6xNag== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvgedrvdekgdefjecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhepfffhvfevuffkfhggtggujgesghdtre ertddtvdenucfhrhhomheprfgrthhrihgtkhcuufhtvghinhhhrghrughtuceophhssehp khhsrdhimheqnecuggftrfgrthhtvghrnhepueektdevtdffveeljeetgfehheeigeekle duvdeffeeghefgledttdehjeelffetnecuvehluhhsthgvrhfuihiivgepudenucfrrghr rghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 7 Nov 2022 07:16:37 -0500 (EST) Received: by pks.im (OpenSMTPD) with ESMTPSA id d5391083 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 7 Nov 2022 12:16:34 +0000 (UTC) Date: Mon, 7 Nov 2022 13:16:35 +0100 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Junio C Hamano , =?iso-8859-1?q?=C6var_Arnfj=F6r=F0?= Bjarmason , Taylor Blau , Jeff King Subject: [PATCH v3 4/6] revision: add new parameter to exclude hidden refs Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Users can optionally hide refs from remote users in git-upload-pack(1), git-receive-pack(1) and others via the `transfer.hideRefs`, but there is not an easy way to obtain the list of all visible or hidden refs right now. We'll require just that though for a performance improvement in our connectivity check. Add a new option `--exclude-hidden=` that excludes any hidden refs from the next pseudo-ref like `--all` or `--branches`. Signed-off-by: Patrick Steinhardt --- Documentation/rev-list-options.txt | 7 ++ builtin/rev-list.c | 1 + revision.c | 43 +++++++- revision.h | 9 +- t/t6021-rev-list-exclude-hidden.sh | 159 +++++++++++++++++++++++++++++ 5 files changed, 217 insertions(+), 2 deletions(-) create mode 100755 t/t6021-rev-list-exclude-hidden.sh diff --git a/Documentation/rev-list-options.txt b/Documentation/rev-list-options.txt index 1837509566..a178956613 100644 --- a/Documentation/rev-list-options.txt +++ b/Documentation/rev-list-options.txt @@ -195,6 +195,13 @@ respectively, and they must begin with `refs/` when applied to `--glob` or `--all`. If a trailing '/{asterisk}' is intended, it must be given explicitly. +--exclude-hidden=[transfer|receive|uploadpack]:: + Do not include refs that have been hidden via either one of + `transfer.hideRefs`, `receive.hideRefs` or `uploadpack.hideRefs` that + the next `--all`, `--branches`, `--tags`, `--remotes` or `--glob` would + otherwise consider. This option is cleared when seeing one of these + pseudo-refs. + --reflog:: Pretend as if all objects mentioned by reflogs are listed on the command line as ``. diff --git a/builtin/rev-list.c b/builtin/rev-list.c index 3acd93f71e..9eace06385 100644 --- a/builtin/rev-list.c +++ b/builtin/rev-list.c @@ -38,6 +38,7 @@ static const char rev_list_usage[] = " --tags\n" " --remotes\n" " --stdin\n" +" --exclude-hidden=[transfer|receive|uploadpack]\n" " --quiet\n" " ordering output:\n" " --topo-order\n" diff --git a/revision.c b/revision.c index e5eaaa24ba..45652f9b0b 100644 --- a/revision.c +++ b/revision.c @@ -1,4 +1,5 @@ #include "cache.h" +#include "config.h" #include "object-store.h" #include "tag.h" #include "blob.h" @@ -1519,22 +1520,30 @@ static void add_rev_cmdline_list(struct rev_info *revs, int ref_excluded(const struct ref_exclusions *exclusions, const char *path) { + const char *stripped_path = strip_namespace(path); struct string_list_item *item; + for_each_string_list_item(item, &exclusions->excluded_refs) { if (!wildmatch(item->string, path, 0)) return 1; } + + if (ref_is_hidden(stripped_path, path, &exclusions->hidden_refs)) + return 1; + return 0; } void init_ref_exclusions(struct ref_exclusions *exclusions) { string_list_init_dup(&exclusions->excluded_refs); + string_list_init_dup(&exclusions->hidden_refs); } void clear_ref_exclusions(struct ref_exclusions *exclusions) { string_list_clear(&exclusions->excluded_refs, 0); + string_list_clear(&exclusions->hidden_refs, 1); } void add_ref_exclusion(struct ref_exclusions *exclusions, const char *exclude) @@ -1542,6 +1551,35 @@ void add_ref_exclusion(struct ref_exclusions *exclusions, const char *exclude) string_list_append(&exclusions->excluded_refs, exclude); } +struct exclude_hidden_refs_cb { + struct ref_exclusions *exclusions; + const char *section; +}; + +static int hide_refs_config(const char *var, const char *value, void *cb_data) +{ + struct exclude_hidden_refs_cb *cb = cb_data; + return parse_hide_refs_config(var, value, cb->section, + &cb->exclusions->hidden_refs); +} + +void exclude_hidden_refs(struct ref_exclusions *exclusions, const char *section) +{ + struct exclude_hidden_refs_cb cb; + + if (strcmp(section, "transfer") && strcmp(section, "receive") && + strcmp(section, "uploadpack")) + die(_("unsupported section for hidden refs: %s"), section); + + if (exclusions->hidden_refs.nr) + die(_("--exclude-hidden= passed more than once")); + + cb.exclusions = exclusions; + cb.section = section; + + git_config(hide_refs_config, &cb); +} + struct all_refs_cb { int all_flags; int warned_bad_reflog; @@ -2220,7 +2258,7 @@ static int handle_revision_opt(struct rev_info *revs, int argc, const char **arg !strcmp(arg, "--bisect") || starts_with(arg, "--glob=") || !strcmp(arg, "--indexed-objects") || !strcmp(arg, "--alternate-refs") || - starts_with(arg, "--exclude=") || + starts_with(arg, "--exclude=") || starts_with(arg, "--exclude-hidden=") || starts_with(arg, "--branches=") || starts_with(arg, "--tags=") || starts_with(arg, "--remotes=") || starts_with(arg, "--no-walk=")) { @@ -2709,6 +2747,9 @@ static int handle_revision_pseudo_opt(struct rev_info *revs, } else if ((argcount = parse_long_opt("exclude", argv, &optarg))) { add_ref_exclusion(&revs->ref_excludes, optarg); return argcount; + } else if ((argcount = parse_long_opt("exclude-hidden", argv, &optarg))) { + exclude_hidden_refs(&revs->ref_excludes, optarg); + return argcount; } else if (skip_prefix(arg, "--branches=", &optarg)) { struct all_refs_cb cb; init_all_refs_cb(&cb, revs, *flags); diff --git a/revision.h b/revision.h index 87d6824c55..fef5e063d1 100644 --- a/revision.h +++ b/revision.h @@ -87,6 +87,12 @@ struct ref_exclusions { * patterns matches, the reference will be excluded. */ struct string_list excluded_refs; + + /* + * Hidden refs is a list of patterns that is to be hidden via + * `ref_is_hidden()`. + */ + struct string_list hidden_refs; }; struct oidset; @@ -449,12 +455,13 @@ void show_object_with_name(FILE *, struct object *, const char *); /** * Helpers to check if a reference should be excluded. */ -#define REF_EXCLUSIONS_INIT { .excluded_refs = STRING_LIST_INIT_DUP } +#define REF_EXCLUSIONS_INIT { .excluded_refs = STRING_LIST_INIT_DUP, .hidden_refs = STRING_LIST_INIT_DUP } int ref_excluded(const struct ref_exclusions *exclusions, const char *path); void init_ref_exclusions(struct ref_exclusions *); void clear_ref_exclusions(struct ref_exclusions *); void add_ref_exclusion(struct ref_exclusions *, const char *exclude); +void exclude_hidden_refs(struct ref_exclusions *, const char *section); /** * This function can be used if you want to add commit objects as revision diff --git a/t/t6021-rev-list-exclude-hidden.sh b/t/t6021-rev-list-exclude-hidden.sh new file mode 100755 index 0000000000..d08fc2da93 --- /dev/null +++ b/t/t6021-rev-list-exclude-hidden.sh @@ -0,0 +1,159 @@ +#!/bin/sh + +test_description='git rev-list --exclude-hidden test' + +TEST_PASSES_SANITIZE_LEAK=true +. ./test-lib.sh + +test_expect_success 'setup' ' + test_commit_bulk --id=commit --ref=refs/heads/main 1 && + COMMIT=$(git rev-parse refs/heads/main) && + test_commit_bulk --id=tag --ref=refs/tags/lightweight 1 && + TAG=$(git rev-parse refs/tags/lightweight) && + test_commit_bulk --id=hidden --ref=refs/hidden/commit 1 && + HIDDEN=$(git rev-parse refs/hidden/commit) && + test_commit_bulk --id=namespace --ref=refs/namespaces/namespace/refs/namespaced/commit 1 && + NAMESPACE=$(git rev-parse refs/namespaces/namespace/refs/namespaced/commit) +' + +test_expect_success 'invalid section' ' + echo "fatal: unsupported section for hidden refs: unsupported" >expected && + test_must_fail git rev-list --exclude-hidden=unsupported 2>err && + test_cmp expected err +' + +test_expect_success 'passed multiple times' ' + echo "fatal: --exclude-hidden= passed more than once" >expected && + test_must_fail git -c transfer.hideRefs=refs/hidden/ rev-list --exclude-hidden=transfer --exclude-hidden=transfer 2>err && + test_cmp expected err +' + +test_expect_success '--exclude-hidden without hiddenRefs' ' + git rev-list --exclude-hidden=transfer --all >out && + cat >expected <<-EOF && + $NAMESPACE + $HIDDEN + $TAG + $COMMIT + EOF + test_cmp expected out +' + +test_expect_success 'hidden via transfer.hideRefs' ' + git -c transfer.hideRefs=refs/hidden/ rev-list --exclude-hidden=transfer --all >out && + cat >expected <<-EOF && + $NAMESPACE + $TAG + $COMMIT + EOF + test_cmp expected out +' + +test_expect_success '--all --exclude-hidden=transfer --not --all without hidden refs' ' + git rev-list --all --exclude-hidden=transfer --not --all >out && + test_must_be_empty out +' + +test_expect_success '--all --exclude-hidden=transfer --not --all with hidden ref' ' + git -c transfer.hideRefs=refs/hidden/ rev-list --all --exclude-hidden=transfer --not --all >out && + cat >expected <<-EOF && + $HIDDEN + EOF + test_cmp expected out +' + +test_expect_success '--exclude-hidden with --exclude' ' + git -c transfer.hideRefs=refs/hidden/ rev-list --exclude=refs/tags/* --exclude-hidden=transfer --all >out && + cat >expected <<-EOF && + $NAMESPACE + $COMMIT + EOF + test_cmp expected out +' + +test_expect_success '--exclude-hidden is reset' ' + git -c transfer.hideRefs=refs/ rev-list --exclude-hidden=transfer --all --all >out && + cat >expected <<-EOF && + $NAMESPACE + $HIDDEN + $TAG + $COMMIT + EOF + test_cmp expected out +' + +test_expect_success '--exclude-hidden operates on stripped refs by default' ' + GIT_NAMESPACE=namespace git -c transfer.hideRefs=refs/namespaced/ rev-list --exclude-hidden=transfer --all >out && + cat >expected <<-EOF && + $HIDDEN + $TAG + $COMMIT + EOF + test_cmp expected out +' + +test_expect_success '--exclude-hidden does not hide namespace by default' ' + GIT_NAMESPACE=namespace git -c transfer.hideRefs=refs/namespaces/namespace/ rev-list --exclude-hidden=transfer --all >out && + cat >expected <<-EOF && + $NAMESPACE + $HIDDEN + $TAG + $COMMIT + EOF + test_cmp expected out +' + +test_expect_success '--exclude-hidden= may operate on unstripped refs' ' + GIT_NAMESPACE=namespace git -c transfer.hideRefs=^refs/namespaces/namespace/ rev-list --exclude-hidden=transfer --all >out && + cat >expected <<-EOF && + $HIDDEN + $TAG + $COMMIT + EOF + test_cmp expected out +' + +for section in receive uploadpack +do + test_expect_success "hidden via $section.hideRefs" ' + git -c $section.hideRefs=refs/hidden/ rev-list --exclude-hidden=$section --all >out && + cat >expected <<-EOF && + $NAMESPACE + $TAG + $COMMIT + EOF + test_cmp expected out + ' + + test_expect_success "--exclude-hidden=$section respects transfer.hideRefs" ' + git -c transfer.hideRefs=refs/hidden/ rev-list --exclude-hidden=$section --all >out && + cat >expected <<-EOF && + $NAMESPACE + $TAG + $COMMIT + EOF + test_cmp expected out + ' + + test_expect_success "--exclude-hidden=transfer ignores $section.hideRefs" ' + git -c $section.hideRefs=refs/hidden/ rev-list --exclude-hidden=transfer --all >out && + cat >expected <<-EOF && + $NAMESPACE + $HIDDEN + $TAG + $COMMIT + EOF + test_cmp expected out + ' + + test_expect_success "--exclude-hidden=$section respects both transfer.hideRefs and $section.hideRefs" ' + git -c transfer.hideRefs=refs/tags/ -c $section.hideRefs=refs/hidden/ rev-list --exclude-hidden=$section --all >out && + cat >expected <<-EOF && + $NAMESPACE + $COMMIT + EOF + test_cmp expected out + ' +done + +test_done From patchwork Mon Nov 7 12:16:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13034298 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D34F2C433FE for ; Mon, 7 Nov 2022 12:17:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232086AbiKGMRh (ORCPT ); Mon, 7 Nov 2022 07:17:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44948 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232009AbiKGMRW (ORCPT ); Mon, 7 Nov 2022 07:17:22 -0500 Received: from wout3-smtp.messagingengine.com (wout3-smtp.messagingengine.com [64.147.123.19]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 362DF6246 for ; Mon, 7 Nov 2022 04:17:21 -0800 (PST) Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.west.internal (Postfix) with ESMTP id 7B1553200988; Mon, 7 Nov 2022 07:16:42 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute5.internal (MEProxy); Mon, 07 Nov 2022 07:16:42 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:date:date:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to; s=fm1; t=1667823402; x=1667909802; bh=hPgQwZl+eQ 5j32YpCfjyDbO7oMTK58Pf4ETDVlNoaZo=; b=Y8c2dXJ6igfqXabuvVKGILuCp2 hZvyT99kDOTpBjG1B1iL00pdLeY+jNHGGDyDS8vi3ctevN0e5EuKEClvCMv4mNfx pcpBHfUIXc9y0dVAE2XRzkndCKrZzlk9xw2ztd50SQ9T+9WANSSYziL/KuY7Xucx k/37go1q0iXMQAkXMLLicwB8HCFxm8zSurI7q44ORfmmN41qHaRxKvdHxI+M489s dy189QBnUu1In1SyHFUMMEajSoOd4O9Ba0cu41F0zRQ/bDeTSZKERfPMEnwR28DV q33Pn+rvNwcBvYYowNtt4vbV+L8zTFkan6lcj+PcBDXt7nyCyxzjiBhDAdkg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:date:date:feedback-id :feedback-id:from:from:in-reply-to:in-reply-to:message-id :mime-version:references:reply-to:sender:subject:subject:to:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1667823402; x=1667909802; bh=hPgQwZl+eQ5j32YpCfjyDbO7oMTK 58Pf4ETDVlNoaZo=; b=KZrQ6fFofAGlKCbNj++2SikEgJ5R2VFuEj8/4i4ivthE 55sau1Qzo9KSF8sYxaoQrUuRe4BSmnCSOJHYN1xvjP7xt3qfK+T5LwwppaYg2I9+ ZCa9nDrtmxzLjRWjtPvPN7CR9J+MuHFTWJpPj7B6ZaCyydyXgCnkP4rwzQVWgjMp rTdFXTzTHjuaG5oGMziUVgUi9m3EoUDAQgKCOENjO5155CBKiLJoUpGJVWxjyL/e 7Q0sQGiQ/oyluKBwdR7EPrux/NJ0Raiotz0VWK6jKv2ha3Fl8SrX5LmkheOw8dnl XTQyJ1k/a916w5+wAB61+SJ05am/z3YLtKN/yDl+HQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvgedrvdekgdefiecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhepfffhvfevuffkfhggtggujgesghdtre ertddtvdenucfhrhhomheprfgrthhrihgtkhcuufhtvghinhhhrghrughtuceophhssehp khhsrdhimheqnecuggftrfgrthhtvghrnhepueektdevtdffveeljeetgfehheeigeekle duvdeffeeghefgledttdehjeelffetnecuvehluhhsthgvrhfuihiivgepudenucfrrghr rghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 7 Nov 2022 07:16:40 -0500 (EST) Received: by pks.im (OpenSMTPD) with ESMTPSA id 382f98ce (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 7 Nov 2022 12:16:38 +0000 (UTC) Date: Mon, 7 Nov 2022 13:16:39 +0100 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Junio C Hamano , =?iso-8859-1?q?=C6var_Arnfj=F6r=F0?= Bjarmason , Taylor Blau , Jeff King Subject: [PATCH v3 5/6] revparse: add `--exclude-hidden=` option Message-ID: <68a5e563045f1b3e6db7304206d95ebebcacbe6f.1667823042.git.ps@pks.im> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Add a new `--exclude-hidden=` option that is similar to the one we just added to git-rev-list(1). Given a seciton name `transfer`, `uploadpack` or `receive` as argument, it causes us to exclude all references that would be hidden by the respective `$seciton.hideRefs` configuration. Signed-off-by: Patrick Steinhardt --- Documentation/git-rev-parse.txt | 7 +++++++ builtin/rev-parse.c | 4 ++++ t/t6018-rev-list-glob.sh | 8 ++++++++ 3 files changed, 19 insertions(+) diff --git a/Documentation/git-rev-parse.txt b/Documentation/git-rev-parse.txt index 6b8ca085aa..a016cb5abe 100644 --- a/Documentation/git-rev-parse.txt +++ b/Documentation/git-rev-parse.txt @@ -197,6 +197,13 @@ respectively, and they must begin with `refs/` when applied to `--glob` or `--all`. If a trailing '/{asterisk}' is intended, it must be given explicitly. +--exclude-hidden=[transfer|receive|uploadpack]:: + Do not include refs that have been hidden via either one of + `transfer.hideRefs`, `receive.hideRefs` or `uploadpack.hideRefs` that + the next `--all`, `--branches`, `--tags`, `--remotes` or `--glob` would + otherwise consider. This option is cleared when seeing one of these + pseudo-refs. + --disambiguate=:: Show every object whose name begins with the given prefix. The must be at least 4 hexadecimal digits long to diff --git a/builtin/rev-parse.c b/builtin/rev-parse.c index 7fa5b6991b..49730c7a23 100644 --- a/builtin/rev-parse.c +++ b/builtin/rev-parse.c @@ -895,6 +895,10 @@ int cmd_rev_parse(int argc, const char **argv, const char *prefix) add_ref_exclusion(&ref_excludes, arg); continue; } + if (skip_prefix(arg, "--exclude-hidden=", &arg)) { + exclude_hidden_refs(&ref_excludes, arg); + continue; + } if (!strcmp(arg, "--show-toplevel")) { const char *work_tree = get_git_work_tree(); if (work_tree) diff --git a/t/t6018-rev-list-glob.sh b/t/t6018-rev-list-glob.sh index e1abc5c2b3..f92616de12 100755 --- a/t/t6018-rev-list-glob.sh +++ b/t/t6018-rev-list-glob.sh @@ -187,6 +187,14 @@ test_expect_success 'rev-parse --exclude=ref with --remotes=glob' ' compare rev-parse "--exclude=upstream/x --remotes=upstream/*" "upstream/one upstream/two" ' +test_expect_success 'rev-parse --exclude-hidden= with --all' ' + compare "-c transfer.hideRefs=refs/remotes/ rev-parse" "--exclude-hidden=transfer --all" "--branches --tags" +' + +test_expect_success 'rev-parse --exclude-hidden= with --all' ' + compare "-c transfer.hideRefs=refs/heads/subspace/ rev-parse" "--exclude-hidden=transfer --all" "--exclude=refs/heads/subspace/* --all" +' + test_expect_success 'rev-list --exclude=glob with --branches=glob' ' compare rev-list "--exclude=subspace-* --branches=sub*" "subspace/one subspace/two" ' From patchwork Mon Nov 7 12:16:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13034295 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 696F6C4332F for ; Mon, 7 Nov 2022 12:17:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232034AbiKGMRe (ORCPT ); Mon, 7 Nov 2022 07:17:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44936 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231990AbiKGMRW (ORCPT ); Mon, 7 Nov 2022 07:17:22 -0500 Received: from wout3-smtp.messagingengine.com (wout3-smtp.messagingengine.com [64.147.123.19]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2C5FB55A8 for ; Mon, 7 Nov 2022 04:17:21 -0800 (PST) Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailout.west.internal (Postfix) with ESMTP id E15453200997; Mon, 7 Nov 2022 07:16:46 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute2.internal (MEProxy); Mon, 07 Nov 2022 07:16:47 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:date:date:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to; s=fm1; t=1667823406; x=1667909806; bh=a0myhg9ixf v3MMs55HXIdrPjUze9f5kr/SI8SroqRbI=; b=EwywDwUO7y3tL/P+lXg/4QRgdd hiLDEgXHwcFaSunAfmyupg7nwIWlWKhXrHVGGWsK0IbztkpPaeTYeQ6+zW1WpWja l/fOTbGl9jCjtLsGfjFZTk9pRDvi+yCkBkU+5to4Cp+VPgePwd1o+wAXAkA9U0NA N7jbezfdKxyg+8XQiuYAAhWWMs6msG+DdUgbamKId89OVabrhsvv/uRJ3BdXOJ1o ed1xuqkJos/ahWnLh8lYPg3HwhRcvKsNBvrZuUu2Cgvsq7XbIE3FHMxNR+unK2Sd sGz+RPRRfFcoYQzqMKxzmNLp2LSnOIc5j2NqsFR/cQjcmcnri6D1c6fgpk6g== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:date:date:feedback-id :feedback-id:from:from:in-reply-to:in-reply-to:message-id :mime-version:references:reply-to:sender:subject:subject:to:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1667823406; x=1667909806; bh=a0myhg9ixfv3MMs55HXIdrPjUze9 f5kr/SI8SroqRbI=; b=CyAqm0znPhbzKd64bZyVAUvrcNzYPMTyNba6/AzYB3l5 XEp5q97MIiVhmNTnbleI1MKrkzKNTF+ghyvsomcKJchAkUgUiLVSAAcUBeekZZB5 TDIAojRdMJgLM9E/FPWY1YQ6snv62KJWvY8TJ2sS47UisBF1Po9jwrgm66DmuBXx HIRk2oP+HYVF1PYoVoyR7YfQliLAbd4Jr0OJCSilRlnPNUu6/9xV97wmir20aWOU JSYScBLit7Dfd5d6Hhu5WYBOhtZQH6REeoSvYk54EcM5FX5SCaHO8BJJjJEGUbo/ mVelgFKYlEB1L1ym8HYBMqXo1xG1Z5QRAdpGFs14GA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvgedrvdekgdefjecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhepfffhvfevuffkfhggtggujgesghdtre ertddtjeenucfhrhhomheprfgrthhrihgtkhcuufhtvghinhhhrghrughtuceophhssehp khhsrdhimheqnecuggftrfgrthhtvghrnhepteeuvefhhfdufedvgeeiueeileegtdfhge eftdeuveejjedtgfejhedujeeutddunecuvehluhhsthgvrhfuihiivgeptdenucfrrghr rghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 7 Nov 2022 07:16:45 -0500 (EST) Received: by pks.im (OpenSMTPD) with ESMTPSA id 96414fd8 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Mon, 7 Nov 2022 12:16:42 +0000 (UTC) Date: Mon, 7 Nov 2022 13:16:43 +0100 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Junio C Hamano , =?iso-8859-1?q?=C6var_Arnfj=F6r=F0?= Bjarmason , Taylor Blau , Jeff King Subject: [PATCH v3 6/6] receive-pack: only use visible refs for connectivity check Message-ID: <9d154495593606532ecec62f29c9c34dc2da8917.1667823042.git.ps@pks.im> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When serving a push, git-receive-pack(1) needs to verify that the packfile sent by the client contains all objects that are required by the updated references. This connectivity check works by marking all preexisting references as uninteresting and using the new reference tips as starting point for a graph walk. Marking all preexisting references as uninteresting can be a problem when it comes to performance. Git forges tend to do internal bookkeeping to keep alive sets of objects for internal use or make them easy to find via certain references. These references are typically hidden away from the user so that they are neither advertised nor writeable. At GitLab, we have one particular repository that contains a total of 7 million references, of which 6.8 million are indeed internal references. With the current connectivity check we are forced to load all these references in order to mark them as uninteresting, and this alone takes around 15 seconds to compute. We can optimize this by only taking into account the set of visible refs when marking objects as uninteresting. This means that we may now walk more objects until we hit any object that is marked as uninteresting. But it is rather unlikely that clients send objects that make large parts of objects reachable that have previously only ever been hidden, whereas the common case is to push incremental changes that build on top of the visible object graph. This provides a huge boost to performance in the mentioned repository, where the vast majority of its refs hidden. Pushing a new commit into this repo with `transfer.hideRefs` set up to hide 6.8 million of 7 refs as it is configured in Gitaly leads to a 4.5-fold speedup: Benchmark 1: main Time (mean ± σ): 30.977 s ± 0.157 s [User: 30.226 s, System: 1.083 s] Range (min … max): 30.796 s … 31.071 s 3 runs Benchmark 2: pks-connectivity-check-hide-refs Time (mean ± σ): 6.799 s ± 0.063 s [User: 6.803 s, System: 0.354 s] Range (min … max): 6.729 s … 6.850 s 3 runs Summary 'pks-connectivity-check-hide-refs' ran 4.56 ± 0.05 times faster than 'main' As we mostly go through the same codepaths even in the case where there are no hidden refs at all compared to the code before there is no change in performance when no refs are hidden: Benchmark 1: main Time (mean ± σ): 48.188 s ± 0.432 s [User: 49.326 s, System: 5.009 s] Range (min … max): 47.706 s … 48.539 s 3 runs Benchmark 2: pks-connectivity-check-hide-refs Time (mean ± σ): 48.027 s ± 0.500 s [User: 48.934 s, System: 5.025 s] Range (min … max): 47.504 s … 48.500 s 3 runs Summary 'pks-connectivity-check-hide-refs' ran 1.00 ± 0.01 times faster than 'main' Signed-off-by: Patrick Steinhardt --- builtin/receive-pack.c | 2 ++ connected.c | 3 +++ connected.h | 7 +++++++ 3 files changed, 12 insertions(+) diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c index 1f3efc58fb..77ab40f123 100644 --- a/builtin/receive-pack.c +++ b/builtin/receive-pack.c @@ -1929,6 +1929,8 @@ static void execute_commands(struct command *commands, opt.err_fd = err_fd; opt.progress = err_fd && !quiet; opt.env = tmp_objdir_env(tmp_objdir); + opt.exclude_hidden_refs_section = "receive"; + if (check_connected(iterate_receive_command_list, &data, &opt)) set_connectivity_errors(commands, si); diff --git a/connected.c b/connected.c index 74a20cb32e..4f6388eed7 100644 --- a/connected.c +++ b/connected.c @@ -100,6 +100,9 @@ int check_connected(oid_iterate_fn fn, void *cb_data, strvec_push(&rev_list.args, "--exclude-promisor-objects"); if (!opt->is_deepening_fetch) { strvec_push(&rev_list.args, "--not"); + if (opt->exclude_hidden_refs_section) + strvec_pushf(&rev_list.args, "--exclude-hidden=%s", + opt->exclude_hidden_refs_section); strvec_push(&rev_list.args, "--all"); } strvec_push(&rev_list.args, "--quiet"); diff --git a/connected.h b/connected.h index 6e59c92aa3..16b2c84f2e 100644 --- a/connected.h +++ b/connected.h @@ -46,6 +46,13 @@ struct check_connected_options { * during a fetch. */ unsigned is_deepening_fetch : 1; + + /* + * If not NULL, use `--exclude-hidden=$section` to exclude all refs + * hidden via the `$section.hideRefs` config from the set of + * already-reachable refs. + */ + const char *exclude_hidden_refs_section; }; #define CHECK_CONNECTED_INIT { 0 }