From patchwork Tue May 28 06:31:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13676165 Received: from fhigh3-smtp.messagingengine.com (fhigh3-smtp.messagingengine.com [103.168.172.154]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 40D6E8F48 for ; Tue, 28 May 2024 06:31:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.154 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716877880; cv=none; b=VOvUTPFmb++tI+o6yb+fCKA/zJTDYf/jNgNRRFxdQ9OPfQhufiRQ/38pSFNFBNUyrI4RlmyzGBVrcXSRHfmQy7rCc4mCJL2j1HXeukmqPz+YDDO4lPqF3WGHbhupI5maScFE8SZoSNK+InjbHWZj2GmavZ9OKpiKtL8W9eRFgjA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716877880; c=relaxed/simple; bh=2stneM+1FbI5Dcv9aozRKngsDk9dZupO8nJlWDE+PWc=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=pF3v3vXA6LX+VQLRs+fEyrPfcBrXUq+XqxvhrgGq23QzjdUscwQjjLmqnpu7ILYQLdmkETdcmYjwQvfqzSW47WCSe3sEtyNsY9oaRGHcE4I8HltXGGjVDOxAbebtOUbnHAGgc3RQzYBOzbaw+CSM8xHsOmBdiBpplKljzVXJhoY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=D6FTCDO3; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=gXtf+Guh; arc=none smtp.client-ip=103.168.172.154 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="D6FTCDO3"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="gXtf+Guh" Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailfhigh.nyi.internal (Postfix) with ESMTP id 5B6121140101; Tue, 28 May 2024 02:31:18 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Tue, 28 May 2024 02:31:18 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm1; t=1716877878; x=1716964278; bh=ouv2cJr261 +nZpT1zWanSfoBjX8kP/6uFyEfCI8OzcE=; b=D6FTCDO3xfdRNXfb1A3S+yoEah Rdtk97gXP4sq1bVnhqABp/FXPTvk5ZRADs29ScR9+thNg0f62UejJyDpn1xDWJGQ sH/Ac69pHjplXZYxxiwJCZTdygBTY3yPt9LhCxYCBcp6x+QJZ0LbQG8KV69/tE+A ZLfjtkTZA2S2mZVvFoWpHSehTx90rBwBL+RKcyjPL4oemt3+nalkk6S68uEuGqfu Bert8PxAvTLQ7YuI0K9+U56/P/CFmA/L63mbASDAJBzqqcrNxmUedC4AXA2+NVBu Z3LoJz49J21Y5CZEtyx3E0IFEjUHqE4J3C7T+rGWZEypUTkYH30SuLdHdiLg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1716877878; x=1716964278; bh=ouv2cJr261+nZpT1zWanSfoBjX8k P/6uFyEfCI8OzcE=; b=gXtf+GuhbWLL9DnRjK/ZrS/HIGwlJR+WU1/5Q+9eay0g lDjh5ScbI+a2XEIgU3Mxd4KR4GB6pqSrJn90I927euM+6irDgAzPp3nWq6bWzVnb kPYeUfOfJv1/B/ePlRZp18q6MNPAjbp5HEv9ufEidMBfb5RXPo+BJaa4F0zT3zE3 9EBzuPm3Wv4wS1vN2SlyRjgQHGEUpsE0h0G+FaYZm/zPHsOGF92FlLMZZcjPuSK4 p1oiL/mKl5E71gyxigc2+alESmpfMNsDoEfnXjUsH9l5zfhGUUwIG/r5KRgP/co3 /50Re48BObJ1/AxASiF8jQxlbihW1avBesLRUzGV8g== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdejhedguddtiecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpeffhffvvefukfhfgggtuggjsehgtderredttddvnecuhfhrohhmpefrrght rhhitghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrihhmqeenucggtffrrghtth gvrhhnpeeukedtvedtffevleejtefgheehieegkeeluddvfeefgeehgfeltddtheejleff teenucevlhhushhtvghrufhiiigvpedunecurfgrrhgrmhepmhgrihhlfhhrohhmpehpsh esphhkshdrihhm X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 28 May 2024 02:31:16 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id e54423b6 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Tue, 28 May 2024 06:31:05 +0000 (UTC) Date: Tue, 28 May 2024 08:31:14 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Eric Sunshine , Junio C Hamano , Ramsay Jones , Justin Tobler Subject: [PATCH v3 01/12] setup: unset ref storage when reinitializing repository version Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: When reinitializing a repository's version we may end up unsetting the hash algorithm when it matches the default hash algorithm. If we didn't do that then the previously configured value might remain intact. While the same issue exists for the ref storage extension, we don't do this here. This has been fine for most of the part because it is not supported to re-initialize a repository with a different ref storage format anyway. We're about to introduce a new command to migrate ref storages though, so this is about to become an issue there. Prepare for this and unset the ref storage format when reinitializing a repository with the "files" format. Signed-off-by: Patrick Steinhardt --- setup.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/setup.c b/setup.c index 7975230ffb..8c84ec9d4b 100644 --- a/setup.c +++ b/setup.c @@ -2028,6 +2028,8 @@ void initialize_repository_version(int hash_algo, if (ref_storage_format != REF_STORAGE_FORMAT_FILES) git_config_set("extensions.refstorage", ref_storage_format_to_name(ref_storage_format)); + else if (reinit) + git_config_set_gently("extensions.refstorage", NULL); } static int is_reinit(void) From patchwork Tue May 28 06:31:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13676166 Received: from fhigh3-smtp.messagingengine.com (fhigh3-smtp.messagingengine.com [103.168.172.154]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DB1628F48 for ; Tue, 28 May 2024 06:31:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.154 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716877885; cv=none; b=iIF7cOvlJ//7MomPOqxOK1sJKeAaE6fVzhcIwzEyeBoW8QTk5RKwPUlOZa8pQwICSh8RK84dynC5KEah3TvHKoGYZZaaoO08IGX6DGhOsCYjUbM4VaVm4isKf1LttXajpQiMZXNKogH3PvsT6EfH0rMSCWnu28qKQXx8zpVtJB8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716877885; c=relaxed/simple; bh=CkHKfHecGorTp9N5BTWAWpswkW4Sh1icE5PKbKW8th8=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=moPy2nY/Tx59guJF3k/2OrmTYB62/AhjL99uTegljvApjwX9/6Rtu4L11rTWaW3oQivKrOKfZfTpPHuqZi6SDtXKs1YUjmv9rKIBLPbOa9Y4hwgKFyVxVZq6XjKeNDQcWWr4aw25ddXaIBWdAWPdq6JizyMZAKTj1KHZZCe7C4A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=oJVR6y0/; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=k8VHQy0N; arc=none smtp.client-ip=103.168.172.154 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="oJVR6y0/"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="k8VHQy0N" Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailfhigh.nyi.internal (Postfix) with ESMTP id D974B11400AF; Tue, 28 May 2024 02:31:22 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Tue, 28 May 2024 02:31:22 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm1; t=1716877882; x=1716964282; bh=UBd586fuFL siuMknaZkUi5d9i4IYO1TZEFA9oIyVcQU=; b=oJVR6y0/uW7dYGuQtQdyTDp0KL iiOrGhzjdehMY4u0ltGumymo/WWRXV5s9sIB+XJbWbhtFhu7x5eEolgvzGZReUgA ya2OXATPO93SkexG3DibXD2R+4a2f63aYXTEPeXktVYpFO2NUFkSfrQOHNHS0JtM IQGz/7xtiNnPBavWH6EppbyB7ZF3Wf6xp8gNIv8A3Qkn6b7uJ0+fPjTjM3oxHeh8 AJ6Ynoa9WGUb2uIaNYe1khwR3s3b48IfL/F+qPv+oi7KVxERQ09EQjV1nZBH91Zp B+30vV0gO64dSm5/xenIjnizrBamLUZgoERnJKzfSmdESrafpABtdnrV8C0w== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1716877882; x=1716964282; bh=UBd586fuFLsiuMknaZkUi5d9i4IY O1TZEFA9oIyVcQU=; b=k8VHQy0NW6STldyHdF3fk4zm/5svTmKevcTupZMAGbch 7lsewZNN4aie8GkPiM9FlqCgfaylFPpf61eXscnZRZh547DSN7LDL5qUry21YoT/ c3c6tqBcuQ6VEA5hEkzEWEUsoohFDIujmSvcqTVkmrA/MQsfYYIIx2OZdorFB7Xa McfAq4AFL+2plqCLuz4LMxdpkbKK2bqKpsptCK/8ni7qqJ3y4hV5uL5r3J+9OhBD 08yAObsX7jDQzEUtd4TyxG8iDDXB4/lwi3FTYNd53LczXb3vRzXPjnYHjtdIgyyF 3E07UzL7aiGlLwRIfaIBVNIRJ5OUKp3aUFPsjOThXw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdejhedguddtjecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpeffhffvvefukfhfgggtuggjsehgtderredttddvnecuhfhrohhmpefrrght rhhitghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrihhmqeenucggtffrrghtth gvrhhnpeeukedtvedtffevleejtefgheehieegkeeluddvfeefgeehgfeltddtheejleff teenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehpsh esphhkshdrihhm X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 28 May 2024 02:31:21 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id 252152e8 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Tue, 28 May 2024 06:31:10 +0000 (UTC) Date: Tue, 28 May 2024 08:31:19 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Eric Sunshine , Junio C Hamano , Ramsay Jones , Justin Tobler Subject: [PATCH v3 02/12] refs: convert ref storage format to an enum Message-ID: <7989e82dcd89c92f23de7ab4ebde96124aa6fe12.1716877224.git.ps@pks.im> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: The ref storage format is tracked as a simple unsigned integer, which makes it harder than necessary to discover what that integer actually is or where its values are defined. Convert the ref storage format to instead be an enum. Signed-off-by: Patrick Steinhardt --- builtin/clone.c | 2 +- builtin/init-db.c | 2 +- refs.c | 7 ++++--- refs.h | 10 ++++++++-- repository.c | 3 ++- repository.h | 10 ++++------ setup.c | 8 ++++---- setup.h | 9 +++++---- 8 files changed, 29 insertions(+), 22 deletions(-) diff --git a/builtin/clone.c b/builtin/clone.c index 1e07524c53..e808e02017 100644 --- a/builtin/clone.c +++ b/builtin/clone.c @@ -970,7 +970,7 @@ int cmd_clone(int argc, const char **argv, const char *prefix) int submodule_progress; int filter_submodules = 0; int hash_algo; - unsigned int ref_storage_format = REF_STORAGE_FORMAT_UNKNOWN; + enum ref_storage_format ref_storage_format = REF_STORAGE_FORMAT_UNKNOWN; const int do_not_override_repo_unix_permissions = -1; const char *template_dir; char *template_dir_dup = NULL; diff --git a/builtin/init-db.c b/builtin/init-db.c index 0170469b84..582dcf20f8 100644 --- a/builtin/init-db.c +++ b/builtin/init-db.c @@ -81,7 +81,7 @@ int cmd_init_db(int argc, const char **argv, const char *prefix) const char *ref_format = NULL; const char *initial_branch = NULL; int hash_algo = GIT_HASH_UNKNOWN; - unsigned int ref_storage_format = REF_STORAGE_FORMAT_UNKNOWN; + enum ref_storage_format ref_storage_format = REF_STORAGE_FORMAT_UNKNOWN; int init_shared_repository = -1; const struct option init_db_options[] = { OPT_STRING(0, "template", &template_dir, N_("template-directory"), diff --git a/refs.c b/refs.c index 31032588e0..e6db85a165 100644 --- a/refs.c +++ b/refs.c @@ -37,14 +37,15 @@ static const struct ref_storage_be *refs_backends[] = { [REF_STORAGE_FORMAT_REFTABLE] = &refs_be_reftable, }; -static const struct ref_storage_be *find_ref_storage_backend(unsigned int ref_storage_format) +static const struct ref_storage_be *find_ref_storage_backend( + enum ref_storage_format ref_storage_format) { if (ref_storage_format < ARRAY_SIZE(refs_backends)) return refs_backends[ref_storage_format]; return NULL; } -unsigned int ref_storage_format_by_name(const char *name) +enum ref_storage_format ref_storage_format_by_name(const char *name) { for (unsigned int i = 0; i < ARRAY_SIZE(refs_backends); i++) if (refs_backends[i] && !strcmp(refs_backends[i]->name, name)) @@ -52,7 +53,7 @@ unsigned int ref_storage_format_by_name(const char *name) return REF_STORAGE_FORMAT_UNKNOWN; } -const char *ref_storage_format_to_name(unsigned int ref_storage_format) +const char *ref_storage_format_to_name(enum ref_storage_format ref_storage_format) { const struct ref_storage_be *be = find_ref_storage_backend(ref_storage_format); if (!be) diff --git a/refs.h b/refs.h index fe7f0db35e..a7afa9bede 100644 --- a/refs.h +++ b/refs.h @@ -11,8 +11,14 @@ struct string_list; struct string_list_item; struct worktree; -unsigned int ref_storage_format_by_name(const char *name); -const char *ref_storage_format_to_name(unsigned int ref_storage_format); +enum ref_storage_format { + REF_STORAGE_FORMAT_UNKNOWN, + REF_STORAGE_FORMAT_FILES, + REF_STORAGE_FORMAT_REFTABLE, +}; + +enum ref_storage_format ref_storage_format_by_name(const char *name); +const char *ref_storage_format_to_name(enum ref_storage_format ref_storage_format); /* * Resolve a reference, recursively following symbolic refererences. diff --git a/repository.c b/repository.c index d29b0304fb..166863f852 100644 --- a/repository.c +++ b/repository.c @@ -124,7 +124,8 @@ void repo_set_compat_hash_algo(struct repository *repo, int algo) repo_read_loose_object_map(repo); } -void repo_set_ref_storage_format(struct repository *repo, unsigned int format) +void repo_set_ref_storage_format(struct repository *repo, + enum ref_storage_format format) { repo->ref_storage_format = format; } diff --git a/repository.h b/repository.h index 4bd8969005..a35cd77c35 100644 --- a/repository.h +++ b/repository.h @@ -1,6 +1,7 @@ #ifndef REPOSITORY_H #define REPOSITORY_H +#include "refs.h" #include "strmap.h" struct config_set; @@ -26,10 +27,6 @@ enum fetch_negotiation_setting { FETCH_NEGOTIATION_NOOP, }; -#define REF_STORAGE_FORMAT_UNKNOWN 0 -#define REF_STORAGE_FORMAT_FILES 1 -#define REF_STORAGE_FORMAT_REFTABLE 2 - struct repo_settings { int initialized; @@ -181,7 +178,7 @@ struct repository { const struct git_hash_algo *compat_hash_algo; /* Repository's reference storage format, as serialized on disk. */ - unsigned int ref_storage_format; + enum ref_storage_format ref_storage_format; /* A unique-id for tracing purposes. */ int trace2_repo_id; @@ -220,7 +217,8 @@ void repo_set_gitdir(struct repository *repo, const char *root, void repo_set_worktree(struct repository *repo, const char *path); void repo_set_hash_algo(struct repository *repo, int algo); void repo_set_compat_hash_algo(struct repository *repo, int compat_algo); -void repo_set_ref_storage_format(struct repository *repo, unsigned int format); +void repo_set_ref_storage_format(struct repository *repo, + enum ref_storage_format format); void initialize_repository(struct repository *repo); RESULT_MUST_BE_USED int repo_init(struct repository *r, const char *gitdir, const char *worktree); diff --git a/setup.c b/setup.c index 8c84ec9d4b..b49ee3e95f 100644 --- a/setup.c +++ b/setup.c @@ -1997,7 +1997,7 @@ static int needs_work_tree_config(const char *git_dir, const char *work_tree) } void initialize_repository_version(int hash_algo, - unsigned int ref_storage_format, + enum ref_storage_format ref_storage_format, int reinit) { char repo_version_string[10]; @@ -2044,7 +2044,7 @@ static int is_reinit(void) return ret; } -void create_reference_database(unsigned int ref_storage_format, +void create_reference_database(enum ref_storage_format ref_storage_format, const char *initial_branch, int quiet) { struct strbuf err = STRBUF_INIT; @@ -2243,7 +2243,7 @@ static void validate_hash_algorithm(struct repository_format *repo_fmt, int hash } static void validate_ref_storage_format(struct repository_format *repo_fmt, - unsigned int format) + enum ref_storage_format format) { const char *name = getenv("GIT_DEFAULT_REF_FORMAT"); @@ -2263,7 +2263,7 @@ static void validate_ref_storage_format(struct repository_format *repo_fmt, int init_db(const char *git_dir, const char *real_git_dir, const char *template_dir, int hash, - unsigned int ref_storage_format, + enum ref_storage_format ref_storage_format, const char *initial_branch, int init_shared_repository, unsigned int flags) { diff --git a/setup.h b/setup.h index b3fd3bf45a..cd8dbc2497 100644 --- a/setup.h +++ b/setup.h @@ -1,6 +1,7 @@ #ifndef SETUP_H #define SETUP_H +#include "refs.h" #include "string-list.h" int is_inside_git_dir(void); @@ -128,7 +129,7 @@ struct repository_format { int is_bare; int hash_algo; int compat_hash_algo; - unsigned int ref_storage_format; + enum ref_storage_format ref_storage_format; int sparse_index; char *work_tree; struct string_list unknown_extensions; @@ -192,13 +193,13 @@ const char *get_template_dir(const char *option_template); int init_db(const char *git_dir, const char *real_git_dir, const char *template_dir, int hash_algo, - unsigned int ref_storage_format, + enum ref_storage_format ref_storage_format, const char *initial_branch, int init_shared_repository, unsigned int flags); void initialize_repository_version(int hash_algo, - unsigned int ref_storage_format, + enum ref_storage_format ref_storage_format, int reinit); -void create_reference_database(unsigned int ref_storage_format, +void create_reference_database(enum ref_storage_format ref_storage_format, const char *initial_branch, int quiet); /* From patchwork Tue May 28 06:31:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13676167 Received: from fhigh3-smtp.messagingengine.com (fhigh3-smtp.messagingengine.com [103.168.172.154]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7483C8F48 for ; Tue, 28 May 2024 06:31:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.154 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716877891; cv=none; b=GhFs7yRPGG2FctMaQUVc6nf7fVukjLlY1OpGLyHJWY8yhJ0fUAPgUCnb/IWZ5dZSE+fyV6D3NF2k4V079DwVa/fWVh0BMzfrtdV7Yq5gdQTfRSO8fK9ATNqbOPekIvNfcyadxXJ+A7UUta9wCxdZt+d+kRfw8lLgokSkI11R/tA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716877891; c=relaxed/simple; bh=ncJi5SG/hCtAMgfh3v0Hi9ICYgr0SMqJtEVtPUQzRHk=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=chMHB/6nhWJmViTjVPaTnBKUTHYBydSrQcqhvD6LkHHVvlkQmV2QWj+hdj5T3X4ynomzVno77Y5Axs0s6PtFkr4rTgAmRPMgLWd6T/v4i4F0rNyycfExSf59tRJ2A6swNliS85Xzct+FzvnY7MmMYQaIEgmCkH+qYmQ++2xeT6o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=mLvCfnlT; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=KWOVlINX; arc=none smtp.client-ip=103.168.172.154 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="mLvCfnlT"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="KWOVlINX" Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailfhigh.nyi.internal (Postfix) with ESMTP id 70C571140097; Tue, 28 May 2024 02:31:28 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Tue, 28 May 2024 02:31:28 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm1; t=1716877888; x=1716964288; bh=9zit0z297j uSdpm+9/RhIkMiNGITMBZBn2guNDJtXxM=; b=mLvCfnlTfRk5EQZ5cApm4fgoiq dfL1DzyChgaWWd7GgufR1GjHIHNwI9/f6J5G5t1Br2j7Lwmhmrmut3crFqv3JYm1 zItCQFgtHdVAxJZY6GwkIYHZ6FmNV6kMNttwf6zmkS8bJ5IlWbXVqTX5S8TKX6zI TcfZEkoWVLaFR9t7SwSDnO8K/dK4S9XhNbzwC2Yfgj0ZJ83F0EHmOcSQcY3E1ymz M0wHRchOFCtsBs9NOYRqwczXcsFUoLi9AtRdJ/lvw/NvkM09aUq9701eGMt6dlcz IHTXw9pHwZRzrFi1Eo165McPNE1I2SboYJhhdoYDPDBjTZ/PWTFKHy3pd3lw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1716877888; x=1716964288; bh=9zit0z297juSdpm+9/RhIkMiNGIT MBZBn2guNDJtXxM=; b=KWOVlINXV1nvrwb6UPI18Hukc4RrV4t9CIjnD+Eq/EV4 Yjta42S8xjwg23bVkdBBUQjsCXVIrHHCkwBh0IEvoiPqc3MM7eBTuynGDeVpXeoj JkPDXAcZ2KAjW70u8wve7hMQGCXPoi+Fe63A6VvAlOFwlrBoQVNyoGNlp2DPXq0q rXnTfr0r/vLvVGp+74cJgGQr0CZeUGMpbYoSSTR5fMCkCQ9hbJrCdGx56a65lY8K 8S4oWQ3Nzh/1i4E0VEkgxEoq3iKSngpfIFcJsav4ErkSO/ROTcXBATcxi1b+amnm Ny2PV9U5xOuhIoWuQzMjbZxWcFdD3GMoJ1zrYAPzFQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdejhedguddtiecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpeffhffvvefukfhfgggtuggjsehgtderredttddvnecuhfhrohhmpefrrght rhhitghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrihhmqeenucggtffrrghtth gvrhhnpeeukedtvedtffevleejtefgheehieegkeeluddvfeefgeehgfeltddtheejleff teenucevlhhushhtvghrufhiiigvpedvnecurfgrrhgrmhepmhgrihhlfhhrohhmpehpsh esphhkshdrihhm X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 28 May 2024 02:31:27 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id 81d258d2 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Tue, 28 May 2024 06:31:15 +0000 (UTC) Date: Tue, 28 May 2024 08:31:24 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Eric Sunshine , Junio C Hamano , Ramsay Jones , Justin Tobler Subject: [PATCH v3 03/12] refs: pass storage format to `ref_store_init()` explicitly Message-ID: <7d1a86292c205bb47bffa93aa293a3575836f9fb.1716877224.git.ps@pks.im> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: We're about to introduce logic to migrate refs from one storage format to another one. This will require us to initialize a ref store with a different format than the one used by the passed-in repository. Prepare for this by accepting the desired ref storage format as parameter. Signed-off-by: Patrick Steinhardt --- refs.c | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/refs.c b/refs.c index e6db85a165..7c3f4df457 100644 --- a/refs.c +++ b/refs.c @@ -1894,13 +1894,14 @@ static struct ref_store *lookup_ref_store_map(struct strmap *map, * gitdir. */ static struct ref_store *ref_store_init(struct repository *repo, + enum ref_storage_format format, const char *gitdir, unsigned int flags) { const struct ref_storage_be *be; struct ref_store *refs; - be = find_ref_storage_backend(repo->ref_storage_format); + be = find_ref_storage_backend(format); if (!be) BUG("reference backend is unknown"); @@ -1922,7 +1923,8 @@ struct ref_store *get_main_ref_store(struct repository *r) if (!r->gitdir) BUG("attempting to get main_ref_store outside of repository"); - r->refs_private = ref_store_init(r, r->gitdir, REF_STORE_ALL_CAPS); + r->refs_private = ref_store_init(r, r->ref_storage_format, + r->gitdir, REF_STORE_ALL_CAPS); r->refs_private = maybe_debug_wrap_ref_store(r->gitdir, r->refs_private); return r->refs_private; } @@ -1982,7 +1984,8 @@ struct ref_store *repo_get_submodule_ref_store(struct repository *repo, free(subrepo); goto done; } - refs = ref_store_init(subrepo, submodule_sb.buf, + refs = ref_store_init(subrepo, the_repository->ref_storage_format, + submodule_sb.buf, REF_STORE_READ | REF_STORE_ODB); register_ref_store_map(&repo->submodule_ref_stores, "submodule", refs, submodule); @@ -2011,12 +2014,12 @@ struct ref_store *get_worktree_ref_store(const struct worktree *wt) struct strbuf common_path = STRBUF_INIT; strbuf_git_common_path(&common_path, wt->repo, "worktrees/%s", wt->id); - refs = ref_store_init(wt->repo, common_path.buf, - REF_STORE_ALL_CAPS); + refs = ref_store_init(wt->repo, wt->repo->ref_storage_format, + common_path.buf, REF_STORE_ALL_CAPS); strbuf_release(&common_path); } else { - refs = ref_store_init(wt->repo, wt->repo->commondir, - REF_STORE_ALL_CAPS); + refs = ref_store_init(wt->repo, the_repository->ref_storage_format, + wt->repo->commondir, REF_STORE_ALL_CAPS); } if (refs) From patchwork Tue May 28 06:31:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13676168 Received: from fout5-smtp.messagingengine.com (fout5-smtp.messagingengine.com [103.168.172.148]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C9F5261FD9 for ; Tue, 28 May 2024 06:31:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.148 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716877895; cv=none; b=qtdeINzns+qZ1NhGGOqhUyBNDFuJkUxlOVAonPV9lrL2ZnFJxFxV68fyE5g3tKF77ApNFzBavWvOLTnBXrweGwTDkDG1yRgz7PDKWLN/yw9/o/B23ILMYh6dRB5nMRudY82THs2UyTZqfG+30oZpAZO21UjVUHSfHFI+WhTDzco= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716877895; c=relaxed/simple; bh=aahlWehhd3PjFBs+oXFWyRRmv+tBK1McDj8kEVskpVM=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=U/TJXGIYKLPIdpmLvgGtFwtU/N9prp2rm8asRM2ddyNyQbEw7fwG0a9pkD/cJokXlwyczbSRGgOuWOgK1VlICK0I5qaXdQhbF+gVhqnH0LzeZccFAvhIwmYL48bMdrFiKa3V1w4EzFfj1e8aFQdD8av+lYbUBWRPzWV5DSBkeFI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=cbtHedKm; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=X8j6q8Db; arc=none smtp.client-ip=103.168.172.148 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="cbtHedKm"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="X8j6q8Db" Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailfout.nyi.internal (Postfix) with ESMTP id E2AEF13800A6; Tue, 28 May 2024 02:31:32 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Tue, 28 May 2024 02:31:32 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm1; t=1716877892; x=1716964292; bh=qPvNXdojrv pZr/XubcBo061Oa8wmJWPmTTBLJXDKCAA=; b=cbtHedKmA1CMPez7q/lxnv4cSm W7b53Y4BgDTConS+TOGc4HrZEqYXvxlq6hBcBFDlQHct9mqqdVLxRQ+nXxcRBcb+ 41iY0GT1gWWWUBsH5UsepoPF6mSRoScfUgM98PO7IXVelwNUri+Dwk3zoMc6hMNh rZSGzjcgP6oPLi6OqTBRHERkteR+TMWuz2mTVK8wYttEOAXMuBHZt8gonjmmu7I/ yjGCt5+tqkmz2lPs5rvGNkkLXg/9z9JISiybvII5feTORLDgKH3GSYbve6Mq3AON G9I84yAXjwAiBKjpKYZ5vJjDMrZ1xLsFuUOGa9L0QzjSwLkbFKNow4jrNbOg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1716877892; x=1716964292; bh=qPvNXdojrvpZr/XubcBo061Oa8wm JWPmTTBLJXDKCAA=; b=X8j6q8DbLmFXLrCv1fXLhGR/kKR8Yx6SW3+4btXSHam4 gfF82iPEuQPdkzBZ8fbOzktxLeavtPMcV/jHYiu3DCH7QVuj8E/87+6i7I3st0GN DFSWN6DaBVCTUGHQ5QwUWd+yYD7sWYZD/Af6ShT3AdmmKD4FSVGgm3yN1eFtjEbz W1NI9YnACMnvnAWZAFiLPaSqOJ10exRq+2zs6mALAZgArqzS7OElU0tgySp8ZsEY fB1wsBBM2RVSxHT0t7O+ChvbHKSxlBZDYu4sBqk65zdk8yW4OP68Na5CxHjUl0BC Q7YC51oXG0epijYRro8OLCE0yA8s1QW1MVtyMjOHYg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdejhedguddtjecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpeffhffvvefukfhfgggtuggjsehgtderredttddvnecuhfhrohhmpefrrght rhhitghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrihhmqeenucggtffrrghtth gvrhhnpeeukedtvedtffevleejtefgheehieegkeeluddvfeefgeehgfeltddtheejleff teenucevlhhushhtvghrufhiiigvpedunecurfgrrhgrmhepmhgrihhlfhhrohhmpehpsh esphhkshdrihhm X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 28 May 2024 02:31:31 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id 31f9c575 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Tue, 28 May 2024 06:31:19 +0000 (UTC) Date: Tue, 28 May 2024 08:31:29 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Eric Sunshine , Junio C Hamano , Ramsay Jones , Justin Tobler Subject: [PATCH v3 04/12] refs: allow to skip creation of reflog entries Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: The ref backends do not have any way to disable the creation of reflog entries. This will be required for upcoming ref format migration logic so that we do not create any entries that didn't exist in the original ref database. Provide a new `REF_SKIP_CREATE_REFLOG` flag that allows the caller to disable reflog entry creation. Signed-off-by: Patrick Steinhardt --- refs.c | 6 ++++++ refs.h | 8 +++++++- refs/files-backend.c | 4 ++++ refs/reftable-backend.c | 3 ++- t/helper/test-ref-store.c | 1 + 5 files changed, 20 insertions(+), 2 deletions(-) diff --git a/refs.c b/refs.c index 7c3f4df457..66e9585767 100644 --- a/refs.c +++ b/refs.c @@ -1194,6 +1194,12 @@ int ref_transaction_update(struct ref_transaction *transaction, { assert(err); + if ((flags & REF_FORCE_CREATE_REFLOG) && + (flags & REF_SKIP_CREATE_REFLOG)) { + strbuf_addstr(err, _("refusing to force and skip creation of reflog")); + return -1; + } + if (!(flags & REF_SKIP_REFNAME_VERIFICATION) && ((new_oid && !is_null_oid(new_oid)) ? check_refname_format(refname, REFNAME_ALLOW_ONELEVEL) : diff --git a/refs.h b/refs.h index a7afa9bede..50a2b3ab09 100644 --- a/refs.h +++ b/refs.h @@ -659,13 +659,19 @@ struct ref_transaction *ref_store_transaction_begin(struct ref_store *refs, */ #define REF_SKIP_REFNAME_VERIFICATION (1 << 11) +/* + * Skip creation of a reflog entry, even if it would have otherwise been + * created. + */ +#define REF_SKIP_CREATE_REFLOG (1 << 12) + /* * Bitmask of all of the flags that are allowed to be passed in to * ref_transaction_update() and friends: */ #define REF_TRANSACTION_UPDATE_ALLOWED_FLAGS \ (REF_NO_DEREF | REF_FORCE_CREATE_REFLOG | REF_SKIP_OID_VERIFICATION | \ - REF_SKIP_REFNAME_VERIFICATION) + REF_SKIP_REFNAME_VERIFICATION | REF_SKIP_CREATE_REFLOG) /* * Add a reference update to transaction. `new_oid` is the value that diff --git a/refs/files-backend.c b/refs/files-backend.c index 73380d7e99..bd0d63bcba 100644 --- a/refs/files-backend.c +++ b/refs/files-backend.c @@ -1750,6 +1750,9 @@ static int files_log_ref_write(struct files_ref_store *refs, { int logfd, result; + if (flags & REF_SKIP_CREATE_REFLOG) + return 0; + if (log_all_ref_updates == LOG_REFS_UNSET) log_all_ref_updates = is_bare_repository() ? LOG_REFS_NONE : LOG_REFS_NORMAL; @@ -2251,6 +2254,7 @@ static int split_head_update(struct ref_update *update, struct ref_update *new_update; if ((update->flags & REF_LOG_ONLY) || + (update->flags & REF_SKIP_CREATE_REFLOG) || (update->flags & REF_IS_PRUNING) || (update->flags & REF_UPDATE_VIA_HEAD)) return 0; diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c index f6edfdf5b3..bffed9257f 100644 --- a/refs/reftable-backend.c +++ b/refs/reftable-backend.c @@ -1103,7 +1103,8 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data if (ret) goto done; - } else if (u->flags & REF_HAVE_NEW && + } else if (!(u->flags & REF_SKIP_CREATE_REFLOG) && + (u->flags & REF_HAVE_NEW) && (u->flags & REF_FORCE_CREATE_REFLOG || should_write_log(&arg->refs->base, u->refname))) { struct reftable_log_record *log; diff --git a/t/helper/test-ref-store.c b/t/helper/test-ref-store.c index c9efd74c2b..ad24300170 100644 --- a/t/helper/test-ref-store.c +++ b/t/helper/test-ref-store.c @@ -126,6 +126,7 @@ static struct flag_definition transaction_flags[] = { FLAG_DEF(REF_FORCE_CREATE_REFLOG), FLAG_DEF(REF_SKIP_OID_VERIFICATION), FLAG_DEF(REF_SKIP_REFNAME_VERIFICATION), + FLAG_DEF(REF_SKIP_CREATE_REFLOG), { NULL, 0 } }; From patchwork Tue May 28 06:31:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13676169 Received: from fhigh3-smtp.messagingengine.com (fhigh3-smtp.messagingengine.com [103.168.172.154]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 549F861FD9 for ; Tue, 28 May 2024 06:31:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.154 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716877899; cv=none; b=LjSZehifZkjwSpdXJ5oQT2yN/DCwVkRoN6b662UKcpXm17Zbf8oe8peq/U+NRma/m8kTjajRNdFXjctcHY5KtLVl6wGXXh+zOMI1VHJXqw2v5a9R9rLePgBP5OJpvwYOpX2ge0JoF2dX0JMyXQ72l0WN68HARHNMjZ1gy+MKZvs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716877899; c=relaxed/simple; bh=tIKKfkl+zVN5uIyHdHz/LW1ix6M2nEQ/QvFCB1G0wfQ=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=t5AfnRY6A1tlKnG7iD1hGGwwdqg5cr2XHKbpXiEwFPI+i0g8seV+TfSBRxw0RoloLquRWltNakrn/a/0Uas37h4jxRgOumkJRjZO1q7RNRnvMZ8TH++naBuznh2vl1qeKIcGgiT/Mos/srNIpNKK5lO8G+3+M3eLPs5RW77KjeY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=MiYswOB8; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=CNlgf+lY; arc=none smtp.client-ip=103.168.172.154 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="MiYswOB8"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="CNlgf+lY" Received: from compute6.internal (compute6.nyi.internal [10.202.2.47]) by mailfhigh.nyi.internal (Postfix) with ESMTP id 6D2A21140081; Tue, 28 May 2024 02:31:37 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute6.internal (MEProxy); Tue, 28 May 2024 02:31:37 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm1; t=1716877897; x=1716964297; bh=TQx252N5ml VC7UBBN6DGXps2ibKc8Pw1jQNc8MBnZGU=; b=MiYswOB8eL5CQrLi6jKy65MgzT mRT63zj5os0c6mDto6bbr1EXFyhTQbO2TeHTpf+Oluxl/T3dAZQZpdhflO1Apth/ 88yMne1Eg7DV5xhzenOlSdp1YWYW7iulzexJ8h3cumCakY9N2Qt6ovW3S8TSRKJC InHKPS7XFUDel2NE+cNfegpRp2P9pROnk+0EBgij0BlANoAuPhapAmRaQt2CMSew iE6P2CzSFeab85KJsesEW6XMwr70ewoXy7j9wLXoIKcKUzMy3fYcydkK7H7NoQB6 AtgXCP4w37NIThFsvRZ8UClsufGu2NO74wcm/qT9w4ezh8ZVBM4VFzGfRW5A== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1716877897; x=1716964297; bh=TQx252N5mlVC7UBBN6DGXps2ibKc 8Pw1jQNc8MBnZGU=; b=CNlgf+lYZtV6f9TucYDRDehJtFOFFrtLXmna5dJFr2Pn MWgAQcvDDDnYD7XXlOGob+QkaNJgR7RLDGXquYhrRfHZNzp605WQeDc55jQPrQLG p+t2y5du2R5HmexnCSvvF4jS1L+9yQT7Xg6BfLchXKpwbUh1zFiKb+Sjf1SnLKyB mhSS878gxGvjyjFyTtRkmHTqgURVMK7vmPmfd2B/XTG2jO42nQofC3Rd8AbI5Rc/ YhVkowCwhZTHEpzZUXTEJu77KFzR1+i7Uyjy628Z7dVd22KkK5Q6YixQNYnYKhVH TiY3VYymYxYMxJTcbD8rT1SGOezWHNGtlaVLIpkUKg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdejhedguddtjecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpeffhffvvefukfhfgggtuggjsehgtderredttddvnecuhfhrohhmpefrrght rhhitghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrihhmqeenucggtffrrghtth gvrhhnpeeukedtvedtffevleejtefgheehieegkeeluddvfeefgeehgfeltddtheejleff teenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehpsh esphhkshdrihhm X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 28 May 2024 02:31:36 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id 200c9986 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Tue, 28 May 2024 06:31:24 +0000 (UTC) Date: Tue, 28 May 2024 08:31:34 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Eric Sunshine , Junio C Hamano , Ramsay Jones , Justin Tobler Subject: [PATCH v3 05/12] refs/files: refactor `add_pseudoref_and_head_entries()` Message-ID: <7f9ce5af2eb316477160224aa087ba6290cad672.1716877224.git.ps@pks.im> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: The `add_pseudoref_and_head_entries()` function accepts both the ref store as well as a directory name as input. This is unnecessary though as the ref store already uniquely identifies the root directory of the ref store anyway. Furthermore, the function is misnamed now that we have clarified the meaning of pseudorefs as it doesn't add pseudorefs, but root refs. Rename it accordingly. Signed-off-by: Patrick Steinhardt --- refs/files-backend.c | 15 ++++++--------- 1 file changed, 6 insertions(+), 9 deletions(-) diff --git a/refs/files-backend.c b/refs/files-backend.c index bd0d63bcba..b4e5437ffe 100644 --- a/refs/files-backend.c +++ b/refs/files-backend.c @@ -324,16 +324,14 @@ static void loose_fill_ref_dir(struct ref_store *ref_store, } /* - * Add pseudorefs to the ref dir by parsing the directory for any files - * which follow the pseudoref syntax. + * Add root refs to the ref dir by parsing the directory for any files which + * follow the root ref syntax. */ -static void add_pseudoref_and_head_entries(struct ref_store *ref_store, - struct ref_dir *dir, - const char *dirname) +static void add_root_refs(struct files_ref_store *refs, + struct ref_dir *dir) { - struct files_ref_store *refs = - files_downcast(ref_store, REF_STORE_READ, "fill_ref_dir"); struct strbuf path = STRBUF_INIT, refname = STRBUF_INIT; + const char *dirname = refs->loose->root->name; struct dirent *de; size_t dirnamelen; DIR *d; @@ -388,8 +386,7 @@ static struct ref_cache *get_loose_ref_cache(struct files_ref_store *refs, dir = get_ref_dir(refs->loose->root); if (flags & DO_FOR_EACH_INCLUDE_ROOT_REFS) - add_pseudoref_and_head_entries(dir->cache->ref_store, dir, - refs->loose->root->name); + add_root_refs(refs, dir); /* * Add an incomplete entry for "refs/" (to be filled From patchwork Tue May 28 06:31:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13676170 Received: from fhigh3-smtp.messagingengine.com (fhigh3-smtp.messagingengine.com [103.168.172.154]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CD05361FD9 for ; Tue, 28 May 2024 06:31:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.154 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716877905; cv=none; b=K2rA0mVvw+ouSxkl5phZOkuxtSKhD6btSM9MShYE6nvYhkPUr5QOihiwm6Fu7TEq6sv8WE67gdlvE/mhBr9J1aX+GiiijQmClYwxbAabWJQQ4BX8hNmTZKnCGWhMF2FbuqScKlv5fnozAXlI4sey61SCizqSGh0rlarLeuyeB0g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716877905; c=relaxed/simple; bh=wxqrIvHl4mcgF1rWswr01x79mDPh2xAHD9lQ6LY5Hfo=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=MitrQTMHoAYFqLcc5SY7+XDsD9uaPzNElnXJz5OJGz+FCNm+/4U6x8B5scb/7Dp18aKdiHhO+gPWr0bflId0vayLdHhEaCTY3bwbC+wE7RIHmTFsWXnQSnKSO+yejcasb9iNS/UvlDHN1wp4jg6n+aGOVBXuYCa5KEt2jrl3WT4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=HOmsdAkm; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=bKY5U6rA; arc=none smtp.client-ip=103.168.172.154 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="HOmsdAkm"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="bKY5U6rA" Received: from compute6.internal (compute6.nyi.internal [10.202.2.47]) by mailfhigh.nyi.internal (Postfix) with ESMTP id DB3AF1140081; Tue, 28 May 2024 02:31:42 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute6.internal (MEProxy); Tue, 28 May 2024 02:31:42 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm1; t=1716877902; x=1716964302; bh=i6v8j4buon QvYUSr0NORYQlNMMWNXvrzs9C9tw3SREo=; b=HOmsdAkmni2AMpNMGQ02wlTj0T 8gGwAjTxpswVI5nXMkuNEJ1SJh0z1bmlcGH7AVA8WcLQlKlnCULRU/bE4IXiSjvy hBcgtp2OfM0XYR1F5LxPBoQcxkcP3aAfLoqcPEovkiHUyJPvYlRJkXRFGRBdSg0S uZdtFH3FxeRk7oZsYS9xpvCNze3wxaiMstHwQq+gCfnIdZE5NPlwt2yKhedjl0hj 0XN8FtUGMr+SKuiVxaTY+4OrgDKkQN/rlTlUrIeWoVLWXUc913kGa4vnn7IG575T P069bYjyiQhHstnKpPy4evopilMGMAucebwgkwRHqFTZZp8CEYYzYaqNx6hA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1716877902; x=1716964302; bh=i6v8j4buonQvYUSr0NORYQlNMMWN Xvrzs9C9tw3SREo=; b=bKY5U6rA/U5VnQmHYjNt60XRxfaDLLyqLWsv6qlqcO7a x8JMbVctPzzWkRFyCz/RnXJNi36EdAsfgfb2+zkJewLxpgVXXjzsYyLhLvt+vD85 Ug6jF7e8douZFWqbzbl5h3lW9vzI8rUX4Y2vPOludRyvxrpU5xj2kAMtnPTRVcOx tUFReaO4hKQcspQ+APr8hlC0NTcwcnrxLuiskHA0VeF2Al1/ladKBqGFpy1n6V2L 05K9kArSCku6TB81LBBxk+TvX/wqslhh9300moWlibqe79U0H4xJ5JOgJ6P6JAHT lToWO+uLlN3g21GM77z5ysc7vskxjzeGV0tQl754gA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdejhedguddtjecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpeffhffvvefukfhfgggtuggjsehgtderredttddvnecuhfhrohhmpefrrght rhhitghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrihhmqeenucggtffrrghtth gvrhhnpeeukedtvedtffevleejtefgheehieegkeeluddvfeefgeehgfeltddtheejleff teenucevlhhushhtvghrufhiiigvpedunecurfgrrhgrmhepmhgrihhlfhhrohhmpehpsh esphhkshdrihhm X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 28 May 2024 02:31:41 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id 72be3910 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Tue, 28 May 2024 06:31:29 +0000 (UTC) Date: Tue, 28 May 2024 08:31:38 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Eric Sunshine , Junio C Hamano , Ramsay Jones , Justin Tobler Subject: [PATCH v3 06/12] refs/files: extract function to iterate through root refs Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Extract a new function that can be used to iterate through all root refs known to the "files" backend. This will be used in the next commit, where we start to teach ref backends to remove themselves. Signed-off-by: Patrick Steinhardt --- refs/files-backend.c | 49 ++++++++++++++++++++++++++++++++++++-------- 1 file changed, 40 insertions(+), 9 deletions(-) diff --git a/refs/files-backend.c b/refs/files-backend.c index b4e5437ffe..b7268b26c8 100644 --- a/refs/files-backend.c +++ b/refs/files-backend.c @@ -323,17 +323,15 @@ static void loose_fill_ref_dir(struct ref_store *ref_store, add_per_worktree_entries_to_dir(dir, dirname); } -/* - * Add root refs to the ref dir by parsing the directory for any files which - * follow the root ref syntax. - */ -static void add_root_refs(struct files_ref_store *refs, - struct ref_dir *dir) +static int for_each_root_ref(struct files_ref_store *refs, + int (*cb)(const char *refname, void *cb_data), + void *cb_data) { struct strbuf path = STRBUF_INIT, refname = STRBUF_INIT; const char *dirname = refs->loose->root->name; struct dirent *de; size_t dirnamelen; + int ret; DIR *d; files_ref_path(refs, &path, dirname); @@ -341,7 +339,7 @@ static void add_root_refs(struct files_ref_store *refs, d = opendir(path.buf); if (!d) { strbuf_release(&path); - return; + return -1; } strbuf_addstr(&refname, dirname); @@ -357,14 +355,47 @@ static void add_root_refs(struct files_ref_store *refs, strbuf_addstr(&refname, de->d_name); dtype = get_dtype(de, &path, 1); - if (dtype == DT_REG && is_root_ref(de->d_name)) - loose_fill_ref_dir_regular_file(refs, refname.buf, dir); + if (dtype == DT_REG && is_root_ref(de->d_name)) { + ret = cb(refname.buf, cb_data); + if (ret) + goto done; + } strbuf_setlen(&refname, dirnamelen); } + +done: strbuf_release(&refname); strbuf_release(&path); closedir(d); + return ret; +} + +struct fill_root_ref_data { + struct files_ref_store *refs; + struct ref_dir *dir; +}; + +static int fill_root_ref(const char *refname, void *cb_data) +{ + struct fill_root_ref_data *data = cb_data; + loose_fill_ref_dir_regular_file(data->refs, refname, data->dir); + return 0; +} + +/* + * Add root refs to the ref dir by parsing the directory for any files which + * follow the root ref syntax. + */ +static void add_root_refs(struct files_ref_store *refs, + struct ref_dir *dir) +{ + struct fill_root_ref_data data = { + .refs = refs, + .dir = dir, + }; + + for_each_root_ref(refs, fill_root_ref, &data); } static struct ref_cache *get_loose_ref_cache(struct files_ref_store *refs, From patchwork Tue May 28 06:31:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13676171 Received: from fhigh3-smtp.messagingengine.com (fhigh3-smtp.messagingengine.com [103.168.172.154]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 61BD054FAF for ; Tue, 28 May 2024 06:31:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.154 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716877909; cv=none; b=QgJMhXX31Ctf+Kr5uzw5ijTrV+AWIJsjJaTDH915bhBSxUxJEPByDnvOMmB8PEoYHxumfr1M9z3bt+2DTqFFMVONXu/X2o5pfynCLvq+0AerRq+7y2ND8z6Jcwim6TffTPcFyqR0/MLQYwSb2IpIhVCW0O5bWv88UOjEMyRM/C0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716877909; c=relaxed/simple; bh=f72jjkM0VSaAUISQoj6EIPh0ksiR7hCg81keBWnYfPU=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=IgSUR4rHKIYqMQFbAYr5nv1uBXdqeRkiPMmAzJOjSHoX5i5w5zo5hrbf/dEI9zNkwZsYkTUxkMDKZi4On12pvs5ZH41N/OX7OiXoMCBkNkoPUeVAjWjuQ+ZBptIpjcKOvSf0wJXr1aKNtacKuwUZukpJgAfmElGYe9N1NFTMAx4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=nh8iuQ/h; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=olxgs/nh; arc=none smtp.client-ip=103.168.172.154 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="nh8iuQ/h"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="olxgs/nh" Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailfhigh.nyi.internal (Postfix) with ESMTP id 6DCBB11400F8; Tue, 28 May 2024 02:31:47 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Tue, 28 May 2024 02:31:47 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm1; t=1716877907; x=1716964307; bh=j7b/v601X6 B9xpJrXaRk2E9HOyqch+xRrmvacmR+SgE=; b=nh8iuQ/hPYn+F5iuAFixNoLWGg dVoK93ifDDo+wy0+lsfne7GzO/s9t3Augrp0fA2Mhz9EEh35R5ENcQSHknPuu1kI tW5jeIE/IrY5rNYoRV/rUm4OZbLVeLPObM6Gtym/l6QbncZlqEzg/xxCNoSrY87G Foyp29y/gBpzB7lTtxadOIzMH/HS6saR/qd8o0UCBdjni1w3RzfjNX226SOHEi7C yM7jXnxbj94Q6yXjC3IZ92DTL/EmbfaFzBkPIKMOudHKssoNzT4irNjXM34dGrDl qC5OLAYakJCRBIxP14qoSKIZqios304B+BLkYxtZGhRgLQOkh889hgN/U44A== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1716877907; x=1716964307; bh=j7b/v601X6B9xpJrXaRk2E9HOyqc h+xRrmvacmR+SgE=; b=olxgs/nhd+xbRwUD5cClTOaMyHpKPFRbfRu2zp6KYs9f NXsS3yDC/gTqqFep7dgkc9E2JGgrBEQhmE8p5+RAmwYops6BQU4Vg+fqdm5ospKg r88ptW6qkbWJpckIxycnnQErjhoU3Fq9eF/cLdu18grZu0wgREvpq7E98GAoaJlD fMzgHnoesyOaYYlZp46TPag+hUVq1E+b5YQApJ9Cs6DPiQpFYZOOfpgDADXYmEBc 6xOgFEg0r30Rx+BIea5N3KeXiUNxERwpI+jvJI014oQyw409OQ7tptw7GwM2dxYw 2+NaRfGiVLhUEQdWL/j2cl/dmbhgKfTD5k2OzrHD+A== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdejhedguddtiecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpeffhffvvefukfhfgggtuggjsehgtderredttddvnecuhfhrohhmpefrrght rhhitghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrihhmqeenucggtffrrghtth gvrhhnpeeukedtvedtffevleejtefgheehieegkeeluddvfeefgeehgfeltddtheejleff teenucevlhhushhtvghrufhiiigvpeefnecurfgrrhgrmhepmhgrihhlfhhrohhmpehpsh esphhkshdrihhm X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 28 May 2024 02:31:46 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id 8920d10d (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Tue, 28 May 2024 06:31:34 +0000 (UTC) Date: Tue, 28 May 2024 08:31:43 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Eric Sunshine , Junio C Hamano , Ramsay Jones , Justin Tobler Subject: [PATCH v3 07/12] refs/files: fix NULL pointer deref when releasing ref store Message-ID: <56baa798fbb2838227fbc08ed54f200e38545d28.1716877224.git.ps@pks.im> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: The `free_ref_cache()` function is not `NULL` safe and will thus segfault when being passed such a pointer. This can easily happen when trying to release a partially initialized "files" ref store. Fix this. Signed-off-by: Patrick Steinhardt --- refs/ref-cache.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/refs/ref-cache.c b/refs/ref-cache.c index b6c53fc8ed..4ce519bbc8 100644 --- a/refs/ref-cache.c +++ b/refs/ref-cache.c @@ -71,6 +71,8 @@ static void free_ref_entry(struct ref_entry *entry) void free_ref_cache(struct ref_cache *cache) { + if (!cache) + return; free_ref_entry(cache->root); free(cache); } From patchwork Tue May 28 06:31:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13676172 Received: from fout5-smtp.messagingengine.com (fout5-smtp.messagingengine.com [103.168.172.148]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E202061FE8 for ; Tue, 28 May 2024 06:31:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.148 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716877914; cv=none; b=ide4qkDmGe/4o/sjRLD2Ith/CEeymftYYi8gZFQQ4P0BhgorWp6kuyULvUcyUIo0i/3ODPqYFOsbk3xrjUuq/FxDfh6nEJQ2ezo5WJrNWbRU/T0+7X2i3fsDDw7TSZEUT6FVmllf5ldq8fiYUiWu4L4A7tMgXI2If2qzKBWnr9M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716877914; c=relaxed/simple; bh=LRI1dHBLKmgxfkkpF5Mo/Q7pnJZmfIq4Ew3kPsExY/I=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=nO3v66shMceBrokx1StoFeRtTSb+AMmdBzewyCdWsmFmGjOYHnJoEQdrreMN83qhZVYIylKgdIr9ti+FLR+DeJTnYQ8LMgjzxucl6l70rK0ldkcqm+LwW3JAhItUAAZ2peuPLJo0SN2iHaoE3oLRRHrcfWH1HGx2OAoI6yRudG8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=aR5WhfzW; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=isMhRNrv; arc=none smtp.client-ip=103.168.172.148 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="aR5WhfzW"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="isMhRNrv" Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailfout.nyi.internal (Postfix) with ESMTP id 04B5413800A5; Tue, 28 May 2024 02:31:52 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Tue, 28 May 2024 02:31:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm1; t=1716877912; x=1716964312; bh=167YRLpudP Fy8Wnmw43oFng2aVUg+Qzo1OsDMCm+no8=; b=aR5WhfzWk131ubAvrBK/i7PxG7 oGRvJaC+FnHN9az59wAioOJXIsJMpe9p8AdQGIJeR0gJPgGYX85g2O3HisCgwLF1 /j5S5LzCL2wmRSCm4YYK/W3whlDV0v/W4rtE2sTQGsI8vUiO+3cd7bPgvZHPJDsd D08pzKb0f3kgYc5IEKFMazkN0VTfpFmuRAOfC9s7NW4oJCD2PdwkqfqJoyUHAGX1 2X2tQgAEuAgqQWxx0Jmmhd6ADsuWyBJ0YJU4ShbpCm5PdMZ5idweeH+sjwdWWvYm 4LBAt/424VRklP+rMG4dt7O+7o+Q2CXJeierqquDV591yHP8rKgR5sQlYfPQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1716877912; x=1716964312; bh=167YRLpudPFy8Wnmw43oFng2aVUg +Qzo1OsDMCm+no8=; b=isMhRNrvkCW1HYOR6gW2Li+IK8YtKdnGnn8o8VAx3q7O oJ4ccm01ogsh19lJqExUNNZ1u4OUT1S1XHzSj89wA3s7OW41VPlcAqAfDUAuN1f/ e/a5O/0HGsLOZqBSIe8i7VUmOGfKrB7fpI+93e9uUvEloXYn2TglAEHBm/xZPCG/ 8g7fIITrhZF51AgcMvuE24Xtr/DeqTQkiN7owXCjf6b6rIoTeHoFAeVDs3OMYkFf FnVMNjFe9/3vwbSOLpLe3CKoiERa2LWnV9GHUaP3OLljS1c6kU9TGhrS77qQBtmB t96E0TCjDNceDMZSnPqZ3rmyClIZcMl70k0R9oP71Q== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdejhedguddtiecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpeffhffvvefukfhfgggtuggjsehgtderredttddvnecuhfhrohhmpefrrght rhhitghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrihhmqeenucggtffrrghtth gvrhhnpeeukedtvedtffevleejtefgheehieegkeeluddvfeefgeehgfeltddtheejleff teenucevlhhushhtvghrufhiiigvpeefnecurfgrrhgrmhepmhgrihhlfhhrohhmpehpsh esphhkshdrihhm X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 28 May 2024 02:31:50 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id 4add8acb (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Tue, 28 May 2024 06:31:39 +0000 (UTC) Date: Tue, 28 May 2024 08:31:48 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Eric Sunshine , Junio C Hamano , Ramsay Jones , Justin Tobler Subject: [PATCH v3 08/12] reftable: inline `merged_table_release()` Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: The function `merged_table_release()` releases a merged table, whereas `reftable_merged_table_free()` releases a merged table and then also free's its pointer. But all callsites of `merged_table_release()` are in fact followed by `reftable_merged_table_free()`, which is redundant. Inline `merged_table_release()` into `reftable_merged_table_free()` to get rid of this redundance. Signed-off-by: Patrick Steinhardt --- reftable/merged.c | 12 ++---------- reftable/merged.h | 2 -- reftable/stack.c | 8 ++------ 3 files changed, 4 insertions(+), 18 deletions(-) diff --git a/reftable/merged.c b/reftable/merged.c index f85a24c678..804fdc0de0 100644 --- a/reftable/merged.c +++ b/reftable/merged.c @@ -207,19 +207,11 @@ int reftable_new_merged_table(struct reftable_merged_table **dest, return 0; } -/* clears the list of subtable, without affecting the readers themselves. */ -void merged_table_release(struct reftable_merged_table *mt) -{ - FREE_AND_NULL(mt->stack); - mt->stack_len = 0; -} - void reftable_merged_table_free(struct reftable_merged_table *mt) { - if (!mt) { + if (!mt) return; - } - merged_table_release(mt); + FREE_AND_NULL(mt->stack); reftable_free(mt); } diff --git a/reftable/merged.h b/reftable/merged.h index a2571dbc99..9db45c3196 100644 --- a/reftable/merged.h +++ b/reftable/merged.h @@ -24,6 +24,4 @@ struct reftable_merged_table { uint64_t max; }; -void merged_table_release(struct reftable_merged_table *mt); - #endif diff --git a/reftable/stack.c b/reftable/stack.c index a59ebe038d..984fd866d0 100644 --- a/reftable/stack.c +++ b/reftable/stack.c @@ -261,10 +261,8 @@ static int reftable_stack_reload_once(struct reftable_stack *st, char **names, new_tables = NULL; st->readers_len = new_readers_len; - if (st->merged) { - merged_table_release(st->merged); + if (st->merged) reftable_merged_table_free(st->merged); - } if (st->readers) { reftable_free(st->readers); } @@ -968,10 +966,8 @@ static int stack_write_compact(struct reftable_stack *st, done: reftable_iterator_destroy(&it); - if (mt) { - merged_table_release(mt); + if (mt) reftable_merged_table_free(mt); - } reftable_ref_record_release(&ref); reftable_log_record_release(&log); st->stats.entries_written += entries; From patchwork Tue May 28 06:31:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13676173 Received: from fout5-smtp.messagingengine.com (fout5-smtp.messagingengine.com [103.168.172.148]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5DA2361FDE for ; Tue, 28 May 2024 06:31:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.148 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716877919; cv=none; b=S5uFjUcNSUvt3c4qVWNUsPIniJciTLnJP8lCZUAaiS6mWb8c5Yk6nB6rMkKlG2q0tv4Z6BhQyW1NpAjFl2Bkl/Qy0tAtV4VZoI0U3Ai63KgFokdx/oaq7BmIcsHWBdvD+VuT0DVFFOCkq6aKPLY+Q0YKa6zT4eMZhaPKzlQXW2w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716877919; c=relaxed/simple; bh=KYxgWFQd2yEOFbxkf5/aGx4ojecxXw9HDziYxW2OeSQ=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=YFo6lZQAG8z46VuuO1Tmy/2pLTDravm2fSJwmUsr9IDD0xaqyqa5G6x0xhzIk9Rd3vxKw7tIV4pZpiCjPbMpjtcMkNGxs0DSbfyDE3CWk0TowguFCB+nNkl5V/XWyTjRAr3jdg3mYP6NBhHXdXml4APgYQA8wmjEFGqFEwTP7xs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=BdTmKemX; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=R1A5+rG3; arc=none smtp.client-ip=103.168.172.148 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="BdTmKemX"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="R1A5+rG3" Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailfout.nyi.internal (Postfix) with ESMTP id 83F241380085; Tue, 28 May 2024 02:31:57 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute5.internal (MEProxy); Tue, 28 May 2024 02:31:57 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm1; t=1716877917; x=1716964317; bh=jAhxTsekDL tmz80IOvYM50ZSR7YylkE5W7o9nE97oqk=; b=BdTmKemXY120oQ6IYtPd9vSj+o o25KBIzcltOqRbvJ3i5Ag7nwVMA8OXkX34hpqCa65JlWEvhtl9Sb4EpMu898Boz9 MHuHHtxZjpxzyRZ+D4EfVXVjMv4AT+7Bh9zsk6elr5rXbjFwu9LRxy9SPLqpGq/V NBdiOGWWT8e9oHo+4znCK3KYVqb4EghOn8GbLzpp3r4P5LviTTydOpjSe1tG65Ns OpUmvJrQ+3dhmmAtCFi2lBajAdSmSqph82t6JD8kfXsj7KsIvOvOzzdSXBBW9Ctv wUMA5t5dPPH29HK4DMPCGSjw4H2BcbkfTiZ7gdEUkgJ87JRKfgOadikJwtPQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1716877917; x=1716964317; bh=jAhxTsekDLtmz80IOvYM50ZSR7Yy lkE5W7o9nE97oqk=; b=R1A5+rG3eF6WuZI/cwDKLOlw0L/Q4Zvo48Vw0l8/lKTX XxitV1EwdlbAG8oJ73rnyLZK4eZex3rYju9xoPTfuUIFE9aHYluvP35jSnexbaZh aiD9xen30WBmb6KgTIAFRrDDjmSZfA4IZg+e22R4+MISz9gxW2meo5eNjFj9A0u4 Ujwnyfs45WHShxZly/iD66FwNNuZVcWIGADPM4tSUkTAfqKXlqdQBFAMV11FaZ0L Cis57fZceevlnLoS4Js91gizHCdoB41zzzYoM8faZuWDK5qX+rKx3qD6FOZAfM2R cGeAgRbY+rH1197bjovd/iSaRCJUHvuVY1ESKl+wMw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdejhedguddtjecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpeffhffvvefukfhfgggtuggjsehgtderredttddvnecuhfhrohhmpefrrght rhhitghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrihhmqeenucggtffrrghtth gvrhhnpeeukedtvedtffevleejtefgheehieegkeeluddvfeefgeehgfeltddtheejleff teenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehpsh esphhkshdrihhm X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 28 May 2024 02:31:56 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id 4afbcb45 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Tue, 28 May 2024 06:31:43 +0000 (UTC) Date: Tue, 28 May 2024 08:31:53 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Eric Sunshine , Junio C Hamano , Ramsay Jones , Justin Tobler Subject: [PATCH v3 09/12] worktree: don't store main worktree twice Message-ID: <7a89aae51526faefc0ccce0a73cbd54aec8e8879.1716877224.git.ps@pks.im> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: In `get_worktree_ref_store()` we either return the repository's main ref store, or we look up the ref store via the map of worktree ref stores. Which of these worktrees gets picked depends on the `is_current` bit of the worktree, which indicates whether the worktree is the one that corresponds to `the_repository`. The bit is getting set in `get_worktrees()`, but only after we have computed the list of all worktrees. This is too late though, because at that time we have already called `get_worktree_ref_store()` on each of the worktrees via `add_head_info()`. The consequence is that the current worktree will not have been marked accordingly, which means that we did not use the main ref store, but instead created a new ref store. We thus have two separate ref stores now that map to the same ref database. Fix this by setting `is_current` before we call `add_head_info()`. Signed-off-by: Patrick Steinhardt --- worktree.c | 29 +++++++++++------------------ 1 file changed, 11 insertions(+), 18 deletions(-) diff --git a/worktree.c b/worktree.c index 12eadacc61..70844d023a 100644 --- a/worktree.c +++ b/worktree.c @@ -53,6 +53,15 @@ static void add_head_info(struct worktree *wt) wt->is_detached = 1; } +static int is_current_worktree(struct worktree *wt) +{ + char *git_dir = absolute_pathdup(get_git_dir()); + const char *wt_git_dir = get_worktree_git_dir(wt); + int is_current = !fspathcmp(git_dir, absolute_path(wt_git_dir)); + free(git_dir); + return is_current; +} + /** * get the main worktree */ @@ -76,6 +85,7 @@ static struct worktree *get_main_worktree(int skip_reading_head) */ worktree->is_bare = (is_bare_repository_cfg == 1) || is_bare_repository(); + worktree->is_current = is_current_worktree(worktree); if (!skip_reading_head) add_head_info(worktree); return worktree; @@ -102,6 +112,7 @@ struct worktree *get_linked_worktree(const char *id, worktree->repo = the_repository; worktree->path = strbuf_detach(&worktree_path, NULL); worktree->id = xstrdup(id); + worktree->is_current = is_current_worktree(worktree); if (!skip_reading_head) add_head_info(worktree); @@ -111,23 +122,6 @@ struct worktree *get_linked_worktree(const char *id, return worktree; } -static void mark_current_worktree(struct worktree **worktrees) -{ - char *git_dir = absolute_pathdup(get_git_dir()); - int i; - - for (i = 0; worktrees[i]; i++) { - struct worktree *wt = worktrees[i]; - const char *wt_git_dir = get_worktree_git_dir(wt); - - if (!fspathcmp(git_dir, absolute_path(wt_git_dir))) { - wt->is_current = 1; - break; - } - } - free(git_dir); -} - /* * NEEDSWORK: This function exists so that we can look up metadata of a * worktree without trying to access any of its internals like the refdb. It @@ -164,7 +158,6 @@ static struct worktree **get_worktrees_internal(int skip_reading_head) ALLOC_GROW(list, counter + 1, alloc); list[counter] = NULL; - mark_current_worktree(list); return list; } From patchwork Tue May 28 06:31:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13676174 Received: from fhigh3-smtp.messagingengine.com (fhigh3-smtp.messagingengine.com [103.168.172.154]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D27966D1A6 for ; Tue, 28 May 2024 06:32:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.154 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716877924; cv=none; b=R9mA10CSTePOqdYjaeQMSHBvBZmM01cYeBe9lo26yrWT9y0MVOgkxXS1SO2jJSeNPXEVFehpbQ7Q9rAzHAHPgN/fg5yWXuhl1IJpy9mhUZrN9dodQvu9anr43fzoikG4xT1/gFbTwirUELYQym4XRHgCgz2FfPrsSRgHXV0GuYw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716877924; c=relaxed/simple; bh=TZk7JWVuxvJZSSdoS7+17NAzZiiGsHTJ41f3XH9UA38=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=N/uCqfOQzhqBlhFhUxsRpO4ZrVq8HzSu/VeHbaRCgabSCtf9OyWgNGywPsf3DVmHFV8RBPtqMkcPObVFrxCVNeAz7XHzLbtk5lY7b3wX+zBOxrQE/p+Wjb6GNfWTdmXy9I4q3iBufCBzzsURKUKS1zdPNdxKvXlxUwRtK11SqzY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=RDrrL+m0; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=REC5m8CW; arc=none smtp.client-ip=103.168.172.154 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="RDrrL+m0"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="REC5m8CW" Received: from compute6.internal (compute6.nyi.internal [10.202.2.47]) by mailfhigh.nyi.internal (Postfix) with ESMTP id 144DB1140081; Tue, 28 May 2024 02:32:02 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute6.internal (MEProxy); Tue, 28 May 2024 02:32:02 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm1; t=1716877922; x=1716964322; bh=2nFvyedBzw ijb5VkTKWmX4Qp+0moe1i7fiPBCTu90cU=; b=RDrrL+m0eZXTfYUyh8aFvGSwFM ap/Mbi+8Fi2B2+naD6BgwIbPUmFNKt6PvZhqwlUj3ILJf9fUMPkH/tKgUSHBsykY WYgjrRGkihO3++uTmuBciMZoV12jhDE8rswZeQKrn8/tFNWZgkOFt6PBZMZETBk+ nB6c5dFpcTuh0mTCVyEsH7gfgRNGwepi2MEdyqxpmF4oU5d2VD9SqXRSL2MZhTUP YYo6m567gVcp5mV1jTKA/JQ5rheQUTMQuUTOcx5OxtuNpgrRIR05/Nuvt3Guqw91 XHSSa4hWr3WcP+2YU1pKJtmNr9bq9dRXSOeWfsBizTIOF+kpF+yve5PoTqgA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1716877922; x=1716964322; bh=2nFvyedBzwijb5VkTKWmX4Qp+0mo e1i7fiPBCTu90cU=; b=REC5m8CWLb9MhvEsCiZUyyWDwIPzkfFzddxQGVB3Y/p+ t4u3qPbDbOvFYcuc8h2y8b2aabsfuXBqLt/tDdtlMsvx2ryqsVdwd7L555qs9raS LBWaMsuWRyKBftOHCb3XK1SyoGEr+4LoVl66kIEyGkNmoHyNE2Qw+wkHJSMpVcNp ptP8ODccxtlC/SSNXlPQXbRy21RmUfpYn5YlbtIPHaSslchG4p/1T4AHOt2m+68B KrF6nEhdzkUFr8uBK8L6Y8Gx9CXRpGICYmTJfXcqRVi33JONK0IEBlAxy5ODyUGv +Jgcyy3SxbC4Vi9PNAUcKquM32Chbd2KVi0ozVHvPw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdejhedguddtjecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpeffhffvvefukfhfgggtuggjsehgtderredttddvnecuhfhrohhmpefrrght rhhitghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrihhmqeenucggtffrrghtth gvrhhnpeeukedtvedtffevleejtefgheehieegkeeluddvfeefgeehgfeltddtheejleff teenucevlhhushhtvghrufhiiigvpedvnecurfgrrhgrmhepmhgrihhlfhhrohhmpehpsh esphhkshdrihhm X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 28 May 2024 02:32:00 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id 0f1c447e (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Tue, 28 May 2024 06:31:48 +0000 (UTC) Date: Tue, 28 May 2024 08:31:58 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Eric Sunshine , Junio C Hamano , Ramsay Jones , Justin Tobler Subject: [PATCH v3 10/12] refs: implement removal of ref storages Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: We're about to introduce logic to migrate ref storages. One part of the migration will be to delete the files that are part of the old ref storage format. We don't yet have a way to delete such data generically across ref backends though. Implement a new `delete` callback and expose it via a new `ref_storage_delete()` function. Signed-off-by: Patrick Steinhardt --- refs.c | 5 ++++ refs.h | 5 ++++ refs/files-backend.c | 63 +++++++++++++++++++++++++++++++++++++++++ refs/packed-backend.c | 15 ++++++++++ refs/refs-internal.h | 7 +++++ refs/reftable-backend.c | 52 ++++++++++++++++++++++++++++++++++ 6 files changed, 147 insertions(+) diff --git a/refs.c b/refs.c index 66e9585767..9b112b0527 100644 --- a/refs.c +++ b/refs.c @@ -1861,6 +1861,11 @@ int ref_store_create_on_disk(struct ref_store *refs, int flags, struct strbuf *e return refs->be->create_on_disk(refs, flags, err); } +int ref_store_remove_on_disk(struct ref_store *refs, struct strbuf *err) +{ + return refs->be->remove_on_disk(refs, err); +} + int repo_resolve_gitlink_ref(struct repository *r, const char *submodule, const char *refname, struct object_id *oid) diff --git a/refs.h b/refs.h index 50a2b3ab09..61ee7b7a15 100644 --- a/refs.h +++ b/refs.h @@ -129,6 +129,11 @@ int ref_store_create_on_disk(struct ref_store *refs, int flags, struct strbuf *e */ void ref_store_release(struct ref_store *ref_store); +/* + * Remove the ref store from disk. This deletes all associated data. + */ +int ref_store_remove_on_disk(struct ref_store *refs, struct strbuf *err); + /* * Return the peeled value of the oid currently being iterated via * for_each_ref(), etc. This is equivalent to calling: diff --git a/refs/files-backend.c b/refs/files-backend.c index b7268b26c8..cb752d32b6 100644 --- a/refs/files-backend.c +++ b/refs/files-backend.c @@ -3340,11 +3340,74 @@ static int files_ref_store_create_on_disk(struct ref_store *ref_store, return 0; } +struct remove_one_root_ref_data { + const char *gitdir; + struct strbuf *err; +}; + +static int remove_one_root_ref(const char *refname, + void *cb_data) +{ + struct remove_one_root_ref_data *data = cb_data; + struct strbuf buf = STRBUF_INIT; + int ret = 0; + + strbuf_addf(&buf, "%s/%s", data->gitdir, refname); + + ret = unlink(buf.buf); + if (ret < 0) + strbuf_addf(data->err, "could not delete %s: %s\n", + refname, strerror(errno)); + + strbuf_release(&buf); + return ret; +} + +static int files_ref_store_remove_on_disk(struct ref_store *ref_store, + struct strbuf *err) +{ + struct files_ref_store *refs = + files_downcast(ref_store, REF_STORE_WRITE, "remove"); + struct remove_one_root_ref_data data = { + .gitdir = refs->base.gitdir, + .err = err, + }; + struct strbuf sb = STRBUF_INIT; + int ret = 0; + + strbuf_addf(&sb, "%s/refs", refs->base.gitdir); + if (remove_dir_recursively(&sb, 0) < 0) { + strbuf_addf(err, "could not delete refs: %s", + strerror(errno)); + ret = -1; + } + strbuf_reset(&sb); + + strbuf_addf(&sb, "%s/logs", refs->base.gitdir); + if (remove_dir_recursively(&sb, 0) < 0) { + strbuf_addf(err, "could not delete logs: %s", + strerror(errno)); + ret = -1; + } + strbuf_reset(&sb); + + ret = for_each_root_ref(refs, remove_one_root_ref, &data); + if (ret < 0) + ret = -1; + + if (ref_store_remove_on_disk(refs->packed_ref_store, err) < 0) + ret = -1; + + strbuf_release(&sb); + return ret; +} + struct ref_storage_be refs_be_files = { .name = "files", .init = files_ref_store_init, .release = files_ref_store_release, .create_on_disk = files_ref_store_create_on_disk, + .remove_on_disk = files_ref_store_remove_on_disk, .transaction_prepare = files_transaction_prepare, .transaction_finish = files_transaction_finish, diff --git a/refs/packed-backend.c b/refs/packed-backend.c index 2789fd92f5..c4c1e36aa2 100644 --- a/refs/packed-backend.c +++ b/refs/packed-backend.c @@ -1,5 +1,6 @@ #include "../git-compat-util.h" #include "../config.h" +#include "../dir.h" #include "../gettext.h" #include "../hash.h" #include "../hex.h" @@ -1266,6 +1267,19 @@ static int packed_ref_store_create_on_disk(struct ref_store *ref_store UNUSED, return 0; } +static int packed_ref_store_remove_on_disk(struct ref_store *ref_store, + struct strbuf *err) +{ + struct packed_ref_store *refs = packed_downcast(ref_store, 0, "remove"); + + if (remove_path(refs->path) < 0) { + strbuf_addstr(err, "could not delete packed-refs"); + return -1; + } + + return 0; +} + /* * Write the packed refs from the current snapshot to the packed-refs * tempfile, incorporating any changes from `updates`. `updates` must @@ -1724,6 +1738,7 @@ struct ref_storage_be refs_be_packed = { .init = packed_ref_store_init, .release = packed_ref_store_release, .create_on_disk = packed_ref_store_create_on_disk, + .remove_on_disk = packed_ref_store_remove_on_disk, .transaction_prepare = packed_transaction_prepare, .transaction_finish = packed_transaction_finish, diff --git a/refs/refs-internal.h b/refs/refs-internal.h index 33749fbd83..cbcb6f9c36 100644 --- a/refs/refs-internal.h +++ b/refs/refs-internal.h @@ -517,6 +517,12 @@ typedef int ref_store_create_on_disk_fn(struct ref_store *refs, int flags, struct strbuf *err); +/* + * Remove the reference store from disk. + */ +typedef int ref_store_remove_on_disk_fn(struct ref_store *refs, + struct strbuf *err); + typedef int ref_transaction_prepare_fn(struct ref_store *refs, struct ref_transaction *transaction, struct strbuf *err); @@ -649,6 +655,7 @@ struct ref_storage_be { ref_store_init_fn *init; ref_store_release_fn *release; ref_store_create_on_disk_fn *create_on_disk; + ref_store_remove_on_disk_fn *remove_on_disk; ref_transaction_prepare_fn *transaction_prepare; ref_transaction_finish_fn *transaction_finish; diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c index bffed9257f..e555be4671 100644 --- a/refs/reftable-backend.c +++ b/refs/reftable-backend.c @@ -1,6 +1,7 @@ #include "../git-compat-util.h" #include "../abspath.h" #include "../chdir-notify.h" +#include "../dir.h" #include "../environment.h" #include "../gettext.h" #include "../hash.h" @@ -343,6 +344,56 @@ static int reftable_be_create_on_disk(struct ref_store *ref_store, return 0; } +static int reftable_be_remove_on_disk(struct ref_store *ref_store, + struct strbuf *err) +{ + struct reftable_ref_store *refs = + reftable_be_downcast(ref_store, REF_STORE_WRITE, "remove"); + struct strbuf sb = STRBUF_INIT; + int ret = 0; + + /* + * Release the ref store such that all stacks are closed. This is + * required so that the "tables.list" file is not open anymore, which + * would otherwise make it impossible to remove the file on Windows. + */ + reftable_be_release(ref_store); + + strbuf_addf(&sb, "%s/reftable", refs->base.gitdir); + if (remove_dir_recursively(&sb, 0) < 0) { + strbuf_addf(err, "could not delete reftables: %s", + strerror(errno)); + ret = -1; + } + strbuf_reset(&sb); + + strbuf_addf(&sb, "%s/HEAD", refs->base.gitdir); + if (unlink(sb.buf) < 0) { + strbuf_addf(err, "could not delete stub HEAD: %s", + strerror(errno)); + ret = -1; + } + strbuf_reset(&sb); + + strbuf_addf(&sb, "%s/refs/heads", refs->base.gitdir); + if (unlink(sb.buf) < 0) { + strbuf_addf(err, "could not delete stub heads: %s", + strerror(errno)); + ret = -1; + } + strbuf_reset(&sb); + + strbuf_addf(&sb, "%s/refs", refs->base.gitdir); + if (rmdir(sb.buf) < 0) { + strbuf_addf(err, "could not delete stub heads: %s", + strerror(errno)); + ret = -1; + } + + strbuf_release(&sb); + return ret; +} + struct reftable_ref_iterator { struct ref_iterator base; struct reftable_ref_store *refs; @@ -2196,6 +2247,7 @@ struct ref_storage_be refs_be_reftable = { .init = reftable_be_init, .release = reftable_be_release, .create_on_disk = reftable_be_create_on_disk, + .remove_on_disk = reftable_be_remove_on_disk, .transaction_prepare = reftable_be_transaction_prepare, .transaction_finish = reftable_be_transaction_finish, From patchwork Tue May 28 06:32:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13676177 Received: from fhigh3-smtp.messagingengine.com (fhigh3-smtp.messagingengine.com [103.168.172.154]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 71C9B71739 for ; Tue, 28 May 2024 06:32:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.154 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716877929; cv=none; b=Z2XeIF7iy7nvCF1M39ud81u518K3qviGFU0U6lPy9QSnWhs+4fgnAozKZ/49lR3IFb1g0BnA2cL/GMci2NG8uY6/gqPbtxJuevnZ33n9bBdjt61pIYfEFg7CK6msGBDSzw1JogAXmHarhNlaSxyZBbMc22DHZzyUF3ZLLmnLDD0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716877929; c=relaxed/simple; bh=TqcQEP5RW1UpAtHAjK1Fmf/i5HszT3fr7yfY49W+XwI=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=ZmaOGKBLhWI6XRIzLy/xvPmzNmQv3nzddBXLtfP+gsF3/N9FNCcyb28ow/VgU966ffCVGRcPyzMBhkjPMrxxo4XVua2vI2Lxt92WOFUSljHiehoK4z1ge6Y8iij9anm30r1gjAmc2oDOXDnmCmagWMcTKPDF7c6oF1+RD17ASzA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=dIixkLrV; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=gn2pg/Qs; arc=none smtp.client-ip=103.168.172.154 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="dIixkLrV"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="gn2pg/Qs" Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailfhigh.nyi.internal (Postfix) with ESMTP id 8D6A611400D8; Tue, 28 May 2024 02:32:06 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute5.internal (MEProxy); Tue, 28 May 2024 02:32:06 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm1; t=1716877926; x=1716964326; bh=z6fQLnpixC IcEw9Wy+WbekPSgke18x8U3wMCT2oUyl8=; b=dIixkLrVbbJwDsWwkI76H6TLN6 ZlqMQCDB0IyhU4lzd+R2YHldVzsWuXfmpxEymAHCF7lDRGw4peGjaKlkcXcIWP/k Z7/M7AUxCXbZGKOslpkMAjRmn5CoBv2216H20I8vkTucKna5SjOSngDEHF8Ntfo+ SeXKvaZu6jbNoDK5ZW5PgQjs7GKlkSabSV1zZRTCTlVW1PQ7W9pk6ORu9EsjIXB4 LPXjdwsY8cLJnCa0KzLnd2YcTtZWHrpRsEgcyYzHhfNZXOpq3HnsMIsV5AFDS9B0 OPVaoeb7nN5fLBjYzg5d0PljoaQq3ty3uRgyh311bPriq7oh31QX88LRvdUA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1716877926; x=1716964326; bh=z6fQLnpixCIcEw9Wy+WbekPSgke1 8x8U3wMCT2oUyl8=; b=gn2pg/Qs/xTKbpcSlWYrR/zS9sdijSVBqlDuIWLD4L7I UvYw+Z451PY6AOStoF8DGO2DJZ1MCaq4krn2eHsB+NA3JoWm8t3goL4WrSqCY3ye 2dYJ+0PeP/UdbBUAIOyGcLJ969WldRvfnaMRysQvurFICgORs5/ngGUuhWp7jOfY h+jzWmR2HEu9v3Mf/soVkI5QcYskpdZkMiP3fpAq6+e5eEBlamrnY4lmbTQrEVyw kBDoltOsublkno4eRatsS2ftjdueAPvXddVQOd+7Vfy+XYrFo3DJ2/LAoTV8VH7O 0DgGHKhZW6T2YIveUMHHrKvqCpc47YOSxtmSYCK4Pg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdejhedguddtjecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpeffhffvvefukfhfgggtuggjsehgtderredttddvnecuhfhrohhmpefrrght rhhitghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrihhmqeenucggtffrrghtth gvrhhnpeeukedtvedtffevleejtefgheehieegkeeluddvfeefgeehgfeltddtheejleff teenucevlhhushhtvghrufhiiigvpedunecurfgrrhgrmhepmhgrihhlfhhrohhmpehpsh esphhkshdrihhm X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 28 May 2024 02:32:05 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id 3623a6c5 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Tue, 28 May 2024 06:31:53 +0000 (UTC) Date: Tue, 28 May 2024 08:32:02 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Eric Sunshine , Junio C Hamano , Ramsay Jones , Justin Tobler Subject: [PATCH v3 11/12] refs: implement logic to migrate between ref storage formats Message-ID: <1f26051eff8b7c18bb7114803454611272f84e19.1716877224.git.ps@pks.im> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: With the introduction of the new "reftable" backend, users may want to migrate repositories between the backends without having to recreate the whole repository. Add the logic to do so. The implementation is generic and works with arbitrary ref storage formats so that a backend does not need to implement any migration logic. It does have a few limitations though: - We do not migrate repositories with worktrees, because worktrees have separate ref storages. It makes the overall affair more complex if we have to migrate multiple storages at once. - We do not migrate reflogs, because we have no interfaces to write many reflog entries. - We do not lock the repository for concurrent access, and thus concurrent writes may end up with weird in-between states. There is no way to fully lock the "files" backend for writes due to its format, and thus we punt on this topic altogether and defer to the user to avoid those from happening. In other words, this version is a minimum viable product for migrating a repository's ref storage format. It works alright for bare repos, which often have neither worktrees nor reflogs. But it will not work for many other repositories without some preparations. These limitations are not set into stone though, and ideally we will eventually address them over time. The logic is not yet used by anything, and thus there are no tests for it. Those will be added in the next commit. Signed-off-by: Patrick Steinhardt --- refs.c | 305 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++ refs.h | 18 ++++ 2 files changed, 323 insertions(+) diff --git a/refs.c b/refs.c index 9b112b0527..f7c7765d23 100644 --- a/refs.c +++ b/refs.c @@ -2570,3 +2570,308 @@ int ref_update_check_old_target(const char *referent, struct ref_update *update, referent, update->old_target); return -1; } + +struct migration_data { + struct ref_store *old_refs; + struct ref_transaction *transaction; + struct strbuf *errbuf; +}; + +static int migrate_one_ref(const char *refname, const struct object_id *oid, + int flags, void *cb_data) +{ + struct migration_data *data = cb_data; + struct strbuf symref_target = STRBUF_INIT; + int ret; + + if (flags & REF_ISSYMREF) { + ret = refs_read_symbolic_ref(data->old_refs, refname, &symref_target); + if (ret < 0) + goto done; + + ret = ref_transaction_update(data->transaction, refname, NULL, null_oid(), + symref_target.buf, NULL, + REF_SKIP_CREATE_REFLOG | REF_NO_DEREF, NULL, data->errbuf); + if (ret < 0) + goto done; + } else { + ret = ref_transaction_create(data->transaction, refname, oid, + REF_SKIP_CREATE_REFLOG | REF_SKIP_OID_VERIFICATION, + NULL, data->errbuf); + if (ret < 0) + goto done; + } + +done: + strbuf_release(&symref_target); + return ret; +} + +static int move_files(const char *from_path, const char *to_path, struct strbuf *errbuf) +{ + struct strbuf from_buf = STRBUF_INIT, to_buf = STRBUF_INIT; + size_t from_len, to_len; + DIR *from_dir; + int ret; + + from_dir = opendir(from_path); + if (!from_dir) { + strbuf_addf(errbuf, "could not open source directory '%s': %s", + from_path, strerror(errno)); + ret = -1; + goto done; + } + + strbuf_addstr(&from_buf, from_path); + strbuf_complete(&from_buf, '/'); + from_len = from_buf.len; + + strbuf_addstr(&to_buf, to_path); + strbuf_complete(&to_buf, '/'); + to_len = to_buf.len; + + while (1) { + struct dirent *ent; + + errno = 0; + ent = readdir(from_dir); + if (!ent) + break; + + if (!strcmp(ent->d_name, ".") || + !strcmp(ent->d_name, "..")) + continue; + + strbuf_setlen(&from_buf, from_len); + strbuf_addstr(&from_buf, ent->d_name); + + strbuf_setlen(&to_buf, to_len); + strbuf_addstr(&to_buf, ent->d_name); + + ret = rename(from_buf.buf, to_buf.buf); + if (ret < 0) { + strbuf_addf(errbuf, "could not link file '%s' to '%s': %s", + from_buf.buf, to_buf.buf, strerror(errno)); + goto done; + } + } + + if (errno) { + strbuf_addf(errbuf, "could not read entry from directory '%s': %s", + from_path, strerror(errno)); + ret = -1; + goto done; + } + + ret = 0; + +done: + strbuf_release(&from_buf); + strbuf_release(&to_buf); + if (from_dir) + closedir(from_dir); + return ret; +} + +static int count_reflogs(const char *reflog UNUSED, void *payload) +{ + size_t *reflog_count = payload; + (*reflog_count)++; + return 0; +} + +static int has_worktrees(void) +{ + struct worktree **worktrees = get_worktrees(); + int ret = 0; + size_t i; + + for (i = 0; worktrees[i]; i++) { + if (is_main_worktree(worktrees[i])) + continue; + ret = 1; + } + + free_worktrees(worktrees); + return ret; +} + +int repo_migrate_ref_storage_format(struct repository *repo, + enum ref_storage_format format, + unsigned int flags, + struct strbuf *errbuf) +{ + struct ref_store *old_refs = NULL, *new_refs = NULL; + struct ref_transaction *transaction = NULL; + struct strbuf buf = STRBUF_INIT; + struct migration_data data; + size_t reflog_count = 0; + char *new_gitdir = NULL; + int did_migrate_refs = 0; + int ret; + + old_refs = get_main_ref_store(repo); + + /* + * We do not have any interfaces that would allow us to write many + * reflog entries. Once we have them we can remove this restriction. + */ + if (refs_for_each_reflog(old_refs, count_reflogs, &reflog_count) < 0) { + strbuf_addstr(errbuf, "cannot count reflogs"); + ret = -1; + goto done; + } + if (reflog_count) { + strbuf_addstr(errbuf, "migrating reflogs is not supported yet"); + ret = -1; + goto done; + } + + /* + * Worktrees complicate the migration because every worktree has a + * separate ref storage. While it should be feasible to implement, this + * is pushed out to a future iteration. + * + * TODO: we should really be passing the caller-provided repository to + * `has_worktrees()`, but our worktree subsystem doesn't yet support + * that. + */ + if (has_worktrees()) { + strbuf_addstr(errbuf, "migrating repositories with worktrees is not supported yet"); + ret = -1; + goto done; + } + + /* + * The overall logic looks like this: + * + * 1. Set up a new temporary directory and initialize it with the new + * format. This is where all refs will be migrated into. + * + * 2. Enumerate all refs and write them into the new ref storage. + * This operation is safe as we do not yet modify the main + * repository. + * + * 3. If we're in dry-run mode then we are done and can hand over the + * directory to the caller for inspection. If not, we now start + * with the destructive part. + * + * 4. Delete the old ref storage from disk. As we have a copy of refs + * in the new ref storage it's okay(ish) if we now get interrupted + * as there is an equivalent copy of all refs available. + * + * 5. Move the new ref storage files into place. + * + * 6. Change the repository format to the new ref format. + */ + strbuf_addf(&buf, "%s/%s", old_refs->gitdir, "ref_migration.XXXXXX"); + new_gitdir = mkdtemp(xstrdup(buf.buf)); + if (!new_gitdir) { + strbuf_addf(errbuf, "cannot create migration directory: %s", + strerror(errno)); + ret = -1; + goto done; + } + + new_refs = ref_store_init(repo, format, new_gitdir, + REF_STORE_ALL_CAPS); + ret = ref_store_create_on_disk(new_refs, 0, errbuf); + if (ret < 0) + goto done; + + transaction = ref_store_transaction_begin(new_refs, errbuf); + if (!transaction) + goto done; + + data.old_refs = old_refs; + data.transaction = transaction; + data.errbuf = errbuf; + + /* + * We need to use the internal `do_for_each_ref()` here so that we can + * also include broken refs and symrefs. These would otherwise be + * skipped silently. + * + * Ideally, we would do this call while locking the old ref storage + * such that there cannot be any concurrent modifications. We do not + * have the infra for that though, and the "files" backend does not + * allow for a central lock due to its design. It's thus on the user to + * ensure that there are no concurrent writes. + */ + ret = do_for_each_ref(old_refs, "", NULL, migrate_one_ref, 0, + DO_FOR_EACH_INCLUDE_ROOT_REFS | DO_FOR_EACH_INCLUDE_BROKEN, + &data); + if (ret < 0) + goto done; + + /* + * TODO: we might want to migrate to `initial_ref_transaction_commit()` + * here, which is more efficient for the files backend because it would + * write new refs into the packed-refs file directly. At this point, + * the files backend doesn't handle pseudo-refs and symrefs correctly + * though, so this requires some more work. + */ + ret = ref_transaction_commit(transaction, errbuf); + if (ret < 0) + goto done; + did_migrate_refs = 1; + + if (flags & REPO_MIGRATE_REF_STORAGE_FORMAT_DRYRUN) { + printf(_("Finished dry-run migration of refs, " + "the result can be found at '%s'\n"), new_gitdir); + ret = 0; + goto done; + } + + /* + * Until now we were in the non-destructive phase, where we only + * populated the new ref store. From hereon though we are about + * to get hands by deleting the old ref store and then moving + * the new one into place. + * + * Assuming that there were no concurrent writes, the new ref + * store should have all information. So if we fail from hereon + * we may be in an in-between state, but it would still be able + * to recover by manually moving remaining files from the + * temporary migration directory into place. + */ + ret = ref_store_remove_on_disk(old_refs, errbuf); + if (ret < 0) + goto done; + + ret = move_files(new_gitdir, old_refs->gitdir, errbuf); + if (ret < 0) + goto done; + + if (rmdir(new_gitdir) < 0) + warning_errno(_("could not remove temporary migration directory '%s'"), + new_gitdir); + + /* + * We have migrated the repository, so we now need to adjust the + * repository format so that clients will use the new ref store. + * We also need to swap out the repository's main ref store. + */ + initialize_repository_version(hash_algo_by_ptr(repo->hash_algo), format, 1); + + free(new_refs->gitdir); + new_refs->gitdir = xstrdup(old_refs->gitdir); + repo->refs_private = new_refs; + ref_store_release(old_refs); + + ret = 0; + +done: + if (ret && did_migrate_refs) { + strbuf_complete(errbuf, '\n'); + strbuf_addf(errbuf, _("migrated refs can be found at '%s'"), + new_gitdir); + } + + if (ret && new_refs) + ref_store_release(new_refs); + ref_transaction_free(transaction); + strbuf_release(&buf); + free(new_gitdir); + return ret; +} diff --git a/refs.h b/refs.h index 61ee7b7a15..76d25df4de 100644 --- a/refs.h +++ b/refs.h @@ -1070,6 +1070,24 @@ int is_root_ref(const char *refname); */ int is_pseudo_ref(const char *refname); +/* + * The following flags can be passed to `repo_migrate_ref_storage_format()`: + * + * - REPO_MIGRATE_REF_STORAGE_FORMAT_DRYRUN: perform a dry-run migration + * without touching the main repository. The result will be written into a + * temporary ref storage directory. + */ +#define REPO_MIGRATE_REF_STORAGE_FORMAT_DRYRUN (1 << 0) + +/* + * Migrate the ref storage format used by the repository to the + * specified one. + */ +int repo_migrate_ref_storage_format(struct repository *repo, + enum ref_storage_format format, + unsigned int flags, + struct strbuf *err); + /* * The following functions have been removed in Git v2.45 in favor of functions * that receive a `ref_store` as parameter. The intent of this section is From patchwork Tue May 28 06:32:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13676179 Received: from fout5-smtp.messagingengine.com (fout5-smtp.messagingengine.com [103.168.172.148]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 026B08F48 for ; Tue, 28 May 2024 06:32:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.148 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716877934; cv=none; b=eK8cY0LV0IpsBifgjiew/BJg9xLb5wv9R/FVomplXbGxw65IlkiZ93/PMUF6HaXOy05a6fosPoZZxdnhltLQbAVNV6L6QASCqm410QyGtwlU5oWuHHxgVg1s5yStfbX7an8DITJvaSf4brO6p0r/NvX8xyrIhdY+Dny28a9JCjA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716877934; c=relaxed/simple; bh=3lB0mQM9rGKArezw/ZCMRlnz7vVf+aNYVzvpOrbVg/0=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=C2Ml0rghf197V5vgPt0dIYLTc7cwiaW5jejXyDmMfTD6RY30omBgRL8ZcbLFLYLGqKLl9uM/MkmFr4XYhBOc5M+ltVFIT7hc5LAOk7SzUftirE+0AVoSgJ3vax8mqTWVSk0Rcvp4d0KeJLg3V49Vhh2ezUwFnv9oZGAzdA055Ac= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=lTNcBmcE; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=kSw0Kpll; arc=none smtp.client-ip=103.168.172.148 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="lTNcBmcE"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="kSw0Kpll" Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailfout.nyi.internal (Postfix) with ESMTP id 184F813800A8; Tue, 28 May 2024 02:32:11 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute5.internal (MEProxy); Tue, 28 May 2024 02:32:11 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm1; t=1716877931; x=1716964331; bh=6M7SvIoSC5 wS2sG80EEbJpkVHwUJl0BzbhebbmRvk9g=; b=lTNcBmcE5h/wd/9VhVSAKNDE01 0S6wkge7wFdUuNFpGmowwcpD2W92tIWxNOpti4xMlxObDlNUFGBkVFSDwxt55i4o EtE3a1Q5f8WJyKGNGL7K7+0glvR9dWzvlPOJaA1FkS17bMQcSg/EZPCZv45AjkKZ 4B9p7ZmjUubnCGm8mv9jqwLo7A21/PSikaqYJqBGoa9uhL+cYiKaXpxMi5YcCHTC fG+xpJtP4J/JhX+Tvq/6dY4Kz+QRFZPstt0F1NOmXiirg8DFkhV8Vylj2Qo80mrp 76rQOiziejr2M6JeJkEd7eRqMl6NqwkLeNeIEouwDq30dbqvAbGtLzIrA0bQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1716877931; x=1716964331; bh=6M7SvIoSC5wS2sG80EEbJpkVHwUJ l0BzbhebbmRvk9g=; b=kSw0KpllqZxVjA08K1NrWC+XmMkdtpcrMBQ7xBAMk5kt ut2mxQ/qL+CswwnLvu8rXEWg6nH7G+GRwcAO+pSaEhCettslzJxH/hVO2L5G6DKQ T4BIBCYsS0pFqci6j0HFTDpvITSmAIUv2zovybR6rW74ElNp0rxbamlpnPV7g1X4 KAjnP3pu2Whdp2WZcXGYUoy+7MnzyBWXsTMtd/BhGo8QZrxpuLEpVwCBWVsqWoCX 3UrSTD/3uZFKYfNF1mAqKnRMrDBVQvEh/tJtuRBkD4juSikRVvVpQ7ppnExKenBb MSPEySU/5/ptA6va5RXC9Y/w+kzWmclhyheBbLIV9A== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdejhedguddtjecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpeffhffvvefukfhfgggtuggjsehgtderredttddvnecuhfhrohhmpefrrght rhhitghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrihhmqeenucggtffrrghtth gvrhhnpeeukedtvedtffevleejtefgheehieegkeeluddvfeefgeehgfeltddtheejleff teenucevlhhushhtvghrufhiiigvpedvnecurfgrrhgrmhepmhgrihhlfhhrohhmpehpsh esphhkshdrihhm X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 28 May 2024 02:32:09 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id 8d54e190 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Tue, 28 May 2024 06:31:58 +0000 (UTC) Date: Tue, 28 May 2024 08:32:07 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Eric Sunshine , Junio C Hamano , Ramsay Jones , Justin Tobler Subject: [PATCH v3 12/12] builtin/refs: new command to migrate ref storage formats Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Introduce a new command that allows the user to migrate a repository between ref storage formats. This new command is implemented as part of a new git-refs(1) executable. This is due to two reasons: - There is no good place to put the migration logic in existing commands. git-maintenance(1) felt unwieldy, and git-pack-refs(1) is not the correct place to put it, either. - I had it in my mind to create a new low-level command for accessing refs for quite a while already. git-refs(1) is that command and can over time grow more functionality relating to refs. This should help discoverability by consolidating low-level access to refs into a single executable. As mentioned in the preceding commit that introduces the ref storage format migration logic, the new `git refs migrate` command still has a bunch of restrictions. These restrictions are documented accordingly. Signed-off-by: Patrick Steinhardt --- .gitignore | 1 + Documentation/git-refs.txt | 62 ++++++++++ Makefile | 1 + builtin.h | 1 + builtin/refs.c | 75 ++++++++++++ git.c | 1 + t/t1460-refs-migrate.sh | 243 +++++++++++++++++++++++++++++++++++++ 7 files changed, 384 insertions(+) create mode 100644 Documentation/git-refs.txt create mode 100644 builtin/refs.c create mode 100755 t/t1460-refs-migrate.sh diff --git a/.gitignore b/.gitignore index 612c0f6a0f..8caf3700c2 100644 --- a/.gitignore +++ b/.gitignore @@ -126,6 +126,7 @@ /git-rebase /git-receive-pack /git-reflog +/git-refs /git-remote /git-remote-http /git-remote-https diff --git a/Documentation/git-refs.txt b/Documentation/git-refs.txt new file mode 100644 index 0000000000..3e9c05185a --- /dev/null +++ b/Documentation/git-refs.txt @@ -0,0 +1,62 @@ +git-refs(1) +=========== + +NAME +---- + +git-refs - Low-level access to refs + +SYNOPSIS +-------- + +[verse] +'git refs migrate' --ref-format= [--dry-run] + +DESCRIPTION +----------- + +This command provides low-level access to refs. + +COMMANDS +-------- + +migrate:: + Migrate ref store between different formats. + +OPTIONS +------- + +The following options are specific to 'git refs migrate': + +--ref-format=:: + The ref format to migrate the ref store to. Can be one of: ++ +include::ref-storage-format.txt[] + +--dry-run:: + Perform the migration, but do not modify the repository. The migrated + refs will be written into a separate directory that can be inspected + separately. The name of the directory will be reported on stdout. This + can be used to double check that the migration works as expected before + performing the actual migration. + +KNOWN LIMITATIONS +----------------- + +The ref format migration has several known limitations in its current form: + +* It is not possible to migrate repositories that have reflogs. + +* It is not possible to migrate repositories that have worktrees. + +* There is no way to block concurrent writes to the repository during an + ongoing migration. Concurrent writes can lead to an inconsistent migrated + state. Users are expected to block writes on a higher level. If your + repository is registered for scheduled maintenance, it is recommended to + unregister it first with git-maintenance(1). + +These limitations may eventually be lifted. + +GIT +--- +Part of the linkgit:git[1] suite diff --git a/Makefile b/Makefile index cf504963c2..2d702b552c 100644 --- a/Makefile +++ b/Makefile @@ -1283,6 +1283,7 @@ BUILTIN_OBJS += builtin/read-tree.o BUILTIN_OBJS += builtin/rebase.o BUILTIN_OBJS += builtin/receive-pack.o BUILTIN_OBJS += builtin/reflog.o +BUILTIN_OBJS += builtin/refs.o BUILTIN_OBJS += builtin/remote-ext.o BUILTIN_OBJS += builtin/remote-fd.o BUILTIN_OBJS += builtin/remote.o diff --git a/builtin.h b/builtin.h index 28280636da..7eda9b2486 100644 --- a/builtin.h +++ b/builtin.h @@ -207,6 +207,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix); int cmd_rebase__interactive(int argc, const char **argv, const char *prefix); int cmd_receive_pack(int argc, const char **argv, const char *prefix); int cmd_reflog(int argc, const char **argv, const char *prefix); +int cmd_refs(int argc, const char **argv, const char *prefix); int cmd_remote(int argc, const char **argv, const char *prefix); int cmd_remote_ext(int argc, const char **argv, const char *prefix); int cmd_remote_fd(int argc, const char **argv, const char *prefix); diff --git a/builtin/refs.c b/builtin/refs.c new file mode 100644 index 0000000000..46dcd150d4 --- /dev/null +++ b/builtin/refs.c @@ -0,0 +1,75 @@ +#include "builtin.h" +#include "parse-options.h" +#include "refs.h" +#include "repository.h" +#include "strbuf.h" + +#define REFS_MIGRATE_USAGE \ + N_("git refs migrate --ref-format= [--dry-run]") + +static int cmd_refs_migrate(int argc, const char **argv, const char *prefix) +{ + const char * const migrate_usage[] = { + REFS_MIGRATE_USAGE, + NULL, + }; + const char *format_str = NULL; + enum ref_storage_format format; + unsigned int flags = 0; + struct option options[] = { + OPT_STRING_F(0, "ref-format", &format_str, N_("format"), + N_("specify the reference format to convert to"), + PARSE_OPT_NONEG), + OPT_BIT(0, "dry-run", &flags, + N_("perform a non-destructive dry-run"), + REPO_MIGRATE_REF_STORAGE_FORMAT_DRYRUN), + OPT_END(), + }; + struct strbuf errbuf = STRBUF_INIT; + int err; + + argc = parse_options(argc, argv, prefix, options, migrate_usage, 0); + if (argc) + usage(_("too many arguments")); + if (!format_str) + usage(_("missing --ref-format=")); + + format = ref_storage_format_by_name(format_str); + if (format == REF_STORAGE_FORMAT_UNKNOWN) { + err = error(_("unknown ref storage format '%s'"), format_str); + goto out; + } + + if (the_repository->ref_storage_format == format) { + err = error(_("repository already uses '%s' format"), + ref_storage_format_to_name(format)); + goto out; + } + + if (repo_migrate_ref_storage_format(the_repository, format, flags, &errbuf) < 0) { + err = error("%s", errbuf.buf); + goto out; + } + + err = 0; + +out: + strbuf_release(&errbuf); + return err; +} + +int cmd_refs(int argc, const char **argv, const char *prefix) +{ + const char * const refs_usage[] = { + REFS_MIGRATE_USAGE, + NULL, + }; + parse_opt_subcommand_fn *fn = NULL; + struct option opts[] = { + OPT_SUBCOMMAND("migrate", &fn, cmd_refs_migrate), + OPT_END(), + }; + + argc = parse_options(argc, argv, prefix, opts, refs_usage, 0); + return fn(argc, argv, prefix); +} diff --git a/git.c b/git.c index 637c61ca9c..683bb69194 100644 --- a/git.c +++ b/git.c @@ -594,6 +594,7 @@ static struct cmd_struct commands[] = { { "rebase", cmd_rebase, RUN_SETUP | NEED_WORK_TREE }, { "receive-pack", cmd_receive_pack }, { "reflog", cmd_reflog, RUN_SETUP }, + { "refs", cmd_refs, RUN_SETUP }, { "remote", cmd_remote, RUN_SETUP }, { "remote-ext", cmd_remote_ext, NO_PARSEOPT }, { "remote-fd", cmd_remote_fd, NO_PARSEOPT }, diff --git a/t/t1460-refs-migrate.sh b/t/t1460-refs-migrate.sh new file mode 100755 index 0000000000..f7c0783d30 --- /dev/null +++ b/t/t1460-refs-migrate.sh @@ -0,0 +1,243 @@ +#!/bin/sh + +test_description='migration of ref storage backends' + +GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main +export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME + +TEST_PASSES_SANITIZE_LEAK=true +. ./test-lib.sh + +test_migration () { + git -C "$1" for-each-ref --include-root-refs \ + --format='%(refname) %(objectname) %(symref)' >expect && + git -C "$1" refs migrate --ref-format="$2" && + git -C "$1" for-each-ref --include-root-refs \ + --format='%(refname) %(objectname) %(symref)' >actual && + test_cmp expect actual && + + git -C "$1" rev-parse --show-ref-format >actual && + echo "$2" >expect && + test_cmp expect actual +} + +test_expect_success 'setup' ' + rm -rf .git && + # The migration does not yet support reflogs. + git config --global core.logAllRefUpdates false +' + +test_expect_success "superfluous arguments" ' + test_when_finished "rm -rf repo" && + git init repo && + test_must_fail git -C repo refs migrate foo 2>err && + cat >expect <<-EOF && + usage: too many arguments + EOF + test_cmp expect err +' + +test_expect_success "missing ref storage format" ' + test_when_finished "rm -rf repo" && + git init repo && + test_must_fail git -C repo refs migrate 2>err && + cat >expect <<-EOF && + usage: missing --ref-format= + EOF + test_cmp expect err +' + +test_expect_success "unknown ref storage format" ' + test_when_finished "rm -rf repo" && + git init repo && + test_must_fail git -C repo refs migrate \ + --ref-format=unknown 2>err && + cat >expect <<-EOF && + error: unknown ref storage format ${SQ}unknown${SQ} + EOF + test_cmp expect err +' + +ref_formats="files reftable" +for from_format in $ref_formats +do + for to_format in $ref_formats + do + if test "$from_format" = "$to_format" + then + continue + fi + + test_expect_success "$from_format: migration to same format fails" ' + test_when_finished "rm -rf repo" && + git init --ref-format=$from_format repo && + test_must_fail git -C repo refs migrate \ + --ref-format=$from_format 2>err && + cat >expect <<-EOF && + error: repository already uses ${SQ}$from_format${SQ} format + EOF + test_cmp expect err + ' + + test_expect_success "$from_format -> $to_format: migration with reflog fails" ' + test_when_finished "rm -rf repo" && + git init --ref-format=$from_format repo && + test_config -C repo core.logAllRefUpdates true && + test_commit -C repo logged && + test_must_fail git -C repo refs migrate \ + --ref-format=$to_format 2>err && + cat >expect <<-EOF && + error: migrating reflogs is not supported yet + EOF + test_cmp expect err + ' + + test_expect_success "$from_format -> $to_format: migration with worktree fails" ' + test_when_finished "rm -rf repo" && + git init --ref-format=$from_format repo && + git -C repo worktree add wt && + test_must_fail git -C repo refs migrate \ + --ref-format=$to_format 2>err && + cat >expect <<-EOF && + error: migrating repositories with worktrees is not supported yet + EOF + test_cmp expect err + ' + + test_expect_success "$from_format -> $to_format: unborn HEAD" ' + test_when_finished "rm -rf repo" && + git init --ref-format=$from_format repo && + test_migration repo "$to_format" + ' + + test_expect_success "$from_format -> $to_format: single ref" ' + test_when_finished "rm -rf repo" && + git init --ref-format=$from_format repo && + test_commit -C repo initial && + test_migration repo "$to_format" + ' + + test_expect_success "$from_format -> $to_format: bare repository" ' + test_when_finished "rm -rf repo repo.git" && + git init --ref-format=$from_format repo && + test_commit -C repo initial && + git clone --ref-format=$from_format --mirror repo repo.git && + test_migration repo.git "$to_format" + ' + + test_expect_success "$from_format -> $to_format: dangling symref" ' + test_when_finished "rm -rf repo" && + git init --ref-format=$from_format repo && + test_commit -C repo initial && + git -C repo symbolic-ref BROKEN_HEAD refs/heads/nonexistent && + test_migration repo "$to_format" && + echo refs/heads/nonexistent >expect && + git -C repo symbolic-ref BROKEN_HEAD >actual && + test_cmp expect actual + ' + + test_expect_success "$from_format -> $to_format: broken ref" ' + test_when_finished "rm -rf repo" && + git init --ref-format=$from_format repo && + test_commit -C repo initial && + test-tool -C repo ref-store main update-ref "" refs/heads/broken \ + "$(test_oid 001)" "$ZERO_OID" REF_SKIP_CREATE_REFLOG,REF_SKIP_OID_VERIFICATION && + test_migration repo "$to_format" && + test_oid 001 >expect && + git -C repo rev-parse refs/heads/broken >actual && + test_cmp expect actual + ' + + test_expect_success "$from_format -> $to_format: pseudo-refs" ' + test_when_finished "rm -rf repo" && + git init --ref-format=$from_format repo && + test_commit -C repo initial && + git -C repo update-ref FOO_HEAD HEAD && + test_migration repo "$to_format" + ' + + test_expect_success "$from_format -> $to_format: special refs are left alone" ' + test_when_finished "rm -rf repo" && + git init --ref-format=$from_format repo && + test_commit -C repo initial && + git -C repo rev-parse HEAD >repo/.git/MERGE_HEAD && + git -C repo rev-parse MERGE_HEAD && + test_migration repo "$to_format" && + test_path_is_file repo/.git/MERGE_HEAD + ' + + test_expect_success "$from_format -> $to_format: a bunch of refs" ' + test_when_finished "rm -rf repo" && + git init --ref-format=$from_format repo && + + test_commit -C repo initial && + cat >input <<-EOF && + create FOO_HEAD HEAD + create refs/heads/branch-1 HEAD + create refs/heads/branch-2 HEAD + create refs/heads/branch-3 HEAD + create refs/heads/branch-4 HEAD + create refs/tags/tag-1 HEAD + create refs/tags/tag-2 HEAD + EOF + git -C repo update-ref --stdin $to_format: dry-run migration does not modify repository" ' + test_when_finished "rm -rf repo" && + git init --ref-format=$from_format repo && + test_commit -C repo initial && + git -C repo refs migrate --dry-run \ + --ref-format=$to_format >output && + grep "Finished dry-run migration of refs" output && + test_path_is_dir repo/.git/ref_migration.* && + echo $from_format >expect && + git -C repo rev-parse --show-ref-format >actual && + test_cmp expect actual + ' + done +done + +test_expect_success 'migrating from files format deletes backend files' ' + test_when_finished "rm -rf repo" && + git init --ref-format=files repo && + test_commit -C repo first && + git -C repo pack-refs --all && + test_commit -C repo second && + git -C repo update-ref ORIG_HEAD HEAD && + git -C repo rev-parse HEAD >repo/.git/FETCH_HEAD && + + test_path_is_file repo/.git/HEAD && + test_path_is_file repo/.git/ORIG_HEAD && + test_path_is_file repo/.git/refs/heads/main && + test_path_is_file repo/.git/packed-refs && + + test_migration repo reftable && + + echo "ref: refs/heads/.invalid" >expect && + test_cmp expect repo/.git/HEAD && + echo "this repository uses the reftable format" >expect && + test_cmp expect repo/.git/refs/heads && + test_path_is_file repo/.git/FETCH_HEAD && + test_path_is_missing repo/.git/ORIG_HEAD && + test_path_is_missing repo/.git/refs/heads/main && + test_path_is_missing repo/.git/logs && + test_path_is_missing repo/.git/packed-refs +' + +test_expect_success 'migrating from reftable format deletes backend files' ' + test_when_finished "rm -rf repo" && + git init --ref-format=reftable repo && + test_commit -C repo first && + + test_path_is_dir repo/.git/reftable && + test_migration repo files && + + test_path_is_missing repo/.git/reftable && + echo "ref: refs/heads/main" >expect && + test_cmp expect repo/.git/HEAD && + test_path_is_file repo/.git/refs/heads/main +' + +test_done