From patchwork Thu Jun 6 05:28:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13687818 Received: from wfout6-smtp.messagingengine.com (wfout6-smtp.messagingengine.com [64.147.123.149]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2B3B83B1BC for ; Thu, 6 Jun 2024 05:29:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=64.147.123.149 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717651744; cv=none; b=rGNSSIbG6Rk0yJ3NVMVlBODUIaubIM8jpq/b7kKSQ6jlvJmAU/c8Rg+Qor3SJ5MapTXwdPKoOqBAkNu+c44olQlyV3UE5iS8usd8SvGolNfEO4mLzH4Sfs91j2clTcfACR04F7QB96sS5WKJUx6Qo9h9z4hhAMbvxfW/X1EiNFQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717651744; c=relaxed/simple; bh=Nq/8gJa+Kq3l8OiehupsScLUIBTADIvm5y7xr37Drn0=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=gF0B2wXHTmFClEWa8tyWTmP06bU0O4yBfHStd01mg70K4Du8sWs3Cn6JsyS3XxqSEhNfLtZkbIgM86EfG9eHkN05IeJLqpmYxDvUOJaNQs/2F6+rGY1TtOc5enSY3TtuSRVmynyWXOa+qSNvIJfHWx6CUhjb+7jFT7HDNJUmvrA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=Ez8k9f4L; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=MnBHEiv2; arc=none smtp.client-ip=64.147.123.149 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="Ez8k9f4L"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="MnBHEiv2" Received: from compute6.internal (compute6.nyi.internal [10.202.2.47]) by mailfout.west.internal (Postfix) with ESMTP id EFB3D1C001D0; Thu, 6 Jun 2024 01:29:01 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute6.internal (MEProxy); Thu, 06 Jun 2024 01:29:02 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm1; t=1717651741; x=1717738141; bh=qDOiKOuhcZ EX7/k455COCf87MTP6jZPoNKaStQCtCds=; b=Ez8k9f4LRJP+I8eU6djPaa7MCy huMpiueiaOecRNmmNTMI8NquCPzLJvskwLHJ7a80Qyz6pbShUikHPzY86a4S0P2D Sf7oIYdALzViCVUDcblQuzQJ/gO2sSNYFWzGRn34fg2nUCgcsxgFlQ/Iky3IbeoV zC6IHBtoCPKtxsN2BNOCkcdYxTbuO8MlwWsDDCLZJKcHh2rYy7ZR5e+f3jZ0OeUg zU47K0yCjaBmxXHNRwEwf6zDrRdsLytuf7vO+Edeq09b5TT3ak8RcPHUHEGEcqi/ xLkxiRvxnRJum0V3APXuASXcyuU/yNLvbM8Ekkv68H2SKlgabne8oaPbbC8Q== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1717651741; x=1717738141; bh=qDOiKOuhcZEX7/k455COCf87MTP6 jZPoNKaStQCtCds=; b=MnBHEiv2cFcp743V86u5k6fjHSsgzhTQ2xlVprvhMPvV Zg4h8AWqfjE7MnrXF2Kc/8cwkaZ0iVB/2tbiNCqb9JfCv5HbAJ+nGBejH50+xNG6 31nRQQE442Kxrw8B5UtpPPnI+4V3ndT8Rw0tPeuMmoyiNaRyE3CxcUBAas2nNzSL 0B2ZDMhhq7VriT4mQ+fgw0d+xYnzvjTDSedR9MvxXR0+te4bqzuwY4F5BYIT8UW0 2Fkv/+L3OEMisHghbEP3M0cLhsWNuOTTGIzc6qBUNEuurFhcpjvbiy6nYlWPj2Fq MCOLVbFw4m+5G62OlTmLU0iappa4Aq1gjBaT20kr3g== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdeljedgleeiucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtvdenucfhrhhomheprfgrthhr ihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimheqnecuggftrfgrthhtvg hrnhepueektdevtdffveeljeetgfehheeigeekleduvdeffeeghefgledttdehjeelffet necuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepphhsse hpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 6 Jun 2024 01:29:00 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id a5fca1da (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Thu, 6 Jun 2024 05:28:29 +0000 (UTC) Date: Thu, 6 Jun 2024 07:28:57 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Eric Sunshine , Junio C Hamano , Ramsay Jones , Justin Tobler , Karthik Nayak , Jeff King Subject: [PATCH v5 01/12] setup: unset ref storage when reinitializing repository version Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: When reinitializing a repository's version we may end up unsetting the hash algorithm when it matches the default hash algorithm. If we didn't do that then the previously configured value might remain intact. While the same issue exists for the ref storage extension, we don't do this here. This has been fine for most of the part because it is not supported to re-initialize a repository with a different ref storage format anyway. We're about to introduce a new command to migrate ref storages though, so this is about to become an issue there. Prepare for this and unset the ref storage format when reinitializing a repository with the "files" format. Signed-off-by: Patrick Steinhardt --- setup.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/setup.c b/setup.c index 7975230ffb..8c84ec9d4b 100644 --- a/setup.c +++ b/setup.c @@ -2028,6 +2028,8 @@ void initialize_repository_version(int hash_algo, if (ref_storage_format != REF_STORAGE_FORMAT_FILES) git_config_set("extensions.refstorage", ref_storage_format_to_name(ref_storage_format)); + else if (reinit) + git_config_set_gently("extensions.refstorage", NULL); } static int is_reinit(void) From patchwork Thu Jun 6 05:29:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13687819 Received: from wfhigh1-smtp.messagingengine.com (wfhigh1-smtp.messagingengine.com [64.147.123.152]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E965F37165 for ; Thu, 6 Jun 2024 05:29:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=64.147.123.152 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717651749; cv=none; b=OJD4UzzitH8Hr5WNiNYn+j1aQ3PiQ6mhhSM6tDUj6z4SaZVs6iMEi5KAe2LxeBHnaMGemP5Xswwe58yZJTBd/Q8D8reSmbW+Nan19Nf6ASzI14Fm/b2RIgK5qACGEt6uteNnx0DSrkBoDtXv+hY86IidWcVFchYM3fPZvsG68YI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717651749; c=relaxed/simple; bh=4mZ6dYhPboWNNR3zp51vV4n94sLfwBmUGcvQObk5/I0=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=pog89pjeOWEwzHnzSWTQpYLpmQThiWwzT3/4kanJGYpvfVG7Fqyv+IhdmTCI5yynwdj7qpkp7brdXrxpQl5z6LWz0CZgaxFpFv38eF/Sm+69KbNXHDGSAllL1UE2etpUTJxjNFGFTrG1dRpKrOlBwTK39wX4mhKEqAtxBRDt7f8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=U8Q80xpg; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=aYEUedfO; arc=none smtp.client-ip=64.147.123.152 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="U8Q80xpg"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="aYEUedfO" Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailfhigh.west.internal (Postfix) with ESMTP id CBF851800196; Thu, 6 Jun 2024 01:29:06 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Thu, 06 Jun 2024 01:29:07 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm1; t=1717651746; x=1717738146; bh=dnOYK+t8m8 yUHZU96a9PmpWMeg5XmeVJ/z1Y3a95+6A=; b=U8Q80xpgK0PWLzrWX/aZ+vQxxB WLBsDma0OyrZCFODU83IndVtI2wOHqVAXoGrdJVagI8p5TQ1JNbMfC1m1NL9CaSW GpWJyqvSwCLgUpgtNPcILfapq7Za6KcXv9ODTw+uiMPXT7nT0VehaxZcYpcqcRFh M7z7QvoJ5UZS0AOqutn8jaTOcYfwh0bF9lp08KDEimgZWPB69sPcWyMRloQriNoS grFPWvuethZSit8nHA0PusJcjbzN/eMAqqn7HoPY6wuXSRS5fvu4QkUTnXq0U7LU LjRMsOtsoO+TOhQEj6elG6SC7N8mamalbYmhFxJJSfYC4Wt0XTQesPVzm2/A== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1717651746; x=1717738146; bh=dnOYK+t8m8yUHZU96a9PmpWMeg5X meVJ/z1Y3a95+6A=; b=aYEUedfOJ4idaDdqGGLiMA0AKDsfY+OTpL145M0WBSjD Vvb6ieH29G2TzqM/JNA3OesrkFowZrsbDBLoQZdOdn79HXWSeI39amD1L9lM3g/i Pq5VonFNnzVgfZ/f4YtPF2NdldEYRZJVSLFzT1iXiP9IyjYuWxqi/ayRPx4gk6ZR fErywzMdvXRgPOLy6BG467uwnh2gMntEfEj0krqk0FwqwUb/02f3ZAFg5+h5TxxB j3jL4D6yw6uudQgEA0JejwCbqJWZBOPcHmpcC5Uma0FO2seq8vZSj6SceVxn1/im xpaPrevka97pAywzTsgCXSzKtmfn/qOCy6txp6F/HA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdeljedgleeiucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtvdenucfhrhhomheprfgrthhr ihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimheqnecuggftrfgrthhtvg hrnhepueektdevtdffveeljeetgfehheeigeekleduvdeffeeghefgledttdehjeelffet necuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepphhsse hpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 6 Jun 2024 01:29:04 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id e145832b (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Thu, 6 Jun 2024 05:28:34 +0000 (UTC) Date: Thu, 6 Jun 2024 07:29:01 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Eric Sunshine , Junio C Hamano , Ramsay Jones , Justin Tobler , Karthik Nayak , Jeff King Subject: [PATCH v5 02/12] refs: convert ref storage format to an enum Message-ID: <7989e82dcd89c92f23de7ab4ebde96124aa6fe12.1717649802.git.ps@pks.im> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: The ref storage format is tracked as a simple unsigned integer, which makes it harder than necessary to discover what that integer actually is or where its values are defined. Convert the ref storage format to instead be an enum. Signed-off-by: Patrick Steinhardt --- builtin/clone.c | 2 +- builtin/init-db.c | 2 +- refs.c | 7 ++++--- refs.h | 10 ++++++++-- repository.c | 3 ++- repository.h | 10 ++++------ setup.c | 8 ++++---- setup.h | 9 +++++---- 8 files changed, 29 insertions(+), 22 deletions(-) diff --git a/builtin/clone.c b/builtin/clone.c index 1e07524c53..e808e02017 100644 --- a/builtin/clone.c +++ b/builtin/clone.c @@ -970,7 +970,7 @@ int cmd_clone(int argc, const char **argv, const char *prefix) int submodule_progress; int filter_submodules = 0; int hash_algo; - unsigned int ref_storage_format = REF_STORAGE_FORMAT_UNKNOWN; + enum ref_storage_format ref_storage_format = REF_STORAGE_FORMAT_UNKNOWN; const int do_not_override_repo_unix_permissions = -1; const char *template_dir; char *template_dir_dup = NULL; diff --git a/builtin/init-db.c b/builtin/init-db.c index 0170469b84..582dcf20f8 100644 --- a/builtin/init-db.c +++ b/builtin/init-db.c @@ -81,7 +81,7 @@ int cmd_init_db(int argc, const char **argv, const char *prefix) const char *ref_format = NULL; const char *initial_branch = NULL; int hash_algo = GIT_HASH_UNKNOWN; - unsigned int ref_storage_format = REF_STORAGE_FORMAT_UNKNOWN; + enum ref_storage_format ref_storage_format = REF_STORAGE_FORMAT_UNKNOWN; int init_shared_repository = -1; const struct option init_db_options[] = { OPT_STRING(0, "template", &template_dir, N_("template-directory"), diff --git a/refs.c b/refs.c index 31032588e0..e6db85a165 100644 --- a/refs.c +++ b/refs.c @@ -37,14 +37,15 @@ static const struct ref_storage_be *refs_backends[] = { [REF_STORAGE_FORMAT_REFTABLE] = &refs_be_reftable, }; -static const struct ref_storage_be *find_ref_storage_backend(unsigned int ref_storage_format) +static const struct ref_storage_be *find_ref_storage_backend( + enum ref_storage_format ref_storage_format) { if (ref_storage_format < ARRAY_SIZE(refs_backends)) return refs_backends[ref_storage_format]; return NULL; } -unsigned int ref_storage_format_by_name(const char *name) +enum ref_storage_format ref_storage_format_by_name(const char *name) { for (unsigned int i = 0; i < ARRAY_SIZE(refs_backends); i++) if (refs_backends[i] && !strcmp(refs_backends[i]->name, name)) @@ -52,7 +53,7 @@ unsigned int ref_storage_format_by_name(const char *name) return REF_STORAGE_FORMAT_UNKNOWN; } -const char *ref_storage_format_to_name(unsigned int ref_storage_format) +const char *ref_storage_format_to_name(enum ref_storage_format ref_storage_format) { const struct ref_storage_be *be = find_ref_storage_backend(ref_storage_format); if (!be) diff --git a/refs.h b/refs.h index fe7f0db35e..a7afa9bede 100644 --- a/refs.h +++ b/refs.h @@ -11,8 +11,14 @@ struct string_list; struct string_list_item; struct worktree; -unsigned int ref_storage_format_by_name(const char *name); -const char *ref_storage_format_to_name(unsigned int ref_storage_format); +enum ref_storage_format { + REF_STORAGE_FORMAT_UNKNOWN, + REF_STORAGE_FORMAT_FILES, + REF_STORAGE_FORMAT_REFTABLE, +}; + +enum ref_storage_format ref_storage_format_by_name(const char *name); +const char *ref_storage_format_to_name(enum ref_storage_format ref_storage_format); /* * Resolve a reference, recursively following symbolic refererences. diff --git a/repository.c b/repository.c index d29b0304fb..166863f852 100644 --- a/repository.c +++ b/repository.c @@ -124,7 +124,8 @@ void repo_set_compat_hash_algo(struct repository *repo, int algo) repo_read_loose_object_map(repo); } -void repo_set_ref_storage_format(struct repository *repo, unsigned int format) +void repo_set_ref_storage_format(struct repository *repo, + enum ref_storage_format format) { repo->ref_storage_format = format; } diff --git a/repository.h b/repository.h index 4bd8969005..a35cd77c35 100644 --- a/repository.h +++ b/repository.h @@ -1,6 +1,7 @@ #ifndef REPOSITORY_H #define REPOSITORY_H +#include "refs.h" #include "strmap.h" struct config_set; @@ -26,10 +27,6 @@ enum fetch_negotiation_setting { FETCH_NEGOTIATION_NOOP, }; -#define REF_STORAGE_FORMAT_UNKNOWN 0 -#define REF_STORAGE_FORMAT_FILES 1 -#define REF_STORAGE_FORMAT_REFTABLE 2 - struct repo_settings { int initialized; @@ -181,7 +178,7 @@ struct repository { const struct git_hash_algo *compat_hash_algo; /* Repository's reference storage format, as serialized on disk. */ - unsigned int ref_storage_format; + enum ref_storage_format ref_storage_format; /* A unique-id for tracing purposes. */ int trace2_repo_id; @@ -220,7 +217,8 @@ void repo_set_gitdir(struct repository *repo, const char *root, void repo_set_worktree(struct repository *repo, const char *path); void repo_set_hash_algo(struct repository *repo, int algo); void repo_set_compat_hash_algo(struct repository *repo, int compat_algo); -void repo_set_ref_storage_format(struct repository *repo, unsigned int format); +void repo_set_ref_storage_format(struct repository *repo, + enum ref_storage_format format); void initialize_repository(struct repository *repo); RESULT_MUST_BE_USED int repo_init(struct repository *r, const char *gitdir, const char *worktree); diff --git a/setup.c b/setup.c index 8c84ec9d4b..b49ee3e95f 100644 --- a/setup.c +++ b/setup.c @@ -1997,7 +1997,7 @@ static int needs_work_tree_config(const char *git_dir, const char *work_tree) } void initialize_repository_version(int hash_algo, - unsigned int ref_storage_format, + enum ref_storage_format ref_storage_format, int reinit) { char repo_version_string[10]; @@ -2044,7 +2044,7 @@ static int is_reinit(void) return ret; } -void create_reference_database(unsigned int ref_storage_format, +void create_reference_database(enum ref_storage_format ref_storage_format, const char *initial_branch, int quiet) { struct strbuf err = STRBUF_INIT; @@ -2243,7 +2243,7 @@ static void validate_hash_algorithm(struct repository_format *repo_fmt, int hash } static void validate_ref_storage_format(struct repository_format *repo_fmt, - unsigned int format) + enum ref_storage_format format) { const char *name = getenv("GIT_DEFAULT_REF_FORMAT"); @@ -2263,7 +2263,7 @@ static void validate_ref_storage_format(struct repository_format *repo_fmt, int init_db(const char *git_dir, const char *real_git_dir, const char *template_dir, int hash, - unsigned int ref_storage_format, + enum ref_storage_format ref_storage_format, const char *initial_branch, int init_shared_repository, unsigned int flags) { diff --git a/setup.h b/setup.h index b3fd3bf45a..cd8dbc2497 100644 --- a/setup.h +++ b/setup.h @@ -1,6 +1,7 @@ #ifndef SETUP_H #define SETUP_H +#include "refs.h" #include "string-list.h" int is_inside_git_dir(void); @@ -128,7 +129,7 @@ struct repository_format { int is_bare; int hash_algo; int compat_hash_algo; - unsigned int ref_storage_format; + enum ref_storage_format ref_storage_format; int sparse_index; char *work_tree; struct string_list unknown_extensions; @@ -192,13 +193,13 @@ const char *get_template_dir(const char *option_template); int init_db(const char *git_dir, const char *real_git_dir, const char *template_dir, int hash_algo, - unsigned int ref_storage_format, + enum ref_storage_format ref_storage_format, const char *initial_branch, int init_shared_repository, unsigned int flags); void initialize_repository_version(int hash_algo, - unsigned int ref_storage_format, + enum ref_storage_format ref_storage_format, int reinit); -void create_reference_database(unsigned int ref_storage_format, +void create_reference_database(enum ref_storage_format ref_storage_format, const char *initial_branch, int quiet); /* From patchwork Thu Jun 6 05:29:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13687820 Received: from wfhigh1-smtp.messagingengine.com (wfhigh1-smtp.messagingengine.com [64.147.123.152]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C00E437165 for ; Thu, 6 Jun 2024 05:29:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=64.147.123.152 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717651754; cv=none; b=q0lW4Ue0uPrSbkdx1HfthsPiD+VC1PR3Vvm28kKUDHxiz4UVgTZSj26GvIovG5hvNehufSX2z5YtPEbRqMUfdA2EelSJgek6V9XZNQxbw2/hNIhaIG7004gQtXfGWO/QANn41GRdJO+andk/SJtO7tzaDh87yj2Qc+2zrYLlOlY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717651754; c=relaxed/simple; bh=P396GJxafiCA4CB43/bFc3y+v3QTVlFda+K1xzNsxbA=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=H4N/eg7P3rxW5xoTbfYk4xBNNeIEjQL5T2OfwTKkhY8pc4jG7NmKaPSd5JixAat7Yq1U33rd9TuUTjqgYTlqOJQwFT2ktvrGWFIeUcXDFZ3GpI1ViYpPl7qvW+xcRWQ1yT7wYdZdAqikGZJZ3Rj0VEe4g1ZBNW5Z8CdI1+nAvoY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=Wz4APpTQ; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=IPRf1WQl; arc=none smtp.client-ip=64.147.123.152 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="Wz4APpTQ"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="IPRf1WQl" Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailfhigh.west.internal (Postfix) with ESMTP id 820A11800199; Thu, 6 Jun 2024 01:29:11 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Thu, 06 Jun 2024 01:29:12 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm1; t=1717651751; x=1717738151; bh=pmBuIsPBzA UKvKNYkK6Q7y+9Uqyt1PlFgEzYDa+/Q/A=; b=Wz4APpTQKU94VbjeAiL2wZKfqT VIKMyWDsVdUfpMgztDXmQEVL/PjDmA2HUbUE4hjlMZ/Yc2IOMisd09nvdUDHFDRK +oQ215h3R8WZSWx+PSLdoZQSB08ugGGJ3D1XSwMAsN2cfcxHkj6M6qoSiyCV0Aeq epCGY0BbDua8KdJSIDRjpL4VjvXiqvDx7UV44hVtmbpU2jwwvx6k1Gn0wzvu4+nO CWdJBF7/l8c8n/BxOgzZMXx1C29C4vRcrqKff2Mp9Pnup65wlM6lyDF0MA7+DeKe tFwVizCMh5v95mkD2JW57tEi+M+qZNOjlGFB/2/mbAbodCoNwPgRNchwnurQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1717651751; x=1717738151; bh=pmBuIsPBzAUKvKNYkK6Q7y+9Uqyt 1PlFgEzYDa+/Q/A=; b=IPRf1WQlDkiCX97T6cFkejpKCGhpsZe1Co64aFl/jtys 7AKzN0abUqUGNHaeH9B12oB/SkPmrl91rr9XnUu9MGMg2ax49lt8WepBAVQ16i+/ d4Cq2kDEIgNSDxq6K5Z+ejveoi4bsUCx3zp5JV5RXn1ZMUbuCOmyRrjVFC5tGhz0 73LSUpv27NPhXzh4sFUS8LMqbr1wVOOpR6sKVI9zeqMQhR+ZltoXDnyNrxv9fwLQ 7yWflw2WKGTo0L5ak5MmXg+qQvNYnN4/sohLDnVTJDUzSigH3d/YR2MbXoUfh2Rg 5yV+fH3Lz0ANciFNoVh1zcMSoa/F4e1i/DzKHfYrow== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdeljedgleeiucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtvdenucfhrhhomheprfgrthhr ihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimheqnecuggftrfgrthhtvg hrnhepueektdevtdffveeljeetgfehheeigeekleduvdeffeeghefgledttdehjeelffet necuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepphhsse hpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 6 Jun 2024 01:29:09 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id 05131f72 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Thu, 6 Jun 2024 05:28:39 +0000 (UTC) Date: Thu, 6 Jun 2024 07:29:06 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Eric Sunshine , Junio C Hamano , Ramsay Jones , Justin Tobler , Karthik Nayak , Jeff King Subject: [PATCH v5 03/12] refs: pass storage format to `ref_store_init()` explicitly Message-ID: <26005abb280805eaced2ca146f004fe01ea039f5.1717649802.git.ps@pks.im> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: We're about to introduce logic to migrate refs from one storage format to another one. This will require us to initialize a ref store with a different format than the one used by the passed-in repository. Prepare for this by accepting the desired ref storage format as parameter. Signed-off-by: Patrick Steinhardt --- refs.c | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/refs.c b/refs.c index e6db85a165..423684b8b8 100644 --- a/refs.c +++ b/refs.c @@ -1891,16 +1891,17 @@ static struct ref_store *lookup_ref_store_map(struct strmap *map, /* * Create, record, and return a ref_store instance for the specified - * gitdir. + * gitdir using the given ref storage format. */ static struct ref_store *ref_store_init(struct repository *repo, + enum ref_storage_format format, const char *gitdir, unsigned int flags) { const struct ref_storage_be *be; struct ref_store *refs; - be = find_ref_storage_backend(repo->ref_storage_format); + be = find_ref_storage_backend(format); if (!be) BUG("reference backend is unknown"); @@ -1922,7 +1923,8 @@ struct ref_store *get_main_ref_store(struct repository *r) if (!r->gitdir) BUG("attempting to get main_ref_store outside of repository"); - r->refs_private = ref_store_init(r, r->gitdir, REF_STORE_ALL_CAPS); + r->refs_private = ref_store_init(r, r->ref_storage_format, + r->gitdir, REF_STORE_ALL_CAPS); r->refs_private = maybe_debug_wrap_ref_store(r->gitdir, r->refs_private); return r->refs_private; } @@ -1982,7 +1984,8 @@ struct ref_store *repo_get_submodule_ref_store(struct repository *repo, free(subrepo); goto done; } - refs = ref_store_init(subrepo, submodule_sb.buf, + refs = ref_store_init(subrepo, the_repository->ref_storage_format, + submodule_sb.buf, REF_STORE_READ | REF_STORE_ODB); register_ref_store_map(&repo->submodule_ref_stores, "submodule", refs, submodule); @@ -2011,12 +2014,12 @@ struct ref_store *get_worktree_ref_store(const struct worktree *wt) struct strbuf common_path = STRBUF_INIT; strbuf_git_common_path(&common_path, wt->repo, "worktrees/%s", wt->id); - refs = ref_store_init(wt->repo, common_path.buf, - REF_STORE_ALL_CAPS); + refs = ref_store_init(wt->repo, wt->repo->ref_storage_format, + common_path.buf, REF_STORE_ALL_CAPS); strbuf_release(&common_path); } else { - refs = ref_store_init(wt->repo, wt->repo->commondir, - REF_STORE_ALL_CAPS); + refs = ref_store_init(wt->repo, the_repository->ref_storage_format, + wt->repo->commondir, REF_STORE_ALL_CAPS); } if (refs) From patchwork Thu Jun 6 05:29:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13687821 Received: from wfhigh1-smtp.messagingengine.com (wfhigh1-smtp.messagingengine.com [64.147.123.152]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5666C37165 for ; Thu, 6 Jun 2024 05:29:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=64.147.123.152 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717651757; cv=none; b=sKh4Acgbc/nOz7CsgVydi1KTxfrHv0KHGS5pgMFPXTeCulky2xCbRYp9Zy7Izy3hUIRbUbgFilqXkJ07VRK3AWl5UjL5TZEkJFlLIEpNYZ27ixC2z5d2+dMZc7FWOD4/6hWGWBk6KW1IZ7muMsbwXkTMq6OIuz6WxcXk7PF1qO4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717651757; c=relaxed/simple; bh=Faatj/CchbgIN/DijmcNsYo159kEr5ryRlYN03cc4XM=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=evEEOdwic5xdR9FTUkjECcDHE+WIGTOf2ppkdxl8pt7I7p1szq6DwwleXwG59k3wEaVfm0UtvxDcx8mTRgahzxnPWNN6OSHaqMkJheiXA/+b5WFsB0Kjf0QCx2yTyorgWEp7Edn87lqs1AeVmZVa/V6BHZh5hRW2oCkBW+/es2U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=Y+mrjE78; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=gLYrR+vq; arc=none smtp.client-ip=64.147.123.152 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="Y+mrjE78"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="gLYrR+vq" Received: from compute6.internal (compute6.nyi.internal [10.202.2.47]) by mailfhigh.west.internal (Postfix) with ESMTP id 34887180019D; Thu, 6 Jun 2024 01:29:15 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute6.internal (MEProxy); Thu, 06 Jun 2024 01:29:15 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm1; t=1717651754; x=1717738154; bh=dI+Q96esZj zF5B7TXZbygU9teTOcUNMShCKLNyfR7Zk=; b=Y+mrjE78PnKQqyiSnkAfl/Pn+f 9OdD9X4OH5joH1x6+vVCCtwUvGFTEnUtpE+JNAJ43lHhBQl2QNCaQ81tp9WZVtG3 W2FeheWRbBKhaDX5nx+Jyxblq3iEXvoPrYs3rxLSJgsxeapGQXE3z9j1DKXDXsSm M6j1r2ZyttO84zneqF4/rzinXSUD1WlWyJbcnvppk9Tz8AzhDICEYKVk6vjH4q/p n99rFcJ9uvn4j2Lj6GIzrred9KQtAB7JPDLBv+nGmMseQXW2/g35OllG87l3ViVF lH3f7r9cthUPPaFEoua3mcyY0/KEFFZESBd5qe9YK0xtIQRP8MuIo34qb22g== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1717651754; x=1717738154; bh=dI+Q96esZjzF5B7TXZbygU9teTOc UNMShCKLNyfR7Zk=; b=gLYrR+vqgL9XohR6pAXm/SVXZcx954CoyKfuSzuKrsHb F02ZoifGVnZjNFJ1YNZWU2iBKqRZ1sSr7ip630aikkmzML4YUfbZAfVtsS/nSFoc N4+RufCZPVKd/4RjuTuCDR0zlTOvI5PoRjpYKn2uI+ohC/+QmweBJBKSDtjEG7yO VSEHWsIJ/bArWcN1iuMuR4TgOkbusU+2DYK1HEf+h4FUS3/IMSyTuP0F90SqLPel /KJqFxDMih+KVy9UTBkgC8RtiB5097EwNirh7wCbN/ZGYUQ6rtXAMAMeM+DSqkaz p1LN+CbTyjrHyIF3pOJuokm2SvNScNB3Yq1qdy/x0g== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdeljedgleeiucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtvdenucfhrhhomheprfgrthhr ihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimheqnecuggftrfgrthhtvg hrnhepueektdevtdffveeljeetgfehheeigeekleduvdeffeeghefgledttdehjeelffet necuvehluhhsthgvrhfuihiivgepudenucfrrghrrghmpehmrghilhhfrhhomhepphhsse hpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 6 Jun 2024 01:29:13 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id 5efb1c21 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Thu, 6 Jun 2024 05:28:43 +0000 (UTC) Date: Thu, 6 Jun 2024 07:29:11 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Eric Sunshine , Junio C Hamano , Ramsay Jones , Justin Tobler , Karthik Nayak , Jeff King Subject: [PATCH v5 04/12] refs: allow to skip creation of reflog entries Message-ID: <053f1be657dd7a8b9ebc3251f09123e815f9a24c.1717649802.git.ps@pks.im> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: The ref backends do not have any way to disable the creation of reflog entries. This will be required for upcoming ref format migration logic so that we do not create any entries that didn't exist in the original ref database. Provide a new `REF_SKIP_CREATE_REFLOG` flag that allows the caller to disable reflog entry creation. Signed-off-by: Patrick Steinhardt --- refs.c | 6 ++++++ refs.h | 8 +++++++- refs/files-backend.c | 4 ++++ refs/reftable-backend.c | 3 ++- t/helper/test-ref-store.c | 1 + 5 files changed, 20 insertions(+), 2 deletions(-) diff --git a/refs.c b/refs.c index 423684b8b8..fa3b0a82d4 100644 --- a/refs.c +++ b/refs.c @@ -1194,6 +1194,12 @@ int ref_transaction_update(struct ref_transaction *transaction, { assert(err); + if ((flags & REF_FORCE_CREATE_REFLOG) && + (flags & REF_SKIP_CREATE_REFLOG)) { + strbuf_addstr(err, _("refusing to force and skip creation of reflog")); + return -1; + } + if (!(flags & REF_SKIP_REFNAME_VERIFICATION) && ((new_oid && !is_null_oid(new_oid)) ? check_refname_format(refname, REFNAME_ALLOW_ONELEVEL) : diff --git a/refs.h b/refs.h index a7afa9bede..50a2b3ab09 100644 --- a/refs.h +++ b/refs.h @@ -659,13 +659,19 @@ struct ref_transaction *ref_store_transaction_begin(struct ref_store *refs, */ #define REF_SKIP_REFNAME_VERIFICATION (1 << 11) +/* + * Skip creation of a reflog entry, even if it would have otherwise been + * created. + */ +#define REF_SKIP_CREATE_REFLOG (1 << 12) + /* * Bitmask of all of the flags that are allowed to be passed in to * ref_transaction_update() and friends: */ #define REF_TRANSACTION_UPDATE_ALLOWED_FLAGS \ (REF_NO_DEREF | REF_FORCE_CREATE_REFLOG | REF_SKIP_OID_VERIFICATION | \ - REF_SKIP_REFNAME_VERIFICATION) + REF_SKIP_REFNAME_VERIFICATION | REF_SKIP_CREATE_REFLOG) /* * Add a reference update to transaction. `new_oid` is the value that diff --git a/refs/files-backend.c b/refs/files-backend.c index 73380d7e99..bd0d63bcba 100644 --- a/refs/files-backend.c +++ b/refs/files-backend.c @@ -1750,6 +1750,9 @@ static int files_log_ref_write(struct files_ref_store *refs, { int logfd, result; + if (flags & REF_SKIP_CREATE_REFLOG) + return 0; + if (log_all_ref_updates == LOG_REFS_UNSET) log_all_ref_updates = is_bare_repository() ? LOG_REFS_NONE : LOG_REFS_NORMAL; @@ -2251,6 +2254,7 @@ static int split_head_update(struct ref_update *update, struct ref_update *new_update; if ((update->flags & REF_LOG_ONLY) || + (update->flags & REF_SKIP_CREATE_REFLOG) || (update->flags & REF_IS_PRUNING) || (update->flags & REF_UPDATE_VIA_HEAD)) return 0; diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c index f6edfdf5b3..bffed9257f 100644 --- a/refs/reftable-backend.c +++ b/refs/reftable-backend.c @@ -1103,7 +1103,8 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data if (ret) goto done; - } else if (u->flags & REF_HAVE_NEW && + } else if (!(u->flags & REF_SKIP_CREATE_REFLOG) && + (u->flags & REF_HAVE_NEW) && (u->flags & REF_FORCE_CREATE_REFLOG || should_write_log(&arg->refs->base, u->refname))) { struct reftable_log_record *log; diff --git a/t/helper/test-ref-store.c b/t/helper/test-ref-store.c index c9efd74c2b..ad24300170 100644 --- a/t/helper/test-ref-store.c +++ b/t/helper/test-ref-store.c @@ -126,6 +126,7 @@ static struct flag_definition transaction_flags[] = { FLAG_DEF(REF_FORCE_CREATE_REFLOG), FLAG_DEF(REF_SKIP_OID_VERIFICATION), FLAG_DEF(REF_SKIP_REFNAME_VERIFICATION), + FLAG_DEF(REF_SKIP_CREATE_REFLOG), { NULL, 0 } }; From patchwork Thu Jun 6 05:29:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13687822 Received: from wfhigh1-smtp.messagingengine.com (wfhigh1-smtp.messagingengine.com [64.147.123.152]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0E67139FEF for ; Thu, 6 Jun 2024 05:29:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=64.147.123.152 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717651763; cv=none; b=tINq9FNd8PSiQfbcuPq5Xu2XjDjVCeGy5yCnfPKzf30E/VeJvTzWIgtoeafgBeYqyco1Hg0GGesyvlQB6jJuG3reEZ0TpFvOQNMtcKCyxaxoWjo4EGYvFqGiZLp+XDER6v5bMhbYt0a1PZLcbiiGpid9EbZ6qNC72gWJQHDA/uw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717651763; c=relaxed/simple; bh=+541hOD0PpjWPWOp+/tlFdKd5rrJ4pkL20JN6vdUhqg=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=blHymrkvLSDh2/5CgY7loIOnf0HJSSMHy9gKDfnrw58I/HwRgOZTx7oKP0jnrD5uNUwn7d/3RwKe3D6DTmWSeT+o4hFE6RkHJWA0us+88apHbJdcASLUxl5o8nEbGsX9gd0G8QhC3hcDhNbYGBxWWEiOCJg0Ko2zmF1ae0QvD2A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=X7ZBnSy3; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=azaCatxY; arc=none smtp.client-ip=64.147.123.152 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="X7ZBnSy3"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="azaCatxY" Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailfhigh.west.internal (Postfix) with ESMTP id DF5BE180019A; Thu, 6 Jun 2024 01:29:20 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Thu, 06 Jun 2024 01:29:21 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm1; t=1717651760; x=1717738160; bh=7OXMiN6uPD IA2p9Ka0BkFmIPIbeyBY0irdGZXib6i5Y=; b=X7ZBnSy3pwkXU75X241b1jfmqd 5G4kRCbgdXg0ntQC/0SsnSBfAxY0D0+AOfDRAsBzFNfOBQB/rB8qXn35QbYPJeo6 3j9ggAiPKFtg36QSaz/N0I/wok0INvSnTHVhflgOEYDBQi3L4K2SyFbGdokEP2e7 iT2f+DvwjKqG5KlIfdRE9XHVd0AXpapDD+wbwPXTRulpOgfMBh6KGvHjrZe3siMI N+92WHTSYcRxecyrUFXJ4hyyypA89DjHQyVU/yDkQieKuY4+3imK4yIaXB0n6t3d 9AVEnSucbShMhtJgRoVhTvKijNQ+TcEJRK88bnrtSUnsLP0PTOsHS2ZeXrFA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1717651760; x=1717738160; bh=7OXMiN6uPDIA2p9Ka0BkFmIPIbey BY0irdGZXib6i5Y=; b=azaCatxYuahED4b60noCvQQEe9RKVSNb7nQMWwQXrVv8 dPbs2cK99a3upeCWSU7zzzZHqrnNiyfzrzffPUTmHeR3EY74dmaH1Q0AkWI2KHP9 HwVH9PglPPtFpFN8JCpZVtjt33eAwPP59IRa7RZemX1PrUnMRPcuHY5uFnavlyJI KC+Z3AmuixTN7zXXPdUHgt3TuMHzjnuPqPC5IqigVpg1mOaTLSrC90+pIremJqOL F0339vypt1B/P7kjKHRgIr/OGncQlOC57EaQvgNGVc9i59KoYCOWwCO1yETFpL60 AU3qNL7RC3F7WPIfpGHVE/xUhNmywlo67In1LFSUNA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdeljedgleeiucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtvdenucfhrhhomheprfgrthhr ihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimheqnecuggftrfgrthhtvg hrnhepueektdevtdffveeljeetgfehheeigeekleduvdeffeeghefgledttdehjeelffet necuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepphhsse hpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 6 Jun 2024 01:29:18 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id bdda3277 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Thu, 6 Jun 2024 05:28:48 +0000 (UTC) Date: Thu, 6 Jun 2024 07:29:16 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Eric Sunshine , Junio C Hamano , Ramsay Jones , Justin Tobler , Karthik Nayak , Jeff King Subject: [PATCH v5 05/12] refs/files: refactor `add_pseudoref_and_head_entries()` Message-ID: <29147da2b9c3e67d1fa780d25f81ad6cae8b1b77.1717649802.git.ps@pks.im> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: The `add_pseudoref_and_head_entries()` function accepts both the ref store as well as a directory name as input. This is unnecessary though as the ref store already uniquely identifies the root directory of the ref store anyway. Furthermore, the function is misnamed now that we have clarified the meaning of pseudorefs as it doesn't add pseudorefs, but root refs. Rename it accordingly. Signed-off-by: Patrick Steinhardt --- refs/files-backend.c | 15 ++++++--------- 1 file changed, 6 insertions(+), 9 deletions(-) diff --git a/refs/files-backend.c b/refs/files-backend.c index bd0d63bcba..b4e5437ffe 100644 --- a/refs/files-backend.c +++ b/refs/files-backend.c @@ -324,16 +324,14 @@ static void loose_fill_ref_dir(struct ref_store *ref_store, } /* - * Add pseudorefs to the ref dir by parsing the directory for any files - * which follow the pseudoref syntax. + * Add root refs to the ref dir by parsing the directory for any files which + * follow the root ref syntax. */ -static void add_pseudoref_and_head_entries(struct ref_store *ref_store, - struct ref_dir *dir, - const char *dirname) +static void add_root_refs(struct files_ref_store *refs, + struct ref_dir *dir) { - struct files_ref_store *refs = - files_downcast(ref_store, REF_STORE_READ, "fill_ref_dir"); struct strbuf path = STRBUF_INIT, refname = STRBUF_INIT; + const char *dirname = refs->loose->root->name; struct dirent *de; size_t dirnamelen; DIR *d; @@ -388,8 +386,7 @@ static struct ref_cache *get_loose_ref_cache(struct files_ref_store *refs, dir = get_ref_dir(refs->loose->root); if (flags & DO_FOR_EACH_INCLUDE_ROOT_REFS) - add_pseudoref_and_head_entries(dir->cache->ref_store, dir, - refs->loose->root->name); + add_root_refs(refs, dir); /* * Add an incomplete entry for "refs/" (to be filled From patchwork Thu Jun 6 05:29:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13687823 Received: from wfout6-smtp.messagingengine.com (wfout6-smtp.messagingengine.com [64.147.123.149]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AB9681EB3E for ; Thu, 6 Jun 2024 05:29:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=64.147.123.149 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717651768; cv=none; b=cMgUd5tD7pDlRybJo5c2S6rPSdkAMCTl4W5wT4yCn8Utuvbszrwh7VlhBQElkfF26KV3Zg8EforxqNUyXdSAP0QLFkcPuWDVu/YRIMK/9/agFKDrbw50sNdtzLTg1c718Ck2FsydEO4vhwend3ngHSWZpTWCGmcsr+O82Z4Q9SE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717651768; c=relaxed/simple; bh=KIEow5x8ANkf91C51IUkchPizzQarlOpQ8kcOeN1XpE=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=dXzIKgRZ3Hq91aB7+Kue43fazHQtVas51k4buZtr4BChEBiiCRN4OMoOLeIRfprPupN3Lr6Dp0fPnpIbB3svrXy+X9p9nhUzRxEAtC0LCFHo4OtWikNAVu8vYV9WhsQxbiHSFa23HQTMojjW6mI/TuKP0ByiZ4wmkidCeU1+hS0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=V1REwk4i; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=fH7VtIP3; arc=none smtp.client-ip=64.147.123.149 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="V1REwk4i"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="fH7VtIP3" Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailfout.west.internal (Postfix) with ESMTP id 9A3CC1C001D3; Thu, 6 Jun 2024 01:29:25 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Thu, 06 Jun 2024 01:29:26 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm1; t=1717651765; x=1717738165; bh=WQJP1pnefI aouRU8eOongE19YDHwuzNv0UWTEwiZDgk=; b=V1REwk4iowKxpqLm0ql+dHdSkB h4sp75g8iicCCBZfwS5WmgYc8WcC4t4TABKO9CBg9XHD0AHSiE5B9TgnnuC8tx4M Lpayb+20YZkIEIel/OBtXyToEFNOTO3x5IUbZ3/stbxqhrFQmEfqmP8UFjDd+dHB 1SrNlDl8907pmrSI2rXIz48aJ8ZSq6K6gvjVLioTs4zC36XdP/xTXcm/4sUPB0Tx x/GNXT5m7Ar6cve/cBDKpwab46OJ20JTaEf8Ktwf+DzUVNRslPZMnisMI18ofZLf KSzyakXKFXho4J/A0HRTOiquZIrWhXVssad/gUUCSxXWbtybjGPKjY7SjMyQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1717651765; x=1717738165; bh=WQJP1pnefIaouRU8eOongE19YDHw uzNv0UWTEwiZDgk=; b=fH7VtIP3kWr0dPl1eIA9AI5vHSvQYS08v3W8uavMVEm/ yGR96icOAv2gbMMUvSUuwaRT00nJDe/Wd3rqgGQ72zmNiYa+RSRo58+IoTHUvWzl aSrgAct5j8+VbjUtxxZy5EhbqPtwX44mXMv/WZDOxpKPu9Phf5VumNijMmDBwEix FtURXsrqc3zMCVHMAxMH25Z/+hAW30gzMZVvwQNYHD31k2iFjcVziv9DNvh6eyH/ 9qgvvSrE3soZ9JiXobue2n7VchVpFgbmJqqU0nRhvNU3fjullDw69pRx6ArEBvyu e2KNZzubvm+RHRNAElRX/Iwvq8iUuUZhjSSxTZkLYw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdeljedgleeiucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtvdenucfhrhhomheprfgrthhr ihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimheqnecuggftrfgrthhtvg hrnhepueektdevtdffveeljeetgfehheeigeekleduvdeffeeghefgledttdehjeelffet necuvehluhhsthgvrhfuihiivgepudenucfrrghrrghmpehmrghilhhfrhhomhepphhsse hpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 6 Jun 2024 01:29:23 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id c501f3c2 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Thu, 6 Jun 2024 05:28:53 +0000 (UTC) Date: Thu, 6 Jun 2024 07:29:20 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Eric Sunshine , Junio C Hamano , Ramsay Jones , Justin Tobler , Karthik Nayak , Jeff King Subject: [PATCH v5 06/12] refs/files: extract function to iterate through root refs Message-ID: <86cf0c84b121f23e7cbc3ba96299d9264123b485.1717649802.git.ps@pks.im> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Extract a new function that can be used to iterate through all root refs known to the "files" backend. This will be used in the next commit, where we start to teach ref backends to remove themselves. Signed-off-by: Patrick Steinhardt --- refs/files-backend.c | 51 ++++++++++++++++++++++++++++++++++++-------- 1 file changed, 42 insertions(+), 9 deletions(-) diff --git a/refs/files-backend.c b/refs/files-backend.c index b4e5437ffe..de8cc83174 100644 --- a/refs/files-backend.c +++ b/refs/files-backend.c @@ -323,17 +323,15 @@ static void loose_fill_ref_dir(struct ref_store *ref_store, add_per_worktree_entries_to_dir(dir, dirname); } -/* - * Add root refs to the ref dir by parsing the directory for any files which - * follow the root ref syntax. - */ -static void add_root_refs(struct files_ref_store *refs, - struct ref_dir *dir) +static int for_each_root_ref(struct files_ref_store *refs, + int (*cb)(const char *refname, void *cb_data), + void *cb_data) { struct strbuf path = STRBUF_INIT, refname = STRBUF_INIT; const char *dirname = refs->loose->root->name; struct dirent *de; size_t dirnamelen; + int ret; DIR *d; files_ref_path(refs, &path, dirname); @@ -341,7 +339,7 @@ static void add_root_refs(struct files_ref_store *refs, d = opendir(path.buf); if (!d) { strbuf_release(&path); - return; + return -1; } strbuf_addstr(&refname, dirname); @@ -357,14 +355,49 @@ static void add_root_refs(struct files_ref_store *refs, strbuf_addstr(&refname, de->d_name); dtype = get_dtype(de, &path, 1); - if (dtype == DT_REG && is_root_ref(de->d_name)) - loose_fill_ref_dir_regular_file(refs, refname.buf, dir); + if (dtype == DT_REG && is_root_ref(de->d_name)) { + ret = cb(refname.buf, cb_data); + if (ret) + goto done; + } strbuf_setlen(&refname, dirnamelen); } + + ret = 0; + +done: strbuf_release(&refname); strbuf_release(&path); closedir(d); + return ret; +} + +struct fill_root_ref_data { + struct files_ref_store *refs; + struct ref_dir *dir; +}; + +static int fill_root_ref(const char *refname, void *cb_data) +{ + struct fill_root_ref_data *data = cb_data; + loose_fill_ref_dir_regular_file(data->refs, refname, data->dir); + return 0; +} + +/* + * Add root refs to the ref dir by parsing the directory for any files which + * follow the root ref syntax. + */ +static void add_root_refs(struct files_ref_store *refs, + struct ref_dir *dir) +{ + struct fill_root_ref_data data = { + .refs = refs, + .dir = dir, + }; + + for_each_root_ref(refs, fill_root_ref, &data); } static struct ref_cache *get_loose_ref_cache(struct files_ref_store *refs, From patchwork Thu Jun 6 05:29:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13687824 Received: from wfhigh1-smtp.messagingengine.com (wfhigh1-smtp.messagingengine.com [64.147.123.152]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5F13A1EB3E for ; Thu, 6 Jun 2024 05:29:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=64.147.123.152 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717651771; cv=none; b=fsnjSO0dBGs1KMJZoi3QAOL5nCJqj6iiA+lenJPP1P3ki6GTEGmCkDhEEr0X9Jg0hB9PqG5sXMPqwCO+uKQI1o7iG/RKg2sQvyq/SSykzTj/34kXvha8Kwi0MntNYy3NdjqFMYF3fdtfeYtdk9JnhWtTB2aotLSMPdhAKEu6lrY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717651771; c=relaxed/simple; bh=1YdQl7j7f+9Y19Av3QmNoFuFJrRzizkSS0fFsV+HNdY=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=Ki/I7bVXeTVvZPuRAkL3ejahe5mVnxdchKcCsBHt2/1s1gFv1mVnI2pqLsRa1fbyZDLXRjWcX9RjvhKL06mDFMSylts7lDj5WDBhtPo1rk1s5DX/Wyfg3VcXZCUowoTCC8IzSO5jl+diuz3wIpt2B3aqP7BKFTWadQXXUf5up20= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=dZ93h024; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=jmsuNm+X; arc=none smtp.client-ip=64.147.123.152 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="dZ93h024"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="jmsuNm+X" Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailfhigh.west.internal (Postfix) with ESMTP id 687EB180019C; Thu, 6 Jun 2024 01:29:29 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Thu, 06 Jun 2024 01:29:29 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm1; t=1717651769; x=1717738169; bh=rK+Q01T0xB NzBMGjzOJmAYTcwS9seinEch1wZkGBYBg=; b=dZ93h024MHAEZE/mvVzb+SjMq/ JzN8PBSoSom/KirHZbieXgMMuGVWjbnx76qk/PDCMFGDamxjfVHXVJSBK+/vwh4b z8l/hD5rRcS6o/yZiorK3mJpOVaBmW9ye+Oo5Wq5Mm0ETMsE0rDeKWrB154w24F9 M4gkZGzg0dGDY/0hCvOxrVfXK3aN8A9lX7GNZwMOzVMiqzMjQu8mK9HiRbRXdwdg 2gJVFHYxDsl0s12OCh8vWh+OmxsjS040MVTupkqlSOUGRH1PuRRHzuYRtZPc9PSV V23vQPIaGIKvfMguptuAsIhGbOMmgEgJ9UhTh5/k4QKjFThcP8H//7ZThdgw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1717651769; x=1717738169; bh=rK+Q01T0xBNzBMGjzOJmAYTcwS9s einEch1wZkGBYBg=; b=jmsuNm+XvpMq+YCVTmNHzALwMRpxSF6lw5YF0Jd8FbqT AfUeoOq/5urDj+SZMRCzXoW1mZj396n3AwvPFuwLVS/50dI09gjeP7Ckhth0NS7R GKDJgaApsHpZu0TOpp2kUq7+MX27pSZUUjQUfE2jUjjWDW8pGXVVY9PJN6jN7kKl q1p1Teg84Z3wYqBa5Gmuhv2JxhPdhkys2rw90SS6laRIpc/0EeFSl4RezdPOPm76 fNKg8QN/87dTzaenB634Rz+kH1JeOjDgVE+LpLqTdXiUwRgejgrsTpvAk4w4b07Z FNUKYBr44GmLQOQ7wM6ZyQ4pAGUQ1Pzlg+m4dDRQdg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdeljedgleeiucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtvdenucfhrhhomheprfgrthhr ihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimheqnecuggftrfgrthhtvg hrnhepueektdevtdffveeljeetgfehheeigeekleduvdeffeeghefgledttdehjeelffet necuvehluhhsthgvrhfuihiivgepudenucfrrghrrghmpehmrghilhhfrhhomhepphhsse hpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 6 Jun 2024 01:29:27 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id 7ffdc90c (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Thu, 6 Jun 2024 05:28:57 +0000 (UTC) Date: Thu, 6 Jun 2024 07:29:25 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Eric Sunshine , Junio C Hamano , Ramsay Jones , Justin Tobler , Karthik Nayak , Jeff King Subject: [PATCH v5 07/12] refs/files: fix NULL pointer deref when releasing ref store Message-ID: <6b0aaf2ac829fe2c223a7e7b6ea0d86732a6a0eb.1717649802.git.ps@pks.im> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: The `free_ref_cache()` function is not `NULL` safe and will thus segfault when being passed such a pointer. This can easily happen when trying to release a partially initialized "files" ref store. Fix this. Signed-off-by: Patrick Steinhardt --- refs/ref-cache.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/refs/ref-cache.c b/refs/ref-cache.c index b6c53fc8ed..4ce519bbc8 100644 --- a/refs/ref-cache.c +++ b/refs/ref-cache.c @@ -71,6 +71,8 @@ static void free_ref_entry(struct ref_entry *entry) void free_ref_cache(struct ref_cache *cache) { + if (!cache) + return; free_ref_entry(cache->root); free(cache); } From patchwork Thu Jun 6 05:29:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13687825 Received: from wfhigh1-smtp.messagingengine.com (wfhigh1-smtp.messagingengine.com [64.147.123.152]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 43F2617BD5 for ; Thu, 6 Jun 2024 05:29:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=64.147.123.152 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717651776; cv=none; b=KRjFPnhn/Qx9569+ASQo+I6hXr6YHbj2MKi7OiFBMZITC1MbkQV1slQjjAxzoo3GaWaVW/vfaAvq1JRFCSTdcsXMTJSeHB4WH0ssvnOiJFUF1J+NVaTTWF8XIBoazHCH6Nisw3K9Wd4+uoB6NEU+dBzgdz3C9EXIs8rgyz4tdsY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717651776; c=relaxed/simple; bh=oV2/weQzu0bIrxiLBXZBdtTc6NO85mAyG01a0zRnO/E=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=G4zZIaYmKVPUJ3ZbVq95KS0jVUA7lNWPv6RAD9U8rpkcUElTok/FPTIZlLnukunYFg35s8nFgUJqkMjQBd2vEwsRaO9is6D5f7f2MY9+MA/4A3/0a87239xkthCWEBob8OvkQHWbXzZ1r+3iM17JY/TKvJYm1ykKLvK67SKgR5E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=eNGt+PYF; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=EKoCI5dB; arc=none smtp.client-ip=64.147.123.152 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="eNGt+PYF"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="EKoCI5dB" Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailfhigh.west.internal (Postfix) with ESMTP id 184991800199; Thu, 6 Jun 2024 01:29:34 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Thu, 06 Jun 2024 01:29:34 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm1; t=1717651773; x=1717738173; bh=NmXC40nQw3 ibAZH4sTdQfldyCVWUHiXgzFDFfceOW4I=; b=eNGt+PYFDdhcQEsYcA0k1+UxHH siX5zColR0JNg+hTGXC3nmDgMJYGjww8ohR82ePw1UNleKvcsJdahXfHiH1oc7nC ROxBhIExyXp1CHWdyo15gePgqm9TbpdacBRR0pyKEtXDbeH86jtQs0PMFtnSNIYh P7Gg9yEOw14w3hp0D7QDVqwHvxkM52QKYgfE2H19dfzDHwv+QNhRThxoOiWaXhxH LK4Y5pi/V2f56WpFfY3UGUojAaQM8w/thFJk99jy6g0FJPK+mdUQDPQQxyh613zR FASixRbYkNPrE6Ww80Qwi1bXQbX3BleFpDls6zsIIifUYQrfv+Rhq06OfKJg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1717651773; x=1717738173; bh=NmXC40nQw3ibAZH4sTdQfldyCVWU HiXgzFDFfceOW4I=; b=EKoCI5dB1EbZ0poJQIOZBYtMXn+kY6mVvpsMf+WtQ0FN Ssn2Zg8MgsRluUf4DSW8VAwDSHNlsiWDwvkZXfSN/joGrVeuaGLLVuFbv5cy/03q zAQ+b7t6tOfSfkQAalkMOfwILJ/YlckbcKSkPXjxhNBwzKMorJ/zFSV3CLe0Yb1E rXvXKibk8eo8+5gKLVkPlAuG9mXaVsRm8xP4YbQ+Zp3+xEkwUL2AGJumEfsf2J7B lPxKRNOUt8VOOdb/nhlue1ZatVJSk4sRgDWyHltNSM3N/lCdeiw/W76sG9jqufMB 19xC0BQUYMkPPFlyaueRO6qLWa96i9wgF6oETaIE+w== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdeljedgleeiucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtvdenucfhrhhomheprfgrthhr ihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimheqnecuggftrfgrthhtvg hrnhepueektdevtdffveeljeetgfehheeigeekleduvdeffeeghefgledttdehjeelffet necuvehluhhsthgvrhfuihiivgepvdenucfrrghrrghmpehmrghilhhfrhhomhepphhsse hpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 6 Jun 2024 01:29:32 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id 94414816 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Thu, 6 Jun 2024 05:29:02 +0000 (UTC) Date: Thu, 6 Jun 2024 07:29:30 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Eric Sunshine , Junio C Hamano , Ramsay Jones , Justin Tobler , Karthik Nayak , Jeff King Subject: [PATCH v5 08/12] reftable: inline `merged_table_release()` Message-ID: <0690d5eae983f53b1d4223e7a0c722b6b22e8bee.1717649802.git.ps@pks.im> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: The function `merged_table_release()` releases a merged table, whereas `reftable_merged_table_free()` releases a merged table and then also free's its pointer. But all callsites of `merged_table_release()` are in fact followed by `reftable_merged_table_free()`, which is redundant. Inline `merged_table_release()` into `reftable_merged_table_free()` to get rid of this redundance. Signed-off-by: Patrick Steinhardt --- reftable/merged.c | 12 ++---------- reftable/merged.h | 2 -- reftable/stack.c | 8 ++------ 3 files changed, 4 insertions(+), 18 deletions(-) diff --git a/reftable/merged.c b/reftable/merged.c index f85a24c678..804fdc0de0 100644 --- a/reftable/merged.c +++ b/reftable/merged.c @@ -207,19 +207,11 @@ int reftable_new_merged_table(struct reftable_merged_table **dest, return 0; } -/* clears the list of subtable, without affecting the readers themselves. */ -void merged_table_release(struct reftable_merged_table *mt) -{ - FREE_AND_NULL(mt->stack); - mt->stack_len = 0; -} - void reftable_merged_table_free(struct reftable_merged_table *mt) { - if (!mt) { + if (!mt) return; - } - merged_table_release(mt); + FREE_AND_NULL(mt->stack); reftable_free(mt); } diff --git a/reftable/merged.h b/reftable/merged.h index a2571dbc99..9db45c3196 100644 --- a/reftable/merged.h +++ b/reftable/merged.h @@ -24,6 +24,4 @@ struct reftable_merged_table { uint64_t max; }; -void merged_table_release(struct reftable_merged_table *mt); - #endif diff --git a/reftable/stack.c b/reftable/stack.c index a59ebe038d..984fd866d0 100644 --- a/reftable/stack.c +++ b/reftable/stack.c @@ -261,10 +261,8 @@ static int reftable_stack_reload_once(struct reftable_stack *st, char **names, new_tables = NULL; st->readers_len = new_readers_len; - if (st->merged) { - merged_table_release(st->merged); + if (st->merged) reftable_merged_table_free(st->merged); - } if (st->readers) { reftable_free(st->readers); } @@ -968,10 +966,8 @@ static int stack_write_compact(struct reftable_stack *st, done: reftable_iterator_destroy(&it); - if (mt) { - merged_table_release(mt); + if (mt) reftable_merged_table_free(mt); - } reftable_ref_record_release(&ref); reftable_log_record_release(&log); st->stats.entries_written += entries; From patchwork Thu Jun 6 05:29:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13687826 Received: from wfhigh1-smtp.messagingengine.com (wfhigh1-smtp.messagingengine.com [64.147.123.152]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B901917BD5 for ; Thu, 6 Jun 2024 05:29:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=64.147.123.152 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717651781; cv=none; b=dytvW+DUXfyUoZ1TyR7QN7gcI+Z1Ejoxr+e0+wvMJTJb9nxkNU9garA0Z0ubwpYZujKL52q8FdtPgcAwTnf4yvksV692VGb2XzBPFJ5fd0ssmd02+YGhH81rfLPcbFSM5hAGmYY5XhQSCQuZQx0TpGwEuVd0fKKm9o4DmMf0FOw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717651781; c=relaxed/simple; bh=ltfm9xLu6ItC0rks+3zNvnGzNe+bg5msMbWMaiRVW5w=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=bFEF5c5r4JmXBdqCA/NG6kLdwmPatytvfc0aw5u/GN9cf6C2bUs7d8v/uCdxpfBcj2dm6P1Vdy4VMtN6KBGpW9uapxbA1Yrv1OruocCL218rlS+hwoSJMC0FsoRECeRBG9NF8XEMLDPVNnGJkMN5/DiacAJUy4ebsIqNjODL35Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=Z7HugnK+; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=Rvfze/Sb; arc=none smtp.client-ip=64.147.123.152 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="Z7HugnK+"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="Rvfze/Sb" Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailfhigh.west.internal (Postfix) with ESMTP id CE57E1800199; Thu, 6 Jun 2024 01:29:38 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Thu, 06 Jun 2024 01:29:39 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm1; t=1717651778; x=1717738178; bh=7nH7QkcpSx Mrj8Q7vaunoijEXOhjlP3vVM+tZfZS6mk=; b=Z7HugnK+d6hQly6pYhcxhvSt/8 F9sW9Cu4gkeERvh4xGpHJlezzwapmyIQ2i7Um0l2ac+QjSOsMdMe/GgGKgOtnQVq RhE5UFHdo/RcM/M/aOd/O0xvHbwXma2ozyeM7SWxsF0NS625HcztJoJCjB7aw52f MGliigs35KnaAOcV9gGXifqRou5OpBq/EhbudKYIzFCtu0pjWY1n1RC9N//nU/8H XlrkRQOcqIAMaw3B6EhxO7X+jfTAS+1gOV93rxXW7/MRzOqtoKT9Z51MSsnW3t40 iz9oiVHZcd8XXI5SXRlyvcSEyVOvQPMjZp2eS+RbrW48QUhtKtsdW7cCo6+Q== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1717651778; x=1717738178; bh=7nH7QkcpSxMrj8Q7vaunoijEXOhj lP3vVM+tZfZS6mk=; b=Rvfze/SbHRiMHI9OvWliyBLCw1VjfvZBWdIl/In7HXcA bXZlAd2hoP3evJRgSwpfYsuI6NJyjMDy1E5dcxebjpyVzsujqc5Fk0U/Njl8lBkC Ch2LXMrUaa2xcx/rsShU7xfE46zlTsHWr+lTks+R+3xK6qlkLnlaiapdI6GNOnjj 4p96pnfz005PrrBwWgv2yBhtvCdlXSMHV5j5Vks+ElZzp+1BT3TkmzEcpdgEuVyR iNuLpE28O3toWTEkYxNnMVctnQkR7EbRVzFh8JCXHPTMDAQWR88OM4ZQljIP3pvF QDunacvwTNsmmIiKJidRgvu+maIFfXBmylFU3Hmvhw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdeljedgleeiucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtvdenucfhrhhomheprfgrthhr ihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimheqnecuggftrfgrthhtvg hrnhepueektdevtdffveeljeetgfehheeigeekleduvdeffeeghefgledttdehjeelffet necuvehluhhsthgvrhfuihiivgepfeenucfrrghrrghmpehmrghilhhfrhhomhepphhsse hpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 6 Jun 2024 01:29:36 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id b00dff25 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Thu, 6 Jun 2024 05:29:07 +0000 (UTC) Date: Thu, 6 Jun 2024 07:29:34 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Eric Sunshine , Junio C Hamano , Ramsay Jones , Justin Tobler , Karthik Nayak , Jeff King Subject: [PATCH v5 09/12] worktree: don't store main worktree twice Message-ID: <89699a641d0487140152159b50835e9c4fb28efe.1717649802.git.ps@pks.im> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: In `get_worktree_ref_store()` we either return the repository's main ref store, or we look up the ref store via the map of worktree ref stores. Which of these worktrees gets picked depends on the `is_current` bit of the worktree, which indicates whether the worktree is the one that corresponds to `the_repository`. The bit is getting set in `get_worktrees()`, but only after we have computed the list of all worktrees. This is too late though, because at that time we have already called `get_worktree_ref_store()` on each of the worktrees via `add_head_info()`. The consequence is that the current worktree will not have been marked accordingly, which means that we did not use the main ref store, but instead created a new ref store. We thus have two separate ref stores now that map to the same ref database. Fix this by setting `is_current` before we call `add_head_info()`. Signed-off-by: Patrick Steinhardt --- worktree.c | 29 +++++++++++------------------ 1 file changed, 11 insertions(+), 18 deletions(-) diff --git a/worktree.c b/worktree.c index 12eadacc61..70844d023a 100644 --- a/worktree.c +++ b/worktree.c @@ -53,6 +53,15 @@ static void add_head_info(struct worktree *wt) wt->is_detached = 1; } +static int is_current_worktree(struct worktree *wt) +{ + char *git_dir = absolute_pathdup(get_git_dir()); + const char *wt_git_dir = get_worktree_git_dir(wt); + int is_current = !fspathcmp(git_dir, absolute_path(wt_git_dir)); + free(git_dir); + return is_current; +} + /** * get the main worktree */ @@ -76,6 +85,7 @@ static struct worktree *get_main_worktree(int skip_reading_head) */ worktree->is_bare = (is_bare_repository_cfg == 1) || is_bare_repository(); + worktree->is_current = is_current_worktree(worktree); if (!skip_reading_head) add_head_info(worktree); return worktree; @@ -102,6 +112,7 @@ struct worktree *get_linked_worktree(const char *id, worktree->repo = the_repository; worktree->path = strbuf_detach(&worktree_path, NULL); worktree->id = xstrdup(id); + worktree->is_current = is_current_worktree(worktree); if (!skip_reading_head) add_head_info(worktree); @@ -111,23 +122,6 @@ struct worktree *get_linked_worktree(const char *id, return worktree; } -static void mark_current_worktree(struct worktree **worktrees) -{ - char *git_dir = absolute_pathdup(get_git_dir()); - int i; - - for (i = 0; worktrees[i]; i++) { - struct worktree *wt = worktrees[i]; - const char *wt_git_dir = get_worktree_git_dir(wt); - - if (!fspathcmp(git_dir, absolute_path(wt_git_dir))) { - wt->is_current = 1; - break; - } - } - free(git_dir); -} - /* * NEEDSWORK: This function exists so that we can look up metadata of a * worktree without trying to access any of its internals like the refdb. It @@ -164,7 +158,6 @@ static struct worktree **get_worktrees_internal(int skip_reading_head) ALLOC_GROW(list, counter + 1, alloc); list[counter] = NULL; - mark_current_worktree(list); return list; } From patchwork Thu Jun 6 05:29:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13687827 Received: from wfout6-smtp.messagingengine.com (wfout6-smtp.messagingengine.com [64.147.123.149]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AE6DD38F9C for ; Thu, 6 Jun 2024 05:29:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=64.147.123.149 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717651786; cv=none; b=AhdQTr5otke4GMl6SDqo7VZlR6BW1w1tYYKd1buOPZ7mdUlhKncdMkX6deiIfwRjQMmqlKAsoDmfXydh7C5egK6mjfumd1zbiYcgstuWW/4OfD0r3MdsVNchHDX4urV4nlQTEWhARAAp5GTz9hivYjPmDfJGDFOM35fcqTV+WFw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717651786; c=relaxed/simple; bh=GWsPpmc/e8WlZsLw60fwdL0NJea4HGKtxXdmiTChUmM=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=IiKRudbfxeAMkQdTFBRD3xOONDvEFlVqAEAqdbUVHpBkVisg4azk0eia/1RW61yYolu1tGDrcAiz+J/NNjXWYGSorms7RWwGHpqcUPzbZJrst6luF8QLY2RfWUA3jFBuzLk8f3n5GsRWnv0YGCdlQvzqaftaFkviBgdTfhtdSA8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=fHzk787e; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=CLyoRi5M; arc=none smtp.client-ip=64.147.123.149 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="fHzk787e"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="CLyoRi5M" Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailfout.west.internal (Postfix) with ESMTP id 99B3E1C001CF; Thu, 6 Jun 2024 01:29:43 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Thu, 06 Jun 2024 01:29:44 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm1; t=1717651783; x=1717738183; bh=YBjFRcmq/9 dGqKp+ruDeWDGb9J0m4MXTI9lpZeQ36hM=; b=fHzk787eNwxUL9dl2qP+lhsiO6 N17FYf15P5sW531cyOjdTUZL6Swe4ZDHaoZK4BuWIMVeUbjiB9C4KnscXvgaXKK6 z9lbRRr2ddboJnTYw3V1an43ESGfWM6mNiJzrY12YaAyOAeuS/wr6AfIldkkMmYZ Wo+NM6xrhqzLpkq4+itnbaMyjClJAHaCYpe1PLiT8kWyPm8pBKMHm+q2CEJY3VUY T9zvVh3VKT1isVrsF+n7JVkahNmM9iykUNbEBBDqkfGIWJsVfrNI7r1dy7SECQsU +7Vz8mPzU5+y8k2Dzrr7rpa5vmNMTXVI3p/rz7JIwZKivv/Z6ohhv20VeWgQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1717651783; x=1717738183; bh=YBjFRcmq/9dGqKp+ruDeWDGb9J0m 4MXTI9lpZeQ36hM=; b=CLyoRi5MYYUCPOVGuI/ncw0DcGHh+sPfVJkvzOs5CQVe 1ybre0EWfHJgd47JYkLFN0w0rQ3oq7qHSrFo3nsEs7a+6BHF605Rbq6XUYTdYrBq oOVBSvvtUlAn7XXZ44sgBduIskEbgMNnRBJbjCp55XlVftNilQMGelWYUY2ANJvD TazS41eFuYAesl3Sv20d4+hp60sjq9/MruqNAsqGkyLs+d4MxAkym1/scSrrY74A sKgXgmRl3P7jZTJhqyRKaAvJ/SJ/fxCtP68G2tbLbtEHxzq+z7nIQcsypMhpj/VG s11dwDnR2TtGZixWjc63nzXG7TtCawholnNq2RaLvQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdeljedgleeiucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtvdenucfhrhhomheprfgrthhr ihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimheqnecuggftrfgrthhtvg hrnhepueektdevtdffveeljeetgfehheeigeekleduvdeffeeghefgledttdehjeelffet necuvehluhhsthgvrhfuihiivgepgeenucfrrghrrghmpehmrghilhhfrhhomhepphhsse hpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 6 Jun 2024 01:29:41 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id 82111a79 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Thu, 6 Jun 2024 05:29:11 +0000 (UTC) Date: Thu, 6 Jun 2024 07:29:39 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Eric Sunshine , Junio C Hamano , Ramsay Jones , Justin Tobler , Karthik Nayak , Jeff King Subject: [PATCH v5 10/12] refs: implement removal of ref storages Message-ID: <7b5fee21856d970bdf6cd423d2b354e610e78df6.1717649802.git.ps@pks.im> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: We're about to introduce logic to migrate ref storages. One part of the migration will be to delete the files that are part of the old ref storage format. We don't yet have a way to delete such data generically across ref backends though. Implement a new `delete` callback and expose it via a new `ref_storage_delete()` function. Signed-off-by: Patrick Steinhardt --- refs.c | 5 ++++ refs.h | 5 ++++ refs/files-backend.c | 62 +++++++++++++++++++++++++++++++++++++++++ refs/packed-backend.c | 15 ++++++++++ refs/refs-internal.h | 7 +++++ refs/reftable-backend.c | 52 ++++++++++++++++++++++++++++++++++ 6 files changed, 146 insertions(+) diff --git a/refs.c b/refs.c index fa3b0a82d4..31fd391214 100644 --- a/refs.c +++ b/refs.c @@ -1861,6 +1861,11 @@ int ref_store_create_on_disk(struct ref_store *refs, int flags, struct strbuf *e return refs->be->create_on_disk(refs, flags, err); } +int ref_store_remove_on_disk(struct ref_store *refs, struct strbuf *err) +{ + return refs->be->remove_on_disk(refs, err); +} + int repo_resolve_gitlink_ref(struct repository *r, const char *submodule, const char *refname, struct object_id *oid) diff --git a/refs.h b/refs.h index 50a2b3ab09..61ee7b7a15 100644 --- a/refs.h +++ b/refs.h @@ -129,6 +129,11 @@ int ref_store_create_on_disk(struct ref_store *refs, int flags, struct strbuf *e */ void ref_store_release(struct ref_store *ref_store); +/* + * Remove the ref store from disk. This deletes all associated data. + */ +int ref_store_remove_on_disk(struct ref_store *refs, struct strbuf *err); + /* * Return the peeled value of the oid currently being iterated via * for_each_ref(), etc. This is equivalent to calling: diff --git a/refs/files-backend.c b/refs/files-backend.c index de8cc83174..e663781199 100644 --- a/refs/files-backend.c +++ b/refs/files-backend.c @@ -3342,11 +3342,73 @@ static int files_ref_store_create_on_disk(struct ref_store *ref_store, return 0; } +struct remove_one_root_ref_data { + const char *gitdir; + struct strbuf *err; +}; + +static int remove_one_root_ref(const char *refname, + void *cb_data) +{ + struct remove_one_root_ref_data *data = cb_data; + struct strbuf buf = STRBUF_INIT; + int ret = 0; + + strbuf_addf(&buf, "%s/%s", data->gitdir, refname); + + ret = unlink(buf.buf); + if (ret < 0) + strbuf_addf(data->err, "could not delete %s: %s\n", + refname, strerror(errno)); + + strbuf_release(&buf); + return ret; +} + +static int files_ref_store_remove_on_disk(struct ref_store *ref_store, + struct strbuf *err) +{ + struct files_ref_store *refs = + files_downcast(ref_store, REF_STORE_WRITE, "remove"); + struct remove_one_root_ref_data data = { + .gitdir = refs->base.gitdir, + .err = err, + }; + struct strbuf sb = STRBUF_INIT; + int ret = 0; + + strbuf_addf(&sb, "%s/refs", refs->base.gitdir); + if (remove_dir_recursively(&sb, 0) < 0) { + strbuf_addf(err, "could not delete refs: %s", + strerror(errno)); + ret = -1; + } + strbuf_reset(&sb); + + strbuf_addf(&sb, "%s/logs", refs->base.gitdir); + if (remove_dir_recursively(&sb, 0) < 0) { + strbuf_addf(err, "could not delete logs: %s", + strerror(errno)); + ret = -1; + } + strbuf_reset(&sb); + + if (for_each_root_ref(refs, remove_one_root_ref, &data) < 0) + ret = -1; + + if (ref_store_remove_on_disk(refs->packed_ref_store, err) < 0) + ret = -1; + + strbuf_release(&sb); + return ret; +} + struct ref_storage_be refs_be_files = { .name = "files", .init = files_ref_store_init, .release = files_ref_store_release, .create_on_disk = files_ref_store_create_on_disk, + .remove_on_disk = files_ref_store_remove_on_disk, .transaction_prepare = files_transaction_prepare, .transaction_finish = files_transaction_finish, diff --git a/refs/packed-backend.c b/refs/packed-backend.c index 2789fd92f5..c4c1e36aa2 100644 --- a/refs/packed-backend.c +++ b/refs/packed-backend.c @@ -1,5 +1,6 @@ #include "../git-compat-util.h" #include "../config.h" +#include "../dir.h" #include "../gettext.h" #include "../hash.h" #include "../hex.h" @@ -1266,6 +1267,19 @@ static int packed_ref_store_create_on_disk(struct ref_store *ref_store UNUSED, return 0; } +static int packed_ref_store_remove_on_disk(struct ref_store *ref_store, + struct strbuf *err) +{ + struct packed_ref_store *refs = packed_downcast(ref_store, 0, "remove"); + + if (remove_path(refs->path) < 0) { + strbuf_addstr(err, "could not delete packed-refs"); + return -1; + } + + return 0; +} + /* * Write the packed refs from the current snapshot to the packed-refs * tempfile, incorporating any changes from `updates`. `updates` must @@ -1724,6 +1738,7 @@ struct ref_storage_be refs_be_packed = { .init = packed_ref_store_init, .release = packed_ref_store_release, .create_on_disk = packed_ref_store_create_on_disk, + .remove_on_disk = packed_ref_store_remove_on_disk, .transaction_prepare = packed_transaction_prepare, .transaction_finish = packed_transaction_finish, diff --git a/refs/refs-internal.h b/refs/refs-internal.h index 33749fbd83..cbcb6f9c36 100644 --- a/refs/refs-internal.h +++ b/refs/refs-internal.h @@ -517,6 +517,12 @@ typedef int ref_store_create_on_disk_fn(struct ref_store *refs, int flags, struct strbuf *err); +/* + * Remove the reference store from disk. + */ +typedef int ref_store_remove_on_disk_fn(struct ref_store *refs, + struct strbuf *err); + typedef int ref_transaction_prepare_fn(struct ref_store *refs, struct ref_transaction *transaction, struct strbuf *err); @@ -649,6 +655,7 @@ struct ref_storage_be { ref_store_init_fn *init; ref_store_release_fn *release; ref_store_create_on_disk_fn *create_on_disk; + ref_store_remove_on_disk_fn *remove_on_disk; ref_transaction_prepare_fn *transaction_prepare; ref_transaction_finish_fn *transaction_finish; diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c index bffed9257f..da6b3162f3 100644 --- a/refs/reftable-backend.c +++ b/refs/reftable-backend.c @@ -1,6 +1,7 @@ #include "../git-compat-util.h" #include "../abspath.h" #include "../chdir-notify.h" +#include "../dir.h" #include "../environment.h" #include "../gettext.h" #include "../hash.h" @@ -343,6 +344,56 @@ static int reftable_be_create_on_disk(struct ref_store *ref_store, return 0; } +static int reftable_be_remove_on_disk(struct ref_store *ref_store, + struct strbuf *err) +{ + struct reftable_ref_store *refs = + reftable_be_downcast(ref_store, REF_STORE_WRITE, "remove"); + struct strbuf sb = STRBUF_INIT; + int ret = 0; + + /* + * Release the ref store such that all stacks are closed. This is + * required so that the "tables.list" file is not open anymore, which + * would otherwise make it impossible to remove the file on Windows. + */ + reftable_be_release(ref_store); + + strbuf_addf(&sb, "%s/reftable", refs->base.gitdir); + if (remove_dir_recursively(&sb, 0) < 0) { + strbuf_addf(err, "could not delete reftables: %s", + strerror(errno)); + ret = -1; + } + strbuf_reset(&sb); + + strbuf_addf(&sb, "%s/HEAD", refs->base.gitdir); + if (unlink(sb.buf) < 0) { + strbuf_addf(err, "could not delete stub HEAD: %s", + strerror(errno)); + ret = -1; + } + strbuf_reset(&sb); + + strbuf_addf(&sb, "%s/refs/heads", refs->base.gitdir); + if (unlink(sb.buf) < 0) { + strbuf_addf(err, "could not delete stub heads: %s", + strerror(errno)); + ret = -1; + } + strbuf_reset(&sb); + + strbuf_addf(&sb, "%s/refs", refs->base.gitdir); + if (rmdir(sb.buf) < 0) { + strbuf_addf(err, "could not delete refs directory: %s", + strerror(errno)); + ret = -1; + } + + strbuf_release(&sb); + return ret; +} + struct reftable_ref_iterator { struct ref_iterator base; struct reftable_ref_store *refs; @@ -2196,6 +2247,7 @@ struct ref_storage_be refs_be_reftable = { .init = reftable_be_init, .release = reftable_be_release, .create_on_disk = reftable_be_create_on_disk, + .remove_on_disk = reftable_be_remove_on_disk, .transaction_prepare = reftable_be_transaction_prepare, .transaction_finish = reftable_be_transaction_finish, From patchwork Thu Jun 6 05:29:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13687828 Received: from wfhigh1-smtp.messagingengine.com (wfhigh1-smtp.messagingengine.com [64.147.123.152]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 588D738F9C for ; Thu, 6 Jun 2024 05:29:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=64.147.123.152 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717651792; cv=none; b=Vcdp0bmHe5bpfdb7bSohavwga3f0wg/udLoO7DUZoK54grVaffJkyOxhxYtF1GoC9ESy99CB7rZ56xU2ktbAS2c+WEkky4S/qSCIwFjgfKl3dP8nVb/Szz5+1jWkjddNqp0AU3TIEpKJGhgs5n0nLTYOO5l9oqJVCs4W9fOccHA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717651792; c=relaxed/simple; bh=1tr9XhwRvfMdT/ifnal/9H4ylbW1BX/8wNio8OszqXo=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=j4NRloCLmUo/FeBOM75EUOIi0je4Y7L3D7+VrabmaQGOD/cMEiup8eG6YC/FNbAJxU2enJ55SfgZDE2ud8nUifI68B6g5GKLjoFfVH8C2AIO51rbESjPnbhkPUG2DO+iw48b3GexxKcIm6dD6J9RNq+aNTmBPvpkUkNngIPAC7M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=CKHb2+G3; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=FLLrRO+K; arc=none smtp.client-ip=64.147.123.152 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="CKHb2+G3"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="FLLrRO+K" Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailfhigh.west.internal (Postfix) with ESMTP id 5B16E180019C; Thu, 6 Jun 2024 01:29:49 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Thu, 06 Jun 2024 01:29:49 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm1; t=1717651789; x=1717738189; bh=o2qBq0Y8NU 8IavnD7S1O7juVcY8HOGyY7cdGbq5FODc=; b=CKHb2+G3M3JoH5aXy2cc7xwbKX QnMdN4/6trn/LjVdItOAm341lB6vpuxCHg7U02J41wzTkdjtNumsWfxtdiVExTR/ xa/kY2SYJlxDKLs99jnLHTVdGImKbDeaUX93qMZoJmcTZ9il/lhyvg4B9JVMSa47 /+dzM383eYYWSkzRjSnQemeXuBV4OtCk4jt1LSDTU9EXL6Y6/7ZUjT4AvFIm656t U0c+x5X1luB/FvShTSMRId3hZPxsErUQHtiIyTQrEAErRzWOlsY+NC54TBaXeLOh ydshptS7QWSiyGlsCKg2aHy1fX5IfMxefcIQPC01u2/QtxlNwD0I2gU0ep9w== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1717651789; x=1717738189; bh=o2qBq0Y8NU8IavnD7S1O7juVcY8H OGyY7cdGbq5FODc=; b=FLLrRO+KXuzqyDyIrvVx0EWlnjT3HxmdGWooQ3przo1S FGNKtdOhi+i8XrrIkmXfhw5tIS0KAXpl4fJ0kZAnhZ/S9hBNLnUw6FvIWVXkL/4l 8r49+/t03nicVL9NDiIQyJT0Sh1kASANTM9ofJzfHvnNS2H5n9EeK+hEVGDDYMpL DvGFAOe22TQiNWn4tDIgbu9vtMfg+X3rvO3hYZ27zJe8idEPoGJJ/x/vWVBWWVpJ DXGfmBeM4O4s6f72bsCkLl2xZ8z6QD+50w8QFanixz6cEBcf8XD4oHIy+9hzJDhG IDNBKy9IKRUSHSUKTEQ3vrPARrsBrwnHl0/lWM76jw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdeljedgleeiucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtvdenucfhrhhomheprfgrthhr ihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimheqnecuggftrfgrthhtvg hrnhepueektdevtdffveeljeetgfehheeigeekleduvdeffeeghefgledttdehjeelffet necuvehluhhsthgvrhfuihiivgepfeenucfrrghrrghmpehmrghilhhfrhhomhepphhsse hpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 6 Jun 2024 01:29:47 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id 4f99c112 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Thu, 6 Jun 2024 05:29:17 +0000 (UTC) Date: Thu, 6 Jun 2024 07:29:45 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Eric Sunshine , Junio C Hamano , Ramsay Jones , Justin Tobler , Karthik Nayak , Jeff King Subject: [PATCH v5 11/12] refs: implement logic to migrate between ref storage formats Message-ID: <893d99e98ed820ad210540de190d706cc057b2ee.1717649803.git.ps@pks.im> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: With the introduction of the new "reftable" backend, users may want to migrate repositories between the backends without having to recreate the whole repository. Add the logic to do so. The implementation is generic and works with arbitrary ref storage formats so that a backend does not need to implement any migration logic. It does have a few limitations though: - We do not migrate repositories with worktrees, because worktrees have separate ref storages. It makes the overall affair more complex if we have to migrate multiple storages at once. - We do not migrate reflogs, because we have no interfaces to write many reflog entries. - We do not lock the repository for concurrent access, and thus concurrent writes may end up with weird in-between states. There is no way to fully lock the "files" backend for writes due to its format, and thus we punt on this topic altogether and defer to the user to avoid those from happening. In other words, this version is a minimum viable product for migrating a repository's ref storage format. It works alright for bare repos, which often have neither worktrees nor reflogs. But it will not work for many other repositories without some preparations. These limitations are not set into stone though, and ideally we will eventually address them over time. The logic is not yet used by anything, and thus there are no tests for it. Those will be added in the next commit. Signed-off-by: Patrick Steinhardt --- refs.c | 308 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++ refs.h | 18 ++++ 2 files changed, 326 insertions(+) diff --git a/refs.c b/refs.c index 31fd391214..1304d3dd87 100644 --- a/refs.c +++ b/refs.c @@ -2570,3 +2570,311 @@ int ref_update_check_old_target(const char *referent, struct ref_update *update, referent, update->old_target); return -1; } + +struct migration_data { + struct ref_store *old_refs; + struct ref_transaction *transaction; + struct strbuf *errbuf; +}; + +static int migrate_one_ref(const char *refname, const struct object_id *oid, + int flags, void *cb_data) +{ + struct migration_data *data = cb_data; + struct strbuf symref_target = STRBUF_INIT; + int ret; + + if (flags & REF_ISSYMREF) { + ret = refs_read_symbolic_ref(data->old_refs, refname, &symref_target); + if (ret < 0) + goto done; + + ret = ref_transaction_update(data->transaction, refname, NULL, null_oid(), + symref_target.buf, NULL, + REF_SKIP_CREATE_REFLOG | REF_NO_DEREF, NULL, data->errbuf); + if (ret < 0) + goto done; + } else { + ret = ref_transaction_create(data->transaction, refname, oid, + REF_SKIP_CREATE_REFLOG | REF_SKIP_OID_VERIFICATION, + NULL, data->errbuf); + if (ret < 0) + goto done; + } + +done: + strbuf_release(&symref_target); + return ret; +} + +static int move_files(const char *from_path, const char *to_path, struct strbuf *errbuf) +{ + struct strbuf from_buf = STRBUF_INIT, to_buf = STRBUF_INIT; + size_t from_len, to_len; + DIR *from_dir; + int ret; + + from_dir = opendir(from_path); + if (!from_dir) { + strbuf_addf(errbuf, "could not open source directory '%s': %s", + from_path, strerror(errno)); + ret = -1; + goto done; + } + + strbuf_addstr(&from_buf, from_path); + strbuf_complete(&from_buf, '/'); + from_len = from_buf.len; + + strbuf_addstr(&to_buf, to_path); + strbuf_complete(&to_buf, '/'); + to_len = to_buf.len; + + while (1) { + struct dirent *ent; + + errno = 0; + ent = readdir(from_dir); + if (!ent) + break; + + if (!strcmp(ent->d_name, ".") || + !strcmp(ent->d_name, "..")) + continue; + + strbuf_setlen(&from_buf, from_len); + strbuf_addstr(&from_buf, ent->d_name); + + strbuf_setlen(&to_buf, to_len); + strbuf_addstr(&to_buf, ent->d_name); + + ret = rename(from_buf.buf, to_buf.buf); + if (ret < 0) { + strbuf_addf(errbuf, "could not link file '%s' to '%s': %s", + from_buf.buf, to_buf.buf, strerror(errno)); + goto done; + } + } + + if (errno) { + strbuf_addf(errbuf, "could not read entry from directory '%s': %s", + from_path, strerror(errno)); + ret = -1; + goto done; + } + + ret = 0; + +done: + strbuf_release(&from_buf); + strbuf_release(&to_buf); + if (from_dir) + closedir(from_dir); + return ret; +} + +static int count_reflogs(const char *reflog UNUSED, void *payload) +{ + size_t *reflog_count = payload; + (*reflog_count)++; + return 0; +} + +static int has_worktrees(void) +{ + struct worktree **worktrees = get_worktrees(); + int ret = 0; + size_t i; + + for (i = 0; worktrees[i]; i++) { + if (is_main_worktree(worktrees[i])) + continue; + ret = 1; + } + + free_worktrees(worktrees); + return ret; +} + +int repo_migrate_ref_storage_format(struct repository *repo, + enum ref_storage_format format, + unsigned int flags, + struct strbuf *errbuf) +{ + struct ref_store *old_refs = NULL, *new_refs = NULL; + struct ref_transaction *transaction = NULL; + struct strbuf new_gitdir = STRBUF_INIT; + struct migration_data data; + size_t reflog_count = 0; + int did_migrate_refs = 0; + int ret; + + if (repo->ref_storage_format == format) { + strbuf_addstr(errbuf, "current and new ref storage format are equal"); + ret = -1; + goto done; + } + + old_refs = get_main_ref_store(repo); + + /* + * We do not have any interfaces that would allow us to write many + * reflog entries. Once we have them we can remove this restriction. + */ + if (refs_for_each_reflog(old_refs, count_reflogs, &reflog_count) < 0) { + strbuf_addstr(errbuf, "cannot count reflogs"); + ret = -1; + goto done; + } + if (reflog_count) { + strbuf_addstr(errbuf, "migrating reflogs is not supported yet"); + ret = -1; + goto done; + } + + /* + * Worktrees complicate the migration because every worktree has a + * separate ref storage. While it should be feasible to implement, this + * is pushed out to a future iteration. + * + * TODO: we should really be passing the caller-provided repository to + * `has_worktrees()`, but our worktree subsystem doesn't yet support + * that. + */ + if (has_worktrees()) { + strbuf_addstr(errbuf, "migrating repositories with worktrees is not supported yet"); + ret = -1; + goto done; + } + + /* + * The overall logic looks like this: + * + * 1. Set up a new temporary directory and initialize it with the new + * format. This is where all refs will be migrated into. + * + * 2. Enumerate all refs and write them into the new ref storage. + * This operation is safe as we do not yet modify the main + * repository. + * + * 3. If we're in dry-run mode then we are done and can hand over the + * directory to the caller for inspection. If not, we now start + * with the destructive part. + * + * 4. Delete the old ref storage from disk. As we have a copy of refs + * in the new ref storage it's okay(ish) if we now get interrupted + * as there is an equivalent copy of all refs available. + * + * 5. Move the new ref storage files into place. + * + * 6. Change the repository format to the new ref format. + */ + strbuf_addf(&new_gitdir, "%s/%s", old_refs->gitdir, "ref_migration.XXXXXX"); + if (!mkdtemp(new_gitdir.buf)) { + strbuf_addf(errbuf, "cannot create migration directory: %s", + strerror(errno)); + ret = -1; + goto done; + } + + new_refs = ref_store_init(repo, format, new_gitdir.buf, + REF_STORE_ALL_CAPS); + ret = ref_store_create_on_disk(new_refs, 0, errbuf); + if (ret < 0) + goto done; + + transaction = ref_store_transaction_begin(new_refs, errbuf); + if (!transaction) + goto done; + + data.old_refs = old_refs; + data.transaction = transaction; + data.errbuf = errbuf; + + /* + * We need to use the internal `do_for_each_ref()` here so that we can + * also include broken refs and symrefs. These would otherwise be + * skipped silently. + * + * Ideally, we would do this call while locking the old ref storage + * such that there cannot be any concurrent modifications. We do not + * have the infra for that though, and the "files" backend does not + * allow for a central lock due to its design. It's thus on the user to + * ensure that there are no concurrent writes. + */ + ret = do_for_each_ref(old_refs, "", NULL, migrate_one_ref, 0, + DO_FOR_EACH_INCLUDE_ROOT_REFS | DO_FOR_EACH_INCLUDE_BROKEN, + &data); + if (ret < 0) + goto done; + + /* + * TODO: we might want to migrate to `initial_ref_transaction_commit()` + * here, which is more efficient for the files backend because it would + * write new refs into the packed-refs file directly. At this point, + * the files backend doesn't handle pseudo-refs and symrefs correctly + * though, so this requires some more work. + */ + ret = ref_transaction_commit(transaction, errbuf); + if (ret < 0) + goto done; + did_migrate_refs = 1; + + if (flags & REPO_MIGRATE_REF_STORAGE_FORMAT_DRYRUN) { + printf(_("Finished dry-run migration of refs, " + "the result can be found at '%s'\n"), new_gitdir.buf); + ret = 0; + goto done; + } + + /* + * Until now we were in the non-destructive phase, where we only + * populated the new ref store. From hereon though we are about + * to get hands by deleting the old ref store and then moving + * the new one into place. + * + * Assuming that there were no concurrent writes, the new ref + * store should have all information. So if we fail from hereon + * we may be in an in-between state, but it would still be able + * to recover by manually moving remaining files from the + * temporary migration directory into place. + */ + ret = ref_store_remove_on_disk(old_refs, errbuf); + if (ret < 0) + goto done; + + ret = move_files(new_gitdir.buf, old_refs->gitdir, errbuf); + if (ret < 0) + goto done; + + if (rmdir(new_gitdir.buf) < 0) + warning_errno(_("could not remove temporary migration directory '%s'"), + new_gitdir.buf); + + /* + * We have migrated the repository, so we now need to adjust the + * repository format so that clients will use the new ref store. + * We also need to swap out the repository's main ref store. + */ + initialize_repository_version(hash_algo_by_ptr(repo->hash_algo), format, 1); + + free(new_refs->gitdir); + new_refs->gitdir = xstrdup(old_refs->gitdir); + repo->refs_private = new_refs; + ref_store_release(old_refs); + + ret = 0; + +done: + if (ret && did_migrate_refs) { + strbuf_complete(errbuf, '\n'); + strbuf_addf(errbuf, _("migrated refs can be found at '%s'"), + new_gitdir.buf); + } + + if (ret && new_refs) + ref_store_release(new_refs); + ref_transaction_free(transaction); + strbuf_release(&new_gitdir); + return ret; +} diff --git a/refs.h b/refs.h index 61ee7b7a15..76d25df4de 100644 --- a/refs.h +++ b/refs.h @@ -1070,6 +1070,24 @@ int is_root_ref(const char *refname); */ int is_pseudo_ref(const char *refname); +/* + * The following flags can be passed to `repo_migrate_ref_storage_format()`: + * + * - REPO_MIGRATE_REF_STORAGE_FORMAT_DRYRUN: perform a dry-run migration + * without touching the main repository. The result will be written into a + * temporary ref storage directory. + */ +#define REPO_MIGRATE_REF_STORAGE_FORMAT_DRYRUN (1 << 0) + +/* + * Migrate the ref storage format used by the repository to the + * specified one. + */ +int repo_migrate_ref_storage_format(struct repository *repo, + enum ref_storage_format format, + unsigned int flags, + struct strbuf *err); + /* * The following functions have been removed in Git v2.45 in favor of functions * that receive a `ref_store` as parameter. The intent of this section is From patchwork Thu Jun 6 05:29:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13687829 Received: from wfhigh1-smtp.messagingengine.com (wfhigh1-smtp.messagingengine.com [64.147.123.152]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1C12138F9C for ; Thu, 6 Jun 2024 05:29:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=64.147.123.152 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717651797; cv=none; b=Rt/8ABnguq07wc9qNbAmsHQPOl9s5540SdPDHA2RN3vDp3zJbuLJUdnROViZOg4iEhe2J8AA+nAwRRc6qRNbR7oZL1MaIcAlzkFBxCFWJxZAlJVRuyxNUOw6CZwnlYFe9Qy11Zqbnz4Fjdxk8tD1obVfgQFg+zaohlS49pb3lbw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717651797; c=relaxed/simple; bh=Ls33Nz1zgdq9zLOU6DLSCaa2hPQ6tcjWvTownYXN0Q4=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=XAWAEN+mkUI2lat3DvFZ+slz/tQX3Fl5JTwrv8IWs5FP6bJTT+laPkGSGH5lzkKLWOj09nEDvAerzaw3IHhPEEcF69E6VRgOWvTjNuSmdRr4zlPCy5v3b21JASrExCRXfz/fSgcRZoYi82EbZFE6XVeEtsJVDwja759ksM8aLRg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=NZf6+Fwb; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=JeHdBhqg; arc=none smtp.client-ip=64.147.123.152 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="NZf6+Fwb"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="JeHdBhqg" Received: from compute6.internal (compute6.nyi.internal [10.202.2.47]) by mailfhigh.west.internal (Postfix) with ESMTP id 0FE78180019C; Thu, 6 Jun 2024 01:29:54 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute6.internal (MEProxy); Thu, 06 Jun 2024 01:29:54 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm1; t=1717651793; x=1717738193; bh=T0Ujyg11Cq 5UxLnkwmBGwMis6T8o8hBnXP+4hoGT3E8=; b=NZf6+Fwb88qWHFcLMZxpcqO5sf mabV0JPbpug3cY68IoEYu4/Kmd+GNFrtdVitWf4V6AuVUswX3+Yjmh07qtdGiIQH OQo0r3OMFSKXdDpC3DCwrrvXXL719arwp8ITL29NNrhNt8MIEpomL5+uPE+5gaGd yA1aUbRCxpKW50cNYcfjRaIP84/Xx721PFrI+jCFkpU52SHy8kM78hWIN4ZwSk9e NVMsCk6Z/K8vdgkZHOjkS+oKKqC9VmuqPN6dN5XrM5/UkwsYGLlx1YzWg+KRQJKM 0fNdrO1TucP4I70IMw9IF/Fqvao3WwYLVs5MR3hm9pZwtispDHkK0nRRKAMw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1717651793; x=1717738193; bh=T0Ujyg11Cq5UxLnkwmBGwMis6T8o 8hBnXP+4hoGT3E8=; b=JeHdBhqgKDgEGcpv582DYuqauomqZPPur/m4ngO7Vhfl bv3AUMS64Z92x+FVu2AlegLI5BnfEoGdKY2O57rT6oyiWWOcPhWd21/w6LJ02d1t 1NRIJETrWYpMFt+dXdln0mrjiH9jbZxa+m0GsWNbMc+6djMHTyKQkp1dg9p2XezF gUtR4dp9igb9qXLDItIia2A7O2sYtzuf2L263PXkp8odr1//rGDbvtG3Xe7OvMAw v4TzuXK9AESYRrVWETBH3PfSgdy1nNdfSF4f4a8IiAacr9iaE/NbGhAbHMJBzt/T 3UaZrE9VgV0HHMZKz4DgJtCSJcc/k8Wf8WLMSD37Bg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvdeljedgleeiucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtvdenucfhrhhomheprfgrthhr ihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimheqnecuggftrfgrthhtvg hrnhepueektdevtdffveeljeetgfehheeigeekleduvdeffeeghefgledttdehjeelffet necuvehluhhsthgvrhfuihiivgepvdenucfrrghrrghmpehmrghilhhfrhhomhepphhsse hpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 6 Jun 2024 01:29:52 -0400 (EDT) Received: by localhost (OpenSMTPD) with ESMTPSA id 9501e0a7 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Thu, 6 Jun 2024 05:29:21 +0000 (UTC) Date: Thu, 6 Jun 2024 07:29:49 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Eric Sunshine , Junio C Hamano , Ramsay Jones , Justin Tobler , Karthik Nayak , Jeff King Subject: [PATCH v5 12/12] builtin/refs: new command to migrate ref storage formats Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Introduce a new command that allows the user to migrate a repository between ref storage formats. This new command is implemented as part of a new git-refs(1) executable. This is due to two reasons: - There is no good place to put the migration logic in existing commands. git-maintenance(1) felt unwieldy, and git-pack-refs(1) is not the correct place to put it, either. - I had it in my mind to create a new low-level command for accessing refs for quite a while already. git-refs(1) is that command and can over time grow more functionality relating to refs. This should help discoverability by consolidating low-level access to refs into a single executable. As mentioned in the preceding commit that introduces the ref storage format migration logic, the new `git refs migrate` command still has a bunch of restrictions. These restrictions are documented accordingly. Signed-off-by: Patrick Steinhardt --- .gitignore | 1 + Documentation/git-refs.txt | 61 ++++++++++ Makefile | 1 + builtin.h | 1 + builtin/refs.c | 75 ++++++++++++ command-list.txt | 1 + git.c | 1 + t/t1460-refs-migrate.sh | 243 +++++++++++++++++++++++++++++++++++++ 8 files changed, 384 insertions(+) create mode 100644 Documentation/git-refs.txt create mode 100644 builtin/refs.c create mode 100755 t/t1460-refs-migrate.sh diff --git a/.gitignore b/.gitignore index 612c0f6a0f..8caf3700c2 100644 --- a/.gitignore +++ b/.gitignore @@ -126,6 +126,7 @@ /git-rebase /git-receive-pack /git-reflog +/git-refs /git-remote /git-remote-http /git-remote-https diff --git a/Documentation/git-refs.txt b/Documentation/git-refs.txt new file mode 100644 index 0000000000..5b99e04385 --- /dev/null +++ b/Documentation/git-refs.txt @@ -0,0 +1,61 @@ +git-refs(1) +=========== + +NAME +---- +git-refs - Low-level access to refs + + +SYNOPSIS +-------- +[verse] +'git refs migrate' --ref-format= [--dry-run] + +DESCRIPTION +----------- + +This command provides low-level access to refs. + +COMMANDS +-------- + +migrate:: + Migrate ref store between different formats. + +OPTIONS +------- + +The following options are specific to 'git refs migrate': + +--ref-format=:: + The ref format to migrate the ref store to. Can be one of: ++ +include::ref-storage-format.txt[] + +--dry-run:: + Perform the migration, but do not modify the repository. The migrated + refs will be written into a separate directory that can be inspected + separately. The name of the directory will be reported on stdout. This + can be used to double check that the migration works as expected before + performing the actual migration. + +KNOWN LIMITATIONS +----------------- + +The ref format migration has several known limitations in its current form: + +* It is not possible to migrate repositories that have reflogs. + +* It is not possible to migrate repositories that have worktrees. + +* There is no way to block concurrent writes to the repository during an + ongoing migration. Concurrent writes can lead to an inconsistent migrated + state. Users are expected to block writes on a higher level. If your + repository is registered for scheduled maintenance, it is recommended to + unregister it first with git-maintenance(1). + +These limitations may eventually be lifted. + +GIT +--- +Part of the linkgit:git[1] suite diff --git a/Makefile b/Makefile index cf504963c2..2d702b552c 100644 --- a/Makefile +++ b/Makefile @@ -1283,6 +1283,7 @@ BUILTIN_OBJS += builtin/read-tree.o BUILTIN_OBJS += builtin/rebase.o BUILTIN_OBJS += builtin/receive-pack.o BUILTIN_OBJS += builtin/reflog.o +BUILTIN_OBJS += builtin/refs.o BUILTIN_OBJS += builtin/remote-ext.o BUILTIN_OBJS += builtin/remote-fd.o BUILTIN_OBJS += builtin/remote.o diff --git a/builtin.h b/builtin.h index 28280636da..7eda9b2486 100644 --- a/builtin.h +++ b/builtin.h @@ -207,6 +207,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix); int cmd_rebase__interactive(int argc, const char **argv, const char *prefix); int cmd_receive_pack(int argc, const char **argv, const char *prefix); int cmd_reflog(int argc, const char **argv, const char *prefix); +int cmd_refs(int argc, const char **argv, const char *prefix); int cmd_remote(int argc, const char **argv, const char *prefix); int cmd_remote_ext(int argc, const char **argv, const char *prefix); int cmd_remote_fd(int argc, const char **argv, const char *prefix); diff --git a/builtin/refs.c b/builtin/refs.c new file mode 100644 index 0000000000..46dcd150d4 --- /dev/null +++ b/builtin/refs.c @@ -0,0 +1,75 @@ +#include "builtin.h" +#include "parse-options.h" +#include "refs.h" +#include "repository.h" +#include "strbuf.h" + +#define REFS_MIGRATE_USAGE \ + N_("git refs migrate --ref-format= [--dry-run]") + +static int cmd_refs_migrate(int argc, const char **argv, const char *prefix) +{ + const char * const migrate_usage[] = { + REFS_MIGRATE_USAGE, + NULL, + }; + const char *format_str = NULL; + enum ref_storage_format format; + unsigned int flags = 0; + struct option options[] = { + OPT_STRING_F(0, "ref-format", &format_str, N_("format"), + N_("specify the reference format to convert to"), + PARSE_OPT_NONEG), + OPT_BIT(0, "dry-run", &flags, + N_("perform a non-destructive dry-run"), + REPO_MIGRATE_REF_STORAGE_FORMAT_DRYRUN), + OPT_END(), + }; + struct strbuf errbuf = STRBUF_INIT; + int err; + + argc = parse_options(argc, argv, prefix, options, migrate_usage, 0); + if (argc) + usage(_("too many arguments")); + if (!format_str) + usage(_("missing --ref-format=")); + + format = ref_storage_format_by_name(format_str); + if (format == REF_STORAGE_FORMAT_UNKNOWN) { + err = error(_("unknown ref storage format '%s'"), format_str); + goto out; + } + + if (the_repository->ref_storage_format == format) { + err = error(_("repository already uses '%s' format"), + ref_storage_format_to_name(format)); + goto out; + } + + if (repo_migrate_ref_storage_format(the_repository, format, flags, &errbuf) < 0) { + err = error("%s", errbuf.buf); + goto out; + } + + err = 0; + +out: + strbuf_release(&errbuf); + return err; +} + +int cmd_refs(int argc, const char **argv, const char *prefix) +{ + const char * const refs_usage[] = { + REFS_MIGRATE_USAGE, + NULL, + }; + parse_opt_subcommand_fn *fn = NULL; + struct option opts[] = { + OPT_SUBCOMMAND("migrate", &fn, cmd_refs_migrate), + OPT_END(), + }; + + argc = parse_options(argc, argv, prefix, opts, refs_usage, 0); + return fn(argc, argv, prefix); +} diff --git a/command-list.txt b/command-list.txt index c4cd0f352b..e0bb87b3b5 100644 --- a/command-list.txt +++ b/command-list.txt @@ -157,6 +157,7 @@ git-read-tree plumbingmanipulators git-rebase mainporcelain history git-receive-pack synchelpers git-reflog ancillarymanipulators complete +git-refs ancillarymanipulators complete git-remote ancillarymanipulators complete git-repack ancillarymanipulators complete git-replace ancillarymanipulators complete diff --git a/git.c b/git.c index 637c61ca9c..683bb69194 100644 --- a/git.c +++ b/git.c @@ -594,6 +594,7 @@ static struct cmd_struct commands[] = { { "rebase", cmd_rebase, RUN_SETUP | NEED_WORK_TREE }, { "receive-pack", cmd_receive_pack }, { "reflog", cmd_reflog, RUN_SETUP }, + { "refs", cmd_refs, RUN_SETUP }, { "remote", cmd_remote, RUN_SETUP }, { "remote-ext", cmd_remote_ext, NO_PARSEOPT }, { "remote-fd", cmd_remote_fd, NO_PARSEOPT }, diff --git a/t/t1460-refs-migrate.sh b/t/t1460-refs-migrate.sh new file mode 100755 index 0000000000..f7c0783d30 --- /dev/null +++ b/t/t1460-refs-migrate.sh @@ -0,0 +1,243 @@ +#!/bin/sh + +test_description='migration of ref storage backends' + +GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main +export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME + +TEST_PASSES_SANITIZE_LEAK=true +. ./test-lib.sh + +test_migration () { + git -C "$1" for-each-ref --include-root-refs \ + --format='%(refname) %(objectname) %(symref)' >expect && + git -C "$1" refs migrate --ref-format="$2" && + git -C "$1" for-each-ref --include-root-refs \ + --format='%(refname) %(objectname) %(symref)' >actual && + test_cmp expect actual && + + git -C "$1" rev-parse --show-ref-format >actual && + echo "$2" >expect && + test_cmp expect actual +} + +test_expect_success 'setup' ' + rm -rf .git && + # The migration does not yet support reflogs. + git config --global core.logAllRefUpdates false +' + +test_expect_success "superfluous arguments" ' + test_when_finished "rm -rf repo" && + git init repo && + test_must_fail git -C repo refs migrate foo 2>err && + cat >expect <<-EOF && + usage: too many arguments + EOF + test_cmp expect err +' + +test_expect_success "missing ref storage format" ' + test_when_finished "rm -rf repo" && + git init repo && + test_must_fail git -C repo refs migrate 2>err && + cat >expect <<-EOF && + usage: missing --ref-format= + EOF + test_cmp expect err +' + +test_expect_success "unknown ref storage format" ' + test_when_finished "rm -rf repo" && + git init repo && + test_must_fail git -C repo refs migrate \ + --ref-format=unknown 2>err && + cat >expect <<-EOF && + error: unknown ref storage format ${SQ}unknown${SQ} + EOF + test_cmp expect err +' + +ref_formats="files reftable" +for from_format in $ref_formats +do + for to_format in $ref_formats + do + if test "$from_format" = "$to_format" + then + continue + fi + + test_expect_success "$from_format: migration to same format fails" ' + test_when_finished "rm -rf repo" && + git init --ref-format=$from_format repo && + test_must_fail git -C repo refs migrate \ + --ref-format=$from_format 2>err && + cat >expect <<-EOF && + error: repository already uses ${SQ}$from_format${SQ} format + EOF + test_cmp expect err + ' + + test_expect_success "$from_format -> $to_format: migration with reflog fails" ' + test_when_finished "rm -rf repo" && + git init --ref-format=$from_format repo && + test_config -C repo core.logAllRefUpdates true && + test_commit -C repo logged && + test_must_fail git -C repo refs migrate \ + --ref-format=$to_format 2>err && + cat >expect <<-EOF && + error: migrating reflogs is not supported yet + EOF + test_cmp expect err + ' + + test_expect_success "$from_format -> $to_format: migration with worktree fails" ' + test_when_finished "rm -rf repo" && + git init --ref-format=$from_format repo && + git -C repo worktree add wt && + test_must_fail git -C repo refs migrate \ + --ref-format=$to_format 2>err && + cat >expect <<-EOF && + error: migrating repositories with worktrees is not supported yet + EOF + test_cmp expect err + ' + + test_expect_success "$from_format -> $to_format: unborn HEAD" ' + test_when_finished "rm -rf repo" && + git init --ref-format=$from_format repo && + test_migration repo "$to_format" + ' + + test_expect_success "$from_format -> $to_format: single ref" ' + test_when_finished "rm -rf repo" && + git init --ref-format=$from_format repo && + test_commit -C repo initial && + test_migration repo "$to_format" + ' + + test_expect_success "$from_format -> $to_format: bare repository" ' + test_when_finished "rm -rf repo repo.git" && + git init --ref-format=$from_format repo && + test_commit -C repo initial && + git clone --ref-format=$from_format --mirror repo repo.git && + test_migration repo.git "$to_format" + ' + + test_expect_success "$from_format -> $to_format: dangling symref" ' + test_when_finished "rm -rf repo" && + git init --ref-format=$from_format repo && + test_commit -C repo initial && + git -C repo symbolic-ref BROKEN_HEAD refs/heads/nonexistent && + test_migration repo "$to_format" && + echo refs/heads/nonexistent >expect && + git -C repo symbolic-ref BROKEN_HEAD >actual && + test_cmp expect actual + ' + + test_expect_success "$from_format -> $to_format: broken ref" ' + test_when_finished "rm -rf repo" && + git init --ref-format=$from_format repo && + test_commit -C repo initial && + test-tool -C repo ref-store main update-ref "" refs/heads/broken \ + "$(test_oid 001)" "$ZERO_OID" REF_SKIP_CREATE_REFLOG,REF_SKIP_OID_VERIFICATION && + test_migration repo "$to_format" && + test_oid 001 >expect && + git -C repo rev-parse refs/heads/broken >actual && + test_cmp expect actual + ' + + test_expect_success "$from_format -> $to_format: pseudo-refs" ' + test_when_finished "rm -rf repo" && + git init --ref-format=$from_format repo && + test_commit -C repo initial && + git -C repo update-ref FOO_HEAD HEAD && + test_migration repo "$to_format" + ' + + test_expect_success "$from_format -> $to_format: special refs are left alone" ' + test_when_finished "rm -rf repo" && + git init --ref-format=$from_format repo && + test_commit -C repo initial && + git -C repo rev-parse HEAD >repo/.git/MERGE_HEAD && + git -C repo rev-parse MERGE_HEAD && + test_migration repo "$to_format" && + test_path_is_file repo/.git/MERGE_HEAD + ' + + test_expect_success "$from_format -> $to_format: a bunch of refs" ' + test_when_finished "rm -rf repo" && + git init --ref-format=$from_format repo && + + test_commit -C repo initial && + cat >input <<-EOF && + create FOO_HEAD HEAD + create refs/heads/branch-1 HEAD + create refs/heads/branch-2 HEAD + create refs/heads/branch-3 HEAD + create refs/heads/branch-4 HEAD + create refs/tags/tag-1 HEAD + create refs/tags/tag-2 HEAD + EOF + git -C repo update-ref --stdin $to_format: dry-run migration does not modify repository" ' + test_when_finished "rm -rf repo" && + git init --ref-format=$from_format repo && + test_commit -C repo initial && + git -C repo refs migrate --dry-run \ + --ref-format=$to_format >output && + grep "Finished dry-run migration of refs" output && + test_path_is_dir repo/.git/ref_migration.* && + echo $from_format >expect && + git -C repo rev-parse --show-ref-format >actual && + test_cmp expect actual + ' + done +done + +test_expect_success 'migrating from files format deletes backend files' ' + test_when_finished "rm -rf repo" && + git init --ref-format=files repo && + test_commit -C repo first && + git -C repo pack-refs --all && + test_commit -C repo second && + git -C repo update-ref ORIG_HEAD HEAD && + git -C repo rev-parse HEAD >repo/.git/FETCH_HEAD && + + test_path_is_file repo/.git/HEAD && + test_path_is_file repo/.git/ORIG_HEAD && + test_path_is_file repo/.git/refs/heads/main && + test_path_is_file repo/.git/packed-refs && + + test_migration repo reftable && + + echo "ref: refs/heads/.invalid" >expect && + test_cmp expect repo/.git/HEAD && + echo "this repository uses the reftable format" >expect && + test_cmp expect repo/.git/refs/heads && + test_path_is_file repo/.git/FETCH_HEAD && + test_path_is_missing repo/.git/ORIG_HEAD && + test_path_is_missing repo/.git/refs/heads/main && + test_path_is_missing repo/.git/logs && + test_path_is_missing repo/.git/packed-refs +' + +test_expect_success 'migrating from reftable format deletes backend files' ' + test_when_finished "rm -rf repo" && + git init --ref-format=reftable repo && + test_commit -C repo first && + + test_path_is_dir repo/.git/reftable && + test_migration repo files && + + test_path_is_missing repo/.git/reftable && + echo "ref: refs/heads/main" >expect && + test_cmp expect repo/.git/HEAD && + test_path_is_file repo/.git/refs/heads/main +' + +test_done