From patchwork Wed Dec 6 12:39:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13481737 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="HPeS6G19"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="G+6FJ5Ga" Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 209FC181 for ; Wed, 6 Dec 2023 04:39:52 -0800 (PST) Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id 2F3365C0296 for ; Wed, 6 Dec 2023 07:39:51 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Wed, 06 Dec 2023 07:39:51 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to; s=fm1; t=1701866391; x=1701952791; bh=mD WPZ/dPwJZR775DsjlLpFpvUIiKFd5MWq2ju+gi9bY=; b=HPeS6G193lQ5muDar0 jWedj5hgvGaBb5xmhaAvi5F3jLGSwIVyZ2autyNNHdHGroQdFvONGDJUap+Eh50L NoxQnMb27VsNM8rfHLNSNTW6yKYEIeKwD+a+qbEXKsNlgAmT0oW2Nc5RRX8yAcAM LiKlrlH0pahZwN7IYlW2uVfJS5vuuBMOJ1TPOwy0D8uXCu7m3pyUGC6NrlUQxXGt XDGFmXib7eCDyHymKf8UbPutcyNjn8Z9nET0HaRosI0Lu57qImG+j1UDQ9mC+sGw YpaifYypzr5Fc04JVqUWxgLwfy6cLgxTBBl3MmL7yhnGctk6O3LczUBaOy3ZOGjf h+ZA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; t=1701866391; x=1701952791; bh=mDWPZ/dPwJZR7 75DsjlLpFpvUIiKFd5MWq2ju+gi9bY=; b=G+6FJ5GaGK65mlISOtsUZyWiK5fPj Fi69427q+sDR/db0BN1YXrUUMHw4dM+Pl+MWzd+bLkvGTnc+IcApxecoQ9U+lexS /PzjVKQVZZ5NXrP334/4cYxys3Lx/6DHsgaiaJ32w6KCIjvCD+xBa8suXkDwHdEN omQArWKbKDRbrpme8IoYcrMjQ/0nP4NoeHys9t/mFCwNvu+NI5hz2S2SgIctUgDy nfO4FYxAz+jq1OpJ0qZCg6eskdPB0wKjhvHUcKN45ucP9BPYNqsPrAKfqaLQg+Uk uOeCrj6q2R0RC/vSHiHWb0K/y2gMNDzvgWrEnvG3aEWnHqkumAbLgT6rQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrudektddggeegucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpeffhffvuffkfhggtggujgesghdtre ertddtvdenucfhrhhomheprfgrthhrihgtkhcuufhtvghinhhhrghrughtuceophhssehp khhsrdhimheqnecuggftrfgrthhtvghrnhepheeghfdtfeeuffehkefgffduleffjedthf dvjeektdfhhedvlefgtefgvdettdfhnecuvehluhhsthgvrhfuihiivgeptdenucfrrghr rghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Wed, 6 Dec 2023 07:39:50 -0500 (EST) Received: by vm-mail (OpenSMTPD) with ESMTPSA id 9cbcade1 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO) for ; Wed, 6 Dec 2023 12:38:24 +0000 (UTC) Date: Wed, 6 Dec 2023 13:39:48 +0100 From: Patrick Steinhardt To: git@vger.kernel.org Subject: [PATCH 1/7] setup: extract function to create the refdb Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: We're about to let callers skip creation of the reference database when calling `init_db()`. Extract the logic into a standalone function so that it becomes easier to do this refactoring. Signed-off-by: Patrick Steinhardt --- setup.c | 95 ++++++++++++++++++++++++++++++++++----------------------- 1 file changed, 57 insertions(+), 38 deletions(-) diff --git a/setup.c b/setup.c index fc592dc6dd..9fcb64159f 100644 --- a/setup.c +++ b/setup.c @@ -1885,6 +1885,60 @@ void initialize_repository_version(int hash_algo, int reinit) git_config_set_gently("extensions.objectformat", NULL); } +static int is_reinit(void) +{ + struct strbuf buf = STRBUF_INIT; + char junk[2]; + int ret; + + git_path_buf(&buf, "HEAD"); + ret = !access(buf.buf, R_OK) || readlink(buf.buf, junk, sizeof(junk) - 1) != -1; + strbuf_release(&buf); + return ret; +} + +static void create_reference_database(const char *initial_branch, int quiet) +{ + struct strbuf err = STRBUF_INIT; + int reinit = is_reinit(); + + /* + * We need to create a "refs" dir in any case so that older + * versions of git can tell that this is a repository. + */ + safe_create_dir(git_path("refs"), 1); + adjust_shared_perm(git_path("refs")); + + if (refs_init_db(&err)) + die("failed to set up refs db: %s", err.buf); + + /* + * Point the HEAD symref to the initial branch with if HEAD does + * not yet exist. + */ + if (!reinit) { + char *ref; + + if (!initial_branch) + initial_branch = git_default_branch_name(quiet); + + ref = xstrfmt("refs/heads/%s", initial_branch); + if (check_refname_format(ref, 0) < 0) + die(_("invalid initial branch name: '%s'"), + initial_branch); + + if (create_symref("HEAD", ref, NULL) < 0) + exit(1); + free(ref); + } + + if (reinit && initial_branch) + warning(_("re-init: ignored --initial-branch=%s"), + initial_branch); + + strbuf_release(&err); +} + static int create_default_files(const char *template_path, const char *original_git_dir, const char *initial_branch, @@ -1896,10 +1950,8 @@ static int create_default_files(const char *template_path, struct stat st1; struct strbuf buf = STRBUF_INIT; char *path; - char junk[2]; int reinit; int filemode; - struct strbuf err = STRBUF_INIT; const char *init_template_dir = NULL; const char *work_tree = get_git_work_tree(); @@ -1919,6 +1971,8 @@ static int create_default_files(const char *template_path, reset_shared_repository(); git_config(git_default_config, NULL); + reinit = is_reinit(); + /* * We must make sure command-line options continue to override any * values we might have just re-read from the config. @@ -1962,39 +2016,7 @@ static int create_default_files(const char *template_path, adjust_shared_perm(get_git_dir()); } - /* - * We need to create a "refs" dir in any case so that older - * versions of git can tell that this is a repository. - */ - safe_create_dir(git_path("refs"), 1); - adjust_shared_perm(git_path("refs")); - - if (refs_init_db(&err)) - die("failed to set up refs db: %s", err.buf); - - /* - * Point the HEAD symref to the initial branch with if HEAD does - * not yet exist. - */ - path = git_path_buf(&buf, "HEAD"); - reinit = (!access(path, R_OK) - || readlink(path, junk, sizeof(junk)-1) != -1); - if (!reinit) { - char *ref; - - if (!initial_branch) - initial_branch = git_default_branch_name(quiet); - - ref = xstrfmt("refs/heads/%s", initial_branch); - if (check_refname_format(ref, 0) < 0) - die(_("invalid initial branch name: '%s'"), - initial_branch); - - if (create_symref("HEAD", ref, NULL) < 0) - exit(1); - free(ref); - } - + create_reference_database(initial_branch, quiet); initialize_repository_version(fmt->hash_algo, 0); /* Check filemode trustability */ @@ -2158,9 +2180,6 @@ int init_db(const char *git_dir, const char *real_git_dir, prev_bare_repository, init_shared_repository, flags & INIT_DB_QUIET); - if (reinit && initial_branch) - warning(_("re-init: ignored --initial-branch=%s"), - initial_branch); create_object_directory(); From patchwork Wed Dec 6 12:39:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13481738 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="qggpKXq8"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="XRAFqlAo" Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A2E0011F for ; Wed, 6 Dec 2023 04:39:56 -0800 (PST) Received: from compute6.internal (compute6.nyi.internal [10.202.2.47]) by mailout.nyi.internal (Postfix) with ESMTP id 1A7625C0296 for ; Wed, 6 Dec 2023 07:39:56 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute6.internal (MEProxy); Wed, 06 Dec 2023 07:39:56 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to; s=fm1; t=1701866396; x=1701952796; bh=CM Kql7Y6w2gk0ZgTan/FbtV8+K23/QmNr1OxXreKeuE=; b=qggpKXq8B841nnedva oKUhwjZtWMR02vLJSn0B2S1nRU9lPecur36QRgojhxgMBcCCc3aYdRDmuyRZGVYQ S5DLYuz0l5vKS9blPVjKNBl4Sd255S8SJY4Q067vrHnWnRCo45P8yY2X0pMqvMu8 eFiMfSKCN9ABQ1RkLZXY5h2Pf9kUIZJO23GyU+umFAM+pF/rstzCZvjszyYzJYYL Z+SWDk3AZEREiIOV7toVQllE8N8O8uG5k5kGiFUoeWx5zBlzF01RPMZIa/dwOHS8 MvYfRoVg+UZLJl1/uM1MF0YA4ze9qfO31BlnGnl4m066BJpKnZO3+e7292c4eG3v A9DQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; t=1701866396; x=1701952796; bh=CMKql7Y6w2gk0 ZgTan/FbtV8+K23/QmNr1OxXreKeuE=; b=XRAFqlAoMnKDuRmN05F1DlupKySmn AbgdkioyVmAJzuR6VVmnL3t4ftdQ7lY+tAF+Bs+PdsXBi4p037kE5ba7yawstLeG 2bywoF4WJ/+SW8+kGtZiE7JoDNppjr1zb4FKMCNqeKDPtTcPZIbCsob7XUAHrmPb ClY3yIE2o7FQmKnm/wHAAhfdvKrNzRodvFerrZyYqTR1dCyfl6ki/BiW1oow8k1g GILPwUy0vb+TwWnkcVoabS7s9mi/nRmuyOZeBLWNas42oWylcRsu2IjaP6KCt+VF 4FNgZoilg/1ziDmk8wOXPn4XfE2M5eOVaAPuC1pU/FzLxITn1NSriYVYQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrudektddggeegucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpeffhffvuffkfhggtggujgesghdtre ertddtvdenucfhrhhomheprfgrthhrihgtkhcuufhtvghinhhhrghrughtuceophhssehp khhsrdhimheqnecuggftrfgrthhtvghrnhepheeghfdtfeeuffehkefgffduleffjedthf dvjeektdfhhedvlefgtefgvdettdfhnecuvehluhhsthgvrhfuihiivgeptdenucfrrghr rghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Wed, 6 Dec 2023 07:39:55 -0500 (EST) Received: by vm-mail (OpenSMTPD) with ESMTPSA id 6b8b4ed6 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO) for ; Wed, 6 Dec 2023 12:38:29 +0000 (UTC) Date: Wed, 6 Dec 2023 13:39:53 +0100 From: Patrick Steinhardt To: git@vger.kernel.org Subject: [PATCH 2/7] setup: allow skipping creation of the refdb Message-ID: <090c52423e3de9c4e94c6694bb18fb8510a3ad08.1701863960.git.ps@pks.im> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Allow callers to skip creation of the reference database via a new flag `INIT_DB_SKIP_REFDB`, which is required for git-clone(1) so that we can create it at a later point once the object format has been discovered from the remote repository. Note that we also uplift the call to `create_reference_database()` into `init_db()`, which makes it easier to handle the new flag for us. This changes the order in which we do initialization so that we now set up the Git configuration before we create the reference database. In practice this move should not result in any change in behaviour. Signed-off-by: Patrick Steinhardt --- setup.c | 13 +++++-------- setup.h | 5 +++-- 2 files changed, 8 insertions(+), 10 deletions(-) diff --git a/setup.c b/setup.c index 9fcb64159f..a80fc09b9c 100644 --- a/setup.c +++ b/setup.c @@ -1941,11 +1941,9 @@ static void create_reference_database(const char *initial_branch, int quiet) static int create_default_files(const char *template_path, const char *original_git_dir, - const char *initial_branch, const struct repository_format *fmt, int prev_bare_repository, - int init_shared_repository, - int quiet) + int init_shared_repository) { struct stat st1; struct strbuf buf = STRBUF_INIT; @@ -2016,7 +2014,6 @@ static int create_default_files(const char *template_path, adjust_shared_perm(get_git_dir()); } - create_reference_database(initial_branch, quiet); initialize_repository_version(fmt->hash_algo, 0); /* Check filemode trustability */ @@ -2176,11 +2173,11 @@ int init_db(const char *git_dir, const char *real_git_dir, validate_hash_algorithm(&repo_fmt, hash); reinit = create_default_files(template_dir, original_git_dir, - initial_branch, &repo_fmt, - prev_bare_repository, - init_shared_repository, - flags & INIT_DB_QUIET); + &repo_fmt, prev_bare_repository, + init_shared_repository); + if (!(flags & INIT_DB_SKIP_REFDB)) + create_reference_database(initial_branch, flags & INIT_DB_QUIET); create_object_directory(); if (get_shared_repository()) { diff --git a/setup.h b/setup.h index b48cf1c43b..cbf538286b 100644 --- a/setup.h +++ b/setup.h @@ -169,8 +169,9 @@ int verify_repository_format(const struct repository_format *format, */ void check_repository_format(struct repository_format *fmt); -#define INIT_DB_QUIET 0x0001 -#define INIT_DB_EXIST_OK 0x0002 +#define INIT_DB_QUIET (1 << 0) +#define INIT_DB_EXIST_OK (1 << 1) +#define INIT_DB_SKIP_REFDB (1 << 2) int init_db(const char *git_dir, const char *real_git_dir, const char *template_dir, int hash_algo, From patchwork Wed Dec 6 12:39:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13481739 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="hNt2LUmZ"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="nSSuuZbQ" Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 70E1211F for ; Wed, 6 Dec 2023 04:40:00 -0800 (PST) Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailout.nyi.internal (Postfix) with ESMTP id DD08B5C0288 for ; Wed, 6 Dec 2023 07:39:59 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute2.internal (MEProxy); Wed, 06 Dec 2023 07:39:59 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to; s=fm1; t=1701866399; x=1701952799; bh=01 GGAJOWd3Swn+hAXwTOpobA1AKOmJMucNCSXBa1SjE=; b=hNt2LUmZfKwkT47Jii LTCuN2G819isVokYgoWmLACP6a7KUJ7RiweOAC17wmKco1cTSiZbcqNoAWpTAyU+ EMHCDbsl9N4s09Hn9sq3yN8kvoHa2a9vnFhiFa2Mry2A7ERvQVV5BPZa/rOoAvls W3yFkohTKyE89bvMScC6taScmlBz8XSI+zRI6U26ZXd+Cts0GNosI1tRYcRrQ0kE x0i02PDr4XFg0LVzuvcZ+6DzyuOAdPpPDKo8VMuBt7IVN/Hf2A27OLMXBBdICays OmsS4O4PzKC7Em/0RwWOaO0ZpTNYH1G3rWW6d9UTAMvCcIfY8W25cXp5uLPY213V 08sA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; t=1701866399; x=1701952799; bh=01GGAJOWd3Swn +hAXwTOpobA1AKOmJMucNCSXBa1SjE=; b=nSSuuZbQoKf0EGfiJODxoc2erRNen rm6JAtfSX2dKsVywbH7uIAtCMR35IjY8EBkS2swKwhX0e9iTLi9VxiisnchMbWpO YrqMnI4AB8VAV5DpGJd0zfmxFUrQtpXtJHwILIUAqNBpO8Z+Y2fc8O5Qpcnz/IrH NraQgWpou9/GAQwz7ac4SWEtOUNYpm1z2UCye+2rs7/v5tiPizECJRwEO+j373v7 Zk3QTnXtuH2hbe5dPAJHrWGw5HDd5ED8jQSIaJbMVhWJdunFpA1kNkEW77u/iWpp x/iyx9HMrAoXwmjT1Xn04j8K21WA1PuwOHgft5psZiHhu5bRorcq1KBmw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrudektddggeegucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpeffhffvuffkfhggtggujgesghdtre ertddtvdenucfhrhhomheprfgrthhrihgtkhcuufhtvghinhhhrghrughtuceophhssehp khhsrdhimheqnecuggftrfgrthhtvghrnhepheeghfdtfeeuffehkefgffduleffjedthf dvjeektdfhhedvlefgtefgvdettdfhnecuvehluhhsthgvrhfuihiivgeptdenucfrrghr rghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Wed, 6 Dec 2023 07:39:59 -0500 (EST) Received: by vm-mail (OpenSMTPD) with ESMTPSA id b377bc1a (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO) for ; Wed, 6 Dec 2023 12:38:33 +0000 (UTC) Date: Wed, 6 Dec 2023 13:39:57 +0100 From: Patrick Steinhardt To: git@vger.kernel.org Subject: [PATCH 3/7] remote-curl: rediscover repository when fetching refs Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: We're about to change git-clone(1) so that we set up the reference database at a later point. This change will cause git-remote-curl(1) to not detect the repository anymore due to "HEAD" not having been created yet at the time it spawns, and thus causes it to error out once it is asked to fetch the references. We can address this issue by trying to re-discover the Git repository in case none was detected at startup time. With this change, the clone will look as following: 1. git-clone(1) sets up the initial repository, excluding the reference database. 2. git-clone(1) spawns git-remote-curl(1), which will be unable to detect the repository due to a missing "HEAD". 3. git-clone(1) asks git-remote-curl(1) to list remote references. This works just fine as this step does not require a local repository 4. git-clone(1) creates the reference database as it has now learned about the object format. 5. git-clone(1) asks git-remote-curl(1) to fetch the remote packfile. The latter notices that it doesn't have a repository available, but it now knows to try and re-discover it. If the re-discovery succeeds in the last step we can continue with the clone. Signed-off-by: Patrick Steinhardt --- remote-curl.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/remote-curl.c b/remote-curl.c index ef05752ca5..fc29757b65 100644 --- a/remote-curl.c +++ b/remote-curl.c @@ -1564,8 +1564,11 @@ int cmd_main(int argc, const char **argv) if (buf.len == 0) break; if (starts_with(buf.buf, "fetch ")) { - if (nongit) - die(_("remote-curl: fetch attempted without a local repo")); + if (nongit) { + setup_git_directory_gently(&nongit); + if (nongit) + die(_("remote-curl: fetch attempted without a local repo")); + } parse_fetch(&buf); } else if (!strcmp(buf.buf, "list") || starts_with(buf.buf, "list ")) { From patchwork Wed Dec 6 12:40:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13481740 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="Fx+iUj3G"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="YlVshOs0" Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 91A8DD3 for ; Wed, 6 Dec 2023 04:40:04 -0800 (PST) Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.nyi.internal (Postfix) with ESMTP id 071BA5C0297 for ; Wed, 6 Dec 2023 07:40:04 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute3.internal (MEProxy); Wed, 06 Dec 2023 07:40:04 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to; s=fm1; t=1701866404; x=1701952804; bh=1L c5HAo25ugq7oeSNUCiN+GW+jVQgSNYcNqCD/LmB60=; b=Fx+iUj3GasCgfPYrAd b3ry7D7tJ2IcrocR9wKqQ625PzCc+A35L6P4FfgLOyVQ04zq6Tr/1JVLYe2uifr/ tS459aHCzOguHoJeoi0sZ4Jo53WttOBT1EGhl0jkPQvlZP/BKgQQkb5C5bK5Y2yV dORRap31KydMWo59OAM/+y+AGWsgFeRjSHweIfbw/kcaT7c5oT/MKf0UXiXyq9/m 9kBQlMl61dG25u6PKtPpnedvo4Io124hpLWGxMylN4YrUPvdB4qLUiWizY04ipEn +i0HP8UlAcGf56Ar3ZKYNbfeUaIKJ1DPs3n9WjCy5GOTL0y8RA149e//4ERf0JFM 8k0Q== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; t=1701866404; x=1701952804; bh=1Lc5HAo25ugq7 oeSNUCiN+GW+jVQgSNYcNqCD/LmB60=; b=YlVshOs0iZziEauztqGyqLCfJoST3 JDMAzikXk9L4bFcLLNBT5pJIx6hvOO3R95jhddgzSMGWre1RrB1OpvZR2l1J6LdP o/JFzKNpjsUBFzTcYgLjqUp+SSYi5g8Mtgpqgesg61oL/nRAkHko8pXoFYH16DQZ H9VjVCR9nzhVjSpamMPDHW+g7HNZl9LJRBCFVw/8nJLEb49Cc/YYNPEUHAuqj8s8 Bgg2gr3bDbg+WRIJisQ6hNgnVHqWAwr36IGVYVKET+DzQ6D5eW5Ug3B88v7Bc/1u R4+/YOYX7himZWF+nd1AKWXi/duoM7BvGOOV9jANcjxLaloBC9Ga9+94g== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrudektddggeegucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpeffhffvuffkfhggtggujgesghdtre ertddtvdenucfhrhhomheprfgrthhrihgtkhcuufhtvghinhhhrghrughtuceophhssehp khhsrdhimheqnecuggftrfgrthhtvghrnhepheeghfdtfeeuffehkefgffduleffjedthf dvjeektdfhhedvlefgtefgvdettdfhnecuvehluhhsthgvrhfuihiivgeptdenucfrrghr rghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Wed, 6 Dec 2023 07:40:03 -0500 (EST) Received: by vm-mail (OpenSMTPD) with ESMTPSA id 6701feb7 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO) for ; Wed, 6 Dec 2023 12:38:37 +0000 (UTC) Date: Wed, 6 Dec 2023 13:40:01 +0100 From: Patrick Steinhardt To: git@vger.kernel.org Subject: [PATCH 4/7] builtin/clone: fix bundle URIs with mismatching object formats Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: We create the reference database in git-clone(1) quite early before connecting to the remote repository. Given that we do not yet know about the object format that the remote repository uses at that point in time the consequence is that the refdb may be initialized with the wrong object format. This is not a problem in the context of the files backend as we do not encode the object format anywhere, and furthermore the only reference that we write between initializing the refdb and learning about the object format is the "HEAD" symref. It will become a problem though once we land the reftable backend, which indeed does require to know about the proper object format at the time of creation. We thus need to rearrange the logic in git-clone(1) so that we only initialize the refdb once we have learned about the actual object format. As a first step, move listing of remote references to happen earlier, which also allow us to set up the hash algorithm of the repository earlier now. While we aim to execute this logic as late as possible until after most of the setup has happened already, detection of the object format and thus later the setup of the reference database must happen before any other logic that may spawn Git commands or otherwise these Git commands may not recognize the repository as such. The first Git step where we expect the repository to be fully initalized is when we fetch bundles via bundle URIs. Funny enough, the comments there also state that "the_repository must match the cloned repo", which is indeed not necessarily the case for the hash algorithm right now. So in practice it is the right thing to detect the remote's object format before downloading bundle URIs anyway, and not doing so causes clones with bundle URIS to fail when the local default object format does not match the remote repository's format. Unfortunately though, this creates a new issue: downloading bundles may take a long time, so if we list refs beforehand they might've grown stale meanwhile. It is not clear how to solve this issue except for a second reference listing though after we have downloaded the bundles, which may be an expensive thing to do. Arguably though, it's preferable to have a staleness issue compared to being unable to clone a repository altogether. Signed-off-by: Patrick Steinhardt --- builtin/clone.c | 48 ++++++++++++++++++------------------- t/t5558-clone-bundle-uri.sh | 18 ++++++++++++++ 2 files changed, 41 insertions(+), 25 deletions(-) diff --git a/builtin/clone.c b/builtin/clone.c index c6357af949..d188650881 100644 --- a/builtin/clone.c +++ b/builtin/clone.c @@ -1266,6 +1266,26 @@ int cmd_clone(int argc, const char **argv, const char *prefix) if (transport->smart_options && !deepen && !filter_options.choice) transport->smart_options->check_self_contained_and_connected = 1; + strvec_push(&transport_ls_refs_options.ref_prefixes, "HEAD"); + refspec_ref_prefixes(&remote->fetch, + &transport_ls_refs_options.ref_prefixes); + if (option_branch) + expand_ref_prefix(&transport_ls_refs_options.ref_prefixes, + option_branch); + if (!option_no_tags) + strvec_push(&transport_ls_refs_options.ref_prefixes, + "refs/tags/"); + + refs = transport_get_remote_refs(transport, &transport_ls_refs_options); + + /* + * Now that we know what algorithm the remote side is using, let's set + * ours to the same thing. + */ + hash_algo = hash_algo_by_ptr(transport_get_hash_algo(transport)); + initialize_repository_version(hash_algo, 1); + repo_set_hash_algo(the_repository, hash_algo); + /* * Before fetching from the remote, download and install bundle * data from the --bundle-uri option. @@ -1281,24 +1301,7 @@ int cmd_clone(int argc, const char **argv, const char *prefix) bundle_uri); else if (has_heuristic) git_config_set_gently("fetch.bundleuri", bundle_uri); - } - - strvec_push(&transport_ls_refs_options.ref_prefixes, "HEAD"); - refspec_ref_prefixes(&remote->fetch, - &transport_ls_refs_options.ref_prefixes); - if (option_branch) - expand_ref_prefix(&transport_ls_refs_options.ref_prefixes, - option_branch); - if (!option_no_tags) - strvec_push(&transport_ls_refs_options.ref_prefixes, - "refs/tags/"); - - refs = transport_get_remote_refs(transport, &transport_ls_refs_options); - - if (refs) - mapped_refs = wanted_peer_refs(refs, &remote->fetch); - - if (!bundle_uri) { + } else { /* * Populate transport->got_remote_bundle_uri and * transport->bundle_uri. We might get nothing. @@ -1319,13 +1322,8 @@ int cmd_clone(int argc, const char **argv, const char *prefix) } } - /* - * Now that we know what algorithm the remote side is using, - * let's set ours to the same thing. - */ - hash_algo = hash_algo_by_ptr(transport_get_hash_algo(transport)); - initialize_repository_version(hash_algo, 1); - repo_set_hash_algo(the_repository, hash_algo); + if (refs) + mapped_refs = wanted_peer_refs(refs, &remote->fetch); if (mapped_refs) { /* diff --git a/t/t5558-clone-bundle-uri.sh b/t/t5558-clone-bundle-uri.sh index 996a08e90c..1ca5f745e7 100755 --- a/t/t5558-clone-bundle-uri.sh +++ b/t/t5558-clone-bundle-uri.sh @@ -33,6 +33,15 @@ test_expect_success 'clone with path bundle' ' test_cmp expect actual ' +test_expect_success 'clone with path bundle and non-default hash' ' + test_when_finished "rm -rf clone-path-non-default-hash" && + GIT_DEFAULT_HASH=sha256 git clone --bundle-uri="clone-from/B.bundle" \ + clone-from clone-path-non-default-hash && + git -C clone-path-non-default-hash rev-parse refs/bundles/topic >actual && + git -C clone-from rev-parse topic >expect && + test_cmp expect actual +' + test_expect_success 'clone with file:// bundle' ' git clone --bundle-uri="file://$(pwd)/clone-from/B.bundle" \ clone-from clone-file && @@ -284,6 +293,15 @@ test_expect_success 'clone HTTP bundle' ' test_config -C clone-http log.excludedecoration refs/bundle/ ' +test_expect_success 'clone HTTP bundle with non-default hash' ' + test_when_finished "rm -rf clone-http-non-default-hash" && + GIT_DEFAULT_HASH=sha256 git clone --bundle-uri="$HTTPD_URL/B.bundle" \ + "$HTTPD_URL/smart/fetch.git" clone-http-non-default-hash && + git -C clone-http-non-default-hash rev-parse refs/bundles/topic >actual && + git -C clone-from rev-parse topic >expect && + test_cmp expect actual +' + test_expect_success 'clone bundle list (HTTP, no heuristic)' ' test_when_finished rm -f trace*.txt && From patchwork Wed Dec 6 12:40:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13481741 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="s+2HgnaM"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="bJLLs2he" Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A4C8111F for ; Wed, 6 Dec 2023 04:40:09 -0800 (PST) Received: from compute7.internal (compute7.nyi.internal [10.202.2.48]) by mailout.nyi.internal (Postfix) with ESMTP id EDAF65C0297 for ; Wed, 6 Dec 2023 07:40:08 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute7.internal (MEProxy); Wed, 06 Dec 2023 07:40:08 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to; s=fm1; t=1701866408; x=1701952808; bh=fe +aRzJVFhYmEZgmnb1uwhmMIYlwS6wyQrnp+ZPJ61k=; b=s+2HgnaMkU84AY1fjF 9DP2FSOByqBXVsIzv5Bi/Nubaqe2Z1M6DHkw6LHCLYuGJgmE5UV8I3I+pw0dDTQB VhzYamWpdtzqn5bQ1NlH0eTqaaBO3khdDeOoyCo8nx3tjG33htjtQivFN6o2eo4A F/UNTYq/6LU5ErtHHgZ4xY2r1ONaqS6IyycJOlRDU67l611lvGGgGar3NPVoyvBA XPfNM4YXuww7k4f0rmI4936y60oy7flCTM44vCBHvVaNpzyTZB8J/Fu49VgeBrQo YiodebAY6/H5s/srdSfwew/JohhQo022WZhr3e7vpSRRZI1IrZuJl3YyUNqfkxf4 sCKg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; t=1701866408; x=1701952808; bh=fe+aRzJVFhYmE Zgmnb1uwhmMIYlwS6wyQrnp+ZPJ61k=; b=bJLLs2helPwIfCz1IfQnjMRRLGzs6 +87Do1Bw1Eaw6lP29sRXOXOOKnWGC0FUXXsaShVfOBHEPvi1BWTHURmt/RMVTr98 WW0eAz/Ciql3fi0jG3MxzHcjnGIJEx25VW8H1yEIEIQauepj9B9XMReotTruzl1O 9kCkhJt+q/FR2pG3r/vAQGe0QC0nVZtgFj9xn/2XvQaVTs+ZPnr2mDi6adjYtvB7 J5a2Wr0Qj/fSpCq5HMyfwr41pmjBUFlZhyCLQtyjrzxlAGYqGQOVvoLTux+sVoNz 14hgBkhVwtIgvbfiKn6VvRhJAV4qccn1lvuoJUICduidFn4Xo0JMvs/Qw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrudektddggeegucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpeffhffvuffkfhggtggujgesghdtre ertddtvdenucfhrhhomheprfgrthhrihgtkhcuufhtvghinhhhrghrughtuceophhssehp khhsrdhimheqnecuggftrfgrthhtvghrnhepheeghfdtfeeuffehkefgffduleffjedthf dvjeektdfhhedvlefgtefgvdettdfhnecuvehluhhsthgvrhfuihiivgeptdenucfrrghr rghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Wed, 6 Dec 2023 07:40:08 -0500 (EST) Received: by vm-mail (OpenSMTPD) with ESMTPSA id c06127c9 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO) for ; Wed, 6 Dec 2023 12:38:41 +0000 (UTC) Date: Wed, 6 Dec 2023 13:40:06 +0100 From: Patrick Steinhardt To: git@vger.kernel.org Subject: [PATCH 5/7] builtin/clone: set up sparse checkout later Message-ID: <703ff77eea6daed0d44908e6a8a20aaa64af99d6.1701863960.git.ps@pks.im> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: When asked to do a sparse checkout, then git-clone(1) will spawn `git sparse-checkout set` to set up the configuration accordingly. This requires a proper Git repository or otherwise the command will fail. But as we are about to move creation of the reference database to a later point, this prerequisite will not hold anymore. Move the logic to a later point in time where we know to have created the reference database already. Signed-off-by: Patrick Steinhardt --- builtin/clone.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/builtin/clone.c b/builtin/clone.c index d188650881..9c60923f31 100644 --- a/builtin/clone.c +++ b/builtin/clone.c @@ -1185,9 +1185,6 @@ int cmd_clone(int argc, const char **argv, const char *prefix) if (option_required_reference.nr || option_optional_reference.nr) setup_reference(); - if (option_sparse_checkout && git_sparse_checkout_init(dir)) - return 1; - remote = remote_get(remote_name); refspec_appendf(&remote->fetch, "+%s*:%s*", src_ref_prefix, @@ -1426,6 +1423,9 @@ int cmd_clone(int argc, const char **argv, const char *prefix) dissociate_from_references(); } + if (option_sparse_checkout && git_sparse_checkout_init(dir)) + return 1; + junk_mode = JUNK_LEAVE_REPO; err = checkout(submodule_progress, filter_submodules); From patchwork Wed Dec 6 12:40:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13481742 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="s4dtbI+d"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="FzZB/3+z" Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8C3EBD3 for ; Wed, 6 Dec 2023 04:40:13 -0800 (PST) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 04D155C0296 for ; Wed, 6 Dec 2023 07:40:13 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute1.internal (MEProxy); Wed, 06 Dec 2023 07:40:13 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to; s=fm1; t=1701866413; x=1701952813; bh=gv lWqrz39eBQx6tdLh+1nsoIyVICx9Gqa/3MxQFqkoc=; b=s4dtbI+dzAPpE7XS8N rPkpI2A9mAqkAyfzBhE3vZC3loQm82YZ6lH0XyuSSilYdYp0wgiqHWPYDucSYIv9 Xlj64ixGlfRP60+jCrz3p20nxaurVL74z8XDfBxh61rhApZnUV0liCXczMj/L3Re dzeFZtaWhxht3ztEYku+5ZJ17QtFyBQKhqLq3iTfW1FYbWWHq937EV3ASFQeGinX t94kAPn1aQWGVIlkjeKj97n9rmmcUbCvqeO7642t8ZYcgcV8VL6mzSUyztP2FWSW +qk+yho3XGtRkyyp7rZp7n4uQnKUKzBJzUjD/HPkKLcX/XxETv6zaLCfjtgbIAFl zfuA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; t=1701866413; x=1701952813; bh=gvlWqrz39eBQx 6tdLh+1nsoIyVICx9Gqa/3MxQFqkoc=; b=FzZB/3+zkL4/HoHETyqY7QZUpzaU3 PuyXcbyiwXDoNbjUx/jDWOvqPueOnL2ll3UUz4SCrGt8I1qAzMRdfpFJC7yj++Yt x7VX55WErYXb1AQlsGETv4ogMxd2Q4llK8l92Iint1H0XgWzn52bQIWY+IzEn3p0 1oTZIE7ZN1cz0M/mTnFBEF0F2nW0BRrkkXK1Ff6/Z839ooedZjuOk3155KpDPxmX ggnlZYTl3DINl9EBsPwDdGPLgrB7EDlUUj5Sld+t/nCsH/7LvMTpZw/mjy7Fudli 6v0nJhUkd7tmo7tHJXkDmDV9U+h4uZLKW8k0zzTWX2EIbMDL6AG6J+Yag== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrudektddggeegucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpeffhffvuffkfhggtggujgesghdtre ertddtvdenucfhrhhomheprfgrthhrihgtkhcuufhtvghinhhhrghrughtuceophhssehp khhsrdhimheqnecuggftrfgrthhtvghrnhepheeghfdtfeeuffehkefgffduleffjedthf dvjeektdfhhedvlefgtefgvdettdfhnecuvehluhhsthgvrhfuihiivgeptdenucfrrghr rghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Wed, 6 Dec 2023 07:40:12 -0500 (EST) Received: by vm-mail (OpenSMTPD) with ESMTPSA id 7babb740 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO) for ; Wed, 6 Dec 2023 12:38:46 +0000 (UTC) Date: Wed, 6 Dec 2023 13:40:10 +0100 From: Patrick Steinhardt To: git@vger.kernel.org Subject: [PATCH 6/7] builtin/clone: skip reading HEAD when retrieving remote Message-ID: <6c919fb19c4e0a723c23b407717fda7649bb360f.1701863960.git.ps@pks.im> References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: After we have set up the remote configuration in git-clone(1) we'll call `remote_get()` to read the remote from the on-disk configuration. But next to reading the on-disk configuration, `remote_get()` will also cause us to try and read the repository's HEAD reference so that we can figure out the current branch. Besides being pointless in git-clone(1) because we're operating in an empty repository anyway, this will also break once we move creation of the reference database to a later point in time. Refactor the code to introduce a new `remote_get_early()` function that will skip reading the HEAD reference to address this issue. Signed-off-by: Patrick Steinhardt --- builtin/clone.c | 2 +- remote.c | 26 ++++++++++++++++---------- remote.h | 1 + 3 files changed, 18 insertions(+), 11 deletions(-) diff --git a/builtin/clone.c b/builtin/clone.c index 9c60923f31..06966c5d4c 100644 --- a/builtin/clone.c +++ b/builtin/clone.c @@ -1185,7 +1185,7 @@ int cmd_clone(int argc, const char **argv, const char *prefix) if (option_required_reference.nr || option_optional_reference.nr) setup_reference(); - remote = remote_get(remote_name); + remote = remote_get_early(remote_name); refspec_appendf(&remote->fetch, "+%s*:%s*", src_ref_prefix, branch_top.buf); diff --git a/remote.c b/remote.c index abb24822be..051d0a64a0 100644 --- a/remote.c +++ b/remote.c @@ -509,7 +509,7 @@ static void alias_all_urls(struct remote_state *remote_state) } } -static void read_config(struct repository *repo) +static void read_config(struct repository *repo, int early) { int flag; @@ -518,7 +518,7 @@ static void read_config(struct repository *repo) repo->remote_state->initialized = 1; repo->remote_state->current_branch = NULL; - if (startup_info->have_repository) { + if (startup_info->have_repository && !early) { const char *head_ref = refs_resolve_ref_unsafe( get_main_ref_store(repo), "HEAD", 0, NULL, &flag); if (head_ref && (flag & REF_ISSYMREF) && @@ -561,7 +561,7 @@ static const char *remotes_remote_for_branch(struct remote_state *remote_state, const char *remote_for_branch(struct branch *branch, int *explicit) { - read_config(the_repository); + read_config(the_repository, 0); die_on_missing_branch(the_repository, branch); return remotes_remote_for_branch(the_repository->remote_state, branch, @@ -587,7 +587,7 @@ remotes_pushremote_for_branch(struct remote_state *remote_state, const char *pushremote_for_branch(struct branch *branch, int *explicit) { - read_config(the_repository); + read_config(the_repository, 0); die_on_missing_branch(the_repository, branch); return remotes_pushremote_for_branch(the_repository->remote_state, @@ -599,7 +599,7 @@ static struct remote *remotes_remote_get(struct remote_state *remote_state, const char *remote_ref_for_branch(struct branch *branch, int for_push) { - read_config(the_repository); + read_config(the_repository, 0); die_on_missing_branch(the_repository, branch); if (branch) { @@ -709,7 +709,13 @@ remotes_remote_get(struct remote_state *remote_state, const char *name) struct remote *remote_get(const char *name) { - read_config(the_repository); + read_config(the_repository, 0); + return remotes_remote_get(the_repository->remote_state, name); +} + +struct remote *remote_get_early(const char *name) +{ + read_config(the_repository, 1); return remotes_remote_get(the_repository->remote_state, name); } @@ -722,7 +728,7 @@ remotes_pushremote_get(struct remote_state *remote_state, const char *name) struct remote *pushremote_get(const char *name) { - read_config(the_repository); + read_config(the_repository, 0); return remotes_pushremote_get(the_repository->remote_state, name); } @@ -738,7 +744,7 @@ int remote_is_configured(struct remote *remote, int in_repo) int for_each_remote(each_remote_fn fn, void *priv) { int i, result = 0; - read_config(the_repository); + read_config(the_repository, 0); for (i = 0; i < the_repository->remote_state->remotes_nr && !result; i++) { struct remote *remote = @@ -1831,7 +1837,7 @@ struct branch *branch_get(const char *name) { struct branch *ret; - read_config(the_repository); + read_config(the_repository, 0); if (!name || !*name || !strcmp(name, "HEAD")) ret = the_repository->remote_state->current_branch; else @@ -1973,7 +1979,7 @@ static const char *branch_get_push_1(struct remote_state *remote_state, const char *branch_get_push(struct branch *branch, struct strbuf *err) { - read_config(the_repository); + read_config(the_repository, 0); die_on_missing_branch(the_repository, branch); if (!branch) diff --git a/remote.h b/remote.h index cdc8b1db42..79353ba226 100644 --- a/remote.h +++ b/remote.h @@ -118,6 +118,7 @@ struct remote { * and configuration. */ struct remote *remote_get(const char *name); +struct remote *remote_get_early(const char *name); struct remote *pushremote_get(const char *name); int remote_is_configured(struct remote *remote, int in_repo); From patchwork Wed Dec 6 12:40:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13481743 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="HwdzjbNc"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="kXpbIAbs" Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7363911F for ; Wed, 6 Dec 2023 04:40:18 -0800 (PST) Received: from compute6.internal (compute6.nyi.internal [10.202.2.47]) by mailout.nyi.internal (Postfix) with ESMTP id E09135C029B for ; Wed, 6 Dec 2023 07:40:17 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute6.internal (MEProxy); Wed, 06 Dec 2023 07:40:17 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to; s=fm1; t=1701866417; x=1701952817; bh=Si NMrssfWAqK6wKLQudCbA7utvmo8GD+9E6sFgkqLN0=; b=HwdzjbNc3MNq/tEFVq PR8aoS8IdJ/TN6kNp8kyBeq3RrJ07auvbQlfNP63IHXIz8oRvZNkuh/AhkunRW8b HfPYNX/4w6zQtHMFW1odPVKfUzOI+v9Uuu0JqDXs3p9XtEHMbDFpr00IGrHJS9Ji rucYxIGjOvxQ9QK73seW7cF5rAtpVXXlHqTlcn9Np8MDN+DQ/f6a+9LEowvjmOxo stdzHqWJJ2B1kvZxjmXueWfDRT8FkrUJWlbWHk0yi2An9MGjApQM2WaNTIKbgD1H mUjAi9YOzfjRT7vvWfXrcIP6aV6niHDeEr7ys1cI4nicAMhjhfWCZZIYAR/ebAAl QPqQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; t=1701866417; x=1701952817; bh=SiNMrssfWAqK6 wKLQudCbA7utvmo8GD+9E6sFgkqLN0=; b=kXpbIAbsXuRQsTIGAYzkPGyPEaLTW lY3HYJHjGpPYw15wAtYodPJTT2BW+8/HrZTG58rcW/8qElGuLD6pTcuK9l81BoBr 5Bpk79EAazHkcijRsWThtud6/LGJKVeKavNLhD766pQdn7TMAoLht5K9QYF9UZGN 6VDI8GCYUMbNj++AMg+0w0RoK5PPwHCibic1lJUPjGf3hvMcK3XUv/Z/qfKuNvUu txX3/Np9aTJWN5pjPPDrNJdcCEBu22vzYFpI1nMDb21oU4cR3UGw/5p1Z/HKl1qf RDrdMHE9DvZjklynx9lMtVrE5CFgyjKv7mYksI6K1OY32dj7+ohgC4xmA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrudektddggeegucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpeffhffvuffkfhggtggujgesghdtre ertddtvdenucfhrhhomheprfgrthhrihgtkhcuufhtvghinhhhrghrughtuceophhssehp khhsrdhimheqnecuggftrfgrthhtvghrnhepheeghfdtfeeuffehkefgffduleffjedthf dvjeektdfhhedvlefgtefgvdettdfhnecuvehluhhsthgvrhfuihiivgepudenucfrrghr rghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Wed, 6 Dec 2023 07:40:17 -0500 (EST) Received: by vm-mail (OpenSMTPD) with ESMTPSA id 482bd64b (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO) for ; Wed, 6 Dec 2023 12:38:51 +0000 (UTC) Date: Wed, 6 Dec 2023 13:40:15 +0100 From: Patrick Steinhardt To: git@vger.kernel.org Subject: [PATCH 7/7] builtin/clone: create the refdb with the correct object format Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: We're currently creating the reference database with a potentially incorrect object format when the remote repository's object format is different from the local default object format. This works just fine for now because the files backend never records the object format anywhere. But this logic will fail with any new reference backend that encodes this information in some form either on-disk or in-memory. The preceding commits have reshuffled code in git-clone(1) so that there is no code path that will access the reference database before we have detected the remote's object format. With these refactorings we can now defer initialization of the reference database until after we have learned the remote's object format and thus initialize it with the correct format from the get-go. These refactorings are required to make git-clone(1) work with the upcoming reftable backend when cloning repositories with the SHA256 object format. Signed-off-by: Patrick Steinhardt --- builtin/clone.c | 9 ++++++++- setup.c | 2 +- setup.h | 1 + 3 files changed, 10 insertions(+), 2 deletions(-) diff --git a/builtin/clone.c b/builtin/clone.c index 06966c5d4c..fd052b8b54 100644 --- a/builtin/clone.c +++ b/builtin/clone.c @@ -1097,8 +1097,14 @@ int cmd_clone(int argc, const char **argv, const char *prefix) } } + /* + * Initialize the repository, but skip initializing the reference + * database. We do not yet know about the object format of the + * repository, and reference backends may persist that information into + * their on-disk data structures. + */ init_db(git_dir, real_git_dir, option_template, GIT_HASH_UNKNOWN, NULL, - do_not_override_repo_unix_permissions, INIT_DB_QUIET); + do_not_override_repo_unix_permissions, INIT_DB_QUIET | INIT_DB_SKIP_REFDB); if (real_git_dir) { free((char *)git_dir); @@ -1282,6 +1288,7 @@ int cmd_clone(int argc, const char **argv, const char *prefix) hash_algo = hash_algo_by_ptr(transport_get_hash_algo(transport)); initialize_repository_version(hash_algo, 1); repo_set_hash_algo(the_repository, hash_algo); + create_reference_database(NULL, 1); /* * Before fetching from the remote, download and install bundle diff --git a/setup.c b/setup.c index a80fc09b9c..e1d0ce29c6 100644 --- a/setup.c +++ b/setup.c @@ -1897,7 +1897,7 @@ static int is_reinit(void) return ret; } -static void create_reference_database(const char *initial_branch, int quiet) +void create_reference_database(const char *initial_branch, int quiet) { struct strbuf err = STRBUF_INIT; int reinit = is_reinit(); diff --git a/setup.h b/setup.h index cbf538286b..3f0f17c351 100644 --- a/setup.h +++ b/setup.h @@ -178,6 +178,7 @@ int init_db(const char *git_dir, const char *real_git_dir, const char *initial_branch, int init_shared_repository, unsigned int flags); void initialize_repository_version(int hash_algo, int reinit); +void create_reference_database(const char *initial_branch, int quiet); /* * NOTE NOTE NOTE!!