From patchwork Thu Dec 28 09:57:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13505775 Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8BDBF63B1 for ; Thu, 28 Dec 2023 09:57:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="lTiQVCag"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="mER9cv0u" Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailout.nyi.internal (Postfix) with ESMTP id 95B635C0080; Thu, 28 Dec 2023 04:57:27 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute2.internal (MEProxy); Thu, 28 Dec 2023 04:57:27 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm2; t=1703757447; x=1703843847; bh=B19MDGHCdz 3g7oTy/4t5EtLlPwU9hnEETti4CfOhdyg=; b=lTiQVCagKX4cujgBrX+JKdKV7L OTKfD3YVVfHHVrcbB3iXsH0fj1ovcyXvqFUJ6iR1sSK5gj0aBADeRYXMsxsmsQrd iBmB0JgtjSVnR5iegx4Shd/eQN9DKuQNSx6PXn+BGVe8XPQgMik1kA421JFPj4h4 pdIWLjdQYr3DP84nY8uQqmj2uHEZ6yeag0jui4NeLJj7FedOMAbxPD5leM+UqXO1 tsTGRkb+KWg+zZJLJGuwYpGiPbFit02xIltelFENmiHtKDkdsuFtNlZQu6+CCGBw 0qaLlPAQ9E/a5CsZR1ASpQ5MBieo5EIUHHnhgAl20neeUByV8lSWMoPB6Wiw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; t=1703757447; x=1703843847; bh=B19MDGHCdz3g7oTy/4t5EtLlPwU9 hnEETti4CfOhdyg=; b=mER9cv0ux7cYz7D3h5aQHI2cypJoFCwnR7RSA5xCT/7m 6xM5C2ua72O5cemB40TdVEQUwkuZSQ9fHoPBzri11C02hXqEVYTklJrBYmEgTE3H rmsGx8uhLZQ9XNl5X96V2AieXJYdouFnoz0fo2M1k780OV5Dg/wRAMEq7N1+NyOC a2z7JZN0MG5A6cpmXm13TwMtn5SzCzofQ+9u//3rYDRij+s/0QquLOd79lb4o/EH elevUWdiSsBzqrXKCUwnD6XdK1s/ltaokL2T5dHkckzsTPfx6UdwdXMQBipNCv9V EqA1D32BvIJl5DQEHIYz95hSGf4EDSulwCfGHve+Gg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrvdefuddguddtucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvfevuffkfhggtggujgesghdtreertddtvdenucfhrhhomheprfgrthhr ihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimheqnecuggftrfgrthhtvg hrnhepueektdevtdffveeljeetgfehheeigeekleduvdeffeeghefgledttdehjeelffet necuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepphhsse hpkhhsrdhimh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 28 Dec 2023 04:57:26 -0500 (EST) Received: by vm-mail (OpenSMTPD) with ESMTPSA id e6df7380 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Thu, 28 Dec 2023 09:55:10 +0000 (UTC) Date: Thu, 28 Dec 2023 10:57:23 +0100 From: Patrick Steinhardt To: git@vger.kernel.org Cc: Karthik Nayak , Junio C Hamano Subject: [PATCH v2 00/12] Introduce `refStorage` extension Message-ID: References: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: Hi, this is the second version of my patch series that introduces the new `refStorage` extension. This extension will be used for the upcoming reftable backend. Changes compared to v2: - Fixed various typos in commit messages. - Fixed redundant information when the refstorage extension's value isn't understood. - Introduced `repo_set_ref_storage_format()`. Thanks for the feedback so far! Patrick Patrick Steinhardt (12): t: introduce DEFAULT_REPO_FORMAT prereq worktree: skip reading HEAD when repairing worktrees refs: refactor logic to look up storage backends setup: start tracking ref storage format setup: set repository's formats on init setup: introduce "extensions.refStorage" extension setup: introduce GIT_DEFAULT_REF_FORMAT envvar t: introduce GIT_TEST_DEFAULT_REF_FORMAT envvar builtin/rev-parse: introduce `--show-ref-format` flag builtin/init: introduce `--ref-format=` value flag builtin/clone: introduce `--ref-format=` value flag t9500: write "extensions.refstorage" into config Documentation/config/extensions.txt | 11 +++ Documentation/git-clone.txt | 6 ++ Documentation/git-init.txt | 7 ++ Documentation/git-rev-parse.txt | 3 + Documentation/git.txt | 5 ++ Documentation/ref-storage-format.txt | 1 + .../technical/repository-version.txt | 5 ++ builtin/clone.c | 17 ++++- builtin/init-db.c | 15 +++- builtin/rev-parse.c | 4 ++ refs.c | 34 ++++++--- refs.h | 3 + refs/debug.c | 1 - refs/files-backend.c | 1 - refs/packed-backend.c | 1 - refs/refs-internal.h | 1 - repository.c | 6 ++ repository.h | 7 ++ setup.c | 63 +++++++++++++++-- setup.h | 9 ++- t/README | 3 + t/t0001-init.sh | 70 +++++++++++++++++++ t/t1500-rev-parse.sh | 17 +++++ t/t3200-branch.sh | 2 +- t/t5601-clone.sh | 17 +++++ t/t9500-gitweb-standalone-no-errors.sh | 5 ++ t/test-lib-functions.sh | 5 ++ t/test-lib.sh | 15 +++- worktree.c | 24 ++++--- 29 files changed, 323 insertions(+), 35 deletions(-) create mode 100644 Documentation/ref-storage-format.txt Range-diff against v1: 1: 239ca38efd ! 1: 3613439cb7 t: introduce DEFAULT_REPO_FORMAT prereq @@ Commit message repository format or otherwise they would fail to run, e.g. because they fail to detect the correct hash function. While the hash function is the only extension right now that creates problems like this, we are about - to add a second extensions for the ref format. + to add a second extension for the ref format. Introduce a new DEFAULT_REPO_FORMAT prereq that can easily be amended whenever we add new format extensions. Next to making any such changes 2: e895091025 ! 2: ecf4f1ddee worktree: skip reading HEAD when repairing worktrees @@ Commit message worktree: skip reading HEAD when repairing worktrees When calling `git init --separate-git-dir=` on a preexisting - repository, then we move the Git directory of that repository to the new - path specified by the user. If there are worktrees present in the Git - repository, we need to repair the worktrees so that their gitlinks point - to the new location of the repository. + repository, we move the Git directory of that repository to the new path + specified by the user. If there are worktrees present in the repository, + we need to repair the worktrees so that their gitlinks point to the new + location of the repository. This repair logic will load repositories via `get_worktrees()`, which will enumerate up and initialize all worktrees. Part of initialization @@ Commit message format, which does not work. We do not require the worktree HEADs at all to repair worktrees. So - let's fix issue this by skipping over the step that reads them. + let's fix this issue by skipping over the step that reads them. Signed-off-by: Patrick Steinhardt 3: f712d5ef5b ! 3: 12329b99b7 refs: refactor logic to look up storage backends @@ refs.c - for (be = refs_backends; be; be = be->next) - if (!strcmp(be->name, name)) - return be; -+ if (ref_storage_format && ref_storage_format < ARRAY_SIZE(refs_backends)) ++ if (ref_storage_format < ARRAY_SIZE(refs_backends)) + return refs_backends[ref_storage_format]; return NULL; } @@ refs.c: static struct ref_store *ref_store_init(struct repository *repo, if (!be) - BUG("reference backend %s is unknown", be_name); -+ BUG("reference backend %s is unknown", ref_storage_format_to_name(format)); ++ BUG("reference backend is unknown"); refs = be->init(repo, gitdir, flags); return refs; 4: 6564659d40 ! 4: ddd099fbaf setup: start tracking ref storage format when @@ Metadata Author: Patrick Steinhardt ## Commit message ## - setup: start tracking ref storage format when + setup: start tracking ref storage format In order to discern which ref storage format a repository is supposed to use we need to start setting up and/or discovering the format. This @@ Commit message - The first path is when we create a repository via `init_db()`. When we are re-initializing a preexisting repository we need to retain the previously used ref storage format -- if the user asked for a - different format then this indicates an erorr and we error out. + different format then this indicates an error and we error out. Otherwise we either initialize the repository with the format asked for by the user or the default format, which currently is the "files" backend. @@ refs.c: static struct ref_store *ref_store_init(struct repository *repo, + be = find_ref_storage_backend(repo->ref_storage_format); if (!be) -- BUG("reference backend %s is unknown", ref_storage_format_to_name(format)); -+ BUG("reference backend is unknown"); - - refs = be->init(repo, gitdir, flags); - return refs; - - ## refs.h ## -@@ refs.h: struct string_list; - struct string_list_item; - struct worktree; - -+int default_ref_storage_format(void); - int ref_storage_format_by_name(const char *name); - const char *ref_storage_format_to_name(int ref_storage_format); + BUG("reference backend is unknown"); ## repository.c ## +@@ repository.c: void repo_set_hash_algo(struct repository *repo, int hash_algo) + repo->hash_algo = &hash_algos[hash_algo]; + } + ++void repo_set_ref_storage_format(struct repository *repo, int format) ++{ ++ repo->ref_storage_format = format; ++} ++ + /* + * Attempt to resolve and set the provided 'gitdir' for repository 'repo'. + * Return 0 upon success and a non-zero value upon failure. @@ repository.c: int repo_init(struct repository *repo, goto error; repo_set_hash_algo(repo, format.hash_algo); -+ repo->ref_storage_format = format.ref_storage_format; ++ repo_set_ref_storage_format(repo, format.ref_storage_format); repo->repository_format_worktree_config = format.worktree_config; /* take ownership of format.partial_clone */ @@ repository.h: struct repository { /* A unique-id for tracing purposes. */ int trace2_repo_id; +@@ repository.h: void repo_set_gitdir(struct repository *repo, const char *root, + const struct set_gitdir_args *extra_args); + void repo_set_worktree(struct repository *repo, const char *path); + void repo_set_hash_algo(struct repository *repo, int algo); ++void repo_set_ref_storage_format(struct repository *repo, int format); + void initialize_the_repository(void); + RESULT_MUST_BE_USED + int repo_init(struct repository *r, const char *gitdir, const char *worktree); ## setup.c ## @@ setup.c: const char *setup_git_directory_gently(int *nongit_ok) } if (startup_info->have_repository) { repo_set_hash_algo(the_repository, repo_fmt.hash_algo); -+ the_repository->ref_storage_format = -+ repo_fmt.ref_storage_format; ++ repo_set_ref_storage_format(the_repository, ++ repo_fmt.ref_storage_format); the_repository->repository_format_worktree_config = repo_fmt.worktree_config; /* take ownership of repo_fmt.partial_clone */ @@ setup.c: void check_repository_format(struct repository_format *fmt) check_repository_format_gently(get_git_dir(), fmt, NULL); startup_info->have_repository = 1; repo_set_hash_algo(the_repository, fmt->hash_algo); -+ the_repository->ref_storage_format = -+ fmt->ref_storage_format; ++ repo_set_ref_storage_format(the_repository, ++ fmt->ref_storage_format); the_repository->repository_format_worktree_config = fmt->worktree_config; the_repository->repository_format_partial_clone = @@ setup.c: void create_reference_database(const char *initial_branch, int quiet) safe_create_dir(git_path("refs"), 1); adjust_shared_perm(git_path("refs")); -+ the_repository->ref_storage_format = ref_storage_format; ++ repo_set_ref_storage_format(the_repository, ref_storage_format); if (refs_init_db(&err)) die("failed to set up refs db: %s", err.buf); 5: f90a63d63c ! 5: 01a1e58a97 setup: set repository's formats on init @@ setup.c: int init_db(const char *git_dir, const char *real_git_dir, + * Now that we have set up both the hash algorithm and the ref storage + * format we can update the repository's settings accordingly. + */ -+ the_repository->hash_algo = &hash_algos[repo_fmt.hash_algo]; -+ the_repository->ref_storage_format = repo_fmt.ref_storage_format; ++ repo_set_hash_algo(the_repository, repo_fmt.hash_algo); ++ repo_set_ref_storage_format(the_repository, repo_fmt.ref_storage_format); + if (!(flags & INIT_DB_SKIP_REFDB)) create_reference_database(repo_fmt.ref_storage_format, 6: beeb182f28 = 6: 0a586fa648 setup: introduce "extensions.refStorage" extension 7: dd91a75da4 = 7: 6d8754f73a setup: introduce GIT_DEFAULT_REF_FORMAT envvar 8: ed3bf008cd = 8: c645932f3d t: introduce GIT_TEST_DEFAULT_REF_FORMAT envvar 9: 8a3d950d69 = 9: 761d647770 builtin/rev-parse: introduce `--show-ref-format` flag 10: 4d98b53553 = 10: e382b5bf08 builtin/init: introduce `--ref-format=` value flag 11: 71cf0ce827 = 11: 257233658d builtin/clone: introduce `--ref-format=` value flag 12: bbe2fbb154 ! 12: b8cd06ec53 t9500: write "extensions.refstorage" into config @@ Commit message t9500: write "extensions.refstorage" into config In t9500 we're writing a custom configuration that sets up gitweb. This - requires us manually ensure that the repository format is configured as - required, including both the repository format version and extensions. - With the introduction of the "extensions.refStorage" extension we need - to update the test to also write this new one. + requires us to manually ensure that the repository format is configured + as required, including both the repository format version and + extensions. With the introduction of the "extensions.refStorage" + extension we need to update the test to also write this new one. Signed-off-by: Patrick Steinhardt base-commit: e79552d19784ee7f4bbce278fe25f93fbda196fa