From patchwork Tue Nov 26 06:42:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13885503 Received: from fhigh-a7-smtp.messagingengine.com (fhigh-a7-smtp.messagingengine.com [103.168.172.158]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6B524539A for ; Tue, 26 Nov 2024 06:43:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.158 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732603394; cv=none; b=RFG9wVC41pLE3ftrDB/UuE+3/bUrr7hzCIf38fxUvvxnf0PSUfYDEJgVuqkYOAppasF3ygshWmMUO2/NBPieRnTMzm0SDYjAjPcG253pjjiEa6TregkEl7JU1HCegPY2wSw4PFOYYYhoB7EuZZgnL8e/eN0P4vXW2/Gtrh5kygo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732603394; c=relaxed/simple; bh=RY7R45HryQCGEurXCC5y44erK3ifKcC6cc4buSp1nQ0=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=tDtYy9Vi4gdSVrQDF702bap36NAGAQesABT8y+SJAWJp1TqLTw84gkeiUMkTy2kkNj/TzMmZGGiXyc2Tnx+v3pscVLEJV6z9Z79nyBWp5RJCZxyiWxrgvGtkTcjNGCVboyOSGdBt1b6irgTB1+F8qTvz4adb92q2WPr2toUJ7wk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=mEaIry8C; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=xLsEzAaL; arc=none smtp.client-ip=103.168.172.158 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="mEaIry8C"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="xLsEzAaL" Received: from phl-compute-01.internal (phl-compute-01.phl.internal [10.202.2.41]) by mailfhigh.phl.internal (Postfix) with ESMTP id 576781140195; Tue, 26 Nov 2024 01:43:10 -0500 (EST) Received: from phl-mailfrontend-02 ([10.202.2.163]) by phl-compute-01.internal (MEProxy); Tue, 26 Nov 2024 01:43:10 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm1; t=1732603390; x=1732689790; bh=X/1UpB6qwXH7Qlu/+bguFnTwdCuZMyX56PW7CQkIpZQ=; b= mEaIry8Cde4k22FPpaNfbDjXmMzKVTK5b9Rv0QPGjJ3QFC07gJW4xmcST5coN3pR dsTJhn5MofkjsjxE7mptXhOxIitZmV+NEWaNdKZE64it+i142TNwZ3IfXiTioZPs rIYKT740qG0VyOM5irlahYBC9lViz3fnMPe2KKSM6T4ywufeZv88wee/qiqeQAZl JhzGG/BrC3bpKQ9nNIEd2nA2OY1Wbe9orPM5TpnGup609D5nMr74s0n1jndllEjc Cnwv1Dc37QRNYFy78HHj72yTsT+AvaMjdtN0wSuzXZZsd0+2vHDjLZkEM/xjW5yi OfehPMMBIE18i+Qe04mrAw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1732603390; x= 1732689790; bh=X/1UpB6qwXH7Qlu/+bguFnTwdCuZMyX56PW7CQkIpZQ=; b=x LsEzAaLaO39Ihm2ythKfwl2xXGhQXR8fH/lLnE555ROzkKtuWzqBLdH6lDunVieK 6KUusvGRz4p6cMWZFifY4Pt4tv+GUFpp8VC8MjpL6EeOlCsb3vnmjIcFsgvdnkhX dgW67ERjFdpsHv2alLHdMZYKOz9fZDJoTeKe045wUvqDxH7JLiXSzGH+qdO1l0Nq p7snqBfGFF7TeGvCpACswrEs+q/gJ/YpodurkFbVmDZiyGAW0AcDySIXGZwkEejE 16SEbjk0OWFRj2cChiNIJ6wrLUK1P/oW1CjnjrJ77Dyc5AQHpjDaixg58mTe9Wy2 AgPaAWc0dBYnGcIVPzoxg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefuddrgeeigdeliecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpggftfghnshhusghstghrihgsvgdpuffr tefokffrpgfnqfghnecuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnth hsucdlqddutddtmdenucfjughrpefhfffugggtgffkfhgjvfevofesthejredtredtjeen ucfhrhhomheprfgrthhrihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimh eqnecuggftrfgrthhtvghrnhepffeuiedujedvkeehuedvkeefffeivdeuleetkeduheej teekgedvudfgtdfgieelnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrg hilhhfrhhomhepphhssehpkhhsrdhimhdpnhgspghrtghpthhtohepfedpmhhouggvpehs mhhtphhouhhtpdhrtghpthhtohepghhithesvhhgvghrrdhkvghrnhgvlhdrohhrghdprh gtphhtthhopehgihhtshhtvghrsehpohgsohigrdgtohhmpdhrtghpthhtohepkhgrrhht hhhikhdrudekkeesghhmrghilhdrtghomh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 26 Nov 2024 01:43:09 -0500 (EST) Received: by vm-mail (OpenSMTPD) with ESMTPSA id 2f4f44ba (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Tue, 26 Nov 2024 06:42:05 +0000 (UTC) From: Patrick Steinhardt Date: Tue, 26 Nov 2024 07:42:52 +0100 Subject: [PATCH v4 01/10] refs/reftable: encapsulate reftable stack Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241126-pks-reftable-backend-reuse-iter-v4-1-b17fd27df126@pks.im> References: <20241126-pks-reftable-backend-reuse-iter-v4-0-b17fd27df126@pks.im> In-Reply-To: <20241126-pks-reftable-backend-reuse-iter-v4-0-b17fd27df126@pks.im> To: git@vger.kernel.org Cc: karthik nayak , Junio C Hamano X-Mailer: b4 0.14.2 The reftable ref store needs to keep track of multiple stacks, one for the main worktree and an arbitrary number of stacks for worktrees. This is done by storing pointers to `struct reftable_stack`, which we then access directly. Wrap the stack in a new `struct reftable_backend`. This will allow us to attach more data to each respective stack in subsequent commits. Signed-off-by: Patrick Steinhardt --- refs/reftable-backend.c | 135 +++++++++++++++++++++++++++--------------------- 1 file changed, 76 insertions(+), 59 deletions(-) diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c index f560bc2b67857d785294e6b5699383a256f30813..acd26f8928d18396f78a2d39ad0e0c1796d5a409 100644 --- a/refs/reftable-backend.c +++ b/refs/reftable-backend.c @@ -34,24 +34,41 @@ */ #define REF_UPDATE_VIA_HEAD (1 << 8) +struct reftable_backend { + struct reftable_stack *stack; +}; + +static int reftable_backend_init(struct reftable_backend *be, + const char *path, + const struct reftable_write_options *opts) +{ + return reftable_new_stack(&be->stack, path, opts); +} + +static void reftable_backend_release(struct reftable_backend *be) +{ + reftable_stack_destroy(be->stack); + be->stack = NULL; +} + struct reftable_ref_store { struct ref_store base; /* - * The main stack refers to the common dir and thus contains common + * The main backend refers to the common dir and thus contains common * refs as well as refs of the main repository. */ - struct reftable_stack *main_stack; + struct reftable_backend main_backend; /* - * The worktree stack refers to the gitdir in case the refdb is opened + * The worktree backend refers to the gitdir in case the refdb is opened * via a worktree. It thus contains the per-worktree refs. */ - struct reftable_stack *worktree_stack; + struct reftable_backend worktree_backend; /* - * Map of worktree stacks by their respective worktree names. The map + * Map of worktree backends by their respective worktree names. The map * is populated lazily when we try to resolve `worktrees/$worktree` refs. */ - struct strmap worktree_stacks; + struct strmap worktree_backends; struct reftable_write_options write_options; unsigned int store_flags; @@ -97,21 +114,21 @@ static struct reftable_ref_store *reftable_be_downcast(struct ref_store *ref_sto * like `worktrees/$worktree/refs/heads/foo` as worktree stacks will store * those references in their normalized form. */ -static struct reftable_stack *stack_for(struct reftable_ref_store *store, - const char *refname, - const char **rewritten_ref) +static struct reftable_backend *backend_for(struct reftable_ref_store *store, + const char *refname, + const char **rewritten_ref) { const char *wtname; int wtname_len; if (!refname) - return store->main_stack; + return &store->main_backend; switch (parse_worktree_ref(refname, &wtname, &wtname_len, rewritten_ref)) { case REF_WORKTREE_OTHER: { static struct strbuf wtname_buf = STRBUF_INIT; struct strbuf wt_dir = STRBUF_INIT; - struct reftable_stack *stack; + struct reftable_backend *be; /* * We're using a static buffer here so that we don't need to @@ -125,37 +142,39 @@ static struct reftable_stack *stack_for(struct reftable_ref_store *store, /* * There is an edge case here: when the worktree references the * current worktree, then we set up the stack once via - * `worktree_stacks` and once via `worktree_stack`. This is + * `worktree_backends` and once via `worktree_backend`. This is * wasteful, but in the reading case it shouldn't matter. And * in the writing case we would notice that the stack is locked * already and error out when trying to write a reference via * both stacks. */ - stack = strmap_get(&store->worktree_stacks, wtname_buf.buf); - if (!stack) { + be = strmap_get(&store->worktree_backends, wtname_buf.buf); + if (!be) { strbuf_addf(&wt_dir, "%s/worktrees/%s/reftable", store->base.repo->commondir, wtname_buf.buf); - store->err = reftable_new_stack(&stack, wt_dir.buf, - &store->write_options); + CALLOC_ARRAY(be, 1); + store->err = reftable_backend_init(be, wt_dir.buf, + &store->write_options); assert(store->err != REFTABLE_API_ERROR); - strmap_put(&store->worktree_stacks, wtname_buf.buf, stack); + + strmap_put(&store->worktree_backends, wtname_buf.buf, be); } strbuf_release(&wt_dir); - return stack; + return be; } case REF_WORKTREE_CURRENT: /* * If there is no worktree stack then we're currently in the * main worktree. We thus return the main stack in that case. */ - if (!store->worktree_stack) - return store->main_stack; - return store->worktree_stack; + if (!store->worktree_backend.stack) + return &store->main_backend; + return &store->worktree_backend; case REF_WORKTREE_MAIN: case REF_WORKTREE_SHARED: - return store->main_stack; + return &store->main_backend; default: BUG("unhandled worktree reference type"); } @@ -292,7 +311,7 @@ static struct ref_store *reftable_be_init(struct repository *repo, umask(mask); base_ref_store_init(&refs->base, repo, gitdir, &refs_be_reftable); - strmap_init(&refs->worktree_stacks); + strmap_init(&refs->worktree_backends); refs->store_flags = store_flags; refs->log_all_ref_updates = repo_settings_get_log_all_ref_updates(repo); @@ -337,8 +356,8 @@ static struct ref_store *reftable_be_init(struct repository *repo, strbuf_realpath(&path, gitdir, 0); } strbuf_addstr(&path, "/reftable"); - refs->err = reftable_new_stack(&refs->main_stack, path.buf, - &refs->write_options); + refs->err = reftable_backend_init(&refs->main_backend, path.buf, + &refs->write_options); if (refs->err) goto done; @@ -354,8 +373,8 @@ static struct ref_store *reftable_be_init(struct repository *repo, strbuf_reset(&path); strbuf_addf(&path, "%s/reftable", gitdir); - refs->err = reftable_new_stack(&refs->worktree_stack, path.buf, - &refs->write_options); + refs->err = reftable_backend_init(&refs->worktree_backend, path.buf, + &refs->write_options); if (refs->err) goto done; } @@ -374,19 +393,17 @@ static void reftable_be_release(struct ref_store *ref_store) struct strmap_entry *entry; struct hashmap_iter iter; - if (refs->main_stack) { - reftable_stack_destroy(refs->main_stack); - refs->main_stack = NULL; - } + if (refs->main_backend.stack) + reftable_backend_release(&refs->main_backend); + if (refs->worktree_backend.stack) + reftable_backend_release(&refs->worktree_backend); - if (refs->worktree_stack) { - reftable_stack_destroy(refs->worktree_stack); - refs->worktree_stack = NULL; + strmap_for_each_entry(&refs->worktree_backends, &iter, entry) { + struct reftable_backend *be = entry->value; + reftable_backend_release(be); + free(be); } - - strmap_for_each_entry(&refs->worktree_stacks, &iter, entry) - reftable_stack_destroy(entry->value); - strmap_clear(&refs->worktree_stacks, 0); + strmap_clear(&refs->worktree_backends, 0); } static int reftable_be_create_on_disk(struct ref_store *ref_store, @@ -781,7 +798,7 @@ static struct ref_iterator *reftable_be_iterator_begin(struct ref_store *ref_sto required_flags |= REF_STORE_ODB; refs = reftable_be_downcast(ref_store, required_flags, "ref_iterator_begin"); - main_iter = ref_iterator_for_stack(refs, refs->main_stack, prefix, + main_iter = ref_iterator_for_stack(refs, refs->main_backend.stack, prefix, exclude_patterns, flags); /* @@ -789,14 +806,14 @@ static struct ref_iterator *reftable_be_iterator_begin(struct ref_store *ref_sto * right now. If we aren't, then we return the common reftable * iterator, only. */ - if (!refs->worktree_stack) + if (!refs->worktree_backend.stack) return &main_iter->base; /* * Otherwise we merge both the common and the per-worktree refs into a * single iterator. */ - worktree_iter = ref_iterator_for_stack(refs, refs->worktree_stack, prefix, + worktree_iter = ref_iterator_for_stack(refs, refs->worktree_backend.stack, prefix, exclude_patterns, flags); return merge_ref_iterator_begin(&worktree_iter->base, &main_iter->base, ref_iterator_select, NULL); @@ -811,7 +828,7 @@ static int reftable_be_read_raw_ref(struct ref_store *ref_store, { struct reftable_ref_store *refs = reftable_be_downcast(ref_store, REF_STORE_READ, "read_raw_ref"); - struct reftable_stack *stack = stack_for(refs, refname, &refname); + struct reftable_stack *stack = backend_for(refs, refname, &refname)->stack; int ret; if (refs->err < 0) @@ -838,7 +855,7 @@ static int reftable_be_read_symbolic_ref(struct ref_store *ref_store, { struct reftable_ref_store *refs = reftable_be_downcast(ref_store, REF_STORE_READ, "read_symbolic_ref"); - struct reftable_stack *stack = stack_for(refs, refname, &refname); + struct reftable_stack *stack = backend_for(refs, refname, &refname)->stack; struct reftable_ref_record ref = {0}; int ret; @@ -898,7 +915,7 @@ static int prepare_transaction_update(struct write_transaction_table_arg **out, struct ref_update *update, struct strbuf *err) { - struct reftable_stack *stack = stack_for(refs, update->refname, NULL); + struct reftable_stack *stack = backend_for(refs, update->refname, NULL)->stack; struct write_transaction_table_arg *arg = NULL; size_t i; int ret; @@ -1031,7 +1048,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store, goto done; } - ret = read_ref_without_reload(refs, stack_for(refs, "HEAD", NULL), "HEAD", + ret = read_ref_without_reload(refs, backend_for(refs, "HEAD", NULL)->stack, "HEAD", &head_oid, &head_referent, &head_type); if (ret < 0) goto done; @@ -1043,7 +1060,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store, struct reftable_stack *stack; const char *rewritten_ref; - stack = stack_for(refs, u->refname, &rewritten_ref); + stack = backend_for(refs, u->refname, &rewritten_ref)->stack; /* Verify that the new object ID is valid. */ if ((u->flags & REF_HAVE_NEW) && !is_null_oid(&u->new_oid) && @@ -1525,9 +1542,9 @@ static int reftable_be_pack_refs(struct ref_store *ref_store, if (refs->err) return refs->err; - stack = refs->worktree_stack; + stack = refs->worktree_backend.stack; if (!stack) - stack = refs->main_stack; + stack = refs->main_backend.stack; if (opts->flags & PACK_REFS_AUTO) ret = reftable_stack_auto_compact(stack); @@ -1782,7 +1799,7 @@ static int reftable_be_rename_ref(struct ref_store *ref_store, { struct reftable_ref_store *refs = reftable_be_downcast(ref_store, REF_STORE_WRITE, "rename_ref"); - struct reftable_stack *stack = stack_for(refs, newrefname, &newrefname); + struct reftable_stack *stack = backend_for(refs, newrefname, &newrefname)->stack; struct write_copy_arg arg = { .refs = refs, .stack = stack, @@ -1814,7 +1831,7 @@ static int reftable_be_copy_ref(struct ref_store *ref_store, { struct reftable_ref_store *refs = reftable_be_downcast(ref_store, REF_STORE_WRITE, "copy_ref"); - struct reftable_stack *stack = stack_for(refs, newrefname, &newrefname); + struct reftable_stack *stack = backend_for(refs, newrefname, &newrefname)->stack; struct write_copy_arg arg = { .refs = refs, .stack = stack, @@ -1952,11 +1969,11 @@ static struct ref_iterator *reftable_be_reflog_iterator_begin(struct ref_store * reftable_be_downcast(ref_store, REF_STORE_READ, "reflog_iterator_begin"); struct reftable_reflog_iterator *main_iter, *worktree_iter; - main_iter = reflog_iterator_for_stack(refs, refs->main_stack); - if (!refs->worktree_stack) + main_iter = reflog_iterator_for_stack(refs, refs->main_backend.stack); + if (!refs->worktree_backend.stack) return &main_iter->base; - worktree_iter = reflog_iterator_for_stack(refs, refs->worktree_stack); + worktree_iter = reflog_iterator_for_stack(refs, refs->worktree_backend.stack); return merge_ref_iterator_begin(&worktree_iter->base, &main_iter->base, ref_iterator_select, NULL); @@ -1995,7 +2012,7 @@ static int reftable_be_for_each_reflog_ent_reverse(struct ref_store *ref_store, { struct reftable_ref_store *refs = reftable_be_downcast(ref_store, REF_STORE_READ, "for_each_reflog_ent_reverse"); - struct reftable_stack *stack = stack_for(refs, refname, &refname); + struct reftable_stack *stack = backend_for(refs, refname, &refname)->stack; struct reftable_log_record log = {0}; struct reftable_iterator it = {0}; int ret; @@ -2035,7 +2052,7 @@ static int reftable_be_for_each_reflog_ent(struct ref_store *ref_store, { struct reftable_ref_store *refs = reftable_be_downcast(ref_store, REF_STORE_READ, "for_each_reflog_ent"); - struct reftable_stack *stack = stack_for(refs, refname, &refname); + struct reftable_stack *stack = backend_for(refs, refname, &refname)->stack; struct reftable_log_record *logs = NULL; struct reftable_iterator it = {0}; size_t logs_alloc = 0, logs_nr = 0, i; @@ -2084,7 +2101,7 @@ static int reftable_be_reflog_exists(struct ref_store *ref_store, { struct reftable_ref_store *refs = reftable_be_downcast(ref_store, REF_STORE_READ, "reflog_exists"); - struct reftable_stack *stack = stack_for(refs, refname, &refname); + struct reftable_stack *stack = backend_for(refs, refname, &refname)->stack; struct reftable_log_record log = {0}; struct reftable_iterator it = {0}; int ret; @@ -2169,7 +2186,7 @@ static int reftable_be_create_reflog(struct ref_store *ref_store, { struct reftable_ref_store *refs = reftable_be_downcast(ref_store, REF_STORE_WRITE, "create_reflog"); - struct reftable_stack *stack = stack_for(refs, refname, &refname); + struct reftable_stack *stack = backend_for(refs, refname, &refname)->stack; struct write_reflog_existence_arg arg = { .refs = refs, .stack = stack, @@ -2243,7 +2260,7 @@ static int reftable_be_delete_reflog(struct ref_store *ref_store, { struct reftable_ref_store *refs = reftable_be_downcast(ref_store, REF_STORE_WRITE, "delete_reflog"); - struct reftable_stack *stack = stack_for(refs, refname, &refname); + struct reftable_stack *stack = backend_for(refs, refname, &refname)->stack; struct write_reflog_delete_arg arg = { .stack = stack, .refname = refname, @@ -2352,7 +2369,7 @@ static int reftable_be_reflog_expire(struct ref_store *ref_store, */ struct reftable_ref_store *refs = reftable_be_downcast(ref_store, REF_STORE_WRITE, "reflog_expire"); - struct reftable_stack *stack = stack_for(refs, refname, &refname); + struct reftable_stack *stack = backend_for(refs, refname, &refname)->stack; struct reftable_log_record *logs = NULL; struct reftable_log_record *rewritten = NULL; struct reftable_ref_record ref_record = {0}; From patchwork Tue Nov 26 06:42:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13885504 Received: from fhigh-a7-smtp.messagingengine.com (fhigh-a7-smtp.messagingengine.com [103.168.172.158]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5B92017CA17 for ; Tue, 26 Nov 2024 06:43:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.158 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732603394; cv=none; b=C6LlAWhBxaLYH7lQYBuG/MtHF9I5x6pmdHrb8E++TvA/vp9SzyuIHfqEsSOcNeL+WrhHETIPth/AzwyoGrDKAXq1S22lBV/XmCbNPG2573EKK2CBqH87WYjcYa/7az2TChHOTuGEkqQ0gm8xhrvIV0QiY/sfbToichBfAqy+OHk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732603394; c=relaxed/simple; bh=hvRMD0+vP2mFq5CwXaA8uS4HvDpj0aFPiTQ4GYbKA+Y=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=bY3DQSyD039JlTV74ISz8g4SrHFQsdeFAzIEQKBhrmX400mPwjFDiJn/P8yKuaHcTOlT0LNVvrBmNwdXbUQy72P0ZWjGOM7AIFm8ELWDMuGoZNAB04rsbsncG4VKyeDk00xVMdwCLqok/TdAtfQbtEYfLEPysn8ZrXoXqi62olo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=K2nrkKTl; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=j9a3QdvB; arc=none smtp.client-ip=103.168.172.158 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="K2nrkKTl"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="j9a3QdvB" Received: from phl-compute-08.internal (phl-compute-08.phl.internal [10.202.2.48]) by mailfhigh.phl.internal (Postfix) with ESMTP id 7832411401D2; Tue, 26 Nov 2024 01:43:11 -0500 (EST) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-08.internal (MEProxy); Tue, 26 Nov 2024 01:43:11 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm1; t=1732603391; x=1732689791; bh=JpL1DK9kU8ndm+YSA6TWQL5xZVARfwZ9Ykp30gjS3lA=; b= K2nrkKTlzdokfiJQj+Ea2qyKU0gxtM5irZaPcztZJGP1O7lMWji6l5VCar9U8CDR 10DOzcJeVczGzIpvUEJUOSAtnLbL7SzpQRbAKg7KbdudjtwqRfcu8DjHB9/zTLgV a4RV17a9BQvaasi+oQUgHxUsBzc6Qo01VsNqDMM/GH8+/Xc33huAmGMzbs1vlYEh uUTCwfcFzfWRJfpIE0R4YfZDT+royTAbGa0gihEAziihUzDJQwAPZH6PXTcI9B5T XhYUsxJNm/PxPcuooR3Nhlo1IyUtkAE5uILGffNJfmCL2psbwF4M6TzXYQ3Qh1+G 0cKHslE17qTSbDBsTDCkUA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1732603391; x= 1732689791; bh=JpL1DK9kU8ndm+YSA6TWQL5xZVARfwZ9Ykp30gjS3lA=; b=j 9a3QdvBo9mRdA030BCVmjpV64C5fDJKqfWMwcgVEA9GCp5g1l8i9hmFqkDOaVCLM 2TZ+v/qVPN+6t9ajCtH7XbaQXhqtfqL8EBU4PjE0LWPo7ndqDMyQP5PXczsOu9KK dLIW7W6Bhyeszp9HDFD3t1QAKiFH/inLrBPWWALPTBjywUmFqJbOtM6A3E7l8F+y gX3helr40yNlo199eMIaD4zDOambqJgJq58hsDtzVbBO9wNuiAKUnIAw/XsrYiFh be2DWFkDNMVw7YcO5zvwfySY2qCvozC9ziDwuot9Lr7E/P1yvB6k7iqylJkpT59J YGxucjZIDSBB4TSXIlM+w== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefuddrgeeigdeliecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpggftfghnshhusghstghrihgsvgdpuffr tefokffrpgfnqfghnecuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnth hsucdlqddutddtmdenucfjughrpefhfffugggtgffkfhgjvfevofesthejredtredtjeen ucfhrhhomheprfgrthhrihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimh eqnecuggftrfgrthhtvghrnhepffeuiedujedvkeehuedvkeefffeivdeuleetkeduheej teekgedvudfgtdfgieelnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrg hilhhfrhhomhepphhssehpkhhsrdhimhdpnhgspghrtghpthhtohepfedpmhhouggvpehs mhhtphhouhhtpdhrtghpthhtohepghhithhsthgvrhesphhosghogidrtghomhdprhgtph htthhopehgihhtsehvghgvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtohepkhgrrhht hhhikhdrudekkeesghhmrghilhdrtghomh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 26 Nov 2024 01:43:10 -0500 (EST) Received: by vm-mail (OpenSMTPD) with ESMTPSA id caf1e51c (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Tue, 26 Nov 2024 06:42:06 +0000 (UTC) From: Patrick Steinhardt Date: Tue, 26 Nov 2024 07:42:53 +0100 Subject: [PATCH v4 02/10] refs/reftable: handle reloading stacks in the reftable backend Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241126-pks-reftable-backend-reuse-iter-v4-2-b17fd27df126@pks.im> References: <20241126-pks-reftable-backend-reuse-iter-v4-0-b17fd27df126@pks.im> In-Reply-To: <20241126-pks-reftable-backend-reuse-iter-v4-0-b17fd27df126@pks.im> To: git@vger.kernel.org Cc: karthik nayak , Junio C Hamano X-Mailer: b4 0.14.2 When accessing a stack we almost always have to reload the stack before reading data from it. This is mostly because Git does not have a notification mechanism for when underlying data has been changed, and thus we are forced to opportunistically reload the stack every single time to account for any changes that may have happened concurrently. Handle the reload internally in `backend_for()`. For one this forces callsites to think about whether or not they need to reload the stack. But second this makes the logic to access stacks more self-contained by letting the `struct reftable_backend` manage themselves. Update callsites where we don't reload the stack to document why we don't. In some cases it's unclear whether it is the right thing to do in the first place, but fixing that is outside of the scope of this patch series. Signed-off-by: Patrick Steinhardt --- refs/reftable-backend.c | 184 +++++++++++++++++++++++++++++++++--------------- 1 file changed, 126 insertions(+), 58 deletions(-) diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c index acd26f8928d18396f78a2d39ad0e0c1796d5a409..8be9cc43c990212b0b4f8d0b473f84270ff4787d 100644 --- a/refs/reftable-backend.c +++ b/refs/reftable-backend.c @@ -114,21 +114,25 @@ static struct reftable_ref_store *reftable_be_downcast(struct ref_store *ref_sto * like `worktrees/$worktree/refs/heads/foo` as worktree stacks will store * those references in their normalized form. */ -static struct reftable_backend *backend_for(struct reftable_ref_store *store, - const char *refname, - const char **rewritten_ref) +static int backend_for(struct reftable_backend **out, + struct reftable_ref_store *store, + const char *refname, + const char **rewritten_ref, + int reload) { + struct reftable_backend *be; const char *wtname; int wtname_len; - if (!refname) - return &store->main_backend; + if (!refname) { + be = &store->main_backend; + goto out; + } switch (parse_worktree_ref(refname, &wtname, &wtname_len, rewritten_ref)) { case REF_WORKTREE_OTHER: { static struct strbuf wtname_buf = STRBUF_INIT; struct strbuf wt_dir = STRBUF_INIT; - struct reftable_backend *be; /* * We're using a static buffer here so that we don't need to @@ -162,7 +166,7 @@ static struct reftable_backend *backend_for(struct reftable_ref_store *store, } strbuf_release(&wt_dir); - return be; + goto out; } case REF_WORKTREE_CURRENT: /* @@ -170,14 +174,27 @@ static struct reftable_backend *backend_for(struct reftable_ref_store *store, * main worktree. We thus return the main stack in that case. */ if (!store->worktree_backend.stack) - return &store->main_backend; - return &store->worktree_backend; + be = &store->main_backend; + else + be = &store->worktree_backend; + goto out; case REF_WORKTREE_MAIN: case REF_WORKTREE_SHARED: - return &store->main_backend; + be = &store->main_backend; + goto out; default: BUG("unhandled worktree reference type"); } + +out: + if (reload) { + int ret = reftable_stack_reload(be->stack); + if (ret) + return ret; + } + *out = be; + + return 0; } static int should_write_log(struct reftable_ref_store *refs, const char *refname) @@ -828,17 +845,17 @@ static int reftable_be_read_raw_ref(struct ref_store *ref_store, { struct reftable_ref_store *refs = reftable_be_downcast(ref_store, REF_STORE_READ, "read_raw_ref"); - struct reftable_stack *stack = backend_for(refs, refname, &refname)->stack; + struct reftable_backend *be; int ret; if (refs->err < 0) return refs->err; - ret = reftable_stack_reload(stack); + ret = backend_for(&be, refs, refname, &refname, 1); if (ret) return ret; - ret = read_ref_without_reload(refs, stack, refname, oid, referent, type); + ret = read_ref_without_reload(refs, be->stack, refname, oid, referent, type); if (ret < 0) return ret; if (ret > 0) { @@ -855,15 +872,15 @@ static int reftable_be_read_symbolic_ref(struct ref_store *ref_store, { struct reftable_ref_store *refs = reftable_be_downcast(ref_store, REF_STORE_READ, "read_symbolic_ref"); - struct reftable_stack *stack = backend_for(refs, refname, &refname)->stack; struct reftable_ref_record ref = {0}; + struct reftable_backend *be; int ret; - ret = reftable_stack_reload(stack); + ret = backend_for(&be, refs, refname, &refname, 1); if (ret) return ret; - ret = reftable_stack_read_ref(stack, refname, &ref); + ret = reftable_stack_read_ref(be->stack, refname, &ref); if (ret == 0 && ref.value_type == REFTABLE_REF_SYMREF) strbuf_addstr(referent, ref.value.symref); else @@ -880,7 +897,7 @@ struct reftable_transaction_update { struct write_transaction_table_arg { struct reftable_ref_store *refs; - struct reftable_stack *stack; + struct reftable_backend *be; struct reftable_addition *addition; struct reftable_transaction_update *updates; size_t updates_nr; @@ -915,27 +932,37 @@ static int prepare_transaction_update(struct write_transaction_table_arg **out, struct ref_update *update, struct strbuf *err) { - struct reftable_stack *stack = backend_for(refs, update->refname, NULL)->stack; struct write_transaction_table_arg *arg = NULL; + struct reftable_backend *be; size_t i; int ret; + /* + * This function gets called in a loop, and we don't want to repeatedly + * reload the stack for every single ref update. Instead, we manually + * reload further down in the case where we haven't yet prepared the + * specific `reftable_backend`. + */ + ret = backend_for(&be, refs, update->refname, NULL, 0); + if (ret) + return ret; + /* * Search for a preexisting stack update. If there is one then we add * the update to it, otherwise we set up a new stack update. */ for (i = 0; !arg && i < tx_data->args_nr; i++) - if (tx_data->args[i].stack == stack) + if (tx_data->args[i].be == be) arg = &tx_data->args[i]; if (!arg) { struct reftable_addition *addition; - ret = reftable_stack_reload(stack); + ret = reftable_stack_reload(be->stack); if (ret) return ret; - ret = reftable_stack_new_addition(&addition, stack, + ret = reftable_stack_new_addition(&addition, be->stack, REFTABLE_STACK_NEW_ADDITION_RELOAD); if (ret) { if (ret == REFTABLE_LOCK_ERROR) @@ -947,7 +974,7 @@ static int prepare_transaction_update(struct write_transaction_table_arg **out, tx_data->args_alloc); arg = &tx_data->args[tx_data->args_nr++]; arg->refs = refs; - arg->stack = stack; + arg->be = be; arg->addition = addition; arg->updates = NULL; arg->updates_nr = 0; @@ -1002,6 +1029,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store, struct strbuf referent = STRBUF_INIT, head_referent = STRBUF_INIT; struct string_list affected_refnames = STRING_LIST_INIT_NODUP; struct reftable_transaction_data *tx_data = NULL; + struct reftable_backend *be; struct object_id head_oid; unsigned int head_type = 0; size_t i; @@ -1048,7 +1076,22 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store, goto done; } - ret = read_ref_without_reload(refs, backend_for(refs, "HEAD", NULL)->stack, "HEAD", + /* + * TODO: it's dubious whether we should reload the stack that "HEAD" + * belongs to or not. In theory, it may happen that we only modify + * stacks which are _not_ part of the "HEAD" stack. In that case we + * wouldn't have prepared any transaction for its stack and would not + * have reloaded it, which may mean that it is stale. + * + * On the other hand, reloading that stack without locking it feels + * wrong, too, as the value of "HEAD" could be modified concurrently at + * any point in time. + */ + ret = backend_for(&be, refs, "HEAD", NULL, 0); + if (ret) + goto done; + + ret = read_ref_without_reload(refs, be->stack, "HEAD", &head_oid, &head_referent, &head_type); if (ret < 0) goto done; @@ -1057,10 +1100,18 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store, for (i = 0; i < transaction->nr; i++) { struct ref_update *u = transaction->updates[i]; struct object_id current_oid = {0}; - struct reftable_stack *stack; const char *rewritten_ref; - stack = backend_for(refs, u->refname, &rewritten_ref)->stack; + /* + * There is no need to reload the respective backends here as + * we have already reloaded them when preparing the transaction + * update. And given that the stacks have been locked there + * shouldn't have been any concurrent modifications of the + * stack. + */ + ret = backend_for(&be, refs, u->refname, &rewritten_ref, 0); + if (ret) + goto done; /* Verify that the new object ID is valid. */ if ((u->flags & REF_HAVE_NEW) && !is_null_oid(&u->new_oid) && @@ -1116,7 +1167,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store, string_list_insert(&affected_refnames, new_update->refname); } - ret = read_ref_without_reload(refs, stack, rewritten_ref, + ret = read_ref_without_reload(refs, be->stack, rewritten_ref, ¤t_oid, &referent, &u->type); if (ret < 0) goto done; @@ -1318,7 +1369,7 @@ static int transaction_update_cmp(const void *a, const void *b) static int write_transaction_table(struct reftable_writer *writer, void *cb_data) { struct write_transaction_table_arg *arg = cb_data; - uint64_t ts = reftable_stack_next_update_index(arg->stack); + uint64_t ts = reftable_stack_next_update_index(arg->be->stack); struct reftable_log_record *logs = NULL; struct ident_split committer_ident = {0}; size_t logs_nr = 0, logs_alloc = 0, i; @@ -1354,7 +1405,7 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data struct reftable_log_record log = {0}; struct reftable_iterator it = {0}; - ret = reftable_stack_init_log_iterator(arg->stack, &it); + ret = reftable_stack_init_log_iterator(arg->be->stack, &it); if (ret < 0) goto done; @@ -1799,10 +1850,9 @@ static int reftable_be_rename_ref(struct ref_store *ref_store, { struct reftable_ref_store *refs = reftable_be_downcast(ref_store, REF_STORE_WRITE, "rename_ref"); - struct reftable_stack *stack = backend_for(refs, newrefname, &newrefname)->stack; + struct reftable_backend *be; struct write_copy_arg arg = { .refs = refs, - .stack = stack, .oldname = oldrefname, .newname = newrefname, .logmsg = logmsg, @@ -1814,10 +1864,11 @@ static int reftable_be_rename_ref(struct ref_store *ref_store, if (ret < 0) goto done; - ret = reftable_stack_reload(stack); + ret = backend_for(&be, refs, newrefname, &newrefname, 1); if (ret) goto done; - ret = reftable_stack_add(stack, &write_copy_table, &arg); + arg.stack = be->stack; + ret = reftable_stack_add(be->stack, &write_copy_table, &arg); done: assert(ret != REFTABLE_API_ERROR); @@ -1831,10 +1882,9 @@ static int reftable_be_copy_ref(struct ref_store *ref_store, { struct reftable_ref_store *refs = reftable_be_downcast(ref_store, REF_STORE_WRITE, "copy_ref"); - struct reftable_stack *stack = backend_for(refs, newrefname, &newrefname)->stack; + struct reftable_backend *be; struct write_copy_arg arg = { .refs = refs, - .stack = stack, .oldname = oldrefname, .newname = newrefname, .logmsg = logmsg, @@ -1845,10 +1895,11 @@ static int reftable_be_copy_ref(struct ref_store *ref_store, if (ret < 0) goto done; - ret = reftable_stack_reload(stack); + ret = backend_for(&be, refs, newrefname, &newrefname, 1); if (ret) goto done; - ret = reftable_stack_add(stack, &write_copy_table, &arg); + arg.stack = be->stack; + ret = reftable_stack_add(be->stack, &write_copy_table, &arg); done: assert(ret != REFTABLE_API_ERROR); @@ -2012,15 +2063,23 @@ static int reftable_be_for_each_reflog_ent_reverse(struct ref_store *ref_store, { struct reftable_ref_store *refs = reftable_be_downcast(ref_store, REF_STORE_READ, "for_each_reflog_ent_reverse"); - struct reftable_stack *stack = backend_for(refs, refname, &refname)->stack; struct reftable_log_record log = {0}; struct reftable_iterator it = {0}; + struct reftable_backend *be; int ret; if (refs->err < 0) return refs->err; - ret = reftable_stack_init_log_iterator(stack, &it); + /* + * TODO: we should adapt this callsite to reload the stack. There is no + * obvious reason why we shouldn't. + */ + ret = backend_for(&be, refs, refname, &refname, 0); + if (ret) + goto done; + + ret = reftable_stack_init_log_iterator(be->stack, &it); if (ret < 0) goto done; @@ -2052,16 +2111,24 @@ static int reftable_be_for_each_reflog_ent(struct ref_store *ref_store, { struct reftable_ref_store *refs = reftable_be_downcast(ref_store, REF_STORE_READ, "for_each_reflog_ent"); - struct reftable_stack *stack = backend_for(refs, refname, &refname)->stack; struct reftable_log_record *logs = NULL; struct reftable_iterator it = {0}; + struct reftable_backend *be; size_t logs_alloc = 0, logs_nr = 0, i; int ret; if (refs->err < 0) return refs->err; - ret = reftable_stack_init_log_iterator(stack, &it); + /* + * TODO: we should adapt this callsite to reload the stack. There is no + * obvious reason why we shouldn't. + */ + ret = backend_for(&be, refs, refname, &refname, 0); + if (ret) + goto done; + + ret = reftable_stack_init_log_iterator(be->stack, &it); if (ret < 0) goto done; @@ -2101,20 +2168,20 @@ static int reftable_be_reflog_exists(struct ref_store *ref_store, { struct reftable_ref_store *refs = reftable_be_downcast(ref_store, REF_STORE_READ, "reflog_exists"); - struct reftable_stack *stack = backend_for(refs, refname, &refname)->stack; struct reftable_log_record log = {0}; struct reftable_iterator it = {0}; + struct reftable_backend *be; int ret; ret = refs->err; if (ret < 0) goto done; - ret = reftable_stack_reload(stack); + ret = backend_for(&be, refs, refname, &refname, 1); if (ret < 0) goto done; - ret = reftable_stack_init_log_iterator(stack, &it); + ret = reftable_stack_init_log_iterator(be->stack, &it); if (ret < 0) goto done; @@ -2186,10 +2253,9 @@ static int reftable_be_create_reflog(struct ref_store *ref_store, { struct reftable_ref_store *refs = reftable_be_downcast(ref_store, REF_STORE_WRITE, "create_reflog"); - struct reftable_stack *stack = backend_for(refs, refname, &refname)->stack; + struct reftable_backend *be; struct write_reflog_existence_arg arg = { .refs = refs, - .stack = stack, .refname = refname, }; int ret; @@ -2198,11 +2264,12 @@ static int reftable_be_create_reflog(struct ref_store *ref_store, if (ret < 0) goto done; - ret = reftable_stack_reload(stack); + ret = backend_for(&be, refs, refname, &refname, 1); if (ret) goto done; + arg.stack = be->stack; - ret = reftable_stack_add(stack, &write_reflog_existence_table, &arg); + ret = reftable_stack_add(be->stack, &write_reflog_existence_table, &arg); done: return ret; @@ -2260,17 +2327,18 @@ static int reftable_be_delete_reflog(struct ref_store *ref_store, { struct reftable_ref_store *refs = reftable_be_downcast(ref_store, REF_STORE_WRITE, "delete_reflog"); - struct reftable_stack *stack = backend_for(refs, refname, &refname)->stack; + struct reftable_backend *be; struct write_reflog_delete_arg arg = { - .stack = stack, .refname = refname, }; int ret; - ret = reftable_stack_reload(stack); + ret = backend_for(&be, refs, refname, &refname, 1); if (ret) return ret; - ret = reftable_stack_add(stack, &write_reflog_delete_table, &arg); + arg.stack = be->stack; + + ret = reftable_stack_add(be->stack, &write_reflog_delete_table, &arg); assert(ret != REFTABLE_API_ERROR); return ret; @@ -2369,13 +2437,13 @@ static int reftable_be_reflog_expire(struct ref_store *ref_store, */ struct reftable_ref_store *refs = reftable_be_downcast(ref_store, REF_STORE_WRITE, "reflog_expire"); - struct reftable_stack *stack = backend_for(refs, refname, &refname)->stack; struct reftable_log_record *logs = NULL; struct reftable_log_record *rewritten = NULL; struct reftable_ref_record ref_record = {0}; struct reftable_iterator it = {0}; struct reftable_addition *add = NULL; struct reflog_expiry_arg arg = {0}; + struct reftable_backend *be; struct object_id oid = {0}; uint8_t *last_hash = NULL; size_t logs_nr = 0, logs_alloc = 0, i; @@ -2384,11 +2452,11 @@ static int reftable_be_reflog_expire(struct ref_store *ref_store, if (refs->err < 0) return refs->err; - ret = reftable_stack_reload(stack); + ret = backend_for(&be, refs, refname, &refname, 1); if (ret < 0) goto done; - ret = reftable_stack_init_log_iterator(stack, &it); + ret = reftable_stack_init_log_iterator(be->stack, &it); if (ret < 0) goto done; @@ -2396,11 +2464,11 @@ static int reftable_be_reflog_expire(struct ref_store *ref_store, if (ret < 0) goto done; - ret = reftable_stack_new_addition(&add, stack, 0); + ret = reftable_stack_new_addition(&add, be->stack, 0); if (ret < 0) goto done; - ret = reftable_stack_read_ref(stack, refname, &ref_record); + ret = reftable_stack_read_ref(be->stack, refname, &ref_record); if (ret < 0) goto done; if (reftable_ref_record_val1(&ref_record)) @@ -2479,8 +2547,8 @@ static int reftable_be_reflog_expire(struct ref_store *ref_store, arg.refs = refs; arg.records = rewritten; arg.len = logs_nr; - arg.stack = stack, - arg.refname = refname, + arg.stack = be->stack; + arg.refname = refname; ret = reftable_addition_add(add, &write_reflog_expiry_table, &arg); if (ret < 0) From patchwork Tue Nov 26 06:42:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13885502 Received: from fhigh-a7-smtp.messagingengine.com (fhigh-a7-smtp.messagingengine.com [103.168.172.158]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6B9B4180A80 for ; Tue, 26 Nov 2024 06:43:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.158 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732603394; cv=none; b=jLX0kh02UFu8QsQMu53DWc9T8+VMdQZW0F+TCKGr6RAxaOkfl/P7tR9QBCDrVYEei+BZgNpa4zXBHdR1oTFNFKBmIbogAvj/GSh1ur+yG8En1AGgl250JpirvgSqGU/V38OjEJ/GajaWw3jGXKesRzukwdjmsUC/fwN5qcwKUAI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732603394; c=relaxed/simple; bh=caHEs5YS+sovNq5SdAZZYy8gNO5YDOU03xMLNEbe+g0=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=n1XR/yghigeNSAUWplbSXWDfRGdEtdvPpcujPIfkWf2YyvIoIIREstpnNDqU1PbAF4PJdbu80pXKjKFXy0twTFEHOpbR2Gd4G98NW8v5k0cGqCk5I8HMFvxVpibTnYlhFr+pQ8lTAOI6KBX9QmYSZsMD+JbrV0HPQz6m7ZX6/MU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=E4U6AXs9; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=HbvfG5/s; arc=none smtp.client-ip=103.168.172.158 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="E4U6AXs9"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="HbvfG5/s" Received: from phl-compute-01.internal (phl-compute-01.phl.internal [10.202.2.41]) by mailfhigh.phl.internal (Postfix) with ESMTP id 85BCA11401C6; Tue, 26 Nov 2024 01:43:11 -0500 (EST) Received: from phl-mailfrontend-02 ([10.202.2.163]) by phl-compute-01.internal (MEProxy); Tue, 26 Nov 2024 01:43:11 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm1; t=1732603391; x=1732689791; bh=f3l+ZSUHDHw7zaVisfjA/AS4PEEsPrmN+yAyIINkxmg=; b= E4U6AXs9eGbF+zk4ZQ7O3x7OXgOW2LSZ+DUUYalLlrMNipmnPitTx80SMrr/WoMH 6S5FxqsmBvc7Guf+1WqGjfD9dsgM3XnRoS9yhn93ULgtwXERDi3+Q9Ojx1aZ5eMh 7byqLnAdW/QIt/NBjzxaqilfUomVQn6T5WyFSTO1MzClWktSaQ4ZSCo7piU0cRML OHU0OX2O+TT19GjFvPMGXkLSItxLB4c9sfeGcnA/1mRXpdEyE9hid+sb9NyC+sRs t3eEQmmp1ITkz0mbu4N5pEfSr/+SD3N+uWVsIZUgQbkQRI1qbH1p4vprFSknxmQP oioW5mhLoGfgck2wLTlJlQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1732603391; x= 1732689791; bh=f3l+ZSUHDHw7zaVisfjA/AS4PEEsPrmN+yAyIINkxmg=; b=H bvfG5/st7RNGie9tqMOwcfw7YSvlhcALJhyIqkUXVoTSQxlDaA+THx25Ehn+XKW6 Iqf4x8zIDQoRm7eAyliNcFZn+nsnBFEydIrVytJ9ge5EKzpuH+nlJ+d8Nj9O7zRl oTDl9DdxysSTnBIYeCbhev/pGiRuteyhL3GKzr4ELXPa7YYdGb/n3JPJ5JebzZ4O FiCXSgmRw+oGEp931/AgVO6wcuDCvPPRgh3y8EJO7gG/Qjt48CFPVFfvZTb7JxvL NQ/so30DWR3waFqXCaR0lXFL1VC27wPkXrgxJeLT6M3p1Bg0WzSEGoQ7TH1bV8Sq SUDukQ4M0Uf8hx9XXlWyA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefuddrgeeigdeliecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpggftfghnshhusghstghrihgsvgdpuffr tefokffrpgfnqfghnecuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnth hsucdlqddutddtmdenucfjughrpefhfffugggtgffkfhgjvfevofesthejredtredtjeen ucfhrhhomheprfgrthhrihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimh eqnecuggftrfgrthhtvghrnhepffeuiedujedvkeehuedvkeefffeivdeuleetkeduheej teekgedvudfgtdfgieelnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrg hilhhfrhhomhepphhssehpkhhsrdhimhdpnhgspghrtghpthhtohepfedpmhhouggvpehs mhhtphhouhhtpdhrtghpthhtohepghhithesvhhgvghrrdhkvghrnhgvlhdrohhrghdprh gtphhtthhopehgihhtshhtvghrsehpohgsohigrdgtohhmpdhrtghpthhtohepkhgrrhht hhhikhdrudekkeesghhmrghilhdrtghomh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 26 Nov 2024 01:43:10 -0500 (EST) Received: by vm-mail (OpenSMTPD) with ESMTPSA id bee29068 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Tue, 26 Nov 2024 06:42:07 +0000 (UTC) From: Patrick Steinhardt Date: Tue, 26 Nov 2024 07:42:54 +0100 Subject: [PATCH v4 03/10] reftable/stack: add accessor for the hash ID Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241126-pks-reftable-backend-reuse-iter-v4-3-b17fd27df126@pks.im> References: <20241126-pks-reftable-backend-reuse-iter-v4-0-b17fd27df126@pks.im> In-Reply-To: <20241126-pks-reftable-backend-reuse-iter-v4-0-b17fd27df126@pks.im> To: git@vger.kernel.org Cc: karthik nayak , Junio C Hamano X-Mailer: b4 0.14.2 Add an accessor function that allows callers to access the hash ID of a reftable stack. This function will be used in a subsequent commit. Signed-off-by: Patrick Steinhardt --- reftable/reftable-stack.h | 3 +++ reftable/stack.c | 5 +++++ 2 files changed, 8 insertions(+) diff --git a/reftable/reftable-stack.h b/reftable/reftable-stack.h index 54787f2ef530406a7970db058c3a0cf456897978..ae14270ea74108cd4c314ec38e7d5c9a4e731481 100644 --- a/reftable/reftable-stack.h +++ b/reftable/reftable-stack.h @@ -149,4 +149,7 @@ struct reftable_compaction_stats { struct reftable_compaction_stats * reftable_stack_compaction_stats(struct reftable_stack *st); +/* Return the hash of the stack. */ +enum reftable_hash reftable_stack_hash_id(struct reftable_stack *st); + #endif diff --git a/reftable/stack.c b/reftable/stack.c index 1fffd75630266c0d3d1e4a2d037b5cf90455529d..d97b64a40d4ad05cfd9e6f33e8ba1e713281ef6d 100644 --- a/reftable/stack.c +++ b/reftable/stack.c @@ -1791,3 +1791,8 @@ int reftable_stack_clean(struct reftable_stack *st) reftable_addition_destroy(add); return err; } + +enum reftable_hash reftable_stack_hash_id(struct reftable_stack *st) +{ + return reftable_merged_table_hash_id(st->merged); +} From patchwork Tue Nov 26 06:42:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13885506 Received: from fhigh-a7-smtp.messagingengine.com (fhigh-a7-smtp.messagingengine.com [103.168.172.158]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 814801885A5 for ; Tue, 26 Nov 2024 06:43:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.158 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732603395; cv=none; b=BNtOVD+a1GdinUuHrtCFwkD4c5pK51oQphMAs7A9vttBJ7FstLkqqjwruhR03dVfscTJWDX4QFAjgXalsskT0Eo5xZFZvTQjBkVfZRsteibC0HqgAg4zKeAgJ4y8tkc91SqBQgVJyrmP6amupOxvOtGBf8gsd7Q1gRJsiyKQbv4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732603395; c=relaxed/simple; bh=RFo09bCeKwC1AMnDNbDWDRKTr8sVVUaA8jbeX9CH+IQ=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=aIzqUmDPcnp1VXcRQLVZEwvn3wpgZWnlvbl6Rape3oDajI6IHVBT4tN/gyJqQfkLCNn6rNUzn5b2J1LZOtZlqdPrn5Wt+FJAnV8xBBgbd7WJprRE31Ca5lmq7gnsvlZr26EVFGc9vbthkAAK+AXLJ0a6VPgFw0kzEn1yHWPV4XU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=rNZPklbq; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=Uh2ayCXA; arc=none smtp.client-ip=103.168.172.158 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="rNZPklbq"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="Uh2ayCXA" Received: from phl-compute-11.internal (phl-compute-11.phl.internal [10.202.2.51]) by mailfhigh.phl.internal (Postfix) with ESMTP id B0E9811401CA; Tue, 26 Nov 2024 01:43:12 -0500 (EST) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-11.internal (MEProxy); Tue, 26 Nov 2024 01:43:12 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm1; t=1732603392; x=1732689792; bh=jFhBmt/arKudzy7yUBrJ0sfTMcPZZePIlJUtd9irujI=; b= rNZPklbq130aZIfcLQBWda53VRNfmCjXJkWTRDuLkJjlKxE9n6ANVo7zK0nMRBdp Edd1s3lvC5QFtOhUVHENLp7a+kO2ATxOfkbBhoAaosdoceXV3zXfxZw/6FjCxYde KOkwlXYM4eXsZLmHY+y33RpVZ6g62FUpAh+MIBiHUWDaShJWuKUBJiyiwlUsuqMP PXqOkCvaALTPw2pvQHHLgXCBBSzCuOsR8WGDN6UPzj4tiLjqkjzDFnSu/db5cm9a WH0y6PgafJjUgqPkXPTkEfpep23PmdwBMxgzw5Vpg5QFo1sdwcJNHkZtwxFUE0DN THz691b2QOHa9mvExYlohg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1732603392; x= 1732689792; bh=jFhBmt/arKudzy7yUBrJ0sfTMcPZZePIlJUtd9irujI=; b=U h2ayCXAtJY7ZBXlSSuu9PAZ4yCzTN7xEgP6Tu97YDzCU3HK3Dbd6y6fITIaq7KXf KRVJPV6FRuMmH+rwFANEjnTEvs+ndOuvDpRlV91cpPtArjbfTWohusloo4ODeDRo kKxeSCJRZDfne9ndWxWaliGPdpid6W9fjkwb3OtP07LDIrpy4ZHZdKqFK3BijVBS 8AOZJgowZbBisjlRRkNdpcKnyR/dsGVrWtxbhyu531LjF+Xs6zsoWUUl2aNpzSqw +r+5L5FghIHVF1rgI5THHPggj46Ry2W+mjKzPiqjbz6ogQMTmDHm7wUzNWGb4guu Tx4Z9FMgc8Ios9uVDZu0g== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefuddrgeeigdeliecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpggftfghnshhusghstghrihgsvgdpuffr tefokffrpgfnqfghnecuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnth hsucdlqddutddtmdenucfjughrpefhfffugggtgffkfhgjvfevofesthejredtredtjeen ucfhrhhomheprfgrthhrihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimh eqnecuggftrfgrthhtvghrnhephfefieevffduhfejvdevfffhudekhfdvkeevveegueei teetveeglefghfevtdfgnecuffhomhgrihhnpehuphgurghtvgdrthiinecuvehluhhsth gvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimhdp nhgspghrtghpthhtohepfedpmhhouggvpehsmhhtphhouhhtpdhrtghpthhtohepghhith esvhhgvghrrdhkvghrnhgvlhdrohhrghdprhgtphhtthhopehgihhtshhtvghrsehpohgs ohigrdgtohhmpdhrtghpthhtohepkhgrrhhthhhikhdrudekkeesghhmrghilhdrtghomh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 26 Nov 2024 01:43:11 -0500 (EST) Received: by vm-mail (OpenSMTPD) with ESMTPSA id 58ca4687 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Tue, 26 Nov 2024 06:42:08 +0000 (UTC) From: Patrick Steinhardt Date: Tue, 26 Nov 2024 07:42:55 +0100 Subject: [PATCH v4 04/10] refs/reftable: figure out hash via `reftable_stack` Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241126-pks-reftable-backend-reuse-iter-v4-4-b17fd27df126@pks.im> References: <20241126-pks-reftable-backend-reuse-iter-v4-0-b17fd27df126@pks.im> In-Reply-To: <20241126-pks-reftable-backend-reuse-iter-v4-0-b17fd27df126@pks.im> To: git@vger.kernel.org Cc: karthik nayak , Junio C Hamano X-Mailer: b4 0.14.2 The function `read_ref_without_reload()` accepts a ref store as input only so that we can figure out the hash function used by it. This is duplicate information though because the reftable stack knows about its hash function, too. Drop the superfluous parameter to simplify the calling convention a bit. Signed-off-by: Patrick Steinhardt --- refs/reftable-backend.c | 26 +++++++++++++++++++------- 1 file changed, 19 insertions(+), 7 deletions(-) diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c index 8be9cc43c990212b0b4f8d0b473f84270ff4787d..302de622d44ccbe8eceee0654f11ad7184791eba 100644 --- a/refs/reftable-backend.c +++ b/refs/reftable-backend.c @@ -243,8 +243,7 @@ static void fill_reftable_log_record(struct reftable_log_record *log, const stru log->value.update.tz_offset = sign * atoi(tz_begin); } -static int read_ref_without_reload(struct reftable_ref_store *refs, - struct reftable_stack *stack, +static int read_ref_without_reload(struct reftable_stack *stack, const char *refname, struct object_id *oid, struct strbuf *referent, @@ -262,8 +261,21 @@ static int read_ref_without_reload(struct reftable_ref_store *refs, strbuf_addstr(referent, ref.value.symref); *type |= REF_ISSYMREF; } else if (reftable_ref_record_val1(&ref)) { + unsigned int hash_id; + + switch (reftable_stack_hash_id(stack)) { + case REFTABLE_HASH_SHA1: + hash_id = GIT_HASH_SHA1; + break; + case REFTABLE_HASH_SHA256: + hash_id = GIT_HASH_SHA256; + break; + default: + BUG("unhandled hash ID %d", reftable_stack_hash_id(stack)); + } + oidread(oid, reftable_ref_record_val1(&ref), - refs->base.repo->hash_algo); + &hash_algos[hash_id]); } else { /* We got a tombstone, which should not happen. */ BUG("unhandled reference value type %d", ref.value_type); @@ -855,7 +867,7 @@ static int reftable_be_read_raw_ref(struct ref_store *ref_store, if (ret) return ret; - ret = read_ref_without_reload(refs, be->stack, refname, oid, referent, type); + ret = read_ref_without_reload(be->stack, refname, oid, referent, type); if (ret < 0) return ret; if (ret > 0) { @@ -1091,7 +1103,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store, if (ret) goto done; - ret = read_ref_without_reload(refs, be->stack, "HEAD", + ret = read_ref_without_reload(be->stack, "HEAD", &head_oid, &head_referent, &head_type); if (ret < 0) goto done; @@ -1167,7 +1179,7 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store, string_list_insert(&affected_refnames, new_update->refname); } - ret = read_ref_without_reload(refs, be->stack, rewritten_ref, + ret = read_ref_without_reload(be->stack, rewritten_ref, ¤t_oid, &referent, &u->type); if (ret < 0) goto done; @@ -1733,7 +1745,7 @@ static int write_copy_table(struct reftable_writer *writer, void *cb_data) memcpy(logs[logs_nr].value.update.old_hash, old_ref.value.val1, GIT_MAX_RAWSZ); logs_nr++; - ret = read_ref_without_reload(arg->refs, arg->stack, "HEAD", &head_oid, + ret = read_ref_without_reload(arg->stack, "HEAD", &head_oid, &head_referent, &head_type); if (ret < 0) goto done; From patchwork Tue Nov 26 06:42:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13885505 Received: from fhigh-a7-smtp.messagingengine.com (fhigh-a7-smtp.messagingengine.com [103.168.172.158]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 814FE1885B8 for ; Tue, 26 Nov 2024 06:43:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.158 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732603395; cv=none; b=NS1nD+Cz6w9Xw6WCz4TektYaClsB20RfgtsPOPLveheZJs9hMsyUkW+FGIDTQL3kDw26AT8cbxR6xTxNSUVBClmwk6LZNc1TSKdJvsMZo90Hfbgs1/73KsoWh0dVJh/X3KBjcdmDxdiWhUiajMDUOHYyB9VlBnw2Y0JMuFOSgpM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732603395; c=relaxed/simple; bh=vMaXWD6ttWCSMPHXBiy/O+nxK4ISvrAJ2Md30isUYzg=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=dvSKs0zhSqkm0U/zjcPDswvhVM6+Q0B8KZLRA68cKXTQT8zBqLD/pMgTYJNv/gTUFRmfcFB7A4Q+Yd6VGvIVBi8Gc1U4ma56GDOn7IlSYICTIRRDt7ahK1rUdDL3vQeRyGB7zfpHa19GaN5s/5YEWMbwn84s69xi8YiPtV91Adw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=PexhGmkJ; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=aqdTmw93; arc=none smtp.client-ip=103.168.172.158 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="PexhGmkJ"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="aqdTmw93" Received: from phl-compute-04.internal (phl-compute-04.phl.internal [10.202.2.44]) by mailfhigh.phl.internal (Postfix) with ESMTP id C462911401DB; Tue, 26 Nov 2024 01:43:12 -0500 (EST) Received: from phl-mailfrontend-02 ([10.202.2.163]) by phl-compute-04.internal (MEProxy); Tue, 26 Nov 2024 01:43:12 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm1; t=1732603392; x=1732689792; bh=rrpKqUUtgI5A79upr2He4K8JA4XHxs+3PqRNii+aRuA=; b= PexhGmkJKyiKw8PhHroxN0Beey1fIher8lpYhRrZGTwpmJw8Qhz2K+oC/mvRmhN9 UAOZlq4xT84e+WC8fNBBQCkMTP4kDSumYuSEcwBa+kMwf3433iUIu8WQrZo8aSzx VBFs2+WlGcSkLY3A17ICT7YDKxVDdW1V6Tp5HdXuOX97dBnn4GfBigR69mPJCs4s fULFh6cTXSrj3g1Lxy0fLywKlO1tpsf9Gt1/dgw6m8p2O2BxBDvfWinIMQKR7uk7 Q3mvMoZj5w0F2hYKwmVAt0+iRpwXDfxPnIpHfehwNz/mVujzgtzRbVgkjberHL0L rc7gJ2/E6UexgKrLIyXaEg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1732603392; x= 1732689792; bh=rrpKqUUtgI5A79upr2He4K8JA4XHxs+3PqRNii+aRuA=; b=a qdTmw931GUfL8WOp8wRPd6xPp8diNHLaG0OgAAvgNZhR329G8Ligle21NVhIj5cc tR7CpFBum4lu7GJSYFiRTieUfrp11t2MXUJ0SmqnYeSBBZ92ptauKOZP17pBQHr6 ny62Y/DPvMJ37qYPHGP9hyVmBxEUU4WTTmSicyagzIaCtnszPK9axK3SMyFkDv93 /JJOz+4X6wV37aHfGV+dckXucFNOxIde3T+GWZTGm8MPY5JzsCGKAbAAy5nesJKC 3q/Cb1i5gtr/PXDxZUsACo369xMjpR3rqUE/2pBdzKOnl80fMvP3OVbXjMjNAToh qbTNiBZpX4NeHAG2UUfTw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefuddrgeeigdelhecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpggftfghnshhusghstghrihgsvgdpuffr tefokffrpgfnqfghnecuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnth hsucdlqddutddtmdenucfjughrpefhfffugggtgffkfhgjvfevofesthejredtredtjeen ucfhrhhomheprfgrthhrihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimh eqnecuggftrfgrthhtvghrnhephfefieevffduhfejvdevfffhudekhfdvkeevveegueei teetveeglefghfevtdfgnecuffhomhgrihhnpehuphgurghtvgdrthiinecuvehluhhsth gvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimhdp nhgspghrtghpthhtohepfedpmhhouggvpehsmhhtphhouhhtpdhrtghpthhtohepghhith hsthgvrhesphhosghogidrtghomhdprhgtphhtthhopehgihhtsehvghgvrhdrkhgvrhhn vghlrdhorhhgpdhrtghpthhtohepkhgrrhhthhhikhdrudekkeesghhmrghilhdrtghomh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 26 Nov 2024 01:43:11 -0500 (EST) Received: by vm-mail (OpenSMTPD) with ESMTPSA id 1c3a0c50 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Tue, 26 Nov 2024 06:42:08 +0000 (UTC) From: Patrick Steinhardt Date: Tue, 26 Nov 2024 07:42:56 +0100 Subject: [PATCH v4 05/10] refs/reftable: read references via `struct reftable_backend` Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241126-pks-reftable-backend-reuse-iter-v4-5-b17fd27df126@pks.im> References: <20241126-pks-reftable-backend-reuse-iter-v4-0-b17fd27df126@pks.im> In-Reply-To: <20241126-pks-reftable-backend-reuse-iter-v4-0-b17fd27df126@pks.im> To: git@vger.kernel.org Cc: karthik nayak , Junio C Hamano X-Mailer: b4 0.14.2 Refactor `read_ref_without_reload()` to accept `struct reftable_backend` as parameter instead of `struct reftable_stack`. Rename the function to `reftable_backend_read_ref()` to clarify its scope and move it close to other functions operating on `struct reftable_backend`. This change allows us to implement an additional caching layer when reading refs where we can reuse reftable iterators. Signed-off-by: Patrick Steinhardt --- refs/reftable-backend.c | 122 +++++++++++++++++++++++------------------------- 1 file changed, 59 insertions(+), 63 deletions(-) diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c index 302de622d44ccbe8eceee0654f11ad7184791eba..88910207b87b1fa5bbbedebc8817f8afab937b77 100644 --- a/refs/reftable-backend.c +++ b/refs/reftable-backend.c @@ -51,6 +51,50 @@ static void reftable_backend_release(struct reftable_backend *be) be->stack = NULL; } +static int reftable_backend_read_ref(struct reftable_backend *be, + const char *refname, + struct object_id *oid, + struct strbuf *referent, + unsigned int *type) +{ + struct reftable_ref_record ref = {0}; + int ret; + + ret = reftable_stack_read_ref(be->stack, refname, &ref); + if (ret) + goto done; + + if (ref.value_type == REFTABLE_REF_SYMREF) { + strbuf_reset(referent); + strbuf_addstr(referent, ref.value.symref); + *type |= REF_ISSYMREF; + } else if (reftable_ref_record_val1(&ref)) { + unsigned int hash_id; + + switch (reftable_stack_hash_id(be->stack)) { + case REFTABLE_HASH_SHA1: + hash_id = GIT_HASH_SHA1; + break; + case REFTABLE_HASH_SHA256: + hash_id = GIT_HASH_SHA256; + break; + default: + BUG("unhandled hash ID %d", reftable_stack_hash_id(be->stack)); + } + + oidread(oid, reftable_ref_record_val1(&ref), + &hash_algos[hash_id]); + } else { + /* We got a tombstone, which should not happen. */ + BUG("unhandled reference value type %d", ref.value_type); + } + +done: + assert(ret != REFTABLE_API_ERROR); + reftable_ref_record_release(&ref); + return ret; +} + struct reftable_ref_store { struct ref_store base; @@ -243,50 +287,6 @@ static void fill_reftable_log_record(struct reftable_log_record *log, const stru log->value.update.tz_offset = sign * atoi(tz_begin); } -static int read_ref_without_reload(struct reftable_stack *stack, - const char *refname, - struct object_id *oid, - struct strbuf *referent, - unsigned int *type) -{ - struct reftable_ref_record ref = {0}; - int ret; - - ret = reftable_stack_read_ref(stack, refname, &ref); - if (ret) - goto done; - - if (ref.value_type == REFTABLE_REF_SYMREF) { - strbuf_reset(referent); - strbuf_addstr(referent, ref.value.symref); - *type |= REF_ISSYMREF; - } else if (reftable_ref_record_val1(&ref)) { - unsigned int hash_id; - - switch (reftable_stack_hash_id(stack)) { - case REFTABLE_HASH_SHA1: - hash_id = GIT_HASH_SHA1; - break; - case REFTABLE_HASH_SHA256: - hash_id = GIT_HASH_SHA256; - break; - default: - BUG("unhandled hash ID %d", reftable_stack_hash_id(stack)); - } - - oidread(oid, reftable_ref_record_val1(&ref), - &hash_algos[hash_id]); - } else { - /* We got a tombstone, which should not happen. */ - BUG("unhandled reference value type %d", ref.value_type); - } - -done: - assert(ret != REFTABLE_API_ERROR); - reftable_ref_record_release(&ref); - return ret; -} - static int reftable_be_config(const char *var, const char *value, const struct config_context *ctx, void *_opts) @@ -867,7 +867,7 @@ static int reftable_be_read_raw_ref(struct ref_store *ref_store, if (ret) return ret; - ret = read_ref_without_reload(be->stack, refname, oid, referent, type); + ret = reftable_backend_read_ref(be, refname, oid, referent, type); if (ret < 0) return ret; if (ret > 0) { @@ -1103,8 +1103,8 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store, if (ret) goto done; - ret = read_ref_without_reload(be->stack, "HEAD", - &head_oid, &head_referent, &head_type); + ret = reftable_backend_read_ref(be, "HEAD", &head_oid, + &head_referent, &head_type); if (ret < 0) goto done; ret = 0; @@ -1179,8 +1179,8 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store, string_list_insert(&affected_refnames, new_update->refname); } - ret = read_ref_without_reload(be->stack, rewritten_ref, - ¤t_oid, &referent, &u->type); + ret = reftable_backend_read_ref(be, rewritten_ref, + ¤t_oid, &referent, &u->type); if (ret < 0) goto done; if (ret > 0 && !ref_update_expects_existing_old_ref(u)) { @@ -1638,7 +1638,7 @@ struct write_create_symref_arg { struct write_copy_arg { struct reftable_ref_store *refs; - struct reftable_stack *stack; + struct reftable_backend *be; const char *oldname; const char *newname; const char *logmsg; @@ -1663,7 +1663,7 @@ static int write_copy_table(struct reftable_writer *writer, void *cb_data) if (split_ident_line(&committer_ident, committer_info, strlen(committer_info))) BUG("failed splitting committer info"); - if (reftable_stack_read_ref(arg->stack, arg->oldname, &old_ref)) { + if (reftable_stack_read_ref(arg->be->stack, arg->oldname, &old_ref)) { ret = error(_("refname %s not found"), arg->oldname); goto done; } @@ -1702,7 +1702,7 @@ static int write_copy_table(struct reftable_writer *writer, void *cb_data) * the old branch and the creation of the new branch, and we cannot do * two changes to a reflog in a single update. */ - deletion_ts = creation_ts = reftable_stack_next_update_index(arg->stack); + deletion_ts = creation_ts = reftable_stack_next_update_index(arg->be->stack); if (arg->delete_old) creation_ts++; reftable_writer_set_limits(writer, deletion_ts, creation_ts); @@ -1745,8 +1745,8 @@ static int write_copy_table(struct reftable_writer *writer, void *cb_data) memcpy(logs[logs_nr].value.update.old_hash, old_ref.value.val1, GIT_MAX_RAWSZ); logs_nr++; - ret = read_ref_without_reload(arg->stack, "HEAD", &head_oid, - &head_referent, &head_type); + ret = reftable_backend_read_ref(arg->be, "HEAD", &head_oid, + &head_referent, &head_type); if (ret < 0) goto done; append_head_reflog = (head_type & REF_ISSYMREF) && !strcmp(head_referent.buf, arg->oldname); @@ -1789,7 +1789,7 @@ static int write_copy_table(struct reftable_writer *writer, void *cb_data) * copy over all log entries from the old reflog. Last but not least, * when renaming we also have to delete all the old reflog entries. */ - ret = reftable_stack_init_log_iterator(arg->stack, &it); + ret = reftable_stack_init_log_iterator(arg->be->stack, &it); if (ret < 0) goto done; @@ -1862,7 +1862,6 @@ static int reftable_be_rename_ref(struct ref_store *ref_store, { struct reftable_ref_store *refs = reftable_be_downcast(ref_store, REF_STORE_WRITE, "rename_ref"); - struct reftable_backend *be; struct write_copy_arg arg = { .refs = refs, .oldname = oldrefname, @@ -1876,11 +1875,10 @@ static int reftable_be_rename_ref(struct ref_store *ref_store, if (ret < 0) goto done; - ret = backend_for(&be, refs, newrefname, &newrefname, 1); + ret = backend_for(&arg.be, refs, newrefname, &newrefname, 1); if (ret) goto done; - arg.stack = be->stack; - ret = reftable_stack_add(be->stack, &write_copy_table, &arg); + ret = reftable_stack_add(arg.be->stack, &write_copy_table, &arg); done: assert(ret != REFTABLE_API_ERROR); @@ -1894,7 +1892,6 @@ static int reftable_be_copy_ref(struct ref_store *ref_store, { struct reftable_ref_store *refs = reftable_be_downcast(ref_store, REF_STORE_WRITE, "copy_ref"); - struct reftable_backend *be; struct write_copy_arg arg = { .refs = refs, .oldname = oldrefname, @@ -1907,11 +1904,10 @@ static int reftable_be_copy_ref(struct ref_store *ref_store, if (ret < 0) goto done; - ret = backend_for(&be, refs, newrefname, &newrefname, 1); + ret = backend_for(&arg.be, refs, newrefname, &newrefname, 1); if (ret) goto done; - arg.stack = be->stack; - ret = reftable_stack_add(be->stack, &write_copy_table, &arg); + ret = reftable_stack_add(arg.be->stack, &write_copy_table, &arg); done: assert(ret != REFTABLE_API_ERROR); From patchwork Tue Nov 26 06:42:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13885507 Received: from fhigh-a7-smtp.messagingengine.com (fhigh-a7-smtp.messagingengine.com [103.168.172.158]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BC4A918FDDC for ; Tue, 26 Nov 2024 06:43:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.158 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732603396; cv=none; b=alw8ibN0lLIH1CZQtKtWFQsdguq2EXFeQM5t0Hy3TxeHXaC3RKVpk7bms3d9dE7LjHBlcecLYsDi6q7Q0ZUG9zDRRrmcVKNnIcpTZku2G/a5I4a5jZeh0ASWinLlrK0sVKuaBgkFiXycKuuQsbuvMsh6Pw4BIrOmoShHt/T7a7M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732603396; c=relaxed/simple; bh=HPm24ebYEPF4LM0/oL1T6M14ASdt/4YWdg93+NLSa9U=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=PtDJ2kuF3cn8Wh3m3SShPZGk063ACElOWxURV9NqYCGuYlVPtMLebBX+tZk9pzJ9kyd20t0R8Y+sJWwliIU/gV4+2dM4yGLVVhao/ABiqOAs3bCX2YLQ2XsQB7pqeXTYGziBx/+agrfW6C/F6PTf04BXyq5nTNbt4mM4njK+J8E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=wfmCDdhB; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=LzvxD+T9; arc=none smtp.client-ip=103.168.172.158 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="wfmCDdhB"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="LzvxD+T9" Received: from phl-compute-01.internal (phl-compute-01.phl.internal [10.202.2.41]) by mailfhigh.phl.internal (Postfix) with ESMTP id DC09211401B8; Tue, 26 Nov 2024 01:43:13 -0500 (EST) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-01.internal (MEProxy); Tue, 26 Nov 2024 01:43:13 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm1; t=1732603393; x=1732689793; bh=ZdkV5Vp5z8qbHtIBvzfRn87LvMoaBaEagc6vMF77zAQ=; b= wfmCDdhB8VJ6zq4wY9ONvMIhfCKNTtutFXor3f5JSYwPR+FkM0zsH0tuuKmv6Ngz 2Nzp/TFdG/9iD8vr66gyRcUSWMi04SORo/9by03DtbvVd1zMwZARyECOaVsm1qN7 uOsxebQEIXtPj2WiAp42xug8VCTL4rE6+juISzPtB4L+bvr5RCd1qLHf+inhnpy0 lShzhKqtsbkvnTzs1pqJwPrUhfHV9oVmR13Axi0Ex9Ob7apfP4yd8zgFEYhtI8fY w7/BoTpJOky2TCgrOhUVefoUY778irlh5mB8RhStysQw1Z3Kt07fpQusiaTFV0GG xGl8xGbdFmYczvUZ+Vjgig== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1732603393; x= 1732689793; bh=ZdkV5Vp5z8qbHtIBvzfRn87LvMoaBaEagc6vMF77zAQ=; b=L zvxD+T95wtuaF1gb1dveDlAJwfwL3vBuYkU9nU+wE5THtQqsgbE8sO1BnkBpK+0M ZPBvgxSTMyuMW4WJ3P5YmchrZ/zk03qcazglKJshy/xipTCWrU/e7EPUWxQIx8w+ Etvg/sKT/K9G2glTmpeGg3e0WRtpN1PUjSrr+afml6erS/hlxO57CaoaLuHCUbcD VAmoSU3WNfybB/4vP3wcV+lP6K3So5KRKb0hepxS7RRHmqHhCv8uG1qe66Nv1Kxr fXcXoMxdHJWKIVHi/ALATMRQ/NgdhuR/XKDj5j8CiqWc/Il+ynPOmZSNBqqbdZwN no/7P6z4I/2yABOC6ehWA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefuddrgeeigdeliecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpggftfghnshhusghstghrihgsvgdpuffr tefokffrpgfnqfghnecuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnth hsucdlqddutddtmdenucfjughrpefhfffugggtgffkfhgjvfevofesthejredtredtjeen ucfhrhhomheprfgrthhrihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimh eqnecuggftrfgrthhtvghrnhepffeuiedujedvkeehuedvkeefffeivdeuleetkeduheej teekgedvudfgtdfgieelnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrg hilhhfrhhomhepphhssehpkhhsrdhimhdpnhgspghrtghpthhtohepfedpmhhouggvpehs mhhtphhouhhtpdhrtghpthhtohepkhgrrhhthhhikhdrudekkeesghhmrghilhdrtghomh dprhgtphhtthhopehgihhtsehvghgvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtohep ghhithhsthgvrhesphhosghogidrtghomh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 26 Nov 2024 01:43:12 -0500 (EST) Received: by vm-mail (OpenSMTPD) with ESMTPSA id e286c882 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Tue, 26 Nov 2024 06:42:09 +0000 (UTC) From: Patrick Steinhardt Date: Tue, 26 Nov 2024 07:42:57 +0100 Subject: [PATCH v4 06/10] refs/reftable: refactor reading symbolic refs to use reftable backend Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241126-pks-reftable-backend-reuse-iter-v4-6-b17fd27df126@pks.im> References: <20241126-pks-reftable-backend-reuse-iter-v4-0-b17fd27df126@pks.im> In-Reply-To: <20241126-pks-reftable-backend-reuse-iter-v4-0-b17fd27df126@pks.im> To: git@vger.kernel.org Cc: karthik nayak , Junio C Hamano X-Mailer: b4 0.14.2 Refactor the callback function that reads symbolic references in the reftable backend to use `reftable_backend_read_ref()` instead of accessing the reftable stack directly. This ensures that the function will benefit from the new caching layer that we're about to introduce. Signed-off-by: Patrick Steinhardt --- refs/reftable-backend.c | 11 ++++------- 1 file changed, 4 insertions(+), 7 deletions(-) diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c index 88910207b87b1fa5bbbedebc8817f8afab937b77..2d06620ac8b1b7f07783cd5873dbe0fe67b84bd6 100644 --- a/refs/reftable-backend.c +++ b/refs/reftable-backend.c @@ -884,21 +884,18 @@ static int reftable_be_read_symbolic_ref(struct ref_store *ref_store, { struct reftable_ref_store *refs = reftable_be_downcast(ref_store, REF_STORE_READ, "read_symbolic_ref"); - struct reftable_ref_record ref = {0}; struct reftable_backend *be; + struct object_id oid; + unsigned int type = 0; int ret; ret = backend_for(&be, refs, refname, &refname, 1); if (ret) return ret; - ret = reftable_stack_read_ref(be->stack, refname, &ref); - if (ret == 0 && ref.value_type == REFTABLE_REF_SYMREF) - strbuf_addstr(referent, ref.value.symref); - else + ret = reftable_backend_read_ref(be, refname, &oid, referent, &type); + if (type != REF_ISSYMREF) ret = -1; - - reftable_ref_record_release(&ref); return ret; } From patchwork Tue Nov 26 06:42:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13885508 Received: from fout-a4-smtp.messagingengine.com (fout-a4-smtp.messagingengine.com [103.168.172.147]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0E71B1917E4 for ; Tue, 26 Nov 2024 06:43:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.147 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732603396; cv=none; b=CbdyC4uZaxaWqYOFPOpmgFHWEr0fJDXlLt0M9jHhVxRROhUEtFXKZd6Y8ERk8pEgPMT8Qbw8xKz88Lhszq5+2sSPIKGITiWKAGAKfmdXB3gQeM3GDG8Zg0hLdpIpN1apXvhRrLkLy/o+Rg6obQelUBvytuy/d48mgqzXWrve4uQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732603396; c=relaxed/simple; bh=R3cDBKXuIRR7EXwXLRGQ4uF8Qsfofxg/vk0M/9T+gqQ=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=RgkkGdG9R18PHHTdZgZpjdTKHH7nBv09tz8ZEiH6U7/b0S5Lto/wkezHh7H/VbPh4a5/xdNWBLEVnCreRNT6zYhBNg/MWEIjIjOou1zRePeYUBM81HuwcUtqiGTt+HY1H0YruRzEjWfUOoiULe3bxdER1lMd1+yJS4giSrrmLzY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=TDRLpqk/; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=KnXTOf60; arc=none smtp.client-ip=103.168.172.147 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="TDRLpqk/"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="KnXTOf60" Received: from phl-compute-03.internal (phl-compute-03.phl.internal [10.202.2.43]) by mailfout.phl.internal (Postfix) with ESMTP id 0A17413805E8; Tue, 26 Nov 2024 01:43:14 -0500 (EST) Received: from phl-mailfrontend-02 ([10.202.2.163]) by phl-compute-03.internal (MEProxy); Tue, 26 Nov 2024 01:43:14 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm1; t=1732603394; x=1732689794; bh=V4Pogm/Mo+P/f1Sw/VTI+xc4fMJmcMPGZxoA+Bjidms=; b= TDRLpqk/1Sh59aJkjieA9x7No4pjOtoNZIy2v7kxbn9SXg55SXfy0GU4cbn3SSrR zfK1E6b4jXD44Gm2MhWGgeCcDO2YzV5oikqpbUoG69ObrEMQLUAbt4rNIy02iqc5 +K/CmI/p9skF2s6kS8FdoIDIaWaBFiSBvpyauHTRh44I7vn5fUIh04mXwQxxvUwF HJJhtRPsBHdAOP7yNEU7ldYgPYVDh1IlZ2yv6/IflgRlykvTRrs23szLIoUdKXlC ODLja2cDXa8RuKlky/M0wI3cxTVkWB/9LVqqdvQocglF2lgHG+8Z9C1mx3iCmtIw BGKYjnRSHJ33C+43ki0aEQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1732603394; x= 1732689794; bh=V4Pogm/Mo+P/f1Sw/VTI+xc4fMJmcMPGZxoA+Bjidms=; b=K nXTOf60Pry6DnZcL7NUzKcAR8rXyjrRK/pUIjBYDJ0Yj0kj9qu5XoAkeCavcUNQX iDA/ZjjfNdEx8Lklo9CJP6MTb2GUNxuaDDxgvnxE1slCL+QrwRl964Ph72dPyawZ BuF0kJuh97BdaB0KZx4UyekDLz/1BqKlEB/MAyZlGrTxGmRQajfpUq5jVxp8UrI9 jcqAfmthlQuIa6ccLFsCPGBkUGr5aG8GnclPHhS9FhIuw5YmCbodp4n5g8zohKO5 UvLi9yqzgoV7UeafOzfHcOK5VxiRMlX6pKo8mkXgWKiwLCxYeKTTe+WRAP5wjI/U odu+u3eV0KibEn3NHUr4g== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefuddrgeeigdeliecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpggftfghnshhusghstghrihgsvgdpuffr tefokffrpgfnqfghnecuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnth hsucdlqddutddtmdenucfjughrpefhfffugggtgffkfhgjvfevofesthejredtredtjeen ucfhrhhomheprfgrthhrihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimh eqnecuggftrfgrthhtvghrnhepffeuiedujedvkeehuedvkeefffeivdeuleetkeduheej teekgedvudfgtdfgieelnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrg hilhhfrhhomhepphhssehpkhhsrdhimhdpnhgspghrtghpthhtohepfedpmhhouggvpehs mhhtphhouhhtpdhrtghpthhtohepkhgrrhhthhhikhdrudekkeesghhmrghilhdrtghomh dprhgtphhtthhopehgihhtsehvghgvrhdrkhgvrhhnvghlrdhorhhgpdhrtghpthhtohep ghhithhsthgvrhesphhosghogidrtghomh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 26 Nov 2024 01:43:12 -0500 (EST) Received: by vm-mail (OpenSMTPD) with ESMTPSA id 192deba0 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Tue, 26 Nov 2024 06:42:10 +0000 (UTC) From: Patrick Steinhardt Date: Tue, 26 Nov 2024 07:42:58 +0100 Subject: [PATCH v4 07/10] refs/reftable: refactor reflog expiry to use reftable backend Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241126-pks-reftable-backend-reuse-iter-v4-7-b17fd27df126@pks.im> References: <20241126-pks-reftable-backend-reuse-iter-v4-0-b17fd27df126@pks.im> In-Reply-To: <20241126-pks-reftable-backend-reuse-iter-v4-0-b17fd27df126@pks.im> To: git@vger.kernel.org Cc: karthik nayak , Junio C Hamano X-Mailer: b4 0.14.2 Refactor the callback function that expires reflog entries in the reftable backend to use `reftable_backend_read_ref()` instead of accessing the reftable stack directly. This ensures that the function will benefit from the new caching layer that we're about to introduce. Signed-off-by: Patrick Steinhardt --- refs/reftable-backend.c | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c index 2d06620ac8b1b7f07783cd5873dbe0fe67b84bd6..b6638d43028d11ae7621dcd4e0dbf1d3174743b7 100644 --- a/refs/reftable-backend.c +++ b/refs/reftable-backend.c @@ -2444,14 +2444,15 @@ static int reftable_be_reflog_expire(struct ref_store *ref_store, reftable_be_downcast(ref_store, REF_STORE_WRITE, "reflog_expire"); struct reftable_log_record *logs = NULL; struct reftable_log_record *rewritten = NULL; - struct reftable_ref_record ref_record = {0}; struct reftable_iterator it = {0}; struct reftable_addition *add = NULL; struct reflog_expiry_arg arg = {0}; struct reftable_backend *be; struct object_id oid = {0}; + struct strbuf referent = STRBUF_INIT; uint8_t *last_hash = NULL; size_t logs_nr = 0, logs_alloc = 0, i; + unsigned int type = 0; int ret; if (refs->err < 0) @@ -2473,12 +2474,9 @@ static int reftable_be_reflog_expire(struct ref_store *ref_store, if (ret < 0) goto done; - ret = reftable_stack_read_ref(be->stack, refname, &ref_record); + ret = reftable_backend_read_ref(be, refname, &oid, &referent, &type); if (ret < 0) goto done; - if (reftable_ref_record_val1(&ref_record)) - oidread(&oid, reftable_ref_record_val1(&ref_record), - ref_store->repo->hash_algo); prepare_fn(refname, &oid, policy_cb_data); while (1) { @@ -2545,8 +2543,7 @@ static int reftable_be_reflog_expire(struct ref_store *ref_store, } } - if (flags & EXPIRE_REFLOGS_UPDATE_REF && last_hash && - reftable_ref_record_val1(&ref_record)) + if (flags & EXPIRE_REFLOGS_UPDATE_REF && last_hash && !is_null_oid(&oid)) oidread(&arg.update_oid, last_hash, ref_store->repo->hash_algo); arg.refs = refs; @@ -2571,11 +2568,11 @@ static int reftable_be_reflog_expire(struct ref_store *ref_store, cleanup_fn(policy_cb_data); assert(ret != REFTABLE_API_ERROR); - reftable_ref_record_release(&ref_record); reftable_iterator_destroy(&it); reftable_addition_destroy(add); for (i = 0; i < logs_nr; i++) reftable_log_record_release(&logs[i]); + strbuf_release(&referent); free(logs); free(rewritten); return ret; From patchwork Tue Nov 26 06:42:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13885509 Received: from fhigh-a7-smtp.messagingengine.com (fhigh-a7-smtp.messagingengine.com [103.168.172.158]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B57B7192B96 for ; Tue, 26 Nov 2024 06:43:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.158 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732603397; cv=none; b=AEU3ojBWpXUF/HUXWhyoFGnPjR5EA6UCm5y4c/VhRSSYnvTf4fyen+7aOhS2x0M3STg5i2Raiu51ivnhFdT6qwC28Qs3bfSjDCHdyJ/bFqp/s3c83LnzLHWCDjHQvPBbuBVTM8/KfekBqzWopGOIDd3MqVfgHaU/9KEwVeDGvls= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732603397; c=relaxed/simple; bh=TcZnBRzW8ug0Ms+nJ4aZxM5qOe0YNDVDDbsX31N/3ug=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=qRPDm6mjBlPcxWPaDm7KbOclbMbTLg0p5c1lFBC7SaswN7B2yFe9HWndBk1KmkheZctQ6uyn1cac6E7zL5JwuIbvd6pygIZHcoMSKv7wtpnb6uUI/RZR5C0XoFlqtOlb9zEJbIjgOc97GaKsND/rZZl1mMo7gYl/cLDZVV0qtMA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=CLgMuQz6; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=Dluf6Icu; arc=none smtp.client-ip=103.168.172.158 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="CLgMuQz6"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="Dluf6Icu" Received: from phl-compute-10.internal (phl-compute-10.phl.internal [10.202.2.50]) by mailfhigh.phl.internal (Postfix) with ESMTP id 1B63F1140195; Tue, 26 Nov 2024 01:43:15 -0500 (EST) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-10.internal (MEProxy); Tue, 26 Nov 2024 01:43:15 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm1; t=1732603395; x=1732689795; bh=Msflp6uCQEerTTNcabEtzkLfRmj3FmMpoD+7j9+/N/g=; b= CLgMuQz6p+g6rV5Bj7RU80eXIUE5O/efm1O+MEVtHuVvrClP2L7i3v7WBpfSLjRd 3D2C2GdfpknUpNKWMfnA+MIChER+Rcx5+jmEpOr8StrGunydrnpOAWkxUXj12f70 UWnp9S6zI10qBMKdoyeAI7YX6pPFnt2OFeMNMuStSYC7lvCowxTG5cS1eeIpyT09 uCl6dRhU3qXX+Ov+TeLsQoZd7D1j2dI3kWtNOAs8HODoTEewKogEbOossLrvCQFB CRQU5zd2/OOik2J4kpvtv3HuPfSVnm+E3YSz75jkbNHLj+9FR8Kwa0dvPDSI6oF8 7dsRiNgmG7j99VakND+Abw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1732603395; x= 1732689795; bh=Msflp6uCQEerTTNcabEtzkLfRmj3FmMpoD+7j9+/N/g=; b=D luf6IcuPpZEVuVWT49zEBKoE8Sy0hQaKvSSoLe0oMRKVfj2sMvaNbsZW01iYLS1r LV7dWN/Udu8y6BAsfn+tii8muno60kSyGphXQl27/AY66bUxIooi3+7w81UufEo3 kmSrnYkjziL5GhYTmWxMX3YKTKpMRKOOqZmG6D1BkKIav4PXwEJvMB8dBXdd87Z/ O5EoNZ1UPMArs8NvdCQ79CPEfyEaOuas0UgguHoarhC2YLsSF8yaipdziFS8I8mZ 02Obaq4fmpp3MDPzso3Rh2qj0BOAyA7ChOb+M9QUCqMvoAwUtE/jtQQCrr9Jk2Ui UpRJXsRXci1yGEs7uewjA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefuddrgeeigdeliecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpggftfghnshhusghstghrihgsvgdpuffr tefokffrpgfnqfghnecuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnth hsucdlqddutddtmdenucfjughrpefhfffugggtgffkfhgjvfevofesthejredtredtjeen ucfhrhhomheprfgrthhrihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimh eqnecuggftrfgrthhtvghrnhepffeuiedujedvkeehuedvkeefffeivdeuleetkeduheej teekgedvudfgtdfgieelnecuvehluhhsthgvrhfuihiivgepudenucfrrghrrghmpehmrg hilhhfrhhomhepphhssehpkhhsrdhimhdpnhgspghrtghpthhtohepfedpmhhouggvpehs mhhtphhouhhtpdhrtghpthhtohepghhithesvhhgvghrrdhkvghrnhgvlhdrohhrghdprh gtphhtthhopehgihhtshhtvghrsehpohgsohigrdgtohhmpdhrtghpthhtohepkhgrrhht hhhikhdrudekkeesghhmrghilhdrtghomh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 26 Nov 2024 01:43:14 -0500 (EST) Received: by vm-mail (OpenSMTPD) with ESMTPSA id 154b1dfe (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Tue, 26 Nov 2024 06:42:11 +0000 (UTC) From: Patrick Steinhardt Date: Tue, 26 Nov 2024 07:42:59 +0100 Subject: [PATCH v4 08/10] reftable/stack: add mechanism to notify callers on reload Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241126-pks-reftable-backend-reuse-iter-v4-8-b17fd27df126@pks.im> References: <20241126-pks-reftable-backend-reuse-iter-v4-0-b17fd27df126@pks.im> In-Reply-To: <20241126-pks-reftable-backend-reuse-iter-v4-0-b17fd27df126@pks.im> To: git@vger.kernel.org Cc: karthik nayak , Junio C Hamano X-Mailer: b4 0.14.2 Reftable stacks are reloaded in two cases: - When calling `reftable_stack_reload()`, if the stat-cache tells us that the stack has been modified. - When committing a reftable addition. While callers can figure out the second case, they do not have a mechanism to figure out whether `reftable_stack_reload()` led to an actual reload of the on-disk data. All they can do is thus to assume that data is always being reloaded in that case. Improve the situation by introducing a new `on_reload()` callback to the reftable options. If provided, the function will be invoked every time the stack has indeed been reloaded. This allows callers to invalidate data that depends on the current stack data. Signed-off-by: Patrick Steinhardt --- reftable/reftable-writer.h | 9 +++++++++ reftable/stack.c | 4 ++++ 2 files changed, 13 insertions(+) diff --git a/reftable/reftable-writer.h b/reftable/reftable-writer.h index c85ef5a5bd14595d75f99457fef4407040e197c5..5f9afa620bb00de66c311765fb0ae8c6f56401ae 100644 --- a/reftable/reftable-writer.h +++ b/reftable/reftable-writer.h @@ -68,6 +68,15 @@ struct reftable_write_options { * fsync(3P) when unset. */ int (*fsync)(int fd); + + /* + * Callback function to execute whenever the stack is being reloaded. + * This can be used e.g. to discard cached information that relies on + * the old stack's data. The payload data will be passed as argument to + * the callback. + */ + void (*on_reload)(void *payload); + void *on_reload_payload; }; /* reftable_block_stats holds statistics for a single block type */ diff --git a/reftable/stack.c b/reftable/stack.c index d97b64a40d4ad05cfd9e6f33e8ba1e713281ef6d..5384ca9de0d1f064aebcb09308a74cc397b37463 100644 --- a/reftable/stack.c +++ b/reftable/stack.c @@ -548,6 +548,10 @@ static int reftable_stack_reload_maybe_reuse(struct reftable_stack *st, close(fd); free_names(names); free_names(names_after); + + if (st->opts.on_reload) + st->opts.on_reload(st->opts.on_reload_payload); + return err; } From patchwork Tue Nov 26 06:43:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13885510 Received: from fhigh-a7-smtp.messagingengine.com (fhigh-a7-smtp.messagingengine.com [103.168.172.158]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 34DC21957FF for ; Tue, 26 Nov 2024 06:43:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.158 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732603399; cv=none; b=IbSrdciR2V+vjOtbpQUJVaP/HQNzNIZxPTKOTbWcRi7s7OSrGe40juS0kFWb4jIHCpQZl+QgOqjS0vfrg3pWfTBr/FFBwmwdG/pWfWy1ci6bdWIaYqeQ33p+xp54COPIoTti4UsvupHMOV++O2cJSCjtvA4DK5xCPAS8x6oKcJI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732603399; c=relaxed/simple; bh=uUPaBBPP0gxrd2yBLKHUDE++1sBdqt0WVcimtJ5GcFY=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=KkT56YmwSLsTLNnWCHAG0c9zBW+MoG/MTEdHJgfJO1lDVz6MWi1qAYyMm78L4ZuNIbNx/7REeR1uF5YdMVfWarDdu4zDtJ5gezIzG4eS/cmZnQzzWfY4iI4mLvVkDacRaGwzLdRU1nx1oQqjhPDBqI7+zVLFHGC6A8YYr/du4zA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=gON6qz5q; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=r/hQX59w; arc=none smtp.client-ip=103.168.172.158 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="gON6qz5q"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="r/hQX59w" Received: from phl-compute-10.internal (phl-compute-10.phl.internal [10.202.2.50]) by mailfhigh.phl.internal (Postfix) with ESMTP id 3E11C11401CA; Tue, 26 Nov 2024 01:43:16 -0500 (EST) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-10.internal (MEProxy); Tue, 26 Nov 2024 01:43:16 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm1; t=1732603396; x=1732689796; bh=dX17a32p2VvqyhiVIIa/68qE5g0l3FBuVpmR3eWFDyQ=; b= gON6qz5qfn1ljjwznz/b2gcMI8yKfIIsYmDNUuGcDVqg9GYCbV+TMz6bF7ANW2mY 3YrvRAhIiUYUoBuchpd4tM7aGt5qXkz36tuvnO6YXe55MTPcv6vpDVVge4bf2P21 dWyqjIEVCj/qZpNcynYrdLrqimFuOIfugek2pAFHeGV49rtzJVDjCbLoXKFK5vae dNEaBSuXdDNTrD9UkjtGtbakiXvXsra/hoT4275szbSNryNFKRpEauw3Y7pr2Z7q qwpc/g2dKkoQMpMQF213uNLUPoEnMl8qMqcZrgFubP1RvQ+2D6+YAeZ5iGPGPnrx IaphyMUuArM5NjNrjUIPLw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1732603396; x= 1732689796; bh=dX17a32p2VvqyhiVIIa/68qE5g0l3FBuVpmR3eWFDyQ=; b=r /hQX59wW81BDMo+PtAV6qHc7qU6DzDcc6BwWdrOFinzNia9ZE7QGD8tWz1sWJR1V 2VdjAVTss5I7gxMapjWWCYouN805crqCiuy/Jzscf3xcn0LQGcu5lSeNLZxbVlDN OiwG+Vu52J41tW1OBKCfrBEGj+n5FTY9jerbaEGpnYiY1yj57d6afjL8f2kt75KW QUx2phQ1K2tdwtCIGjk21ccTlamnuBv/YeYHv6jbKLkYBK/3EkXMWYnK9OjzySBB Xc4RnHnvnH3ANP9HVShB/SrAqmKtLEPOiYha6Cm54T5zBYdhUO/i/2h+iBqf+2ly HBKZF4bqKZy9S7Y+SnwLg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefuddrgeeigdeliecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpggftfghnshhusghstghrihgsvgdpuffr tefokffrpgfnqfghnecuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnth hsucdlqddutddtmdenucfjughrpefhfffugggtgffkfhgjvfevofesthejredtredtjeen ucfhrhhomheprfgrthhrihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimh eqnecuggftrfgrthhtvghrnhepffeuiedujedvkeehuedvkeefffeivdeuleetkeduheej teekgedvudfgtdfgieelnecuvehluhhsthgvrhfuihiivgepudenucfrrghrrghmpehmrg hilhhfrhhomhepphhssehpkhhsrdhimhdpnhgspghrtghpthhtohepfedpmhhouggvpehs mhhtphhouhhtpdhrtghpthhtohepkhgrrhhthhhikhdrudekkeesghhmrghilhdrtghomh dprhgtphhtthhopehgihhtshhtvghrsehpohgsohigrdgtohhmpdhrtghpthhtohepghhi thesvhhgvghrrdhkvghrnhgvlhdrohhrgh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 26 Nov 2024 01:43:15 -0500 (EST) Received: by vm-mail (OpenSMTPD) with ESMTPSA id 98dd281c (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Tue, 26 Nov 2024 06:42:12 +0000 (UTC) From: Patrick Steinhardt Date: Tue, 26 Nov 2024 07:43:00 +0100 Subject: [PATCH v4 09/10] reftable/merged: drain priority queue on reseek Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241126-pks-reftable-backend-reuse-iter-v4-9-b17fd27df126@pks.im> References: <20241126-pks-reftable-backend-reuse-iter-v4-0-b17fd27df126@pks.im> In-Reply-To: <20241126-pks-reftable-backend-reuse-iter-v4-0-b17fd27df126@pks.im> To: git@vger.kernel.org Cc: karthik nayak , Junio C Hamano X-Mailer: b4 0.14.2 In 5bf96e0c39 (reftable/generic: move seeking of records into the iterator, 2024-05-13) we have refactored the reftable codebase such that iterators can be initialized once and then re-seeked multiple times. This feature is used by 1869525066 (refs/reftable: wire up support for exclude patterns, 2024-09-16) in order to skip records based on exclude patterns provided by the caller. The logic to re-seek the merged iterator is insufficient though because we don't drain the priority queue on a re-seek. This means that the queue may contain stale entries and thus reading the next record in the queue will return the wrong entry. While this is an obvious bug, it is harmless in the context of above exclude patterns: - If the queue contained stale entries that match the pattern then the caller would already know to filter out such refs. This is because our codebase is prepared to handle backends that don't have a way to efficiently implement exclude patterns. - If the queue contained stale entries that don't match the pattern we'd eventually filter out any duplicates. This is because the reftable code discards items with the same ref name and sorts any remaining entries properly. So things happen to work in this context regardless of the bug, and there is no other use case yet where we re-seek iterators. We're about to introduce a caching mechanism though where iterators are reused by the reftable backend, and that will expose the bug. Fix the issue by draining the priority queue when seeking and add a testcase that surfaces the issue. Signed-off-by: Patrick Steinhardt --- reftable/merged.c | 2 ++ t/unit-tests/t-reftable-merged.c | 73 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 75 insertions(+) diff --git a/reftable/merged.c b/reftable/merged.c index 5b93e20f42945300abbc1a036bbdf067fced7854..bb0836e3443271f9c0d5ba5582c78694d437ddc2 100644 --- a/reftable/merged.c +++ b/reftable/merged.c @@ -66,6 +66,8 @@ static int merged_iter_seek(struct merged_iter *mi, struct reftable_record *want int err; mi->advance_index = -1; + while (!merged_iter_pqueue_is_empty(mi->pq)) + merged_iter_pqueue_remove(&mi->pq); for (size_t i = 0; i < mi->subiters_len; i++) { err = iterator_seek(&mi->subiters[i].iter, want); diff --git a/t/unit-tests/t-reftable-merged.c b/t/unit-tests/t-reftable-merged.c index 2591b5e59745536a205271491f747875e04c5a3f..a12bd0e1a3bdeda82bcbc6259e679df4f232e3e2 100644 --- a/t/unit-tests/t-reftable-merged.c +++ b/t/unit-tests/t-reftable-merged.c @@ -273,6 +273,78 @@ static void t_merged_seek_multiple_times(void) reftable_free(sources); } +static void t_merged_seek_multiple_times_without_draining(void) +{ + struct reftable_ref_record r1[] = { + { + .refname = (char *) "a", + .update_index = 1, + .value_type = REFTABLE_REF_VAL1, + .value.val1 = { 1 }, + }, + { + .refname = (char *) "c", + .update_index = 1, + .value_type = REFTABLE_REF_VAL1, + .value.val1 = { 2 }, + } + }; + struct reftable_ref_record r2[] = { + { + .refname = (char *) "b", + .update_index = 2, + .value_type = REFTABLE_REF_VAL1, + .value.val1 = { 3 }, + }, + { + .refname = (char *) "d", + .update_index = 2, + .value_type = REFTABLE_REF_VAL1, + .value.val1 = { 4 }, + }, + }; + struct reftable_ref_record *refs[] = { + r1, r2, + }; + size_t sizes[] = { + ARRAY_SIZE(r1), ARRAY_SIZE(r2), + }; + struct reftable_buf bufs[] = { + REFTABLE_BUF_INIT, REFTABLE_BUF_INIT, + }; + struct reftable_block_source *sources = NULL; + struct reftable_reader **readers = NULL; + struct reftable_ref_record rec = { 0 }; + struct reftable_iterator it = { 0 }; + struct reftable_merged_table *mt; + int err; + + mt = merged_table_from_records(refs, &sources, &readers, sizes, bufs, 2); + merged_table_init_iter(mt, &it, BLOCK_TYPE_REF); + + err = reftable_iterator_seek_ref(&it, "b"); + check(!err); + err = reftable_iterator_next_ref(&it, &rec); + check(!err); + err = reftable_ref_record_equal(&rec, &r2[0], REFTABLE_HASH_SIZE_SHA1); + check(err == 1); + + err = reftable_iterator_seek_ref(&it, "a"); + check(!err); + err = reftable_iterator_next_ref(&it, &rec); + check(!err); + err = reftable_ref_record_equal(&rec, &r1[0], REFTABLE_HASH_SIZE_SHA1); + check(err == 1); + + for (size_t i = 0; i < ARRAY_SIZE(bufs); i++) + reftable_buf_release(&bufs[i]); + readers_destroy(readers, ARRAY_SIZE(refs)); + reftable_ref_record_release(&rec); + reftable_iterator_destroy(&it); + reftable_merged_table_free(mt); + reftable_free(sources); +} + static struct reftable_merged_table * merged_table_from_log_records(struct reftable_log_record **logs, struct reftable_block_source **source, @@ -467,6 +539,7 @@ int cmd_main(int argc UNUSED, const char *argv[] UNUSED) TEST(t_merged_logs(), "merged table with multiple log updates for same ref"); TEST(t_merged_refs(), "merged table with multiple updates to same ref"); TEST(t_merged_seek_multiple_times(), "merged table can seek multiple times"); + TEST(t_merged_seek_multiple_times_without_draining(), "merged table can seek multiple times without draining"); TEST(t_merged_single_record(), "ref occurring in only one record can be fetched"); return test_done(); From patchwork Tue Nov 26 06:43:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 13885511 Received: from fhigh-a7-smtp.messagingengine.com (fhigh-a7-smtp.messagingengine.com [103.168.172.158]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5140C199EA2 for ; Tue, 26 Nov 2024 06:43:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=103.168.172.158 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732603401; cv=none; b=JgckDnXFixSHkkbr189/tHUvH/0OFPok4Qj0mjfV1CvlAHkg6pikS9bHbeL6cQvewBOkNci6MHcPUpFw/FIP1sVVXrBHnEGT86JRGY+pTsuD9rS7phFNhb//M5aoF7x4qn7XLD6qOXF48nTv2GJH9sXv/H7xZWLRElQQe6T/Yls= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732603401; c=relaxed/simple; bh=GS29QW5K5eJC2D3Q8zHhdR1bVjNFUz8vKrFJz4uzKPI=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=UZDVD97UMTvf/oxMon33wUkRbOYlvAVqESL8weO4EilbqsOu1ZjfsqJsDNALsEapCokyvRPC2P2MtcF4P84fv34cjTM3LUtpFd4buLvZ4sYx0rPltw2toKxRFVEgUDGG6mG9T5zg6S1xEH+9YBpefRJMByy2WZj+ibnYQWq3AYY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im; spf=pass smtp.mailfrom=pks.im; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b=nmM0n7v6; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=fJtI6/zY; arc=none smtp.client-ip=103.168.172.158 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=pks.im Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=pks.im Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="nmM0n7v6"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="fJtI6/zY" Received: from phl-compute-09.internal (phl-compute-09.phl.internal [10.202.2.49]) by mailfhigh.phl.internal (Postfix) with ESMTP id 75A2E1140195; Tue, 26 Nov 2024 01:43:17 -0500 (EST) Received: from phl-mailfrontend-01 ([10.202.2.162]) by phl-compute-09.internal (MEProxy); Tue, 26 Nov 2024 01:43:17 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=cc:cc :content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm1; t=1732603397; x=1732689797; bh=tXpqkJU3/48Dq0e5hPz42ubC41ZA//uEhX92CRN7jsA=; b= nmM0n7v6aO4hgDO36gkBw/iv8gtsH/TMymjPD01/i2Kx9laAIf4AQf2YWZu8mlNE PFxPxTGM+C6viGdm7yVdOIDXxB0tJuojLrwEUzU3ANTY5xo63LhvT2FW4dOyLZPc N/d5wCR6hMjB08svjNYajyorj8wGCWb5Z/1PZ8dPyxDJ1CAPSL8/9Ja8/9gJyHPR pml1BYNAfKvdqf/Fjn7zN/r/yBqG53Pb/K86SWNuPr+f8Nmy3yb+ajrehHy9XQ1x bM6RsuhMCMlIrvsoGKNun8fDzRz9HMlWj+S7+lG4w3+fINMvPO5GsT21dTZ0xZ1K 5UemFkI1o4mGKR4qmeShRA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1732603397; x= 1732689797; bh=tXpqkJU3/48Dq0e5hPz42ubC41ZA//uEhX92CRN7jsA=; b=f JtI6/zYOTT9ngzWz69/K+IRgyrwzaPWTCut2/3CgQPJ/3+0KHy1EZV97ultWK7aY gp1NZE9qAmDPb6EG+L0rKSCzHvMc1qXGXLlbXERGb/I/aa1wEW690CzadZGM5cwS unj6JOqlzu8jiE/J8nf6aVA2zcA0U4CHbOcuCIGKgx6L5SjgxtrLpJiXLmk5Af1G ZcxWWMh7RvZlKPd2AnmWzNA2lytjhMFh1WjhenwFzNMuqT5lZ1eM9JyzTgzbzsA/ nNAXcQpgfJfVj6J2MVMNoiSnhK294Ba6I+Dwd3hE92XoEhRHK5fuIYwTGajori7K ZkgUffW+F7M4Xhj7LLUWQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeefuddrgeeigdeliecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpggftfghnshhusghstghrihgsvgdpuffr tefokffrpgfnqfghnecuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnth hsucdlqddutddtmdenucfjughrpefhfffugggtgffkfhgjvfevofesthekredtredtjeen ucfhrhhomheprfgrthhrihgtkhcuufhtvghinhhhrghrughtuceophhssehpkhhsrdhimh eqnecuggftrfgrthhtvghrnhepfefhueegleehfeejkedtffehvdfhvdetfefgtdduffdu veevteegueeutdekhfegnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrg hilhhfrhhomhepphhssehpkhhsrdhimhdpnhgspghrtghpthhtohepfedpmhhouggvpehs mhhtphhouhhtpdhrtghpthhtohepghhithesvhhgvghrrdhkvghrnhgvlhdrohhrghdprh gtphhtthhopehkrghrthhhihhkrddukeeksehgmhgrihhlrdgtohhmpdhrtghpthhtohep ghhithhsthgvrhesphhosghogidrtghomh X-ME-Proxy: Feedback-ID: i197146af:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 26 Nov 2024 01:43:16 -0500 (EST) Received: by vm-mail (OpenSMTPD) with ESMTPSA id 7fa5a86c (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Tue, 26 Nov 2024 06:42:13 +0000 (UTC) From: Patrick Steinhardt Date: Tue, 26 Nov 2024 07:43:01 +0100 Subject: [PATCH v4 10/10] refs/reftable: reuse iterators when reading refs Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241126-pks-reftable-backend-reuse-iter-v4-10-b17fd27df126@pks.im> References: <20241126-pks-reftable-backend-reuse-iter-v4-0-b17fd27df126@pks.im> In-Reply-To: <20241126-pks-reftable-backend-reuse-iter-v4-0-b17fd27df126@pks.im> To: git@vger.kernel.org Cc: karthik nayak , Junio C Hamano X-Mailer: b4 0.14.2 When reading references the reftable backend has to: 1. Create a new ref iterator. 2. Seek the iterator to the record we're searching for. 3. Read the record. We cannot really avoid the last two steps, but re-creating the iterator every single time we want to read a reference is kind of expensive and a waste of resources. We couldn't help it in the past though because it was not possible to reuse iterators. But starting with 5bf96e0c39 (reftable/generic: move seeking of records into the iterator, 2024-05-13) we have split up the iterator lifecycle such that creating the iterator and seeking are two different concerns. Refactor the code such that we cache iterators in the reftable backend. This cache is invalidated whenever the respective stack is reloaded such that we know to recreate the iterator in that case. This leads to a sizeable speedup when creating many refs, which requires a lot of random reference reads: Benchmark 1: update-ref: create many refs (refcount = 100000, revision = master) Time (mean ± σ): 1.793 s ± 0.010 s [User: 0.954 s, System: 0.835 s] Range (min … max): 1.781 s … 1.811 s 10 runs Benchmark 2: update-ref: create many refs (refcount = 100000, revision = HEAD) Time (mean ± σ): 1.680 s ± 0.013 s [User: 0.846 s, System: 0.831 s] Range (min … max): 1.664 s … 1.702 s 10 runs Summary update-ref: create many refs (refcount = 100000, revision = HEAD) ran 1.07 ± 0.01 times faster than update-ref: create many refs (refcount = 100000, revision = master) While 7% is not a huge win, you have to consider that the benchmark is _writing_ data, so _reading_ references is only one part of what we do. Flame graphs show that we spend around 40% of our time reading refs, so the speedup when reading refs is approximately ~2.5x that. I could not find better benchmarks where we perform a lot of random ref reads. You can also see a sizeable impact on memory usage when creating 100k references. Before this change: HEAP SUMMARY: in use at exit: 19,112,538 bytes in 200,170 blocks total heap usage: 8,400,426 allocs, 8,200,256 frees, 454,367,048 bytes allocated After this change: HEAP SUMMARY: in use at exit: 674,416 bytes in 169 blocks total heap usage: 7,929,872 allocs, 7,929,703 frees, 281,509,985 bytes allocated As an additional factor, this refactoring opens up the possibility for more performance optimizations in how we re-seek iterators. Any change that allows us to optimize re-seeking by e.g. reusing data structures would thus also directly speed up random reads. Signed-off-by: Patrick Steinhardt --- refs/reftable-backend.c | 32 +++++++++++++++++++++++++++++--- 1 file changed, 29 insertions(+), 3 deletions(-) diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c index b6638d43028d11ae7621dcd4e0dbf1d3174743b7..a80c334f384acd0d72a1c4ef9e59c93600dd6db4 100644 --- a/refs/reftable-backend.c +++ b/refs/reftable-backend.c @@ -36,19 +36,30 @@ struct reftable_backend { struct reftable_stack *stack; + struct reftable_iterator it; }; +static void reftable_backend_on_reload(void *payload) +{ + struct reftable_backend *be = payload; + reftable_iterator_destroy(&be->it); +} + static int reftable_backend_init(struct reftable_backend *be, const char *path, - const struct reftable_write_options *opts) + const struct reftable_write_options *_opts) { - return reftable_new_stack(&be->stack, path, opts); + struct reftable_write_options opts = *_opts; + opts.on_reload = reftable_backend_on_reload; + opts.on_reload_payload = be; + return reftable_new_stack(&be->stack, path, &opts); } static void reftable_backend_release(struct reftable_backend *be) { reftable_stack_destroy(be->stack); be->stack = NULL; + reftable_iterator_destroy(&be->it); } static int reftable_backend_read_ref(struct reftable_backend *be, @@ -60,10 +71,25 @@ static int reftable_backend_read_ref(struct reftable_backend *be, struct reftable_ref_record ref = {0}; int ret; - ret = reftable_stack_read_ref(be->stack, refname, &ref); + if (!be->it.ops) { + ret = reftable_stack_init_ref_iterator(be->stack, &be->it); + if (ret) + goto done; + } + + ret = reftable_iterator_seek_ref(&be->it, refname); if (ret) goto done; + ret = reftable_iterator_next_ref(&be->it, &ref); + if (ret) + goto done; + + if (strcmp(ref.refname, refname)) { + ret = 1; + goto done; + } + if (ref.value_type == REFTABLE_REF_SYMREF) { strbuf_reset(referent); strbuf_addstr(referent, ref.value.symref);