From patchwork Mon Dec 9 11:07:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karthik Nayak X-Patchwork-Id: 13899318 Received: from mail-ej1-f50.google.com (mail-ej1-f50.google.com [209.85.218.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4E22C21B8F5 for ; Mon, 9 Dec 2024 11:07:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733742447; cv=none; b=tJqX2gYTm7n293hSMe6J9JlMY+ivS2cgjmKUiZyizsS/mFG4cPnPpuE3cyRYkaegEVDpZ6Fr4PofpKfjGeT+N60MkeENWmWiqShiWgjyfB5f/YoIiggD5tRw/HCqftJzMWIk9j7+1mn7x+kaSsgXR4hpCdQoAfSwfPvrpLWDDiM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733742447; c=relaxed/simple; bh=/XjJV0LYQeg4I9hOsy8SjFCeo5ZquFL0ZugMDhGlLLA=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=ooH6LjdLe7CGKYcVynCeylWL/7SMYAySL2F7pUz7L38EnCB6Y3QhijXdbHCu/ljfwUSGRtrh/ixn0TfS/Z/EYasAVfauu2WGSmDFOxSulEWkfu9LeBb5eIi1rA+yOoUgVfz0JoDLEK0OWiLZg5QG6kD1uOwhEzAW9TryhBFrL/w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=OdiVx2/a; arc=none smtp.client-ip=209.85.218.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="OdiVx2/a" Received: by mail-ej1-f50.google.com with SMTP id a640c23a62f3a-aa5f1909d6fso798886966b.3 for ; Mon, 09 Dec 2024 03:07:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1733742443; x=1734347243; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=OqCUFtoQIVaYaTWtnFeSnYm3Lb6j6vYL1NqxANp+6Oo=; b=OdiVx2/az1vOAcy8tj90b0ZQJ/fHH62mPYWN3upjeMXN0HO0UUfexipHDNhaqX/45d D3qnRB8bvHNic6WdkdkS4OGr76axsXg5+42fR2LkcuzUsVl27wlKrSRRFxGq5S/GgNPo 5t3EoA8AuSx/SUOyYaefY34ZDxQ5Qzdr7ocE7tI0JTRRmF973Lr/Fi6mSdOru9Z4gRN3 iRvCeTUwDqRBwHDFQPr2WVf0SjWQK1MFxc5/4xzmmUjuYLsLsYhypi6T99/Vstw1tpLc OrJ5VOX7Pbtt6w2Chq1LAMSNBrakSX4kHEB4C5aeqenO5XiC9Cr6DCBNlI85LWRqF9sH BhIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733742443; x=1734347243; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=OqCUFtoQIVaYaTWtnFeSnYm3Lb6j6vYL1NqxANp+6Oo=; b=uAU3dRAz0WEoB8Oq9k7WIMeoE0hXDWWCpY6MIF1R9w+r9tj2Rniz5XEbbrJpKwRdUX sRQwnSyFI4s2I7BqKCQGf4yOOIoTgKbCGCxH1tdVx6MItxZ/y42WEcvVt9f3wOKvvS2Y rheHi7ficGoJO9Lm7J9a3nH9VVWu5euBvrqlzzIKKSKWqtx0NRbBUxOaVNW3uGSxA9IP fGBUBqTzCUMeQHJ5IIr+wZPPCDpX19crmx2tQ3ZK09luonj+Mppd3vgc3tNwQjr8GxAi GJL4fYf9BgADEShb2SA5FUMKjlAOv8dskJU5J5OvWHiHXsG0GJVkwNx4ZG7ywK6s//4+ ji+w== X-Gm-Message-State: AOJu0YyPgXQjg8wupHzSl8KUS8BcAPZNnHmMFrqsJ5Qv+9SOh+5Uaqw8 jK08iob4HGWyjN3cMD0VgWHGibWDSjZh3XmGY6Unntd0QWCaJzY+F1JguKGc X-Gm-Gg: ASbGncvoUTgrkevlLylL4Rfm4uc6QKpiOXfeve53PNPsRAoi4r4yb3qRxNe6n21KdVh C4BYWVPyFLkEeSOnnjC+i4BaMFjUxbjYGjrcfKW6WXQYMWw8M2S9/i2zI+OjUfcPWFMh0Yyk2VX wvQPAarNl3NMBQk6I2zhDP9RGskwDoHoEBhPWlnTc+UspKQATZ2+E5/rD/iVG5uPyP6k0iapBAh KAyqGy9Wpfdz8WdxSfs7AQKe8gyLku041Et8wueGEm+aNSpJomKAHwDoleET1A= X-Google-Smtp-Source: AGHT+IH/LGwku0+z26M1LkmYjoAhMZknSKDEDqcKg0u6PY4+vK2VHNlfCPJUJ6O8DurLKgNjrm1Qpg== X-Received: by 2002:a17:906:3092:b0:aa6:75e1:186b with SMTP id a640c23a62f3a-aa675e1433emr441932066b.53.1733742443200; Mon, 09 Dec 2024 03:07:23 -0800 (PST) Received: from [192.168.178.20] ([2a02:2455:8256:2d00:9c39:c2d7:aedd:294d]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aa672d377fbsm257013366b.75.2024.12.09.03.07.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Dec 2024 03:07:22 -0800 (PST) From: Karthik Nayak Date: Mon, 09 Dec 2024 12:07:15 +0100 Subject: [PATCH 1/7] refs: include committer info in `ref_update` struct Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241209-320-git-refs-migrate-reflogs-v1-1-d4bc37ee860f@gmail.com> References: <20241209-320-git-refs-migrate-reflogs-v1-0-d4bc37ee860f@gmail.com> In-Reply-To: <20241209-320-git-refs-migrate-reflogs-v1-0-d4bc37ee860f@gmail.com> To: git@vger.kernel.org Cc: Karthik Nayak , toon@iotcl.com, Christian Couder X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=5660; i=karthik.188@gmail.com; h=from:subject:message-id; bh=/XjJV0LYQeg4I9hOsy8SjFCeo5ZquFL0ZugMDhGlLLA=; b=owEB7QES/pANAwAIAT7VnySORox/AcsmYgBnVs9nRglhuUcpGwqSf34DJv7UMeHH3xD9gi9A7 XDaLxRi5fKJAbMEAAEIAB0WIQRXzkx/Y3VxD8tlxgY+1Z8kjkaMfwUCZ1bPZwAKCRA+1Z8kjkaM f2ADC/48BdrgEu4Egsnfeb0tY4io55CqLIS61+nsnLyaXjXpBrSh4TWT4R75VZ2k2eYfEyyVCf2 r4KKDlObsI6yfXMbxDJICMcFyjughnKsMDzzOgYxHz/zKfc/N9qixFlapymipVwlSX0MPr/ujbN PvUdDm2TOd241ZBxhC+evBaZMW32M99fShUbi9KYsfQYbQidCHKMSi/Jq5VviXE0t/PX8a+BOq5 56G0A4rtSdiwMzZGHSmsAm/+HeE2IKBdqIteCXhw6sv2RLSFAVq3zzQQ2s4eWxSbjRshAylUzfW 9saLf4172gupUrPDeUfURG7ju7+Nvui+9gdWW14qBMt78+xBjNrn2mEhSyVdrDh4Gr4ynXlYGrt eMTRAt5VQx9XbFhYl9I/d4hQo/cAY/Ts/o9iKOJ6dpzilXMU60sYBcn1NHonMjXqR1FzL/fW7+Z rBpweUJNJfSjbQSColhryBAduSZDGH55YHQuV0ng/taTXYEekE03jjEGCyXVSohuxRrNE= X-Developer-Key: i=karthik.188@gmail.com; a=openpgp; fpr=57CE4C7F6375710FCB65C6063ED59F248E468C7F The reference backends obtain the committer information from `git_committer_info(0)` when adding a reflog. The upcoming patches introduce support for migrating reflogs between the reference backends. This requires an interface to creating reflogs, including custom committer information. Add a new field `committer_info` to the `ref_update` struct, which is then used by the reference backends. If there is no `committer_info` provided, the reference backends default to using `git_committer_info(0)`. The field itself cannot be set to `git_committer_info(0)` since the values are dynamic and must be obtained right when the reflog is being committed. Signed-off-by: Karthik Nayak --- refs.c | 1 + refs/files-backend.c | 22 +++++++++++++--------- refs/refs-internal.h | 1 + refs/reftable-backend.c | 12 +++++++++++- 4 files changed, 26 insertions(+), 10 deletions(-) diff --git a/refs.c b/refs.c index 762f3e324d59c60cd4f05c2f257e54de8deb00e5..f003e51c6bf5229bfbce8ce61ffad7cdba0572e0 100644 --- a/refs.c +++ b/refs.c @@ -1151,6 +1151,7 @@ void ref_transaction_free(struct ref_transaction *transaction) for (i = 0; i < transaction->nr; i++) { free(transaction->updates[i]->msg); + free(transaction->updates[i]->committer_info); free((char *)transaction->updates[i]->new_target); free((char *)transaction->updates[i]->old_target); free(transaction->updates[i]); diff --git a/refs/files-backend.c b/refs/files-backend.c index 64f51f0da905a9a8a1ac4109c6b0a9a85a355db7..13f8539e6caa923cd4834775fcb0cd7f90d82014 100644 --- a/refs/files-backend.c +++ b/refs/files-backend.c @@ -1858,6 +1858,9 @@ static int log_ref_write_fd(int fd, const struct object_id *old_oid, struct strbuf sb = STRBUF_INIT; int ret = 0; + if (!committer) + committer = git_committer_info(0); + strbuf_addf(&sb, "%s %s %s", oid_to_hex(old_oid), oid_to_hex(new_oid), committer); if (msg && *msg) { strbuf_addch(&sb, '\t'); @@ -1871,8 +1874,10 @@ static int log_ref_write_fd(int fd, const struct object_id *old_oid, } static int files_log_ref_write(struct files_ref_store *refs, - const char *refname, const struct object_id *old_oid, - const struct object_id *new_oid, const char *msg, + const char *refname, + const struct object_id *old_oid, + const struct object_id *new_oid, + const char *committer_info, const char *msg, int flags, struct strbuf *err) { int logfd, result; @@ -1889,8 +1894,7 @@ static int files_log_ref_write(struct files_ref_store *refs, if (logfd < 0) return 0; - result = log_ref_write_fd(logfd, old_oid, new_oid, - git_committer_info(0), msg); + result = log_ref_write_fd(logfd, old_oid, new_oid, committer_info, msg); if (result) { struct strbuf sb = STRBUF_INIT; int save_errno = errno; @@ -1974,8 +1978,7 @@ static int commit_ref_update(struct files_ref_store *refs, files_assert_main_repository(refs, "commit_ref_update"); clear_loose_ref_cache(refs); - if (files_log_ref_write(refs, lock->ref_name, - &lock->old_oid, oid, + if (files_log_ref_write(refs, lock->ref_name, &lock->old_oid, oid, NULL, logmsg, flags, err)) { char *old_msg = strbuf_detach(err, NULL); strbuf_addf(err, "cannot update the ref '%s': %s", @@ -2007,8 +2010,8 @@ static int commit_ref_update(struct files_ref_store *refs, if (head_ref && (head_flag & REF_ISSYMREF) && !strcmp(head_ref, lock->ref_name)) { struct strbuf log_err = STRBUF_INIT; - if (files_log_ref_write(refs, "HEAD", - &lock->old_oid, oid, + if (files_log_ref_write(refs, "HEAD", &lock->old_oid, + oid, git_committer_info(0), logmsg, flags, &log_err)) { error("%s", log_err.buf); strbuf_release(&log_err); @@ -2969,7 +2972,8 @@ static int parse_and_write_reflog(struct files_ref_store *refs, } if (files_log_ref_write(refs, lock->ref_name, &lock->old_oid, - &update->new_oid, update->msg, update->flags, err)) { + &update->new_oid, update->committer_info, + update->msg, update->flags, err)) { char *old_msg = strbuf_detach(err, NULL); strbuf_addf(err, "cannot update the ref '%s': %s", diff --git a/refs/refs-internal.h b/refs/refs-internal.h index 58aa56d1b27c85d606ed7c8c0d908e4b87d1066b..0fd95cdacd99e4a728c22f5286f6b3f0f360c110 100644 --- a/refs/refs-internal.h +++ b/refs/refs-internal.h @@ -113,6 +113,7 @@ struct ref_update { void *backend_data; unsigned int type; char *msg; + char *committer_info; /* * If this ref_update was split off of a symref update via diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c index 647ef9b05b1dc9a376ed054330b487f7595c5caa..e882602487c66261d586a94101bb1b4e9a2ed60e 100644 --- a/refs/reftable-backend.c +++ b/refs/reftable-backend.c @@ -1379,11 +1379,21 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data } if (create_reflog) { + struct ident_split c; + ALLOC_GROW(logs, logs_nr + 1, logs_alloc); log = &logs[logs_nr++]; memset(log, 0, sizeof(*log)); - fill_reftable_log_record(log, &committer_ident); + if (u->committer_info) { + if (split_ident_line(&c, u->committer_info, + strlen(u->committer_info))) + BUG("failed splitting committer info"); + } else { + c = committer_ident; + } + + fill_reftable_log_record(log, &c); log->update_index = ts; log->refname = xstrdup(u->refname); memcpy(log->value.update.new_hash, From patchwork Mon Dec 9 11:07:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karthik Nayak X-Patchwork-Id: 13899320 Received: from mail-ej1-f50.google.com (mail-ej1-f50.google.com [209.85.218.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ABFF621B91C for ; Mon, 9 Dec 2024 11:07:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733742450; cv=none; b=cT0+/dxqvgAH5MPNf7E14+RgAeMbGKIOG7vCAO3DzfOESEOb+6ad+jl432qX3iXTEevYDMbFhCdncBZyrSGH7AVXpcsbi7KCNC5ra4zid4v2/EfAf4RC7fn+XJk1s3vE0AVTDnAUvmTf3zCgYjiThyidRP1Xay/5i1mWZLCEgu8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733742450; c=relaxed/simple; bh=IB9h00v4zfs7q+FeZZk33xlgGFW2QmXAcKQGj56BkwY=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=sd7e10EQyBp7NCPedZZ8T/GshJ45cFQlI/HzcGfcjtwkWa8Okumtm0Z9OAil7p3I3FouX4FaxpW4+jCI/ccbqcIOmeRqauNazejAYPjSTeK2C66mF3VfxW36j2GPgHjPAx5RAiLTw98yfemB3aiUCUlXyltw30xcqyC/jrttaWI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=b4UWgE+N; arc=none smtp.client-ip=209.85.218.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="b4UWgE+N" Received: by mail-ej1-f50.google.com with SMTP id a640c23a62f3a-aa64f3c5a05so350227966b.3 for ; Mon, 09 Dec 2024 03:07:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1733742445; x=1734347245; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=lUvb5dIPp19cc7hdfTanzPMDqohb3VapJTcypa4CDlg=; b=b4UWgE+NrX5SCdAQoyVYyhwZzSfHPWoCWugfoD1/VJ7RJSnna6VgitUuI8wO/HL52K efrYSnILrwdTQmGYKWbEynONChoGtnpkzC0bnfAZg1J1qyQEEsuH7T3cdfyOtEmC6ps3 uaECc0vclO1PbGXtr9I224ELEXS3BQV71y/7oNnNlPRIWqTSzicbVEbxIGInLk7cO0v2 1v/dwvm9tulM8pXXohVSqINn+8O+96qWrL0sduNOTpvbi3xGG29oqsXfbLSqcFTLYgLt nKDmKd+KixhD78NHi3U30gkLjNlh12SBwemZ+83h0B4wpS79o3hty3PYn9UZKCyqOLn5 AwzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733742445; x=1734347245; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=lUvb5dIPp19cc7hdfTanzPMDqohb3VapJTcypa4CDlg=; b=rHmqbGnKWh8n/08Ukhesyu0X41dIFVmfNHuxN+7zX+ZcthgW2r3Y2B0EuzLGxqGofZ E51G5W/aIyzpg+dfFdWa5lPNDDhlLS99F8Ab7d27OPwI/fu6jnuJ0mYREwXY1L6VOjnU lsW0lQpIhU5QBIGb6Z6pN7LrOPUzyOf0m8zhvlmY3Kch9cKvVEK2P6F73RtGAN6HdO/U UO60ilrPvS6C9IAnUfOK0sADN10vhTqTeLIAamWemhppk0cdyKWDOVGgGuxW/1kdtupN H5Z6CzrK9IynTnLUyioG7Nw7GAMZt3qOVLkUmqVobFpinFQbqhxDEgbPv81lumMLuQaM UWHA== X-Gm-Message-State: AOJu0YzwmawWk/PZz1n+MEtDczwhjr7ktVpHqkIDLotZs6UzfgHNRKpS xGq0kmymPOzEJWP9kyPxwJW4JQKjYa3AGtt+wBTE9QdAeQ6rjhx5vEB7NrOo X-Gm-Gg: ASbGncuPQVQRXOsA4nJ9ka91W7r85Z3CqiAPxEd2XQbjmCbPwQfX1vU6ygIOUhT1i0Y Z3Sq3fHRkdG7QgMOqdF/wra/saLd34WQoy9bHHMgS2Mrfo1lrAcS6fXpVQqqskzBKHf/CnZl2If uIzGuNwxK2t0u53X8X0cxVLiNiACpqfidmcxJa+c17gLEWsMrscpMQ8vqQ7dYixcmB1p6mYmG11 yfjis5c7+eu4tgrXJ8zYtJFuVOLQqUYZxp8aIJPxmjZyBaN2YNsnkntpQwT6ZY= X-Google-Smtp-Source: AGHT+IGivM0pKNa6dg6VN+MmXHpcvUfNkaDjY62c2dF5lN+hdkxjJgm+iaxjaFlu3vQWH+WhHlIerA== X-Received: by 2002:a17:906:23e9:b0:aa6:8ce6:1928 with SMTP id a640c23a62f3a-aa68ce619ddmr249798866b.48.1733742444220; Mon, 09 Dec 2024 03:07:24 -0800 (PST) Received: from [192.168.178.20] ([2a02:2455:8256:2d00:9c39:c2d7:aedd:294d]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aa672d377fbsm257013366b.75.2024.12.09.03.07.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Dec 2024 03:07:23 -0800 (PST) From: Karthik Nayak Date: Mon, 09 Dec 2024 12:07:16 +0100 Subject: [PATCH 2/7] refs: add `index` field to `struct ref_udpate` Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241209-320-git-refs-migrate-reflogs-v1-2-d4bc37ee860f@gmail.com> References: <20241209-320-git-refs-migrate-reflogs-v1-0-d4bc37ee860f@gmail.com> In-Reply-To: <20241209-320-git-refs-migrate-reflogs-v1-0-d4bc37ee860f@gmail.com> To: git@vger.kernel.org Cc: Karthik Nayak , toon@iotcl.com, Christian Couder X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=2464; i=karthik.188@gmail.com; h=from:subject:message-id; bh=IB9h00v4zfs7q+FeZZk33xlgGFW2QmXAcKQGj56BkwY=; b=owEB7QES/pANAwAIAT7VnySORox/AcsmYgBnVs9nG+ho1weYLZZriqxo2sVBFAN1zW+svdMi+ zUwXj5aHaiJAbMEAAEIAB0WIQRXzkx/Y3VxD8tlxgY+1Z8kjkaMfwUCZ1bPZwAKCRA+1Z8kjkaM f79RC/9u7GVl6hptX9bIn5PXVkQ41tTdhRYisFi1S7G84pqfqwfBKUwekn6gkWtl8g5ILv8j2IU c1kSj8NYXESS/4abO62F/f/Ms/3/Dm2URPWcAzLARdgArkNX7kkdGWkGPi5E7s6tGW5D/ax/kbb HNQpRGBFqLX8mRXTxMxPfYh2CHF6KBz++x0GVVjSQJPHRCCpX/+OfX37rjyOESTSZpcAltQ+3dW Nh2UhTpXAv6sxgpv+VhVJ+Blh5g3ITwU6ACkuuNktuFLgElN6H4vTzxoCninc0UqQNsXqEuY2cN FD9nzQFvIM35cV7wtL/d2v4uW2uD7IjX+VgvPzWtdWBNJwN1og6XXcnjG5JSNEwZC1jZlmUr+V9 f1hUqCGOSB7bFMzzQv+OAT43in2C9nqtmI+MiWG/Cdtl5MJevDXGAc1MK8lkokTY+9A63cpNW1W apbkVTE0K075Ej1nBdO9N53/t6KjnEcQuQ94mzWvL1zpoIQ4cBkY5DsqcPQlb/JH9Ijaw= X-Developer-Key: i=karthik.188@gmail.com; a=openpgp; fpr=57CE4C7F6375710FCB65C6063ED59F248E468C7F The reftable backend, sorts its updates by refname before applying them, this ensures that the references are stored sorted. When migrating reflogs from one backend to another, the order of the reflogs must be maintained. Add a new `index` field to the `ref_update` struct to facilitate this. This field is used in the reftable backend's sort comparison function `transaction_update_cmp`, to ensure that indexed fields maintain their order. Signed-off-by: Karthik Nayak --- refs/refs-internal.h | 7 +++++++ refs/reftable-backend.c | 13 +++++++++++-- 2 files changed, 18 insertions(+), 2 deletions(-) diff --git a/refs/refs-internal.h b/refs/refs-internal.h index 0fd95cdacd99e4a728c22f5286f6b3f0f360c110..f5c733d099f0c6f1076a25f4f77d9d5eb345ec87 100644 --- a/refs/refs-internal.h +++ b/refs/refs-internal.h @@ -115,6 +115,13 @@ struct ref_update { char *msg; char *committer_info; + /* + * The index overrides the default sort algorithm. This is needed + * when migrating reflogs and we want to ensure we carry over the + * same order. + */ + unsigned int index; + /* * If this ref_update was split off of a symref update via * split_symref_update(), then this member points at that diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c index e882602487c66261d586a94101bb1b4e9a2ed60e..c008f20be719fec3af6a8f81c821cb9c263764d7 100644 --- a/refs/reftable-backend.c +++ b/refs/reftable-backend.c @@ -1279,8 +1279,17 @@ static int reftable_be_transaction_abort(struct ref_store *ref_store UNUSED, static int transaction_update_cmp(const void *a, const void *b) { - return strcmp(((struct reftable_transaction_update *)a)->update->refname, - ((struct reftable_transaction_update *)b)->update->refname); + struct reftable_transaction_update *update_a = (struct reftable_transaction_update *)a; + struct reftable_transaction_update *update_b = (struct reftable_transaction_update *)b; + + /* + * If there is an index set, it should take preference (default is 0). + * This ensures that updates with indexes are sorted amongst themselves. + */ + if (update_a->update->index || update_b->update->index) + return update_a->update->index - update_b->update->index; + + return strcmp(update_a->update->refname, update_b->update->refname); } static int write_transaction_table(struct reftable_writer *writer, void *cb_data) From patchwork Mon Dec 9 11:07:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karthik Nayak X-Patchwork-Id: 13899319 Received: from mail-wr1-f48.google.com (mail-wr1-f48.google.com [209.85.221.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2D63D21B8F0 for ; Mon, 9 Dec 2024 11:07:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.48 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733742450; cv=none; b=I0avW0JxmsDYI8zz5mxTfH7t6OXuuoSRg8ZfQk6UshXBhZpcW49rIjZ+1sf81jdKF2wLsB5mlAIHJuMsEQpwOOfBewzWjCrcR3HbmK4DPT0GBbb8HubTgCFMjMHeFZqgMnZMqLjJ5k+igHxtuRTquD7c8d5XcQRxyGw3uq9Qiqo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733742450; c=relaxed/simple; bh=OBQYAEKRSd2fBN/oYIGCe0psMJew4nwiAdtZVB0QIfQ=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=RI8HIu6ESv+dkI1QoDrQjx1F7EeW955ogrYij9xEDC5miqWJQTZ57FZIxqmCu/CR95B3bwAtblO85/sMGqqiEWQhcjPCzBUYqflxsNWwTSocOELA6MDfo0ylC9QIDdd6JNwkwOZ+SYEpsm3ZSwREn6LwTsNMiNzJk5mwpC/TQcA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=E7Hrho7w; arc=none smtp.client-ip=209.85.221.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="E7Hrho7w" Received: by mail-wr1-f48.google.com with SMTP id ffacd0b85a97d-385e2880606so3488388f8f.3 for ; Mon, 09 Dec 2024 03:07:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1733742445; x=1734347245; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=CCzOtJbDEsEaJ1UVqvV5ZSCi9dBAZdai9+heqibwEPw=; b=E7Hrho7wKlI9nvlge6FckW9p1mBWRrqhlE0Sxz/R8zAQ2mHAbGPorPUWoIU8jGk0OT IQd/IwNJeZmoeVBQPJXPswRHadDsFmvjNtxhic/49otwtRvhAys8IJCdMwcDjuY3JgRb ZKFVU5MA/ID/OaaTLLhlpObKjBIdpncl6Z/wW0+oiIqXMTvdR2QdPYBwajxlm0mXG1sJ DG7Ypq2xg9ncsxPLm94x/VMDnAOG+soZ3CQ8FeaEMd6j1j/Z7vievlwu0ZOpr2ibizLR oq4JgJxITWgBo1kD9WDZvsi8bX89aLHJmw0RsJ0wUmj4H3vhlQs0HFrZRwv3IDZn1q0l b2vw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733742445; x=1734347245; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=CCzOtJbDEsEaJ1UVqvV5ZSCi9dBAZdai9+heqibwEPw=; b=bCEIO5StIoRQ9M2TUn4HfAov5thmfZkisoyTLsCsjZBl8cyDp3UtAkpwaF/GOL1/G0 cTLzsMxDlYl6XO56n6jtvyw9dy0Ek2Ot8/Nbhpi0QYoCBwo/dJTHn6CnaoeJEKlDl6JY zebkCDNTv2Y815FrWxBm0RtJuViyeateWLNAahLYJK/y8l3FHeQvY2EvgTt5WZ5/c/tN 8FG4lJnJf0hPpVdh3U58hk+iILjB+U8K5PE/mX1vpUuAC4BkDfOdSPEMCFUVL+gnfdbr mJo6nSPnVugNum3X7M/CilhwBK4WLCgPIjsjFs6BegCcU2hZpgo6NdCfTcQPHBbYqQ/t /zAg== X-Gm-Message-State: AOJu0YzrlhbU6j0xohVKwYBJhWDbP8LOvU/JIfM2ftEM2LYGt04cu1Wb fZC5EL7Ehzd9lMHQ+azGfFG09iGCUQ5SPamdc+gF8H8o+GF4HorfmSYH7XP8 X-Gm-Gg: ASbGnctLM9r7LzLp06tVJhA2AV3TTi5V0ctmxgJivcGow7JIU64EivnNTirKNH21ltJ v6jLnb7Fs8I+kkX+l5OqtPrRB04Z9r0an7IaHeuuf18md9Ol02SfVS5tbz5O7EAn0hUOw6mHpF0 YIshC6JQjA9UpXDESpTFofYc8TwSHKjE0dKSkAmcHwDAvkZvTjmRPRhTxXPSjAA6/OyW/JJO8Wn stqsmHEFwsDNim+q0CZL3u6GCRI91M2z2hDPVOduIDD4Qett4ulOtVCpLqVn24= X-Google-Smtp-Source: AGHT+IFnSXSTOiiZH0DdlcVV809FelGCI1kJTnwUGwnVScAQHuzqOe+H0sIsu0cS43dr1GbpaTC5Cg== X-Received: by 2002:a5d:588f:0:b0:385:d143:138b with SMTP id ffacd0b85a97d-3862b3db6f0mr7785267f8f.51.1733742445120; Mon, 09 Dec 2024 03:07:25 -0800 (PST) Received: from [192.168.178.20] ([2a02:2455:8256:2d00:9c39:c2d7:aedd:294d]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aa672d377fbsm257013366b.75.2024.12.09.03.07.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Dec 2024 03:07:24 -0800 (PST) From: Karthik Nayak Date: Mon, 09 Dec 2024 12:07:17 +0100 Subject: [PATCH 3/7] refs/files: add count field to ref_lock Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241209-320-git-refs-migrate-reflogs-v1-3-d4bc37ee860f@gmail.com> References: <20241209-320-git-refs-migrate-reflogs-v1-0-d4bc37ee860f@gmail.com> In-Reply-To: <20241209-320-git-refs-migrate-reflogs-v1-0-d4bc37ee860f@gmail.com> To: git@vger.kernel.org Cc: Karthik Nayak , toon@iotcl.com, Christian Couder X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=5471; i=karthik.188@gmail.com; h=from:subject:message-id; bh=OBQYAEKRSd2fBN/oYIGCe0psMJew4nwiAdtZVB0QIfQ=; b=owEB7QES/pANAwAIAT7VnySORox/AcsmYgBnVs9nloZS/hXsd91A61GnfWEN2eW1GXa+r+vti b+LorWX//eJAbMEAAEIAB0WIQRXzkx/Y3VxD8tlxgY+1Z8kjkaMfwUCZ1bPZwAKCRA+1Z8kjkaM f23VC/4y2tYHVnGzuRijbx68CqsXanU7NRTuyEf62Hn/cSNqrZCpTVRPhtF3CXNIj8lWaxymzNh 7HNIoGk1JVHtp+iEZ0yax7Kq+lx+SRX3JzmBNhPqvYwQfjYedwYqbMSC906W2fleupmBQsAd9to S1fQuOFAlHkJCYOTHobT6wxkIBGJY0g+4rQ7umcu4PWRU4pC5CUq0slSpDLKSY0GK2CJAgzbnJ7 btoc47lwsUZlrypzl2hEB1kqYMLUMv67odGrlOoXG4bpnz0uGoSUNLO6ncLiTv19X3aGs4EDYxn y1QgpZ2ogg5kMEYeJiH5YMVjJcXfc6UGa0WGXhuM8089slvH+hXFJKd+lGVTyOwVF5vTG0B7rsu 8jdcLtjsU+Vzpbnj8aYg2hDlQzLlZa2YQc1y3aFqq6ybDs8/3zqdvXBLb/IRjDIpdRrmtzpTJ9r kilH3g+qHrWFE6li618QnEwm5iRKB9pWhzev7UTlCyuIaPFVLP78Q8nal0q/WTHJHhbH4= X-Developer-Key: i=karthik.188@gmail.com; a=openpgp; fpr=57CE4C7F6375710FCB65C6063ED59F248E468C7F When refs are updated in the files-backend, a lock is obtained for the corresponding file path. This is the case even for reflogs, i.e. a lock is obtained on the reference path instead of the reflog path. This works, since generally, reflogs are updated alongside the ref. The upcoming patches will add support for reflog updates in ref transaction. This means, in a particular transaction we want to have ref updates and reflog updates. For refs, in a given transaction there can only be one update. But, we can theoretically have multiple reflog updates in a given transaction. The current flow does not support this, because currently refs & reflogs are treated as a single entity and capture the lock together. To separate this, add a count field to ref_lock. With this, multiple updates can hold onto a single ref_lock and the lock will only be released when all of them release the lock. This patch only adds the `count` field to `ref_lock` and adds the logic to increment and decrement the lock. In a follow up commit, we'll separate the reflog update logic from ref updates and utilize this functionality. Signed-off-by: Karthik Nayak --- refs/files-backend.c | 59 +++++++++++++++++++++++++++++++++++----------------- 1 file changed, 40 insertions(+), 19 deletions(-) diff --git a/refs/files-backend.c b/refs/files-backend.c index 13f8539e6caa923cd4834775fcb0cd7f90d82014..9c929c1ac33bc62a75620e684a809d46b574f1c6 100644 --- a/refs/files-backend.c +++ b/refs/files-backend.c @@ -71,6 +71,8 @@ struct ref_lock { char *ref_name; struct lock_file lk; struct object_id old_oid; + /* count keeps track of users of the lock */ + unsigned int count; }; struct files_ref_store { @@ -638,9 +640,12 @@ int parse_loose_ref_contents(const struct git_hash_algo *algop, static void unlock_ref(struct ref_lock *lock) { - rollback_lock_file(&lock->lk); - free(lock->ref_name); - free(lock); + lock->count--; + if (!lock->count) { + rollback_lock_file(&lock->lk); + free(lock->ref_name); + free(lock); + } } /* @@ -696,6 +701,7 @@ static int lock_raw_ref(struct files_ref_store *refs, *lock_p = CALLOC_ARRAY(lock, 1); lock->ref_name = xstrdup(refname); + lock->count = 1; files_ref_path(refs, &ref_file, refname); retry: @@ -1169,6 +1175,7 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs, goto error_return; lock->ref_name = xstrdup(refname); + lock->count = 1; if (raceproof_create_file(ref_file.buf, create_reflock, &lock->lk)) { unable_to_lock_message(ref_file.buf, errno, err); @@ -2535,6 +2542,12 @@ static int check_old_oid(struct ref_update *update, struct object_id *oid, return -1; } +struct files_transaction_backend_data { + struct ref_transaction *packed_transaction; + int packed_refs_locked; + struct strmap ref_locks; +}; + /* * Prepare for carrying out update: * - Lock the reference referred to by update. @@ -2557,11 +2570,14 @@ static int lock_ref_for_update(struct files_ref_store *refs, { struct strbuf referent = STRBUF_INIT; int mustexist = ref_update_expects_existing_old_ref(update); + struct files_transaction_backend_data *backend_data; int ret = 0; struct ref_lock *lock; files_assert_main_repository(refs, "lock_ref_for_update"); + backend_data = transaction->backend_data; + if ((update->flags & REF_HAVE_NEW) && ref_update_has_null_new_value(update)) update->flags |= REF_DELETING; @@ -2572,18 +2588,25 @@ static int lock_ref_for_update(struct files_ref_store *refs, goto out; } - ret = lock_raw_ref(refs, update->refname, mustexist, - affected_refnames, - &lock, &referent, - &update->type, err); - if (ret) { - char *reason; + lock = strmap_get(&backend_data->ref_locks, update->refname); + if (lock) { + lock->count = lock->count + 1; + } else { + ret = lock_raw_ref(refs, update->refname, mustexist, + affected_refnames, + &lock, &referent, + &update->type, err); + if (ret) { + char *reason; + + reason = strbuf_detach(err, NULL); + strbuf_addf(err, "cannot lock ref '%s': %s", + ref_update_original_update_refname(update), reason); + free(reason); + goto out; + } - reason = strbuf_detach(err, NULL); - strbuf_addf(err, "cannot lock ref '%s': %s", - ref_update_original_update_refname(update), reason); - free(reason); - goto out; + strmap_put(&backend_data->ref_locks, update->refname, lock); } update->backend_data = lock; @@ -2730,11 +2753,6 @@ static int lock_ref_for_update(struct files_ref_store *refs, return ret; } -struct files_transaction_backend_data { - struct ref_transaction *packed_transaction; - int packed_refs_locked; -}; - /* * Unlock any references in `transaction` that are still locked, and * mark the transaction closed. @@ -2767,6 +2785,8 @@ static void files_transaction_cleanup(struct files_ref_store *refs, if (backend_data->packed_refs_locked) packed_refs_unlock(refs->packed_ref_store); + strmap_clear(&backend_data->ref_locks, 0); + free(backend_data); } @@ -2796,6 +2816,7 @@ static int files_transaction_prepare(struct ref_store *ref_store, goto cleanup; CALLOC_ARRAY(backend_data, 1); + strmap_init(&backend_data->ref_locks); transaction->backend_data = backend_data; /* From patchwork Mon Dec 9 11:07:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karthik Nayak X-Patchwork-Id: 13899321 Received: from mail-ed1-f49.google.com (mail-ed1-f49.google.com [209.85.208.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 29E7A21B91E for ; Mon, 9 Dec 2024 11:07:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.49 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733742451; cv=none; b=WqCsvgolkLFWa31wc7aAHnLpLa/5trNPCKUogYbDU5bMiW0lIr5QV/PEESPKo7aHAesQd3hl5ne7WMyhLA+zc/7qjD3HxhuHSspWJ6frNj0ca5YbVEiyKl0aC3D4IpTYVBoxjnLR7VFJ338yL1pifSyrOpxBxu7uHTvrxFgN4Qw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733742451; c=relaxed/simple; bh=YCQQd9ym7xx16rPamjRXYJWagS28PovIK7X844HDYxs=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=JN/jb+3SfiC2/7/0wMC+R4Alza47J9QTl522XDuV22Atsq7sp3+Z/kdHC8X9T4qIEgrA8TJHXuNFlm+sWtUlrNm2OrzARjky1qOFaiQxpP3d9+zlSuKHgJq6/I1OrzPbln5p62IZPbvivkeXuh64zgXUY3jd3Jw9lJcmKCPEoiU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=UATBTITx; arc=none smtp.client-ip=209.85.208.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="UATBTITx" Received: by mail-ed1-f49.google.com with SMTP id 4fb4d7f45d1cf-5d414b8af7bso339278a12.0 for ; Mon, 09 Dec 2024 03:07:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1733742446; x=1734347246; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=DjzDeyeOaWumKUh5rU1ojNga2tt2q8VFzErpCwuxoMk=; b=UATBTITxkVBgy9wcL23vG/bqmZUbi0LEek15Q4jSoz9pzz1BADrGDsGwA7iE0HPN7k 4y/zQtTgqg4pe9fuNFgKCGdzNk5Xc1rRXWSOraxXoe5exitlrPzcC9OjQgXPswnCQCmP LKD1+/k2ey3vSS9geUnENFN8fQRSChvlyd/WWU4uoz4we75Uol2I0QeGDhr1c40D/Rvu CC8XAg7im/M+1OzKLRJnGZIIcrZww17crX25T1Bwub9XUo2m62/vAQODgAGRVRIZrOeP QO2yL+H8EQdv4jCQPA/Mjc73PgWeYQVYTVXaIqsiNrbwya//dqiD+jI9yFdCcdllVwDN vEAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733742446; x=1734347246; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=DjzDeyeOaWumKUh5rU1ojNga2tt2q8VFzErpCwuxoMk=; b=R3y68F0H2YPpVP8Cg+wxW0+E3s7F2uMek2CPkioQiu4yOwiEh0vKqDS89A5/PAUk5t MTxGDxKHN1pOh10vhbjU31Nw7tnmD4kvZBP04sk6W2K5RCdCrLbOO//vmobLDP8ByxMg nVrpdlsvzNopscwWM5nWFRpDIuH/qcjuajE+suGlI2bGJdVhRchZ9IeVY6i4PeUhuj9g m3HMnvBOT+spyqXyAZTKep6JCS6b87mXJvZhVl+ivk/0ts6osczp7ykaNMuDnDe0ACSf F0kn5Xjro2shlrT5R/D3Mf5A7xLcfmE+01sd79a4lrFcjZltb5Y27u98HVJKgTOciKDg Jrpw== X-Gm-Message-State: AOJu0YwUAoR65BTn8Z5fc2k7Fhl0l3MVO+kAUwYFaJ6p1+aIArw2dQOD 6g6OcLREeqBS8d61jC3cF8SAHP1aqfhg9diMJfCBZgwc23yn/MIaHzrPf5h0 X-Gm-Gg: ASbGnctcFoZECOlu9ORyUkxqBcyqe9MnTPVoO5S42L6CBvJogF/TdOuYU0Zw2BCHvQV EP9dE0S5uK/cFknj4Eq6qYIpmgWrALmcKRVrfgsRdZszsRli9arNqH/PPrKfFLBc05d9fl6isdd JSrOyGo24I41dRKFP9wyhokU3iDMoTJfsu159ubG8E4s6+gJ440y7e0AoNIGXHsEnbYjM886moS cuF6HCCoRB0fKx4qmogwxItJQEbRZrOpTsuaWxDbQdGMSUmZDKbkLg/wR84Bac= X-Google-Smtp-Source: AGHT+IFHQ2AuiUUmt0nkzl/AANEuKrWbVwHJx8ziaCKvBsyjga0hBSojqdVmGKQ9OIUwoPyzyVKUww== X-Received: by 2002:a17:907:9145:b0:aa6:8f39:5c with SMTP id a640c23a62f3a-aa68f3913c7mr180584366b.17.1733742445921; Mon, 09 Dec 2024 03:07:25 -0800 (PST) Received: from [192.168.178.20] ([2a02:2455:8256:2d00:9c39:c2d7:aedd:294d]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aa672d377fbsm257013366b.75.2024.12.09.03.07.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Dec 2024 03:07:25 -0800 (PST) From: Karthik Nayak Date: Mon, 09 Dec 2024 12:07:18 +0100 Subject: [PATCH 4/7] refs: extract out refname verification in transactions Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241209-320-git-refs-migrate-reflogs-v1-4-d4bc37ee860f@gmail.com> References: <20241209-320-git-refs-migrate-reflogs-v1-0-d4bc37ee860f@gmail.com> In-Reply-To: <20241209-320-git-refs-migrate-reflogs-v1-0-d4bc37ee860f@gmail.com> To: git@vger.kernel.org Cc: Karthik Nayak , toon@iotcl.com, Christian Couder X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=2669; i=karthik.188@gmail.com; h=from:subject:message-id; bh=YCQQd9ym7xx16rPamjRXYJWagS28PovIK7X844HDYxs=; b=owEB7QES/pANAwAIAT7VnySORox/AcsmYgBnVs9nUeZbXtgT6WErZe6adHPZCsBfQfQJvcTJ+ ThLyLD06UOJAbMEAAEIAB0WIQRXzkx/Y3VxD8tlxgY+1Z8kjkaMfwUCZ1bPZwAKCRA+1Z8kjkaM f8P1C/9uRx518JkTI4lNKuEsTkSbbpd4863z2iawx95Ooa1PFIbfn8zJ7yZO3n37kAryaqwaXCw a+MS3R0qRjVdQxox9ryZ0amNTNNUQ30bRep12k9aMcU2nuZZawd+LyaEAvdjbSvAizoNEk4SdUZ ZcW7vtiomXvZq7OvXb8+qKpmXGHExw5+Hn7Bx/7EWvyg/wxkJ5M6mbId8UAebx3DaXL0YUsUKlv /TmFACbQHJAfHgCum+jhLuh0V1xKqwF7L+cEG02hJNmVvmsn2ivJn7Jw4EKTPr87ktMimEgM4zg GJJkxVO8122soiVyr+d/fxDsuPX1rFwYODLZqYVWuWpDgSHZRYrHq1GeWGjdJO6klDap5c3+Kt4 dZivWdxQUOvzcb0fEVlD5rLpkYvRRifjgszRy17r47+RCgQX7z8H2xZBi2554gPgJr01jll5wCV WJE79uKzpSMO+9T5dMfJ/jpOTIWT85/IeySYIWeNZbOhHODNSJVjRyHKxiitm72noCasA= X-Developer-Key: i=karthik.188@gmail.com; a=openpgp; fpr=57CE4C7F6375710FCB65C6063ED59F248E468C7F Unless the `REF_SKIP_REFNAME_VERIFICATION` flag is set for an update, the refname of the update is verified for: - Ensuring it is not a pseudoref. - Checking the refname format. These checks are also be needed in a following commit where the function to add reflog updates to the transaction is introduced. Extract the code out into a new static function. Signed-off-by: Karthik Nayak --- refs.c | 43 ++++++++++++++++++++++++++++--------------- 1 file changed, 28 insertions(+), 15 deletions(-) diff --git a/refs.c b/refs.c index f003e51c6bf5229bfbce8ce61ffad7cdba0572e0..732c236a3fd0cf324cc172b48d3d54f6dbadf4a4 100644 --- a/refs.c +++ b/refs.c @@ -1196,6 +1196,29 @@ struct ref_update *ref_transaction_add_update( return update; } +static int transaction_refname_verification(const char *refname, + const struct object_id *new_oid, + unsigned int flags, + struct strbuf *err) +{ + if (flags & REF_SKIP_REFNAME_VERIFICATION) + return 0; + + if (is_pseudo_ref(refname)) { + strbuf_addf(err, _("refusing to update pseudoref '%s'"), + refname); + return -1; + } else if ((new_oid && !is_null_oid(new_oid)) ? + check_refname_format(refname, REFNAME_ALLOW_ONELEVEL) : + !refname_is_safe(refname)) { + strbuf_addf(err, _("refusing to update ref with bad name '%s'"), + refname); + return -1; + } + + return 0; +} + int ref_transaction_update(struct ref_transaction *transaction, const char *refname, const struct object_id *new_oid, @@ -1205,6 +1228,8 @@ int ref_transaction_update(struct ref_transaction *transaction, unsigned int flags, const char *msg, struct strbuf *err) { + int ret; + assert(err); if ((flags & REF_FORCE_CREATE_REFLOG) && @@ -1213,21 +1238,9 @@ int ref_transaction_update(struct ref_transaction *transaction, return -1; } - if (!(flags & REF_SKIP_REFNAME_VERIFICATION) && - ((new_oid && !is_null_oid(new_oid)) ? - check_refname_format(refname, REFNAME_ALLOW_ONELEVEL) : - !refname_is_safe(refname))) { - strbuf_addf(err, _("refusing to update ref with bad name '%s'"), - refname); - return -1; - } - - if (!(flags & REF_SKIP_REFNAME_VERIFICATION) && - is_pseudo_ref(refname)) { - strbuf_addf(err, _("refusing to update pseudoref '%s'"), - refname); - return -1; - } + ret = transaction_refname_verification(refname, new_oid, flags, err); + if (ret) + return ret; if (flags & ~REF_TRANSACTION_UPDATE_ALLOWED_FLAGS) BUG("illegal flags 0x%x passed to ref_transaction_update()", flags); From patchwork Mon Dec 9 11:07:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karthik Nayak X-Patchwork-Id: 13899322 Received: from mail-ej1-f49.google.com (mail-ej1-f49.google.com [209.85.218.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 131EB21B8F5 for ; Mon, 9 Dec 2024 11:07:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.49 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733742451; cv=none; b=P06gQBVJuZOgtHg3hJG0zxIXSXKQSgFjkax5Kc5TDLsMUA0px146ctL6AYlYclJvtZIu8AwXzf6d1FwQu1r9vQpKjp6KligL7LGFBr3lLZItJmkeyw0BRatksBNtiLOE1O2r8u8NRKyILScbS/sBNsiYui7vB4KQoshdPoiTBcc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733742451; c=relaxed/simple; bh=7cR6epBjK5+cted6JKC8pV6nxzM8nDK5/Q4THQFp+hU=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=mXUDnxoRby9FG1/kFTG3y8U/Z1R6FJFnLzGUFutS68KXA83jNWSpYWT7tcfLgDFD02wNdfqeCEbJGqCp0HZKyngvxU/RgvNZUovBTawMZTOR4IUiaTnRlXSh2ZCsaElz/lCFhmN9TfQ7cEPdYivk0Xh8WpngVX8yKE5R/q7M5F4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=cqc/AlBC; arc=none smtp.client-ip=209.85.218.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="cqc/AlBC" Received: by mail-ej1-f49.google.com with SMTP id a640c23a62f3a-aa6997f33e4so35582166b.3 for ; Mon, 09 Dec 2024 03:07:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1733742447; x=1734347247; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=UYHhFcf2edhwtlmaD7CZzQxwc9C/jKZzn5dUQh3fjkc=; b=cqc/AlBCVR821o2fkmbcCFC2iHkXJMBd1npAGI50Cc0TAzuxME4Wyb9NkwTyzXvVpU KQIjgK97OV2GSjuwVRI4jbHSm8AUXImX6hQ6YjMT9tZJhS0Gj94JSJEUGv+bkibg1hz2 j6BzrdnP78jsLTPfslF0wy3YyYHKoyjg+9YG652V1mQArCGBRNjmVbg4CgcIsMDflETM og7E2ZbP/xM6F1iWH9prAZa+GKX+wvx+172A/WTbtHCtPUPuy6kFpiE7TC0XOB5kGqVg ze2ztlRP5/dfU8XVMyt6ssrT0o5dqw9xtnaJBAKDUz4r6FZe59LP3ZV3GhijrY7JD4pN qzqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733742447; x=1734347247; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UYHhFcf2edhwtlmaD7CZzQxwc9C/jKZzn5dUQh3fjkc=; b=MLB68qyH1bTTBBmwAhMIGw6msVHImsCyd/DVchiewZk2i1OOGofAYOOpgFTD/QDd9z kQVwC2PLkBTyh+ZTljgOvwv8GqwDTeiXsEdvTNYJgX4y/SB+vE1ypYxjewQ5r3wPa5nj 4dy0pli+SXiTNsOGWYGpTmagLHieiHdPWX4xZyuBWMpUEG+rY00JQzrSUufdlX/wpwAu Vz0ciws7k+xqHUcCw0eufHWwvMu7L74Mtt1Vgbf4eOKZgswxx7gOmt2ymzDUCmOUGk68 6NBz0M4KZ43tiWpgsFSWt4UJP+ejri9kjjNkDxcHZMOD5e2B5ey99TdyT3Mf0r7+KLuP CjxA== X-Gm-Message-State: AOJu0Yx/8WNRf4o1fESJitTJlZpQFZPYwIcGjrQ78eOn+vBxvdnUSvvA kXfuRZOkeJB+SbyEajuv8ZC18BRzb2COfDYWnaLA5GjJHSXLBHDsQHhZQvrK X-Gm-Gg: ASbGncuENbAcJdqMkNEm9tc4e9j+9wgdMy8TRWSCCf8k9jhejAGGfrodvIngoNUTBfU RLtEIVP0hhtpWsjvLdJQnJ6fiOxne4biOnPi62rH6FpLn42rJ/JJKKy4dPhSXRX5u0XLtPtAKNe IGo6E2i1zUhqKyDFwibpusr0nDllfJxrCUdiMutx7WjowL8Nf9igvKx/0rndI4zaUqVRPrYtt/6 NAYpNUzIhmBGrB+Vp3t4J/lEuiIP7PKJ+cClyNci2rfj6/58Jp8cyBBu8Izj4I= X-Google-Smtp-Source: AGHT+IGG/t7PjiinfWcRs1DqA4NC5z/0CB4WM1MUP7WS+n4KcCWyWT83JWfzTPgWrsy6xhHQJlYwvA== X-Received: by 2002:a17:906:1da1:b0:aa6:33cf:b389 with SMTP id a640c23a62f3a-aa63a09beffmr1047699166b.34.1733742446734; Mon, 09 Dec 2024 03:07:26 -0800 (PST) Received: from [192.168.178.20] ([2a02:2455:8256:2d00:9c39:c2d7:aedd:294d]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aa672d377fbsm257013366b.75.2024.12.09.03.07.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Dec 2024 03:07:26 -0800 (PST) From: Karthik Nayak Date: Mon, 09 Dec 2024 12:07:19 +0100 Subject: [PATCH 5/7] refs: introduce the `ref_transaction_update_reflog` function Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241209-320-git-refs-migrate-reflogs-v1-5-d4bc37ee860f@gmail.com> References: <20241209-320-git-refs-migrate-reflogs-v1-0-d4bc37ee860f@gmail.com> In-Reply-To: <20241209-320-git-refs-migrate-reflogs-v1-0-d4bc37ee860f@gmail.com> To: git@vger.kernel.org Cc: Karthik Nayak , toon@iotcl.com, Christian Couder X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=12796; i=karthik.188@gmail.com; h=from:subject:message-id; bh=7cR6epBjK5+cted6JKC8pV6nxzM8nDK5/Q4THQFp+hU=; b=owEB7QES/pANAwAIAT7VnySORox/AcsmYgBnVs9nUC4aA8kuB8KWiMLJcOQXW2odEaMtyO/7c GMgwsFceEyJAbMEAAEIAB0WIQRXzkx/Y3VxD8tlxgY+1Z8kjkaMfwUCZ1bPZwAKCRA+1Z8kjkaM f02KC/4ud6Ozt2iAZW1hKiN2SXGyAcmvRKvS235hyF7IquCU0NZwKqapBplNOWwnMvkT7ioomgc gQ3YZMCW1+kYSW4yIF6y6fTjO7UHxOHcO1buL5F/LmcLAgLHGtbTbTEUY0Lkl82OLurBPM/t7Dt c0YzfyQIb4+etVhP/SMIp3cVApCfPVrvAhMmyyfdm6C2xodisnA0bUBqX8TQbB3ygmDfxOJ3q8C xHwhvM8NX0b2FtP4ySals/S9SOCivjDr91gvgT/5MJbsCQHNKXJlxZKEqpmlIsC6z68/LDitphb lhF3yvhavZo/9Oj2MAUSQeMQhBr37iNfwjJ5H7G9BDXvU6sBp9RBDC7zosqF1kVocfXOwQ9aqOC ivhQUt81O7EBFl/lJoTd11itiogs3YDIJxQ5ty5K8Ule3/KQ8Yu3w4qWWN0YKwwgUGvUqQKyClu TH+krJnGoL64vgqYPAMlqdjtZgO4Nlrm6N7aosGVh5BXCDExISIfyah+KMTCwJpFz73cI= X-Developer-Key: i=karthik.188@gmail.com; a=openpgp; fpr=57CE4C7F6375710FCB65C6063ED59F248E468C7F Introduce a new function `ref_transaction_update_reflog`, for clients to add a reflog update to a transaction. While the existing function `ref_transaction_update` also allows clients to add a reflog entry, this function does a few things more, It: - Enforces that only a reflog entry is added and does not update the ref itself. - Allows the users to also provide the committer information. This means clients can add reflog entries with custom committer information. A follow up commit will utilize this function to add reflog support to `git refs migrate`. Signed-off-by: Karthik Nayak --- refs.c | 89 +++++++++++++++++++++++++++++++++++++------------ refs.h | 12 +++++++ refs/files-backend.c | 48 +++++++++++++++----------- refs/refs-internal.h | 16 +++++---- refs/reftable-backend.c | 6 ++-- 5 files changed, 122 insertions(+), 49 deletions(-) diff --git a/refs.c b/refs.c index 732c236a3fd0cf324cc172b48d3d54f6dbadf4a4..602a65873181a90751def525608a7fa7bea59562 100644 --- a/refs.c +++ b/refs.c @@ -1160,13 +1160,15 @@ void ref_transaction_free(struct ref_transaction *transaction) free(transaction); } -struct ref_update *ref_transaction_add_update( - struct ref_transaction *transaction, - const char *refname, unsigned int flags, - const struct object_id *new_oid, - const struct object_id *old_oid, - const char *new_target, const char *old_target, - const char *msg) +struct ref_update *ref_transaction_add_update(struct ref_transaction *transaction, + const char *refname, + unsigned int flags, + const struct object_id *new_oid, + const struct object_id *old_oid, + const char *new_target, + const char *old_target, + const char *committer_info, + const char *msg) { struct ref_update *update; @@ -1190,8 +1192,15 @@ struct ref_update *ref_transaction_add_update( oidcpy(&update->new_oid, new_oid); if ((flags & REF_HAVE_OLD) && old_oid) oidcpy(&update->old_oid, old_oid); - if (!(flags & REF_SKIP_CREATE_REFLOG)) + if (!(flags & REF_SKIP_CREATE_REFLOG)) { + if (committer_info) { + struct strbuf sb = STRBUF_INIT; + strbuf_addstr(&sb, committer_info); + update->committer_info = strbuf_detach(&sb, NULL); + } + update->msg = normalize_reflog_message(msg); + } return update; } @@ -1199,20 +1208,29 @@ struct ref_update *ref_transaction_add_update( static int transaction_refname_verification(const char *refname, const struct object_id *new_oid, unsigned int flags, + unsigned int reflog, struct strbuf *err) { if (flags & REF_SKIP_REFNAME_VERIFICATION) return 0; if (is_pseudo_ref(refname)) { - strbuf_addf(err, _("refusing to update pseudoref '%s'"), - refname); + if (reflog) + strbuf_addf(err, _("refusing to update reflog for pseudoref '%s'"), + refname); + else + strbuf_addf(err, _("refusing to update pseudoref '%s'"), + refname); return -1; } else if ((new_oid && !is_null_oid(new_oid)) ? check_refname_format(refname, REFNAME_ALLOW_ONELEVEL) : !refname_is_safe(refname)) { - strbuf_addf(err, _("refusing to update ref with bad name '%s'"), - refname); + if (reflog) + strbuf_addf(err, _("refusing to update reflog with bad name '%s'"), + refname); + else + strbuf_addf(err, _("refusing to update ref with bad name '%s'"), + refname); return -1; } @@ -1238,7 +1256,7 @@ int ref_transaction_update(struct ref_transaction *transaction, return -1; } - ret = transaction_refname_verification(refname, new_oid, flags, err); + ret = transaction_refname_verification(refname, new_oid, flags, 0, err); if (ret) return ret; @@ -1255,18 +1273,47 @@ int ref_transaction_update(struct ref_transaction *transaction, flags |= (new_oid ? REF_HAVE_NEW : 0) | (old_oid ? REF_HAVE_OLD : 0); flags |= (new_target ? REF_HAVE_NEW : 0) | (old_target ? REF_HAVE_OLD : 0); - ref_transaction_add_update(transaction, refname, flags, - new_oid, old_oid, new_target, - old_target, msg); + ref_transaction_add_update(transaction, refname, flags, new_oid, + old_oid, new_target, old_target, NULL, msg); + return 0; +} + +int ref_transaction_update_reflog(struct ref_transaction *transaction, + const char *refname, + const struct object_id *new_oid, + const struct object_id *old_oid, + const char *committer_info, unsigned int flags, + const char *msg, unsigned int index, + struct strbuf *err) +{ + struct ref_update *update; + int ret; + + assert(err); + + ret = transaction_refname_verification(refname, new_oid, flags, 1, err); + if (ret) + return ret; + + flags |= REF_LOG_ONLY | REF_NO_DEREF; + + update = ref_transaction_add_update(transaction, refname, flags, + new_oid, old_oid, NULL, NULL, + committer_info, msg); + /* + * While we do set the old_oid value, we unset the flag to skip + * old_oid verification which only makes sense for refs. + */ + update->flags &= ~REF_HAVE_OLD; + update->index = index; + return 0; } int ref_transaction_create(struct ref_transaction *transaction, - const char *refname, - const struct object_id *new_oid, - const char *new_target, - unsigned int flags, const char *msg, - struct strbuf *err) + const char *refname, const struct object_id *new_oid, + const char *new_target, unsigned int flags, + const char *msg, struct strbuf *err) { if (new_oid && new_target) BUG("create called with both new_oid and new_target set"); diff --git a/refs.h b/refs.h index a5bedf48cf6de91005a7e8d0bf58ca98350397a6..b86d2cd87be33f7bb1b31fce711d6c7c8d9491c9 100644 --- a/refs.h +++ b/refs.h @@ -727,6 +727,18 @@ int ref_transaction_update(struct ref_transaction *transaction, unsigned int flags, const char *msg, struct strbuf *err); +/* + * Similar to `ref_transaction_update`, but this function is only for adding + * a reflog updates. Supports providing custom committer information. + */ +int ref_transaction_update_reflog(struct ref_transaction *transaction, + const char *refname, + const struct object_id *new_oid, + const struct object_id *old_oid, + const char *committer_info, unsigned int flags, + const char *msg, unsigned int index, + struct strbuf *err); + /* * Add a reference creation to transaction. new_oid is the value that * the reference should have after the update; it must not be diff --git a/refs/files-backend.c b/refs/files-backend.c index 9c929c1ac33bc62a75620e684a809d46b574f1c6..32975e0fd7a03ab8ddf99c0a68af99921d3f5090 100644 --- a/refs/files-backend.c +++ b/refs/files-backend.c @@ -1268,10 +1268,10 @@ static void prune_ref(struct files_ref_store *refs, struct ref_to_prune *r) transaction = ref_store_transaction_begin(&refs->base, 0, &err); if (!transaction) goto cleanup; - ref_transaction_add_update( - transaction, r->name, - REF_NO_DEREF | REF_HAVE_NEW | REF_HAVE_OLD | REF_IS_PRUNING, - null_oid(), &r->oid, NULL, NULL, NULL); + ref_transaction_add_update(transaction, r->name, + REF_NO_DEREF | REF_HAVE_NEW | REF_HAVE_OLD | + REF_IS_PRUNING, null_oid(), &r->oid, NULL, + NULL, NULL, NULL); if (ref_transaction_commit(transaction, &err)) goto cleanup; @@ -2418,7 +2418,7 @@ static int split_head_update(struct ref_update *update, transaction, "HEAD", update->flags | REF_LOG_ONLY | REF_NO_DEREF, &update->new_oid, &update->old_oid, - NULL, NULL, update->msg); + NULL, NULL, update->committer_info, update->msg); /* * Add "HEAD". This insertion is O(N) in the transaction @@ -2482,7 +2482,8 @@ static int split_symref_update(struct ref_update *update, transaction, referent, new_flags, update->new_target ? NULL : &update->new_oid, update->old_target ? NULL : &update->old_oid, - update->new_target, update->old_target, update->msg); + update->new_target, update->old_target, NULL, + update->msg); new_update->parent_update = update; @@ -2911,11 +2912,11 @@ static int files_transaction_prepare(struct ref_store *ref_store, packed_transaction; } - ref_transaction_add_update( - packed_transaction, update->refname, - REF_HAVE_NEW | REF_NO_DEREF, - &update->new_oid, NULL, - NULL, NULL, NULL); + ref_transaction_add_update(packed_transaction, + update->refname, + REF_HAVE_NEW | REF_NO_DEREF, + &update->new_oid, NULL, NULL, + NULL, NULL, NULL); } } @@ -3080,10 +3081,12 @@ static int files_transaction_finish_initial(struct files_ref_store *refs, } /* - * packed-refs don't support symbolic refs and root refs, so we - * have to queue these references via the loose transaction. + * packed-refs don't support symbolic refs, root refs and reflogs, + * so we have to queue these references via the loose transaction. */ - if (update->new_target || is_root_ref(update->refname)) { + if (update->new_target || + is_root_ref(update->refname) || + (update->flags & REF_LOG_ONLY)) { if (!loose_transaction) { loose_transaction = ref_store_transaction_begin(&refs->base, 0, err); if (!loose_transaction) { @@ -3092,15 +3095,22 @@ static int files_transaction_finish_initial(struct files_ref_store *refs, } } - ref_transaction_add_update(loose_transaction, update->refname, - update->flags & ~REF_HAVE_OLD, - update->new_target ? NULL : &update->new_oid, NULL, - update->new_target, NULL, NULL); + if (update->flags & REF_LOG_ONLY) + ref_transaction_add_update(loose_transaction, update->refname, + update->flags, &update->new_oid, + &update->old_oid, NULL, NULL, + update->committer_info, update->msg); + else + ref_transaction_add_update(loose_transaction, update->refname, + update->flags & ~REF_HAVE_OLD, + update->new_target ? NULL : &update->new_oid, NULL, + update->new_target, NULL, update->committer_info, + NULL); } else { ref_transaction_add_update(packed_transaction, update->refname, update->flags & ~REF_HAVE_OLD, &update->new_oid, &update->old_oid, - NULL, NULL, NULL); + NULL, NULL, update->committer_info, NULL); } } diff --git a/refs/refs-internal.h b/refs/refs-internal.h index f5c733d099f0c6f1076a25f4f77d9d5eb345ec87..82c1387d1e6ab3658b31fe99c95f98645ff1ebf1 100644 --- a/refs/refs-internal.h +++ b/refs/refs-internal.h @@ -156,13 +156,15 @@ int ref_update_reject_duplicates(struct string_list *refnames, * dereferenced if the REF_HAVE_NEW and REF_HAVE_OLD bits, * respectively, are set in flags. */ -struct ref_update *ref_transaction_add_update( - struct ref_transaction *transaction, - const char *refname, unsigned int flags, - const struct object_id *new_oid, - const struct object_id *old_oid, - const char *new_target, const char *old_target, - const char *msg); +struct ref_update *ref_transaction_add_update(struct ref_transaction *transaction, + const char *refname, + unsigned int flags, + const struct object_id *new_oid, + const struct object_id *old_oid, + const char *new_target, + const char *old_target, + const char *committer_info, + const char *msg); /* * Transaction states. diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c index c008f20be719fec3af6a8f81c821cb9c263764d7..b2e3ba877de9e59fea5a4d066eb13e60ef22a32b 100644 --- a/refs/reftable-backend.c +++ b/refs/reftable-backend.c @@ -1078,7 +1078,8 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store, new_update = ref_transaction_add_update( transaction, "HEAD", u->flags | REF_LOG_ONLY | REF_NO_DEREF, - &u->new_oid, &u->old_oid, NULL, NULL, u->msg); + &u->new_oid, &u->old_oid, NULL, NULL, NULL, + u->msg); string_list_insert(&affected_refnames, new_update->refname); } @@ -1161,7 +1162,8 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store, transaction, referent.buf, new_flags, u->new_target ? NULL : &u->new_oid, u->old_target ? NULL : &u->old_oid, - u->new_target, u->old_target, u->msg); + u->new_target, u->old_target, + u->committer_info, u->msg); new_update->parent_update = u; From patchwork Mon Dec 9 11:07:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karthik Nayak X-Patchwork-Id: 13899323 Received: from mail-ej1-f53.google.com (mail-ej1-f53.google.com [209.85.218.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CFADB21C17C for ; Mon, 9 Dec 2024 11:07:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.53 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733742451; cv=none; b=r+7QBvCHPku7sm3ziWiVxFv6Qbshl3IEApVFD4yKXMGlouLZoOgM/e/dsWGu8ZkfY+WvGKCaHEDJoX904y1cPRNvM0wsV6jCM51AfxH9XVlKsju20lvyhFmQ/5BnqLuyYxggNL6BghNpPn+kEg5vO5cim6SXFZY/ZTtOo6QEBoI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733742451; c=relaxed/simple; bh=WGQz9kGlhiqH8Cl10higM364cpuFdIUZl1NSy3ZRwIg=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=oQ8tfOmKr1m9zGjbAj8gZmQ97CYXxI3gtUrnQ/g8GMek1tzOPpy7c/um/7nBVW8zaHpeDwrEmncFWOr+KdVFSAthIM3lrSN3vpC5pRoML/LLVtHCAF5LT+6MavVOFlrGVdhcwdrwyTJsKPNZfu3us9DqAX+GMGfRA0RB4kAqcLc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=iLU5VMKO; arc=none smtp.client-ip=209.85.218.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="iLU5VMKO" Received: by mail-ej1-f53.google.com with SMTP id a640c23a62f3a-aa66e4d1d5aso231790166b.2 for ; Mon, 09 Dec 2024 03:07:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1733742448; x=1734347248; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=mxj3xT5a2DekMcb+oyYc7qlHEsze3X91betL5eqZT28=; b=iLU5VMKOnInRX1wQSHeUMuQnrvCdbTTX0hfJJ0QhtxHMrWNoyKhj7Sxq9fyyYLAIyg VCIKlgYPViXVIwYPXKo/dRhIQZ696I//uyFXRJidfF65LKeUeG+KaL84MZYvCGDC2sNN s6D65UIWzY9HyH0LMdqlRaYks89aQ7V96wHyKFVLxVxuW/SkU25gEmwxKHxjJnzzlQsK bpdmvKEV2J1WZlhl6QcH891UC0m7oY9w4bTYoJpeNNIKHqddOLMv9FE0O1wzaJhMx0au GUuvzA2tUMsCzjgN1DGog5MEOcpGtrvP5hIATv7XNdYiRBfd4cKPCG+kkxSyM4TOWIHn QJ/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733742448; x=1734347248; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=mxj3xT5a2DekMcb+oyYc7qlHEsze3X91betL5eqZT28=; b=wkgLSdZJYZXg5xPzJZ17+hMjIvycCvEmMhub1Cb6kA0DfA/yv+WlHn6IOoGI8GPsnO 8ELij9ZTF8wEy/Ns4St9yh7GTOkRWod02p/tnHeyAKsq70IDhwvbv7RWPft4Hz/ElrLo DhBRJOK9d7giKYrvKbaIWEBIuRSxLPA5tIEf5aHJsXfo6EUwfs1UTeXf/dv+D0lkpyYJ RJ0ZVzQjN0oLfC59nha3UvkmXehUuxushRGmmvr9l5mOYXCpEB98/GM/qwXoe9CVRzxD o2fLUirJtIxAvjDiRGtvtmdauVKEvB+N3Exr6sqSI2YVJSReL4GFLzQ1tEEel/SYkcBR 4c7w== X-Gm-Message-State: AOJu0Yw988riH2FFIwM3eRPw1yjaOXrTRxJmfOVlblhsxXYSntXn1lnD alRFXfXPWKJAQPLBQxnGc5iLrdIp1Eb649ma6BkeHgA07+MwyQxqRlSYd8kU X-Gm-Gg: ASbGncuC4NqQ390SkC3HUC9/5nQy16/luqA1paH2VdM2BkAKIchAEpzdwvk7NGULXUZ DJOT0J4uz2PO8EDx508q/FfxZ7Ho9peFU/lysmVeCrwKB5TBCVA9LO5wtrzK6zePvWEhYDXEOe4 kKvziJjuxt8zv5B05ftsFW1+OzpnovUDLAU3VvwffE7ONu1I2rzOHcNuNC4gzZd1OUWFJyCSFkf +NYluEVdXQyoO+WTG+TzduUe1enFkXFjzLdTVMZP6u0nQk7bMCMGwp6LhIo/hc= X-Google-Smtp-Source: AGHT+IE74Lz9ugFcNGatg+1a3ZSJI9T4n755mo4wo2v1SwyZGixfYSUvJDELm4iFBG6bjgq8eWsqPg== X-Received: by 2002:a17:906:23e9:b0:aa6:8d51:8fdb with SMTP id a640c23a62f3a-aa68d519148mr252471766b.19.1733742447599; Mon, 09 Dec 2024 03:07:27 -0800 (PST) Received: from [192.168.178.20] ([2a02:2455:8256:2d00:9c39:c2d7:aedd:294d]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aa672d377fbsm257013366b.75.2024.12.09.03.07.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Dec 2024 03:07:27 -0800 (PST) From: Karthik Nayak Date: Mon, 09 Dec 2024 12:07:20 +0100 Subject: [PATCH 6/7] refs: allow multiple reflog entries for the same refname Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241209-320-git-refs-migrate-reflogs-v1-6-d4bc37ee860f@gmail.com> References: <20241209-320-git-refs-migrate-reflogs-v1-0-d4bc37ee860f@gmail.com> In-Reply-To: <20241209-320-git-refs-migrate-reflogs-v1-0-d4bc37ee860f@gmail.com> To: git@vger.kernel.org Cc: Karthik Nayak , toon@iotcl.com, Christian Couder X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=4867; i=karthik.188@gmail.com; h=from:subject:message-id; bh=WGQz9kGlhiqH8Cl10higM364cpuFdIUZl1NSy3ZRwIg=; b=owEB7AET/pANAwAIAT7VnySORox/AcsmYgBnVs9nU1I5k1k6BuP8eZCylXejKtGvaRsIhG9sZ 54TDSeDYU2JAbIEAAEIAB0WIQRXzkx/Y3VxD8tlxgY+1Z8kjkaMfwUCZ1bPZwAKCRA+1Z8kjkaM fx0BC/YmzyWedPtd/GpLQu00xB+SdOJxG5lxQOQa8k5DQg/g+1BTQIjgo6dPUlTViUHzHQV5zdg tMkB+MYNFhZ/hxFL7AHejODZB9xumPFdIUVbQlu0s94cEkiiVp1qXpBHKm7FPDLFiL5gkrSoxMQ mMsa+75/40TB/t9aBvDFXHmxZldsATPYje2Fnq98tD2yOk1/YpIUOus2dd49a0PKOM1a1ELwxbv kCR+sZjo/Rf5CO5G59y2KhpH24arVAWa2a6BGYL/TNkMP1tCcrbdMS6rxF86w9uWf2ezILyuuQe EOMuKgKX6ZQ+Pw999wCbrIBjxiw9qLBdXXAWAZVKxywxgONXYm6XctbhZnn0DKgk/urGGYFTswp Nmsbrf97k8COvzi1Hx4qdCq7+BmN+MF3Hvol2TQVAqHVc9TnXh6lcHg12pyE6AS0vmNw+V5t3WU q8577eEzy1geWVr2q1q5Dlc0k+66M+QhN2SJcVeaSVsLj5jnQJZDPSxD8Gp1wP74LT+A== X-Developer-Key: i=karthik.188@gmail.com; a=openpgp; fpr=57CE4C7F6375710FCB65C6063ED59F248E468C7F The reference transaction only allows a update for a given reference to avoid conflicts. This, however, isn't an issue for reflogs. There are no conflicts to be resolved in reflogs and when migrating reflogs between backends we'd have multiple reflog entries for the same refname. So allow multiple reflog updates within a single transaction. Also the reflog creation logic isn't exposed to the end user. While this might change in the future, currently, this reduces the scope of issues to think about. This is required to add reflog migration support to `git refs migrate` which currently doesn't support it. Signed-off-by: Karthik Nayak --- refs/files-backend.c | 15 +++++++++++---- refs/reftable-backend.c | 16 +++++++++++++--- 2 files changed, 24 insertions(+), 7 deletions(-) diff --git a/refs/files-backend.c b/refs/files-backend.c index 32975e0fd7a03ab8ddf99c0a68af99921d3f5090..10fba1e97b967fbc04c62a0a6d7d9648ce1c51fb 100644 --- a/refs/files-backend.c +++ b/refs/files-backend.c @@ -2612,6 +2612,9 @@ static int lock_ref_for_update(struct files_ref_store *refs, update->backend_data = lock; + if (update->flags & REF_LOG_ONLY) + goto out; + if (update->type & REF_ISSYMREF) { if (update->flags & REF_NO_DEREF) { /* @@ -2830,13 +2833,16 @@ static int files_transaction_prepare(struct ref_store *ref_store, */ for (i = 0; i < transaction->nr; i++) { struct ref_update *update = transaction->updates[i]; - struct string_list_item *item = - string_list_append(&affected_refnames, update->refname); + struct string_list_item *item; if ((update->flags & REF_IS_PRUNING) && !(update->flags & REF_NO_DEREF)) BUG("REF_IS_PRUNING set without REF_NO_DEREF"); + if (update->flags & REF_LOG_ONLY) + continue; + + item = string_list_append(&affected_refnames, update->refname); /* * We store a pointer to update in item->util, but at * the moment we never use the value of this field @@ -3036,8 +3042,9 @@ static int files_transaction_finish_initial(struct files_ref_store *refs, /* Fail if a refname appears more than once in the transaction: */ for (i = 0; i < transaction->nr; i++) - string_list_append(&affected_refnames, - transaction->updates[i]->refname); + if (!(transaction->updates[i]->flags & REF_LOG_ONLY)) + string_list_append(&affected_refnames, + transaction->updates[i]->refname); string_list_sort(&affected_refnames); if (ref_update_reject_duplicates(&affected_refnames, err)) { ret = TRANSACTION_GENERIC_ERROR; diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c index b2e3ba877de9e59fea5a4d066eb13e60ef22a32b..d9d2e28122a00ddd7f835c35a5851e390761885b 100644 --- a/refs/reftable-backend.c +++ b/refs/reftable-backend.c @@ -990,8 +990,9 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store, if (ret) goto done; - string_list_append(&affected_refnames, - transaction->updates[i]->refname); + if (!(transaction->updates[i]->flags & REF_LOG_ONLY)) + string_list_append(&affected_refnames, + transaction->updates[i]->refname); } /* @@ -1302,6 +1303,7 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data struct ident_split committer_ident = {0}; size_t logs_nr = 0, logs_alloc = 0, i; const char *committer_info; + struct strintmap logs_ts; int ret = 0; committer_info = git_committer_info(0); @@ -1310,6 +1312,8 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data QSORT(arg->updates, arg->updates_nr, transaction_update_cmp); + strintmap_init(&logs_ts, ts); + reftable_writer_set_limits(writer, ts, ts); for (i = 0; i < arg->updates_nr; i++) { @@ -1391,6 +1395,7 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data if (create_reflog) { struct ident_split c; + uint64_t update_index; ALLOC_GROW(logs, logs_nr + 1, logs_alloc); log = &logs[logs_nr++]; @@ -1405,7 +1410,11 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data } fill_reftable_log_record(log, &c); - log->update_index = ts; + + update_index = strintmap_get(&logs_ts, u->refname); + log->update_index = update_index; + strintmap_set(&logs_ts, u->refname, update_index+1); + log->refname = xstrdup(u->refname); memcpy(log->value.update.new_hash, u->new_oid.hash, GIT_MAX_RAWSZ); @@ -1476,6 +1485,7 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data done: assert(ret != REFTABLE_API_ERROR); + strintmap_clear(&logs_ts); for (i = 0; i < logs_nr; i++) reftable_log_record_release(&logs[i]); free(logs); From patchwork Mon Dec 9 11:07:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karthik Nayak X-Patchwork-Id: 13899324 Received: from mail-ej1-f47.google.com (mail-ej1-f47.google.com [209.85.218.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B6CED21C199 for ; Mon, 9 Dec 2024 11:07:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.47 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733742452; cv=none; b=eg5JDZ8IjTs6pHUe5gtqWByVUn2v3kGbyOhPuZGnAJ8GhYkmAFeknuEVa3TzEm/w4q+H8TbCZHXLHKGoZwiv6AYusfxG3arIFxFaCzHM3YUT5+GeyoRniYJPzJTVJd19aqo08GyGcjg1BaASiYWxMaucq6/lUmE8ssE5FrkjxaE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733742452; c=relaxed/simple; bh=dTMkc7fTSLRLIW8QEzzuJZURYqfHWikDw5kU0cagkdI=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=hF1MncOyaoCetj+XT/tkfuzJRh+zW344QyYLryaJQRi/9iTJa6aeAMOLAM/ag85HFcddu7QbkPc9CscGcgqvRWizlhI1MM/WuJr3j3subUKXB9vO8lYQ6xxQHzGlKVJ7ai2ZY1rz/J32n3EPos4UydGNlTpQJa77HvqUnr0+tpQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=B1hTg30e; arc=none smtp.client-ip=209.85.218.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="B1hTg30e" Received: by mail-ej1-f47.google.com with SMTP id a640c23a62f3a-aa6997f33e4so35587366b.3 for ; Mon, 09 Dec 2024 03:07:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1733742449; x=1734347249; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=nstbR1x/Ctx0E8lCauhKUrE1x/Cv1cSQvkF0rT8B7aE=; b=B1hTg30eV/w9covnBO3l4/TerLPUXOwLOg5dRcu1wbgk4bPkOz/bESRE2Jt3j63unp hcDDp/XNZICIn8qiDt8blbCPR005dY3L3+XnLxdgW/YN+BhMsNrJDbYCD+IHsBUhf9yD 7F4wJuaWiEpYdawQnv4xCrtAebng5vS5TukzwLjDnewCgy1LGMw+kqZA2kIcsEi/m80O +/2JUbjRr4fejH5A0QnddUXJsl8P8P35JaQjz+3as6RmYVhSk/1CVp3z86LYWZFDO9By 8dxLoVrO99jAcf834Neo6JQChJjauuJVnKpHkB5NJGQ8uRs0GMTRVoboOF7raD7iZzJg h8CA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733742449; x=1734347249; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nstbR1x/Ctx0E8lCauhKUrE1x/Cv1cSQvkF0rT8B7aE=; b=ve40iR2wVXooW/bZTKvKv3JOfIuQqaAHTTe7AE2iTzrK1hyJ1cRG8ZADIj9DxafWkZ 2re7MHrG8fIr2SSMyeTRppg1j7kePs/9PFAVd0tZ7owALOL6PACMJjD/gLk4OtogGvCK dhbNjGclwIIyTqtXm4SR2GMfG9KcvxJIeM/0W9YiheYAjh+zWVUB4OytdIgzcr500brq 5+TJdb6rJ7wJ1LtpXmdty3tO+WW/d2klwuBw06NFvIINODo3hByD90qJskuR+/JJAR+y u/02GjRZXorc3h0wakYzyIpt1RxfOencOGbDMGZn2QsVeb/2DGKpnn7bBGOufSd3BGhV c+aA== X-Gm-Message-State: AOJu0YxwJBfq9tmBC1Xop2wJ3DMjYKX4KHyF+9KLQjG2ts1fjUk6kmYc UPzGmi9v+PEmfaMoHSmGO8mq18IKBWCa0zQj4kVjlwXaLMxAD2Plq5T8aS4Y X-Gm-Gg: ASbGncvnlr3FZlvRiOaizF7Jehaa58LiHu9KlSikvab5SMHiMLK6kovoehU2cEqAwgc kyIRshYu9955v9opjOc2wwZoPZZv/6P5bLjmYEvSndYaIO0kGCMLQifshyyjNrOmbIJUpKUOjyr YT5QjBFqTsMx+28zfjYLx27vF107ZoLktbOknKKDPhokYlchtArpti2Hdsg5PqeLPweXzRYXGy/ 950KnXYSUcZGFsEViT+LXNjAMkbKP4kei6ZatjmKJonhMMw0Uvxj1pFV5tk7XM= X-Google-Smtp-Source: AGHT+IFFUB3YSHykz4WghB0Xg+gTeIk8/WcqDxCzxDh0hfs9GamrS4nm2SV2FGu0/HZR+0JlqglrOA== X-Received: by 2002:a17:907:75c2:b0:aa6:40a4:b13 with SMTP id a640c23a62f3a-aa640a40bdcmr1101085666b.59.1733742448704; Mon, 09 Dec 2024 03:07:28 -0800 (PST) Received: from [192.168.178.20] ([2a02:2455:8256:2d00:9c39:c2d7:aedd:294d]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aa672d377fbsm257013366b.75.2024.12.09.03.07.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Dec 2024 03:07:28 -0800 (PST) From: Karthik Nayak Date: Mon, 09 Dec 2024 12:07:21 +0100 Subject: [PATCH 7/7] refs: add support for migrating reflogs Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241209-320-git-refs-migrate-reflogs-v1-7-d4bc37ee860f@gmail.com> References: <20241209-320-git-refs-migrate-reflogs-v1-0-d4bc37ee860f@gmail.com> In-Reply-To: <20241209-320-git-refs-migrate-reflogs-v1-0-d4bc37ee860f@gmail.com> To: git@vger.kernel.org Cc: Karthik Nayak , toon@iotcl.com, Christian Couder X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=10371; i=karthik.188@gmail.com; h=from:subject:message-id; bh=dTMkc7fTSLRLIW8QEzzuJZURYqfHWikDw5kU0cagkdI=; b=owEB7QES/pANAwAIAT7VnySORox/AcsmYgBnVs9nsXvzILlQNXVz1PPrLehe00AsNv/rOy6IU rx7CLo5e7qJAbMEAAEIAB0WIQRXzkx/Y3VxD8tlxgY+1Z8kjkaMfwUCZ1bPZwAKCRA+1Z8kjkaM fxXnC/sHEdPXy+Xf9yswAcq8TWlqUErU9WsRhC72gshfDuBNrb7QZg1vTErTWEa4A4d070UFhei wDQLWQMGiXXIk62aaGOaol9S1HgMuvZUjStZvQd5D1Nr1ShHjby1qx/BAuMePl1F7pMLl34coq5 SclgrbqgJ65w9C69erCXTU/YP+J3Fu1FfSCMZRory5k9mve64yYagoyZg02Nq1GKqioAZV8PvC4 Me9MXl4iiS6pNjKa+14bJy7d4+9NvM42+D8kv+OPvXgfKxVYs/S+JSnqidigQrpJ9uBBZuJC2bK QXjCBqXGT4wyNBjMpvZ0cYQmYRjxxTBO5vW9Yxp7HD297pjq0W5aj723/DmCwQlv0xMB326DPun wfcj8l/R9Kt/zJXivS+Rhv73b9WkCroNwwQeqfcN0KbXFkU7yNaaa5OF/v6an1MZ97xfZb5oeTF yKDtem+PAG+3b5ag5//tmfofs8aOBuwD0GVoxpa0rPWt67VQNngDWN/wx/DigBH0ir1jY= X-Developer-Key: i=karthik.188@gmail.com; a=openpgp; fpr=57CE4C7F6375710FCB65C6063ED59F248E468C7F The `git refs migrate` command was introduced in 25a0023f28 (builtin/refs: new command to migrate ref storage formats, 2024-06-06) to support migrating from one reference backend to another. One limitation of the command was that it didn't support migrating repositories which contained reflogs. A previous commit, added support for adding reflog updates in ref transactions. Using the added functionality bake in reflog support for `git refs migrate`. To ensure that the order of the reflogs is maintained during the migration, we add the index for each reflog update as we iterate over the reflogs from the old reference backend. This is to ensure that the order is maintained in the new backend. Signed-off-by: Karthik Nayak --- Documentation/git-refs.txt | 2 -- refs.c | 81 ++++++++++++++++++++++++++++++++-------------- t/t1460-refs-migrate.sh | 73 ++++++++++++++++++++++++++++------------- 3 files changed, 107 insertions(+), 49 deletions(-) diff --git a/Documentation/git-refs.txt b/Documentation/git-refs.txt index ce31f93061db5e5d16aca516dd3d15f6527db870..9829984b0a4c4f54ec7f9b6c6c7072f62b1d198d 100644 --- a/Documentation/git-refs.txt +++ b/Documentation/git-refs.txt @@ -57,8 +57,6 @@ KNOWN LIMITATIONS The ref format migration has several known limitations in its current form: -* It is not possible to migrate repositories that have reflogs. - * It is not possible to migrate repositories that have worktrees. * There is no way to block concurrent writes to the repository during an diff --git a/refs.c b/refs.c index 602a65873181a90751def525608a7fa7bea59562..4d10c7276391e8e85c66bd626bb0ecfec0941c6d 100644 --- a/refs.c +++ b/refs.c @@ -30,6 +30,7 @@ #include "date.h" #include "commit.h" #include "wildmatch.h" +#include "ident.h" /* * List of all available backends @@ -2687,6 +2688,7 @@ int ref_update_check_old_target(const char *referent, struct ref_update *update, } struct migration_data { + unsigned int index; struct ref_store *old_refs; struct ref_transaction *transaction; struct strbuf *errbuf; @@ -2722,6 +2724,53 @@ static int migrate_one_ref(const char *refname, const char *referent UNUSED, con return ret; } +struct reflog_migration_data { + unsigned int *index; + const char *refname; + struct ref_store *old_refs; + struct ref_transaction *transaction; + struct strbuf *errbuf; +}; + +static int migrate_one_reflog_entry(struct object_id *old_oid, + struct object_id *new_oid, + const char *committer, + timestamp_t timestamp, int tz, + const char *msg, void *cb_data) +{ + struct reflog_migration_data *data = cb_data; + struct strbuf sb = STRBUF_INIT; + const char *date; + int ret; + + date = show_date(timestamp, tz, DATE_MODE(NORMAL)); + /* committer contains name and email */ + strbuf_addstr(&sb, fmt_ident("", committer, WANT_BLANK_IDENT, date, 0)); + + ret = ref_transaction_update_reflog(data->transaction, data->refname, + new_oid, old_oid, sb.buf, + REF_HAVE_NEW | REF_HAVE_OLD, msg, + (*data->index)++, data->errbuf); + strbuf_release(&sb); + + return ret; +} + +static int migrate_one_reflog(const char *refname, void *cb_data) +{ + struct migration_data *migration_data = cb_data; + struct reflog_migration_data data; + + data.refname = refname; + data.old_refs = migration_data->old_refs; + data.transaction = migration_data->transaction; + data.errbuf = migration_data->errbuf; + data.index = &migration_data->index; + + return refs_for_each_reflog_ent(migration_data->old_refs, refname, + migrate_one_reflog_entry, &data); +} + static int move_files(const char *from_path, const char *to_path, struct strbuf *errbuf) { struct strbuf from_buf = STRBUF_INIT, to_buf = STRBUF_INIT; @@ -2788,13 +2837,6 @@ static int move_files(const char *from_path, const char *to_path, struct strbuf return ret; } -static int count_reflogs(const char *reflog UNUSED, void *payload) -{ - size_t *reflog_count = payload; - (*reflog_count)++; - return 0; -} - static int has_worktrees(void) { struct worktree **worktrees = get_worktrees(); @@ -2820,7 +2862,6 @@ int repo_migrate_ref_storage_format(struct repository *repo, struct ref_transaction *transaction = NULL; struct strbuf new_gitdir = STRBUF_INIT; struct migration_data data; - size_t reflog_count = 0; int did_migrate_refs = 0; int ret; @@ -2832,21 +2873,6 @@ int repo_migrate_ref_storage_format(struct repository *repo, old_refs = get_main_ref_store(repo); - /* - * We do not have any interfaces that would allow us to write many - * reflog entries. Once we have them we can remove this restriction. - */ - if (refs_for_each_reflog(old_refs, count_reflogs, &reflog_count) < 0) { - strbuf_addstr(errbuf, "cannot count reflogs"); - ret = -1; - goto done; - } - if (reflog_count) { - strbuf_addstr(errbuf, "migrating reflogs is not supported yet"); - ret = -1; - goto done; - } - /* * Worktrees complicate the migration because every worktree has a * separate ref storage. While it should be feasible to implement, this @@ -2868,8 +2894,8 @@ int repo_migrate_ref_storage_format(struct repository *repo, * 1. Set up a new temporary directory and initialize it with the new * format. This is where all refs will be migrated into. * - * 2. Enumerate all refs and write them into the new ref storage. - * This operation is safe as we do not yet modify the main + * 2. Enumerate all refs and reflogs and write them into the new ref + * storage. This operation is safe as we do not yet modify the main * repository. * * 3. If we're in dry-run mode then we are done and can hand over the @@ -2924,6 +2950,11 @@ int repo_migrate_ref_storage_format(struct repository *repo, if (ret < 0) goto done; + data.index = 1; + ret = refs_for_each_reflog(old_refs, migrate_one_reflog, &data); + if (ret < 0) + goto done; + ret = ref_transaction_commit(transaction, errbuf); if (ret < 0) goto done; diff --git a/t/t1460-refs-migrate.sh b/t/t1460-refs-migrate.sh index 1bfff3a7afd5acc470424dfe7ec3e97d45f5c481..f59bc4860f19c4af82dc6f2984bdb69d61fe3ec2 100755 --- a/t/t1460-refs-migrate.sh +++ b/t/t1460-refs-migrate.sh @@ -7,23 +7,44 @@ export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME . ./test-lib.sh +# Migrate the provided repository from one format to the other and +# verify that the references and logs are migrated over correctly. +# Usage: test_migration +# is the relative path to the repo to be migrated. +# is the ref format to be migrated to. +# (true or false) whether to skip reflog verification. test_migration () { - git -C "$1" for-each-ref --include-root-refs \ + repo=$1 && + format=$2 && + skip_reflog_verify=${3:-false} && + git -C "$repo" for-each-ref --include-root-refs \ --format='%(refname) %(objectname) %(symref)' >expect && - git -C "$1" refs migrate --ref-format="$2" && - git -C "$1" for-each-ref --include-root-refs \ + if ! $skip_reflog_verify + then + git -C "$repo" reflog --all >expect_logs && + git -C "$repo" reflog list >expect_log_list + fi && + + git -C "$repo" refs migrate --ref-format="$2" && + + git -C "$repo" for-each-ref --include-root-refs \ --format='%(refname) %(objectname) %(symref)' >actual && test_cmp expect actual && + if ! $skip_reflog_verify + then + git -C "$repo" reflog --all >actual_logs && + git -C "$repo" reflog list >actual_log_list && + test_cmp expect_logs actual_logs && + test_cmp expect_log_list actual_log_list + fi && - git -C "$1" rev-parse --show-ref-format >actual && - echo "$2" >expect && + git -C "$repo" rev-parse --show-ref-format >actual && + echo "$format" >expect && test_cmp expect actual } test_expect_success 'setup' ' - rm -rf .git && - # The migration does not yet support reflogs. - git config --global core.logAllRefUpdates false + rm -rf .git ' test_expect_success "superfluous arguments" ' @@ -78,19 +99,6 @@ do test_cmp expect err ' - test_expect_success "$from_format -> $to_format: migration with reflog fails" ' - test_when_finished "rm -rf repo" && - git init --ref-format=$from_format repo && - test_config -C repo core.logAllRefUpdates true && - test_commit -C repo logged && - test_must_fail git -C repo refs migrate \ - --ref-format=$to_format 2>err && - cat >expect <<-EOF && - error: migrating reflogs is not supported yet - EOF - test_cmp expect err - ' - test_expect_success "$from_format -> $to_format: migration with worktree fails" ' test_when_finished "rm -rf repo" && git init --ref-format=$from_format repo && @@ -141,7 +149,7 @@ do test_commit -C repo initial && test-tool -C repo ref-store main update-ref "" refs/heads/broken \ "$(test_oid 001)" "$ZERO_OID" REF_SKIP_CREATE_REFLOG,REF_SKIP_OID_VERIFICATION && - test_migration repo "$to_format" && + test_migration repo "$to_format" true && test_oid 001 >expect && git -C repo rev-parse refs/heads/broken >actual && test_cmp expect actual @@ -195,6 +203,27 @@ do git -C repo rev-parse --show-ref-format >actual && test_cmp expect actual ' + + test_expect_success "$from_format -> $to_format: reflogs of symrefs with target deleted" ' + test_when_finished "rm -rf repo" && + git init --ref-format=$from_format repo && + test_commit -C repo initial && + git -C repo branch branch-1 HEAD && + git -C repo symbolic-ref refs/heads/symref refs/heads/branch-1 && + cat >input <<-EOF && + delete refs/heads/branch-1 + EOF + git -C repo update-ref --stdin $to_format: reflogs order is retained" ' + test_when_finished "rm -rf repo" && + git init --ref-format=$from_format repo && + test_commit --date "100005000 +0700" --no-tag -C repo initial && + test_commit --date "100003000 +0700" --no-tag -C repo second && + test_migration repo "$to_format" + ' done done