From patchwork Sun Dec 15 16:25:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karthik Nayak X-Patchwork-Id: 13908806 Received: from mail-ej1-f47.google.com (mail-ej1-f47.google.com [209.85.218.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9C25019048D for ; Sun, 15 Dec 2024 16:25:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.47 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734279957; cv=none; b=CLNEChDF8veUXnBpjhdRFo9etSH7SpFegwxUMi7a6CyYXn9qdCHXyh7NXbKyT8ryIzaYI5PKXx30EgpZZ6rf+OnqEcL0NLqGjiGkrgQgDOC25PPvdpBh8xL3GHW/h+4zpb1+Dm5IXPyzLOTg8ikGEJhWP5KmHDYEJ4eonp880Vw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734279957; c=relaxed/simple; bh=1zQAW1gf6s8EkfhWmsjwMbxZIi5DU5Z49Kos2YDhOaQ=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=nYhu1vUALdfvM8+kam7BAJSQrbhBM2YARov3c/zTMhacKK0dAY2g7bZei67Q9n+cXauFmXyFZLmolKqesWCMiutwV/ndtgUqPvQh7hjIZPcyzWq/WYD8O1ZO5dbrqWh0aqZMHjH4kDa1YJ6m9RnoFE6KAXAqAEu9B2bQPazZorE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=RUYwXycc; arc=none smtp.client-ip=209.85.218.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="RUYwXycc" Received: by mail-ej1-f47.google.com with SMTP id a640c23a62f3a-aa68b513abcso550147966b.0 for ; Sun, 15 Dec 2024 08:25:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734279954; x=1734884754; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=2yP66MpHoRJMrRT8Yqjo1czS94YPdDgttlZIZ/ZdMEA=; b=RUYwXycc9TQOJGVHOwGfK3yNtOXX6GpMxlgIwTDeEHACGtk06UiS4hCo+S22vNFu5F 0x1v82yzBVz2CPdEHXahRDJwGNqEXN02BVzPEwBcAVKzmXL/ELAjJS29b9zceBcJKFjD IdLLLhxS4AI+WrP204LQ+W4Z7i4nIUSo2yQAecv+CtVjCw8MY5gBxpEDwIyKULdDSwt6 7uFqjq73BJmAVMS3NY4qq9yFJg1VJmGdDeGBH3IpK41Wx7di35ig7hFyNcQn5VYlTWQR lsDdOQp+cdDxhBJeogtfc3q1z8WqlKH4Kfv0wCyQ4E3T6tK/t0QI6Fsuqq1SjjNUSqWL fPXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734279954; x=1734884754; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2yP66MpHoRJMrRT8Yqjo1czS94YPdDgttlZIZ/ZdMEA=; b=outSHc/P+unDn4yDgwEYoNjUTpPVnN+Q+dqVo+niOfxBUubxPPFBX7093jekrGPa5Q cB/429NF/HhLYtn9m6p9W0ahilCBeAuTIEHHltEGjjnJ2lr56L/UyRt6VJs+Mv3+urIK Rvb2a37aSiyHPwBW6Co46MM6FyomNDnzutzWO8YurtvRw7RfUmvjV5VbQ7Z+ELTLtzMm bAETyLQpq9UpTqsOLvFSnMAWDZjzZKQhY2SeMX3otBW6lj24z45UCazjI1v6LXpaEMYM 5dd7b8gwti3Nbkat1GH/zC5OIQHptKdsNBd6CsFysMy8PpGquXo+k21B2cYcSnn91OeU mMIw== X-Gm-Message-State: AOJu0YzoQpaKc0sfbvIevni+BuTnwDoTCHQeIMLR5XgV9GuB0XdNJxR/ cST0aRx87PXQNIRWU8pv+xrK6rX70srNmuV16+DPTuq2FQKsVUzU X-Gm-Gg: ASbGncus9rUOBSVodbMe4YLonz6VT0TSy6ZHOQvRbs1fH03zOy6Fa4qyAgF5JJmex/P 4iDB5RWUGBQ8siPsIXMQkCEZLgqGAvdhji2p4AcnBOWt1Yc5uXObpshO+C0NWtq0voQjtjxIpOb m+tO8Mm0ZwSG8FWV4pMAv8u5hGD8ilfiEZM68W65lW1ypM7cfzsCVvBEKDzGyBq+8jEm03tXjW5 6h9oJY/ZkWNIrpLkQDYGWeYpKp77557Daa7Kgl+g7ylbKMd72BU78e50wU2QmHfkXrnnA== X-Google-Smtp-Source: AGHT+IECQ8vxblJuypY/szevXuErHCytawNkw8N4HPlpTw9GZULsegtJCYd7rHPK85xeEelmtHr9Og== X-Received: by 2002:a17:906:329a:b0:aa6:8602:9514 with SMTP id a640c23a62f3a-aab77e825a0mr918894166b.48.1734279953606; Sun, 15 Dec 2024 08:25:53 -0800 (PST) Received: from [192.168.178.20] ([2a02:2455:8256:2d00:9c39:c2d7:aedd:294d]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aab9638ad1asm224887866b.139.2024.12.15.08.25.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 15 Dec 2024 08:25:53 -0800 (PST) From: Karthik Nayak Date: Sun, 15 Dec 2024 17:25:38 +0100 Subject: [PATCH v3 1/8] refs: include committer info in `ref_update` struct Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241215-320-git-refs-migrate-reflogs-v3-1-4127fe707b98@gmail.com> References: <20241215-320-git-refs-migrate-reflogs-v3-0-4127fe707b98@gmail.com> In-Reply-To: <20241215-320-git-refs-migrate-reflogs-v3-0-4127fe707b98@gmail.com> To: git@vger.kernel.org Cc: Karthik Nayak , ps@pks.im, Christian Couder X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=5688; i=karthik.188@gmail.com; h=from:subject:message-id; bh=1zQAW1gf6s8EkfhWmsjwMbxZIi5DU5Z49Kos2YDhOaQ=; b=owEB7QES/pANAwAIAT7VnySORox/AcsmYgBnXwMNdjAbsJp6FDTVgPSv/bXPt7G9p6LW+6ES/ 5d05Gmop1yJAbMEAAEIAB0WIQRXzkx/Y3VxD8tlxgY+1Z8kjkaMfwUCZ18DDQAKCRA+1Z8kjkaM fxLlDACHzGUWgc3jfthCD0VG28S3UCR4Pbwi1jmCWmX8rvJA0vPr/SKwnJ4c6XQQdmPFJ9EOc0I aU/WtnW6VsRE/GxInuNTg/j88D320PQQgcC5ustrgbH43a00cnKr0GQtb208MlcU/j/wGCruzBd 1eQgSdMq7lFE9S/7zttkrxbQnOr/jRXODAR8HFqtaudI3arN5h5Ngb3CfJf1+e/Oxo8djUdVS5U +z58rCFx8+XDICVPtvpAX8urupXycbY43qcbnRODn1EjWiRA6B53s8JEr5cqoBEKEeZ7JgcHw5b Kdysf6PMar7muZdJw8WAYywKXM70W32ZZmGZIon4/SkoCLUYJ2DXnEcyF3mYEyaTP4RbB7N1Uyg kifErHyjLJHisCXPVbKXMozYZZjOjSeLzbaKDRclfMcoHjZdezqMvCfMMhxtX62og7FYNgVLCJL dO3EnhXEfT/LxnxMqqYO8CLbsTw6YnKwqIQQyIfmqlhLgDSAqcjXkhZCpsJAVhF0s0QkA= X-Developer-Key: i=karthik.188@gmail.com; a=openpgp; fpr=57CE4C7F6375710FCB65C6063ED59F248E468C7F The reference backends obtain the committer information from `git_committer_info(0)` when adding a reflog. The upcoming patches introduce support for migrating reflogs between the reference backends. This requires an interface to creating reflogs, including custom committer information. Add a new field `committer_info` to the `ref_update` struct, which is then used by the reference backends. If there is no `committer_info` provided, the reference backends default to using `git_committer_info(0)`. The field itself cannot be set to `git_committer_info(0)` since the values are dynamic and must be obtained right when the reflog is being committed. Signed-off-by: Karthik Nayak --- refs.c | 1 + refs/files-backend.c | 24 ++++++++++++++---------- refs/refs-internal.h | 1 + refs/reftable-backend.c | 12 +++++++++++- 4 files changed, 27 insertions(+), 11 deletions(-) diff --git a/refs.c b/refs.c index 762f3e324d59c60cd4f05c2f257e54de8deb00e5..f003e51c6bf5229bfbce8ce61ffad7cdba0572e0 100644 --- a/refs.c +++ b/refs.c @@ -1151,6 +1151,7 @@ void ref_transaction_free(struct ref_transaction *transaction) for (i = 0; i < transaction->nr; i++) { free(transaction->updates[i]->msg); + free(transaction->updates[i]->committer_info); free((char *)transaction->updates[i]->new_target); free((char *)transaction->updates[i]->old_target); free(transaction->updates[i]); diff --git a/refs/files-backend.c b/refs/files-backend.c index 64f51f0da905a9a8a1ac4109c6b0a9a85a355db7..6078668c99ee254e794e3ba49689aa34e6022efd 100644 --- a/refs/files-backend.c +++ b/refs/files-backend.c @@ -1858,6 +1858,9 @@ static int log_ref_write_fd(int fd, const struct object_id *old_oid, struct strbuf sb = STRBUF_INIT; int ret = 0; + if (!committer) + committer = git_committer_info(0); + strbuf_addf(&sb, "%s %s %s", oid_to_hex(old_oid), oid_to_hex(new_oid), committer); if (msg && *msg) { strbuf_addch(&sb, '\t'); @@ -1871,8 +1874,10 @@ static int log_ref_write_fd(int fd, const struct object_id *old_oid, } static int files_log_ref_write(struct files_ref_store *refs, - const char *refname, const struct object_id *old_oid, - const struct object_id *new_oid, const char *msg, + const char *refname, + const struct object_id *old_oid, + const struct object_id *new_oid, + const char *committer_info, const char *msg, int flags, struct strbuf *err) { int logfd, result; @@ -1889,8 +1894,7 @@ static int files_log_ref_write(struct files_ref_store *refs, if (logfd < 0) return 0; - result = log_ref_write_fd(logfd, old_oid, new_oid, - git_committer_info(0), msg); + result = log_ref_write_fd(logfd, old_oid, new_oid, committer_info, msg); if (result) { struct strbuf sb = STRBUF_INIT; int save_errno = errno; @@ -1974,8 +1978,7 @@ static int commit_ref_update(struct files_ref_store *refs, files_assert_main_repository(refs, "commit_ref_update"); clear_loose_ref_cache(refs); - if (files_log_ref_write(refs, lock->ref_name, - &lock->old_oid, oid, + if (files_log_ref_write(refs, lock->ref_name, &lock->old_oid, oid, NULL, logmsg, flags, err)) { char *old_msg = strbuf_detach(err, NULL); strbuf_addf(err, "cannot update the ref '%s': %s", @@ -2007,9 +2010,9 @@ static int commit_ref_update(struct files_ref_store *refs, if (head_ref && (head_flag & REF_ISSYMREF) && !strcmp(head_ref, lock->ref_name)) { struct strbuf log_err = STRBUF_INIT; - if (files_log_ref_write(refs, "HEAD", - &lock->old_oid, oid, - logmsg, flags, &log_err)) { + if (files_log_ref_write(refs, "HEAD", &lock->old_oid, + oid, NULL, logmsg, flags, + &log_err)) { error("%s", log_err.buf); strbuf_release(&log_err); } @@ -2969,7 +2972,8 @@ static int parse_and_write_reflog(struct files_ref_store *refs, } if (files_log_ref_write(refs, lock->ref_name, &lock->old_oid, - &update->new_oid, update->msg, update->flags, err)) { + &update->new_oid, update->committer_info, + update->msg, update->flags, err)) { char *old_msg = strbuf_detach(err, NULL); strbuf_addf(err, "cannot update the ref '%s': %s", diff --git a/refs/refs-internal.h b/refs/refs-internal.h index 58aa56d1b27c85d606ed7c8c0d908e4b87d1066b..0fd95cdacd99e4a728c22f5286f6b3f0f360c110 100644 --- a/refs/refs-internal.h +++ b/refs/refs-internal.h @@ -113,6 +113,7 @@ struct ref_update { void *backend_data; unsigned int type; char *msg; + char *committer_info; /* * If this ref_update was split off of a symref update via diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c index 647ef9b05b1dc9a376ed054330b487f7595c5caa..e882602487c66261d586a94101bb1b4e9a2ed60e 100644 --- a/refs/reftable-backend.c +++ b/refs/reftable-backend.c @@ -1379,11 +1379,21 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data } if (create_reflog) { + struct ident_split c; + ALLOC_GROW(logs, logs_nr + 1, logs_alloc); log = &logs[logs_nr++]; memset(log, 0, sizeof(*log)); - fill_reftable_log_record(log, &committer_ident); + if (u->committer_info) { + if (split_ident_line(&c, u->committer_info, + strlen(u->committer_info))) + BUG("failed splitting committer info"); + } else { + c = committer_ident; + } + + fill_reftable_log_record(log, &c); log->update_index = ts; log->refname = xstrdup(u->refname); memcpy(log->value.update.new_hash, From patchwork Sun Dec 15 16:25:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karthik Nayak X-Patchwork-Id: 13908807 Received: from mail-ej1-f42.google.com (mail-ej1-f42.google.com [209.85.218.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4543F190493 for ; Sun, 15 Dec 2024 16:25:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734279958; cv=none; b=qukmm7hgZGCunGgdcxv1DGLTjgk1kVbyJVigoqKNIS6kv0784VsMOQ8Jfq+449hAIMwJBwAtoXn+8fvQMuAAUwPJ0XT82IAetwW6SRRWU83uyD2eR1zBXVoPtGKEiggdAvfKe2Z+aYNd0WXogFh0/9JJVsVOAqdBl+TNMklGd4s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734279958; c=relaxed/simple; bh=IB9h00v4zfs7q+FeZZk33xlgGFW2QmXAcKQGj56BkwY=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=JYBMoc7lrVNbCRwBIT9hluP7aw6Y+xLvaglU0QRVBQylcKDoPxLppVeIbhqVprcNNHmqJn15v1EE7EFBYdpXzNw78x860/5nRXvgiINgTpqfLsM/wE1fXtbmmbkUKuOfYnV0uqy1P7b4wDgtd5bYO3kHLwXZ4MEOvauVbKBWt2o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=J04VRP/y; arc=none smtp.client-ip=209.85.218.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="J04VRP/y" Received: by mail-ej1-f42.google.com with SMTP id a640c23a62f3a-aa66e4d1d5aso539020766b.2 for ; Sun, 15 Dec 2024 08:25:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734279954; x=1734884754; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=lUvb5dIPp19cc7hdfTanzPMDqohb3VapJTcypa4CDlg=; b=J04VRP/yE9gdoeJbA12JdSuoYHBxI2hecsaV6WNOlsEMcGnev+tDTrdY6FuVUhQodP SjbZxZh+cKLfidA4LRJj4902qNO3R9Rc5RrvxMQUxdDyu1LN/FHo/7wEYZd5h/KJ+wTM P1h9ELG8+gFiJs73cQLbHb3dybbja4qIAM38bHJjNgvSVH3LekWLZ82dqMvJsdRL0WqA hZMwPJcN0KJYuw3oL0TI15GQMKeu+T9Y4dWXFHd1YRfvzjkRTfRxJk9IgDxXC32/5Qtq PuZ8PcX0VpQts8DxtD4RsSNZ4CIqj3rv4/4/rVYCRuj6LA8EJ+P6FMsCEerzbh0zVYQ3 nMcQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734279954; x=1734884754; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=lUvb5dIPp19cc7hdfTanzPMDqohb3VapJTcypa4CDlg=; b=ViN5LEoOjsSV0CLOQaU1j2iCeIcop7wKEK9F7wM+1coL+K5M4FUwKZg74nQEudBlxh u6WcA6V0f/69J4Iul/yb/ldrqKINx6leCHY57cKpGlJe4sAK3dB18NuXRThJIkMpoSd2 3xI/PxUQqzHWl2l+TpsZrogsQbqkdYFf00KF7ydXVgGDfMTN+9eUNASq+EaIvDGYbbM3 t3LfsPjjzNt2/ldQK7/oDiSes5OrH1TyV8SvQ8x0IAUq79SH0HzrJO6UjnQFSZzrDCfG 8B8VjjzuA8b1MgQj0MEgn89NOG5xF0GF2Z+qkHSBJQfYxi2n5ZfSw1cE0PEF7KSFKu5a xBMg== X-Gm-Message-State: AOJu0YzPVS5HX2dQmVXz/2TUFdqHTUaIiBCul6SxbSk3KD1pJ2SUNDEj QGT/vE3sd5/xWd/1H5qZiXJFo0brLL4DcOmAItC/SXZbu4pJ5veu2s3gvwXT X-Gm-Gg: ASbGncue6kB0FOjh0IHzB/G34nv7t/3KzmYySdAZY4fe6eQJowLPApis12ih3UZp0T6 SUZp/E1iUd4dPxY/isB9A/WoynXvusiTAwuIUf4D/SoMUO19xZqf0GDEoo2iWDB5tZHNv/Mu3Ud ceNZiHhbSTzHy1O8uKscwn1DWRQE6VYnrEN7wndw2rFTmtU39Hv4iLVOPBkDL4lTsrnYd6SROuI AZF8cqwonChv57FGEJcJRBH+gBsx9gAntD3q3M2TB44b7bfzJk2pn740d4rvf0AAmOmzg== X-Google-Smtp-Source: AGHT+IEBhSHIGMVvRwHMqzguIOCKx100tSwpM4AGGa8OBr+yz6/goeONJ6ummHslPTgH4C1bKmNw3w== X-Received: by 2002:a17:907:2da4:b0:aa6:23ba:d8c8 with SMTP id a640c23a62f3a-aab778c1e88mr988215066b.11.1734279954316; Sun, 15 Dec 2024 08:25:54 -0800 (PST) Received: from [192.168.178.20] ([2a02:2455:8256:2d00:9c39:c2d7:aedd:294d]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aab9638ad1asm224887866b.139.2024.12.15.08.25.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 15 Dec 2024 08:25:54 -0800 (PST) From: Karthik Nayak Date: Sun, 15 Dec 2024 17:25:39 +0100 Subject: [PATCH v3 2/8] refs: add `index` field to `struct ref_udpate` Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241215-320-git-refs-migrate-reflogs-v3-2-4127fe707b98@gmail.com> References: <20241215-320-git-refs-migrate-reflogs-v3-0-4127fe707b98@gmail.com> In-Reply-To: <20241215-320-git-refs-migrate-reflogs-v3-0-4127fe707b98@gmail.com> To: git@vger.kernel.org Cc: Karthik Nayak , ps@pks.im, Christian Couder X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=2464; i=karthik.188@gmail.com; h=from:subject:message-id; bh=IB9h00v4zfs7q+FeZZk33xlgGFW2QmXAcKQGj56BkwY=; b=owEB7QES/pANAwAIAT7VnySORox/AcsmYgBnXwMNi0jRUAToXzhracDci5bYYFEf843L++Vin wnZZ33yPtmJAbMEAAEIAB0WIQRXzkx/Y3VxD8tlxgY+1Z8kjkaMfwUCZ18DDQAKCRA+1Z8kjkaM fxxbC/4lmJNV09bcvVJghGfJ/HfRmEMZ9Tgf3C4V66G1jAeQR7aekpcqdjIcMV4h2qSKHzasAPk Tp33OEkXra/XJAnjpCt1em5VlYtxCYPLu2TzU21hzcCnLtpqZwKXWGGfjuXjwodS9VtlNW+emuL b0B5XMjCSuIo4iD/9pdPPJ7qGmSgYQx7sjjOEUm3BxrabGhWALdnZpxrJZwQ/OWtHjxE/4i/3uz 6/t+vkZ0z7UppATLfZ2ym8uGEKOdYoSXcQCUbjwoWTJLx/zmlFLeLyKskGJNys9WO+b7csDx5rE 4k1Hpf1EG/CGFlNfh5Y0x2OTU+L7wdF1gVe7NpfvHjbFuRt6sjr6cFaFpxuwea9W5TIlNMCHvYO COg+EIIGtwC6VSWG24Phhgrl7gT7Ax8dWwuYr6WTfLcMfmrvAPwTnQ5z2NeB7l5Cs+jCpEx1EZv J3qOBmFQ77mf54qy3qadhDZFRrdJr5ZGp3FltADf/VMt2awilEnbEQbBRfmSCpZKJorh8= X-Developer-Key: i=karthik.188@gmail.com; a=openpgp; fpr=57CE4C7F6375710FCB65C6063ED59F248E468C7F The reftable backend, sorts its updates by refname before applying them, this ensures that the references are stored sorted. When migrating reflogs from one backend to another, the order of the reflogs must be maintained. Add a new `index` field to the `ref_update` struct to facilitate this. This field is used in the reftable backend's sort comparison function `transaction_update_cmp`, to ensure that indexed fields maintain their order. Signed-off-by: Karthik Nayak --- refs/refs-internal.h | 7 +++++++ refs/reftable-backend.c | 13 +++++++++++-- 2 files changed, 18 insertions(+), 2 deletions(-) diff --git a/refs/refs-internal.h b/refs/refs-internal.h index 0fd95cdacd99e4a728c22f5286f6b3f0f360c110..f5c733d099f0c6f1076a25f4f77d9d5eb345ec87 100644 --- a/refs/refs-internal.h +++ b/refs/refs-internal.h @@ -115,6 +115,13 @@ struct ref_update { char *msg; char *committer_info; + /* + * The index overrides the default sort algorithm. This is needed + * when migrating reflogs and we want to ensure we carry over the + * same order. + */ + unsigned int index; + /* * If this ref_update was split off of a symref update via * split_symref_update(), then this member points at that diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c index e882602487c66261d586a94101bb1b4e9a2ed60e..c008f20be719fec3af6a8f81c821cb9c263764d7 100644 --- a/refs/reftable-backend.c +++ b/refs/reftable-backend.c @@ -1279,8 +1279,17 @@ static int reftable_be_transaction_abort(struct ref_store *ref_store UNUSED, static int transaction_update_cmp(const void *a, const void *b) { - return strcmp(((struct reftable_transaction_update *)a)->update->refname, - ((struct reftable_transaction_update *)b)->update->refname); + struct reftable_transaction_update *update_a = (struct reftable_transaction_update *)a; + struct reftable_transaction_update *update_b = (struct reftable_transaction_update *)b; + + /* + * If there is an index set, it should take preference (default is 0). + * This ensures that updates with indexes are sorted amongst themselves. + */ + if (update_a->update->index || update_b->update->index) + return update_a->update->index - update_b->update->index; + + return strcmp(update_a->update->refname, update_b->update->refname); } static int write_transaction_table(struct reftable_writer *writer, void *cb_data) From patchwork Sun Dec 15 16:25:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karthik Nayak X-Patchwork-Id: 13908808 Received: from mail-ed1-f52.google.com (mail-ed1-f52.google.com [209.85.208.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 233A81922D7 for ; Sun, 15 Dec 2024 16:25:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.52 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734279959; cv=none; b=jilkwLgeRslCeeWEFAFoXd7sK5Ml32mJ87RyhQLtTLaJV1SpbusaQ+iN6O2lB5vb3VJt6t4EPh9Jmvcgkoortrky0EJmNXyOosoTPMu4oBaUXeT0k5HD4Lgz38hKh14x+r7PshAdYdIQrfpVPdH4Ic/1nF3bNGuOZK7FiYQwZWE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734279959; c=relaxed/simple; bh=ERhO5bUagvCcVWd1ajCqhTTv/qHXvpyDSomL3moOTTM=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=kEWvxYzZC4ngY80RKvn9jNQy510Bl/Xg2+WyCy1gmc1p2Vm/CZefdL2eJB5Lpp5hsNrsm9joBRZPFwJh3Mtw9acjjivNAiuthNgulmH1j7EXmeuH8BehItkuoPILsTsZ2TiKSJIJNS2HNRErKfIdpkC1kpihi/0ZIShzYmzRYbA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=AexyhYG3; arc=none smtp.client-ip=209.85.208.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="AexyhYG3" Received: by mail-ed1-f52.google.com with SMTP id 4fb4d7f45d1cf-5d414b8af7bso6924791a12.0 for ; Sun, 15 Dec 2024 08:25:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734279955; x=1734884755; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=5gNfOmPXL16KqA+CPu0YSmndFRynLUkndelIwKCAq4k=; b=AexyhYG3tRZXEXpV63hE7SLakyTOPywUWj7yClHCl/11qZ/Ri0t6jTcrTLSyc6p3lW gRkGrg+mqUYaLGk5ZU9ZUaHXMMor8mSgMJRKgq89mHB6XW3Y41GesfJDuyiHYZCX2JHL buUtWlahiMRQPay9or5Kopyys/cIXs47GS6Fs7Sj78MrOArIyFVWsxdNQMiJ9SUC/XAU wreil9dXmjXOo/NrtEj1Jn8d1kQKtGYa1pu8NcLrHOH7lL9Ktq15v958t0vY+XWtBtXp bk7tYS9rWRLr4UsBz3w3gwii4pOCHuvCFPj+/9Dzhl3wHv9KfnTqWI+q8d5pXFMLTzFB WmFw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734279955; x=1734884755; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=5gNfOmPXL16KqA+CPu0YSmndFRynLUkndelIwKCAq4k=; b=GljMd6i+sud7Gah1cg+VH7nnqGjve5bq4twH3xGyCfo/DhH/Vsx9mASq4HCklpZ6Pf H7EBnv44SAuinWVewqOKwxWRMR0Q/fdO1fTVrIfmZUbFCRWZjzb4+7DifXiIH/nNa37t llWlDHn8UW89SjN8ppHk6pz5fAuQKohE89lexWzwJ8X70giF86uDe0VYqb5D9D5/Zm60 njmIgJbJFUWf9cVGFKmuda9BktyxRc2XdNhbGBFEzFNNVx+3ZHNVFcvSwV8OHcl6wYje YN7NFhWvVsz+TdLOZiEugZF0ui5+LyntTJXbyWg3Vi5zxzQu/I9OA2IaxWzET5JKAqd+ eqrw== X-Gm-Message-State: AOJu0YzTCWfP8kWM+7OTmM0yeunPFOMEVEZJHT2CgijazmjOLgQBx3U4 h9Nt7JPRanx5daQ8u1hvATBv8lUOuKp2ktSBvCc1K0VVZE6A93FZ X-Gm-Gg: ASbGncu2FLkN4k9xiIuEUSMo22K7KzZM3I4oWj5CDjohljMoTEavadq7TZedRISJdlE 4YFhFvP4qg9V32gxoprBWKUdZSLKAL4aSVYqE8kQ+Y5nsPmhJO1wAWqxfWh/G+3iuUO2IqTSLjQ 6UVAj3JjKQCTqBBML28rIoKFqyT/IuAFnaoKthwWxfSnANWQlv/y60GlFLRZInGwjYbZa3plz/w i3tUlp1BJDE6GB3/M0Nr61/Ch3mQ0Bg8giORUqpXeUm8vQyW+LgiKmqRHjLHJ8gfYqOXQ== X-Google-Smtp-Source: AGHT+IGKxmZxRsH0dk7vo1ktlgWM2MkwGv77SSdjxhYqgeVNVJxxfAmCFXkgupcGBlWLmdESqY5RnQ== X-Received: by 2002:a17:907:9491:b0:a9a:9df:5580 with SMTP id a640c23a62f3a-aab7792d75cmr840250966b.19.1734279955063; Sun, 15 Dec 2024 08:25:55 -0800 (PST) Received: from [192.168.178.20] ([2a02:2455:8256:2d00:9c39:c2d7:aedd:294d]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aab9638ad1asm224887866b.139.2024.12.15.08.25.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 15 Dec 2024 08:25:54 -0800 (PST) From: Karthik Nayak Date: Sun, 15 Dec 2024 17:25:40 +0100 Subject: [PATCH v3 3/8] refs/files: add count field to ref_lock Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241215-320-git-refs-migrate-reflogs-v3-3-4127fe707b98@gmail.com> References: <20241215-320-git-refs-migrate-reflogs-v3-0-4127fe707b98@gmail.com> In-Reply-To: <20241215-320-git-refs-migrate-reflogs-v3-0-4127fe707b98@gmail.com> To: git@vger.kernel.org Cc: Karthik Nayak , ps@pks.im, Christian Couder X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=5670; i=karthik.188@gmail.com; h=from:subject:message-id; bh=ERhO5bUagvCcVWd1ajCqhTTv/qHXvpyDSomL3moOTTM=; b=owEB7QES/pANAwAIAT7VnySORox/AcsmYgBnXwMOxxBE3KzCwxPHSm2GTWOF4FpR7q7tkcMQN gVL5pSyNiyJAbMEAAEIAB0WIQRXzkx/Y3VxD8tlxgY+1Z8kjkaMfwUCZ18DDgAKCRA+1Z8kjkaM f1e6C/kBt6GPim/504m37VFnzi/cFOTdbmPihOzwOCnFyHH3Sau45sxfrAs73eEfLDB6OfeSG8K C+MZiDDDjTzooBkPDhz6XMEloGNjBM2LT1joiudXyxKh+wuvmmvsPCMOgGJRmaZJ8562JB+qIme KP8gL4K65KrY6sv/8/xGgtaTsfeQnELVD9HWZCqHek4K2npBdmjnb632tRv+Ka3iD2UGCMxlK3i xErdQxiz33Y0b9YRNsIcjwMGch+1uWiCZ9YR1mc8CWL6lvpguOimwN+CsYMWVMY1LGFtGTUON0n a5guYd7xvWV6p4oH9u7Ded8zMXqae2ipVNBBMlyozqigVBO9PSTgCBIloFj6ocX1dWG9i9qmOIJ 2/gxsd+pgwR9pqHMX6L23wn+tFlyjUTYqgzwwwoj6lSAVtMGJ76YxmQFz1gTa+GpJS7X4SwwByd 3DqgVsQAr7nuJCO1oC3JvNs4r0+zPJ+Ool/lBGi8J2U8POmdSYZ71iZTfr8TMTpvrCJo4= X-Developer-Key: i=karthik.188@gmail.com; a=openpgp; fpr=57CE4C7F6375710FCB65C6063ED59F248E468C7F When refs are updated in the files-backend, a lock is obtained for the corresponding file path. This is the case even for reflogs, i.e. a lock is obtained on the reference path instead of the reflog path. This works, since generally, reflogs are updated alongside the ref. The upcoming patches will add support for reflog updates in ref transaction. This means, in a particular transaction we want to have ref updates and reflog updates. For a given ref in a given transaction there can be at most one update. But we can theoretically have multiple reflog updates for a given ref in a given transaction. A great example of this would be when migrating reflogs from one backend to another. There we would batch all the reflog updates for a given reference in a single transaction. The current flow does not support this, because currently refs & reflogs are treated as a single entity and capture the lock together. To separate this, add a count field to ref_lock. With this, multiple updates can hold onto a single ref_lock and the lock will only be released when all of them release the lock. This patch only adds the `count` field to `ref_lock` and adds the logic to increment and decrement the lock. In a follow up commit, we'll separate the reflog update logic from ref updates and utilize this functionality. Signed-off-by: Karthik Nayak --- refs/files-backend.c | 58 +++++++++++++++++++++++++++++++++++----------------- 1 file changed, 39 insertions(+), 19 deletions(-) diff --git a/refs/files-backend.c b/refs/files-backend.c index 6078668c99ee254e794e3ba49689aa34e6022efd..02cb4907d8659e87a227fed4f60a5f6606be8764 100644 --- a/refs/files-backend.c +++ b/refs/files-backend.c @@ -71,6 +71,7 @@ struct ref_lock { char *ref_name; struct lock_file lk; struct object_id old_oid; + unsigned int count; /* track users of the lock (ref update + reflog updates) */ }; struct files_ref_store { @@ -638,9 +639,12 @@ int parse_loose_ref_contents(const struct git_hash_algo *algop, static void unlock_ref(struct ref_lock *lock) { - rollback_lock_file(&lock->lk); - free(lock->ref_name); - free(lock); + lock->count--; + if (!lock->count) { + rollback_lock_file(&lock->lk); + free(lock->ref_name); + free(lock); + } } /* @@ -696,6 +700,7 @@ static int lock_raw_ref(struct files_ref_store *refs, *lock_p = CALLOC_ARRAY(lock, 1); lock->ref_name = xstrdup(refname); + lock->count = 1; files_ref_path(refs, &ref_file, refname); retry: @@ -1169,6 +1174,7 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs, goto error_return; lock->ref_name = xstrdup(refname); + lock->count = 1; if (raceproof_create_file(ref_file.buf, create_reflock, &lock->lk)) { unable_to_lock_message(ref_file.buf, errno, err); @@ -2535,6 +2541,12 @@ static int check_old_oid(struct ref_update *update, struct object_id *oid, return -1; } +struct files_transaction_backend_data { + struct ref_transaction *packed_transaction; + int packed_refs_locked; + struct strmap ref_locks; +}; + /* * Prepare for carrying out update: * - Lock the reference referred to by update. @@ -2557,11 +2569,14 @@ static int lock_ref_for_update(struct files_ref_store *refs, { struct strbuf referent = STRBUF_INIT; int mustexist = ref_update_expects_existing_old_ref(update); + struct files_transaction_backend_data *backend_data; int ret = 0; struct ref_lock *lock; files_assert_main_repository(refs, "lock_ref_for_update"); + backend_data = transaction->backend_data; + if ((update->flags & REF_HAVE_NEW) && ref_update_has_null_new_value(update)) update->flags |= REF_DELETING; @@ -2572,18 +2587,25 @@ static int lock_ref_for_update(struct files_ref_store *refs, goto out; } - ret = lock_raw_ref(refs, update->refname, mustexist, - affected_refnames, - &lock, &referent, - &update->type, err); - if (ret) { - char *reason; + lock = strmap_get(&backend_data->ref_locks, update->refname); + if (lock) { + lock->count++; + } else { + ret = lock_raw_ref(refs, update->refname, mustexist, + affected_refnames, + &lock, &referent, + &update->type, err); + if (ret) { + char *reason; + + reason = strbuf_detach(err, NULL); + strbuf_addf(err, "cannot lock ref '%s': %s", + ref_update_original_update_refname(update), reason); + free(reason); + goto out; + } - reason = strbuf_detach(err, NULL); - strbuf_addf(err, "cannot lock ref '%s': %s", - ref_update_original_update_refname(update), reason); - free(reason); - goto out; + strmap_put(&backend_data->ref_locks, update->refname, lock); } update->backend_data = lock; @@ -2730,11 +2752,6 @@ static int lock_ref_for_update(struct files_ref_store *refs, return ret; } -struct files_transaction_backend_data { - struct ref_transaction *packed_transaction; - int packed_refs_locked; -}; - /* * Unlock any references in `transaction` that are still locked, and * mark the transaction closed. @@ -2767,6 +2784,8 @@ static void files_transaction_cleanup(struct files_ref_store *refs, if (backend_data->packed_refs_locked) packed_refs_unlock(refs->packed_ref_store); + strmap_clear(&backend_data->ref_locks, 0); + free(backend_data); } @@ -2796,6 +2815,7 @@ static int files_transaction_prepare(struct ref_store *ref_store, goto cleanup; CALLOC_ARRAY(backend_data, 1); + strmap_init(&backend_data->ref_locks); transaction->backend_data = backend_data; /* From patchwork Sun Dec 15 16:25:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karthik Nayak X-Patchwork-Id: 13908809 Received: from mail-ej1-f54.google.com (mail-ej1-f54.google.com [209.85.218.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BAED0192593 for ; Sun, 15 Dec 2024 16:25:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.54 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734279959; cv=none; b=KCX7WwfeENK+y3smO7+s87yG/Ngl7tFWVR1wjk7Kyt0rB0gQfYqnuXN+qTQkWKUgdV8Uqvn1tnNDwlK5um53+cSAee7x57hlPpbKb6CckwXYh780Z3UonIYO1t+eEldQ/TeLumlR0DHPS86tzBZK0cnk/es0bmTNxvH5HaYaeXA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734279959; c=relaxed/simple; bh=NoEckIGxQa5jQrZ0dQ7BHqCovFckF/qXPjbGFqkkuA0=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=mPpOemIDDsEMTHYd06KDDWyhNK+/oH9LKpiEK88flt5RgS6jlfzOUSkJb+wuyBQ34uexQa6rLEBAEz4dFF6a2xiKtehHU2IkZXtrOx64ReZu1zRNcd5U+maMBBOGihUj+PasLW8FAJrrCz15F7GIK5d3SXPNgWXB748USxGrPbc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Ol/Nv/zo; arc=none smtp.client-ip=209.85.218.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Ol/Nv/zo" Received: by mail-ej1-f54.google.com with SMTP id a640c23a62f3a-aa662795ca3so850509766b.1 for ; Sun, 15 Dec 2024 08:25:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734279956; x=1734884756; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=Io5XE6RXiBUov+SaLOGtphgg3UciL24hpxxR053Wce4=; b=Ol/Nv/zoJTIaF7TIxjkUobZVJ72qdVAzE3knI268Otl+C92k0ko1XhCI9Y5FWlzuZx q7tykWY6oOoDHq3wvqARpCziRiI+GPebqd7Y5JJbGx5ldqXcYjHWxNGlw8FSaHYcbZ/a RO5K75CaGvt8+pwZjcP12srpTgh0zAd3bqxeE3GH8huucUxI1jL4tkIdObFB3pT0f11D EBIzBWf+YImL438SXmsbmtm4MzVN/zZmdBS/k0QOaDZTXUM/OrYrXAg/YOBSSitTwt8J 7pItz3wWSEW3xCn1sYiiT2H+EcctcgUx9vdbSbUIbr3lnUkVqKdso227222IpAjtJQux HhQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734279956; x=1734884756; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Io5XE6RXiBUov+SaLOGtphgg3UciL24hpxxR053Wce4=; b=EbxeDneB54qP3MTe9w9kb72axxQOO1aHuEO2Bq9fsM3T0pQDfsOJHHa5OICj2nGo3M cukNrBtHXH5+P7NISPLqogKnusHvUa3Wb0K8bKdjZeeUBMXzg9En562nb4jHmU7kL2ud XD1eLouqTg6AWo5cI5OIxjClx+rgCizjHWbnbPy1H3cJYJHuhb4Uac2XaA6AEFMIbOFn JlR32IEHqBADrYXnPheU1xY3LqvLBCDUrAiG7fiBPddOgkQiEuLVLrHw5Fq5m8G92BzJ H3QZzMLb1kwcCoEzhi4+5Cb3fyemv6fDFTqIlvvYQY77MtMf422XhQG5QjhTdDNutfix fePA== X-Gm-Message-State: AOJu0YxSMbwevdyuqYgqUWZ0WM3Eix09T+cOlhoVtwDaMLw4i+XnDS9c zhqCsqQSzNMAGoVBttpe+LtvAA9cCONYKFezE9sqhK0jdVM+RY0W X-Gm-Gg: ASbGncshujoqYYJuP2Jkr8JZoily9prMROKriNqjaUmA73GPcwg/ZSPBYtnlAoKzfVC yg8ccD0sAqLyTcctB0HK1+KFcX/le0zM8Dlkul3jo5ltApWRFunuhCDL78cKvW0sDllELcJs8OX gOPSADAbQp2rHxrb8uGiKgKGz/QiuudU5kA3P83Qzjn2QY+TxKzDMk0H1/oUqegtLXWaIxXZc87 8ozLVhOksXmpRMMjCHaamyi38xmBUoKDTSmrQlEK9Spv6rhf86UmdkF8ih5Xbtaw6vhAg== X-Google-Smtp-Source: AGHT+IHX4TVNU4s9QWZP0OvrrdjTEBrkhVdocyz+5grGI/M+nUl+ozWx1yXvtwFU7G7Xj8+DHDGxuw== X-Received: by 2002:a17:907:1c07:b0:aa6:7c36:3428 with SMTP id a640c23a62f3a-aab7ad158b3mr916404566b.0.1734279955843; Sun, 15 Dec 2024 08:25:55 -0800 (PST) Received: from [192.168.178.20] ([2a02:2455:8256:2d00:9c39:c2d7:aedd:294d]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aab9638ad1asm224887866b.139.2024.12.15.08.25.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 15 Dec 2024 08:25:55 -0800 (PST) From: Karthik Nayak Date: Sun, 15 Dec 2024 17:25:41 +0100 Subject: [PATCH v3 4/8] refs: extract out refname verification in transactions Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241215-320-git-refs-migrate-reflogs-v3-4-4127fe707b98@gmail.com> References: <20241215-320-git-refs-migrate-reflogs-v3-0-4127fe707b98@gmail.com> In-Reply-To: <20241215-320-git-refs-migrate-reflogs-v3-0-4127fe707b98@gmail.com> To: git@vger.kernel.org Cc: Karthik Nayak , ps@pks.im, Christian Couder X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=2366; i=karthik.188@gmail.com; h=from:subject:message-id; bh=NoEckIGxQa5jQrZ0dQ7BHqCovFckF/qXPjbGFqkkuA0=; b=owEB7QES/pANAwAIAT7VnySORox/AcsmYgBnXwMOpgREDkZXHPSB4e0LhKgyiQHnfmGTqIAjq 33AKX+3snaJAbMEAAEIAB0WIQRXzkx/Y3VxD8tlxgY+1Z8kjkaMfwUCZ18DDgAKCRA+1Z8kjkaM f/yeC/90ICif0WavWU0cffLuAWqZ//1TAutknYYOxAb6uDAPB1d/FfeTmNbYvC67sFUwDPFriHf 4OohTEZ3x5v+ozR2QiX69z50iVZSFr1yK8mn3foTJjHjSQ+JRFgKowRNBeQDBGFW55QZjOpOOgL cYmfAq1DcdB9V+UBc0gJxABZFVH+l/CSchPVV78+2Sef1bSZ3C/vX3dxNK6e8NafvqSY583QxP+ kWTJu2cRhCjJYSoywiMvISg39b0iYoaUEX5dWsOV1baQ/xljNnfABeuP8N/DgRBG+Ist0aqIjPd Kj2+7GZXiH5awX9LtykeJ5hCOyjDdEWwEX5/HJ3LDT7fDvTXC1JuR9ptUKKGFAOrq5YRdu77OyW uuMUdJce7qrmmhBIBDeB9TZTcitiZ19il1L5zstqddWO4lmbxEEXrOK8/7n5i0J/Xp4vqDaB5qP 3Ey8oO+0ScEHtKD+I2M5SpgPfMMaWCMswdYZiSlvKY2x9dWYygh6cC9d9+lRTr9HRuFLE= X-Developer-Key: i=karthik.188@gmail.com; a=openpgp; fpr=57CE4C7F6375710FCB65C6063ED59F248E468C7F Unless the `REF_SKIP_REFNAME_VERIFICATION` flag is set for an update, the refname of the update is verified for: - Ensuring it is not a pseudoref. - Checking the refname format. These checks will also be needed in a following commit where the function to add reflog updates to the transaction is introduced. Extract the code out into a new static function. Signed-off-by: Karthik Nayak --- refs.c | 37 +++++++++++++++++++++++-------------- 1 file changed, 23 insertions(+), 14 deletions(-) diff --git a/refs.c b/refs.c index f003e51c6bf5229bfbce8ce61ffad7cdba0572e0..9c9f4940c60d3cdd34ce8f1e668d17b9da3cd801 100644 --- a/refs.c +++ b/refs.c @@ -1196,6 +1196,28 @@ struct ref_update *ref_transaction_add_update( return update; } +static int transaction_refname_valid(const char *refname, + const struct object_id *new_oid, + unsigned int flags, struct strbuf *err) +{ + if (flags & REF_SKIP_REFNAME_VERIFICATION) + return 1; + + if (is_pseudo_ref(refname)) { + strbuf_addf(err, _("refusing to update pseudoref '%s'"), + refname); + return 0; + } else if ((new_oid && !is_null_oid(new_oid)) ? + check_refname_format(refname, REFNAME_ALLOW_ONELEVEL) : + !refname_is_safe(refname)) { + strbuf_addf(err, _("refusing to update ref with bad name '%s'"), + refname); + return 0; + } + + return 1; +} + int ref_transaction_update(struct ref_transaction *transaction, const char *refname, const struct object_id *new_oid, @@ -1213,21 +1235,8 @@ int ref_transaction_update(struct ref_transaction *transaction, return -1; } - if (!(flags & REF_SKIP_REFNAME_VERIFICATION) && - ((new_oid && !is_null_oid(new_oid)) ? - check_refname_format(refname, REFNAME_ALLOW_ONELEVEL) : - !refname_is_safe(refname))) { - strbuf_addf(err, _("refusing to update ref with bad name '%s'"), - refname); + if (!transaction_refname_valid(refname, new_oid, flags, err)) return -1; - } - - if (!(flags & REF_SKIP_REFNAME_VERIFICATION) && - is_pseudo_ref(refname)) { - strbuf_addf(err, _("refusing to update pseudoref '%s'"), - refname); - return -1; - } if (flags & ~REF_TRANSACTION_UPDATE_ALLOWED_FLAGS) BUG("illegal flags 0x%x passed to ref_transaction_update()", flags); From patchwork Sun Dec 15 16:25:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karthik Nayak X-Patchwork-Id: 13908810 Received: from mail-ej1-f52.google.com (mail-ej1-f52.google.com [209.85.218.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 800F3193071 for ; Sun, 15 Dec 2024 16:25:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.52 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734279960; cv=none; b=exj7DXn3D8x+ZSnkH8hTvzc6AY9Ju0B3mRd54aqbLtrwOumR9X5h+VxkWuC7HswBVhNrgW/qawcibxV9nLKCL0nO3k7lu6mt+ok0x8a/TwgTXHG986PzB8go9qpyVBA74lMwA9NB8jrAuxRqK/zdofnOc3ljj1szA54r8WyoMu8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734279960; c=relaxed/simple; bh=ATy4ssKxG+1kuX/UpGVdpNwF/duWxcYMcv9ZJ2nZT2Y=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=Zkzq9lTtxJzOpmzRTaQ/+YHlDQdFI5z9VmhoTO9rAnBIvip/qHtHKIdg5fUiNw+JCXO/OLUGtW6lBxHTDSBDu/iPSPoDW1UFUWca8A9OoS5VjUMzywm4kJktX5PCv9Q8y8gLsfNQmVUtdupZ+TLOyW8dlBECYQQjIz0EqYcVsck= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=OuUdj1aL; arc=none smtp.client-ip=209.85.218.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="OuUdj1aL" Received: by mail-ej1-f52.google.com with SMTP id a640c23a62f3a-aa670ffe302so632976366b.2 for ; Sun, 15 Dec 2024 08:25:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734279957; x=1734884757; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=W1ch3s3JnDSoPZxdCqvaf+8QiC2QUWbiyEX7XSSiSX0=; b=OuUdj1aL1nz1LKZQQGlBw71viY/AcmmF6HUXPzU6w2TRU8zQAER1cbMUGVPA+gUC0u 8UqxXduPJLaO9xEEPV0o3LlUbwdqb4yDZBSBM6dB0DGpHlN9HjmZx0eBrETmJYftc4jy JgxTmrTO2+ooQrUoWgvhkRGoMsZpC4O6OK3HQ2U1ttXT/GRpFtfCy1lE1DzssQ0Ukn4R Mh+AQEbj3SMI5+6PYyghZmMlJOvoxWTMkglVg2yU79Ma4QY7ytdzpSd/u3FkTRMQ08Hv P5Iomx2NS07C5IK0gJPJ/2iez8TNJjEdrTrc/24c0FYpY25uZiJntH4g/AN1Hzx+Vokv hZ/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734279957; x=1734884757; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=W1ch3s3JnDSoPZxdCqvaf+8QiC2QUWbiyEX7XSSiSX0=; b=QWc1CVHHAPYjbDfdiCcSIfStsHwL54qH/5hnj7OnxmXlrmiOJH7L8+mAAD4zFJMQHk vNNEOprXcc4u2xZPC3PqS/NDmdbKQgQ8sUSdnahf21gPsCQjLHhNrg8CG9B0qZ/pDQtm jw/Ex5TDTh/Av9ro/+n7bW48Hw5wu4b/y4/GGBfKkid6P5rX4skcCQ1MXavfyzVVBTGe siX1QPsbgLh1lNSvheXdxwSvpfyojiGGbUtfaXLEES0hhevP20UttQt2h9Sz4n9Mx4It 9rW8zvCad6R6PzHlWk3//Qec2l5vEIYzJd5vu0pcGS/tOKdHnXIByUBjJLQp3mUpyBia kGiA== X-Gm-Message-State: AOJu0YxMyHGCFK/SKeWLEF8BpQpLnKBcEutoVFEF2zvzGbPWwe10Rf1T 6wORnjZNBI+gMKvr0ybRLzeFGnPxZdEWVLHqDFJyOLBTLOvMhnpf X-Gm-Gg: ASbGncs2Jz4+f3omtH6AOUGntSdjSS0M8PWfsHO91pKpKiMYxGjT0jI9QnGgVGH+wAq ITiTFNm4bNLAYOynpS89Lc70Xtjvj28RjLzjptGVcrBgZXlU/bE5OwT3ujIjGfjK4/sPbN7ZK1x yuEa9zurNMOrwxTSTkzntVh1bF6lAITwbbA2AsBgaGIxulcDo9slSeYCWVlgc1D1NaB5Mmca6Sp jjizblD2bLS0C6V3pej3T835URvhfCMXJmM5/6J9vQHePX9EgouL6tkBIIAPrsKR+tpRA== X-Google-Smtp-Source: AGHT+IHvjA7ERkDCg9C5D2u7P8k2GkxwBJJ56JYN4aX9cnZoy1TAmegisBA25W5xOau36oWK+vy9IQ== X-Received: by 2002:a17:907:780a:b0:aa6:8520:7187 with SMTP id a640c23a62f3a-aab779c1398mr960499866b.36.1734279956571; Sun, 15 Dec 2024 08:25:56 -0800 (PST) Received: from [192.168.178.20] ([2a02:2455:8256:2d00:9c39:c2d7:aedd:294d]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aab9638ad1asm224887866b.139.2024.12.15.08.25.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 15 Dec 2024 08:25:56 -0800 (PST) From: Karthik Nayak Date: Sun, 15 Dec 2024 17:25:42 +0100 Subject: [PATCH v3 5/8] refs: add `committer_info` to `ref_transaction_add_update()` Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241215-320-git-refs-migrate-reflogs-v3-5-4127fe707b98@gmail.com> References: <20241215-320-git-refs-migrate-reflogs-v3-0-4127fe707b98@gmail.com> In-Reply-To: <20241215-320-git-refs-migrate-reflogs-v3-0-4127fe707b98@gmail.com> To: git@vger.kernel.org Cc: Karthik Nayak , ps@pks.im, Christian Couder X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=5487; i=karthik.188@gmail.com; h=from:subject:message-id; bh=ATy4ssKxG+1kuX/UpGVdpNwF/duWxcYMcv9ZJ2nZT2Y=; b=owEB7QES/pANAwAIAT7VnySORox/AcsmYgBnXwMOmyYBfliGGIhJ2n/3fF+4NkZFkl5FNf797 K9yRLi3rXSJAbMEAAEIAB0WIQRXzkx/Y3VxD8tlxgY+1Z8kjkaMfwUCZ18DDgAKCRA+1Z8kjkaM f1c9DACKtASTy9GGPJo6D6RZjFuc22IPvDbdhw56gN2IgcuLZpvITKB0Ws+OlqAPD1aNb1JakB3 HyWrIC0SdtL6S/oqxVgGzXbXF9390ASRzjAXE2F0S7J7kdiA8j3nkyyoGDWyo9MQQopYQ5ckcHQ eIOVsMPiudt75TqSNxIrt2uc3/RaSVNqk4TmSNsWfuLK/2Lbhs+bhequl0E42Uwh4mgFhd9eF7L AeobmACTrjhXNr21L3WPTiTtm1Q2/0f8yuF3hEBQ6WpA5+RZmga6EhcVUw4/fVa03aYTzith8j5 Kk4zutokN7jGuyAOCq9QOUfKIQPE6O2LAH+VSWGGE+jfq4nPWqIvioOET08ISNyoV1NVzCZO/Q/ z2bIxXuhL6sE+264BNAt/iAqQ9+oo4Q8/IKqZPlKwnXCSmKVmvTQEuBahsrLMIx0XS2vGHkRBkz SyFra/C6etm0yrbJFLzr8jvDWm5eh9WbQMHfADFa7W8Ly96b1AZ4EnL2zfTudwpk2jgvM= X-Developer-Key: i=karthik.188@gmail.com; a=openpgp; fpr=57CE4C7F6375710FCB65C6063ED59F248E468C7F The `ref_transaction_add_update()` creates the `ref_update` struct. To facilitate addition of reflogs in the next commit, the function needs to accommodate setting the `committer_info` field in the struct. So modify the function to also take `committer_info` as an argument and set it accordingly. Signed-off-by: Karthik Nayak --- refs.c | 7 +++++-- refs/files-backend.c | 14 ++++++++------ refs/refs-internal.h | 1 + refs/reftable-backend.c | 6 ++++-- 4 files changed, 18 insertions(+), 10 deletions(-) diff --git a/refs.c b/refs.c index 9c9f4940c60d3cdd34ce8f1e668d17b9da3cd801..782bf1090af65196263a3c35ed18d878bb4f2967 100644 --- a/refs.c +++ b/refs.c @@ -1166,6 +1166,7 @@ struct ref_update *ref_transaction_add_update( const struct object_id *new_oid, const struct object_id *old_oid, const char *new_target, const char *old_target, + const char *committer_info, const char *msg) { struct ref_update *update; @@ -1190,8 +1191,10 @@ struct ref_update *ref_transaction_add_update( oidcpy(&update->new_oid, new_oid); if ((flags & REF_HAVE_OLD) && old_oid) oidcpy(&update->old_oid, old_oid); - if (!(flags & REF_SKIP_CREATE_REFLOG)) + if (!(flags & REF_SKIP_CREATE_REFLOG)) { + update->committer_info = xstrdup_or_null(committer_info); update->msg = normalize_reflog_message(msg); + } return update; } @@ -1253,7 +1256,7 @@ int ref_transaction_update(struct ref_transaction *transaction, ref_transaction_add_update(transaction, refname, flags, new_oid, old_oid, new_target, - old_target, msg); + old_target, NULL, msg); return 0; } diff --git a/refs/files-backend.c b/refs/files-backend.c index 02cb4907d8659e87a227fed4f60a5f6606be8764..255fed8354cae982f785b1b85340e2a1eeecf2a6 100644 --- a/refs/files-backend.c +++ b/refs/files-backend.c @@ -1270,7 +1270,7 @@ static void prune_ref(struct files_ref_store *refs, struct ref_to_prune *r) ref_transaction_add_update( transaction, r->name, REF_NO_DEREF | REF_HAVE_NEW | REF_HAVE_OLD | REF_IS_PRUNING, - null_oid(), &r->oid, NULL, NULL, NULL); + null_oid(), &r->oid, NULL, NULL, NULL, NULL); if (ref_transaction_commit(transaction, &err)) goto cleanup; @@ -2417,7 +2417,7 @@ static int split_head_update(struct ref_update *update, transaction, "HEAD", update->flags | REF_LOG_ONLY | REF_NO_DEREF, &update->new_oid, &update->old_oid, - NULL, NULL, update->msg); + NULL, NULL, update->committer_info, update->msg); /* * Add "HEAD". This insertion is O(N) in the transaction @@ -2481,7 +2481,8 @@ static int split_symref_update(struct ref_update *update, transaction, referent, new_flags, update->new_target ? NULL : &update->new_oid, update->old_target ? NULL : &update->old_oid, - update->new_target, update->old_target, update->msg); + update->new_target, update->old_target, NULL, + update->msg); new_update->parent_update = update; @@ -2914,7 +2915,7 @@ static int files_transaction_prepare(struct ref_store *ref_store, packed_transaction, update->refname, REF_HAVE_NEW | REF_NO_DEREF, &update->new_oid, NULL, - NULL, NULL, NULL); + NULL, NULL, NULL, NULL); } } @@ -3094,12 +3095,13 @@ static int files_transaction_finish_initial(struct files_ref_store *refs, ref_transaction_add_update(loose_transaction, update->refname, update->flags & ~REF_HAVE_OLD, update->new_target ? NULL : &update->new_oid, NULL, - update->new_target, NULL, NULL); + update->new_target, NULL, update->committer_info, + NULL); } else { ref_transaction_add_update(packed_transaction, update->refname, update->flags & ~REF_HAVE_OLD, &update->new_oid, &update->old_oid, - NULL, NULL, NULL); + NULL, NULL, update->committer_info, NULL); } } diff --git a/refs/refs-internal.h b/refs/refs-internal.h index f5c733d099f0c6f1076a25f4f77d9d5eb345ec87..79b287c5ec5c7d8f759869cf93cda405640186dc 100644 --- a/refs/refs-internal.h +++ b/refs/refs-internal.h @@ -162,6 +162,7 @@ struct ref_update *ref_transaction_add_update( const struct object_id *new_oid, const struct object_id *old_oid, const char *new_target, const char *old_target, + const char *committer_info, const char *msg); /* diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c index c008f20be719fec3af6a8f81c821cb9c263764d7..b2e3ba877de9e59fea5a4d066eb13e60ef22a32b 100644 --- a/refs/reftable-backend.c +++ b/refs/reftable-backend.c @@ -1078,7 +1078,8 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store, new_update = ref_transaction_add_update( transaction, "HEAD", u->flags | REF_LOG_ONLY | REF_NO_DEREF, - &u->new_oid, &u->old_oid, NULL, NULL, u->msg); + &u->new_oid, &u->old_oid, NULL, NULL, NULL, + u->msg); string_list_insert(&affected_refnames, new_update->refname); } @@ -1161,7 +1162,8 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store, transaction, referent.buf, new_flags, u->new_target ? NULL : &u->new_oid, u->old_target ? NULL : &u->old_oid, - u->new_target, u->old_target, u->msg); + u->new_target, u->old_target, + u->committer_info, u->msg); new_update->parent_update = u; From patchwork Sun Dec 15 16:25:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karthik Nayak X-Patchwork-Id: 13908811 Received: from mail-ej1-f47.google.com (mail-ej1-f47.google.com [209.85.218.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 34EB3193426 for ; Sun, 15 Dec 2024 16:25:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.47 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734279961; cv=none; b=u3V9WzTZU/l/YDkfKk83NCWj/VR+5MI4iWPTOI+URwCNV3L6JMdXXX3/lMi4CZqUG3YfDViGF/IqVoNUPAhunlCGwFHmazg65gS0IszaugucjuBdzhMGi4a9P1COevPfZ8BSPCevMSm+IcHe3mrDCDvLnTnuj8ptXFQp5WkTo+Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734279961; c=relaxed/simple; bh=KzZfc3eupGdaCgu8IhgU0wTKwFN0zZpW2VhtYjGjgj0=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=YX/kdakQ5ZW3I2BS/KPng6dG3OzjY7I7IcsJSjsnS7WiW5RQqrejinGKosXu7ogho+p6+HNk0Ba2++JgAlgLA6u2xW8T6iynmo6vR76H3DmSOP+Jl313rQ58Dmrdw7qXnw8NwlmuCLX/C7OIDarGP94qloa0XqYPowSOsjWjju8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=clvCVcdP; arc=none smtp.client-ip=209.85.218.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="clvCVcdP" Received: by mail-ej1-f47.google.com with SMTP id a640c23a62f3a-aa662795ca3so850511366b.1 for ; Sun, 15 Dec 2024 08:25:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734279957; x=1734884757; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=JbLgtM6fZRugS/23dnDGKiBeriksEAzT/0GxzT9Q5yo=; b=clvCVcdPatptCljTqGPcyNhkvfDSnPb5cROvLmFpEn0GR0voROjoRMcEpHKDgNvSQ1 kLEXDJcPkGQl5O1ppS+b7H/cdHj9ncmy29CQy0oz8dxbN+FSI/v043rCbkK6XZQLs0NV /6yL96uyLwHQOACjuy7PYiWIYl167KQSTzR35e9erzqYZ0T8lGS3EbDfnH2nKWyyz60Y tChTW4+08CbfZCYu4fCBT5O7DwTcmQDGl/4QoiYSo/PnmLXNJdf/v2gt2xqfKR4i1Ta1 MSmp8W5win+a+9TrrR0VZhcL4tNiGwevCL3yPPm4ic0i+AJHMXt2+V0TZeRf7T+XZ52C LkVg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734279957; x=1734884757; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JbLgtM6fZRugS/23dnDGKiBeriksEAzT/0GxzT9Q5yo=; b=hX3fkXN3iWFZ0T6P3kNSzqEJEY1nvm59IOhBrKuZZxbbigY7XHM7uwfqnhXFL+svIq t+3ZsVp3ryyK56bysYzmzFtTB/7bOYH/CB+QNoJ2HaH7BQb40LGSquYERba4pB/OLEBv +oBZe+fi5oEG4CZ+w9RVPlLPwFTtpFhXlRExsKGMOhZZx8LBDTMVxnDMp7B0wh7IaGr5 RUnh3pTavn15CZrBn4HnhmFJehqpVgmCG3BRBeZI3dM7SWcqt+Hffh9UvKm0ffu189NA yn8ExdjxqSu8cqjx6Z13+rqT6QIHE0TZKqt3DybFH+szZpwVSLTQqF1aNpbXsGVWqBSN lmsA== X-Gm-Message-State: AOJu0YzpasAbWIFj2W7fQW+UusZFC5+3LTPM+KlUmClkAJCQuCGxS+nC LfJRcOZF8JyVIfJVf/MGQZrGUM1o8I3yQEg5jzCKWkiR9S2Q6ivN X-Gm-Gg: ASbGncs6/AoBLY8+4FD7/LcsRU6MZfPbq5RF1z6BYscDNePvRdzHLkI1aOAFG9pSwaV qkryfN4Ybr2BuMACy83Y9GPNvTguUC5GvpI7zmZr7zQSwUhocFnd89vZthO2DFAJEdo9r6SGH2k 3GXoBFlWdjKbCfatk0C+u7duO5LCSxIIZTY/4WH2U2IFbEyhLtrgp7tVcqjKjdp7m9SEa1JliD2 d0Err00CPEypDcXTAgtwKRN+lke3eOAhBsEtN/52m9TR87zSaq9l22/5MT3sgKF7Ex3qw== X-Google-Smtp-Source: AGHT+IFU75ruSZpv/8WxWSQJDO9VJ/dqcDlz6/0q9LM0vRcCnMI0QWD2DoG3FAa/fKAG9f56Wx7PiA== X-Received: by 2002:a17:907:1c9e:b0:aa6:489e:5848 with SMTP id a640c23a62f3a-aab7b8485admr849445266b.25.1734279957297; Sun, 15 Dec 2024 08:25:57 -0800 (PST) Received: from [192.168.178.20] ([2a02:2455:8256:2d00:9c39:c2d7:aedd:294d]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aab9638ad1asm224887866b.139.2024.12.15.08.25.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 15 Dec 2024 08:25:56 -0800 (PST) From: Karthik Nayak Date: Sun, 15 Dec 2024 17:25:43 +0100 Subject: [PATCH v3 6/8] refs: introduce the `ref_transaction_update_reflog` function Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241215-320-git-refs-migrate-reflogs-v3-6-4127fe707b98@gmail.com> References: <20241215-320-git-refs-migrate-reflogs-v3-0-4127fe707b98@gmail.com> In-Reply-To: <20241215-320-git-refs-migrate-reflogs-v3-0-4127fe707b98@gmail.com> To: git@vger.kernel.org Cc: Karthik Nayak , ps@pks.im, Christian Couder X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=6145; i=karthik.188@gmail.com; h=from:subject:message-id; bh=KzZfc3eupGdaCgu8IhgU0wTKwFN0zZpW2VhtYjGjgj0=; b=owEB7QES/pANAwAIAT7VnySORox/AcsmYgBnXwMOgPn2PqpXM7/2RLuD6z7dlzrfvePMx7Fu2 l8rZbpUjd2JAbMEAAEIAB0WIQRXzkx/Y3VxD8tlxgY+1Z8kjkaMfwUCZ18DDgAKCRA+1Z8kjkaM f7QfDACe1yHb05TYla5iJel8LrcjvpXfhMOdJ5h2iZA++ai1uqAFlUWxAn3xmOPkUjIDLgAc1UL xsvO6wkeTRfYEI/g4dFzzLOtUzExThjdgMo3DdvgLMWqUg+Xhfn+58ZmGmUpRupDGdPGwhs0QlT bt/Xii5Bxt0Di4Koj78wXuNgVSf4d9xBUstj58yh22LVNchWenE5/gP8B6kcs9zwCLiK8B9Z58S puE/TRqDRKzMNf8k/m2Qz7M4yJ742IPXB5hq9pbEaSS6CP205K0GyALjY3gqPpWUjpuiyCeBGPY mEbqOb6Imypo1Je3pcRGM6TRpyHNVBqTjzbpELT7DQRizVbZwqqeZqt6SsztJiZ45wMNDVcIhqJ acuYIXzhES8QcfJ5rgngdP4J8/gWf+IS65kvIAne0Edo7doBEyQJUijyog1onwV/YtI2q9Hae9V MeXBcIbFC8OMzxWwwj7+8YSgj2Yc82k0zcbMKNZ37mgGXOEjwqNVMwDhsF1M3yE7oPyTY= X-Developer-Key: i=karthik.188@gmail.com; a=openpgp; fpr=57CE4C7F6375710FCB65C6063ED59F248E468C7F Introduce a new function `ref_transaction_update_reflog`, for clients to add a reflog update to a transaction. While the existing function `ref_transaction_update` also allows clients to add a reflog entry, this function does a few things more, It: - Enforces that only a reflog entry is added and does not update the ref itself. - Allows the users to also provide the committer information. This means clients can add reflog entries with custom committer information. The `transaction_refname_valid()` function also modifies the error message selectively based on the type of the update. This change also affects reflog updates which go through `ref_transaction_update()`. A follow up commit will utilize this function to add reflog support to `git refs migrate`. Signed-off-by: Karthik Nayak --- refs.c | 39 +++++++++++++++++++++++++++++++++++---- refs.h | 14 ++++++++++++++ refs/files-backend.c | 24 ++++++++++++++++-------- 3 files changed, 65 insertions(+), 12 deletions(-) diff --git a/refs.c b/refs.c index 782bf1090af65196263a3c35ed18d878bb4f2967..8b3882cff17e5e3b0376f75654e32f81a23e5cb2 100644 --- a/refs.c +++ b/refs.c @@ -1207,14 +1207,14 @@ static int transaction_refname_valid(const char *refname, return 1; if (is_pseudo_ref(refname)) { - strbuf_addf(err, _("refusing to update pseudoref '%s'"), - refname); + const char *what = flags & REF_LOG_ONLY ? "reflog for pseudoref" : "pseudoref"; + strbuf_addf(err, _("refusing to update %s '%s'"), what, refname); return 0; } else if ((new_oid && !is_null_oid(new_oid)) ? check_refname_format(refname, REFNAME_ALLOW_ONELEVEL) : !refname_is_safe(refname)) { - strbuf_addf(err, _("refusing to update ref with bad name '%s'"), - refname); + const char *what = flags & REF_LOG_ONLY ? "reflog with bad name" : "ref with bad name"; + strbuf_addf(err, _("refusing to update %s '%s'"), what, refname); return 0; } @@ -1257,6 +1257,37 @@ int ref_transaction_update(struct ref_transaction *transaction, ref_transaction_add_update(transaction, refname, flags, new_oid, old_oid, new_target, old_target, NULL, msg); + + return 0; +} + +int ref_transaction_update_reflog(struct ref_transaction *transaction, + const char *refname, + const struct object_id *new_oid, + const struct object_id *old_oid, + const char *committer_info, unsigned int flags, + const char *msg, unsigned int index, + struct strbuf *err) +{ + struct ref_update *update; + + assert(err); + + flags |= REF_LOG_ONLY | REF_NO_DEREF; + + if (!transaction_refname_valid(refname, new_oid, flags, err)) + return -1; + + update = ref_transaction_add_update(transaction, refname, flags, + new_oid, old_oid, NULL, NULL, + committer_info, msg); + /* + * While we do set the old_oid value, we unset the flag to skip + * old_oid verification which only makes sense for refs. + */ + update->flags &= ~REF_HAVE_OLD; + update->index = index; + return 0; } diff --git a/refs.h b/refs.h index a5bedf48cf6de91005a7e8d0bf58ca98350397a6..b0dfc65ed2e59c4b66967840339f81e7746a96d3 100644 --- a/refs.h +++ b/refs.h @@ -727,6 +727,20 @@ int ref_transaction_update(struct ref_transaction *transaction, unsigned int flags, const char *msg, struct strbuf *err); +/* + * Similar to`ref_transaction_update`, but this function is only for adding + * a reflog update. Supports providing custom committer information. The index + * field can be utiltized to order updates as desired. When not used, the + * updates default to being ordered by refname. + */ +int ref_transaction_update_reflog(struct ref_transaction *transaction, + const char *refname, + const struct object_id *new_oid, + const struct object_id *old_oid, + const char *committer_info, unsigned int flags, + const char *msg, unsigned int index, + struct strbuf *err); + /* * Add a reference creation to transaction. new_oid is the value that * the reference should have after the update; it must not be diff --git a/refs/files-backend.c b/refs/files-backend.c index 255fed8354cae982f785b1b85340e2a1eeecf2a6..c11213f52065bcf2fa7612df8f9500692ee2d02c 100644 --- a/refs/files-backend.c +++ b/refs/files-backend.c @@ -3080,10 +3080,12 @@ static int files_transaction_finish_initial(struct files_ref_store *refs, } /* - * packed-refs don't support symbolic refs and root refs, so we - * have to queue these references via the loose transaction. + * packed-refs don't support symbolic refs, root refs and reflogs, + * so we have to queue these references via the loose transaction. */ - if (update->new_target || is_root_ref(update->refname)) { + if (update->new_target || + is_root_ref(update->refname) || + (update->flags & REF_LOG_ONLY)) { if (!loose_transaction) { loose_transaction = ref_store_transaction_begin(&refs->base, 0, err); if (!loose_transaction) { @@ -3092,11 +3094,17 @@ static int files_transaction_finish_initial(struct files_ref_store *refs, } } - ref_transaction_add_update(loose_transaction, update->refname, - update->flags & ~REF_HAVE_OLD, - update->new_target ? NULL : &update->new_oid, NULL, - update->new_target, NULL, update->committer_info, - NULL); + if (update->flags & REF_LOG_ONLY) + ref_transaction_add_update(loose_transaction, update->refname, + update->flags, &update->new_oid, + &update->old_oid, NULL, NULL, + update->committer_info, update->msg); + else + ref_transaction_add_update(loose_transaction, update->refname, + update->flags & ~REF_HAVE_OLD, + update->new_target ? NULL : &update->new_oid, NULL, + update->new_target, NULL, update->committer_info, + NULL); } else { ref_transaction_add_update(packed_transaction, update->refname, update->flags & ~REF_HAVE_OLD, From patchwork Sun Dec 15 16:25:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karthik Nayak X-Patchwork-Id: 13908812 Received: from mail-ej1-f44.google.com (mail-ej1-f44.google.com [209.85.218.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DD9C21940B3 for ; Sun, 15 Dec 2024 16:25:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.44 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734279961; cv=none; b=bnegeeNTVh0y3TsBnKDZDLJ94aoR7RgrbFHg7rFeZHTENpf4wRf0lFMkr/H7eI7rtC6qNFm13zGz66ZXBPQnL/sHt2RpKu6Xi0oTuErfbNyGa0mgJCTby1HgCvHO/U+/ZbDEmrfouWVX9m+iaLBxRHrfzObTHKJpKonRHRrlFf8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734279961; c=relaxed/simple; bh=jDPMqKatsQhg+EBjNPw/kC4fYprM2j3L9khrIu5BB0Q=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=bLGc0CyTUXKNGj/hu4HeibY0NrN2CKAp0bZwfUyAJWeBz6tnPldqk+o/+N0OvciGQyMzYyJwT/SIGBX2MT6/XsKGRPDNIAoy4s04/clJaFhjtuHCRdhqLXrEkKKvmoyJ2eXq3XnQut1hBV4uUNTP+Lty9T8DQ0tiF3diPVFyxP0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=SS0VR5le; arc=none smtp.client-ip=209.85.218.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="SS0VR5le" Received: by mail-ej1-f44.google.com with SMTP id a640c23a62f3a-aab925654d9so336669766b.2 for ; Sun, 15 Dec 2024 08:25:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734279958; x=1734884758; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=hFja4Dce2bjeu8KBc7aHCgLjbi7WVN8kJntGzJ+xVtg=; b=SS0VR5leARAq0rq78Wquk2MjLjhTpyvq3jHXjNnbEr1bpxk/DeiZb7BXokLRqtA7IN U3lZFNddDRPiwq/CuWwHnY3Ob/C6FKcotJ8x+TyzrdekHbLG34cBvZId7w6Vo/DRfpmd nd5xZTV/N3UcD/gnHI2NmSb38Nv8uf9baOSMOFiQ+sugQbyB+sok/AQE0BLCsG0XIfuP Trtu6YhGYwa/T+1PpkxJbURVlEQ/3Vc8KyQ0l+nQwR4SCYmsC1hXHFMzCFDJ8a1BWnck 1YzaGVO6JCZIMbBwkQIejvV79JwFShs+/N33xBI7KDASPoKbUv4x6euQkt4aVbmHAkJQ fBMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734279958; x=1734884758; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hFja4Dce2bjeu8KBc7aHCgLjbi7WVN8kJntGzJ+xVtg=; b=Ws/dk0B7R85Wi3WFpXp5MzSCTBYg1FO7pLEm8nGEGnLu5hHY1ElAmJqgiYB1M+mpjL kqT6+Wf0fZb8PnAa6/QWtJNzlGng2bTNKfY/4ohiHLcAFJeipaseXi/uhrZxKLYxf6Ma xpSJU5bBTSfA4VWP7T4BHpm94zua26OGKzF56IwSobWWIwLgtbhR1xzdN+zc+67GLq8E jdLsi+9xb6xv2jdvV3jaioFcW4DtgJPcju+IulmaMJ3nD8WxvQFK22q6cMXDyonSjvHG phTIf6MhBEqJI7rx5h0Yj/RumZcOv1KZZDwY0xoQlP/BTS19ZTBA+6p9xc00jP9eo9D/ NxEw== X-Gm-Message-State: AOJu0YysNtcxwEDGAsBh2y+xhdMUbdkNzF34eWb8UAzSeXEiob4CODYS bmu9l/vGlOoHdpv+vNdNP19d3u8dYwD0XP7rr7Ogg/mS/8K7n5D0 X-Gm-Gg: ASbGncvRLZchKmiT0H9WB5S/HjeMC2WtNb2rN7QI3kNDeVv2AGSk5xKHqctjD5IRF2I XgDCN+b1be6G2Mzd6IzLr5eYnZ/0LkiHsdifMivDEOHFZHF9AA6K0VseMBesTIZqoyq+gPMfWN7 kFntvrxL7RKevep+1DE8iYI7KPksNe4jw741o5bssidVvWb4yGydWjajkcSB/Z5qcvq+2PCoT0n 4gkzuTQpnxN01wRx7o58WwjyEZb8fuDzhsI5Dj9BhF2MP9E4KFghzcfF9CgxSi74fsSuw== X-Google-Smtp-Source: AGHT+IEpbq135coc0PrsYDSJfTmgYHumdNb87a2VFfzxX8OFxH2RktPZ936y8B7CeGAMfolb1GavPQ== X-Received: by 2002:a17:906:f58e:b0:aa6:2fdc:db1a with SMTP id a640c23a62f3a-aab77e8ab9bmr861133166b.38.1734279958043; Sun, 15 Dec 2024 08:25:58 -0800 (PST) Received: from [192.168.178.20] ([2a02:2455:8256:2d00:9c39:c2d7:aedd:294d]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aab9638ad1asm224887866b.139.2024.12.15.08.25.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 15 Dec 2024 08:25:57 -0800 (PST) From: Karthik Nayak Date: Sun, 15 Dec 2024 17:25:44 +0100 Subject: [PATCH v3 7/8] refs: allow multiple reflog entries for the same refname Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241215-320-git-refs-migrate-reflogs-v3-7-4127fe707b98@gmail.com> References: <20241215-320-git-refs-migrate-reflogs-v3-0-4127fe707b98@gmail.com> In-Reply-To: <20241215-320-git-refs-migrate-reflogs-v3-0-4127fe707b98@gmail.com> To: git@vger.kernel.org Cc: Karthik Nayak , ps@pks.im, Christian Couder X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=4945; i=karthik.188@gmail.com; h=from:subject:message-id; bh=jDPMqKatsQhg+EBjNPw/kC4fYprM2j3L9khrIu5BB0Q=; b=owEB7QES/pANAwAIAT7VnySORox/AcsmYgBnXwMOb82QdwdoIBEfJT+tJHkyOa/fgHvHcYBcU Nssp2/DTSaJAbMEAAEIAB0WIQRXzkx/Y3VxD8tlxgY+1Z8kjkaMfwUCZ18DDgAKCRA+1Z8kjkaM f13pC/9YIwIdj362MmG8xJEM96iS1PWxavA0ollutg+IXF8iM55G1jdFGBoXqJw+PtKDZmbvX5B cLQE1QlywTRWZ+madOEmF3raZ82gXIqNe0QXNeoeJvJg3QIMBc8ZmLLX35lOjZkYYrTvYEOfjS6 7O5tkppDjfUOwzDy+UYR3O6E0FyGJlUZK5xjp+bFJzOuHDU7tVivcAir1t0kHdlm3ZJ7KpuPYcE 6HIduhEPW6aVpgGNMNryTJcke9vnquK0V9P/slBAmOG9JFI/nuF2rOTOizVnwTjfnSqqtGhq6fC TnQG+lK3VKpSrU2hwFTCeeXoDNiuM8um8NkFI98jq8h/mBiUrD1pTMIeq1Kma0UEBNA7zUA+Fvn uVbCZvRp/QfhvnOUthRRHXvlscH39B1KrCWYE14uwcNrMJazO4DXjD34SUBToo0aEL+MnbifnjR VbDm76Vt1zI16W16/F/RD+XfI8jmpHnCyjjb1YA7twElhfCkv7m0DVlPyv0jZwbuXaoWk= X-Developer-Key: i=karthik.188@gmail.com; a=openpgp; fpr=57CE4C7F6375710FCB65C6063ED59F248E468C7F The reference transaction only allows a single update for a given reference to avoid conflicts. This, however, isn't an issue for reflogs. There are no conflicts to be resolved in reflogs and when migrating reflogs between backends we'd have multiple reflog entries for the same refname. So allow multiple reflog updates within a single transaction. Also the reflog creation logic isn't exposed to the end user. While this might change in the future, currently, this reduces the scope of issues to think about. In the reftable backend, the writer sorts all updates based on the update_index before writing to the block. When there are multiple reflogs for a given refname, it is essential that the order of the reflogs is maintained. So add the `index` value to the `update_index`. The `index` field is only set when multiple reflog entries for a given refname are added and as such in most scenarios the old behavior remains. This is required to add reflog migration support to `git refs migrate`. Signed-off-by: Karthik Nayak --- refs/files-backend.c | 15 +++++++++++---- refs/reftable-backend.c | 22 +++++++++++++++++++--- 2 files changed, 30 insertions(+), 7 deletions(-) diff --git a/refs/files-backend.c b/refs/files-backend.c index c11213f52065bcf2fa7612df8f9500692ee2d02c..8953d1c6d37b13b0db701888b3db92fd87a68aaa 100644 --- a/refs/files-backend.c +++ b/refs/files-backend.c @@ -2611,6 +2611,9 @@ static int lock_ref_for_update(struct files_ref_store *refs, update->backend_data = lock; + if (update->flags & REF_LOG_ONLY) + goto out; + if (update->type & REF_ISSYMREF) { if (update->flags & REF_NO_DEREF) { /* @@ -2829,13 +2832,16 @@ static int files_transaction_prepare(struct ref_store *ref_store, */ for (i = 0; i < transaction->nr; i++) { struct ref_update *update = transaction->updates[i]; - struct string_list_item *item = - string_list_append(&affected_refnames, update->refname); + struct string_list_item *item; if ((update->flags & REF_IS_PRUNING) && !(update->flags & REF_NO_DEREF)) BUG("REF_IS_PRUNING set without REF_NO_DEREF"); + if (update->flags & REF_LOG_ONLY) + continue; + + item = string_list_append(&affected_refnames, update->refname); /* * We store a pointer to update in item->util, but at * the moment we never use the value of this field @@ -3035,8 +3041,9 @@ static int files_transaction_finish_initial(struct files_ref_store *refs, /* Fail if a refname appears more than once in the transaction: */ for (i = 0; i < transaction->nr; i++) - string_list_append(&affected_refnames, - transaction->updates[i]->refname); + if (!(transaction->updates[i]->flags & REF_LOG_ONLY)) + string_list_append(&affected_refnames, + transaction->updates[i]->refname); string_list_sort(&affected_refnames); if (ref_update_reject_duplicates(&affected_refnames, err)) { ret = TRANSACTION_GENERIC_ERROR; diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c index b2e3ba877de9e59fea5a4d066eb13e60ef22a32b..bec5962debea7b62572d08f6fa8fd38ab4cd8af6 100644 --- a/refs/reftable-backend.c +++ b/refs/reftable-backend.c @@ -990,8 +990,9 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store, if (ret) goto done; - string_list_append(&affected_refnames, - transaction->updates[i]->refname); + if (!(transaction->updates[i]->flags & REF_LOG_ONLY)) + string_list_append(&affected_refnames, + transaction->updates[i]->refname); } /* @@ -1301,6 +1302,7 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data struct reftable_log_record *logs = NULL; struct ident_split committer_ident = {0}; size_t logs_nr = 0, logs_alloc = 0, i; + uint64_t max_update_index = ts; const char *committer_info; int ret = 0; @@ -1405,7 +1407,19 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data } fill_reftable_log_record(log, &c); - log->update_index = ts; + + /* + * Updates are sorted by the writer. So updates for the same + * refname need to contain different update indices. + */ + log->update_index = ts + u->index; + + /* + * Note the max update_index so the limit can be set later on. + */ + if (log->update_index > max_update_index) + max_update_index = log->update_index; + log->refname = xstrdup(u->refname); memcpy(log->value.update.new_hash, u->new_oid.hash, GIT_MAX_RAWSZ); @@ -1469,6 +1483,8 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data * and log blocks. */ if (logs) { + reftable_writer_set_limits(writer, ts, max_update_index); + ret = reftable_writer_add_logs(writer, logs, logs_nr); if (ret < 0) goto done; From patchwork Sun Dec 15 16:25:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karthik Nayak X-Patchwork-Id: 13908813 Received: from mail-ed1-f41.google.com (mail-ed1-f41.google.com [209.85.208.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D31F5194A48 for ; Sun, 15 Dec 2024 16:26:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.41 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734279963; cv=none; b=qOwU5LZQsQhVhIL0bpQBZddHDEtX2xOZGadaSO0wEZX6RBqGajrZNnRalv1WaS3NpUdSCtwYaNwrhNOISAZse56SqZKDRraZoK436J01UBZyh8YUd61wqD44tPkLKB8vVb2Vf9yYAqN1PzLrRcbBk73F5NCv2yiWnM26QhmERRg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734279963; c=relaxed/simple; bh=zW/FnKkrM2PUKFsvbiMZ/XJ09fEZtN+WkaFko91Fvzo=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=HTUGlYXXnWfnqcqcKj3Gbzc3E3mSoYFuBZW3ro63hqpc7rG6HJmY1onSsXhM1lFeJQREJHtesmUApxozfvLnWaWOavzeq3WSRoGEyAvFzwYy0HjIFs6KIi3YzKDcwAa95baCEgJ6BVbRSzJWt2FF4MguHtc1YSo/G5+ZbxgDLkM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=RP0jriJ0; arc=none smtp.client-ip=209.85.208.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="RP0jriJ0" Received: by mail-ed1-f41.google.com with SMTP id 4fb4d7f45d1cf-5d27243ba8bso6260673a12.2 for ; Sun, 15 Dec 2024 08:26:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734279959; x=1734884759; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=xzx/hpb0tlRTDTgKeQlXGTFN5b9cnPUAa/S44t41PsI=; b=RP0jriJ0Dd/dYD7BByYUgO5oEJEBfR03delVjBJHWoi0FGlu5sJVYi5dfkJNLJjvBy 3j6i4Lx3BGjGJ0H8lMP4xSP1gxlb9LEQn3DgsgRqEBFYw9aZIAJB1fDrrTz/nYz9GRhs Fk8/MT+ITTCedGCy4xDMUqiBvEkfdZptbXGLF4m+T/h002IBss2s2gNORParTMie0zpP DNBNe43VWBNfcjZ1btmJqcKPi8ES3Zbu79YdCDzcA8/J00ouBiLoNv/50stNnnXLJqMM IPWNBZlQehmD1RvyeBfYrsiurc+XYgdM/1Kq7A4tYbHiO9HJo9HWU1/jFqiw71xvKZKU LTTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734279959; x=1734884759; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xzx/hpb0tlRTDTgKeQlXGTFN5b9cnPUAa/S44t41PsI=; b=hgxhxJWFIId7IYVeUMEQJ8T0QYCe3tlraBkphgnwRtJgYYMfKoWI12R3u24ESu670W NGZP8yYssZPHx5NAaUk6kYCF61QM1P0TKDE1HEkfJLNdxH29OJ6mZNBD9Px8FnP6h+Ul w2g8ldV112yyH1zplW4SmDKH0YjspZOmJ8/oMQEDygPqqdrLMVoY4ipynqlYuwnhtAMS 4qiGN4f0gfsX9eLxKbE/1NbyDW4qgrizgY/LSQKnqFjPyzs2q21oYl9ZSVaePWXm540x cid/1smWORAQEGSKfxqyqpszHH9Zdv9/kCHAU0fivUi+mG7OvYC3V6H8AHBDdF2NOl5/ C3qA== X-Gm-Message-State: AOJu0YwZDXMvMwlkJbvks+Vra+nTDtyVYC6f6lmgeC0fyV3NCZM/NzjT d262hVnb/W6I+vbS8pR1ti03IPs8caZvp8Toy+uALAY0TiyVJpY/ X-Gm-Gg: ASbGnctIjedDWpA6g63N+5AOBjwLZhl9K2WtpE2EHCW4wODlgwBpvUMCCF1PbdJho+t LXeht01n88F90lIqU8QKO4N8AWi9tK/XEcXymcFz6oLbpaDXKssqgAusU8f/qbEgXDFiyE76a0j 217e+cV8soRC1N41lz7zwgsNFjgMzRH8547DNNnfeYcrabPvLVllI3GHWRTOUate0uuwSkbTXYm fVuq/XmEtzSZyM+uusz+40vjZPiydFvr1nv2oAPbxRDroLZFglyZD/tgfTfbcP7JKz2iw== X-Google-Smtp-Source: AGHT+IFO44eVBYAJtHmJkxXFIkaaAd2JgTbLYvx0J177wsqTOG3Dwi+ZGIAtgLuUB47aRvJsg6qhbA== X-Received: by 2002:a17:907:7f17:b0:aa6:519c:ef9a with SMTP id a640c23a62f3a-aab77ead24bmr888229566b.53.1734279958814; Sun, 15 Dec 2024 08:25:58 -0800 (PST) Received: from [192.168.178.20] ([2a02:2455:8256:2d00:9c39:c2d7:aedd:294d]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aab9638ad1asm224887866b.139.2024.12.15.08.25.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 15 Dec 2024 08:25:58 -0800 (PST) From: Karthik Nayak Date: Sun, 15 Dec 2024 17:25:45 +0100 Subject: [PATCH v3 8/8] refs: add support for migrating reflogs Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241215-320-git-refs-migrate-reflogs-v3-8-4127fe707b98@gmail.com> References: <20241215-320-git-refs-migrate-reflogs-v3-0-4127fe707b98@gmail.com> In-Reply-To: <20241215-320-git-refs-migrate-reflogs-v3-0-4127fe707b98@gmail.com> To: git@vger.kernel.org Cc: Karthik Nayak , ps@pks.im, Christian Couder X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=11509; i=karthik.188@gmail.com; h=from:subject:message-id; bh=zW/FnKkrM2PUKFsvbiMZ/XJ09fEZtN+WkaFko91Fvzo=; b=owEB7QES/pANAwAIAT7VnySORox/AcsmYgBnXwMPBAQCceFdbcLbEN7CsRB8GgrBga+8Ja6T/ hUlBBLXZMGJAbMEAAEIAB0WIQRXzkx/Y3VxD8tlxgY+1Z8kjkaMfwUCZ18DDwAKCRA+1Z8kjkaM f5TwC/9idQ5f6+sMIBIhv5P7V+8MDoqcuZEPu7ejLzf10xptnbo+4MRbAc0eJWBFwuYrtPxmLWo PVSlvto4x3+T/BSHAPrYbpzVhJhY3NmWzueBqprIEB1NcW8lk/dScwyfQqvyyi3ZfdmSkJ8BlZJ oCMiWt+EdjXy05EJxm4LpGcneBQUP/iyKr/2F6D7n/mv0mQQxIVoCVPQlVA0da4CPXkW/y+dZIz xGWptBTXJ+d6LT9NjFc5hC7CGGxlRIbDPTKTVWRA3i/ccDLWzyr9uvwKolod8Wk7NBTEXcAiP24 +FxMGnb3KCJl0R05rnhhGzHemj22vNRmE6x9tcx4+GV4l9JzITPhwH/opXFid++IaD8uGyoTxy+ rng8pqycEcFfaQFzJCCVRPbfx2gH4M8SLwOz40qeLzEVVegUa8Ywwo8VEPUEl2XKq8d2A5vDxKF o2xlovzcLDOxsXfxnn6JoWKMFmpdT8RlTwjbl7Z3QSrr+zC54hBFWrbN02rv59y6+Y3Tk= X-Developer-Key: i=karthik.188@gmail.com; a=openpgp; fpr=57CE4C7F6375710FCB65C6063ED59F248E468C7F The `git refs migrate` command was introduced in 25a0023f28 (builtin/refs: new command to migrate ref storage formats, 2024-06-06) to support migrating from one reference backend to another. One limitation of the command was that it didn't support migrating repositories which contained reflogs. A previous commit, added support for adding reflog updates in ref transactions. Using the added functionality bake in reflog support for `git refs migrate`. To ensure that the order of the reflogs is maintained during the migration, we add the index for each reflog update as we iterate over the reflogs from the old reference backend. This is to ensure that the order is maintained in the new backend. Signed-off-by: Karthik Nayak --- Documentation/git-refs.txt | 2 -- refs.c | 89 ++++++++++++++++++++++++++++++++-------------- t/t1460-refs-migrate.sh | 73 +++++++++++++++++++++++++------------ 3 files changed, 113 insertions(+), 51 deletions(-) diff --git a/Documentation/git-refs.txt b/Documentation/git-refs.txt index ce31f93061db5e5d16aca516dd3d15f6527db870..9829984b0a4c4f54ec7f9b6c6c7072f62b1d198d 100644 --- a/Documentation/git-refs.txt +++ b/Documentation/git-refs.txt @@ -57,8 +57,6 @@ KNOWN LIMITATIONS The ref format migration has several known limitations in its current form: -* It is not possible to migrate repositories that have reflogs. - * It is not possible to migrate repositories that have worktrees. * There is no way to block concurrent writes to the repository during an diff --git a/refs.c b/refs.c index 8b3882cff17e5e3b0376f75654e32f81a23e5cb2..4a74f7c7bd0314ad8e6c4cbea436df934b2c7f88 100644 --- a/refs.c +++ b/refs.c @@ -30,6 +30,7 @@ #include "date.h" #include "commit.h" #include "wildmatch.h" +#include "ident.h" /* * List of all available backends @@ -2673,6 +2674,7 @@ struct migration_data { struct ref_store *old_refs; struct ref_transaction *transaction; struct strbuf *errbuf; + struct strbuf sb; }; static int migrate_one_ref(const char *refname, const char *referent UNUSED, const struct object_id *oid, @@ -2705,6 +2707,52 @@ static int migrate_one_ref(const char *refname, const char *referent UNUSED, con return ret; } +struct reflog_migration_data { + unsigned int index; + const char *refname; + struct ref_store *old_refs; + struct ref_transaction *transaction; + struct strbuf *errbuf; + struct strbuf *sb; +}; + +static int migrate_one_reflog_entry(struct object_id *old_oid, + struct object_id *new_oid, + const char *committer, + timestamp_t timestamp, int tz, + const char *msg, void *cb_data) +{ + struct reflog_migration_data *data = cb_data; + const char *date; + int ret; + + date = show_date(timestamp, tz, DATE_MODE(NORMAL)); + strbuf_reset(data->sb); + /* committer contains name and email */ + strbuf_addstr(data->sb, fmt_ident("", committer, WANT_BLANK_IDENT, date, 0)); + + ret = ref_transaction_update_reflog(data->transaction, data->refname, + new_oid, old_oid, data->sb->buf, + REF_HAVE_NEW | REF_HAVE_OLD, msg, + data->index++, data->errbuf); + return ret; +} + +static int migrate_one_reflog(const char *refname, void *cb_data) +{ + struct migration_data *migration_data = cb_data; + struct reflog_migration_data data; + + data.refname = refname; + data.old_refs = migration_data->old_refs; + data.transaction = migration_data->transaction; + data.errbuf = migration_data->errbuf; + data.sb = &migration_data->sb; + + return refs_for_each_reflog_ent(migration_data->old_refs, refname, + migrate_one_reflog_entry, &data); +} + static int move_files(const char *from_path, const char *to_path, struct strbuf *errbuf) { struct strbuf from_buf = STRBUF_INIT, to_buf = STRBUF_INIT; @@ -2771,13 +2819,6 @@ static int move_files(const char *from_path, const char *to_path, struct strbuf return ret; } -static int count_reflogs(const char *reflog UNUSED, void *payload) -{ - size_t *reflog_count = payload; - (*reflog_count)++; - return 0; -} - static int has_worktrees(void) { struct worktree **worktrees = get_worktrees(); @@ -2803,7 +2844,6 @@ int repo_migrate_ref_storage_format(struct repository *repo, struct ref_transaction *transaction = NULL; struct strbuf new_gitdir = STRBUF_INIT; struct migration_data data; - size_t reflog_count = 0; int did_migrate_refs = 0; int ret; @@ -2815,21 +2855,6 @@ int repo_migrate_ref_storage_format(struct repository *repo, old_refs = get_main_ref_store(repo); - /* - * We do not have any interfaces that would allow us to write many - * reflog entries. Once we have them we can remove this restriction. - */ - if (refs_for_each_reflog(old_refs, count_reflogs, &reflog_count) < 0) { - strbuf_addstr(errbuf, "cannot count reflogs"); - ret = -1; - goto done; - } - if (reflog_count) { - strbuf_addstr(errbuf, "migrating reflogs is not supported yet"); - ret = -1; - goto done; - } - /* * Worktrees complicate the migration because every worktree has a * separate ref storage. While it should be feasible to implement, this @@ -2855,17 +2880,21 @@ int repo_migrate_ref_storage_format(struct repository *repo, * This operation is safe as we do not yet modify the main * repository. * - * 3. If we're in dry-run mode then we are done and can hand over the + * 3. Enumerate all reflogs and write them into the new ref storage. + * This operation is safe as we do not yet modify the main + * repository. + * + * 4. If we're in dry-run mode then we are done and can hand over the * directory to the caller for inspection. If not, we now start * with the destructive part. * - * 4. Delete the old ref storage from disk. As we have a copy of refs + * 5. Delete the old ref storage from disk. As we have a copy of refs * in the new ref storage it's okay(ish) if we now get interrupted * as there is an equivalent copy of all refs available. * - * 5. Move the new ref storage files into place. + * 6. Move the new ref storage files into place. * - * 6. Change the repository format to the new ref format. + * 7. Change the repository format to the new ref format. */ strbuf_addf(&new_gitdir, "%s/%s", old_refs->gitdir, "ref_migration.XXXXXX"); if (!mkdtemp(new_gitdir.buf)) { @@ -2889,6 +2918,7 @@ int repo_migrate_ref_storage_format(struct repository *repo, data.old_refs = old_refs; data.transaction = transaction; data.errbuf = errbuf; + strbuf_init(&data.sb, 0); /* * We need to use the internal `do_for_each_ref()` here so that we can @@ -2907,6 +2937,10 @@ int repo_migrate_ref_storage_format(struct repository *repo, if (ret < 0) goto done; + ret = refs_for_each_reflog(old_refs, migrate_one_reflog, &data); + if (ret < 0) + goto done; + ret = ref_transaction_commit(transaction, errbuf); if (ret < 0) goto done; @@ -2982,6 +3016,7 @@ int repo_migrate_ref_storage_format(struct repository *repo, } ref_transaction_free(transaction); strbuf_release(&new_gitdir); + strbuf_release(&data.sb); return ret; } diff --git a/t/t1460-refs-migrate.sh b/t/t1460-refs-migrate.sh index 1bfff3a7afd5acc470424dfe7ec3e97d45f5c481..f59bc4860f19c4af82dc6f2984bdb69d61fe3ec2 100755 --- a/t/t1460-refs-migrate.sh +++ b/t/t1460-refs-migrate.sh @@ -7,23 +7,44 @@ export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME . ./test-lib.sh +# Migrate the provided repository from one format to the other and +# verify that the references and logs are migrated over correctly. +# Usage: test_migration +# is the relative path to the repo to be migrated. +# is the ref format to be migrated to. +# (true or false) whether to skip reflog verification. test_migration () { - git -C "$1" for-each-ref --include-root-refs \ + repo=$1 && + format=$2 && + skip_reflog_verify=${3:-false} && + git -C "$repo" for-each-ref --include-root-refs \ --format='%(refname) %(objectname) %(symref)' >expect && - git -C "$1" refs migrate --ref-format="$2" && - git -C "$1" for-each-ref --include-root-refs \ + if ! $skip_reflog_verify + then + git -C "$repo" reflog --all >expect_logs && + git -C "$repo" reflog list >expect_log_list + fi && + + git -C "$repo" refs migrate --ref-format="$2" && + + git -C "$repo" for-each-ref --include-root-refs \ --format='%(refname) %(objectname) %(symref)' >actual && test_cmp expect actual && + if ! $skip_reflog_verify + then + git -C "$repo" reflog --all >actual_logs && + git -C "$repo" reflog list >actual_log_list && + test_cmp expect_logs actual_logs && + test_cmp expect_log_list actual_log_list + fi && - git -C "$1" rev-parse --show-ref-format >actual && - echo "$2" >expect && + git -C "$repo" rev-parse --show-ref-format >actual && + echo "$format" >expect && test_cmp expect actual } test_expect_success 'setup' ' - rm -rf .git && - # The migration does not yet support reflogs. - git config --global core.logAllRefUpdates false + rm -rf .git ' test_expect_success "superfluous arguments" ' @@ -78,19 +99,6 @@ do test_cmp expect err ' - test_expect_success "$from_format -> $to_format: migration with reflog fails" ' - test_when_finished "rm -rf repo" && - git init --ref-format=$from_format repo && - test_config -C repo core.logAllRefUpdates true && - test_commit -C repo logged && - test_must_fail git -C repo refs migrate \ - --ref-format=$to_format 2>err && - cat >expect <<-EOF && - error: migrating reflogs is not supported yet - EOF - test_cmp expect err - ' - test_expect_success "$from_format -> $to_format: migration with worktree fails" ' test_when_finished "rm -rf repo" && git init --ref-format=$from_format repo && @@ -141,7 +149,7 @@ do test_commit -C repo initial && test-tool -C repo ref-store main update-ref "" refs/heads/broken \ "$(test_oid 001)" "$ZERO_OID" REF_SKIP_CREATE_REFLOG,REF_SKIP_OID_VERIFICATION && - test_migration repo "$to_format" && + test_migration repo "$to_format" true && test_oid 001 >expect && git -C repo rev-parse refs/heads/broken >actual && test_cmp expect actual @@ -195,6 +203,27 @@ do git -C repo rev-parse --show-ref-format >actual && test_cmp expect actual ' + + test_expect_success "$from_format -> $to_format: reflogs of symrefs with target deleted" ' + test_when_finished "rm -rf repo" && + git init --ref-format=$from_format repo && + test_commit -C repo initial && + git -C repo branch branch-1 HEAD && + git -C repo symbolic-ref refs/heads/symref refs/heads/branch-1 && + cat >input <<-EOF && + delete refs/heads/branch-1 + EOF + git -C repo update-ref --stdin $to_format: reflogs order is retained" ' + test_when_finished "rm -rf repo" && + git init --ref-format=$from_format repo && + test_commit --date "100005000 +0700" --no-tag -C repo initial && + test_commit --date "100003000 +0700" --no-tag -C repo second && + test_migration repo "$to_format" + ' done done