From patchwork Fri Dec 13 10:36:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karthik Nayak X-Patchwork-Id: 13906827 Received: from mail-ej1-f47.google.com (mail-ej1-f47.google.com [209.85.218.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DDB441BE251 for ; Fri, 13 Dec 2024 10:36:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.47 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734086215; cv=none; b=BVL09Xj9RRawNOagvmGuR6FsJZ2PrNNsAtQ4v203kGgxMptJpSfFcIgm1gc1uT7Y/hzdhGZC8f7XuYdfSvSVVEIXQT5luBd3GIWbEy3XFM1krOUAVju0IhubwMO4XfTEZlMvWnFCHUwFOI1VUHjiXmrN5BCc6b5V7JHoHrjzjgA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734086215; c=relaxed/simple; bh=1zQAW1gf6s8EkfhWmsjwMbxZIi5DU5Z49Kos2YDhOaQ=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=cpdLVQ6fGm769byNmcgK4o+yKy+VvPLoHBJ4tM8uKF3N567kzwqWHxmM30XlWNaXUqHIC0Vdt98PBoqJN1gfgWI9BNQxFMtzZhanOQKGnOA/3bYDagBdjA1XJNPiEsPb2iIYFy0RBnC4A5UdP7FFWWTd5fkRQVg/ZMVgcOUtHvA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=TU6/rvIE; arc=none smtp.client-ip=209.85.218.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="TU6/rvIE" Received: by mail-ej1-f47.google.com with SMTP id a640c23a62f3a-aa5f1909d6fso287602966b.3 for ; Fri, 13 Dec 2024 02:36:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734086212; x=1734691012; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=2yP66MpHoRJMrRT8Yqjo1czS94YPdDgttlZIZ/ZdMEA=; b=TU6/rvIEWR3IZwI3TGL6mlUezUaheBY8VHZSK9a8teFctKqvg/yoAOfe2onYgZip8W dAveA3kFxOTPUkNvpWPvNR6ThokA/vszoInwVtWYIJXfDPoEh2CD+NKOHz4zioxNJWd3 Ns/dv/SP9kZiPQ9KsQDzW2FFCB7kZmFXgLPtFrsST5bpAi5Yaz7aJkS+8WLzQEZ0HFvY 7nDpdcRjN9/k/vu8E1Mr8v+Lwczo6uetu/b3WmR2vm5m1lLgdZYUWq55rLaAoZuY6kSq mPQVc1so1jNFETCGOb/sMkojHELvJ4tGAxiy4kn6fHEApja2ogZF2rUiX3wZ/scotYXf 66gw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734086212; x=1734691012; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2yP66MpHoRJMrRT8Yqjo1czS94YPdDgttlZIZ/ZdMEA=; b=P7HFapmqE2+ohct2EjND1shZQtVZ2sb8N/AoAFaOictyeNSyf8zlhPQs63Xo+rSOme EecgV5aUua2EKVhsuHYusnEti9x/xwCdZTzD7KuOodkr2m7LzCLTjZ0wsv+U7+4nSZVH u6E0gbQSzB1gi2p54Qsv+XgyyyMzXtS04FaND/vf2xvUw9L+f3kvVwd7TSEDxq2PNC3s G4Z6wFT4Aq9NxTboIT1fmtUFtJzCqK0ivx6qUo6Vj5C+RzrQtXHJmfdEgcV4+tYHJ+c1 nmqBnWLhy1GLWxVASukqbgTZz0J/gkhC+3OebpITSPNpyQEKvhrrWQXcHze288diPPRB y73Q== X-Gm-Message-State: AOJu0YwxqX82QpjlrKbCcBjCek1nsHzEq4gPDUHT3tj+HjUtNy59L/ac 7T0LZSZRkLU+tI5fRHxC1yaX/UglNGqZP8lSflLNFsFuvFhXFZdz X-Gm-Gg: ASbGnctZ3G0gaGSJ7YhIgmjroMjQuASvh0fJnoQaHYbJdrqogTB9/1yKYsvaG0BEhr4 k5lRZQ2MKdsZgRou5lI0KF2871y6GCIXhOA492IdHj31W92YdECtpXH7+tCMoFY6pR6DX6EBlRW +BSy5AJ3a/dPxk97S+kk6uLBGm8JcZBoeIbcZN/VBtvE/IBggAcMXfFybPBL3F2Yn0+HUPa/6MS lZ7qwma0qNsUDtTOsQ7j/iFvQ4T/xMk5RjI5WDM0lzsMie/2hMtWKFe25g5C3bLd/oyZA== X-Google-Smtp-Source: AGHT+IHD1ss3qRydthDvpeokvQg4m5BQ34B2qytobXwm48isVD+DXXbGb1elPCkQRWyazKieeHlUdg== X-Received: by 2002:a17:907:7741:b0:aa5:4982:abc8 with SMTP id a640c23a62f3a-aab77989e01mr162590366b.22.1734086211825; Fri, 13 Dec 2024 02:36:51 -0800 (PST) Received: from [192.168.178.20] ([2a02:2455:8256:2d00:9c39:c2d7:aedd:294d]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aa6883f65c3sm686785266b.157.2024.12.13.02.36.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Dec 2024 02:36:51 -0800 (PST) From: Karthik Nayak Date: Fri, 13 Dec 2024 11:36:46 +0100 Subject: [PATCH v2 1/8] refs: include committer info in `ref_update` struct Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241213-320-git-refs-migrate-reflogs-v2-1-f28312cdb6c0@gmail.com> References: <20241213-320-git-refs-migrate-reflogs-v2-0-f28312cdb6c0@gmail.com> In-Reply-To: <20241213-320-git-refs-migrate-reflogs-v2-0-f28312cdb6c0@gmail.com> To: git@vger.kernel.org Cc: Karthik Nayak , ps@pks.im, Christian Couder X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=5688; i=karthik.188@gmail.com; h=from:subject:message-id; bh=1zQAW1gf6s8EkfhWmsjwMbxZIi5DU5Z49Kos2YDhOaQ=; b=kA0DAAgBPtWfJI5GjH8ByyZiAGdcDkCghjNk1Co6n6ltY6seXiRIcOxWA2E33Ur7K1uln7j5I YkBswQAAQgAHRYhBFfOTH9jdXEPy2XGBj7VnySORox/BQJnXA5AAAoJED7VnySORox/uVgL/1Mv a+bVnzw2D2aKoybeRbLDRv+TDGGLd2R2uq1ps1kbs0HlZqbrYV+dkilnikv7NOU8xW6JVgvt+ML 2rqT158KkZ/cVJLyZldSuUehk7ZNrv5Yq+yteR05ghYyuVacguEWMvhIAlRAnhqfAFap6oGpr45 XiwW3XyM3vFtA3rFzOFmuXh5sSNl/WlakGSfkC3GqYKhLU3Xxijgu0w+BBff6CRqfz4l93nnbsF eucNXM3EgHm5TpIEmXdJGfPWy+AvUTohyJRCnrZa/HS/YcpTQpX3lZZWlGRp6ksLTuXg7QRQQfc Gf74vcJrA4R/aUwjnJYCHzqu/gW+8Lz7mrXtZCPP/kQQUoBmnXd0tkMbiA59YOKhr/1U9tD2MG/ FW+wIw+VOHLtoourvLui0mnwKEYsRrkBg86NjSYgtWW3iuS4t0Mslcfu4jUyuqy5VMIbpi6Y4ex kkOjGMd9q7ICY94BwyamFUaAHYNME7MfeqvQMcl6ctVbm7DrGAkqjefbJOoQ== X-Developer-Key: i=karthik.188@gmail.com; a=openpgp; fpr=57CE4C7F6375710FCB65C6063ED59F248E468C7F The reference backends obtain the committer information from `git_committer_info(0)` when adding a reflog. The upcoming patches introduce support for migrating reflogs between the reference backends. This requires an interface to creating reflogs, including custom committer information. Add a new field `committer_info` to the `ref_update` struct, which is then used by the reference backends. If there is no `committer_info` provided, the reference backends default to using `git_committer_info(0)`. The field itself cannot be set to `git_committer_info(0)` since the values are dynamic and must be obtained right when the reflog is being committed. Signed-off-by: Karthik Nayak --- refs.c | 1 + refs/files-backend.c | 24 ++++++++++++++---------- refs/refs-internal.h | 1 + refs/reftable-backend.c | 12 +++++++++++- 4 files changed, 27 insertions(+), 11 deletions(-) diff --git a/refs.c b/refs.c index 762f3e324d59c60cd4f05c2f257e54de8deb00e5..f003e51c6bf5229bfbce8ce61ffad7cdba0572e0 100644 --- a/refs.c +++ b/refs.c @@ -1151,6 +1151,7 @@ void ref_transaction_free(struct ref_transaction *transaction) for (i = 0; i < transaction->nr; i++) { free(transaction->updates[i]->msg); + free(transaction->updates[i]->committer_info); free((char *)transaction->updates[i]->new_target); free((char *)transaction->updates[i]->old_target); free(transaction->updates[i]); diff --git a/refs/files-backend.c b/refs/files-backend.c index 64f51f0da905a9a8a1ac4109c6b0a9a85a355db7..6078668c99ee254e794e3ba49689aa34e6022efd 100644 --- a/refs/files-backend.c +++ b/refs/files-backend.c @@ -1858,6 +1858,9 @@ static int log_ref_write_fd(int fd, const struct object_id *old_oid, struct strbuf sb = STRBUF_INIT; int ret = 0; + if (!committer) + committer = git_committer_info(0); + strbuf_addf(&sb, "%s %s %s", oid_to_hex(old_oid), oid_to_hex(new_oid), committer); if (msg && *msg) { strbuf_addch(&sb, '\t'); @@ -1871,8 +1874,10 @@ static int log_ref_write_fd(int fd, const struct object_id *old_oid, } static int files_log_ref_write(struct files_ref_store *refs, - const char *refname, const struct object_id *old_oid, - const struct object_id *new_oid, const char *msg, + const char *refname, + const struct object_id *old_oid, + const struct object_id *new_oid, + const char *committer_info, const char *msg, int flags, struct strbuf *err) { int logfd, result; @@ -1889,8 +1894,7 @@ static int files_log_ref_write(struct files_ref_store *refs, if (logfd < 0) return 0; - result = log_ref_write_fd(logfd, old_oid, new_oid, - git_committer_info(0), msg); + result = log_ref_write_fd(logfd, old_oid, new_oid, committer_info, msg); if (result) { struct strbuf sb = STRBUF_INIT; int save_errno = errno; @@ -1974,8 +1978,7 @@ static int commit_ref_update(struct files_ref_store *refs, files_assert_main_repository(refs, "commit_ref_update"); clear_loose_ref_cache(refs); - if (files_log_ref_write(refs, lock->ref_name, - &lock->old_oid, oid, + if (files_log_ref_write(refs, lock->ref_name, &lock->old_oid, oid, NULL, logmsg, flags, err)) { char *old_msg = strbuf_detach(err, NULL); strbuf_addf(err, "cannot update the ref '%s': %s", @@ -2007,9 +2010,9 @@ static int commit_ref_update(struct files_ref_store *refs, if (head_ref && (head_flag & REF_ISSYMREF) && !strcmp(head_ref, lock->ref_name)) { struct strbuf log_err = STRBUF_INIT; - if (files_log_ref_write(refs, "HEAD", - &lock->old_oid, oid, - logmsg, flags, &log_err)) { + if (files_log_ref_write(refs, "HEAD", &lock->old_oid, + oid, NULL, logmsg, flags, + &log_err)) { error("%s", log_err.buf); strbuf_release(&log_err); } @@ -2969,7 +2972,8 @@ static int parse_and_write_reflog(struct files_ref_store *refs, } if (files_log_ref_write(refs, lock->ref_name, &lock->old_oid, - &update->new_oid, update->msg, update->flags, err)) { + &update->new_oid, update->committer_info, + update->msg, update->flags, err)) { char *old_msg = strbuf_detach(err, NULL); strbuf_addf(err, "cannot update the ref '%s': %s", diff --git a/refs/refs-internal.h b/refs/refs-internal.h index 58aa56d1b27c85d606ed7c8c0d908e4b87d1066b..0fd95cdacd99e4a728c22f5286f6b3f0f360c110 100644 --- a/refs/refs-internal.h +++ b/refs/refs-internal.h @@ -113,6 +113,7 @@ struct ref_update { void *backend_data; unsigned int type; char *msg; + char *committer_info; /* * If this ref_update was split off of a symref update via diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c index 647ef9b05b1dc9a376ed054330b487f7595c5caa..e882602487c66261d586a94101bb1b4e9a2ed60e 100644 --- a/refs/reftable-backend.c +++ b/refs/reftable-backend.c @@ -1379,11 +1379,21 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data } if (create_reflog) { + struct ident_split c; + ALLOC_GROW(logs, logs_nr + 1, logs_alloc); log = &logs[logs_nr++]; memset(log, 0, sizeof(*log)); - fill_reftable_log_record(log, &committer_ident); + if (u->committer_info) { + if (split_ident_line(&c, u->committer_info, + strlen(u->committer_info))) + BUG("failed splitting committer info"); + } else { + c = committer_ident; + } + + fill_reftable_log_record(log, &c); log->update_index = ts; log->refname = xstrdup(u->refname); memcpy(log->value.update.new_hash, From patchwork Fri Dec 13 10:36:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karthik Nayak X-Patchwork-Id: 13906828 Received: from mail-ej1-f52.google.com (mail-ej1-f52.google.com [209.85.218.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8B2471BF311 for ; Fri, 13 Dec 2024 10:36:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.52 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734086216; cv=none; b=inWS5DejZ8/E7zWk2kZqbHkFVeI2GwsCRVhKmDRblQ+2XaEVC8ms3Hpf1wnfzQKeVgWsW1BDBXrt0Teb2MWqSJ4B2TH1DHKw5NpNVpaU/IoIg9mukyaylERFGg/dAjM+TPGeFGhWYRI5ShaFNGpwci3kX7mVhUzUHViZ5d2rvnk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734086216; c=relaxed/simple; bh=IB9h00v4zfs7q+FeZZk33xlgGFW2QmXAcKQGj56BkwY=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=aW22S4xZvRcUf1UZyuxJsIEO6AbaHuDYTsR02Mb1S2bIBxup6Vyc0kruy/MZaPG72u/cKz55+HdDzNsQGqdc3rtbkzQ7Mii9eJ2DwSgYtu+4fd51Z79LShXyCuaA6wDj0QCGDgkE7JRjZE7CyYRv38ik+3HEHwsgpCRz5IYlmBs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=f9fjLkZ1; arc=none smtp.client-ip=209.85.218.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="f9fjLkZ1" Received: by mail-ej1-f52.google.com with SMTP id a640c23a62f3a-aa6b4cc7270so203348466b.0 for ; Fri, 13 Dec 2024 02:36:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734086213; x=1734691013; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=lUvb5dIPp19cc7hdfTanzPMDqohb3VapJTcypa4CDlg=; b=f9fjLkZ1qZI79NMpRjZTrOVjjm2M9XG3WQIJ8VOdpRXsb7YOAf4bnnPWutK5dRlPxe EBdFnB6UlG5Jg91HvUqbljWEJV9JI1nYvlNnnjjne8nLrfigHlJ9jCiuXbGcAGS5+7Q3 RgFuw6cR3ZjBnRMwYu4vkYOwg/KfpeykduMnhHWbzsMuWw69JofTEehWqfjscsEGFW5M 09cQVw+fWMOxUHpmCemosgE9wKbCL1Bac64ZrNL8QsKauhskvomTxLorAJy4ilDJpmct wFTonyhBkNvGKFMf7UhWTllRKzpM28KVFys09q9gaO2RRwjcNFhSkJsMOHlUFyYAIogy m0Qw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734086213; x=1734691013; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=lUvb5dIPp19cc7hdfTanzPMDqohb3VapJTcypa4CDlg=; b=exEWFAoXlROAUYYtx2clVQIXUrSAySJUlSxjThFvC8Qb5c3Iu3XPGVIanldIZIajRv juAmNfk/OmVHMx1o4vSg0B0ww0mefMPMldqW0jWqTFDOYwYOebKEOJic5KZHtuPu63qW chlfGgvmezGnO4E07RedXD2K3gng0t7dc6CHIINEaMLh0n63MnHpMHO26E2RWp219Bae dTUcA3r6TN54BdgpkN39tBoWWnGuIW0X3IgBsqB1rFsrZrNzr49Lu729Jg7JdZb+j1te HY1z8azH5FMXkb7N+PTBe/eSii/xwMckxETNVrnL01CRVozZgqEVW+mNmZHWhRUyN3hC zzMg== X-Gm-Message-State: AOJu0YyEteftetr6F7bNN/1T3z8QsK2UBwiw1xuUko5MtdQSgbcve+eP 1OWaDXrBSYhHNAzqUnEF2FJdfX2mxjybJGVYvKchsbBoPlL2ZsRo X-Gm-Gg: ASbGncudCmVa7epH38pjVeVMBZO2afiAaaSqKu+AAoKrygxH/9QJmIeZtXq4aUdhJvG oS7uqxrYiWs0/qteUuxBOMcRiBeDqkbC7SVVPbIowPY+muqkgniMyQknewSrtQaY3vv5+tqBN58 wnTZIWj5U68RaMqkGD43nzwwpD8KPeUH/UNiRC33SzETlzaTxuXhnzBAszEvCPQTj5USOaqYhET 0cuQiyEyjzcoDbq87vAjmHwBC3b7J8u17T60jJqoz2hWLoSSa0EOyRlnfrLMgIeS01/7Q== X-Google-Smtp-Source: AGHT+IEN+dREHKqPS35kHu+8IjyhBAZWX+6Dy3uAujderjC6DYdhs8d5zz/2HdQ3Vdjnp1Xq33XJ/Q== X-Received: by 2002:a05:6402:4499:b0:5d4:55e:f99e with SMTP id 4fb4d7f45d1cf-5d63c3405bfmr4964903a12.18.1734086212576; Fri, 13 Dec 2024 02:36:52 -0800 (PST) Received: from [192.168.178.20] ([2a02:2455:8256:2d00:9c39:c2d7:aedd:294d]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aa6883f65c3sm686785266b.157.2024.12.13.02.36.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Dec 2024 02:36:52 -0800 (PST) From: Karthik Nayak Date: Fri, 13 Dec 2024 11:36:47 +0100 Subject: [PATCH v2 2/8] refs: add `index` field to `struct ref_udpate` Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241213-320-git-refs-migrate-reflogs-v2-2-f28312cdb6c0@gmail.com> References: <20241213-320-git-refs-migrate-reflogs-v2-0-f28312cdb6c0@gmail.com> In-Reply-To: <20241213-320-git-refs-migrate-reflogs-v2-0-f28312cdb6c0@gmail.com> To: git@vger.kernel.org Cc: Karthik Nayak , ps@pks.im, Christian Couder X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=2464; i=karthik.188@gmail.com; h=from:subject:message-id; bh=IB9h00v4zfs7q+FeZZk33xlgGFW2QmXAcKQGj56BkwY=; b=owEB7QES/pANAwAIAT7VnySORox/AcsmYgBnXA5Aa7BSAc/6jK6ehBVCRTe5kV2//CGuMkz6/ azqAygE6WyJAbMEAAEIAB0WIQRXzkx/Y3VxD8tlxgY+1Z8kjkaMfwUCZ1wOQAAKCRA+1Z8kjkaM fxbBC/43I91fM4b/7lZG7of6mQBZJ26I06P/12OrtGJIdWjywc8a3u7/GAGco+iUTJZh5PQ7OWa 04sS880sR4LhdLTUYf03VyCklKegm/ULYmAAtYlpb1SdNoa2ECnc4LgAAnYmUlQPPhdmpqgAqpT 5+48LIgoteGvwOfi4spuV2HRyJjB3IQMT894D9zggA6VoVJhiw+pGUXL+0SvcTvlbP8ihcm8lKK 8i0Zy5MkvKPHGWvylAlmV9T8RDL9ehFSOBEWDmt9OoHJsOZAr15juvbOZAFH9HcmSwBN3YAD8sY yWM7LRHkiu5jQJ/8QeMsMyhoAr7hkNt8PArVln5fiCZRyCi08sUsEC8uJ0kTBnbGhKbToR8IQcp 7wcE04n/BCR+UWUdClT5SOyZfM31/e8t5126qJntUiIsCICva+dN0f7ErKkDQI+iolyHLykSZNR iFQYAizgYtRHVfuAJuKCVqGehDsoD4j0TlVIWRCQzQ0p+DhpX3m/+EWSwe+R1IeKKXGT8= X-Developer-Key: i=karthik.188@gmail.com; a=openpgp; fpr=57CE4C7F6375710FCB65C6063ED59F248E468C7F The reftable backend, sorts its updates by refname before applying them, this ensures that the references are stored sorted. When migrating reflogs from one backend to another, the order of the reflogs must be maintained. Add a new `index` field to the `ref_update` struct to facilitate this. This field is used in the reftable backend's sort comparison function `transaction_update_cmp`, to ensure that indexed fields maintain their order. Signed-off-by: Karthik Nayak --- refs/refs-internal.h | 7 +++++++ refs/reftable-backend.c | 13 +++++++++++-- 2 files changed, 18 insertions(+), 2 deletions(-) diff --git a/refs/refs-internal.h b/refs/refs-internal.h index 0fd95cdacd99e4a728c22f5286f6b3f0f360c110..f5c733d099f0c6f1076a25f4f77d9d5eb345ec87 100644 --- a/refs/refs-internal.h +++ b/refs/refs-internal.h @@ -115,6 +115,13 @@ struct ref_update { char *msg; char *committer_info; + /* + * The index overrides the default sort algorithm. This is needed + * when migrating reflogs and we want to ensure we carry over the + * same order. + */ + unsigned int index; + /* * If this ref_update was split off of a symref update via * split_symref_update(), then this member points at that diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c index e882602487c66261d586a94101bb1b4e9a2ed60e..c008f20be719fec3af6a8f81c821cb9c263764d7 100644 --- a/refs/reftable-backend.c +++ b/refs/reftable-backend.c @@ -1279,8 +1279,17 @@ static int reftable_be_transaction_abort(struct ref_store *ref_store UNUSED, static int transaction_update_cmp(const void *a, const void *b) { - return strcmp(((struct reftable_transaction_update *)a)->update->refname, - ((struct reftable_transaction_update *)b)->update->refname); + struct reftable_transaction_update *update_a = (struct reftable_transaction_update *)a; + struct reftable_transaction_update *update_b = (struct reftable_transaction_update *)b; + + /* + * If there is an index set, it should take preference (default is 0). + * This ensures that updates with indexes are sorted amongst themselves. + */ + if (update_a->update->index || update_b->update->index) + return update_a->update->index - update_b->update->index; + + return strcmp(update_a->update->refname, update_b->update->refname); } static int write_transaction_table(struct reftable_writer *writer, void *cb_data) From patchwork Fri Dec 13 10:36:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karthik Nayak X-Patchwork-Id: 13906830 Received: from mail-ed1-f41.google.com (mail-ed1-f41.google.com [209.85.208.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3DECF1BFE03 for ; Fri, 13 Dec 2024 10:36:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.41 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734086217; cv=none; b=Gsq8qo/m5BnLO8GAkrDl3wEcbI6b/7DDWFtDHCqhuMt+bBvgsXBFTrfiejBuqbz2Wp/ave4dyalPZmmBQsprACAnco6FJXZY5dGhnEbGju2tGTGeFvPaS/xluaDw/wnooHPU3CnWVWJr3fHfotZXfbm8c34Tb90QPWQ8JGv4WZ0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734086217; c=relaxed/simple; bh=ERhO5bUagvCcVWd1ajCqhTTv/qHXvpyDSomL3moOTTM=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=MHB9eLmwEOeETdqwJ5IBO848cAr6FHVAu1c/5s3LSC2oEw/BSmochdvi9u8BMJGry7W6L1Ha2PYEm/2dAsTS45rBXOTJFZvm6mMWuJLIc1LjfCu4MnuPUxgbOoudtVlKkxeWa4GnqvVELRZRLOdHoaWLBBae5/7oUsh2qlJI9eM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=g6/LZF24; arc=none smtp.client-ip=209.85.208.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="g6/LZF24" Received: by mail-ed1-f41.google.com with SMTP id 4fb4d7f45d1cf-5d3f57582a2so5218822a12.1 for ; Fri, 13 Dec 2024 02:36:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734086213; x=1734691013; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=5gNfOmPXL16KqA+CPu0YSmndFRynLUkndelIwKCAq4k=; b=g6/LZF24eZZOfJrx7j7rlSwID3hngJJZnjPoU7YemcQQLR1pABEmSkm/QQ/shmQYEy jopQSPy6sprioPSj7QXgVET6dJZfQ9H7bmXWzl5Wm51f4yTHfP7o1b9Ji/rTgngaWNkW n8pBM2cokOqMPvjrmSmFMUeSKHbdbm7+84wM3UgF9j3smHg6Hmcmi+gHR3e0YExz/oRO bngZ9Ds1DRoCSNZEBL32QPTZAas6aJul8PZ5HS+5VDJWqwBHU8dT69QNvzP+IP5WJ10k i8717yEZjlcboP9/YXOdsByA2ylvc/XuH4NYNpRMzr8QkUGWi2dfNLvEhket4GLH7c5A mxkA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734086213; x=1734691013; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=5gNfOmPXL16KqA+CPu0YSmndFRynLUkndelIwKCAq4k=; b=reERBam5+VfFShSPEh7etaQJMlUuh44cC7C32Xz9ESdDOjLEOrNe48dggd1SVYUUVK hpR+aCVOtYiK2y4Fq6ma60Tp6e+v16vkOfP0QazK6LmaEXpV4dXlOBBHnCe1tkqPhomw GEno2zdgktowDpuHnS9+s+2OupcfchbZhkWb9RzI+/yojvALAoOfB2tgHsmKjq/zfcxy JbrwD6Zpa3UmxWY/bPEjcVuTeuGRlghveGA+fJh3spaLLGgUimKOCxdEDTanFN6YWq7+ C8259IwG5VhSdbje3ywYjP3RKwmLgeYnlr0CTDypc86vNPYYQ4dQetesfbv/IjsMd5Lk AUiQ== X-Gm-Message-State: AOJu0YwhLORCWZrT8kayhHxyyM72zgjOP6WIVU4tiq4hRkMR5qaMDaiF ozuAyPKZHDiOncI1YpfruZ8YkTrglV2oJctDTqK33pDMU8wRmiXU X-Gm-Gg: ASbGnctzefEMirD0ygKo8hZpaXWSzTpIveKsPag657J8faTaSeI59r9OyDdzbTuISuU gh2GNzbqAXyKKfXt9m6qdAdKiQHVDVKTJ/TXHPpQK3WqLnTzXHUDmvw4GRTstTXrGutp8/RRPIM VLkote1AfExfy1uArZy67uJPaM+0yi6pvSy4LM2TrieSwzlzvtSe9JNMKn0nrRyNJas8aCMtZ/n jbuH8owtx14aMgW6xHQsXwU0CjD8VRH0voh+5etulW/EDsICRm3kvIWk2c8yc5lrG74Gg== X-Google-Smtp-Source: AGHT+IFqGOv8SgIjyhXFKeUndVDGplwQvUWhqgL5D9EIrZETyhjsw2DFSBhZRqZu3bkh8u9umWcVlg== X-Received: by 2002:a17:907:7e92:b0:aa6:a9fe:46de with SMTP id a640c23a62f3a-aab7b777564mr189714966b.19.1734086213335; Fri, 13 Dec 2024 02:36:53 -0800 (PST) Received: from [192.168.178.20] ([2a02:2455:8256:2d00:9c39:c2d7:aedd:294d]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aa6883f65c3sm686785266b.157.2024.12.13.02.36.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Dec 2024 02:36:53 -0800 (PST) From: Karthik Nayak Date: Fri, 13 Dec 2024 11:36:48 +0100 Subject: [PATCH v2 3/8] refs/files: add count field to ref_lock Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241213-320-git-refs-migrate-reflogs-v2-3-f28312cdb6c0@gmail.com> References: <20241213-320-git-refs-migrate-reflogs-v2-0-f28312cdb6c0@gmail.com> In-Reply-To: <20241213-320-git-refs-migrate-reflogs-v2-0-f28312cdb6c0@gmail.com> To: git@vger.kernel.org Cc: Karthik Nayak , ps@pks.im, Christian Couder X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=5670; i=karthik.188@gmail.com; h=from:subject:message-id; bh=ERhO5bUagvCcVWd1ajCqhTTv/qHXvpyDSomL3moOTTM=; b=owEB7QES/pANAwAIAT7VnySORox/AcsmYgBnXA5AtUAbbN803ICOyPKDnHQBP9z8G5c0kFS5h L0aDzeDtuKJAbMEAAEIAB0WIQRXzkx/Y3VxD8tlxgY+1Z8kjkaMfwUCZ1wOQAAKCRA+1Z8kjkaM f4PPC/4oxeHHeFFzbQdmjvLnIzINPSJXSL4PfzIvZq6lZbZVw1mHwV+54w98e0DASm08wFNkBOM DEspVSEANdUVwiJCtX0QX62DLnawkgsVp6aiYoK7CDjG63opmsDtkiINNWuWpY7NwNNt7M0mREs eUYefblq+/oRss3hFpdkU6msfdVxJRXMGrz4uQDTURtwn10yd8ZcQn8tc5E/5YnQ4/ivuiBy7Xe akkXHFfskr/c3XuAB3ZW9QmBpJlnPlNUeMsdDtc7TNnOLLDrBrVfmP6Zgl/p3eCJS67DruEDzJ/ FMLfudwCMYMz+JoHQXQhxtnaXAUJuxW1GYjkghjqklxRhNAKS7s8UYBLB+e6VpdiLZns3vdbCUN hfy9X4XZTC4aNFHUDlrb7vdLLmJG8GB+5au9UYMVWBiy3/F9tvWziaMB0deh0l8CdnB0S+cFOld knn2ft/XGsGjpslCtpRK5gudwxgQF05yF64NA3WQffGX6a9EOHsAoOJENRSDmzNURlZ10= X-Developer-Key: i=karthik.188@gmail.com; a=openpgp; fpr=57CE4C7F6375710FCB65C6063ED59F248E468C7F When refs are updated in the files-backend, a lock is obtained for the corresponding file path. This is the case even for reflogs, i.e. a lock is obtained on the reference path instead of the reflog path. This works, since generally, reflogs are updated alongside the ref. The upcoming patches will add support for reflog updates in ref transaction. This means, in a particular transaction we want to have ref updates and reflog updates. For a given ref in a given transaction there can be at most one update. But we can theoretically have multiple reflog updates for a given ref in a given transaction. A great example of this would be when migrating reflogs from one backend to another. There we would batch all the reflog updates for a given reference in a single transaction. The current flow does not support this, because currently refs & reflogs are treated as a single entity and capture the lock together. To separate this, add a count field to ref_lock. With this, multiple updates can hold onto a single ref_lock and the lock will only be released when all of them release the lock. This patch only adds the `count` field to `ref_lock` and adds the logic to increment and decrement the lock. In a follow up commit, we'll separate the reflog update logic from ref updates and utilize this functionality. Signed-off-by: Karthik Nayak --- refs/files-backend.c | 58 +++++++++++++++++++++++++++++++++++----------------- 1 file changed, 39 insertions(+), 19 deletions(-) diff --git a/refs/files-backend.c b/refs/files-backend.c index 6078668c99ee254e794e3ba49689aa34e6022efd..02cb4907d8659e87a227fed4f60a5f6606be8764 100644 --- a/refs/files-backend.c +++ b/refs/files-backend.c @@ -71,6 +71,7 @@ struct ref_lock { char *ref_name; struct lock_file lk; struct object_id old_oid; + unsigned int count; /* track users of the lock (ref update + reflog updates) */ }; struct files_ref_store { @@ -638,9 +639,12 @@ int parse_loose_ref_contents(const struct git_hash_algo *algop, static void unlock_ref(struct ref_lock *lock) { - rollback_lock_file(&lock->lk); - free(lock->ref_name); - free(lock); + lock->count--; + if (!lock->count) { + rollback_lock_file(&lock->lk); + free(lock->ref_name); + free(lock); + } } /* @@ -696,6 +700,7 @@ static int lock_raw_ref(struct files_ref_store *refs, *lock_p = CALLOC_ARRAY(lock, 1); lock->ref_name = xstrdup(refname); + lock->count = 1; files_ref_path(refs, &ref_file, refname); retry: @@ -1169,6 +1174,7 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs, goto error_return; lock->ref_name = xstrdup(refname); + lock->count = 1; if (raceproof_create_file(ref_file.buf, create_reflock, &lock->lk)) { unable_to_lock_message(ref_file.buf, errno, err); @@ -2535,6 +2541,12 @@ static int check_old_oid(struct ref_update *update, struct object_id *oid, return -1; } +struct files_transaction_backend_data { + struct ref_transaction *packed_transaction; + int packed_refs_locked; + struct strmap ref_locks; +}; + /* * Prepare for carrying out update: * - Lock the reference referred to by update. @@ -2557,11 +2569,14 @@ static int lock_ref_for_update(struct files_ref_store *refs, { struct strbuf referent = STRBUF_INIT; int mustexist = ref_update_expects_existing_old_ref(update); + struct files_transaction_backend_data *backend_data; int ret = 0; struct ref_lock *lock; files_assert_main_repository(refs, "lock_ref_for_update"); + backend_data = transaction->backend_data; + if ((update->flags & REF_HAVE_NEW) && ref_update_has_null_new_value(update)) update->flags |= REF_DELETING; @@ -2572,18 +2587,25 @@ static int lock_ref_for_update(struct files_ref_store *refs, goto out; } - ret = lock_raw_ref(refs, update->refname, mustexist, - affected_refnames, - &lock, &referent, - &update->type, err); - if (ret) { - char *reason; + lock = strmap_get(&backend_data->ref_locks, update->refname); + if (lock) { + lock->count++; + } else { + ret = lock_raw_ref(refs, update->refname, mustexist, + affected_refnames, + &lock, &referent, + &update->type, err); + if (ret) { + char *reason; + + reason = strbuf_detach(err, NULL); + strbuf_addf(err, "cannot lock ref '%s': %s", + ref_update_original_update_refname(update), reason); + free(reason); + goto out; + } - reason = strbuf_detach(err, NULL); - strbuf_addf(err, "cannot lock ref '%s': %s", - ref_update_original_update_refname(update), reason); - free(reason); - goto out; + strmap_put(&backend_data->ref_locks, update->refname, lock); } update->backend_data = lock; @@ -2730,11 +2752,6 @@ static int lock_ref_for_update(struct files_ref_store *refs, return ret; } -struct files_transaction_backend_data { - struct ref_transaction *packed_transaction; - int packed_refs_locked; -}; - /* * Unlock any references in `transaction` that are still locked, and * mark the transaction closed. @@ -2767,6 +2784,8 @@ static void files_transaction_cleanup(struct files_ref_store *refs, if (backend_data->packed_refs_locked) packed_refs_unlock(refs->packed_ref_store); + strmap_clear(&backend_data->ref_locks, 0); + free(backend_data); } @@ -2796,6 +2815,7 @@ static int files_transaction_prepare(struct ref_store *ref_store, goto cleanup; CALLOC_ARRAY(backend_data, 1); + strmap_init(&backend_data->ref_locks); transaction->backend_data = backend_data; /* From patchwork Fri Dec 13 10:36:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karthik Nayak X-Patchwork-Id: 13906831 Received: from mail-ej1-f50.google.com (mail-ej1-f50.google.com [209.85.218.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F26971C07E2 for ; Fri, 13 Dec 2024 10:36:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734086217; cv=none; b=i89bLtvP70ANKPvxCzPIoxWo2Bwchx2WQC7d5WiCC9XdWWDEIjnL2j3NVms+IibkJBX6mFLiY5jfM46+3v1B1lt2xAFYiMVB0ssUD3vrjpPnTchOq89fONvAt21awz2OokkD7j9FCuVx5uA6ZaHa+FQ+EKLZrC4qJVvoLBWi43o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734086217; c=relaxed/simple; bh=NoEckIGxQa5jQrZ0dQ7BHqCovFckF/qXPjbGFqkkuA0=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=dt9TBSy0VKcYXjWWCqxBHVLCKx/ndKJyIbYc3fIC3OXmGrGFELEC+sRU2bvZUpNrhYO/6gHbBJs4R+cQF6xcpPFMUp+Cz3teEUsiKNPVJlCnkLd5Y/SVWDY2AdkJQHv5yJaYw0PJHSgHB+LPlVP3zCeEC5B9RIiqQpdnEFPymtk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=T2gKCsyE; arc=none smtp.client-ip=209.85.218.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="T2gKCsyE" Received: by mail-ej1-f50.google.com with SMTP id a640c23a62f3a-aa6997f33e4so237149266b.3 for ; Fri, 13 Dec 2024 02:36:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734086214; x=1734691014; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=Io5XE6RXiBUov+SaLOGtphgg3UciL24hpxxR053Wce4=; b=T2gKCsyEJSPq7d6cNdl/3KluV6lws3yBx0zyo+EO2U0BbAySZpK/i8gzRUuB76PAo+ 79zpsvLnWDDP2/35f6fDm4Ip4M222Usp36oTVOnEob00CBcWEOG/pUZ8GMKwBMxHUyGb 3PVmeHc6yORfO2+kTL6wtEpsB1dahtI9QaUxLMPT8W6E2h0hFg5JQx4xx+1yFQ78yPxw +I5kctSGgqgZ1FXqc+RoyVNdWFvdjuJueS5I02k2356DPVnTHIolhIempS/ZCTvnJYDh eGtKRhif2OeHiwaDnDnA7PcRo4UsH67p4n01G06khxnvDO4DdUtOLNJKjjM4QfbOPIwb uUdw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734086214; x=1734691014; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Io5XE6RXiBUov+SaLOGtphgg3UciL24hpxxR053Wce4=; b=v4uOvOBuAKeRYXelExMILxAmWuSW8HiJrxac4/UAN1ZN+Ka38qt1nxYlLhoNvF0NNt yWis50QSxA6QM5zrbvjZoqnU09D24Aj4BD/HnfxFbm4vNkYRj3utR/mKWp3b7L3H9EI6 td6X7Mn9JJc3tEFLskkLWj1+vn1WrovG0EBkc66Y4xXIM9iiMod4Pfd8EkKqytEnusKl lLE7x/PM+KUXn5G7uLjoQEXJX3z0RUhDPejFDeRku1TQ2/yenpvvUFf7Xurr0Dy0poFT EWD1wR3kN/mrwT+Ayatdxn/hPo0hcGYRpK5L31I2MoILtD9YAgeESNsDyxV65DiokKej uJZw== X-Gm-Message-State: AOJu0Ywbu8SLzH2xl/eCTkt7dsu01ug5CZ/rFZYpH0tvni1/dbIS8Inh p4KMwElJ6NLIyzUepBr6lLJiVQqdmVWXWqB/hzP0zAYkmtlKb3JF X-Gm-Gg: ASbGnct7+kY4yZane+04zq85YFqr80E3JVgLMZNbeWbem323ry1Pb202o+QfQNuP5ij jYMp8hfotpEFmMUGN+IJaW2VJhnTRO/NVnTYOfRnB6GXeLC53uvYR6JkqInD4tzwqfUkS10sRe0 GwVwIR5KhT3a6q1nyNm8RHkoD0de0wIKkewQRidWbYDJYeYXIY4Vz97vpcFe8FL1rAJ6nPqxuKm nslEicGp35AZloBkfcOsiUr0uQEux0NgDnrGjloo4i9IXUMQjKDVg/hJVL1WGGMWFo4yA== X-Google-Smtp-Source: AGHT+IHQjSYHlm9jzVcydVJ3Nx2wx5TdvVFHVXT7YVOSVOsxBDLp08UlTY38R0nkF4vthvObGOxPxA== X-Received: by 2002:a17:907:7813:b0:aa6:66bc:8788 with SMTP id a640c23a62f3a-aab77e8a87emr192418266b.45.1734086214004; Fri, 13 Dec 2024 02:36:54 -0800 (PST) Received: from [192.168.178.20] ([2a02:2455:8256:2d00:9c39:c2d7:aedd:294d]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aa6883f65c3sm686785266b.157.2024.12.13.02.36.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Dec 2024 02:36:53 -0800 (PST) From: Karthik Nayak Date: Fri, 13 Dec 2024 11:36:49 +0100 Subject: [PATCH v2 4/8] refs: extract out refname verification in transactions Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241213-320-git-refs-migrate-reflogs-v2-4-f28312cdb6c0@gmail.com> References: <20241213-320-git-refs-migrate-reflogs-v2-0-f28312cdb6c0@gmail.com> In-Reply-To: <20241213-320-git-refs-migrate-reflogs-v2-0-f28312cdb6c0@gmail.com> To: git@vger.kernel.org Cc: Karthik Nayak , ps@pks.im, Christian Couder X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=2366; i=karthik.188@gmail.com; h=from:subject:message-id; bh=NoEckIGxQa5jQrZ0dQ7BHqCovFckF/qXPjbGFqkkuA0=; b=owEB7QES/pANAwAIAT7VnySORox/AcsmYgBnXA5Ax1IR50x9djyLZSMGTFu7b5A38pHgA+dvB 2nTw5gUl3WJAbMEAAEIAB0WIQRXzkx/Y3VxD8tlxgY+1Z8kjkaMfwUCZ1wOQAAKCRA+1Z8kjkaM f34EC/4h/JAQ4GJ7M3NRjEEjkaijnGAGlpAKu8XOcj4ZPnKdQeKL3PizlMHnSvC7mkY7t6Gx1F9 0NMDg/FO2F6QLLHzOmb3PphEZz3pA9C75mRhZOGPFWrvfmD4rt2nkisFMLhd6x06tLU6Gy80nqe qPadXg3BNEofUb1EFo+nJStRSyPHk8kPUEF+wL3bp3+BY+aSSEgb4NmR9xv0jfE6Q1I3GJ56gqu pyyp090KdqKYOPPPuUe0v5VP7D9NtyKT9+JnVXbUQbE8BzQ3GzZeJi/vCVga6SLB22NAT3Dp0t6 p7f+gVYR8vt+QM2o4k2onO2U2+oTyXuVN5aqZ6160faUajnxpUIOAleTwJPQCXAym3AJffqO8QL oT3t11UH1y0+yCwjqO0SQj6tf0sPF0CjK79bB61cesfae4Vr8uNJVPAaF8yvcQrfr+fZLCYN5Wx 3sDH6sjEUAC8olMyXQCA7XYnhjwxwuYnHYwpwkXV9g/nP+GkA4kAYdjwWEoFuDWjCouBo= X-Developer-Key: i=karthik.188@gmail.com; a=openpgp; fpr=57CE4C7F6375710FCB65C6063ED59F248E468C7F Unless the `REF_SKIP_REFNAME_VERIFICATION` flag is set for an update, the refname of the update is verified for: - Ensuring it is not a pseudoref. - Checking the refname format. These checks will also be needed in a following commit where the function to add reflog updates to the transaction is introduced. Extract the code out into a new static function. Signed-off-by: Karthik Nayak --- refs.c | 37 +++++++++++++++++++++++-------------- 1 file changed, 23 insertions(+), 14 deletions(-) diff --git a/refs.c b/refs.c index f003e51c6bf5229bfbce8ce61ffad7cdba0572e0..9c9f4940c60d3cdd34ce8f1e668d17b9da3cd801 100644 --- a/refs.c +++ b/refs.c @@ -1196,6 +1196,28 @@ struct ref_update *ref_transaction_add_update( return update; } +static int transaction_refname_valid(const char *refname, + const struct object_id *new_oid, + unsigned int flags, struct strbuf *err) +{ + if (flags & REF_SKIP_REFNAME_VERIFICATION) + return 1; + + if (is_pseudo_ref(refname)) { + strbuf_addf(err, _("refusing to update pseudoref '%s'"), + refname); + return 0; + } else if ((new_oid && !is_null_oid(new_oid)) ? + check_refname_format(refname, REFNAME_ALLOW_ONELEVEL) : + !refname_is_safe(refname)) { + strbuf_addf(err, _("refusing to update ref with bad name '%s'"), + refname); + return 0; + } + + return 1; +} + int ref_transaction_update(struct ref_transaction *transaction, const char *refname, const struct object_id *new_oid, @@ -1213,21 +1235,8 @@ int ref_transaction_update(struct ref_transaction *transaction, return -1; } - if (!(flags & REF_SKIP_REFNAME_VERIFICATION) && - ((new_oid && !is_null_oid(new_oid)) ? - check_refname_format(refname, REFNAME_ALLOW_ONELEVEL) : - !refname_is_safe(refname))) { - strbuf_addf(err, _("refusing to update ref with bad name '%s'"), - refname); + if (!transaction_refname_valid(refname, new_oid, flags, err)) return -1; - } - - if (!(flags & REF_SKIP_REFNAME_VERIFICATION) && - is_pseudo_ref(refname)) { - strbuf_addf(err, _("refusing to update pseudoref '%s'"), - refname); - return -1; - } if (flags & ~REF_TRANSACTION_UPDATE_ALLOWED_FLAGS) BUG("illegal flags 0x%x passed to ref_transaction_update()", flags); From patchwork Fri Dec 13 10:36:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karthik Nayak X-Patchwork-Id: 13906832 Received: from mail-ej1-f48.google.com (mail-ej1-f48.google.com [209.85.218.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9B3441C1F03 for ; Fri, 13 Dec 2024 10:36:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.48 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734086218; cv=none; b=o7JoL5gM2KQElb4NFAaAp+GkUy3hzX0jOdaJTmbdxs3jj8kCLdVdqnPS9TcQuGWNx40H40cL/kZ0rGSHR3q1p3r72D/g6vQ56mzstqkT1HtlpZySeuGFEMQneClPs5/UBTsZVrbYN9wtXKULxOesvBKXnXIEtdIZY6Jby3wEVSY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734086218; c=relaxed/simple; bh=OqbDd3jsfWs8s5oNdnPUTcLB6APdMSxltVwJQdqRtRM=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=jquQQo7rHyy1nJxKy1aIm5qiy1VPdhLjMa6caosso7GKXKvyBWl3jtLHxVNcCffZ21TWmcVOesefuksjk6j5Bg5QAx18wgjItOUdfJMBMpI9rrSXSHJ2BEere7+0aOpEzotYWDGqyigQFudW6tt5AbsT0myoJL5H+Ojul2oksm4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=CAxjsp9T; arc=none smtp.client-ip=209.85.218.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="CAxjsp9T" Received: by mail-ej1-f48.google.com with SMTP id a640c23a62f3a-aa692211331so311031666b.1 for ; Fri, 13 Dec 2024 02:36:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734086215; x=1734691015; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=HvQRFT0vg1nsS3aygyDwNZGzsFcPMBGk2RfdqS0CpN0=; b=CAxjsp9TrOm3FeIe1L5wQy64mC3WyQCU5/3Kiw8/xHLkdT1F1RclcSsoe0mI7Rloh4 94sJfxHVZSgo+1fiqhK7DzCDLHTkBhrV1feDfmOKF5Cxc7CzbeB87kPGICr+un0nijYU RC8mQ7Ms1Iu6sJEd5/C4eWEhODRQxQqlNQ1bvqXzQiAlHv5MM6381Wo/ArvkDo1tTaUu ykDfQ10MrVDtweGZbTZXj0/7fyob3DDmZN7+FneysqTLtUBQGkYXd0yNpk6a/GlLRZsI VFmsmBiZw8FfXvZOJZfCrIxYkC/j+CQUz2U3biicLW67mUyisvWn638Ytfb2wIphtRLO 6Xxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734086215; x=1734691015; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HvQRFT0vg1nsS3aygyDwNZGzsFcPMBGk2RfdqS0CpN0=; b=qJtcIkFTwvuUEItucSxlgiZ36wLpSp+YV5SDqerIgje7JfUaisKZ+est668leorufw ok13vNt17keAjXmL4j1PgQbTX2GnAtbpGAORjiej3GpfF+H0SZOnz5iFWIh1rEF32fhM P12cUwu2nvXCMABCimkIDwI2w5mUjYbTh+NYe9fD6dW5Cfr0S9XCYsQ6bIzM6JJ5EVus 5Br1PS2bl6eYDN1PB/PEtHRQ5jccxeqG3ZBUymwIRj8tKP7dmQR2xq86ncJV74UWT7j2 ft7jbCO48krwD+AMKJ1hkk5sYF0cAapYfv7ksG24eTsXM5PKDDci5pNyA/fBpggeheKg 9frQ== X-Gm-Message-State: AOJu0YzHRk6V3bce8UZaJTWE5GuMGBex8ohW2JGzLYximDcVyKjbJADf GH6Set30fMhKLoKkR+MozWJ0wOmwaxZneG2Oqxs77AHn84Pqo5Yj X-Gm-Gg: ASbGncvuM2RSmSjexnkSzgyxTEBQW2VAneHxd2w5FTfoRb/ApfiDIRQF6YUE+SA75Hz YNcYB43bKaMtU7HacplljSK7b+q2kJHjbR4QNWkPkfUsXc2XsxbEf/8h88UU2L9oxAB6md3PPw9 e6YqLQlgZwQV5A6H+g1dmXDcbh9DSgL9UjFqYvK3i5bGxEVnT2vmHGBikCa2t6GKMV3IP26qh+t q8Hp2CRLeIe0aCzgy1OQ7kBlzz8QgQcATYvYbj9VTfycqQKjdNYE35yzmsMvXd3hDTCqw== X-Google-Smtp-Source: AGHT+IGo8KjwbWZK08sx8dLQUpPpCbCwl4+FehMYeIR43yPAEyhAyeH4otJ0wUXOp09sWhYJQl6AXA== X-Received: by 2002:a17:907:1c1a:b0:aa6:9540:5714 with SMTP id a640c23a62f3a-aa6c427589emr509985966b.25.1734086214739; Fri, 13 Dec 2024 02:36:54 -0800 (PST) Received: from [192.168.178.20] ([2a02:2455:8256:2d00:9c39:c2d7:aedd:294d]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aa6883f65c3sm686785266b.157.2024.12.13.02.36.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Dec 2024 02:36:54 -0800 (PST) From: Karthik Nayak Date: Fri, 13 Dec 2024 11:36:50 +0100 Subject: [PATCH v2 5/8] refs: add `committer_info` to `ref_transaction_add_update()` Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241213-320-git-refs-migrate-reflogs-v2-5-f28312cdb6c0@gmail.com> References: <20241213-320-git-refs-migrate-reflogs-v2-0-f28312cdb6c0@gmail.com> In-Reply-To: <20241213-320-git-refs-migrate-reflogs-v2-0-f28312cdb6c0@gmail.com> To: git@vger.kernel.org Cc: Karthik Nayak , ps@pks.im, Christian Couder X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=5509; i=karthik.188@gmail.com; h=from:subject:message-id; bh=OqbDd3jsfWs8s5oNdnPUTcLB6APdMSxltVwJQdqRtRM=; b=owEB7QES/pANAwAIAT7VnySORox/AcsmYgBnXA5AqjrAOASHoZn99LJYQSLKci2tz2Anp4Mst J5o2LYXEbGJAbMEAAEIAB0WIQRXzkx/Y3VxD8tlxgY+1Z8kjkaMfwUCZ1wOQAAKCRA+1Z8kjkaM f9kXC/9Z6hhcAqe3xrFbQwjIGe3zUI9VgYvcuNT5xNXACR+UBwNYdlYDG+KaVT9/j8vv9FadUZn lRAYSgel5Q/+HfOq/e3ga6YMnVoCvHZCWgt1OA1HI4gfCCn9z1fGsbwzs1YtwkzfixnYR0e0NlV VtrF1kCVOj7Ih+xEABw23RZm8JDdySOtsClciFJ7/vT4PO+GYuO7H6BCz70YJfdhn1bnzZEotTE t5kO5fngvrSKPYA1pDojjy0xmfpWOKo1jh2C0bVyKe084y2ZShsOI5W7foqz1moYFlNfsa60EKo li8yqYeMO2MbC3+emLoNjvajzWbEXXQW7Ebm39demxjAPCFA4+/9+ixTMlKek1eG1FSLESdjtH4 w+RzJZRzu7CasITIGud2OZTYWKo4xS8H4S54Ikz5xCOoQbu+q+ZyWHfH6XonfheLV1nW+LIMgzv oBLQQhzF8g7nZ+/fU/utrnQG4NLFENefzvYLQHA6DOoiGCHsvri+nKoAwuslLPGIretbQ= X-Developer-Key: i=karthik.188@gmail.com; a=openpgp; fpr=57CE4C7F6375710FCB65C6063ED59F248E468C7F The `ref_transaction_add_update()` creates the `ref_update` struct. To facilitate addition of reflogs in the next commit, the function needs to accommodate setting the `committer_info` field in the struct. So modify the function to also take `committer_info` as an argument and set it accordingly. Signed-off-by: Karthik Nayak --- refs.c | 9 +++++++-- refs/files-backend.c | 14 ++++++++------ refs/refs-internal.h | 1 + refs/reftable-backend.c | 6 ++++-- 4 files changed, 20 insertions(+), 10 deletions(-) diff --git a/refs.c b/refs.c index 9c9f4940c60d3cdd34ce8f1e668d17b9da3cd801..428ca256f3e5860554e9a7fa42a8368bb2689b31 100644 --- a/refs.c +++ b/refs.c @@ -1166,6 +1166,7 @@ struct ref_update *ref_transaction_add_update( const struct object_id *new_oid, const struct object_id *old_oid, const char *new_target, const char *old_target, + const char *committer_info, const char *msg) { struct ref_update *update; @@ -1190,8 +1191,12 @@ struct ref_update *ref_transaction_add_update( oidcpy(&update->new_oid, new_oid); if ((flags & REF_HAVE_OLD) && old_oid) oidcpy(&update->old_oid, old_oid); - if (!(flags & REF_SKIP_CREATE_REFLOG)) + if (!(flags & REF_SKIP_CREATE_REFLOG)) { + if (committer_info) + update->committer_info = xstrdup(committer_info); + update->msg = normalize_reflog_message(msg); + } return update; } @@ -1253,7 +1258,7 @@ int ref_transaction_update(struct ref_transaction *transaction, ref_transaction_add_update(transaction, refname, flags, new_oid, old_oid, new_target, - old_target, msg); + old_target, NULL, msg); return 0; } diff --git a/refs/files-backend.c b/refs/files-backend.c index 02cb4907d8659e87a227fed4f60a5f6606be8764..255fed8354cae982f785b1b85340e2a1eeecf2a6 100644 --- a/refs/files-backend.c +++ b/refs/files-backend.c @@ -1270,7 +1270,7 @@ static void prune_ref(struct files_ref_store *refs, struct ref_to_prune *r) ref_transaction_add_update( transaction, r->name, REF_NO_DEREF | REF_HAVE_NEW | REF_HAVE_OLD | REF_IS_PRUNING, - null_oid(), &r->oid, NULL, NULL, NULL); + null_oid(), &r->oid, NULL, NULL, NULL, NULL); if (ref_transaction_commit(transaction, &err)) goto cleanup; @@ -2417,7 +2417,7 @@ static int split_head_update(struct ref_update *update, transaction, "HEAD", update->flags | REF_LOG_ONLY | REF_NO_DEREF, &update->new_oid, &update->old_oid, - NULL, NULL, update->msg); + NULL, NULL, update->committer_info, update->msg); /* * Add "HEAD". This insertion is O(N) in the transaction @@ -2481,7 +2481,8 @@ static int split_symref_update(struct ref_update *update, transaction, referent, new_flags, update->new_target ? NULL : &update->new_oid, update->old_target ? NULL : &update->old_oid, - update->new_target, update->old_target, update->msg); + update->new_target, update->old_target, NULL, + update->msg); new_update->parent_update = update; @@ -2914,7 +2915,7 @@ static int files_transaction_prepare(struct ref_store *ref_store, packed_transaction, update->refname, REF_HAVE_NEW | REF_NO_DEREF, &update->new_oid, NULL, - NULL, NULL, NULL); + NULL, NULL, NULL, NULL); } } @@ -3094,12 +3095,13 @@ static int files_transaction_finish_initial(struct files_ref_store *refs, ref_transaction_add_update(loose_transaction, update->refname, update->flags & ~REF_HAVE_OLD, update->new_target ? NULL : &update->new_oid, NULL, - update->new_target, NULL, NULL); + update->new_target, NULL, update->committer_info, + NULL); } else { ref_transaction_add_update(packed_transaction, update->refname, update->flags & ~REF_HAVE_OLD, &update->new_oid, &update->old_oid, - NULL, NULL, NULL); + NULL, NULL, update->committer_info, NULL); } } diff --git a/refs/refs-internal.h b/refs/refs-internal.h index f5c733d099f0c6f1076a25f4f77d9d5eb345ec87..79b287c5ec5c7d8f759869cf93cda405640186dc 100644 --- a/refs/refs-internal.h +++ b/refs/refs-internal.h @@ -162,6 +162,7 @@ struct ref_update *ref_transaction_add_update( const struct object_id *new_oid, const struct object_id *old_oid, const char *new_target, const char *old_target, + const char *committer_info, const char *msg); /* diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c index c008f20be719fec3af6a8f81c821cb9c263764d7..b2e3ba877de9e59fea5a4d066eb13e60ef22a32b 100644 --- a/refs/reftable-backend.c +++ b/refs/reftable-backend.c @@ -1078,7 +1078,8 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store, new_update = ref_transaction_add_update( transaction, "HEAD", u->flags | REF_LOG_ONLY | REF_NO_DEREF, - &u->new_oid, &u->old_oid, NULL, NULL, u->msg); + &u->new_oid, &u->old_oid, NULL, NULL, NULL, + u->msg); string_list_insert(&affected_refnames, new_update->refname); } @@ -1161,7 +1162,8 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store, transaction, referent.buf, new_flags, u->new_target ? NULL : &u->new_oid, u->old_target ? NULL : &u->old_oid, - u->new_target, u->old_target, u->msg); + u->new_target, u->old_target, + u->committer_info, u->msg); new_update->parent_update = u; From patchwork Fri Dec 13 10:36:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karthik Nayak X-Patchwork-Id: 13906833 Received: from mail-ej1-f51.google.com (mail-ej1-f51.google.com [209.85.218.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 63E1A1CEEA8 for ; Fri, 13 Dec 2024 10:36:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734086219; cv=none; b=Yfzn2mJjIc+P9wuFYinJnx7keqJzseS0UrRDCasGP2CGU5KO1mh9evWpdXFEwgInOLerX4gxXTWC6PSBceg1hb0I3tOclDl9rtkn1c08GGjx7baEu8/n9vhzrnI8j6uxXthDWmitYEc9oQskX/yxAy+lU/iZErTyORtFg4oOnTc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734086219; c=relaxed/simple; bh=3aiM2iKCsTC1wp82pEkuMQZAxTWB2iCSybVMKz62p7s=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=LgIZm7UuRVJkh/z6TG7Jmbi7vMAuRDTE+gOR/e8jntMox1rpOB05ZSfmzCdQ+wHEgYJ8AeREuSv2i1sCGEfvlDDs0jkfbzqFgFeGkOKWAygV55MsMmPLa40fkSjDkdgHon1torsHpnY5zP/ZQexcBMLKmf3c66fU2XydCpEoKkM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=SOcR+DvF; arc=none smtp.client-ip=209.85.218.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="SOcR+DvF" Received: by mail-ej1-f51.google.com with SMTP id a640c23a62f3a-aa6c0dbce1fso209857166b.2 for ; Fri, 13 Dec 2024 02:36:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734086216; x=1734691016; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=G1AU86NkiDthAajwqrAXEV0XCKO0VDecZ2AEpp6U//0=; b=SOcR+DvFbTf3dSgJfHtrMEpEkY4hLBggBXT6auMUZDbcY8KPxDA2g0uwuhl9+W67lF geWeUjwJwDGq4jWdOmWjKkTNYuk7F34rbq5x1sBbmEVl/vM4YTSXgVeRLQJ9wqUOstBC zMyGNEjIoPRmkQwKLyDtK0dcuPPKPyZIsKMNvqrf0tH5gKHtHJhJ0KMZhM/pA0Mcq/m8 QJxiduIhtF7KY7GA4rf/dEd3TehrhTqPLprqiZaXdTTUvCjtBfIIQWQygFE9e9o3YeYn jXGOJWGcGNSY1LtdUdM5WBpQ719D0Qx43uZxBd0GD+6ZZjXWhBLskUZ5jtokGWwDcW09 WJkw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734086216; x=1734691016; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=G1AU86NkiDthAajwqrAXEV0XCKO0VDecZ2AEpp6U//0=; b=TzbqnnVZES5tcMvYPhuMHatHW+iqGYf8avyaM7G6WW2CBkEyH1fFFV3t3JVb97dHGe +QIRVN90qzkLRLiuKTcAzuzSt848KJR/cL189V5q3DzPfReLzHFjD2nrrJTFqpUycrF3 HVpPv9XUqYpdzZZWP4Kod2gyXkiOil3yt3gk9B61RWyXXNMOQoULprfLXKzPSggZbySV UhChrVHVIqypBaKD1p84KtOVeKUs2jJ08ccoXy1oT1T1JSyV/lGN3XgKC6bjTOK0v081 YV49ovlamWcehv7LlvAxUpfNcEB/rZ2WxVckfX4GHTODf1KsWQQKQheSXFOGpvExKbae BiNw== X-Gm-Message-State: AOJu0Yyk/m6JVuA/ICD4+ifFvHYn7WgW04vwvHlYwPb1PH2UJTmkBWHd 4lfuJBgBTTvezpQcPkdXLqiggozhPJ2dPwfiHlKmmqiRZkV2EQol X-Gm-Gg: ASbGncsW3NXpWAAFGnIsWx4yPPVBRDvv6ht6mEFFSYUcndjeq346GGVB1jJcmfIddMy IWUBOnBrJhSzhxseo3B8tZid9lywDamoIdM27gcCgD9ZTwEAVB/NLZQCCJM8v8Tt0PyAkVxllfV MULmoB9/qDmMY9/UoZEsURM0U2WZx1nVoKer6tUUnEp17+m8Oeh0Gzm9OjE9QflyRxe2Q+8efIW QOJztBLLeig9M9JSvlacKmjZNTktnEe+x6aw5sO8DCWOLijaYjLjre+B+0DYfTXdAMLFA== X-Google-Smtp-Source: AGHT+IF82MqGPA7OPdZz3P1afszwa9n+ofh+6b0rCET7SnHArLdAgihRzAMdC82Le/W03M/eVJ4n7A== X-Received: by 2002:a05:6402:270f:b0:5d0:8f1c:d9d7 with SMTP id 4fb4d7f45d1cf-5d63c3007e4mr4783167a12.4.1734086215460; Fri, 13 Dec 2024 02:36:55 -0800 (PST) Received: from [192.168.178.20] ([2a02:2455:8256:2d00:9c39:c2d7:aedd:294d]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aa6883f65c3sm686785266b.157.2024.12.13.02.36.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Dec 2024 02:36:55 -0800 (PST) From: Karthik Nayak Date: Fri, 13 Dec 2024 11:36:51 +0100 Subject: [PATCH v2 6/8] refs: introduce the `ref_transaction_update_reflog` function Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241213-320-git-refs-migrate-reflogs-v2-6-f28312cdb6c0@gmail.com> References: <20241213-320-git-refs-migrate-reflogs-v2-0-f28312cdb6c0@gmail.com> In-Reply-To: <20241213-320-git-refs-migrate-reflogs-v2-0-f28312cdb6c0@gmail.com> To: git@vger.kernel.org Cc: Karthik Nayak , ps@pks.im, Christian Couder X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=6372; i=karthik.188@gmail.com; h=from:subject:message-id; bh=3aiM2iKCsTC1wp82pEkuMQZAxTWB2iCSybVMKz62p7s=; b=owEB7QES/pANAwAIAT7VnySORox/AcsmYgBnXA5BwJcGSf/ohWwQ8RxdWaRhvm6z8Fu7/fRQR +S3bwvCLVmJAbMEAAEIAB0WIQRXzkx/Y3VxD8tlxgY+1Z8kjkaMfwUCZ1wOQQAKCRA+1Z8kjkaM f2gBDAChPrNL4jeY87Uu/ixIAS2CTC0jzTibIV0KJNivEgv1lCwR0nUm7vjOidKM+JL+3oT4ZIP egOizca4D6IVY5dL/i5Vh3IHnqgYpt9mDFxlENzfdiLhVg/wj+c+anBWCY9enxNPIQJLIclwJTO 69iFF2EI8Z8EK9m0Izdk7VDX3RHuQ0z342E8Y63qEFBSkxBFSYpJYsllpfp9x2zlukZEU8vinm/ LDuR3v+6Zg9736J636Lo+ETrP4XOA7X31wSRUCzBlu6Y5NtzXE6IiwVVPv8sB+4SwnPhkBX3jjF gz09n0QCqk/H4OpgI+x5z+x/GYStJIQskn5LFdcCKb/d7hx2fpIzKXJIvKVaj2+eSthKGaDaAW4 zyh3KCTAgaNVHSjpg8Lo1aqJklAVl8Jii3tVx09LXO7zLSG+Z1ckEteopBkx6JDgTI/3OakLUy5 uaHuZQQTL2zapK9xtmxaBAswNc+EYLfAVjNn7kY0xhbFzRy4KvAOrDo2wgnoXUPuA5dPI= X-Developer-Key: i=karthik.188@gmail.com; a=openpgp; fpr=57CE4C7F6375710FCB65C6063ED59F248E468C7F Introduce a new function `ref_transaction_update_reflog`, for clients to add a reflog update to a transaction. While the existing function `ref_transaction_update` also allows clients to add a reflog entry, this function does a few things more, It: - Enforces that only a reflog entry is added and does not update the ref itself. - Allows the users to also provide the committer information. This means clients can add reflog entries with custom committer information. A follow up commit will utilize this function to add reflog support to `git refs migrate`. Signed-off-by: Karthik Nayak --- refs.c | 44 ++++++++++++++++++++++++++++++++++++++------ refs.h | 12 ++++++++++++ refs/files-backend.c | 24 ++++++++++++++++-------- 3 files changed, 66 insertions(+), 14 deletions(-) diff --git a/refs.c b/refs.c index 428ca256f3e5860554e9a7fa42a8368bb2689b31..9f539369bc94a25594adc3e95847f2fe72f58a08 100644 --- a/refs.c +++ b/refs.c @@ -1203,20 +1203,21 @@ struct ref_update *ref_transaction_add_update( static int transaction_refname_valid(const char *refname, const struct object_id *new_oid, - unsigned int flags, struct strbuf *err) + unsigned int flags, unsigned int reflog, + struct strbuf *err) { if (flags & REF_SKIP_REFNAME_VERIFICATION) return 1; if (is_pseudo_ref(refname)) { - strbuf_addf(err, _("refusing to update pseudoref '%s'"), - refname); + const char *what = reflog ? "reflog for pseudoref" : "pseudoref"; + strbuf_addf(err, _("refusing to update %s '%s'"), what, refname); return 0; } else if ((new_oid && !is_null_oid(new_oid)) ? check_refname_format(refname, REFNAME_ALLOW_ONELEVEL) : !refname_is_safe(refname)) { - strbuf_addf(err, _("refusing to update ref with bad name '%s'"), - refname); + const char *what = reflog ? "reflog with bad name" : "ref with bad name"; + strbuf_addf(err, _("refusing to update %s '%s'"), what, refname); return 0; } @@ -1240,7 +1241,7 @@ int ref_transaction_update(struct ref_transaction *transaction, return -1; } - if (!transaction_refname_valid(refname, new_oid, flags, err)) + if (!transaction_refname_valid(refname, new_oid, flags, 0, err)) return -1; if (flags & ~REF_TRANSACTION_UPDATE_ALLOWED_FLAGS) @@ -1259,6 +1260,37 @@ int ref_transaction_update(struct ref_transaction *transaction, ref_transaction_add_update(transaction, refname, flags, new_oid, old_oid, new_target, old_target, NULL, msg); + + return 0; +} + +int ref_transaction_update_reflog(struct ref_transaction *transaction, + const char *refname, + const struct object_id *new_oid, + const struct object_id *old_oid, + const char *committer_info, unsigned int flags, + const char *msg, unsigned int index, + struct strbuf *err) +{ + struct ref_update *update; + + assert(err); + + if (!transaction_refname_valid(refname, new_oid, flags, 1, err)) + return -1; + + flags |= REF_LOG_ONLY | REF_NO_DEREF; + + update = ref_transaction_add_update(transaction, refname, flags, + new_oid, old_oid, NULL, NULL, + committer_info, msg); + /* + * While we do set the old_oid value, we unset the flag to skip + * old_oid verification which only makes sense for refs. + */ + update->flags &= ~REF_HAVE_OLD; + update->index = index; + return 0; } diff --git a/refs.h b/refs.h index a5bedf48cf6de91005a7e8d0bf58ca98350397a6..67f8b3eef3f2101409e5cc6eb2241d99e9f7d95c 100644 --- a/refs.h +++ b/refs.h @@ -727,6 +727,18 @@ int ref_transaction_update(struct ref_transaction *transaction, unsigned int flags, const char *msg, struct strbuf *err); +/* + * Similar to`ref_transaction_update`, but this function is only for adding + * a reflog update. Supports providing custom committer information. + */ +int ref_transaction_update_reflog(struct ref_transaction *transaction, + const char *refname, + const struct object_id *new_oid, + const struct object_id *old_oid, + const char *committer_info, unsigned int flags, + const char *msg, unsigned int index, + struct strbuf *err); + /* * Add a reference creation to transaction. new_oid is the value that * the reference should have after the update; it must not be diff --git a/refs/files-backend.c b/refs/files-backend.c index 255fed8354cae982f785b1b85340e2a1eeecf2a6..c11213f52065bcf2fa7612df8f9500692ee2d02c 100644 --- a/refs/files-backend.c +++ b/refs/files-backend.c @@ -3080,10 +3080,12 @@ static int files_transaction_finish_initial(struct files_ref_store *refs, } /* - * packed-refs don't support symbolic refs and root refs, so we - * have to queue these references via the loose transaction. + * packed-refs don't support symbolic refs, root refs and reflogs, + * so we have to queue these references via the loose transaction. */ - if (update->new_target || is_root_ref(update->refname)) { + if (update->new_target || + is_root_ref(update->refname) || + (update->flags & REF_LOG_ONLY)) { if (!loose_transaction) { loose_transaction = ref_store_transaction_begin(&refs->base, 0, err); if (!loose_transaction) { @@ -3092,11 +3094,17 @@ static int files_transaction_finish_initial(struct files_ref_store *refs, } } - ref_transaction_add_update(loose_transaction, update->refname, - update->flags & ~REF_HAVE_OLD, - update->new_target ? NULL : &update->new_oid, NULL, - update->new_target, NULL, update->committer_info, - NULL); + if (update->flags & REF_LOG_ONLY) + ref_transaction_add_update(loose_transaction, update->refname, + update->flags, &update->new_oid, + &update->old_oid, NULL, NULL, + update->committer_info, update->msg); + else + ref_transaction_add_update(loose_transaction, update->refname, + update->flags & ~REF_HAVE_OLD, + update->new_target ? NULL : &update->new_oid, NULL, + update->new_target, NULL, update->committer_info, + NULL); } else { ref_transaction_add_update(packed_transaction, update->refname, update->flags & ~REF_HAVE_OLD, From patchwork Fri Dec 13 10:36:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karthik Nayak X-Patchwork-Id: 13906834 Received: from mail-ed1-f45.google.com (mail-ed1-f45.google.com [209.85.208.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2DE481DB520 for ; Fri, 13 Dec 2024 10:36:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.45 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734086220; cv=none; b=k/tS9AxbZFVj42fYjhsvcbFgUCRQFFiu0uO/PdlSoAf3QbOC0sSWONSmNpVXEkq9cr7nsx7EnhYwMKqIIHm9w6y6+uMNtFJ4sGndCYTsOwosq62M5a9S7JB051zx3eSog42pDoGVe+CcRX0ptc+hoaVBSnwgcBHJ1lF+n9XMgH8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734086220; c=relaxed/simple; bh=f/A/BCqKAqzM0pLs1kG9GIyqPDtQjwybZoyZG7dxK2A=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=WtE2Lvq1nJqlQbhn+zmSfTzh1cOC9QjH5FKTDyqb9H1f9hXA6O1+F6Re0zCU1fwA9bpWiJTIl69lGrkIClu6S6fJXua57H38eUHwCs8mvENzHBXCA98svBrZyxrZmgGRIDuqMFcmDf6EKJOCeipzAkRP1kin9PJGOJzOw7RJzPA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=WxoV1fB5; arc=none smtp.client-ip=209.85.208.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="WxoV1fB5" Received: by mail-ed1-f45.google.com with SMTP id 4fb4d7f45d1cf-5d34030ebb2so2773017a12.1 for ; Fri, 13 Dec 2024 02:36:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734086216; x=1734691016; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=sBFYSTUfNJIIVk7ZutMhrrImhHuoVlb/4NnmMbqvDNQ=; b=WxoV1fB5/y9OdKYMqj+PsUmxjF+ShHiPZ2FHMzruCODb7F6ndR4v3SKtpsX5w5PMnV CLWa6h1AqjGTu4MmHLYdfaEEv22N+BHqqhJBERXbT18J8S2wEOianTPwR+aXmbdgMcPm 6F4p1u2UWgWKvmXWKFLLSK1Av1XaRt6eOyN9P3QzZ2J0JsTQIMDj3n0FBcuc+oH3iY/2 tz/JA5MRW7YwWGrwIZEQ9Nga0TYKniNJhU64aYXJppyBwsRjNhXk5R/m3pNsyB+kZ5qV yWQvAuoqx7sAmAdQVpz72yL9++6ZG2HtRcEg+SDmM/RJNdTfVPJ0GihCqPLHSOK7pmTq 4bHQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734086216; x=1734691016; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=sBFYSTUfNJIIVk7ZutMhrrImhHuoVlb/4NnmMbqvDNQ=; b=iNTzSZ0LbcRt8+aXPAF8Er+oX6BchApecT3nMYlHx79oVfJtlk0w36WhlczuegH1mx yk9VtByTMlzREstDBDqRXZbcj2gM6AHz3ugtqfq0UtTldWbd4D7GK8tImewx2Erz/zC6 A+HhXHhEPQIv1qd6dc0DbNLaqkP00BxYFMa3B1JSkGTQdpG+KGx5EN9FAiEGwE+uciC0 YsZ6kmzHvBUn1ZQ1NVY5SNb+kZsWMufKXQHEMZyDFKdP7FjVhRqQ5wbcAn4b86RkG394 00139tcA+4JFixZkjMMdPS+BQeQ5Mq4W78S65FvTbcEq85Zhp+agkEcGnDm3IejEPwgJ EeCw== X-Gm-Message-State: AOJu0YxAPBtB7TxOtK0DYt/5M2CgdIB65zNBTjsJPp+WmUymTjMpYcvI o2JVAMKDrAhMthpSLA08LnXcOVNWjAzwLn+wV4W2OvNl++nNTYby X-Gm-Gg: ASbGncv7phLdSGfRcrrFe8kgvfGmAgNgmZB5zieZvRFoYLGKd36rKhnCALaqtckEz0o pwtDkC54iazQdya5b/wG8Bo8WG3HDrmSLLKcXGzVCOAd2zUpVZkjYOzcF9vlqz1/q85QsdbXfgM Wjf0pw75pVzA5aWOe3EJ2MIEKHFAxOhECFhy8ZPflu4WwNlpeqf9jV8NvheQBATb5HOjuLrJePn x7xb+z+namlmjqgFZdFHwfclheecsz1cIMQolFcd1TZzQ8pd8uSqHgRVymN/xrYSKNHQw== X-Google-Smtp-Source: AGHT+IGL8GSwgdNNgsJmE92GMvA3hnymUZiQIfucOXz9d6XmK14yI4NlWC/BDHuGBH/tNk7W1IM0ug== X-Received: by 2002:a17:906:32ce:b0:aa6:77e6:ea3d with SMTP id a640c23a62f3a-aab77e7b4d0mr194019966b.45.1734086216184; Fri, 13 Dec 2024 02:36:56 -0800 (PST) Received: from [192.168.178.20] ([2a02:2455:8256:2d00:9c39:c2d7:aedd:294d]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aa6883f65c3sm686785266b.157.2024.12.13.02.36.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Dec 2024 02:36:55 -0800 (PST) From: Karthik Nayak Date: Fri, 13 Dec 2024 11:36:52 +0100 Subject: [PATCH v2 7/8] refs: allow multiple reflog entries for the same refname Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241213-320-git-refs-migrate-reflogs-v2-7-f28312cdb6c0@gmail.com> References: <20241213-320-git-refs-migrate-reflogs-v2-0-f28312cdb6c0@gmail.com> In-Reply-To: <20241213-320-git-refs-migrate-reflogs-v2-0-f28312cdb6c0@gmail.com> To: git@vger.kernel.org Cc: Karthik Nayak , ps@pks.im, Christian Couder X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=4948; i=karthik.188@gmail.com; h=from:subject:message-id; bh=f/A/BCqKAqzM0pLs1kG9GIyqPDtQjwybZoyZG7dxK2A=; b=owEB7QES/pANAwAIAT7VnySORox/AcsmYgBnXA5BSPsR2ftTdYnX7sPEb2WvT/fyZbpqSAU2C 6+NMFxkEB2JAbMEAAEIAB0WIQRXzkx/Y3VxD8tlxgY+1Z8kjkaMfwUCZ1wOQQAKCRA+1Z8kjkaM f0D7C/4wlB2jQIJyKNhLs/CppFAN47y8APmf1hv5wG4D4XTkkCYDM1CBeEx8XC6zu8P4lv+8zRt X08EPsCwdZZvz+FDUKQlSe9ryXWcxpYZGNpnIdSNFzXPI4QZFIv8GvfWOSTAF5PWA/OnPp/y+Ss HAI3obpzSRRtIaKDD/npe9plNdMASwLwkY7aLEjUJ52gyi05JggILPowhlCmjbqA+ycyeXAo+6W WW1Z/hWJUfVt8p27NuOYgoXRYPhyW/O1ETFxPrYdDg2aCKZRdGDvYdRCUOudIhjXXO+EdTw1WcI o8NdMHVoNnjuMzQ8WyoC9qt1nWetMWoa8ACMU3UH8/oUmGdVbio19NtObygPBMjpWn8tsA23rCb qD665uJ/+OAAQzf/yDck9fIH1/pmA+AylHzpTaupHwDfKMR0eku08zskcEXym/G2tvXZmegZXCe 476gVt+i6fHHK1Q9FCCk7O3wAEZgKkZwjvhyLjNJ3YdhsFyL7e6wr4l/aa2FRPrrKoXE4= X-Developer-Key: i=karthik.188@gmail.com; a=openpgp; fpr=57CE4C7F6375710FCB65C6063ED59F248E468C7F The reference transaction only allows a single update for a given reference to avoid conflicts. This, however, isn't an issue for reflogs. There are no conflicts to be resolved in reflogs and when migrating reflogs between backends we'd have multiple reflog entries for the same refname. So allow multiple reflog updates within a single transaction. Also the reflog creation logic isn't exposed to the end user. While this might change in the future, currently, this reduces the scope of issues to think about. In the reftable backend, the writer sorts all updates based on the update_index before writing to the block. When there are multiple reflogs for a given refname, it is essential that the order of the reflogs is maintained. So add the `index` value to the `update_index`. The `index` field is only be set when multiple reflog entries for a given refname are added and as such in most scenarios the old behavior remains. This is required to add reflog migration support to `git refs migrate`. Signed-off-by: Karthik Nayak --- refs/files-backend.c | 15 +++++++++++---- refs/reftable-backend.c | 22 +++++++++++++++++++--- 2 files changed, 30 insertions(+), 7 deletions(-) diff --git a/refs/files-backend.c b/refs/files-backend.c index c11213f52065bcf2fa7612df8f9500692ee2d02c..8953d1c6d37b13b0db701888b3db92fd87a68aaa 100644 --- a/refs/files-backend.c +++ b/refs/files-backend.c @@ -2611,6 +2611,9 @@ static int lock_ref_for_update(struct files_ref_store *refs, update->backend_data = lock; + if (update->flags & REF_LOG_ONLY) + goto out; + if (update->type & REF_ISSYMREF) { if (update->flags & REF_NO_DEREF) { /* @@ -2829,13 +2832,16 @@ static int files_transaction_prepare(struct ref_store *ref_store, */ for (i = 0; i < transaction->nr; i++) { struct ref_update *update = transaction->updates[i]; - struct string_list_item *item = - string_list_append(&affected_refnames, update->refname); + struct string_list_item *item; if ((update->flags & REF_IS_PRUNING) && !(update->flags & REF_NO_DEREF)) BUG("REF_IS_PRUNING set without REF_NO_DEREF"); + if (update->flags & REF_LOG_ONLY) + continue; + + item = string_list_append(&affected_refnames, update->refname); /* * We store a pointer to update in item->util, but at * the moment we never use the value of this field @@ -3035,8 +3041,9 @@ static int files_transaction_finish_initial(struct files_ref_store *refs, /* Fail if a refname appears more than once in the transaction: */ for (i = 0; i < transaction->nr; i++) - string_list_append(&affected_refnames, - transaction->updates[i]->refname); + if (!(transaction->updates[i]->flags & REF_LOG_ONLY)) + string_list_append(&affected_refnames, + transaction->updates[i]->refname); string_list_sort(&affected_refnames); if (ref_update_reject_duplicates(&affected_refnames, err)) { ret = TRANSACTION_GENERIC_ERROR; diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c index b2e3ba877de9e59fea5a4d066eb13e60ef22a32b..bec5962debea7b62572d08f6fa8fd38ab4cd8af6 100644 --- a/refs/reftable-backend.c +++ b/refs/reftable-backend.c @@ -990,8 +990,9 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store, if (ret) goto done; - string_list_append(&affected_refnames, - transaction->updates[i]->refname); + if (!(transaction->updates[i]->flags & REF_LOG_ONLY)) + string_list_append(&affected_refnames, + transaction->updates[i]->refname); } /* @@ -1301,6 +1302,7 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data struct reftable_log_record *logs = NULL; struct ident_split committer_ident = {0}; size_t logs_nr = 0, logs_alloc = 0, i; + uint64_t max_update_index = ts; const char *committer_info; int ret = 0; @@ -1405,7 +1407,19 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data } fill_reftable_log_record(log, &c); - log->update_index = ts; + + /* + * Updates are sorted by the writer. So updates for the same + * refname need to contain different update indices. + */ + log->update_index = ts + u->index; + + /* + * Note the max update_index so the limit can be set later on. + */ + if (log->update_index > max_update_index) + max_update_index = log->update_index; + log->refname = xstrdup(u->refname); memcpy(log->value.update.new_hash, u->new_oid.hash, GIT_MAX_RAWSZ); @@ -1469,6 +1483,8 @@ static int write_transaction_table(struct reftable_writer *writer, void *cb_data * and log blocks. */ if (logs) { + reftable_writer_set_limits(writer, ts, max_update_index); + ret = reftable_writer_add_logs(writer, logs, logs_nr); if (ret < 0) goto done; From patchwork Fri Dec 13 10:36:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Karthik Nayak X-Patchwork-Id: 13906835 Received: from mail-ej1-f44.google.com (mail-ej1-f44.google.com [209.85.218.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C534B1BFE03 for ; Fri, 13 Dec 2024 10:36:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.44 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734086220; cv=none; b=Ns3SBYLJOWNtZ7A6/uhkHdF1M5aiimMXVaHMFniKQwU6aeFqWJ+ROqg5D7+WFdj7vi0QE1xqxvcZRnltUjrHOan6ywwA2d+uLn9GMER833po22eOZLKJ15ewlAA493z5ziFbYrF30fEc9k2ovzX7hLZjA3gl724NclJb5myQPKY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734086220; c=relaxed/simple; bh=Y66dCnSC946I5gvH4/GQmG/6YgI4VpaE3ph4AtJnKe8=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=FLYgw5amvNeZTgd28c2A8gx2KfjOaF0qkHRM1jBgMKHx32S1KDVMnvxLlHN6/XsT7HMptuBvANQ9jxUcdkjTpDH1+/1/cSES/NUnJSISCoGoOTOnD17FvV4SAhGj2UP3SrpBjaxk8AQS7cnVuCClU7w8xU2dYCLzerWzIkZRML0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=dMOjgYeT; arc=none smtp.client-ip=209.85.218.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="dMOjgYeT" Received: by mail-ej1-f44.google.com with SMTP id a640c23a62f3a-aa692211331so311037266b.1 for ; Fri, 13 Dec 2024 02:36:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734086217; x=1734691017; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=O2xJZv3kmXE2qjjDNYApZXtq0V9WPB5zDJrYsdgLbgc=; b=dMOjgYeTqVMzyKPkGW+1wOWFwMyzg3Ghz7v+ohL6yAcBdbN9P0P2OhzkyUg+Jbz3FZ khk2p3Wam9dwyS70dJJLZt/IT7n7m8o5LPuoBqnEI8Kgo3M4DQjy4FjG06L009Je7cNd aAcFJoz/w8htaN5Uqol/qs0PCMCy43HelDvwLEql0DjHZvoYOuxNf4/zx+vWNGvgocYt yATmNl8La6aCuGNZWwMGebVYZN89laN60Kv9Fg5Los0pyzE+xJ/InjOgYKpeH//2SWgI KHdRRjQKsT11FXS07J49QWJfrQpf7sQM4Yu86FgQt8g9a5d/OYzECxsWSymMzkQhoNYs ttnA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734086217; x=1734691017; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=O2xJZv3kmXE2qjjDNYApZXtq0V9WPB5zDJrYsdgLbgc=; b=jkmdwUt9qP/HtUXbo0YRbdYyJsvLeRNnPVH0ZeELIFt+Nz+vmA6dz49vU2BOFMIDWC gZz6PNYFOQ9gPkRRuQfSE1LKAHf2eCWtD308xZ/2gMCxnYx6SnYGs7Etu9D1KRu3+4Tw P/XV6YsYYzRy70UqJPvFs+Zil9ZKV9ODIUmoELk6tHlpu8N8wnMGz05vt4MxJyDACX4A EwhB3lx0WJ9e3vSQ+BzLnVjpXm6VFv0rMEPZnwOTOrmXFCmlw916Zv8yKInl7ZZ9YOKH KotxtiBpj66mxG8xhmjKtHDVq89bCciGgntJTpo/2dW5R3Sp/Cdrc8/cqLFNjDgOgzo5 efPQ== X-Gm-Message-State: AOJu0YxMF6lwks9WFE22mxT6AXAftfWYLXv75dDB9nqkT6hGVcXGTIeg aQ6SkVeOb233+wBsU4roJ4FsNfmNNK6waWHPoEq+8YqdZ5G7qtGVvYa/GZBL X-Gm-Gg: ASbGncvNkM7VlqqodWONLKgD4pflL8aHSVyXZ/pV2hwPrOKlNADoI2omUmQNqo9wYrw krjtuDfoiEK7JLzKT1TNUkeeLMl12QqWtJYBf8OxbJAbLRBCvTQuNFUJ9nclugMlED6AoiOP7mG bKSgE4aLtJurhZOnkneBcEUGcnEJ1P2XXcJjSzdgU1hFh7h37HagInYG9mL4Htu0+P3NIyGOShP O7HHNaogogbOufUAWIuamIhMEATgwO7ns6RrcDeDZBQqTRE5J4pf4keHiCY/E2HCIqcLw== X-Google-Smtp-Source: AGHT+IGuRwxZTw8LBy7tl3jV450KXzIxXf3uWo/N2ckbNaVgpe54CO3TJ1rUfpjkbnxcoLwuQw0ORA== X-Received: by 2002:a17:907:d92:b0:aa6:ade8:2d8c with SMTP id a640c23a62f3a-aa6c40ba702mr741321166b.5.1734086216949; Fri, 13 Dec 2024 02:36:56 -0800 (PST) Received: from [192.168.178.20] ([2a02:2455:8256:2d00:9c39:c2d7:aedd:294d]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aa6883f65c3sm686785266b.157.2024.12.13.02.36.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Dec 2024 02:36:56 -0800 (PST) From: Karthik Nayak Date: Fri, 13 Dec 2024 11:36:53 +0100 Subject: [PATCH v2 8/8] refs: add support for migrating reflogs Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241213-320-git-refs-migrate-reflogs-v2-8-f28312cdb6c0@gmail.com> References: <20241213-320-git-refs-migrate-reflogs-v2-0-f28312cdb6c0@gmail.com> In-Reply-To: <20241213-320-git-refs-migrate-reflogs-v2-0-f28312cdb6c0@gmail.com> To: git@vger.kernel.org Cc: Karthik Nayak , ps@pks.im, Christian Couder X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=11041; i=karthik.188@gmail.com; h=from:subject:message-id; bh=Y66dCnSC946I5gvH4/GQmG/6YgI4VpaE3ph4AtJnKe8=; b=owEB7QES/pANAwAIAT7VnySORox/AcsmYgBnXA5BKnm+9CtHZQsTPxdaYljFytP9GA+e/LfJR sh3TVvYMfaJAbMEAAEIAB0WIQRXzkx/Y3VxD8tlxgY+1Z8kjkaMfwUCZ1wOQQAKCRA+1Z8kjkaM f9Z7C/9jg3xVNPrpgWogvVpBj7qMwnHxsb1/L+gB2CD2RFCdQt0fhIDvpLVyp32EuY9G406GHf/ /6lvaWM/tBW643WWR0KdzevxIaLAOC3llx+/UvbjHDbRPCa1lqmTYXGD0kZxSgIn8NVuUmGQvaV 8EO3YkMi4sL/3vgTM5oYP3zUrlsrXPySQeso64pck/+F9m9XAVK2067lksowMBexHDc9rdd4pCD h6SAkdGlB2yyVpaGG/pl1yFZnd8oDSN7JSH2k/9KxeSnhU4DRlCa/+Twhq/tFpvJJgZcDbNDRMV u2e9kxOCgz+6W9F/rWdERrJnCha8al5+2ePb1SFeDHDXfogI/gg9Twb1q4zevJ62qOceCNuu5jt iKzimobfLGYr+bIxvt4wv39Lld5WWpqJtp42QLDuxx+A5JbyEyp90DYzAIfZmS728K2qrUyvhFN 1VXLneznzj6qb41aozidTDW//U9CTnEJ2wNkfRNgXXKBheEtqnFPTlXKb5nmL6h0jO310= X-Developer-Key: i=karthik.188@gmail.com; a=openpgp; fpr=57CE4C7F6375710FCB65C6063ED59F248E468C7F The `git refs migrate` command was introduced in 25a0023f28 (builtin/refs: new command to migrate ref storage formats, 2024-06-06) to support migrating from one reference backend to another. One limitation of the command was that it didn't support migrating repositories which contained reflogs. A previous commit, added support for adding reflog updates in ref transactions. Using the added functionality bake in reflog support for `git refs migrate`. To ensure that the order of the reflogs is maintained during the migration, we add the index for each reflog update as we iterate over the reflogs from the old reference backend. This is to ensure that the order is maintained in the new backend. Signed-off-by: Karthik Nayak --- Documentation/git-refs.txt | 2 -- refs.c | 89 ++++++++++++++++++++++++++++++++-------------- t/t1460-refs-migrate.sh | 73 +++++++++++++++++++++++++------------ 3 files changed, 113 insertions(+), 51 deletions(-) diff --git a/Documentation/git-refs.txt b/Documentation/git-refs.txt index ce31f93061db5e5d16aca516dd3d15f6527db870..9829984b0a4c4f54ec7f9b6c6c7072f62b1d198d 100644 --- a/Documentation/git-refs.txt +++ b/Documentation/git-refs.txt @@ -57,8 +57,6 @@ KNOWN LIMITATIONS The ref format migration has several known limitations in its current form: -* It is not possible to migrate repositories that have reflogs. - * It is not possible to migrate repositories that have worktrees. * There is no way to block concurrent writes to the repository during an diff --git a/refs.c b/refs.c index 9f539369bc94a25594adc3e95847f2fe72f58a08..f19292d50f0003881220e8f7cfcf6c7eb4b2e749 100644 --- a/refs.c +++ b/refs.c @@ -30,6 +30,7 @@ #include "date.h" #include "commit.h" #include "wildmatch.h" +#include "ident.h" /* * List of all available backends @@ -2673,6 +2674,7 @@ int ref_update_check_old_target(const char *referent, struct ref_update *update, } struct migration_data { + unsigned int reflog_index; struct ref_store *old_refs; struct ref_transaction *transaction; struct strbuf *errbuf; @@ -2708,6 +2710,53 @@ static int migrate_one_ref(const char *refname, const char *referent UNUSED, con return ret; } +struct reflog_migration_data { + unsigned int *index; + const char *refname; + struct ref_store *old_refs; + struct ref_transaction *transaction; + struct strbuf *errbuf; +}; + +static int migrate_one_reflog_entry(struct object_id *old_oid, + struct object_id *new_oid, + const char *committer, + timestamp_t timestamp, int tz, + const char *msg, void *cb_data) +{ + struct reflog_migration_data *data = cb_data; + struct strbuf sb = STRBUF_INIT; + const char *date; + int ret; + + date = show_date(timestamp, tz, DATE_MODE(NORMAL)); + /* committer contains name and email */ + strbuf_addstr(&sb, fmt_ident("", committer, WANT_BLANK_IDENT, date, 0)); + + ret = ref_transaction_update_reflog(data->transaction, data->refname, + new_oid, old_oid, sb.buf, + REF_HAVE_NEW | REF_HAVE_OLD, msg, + (*data->index)++, data->errbuf); + strbuf_release(&sb); + + return ret; +} + +static int migrate_one_reflog(const char *refname, void *cb_data) +{ + struct migration_data *migration_data = cb_data; + struct reflog_migration_data data; + + data.refname = refname; + data.old_refs = migration_data->old_refs; + data.transaction = migration_data->transaction; + data.errbuf = migration_data->errbuf; + data.index = &migration_data->reflog_index; + + return refs_for_each_reflog_ent(migration_data->old_refs, refname, + migrate_one_reflog_entry, &data); +} + static int move_files(const char *from_path, const char *to_path, struct strbuf *errbuf) { struct strbuf from_buf = STRBUF_INIT, to_buf = STRBUF_INIT; @@ -2774,13 +2823,6 @@ static int move_files(const char *from_path, const char *to_path, struct strbuf return ret; } -static int count_reflogs(const char *reflog UNUSED, void *payload) -{ - size_t *reflog_count = payload; - (*reflog_count)++; - return 0; -} - static int has_worktrees(void) { struct worktree **worktrees = get_worktrees(); @@ -2806,7 +2848,6 @@ int repo_migrate_ref_storage_format(struct repository *repo, struct ref_transaction *transaction = NULL; struct strbuf new_gitdir = STRBUF_INIT; struct migration_data data; - size_t reflog_count = 0; int did_migrate_refs = 0; int ret; @@ -2818,21 +2859,6 @@ int repo_migrate_ref_storage_format(struct repository *repo, old_refs = get_main_ref_store(repo); - /* - * We do not have any interfaces that would allow us to write many - * reflog entries. Once we have them we can remove this restriction. - */ - if (refs_for_each_reflog(old_refs, count_reflogs, &reflog_count) < 0) { - strbuf_addstr(errbuf, "cannot count reflogs"); - ret = -1; - goto done; - } - if (reflog_count) { - strbuf_addstr(errbuf, "migrating reflogs is not supported yet"); - ret = -1; - goto done; - } - /* * Worktrees complicate the migration because every worktree has a * separate ref storage. While it should be feasible to implement, this @@ -2858,17 +2884,21 @@ int repo_migrate_ref_storage_format(struct repository *repo, * This operation is safe as we do not yet modify the main * repository. * - * 3. If we're in dry-run mode then we are done and can hand over the + * 3. Enumerate all reflogs and write them into the new ref storage. + * This operation is safe as we do not yet modify the main + * repository. + * + * 4. If we're in dry-run mode then we are done and can hand over the * directory to the caller for inspection. If not, we now start * with the destructive part. * - * 4. Delete the old ref storage from disk. As we have a copy of refs + * 5. Delete the old ref storage from disk. As we have a copy of refs * in the new ref storage it's okay(ish) if we now get interrupted * as there is an equivalent copy of all refs available. * - * 5. Move the new ref storage files into place. + * 6. Move the new ref storage files into place. * - * 6. Change the repository format to the new ref format. + * 7. Change the repository format to the new ref format. */ strbuf_addf(&new_gitdir, "%s/%s", old_refs->gitdir, "ref_migration.XXXXXX"); if (!mkdtemp(new_gitdir.buf)) { @@ -2910,6 +2940,11 @@ int repo_migrate_ref_storage_format(struct repository *repo, if (ret < 0) goto done; + data.reflog_index = 1; + ret = refs_for_each_reflog(old_refs, migrate_one_reflog, &data); + if (ret < 0) + goto done; + ret = ref_transaction_commit(transaction, errbuf); if (ret < 0) goto done; diff --git a/t/t1460-refs-migrate.sh b/t/t1460-refs-migrate.sh index 1bfff3a7afd5acc470424dfe7ec3e97d45f5c481..f59bc4860f19c4af82dc6f2984bdb69d61fe3ec2 100755 --- a/t/t1460-refs-migrate.sh +++ b/t/t1460-refs-migrate.sh @@ -7,23 +7,44 @@ export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME . ./test-lib.sh +# Migrate the provided repository from one format to the other and +# verify that the references and logs are migrated over correctly. +# Usage: test_migration +# is the relative path to the repo to be migrated. +# is the ref format to be migrated to. +# (true or false) whether to skip reflog verification. test_migration () { - git -C "$1" for-each-ref --include-root-refs \ + repo=$1 && + format=$2 && + skip_reflog_verify=${3:-false} && + git -C "$repo" for-each-ref --include-root-refs \ --format='%(refname) %(objectname) %(symref)' >expect && - git -C "$1" refs migrate --ref-format="$2" && - git -C "$1" for-each-ref --include-root-refs \ + if ! $skip_reflog_verify + then + git -C "$repo" reflog --all >expect_logs && + git -C "$repo" reflog list >expect_log_list + fi && + + git -C "$repo" refs migrate --ref-format="$2" && + + git -C "$repo" for-each-ref --include-root-refs \ --format='%(refname) %(objectname) %(symref)' >actual && test_cmp expect actual && + if ! $skip_reflog_verify + then + git -C "$repo" reflog --all >actual_logs && + git -C "$repo" reflog list >actual_log_list && + test_cmp expect_logs actual_logs && + test_cmp expect_log_list actual_log_list + fi && - git -C "$1" rev-parse --show-ref-format >actual && - echo "$2" >expect && + git -C "$repo" rev-parse --show-ref-format >actual && + echo "$format" >expect && test_cmp expect actual } test_expect_success 'setup' ' - rm -rf .git && - # The migration does not yet support reflogs. - git config --global core.logAllRefUpdates false + rm -rf .git ' test_expect_success "superfluous arguments" ' @@ -78,19 +99,6 @@ do test_cmp expect err ' - test_expect_success "$from_format -> $to_format: migration with reflog fails" ' - test_when_finished "rm -rf repo" && - git init --ref-format=$from_format repo && - test_config -C repo core.logAllRefUpdates true && - test_commit -C repo logged && - test_must_fail git -C repo refs migrate \ - --ref-format=$to_format 2>err && - cat >expect <<-EOF && - error: migrating reflogs is not supported yet - EOF - test_cmp expect err - ' - test_expect_success "$from_format -> $to_format: migration with worktree fails" ' test_when_finished "rm -rf repo" && git init --ref-format=$from_format repo && @@ -141,7 +149,7 @@ do test_commit -C repo initial && test-tool -C repo ref-store main update-ref "" refs/heads/broken \ "$(test_oid 001)" "$ZERO_OID" REF_SKIP_CREATE_REFLOG,REF_SKIP_OID_VERIFICATION && - test_migration repo "$to_format" && + test_migration repo "$to_format" true && test_oid 001 >expect && git -C repo rev-parse refs/heads/broken >actual && test_cmp expect actual @@ -195,6 +203,27 @@ do git -C repo rev-parse --show-ref-format >actual && test_cmp expect actual ' + + test_expect_success "$from_format -> $to_format: reflogs of symrefs with target deleted" ' + test_when_finished "rm -rf repo" && + git init --ref-format=$from_format repo && + test_commit -C repo initial && + git -C repo branch branch-1 HEAD && + git -C repo symbolic-ref refs/heads/symref refs/heads/branch-1 && + cat >input <<-EOF && + delete refs/heads/branch-1 + EOF + git -C repo update-ref --stdin $to_format: reflogs order is retained" ' + test_when_finished "rm -rf repo" && + git init --ref-format=$from_format repo && + test_commit --date "100005000 +0700" --no-tag -C repo initial && + test_commit --date "100003000 +0700" --no-tag -C repo second && + test_migration repo "$to_format" + ' done done