From patchwork Tue Jun 2 08:25:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Patrick Steinhardt X-Patchwork-Id: 11583375 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 10F38912 for ; Tue, 2 Jun 2020 08:25:01 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E7570206C3 for ; Tue, 2 Jun 2020 08:25:00 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=pks.im header.i=@pks.im header.b="Jwr8M/d8"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="x2vhZs08" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726179AbgFBIY7 (ORCPT ); Tue, 2 Jun 2020 04:24:59 -0400 Received: from out4-smtp.messagingengine.com ([66.111.4.28]:55611 "EHLO out4-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725811AbgFBIY6 (ORCPT ); Tue, 2 Jun 2020 04:24:58 -0400 Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.nyi.internal (Postfix) with ESMTP id 303035C00A9 for ; Tue, 2 Jun 2020 04:24:57 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute3.internal (MEProxy); Tue, 02 Jun 2020 04:24:57 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pks.im; h=date :from:to:subject:message-id:mime-version:content-type; s=fm1; bh=CBvkvbMJYc1o7V9yreARQPJi4NBTIAE++qUtdeqqpeE=; b=Jwr8M/d8nvna HJK2IUx6yqn0HJt2s9gmgzAKcDeO02yBjQE0NJu5Fxkj4i0Wn+QMKPdrNfBenIwL SORlZdt/iop8F2poOK6MxaUqvtKQeiVM1EL9XVa/TO/KpsAWUSLl1JBmQPbAEjq4 1/wNELCa0DKMhKt4ieWVKheKh0tgTAO+34FwaJBCw8X7QTbDKlWDjqW/BPAUuP64 1LtYlGHNeCXYhVBOxydJAQPktrlMQporSmynFZb7BPszpGhVvbMikNnQ4g07AnSF mrPOQ0yhTDVbe1OAZ/qiFQzzk86GusYjcTNic6OZQgDHYIIXV+U+H7kgF89sZZUV zW6Kzt5UlA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-type:date:from:message-id :mime-version:subject:to:x-me-proxy:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm2; bh=CBvkvbMJYc1o7V9yreARQPJi4NBTI AE++qUtdeqqpeE=; b=x2vhZs08w7cuFGm+OrjsHTiMKoOf4U5HibfPhzfYihRfF +42RLrc9Tk7HRlOJcvyDcYyXUY+Vw17TG3/kslBhcOhBpeY2HPGXYUvKKi/5Q8PC tdVgLD0VL4Wc3wUGmvBF1Pm1RS1QdcmxrhJai6Z/QEgupyH40Fu0s3DDZ4lz94Z2 lwNVc4JI10ypySZtrBTJf1szCr88lCFD7Ii1LIEGXjfAUbUMkVPhjvkqzmS1J8rl QArSA6NNufIGwYcpCS/R+bek2GehvuwqOjSPLjPDl8HU9hmfHWyidaPcaC+4S7TM 8p8tTYKxqmrIAysPtvTLt7dOJo4wq0+ahtCEdlfUA== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduhedrudefjecutefuodetggdotefrodftvfcurf hrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecuuegr ihhlohhuthemuceftddtnecunecujfgurhepfffhvffukfggtggusehgtderredttddvne cuhfhrohhmpefrrghtrhhitghkucfuthgvihhnhhgrrhguthcuoehpshesphhkshdrihhm qeenucggtffrrghtthgvrhhnpeejieefvdeuleffgfejudffvdeghfeigfejgfdvvdefud evffefveffhffgkeeiffenucfkphepjeekrdehhedruddtiedrhedunecuvehluhhsthgv rhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepphhssehpkhhsrdhimh X-ME-Proxy: Received: from vm-mail.pks.im (x4e376a33.dyn.telefonica.de [78.55.106.51]) by mail.messagingengine.com (Postfix) with ESMTPA id 6BCB33280060 for ; Tue, 2 Jun 2020 04:24:56 -0400 (EDT) Received: from localhost (tanuki [10.192.0.23]) by vm-mail.pks.im (OpenSMTPD) with ESMTPSA id ec42fb47 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO) for ; Tue, 2 Jun 2020 08:24:54 +0000 (UTC) Date: Tue, 2 Jun 2020 10:25:44 +0200 From: Patrick Steinhardt To: git@vger.kernel.org Subject: [PATCH] refs: implement reference transaction hooks Message-ID: <1d1a94426f95d842e0e3ea6a1569c0c45239229c.1591086316.git.ps@pks.im> MIME-Version: 1.0 Content-Disposition: inline Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org The low-level reference transactions used to update references are currently completely opaque to the user. While certainly desirable in most usecases, there are some which might want to hook into the transaction to observe all queued reference updates as well as observing the abortion or commit of a prepared transaction. One such usecase would be to have a set of replicas of a given Git repository, where we perform Git operations on all of the repositories at once and expect the outcome to be the same in all of them. While there exist hooks already for a certain subset of Git commands that could be used to implement a voting mechanism for this, many others currently don't have any mechanism for this. The above scenario is the motivation for a set of three new hooks that reach directly into Git's reference transaction. Each of the following new hooks (currently) doesn't accept any parameters and receives the set of queued reference updates via stdin: - ref-transaction-prepared gets called when all reference updates have been queued. At this stage, the hook may decide to abort the transaction prematurely by returning a non-zero status code. - ref-transaction-committed gets called when a reference transaction was transmitted and all queued updates have been persisted. - ref-transaction-aborted gets called when a reference transaction was aborted and all queued updates have been rolled back. Given the usecase described above, a voting mechanism can now be implemented as a "ref-transaction-prepared" hook: as soon as it gets called, it will take all of stdin and use it to cast a vote to a central service. When all replicas of the repository agree, the hook will exit with zero, otherwise it will abort the transaction by returning non-zero. The most important upside is that this will catch _all_ commands writing references at once, allowing to implement strong consistency for reference updates via a single mechanism. Signed-off-by: Patrick Steinhardt --- Documentation/githooks.txt | 51 ++++++++++++++++++ refs.c | 67 +++++++++++++++++++++++- t/t1416-ref-transaction-hooks.sh | 88 ++++++++++++++++++++++++++++++++ 3 files changed, 204 insertions(+), 2 deletions(-) create mode 100755 t/t1416-ref-transaction-hooks.sh diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt index 81f2a87e88..48f8446943 100644 --- a/Documentation/githooks.txt +++ b/Documentation/githooks.txt @@ -404,6 +404,57 @@ Both standard output and standard error output are forwarded to `git send-pack` on the other end, so you can simply `echo` messages for the user. +ref-transaction-prepared +~~~~~~~~~~~~~~~~~~~~~~~~ + +This hook is invoked by any Git command that performs reference +updates. It executes as soon as all reference updates were queued to +the transaction and locked on disk. This hook executes for every +reference transaction that is being prepared and may thus get called +multiple times. + +It takes no arguments, but for each ref to be updated it receives on +standard input a line of the format: + + SP SP LF + +If the hook exits with a non-zero status, the transaction is aborted +and the command exits immediately. The +<> hook is not +executed in that case. + +[[ref-transaction-aborted]] +ref-transaction-aborted +~~~~~~~~~~~~~~~~~~~~~~~ + +This hook is invoked by any Git command that performs reference +updates. It executes as soon as a reference transaction is aborted and +after all reference locks were released and any changes made to +references were rolled back. The hook may get called multiple times or +never in case no transaction was aborted. + +The hook takes no arguments, but for each ref to be updated it +receives on standard input a line of the format: + + SP SP LF + +The hook's exit code is discarded by Git. + +ref-transaction-committed +~~~~~~~~~~~~~~~~~~~~~~~~~ + +This hook is invoked by any Git command that performs reference +updates. It executes as soon as a reference transaction is committed, +persisting all changes to disk and releasing any locks. The hook may +get called multiple times or never in case no transaction was aborted. + +The hook takes no arguments, but for each ref to be updated it +receives on standard input a line of the format: + + SP SP LF + +The hook's exit code is discarded by Git. + push-to-checkout ~~~~~~~~~~~~~~~~ diff --git a/refs.c b/refs.c index 224ff66c7b..e41fa7ea55 100644 --- a/refs.c +++ b/refs.c @@ -9,6 +9,7 @@ #include "iterator.h" #include "refs.h" #include "refs/refs-internal.h" +#include "run-command.h" #include "object-store.h" #include "object.h" #include "tag.h" @@ -16,6 +17,7 @@ #include "worktree.h" #include "argv-array.h" #include "repository.h" +#include "sigchain.h" /* * List of all available backends @@ -1986,10 +1988,56 @@ int ref_update_reject_duplicates(struct string_list *refnames, return 0; } +static int run_transaction_hook(struct ref_transaction *transaction, + const char *hook_name) +{ + struct child_process proc = CHILD_PROCESS_INIT; + struct strbuf buf = STRBUF_INIT; + const char *argv[2]; + int code, i; + + argv[0] = find_hook(hook_name); + if (!argv[0]) + return 0; + + argv[1] = NULL; + + proc.argv = argv; + proc.in = -1; + proc.stdout_to_stderr = 1; + proc.trace2_hook_name = hook_name; + + code = start_command(&proc); + if (code) + return code; + + sigchain_push(SIGPIPE, SIG_IGN); + + for (i = 0; i < transaction->nr; i++) { + struct ref_update *update = transaction->updates[i]; + + strbuf_reset(&buf); + strbuf_addf(&buf, "%s %s %s\n", + oid_to_hex(&update->old_oid), + oid_to_hex(&update->new_oid), + update->refname); + + if (write_in_full(proc.in, buf.buf, buf.len) < 0) + break; + } + + close(proc.in); + sigchain_pop(SIGPIPE); + + strbuf_release(&buf); + return finish_command(&proc); +} + int ref_transaction_prepare(struct ref_transaction *transaction, struct strbuf *err) { struct ref_store *refs = transaction->ref_store; + int ret; switch (transaction->state) { case REF_TRANSACTION_OPEN: @@ -2012,7 +2060,17 @@ int ref_transaction_prepare(struct ref_transaction *transaction, return -1; } - return refs->be->transaction_prepare(refs, transaction, err); + ret = refs->be->transaction_prepare(refs, transaction, err); + if (ret) + return ret; + + ret = run_transaction_hook(transaction, "ref-transaction-prepared"); + if (ret) { + ref_transaction_abort(transaction, err); + die(_("ref updates aborted by hook")); + } + + return 0; } int ref_transaction_abort(struct ref_transaction *transaction, @@ -2036,6 +2094,8 @@ int ref_transaction_abort(struct ref_transaction *transaction, break; } + run_transaction_hook(transaction, "ref-transaction-aborted"); + ref_transaction_free(transaction); return ret; } @@ -2064,7 +2124,10 @@ int ref_transaction_commit(struct ref_transaction *transaction, break; } - return refs->be->transaction_finish(refs, transaction, err); + ret = refs->be->transaction_finish(refs, transaction, err); + if (!ret) + run_transaction_hook(transaction, "ref-transaction-committed"); + return ret; } int refs_verify_refname_available(struct ref_store *refs, diff --git a/t/t1416-ref-transaction-hooks.sh b/t/t1416-ref-transaction-hooks.sh new file mode 100755 index 0000000000..b6df5fc883 --- /dev/null +++ b/t/t1416-ref-transaction-hooks.sh @@ -0,0 +1,88 @@ +#!/bin/sh + +test_description='reference transaction hooks' + +. ./test-lib.sh + +create_commit () +{ + test_tick && + T=$(git write-tree) && + sha1=$(echo message | git commit-tree $T) && + echo $sha1 +} + +test_expect_success setup ' + mkdir -p .git/hooks +' + +test_expect_success 'prepared hook allows updating ref' ' + test_when_finished "rm .git/hooks/ref-transaction-prepared" && + write_script .git/hooks/ref-transaction-prepared <<-\EOF && + exit 0 + EOF + C=$(create_commit) && + git update-ref HEAD $C +' + +test_expect_success 'prepared hook aborts updating ref' ' + test_when_finished "rm .git/hooks/ref-transaction-prepared" && + write_script .git/hooks/ref-transaction-prepared <<-\EOF && + exit 1 + EOF + C=$(create_commit) && + test_must_fail git update-ref HEAD $C 2>err && + grep "ref updates aborted by hook" err +' + +test_expect_success 'prepared hook gets all queued updates' ' + test_when_finished "rm .git/hooks/ref-transaction-prepared actual" && + write_script .git/hooks/ref-transaction-prepared <<-\EOF && + while read -r line; do printf "%s\n" "$line"; done >actual + EOF + C=$(create_commit) && + cat >expect <<-EOF && + $ZERO_OID $C HEAD + $ZERO_OID $C refs/heads/master + EOF + git update-ref HEAD $C <<-EOF && + update HEAD $ZERO_OID $C + update refs/heads/master $ZERO_OID $C + EOF + test_cmp expect actual +' + +test_expect_success 'committed hook gets all queued updates' ' + test_when_finished "rm .git/hooks/ref-transaction-committed actual" && + write_script .git/hooks/ref-transaction-committed <<-\EOF && + while read -r line; do printf "%s\n" "$line"; done >actual + EOF + C=$(create_commit) && + cat >expect <<-EOF && + $ZERO_OID $C HEAD + $ZERO_OID $C refs/heads/master + EOF + git update-ref HEAD $C && + test_cmp expect actual +' + +test_expect_success 'aborted hook gets all queued updates' ' + test_when_finished "rm .git/hooks/ref-transaction-aborted actual" && + write_script .git/hooks/ref-transaction-aborted <<-\EOF && + while read -r line; do printf "%s\n" "$line"; done >actual + EOF + C=$(create_commit) && + cat >expect <<-EOF && + $ZERO_OID $C HEAD + $ZERO_OID $C refs/heads/master + EOF + git update-ref --stdin <<-EOF && + start + update HEAD $C $ZERO_OID + update refs/heads/master $C $ZERO_OID + abort + EOF + test_cmp expect actual +' + +test_done