From patchwork Tue Dec 22 00:02:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 11985419 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C5FC9C433E0 for ; Tue, 22 Dec 2020 00:03:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8A3ED22525 for ; Tue, 22 Dec 2020 00:03:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726117AbgLVADI (ORCPT ); Mon, 21 Dec 2020 19:03:08 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47834 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725844AbgLVADH (ORCPT ); Mon, 21 Dec 2020 19:03:07 -0500 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 12453C0613D6 for ; Mon, 21 Dec 2020 16:02:27 -0800 (PST) Received: by mail-yb1-xb4a.google.com with SMTP id g67so15735832ybb.9 for ; Mon, 21 Dec 2020 16:02:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=FDC3b5la1dUCH0jY1gWHeWm5B7/npxZnWSKi8v9mPAg=; b=TSAy6pckljszGZeBT23aEUz6Lg4JttCOSVT4qGbhkpxsgy4cy1ibAO5YWhpfwDA4we 3yHkbs/Ak1EJ926e+5zZpKkXJ3pfjJVsN1kcXm3C+ZlJeH++k0QD1hV0ERWrqj4l1lxL BxnYCRL8OUkp0mN9+cAQxSXp4Q34EQgnv6AIPyAA114+9f39dvpcqCcvRJWsXsQB6vB8 ZRCwI0OhCpuTYVE8uTWtQlLOmpg2L/kZ6iynQm4sKQeEcKrn6GCuMDUVxx8e+Kq9ctht OQgGDiojvYNlVsqRv9u8gyg6la0hg0tcrlCZ/h9+weYMzGA3QxQQ00i/HRwcdgnUOdEz iLNA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=FDC3b5la1dUCH0jY1gWHeWm5B7/npxZnWSKi8v9mPAg=; b=ByLL+14vyG1trT5kKYa3g1/O1/88TGh2JQR2ELiFECoSPZBUv2miQk6U7nYSS/QocI /4GmthvI8PKYvs67ne3chzyyH8V2Ea/e4Cj+KXJA5y9vNtYC7tz9qJtZhlmliHnC3MPW 4W3r5KwztAyaTO/5CTFGxCFUXwrgiX9bn8OQUZunCyI8aDNFIOdtNWitDldlb5EzNHWm v5wtJoDr1dmXHxYKDPvO0F5a6rLZ4c7GCSF2JVn92dJxMo8j1/iOVdaC5bkahPEzZb4p WM7Sy71Pqdy1Ia59aF6jKRkDYgcmg4NVueuu2PRkXqbJfz65+iP903ZDgqFK5gDWnhQc s38Q== X-Gm-Message-State: AOAM532YmHHmz5cEjrsl47MOPPJ7xXiur+mbs34n7+WR3Zdnx8oUeX/7 MPuqxnqgOv/Xz42kTiX8Nq3nK3JHPSOzhIdFrKsVn+n/KIOmS1o+cdb2fu5ktTjY8vZbsy1qwSB +6KCEsarlb0SWR6x1nHC8xl1Y7l7P6sLZNFZKUss0Grg/Yo073kE71oD8VGJu0TOIBrUZpbunug == X-Google-Smtp-Source: ABdhPJxZJRJ/R22Rj0IfLeKBDhBAPbZwki7EHq21mxTPuWOaPfgMxEo78j7NWnlqYESYqkLz+8FS54Y5l2zjIopTHnE= Sender: "emilyshaffer via sendgmr" X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:1ea0:b8ff:fe77:f690]) (user=emilyshaffer job=sendgmr) by 2002:a25:ade2:: with SMTP id d34mr25319111ybe.15.1608595346204; Mon, 21 Dec 2020 16:02:26 -0800 (PST) Date: Mon, 21 Dec 2020 16:02:04 -0800 In-Reply-To: <20201222000220.1491091-1-emilyshaffer@google.com> Message-Id: <20201222000220.1491091-2-emilyshaffer@google.com> Mime-Version: 1.0 References: <20201222000220.1491091-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.29.2.490.gc7ae633391 Subject: [PATCH v7 01/17] doc: propose hooks managed by the config From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Begin a design document for config-based hooks, managed via git-hook. Focus on an overview of the implementation and motivation for design decisions. Briefly discuss the alternatives considered before this point. Also, attempt to redefine terms to fit into a multihook world. Signed-off-by: Emily Shaffer --- Notes: Since v6, checked for inconsistencies with implementation and added lots of caveats about whether 'git hook add' and 'git hook edit' will ever materialize. Hopefully this reflects reality now; please review accordingly. Since v6, checked for inconsistencies with implementation and added lots of caveats about whether 'git hook add' and 'git hook edit' will ever materialize. Hopefully this reflects reality now; please review accordingly. Since v4, addressed comments from Jonathan Tan about wording. However, I have not addressed AEvar's comments or done a full re-review of this document. I wanted to get the rest of the series out for initial review first. - Emily Since v4, addressed comments from Jonathan Tan about wording. Documentation/Makefile | 1 + .../technical/config-based-hooks.txt | 355 ++++++++++++++++++ 2 files changed, 356 insertions(+) create mode 100644 Documentation/technical/config-based-hooks.txt diff --git a/Documentation/Makefile b/Documentation/Makefile index 69dbe4bb0b..505d318da1 100644 --- a/Documentation/Makefile +++ b/Documentation/Makefile @@ -81,6 +81,7 @@ SP_ARTICLES += $(API_DOCS) TECH_DOCS += MyFirstContribution TECH_DOCS += MyFirstObjectWalk TECH_DOCS += SubmittingPatches +TECH_DOCS += technical/config-based-hooks TECH_DOCS += technical/hash-function-transition TECH_DOCS += technical/http-protocol TECH_DOCS += technical/index-format diff --git a/Documentation/technical/config-based-hooks.txt b/Documentation/technical/config-based-hooks.txt new file mode 100644 index 0000000000..3217faba47 --- /dev/null +++ b/Documentation/technical/config-based-hooks.txt @@ -0,0 +1,355 @@ +Configuration-based hook management +=================================== +:sectanchors: + +[[motivation]] +== Motivation + +Replace the .git/hook/hookname path as the only source of hooks to execute; +allow users to define hooks using config files, in a way which is friendly to +users with multiple repos which have similar needs. + +Redefine "hook" as an event rather than a single script, allowing users to +perform multiple unrelated actions on a single event. + +Take a step closer to safety when copying zipped Git repositories from untrusted +users by making it more apparent to users which scripts will be run during +normal Git operations. + +Make it easier for users to discover Git's hook feature and automate their +workflows. + +[[user-interfaces]] +== User interfaces + +[[config-schema]] +=== Config schema + +Hooks can be introduced by editing the configuration manually. There are two new +sections added, `hook` and `hookcmd`. + +[[config-schema-hook]] +==== `hook` + +Primarily contains subsections for each hook event. The order of variables in +these subsections defines the hook command execution order; hook commands can be +specified by setting the value directly to the command if no additional +configuration is needed, or by setting the value as the name of a `hookcmd`. If +Git does not find a `hookcmd` whose subsection matches the value of the given +command string, Git will try to execute the string directly. Hooks are executed +by passing the resolved command string to the shell. In the future, hook event +subsections could also contain per-hook-event settings; see +<> for more details. + +Also contains top-level hook execution settings, for example, `hook.runHookDir`. +(These settings are described more in <>.) + +---- +[hook "pre-commit"] + command = perl-linter + command = /usr/bin/git-secrets --pre-commit + +[hook "pre-applypatch"] + command = perl-linter + # for illustration purposes; error behavior isn't planned yet + error = ignore + +[hook] + runHookDir = interactive +---- + +[[config-schema-hookcmd]] +==== `hookcmd` + +Defines a hook command and its attributes, which will be used when a hook event +occurs. Unqualified attributes are assumed to apply to this hook during all hook +events, but event-specific attributes can also be supplied. The example runs +`/usr/bin/lint-it --language=perl `, but for repos which +include this config, the hook command will be skipped for all events. +Theoretically, the last line could be used to "un-skip" the hook command for +`pre-commit` hooks, but this hasn't been scoped or implemented yet. + +---- +[hookcmd "perl-linter"] + command = /usr/bin/lint-it --language=perl + skip = true + # for illustration purposes; below hasn't been defined yet + pre-commit-skip = false +---- + +[[command-line-api]] +=== Command-line API + +Users should be able to view, run, reorder, and create hook commands via the +command line. External tools should be able to view a list of hooks in the +correct order to run. Modifier commands (`edit` and `add`) have not been +implemented yet and may not be if manually editing the config proves usable +enough. + +*`git hook list `* + +*`git hook run [-a ]... [-e ]...`* + +*`git hook edit `* + +*`git hook add `* + +[[hook-editor]] +=== Hook editor + +The tool which is presented by `git hook edit `. Ideally, this +tool should be easier to use than manually editing the config, and then produce +a concise config afterwards. It may take a form similar to `git rebase +--interactive`. This has not been designed or implemented yet and may not be if +the config proves usable enough. + +[[implementation]] +== Implementation + +[[library]] +=== Library + +`hook.c` and `hook.h` are responsible for interacting with the config files. The +hook library provides a basic API to call all hooks in config order with more +complex options passed via `struct run_hooks_opt`: + +*`int run_hooks(const char *hookname, struct run_hooks_opt *options)`* + +`struct run_hooks_opt` allows callers to set: + +- environment variables +- command-line arguments +- behavior for the hook command provided by `run-command.h:find_hook()` (see + below) +- a method to provide stdin to each hook, either via a file containing stdin, a + `struct string_list` containing a list of lines to print, or a callback + function to allow the caller to populate stdin manually +- a method to process stdout from each hook, e.g. for printing to sideband + during a network operation +- parallelism +- a custom working directory for hooks to execute in + +And this struct can be extended with more options as necessary in the future. + +The "legacy" hook provided by `run-command.h:find_hook()` - that is, the hook +present in `.git/hooks/` or +`$(git config --get core.hooksPath)/` - can be handled in a number of +ways, providing an avenue to deprecate these "legacy" hooks if desired. The +handling is based on a config `hook.runHookDir`, which is checked against a +number of cases: + +- "no": the legacy hook will not be run +- "interactive": Git will prompt the user before running the legacy hook +- "warn": Git will print a warning to stderr before running the legacy hook +- "yes" (default): Git will silently run the legacy hook + +In case this list is expanded in the future, if a value for `hook.runHookDir` is +given which Git does not recognize, Git should discard that config entry. For +example, if "warn" was specified at system level and "junk" was specified at +global level, Git would resolve the value to "warn"; if the only time the config +was set was to "junk", Git would use the default value of "yes". + +`struct hookcmd` is expected to grow in size over time as more functionality is +added to hooks; so that other parts of the code don't need to understand the +config schema, `struct hookcmd` should contain logical values instead of string +pairs. + +By default, hook parallelism is chosen based on the semantics of each hook; +callsites initialize their `struct run_hooks_opt` via one of two macros, +`RUN_HOOKS_OPT_INIT_SYNC` or `RUN_HOOKS_OPT_INIT_ASYNC`. The default number of +jobs can be configured in `hook.jobs`; this config applies across all hook +events. If unset, the value of `online_cpus()` (equivalent to `nproc`) is used. + +[[builtin]] +=== Builtin + +`builtin/hook.c` is responsible for providing the frontend. It's responsible for +formatting user-provided data and then calling the library API to set the +configs as appropriate. The builtin frontend is not responsible for calling the +config directly, so that other areas of Git can rely on the hook library to +understand the most recent config schema for hooks. + +[[migration]] +=== Migration path + +[[stage-0]] +==== Stage 0 + +Hooks are called by running `run-command.h:find_hook()` with the hookname and +executing the result. The hook library and builtin do not exist. Hooks only +exist as specially named scripts within `.git/hooks/`. + +[[stage-1]] +==== Stage 1 + +`git hook list --porcelain ` is implemented. `hook.h:run_hooks()` is +taught to include `run-command.h:find_hook()` at the end; calls to `find_hook()` +are replaced with calls to `run_hooks()`. Users can opt-in to config-based hooks +simply by creating some in their config; otherwise users should remain +unaffected by the change. + +[[stage-2]] +==== Stage 2 + +The call to `find_hook()` inside of `run_hooks()` learns to check for a config, +`hook.runHookDir`. Users can opt into managing their hooks completely via the +config this way. + +[[stage-3]] +==== Stage 3 + +`.git/hooks` is removed from the template and the hook directory is considered +deprecated. To avoid breaking older repos, the default of `hook.runHookDir` is +not changed, and `find_hook()` is not removed. + +[[caveats]] +== Caveats + +[[security]] +=== Security and repo config + +Part of the motivation behind this refactor is to mitigate hooks as an attack +vector;footnote:[https://lore.kernel.org/git/20171002234517.GV19555@aiede.mtv.corp.google.com/] +however, as the design stands, users can still provide hooks in the repo-level +config, which is included when a repo is zipped and sent elsewhere. The +security of the repo-level config is still under discussion; this design +generally assumes the repo-level config is secure, which is not true yet. The +goal is to avoid an overcomplicated design to work around a problem which has +ceased to exist. + +[[ease-of-use]] +=== Ease of use + +The config schema is nontrivial; that's why it's important for the `git hook` +modifier commands to be usable. Contributors with UX expertise are encouraged to +share their suggestions. + +[[alternatives]] +== Alternative approaches + +A previous summary of alternatives exists in the +archives.footnote:[https://lore.kernel.org/git/20191116011125.GG22855@google.com] + +[[status-quo]] +=== Status quo + +Today users can implement multihooks themselves by using a "trampoline script" +as their hook, and pointing that script to a directory or list of other scripts +they wish to run. + +[[hook-directories]] +=== Hook directories + +Other contributors have suggested Git learn about the existence of a directory +such as `.git/hooks/.d` and execute those hooks in alphabetical order. + +[[comparison]] +=== Comparison table + +.Comparison of alternatives +|=== +|Feature |Config-based hooks |Hook directories |Status quo + +|Supports multiple hooks +|Natively +|Natively +|With user effort + +|Safer for zipped repos +|A little +|No +|No + +|Previous hooks just work +|If configured +|Yes +|Yes + +|Can install one hook to many repos +|Yes +|No +|No + +|Discoverability +|Better (in `git help git`) +|Same as before +|Same as before + +|Hard to run unexpected hook +|If configured +|No +|No +|=== + +[[future-work]] +== Future work + +[[execution-ordering]] +=== Execution ordering + +We may find that config order is insufficient for some users; for example, +config order makes it difficult to add a new hook to the system or global config +which runs at the end of the hook list. A new ordering schema should be: + +1) Specified by a `hook.order` config, so that users will not unexpectedly see +their order change; + +2) Either dependency or numerically based. + +Dependency-based ordering is prone to classic linked-list problems, like a +cycles and handling of missing dependencies. But, it paves the way for enabling +parallelization if some tasks truly depend on others. + +Numerical ordering makes it tricky for Git to generate suggested ordering +numbers for each command, but is easy to determine a definitive order. + +[[parallelization]] +=== Parallelization with dependencies + +Currently hooks use a naive parallelization scheme or are run in series. But if +one hook depends on another's output, then users will want to specify those +dependencies. If we decide to solve this problem, we may want to look to modern +build systems for inspiration on how to manage dependencies and parallel tasks. + +[[securing-hookdir-hooks]] +=== Securing hookdir hooks + +With the design as written in this doc, it's still possible for a malicious user +to modify `.git/config` to include `hook.pre-receive.command = rm -rf /`, then +zip their repo and send it to another user. It may be necessary to teach Git to +only allow inlined hooks like this if they were configured outside of the local +scope (in other words, only run hookcmds, and only allow hookcmds to be +configured in global or system scope); or another approach, like a list of safe +projects, might be useful. It may also be sufficient (or at least useful) to +teach a `hook.disableAll` config or similar flag to the Git executable. + +[[submodule-inheritance]] +=== Submodule inheritance + +It's possible some submodules may want to run the identical set of hooks that +their superrepo runs. While a globally-configured hook set is helpful, it's not +a great solution for users who have multiple repos-with-submodules under the +same user. It would be useful for submodules to learn how to run hooks from +their superrepo's config, or inherit that hook setting. + +[[per-hook-event-settings]] +=== Per-hook-event settings + +It might be desirable to keep settings specifically for some hook events, but +not for others - for example, a user may wish to disable hookdir hooks for all +events but pre-commit, which they haven't had time to convert yet; or, a user +may wish for execution order settings to differ based on hook event. In that +case, it would be useful to set something like `hook.pre-commit.executionOrder` +which would not apply to the 'prepare-commit-msg' hook, for example. + +[[glossary]] +== Glossary + +*hook event* + +A point during Git's execution where user scripts may be run, for example, +_prepare-commit-msg_ or _pre-push_. + +*hook command* + +A user script or executable which will be run on one or more hook events. From patchwork Tue Dec 22 00:02:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 11985421 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 82121C433E6 for ; Tue, 22 Dec 2020 00:03:10 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 52E4322525 for ; Tue, 22 Dec 2020 00:03:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726209AbgLVADJ (ORCPT ); Mon, 21 Dec 2020 19:03:09 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47842 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725844AbgLVADJ (ORCPT ); Mon, 21 Dec 2020 19:03:09 -0500 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C995FC061793 for ; Mon, 21 Dec 2020 16:02:28 -0800 (PST) Received: by mail-yb1-xb4a.google.com with SMTP id a206so15773624ybg.0 for ; Mon, 21 Dec 2020 16:02:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=isjX3r5+stGttIjq0LNRR8TugCAIuFE1YPQWpSzFGCw=; b=GbS+1WoVosn4KcK5M1WiRuSuqFxklMwj5JrJmK2TRXQTPB2Amh5X0fXVpUQwTMg3Sc /R9fcwufaAXXzaPW23adrRztLiEHNU9qMTtFOyf8z6cl/cSc0M4mh6rfLE6LR3L7kDZn Ac+N7fuqxB0fN7Ca0zcb8ngoA4Rdxdn0qCMQgnsipjPhHzTkvnqcflRCgmW7768ng8YQ AIfv8fNhdStxT+YMZhKu5PTibuvZK49nigmGfLjW2nhYO6ns5iew41YR5QQxwuMA8ieM vVlGhSqXyfPqZZTtARa0rYx6v0DavqT1iczElTVshR0WoW54EFDzE4eignkvLT6cEYM2 7mEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=isjX3r5+stGttIjq0LNRR8TugCAIuFE1YPQWpSzFGCw=; b=AKELtkdL30GWX7UX2lw/XCdwGel4NXOqnlBLhMSWWYnTXw8TuXuTnCW4oHj1E61XQp zJmqsENK22vMOB7qdSw+5y0tY2FwuniLKvgdqKriB7GFCKtEpFk4md1UZz/qu5O5UQ1v DJCeypw6fvyRbi7eAGow1/LbnJo0TFOs3/o8tXMdV3DAxseXl3ns1/miR1sBXm+lZQ7A rL9r6Put6NW7rfIVHUP/gm9yto1WsZuSE1YT43PoN8uLSmT1RRj6+dedUp+/gcBzM1TC j37Go04cknnI7FoO5Lbfcct2Eom6b7f1IxP1OGwOu+J8nUCBgukNQ6wTii7PFEB5JxxW H5Sw== X-Gm-Message-State: AOAM531YeNMH8MD6m7aD+UZtQ1WO96wVEwQbMvkn9zdIyJ3dUeZfOyGb LvI896JqAV+sm7TGkSSRqlI16o4+EdITrBGyfSuqG0Xbany2a5dLHYTYiPiIrubcaUW/K/nwx/v RX/hh3v2geahH496CmV3J0lGkgijj4SNP74hpnefqvoiXRKOSOpJk5q48YGycJ7PocOQo8cX6wA == X-Google-Smtp-Source: ABdhPJw5cY9sIIb7dVt9jYQNYboFNHwSzSoubdEC0NqI4n9qdsKmI3cf5O753D2JfhUPJ5Zg9E/jpk9iW0OKmkvXnt0= Sender: "emilyshaffer via sendgmr" X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:1ea0:b8ff:fe77:f690]) (user=emilyshaffer job=sendgmr) by 2002:a25:ab74:: with SMTP id u107mr25186356ybi.44.1608595348002; Mon, 21 Dec 2020 16:02:28 -0800 (PST) Date: Mon, 21 Dec 2020 16:02:05 -0800 In-Reply-To: <20201222000220.1491091-1-emilyshaffer@google.com> Message-Id: <20201222000220.1491091-3-emilyshaffer@google.com> Mime-Version: 1.0 References: <20201222000220.1491091-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.29.2.490.gc7ae633391 Subject: [PATCH v7 02/17] hook: scaffolding for git-hook subcommand From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Introduce infrastructure for a new subcommand, git-hook, which will be used to ease config-based hook management. This command will handle parsing configs to compose a list of hooks to run for a given event, as well as adding or modifying hook configs in an interactive fashion. Signed-off-by: Emily Shaffer --- Notes: Since v4, mainly changed to RUN_SETUP_GENTLY so that 'git hook list' can be executed outside of a repo. .gitignore | 1 + Documentation/git-hook.txt | 20 ++++++++++++++++++++ Makefile | 1 + builtin.h | 1 + builtin/hook.c | 21 +++++++++++++++++++++ command-list.txt | 1 + git.c | 1 + t/t1360-config-based-hooks.sh | 11 +++++++++++ 8 files changed, 57 insertions(+) create mode 100644 Documentation/git-hook.txt create mode 100644 builtin/hook.c create mode 100755 t/t1360-config-based-hooks.sh diff --git a/.gitignore b/.gitignore index 3dcdb6bb5a..3608c35b73 100644 --- a/.gitignore +++ b/.gitignore @@ -76,6 +76,7 @@ /git-grep /git-hash-object /git-help +/git-hook /git-http-backend /git-http-fetch /git-http-push diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt new file mode 100644 index 0000000000..9eeab0009d --- /dev/null +++ b/Documentation/git-hook.txt @@ -0,0 +1,20 @@ +git-hook(1) +=========== + +NAME +---- +git-hook - Manage configured hooks + +SYNOPSIS +-------- +[verse] +'git hook' + +DESCRIPTION +----------- +A placeholder command. Later, you will be able to list, add, and modify hooks +with this command. + +GIT +--- +Part of the linkgit:git[1] suite diff --git a/Makefile b/Makefile index 6fb86c5862..24cee44400 100644 --- a/Makefile +++ b/Makefile @@ -1101,6 +1101,7 @@ BUILTIN_OBJS += builtin/get-tar-commit-id.o BUILTIN_OBJS += builtin/grep.o BUILTIN_OBJS += builtin/hash-object.o BUILTIN_OBJS += builtin/help.o +BUILTIN_OBJS += builtin/hook.o BUILTIN_OBJS += builtin/index-pack.o BUILTIN_OBJS += builtin/init-db.o BUILTIN_OBJS += builtin/interpret-trailers.o diff --git a/builtin.h b/builtin.h index b6ce981b73..8df1d36a7a 100644 --- a/builtin.h +++ b/builtin.h @@ -163,6 +163,7 @@ int cmd_get_tar_commit_id(int argc, const char **argv, const char *prefix); int cmd_grep(int argc, const char **argv, const char *prefix); int cmd_hash_object(int argc, const char **argv, const char *prefix); int cmd_help(int argc, const char **argv, const char *prefix); +int cmd_hook(int argc, const char **argv, const char *prefix); int cmd_index_pack(int argc, const char **argv, const char *prefix); int cmd_init_db(int argc, const char **argv, const char *prefix); int cmd_interpret_trailers(int argc, const char **argv, const char *prefix); diff --git a/builtin/hook.c b/builtin/hook.c new file mode 100644 index 0000000000..b2bbc84d4d --- /dev/null +++ b/builtin/hook.c @@ -0,0 +1,21 @@ +#include "cache.h" + +#include "builtin.h" +#include "parse-options.h" + +static const char * const builtin_hook_usage[] = { + N_("git hook"), + NULL +}; + +int cmd_hook(int argc, const char **argv, const char *prefix) +{ + struct option builtin_hook_options[] = { + OPT_END(), + }; + + argc = parse_options(argc, argv, prefix, builtin_hook_options, + builtin_hook_usage, 0); + + return 0; +} diff --git a/command-list.txt b/command-list.txt index 9379b02e5e..75909bf602 100644 --- a/command-list.txt +++ b/command-list.txt @@ -103,6 +103,7 @@ git-grep mainporcelain info git-gui mainporcelain git-hash-object plumbingmanipulators git-help ancillaryinterrogators complete +git-hook mainporcelain git-http-backend synchingrepositories git-http-fetch synchelpers git-http-push synchelpers diff --git a/git.c b/git.c index a00a0a4d94..9d1768b8e8 100644 --- a/git.c +++ b/git.c @@ -525,6 +525,7 @@ static struct cmd_struct commands[] = { { "grep", cmd_grep, RUN_SETUP_GENTLY }, { "hash-object", cmd_hash_object }, { "help", cmd_help }, + { "hook", cmd_hook, RUN_SETUP_GENTLY }, { "index-pack", cmd_index_pack, RUN_SETUP_GENTLY | NO_PARSEOPT }, { "init", cmd_init_db }, { "init-db", cmd_init_db }, diff --git a/t/t1360-config-based-hooks.sh b/t/t1360-config-based-hooks.sh new file mode 100755 index 0000000000..34b0df5216 --- /dev/null +++ b/t/t1360-config-based-hooks.sh @@ -0,0 +1,11 @@ +#!/bin/bash + +test_description='config-managed multihooks, including git-hook command' + +. ./test-lib.sh + +test_expect_success 'git hook command does not crash' ' + git hook +' + +test_done From patchwork Tue Dec 22 00:02:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 11985423 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 41ADFC433DB for ; Tue, 22 Dec 2020 00:03:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 03F2922525 for ; Tue, 22 Dec 2020 00:03:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726351AbgLVADL (ORCPT ); Mon, 21 Dec 2020 19:03:11 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47848 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725844AbgLVADK (ORCPT ); Mon, 21 Dec 2020 19:03:10 -0500 Received: from mail-pg1-x54a.google.com (mail-pg1-x54a.google.com [IPv6:2607:f8b0:4864:20::54a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8584EC061282 for ; Mon, 21 Dec 2020 16:02:30 -0800 (PST) Received: by mail-pg1-x54a.google.com with SMTP id 26so7473085pgl.2 for ; Mon, 21 Dec 2020 16:02:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=E63ELr0gZbKJ9atizmHStTPOQsdrLPSX1yQ9VPIzHGQ=; b=gVAbPADcJ7eA4JeRLeR5u5/rcPJ7b0Mr2yWk+2dca3eCEKWztL0uovz3dU/xXJv9Ay iTJ4UqSz2OcOrPCL/0xa9mIGvPv/rZFS9Mjjm8Bm0cckEwfbs31mvnm7FEuFxtAYCPKq YCaFU/WL+aBAXrwmQp6aymmCLkhsH3maN/rnlMy2TpPNeu8AJqKwlyRu1O/QMa9iBp8c Oz+07zNoUQXwLtz6M69Dqgrw02LNj9mr+ptFgcsCa1SeekK2NC4qOt0BMia96rs8ydc7 0afKe8fqOZbSC5f7czOGXw5E7u0mNOcoJ6ONyuXY/k6n803EP3r1oBjAZT/07NKOPfrY I3LA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=E63ELr0gZbKJ9atizmHStTPOQsdrLPSX1yQ9VPIzHGQ=; b=d5UBAnIbH1oa2AmUrgO5Z8kzFKtJzBmJeT99rEozHMMfO7ZW0BDDmWR8BbX0cpVH0S Pax7sWL9ba2LeYLCStARaV9+PZj5lLkiegnV7/BIr/aIgWPmuEct9Qu7MkO4FaJUAw4J ua7qmRMXAT55jqx37n3ih66P9KikYmvwAOaGDimoEnZzHwAw19R6JJ4BjKSh4E0WCN83 1IiLHAtPC+XD3UivEw8o0dWLRrqdIXnA1bMtUAkoP2AVaPGSlVeXrEK7M/zkM9HCxw53 5p/atqtsFFTIBOLa1uezV0I06D3JfXQLbuMsCIDXPHLOj+j/dwUX/nEASnCvOpNQ8keB jMJQ== X-Gm-Message-State: AOAM531DbY39g/JLDCH6a2FSadNI9scYwpVOpZcU62GyAvbgFuOk2SFc CkpnGenEHWGSroMgZLjjEMY2HZRe+HZviZ2g9PUKrxZo6YQlqgSBslIaOnoBqNSoazq+hYd4Pb0 2EMiPzSG6UEkdB0S1J73JSCUvfjILLwS/Rr+jYbdBzL06N/ZAdCFQULM4ZpyngOz/ZKi3ut7AwQ == X-Google-Smtp-Source: ABdhPJw8oPKeGANVMDAhPmRnDIjd0DOXI8J4wck9+h5u4IBiyzdXbIZYIMYcEZ/OzcEQdikl12JszuEbXH1tIcOLEio= Sender: "emilyshaffer via sendgmr" X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:1ea0:b8ff:fe77:f690]) (user=emilyshaffer job=sendgmr) by 2002:a17:902:ba84:b029:dc:f27:dd4e with SMTP id k4-20020a170902ba84b02900dc0f27dd4emr18226280pls.61.1608595349973; Mon, 21 Dec 2020 16:02:29 -0800 (PST) Date: Mon, 21 Dec 2020 16:02:06 -0800 In-Reply-To: <20201222000220.1491091-1-emilyshaffer@google.com> Message-Id: <20201222000220.1491091-4-emilyshaffer@google.com> Mime-Version: 1.0 References: <20201222000220.1491091-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.29.2.490.gc7ae633391 Subject: [PATCH v7 03/17] hook: add list command From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Teach 'git hook list ', which checks the known configs in order to create an ordered list of hooks to run on a given hook event. Multiple commands can be specified for a given hook by providing multiple "hook..command = " lines. Hooks will be run in config order. If more properties need to be set on a given hook in the future, commands can also be specified by providing "hook..command = ", as well as a "[hookcmd ]" subsection; at minimum, this subsection must contain a "hookcmd..command = " line. For example: $ git config --list | grep ^hook hook.pre-commit.command=baz hook.pre-commit.command=~/bar.sh hookcmd.baz.command=~/baz/from/hookcmd.sh $ git hook list pre-commit global: ~/baz/from/hookcmd.sh local: ~/bar.sh Signed-off-by: Emily Shaffer --- Notes: Since v4, updated the sample in the commit message to reflect reality better. Since v4, more work on the documentation. Also a slight change to the output format (space instead of tab). Documentation/config/hook.txt | 9 +++ Documentation/git-hook.txt | 59 ++++++++++++++++- Makefile | 1 + builtin/hook.c | 56 +++++++++++++++-- hook.c | 115 ++++++++++++++++++++++++++++++++++ hook.h | 26 ++++++++ t/t1360-config-based-hooks.sh | 81 +++++++++++++++++++++++- 7 files changed, 338 insertions(+), 9 deletions(-) create mode 100644 Documentation/config/hook.txt create mode 100644 hook.c create mode 100644 hook.h diff --git a/Documentation/config/hook.txt b/Documentation/config/hook.txt new file mode 100644 index 0000000000..71449ecbc7 --- /dev/null +++ b/Documentation/config/hook.txt @@ -0,0 +1,9 @@ +hook..command:: + A command to execute during the hook event. This can be an + executable on your device, a oneliner for your shell, or the name of a + hookcmd. See linkgit:git-hook[1]. + +hookcmd..command:: + A command to execute during a hook for which has been specified + as a command. This can be an executable on your device or a oneliner for + your shell. See linkgit:git-hook[1]. diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt index 9eeab0009d..f19875ed68 100644 --- a/Documentation/git-hook.txt +++ b/Documentation/git-hook.txt @@ -8,12 +8,65 @@ git-hook - Manage configured hooks SYNOPSIS -------- [verse] -'git hook' +'git hook' list DESCRIPTION ----------- -A placeholder command. Later, you will be able to list, add, and modify hooks -with this command. +You can list configured hooks with this command. Later, you will be able to run, +add, and modify hooks with this command. + +This command parses the default configuration files for sections `hook` and +`hookcmd`. `hook` is used to describe the commands which will be run during a +particular hook event; commands are run in the order Git encounters them during +the configuration parse (see linkgit:git-config[1]). `hookcmd` is used to +describe attributes of a specific command. If additional attributes don't need +to be specified, a command to run can be specified directly in the `hook` +section; if a `hookcmd` by that name isn't found, Git will attempt to run the +provided value directly. For example: + +Global config +---- + [hook "post-commit"] + command = "linter" + command = "~/typocheck.sh" + + [hookcmd "linter"] + command = "/bin/linter --c" +---- + +Local config +---- + [hook "prepare-commit-msg"] + command = "linter" + [hook "post-commit"] + command = "python ~/run-test-suite.py" +---- + +With these configs, you'd then see: + +---- +$ git hook list "post-commit" +global: /bin/linter --c +global: ~/typocheck.sh +local: python ~/run-test-suite.py + +$ git hook list "prepare-commit-msg" +local: /bin/linter --c +---- + +COMMANDS +-------- + +list ``:: + +List the hooks which have been configured for ``. Hooks appear +in the order they should be run, and print the config scope where the relevant +`hook..command` was specified, not the `hookcmd` (if applicable). +This output is human-readable and the format is subject to change over time. + +CONFIGURATION +------------- +include::config/hook.txt[] GIT --- diff --git a/Makefile b/Makefile index 24cee44400..d9f43dc8fe 100644 --- a/Makefile +++ b/Makefile @@ -904,6 +904,7 @@ LIB_OBJS += grep.o LIB_OBJS += hashmap.o LIB_OBJS += help.o LIB_OBJS += hex.o +LIB_OBJS += hook.o LIB_OBJS += ident.o LIB_OBJS += json-writer.o LIB_OBJS += kwset.o diff --git a/builtin/hook.c b/builtin/hook.c index b2bbc84d4d..4d36de52f8 100644 --- a/builtin/hook.c +++ b/builtin/hook.c @@ -1,21 +1,69 @@ #include "cache.h" #include "builtin.h" +#include "config.h" +#include "hook.h" #include "parse-options.h" +#include "strbuf.h" static const char * const builtin_hook_usage[] = { - N_("git hook"), + N_("git hook list "), NULL }; -int cmd_hook(int argc, const char **argv, const char *prefix) +static int list(int argc, const char **argv, const char *prefix) { - struct option builtin_hook_options[] = { + struct list_head *head, *pos; + struct hook *item; + struct strbuf hookname = STRBUF_INIT; + + struct option list_options[] = { OPT_END(), }; - argc = parse_options(argc, argv, prefix, builtin_hook_options, + argc = parse_options(argc, argv, prefix, list_options, builtin_hook_usage, 0); + if (argc < 1) { + usage_msg_opt(_("You must specify a hook event name to list."), + builtin_hook_usage, list_options); + } + + strbuf_addstr(&hookname, argv[0]); + + head = hook_list(&hookname); + + if (list_empty(head)) { + printf(_("no commands configured for hook '%s'\n"), + hookname.buf); + strbuf_release(&hookname); + return 0; + } + + list_for_each(pos, head) { + item = list_entry(pos, struct hook, list); + if (item) + printf("%s: %s\n", + config_scope_name(item->origin), + item->command.buf); + } + + clear_hook_list(head); + strbuf_release(&hookname); + return 0; } + +int cmd_hook(int argc, const char **argv, const char *prefix) +{ + struct option builtin_hook_options[] = { + OPT_END(), + }; + if (argc < 2) + usage_with_options(builtin_hook_usage, builtin_hook_options); + + if (!strcmp(argv[1], "list")) + return list(argc - 1, argv + 1, prefix); + + usage_with_options(builtin_hook_usage, builtin_hook_options); +} diff --git a/hook.c b/hook.c new file mode 100644 index 0000000000..937dc768c8 --- /dev/null +++ b/hook.c @@ -0,0 +1,115 @@ +#include "cache.h" + +#include "hook.h" +#include "config.h" + +void free_hook(struct hook *ptr) +{ + if (ptr) { + strbuf_release(&ptr->command); + free(ptr); + } +} + +static void append_or_move_hook(struct list_head *head, const char *command) +{ + struct list_head *pos = NULL, *tmp = NULL; + struct hook *to_add = NULL; + + /* + * remove the prior entry with this command; we'll replace it at the + * end. + */ + list_for_each_safe(pos, tmp, head) { + struct hook *it = list_entry(pos, struct hook, list); + if (!strcmp(it->command.buf, command)) { + list_del(pos); + /* we'll simply move the hook to the end */ + to_add = it; + } + } + + if (!to_add) { + /* adding a new hook, not moving an old one */ + to_add = xmalloc(sizeof(struct hook)); + strbuf_init(&to_add->command, 0); + strbuf_addstr(&to_add->command, command); + } + + /* re-set the scope so we show where an override was specified */ + to_add->origin = current_config_scope(); + + list_add_tail(&to_add->list, pos); +} + +static void remove_hook(struct list_head *to_remove) +{ + struct hook *hook_to_remove = list_entry(to_remove, struct hook, list); + list_del(to_remove); + free_hook(hook_to_remove); +} + +void clear_hook_list(struct list_head *head) +{ + struct list_head *pos, *tmp; + list_for_each_safe(pos, tmp, head) + remove_hook(pos); +} + +struct hook_config_cb +{ + struct strbuf *hookname; + struct list_head *list; +}; + +static int hook_config_lookup(const char *key, const char *value, void *cb_data) +{ + struct hook_config_cb *data = cb_data; + const char *hook_key = data->hookname->buf; + struct list_head *head = data->list; + + if (!strcmp(key, hook_key)) { + const char *command = value; + struct strbuf hookcmd_name = STRBUF_INIT; + + /* Check if a hookcmd with that name exists. */ + strbuf_addf(&hookcmd_name, "hookcmd.%s.command", command); + git_config_get_value(hookcmd_name.buf, &command); + + if (!command) { + strbuf_release(&hookcmd_name); + BUG("git_config_get_value overwrote a string it shouldn't have"); + } + + /* + * TODO: implement an option-getting callback, e.g. + * get configs by pattern hookcmd.$value.* + * for each key+value, do_callback(key, value, cb_data) + */ + + append_or_move_hook(head, command); + + strbuf_release(&hookcmd_name); + } + + return 0; +} + +struct list_head* hook_list(const struct strbuf* hookname) +{ + struct strbuf hook_key = STRBUF_INIT; + struct list_head *hook_head = xmalloc(sizeof(struct list_head)); + struct hook_config_cb cb_data = { &hook_key, hook_head }; + + INIT_LIST_HEAD(hook_head); + + if (!hookname) + return NULL; + + strbuf_addf(&hook_key, "hook.%s.command", hookname->buf); + + git_config(hook_config_lookup, (void*)&cb_data); + + strbuf_release(&hook_key); + return hook_head; +} diff --git a/hook.h b/hook.h new file mode 100644 index 0000000000..8ffc4f14b6 --- /dev/null +++ b/hook.h @@ -0,0 +1,26 @@ +#include "config.h" +#include "list.h" +#include "strbuf.h" + +struct hook +{ + struct list_head list; + /* + * Config file which holds the hook.*.command definition. + * (This has nothing to do with the hookcmd..* configs.) + */ + enum config_scope origin; + /* The literal command to run. */ + struct strbuf command; +}; + +/* + * Provides a linked list of 'struct hook' detailing commands which should run + * in response to the 'hookname' event, in execution order. + */ +struct list_head* hook_list(const struct strbuf *hookname); + +/* Free memory associated with a 'struct hook' */ +void free_hook(struct hook *ptr); +/* Empties the list at 'head', calling 'free_hook()' on each entry */ +void clear_hook_list(struct list_head *head); diff --git a/t/t1360-config-based-hooks.sh b/t/t1360-config-based-hooks.sh index 34b0df5216..6e4a3e763f 100755 --- a/t/t1360-config-based-hooks.sh +++ b/t/t1360-config-based-hooks.sh @@ -4,8 +4,85 @@ test_description='config-managed multihooks, including git-hook command' . ./test-lib.sh -test_expect_success 'git hook command does not crash' ' - git hook +ROOT= +if test_have_prereq MINGW +then + # In Git for Windows, Unix-like paths work only in shell scripts; + # `git.exe`, however, will prefix them with the pseudo root directory + # (of the Unix shell). Let's accommodate for that. + ROOT="$(cd / && pwd)" +fi + +setup_hooks () { + test_config hook.pre-commit.command "/path/ghi" --add + test_config_global hook.pre-commit.command "/path/def" --add +} + +setup_hookcmd () { + test_config hook.pre-commit.command "abc" --add + test_config_global hookcmd.abc.command "/path/abc" --add +} + +test_expect_success 'git hook rejects commands without a mode' ' + test_must_fail git hook pre-commit +' + + +test_expect_success 'git hook rejects commands without a hookname' ' + test_must_fail git hook list +' + +test_expect_success 'git hook runs outside of a repo' ' + setup_hooks && + + cat >expected <<-EOF && + global: $ROOT/path/def + EOF + + nongit git config --list --global && + + nongit git hook list pre-commit >actual && + test_cmp expected actual +' + +test_expect_success 'git hook list orders by config order' ' + setup_hooks && + + cat >expected <<-EOF && + global: $ROOT/path/def + local: $ROOT/path/ghi + EOF + + git hook list pre-commit >actual && + test_cmp expected actual +' + +test_expect_success 'git hook list dereferences a hookcmd' ' + setup_hooks && + setup_hookcmd && + + cat >expected <<-EOF && + global: $ROOT/path/def + local: $ROOT/path/ghi + local: $ROOT/path/abc + EOF + + git hook list pre-commit >actual && + test_cmp expected actual +' + +test_expect_success 'git hook list reorders on duplicate commands' ' + setup_hooks && + + test_config hook.pre-commit.command "/path/def" --add && + + cat >expected <<-EOF && + local: $ROOT/path/ghi + local: $ROOT/path/def + EOF + + git hook list pre-commit >actual && + test_cmp expected actual ' test_done From patchwork Tue Dec 22 00:02:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 11985425 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EE2BDC433E6 for ; Tue, 22 Dec 2020 00:03:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id ACAEA22525 for ; Tue, 22 Dec 2020 00:03:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726365AbgLVADN (ORCPT ); Mon, 21 Dec 2020 19:03:13 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47856 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725844AbgLVADM (ORCPT ); Mon, 21 Dec 2020 19:03:12 -0500 Received: from mail-pg1-x54a.google.com (mail-pg1-x54a.google.com [IPv6:2607:f8b0:4864:20::54a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 72F20C061285 for ; Mon, 21 Dec 2020 16:02:32 -0800 (PST) Received: by mail-pg1-x54a.google.com with SMTP id w4so7493745pgc.7 for ; Mon, 21 Dec 2020 16:02:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=c/vmugnHQwHPjeFKAYrWXfuwZcwZYECjyB5o35wtJbI=; b=sfWF1oPnFkGE2rClcPfaSVBzY8RCItf2op4kRQcU83GWzSkkaKcEjJL0mLpV66N/zd Oq0RouJ8HTsbaHIezPiyHSPn75LhnBDphnr7zwTJghovvxu18NPOScKJJX7bZITSWzik vkg+TZnD510r8YsP2Vlc6lGMFe6LcQ69Ybh9pd5Hx79eeaW6Dy+xgxi+pgCDqjmZobmv qM9QBYiQn3JYRgWwIAn60kZwa4+4gpqN1JbbFg9beTCA739Xw4XlCllB891HkU6Qpsjv JYUsi3EF3N+CEvUWm99VCnQpjVUjE7xlAP4+KDJP0uSKsyQnGvT6tXtUjo2W0dGkDLpf ZbnQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=c/vmugnHQwHPjeFKAYrWXfuwZcwZYECjyB5o35wtJbI=; b=lvQb+NrpHKeWFGBm1sdZAU+9lfJ0pQaCocSCWKtcJ8wmTdta9goIlzkCCNU8UFzFGR FErHPB12QAgAwIdnBHF19UBpEpVWinSMqM+CTzI0MHWSe5VPW2xqwbDclK4KPNmKBwsw 7M3bzbwEGqi/aOQDuaBreLP31jXthjJ8nKYk1jRWX9z15xVVA5D5zDItoiZkA0bWvUTD iiHBVo4bqj0zblLAVha2qF7QP5JwIxUVXRhTIMKOr9JxorPSZT83h0Yn00BTLGJxN10h PJ06UjTU78NoWWkmQP4wonuKqXUn0bhIJGuzqnVLQTQJO1/7U6WJ0EAUxaCi9X4fk+wH TPoQ== X-Gm-Message-State: AOAM533ooaH5RGJTyf/JrnH7GJ642zYR/VdDXsjqcd8o8C/usTFJcjLL UIgO0FUxW0iUoz6EDc3U8tEUT4ZKbp1AoL8V1ZjYF9FFq2UThp95k6holvMkR7alFkpUWKW5FlQ uQqprdkyzU4b/tQj4d5lCxKSSOaI9n8wFcdi0PL4GPbc0vCAhbXCEXoiPH+NPuXQsumZQs7SfXg == X-Google-Smtp-Source: ABdhPJxJwWOojbDCzZc/oXe4MXqVkXABiSVN6F3dXkqg/rwnZHGlNl61NAzAg7QOYAzLq7nG8TSPiA+jnGc50NefudE= Sender: "emilyshaffer via sendgmr" X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:1ea0:b8ff:fe77:f690]) (user=emilyshaffer job=sendgmr) by 2002:a62:ae0c:0:b029:1a5:819d:9ac5 with SMTP id q12-20020a62ae0c0000b02901a5819d9ac5mr17318189pff.26.1608595351862; Mon, 21 Dec 2020 16:02:31 -0800 (PST) Date: Mon, 21 Dec 2020 16:02:07 -0800 In-Reply-To: <20201222000220.1491091-1-emilyshaffer@google.com> Message-Id: <20201222000220.1491091-5-emilyshaffer@google.com> Mime-Version: 1.0 References: <20201222000220.1491091-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.29.2.490.gc7ae633391 Subject: [PATCH v7 04/17] hook: include hookdir hook in list From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Historically, hooks are declared by placing an executable into $GIT_DIR/hooks/$HOOKNAME (or $HOOKDIR/$HOOKNAME). Although hooks taken from the config are more featureful than hooks placed in the $HOOKDIR, those hooks should not stop working for users who already have them. Legacy hooks should be run directly, not in shell. We know that they are a path to an executable, not a oneliner script - and running them directly takes care of path quoting concerns for us for free. Signed-off-by: Emily Shaffer --- Notes: Newly split into its own commit since v4, and taking place much sooner. An unfortunate side effect of adding this support *before* the hook.runHookDir support is that the labels on the list are not clear - because we aren't yet flagging which hooks are from the hookdir versus the config. I suppose we could move the addition of that field to the struct hook up to this patch, but it didn't make a lot of sense to me to do it just for cosmetic purposes. builtin/hook.c | 18 ++++++++++++++---- hook.c | 15 +++++++++++++++ hook.h | 1 + t/t1360-config-based-hooks.sh | 19 +++++++++++++++++++ 4 files changed, 49 insertions(+), 4 deletions(-) diff --git a/builtin/hook.c b/builtin/hook.c index 4d36de52f8..a0013ae4d7 100644 --- a/builtin/hook.c +++ b/builtin/hook.c @@ -16,6 +16,7 @@ static int list(int argc, const char **argv, const char *prefix) struct list_head *head, *pos; struct hook *item; struct strbuf hookname = STRBUF_INIT; + struct strbuf hookdir_annotation = STRBUF_INIT; struct option list_options[] = { OPT_END(), @@ -42,10 +43,17 @@ static int list(int argc, const char **argv, const char *prefix) list_for_each(pos, head) { item = list_entry(pos, struct hook, list); - if (item) - printf("%s: %s\n", - config_scope_name(item->origin), - item->command.buf); + if (item) { + /* Don't translate 'hookdir' - it matches the config */ + printf("%s: %s%s\n", + (item->from_hookdir + ? "hookdir" + : config_scope_name(item->origin)), + item->command.buf, + (item->from_hookdir + ? hookdir_annotation.buf + : "")); + } } clear_hook_list(head); @@ -62,6 +70,8 @@ int cmd_hook(int argc, const char **argv, const char *prefix) if (argc < 2) usage_with_options(builtin_hook_usage, builtin_hook_options); + git_config(git_default_config, NULL); + if (!strcmp(argv[1], "list")) return list(argc - 1, argv + 1, prefix); diff --git a/hook.c b/hook.c index 937dc768c8..ffbdcfd987 100644 --- a/hook.c +++ b/hook.c @@ -2,6 +2,7 @@ #include "hook.h" #include "config.h" +#include "run-command.h" void free_hook(struct hook *ptr) { @@ -34,6 +35,7 @@ static void append_or_move_hook(struct list_head *head, const char *command) to_add = xmalloc(sizeof(struct hook)); strbuf_init(&to_add->command, 0); strbuf_addstr(&to_add->command, command); + to_add->from_hookdir = 0; } /* re-set the scope so we show where an override was specified */ @@ -100,6 +102,7 @@ struct list_head* hook_list(const struct strbuf* hookname) struct strbuf hook_key = STRBUF_INIT; struct list_head *hook_head = xmalloc(sizeof(struct list_head)); struct hook_config_cb cb_data = { &hook_key, hook_head }; + const char *legacy_hook_path = NULL; INIT_LIST_HEAD(hook_head); @@ -110,6 +113,18 @@ struct list_head* hook_list(const struct strbuf* hookname) git_config(hook_config_lookup, (void*)&cb_data); + if (have_git_dir()) + legacy_hook_path = find_hook(hookname->buf); + + /* Unconditionally add legacy hook, but annotate it. */ + if (legacy_hook_path) { + struct hook *legacy_hook; + + append_or_move_hook(hook_head, absolute_path(legacy_hook_path)); + legacy_hook = list_entry(hook_head->prev, struct hook, list); + legacy_hook->from_hookdir = 1; + } + strbuf_release(&hook_key); return hook_head; } diff --git a/hook.h b/hook.h index 8ffc4f14b6..5750634c83 100644 --- a/hook.h +++ b/hook.h @@ -12,6 +12,7 @@ struct hook enum config_scope origin; /* The literal command to run. */ struct strbuf command; + int from_hookdir; }; /* diff --git a/t/t1360-config-based-hooks.sh b/t/t1360-config-based-hooks.sh index 6e4a3e763f..0f12af4659 100755 --- a/t/t1360-config-based-hooks.sh +++ b/t/t1360-config-based-hooks.sh @@ -23,6 +23,14 @@ setup_hookcmd () { test_config_global hookcmd.abc.command "/path/abc" --add } +setup_hookdir () { + mkdir .git/hooks + write_script .git/hooks/pre-commit <<-EOF + echo \"Legacy Hook\" + EOF + test_when_finished rm -rf .git/hooks +} + test_expect_success 'git hook rejects commands without a mode' ' test_must_fail git hook pre-commit ' @@ -85,4 +93,15 @@ test_expect_success 'git hook list reorders on duplicate commands' ' test_cmp expected actual ' +test_expect_success 'git hook list shows hooks from the hookdir' ' + setup_hookdir && + + cat >expected <<-EOF && + hookdir: $(pwd)/.git/hooks/pre-commit + EOF + + git hook list pre-commit >actual && + test_cmp expected actual +' + test_done From patchwork Tue Dec 22 00:02:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 11985427 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1B511C433E0 for ; Tue, 22 Dec 2020 00:03:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D3D6622AED for ; Tue, 22 Dec 2020 00:03:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726427AbgLVADq (ORCPT ); Mon, 21 Dec 2020 19:03:46 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47940 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725844AbgLVADp (ORCPT ); Mon, 21 Dec 2020 19:03:45 -0500 Received: from mail-pl1-x64a.google.com (mail-pl1-x64a.google.com [IPv6:2607:f8b0:4864:20::64a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 74FD5C061248 for ; Mon, 21 Dec 2020 16:02:34 -0800 (PST) Received: by mail-pl1-x64a.google.com with SMTP id x12so6526590plr.21 for ; Mon, 21 Dec 2020 16:02:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=EmHsBf1+1NjuozJFvJ8OL3GYhDVYbAr1m6xHT89QZNg=; b=Jb5ObFzMMFFTPyxzyYcc753fQt17WKh8mQyycYlkjw5ej11qGk4+ipIyc7NGpfeFo5 +mSLx8jMR+UL3Sn8XMJ6mElAuOOpJQMD1mh04Gm+kLe7hvO7R2Mp1Y/sP8qQLaBCJVy9 //xz0nczQ6Q4E9N8LpJ6IQTLr/Jy4bMy0dJuo/KBOCkYWe7IEf42mOYzHLJA1rwzJKTc v5r4X+NUpVEn7Li3ZcHurzHWdAa530FsKU0M7nVzrVpJRQaVuF7AET8H1pHFa779kRjG JLJHNt7APdoZ+yCBCEHl5vcIyofeORMW3COny2sEf6/lqas3T8hiE/fXc7JZpgtkyGnV kE2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=EmHsBf1+1NjuozJFvJ8OL3GYhDVYbAr1m6xHT89QZNg=; b=K5OIpEGFkHYUt3oY4wDfpqVhBiia6cvfBT7BhwYjf9+y93AZxiRJT+B80dE1kmSZv7 hV23mbue7H141vdWBbABvZoFI6vP+L9/zwoOHwE05oS+0+xCPkHsZXi+q6SOoaREAAK4 vIxw3rU5W1LMZ3vgbH+cSZqmHbWEX5hDSZiaNsd4vCHTg24QZu+DXAYbW1S2lf6GgNPj q0pFwtUTyc51fazV5/gOncMUp7AbzPsq7/ieDbkb3uSEfS01e/WL+XdyAuGJgtmEhuzO invrSlujSM67yf908bdg6r4aUG3WubX4rrekFC8G/kC22OXart21t7QNOGtWhSpS0QD4 ZRxw== X-Gm-Message-State: AOAM532Z6y2S/Pv2zuNObBTzEdsdKN9Xta0jYWzUopGmJ7faZsOaU5qF nfmxd9Qj2tKDKeNrQz7V6LG2kpmr1tV+tcjibnvvcHodurpA5QmWXedFv+8xiZEw+X+DGR3PjQj 4B//zZOLTeG4bdpRxVuKIBhiityowBRP3+d4ddf1jYtDsUGcqXnKBdQ39DBukC6KVHFu/2RT22Q == X-Google-Smtp-Source: ABdhPJxshcycMiziB8jasD+GYfBSrPOglW2kXAm7xArODkvYgKFa5m1IpAKoqVs8EOKzikXnhVOKEtfMQi4oKmzIckM= Sender: "emilyshaffer via sendgmr" X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:1ea0:b8ff:fe77:f690]) (user=emilyshaffer job=sendgmr) by 2002:a17:902:8bc8:b029:dc:36d4:fba8 with SMTP id r8-20020a1709028bc8b02900dc36d4fba8mr11166276plo.82.1608595353882; Mon, 21 Dec 2020 16:02:33 -0800 (PST) Date: Mon, 21 Dec 2020 16:02:08 -0800 In-Reply-To: <20201222000220.1491091-1-emilyshaffer@google.com> Message-Id: <20201222000220.1491091-6-emilyshaffer@google.com> Mime-Version: 1.0 References: <20201222000220.1491091-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.29.2.490.gc7ae633391 Subject: [PATCH v7 05/17] hook: respect hook.runHookDir From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Include hooks specified in the hook directory in the list of hooks to run. These hooks do need to be treated differently from config-specified ones - they do not need to run in a shell, and later on may be disabled or warned about based on a config setting. Because they are at least as local as the local config, we'll run them last - to keep the hook execution order from global to local. Signed-off-by: Emily Shaffer --- Notes: Newly split into its own commit since v4, and taking place much sooner. An unfortunate side effect of adding this support *before* the hook.runHookDir support is that the labels on the list are not clear - because we aren't yet flagging which hooks are from the hookdir versus the config. I suppose we could move the addition of that field to the struct hook up to this patch, but it didn't make a lot of sense to me to do it just for cosmetic purposes. Documentation/config/hook.txt | 5 ++++ builtin/hook.c | 54 +++++++++++++++++++++++++++++++++-- hook.c | 21 ++++++++++++++ hook.h | 15 ++++++++++ t/t1360-config-based-hooks.sh | 43 ++++++++++++++++++++++++++++ 5 files changed, 135 insertions(+), 3 deletions(-) diff --git a/Documentation/config/hook.txt b/Documentation/config/hook.txt index 71449ecbc7..75312754ae 100644 --- a/Documentation/config/hook.txt +++ b/Documentation/config/hook.txt @@ -7,3 +7,8 @@ hookcmd..command:: A command to execute during a hook for which has been specified as a command. This can be an executable on your device or a oneliner for your shell. See linkgit:git-hook[1]. + +hook.runHookDir:: + Controls how hooks contained in your hookdir are executed. Can be any of + "yes", "warn", "interactive", or "no". Defaults to "yes". See + linkgit:git-hook[1] and linkgit:git-config[1] "core.hooksPath"). diff --git a/builtin/hook.c b/builtin/hook.c index a0013ae4d7..d087e6f5b0 100644 --- a/builtin/hook.c +++ b/builtin/hook.c @@ -11,6 +11,8 @@ static const char * const builtin_hook_usage[] = { NULL }; +static enum hookdir_opt should_run_hookdir; + static int list(int argc, const char **argv, const char *prefix) { struct list_head *head, *pos; @@ -41,6 +43,26 @@ static int list(int argc, const char **argv, const char *prefix) return 0; } + switch (should_run_hookdir) { + case HOOKDIR_NO: + strbuf_addstr(&hookdir_annotation, _(" (will not run)")); + break; + case HOOKDIR_INTERACTIVE: + strbuf_addstr(&hookdir_annotation, _(" (will prompt)")); + break; + case HOOKDIR_WARN: + case HOOKDIR_UNKNOWN: + strbuf_addstr(&hookdir_annotation, _(" (will warn)")); + break; + case HOOKDIR_YES: + /* + * The default behavior should agree with + * hook.c:configured_hookdir_opt(). + */ + default: + break; + } + list_for_each(pos, head) { item = list_entry(pos, struct hook, list); if (item) { @@ -64,16 +86,42 @@ static int list(int argc, const char **argv, const char *prefix) int cmd_hook(int argc, const char **argv, const char *prefix) { + const char *run_hookdir = NULL; + struct option builtin_hook_options[] = { + OPT_STRING(0, "run-hookdir", &run_hookdir, N_("option"), + N_("what to do with hooks found in the hookdir")), OPT_END(), }; - if (argc < 2) + + argc = parse_options(argc, argv, prefix, builtin_hook_options, + builtin_hook_usage, 0); + + /* after the parse, we should have " " */ + if (argc < 1) usage_with_options(builtin_hook_usage, builtin_hook_options); git_config(git_default_config, NULL); - if (!strcmp(argv[1], "list")) - return list(argc - 1, argv + 1, prefix); + + /* argument > config */ + if (run_hookdir) + if (!strcmp(run_hookdir, "no")) + should_run_hookdir = HOOKDIR_NO; + else if (!strcmp(run_hookdir, "yes")) + should_run_hookdir = HOOKDIR_YES; + else if (!strcmp(run_hookdir, "warn")) + should_run_hookdir = HOOKDIR_WARN; + else if (!strcmp(run_hookdir, "interactive")) + should_run_hookdir = HOOKDIR_INTERACTIVE; + else + die(_("'%s' is not a valid option for --run-hookdir " + "(yes, warn, interactive, no)"), run_hookdir); + else + should_run_hookdir = configured_hookdir_opt(); + + if (!strcmp(argv[0], "list")) + return list(argc, argv, prefix); usage_with_options(builtin_hook_usage, builtin_hook_options); } diff --git a/hook.c b/hook.c index ffbdcfd987..ed52e85159 100644 --- a/hook.c +++ b/hook.c @@ -97,6 +97,27 @@ static int hook_config_lookup(const char *key, const char *value, void *cb_data) return 0; } +enum hookdir_opt configured_hookdir_opt(void) +{ + const char *key; + if (git_config_get_value("hook.runhookdir", &key)) + return HOOKDIR_YES; /* by default, just run it. */ + + if (!strcmp(key, "no")) + return HOOKDIR_NO; + + if (!strcmp(key, "yes")) + return HOOKDIR_YES; + + if (!strcmp(key, "warn")) + return HOOKDIR_WARN; + + if (!strcmp(key, "interactive")) + return HOOKDIR_INTERACTIVE; + + return HOOKDIR_UNKNOWN; +} + struct list_head* hook_list(const struct strbuf* hookname) { struct strbuf hook_key = STRBUF_INIT; diff --git a/hook.h b/hook.h index 5750634c83..ccdf6272f2 100644 --- a/hook.h +++ b/hook.h @@ -21,6 +21,21 @@ struct hook */ struct list_head* hook_list(const struct strbuf *hookname); +enum hookdir_opt +{ + HOOKDIR_NO, + HOOKDIR_WARN, + HOOKDIR_INTERACTIVE, + HOOKDIR_YES, + HOOKDIR_UNKNOWN, +}; + +/* + * Provides the hookdir_opt specified in the config without consulting any + * command line arguments. + */ +enum hookdir_opt configured_hookdir_opt(void); + /* Free memory associated with a 'struct hook' */ void free_hook(struct hook *ptr); /* Empties the list at 'head', calling 'free_hook()' on each entry */ diff --git a/t/t1360-config-based-hooks.sh b/t/t1360-config-based-hooks.sh index 0f12af4659..91127a50a4 100755 --- a/t/t1360-config-based-hooks.sh +++ b/t/t1360-config-based-hooks.sh @@ -104,4 +104,47 @@ test_expect_success 'git hook list shows hooks from the hookdir' ' test_cmp expected actual ' +test_expect_success 'hook.runHookDir = no is respected by list' ' + setup_hookdir && + + test_config hook.runHookDir "no" && + + cat >expected <<-EOF && + hookdir: $(pwd)/.git/hooks/pre-commit (will not run) + EOF + + git hook list pre-commit >actual && + # the hookdir annotation is translated + test_i18ncmp expected actual +' + +test_expect_success 'hook.runHookDir = warn is respected by list' ' + setup_hookdir && + + test_config hook.runHookDir "warn" && + + cat >expected <<-EOF && + hookdir: $(pwd)/.git/hooks/pre-commit (will warn) + EOF + + git hook list pre-commit >actual && + # the hookdir annotation is translated + test_i18ncmp expected actual +' + + +test_expect_success 'hook.runHookDir = interactive is respected by list' ' + setup_hookdir && + + test_config hook.runHookDir "interactive" && + + cat >expected <<-EOF && + hookdir: $(pwd)/.git/hooks/pre-commit (will prompt) + EOF + + git hook list pre-commit >actual && + # the hookdir annotation is translated + test_i18ncmp expected actual +' + test_done From patchwork Tue Dec 22 00:02:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 11985429 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 905CCC433DB for ; Tue, 22 Dec 2020 00:03:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5E4F7229C6 for ; Tue, 22 Dec 2020 00:03:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726387AbgLVADq (ORCPT ); Mon, 21 Dec 2020 19:03:46 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47942 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725852AbgLVADp (ORCPT ); Mon, 21 Dec 2020 19:03:45 -0500 Received: from mail-pj1-x104a.google.com (mail-pj1-x104a.google.com [IPv6:2607:f8b0:4864:20::104a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 80C70C061257 for ; Mon, 21 Dec 2020 16:02:36 -0800 (PST) Received: by mail-pj1-x104a.google.com with SMTP id c1so313942pjo.6 for ; Mon, 21 Dec 2020 16:02:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=l4nwN7UBxNOkLOLmU9E9t2L7T24QGATpAFRP9+Jp3qQ=; b=qVQ9hC5YElCIBgO5Kz6zFUBSPFfoZadJ5d+jq0ZUklelrslIfcU5XPVZ8+g5F1Hha8 QHsL+Tj11vuNPY0W3HGULtoQHctEGrdQJ2ZRCZLudZBvt8S+HC1Bu9GORIgVxFOqbBbr g1CSAXLKoSGE34WsFYImheqpdkxAsBqrqa+G41cElrOp5/149UBTWhBSckLGbr9bEqDi B1+rmffYH07WyTbQ6FTwji2ked97rLGFRx9mObIgeZJbg96xwpL74/STNkMH4qER44W7 FQCfdzPtrmBu/uMJWjY4gOaUJ+hsbtFMGF/5WYEXODYOg3IjylA4xM2qxnD4G0wlk+f1 BnYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=l4nwN7UBxNOkLOLmU9E9t2L7T24QGATpAFRP9+Jp3qQ=; b=S669Px6uTKi4vb6fHTn5WcAx/r5sdD39fHepgPQJe1WJczWLBrQo9DhTedSd5XswLk 5DsYGjATsz/KH8WXGpvRZzIHMyAs/jpHO3XIH5Vr0vxmJzvVurBgPoPWch/CRZtmwKiO lDe722jVBGJpbWwpO7FsgYwyaL1Ud1u569hHdub/x5inT7Tz/ndNmUOHyHOBOkE8TTJw R3lGBWe1KfherQC0Ws+oAfAuOcDCmWNjJ0Lkcf+CQQAIrKdcJvMUF8oL0ck5SA3GsbVm 0gclOgByEAUVnXbsMZNZ8YUWguFJPRtMbWm/nHYlibn5w8hhUi2NyWWaH7UplA3BMPKu g7mg== X-Gm-Message-State: AOAM533L/XN5p4XBghJaK3RISiigx/IQXjR3BKanmTV4ykKgE90Ow0iN itpXaT787vDHRtofFQWQ3fIHqsOr+oCm56fVzvNYQgAvl7TUimR35M1J5GWDeObM6Xpvys+Jtx2 9zBm+Dcr0Ke/Od3lW7zL+jdgRYaAf/A+BR5z8IPVkq1PKbwAJ/dO4ShUjZkug0dc2YJKPUfItBA == X-Google-Smtp-Source: ABdhPJwXCxUhI3aqqaqkvudkLjp3sgJ/+/+AkpESkXzyyu/TTpbqMFc5CB9x/w5dcF8sEoTZ1votpeCRwhDFHuK3BeE= Sender: "emilyshaffer via sendgmr" X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:1ea0:b8ff:fe77:f690]) (user=emilyshaffer job=sendgmr) by 2002:a17:902:b189:b029:dc:4102:4edf with SMTP id s9-20020a170902b189b02900dc41024edfmr4964766plr.80.1608595355924; Mon, 21 Dec 2020 16:02:35 -0800 (PST) Date: Mon, 21 Dec 2020 16:02:09 -0800 In-Reply-To: <20201222000220.1491091-1-emilyshaffer@google.com> Message-Id: <20201222000220.1491091-7-emilyshaffer@google.com> Mime-Version: 1.0 References: <20201222000220.1491091-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.29.2.490.gc7ae633391 Subject: [PATCH v7 06/17] hook: implement hookcmd..skip From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org If a user wants a specific repo to skip execution of a hook which is set at a global or system level, they can now do so by specifying 'skip' in their repo config: ~/.gitconfig [hook.pre-commit] command = skippable-oneliner command = skippable-hookcmd [hookcmd.skippable-hookcmd] command = foo.sh $GIT_DIR/.git/config [hookcmd.skippable-oneliner] skip = true [hookcmd.skippable-hookcmd] skip = true Later it may make sense to add an option like "hookcmd..-skip" - but for simplicity, let's start with a universal skip setting like this. Signed-off-by: Emily Shaffer --- Notes: In addition to being handy for turning off global hooks one project doesn't care about, this setting will be necessary much later for the 'proc-receive' hook, which can only cope with up to one hook being specified. New since v4. During the Google team's review club I was reminded about this whole 'skip' option I never implemented. It's true that it's impossible to exclude a given hook without this; however, I think I have some more work to do on it, so consider it RFC for now and tell me what you think :) - Emily During the Google team's review club this week I was reminded about this whole 'skip' option I never implemented. It's true that it's impossible to exclude a given hook without this; however, I think we have some more work to do on it, so consider it RFC for now and tell me what you think :) - Emily hook.c | 37 +++++++++++++++++++++++++---------- t/t1360-config-based-hooks.sh | 23 ++++++++++++++++++++++ 2 files changed, 50 insertions(+), 10 deletions(-) diff --git a/hook.c b/hook.c index ed52e85159..d262503725 100644 --- a/hook.c +++ b/hook.c @@ -12,23 +12,24 @@ void free_hook(struct hook *ptr) } } -static void append_or_move_hook(struct list_head *head, const char *command) +static struct hook* find_hook_by_command(struct list_head *head, const char *command) { struct list_head *pos = NULL, *tmp = NULL; - struct hook *to_add = NULL; + struct hook *found = NULL; - /* - * remove the prior entry with this command; we'll replace it at the - * end. - */ list_for_each_safe(pos, tmp, head) { struct hook *it = list_entry(pos, struct hook, list); if (!strcmp(it->command.buf, command)) { list_del(pos); - /* we'll simply move the hook to the end */ - to_add = it; + found = it; } } + return found; +} + +static void append_or_move_hook(struct list_head *head, const char *command) +{ + struct hook *to_add = find_hook_by_command(head, command); if (!to_add) { /* adding a new hook, not moving an old one */ @@ -41,7 +42,7 @@ static void append_or_move_hook(struct list_head *head, const char *command) /* re-set the scope so we show where an override was specified */ to_add->origin = current_config_scope(); - list_add_tail(&to_add->list, pos); + list_add_tail(&to_add->list, head); } static void remove_hook(struct list_head *to_remove) @@ -73,8 +74,18 @@ static int hook_config_lookup(const char *key, const char *value, void *cb_data) if (!strcmp(key, hook_key)) { const char *command = value; struct strbuf hookcmd_name = STRBUF_INIT; + int skip = 0; + + /* + * Check if we're removing that hook instead. Hookcmds are + * removed by name, and inlined hooks are removed by command + * content. + */ + strbuf_addf(&hookcmd_name, "hookcmd.%s.skip", command); + git_config_get_bool(hookcmd_name.buf, &skip); /* Check if a hookcmd with that name exists. */ + strbuf_reset(&hookcmd_name); strbuf_addf(&hookcmd_name, "hookcmd.%s.command", command); git_config_get_value(hookcmd_name.buf, &command); @@ -89,7 +100,13 @@ static int hook_config_lookup(const char *key, const char *value, void *cb_data) * for each key+value, do_callback(key, value, cb_data) */ - append_or_move_hook(head, command); + if (skip) { + struct hook *to_remove = find_hook_by_command(head, command); + if (to_remove) + remove_hook(&(to_remove->list)); + } else { + append_or_move_hook(head, command); + } strbuf_release(&hookcmd_name); } diff --git a/t/t1360-config-based-hooks.sh b/t/t1360-config-based-hooks.sh index 91127a50a4..ebd3bc623f 100755 --- a/t/t1360-config-based-hooks.sh +++ b/t/t1360-config-based-hooks.sh @@ -132,6 +132,29 @@ test_expect_success 'hook.runHookDir = warn is respected by list' ' test_i18ncmp expected actual ' +test_expect_success 'git hook list removes skipped hookcmd' ' + setup_hookcmd && + test_config hookcmd.abc.skip "true" --add && + + cat >expected <<-EOF && + no commands configured for hook '\''pre-commit'\'' + EOF + + git hook list pre-commit >actual && + test_i18ncmp expected actual +' + +test_expect_success 'git hook list removes skipped inlined hook' ' + setup_hooks && + test_config hookcmd."$ROOT/path/ghi".skip "true" --add && + + cat >expected <<-EOF && + global: $ROOT/path/def + EOF + + git hook list pre-commit >actual && + test_cmp expected actual +' test_expect_success 'hook.runHookDir = interactive is respected by list' ' setup_hookdir && From patchwork Tue Dec 22 00:02:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 11985433 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 398D6C433DB for ; Tue, 22 Dec 2020 00:03:50 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0A01122AED for ; Tue, 22 Dec 2020 00:03:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726472AbgLVADs (ORCPT ); Mon, 21 Dec 2020 19:03:48 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47950 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725782AbgLVADr (ORCPT ); Mon, 21 Dec 2020 19:03:47 -0500 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8F9FFC0611C5 for ; Mon, 21 Dec 2020 16:02:38 -0800 (PST) Received: by mail-yb1-xb4a.google.com with SMTP id d187so15869762ybc.6 for ; Mon, 21 Dec 2020 16:02:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=tYBIp72At1pPq2NwvWzTuSgSJkDuk44JQzLmJTw7OgE=; b=JeUmfwTV0dWITO3SyzdQRYlmsRb5VIPo0ZyVbeEo6Mt15og8mjGOXdW7JnUFeApqKE NADhDOb9hu7524WGJVJ8fUhgUxJbxt95eQP01iCYtZPyheSd9J32b/5qxWtavwBAq3km CDPbdrVmY7thLIeUb4gy5kbwSjsAJRTAhpqMGSauCPa+p0aRwN8nUKRo+R5gL/cK2wTa kvqOcYhpgKWsDTqGU/g72pt5z13a0bj0AMLMoqSjWvSko+NvJW++pCk5OQlV/1DEEX1v eR5/GvU6JX5VxUw5hGLxrKHHd1HMttkTqbBdSkJ59HCBW3IYchmTDZPVntm1Tvq5wiml zTYQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=tYBIp72At1pPq2NwvWzTuSgSJkDuk44JQzLmJTw7OgE=; b=dgYuL5Hvno2deB3XAH5/tn2V3xumYJ+sKiu6etP2ieZhl8Hx4Ee36y22CUwV6Ys3fj VfinNR/VXIg6c9XSmlyPEbMLIYTU6uDI3ws35O/Uni9Xq3Od21DU7dnjuqEOGN/c+pmb RTUJ++jDnVR3LKTgo54VsbSVQlCXW22UNjf+TN6RQwJtmRk0PF9r77F0EvJZvEy1TjrV /Pvytc/P+haQjth8fUHRhdy+TnQpC6u73jpG4YPMAIjM+V9gi0t4v+piE/VPBlhU63xS TMA/J+fy/JTk59kAT9AXqksjPMHMimXDAeaFt8mp9fzx0s+eJKnEq3ZFPOHTJR1cAQGu 3NnA== X-Gm-Message-State: AOAM5336W51pkWvzu0DnOrADKYBRrHAJWgv+tRCeSesLc7HM7QpsxEzE B4L+v2lsM0wMLEk6UeikenFpwF9eYB5Xupi2epBC7HmS6vi/VlAfMCi04Etarp6cm28+Y+2l6RI CiPAw3xMngia6QOCDYdrmJd2KRbc+JBXT+zzEHPfLi8g4spqlCsFEJxLwrt4QgiUj1w1yM0JIOg == X-Google-Smtp-Source: ABdhPJyCRj3HEw7bXs9gv5pB/vK2ev4FRo/XX4FxVNtUNmPkp8Vh3rSKUM0qLMmFvyQ5XhvphF6KzDws1HboqTWjPVU= Sender: "emilyshaffer via sendgmr" X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:1ea0:b8ff:fe77:f690]) (user=emilyshaffer job=sendgmr) by 2002:a25:ae14:: with SMTP id a20mr26796830ybj.410.1608595357796; Mon, 21 Dec 2020 16:02:37 -0800 (PST) Date: Mon, 21 Dec 2020 16:02:10 -0800 In-Reply-To: <20201222000220.1491091-1-emilyshaffer@google.com> Message-Id: <20201222000220.1491091-8-emilyshaffer@google.com> Mime-Version: 1.0 References: <20201222000220.1491091-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.29.2.490.gc7ae633391 Subject: [PATCH v7 07/17] parse-options: parse into strvec From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org parse-options already knows how to read into a string_list, and it knows how to read into an strvec as a passthrough (that is, including the argument as well as its value). string_list and strvec serve similar purposes but are somewhat painful to convert between; so, let's teach parse-options to read values of string arguments directly into an strvec without preserving the argument name. This is useful if collecting generic arguments to pass through to another command, for example, 'git hook run --arg "--quiet" --arg "--format=pretty" some-hook'. The resulting strvec would contain { "--quiet", "--format=pretty" }. The implementation is based on that of OPT_STRING_LIST. Signed-off-by: Emily Shaffer --- Notes: Since v4, fixed one or two more places where I missed the argv_array->strvec rename. Documentation/technical/api-parse-options.txt | 5 +++++ parse-options-cb.c | 16 ++++++++++++++++ parse-options.h | 4 ++++ 3 files changed, 25 insertions(+) diff --git a/Documentation/technical/api-parse-options.txt b/Documentation/technical/api-parse-options.txt index 5a60bbfa7f..679bd98629 100644 --- a/Documentation/technical/api-parse-options.txt +++ b/Documentation/technical/api-parse-options.txt @@ -173,6 +173,11 @@ There are some macros to easily define options: The string argument is stored as an element in `string_list`. Use of `--no-option` will clear the list of preceding values. +`OPT_STRVEC(short, long, &struct strvec, arg_str, description)`:: + Introduce an option with a string argument. + The string argument is stored as an element in `strvec`. + Use of `--no-option` will clear the list of preceding values. + `OPT_INTEGER(short, long, &int_var, description)`:: Introduce an option with integer argument. The integer is put into `int_var`. diff --git a/parse-options-cb.c b/parse-options-cb.c index 4542d4d3f9..c2451dfb1b 100644 --- a/parse-options-cb.c +++ b/parse-options-cb.c @@ -207,6 +207,22 @@ int parse_opt_string_list(const struct option *opt, const char *arg, int unset) return 0; } +int parse_opt_strvec(const struct option *opt, const char *arg, int unset) +{ + struct strvec *v = opt->value; + + if (unset) { + strvec_clear(v); + return 0; + } + + if (!arg) + return -1; + + strvec_push(v, arg); + return 0; +} + int parse_opt_noop_cb(const struct option *opt, const char *arg, int unset) { return 0; diff --git a/parse-options.h b/parse-options.h index 7030d8f3da..75cc8c7c96 100644 --- a/parse-options.h +++ b/parse-options.h @@ -177,6 +177,9 @@ struct option { #define OPT_STRING_LIST(s, l, v, a, h) \ { OPTION_CALLBACK, (s), (l), (v), (a), \ (h), 0, &parse_opt_string_list } +#define OPT_STRVEC(s, l, v, a, h) \ + { OPTION_CALLBACK, (s), (l), (v), (a), \ + (h), 0, &parse_opt_strvec } #define OPT_UYN(s, l, v, h) { OPTION_CALLBACK, (s), (l), (v), NULL, \ (h), PARSE_OPT_NOARG, &parse_opt_tertiary } #define OPT_EXPIRY_DATE(s, l, v, h) \ @@ -296,6 +299,7 @@ int parse_opt_commits(const struct option *, const char *, int); int parse_opt_commit(const struct option *, const char *, int); int parse_opt_tertiary(const struct option *, const char *, int); int parse_opt_string_list(const struct option *, const char *, int); +int parse_opt_strvec(const struct option *, const char *, int); int parse_opt_noop_cb(const struct option *, const char *, int); enum parse_opt_result parse_opt_unknown_cb(struct parse_opt_ctx_t *ctx, const struct option *, From patchwork Tue Dec 22 00:02:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 11985431 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9ED9BC433E6 for ; Tue, 22 Dec 2020 00:03:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5043522AED for ; Tue, 22 Dec 2020 00:03:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726486AbgLVADt (ORCPT ); Mon, 21 Dec 2020 19:03:49 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47952 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725844AbgLVADr (ORCPT ); Mon, 21 Dec 2020 19:03:47 -0500 Received: from mail-pg1-x54a.google.com (mail-pg1-x54a.google.com [IPv6:2607:f8b0:4864:20::54a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5B83CC0611CA for ; Mon, 21 Dec 2020 16:02:40 -0800 (PST) Received: by mail-pg1-x54a.google.com with SMTP id z4so7463598pgr.22 for ; Mon, 21 Dec 2020 16:02:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=agVQuhnOaLaKGub8ioDJ+ios7d1WojS1yaOceDnDMHQ=; b=tGBgaU7tDf9Zk5c/lWHTTdma3gvMPlrpjj1uRdIUXuBPe0px89hVveDgVFfvPQm5Ol ZrPE9lEFc7J3DEm3xHW61GwijWOE/bzFDBeTC22m1CgbhY62rJ+luKnoMPebbrEigTsw 4cfn/XIMwk+87Rsq8WA7+FA8vAvni7YCmRc4ZDhE1vUthrLyq574tdjVewsRwg27+Ojm SdNIU9aX3svONVTwxYcy/eSd+VMeR+ccHssPADQQJu7sr4HmcQk2ViwHb8N0BXaAVinA olBaI8xC8NfBn57Oc0UaiBzBSmyocR/qn3VV2QA5tFHhKCwwRlZPGunZmKlCXXPld0fg InjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=agVQuhnOaLaKGub8ioDJ+ios7d1WojS1yaOceDnDMHQ=; b=aUY9ofENXIkK+4NQk90fxWXlNBCJGZVlJF6h4wXiwcXLlXxSAYEbMm1u/Iwaze3gYW lZD0hY8DsCdNEWqLfA+MvZnaQBwWZKEzrKBaiV5jwwCTBYH+nnVxVOHxa7liekmCTYK8 xTGlgvQeJ1B3HNiePoBrRa9dKxXfbSXkrfeiSyhoZ+QWQz/OAXBFnDpEKFeq9V8/EGUz dzdpsMfwXx+mXE4JhPsrwuwMUwubARgS2GrdUIyhdSrk5mokCm9ORQmOlVU8SLuiVoK6 Vvr1Y1zcikk/Cl59GqxGSSLa5Ib/R2q/efP7xQ5To1vXpI2o4hFvR5VExednPy9G3ZNU uLvg== X-Gm-Message-State: AOAM530+3KRYSD1rFn7J6y51EnDfFpz4bGJKJ+rI3jxi6cXmSOBISxhE NYGJJ2p/t4h5EPk2+LHVhlcXCPwEl5MyNKsquT5R7Urya8+hfBboAuXu5m2pX0Boor1e+Av8/8i cCvm1yKNqsaExE9k6Aq6sxmL0ul6IlxKqo0pqmjEBnH6UqLFJZSjS6O57AlnRd9F87/X/eRML4A == X-Google-Smtp-Source: ABdhPJy6wuICcfgXGKQDzi7xntecMMpSNMFo+taoVB6Q5vUDamSmFGJkJYVO/hum/BB3yFFCJ2J8nZan+En9yWSy5zw= Sender: "emilyshaffer via sendgmr" X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:1ea0:b8ff:fe77:f690]) (user=emilyshaffer job=sendgmr) by 2002:a17:902:8203:b029:dc:3371:6b04 with SMTP id x3-20020a1709028203b02900dc33716b04mr14206532pln.81.1608595359788; Mon, 21 Dec 2020 16:02:39 -0800 (PST) Date: Mon, 21 Dec 2020 16:02:11 -0800 In-Reply-To: <20201222000220.1491091-1-emilyshaffer@google.com> Message-Id: <20201222000220.1491091-9-emilyshaffer@google.com> Mime-Version: 1.0 References: <20201222000220.1491091-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.29.2.490.gc7ae633391 Subject: [PATCH v7 08/17] hook: add 'run' subcommand From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In order to enable hooks to be run as an external process, by a standalone Git command, or by tools which wrap Git, provide an external means to run all configured hook commands for a given hook event. For now, the hook commands will run in config order, in series. As alternate ordering or parallelism is supported in the future, we should add knobs to use those to the command line as well. As with the legacy hook implementation, all stdout generated by hook commands is redirected to stderr. Piping from stdin is not yet supported. Legacy hooks (those present in $GITDIR/hooks) are run at the end of the execution list. For now, there is no way to disable them. Users may wish to provide hook commands like 'git config hook.pre-commit.command "~/linter.sh --pre-commit"'. To enable this, config-defined hooks are run in a shell. (Since hooks in $GITDIR/hooks can't be specified with included arguments or paths which need expansion like this, they are run without a shell instead.) Signed-off-by: Emily Shaffer --- Notes: Since v4, updated the docs, and did less local application of single quotes. In order for hookdir hooks to run successfully with a space in the path, though, they must not be run with 'sh -c'. So we can treat the hookdir hooks specially, and warn users via doc about special considerations for configured hooks with spaces in their path. Documentation/git-hook.txt | 31 +++++++++- builtin/hook.c | 48 ++++++++++++++- hook.c | 112 ++++++++++++++++++++++++++++++++++ hook.h | 32 ++++++++++ t/t1360-config-based-hooks.sh | 65 +++++++++++++++++++- 5 files changed, 281 insertions(+), 7 deletions(-) diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt index f19875ed68..18a817d832 100644 --- a/Documentation/git-hook.txt +++ b/Documentation/git-hook.txt @@ -9,11 +9,12 @@ SYNOPSIS -------- [verse] 'git hook' list +'git hook' run [(-e|--env)=...] [(-a|--arg)=...] DESCRIPTION ----------- -You can list configured hooks with this command. Later, you will be able to run, -add, and modify hooks with this command. +You can list and run configured hooks with this command. Later, you will be able +to add and modify hooks with this command. This command parses the default configuration files for sections `hook` and `hookcmd`. `hook` is used to describe the commands which will be run during a @@ -64,6 +65,32 @@ in the order they should be run, and print the config scope where the relevant `hook..command` was specified, not the `hookcmd` (if applicable). This output is human-readable and the format is subject to change over time. +run [(-e|--env)=...] [(-a|--arg)=...] ``:: + +Runs hooks configured for ``, in the same order displayed by `git +hook list`. Hooks configured this way are run prepended with `sh -c`, so paths +containing special characters or spaces should be wrapped in single quotes: +`command = '/my/path with spaces/script.sh' some args`. + +OPTIONS +------- +--run-hookdir:: + Overrides the hook.runHookDir config. Must be 'yes', 'warn', + 'interactive', or 'no'. Specifies how to handle hooks located in the Git + hook directory (core.hooksPath). + +-a:: +--arg:: + Only valid for `run`. ++ +Specify arguments to pass to every hook that is run. + +-e:: +--env:: + Only valid for `run`. ++ +Specify environment variables to set for every hook that is run. + CONFIGURATION ------------- include::config/hook.txt[] diff --git a/builtin/hook.c b/builtin/hook.c index d087e6f5b0..07ba00e07a 100644 --- a/builtin/hook.c +++ b/builtin/hook.c @@ -5,9 +5,11 @@ #include "hook.h" #include "parse-options.h" #include "strbuf.h" +#include "strvec.h" static const char * const builtin_hook_usage[] = { N_("git hook list "), + N_("git hook run [(-e|--env)=...] [(-a|--arg)=...] "), NULL }; @@ -84,6 +86,46 @@ static int list(int argc, const char **argv, const char *prefix) return 0; } +static int run(int argc, const char **argv, const char *prefix) +{ + struct strbuf hookname = STRBUF_INIT; + struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT; + int rc = 0; + + struct option run_options[] = { + OPT_STRVEC('e', "env", &opt.env, N_("var"), + N_("environment variables for hook to use")), + OPT_STRVEC('a', "arg", &opt.args, N_("args"), + N_("argument to pass to hook")), + OPT_END(), + }; + + /* + * While it makes sense to list hooks out-of-repo, it doesn't make sense + * to execute them. Hooks usually want to look at repository artifacts. + */ + if (!have_git_dir()) + usage_msg_opt(_("You must be in a Git repo to execute hooks."), + builtin_hook_usage, run_options); + + argc = parse_options(argc, argv, prefix, run_options, + builtin_hook_usage, 0); + + if (argc < 1) + usage_msg_opt(_("You must specify a hook event to run."), + builtin_hook_usage, run_options); + + strbuf_addstr(&hookname, argv[0]); + opt.run_hookdir = should_run_hookdir; + + rc = run_hooks(hookname.buf, &opt); + + strbuf_release(&hookname); + run_hooks_opt_clear(&opt); + + return rc; +} + int cmd_hook(int argc, const char **argv, const char *prefix) { const char *run_hookdir = NULL; @@ -95,10 +137,10 @@ int cmd_hook(int argc, const char **argv, const char *prefix) }; argc = parse_options(argc, argv, prefix, builtin_hook_options, - builtin_hook_usage, 0); + builtin_hook_usage, PARSE_OPT_KEEP_UNKNOWN); /* after the parse, we should have " " */ - if (argc < 1) + if (argc < 2) usage_with_options(builtin_hook_usage, builtin_hook_options); git_config(git_default_config, NULL); @@ -122,6 +164,8 @@ int cmd_hook(int argc, const char **argv, const char *prefix) if (!strcmp(argv[0], "list")) return list(argc, argv, prefix); + if (!strcmp(argv[0], "run")) + return run(argc, argv, prefix); usage_with_options(builtin_hook_usage, builtin_hook_options); } diff --git a/hook.c b/hook.c index d262503725..5836bbb739 100644 --- a/hook.c +++ b/hook.c @@ -3,6 +3,7 @@ #include "hook.h" #include "config.h" #include "run-command.h" +#include "prompt.h" void free_hook(struct hook *ptr) { @@ -135,6 +136,56 @@ enum hookdir_opt configured_hookdir_opt(void) return HOOKDIR_UNKNOWN; } +static int should_include_hookdir(const char *path, enum hookdir_opt cfg) +{ + struct strbuf prompt = STRBUF_INIT; + /* + * If the path doesn't exist, don't bother adding the empty hook and + * don't bother checking the config or prompting the user. + */ + if (!path) + return 0; + + switch (cfg) + { + case HOOKDIR_NO: + return 0; + case HOOKDIR_UNKNOWN: + fprintf(stderr, + _("Unrecognized value for 'hook.runHookDir'. " + "Is there a typo? ")); + /* FALLTHROUGH */ + case HOOKDIR_WARN: + fprintf(stderr, _("Running legacy hook at '%s'\n"), + path); + return 1; + case HOOKDIR_INTERACTIVE: + do { + /* + * TRANSLATORS: Make sure to include [Y] and [n] + * in your translation. Only English input is + * accepted. Default option is "yes". + */ + fprintf(stderr, _("Run '%s'? [Yn] "), path); + git_read_line_interactively(&prompt); + strbuf_tolower(&prompt); + if (starts_with(prompt.buf, "n")) { + strbuf_release(&prompt); + return 0; + } else if (starts_with(prompt.buf, "y")) { + strbuf_release(&prompt); + return 1; + } + /* otherwise, we didn't understand the input */ + } while (prompt.len); /* an empty reply means "Yes" */ + strbuf_release(&prompt); + return 1; + case HOOKDIR_YES: + default: + return 1; + } +} + struct list_head* hook_list(const struct strbuf* hookname) { struct strbuf hook_key = STRBUF_INIT; @@ -166,3 +217,64 @@ struct list_head* hook_list(const struct strbuf* hookname) strbuf_release(&hook_key); return hook_head; } + +void run_hooks_opt_init(struct run_hooks_opt *o) +{ + strvec_init(&o->env); + strvec_init(&o->args); + o->run_hookdir = configured_hookdir_opt(); +} + +void run_hooks_opt_clear(struct run_hooks_opt *o) +{ + strvec_clear(&o->env); + strvec_clear(&o->args); +} + +int run_hooks(const char *hookname, struct run_hooks_opt *options) +{ + struct strbuf hookname_str = STRBUF_INIT; + struct list_head *to_run, *pos = NULL, *tmp = NULL; + int rc = 0; + + if (!options) + BUG("a struct run_hooks_opt must be provided to run_hooks"); + + strbuf_addstr(&hookname_str, hookname); + + to_run = hook_list(&hookname_str); + + list_for_each_safe(pos, tmp, to_run) { + struct child_process hook_proc = CHILD_PROCESS_INIT; + struct hook *hook = list_entry(pos, struct hook, list); + + hook_proc.env = options->env.v; + hook_proc.no_stdin = 1; + hook_proc.stdout_to_stderr = 1; + hook_proc.trace2_hook_name = hook->command.buf; + hook_proc.use_shell = 1; + + if (hook->from_hookdir) { + if (!should_include_hookdir(hook->command.buf, options->run_hookdir)) + continue; + /* + * Commands from the config could be oneliners, but we know + * for certain that hookdir commands are not. + */ + hook_proc.use_shell = 0; + } + + /* add command */ + strvec_push(&hook_proc.args, hook->command.buf); + + /* + * add passed-in argv, without expanding - let the user get back + * exactly what they put in + */ + strvec_pushv(&hook_proc.args, options->args.v); + + rc |= run_command(&hook_proc); + } + + return rc; +} diff --git a/hook.h b/hook.h index ccdf6272f2..259662968f 100644 --- a/hook.h +++ b/hook.h @@ -1,6 +1,7 @@ #include "config.h" #include "list.h" #include "strbuf.h" +#include "strvec.h" struct hook { @@ -36,6 +37,37 @@ enum hookdir_opt */ enum hookdir_opt configured_hookdir_opt(void); +struct run_hooks_opt +{ + /* Environment vars to be set for each hook */ + struct strvec env; + + /* Args to be passed to each hook */ + struct strvec args; + + /* + * How should the hookdir be handled? + * Leave the RUN_HOOKS_OPT_INIT default in most cases; this only needs + * to be overridden if the user can override it at the command line. + */ + enum hookdir_opt run_hookdir; +}; + +#define RUN_HOOKS_OPT_INIT { \ + .env = STRVEC_INIT, \ + .args = STRVEC_INIT, \ + .run_hookdir = configured_hookdir_opt() \ +} + +void run_hooks_opt_init(struct run_hooks_opt *o); +void run_hooks_opt_clear(struct run_hooks_opt *o); + +/* + * Runs all hooks associated to the 'hookname' event in order. Each hook will be + * passed 'env' and 'args'. + */ +int run_hooks(const char *hookname, struct run_hooks_opt *options); + /* Free memory associated with a 'struct hook' */ void free_hook(struct hook *ptr); /* Empties the list at 'head', calling 'free_hook()' on each entry */ diff --git a/t/t1360-config-based-hooks.sh b/t/t1360-config-based-hooks.sh index ebd3bc623f..5b3003d59b 100755 --- a/t/t1360-config-based-hooks.sh +++ b/t/t1360-config-based-hooks.sh @@ -115,7 +115,10 @@ test_expect_success 'hook.runHookDir = no is respected by list' ' git hook list pre-commit >actual && # the hookdir annotation is translated - test_i18ncmp expected actual + test_i18ncmp expected actual && + + git hook run pre-commit 2>actual && + test_must_be_empty actual ' test_expect_success 'hook.runHookDir = warn is respected by list' ' @@ -129,6 +132,14 @@ test_expect_success 'hook.runHookDir = warn is respected by list' ' git hook list pre-commit >actual && # the hookdir annotation is translated + test_i18ncmp expected actual && + + cat >expected <<-EOF && + Running legacy hook at '\''$(pwd)/.git/hooks/pre-commit'\'' + "Legacy Hook" + EOF + + git hook run pre-commit 2>actual && test_i18ncmp expected actual ' @@ -156,7 +167,7 @@ test_expect_success 'git hook list removes skipped inlined hook' ' test_cmp expected actual ' -test_expect_success 'hook.runHookDir = interactive is respected by list' ' +test_expect_success 'hook.runHookDir = interactive is respected by list and run' ' setup_hookdir && test_config hook.runHookDir "interactive" && @@ -167,7 +178,55 @@ test_expect_success 'hook.runHookDir = interactive is respected by list' ' git hook list pre-commit >actual && # the hookdir annotation is translated - test_i18ncmp expected actual + test_i18ncmp expected actual && + + test_write_lines n | git hook run pre-commit 2>actual && + ! grep "Legacy Hook" actual && + + test_write_lines y | git hook run pre-commit 2>actual && + grep "Legacy Hook" actual +' + +test_expect_success 'inline hook definitions execute oneliners' ' + test_config hook.pre-commit.command "echo \"Hello World\"" && + + echo "Hello World" >expected && + + # hooks are run with stdout_to_stderr = 1 + git hook run pre-commit 2>actual && + test_cmp expected actual +' + +test_expect_success 'inline hook definitions resolve paths' ' + write_script sample-hook.sh <<-EOF && + echo \"Sample Hook\" + EOF + + test_when_finished "rm sample-hook.sh" && + + test_config hook.pre-commit.command "\"$(pwd)/sample-hook.sh\"" && + + echo \"Sample Hook\" >expected && + + # hooks are run with stdout_to_stderr = 1 + git hook run pre-commit 2>actual && + test_cmp expected actual +' + +test_expect_success 'hookdir hook included in git hook run' ' + setup_hookdir && + + echo \"Legacy Hook\" >expected && + + # hooks are run with stdout_to_stderr = 1 + git hook run pre-commit 2>actual && + test_cmp expected actual +' + +test_expect_success 'out-of-repo runs excluded' ' + setup_hooks && + + nongit test_must_fail git hook run pre-commit ' test_done From patchwork Tue Dec 22 00:02:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 11985435 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2A820C433E0 for ; Tue, 22 Dec 2020 00:03:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id F3E5622AED for ; Tue, 22 Dec 2020 00:03:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726507AbgLVADu (ORCPT ); Mon, 21 Dec 2020 19:03:50 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47956 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725844AbgLVADt (ORCPT ); Mon, 21 Dec 2020 19:03:49 -0500 Received: from mail-qk1-x749.google.com (mail-qk1-x749.google.com [IPv6:2607:f8b0:4864:20::749]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 02537C0611CB for ; Mon, 21 Dec 2020 16:02:43 -0800 (PST) Received: by mail-qk1-x749.google.com with SMTP id l138so10174904qke.4 for ; Mon, 21 Dec 2020 16:02:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=TXJQA/b51PKKHbz3F5UC4bTZR/zxTZx2d/9xPC06otY=; b=mj3nf7vzVLZdGmxpNUzS564PUhPm5akLh/enNJ3yF8MLR8x3BhL805LMqdH6IkGcCz 84eesLQwXzd3MZkEX8wA4rdS8OwC6QYuDNmdd2ps7Hulx9xMcd/tRjv+h9jIXegMba3a V6duwq94HNIF8zrMd3pJSoYUYml0n97RYD9gXQGD8rKRmDo2QbjMmvW6q5lbQQH1t40m VVLCogAAGel1k+v3B7e4X5ZXRLkPcXYbE5kpTb8GqNZ/SJVRYZ1uIr9okMQTN1A2ZTLH hBGejLmzgDBxa5/LRIKUV7oLqZZqC8TZ7KjIezaK6SwIE44b5SXACYxTt835PRpyCVxg q5qw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=TXJQA/b51PKKHbz3F5UC4bTZR/zxTZx2d/9xPC06otY=; b=V2+EV7x6ER6UJtHy2XjJ/BHVVYmHqbZqUTR1VXtS/BAsQKUaT9Y4WA2rIsxG9Ylg5H h8jdEkBsbF5w/x7ZQn6VIUtXDEO4Dmqs02m626sOUFOKM9uwHUwxnxmiVJW8wODACdJT 3hm9z3Un+CFpPdasodH0UHAUV08qYLWgHNXrg87xoXvyiTtXK6Ris8INKSHlm2HU/Mtb iy7jgXV16soPDtLe+mWI9FCWw4AkTgHcSD1IrITqSCCBdM8K0TevZCyfXW/nR6dJTnjU lQFHU3RDtDwFuz41BURsOK4M80N8ce2xxbn+alc2VL6Y5RkfTPoaH2wPyWfTExsZgvSl tHvg== X-Gm-Message-State: AOAM530ir9B5V5l8N6rRnB1cbrSlPfb6sLXDOv00fN71PqWU+Wp8VN3p UuDI0qFLQvN3TyBXsWKVDB1UUUEq0yi+1DDzBtr0rIaG5QqF/PmQPd5zs0hL7t3cmJn6V6jB+9T rnxHtYqp/59R8wTxxQpWOIrGxHm6VU+dz9rshJ1lG+cMwbWGhTWzqOzeXVfp5R+pXX26DF9zEeQ == X-Google-Smtp-Source: ABdhPJzeCy4sHUz2FiaSXGX2n1UCsj9+fDIZzWCgQfbFmDNa0omaTbZNPdt7yCoeJrElRI7qhVRRIbQUs6QGbHFEI28= Sender: "emilyshaffer via sendgmr" X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:1ea0:b8ff:fe77:f690]) (user=emilyshaffer job=sendgmr) by 2002:a05:6214:24:: with SMTP id b4mr19752680qvr.29.1608595362123; Mon, 21 Dec 2020 16:02:42 -0800 (PST) Date: Mon, 21 Dec 2020 16:02:12 -0800 In-Reply-To: <20201222000220.1491091-1-emilyshaffer@google.com> Message-Id: <20201222000220.1491091-10-emilyshaffer@google.com> Mime-Version: 1.0 References: <20201222000220.1491091-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.29.2.490.gc7ae633391 Subject: [PATCH v7 09/17] hook: replace find_hook() with hook_exists() From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Add a helper to easily determine whether any hooks exist for a given hook event. Many callers want to check whether some state could be modified by a hook; that check should include the config-based hooks as well. Optimize by checking the config directly. Since commands which execute hooks might want to take args to replace 'hook.runHookDir', let 'hook_exists()' mirror the behavior of 'hook.runHookDir'. Signed-off-by: Emily Shaffer --- builtin/bugreport.c | 4 ++-- hook.c | 15 +++++++++++++++ hook.h | 9 +++++++++ 3 files changed, 26 insertions(+), 2 deletions(-) diff --git a/builtin/bugreport.c b/builtin/bugreport.c index ad3cc9c02f..2fe65d8f1e 100644 --- a/builtin/bugreport.c +++ b/builtin/bugreport.c @@ -3,7 +3,7 @@ #include "strbuf.h" #include "help.h" #include "compat/compiler.h" -#include "run-command.h" +#include "hook.h" static void get_system_info(struct strbuf *sys_info) @@ -82,7 +82,7 @@ static void get_populated_hooks(struct strbuf *hook_info, int nongit) } for (i = 0; i < ARRAY_SIZE(hook); i++) - if (find_hook(hook[i])) + if (hook_exists(hook[i], configured_hookdir_opt())) strbuf_addf(hook_info, "%s\n", hook[i]); } diff --git a/hook.c b/hook.c index 5836bbb739..fbb69706d8 100644 --- a/hook.c +++ b/hook.c @@ -225,6 +225,21 @@ void run_hooks_opt_init(struct run_hooks_opt *o) o->run_hookdir = configured_hookdir_opt(); } +int hook_exists(const char *hookname, enum hookdir_opt should_run_hookdir) +{ + const char *value = NULL; /* throwaway */ + struct strbuf hook_key = STRBUF_INIT; + + int could_run_hookdir = (should_run_hookdir == HOOKDIR_INTERACTIVE || + should_run_hookdir == HOOKDIR_WARN || + should_run_hookdir == HOOKDIR_YES) + && !!find_hook(hookname); + + strbuf_addf(&hook_key, "hook.%s.command", hookname); + + return (!git_config_get_value(hook_key.buf, &value)) || could_run_hookdir; +} + void run_hooks_opt_clear(struct run_hooks_opt *o) { strvec_clear(&o->env); diff --git a/hook.h b/hook.h index 259662968f..762b6fadad 100644 --- a/hook.h +++ b/hook.h @@ -62,6 +62,15 @@ struct run_hooks_opt void run_hooks_opt_init(struct run_hooks_opt *o); void run_hooks_opt_clear(struct run_hooks_opt *o); +/* + * Returns 1 if any hooks are specified in the config or if a hook exists in the + * hookdir. Typically, invoke hook_exsts() like: + * hook_exists(hookname, configured_hookdir_opt()); + * Like with run_hooks, if you take a --run-hookdir flag, reflect that + * user-specified behavior here instead. + */ +int hook_exists(const char *hookname, enum hookdir_opt should_run_hookdir); + /* * Runs all hooks associated to the 'hookname' event in order. Each hook will be * passed 'env' and 'args'. From patchwork Tue Dec 22 00:02:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 11985437 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EDA5BC43381 for ; Tue, 22 Dec 2020 00:03:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B587722AED for ; Tue, 22 Dec 2020 00:03:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726514AbgLVADv (ORCPT ); Mon, 21 Dec 2020 19:03:51 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47960 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726487AbgLVADt (ORCPT ); Mon, 21 Dec 2020 19:03:49 -0500 Received: from mail-qv1-xf49.google.com (mail-qv1-xf49.google.com [IPv6:2607:f8b0:4864:20::f49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ABF10C0611CC for ; Mon, 21 Dec 2020 16:02:44 -0800 (PST) Received: by mail-qv1-xf49.google.com with SMTP id j24so9340605qvg.8 for ; Mon, 21 Dec 2020 16:02:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=hqBG27AjgoiYQ8SVuvV/IoKUFszFAUAXY6rgXEtHHxY=; b=DFR8SSpn4LyDLhPk6hrLum+hDl3dAbg0ohOZTYuKrtQQQTabpukPL1uoRtXTtjLu3m Yj/pNH9qiPoE0EoieRyIrSOdQ2aEu1wtj9SAfOI4S7rOt57mdK1/qmD5sn+OF6eiatq5 7hc5SEejVNOSYOZPB0zA354+qfeLc/oGLleZHdI3/eW45J8nRqJ1RE/JZjaiSZgZCKs+ 6fi/usPHgFaVg34O6BdMqSBsQtT4AeNc7tsR5dNf8mZxWdhdbQSL9vPaKFpHRCk5TC/c TaHGa08QxRq2sdSUVVXp0A/tQ2+mon8FZxasEzW1CUXPa3gcck8yVbu58iukPgLoeqba h8kg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=hqBG27AjgoiYQ8SVuvV/IoKUFszFAUAXY6rgXEtHHxY=; b=kw5o7qLcqyTAhljR/NNm+cyTN5Y6GotIVLIgAC6EhFCRcTVRWyNNwIfG6ZqWwfteD7 VX4NeKZR82EKKnsbH8oFKfMToMecuMbVUjeOTEMnLEbk9miVtagx/K5UsnnWD5tN2NFB q0ZexvTBkLTEwPgGqvuQjInZUb+2jnZTuRINHsMR7SYg10NBdMZJJbY4e5ua7LdjORsP nb2rcOCfdy2/EPjIHE8G7689awBq2xEhaslZNhsOZpHr782Xt4Rr1KG+eYgLG2cIcYtK cI6B+MKPqweZ88fWIfn/vtPk+6HwcNlnnD6Q7GIrm45thIhXNgGutMAmdh5aLX7PISnp uX4g== X-Gm-Message-State: AOAM531ut+qH5nL83LXj93bqaDDCzk48Fx06HdtxBHzyemJ7CqNQFaOs Du3/PjlTubT1qeNma0Vdd0iY26/160BS6+Kg2KdUX80zaWQTCeREGakUYeSRPI5wR2luHkWKWAK ngbz2VzDnfPT40rayUdWQvDvq9AY52Talla+dQ6wDtJoqoFHiiEr7vawW6eY6iL6vX68zzQMeAg == X-Google-Smtp-Source: ABdhPJzTlW/r12a1SenaeSMiydoaavVRaIzlngsrUR4XPt+tLpKOExIFCmxvnkS4zBJ3dFDW9Zzcm3e3zcoMWjIkUJ4= Sender: "emilyshaffer via sendgmr" X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:1ea0:b8ff:fe77:f690]) (user=emilyshaffer job=sendgmr) by 2002:ad4:45a9:: with SMTP id y9mr19540991qvu.15.1608595363847; Mon, 21 Dec 2020 16:02:43 -0800 (PST) Date: Mon, 21 Dec 2020 16:02:13 -0800 In-Reply-To: <20201222000220.1491091-1-emilyshaffer@google.com> Message-Id: <20201222000220.1491091-11-emilyshaffer@google.com> Mime-Version: 1.0 References: <20201222000220.1491091-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.29.2.490.gc7ae633391 Subject: [PATCH v7 10/17] hook: support passing stdin to hooks From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Some hooks (such as post-rewrite) need to take input via stdin. Previously, callers provided stdin to hooks by setting run-command.h:child_process.in, which takes a FD. Callers would open the file in question themselves before calling run-command(). However, since we will now need to seek to the front of the file and read it again for every hook which runs, hook.h:run_command() takes a path and handles FD management itself. Since this file is opened for read only, it should not prevent later parallel execution support. On the frontend, this is supported by asking for a file path, rather than by reading stdin. Reading directly from stdin would involve caching the entire stdin (to memory or to disk) and reading it back from the beginning to each hook. We'd want to support cases like insufficient memory or storage for the file. While this may prove useful later, for now the path of least resistance is to just ask the user to make this interim file themselves. Signed-off-by: Emily Shaffer --- Documentation/git-hook.txt | 11 +++++++++-- builtin/hook.c | 5 ++++- hook.c | 7 ++++++- hook.h | 9 +++++++-- t/t1360-config-based-hooks.sh | 24 ++++++++++++++++++++++++ 5 files changed, 50 insertions(+), 6 deletions(-) diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt index 18a817d832..cce30a80d0 100644 --- a/Documentation/git-hook.txt +++ b/Documentation/git-hook.txt @@ -9,7 +9,8 @@ SYNOPSIS -------- [verse] 'git hook' list -'git hook' run [(-e|--env)=...] [(-a|--arg)=...] +'git hook' run [(-e|--env)=...] [(-a|--arg)=...] [--to-stdin=] + DESCRIPTION ----------- @@ -65,7 +66,7 @@ in the order they should be run, and print the config scope where the relevant `hook..command` was specified, not the `hookcmd` (if applicable). This output is human-readable and the format is subject to change over time. -run [(-e|--env)=...] [(-a|--arg)=...] ``:: +run [(-e|--env)=...] [(-a|--arg)=...] [--to-stdin=] ``:: Runs hooks configured for ``, in the same order displayed by `git hook list`. Hooks configured this way are run prepended with `sh -c`, so paths @@ -91,6 +92,12 @@ Specify arguments to pass to every hook that is run. + Specify environment variables to set for every hook that is run. +--to-stdin:: + Only valid for `run`. ++ +Specify a file which will be streamed into stdin for every hook that is run. +Each hook will receive the entire file from beginning to EOF. + CONFIGURATION ------------- include::config/hook.txt[] diff --git a/builtin/hook.c b/builtin/hook.c index 07ba00e07a..be104f2938 100644 --- a/builtin/hook.c +++ b/builtin/hook.c @@ -9,7 +9,8 @@ static const char * const builtin_hook_usage[] = { N_("git hook list "), - N_("git hook run [(-e|--env)=...] [(-a|--arg)=...] "), + N_("git hook run [(-e|--env)=...] [(-a|--arg)=...]" + "[--to-stdin=] "), NULL }; @@ -97,6 +98,8 @@ static int run(int argc, const char **argv, const char *prefix) N_("environment variables for hook to use")), OPT_STRVEC('a', "arg", &opt.args, N_("args"), N_("argument to pass to hook")), + OPT_STRING(0, "to-stdin", &opt.path_to_stdin, N_("path"), + N_("file to read into hooks' stdin")), OPT_END(), }; diff --git a/hook.c b/hook.c index fbb69706d8..ce5c443206 100644 --- a/hook.c +++ b/hook.c @@ -263,8 +263,13 @@ int run_hooks(const char *hookname, struct run_hooks_opt *options) struct child_process hook_proc = CHILD_PROCESS_INIT; struct hook *hook = list_entry(pos, struct hook, list); + /* reopen the file for stdin; run_command closes it. */ + if (options->path_to_stdin) + hook_proc.in = xopen(options->path_to_stdin, O_RDONLY); + else + hook_proc.no_stdin = 1; + hook_proc.env = options->env.v; - hook_proc.no_stdin = 1; hook_proc.stdout_to_stderr = 1; hook_proc.trace2_hook_name = hook->command.buf; hook_proc.use_shell = 1; diff --git a/hook.h b/hook.h index 762b6fadad..e22a6db832 100644 --- a/hook.h +++ b/hook.h @@ -51,11 +51,15 @@ struct run_hooks_opt * to be overridden if the user can override it at the command line. */ enum hookdir_opt run_hookdir; + + /* Path to file which should be piped to stdin for each hook */ + const char *path_to_stdin; }; #define RUN_HOOKS_OPT_INIT { \ - .env = STRVEC_INIT, \ + .env = STRVEC_INIT, \ .args = STRVEC_INIT, \ + .path_to_stdin = NULL, \ .run_hookdir = configured_hookdir_opt() \ } @@ -73,7 +77,8 @@ int hook_exists(const char *hookname, enum hookdir_opt should_run_hookdir); /* * Runs all hooks associated to the 'hookname' event in order. Each hook will be - * passed 'env' and 'args'. + * passed 'env' and 'args'. The file at 'stdin_path' will be closed and reopened + * for each hook that runs. */ int run_hooks(const char *hookname, struct run_hooks_opt *options); diff --git a/t/t1360-config-based-hooks.sh b/t/t1360-config-based-hooks.sh index 5b3003d59b..c672269ee4 100755 --- a/t/t1360-config-based-hooks.sh +++ b/t/t1360-config-based-hooks.sh @@ -229,4 +229,28 @@ test_expect_success 'out-of-repo runs excluded' ' nongit test_must_fail git hook run pre-commit ' +test_expect_success 'stdin to multiple hooks' ' + git config --add hook.test.command "xargs -P1 -I% echo a%" && + git config --add hook.test.command "xargs -P1 -I% echo b%" && + test_when_finished "test_unconfig hook.test.command" && + + cat >input <<-EOF && + 1 + 2 + 3 + EOF + + cat >expected <<-EOF && + a1 + a2 + a3 + b1 + b2 + b3 + EOF + + git hook run --to-stdin=input test 2>actual && + test_cmp expected actual +' + test_done From patchwork Tue Dec 22 00:02:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 11985441 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B6A73C433DB for ; Tue, 22 Dec 2020 00:03:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8AA3F22AED for ; Tue, 22 Dec 2020 00:03:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726529AbgLVADw (ORCPT ); Mon, 21 Dec 2020 19:03:52 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47964 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726487AbgLVADv (ORCPT ); Mon, 21 Dec 2020 19:03:51 -0500 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C4D40C0611CD for ; Mon, 21 Dec 2020 16:02:46 -0800 (PST) Received: by mail-yb1-xb49.google.com with SMTP id w8so15694786ybj.14 for ; Mon, 21 Dec 2020 16:02:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=xksZ8TN+Bx5zxuwy/jMdxbToJEO3CXMa1+UlyOml024=; b=cXnu/3plmkjViH15W08V7wwwypVbR6t47iHNOv4igl3Cua4vXG7CmKx5r4cj+qjrlc d8wOK1zR4r2wjXMJYqRyCQBFC3W8LNfqMENUkvg0Z+F20AHZ5TOt/G5y2Q7UnkfRKPz+ mFRKiRwtS7AByAaXkVXDWz69oGig+qVcMb9X6QmI3X2pEYb5tpLsVeMvalsFefNoatt6 y/bNDhuAhjq/UUuCTXu3hp0725g3o+6huRiGkm0MTc1Dthh6yzS3j3Wvj95DDdcEGrHl dFaKnD/n96kRUlSU/3vHPWxnWlsHy2mvEOoEKFUnpEQfX3xHdliU7gkcbsK3u8QxlJON l3eA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=xksZ8TN+Bx5zxuwy/jMdxbToJEO3CXMa1+UlyOml024=; b=oFJunwGbP+S1WbJpq0MyF/7WVSpQbuucv4jRmqdIMqjKmUAv6R4xNYAR+YEr7203EV EH0yGfa+gdh4YWxwltO6tYjDXlPj6LQUeszmtzTzLN1U0HHlploQ5vFyDjWBUUwmAuLy fEYWBxIKLg5Iz59Ru/w/IQJJSr/xOAZ6a1wKGdHg3yr71IPHgFKrDjL/mt3uA+86o7sP SjKUdNNALGFuEzGnIra/95IzOoMukVCGhB2WDltGZHd1v+LOj4F3Sr5jcgZ6P95cQx6r 1An7/2ScUVd7OWw5FXn6i143xsdpTHHX06JbDcSj8u8gfaiaCEnz31FfXAv7Ri6V3bWY cOww== X-Gm-Message-State: AOAM532j8wbHJzrtyIOHMOQFVerjsv841uK3zOSqMhwIu9cfRPn/SIO9 nYxmXO2iieHhOJqZRXOVtchm7bdiohMtT4/YVMJMCrKQcH/sGs9m8wqLPcIXjYMQDSZp3TQDJTD cEmsAsy3ZLT3s/NQnp2JWFhULtNV1xTs5ViJkJvW6xNcyIFW+oJnl4FitCNqu9P33zVYGz9Avaw == X-Google-Smtp-Source: ABdhPJwyxhVhdtc6MQEaxr7gMbUVHRg13FR3GKmfdZri/AFaXJIkGahydwGCQNm1fs4NOUJxADBApQoqbrw44Tzkklg= Sender: "emilyshaffer via sendgmr" X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:1ea0:b8ff:fe77:f690]) (user=emilyshaffer job=sendgmr) by 2002:a5b:101:: with SMTP id 1mr24847735ybx.507.1608595365912; Mon, 21 Dec 2020 16:02:45 -0800 (PST) Date: Mon, 21 Dec 2020 16:02:14 -0800 In-Reply-To: <20201222000220.1491091-1-emilyshaffer@google.com> Message-Id: <20201222000220.1491091-12-emilyshaffer@google.com> Mime-Version: 1.0 References: <20201222000220.1491091-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.29.2.490.gc7ae633391 Subject: [PATCH v7 11/17] run-command: allow stdin for run_processes_parallel From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org While it makes sense not to inherit stdin from the parent process to avoid deadlocking, it's not necessary to completely ban stdin to children. An informed user should be able to configure stdin safely. By setting `some_child.process.no_stdin=1` before calling `get_next_task()` we provide a reasonable default behavior but enable users to set up stdin streaming for themselves during the callback. `some_child.process.stdout_to_stderr`, however, remains unmodifiable by `get_next_task()` - the rest of the run_processes_parallel() API depends on child output in stderr. Signed-off-by: Emily Shaffer --- run-command.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/run-command.c b/run-command.c index ea4d0fb4b1..80c8c97bc1 100644 --- a/run-command.c +++ b/run-command.c @@ -1683,6 +1683,9 @@ static int pp_start_one(struct parallel_processes *pp) if (i == pp->max_processes) BUG("bookkeeping is hard"); + /* disallow by default, but allow users to set up stdin if they wish */ + pp->children[i].process.no_stdin = 1; + code = pp->get_next_task(&pp->children[i].process, &pp->children[i].err, pp->data, @@ -1694,7 +1697,6 @@ static int pp_start_one(struct parallel_processes *pp) } pp->children[i].process.err = -1; pp->children[i].process.stdout_to_stderr = 1; - pp->children[i].process.no_stdin = 1; if (start_command(&pp->children[i].process)) { code = pp->start_failure(&pp->children[i].err, From patchwork Tue Dec 22 00:02:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 11985439 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A5CB9C433E6 for ; Tue, 22 Dec 2020 00:03:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 58ED922AED for ; Tue, 22 Dec 2020 00:03:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726525AbgLVADv (ORCPT ); Mon, 21 Dec 2020 19:03:51 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47966 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726515AbgLVADv (ORCPT ); Mon, 21 Dec 2020 19:03:51 -0500 Received: from mail-qk1-x74a.google.com (mail-qk1-x74a.google.com [IPv6:2607:f8b0:4864:20::74a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C1FEFC0611CE for ; Mon, 21 Dec 2020 16:02:48 -0800 (PST) Received: by mail-qk1-x74a.google.com with SMTP id k126so10205272qkf.8 for ; Mon, 21 Dec 2020 16:02:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=0mvt9+rG8Pl/+CQJWtaLV5p670AG+aRiSc9AkfIPHAs=; b=Hl55flAjPqniIFmona4K4OsZQga1OzaHKID9M9Y2jHLvJbM34S9wBgOFHLiM4V+8bP 0U7awdRBdPMUOjF3DvpSS5I+rfU+bUeOn+XvKBLuMshJHEW9jkjMDe/B/vibtzY4wVsx 2NhYKcx52zUsWzozqlmpj72GsG5jdTg4r6K6aKiqhNd3XFsNlZDO5Zi1+OpYhTrNqiWk U/polrXE2NVVUR38GIG3WvfhnW7BwscSVxH+Ke5JiWyjTYFzLhmmycpEyj7R15acu/iS x02wMTgZqbGiN7b/GCiktnWlKZu8qIILeyWSfYP4RfcKaoyZBlbH3EMpNPAZTLF/YtH3 vLEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=0mvt9+rG8Pl/+CQJWtaLV5p670AG+aRiSc9AkfIPHAs=; b=mYfApQcmKVy+osenAiy4zlUMB6LwtVgfxc7k44wB7yd5IvShtDEXLSFo1PwsheBe33 /knnuiHXnPVGxfidcdKYvvbWjvaplsO2ph2qcoPxuZd3Fqy5cZK0X8J1Q+KgXCZkuBQa Ge2wAdFwutGG0gmgcy5Ftp7/8nQThdeIuQ39Hd0wB4swHc2UTrQrIO0Ovn/eTKOJmbIQ hhCwzgV1PYkLN3UOVNDXNzY83rOT9jf+AccfHULgni7z1ODkjVY0r24Ck9L5VWhaDKae nTcK+5K/4VqMmg2EetQ4VJsBRzIT8k5EmglbZGdfxQrtOxBtPyEe3+fTTMnfDNXRmjDf ueHg== X-Gm-Message-State: AOAM531jyhU48wquuhQQnSqLbWVMezTe5e7GcdoA5iZVMrszhS34UeW/ DmMC7s/bvI+VvjN8idjx/B1a0HiZPa/mR9/iVsqOTcac2MNrcTY8vFPMTjaeiU+qwGC32ir8dnz TFo+HXCMO/duqPgo8EZHxzijCAia1DJ/rTh5k3y4UwX6ZfeoUw3d60fNJHblvNJguLjcrVbbcjA == X-Google-Smtp-Source: ABdhPJyzizdXAvSprghXvy1W1QNfzbcfqj6X8Mpj04wm3G/oOy851LHbbveSQUMyZW+6rQMwUQURsXTUaMnIgbKGedE= Sender: "emilyshaffer via sendgmr" X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:1ea0:b8ff:fe77:f690]) (user=emilyshaffer job=sendgmr) by 2002:ad4:4e4d:: with SMTP id eb13mr19283681qvb.6.1608595367810; Mon, 21 Dec 2020 16:02:47 -0800 (PST) Date: Mon, 21 Dec 2020 16:02:15 -0800 In-Reply-To: <20201222000220.1491091-1-emilyshaffer@google.com> Message-Id: <20201222000220.1491091-13-emilyshaffer@google.com> Mime-Version: 1.0 References: <20201222000220.1491091-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.29.2.490.gc7ae633391 Subject: [PATCH v7 12/17] hook: allow parallel hook execution From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In many cases, there's no reason not to allow hooks to execute in parallel. run_processes_parallel() is well-suited - it's a task queue that runs its housekeeping in series, which means users don't need to worry about thread safety on their callback data. True multithreaded execution with the async_* functions isn't necessary here. Synchronous hook execution can be achieved by only allowing 1 job to run at a time. Teach run_hooks() to use that function for simple hooks which don't require stdin or capture of stderr. Signed-off-by: Emily Shaffer --- Notes: Per AEvar's request - parallel hook execution on day zero. In most ways run_processes_parallel() worked great for me - but it didn't have great support for hooks where we pipe to and from. I had to add this support later in the series. Since I modified an existing and in-use library I'd appreciate a keen look on these patches. - Emily Documentation/config/hook.txt | 5 ++ Documentation/git-hook.txt | 14 +++- builtin/hook.c | 6 +- hook.c | 142 ++++++++++++++++++++++++++-------- hook.h | 28 ++++++- 5 files changed, 157 insertions(+), 38 deletions(-) diff --git a/Documentation/config/hook.txt b/Documentation/config/hook.txt index 75312754ae..a423d13781 100644 --- a/Documentation/config/hook.txt +++ b/Documentation/config/hook.txt @@ -12,3 +12,8 @@ hook.runHookDir:: Controls how hooks contained in your hookdir are executed. Can be any of "yes", "warn", "interactive", or "no". Defaults to "yes". See linkgit:git-hook[1] and linkgit:git-config[1] "core.hooksPath"). + +hook.jobs:: + Specifies how many hooks can be run simultaneously during parallelized + hook execution. If unspecified, defaults to the number of processors on + the current system. diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt index cce30a80d0..01cee4ad81 100644 --- a/Documentation/git-hook.txt +++ b/Documentation/git-hook.txt @@ -10,7 +10,7 @@ SYNOPSIS [verse] 'git hook' list 'git hook' run [(-e|--env)=...] [(-a|--arg)=...] [--to-stdin=] - + [(-j|--jobs) ] DESCRIPTION ----------- @@ -66,7 +66,7 @@ in the order they should be run, and print the config scope where the relevant `hook..command` was specified, not the `hookcmd` (if applicable). This output is human-readable and the format is subject to change over time. -run [(-e|--env)=...] [(-a|--arg)=...] [--to-stdin=] ``:: +run [(-e|--env)=...] [(-a|--arg)=...] [--to-stdin=] [(-j|--jobs)] ``:: Runs hooks configured for ``, in the same order displayed by `git hook list`. Hooks configured this way are run prepended with `sh -c`, so paths @@ -98,6 +98,16 @@ Specify environment variables to set for every hook that is run. Specify a file which will be streamed into stdin for every hook that is run. Each hook will receive the entire file from beginning to EOF. +-j:: +--jobs:: + Only valid for `run`. ++ +Specify how many hooks to run simultaneously. If this flag is not specified, use +the value of the `hook.jobs` config. If the config is not specified, use the +number of CPUs on the current system. Some hooks may be ineligible for +parallelization: for example, 'commit-msg' intends hooks modify the commit +message body and cannot be parallelized. + CONFIGURATION ------------- include::config/hook.txt[] diff --git a/builtin/hook.c b/builtin/hook.c index be104f2938..7fbc84ab64 100644 --- a/builtin/hook.c +++ b/builtin/hook.c @@ -10,7 +10,7 @@ static const char * const builtin_hook_usage[] = { N_("git hook list "), N_("git hook run [(-e|--env)=...] [(-a|--arg)=...]" - "[--to-stdin=] "), + "[--to-stdin=] [(-j|--jobs) ] "), NULL }; @@ -90,7 +90,7 @@ static int list(int argc, const char **argv, const char *prefix) static int run(int argc, const char **argv, const char *prefix) { struct strbuf hookname = STRBUF_INIT; - struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT; + struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_ASYNC; int rc = 0; struct option run_options[] = { @@ -100,6 +100,8 @@ static int run(int argc, const char **argv, const char *prefix) N_("argument to pass to hook")), OPT_STRING(0, "to-stdin", &opt.path_to_stdin, N_("path"), N_("file to read into hooks' stdin")), + OPT_INTEGER('j', "jobs", &opt.jobs, + N_("run up to hooks simultaneously")), OPT_END(), }; diff --git a/hook.c b/hook.c index ce5c443206..b190afa33b 100644 --- a/hook.c +++ b/hook.c @@ -136,6 +136,14 @@ enum hookdir_opt configured_hookdir_opt(void) return HOOKDIR_UNKNOWN; } +int configured_hook_jobs(void) +{ + int n = online_cpus(); + git_config_get_int("hook.jobs", &n); + + return n; +} + static int should_include_hookdir(const char *path, enum hookdir_opt cfg) { struct strbuf prompt = STRBUF_INIT; @@ -223,6 +231,7 @@ void run_hooks_opt_init(struct run_hooks_opt *o) strvec_init(&o->env); strvec_init(&o->args); o->run_hookdir = configured_hookdir_opt(); + o->jobs = configured_hook_jobs(); } int hook_exists(const char *hookname, enum hookdir_opt should_run_hookdir) @@ -246,11 +255,96 @@ void run_hooks_opt_clear(struct run_hooks_opt *o) strvec_clear(&o->args); } + +static int pick_next_hook(struct child_process *cp, + struct strbuf *out, + void *pp_cb, + void **pp_task_cb) +{ + struct hook_cb_data *hook_cb = pp_cb; + + struct hook *hook = list_entry(hook_cb->run_me, struct hook, list); + + if (hook_cb->head == hook_cb->run_me) + return 0; + + cp->env = hook_cb->options->env.v; + cp->stdout_to_stderr = 1; + cp->trace2_hook_name = hook->command.buf; + + /* reopen the file for stdin; run_command closes it. */ + if (hook_cb->options->path_to_stdin) { + cp->no_stdin = 0; + cp->in = xopen(hook_cb->options->path_to_stdin, O_RDONLY); + } else { + cp->no_stdin = 1; + } + + /* + * Commands from the config could be oneliners, but we know + * for certain that hookdir commands are not. + */ + if (hook->from_hookdir) + cp->use_shell = 0; + else + cp->use_shell = 1; + + /* add command */ + strvec_push(&cp->args, hook->command.buf); + + /* + * add passed-in argv, without expanding - let the user get back + * exactly what they put in + */ + strvec_pushv(&cp->args, hook_cb->options->args.v); + + /* Provide context for errors if necessary */ + *pp_task_cb = hook; + + /* Get the next entry ready */ + hook_cb->run_me = hook_cb->run_me->next; + + return 1; +} + +static int notify_start_failure(struct strbuf *out, + void *pp_cb, + void *pp_task_cp) +{ + struct hook_cb_data *hook_cb = pp_cb; + struct hook *attempted = pp_task_cp; + + /* |= rc in cb */ + hook_cb->rc |= 1; + + strbuf_addf(out, _("Couldn't start '%s', configured in '%s'\n"), + attempted->command.buf, + attempted->from_hookdir ? "hookdir" + : config_scope_name(attempted->origin)); + + /* NEEDSWORK: if halt_on_error is desired, do it here. */ + return 0; +} + +static int notify_hook_finished(int result, + struct strbuf *out, + void *pp_cb, + void *pp_task_cb) +{ + struct hook_cb_data *hook_cb = pp_cb; + + /* |= rc in cb */ + hook_cb->rc |= result; + + /* NEEDSWORK: if halt_on_error is desired, do it here. */ + return 0; +} + int run_hooks(const char *hookname, struct run_hooks_opt *options) { struct strbuf hookname_str = STRBUF_INIT; struct list_head *to_run, *pos = NULL, *tmp = NULL; - int rc = 0; + struct hook_cb_data cb_data = { 0, NULL, NULL, options }; if (!options) BUG("a struct run_hooks_opt must be provided to run_hooks"); @@ -260,41 +354,23 @@ int run_hooks(const char *hookname, struct run_hooks_opt *options) to_run = hook_list(&hookname_str); list_for_each_safe(pos, tmp, to_run) { - struct child_process hook_proc = CHILD_PROCESS_INIT; struct hook *hook = list_entry(pos, struct hook, list); - /* reopen the file for stdin; run_command closes it. */ - if (options->path_to_stdin) - hook_proc.in = xopen(options->path_to_stdin, O_RDONLY); - else - hook_proc.no_stdin = 1; - - hook_proc.env = options->env.v; - hook_proc.stdout_to_stderr = 1; - hook_proc.trace2_hook_name = hook->command.buf; - hook_proc.use_shell = 1; - - if (hook->from_hookdir) { - if (!should_include_hookdir(hook->command.buf, options->run_hookdir)) - continue; - /* - * Commands from the config could be oneliners, but we know - * for certain that hookdir commands are not. - */ - hook_proc.use_shell = 0; - } - - /* add command */ - strvec_push(&hook_proc.args, hook->command.buf); + if (hook->from_hookdir && + !should_include_hookdir(hook->command.buf, options->run_hookdir)) + list_del(pos); + } - /* - * add passed-in argv, without expanding - let the user get back - * exactly what they put in - */ - strvec_pushv(&hook_proc.args, options->args.v); + cb_data.head = to_run; + cb_data.run_me = to_run->next; - rc |= run_command(&hook_proc); - } + run_processes_parallel_tr2(options->jobs, + pick_next_hook, + notify_start_failure, + notify_hook_finished, + &cb_data, + "hook", + hookname); - return rc; + return cb_data.rc; } diff --git a/hook.h b/hook.h index e22a6db832..0d973d090f 100644 --- a/hook.h +++ b/hook.h @@ -37,6 +37,9 @@ enum hookdir_opt */ enum hookdir_opt configured_hookdir_opt(void); +/* Provides the number of threads to use for parallel hook execution. */ +int configured_hook_jobs(void); + struct run_hooks_opt { /* Environment vars to be set for each hook */ @@ -54,15 +57,38 @@ struct run_hooks_opt /* Path to file which should be piped to stdin for each hook */ const char *path_to_stdin; + + /* Number of threads to parallelize across */ + int jobs; }; -#define RUN_HOOKS_OPT_INIT { \ +/* + * Callback provided to feed_pipe_fn and consume_sideband_fn. + */ +struct hook_cb_data { + int rc; + struct list_head *head; + struct list_head *run_me; + struct run_hooks_opt *options; +}; + +#define RUN_HOOKS_OPT_INIT_SYNC { \ .env = STRVEC_INIT, \ .args = STRVEC_INIT, \ .path_to_stdin = NULL, \ + .jobs = 1, \ .run_hookdir = configured_hookdir_opt() \ } +#define RUN_HOOKS_OPT_INIT_ASYNC { \ + .env = STRVEC_INIT, \ + .args = STRVEC_INIT, \ + .path_to_stdin = NULL, \ + .jobs = configured_hook_jobs(), \ + .run_hookdir = configured_hookdir_opt() \ +} + + void run_hooks_opt_init(struct run_hooks_opt *o); void run_hooks_opt_clear(struct run_hooks_opt *o); From patchwork Tue Dec 22 00:02:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 11985445 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 08B28C433DB for ; Tue, 22 Dec 2020 00:03:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CE1A322AED for ; Tue, 22 Dec 2020 00:03:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726551AbgLVADz (ORCPT ); Mon, 21 Dec 2020 19:03:55 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47974 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726487AbgLVADw (ORCPT ); Mon, 21 Dec 2020 19:03:52 -0500 Received: from mail-qt1-x84a.google.com (mail-qt1-x84a.google.com [IPv6:2607:f8b0:4864:20::84a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 83F5EC0611CF for ; Mon, 21 Dec 2020 16:02:50 -0800 (PST) Received: by mail-qt1-x84a.google.com with SMTP id w3so9128895qti.17 for ; Mon, 21 Dec 2020 16:02:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=eOkRt3sHO+DZ9MfMNfj2MOvU3rJSy7G5czli6umbxR4=; b=AkzPb5X3A6kpRMK+irevjvYUy9huT3VUjClyE1b/B3G6DUNKVgRY7wtmdFGxqBddJt mys6yvKKRQ+iK8HXp7bcgZ+KvlEEtQoZ8OSLOmZDZjMTviXgWolqpE/TzUEPvm0ADsl9 f4aoKv9L+VGvGizNBw15xLaANZb4u2K3/seD4O/fOBRpikxODd6tbnTdBhBZS/qVOccK oDARVZr3xUU538wHogGOOmje1N/mIf+FzGoWx7SiolLb+3byIMdTiHx/NFMTxG5NeHT9 +tqf/0QM6RoiRjveetuSOPAPRH9poynPNAJiAo2ra6AlWQzqBvRTJcP22vWqr8bcxKov Z9XQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=eOkRt3sHO+DZ9MfMNfj2MOvU3rJSy7G5czli6umbxR4=; b=tKoYRBAtjc1K3IThsqNkc2FTGcqYDJP8ONzbfPWDmer7As1smvhEadIDgrPYJkqBvr wta4okXrwRKAJCsHw8WnhyFj7kfMOhyta+bpPmDZ3f8r+reFPkyxmdjTT4LlfGpeiY8h R2IYD2vgFjE7nkyLs+WLf4jTyGLZ0dqNFK11xh14km0/Urs8KaFX9BcOfgdrgtImBjMc GK2BzZyi6fz47fm0QCDEOGnWnamglnBrtl69DFrehS8RweDAqe1P6DwFatGFTkdXKC3a GiKybU2KJsstfyuf6sA7cLCDCjjsHZlbD5RGY1ep62J9HeqYI022mIhjZNHZQQ2lBOQl z+6w== X-Gm-Message-State: AOAM5300W/h8jCUcSjHkxJw4t8XenGoGRc8a9H0rwgnFrlx1BXz0Gn4X aYyOOQJqzDDeO5kPgkimf4nucrV3lKdRzz6uYjnLbOpK/2B8TAeLFFB75oZIolOVWDezF6DWVzg DnwZpmCP0xLusN4Bf/rRYPqa8qcO2V4ZmCUeRmzrpnGoSxfljF3a7aguWxIjOq/C3MsKTlMMJRg == X-Google-Smtp-Source: ABdhPJx3dY6P7iKxcP9In3280iYF5LpyVc8J74PR/RpHzfpUlV7N/bacsW9OXhWaoufMlUKZp+TpWLKR65oCwwV3Sz8= Sender: "emilyshaffer via sendgmr" X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:1ea0:b8ff:fe77:f690]) (user=emilyshaffer job=sendgmr) by 2002:a0c:da87:: with SMTP id z7mr19381539qvj.41.1608595369674; Mon, 21 Dec 2020 16:02:49 -0800 (PST) Date: Mon, 21 Dec 2020 16:02:16 -0800 In-Reply-To: <20201222000220.1491091-1-emilyshaffer@google.com> Message-Id: <20201222000220.1491091-14-emilyshaffer@google.com> Mime-Version: 1.0 References: <20201222000220.1491091-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.29.2.490.gc7ae633391 Subject: [PATCH v7 13/17] hook: allow specifying working directory for hooks From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Hooks like "post-checkout" require that hooks have a different working directory than the initial process. Pipe that directly through to struct child_process. Because we can just run 'git -C hook run ...' it shouldn't be necessary to pipe this option through the frontend. In fact, this reduces the possibility of users running hooks which affect some part of the filesystem outside of the repo in question. Signed-off-by: Emily Shaffer --- Notes: Needed later for "post-checkout" conversion. hook.c | 1 + hook.h | 5 +++++ 2 files changed, 6 insertions(+) diff --git a/hook.c b/hook.c index b190afa33b..eea90ec1d0 100644 --- a/hook.c +++ b/hook.c @@ -271,6 +271,7 @@ static int pick_next_hook(struct child_process *cp, cp->env = hook_cb->options->env.v; cp->stdout_to_stderr = 1; cp->trace2_hook_name = hook->command.buf; + cp->dir = hook_cb->options->dir; /* reopen the file for stdin; run_command closes it. */ if (hook_cb->options->path_to_stdin) { diff --git a/hook.h b/hook.h index 0d973d090f..8a7542610c 100644 --- a/hook.h +++ b/hook.h @@ -60,6 +60,9 @@ struct run_hooks_opt /* Number of threads to parallelize across */ int jobs; + + /* Path to initial working directory for subprocess */ + const char *dir; }; /* @@ -77,6 +80,7 @@ struct hook_cb_data { .args = STRVEC_INIT, \ .path_to_stdin = NULL, \ .jobs = 1, \ + .dir = NULL, \ .run_hookdir = configured_hookdir_opt() \ } @@ -85,6 +89,7 @@ struct hook_cb_data { .args = STRVEC_INIT, \ .path_to_stdin = NULL, \ .jobs = configured_hook_jobs(), \ + .dir = NULL, \ .run_hookdir = configured_hookdir_opt() \ } From patchwork Tue Dec 22 00:02:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 11985443 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 16440C433E0 for ; Tue, 22 Dec 2020 00:03:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D60FE22AED for ; Tue, 22 Dec 2020 00:03:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725961AbgLVADy (ORCPT ); Mon, 21 Dec 2020 19:03:54 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47976 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726531AbgLVADx (ORCPT ); Mon, 21 Dec 2020 19:03:53 -0500 Received: from mail-pl1-x64a.google.com (mail-pl1-x64a.google.com [IPv6:2607:f8b0:4864:20::64a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 35E53C0611D0 for ; Mon, 21 Dec 2020 16:02:52 -0800 (PST) Received: by mail-pl1-x64a.google.com with SMTP id d6so6535377plr.17 for ; Mon, 21 Dec 2020 16:02:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=Nv71DuML1QIaSxo4P2u/LBfhdUA0RnaHc/b/8aYsNa4=; b=p2E3Uty/3GVxH5WovAhIzTCB/CPuKpTTIp2YgqBkDt0Ya1psOmS3OBL8+dNOGYqzGC l79qNl18P+qhQ/A1SuPYFeO1P4xgf929Hw6d8IxAr7SA8RNKoJJqWmuY64fqlGR3R4w6 Hye7t6qLul8vka7eFMSR/VA+yhG9/uhQ7FfiVt67bVtiWiblq1PE3GItO09UYlm+9U42 F2iKuJbkkoRpU1OnekWxPN+O+3E7fPaOqoiOYfDm35ZaJHvhCxh1B3cm86PMlubtxDn0 fJFDiVFSPa4V1LijEnuZsJ0y/ukky3yJovfXf2QvdhcygTSchfp89GDbIkjUiVieWEBv gwZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=Nv71DuML1QIaSxo4P2u/LBfhdUA0RnaHc/b/8aYsNa4=; b=sLIRNcuGCHmoPbrsEw5ZQvi0NHm8U4trqpwJY0AUYpXBdGam0V1ALKS4Es3T2Ggc94 JSP7V0v8ZauRw+upeXaIoPnPahnIJh4N6lyjWALf9K+b4UTyzRhI+sck+1ggwF05W+aw IiA+tNjAT3Zkr2vQbTMIlt6YqUrd6xeFjx/Mvm/ib4qKvqjlUCcWoi9pvuFvh3ufgEN+ +4AQzng5iNgsEPQor5uwUMVyt35dvCaJ+sIYz6o56fsPpsvvim8onyJZ/ZD9mZWd62RH bsMqr//R+Ph1fZI9ncudtSZtDdnysN20MKb3fggQ9m2S0614MjRMsnmqbkyLju51Yexv SCbA== X-Gm-Message-State: AOAM530CLk9W3tplNeFlmqOQwllxyv9C4I1hzASxbkL1oHCZvJTY7W2q OnD+Bu1XIYSvEEtLdmQBaXodEvJ6jPybAxA7wN3V1ljJtwOZS+Sf+uV+zEzGKZdojGeWPutWVUn AmNfymfMk2MHqFSAS+IKukSloXdi9QoU9gnJ+Gx5khwAbUmtiUUhWHR5Z6l7xSa7NHwD9q5e4vA == X-Google-Smtp-Source: ABdhPJw20mAWDwFCiF/N+4MxjpCin950DiPCwUGSRjApCcsJmlDSYLoF5jEFuxCXPl4WsW2LBCzI3yhWQcyImBbAUh8= Sender: "emilyshaffer via sendgmr" X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:1ea0:b8ff:fe77:f690]) (user=emilyshaffer job=sendgmr) by 2002:a63:5004:: with SMTP id e4mr17327167pgb.338.1608595371555; Mon, 21 Dec 2020 16:02:51 -0800 (PST) Date: Mon, 21 Dec 2020 16:02:17 -0800 In-Reply-To: <20201222000220.1491091-1-emilyshaffer@google.com> Message-Id: <20201222000220.1491091-15-emilyshaffer@google.com> Mime-Version: 1.0 References: <20201222000220.1491091-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.29.2.490.gc7ae633391 Subject: [PATCH v7 14/17] run-command: add stdin callback for parallelization From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org If a user of the run_processes_parallel() API wants to pipe a large amount of information to stdin of each parallel command, that information could exceed the buffer of the pipe allocated for that process's stdin. Generally this is solved by repeatedly writing to child_process.in between calls to start_command() and finish_command(); run_processes_parallel() did not provide users an opportunity to access child_process at that time. Because the data might be extremely large (for example, a list of all refs received during a push from a client) simply taking a string_list or strbuf is not as scalable as using a callback; the rest of the run_processes_parallel() API also uses callbacks, so making this feature match the rest of the API reduces mental load on the user. Signed-off-by: Emily Shaffer --- builtin/fetch.c | 1 + builtin/submodule--helper.c | 2 +- hook.c | 1 + run-command.c | 54 +++++++++++++++++++++++++++++++++++-- run-command.h | 17 +++++++++++- submodule.c | 1 + t/helper/test-run-command.c | 31 ++++++++++++++++++--- t/t0061-run-command.sh | 30 +++++++++++++++++++++ 8 files changed, 129 insertions(+), 8 deletions(-) diff --git a/builtin/fetch.c b/builtin/fetch.c index ecf8537605..5e153b5193 100644 --- a/builtin/fetch.c +++ b/builtin/fetch.c @@ -1647,6 +1647,7 @@ static int fetch_multiple(struct string_list *list, int max_children) result = run_processes_parallel_tr2(max_children, &fetch_next_remote, &fetch_failed_to_start, + NULL, &fetch_finished, &state, "fetch", "parallel/fetch"); diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c index c30896c897..bb623c1852 100644 --- a/builtin/submodule--helper.c +++ b/builtin/submodule--helper.c @@ -2294,7 +2294,7 @@ static int update_submodules(struct submodule_update_clone *suc) int i; run_processes_parallel_tr2(suc->max_jobs, update_clone_get_next_task, - update_clone_start_failure, + update_clone_start_failure, NULL, update_clone_task_finished, suc, "submodule", "parallel/update"); diff --git a/hook.c b/hook.c index eea90ec1d0..312ede1251 100644 --- a/hook.c +++ b/hook.c @@ -368,6 +368,7 @@ int run_hooks(const char *hookname, struct run_hooks_opt *options) run_processes_parallel_tr2(options->jobs, pick_next_hook, notify_start_failure, + NULL, notify_hook_finished, &cb_data, "hook", diff --git a/run-command.c b/run-command.c index 80c8c97bc1..7b65c087f8 100644 --- a/run-command.c +++ b/run-command.c @@ -1548,6 +1548,7 @@ struct parallel_processes { get_next_task_fn get_next_task; start_failure_fn start_failure; + feed_pipe_fn feed_pipe; task_finished_fn task_finished; struct { @@ -1575,6 +1576,13 @@ static int default_start_failure(struct strbuf *out, return 0; } +static int default_feed_pipe(struct strbuf *pipe, + void *pp_cb, + void *pp_task_cb) +{ + return 1; +} + static int default_task_finished(int result, struct strbuf *out, void *pp_cb, @@ -1605,6 +1613,7 @@ static void pp_init(struct parallel_processes *pp, int n, get_next_task_fn get_next_task, start_failure_fn start_failure, + feed_pipe_fn feed_pipe, task_finished_fn task_finished, void *data) { @@ -1623,6 +1632,7 @@ static void pp_init(struct parallel_processes *pp, pp->get_next_task = get_next_task; pp->start_failure = start_failure ? start_failure : default_start_failure; + pp->feed_pipe = feed_pipe ? feed_pipe : default_feed_pipe; pp->task_finished = task_finished ? task_finished : default_task_finished; pp->nr_processes = 0; @@ -1715,6 +1725,37 @@ static int pp_start_one(struct parallel_processes *pp) return 0; } +static void pp_buffer_stdin(struct parallel_processes *pp) +{ + int i; + struct strbuf sb = STRBUF_INIT; + + /* Buffer stdin for each pipe. */ + for (i = 0; i < pp->max_processes; i++) { + if (pp->children[i].state == GIT_CP_WORKING && + pp->children[i].process.in > 0) { + int done; + strbuf_reset(&sb); + done = pp->feed_pipe(&sb, pp->data, + pp->children[i].data); + if (sb.len) { + if (write_in_full(pp->children[i].process.in, + sb.buf, sb.len) < 0) { + if (errno != EPIPE) + die_errno("write"); + done = 1; + } + } + if (done) { + close(pp->children[i].process.in); + pp->children[i].process.in = 0; + } + } + } + + strbuf_release(&sb); +} + static void pp_buffer_stderr(struct parallel_processes *pp, int output_timeout) { int i; @@ -1779,6 +1820,7 @@ static int pp_collect_finished(struct parallel_processes *pp) pp->nr_processes--; pp->children[i].state = GIT_CP_FREE; pp->pfd[i].fd = -1; + pp->children[i].process.in = 0; child_process_init(&pp->children[i].process); if (i != pp->output_owner) { @@ -1812,6 +1854,7 @@ static int pp_collect_finished(struct parallel_processes *pp) int run_processes_parallel(int n, get_next_task_fn get_next_task, start_failure_fn start_failure, + feed_pipe_fn feed_pipe, task_finished_fn task_finished, void *pp_cb) { @@ -1820,7 +1863,9 @@ int run_processes_parallel(int n, int spawn_cap = 4; struct parallel_processes pp; - pp_init(&pp, n, get_next_task, start_failure, task_finished, pp_cb); + sigchain_push(SIGPIPE, SIG_IGN); + + pp_init(&pp, n, get_next_task, start_failure, feed_pipe, task_finished, pp_cb); while (1) { for (i = 0; i < spawn_cap && !pp.shutdown && @@ -1837,6 +1882,7 @@ int run_processes_parallel(int n, } if (!pp.nr_processes) break; + pp_buffer_stdin(&pp); pp_buffer_stderr(&pp, output_timeout); pp_output(&pp); code = pp_collect_finished(&pp); @@ -1848,11 +1894,15 @@ int run_processes_parallel(int n, } pp_cleanup(&pp); + + sigchain_pop(SIGPIPE); + return 0; } int run_processes_parallel_tr2(int n, get_next_task_fn get_next_task, start_failure_fn start_failure, + feed_pipe_fn feed_pipe, task_finished_fn task_finished, void *pp_cb, const char *tr2_category, const char *tr2_label) { @@ -1862,7 +1912,7 @@ int run_processes_parallel_tr2(int n, get_next_task_fn get_next_task, ((n < 1) ? online_cpus() : n)); result = run_processes_parallel(n, get_next_task, start_failure, - task_finished, pp_cb); + feed_pipe, task_finished, pp_cb); trace2_region_leave(tr2_category, tr2_label, NULL); diff --git a/run-command.h b/run-command.h index 6472b38bde..e058c0e2c8 100644 --- a/run-command.h +++ b/run-command.h @@ -436,6 +436,20 @@ typedef int (*start_failure_fn)(struct strbuf *out, void *pp_cb, void *pp_task_cb); +/** + * This callback is called repeatedly on every child process who requests + * start_command() to create a pipe by setting child_process.in < 0. + * + * pp_cb is the callback cookie as passed into run_processes_parallel, and + * pp_task_cb is the callback cookie as passed into get_next_task_fn. + * The contents of 'send' will be read into the pipe and passed to the pipe. + * + * Return nonzero to close the pipe. + */ +typedef int (*feed_pipe_fn)(struct strbuf *pipe, + void *pp_cb, + void *pp_task_cb); + /** * This callback is called on every child process that finished processing. * @@ -470,10 +484,11 @@ typedef int (*task_finished_fn)(int result, int run_processes_parallel(int n, get_next_task_fn, start_failure_fn, + feed_pipe_fn, task_finished_fn, void *pp_cb); int run_processes_parallel_tr2(int n, get_next_task_fn, start_failure_fn, - task_finished_fn, void *pp_cb, + feed_pipe_fn, task_finished_fn, void *pp_cb, const char *tr2_category, const char *tr2_label); #endif diff --git a/submodule.c b/submodule.c index b3bb59f066..953f41818c 100644 --- a/submodule.c +++ b/submodule.c @@ -1638,6 +1638,7 @@ int fetch_populated_submodules(struct repository *r, run_processes_parallel_tr2(max_parallel_jobs, get_next_submodule, fetch_start_failure, + NULL, fetch_finish, &spf, "submodule", "parallel/fetch"); diff --git a/t/helper/test-run-command.c b/t/helper/test-run-command.c index 7ae03dc712..9348184d30 100644 --- a/t/helper/test-run-command.c +++ b/t/helper/test-run-command.c @@ -32,8 +32,13 @@ static int parallel_next(struct child_process *cp, return 0; strvec_pushv(&cp->args, d->argv); + cp->in = d->in; + cp->no_stdin = d->no_stdin; strbuf_addstr(err, "preloaded output of a child\n"); number_callbacks++; + + *task_cb = xmalloc(sizeof(int)); + *(int*)(*task_cb) = 2; return 1; } @@ -55,6 +60,17 @@ static int task_finished(int result, return 1; } +static int test_stdin(struct strbuf *pipe, void *cb, void *task_cb) +{ + int *lines_remaining = task_cb; + + if (*lines_remaining) + strbuf_addf(pipe, "sample stdin %d\n", --(*lines_remaining)); + + return !(*lines_remaining); +} + + struct testsuite { struct string_list tests, failed; int next; @@ -185,7 +201,7 @@ static int testsuite(int argc, const char **argv) suite.tests.nr, max_jobs); ret = run_processes_parallel(max_jobs, next_test, test_failed, - test_finished, &suite); + test_stdin, test_finished, &suite); if (suite.failed.nr > 0) { ret = 1; @@ -413,15 +429,22 @@ int cmd__run_command(int argc, const char **argv) if (!strcmp(argv[1], "run-command-parallel")) exit(run_processes_parallel(jobs, parallel_next, - NULL, NULL, &proc)); + NULL, NULL, NULL, &proc)); if (!strcmp(argv[1], "run-command-abort")) exit(run_processes_parallel(jobs, parallel_next, - NULL, task_finished, &proc)); + NULL, NULL, task_finished, &proc)); if (!strcmp(argv[1], "run-command-no-jobs")) exit(run_processes_parallel(jobs, no_job, - NULL, task_finished, &proc)); + NULL, NULL, task_finished, &proc)); + + if (!strcmp(argv[1], "run-command-stdin")) { + proc.in = -1; + proc.no_stdin = 0; + exit (run_processes_parallel(jobs, parallel_next, NULL, + test_stdin, NULL, &proc)); + } fprintf(stderr, "check usage\n"); return 1; diff --git a/t/t0061-run-command.sh b/t/t0061-run-command.sh index 7d599675e3..87759482ad 100755 --- a/t/t0061-run-command.sh +++ b/t/t0061-run-command.sh @@ -143,6 +143,36 @@ test_expect_success 'run_command runs in parallel with more tasks than jobs avai test_cmp expect actual ' +cat >expect <<-EOF +preloaded output of a child +listening for stdin: +sample stdin 1 +sample stdin 0 +preloaded output of a child +listening for stdin: +sample stdin 1 +sample stdin 0 +preloaded output of a child +listening for stdin: +sample stdin 1 +sample stdin 0 +preloaded output of a child +listening for stdin: +sample stdin 1 +sample stdin 0 +EOF + +test_expect_success 'run_command listens to stdin' ' + write_script stdin-script <<-\EOF && + echo "listening for stdin:" + while read line; do + echo "$line" + done + EOF + test-tool run-command run-command-stdin 2 ./stdin-script 2>actual && + test_cmp expect actual +' + cat >expect <<-EOF preloaded output of a child asking for a quick stop From patchwork Tue Dec 22 00:02:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 11985447 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 24B4CC433E0 for ; Tue, 22 Dec 2020 00:04:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E8AA2229C6 for ; Tue, 22 Dec 2020 00:04:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726560AbgLVAEB (ORCPT ); Mon, 21 Dec 2020 19:04:01 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47942 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725780AbgLVAEA (ORCPT ); Mon, 21 Dec 2020 19:04:00 -0500 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2A77BC0611E4 for ; Mon, 21 Dec 2020 16:02:54 -0800 (PST) Received: by mail-yb1-xb4a.google.com with SMTP id c9so15654112ybs.8 for ; Mon, 21 Dec 2020 16:02:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=2ZAE60cky6qzXQ9u1UPgPFumiFq3z7RBjfFezd3lxus=; b=R+Kih8mFcTJmT53WfDbXG3jDF8I1Wa052z2d0Mpki0CVgtWrqpeNk1utgEUSKj+5qs eBXeiD7bzPNcIAcXc2hzxUxJLefixBFwX7G9Dp7a1otQXe0PD1xDi2eSIk/mN6XKjDVo m+yBJzlDbHLxs9yx/9NnU2LYpU012tHEAjYzA8md/bJcnCFo/Jrg1jcFGI6htgFoAye+ kjrJO+GBaf2nB2ITGO1hrbHosFItgUGcvVg3ftXybW44y4wx9YCnId0zpCDmGRTUTNn/ J/agsMfcT5iyoJ5OK2drEQURAo2DmW6ouY3eJBXPtEYcT0pH2gAS/R7Zz6BHbN6Kbguu dMFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=2ZAE60cky6qzXQ9u1UPgPFumiFq3z7RBjfFezd3lxus=; b=qysPTojliyXr7zoe0TvJDPIsRFHhSOn26TZFlc+4EttlCZhW94b7OQiTPfqN3WIMUh Y5dxSeb4LBgd0JIl0D8Ca10cn9Gkw/W4MrAKd01ISJ7cdIc26uM85Xjl4V1DVnotxY2h 3Rk1hiRuyzXJyIunboMXj3jPf2JHf9FADqptILW7JnOV8ZLGKUwxjpGZvnjZB6JYWQpn JSq/2aIwCFmJMvtKMKmnsa9XjD54iewLtc1yjIrSo0JPPksONFpmPLwjlLdEjLk7PbI3 JFL+XjHlYJ9nUxCu3kcR56OBtGiRnzkgtCnl+4BcAIa59T3tvsYnzR0JlYNIaM5Lk2pL Ivug== X-Gm-Message-State: AOAM531IaKOmgm7ZsiiV6PanZNgFLBHPhOAdbLGbO+hZLM9d64pgxjCE H/FDYtnBa128ZokwmKY6kGzVKaLGAztia55q2Y3bt1/gHDRLV/YFrlHvnMgUV1fKYzb+iRXGe1H 98mb2+HbU1LJZsU19NENbegBa5RJwZL0ltNUN8lSwOUQJUpfjySb66zjDpDCGMxddNaZaAukbaA == X-Google-Smtp-Source: ABdhPJwGOEyUIsq1MbYoCCFixToP3NwTjsEHASVhrc6ceG5/tI0xHO0AzyeSedadKkAs7y6YagaZxBhFPpm20mJh0bs= Sender: "emilyshaffer via sendgmr" X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:1ea0:b8ff:fe77:f690]) (user=emilyshaffer job=sendgmr) by 2002:a25:9b45:: with SMTP id u5mr13924866ybo.331.1608595373352; Mon, 21 Dec 2020 16:02:53 -0800 (PST) Date: Mon, 21 Dec 2020 16:02:18 -0800 In-Reply-To: <20201222000220.1491091-1-emilyshaffer@google.com> Message-Id: <20201222000220.1491091-16-emilyshaffer@google.com> Mime-Version: 1.0 References: <20201222000220.1491091-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.29.2.490.gc7ae633391 Subject: [PATCH v7 15/17] hook: provide stdin by string_list or callback From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In cases where a hook requires only a small amount of information via stdin, it should be simple for users to provide a string_list alone. But in more complicated cases where the stdin is too large to hold in memory, let's provide a callback the users can populate line after line with instead. Signed-off-by: Emily Shaffer --- hook.c | 39 ++++++++++++++++++++++++++++++++++++++- hook.h | 25 +++++++++++++++++++++++++ 2 files changed, 63 insertions(+), 1 deletion(-) diff --git a/hook.c b/hook.c index 312ede1251..b63a34d0a6 100644 --- a/hook.c +++ b/hook.c @@ -9,6 +9,7 @@ void free_hook(struct hook *ptr) { if (ptr) { strbuf_release(&ptr->command); + free(ptr->feed_pipe_cb_data); free(ptr); } } @@ -38,6 +39,7 @@ static void append_or_move_hook(struct list_head *head, const char *command) strbuf_init(&to_add->command, 0); strbuf_addstr(&to_add->command, command); to_add->from_hookdir = 0; + to_add->feed_pipe_cb_data = NULL; } /* re-set the scope so we show where an override was specified */ @@ -253,9 +255,32 @@ void run_hooks_opt_clear(struct run_hooks_opt *o) { strvec_clear(&o->env); strvec_clear(&o->args); + string_list_clear(&o->str_stdin, 0); } +static int pipe_from_string_list(struct strbuf *pipe, void *pp_cb, void *pp_task_cb) +{ + int *item_idx; + struct hook *ctx = pp_task_cb; + struct string_list *to_pipe = &((struct hook_cb_data*)pp_cb)->options->str_stdin; + + /* Bootstrap the state manager if necessary. */ + if (!ctx->feed_pipe_cb_data) { + ctx->feed_pipe_cb_data = xmalloc(sizeof(unsigned int)); + *(int*)ctx->feed_pipe_cb_data = 0; + } + + item_idx = ctx->feed_pipe_cb_data; + + if (*item_idx < to_pipe->nr) { + strbuf_addf(pipe, "%s\n", to_pipe->items[*item_idx].string); + (*item_idx)++; + return 0; + } + return 1; +} + static int pick_next_hook(struct child_process *cp, struct strbuf *out, void *pp_cb, @@ -277,6 +302,10 @@ static int pick_next_hook(struct child_process *cp, if (hook_cb->options->path_to_stdin) { cp->no_stdin = 0; cp->in = xopen(hook_cb->options->path_to_stdin, O_RDONLY); + } else if (hook_cb->options->feed_pipe) { + /* ask for start_command() to make a pipe for us */ + cp->in = -1; + cp->no_stdin = 0; } else { cp->no_stdin = 1; } @@ -350,6 +379,14 @@ int run_hooks(const char *hookname, struct run_hooks_opt *options) if (!options) BUG("a struct run_hooks_opt must be provided to run_hooks"); + if ((options->path_to_stdin && options->str_stdin.nr) || + (options->path_to_stdin && options->feed_pipe) || + (options->str_stdin.nr && options->feed_pipe)) + BUG("choose only one method to populate stdin"); + + if (options->str_stdin.nr) + options->feed_pipe = &pipe_from_string_list; + strbuf_addstr(&hookname_str, hookname); to_run = hook_list(&hookname_str); @@ -368,7 +405,7 @@ int run_hooks(const char *hookname, struct run_hooks_opt *options) run_processes_parallel_tr2(options->jobs, pick_next_hook, notify_start_failure, - NULL, + options->feed_pipe, notify_hook_finished, &cb_data, "hook", diff --git a/hook.h b/hook.h index 8a7542610c..0ac83fa7ca 100644 --- a/hook.h +++ b/hook.h @@ -2,6 +2,7 @@ #include "list.h" #include "strbuf.h" #include "strvec.h" +#include "run-command.h" struct hook { @@ -14,6 +15,12 @@ struct hook /* The literal command to run. */ struct strbuf command; int from_hookdir; + + /* + * Use this to keep state for your feed_pipe_fn if you are using + * run_hooks_opt.feed_pipe. Otherwise, do not touch it. + */ + void *feed_pipe_cb_data; }; /* @@ -57,12 +64,24 @@ struct run_hooks_opt /* Path to file which should be piped to stdin for each hook */ const char *path_to_stdin; + /* Pipe each string to stdin, separated by newlines */ + struct string_list str_stdin; + /* + * Callback and state pointer to ask for more content to pipe to stdin. + * Will be called repeatedly, for each hook. See + * hook.c:pipe_from_stdin() for an example. Keep per-hook state in + * hook.feed_pipe_cb_data (per process). Keep initialization context in + * feed_pipe_ctx (shared by all processes). + */ + feed_pipe_fn feed_pipe; + void *feed_pipe_ctx; /* Number of threads to parallelize across */ int jobs; /* Path to initial working directory for subprocess */ const char *dir; + }; /* @@ -81,6 +100,9 @@ struct hook_cb_data { .path_to_stdin = NULL, \ .jobs = 1, \ .dir = NULL, \ + .str_stdin = STRING_LIST_INIT_DUP, \ + .feed_pipe = NULL, \ + .feed_pipe_ctx = NULL, \ .run_hookdir = configured_hookdir_opt() \ } @@ -90,6 +112,9 @@ struct hook_cb_data { .path_to_stdin = NULL, \ .jobs = configured_hook_jobs(), \ .dir = NULL, \ + .str_stdin = STRING_LIST_INIT_DUP, \ + .feed_pipe = NULL, \ + .feed_pipe_ctx = NULL, \ .run_hookdir = configured_hookdir_opt() \ } From patchwork Tue Dec 22 00:02:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 11985449 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E2A45C433DB for ; Tue, 22 Dec 2020 00:04:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A79F2229C6 for ; Tue, 22 Dec 2020 00:04:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726556AbgLVAEB (ORCPT ); Mon, 21 Dec 2020 19:04:01 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47940 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726314AbgLVAEA (ORCPT ); Mon, 21 Dec 2020 19:04:00 -0500 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1F085C0611BB for ; Mon, 21 Dec 2020 16:02:56 -0800 (PST) Received: by mail-yb1-xb4a.google.com with SMTP id l8so15802285ybj.16 for ; Mon, 21 Dec 2020 16:02:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=csQKTUt/cuwg2RX/ZtiJotCBX/bK2XzRgF0aBFgOdck=; b=MTeQhXHlQxLEvOT/bwqljticuL+13oSvDK8t/AzxbGIhBRFony7+Xtc0bFAKeLYFJQ 32v8czjeHol4q3tYMKzn3j06ZGD2hYY8zC4xKmbDxccvOHN4KHcauVdd+CQ6pP5SJNSm Qw9exOnV+Bpl8GCAEBayuk67sglfvE7LJS+YBLU6AlGAwXtElf6Xch7aTHov5JpS9qLd WL5favpGmx8hpyEKgM0z6qk93Utmv/R3+SvzT0OdpagMvyjRd8ii2JTZBNYhkeEqe/Cv 6971uun+GexlrrlA7i8/75RvzQ7jfVSylYfTdkjvAHWHVuQwRSzdLEj9OPSyOmawMjrI EZEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=csQKTUt/cuwg2RX/ZtiJotCBX/bK2XzRgF0aBFgOdck=; b=X4IDAnKmmYA0CbmI9bGdrmeUtTw4WkhSfZaA8Jt2GYZAw/fGlC9ngJAxB4kqsj4d8g kTX4RtrGWdAZdBgIBgLuzHwX/Uy4awnn03XPKchQhJ4LQdMSvkX3WNGS/pYVNTtVABYu YUGOxLpfL9u0RmDaB1fyHE1birDCetTXELw7vpregDbtama60wUvNfYIyNCHdqdCUJjs 0bWW47/oBpO755uKD1j//GXlaUpJVeejxXRIYRfAhi/y5zHwKOSgb9QZHRSpQ3Z7O1PJ aEvQGIZcoD6pWHrBG7rOJM9QWrhpj5vmdJj28mG4CTyRbGVCTxzlP95268YxLtPDzuFo dZJw== X-Gm-Message-State: AOAM531AcPUAieOkdSD5PRwgkR3zFYY1Kd50qlO3vyKG7GzoD1rtl0r9 Erk5KdSVBGEquJdoB0knsMG8Jsrkujl7TO/WBsmKhN3Qp/F/YuBVRbJKLkRyvlswJWfkbmhNRx+ SIrjJnoblTbapmo0gOFX1KKMwtpgo+Qbkof9xOWr/qqeSNCg1WuVGPwIjVh0E3i0PSBiPZwvvpA == X-Google-Smtp-Source: ABdhPJwUUEtxg5Exj6H4IHuXcXSPB5w+kZ22ouvxEKvmwUoPNHLVtHDd+QBAEX9mfz6+63shh72ncjRi9XQPXuSlWAE= Sender: "emilyshaffer via sendgmr" X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:1ea0:b8ff:fe77:f690]) (user=emilyshaffer job=sendgmr) by 2002:a25:6c8a:: with SMTP id h132mr25199009ybc.263.1608595375262; Mon, 21 Dec 2020 16:02:55 -0800 (PST) Date: Mon, 21 Dec 2020 16:02:19 -0800 In-Reply-To: <20201222000220.1491091-1-emilyshaffer@google.com> Message-Id: <20201222000220.1491091-17-emilyshaffer@google.com> Mime-Version: 1.0 References: <20201222000220.1491091-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.29.2.490.gc7ae633391 Subject: [PATCH v7 16/17] run-command: allow capturing of collated output From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Some callers, for example server-side hooks which wish to relay hook output to clients across a transport, want to capture what would normally print to stderr and do something else with it. Allow that via a callback. By calling the callback regardless of whether there's output available, we allow clients to send e.g. a keepalive if necessary. Because we expose a strbuf, not a fd or FILE*, there's no need to create a temporary pipe or similar - we can just skip the print to stderr and instead hand it to the caller. Signed-off-by: Emily Shaffer --- Notes: Originally when writing this patch I attempted to use a pipe in memory - but managing its lifetime was actually pretty tricky, and I found I could achieve the same thing with less code by doing it this way. Critique welcome, including "no, you really need to do it with a pipe". builtin/fetch.c | 2 +- builtin/submodule--helper.c | 2 +- hook.c | 1 + run-command.c | 33 +++++++++++++++++++++++++-------- run-command.h | 18 +++++++++++++++++- submodule.c | 2 +- t/helper/test-run-command.c | 25 ++++++++++++++++++++----- t/t0061-run-command.sh | 7 +++++++ 8 files changed, 73 insertions(+), 17 deletions(-) diff --git a/builtin/fetch.c b/builtin/fetch.c index 5e153b5193..6a634085d9 100644 --- a/builtin/fetch.c +++ b/builtin/fetch.c @@ -1647,7 +1647,7 @@ static int fetch_multiple(struct string_list *list, int max_children) result = run_processes_parallel_tr2(max_children, &fetch_next_remote, &fetch_failed_to_start, - NULL, + NULL, NULL, &fetch_finished, &state, "fetch", "parallel/fetch"); diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c index bb623c1852..8c543d33fd 100644 --- a/builtin/submodule--helper.c +++ b/builtin/submodule--helper.c @@ -2294,7 +2294,7 @@ static int update_submodules(struct submodule_update_clone *suc) int i; run_processes_parallel_tr2(suc->max_jobs, update_clone_get_next_task, - update_clone_start_failure, NULL, + update_clone_start_failure, NULL, NULL, update_clone_task_finished, suc, "submodule", "parallel/update"); diff --git a/hook.c b/hook.c index b63a34d0a6..1439322a29 100644 --- a/hook.c +++ b/hook.c @@ -406,6 +406,7 @@ int run_hooks(const char *hookname, struct run_hooks_opt *options) pick_next_hook, notify_start_failure, options->feed_pipe, + NULL, notify_hook_finished, &cb_data, "hook", diff --git a/run-command.c b/run-command.c index 7b65c087f8..0dce6bec83 100644 --- a/run-command.c +++ b/run-command.c @@ -1549,6 +1549,7 @@ struct parallel_processes { get_next_task_fn get_next_task; start_failure_fn start_failure; feed_pipe_fn feed_pipe; + consume_sideband_fn consume_sideband; task_finished_fn task_finished; struct { @@ -1614,6 +1615,7 @@ static void pp_init(struct parallel_processes *pp, get_next_task_fn get_next_task, start_failure_fn start_failure, feed_pipe_fn feed_pipe, + consume_sideband_fn consume_sideband, task_finished_fn task_finished, void *data) { @@ -1634,6 +1636,7 @@ static void pp_init(struct parallel_processes *pp, pp->start_failure = start_failure ? start_failure : default_start_failure; pp->feed_pipe = feed_pipe ? feed_pipe : default_feed_pipe; pp->task_finished = task_finished ? task_finished : default_task_finished; + pp->consume_sideband = consume_sideband; pp->nr_processes = 0; pp->output_owner = 0; @@ -1670,7 +1673,10 @@ static void pp_cleanup(struct parallel_processes *pp) * When get_next_task added messages to the buffer in its last * iteration, the buffered output is non empty. */ - strbuf_write(&pp->buffered_output, stderr); + if (pp->consume_sideband) + pp->consume_sideband(&pp->buffered_output, pp->data); + else + strbuf_write(&pp->buffered_output, stderr); strbuf_release(&pp->buffered_output); sigchain_pop_common(); @@ -1786,9 +1792,13 @@ static void pp_buffer_stderr(struct parallel_processes *pp, int output_timeout) static void pp_output(struct parallel_processes *pp) { int i = pp->output_owner; + if (pp->children[i].state == GIT_CP_WORKING && pp->children[i].err.len) { - strbuf_write(&pp->children[i].err, stderr); + if (pp->consume_sideband) + pp->consume_sideband(&pp->children[i].err, pp->data); + else + strbuf_write(&pp->children[i].err, stderr); strbuf_reset(&pp->children[i].err); } } @@ -1827,11 +1837,15 @@ static int pp_collect_finished(struct parallel_processes *pp) strbuf_addbuf(&pp->buffered_output, &pp->children[i].err); strbuf_reset(&pp->children[i].err); } else { - strbuf_write(&pp->children[i].err, stderr); + /* Output errors, then all other finished child processes */ + if (pp->consume_sideband) { + pp->consume_sideband(&pp->children[i].err, pp->data); + pp->consume_sideband(&pp->buffered_output, pp->data); + } else { + strbuf_write(&pp->children[i].err, stderr); + strbuf_write(&pp->buffered_output, stderr); + } strbuf_reset(&pp->children[i].err); - - /* Output all other finished child processes */ - strbuf_write(&pp->buffered_output, stderr); strbuf_reset(&pp->buffered_output); /* @@ -1855,6 +1869,7 @@ int run_processes_parallel(int n, get_next_task_fn get_next_task, start_failure_fn start_failure, feed_pipe_fn feed_pipe, + consume_sideband_fn consume_sideband, task_finished_fn task_finished, void *pp_cb) { @@ -1865,7 +1880,7 @@ int run_processes_parallel(int n, sigchain_push(SIGPIPE, SIG_IGN); - pp_init(&pp, n, get_next_task, start_failure, feed_pipe, task_finished, pp_cb); + pp_init(&pp, n, get_next_task, start_failure, feed_pipe, consume_sideband, task_finished, pp_cb); while (1) { for (i = 0; i < spawn_cap && !pp.shutdown && @@ -1903,6 +1918,7 @@ int run_processes_parallel(int n, int run_processes_parallel_tr2(int n, get_next_task_fn get_next_task, start_failure_fn start_failure, feed_pipe_fn feed_pipe, + consume_sideband_fn consume_sideband, task_finished_fn task_finished, void *pp_cb, const char *tr2_category, const char *tr2_label) { @@ -1912,7 +1928,8 @@ int run_processes_parallel_tr2(int n, get_next_task_fn get_next_task, ((n < 1) ? online_cpus() : n)); result = run_processes_parallel(n, get_next_task, start_failure, - feed_pipe, task_finished, pp_cb); + feed_pipe, consume_sideband, + task_finished, pp_cb); trace2_region_leave(tr2_category, tr2_label, NULL); diff --git a/run-command.h b/run-command.h index e058c0e2c8..2ad8271f56 100644 --- a/run-command.h +++ b/run-command.h @@ -450,6 +450,20 @@ typedef int (*feed_pipe_fn)(struct strbuf *pipe, void *pp_cb, void *pp_task_cb); +/** + * If this callback is provided, instead of collating process output to stderr, + * they will be collated into a new pipe. consume_sideband_fn will be called + * repeatedly. When output is available on that pipe, it will be contained in + * 'output'. But it will be called with an empty 'output' too, to allow for + * keepalives or similar operations if necessary. + * + * pp_cb is the callback cookie as passed into run_processes_parallel. + * + * Since this callback is provided with the collated output, no task cookie is + * provided. + */ +typedef void (*consume_sideband_fn)(struct strbuf *output, void *pp_cb); + /** * This callback is called on every child process that finished processing. * @@ -485,10 +499,12 @@ int run_processes_parallel(int n, get_next_task_fn, start_failure_fn, feed_pipe_fn, + consume_sideband_fn, task_finished_fn, void *pp_cb); int run_processes_parallel_tr2(int n, get_next_task_fn, start_failure_fn, - feed_pipe_fn, task_finished_fn, void *pp_cb, + feed_pipe_fn, consume_sideband_fn, + task_finished_fn, void *pp_cb, const char *tr2_category, const char *tr2_label); #endif diff --git a/submodule.c b/submodule.c index 953f41818c..215bff22d9 100644 --- a/submodule.c +++ b/submodule.c @@ -1638,7 +1638,7 @@ int fetch_populated_submodules(struct repository *r, run_processes_parallel_tr2(max_parallel_jobs, get_next_submodule, fetch_start_failure, - NULL, + NULL, NULL, fetch_finish, &spf, "submodule", "parallel/fetch"); diff --git a/t/helper/test-run-command.c b/t/helper/test-run-command.c index 9348184d30..d53db6d11c 100644 --- a/t/helper/test-run-command.c +++ b/t/helper/test-run-command.c @@ -51,6 +51,16 @@ static int no_job(struct child_process *cp, return 0; } +static void test_consume_sideband(struct strbuf *output, void *cb) +{ + FILE *sideband; + + sideband = fopen("./sideband", "a"); + + strbuf_write(output, sideband); + fclose(sideband); +} + static int task_finished(int result, struct strbuf *err, void *pp_cb, @@ -201,7 +211,7 @@ static int testsuite(int argc, const char **argv) suite.tests.nr, max_jobs); ret = run_processes_parallel(max_jobs, next_test, test_failed, - test_stdin, test_finished, &suite); + test_stdin, NULL, test_finished, &suite); if (suite.failed.nr > 0) { ret = 1; @@ -429,23 +439,28 @@ int cmd__run_command(int argc, const char **argv) if (!strcmp(argv[1], "run-command-parallel")) exit(run_processes_parallel(jobs, parallel_next, - NULL, NULL, NULL, &proc)); + NULL, NULL, NULL, NULL, &proc)); if (!strcmp(argv[1], "run-command-abort")) exit(run_processes_parallel(jobs, parallel_next, - NULL, NULL, task_finished, &proc)); + NULL, NULL, NULL, task_finished, &proc)); if (!strcmp(argv[1], "run-command-no-jobs")) exit(run_processes_parallel(jobs, no_job, - NULL, NULL, task_finished, &proc)); + NULL, NULL, NULL, task_finished, &proc)); if (!strcmp(argv[1], "run-command-stdin")) { proc.in = -1; proc.no_stdin = 0; exit (run_processes_parallel(jobs, parallel_next, NULL, - test_stdin, NULL, &proc)); + test_stdin, NULL, NULL, &proc)); } + if (!strcmp(argv[1], "run-command-sideband")) + exit(run_processes_parallel(jobs, parallel_next, NULL, NULL, + test_consume_sideband, NULL, + &proc)); + fprintf(stderr, "check usage\n"); return 1; } diff --git a/t/t0061-run-command.sh b/t/t0061-run-command.sh index 87759482ad..e99f6c7f44 100755 --- a/t/t0061-run-command.sh +++ b/t/t0061-run-command.sh @@ -143,6 +143,13 @@ test_expect_success 'run_command runs in parallel with more tasks than jobs avai test_cmp expect actual ' +test_expect_success 'run_command can divert output' ' + test_when_finished rm sideband && + test-tool run-command run-command-sideband 3 sh -c "printf \"%s\n%s\n\" Hello World" 2>actual && + test_must_be_empty actual && + test_cmp expect sideband +' + cat >expect <<-EOF preloaded output of a child listening for stdin: From patchwork Tue Dec 22 00:02:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 11985451 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4A784C433E0 for ; Tue, 22 Dec 2020 00:04:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 138FA22AED for ; Tue, 22 Dec 2020 00:04:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726564AbgLVAE0 (ORCPT ); Mon, 21 Dec 2020 19:04:26 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48060 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726048AbgLVAE0 (ORCPT ); Mon, 21 Dec 2020 19:04:26 -0500 Received: from mail-pl1-x649.google.com (mail-pl1-x649.google.com [IPv6:2607:f8b0:4864:20::649]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CF751C0619D2 for ; Mon, 21 Dec 2020 16:02:57 -0800 (PST) Received: by mail-pl1-x649.google.com with SMTP id y5so5922001plr.19 for ; Mon, 21 Dec 2020 16:02:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=S/Y46cKYes4W6B2lDw3dstWxrxCcTY2+4b0qG4ApfCM=; b=G64uK3cEGA0iD3aw4dRzK/eAVAKXcJUZDhhmF6/1e/jNKVbYYX4hjGC14G5jcmf0pm sB/U92UDE9RTUNFIfbYclyg3BHcyPaom94j5tZJ+AKTMNbdt/emTCBeb+yPTMye8u0iN SPoaLMhye6z/BDAmAeNEue3wyjDxkqFQ9gD34JpbVrACGj6hxNiULyoOSCyuckTEErx9 /nct2i3GSUi9wg0RZFNRkcFwFy5Uxb26dun9tgRmkbW5YJEZlePQmgtB7NI/rj8GnCMa yBvAoXRO/Ry8Gj5AmRTHyZeg+9SXSZZL5pTnTmLNlOQ+iielDhBN63/uB9CuO9CO7xPd Tydw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=S/Y46cKYes4W6B2lDw3dstWxrxCcTY2+4b0qG4ApfCM=; b=JSlwDFYXJRqV+ibzkb7ihgD4LcjA7V/+U0k2K15R+SB5JGU4lhK9AIvbByoeaO4lG0 HtPYXYqY6mg/2V2kekpyZ/HuaMwJ8GI6Tu92adJ8OfHD1ImEURCMu+BrBhHBAEZKj39u GEstnW427X0jLpIIe/z/114oEujQjJjlZJb464TtLsqWMe3Cyynv2jgzxDhMGJYPo4MH TEjygr5fIdfFldmG/K06fFMrbPXjeXsRoFlNTgy0Kv2lZ+eRBYvURsRas3TsvUY5ZELc Dwtv8YQd5CVBBAIYERP+iOEC5T8o34v0fBvTfO/kKgwVFz/jINV4iiX0gBtv7tezTQGM oqJw== X-Gm-Message-State: AOAM533CtQvsHSy8s2xOxx8SoXK60NJrfMlsgVQ3VDwZa611ElBhGaQ+ u32xSbbH2N7MZuQ1VJDaueBtOc5qRCMEOCE1+oXT/D+qzchDv5MiPs/oA4BQWzp6cBmbgY5zey/ a0PtsfcXo8btAPQi6+KUcB+veh0HIap7TWostkcCzHZwvpLWg7sxNn1eduDOa4QM4akuBO7cMGQ == X-Google-Smtp-Source: ABdhPJzncXfS5ZIBv+mHiYZn1z2Gn5FINzXIH+DEh7YnoTLNzAW0Wbl3+kYnWBHii/cSfsPjtnSybMrwD9EHcZbx5KE= Sender: "emilyshaffer via sendgmr" X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:1ea0:b8ff:fe77:f690]) (user=emilyshaffer job=sendgmr) by 2002:a17:90b:128f:: with SMTP id fw15mr19389373pjb.91.1608595377251; Mon, 21 Dec 2020 16:02:57 -0800 (PST) Date: Mon, 21 Dec 2020 16:02:20 -0800 In-Reply-To: <20201222000220.1491091-1-emilyshaffer@google.com> Message-Id: <20201222000220.1491091-18-emilyshaffer@google.com> Mime-Version: 1.0 References: <20201222000220.1491091-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.29.2.490.gc7ae633391 Subject: [PATCH v7 17/17] hooks: allow callers to capture output From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Some server-side hooks will require capturing output to send over sideband instead of printing directly to stderr. Expose that capability. Signed-off-by: Emily Shaffer --- Notes: You can see this in practice in the conversions for some of the push hooks, like 'receive-pack'. hook.c | 2 +- hook.h | 10 ++++++++++ 2 files changed, 11 insertions(+), 1 deletion(-) diff --git a/hook.c b/hook.c index 1439322a29..dc241f7ec5 100644 --- a/hook.c +++ b/hook.c @@ -406,7 +406,7 @@ int run_hooks(const char *hookname, struct run_hooks_opt *options) pick_next_hook, notify_start_failure, options->feed_pipe, - NULL, + options->consume_sideband, notify_hook_finished, &cb_data, "hook", diff --git a/hook.h b/hook.h index 0ac83fa7ca..1e92379bb8 100644 --- a/hook.h +++ b/hook.h @@ -76,6 +76,14 @@ struct run_hooks_opt feed_pipe_fn feed_pipe; void *feed_pipe_ctx; + /* + * Populate this to capture output and prevent it from being printed to + * stderr. This will be passed directly through to + * run_command:run_parallel_processes(). See t/helper/test-run-command.c + * for an example. + */ + consume_sideband_fn consume_sideband; + /* Number of threads to parallelize across */ int jobs; @@ -103,6 +111,7 @@ struct hook_cb_data { .str_stdin = STRING_LIST_INIT_DUP, \ .feed_pipe = NULL, \ .feed_pipe_ctx = NULL, \ + .consume_sideband = NULL, \ .run_hookdir = configured_hookdir_opt() \ } @@ -115,6 +124,7 @@ struct hook_cb_data { .str_stdin = STRING_LIST_INIT_DUP, \ .feed_pipe = NULL, \ .feed_pipe_ctx = NULL, \ + .consume_sideband = NULL, \ .run_hookdir = configured_hookdir_opt() \ } From patchwork Mon Dec 28 22:37:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 11991749 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-31.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6288DC433DB for ; Mon, 28 Dec 2020 23:23:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1A3A1222F9 for ; Mon, 28 Dec 2020 23:23:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729927AbgL1Wzj (ORCPT ); Mon, 28 Dec 2020 17:55:39 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53286 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729649AbgL1WiE (ORCPT ); Mon, 28 Dec 2020 17:38:04 -0500 Received: from mail-qt1-x84a.google.com (mail-qt1-x84a.google.com [IPv6:2607:f8b0:4864:20::84a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D886EC0613D6 for ; Mon, 28 Dec 2020 14:37:23 -0800 (PST) Received: by mail-qt1-x84a.google.com with SMTP id h7so6197393qtn.21 for ; Mon, 28 Dec 2020 14:37:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:subject:from:to:cc; bh=nfrg2zG1zF4wLlLGq+Xouw/vEtdcn+A9nGN2VNincK8=; b=VFoPJ59XUI6thhgIGVtYBkrkxWLYQ8BrCCQqIWK6eCgFj2iXOhVTXKfSrXKwLling5 WLNBzYTNTYPmXnuqJpi9pVZ0gex2k37BhTWkJ91R103thjaouF0o9ZiuxZeROvjHK/Uw yb82YRMRSIgJvyDRgP9pZKBgGAfM/NVCzj06XncOy2sZ2VwBriQOX2acCvBsgusudSat PTRYoXHX54AwO3khXuvpYcgc9YxMc8I7dOsBDUrtleNNtCUdcjEQkjDKyCtsiop8H3tb 1+819eJjuta4/SegzAHTy9Y4V4nHrO4tYelticWu1Uk5hU3qvlD8IVJENuA6yq3D2v/T 1vGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :subject:from:to:cc; bh=nfrg2zG1zF4wLlLGq+Xouw/vEtdcn+A9nGN2VNincK8=; b=afp7fg5ljiL5y0C2YTjzvf1k7TQeY+J9eUgjWWbT3dVoNu85KyrrHJxQkvLAJtYkzi CgOeVSW0Kp0mhNO6WaTwEgbbC9NoQ3xTtAkFO7rwnanEh+hKbJNh4WLEmALc0MMn1lEQ 3vAm0wZoofBz8yy2Xl8LkDaL5H7Z9GoBV4VzkYxxdF5KlSE3eMUh8zf5CMrCR8jdBBI+ X8Vj/psPyQOZxSIc71qsbANArRRNR46X+ZrS3MBmVOD8HzfPpRFQ7RydhwCadSGw3pjy LsXUB8rQ6YRwQXE/Y1gFncgRe3zorGPm1LsQCWUaT5IpYjuRF/cTd/jap8mpOU0OeRJP ufxg== X-Gm-Message-State: AOAM530AY0GvYuG/ePw5m6RJR9a6FyOu3Owe2t+zcfNwbf00ZRKonJoV GGPKzD0ggEMAHOMCIaKoYky2XXNWz7qF7KoNaHwDm9I3779Dq31mgdl3Poiriy+j24WChUeX2BX hbrmdH/1PbPujF6Fw+x7EGlc0CpaxctLRByyzsVaUcK9TQFlaOO/RLYJXmS/YAQtY5GY2tcaxig == X-Google-Smtp-Source: ABdhPJzmmlPcNjQLS0BuQqNtgQ8v5+UuabZju+rsxEDpWoB0ovguyuKA2klKbGUmvy9qUC3UQWzzxWSIisMHM7Ki7pU= Sender: "emilyshaffer via sendgmr" X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:1ea0:b8ff:fe77:f690]) (user=emilyshaffer job=sendgmr) by 2002:a0c:ac44:: with SMTP id m4mr49298794qvb.45.1609195042589; Mon, 28 Dec 2020 14:37:22 -0800 (PST) Date: Mon, 28 Dec 2020 14:37:16 -0800 In-Reply-To: <20201222000220.1491091-1-emilyshaffer@google.com> Message-Id: <20201228223716.275101-1-emilyshaffer@google.com> Mime-Version: 1.0 X-Mailer: git-send-email 2.29.2.490.gc7ae633391 Subject: [PATCH v3 18/17] doc: make git-hook.txt point of truth From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer , Jeff King , Junio C Hamano , James Ramsay , Jonathan Nieder , "brian m. carlson" , " =?utf-8?b?w4Z2YXIgQXJu?= =?utf-8?b?ZmrDtnLDsCBCamFybWFzb24=?= " , Phillip Wood , Josh Steadmon , Johannes Schindelin Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org By showing the list of all hooks in 'git help hook' for users to refer to, 'git help hook' becomes a one-stop shop for hook authorship. Since some may still have muscle memory for 'git help githooks', though, reference the 'git hook' commands and otherwise don't remove content. Signed-off-by: Emily Shaffer --- Sorry for the wonky subject. It seemed unnecessary to send the entirety of the topic again when there were no changes. (I was able to push this patch to my fork without -f, so indeed the rest is unchanged.) I'd really prefer if 'git help githooks' opens the 'git-hook.txt' manpage, but I couldn't figure out how to do that. This seemed like the next best thing to me - users can still find information at the old manpage, but get a little nudge towards the new manpage in case they didn't notice it. Extremely open to other notes about the direction of these docs; I'm not confident in anything except that it'd be annoying to have 'git help githooks' remain unchanged. - Emily Documentation/git-hook.txt | 5 + Documentation/githooks.txt | 701 +------------------------------------ 2 files changed, 13 insertions(+), 693 deletions(-) diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt index 01cee4ad81..cb8e383ec9 100644 --- a/Documentation/git-hook.txt +++ b/Documentation/git-hook.txt @@ -112,6 +112,11 @@ CONFIGURATION ------------- include::config/hook.txt[] +HOOKS +----- + +include::native-hooks.txt[] + GIT --- Part of the linkgit:git[1] suite diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt index c450f7a27e..4d06369924 100644 --- a/Documentation/githooks.txt +++ b/Documentation/githooks.txt @@ -8,15 +8,20 @@ githooks - Hooks used by Git SYNOPSIS -------- $GIT_DIR/hooks/* (or \`git config core.hooksPath`/*) +[verse] +'git hook' list +'git hook' run DESCRIPTION ----------- -Hooks are programs you can place in a hooks directory to trigger -actions at certain points in git's execution. Hooks that don't have +Hooks are programs you can place in a hooks directory or specify in the config +to trigger actions at certain points in Git's execution. Hooks that don't have the executable bit set are ignored. +For information about specifying hooks in the config, check linkgit:git-hook[1]. + By default the hooks directory is `$GIT_DIR/hooks`, but that can be changed via the `core.hooksPath` configuration variable (see linkgit:git-config[1]). @@ -42,697 +47,7 @@ The currently supported hooks are described below. HOOKS ----- -applypatch-msg -~~~~~~~~~~~~~~ - -This hook is invoked by linkgit:git-am[1]. It takes a single -parameter, the name of the file that holds the proposed commit -log message. Exiting with a non-zero status causes `git am` to abort -before applying the patch. - -The hook is allowed to edit the message file in place, and can -be used to normalize the message into some project standard -format. It can also be used to refuse the commit after inspecting -the message file. - -The default 'applypatch-msg' hook, when enabled, runs the -'commit-msg' hook, if the latter is enabled. - -Hooks run during 'applypatch-msg' will not be parallelized. - -pre-applypatch -~~~~~~~~~~~~~~ - -This hook is invoked by linkgit:git-am[1]. It takes no parameter, and is -invoked after the patch is applied, but before a commit is made. - -If it exits with non-zero status, then the working tree will not be -committed after applying the patch. - -It can be used to inspect the current working tree and refuse to -make a commit if it does not pass certain test. - -The default 'pre-applypatch' hook, when enabled, runs the -'pre-commit' hook, if the latter is enabled. - -Hooks run during 'pre-applypatch' will be run in parallel by default. - -post-applypatch -~~~~~~~~~~~~~~~ - -This hook is invoked by linkgit:git-am[1]. It takes no parameter, -and is invoked after the patch is applied and a commit is made. - -This hook is meant primarily for notification, and cannot affect -the outcome of `git am`. - -Hooks run during 'post-applypatch' will be run in parallel by default. - -pre-commit -~~~~~~~~~~ - -This hook is invoked by linkgit:git-commit[1], and can be bypassed -with the `--no-verify` option. It takes no parameters, and is -invoked before obtaining the proposed commit log message and -making a commit. Exiting with a non-zero status from this script -causes the `git commit` command to abort before creating a commit. - -The default 'pre-commit' hook, when enabled, catches introduction -of lines with trailing whitespaces and aborts the commit when -such a line is found. - -All the `git commit` hooks are invoked with the environment -variable `GIT_EDITOR=:` if the command will not bring up an editor -to modify the commit message. - -The default 'pre-commit' hook, when enabled--and with the -`hooks.allownonascii` config option unset or set to false--prevents -the use of non-ASCII filenames. - -Hooks executed during 'pre-commit' will not be parallelized. - -pre-merge-commit -~~~~~~~~~~~~~~~~ - -This hook is invoked by linkgit:git-merge[1], and can be bypassed -with the `--no-verify` option. It takes no parameters, and is -invoked after the merge has been carried out successfully and before -obtaining the proposed commit log message to -make a commit. Exiting with a non-zero status from this script -causes the `git merge` command to abort before creating a commit. - -The default 'pre-merge-commit' hook, when enabled, runs the -'pre-commit' hook, if the latter is enabled. - -This hook is invoked with the environment variable -`GIT_EDITOR=:` if the command will not bring up an editor -to modify the commit message. - -If the merge cannot be carried out automatically, the conflicts -need to be resolved and the result committed separately (see -linkgit:git-merge[1]). At that point, this hook will not be executed, -but the 'pre-commit' hook will, if it is enabled. - -Hooks executed during 'pre-merge-commit' will not be parallelized. - -prepare-commit-msg -~~~~~~~~~~~~~~~~~~ - -This hook is invoked by linkgit:git-commit[1] right after preparing the -default log message, and before the editor is started. - -It takes one to three parameters. The first is the name of the file -that contains the commit log message. The second is the source of the commit -message, and can be: `message` (if a `-m` or `-F` option was -given); `template` (if a `-t` option was given or the -configuration option `commit.template` is set); `merge` (if the -commit is a merge or a `.git/MERGE_MSG` file exists); `squash` -(if a `.git/SQUASH_MSG` file exists); or `commit`, followed by -a commit SHA-1 (if a `-c`, `-C` or `--amend` option was given). - -If the exit status is non-zero, `git commit` will abort. - -The purpose of the hook is to edit the message file in place, and -it is not suppressed by the `--no-verify` option. A non-zero exit -means a failure of the hook and aborts the commit. It should not -be used as replacement for pre-commit hook. - -The sample `prepare-commit-msg` hook that comes with Git removes the -help message found in the commented portion of the commit template. - -Hooks executed during 'prepare-commit-msg' will not be parallelized. - -commit-msg -~~~~~~~~~~ - -This hook is invoked by linkgit:git-commit[1] and linkgit:git-merge[1], and can be -bypassed with the `--no-verify` option. It takes a single parameter, -the name of the file that holds the proposed commit log message. -Exiting with a non-zero status causes the command to abort. - -The hook is allowed to edit the message file in place, and can be used -to normalize the message into some project standard format. It -can also be used to refuse the commit after inspecting the message -file. - -The default 'commit-msg' hook, when enabled, detects duplicate -`Signed-off-by` trailers, and aborts the commit if one is found. - -Hooks executed during 'commit-msg' will not be parallelized. - -post-commit -~~~~~~~~~~~ - -This hook is invoked by linkgit:git-commit[1]. It takes no parameters, and is -invoked after a commit is made. - -This hook is meant primarily for notification, and cannot affect -the outcome of `git commit`. - -Hooks executed during 'post-commit' will run in parallel by default. - -pre-rebase -~~~~~~~~~~ - -This hook is called by linkgit:git-rebase[1] and can be used to prevent a -branch from getting rebased. The hook may be called with one or -two parameters. The first parameter is the upstream from which -the series was forked. The second parameter is the branch being -rebased, and is not set when rebasing the current branch. - -Hooks executed during 'pre-rebase' will run in parallel by default. - -post-checkout -~~~~~~~~~~~~~ - -This hook is invoked when a linkgit:git-checkout[1] or -linkgit:git-switch[1] is run after having updated the -worktree. The hook is given three parameters: the ref of the previous HEAD, -the ref of the new HEAD (which may or may not have changed), and a flag -indicating whether the checkout was a branch checkout (changing branches, -flag=1) or a file checkout (retrieving a file from the index, flag=0). -This hook cannot affect the outcome of `git switch` or `git checkout`, -other than that the hook's exit status becomes the exit status of -these two commands. - -It is also run after linkgit:git-clone[1], unless the `--no-checkout` (`-n`) option is -used. The first parameter given to the hook is the null-ref, the second the -ref of the new HEAD and the flag is always 1. Likewise for `git worktree add` -unless `--no-checkout` is used. - -This hook can be used to perform repository validity checks, auto-display -differences from the previous HEAD if different, or set working dir metadata -properties. - -Hooks executed during 'post-checkout' will not be parallelized. - -post-merge -~~~~~~~~~~ - -This hook is invoked by linkgit:git-merge[1], which happens when a `git pull` -is done on a local repository. The hook takes a single parameter, a status -flag specifying whether or not the merge being done was a squash merge. -This hook cannot affect the outcome of `git merge` and is not executed, -if the merge failed due to conflicts. - -This hook can be used in conjunction with a corresponding pre-commit hook to -save and restore any form of metadata associated with the working tree -(e.g.: permissions/ownership, ACLS, etc). See contrib/hooks/setgitperms.perl -for an example of how to do this. - -Hooks executed during 'post-merge' will run in parallel by default. - -pre-push -~~~~~~~~ - -This hook is called by linkgit:git-push[1] and can be used to prevent -a push from taking place. The hook is called with two parameters -which provide the name and location of the destination remote, if a -named remote is not being used both values will be the same. - -Information about what is to be pushed is provided on the hook's standard -input with lines of the form: - - SP SP SP LF - -For instance, if the command +git push origin master:foreign+ were run the -hook would receive a line like the following: - - refs/heads/master 67890 refs/heads/foreign 12345 - -although the full, 40-character SHA-1s would be supplied. If the foreign ref -does not yet exist the `` will be 40 `0`. If a ref is to be -deleted, the `` will be supplied as `(delete)` and the `` will be 40 `0`. If the local commit was specified by something other -than a name which could be expanded (such as `HEAD~`, or a SHA-1) it will be -supplied as it was originally given. - -If this hook exits with a non-zero status, `git push` will abort without -pushing anything. Information about why the push is rejected may be sent -to the user by writing to standard error. - -Hooks executed during 'pre-push' will run in parallel by default. - -[[pre-receive]] -pre-receive -~~~~~~~~~~~ - -This hook is invoked by linkgit:git-receive-pack[1] when it reacts to -`git push` and updates reference(s) in its repository. -Just before starting to update refs on the remote repository, the -pre-receive hook is invoked. Its exit status determines the success -or failure of the update. - -This hook executes once for the receive operation. It takes no -arguments, but for each ref to be updated it receives on standard -input a line of the format: - - SP SP LF - -where `` is the old object name stored in the ref, -`` is the new object name to be stored in the ref and -`` is the full name of the ref. -When creating a new ref, `` is 40 `0`. - -If the hook exits with non-zero status, none of the refs will be -updated. If the hook exits with zero, updating of individual refs can -still be prevented by the <> hook. - -Both standard output and standard error output are forwarded to -`git send-pack` on the other end, so you can simply `echo` messages -for the user. - -The number of push options given on the command line of -`git push --push-option=...` can be read from the environment -variable `GIT_PUSH_OPTION_COUNT`, and the options themselves are -found in `GIT_PUSH_OPTION_0`, `GIT_PUSH_OPTION_1`,... -If it is negotiated to not use the push options phase, the -environment variables will not be set. If the client selects -to use push options, but doesn't transmit any, the count variable -will be set to zero, `GIT_PUSH_OPTION_COUNT=0`. - -See the section on "Quarantine Environment" in -linkgit:git-receive-pack[1] for some caveats. - -Hooks executed during 'pre-receive' will not be parallelized. - -[[update]] -update -~~~~~~ - -This hook is invoked by linkgit:git-receive-pack[1] when it reacts to -`git push` and updates reference(s) in its repository. -Just before updating the ref on the remote repository, the update hook -is invoked. Its exit status determines the success or failure of -the ref update. - -The hook executes once for each ref to be updated, and takes -three parameters: - - - the name of the ref being updated, - - the old object name stored in the ref, - - and the new object name to be stored in the ref. - -A zero exit from the update hook allows the ref to be updated. -Exiting with a non-zero status prevents `git receive-pack` -from updating that ref. - -This hook can be used to prevent 'forced' update on certain refs by -making sure that the object name is a commit object that is a -descendant of the commit object named by the old object name. -That is, to enforce a "fast-forward only" policy. - -It could also be used to log the old..new status. However, it -does not know the entire set of branches, so it would end up -firing one e-mail per ref when used naively, though. The -<> hook is more suited to that. - -In an environment that restricts the users' access only to git -commands over the wire, this hook can be used to implement access -control without relying on filesystem ownership and group -membership. See linkgit:git-shell[1] for how you might use the login -shell to restrict the user's access to only git commands. - -Both standard output and standard error output are forwarded to -`git send-pack` on the other end, so you can simply `echo` messages -for the user. - -The default 'update' hook, when enabled--and with -`hooks.allowunannotated` config option unset or set to false--prevents -unannotated tags to be pushed. - -Hooks executed during 'update' are run in parallel by default. - -[[proc-receive]] -proc-receive -~~~~~~~~~~~~ - -This hook is invoked by linkgit:git-receive-pack[1]. If the server has -set the multi-valued config variable `receive.procReceiveRefs`, and the -commands sent to 'receive-pack' have matching reference names, these -commands will be executed by this hook, instead of by the internal -`execute_commands()` function. This hook is responsible for updating -the relevant references and reporting the results back to 'receive-pack'. - -This hook executes once for the receive operation. It takes no -arguments, but uses a pkt-line format protocol to communicate with -'receive-pack' to read commands, push-options and send results. In the -following example for the protocol, the letter 'S' stands for -'receive-pack' and the letter 'H' stands for this hook. - - # Version and features negotiation. - S: PKT-LINE(version=1\0push-options atomic...) - S: flush-pkt - H: PKT-LINE(version=1\0push-options...) - H: flush-pkt - - # Send commands from server to the hook. - S: PKT-LINE( ) - S: ... ... - S: flush-pkt - # Send push-options only if the 'push-options' feature is enabled. - S: PKT-LINE(push-option) - S: ... ... - S: flush-pkt - - # Receive result from the hook. - # OK, run this command successfully. - H: PKT-LINE(ok ) - # NO, I reject it. - H: PKT-LINE(ng ) - # Fall through, let 'receive-pack' to execute it. - H: PKT-LINE(ok ) - H: PKT-LINE(option fall-through) - # OK, but has an alternate reference. The alternate reference name - # and other status can be given in option directives. - H: PKT-LINE(ok ) - H: PKT-LINE(option refname ) - H: PKT-LINE(option old-oid ) - H: PKT-LINE(option new-oid ) - H: PKT-LINE(option forced-update) - H: ... ... - H: flush-pkt - -Each command for the 'proc-receive' hook may point to a pseudo-reference -and always has a zero-old as its old-oid, while the 'proc-receive' hook -may update an alternate reference and the alternate reference may exist -already with a non-zero old-oid. For this case, this hook will use -"option" directives to report extended attributes for the reference given -by the leading "ok" directive. - -The report of the commands of this hook should have the same order as -the input. The exit status of the 'proc-receive' hook only determines -the success or failure of the group of commands sent to it, unless -atomic push is in use. - -It is forbidden to specify more than one hook for 'proc-receive'. If a -globally-configured 'proc-receive' must be overridden, use -'hookcmd..skip = true' to ignore it. - -[[post-receive]] -post-receive -~~~~~~~~~~~~ - -This hook is invoked by linkgit:git-receive-pack[1] when it reacts to -`git push` and updates reference(s) in its repository. -It executes on the remote repository once after all the refs have -been updated. - -This hook executes once for the receive operation. It takes no -arguments, but gets the same information as the -<> -hook does on its standard input. - -This hook does not affect the outcome of `git receive-pack`, as it -is called after the real work is done. - -This supersedes the <> hook in that it gets -both old and new values of all the refs in addition to their -names. - -Both standard output and standard error output are forwarded to -`git send-pack` on the other end, so you can simply `echo` messages -for the user. - -The default 'post-receive' hook is empty, but there is -a sample script `post-receive-email` provided in the `contrib/hooks` -directory in Git distribution, which implements sending commit -emails. - -The number of push options given on the command line of -`git push --push-option=...` can be read from the environment -variable `GIT_PUSH_OPTION_COUNT`, and the options themselves are -found in `GIT_PUSH_OPTION_0`, `GIT_PUSH_OPTION_1`,... -If it is negotiated to not use the push options phase, the -environment variables will not be set. If the client selects -to use push options, but doesn't transmit any, the count variable -will be set to zero, `GIT_PUSH_OPTION_COUNT=0`. - -Hooks executed during 'post-receive' are run in parallel by default. - -[[post-update]] -post-update -~~~~~~~~~~~ - -This hook is invoked by linkgit:git-receive-pack[1] when it reacts to -`git push` and updates reference(s) in its repository. -It executes on the remote repository once after all the refs have -been updated. - -It takes a variable number of parameters, each of which is the -name of ref that was actually updated. - -This hook is meant primarily for notification, and cannot affect -the outcome of `git receive-pack`. - -The 'post-update' hook can tell what are the heads that were pushed, -but it does not know what their original and updated values are, -so it is a poor place to do log old..new. The -<> hook does get both original and -updated values of the refs. You might consider it instead if you need -them. - -When enabled, the default 'post-update' hook runs -`git update-server-info` to keep the information used by dumb -transports (e.g., HTTP) up to date. If you are publishing -a Git repository that is accessible via HTTP, you should -probably enable this hook. - -Both standard output and standard error output are forwarded to -`git send-pack` on the other end, so you can simply `echo` messages -for the user. - -Hooks run during 'post-update' will be run in parallel by default. - -reference-transaction -~~~~~~~~~~~~~~~~~~~~~ - -This hook is invoked by any Git command that performs reference -updates. It executes whenever a reference transaction is prepared, -committed or aborted and may thus get called multiple times. - -The hook takes exactly one argument, which is the current state the -given reference transaction is in: - - - "prepared": All reference updates have been queued to the - transaction and references were locked on disk. - - - "committed": The reference transaction was committed and all - references now have their respective new value. - - - "aborted": The reference transaction was aborted, no changes - were performed and the locks have been released. - -For each reference update that was added to the transaction, the hook -receives on standard input a line of the format: - - SP SP LF - -The exit status of the hook is ignored for any state except for the -"prepared" state. In the "prepared" state, a non-zero exit status will -cause the transaction to be aborted. The hook will not be called with -"aborted" state in that case. - -Hooks run during 'reference-transaction' will be run in parallel by default. - -push-to-checkout -~~~~~~~~~~~~~~~~ - -This hook is invoked by linkgit:git-receive-pack[1] when it reacts to -`git push` and updates reference(s) in its repository, and when -the push tries to update the branch that is currently checked out -and the `receive.denyCurrentBranch` configuration variable is set to -`updateInstead`. Such a push by default is refused if the working -tree and the index of the remote repository has any difference from -the currently checked out commit; when both the working tree and the -index match the current commit, they are updated to match the newly -pushed tip of the branch. This hook is to be used to override the -default behaviour. - -The hook receives the commit with which the tip of the current -branch is going to be updated. It can exit with a non-zero status -to refuse the push (when it does so, it must not modify the index or -the working tree). Or it can make any necessary changes to the -working tree and to the index to bring them to the desired state -when the tip of the current branch is updated to the new commit, and -exit with a zero status. - -For example, the hook can simply run `git read-tree -u -m HEAD "$1"` -in order to emulate `git fetch` that is run in the reverse direction -with `git push`, as the two-tree form of `git read-tree -u -m` is -essentially the same as `git switch` or `git checkout` -that switches branches while -keeping the local changes in the working tree that do not interfere -with the difference between the branches. - -Hooks executed during 'push-to-checkout' will not be parallelized. - -pre-auto-gc -~~~~~~~~~~~ - -This hook is invoked by `git gc --auto` (see linkgit:git-gc[1]). It -takes no parameter, and exiting with non-zero status from this script -causes the `git gc --auto` to abort. - -Hooks run during 'pre-auto-gc' will be run in parallel by default. - -post-rewrite -~~~~~~~~~~~~ - -This hook is invoked by commands that rewrite commits -(linkgit:git-commit[1] when called with `--amend` and -linkgit:git-rebase[1]; however, full-history (re)writing tools like -linkgit:git-fast-import[1] or -https://github.com/newren/git-filter-repo[git-filter-repo] typically -do not call it!). Its first argument denotes the command it was -invoked by: currently one of `amend` or `rebase`. Further -command-dependent arguments may be passed in the future. - -The hook receives a list of the rewritten commits on stdin, in the -format - - SP [ SP ] LF - -The 'extra-info' is again command-dependent. If it is empty, the -preceding SP is also omitted. Currently, no commands pass any -'extra-info'. - -The hook always runs after the automatic note copying (see -"notes.rewrite." in linkgit:git-config[1]) has happened, and -thus has access to these notes. - -Hooks run during 'post-rewrite' will be run in parallel by default. - -The following command-specific comments apply: - -rebase:: - For the 'squash' and 'fixup' operation, all commits that were - squashed are listed as being rewritten to the squashed commit. - This means that there will be several lines sharing the same - 'new-sha1'. -+ -The commits are guaranteed to be listed in the order that they were -processed by rebase. - -sendemail-validate -~~~~~~~~~~~~~~~~~~ - -This hook is invoked by linkgit:git-send-email[1]. It takes a single parameter, -the name of the file that holds the e-mail to be sent. Exiting with a -non-zero status causes `git send-email` to abort before sending any -e-mails. - -fsmonitor-watchman -~~~~~~~~~~~~~~~~~~ - -This hook is invoked when the configuration option `core.fsmonitor` is -set to `.git/hooks/fsmonitor-watchman` or `.git/hooks/fsmonitor-watchmanv2` -depending on the version of the hook to use. - -Version 1 takes two arguments, a version (1) and the time in elapsed -nanoseconds since midnight, January 1, 1970. - -Version 2 takes two arguments, a version (2) and a token that is used -for identifying changes since the token. For watchman this would be -a clock id. This version must output to stdout the new token followed -by a NUL before the list of files. - -The hook should output to stdout the list of all files in the working -directory that may have changed since the requested time. The logic -should be inclusive so that it does not miss any potential changes. -The paths should be relative to the root of the working directory -and be separated by a single NUL. - -It is OK to include files which have not actually changed. All changes -including newly-created and deleted files should be included. When -files are renamed, both the old and the new name should be included. - -Git will limit what files it checks for changes as well as which -directories are checked for untracked files based on the path names -given. - -An optimized way to tell git "all files have changed" is to return -the filename `/`. - -The exit status determines whether git will use the data from the -hook to limit its search. On error, it will fall back to verifying -all files and folders. - -p4-changelist -~~~~~~~~~~~~~ - -This hook is invoked by `git-p4 submit`. - -The `p4-changelist` hook is executed after the changelist -message has been edited by the user. It can be bypassed with the -`--no-verify` option. It takes a single parameter, the name -of the file that holds the proposed changelist text. Exiting -with a non-zero status causes the command to abort. - -The hook is allowed to edit the changelist file and can be used -to normalize the text into some project standard format. It can -also be used to refuse the Submit after inspect the message file. - -Run `git-p4 submit --help` for details. - -p4-prepare-changelist -~~~~~~~~~~~~~~~~~~~~~ - -This hook is invoked by `git-p4 submit`. - -The `p4-prepare-changelist` hook is executed right after preparing -the default changelist message and before the editor is started. -It takes one parameter, the name of the file that contains the -changelist text. Exiting with a non-zero status from the script -will abort the process. - -The purpose of the hook is to edit the message file in place, -and it is not supressed by the `--no-verify` option. This hook -is called even if `--prepare-p4-only` is set. - -Run `git-p4 submit --help` for details. - -p4-post-changelist -~~~~~~~~~~~~~~~~~~ - -This hook is invoked by `git-p4 submit`. - -The `p4-post-changelist` hook is invoked after the submit has -successfully occurred in P4. It takes no parameters and is meant -primarily for notification and cannot affect the outcome of the -git p4 submit action. - -Run `git-p4 submit --help` for details. - -p4-pre-submit -~~~~~~~~~~~~~ - -This hook is invoked by `git-p4 submit`. It takes no parameters and nothing -from standard input. Exiting with non-zero status from this script prevent -`git-p4 submit` from launching. It can be bypassed with the `--no-verify` -command line option. Run `git-p4 submit --help` for details. - - - -post-index-change -~~~~~~~~~~~~~~~~~ - -This hook is invoked when the index is written in read-cache.c -do_write_locked_index. - -The first parameter passed to the hook is the indicator for the -working directory being updated. "1" meaning working directory -was updated or "0" when the working directory was not updated. - -The second parameter passed to the hook is the indicator for whether -or not the index was updated and the skip-worktree bit could have -changed. "1" meaning skip-worktree bits could have been updated -and "0" meaning they were not. - -Only one parameter should be set to "1" when the hook runs. The hook -running passing "1", "1" should not be possible. - -Hooks run during 'post-index-change' will be run in parallel by default. +include::native-hooks.txt[] GIT ---