From patchwork Thu May 27 00:08:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 12282991 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7C878C47082 for ; Thu, 27 May 2021 00:09:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5CDDF613BE for ; Thu, 27 May 2021 00:09:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234789AbhE0AKj (ORCPT ); Wed, 26 May 2021 20:10:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51512 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234122AbhE0AKi (ORCPT ); Wed, 26 May 2021 20:10:38 -0400 Received: from mail-qt1-x849.google.com (mail-qt1-x849.google.com [IPv6:2607:f8b0:4864:20::849]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 68124C06175F for ; Wed, 26 May 2021 17:09:04 -0700 (PDT) Received: by mail-qt1-x849.google.com with SMTP id h12-20020ac8776c0000b02901f1228fdb1bso1666878qtu.6 for ; Wed, 26 May 2021 17:09:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=u5/aafDQcesqyHdnWrsLu7UyqcAKkTU3A+5Gbanp5Nc=; b=eJNArjXLjAzsq7v4+QMbiATyPky+HF3LvstK/ccQvGmEeabq9DB6bRdkRwt+EA3Yhc xwB7FPiVZyNcEVMe7X0Bebb4tKRqXwNxjW6CVtPbpSFwuhCtA2CQCTiZTb8Xrp9DO/L6 OuoZP7ASHvCmJDJe1D6J58hofY9yki+w2JYwa003gHpd6V6mQycvrXxPx+CFOrUZBJH4 UVDz0dKdLSeGUG4FwsNFLisvVkJ8RGzAqDN8YZvCSWW1YxK90JM4Bm6gbRIDwYtFdRub Xl9SAyUGINUUopmdD9RtkkxgIdHf/lVre4r55XfPx2WSZ0e+GMuZrfyzAoci6EM1ESxg dVOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=u5/aafDQcesqyHdnWrsLu7UyqcAKkTU3A+5Gbanp5Nc=; b=rjT4fmYDXgBmuy9z04MaalpIeMLVFJKvzyoM+I/p6witMnfDvGUFOp3WJr/Iu+eKsm K6Lz5g3xvhh2WVYoIObtR78OymrQN62fm0dILcmSrOxpx3+AfeNHKl4gsvdjh0StlxwB GNVMqpX2RfcJo3XrZ2G5nFMwInV8Bn93rxw5h8tzd6vTaRu3qB/R9jg36Bvst3kr1LTn sbW5PCCmV4vtBJZDdBxTwhE1k3qx8fzyiKCK1CYe/oeEQ8ix33ocu/c78PSXCtt0dYIP aXdGaz4uhCWt+f+fWkJaOuo8wXBZMzNG6JgWj5DdUqADjPxWs8ywa+7jj38RN9HTq2oj EPpA== X-Gm-Message-State: AOAM532D8k8XH4B5dU1naBFjf7D03lOjdXm1HYJWmPgexzLLlNNemr2R 1s98meBTkzvQz4Njkrx3OkaqGMeVi6wET9TzS6Nq2rfnWKcqy75LL6Y0J6GIVdaFfIhQNonOag7 z/FaIbpb3144caygZEGfT2DEWEhmfgmKPG6TT/uu/R8aCVZF/dOuPhYTzeHg9QGMZ6fIbscddEQ == X-Google-Smtp-Source: ABdhPJxXkvSKGgmSfjdDvsSR1Jm/XNGwD0p+EAnpfdj9Rl+KXAAsMIZMm0YiB5Z7Iaop/FlFY8OAAEaEo96MZezcJJc= X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:c1a4:4d87:8b5a:d12c]) (user=emilyshaffer job=sendgmr) by 2002:a0c:ab88:: with SMTP id j8mr809979qvb.23.1622074143463; Wed, 26 May 2021 17:09:03 -0700 (PDT) Date: Wed, 26 May 2021 17:08:20 -0700 In-Reply-To: <20210527000856.695702-1-emilyshaffer@google.com> Message-Id: <20210527000856.695702-2-emilyshaffer@google.com> Mime-Version: 1.0 References: <20210527000856.695702-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v9 01/37] doc: propose hooks managed by the config From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Begin a design document for config-based hooks, managed via git-hook. Focus on an overview of the implementation and motivation for design decisions. Briefly discuss the alternatives considered before this point. Also, attempt to redefine terms to fit into a multihook world. Signed-off-by: Emily Shaffer --- Notes: Since v6, checked for inconsistencies with implementation and added lots of caveats about whether 'git hook add' and 'git hook edit' will ever materialize. Hopefully this reflects reality now; please review accordingly. Since v6, checked for inconsistencies with implementation and added lots of caveats about whether 'git hook add' and 'git hook edit' will ever materialize. Hopefully this reflects reality now; please review accordingly. Since v4, addressed comments from Jonathan Tan about wording. However, I have not addressed AEvar's comments or done a full re-review of this document. I wanted to get the rest of the series out for initial review first. - Emily Since v4, addressed comments from Jonathan Tan about wording. Documentation/Makefile | 1 + .../technical/config-based-hooks.txt | 369 ++++++++++++++++++ 2 files changed, 370 insertions(+) create mode 100644 Documentation/technical/config-based-hooks.txt diff --git a/Documentation/Makefile b/Documentation/Makefile index 2aae4c9cbb..5d19eddb0e 100644 --- a/Documentation/Makefile +++ b/Documentation/Makefile @@ -90,6 +90,7 @@ SP_ARTICLES += $(API_DOCS) TECH_DOCS += MyFirstContribution TECH_DOCS += MyFirstObjectWalk TECH_DOCS += SubmittingPatches +TECH_DOCS += technical/config-based-hooks TECH_DOCS += technical/hash-function-transition TECH_DOCS += technical/http-protocol TECH_DOCS += technical/index-format diff --git a/Documentation/technical/config-based-hooks.txt b/Documentation/technical/config-based-hooks.txt new file mode 100644 index 0000000000..1f973117e4 --- /dev/null +++ b/Documentation/technical/config-based-hooks.txt @@ -0,0 +1,369 @@ +Configuration-based hook management +=================================== +:sectanchors: + +[[motivation]] +== Motivation + +Replace the `.git/hook/hookname` path as the only source of hooks to execute; +allow users to define hooks using config files, in a way which is friendly to +users with multiple repos which have similar needs - hooks can be easily shared +between multiple Git repos. + +Redefine "hook" as an event rather than a single script, allowing users to +perform multiple unrelated actions on a single event. + +Make it easier for users to discover Git's hook feature and automate their +workflows. + +[[user-interfaces]] +== User interfaces + +[[config-schema]] +=== Config schema + +Hooks can be introduced by editing the configuration manually. There are two new +sections added, `hook` and `hookcmd`. + +[[config-schema-hook]] +==== `hook` + +Primarily contains subsections for each hook event. The order of variables in +these subsections defines the hook command execution order; hook commands can be +specified by setting the value directly to the command if no additional +configuration is needed, or by setting the value as the name of a `hookcmd`. If +Git does not find a `hookcmd` whose subsection matches the value of the given +command string, Git will try to execute the string directly. Hooks are executed +by passing the resolved command string to the shell. In the future, hook event +subsections could also contain per-hook-event settings; see +<> for more details. + +Also contains top-level hook execution settings, for example, `hook.runHookDir`. +(These settings are described more in <>.) + +---- +[hook "pre-commit"] + command = perl-linter + command = /usr/bin/git-secrets --pre-commit + +[hook "pre-applypatch"] + command = perl-linter + # for illustration purposes; error behavior isn't planned yet + error = ignore + +[hook] + runHookDir = interactive +---- + +[[config-schema-hookcmd]] +==== `hookcmd` + +Defines a hook command and its attributes, which will be used when a hook event +occurs. Unqualified attributes are assumed to apply to this hook during all hook +events, but event-specific attributes can also be supplied. The example runs +`/usr/bin/lint-it --language=perl `, but for repos which +include this config, the hook command will be skipped for all events. +Theoretically, the last line could be used to "un-skip" the hook command for +`pre-commit` hooks, but this hasn't been scoped or implemented yet. + +---- +[hookcmd "perl-linter"] + command = /usr/bin/lint-it --language=perl + skip = true + # for illustration purposes; below hasn't been defined yet + pre-commit-skip = false +---- + +[[command-line-api]] +=== Command-line API + +Users should be able to view, run, reorder, and create hook commands via the +command line. External tools should be able to view a list of hooks in the +correct order to run. Modifier commands (`edit` and `add`) have not been +implemented yet and may not be if manually editing the config proves usable +enough. + +*`git hook list `* + +*`git hook run [-a ]... [-e ]...`* + +*`git hook edit `* + +*`git hook add `* + +[[hook-editor]] +=== Hook editor + +The tool which is presented by `git hook edit `. Ideally, this +tool should be easier to use than manually editing the config, and then produce +a concise config afterwards. It may take a form similar to `git rebase +--interactive`. This has not been designed or implemented yet and may not be if +the config proves usable enough. + +[[implementation]] +== Implementation + +[[library]] +=== Library + +`hook.c` and `hook.h` are responsible for interacting with the config files. The +hook library provides a basic API to call all hooks in config order with more +complex options passed via `struct run_hooks_opt`: + +*`int run_hooks(const char *hookname, struct run_hooks_opt *options)`* + +`struct run_hooks_opt` allows callers to set: + +- environment variables +- command-line arguments +- behavior for the hook command provided by `run-command.h:find_hook()` (see + below) +- a method to provide stdin to each hook, either via a file containing stdin, a + `struct string_list` containing a list of lines to print, or a callback + function to allow the caller to populate stdin manually +- a method to process stdout from each hook, e.g. for printing to sideband + during a network operation +- parallelism +- a custom working directory for hooks to execute in + +And this struct can be extended with more options as necessary in the future. + +The "legacy" hook provided by `run-command.h:find_hook()` - that is, the hook +present in `.git/hooks/` or +`$(git config --get core.hooksPath)/` - can be handled in a number of +ways, providing an avenue to deprecate these "legacy" hooks if desired. The +handling is based on a config `hook.runHookDir`, which is checked against a +number of cases: + +- "no": the legacy hook will not be run +- "error": Git will print a warning to stderr before ignoring the legacy hook +- "interactive": Git will prompt the user before running the legacy hook +- "warn": Git will print a warning to stderr before running the legacy hook +- "yes" (default): Git will silently run the legacy hook + +In case this list is expanded in the future, if a value for `hook.runHookDir` is +given which Git does not recognize, Git should discard that config entry. For +example, if "warn" was specified at system level and "junk" was specified at +global level, Git would resolve the value to "warn"; if the only time the config +was set was to "junk", Git would use the default value of "yes" (but print a +warning to the user first to let them know their value is wrong). + +`struct hookcmd` is expected to grow in size over time as more functionality is +added to hooks; so that other parts of the code don't need to understand the +config schema, `struct hookcmd` should contain logical values instead of string +pairs. + +By default, hook parallelism is chosen based on the semantics of each hook; +callsites initialize their `struct run_hooks_opt` via one of two macros, +`RUN_HOOKS_OPT_INIT_SYNC` or `RUN_HOOKS_OPT_INIT_ASYNC`. The default number of +jobs can be configured in `hook.jobs`; this config applies across all hook +events. If unset, the value of `online_cpus()` (equivalent to `nproc`) is used. + +[[builtin]] +=== Builtin + +`builtin/hook.c` is responsible for providing the frontend. It's responsible for +formatting user-provided data and then calling the library API to set the +configs as appropriate. The builtin frontend is not responsible for calling the +config directly, so that other areas of Git can rely on the hook library to +understand the most recent config schema for hooks. + +[[migration]] +=== Migration path + +[[stage-0]] +==== Stage 0 + +Hooks are called by running `run-command.h:find_hook()` with the hookname and +executing the result. The hook library and builtin do not exist. Hooks only +exist as specially named scripts within `.git/hooks/`. + +[[stage-1]] +==== Stage 1 + +`git hook list --porcelain ` is implemented. `hook.h:run_hooks()` is +taught to include `run-command.h:find_hook()` at the end; calls to `find_hook()` +are replaced with calls to `run_hooks()`. Users can opt-in to config-based hooks +simply by creating some in their config; otherwise users should remain +unaffected by the change. + +[[stage-2]] +==== Stage 2 + +The call to `find_hook()` inside of `run_hooks()` learns to check for a config, +`hook.runHookDir`. Users can opt into managing their hooks completely via the +config this way. + +[[stage-3]] +==== Stage 3 + +`.git/hooks` is removed from the template and the hook directory is considered +deprecated. To avoid breaking older repos, the default of `hook.runHookDir` is +not changed, and `find_hook()` is not removed. + +[[caveats]] +== Caveats + +[[security]] +=== Security and repo config + +Part of the motivation behind this refactor is to mitigate hooks as an attack +vector.footnote:[https://lore.kernel.org/git/20171002234517.GV19555@aiede.mtv.corp.google.com/] +However, as the design stands, users can still provide hooks in the repo-level +config, which is included when a repo is zipped and sent elsewhere. The +security of the repo-level config is still under discussion; this design +generally assumes the repo-level config is secure, which is not true yet. This +assumption was made to avoid overcomplicating the design. So, this series +doesn't particularly improve security or resistance to zip attacks. + +[[ease-of-use]] +=== Ease of use + +The config schema is nontrivial; that's why it's important for the `git hook` +modifier commands to be usable. Contributors with UX expertise are encouraged to +share their suggestions. + +[[alternatives]] +== Alternative approaches + +A previous summary of alternatives exists in the +archives.footnote:[https://lore.kernel.org/git/20191116011125.GG22855@google.com] + +The table below shows a number of goals and how they might be achieved with +config-based hooks, by implementing directory support (i.e. +'.git/hooks/pre-commit.d'), or as hooks are run today. + +.Comparison of alternatives +|=== +|Feature |Config-based hooks |Hook directories |Status quo + +|Supports multiple hooks +|Natively +|Natively +|With user effort + +|Supports parallelization +|Natively +|Natively +|No (user's multihook trampoline script would need to handle parallelism) + +|Safer for zipped repos +|A little +|No +|No + +|Previous hooks just work +|If configured +|Yes +|Yes + +|Can install one hook to many repos +|Yes +|With symlinks or core.hooksPath +|With symlinks or core.hooksPath + +|Discoverability +|Findable with 'git help git' or tab-completion via 'git hook' subcommand +|Findable via improved documentation +|Same as before + +|Hard to run unexpected hook +|If configured +|Could be made to warn or look for a config +|No +|=== + +[[status-quo]] +=== Status quo + +Today users can implement multihooks themselves by using a "trampoline script" +as their hook, and pointing that script to a directory or list of other scripts +they wish to run. + +[[hook-directories]] +=== Hook directories + +Other contributors have suggested Git learn about the existence of a directory +such as `.git/hooks/.d` and execute those hooks in alphabetical order. + +[[future-work]] +== Future work + +[[execution-ordering]] +=== Execution ordering + +We may find that config order is insufficient for some users; for example, +config order makes it difficult to add a new hook to the system or global config +which runs at the end of the hook list. A new ordering schema should be: + +1) Specified by a `hook.order` config, so that users will not unexpectedly see +their order change; + +2) Either dependency or numerically based. + +Dependency-based ordering is prone to classic linked-list problems, like a +cycles and handling of missing dependencies. But, it paves the way for enabling +parallelization if some tasks truly depend on others. + +Numerical ordering makes it tricky for Git to generate suggested ordering +numbers for each command, but is easy to determine a definitive order. + +[[parallelization]] +=== Parallelization with dependencies + +Currently hooks use a naive parallelization scheme or are run in series. But if +one hook depends on another's output, then users will want to specify those +dependencies. If we decide to solve this problem, we may want to look to modern +build systems for inspiration on how to manage dependencies and parallel tasks. + +[[nontrivial-hooks]] +=== Multihooks and nontrivial output + +Some hooks - like 'proc-receive' - don't lend themselves well to multihooks at +all. In the case of 'proc-receive', for now, multiple hook definitions are +disallowed. In the future we might be able to conceive a better approach, for +example, running the hooks in series and using the output from one hook as the +input to the next. + +[[securing-hookdir-hooks]] +=== Securing hookdir hooks + +With the design as written in this doc, it's still possible for a malicious user +to modify `.git/config` to include `hook.pre-receive.command = rm -rf /`, then +zip their repo and send it to another user. It may be necessary to teach Git to +only allow inlined hooks like this if they were configured outside of the local +scope (in other words, only run hookcmds, and only allow hookcmds to be +configured in global or system scope); or another approach, like a list of safe +projects, might be useful. It may also be sufficient (or at least useful) to +teach a `hook.disableAll` config or similar flag to the Git executable. + +[[submodule-inheritance]] +=== Submodule inheritance + +It's possible some submodules may want to run the identical set of hooks that +their superrepo runs. While a globally-configured hook set is helpful, it's not +a great solution for users who have multiple repos-with-submodules under the +same user. It would be useful for submodules to learn how to run hooks from +their superrepo's config, or inherit that hook setting. + +[[per-hook-event-settings]] +=== Per-hook-event settings + +It might be desirable to keep settings specifically for some hook events, but +not for others - for example, a user may wish to disable hookdir hooks for all +events but pre-commit, which they haven't had time to convert yet; or, a user +may wish for execution order settings to differ based on hook event. In that +case, it would be useful to set something like `hook.pre-commit.executionOrder` +which would not apply to the 'prepare-commit-msg' hook, for example. + +[[glossary]] +== Glossary + +*hook event* + +A point during Git's execution where user scripts may be run, for example, +_prepare-commit-msg_ or _pre-push_. + +*hook command* + +A user script or executable which will be run on one or more hook events. From patchwork Thu May 27 00:08:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 12282993 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 921A6C47082 for ; Thu, 27 May 2021 00:09:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 622CE613CA for ; Thu, 27 May 2021 00:09:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235005AbhE0AKu (ORCPT ); Wed, 26 May 2021 20:10:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51520 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234727AbhE0AKj (ORCPT ); Wed, 26 May 2021 20:10:39 -0400 Received: from mail-qk1-x74a.google.com (mail-qk1-x74a.google.com [IPv6:2607:f8b0:4864:20::74a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5A04FC061574 for ; Wed, 26 May 2021 17:09:06 -0700 (PDT) Received: by mail-qk1-x74a.google.com with SMTP id z2-20020a3765020000b02903a5f51b1c74so2094516qkb.7 for ; Wed, 26 May 2021 17:09:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=XRTBZT1WjPz6Hih0Jb6m3QfLt0YoXU8k1EGrJ0ZUBSw=; b=LcLWjnD7ylPU+fxto5OQenZEZbH+VLDiIwm6MKYJTTT+Mbeq27AXayc7lyZSrsLwhB 5C3PAoNhLXJLGxXKCy9zN0t6mBdEWLPUS2YyliIrVWG1+JR7TciXoxZZnVBaBS8YrmYu SmL4/3TVHSK5vjIKlhva5/OrWHM0LFIu+y+T4Tqo0uqtj26SPIaDSKoGGNgI5BryzVdE fyjV1VRlI8LhG66YOS5xjgpq4qUZ0b7rrlVJiD+AwvRmwJ1iOCGpUuhp9+yeq6psAHhx nGmp7/UkEajK5mWzmx0Rno+gInDRn9HgvzMgLC6zbSujJOgoOz291MV31iMnicO+gNJn NOfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=XRTBZT1WjPz6Hih0Jb6m3QfLt0YoXU8k1EGrJ0ZUBSw=; b=jWyUCe06jwHw2gR2auBS5ROrNhY2/XDc9g6PJNcCE7Rs21a0CAkvUDGESbzV6+P5ZT L/cfe7aLnZEuBTY5dd60au8eTI604eJYkMKi/IAYtWzwtzngjETRnzhI1i/5VQxz8LXR ph1kQCCxtsF+UmofkAoSLoY2Ho/+LXTjl2/gGvudnVYYmVoGXihVZAVZoa11TM1BuDp/ TIg7PCdGjIwXGQ3Dokg+SnfgW+gbXRUjYiQhkvIsxywFUDf7bu11oa1EyANK+/53ItzE BLCWoYGe8FLon9L7/9qFrfp0VuIdp3LR0Z1KaCgMbhBV0dBW81vsy4HAU/vHugk3D6bp sudA== X-Gm-Message-State: AOAM532tgb2f7IcfJ34EN3etbub4NJZJzn+6YLZz6qYQArlIm62fiQ6j QWuXh72LOEkX0ZRQgd2UraLE4YPeXPJHGNytvvtbO/lNilKdll9LhKFUR15biOh1dSPRbVqUXhy 3gfMBKIMXlZlNmxnnfXUq7b2ejb1+zQb0vLTmO5hwY6OBAdx6iNdS9MVYPbaHLnMCljdCKMNijQ == X-Google-Smtp-Source: ABdhPJz7xYtC93lj5pWVL8m1qYi2W6h6PDl3ZP2ukO4V4TeCCnipNwGSKB2Gk8hAijCtaC9uuaZhhsU+xW3uc6B7aaY= X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:c1a4:4d87:8b5a:d12c]) (user=emilyshaffer job=sendgmr) by 2002:a05:6214:d87:: with SMTP id e7mr515781qve.53.1622074145397; Wed, 26 May 2021 17:09:05 -0700 (PDT) Date: Wed, 26 May 2021 17:08:21 -0700 In-Reply-To: <20210527000856.695702-1-emilyshaffer@google.com> Message-Id: <20210527000856.695702-3-emilyshaffer@google.com> Mime-Version: 1.0 References: <20210527000856.695702-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v9 02/37] hook: introduce git-hook subcommand From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Add a new subcommand, git-hook, which will be used to ease config-based hook management. This command will handle parsing configs to compose a list of hooks to run for a given event, as well as adding or modifying hook configs in an interactive fashion. Start with 'git hook list ', which checks the known configs in order to create an ordered list of hooks to run on a given hook event. Multiple commands can be specified for a given hook by providing multiple "hook..command = " lines. Hooks will be run in config order. If more properties need to be set on a given hook in the future, commands can also be specified by providing "hook..command = ", as well as a "[hookcmd ]" subsection; this subsection should contain a "hookcmd..command = " line. For example: $ git config --list | grep ^hook hook.pre-commit.command=baz hook.pre-commit.command=~/bar.sh hookcmd.baz.command=~/baz/from/hookcmd.sh $ git hook list pre-commit global: ~/baz/from/hookcmd.sh local: ~/bar.sh Signed-off-by: Emily Shaffer --- Notes: Since v4, mainly changed to RUN_SETUP_GENTLY so that 'git hook list' can be executed outside of a repo. .gitignore | 1 + Documentation/config/hook.txt | 9 +++ Documentation/git-hook.txt | 73 +++++++++++++++++++++ Makefile | 2 + builtin.h | 1 + builtin/hook.c | 65 ++++++++++++++++++ command-list.txt | 1 + git.c | 1 + hook.c | 120 ++++++++++++++++++++++++++++++++++ hook.h | 25 +++++++ t/t1360-config-based-hooks.sh | 88 +++++++++++++++++++++++++ 11 files changed, 386 insertions(+) create mode 100644 Documentation/config/hook.txt create mode 100644 Documentation/git-hook.txt create mode 100644 builtin/hook.c create mode 100644 hook.c create mode 100644 hook.h create mode 100755 t/t1360-config-based-hooks.sh diff --git a/.gitignore b/.gitignore index 311841f9be..de39dc9961 100644 --- a/.gitignore +++ b/.gitignore @@ -77,6 +77,7 @@ /git-grep /git-hash-object /git-help +/git-hook /git-http-backend /git-http-fetch /git-http-push diff --git a/Documentation/config/hook.txt b/Documentation/config/hook.txt new file mode 100644 index 0000000000..71449ecbc7 --- /dev/null +++ b/Documentation/config/hook.txt @@ -0,0 +1,9 @@ +hook..command:: + A command to execute during the hook event. This can be an + executable on your device, a oneliner for your shell, or the name of a + hookcmd. See linkgit:git-hook[1]. + +hookcmd..command:: + A command to execute during a hook for which has been specified + as a command. This can be an executable on your device or a oneliner for + your shell. See linkgit:git-hook[1]. diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt new file mode 100644 index 0000000000..f19875ed68 --- /dev/null +++ b/Documentation/git-hook.txt @@ -0,0 +1,73 @@ +git-hook(1) +=========== + +NAME +---- +git-hook - Manage configured hooks + +SYNOPSIS +-------- +[verse] +'git hook' list + +DESCRIPTION +----------- +You can list configured hooks with this command. Later, you will be able to run, +add, and modify hooks with this command. + +This command parses the default configuration files for sections `hook` and +`hookcmd`. `hook` is used to describe the commands which will be run during a +particular hook event; commands are run in the order Git encounters them during +the configuration parse (see linkgit:git-config[1]). `hookcmd` is used to +describe attributes of a specific command. If additional attributes don't need +to be specified, a command to run can be specified directly in the `hook` +section; if a `hookcmd` by that name isn't found, Git will attempt to run the +provided value directly. For example: + +Global config +---- + [hook "post-commit"] + command = "linter" + command = "~/typocheck.sh" + + [hookcmd "linter"] + command = "/bin/linter --c" +---- + +Local config +---- + [hook "prepare-commit-msg"] + command = "linter" + [hook "post-commit"] + command = "python ~/run-test-suite.py" +---- + +With these configs, you'd then see: + +---- +$ git hook list "post-commit" +global: /bin/linter --c +global: ~/typocheck.sh +local: python ~/run-test-suite.py + +$ git hook list "prepare-commit-msg" +local: /bin/linter --c +---- + +COMMANDS +-------- + +list ``:: + +List the hooks which have been configured for ``. Hooks appear +in the order they should be run, and print the config scope where the relevant +`hook..command` was specified, not the `hookcmd` (if applicable). +This output is human-readable and the format is subject to change over time. + +CONFIGURATION +------------- +include::config/hook.txt[] + +GIT +--- +Part of the linkgit:git[1] suite diff --git a/Makefile b/Makefile index c3565fc0f8..a6b71a0fbe 100644 --- a/Makefile +++ b/Makefile @@ -901,6 +901,7 @@ LIB_OBJS += hash-lookup.o LIB_OBJS += hashmap.o LIB_OBJS += help.o LIB_OBJS += hex.o +LIB_OBJS += hook.o LIB_OBJS += ident.o LIB_OBJS += json-writer.o LIB_OBJS += kwset.o @@ -1101,6 +1102,7 @@ BUILTIN_OBJS += builtin/get-tar-commit-id.o BUILTIN_OBJS += builtin/grep.o BUILTIN_OBJS += builtin/hash-object.o BUILTIN_OBJS += builtin/help.o +BUILTIN_OBJS += builtin/hook.o BUILTIN_OBJS += builtin/index-pack.o BUILTIN_OBJS += builtin/init-db.o BUILTIN_OBJS += builtin/interpret-trailers.o diff --git a/builtin.h b/builtin.h index 16ecd5586f..91740c1514 100644 --- a/builtin.h +++ b/builtin.h @@ -164,6 +164,7 @@ int cmd_get_tar_commit_id(int argc, const char **argv, const char *prefix); int cmd_grep(int argc, const char **argv, const char *prefix); int cmd_hash_object(int argc, const char **argv, const char *prefix); int cmd_help(int argc, const char **argv, const char *prefix); +int cmd_hook(int argc, const char **argv, const char *prefix); int cmd_index_pack(int argc, const char **argv, const char *prefix); int cmd_init_db(int argc, const char **argv, const char *prefix); int cmd_interpret_trailers(int argc, const char **argv, const char *prefix); diff --git a/builtin/hook.c b/builtin/hook.c new file mode 100644 index 0000000000..79e150437e --- /dev/null +++ b/builtin/hook.c @@ -0,0 +1,65 @@ +#include "cache.h" +#include "builtin.h" +#include "config.h" +#include "hook.h" +#include "parse-options.h" +#include "strbuf.h" + +static const char * const builtin_hook_usage[] = { + N_("git hook list "), + NULL +}; + +static int list(int argc, const char **argv, const char *prefix) +{ + struct list_head *head, *pos; + const char *hookname = NULL; + + struct option list_options[] = { + OPT_END(), + }; + + argc = parse_options(argc, argv, prefix, list_options, + builtin_hook_usage, 0); + + if (argc < 1) { + usage_msg_opt(_("You must specify a hook event name to list."), + builtin_hook_usage, list_options); + } + + hookname = argv[0]; + + head = hook_list(hookname); + + if (list_empty(head)) { + printf(_("no commands configured for hook '%s'\n"), + hookname); + return 0; + } + + list_for_each(pos, head) { + struct hook *item = list_entry(pos, struct hook, list); + if (item) + printf("%s: %s\n", + config_scope_name(item->origin), + item->command.buf); + } + + clear_hook_list(head); + + return 0; +} + +int cmd_hook(int argc, const char **argv, const char *prefix) +{ + struct option builtin_hook_options[] = { + OPT_END(), + }; + if (argc < 2) + usage_with_options(builtin_hook_usage, builtin_hook_options); + + if (!strcmp(argv[1], "list")) + return list(argc - 1, argv + 1, prefix); + + usage_with_options(builtin_hook_usage, builtin_hook_options); +} diff --git a/command-list.txt b/command-list.txt index a289f09ed6..9ccd8e5aeb 100644 --- a/command-list.txt +++ b/command-list.txt @@ -103,6 +103,7 @@ git-grep mainporcelain info git-gui mainporcelain git-hash-object plumbingmanipulators git-help ancillaryinterrogators complete +git-hook mainporcelain git-http-backend synchingrepositories git-http-fetch synchelpers git-http-push synchelpers diff --git a/git.c b/git.c index 18bed9a996..39988ee3b0 100644 --- a/git.c +++ b/git.c @@ -538,6 +538,7 @@ static struct cmd_struct commands[] = { { "grep", cmd_grep, RUN_SETUP_GENTLY }, { "hash-object", cmd_hash_object }, { "help", cmd_help }, + { "hook", cmd_hook, RUN_SETUP_GENTLY }, { "index-pack", cmd_index_pack, RUN_SETUP_GENTLY | NO_PARSEOPT }, { "init", cmd_init_db }, { "init-db", cmd_init_db }, diff --git a/hook.c b/hook.c new file mode 100644 index 0000000000..d3e28aa73a --- /dev/null +++ b/hook.c @@ -0,0 +1,120 @@ +#include "cache.h" + +#include "hook.h" +#include "config.h" + +void free_hook(struct hook *ptr) +{ + if (ptr) { + strbuf_release(&ptr->command); + free(ptr); + } +} + +static void append_or_move_hook(struct list_head *head, const char *command) +{ + struct list_head *pos = NULL, *tmp = NULL; + struct hook *to_add = NULL; + + /* + * remove the prior entry with this command; we'll replace it at the + * end. + */ + list_for_each_safe(pos, tmp, head) { + struct hook *it = list_entry(pos, struct hook, list); + if (!strcmp(it->command.buf, command)) { + list_del(pos); + /* we'll simply move the hook to the end */ + to_add = it; + break; + } + } + + if (!to_add) { + /* adding a new hook, not moving an old one */ + to_add = xmalloc(sizeof(*to_add)); + strbuf_init(&to_add->command, 0); + strbuf_addstr(&to_add->command, command); + } + + /* re-set the scope so we show where an override was specified */ + to_add->origin = current_config_scope(); + + list_add_tail(&to_add->list, head); +} + +static void remove_hook(struct list_head *to_remove) +{ + struct hook *hook_to_remove = list_entry(to_remove, struct hook, list); + list_del(to_remove); + free_hook(hook_to_remove); +} + +void clear_hook_list(struct list_head *head) +{ + struct list_head *pos, *tmp; + list_for_each_safe(pos, tmp, head) + remove_hook(pos); +} + +struct hook_config_cb +{ + struct strbuf *hookname; + struct list_head *list; +}; + +static int hook_config_lookup(const char *key, const char *value, void *cb_data) +{ + struct hook_config_cb *data = cb_data; + const char *hook_key = data->hookname->buf; + struct list_head *head = data->list; + + if (!strcmp(key, hook_key)) { + const char *command = value; + struct strbuf hookcmd_name = STRBUF_INIT; + + /* + * Check if a hookcmd with that name exists. If it doesn't, + * 'git_config_get_value()' is documented not to touch &command, + * so we don't need to do anything. + */ + strbuf_addf(&hookcmd_name, "hookcmd.%s.command", command); + git_config_get_value(hookcmd_name.buf, &command); + + if (!command) { + strbuf_release(&hookcmd_name); + BUG("git_config_get_value overwrote a string it shouldn't have"); + } + + /* + * TODO: implement an option-getting callback, e.g. + * get configs by pattern hookcmd.$value.* + * for each key+value, do_callback(key, value, cb_data) + */ + + append_or_move_hook(head, command); + + strbuf_release(&hookcmd_name); + } + + return 0; +} + +struct list_head* hook_list(const char* hookname) +{ + struct strbuf hook_key = STRBUF_INIT; + struct list_head *hook_head = xmalloc(sizeof(struct list_head)); + struct hook_config_cb cb_data = { &hook_key, hook_head }; + + INIT_LIST_HEAD(hook_head); + + if (!hookname) + return NULL; + + strbuf_addf(&hook_key, "hook.%s.command", hookname); + + git_config(hook_config_lookup, &cb_data); + + strbuf_release(&hook_key); + return hook_head; +} diff --git a/hook.h b/hook.h new file mode 100644 index 0000000000..042cab8446 --- /dev/null +++ b/hook.h @@ -0,0 +1,25 @@ +#include "config.h" +#include "list.h" +#include "strbuf.h" + +struct hook { + struct list_head list; + /* + * Config file which holds the hook.*.command definition. + * (This has nothing to do with the hookcmd..* configs.) + */ + enum config_scope origin; + /* The literal command to run. */ + struct strbuf command; +}; + +/* + * Provides a linked list of 'struct hook' detailing commands which should run + * in response to the 'hookname' event, in execution order. + */ +struct list_head* hook_list(const char *hookname); + +/* Free memory associated with a 'struct hook' */ +void free_hook(struct hook *ptr); +/* Empties the list at 'head', calling 'free_hook()' on each entry */ +void clear_hook_list(struct list_head *head); diff --git a/t/t1360-config-based-hooks.sh b/t/t1360-config-based-hooks.sh new file mode 100755 index 0000000000..6e4a3e763f --- /dev/null +++ b/t/t1360-config-based-hooks.sh @@ -0,0 +1,88 @@ +#!/bin/bash + +test_description='config-managed multihooks, including git-hook command' + +. ./test-lib.sh + +ROOT= +if test_have_prereq MINGW +then + # In Git for Windows, Unix-like paths work only in shell scripts; + # `git.exe`, however, will prefix them with the pseudo root directory + # (of the Unix shell). Let's accommodate for that. + ROOT="$(cd / && pwd)" +fi + +setup_hooks () { + test_config hook.pre-commit.command "/path/ghi" --add + test_config_global hook.pre-commit.command "/path/def" --add +} + +setup_hookcmd () { + test_config hook.pre-commit.command "abc" --add + test_config_global hookcmd.abc.command "/path/abc" --add +} + +test_expect_success 'git hook rejects commands without a mode' ' + test_must_fail git hook pre-commit +' + + +test_expect_success 'git hook rejects commands without a hookname' ' + test_must_fail git hook list +' + +test_expect_success 'git hook runs outside of a repo' ' + setup_hooks && + + cat >expected <<-EOF && + global: $ROOT/path/def + EOF + + nongit git config --list --global && + + nongit git hook list pre-commit >actual && + test_cmp expected actual +' + +test_expect_success 'git hook list orders by config order' ' + setup_hooks && + + cat >expected <<-EOF && + global: $ROOT/path/def + local: $ROOT/path/ghi + EOF + + git hook list pre-commit >actual && + test_cmp expected actual +' + +test_expect_success 'git hook list dereferences a hookcmd' ' + setup_hooks && + setup_hookcmd && + + cat >expected <<-EOF && + global: $ROOT/path/def + local: $ROOT/path/ghi + local: $ROOT/path/abc + EOF + + git hook list pre-commit >actual && + test_cmp expected actual +' + +test_expect_success 'git hook list reorders on duplicate commands' ' + setup_hooks && + + test_config hook.pre-commit.command "/path/def" --add && + + cat >expected <<-EOF && + local: $ROOT/path/ghi + local: $ROOT/path/def + EOF + + git hook list pre-commit >actual && + test_cmp expected actual +' + +test_done From patchwork Thu May 27 00:08:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 12282995 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C3D69C47088 for ; Thu, 27 May 2021 00:09:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9BFFC6128D for ; Thu, 27 May 2021 00:09:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234314AbhE0AKv (ORCPT ); Wed, 26 May 2021 20:10:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51526 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234122AbhE0AKl (ORCPT ); Wed, 26 May 2021 20:10:41 -0400 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 11A2BC06175F for ; Wed, 26 May 2021 17:09:08 -0700 (PDT) Received: by mail-yb1-xb4a.google.com with SMTP id d20-20020a25add40000b02904f8960b23e8so3676713ybe.6 for ; Wed, 26 May 2021 17:09:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=RQEPhYSW+pnU2Ya2fKl3UKob65qngob4CXBszDyv+Tg=; b=KWhocE/ezycq+UZT0UJeNdGGf+/+8OXs2/9gniyxgiDkyGXVgyWY6RJAZuGY3uIU4n pmvY/JoUli1n2gCczU4Ip120Om6WgP6Xe318t7yliOlY1vFBx8eDi/3kjq58MkXiygVD 0f+iTl639pSMUZ1M54nQu+LH/geY37oAdw5j9SMKix1ghQaUnHY0BdKcnk74AePTf2oq UhTNM4hJbnJ87fN6jDQfiLpomGSPUANS08VnwlkFCi1sTbgI9ap3DxNUAQPvtkGk9uhH 0HqTzJ3k7GRN6nvtJHoVqFNhYfJbMLZGVVCI57Xtw4fU6rn0kBZyHKoGxqaqQ7Nk+iIb T5fg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=RQEPhYSW+pnU2Ya2fKl3UKob65qngob4CXBszDyv+Tg=; b=RgTudwwQA/CzCfdJ6KSxZeqvXpn077EelEGHvx9YSoZWj1t88RWJpum1zdBBEeQk13 FTc2Jqn4vOsjSfzrT+6HIFzaWE8wZSqFGvH0uly4q8W+lieoYjeOoMhx0nXHAbhvtJbE DhWQIn5qxplQtcGE3jsEhHQRAeoGSwXu5m+xn2WCwBCKCce91WoZBRSVlTYjr9ygo8Lr PRNRHWzRujAaUGAWcjsVThumocj9ccQY9QNgoRPyBhQcG082vz2DtyJFIhxnO5f1rW/i LVVefLN1cnUw1dNl2VBnc+SGENcuTnS84j9S1bNxXCsUw5oqwZU+unCxTaC2Sye+en9h u9FA== X-Gm-Message-State: AOAM5303updZJZnNFzOcDfWkaQpHWKL7jrSdj5upiMaMCDSON/wF190x P59PjsIhfftqbrPBSAUlvAOOYAPUG4z2H8Y52HXjqTPgq+3VlgkjxT5tSlJ7hTyYvmgJNPEWVWz SrdjMwDIrFpZfQTuNV4goCIprzX2p9ddPnI2ga33vGhVSzlig97H0O1IQLu5CeToqXnUAY/xmbg == X-Google-Smtp-Source: ABdhPJzwgQ47zNeGw1zznHfhSgtnDQrf2LNA+nAdjCeLaNmI85qB6LhM4rkYfAlTmiXcjEOtO7E3MpZfhNQv9iX65jI= X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:c1a4:4d87:8b5a:d12c]) (user=emilyshaffer job=sendgmr) by 2002:a05:6902:1005:: with SMTP id w5mr825103ybt.233.1622074147180; Wed, 26 May 2021 17:09:07 -0700 (PDT) Date: Wed, 26 May 2021 17:08:22 -0700 In-Reply-To: <20210527000856.695702-1-emilyshaffer@google.com> Message-Id: <20210527000856.695702-4-emilyshaffer@google.com> Mime-Version: 1.0 References: <20210527000856.695702-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v9 03/37] hook: include hookdir hook in list From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Historically, hooks are declared by placing an executable into $GIT_DIR/hooks/$HOOKNAME (or $HOOKDIR/$HOOKNAME). Although hooks taken from the config are more featureful than hooks placed in the $HOOKDIR, those hooks should not stop working for users who already have them. Let's list them to the user, but instead of displaying a config scope (e.g. "global: blah") we can prefix them with "hookdir:". Signed-off-by: Emily Shaffer --- builtin/hook.c | 18 +++++++++++++++--- hook.c | 17 +++++++++++++++++ hook.h | 1 + t/t1360-config-based-hooks.sh | 19 +++++++++++++++++++ 4 files changed, 52 insertions(+), 3 deletions(-) diff --git a/builtin/hook.c b/builtin/hook.c index 79e150437e..e82725f0a6 100644 --- a/builtin/hook.c +++ b/builtin/hook.c @@ -39,10 +39,20 @@ static int list(int argc, const char **argv, const char *prefix) list_for_each(pos, head) { struct hook *item = list_entry(pos, struct hook, list); - if (item) - printf("%s: %s\n", - config_scope_name(item->origin), + item = list_entry(pos, struct hook, list); + if (item) { + /* + * TRANSLATORS: ": ". Both fields + * should be left untranslated; config scope matches the + * output of 'git config --show-scope'. Marked for + * translation to provide better RTL support later. + */ + printf(_("%s: %s\n"), + (item->from_hookdir + ? "hookdir" + : config_scope_name(item->origin)), item->command.buf); + } } clear_hook_list(head); @@ -58,6 +68,8 @@ int cmd_hook(int argc, const char **argv, const char *prefix) if (argc < 2) usage_with_options(builtin_hook_usage, builtin_hook_options); + git_config(git_default_config, NULL); + if (!strcmp(argv[1], "list")) return list(argc - 1, argv + 1, prefix); diff --git a/hook.c b/hook.c index d3e28aa73a..b4994fc108 100644 --- a/hook.c +++ b/hook.c @@ -2,6 +2,7 @@ #include "hook.h" #include "config.h" +#include "run-command.h" void free_hook(struct hook *ptr) { @@ -35,6 +36,7 @@ static void append_or_move_hook(struct list_head *head, const char *command) to_add = xmalloc(sizeof(*to_add)); strbuf_init(&to_add->command, 0); strbuf_addstr(&to_add->command, command); + to_add->from_hookdir = 0; } /* re-set the scope so we show where an override was specified */ @@ -115,6 +117,21 @@ struct list_head* hook_list(const char* hookname) git_config(hook_config_lookup, &cb_data); + if (have_git_dir()) { + const char *legacy_hook_path = find_hook(hookname); + + /* Unconditionally add legacy hook, but annotate it. */ + if (legacy_hook_path) { + struct hook *legacy_hook; + + append_or_move_hook(hook_head, + absolute_path(legacy_hook_path)); + legacy_hook = list_entry(hook_head->prev, struct hook, + list); + legacy_hook->from_hookdir = 1; + } + } + strbuf_release(&hook_key); return hook_head; } diff --git a/hook.h b/hook.h index 042cab8446..b6c5480325 100644 --- a/hook.h +++ b/hook.h @@ -11,6 +11,7 @@ struct hook { enum config_scope origin; /* The literal command to run. */ struct strbuf command; + unsigned from_hookdir : 1; }; /* diff --git a/t/t1360-config-based-hooks.sh b/t/t1360-config-based-hooks.sh index 6e4a3e763f..0f12af4659 100755 --- a/t/t1360-config-based-hooks.sh +++ b/t/t1360-config-based-hooks.sh @@ -23,6 +23,14 @@ setup_hookcmd () { test_config_global hookcmd.abc.command "/path/abc" --add } +setup_hookdir () { + mkdir .git/hooks + write_script .git/hooks/pre-commit <<-EOF + echo \"Legacy Hook\" + EOF + test_when_finished rm -rf .git/hooks +} + test_expect_success 'git hook rejects commands without a mode' ' test_must_fail git hook pre-commit ' @@ -85,4 +93,15 @@ test_expect_success 'git hook list reorders on duplicate commands' ' test_cmp expected actual ' +test_expect_success 'git hook list shows hooks from the hookdir' ' + setup_hookdir && + + cat >expected <<-EOF && + hookdir: $(pwd)/.git/hooks/pre-commit + EOF + + git hook list pre-commit >actual && + test_cmp expected actual +' + test_done From patchwork Thu May 27 00:08:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 12282997 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-21.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 82A7BC47089 for ; Thu, 27 May 2021 00:09:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 68FD46128D for ; Thu, 27 May 2021 00:09:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233935AbhE0AK6 (ORCPT ); Wed, 26 May 2021 20:10:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51538 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233972AbhE0AKo (ORCPT ); Wed, 26 May 2021 20:10:44 -0400 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 14924C061760 for ; Wed, 26 May 2021 17:09:10 -0700 (PDT) Received: by mail-yb1-xb4a.google.com with SMTP id e198-20020a25e7cf0000b029052a1d34778fso3617630ybh.13 for ; Wed, 26 May 2021 17:09:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=5u3Jomj2WzIKSJPoNZOXmDBZeXPSqG0sey2cRsleXNE=; b=ugeCRYEB847rmGpoazcpjcx2ccUDCFFUb2Io+GV0lyMpaH0X2lGVcIKJNoH22AnVAE rOncj+qd6nD5RdJNhUxHWQVVdr0Zh9whd6ypcebtoXa7lALYcNwgKGbx3m84UEr3me0S crT1k09T6/zqLuRpuSMCq9QADLhayYEASampEuU8KP8/NgPbSLIKsHaNUXotMKQmQZ86 Xl6JIUVgqWVsg08SL27mRB+xi7/vHZBkAzrjElmSrgItDBjDfjYLd2HiUuuUybbdZc/q 39kp8BYsh6LiLmHTsEuECJmWXXs2Qi0tIzXGJU2nJYYjGhkTsL7rZVJHoAAhFPx4bzAQ yPvw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=5u3Jomj2WzIKSJPoNZOXmDBZeXPSqG0sey2cRsleXNE=; b=Fl71G0JRpS08+eWcZzuSnChbCPfzsHEMjEtdG7y5dECLvbJ5z5g+Z6+qYJXAjzjj5g AYrhJu0Xd9VUymt7Bsi1VZB/vcqarrhgYw4geQORZfF3IhaIwXaWURGhMn5OIwBmFK09 p7vA85lyJsxX8D9EiCMNmFNgjmZPxUQaYxkq2ZhYXVHsq8IDCCOpa43klF4KEr7AkXZH C9XT9+fznZWU8+k2/cotgkZr9JdOghjCdHy9Euq0xL4whE77WFdMmkGSfYXJxM8vqSao ESX9bfPeieFlctwqMWxphJhNOSZ7xK3VZeQGOpnMV4oyxoX0cCXk1CM79ttbjPhQ4T9w HGmw== X-Gm-Message-State: AOAM531BwXOdMufRQoJW6C3BXynBe4MPvyGIUHI9Mbd5zV9XLH1crIoB yVb0kVB05B0PfAdBFDl9ci3v5uTjAQ8yTRzcwTgu1xr5u6nimoQiXgIQIM5Zb0w0oi1UQLWyttW Kb3uvYckzIEtgExDUN1WdDTzD856ScecuWv7f4Qj/XKPKBssdLxLCTqCAXUnc1d/0yDdy+Oy86A == X-Google-Smtp-Source: ABdhPJxa/hPhqJXJs7nfNaEAQFzJrcpF/Z/6ZBi1lis28IWAYSBngVoEREfcEtUwM2V/RK9r6JYMT3ggJbQ4N2a1XuU= X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:c1a4:4d87:8b5a:d12c]) (user=emilyshaffer job=sendgmr) by 2002:a25:df8e:: with SMTP id w136mr893269ybg.404.1622074149235; Wed, 26 May 2021 17:09:09 -0700 (PDT) Date: Wed, 26 May 2021 17:08:23 -0700 In-Reply-To: <20210527000856.695702-1-emilyshaffer@google.com> Message-Id: <20210527000856.695702-5-emilyshaffer@google.com> Mime-Version: 1.0 References: <20210527000856.695702-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v9 04/37] hook: teach hook.runHookDir From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org For now, just give a hint about how these hooks will be run in 'git hook list'. Later on, though, we will pay attention to this enum when running the hooks. --- Notes: Since v7, tidied up the behavior of the HOOK_UNKNOWN flag and added a test to enforce it - now it matches the design doc much better. Also, thanks Jonathan Tan for pointing out that the commit message made no sense and was targeted for a different change. Rewrote the commit message now. Plus, added HOOK_ERROR flag per Junio and Jonathan Nieder. Newly split into its own commit since v4, and taking place much sooner. An unfortunate side effect of adding this support *before* the hook.runHookDir support is that the labels on the list are not clear - because we aren't yet flagging which hooks are from the hookdir versus the config. I suppose we could move the addition of that field to the struct hook up to this patch, but it didn't make a lot of sense to me to do it just for cosmetic purposes. Since v7, tidied up the behavior of the HOOK_UNKNOWN flag and added a test to enforce it - now it matches the design doc much better. Also, thanks Jonathan Tan for pointing out that the commit message made no sense and was targeted for a different change. Rewrote the commit message now. Newly split into its own commit since v4, and taking place much sooner. An unfortunate side effect of adding this support *before* the hook.runHookDir support is that the labels on the list are not clear - because we aren't yet flagging which hooks are from the hookdir versus the config. I suppose we could move the addition of that field to the struct hook up to this patch, but it didn't make a lot of sense to me to do it just for cosmetic purposes. Documentation/config/hook.txt | 5 ++ builtin/hook.c | 98 ++++++++++++++++++++++++++++++----- hook.c | 24 +++++++++ hook.h | 22 ++++++++ t/t1360-config-based-hooks.sh | 71 +++++++++++++++++++++++++ 5 files changed, 206 insertions(+), 14 deletions(-) diff --git a/Documentation/config/hook.txt b/Documentation/config/hook.txt index 71449ecbc7..75312754ae 100644 --- a/Documentation/config/hook.txt +++ b/Documentation/config/hook.txt @@ -7,3 +7,8 @@ hookcmd..command:: A command to execute during a hook for which has been specified as a command. This can be an executable on your device or a oneliner for your shell. See linkgit:git-hook[1]. + +hook.runHookDir:: + Controls how hooks contained in your hookdir are executed. Can be any of + "yes", "warn", "interactive", or "no". Defaults to "yes". See + linkgit:git-hook[1] and linkgit:git-config[1] "core.hooksPath"). diff --git a/builtin/hook.c b/builtin/hook.c index e82725f0a6..b1e63a9576 100644 --- a/builtin/hook.c +++ b/builtin/hook.c @@ -10,10 +10,13 @@ static const char * const builtin_hook_usage[] = { NULL }; +static enum hookdir_opt should_run_hookdir; + static int list(int argc, const char **argv, const char *prefix) { struct list_head *head, *pos; const char *hookname = NULL; + struct strbuf hookdir_annotation = STRBUF_INIT; struct option list_options[] = { OPT_END(), @@ -41,37 +44,104 @@ static int list(int argc, const char **argv, const char *prefix) struct hook *item = list_entry(pos, struct hook, list); item = list_entry(pos, struct hook, list); if (item) { - /* - * TRANSLATORS: ": ". Both fields - * should be left untranslated; config scope matches the - * output of 'git config --show-scope'. Marked for - * translation to provide better RTL support later. - */ - printf(_("%s: %s\n"), - (item->from_hookdir - ? "hookdir" - : config_scope_name(item->origin)), - item->command.buf); + if (item->from_hookdir) { + /* + * TRANSLATORS: do not translate 'hookdir' as + * it matches the config setting. + */ + switch (should_run_hookdir) { + case HOOKDIR_NO: + printf(_("hookdir: %s (will not run)\n"), + item->command.buf); + break; + case HOOKDIR_ERROR: + printf(_("hookdir: %s (will error and not run)\n"), + item->command.buf); + break; + case HOOKDIR_INTERACTIVE: + printf(_("hookdir: %s (will prompt)\n"), + item->command.buf); + break; + case HOOKDIR_WARN: + printf(_("hookdir: %s (will warn but run)\n"), + item->command.buf); + break; + case HOOKDIR_YES: + /* + * The default behavior should agree with + * hook.c:configured_hookdir_opt(). HOOKDIR_UNKNOWN should just + * do the default behavior. + */ + case HOOKDIR_UNKNOWN: + default: + printf(_("hookdir: %s\n"), + item->command.buf); + break; + } + } else { + /* + * TRANSLATORS: ": ". Both fields + * should be left untranslated; config scope matches the + * output of 'git config --show-scope'. Marked for + * translation to provide better RTL support later. + */ + printf(_("%s: %s\n"), + config_scope_name(item->origin), + item->command.buf); + } } } clear_hook_list(head); + strbuf_release(&hookdir_annotation); return 0; } int cmd_hook(int argc, const char **argv, const char *prefix) { + const char *run_hookdir = NULL; + struct option builtin_hook_options[] = { + OPT_STRING(0, "run-hookdir", &run_hookdir, N_("option"), + N_("what to do with hooks found in the hookdir")), OPT_END(), }; - if (argc < 2) + + argc = parse_options(argc, argv, prefix, builtin_hook_options, + builtin_hook_usage, 0); + + /* after the parse, we should have " " */ + if (argc < 1) usage_with_options(builtin_hook_usage, builtin_hook_options); git_config(git_default_config, NULL); - if (!strcmp(argv[1], "list")) - return list(argc - 1, argv + 1, prefix); + + /* argument > config */ + if (run_hookdir) + if (!strcmp(run_hookdir, "no")) + should_run_hookdir = HOOKDIR_NO; + else if (!strcmp(run_hookdir, "error")) + should_run_hookdir = HOOKDIR_ERROR; + else if (!strcmp(run_hookdir, "yes")) + should_run_hookdir = HOOKDIR_YES; + else if (!strcmp(run_hookdir, "warn")) + should_run_hookdir = HOOKDIR_WARN; + else if (!strcmp(run_hookdir, "interactive")) + should_run_hookdir = HOOKDIR_INTERACTIVE; + else + /* + * TRANSLATORS: leave "yes/warn/interactive/no" + * untranslated; the strings are compared literally. + */ + die(_("'%s' is not a valid option for --run-hookdir " + "(yes, warn, interactive, no)"), run_hookdir); + else + should_run_hookdir = configured_hookdir_opt(); + + if (!strcmp(argv[0], "list")) + return list(argc, argv, prefix); usage_with_options(builtin_hook_usage, builtin_hook_options); } diff --git a/hook.c b/hook.c index b4994fc108..030051cab2 100644 --- a/hook.c +++ b/hook.c @@ -102,6 +102,30 @@ static int hook_config_lookup(const char *key, const char *value, void *cb_data) return 0; } +enum hookdir_opt configured_hookdir_opt(void) +{ + const char *key; + if (git_config_get_value("hook.runhookdir", &key)) + return HOOKDIR_YES; /* by default, just run it. */ + + if (!strcmp(key, "no")) + return HOOKDIR_NO; + + if (!strcmp(key, "error")) + return HOOKDIR_ERROR; + + if (!strcmp(key, "yes")) + return HOOKDIR_YES; + + if (!strcmp(key, "warn")) + return HOOKDIR_WARN; + + if (!strcmp(key, "interactive")) + return HOOKDIR_INTERACTIVE; + + return HOOKDIR_UNKNOWN; +} + struct list_head* hook_list(const char* hookname) { struct strbuf hook_key = STRBUF_INIT; diff --git a/hook.h b/hook.h index b6c5480325..7f2b2ee8f2 100644 --- a/hook.h +++ b/hook.h @@ -20,6 +20,28 @@ struct hook { */ struct list_head* hook_list(const char *hookname); +enum hookdir_opt +{ + HOOKDIR_NO, + HOOKDIR_ERROR, + HOOKDIR_WARN, + HOOKDIR_INTERACTIVE, + HOOKDIR_YES, + HOOKDIR_UNKNOWN, +}; + +/* + * Provides the hookdir_opt specified in the config without consulting any + * command line arguments. + */ +enum hookdir_opt configured_hookdir_opt(void); + +/* + * Provides the hookdir_opt specified in the config without consulting any + * command line arguments. + */ +enum hookdir_opt configured_hookdir_opt(void); + /* Free memory associated with a 'struct hook' */ void free_hook(struct hook *ptr); /* Empties the list at 'head', calling 'free_hook()' on each entry */ diff --git a/t/t1360-config-based-hooks.sh b/t/t1360-config-based-hooks.sh index 0f12af4659..141e6f7590 100755 --- a/t/t1360-config-based-hooks.sh +++ b/t/t1360-config-based-hooks.sh @@ -104,4 +104,75 @@ test_expect_success 'git hook list shows hooks from the hookdir' ' test_cmp expected actual ' +test_expect_success 'hook.runHookDir = no is respected by list' ' + setup_hookdir && + + test_config hook.runHookDir "no" && + + cat >expected <<-EOF && + hookdir: $(pwd)/.git/hooks/pre-commit (will not run) + EOF + + git hook list pre-commit >actual && + # the hookdir annotation is translated + test_cmp expected actual +' + +test_expect_success 'hook.runHookDir = error is respected by list' ' + setup_hookdir && + + test_config hook.runHookDir "error" && + + cat >expected <<-EOF && + hookdir: $(pwd)/.git/hooks/pre-commit (will error and not run) + EOF + + git hook list pre-commit >actual && + # the hookdir annotation is translated + test_cmp expected actual +' + +test_expect_success 'hook.runHookDir = warn is respected by list' ' + setup_hookdir && + + test_config hook.runHookDir "warn" && + + cat >expected <<-EOF && + hookdir: $(pwd)/.git/hooks/pre-commit (will warn but run) + EOF + + git hook list pre-commit >actual && + # the hookdir annotation is translated + test_cmp expected actual +' + + +test_expect_success 'hook.runHookDir = interactive is respected by list' ' + setup_hookdir && + + test_config hook.runHookDir "interactive" && + + cat >expected <<-EOF && + hookdir: $(pwd)/.git/hooks/pre-commit (will prompt) + EOF + + git hook list pre-commit >actual && + # the hookdir annotation is translated + test_cmp expected actual +' + +test_expect_success 'hook.runHookDir is tolerant to unknown values' ' + setup_hookdir && + + test_config hook.runHookDir "junk" && + + cat >expected <<-EOF && + hookdir: $(pwd)/.git/hooks/pre-commit + EOF + + git hook list pre-commit >actual && + # the hookdir annotation is translated + test_cmp expected actual +' + test_done From patchwork Thu May 27 00:08:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 12282999 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A202BC47082 for ; Thu, 27 May 2021 00:09:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7ED4A613BE for ; Thu, 27 May 2021 00:09:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234979AbhE0AK7 (ORCPT ); Wed, 26 May 2021 20:10:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51570 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234972AbhE0AKt (ORCPT ); Wed, 26 May 2021 20:10:49 -0400 Received: from mail-qk1-x749.google.com (mail-qk1-x749.google.com [IPv6:2607:f8b0:4864:20::749]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 08502C06138B for ; Wed, 26 May 2021 17:09:12 -0700 (PDT) Received: by mail-qk1-x749.google.com with SMTP id s123-20020a3777810000b02902e9adec2313so2105613qkc.4 for ; Wed, 26 May 2021 17:09:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=SXHg8b+uFdBO2stA984yAarTPvmFraUymxfuXjYJCTI=; b=pH5tQ9wdXB8e5nUYEje2Hg1JvNXr+nhqDtrMZxps++/nnVRLfgiec24BSSneALCsAw vhtHGyzdJtQy+4MKjDxRFQkZ0IaP8P08Im4VgYIuAEGSeCmeJ1SD8fviBj2BL78WDiow nrcPQ7rJj2paB5Vv1peV6BiWDPTqvBN8r5eJYurTVITY+6dTCI0vnwdDt06f1K1yCYSw 6QNUnf7qin6yIADEslsGwfVvmKbhljxMS2YwltSvsCgR0xuZTGal0LOgwcE73JEEEtoa BSv73/BVqlYVC9gDiGL3anfF/cybwCVBfAuVBDjG6QMYWIUvvjwQGQpTSStDpvoSnNSZ D5sA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=SXHg8b+uFdBO2stA984yAarTPvmFraUymxfuXjYJCTI=; b=Fh/tf4c/1/zfsPm/LItXh/TnmTWjXjCWf91k2LpEblGqQ26JZe3Oflk5zase3JSdZh djvqXkjlh9bycSWSMZkkWFDebBsEaIqQT98cv9Pq3u4O5rq0JKibUHM1PgaWoy7ZmnK5 mxNVaKiUup0jpzk24FlFxDc0B4Wh9ZNX7mZIYbS1Yjl1WlhP/H/hkJVFGvWp5obNbwLZ hjgOsyLUslh7DPvL8oiwV8HG783rU5iizkGL6s3qtPx0N77KVk3Usm1rmoch9OMXeMBq Mb8Y6nkAH1HGqGIVJnCirykIceqDFu08jLVcv2/7GbyU+QNPK4ywCsViQG/RyaUhk6NF Smng== X-Gm-Message-State: AOAM530sDXeuXFkpi2Mm17lL7dU1T9bIIkM7lcudub6G6HWJ65j/a6VU NBnPPfzXxio5lYT7dfHUFYFajABlNhF9ZEqgLQR3LfGXUCYMCTwfzC1q8ABjOEkBE+wkS9pBbru ShmV0Q/YngruQ1A+zi15qJsY6ah1jqvWueBM+M200sCSBVXUxvDx8WxLe1RwOFyjQ2Hx4VHYJMg == X-Google-Smtp-Source: ABdhPJx+rAxrXF19GEwkxeJNuwOWdUcjKKIvgIU5G/mXpJc++oO6bTTfZCFI85T4XZqmw5aExaVc0tDCR9coe3x9DSk= X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:c1a4:4d87:8b5a:d12c]) (user=emilyshaffer job=sendgmr) by 2002:a05:6214:19c8:: with SMTP id j8mr1081111qvc.42.1622074151120; Wed, 26 May 2021 17:09:11 -0700 (PDT) Date: Wed, 26 May 2021 17:08:24 -0700 In-Reply-To: <20210527000856.695702-1-emilyshaffer@google.com> Message-Id: <20210527000856.695702-6-emilyshaffer@google.com> Mime-Version: 1.0 References: <20210527000856.695702-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v9 05/37] hook: implement hookcmd..skip From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org If a user wants a specific repo to skip execution of a hook which is set at a global or system level, they will be able to do so by specifying 'skip' in their repo config: ~/.gitconfig [hook.pre-commit] command = skippable-oneliner command = skippable-hookcmd [hookcmd.skippable-hookcmd] command = foo.sh $GIT_DIR/.git/config [hookcmd.skippable-oneliner] skip = true [hookcmd.skippable-hookcmd] skip = true Later it may make sense to add an option like "hookcmd..-skip" - but for simplicity, let's start with a universal skip setting like this. Signed-off-by: Emily Shaffer --- Documentation/config/hook.txt | 8 ++++++++ Documentation/git-hook.txt | 33 +++++++++++++++++++++++++++++++++ hook.c | 35 ++++++++++++++++++++++++++--------- t/t1360-config-based-hooks.sh | 35 +++++++++++++++++++++++++++++++++++ 4 files changed, 102 insertions(+), 9 deletions(-) diff --git a/Documentation/config/hook.txt b/Documentation/config/hook.txt index 75312754ae..8b12512e33 100644 --- a/Documentation/config/hook.txt +++ b/Documentation/config/hook.txt @@ -8,6 +8,14 @@ hookcmd..command:: as a command. This can be an executable on your device or a oneliner for your shell. See linkgit:git-hook[1]. +hookcmd..skip:: + Specify this boolean to remove a command from earlier in the execution + order. Useful if you want to make a single repo an exception to hook + configured at the system or global scope. If there is no hookcmd + specified for the command you want to skip, you can use the value of + `hook..command` as as a shortcut. The "skip" setting + must be specified after the "hook..command" to have an effect. + hook.runHookDir:: Controls how hooks contained in your hookdir are executed. Can be any of "yes", "warn", "interactive", or "no". Defaults to "yes". See diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt index f19875ed68..c84520cb38 100644 --- a/Documentation/git-hook.txt +++ b/Documentation/git-hook.txt @@ -54,6 +54,39 @@ $ git hook list "prepare-commit-msg" local: /bin/linter --c ---- +If there is a command you wish to run in most cases but have one or two +exceptional repos where it should be skipped, you can use specify +`hookcmd..skip`, for example: + +System config +---- + [hook "pre-commit"] + command = check-for-secrets + + [hookcmd "check-for-secrets"] + command = /bin/secret-checker --aggressive +---- + +Local config +---- + [hookcmd "check-for-secrets"] + skip = true + # This works for inlined hook commands, too: + [hookcmd "~/typocheck.sh"] + skip = true +---- + +After these configs are added, the hook list becomes: + +---- +$ git hook list "post-commit" +global: /bin/linter --c +local: python ~/run-test-suite.py + +$ git hook list "pre-commit" +no commands configured for hook 'pre-commit' +---- + COMMANDS -------- diff --git a/hook.c b/hook.c index 030051cab2..65cbad8dba 100644 --- a/hook.c +++ b/hook.c @@ -12,24 +12,25 @@ void free_hook(struct hook *ptr) } } -static void append_or_move_hook(struct list_head *head, const char *command) +static struct hook * find_hook_by_command(struct list_head *head, const char *command) { struct list_head *pos = NULL, *tmp = NULL; - struct hook *to_add = NULL; + struct hook *found = NULL; - /* - * remove the prior entry with this command; we'll replace it at the - * end. - */ list_for_each_safe(pos, tmp, head) { struct hook *it = list_entry(pos, struct hook, list); if (!strcmp(it->command.buf, command)) { list_del(pos); - /* we'll simply move the hook to the end */ - to_add = it; + found = it; break; } } + return found; +} + +static void append_or_move_hook(struct list_head *head, const char *command) +{ + struct hook *to_add = find_hook_by_command(head, command); if (!to_add) { /* adding a new hook, not moving an old one */ @@ -74,12 +75,22 @@ static int hook_config_lookup(const char *key, const char *value, void *cb_data) if (!strcmp(key, hook_key)) { const char *command = value; struct strbuf hookcmd_name = STRBUF_INIT; + int skip = 0; + + /* + * Check if we're removing that hook instead. Hookcmds are + * removed by name, and inlined hooks are removed by command + * content. + */ + strbuf_addf(&hookcmd_name, "hookcmd.%s.skip", command); + git_config_get_bool(hookcmd_name.buf, &skip); /* * Check if a hookcmd with that name exists. If it doesn't, * 'git_config_get_value()' is documented not to touch &command, * so we don't need to do anything. */ + strbuf_reset(&hookcmd_name); strbuf_addf(&hookcmd_name, "hookcmd.%s.command", command); git_config_get_value(hookcmd_name.buf, &command); @@ -94,7 +105,13 @@ static int hook_config_lookup(const char *key, const char *value, void *cb_data) * for each key+value, do_callback(key, value, cb_data) */ - append_or_move_hook(head, command); + if (skip) { + struct hook *to_remove = find_hook_by_command(head, command); + if (to_remove) + remove_hook(&(to_remove->list)); + } else { + append_or_move_hook(head, command); + } strbuf_release(&hookcmd_name); } diff --git a/t/t1360-config-based-hooks.sh b/t/t1360-config-based-hooks.sh index 141e6f7590..33ac27aa97 100755 --- a/t/t1360-config-based-hooks.sh +++ b/t/t1360-config-based-hooks.sh @@ -146,6 +146,41 @@ test_expect_success 'hook.runHookDir = warn is respected by list' ' test_cmp expected actual ' +test_expect_success 'git hook list removes skipped hookcmd' ' + setup_hookcmd && + test_config hookcmd.abc.skip "true" --add && + + cat >expected <<-EOF && + no commands configured for hook '\''pre-commit'\'' + EOF + + git hook list pre-commit >actual && + test_cmp expected actual +' + +test_expect_success 'git hook list ignores skip referring to unused hookcmd' ' + test_config hookcmd.abc.command "/path/abc" --add && + test_config hookcmd.abc.skip "true" --add && + + cat >expected <<-EOF && + no commands configured for hook '\''pre-commit'\'' + EOF + + git hook list pre-commit >actual && + test_cmp expected actual +' + +test_expect_success 'git hook list removes skipped inlined hook' ' + setup_hooks && + test_config hookcmd."$ROOT/path/ghi".skip "true" --add && + + cat >expected <<-EOF && + global: $ROOT/path/def + EOF + + git hook list pre-commit >actual && + test_cmp expected actual +' test_expect_success 'hook.runHookDir = interactive is respected by list' ' setup_hookdir && From patchwork Thu May 27 00:08:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 12283001 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 23577C47089 for ; Thu, 27 May 2021 00:09:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 069F0613BE for ; Thu, 27 May 2021 00:09:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235030AbhE0ALB (ORCPT ); Wed, 26 May 2021 20:11:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51576 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233936AbhE0AKv (ORCPT ); Wed, 26 May 2021 20:10:51 -0400 Received: from mail-qk1-x74a.google.com (mail-qk1-x74a.google.com [IPv6:2607:f8b0:4864:20::74a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D95E5C06138C for ; Wed, 26 May 2021 17:09:13 -0700 (PDT) Received: by mail-qk1-x74a.google.com with SMTP id r25-20020a05620a03d9b02903a58bfe037cso2080661qkm.15 for ; Wed, 26 May 2021 17:09:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=OlFFOx1J937SauwGyV2zTpiyzLPKZyfZEkDezuJbeHI=; b=cTjvaYtf9OTX9Notft4xoUUmP11j05EqqXCKD9Uaj1hyv9erlzCUmddkw+sGPuTSRQ rTrE/n+LqSLbx9G9A/N8cQMeQg3fCQK7Zf18+goJaT/F3Ojxi51YGC4iC5t4Z5CAwBN+ g/DkdzN70yxJj2nIPZdqTDkmqJKf9n7mylTw3DQ4gEP52HnQ7r9GUPYi+I/37iORo9mw i4l45iGmn2I3shcVvuwADo68Zw6Iqt5uTeQ9wrSPU+26GcFKWYzrmJcnYg3xSNpaUS7y mKzmMbEzgphK/so+lTF0vUR+7EeVmyPXYh390yNnU4T69u47F7U5hXWjtNyBhROahBqN ulCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=OlFFOx1J937SauwGyV2zTpiyzLPKZyfZEkDezuJbeHI=; b=UWFx3H8jv7tyFtF9syk0MxHdclXgpYksB/XyC/F83djODCVlCFkGdAJPNb35AB3QJd SpadXPWlqdp6Q2o+0WGBcp/cpvISQY/oDMH1WLAn55eDQnwQXWFp2qB757RpcQTmbd9j qdVvktNPfQkB5QXKjvZME6xtwqvOs7II7gqlhFHsWi64D1ynMxI/iKTacMjYNWtpuT5X JOddq4GHCCjChm83jw9kl9GM+Tkndt9Dg6YPxQVCsxLqDtke/B/1RcdI0wUvDMaBI4PM Z+SOD3NJpQcXEVLiNVoFJhMJzKkPRgvQAcQzt/6k89POeya+34QM+XhSB3tanAVCfc3i yxfA== X-Gm-Message-State: AOAM530P8UarVJ+FmhKf3OR9zVkkBZSxgTWihGrI0I/aTAdcNqMHi+S/ U7ST/KWa/x5KbigJwal6fDGIO4R1/BpaHMtxhKdNx6M/rSmPNE0cyrAqBJYQYRfPadPrN74wAum 6C4+6XBhlL7ip3UXnWXs8HPA9fN16GHCoYFm9J1zoS2nWhVQsA/FRwYP4qWPBAWV1NMwx8ap4LQ == X-Google-Smtp-Source: ABdhPJyJdaVE5kwJ5j6c7qM+2aghQYEoEbwEIYKRh5Y9rKXUJubNNkdG13HXOToAuC+FCokZOF6GgFJa+eGfT6Qc+C0= X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:c1a4:4d87:8b5a:d12c]) (user=emilyshaffer job=sendgmr) by 2002:ad4:5fcb:: with SMTP id jq11mr784371qvb.61.1622074152950; Wed, 26 May 2021 17:09:12 -0700 (PDT) Date: Wed, 26 May 2021 17:08:25 -0700 In-Reply-To: <20210527000856.695702-1-emilyshaffer@google.com> Message-Id: <20210527000856.695702-7-emilyshaffer@google.com> Mime-Version: 1.0 References: <20210527000856.695702-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v9 06/37] parse-options: parse into strvec From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org parse-options already knows how to read into a string_list, and it knows how to read into an strvec as a passthrough (that is, including the argument as well as its value). string_list and strvec serve similar purposes but are somewhat painful to convert between; so, let's teach parse-options to read values of string arguments directly into an strvec without preserving the argument name. This is useful if collecting generic arguments to pass through to another command, for example, 'git hook run --arg "--quiet" --arg "--format=pretty" some-hook'. The resulting strvec would contain { "--quiet", "--format=pretty" }. The implementation is based on that of OPT_STRING_LIST. Signed-off-by: Emily Shaffer --- Notes: Since v7, updated the reference doc to make the intended usage for OPT_STRVEC more clear. Since v4, fixed one or two more places where I missed the argv_array->strvec rename. Documentation/technical/api-parse-options.txt | 7 +++++ parse-options-cb.c | 16 +++++++++++ parse-options.h | 4 +++ t/helper/test-parse-options.c | 6 +++++ t/t0040-parse-options.sh | 27 +++++++++++++++++++ 5 files changed, 60 insertions(+) diff --git a/Documentation/technical/api-parse-options.txt b/Documentation/technical/api-parse-options.txt index 5a60bbfa7f..f79b17e7fc 100644 --- a/Documentation/technical/api-parse-options.txt +++ b/Documentation/technical/api-parse-options.txt @@ -173,6 +173,13 @@ There are some macros to easily define options: The string argument is stored as an element in `string_list`. Use of `--no-option` will clear the list of preceding values. +`OPT_STRVEC(short, long, &struct strvec, arg_str, description)`:: + Introduce an option with a string argument, meant to be specified + multiple times. + The string argument is stored as an element in `strvec`, and later + arguments are added to the same `strvec`. + Use of `--no-option` will clear the list of preceding values. + `OPT_INTEGER(short, long, &int_var, description)`:: Introduce an option with integer argument. The integer is put into `int_var`. diff --git a/parse-options-cb.c b/parse-options-cb.c index 3c811e1e4a..8227499eb6 100644 --- a/parse-options-cb.c +++ b/parse-options-cb.c @@ -207,6 +207,22 @@ int parse_opt_string_list(const struct option *opt, const char *arg, int unset) return 0; } +int parse_opt_strvec(const struct option *opt, const char *arg, int unset) +{ + struct strvec *v = opt->value; + + if (unset) { + strvec_clear(v); + return 0; + } + + if (!arg) + return -1; + + strvec_push(v, arg); + return 0; +} + int parse_opt_noop_cb(const struct option *opt, const char *arg, int unset) { return 0; diff --git a/parse-options.h b/parse-options.h index a845a9d952..fcb0f1f31e 100644 --- a/parse-options.h +++ b/parse-options.h @@ -178,6 +178,9 @@ struct option { #define OPT_STRING_LIST(s, l, v, a, h) \ { OPTION_CALLBACK, (s), (l), (v), (a), \ (h), 0, &parse_opt_string_list } +#define OPT_STRVEC(s, l, v, a, h) \ + { OPTION_CALLBACK, (s), (l), (v), (a), \ + (h), 0, &parse_opt_strvec } #define OPT_UYN(s, l, v, h) { OPTION_CALLBACK, (s), (l), (v), NULL, \ (h), PARSE_OPT_NOARG, &parse_opt_tertiary } #define OPT_EXPIRY_DATE(s, l, v, h) \ @@ -297,6 +300,7 @@ int parse_opt_commits(const struct option *, const char *, int); int parse_opt_commit(const struct option *, const char *, int); int parse_opt_tertiary(const struct option *, const char *, int); int parse_opt_string_list(const struct option *, const char *, int); +int parse_opt_strvec(const struct option *, const char *, int); int parse_opt_noop_cb(const struct option *, const char *, int); enum parse_opt_result parse_opt_unknown_cb(struct parse_opt_ctx_t *ctx, const struct option *, diff --git a/t/helper/test-parse-options.c b/t/helper/test-parse-options.c index 2051ce57db..922af56156 100644 --- a/t/helper/test-parse-options.c +++ b/t/helper/test-parse-options.c @@ -2,6 +2,7 @@ #include "cache.h" #include "parse-options.h" #include "string-list.h" +#include "strvec.h" #include "trace2.h" static int boolean = 0; @@ -15,6 +16,7 @@ static char *string = NULL; static char *file = NULL; static int ambiguous; static struct string_list list = STRING_LIST_INIT_NODUP; +static struct strvec vector = STRVEC_INIT; static struct { int called; @@ -133,6 +135,7 @@ int cmd__parse_options(int argc, const char **argv) OPT_STRING('o', NULL, &string, "str", "get another string"), OPT_NOOP_NOARG(0, "obsolete"), OPT_STRING_LIST(0, "list", &list, "str", "add str to list"), + OPT_STRVEC(0, "vector", &vector, "str", "add str to strvec"), OPT_GROUP("Magic arguments"), OPT_ARGUMENT("quux", NULL, "means --quux"), OPT_NUMBER_CALLBACK(&integer, "set integer to NUM", @@ -183,6 +186,9 @@ int cmd__parse_options(int argc, const char **argv) for (i = 0; i < list.nr; i++) show(&expect, &ret, "list: %s", list.items[i].string); + for (i = 0; i < vector.nr; i++) + show(&expect, &ret, "vector: %s", vector.v[i]); + for (i = 0; i < argc; i++) show(&expect, &ret, "arg %02d: %s", i, argv[i]); diff --git a/t/t0040-parse-options.sh b/t/t0040-parse-options.sh index ad4746d899..485e0170bf 100755 --- a/t/t0040-parse-options.sh +++ b/t/t0040-parse-options.sh @@ -35,6 +35,7 @@ String options --st get another string (pervert ordering) -o get another string --list add str to list + --vector add str to strvec Magic arguments --quux means --quux @@ -386,6 +387,32 @@ test_expect_success '--no-list resets list' ' test_cmp expect output ' +cat >expect <<\EOF +boolean: 0 +integer: 0 +magnitude: 0 +timestamp: 0 +string: (not set) +abbrev: 7 +verbose: -1 +quiet: 0 +dry run: no +file: (not set) +vector: foo +vector: bar +vector: baz +EOF +test_expect_success '--vector keeps list of strings' ' + test-tool parse-options --vector foo --vector=bar --vector=baz >output && + test_cmp expect output +' + +test_expect_success '--no-vector resets list' ' + test-tool parse-options --vector=other --vector=irrelevant --vector=options \ + --no-vector --vector=foo --vector=bar --vector=baz >output && + test_cmp expect output +' + test_expect_success 'multiple quiet levels' ' test-tool parse-options --expect="quiet: 3" -q -q -q ' From patchwork Thu May 27 00:08:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 12283005 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 28AE2C47082 for ; Thu, 27 May 2021 00:09:33 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0F6346128D for ; Thu, 27 May 2021 00:09:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235038AbhE0ALD (ORCPT ); Wed, 26 May 2021 20:11:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51578 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234727AbhE0AKv (ORCPT ); Wed, 26 May 2021 20:10:51 -0400 Received: from mail-qv1-xf49.google.com (mail-qv1-xf49.google.com [IPv6:2607:f8b0:4864:20::f49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CCA2FC06138D for ; Wed, 26 May 2021 17:09:15 -0700 (PDT) Received: by mail-qv1-xf49.google.com with SMTP id n12-20020a0c8c0c0000b02901edb8963d4dso2545924qvb.18 for ; Wed, 26 May 2021 17:09:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=q2FyjGuAwIdBJraOhx4+uUvajHAeUPiaFGbAAleSHOA=; b=mmDW0OzaD1Yl5j5HWxBm233FuntXefTlp7JgvrQ1X5LYqFwzaZFjD7Edwb3xJbwRs3 +bWdzEuCsb1VSnpcnndXX5i3IcSt2qlNENLP7aXk39E5eP6hE6ZYtFS/d9VIi3fQPryR mGhfVbaNoSe0SIKmMDfnM7dN5/QMI3tvrPpOldIRwA/jvw326A9hTVPkdtFL4txvChWo VUWYfwXFQY+Muk5dQ5SoXhziVg9SuQ7PGDWA4YtRF7saD9O/vN7pw0SuL2Xa0PPX/dQ1 yfV9Hp4XZJgVZ+Z9G/mXWRO6nu1KSXn02vAB6I5InbCY0ePnhOc77qdE7EFx27Hmjui2 LRjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=q2FyjGuAwIdBJraOhx4+uUvajHAeUPiaFGbAAleSHOA=; b=Q6Cap3HZQ+L2Fyxo3qmme+Spldl47CtInEzpjCZw+g+R3/vh6fN3wiqhQshXZm46dW upGkhbiTmZl8Btdi/l5R++Aj88LRPladUP1scmvt+Sh410jiXgRF21gWUZonD2h8xrwo VxkoC/4WRdobhg2hWfFJTxOUD4kbzSnDtxBe3U+bj68BAjJg5d1nkTMR8OS3mN9Pp1Rb TbbajzuyANzT6AZmYZsJxgTYqI9aniyBIA5PEqb5MCThceT+yWC/RE9scsXC3ZUedRim r0BEDgDaTmGxgP21zXgt6quh41aeyR6iEW5lDBTF5TMEldpBnFpyi12n1pjipi24IXoz BZkA== X-Gm-Message-State: AOAM5315EwstGmL0GZdi9FSzb/SalZ3YJf/ND7LNVcTqWOuZI7ZJ/jV/ FTT48+Ekfk09z7jvH6bCJj5SI/PusK920EZU78qL+SpN4i0CpNvR4gnHQ8rmX/1QG7jdj8t4Mxl VHwGfbgiHLFDpEDpSiHKIjXXYZTyVB0LCGKQ1MV+CmG6K//L8GpvnebFvj+7iduaizFLNtHlR3w == X-Google-Smtp-Source: ABdhPJyc0Xj4GOHvNo7gOBHFWz8tZRcLFptatTCNboZUirZd3U2YRFT7gKTZcgTQR1sN5/d6xAcw/8JMrLevj/in1jk= X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:c1a4:4d87:8b5a:d12c]) (user=emilyshaffer job=sendgmr) by 2002:a05:6214:c6c:: with SMTP id t12mr552655qvj.34.1622074154810; Wed, 26 May 2021 17:09:14 -0700 (PDT) Date: Wed, 26 May 2021 17:08:26 -0700 In-Reply-To: <20210527000856.695702-1-emilyshaffer@google.com> Message-Id: <20210527000856.695702-8-emilyshaffer@google.com> Mime-Version: 1.0 References: <20210527000856.695702-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v9 07/37] hook: add 'run' subcommand From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In order to enable hooks to be run as an external process, by a standalone Git command, or by tools which wrap Git, provide an external means to run all configured hook commands for a given hook event. For now, the hook commands will run in config order, in series. As alternate ordering or parallelism is supported in the future, we should add knobs to use those to the command line as well. As with the legacy hook implementation, all stdout generated by hook commands is redirected to stderr. Piping from stdin is not yet supported. Legacy hooks (those present in $GITDIR/hooks) are run at the end of the execution list. They can be disabled, or made to print warnings, or to prompt before running, with the 'hook.runHookDir' config. Users may wish to provide hook commands like 'git config hook.pre-commit.command "~/linter.sh --pre-commit"'. To enable this, config-defined hooks are run in a shell. (Since hooks in $GITDIR/hooks can't be specified with included arguments or paths which need expansion like this, they are run without a shell instead.) Signed-off-by: Emily Shaffer --- Notes: Since v7, added support for "error" hook.runHookDir setting. Since v4, updated the docs, and did less local application of single quotes. In order for hookdir hooks to run successfully with a space in the path, though, they must not be run with 'sh -c'. So we can treat the hookdir hooks specially, and warn users via doc about special considerations for configured hooks with spaces in their path. Documentation/git-hook.txt | 31 +++++++- builtin/hook.c | 42 ++++++++++- hook.c | 137 +++++++++++++++++++++++++++++++++- hook.h | 26 ++++++- t/t1360-config-based-hooks.sh | 96 +++++++++++++++++++++++- 5 files changed, 320 insertions(+), 12 deletions(-) diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt index c84520cb38..8f96c347ea 100644 --- a/Documentation/git-hook.txt +++ b/Documentation/git-hook.txt @@ -9,11 +9,12 @@ SYNOPSIS -------- [verse] 'git hook' list +'git hook' run [(-e|--env)=...] [(-a|--arg)=...] DESCRIPTION ----------- -You can list configured hooks with this command. Later, you will be able to run, -add, and modify hooks with this command. +You can list and run configured hooks with this command. Later, you will be able +to add and modify hooks with this command. This command parses the default configuration files for sections `hook` and `hookcmd`. `hook` is used to describe the commands which will be run during a @@ -97,6 +98,32 @@ in the order they should be run, and print the config scope where the relevant `hook..command` was specified, not the `hookcmd` (if applicable). This output is human-readable and the format is subject to change over time. +run [(-e|--env)=...] [(-a|--arg)=...] ``:: + +Runs hooks configured for ``, in the same order displayed by `git +hook list`. Hooks configured this way may be run prepended with `sh -c`, so +paths containing special characters or spaces should be wrapped in single +quotes: `command = '/my/path with spaces/script.sh' some args`. + +OPTIONS +------- +--run-hookdir:: + Overrides the hook.runHookDir config. Must be 'yes', 'warn', + 'interactive', or 'no'. Specifies how to handle hooks located in the Git + hook directory (core.hooksPath). + +-a:: +--arg:: + Only valid for `run`. ++ +Specify arguments to pass to every hook that is run. + +-e:: +--env:: + Only valid for `run`. ++ +Specify environment variables to set for every hook that is run. + CONFIGURATION ------------- include::config/hook.txt[] diff --git a/builtin/hook.c b/builtin/hook.c index b1e63a9576..4673c9091c 100644 --- a/builtin/hook.c +++ b/builtin/hook.c @@ -4,9 +4,11 @@ #include "hook.h" #include "parse-options.h" #include "strbuf.h" +#include "strvec.h" static const char * const builtin_hook_usage[] = { N_("git hook list "), + N_("git hook run [(-e|--env)=...] [(-a|--arg)=...] "), NULL }; @@ -98,6 +100,40 @@ static int list(int argc, const char **argv, const char *prefix) return 0; } +static int run(int argc, const char **argv, const char *prefix) +{ + struct strbuf hookname = STRBUF_INIT; + struct run_hooks_opt opt; + int rc = 0; + + struct option run_options[] = { + OPT_STRVEC('e', "env", &opt.env, N_("var"), + N_("environment variables for hook to use")), + OPT_STRVEC('a', "arg", &opt.args, N_("args"), + N_("argument to pass to hook")), + OPT_END(), + }; + + run_hooks_opt_init(&opt); + + argc = parse_options(argc, argv, prefix, run_options, + builtin_hook_usage, 0); + + if (argc < 1) + usage_msg_opt(_("You must specify a hook event to run."), + builtin_hook_usage, run_options); + + strbuf_addstr(&hookname, argv[0]); + opt.run_hookdir = should_run_hookdir; + + rc = run_hooks(hookname.buf, &opt); + + strbuf_release(&hookname); + run_hooks_opt_clear(&opt); + + return rc; +} + int cmd_hook(int argc, const char **argv, const char *prefix) { const char *run_hookdir = NULL; @@ -109,10 +145,10 @@ int cmd_hook(int argc, const char **argv, const char *prefix) }; argc = parse_options(argc, argv, prefix, builtin_hook_options, - builtin_hook_usage, 0); + builtin_hook_usage, PARSE_OPT_KEEP_UNKNOWN); /* after the parse, we should have " " */ - if (argc < 1) + if (argc < 2) usage_with_options(builtin_hook_usage, builtin_hook_options); git_config(git_default_config, NULL); @@ -142,6 +178,8 @@ int cmd_hook(int argc, const char **argv, const char *prefix) if (!strcmp(argv[0], "list")) return list(argc, argv, prefix); + if (!strcmp(argv[0], "run")) + return run(argc, argv, prefix); usage_with_options(builtin_hook_usage, builtin_hook_options); } diff --git a/hook.c b/hook.c index 65cbad8dba..b631da659b 100644 --- a/hook.c +++ b/hook.c @@ -3,13 +3,13 @@ #include "hook.h" #include "config.h" #include "run-command.h" +#include "prompt.h" void free_hook(struct hook *ptr) { - if (ptr) { + if (ptr) strbuf_release(&ptr->command); - free(ptr); - } + free(ptr); } static struct hook * find_hook_by_command(struct list_head *head, const char *command) @@ -143,6 +143,70 @@ enum hookdir_opt configured_hookdir_opt(void) return HOOKDIR_UNKNOWN; } +static int should_include_hookdir(const char *path, enum hookdir_opt cfg) +{ + struct strbuf prompt = STRBUF_INIT; + /* + * If the path doesn't exist, don't bother adding the empty hook and + * don't bother checking the config or prompting the user. + */ + if (!path) + return 0; + + switch (cfg) + { + case HOOKDIR_ERROR: + fprintf(stderr, _("Skipping legacy hook at '%s'\n"), + path); + /* FALLTHROUGH */ + case HOOKDIR_NO: + return 0; + case HOOKDIR_WARN: + fprintf(stderr, _("Running legacy hook at '%s'\n"), + path); + return 1; + case HOOKDIR_INTERACTIVE: + do { + /* + * TRANSLATORS: Make sure to include [Y] and [n] + * in your translation. Only English input is + * accepted. Default option is "yes". + */ + fprintf(stderr, _("Run '%s'? [Y/n] "), path); + git_read_line_interactively(&prompt); + /* + * In case of prompt = '' - that is, user hit enter, + * saying "yes I want the default" - strncasecmp will + * return 0 regardless. So list the default first. + * + * Case insensitively, accept "y", "ye", or "yes" as + * "yes"; accept "n" or "no" as "no". + */ + if (!strncasecmp(prompt.buf, "yes", prompt.len)) { + strbuf_release(&prompt); + return 1; + } else if (!strncasecmp(prompt.buf, "no", prompt.len)) { + strbuf_release(&prompt); + return 0; + } + /* otherwise, we didn't understand the input */ + } while (prompt.len); /* an empty reply means default (yes) */ + return 1; + /* + * HOOKDIR_UNKNOWN should match the default behavior, but let's + * give a heads up to the user. + */ + case HOOKDIR_UNKNOWN: + fprintf(stderr, + _("Unrecognized value for 'hook.runHookDir'. " + "Is there a typo? ")); + /* FALLTHROUGH */ + case HOOKDIR_YES: + default: + return 1; + } +} + struct list_head* hook_list(const char* hookname) { struct strbuf hook_key = STRBUF_INIT; @@ -176,3 +240,70 @@ struct list_head* hook_list(const char* hookname) strbuf_release(&hook_key); return hook_head; } + +void run_hooks_opt_init(struct run_hooks_opt *o) +{ + strvec_init(&o->env); + strvec_init(&o->args); + o->run_hookdir = configured_hookdir_opt(); +} + +void run_hooks_opt_clear(struct run_hooks_opt *o) +{ + strvec_clear(&o->env); + strvec_clear(&o->args); +} + +static void prepare_hook_cp(const char *hookname, struct hook *hook, + struct run_hooks_opt *options, + struct child_process *cp) +{ + if (!hook) + return; + + cp->no_stdin = 1; + cp->env = options->env.v; + cp->stdout_to_stderr = 1; + cp->trace2_hook_name = hookname; + + /* + * Commands from the config could be oneliners, but we know + * for certain that hookdir commands are not. + */ + cp->use_shell = !hook->from_hookdir; + + /* add command */ + strvec_push(&cp->args, hook->command.buf); + + /* + * add passed-in argv, without expanding - let the user get back + * exactly what they put in + */ + strvec_pushv(&cp->args, options->args.v); +} + +int run_hooks(const char *hookname, struct run_hooks_opt *options) +{ + struct list_head *to_run, *pos = NULL, *tmp = NULL; + int rc = 0; + + if (!options) + BUG("a struct run_hooks_opt must be provided to run_hooks"); + + to_run = hook_list(hookname); + + list_for_each_safe(pos, tmp, to_run) { + struct child_process hook_proc = CHILD_PROCESS_INIT; + struct hook *hook = list_entry(pos, struct hook, list); + + if (hook->from_hookdir && + !should_include_hookdir(hook->command.buf, options->run_hookdir)) + continue; + + prepare_hook_cp(hookname, hook, options, &hook_proc); + + rc |= run_command(&hook_proc); + } + + return rc; +} diff --git a/hook.h b/hook.h index 7f2b2ee8f2..fb5132305f 100644 --- a/hook.h +++ b/hook.h @@ -1,6 +1,7 @@ #include "config.h" #include "list.h" #include "strbuf.h" +#include "strvec.h" struct hook { struct list_head list; @@ -36,11 +37,30 @@ enum hookdir_opt */ enum hookdir_opt configured_hookdir_opt(void); +struct run_hooks_opt +{ + /* Environment vars to be set for each hook */ + struct strvec env; + + /* Args to be passed to each hook */ + struct strvec args; + + /* + * How should the hookdir be handled? + * Leave the RUN_HOOKS_OPT_INIT default in most cases; this only needs + * to be overridden if the user can override it at the command line. + */ + enum hookdir_opt run_hookdir; +}; + +void run_hooks_opt_init(struct run_hooks_opt *o); +void run_hooks_opt_clear(struct run_hooks_opt *o); + /* - * Provides the hookdir_opt specified in the config without consulting any - * command line arguments. + * Runs all hooks associated to the 'hookname' event in order. Each hook will be + * passed 'env' and 'args'. */ -enum hookdir_opt configured_hookdir_opt(void); +int run_hooks(const char *hookname, struct run_hooks_opt *options); /* Free memory associated with a 'struct hook' */ void free_hook(struct hook *ptr); diff --git a/t/t1360-config-based-hooks.sh b/t/t1360-config-based-hooks.sh index 33ac27aa97..3dddd41e4f 100755 --- a/t/t1360-config-based-hooks.sh +++ b/t/t1360-config-based-hooks.sh @@ -115,7 +115,10 @@ test_expect_success 'hook.runHookDir = no is respected by list' ' git hook list pre-commit >actual && # the hookdir annotation is translated - test_cmp expected actual + test_cmp expected actual && + + git hook run pre-commit 2>actual && + test_must_be_empty actual ' test_expect_success 'hook.runHookDir = error is respected by list' ' @@ -129,6 +132,13 @@ test_expect_success 'hook.runHookDir = error is respected by list' ' git hook list pre-commit >actual && # the hookdir annotation is translated + test_cmp expected actual && + + cat >expected <<-EOF && + Skipping legacy hook at '\''$(pwd)/.git/hooks/pre-commit'\'' + EOF + + git hook run pre-commit 2>actual && test_cmp expected actual ' @@ -143,6 +153,14 @@ test_expect_success 'hook.runHookDir = warn is respected by list' ' git hook list pre-commit >actual && # the hookdir annotation is translated + test_cmp expected actual && + + cat >expected <<-EOF && + Running legacy hook at '\''$(pwd)/.git/hooks/pre-commit'\'' + "Legacy Hook" + EOF + + git hook run pre-commit 2>actual && test_cmp expected actual ' @@ -182,7 +200,7 @@ test_expect_success 'git hook list removes skipped inlined hook' ' test_cmp expected actual ' -test_expect_success 'hook.runHookDir = interactive is respected by list' ' +test_expect_success 'hook.runHookDir = interactive is respected by list and run' ' setup_hookdir && test_config hook.runHookDir "interactive" && @@ -193,9 +211,83 @@ test_expect_success 'hook.runHookDir = interactive is respected by list' ' git hook list pre-commit >actual && # the hookdir annotation is translated + test_cmp expected actual && + + test_write_lines n | git hook run pre-commit 2>actual && + ! grep "Legacy Hook" actual && + + test_write_lines y | git hook run pre-commit 2>actual && + grep "Legacy Hook" actual +' + +test_expect_success 'inline hook definitions execute oneliners' ' + test_config hook.pre-commit.command "echo \"Hello World\"" && + + echo "Hello World" >expected && + + # hooks are run with stdout_to_stderr = 1 + git hook run pre-commit 2>actual && + test_cmp expected actual +' + +test_expect_success 'inline hook definitions resolve paths' ' + write_script sample-hook.sh <<-EOF && + echo \"Sample Hook\" + EOF + + test_when_finished "rm sample-hook.sh" && + + test_config hook.pre-commit.command "\"$(pwd)/sample-hook.sh\"" && + + echo \"Sample Hook\" >expected && + + # hooks are run with stdout_to_stderr = 1 + git hook run pre-commit 2>actual && + test_cmp expected actual +' + +test_expect_success 'git hook run can pass args and env vars' ' + write_script sample-hook.sh <<-\EOF && + echo $1 + echo $2 + echo $TEST_ENV_1 + echo $TEST_ENV_2 + EOF + + test_config hook.pre-commit.command "\"$(pwd)/sample-hook.sh\"" && + + cat >expected <<-EOF && + arg1 + arg2 + env1 + env2 + EOF + + git hook run --arg arg1 \ + --env TEST_ENV_1=env1 \ + -a arg2 \ + -e TEST_ENV_2=env2 \ + pre-commit 2>actual && + test_cmp expected actual ' +test_expect_success 'hookdir hook included in git hook run' ' + setup_hookdir && + + echo \"Legacy Hook\" >expected && + + # hooks are run with stdout_to_stderr = 1 + git hook run pre-commit 2>actual && + test_cmp expected actual +' + +test_expect_success 'out-of-repo runs excluded' ' + setup_hooks && + + nongit test_must_fail git hook run pre-commit +' + test_expect_success 'hook.runHookDir is tolerant to unknown values' ' setup_hookdir && From patchwork Thu May 27 00:08:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 12283003 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 96FE4C47088 for ; Thu, 27 May 2021 00:09:33 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7922B613BE for ; Thu, 27 May 2021 00:09:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235047AbhE0ALE (ORCPT ); Wed, 26 May 2021 20:11:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51580 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233764AbhE0AKv (ORCPT ); Wed, 26 May 2021 20:10:51 -0400 Received: from mail-qk1-x74a.google.com (mail-qk1-x74a.google.com [IPv6:2607:f8b0:4864:20::74a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 945DDC061574 for ; Wed, 26 May 2021 17:09:17 -0700 (PDT) Received: by mail-qk1-x74a.google.com with SMTP id i141-20020a379f930000b02902e94f6d938dso2103194qke.5 for ; Wed, 26 May 2021 17:09:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=wOw1tZPeGzt3zbyODH5czmUPBe+O1PfnhDXsKLtPqtc=; b=nmMKey+ANLGLQkPFp3mGmd3ZkkSBnH3j/5t0yvK1PBfU3flc9KxWSaVXYAsjoQVtLa HylrdwPS2dHWmV+njRxjH1rptVckhqpInp4rE3bLIsDKYkS8C7l76nDjo6MDHAY95FXc lp/XxNcgW+r/n/ACTaUT2EWybist26ToY+f5iSvIOC2462nn/4FiSgtauAVvqN6BihS/ s5vOrDLiWIy3sUGHfi7IUKOUbH0rcNrqNzsyfSxzToLD5s/NENAyNZE8DHzzkqGqediP PgsiBEq2X9ytjn0a0QBHkT71DRsLZYgBjIaJpHB5qRviM285e+ptKwSboX0Waf/yjjFA wlYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=wOw1tZPeGzt3zbyODH5czmUPBe+O1PfnhDXsKLtPqtc=; b=qTF8ac6RAVSFI2zmtVsSEgcXvMP5PbGeatwIhYZ9cXO5AREVysxLyHUxfaZG6itYX3 nKD6RDUto4qvJuSayiTlk18y3fwvh/gLF7sfdPw/iDX2xZzqKunGAA/tAgWmfnNOlasf o7Ss+S2Mskbijf69Gow2d3pJepCFcaXmD1sYq43/8iR0dCk8SbJAKridAvfaghG87t5S z1NpFh02ob/m2JxJIZFCOl4csuSuT5vFreXfjcQTx1iV/pzL0AF/bV64zCqyExirhA5n IPz9X3jghkF10AAQB4b8FrtzOz+M/HhYyHWU/B6AaLiCoOoPfqvYow3FZByoW7St3Oam GrTA== X-Gm-Message-State: AOAM531EkBES7zIs8mNNlXI418XGheQigfH2i+PmuBcJk4jV1UOc0gAO H63zW60EWldl+IqqrWleTSeFGvqw/vjLCWLlVprMrGEqz3qd4SXzu2zdiAM3udKKq0HsxkfGt/Q vUcacyyb79B+kY20njzYp+2VtmtXeyzvhKvUUT3qwgyDBLgwYu9sKQn2pUbvr2yqAeEr5aNrGVg == X-Google-Smtp-Source: ABdhPJzkz6JJY+gMyHFb5ezkYgqRJCI29HShEO4ekhOsXmTKmqQSkEVs5u9cBY9AkxWivkC89zvUt5/foBC18zAOJWo= X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:c1a4:4d87:8b5a:d12c]) (user=emilyshaffer job=sendgmr) by 2002:a05:6214:391:: with SMTP id l17mr844887qvy.22.1622074156713; Wed, 26 May 2021 17:09:16 -0700 (PDT) Date: Wed, 26 May 2021 17:08:27 -0700 In-Reply-To: <20210527000856.695702-1-emilyshaffer@google.com> Message-Id: <20210527000856.695702-9-emilyshaffer@google.com> Mime-Version: 1.0 References: <20210527000856.695702-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v9 08/37] hook: introduce hook_exists() From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Add a helper to easily determine whether any hooks exist for a given hook event. Many callers want to check whether some state could be modified by a hook; that check should include the config-based hooks as well. Optimize by checking the config directly. Since commands which execute hooks might want to take args to replace 'hook.runHookDir', let 'hook_exists()' take a hookdir_opt to override that config. In some cases, external callers today use find_hook() to discover the location of a hook and then run it manually with run-command.h (that is, not with run_hook_le()). Later, those cases will call hook.h:run_hook() directly instead. Once the entire codebase is using hook_exists() instead of find_hook(), find_hook() can be safely rolled into hook_exists() and removed from run-command.h. Signed-off-by: Emily Shaffer --- hook.c | 19 +++++++++++++++++++ hook.h | 10 ++++++++++ 2 files changed, 29 insertions(+) diff --git a/hook.c b/hook.c index b631da659b..008167dbe5 100644 --- a/hook.c +++ b/hook.c @@ -248,6 +248,25 @@ void run_hooks_opt_init(struct run_hooks_opt *o) o->run_hookdir = configured_hookdir_opt(); } +int hook_exists(const char *hookname, enum hookdir_opt should_run_hookdir) +{ + const char *value = NULL; /* throwaway */ + struct strbuf hook_key = STRBUF_INIT; + int could_run_hookdir; + + if (should_run_hookdir == HOOKDIR_USE_CONFIG) + should_run_hookdir = configured_hookdir_opt(); + + could_run_hookdir = (should_run_hookdir == HOOKDIR_INTERACTIVE || + should_run_hookdir == HOOKDIR_WARN || + should_run_hookdir == HOOKDIR_YES) + && !!find_hook(hookname); + + strbuf_addf(&hook_key, "hook.%s.command", hookname); + + return (!git_config_get_value(hook_key.buf, &value)) || could_run_hookdir; +} + void run_hooks_opt_clear(struct run_hooks_opt *o) { strvec_clear(&o->env); diff --git a/hook.h b/hook.h index fb5132305f..5f770b53ed 100644 --- a/hook.h +++ b/hook.h @@ -23,6 +23,7 @@ struct list_head* hook_list(const char *hookname); enum hookdir_opt { + HOOKDIR_USE_CONFIG, HOOKDIR_NO, HOOKDIR_ERROR, HOOKDIR_WARN, @@ -56,6 +57,15 @@ struct run_hooks_opt void run_hooks_opt_init(struct run_hooks_opt *o); void run_hooks_opt_clear(struct run_hooks_opt *o); +/* + * Returns 1 if any hooks are specified in the config or if a hook exists in the + * hookdir. Typically, invoke hook_exsts() like: + * hook_exists(hookname, configured_hookdir_opt()); + * Like with run_hooks, if you take a --run-hookdir flag, reflect that + * user-specified behavior here instead. + */ +int hook_exists(const char *hookname, enum hookdir_opt should_run_hookdir); + /* * Runs all hooks associated to the 'hookname' event in order. Each hook will be * passed 'env' and 'args'. From patchwork Thu May 27 00:08:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 12283007 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4E1ADC47089 for ; Thu, 27 May 2021 00:09:34 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2BFC6613BE for ; Thu, 27 May 2021 00:09:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235050AbhE0ALF (ORCPT ); Wed, 26 May 2021 20:11:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51568 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234122AbhE0AKy (ORCPT ); Wed, 26 May 2021 20:10:54 -0400 Received: from mail-qv1-xf49.google.com (mail-qv1-xf49.google.com [IPv6:2607:f8b0:4864:20::f49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 65105C06175F for ; Wed, 26 May 2021 17:09:19 -0700 (PDT) Received: by mail-qv1-xf49.google.com with SMTP id h11-20020a0ceecb0000b0290211ed54e716so2566747qvs.9 for ; Wed, 26 May 2021 17:09:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=g4Md1XkQ5N67CgVq854t25haHVEaoZQdKXfMwtnQtVs=; b=L/HgEFUjQr5uh+BvYC6YXjTTNtQgMUgiCTONFyTJx/21jIRYA2CHKlwqPDIIl0yYNt gsn967pEeIQoBSDgADJRpCKWuQd+dHSz9mveRb9T0pDNkK9TtQbJYZg8frVpvwmga0Wx f7NnojXDZediMFy2AqY966rBIKO2K2NpL1Qac/krVJenVLfJCmcwRdy1Ok41nFo/obH3 oL0JcuQocBr/NdFLnJWmBbDPzEX96e163EifPtHYa0Xl2uhJ/rGXFXGu17qrP3NjzVGb FrCGGflVyWwm3+euWoHG9pRA/zcUsGTkXTwg8IFb14InCNEy4aRIrvdJthK4t6UeTXHD lLlA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=g4Md1XkQ5N67CgVq854t25haHVEaoZQdKXfMwtnQtVs=; b=e3GM5reOYhwdWIyRvz+LGWSgSMI/1PI8CDx+zIIU+2gQapYp2+P7WyLqP5XlVy6hzq bWXVFCITvO5HnvxDRy0dZaFjaNYkCox5QWuym51G1w0hDPS1Bn/++7lBoZ3b6H0TGYIk LUhuV2HVX7jubeVcIeZwvNSxazKOzmv/OMeVq26JsTtwRKpqvPH7PO13ZorUuD197cIC 6IDLa3nO+SBiaTJ+vAkv++Zg33Le2FOquDdau+78roCdcdo8ZbtdWJEr5M1aNmhlxhIV VAdZsEmfnEE/ZWNqet4drSzmRVEQfLyycF/sat3NoyAkLiQ/ak6F6jIgSgXX3vH6GQ0r TcSQ== X-Gm-Message-State: AOAM5312+yC6NUu+lPsuU1hX9AtaS+eFY2uQgA4lG9YyK5kmHY3crxtd WVUtprhL5IjXl6Eg0winSfwoAe32iQ68mWMkl6H+br7Fy37KOevRRdi6Sm/ylUgeTuQ3YeCbOor 4mMsDIBhLHsPquVsxMrYRXNWrAntXqcThAxYQyKIf9pVdItkVJAbOZ2kCkhE8MmVLzSXu7sEdQA == X-Google-Smtp-Source: ABdhPJzCSwoOG9rvxkGKjOvCREz6eAl+azuVJCzguJ1WDHnOQn4tZfSkgS2MopGLCZNSbVAqENNwLVrTM1nCk7B6wns= X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:c1a4:4d87:8b5a:d12c]) (user=emilyshaffer job=sendgmr) by 2002:a0c:efcf:: with SMTP id a15mr1040880qvt.55.1622074158505; Wed, 26 May 2021 17:09:18 -0700 (PDT) Date: Wed, 26 May 2021 17:08:28 -0700 In-Reply-To: <20210527000856.695702-1-emilyshaffer@google.com> Message-Id: <20210527000856.695702-10-emilyshaffer@google.com> Mime-Version: 1.0 References: <20210527000856.695702-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v9 09/37] hook: support passing stdin to hooks From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Some hooks (such as post-rewrite) need to take input via stdin. Previously, callers provided stdin to hooks by setting run-command.h:child_process.in, which takes a FD. Callers would open the file in question themselves before calling run-command(). However, since we will now need to seek to the front of the file and read it again for every hook which runs, hook.h:run_command() takes a path and handles FD management itself. Since this file is opened for read only, it should not prevent later parallel execution support. On the frontend, this is supported by asking for a file path, rather than by reading stdin. Reading directly from stdin would involve caching the entire stdin (to memory or to disk) and reading it back from the beginning to each hook. We'd want to support cases like insufficient memory or storage for the file. While this may prove useful later, for now the path of least resistance is to just ask the user to make this interim file themselves. Signed-off-by: Emily Shaffer --- Documentation/git-hook.txt | 11 +++++++++-- builtin/hook.c | 5 ++++- hook.c | 8 +++++++- hook.h | 6 +++++- t/t1360-config-based-hooks.sh | 24 ++++++++++++++++++++++++ 5 files changed, 49 insertions(+), 5 deletions(-) diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt index 8f96c347ea..96a857c682 100644 --- a/Documentation/git-hook.txt +++ b/Documentation/git-hook.txt @@ -9,7 +9,8 @@ SYNOPSIS -------- [verse] 'git hook' list -'git hook' run [(-e|--env)=...] [(-a|--arg)=...] +'git hook' run [(-e|--env)=...] [(-a|--arg)=...] [--to-stdin=] + DESCRIPTION ----------- @@ -98,7 +99,7 @@ in the order they should be run, and print the config scope where the relevant `hook..command` was specified, not the `hookcmd` (if applicable). This output is human-readable and the format is subject to change over time. -run [(-e|--env)=...] [(-a|--arg)=...] ``:: +run [(-e|--env)=...] [(-a|--arg)=...] [--to-stdin=] ``:: Runs hooks configured for ``, in the same order displayed by `git hook list`. Hooks configured this way may be run prepended with `sh -c`, so @@ -124,6 +125,12 @@ Specify arguments to pass to every hook that is run. + Specify environment variables to set for every hook that is run. +--to-stdin:: + Only valid for `run`. ++ +Specify a file which will be streamed into stdin for every hook that is run. +Each hook will receive the entire file from beginning to EOF. + CONFIGURATION ------------- include::config/hook.txt[] diff --git a/builtin/hook.c b/builtin/hook.c index 4673c9091c..0d9c052e84 100644 --- a/builtin/hook.c +++ b/builtin/hook.c @@ -8,7 +8,8 @@ static const char * const builtin_hook_usage[] = { N_("git hook list "), - N_("git hook run [(-e|--env)=...] [(-a|--arg)=...] "), + N_("git hook run [(-e|--env)=...] [(-a|--arg)=...]" + "[--to-stdin=] "), NULL }; @@ -111,6 +112,8 @@ static int run(int argc, const char **argv, const char *prefix) N_("environment variables for hook to use")), OPT_STRVEC('a', "arg", &opt.args, N_("args"), N_("argument to pass to hook")), + OPT_STRING(0, "to-stdin", &opt.path_to_stdin, N_("path"), + N_("file to read into hooks' stdin")), OPT_END(), }; diff --git a/hook.c b/hook.c index 008167dbe5..a2eda57fb9 100644 --- a/hook.c +++ b/hook.c @@ -245,6 +245,7 @@ void run_hooks_opt_init(struct run_hooks_opt *o) { strvec_init(&o->env); strvec_init(&o->args); + o->path_to_stdin = NULL; o->run_hookdir = configured_hookdir_opt(); } @@ -280,7 +281,12 @@ static void prepare_hook_cp(const char *hookname, struct hook *hook, if (!hook) return; - cp->no_stdin = 1; + /* reopen the file for stdin; run_command closes it. */ + if (options->path_to_stdin) + cp->in = xopen(options->path_to_stdin, O_RDONLY); + else + cp->no_stdin = 1; + cp->env = options->env.v; cp->stdout_to_stderr = 1; cp->trace2_hook_name = hookname; diff --git a/hook.h b/hook.h index 5f770b53ed..37147f4c5e 100644 --- a/hook.h +++ b/hook.h @@ -52,6 +52,9 @@ struct run_hooks_opt * to be overridden if the user can override it at the command line. */ enum hookdir_opt run_hookdir; + + /* Path to file which should be piped to stdin for each hook */ + const char *path_to_stdin; }; void run_hooks_opt_init(struct run_hooks_opt *o); @@ -68,7 +71,8 @@ int hook_exists(const char *hookname, enum hookdir_opt should_run_hookdir); /* * Runs all hooks associated to the 'hookname' event in order. Each hook will be - * passed 'env' and 'args'. + * passed 'env' and 'args'. The file at 'stdin_path' will be closed and reopened + * for each hook that runs. */ int run_hooks(const char *hookname, struct run_hooks_opt *options); diff --git a/t/t1360-config-based-hooks.sh b/t/t1360-config-based-hooks.sh index 3dddd41e4f..43917172d7 100755 --- a/t/t1360-config-based-hooks.sh +++ b/t/t1360-config-based-hooks.sh @@ -302,4 +302,28 @@ test_expect_success 'hook.runHookDir is tolerant to unknown values' ' test_cmp expected actual ' +test_expect_success 'stdin to multiple hooks' ' + git config --add hook.test.command "xargs -P1 -I% echo a%" && + git config --add hook.test.command "xargs -P1 -I% echo b%" && + test_when_finished "test_unconfig hook.test.command" && + + cat >input <<-EOF && + 1 + 2 + 3 + EOF + + cat >expected <<-EOF && + a1 + a2 + a3 + b1 + b2 + b3 + EOF + + git hook run --to-stdin=input test 2>actual && + test_cmp expected actual +' + test_done From patchwork Thu May 27 00:08:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 12283009 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3D665C47088 for ; Thu, 27 May 2021 00:09:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1845C613BE for ; Thu, 27 May 2021 00:09:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235064AbhE0ALG (ORCPT ); Wed, 26 May 2021 20:11:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51610 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235017AbhE0AK5 (ORCPT ); Wed, 26 May 2021 20:10:57 -0400 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3F5ACC061761 for ; Wed, 26 May 2021 17:09:21 -0700 (PDT) Received: by mail-yb1-xb4a.google.com with SMTP id u7-20020a259b470000b02904dca50820c2so3655437ybo.11 for ; Wed, 26 May 2021 17:09:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=JXaUam9jeE2CtQntpC2GRDCstNDRkBt3WO/Vq6o61SQ=; b=VKOUgtpeV5WMe7ZvGIO+dFAxBcbAzsvFWUQ/GoGsW9j44Ao6ffrHuT0uPMeLCEqWDS teM7r3CfCO8AIqbSLC3pM4zqaAkFat4+6GUnIZToCFvWaUQJQew9pQRWQIjujxyUgbK5 7drgeXQ1zasOhSdzNkJV5pvD9tIDAdQKiZ5fKTSroTttyZYQQUIz0JLiIwXU4KZsWxeO 7JWPTW6YOVacCfnWeBi27gdjyQYzYyxf42YdCWe/ny5ZOPc4K2aogH0LpKqaPTVAK0h4 6RQgeupwBf5jm8P5/bd/cVJUHqgweQJXd///D48hp9pD5S0F30XTv/Pbeslvw695nlfp aMJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=JXaUam9jeE2CtQntpC2GRDCstNDRkBt3WO/Vq6o61SQ=; b=CFEpcW8H+gyHGsdmxuhprvO5fzxYmDrtaCbliip4PNfm6AR0PLITsFJb1GuHRqk6c6 901XUB/lP5Mmacs76cYHj7tny4v60euoljqkT8hLKYh2iU/naFZMI4/QB7M4pLv8p2sc LsiUpQp72OVTLtEMl2vpLqUbiLeQGzrkJtCfA5JYFgeGBIRjEjmCKnVZDNsUV9m/fsrl b3NqN/C6mcc4KFpfbDP0Rdm67fTk3xpBqzIozEwDztM5LIeO9rWH+7KrpJ2iKd/R4sWd X7vS5lA4KI/+z3OYST2rjw6snyCIEjl43NADONWPFA9nK9JN8VfdMyLQnoDETzYwoT9k /HGg== X-Gm-Message-State: AOAM531ZjDL/nqbl8/hbxYvXQNbcF3eAOVCI/qnyAEAHWxRB7p2bFRTC ZMN1aj8kmgnMVNmSALCs8DKqc2i7gzwR1P6B4ZqUnWVDkYu0VQkYBtOqxcvbzms+hWrALXGlOvk ct3TSSyH1oRDUOCiy2m17Z7l/qXnyxkY7/ZqsWsOZG3M6Av0k735W5WN3TI3AhshU3FXPCpdEOQ == X-Google-Smtp-Source: ABdhPJywyxx3Q/X50IQRGggntxamuoP/DgknCwNEesl0JTH+UCDACYFv16IPrYXy4Ciaeo9PQtCBk9PinYlFQqzutDI= X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:c1a4:4d87:8b5a:d12c]) (user=emilyshaffer job=sendgmr) by 2002:a25:8804:: with SMTP id c4mr827637ybl.469.1622074160424; Wed, 26 May 2021 17:09:20 -0700 (PDT) Date: Wed, 26 May 2021 17:08:29 -0700 In-Reply-To: <20210527000856.695702-1-emilyshaffer@google.com> Message-Id: <20210527000856.695702-11-emilyshaffer@google.com> Mime-Version: 1.0 References: <20210527000856.695702-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v9 10/37] run-command: allow stdin for run_processes_parallel From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org While it makes sense not to inherit stdin from the parent process to avoid deadlocking, it's not necessary to completely ban stdin to children. An informed user should be able to configure stdin safely. By setting `some_child.process.no_stdin=1` before calling `get_next_task()` we provide a reasonable default behavior but enable users to set up stdin streaming for themselves during the callback. `some_child.process.stdout_to_stderr`, however, remains unmodifiable by `get_next_task()` - the rest of the run_processes_parallel() API depends on child output in stderr. Signed-off-by: Emily Shaffer --- run-command.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/run-command.c b/run-command.c index be6bc128cd..5237984ebe 100644 --- a/run-command.c +++ b/run-command.c @@ -1693,6 +1693,14 @@ static int pp_start_one(struct parallel_processes *pp) if (i == pp->max_processes) BUG("bookkeeping is hard"); + /* + * By default, do not inherit stdin from the parent process - otherwise, + * all children would share stdin! Users may overwrite this to provide + * something to the child's stdin by having their 'get_next_task' + * callback assign 0 to .no_stdin and an appropriate integer to .in. + */ + pp->children[i].process.no_stdin = 1; + code = pp->get_next_task(&pp->children[i].process, &pp->children[i].err, pp->data, @@ -1704,7 +1712,6 @@ static int pp_start_one(struct parallel_processes *pp) } pp->children[i].process.err = -1; pp->children[i].process.stdout_to_stderr = 1; - pp->children[i].process.no_stdin = 1; if (start_command(&pp->children[i].process)) { code = pp->start_failure(&pp->children[i].err, From patchwork Thu May 27 00:08:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 12283011 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 384EDC47082 for ; Thu, 27 May 2021 00:09:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1C8B0613CA for ; Thu, 27 May 2021 00:09:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235099AbhE0ALT (ORCPT ); Wed, 26 May 2021 20:11:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51616 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234827AbhE0AK7 (ORCPT ); Wed, 26 May 2021 20:10:59 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C8341C061763 for ; Wed, 26 May 2021 17:09:23 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id 129-20020a2501870000b0290526b90a2992so3660669ybb.10 for ; Wed, 26 May 2021 17:09:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=vBF1RmMFwQJr3rNl8w+O7c1zzESPX+uCdyJN2Ei8ia8=; b=E/VstNFIjpZ/D+A2Ymc+aXHtcDSTCUIRi0aB8n4aHLrNWb+Hb/kN0xQY8E3q5JYIGn It6UxNtT7tch4DTVJi8x+pTfqb8zC9oqOmZR4MmUTyocS1zGZ9sXoZTD7nVu7kHh8A4t 8jqbMcMzv3NPZdgDomBwM0EX7IJpqOXf7qCm6mNQKAUc73+O7KEwZ5jaeledbcRhjTLP fF45gSAllgL/NEL//Sgn2A0maWIBLjR7NKzJk6Mw+Vjeaf150EHg+xcIlVPUYMIwe6wb zbl/t3Xrbqs8+nnVksXn2TSNv//XH7wb4LrbZxOT/nohOtpEYgdAcMe/jPm97N6qojDy 0Xbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=vBF1RmMFwQJr3rNl8w+O7c1zzESPX+uCdyJN2Ei8ia8=; b=AN5l0Bml6Pyb/NI4jBnCtEknC8g0JURAkz1hYWtkfzOrROI8seAH7C63WdxwuMPEAS v2LQK71IU/1v1m605S8sYFNwQ8EYzFvU8kNvMWpfgBcac5J1FTdF6B9NshxMEEFelhkJ u8wSlpEowgDBp0YZkTxbWj/qgwSSvfiLN9plnGyc7iOrEh40+FNBepHeD/1NBm3Pe9Yh BQRTjx+p+sVu2reKJES1czbiN/74PGscBILYkpn3RXaPycToLXzqerOEP7MB5D7/1aWH 28KiilZoxxIDMWlhIs/12jbnak+dJ/RW52AEFBv3QtUdOVir9aWonBaU0Wp5CiUMebO7 V5Sg== X-Gm-Message-State: AOAM530lkCqmvnTRNFJDeyJAIhjKu3qpJWMHFhLE7d2Az4IeQUEqeefT ILrAqMnVMA0zGbZ5sm4rAXD+cl1ZG8wBGami9+MhvllTwRw+EflTI7ACS63hyhqtuaiL6/WXUct s8RJo6ZJCQtfnv3NdK2UR7bsIBmgHSsbVcoac+f8Pgn7G4iD83jNI6OY9JxmqoRQ1qg3Ve9Lngg == X-Google-Smtp-Source: ABdhPJx74tXQ89xzhepUVfvTVfn3gX3MWEmrwc/6phZRjkOPqA400FWqhM1PBeXkZVBIhJc5AMfYlLAmI1bXBoqJ9f0= X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:c1a4:4d87:8b5a:d12c]) (user=emilyshaffer job=sendgmr) by 2002:a25:9942:: with SMTP id n2mr952158ybo.230.1622074162296; Wed, 26 May 2021 17:09:22 -0700 (PDT) Date: Wed, 26 May 2021 17:08:30 -0700 In-Reply-To: <20210527000856.695702-1-emilyshaffer@google.com> Message-Id: <20210527000856.695702-12-emilyshaffer@google.com> Mime-Version: 1.0 References: <20210527000856.695702-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v9 11/37] hook: allow parallel hook execution From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In many cases, there's no reason not to allow hooks to execute in parallel. run_processes_parallel() is well-suited - it's a task queue that runs its housekeeping in series, which means users don't need to worry about thread safety on their callback data. True multithreaded execution with the async_* functions isn't necessary here. Synchronous hook execution can be achieved by only allowing 1 job to run at a time. Teach run_hooks() to use that function for simple hooks which don't require stdin or capture of stderr. Signed-off-by: Emily Shaffer --- Notes: Per AEvar's request - parallel hook execution on day zero. In most ways run_processes_parallel() worked great for me - but it didn't have great support for hooks where we pipe to and from. I had to add this support later in the series. Since I modified an existing and in-use library I'd appreciate a keen look on these patches. - Emily Documentation/config/hook.txt | 5 ++ Documentation/git-hook.txt | 14 ++++- builtin/hook.c | 6 +- hook.c | 111 ++++++++++++++++++++++++++++------ hook.h | 22 ++++++- 5 files changed, 134 insertions(+), 24 deletions(-) diff --git a/Documentation/config/hook.txt b/Documentation/config/hook.txt index 8b12512e33..4f66bb35cf 100644 --- a/Documentation/config/hook.txt +++ b/Documentation/config/hook.txt @@ -20,3 +20,8 @@ hook.runHookDir:: Controls how hooks contained in your hookdir are executed. Can be any of "yes", "warn", "interactive", or "no". Defaults to "yes". See linkgit:git-hook[1] and linkgit:git-config[1] "core.hooksPath"). + +hook.jobs:: + Specifies how many hooks can be run simultaneously during parallelized + hook execution. If unspecified, defaults to the number of processors on + the current system. diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt index 96a857c682..81b8e94994 100644 --- a/Documentation/git-hook.txt +++ b/Documentation/git-hook.txt @@ -10,7 +10,7 @@ SYNOPSIS [verse] 'git hook' list 'git hook' run [(-e|--env)=...] [(-a|--arg)=...] [--to-stdin=] - + [(-j|--jobs) ] DESCRIPTION ----------- @@ -99,7 +99,7 @@ in the order they should be run, and print the config scope where the relevant `hook..command` was specified, not the `hookcmd` (if applicable). This output is human-readable and the format is subject to change over time. -run [(-e|--env)=...] [(-a|--arg)=...] [--to-stdin=] ``:: +run [(-e|--env)=...] [(-a|--arg)=...] [--to-stdin=] [(-j|--jobs)] ``:: Runs hooks configured for ``, in the same order displayed by `git hook list`. Hooks configured this way may be run prepended with `sh -c`, so @@ -131,6 +131,16 @@ Specify environment variables to set for every hook that is run. Specify a file which will be streamed into stdin for every hook that is run. Each hook will receive the entire file from beginning to EOF. +-j:: +--jobs:: + Only valid for `run`. ++ +Specify how many hooks to run simultaneously. If this flag is not specified, use +the value of the `hook.jobs` config. If the config is not specified, use the +number of CPUs on the current system. Some hooks may be ineligible for +parallelization: for example, 'commit-msg' intends hooks modify the commit +message body and cannot be parallelized. + CONFIGURATION ------------- include::config/hook.txt[] diff --git a/builtin/hook.c b/builtin/hook.c index 0d9c052e84..c79a961e80 100644 --- a/builtin/hook.c +++ b/builtin/hook.c @@ -9,7 +9,7 @@ static const char * const builtin_hook_usage[] = { N_("git hook list "), N_("git hook run [(-e|--env)=...] [(-a|--arg)=...]" - "[--to-stdin=] "), + "[--to-stdin=] [(-j|--jobs) ] "), NULL }; @@ -114,10 +114,12 @@ static int run(int argc, const char **argv, const char *prefix) N_("argument to pass to hook")), OPT_STRING(0, "to-stdin", &opt.path_to_stdin, N_("path"), N_("file to read into hooks' stdin")), + OPT_INTEGER('j', "jobs", &opt.jobs, + N_("run up to hooks simultaneously")), OPT_END(), }; - run_hooks_opt_init(&opt); + run_hooks_opt_init_async(&opt); argc = parse_options(argc, argv, prefix, run_options, builtin_hook_usage, 0); diff --git a/hook.c b/hook.c index a2eda57fb9..784529fe2b 100644 --- a/hook.c +++ b/hook.c @@ -143,6 +143,14 @@ enum hookdir_opt configured_hookdir_opt(void) return HOOKDIR_UNKNOWN; } +int configured_hook_jobs(void) +{ + int n = online_cpus(); + git_config_get_int("hook.jobs", &n); + + return n; +} + static int should_include_hookdir(const char *path, enum hookdir_opt cfg) { struct strbuf prompt = STRBUF_INIT; @@ -241,12 +249,19 @@ struct list_head* hook_list(const char* hookname) return hook_head; } -void run_hooks_opt_init(struct run_hooks_opt *o) +void run_hooks_opt_init_sync(struct run_hooks_opt *o) { strvec_init(&o->env); strvec_init(&o->args); o->path_to_stdin = NULL; o->run_hookdir = configured_hookdir_opt(); + o->jobs = 1; +} + +void run_hooks_opt_init_async(struct run_hooks_opt *o) +{ + run_hooks_opt_init_sync(o); + o->jobs = configured_hook_jobs(); } int hook_exists(const char *hookname, enum hookdir_opt should_run_hookdir) @@ -274,22 +289,28 @@ void run_hooks_opt_clear(struct run_hooks_opt *o) strvec_clear(&o->args); } -static void prepare_hook_cp(const char *hookname, struct hook *hook, - struct run_hooks_opt *options, - struct child_process *cp) +static int pick_next_hook(struct child_process *cp, + struct strbuf *out, + void *pp_cb, + void **pp_task_cb) { + struct hook_cb_data *hook_cb = pp_cb; + struct hook *hook = hook_cb->run_me; + if (!hook) - return; + return 0; /* reopen the file for stdin; run_command closes it. */ - if (options->path_to_stdin) - cp->in = xopen(options->path_to_stdin, O_RDONLY); - else + if (hook_cb->options->path_to_stdin) { + cp->no_stdin = 0; + cp->in = xopen(hook_cb->options->path_to_stdin, O_RDONLY); + } else { cp->no_stdin = 1; + } - cp->env = options->env.v; + cp->env = hook_cb->options->env.v; cp->stdout_to_stderr = 1; - cp->trace2_hook_name = hookname; + cp->trace2_hook_name = hook_cb->hookname; /* * Commands from the config could be oneliners, but we know @@ -304,13 +325,58 @@ static void prepare_hook_cp(const char *hookname, struct hook *hook, * add passed-in argv, without expanding - let the user get back * exactly what they put in */ - strvec_pushv(&cp->args, options->args.v); + strvec_pushv(&cp->args, hook_cb->options->args.v); + + /* Provide context for errors if necessary */ + *pp_task_cb = hook; + + /* Get the next entry ready */ + if (hook_cb->run_me->list.next == hook_cb->head) + hook_cb->run_me = NULL; + else + hook_cb->run_me = list_entry(hook_cb->run_me->list.next, + struct hook, list); + + return 1; +} + +static int notify_start_failure(struct strbuf *out, + void *pp_cb, + void *pp_task_cp) +{ + struct hook_cb_data *hook_cb = pp_cb; + struct hook *attempted = pp_task_cp; + + /* |= rc in cb */ + hook_cb->rc |= 1; + + strbuf_addf(out, _("Couldn't start '%s', configured in '%s'\n"), + attempted->command.buf, + attempted->from_hookdir ? "hookdir" + : config_scope_name(attempted->origin)); + + /* NEEDSWORK: if halt_on_error is desired, do it here. */ + return 0; +} + +static int notify_hook_finished(int result, + struct strbuf *out, + void *pp_cb, + void *pp_task_cb) +{ + struct hook_cb_data *hook_cb = pp_cb; + + /* |= rc in cb */ + hook_cb->rc |= result; + + /* NEEDSWORK: if halt_on_error is desired, do it here. */ + return 0; } int run_hooks(const char *hookname, struct run_hooks_opt *options) { struct list_head *to_run, *pos = NULL, *tmp = NULL; - int rc = 0; + struct hook_cb_data cb_data = { 0, hookname, NULL, NULL, options }; if (!options) BUG("a struct run_hooks_opt must be provided to run_hooks"); @@ -318,17 +384,26 @@ int run_hooks(const char *hookname, struct run_hooks_opt *options) to_run = hook_list(hookname); list_for_each_safe(pos, tmp, to_run) { - struct child_process hook_proc = CHILD_PROCESS_INIT; struct hook *hook = list_entry(pos, struct hook, list); if (hook->from_hookdir && !should_include_hookdir(hook->command.buf, options->run_hookdir)) - continue; + list_del(pos); + } + + if (list_empty(to_run)) + return 0; - prepare_hook_cp(hookname, hook, options, &hook_proc); + cb_data.head = to_run; + cb_data.run_me = list_entry(to_run->next, struct hook, list); - rc |= run_command(&hook_proc); - } + run_processes_parallel_tr2(options->jobs, + pick_next_hook, + notify_start_failure, + notify_hook_finished, + &cb_data, + "hook", + hookname); - return rc; + return cb_data.rc; } diff --git a/hook.h b/hook.h index 37147f4c5e..cdb1ac5510 100644 --- a/hook.h +++ b/hook.h @@ -38,6 +38,9 @@ enum hookdir_opt */ enum hookdir_opt configured_hookdir_opt(void); +/* Provides the number of threads to use for parallel hook execution. */ +int configured_hook_jobs(void); + struct run_hooks_opt { /* Environment vars to be set for each hook */ @@ -48,16 +51,31 @@ struct run_hooks_opt /* * How should the hookdir be handled? - * Leave the RUN_HOOKS_OPT_INIT default in most cases; this only needs + * Leave the run_hooks_opt_init_*() default in most cases; this only needs * to be overridden if the user can override it at the command line. */ enum hookdir_opt run_hookdir; /* Path to file which should be piped to stdin for each hook */ const char *path_to_stdin; + + /* Number of threads to parallelize across */ + int jobs; +}; + +/* + * Callback provided to feed_pipe_fn and consume_sideband_fn. + */ +struct hook_cb_data { + int rc; + const char *hookname; + struct list_head *head; + struct hook *run_me; + struct run_hooks_opt *options; }; -void run_hooks_opt_init(struct run_hooks_opt *o); +void run_hooks_opt_init_sync(struct run_hooks_opt *o); +void run_hooks_opt_init_async(struct run_hooks_opt *o); void run_hooks_opt_clear(struct run_hooks_opt *o); /* From patchwork Thu May 27 00:08:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 12283013 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DE0F1C47088 for ; Thu, 27 May 2021 00:09:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C0062613CD for ; Thu, 27 May 2021 00:09:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235018AbhE0ALU (ORCPT ); Wed, 26 May 2021 20:11:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51618 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234832AbhE0AK7 (ORCPT ); Wed, 26 May 2021 20:10:59 -0400 Received: from mail-qt1-x84a.google.com (mail-qt1-x84a.google.com [IPv6:2607:f8b0:4864:20::84a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 256B2C0613CE for ; Wed, 26 May 2021 17:09:25 -0700 (PDT) Received: by mail-qt1-x84a.google.com with SMTP id g21-20020ac870d50000b0290230013e97e6so1654931qtp.21 for ; Wed, 26 May 2021 17:09:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=EiorRrSyDgR1pu3YWn64sSiUsHH2SGNVEqe3tVzGYLE=; b=UPOSvGbnj4XZFuEOOsREN/ruVB4ss7oImM6PfrXYKFS4Yjuloi8vT2VaLPg+5g4PCS OODV9oi7K+RXj7W65nCoi0dDy5sKJYa25+tW50a7c4ecPkBzQeF0/j82FQa7nxTpGMBm BPoTJc1LLK195EE8RV/IuzPHvoIaefhqd7zlVP+FelioCHkHn04B+u3GN60AUqVqf/TH Bvg0gjQlsNltTlSi/H9j1EI7uyI1p7DryGFgPunBMCJrPO9RC6bvtGhT6U+zd1esZ+pq tIDIy/+UwfWC73htNIFVdSp4pVkLnluwUDFDm9xLAOEbze0BJYsUXWkSWpws3F2WQv7Z nK4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=EiorRrSyDgR1pu3YWn64sSiUsHH2SGNVEqe3tVzGYLE=; b=ARTG1nVenYEAbnxW1Uq0VNNN/1ZvBswXa3SuB2ZuEMVxVUHXHdw/+z0P0iScbmr37y RCo1RGwc733/1U9VNi2g3WdTW8EZWpiISMLC9ai/pfI0cVQmXq5QiWuSbBLGrHsbzQ7o +y8+tarDtrxGK6E00VDntQqqRVrIsCQc80NI9WUKDuXO5W/f0EjyoaDdwHciebnxShtx IEVsODRCYR8fZj+MgaZgQyYs4ctQuhvN7Zvcl+ggBA85ItgCIAjKoMJOJemvDqwLvWS/ wQ4meT6q2IEfLW7uZ+6Q2+6ySBJzBJrRaRc8aaubcpjars/Ep6HdPtSeRbZPyRDUfuQ1 pONw== X-Gm-Message-State: AOAM530NhtaA4UJm0ChUC20nUw1mBAbA+2y1Nwdn/LLsFZBLwbsDhMlS xPSpHgr9XodXXzZubnOeUFrRHfudrwKXNXc+k1rdJ6XyAFoHNhF99au3NMFnaI3jye+LclrF/pm Ji8uDYVr6Z603CKz9KVZR8HZSQRnKttGWcvz4WHUoEwRWW0YCXmr/qj/G6PTHAkqj1U5ITDdQug == X-Google-Smtp-Source: ABdhPJy6q6csvpE3ZOqt+lDxsBk03OPm48nQuG9QPRK60znOwS47njw1F/b/s8vSAF0oACFapDjTMAFWDcvvfHw4cqM= X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:c1a4:4d87:8b5a:d12c]) (user=emilyshaffer job=sendgmr) by 2002:a05:6214:766:: with SMTP id f6mr537139qvz.17.1622074164244; Wed, 26 May 2021 17:09:24 -0700 (PDT) Date: Wed, 26 May 2021 17:08:31 -0700 In-Reply-To: <20210527000856.695702-1-emilyshaffer@google.com> Message-Id: <20210527000856.695702-13-emilyshaffer@google.com> Mime-Version: 1.0 References: <20210527000856.695702-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v9 12/37] hook: allow specifying working directory for hooks From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Hooks like "post-checkout" require that hooks have a different working directory than the initial process. Pipe that directly through to struct child_process. Because we can just run 'git -C hook run ...' it shouldn't be necessary to pipe this option through the frontend. In fact, this reduces the possibility of users running hooks which affect some part of the filesystem outside of the repo in question. Signed-off-by: Emily Shaffer --- Notes: Needed later for "post-checkout" conversion. hook.c | 2 ++ hook.h | 3 +++ 2 files changed, 5 insertions(+) diff --git a/hook.c b/hook.c index 784529fe2b..920563607d 100644 --- a/hook.c +++ b/hook.c @@ -256,6 +256,7 @@ void run_hooks_opt_init_sync(struct run_hooks_opt *o) o->path_to_stdin = NULL; o->run_hookdir = configured_hookdir_opt(); o->jobs = 1; + o->dir = NULL; } void run_hooks_opt_init_async(struct run_hooks_opt *o) @@ -311,6 +312,7 @@ static int pick_next_hook(struct child_process *cp, cp->env = hook_cb->options->env.v; cp->stdout_to_stderr = 1; cp->trace2_hook_name = hook_cb->hookname; + cp->dir = hook_cb->options->dir; /* * Commands from the config could be oneliners, but we know diff --git a/hook.h b/hook.h index cdb1ac5510..218b9e1721 100644 --- a/hook.h +++ b/hook.h @@ -61,6 +61,9 @@ struct run_hooks_opt /* Number of threads to parallelize across */ int jobs; + + /* Path to initial working directory for subprocess */ + const char *dir; }; /* From patchwork Thu May 27 00:08:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 12283015 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A90C0C47089 for ; Thu, 27 May 2021 00:09:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 831E0613CD for ; Thu, 27 May 2021 00:09:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234996AbhE0ALU (ORCPT ); Wed, 26 May 2021 20:11:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51626 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233972AbhE0ALA (ORCPT ); Wed, 26 May 2021 20:11:00 -0400 Received: from mail-qv1-xf49.google.com (mail-qv1-xf49.google.com [IPv6:2607:f8b0:4864:20::f49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EC3C5C06138F for ; Wed, 26 May 2021 17:09:26 -0700 (PDT) Received: by mail-qv1-xf49.google.com with SMTP id d3-20020a0cdb030000b029021313001eeeso2365010qvk.15 for ; Wed, 26 May 2021 17:09:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=X5Ci3i4EAIEdram3u8mM7KNJ47PQuBSwRytFYjIp628=; b=RTV/fgv+i49+192AQzUPSGTfXBWvvzWIsF3632X2SRVstgdMAQr01E8wKYwsn/IVJV uCupEe6w22/Kigc29a8J6G6/fITq61CbeRkvmkIxIKjXW0g9xbRttK/1itccvYPXdrAz 3lJsP3RQtwdlPM2ISdGqOK6POX98kaOszB3K42oCqX3sa0PPgTNZHsaVQROKjiakUbxt JVgQdR7FQwK6FqkSkdFuFP462CGBg2KNVCUtL7HuD5nHc7m9JGtLLZsJF/HV/T3o6beB drg4wDXXHO60yasWdAKwoYgR0pSMwsWUuu4QqL4sOLvDAvokpzdkOsWC0davAPGDaoOA I92g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=X5Ci3i4EAIEdram3u8mM7KNJ47PQuBSwRytFYjIp628=; b=JmL2YZ9/r/G35CZ4/n5pSoND2HvD+ozxEJCwNgq/ROLcOlDLXmypXhkopGfqBT8dhE V6g0vTkZsbNY7eyJwm4gY1tUiorb+SvKT8upFq0JFCn0VOmS3s4NM5yX1VNub3qGFkWU qu+PhYG0DUOzId9ciMfp9Ovzze6DzGH4p8HR4BZUuAanecLXbhic6sU0JdQW9STwXjFl eKLsGFVV7SSlsDfFsPJqH/o5YyNFpkG7RTETF3mpH4rX7HVtdyBbn1ldCSkfzbs5BGsx blq0LblU2GrsdzQnsDKP7yjA91ID6QJkZ+sislFMbRNcY7MHLbU09zkZV0SKtR2lYhkt tyVQ== X-Gm-Message-State: AOAM532YKWFO4Bnz5wcr1MElHalNdppbort3I+cHClKATG2zcRhmQySJ k/Teq3QqReSHD61AOZysF/Twnfgm9sCuO773jMiYbUT2HeGnGX8QMigr3c72yYA7DjxteghRe6z LiYrPlZ3cLgfHinbts+T4whcNYdsrthe6UDWuNvUQhs0V6U4NoXZzCV3EhvvPFQ0Sj32yIQIGhg == X-Google-Smtp-Source: ABdhPJy0uKrXAl17MdO2FcX/QOKiE+QR9CoHL1xbuvFeyc47grfSIwbs8ZpKMGdAMaCLhgEpiiZmiR/EibYqxq5bpdo= X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:c1a4:4d87:8b5a:d12c]) (user=emilyshaffer job=sendgmr) by 2002:a0c:e1ca:: with SMTP id v10mr512836qvl.47.1622074166092; Wed, 26 May 2021 17:09:26 -0700 (PDT) Date: Wed, 26 May 2021 17:08:32 -0700 In-Reply-To: <20210527000856.695702-1-emilyshaffer@google.com> Message-Id: <20210527000856.695702-14-emilyshaffer@google.com> Mime-Version: 1.0 References: <20210527000856.695702-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v9 13/37] run-command: add stdin callback for parallelization From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org If a user of the run_processes_parallel() API wants to pipe a large amount of information to stdin of each parallel command, that information could exceed the buffer of the pipe allocated for that process's stdin. Generally this is solved by repeatedly writing to child_process.in between calls to start_command() and finish_command(); run_processes_parallel() did not provide users an opportunity to access child_process at that time. Because the data might be extremely large (for example, a list of all refs received during a push from a client) simply taking a string_list or strbuf is not as scalable as using a callback; the rest of the run_processes_parallel() API also uses callbacks, so making this feature match the rest of the API reduces mental load on the user. Signed-off-by: Emily Shaffer --- builtin/fetch.c | 1 + builtin/submodule--helper.c | 2 +- hook.c | 1 + run-command.c | 54 +++++++++++++++++++++++++++++++++++-- run-command.h | 17 +++++++++++- submodule.c | 1 + t/helper/test-run-command.c | 31 ++++++++++++++++++--- t/t0061-run-command.sh | 30 +++++++++++++++++++++ 8 files changed, 129 insertions(+), 8 deletions(-) diff --git a/builtin/fetch.c b/builtin/fetch.c index dfde96a435..a07816b650 100644 --- a/builtin/fetch.c +++ b/builtin/fetch.c @@ -1817,6 +1817,7 @@ static int fetch_multiple(struct string_list *list, int max_children) result = run_processes_parallel_tr2(max_children, &fetch_next_remote, &fetch_failed_to_start, + NULL, &fetch_finished, &state, "fetch", "parallel/fetch"); diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c index d55f6262e9..8d1e731073 100644 --- a/builtin/submodule--helper.c +++ b/builtin/submodule--helper.c @@ -2295,7 +2295,7 @@ static int update_submodules(struct submodule_update_clone *suc) int i; run_processes_parallel_tr2(suc->max_jobs, update_clone_get_next_task, - update_clone_start_failure, + update_clone_start_failure, NULL, update_clone_task_finished, suc, "submodule", "parallel/update"); diff --git a/hook.c b/hook.c index 920563607d..1fa7976583 100644 --- a/hook.c +++ b/hook.c @@ -402,6 +402,7 @@ int run_hooks(const char *hookname, struct run_hooks_opt *options) run_processes_parallel_tr2(options->jobs, pick_next_hook, notify_start_failure, + NULL, notify_hook_finished, &cb_data, "hook", diff --git a/run-command.c b/run-command.c index 5237984ebe..6dd33caa57 100644 --- a/run-command.c +++ b/run-command.c @@ -1558,6 +1558,7 @@ struct parallel_processes { get_next_task_fn get_next_task; start_failure_fn start_failure; + feed_pipe_fn feed_pipe; task_finished_fn task_finished; struct { @@ -1585,6 +1586,13 @@ static int default_start_failure(struct strbuf *out, return 0; } +static int default_feed_pipe(struct strbuf *pipe, + void *pp_cb, + void *pp_task_cb) +{ + return 1; +} + static int default_task_finished(int result, struct strbuf *out, void *pp_cb, @@ -1615,6 +1623,7 @@ static void pp_init(struct parallel_processes *pp, int n, get_next_task_fn get_next_task, start_failure_fn start_failure, + feed_pipe_fn feed_pipe, task_finished_fn task_finished, void *data) { @@ -1633,6 +1642,7 @@ static void pp_init(struct parallel_processes *pp, pp->get_next_task = get_next_task; pp->start_failure = start_failure ? start_failure : default_start_failure; + pp->feed_pipe = feed_pipe ? feed_pipe : default_feed_pipe; pp->task_finished = task_finished ? task_finished : default_task_finished; pp->nr_processes = 0; @@ -1730,6 +1740,37 @@ static int pp_start_one(struct parallel_processes *pp) return 0; } +static void pp_buffer_stdin(struct parallel_processes *pp) +{ + int i; + struct strbuf sb = STRBUF_INIT; + + /* Buffer stdin for each pipe. */ + for (i = 0; i < pp->max_processes; i++) { + if (pp->children[i].state == GIT_CP_WORKING && + pp->children[i].process.in > 0) { + int done; + strbuf_reset(&sb); + done = pp->feed_pipe(&sb, pp->data, + pp->children[i].data); + if (sb.len) { + if (write_in_full(pp->children[i].process.in, + sb.buf, sb.len) < 0) { + if (errno != EPIPE) + die_errno("write"); + done = 1; + } + } + if (done) { + close(pp->children[i].process.in); + pp->children[i].process.in = 0; + } + } + } + + strbuf_release(&sb); +} + static void pp_buffer_stderr(struct parallel_processes *pp, int output_timeout) { int i; @@ -1794,6 +1835,7 @@ static int pp_collect_finished(struct parallel_processes *pp) pp->nr_processes--; pp->children[i].state = GIT_CP_FREE; pp->pfd[i].fd = -1; + pp->children[i].process.in = 0; child_process_init(&pp->children[i].process); if (i != pp->output_owner) { @@ -1827,6 +1869,7 @@ static int pp_collect_finished(struct parallel_processes *pp) int run_processes_parallel(int n, get_next_task_fn get_next_task, start_failure_fn start_failure, + feed_pipe_fn feed_pipe, task_finished_fn task_finished, void *pp_cb) { @@ -1835,7 +1878,9 @@ int run_processes_parallel(int n, int spawn_cap = 4; struct parallel_processes pp; - pp_init(&pp, n, get_next_task, start_failure, task_finished, pp_cb); + sigchain_push(SIGPIPE, SIG_IGN); + + pp_init(&pp, n, get_next_task, start_failure, feed_pipe, task_finished, pp_cb); while (1) { for (i = 0; i < spawn_cap && !pp.shutdown && @@ -1852,6 +1897,7 @@ int run_processes_parallel(int n, } if (!pp.nr_processes) break; + pp_buffer_stdin(&pp); pp_buffer_stderr(&pp, output_timeout); pp_output(&pp); code = pp_collect_finished(&pp); @@ -1863,11 +1909,15 @@ int run_processes_parallel(int n, } pp_cleanup(&pp); + + sigchain_pop(SIGPIPE); + return 0; } int run_processes_parallel_tr2(int n, get_next_task_fn get_next_task, start_failure_fn start_failure, + feed_pipe_fn feed_pipe, task_finished_fn task_finished, void *pp_cb, const char *tr2_category, const char *tr2_label) { @@ -1877,7 +1927,7 @@ int run_processes_parallel_tr2(int n, get_next_task_fn get_next_task, ((n < 1) ? online_cpus() : n)); result = run_processes_parallel(n, get_next_task, start_failure, - task_finished, pp_cb); + feed_pipe, task_finished, pp_cb); trace2_region_leave(tr2_category, tr2_label, NULL); diff --git a/run-command.h b/run-command.h index d08414a92e..1e3cf0999f 100644 --- a/run-command.h +++ b/run-command.h @@ -443,6 +443,20 @@ typedef int (*start_failure_fn)(struct strbuf *out, void *pp_cb, void *pp_task_cb); +/** + * This callback is called repeatedly on every child process who requests + * start_command() to create a pipe by setting child_process.in < 0. + * + * pp_cb is the callback cookie as passed into run_processes_parallel, and + * pp_task_cb is the callback cookie as passed into get_next_task_fn. + * The contents of 'send' will be read into the pipe and passed to the pipe. + * + * Return nonzero to close the pipe. + */ +typedef int (*feed_pipe_fn)(struct strbuf *pipe, + void *pp_cb, + void *pp_task_cb); + /** * This callback is called on every child process that finished processing. * @@ -477,10 +491,11 @@ typedef int (*task_finished_fn)(int result, int run_processes_parallel(int n, get_next_task_fn, start_failure_fn, + feed_pipe_fn, task_finished_fn, void *pp_cb); int run_processes_parallel_tr2(int n, get_next_task_fn, start_failure_fn, - task_finished_fn, void *pp_cb, + feed_pipe_fn, task_finished_fn, void *pp_cb, const char *tr2_category, const char *tr2_label); #endif diff --git a/submodule.c b/submodule.c index 0b1d9c1dde..ea026a8195 100644 --- a/submodule.c +++ b/submodule.c @@ -1645,6 +1645,7 @@ int fetch_populated_submodules(struct repository *r, run_processes_parallel_tr2(max_parallel_jobs, get_next_submodule, fetch_start_failure, + NULL, fetch_finish, &spf, "submodule", "parallel/fetch"); diff --git a/t/helper/test-run-command.c b/t/helper/test-run-command.c index 7ae03dc712..9348184d30 100644 --- a/t/helper/test-run-command.c +++ b/t/helper/test-run-command.c @@ -32,8 +32,13 @@ static int parallel_next(struct child_process *cp, return 0; strvec_pushv(&cp->args, d->argv); + cp->in = d->in; + cp->no_stdin = d->no_stdin; strbuf_addstr(err, "preloaded output of a child\n"); number_callbacks++; + + *task_cb = xmalloc(sizeof(int)); + *(int*)(*task_cb) = 2; return 1; } @@ -55,6 +60,17 @@ static int task_finished(int result, return 1; } +static int test_stdin(struct strbuf *pipe, void *cb, void *task_cb) +{ + int *lines_remaining = task_cb; + + if (*lines_remaining) + strbuf_addf(pipe, "sample stdin %d\n", --(*lines_remaining)); + + return !(*lines_remaining); +} + + struct testsuite { struct string_list tests, failed; int next; @@ -185,7 +201,7 @@ static int testsuite(int argc, const char **argv) suite.tests.nr, max_jobs); ret = run_processes_parallel(max_jobs, next_test, test_failed, - test_finished, &suite); + test_stdin, test_finished, &suite); if (suite.failed.nr > 0) { ret = 1; @@ -413,15 +429,22 @@ int cmd__run_command(int argc, const char **argv) if (!strcmp(argv[1], "run-command-parallel")) exit(run_processes_parallel(jobs, parallel_next, - NULL, NULL, &proc)); + NULL, NULL, NULL, &proc)); if (!strcmp(argv[1], "run-command-abort")) exit(run_processes_parallel(jobs, parallel_next, - NULL, task_finished, &proc)); + NULL, NULL, task_finished, &proc)); if (!strcmp(argv[1], "run-command-no-jobs")) exit(run_processes_parallel(jobs, no_job, - NULL, task_finished, &proc)); + NULL, NULL, task_finished, &proc)); + + if (!strcmp(argv[1], "run-command-stdin")) { + proc.in = -1; + proc.no_stdin = 0; + exit (run_processes_parallel(jobs, parallel_next, NULL, + test_stdin, NULL, &proc)); + } fprintf(stderr, "check usage\n"); return 1; diff --git a/t/t0061-run-command.sh b/t/t0061-run-command.sh index 7d599675e3..87759482ad 100755 --- a/t/t0061-run-command.sh +++ b/t/t0061-run-command.sh @@ -143,6 +143,36 @@ test_expect_success 'run_command runs in parallel with more tasks than jobs avai test_cmp expect actual ' +cat >expect <<-EOF +preloaded output of a child +listening for stdin: +sample stdin 1 +sample stdin 0 +preloaded output of a child +listening for stdin: +sample stdin 1 +sample stdin 0 +preloaded output of a child +listening for stdin: +sample stdin 1 +sample stdin 0 +preloaded output of a child +listening for stdin: +sample stdin 1 +sample stdin 0 +EOF + +test_expect_success 'run_command listens to stdin' ' + write_script stdin-script <<-\EOF && + echo "listening for stdin:" + while read line; do + echo "$line" + done + EOF + test-tool run-command run-command-stdin 2 ./stdin-script 2>actual && + test_cmp expect actual +' + cat >expect <<-EOF preloaded output of a child asking for a quick stop From patchwork Thu May 27 00:08:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 12283017 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6F9B4C47088 for ; Thu, 27 May 2021 00:09:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 53659613BE for ; Thu, 27 May 2021 00:09:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235102AbhE0AL0 (ORCPT ); Wed, 26 May 2021 20:11:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51578 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233936AbhE0ALB (ORCPT ); Wed, 26 May 2021 20:11:01 -0400 Received: from mail-qk1-x74a.google.com (mail-qk1-x74a.google.com [IPv6:2607:f8b0:4864:20::74a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CCCC8C06138C for ; Wed, 26 May 2021 17:09:28 -0700 (PDT) Received: by mail-qk1-x74a.google.com with SMTP id u9-20020a05620a4549b02902e956c2a3c8so2066970qkp.20 for ; Wed, 26 May 2021 17:09:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=osfYIeU0d2TJ9wVVo18P/YFkWV2BzEx8rkzomjPT40g=; b=rM/0XABWDppTCMRkqz65ehllrcaJa4VHxjcOCqpDaP2ixW0TONIoob1BTXs9kBTBA+ Ssak6DXRN6ePXi/037WBvozlNKhKlKszOv/fhDHsgrfvwoCZI0BPKvol1BMWZJnD8cGr NhJ3pkLrR4kFuWkTMy3j1L3HNYCwFpmE5nwmZ4hIbx+4w/RkrVUU7TI77U7w8qV2pFry ecqHw9t1nkC/1cbYjSK8TNSwL/VGkK2htGKk+GV0upetQdqc7PcIfENGoT7E0s+iNMxj S9H5FzFuZH36z2CZYCsBDt3vPzqJkc0gelvtlci0utisdWRsAWL9a+6ulZIZdMbHi1JL Hrmg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=osfYIeU0d2TJ9wVVo18P/YFkWV2BzEx8rkzomjPT40g=; b=bRRnladeMCaw+qHhh2NHLZ5X0eTVbMGSDPLVZKq++1/PU8C1JEgzeoch3NXKn5gMFg BNtRUTHI6CPH+Fk1dYgZSom8b275zKMYaMgcgJlKoPl4uM/k5jZoN8QZShm9J7qxrnFf /E0x8NXBnppWKEYih59wwVdfXa8xlnjPiH6i8gmd3FD5O4FeOtre8dBw9GxwB4qhtXf3 7piAvCfahJB025a1qIQ/Wx/0Q/C41X4m9qoA+lRQhAuKhOhrhoVDK0JnBhT/1px8Sw6p Zwk6zXmqpCu328otPj2fs29TIXyBrKqXpf3exMVlCSIk+A16WfLJcQIx4l7zU/5JZhsy xqnw== X-Gm-Message-State: AOAM5337ob8KMNXORx27smVlvh/VdLyd/leZjTyky27g+qSjhPZaYFgF 2r1khUR5X40pswZFqCl68Js0AMAGeGvZYS4BINDM1mipVxn97mv10a3NZmpMYMrVTiYzc1+i7HI AMqV8hae872IDMdaQ0Mk/Zdk6nhYVPzTSgpg8/Yzb7DD7GyD8i5ZMs8MGrsBpp0LG9dupqB5d/w == X-Google-Smtp-Source: ABdhPJx6GosKm5Qyn7oju9FB53x6aZNhVtathjZLzVqENEWAto7wDTHZ4K/IoFl58L1jKmMgeRSq+dB8YuezYTRDREc= X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:c1a4:4d87:8b5a:d12c]) (user=emilyshaffer job=sendgmr) by 2002:a0c:ee23:: with SMTP id l3mr1004986qvs.55.1622074167964; Wed, 26 May 2021 17:09:27 -0700 (PDT) Date: Wed, 26 May 2021 17:08:33 -0700 In-Reply-To: <20210527000856.695702-1-emilyshaffer@google.com> Message-Id: <20210527000856.695702-15-emilyshaffer@google.com> Mime-Version: 1.0 References: <20210527000856.695702-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v9 14/37] hook: provide stdin by string_list or callback From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In cases where a hook requires only a small amount of information via stdin, it should be simple for users to provide a string_list alone. But in more complicated cases where the stdin is too large to hold in memory, let's provide a callback the users can populate line after line with instead. Signed-off-by: Emily Shaffer --- hook.c | 38 ++++++++++++++++++++++++++++++++++++-- hook.h | 28 ++++++++++++++++++++++++++++ 2 files changed, 64 insertions(+), 2 deletions(-) diff --git a/hook.c b/hook.c index 1fa7976583..8aa0d4fdf2 100644 --- a/hook.c +++ b/hook.c @@ -7,8 +7,10 @@ void free_hook(struct hook *ptr) { - if (ptr) + if (ptr) { strbuf_release(&ptr->command); + free(ptr->feed_pipe_cb_data); + } free(ptr); } @@ -38,6 +40,7 @@ static void append_or_move_hook(struct list_head *head, const char *command) strbuf_init(&to_add->command, 0); strbuf_addstr(&to_add->command, command); to_add->from_hookdir = 0; + to_add->feed_pipe_cb_data = NULL; } /* re-set the scope so we show where an override was specified */ @@ -257,6 +260,8 @@ void run_hooks_opt_init_sync(struct run_hooks_opt *o) o->run_hookdir = configured_hookdir_opt(); o->jobs = 1; o->dir = NULL; + o->feed_pipe = NULL; + o->feed_pipe_ctx = NULL; } void run_hooks_opt_init_async(struct run_hooks_opt *o) @@ -290,6 +295,28 @@ void run_hooks_opt_clear(struct run_hooks_opt *o) strvec_clear(&o->args); } +int pipe_from_string_list(struct strbuf *pipe, void *pp_cb, void *pp_task_cb) +{ + int *item_idx; + struct hook *ctx = pp_task_cb; + struct string_list *to_pipe = ((struct hook_cb_data*)pp_cb)->options->feed_pipe_ctx; + + /* Bootstrap the state manager if necessary. */ + if (!ctx->feed_pipe_cb_data) { + ctx->feed_pipe_cb_data = xmalloc(sizeof(unsigned int)); + *(int*)ctx->feed_pipe_cb_data = 0; + } + + item_idx = ctx->feed_pipe_cb_data; + + if (*item_idx < to_pipe->nr) { + strbuf_addf(pipe, "%s\n", to_pipe->items[*item_idx].string); + (*item_idx)++; + return 0; + } + return 1; +} + static int pick_next_hook(struct child_process *cp, struct strbuf *out, void *pp_cb, @@ -305,6 +332,10 @@ static int pick_next_hook(struct child_process *cp, if (hook_cb->options->path_to_stdin) { cp->no_stdin = 0; cp->in = xopen(hook_cb->options->path_to_stdin, O_RDONLY); + } else if (hook_cb->options->feed_pipe) { + /* ask for start_command() to make a pipe for us */ + cp->in = -1; + cp->no_stdin = 0; } else { cp->no_stdin = 1; } @@ -383,6 +414,9 @@ int run_hooks(const char *hookname, struct run_hooks_opt *options) if (!options) BUG("a struct run_hooks_opt must be provided to run_hooks"); + if (options->path_to_stdin && options->feed_pipe) + BUG("choose only one method to populate stdin"); + to_run = hook_list(hookname); list_for_each_safe(pos, tmp, to_run) { @@ -402,7 +436,7 @@ int run_hooks(const char *hookname, struct run_hooks_opt *options) run_processes_parallel_tr2(options->jobs, pick_next_hook, notify_start_failure, - NULL, + options->feed_pipe, notify_hook_finished, &cb_data, "hook", diff --git a/hook.h b/hook.h index 218b9e1721..aba62d4112 100644 --- a/hook.h +++ b/hook.h @@ -2,6 +2,7 @@ #include "list.h" #include "strbuf.h" #include "strvec.h" +#include "run-command.h" struct hook { struct list_head list; @@ -13,6 +14,12 @@ struct hook { /* The literal command to run. */ struct strbuf command; unsigned from_hookdir : 1; + + /* + * Use this to keep state for your feed_pipe_fn if you are using + * run_hooks_opt.feed_pipe. Otherwise, do not touch it. + */ + void *feed_pipe_cb_data; }; /* @@ -58,14 +65,35 @@ struct run_hooks_opt /* Path to file which should be piped to stdin for each hook */ const char *path_to_stdin; + /* + * Callback and state pointer to ask for more content to pipe to stdin. + * Will be called repeatedly, for each hook. See + * hook.c:pipe_from_stdin() for an example. Keep per-hook state in + * hook.feed_pipe_cb_data (per process). Keep initialization context in + * feed_pipe_ctx (shared by all processes). + * + * See 'pipe_from_string_list()' for info about how to specify a + * string_list as the stdin input instead of writing your own handler. + */ + feed_pipe_fn feed_pipe; + void *feed_pipe_ctx; /* Number of threads to parallelize across */ int jobs; /* Path to initial working directory for subprocess */ const char *dir; + }; +/* + * To specify a 'struct string_list', set 'run_hooks_opt.feed_pipe_ctx' to the + * string_list and set 'run_hooks_opt.feed_pipe' to 'pipe_from_string_list()'. + * This will pipe each string in the list to stdin, separated by newlines. (Do + * not inject your own newlines.) + */ +int pipe_from_string_list(struct strbuf *pipe, void *pp_cb, void *pp_task_cb); + /* * Callback provided to feed_pipe_fn and consume_sideband_fn. */ From patchwork Thu May 27 00:08:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 12283019 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DE29FC47082 for ; Thu, 27 May 2021 00:09:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C2CE7613BE for ; Thu, 27 May 2021 00:09:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235054AbhE0AL3 (ORCPT ); Wed, 26 May 2021 20:11:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51646 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235044AbhE0ALE (ORCPT ); Wed, 26 May 2021 20:11:04 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3F0C2C061343 for ; Wed, 26 May 2021 17:09:30 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id n129-20020a2527870000b02904ed02e1aab5so3609276ybn.21 for ; Wed, 26 May 2021 17:09:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=bFgS3cmr97QyGr3WOxnqGB+624xrzbEAm+TUPlN3iVs=; b=rtWNaqzwixQr2zvUQRE0xihkpCDMxWMVrKHcfHMhYnl3V9A2OVJLwXxPIHNk3u8x5s 3toeBQSfOSrkxG4k0ojSj/AgJkF+ksjdDA6WiJvHDe75q7+p/S2MEIebGtOlWGNYA67L o+Xd3c4AsK0wao+OEJ9cvmwVe9C6M8cDLOPXJqsKrB3U5eAjZuRs++AgYeP2zgqhSQfe D4QpWA/t13AJsh/4u2XAbp3SeYOVjX57+Jz44FMWtWnLB2EsRJ4IGGdkUwGgt7LUk9nQ /2qWBXV46BoLexRJu6ysw3at20ibihSpU1jZhLdzQ8RoSfzizJ6FN3B8BvvG/EBrfyeM zOzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=bFgS3cmr97QyGr3WOxnqGB+624xrzbEAm+TUPlN3iVs=; b=oar0ONCk4DGZNVaT8GtoCX3tbeeWm+WYJR8l6Jzr9eRovfYuDnq9+7Z4nBhbd2Cn6k 9InHEIpfA/wCb1XafKIGv1q+2DIHETA9jREkxeGDldMmyQDbCpkCSkz0669UL2atmrXV XRsZtFX4OZh6XNMcGqKGdCYvZSu6cFuLV+vnErwO2irl5CfAmf4XNGSIy7/ES71Gtjx8 akxoYQepTKFMIXKygoSMrPnmxMoI4lmzEgwBqW6z46IZo/Y614B62KiFh6cXATjYaXMo ZJQzVLiFpJ+E8BVxot2YoDNLkdtSbWvXDEYKKzyI5d3g+F51eTumB55EG4D8MFsSCe8d AwAQ== X-Gm-Message-State: AOAM531O7aCkmq9lv0wm7PH+LMLLxqZa79Wu71P607g4iY3LhmpQfhdv IJhFL+BOhh2XuCppu0i4NtnUdjMXUJJix/U5KM2O6+P/UA6gqJo0xkti4NDqX3TtPh0uQ2rSx3+ uPtUjYseZ5E4JJciOmxT7/hYNsmWOp7DlYTu6waeuHTWV8ghCNlOWKvAtCZOE8nnPQScmwIBByg == X-Google-Smtp-Source: ABdhPJyVpNfxv9f1H+PlVRoGSxojZeuUV8JAX7go2FdCTEBfr2cqPFK/vOpfaTLEVqFFYoYIrKZUtmOekflTo6FIAuA= X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:c1a4:4d87:8b5a:d12c]) (user=emilyshaffer job=sendgmr) by 2002:a25:c012:: with SMTP id c18mr850392ybf.41.1622074170073; Wed, 26 May 2021 17:09:30 -0700 (PDT) Date: Wed, 26 May 2021 17:08:34 -0700 In-Reply-To: <20210527000856.695702-1-emilyshaffer@google.com> Message-Id: <20210527000856.695702-16-emilyshaffer@google.com> Mime-Version: 1.0 References: <20210527000856.695702-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v9 15/37] run-command: allow capturing of collated output From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Some callers, for example server-side hooks which wish to relay hook output to clients across a transport, want to capture what would normally print to stderr and do something else with it. Allow that via a callback. By calling the callback regardless of whether there's output available, we allow clients to send e.g. a keepalive if necessary. Because we expose a strbuf, not a fd or FILE*, there's no need to create a temporary pipe or similar - we can just skip the print to stderr and instead hand it to the caller. Signed-off-by: Emily Shaffer --- Notes: Originally when writing this patch I attempted to use a pipe in memory - but managing its lifetime was actually pretty tricky, and I found I could achieve the same thing with less code by doing it this way. Critique welcome, including "no, you really need to do it with a pipe". builtin/fetch.c | 2 +- builtin/submodule--helper.c | 2 +- hook.c | 1 + run-command.c | 33 +++++++++++++++++++++++++-------- run-command.h | 18 +++++++++++++++++- submodule.c | 2 +- t/helper/test-run-command.c | 25 ++++++++++++++++++++----- t/t0061-run-command.sh | 7 +++++++ 8 files changed, 73 insertions(+), 17 deletions(-) diff --git a/builtin/fetch.c b/builtin/fetch.c index a07816b650..769af53ca4 100644 --- a/builtin/fetch.c +++ b/builtin/fetch.c @@ -1817,7 +1817,7 @@ static int fetch_multiple(struct string_list *list, int max_children) result = run_processes_parallel_tr2(max_children, &fetch_next_remote, &fetch_failed_to_start, - NULL, + NULL, NULL, &fetch_finished, &state, "fetch", "parallel/fetch"); diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c index 8d1e731073..fef8392e1d 100644 --- a/builtin/submodule--helper.c +++ b/builtin/submodule--helper.c @@ -2295,7 +2295,7 @@ static int update_submodules(struct submodule_update_clone *suc) int i; run_processes_parallel_tr2(suc->max_jobs, update_clone_get_next_task, - update_clone_start_failure, NULL, + update_clone_start_failure, NULL, NULL, update_clone_task_finished, suc, "submodule", "parallel/update"); diff --git a/hook.c b/hook.c index 8aa0d4fdf2..d48071bb1d 100644 --- a/hook.c +++ b/hook.c @@ -437,6 +437,7 @@ int run_hooks(const char *hookname, struct run_hooks_opt *options) pick_next_hook, notify_start_failure, options->feed_pipe, + NULL, notify_hook_finished, &cb_data, "hook", diff --git a/run-command.c b/run-command.c index 6dd33caa57..27135defb8 100644 --- a/run-command.c +++ b/run-command.c @@ -1559,6 +1559,7 @@ struct parallel_processes { get_next_task_fn get_next_task; start_failure_fn start_failure; feed_pipe_fn feed_pipe; + consume_sideband_fn consume_sideband; task_finished_fn task_finished; struct { @@ -1624,6 +1625,7 @@ static void pp_init(struct parallel_processes *pp, get_next_task_fn get_next_task, start_failure_fn start_failure, feed_pipe_fn feed_pipe, + consume_sideband_fn consume_sideband, task_finished_fn task_finished, void *data) { @@ -1644,6 +1646,7 @@ static void pp_init(struct parallel_processes *pp, pp->start_failure = start_failure ? start_failure : default_start_failure; pp->feed_pipe = feed_pipe ? feed_pipe : default_feed_pipe; pp->task_finished = task_finished ? task_finished : default_task_finished; + pp->consume_sideband = consume_sideband; pp->nr_processes = 0; pp->output_owner = 0; @@ -1680,7 +1683,10 @@ static void pp_cleanup(struct parallel_processes *pp) * When get_next_task added messages to the buffer in its last * iteration, the buffered output is non empty. */ - strbuf_write(&pp->buffered_output, stderr); + if (pp->consume_sideband) + pp->consume_sideband(&pp->buffered_output, pp->data); + else + strbuf_write(&pp->buffered_output, stderr); strbuf_release(&pp->buffered_output); sigchain_pop_common(); @@ -1801,9 +1807,13 @@ static void pp_buffer_stderr(struct parallel_processes *pp, int output_timeout) static void pp_output(struct parallel_processes *pp) { int i = pp->output_owner; + if (pp->children[i].state == GIT_CP_WORKING && pp->children[i].err.len) { - strbuf_write(&pp->children[i].err, stderr); + if (pp->consume_sideband) + pp->consume_sideband(&pp->children[i].err, pp->data); + else + strbuf_write(&pp->children[i].err, stderr); strbuf_reset(&pp->children[i].err); } } @@ -1842,11 +1852,15 @@ static int pp_collect_finished(struct parallel_processes *pp) strbuf_addbuf(&pp->buffered_output, &pp->children[i].err); strbuf_reset(&pp->children[i].err); } else { - strbuf_write(&pp->children[i].err, stderr); + /* Output errors, then all other finished child processes */ + if (pp->consume_sideband) { + pp->consume_sideband(&pp->children[i].err, pp->data); + pp->consume_sideband(&pp->buffered_output, pp->data); + } else { + strbuf_write(&pp->children[i].err, stderr); + strbuf_write(&pp->buffered_output, stderr); + } strbuf_reset(&pp->children[i].err); - - /* Output all other finished child processes */ - strbuf_write(&pp->buffered_output, stderr); strbuf_reset(&pp->buffered_output); /* @@ -1870,6 +1884,7 @@ int run_processes_parallel(int n, get_next_task_fn get_next_task, start_failure_fn start_failure, feed_pipe_fn feed_pipe, + consume_sideband_fn consume_sideband, task_finished_fn task_finished, void *pp_cb) { @@ -1880,7 +1895,7 @@ int run_processes_parallel(int n, sigchain_push(SIGPIPE, SIG_IGN); - pp_init(&pp, n, get_next_task, start_failure, feed_pipe, task_finished, pp_cb); + pp_init(&pp, n, get_next_task, start_failure, feed_pipe, consume_sideband, task_finished, pp_cb); while (1) { for (i = 0; i < spawn_cap && !pp.shutdown && @@ -1918,6 +1933,7 @@ int run_processes_parallel(int n, int run_processes_parallel_tr2(int n, get_next_task_fn get_next_task, start_failure_fn start_failure, feed_pipe_fn feed_pipe, + consume_sideband_fn consume_sideband, task_finished_fn task_finished, void *pp_cb, const char *tr2_category, const char *tr2_label) { @@ -1927,7 +1943,8 @@ int run_processes_parallel_tr2(int n, get_next_task_fn get_next_task, ((n < 1) ? online_cpus() : n)); result = run_processes_parallel(n, get_next_task, start_failure, - feed_pipe, task_finished, pp_cb); + feed_pipe, consume_sideband, + task_finished, pp_cb); trace2_region_leave(tr2_category, tr2_label, NULL); diff --git a/run-command.h b/run-command.h index 1e3cf0999f..ebc4a95a94 100644 --- a/run-command.h +++ b/run-command.h @@ -457,6 +457,20 @@ typedef int (*feed_pipe_fn)(struct strbuf *pipe, void *pp_cb, void *pp_task_cb); +/** + * If this callback is provided, instead of collating process output to stderr, + * they will be collated into a new pipe. consume_sideband_fn will be called + * repeatedly. When output is available on that pipe, it will be contained in + * 'output'. But it will be called with an empty 'output' too, to allow for + * keepalives or similar operations if necessary. + * + * pp_cb is the callback cookie as passed into run_processes_parallel. + * + * Since this callback is provided with the collated output, no task cookie is + * provided. + */ +typedef void (*consume_sideband_fn)(struct strbuf *output, void *pp_cb); + /** * This callback is called on every child process that finished processing. * @@ -492,10 +506,12 @@ int run_processes_parallel(int n, get_next_task_fn, start_failure_fn, feed_pipe_fn, + consume_sideband_fn, task_finished_fn, void *pp_cb); int run_processes_parallel_tr2(int n, get_next_task_fn, start_failure_fn, - feed_pipe_fn, task_finished_fn, void *pp_cb, + feed_pipe_fn, consume_sideband_fn, + task_finished_fn, void *pp_cb, const char *tr2_category, const char *tr2_label); #endif diff --git a/submodule.c b/submodule.c index ea026a8195..7fe0c8f7c9 100644 --- a/submodule.c +++ b/submodule.c @@ -1645,7 +1645,7 @@ int fetch_populated_submodules(struct repository *r, run_processes_parallel_tr2(max_parallel_jobs, get_next_submodule, fetch_start_failure, - NULL, + NULL, NULL, fetch_finish, &spf, "submodule", "parallel/fetch"); diff --git a/t/helper/test-run-command.c b/t/helper/test-run-command.c index 9348184d30..d53db6d11c 100644 --- a/t/helper/test-run-command.c +++ b/t/helper/test-run-command.c @@ -51,6 +51,16 @@ static int no_job(struct child_process *cp, return 0; } +static void test_consume_sideband(struct strbuf *output, void *cb) +{ + FILE *sideband; + + sideband = fopen("./sideband", "a"); + + strbuf_write(output, sideband); + fclose(sideband); +} + static int task_finished(int result, struct strbuf *err, void *pp_cb, @@ -201,7 +211,7 @@ static int testsuite(int argc, const char **argv) suite.tests.nr, max_jobs); ret = run_processes_parallel(max_jobs, next_test, test_failed, - test_stdin, test_finished, &suite); + test_stdin, NULL, test_finished, &suite); if (suite.failed.nr > 0) { ret = 1; @@ -429,23 +439,28 @@ int cmd__run_command(int argc, const char **argv) if (!strcmp(argv[1], "run-command-parallel")) exit(run_processes_parallel(jobs, parallel_next, - NULL, NULL, NULL, &proc)); + NULL, NULL, NULL, NULL, &proc)); if (!strcmp(argv[1], "run-command-abort")) exit(run_processes_parallel(jobs, parallel_next, - NULL, NULL, task_finished, &proc)); + NULL, NULL, NULL, task_finished, &proc)); if (!strcmp(argv[1], "run-command-no-jobs")) exit(run_processes_parallel(jobs, no_job, - NULL, NULL, task_finished, &proc)); + NULL, NULL, NULL, task_finished, &proc)); if (!strcmp(argv[1], "run-command-stdin")) { proc.in = -1; proc.no_stdin = 0; exit (run_processes_parallel(jobs, parallel_next, NULL, - test_stdin, NULL, &proc)); + test_stdin, NULL, NULL, &proc)); } + if (!strcmp(argv[1], "run-command-sideband")) + exit(run_processes_parallel(jobs, parallel_next, NULL, NULL, + test_consume_sideband, NULL, + &proc)); + fprintf(stderr, "check usage\n"); return 1; } diff --git a/t/t0061-run-command.sh b/t/t0061-run-command.sh index 87759482ad..e99f6c7f44 100755 --- a/t/t0061-run-command.sh +++ b/t/t0061-run-command.sh @@ -143,6 +143,13 @@ test_expect_success 'run_command runs in parallel with more tasks than jobs avai test_cmp expect actual ' +test_expect_success 'run_command can divert output' ' + test_when_finished rm sideband && + test-tool run-command run-command-sideband 3 sh -c "printf \"%s\n%s\n\" Hello World" 2>actual && + test_must_be_empty actual && + test_cmp expect sideband +' + cat >expect <<-EOF preloaded output of a child listening for stdin: From patchwork Thu May 27 00:08:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 12283021 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 28FB9C47088 for ; Thu, 27 May 2021 00:10:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1192D613CD for ; Thu, 27 May 2021 00:10:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234457AbhE0ALb (ORCPT ); Wed, 26 May 2021 20:11:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51604 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235057AbhE0ALG (ORCPT ); Wed, 26 May 2021 20:11:06 -0400 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DAD28C061760 for ; Wed, 26 May 2021 17:09:32 -0700 (PDT) Received: by mail-yb1-xb4a.google.com with SMTP id o12-20020a5b050c0000b02904f4a117bd74so3621205ybp.17 for ; Wed, 26 May 2021 17:09:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=uTnTkdDSTMFFNbnr5l+txoXk6PxqNXrGkNvjBeciYNk=; b=oIyKtJBrVWrjgTfU2P7B9IC3zoBo4adpHdVYMg4S8KhPEbrykGkKkfcFdV3qFJUqGH jzHAxV+yZPrYznocoxbKTSMjf+M/BZ2ko1HW5N9WVdunUKhM4VZeeDdHZ61TfTdeYppW n7B1DBniyqakfdjEJNQWKKFIBYPqEV0yHF6oPoE57EboBZvcJ2rf+FNrsimPP8dcoSLc xZTNqSilnE7sgkFv95jooO/8lKpgGqY4UQI07LylXfty2Sv0kOvAdiLzzU3FqdeXaYcA J9FabWLHcUGWFuZTIwvphAdmjDS7nwgpB4yg6O+YgXVsR4Ch5XUFtKlPDWTQD2ZB0Y8X F4gw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=uTnTkdDSTMFFNbnr5l+txoXk6PxqNXrGkNvjBeciYNk=; b=hzoa+OFnv/VbFPldEWFpXHl+Ko0c+7uFbKK4MsYyjES+CBSrOe5ISBCmlvQQfYjNzk AXKwnN2O3w0ajac9y+Wr3qhUtseh13PQB6j9EkxLVHngLcRYs20g3XMGbivOGx+ofadr B48Au4s9swKIO16jhCuc+i0vaLt0R3z3WqBVaTQDHlPlfJAcODgwuhn05OxkP2lihwBr LALGAx54t8j3U3AjU/ACT337tT+Ue/ZebSg6qAg5npFoKT/uzVcpvyhnWPm7UuT6JMPn Q2OTy0eNitOhEs0fNEnuo8Quv+k0BTKsBG3BRiruICpR0MHMWAxQJDggxuwbrGla/+wI i/mQ== X-Gm-Message-State: AOAM531QmBwn7vr1aC9d8MaKaKJOMDgxcziJk3e5vHK9iwxxap2kgSQ/ JWksVHV27px7yt8SRDApwX+6vuDkU4E7Zebwu4Rj3te5XPkmaqx01VTCBos+C7pUUja+GN0atOm L0hiA/2rGnx8+hIP2CIRVA29nfJKRwrG9sIrKWfovNYcDbP9l3knBFXdb5t75A844EimWfkaTIQ == X-Google-Smtp-Source: ABdhPJwN+igzAO+v8OyWF00UPrfjtL+dWZkYAzWFRKJhjZHQMwXXdn5dsOU984CQ/Z81AvKpQYLUTeViARHHC6QBdMU= X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:c1a4:4d87:8b5a:d12c]) (user=emilyshaffer job=sendgmr) by 2002:a5b:7c8:: with SMTP id t8mr849376ybq.49.1622074172053; Wed, 26 May 2021 17:09:32 -0700 (PDT) Date: Wed, 26 May 2021 17:08:35 -0700 In-Reply-To: <20210527000856.695702-1-emilyshaffer@google.com> Message-Id: <20210527000856.695702-17-emilyshaffer@google.com> Mime-Version: 1.0 References: <20210527000856.695702-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v9 16/37] hooks: allow callers to capture output From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Some server-side hooks will require capturing output to send over sideband instead of printing directly to stderr. Expose that capability. Signed-off-by: Emily Shaffer --- Notes: You can see this in practice in the conversions for some of the push hooks, like 'receive-pack'. hook.c | 3 ++- hook.h | 8 ++++++++ 2 files changed, 10 insertions(+), 1 deletion(-) diff --git a/hook.c b/hook.c index d48071bb1d..27da1fdb32 100644 --- a/hook.c +++ b/hook.c @@ -262,6 +262,7 @@ void run_hooks_opt_init_sync(struct run_hooks_opt *o) o->dir = NULL; o->feed_pipe = NULL; o->feed_pipe_ctx = NULL; + o->consume_sideband = NULL; } void run_hooks_opt_init_async(struct run_hooks_opt *o) @@ -437,7 +438,7 @@ int run_hooks(const char *hookname, struct run_hooks_opt *options) pick_next_hook, notify_start_failure, options->feed_pipe, - NULL, + options->consume_sideband, notify_hook_finished, &cb_data, "hook", diff --git a/hook.h b/hook.h index aba62d4112..f32189380a 100644 --- a/hook.h +++ b/hook.h @@ -78,6 +78,14 @@ struct run_hooks_opt feed_pipe_fn feed_pipe; void *feed_pipe_ctx; + /* + * Populate this to capture output and prevent it from being printed to + * stderr. This will be passed directly through to + * run_command:run_parallel_processes(). See t/helper/test-run-command.c + * for an example. + */ + consume_sideband_fn consume_sideband; + /* Number of threads to parallelize across */ int jobs; From patchwork Thu May 27 00:08:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 12283023 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EA86DC47082 for ; Thu, 27 May 2021 00:10:01 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C1A46613CD for ; Thu, 27 May 2021 00:10:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235072AbhE0ALc (ORCPT ); Wed, 26 May 2021 20:11:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51668 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235073AbhE0ALJ (ORCPT ); Wed, 26 May 2021 20:11:09 -0400 Received: from mail-qv1-xf4a.google.com (mail-qv1-xf4a.google.com [IPv6:2607:f8b0:4864:20::f4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ED2A5C061345 for ; Wed, 26 May 2021 17:09:34 -0700 (PDT) Received: by mail-qv1-xf4a.google.com with SMTP id n12-20020a0c8c0c0000b02901edb8963d4dso2546467qvb.18 for ; Wed, 26 May 2021 17:09:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=uO0yNrmc7QwXPsixtRLiwPhdxFgoz9yKm6nIA0ICH0g=; b=YIIU60K3MiHV0BqEZgfGySsih3YTrvPnS4NolQHWt48U5pIxhaXdiuaz4BgFYoCgnJ LjQBEs8rjarQDJFqVt6RWgfb+wpf/HLSHWOY/CZG1+7g1AFzf1954zG37dmYxBnKiFhI fDYcClhrWBsVNoO/5KBYUcEvSKoTFe3hm4o2N/S7vq3HvSCY80mpv/gYAp2dD6vZ0phO bRRZoJOlH9wsJrfqOMwuHquFlXihvnGU8IqW5MEeX8zfC+aOKOJ9aKcPvr8HKm3rBUbi iL/ooAFRgaRqiLDSq43omenTtCX1sia67SEZcmWcwjKLYJwuRaT3MHgbP+5rbJKTpVpd i/eA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=uO0yNrmc7QwXPsixtRLiwPhdxFgoz9yKm6nIA0ICH0g=; b=P+YMuxMyVcgtXv2PGF5HNR4JdOntRAHULM8nBiXWyhy5N+Dj++Ht5G8USE5WtY38gq iSK2rw5r1XmyfCJE+jKee8+vQgo9KRuEFbnWCnWIzXpH6R1QpfkiZgzeAF45FljUi6o2 giCKcNmQpW5qobBuI0hrJH7+QW46C8PYYCnu33Rrjlkgjcf3Y0NCvekbPyZXBeXmL3v9 kuHtxgnzahra2DKdXEivuGxZQYlFybki8GIghgWx7CzdcwDm8yh5zMQh+FTYpeU0QvOS VEyLSSk9B9TYyni9AmsO4SuxSs1RK0MDKAGwpfPr3KesZFpV8JM9jkiQP0lriJNwMRjZ wE9g== X-Gm-Message-State: AOAM5321hHHoaGAAoE4dFy+iTDe/yX645DCnnL1PKtLtNN38XoVwhusy RYskx6FVPYH6YdmWVBTf1XIEzXcYOxc1IORwoPkiSrH8uT6gz8dTpzdpUivoXeMvz9njnfybx1g NuFMtTG9s6V3RQhTLmzMzPgUb5adliJJNFJ00m8M4oITZ5pPZ2zhU7YqgLSUKwexL2W5acLYEtw == X-Google-Smtp-Source: ABdhPJyDI+xqAA7ik4mekh5GtfeO+1eqV2NiDB9Hb/rLEsmROYAb7/EcQ78BmvM23ieu4p8Hu6pabfTaAWPynXtgXWY= X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:c1a4:4d87:8b5a:d12c]) (user=emilyshaffer job=sendgmr) by 2002:a0c:d80f:: with SMTP id h15mr808415qvj.17.1622074174067; Wed, 26 May 2021 17:09:34 -0700 (PDT) Date: Wed, 26 May 2021 17:08:36 -0700 In-Reply-To: <20210527000856.695702-1-emilyshaffer@google.com> Message-Id: <20210527000856.695702-18-emilyshaffer@google.com> Mime-Version: 1.0 References: <20210527000856.695702-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v9 17/37] commit: use config-based hooks From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org As part of the adoption of config-based hooks, teach run_commit_hook() to call hook.h instead of run-command.h. This covers 'pre-commit', 'commit-msg', and 'prepare-commit-msg'. Additionally, ask the hook library - not run-command - whether any hooks will be run, as it's possible hooks may exist in the config but not the hookdir. Because all but 'post-commit' hooks are expected to make some state change, force all but 'post-commit' hook to run in series. 'post-commit' "is meant primarily for notification, and cannot affect the outcome of `git commit`," so it is fine to run in parallel. Signed-off-by: Emily Shaffer --- Documentation/githooks.txt | 13 +++++++++++ builtin/commit.c | 11 +++++----- builtin/merge.c | 9 ++++---- commit.c | 22 ++++++++++++++----- commit.h | 3 ++- sequencer.c | 7 +++--- ...3-pre-commit-and-pre-merge-commit-hooks.sh | 17 ++++++++++++-- 7 files changed, 61 insertions(+), 21 deletions(-) diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt index b51959ff94..4af202b366 100644 --- a/Documentation/githooks.txt +++ b/Documentation/githooks.txt @@ -103,6 +103,8 @@ The default 'pre-commit' hook, when enabled--and with the `hooks.allownonascii` config option unset or set to false--prevents the use of non-ASCII filenames. +Hooks executed during 'pre-commit' will not be parallelized. + pre-merge-commit ~~~~~~~~~~~~~~~~ @@ -125,6 +127,8 @@ need to be resolved and the result committed separately (see linkgit:git-merge[1]). At that point, this hook will not be executed, but the 'pre-commit' hook will, if it is enabled. +Hooks executed during 'pre-merge-commit' will not be parallelized. + prepare-commit-msg ~~~~~~~~~~~~~~~~~~ @@ -150,6 +154,9 @@ be used as replacement for pre-commit hook. The sample `prepare-commit-msg` hook that comes with Git removes the help message found in the commented portion of the commit template. +Hooks executed during 'prepare-commit-msg' will not be parallelized, because +hooks are expected to edit the file containing the commit log message. + commit-msg ~~~~~~~~~~ @@ -166,6 +173,9 @@ file. The default 'commit-msg' hook, when enabled, detects duplicate `Signed-off-by` trailers, and aborts the commit if one is found. +Hooks executed during 'commit-msg' will not be parallelized, because hooks are +expected to edit the file containing the proposed commit log message. + post-commit ~~~~~~~~~~~ @@ -175,6 +185,9 @@ invoked after a commit is made. This hook is meant primarily for notification, and cannot affect the outcome of `git commit`. +Hooks executed during 'post-commit' will run in parallel, unless hook.jobs is +configured to 1. + pre-rebase ~~~~~~~~~~ diff --git a/builtin/commit.c b/builtin/commit.c index 190d215d43..7e01802961 100644 --- a/builtin/commit.c +++ b/builtin/commit.c @@ -36,6 +36,7 @@ #include "help.h" #include "commit-reach.h" #include "commit-graph.h" +#include "hook.h" static const char * const builtin_commit_usage[] = { N_("git commit [] [--] ..."), @@ -728,7 +729,7 @@ static int prepare_to_commit(const char *index_file, const char *prefix, /* This checks and barfs if author is badly specified */ determine_author_info(author_ident); - if (!no_verify && run_commit_hook(use_editor, index_file, "pre-commit", NULL)) + if (!no_verify && run_commit_hook(use_editor, 0, index_file, "pre-commit", NULL)) return 0; if (squash_message) { @@ -1044,7 +1045,7 @@ static int prepare_to_commit(const char *index_file, const char *prefix, return 0; } - if (!no_verify && find_hook("pre-commit")) { + if (!no_verify && hook_exists("pre-commit", HOOKDIR_USE_CONFIG)) { /* * Re-read the index as pre-commit hook could have updated it, * and write it out as a tree. We must do this before we invoke @@ -1059,7 +1060,7 @@ static int prepare_to_commit(const char *index_file, const char *prefix, return 0; } - if (run_commit_hook(use_editor, index_file, "prepare-commit-msg", + if (run_commit_hook(use_editor, 0, index_file, "prepare-commit-msg", git_path_commit_editmsg(), hook_arg1, hook_arg2, NULL)) return 0; @@ -1076,7 +1077,7 @@ static int prepare_to_commit(const char *index_file, const char *prefix, } if (!no_verify && - run_commit_hook(use_editor, index_file, "commit-msg", git_path_commit_editmsg(), NULL)) { + run_commit_hook(use_editor, 0, index_file, "commit-msg", git_path_commit_editmsg(), NULL)) { return 0; } @@ -1829,7 +1830,7 @@ int cmd_commit(int argc, const char **argv, const char *prefix) repo_rerere(the_repository, 0); run_auto_maintenance(quiet); - run_commit_hook(use_editor, get_index_file(), "post-commit", NULL); + run_commit_hook(use_editor, 1, get_index_file(), "post-commit", NULL); if (amend && !no_post_rewrite) { commit_post_rewrite(the_repository, current_head, &oid); } diff --git a/builtin/merge.c b/builtin/merge.c index eddb8ae70d..182dea418a 100644 --- a/builtin/merge.c +++ b/builtin/merge.c @@ -43,6 +43,7 @@ #include "commit-reach.h" #include "wt-status.h" #include "commit-graph.h" +#include "hook.h" #define DEFAULT_TWOHEAD (1<<0) #define DEFAULT_OCTOPUS (1<<1) @@ -841,14 +842,14 @@ static void prepare_to_commit(struct commit_list *remoteheads) struct strbuf msg = STRBUF_INIT; const char *index_file = get_index_file(); - if (!no_verify && run_commit_hook(0 < option_edit, index_file, "pre-merge-commit", NULL)) + if (!no_verify && run_commit_hook(0 < option_edit, 0, index_file, "pre-merge-commit", NULL)) abort_commit(remoteheads, NULL); /* * Re-read the index as pre-merge-commit hook could have updated it, * and write it out as a tree. We must do this before we invoke * the editor and after we invoke run_status above. */ - if (find_hook("pre-merge-commit")) + if (hook_exists("pre-merge-commit", HOOKDIR_USE_CONFIG)) discard_cache(); read_cache_from(index_file); strbuf_addbuf(&msg, &merge_msg); @@ -869,7 +870,7 @@ static void prepare_to_commit(struct commit_list *remoteheads) append_signoff(&msg, ignore_non_trailer(msg.buf, msg.len), 0); write_merge_heads(remoteheads); write_file_buf(git_path_merge_msg(the_repository), msg.buf, msg.len); - if (run_commit_hook(0 < option_edit, get_index_file(), "prepare-commit-msg", + if (run_commit_hook(0 < option_edit, 0, get_index_file(), "prepare-commit-msg", git_path_merge_msg(the_repository), "merge", NULL)) abort_commit(remoteheads, NULL); if (0 < option_edit) { @@ -877,7 +878,7 @@ static void prepare_to_commit(struct commit_list *remoteheads) abort_commit(remoteheads, NULL); } - if (!no_verify && run_commit_hook(0 < option_edit, get_index_file(), + if (!no_verify && run_commit_hook(0 < option_edit, 0, get_index_file(), "commit-msg", git_path_merge_msg(the_repository), NULL)) abort_commit(remoteheads, NULL); diff --git a/commit.c b/commit.c index 8ea55a447f..0da5b7e7f1 100644 --- a/commit.c +++ b/commit.c @@ -21,6 +21,7 @@ #include "commit-reach.h" #include "run-command.h" #include "shallow.h" +#include "hook.h" static struct commit_extra_header *read_commit_extra_header_lines(const char *buf, size_t len, const char **); @@ -1695,25 +1696,34 @@ size_t ignore_non_trailer(const char *buf, size_t len) return boc ? len - boc : len - cutoff; } -int run_commit_hook(int editor_is_used, const char *index_file, +int run_commit_hook(int editor_is_used, int parallelize, const char *index_file, const char *name, ...) { - struct strvec hook_env = STRVEC_INIT; + struct run_hooks_opt opt; va_list args; + const char *arg; int ret; - strvec_pushf(&hook_env, "GIT_INDEX_FILE=%s", index_file); + run_hooks_opt_init_sync(&opt); + + if (parallelize) + opt.jobs = configured_hook_jobs(); + + strvec_pushf(&opt.env, "GIT_INDEX_FILE=%s", index_file); /* * Let the hook know that no editor will be launched. */ if (!editor_is_used) - strvec_push(&hook_env, "GIT_EDITOR=:"); + strvec_push(&opt.env, "GIT_EDITOR=:"); va_start(args, name); - ret = run_hook_ve(hook_env.v, name, args); + while ((arg = va_arg(args, const char *))) + strvec_push(&opt.args, arg); va_end(args); - strvec_clear(&hook_env); + + ret = run_hooks(name, &opt); + run_hooks_opt_clear(&opt); return ret; } diff --git a/commit.h b/commit.h index df42eb434f..a90c094ec2 100644 --- a/commit.h +++ b/commit.h @@ -363,7 +363,8 @@ int compare_commits_by_commit_date(const void *a_, const void *b_, void *unused) int compare_commits_by_gen_then_commit_date(const void *a_, const void *b_, void *unused); LAST_ARG_MUST_BE_NULL -int run_commit_hook(int editor_is_used, const char *index_file, const char *name, ...); +int run_commit_hook(int editor_is_used, int parallelize, const char *index_file, + const char *name, ...); /* Sign a commit or tag buffer, storing the result in a header. */ int sign_with_header(struct strbuf *buf, const char *keyid); diff --git a/sequencer.c b/sequencer.c index 0bec01cf38..72234af8ed 100644 --- a/sequencer.c +++ b/sequencer.c @@ -34,6 +34,7 @@ #include "commit-reach.h" #include "rebase-interactive.h" #include "reset.h" +#include "hook.h" #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION" @@ -1207,7 +1208,7 @@ static int run_prepare_commit_msg_hook(struct repository *r, } else { arg1 = "message"; } - if (run_commit_hook(0, r->index_file, "prepare-commit-msg", name, + if (run_commit_hook(0, 0, r->index_file, "prepare-commit-msg", name, arg1, arg2, NULL)) ret = error(_("'prepare-commit-msg' hook failed")); @@ -1445,7 +1446,7 @@ static int try_to_commit(struct repository *r, } } - if (find_hook("prepare-commit-msg")) { + if (hook_exists("prepare-commit-msg", HOOKDIR_USE_CONFIG)) { res = run_prepare_commit_msg_hook(r, msg, hook_commit); if (res) goto out; @@ -1537,7 +1538,7 @@ static int try_to_commit(struct repository *r, goto out; } - run_commit_hook(0, r->index_file, "post-commit", NULL); + run_commit_hook(0, 1, r->index_file, "post-commit", NULL); if (flags & AMEND_MSG) commit_post_rewrite(r, current_head, oid); diff --git a/t/t7503-pre-commit-and-pre-merge-commit-hooks.sh b/t/t7503-pre-commit-and-pre-merge-commit-hooks.sh index 606d8d0f08..e9e3713033 100755 --- a/t/t7503-pre-commit-and-pre-merge-commit-hooks.sh +++ b/t/t7503-pre-commit-and-pre-merge-commit-hooks.sh @@ -8,8 +8,8 @@ export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME . ./test-lib.sh HOOKDIR="$(git rev-parse --git-dir)/hooks" -PRECOMMIT="$HOOKDIR/pre-commit" -PREMERGE="$HOOKDIR/pre-merge-commit" +PRECOMMIT="$(pwd)/$HOOKDIR/pre-commit" +PREMERGE="$(pwd)/$HOOKDIR/pre-merge-commit" # Prepare sample scripts that write their $0 to actual_hooks test_expect_success 'sample script setup' ' @@ -106,6 +106,19 @@ test_expect_success 'with succeeding hook' ' test_cmp expected_hooks actual_hooks ' +# NEEDSWORK: when 'git hook add' and 'git hook remove' have been added, use that +# instead +test_expect_success 'with succeeding hook (config-based)' ' + test_when_finished "git config --unset hook.pre-commit.command success.sample" && + test_when_finished "rm -f expected_hooks actual_hooks" && + git config hook.pre-commit.command "$HOOKDIR/success.sample" && + echo "$HOOKDIR/success.sample" >expected_hooks && + echo "more" >>file && + git add file && + git commit -m "more" && + test_cmp expected_hooks actual_hooks +' + test_expect_success 'with succeeding hook (merge)' ' test_when_finished "rm -f \"$PREMERGE\" expected_hooks actual_hooks" && cp "$HOOKDIR/success.sample" "$PREMERGE" && From patchwork Thu May 27 00:08:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 12283025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C9ABCC47082 for ; Thu, 27 May 2021 00:10:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A7FF06128D for ; Thu, 27 May 2021 00:10:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235116AbhE0ALf (ORCPT ); Wed, 26 May 2021 20:11:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51676 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235075AbhE0ALK (ORCPT ); Wed, 26 May 2021 20:11:10 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D910AC061347 for ; Wed, 26 May 2021 17:09:36 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id i8-20020a25b2080000b0290523c9c81ba5so3630663ybj.20 for ; Wed, 26 May 2021 17:09:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=pIxbakJrlVOPkzM2S4hipsY2erGboEvpJ+ELV0xPoXQ=; b=p+I8JeRC/LUjExo1PvG7U53kcGex6CgfKpsHxee2W8JD7qu6oIwWuymBamc9WtRYM4 LSA0dhugDSW20jiXcwaGolFoLHRI0Hu+kfJ6esWYDl20xkv1fc1UTduwOqbY8cLBB6Zx T8Dt6sNIRY2DvWgYr9oOgJc+1lZtQ+IGMhXInNZXqa1gwGSzcZChjB9xeg7IPPqnNrng PJDG67m5JAK6zIcIJHhKMJyYyOGpbkHqv+4vYhxdOjXUK4V6o6NIfZZstBfPsuxoa51w BJ3cJARXg+BY7k5B2WpD2RqXfDNcIqch9tGLoUPaOXu+QZZQIrTh6pDL4I+XBMLi99d0 dghQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=pIxbakJrlVOPkzM2S4hipsY2erGboEvpJ+ELV0xPoXQ=; b=O6c3fNVN6lEXNvQ4HCZXXC9sLrhsXm9Y8Nb+7Loz9XVM2uZtFv5cUUsSVP0RPvhrSp 46uJW8wlJ+8BsO5iLekzr1gjda57M8VuTbpDjXRvOzFrS6ulJUW2f0AwcwGkeCxTn7AH OvZhfXHKRpT1VV14dxyfHWRyXX/nL8hXIPbuqko2xdqy0NUK6+g1pEx72GbNSByoQloA gZe4RfzPZUGIi9NghmR/GCF61+tjdRQFNKuQX85PVBzPpQireMWLPk54kBU1QTfyZp6T x7G889SiBHpp+qguoETKufMAh2xdET2sC/EOTTBeV5LWaOmc08jHTZrePPzGmIoEhhGc 4WVg== X-Gm-Message-State: AOAM532rHn9XM+Ax29K20c8sBKwdwYWVaohc/t4A52PfFxUK6sQ+gAgK Q62lWrWWUCtQxkIZA4NAfA0IKy6NFGyGySWFw8cqMh+I/HPVY40GOR8hjYaXANmBTg5J2JLCqj7 CgPj5bv34iSTDxB6z3v2py/ypMxabp5IA+GwOLN9v4LK+po0Hwa/CXW2uo2BxqdMHl3UtXTvl0Q == X-Google-Smtp-Source: ABdhPJwmN8V7t0EXlsM16KZMZgOmEGFv84ot9ii4laGvlYz/3JtsFiod1BhnuAhiHR7I84ehPLfwkRptQ6yK14rItjs= X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:c1a4:4d87:8b5a:d12c]) (user=emilyshaffer job=sendgmr) by 2002:a25:3103:: with SMTP id x3mr930807ybx.8.1622074176012; Wed, 26 May 2021 17:09:36 -0700 (PDT) Date: Wed, 26 May 2021 17:08:37 -0700 In-Reply-To: <20210527000856.695702-1-emilyshaffer@google.com> Message-Id: <20210527000856.695702-19-emilyshaffer@google.com> Mime-Version: 1.0 References: <20210527000856.695702-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v9 18/37] am: convert applypatch hooks to use config From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Teach pre-applypatch, post-applypatch, and applypatch-msg to use the hook.h library instead of the run-command.h library. This enables use of hooks specified in the config, in addition to those in the hookdir. These three hooks are called only by builtin/am.c. Signed-off-by: Emily Shaffer --- Documentation/githooks.txt | 9 +++++++++ builtin/am.c | 21 ++++++++++++++++++--- 2 files changed, 27 insertions(+), 3 deletions(-) diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt index 4af202b366..ec5020bfb6 100644 --- a/Documentation/githooks.txt +++ b/Documentation/githooks.txt @@ -58,6 +58,9 @@ the message file. The default 'applypatch-msg' hook, when enabled, runs the 'commit-msg' hook, if the latter is enabled. +Hooks run during 'applypatch-msg' will not be parallelized, because hooks are +expected to edit the file holding the commit log message. + pre-applypatch ~~~~~~~~~~~~~~ @@ -73,6 +76,9 @@ make a commit if it does not pass certain test. The default 'pre-applypatch' hook, when enabled, runs the 'pre-commit' hook, if the latter is enabled. +Hooks run during 'pre-applypatch' will be run in parallel, unless hook.jobs is +configured to 1. + post-applypatch ~~~~~~~~~~~~~~~ @@ -82,6 +88,9 @@ and is invoked after the patch is applied and a commit is made. This hook is meant primarily for notification, and cannot affect the outcome of `git am`. +Hooks run during 'post-applypatch' will be run in parallel, unless hook.jobs is +configured to 1. + pre-commit ~~~~~~~~~~ diff --git a/builtin/am.c b/builtin/am.c index 0b2d886c81..d84791859c 100644 --- a/builtin/am.c +++ b/builtin/am.c @@ -33,6 +33,7 @@ #include "string-list.h" #include "packfile.h" #include "repository.h" +#include "hook.h" /** * Returns the length of the first line of msg. @@ -444,9 +445,14 @@ static void am_destroy(const struct am_state *state) static int run_applypatch_msg_hook(struct am_state *state) { int ret; + struct run_hooks_opt opt; + + run_hooks_opt_init_sync(&opt); assert(state->msg); - ret = run_hook_le(NULL, "applypatch-msg", am_path(state, "final-commit"), NULL); + strvec_push(&opt.args, am_path(state, "final-commit")); + ret = run_hooks("applypatch-msg", &opt); + run_hooks_opt_clear(&opt); if (!ret) { FREE_AND_NULL(state->msg); @@ -1606,9 +1612,16 @@ static void do_commit(const struct am_state *state) struct commit_list *parents = NULL; const char *reflog_msg, *author, *committer = NULL; struct strbuf sb = STRBUF_INIT; + struct run_hooks_opt hook_opt; + + run_hooks_opt_init_async(&hook_opt); - if (run_hook_le(NULL, "pre-applypatch", NULL)) + if (run_hooks("pre-applypatch", &hook_opt)) { + run_hooks_opt_clear(&hook_opt); exit(1); + } + + run_hooks_opt_clear(&hook_opt); if (write_cache_as_tree(&tree, 0, NULL)) die(_("git write-tree failed to write a tree")); @@ -1659,8 +1672,10 @@ static void do_commit(const struct am_state *state) fclose(fp); } - run_hook_le(NULL, "post-applypatch", NULL); + run_hooks_opt_init_async(&hook_opt); + run_hooks("post-applypatch", &hook_opt); + run_hooks_opt_clear(&hook_opt); strbuf_release(&sb); } From patchwork Thu May 27 00:08:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 12283027 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1D6F1C47088 for ; Thu, 27 May 2021 00:10:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EFE45613CA for ; Thu, 27 May 2021 00:10:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235143AbhE0ALr (ORCPT ); Wed, 26 May 2021 20:11:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51712 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234503AbhE0ALT (ORCPT ); Wed, 26 May 2021 20:11:19 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C210BC06134A for ; Wed, 26 May 2021 17:09:38 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id k34-20020a25b2a20000b02905149e86803eso3642062ybj.9 for ; Wed, 26 May 2021 17:09:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=btlcOqWp2Hrr3jczh4cqmNxLjf86ggWODpJ2RlSu0+c=; b=HyDy3Xk5/RcHwzvBaXH+PXNmLKMnMMsFUncX1+K9rcGfTBF5yqHrR8A6yT4/1LlBF+ dJObK102X1avKZLscULnvHl8J1h0kUyNMzurLbTh31wO89WBb9dZGGK4YLVU+ENdmmfJ nvOU9HXHXqu5MCL4rI6sbjPnr5BpysCecjlZ81mYHLTOpdWs9PTBwhIxFrUu4mfvxFN/ bKgICq7XYhm/5Iz9pa55vhcmQBoqILbDrNyyhmCHI6cdKkIFZy+dAf7NNrWuM61TERRA b4HTxkJm6xLecBTexssohYYSL8ta92cG22h7zH5IBcYWCEXjqpZHGOecQdmL98c/xiHV BcgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=btlcOqWp2Hrr3jczh4cqmNxLjf86ggWODpJ2RlSu0+c=; b=Ic5HSd7UW7WFtpTIa3DYUqtr2JvTOI19GbgvBMdwviD/fjPAt3nYtDJQUzLOZPaBE2 hO5Cr+F57ZTK0cs6wyru7n9AOFjtfvgN5p8uvlE/GWyGTBwR7xiAODcT848+ZRyyILg2 Gophhd9/0OdMeB+ycQv+32PWMWqoMhA9FRnF51L94qOOGXP5RgJVsMflJrq3TlNfXc1O VeE+IkY8KpZpTZQfoDN1YuJSn8N6vQ9tcfUrKEWDD0tp8NLtGCWnBB8gT2Xrz6Z7hFuU vIqtj2B6zjNX93bJMRdlxXJCjNQtHnGATX6cMokhFmFOnEGrEvX73GPP3LJ28IOARHF2 kaeg== X-Gm-Message-State: AOAM530Wvdv9Usky2k8L1oX4OppYne92wLu4gKTL52ud35W9Ck5JAQid P/EvaUNX3j0ZYnrfJ+pH7eGQugeGuXep5Q+3CPmlLGUKA4OxImypOxByjRlvSmByyM3KEPsSwrm 9dku87ZfVabsTEbD6mLJEjzc79EP21IrNK7Pv1r2M9lkPaZPcBijwmzhkCkGslWHAWjyBytk4+A == X-Google-Smtp-Source: ABdhPJxWm3lCJZh0ACxr6PJy7NH5ko5sWrGymAKqZbM9x62mrMlnRh2fM/9sY2bRn/8SDkdzb4CBzq2+kObucYNplCM= X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:c1a4:4d87:8b5a:d12c]) (user=emilyshaffer job=sendgmr) by 2002:a25:8b12:: with SMTP id i18mr858896ybl.162.1622074177910; Wed, 26 May 2021 17:09:37 -0700 (PDT) Date: Wed, 26 May 2021 17:08:38 -0700 In-Reply-To: <20210527000856.695702-1-emilyshaffer@google.com> Message-Id: <20210527000856.695702-20-emilyshaffer@google.com> Mime-Version: 1.0 References: <20210527000856.695702-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v9 19/37] merge: use config-based hooks for post-merge hook From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Teach post-merge to use the hook.h library instead of the run-command.h library to run hooks. This means that post-merge hooks can come from the config as well as from the hookdir. post-merge is invoked only from builtin/merge.c. Signed-off-by: Emily Shaffer --- Documentation/githooks.txt | 3 +++ builtin/merge.c | 6 +++++- 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt index ec5020bfb6..c904b160dc 100644 --- a/Documentation/githooks.txt +++ b/Documentation/githooks.txt @@ -242,6 +242,9 @@ save and restore any form of metadata associated with the working tree (e.g.: permissions/ownership, ACLS, etc). See contrib/hooks/setgitperms.perl for an example of how to do this. +Hooks executed during 'post-merge' will run in parallel, unless hook.jobs is +configured to 1. + pre-push ~~~~~~~~ diff --git a/builtin/merge.c b/builtin/merge.c index 182dea418a..7a524cb3e3 100644 --- a/builtin/merge.c +++ b/builtin/merge.c @@ -448,6 +448,7 @@ static void finish(struct commit *head_commit, const struct object_id *new_head, const char *msg) { struct strbuf reflog_message = STRBUF_INIT; + struct run_hooks_opt opt; const struct object_id *head = &head_commit->object.oid; if (!msg) @@ -489,7 +490,10 @@ static void finish(struct commit *head_commit, } /* Run a post-merge hook */ - run_hook_le(NULL, "post-merge", squash ? "1" : "0", NULL); + run_hooks_opt_init_async(&opt); + strvec_push(&opt.args, squash ? "1" : "0"); + run_hooks("post-merge", &opt); + run_hooks_opt_clear(&opt); apply_autostash(git_path_merge_autostash(the_repository)); strbuf_release(&reflog_message); From patchwork Thu May 27 00:08:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 12283029 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 12B18C47088 for ; Thu, 27 May 2021 00:10:20 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EB37C6128D for ; Thu, 27 May 2021 00:10:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235147AbhE0ALu (ORCPT ); Wed, 26 May 2021 20:11:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51576 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233972AbhE0ALV (ORCPT ); Wed, 26 May 2021 20:11:21 -0400 Received: from mail-qv1-xf4a.google.com (mail-qv1-xf4a.google.com [IPv6:2607:f8b0:4864:20::f4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B58F8C06134E for ; Wed, 26 May 2021 17:09:40 -0700 (PDT) Received: by mail-qv1-xf4a.google.com with SMTP id z93-20020a0ca5e60000b02901ec19d8ff47so2569216qvz.8 for ; Wed, 26 May 2021 17:09:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=wnCTag5HKGK8XoqlyYcgrX7r6UraKb8JZF3vVdV1iQQ=; b=fMO08spC8acMpqhB049c0oNKJhNBHktz9mXqeTIhhqbqgvPuI/MXtBRHOMeLh44anl u3aop3NpKrwfmzfJBOTU0a7OMlPjixCdViOHT0cyLA8bdO9oemGq5T8PIbtsxQM/iisG oavgyT1hzQwzR89E7o1V63KkicA6IwFoytdgBRyIFluydf8elSBUzZa387kUXJRsRg8B 0RFebfNbUMzww7cU54P8uh+SLocpNQEvNeGppeZu3BDW+6YnKo6Z7B15MMQ7/hogh+Nw 32By5I+VbI/wJgynjJZHAcIei5u4DcUtuv4YHGVYDreE2n2TjVddirvjiSHUNZqBidoD zbpA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=wnCTag5HKGK8XoqlyYcgrX7r6UraKb8JZF3vVdV1iQQ=; b=NJUgN560gYLsYqvTQaXfg8wU3NOhrMaYrXUVYAcId2xgB1n5SjknWqL1k1Hwf+f3AL sz8e+0RUkzj4jcqEti68r+DOpOZOVlxpGJtyvkvnsOXsZ0uOn2ITjn1rbCVCA/lGVo1O ipH5e7Ls43J4xw5jSndkND22HgaqbPJ9yUgbZrreqdrzvY/nRUYHaxN+53OaxhYXSvPg tQYlbfambXvYmpPgvZNg6h5TUwM6bhoQ5UjBckBrk9hzxQQXmwWBQln/hv9OjX1fKJxW Zv6Hd72AsS+W2X+Vv3gfC7fYtgyNjO8goMjCwL9U5wRyZnat4UnYTjj6Hql6H+Pddl7w qEiQ== X-Gm-Message-State: AOAM530PEZzeKt1MOUj2/yiEZEiu0dOkIxjl5t7JAyjLkv1q2w2UHn/t 5qLG4cblEocc6yhJJ1M6f7I9W5Fg3QLUzR77BgIJCro2vpVHm5XtDQIbWnI33Dp/bBptONURVTT lF/PLAqX6+2i/xNc5bHKsl9sfDFqZexK2Q5p0tsU0Nz/4L04YOq1WcGylyXoRjEFWpXUE22fcoQ == X-Google-Smtp-Source: ABdhPJx1eIlrQOWIDUwY2RxHkIqpyajuIKUpzdMlvqOKMthpic00dg5avMNI46f/JZ+mGzPPxLM+ZxaomIxPNluGwwA= X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:c1a4:4d87:8b5a:d12c]) (user=emilyshaffer job=sendgmr) by 2002:ad4:4c45:: with SMTP id cs5mr571022qvb.6.1622074179837; Wed, 26 May 2021 17:09:39 -0700 (PDT) Date: Wed, 26 May 2021 17:08:39 -0700 In-Reply-To: <20210527000856.695702-1-emilyshaffer@google.com> Message-Id: <20210527000856.695702-21-emilyshaffer@google.com> Mime-Version: 1.0 References: <20210527000856.695702-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v9 20/37] gc: use hook library for pre-auto-gc hook From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Using the hook.h library instead of the run-command.h library to run pre-auto-gc means that those hooks can be set up in config files, as well as in the hookdir. pre-auto-gc is called only from builtin/gc.c. Signed-off-by: Emily Shaffer --- Documentation/githooks.txt | 3 +++ builtin/gc.c | 9 ++++++++- 2 files changed, 11 insertions(+), 1 deletion(-) diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt index c904b160dc..d77170dafb 100644 --- a/Documentation/githooks.txt +++ b/Documentation/githooks.txt @@ -569,6 +569,9 @@ This hook is invoked by `git gc --auto` (see linkgit:git-gc[1]). It takes no parameter, and exiting with non-zero status from this script causes the `git gc --auto` to abort. +Hooks run during 'pre-auto-gc' will be run in parallel, unless hook.jobs is +configured to 1. + post-rewrite ~~~~~~~~~~~~ diff --git a/builtin/gc.c b/builtin/gc.c index f05d2f0a1a..16890b097c 100644 --- a/builtin/gc.c +++ b/builtin/gc.c @@ -32,6 +32,7 @@ #include "remote.h" #include "object-store.h" #include "exec-cmd.h" +#include "hook.h" #define FAILED_RUN "failed to run %s" @@ -348,6 +349,8 @@ static void add_repack_incremental_option(void) static int need_to_gc(void) { + struct run_hooks_opt hook_opt; + /* * Setting gc.auto to 0 or negative can disable the * automatic gc. @@ -394,8 +397,12 @@ static int need_to_gc(void) else return 0; - if (run_hook_le(NULL, "pre-auto-gc", NULL)) + run_hooks_opt_init_async(&hook_opt); + if (run_hooks("pre-auto-gc", &hook_opt)) { + run_hooks_opt_clear(&hook_opt); return 0; + } + run_hooks_opt_clear(&hook_opt); return 1; } From patchwork Thu May 27 00:08:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 12283031 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4A79DC47088 for ; Thu, 27 May 2021 00:10:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1B84D6128D for ; Thu, 27 May 2021 00:10:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235153AbhE0AL7 (ORCPT ); Wed, 26 May 2021 20:11:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51578 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235024AbhE0ALX (ORCPT ); Wed, 26 May 2021 20:11:23 -0400 Received: from mail-qt1-x84a.google.com (mail-qt1-x84a.google.com [IPv6:2607:f8b0:4864:20::84a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 90203C06134F for ; Wed, 26 May 2021 17:09:42 -0700 (PDT) Received: by mail-qt1-x84a.google.com with SMTP id j19-20020ac85f930000b029021f033edf60so1663559qta.10 for ; Wed, 26 May 2021 17:09:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=g8vUeaNpFTZY8ypWo63L8C3N26h6uVvcs49EsDCGhCs=; b=MDwhTybXgB0qCAAllpB6UrQrwfytS6YOK/AXVV2jUqjd7Ruq5qdhW+FBfPit2nchFp m/ZBeuTmH1hwahT9cOFHUrZSaoD8vjv1O0zPIaks/TXQ8D+LhtjP1lsXA858kpIKEUxl Og5F2VNXoyU61zAr2llCTP1uFA4anH/Xf4qPS0G16iP5ymhYoveaZpOeL59udVHscu+S 5FOS98FjoapL97ExjFl0E1TRyPyzqDiuX0c3qCYSvavebzE2UnJSovbULmOv4xk0Uswb T+4XkrOeE4I9EypOLItgiPg2sVAVOMzwbzgxX8CJM1LO0GU3SHZ4hURCMmD77mIPZWq+ o3Zg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=g8vUeaNpFTZY8ypWo63L8C3N26h6uVvcs49EsDCGhCs=; b=Y5yVtB1aHDlWjyyrnm0UMa2U8x+rZ+Z6TL2KrQ9Wwf9VEjASJPBUUfOZVPLAbhs0Nk v6oTYzy49XjOyBlYZaBuFqzSp6I2Qi9JlsJ8GTvyOUik24r1RRCBetadNG2Goh/QZDuN TnxwSyy4Ni7CmvMcwPQsup2AMiOKlMJ29gPYneJprgpRXgsRlscBo9rssGYlMzTfK0Zx 6pQEIXD3/cRvhfy6Vusts4/cSOg28XXpIiwlaOY9+GObXMqFpYRbohh3wiIfhqXxOBiF vvLb1xTzG6kVVNkemxc/4K9j4sdMuSut6EdFeD7+/TDeJ506q+cBSTOo9x9n+gnNDhzt VpFg== X-Gm-Message-State: AOAM531JJWiWqcMmdkuQ7lEbHXvbK+x1Jh22bqevrUq7y0kM7R89ugaE pgn42qbLknOhjjpwGCilyxQxqSztIKESwzGlRFAx4Jb+9msAna4nUPnzuiq4QHp/YWkiggz/IGR c9c/s5GOFht7WHLY7MqnrgpvruCKgAVjP9tk61c9F5OOyfC5h9r6x97IFLKeWQvXWmcNINJjQvw == X-Google-Smtp-Source: ABdhPJw9n9uRH8P9PsSyuRAiIO/0KoBO7j82pNZmlXd3pXMLm40Mbj30oFTV0+ePJsiDlNzlECSrXWy1WBvgiNA8heA= X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:c1a4:4d87:8b5a:d12c]) (user=emilyshaffer job=sendgmr) by 2002:a0c:eb4d:: with SMTP id c13mr835540qvq.48.1622074181620; Wed, 26 May 2021 17:09:41 -0700 (PDT) Date: Wed, 26 May 2021 17:08:40 -0700 In-Reply-To: <20210527000856.695702-1-emilyshaffer@google.com> Message-Id: <20210527000856.695702-22-emilyshaffer@google.com> Mime-Version: 1.0 References: <20210527000856.695702-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v9 21/37] rebase: teach pre-rebase to use hook.h From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org By using hook.h instead of run-command.h to run hooks, pre-rebase hooks can now be specified in the config as well as in the hookdir. pre-rebase is not called anywhere besides builtin/rebase.c. Signed-off-by: Emily Shaffer --- Documentation/githooks.txt | 3 +++ builtin/rebase.c | 10 ++++++++-- 2 files changed, 11 insertions(+), 2 deletions(-) diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt index d77170dafb..7ae24d65ec 100644 --- a/Documentation/githooks.txt +++ b/Documentation/githooks.txt @@ -206,6 +206,9 @@ two parameters. The first parameter is the upstream from which the series was forked. The second parameter is the branch being rebased, and is not set when rebasing the current branch. +Hooks executed during 'pre-rebase' will run in parallel, unless hook.jobs is +configured to 1. + post-checkout ~~~~~~~~~~~~~ diff --git a/builtin/rebase.c b/builtin/rebase.c index 12f093121d..fe9f144cad 100644 --- a/builtin/rebase.c +++ b/builtin/rebase.c @@ -28,6 +28,7 @@ #include "sequencer.h" #include "rebase-interactive.h" #include "reset.h" +#include "hook.h" #define DEFAULT_REFLOG_ACTION "rebase" @@ -1313,6 +1314,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix) char *squash_onto_name = NULL; int reschedule_failed_exec = -1; int allow_preemptive_ff = 1; + struct run_hooks_opt hook_opt; struct option builtin_rebase_options[] = { OPT_STRING(0, "onto", &options.onto_name, N_("revision"), @@ -2022,10 +2024,14 @@ int cmd_rebase(int argc, const char **argv, const char *prefix) } /* If a hook exists, give it a chance to interrupt*/ + run_hooks_opt_init_async(&hook_opt); + strvec_pushl(&hook_opt.args, options.upstream_arg, argc ? argv[0] : NULL, NULL); if (!ok_to_skip_pre_rebase && - run_hook_le(NULL, "pre-rebase", options.upstream_arg, - argc ? argv[0] : NULL, NULL)) + run_hooks("pre-rebase", &hook_opt)) { + run_hooks_opt_clear(&hook_opt); die(_("The pre-rebase hook refused to rebase.")); + } + run_hooks_opt_clear(&hook_opt); if (options.flags & REBASE_DIFFSTAT) { struct diff_options opts; From patchwork Thu May 27 00:08:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 12283033 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 23D28C47082 for ; Thu, 27 May 2021 00:10:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 093A5613CD for ; Thu, 27 May 2021 00:10:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235157AbhE0AMA (ORCPT ); Wed, 26 May 2021 20:12:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51642 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235031AbhE0AL1 (ORCPT ); Wed, 26 May 2021 20:11:27 -0400 Received: from mail-qt1-x84a.google.com (mail-qt1-x84a.google.com [IPv6:2607:f8b0:4864:20::84a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B4E41C061350 for ; Wed, 26 May 2021 17:09:44 -0700 (PDT) Received: by mail-qt1-x84a.google.com with SMTP id f3-20020ac849830000b02901e0f0a55411so1673483qtq.9 for ; Wed, 26 May 2021 17:09:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=FejbK8e94WcgIFCcypxbkEKEaBwHy1lP2msQLc2cmUQ=; b=rMDgGqNp+4HYo64RaeIwfGNoJT8OOk7zKyt0IJtDSVmc3oTZFa5LeUgnGGd0AuTYtv Au/CWByC5WGgi5Y098+gQECeWDwef/1Ri2lWKHqFgHWEb6Iz2yRr/q0i/2xHbv4pI9pu zbv9bIO53ee8snoRKei/Brttl4FVQPz7n9S2s6h4IvZq3i3XWuqNfy3TXADReW+auZix px3wFI1cPceyu9gHcx+uUiW+mS3YyvfkHt5SCkvM3mU5jZY/tNQ1R1v8ADepk/3e0pzf nrmmRZcbSdz/n+ZZxvRhCE0RZlQXai6LIAjV0nqjuue/+RWfCwbsI9wnEjoc9doJzcav KRQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=FejbK8e94WcgIFCcypxbkEKEaBwHy1lP2msQLc2cmUQ=; b=WLwwtdFjIJ+Gip+X2qDXvr/BcEjbT9qS4b2w1F9Q2hx2AzZx54Y22loeIjHJRGGEa1 4lCQG5kmxYuzYp3F0YPa0R1FxqZ/sJ0aE01k7KwUewji0SRw1zrNzLv5iMrN3UuJbdBa /KfnDfKwGfftLy0YrOb7YZoWDCW9NAIwYo+9yGF/5qgSQas2uhtl1Jd0j0r/FIZILNhK jQtSDW+K4OuZoiqEZBDXL0c3NOYB4xuIW+A8lVZuxtwV884vq50rd8XyaW/yvc2RBO5n sdwsmmAqYR4/0Wo2Fe5fLLHILjAXSL+ix1isM7T+tLTCJxBTCzYoV2Cn1qHfLz9fRWrs hmNQ== X-Gm-Message-State: AOAM532qtXSuO2jLfF6C2TZCPH6wuPr7t9KjNbhba5D/H9iWq20LjxLn Wt+OZezM2kPpCCw+wYlQoOIofwQidPbcuacLzDfLKkkYNJ1csTQbIgyqpUKDBY6Ku+GCv+zMqqA Z1I5sEAygJfEitcAaGClCHsc33xnQ8GHZu814Pyu4CUZNRc3ISoHxDS8yFGOxAGJMfUa/jeWy4w == X-Google-Smtp-Source: ABdhPJx0pVEyU/ye0y8Ug5mKXD2LbdJcgLNhPyD7uGHQsAORj2qSKxOGCQfDVVazJ6XVbhFQCGFLeWRcy9eV6LjAycs= X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:c1a4:4d87:8b5a:d12c]) (user=emilyshaffer job=sendgmr) by 2002:a0c:fa4e:: with SMTP id k14mr807886qvo.51.1622074183805; Wed, 26 May 2021 17:09:43 -0700 (PDT) Date: Wed, 26 May 2021 17:08:41 -0700 In-Reply-To: <20210527000856.695702-1-emilyshaffer@google.com> Message-Id: <20210527000856.695702-23-emilyshaffer@google.com> Mime-Version: 1.0 References: <20210527000856.695702-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v9 22/37] read-cache: convert post-index-change hook to use config From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org By using hook.h instead of run-command.h to run, post-index-change hooks can now be specified in the config in addition to the hookdir. post-index-change is not run anywhere besides in read-cache.c. Signed-off-by: Emily Shaffer --- Documentation/githooks.txt | 3 +++ read-cache.c | 14 +++++++++++--- 2 files changed, 14 insertions(+), 3 deletions(-) diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt index 7ae24d65ec..5efa25a44a 100644 --- a/Documentation/githooks.txt +++ b/Documentation/githooks.txt @@ -729,6 +729,9 @@ and "0" meaning they were not. Only one parameter should be set to "1" when the hook runs. The hook running passing "1", "1" should not be possible. +Hooks run during 'post-index-change' will be run in parallel, unless hook.jobs +is configured to 1. + GIT --- Part of the linkgit:git[1] suite diff --git a/read-cache.c b/read-cache.c index 1b3c2eb408..6a5c9403f4 100644 --- a/read-cache.c +++ b/read-cache.c @@ -26,6 +26,8 @@ #include "thread-utils.h" #include "progress.h" #include "sparse-index.h" +#include "hook.h" +>>>>>>> 9524a9d29d (read-cache: convert post-index-change hook to use config) /* Mask for the name length in ce_flags in the on-disk index */ @@ -3131,6 +3133,7 @@ static int do_write_locked_index(struct index_state *istate, struct lock_file *l { int ret; int was_full = !istate->sparse_index; + struct run_hooks_opt hook_opt; ret = convert_to_sparse(istate); @@ -3159,9 +3162,14 @@ static int do_write_locked_index(struct index_state *istate, struct lock_file *l else ret = close_lock_file_gently(lock); - run_hook_le(NULL, "post-index-change", - istate->updated_workdir ? "1" : "0", - istate->updated_skipworktree ? "1" : "0", NULL); + run_hooks_opt_init_async(&hook_opt); + strvec_pushl(&hook_opt.args, + istate->updated_workdir ? "1" : "0", + istate->updated_skipworktree ? "1" : "0", + NULL); + run_hooks("post-index-change", &hook_opt); + run_hooks_opt_clear(&hook_opt); + istate->updated_workdir = 0; istate->updated_skipworktree = 0; From patchwork Thu May 27 00:08:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 12283035 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 462D5C47089 for ; Thu, 27 May 2021 00:10:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2B3C9613BE for ; Thu, 27 May 2021 00:10:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235161AbhE0AMB (ORCPT ); Wed, 26 May 2021 20:12:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51644 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235034AbhE0AL1 (ORCPT ); Wed, 26 May 2021 20:11:27 -0400 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 89AE1C061351 for ; Wed, 26 May 2021 17:09:46 -0700 (PDT) Received: by mail-yb1-xb4a.google.com with SMTP id v184-20020a257ac10000b02904f84a5c5297so3633345ybc.16 for ; Wed, 26 May 2021 17:09:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=iLOtjpzDnEaWUXWYBndYmwd+SM1Z7rlGSIHzSaeSOSM=; b=Pg1uVs/b5OhtBq4YKVNDQVz2eHh7ejeSNgZol5R/XBPPwQ9Eo9gF4htjraSbAQg3zB skooIxu2unTxTlSqINTlQF+VXZv4UbIQgSk3UGi4O1qFBG/BkaAZ2BDyGM4FLAlfN0WJ ZG8wlRG2xKHAJ4+WggaHT4mt+a6QEKYNrCIYBUafPV4dneFzpEu1qAVxq4r0M1AhejQk 0SxmYPqnzpY+zemjdsbZWHIN1IMi8NhxgSdE93rWZq+9qOharjqihL5pl0lqQUDz0REZ jKUJsN/Qg+lzyTzaY7em5JfRVNPTxem8EfJiKdphGtkMkahwwpWJ/vmQdRM0vcukLPdc v3lQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=iLOtjpzDnEaWUXWYBndYmwd+SM1Z7rlGSIHzSaeSOSM=; b=G3NyDcQOu8eI1nrWw6/5T7aDOMGYZBh37dcvpNwieig8RYbRBV8NIdEr6zMget8vKi ibJT+xJTgzaVm3Xjs3ZUphwyzNbeqKTYkVziuigvqAJWGppurGLqtlAXvybgdcBwJnkw eW3+97VU/1kNp5slDbPbXKQL+L6TmPNAbMNk8VRVli0ZW8Sp22KjXAFi7ey37VhdBGHp gwechq7bFyKC7RClJHKSoWjfTcru9fztkLb+E+LyvlcvFsMKN3xdmFS21zE0NYH2CPcr IDk5wI+6C4MRz8QZdPaoUDWO/LW2oJMa4V/l5Y18N7Ln17qF4OaGsSq5Hm0Zpkekd9yY ms+Q== X-Gm-Message-State: AOAM530QmW6luCTdM8jY6K9+UHGG5u7C/d0kYCDFlXziDBDwZvaarh10 4CCx+NalC++OTwgA0lFdHXUhucYcWl6sEZRC8zDSmrugGVfvEtX4ktJwN6IRCsjg8fExlVbkFhq zRLMv0VvHyAi+S5fc9Jec0ox9c9lkmwvLxzyYmKCFyFIBW5UgYC4A4mi/XYG/Hn8VUONnDybAQw == X-Google-Smtp-Source: ABdhPJwJaDJaqlAzrzVWw878c+lU3yG2D7bBSRUDFy6Zd9oO9I9ahzOLr/gC0gdp9akoY2DjO8qfr8CERsXcxWW55HI= X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:c1a4:4d87:8b5a:d12c]) (user=emilyshaffer job=sendgmr) by 2002:a25:a1c4:: with SMTP id a62mr866486ybi.338.1622074185655; Wed, 26 May 2021 17:09:45 -0700 (PDT) Date: Wed, 26 May 2021 17:08:42 -0700 In-Reply-To: <20210527000856.695702-1-emilyshaffer@google.com> Message-Id: <20210527000856.695702-24-emilyshaffer@google.com> Mime-Version: 1.0 References: <20210527000856.695702-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v9 23/37] receive-pack: convert push-to-checkout hook to hook.h From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org By using hook.h instead of run-command.h to invoke push-to-checkout, hooks can now be specified in the config as well as in the hookdir. push-to-checkout is not called anywhere but in builtin/receive-pack.c. Signed-off-by: Emily Shaffer --- Documentation/githooks.txt | 1 + builtin/receive-pack.c | 17 +++++++++++++---- 2 files changed, 14 insertions(+), 4 deletions(-) diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt index 5efa25a44a..17ffeebf07 100644 --- a/Documentation/githooks.txt +++ b/Documentation/githooks.txt @@ -564,6 +564,7 @@ that switches branches while keeping the local changes in the working tree that do not interfere with the difference between the branches. +Hooks executed during 'push-to-checkout' will not be parallelized. pre-auto-gc ~~~~~~~~~~~ diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c index a34742513a..09e7cdee45 100644 --- a/builtin/receive-pack.c +++ b/builtin/receive-pack.c @@ -29,6 +29,7 @@ #include "commit-reach.h" #include "worktree.h" #include "shallow.h" +#include "hook.h" static const char * const receive_pack_usage[] = { N_("git receive-pack "), @@ -1435,12 +1436,20 @@ static const char *push_to_checkout(unsigned char *hash, struct strvec *env, const char *work_tree) { + struct run_hooks_opt opt; + + run_hooks_opt_init_sync(&opt); + strvec_pushf(env, "GIT_WORK_TREE=%s", absolute_path(work_tree)); - if (run_hook_le(env->v, push_to_checkout_hook, - hash_to_hex(hash), NULL)) + strvec_pushv(&opt.env, env->v); + strvec_push(&opt.args, hash_to_hex(hash)); + if (run_hooks(push_to_checkout_hook, &opt)) { + run_hooks_opt_clear(&opt); return "push-to-checkout hook declined"; - else + } else { + run_hooks_opt_clear(&opt); return NULL; + } } static const char *update_worktree(unsigned char *sha1, const struct worktree *worktree) @@ -1464,7 +1473,7 @@ static const char *update_worktree(unsigned char *sha1, const struct worktree *w strvec_pushf(&env, "GIT_DIR=%s", absolute_path(git_dir)); - if (!find_hook(push_to_checkout_hook)) + if (!hook_exists(push_to_checkout_hook, HOOKDIR_USE_CONFIG)) retval = push_to_deploy(sha1, &env, work_tree); else retval = push_to_checkout(sha1, &env, work_tree); From patchwork Thu May 27 00:08:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 12283037 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 14920C47082 for ; Thu, 27 May 2021 00:10:35 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EBF916128D for ; Thu, 27 May 2021 00:10:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235037AbhE0AMD (ORCPT ); Wed, 26 May 2021 20:12:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51652 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235103AbhE0AL2 (ORCPT ); Wed, 26 May 2021 20:11:28 -0400 Received: from mail-qk1-x74a.google.com (mail-qk1-x74a.google.com [IPv6:2607:f8b0:4864:20::74a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6D375C061763 for ; Wed, 26 May 2021 17:09:48 -0700 (PDT) Received: by mail-qk1-x74a.google.com with SMTP id o14-20020a05620a0d4eb02903a5eee61155so1822985qkl.9 for ; Wed, 26 May 2021 17:09:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=XoaFu9RjJSgGuLhD0KFOCrlNkYT8AahKe3FsNktkh5o=; b=Q8ouFtMakxxOFF6QpPXVdp99beqBBoKkByZi7M/tVJQDQrqkIWskhg/3L11+9aWhxF DMKARGtYdTlgZBn6oZk5HI+rzTIVBqQ/PcGmP/IdmTehOTl/s2YuuC9MfVtnjXb7RKtj tNADYFtSnlVb53RApgOIjKyvClI42biiEJo1VkJJdCQm06Xo+ULUMCuLzL+LFT0+I8Ky B3+Twt+RseJXWde2Zafy2sujTN/h4FLXZZRVDav/7KlWACn0PQ1kR7uI5rfgrIhWHCcY b2esWbgjGmn63Y+/4n15mYWiFcbuN3GTQVspU5ImYj0N/kfpIcoY8P5cde3Of3qHF7z9 mwbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=XoaFu9RjJSgGuLhD0KFOCrlNkYT8AahKe3FsNktkh5o=; b=DGKCq5zK3WpPK+gV95br+mDU4fbQaGRlhZdv+obOw+k5YVH4QsJct+IhjHilT7Jyyv tTBvYmarLLLEPBQ68PB8nLZ6Au6BVxMNO2MpJNUw2Ban9DdrB2+KktkeHyYrXwqpzx2C RCcl8//WxNcAKB03dWCMxcn1bRHAgA8iwPtu0Sw/+79FOWswDbnklaeIQt7+eH7bMszO GdIIfkZeGx7kRP+eT879hQYpUWYO8R4aVZGkLhtzTm3qfuw9WU1+e92MBeekOU0ZcNSl i1buHuIs49Yt/aWKy0qg3AnEBV8qa90VhphSHQhrXyVqBhRygmzNCKw2GEnStdMh23v8 Sr5g== X-Gm-Message-State: AOAM5328DIvTbSM8uX7jSe9OVP13DoRze1N0aAzcBPP9o42JWTuSllhQ mS3OP/HcZnK2IcDJsi2pjXBvXHMgBUDz42l1MwIL/4k7nMhGgSw32GydzINw07f0754jP3xstay +rH2MNo/B8i9xodg/3dIGCDxr7dgNKvRl77mil3C8Zu5EAcDl647UoWx8y/8OhOF862UOPgUr0g == X-Google-Smtp-Source: ABdhPJzq5GzLLsa+0fk0kodVExYaUYlSGipXYeKfwieUn9GouwG0G5L/tcuyWAfPDAw+27IeT5lhF1Q34YBO1bpYskA= X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:c1a4:4d87:8b5a:d12c]) (user=emilyshaffer job=sendgmr) by 2002:a05:6214:1421:: with SMTP id o1mr529752qvx.33.1622074187489; Wed, 26 May 2021 17:09:47 -0700 (PDT) Date: Wed, 26 May 2021 17:08:43 -0700 In-Reply-To: <20210527000856.695702-1-emilyshaffer@google.com> Message-Id: <20210527000856.695702-25-emilyshaffer@google.com> Mime-Version: 1.0 References: <20210527000856.695702-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v9 24/37] git-p4: use 'git hook' to run hooks From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Instead of duplicating the behavior of run-command.h:run_hook_le() in Python, we can directly call 'git hook run'. As a bonus, this means git-p4 learns how to find hook specifications from the Git config as well as from the hookdir. Signed-off-by: Emily Shaffer --- Notes: Maybe there is a better way to do this - I had a hard time getting this to run locally, and Python is not my forte, so if anybody has a better approach I'd love to just take that patch instead :) Since v6, removed the developer debug print statements.... :X Maybe there is a better way to do this - I had a hard time getting this to run locally, and Python is not my forte, so if anybody has a better approach I'd love to just take that patch instead :) git-p4.py | 67 +++++-------------------------------------------------- 1 file changed, 6 insertions(+), 61 deletions(-) diff --git a/git-p4.py b/git-p4.py index d34a1946b7..b7192d1153 100755 --- a/git-p4.py +++ b/git-p4.py @@ -208,70 +208,15 @@ def decode_path(path): def run_git_hook(cmd, param=[]): """Execute a hook if the hook exists.""" - if verbose: - sys.stderr.write("Looking for hook: %s\n" % cmd) - sys.stderr.flush() - - hooks_path = gitConfig("core.hooksPath") - if len(hooks_path) <= 0: - hooks_path = os.path.join(os.environ["GIT_DIR"], "hooks") - - if not isinstance(param, list): - param=[param] - - # resolve hook file name, OS depdenent - hook_file = os.path.join(hooks_path, cmd) - if platform.system() == 'Windows': - if not os.path.isfile(hook_file): - # look for the file with an extension - files = glob.glob(hook_file + ".*") - if not files: - return True - files.sort() - hook_file = files.pop() - while hook_file.upper().endswith(".SAMPLE"): - # The file is a sample hook. We don't want it - if len(files) > 0: - hook_file = files.pop() - else: - return True - - if not os.path.isfile(hook_file) or not os.access(hook_file, os.X_OK): + if not cmd: return True - return run_hook_command(hook_file, param) == 0 - -def run_hook_command(cmd, param): - """Executes a git hook command - cmd = the command line file to be executed. This can be - a file that is run by OS association. - - param = a list of parameters to pass to the cmd command - - On windows, the extension is checked to see if it should - be run with the Git for Windows Bash shell. If there - is no file extension, the file is deemed a bash shell - and will be handed off to sh.exe. Otherwise, Windows - will be called with the shell to handle the file assocation. - - For non Windows operating systems, the file is called - as an executable. - """ - cli = [cmd] + param - use_shell = False - if platform.system() == 'Windows': - (root,ext) = os.path.splitext(cmd) - if ext == "": - exe_path = os.environ.get("EXEPATH") - if exe_path is None: - exe_path = "" - else: - exe_path = os.path.join(exe_path, "bin") - cli = [os.path.join(exe_path, "SH.EXE")] + cli - else: - use_shell = True - return subprocess.call(cli, shell=use_shell) + """args are specified with -a -a -a """ + args = (['git', 'hook', 'run'] + + ["-a" + arg for arg in param] + + [cmd]) + return subprocess.call(args) == 0 def write_pipe(c, stdin): if verbose: From patchwork Thu May 27 00:08:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 12283039 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C7196C47082 for ; Thu, 27 May 2021 00:10:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A75C7613BE for ; Thu, 27 May 2021 00:10:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235100AbhE0AMQ (ORCPT ); Wed, 26 May 2021 20:12:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51568 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235108AbhE0AL3 (ORCPT ); Wed, 26 May 2021 20:11:29 -0400 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 342BBC06138F for ; Wed, 26 May 2021 17:09:50 -0700 (PDT) Received: by mail-yb1-xb4a.google.com with SMTP id s8-20020a5b04480000b029049fb35700b9so3713836ybp.5 for ; Wed, 26 May 2021 17:09:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=aoqljucL2LDctk2U9RJTSfIh18IB7ZyjcWgZzUGaKAE=; b=Ec4vFSUxq2uVWPJqx45e/PagaaJ/Dj92+gOLWRmuPBg9zYqYjMDVuOFvU0BICLuaSj splQeFpKDnGavjkctJ8zObmuh6vVVnqXiwLVjrgvXagkcbdmBZyWp/y2XLhLlWavGodu lJGPOtcPD/7dNG0rlwuhr4v21wM/jhOpKo0nRw+S4/nYXVlQFQe4GnZdMQ2BxSkMd1gS AqG5z5iNlriaj8X3kKB9X0X08y7VEP6kGybxWTEfjFKQXYEb5M4+dsYdwpuwn6cWU6zB TeDvZSOpqwv5LMHoYvTC3enStjw3Ie2l4xGaL+Zq/lZpe1J+GHWZaXT7s/nKoRqqk12Q vtqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=aoqljucL2LDctk2U9RJTSfIh18IB7ZyjcWgZzUGaKAE=; b=QbYFnaqNDxbjdeYaaKXtQcjFP0k6xtQI35ZdMfqGngpmo1I8r4WAVdgB75m2HoWFjc Oes8N56ijTKPRnnYVuhA2Zm6z/LXzbZip99nYjJ1fLJ7Ezwx6BgduQrY2Zq2xyfA8cxc HyceR+x76YCWAe/mXL6heFszRT60IV4YNH1MUpMWSk5/bo3FVCI0wK84av61T1yZgOHg Z6FVMUQhqC12IyTW2mvnllfmgrio4DpqRIA3JY/2afloUMBXG5JsFtJvQMsgbjmcFlMG rNvTypbwCqsC0edEklfrQFS0W8yoeVKKmXNOwbZXsNNxBqUr+fQvUDuADO9TG+k0Gs2b OmXg== X-Gm-Message-State: AOAM530x9kV2kNROaM44c+NGYYlOE3JPEXX005Or9+dz44UrH3pu1i1N HRloVMJXYrTlVbbU8bD5Wgwk0js77DUJY/03ID89hiImnxiI7VlugepdHb0HsSUIG5atKHNWRYp tnbVbcPLV/D+jf8YrE+lPWr6L8jfOp3WjmuVlSVU9Azl7ir/mCb4VuMp2hQrTAqSpNCBzRE1XDQ == X-Google-Smtp-Source: ABdhPJykUXdjgjNWiaYC+zQipOvgILHrAXCGxDP5XVQfeAw2kHOAa6eEi5Q7xGVF6aTaROVk7yc7IRSuXCkhZ6dxyf8= X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:c1a4:4d87:8b5a:d12c]) (user=emilyshaffer job=sendgmr) by 2002:a25:b701:: with SMTP id t1mr871501ybj.348.1622074189337; Wed, 26 May 2021 17:09:49 -0700 (PDT) Date: Wed, 26 May 2021 17:08:44 -0700 In-Reply-To: <20210527000856.695702-1-emilyshaffer@google.com> Message-Id: <20210527000856.695702-26-emilyshaffer@google.com> Mime-Version: 1.0 References: <20210527000856.695702-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v9 25/37] hooks: convert 'post-checkout' hook to hook library From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org By using the 'hook.h' library, 'post-checkout' hooks can now be specified in the config as well as in the hook directory. Signed-off-by: Emily Shaffer --- Documentation/githooks.txt | 2 ++ builtin/checkout.c | 19 ++++++++++++++----- builtin/clone.c | 8 ++++++-- builtin/worktree.c | 32 ++++++++++++++++---------------- read-cache.c | 1 - reset.c | 17 +++++++++++++---- 6 files changed, 51 insertions(+), 28 deletions(-) diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt index 17ffeebf07..6a5ff036ff 100644 --- a/Documentation/githooks.txt +++ b/Documentation/githooks.txt @@ -231,6 +231,8 @@ This hook can be used to perform repository validity checks, auto-display differences from the previous HEAD if different, or set working dir metadata properties. +Hooks executed during 'post-checkout' will not be parallelized. + post-merge ~~~~~~~~~~ diff --git a/builtin/checkout.c b/builtin/checkout.c index f4cd7747d3..1797f05a50 100644 --- a/builtin/checkout.c +++ b/builtin/checkout.c @@ -9,6 +9,7 @@ #include "config.h" #include "diff.h" #include "dir.h" +#include "hook.h" #include "ll-merge.h" #include "lockfile.h" #include "merge-recursive.h" @@ -106,13 +107,21 @@ struct branch_info { static int post_checkout_hook(struct commit *old_commit, struct commit *new_commit, int changed) { - return run_hook_le(NULL, "post-checkout", - oid_to_hex(old_commit ? &old_commit->object.oid : null_oid()), - oid_to_hex(new_commit ? &new_commit->object.oid : null_oid()), - changed ? "1" : "0", NULL); + struct run_hooks_opt opt; + int rc; + + run_hooks_opt_init_sync(&opt); + /* "new_commit" can be NULL when checking out from the index before a commit exists. */ - + strvec_pushl(&opt.args, + oid_to_hex(old_commit ? &old_commit->object.oid : null_oid()), + oid_to_hex(new_commit ? &new_commit->object.oid : null_oid()), + changed ? "1" : "0", + NULL); + rc = run_hooks("post-checkout", &opt); + run_hooks_opt_clear(&opt); + return rc; } static int update_some(const struct object_id *oid, struct strbuf *base, diff --git a/builtin/clone.c b/builtin/clone.c index eeb74c0217..2a2a03bf76 100644 --- a/builtin/clone.c +++ b/builtin/clone.c @@ -32,6 +32,7 @@ #include "connected.h" #include "packfile.h" #include "list-objects-filter-options.h" +#include "hook.h" /* * Overall FIXMEs: @@ -775,6 +776,7 @@ static int checkout(int submodule_progress) struct tree *tree; struct tree_desc t; int err = 0; + struct run_hooks_opt hook_opt; if (option_no_checkout) return 0; @@ -820,8 +822,10 @@ static int checkout(int submodule_progress) if (write_locked_index(&the_index, &lock_file, COMMIT_LOCK)) die(_("unable to write new index file")); - err |= run_hook_le(NULL, "post-checkout", oid_to_hex(null_oid()), - oid_to_hex(&oid), "1", NULL); + run_hooks_opt_init_sync(&hook_opt); + strvec_pushl(&hook_opt.args, oid_to_hex(null_oid()), oid_to_hex(&oid), "1", NULL); + err |= run_hooks("post-checkout", &hook_opt); + run_hooks_opt_clear(&hook_opt); if (!err && (option_recurse_submodules.nr > 0)) { struct strvec args = STRVEC_INIT; diff --git a/builtin/worktree.c b/builtin/worktree.c index 976bf8ed06..017b2cfcb5 100644 --- a/builtin/worktree.c +++ b/builtin/worktree.c @@ -13,6 +13,7 @@ #include "utf8.h" #include "worktree.h" #include "quote.h" +#include "hook.h" static const char * const worktree_usage[] = { N_("git worktree add [] []"), @@ -381,22 +382,21 @@ static int add_worktree(const char *path, const char *refname, * is_junk is cleared, but do return appropriate code when hook fails. */ if (!ret && opts->checkout) { - const char *hook = find_hook("post-checkout"); - if (hook) { - const char *env[] = { "GIT_DIR", "GIT_WORK_TREE", NULL }; - cp.git_cmd = 0; - cp.no_stdin = 1; - cp.stdout_to_stderr = 1; - cp.dir = path; - cp.env = env; - cp.argv = NULL; - cp.trace2_hook_name = "post-checkout"; - strvec_pushl(&cp.args, absolute_path(hook), - oid_to_hex(null_oid()), - oid_to_hex(&commit->object.oid), - "1", NULL); - ret = run_command(&cp); - } + struct run_hooks_opt opt; + + run_hooks_opt_init_sync(&opt); + + strvec_pushl(&opt.env, "GIT_DIR", "GIT_WORK_TREE", NULL); + strvec_pushl(&opt.args, + oid_to_hex(null_oid()), + oid_to_hex(&commit->object.oid), + "1", + NULL); + opt.dir = path; + + ret = run_hooks("post-checkout", &opt); + + run_hooks_opt_clear(&opt); } strvec_clear(&child_env); diff --git a/read-cache.c b/read-cache.c index 6a5c9403f4..ebb9c19056 100644 --- a/read-cache.c +++ b/read-cache.c @@ -27,7 +27,6 @@ #include "progress.h" #include "sparse-index.h" #include "hook.h" ->>>>>>> 9524a9d29d (read-cache: convert post-index-change hook to use config) /* Mask for the name length in ce_flags in the on-disk index */ diff --git a/reset.c b/reset.c index 4bea758053..48d45f5b79 100644 --- a/reset.c +++ b/reset.c @@ -7,6 +7,7 @@ #include "tree-walk.h" #include "tree.h" #include "unpack-trees.h" +#include "hook.h" int reset_head(struct repository *r, struct object_id *oid, const char *action, const char *switch_to_branch, unsigned flags, @@ -126,10 +127,18 @@ int reset_head(struct repository *r, struct object_id *oid, const char *action, ret = create_symref("HEAD", switch_to_branch, reflog_head); } - if (run_hook) - run_hook_le(NULL, "post-checkout", - oid_to_hex(orig ? orig : null_oid()), - oid_to_hex(oid), "1", NULL); + if (run_hook) { + struct run_hooks_opt opt; + + run_hooks_opt_init_sync(&opt); + strvec_pushl(&opt.args, + oid_to_hex(orig ? orig : null_oid()), + oid_to_hex(oid), + "1", + NULL); + run_hooks("post-checkout", &opt); + run_hooks_opt_clear(&opt); + } leave_reset_head: strbuf_release(&msg); From patchwork Thu May 27 00:08:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 12283041 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C39D6C4708A for ; Thu, 27 May 2021 00:10:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A50F0613BE for ; Thu, 27 May 2021 00:10:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233316AbhE0AMT (ORCPT ); Wed, 26 May 2021 20:12:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51670 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235065AbhE0ALa (ORCPT ); Wed, 26 May 2021 20:11:30 -0400 Received: from mail-qv1-xf49.google.com (mail-qv1-xf49.google.com [IPv6:2607:f8b0:4864:20::f49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7B152C061353 for ; Wed, 26 May 2021 17:09:52 -0700 (PDT) Received: by mail-qv1-xf49.google.com with SMTP id x2-20020a0cda020000b02901edb4c412fdso2546737qvj.11 for ; Wed, 26 May 2021 17:09:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=ZdDKsphkbuAGRRhJWNf6o04Yx3VUr1hbdG6NAatJSo8=; b=Tlk2mc/RKy0rt6XVGonwGOdredqV4+ZjskuicSzivE0SD2R6hIdSmO+txoXC5W9W7h o1Qle341sg2oYRVtjn92MaclYQ4cAKRF3e57VlzZHZ85e0yUDgqBhIDUD/MI77hfCqDA DRvEV27vHZovLT74bsXRF/i4ue5kyr+Mm0gQ6k8wz1OlQbz810S9f2S+zKbt86W5ECHJ AAzubp2QVNN83IlwxTwRnSo9+DKwJMCB+bt5EI4TmBp49K5b/0e5h0cjEmRgxiKnd+Bi voeDmMy2iWpI7mG1r3Reu312hQuhHOQx34G7sLb+R5phY99Hj7gupAVSHDPX9gBzrOos iwhQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=ZdDKsphkbuAGRRhJWNf6o04Yx3VUr1hbdG6NAatJSo8=; b=Sow/9srLrDG2YoUZi35YtUtUgQ7OZza5Wy6MB3hTb34EF+5z2Nc0hDhqreK45xa3Ie BXaHCFuTFne2fKwqahuLV5F5Qeuv/vwasbrwzKjiSD85J3fo0NzbYl1RZtPskhA8oVBC 8p9CgkFDz1830O64G9IMFU88raouazthdqf++vXWtDhmlz+p1K2hCY7q0IjIMDDCK7YE LborZ4tgN12KXdRhkWadok5wJIro4k3/aB96fioRqWrUYNMVnQSCev4Qpqy/uNHMoaXF s1a5uFPgIXlO7hIbzk9eO00tGl83jZJhbWhI4ZnV0NfOV8OUjMnYUlEBgQ/o16Vus5Fn ieXA== X-Gm-Message-State: AOAM5339Eo8NN1BqXl+yuhvr6AmYKzhqTmGOw6gfYQeFtRJP2ndWFP47 tn8Ky1RW/2NjJ8S82bMubuxBpWpa9hP73T59DplMscPWtsFhj17XLFVkJFlBmDkfVuqwOGmT5jX ILhw8FlI45ySJRoX+dkW+RsK3EjEt/Aj+I5aX7IxpXc2Qkw10YXaeBUHNBaGyyCpgaDj7Nk9iNg == X-Google-Smtp-Source: ABdhPJw86f1W0g3hc1hkNrOwDUdFtY9ihUIg5ekBz2W0zKmWqGIh0UJIv5HWOQ6XDHZNRhCS1jy1hBRYaE6Hq57hxMY= X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:c1a4:4d87:8b5a:d12c]) (user=emilyshaffer job=sendgmr) by 2002:ad4:450f:: with SMTP id k15mr553700qvu.1.1622074191330; Wed, 26 May 2021 17:09:51 -0700 (PDT) Date: Wed, 26 May 2021 17:08:45 -0700 In-Reply-To: <20210527000856.695702-1-emilyshaffer@google.com> Message-Id: <20210527000856.695702-27-emilyshaffer@google.com> Mime-Version: 1.0 References: <20210527000856.695702-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v9 26/37] hook: convert 'post-rewrite' hook to config From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org By using 'hook.h' for 'post-rewrite', we simplify hook invocations by not needing to put together our own 'struct child_process' and we also learn to run hooks specified in the config as well as the hook dir. The signal handling that's being removed by this commit now takes place in run-command.h:run_processes_parallel(), so it is OK to remove them here. Signed-off-by: Emily Shaffer --- Documentation/githooks.txt | 3 ++ builtin/am.c | 18 +++----- sequencer.c | 85 +++++++++++++++++--------------------- 3 files changed, 47 insertions(+), 59 deletions(-) diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt index 6a5ff036ff..156e4809f6 100644 --- a/Documentation/githooks.txt +++ b/Documentation/githooks.txt @@ -603,6 +603,9 @@ The hook always runs after the automatic note copying (see "notes.rewrite." in linkgit:git-config[1]) has happened, and thus has access to these notes. +Hooks run during 'post-rewrite' will be run in parallel, unless hook.jobs is +configured to 1. + The following command-specific comments apply: rebase:: diff --git a/builtin/am.c b/builtin/am.c index d84791859c..d2534f9a1f 100644 --- a/builtin/am.c +++ b/builtin/am.c @@ -469,23 +469,17 @@ static int run_applypatch_msg_hook(struct am_state *state) */ static int run_post_rewrite_hook(const struct am_state *state) { - struct child_process cp = CHILD_PROCESS_INIT; - const char *hook = find_hook("post-rewrite"); + struct run_hooks_opt opt; int ret; - if (!hook) - return 0; + run_hooks_opt_init_async(&opt); - strvec_push(&cp.args, hook); - strvec_push(&cp.args, "rebase"); + strvec_push(&opt.args, "rebase"); + opt.path_to_stdin = am_path(state, "rewritten"); - cp.in = xopen(am_path(state, "rewritten"), O_RDONLY); - cp.stdout_to_stderr = 1; - cp.trace2_hook_name = "post-rewrite"; + ret = run_hooks("post-rewrite", &opt); - ret = run_command(&cp); - - close(cp.in); + run_hooks_opt_clear(&opt); return ret; } diff --git a/sequencer.c b/sequencer.c index 72234af8ed..3fa7668763 100644 --- a/sequencer.c +++ b/sequencer.c @@ -35,6 +35,7 @@ #include "rebase-interactive.h" #include "reset.h" #include "hook.h" +#include "string-list.h" #define GIT_REFLOG_ACTION "GIT_REFLOG_ACTION" @@ -1147,33 +1148,30 @@ int update_head_with_reflog(const struct commit *old_head, static int run_rewrite_hook(const struct object_id *oldoid, const struct object_id *newoid) { - struct child_process proc = CHILD_PROCESS_INIT; - const char *argv[3]; + struct run_hooks_opt opt; + struct strbuf tmp = STRBUF_INIT; + struct string_list to_stdin = STRING_LIST_INIT_DUP; int code; - struct strbuf sb = STRBUF_INIT; - argv[0] = find_hook("post-rewrite"); - if (!argv[0]) - return 0; + run_hooks_opt_init_async(&opt); - argv[1] = "amend"; - argv[2] = NULL; - - proc.argv = argv; - proc.in = -1; - proc.stdout_to_stderr = 1; - proc.trace2_hook_name = "post-rewrite"; - - code = start_command(&proc); - if (code) - return code; - strbuf_addf(&sb, "%s %s\n", oid_to_hex(oldoid), oid_to_hex(newoid)); - sigchain_push(SIGPIPE, SIG_IGN); - write_in_full(proc.in, sb.buf, sb.len); - close(proc.in); - strbuf_release(&sb); - sigchain_pop(SIGPIPE); - return finish_command(&proc); + strvec_push(&opt.args, "amend"); + + strbuf_addf(&tmp, + "%s %s", + oid_to_hex(oldoid), + oid_to_hex(newoid)); + string_list_append(&to_stdin, tmp.buf); + + opt.feed_pipe = pipe_from_string_list; + opt.feed_pipe_ctx = &to_stdin; + + code = run_hooks("post-rewrite", &opt); + + run_hooks_opt_clear(&opt); + strbuf_release(&tmp); + string_list_clear(&to_stdin, 0); + return code; } void commit_post_rewrite(struct repository *r, @@ -4527,30 +4525,23 @@ static int pick_commits(struct repository *r, flush_rewritten_pending(); if (!stat(rebase_path_rewritten_list(), &st) && st.st_size > 0) { - struct child_process child = CHILD_PROCESS_INIT; - const char *post_rewrite_hook = - find_hook("post-rewrite"); - - child.in = open(rebase_path_rewritten_list(), O_RDONLY); - child.git_cmd = 1; - strvec_push(&child.args, "notes"); - strvec_push(&child.args, "copy"); - strvec_push(&child.args, "--for-rewrite=rebase"); + struct child_process notes_cp = CHILD_PROCESS_INIT; + struct run_hooks_opt hook_opt; + + run_hooks_opt_init_async(&hook_opt); + + notes_cp.in = open(rebase_path_rewritten_list(), O_RDONLY); + notes_cp.git_cmd = 1; + strvec_push(¬es_cp.args, "notes"); + strvec_push(¬es_cp.args, "copy"); + strvec_push(¬es_cp.args, "--for-rewrite=rebase"); /* we don't care if this copying failed */ - run_command(&child); - - if (post_rewrite_hook) { - struct child_process hook = CHILD_PROCESS_INIT; - - hook.in = open(rebase_path_rewritten_list(), - O_RDONLY); - hook.stdout_to_stderr = 1; - hook.trace2_hook_name = "post-rewrite"; - strvec_push(&hook.args, post_rewrite_hook); - strvec_push(&hook.args, "rebase"); - /* we don't care if this hook failed */ - run_command(&hook); - } + run_command(¬es_cp); + + hook_opt.path_to_stdin = rebase_path_rewritten_list(); + strvec_push(&hook_opt.args, "rebase"); + run_hooks("post-rewrite", &hook_opt); + run_hooks_opt_clear(&hook_opt); } apply_autostash(rebase_path_autostash()); From patchwork Thu May 27 00:08:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 12283043 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 08274C47088 for ; Thu, 27 May 2021 00:10:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DD70C613BE for ; Thu, 27 May 2021 00:10:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235124AbhE0AMV (ORCPT ); Wed, 26 May 2021 20:12:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51676 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234877AbhE0ALf (ORCPT ); Wed, 26 May 2021 20:11:35 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 31D6BC061355 for ; Wed, 26 May 2021 17:09:54 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id u13-20020a25f80d0000b029051d7fee31cfso3608481ybd.22 for ; Wed, 26 May 2021 17:09:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=EneeTqZADKMKCck0hLyFMbFE6wKTgQCUhaB2t5/Qwvo=; b=eoI2LE5K2ZtBveXSxqDhJMCAx9wQQdpY8gaH1l9KJckwwm6M399hRqs8JjBBUFmgto yJItAbTkgUdQB5+Sr98swa7vlXpcwLFBqKOxMnXish++i+nVQymdGmKBjrzfDK/kZ1+B cD9xbp4PMNP8SWYoG4xjkaxYLrUzg0G7sCx2VBIJjxf57zyJea6QHV3lJrkvNoh2kwzO nfw/k/Yh8euf/IV9uec9ZISZY6BNtrqf4r6o4sLXYYlA+l8t4U08Bb8oqleOicSCwHFv ep0MsXMPNMsKHvnLa2/AOguAqMSwbKQSDLpZ0JIrj3uPUTDq/SYhKZPe+ACHwZtwba4P EzDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=EneeTqZADKMKCck0hLyFMbFE6wKTgQCUhaB2t5/Qwvo=; b=qhovaMx6/cgekYrOTTTPdJPLdUf1YaQLnUvH+MkBaSv8HlFFVVtiZox3llkGreK8zR x7f1TDxiz1vOcWkv88++C6/SVRzA4TKG8gQraSp6qZ3mPAwtl3cbsuo+gu7qJJpgBa0/ 5PG15S4KOxdgXhJD3QECtRv68a1R6h7P/Vbf76PvetV6RwCPjMTD79gK5Cl6wgBQ+Sc8 Umw4pAmeolKfxhcN8OZbcafC9FQJu6dtZgksv8EPwbLMwGv9MxiroxbeR64W1PMMZalE HKY8Hx6v9btBYezSq3KQcUelhtINW/YArBuKwJWv/SbCfgBuR6F3kljZUkngoXPkjzX7 5E7A== X-Gm-Message-State: AOAM533Dhyo8+IRBOpM5IuVsuSrMbVyKEMst9fipKONRiDzgJt8LreyQ rgJcUPavrLzDiNe0zvKWhCY04WSJes+N+EYN14Wf21gXsyX3D8M6TfIThZwgfpj1PXPDZull114 wlX3sNmKlTuYtkk0TsOtbWUwCAfwXewOfI6I8s+iafaowzsyxT7pV+fq3zgddEdHNno7LCkZYtA == X-Google-Smtp-Source: ABdhPJwXxeYsv4UnDAcnyCUOd/AdUJpUgcOzgyeyDfbxOKUR+zvO/C2hHzHcpWBOG9plEpxOSgzW+xSlYMT4kCUFO7I= X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:c1a4:4d87:8b5a:d12c]) (user=emilyshaffer job=sendgmr) by 2002:a25:ba0c:: with SMTP id t12mr893092ybg.158.1622074193240; Wed, 26 May 2021 17:09:53 -0700 (PDT) Date: Wed, 26 May 2021 17:08:46 -0700 In-Reply-To: <20210527000856.695702-1-emilyshaffer@google.com> Message-Id: <20210527000856.695702-28-emilyshaffer@google.com> Mime-Version: 1.0 References: <20210527000856.695702-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v9 27/37] transport: convert pre-push hook to use config From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org By using the hook.h:run_hooks API, pre-push hooks can be specified in the config as well as in the hookdir. Signed-off-by: Emily Shaffer --- Documentation/githooks.txt | 3 ++ transport.c | 59 +++++++++++--------------------------- 2 files changed, 20 insertions(+), 42 deletions(-) diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt index 156e4809f6..543244ec0a 100644 --- a/Documentation/githooks.txt +++ b/Documentation/githooks.txt @@ -279,6 +279,9 @@ If this hook exits with a non-zero status, `git push` will abort without pushing anything. Information about why the push is rejected may be sent to the user by writing to standard error. +Hooks executed during 'pre-push' will run in parallel, unless hook.jobs is +configured to 1. + [[pre-receive]] pre-receive ~~~~~~~~~~~ diff --git a/transport.c b/transport.c index 6cf3da19eb..dbc7bb7820 100644 --- a/transport.c +++ b/transport.c @@ -22,6 +22,7 @@ #include "protocol.h" #include "object-store.h" #include "color.h" +#include "hook.h" static int transport_use_color = -1; static char transport_colors[][COLOR_MAXLEN] = { @@ -1196,31 +1197,15 @@ static void die_with_unpushed_submodules(struct string_list *needs_pushing) static int run_pre_push_hook(struct transport *transport, struct ref *remote_refs) { - int ret = 0, x; + int ret = 0; + struct run_hooks_opt opt; + struct strbuf tmp = STRBUF_INIT; struct ref *r; - struct child_process proc = CHILD_PROCESS_INIT; - struct strbuf buf; - const char *argv[4]; - - if (!(argv[0] = find_hook("pre-push"))) - return 0; - - argv[1] = transport->remote->name; - argv[2] = transport->url; - argv[3] = NULL; - - proc.argv = argv; - proc.in = -1; - proc.trace2_hook_name = "pre-push"; - - if (start_command(&proc)) { - finish_command(&proc); - return -1; - } - - sigchain_push(SIGPIPE, SIG_IGN); + struct string_list to_stdin = STRING_LIST_INIT_DUP; + run_hooks_opt_init_async(&opt); - strbuf_init(&buf, 256); + strvec_push(&opt.args, transport->remote->name); + strvec_push(&opt.args, transport->url); for (r = remote_refs; r; r = r->next) { if (!r->peer_ref) continue; @@ -1229,30 +1214,20 @@ static int run_pre_push_hook(struct transport *transport, if (r->status == REF_STATUS_REJECT_REMOTE_UPDATED) continue; if (r->status == REF_STATUS_UPTODATE) continue; - strbuf_reset(&buf); - strbuf_addf( &buf, "%s %s %s %s\n", + strbuf_reset(&tmp); + strbuf_addf(&tmp, "%s %s %s %s", r->peer_ref->name, oid_to_hex(&r->new_oid), r->name, oid_to_hex(&r->old_oid)); - - if (write_in_full(proc.in, buf.buf, buf.len) < 0) { - /* We do not mind if a hook does not read all refs. */ - if (errno != EPIPE) - ret = -1; - break; - } + string_list_append(&to_stdin, tmp.buf); } - strbuf_release(&buf); - - x = close(proc.in); - if (!ret) - ret = x; - - sigchain_pop(SIGPIPE); + opt.feed_pipe = pipe_from_string_list; + opt.feed_pipe_ctx = &to_stdin; - x = finish_command(&proc); - if (!ret) - ret = x; + ret = run_hooks("pre-push", &opt); + run_hooks_opt_clear(&opt); + strbuf_release(&tmp); + string_list_clear(&to_stdin, 0); return ret; } From patchwork Thu May 27 00:08:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 12283045 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 88064C47089 for ; Thu, 27 May 2021 00:10:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6D563613CA for ; Thu, 27 May 2021 00:10:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235134AbhE0AMY (ORCPT ); Wed, 26 May 2021 20:12:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51684 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235079AbhE0ALj (ORCPT ); Wed, 26 May 2021 20:11:39 -0400 Received: from mail-qk1-x749.google.com (mail-qk1-x749.google.com [IPv6:2607:f8b0:4864:20::749]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1E878C061574 for ; Wed, 26 May 2021 17:09:56 -0700 (PDT) Received: by mail-qk1-x749.google.com with SMTP id o14-20020a05620a130eb02902ea53a6ef80so2098938qkj.6 for ; Wed, 26 May 2021 17:09:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=5qfSluBsSvJIq1htHRAc/V19BZWcOjZE5Zgmui+lgmI=; b=TtynfeWnbFl5rJLlxhZnjx/ikflCy9mQqU1D5Pabr/zfaA9H50AQcImIdeeeLRJMUC fKLD8J9DPANFlUnM7UPsNdABNs+53O+fSkFFG5zK7Im5N9eG/6k3cK5z+GROQbsvb/sr zGd9cr65wu4x+t7rCu4VCev9grqOXKNf9kyNA+P7APZoPoERQFLiYLgE58IaEDxCYm2R ekX8PsXurTYmtk3gcbZWVQn2RAM48ajBiCj65iKTS/ZAOrFR6JzpNq5g0QusGFu1/iJB yBLrsKKW/K/uiVzfaIo7mn4JL68et/8yDZw15GpY7ls6KYVH9w55aSCtOL4pO/4rLyQh rUVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=5qfSluBsSvJIq1htHRAc/V19BZWcOjZE5Zgmui+lgmI=; b=BlwCO/sFqUuseM6d31BL6tAPPeYjT7GXaA1AXHs+M0kH/2DDBLP0eFQAFxpid+EfF/ 6k/Cv2vzmnAfFXXxfeQUZOT0AjJE6TmqG2ja/JBzzOhHMLnSA5uBs0o+7WSZ99Q0eX/g PafvMuTXb5MV8orjHzu3O+4TXx2SrKvE9Cb0v3CMZZe8074kBuOCB0UrjrycAhPQsf6H wKElaYvlS6wpyWt8OkoJfM2l6yc9w0nmGJfYwdWNVy7ekMhPKJUdVUxjtzRMXm36cth5 mqPNJWw2QRCaqx1jp3HjLMoEttj0TwrSUJZmd9ZPSiwia/iDqa/qvXU7eKZmpgNseq1H CNug== X-Gm-Message-State: AOAM5302wjQOotbS2RaJeJEOwOwjzMxhPJ37pHJTK6LC8I4ezi7409KN WCbAnDULrmSJ4F3oKhcRUjWD8TXOHzG34ME3kE3JdZciIPPM0AxdHcG7I06OPWNkWqy6GmAiDBG WVOQ8ZGPemnAsq54zNEzL3FEuyEVaM812kTY2dwUKf092VTDoKCmXDd+yb1kl8U7Ga07w1ZKLqw == X-Google-Smtp-Source: ABdhPJxlebAdyKUHXhbDN9aGSBOn1YHScRdnvPdkE9iAGOMVlzPdrDwAEJYpLCzgY0D42ZenNfha0SO0r8ZSYtzIzso= X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:c1a4:4d87:8b5a:d12c]) (user=emilyshaffer job=sendgmr) by 2002:a0c:f8d1:: with SMTP id h17mr1073542qvo.2.1622074195199; Wed, 26 May 2021 17:09:55 -0700 (PDT) Date: Wed, 26 May 2021 17:08:47 -0700 In-Reply-To: <20210527000856.695702-1-emilyshaffer@google.com> Message-Id: <20210527000856.695702-29-emilyshaffer@google.com> Mime-Version: 1.0 References: <20210527000856.695702-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v9 28/37] reference-transaction: look for hooks in config From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org By using the hook.h library, reference-transaction hooks can be specified in the config instead. The expected output of the test is not fully updated to reflect the absolute path of the hook called because the 'update' hook has not yet been converted to use hook.h. Signed-off-by: Emily Shaffer --- Documentation/githooks.txt | 3 +++ refs.c | 43 +++++++++++++------------------- t/t1416-ref-transaction-hooks.sh | 8 +++--- transport.c | 1 + 4 files changed, 25 insertions(+), 30 deletions(-) diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt index 543244ec0a..edb840dcdd 100644 --- a/Documentation/githooks.txt +++ b/Documentation/githooks.txt @@ -539,6 +539,9 @@ The exit status of the hook is ignored for any state except for the cause the transaction to be aborted. The hook will not be called with "aborted" state in that case. +Hooks run during 'reference-transaction' will be run in parallel, unless +hook.jobs is configured to 1. + push-to-checkout ~~~~~~~~~~~~~~~~ diff --git a/refs.c b/refs.c index 8c9490235e..32e993aaff 100644 --- a/refs.c +++ b/refs.c @@ -18,6 +18,7 @@ #include "strvec.h" #include "repository.h" #include "sigchain.h" +#include "hook.h" /* * List of all available backends @@ -2061,47 +2062,37 @@ int ref_update_reject_duplicates(struct string_list *refnames, static int run_transaction_hook(struct ref_transaction *transaction, const char *state) { - struct child_process proc = CHILD_PROCESS_INIT; struct strbuf buf = STRBUF_INIT; - const char *hook; + struct run_hooks_opt opt; + struct string_list to_stdin = STRING_LIST_INIT_DUP; int ret = 0, i; + char o[GIT_MAX_HEXSZ + 1], n[GIT_MAX_HEXSZ + 1]; - hook = find_hook("reference-transaction"); - if (!hook) - return ret; - - strvec_pushl(&proc.args, hook, state, NULL); - proc.in = -1; - proc.stdout_to_stderr = 1; - proc.trace2_hook_name = "reference-transaction"; + run_hooks_opt_init_async(&opt); - ret = start_command(&proc); - if (ret) + if (!hook_exists("reference-transaction", HOOKDIR_USE_CONFIG)) return ret; - sigchain_push(SIGPIPE, SIG_IGN); + strvec_push(&opt.args, state); for (i = 0; i < transaction->nr; i++) { struct ref_update *update = transaction->updates[i]; + oid_to_hex_r(o, &update->old_oid); + oid_to_hex_r(n, &update->new_oid); strbuf_reset(&buf); - strbuf_addf(&buf, "%s %s %s\n", - oid_to_hex(&update->old_oid), - oid_to_hex(&update->new_oid), - update->refname); - - if (write_in_full(proc.in, buf.buf, buf.len) < 0) { - if (errno != EPIPE) - ret = -1; - break; - } + strbuf_addf(&buf, "%s %s %s", o, n, update->refname); + string_list_append(&to_stdin, buf.buf); } - close(proc.in); - sigchain_pop(SIGPIPE); + opt.feed_pipe = pipe_from_string_list; + opt.feed_pipe_ctx = &to_stdin; + + ret = run_hooks("reference-transaction", &opt); + run_hooks_opt_clear(&opt); strbuf_release(&buf); + string_list_clear(&to_stdin, 0); - ret |= finish_command(&proc); return ret; } diff --git a/t/t1416-ref-transaction-hooks.sh b/t/t1416-ref-transaction-hooks.sh index 6c941027a8..3a90a59143 100755 --- a/t/t1416-ref-transaction-hooks.sh +++ b/t/t1416-ref-transaction-hooks.sh @@ -125,11 +125,11 @@ test_expect_success 'interleaving hook calls succeed' ' cat >expect <<-EOF && hooks/update refs/tags/PRE $ZERO_OID $PRE_OID - hooks/reference-transaction prepared - hooks/reference-transaction committed + $(pwd)/target-repo.git/hooks/reference-transaction prepared + $(pwd)/target-repo.git/hooks/reference-transaction committed hooks/update refs/tags/POST $ZERO_OID $POST_OID - hooks/reference-transaction prepared - hooks/reference-transaction committed + $(pwd)/target-repo.git/hooks/reference-transaction prepared + $(pwd)/target-repo.git/hooks/reference-transaction committed EOF git push ./target-repo.git PRE POST && diff --git a/transport.c b/transport.c index dbc7bb7820..9191107626 100644 --- a/transport.c +++ b/transport.c @@ -1202,6 +1202,7 @@ static int run_pre_push_hook(struct transport *transport, struct strbuf tmp = STRBUF_INIT; struct ref *r; struct string_list to_stdin = STRING_LIST_INIT_DUP; + run_hooks_opt_init_async(&opt); strvec_push(&opt.args, transport->remote->name); From patchwork Thu May 27 00:08:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 12283047 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 153D2C47088 for ; Thu, 27 May 2021 00:11:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EE9A1613BE for ; Thu, 27 May 2021 00:10:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235089AbhE0AMa (ORCPT ); Wed, 26 May 2021 20:12:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51618 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234699AbhE0ALr (ORCPT ); Wed, 26 May 2021 20:11:47 -0400 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EF41AC061760 for ; Wed, 26 May 2021 17:09:57 -0700 (PDT) Received: by mail-yb1-xb4a.google.com with SMTP id k34-20020a25b2a20000b02905149e86803eso3642840ybj.9 for ; Wed, 26 May 2021 17:09:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=I+L8k7IVYOdFyaHd+iQoVEM+WvZ0n/WN8+K97LewaOY=; b=tUhvWzbnJ+N98w16Gce7jw+anLHwwodjjA93L38VGn4gACFdsWGemEJW04DEnbL9zS rIwAkXH9+Ro213OotBPKHMVIAMYNeU+4fsWuYtzkIbKZwcDOOMErRATnGLJjRfSKY2gO jIV+UyVhiBZGKeyI3hqzrh+yPIy/KtMErW9iWyLMZ8Dupc5MqVUj7t2eejoZbwJdbktE xyvYvAOo/mk0TxwtoXRjx2CjvLh8QbM3g5MQrjJmkUrSgr1WkVEVfbyyNdFprnStLSqr HKTIKsptkqInUorsHvbU/JXT+hWccKQ3Sp4soOW9Kq0B12TFxv5xDIL1+hKyoaKD7ptB 5DYg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=I+L8k7IVYOdFyaHd+iQoVEM+WvZ0n/WN8+K97LewaOY=; b=RQOb8BfhLDNVKusw6+lO3zl6dVVmzP6/Q/L0QPX/fv4vBHQyLte8KI9XTJVNDqm9tR FvBw/ZKfVzuH71F689xR9HjIzSZ70xz29Y2FTrWiZeB1LnMt+DkSZ/oSsOkak3SxUwCe H1triNXmKP6jJ8iXYX5Ngz22PYfD8cqOmOzsPPh6NFSzZkYdONv1cwyCqekthC4mC8V6 f0UuohTFkL22k/hbTfHvwgcDWejKREuO90x7rPqfhvN3WzwC1Y5AhjquiPYknJcgKyNs OnXlNnZZbxiIhMAMF6xSiG8WNtgnjx7JAOVLfPAAzz6lY1wfwmLX5TJPSSxd/y1cZLBo cwoQ== X-Gm-Message-State: AOAM530w3gMeW8LRcZHPeuC48pY3l3BgGz0aeLVlRpoJLsPN/YVBitWj d5f4nHOYWm992uh0FN+V/3EE7PsHSYnGnt9D9CRZf4yYZVCKBOZJe/62l0vHE/FSjx9X0l8VIMA giHL+THR55jbVpOKR45PyeKGYHwBqW8vzU6ayo+/X5UHW2ird9+E6mEW3jne34lFV1hy564kksQ == X-Google-Smtp-Source: ABdhPJyOyVx64Zu1l5lBvtJPcQ0S23YHB8PEGFXEZPIYtg2Ksq/ILi0/vVSHPjRWRd1chDODPuAsgAVKQvH3rTESRII= X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:c1a4:4d87:8b5a:d12c]) (user=emilyshaffer job=sendgmr) by 2002:a25:ba93:: with SMTP id s19mr866748ybg.161.1622074197102; Wed, 26 May 2021 17:09:57 -0700 (PDT) Date: Wed, 26 May 2021 17:08:48 -0700 In-Reply-To: <20210527000856.695702-1-emilyshaffer@google.com> Message-Id: <20210527000856.695702-30-emilyshaffer@google.com> Mime-Version: 1.0 References: <20210527000856.695702-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v9 29/37] receive-pack: convert 'update' hook to hook.h From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org By using hook.h to invoke the 'update' hook, now hooks can be specified in the config in addition to the hookdir. Signed-off-by: Emily Shaffer --- Documentation/githooks.txt | 3 ++ builtin/receive-pack.c | 65 ++++++++++++++++++++++---------- t/t1416-ref-transaction-hooks.sh | 4 +- 3 files changed, 50 insertions(+), 22 deletions(-) diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt index edb840dcdd..1d731474ac 100644 --- a/Documentation/githooks.txt +++ b/Documentation/githooks.txt @@ -368,6 +368,9 @@ The default 'update' hook, when enabled--and with `hooks.allowunannotated` config option unset or set to false--prevents unannotated tags to be pushed. +Hooks executed during 'update' are run in parallel, unless hook.jobs is +configured to 1. + [[proc-receive]] proc-receive ~~~~~~~~~~~~ diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c index 09e7cdee45..0f6bc8653f 100644 --- a/builtin/receive-pack.c +++ b/builtin/receive-pack.c @@ -938,33 +938,58 @@ static int run_receive_hook(struct command *commands, return status; } +static void hook_output_to_sideband(struct strbuf *output, void *cb_data) +{ + int keepalive_active = 0; + + if (keepalive_in_sec <= 0) + use_keepalive = KEEPALIVE_NEVER; + if (use_keepalive == KEEPALIVE_ALWAYS) + keepalive_active = 1; + + /* send a keepalive if there is no data to write */ + if (keepalive_active && !output->len) { + static const char buf[] = "0005\1"; + write_or_die(1, buf, sizeof(buf) - 1); + return; + } + + if (use_keepalive == KEEPALIVE_AFTER_NUL && !keepalive_active) { + const char *first_null = memchr(output->buf, '\0', output->len); + if (first_null) { + /* The null bit is excluded. */ + size_t before_null = first_null - output->buf; + size_t after_null = output->len - (before_null + 1); + keepalive_active = 1; + send_sideband(1, 2, output->buf, before_null, use_sideband); + send_sideband(1, 2, first_null + 1, after_null, use_sideband); + + return; + } + } + + send_sideband(1, 2, output->buf, output->len, use_sideband); +} + static int run_update_hook(struct command *cmd) { - const char *argv[5]; - struct child_process proc = CHILD_PROCESS_INIT; + struct run_hooks_opt opt; int code; - argv[0] = find_hook("update"); - if (!argv[0]) - return 0; + run_hooks_opt_init_async(&opt); - argv[1] = cmd->ref_name; - argv[2] = oid_to_hex(&cmd->old_oid); - argv[3] = oid_to_hex(&cmd->new_oid); - argv[4] = NULL; + strvec_pushl(&opt.args, + cmd->ref_name, + oid_to_hex(&cmd->old_oid), + oid_to_hex(&cmd->new_oid), + NULL); - proc.no_stdin = 1; - proc.stdout_to_stderr = 1; - proc.err = use_sideband ? -1 : 0; - proc.argv = argv; - proc.trace2_hook_name = "update"; - - code = start_command(&proc); - if (code) - return code; if (use_sideband) - copy_to_sideband(proc.err, -1, NULL); - return finish_command(&proc); + opt.consume_sideband = hook_output_to_sideband; + + code = run_hooks("update", &opt); + run_hooks_opt_clear(&opt); + return code; } static struct command *find_command_by_refname(struct command *list, diff --git a/t/t1416-ref-transaction-hooks.sh b/t/t1416-ref-transaction-hooks.sh index 3a90a59143..0a3c3e4a86 100755 --- a/t/t1416-ref-transaction-hooks.sh +++ b/t/t1416-ref-transaction-hooks.sh @@ -124,10 +124,10 @@ test_expect_success 'interleaving hook calls succeed' ' EOF cat >expect <<-EOF && - hooks/update refs/tags/PRE $ZERO_OID $PRE_OID + $(pwd)/target-repo.git/hooks/update refs/tags/PRE $ZERO_OID $PRE_OID $(pwd)/target-repo.git/hooks/reference-transaction prepared $(pwd)/target-repo.git/hooks/reference-transaction committed - hooks/update refs/tags/POST $ZERO_OID $POST_OID + $(pwd)/target-repo.git/hooks/update refs/tags/POST $ZERO_OID $POST_OID $(pwd)/target-repo.git/hooks/reference-transaction prepared $(pwd)/target-repo.git/hooks/reference-transaction committed EOF From patchwork Thu May 27 00:08:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 12283049 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8EB3CC47082 for ; Thu, 27 May 2021 00:11:01 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 68534613CA for ; Thu, 27 May 2021 00:11:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235140AbhE0AMc (ORCPT ); Wed, 26 May 2021 20:12:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51626 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235142AbhE0ALr (ORCPT ); Wed, 26 May 2021 20:11:47 -0400 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D99E2C06138A for ; Wed, 26 May 2021 17:09:59 -0700 (PDT) Received: by mail-yb1-xb4a.google.com with SMTP id d89-20020a25a3620000b02904dc8d0450c6so3659021ybi.2 for ; Wed, 26 May 2021 17:09:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=BCtsCK756f1HfBMcyGwWuDrlhVZRh2EV4JKwSxRhS90=; b=V7+ksot/vWNz/fzICntUlmgwjPkWzatCAgSdRtres994Q/teWdKKpey7LRg+caTlJD MbyzajPzGy2Jk//tXFLb8kPcvSzJ3I1KXRt5oRyU6FYIiJwnEq+aI27tHaQmpEW7dQtV E2NX+B65opCApBiq2ncJsFVmvTrW+eWDV8lixRW3orB40M4X/1a9RXj4BFw2acL6Cr5S VAH2yJSxjazTMIWTwvG68GULhwxSzMfBYmpmMgRVm+00lVv5oJKNY/J2rf1wiEHos9YN MvxYYEiMZzSLeqC43heX6KF/3TyEjbbZ81prA2a4lUZea/p7MYS5wnvDsKX40obcjUF8 n5mw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=BCtsCK756f1HfBMcyGwWuDrlhVZRh2EV4JKwSxRhS90=; b=e9zaEX9WpadVNtd4x7dXcTOJE6vKunp5ZjiG36Iv5l32ZSQyz1ILUDTQ+hNB0QVVqo uIMSSJmeHVgZ74WtMZbD+eNJSNnxZBT5eGKFjn2A9kuqgG8YpvbPJTtOj0FO0EEIhhrP zaAhNcSkTMkvocvjj0PKBftwOty+8BKckA4z7b+L69yNrNaz6eNummsnBnOtGCvRPs07 B3BWZzGaA8xm5uxYrmR3bNIzMM4eN6grWGzREo+mZKWY8m+YMWnlSGbQeni6zBsOmY9d VnoVgG4KhO4RgdcDqOVm6Fl6FZt7gU8ooV2bRFrG99SRCt+eCvQVzjpQ5d9HoQ8XiyFn 0Krw== X-Gm-Message-State: AOAM533v+7V1SMY5Mgh1gXYn/aY0XXfZ67YUn4yBySDJz5H+fGM3Cnvz c2erfnCaqNvKI21Lop+FpVP3BzVTErZhswJ/UDjMRgT1PFKPKYD+62qnYIOece5Exi4SkQd99ub PwSpK4rjp4rPq1W0KpQeUfQxHbPzn+nMDFNrzmfi/IuLhtm7RsJG1AY00XFrR9j5cLIDzoX3Uaw == X-Google-Smtp-Source: ABdhPJy9E8yC5LhB3ABmcqF/YYPm6YI7RZjmOzEERI8MPF7U1BxrVnzu/rbuKv2XcSYEW0biHSCm1Q8GLZY9lOWB1gw= X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:c1a4:4d87:8b5a:d12c]) (user=emilyshaffer job=sendgmr) by 2002:a25:53c9:: with SMTP id h192mr824031ybb.73.1622074199030; Wed, 26 May 2021 17:09:59 -0700 (PDT) Date: Wed, 26 May 2021 17:08:49 -0700 In-Reply-To: <20210527000856.695702-1-emilyshaffer@google.com> Message-Id: <20210527000856.695702-31-emilyshaffer@google.com> Mime-Version: 1.0 References: <20210527000856.695702-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v9 30/37] proc-receive: acquire hook list from hook.h From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org The proc-receive hook differs from most other hooks Git invokes because the hook and the parent Git process engage in bidirectional communication via stdin/stdout. This bidirectional communication is unsuitable for multiple hooks, whether they are in series or in parallel, and is incompatible with run-command.h:run_processes_parallel: - The proc-receive hook is intended to modify the state of the Git repo. From 'git help githooks': This [proc-receive] hook is responsible for updating the relevant references and reporting the results back to 'receive-pack'. This prevents parallelization and implies, at least, specific ordering of hook execution. - The proc-receive hook can reject a push by aborting early with an error code. If a former hook ran through the entire push contents successfully but a later hook rejects some of the push, the repo may be left in a partially-updated (and corrupt) state. - The callback model of the run_processes_parallel() API is unsuited to the current implementation of proc-receive, which loops through "send-receive-consider" with the child process. proc-receive today relies on stateful communication with the child process, which would be unwieldy to implement with callbacks and saved state. - Additionally, run_processes_parallel() is designed to collate the output of many child processes into a single output (stderr or callback), and would require significant work to tell the caller which process sent the output, and indeed to collect any output before the child process has exited. So, rather than using hook.h:run_hooks() to invoke the proc-receive hook, receive-pack.c can learn to ask hook.h:hook_list() for the location of a hook to run. This allows users to configure their proc-receive in a global config for all repos if they want, or a local config if they just don't want to use the hookdir. Because running more than one proc-receive hook doesn't make sense from a repo state perspective, we can explicitly ban configuring more than one proc-receive hook at a time. If a user wants to globally configure one proc-receive hook for most of their repos, but override that hook in a single repo, they should use 'skip' to manually remove the global hook in their special repo: ~/.gitconfig: [hook.proc-receive] command = /usr/bin/usual-proc-receive ~/special-repo/.git/config: [hookcmd./usr/bin/usual-proc-receive] skip = true [hook.proc-receive] command = /usr/bin/special-proc-receive Signed-off-by: Emily Shaffer --- Documentation/githooks.txt | 4 ++ builtin/receive-pack.c | 31 ++++++++++++++- t/t5411/test-0015-too-many-hooks-error.sh | 47 +++++++++++++++++++++++ 3 files changed, 80 insertions(+), 2 deletions(-) create mode 100644 t/t5411/test-0015-too-many-hooks-error.sh diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt index 1d731474ac..b6b13afb36 100644 --- a/Documentation/githooks.txt +++ b/Documentation/githooks.txt @@ -433,6 +433,10 @@ the input. The exit status of the 'proc-receive' hook only determines the success or failure of the group of commands sent to it, unless atomic push is in use. +It is forbidden to specify more than one hook for 'proc-receive'. If a +globally-configured 'proc-receive' must be overridden, use +'hookcmd..skip = true' to ignore it. + [[post-receive]] post-receive ~~~~~~~~~~~~ diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c index 0f6bc8653f..5ecc97d8ab 100644 --- a/builtin/receive-pack.c +++ b/builtin/receive-pack.c @@ -1147,11 +1147,38 @@ static int run_proc_receive_hook(struct command *commands, int version = 0; int code; - argv[0] = find_hook("proc-receive"); - if (!argv[0]) { + struct hook *proc_receive = NULL; + struct list_head *pos, *hooks; + + hooks = hook_list("proc-receive"); + + list_for_each(pos, hooks) { + if (proc_receive) { + rp_error("only one 'proc-receive' hook can be specified"); + return -1; + } + proc_receive = list_entry(pos, struct hook, list); + /* check if the hookdir hook should be ignored */ + if (proc_receive->from_hookdir) { + switch (configured_hookdir_opt()) { + case HOOKDIR_INTERACTIVE: + case HOOKDIR_NO: + proc_receive = NULL; + break; + default: + break; + } + } + + } + + if (!proc_receive) { rp_error("cannot find hook 'proc-receive'"); return -1; } + + + argv[0] = proc_receive->command.buf; argv[1] = NULL; proc.argv = argv; diff --git a/t/t5411/test-0015-too-many-hooks-error.sh b/t/t5411/test-0015-too-many-hooks-error.sh new file mode 100644 index 0000000000..2d64534510 --- /dev/null +++ b/t/t5411/test-0015-too-many-hooks-error.sh @@ -0,0 +1,47 @@ +test_expect_success "setup too many proc-receive hooks (ok, $PROTOCOL)" ' + write_script "proc-receive" <<-EOF && + printf >&2 "# proc-receive hook\n" + test-tool proc-receive -v \ + -r "ok refs/for/main/topic" + EOF + + git -C "$upstream" config --add "hook.proc-receive.command" proc-receive && + cp proc-receive "$upstream/hooks/proc-receive" +' + +# Refs of upstream : main(A) +# Refs of workbench: main(A) tags/v123 +# git push : next(A) refs/for/main/topic(A) +test_expect_success "proc-receive: reject more than one configured hook" ' + test_must_fail git -C workbench push origin \ + HEAD:next \ + HEAD:refs/for/main/topic \ + >out 2>&1 && + make_user_friendly_and_stable_output actual && + cat >expect <<-EOF && + remote: # pre-receive hook + remote: pre-receive< refs/heads/next + remote: pre-receive< refs/for/main/topic + remote: error: only one "proc-receive" hook can be specified + remote: # post-receive hook + remote: post-receive< refs/heads/next + To + * [new branch] HEAD -> next + ! [remote rejected] HEAD -> refs/for/main/topic (fail to run proc-receive hook) + EOF + test_cmp expect actual && + git -C "$upstream" show-ref >out && + make_user_friendly_and_stable_output actual && + cat >expect <<-EOF && + refs/heads/main + refs/heads/next + EOF + test_cmp expect actual +' + +# Refs of upstream : main(A) next(A) +# Refs of workbench: main(A) tags/v123 +test_expect_success "cleanup ($PROTOCOL)" ' + git -C "$upstream" config --unset "hook.proc-receive.command" "proc-receive" && + git -C "$upstream" update-ref -d refs/heads/next +' From patchwork Thu May 27 00:08:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 12283051 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DE8D7C47089 for ; Thu, 27 May 2021 00:11:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BD926613CA for ; Thu, 27 May 2021 00:11:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235142AbhE0AMd (ORCPT ); Wed, 26 May 2021 20:12:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51576 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235141AbhE0ALr (ORCPT ); Wed, 26 May 2021 20:11:47 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9A2C7C06138D for ; Wed, 26 May 2021 17:10:01 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id a7-20020a5b00070000b02904ed415d9d84so3684578ybp.0 for ; Wed, 26 May 2021 17:10:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=zWVIvcSXtcuQD0eNgBPbl/bnb52CcZ1/tXzriKUku1c=; b=YjFZepef4Z0F3pL/Tzx/VHtZI4kWtRMijl7LdHKUg+Xi+NBwE9ToO3z17TRF/luw1d z5J8GrFPdohanVPt6L3bmIQeNkMOCKyYtoCnKs+utIQcBJwlcwJdu58/4jObb0IH/2nk wOEot143Bnk4BHpZhfVZpMES458o3H5I+06eZs9eBiO99rEhuWT3XuXrQg39tvBKqyte /Gmi9y5uu+8+ailfPTBIuqUO4dFBRw1cY1j+x36N3WgOXDi28EAcGVbIFjh/sPcBU4pr fjZIZ49yZX+xg4g7AFt96w19DC/EI7MJBlCvb7k0UZxV5luFBHiMQFGGlTYee+z4ndpS hRQg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=zWVIvcSXtcuQD0eNgBPbl/bnb52CcZ1/tXzriKUku1c=; b=UXsnVwiEBUqUKGw3aI/1l+5tOvbe4FxM/ZBZgigjpIgmYA3hoN+dv5ye7MzCHi/HF9 tKfO/Uo1XHTs9WFp5BoWDvCG6LjalmmA34RwgCz7ypFrQDiC69XSEKCxA1gw3YyzDUW4 NF9e3jSz/gugQYRcKyR21ZwL2YE+BoogdlBpiea+P6zyIsTPe2sOwgyKWYx8eXltaWyJ zNru3rC/SdeR6+vZ6J59TT4atzorzWRHTR8N/liKbbL0JQ6Um+/wZzPeZkWL77D8VOLB 6+Cyodi7w+04n74AG+CinxopGCx2Kq8OAdWJ7REcwZqz+yM6wR2wEwIQR2WFVOxSfVWZ ynVg== X-Gm-Message-State: AOAM533oz9Ho7YFIydJcF5JfSs8pQSXzWlpAQvQjhjAaGofE44KXWZuz Fh2jyaLame4nAOKkfPl8VgUszsW4Guo2Pqune6HgGOnK57/lKgEm+zV/suCdS/bG2rVirGged6E Q9ZuODhaLIo6t5GvHf3NuX7Uv//bKv6xRV1R6G5mYfF+M8ah3me35rE3ynu9/Uwm2wqhHb8k7JA == X-Google-Smtp-Source: ABdhPJzkxaLu0AJpZWFFTC6fsdwn/+fDmVhQNFWSfPhtvV+Aq0NOtO8lCwiHXd0YO1dn9PX7me/LSYVN8BUaLCrRvz4= X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:c1a4:4d87:8b5a:d12c]) (user=emilyshaffer job=sendgmr) by 2002:a25:bb85:: with SMTP id y5mr926680ybg.391.1622074200823; Wed, 26 May 2021 17:10:00 -0700 (PDT) Date: Wed, 26 May 2021 17:08:50 -0700 In-Reply-To: <20210527000856.695702-1-emilyshaffer@google.com> Message-Id: <20210527000856.695702-32-emilyshaffer@google.com> Mime-Version: 1.0 References: <20210527000856.695702-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v9 31/37] post-update: use hook.h library From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org By using run_hooks() instead of run_hook_le(), 'post-update' hooks can be specified in the config as well as the hookdir. Signed-off-by: Emily Shaffer --- Documentation/githooks.txt | 3 +++ builtin/receive-pack.c | 26 ++++++++------------------ 2 files changed, 11 insertions(+), 18 deletions(-) diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt index b6b13afb36..281970f910 100644 --- a/Documentation/githooks.txt +++ b/Documentation/githooks.txt @@ -508,6 +508,9 @@ Both standard output and standard error output are forwarded to `git send-pack` on the other end, so you can simply `echo` messages for the user. +Hooks run during 'post-update' will be run in parallel, unless hook.jobs is +configured to 1. + reference-transaction ~~~~~~~~~~~~~~~~~~~~~ diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c index 5ecc97d8ab..4236c0b268 100644 --- a/builtin/receive-pack.c +++ b/builtin/receive-pack.c @@ -1688,33 +1688,23 @@ static const char *update(struct command *cmd, struct shallow_info *si) static void run_update_post_hook(struct command *commands) { struct command *cmd; - struct child_process proc = CHILD_PROCESS_INIT; - const char *hook; + struct run_hooks_opt opt; - hook = find_hook("post-update"); - if (!hook) - return; + run_hooks_opt_init_async(&opt); for (cmd = commands; cmd; cmd = cmd->next) { if (cmd->error_string || cmd->did_not_exist) continue; - if (!proc.args.nr) - strvec_push(&proc.args, hook); - strvec_push(&proc.args, cmd->ref_name); + strvec_push(&opt.args, cmd->ref_name); } - if (!proc.args.nr) + if (!opt.args.nr) return; - proc.no_stdin = 1; - proc.stdout_to_stderr = 1; - proc.err = use_sideband ? -1 : 0; - proc.trace2_hook_name = "post-update"; + if (use_sideband) + opt.consume_sideband = hook_output_to_sideband; - if (!start_command(&proc)) { - if (use_sideband) - copy_to_sideband(proc.err, -1, NULL); - finish_command(&proc); - } + run_hooks("post-update", &opt); + run_hooks_opt_clear(&opt); } static void check_aliased_update_internal(struct command *cmd, From patchwork Thu May 27 00:08:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 12283053 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1ACE9C47088 for ; Thu, 27 May 2021 00:11:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id F03ED613BE for ; Thu, 27 May 2021 00:11:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235160AbhE0AMf (ORCPT ); Wed, 26 May 2021 20:12:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51646 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235156AbhE0AL7 (ORCPT ); Wed, 26 May 2021 20:11:59 -0400 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BEB78C061346 for ; Wed, 26 May 2021 17:10:03 -0700 (PDT) Received: by mail-yb1-xb4a.google.com with SMTP id o200-20020a2541d10000b0290526dd2d31easo3645593yba.4 for ; Wed, 26 May 2021 17:10:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=Yrt3a9RrtGoGN53/ihg3swfx0asc58AVk5hNuFowVEM=; b=kYVAEEaci1GpEMxmEjo1zB/zW7U6gLETZ+/Ngv+JOjhrfwOkd9zD9zDXW6etJxOJEH L64ge4d+OkXex6MzrfZHKn5FQfk05xggwWhYJERJkyzEBD8Ylu7NSCCkn1p1SIWQ6ps4 T/FydVXQ2HC0dfWjYNSivzm/1Cbd7e90blis/MF0SMJdXvzJEu2A5yUxCoyMnbG0Sh/w QLckhbYhzBjM008JT9hNW7uLLU81eV22VTS3y52LgbcFNudfjwyCzDNVitv6NpstMPQu Oi2LQ6/UfbsY4NkictqFl9jSrlZn+twnhSYaPnWQ7y3epyITlypKjAWpoSc9BHUOljkA N7cQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=Yrt3a9RrtGoGN53/ihg3swfx0asc58AVk5hNuFowVEM=; b=esrUHHwWWlrIA4zA3Nl/hdpVFioNCY7cIe7k7Q/ugX6+ULDPL4+/t5Jek6/mp9CxKe imQ345yTJh5Cbm956QXJ6rgPvv+ru9JEzp5La32TuknkUIfObh2IArJm1yekxlFXhCNT 7DMcrH2AcrJIgDmtLSqwzjANZihlqVNVyN7eArZ4rS26UJ0TJk3264zcOWb1WprI1jLv DKEvnjUKuBQmpTuvjQ19zlheIFPEVGw1XP79PYtCLPa5l5ykiwYF7E1w3ITMwIFg0WWm UddgLVFG7gJWAlA4Df7SuIMtQ4hk6dxA4a/PihOrZZdT0WWVNyC+ENrQx3Eb1oAX1Z6X Ss4Q== X-Gm-Message-State: AOAM533oy4HvZh/LalUWZRuTDrK4jykdfVIx/Yf9zR0GShJsiiGqGD2K uW2c966aO3qkby+ZUU7yQdz6vCaOyLCZ/vurki5GJoJH2LgKvxCRjrH4faJE9hXQlaWaShoHFWU id6iwJu1IcsWgwMbqt4waWmt26Zlyc9OV1zI8uxvn6zlkxG/Lte34PhTCOxPRb+dOoECPGX42Cg == X-Google-Smtp-Source: ABdhPJx1GXNJGhK+aFeA2vRo2UCKh8BnTdec0UamguAdM0Vxyyn6Za6h3yDkNLw3p45tOKvQxWhfBZs2I41IMJy5BKk= X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:c1a4:4d87:8b5a:d12c]) (user=emilyshaffer job=sendgmr) by 2002:a25:6f82:: with SMTP id k124mr842135ybc.489.1622074202889; Wed, 26 May 2021 17:10:02 -0700 (PDT) Date: Wed, 26 May 2021 17:08:51 -0700 In-Reply-To: <20210527000856.695702-1-emilyshaffer@google.com> Message-Id: <20210527000856.695702-33-emilyshaffer@google.com> Mime-Version: 1.0 References: <20210527000856.695702-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v9 32/37] receive-pack: convert receive hooks to hook.h From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org By using the hook.h library to run receive hooks, they can be specified in the config as well as in the hookdir. Signed-off-by: Emily Shaffer --- Documentation/githooks.txt | 5 + builtin/receive-pack.c | 199 +++++++++++++++++-------------------- 2 files changed, 97 insertions(+), 107 deletions(-) diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt index 281970f910..78cc4e0872 100644 --- a/Documentation/githooks.txt +++ b/Documentation/githooks.txt @@ -323,6 +323,8 @@ will be set to zero, `GIT_PUSH_OPTION_COUNT=0`. See the section on "Quarantine Environment" in linkgit:git-receive-pack[1] for some caveats. +Hooks executed during 'pre-receive' will not be parallelized. + [[update]] update ~~~~~~ @@ -476,6 +478,9 @@ environment variables will not be set. If the client selects to use push options, but doesn't transmit any, the count variable will be set to zero, `GIT_PUSH_OPTION_COUNT=0`. +Hooks executed during 'post-receive' are run in parallel, unless hook.jobs is +configured to 1. + [[post-update]] post-update ~~~~~~~~~~~ diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c index 4236c0b268..f44b58e456 100644 --- a/builtin/receive-pack.c +++ b/builtin/receive-pack.c @@ -748,7 +748,7 @@ static int check_cert_push_options(const struct string_list *push_options) return retval; } -static void prepare_push_cert_sha1(struct child_process *proc) +static void prepare_push_cert_sha1(struct run_hooks_opt *opt) { static int already_done; @@ -772,110 +772,42 @@ static void prepare_push_cert_sha1(struct child_process *proc) nonce_status = check_nonce(push_cert.buf, bogs); } if (!is_null_oid(&push_cert_oid)) { - strvec_pushf(&proc->env_array, "GIT_PUSH_CERT=%s", + strvec_pushf(&opt->env, "GIT_PUSH_CERT=%s", oid_to_hex(&push_cert_oid)); - strvec_pushf(&proc->env_array, "GIT_PUSH_CERT_SIGNER=%s", + strvec_pushf(&opt->env, "GIT_PUSH_CERT_SIGNER=%s", sigcheck.signer ? sigcheck.signer : ""); - strvec_pushf(&proc->env_array, "GIT_PUSH_CERT_KEY=%s", + strvec_pushf(&opt->env, "GIT_PUSH_CERT_KEY=%s", sigcheck.key ? sigcheck.key : ""); - strvec_pushf(&proc->env_array, "GIT_PUSH_CERT_STATUS=%c", + strvec_pushf(&opt->env, "GIT_PUSH_CERT_STATUS=%c", sigcheck.result); if (push_cert_nonce) { - strvec_pushf(&proc->env_array, + strvec_pushf(&opt->env, "GIT_PUSH_CERT_NONCE=%s", push_cert_nonce); - strvec_pushf(&proc->env_array, + strvec_pushf(&opt->env, "GIT_PUSH_CERT_NONCE_STATUS=%s", nonce_status); if (nonce_status == NONCE_SLOP) - strvec_pushf(&proc->env_array, + strvec_pushf(&opt->env, "GIT_PUSH_CERT_NONCE_SLOP=%ld", nonce_stamp_slop); } } } +struct receive_hook_feed_context { + struct command *cmd; + int skip_broken; +}; + struct receive_hook_feed_state { struct command *cmd; struct ref_push_report *report; int skip_broken; struct strbuf buf; - const struct string_list *push_options; }; -typedef int (*feed_fn)(void *, const char **, size_t *); -static int run_and_feed_hook(const char *hook_name, feed_fn feed, - struct receive_hook_feed_state *feed_state) -{ - struct child_process proc = CHILD_PROCESS_INIT; - struct async muxer; - const char *argv[2]; - int code; - - argv[0] = find_hook(hook_name); - if (!argv[0]) - return 0; - - argv[1] = NULL; - - proc.argv = argv; - proc.in = -1; - proc.stdout_to_stderr = 1; - proc.trace2_hook_name = hook_name; - - if (feed_state->push_options) { - int i; - for (i = 0; i < feed_state->push_options->nr; i++) - strvec_pushf(&proc.env_array, - "GIT_PUSH_OPTION_%d=%s", i, - feed_state->push_options->items[i].string); - strvec_pushf(&proc.env_array, "GIT_PUSH_OPTION_COUNT=%d", - feed_state->push_options->nr); - } else - strvec_pushf(&proc.env_array, "GIT_PUSH_OPTION_COUNT"); - - if (tmp_objdir) - strvec_pushv(&proc.env_array, tmp_objdir_env(tmp_objdir)); - - if (use_sideband) { - memset(&muxer, 0, sizeof(muxer)); - muxer.proc = copy_to_sideband; - muxer.in = -1; - code = start_async(&muxer); - if (code) - return code; - proc.err = muxer.in; - } - - prepare_push_cert_sha1(&proc); - - code = start_command(&proc); - if (code) { - if (use_sideband) - finish_async(&muxer); - return code; - } - - sigchain_push(SIGPIPE, SIG_IGN); - - while (1) { - const char *buf; - size_t n; - if (feed(feed_state, &buf, &n)) - break; - if (write_in_full(proc.in, buf, n) < 0) - break; - } - close(proc.in); - if (use_sideband) - finish_async(&muxer); - - sigchain_pop(SIGPIPE); - - return finish_command(&proc); -} - -static int feed_receive_hook(void *state_, const char **bufp, size_t *sizep) +static int feed_receive_hook(void *state_) { struct receive_hook_feed_state *state = state_; struct command *cmd = state->cmd; @@ -884,9 +816,7 @@ static int feed_receive_hook(void *state_, const char **bufp, size_t *sizep) state->skip_broken && (cmd->error_string || cmd->did_not_exist)) cmd = cmd->next; if (!cmd) - return -1; /* EOF */ - if (!bufp) - return 0; /* OK, can feed something. */ + return 1; /* EOF - close the pipe*/ strbuf_reset(&state->buf); if (!state->report) state->report = cmd->report; @@ -910,32 +840,36 @@ static int feed_receive_hook(void *state_, const char **bufp, size_t *sizep) cmd->ref_name); state->cmd = cmd->next; } - if (bufp) { - *bufp = state->buf.buf; - *sizep = state->buf.len; - } return 0; } -static int run_receive_hook(struct command *commands, - const char *hook_name, - int skip_broken, - const struct string_list *push_options) +static int feed_receive_hook_cb(struct strbuf *pipe, void *pp_cb, void *pp_task_cb) { - struct receive_hook_feed_state state; - int status; - - strbuf_init(&state.buf, 0); - state.cmd = commands; - state.skip_broken = skip_broken; - state.report = NULL; - if (feed_receive_hook(&state, NULL, NULL)) - return 0; - state.cmd = commands; - state.push_options = push_options; - status = run_and_feed_hook(hook_name, feed_receive_hook, &state); - strbuf_release(&state.buf); - return status; + struct hook *hook = pp_task_cb; + struct receive_hook_feed_state *feed_state = hook->feed_pipe_cb_data; + int rc; + + /* first-time setup */ + if (!feed_state) { + struct hook_cb_data *hook_cb = pp_cb; + struct run_hooks_opt *opt = hook_cb->options; + struct receive_hook_feed_context *ctx = opt->feed_pipe_ctx; + if (!ctx) + BUG("run_hooks_opt.feed_pipe_ctx required for receive hook"); + + feed_state = xmalloc(sizeof(struct receive_hook_feed_state)); + strbuf_init(&feed_state->buf, 0); + feed_state->cmd = ctx->cmd; + feed_state->skip_broken = ctx->skip_broken; + feed_state->report = NULL; + + hook->feed_pipe_cb_data = feed_state; + } + + rc = feed_receive_hook(feed_state); + if (!rc) + strbuf_addbuf(pipe, &feed_state->buf); + return rc; } static void hook_output_to_sideband(struct strbuf *output, void *cb_data) @@ -971,6 +905,57 @@ static void hook_output_to_sideband(struct strbuf *output, void *cb_data) send_sideband(1, 2, output->buf, output->len, use_sideband); } +static int run_receive_hook(struct command *commands, + const char *hook_name, + int skip_broken, + const struct string_list *push_options) +{ + struct run_hooks_opt opt; + struct receive_hook_feed_context ctx; + int rc; + struct command *iter = commands; + + run_hooks_opt_init_async(&opt); + + /* if there are no valid commands, don't invoke the hook at all. */ + while (iter && skip_broken && (iter->error_string || iter->did_not_exist)) + iter = iter->next; + if (!iter) + return 0; + + /* pre-receive hooks should run in series as the hook updates refs */ + if (!strcmp(hook_name, "pre-receive")) + opt.jobs = 1; + + if (push_options) { + int i; + for (i = 0; i < push_options->nr; i++) + strvec_pushf(&opt.env, "GIT_PUSH_OPTION_%d=%s", i, + push_options->items[i].string); + strvec_pushf(&opt.env, "GIT_PUSH_OPTION_COUNT=%d", push_options->nr); + } else + strvec_push(&opt.env, "GIT_PUSH_OPTION_COUNT"); + + if (tmp_objdir) + strvec_pushv(&opt.env, tmp_objdir_env(tmp_objdir)); + + prepare_push_cert_sha1(&opt); + + /* set up sideband printer */ + if (use_sideband) + opt.consume_sideband = hook_output_to_sideband; + + /* set up stdin callback */ + ctx.cmd = commands; + ctx.skip_broken = skip_broken; + opt.feed_pipe = feed_receive_hook_cb; + opt.feed_pipe_ctx = &ctx; + + rc = run_hooks(hook_name, &opt); + run_hooks_opt_clear(&opt); + return rc; +} + static int run_update_hook(struct command *cmd) { struct run_hooks_opt opt; From patchwork Thu May 27 00:08:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 12283055 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A54ECC47082 for ; Thu, 27 May 2021 00:11:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8B55F613D2 for ; Thu, 27 May 2021 00:11:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235060AbhE0AMq (ORCPT ); Wed, 26 May 2021 20:12:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51568 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235176AbhE0AMN (ORCPT ); Wed, 26 May 2021 20:12:13 -0400 Received: from mail-qt1-x849.google.com (mail-qt1-x849.google.com [IPv6:2607:f8b0:4864:20::849]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E6BA2C061358 for ; Wed, 26 May 2021 17:10:05 -0700 (PDT) Received: by mail-qt1-x849.google.com with SMTP id b17-20020ac854110000b02901f279c73d75so1678671qtq.2 for ; Wed, 26 May 2021 17:10:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=dWpw275AjK5+uL0aEQ4VjUUzAAQyBB8ZTrC0oh2O7nE=; b=UbqG+GxiTOo+It1iXxgZKvX8SScOwvxUUZwKTvKjgZqZX+SWi8dMUNP/X5Uf+1nrbl 2T4qzbBHb2m1GzFTnHqopDZoPBvxqto9vO84p+TKdPpyPaE9lyTyCwwNHMuGIGKY54Ha NIR0d3wdiGNka/9Tf8iK3+nSwrdJ4nkLF88ShJscGLwrKsx0qalagcNv3Adugq13WYBF ZS4ICtdIa/ZRZceMt9TWhs5iK11IqYsvMKVTU9CVA7yhNtPJvdbdeZWVEm68kxsP7alD deiGoba7wiG0rbiE0Xj2go/fqK2aC+oAKY2uJqv2heOLcdHDWRJIWXoW/zh/cgqQTwhs 34iA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=dWpw275AjK5+uL0aEQ4VjUUzAAQyBB8ZTrC0oh2O7nE=; b=LXeKuzoJmp2Z6CviWC025vuu7XUzG6UX3nawHgo2Ia47l/ORAUb/jxdjCQ4GenrK4F 7SfoX2bFLnffHY63H1W7oWzTyftR7hWyfajoxBsCdVfPgUQZ1vXO27zANCj1slXg9rLL M6UASp30/mE6ToFrqrBewvzgdptq/iCK88AW0pikVeBETxIp7pgfzDCn4Mm9Kf/8mdXr qYsr4b/tBdU5Qyp66EisLwC+E4NtLEW26jdsxR3E6m+RLxx+DewMbHNr/FQYDp5gF89g LAYLRLbpNSzVRlYApV5BfdcgFOQML96uni4vREho7zoEw5qmz6C5tzeUfzcEYsBJth1D XzvQ== X-Gm-Message-State: AOAM532T22At/0dJRRmG+BjASW9U/9a9VhXdOAqsq5RItfJeWkUdNzqt 4YYOmYvOMp3oN1PT20NePGv31APOhf6Fndj86ZGLCArcUxtnGULTmXHOitM6LRueaj7dgc8qJna Tj4HBuwzk5L+QtjJ0+YOM/lM0KzRRHanezeX4+gUNMpWYGcblQwJxOx0GbUr8SZ1uI4jmHPUf1g == X-Google-Smtp-Source: ABdhPJwYVULhj1mA/JGcpMjpJUJ3M3iDeOvc8XxPfjoP4RmoBNL43yp4buuJDJVEmPacU0gg7oT6JxJftaR/HJ0iUxM= X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:c1a4:4d87:8b5a:d12c]) (user=emilyshaffer job=sendgmr) by 2002:a0c:9d0e:: with SMTP id m14mr836637qvf.50.1622074204662; Wed, 26 May 2021 17:10:04 -0700 (PDT) Date: Wed, 26 May 2021 17:08:52 -0700 In-Reply-To: <20210527000856.695702-1-emilyshaffer@google.com> Message-Id: <20210527000856.695702-34-emilyshaffer@google.com> Mime-Version: 1.0 References: <20210527000856.695702-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v9 33/37] bugreport: use hook_exists instead of find_hook From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org By using the helper in hook.h instead of the one in run-command.h, we can also check whether a hook exists in the config - not just whether it exists in the hookdir. Signed-off-by: Emily Shaffer --- builtin/bugreport.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/builtin/bugreport.c b/builtin/bugreport.c index 9915a5841d..190272ba70 100644 --- a/builtin/bugreport.c +++ b/builtin/bugreport.c @@ -3,7 +3,7 @@ #include "strbuf.h" #include "help.h" #include "compat/compiler.h" -#include "run-command.h" +#include "hook.h" static void get_system_info(struct strbuf *sys_info) @@ -82,7 +82,7 @@ static void get_populated_hooks(struct strbuf *hook_info, int nongit) } for (i = 0; i < ARRAY_SIZE(hook); i++) - if (find_hook(hook[i])) + if (hook_exists(hook[i], HOOKDIR_USE_CONFIG)) strbuf_addf(hook_info, "%s\n", hook[i]); } From patchwork Thu May 27 00:08:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 12283057 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7AC7CC47088 for ; Thu, 27 May 2021 00:11:20 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5EBEB613BE for ; Thu, 27 May 2021 00:11:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234435AbhE0AMs (ORCPT ); Wed, 26 May 2021 20:12:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51668 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235056AbhE0AMU (ORCPT ); Wed, 26 May 2021 20:12:20 -0400 Received: from mail-qv1-xf49.google.com (mail-qv1-xf49.google.com [IPv6:2607:f8b0:4864:20::f49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6A263C06135B for ; Wed, 26 May 2021 17:10:07 -0700 (PDT) Received: by mail-qv1-xf49.google.com with SMTP id n12-20020a0cdc8c0000b02901efdf8d3bc7so2534047qvk.23 for ; Wed, 26 May 2021 17:10:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=TJJkj6w7H78AOnljLVFLdOFVsu/TQvCzoH/8PvUwvg8=; b=TMtaV7YaeFAvqwxuC7mR4Nae20UTI6uYi0LfP+t03EaJXWcfU7G7xM3o1+/qc4jWlZ r7OvX01v7jFomjR8I7p43DEetqtOi7Oxv0MyOs9VAQgDNVhb4acCtmNXbhnX6DPQFTtd a0Mu4vhTXtE9RlAZVMEt7cnsVeWXCi20d9ML+dOW2LCpm1KqyIRqQ4Ee0Y6bTPXXYEzG ZPLUz78OSR1YIaHK+HiO/NjRwp9eEzGDIMv53sa95DNciyWo2CcTGWlSqF2T0BpqGW7r mipTfWmmJptH5E9f4tCTfBNbk+BW33AxNDu+beCdLLJpDz6FPMxS7lNL+dcjnY96vaYD CP/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=TJJkj6w7H78AOnljLVFLdOFVsu/TQvCzoH/8PvUwvg8=; b=EwvoVPJCrnqD8tqh9Ti+Va6PCKG09IZZ7o+fAk92RnRiOw7+ZDj/FvFo5OtLktF2g1 MoTrQ3HQ39uTe5WX3+QM+y8sDFHJtaf1LY9x0owkHFug7YbZWThkF1qnEmwWRuIeySfM b1SAAB5ir8ly98S/MdOsTkCttFA1CTmRKFiP2IeAd0A8P/PuWxeYfPhw1sAV1KdRMhPT Itq2XDVtaCDhuRdhc10LgPIaeO//eitjhqtcieZNdL7kc2i0g/QrOJC7l4KJxd8fGCxm mNmG8D386FgmJQ2gTmBiWsTJIH/Szw9Iwe/s86Dbp6SyMjksg2c4H/SjBDGDdSANvdT/ FqgQ== X-Gm-Message-State: AOAM53033buAP3we9n+dKmyd2ZFCt52EoOOB5y11Xdz2iqELD6ytnnu0 5Tf13DhBnSEF374m8AlYZsrd889WCiQj52dAM5ro1li+iGo/vNqYsHC2V75/SfxKsL/gqcHhac2 3riVtnyn9uOSxPP7kYJ0eybeN2SShnYB5ffsPwi47gpYW2thGmOpESx2donwuzBk6AA28PMz9Cw == X-Google-Smtp-Source: ABdhPJzSY0cft/mxodMbWE0nzdpQqCBYSo7W17wc2RBZeGuK18Nm2rGXFrCSYfa4wsFdOfCTHtB4HDQnNlM3UL7Y3nM= X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:c1a4:4d87:8b5a:d12c]) (user=emilyshaffer job=sendgmr) by 2002:ad4:4a83:: with SMTP id h3mr815682qvx.19.1622074206448; Wed, 26 May 2021 17:10:06 -0700 (PDT) Date: Wed, 26 May 2021 17:08:53 -0700 In-Reply-To: <20210527000856.695702-1-emilyshaffer@google.com> Message-Id: <20210527000856.695702-35-emilyshaffer@google.com> Mime-Version: 1.0 References: <20210527000856.695702-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v9 34/37] git-send-email: use 'git hook run' for 'sendemail-validate' From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org By using the new 'git hook run' subcommand to run 'sendemail-validate', we can reduce the boilerplate needed to run this hook in perl. Using config-based hooks also allows us to run 'sendemail-validate' hooks that were configured globally when running 'git send-email' from outside of a Git directory, alongside other benefits like multihooks and parallelization. Signed-off-by: Emily Shaffer --- git-send-email.perl | 26 ++++++-------------------- t/t9001-send-email.sh | 13 +------------ 2 files changed, 7 insertions(+), 32 deletions(-) diff --git a/git-send-email.perl b/git-send-email.perl index 170f42fe21..b55687453e 100755 --- a/git-send-email.perl +++ b/git-send-email.perl @@ -1958,26 +1958,12 @@ sub unique_email_list { sub validate_patch { my ($fn, $xfer_encoding) = @_; - if ($repo) { - my $validate_hook = catfile($repo->hooks_path(), - 'sendemail-validate'); - my $hook_error; - if (-x $validate_hook) { - my $target = abs_path($fn); - # The hook needs a correct cwd and GIT_DIR. - my $cwd_save = cwd(); - chdir($repo->wc_path() or $repo->repo_path()) - or die("chdir: $!"); - local $ENV{"GIT_DIR"} = $repo->repo_path(); - $hook_error = system_or_msg([$validate_hook, $target]); - chdir($cwd_save) or die("chdir: $!"); - } - if ($hook_error) { - die sprintf(__("fatal: %s: rejected by sendemail-validate hook\n" . - "%s\n" . - "warning: no patches were sent\n"), $fn, $hook_error); - } - } + my $target = abs_path($fn); + + system_or_die(["git", "hook", "run", "sendemail-validate", "-j1", "-a", $target], + sprintf(__("fatal: %s: rejected by sendemail-validate hook\n" . + "warning: no patches were sent\n"), + $fn)); # Any long lines will be automatically fixed if we use a suitable transfer # encoding. diff --git a/t/t9001-send-email.sh b/t/t9001-send-email.sh index aa603cf4d0..bdf6472871 100755 --- a/t/t9001-send-email.sh +++ b/t/t9001-send-email.sh @@ -539,7 +539,6 @@ test_expect_success $PREREQ "--validate respects relative core.hooksPath path" ' test_path_is_file my-hooks.ran && cat >expect <<-EOF && fatal: longline.patch: rejected by sendemail-validate hook - fatal: command '"'"'$PWD/my-hooks/sendemail-validate'"'"' died with exit code 1 warning: no patches were sent EOF test_cmp expect actual @@ -557,7 +556,6 @@ test_expect_success $PREREQ "--validate respects absolute core.hooksPath path" ' test_path_is_file my-hooks.ran && cat >expect <<-EOF && fatal: longline.patch: rejected by sendemail-validate hook - fatal: command '"'"'$PWD/my-hooks/sendemail-validate'"'"' died with exit code 1 warning: no patches were sent EOF test_cmp expect actual @@ -2170,16 +2168,7 @@ test_expect_success $PREREQ 'invoke hook' ' mkdir -p .git/hooks && write_script .git/hooks/sendemail-validate <<-\EOF && - # test that we have the correct environment variable, pwd, and - # argument - case "$GIT_DIR" in - *.git) - true - ;; - *) - false - ;; - esac && + # test that we have the correct argument test -f 0001-add-main.patch && grep "add main" "$1" EOF From patchwork Thu May 27 00:08:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 12283059 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 78F6EC47088 for ; Thu, 27 May 2021 00:11:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 576FC6128D for ; Thu, 27 May 2021 00:11:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234836AbhE0AMy (ORCPT ); Wed, 26 May 2021 20:12:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51582 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234704AbhE0AMZ (ORCPT ); Wed, 26 May 2021 20:12:25 -0400 Received: from mail-qk1-x749.google.com (mail-qk1-x749.google.com [IPv6:2607:f8b0:4864:20::749]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2FA0FC061361 for ; Wed, 26 May 2021 17:10:09 -0700 (PDT) Received: by mail-qk1-x749.google.com with SMTP id o14-20020a05620a130eb02902ea53a6ef80so2099262qkj.6 for ; Wed, 26 May 2021 17:10:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=mQthgHLAeS25SjBjBAWzGs1a3kEccUbm8XzgLq4xxuM=; b=SW0Btnt3epyS2GuM8BotnKs+dVhh9IXUAG0yPC/6tqK6z1uJCI7wjrsw1UeINI0tUZ 4MWvk+rRa4YwbCQfRBTzkrsnyezzy8krGQ1KHGTwD8xz4k4tQUfK7u9/BUbBhCoiGHWm giV5V5xeIKCRlIau0p42NTghbr8LT5S7diwSvBmTE12dP532pPGTRQQfTll3BBPUDPQ4 aD9BjKJ2UIY0/7HiuPTLQKFfH5hyuyQX4gTtLxd7JYAsrj1bRaRzcrlDV0wGJrSQtE3l dkF0bJPdliFko3Wz/UuI4lV8hTxUeF1j5h/S9l76gz8cy70jvarsXsGkPWg0AWpLrX8M Hy3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=mQthgHLAeS25SjBjBAWzGs1a3kEccUbm8XzgLq4xxuM=; b=aXuHB7cfgyb2P0Z0NYDqiOqLK4Rbw/kBsdfqIIotXDwOKs+lMVuijqN/Z559WHK5p0 NWyS/tAU7sUS8SK8QmphdqbTFS1zxO9NHSXLGuVHtwBB2AgN04TyslsQPdKUIF8PYNdp SWuVYIcpDA5+ocOjLGdVU7KVwYmSAmxA088vkWsusGsl3JFN0IWrc0gucPQ69lo4UKRQ eWYGpboVsbIRBait3nLlsJpWogS3Xxgq4RtNxrhNm9RsRzs5dsjDaodi4CXKg49pNviN uRUOdhjo4GTYMYD2ympAhqmRBpyp7H5j6crm7y77FVhUMX/Gol9L+XN9cifl6GbVAKup 1+yQ== X-Gm-Message-State: AOAM533/i9Ol2P0+GZzuU5y6Y0GT3zY3C/nDADeNNU7wjH3zFy5/ekZF lbjd+0GIx76r4WvfFRIr807mgmnQ/XYQVZ+Tv/OU4AFX6Pym7a54yr04BjXYv5wrojDiqu1Bx1l d/N7w+cWmXynjuNvlg/azrWepGHP8GL7M3Fags6VoKlHK7GhakbH6bMPFlWLP7NmT8uQ5YNe48g == X-Google-Smtp-Source: ABdhPJzA3wbDCmY7xc72RlEp/VDq+J01qDQU8JiPpnhIIt9PGSqqsq30+EoitG2FjNY+xmC3jHy2v/E+AYNOGDZK6uY= X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:c1a4:4d87:8b5a:d12c]) (user=emilyshaffer job=sendgmr) by 2002:a05:6214:11a3:: with SMTP id u3mr1055931qvv.6.1622074208190; Wed, 26 May 2021 17:10:08 -0700 (PDT) Date: Wed, 26 May 2021 17:08:54 -0700 In-Reply-To: <20210527000856.695702-1-emilyshaffer@google.com> Message-Id: <20210527000856.695702-36-emilyshaffer@google.com> Mime-Version: 1.0 References: <20210527000856.695702-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v9 35/37] run-command: stop thinking about hooks From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org hook.h has replaced all run-command.h hook-related functionality. run-command.h:run_hooks_le/ve and find_hook are no longer used anywhere in the codebase. So, let's delete the dead code - or, in the one case where it's still needed, move it to an internal function in hook.c. Signed-off-by: Emily Shaffer --- hook.c | 39 ++++++++++++++++++++++++++++-- run-command.c | 66 --------------------------------------------------- run-command.h | 24 ------------------- 3 files changed, 37 insertions(+), 92 deletions(-) diff --git a/hook.c b/hook.c index 27da1fdb32..ff80e52edd 100644 --- a/hook.c +++ b/hook.c @@ -218,6 +218,41 @@ static int should_include_hookdir(const char *path, enum hookdir_opt cfg) } } +static const char *find_legacy_hook(const char *name) +{ + static struct strbuf path = STRBUF_INIT; + + strbuf_reset(&path); + strbuf_git_path(&path, "hooks/%s", name); + if (access(path.buf, X_OK) < 0) { + int err = errno; + +#ifdef STRIP_EXTENSION + strbuf_addstr(&path, STRIP_EXTENSION); + if (access(path.buf, X_OK) >= 0) + return path.buf; + if (errno == EACCES) + err = errno; +#endif + + if (err == EACCES && advice_ignored_hook) { + static struct string_list advise_given = STRING_LIST_INIT_DUP; + + if (!string_list_lookup(&advise_given, name)) { + string_list_insert(&advise_given, name); + advise(_("The '%s' hook was ignored because " + "it's not set as executable.\n" + "You can disable this warning with " + "`git config advice.ignoredHook false`."), + path.buf); + } + } + return NULL; + } + return path.buf; +} + + struct list_head* hook_list(const char* hookname) { struct strbuf hook_key = STRBUF_INIT; @@ -234,7 +269,7 @@ struct list_head* hook_list(const char* hookname) git_config(hook_config_lookup, &cb_data); if (have_git_dir()) { - const char *legacy_hook_path = find_hook(hookname); + const char *legacy_hook_path = find_legacy_hook(hookname); /* Unconditionally add legacy hook, but annotate it. */ if (legacy_hook_path) { @@ -283,7 +318,7 @@ int hook_exists(const char *hookname, enum hookdir_opt should_run_hookdir) could_run_hookdir = (should_run_hookdir == HOOKDIR_INTERACTIVE || should_run_hookdir == HOOKDIR_WARN || should_run_hookdir == HOOKDIR_YES) - && !!find_hook(hookname); + && !!find_legacy_hook(hookname); strbuf_addf(&hook_key, "hook.%s.command", hookname); diff --git a/run-command.c b/run-command.c index 27135defb8..2ff76f3c2f 100644 --- a/run-command.c +++ b/run-command.c @@ -1320,72 +1320,6 @@ int async_with_fork(void) #endif } -const char *find_hook(const char *name) -{ - static struct strbuf path = STRBUF_INIT; - - strbuf_reset(&path); - strbuf_git_path(&path, "hooks/%s", name); - if (access(path.buf, X_OK) < 0) { - int err = errno; - -#ifdef STRIP_EXTENSION - strbuf_addstr(&path, STRIP_EXTENSION); - if (access(path.buf, X_OK) >= 0) - return path.buf; - if (errno == EACCES) - err = errno; -#endif - - if (err == EACCES && advice_ignored_hook) { - static struct string_list advise_given = STRING_LIST_INIT_DUP; - - if (!string_list_lookup(&advise_given, name)) { - string_list_insert(&advise_given, name); - advise(_("The '%s' hook was ignored because " - "it's not set as executable.\n" - "You can disable this warning with " - "`git config advice.ignoredHook false`."), - path.buf); - } - } - return NULL; - } - return path.buf; -} - -int run_hook_ve(const char *const *env, const char *name, va_list args) -{ - struct child_process hook = CHILD_PROCESS_INIT; - const char *p; - - p = find_hook(name); - if (!p) - return 0; - - strvec_push(&hook.args, p); - while ((p = va_arg(args, const char *))) - strvec_push(&hook.args, p); - hook.env = env; - hook.no_stdin = 1; - hook.stdout_to_stderr = 1; - hook.trace2_hook_name = name; - - return run_command(&hook); -} - -int run_hook_le(const char *const *env, const char *name, ...) -{ - va_list args; - int ret; - - va_start(args, name); - ret = run_hook_ve(env, name, args); - va_end(args); - - return ret; -} - struct io_pump { /* initialized by caller */ int fd; diff --git a/run-command.h b/run-command.h index ebc4a95a94..7150da851a 100644 --- a/run-command.h +++ b/run-command.h @@ -201,30 +201,6 @@ int finish_command_in_signal(struct child_process *); */ int run_command(struct child_process *); -/* - * Returns the path to the hook file, or NULL if the hook is missing - * or disabled. Note that this points to static storage that will be - * overwritten by further calls to find_hook and run_hook_*. - */ -const char *find_hook(const char *name); - -/** - * Run a hook. - * The first argument is a pathname to an index file, or NULL - * if the hook uses the default index file or no index is needed. - * The second argument is the name of the hook. - * The further arguments correspond to the hook arguments. - * The last argument has to be NULL to terminate the arguments list. - * If the hook does not exist or is not executable, the return - * value will be zero. - * If it is executable, the hook will be executed and the exit - * status of the hook is returned. - * On execution, .stdout_to_stderr and .no_stdin will be set. - */ -LAST_ARG_MUST_BE_NULL -int run_hook_le(const char *const *env, const char *name, ...); -int run_hook_ve(const char *const *env, const char *name, va_list args); - /* * Trigger an auto-gc */ From patchwork Thu May 27 00:08:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 12283061 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 46500C47089 for ; Thu, 27 May 2021 00:11:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2ADCC613CA for ; Thu, 27 May 2021 00:11:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235181AbhE0AM6 (ORCPT ); Wed, 26 May 2021 20:12:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51712 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234870AbhE0AM0 (ORCPT ); Wed, 26 May 2021 20:12:26 -0400 Received: from mail-qk1-x749.google.com (mail-qk1-x749.google.com [IPv6:2607:f8b0:4864:20::749]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 165B4C061362 for ; Wed, 26 May 2021 17:10:11 -0700 (PDT) Received: by mail-qk1-x749.google.com with SMTP id e8-20020a05620a2088b02903a5edeec4d6so2082731qka.11 for ; Wed, 26 May 2021 17:10:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=PiFe8ZbPARTa1BwN5ccCzizd3frBbn/m1Eorw8fljGc=; b=F3g5LVe+B4y4Oh8jfPadohJJg08EoVWB0VxvJ/YUpZw7U+0owhv84Dho6OXOl2WMsJ ZaWd0WUjac4cykVJMOm3pUtg7z832dAEdvBFPEa6zKqfZIVM1KOBsFAyKeoECFgi53uY 5j/RDvuYOxMxZ+rOLDrR9eEOj00GNcC/xFkXZJMEDmP0ghfN1rS3nj3RpBfvWBHWFYIc HbXo7B4yswjIVyOP9jIx6eE3/Jh9Dg9Y4QBHwf9IE4M73wpmZkwTyWAayRKZhZm/z1zw xOJqTc6GKiX4JDd6E27+BYPZprhqZh3VyLQeLaSr6PYuW878q9mV7X7ljlaUaY0kl+KN 3dKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=PiFe8ZbPARTa1BwN5ccCzizd3frBbn/m1Eorw8fljGc=; b=BP4TmAPqiApFjMQ8Is4n84I1zPVWaOW8Ph3mQtZTHTFMUocgzxidpB/XYZTCfv+mSP WE/JI/aVhspMIiW9+/OGC/wFXFv18/Mr8khNkqGsWduQmHpNMNLEwesaoAR3aV/+Rf0P qAR8jSP0V61rEJ7A200nTxKKBAO/YDs7s/jY/X5cP/Kar/0ZeuUyjcz2WDjeRS6bkgP7 gQK36RcsRSa5U821z7YGHlJHUOSqKIqNtbSqThBOalzgI2na/dX0hxvLd+/ltOIm7c6C UKB70Y0/y6/PAc/QBXr4f/uKznqVYIp92BjYH8Wvpt2+xEHx5jp+RMBhWMZ+h5PGQhPF G1rA== X-Gm-Message-State: AOAM530LWQAIIBy8hPrlaCNbNBzaEWN0i3yo0LVBrHI2Ek1qc6Y9vNu1 g72aFh5NzFZunoQw4K3slAb0XvtpqqUQcDgGpdYL4hibtlDi0xaFxb2Imx+IHMgYAzW7cVlXYhS zXiLcON2l151cEMr2t/8kaqSQBI9w+eaRdUCdc4Q1jtMJJZkxHAOJEpsMm48OJUMLmaxwPE3p8w == X-Google-Smtp-Source: ABdhPJzrtJvXldEtNsLkmF04QQWU2lWzDJog277Fk28U7JGpH9dPpq7FPneCKjhgGIkJqzl4fS9SWkT9AEvXKEWXlLA= X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:c1a4:4d87:8b5a:d12c]) (user=emilyshaffer job=sendgmr) by 2002:a05:6214:bce:: with SMTP id ff14mr1040927qvb.1.1622074210127; Wed, 26 May 2021 17:10:10 -0700 (PDT) Date: Wed, 26 May 2021 17:08:55 -0700 In-Reply-To: <20210527000856.695702-1-emilyshaffer@google.com> Message-Id: <20210527000856.695702-37-emilyshaffer@google.com> Mime-Version: 1.0 References: <20210527000856.695702-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v9 36/37] doc: clarify fsmonitor-watchman specification From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org fsmonitor-watchman is not specified in the same way that other hooks are. In fsmonitor.c:query_fsmonitor(), the path stored in 'core_fsmonitor' is executed directly via run-command.h:capture_command(). 'core_fsmonitor' is set during 'git update-index' via config.c:git_config_get_fsmonitor(). Neither builtin/update-index.c, nor config.c, nor fsmonitor.c check that the path given is in '.git/hooks'. None of the existing hook execution code is used by fsmonitor.c to invoke fsmonitor-watchman, because that executable isn't expected to reside in '.git/hooks'. Furthermore, it doesn't make sense to specify the fsmonitor-watchman hook more than once, and that hook itself may soon be superseded by native logic to talk to an fsmonitor daemon directly from the Git executable. (See for more information.) Therefore, let's correct the user-facing documentation around fsmonitor-watchman and clarify that it won't be supported by config-based hooks. Signed-off-by: Emily Shaffer --- Documentation/githooks.txt | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt index 78cc4e0872..42e66d4e2d 100644 --- a/Documentation/githooks.txt +++ b/Documentation/githooks.txt @@ -649,9 +649,12 @@ e-mails. fsmonitor-watchman ~~~~~~~~~~~~~~~~~~ -This hook is invoked when the configuration option `core.fsmonitor` is -set to `.git/hooks/fsmonitor-watchman` or `.git/hooks/fsmonitor-watchmanv2` -depending on the version of the hook to use. +This hook is invoked when the configuration option `core.fsmonitor` is set to a +path containing an executable. It *cannot* be specified via the usual +`hook.fsmonitor-watchman.command` configuration or by providing an executable +in `.git/hooks/fsmonitor-watchman`. The arguments provided to the hook are +determined by the value of the `core.fsmonitorHookVersion` configuration +option. Version 1 takes two arguments, a version (1) and the time in elapsed nanoseconds since midnight, January 1, 1970. From patchwork Thu May 27 00:08:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Emily Shaffer X-Patchwork-Id: 12283063 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A4854C47088 for ; Thu, 27 May 2021 00:11:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8A8AC613BE for ; Thu, 27 May 2021 00:11:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234699AbhE0AND (ORCPT ); Wed, 26 May 2021 20:13:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51618 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235088AbhE0AMb (ORCPT ); Wed, 26 May 2021 20:12:31 -0400 Received: from mail-qt1-x849.google.com (mail-qt1-x849.google.com [IPv6:2607:f8b0:4864:20::849]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D7C79C061364 for ; Wed, 26 May 2021 17:10:12 -0700 (PDT) Received: by mail-qt1-x849.google.com with SMTP id h12-20020ac8776c0000b02901f1228fdb1bso1668014qtu.6 for ; Wed, 26 May 2021 17:10:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=9jaoNTk9pxC7NibgJC3A3I0yeBCl7VuxCgMuBBolul0=; b=j/J/PonNn9ezcKJZGFF3kloADiv5B0huOeFYhS6wJEso4yb1eJTvRXC2NIteW3Zmkc upepo53BGYXNG2PQclCvRu7X7SEcTnABPoR8/EFKiy3Cu7zEMcEDQEGdOmfIlorqe76f bEl7+JhXaagPxtN+kW6Fc6jwoGHFG3UveNsuuvaJvq3bwAGpBADFBjxX8BO/IeGku4cV /OJzz6aZFqo20xGN5fhVUYWlmPUWgdVdydqZ546X4uxBFu5Yd3fvUrvYzFs0pOM1DqHO fP0ROJBOBNVRSC/DQBnj1QTzss6G5OGr3YRdbMpZhyo2+fZMqFI+Eg78gd9kXMp5UG+a PnXQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=9jaoNTk9pxC7NibgJC3A3I0yeBCl7VuxCgMuBBolul0=; b=oPeILXchPMdRTuOpE8miS7WOkonhfWe9axdYYKaQV4q2GiYhTSNByVtfNl8LH//huD PsJb51CgA6jWm36I0de65alHwCCPdgrR3rf2JEwnGxkd5FrZ8d0ePjUp9DKFp5lwMwVI S5sDTGlq6nTYNoCPbp4AKxP9btXZNrTBSr9L+C2Sfn6LotHAovwUtvwpZqPikUo4mhOi RA09Zk6YHdN6eYwYIpp5mUPMcDa8RDlW/38dX/MugbfDCOTF21/nIMjijJM6XTWp6HAB 5vs3q7hsfzOwZLjh11F5uXoZOTwaG4RPdQ9lsuH6rLasBTeT2//Tii6XhgPQjsDkZweU ZGmQ== X-Gm-Message-State: AOAM5314W35UeGFYUnDAl91fmbwJnwvWXyOsFktSXqUU2HF9JaiHAMx8 7BYksI1qQ1abmodBLN6Dg7VOZaryXO+6bwBxTBOXygw75V4z8WS5t2axT93yJm4tKImhZAp4R4j tk/XnjJUfmQxANdGjWlnGrvgftPPB4UAdlEWQy0tHQsmD+RbCqUhDV87AbmbwRyMQJeTlOmdXng == X-Google-Smtp-Source: ABdhPJxvtodiNvY/QmpNMJgAno/HcomOwqDO8y8Vj9D1oCrNJZsPcIWLJU5Q4tJ7YjXY/LpQlcwYyzUlcrFbJdKe6Ic= X-Received: from podkayne.svl.corp.google.com ([2620:15c:2ce:0:c1a4:4d87:8b5a:d12c]) (user=emilyshaffer job=sendgmr) by 2002:a0c:8e4c:: with SMTP id w12mr816661qvb.3.1622074211945; Wed, 26 May 2021 17:10:11 -0700 (PDT) Date: Wed, 26 May 2021 17:08:56 -0700 In-Reply-To: <20210527000856.695702-1-emilyshaffer@google.com> Message-Id: <20210527000856.695702-38-emilyshaffer@google.com> Mime-Version: 1.0 References: <20210527000856.695702-1-emilyshaffer@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v9 37/37] docs: link githooks and git-hook manpages From: Emily Shaffer To: git@vger.kernel.org Cc: Emily Shaffer Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Since users may have an easier time finding 'man githooks' or 'git help githooks' through tab-completion or muscle memory, reference the 'git hook' commands. And in the 'git hook' manual, point users back to 'man githooks' for specifics about the hook events themselves. Signed-off-by: Emily Shaffer --- Documentation/git-hook.txt | 12 ++++++++++++ Documentation/githooks.txt | 7 ++++--- 2 files changed, 16 insertions(+), 3 deletions(-) diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt index 81b8e94994..24e00a6f4a 100644 --- a/Documentation/git-hook.txt +++ b/Documentation/git-hook.txt @@ -17,6 +17,13 @@ DESCRIPTION You can list and run configured hooks with this command. Later, you will be able to add and modify hooks with this command. +In general, when instructions suggest adding a script to +`.git/hooks/`, you can specify it in the config instead by running +`git config --add hook..command ` - this way you can +share the script between multiple repos. That is, `cp ~/my-script.sh +~/project/.git/hooks/pre-commit` would become `git config --add +hook.pre-commit.command ~/my-script.sh`. + This command parses the default configuration files for sections `hook` and `hookcmd`. `hook` is used to describe the commands which will be run during a particular hook event; commands are run in the order Git encounters them during @@ -141,6 +148,11 @@ number of CPUs on the current system. Some hooks may be ineligible for parallelization: for example, 'commit-msg' intends hooks modify the commit message body and cannot be parallelized. +HOOKS +----- +For a list of hooks which can be configured and how they work, see +linkgit:githooks[5]. + CONFIGURATION ------------- include::config/hook.txt[] diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt index 42e66d4e2d..d780cb3b18 100644 --- a/Documentation/githooks.txt +++ b/Documentation/githooks.txt @@ -7,15 +7,16 @@ githooks - Hooks used by Git SYNOPSIS -------- +'git hook' $GIT_DIR/hooks/* (or \`git config core.hooksPath`/*) DESCRIPTION ----------- -Hooks are programs you can place in a hooks directory to trigger -actions at certain points in git's execution. Hooks that don't have -the executable bit set are ignored. +Hooks are programs you can specify in your config (see linkgit:git-hook[1]) or +place in a hooks directory to trigger actions at certain points in git's +execution. Hooks that don't have the executable bit set are ignored. By default the hooks directory is `$GIT_DIR/hooks`, but that can be changed via the `core.hooksPath` configuration variable (see