From patchwork Tue Aug 25 18:40:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Philippe Blain via GitGitGadget X-Patchwork-Id: 11736221 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 70A8E913 for ; Tue, 25 Aug 2020 18:40:22 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 52DBB2074D for ; Tue, 25 Aug 2020 18:40:22 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="sdYgwFQX" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726374AbgHYSkU (ORCPT ); Tue, 25 Aug 2020 14:40:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44412 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726737AbgHYSkL (ORCPT ); Tue, 25 Aug 2020 14:40:11 -0400 Received: from mail-wr1-x441.google.com (mail-wr1-x441.google.com [IPv6:2a00:1450:4864:20::441]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 53AFAC061755 for ; Tue, 25 Aug 2020 11:40:11 -0700 (PDT) Received: by mail-wr1-x441.google.com with SMTP id a5so13880436wrm.6 for ; Tue, 25 Aug 2020 11:40:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=LJaF+KBq6DinuUhew6phz3UG9LLuWbKh+8DAyadGgBc=; b=sdYgwFQX/DTcYdRf7O0+M2uMBJSZDssmvdXznVZt7thLlZNjhZTNResKxaujJqnatM KrwRHSlFsDv/wdvDckpz7YBNpoLHHp3gI5Lf+5NJsW/RxolqTphg9PPgb5+ue22l87R7 UM0SVMjN3GblLiXp1ltaiV7+r1eeTCawGGvO13mYlsc1m9mtWgVZzpxG1GjWS/phDgsg aeb4ucvhTtcjwtO6ywyd86lr7vuRc9+74Pq79xKlM1k7iOEOIye7cqaLalhi9nbvdX8s jQt2luFNd39miV2EsYmDDbahszxWSbLv55KvaEUIMN3oxFNdRcQnjCzTXvrl44pfBhn6 IV+Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=LJaF+KBq6DinuUhew6phz3UG9LLuWbKh+8DAyadGgBc=; b=O3tvET2yYvbomv2DKBuKB5xpc3oy+dFUqqrXPBfrQQPBrLWhLmlpn+ak5TINAOkDW2 oukLlK1yRPNWbcQghAKACmhN195Fmx7GDMJ2PC1SatAr7PyJkLY4CxWDnoqzPFctezCW aqm+KiCNxkr6oM/9yWC1A24ZgRacufu8JUXOYZ1KJON8kbn4lQd/HkFJyRW6Z+kvc35a r24JgkoYvmFJ8leiXNfL9/gBqU9bc5BYDgHhVrOvptkRTYhLpoQ+WNceHkVjpY9g7ol+ Ih9hiLVfWppmXTUTVVBC0ovp3YhSaCME8vJ4gViAvowawCgjyQl7WG1k9VNdgYNZRR78 NGTw== X-Gm-Message-State: AOAM533vFnQY4FCH0t1ctcdn+TsxWJOL8SZF2e8BMCAxVCYcgHdHnnSi wWVj6y6xCSjMeKW0hqLcT6AkUn1cZjE= X-Google-Smtp-Source: ABdhPJxq8JexgBzbHrs0grGuqShjc5ImkeedCtRZt0p8A8MLQbqO0k4gAOMRdr2p/2rOwsGSSvl2QA== X-Received: by 2002:adf:ea0f:: with SMTP id q15mr11652313wrm.113.1598380809658; Tue, 25 Aug 2020 11:40:09 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id n4sm18279439wrv.8.2020.08.25.11.40.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Aug 2020 11:40:09 -0700 (PDT) Message-Id: <0314258c5cbb8fd771c35e433bf6be95297c4597.1598380805.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Derrick Stolee via GitGitGadget" Date: Tue, 25 Aug 2020 18:40:01 +0000 Subject: [PATCH v2 4/7] for-each-repo: run subcommands on configured repos Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: sandals@crustytoothpaste.net, steadmon@google.com, jrnieder@gmail.com, peff@peff.net, congdanhqx@gmail.com, phillip.wood123@gmail.com, emilyshaffer@google.com, sluongng@gmail.com, jonathantanmy@google.com, Derrick Stolee , Derrick Stolee Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee It can be helpful to store a list of repositories in global or system config and then iterate Git commands on that list. Create a new builtin that makes this process simple for experts. We will use this builtin to run scheduled maintenance on all configured repositories in a future change. The test is very simple, but does highlight that the "--" argument is optional. Signed-off-by: Derrick Stolee --- .gitignore | 1 + Documentation/git-for-each-repo.txt | 59 +++++++++++++++++++++++++++++ Makefile | 1 + builtin.h | 1 + builtin/for-each-repo.c | 58 ++++++++++++++++++++++++++++ command-list.txt | 1 + git.c | 1 + t/t0068-for-each-repo.sh | 30 +++++++++++++++ 8 files changed, 152 insertions(+) create mode 100644 Documentation/git-for-each-repo.txt create mode 100644 builtin/for-each-repo.c create mode 100755 t/t0068-for-each-repo.sh diff --git a/.gitignore b/.gitignore index a5808fa30d..5eb2a2be71 100644 --- a/.gitignore +++ b/.gitignore @@ -67,6 +67,7 @@ /git-filter-branch /git-fmt-merge-msg /git-for-each-ref +/git-for-each-repo /git-format-patch /git-fsck /git-fsck-objects diff --git a/Documentation/git-for-each-repo.txt b/Documentation/git-for-each-repo.txt new file mode 100644 index 0000000000..94bd19da26 --- /dev/null +++ b/Documentation/git-for-each-repo.txt @@ -0,0 +1,59 @@ +git-for-each-repo(1) +==================== + +NAME +---- +git-for-each-repo - Run a Git command on a list of repositories + + +SYNOPSIS +-------- +[verse] +'git for-each-repo' --config= [--] + + +DESCRIPTION +----------- +Run a Git command on a list of repositories. The arguments after the +known options or `--` indicator are used as the arguments for the Git +subprocess. + +THIS COMMAND IS EXPERIMENTAL. THE BEHAVIOR MAY CHANGE. + +For example, we could run maintenance on each of a list of repositories +stored in a `maintenance.repo` config variable using + +------------- +git for-each-repo --config=maintenance.repo maintenance run +------------- + +This will run `git -C maintenance run` for each value `` +in the multi-valued config variable `maintenance.repo`. + + +OPTIONS +------- +--config=:: + Use the given config variable as a multi-valued list storing + absolute path names. Iterate on that list of paths to run + the given arguments. ++ +These config values are loaded from system, global, and local Git config, +as available. If `git for-each-repo` is run in a directory that is not a +Git repository, then only the system and global config is used. + + +SUBPROCESS BEHAVIOR +------------------- + +If any `git -C ` subprocess returns a non-zero exit code, +then the `git for-each-repo` process returns that exit code without running +more subprocesses. + +Each `git -C ` subprocess inherits the standard file +descriptors `stdin`, `stdout`, and `stderr`. + + +GIT +--- +Part of the linkgit:git[1] suite diff --git a/Makefile b/Makefile index 65f8cfb236..7c588ff036 100644 --- a/Makefile +++ b/Makefile @@ -1071,6 +1071,7 @@ BUILTIN_OBJS += builtin/fetch-pack.o BUILTIN_OBJS += builtin/fetch.o BUILTIN_OBJS += builtin/fmt-merge-msg.o BUILTIN_OBJS += builtin/for-each-ref.o +BUILTIN_OBJS += builtin/for-each-repo.o BUILTIN_OBJS += builtin/fsck.o BUILTIN_OBJS += builtin/gc.o BUILTIN_OBJS += builtin/get-tar-commit-id.o diff --git a/builtin.h b/builtin.h index 17c1c0ce49..ff7c6e5aa9 100644 --- a/builtin.h +++ b/builtin.h @@ -150,6 +150,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix); int cmd_fetch_pack(int argc, const char **argv, const char *prefix); int cmd_fmt_merge_msg(int argc, const char **argv, const char *prefix); int cmd_for_each_ref(int argc, const char **argv, const char *prefix); +int cmd_for_each_repo(int argc, const char **argv, const char *prefix); int cmd_format_patch(int argc, const char **argv, const char *prefix); int cmd_fsck(int argc, const char **argv, const char *prefix); int cmd_gc(int argc, const char **argv, const char *prefix); diff --git a/builtin/for-each-repo.c b/builtin/for-each-repo.c new file mode 100644 index 0000000000..5bba623ff1 --- /dev/null +++ b/builtin/for-each-repo.c @@ -0,0 +1,58 @@ +#include "cache.h" +#include "config.h" +#include "builtin.h" +#include "parse-options.h" +#include "run-command.h" +#include "string-list.h" + +static const char * const for_each_repo_usage[] = { + N_("git for-each-repo --config= "), + NULL +}; + +static int run_command_on_repo(const char *path, + void *cbdata) +{ + int i; + struct child_process child = CHILD_PROCESS_INIT; + struct strvec *args = (struct strvec *)cbdata; + + child.git_cmd = 1; + strvec_pushl(&child.args, "-C", path, NULL); + + for (i = 0; i < args->nr; i++) + strvec_push(&child.args, args->v[i]); + + return run_command(&child); +} + +int cmd_for_each_repo(int argc, const char **argv, const char *prefix) +{ + static const char *config_key = NULL; + int i, result = 0; + const struct string_list *values; + struct strvec args = STRVEC_INIT; + + const struct option options[] = { + OPT_STRING(0, "config", &config_key, N_("config"), + N_("config key storing a list of repository paths")), + OPT_END() + }; + + argc = parse_options(argc, argv, prefix, options, for_each_repo_usage, + PARSE_OPT_STOP_AT_NON_OPTION); + + if (!config_key) + die(_("missing --config=")); + + for (i = 0; i < argc; i++) + strvec_push(&args, argv[i]); + + values = repo_config_get_value_multi(the_repository, + config_key); + + for (i = 0; !result && i < values->nr; i++) + result = run_command_on_repo(values->items[i].string, &args); + + return result; +} diff --git a/command-list.txt b/command-list.txt index 0e3204e7d1..581499be82 100644 --- a/command-list.txt +++ b/command-list.txt @@ -94,6 +94,7 @@ git-fetch-pack synchingrepositories git-filter-branch ancillarymanipulators git-fmt-merge-msg purehelpers git-for-each-ref plumbinginterrogators +git-for-each-repo plumbinginterrogators git-format-patch mainporcelain git-fsck ancillaryinterrogators complete git-gc mainporcelain diff --git a/git.c b/git.c index 24f250d29a..1cab64b5d1 100644 --- a/git.c +++ b/git.c @@ -511,6 +511,7 @@ static struct cmd_struct commands[] = { { "fetch-pack", cmd_fetch_pack, RUN_SETUP | NO_PARSEOPT }, { "fmt-merge-msg", cmd_fmt_merge_msg, RUN_SETUP }, { "for-each-ref", cmd_for_each_ref, RUN_SETUP }, + { "for-each-repo", cmd_for_each_repo, RUN_SETUP_GENTLY }, { "format-patch", cmd_format_patch, RUN_SETUP }, { "fsck", cmd_fsck, RUN_SETUP }, { "fsck-objects", cmd_fsck, RUN_SETUP }, diff --git a/t/t0068-for-each-repo.sh b/t/t0068-for-each-repo.sh new file mode 100755 index 0000000000..136b4ec839 --- /dev/null +++ b/t/t0068-for-each-repo.sh @@ -0,0 +1,30 @@ +#!/bin/sh + +test_description='git for-each-repo builtin' + +. ./test-lib.sh + +test_expect_success 'run based on configured value' ' + git init one && + git init two && + git init three && + git -C two commit --allow-empty -m "DID NOT RUN" && + git config run.key "$TRASH_DIRECTORY/one" && + git config --add run.key "$TRASH_DIRECTORY/three" && + git for-each-repo --config=run.key commit --allow-empty -m "ran" && + git -C one log -1 --pretty=format:%s >message && + grep ran message && + git -C two log -1 --pretty=format:%s >message && + ! grep ran message && + git -C three log -1 --pretty=format:%s >message && + grep ran message && + git for-each-repo --config=run.key -- commit --allow-empty -m "ran again" && + git -C one log -1 --pretty=format:%s >message && + grep again message && + git -C two log -1 --pretty=format:%s >message && + ! grep again message && + git -C three log -1 --pretty=format:%s >message && + grep again message +' + +test_done