From patchwork Wed Oct 12 21:02:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 13005440 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A8C6BC433FE for ; Wed, 12 Oct 2022 21:03:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229480AbiJLVD0 (ORCPT ); Wed, 12 Oct 2022 17:03:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57636 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229543AbiJLVDY (ORCPT ); Wed, 12 Oct 2022 17:03:24 -0400 Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com [IPv6:2a00:1450:4864:20::335]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 49CD92195 for ; Wed, 12 Oct 2022 14:03:23 -0700 (PDT) Received: by mail-wm1-x335.google.com with SMTP id ay36so11131301wmb.0 for ; Wed, 12 Oct 2022 14:03:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=23xuod5QxoIsDEcWCtQPm2IgVMfSQsPd7mQgv/Z9E1M=; b=diUADpYE46oF+ntgrcTNUE05zrVLwNEDaW7eP9e2qTQQKkwzzPQJ3/EDhT5eQsnT0I Gp06CcRkYwsvSuSWQXSjqam5tUlZIWfVcJRcJt34B9a1RMyftFRVPHTH9ir6PnXzdIUj lF9WB3l+MnPAZYxIVGf1Q7be3ickPjKzUAU9SyXdI0nHI7XcmsHo9llTJJ41M2s+3UXR MevIXdfxEVqhbu3X3BFrOXQD6RfWKeOWMMaRR2HcC1l/q78FMeq5XL7h+S2f0a1B61qz 0JHzm4wdUAbho7ZTAXbHLi7g22kd4cDH6n2/0sOvjH0IqfK/LqHLRpmC1cnMpSj6GUpL 6Ldg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=23xuod5QxoIsDEcWCtQPm2IgVMfSQsPd7mQgv/Z9E1M=; b=n6I8eaAoAlxUDZ5MbcolAR4L9J9umxIZ4tFezPbF24mA5I1qy6rIrqKnh1sK8mASqE KH5rZOD6DiVTIuavSxAfXVX69ufFM4379JxKAuLTWfgc2u3rY3hXNtQ08U7Cd9J23zO3 JDuXXYuJn5XlUC16yAafrODm3LGu7YRBcN2/X9leatBcqI2s8Ibs0jazcLHgtyXKE22u XMs+OfbLRPTTbS5eBjUrpypvmm1PAarjUC5D88/HL6WyrZeFDFEEwXTizGkP/Xj1MCRF XUx4Kwds81OxUCByPMwflUDS6uWvarzXXNio/MgbKtqa+cN1hjfLU7E6MVb7NwsgyPzr 4oFg== X-Gm-Message-State: ACrzQf1fikw5k0/XBP9jLddaGBmsfGhug8kqBQ6lUomfmliGQe+3RIrH nZHSJIbCdNzo2uxvM/jyLDGzTUxOBPGeYQ== X-Google-Smtp-Source: AMsMyM6q1FMQ7WML+4LhQTvIDKRpwQFyFcO1zW8iveGO4uOkZBzX+ICfacDUUFlJecwC7okB50S1DA== X-Received: by 2002:a7b:c30a:0:b0:3c1:bf95:e17b with SMTP id k10-20020a7bc30a000000b003c1bf95e17bmr4184954wmj.31.1665608601548; Wed, 12 Oct 2022 14:03:21 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id b6-20020adff246000000b00228a6ce17b4sm550079wrp.37.2022.10.12.14.03.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Oct 2022 14:03:21 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Calvin Wan , Emily Shaffer , Phillip Wood , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v3 01/15] run-command test helper: use "else if" pattern Date: Wed, 12 Oct 2022 23:02:20 +0200 Message-Id: X-Mailer: git-send-email 2.38.0.971.ge79ff6d20e7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Adjust the cmd__run_command() to use an "if/else if" chain rather than mutually exclusive "if" statements. This non-functional change makes a subsequent commit smaller. Signed-off-by: Ævar Arnfjörð Bjarmason --- t/helper/test-run-command.c | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-) diff --git a/t/helper/test-run-command.c b/t/helper/test-run-command.c index c9283b47afa..390fa4fb724 100644 --- a/t/helper/test-run-command.c +++ b/t/helper/test-run-command.c @@ -427,18 +427,17 @@ int cmd__run_command(int argc, const char **argv) strvec_clear(&proc.args); strvec_pushv(&proc.args, (const char **)argv + 3); - if (!strcmp(argv[1], "run-command-parallel")) + if (!strcmp(argv[1], "run-command-parallel")) { exit(run_processes_parallel(jobs, parallel_next, NULL, NULL, &proc)); - - if (!strcmp(argv[1], "run-command-abort")) + } else if (!strcmp(argv[1], "run-command-abort")) { exit(run_processes_parallel(jobs, parallel_next, NULL, task_finished, &proc)); - - if (!strcmp(argv[1], "run-command-no-jobs")) + } else if (!strcmp(argv[1], "run-command-no-jobs")) { exit(run_processes_parallel(jobs, no_job, NULL, task_finished, &proc)); - - fprintf(stderr, "check usage\n"); - return 1; + } else { + fprintf(stderr, "check usage\n"); + return 1; + } } From patchwork Wed Oct 12 21:02:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 13005442 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9AECAC433FE for ; Wed, 12 Oct 2022 21:03:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229714AbiJLVD3 (ORCPT ); Wed, 12 Oct 2022 17:03:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57644 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229628AbiJLVDZ (ORCPT ); Wed, 12 Oct 2022 17:03:25 -0400 Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com [IPv6:2a00:1450:4864:20::42e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6F762BDB for ; Wed, 12 Oct 2022 14:03:24 -0700 (PDT) Received: by mail-wr1-x42e.google.com with SMTP id r13so27925985wrj.11 for ; Wed, 12 Oct 2022 14:03:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=bUdYd+/n3y2i5wi6BAUzfJxEHaBcqp9427yq15owIfc=; b=WvRsELn2wLSRXgiTbMYyra7YxgMqHBrsM7SUUw1g2KXVlpy+BqGk9SXl+V+OTT/RdB 8l/bk1Y3o7dGxA8+iPxeGVlvD7skvgUDRuBL+FMw5vNdrheKt9FEi4Ye9zE6sEvKb98J DULCxIn5DBb5i1h/ORYK9udIcesjpKcpAk+zjC92nwjK9uiWW/oulQyQfIe76cC724t3 PcXlRG6ocdimZaIMxPj63vovijAQPpVhzoSiW/GALmH06cE57l4asdjE/xyiNW/iXocP cXHVcx/CNfmM0F6lIueavGd/IUAtspU4V8SqZgWDKhW08mRwYXe/LkBheUBgBtkzSlUO Pr9w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bUdYd+/n3y2i5wi6BAUzfJxEHaBcqp9427yq15owIfc=; b=s150slDPL8tb6/h1+sIuxQD3bXf9QBMl54RPTOPYPwYgYmq7gaJQAVhFVS26O7gpZu pznp0rmWdGNkyW2+PgFCOKJBdTUGwndJJEUPNnoe7W/YZbfwtdGjRijbfZXZq+6vZbRW I5HyWx1Tz3d+Ra1Qh6Wusy1hwcKqemFqUpMMH3EEMTZ17VZkrtCWFrkJpuR21cUMClDT l0apAqJQFEmAedfvGaLoTP7mzOj637JbkaFpAW2MmXXUQS2quJLRtGoKKrhM+LEJtR8z pe0yF79VhWmObsOvBEn55WWaJtT4+dNw5JfDVei6iLgTWTqLk8SOlC79J1ZdzP6O2nhU aQ8A== X-Gm-Message-State: ACrzQf1/mDPcr9ZGNIXn/d76CDqfYwce0SnjOOADv6/Ew11ftP/Ci5L3 gtyo10LDrZ/bnyKMHZTHn2nvzVoXvJuWtg== X-Google-Smtp-Source: AMsMyM6jCPlO7Bo7UhlM1MWi5F4UV5NlNjJbWSE2pY3xOt/D2hkYGTMHthx+WmT4zPe9O1UQhgZrJg== X-Received: by 2002:a05:6000:1a85:b0:230:f238:a48c with SMTP id f5-20020a0560001a8500b00230f238a48cmr8828953wry.92.1665608602677; Wed, 12 Oct 2022 14:03:22 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id b6-20020adff246000000b00228a6ce17b4sm550079wrp.37.2022.10.12.14.03.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Oct 2022 14:03:22 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Calvin Wan , Emily Shaffer , Phillip Wood , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v3 02/15] run-command API: have "run_processes_parallel{,_tr2}()" return void Date: Wed, 12 Oct 2022 23:02:21 +0200 Message-Id: X-Mailer: git-send-email 2.38.0.971.ge79ff6d20e7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Change the "run_processes_parallel{,_tr2}()" functions to return void, instead of int. Ever since c553c72eed6 (run-command: add an asynchronous parallel child processor, 2015-12-15) they have unconditionally returned 0. To get a "real" return value out of this function the caller needs to get it via the "task_finished_fn" callback, see the example in hook.c added in 96e7225b310 (hook: add 'run' subcommand, 2021-12-22). So the "result = " and "if (!result)" code added to "builtin/fetch.c" d54dea77dba (fetch: let --jobs= parallelize --multiple, too, 2019-10-05) has always been redundant, we always took that "if" path. Likewise the "ret =" in "t/helper/test-run-command.c" added in be5d88e1128 (test-tool run-command: learn to run (parts of) the testsuite, 2019-10-04) wasn't used, instead we got the return value from the "if (suite.failed.nr > 0)" block seen in the context. Subsequent commits will alter this API interface, getting rid of this always-zero return value makes it easier to understand those changes. Signed-off-by: Ævar Arnfjörð Bjarmason --- builtin/fetch.c | 17 ++++++++--------- run-command.c | 27 +++++++++++---------------- run-command.h | 16 ++++++++-------- t/helper/test-run-command.c | 16 ++++++++-------- 4 files changed, 35 insertions(+), 41 deletions(-) diff --git a/builtin/fetch.c b/builtin/fetch.c index a0fca93bb6a..78043fb67ef 100644 --- a/builtin/fetch.c +++ b/builtin/fetch.c @@ -1953,15 +1953,14 @@ static int fetch_multiple(struct string_list *list, int max_children) struct parallel_fetch_state state = { argv.v, list, 0, 0 }; strvec_push(&argv, "--end-of-options"); - result = run_processes_parallel_tr2(max_children, - &fetch_next_remote, - &fetch_failed_to_start, - &fetch_finished, - &state, - "fetch", "parallel/fetch"); - - if (!result) - result = state.result; + run_processes_parallel_tr2(max_children, + &fetch_next_remote, + &fetch_failed_to_start, + &fetch_finished, + &state, + "fetch", "parallel/fetch"); + + result = state.result; } else for (i = 0; i < list->nr; i++) { const char *name = list->items[i].string; diff --git a/run-command.c b/run-command.c index 5ec3a46dccf..642e6b6e057 100644 --- a/run-command.c +++ b/run-command.c @@ -1783,11 +1783,11 @@ static int pp_collect_finished(struct parallel_processes *pp) return result; } -int run_processes_parallel(int n, - get_next_task_fn get_next_task, - start_failure_fn start_failure, - task_finished_fn task_finished, - void *pp_cb) +void run_processes_parallel(int n, + get_next_task_fn get_next_task, + start_failure_fn start_failure, + task_finished_fn task_finished, + void *pp_cb) { int i, code; int output_timeout = 100; @@ -1834,25 +1834,20 @@ int run_processes_parallel(int n, } pp_cleanup(&pp); - return 0; } -int run_processes_parallel_tr2(int n, get_next_task_fn get_next_task, - start_failure_fn start_failure, - task_finished_fn task_finished, void *pp_cb, - const char *tr2_category, const char *tr2_label) +void run_processes_parallel_tr2(int n, get_next_task_fn get_next_task, + start_failure_fn start_failure, + task_finished_fn task_finished, void *pp_cb, + const char *tr2_category, const char *tr2_label) { - int result; - trace2_region_enter_printf(tr2_category, tr2_label, NULL, "max:%d", ((n < 1) ? online_cpus() : n)); - result = run_processes_parallel(n, get_next_task, start_failure, - task_finished, pp_cb); + run_processes_parallel(n, get_next_task, start_failure, + task_finished, pp_cb); trace2_region_leave(tr2_category, tr2_label, NULL); - - return result; } int run_auto_maintenance(int quiet) diff --git a/run-command.h b/run-command.h index 0e85e5846a5..e76a1b6b5b3 100644 --- a/run-command.h +++ b/run-command.h @@ -485,14 +485,14 @@ typedef int (*task_finished_fn)(int result, * API reads that setting. */ extern int run_processes_parallel_ungroup; -int run_processes_parallel(int n, - get_next_task_fn, - start_failure_fn, - task_finished_fn, - void *pp_cb); -int run_processes_parallel_tr2(int n, get_next_task_fn, start_failure_fn, - task_finished_fn, void *pp_cb, - const char *tr2_category, const char *tr2_label); +void run_processes_parallel(int n, + get_next_task_fn, + start_failure_fn, + task_finished_fn, + void *pp_cb); +void run_processes_parallel_tr2(int n, get_next_task_fn, start_failure_fn, + task_finished_fn, void *pp_cb, + const char *tr2_category, const char *tr2_label); /** * Convenience function which prepares env for a command to be run in a diff --git a/t/helper/test-run-command.c b/t/helper/test-run-command.c index 390fa4fb724..30c474f3243 100644 --- a/t/helper/test-run-command.c +++ b/t/helper/test-run-command.c @@ -192,8 +192,8 @@ static int testsuite(int argc, const char **argv) fprintf(stderr, "Running %"PRIuMAX" tests (%d at a time)\n", (uintmax_t)suite.tests.nr, max_jobs); - ret = run_processes_parallel(max_jobs, next_test, test_failed, - test_finished, &suite); + run_processes_parallel(max_jobs, next_test, test_failed, + test_finished, &suite); if (suite.failed.nr > 0) { ret = 1; @@ -428,16 +428,16 @@ int cmd__run_command(int argc, const char **argv) strvec_pushv(&proc.args, (const char **)argv + 3); if (!strcmp(argv[1], "run-command-parallel")) { - exit(run_processes_parallel(jobs, parallel_next, - NULL, NULL, &proc)); + run_processes_parallel(jobs, parallel_next, NULL, NULL, &proc); } else if (!strcmp(argv[1], "run-command-abort")) { - exit(run_processes_parallel(jobs, parallel_next, - NULL, task_finished, &proc)); + run_processes_parallel(jobs, parallel_next, NULL, + task_finished, &proc); } else if (!strcmp(argv[1], "run-command-no-jobs")) { - exit(run_processes_parallel(jobs, no_job, - NULL, task_finished, &proc)); + run_processes_parallel(jobs, no_job, NULL, task_finished, + &proc); } else { fprintf(stderr, "check usage\n"); return 1; } + exit(0); } From patchwork Wed Oct 12 21:02:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 13005443 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 418C7C43219 for ; Wed, 12 Oct 2022 21:03:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229771AbiJLVDb (ORCPT ); Wed, 12 Oct 2022 17:03:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57656 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229677AbiJLVD0 (ORCPT ); Wed, 12 Oct 2022 17:03:26 -0400 Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 92A11BF1 for ; Wed, 12 Oct 2022 14:03:25 -0700 (PDT) Received: by mail-wr1-x434.google.com with SMTP id j7so27970174wrr.3 for ; Wed, 12 Oct 2022 14:03:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=wMJ0ei29o3W92b760+HvZkT6k/ZhX31NK+F7yQCZdt4=; b=MSK0PC8M8p/SjZJ7f2OXgT5TeM9ZPAVWeXgSN6ZMywDduQ8O9CAZEGHJnTk8cWZm1Y vz9+Cu8BqBi1znltPje1eF9Vhkq2QHHOXOXfEyCG/Aijp3vgbOtXv/5PQPc+QpgV0vzo 2RsQeCY66f1lXDsdVwCxbsEssXIl+8ajq5vkDjBeysLnwgHf60vghWbUnmAncDR+Ec/t cL5VW2dSc2WFSDW8lMIGw4BlW3K3wnr0lHQGDce3wzqcKsdIhTgrWHmIWTneXlPujdqO JdRxqiOHFlBYjjmqqMCVH2vpnIcRd1JbEvXBe+XVTrkoHu3T9bW0fpMrtP3tg8dYYKt5 2reA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wMJ0ei29o3W92b760+HvZkT6k/ZhX31NK+F7yQCZdt4=; b=BsRWPivDjOPmG0FS1i6bHgkfeR6NmPazBgenjro/7rc2NAUZzWQIvSgFyyeordrDUS MJkyDB/PI7WZ+0FaP+Fy+Rb2p+v2v+iWKwvDC8O2Hu+kRUWV9wQsMRUdWoYHjsP2P2Po iZJdKTrtREC/MAMg+eOJScbxFZXjH9GxSGUKOfnb1Os9NFHchjX9XsQFeCDzj4ehGKpB L2SvRpIVzhjAzRU3EL+UCb5IALN1xeVJgIPM/KOW0QKEyH2v5rHDMZOgK5G3VpHdF8UL lWwNkP1NRSYfThIOEkXn03Z6xHXXgvCCxrYqvJibw8ZgZ1ac9h0PjgYMFUannBE0t5Ww 5vRw== X-Gm-Message-State: ACrzQf1h9VSXB8hkpaaMoFW8pSLXxahNVOefJThlLQnI3qfOQzeJHHmn q/+kxDKd6Vu+uaEbpMML3p0yFHSdIE7DXg== X-Google-Smtp-Source: AMsMyM5aDAJIv0NivZUSl6IVJgEuyVST6ZMXnZ1IcgIHEjWqaDNytoThlITYig6wqE4A/hnUYI8YWg== X-Received: by 2002:adf:ef43:0:b0:22d:c507:dd48 with SMTP id c3-20020adfef43000000b0022dc507dd48mr19383034wrp.416.1665608603682; Wed, 12 Oct 2022 14:03:23 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id b6-20020adff246000000b00228a6ce17b4sm550079wrp.37.2022.10.12.14.03.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Oct 2022 14:03:22 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Calvin Wan , Emily Shaffer , Phillip Wood , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v3 03/15] run-command tests: use "return", not "exit" Date: Wed, 12 Oct 2022 23:02:22 +0200 Message-Id: X-Mailer: git-send-email 2.38.0.971.ge79ff6d20e7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Change the "run-command" test helper to "return" instead of calling "exit", see 338abb0f045 (builtins + test helpers: use return instead of exit() in cmd_*, 2021-06-08) Because we'd previously gotten past the SANITIZE=leak check by using exit() here we need to move to "goto cleanup" pattern. Signed-off-by: Ævar Arnfjörð Bjarmason --- t/helper/test-run-command.c | 33 ++++++++++++++++++++++----------- 1 file changed, 22 insertions(+), 11 deletions(-) diff --git a/t/helper/test-run-command.c b/t/helper/test-run-command.c index 30c474f3243..ee509aefa2f 100644 --- a/t/helper/test-run-command.c +++ b/t/helper/test-run-command.c @@ -381,13 +381,14 @@ int cmd__run_command(int argc, const char **argv) { struct child_process proc = CHILD_PROCESS_INIT; int jobs; + int ret; if (argc > 1 && !strcmp(argv[1], "testsuite")) - exit(testsuite(argc - 1, argv + 1)); + return testsuite(argc - 1, argv + 1); if (!strcmp(argv[1], "inherited-handle")) - exit(inherit_handle(argv[0])); + return inherit_handle(argv[0]); if (!strcmp(argv[1], "inherited-handle-child")) - exit(inherit_handle_child()); + return inherit_handle_child(); if (argc >= 2 && !strcmp(argv[1], "quote-stress-test")) return !!quote_stress_test(argc - 1, argv + 1); @@ -404,18 +405,24 @@ int cmd__run_command(int argc, const char **argv) argv += 2; argc -= 2; } - if (argc < 3) - return 1; + if (argc < 3) { + ret = 1; + goto cleanup; + } strvec_pushv(&proc.args, (const char **)argv + 2); if (!strcmp(argv[1], "start-command-ENOENT")) { - if (start_command(&proc) < 0 && errno == ENOENT) - return 0; + if (start_command(&proc) < 0 && errno == ENOENT) { + ret = 0; + goto cleanup; + } fprintf(stderr, "FAIL %s\n", argv[1]); return 1; } - if (!strcmp(argv[1], "run-command")) - exit(run_command(&proc)); + if (!strcmp(argv[1], "run-command")) { + ret = run_command(&proc); + goto cleanup; + } if (!strcmp(argv[1], "--ungroup")) { argv += 1; @@ -436,8 +443,12 @@ int cmd__run_command(int argc, const char **argv) run_processes_parallel(jobs, no_job, NULL, task_finished, &proc); } else { + ret = 1; fprintf(stderr, "check usage\n"); - return 1; + goto cleanup; } - exit(0); + ret = 0; +cleanup: + child_process_clear(&proc); + return ret; } From patchwork Wed Oct 12 21:02:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 13005444 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A1D2AC4332F for ; Wed, 12 Oct 2022 21:03:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229791AbiJLVDe (ORCPT ); Wed, 12 Oct 2022 17:03:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57680 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229566AbiJLVD2 (ORCPT ); Wed, 12 Oct 2022 17:03:28 -0400 Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com [IPv6:2a00:1450:4864:20::435]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BD1FBBDB for ; Wed, 12 Oct 2022 14:03:26 -0700 (PDT) Received: by mail-wr1-x435.google.com with SMTP id u10so27977019wrq.2 for ; Wed, 12 Oct 2022 14:03:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=5i/VlNChALuCT9gMq8XDHo5XRVOUINHsUwh5rkTXCOE=; b=jvBNbLjgavQrhzS6Og/0BnB44jLkJ14n+xVIZOe4qH4FIODozjTTOJAJL3Ce18HXQJ Lis4LLF1I8YuE44L6OZ8fdbBg7ADx/uP4ecTqqH/MmNNHPE+NSp3mMxwxwITCqu8pybm Zxd0c38YvA3AWGEZ7eNrjH9vxfJ2NWbPQH/poyYVRuzEWu/hOePUtpTbMxNhJ4zc4QTI hB4hJBMVIlXgcOS9cHtJPHXhLenW1gshkuBd8vHoTJiokEumJvX1d8lOc+TkvaaHgeb9 8/SRHSAwWf+e3P4wN2BH4Rt6EgyKvklQdVwjXcm3UtGGpECANp5qZT7MhCCb7eFD2mzB uRgQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=5i/VlNChALuCT9gMq8XDHo5XRVOUINHsUwh5rkTXCOE=; b=nT3dq55b3loHq49MIu5wWwnAPvgUdJzfxfCn1sWWWLuT2ckRzNvexxc4W6aqAApUS4 jAUfsFtakP+I9yNEySnWMJ3WcD3B25vLxWyiB2jryrLHuwLW057C+wSboCkkt7m6nCdg 33KK17ZBK4+CKVvbLZxPmh32modj2nFBU29o2FTUHXlOOneTYT3rUIWby7hqz0w6c+Mz Wj8CfFyfLFOF0peyFksp6eCHYk67w+gQGU2DjW/ue4WcKqIY2p02jdOALZbBZDSSTuLS kWQz6neE411bBdxTOFhVrU99PswrUdiqRelQcw81m+vKuwz4ls9zIGIAsWeQ60YjDbgU P8Kw== X-Gm-Message-State: ACrzQf2bltTjnmJHZmJMXL2Rym3D3vAOLsn75fpFWznh+TLCQdp9UcXT kib2kByOlSET8HugKBzY7CJJkadEvjsafg== X-Google-Smtp-Source: AMsMyM6aylExyrGIx/aXGw7DbNT5xRb0LCN9zLqHuwN9HDzbcbdk4rt3W9RU04AhFL8/cFszTqKwPw== X-Received: by 2002:adf:ebcf:0:b0:22c:9eb4:d6f6 with SMTP id v15-20020adfebcf000000b0022c9eb4d6f6mr19270092wrn.251.1665608604770; Wed, 12 Oct 2022 14:03:24 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id b6-20020adff246000000b00228a6ce17b4sm550079wrp.37.2022.10.12.14.03.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Oct 2022 14:03:24 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Calvin Wan , Emily Shaffer , Phillip Wood , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v3 04/15] run-command API: make "n" parameter a "size_t" Date: Wed, 12 Oct 2022 23:02:23 +0200 Message-Id: X-Mailer: git-send-email 2.38.0.971.ge79ff6d20e7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Make the "n" variable added in c553c72eed6 (run-command: add an asynchronous parallel child processor, 2015-12-15) a "size_t". As we'll see in a subsequent commit we do pass "0" here, but never "jobs < 0". We could have made it an "unsigned int", but as we're having to change this let's not leave another case in the codebase where a size_t and "unsigned int" size differ on some platforms. In this case it's likely to never matter, but it's easier to not need to worry about it. After this and preceding changes: make run-command.o DEVOPTS=extra-all CFLAGS=-Wno-unused-parameter Only has one (and new) -Wsigned-compare warning relevant to a comparison about our "n" or "{nr,max}_processes": About using our "n" (size_t) in the same expression as online_cpus() (int). A subsequent commit will adjust & deal with online_cpus() and that warning. The only users of the "n" parameter are: * builtin/fetch.c: defaults to 1, reads from the "fetch.parallel" config. As seen in the code that parses the config added in d54dea77dba (fetch: let --jobs= parallelize --multiple, too, 2019-10-05) will die if the git_config_int() return value is < 0. It will however pass us n = 0, as we'll see in a subsequent commit. * submodule.c: defaults to 1, reads from "submodule.fetchJobs" config. Read via code originally added in a028a1930c6 (fetching submodules: respect `submodule.fetchJobs` config option, 2016-02-29). It now piggy-backs on the the submodule.fetchJobs code and validation added in f20e7c1ea24 (submodule: remove submodule.fetchjobs from submodule-config parsing, 2017-08-02). Like builtin/fetch.c it will die if the git_config_int() return value is < 0, but like builtin/fetch.c it will pass us n = 0. * builtin/submodule--helper.c: defaults to 1. Read via code originally added in 2335b870fa7 (submodule update: expose parallelism to the user, 2016-02-29). Since f20e7c1ea24 (submodule: remove submodule.fetchjobs from submodule-config parsing, 2017-08-02) it shares a config parser and semantics with the submodule.c caller. * hook.c: hardcoded to 1, see 96e7225b310 (hook: add 'run' subcommand, 2021-12-22). * t/helper/test-run-command.c: can be -1 after parsing the arguments, but will then be overridden to online_cpus() before passing it to this API. See be5d88e1128 (test-tool run-command: learn to run (parts of) the testsuite, 2019-10-04). Signed-off-by: Ævar Arnfjörð Bjarmason --- run-command.c | 42 ++++++++++++++++++------------------------ run-command.h | 4 ++-- 2 files changed, 20 insertions(+), 26 deletions(-) diff --git a/run-command.c b/run-command.c index 642e6b6e057..14a6e38e804 100644 --- a/run-command.c +++ b/run-command.c @@ -1500,8 +1500,8 @@ int run_processes_parallel_ungroup; struct parallel_processes { void *data; - int max_processes; - int nr_processes; + size_t max_processes; + size_t nr_processes; get_next_task_fn get_next_task; start_failure_fn start_failure; @@ -1522,7 +1522,7 @@ struct parallel_processes { unsigned shutdown : 1; unsigned ungroup : 1; - int output_owner; + size_t output_owner; struct strbuf buffered_output; /* of finished children */ }; @@ -1543,9 +1543,7 @@ static int default_task_finished(int result, static void kill_children(struct parallel_processes *pp, int signo) { - int i, n = pp->max_processes; - - for (i = 0; i < n; i++) + for (size_t i = 0; i < pp->max_processes; i++) if (pp->children[i].state == GIT_CP_WORKING) kill(pp->children[i].process.pid, signo); } @@ -1560,20 +1558,19 @@ static void handle_children_on_signal(int signo) } static void pp_init(struct parallel_processes *pp, - int n, + size_t n, get_next_task_fn get_next_task, start_failure_fn start_failure, task_finished_fn task_finished, void *data, int ungroup) { - int i; - if (n < 1) n = online_cpus(); pp->max_processes = n; - trace_printf("run_processes_parallel: preparing to run up to %d tasks", n); + trace_printf("run_processes_parallel: preparing to run up to %"PRIuMAX" tasks", + (uintmax_t)n); pp->data = data; if (!get_next_task) @@ -1594,7 +1591,7 @@ static void pp_init(struct parallel_processes *pp, CALLOC_ARRAY(pp->pfd, n); strbuf_init(&pp->buffered_output, 0); - for (i = 0; i < n; i++) { + for (size_t i = 0; i < n; i++) { strbuf_init(&pp->children[i].err, 0); child_process_init(&pp->children[i].process); if (pp->pfd) { @@ -1609,10 +1606,8 @@ static void pp_init(struct parallel_processes *pp, static void pp_cleanup(struct parallel_processes *pp) { - int i; - trace_printf("run_processes_parallel: done"); - for (i = 0; i < pp->max_processes; i++) { + for (size_t i = 0; i < pp->max_processes; i++) { strbuf_release(&pp->children[i].err); child_process_clear(&pp->children[i].process); } @@ -1639,7 +1634,8 @@ static void pp_cleanup(struct parallel_processes *pp) */ static int pp_start_one(struct parallel_processes *pp) { - int i, code; + size_t i; + int code; for (i = 0; i < pp->max_processes; i++) if (pp->children[i].state == GIT_CP_FREE) @@ -1697,7 +1693,7 @@ static void pp_buffer_stderr(struct parallel_processes *pp, int output_timeout) } /* Buffer output from all pipes. */ - for (i = 0; i < pp->max_processes; i++) { + for (size_t i = 0; i < pp->max_processes; i++) { if (pp->children[i].state == GIT_CP_WORKING && pp->pfd[i].revents & (POLLIN | POLLHUP)) { int n = strbuf_read_once(&pp->children[i].err, @@ -1714,7 +1710,7 @@ static void pp_buffer_stderr(struct parallel_processes *pp, int output_timeout) static void pp_output(struct parallel_processes *pp) { - int i = pp->output_owner; + size_t i = pp->output_owner; if (pp->children[i].state == GIT_CP_WORKING && pp->children[i].err.len) { @@ -1725,8 +1721,8 @@ static void pp_output(struct parallel_processes *pp) static int pp_collect_finished(struct parallel_processes *pp) { - int i, code; - int n = pp->max_processes; + int code; + size_t i, n = pp->max_processes; int result = 0; while (pp->nr_processes > 0) { @@ -1783,7 +1779,7 @@ static int pp_collect_finished(struct parallel_processes *pp) return result; } -void run_processes_parallel(int n, +void run_processes_parallel(size_t n, get_next_task_fn get_next_task, start_failure_fn start_failure, task_finished_fn task_finished, @@ -1817,9 +1813,7 @@ void run_processes_parallel(int n, if (!pp.nr_processes) break; if (ungroup) { - int i; - - for (i = 0; i < pp.max_processes; i++) + for (size_t i = 0; i < pp.max_processes; i++) pp.children[i].state = GIT_CP_WAIT_CLEANUP; } else { pp_buffer_stderr(&pp, output_timeout); @@ -1836,7 +1830,7 @@ void run_processes_parallel(int n, pp_cleanup(&pp); } -void run_processes_parallel_tr2(int n, get_next_task_fn get_next_task, +void run_processes_parallel_tr2(size_t n, get_next_task_fn get_next_task, start_failure_fn start_failure, task_finished_fn task_finished, void *pp_cb, const char *tr2_category, const char *tr2_label) diff --git a/run-command.h b/run-command.h index e76a1b6b5b3..6f7604e1146 100644 --- a/run-command.h +++ b/run-command.h @@ -485,12 +485,12 @@ typedef int (*task_finished_fn)(int result, * API reads that setting. */ extern int run_processes_parallel_ungroup; -void run_processes_parallel(int n, +void run_processes_parallel(size_t n, get_next_task_fn, start_failure_fn, task_finished_fn, void *pp_cb); -void run_processes_parallel_tr2(int n, get_next_task_fn, start_failure_fn, +void run_processes_parallel_tr2(size_t n, get_next_task_fn, start_failure_fn, task_finished_fn, void *pp_cb, const char *tr2_category, const char *tr2_label); From patchwork Wed Oct 12 21:02:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 13005445 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BA74EC433FE for ; Wed, 12 Oct 2022 21:04:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229698AbiJLVED (ORCPT ); Wed, 12 Oct 2022 17:04:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57938 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229793AbiJLVDg (ORCPT ); Wed, 12 Oct 2022 17:03:36 -0400 Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A6CF01ADA3 for ; Wed, 12 Oct 2022 14:03:29 -0700 (PDT) Received: by mail-wr1-x434.google.com with SMTP id b4so27972203wrs.1 for ; Wed, 12 Oct 2022 14:03:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Isce64CBtmUTAta7eZ83mkGxGAFgMqp1/1yVHncBawI=; b=ZRKdNHBypH5MivIWGtcILgxcFK/lhHF65WbW9p2dDJ6CcpIWI9rLtVG7nsroN/ULyw x7Q7uo/6HlP+pgCyDFdCAKkiwhnxrV/SMnhSj6UkDGMCVfgGKDNWWTTmfO1pkPB6/8nG Ynngp9Ek5vHRspfGUkrlpxO7o5ejWGhY+Nd69bFS9S1gudR0ISpOKQlpbTUuFAsi3F/Z uBOK8495CsjLfMGVOW+cpN2gCeyG6UcmXnwvItPPNB76TTOveAAnxPbY7mqZmWUOI8tz H+uejGz1v9hjE3GkNsNWfQBvjq9qWX2yOgPbebLkRIG8XDOnPPMlMDiaJK/afEQ76aRt j00A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Isce64CBtmUTAta7eZ83mkGxGAFgMqp1/1yVHncBawI=; b=XT2LnCdj0sRFkpcG14Abdtj+VfmIvrFUAtHtQ7VIlY7xVU1evI54kGf0YoohJRnEIa 5PaNS4O3yN6opHXn1C3VBtml2vuXOPROzwFHIa8wGCISPnUGnSW+XSQ5TwebYTY1CYI0 Gg4JP2tbpuL58nLaVw0wSjvODDMraxvBEioo3WFiX3lcmWk9v6ZOPjHOF4lOfF62qYWn dzynDnzj7PV9ytZqPUvPHpokM6Z5Kdgi7n2WRZtE/fQwtj9x9jFkwGvgw4KOOxJS34FO MZyBxqxpCu+oLwWggJfRRL0ZJv4+SyeBdCHcHl6HV31F4XQrLG4N80ra4s9I7cLwdf4S ejUg== X-Gm-Message-State: ACrzQf2jxc9d8LefzXS8s4UwTR/vwGvWKx6EvHcfEGD5SuNqzHoOdSgg 36POUGrlbQJQo9U+Bj5DE9HSBHw+FIujlw== X-Google-Smtp-Source: AMsMyM64yVkzI7nIH95f2BnqKTEUrPCz/VFjn0rtApHoEKJDc7LuGlHH/ZziV/ufuXUfQNZ7bpX4sQ== X-Received: by 2002:a5d:456b:0:b0:230:9e5b:c64c with SMTP id a11-20020a5d456b000000b002309e5bc64cmr10757241wrc.211.1665608607369; Wed, 12 Oct 2022 14:03:27 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id b6-20020adff246000000b00228a6ce17b4sm550079wrp.37.2022.10.12.14.03.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Oct 2022 14:03:25 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Calvin Wan , Emily Shaffer , Phillip Wood , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v3 05/15] run-command API: don't fall back on online_cpus() Date: Wed, 12 Oct 2022 23:02:24 +0200 Message-Id: X-Mailer: git-send-email 2.38.0.971.ge79ff6d20e7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When a "jobs = 0" is passed let's BUG() out rather than fall back on online_cpus(). The default behavior was added when this API was implemented in c553c72eed6 (run-command: add an asynchronous parallel child processor, 2015-12-15). Most of our code in-tree that scales up to "online_cpus()" by default calls that function by itself. Keeping this default behavior just for the sake of two callers means that we'd need to maintain this one spot where we're second-guessing the config passed down into pp_init(). The preceding commit has an overview of the API callers that passed "jobs = 0". There were only two of them (actually three, but they resolved to these two config parsing codepaths). The "fetch.parallel" caller already had a test for the "fetch.parallel=0" case added in 0353c688189 (fetch: do not run a redundant fetch from submodule, 2022-05-16), but there was no such test for "submodule.fetchJobs". Let's add one here. Signed-off-by: Ævar Arnfjörð Bjarmason --- builtin/fetch.c | 2 ++ run-command.c | 7 +++---- submodule-config.c | 2 ++ t/t5526-fetch-submodules.sh | 8 +++++++- 4 files changed, 14 insertions(+), 5 deletions(-) diff --git a/builtin/fetch.c b/builtin/fetch.c index 78043fb67ef..82f1da14ec1 100644 --- a/builtin/fetch.c +++ b/builtin/fetch.c @@ -122,6 +122,8 @@ static int git_fetch_config(const char *k, const char *v, void *cb) fetch_parallel_config = git_config_int(k, v); if (fetch_parallel_config < 0) die(_("fetch.parallel cannot be negative")); + if (!fetch_parallel_config) + fetch_parallel_config = online_cpus(); return 0; } diff --git a/run-command.c b/run-command.c index 14a6e38e804..14ea409375a 100644 --- a/run-command.c +++ b/run-command.c @@ -1564,8 +1564,8 @@ static void pp_init(struct parallel_processes *pp, task_finished_fn task_finished, void *data, int ungroup) { - if (n < 1) - n = online_cpus(); + if (!n) + BUG("you must provide a non-zero number of processes!"); pp->max_processes = n; @@ -1835,8 +1835,7 @@ void run_processes_parallel_tr2(size_t n, get_next_task_fn get_next_task, task_finished_fn task_finished, void *pp_cb, const char *tr2_category, const char *tr2_label) { - trace2_region_enter_printf(tr2_category, tr2_label, NULL, "max:%d", - ((n < 1) ? online_cpus() : n)); + trace2_region_enter_printf(tr2_category, tr2_label, NULL, "max:%d", n); run_processes_parallel(n, get_next_task, start_failure, task_finished, pp_cb); diff --git a/submodule-config.c b/submodule-config.c index cd7ee236a12..4dc61b3a78a 100644 --- a/submodule-config.c +++ b/submodule-config.c @@ -303,6 +303,8 @@ int parse_submodule_fetchjobs(const char *var, const char *value) int fetchjobs = git_config_int(var, value); if (fetchjobs < 0) die(_("negative values not allowed for submodule.fetchJobs")); + if (!fetchjobs) + fetchjobs = online_cpus(); return fetchjobs; } diff --git a/t/t5526-fetch-submodules.sh b/t/t5526-fetch-submodules.sh index a301b56db89..a5f494dfcfb 100755 --- a/t/t5526-fetch-submodules.sh +++ b/t/t5526-fetch-submodules.sh @@ -714,7 +714,13 @@ test_expect_success 'fetching submodules respects parallel settings' ' GIT_TRACE=$(pwd)/trace.out git fetch && grep "8 tasks" trace.out && GIT_TRACE=$(pwd)/trace.out git fetch --jobs 9 && - grep "9 tasks" trace.out + grep "9 tasks" trace.out && + >trace.out && + + GIT_TRACE=$(pwd)/trace.out git -c submodule.fetchJobs=0 fetch && + grep "preparing to run up to [0-9]* tasks" trace.out && + ! grep "up to 0 tasks" trace.out && + >trace.out ) ' From patchwork Wed Oct 12 21:02:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 13005446 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 95F4EC4332F for ; Wed, 12 Oct 2022 21:04:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229732AbiJLVEF (ORCPT ); Wed, 12 Oct 2022 17:04:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57960 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229806AbiJLVDg (ORCPT ); Wed, 12 Oct 2022 17:03:36 -0400 Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com [IPv6:2a00:1450:4864:20::42a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4E42C1F9EA for ; Wed, 12 Oct 2022 14:03:31 -0700 (PDT) Received: by mail-wr1-x42a.google.com with SMTP id b4so27972289wrs.1 for ; Wed, 12 Oct 2022 14:03:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Qk22E+xafbfAv9PII0Hbn8O45XXGlAeR5o5RzT+CbaA=; b=lWzEj7ZCK4MisbBK7FErhvRAeRCBPTyKw21JXwaW7HCwp6OmEEvXiKT4P2xYo585A/ woX82ZRvdqD9h2FLJ8aeHaUKfxMamp3ruxZOxc8YILs6dst5z5BKZACoPtivqoQT6KOk 619vVtTdgAyI3FPI93KRoEkD2dpIEMdD7mscEUYkeXQxNclOpt4XjcSNf9z+DSYH7Bfg /7XMXQk7D0+1d16dgPJj6ECwTov/uSYpLlrt38F/sgSen9hSBjxCVh723FVcooPjeaoR 6wAlUz29jbZGd5Jc7GpeC1EizkooEn3LWLnz8MzoxKJgfxNu7SaAKhzNpipXjPWrBXFv 4Xvw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Qk22E+xafbfAv9PII0Hbn8O45XXGlAeR5o5RzT+CbaA=; b=aa15lz+YHk0a/kU0hULBZRnVjQOME3uWFts70/OHm3feZ/6lx6QloRSLaPG2NkXfQT afKgDrbTmh2BLq8piv2y9he10LxE+EcIsIJOy2AtD3V2GjNvUuIE4cmoDn8ihKCL+jpP Rkw000Cjxpte2l+sAvx58CgPUBy/bJXl6ZkwGDIZwoh1zVtxDbdVYaisEm10ysbkUFg0 9sMikM9kU+nwaQdylhhciYkZ5hphqSAnS9naifbWfv7ZQiJUXGoE8DX1Ws3mZNOUku5x H28euCs1pXlmWHEoO+4FuoIi7MZkYZpiPTRdky2ZbuLEf0P+v2s4HUAc2Fpw71vyt2d9 UktQ== X-Gm-Message-State: ACrzQf0L63pTiFz7ByDKoOkYa6Hm0u+oeZmhdj/hv4t2M4DRLBzg2nAh 7vrAC/DaYj7Fe2bO9LnNGmLFA0V6R4GUvA== X-Google-Smtp-Source: AMsMyM6rNV7lDZyUtck3JGPHyMvLuJnyrtvKzMsNbeEONUWa2XsoDRVnG9bUApv6psGsiGT3Vd+1xg== X-Received: by 2002:a5d:6508:0:b0:22e:1af4:57f9 with SMTP id x8-20020a5d6508000000b0022e1af457f9mr19097920wru.539.1665608608794; Wed, 12 Oct 2022 14:03:28 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id b6-20020adff246000000b00228a6ce17b4sm550079wrp.37.2022.10.12.14.03.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Oct 2022 14:03:27 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Calvin Wan , Emily Shaffer , Phillip Wood , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v3 06/15] run-command.c: use designated init for pp_init(), add "const" Date: Wed, 12 Oct 2022 23:02:25 +0200 Message-Id: X-Mailer: git-send-email 2.38.0.971.ge79ff6d20e7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Use a designated initializer to initialize those parts of pp_init() that don't need any conditionals for their initialization, this sets us on a path to pp_init() itself into mostly a validation and allocation function. Since we're doing that we can add "const" to some of the members of the "struct parallel_processes", which helps to clarify and self-document this code. E.g. we never alter the "data" pointer we pass t user callbacks, nor (after the preceding change to stop invoking online_cpus()) do we change "max_processes", the same goes for the "ungroup" option. We can also do away with a call to strbuf_init() in favor of macro initialization, and to rely on other fields being NULL'd or zero'd. Making members of a struct "const" rather that the pointer to the struct itself is usually painful, as e.g. it precludes us from incrementally setting up the structure. In this case we only set it up with the assignment in run_process_parallel() and pp_init(), and don't pass the struct pointer around as "const", so making individual members "const" is worth the potential hassle for extra safety. Signed-off-by: Ævar Arnfjörð Bjarmason --- run-command.c | 34 ++++++++++++++-------------------- 1 file changed, 14 insertions(+), 20 deletions(-) diff --git a/run-command.c b/run-command.c index 14ea409375a..b69deb1cc53 100644 --- a/run-command.c +++ b/run-command.c @@ -1498,9 +1498,9 @@ enum child_state { int run_processes_parallel_ungroup; struct parallel_processes { - void *data; + void *const data; - size_t max_processes; + const size_t max_processes; size_t nr_processes; get_next_task_fn get_next_task; @@ -1520,7 +1520,7 @@ struct parallel_processes { struct pollfd *pfd; unsigned shutdown : 1; - unsigned ungroup : 1; + const unsigned ungroup : 1; size_t output_owner; struct strbuf buffered_output; /* of finished children */ @@ -1558,21 +1558,18 @@ static void handle_children_on_signal(int signo) } static void pp_init(struct parallel_processes *pp, - size_t n, get_next_task_fn get_next_task, start_failure_fn start_failure, - task_finished_fn task_finished, - void *data, int ungroup) + task_finished_fn task_finished) { + const size_t n = pp->max_processes; + if (!n) BUG("you must provide a non-zero number of processes!"); - pp->max_processes = n; - trace_printf("run_processes_parallel: preparing to run up to %"PRIuMAX" tasks", (uintmax_t)n); - pp->data = data; if (!get_next_task) BUG("you need to specify a get_next_task function"); pp->get_next_task = get_next_task; @@ -1580,16 +1577,9 @@ static void pp_init(struct parallel_processes *pp, pp->start_failure = start_failure ? start_failure : default_start_failure; pp->task_finished = task_finished ? task_finished : default_task_finished; - pp->nr_processes = 0; - pp->output_owner = 0; - pp->shutdown = 0; - pp->ungroup = ungroup; CALLOC_ARRAY(pp->children, n); - if (pp->ungroup) - pp->pfd = NULL; - else + if (!pp->ungroup) CALLOC_ARRAY(pp->pfd, n); - strbuf_init(&pp->buffered_output, 0); for (size_t i = 0; i < n; i++) { strbuf_init(&pp->children[i].err, 0); @@ -1789,13 +1779,17 @@ void run_processes_parallel(size_t n, int output_timeout = 100; int spawn_cap = 4; int ungroup = run_processes_parallel_ungroup; - struct parallel_processes pp; + struct parallel_processes pp = { + .max_processes = n, + .data = pp_cb, + .buffered_output = STRBUF_INIT, + .ungroup = ungroup, + }; /* unset for the next API user */ run_processes_parallel_ungroup = 0; - pp_init(&pp, n, get_next_task, start_failure, task_finished, pp_cb, - ungroup); + pp_init(&pp, get_next_task, start_failure, task_finished); while (1) { for (i = 0; i < spawn_cap && !pp.shutdown && From patchwork Wed Oct 12 21:02:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 13005448 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EC177C433FE for ; Wed, 12 Oct 2022 21:04:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229828AbiJLVEJ (ORCPT ); Wed, 12 Oct 2022 17:04:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57936 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229865AbiJLVDi (ORCPT ); Wed, 12 Oct 2022 17:03:38 -0400 Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com [IPv6:2a00:1450:4864:20::430]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5FE6362FC for ; Wed, 12 Oct 2022 14:03:31 -0700 (PDT) Received: by mail-wr1-x430.google.com with SMTP id bk15so27930085wrb.13 for ; Wed, 12 Oct 2022 14:03:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=EiOyptwF5GB4fZFtwp91yLbQbuiKvCWrAbIwl9ON5fU=; b=VOPM+aIMEE8WbpQGZjY1KMoBOd8t/j9iK2Pd0byIzgdqYuhOLzto0joNeqKU82xBhH bH0d+w/2w5RJGL3rgyF/vSoh36D49m3qgd0uqw/nRdHgByE6Dd8t3tE3ejFd3+bbp3xf 4pnu2k/eOxIRPFBVunSkNeejgvBDerxRegPlqm/32AnKZvdA7RwjCO2UuFNyggWMMH6E 4gdEFEhSdN8aW7gnZWXW1v5ERGKcMK7dxiWMSnCJ/jvNhSLXqRX9zFnAozQhEa+uXcji TkxaoWdxOr8ZBLesysScHTYsmtv7ET2+erYTLrIQVXBRFdbEyONtna1lDc4mGYnvBvXl h6kg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=EiOyptwF5GB4fZFtwp91yLbQbuiKvCWrAbIwl9ON5fU=; b=xbvGjm+wElLi8StdLqxHMS3Zy1rIdq/JdJgkuwfV4EZfZ20XnYgjIn3TW+lSk3l9i+ XYHekn1IQcr6/YwrGUeshbsyZBrggrsqachaheBSSha1SRk+/vLEzxdbREK3Ky+hhAmv YGX3IcXd025LIRaI9GB1UShu4kTLN2KDsyKIV6y9bimVj07QYTq67VU+Yvrhv0rleB04 tNdQUn3KyDoTTHRapNidllhr5ajk2rM4WfEYYUzHxe0vW48VOayA+MgEbUwc0gqyfprH SjIETA/kSzwjshHyoU0v1UojeLf5rzH6DYHUvrg4U+YuqDQbXsU0/dDDUyqIQkmSMIpg t6/A== X-Gm-Message-State: ACrzQf0/8xlaV0FmX1ZxOm5K8OW5z2cNXqDt0gWLQtGW0apFTb2O+FIq F91l4uG3iqXvkQkRCBvYYlwvD304/dH5NQ== X-Google-Smtp-Source: AMsMyM6H8IqVgp470Z1deAu60mOLP8RYA4EHqzGNA6wx/kKjsT2Qe6HJb5YV0+EBnS3YurxMrpUCNA== X-Received: by 2002:adf:fa10:0:b0:22e:4b8d:81f8 with SMTP id m16-20020adffa10000000b0022e4b8d81f8mr19357244wrr.135.1665608609752; Wed, 12 Oct 2022 14:03:29 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id b6-20020adff246000000b00228a6ce17b4sm550079wrp.37.2022.10.12.14.03.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Oct 2022 14:03:29 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Calvin Wan , Emily Shaffer , Phillip Wood , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v3 07/15] run-command API: have run_process_parallel() take an "opts" struct Date: Wed, 12 Oct 2022 23:02:26 +0200 Message-Id: X-Mailer: git-send-email 2.38.0.971.ge79ff6d20e7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org As noted in fd3aaf53f71 (run-command: add an "ungroup" option to run_process_parallel(), 2022-06-07) which added the "ungroup" passing it to "run_process_parallel()" via the global "run_processes_parallel_ungroup" variable was a compromise to get the smallest possible regression fix for "maint" at the time. This follow-up to that is a start at passing that parameter and others via a new "struct run_process_parallel_opts", as the earlier version[1] of what became fd3aaf53f71 did. Since we need to change all of the occurrences of "n" to "opt->SOMETHING" let's take the opportunity and rename the terse "n" to "processes". We could also have picked "max_processes", "jobs", "threads" etc., but as the API is named "run_processes_parallel()" let's go with "processes". Since the new "run_processes_parallel()" function is able to take an optional "tr2_category" and "tr2_label" via the struct we can at this point migrate all of the users of "run_processes_parallel_tr2()" over to it. But let's not migrate all the API users yet, only the two users that passed the "ungroup" parameter via the "run_processes_parallel_ungroup" global 1. https://lore.kernel.org/git/cover-v2-0.8-00000000000-20220518T195858Z-avarab@gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason --- hook.c | 23 +++++++----- run-command.c | 54 ++++++++++++++++------------ run-command.h | 72 ++++++++++++++++++++++++++++--------- t/helper/test-run-command.c | 31 ++++++++++------ 4 files changed, 121 insertions(+), 59 deletions(-) diff --git a/hook.c b/hook.c index a493939a4fc..a4fa1031f28 100644 --- a/hook.c +++ b/hook.c @@ -114,8 +114,20 @@ int run_hooks_opt(const char *hook_name, struct run_hooks_opt *options) .options = options, }; const char *const hook_path = find_hook(hook_name); - int jobs = 1; int ret = 0; + const struct run_process_parallel_opts opts = { + .tr2_category = "hook", + .tr2_label = hook_name, + + .processes = 1, + .ungroup = 1, + + .get_next_task = pick_next_hook, + .start_failure = notify_start_failure, + .task_finished = notify_hook_finished, + + .data = &cb_data, + }; if (!options) BUG("a struct run_hooks_opt must be provided to run_hooks"); @@ -137,14 +149,7 @@ int run_hooks_opt(const char *hook_name, struct run_hooks_opt *options) cb_data.hook_path = abs_path.buf; } - run_processes_parallel_ungroup = 1; - run_processes_parallel_tr2(jobs, - pick_next_hook, - notify_start_failure, - notify_hook_finished, - &cb_data, - "hook", - hook_name); + run_processes_parallel(&opts); ret = cb_data.rc; cleanup: strbuf_release(&abs_path); diff --git a/run-command.c b/run-command.c index b69deb1cc53..2858ec7bef5 100644 --- a/run-command.c +++ b/run-command.c @@ -1496,7 +1496,6 @@ enum child_state { GIT_CP_WAIT_CLEANUP, }; -int run_processes_parallel_ungroup; struct parallel_processes { void *const data; @@ -1558,11 +1557,12 @@ static void handle_children_on_signal(int signo) } static void pp_init(struct parallel_processes *pp, - get_next_task_fn get_next_task, - start_failure_fn start_failure, - task_finished_fn task_finished) + const struct run_process_parallel_opts *opts) { - const size_t n = pp->max_processes; + const size_t n = opts->processes; + get_next_task_fn get_next_task = opts->get_next_task; + start_failure_fn start_failure = opts->start_failure; + task_finished_fn task_finished = opts->task_finished; if (!n) BUG("you must provide a non-zero number of processes!"); @@ -1769,27 +1769,27 @@ static int pp_collect_finished(struct parallel_processes *pp) return result; } -void run_processes_parallel(size_t n, - get_next_task_fn get_next_task, - start_failure_fn start_failure, - task_finished_fn task_finished, - void *pp_cb) +void run_processes_parallel(const struct run_process_parallel_opts *opts) { int i, code; int output_timeout = 100; int spawn_cap = 4; - int ungroup = run_processes_parallel_ungroup; struct parallel_processes pp = { - .max_processes = n, - .data = pp_cb, + .max_processes = opts->processes, + .data = opts->data, .buffered_output = STRBUF_INIT, - .ungroup = ungroup, + .ungroup = opts->ungroup, }; + /* options */ + const char *tr2_category = opts->tr2_category; + const char *tr2_label = opts->tr2_label; + const int do_trace2 = tr2_category && tr2_label; - /* unset for the next API user */ - run_processes_parallel_ungroup = 0; + if (do_trace2) + trace2_region_enter_printf(tr2_category, tr2_label, NULL, + "max:%d", opts->processes); - pp_init(&pp, get_next_task, start_failure, task_finished); + pp_init(&pp, opts); while (1) { for (i = 0; i < spawn_cap && !pp.shutdown && @@ -1806,7 +1806,7 @@ void run_processes_parallel(size_t n, } if (!pp.nr_processes) break; - if (ungroup) { + if (opts->ungroup) { for (size_t i = 0; i < pp.max_processes; i++) pp.children[i].state = GIT_CP_WAIT_CLEANUP; } else { @@ -1822,19 +1822,27 @@ void run_processes_parallel(size_t n, } pp_cleanup(&pp); + + if (do_trace2) + trace2_region_leave(tr2_category, tr2_label, NULL); } -void run_processes_parallel_tr2(size_t n, get_next_task_fn get_next_task, +void run_processes_parallel_tr2(size_t processes, get_next_task_fn get_next_task, start_failure_fn start_failure, task_finished_fn task_finished, void *pp_cb, const char *tr2_category, const char *tr2_label) { - trace2_region_enter_printf(tr2_category, tr2_label, NULL, "max:%d", n); + const struct run_process_parallel_opts opts = { + .tr2_category = tr2_category, + .tr2_label = tr2_label, + .processes = processes, - run_processes_parallel(n, get_next_task, start_failure, - task_finished, pp_cb); + .get_next_task = get_next_task, + .start_failure = start_failure, + .task_finished = task_finished, + }; - trace2_region_leave(tr2_category, tr2_label, NULL); + run_processes_parallel(&opts); } int run_auto_maintenance(int quiet) diff --git a/run-command.h b/run-command.h index 6f7604e1146..aabdaf684db 100644 --- a/run-command.h +++ b/run-command.h @@ -459,17 +459,64 @@ typedef int (*task_finished_fn)(int result, void *pp_task_cb); /** - * Runs up to n processes at the same time. Whenever a process can be - * started, the callback get_next_task_fn is called to obtain the data + * Option used by run_processes_parallel(), { 0 }-initialized means no + * options. + */ +struct run_process_parallel_opts +{ + /** + * tr2_category & tr2_label: sets the trace2 category and label for + * logging. These must either be unset, or both of them must be set. + */ + const char *tr2_category; + const char *tr2_label; + + /** + * processes: see 'processes' in run_processes_parallel() below. + */ + size_t processes; + + /** + * ungroup: see 'ungroup' in run_processes_parallel() below. + */ + unsigned int ungroup:1; + + /** + * get_next_task: See get_next_task_fn() above. This must be + * specified. + */ + get_next_task_fn get_next_task; + + /** + * start_failure: See start_failure_fn() above. This can be + * NULL to omit any special handling. + */ + start_failure_fn start_failure; + + /** + * task_finished: See task_finished_fn() above. This can be + * NULL to omit any special handling. + */ + task_finished_fn task_finished; + + /** + * data: user data, will be passed as "pp_cb" to the callback + * parameters. + */ + void *data; +}; + +/** + * Options are passed via the "struct run_process_parallel_opts" above. + * + * Runs N 'processes' at the same time. Whenever a process can be + * started, the callback opts.get_next_task is called to obtain the data * required to start another child process. * * The children started via this function run in parallel. Their output * (both stdout and stderr) is routed to stderr in a manner that output * from different tasks does not interleave (but see "ungroup" below). * - * start_failure_fn and task_finished_fn can be NULL to omit any - * special handling. - * * If the "ungroup" option isn't specified, the API will set the * "stdout_to_stderr" parameter in "struct child_process" and provide * the callbacks with a "struct strbuf *out" parameter to write output @@ -479,19 +526,10 @@ typedef int (*task_finished_fn)(int result, * NULL "struct strbuf *out" parameter, and are responsible for * emitting their own output, including dealing with any race * conditions due to writing in parallel to stdout and stderr. - * The "ungroup" option can be enabled by setting the global - * "run_processes_parallel_ungroup" to "1" before invoking - * run_processes_parallel(), it will be set back to "0" as soon as the - * API reads that setting. */ -extern int run_processes_parallel_ungroup; -void run_processes_parallel(size_t n, - get_next_task_fn, - start_failure_fn, - task_finished_fn, - void *pp_cb); -void run_processes_parallel_tr2(size_t n, get_next_task_fn, start_failure_fn, - task_finished_fn, void *pp_cb, +void run_processes_parallel(const struct run_process_parallel_opts *opts); +void run_processes_parallel_tr2(size_t processes, get_next_task_fn, + start_failure_fn, task_finished_fn, void *pp_cb, const char *tr2_category, const char *tr2_label); /** diff --git a/t/helper/test-run-command.c b/t/helper/test-run-command.c index ee509aefa2f..3ecb830f4a8 100644 --- a/t/helper/test-run-command.c +++ b/t/helper/test-run-command.c @@ -136,7 +136,7 @@ static const char * const testsuite_usage[] = { static int testsuite(int argc, const char **argv) { struct testsuite suite = TESTSUITE_INIT; - int max_jobs = 1, i, ret; + int max_jobs = 1, i, ret = 0; DIR *dir; struct dirent *d; struct option options[] = { @@ -152,6 +152,12 @@ static int testsuite(int argc, const char **argv) "write JUnit-style XML files"), OPT_END() }; + struct run_process_parallel_opts opts = { + .get_next_task = next_test, + .start_failure = test_failed, + .task_finished = test_finished, + .data = &suite, + }; argc = parse_options(argc, argv, NULL, options, testsuite_usage, PARSE_OPT_STOP_AT_NON_OPTION); @@ -192,8 +198,8 @@ static int testsuite(int argc, const char **argv) fprintf(stderr, "Running %"PRIuMAX" tests (%d at a time)\n", (uintmax_t)suite.tests.nr, max_jobs); - run_processes_parallel(max_jobs, next_test, test_failed, - test_finished, &suite); + opts.processes = max_jobs; + run_processes_parallel(&opts); if (suite.failed.nr > 0) { ret = 1; @@ -206,7 +212,7 @@ static int testsuite(int argc, const char **argv) string_list_clear(&suite.tests, 0); string_list_clear(&suite.failed, 0); - return !!ret; + return ret; } static uint64_t my_random_next = 1234; @@ -382,6 +388,9 @@ int cmd__run_command(int argc, const char **argv) struct child_process proc = CHILD_PROCESS_INIT; int jobs; int ret; + struct run_process_parallel_opts opts = { + .data = &proc, + }; if (argc > 1 && !strcmp(argv[1], "testsuite")) return testsuite(argc - 1, argv + 1); @@ -427,7 +436,7 @@ int cmd__run_command(int argc, const char **argv) if (!strcmp(argv[1], "--ungroup")) { argv += 1; argc -= 1; - run_processes_parallel_ungroup = 1; + opts.ungroup = 1; } jobs = atoi(argv[2]); @@ -435,18 +444,20 @@ int cmd__run_command(int argc, const char **argv) strvec_pushv(&proc.args, (const char **)argv + 3); if (!strcmp(argv[1], "run-command-parallel")) { - run_processes_parallel(jobs, parallel_next, NULL, NULL, &proc); + opts.get_next_task = parallel_next; } else if (!strcmp(argv[1], "run-command-abort")) { - run_processes_parallel(jobs, parallel_next, NULL, - task_finished, &proc); + opts.get_next_task = parallel_next; + opts.task_finished = task_finished; } else if (!strcmp(argv[1], "run-command-no-jobs")) { - run_processes_parallel(jobs, no_job, NULL, task_finished, - &proc); + opts.get_next_task = no_job; + opts.task_finished = task_finished; } else { ret = 1; fprintf(stderr, "check usage\n"); goto cleanup; } + opts.processes = jobs; + run_processes_parallel(&opts); ret = 0; cleanup: child_process_clear(&proc); From patchwork Wed Oct 12 21:02:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 13005447 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5FA91C4332F for ; Wed, 12 Oct 2022 21:04:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229793AbiJLVEH (ORCPT ); Wed, 12 Oct 2022 17:04:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58022 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229872AbiJLVDi (ORCPT ); Wed, 12 Oct 2022 17:03:38 -0400 Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com [IPv6:2a00:1450:4864:20::430]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9117D4F6AB for ; Wed, 12 Oct 2022 14:03:33 -0700 (PDT) Received: by mail-wr1-x430.google.com with SMTP id j16so27954906wrh.5 for ; Wed, 12 Oct 2022 14:03:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MHCCJOqGjSoeHoqDQ8V5ZtNIvPBBNJ+hIkGAq/Fcg5g=; b=d8LyOz4bFE9qZ1xUVmpG2EKWT5pdYk4tVoBDLXsaVhsyxsVPxCOBVIkCl+F7bxD//y CXNlQTRFNMn+NzOCG7m37Nrokh8/uvNQEpcuq8o9m4O/llsPSkMQDsfei67TaqwGJ/k3 lzC9Y/HfQSTvhQnZ8R2/+PkInqYblJS9XZ3QCClcmib31JiHoy1HX2Dm42muyittNE/y oAedXSAEG35okoBRTUH2jHzaLz4QRPPPM2HgkKRPyp4I34hoTXrCUa6Mv1cCsv5uiEvQ 2llt6dIX0xvWozsIZiHosL5c7czV5J73UHULvh/sVkc8Qv7ec0o7MbCt+yc5mL7siiGn uKzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MHCCJOqGjSoeHoqDQ8V5ZtNIvPBBNJ+hIkGAq/Fcg5g=; b=kl7O3kKXKMbjvRoUoGxaBlf/5h+zDY2mqVeok7Ju0xFLlGnSYpOCcP78hvoXwRjdUZ aKwwyeY4VJJiuRKD9wSqcpOH8FShypeIlcHoV3G7oxClibKsgskA5KOoj4qOYc7XU5eg cwIu/OYEvfbB1EjJoaCP9I8dN3hF5McB+SGdu4r31BKrVaFfg1msvb1uWZlYF8agA0rA x5BtBnQqcJOffO3ral/d16JFVeYMyrIUON5Eovo2EIFx5qMPIMaQgWAleUg6W81Fvnaz py4gjM9bploVXQRxDEtzUWWaBCDykkRb7IYHWUfq8IH6j3lsMoD7YHhN6b0HWB7GEUMO 1Gvw== X-Gm-Message-State: ACrzQf29zkasChkJPSSmRjeRY1ZZMXEgoAlY3s+c/6g8h47JB7wMCFqk +8U4Ibdo9JZSaKQ2kWKE+K5ThQjXf38whg== X-Google-Smtp-Source: AMsMyM74FVa8H4Oi9udUBn64ihJp7cVJM1QSe50j/NvQXDG3Ot+yUoJmY36hxQC21rhxpi7q52fb0Q== X-Received: by 2002:a5d:59ae:0:b0:230:24a1:f295 with SMTP id p14-20020a5d59ae000000b0023024a1f295mr12595884wrr.457.1665608611007; Wed, 12 Oct 2022 14:03:31 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id b6-20020adff246000000b00228a6ce17b4sm550079wrp.37.2022.10.12.14.03.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Oct 2022 14:03:30 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Calvin Wan , Emily Shaffer , Phillip Wood , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v3 08/15] run-command API: move *_tr2() users to "run_processes_parallel()" Date: Wed, 12 Oct 2022 23:02:27 +0200 Message-Id: X-Mailer: git-send-email 2.38.0.971.ge79ff6d20e7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Have the users of the "run_processes_parallel_tr2()" function use "run_processes_parallel()" instead. In preceding commits the latter was refactored to take a "struct run_process_parallel_opts" argument, since the only reason for "run_processes_parallel_tr2()" to exist was to take arguments that are now a part of that struct we can do away with it. See ee4512ed481 (trace2: create new combined trace facility, 2019-02-22) for the addition of the "*_tr2()" variant of the function, it was used by every caller except "t/helper/test-run-command.c".. Signed-off-by: Ævar Arnfjörð Bjarmason --- builtin/fetch.c | 18 ++++++++++++------ builtin/submodule--helper.c | 16 ++++++++++++---- run-command.c | 18 ------------------ run-command.h | 3 --- submodule.c | 18 ++++++++++++------ 5 files changed, 36 insertions(+), 37 deletions(-) diff --git a/builtin/fetch.c b/builtin/fetch.c index 82f1da14ec1..b06e454cbdd 100644 --- a/builtin/fetch.c +++ b/builtin/fetch.c @@ -1953,15 +1953,21 @@ static int fetch_multiple(struct string_list *list, int max_children) if (max_children != 1 && list->nr != 1) { struct parallel_fetch_state state = { argv.v, list, 0, 0 }; + const struct run_process_parallel_opts opts = { + .tr2_category = "fetch", + .tr2_label = "parallel/fetch", + + .processes = max_children, + + .get_next_task = &fetch_next_remote, + .start_failure = &fetch_failed_to_start, + .task_finished = &fetch_finished, + .data = &state, + }; strvec_push(&argv, "--end-of-options"); - run_processes_parallel_tr2(max_children, - &fetch_next_remote, - &fetch_failed_to_start, - &fetch_finished, - &state, - "fetch", "parallel/fetch"); + run_processes_parallel(&opts); result = state.result; } else for (i = 0; i < list->nr; i++) { diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c index 0b4acb442b2..df526525c1e 100644 --- a/builtin/submodule--helper.c +++ b/builtin/submodule--helper.c @@ -2567,12 +2567,20 @@ static int update_submodules(struct update_data *update_data) { int i, ret = 0; struct submodule_update_clone suc = SUBMODULE_UPDATE_CLONE_INIT; + const struct run_process_parallel_opts opts = { + .tr2_category = "submodule", + .tr2_label = "parallel/update", + + .processes = update_data->max_jobs, + + .get_next_task = update_clone_get_next_task, + .start_failure = update_clone_start_failure, + .task_finished = update_clone_task_finished, + .data = &suc, + }; suc.update_data = update_data; - run_processes_parallel_tr2(suc.update_data->max_jobs, update_clone_get_next_task, - update_clone_start_failure, - update_clone_task_finished, &suc, "submodule", - "parallel/update"); + run_processes_parallel(&opts); /* * We saved the output and put it out all at once now. diff --git a/run-command.c b/run-command.c index 2858ec7bef5..646ea22e399 100644 --- a/run-command.c +++ b/run-command.c @@ -1827,24 +1827,6 @@ void run_processes_parallel(const struct run_process_parallel_opts *opts) trace2_region_leave(tr2_category, tr2_label, NULL); } -void run_processes_parallel_tr2(size_t processes, get_next_task_fn get_next_task, - start_failure_fn start_failure, - task_finished_fn task_finished, void *pp_cb, - const char *tr2_category, const char *tr2_label) -{ - const struct run_process_parallel_opts opts = { - .tr2_category = tr2_category, - .tr2_label = tr2_label, - .processes = processes, - - .get_next_task = get_next_task, - .start_failure = start_failure, - .task_finished = task_finished, - }; - - run_processes_parallel(&opts); -} - int run_auto_maintenance(int quiet) { int enabled; diff --git a/run-command.h b/run-command.h index aabdaf684db..e3e1ea01ad9 100644 --- a/run-command.h +++ b/run-command.h @@ -528,9 +528,6 @@ struct run_process_parallel_opts * conditions due to writing in parallel to stdout and stderr. */ void run_processes_parallel(const struct run_process_parallel_opts *opts); -void run_processes_parallel_tr2(size_t processes, get_next_task_fn, - start_failure_fn, task_finished_fn, void *pp_cb, - const char *tr2_category, const char *tr2_label); /** * Convenience function which prepares env for a command to be run in a diff --git a/submodule.c b/submodule.c index bf7a2c79183..f7c71f1f4b1 100644 --- a/submodule.c +++ b/submodule.c @@ -1819,6 +1819,17 @@ int fetch_submodules(struct repository *r, { int i; struct submodule_parallel_fetch spf = SPF_INIT; + const struct run_process_parallel_opts opts = { + .tr2_category = "submodule", + .tr2_label = "parallel/fetch", + + .processes = max_parallel_jobs, + + .get_next_task = get_next_submodule, + .start_failure = fetch_start_failure, + .task_finished = fetch_finish, + .data = &spf, + }; spf.r = r; spf.command_line_option = command_line_option; @@ -1840,12 +1851,7 @@ int fetch_submodules(struct repository *r, calculate_changed_submodule_paths(r, &spf.changed_submodule_names); string_list_sort(&spf.changed_submodule_names); - run_processes_parallel_tr2(max_parallel_jobs, - get_next_submodule, - fetch_start_failure, - fetch_finish, - &spf, - "submodule", "parallel/fetch"); + run_processes_parallel(&opts); if (spf.submodules_with_errors.len > 0) fprintf(stderr, _("Errors during submodule fetch:\n%s"), From patchwork Wed Oct 12 21:02:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 13005449 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3F947C433FE for ; Wed, 12 Oct 2022 21:04:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229837AbiJLVEL (ORCPT ); Wed, 12 Oct 2022 17:04:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57768 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229927AbiJLVDj (ORCPT ); Wed, 12 Oct 2022 17:03:39 -0400 Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com [IPv6:2a00:1450:4864:20::42e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B84151A206 for ; Wed, 12 Oct 2022 14:03:34 -0700 (PDT) Received: by mail-wr1-x42e.google.com with SMTP id bu30so27945479wrb.8 for ; Wed, 12 Oct 2022 14:03:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=tkROGW5xLXvHujNX5ZUKJWZAD26ecJpu623pIbCn9VU=; b=iaqD3qZAjO9+QOB5jLL/jBEuSgwomrJpeX92Dv4onjL4J5URtlE9fGPmvNDxABqqJt HOaEAQqgxNnPAuYcWf3dNAnXsjhEJ3aPKD1ORtHyfDxzs9kXfJri//bVZfbXErWWYIEO jL2y6Bjq3xPLkvbKiwdpQkPHBQNyVt2G0j2cyDPFrFdRbY0UY/ltunTVGhAI6xmljKsU CYZ29mACgwL8aDRHV20D4efMSd8IiiH5SiHvSDHFZIQazTF9iZKiFJ/BLVjtggB/3Tkw hJvj58kO99EuyjIDpkn6A9vVjjLYphGHfUEySpVzYlDLeOdXQpmm2lsZeFWb2TRVEmOP xNyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=tkROGW5xLXvHujNX5ZUKJWZAD26ecJpu623pIbCn9VU=; b=K48xU9XcDDfc7fggjrsM/Zv3llGt7jnvjzPe6qoGMLm26vpA1wAebUK27ajpLosPL7 5K6XnW1RyK3AlMGLLWutcBcLn1Kcl5tf9+ZGCjqDTK/xETDvKOoxF8fr8Yy0moDk7/gW dsJIFhtuC5pNh5Dd+nDT1ciWRShfT4UwPksKv2NMNqmMd/3K205EGogrSJWALQhkaE/n ANcJOSG3nJuCdivUO4Cn+3U9ihsAcVZSXCHV1KsKf5J5mhd3X08Fgx9/sWyf5E6a0h6C Oe2jSzbwsHvwc3pwUkDtAs5x7IKetAyLRGoLv8brjRc+JJgNbD/9xGukvL6IYe9aqnIc UT8w== X-Gm-Message-State: ACrzQf3LeUx+DCHZWWUtsfeilvXbgypVTq8jcThs/hVIqIxDAPRYVYQG in/zqnxhujjocoYISzzaZivZIyoVLYv48g== X-Google-Smtp-Source: AMsMyM4AHbL92OYRuQCGSXoU9g14ChG2uL1CqOaNiKOFZOjl8eSB1CoJgKbCZVtxlBEKOJn3BVcl7g== X-Received: by 2002:a05:6000:1a88:b0:231:bc2a:4f1b with SMTP id f8-20020a0560001a8800b00231bc2a4f1bmr5046633wry.291.1665608612186; Wed, 12 Oct 2022 14:03:32 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id b6-20020adff246000000b00228a6ce17b4sm550079wrp.37.2022.10.12.14.03.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Oct 2022 14:03:31 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Calvin Wan , Emily Shaffer , Phillip Wood , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v3 09/15] run-command.c: make "struct parallel_processes" const if possible Date: Wed, 12 Oct 2022 23:02:28 +0200 Message-Id: X-Mailer: git-send-email 2.38.0.971.ge79ff6d20e7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Add a "const" to two "struct parallel_processes" parameters where we're not modifying anything in "pp". For kill_children() we'll call it from both the signal handler, and from run_processes_parallel() itself. Adding a "const" there makes it clear that we don't need to modify any state when killing our children. Signed-off-by: Ævar Arnfjörð Bjarmason --- run-command.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/run-command.c b/run-command.c index 646ea22e399..a72c4295adc 100644 --- a/run-command.c +++ b/run-command.c @@ -1540,7 +1540,7 @@ static int default_task_finished(int result, return 0; } -static void kill_children(struct parallel_processes *pp, int signo) +static void kill_children(const struct parallel_processes *pp, int signo) { for (size_t i = 0; i < pp->max_processes; i++) if (pp->children[i].state == GIT_CP_WORKING) @@ -1698,7 +1698,7 @@ static void pp_buffer_stderr(struct parallel_processes *pp, int output_timeout) } } -static void pp_output(struct parallel_processes *pp) +static void pp_output(const struct parallel_processes *pp) { size_t i = pp->output_owner; From patchwork Wed Oct 12 21:02:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 13005452 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6AA8EC4332F for ; Wed, 12 Oct 2022 21:04:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229891AbiJLVEU (ORCPT ); Wed, 12 Oct 2022 17:04:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58022 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230050AbiJLVDq (ORCPT ); Wed, 12 Oct 2022 17:03:46 -0400 Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com [IPv6:2a00:1450:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7C2EA65832 for ; Wed, 12 Oct 2022 14:03:35 -0700 (PDT) Received: by mail-wr1-x433.google.com with SMTP id j16so27954990wrh.5 for ; Wed, 12 Oct 2022 14:03:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=pyQuvVPLUhRkE12mMGAs/wCv7zpX+3e+wsxOYmbq5ic=; b=fOG1rtwi5M7Np6C496k4rXtSwA/54spu2F+G/sGw1nE0aU74z4iv65HOYWvEf4CQ2C Qd90T9AtCOFGrR3BQWvd81Xio9VHeScELNnC6R+MxYz0OfRRY69pnDbVd+4EusBowglB FwqWqxNdYp8oRKLNq+XKreF0nHwe4stoHjgxCaY6VMnJzyVi7r5PQGk6kMm0viWfr+wx LGqEQPiJ+x7r/Al+d9xLmXIN5+OTOoisIzrygRXgUNJPV3MLJ5BWJ005kvn4mH/MLJma A/3YUAb7EmYmY/0IxMChLdir/Pr+UE0g1Ad8P1mIKKzb7vXDJ1jJ6NcNR9AEmBpcHXjN EWLQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=pyQuvVPLUhRkE12mMGAs/wCv7zpX+3e+wsxOYmbq5ic=; b=2BQKVJyczkThpVGHgVdXbjA4SkCut3CIySj7hPXYKVnsr1ASsPSPzQUjz90MxOCAlt iPX4E49blUOofsItX6NkEDDlSLwzonI/mDi5czYsGIdK66nYKbdKn6GYbjjqHx6jQ9TN QZztvpwUWFUsKWYt/uvO3T7ImvA2SCzjOhNJpIkwyTfAeaOPktJTbRU8FP/4dPUjWeQX XL0Hjp360ASrAldpuBfoCh8TEqIiHT3sz/65t2493DGvOyzvLV2DxNE6Sb9MoT3BmIu+ B1g0m77zIJf3S31FIVJ9Y6orjYaXRUSLBhM32e80LTjidplSe42dBdD9otJ5MVDwGVQ8 vxxA== X-Gm-Message-State: ACrzQf1pFcioYA7f2hjTmgbCto63vEU+ww93+w9gBWgs8VzIOfPlPdbT wDLmYrx7xyTj2yuClm9WggTNACwkxglQeg== X-Google-Smtp-Source: AMsMyM7drO6zJLcsZXqIUmNAEyWHNoU2I/jFR0HjVWUbpudHEOZZxeiCFeAg6xhdpHpquYfYBEifDg== X-Received: by 2002:adf:fd04:0:b0:22e:4bf6:4a08 with SMTP id e4-20020adffd04000000b0022e4bf64a08mr20459593wrr.619.1665608613151; Wed, 12 Oct 2022 14:03:33 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id b6-20020adff246000000b00228a6ce17b4sm550079wrp.37.2022.10.12.14.03.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Oct 2022 14:03:32 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Calvin Wan , Emily Shaffer , Phillip Wood , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v3 10/15] run-command.c: don't copy *_fn to "struct parallel_processes" Date: Wed, 12 Oct 2022 23:02:29 +0200 Message-Id: X-Mailer: git-send-email 2.38.0.971.ge79ff6d20e7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org The only remaining reason for copying the callbacks in the "struct run_process_parallel_opts" over to the "struct parallel_processes" was to avoid two if/else statements in case the "start_failure" and "task_finished" callbacks were NULL. Let's handle those cases in pp_start_one() and pp_collect_finished() instead, and avoid the default_* stub functions, and the need to copy this data around. Organizing the code like this made more sense before the "struct run_parallel_parallel_opts" existed, as we'd have needed to pass each of these as a separate parameter. Signed-off-by: Ævar Arnfjörð Bjarmason --- run-command.c | 67 +++++++++++++++++++-------------------------------- 1 file changed, 25 insertions(+), 42 deletions(-) diff --git a/run-command.c b/run-command.c index a72c4295adc..e10b1b97390 100644 --- a/run-command.c +++ b/run-command.c @@ -1502,10 +1502,6 @@ struct parallel_processes { const size_t max_processes; size_t nr_processes; - get_next_task_fn get_next_task; - start_failure_fn start_failure; - task_finished_fn task_finished; - struct { enum child_state state; struct child_process process; @@ -1525,21 +1521,6 @@ struct parallel_processes { struct strbuf buffered_output; /* of finished children */ }; -static int default_start_failure(struct strbuf *out, - void *pp_cb, - void *pp_task_cb) -{ - return 0; -} - -static int default_task_finished(int result, - struct strbuf *out, - void *pp_cb, - void *pp_task_cb) -{ - return 0; -} - static void kill_children(const struct parallel_processes *pp, int signo) { for (size_t i = 0; i < pp->max_processes; i++) @@ -1560,9 +1541,6 @@ static void pp_init(struct parallel_processes *pp, const struct run_process_parallel_opts *opts) { const size_t n = opts->processes; - get_next_task_fn get_next_task = opts->get_next_task; - start_failure_fn start_failure = opts->start_failure; - task_finished_fn task_finished = opts->task_finished; if (!n) BUG("you must provide a non-zero number of processes!"); @@ -1570,12 +1548,8 @@ static void pp_init(struct parallel_processes *pp, trace_printf("run_processes_parallel: preparing to run up to %"PRIuMAX" tasks", (uintmax_t)n); - if (!get_next_task) + if (!opts->get_next_task) BUG("you need to specify a get_next_task function"); - pp->get_next_task = get_next_task; - - pp->start_failure = start_failure ? start_failure : default_start_failure; - pp->task_finished = task_finished ? task_finished : default_task_finished; CALLOC_ARRAY(pp->children, n); if (!pp->ungroup) @@ -1622,7 +1596,8 @@ static void pp_cleanup(struct parallel_processes *pp) * <0 no new job was started, user wishes to shutdown early. Use negative code * to signal the children. */ -static int pp_start_one(struct parallel_processes *pp) +static int pp_start_one(struct parallel_processes *pp, + const struct run_process_parallel_opts *opts) { size_t i; int code; @@ -1633,10 +1608,10 @@ static int pp_start_one(struct parallel_processes *pp) if (i == pp->max_processes) BUG("bookkeeping is hard"); - code = pp->get_next_task(&pp->children[i].process, - pp->ungroup ? NULL : &pp->children[i].err, - pp->data, - &pp->children[i].data); + code = opts->get_next_task(&pp->children[i].process, + pp->ungroup ? NULL : &pp->children[i].err, + pp->data, + &pp->children[i].data); if (!code) { if (!pp->ungroup) { strbuf_addbuf(&pp->buffered_output, &pp->children[i].err); @@ -1651,10 +1626,14 @@ static int pp_start_one(struct parallel_processes *pp) pp->children[i].process.no_stdin = 1; if (start_command(&pp->children[i].process)) { - code = pp->start_failure(pp->ungroup ? NULL : - &pp->children[i].err, - pp->data, - pp->children[i].data); + if (opts->start_failure) + code = opts->start_failure(pp->ungroup ? NULL : + &pp->children[i].err, + pp->data, + pp->children[i].data); + else + code = 0; + if (!pp->ungroup) { strbuf_addbuf(&pp->buffered_output, &pp->children[i].err); strbuf_reset(&pp->children[i].err); @@ -1709,7 +1688,8 @@ static void pp_output(const struct parallel_processes *pp) } } -static int pp_collect_finished(struct parallel_processes *pp) +static int pp_collect_finished(struct parallel_processes *pp, + const struct run_process_parallel_opts *opts) { int code; size_t i, n = pp->max_processes; @@ -1724,9 +1704,12 @@ static int pp_collect_finished(struct parallel_processes *pp) code = finish_command(&pp->children[i].process); - code = pp->task_finished(code, pp->ungroup ? NULL : - &pp->children[i].err, pp->data, - pp->children[i].data); + if (opts->task_finished) + code = opts->task_finished(code, pp->ungroup ? NULL : + &pp->children[i].err, pp->data, + pp->children[i].data); + else + code = 0; if (code) result = code; @@ -1795,7 +1778,7 @@ void run_processes_parallel(const struct run_process_parallel_opts *opts) i < spawn_cap && !pp.shutdown && pp.nr_processes < pp.max_processes; i++) { - code = pp_start_one(&pp); + code = pp_start_one(&pp, opts); if (!code) continue; if (code < 0) { @@ -1813,7 +1796,7 @@ void run_processes_parallel(const struct run_process_parallel_opts *opts) pp_buffer_stderr(&pp, output_timeout); pp_output(&pp); } - code = pp_collect_finished(&pp); + code = pp_collect_finished(&pp, opts); if (code) { pp.shutdown = 1; if (code < 0) From patchwork Wed Oct 12 21:02:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 13005451 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7D203C4332F for ; Wed, 12 Oct 2022 21:04:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229872AbiJLVEQ (ORCPT ); Wed, 12 Oct 2022 17:04:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57996 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230044AbiJLVDq (ORCPT ); Wed, 12 Oct 2022 17:03:46 -0400 Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com [IPv6:2a00:1450:4864:20::329]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E692A8BB83 for ; Wed, 12 Oct 2022 14:03:36 -0700 (PDT) Received: by mail-wm1-x329.google.com with SMTP id o20-20020a05600c4fd400b003b4a516c479so79687wmq.1 for ; Wed, 12 Oct 2022 14:03:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=FSMtaeEQbhLwIh/rYpvNzoxFLefkFRBeO5/fQGJSJm8=; b=S2DmaRBabFqd+f4yOeY0NpS/bcdqJcIyq8YyV+b3Xb9ZY1yF+wwY1REY4wN/rtUwlO gpZbaeA/PnZvy87ULaw2tToASLPp82oo7wAcJaKM1YpqDMQM/hXUceVyzv6jDnkTWYf3 79RE/L7ZrEuqzQaGqk5yCp38a3ZQ/aFDB6JGpPauhUyeXlbvZdMDwmmttfQVxb8OE0Vu Ftrqf1iy0Cfk8WRvZzKLAI7pdaMthTeTz3h6to3ZEd1/vkCXJt0g35UuXLphRgHNiA5N HtjriATJJO92RWcsMPZA6HvnWyjEL9IwT9Twvcsntsqo6aJKCiVmIwUDyYCqAFl+8C/h XhCQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FSMtaeEQbhLwIh/rYpvNzoxFLefkFRBeO5/fQGJSJm8=; b=1Y32Lvf29SJIqsN/yhpXftBx8h7cdk2AQN4fkz76XL5Pfgl6zE3JLPJaHZxSJ16mOQ NupCOeYgAZ3AY8o2GuE3+I6T43awbJiAdU5lTRNoIgoD+aPdVboF+U/nTPUsKHmaSiO6 rU+polGbcTdybDU4xTAk+/dDWte6T/xIOM6x2A7iz6wEgfxyjmMBeCgAQ+MkOkHZv+N+ uN3q866u0Rf3KQfDEnDCVuz2hA2ecEuKvWW4So6zMcgW9jqXdbkeQal+y0cVkWB2X9AF 2exfbYzXwjt25GLMjtRYv+PfIQmqI2hJNftmMTFWmhTzQOfz/yKNHoY9MsXfG00lxnEi Omcg== X-Gm-Message-State: ACrzQf1T7cZzanYm6NYMSGcsRWnorYyBsX6n60WkCq8M6TstPzitUlN6 YlaDKW+2Of2dXVMvF00MX41gccquR6ee2Q== X-Google-Smtp-Source: AMsMyM6RTU3gyM1H0VlILEGt/1/jyT34htQwK7S+yofz/JoEZaeCcQeaIYPRT8Wg8FcDSaCHjbCo5A== X-Received: by 2002:a7b:cbc6:0:b0:3c4:20c5:3fc5 with SMTP id n6-20020a7bcbc6000000b003c420c53fc5mr4077786wmi.9.1665608614154; Wed, 12 Oct 2022 14:03:34 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id b6-20020adff246000000b00228a6ce17b4sm550079wrp.37.2022.10.12.14.03.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Oct 2022 14:03:33 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Calvin Wan , Emily Shaffer , Phillip Wood , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v3 11/15] run-command.c: don't copy "ungroup" to "struct parallel_processes" Date: Wed, 12 Oct 2022 23:02:30 +0200 Message-Id: X-Mailer: git-send-email 2.38.0.971.ge79ff6d20e7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org As with the *_fn members removed in the preceding commit, let's not copy the "ungroup" member of the "struct run_process_parallel_opts" over to the "struct parallel_processes". Now that we're passing the "opts" down there's no reason to do so. This makes the code easier to follow, as we have a "const" attribute on the "struct run_process_parallel_opts", but not "struct parallel_processes". We do not alter the "ungroup" argument, so storing it in the non-const structure would make this control flow less obvious. Signed-off-by: Ævar Arnfjörð Bjarmason --- run-command.c | 18 ++++++++---------- 1 file changed, 8 insertions(+), 10 deletions(-) diff --git a/run-command.c b/run-command.c index e10b1b97390..19d5cff8c65 100644 --- a/run-command.c +++ b/run-command.c @@ -1515,7 +1515,6 @@ struct parallel_processes { struct pollfd *pfd; unsigned shutdown : 1; - const unsigned ungroup : 1; size_t output_owner; struct strbuf buffered_output; /* of finished children */ @@ -1552,7 +1551,7 @@ static void pp_init(struct parallel_processes *pp, BUG("you need to specify a get_next_task function"); CALLOC_ARRAY(pp->children, n); - if (!pp->ungroup) + if (!opts->ungroup) CALLOC_ARRAY(pp->pfd, n); for (size_t i = 0; i < n; i++) { @@ -1609,17 +1608,17 @@ static int pp_start_one(struct parallel_processes *pp, BUG("bookkeeping is hard"); code = opts->get_next_task(&pp->children[i].process, - pp->ungroup ? NULL : &pp->children[i].err, + opts->ungroup ? NULL : &pp->children[i].err, pp->data, &pp->children[i].data); if (!code) { - if (!pp->ungroup) { + if (!opts->ungroup) { strbuf_addbuf(&pp->buffered_output, &pp->children[i].err); strbuf_reset(&pp->children[i].err); } return 1; } - if (!pp->ungroup) { + if (!opts->ungroup) { pp->children[i].process.err = -1; pp->children[i].process.stdout_to_stderr = 1; } @@ -1627,14 +1626,14 @@ static int pp_start_one(struct parallel_processes *pp, if (start_command(&pp->children[i].process)) { if (opts->start_failure) - code = opts->start_failure(pp->ungroup ? NULL : + code = opts->start_failure(opts->ungroup ? NULL : &pp->children[i].err, pp->data, pp->children[i].data); else code = 0; - if (!pp->ungroup) { + if (!opts->ungroup) { strbuf_addbuf(&pp->buffered_output, &pp->children[i].err); strbuf_reset(&pp->children[i].err); } @@ -1705,7 +1704,7 @@ static int pp_collect_finished(struct parallel_processes *pp, code = finish_command(&pp->children[i].process); if (opts->task_finished) - code = opts->task_finished(code, pp->ungroup ? NULL : + code = opts->task_finished(code, opts->ungroup ? NULL : &pp->children[i].err, pp->data, pp->children[i].data); else @@ -1722,7 +1721,7 @@ static int pp_collect_finished(struct parallel_processes *pp, pp->pfd[i].fd = -1; child_process_init(&pp->children[i].process); - if (pp->ungroup) { + if (opts->ungroup) { ; /* no strbuf_*() work to do here */ } else if (i != pp->output_owner) { strbuf_addbuf(&pp->buffered_output, &pp->children[i].err); @@ -1761,7 +1760,6 @@ void run_processes_parallel(const struct run_process_parallel_opts *opts) .max_processes = opts->processes, .data = opts->data, .buffered_output = STRBUF_INIT, - .ungroup = opts->ungroup, }; /* options */ const char *tr2_category = opts->tr2_category; From patchwork Wed Oct 12 21:02:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 13005455 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B64E9C4332F for ; Wed, 12 Oct 2022 21:04:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229819AbiJLVE2 (ORCPT ); Wed, 12 Oct 2022 17:04:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58062 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230092AbiJLVDs (ORCPT ); Wed, 12 Oct 2022 17:03:48 -0400 Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com [IPv6:2a00:1450:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 99446BF72 for ; Wed, 12 Oct 2022 14:03:38 -0700 (PDT) Received: by mail-wr1-x433.google.com with SMTP id bp11so15213727wrb.9 for ; Wed, 12 Oct 2022 14:03:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=rwLhduUbt94tVAmhwExY2jaT6aoxaLjnNIoAAroiiHE=; b=KJKtOXoa86NwMeX7JrQSWk8fkCD6oyTEJp6hIo64QZhedKM1PLyA9vDzpTW7GEUqNr /de7+LR7+nfztgFGFXu8DXDs8UYz0an4xWKltG2nZdzGLG87gck4fbjCRSjhXdvaffPC 8PmLdFBfgtJmWjRLpEKrx3JESRS/6rQCyN/hvK/Pd0bcuExkkvE8FjU/ETdMmGeIP8Yq DGxNtPlgdufNTLsc89zWokNxJ1lgtxInu63+u4MAMLosS3fiOQy4duJP2IRBM73drLpq 5IldfLslOxdufXumQwHlQkbRKMHhVma7wG1CnX0QM7cXEwEVKaqTyYBru/+CugjGAMgJ QKSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=rwLhduUbt94tVAmhwExY2jaT6aoxaLjnNIoAAroiiHE=; b=q2p6NExktSyOT4zT3UqYtBlYk3xrTshJKOkE0+rvd4WnVY3HsEsNWCMfjzAfIupOqX C2iOWbSR/HB5hwL0/he8glKYKV7H2DmNpjD8yWcIgQu5Xegb6a5VmLFT4qfiWSKJatMR CjX+xAZ68ACuosAecqAO9gzuCftqVELW6tpBN7SUjNg9TThTdnPE1H/F2ONBBfwR0VY8 sDryoDs2LLaYetKmUETcAZSlcdX8Cxa3mIa5jxauX17nlVtIDMq28mh3ss4rnMpIEImZ q/+W9Jk79Z+r5BZvHfgfFwPNGytMZgOlnfl9u/Nuarr089zQFjYGwMI6m39HtRM7Lhdv k3YQ== X-Gm-Message-State: ACrzQf0eC3Ml6u7sp/2PZ/JzZRK7aAsmbkXtv3G239y70jdlxWo0qFCi BLAqBPNdybkYZBvyDPz2qLs8+RXZ+ulgOQ== X-Google-Smtp-Source: AMsMyM6mfUcCZl8REa628PJ9rbmsHdlNnr8Nw/y/GRqVDqbmpQH2Ax59Ct0Vu1KPKVVSXAeFQZOvqA== X-Received: by 2002:a05:6000:1ac9:b0:22e:3d0c:43ed with SMTP id i9-20020a0560001ac900b0022e3d0c43edmr19225819wry.34.1665608615386; Wed, 12 Oct 2022 14:03:35 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id b6-20020adff246000000b00228a6ce17b4sm550079wrp.37.2022.10.12.14.03.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Oct 2022 14:03:34 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Calvin Wan , Emily Shaffer , Phillip Wood , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v3 12/15] run-command.c: don't copy "data" to "struct parallel_processes" Date: Wed, 12 Oct 2022 23:02:31 +0200 Message-Id: X-Mailer: git-send-email 2.38.0.971.ge79ff6d20e7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org As with the *_fn members removed in a preceding commit, let's not copy the "data" member of the "struct run_process_parallel_opts" over to the "struct parallel_processes". Now that we're passing the "opts" down there's no reason to do so. This makes the code easier to follow, as we have a "const" attribute on the "struct run_process_parallel_opts", but not "struct parallel_processes". We do not alter the "ungroup" argument, so storing it in the non-const structure would make this control flow less obvious. Signed-off-by: Ævar Arnfjörð Bjarmason --- run-command.c | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/run-command.c b/run-command.c index 19d5cff8c65..d382c42f82d 100644 --- a/run-command.c +++ b/run-command.c @@ -1497,8 +1497,6 @@ enum child_state { }; struct parallel_processes { - void *const data; - const size_t max_processes; size_t nr_processes; @@ -1609,7 +1607,7 @@ static int pp_start_one(struct parallel_processes *pp, code = opts->get_next_task(&pp->children[i].process, opts->ungroup ? NULL : &pp->children[i].err, - pp->data, + opts->data, &pp->children[i].data); if (!code) { if (!opts->ungroup) { @@ -1628,7 +1626,7 @@ static int pp_start_one(struct parallel_processes *pp, if (opts->start_failure) code = opts->start_failure(opts->ungroup ? NULL : &pp->children[i].err, - pp->data, + opts->data, pp->children[i].data); else code = 0; @@ -1705,7 +1703,7 @@ static int pp_collect_finished(struct parallel_processes *pp, if (opts->task_finished) code = opts->task_finished(code, opts->ungroup ? NULL : - &pp->children[i].err, pp->data, + &pp->children[i].err, opts->data, pp->children[i].data); else code = 0; @@ -1758,7 +1756,6 @@ void run_processes_parallel(const struct run_process_parallel_opts *opts) int spawn_cap = 4; struct parallel_processes pp = { .max_processes = opts->processes, - .data = opts->data, .buffered_output = STRBUF_INIT, }; /* options */ From patchwork Wed Oct 12 21:02:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 13005450 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 51C7FC4332F for ; Wed, 12 Oct 2022 21:04:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229851AbiJLVEO (ORCPT ); Wed, 12 Oct 2022 17:04:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58006 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229971AbiJLVDl (ORCPT ); Wed, 12 Oct 2022 17:03:41 -0400 Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com [IPv6:2a00:1450:4864:20::32f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9963E4A118 for ; Wed, 12 Oct 2022 14:03:38 -0700 (PDT) Received: by mail-wm1-x32f.google.com with SMTP id c7-20020a05600c0ac700b003c6cad86f38so1914973wmr.2 for ; Wed, 12 Oct 2022 14:03:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=crgGSHXTomG+o2Ee5LO6r8XHkdaAmnrSdV1KNjX9cOI=; b=mz7BOeB4rmPwM7lX3dxvAksRIyAUVdBDjyV02UspLiYuQAchjUI++JCWlA3cQtZ60b oK5gjCT7DMb5sbjUa4J4KQrmYGystgRYIOjFCZOHvsRp7+K7vOsK+VSikAKtQbKEosJD RX6KH/+iWEJAdCGISDsPST/r44vpGbKb0e+znOASCQ3PpZJc5IO+AyIHlaCF1x5/b2Xw 5ZHkKwEu1Imo6R1r4gB2LFKjrVVPz9fvaqn3p1eYCTk7iy5VRiaA2FBFXAJ9O03WIxQ3 iyUI+GeUfJUrWU7NUmGTMhGDxV67sok0dJcuxEl7q1AvI0IJKEAkGGLJLLTYtPXoacNy frkg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=crgGSHXTomG+o2Ee5LO6r8XHkdaAmnrSdV1KNjX9cOI=; b=KyqMnK9/znCLy2KcU7t50NV1wgpKEg9Vjyu723uRlmWlX6aPPN7fbU5eCIxTQkrLPR Yi8Xf2SKq52+EAVigs34jtXZs9NuYfhX3le8bBHyq2eu1toV+AbJaPeMS/jc4H08sMnv niQFkpFmM9rSX6NwBO88RvwddKLsWB7GQk1gb5MtHtckR60W9SoSwNNQiELSEhn1Yba0 zpPCkyM3JEPeBSmXSoxM8gaKSy0EPRAqSg/EhZCL5kZB8jJMqI64iJ0MVYYFx3I9YvN9 HRnZgvAQTnTAgtQ7PvexFJf+sfv7EAh2XXQYPa9e3VPlHFVjBp34sLmBonnRk0y1Lg0z z9xA== X-Gm-Message-State: ACrzQf3YFAB02F+8BoFoT+WHh6USKWowtehMzq7tbwt+hHaeajp42aNj 8RAeC9w4pRJQw1TVAfwIU/0G8X5+uB+vZA== X-Google-Smtp-Source: AMsMyM6onsy0tPrLUe/pXr+Xo71ICDB1j2QG9VvUXPIliXYRtL9Dnb6gkUGoGaQgMWNHZ+RN510EsQ== X-Received: by 2002:a1c:6a08:0:b0:3c4:a83b:bc4 with SMTP id f8-20020a1c6a08000000b003c4a83b0bc4mr4014964wmc.115.1665608616414; Wed, 12 Oct 2022 14:03:36 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id b6-20020adff246000000b00228a6ce17b4sm550079wrp.37.2022.10.12.14.03.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Oct 2022 14:03:35 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Calvin Wan , Emily Shaffer , Phillip Wood , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v3 13/15] run-command.c: use "opts->processes", not "pp->max_processes" Date: Wed, 12 Oct 2022 23:02:32 +0200 Message-Id: X-Mailer: git-send-email 2.38.0.971.ge79ff6d20e7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Neither the "processes" nor "max_processes" members ever change after their initialization, and they're always equivalent, but some existing code used "pp->max_processes" when we were already passing the "opts" to the function, let's use the "opts" directly instead. Signed-off-by: Ævar Arnfjörð Bjarmason --- run-command.c | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/run-command.c b/run-command.c index d382c42f82d..719a5b58e29 100644 --- a/run-command.c +++ b/run-command.c @@ -1599,10 +1599,10 @@ static int pp_start_one(struct parallel_processes *pp, size_t i; int code; - for (i = 0; i < pp->max_processes; i++) + for (i = 0; i < opts->processes; i++) if (pp->children[i].state == GIT_CP_FREE) break; - if (i == pp->max_processes) + if (i == opts->processes) BUG("bookkeeping is hard"); code = opts->get_next_task(&pp->children[i].process, @@ -1689,14 +1689,14 @@ static int pp_collect_finished(struct parallel_processes *pp, const struct run_process_parallel_opts *opts) { int code; - size_t i, n = pp->max_processes; + size_t i; int result = 0; while (pp->nr_processes > 0) { - for (i = 0; i < pp->max_processes; i++) + for (i = 0; i < opts->processes; i++) if (pp->children[i].state == GIT_CP_WAIT_CLEANUP) break; - if (i == pp->max_processes) + if (i == opts->processes) break; code = finish_command(&pp->children[i].process); @@ -1725,6 +1725,8 @@ static int pp_collect_finished(struct parallel_processes *pp, strbuf_addbuf(&pp->buffered_output, &pp->children[i].err); strbuf_reset(&pp->children[i].err); } else { + const size_t n = opts->processes; + strbuf_write(&pp->children[i].err, stderr); strbuf_reset(&pp->children[i].err); @@ -1771,7 +1773,7 @@ void run_processes_parallel(const struct run_process_parallel_opts *opts) while (1) { for (i = 0; i < spawn_cap && !pp.shutdown && - pp.nr_processes < pp.max_processes; + pp.nr_processes < opts->processes; i++) { code = pp_start_one(&pp, opts); if (!code) @@ -1785,7 +1787,7 @@ void run_processes_parallel(const struct run_process_parallel_opts *opts) if (!pp.nr_processes) break; if (opts->ungroup) { - for (size_t i = 0; i < pp.max_processes; i++) + for (size_t i = 0; i < opts->processes; i++) pp.children[i].state = GIT_CP_WAIT_CLEANUP; } else { pp_buffer_stderr(&pp, output_timeout); From patchwork Wed Oct 12 21:02:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 13005453 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E5FE3C433FE for ; Wed, 12 Oct 2022 21:04:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229624AbiJLVEW (ORCPT ); Wed, 12 Oct 2022 17:04:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58284 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230081AbiJLVDr (ORCPT ); Wed, 12 Oct 2022 17:03:47 -0400 Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F0FDB13EAF for ; Wed, 12 Oct 2022 14:03:39 -0700 (PDT) Received: by mail-wr1-x434.google.com with SMTP id u10so27977623wrq.2 for ; Wed, 12 Oct 2022 14:03:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Y2qPtg4voyOuFwXp2Lzn6VX/8MTszyFp6nPZbwyk5+U=; b=IRm2Z0x0PNUmg5zaqY9NEQyrmTelYr+J9/mqAehjuNa2E1khBHVkHueNV4x4v+ChFu 2nK7Zo4+TdmyRwW5h24rwV+73PUCA/9wCoh+C/26KjBGDIsHW5UEpoAz3U+C6hkDe6dE HxD7icv6Tfl9NzUX5X7m+T/ba9BEu1RoYZ7fcIbuqWlRhss1Q3SqFvSL194r7zCEld3H RUM2Mp51QXFdQGNUWfghA7D3geZKfr8bhghnC+xuaNrkaLLgIj+WOWS7y3Xd7+tq2rPP na/dO9w4Z1G7icQ/Z9njrpD4enVA7KW0ibnmE9yHcOOArxR/lwLZLJC9+mShPiphSmCd ZLKQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Y2qPtg4voyOuFwXp2Lzn6VX/8MTszyFp6nPZbwyk5+U=; b=WUrOaT0U+9XGl5LWJgim1VBuJGUliL0/N00WywA5nRLbBRl+4P1GgUirZJBfFvO+ln R1NhCBCIDBs/fMEIebWN0MuWBt/Kmh9dmPPbpDrB/4RuGun7pxuavZz5prH26GWyfdo0 lFeA7BykDj/8qSaYfbumg3tyLNW0irvDg2wjyD41Wr1EdvWLp95THh7+UjjhamVm6Z8u Ot3qVNMSS8Sq4HMOa5rTf9XSpkUVPQyRNoWDogq6DdlnfDdqCw7MFwoMHkfEV8PlKHo7 dwEwpvJKvCzeJh6F7hSVbzG9LOZqM3HP2ZjUGi1LQ+O8B0If3S2kwt30x97lF9vABxQt 9lFg== X-Gm-Message-State: ACrzQf0Qjg9zfn42czkI8Ey1Q+n1xDtAbY9YER+JzI5ozn19b4lQoK5c kww4WPKkj3GWGirQsMAApdDM09cMF40qDw== X-Google-Smtp-Source: AMsMyM5U5EOLNMSf+EZgka/6Y5UhsiEoNy/PRcm1LAy4awRssBvL8d51XB5R6VIr6W62gz9zjeiHhA== X-Received: by 2002:adf:aad1:0:b0:22e:3667:d307 with SMTP id i17-20020adfaad1000000b0022e3667d307mr20344886wrc.359.1665608617482; Wed, 12 Oct 2022 14:03:37 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id b6-20020adff246000000b00228a6ce17b4sm550079wrp.37.2022.10.12.14.03.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Oct 2022 14:03:36 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Calvin Wan , Emily Shaffer , Phillip Wood , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v3 14/15] run-command.c: pass "opts" further down, and use "opts->processes" Date: Wed, 12 Oct 2022 23:02:33 +0200 Message-Id: X-Mailer: git-send-email 2.38.0.971.ge79ff6d20e7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Continue the migration away from the "max_processes" member of "struct parallel_processes" to the "processes" member of the "struct run_process_parallel_opts", in this case we needed to pass the "opts" further down into pp_cleanup() and pp_buffer_stderr(). Signed-off-by: Ævar Arnfjörð Bjarmason --- run-command.c | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/run-command.c b/run-command.c index 719a5b58e29..ce7966394d4 100644 --- a/run-command.c +++ b/run-command.c @@ -1565,10 +1565,11 @@ static void pp_init(struct parallel_processes *pp, sigchain_push_common(handle_children_on_signal); } -static void pp_cleanup(struct parallel_processes *pp) +static void pp_cleanup(struct parallel_processes *pp, + const struct run_process_parallel_opts *opts) { trace_printf("run_processes_parallel: done"); - for (size_t i = 0; i < pp->max_processes; i++) { + for (size_t i = 0; i < opts->processes; i++) { strbuf_release(&pp->children[i].err); child_process_clear(&pp->children[i].process); } @@ -1647,19 +1648,21 @@ static int pp_start_one(struct parallel_processes *pp, return 0; } -static void pp_buffer_stderr(struct parallel_processes *pp, int output_timeout) +static void pp_buffer_stderr(struct parallel_processes *pp, + const struct run_process_parallel_opts *opts, + int output_timeout) { int i; - while ((i = poll(pp->pfd, pp->max_processes, output_timeout)) < 0) { + while ((i = poll(pp->pfd, opts->processes, output_timeout) < 0)) { if (errno == EINTR) continue; - pp_cleanup(pp); + pp_cleanup(pp, opts); die_errno("poll"); } /* Buffer output from all pipes. */ - for (size_t i = 0; i < pp->max_processes; i++) { + for (size_t i = 0; i < opts->processes; i++) { if (pp->children[i].state == GIT_CP_WORKING && pp->pfd[i].revents & (POLLIN | POLLHUP)) { int n = strbuf_read_once(&pp->children[i].err, @@ -1790,7 +1793,7 @@ void run_processes_parallel(const struct run_process_parallel_opts *opts) for (size_t i = 0; i < opts->processes; i++) pp.children[i].state = GIT_CP_WAIT_CLEANUP; } else { - pp_buffer_stderr(&pp, output_timeout); + pp_buffer_stderr(&pp, opts, output_timeout); pp_output(&pp); } code = pp_collect_finished(&pp, opts); @@ -1801,7 +1804,7 @@ void run_processes_parallel(const struct run_process_parallel_opts *opts) } } - pp_cleanup(&pp); + pp_cleanup(&pp, opts); if (do_trace2) trace2_region_leave(tr2_category, tr2_label, NULL); From patchwork Wed Oct 12 21:02:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 13005454 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 54617C4332F for ; Wed, 12 Oct 2022 21:04:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229926AbiJLVE0 (ORCPT ); Wed, 12 Oct 2022 17:04:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57768 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230094AbiJLVDs (ORCPT ); Wed, 12 Oct 2022 17:03:48 -0400 Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 883D56583E for ; Wed, 12 Oct 2022 14:03:41 -0700 (PDT) Received: by mail-wr1-x42c.google.com with SMTP id r13so27926794wrj.11 for ; Wed, 12 Oct 2022 14:03:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Bh0Fg2Xhh1hoQEPYBziWMa+ayY/kNqB+EUXcNRVSGN8=; b=YDs9/9HyXEOJtO4JdKXMKKSzl/oyxaHqgmavdgenSVylA9/8ksx2YjcRe0q1LAOy4w Sb06vzCzJ3I3Xbait5TS+iXePNwGMC1ZwlhKhT1s1UV8eg/fRxNiAMRmg/XwaXyUYqvM V90JUqIHZgR/f0QncrczpT85Q0BOtPy2E7g1iUdVhoIxe6mRvHWpciy6flgFcqivzBm5 VF3jDd7byY35QAKR/fjYqtNBUfILpNVhhgKHsnQJF6xz0Kv81pFOXcEoV/zx8zg4tKdY qGxdDIjXuBvzuneiVa9vrv58sN24yOLjRSCt3G9X7agTxZz3AVOSmtZAYUhdZZl/HmzM 3k4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Bh0Fg2Xhh1hoQEPYBziWMa+ayY/kNqB+EUXcNRVSGN8=; b=4IzO2uWdhbcqPEsN9EAG5wp5MbQVjtFKyYNBFiwSr08L4EepyAe3t1+BT/Zhd9ZEGu Kk3k2o5ZEtRma3EXQAYSrFDuoD9IvfipovoQ3JIZHIb5zUUUcEmyPRn3bFFG2jMmD5e1 wVuxhwE16NQAMwO/Du6hzpjTwTnunpW74h58iGjSo3h67A1YvvMBJMXfRiS3ZtTxG/y2 bSS/bEuhhv3KEpSqnlpz7mZMEGIPNNC0FWFvjjr7Rb4sBgD2iq9IFnMUcDKqo1rd4GPb NNBPrgJCw8UQ5tXFOgR6OnrUne4KpbH+7odL3jMhENGiLa7kVRVY0w2+zVt3oPw1wzFi xHMg== X-Gm-Message-State: ACrzQf26q9N/hde8Nr8XP/f9kmsOoyQHMdAD29fp4L/uuyfRjKXK9Dr3 EpP5YTeX6O9sEZNQgqkbndmn60n+4Tcd/w== X-Google-Smtp-Source: AMsMyM4Q0HRL3x+fpLG2ELHVwblFAlbnOWjN0hMYylGtAdtCgCkpfZL/pavDK+ThSAnw9Ci8WQdiMw== X-Received: by 2002:adf:ecc1:0:b0:231:b755:e4b2 with SMTP id s1-20020adfecc1000000b00231b755e4b2mr5346560wro.196.1665608618433; Wed, 12 Oct 2022 14:03:38 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id b6-20020adff246000000b00228a6ce17b4sm550079wrp.37.2022.10.12.14.03.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Oct 2022 14:03:37 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Calvin Wan , Emily Shaffer , Phillip Wood , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v3 15/15] run-command.c: remove "max_processes", add "const" to signal() handler Date: Wed, 12 Oct 2022 23:02:34 +0200 Message-Id: X-Mailer: git-send-email 2.38.0.971.ge79ff6d20e7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org As with the *_fn members removed in a preceding commit, let's not copy the "processes" member of the "struct run_process_parallel_opts" over to the "struct parallel_processes". In this case we need the number of processes for the kill_children() function, which will be called from a signal handler. To do that adjust this code added in c553c72eed6 (run-command: add an asynchronous parallel child processor, 2015-12-15) so that we use a dedicated "struct parallel_processes_for_signal" for passing data to the signal handler, in addition to the "struct parallel_process" it'll now have access to our "opts" variable. Signed-off-by: Ævar Arnfjörð Bjarmason --- run-command.c | 37 ++++++++++++++++++++++++++----------- 1 file changed, 26 insertions(+), 11 deletions(-) diff --git a/run-command.c b/run-command.c index ce7966394d4..c772acd7431 100644 --- a/run-command.c +++ b/run-command.c @@ -1497,7 +1497,6 @@ enum child_state { }; struct parallel_processes { - const size_t max_processes; size_t nr_processes; struct { @@ -1518,24 +1517,38 @@ struct parallel_processes { struct strbuf buffered_output; /* of finished children */ }; -static void kill_children(const struct parallel_processes *pp, int signo) +struct parallel_processes_for_signal { + const struct run_process_parallel_opts *opts; + const struct parallel_processes *pp; +}; + +static void kill_children(const struct parallel_processes *pp, + const struct run_process_parallel_opts *opts, + int signo) { - for (size_t i = 0; i < pp->max_processes; i++) + for (size_t i = 0; i < opts->processes; i++) if (pp->children[i].state == GIT_CP_WORKING) kill(pp->children[i].process.pid, signo); } -static struct parallel_processes *pp_for_signal; +static void kill_children_signal(const struct parallel_processes_for_signal *pp_sig, + int signo) +{ + kill_children(pp_sig->pp, pp_sig->opts, signo); +} + +static struct parallel_processes_for_signal *pp_for_signal; static void handle_children_on_signal(int signo) { - kill_children(pp_for_signal, signo); + kill_children_signal(pp_for_signal, signo); sigchain_pop(signo); raise(signo); } static void pp_init(struct parallel_processes *pp, - const struct run_process_parallel_opts *opts) + const struct run_process_parallel_opts *opts, + struct parallel_processes_for_signal *pp_sig) { const size_t n = opts->processes; @@ -1561,7 +1574,9 @@ static void pp_init(struct parallel_processes *pp, } } - pp_for_signal = pp; + pp_sig->pp = pp; + pp_sig->opts = opts; + pp_for_signal = pp_sig; sigchain_push_common(handle_children_on_signal); } @@ -1759,8 +1774,8 @@ void run_processes_parallel(const struct run_process_parallel_opts *opts) int i, code; int output_timeout = 100; int spawn_cap = 4; + struct parallel_processes_for_signal pp_sig; struct parallel_processes pp = { - .max_processes = opts->processes, .buffered_output = STRBUF_INIT, }; /* options */ @@ -1772,7 +1787,7 @@ void run_processes_parallel(const struct run_process_parallel_opts *opts) trace2_region_enter_printf(tr2_category, tr2_label, NULL, "max:%d", opts->processes); - pp_init(&pp, opts); + pp_init(&pp, opts, &pp_sig); while (1) { for (i = 0; i < spawn_cap && !pp.shutdown && @@ -1783,7 +1798,7 @@ void run_processes_parallel(const struct run_process_parallel_opts *opts) continue; if (code < 0) { pp.shutdown = 1; - kill_children(&pp, -code); + kill_children(&pp, opts, -code); } break; } @@ -1800,7 +1815,7 @@ void run_processes_parallel(const struct run_process_parallel_opts *opts) if (code) { pp.shutdown = 1; if (code < 0) - kill_children(&pp, -code); + kill_children(&pp, opts,-code); } }