From patchwork Wed Oct 12 09:01:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 13004856 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 40A56C4332F for ; Wed, 12 Oct 2022 09:01:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229873AbiJLJBl (ORCPT ); Wed, 12 Oct 2022 05:01:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33284 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229796AbiJLJBi (ORCPT ); Wed, 12 Oct 2022 05:01:38 -0400 Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com [IPv6:2a00:1450:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F3C6E8C03A for ; Wed, 12 Oct 2022 02:01:37 -0700 (PDT) Received: by mail-wr1-x42d.google.com with SMTP id bk15so25216511wrb.13 for ; Wed, 12 Oct 2022 02:01:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=j9QLRCZIjiujdOGkUnMF91E4vCeJF8lxY/30LSSwR1E=; b=laNL/arbcknqoGuISZl0OLO1pAZpXW+V2JFKCtI4P+/owAjtLfZyF9qEwDQyPUqsjT JT+fShC6VPLKCPQmn2fr+g7cPDR+6dI2wAdWhe1fIUXPDJE/6vVH9SCRL36cdRbjdhvA HTS+snT4+qPNCoi/y9Ya4dXczdwJYYMsmpoSyK0gApxCLhxe4u3IhYhqnvGstmoqid1F RFbwX7m6gwqxv9EN5lKujt2ekiF99CndLxQ49A8HlyX984M+3R5IlqmC9kLzGmoDua4c wjyM7zzQOG3eaxn26CUXHC7/bSRLAW/I7jraIEBDOF3Xt3yTO7zjOXHnc9Lp2DlK+/Ke Tkhg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=j9QLRCZIjiujdOGkUnMF91E4vCeJF8lxY/30LSSwR1E=; b=hh/RbdC9xCxkA2dulW/+sfxSMNodbHrgOt+09+Llb0YaP2kM+4f6xRSM8RZ1GA7HG8 bFnAZrX1fazN56YCMvf18xr313sxvGHuic7jnsWZZ0aZyEMOEzCafRbjhQYtrL1yRsiM sT/uHx7v290nGP3EiOaYsBqI4yQ2vEvCbWMuKEzX+FDTuXgbj22lfFDqthLMYtsqL1dd WRiYGM1TelWF1GzWCCHSyAYWAKPDEBJNzgSMzKF4CS/JOvpIH8oW7G4T7NGTiWpDqsGl +ufP6wPgwVIOD9Mko40zKyAwc/IQIk+pzYBreBZIkGQsT5/mD+vBeX7kwv3l/muQWf9E 2bKQ== X-Gm-Message-State: ACrzQf2vyKWN/0nV3eG/+APpkLxzkaY5F2oBslnTqDBpcU7nWdjz086g e61Ja0mUTTlEVMOvetuj7WOdWz8ZoRZrzQ== X-Google-Smtp-Source: AMsMyM4QbyT2O/KLDgMG9AML91mgr/msShYGuUxsI7lT8Yq4KuJhrHMwHVO0J5vChN5+OwyTVDt8jQ== X-Received: by 2002:a5d:64a2:0:b0:230:c757:e3db with SMTP id m2-20020a5d64a2000000b00230c757e3dbmr7932172wrp.495.1665565295997; Wed, 12 Oct 2022 02:01:35 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id b1-20020a05600c4e0100b003a3170a7af9sm1280362wmq.4.2022.10.12.02.01.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Oct 2022 02:01:35 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Calvin Wan , Emily Shaffer , Phillip Wood , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v2 01/22] hook tests: fix redirection logic error in 96e7225b310 Date: Wed, 12 Oct 2022 11:01:09 +0200 Message-Id: X-Mailer: git-send-email 2.38.0.971.ge79ff6d20e7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org The tests added in 96e7225b310 (hook: add 'run' subcommand, 2021-12-22) were redirecting to "actual" both in the body of the hook itself and in the testing code below. The net result was that the "2>>actual" redirection later in the test wasn't doing anything. Let's have those redirection do what it looks like they're doing. Signed-off-by: Ævar Arnfjörð Bjarmason --- t/t1800-hook.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/t/t1800-hook.sh b/t/t1800-hook.sh index 43fcb7c0bfc..2ef3579fa7c 100755 --- a/t/t1800-hook.sh +++ b/t/t1800-hook.sh @@ -95,7 +95,7 @@ test_expect_success 'git hook run -- out-of-repo runs excluded' ' test_expect_success 'git -c core.hooksPath= hook run' ' mkdir my-hooks && write_script my-hooks/test-hook <<-\EOF && - echo Hook ran $1 >>actual + echo Hook ran $1 EOF cat >expect <<-\EOF && From patchwork Wed Oct 12 09:01:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 13004857 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 761F9C4332F for ; Wed, 12 Oct 2022 09:01:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229862AbiJLJBq (ORCPT ); Wed, 12 Oct 2022 05:01:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33312 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229830AbiJLJBk (ORCPT ); Wed, 12 Oct 2022 05:01:40 -0400 Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com [IPv6:2a00:1450:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 48E768F964 for ; Wed, 12 Oct 2022 02:01:39 -0700 (PDT) Received: by mail-wr1-x42d.google.com with SMTP id w18so25250248wro.7 for ; Wed, 12 Oct 2022 02:01:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=iA4LOYKHGlx0bJrVegfbP1hLoK3qw8N2dy1AH/p4Ilo=; b=cNlTwh4nmEcAWHd0zn8QNUGzBN/ubEmxB0iKyCjduNwIraTWDpdgwpOOk270qRSLk/ /nLL/RKSneRLejNkNri5Jeu+q0TUT6f5OoF5KJnqtUlSJQLVU6jKENQ9RSAT2ndG4dKW Q3hEjio3ince54BZxSamu44SdWO4/nYoRHPAYBcxz7QTwSIpdXsW7kEV3RT/ztkPNWFN c4+OHdKRevTPZvF5UNVEHUciktIoctGVYLTIyL4Qogvj4TdyZoajQauQGw5UcvYVuhbW 3rKiVfrQHEnTzsyIHgCl0jDd4scXmaegt61GEgalxSDXLpGh6Swow+aoSz9qLXptJuJy VF5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=iA4LOYKHGlx0bJrVegfbP1hLoK3qw8N2dy1AH/p4Ilo=; b=e+3SFXnXLNetNRThgGcSMUymh/usy/DKQrTaE1mWYXGDWjJafy2WBpfq10vRACSTrB YzJWAfN3a+NlBB+vloH1EkVAtGV7oUb/aN+kFXynXI6uWxYfxxUE7FK78oxl3KseHQlU oEWjT+YEZ86n/kJ1AtC8Du8QUdmm9uBDai+3v7giOjm/MDa+lsKu1EWguKAjWZq0JOiE jy7x+gnKyiekcLr6ZPboL6zIycouOyn4CwE8AmR+kv4XFaSV8iQFF4tJzr2R+j4gj2Ch ntM05fKDW/GhCta8vy/wmz5aCfSYN/tOgJxL2itGidkbPyFxvRa2tTTh4an4TONd9YMN 6VZQ== X-Gm-Message-State: ACrzQf2TnwZW/vehxYYRlgNm9W+j1402TZpfM2kXBncpSegPqCOpIcL0 LisSu0Up4c0IEmysFM8ck3AKxnViQKdvcA== X-Google-Smtp-Source: AMsMyM6zL+my6VWH30J5hzISXsZt4w2cnAtCHdqTV6GoNZyW/qdWi42BheeVHCDV+0cQ7rwTbUJoXw== X-Received: by 2002:a05:6000:1885:b0:22e:7851:b257 with SMTP id a5-20020a056000188500b0022e7851b257mr17184765wri.418.1665565297447; Wed, 12 Oct 2022 02:01:37 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id b1-20020a05600c4e0100b003a3170a7af9sm1280362wmq.4.2022.10.12.02.01.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Oct 2022 02:01:36 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Calvin Wan , Emily Shaffer , Phillip Wood , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v2 02/22] submodule tests: reset "trace.out" between "grep" invocations Date: Wed, 12 Oct 2022 11:01:10 +0200 Message-Id: X-Mailer: git-send-email 2.38.0.971.ge79ff6d20e7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Fix test patterns added in 62104ba14af (submodules: allow parallel fetching, add tests and documentation, 2015-12-15) and a028a1930c6 (fetching submodules: respect `submodule.fetchJobs` config option, 2016-02-29). In the former case we were leaving a trace.out file at the top-level for any subsequent tests (there are none, currently). Let's clean the file up instead. In the latter case we were testing that a given configuration would result in "N tasks" in the log, but we were grepping through the log for all previous such tests, when we really meant to clear the logs between the "grep" invocations. In practice this resulted in no logic error, as e.g. "--fetch 7" would not print out a "9 tasks" line, but let's be paranoid and stop implicitly assuming that that's the case. Signed-off-by: Ævar Arnfjörð Bjarmason --- t/t5526-fetch-submodules.sh | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/t/t5526-fetch-submodules.sh b/t/t5526-fetch-submodules.sh index a301b56db89..e36f9fdf242 100755 --- a/t/t5526-fetch-submodules.sh +++ b/t/t5526-fetch-submodules.sh @@ -177,6 +177,7 @@ test_expect_success "submodule.recurse option triggers recursive fetch" ' ' test_expect_success "fetch --recurse-submodules -j2 has the same output behaviour" ' + test_when_finished "rm -f trace.out" && add_submodule_commits && ( cd downstream && @@ -704,17 +705,25 @@ test_expect_success "'fetch.recurseSubmodules=on-demand' works also without .git test_expect_success 'fetching submodules respects parallel settings' ' git config fetch.recurseSubmodules true && + test_when_finished "rm -f downstream/trace.out" && ( cd downstream && GIT_TRACE=$(pwd)/trace.out git fetch && grep "1 tasks" trace.out && + >trace.out && + GIT_TRACE=$(pwd)/trace.out git fetch --jobs 7 && grep "7 tasks" trace.out && + >trace.out && + git config submodule.fetchJobs 8 && GIT_TRACE=$(pwd)/trace.out git fetch && grep "8 tasks" trace.out && + >trace.out && + GIT_TRACE=$(pwd)/trace.out git fetch --jobs 9 && - grep "9 tasks" trace.out + grep "9 tasks" trace.out && + >trace.out && ) ' From patchwork Wed Oct 12 09:01:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 13004860 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 76A7DC433FE for ; Wed, 12 Oct 2022 09:01:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229912AbiJLJB5 (ORCPT ); Wed, 12 Oct 2022 05:01:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33470 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229883AbiJLJBq (ORCPT ); Wed, 12 Oct 2022 05:01:46 -0400 Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com [IPv6:2a00:1450:4864:20::42a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7389AB7EF6 for ; Wed, 12 Oct 2022 02:01:41 -0700 (PDT) Received: by mail-wr1-x42a.google.com with SMTP id w18so25250377wro.7 for ; Wed, 12 Oct 2022 02:01:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=mO0ka02dzn6Z8bqt0b0dDJqdFUzg16NpOFAKwIpa94U=; b=V8/0MjS2dBw3nlP/vsgMoKA1jtGA0F2WNUaLv/3JSzPgFZshcLGNO5gablpJu++6zi qxhgDb4DJXc9vhGG9iFiuOeLSfuhhbPkG4V0I+e5q5HEcXVxPqASJSETybqMJXGK+zq/ QUYu2Y9Voxv4M8lrqwoSG6AdxvLFUhYYJegOBLvRQa5nVldZpz1hOMyic2z/2wLPCqpY 51sEBJG/0ostI4mLPUTXWYkUZRb5sLASpDB3iF6U/9JvF3iEp/LTSBQgc2CDP0xx1P8Q 4fFK05B6APG/Y48A7u8UhSCzM55P73EVx6r4maIIINNZ9L9gtJGzeqQkedC9MCEioqto vfpA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=mO0ka02dzn6Z8bqt0b0dDJqdFUzg16NpOFAKwIpa94U=; b=AmMgcFAwPxRMgR4nuNpubE9X/R9i7ghurkeGVuMLJV8yM+nERNpVQPCZvI4fy7KFDz BRfgqZVWAGtjRagvOlJ0Gu94hHkOKwYL0aeJcIiBTw0C+gHSqECDc7L8Gd/0G3ENECa8 8hpPLSj+Cpi4swL88QVmQqVTKtDdddG5nAMv9KpH8I6Tq0eqs0SYuoDj4/WA0FnB33ge IfpO4Cpuv1jI4wSX9d5TAf5PqJ5U2DlkSgDXD79PhI5ovd2BUV2O7+eaoADF1tzRIOC3 Pn6Eu43ARmTAw4sWSYeVyfRZgEP9USbbJe8PtBF7ZfYEJu59rJRJVFLhOXz436F3ZyuU OT9A== X-Gm-Message-State: ACrzQf3s66RLEqBCZ69Ffe/Fo/8Ef9wQaLO1wxaPnO+uk90yxUYl7vNa U8etTDkfKUCI3ys5A6vvqzPgigRGtRkbXw== X-Google-Smtp-Source: AMsMyM66NsxVEPnS7RUU5P/w3+lFiOxXEPy+On576T7wHlyJI2rV2g20KTibbacM8lx9IwoVQBxOsg== X-Received: by 2002:adf:e192:0:b0:232:3648:776d with SMTP id az18-20020adfe192000000b002323648776dmr689252wrb.698.1665565299384; Wed, 12 Oct 2022 02:01:39 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id b1-20020a05600c4e0100b003a3170a7af9sm1280362wmq.4.2022.10.12.02.01.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Oct 2022 02:01:38 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Calvin Wan , Emily Shaffer , Phillip Wood , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v2 03/22] run-command tests: test stdout of run_command_parallel() Date: Wed, 12 Oct 2022 11:01:11 +0200 Message-Id: X-Mailer: git-send-email 2.38.0.971.ge79ff6d20e7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Extend the tests added in c553c72eed6 (run-command: add an asynchronous parallel child processor, 2015-12-15) to test stdout in addition to stderr. When the "ungroup" feature was added in fd3aaf53f71 (run-command: add an "ungroup" option to run_process_parallel(), 2022-06-07) its tests were made to test both the stdout and stderr, but these existing tests were left alone. Let's also exhaustively test our expected output here. Signed-off-by: Ævar Arnfjörð Bjarmason --- t/t0061-run-command.sh | 25 +++++++++++++++---------- 1 file changed, 15 insertions(+), 10 deletions(-) diff --git a/t/t0061-run-command.sh b/t/t0061-run-command.sh index 7b5423eebda..19af082750a 100755 --- a/t/t0061-run-command.sh +++ b/t/t0061-run-command.sh @@ -130,8 +130,9 @@ World EOF test_expect_success 'run_command runs in parallel with more jobs available than tasks' ' - test-tool run-command run-command-parallel 5 sh -c "printf \"%s\n%s\n\" Hello World" 2>actual && - test_cmp expect actual + test-tool run-command run-command-parallel 5 sh -c "printf \"%s\n%s\n\" Hello World" >out 2>err && + test_must_be_empty out && + test_cmp expect err ' test_expect_success 'run_command runs ungrouped in parallel with more jobs available than tasks' ' @@ -141,8 +142,9 @@ test_expect_success 'run_command runs ungrouped in parallel with more jobs avail ' test_expect_success 'run_command runs in parallel with as many jobs as tasks' ' - test-tool run-command run-command-parallel 4 sh -c "printf \"%s\n%s\n\" Hello World" 2>actual && - test_cmp expect actual + test-tool run-command run-command-parallel 4 sh -c "printf \"%s\n%s\n\" Hello World" >out 2>err && + test_must_be_empty out && + test_cmp expect err ' test_expect_success 'run_command runs ungrouped in parallel with as many jobs as tasks' ' @@ -152,8 +154,9 @@ test_expect_success 'run_command runs ungrouped in parallel with as many jobs as ' test_expect_success 'run_command runs in parallel with more tasks than jobs available' ' - test-tool run-command run-command-parallel 3 sh -c "printf \"%s\n%s\n\" Hello World" 2>actual && - test_cmp expect actual + test-tool run-command run-command-parallel 3 sh -c "printf \"%s\n%s\n\" Hello World" >out 2>err && + test_must_be_empty out && + test_cmp expect err ' test_expect_success 'run_command runs ungrouped in parallel with more tasks than jobs available' ' @@ -172,8 +175,9 @@ asking for a quick stop EOF test_expect_success 'run_command is asked to abort gracefully' ' - test-tool run-command run-command-abort 3 false 2>actual && - test_cmp expect actual + test-tool run-command run-command-abort 3 false >out 2>err && + test_must_be_empty out && + test_cmp expect err ' test_expect_success 'run_command is asked to abort gracefully (ungroup)' ' @@ -187,8 +191,9 @@ no further jobs available EOF test_expect_success 'run_command outputs ' ' - test-tool run-command run-command-no-jobs 3 sh -c "printf \"%s\n%s\n\" Hello World" 2>actual && - test_cmp expect actual + test-tool run-command run-command-no-jobs 3 sh -c "printf \"%s\n%s\n\" Hello World" >out 2>err && + test_must_be_empty out && + test_cmp expect err ' test_expect_success 'run_command outputs (ungroup) ' ' From patchwork Wed Oct 12 09:01:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 13004859 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4AC6CC433FE for ; Wed, 12 Oct 2022 09:01:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229909AbiJLJB4 (ORCPT ); Wed, 12 Oct 2022 05:01:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33474 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229888AbiJLJBq (ORCPT ); Wed, 12 Oct 2022 05:01:46 -0400 Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com [IPv6:2a00:1450:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 81A51B8C0A for ; Wed, 12 Oct 2022 02:01:41 -0700 (PDT) Received: by mail-wr1-x42d.google.com with SMTP id bk15so25216800wrb.13 for ; Wed, 12 Oct 2022 02:01:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=23xuod5QxoIsDEcWCtQPm2IgVMfSQsPd7mQgv/Z9E1M=; b=f/9tYqYOKHiwkp+7+6teSZndk7N91LaySXfkbCUzmYfebg+YYbeKQNjbtVXSOpNxEW Lc4ZiVDz0SZaCG9jaBx6hkFqXN2qKknnR7+h0hf57+9XN13DHXQGEKsHC3js3h6U3iIo wZz74fWglYGLVSCtQoooCWXX2GLu9lv9wZmqO3Zd0Mozi6xP1ic4ugBMb773l1Q5CIVE 0tdX6OBIVx+d/9DUEysiq6KUWteYY+1OgdfQ/rs5KEMoR5y3ijcCA1t65yu7MH3V/TZ5 qkQxZ/6TXW5JqyPVZMVy6UHJrAI8i55iGF4AZznTLmpFXj5kb2DzKiIYjSDuXpmdDxSL o9+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=23xuod5QxoIsDEcWCtQPm2IgVMfSQsPd7mQgv/Z9E1M=; b=d66pihe5fvTq2jyNkdc2kC3HXuab6PqUSu1iMSoQ4YvNIrX2r3xz+x92Rf+Rou5ebZ tmE5mxrr+XJDTsqGQGj75D/zM9gEB4Ydlj/wk0EgJTcZC//Wdu1Vs4N0p5Lg9pBSV4Ib ijlDQhG1L/NeAqg99Ucy8F958+181Vjw+tVTY83tKo+TCHZPxLdxYmtGc3XpJ3ApmGql 65J5uWvSSPSGy3AcON4POkQ/tQ2YjmAg7rpClNe//prDJuVxJprsjn3+Cgjj3+PoCbia e4PGuC8ph1WnMIJWZOr1Yx7DeEBFhp0Li31/xQINji/W6R3qJUCFooYz6AEUTK+q2MsA MZPQ== X-Gm-Message-State: ACrzQf3nqtvjltPELkpQXgAEusj3zKBjg1tkLbqcBKiB3gUuw4LSS54T vf3JLTePkRf2ZJjLRqu01NXgjPHDqN3slg== X-Google-Smtp-Source: AMsMyM7BW+32vjW0L0ZJSJ4bEWjkPgMbbv5qzGHtKd3Wr1WMFrW+i/wZFCn3JUsbvzNv0GWCCzpAeQ== X-Received: by 2002:adf:dbc5:0:b0:22c:c605:3b81 with SMTP id e5-20020adfdbc5000000b0022cc6053b81mr17244279wrj.218.1665565300753; Wed, 12 Oct 2022 02:01:40 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id b1-20020a05600c4e0100b003a3170a7af9sm1280362wmq.4.2022.10.12.02.01.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Oct 2022 02:01:40 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Calvin Wan , Emily Shaffer , Phillip Wood , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v2 04/22] run-command test helper: use "else if" pattern Date: Wed, 12 Oct 2022 11:01:12 +0200 Message-Id: X-Mailer: git-send-email 2.38.0.971.ge79ff6d20e7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Adjust the cmd__run_command() to use an "if/else if" chain rather than mutually exclusive "if" statements. This non-functional change makes a subsequent commit smaller. Signed-off-by: Ævar Arnfjörð Bjarmason --- t/helper/test-run-command.c | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-) diff --git a/t/helper/test-run-command.c b/t/helper/test-run-command.c index c9283b47afa..390fa4fb724 100644 --- a/t/helper/test-run-command.c +++ b/t/helper/test-run-command.c @@ -427,18 +427,17 @@ int cmd__run_command(int argc, const char **argv) strvec_clear(&proc.args); strvec_pushv(&proc.args, (const char **)argv + 3); - if (!strcmp(argv[1], "run-command-parallel")) + if (!strcmp(argv[1], "run-command-parallel")) { exit(run_processes_parallel(jobs, parallel_next, NULL, NULL, &proc)); - - if (!strcmp(argv[1], "run-command-abort")) + } else if (!strcmp(argv[1], "run-command-abort")) { exit(run_processes_parallel(jobs, parallel_next, NULL, task_finished, &proc)); - - if (!strcmp(argv[1], "run-command-no-jobs")) + } else if (!strcmp(argv[1], "run-command-no-jobs")) { exit(run_processes_parallel(jobs, no_job, NULL, task_finished, &proc)); - - fprintf(stderr, "check usage\n"); - return 1; + } else { + fprintf(stderr, "check usage\n"); + return 1; + } } From patchwork Wed Oct 12 09:01:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 13004861 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C64DAC4332F for ; Wed, 12 Oct 2022 09:02:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229940AbiJLJCN (ORCPT ); Wed, 12 Oct 2022 05:02:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33676 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229711AbiJLJBw (ORCPT ); Wed, 12 Oct 2022 05:01:52 -0400 Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com [IPv6:2a00:1450:4864:20::330]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8B034BA24C for ; Wed, 12 Oct 2022 02:01:44 -0700 (PDT) Received: by mail-wm1-x330.google.com with SMTP id iv17so10046496wmb.4 for ; Wed, 12 Oct 2022 02:01:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=bUdYd+/n3y2i5wi6BAUzfJxEHaBcqp9427yq15owIfc=; b=etQAnybuDERheBEJY7GdCzQO+XkFXeZY5O+AG3dFqRDvYYVAAo7BnPf8MnrLNFcgWy bZ3Z3aRCtt53lpYbHpw55whYMpvOMExKcGQTYih9hi7ZjaKap3q7l+R+A1Nm+GERlTWs FlfzTl462GJxdK9l+AzNvETldvKK2vsWnWNE1CXe+eWX52bt35T8U9Vxorma9lfOn0uR Je7fyPOMQrqSTKZzCH3qFzaKVtoLRorSJ+tyd7ZDA9qXyPZmkRJN2BXrMoyveR+x3jty Uk/9Oyr0AP3fkAjQ0Z1+Gb80JrDdLSWtJKgcOSrskgWDWbMLFxT9QyglRnBi4rmeypz/ wWww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bUdYd+/n3y2i5wi6BAUzfJxEHaBcqp9427yq15owIfc=; b=sXcC6hyELCg8IvN7SqiNLzUpIBsMVyrOw7yNyY3kxbL5i/ipN1ssD9rzpzBYBdO5Yy g5pe/H+Ddm6wfSqDbP/kEDgXwgYQZvuVEWnDdHnglI6DSHXGmSKDnKbQOAGXQKESx7nw ksRZ/VZmbe/Y2xd5GXQm+WUq2aCtklN1j4VOU3OAO/kMO3wV7Ft1Cu7BDsrujthJCsqo QJLhogGgiqMtaJKm642Ca4tQqrTnuNSFYP9MeWLLdRclYtxsw7+QNcyjZBYdqUrr+Xo6 LJvAuWlAO9qv+UnklWW4UVeVbQbi+V725Qb/0ydgy52vJm548JdFHNY3teDX5QjSj9/A FB/w== X-Gm-Message-State: ACrzQf02Jc4bkNB5ELWfC6opZRPg1Ip6w8mVEBgjoGF1FHkalKtY2la+ y/sEQNAMqGyEJPrtyehNU3wRqjyPa7bukw== X-Google-Smtp-Source: AMsMyM6PTkSHgzsdVX9Pk5WSzMDL+LYUE7KiK+aT57aqDmCqWbRlVHo8NiQGdxhQXqeuSAxNjQBdYg== X-Received: by 2002:a7b:cd14:0:b0:3c6:bf44:770d with SMTP id f20-20020a7bcd14000000b003c6bf44770dmr2013607wmj.35.1665565302330; Wed, 12 Oct 2022 02:01:42 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id b1-20020a05600c4e0100b003a3170a7af9sm1280362wmq.4.2022.10.12.02.01.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Oct 2022 02:01:41 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Calvin Wan , Emily Shaffer , Phillip Wood , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v2 05/22] run-command API: have "run_processes_parallel{,_tr2}()" return void Date: Wed, 12 Oct 2022 11:01:13 +0200 Message-Id: X-Mailer: git-send-email 2.38.0.971.ge79ff6d20e7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Change the "run_processes_parallel{,_tr2}()" functions to return void, instead of int. Ever since c553c72eed6 (run-command: add an asynchronous parallel child processor, 2015-12-15) they have unconditionally returned 0. To get a "real" return value out of this function the caller needs to get it via the "task_finished_fn" callback, see the example in hook.c added in 96e7225b310 (hook: add 'run' subcommand, 2021-12-22). So the "result = " and "if (!result)" code added to "builtin/fetch.c" d54dea77dba (fetch: let --jobs= parallelize --multiple, too, 2019-10-05) has always been redundant, we always took that "if" path. Likewise the "ret =" in "t/helper/test-run-command.c" added in be5d88e1128 (test-tool run-command: learn to run (parts of) the testsuite, 2019-10-04) wasn't used, instead we got the return value from the "if (suite.failed.nr > 0)" block seen in the context. Subsequent commits will alter this API interface, getting rid of this always-zero return value makes it easier to understand those changes. Signed-off-by: Ævar Arnfjörð Bjarmason --- builtin/fetch.c | 17 ++++++++--------- run-command.c | 27 +++++++++++---------------- run-command.h | 16 ++++++++-------- t/helper/test-run-command.c | 16 ++++++++-------- 4 files changed, 35 insertions(+), 41 deletions(-) diff --git a/builtin/fetch.c b/builtin/fetch.c index a0fca93bb6a..78043fb67ef 100644 --- a/builtin/fetch.c +++ b/builtin/fetch.c @@ -1953,15 +1953,14 @@ static int fetch_multiple(struct string_list *list, int max_children) struct parallel_fetch_state state = { argv.v, list, 0, 0 }; strvec_push(&argv, "--end-of-options"); - result = run_processes_parallel_tr2(max_children, - &fetch_next_remote, - &fetch_failed_to_start, - &fetch_finished, - &state, - "fetch", "parallel/fetch"); - - if (!result) - result = state.result; + run_processes_parallel_tr2(max_children, + &fetch_next_remote, + &fetch_failed_to_start, + &fetch_finished, + &state, + "fetch", "parallel/fetch"); + + result = state.result; } else for (i = 0; i < list->nr; i++) { const char *name = list->items[i].string; diff --git a/run-command.c b/run-command.c index 5ec3a46dccf..642e6b6e057 100644 --- a/run-command.c +++ b/run-command.c @@ -1783,11 +1783,11 @@ static int pp_collect_finished(struct parallel_processes *pp) return result; } -int run_processes_parallel(int n, - get_next_task_fn get_next_task, - start_failure_fn start_failure, - task_finished_fn task_finished, - void *pp_cb) +void run_processes_parallel(int n, + get_next_task_fn get_next_task, + start_failure_fn start_failure, + task_finished_fn task_finished, + void *pp_cb) { int i, code; int output_timeout = 100; @@ -1834,25 +1834,20 @@ int run_processes_parallel(int n, } pp_cleanup(&pp); - return 0; } -int run_processes_parallel_tr2(int n, get_next_task_fn get_next_task, - start_failure_fn start_failure, - task_finished_fn task_finished, void *pp_cb, - const char *tr2_category, const char *tr2_label) +void run_processes_parallel_tr2(int n, get_next_task_fn get_next_task, + start_failure_fn start_failure, + task_finished_fn task_finished, void *pp_cb, + const char *tr2_category, const char *tr2_label) { - int result; - trace2_region_enter_printf(tr2_category, tr2_label, NULL, "max:%d", ((n < 1) ? online_cpus() : n)); - result = run_processes_parallel(n, get_next_task, start_failure, - task_finished, pp_cb); + run_processes_parallel(n, get_next_task, start_failure, + task_finished, pp_cb); trace2_region_leave(tr2_category, tr2_label, NULL); - - return result; } int run_auto_maintenance(int quiet) diff --git a/run-command.h b/run-command.h index 0e85e5846a5..e76a1b6b5b3 100644 --- a/run-command.h +++ b/run-command.h @@ -485,14 +485,14 @@ typedef int (*task_finished_fn)(int result, * API reads that setting. */ extern int run_processes_parallel_ungroup; -int run_processes_parallel(int n, - get_next_task_fn, - start_failure_fn, - task_finished_fn, - void *pp_cb); -int run_processes_parallel_tr2(int n, get_next_task_fn, start_failure_fn, - task_finished_fn, void *pp_cb, - const char *tr2_category, const char *tr2_label); +void run_processes_parallel(int n, + get_next_task_fn, + start_failure_fn, + task_finished_fn, + void *pp_cb); +void run_processes_parallel_tr2(int n, get_next_task_fn, start_failure_fn, + task_finished_fn, void *pp_cb, + const char *tr2_category, const char *tr2_label); /** * Convenience function which prepares env for a command to be run in a diff --git a/t/helper/test-run-command.c b/t/helper/test-run-command.c index 390fa4fb724..30c474f3243 100644 --- a/t/helper/test-run-command.c +++ b/t/helper/test-run-command.c @@ -192,8 +192,8 @@ static int testsuite(int argc, const char **argv) fprintf(stderr, "Running %"PRIuMAX" tests (%d at a time)\n", (uintmax_t)suite.tests.nr, max_jobs); - ret = run_processes_parallel(max_jobs, next_test, test_failed, - test_finished, &suite); + run_processes_parallel(max_jobs, next_test, test_failed, + test_finished, &suite); if (suite.failed.nr > 0) { ret = 1; @@ -428,16 +428,16 @@ int cmd__run_command(int argc, const char **argv) strvec_pushv(&proc.args, (const char **)argv + 3); if (!strcmp(argv[1], "run-command-parallel")) { - exit(run_processes_parallel(jobs, parallel_next, - NULL, NULL, &proc)); + run_processes_parallel(jobs, parallel_next, NULL, NULL, &proc); } else if (!strcmp(argv[1], "run-command-abort")) { - exit(run_processes_parallel(jobs, parallel_next, - NULL, task_finished, &proc)); + run_processes_parallel(jobs, parallel_next, NULL, + task_finished, &proc); } else if (!strcmp(argv[1], "run-command-no-jobs")) { - exit(run_processes_parallel(jobs, no_job, - NULL, task_finished, &proc)); + run_processes_parallel(jobs, no_job, NULL, task_finished, + &proc); } else { fprintf(stderr, "check usage\n"); return 1; } + exit(0); } From patchwork Wed Oct 12 09:01:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 13004862 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BAACFC433FE for ; Wed, 12 Oct 2022 09:02:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229827AbiJLJCQ (ORCPT ); Wed, 12 Oct 2022 05:02:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34126 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229691AbiJLJCI (ORCPT ); Wed, 12 Oct 2022 05:02:08 -0400 Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com [IPv6:2a00:1450:4864:20::42b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4860273308 for ; Wed, 12 Oct 2022 02:01:46 -0700 (PDT) Received: by mail-wr1-x42b.google.com with SMTP id u10so25265161wrq.2 for ; Wed, 12 Oct 2022 02:01:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=wMJ0ei29o3W92b760+HvZkT6k/ZhX31NK+F7yQCZdt4=; b=pMPwnSIPSehnr7teV5g6COSm71kTnvhlFvNrMJzCZmYXm2wMX5gR8qNd1uKQ3h2iMC uaRXOGW3AibF5YwBcI5TPIXJB4tBYIuBgsUocPTBAq2VZSDePWBOLjpJXo2Efg9kTF5L uh7NzUM5lL1G8GKn5Dgf7Ew8r5Eu0AlEriPbBKCX9z6rlB6tCcdqthcSUxqkEuEpeIXR 1SPilxGr7n6IzGZVYAOwauI736yojYUKaaM/Ss+qaW7SAr1HIxMDDkbXC/Zh0YVXT2UG SrrMHo6djZx8H9RKQp6I6lIR7US9mYAsAptpfJm3QyVBKLuhLsXt+oLBLmgsXJnnH61p m6zA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wMJ0ei29o3W92b760+HvZkT6k/ZhX31NK+F7yQCZdt4=; b=SGzUSMehgZ2UiDhXYbdePOw/aQFRTmVnaRKiGhIbyf60hznQlB8ccAErdDNYTHqT4F 0ymjLY2u8jdlLQVVc+Y96Qx0Na+nC1TA6gqaMjlc4ZSX4pzKlFd+osKaoT6wkL/7iWr4 W9f0rD5myb0QEHXubdg8ETQS6glsIVcev95TgfTOhNkdjDeDhqtKiEGuTRwRyWIbbs+s RCTK9WhL80h6GVf3LWwFcltA/bQO2fl3cGnaJEHK+00Z/byGzdekdZEUdABeOH8wwdPo syv4j2w6DzuELv5FL/FdvBNcaas6jkY5gj3/suLthUOZJUWsa2+e1yTAfNQm8ruc2lXY z3FQ== X-Gm-Message-State: ACrzQf32j4IzU8lIqAs/Ct2hFDPKD/eRhgigRUS75kvgVmPjhVrJf/fe 42qdVjwkxZEXXHrBP80SvkNqFsBywQtaHA== X-Google-Smtp-Source: AMsMyM5qRGxoACb8UU0ZguBU1MOUrl3g7G0dWCIHMeh5gSTe8TUMBTbpZdaD30p1nOiBJERY+hNxmA== X-Received: by 2002:adf:efc5:0:b0:22e:4a6:2d5b with SMTP id i5-20020adfefc5000000b0022e04a62d5bmr17879037wrp.293.1665565303966; Wed, 12 Oct 2022 02:01:43 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id b1-20020a05600c4e0100b003a3170a7af9sm1280362wmq.4.2022.10.12.02.01.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Oct 2022 02:01:42 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Calvin Wan , Emily Shaffer , Phillip Wood , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v2 06/22] run-command tests: use "return", not "exit" Date: Wed, 12 Oct 2022 11:01:14 +0200 Message-Id: X-Mailer: git-send-email 2.38.0.971.ge79ff6d20e7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Change the "run-command" test helper to "return" instead of calling "exit", see 338abb0f045 (builtins + test helpers: use return instead of exit() in cmd_*, 2021-06-08) Because we'd previously gotten past the SANITIZE=leak check by using exit() here we need to move to "goto cleanup" pattern. Signed-off-by: Ævar Arnfjörð Bjarmason --- t/helper/test-run-command.c | 33 ++++++++++++++++++++++----------- 1 file changed, 22 insertions(+), 11 deletions(-) diff --git a/t/helper/test-run-command.c b/t/helper/test-run-command.c index 30c474f3243..ee509aefa2f 100644 --- a/t/helper/test-run-command.c +++ b/t/helper/test-run-command.c @@ -381,13 +381,14 @@ int cmd__run_command(int argc, const char **argv) { struct child_process proc = CHILD_PROCESS_INIT; int jobs; + int ret; if (argc > 1 && !strcmp(argv[1], "testsuite")) - exit(testsuite(argc - 1, argv + 1)); + return testsuite(argc - 1, argv + 1); if (!strcmp(argv[1], "inherited-handle")) - exit(inherit_handle(argv[0])); + return inherit_handle(argv[0]); if (!strcmp(argv[1], "inherited-handle-child")) - exit(inherit_handle_child()); + return inherit_handle_child(); if (argc >= 2 && !strcmp(argv[1], "quote-stress-test")) return !!quote_stress_test(argc - 1, argv + 1); @@ -404,18 +405,24 @@ int cmd__run_command(int argc, const char **argv) argv += 2; argc -= 2; } - if (argc < 3) - return 1; + if (argc < 3) { + ret = 1; + goto cleanup; + } strvec_pushv(&proc.args, (const char **)argv + 2); if (!strcmp(argv[1], "start-command-ENOENT")) { - if (start_command(&proc) < 0 && errno == ENOENT) - return 0; + if (start_command(&proc) < 0 && errno == ENOENT) { + ret = 0; + goto cleanup; + } fprintf(stderr, "FAIL %s\n", argv[1]); return 1; } - if (!strcmp(argv[1], "run-command")) - exit(run_command(&proc)); + if (!strcmp(argv[1], "run-command")) { + ret = run_command(&proc); + goto cleanup; + } if (!strcmp(argv[1], "--ungroup")) { argv += 1; @@ -436,8 +443,12 @@ int cmd__run_command(int argc, const char **argv) run_processes_parallel(jobs, no_job, NULL, task_finished, &proc); } else { + ret = 1; fprintf(stderr, "check usage\n"); - return 1; + goto cleanup; } - exit(0); + ret = 0; +cleanup: + child_process_clear(&proc); + return ret; } From patchwork Wed Oct 12 09:01:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 13004864 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 07C10C433FE for ; Wed, 12 Oct 2022 09:02:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229563AbiJLJCR (ORCPT ); Wed, 12 Oct 2022 05:02:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33588 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229885AbiJLJCK (ORCPT ); Wed, 12 Oct 2022 05:02:10 -0400 Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com [IPv6:2a00:1450:4864:20::329]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9122BBA900 for ; Wed, 12 Oct 2022 02:01:47 -0700 (PDT) Received: by mail-wm1-x329.google.com with SMTP id bg9-20020a05600c3c8900b003bf249616b0so793329wmb.3 for ; Wed, 12 Oct 2022 02:01:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=oieYuUBWHUvjDP07aFBGjc0KXeNB5ghCqeSuFf4IYe4=; b=J891T86GhpmC/aNpEKhUSX3O3oSFVUcozDzd6DZYisjcgpuOVrTCg4utYoS/HomerU 4Bdnz/i8Hi3BFqMFAiecmf1AolQyC1pz/BQogYuZhhfy5EJbIhBC+Idsc6rZQfphKh/o U+cBJr4WLSI4s/Vx1Q1PWyzoPQiMiFu5lgolIY14rLrk5HtJZWUAyrA1gogdsE1YqebF eJT/Keb2XXudYl4C5nImoc/O1p+NWQzq8KGc3cuJ/B32P98i4dDCRvuXjsl+Me8nd1DQ jyTF+lPdbjTuUTSBB1NJ5YnOfCThCjkKHoLMUSUQxivilwi1kHFF+3gOgwqfQwF1YSns By+Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=oieYuUBWHUvjDP07aFBGjc0KXeNB5ghCqeSuFf4IYe4=; b=v2ogn9VMuxAxdfk7Q5WvyNFTlazBXVEUf2UCYiBp3LgumHH3uYo52nY8SXQtxB1IT4 NKpjGH0zW9YDAJP0/W5Gpz1+rFtB7NB9iYrlV+f8HpgLU3ss/FbgnfAFQzkQD7DVy/Be oZX7kWMHynG5oc1Sp1z9PkrWZs6OJT8U5oi9gfgBBrnIY35hoEcjNzaDMdcYVV4vc3Vb uXzwEMks0p2DIcU9EvwqHF8pl9OX8Kn88i7Yer/GFsSprdTtGPRVUeDQQ/v9Y76YamuK YqvSHhb7BO5OfrZ3FDZw3f2FPKLvlSknDSOsOYHE9kz1SJtrhkVLEo4JoUDoIBouENfJ uxiQ== X-Gm-Message-State: ACrzQf2iEn0ZRy/Df4QDa5KRr1x6v0sZOj6GNLDp2ogNLhCCkrG/LjMp 3L86c2o/btizC5Y6fbLwMsoC/Ub6fj6Yjg== X-Google-Smtp-Source: AMsMyM5hwCSs7jmNvd9xyssBkaMiiIFBMbBbQF03ToXH4t5T/Ibn2ulNFJGVrZuqaEhvX8srYye52w== X-Received: by 2002:a1c:27c6:0:b0:3c2:e6df:c79b with SMTP id n189-20020a1c27c6000000b003c2e6dfc79bmr1949843wmn.14.1665565305171; Wed, 12 Oct 2022 02:01:45 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id b1-20020a05600c4e0100b003a3170a7af9sm1280362wmq.4.2022.10.12.02.01.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Oct 2022 02:01:44 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Calvin Wan , Emily Shaffer , Phillip Wood , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v2 07/22] run-command.c: remove dead assignment in while-loop Date: Wed, 12 Oct 2022 11:01:15 +0200 Message-Id: X-Mailer: git-send-email 2.38.0.971.ge79ff6d20e7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Remove code that's been unused since it was added in c553c72eed6 (run-command: add an asynchronous parallel child processor, 2015-12-15), the next use of "i" in this function is: for (i = 0; ... So we'll always clobber the "i" that's set here. Presumably the "i" assignment is an artifact of WIP code that made it into our tree. A subsequent commit will need to adjust the type of the "i" variable in the otherwise unrelated for-loop, which is why this is being removed now. Signed-off-by: Ævar Arnfjörð Bjarmason --- run-command.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/run-command.c b/run-command.c index 642e6b6e057..bd45828fe2c 100644 --- a/run-command.c +++ b/run-command.c @@ -1689,7 +1689,7 @@ static void pp_buffer_stderr(struct parallel_processes *pp, int output_timeout) { int i; - while ((i = poll(pp->pfd, pp->max_processes, output_timeout)) < 0) { + while (poll(pp->pfd, pp->max_processes, output_timeout) < 0) { if (errno == EINTR) continue; pp_cleanup(pp); From patchwork Wed Oct 12 09:01:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 13004865 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 992F5C4332F for ; Wed, 12 Oct 2022 09:02:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229978AbiJLJC2 (ORCPT ); Wed, 12 Oct 2022 05:02:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34176 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229914AbiJLJCK (ORCPT ); Wed, 12 Oct 2022 05:02:10 -0400 Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6D35BB7EF4 for ; Wed, 12 Oct 2022 02:01:49 -0700 (PDT) Received: by mail-wr1-x42f.google.com with SMTP id n12so25216982wrp.10 for ; Wed, 12 Oct 2022 02:01:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=FipWHUdK7Qq0SoGArMNe6aQYQGaFlVLkfb6pHN+PQak=; b=SaN+nhgublg0lEanDpdvZyhV679tQMjcn6+MrCAySULfkBWoPrbQDcKVz1zSxA9bwY fmHwyMo0elz4q2vdVbW6AUffBInXaHedF9nXtK3Nr0tspsC1A1fz4qJLubdY11L09nGK UMdkY3f8MNAIGWbkr03jhHUf/vCIF79AXfcAY6ibg4ucVyDfxavD0NSsGHRDSP18Y9dI LaH4yKWICOFUTjEqOp19Y/AutsIZpti5pIJBdC6fFuQRrv9GhmJFRBqiaM6cvx7vcbUi yzJQutnY4HzOIEAZFvXYESE3/wvTAerV0x9b2+xsCjMwMF3dW/gfLYFT5Xv2MeHMuJSx BOcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FipWHUdK7Qq0SoGArMNe6aQYQGaFlVLkfb6pHN+PQak=; b=7SW6xt1THOC9xGuZgfrwbr8u+h/Ah5ppmv301jl0mCxnwnkBXT0lsfeq11CIMbCCv9 1S4hxf32nF1iMi2Bq5n+bBwqa39cCdFFJdNgjVvbJOz7m+8KSJKSSDc+ALBCQWMYi2bY d7N/JuXQtWsUppOQ1YW6zS7VTBU+O4ha1353IIUhyb5PLxuRpyU9ZoV10ZGXG/zVRVkA ZsvpEHsHYhtS1JykQlT3sDTbtOdL1+q2IK0gMYoIejCuR/O68K0XuhvDZ5gQUKNXFt+M DSqGU3+d7U1v2+Z/q2/y8sUWjW6bDjx++Z7G+dHMN01UmUtXWo6PeZJLjgzw5lBdkTW2 2p0g== X-Gm-Message-State: ACrzQf0XFbphcHllkIL1tfEk9bx/HhkxEEu+9a/5++0BQeBIl+ZGDFqJ QXABAm2i7gMFB8lb0/g+jdUjDfYbUH092w== X-Google-Smtp-Source: AMsMyM625zStDEjHIKLNlVE3akVgL9m5TgS1hFTx3+2U6S0KEmN7/7dSApUxpvO27Wcq+qwfyico/w== X-Received: by 2002:a5d:64e9:0:b0:22e:7631:bcab with SMTP id g9-20020a5d64e9000000b0022e7631bcabmr18384896wri.36.1665565306579; Wed, 12 Oct 2022 02:01:46 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id b1-20020a05600c4e0100b003a3170a7af9sm1280362wmq.4.2022.10.12.02.01.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Oct 2022 02:01:45 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Calvin Wan , Emily Shaffer , Phillip Wood , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v2 08/22] run-command.c: use C99 "for (TYPE VAR = ..." syntax where useful Date: Wed, 12 Oct 2022 11:01:16 +0200 Message-Id: X-Mailer: git-send-email 2.38.0.971.ge79ff6d20e7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Refactor code in run-command.c where the "i" iteration variable is being compared to an unsigned type, or where it's being shadowed. In a subsequent commit the type of the "n" variable will be changed, this also helps to avoid some of the warnings we've had under "DEVOPTS=extra-all" since 8d133a4653a (strvec: use size_t to store nr and alloc, 2021-09-11). Per the thread ending at [1] we already have this C99 syntax in the v2.38.0 release, per 6563706568b (CodingGuidelines: give deadline for "for (int i = 0; ...", 2022-03-30) we should re-visit the wider use of this syntax for November 2022, meaning within the window of v2.39.0. As of writing it's earlier than that deadline and per [1] we want to "avoid open[ing] the floodgate and deliberately add more [this C99 syntax]". But in this case the use of the syntax solves a real problem of conflating types, which we'd otherwise need to avoid by using differently named iteration variables (and the associated larger refactoring). 1. https://lore.kernel.org/git/xmqqy1wgwkbj.fsf@gitster.g/ Signed-off-by: Ævar Arnfjörð Bjarmason --- run-command.c | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/run-command.c b/run-command.c index bd45828fe2c..7b27e4de78d 100644 --- a/run-command.c +++ b/run-command.c @@ -443,7 +443,6 @@ static char **prep_childenv(const char *const *deltaenv) struct string_list env = STRING_LIST_INIT_DUP; struct strbuf key = STRBUF_INIT; const char *const *p; - int i; /* Construct a sorted string list consisting of the current environ */ for (p = (const char *const *) environ; p && *p; p++) { @@ -476,7 +475,7 @@ static char **prep_childenv(const char *const *deltaenv) /* Create an array of 'char *' to be used as the childenv */ ALLOC_ARRAY(childenv, env.nr + 1); - for (i = 0; i < env.nr; i++) + for (size_t i = 0; i < env.nr; i++) childenv[i] = env.items[i].util; childenv[env.nr] = NULL; @@ -581,7 +580,6 @@ static void trace_add_env(struct strbuf *dst, const char *const *deltaenv) { struct string_list envs = STRING_LIST_INIT_DUP; const char *const *e; - int i; int printed_unset = 0; /* Last one wins, see run-command.c:prep_childenv() for context */ @@ -599,7 +597,7 @@ static void trace_add_env(struct strbuf *dst, const char *const *deltaenv) } /* "unset X Y...;" */ - for (i = 0; i < envs.nr; i++) { + for (size_t i = 0; i < envs.nr; i++) { const char *var = envs.items[i].string; const char *val = envs.items[i].util; @@ -616,7 +614,7 @@ static void trace_add_env(struct strbuf *dst, const char *const *deltaenv) strbuf_addch(dst, ';'); /* ... followed by "A=B C=D ..." */ - for (i = 0; i < envs.nr; i++) { + for (size_t i = 0; i < envs.nr; i++) { const char *var = envs.items[i].string; const char *val = envs.items[i].util; const char *oldval; @@ -1789,7 +1787,7 @@ void run_processes_parallel(int n, task_finished_fn task_finished, void *pp_cb) { - int i, code; + int code; int output_timeout = 100; int spawn_cap = 4; int ungroup = run_processes_parallel_ungroup; @@ -1801,7 +1799,7 @@ void run_processes_parallel(int n, pp_init(&pp, n, get_next_task, start_failure, task_finished, pp_cb, ungroup); while (1) { - for (i = 0; + for (int i = 0; i < spawn_cap && !pp.shutdown && pp.nr_processes < pp.max_processes; i++) { From patchwork Wed Oct 12 09:01:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 13004868 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D092BC433FE for ; Wed, 12 Oct 2022 09:02:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229915AbiJLJCp (ORCPT ); Wed, 12 Oct 2022 05:02:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34284 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229880AbiJLJCO (ORCPT ); Wed, 12 Oct 2022 05:02:14 -0400 Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D352CBB061 for ; Wed, 12 Oct 2022 02:01:50 -0700 (PDT) Received: by mail-wm1-x32b.google.com with SMTP id ay36so10056579wmb.0 for ; Wed, 12 Oct 2022 02:01:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MjFMAeeGU++6ujCSCowCWceZiRYmFLX8xaffSWmvtHk=; b=dYt7CCDqXWCug0te3xu9+U7MNVo1QDbulDErRUwWnBJshobRIk5obrE5N7reGXjr3v tJWqr1+aIEgFRr8HZX+okf2MGLJ1ePD230u6QG/5IYMwjoku35VUw+sXWhE594OOVk10 MPJ3OXelsPb82lbytGfgLiSs+T+o1e6B7wDbJaNV4+XQRrgdyu89MFjltn7Sd4feAzUn 61K6SmpTtrF/gapUm5EyuiN4b2RmS93ZM2eQEbuf+wvACxHk5N6CLbBnLaOdbDc6Q9AH 2ysz2AIvqJlWh7eiPXUzDH6Yto7448CWrkgyx4LUtXipFNM2qej8OxRKVPOFd1pmP2Mx 4lQQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MjFMAeeGU++6ujCSCowCWceZiRYmFLX8xaffSWmvtHk=; b=lY/pEW4O3qJIqmTggm6KmRFEHNoz1u6Ctqw16NpWAxfOqGnbd6Pv4z63xmUHU6Os0V 2kdd234xHi8dHVfHlC79eThiz7rY+TTklRl37p8Qcjb1IVqT/F0PTuIMz2o3VX9eABD7 Kom1Qb2RY27kbAqMqJBeL1bmEpEo5Ztzbmn8h104EsdyLseVsz0j6iXKL2D+HrzHI2zc 2GffZvGe3NPOvk0kBL53ADz+2d55gX5g2arzxAHlKo3kPxVGOrE582EwWwilYyGHCiry h3CopYNj1P2tasI/h4ymOfzZ2Vibyr4/PWmlPSLWfolyZcQXO1HvoYGaBaBnE/gkLvST M3mg== X-Gm-Message-State: ACrzQf00Oqh/DzIzP9kPHpX30etVaHAsNFXhcAwsJOAHKRJGLEV0jC54 DCCGTCmP928kvaltnWQ7vxzsffhDfKDr2Q== X-Google-Smtp-Source: AMsMyM4rIzXTbrGfyl68sOB+A2K7oWOIY/lDW7T8f3NtIWzVGmglHwdNVCvDrD7K7HK2wbsU9gZjRA== X-Received: by 2002:a05:600c:5388:b0:3c5:4c1:a1f6 with SMTP id hg8-20020a05600c538800b003c504c1a1f6mr1999390wmb.11.1665565307675; Wed, 12 Oct 2022 02:01:47 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id b1-20020a05600c4e0100b003a3170a7af9sm1280362wmq.4.2022.10.12.02.01.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Oct 2022 02:01:46 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Calvin Wan , Emily Shaffer , Phillip Wood , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v2 09/22] run-command API: make "n" parameter a "size_t" Date: Wed, 12 Oct 2022 11:01:17 +0200 Message-Id: X-Mailer: git-send-email 2.38.0.971.ge79ff6d20e7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Make the "n" variable added in c553c72eed6 (run-command: add an asynchronous parallel child processor, 2015-12-15) a "size_t". As we'll see in a subsequent commit we do pass "0" here, but never "jobs < 0". We could have made it an "unsigned int", but as we're having to change this let's not leave another case in the codebase where a size_t and "unsigned int" size differ on some platforms. In this case it's likely to never matter, but it's easier to not need to worry about it. After this and preceding changes: make run-command.o DEVOPTS=extra-all CFLAGS=-Wno-unused-parameter Only has one (and new) -Wsigned-compare warning, about a comparison of "i" to online_cpus(), a subsequent commit will adjust & deal with online_cpus() and that warning. The only users of the "n" parameter are: * builtin/fetch.c: defaults to 1, reads from the "fetch.parallel" config. As seen in the code that parses the config added in d54dea77dba (fetch: let --jobs= parallelize --multiple, too, 2019-10-05) will die if the git_config_int() return value is < 0. It will however pass us n = 0, as we'll see in a subsequent commit. * submodule.c: defaults to 1, reads from "submodule.fetchJobs" config. Read via code originally added in a028a1930c6 (fetching submodules: respect `submodule.fetchJobs` config option, 2016-02-29). It now piggy-backs on the the submodule.fetchJobs code and validation added in f20e7c1ea24 (submodule: remove submodule.fetchjobs from submodule-config parsing, 2017-08-02). Like builtin/fetch.c it will die if the git_config_int() return value is < 0, but like builtin/fetch.c it will pass us n = 0. * builtin/submodule--helper.c: defaults to 1. Read via code originally added in 2335b870fa7 (submodule update: expose parallelism to the user, 2016-02-29). Since f20e7c1ea24 (submodule: remove submodule.fetchjobs from submodule-config parsing, 2017-08-02) it shares a config parser and semantics with the submodule.c caller. * hook.c: hardcoded to 1, see 96e7225b310 (hook: add 'run' subcommand, 2021-12-22). * t/helper/test-run-command.c: can be -1 after parsing the arguments, but will then be overridden to online_cpus() before passing it to this API. See be5d88e1128 (test-tool run-command: learn to run (parts of) the testsuite, 2019-10-04). Signed-off-by: Ævar Arnfjörð Bjarmason --- run-command.c | 44 ++++++++++++++++++-------------------------- run-command.h | 4 ++-- 2 files changed, 20 insertions(+), 28 deletions(-) diff --git a/run-command.c b/run-command.c index 7b27e4de78d..5a63008b6e1 100644 --- a/run-command.c +++ b/run-command.c @@ -1498,8 +1498,8 @@ int run_processes_parallel_ungroup; struct parallel_processes { void *data; - int max_processes; - int nr_processes; + size_t max_processes; + size_t nr_processes; get_next_task_fn get_next_task; start_failure_fn start_failure; @@ -1520,7 +1520,7 @@ struct parallel_processes { unsigned shutdown : 1; unsigned ungroup : 1; - int output_owner; + size_t output_owner; struct strbuf buffered_output; /* of finished children */ }; @@ -1541,9 +1541,7 @@ static int default_task_finished(int result, static void kill_children(struct parallel_processes *pp, int signo) { - int i, n = pp->max_processes; - - for (i = 0; i < n; i++) + for (size_t i = 0; i < pp->max_processes; i++) if (pp->children[i].state == GIT_CP_WORKING) kill(pp->children[i].process.pid, signo); } @@ -1558,20 +1556,19 @@ static void handle_children_on_signal(int signo) } static void pp_init(struct parallel_processes *pp, - int n, + size_t n, get_next_task_fn get_next_task, start_failure_fn start_failure, task_finished_fn task_finished, void *data, int ungroup) { - int i; - if (n < 1) n = online_cpus(); pp->max_processes = n; - trace_printf("run_processes_parallel: preparing to run up to %d tasks", n); + trace_printf("run_processes_parallel: preparing to run up to %"PRIuMAX" tasks", + (uintmax_t)n); pp->data = data; if (!get_next_task) @@ -1592,7 +1589,7 @@ static void pp_init(struct parallel_processes *pp, CALLOC_ARRAY(pp->pfd, n); strbuf_init(&pp->buffered_output, 0); - for (i = 0; i < n; i++) { + for (size_t i = 0; i < n; i++) { strbuf_init(&pp->children[i].err, 0); child_process_init(&pp->children[i].process); if (pp->pfd) { @@ -1607,10 +1604,8 @@ static void pp_init(struct parallel_processes *pp, static void pp_cleanup(struct parallel_processes *pp) { - int i; - trace_printf("run_processes_parallel: done"); - for (i = 0; i < pp->max_processes; i++) { + for (size_t i = 0; i < pp->max_processes; i++) { strbuf_release(&pp->children[i].err); child_process_clear(&pp->children[i].process); } @@ -1637,7 +1632,8 @@ static void pp_cleanup(struct parallel_processes *pp) */ static int pp_start_one(struct parallel_processes *pp) { - int i, code; + size_t i; + int code; for (i = 0; i < pp->max_processes; i++) if (pp->children[i].state == GIT_CP_FREE) @@ -1685,8 +1681,6 @@ static int pp_start_one(struct parallel_processes *pp) static void pp_buffer_stderr(struct parallel_processes *pp, int output_timeout) { - int i; - while (poll(pp->pfd, pp->max_processes, output_timeout) < 0) { if (errno == EINTR) continue; @@ -1695,7 +1689,7 @@ static void pp_buffer_stderr(struct parallel_processes *pp, int output_timeout) } /* Buffer output from all pipes. */ - for (i = 0; i < pp->max_processes; i++) { + for (size_t i = 0; i < pp->max_processes; i++) { if (pp->children[i].state == GIT_CP_WORKING && pp->pfd[i].revents & (POLLIN | POLLHUP)) { int n = strbuf_read_once(&pp->children[i].err, @@ -1712,7 +1706,7 @@ static void pp_buffer_stderr(struct parallel_processes *pp, int output_timeout) static void pp_output(struct parallel_processes *pp) { - int i = pp->output_owner; + size_t i = pp->output_owner; if (pp->children[i].state == GIT_CP_WORKING && pp->children[i].err.len) { @@ -1723,8 +1717,8 @@ static void pp_output(struct parallel_processes *pp) static int pp_collect_finished(struct parallel_processes *pp) { - int i, code; - int n = pp->max_processes; + int code; + size_t i, n = pp->max_processes; int result = 0; while (pp->nr_processes > 0) { @@ -1781,7 +1775,7 @@ static int pp_collect_finished(struct parallel_processes *pp) return result; } -void run_processes_parallel(int n, +void run_processes_parallel(size_t n, get_next_task_fn get_next_task, start_failure_fn start_failure, task_finished_fn task_finished, @@ -1815,9 +1809,7 @@ void run_processes_parallel(int n, if (!pp.nr_processes) break; if (ungroup) { - int i; - - for (i = 0; i < pp.max_processes; i++) + for (size_t i = 0; i < pp.max_processes; i++) pp.children[i].state = GIT_CP_WAIT_CLEANUP; } else { pp_buffer_stderr(&pp, output_timeout); @@ -1834,7 +1826,7 @@ void run_processes_parallel(int n, pp_cleanup(&pp); } -void run_processes_parallel_tr2(int n, get_next_task_fn get_next_task, +void run_processes_parallel_tr2(size_t n, get_next_task_fn get_next_task, start_failure_fn start_failure, task_finished_fn task_finished, void *pp_cb, const char *tr2_category, const char *tr2_label) diff --git a/run-command.h b/run-command.h index e76a1b6b5b3..6f7604e1146 100644 --- a/run-command.h +++ b/run-command.h @@ -485,12 +485,12 @@ typedef int (*task_finished_fn)(int result, * API reads that setting. */ extern int run_processes_parallel_ungroup; -void run_processes_parallel(int n, +void run_processes_parallel(size_t n, get_next_task_fn, start_failure_fn, task_finished_fn, void *pp_cb); -void run_processes_parallel_tr2(int n, get_next_task_fn, start_failure_fn, +void run_processes_parallel_tr2(size_t n, get_next_task_fn, start_failure_fn, task_finished_fn, void *pp_cb, const char *tr2_category, const char *tr2_label); From patchwork Wed Oct 12 09:01:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 13004866 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AA985C4332F for ; Wed, 12 Oct 2022 09:02:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229677AbiJLJCa (ORCPT ); Wed, 12 Oct 2022 05:02:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33474 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229934AbiJLJCM (ORCPT ); Wed, 12 Oct 2022 05:02:12 -0400 Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com [IPv6:2a00:1450:4864:20::32f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D342C8C03A for ; Wed, 12 Oct 2022 02:01:51 -0700 (PDT) Received: by mail-wm1-x32f.google.com with SMTP id c3-20020a1c3503000000b003bd21e3dd7aso810005wma.1 for ; Wed, 12 Oct 2022 02:01:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Nes6Yql5QJWPT21WdhN3guFCtnHvRqAOJfIre0tEVmc=; b=nNtgpet70QKyUs/qyWMse98BNY/eR4u7sxC+1oQ1c7ywBUU4KqajN1syHe1p+1SiEh QQIPJLVeqskYTK6HrlGBU/UDuDrEBdUkfxBUtMfXbR7YKChMtvNTKTR1mIEwGFnafI1v QihnazeTt7xuoWK6a/cfJ+VIKqoBXH6Yb8c3WIWdqx+iAganbCBV+Rlyycricc8Wp/aF UjhJg4a1uCAje7i77mvXOQDcAT9AdIOH2b4m3e5V+KYlBR/PMbIsLREMtwTqdBbSBzwf C27F/0JTNaGfFhws0IGQDrBDn2iBhgDv3dkTe2Y5QJtpYCNtJZ5Nd+YwAxPFGRXtzNBe Wkiw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Nes6Yql5QJWPT21WdhN3guFCtnHvRqAOJfIre0tEVmc=; b=OCY+/x92FG30pVETRt3pZ75X2cJ5734T01GSv6uN1NV6IFjnpQFUbJ0cTEqRZdRiSP g43qiTDrZRRSzh1ReO8jgTiFf9C87OvnwHO/VBhkGtCb9M6mjKbOpdhVsgreSBcbGNcF vEe+S/1EA8BCGZgBSV+DB6Uj4Oohvky+CJElon/CGJDBL0C7HT7ZirZzsalf+JCTvYCF wIzzJ/QsfGWCh3g+2DjW6hEURJS9wV89RzYoLciGStM8Ez7JsRFATuwcWbC6KV4GRNNv U7udThjqhMvprj7G2yV3JjmlegOLFCdQICAq1xSHcn44jo5H3GU6fixkhDN6NIDbKKDz yHJQ== X-Gm-Message-State: ACrzQf1bR9/09DHvtks8D+Pfxl3zwPN09/cnHT5BuNpjzlKjS845aURX flVomU6X99YihW/u5qLINUAs7o2MSgeg7A== X-Google-Smtp-Source: AMsMyM7CyJw9usu7TItB+ZmWWuiZWrWbFiy0spCuXdzrWpd15cIZzZv1L3lsa30Jr4ZHdmTm+TqeRA== X-Received: by 2002:a1c:f30d:0:b0:3c3:7c80:67f4 with SMTP id q13-20020a1cf30d000000b003c37c8067f4mr1987501wmq.86.1665565309309; Wed, 12 Oct 2022 02:01:49 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id b1-20020a05600c4e0100b003a3170a7af9sm1280362wmq.4.2022.10.12.02.01.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Oct 2022 02:01:48 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Calvin Wan , Emily Shaffer , Phillip Wood , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v2 10/22] run-command API: don't fall back on online_cpus() Date: Wed, 12 Oct 2022 11:01:18 +0200 Message-Id: X-Mailer: git-send-email 2.38.0.971.ge79ff6d20e7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When a "jobs = 0" is passed let's BUG() out rather than fall back on online_cpus(). The default behavior was added when this API was implemented in c553c72eed6 (run-command: add an asynchronous parallel child processor, 2015-12-15). Most of our code in-tree that scales up to "online_cpus()" by default calls that function by itself. By having these callers of the "run_processes_parallel()" API do the same we can in subsequent commits pass all arguments down as a "const struct". The preceding commit has an overview of the API callers that passed "jobs = 0". There were only two of them (actually three, but they resolved to these two config parsing codepaths). The "fetch.parallel" caller already had a test for the "fetch.parallel=0" case added in 0353c688189 (fetch: do not run a redundant fetch from submodule, 2022-05-16), but there was no such test for "submodule.fetchJobs". Let's add one here. Signed-off-by: Ævar Arnfjörð Bjarmason --- builtin/fetch.c | 2 ++ run-command.c | 7 +++---- submodule-config.c | 2 ++ t/t5526-fetch-submodules.sh | 5 +++++ 4 files changed, 12 insertions(+), 4 deletions(-) diff --git a/builtin/fetch.c b/builtin/fetch.c index 78043fb67ef..82f1da14ec1 100644 --- a/builtin/fetch.c +++ b/builtin/fetch.c @@ -122,6 +122,8 @@ static int git_fetch_config(const char *k, const char *v, void *cb) fetch_parallel_config = git_config_int(k, v); if (fetch_parallel_config < 0) die(_("fetch.parallel cannot be negative")); + if (!fetch_parallel_config) + fetch_parallel_config = online_cpus(); return 0; } diff --git a/run-command.c b/run-command.c index 5a63008b6e1..21c00a48823 100644 --- a/run-command.c +++ b/run-command.c @@ -1562,8 +1562,8 @@ static void pp_init(struct parallel_processes *pp, task_finished_fn task_finished, void *data, int ungroup) { - if (n < 1) - n = online_cpus(); + if (!n) + BUG("you must provide a non-zero number of processes!"); pp->max_processes = n; @@ -1831,8 +1831,7 @@ void run_processes_parallel_tr2(size_t n, get_next_task_fn get_next_task, task_finished_fn task_finished, void *pp_cb, const char *tr2_category, const char *tr2_label) { - trace2_region_enter_printf(tr2_category, tr2_label, NULL, "max:%d", - ((n < 1) ? online_cpus() : n)); + trace2_region_enter_printf(tr2_category, tr2_label, NULL, "max:%d", n); run_processes_parallel(n, get_next_task, start_failure, task_finished, pp_cb); diff --git a/submodule-config.c b/submodule-config.c index cd7ee236a12..4dc61b3a78a 100644 --- a/submodule-config.c +++ b/submodule-config.c @@ -303,6 +303,8 @@ int parse_submodule_fetchjobs(const char *var, const char *value) int fetchjobs = git_config_int(var, value); if (fetchjobs < 0) die(_("negative values not allowed for submodule.fetchJobs")); + if (!fetchjobs) + fetchjobs = online_cpus(); return fetchjobs; } diff --git a/t/t5526-fetch-submodules.sh b/t/t5526-fetch-submodules.sh index e36f9fdf242..98a287ffb90 100755 --- a/t/t5526-fetch-submodules.sh +++ b/t/t5526-fetch-submodules.sh @@ -724,6 +724,11 @@ test_expect_success 'fetching submodules respects parallel settings' ' GIT_TRACE=$(pwd)/trace.out git fetch --jobs 9 && grep "9 tasks" trace.out && >trace.out && + + GIT_TRACE=$(pwd)/trace.out git -c submodule.fetchJobs=0 fetch && + grep "preparing to run up to [0-9]* tasks" trace.out && + ! grep "up to 0 tasks" trace.out && + >trace.out ) ' From patchwork Wed Oct 12 09:01:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 13004867 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 269D0C4332F for ; Wed, 12 Oct 2022 09:02:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229665AbiJLJCn (ORCPT ); Wed, 12 Oct 2022 05:02:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34152 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229906AbiJLJCO (ORCPT ); Wed, 12 Oct 2022 05:02:14 -0400 Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com [IPv6:2a00:1450:4864:20::435]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E2967BA265 for ; Wed, 12 Oct 2022 02:01:53 -0700 (PDT) Received: by mail-wr1-x435.google.com with SMTP id f11so25267501wrm.6 for ; Wed, 12 Oct 2022 02:01:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=SE+yDSHdlLiw5xxz02Ukmxj5wLkJrY5lWnCU5JMCWOU=; b=CjNKm8wvRwrSEsxDTxH/uYuoNpLC7l8TihUlNyh9QD5XIZxogtvZaOZ7Q8uFLEIEOc 43/dorX81X5j34J0zWZJsm7F4eAOksjOpGrGvDMmMLygJEDf2vA2UvmUGyEYZZ13w9nP cncyu5IKQZHsCxYOkxTsIjhYKrmFnf9BxwQfUtvwu8GsdcjkRIWKvhzsYdk7IZSM/yfx GJ1y5LEgBKwmqimOucbBdufPxCJgI5AxQTjB1fq6cPIpGzuerzsDu0SyguPQSDuplL8A aVzJcLTE7Z6zgbhbAZCcvpi31NSGxS9zcPZWnbzarPCubeW0VxBxzupkjGTKdgPZoz70 8/vQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=SE+yDSHdlLiw5xxz02Ukmxj5wLkJrY5lWnCU5JMCWOU=; b=WSC1mJeobbi4Iq+U9be5d24aCscjouv6jBqBHgeoItVQfkxOOst4dTULHy3eNtL7zi wEzuAlTuu+FQEa3O1EINK9ghAd6W+IdFEt2ZMXVTKo/Dvje+bQIZYxtbdnAbcEuCl9/y EximcNZL4z8lWqBE1LKp0Cz6GwQL9y/I7aC6ZlRapbkrV8hQvVWhJ6NITFH+JUgq8PH3 vfvJgTVBgr4YAzoW3xh8RJUgwTt2BN9Vgh+O9ojfemGRmVMGb3fu7HH8MGc/+mdM7O4+ DAUx6GupX2o45Z+Zb8Xw7UAGWZW1JevORyFSWNRtrKGEIfK62VXs/HpEZVpO72BljNdJ lkAw== X-Gm-Message-State: ACrzQf1lOxSVbsYW4r+BRsESs+mbC1WE12v2uDWE3w01tixIjbQomeev AeDrd6niLQONq4OUCU6UVJgN/w4OdxXWSg== X-Google-Smtp-Source: AMsMyM5M2kDZlBbQLJ4xvDVhcS+Fc0rNuq7lgLyv1XhD1kMbSwNnPK9K60LTm6eIJPtDlUQMB2dv+Q== X-Received: by 2002:a5d:526d:0:b0:22e:4f4d:2546 with SMTP id l13-20020a5d526d000000b0022e4f4d2546mr17656919wrc.464.1665565310685; Wed, 12 Oct 2022 02:01:50 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id b1-20020a05600c4e0100b003a3170a7af9sm1280362wmq.4.2022.10.12.02.01.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Oct 2022 02:01:50 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Calvin Wan , Emily Shaffer , Phillip Wood , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v2 11/22] run-command.c: use designated init for pp_init(), add "const" Date: Wed, 12 Oct 2022 11:01:19 +0200 Message-Id: X-Mailer: git-send-email 2.38.0.971.ge79ff6d20e7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Use a designated initializer to initialize those parts of pp_init() that don't need any conditionals for their initialization, this sets us on a path to pp_init() itself into mostly a validation and allocation function. Since we're doing that we can add "const" to some of the members of the "struct parallel_processes", which helps to clarify and self-document this code. E.g. we never alter the "data" pointer we pass t user callbacks, nor (after the preceding change to stop invoking online_cpus()) do we change "max_processes", the same goes for the "ungroup" option. We can also do away with a call to strbuf_init() in favor of macro initialization, and to rely on other fields being NULL'd or zero'd. Making members of a struct "const" rather that the pointer to the struct itself is usually painful, as e.g. it precludes us from incrementally setting up the structure. In this case we only set it up with the assignment in run_process_parallel() and pp_init(), and don't pass the struct pointer around as "const", so making individual members "const" is worth the potential hassle for extra safety. Signed-off-by: Ævar Arnfjörð Bjarmason --- run-command.c | 34 ++++++++++++++-------------------- 1 file changed, 14 insertions(+), 20 deletions(-) diff --git a/run-command.c b/run-command.c index 21c00a48823..6acfd29d8c1 100644 --- a/run-command.c +++ b/run-command.c @@ -1496,9 +1496,9 @@ enum child_state { int run_processes_parallel_ungroup; struct parallel_processes { - void *data; + void *const data; - size_t max_processes; + const size_t max_processes; size_t nr_processes; get_next_task_fn get_next_task; @@ -1518,7 +1518,7 @@ struct parallel_processes { struct pollfd *pfd; unsigned shutdown : 1; - unsigned ungroup : 1; + const unsigned ungroup : 1; size_t output_owner; struct strbuf buffered_output; /* of finished children */ @@ -1556,21 +1556,18 @@ static void handle_children_on_signal(int signo) } static void pp_init(struct parallel_processes *pp, - size_t n, get_next_task_fn get_next_task, start_failure_fn start_failure, - task_finished_fn task_finished, - void *data, int ungroup) + task_finished_fn task_finished) { + const size_t n = pp->max_processes; + if (!n) BUG("you must provide a non-zero number of processes!"); - pp->max_processes = n; - trace_printf("run_processes_parallel: preparing to run up to %"PRIuMAX" tasks", (uintmax_t)n); - pp->data = data; if (!get_next_task) BUG("you need to specify a get_next_task function"); pp->get_next_task = get_next_task; @@ -1578,16 +1575,9 @@ static void pp_init(struct parallel_processes *pp, pp->start_failure = start_failure ? start_failure : default_start_failure; pp->task_finished = task_finished ? task_finished : default_task_finished; - pp->nr_processes = 0; - pp->output_owner = 0; - pp->shutdown = 0; - pp->ungroup = ungroup; CALLOC_ARRAY(pp->children, n); - if (pp->ungroup) - pp->pfd = NULL; - else + if (!pp->ungroup) CALLOC_ARRAY(pp->pfd, n); - strbuf_init(&pp->buffered_output, 0); for (size_t i = 0; i < n; i++) { strbuf_init(&pp->children[i].err, 0); @@ -1785,13 +1775,17 @@ void run_processes_parallel(size_t n, int output_timeout = 100; int spawn_cap = 4; int ungroup = run_processes_parallel_ungroup; - struct parallel_processes pp; + struct parallel_processes pp = { + .max_processes = n, + .data = pp_cb, + .buffered_output = STRBUF_INIT, + .ungroup = ungroup, + }; /* unset for the next API user */ run_processes_parallel_ungroup = 0; - pp_init(&pp, n, get_next_task, start_failure, task_finished, pp_cb, - ungroup); + pp_init(&pp, get_next_task, start_failure, task_finished); while (1) { for (int i = 0; i < spawn_cap && !pp.shutdown && From patchwork Wed Oct 12 09:01:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 13004870 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CCCCAC4332F for ; Wed, 12 Oct 2022 09:02:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229722AbiJLJCu (ORCPT ); Wed, 12 Oct 2022 05:02:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34178 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229769AbiJLJC1 (ORCPT ); Wed, 12 Oct 2022 05:02:27 -0400 Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com [IPv6:2a00:1450:4864:20::42a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 92A2BBBE36 for ; Wed, 12 Oct 2022 02:01:55 -0700 (PDT) Received: by mail-wr1-x42a.google.com with SMTP id a10so25199263wrm.12 for ; Wed, 12 Oct 2022 02:01:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=26HNmMIt+e5BOyybrcX7bo5j81DiUQ9iRBcmvdBxeDw=; b=Ubj/f02vr8EJNUP7KL/dYd/cQvuX5CiFTr35x1D38auewSQMjRxs+WcQhW/W4nCfYq dPg8FyHdbNXdKG3ZGa2ourf7QLpKAbuRisY+sa+51Yiwjcm6f/NP7FLp2ZV/KbZ0hhhF 6klikuHrrxzvKz1LSgDsrjzFwebJ4Wm7uxFs4x/3Seb1coG9P3Lthy/qEuAK34rGyev9 RGEr9Jk5pZrcSw8Eje2ZOU3IqcY5WomL7wVEBlw16+EJUwq0vuIOGViHTA1rnJprq9Yp GlyGn72nY1gAnO4f2pzoCQTFU+U1jvCHrjv5r/RXf+CvA+yyDnRxGLppiE6jJ9oWVyDa LNtw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=26HNmMIt+e5BOyybrcX7bo5j81DiUQ9iRBcmvdBxeDw=; b=PvwtGumQyZkDri9LOaVPr+Y/VjqIRdcNMnBREA+q8vtHLrmDaJRp6P6SEdNO/W4rjr iYeZ9T0HtMMLraVUokctG/q/wFFfIUOkZYmiV21AW/8uShl3Xw6cNbIY7pAE/q5IeUge 1fnNIQutQ04q///ivuxByN5UQN/LOhfyoACmp7O2e2Ax0D7e5VooNXK4HX9GI+eWBbcc oMSTc3czjyFms7MitRGSzROkQqyvRVTVfAadEUuWS3PCsgV1jNkQ11XJhRWg2uD0Hn1H eV5wj+VPOLtdEN4Yp17WghXnMyeAVpiKTzpK9p7tA3s3pkZ8HqoWnz17rwUWoiCW2Zb1 jMCQ== X-Gm-Message-State: ACrzQf33EaKyPFeTQTNTfl3oDj2TwKUH/mZl9Yl+bkgDSzMzr+GrIOFk idnjB1x1vivqLCmCjhLr1N3QPCZR5oUi+w== X-Google-Smtp-Source: AMsMyM4dr6+t6T7aE1htf5z4lvt+x0vK5mEO1Z4POmmjG/e/TgIOqv9uyHXJapAgI9w5vbz9UVI7Tg== X-Received: by 2002:a5d:6484:0:b0:230:7cad:c268 with SMTP id o4-20020a5d6484000000b002307cadc268mr9645994wri.335.1665565312815; Wed, 12 Oct 2022 02:01:52 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id b1-20020a05600c4e0100b003a3170a7af9sm1280362wmq.4.2022.10.12.02.01.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Oct 2022 02:01:51 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Calvin Wan , Emily Shaffer , Phillip Wood , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v2 12/22] run-command API: add nascent "struct run_process_parallel_opts" Date: Wed, 12 Oct 2022 11:01:20 +0200 Message-Id: X-Mailer: git-send-email 2.38.0.971.ge79ff6d20e7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org As noted in fd3aaf53f71 (run-command: add an "ungroup" option to run_process_parallel(), 2022-06-07) which added the "ungroup" passing it to "run_process_parallel()" via the global "run_processes_parallel_ungroup" variable was a compromise to get the smallest possible regression fix for "maint" at the time. This follow-up to that is a start at passing that parameter and others via a new "struct run_process_parallel_opts", as the earlier version[1] of what became fd3aaf53f71 did. For now we're only changing how data is passed internally to "run-command.c", i.e. from "run_process_parallel()" to pp_init(). Subsequent commits will change "run_processes_parallel()" itself, as well as the "run_processes_parallel_tr2()" wrapper function. Since we need to change all of the occurrences of "n" to "opt->SOMETHING" let's take the opportunity and rename the terse "n" to "processes". We could also have picked "max_processes", "jobs", "threads" etc., but as the API is named "run_processes_parallel()" let's go with "processes". 1. https://lore.kernel.org/git/cover-v2-0.8-00000000000-20220518T195858Z-avarab@gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason --- run-command.c | 30 ++++++++++++++++++++---------- run-command.h | 46 +++++++++++++++++++++++++++++++++++++++------- 2 files changed, 59 insertions(+), 17 deletions(-) diff --git a/run-command.c b/run-command.c index 6acfd29d8c1..3042cb26172 100644 --- a/run-command.c +++ b/run-command.c @@ -1556,11 +1556,12 @@ static void handle_children_on_signal(int signo) } static void pp_init(struct parallel_processes *pp, - get_next_task_fn get_next_task, - start_failure_fn start_failure, - task_finished_fn task_finished) + const struct run_process_parallel_opts *opts) { - const size_t n = pp->max_processes; + const size_t n = opts->processes; + get_next_task_fn get_next_task = opts->get_next_task; + start_failure_fn start_failure = opts->start_failure; + task_finished_fn task_finished = opts->task_finished; if (!n) BUG("you must provide a non-zero number of processes!"); @@ -1765,7 +1766,7 @@ static int pp_collect_finished(struct parallel_processes *pp) return result; } -void run_processes_parallel(size_t n, +void run_processes_parallel(size_t processes, get_next_task_fn get_next_task, start_failure_fn start_failure, task_finished_fn task_finished, @@ -1774,18 +1775,27 @@ void run_processes_parallel(size_t n, int code; int output_timeout = 100; int spawn_cap = 4; - int ungroup = run_processes_parallel_ungroup; struct parallel_processes pp = { - .max_processes = n, + .max_processes = processes, .data = pp_cb, .buffered_output = STRBUF_INIT, - .ungroup = ungroup, + .ungroup = run_processes_parallel_ungroup, + }; + const struct run_process_parallel_opts opts = { + .processes = processes, + + .get_next_task = get_next_task, + .start_failure = start_failure, + .task_finished = task_finished, + + .ungroup = run_processes_parallel_ungroup, }; /* unset for the next API user */ run_processes_parallel_ungroup = 0; - pp_init(&pp, get_next_task, start_failure, task_finished); + pp_init(&pp, &opts); + while (1) { for (int i = 0; i < spawn_cap && !pp.shutdown && @@ -1802,7 +1812,7 @@ void run_processes_parallel(size_t n, } if (!pp.nr_processes) break; - if (ungroup) { + if (opts.ungroup) { for (size_t i = 0; i < pp.max_processes; i++) pp.children[i].state = GIT_CP_WAIT_CLEANUP; } else { diff --git a/run-command.h b/run-command.h index 6f7604e1146..03be48e7ce3 100644 --- a/run-command.h +++ b/run-command.h @@ -459,7 +459,42 @@ typedef int (*task_finished_fn)(int result, void *pp_task_cb); /** - * Runs up to n processes at the same time. Whenever a process can be + * Option used by run_processes_parallel(), { 0 }-initialized means no + * options. + */ +struct run_process_parallel_opts +{ + /** + * processes: see 'processes' in run_processes_parallel() below. + */ + size_t processes; + + /** + * ungroup: see 'ungroup' in run_processes_parallel() below. + */ + unsigned int ungroup:1; + + /** + * get_next_task: See get_next_task_fn() above. This must be + * specified. + */ + get_next_task_fn get_next_task; + + /** + * start_failure: See start_failure_fn() above. This can be + * NULL to omit any special handling. + */ + start_failure_fn start_failure; + + /** + * task_finished: See task_finished_fn() above. This can be + * NULL to omit any special handling. + */ + task_finished_fn task_finished; +}; + +/** + * Runs N 'processes' at the same time. Whenever a process can be * started, the callback get_next_task_fn is called to obtain the data * required to start another child process. * @@ -467,9 +502,6 @@ typedef int (*task_finished_fn)(int result, * (both stdout and stderr) is routed to stderr in a manner that output * from different tasks does not interleave (but see "ungroup" below). * - * start_failure_fn and task_finished_fn can be NULL to omit any - * special handling. - * * If the "ungroup" option isn't specified, the API will set the * "stdout_to_stderr" parameter in "struct child_process" and provide * the callbacks with a "struct strbuf *out" parameter to write output @@ -485,13 +517,13 @@ typedef int (*task_finished_fn)(int result, * API reads that setting. */ extern int run_processes_parallel_ungroup; -void run_processes_parallel(size_t n, +void run_processes_parallel(size_t processes, get_next_task_fn, start_failure_fn, task_finished_fn, void *pp_cb); -void run_processes_parallel_tr2(size_t n, get_next_task_fn, start_failure_fn, - task_finished_fn, void *pp_cb, +void run_processes_parallel_tr2(size_t processes, get_next_task_fn, + start_failure_fn, task_finished_fn, void *pp_cb, const char *tr2_category, const char *tr2_label); /** From patchwork Wed Oct 12 09:01:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 13004869 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8C500C433FE for ; Wed, 12 Oct 2022 09:02:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229529AbiJLJCs (ORCPT ); Wed, 12 Oct 2022 05:02:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33590 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229698AbiJLJC1 (ORCPT ); Wed, 12 Oct 2022 05:02:27 -0400 Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C523BBA92A for ; Wed, 12 Oct 2022 02:01:56 -0700 (PDT) Received: by mail-wr1-x432.google.com with SMTP id w18so25251476wro.7 for ; Wed, 12 Oct 2022 02:01:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=uhZnxruwdib1SKtKhYB8j5uaCVItY1O3/SJsSpNO5U4=; b=BSSK8TbLaOlrOE/c5bOwUnKLvYxE1F/WJzvzPyOJgdlViu1Vvo6MF6FQ6tlHLi9KnQ 2y7EWo5y60CVfJ9Um6tHuvrCkrvirAkuXEki1X1k5493JPUyVm4JbYlx5AXTL3wqSXnb 2sL87nNahA8TOmVOB9U/8afqsbCi6hQlz885lVE6UuMtcKpS9uL5leRjykqgbgdkPtuD kUEZWOTPa8qiSTfgnvgFxoYl5pTaauDzek3rbO4Xienuxs+21654bhO21Ll3Pt537BDR sHgjIGsQgo3S7uH99JOQOlGZVzVBiqVAe/cd9lu0fPZvrFGAdvI48WdYM0f3I30aJZx7 rdiQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=uhZnxruwdib1SKtKhYB8j5uaCVItY1O3/SJsSpNO5U4=; b=R2yDYUQMfSK/W4pW4U1KtMqUQAUsgtj3I+8dqltkDC3j5oVkGmn6nTZ40nm+C76gzL WlcO6xxZzAhFe1kbYqmMnT0p3WGUmKvz848maesMvySZyQWHbmdcn5zo2QNTv1jO5rX5 vSke9Y+lQpLdkcjbv0cBxG1A38IP158o1q9453PXoj3USmhxh4FkdzhKM1pkoFZhPzAn 30kZaqWcdDeWMsxltvdJvJmnXGpbw6CjowFwqpmFn66hQrtWbZqQUPwd6GC1wK/AAqCL RESdMBUWkc0XfTG7qKfk/IdsqhKbmdoLjF1AXTZww4vFpMuwlqFaSSZtwaWrNr+MOuB7 T6TQ== X-Gm-Message-State: ACrzQf0nBtsJEspUK36sW9vznUEKfz5n5KcgyO77VFw+7k1JyjHlSeqX Xa871mYbseX1ijzk62KInqbttG4P7rzz6A== X-Google-Smtp-Source: AMsMyM4laT0Ynxiwjt/7XLh22ajZzYxcLsj7au6vhXMdmPMIsqTfB10FERHjfMpyqC7SAzDzRqYYxQ== X-Received: by 2002:adf:f710:0:b0:22c:d668:504e with SMTP id r16-20020adff710000000b0022cd668504emr17373619wrp.98.1665565314515; Wed, 12 Oct 2022 02:01:54 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id b1-20020a05600c4e0100b003a3170a7af9sm1280362wmq.4.2022.10.12.02.01.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Oct 2022 02:01:53 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Calvin Wan , Emily Shaffer , Phillip Wood , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v2 13/22] run-command API: make run_process_parallel{,_tr2}() thin wrappers Date: Wed, 12 Oct 2022 11:01:21 +0200 Message-Id: X-Mailer: git-send-email 2.38.0.971.ge79ff6d20e7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Make the "run_process_parallel()" and "run_process_parallel_tr2()" functions thin wrappers that construct a "struct run_process_parallel_opts" struct, this is in preparation for changing the API users to use a "struct run_process_parallel_opts" directly. Signed-off-by: Ævar Arnfjörð Bjarmason --- run-command.c | 66 ++++++++++++++++++++++++++++++++++----------------- run-command.h | 7 ++++++ 2 files changed, 51 insertions(+), 22 deletions(-) diff --git a/run-command.c b/run-command.c index 3042cb26172..3cdf85876c1 100644 --- a/run-command.c +++ b/run-command.c @@ -1766,36 +1766,30 @@ static int pp_collect_finished(struct parallel_processes *pp) return result; } -void run_processes_parallel(size_t processes, - get_next_task_fn get_next_task, - start_failure_fn start_failure, - task_finished_fn task_finished, - void *pp_cb) +static void run_processes_parallel_1(const struct run_process_parallel_opts *opts, void *pp_cb) { int code; int output_timeout = 100; int spawn_cap = 4; struct parallel_processes pp = { - .max_processes = processes, + .max_processes = opts->processes, .data = pp_cb, .buffered_output = STRBUF_INIT, .ungroup = run_processes_parallel_ungroup, }; - const struct run_process_parallel_opts opts = { - .processes = processes, - - .get_next_task = get_next_task, - .start_failure = start_failure, - .task_finished = task_finished, - - .ungroup = run_processes_parallel_ungroup, - }; + /* options */ + const char *tr2_category = opts->tr2_category; + const char *tr2_label = opts->tr2_label; + const int do_trace2 = tr2_category && tr2_label; /* unset for the next API user */ run_processes_parallel_ungroup = 0; - pp_init(&pp, &opts); + if (do_trace2) + trace2_region_enter_printf(tr2_category, tr2_label, NULL, + "max:%d", opts->processes); + pp_init(&pp, opts); while (1) { for (int i = 0; i < spawn_cap && !pp.shutdown && @@ -1812,7 +1806,7 @@ void run_processes_parallel(size_t processes, } if (!pp.nr_processes) break; - if (opts.ungroup) { + if (opts->ungroup) { for (size_t i = 0; i < pp.max_processes; i++) pp.children[i].state = GIT_CP_WAIT_CLEANUP; } else { @@ -1828,19 +1822,47 @@ void run_processes_parallel(size_t processes, } pp_cleanup(&pp); + + if (do_trace2) + trace2_region_leave(tr2_category, tr2_label, NULL); +} + +void run_processes_parallel(size_t processes, + get_next_task_fn get_next_task, + start_failure_fn start_failure, + task_finished_fn task_finished, + void *pp_cb) +{ + const struct run_process_parallel_opts opts = { + .processes = processes, + .ungroup = run_processes_parallel_ungroup, + + .get_next_task = get_next_task, + .start_failure = start_failure, + .task_finished = task_finished, + }; + + run_processes_parallel_1(&opts, pp_cb); } -void run_processes_parallel_tr2(size_t n, get_next_task_fn get_next_task, +void run_processes_parallel_tr2(size_t processes, get_next_task_fn get_next_task, start_failure_fn start_failure, task_finished_fn task_finished, void *pp_cb, const char *tr2_category, const char *tr2_label) { - trace2_region_enter_printf(tr2_category, tr2_label, NULL, "max:%d", n); + const struct run_process_parallel_opts opts = { + .tr2_category = tr2_category, + .tr2_label = tr2_label, + + .processes = processes, + .ungroup = run_processes_parallel_ungroup, - run_processes_parallel(n, get_next_task, start_failure, - task_finished, pp_cb); + .get_next_task = get_next_task, + .start_failure = start_failure, + .task_finished = task_finished, + }; - trace2_region_leave(tr2_category, tr2_label, NULL); + run_processes_parallel_1(&opts, pp_cb); } int run_auto_maintenance(int quiet) diff --git a/run-command.h b/run-command.h index 03be48e7ce3..bd0b7b70b27 100644 --- a/run-command.h +++ b/run-command.h @@ -464,6 +464,13 @@ typedef int (*task_finished_fn)(int result, */ struct run_process_parallel_opts { + /** + * tr2_category & tr2_label: sets the trace2 category and label for + * logging. These must either be unset, or both of them must be set. + */ + const char *tr2_category; + const char *tr2_label; + /** * processes: see 'processes' in run_processes_parallel() below. */ From patchwork Wed Oct 12 09:01:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 13004871 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id ECAC1C433FE for ; Wed, 12 Oct 2022 09:02:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229724AbiJLJC6 (ORCPT ); Wed, 12 Oct 2022 05:02:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34744 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229975AbiJLJC2 (ORCPT ); Wed, 12 Oct 2022 05:02:28 -0400 Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9C0AEBBF35 for ; Wed, 12 Oct 2022 02:01:59 -0700 (PDT) Received: by mail-wm1-x32c.google.com with SMTP id r8-20020a1c4408000000b003c47d5fd475so798758wma.3 for ; Wed, 12 Oct 2022 02:01:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=HxRZpG2I81ydM6dFAD1SlTvy5KVNSlLWIzWEJPls/rU=; b=CWlWJBWYUTqiGlreln6yyMt6Hh5VHBee7fFUDeu7xFylVhZrOnsxVbKo1uPQajdaYW lvHd0Ostppe7zJket+KUT1OuCmMv5yPuaWCVUe4Zz1yxRH+NMoBfIYsOYKYYPkv0idTG x3OrU+9mekuQBNjwkbQFaJ+WtSbKtlu5GpB6EDOuozt7jq9ldtwC+GwmiS5pWI1Fpghu eSgNWtHhrwmfSZq5u4K1z6VfgHOWZQe1GiAJrdwu/ZFO3tLxnSFNEf4KPbmMXfeTHMuB q9z/MGBvSxC1EvSM1gc3sZUbe+l1ndOrrVXNIZpoyS84Qi8l0CoOnTDtSA+mkjlSWhXZ Ha9g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HxRZpG2I81ydM6dFAD1SlTvy5KVNSlLWIzWEJPls/rU=; b=u2kU6wdrb0fd0HQy/0Ed/VDXVw5gLI93wzgzt7bd+gtZDPKF46dT8jjWxH4owrLdw/ 2t1wW2lsRMEEoMwzpvdVE+jCA20i781VK1zhl1X4FxOPCDLBFXaRoseJkut8+GBZWxZd xW7fFRq73eQwOAoEXN99l/TVT0hZTVMJeslHzuUvdc4J9RSQt6wek1oYWMbqENOHK9QA CQa+/Zt+oDs+tDs1GzTLIAriNqovT1nRR+OarKTfQeG+T+CFgEqGORrVYt8szog87Ziw NinIWJokbV2dgdRifSgotmt0iQ6NlYiYH6rS0QUDJVAYvr6TexPouqX2NMPo2fzyG1XJ 0ZGQ== X-Gm-Message-State: ACrzQf3GVGNXFcl2i2IRq56P8p+TomcEhvDBnngI2evhLIGz9XK64nD6 6PCMZGLIYU2faYmSPwCBM6OQEc4MSOq66w== X-Google-Smtp-Source: AMsMyM7kCu9aLuqUSbD4rAz6oho8hg4D05NA8xJwpjl1IYGlnu5Dtp8jvBpPM5HwvdjTIqa1FLEuXA== X-Received: by 2002:a05:600c:4f0f:b0:3b4:c09b:37c5 with SMTP id l15-20020a05600c4f0f00b003b4c09b37c5mr1992125wmq.181.1665565317242; Wed, 12 Oct 2022 02:01:57 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id b1-20020a05600c4e0100b003a3170a7af9sm1280362wmq.4.2022.10.12.02.01.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Oct 2022 02:01:54 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Calvin Wan , Emily Shaffer , Phillip Wood , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v2 14/22] run-command API: have run_process_parallel() take an "opts" struct Date: Wed, 12 Oct 2022 11:01:22 +0200 Message-Id: X-Mailer: git-send-email 2.38.0.971.ge79ff6d20e7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Have the "run_process_parallel()" function take an "opts" struct, which allows us to eliminate it as a wrapper for "run_processes_parallel_1()". Since the new "run_processes_parallel()" function is able to take an optional "tr2_category" and "tr2_label" via the struct we can at this point migrate all of the users of "run_processes_parallel_tr2()" over to it. But let's not migrate all the API users, only the two users that passed the "ungroup" parameter via the "run_processes_parallel_ungroup" global, allowing us to delete that global in favor of passing "ungroup" via the "opts" struct. As noted in fd3aaf53f71 (run-command: add an "ungroup" option to run_process_parallel(), 2022-06-07) which added "run_processes_parallel_ungroup" passing this as a global was a hack to produce a small regression fix for "maint". Signed-off-by: Ævar Arnfjörð Bjarmason --- hook.c | 23 ++++++++++++++--------- run-command.c | 32 ++++---------------------------- run-command.h | 21 ++++++++++----------- t/helper/test-run-command.c | 31 +++++++++++++++++++++---------- 4 files changed, 49 insertions(+), 58 deletions(-) diff --git a/hook.c b/hook.c index a493939a4fc..a4fa1031f28 100644 --- a/hook.c +++ b/hook.c @@ -114,8 +114,20 @@ int run_hooks_opt(const char *hook_name, struct run_hooks_opt *options) .options = options, }; const char *const hook_path = find_hook(hook_name); - int jobs = 1; int ret = 0; + const struct run_process_parallel_opts opts = { + .tr2_category = "hook", + .tr2_label = hook_name, + + .processes = 1, + .ungroup = 1, + + .get_next_task = pick_next_hook, + .start_failure = notify_start_failure, + .task_finished = notify_hook_finished, + + .data = &cb_data, + }; if (!options) BUG("a struct run_hooks_opt must be provided to run_hooks"); @@ -137,14 +149,7 @@ int run_hooks_opt(const char *hook_name, struct run_hooks_opt *options) cb_data.hook_path = abs_path.buf; } - run_processes_parallel_ungroup = 1; - run_processes_parallel_tr2(jobs, - pick_next_hook, - notify_start_failure, - notify_hook_finished, - &cb_data, - "hook", - hook_name); + run_processes_parallel(&opts); ret = cb_data.rc; cleanup: strbuf_release(&abs_path); diff --git a/run-command.c b/run-command.c index 3cdf85876c1..0956cc572b7 100644 --- a/run-command.c +++ b/run-command.c @@ -1494,7 +1494,6 @@ enum child_state { GIT_CP_WAIT_CLEANUP, }; -int run_processes_parallel_ungroup; struct parallel_processes { void *const data; @@ -1766,25 +1765,22 @@ static int pp_collect_finished(struct parallel_processes *pp) return result; } -static void run_processes_parallel_1(const struct run_process_parallel_opts *opts, void *pp_cb) +void run_processes_parallel(const struct run_process_parallel_opts *opts) { int code; int output_timeout = 100; int spawn_cap = 4; struct parallel_processes pp = { .max_processes = opts->processes, - .data = pp_cb, + .data = opts->data, .buffered_output = STRBUF_INIT, - .ungroup = run_processes_parallel_ungroup, + .ungroup = opts->ungroup, }; /* options */ const char *tr2_category = opts->tr2_category; const char *tr2_label = opts->tr2_label; const int do_trace2 = tr2_category && tr2_label; - /* unset for the next API user */ - run_processes_parallel_ungroup = 0; - if (do_trace2) trace2_region_enter_printf(tr2_category, tr2_label, NULL, "max:%d", opts->processes); @@ -1827,24 +1823,6 @@ static void run_processes_parallel_1(const struct run_process_parallel_opts *opt trace2_region_leave(tr2_category, tr2_label, NULL); } -void run_processes_parallel(size_t processes, - get_next_task_fn get_next_task, - start_failure_fn start_failure, - task_finished_fn task_finished, - void *pp_cb) -{ - const struct run_process_parallel_opts opts = { - .processes = processes, - .ungroup = run_processes_parallel_ungroup, - - .get_next_task = get_next_task, - .start_failure = start_failure, - .task_finished = task_finished, - }; - - run_processes_parallel_1(&opts, pp_cb); -} - void run_processes_parallel_tr2(size_t processes, get_next_task_fn get_next_task, start_failure_fn start_failure, task_finished_fn task_finished, void *pp_cb, @@ -1853,16 +1831,14 @@ void run_processes_parallel_tr2(size_t processes, get_next_task_fn get_next_task const struct run_process_parallel_opts opts = { .tr2_category = tr2_category, .tr2_label = tr2_label, - .processes = processes, - .ungroup = run_processes_parallel_ungroup, .get_next_task = get_next_task, .start_failure = start_failure, .task_finished = task_finished, }; - run_processes_parallel_1(&opts, pp_cb); + run_processes_parallel(&opts); } int run_auto_maintenance(int quiet) diff --git a/run-command.h b/run-command.h index bd0b7b70b27..aabdaf684db 100644 --- a/run-command.h +++ b/run-command.h @@ -498,11 +498,19 @@ struct run_process_parallel_opts * NULL to omit any special handling. */ task_finished_fn task_finished; + + /** + * data: user data, will be passed as "pp_cb" to the callback + * parameters. + */ + void *data; }; /** + * Options are passed via the "struct run_process_parallel_opts" above. + * * Runs N 'processes' at the same time. Whenever a process can be - * started, the callback get_next_task_fn is called to obtain the data + * started, the callback opts.get_next_task is called to obtain the data * required to start another child process. * * The children started via this function run in parallel. Their output @@ -518,17 +526,8 @@ struct run_process_parallel_opts * NULL "struct strbuf *out" parameter, and are responsible for * emitting their own output, including dealing with any race * conditions due to writing in parallel to stdout and stderr. - * The "ungroup" option can be enabled by setting the global - * "run_processes_parallel_ungroup" to "1" before invoking - * run_processes_parallel(), it will be set back to "0" as soon as the - * API reads that setting. */ -extern int run_processes_parallel_ungroup; -void run_processes_parallel(size_t processes, - get_next_task_fn, - start_failure_fn, - task_finished_fn, - void *pp_cb); +void run_processes_parallel(const struct run_process_parallel_opts *opts); void run_processes_parallel_tr2(size_t processes, get_next_task_fn, start_failure_fn, task_finished_fn, void *pp_cb, const char *tr2_category, const char *tr2_label); diff --git a/t/helper/test-run-command.c b/t/helper/test-run-command.c index ee509aefa2f..3ecb830f4a8 100644 --- a/t/helper/test-run-command.c +++ b/t/helper/test-run-command.c @@ -136,7 +136,7 @@ static const char * const testsuite_usage[] = { static int testsuite(int argc, const char **argv) { struct testsuite suite = TESTSUITE_INIT; - int max_jobs = 1, i, ret; + int max_jobs = 1, i, ret = 0; DIR *dir; struct dirent *d; struct option options[] = { @@ -152,6 +152,12 @@ static int testsuite(int argc, const char **argv) "write JUnit-style XML files"), OPT_END() }; + struct run_process_parallel_opts opts = { + .get_next_task = next_test, + .start_failure = test_failed, + .task_finished = test_finished, + .data = &suite, + }; argc = parse_options(argc, argv, NULL, options, testsuite_usage, PARSE_OPT_STOP_AT_NON_OPTION); @@ -192,8 +198,8 @@ static int testsuite(int argc, const char **argv) fprintf(stderr, "Running %"PRIuMAX" tests (%d at a time)\n", (uintmax_t)suite.tests.nr, max_jobs); - run_processes_parallel(max_jobs, next_test, test_failed, - test_finished, &suite); + opts.processes = max_jobs; + run_processes_parallel(&opts); if (suite.failed.nr > 0) { ret = 1; @@ -206,7 +212,7 @@ static int testsuite(int argc, const char **argv) string_list_clear(&suite.tests, 0); string_list_clear(&suite.failed, 0); - return !!ret; + return ret; } static uint64_t my_random_next = 1234; @@ -382,6 +388,9 @@ int cmd__run_command(int argc, const char **argv) struct child_process proc = CHILD_PROCESS_INIT; int jobs; int ret; + struct run_process_parallel_opts opts = { + .data = &proc, + }; if (argc > 1 && !strcmp(argv[1], "testsuite")) return testsuite(argc - 1, argv + 1); @@ -427,7 +436,7 @@ int cmd__run_command(int argc, const char **argv) if (!strcmp(argv[1], "--ungroup")) { argv += 1; argc -= 1; - run_processes_parallel_ungroup = 1; + opts.ungroup = 1; } jobs = atoi(argv[2]); @@ -435,18 +444,20 @@ int cmd__run_command(int argc, const char **argv) strvec_pushv(&proc.args, (const char **)argv + 3); if (!strcmp(argv[1], "run-command-parallel")) { - run_processes_parallel(jobs, parallel_next, NULL, NULL, &proc); + opts.get_next_task = parallel_next; } else if (!strcmp(argv[1], "run-command-abort")) { - run_processes_parallel(jobs, parallel_next, NULL, - task_finished, &proc); + opts.get_next_task = parallel_next; + opts.task_finished = task_finished; } else if (!strcmp(argv[1], "run-command-no-jobs")) { - run_processes_parallel(jobs, no_job, NULL, task_finished, - &proc); + opts.get_next_task = no_job; + opts.task_finished = task_finished; } else { ret = 1; fprintf(stderr, "check usage\n"); goto cleanup; } + opts.processes = jobs; + run_processes_parallel(&opts); ret = 0; cleanup: child_process_clear(&proc); From patchwork Wed Oct 12 09:01:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 13004872 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 88BFBC433FE for ; Wed, 12 Oct 2022 09:03:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229961AbiJLJDJ (ORCPT ); Wed, 12 Oct 2022 05:03:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34746 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229977AbiJLJC2 (ORCPT ); Wed, 12 Oct 2022 05:02:28 -0400 Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7015DBC44A for ; Wed, 12 Oct 2022 02:02:01 -0700 (PDT) Received: by mail-wr1-x432.google.com with SMTP id a3so25307554wrt.0 for ; Wed, 12 Oct 2022 02:02:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=W0exEticuLke4K4IWD+gzWx94T1NJ37WOST9HH/2Gvk=; b=ll+28SGn5hya3lQyu8GDcq6SNgCP1byfcXGRmQxx4sMDm7LMc/SMagTlwhXECt3T0h kCENR5uCRAexhGDlrJRmQd3QAoq7st04iJu+tmPGr3gIrSsuMV5bODysHNQUWpXRpb+p 1YlarGJFAkqX1TtiUL/x82RhtBupJVnXUSxRoyJKxzDf/0PE4vna1+AxjlreTVCo8p8X KUBcrko/BhQnJ1XqqKZzLnEVyJg96PbPzKvrIZowQH5NYgJbSZPUY8TLty86oBDacIiF eW9K7wol6l5jlknJuetUX2UFG60rj5m0jP6zqQ85XhEyxhx5vifqK0ihuZrorg8wsZH8 55ow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=W0exEticuLke4K4IWD+gzWx94T1NJ37WOST9HH/2Gvk=; b=bsGWHc9deivUxSHutYvgzM/wqn1wXhxZxwJwz/ADKF1yOd5nXyrXn22OYyjM+wyACn 1D5a1jTMvFumZMSFDEMA6S+/yT6gyz4LL5dRtVtg0UhXfdaGiL6qeNfJa2CJeB2v4S+k 9ES+VNpGwm3Ec0S8eO2JZRdMCNXBJbaYoSzueOBwb+QrkkiAaSuJq2yrBBEAwe5oXc2f ZOcb8sDCVApfThJrhmCQ+/8tE6SJPnkmsVPLpFsSqQn9wEuZ9CJ/LGqpL6MgwNZ72RWD BMEp0Pqoc15HUVc1nG9f2f/LUYp5V38e/iEm2jPPi+aQY+uIi13h+Vboz54cOEN59ZmL flyQ== X-Gm-Message-State: ACrzQf0oABTlHOl6oE0lukx2ymDNLQKdgX2rpcZgtA9ISm+GoEOS32lK wfGAtR2hCVxk+bHaDn8mpamAOPddO2qY0A== X-Google-Smtp-Source: AMsMyM5Yw7FCzoxCXjnSRqAYF5ctCfzP14nCgk7vk5u65sLItJ40vGqoe3UPTixatQM1PYS5blB3HA== X-Received: by 2002:adf:e192:0:b0:232:3648:776d with SMTP id az18-20020adfe192000000b002323648776dmr690499wrb.698.1665565318582; Wed, 12 Oct 2022 02:01:58 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id b1-20020a05600c4e0100b003a3170a7af9sm1280362wmq.4.2022.10.12.02.01.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Oct 2022 02:01:57 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Calvin Wan , Emily Shaffer , Phillip Wood , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v2 15/22] run-command API: move *_tr2() users to "run_processes_parallel()" Date: Wed, 12 Oct 2022 11:01:23 +0200 Message-Id: X-Mailer: git-send-email 2.38.0.971.ge79ff6d20e7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Have the users of the "run_processes_parallel_tr2()" function use "run_processes_parallel()" instead. In preceding commits the latter was refactored to take a "struct run_process_parallel_opts" argument, since the only reason for "run_processes_parallel_tr2()" to exist was to take arguments that are now a part of that struct we can do away with it. See ee4512ed481 (trace2: create new combined trace facility, 2019-02-22) for the addition of the "*_tr2()" variant of the function, it was used by every caller except "t/helper/test-run-command.c".. Signed-off-by: Ævar Arnfjörð Bjarmason --- builtin/fetch.c | 18 ++++++++++++------ builtin/submodule--helper.c | 16 ++++++++++++---- run-command.c | 18 ------------------ run-command.h | 3 --- submodule.c | 18 ++++++++++++------ 5 files changed, 36 insertions(+), 37 deletions(-) diff --git a/builtin/fetch.c b/builtin/fetch.c index 82f1da14ec1..b06e454cbdd 100644 --- a/builtin/fetch.c +++ b/builtin/fetch.c @@ -1953,15 +1953,21 @@ static int fetch_multiple(struct string_list *list, int max_children) if (max_children != 1 && list->nr != 1) { struct parallel_fetch_state state = { argv.v, list, 0, 0 }; + const struct run_process_parallel_opts opts = { + .tr2_category = "fetch", + .tr2_label = "parallel/fetch", + + .processes = max_children, + + .get_next_task = &fetch_next_remote, + .start_failure = &fetch_failed_to_start, + .task_finished = &fetch_finished, + .data = &state, + }; strvec_push(&argv, "--end-of-options"); - run_processes_parallel_tr2(max_children, - &fetch_next_remote, - &fetch_failed_to_start, - &fetch_finished, - &state, - "fetch", "parallel/fetch"); + run_processes_parallel(&opts); result = state.result; } else for (i = 0; i < list->nr; i++) { diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c index 0b4acb442b2..df526525c1e 100644 --- a/builtin/submodule--helper.c +++ b/builtin/submodule--helper.c @@ -2567,12 +2567,20 @@ static int update_submodules(struct update_data *update_data) { int i, ret = 0; struct submodule_update_clone suc = SUBMODULE_UPDATE_CLONE_INIT; + const struct run_process_parallel_opts opts = { + .tr2_category = "submodule", + .tr2_label = "parallel/update", + + .processes = update_data->max_jobs, + + .get_next_task = update_clone_get_next_task, + .start_failure = update_clone_start_failure, + .task_finished = update_clone_task_finished, + .data = &suc, + }; suc.update_data = update_data; - run_processes_parallel_tr2(suc.update_data->max_jobs, update_clone_get_next_task, - update_clone_start_failure, - update_clone_task_finished, &suc, "submodule", - "parallel/update"); + run_processes_parallel(&opts); /* * We saved the output and put it out all at once now. diff --git a/run-command.c b/run-command.c index 0956cc572b7..64fa0299175 100644 --- a/run-command.c +++ b/run-command.c @@ -1823,24 +1823,6 @@ void run_processes_parallel(const struct run_process_parallel_opts *opts) trace2_region_leave(tr2_category, tr2_label, NULL); } -void run_processes_parallel_tr2(size_t processes, get_next_task_fn get_next_task, - start_failure_fn start_failure, - task_finished_fn task_finished, void *pp_cb, - const char *tr2_category, const char *tr2_label) -{ - const struct run_process_parallel_opts opts = { - .tr2_category = tr2_category, - .tr2_label = tr2_label, - .processes = processes, - - .get_next_task = get_next_task, - .start_failure = start_failure, - .task_finished = task_finished, - }; - - run_processes_parallel(&opts); -} - int run_auto_maintenance(int quiet) { int enabled; diff --git a/run-command.h b/run-command.h index aabdaf684db..e3e1ea01ad9 100644 --- a/run-command.h +++ b/run-command.h @@ -528,9 +528,6 @@ struct run_process_parallel_opts * conditions due to writing in parallel to stdout and stderr. */ void run_processes_parallel(const struct run_process_parallel_opts *opts); -void run_processes_parallel_tr2(size_t processes, get_next_task_fn, - start_failure_fn, task_finished_fn, void *pp_cb, - const char *tr2_category, const char *tr2_label); /** * Convenience function which prepares env for a command to be run in a diff --git a/submodule.c b/submodule.c index bf7a2c79183..f7c71f1f4b1 100644 --- a/submodule.c +++ b/submodule.c @@ -1819,6 +1819,17 @@ int fetch_submodules(struct repository *r, { int i; struct submodule_parallel_fetch spf = SPF_INIT; + const struct run_process_parallel_opts opts = { + .tr2_category = "submodule", + .tr2_label = "parallel/fetch", + + .processes = max_parallel_jobs, + + .get_next_task = get_next_submodule, + .start_failure = fetch_start_failure, + .task_finished = fetch_finish, + .data = &spf, + }; spf.r = r; spf.command_line_option = command_line_option; @@ -1840,12 +1851,7 @@ int fetch_submodules(struct repository *r, calculate_changed_submodule_paths(r, &spf.changed_submodule_names); string_list_sort(&spf.changed_submodule_names); - run_processes_parallel_tr2(max_parallel_jobs, - get_next_submodule, - fetch_start_failure, - fetch_finish, - &spf, - "submodule", "parallel/fetch"); + run_processes_parallel(&opts); if (spf.submodules_with_errors.len > 0) fprintf(stderr, _("Errors during submodule fetch:\n%s"), From patchwork Wed Oct 12 09:01:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 13004873 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 50D41C4332F for ; Wed, 12 Oct 2022 09:03:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229902AbiJLJDK (ORCPT ); Wed, 12 Oct 2022 05:03:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33474 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229517AbiJLJC3 (ORCPT ); Wed, 12 Oct 2022 05:02:29 -0400 Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 39BBA8F964 for ; Wed, 12 Oct 2022 02:02:01 -0700 (PDT) Received: by mail-wr1-x431.google.com with SMTP id bk15so25218174wrb.13 for ; Wed, 12 Oct 2022 02:02:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=BdHbmIww9B7aTqOeTOmToJ+eVkGDXXVo3L8SPSVlDPo=; b=qyq+UVbsGtUaSVgTw1NCeFdueZ8zAUcDlAgXknoz23yxfy0B3dNfOH6S+DH+/DOaEU 3IcuquwcAeNd+MnfIXjzaN4naoDURF7LIxm4+0o5QOr86UjSU9LgwTMkDE3f0UuLX4MF zy6eYWO0SAIRfUzQGxxmacm9r1lO4QVTO0tmNyHB/TrqG3FnkAcTcjfuvaIwAkAylGSm OHUSNoUb9YG2TAD4fC0tCmhMx7bTl9r1zRCVg2gRqJHOVc6TqIT3eN/2oa9oo/cQQrjC coIDqEgL+BG4F3zHFKm4ll6jMC+gapJXW8JhszyxD0UwzpygZgSvO/JdEJ/KgmWVB4wU GWkg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BdHbmIww9B7aTqOeTOmToJ+eVkGDXXVo3L8SPSVlDPo=; b=DFkd/5QmI7LPEeexccAntyEESAipPKIllNgbYx0C0jBKMKGjTMSfbspD2otZteu2js ZKUKud3cgmwvyTOI0qWQEErHuviOR8wd9NIomR6sBTATcmTF+4v2b5r9w5IFpyHiQ1/h +FDqHXWZgdfh5IeXx2DElo2O+EZ2+QQhZ8Ft0x7FtmF9gAvxukdjPTL7Z9OTxVD5LnZn SslTA48kou+ad54Z/rpVyvrzzlEfndl4AEi6Afw8R35XElgbDGrNmbd3LlyoFhRD+VsZ f+DJynv6wv/3fW3dxfKi7/KPSHpT6fg6FImDHKU3iiPmoEgw1gi7dCpGXnlUqVgGN5Be CBtw== X-Gm-Message-State: ACrzQf0qDUTNAsOLSqAlAx2Y37C+WaMfNf6fnsdBZv3oyD6hdYtrK1v0 PFwHsByUxI/wPWmf64NlnGCasDM3qwy6+g== X-Google-Smtp-Source: AMsMyM4WniFzVGqkfY120EtciitgRlUDW9JzHFbPATr6+1Sha8dZ/0dJS9WW1tfCNVfj4KgKDikwGw== X-Received: by 2002:adf:f58b:0:b0:22e:3c4:cf83 with SMTP id f11-20020adff58b000000b0022e03c4cf83mr18192963wro.379.1665565319747; Wed, 12 Oct 2022 02:01:59 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id b1-20020a05600c4e0100b003a3170a7af9sm1280362wmq.4.2022.10.12.02.01.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Oct 2022 02:01:58 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Calvin Wan , Emily Shaffer , Phillip Wood , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v2 16/22] run-command.c: make "struct parallel_processes" const if possible Date: Wed, 12 Oct 2022 11:01:24 +0200 Message-Id: X-Mailer: git-send-email 2.38.0.971.ge79ff6d20e7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Add a "const" to two "struct parallel_processes" parameters where we're not modifying anything in "pp". For kill_children() we'll call it from both the signal handler, and from run_processes_parallel() itself. Adding a "const" there makes it clear that we don't need to modify any state when killing our children. Signed-off-by: Ævar Arnfjörð Bjarmason --- run-command.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/run-command.c b/run-command.c index 64fa0299175..6c343b9b77b 100644 --- a/run-command.c +++ b/run-command.c @@ -1538,7 +1538,7 @@ static int default_task_finished(int result, return 0; } -static void kill_children(struct parallel_processes *pp, int signo) +static void kill_children(const struct parallel_processes *pp, int signo) { for (size_t i = 0; i < pp->max_processes; i++) if (pp->children[i].state == GIT_CP_WORKING) @@ -1694,7 +1694,7 @@ static void pp_buffer_stderr(struct parallel_processes *pp, int output_timeout) } } -static void pp_output(struct parallel_processes *pp) +static void pp_output(const struct parallel_processes *pp) { size_t i = pp->output_owner; From patchwork Wed Oct 12 09:01:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 13004874 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C822DC433FE for ; Wed, 12 Oct 2022 09:03:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229972AbiJLJDN (ORCPT ); Wed, 12 Oct 2022 05:03:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35074 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229934AbiJLJCi (ORCPT ); Wed, 12 Oct 2022 05:02:38 -0400 Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com [IPv6:2a00:1450:4864:20::32d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 28BA6BC455 for ; Wed, 12 Oct 2022 02:02:03 -0700 (PDT) Received: by mail-wm1-x32d.google.com with SMTP id az22-20020a05600c601600b003c6b72797fdso792522wmb.5 for ; Wed, 12 Oct 2022 02:02:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=XXfkWFxR2czHSHKhsFpSxQ+BbDofQrKSG9T2oknMJ00=; b=In9kjOJZqmbNGEqQRVbxNdu7KSPUx3V24/olGlRuiDmZsbdX0+VRK9840YZPieAmq+ /N8jLnKE561fk1xboB6qqGFx+CeuaFlKzNOdRnTCku5iz2dou8eZjNu95nIXQChmYneP MBUrTDWl+IRrSHnpiNLXEJhv71pSPet4HUsF/EwRpSndBDa7u2iuSFIq+vOnmjAZr2Q0 So6sF0cnqGNTIuvtQY7cDY6dTEa4iLew/y+hNRHBcVwIcPzUcb1fDFb5kmJCwqxMrsFg Lw7Wl2fQUD+f+sjj8NxlmhIwfHvqLg3jRxXFo7pjtuPSZOn/4oifdoK/vdpMiWl0bROy Ix/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XXfkWFxR2czHSHKhsFpSxQ+BbDofQrKSG9T2oknMJ00=; b=62pV2CWFGTnWw8QfqZH5eRnlnAQottsV+Az3pLGGSgrcXjM54djjKlGzysrhSVvztj kuPz38EqEDWSQ27WR2iJfibgVDRReuVdnTBqFjGEm1NHZxVUOhw3nc2q9WE9DniunrON 6WdxEieokFgFDjdXF/m96LSa6ornO7VALdYZrPcCcTtLkPggHIN5oo/EXzv/6xskOZsz hXigHNV8/zNO8Bew0rK0KTxvJGivXwlhrtiaZgOs9JjV3rz2afCJ8JXzOce4JZ9leTd+ 7y8+mJgxVRA7Ld8s1WyzdI4SexJYZGaRnirfUKaijh9wmQe4vVgSWWwceYoj2mUtzzKB ZI8w== X-Gm-Message-State: ACrzQf2dkaJXGeyvALFm6JxMnFAik27f4ge++NOSywNsEx+gcqV75sW/ +JCRjskCoHSUzCRcCpa/7mGHFNSL2cUEAg== X-Google-Smtp-Source: AMsMyM7iKiJkxCJ+VycETdEzmNCyUnNbf8U3Iq1PvyfvTHOuPRfQKr3i4VNu8d8LFcm2rzq5Yt2jwA== X-Received: by 2002:a7b:cbc4:0:b0:3b4:fd67:d70d with SMTP id n4-20020a7bcbc4000000b003b4fd67d70dmr2023558wmi.98.1665565321445; Wed, 12 Oct 2022 02:02:01 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id b1-20020a05600c4e0100b003a3170a7af9sm1280362wmq.4.2022.10.12.02.01.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Oct 2022 02:02:00 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Calvin Wan , Emily Shaffer , Phillip Wood , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v2 17/22] run-command.c: don't copy *_fn to "struct parallel_processes" Date: Wed, 12 Oct 2022 11:01:25 +0200 Message-Id: X-Mailer: git-send-email 2.38.0.971.ge79ff6d20e7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org The only remaining reason for copying the callbacks in the "struct run_process_parallel_opts" over to the "struct parallel_processes" was to avoid two if/else statements in case the "start_failure" and "task_finished" callbacks were NULL. Let's handle those cases in pp_start_one() and pp_collect_finished() instead, and avoid the default_* stub functions, and the need to copy this data around. Organizing the code like this made more sense before the "struct run_parallel_parallel_opts" existed, as we'd have needed to pass each of these as a separate parameter. Signed-off-by: Ævar Arnfjörð Bjarmason --- run-command.c | 67 +++++++++++++++++++-------------------------------- 1 file changed, 25 insertions(+), 42 deletions(-) diff --git a/run-command.c b/run-command.c index 6c343b9b77b..4fce1193b66 100644 --- a/run-command.c +++ b/run-command.c @@ -1500,10 +1500,6 @@ struct parallel_processes { const size_t max_processes; size_t nr_processes; - get_next_task_fn get_next_task; - start_failure_fn start_failure; - task_finished_fn task_finished; - struct { enum child_state state; struct child_process process; @@ -1523,21 +1519,6 @@ struct parallel_processes { struct strbuf buffered_output; /* of finished children */ }; -static int default_start_failure(struct strbuf *out, - void *pp_cb, - void *pp_task_cb) -{ - return 0; -} - -static int default_task_finished(int result, - struct strbuf *out, - void *pp_cb, - void *pp_task_cb) -{ - return 0; -} - static void kill_children(const struct parallel_processes *pp, int signo) { for (size_t i = 0; i < pp->max_processes; i++) @@ -1558,9 +1539,6 @@ static void pp_init(struct parallel_processes *pp, const struct run_process_parallel_opts *opts) { const size_t n = opts->processes; - get_next_task_fn get_next_task = opts->get_next_task; - start_failure_fn start_failure = opts->start_failure; - task_finished_fn task_finished = opts->task_finished; if (!n) BUG("you must provide a non-zero number of processes!"); @@ -1568,12 +1546,8 @@ static void pp_init(struct parallel_processes *pp, trace_printf("run_processes_parallel: preparing to run up to %"PRIuMAX" tasks", (uintmax_t)n); - if (!get_next_task) + if (!opts->get_next_task) BUG("you need to specify a get_next_task function"); - pp->get_next_task = get_next_task; - - pp->start_failure = start_failure ? start_failure : default_start_failure; - pp->task_finished = task_finished ? task_finished : default_task_finished; CALLOC_ARRAY(pp->children, n); if (!pp->ungroup) @@ -1620,7 +1594,8 @@ static void pp_cleanup(struct parallel_processes *pp) * <0 no new job was started, user wishes to shutdown early. Use negative code * to signal the children. */ -static int pp_start_one(struct parallel_processes *pp) +static int pp_start_one(struct parallel_processes *pp, + const struct run_process_parallel_opts *opts) { size_t i; int code; @@ -1631,10 +1606,10 @@ static int pp_start_one(struct parallel_processes *pp) if (i == pp->max_processes) BUG("bookkeeping is hard"); - code = pp->get_next_task(&pp->children[i].process, - pp->ungroup ? NULL : &pp->children[i].err, - pp->data, - &pp->children[i].data); + code = opts->get_next_task(&pp->children[i].process, + pp->ungroup ? NULL : &pp->children[i].err, + pp->data, + &pp->children[i].data); if (!code) { if (!pp->ungroup) { strbuf_addbuf(&pp->buffered_output, &pp->children[i].err); @@ -1649,10 +1624,14 @@ static int pp_start_one(struct parallel_processes *pp) pp->children[i].process.no_stdin = 1; if (start_command(&pp->children[i].process)) { - code = pp->start_failure(pp->ungroup ? NULL : - &pp->children[i].err, - pp->data, - pp->children[i].data); + if (opts->start_failure) + code = opts->start_failure(pp->ungroup ? NULL : + &pp->children[i].err, + pp->data, + pp->children[i].data); + else + code = 0; + if (!pp->ungroup) { strbuf_addbuf(&pp->buffered_output, &pp->children[i].err); strbuf_reset(&pp->children[i].err); @@ -1705,7 +1684,8 @@ static void pp_output(const struct parallel_processes *pp) } } -static int pp_collect_finished(struct parallel_processes *pp) +static int pp_collect_finished(struct parallel_processes *pp, + const struct run_process_parallel_opts *opts) { int code; size_t i, n = pp->max_processes; @@ -1720,9 +1700,12 @@ static int pp_collect_finished(struct parallel_processes *pp) code = finish_command(&pp->children[i].process); - code = pp->task_finished(code, pp->ungroup ? NULL : - &pp->children[i].err, pp->data, - pp->children[i].data); + if (opts->task_finished) + code = opts->task_finished(code, pp->ungroup ? NULL : + &pp->children[i].err, pp->data, + pp->children[i].data); + else + code = 0; if (code) result = code; @@ -1791,7 +1774,7 @@ void run_processes_parallel(const struct run_process_parallel_opts *opts) i < spawn_cap && !pp.shutdown && pp.nr_processes < pp.max_processes; i++) { - code = pp_start_one(&pp); + code = pp_start_one(&pp, opts); if (!code) continue; if (code < 0) { @@ -1809,7 +1792,7 @@ void run_processes_parallel(const struct run_process_parallel_opts *opts) pp_buffer_stderr(&pp, output_timeout); pp_output(&pp); } - code = pp_collect_finished(&pp); + code = pp_collect_finished(&pp, opts); if (code) { pp.shutdown = 1; if (code < 0) From patchwork Wed Oct 12 09:01:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 13004875 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B285EC4332F for ; Wed, 12 Oct 2022 09:03:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229975AbiJLJDP (ORCPT ); Wed, 12 Oct 2022 05:03:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35086 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229939AbiJLJCi (ORCPT ); Wed, 12 Oct 2022 05:02:38 -0400 Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com [IPv6:2a00:1450:4864:20::42a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D92F2BBF1A for ; Wed, 12 Oct 2022 02:02:04 -0700 (PDT) Received: by mail-wr1-x42a.google.com with SMTP id a10so25199968wrm.12 for ; Wed, 12 Oct 2022 02:02:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=glP9ttcAeVzVnqaJnd5Y05AI21fF9VTEm3TmGDMARtQ=; b=BbO6TVp/nhXn8BIzvcAgBBoyhOJ0755zfOofnHde0x6MbBJGTnL7YXOhp9q1On6FVE gDFRKi+ut+iN9wBL6ouLlDguAl+IOYfNqUETBCNmE10avQqqyuQPFFfhW3E8NtLlVgCx ql69KoM+0x3wp237LFU0p5T96OGoVdmFUoEsIvSpmD+5XcVoc/kNyCuU9FQdGOi5VvWi mn7b9y4GLWCee9pomwT4hW3WNXP95eruj06nhZO3IGpJ/MKEXIMXZnnP6nwisMawWLcJ 6xjkpBzf2HZEjfFhcQDylgxn5/Dsib6RTHfAU8bB4SniC9Tz6TPesposgIBj3AgE7YU9 JCeg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=glP9ttcAeVzVnqaJnd5Y05AI21fF9VTEm3TmGDMARtQ=; b=4z53N5+xnTdhegv02ak3mdCIC9W/3hzHNh3xI9v9FoP2y/sLo7iUDnC5mvOo+IGrPk edQUsTgKVggggN0H1Hfpu1cFcQq9c+3ZlQFM5BLiJ0gIK60i4MVCvwGsYboN6jXKOOOE Cov8t2K21wTWXHY4lF1Rp8TtNw1Btse5YTpF+ZKKdk4PvYPPLccJnr1gr4wFaUW2WHQv mkHk79Zk0P0t02bGJ9uz6zEx7gRZaPwJloA8VFid89IdMf9ajvfKlpmiyJGDW6IQwAm+ YOtx6NdR+s4Gwulhxkw+kNxIdSJKGq/nuKnbtAFS2ME6ma9dw0nC+OofNs3JzReTsbhQ ZIsw== X-Gm-Message-State: ACrzQf2gnzVGMEyeVnS0JUqYvkMDtGUPzZnPFRrw3jhJ5ORLFo/SdTPW pfLadvXKY0Ap/9dDKl2qgvST4bndVaabtg== X-Google-Smtp-Source: AMsMyM6wLR6pt8+mM7QOUj0zhBqYjnHHWFoKURchWHTpta8W4FbpQEELoz+id3h0EJXxl+hkPxNSTg== X-Received: by 2002:adf:efc5:0:b0:22e:4a6:2d5b with SMTP id i5-20020adfefc5000000b0022e04a62d5bmr17880339wrp.293.1665565323857; Wed, 12 Oct 2022 02:02:03 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id b1-20020a05600c4e0100b003a3170a7af9sm1280362wmq.4.2022.10.12.02.02.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Oct 2022 02:02:02 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Calvin Wan , Emily Shaffer , Phillip Wood , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v2 18/22] run-command.c: don't copy "ungroup" to "struct parallel_processes" Date: Wed, 12 Oct 2022 11:01:26 +0200 Message-Id: X-Mailer: git-send-email 2.38.0.971.ge79ff6d20e7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org As with the *_fn members removed in the preceding commit, let's not copy the "ungroup" member of the "struct run_process_parallel_opts" over to the "struct parallel_processes". Now that we're passing the "opts" down there's no reason to do so. This makes the code easier to follow, as we have a "const" attribute on the "struct run_process_parallel_opts", but not "struct parallel_processes". We do not alter the "ungroup" argument, so storing it in the non-const structure would make this control flow less obvious. Signed-off-by: Ævar Arnfjörð Bjarmason --- run-command.c | 18 ++++++++---------- 1 file changed, 8 insertions(+), 10 deletions(-) diff --git a/run-command.c b/run-command.c index 4fce1193b66..2b1cfbc996f 100644 --- a/run-command.c +++ b/run-command.c @@ -1513,7 +1513,6 @@ struct parallel_processes { struct pollfd *pfd; unsigned shutdown : 1; - const unsigned ungroup : 1; size_t output_owner; struct strbuf buffered_output; /* of finished children */ @@ -1550,7 +1549,7 @@ static void pp_init(struct parallel_processes *pp, BUG("you need to specify a get_next_task function"); CALLOC_ARRAY(pp->children, n); - if (!pp->ungroup) + if (!opts->ungroup) CALLOC_ARRAY(pp->pfd, n); for (size_t i = 0; i < n; i++) { @@ -1607,17 +1606,17 @@ static int pp_start_one(struct parallel_processes *pp, BUG("bookkeeping is hard"); code = opts->get_next_task(&pp->children[i].process, - pp->ungroup ? NULL : &pp->children[i].err, + opts->ungroup ? NULL : &pp->children[i].err, pp->data, &pp->children[i].data); if (!code) { - if (!pp->ungroup) { + if (!opts->ungroup) { strbuf_addbuf(&pp->buffered_output, &pp->children[i].err); strbuf_reset(&pp->children[i].err); } return 1; } - if (!pp->ungroup) { + if (!opts->ungroup) { pp->children[i].process.err = -1; pp->children[i].process.stdout_to_stderr = 1; } @@ -1625,14 +1624,14 @@ static int pp_start_one(struct parallel_processes *pp, if (start_command(&pp->children[i].process)) { if (opts->start_failure) - code = opts->start_failure(pp->ungroup ? NULL : + code = opts->start_failure(opts->ungroup ? NULL : &pp->children[i].err, pp->data, pp->children[i].data); else code = 0; - if (!pp->ungroup) { + if (!opts->ungroup) { strbuf_addbuf(&pp->buffered_output, &pp->children[i].err); strbuf_reset(&pp->children[i].err); } @@ -1701,7 +1700,7 @@ static int pp_collect_finished(struct parallel_processes *pp, code = finish_command(&pp->children[i].process); if (opts->task_finished) - code = opts->task_finished(code, pp->ungroup ? NULL : + code = opts->task_finished(code, opts->ungroup ? NULL : &pp->children[i].err, pp->data, pp->children[i].data); else @@ -1718,7 +1717,7 @@ static int pp_collect_finished(struct parallel_processes *pp, pp->pfd[i].fd = -1; child_process_init(&pp->children[i].process); - if (pp->ungroup) { + if (opts->ungroup) { ; /* no strbuf_*() work to do here */ } else if (i != pp->output_owner) { strbuf_addbuf(&pp->buffered_output, &pp->children[i].err); @@ -1757,7 +1756,6 @@ void run_processes_parallel(const struct run_process_parallel_opts *opts) .max_processes = opts->processes, .data = opts->data, .buffered_output = STRBUF_INIT, - .ungroup = opts->ungroup, }; /* options */ const char *tr2_category = opts->tr2_category; From patchwork Wed Oct 12 09:01:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 13004876 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CEE3CC4332F for ; Wed, 12 Oct 2022 09:03:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229736AbiJLJDk (ORCPT ); Wed, 12 Oct 2022 05:03:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34252 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229990AbiJLJCi (ORCPT ); Wed, 12 Oct 2022 05:02:38 -0400 Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com [IPv6:2a00:1450:4864:20::435]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F0CF1BC45E for ; Wed, 12 Oct 2022 02:02:05 -0700 (PDT) Received: by mail-wr1-x435.google.com with SMTP id f11so25268448wrm.6 for ; Wed, 12 Oct 2022 02:02:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=X/NTPbtC8voArdAzTWPZmQoXgyxV9caBO5BFaYIMXZw=; b=blIPmK2FBzbl6t7EcM6aarvNRVsLn/j971Lcc6m/cJdHcWctCa2z7WOa4BSTJQq46u 3+symio5U550Nmrnhtn6uNm6RA+L6ALhfnOQBGRzUCH5F9lT2uiogw+rqhRmNMpTjzg4 dwfz3nEzU3oo6bYkp8qDSwj/MJe4SCKHFSlhxCraLv5AV+ifBKQlS3c+U++mYjAgN4xW npADykZBldHOoRHGn6Q1Muxp4+jpANbrAaZkNZ8VHft2l+9Tw8rvxFWOs9z87dF/jUmd NPVlBz/ph0NIOPW/BLcMRsq++UQyh4Z+Z38jeQwkqw1i+XXmL5GXM03VJ5QAstiiW55h C47A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=X/NTPbtC8voArdAzTWPZmQoXgyxV9caBO5BFaYIMXZw=; b=7DJAlqBBlBfh3qdlnfJS3C+ETQeJOqV7KUClNcFteXVsPmSLKnXaZsJ+KUIoU3aM3L vXcrGY7MtL9ANrATL7UvtD/URjkmvx9AjiRr9QSneKNIeiAi7SpsemondRMOvaic5uue sDdmq7xZdHdMjXJrjWSX7MddiyJ6BgT54GUT9AcKDJIYHCcwP1SF6uBvYZ00G6GVn/56 OOaJ79K+b9qyP0a/lVLCOVvT+Oypo20ff2gh2PMDQcplgslT8RBH845amzwccCSTN917 T43t85TbfBVN0oBDJUCkgE3FzeNGvrGsIsXNdqXu/LRAFCDyhMLTh/GBzedDDaNwHMWq L4Pw== X-Gm-Message-State: ACrzQf3BvVDM8c6wiAE/CGDJUqUH4I8763Ir+Dcq0WbNEuzsvnRso9wF vqeHe6VtuYtWHWFdHwgaNCfd1FRr6RtDOw== X-Google-Smtp-Source: AMsMyM4DvS1uvdkPDFDOTgIirYQsJ8dZlyQss7Gak70ojE2BSUtZMO1MQAELUa2Ot03bU9ng8OAbYQ== X-Received: by 2002:a5d:64e9:0:b0:22e:7631:bcab with SMTP id g9-20020a5d64e9000000b0022e7631bcabmr18386109wri.36.1665565324891; Wed, 12 Oct 2022 02:02:04 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id b1-20020a05600c4e0100b003a3170a7af9sm1280362wmq.4.2022.10.12.02.02.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Oct 2022 02:02:04 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Calvin Wan , Emily Shaffer , Phillip Wood , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v2 19/22] run-command.c: don't copy "data" to "struct parallel_processes" Date: Wed, 12 Oct 2022 11:01:27 +0200 Message-Id: X-Mailer: git-send-email 2.38.0.971.ge79ff6d20e7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org As with the *_fn members removed in a preceding commit, let's not copy the "data" member of the "struct run_process_parallel_opts" over to the "struct parallel_processes". Now that we're passing the "opts" down there's no reason to do so. This makes the code easier to follow, as we have a "const" attribute on the "struct run_process_parallel_opts", but not "struct parallel_processes". We do not alter the "ungroup" argument, so storing it in the non-const structure would make this control flow less obvious. Signed-off-by: Ævar Arnfjörð Bjarmason --- run-command.c | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/run-command.c b/run-command.c index 2b1cfbc996f..d07ee759b34 100644 --- a/run-command.c +++ b/run-command.c @@ -1495,8 +1495,6 @@ enum child_state { }; struct parallel_processes { - void *const data; - const size_t max_processes; size_t nr_processes; @@ -1607,7 +1605,7 @@ static int pp_start_one(struct parallel_processes *pp, code = opts->get_next_task(&pp->children[i].process, opts->ungroup ? NULL : &pp->children[i].err, - pp->data, + opts->data, &pp->children[i].data); if (!code) { if (!opts->ungroup) { @@ -1626,7 +1624,7 @@ static int pp_start_one(struct parallel_processes *pp, if (opts->start_failure) code = opts->start_failure(opts->ungroup ? NULL : &pp->children[i].err, - pp->data, + opts->data, pp->children[i].data); else code = 0; @@ -1701,7 +1699,7 @@ static int pp_collect_finished(struct parallel_processes *pp, if (opts->task_finished) code = opts->task_finished(code, opts->ungroup ? NULL : - &pp->children[i].err, pp->data, + &pp->children[i].err, opts->data, pp->children[i].data); else code = 0; @@ -1754,7 +1752,6 @@ void run_processes_parallel(const struct run_process_parallel_opts *opts) int spawn_cap = 4; struct parallel_processes pp = { .max_processes = opts->processes, - .data = opts->data, .buffered_output = STRBUF_INIT, }; /* options */ From patchwork Wed Oct 12 09:01:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 13004878 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7CDBAC433FE for ; Wed, 12 Oct 2022 09:03:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229587AbiJLJDo (ORCPT ); Wed, 12 Oct 2022 05:03:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33472 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229949AbiJLJCj (ORCPT ); Wed, 12 Oct 2022 05:02:39 -0400 Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com [IPv6:2a00:1450:4864:20::330]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5BC97BBF1F for ; Wed, 12 Oct 2022 02:02:08 -0700 (PDT) Received: by mail-wm1-x330.google.com with SMTP id n9so4468917wms.1 for ; Wed, 12 Oct 2022 02:02:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=nWiq+1Qk7Pv/yhNrnfikzX8KE+pwJte1q3Q3RPBbztk=; b=gewKUH0N41xcuyCOT+sgXzTzJo5+P8xizw2bd6VPO3FzCOO+FHy67p/eQHksIe/WBb hngUmlUDyTX2XjhVp3v75jURPpmXhKnNHKYrVWAzM53fZA7pBDAt367do3FMo05/Euvc Rjmdvx5+sSzI3ImtZHQ5CZSSG1rZtHhbxNrn0eTTrXtOqt8V3LHycCQtMx9CPT7K6t/5 QSjeexxF1ejMJcHPak8d4/ziXe3tKPGx/G+ggozVLRMcOta045xbvFWQanKBGJUPCHI5 zoBDKkYerBWDg7m8ZqRhoAHH7XGJ8Qj+GSaBQqp3ycXftbY2rS/cOeNLeur2Plwqac3S tAkg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nWiq+1Qk7Pv/yhNrnfikzX8KE+pwJte1q3Q3RPBbztk=; b=cVxuzGRlm1Om2p0fV+1g87Cxb+6S2P+aZ21V2E/Sby063avVRMQ+a5H7Sl2mAdEjfj 1Vxxy7fEfziOG3V7ta1PK6gcNkk2vQ+kTwOFkytZU8fsDXTEJTYN8EaVhtwc051sOR+c zLq2o/JLFeR5NOOmc4AwhgEdwEj0Qx8Pu49tDSpDGXupIlvL3QjA92dYoacuHFs4nwzW 5E+t5EKio43qDEy3lGncv50Xqt2JvJjEXs3+JW+jepyICIOq98ttCRXzQMH5OZ2Bu7lz ctzJITMEDho517o8NwWIp7aW+BSmljpr6scvNQhHWVjbbCaal38T7ln1oC0hdHM4F027 RnTw== X-Gm-Message-State: ACrzQf0XQHa+GwHOtYbOiDmC/oyqqUxUR+UGS8ANZrCw8I96IrJ6ZXFc m4x7Dba1iRf0aaUhS65OyOiv8D55wNZP+Q== X-Google-Smtp-Source: AMsMyM6prcboIIEWnp1fkY9rQWwaE7tVDQWdLShDhtBX4nSontjdku53aEgWRWXaSmwMAUNxkY50Zw== X-Received: by 2002:a1c:f30d:0:b0:3c3:7c80:67f4 with SMTP id q13-20020a1cf30d000000b003c37c8067f4mr1988448wmq.86.1665565326504; Wed, 12 Oct 2022 02:02:06 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id b1-20020a05600c4e0100b003a3170a7af9sm1280362wmq.4.2022.10.12.02.02.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Oct 2022 02:02:05 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Calvin Wan , Emily Shaffer , Phillip Wood , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v2 20/22] run-command.c: use "opts->processes", not "pp->max_processes" Date: Wed, 12 Oct 2022 11:01:28 +0200 Message-Id: X-Mailer: git-send-email 2.38.0.971.ge79ff6d20e7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Neither the "processes" nor "max_processes" members ever change after their initialization, and they're always equivalent, but some existing code used "pp->max_processes" when we were already passing the "opts" to the function, let's use the "opts" directly instead. Signed-off-by: Ævar Arnfjörð Bjarmason --- run-command.c | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/run-command.c b/run-command.c index d07ee759b34..b19b62749e0 100644 --- a/run-command.c +++ b/run-command.c @@ -1597,10 +1597,10 @@ static int pp_start_one(struct parallel_processes *pp, size_t i; int code; - for (i = 0; i < pp->max_processes; i++) + for (i = 0; i < opts->processes; i++) if (pp->children[i].state == GIT_CP_FREE) break; - if (i == pp->max_processes) + if (i == opts->processes) BUG("bookkeeping is hard"); code = opts->get_next_task(&pp->children[i].process, @@ -1685,14 +1685,14 @@ static int pp_collect_finished(struct parallel_processes *pp, const struct run_process_parallel_opts *opts) { int code; - size_t i, n = pp->max_processes; + size_t i; int result = 0; while (pp->nr_processes > 0) { - for (i = 0; i < pp->max_processes; i++) + for (i = 0; i < opts->processes; i++) if (pp->children[i].state == GIT_CP_WAIT_CLEANUP) break; - if (i == pp->max_processes) + if (i == opts->processes) break; code = finish_command(&pp->children[i].process); @@ -1721,6 +1721,8 @@ static int pp_collect_finished(struct parallel_processes *pp, strbuf_addbuf(&pp->buffered_output, &pp->children[i].err); strbuf_reset(&pp->children[i].err); } else { + const size_t n = opts->processes; + strbuf_write(&pp->children[i].err, stderr); strbuf_reset(&pp->children[i].err); @@ -1767,7 +1769,7 @@ void run_processes_parallel(const struct run_process_parallel_opts *opts) while (1) { for (int i = 0; i < spawn_cap && !pp.shutdown && - pp.nr_processes < pp.max_processes; + pp.nr_processes < opts->processes; i++) { code = pp_start_one(&pp, opts); if (!code) @@ -1781,7 +1783,7 @@ void run_processes_parallel(const struct run_process_parallel_opts *opts) if (!pp.nr_processes) break; if (opts->ungroup) { - for (size_t i = 0; i < pp.max_processes; i++) + for (size_t i = 0; i < opts->processes; i++) pp.children[i].state = GIT_CP_WAIT_CLEANUP; } else { pp_buffer_stderr(&pp, output_timeout); From patchwork Wed Oct 12 09:01:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 13004877 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 02A26C433FE for ; Wed, 12 Oct 2022 09:03:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229904AbiJLJDm (ORCPT ); Wed, 12 Oct 2022 05:03:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34266 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229997AbiJLJCk (ORCPT ); Wed, 12 Oct 2022 05:02:40 -0400 Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 93CECB8C27 for ; Wed, 12 Oct 2022 02:02:09 -0700 (PDT) Received: by mail-wr1-x434.google.com with SMTP id bu30so25235481wrb.8 for ; Wed, 12 Oct 2022 02:02:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=hHh9sxSAxHJ86Rf94FzCNVK6qVMsfZrPHIHVMtkKSpQ=; b=EEYQEwIgSj56l73jfLO9ltyiMTjGXlgjdL1j1Lpy6IxoJoR40upFVjrAJpiE5H7ylB le7olY3gIJ2dR7IDApBodhmyCGDmaQupFAGqDDf5j+tI/qhtB0uJJRQfJjNDs0zhDMic /v9eDAojIE3CoyT8CM9seuOG3v0GSdwDq8eSZrFXf6i/DMqh6fa7M9ehBQsy4LRnZVZ6 k+ktgwosdoW5Uh38ypPe+/QrcK7wbh38+0d+aol+QbB2dI7tu8ZDLEm/j+WHnf1rzjl3 xnrJ2sEtH1J+obDKokRyFuxMUr1NMLBK8KngcoFz0kysLO/uRKP9p9ckUT1CjbDKKQQX ukGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hHh9sxSAxHJ86Rf94FzCNVK6qVMsfZrPHIHVMtkKSpQ=; b=pxP552IL8MxsrLtXSRH/wwGLwjpFd6ANuriaP/UolPKom0hZFSde4Dih5rz3tPoHKD 8emhDtTVvwbcNs9VGFDsVNTiQuE8Lec/SUqJVQsbCu0Kr2QaEgq8kzu9tdljbaLQZ3p6 gxI43PfZtmzB5mj9H6sxPPnW7+C9AuqAP5hLQ0SGzsG6ea8E0fVB4i/UPO+lwqnDqcYB /hUAeCD4JI9/pegjqTZFU0sO2PfMT1AIoKR7rCqFl1ZptcqproVj4IL+cIXwCjw/uHSp x7LphVqjIZLxmwNHmbqzk6viH/Z7CfyigWkjiFCHo51utrbVipsTMuiksoNyypgs/+tY z1EQ== X-Gm-Message-State: ACrzQf3nFOi6sOFUue4j0QdXE43xSoFK+wA4/HuSmx5XpSin2hI3EqWp Bit9nx8UZOHdwqmCq3Zs0cDsan29lqqQdA== X-Google-Smtp-Source: AMsMyM5YmV7nUX4jiKFbLChno4zi47khfv1rJj0S050KW1024VY8WdpYXEDsp7DlxHzlF62M5kIzTA== X-Received: by 2002:a5d:4e4f:0:b0:22e:3e3a:a65d with SMTP id r15-20020a5d4e4f000000b0022e3e3aa65dmr17962055wrt.115.1665565327579; Wed, 12 Oct 2022 02:02:07 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id b1-20020a05600c4e0100b003a3170a7af9sm1280362wmq.4.2022.10.12.02.02.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Oct 2022 02:02:06 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Calvin Wan , Emily Shaffer , Phillip Wood , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v2 21/22] run-command.c: pass "opts" further down, and use "opts->processes" Date: Wed, 12 Oct 2022 11:01:29 +0200 Message-Id: X-Mailer: git-send-email 2.38.0.971.ge79ff6d20e7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Continue the migration away from the "max_processes" member of "struct parallel_processes" to the "processes" member of the "struct run_process_parallel_opts", in this case we needed to pass the "opts" further down into pp_cleanup() and pp_buffer_stderr(). Signed-off-by: Ævar Arnfjörð Bjarmason --- run-command.c | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/run-command.c b/run-command.c index b19b62749e0..6b91235ff2d 100644 --- a/run-command.c +++ b/run-command.c @@ -1563,10 +1563,11 @@ static void pp_init(struct parallel_processes *pp, sigchain_push_common(handle_children_on_signal); } -static void pp_cleanup(struct parallel_processes *pp) +static void pp_cleanup(struct parallel_processes *pp, + const struct run_process_parallel_opts *opts) { trace_printf("run_processes_parallel: done"); - for (size_t i = 0; i < pp->max_processes; i++) { + for (size_t i = 0; i < opts->processes; i++) { strbuf_release(&pp->children[i].err); child_process_clear(&pp->children[i].process); } @@ -1645,17 +1646,19 @@ static int pp_start_one(struct parallel_processes *pp, return 0; } -static void pp_buffer_stderr(struct parallel_processes *pp, int output_timeout) +static void pp_buffer_stderr(struct parallel_processes *pp, + const struct run_process_parallel_opts *opts, + int output_timeout) { - while (poll(pp->pfd, pp->max_processes, output_timeout) < 0) { + while (poll(pp->pfd, opts->processes, output_timeout) < 0) { if (errno == EINTR) continue; - pp_cleanup(pp); + pp_cleanup(pp, opts); die_errno("poll"); } /* Buffer output from all pipes. */ - for (size_t i = 0; i < pp->max_processes; i++) { + for (size_t i = 0; i < opts->processes; i++) { if (pp->children[i].state == GIT_CP_WORKING && pp->pfd[i].revents & (POLLIN | POLLHUP)) { int n = strbuf_read_once(&pp->children[i].err, @@ -1786,7 +1789,7 @@ void run_processes_parallel(const struct run_process_parallel_opts *opts) for (size_t i = 0; i < opts->processes; i++) pp.children[i].state = GIT_CP_WAIT_CLEANUP; } else { - pp_buffer_stderr(&pp, output_timeout); + pp_buffer_stderr(&pp, opts, output_timeout); pp_output(&pp); } code = pp_collect_finished(&pp, opts); @@ -1797,7 +1800,7 @@ void run_processes_parallel(const struct run_process_parallel_opts *opts) } } - pp_cleanup(&pp); + pp_cleanup(&pp, opts); if (do_trace2) trace2_region_leave(tr2_category, tr2_label, NULL); From patchwork Wed Oct 12 09:01:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 13004879 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 60724C433FE for ; Wed, 12 Oct 2022 09:03:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229833AbiJLJDq (ORCPT ); Wed, 12 Oct 2022 05:03:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35178 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229711AbiJLJCk (ORCPT ); Wed, 12 Oct 2022 05:02:40 -0400 Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0FE9CB97A4 for ; Wed, 12 Oct 2022 02:02:11 -0700 (PDT) Received: by mail-wr1-x434.google.com with SMTP id b4so25266013wrs.1 for ; Wed, 12 Oct 2022 02:02:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ROCVsX5/tkZdcHQMUTjeGgC1L1zCsLPmUFZsLtReitY=; b=k1iNzqYInVl7RcJFLanH6IDIFh/YHj6fv0VHioMeds72+drs6RqZBfsQ/KMPOiRDT9 L6WD8MG1RocFBAJPSFjNOZyb0ftUwU/nuvg8WPjvjpJCR0CY/1TCnNyiryCTnUq2P06N i45JRyVv3Ug2QA1BtEfthVWRsRcYyOUX4iOY/bl956DL2TqhN6kya5fIymOCtNyJlIl2 W6av8VsjdMJoz80rKdPkcMa9VqK8fpIlYZmDcOcFgXwhsTSoJbNnu5g1TBdm9Rf8oT3q ld+nh5SelXZvWJJlSEG3ZKAkLpBlh2sa85lhcr1ce57OS1ddX732zQ1Ekj0yCiNZicHw 7LPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ROCVsX5/tkZdcHQMUTjeGgC1L1zCsLPmUFZsLtReitY=; b=cD26k7CKAu8PM/P/N5NcWomQXTRzDGh/KBCgF11IFlPRiRXFBTjHWSBpjwJEq8Q18+ i6IznEPTen0WcQA8PdiGoHQK76uthgYYxkR6ap892ku2chko/LrlDUxOYZkhRjHansrK W1FwGjL7hO0dIsJVr13KzJcP35Oxu9bofwLGbP9+n7V0R6stCpuMnTT//CmKlaeZ/a8G ioF1+ngyYHqDn3kmTbw5VizZOL5SnCZw9YI5ZQPOpc16u1yv496LPNqYtZPZW3tYOczf OxWu2gUvr2tIHDAupmJqkmsbmzsmYqnL6oWUhLGWxrrnSawcH7qlIXqvQdKGVcDhYOSK cj4w== X-Gm-Message-State: ACrzQf3+jF3Nh07Hs3pR9tgjXj3j6gSvZ+wTYHxQmrC/2KaQl002Urxk v5pvWx1JMmHlr9aW568EVKwa/NIHaxEevQ== X-Google-Smtp-Source: AMsMyM7Bo+gXSqvDVMERz+KfPRcAdGgfVxcWs+FOt4334WNQ+PtieuqkDCOD64CpyWKIpmXlfD0vQw== X-Received: by 2002:a5d:500a:0:b0:22e:34bd:c907 with SMTP id e10-20020a5d500a000000b0022e34bdc907mr16921913wrt.548.1665565329247; Wed, 12 Oct 2022 02:02:09 -0700 (PDT) Received: from vm.nix.is (vm.nix.is. [2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id b1-20020a05600c4e0100b003a3170a7af9sm1280362wmq.4.2022.10.12.02.02.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Oct 2022 02:02:08 -0700 (PDT) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , Calvin Wan , Emily Shaffer , Phillip Wood , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= Subject: [PATCH v2 22/22] run-command.c: remove "pp->max_processes", add "const" to signal() handler Date: Wed, 12 Oct 2022 11:01:30 +0200 Message-Id: X-Mailer: git-send-email 2.38.0.971.ge79ff6d20e7 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org As with the *_fn members removed in a preceding commit, let's not copy the "processes" member of the "struct run_process_parallel_opts" over to the "struct parallel_processes". In this case we need the number of processes for the kill_children() function, which will be called from a signal handler. To do that adjust this code added in c553c72eed6 (run-command: add an asynchronous parallel child processor, 2015-12-15) so that we use a dedicated "struct parallel_processes_for_signal" for passing data to the signal handler, in addition to the "struct parallel_process" it'll now have access to our "opts" variable. Signed-off-by: Ævar Arnfjörð Bjarmason --- run-command.c | 36 +++++++++++++++++++++++++----------- 1 file changed, 25 insertions(+), 11 deletions(-) diff --git a/run-command.c b/run-command.c index 6b91235ff2d..a6c123ade1a 100644 --- a/run-command.c +++ b/run-command.c @@ -1495,7 +1495,6 @@ enum child_state { }; struct parallel_processes { - const size_t max_processes; size_t nr_processes; struct { @@ -1516,24 +1515,38 @@ struct parallel_processes { struct strbuf buffered_output; /* of finished children */ }; -static void kill_children(const struct parallel_processes *pp, int signo) +struct parallel_processes_for_signal { + const struct run_process_parallel_opts *opts; + const struct parallel_processes *pp; +}; + +static void kill_children(const struct parallel_processes *pp, + const struct run_process_parallel_opts *opts, + int signo) { - for (size_t i = 0; i < pp->max_processes; i++) + for (size_t i = 0; i < opts->processes; i++) if (pp->children[i].state == GIT_CP_WORKING) kill(pp->children[i].process.pid, signo); } -static struct parallel_processes *pp_for_signal; +static void kill_children_signal(const struct parallel_processes_for_signal *pp_sig, + int signo) +{ + kill_children(pp_sig->pp, pp_sig->opts, signo); +} + +static struct parallel_processes_for_signal *pp_for_signal; static void handle_children_on_signal(int signo) { - kill_children(pp_for_signal, signo); + kill_children_signal(pp_for_signal, signo); sigchain_pop(signo); raise(signo); } static void pp_init(struct parallel_processes *pp, - const struct run_process_parallel_opts *opts) + const struct run_process_parallel_opts *opts, + struct parallel_processes_for_signal *pp_sig) { const size_t n = opts->processes; @@ -1559,7 +1572,8 @@ static void pp_init(struct parallel_processes *pp, } } - pp_for_signal = pp; + pp_sig->pp = pp; + pp_sig->opts = opts; sigchain_push_common(handle_children_on_signal); } @@ -1755,8 +1769,8 @@ void run_processes_parallel(const struct run_process_parallel_opts *opts) int code; int output_timeout = 100; int spawn_cap = 4; + struct parallel_processes_for_signal pp_sig; struct parallel_processes pp = { - .max_processes = opts->processes, .buffered_output = STRBUF_INIT, }; /* options */ @@ -1768,7 +1782,7 @@ void run_processes_parallel(const struct run_process_parallel_opts *opts) trace2_region_enter_printf(tr2_category, tr2_label, NULL, "max:%d", opts->processes); - pp_init(&pp, opts); + pp_init(&pp, opts, &pp_sig); while (1) { for (int i = 0; i < spawn_cap && !pp.shutdown && @@ -1779,7 +1793,7 @@ void run_processes_parallel(const struct run_process_parallel_opts *opts) continue; if (code < 0) { pp.shutdown = 1; - kill_children(&pp, -code); + kill_children(&pp, opts, -code); } break; } @@ -1796,7 +1810,7 @@ void run_processes_parallel(const struct run_process_parallel_opts *opts) if (code) { pp.shutdown = 1; if (code < 0) - kill_children(&pp, -code); + kill_children(&pp, opts,-code); } }