| Message ID | patch-07.15-a9810aaa852-20220930T111343Z-avarab@gmail.com (mailing list archive) |
|---|---|
| State | Superseded |
| Headers | show |
| Series | run-command API: pass functions & opts via struct | expand |
> The rename the "n" variable added in c553c72eed6 (run-command: add an > asynchronous parallel child processor, 2015-12-15) to "jobs", and > change the type to an "unsigned int". As we'll see in a subsequent > commit we do pass "0" here, but never "jobs < 0". > - pp->max_processes = n; > + pp->max_processes = jobs; > > - trace_printf("run_processes_parallel: preparing to run up to %d tasks", n); > + trace_printf("run_processes_parallel: preparing to run up to %d tasks", > + jobs); Should we normalize what we call processes/jobs/tasks? They all seem to mean the same thing. I'm leaning towards processes since the function name itself is run_processes_parallel. > - * Runs up to n processes at the same time. Whenever a process can be > + * Runs up to 'jobs' processes at the same time. Whenever a process can be Also should "jobs" be changed to "max_jobs" or whatever normalized name we call it? The comment here can then be changed to: "Runs up to 'max_processes' at the same time."
On 30/09/2022 12:28, Ævar Arnfjörð Bjarmason wrote: > The rename the "n" variable added in c553c72eed6 (run-command: add an > asynchronous parallel child processor, 2015-12-15) to "jobs", and > change the type to an "unsigned int". As we'll see in a subsequent > commit we do pass "0" here, but never "jobs < 0". > > The only users of the "jobs" parameter are: > > * builtin/fetch.c: defaults to 1, reads from the "fetch.parallel" > config. As seen in the code that parses the config added in > d54dea77dba (fetch: let --jobs=<n> parallelize --multiple, too, > 2019-10-05) will die if the git_config_int() return value is < 0. > > It will however pass us jobs = 0, as we'll see in a subsequent > commit. > > * submodule.c: defaults to 1, reads from "submodule.fetchJobs" > config. Read via code originally added in a028a1930c6 (fetching > submodules: respect `submodule.fetchJobs` config option, 2016-02-29). > > It now piggy-backs on the the submodule.fetchJobs code and > validation added in f20e7c1ea24 (submodule: remove > submodule.fetchjobs from submodule-config parsing, 2017-08-02). > > Like builtin/fetch.c it will die if the git_config_int() return > value is < 0, but like builtin/fetch.c it will pass us jobs = 0. > > * builtin/submodule--helper.c: defaults to 1. Read via code > originally added in 2335b870fa7 (submodule update: expose parallelism > to the user, 2016-02-29). > > Since f20e7c1ea24 (submodule: remove submodule.fetchjobs from > submodule-config parsing, 2017-08-02) it shares a config parser and > semantics with the submodule.c caller. > > * hook.c: hardcoded to 1, see 96e7225b310 (hook: add 'run' > subcommand, 2021-12-22). > > * t/helper/test-run-command.c: can be -1 after parsing the arguments, > but will then be overridden to online_cpus() before passing it to > this API. See be5d88e1128 (test-tool run-command: learn to run (parts > of) the testsuite, 2019-10-04). I'm afraid I think this commit is a complete waste of reviewers time. As far as I con see in a later commit all the changes here are rewritten. Why do we need to go from n -> jobs -> opts->jobs rather than just n -> opts->jobs later. Best Wishes Phillip > Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> > --- > run-command.c | 33 +++++++++++++++++---------------- > run-command.h | 6 +++--- > 2 files changed, 20 insertions(+), 19 deletions(-) > > diff --git a/run-command.c b/run-command.c > index 642e6b6e057..80d282dbdb6 100644 > --- a/run-command.c > +++ b/run-command.c > @@ -1500,8 +1500,8 @@ int run_processes_parallel_ungroup; > struct parallel_processes { > void *data; > > - int max_processes; > - int nr_processes; > + unsigned int max_processes; > + unsigned int nr_processes; > > get_next_task_fn get_next_task; > start_failure_fn start_failure; > @@ -1560,20 +1560,21 @@ static void handle_children_on_signal(int signo) > } > > static void pp_init(struct parallel_processes *pp, > - int n, > + unsigned int jobs, > get_next_task_fn get_next_task, > start_failure_fn start_failure, > task_finished_fn task_finished, > void *data, int ungroup) > { > - int i; > + unsigned int i; > > - if (n < 1) > - n = online_cpus(); > + if (jobs < 1) > + jobs = online_cpus(); > > - pp->max_processes = n; > + pp->max_processes = jobs; > > - trace_printf("run_processes_parallel: preparing to run up to %d tasks", n); > + trace_printf("run_processes_parallel: preparing to run up to %d tasks", > + jobs); > > pp->data = data; > if (!get_next_task) > @@ -1587,14 +1588,14 @@ static void pp_init(struct parallel_processes *pp, > pp->output_owner = 0; > pp->shutdown = 0; > pp->ungroup = ungroup; > - CALLOC_ARRAY(pp->children, n); > + CALLOC_ARRAY(pp->children, jobs); > if (pp->ungroup) > pp->pfd = NULL; > else > - CALLOC_ARRAY(pp->pfd, n); > + CALLOC_ARRAY(pp->pfd, jobs); > strbuf_init(&pp->buffered_output, 0); > > - for (i = 0; i < n; i++) { > + for (i = 0; i < jobs; i++) { > strbuf_init(&pp->children[i].err, 0); > child_process_init(&pp->children[i].process); > if (pp->pfd) { > @@ -1783,7 +1784,7 @@ static int pp_collect_finished(struct parallel_processes *pp) > return result; > } > > -void run_processes_parallel(int n, > +void run_processes_parallel(unsigned int jobs, > get_next_task_fn get_next_task, > start_failure_fn start_failure, > task_finished_fn task_finished, > @@ -1798,7 +1799,7 @@ void run_processes_parallel(int n, > /* unset for the next API user */ > run_processes_parallel_ungroup = 0; > > - pp_init(&pp, n, get_next_task, start_failure, task_finished, pp_cb, > + pp_init(&pp, jobs, get_next_task, start_failure, task_finished, pp_cb, > ungroup); > while (1) { > for (i = 0; > @@ -1836,15 +1837,15 @@ void run_processes_parallel(int n, > pp_cleanup(&pp); > } > > -void run_processes_parallel_tr2(int n, get_next_task_fn get_next_task, > +void run_processes_parallel_tr2(unsigned int jobs, get_next_task_fn get_next_task, > start_failure_fn start_failure, > task_finished_fn task_finished, void *pp_cb, > const char *tr2_category, const char *tr2_label) > { > trace2_region_enter_printf(tr2_category, tr2_label, NULL, "max:%d", > - ((n < 1) ? online_cpus() : n)); > + ((jobs < 1) ? online_cpus() : jobs)); > > - run_processes_parallel(n, get_next_task, start_failure, > + run_processes_parallel(jobs, get_next_task, start_failure, > task_finished, pp_cb); > > trace2_region_leave(tr2_category, tr2_label, NULL); > diff --git a/run-command.h b/run-command.h > index e76a1b6b5b3..4502bdc64dc 100644 > --- a/run-command.h > +++ b/run-command.h > @@ -459,7 +459,7 @@ typedef int (*task_finished_fn)(int result, > void *pp_task_cb); > > /** > - * Runs up to n processes at the same time. Whenever a process can be > + * Runs up to 'jobs' processes at the same time. Whenever a process can be > * started, the callback get_next_task_fn is called to obtain the data > * required to start another child process. > * > @@ -485,12 +485,12 @@ typedef int (*task_finished_fn)(int result, > * API reads that setting. > */ > extern int run_processes_parallel_ungroup; > -void run_processes_parallel(int n, > +void run_processes_parallel(unsigned int jobs, > get_next_task_fn, > start_failure_fn, > task_finished_fn, > void *pp_cb); > -void run_processes_parallel_tr2(int n, get_next_task_fn, start_failure_fn, > +void run_processes_parallel_tr2(unsigned int jobs, get_next_task_fn, start_failure_fn, > task_finished_fn, void *pp_cb, > const char *tr2_category, const char *tr2_label); >
diff --git a/run-command.c b/run-command.c index 642e6b6e057..80d282dbdb6 100644 --- a/run-command.c +++ b/run-command.c @@ -1500,8 +1500,8 @@ int run_processes_parallel_ungroup; struct parallel_processes { void *data; - int max_processes; - int nr_processes; + unsigned int max_processes; + unsigned int nr_processes; get_next_task_fn get_next_task; start_failure_fn start_failure; @@ -1560,20 +1560,21 @@ static void handle_children_on_signal(int signo) } static void pp_init(struct parallel_processes *pp, - int n, + unsigned int jobs, get_next_task_fn get_next_task, start_failure_fn start_failure, task_finished_fn task_finished, void *data, int ungroup) { - int i; + unsigned int i; - if (n < 1) - n = online_cpus(); + if (jobs < 1) + jobs = online_cpus(); - pp->max_processes = n; + pp->max_processes = jobs; - trace_printf("run_processes_parallel: preparing to run up to %d tasks", n); + trace_printf("run_processes_parallel: preparing to run up to %d tasks", + jobs); pp->data = data; if (!get_next_task) @@ -1587,14 +1588,14 @@ static void pp_init(struct parallel_processes *pp, pp->output_owner = 0; pp->shutdown = 0; pp->ungroup = ungroup; - CALLOC_ARRAY(pp->children, n); + CALLOC_ARRAY(pp->children, jobs); if (pp->ungroup) pp->pfd = NULL; else - CALLOC_ARRAY(pp->pfd, n); + CALLOC_ARRAY(pp->pfd, jobs); strbuf_init(&pp->buffered_output, 0); - for (i = 0; i < n; i++) { + for (i = 0; i < jobs; i++) { strbuf_init(&pp->children[i].err, 0); child_process_init(&pp->children[i].process); if (pp->pfd) { @@ -1783,7 +1784,7 @@ static int pp_collect_finished(struct parallel_processes *pp) return result; } -void run_processes_parallel(int n, +void run_processes_parallel(unsigned int jobs, get_next_task_fn get_next_task, start_failure_fn start_failure, task_finished_fn task_finished, @@ -1798,7 +1799,7 @@ void run_processes_parallel(int n, /* unset for the next API user */ run_processes_parallel_ungroup = 0; - pp_init(&pp, n, get_next_task, start_failure, task_finished, pp_cb, + pp_init(&pp, jobs, get_next_task, start_failure, task_finished, pp_cb, ungroup); while (1) { for (i = 0; @@ -1836,15 +1837,15 @@ void run_processes_parallel(int n, pp_cleanup(&pp); } -void run_processes_parallel_tr2(int n, get_next_task_fn get_next_task, +void run_processes_parallel_tr2(unsigned int jobs, get_next_task_fn get_next_task, start_failure_fn start_failure, task_finished_fn task_finished, void *pp_cb, const char *tr2_category, const char *tr2_label) { trace2_region_enter_printf(tr2_category, tr2_label, NULL, "max:%d", - ((n < 1) ? online_cpus() : n)); + ((jobs < 1) ? online_cpus() : jobs)); - run_processes_parallel(n, get_next_task, start_failure, + run_processes_parallel(jobs, get_next_task, start_failure, task_finished, pp_cb); trace2_region_leave(tr2_category, tr2_label, NULL); diff --git a/run-command.h b/run-command.h index e76a1b6b5b3..4502bdc64dc 100644 --- a/run-command.h +++ b/run-command.h @@ -459,7 +459,7 @@ typedef int (*task_finished_fn)(int result, void *pp_task_cb); /** - * Runs up to n processes at the same time. Whenever a process can be + * Runs up to 'jobs' processes at the same time. Whenever a process can be * started, the callback get_next_task_fn is called to obtain the data * required to start another child process. * @@ -485,12 +485,12 @@ typedef int (*task_finished_fn)(int result, * API reads that setting. */ extern int run_processes_parallel_ungroup; -void run_processes_parallel(int n, +void run_processes_parallel(unsigned int jobs, get_next_task_fn, start_failure_fn, task_finished_fn, void *pp_cb); -void run_processes_parallel_tr2(int n, get_next_task_fn, start_failure_fn, +void run_processes_parallel_tr2(unsigned int jobs, get_next_task_fn, start_failure_fn, task_finished_fn, void *pp_cb, const char *tr2_category, const char *tr2_label);
The rename the "n" variable added in c553c72eed6 (run-command: add an asynchronous parallel child processor, 2015-12-15) to "jobs", and change the type to an "unsigned int". As we'll see in a subsequent commit we do pass "0" here, but never "jobs < 0". The only users of the "jobs" parameter are: * builtin/fetch.c: defaults to 1, reads from the "fetch.parallel" config. As seen in the code that parses the config added in d54dea77dba (fetch: let --jobs=<n> parallelize --multiple, too, 2019-10-05) will die if the git_config_int() return value is < 0. It will however pass us jobs = 0, as we'll see in a subsequent commit. * submodule.c: defaults to 1, reads from "submodule.fetchJobs" config. Read via code originally added in a028a1930c6 (fetching submodules: respect `submodule.fetchJobs` config option, 2016-02-29). It now piggy-backs on the the submodule.fetchJobs code and validation added in f20e7c1ea24 (submodule: remove submodule.fetchjobs from submodule-config parsing, 2017-08-02). Like builtin/fetch.c it will die if the git_config_int() return value is < 0, but like builtin/fetch.c it will pass us jobs = 0. * builtin/submodule--helper.c: defaults to 1. Read via code originally added in 2335b870fa7 (submodule update: expose parallelism to the user, 2016-02-29). Since f20e7c1ea24 (submodule: remove submodule.fetchjobs from submodule-config parsing, 2017-08-02) it shares a config parser and semantics with the submodule.c caller. * hook.c: hardcoded to 1, see 96e7225b310 (hook: add 'run' subcommand, 2021-12-22). * t/helper/test-run-command.c: can be -1 after parsing the arguments, but will then be overridden to online_cpus() before passing it to this API. See be5d88e1128 (test-tool run-command: learn to run (parts of) the testsuite, 2019-10-04). Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> --- run-command.c | 33 +++++++++++++++++---------------- run-command.h | 6 +++--- 2 files changed, 20 insertions(+), 19 deletions(-)