| Message ID | 310e361371efc156c3aaac94439bdbeaa965155f.1723533091.git.ps@pks.im (mailing list archive) |
|---|---|
| State | Superseded |
| Headers | show |
| Series | builtin/maintenance: fix auto-detach with non-standard tasks | expand |
On Tue Aug 13, 2024 at 5:17 PM AEST, Patrick Steinhardt wrote: > Note that there is one small gotcha here with the "--prune" option. Next > to passing a string, this option also accepts the "--no-prune" option > that overrides the default or configured value. We thus need to discern > between the option not having been passed by the user and the negative > variant of it. This is done by using a simple sentinel value that lets > us discern these cases. > > @@ -644,12 +673,15 @@ int cmd_gc(int argc, const char **argv, const char *prefix) > struct child_process rerere_cmd = CHILD_PROCESS_INIT; > struct maintenance_run_opts opts = {0}; > struct gc_config cfg = GC_CONFIG_INIT; > + const char *prune_expire_sentinel = "sentinel"; > + const char *prune_expire_arg = prune_expire_sentinel; > + int ret; > > struct option builtin_gc_options[] = { > OPT__QUIET(&quiet, N_("suppress progress reporting")), > - { OPTION_STRING, 0, "prune", &cfg.prune_expire, N_("date"), > + { OPTION_STRING, 0, "prune", &prune_expire_arg, N_("date"), > N_("prune unreferenced objects"), > - PARSE_OPT_OPTARG, NULL, (intptr_t)cfg.prune_expire }, > + PARSE_OPT_OPTARG, NULL, (intptr_t)prune_expire_arg }, > OPT_BOOL(0, "cruft", &cfg.cruft_packs, N_("pack unreferenced objects separately")), > OPT_MAGNITUDE(0, "max-cruft-size", &cfg.max_cruft_size, > N_("with --cruft, limit the size of new cruft packs")), I was wondering how the `no-*` options worked since they're not explicitly defined in the `builtin_gc_options` array. I guess they're handled internally by `parse_options()`, and if the `no-` prefix was present, it leaves that argument unset.
On Thu, Aug 15, 2024 at 03:22:04PM +1000, James Liu wrote: > On Tue Aug 13, 2024 at 5:17 PM AEST, Patrick Steinhardt wrote: > > Note that there is one small gotcha here with the "--prune" option. Next > > to passing a string, this option also accepts the "--no-prune" option > > that overrides the default or configured value. We thus need to discern > > between the option not having been passed by the user and the negative > > variant of it. This is done by using a simple sentinel value that lets > > us discern these cases. > > > > @@ -644,12 +673,15 @@ int cmd_gc(int argc, const char **argv, const char *prefix) > > struct child_process rerere_cmd = CHILD_PROCESS_INIT; > > struct maintenance_run_opts opts = {0}; > > struct gc_config cfg = GC_CONFIG_INIT; > > + const char *prune_expire_sentinel = "sentinel"; > > + const char *prune_expire_arg = prune_expire_sentinel; > > + int ret; > > > > struct option builtin_gc_options[] = { > > OPT__QUIET(&quiet, N_("suppress progress reporting")), > > - { OPTION_STRING, 0, "prune", &cfg.prune_expire, N_("date"), > > + { OPTION_STRING, 0, "prune", &prune_expire_arg, N_("date"), > > N_("prune unreferenced objects"), > > - PARSE_OPT_OPTARG, NULL, (intptr_t)cfg.prune_expire }, > > + PARSE_OPT_OPTARG, NULL, (intptr_t)prune_expire_arg }, > > OPT_BOOL(0, "cruft", &cfg.cruft_packs, N_("pack unreferenced objects separately")), > > OPT_MAGNITUDE(0, "max-cruft-size", &cfg.max_cruft_size, > > N_("with --cruft, limit the size of new cruft packs")), > > I was wondering how the `no-*` options worked since they're not > explicitly defined in the `builtin_gc_options` array. I guess > they're handled internally by `parse_options()`, and if the `no-` prefix > was present, it leaves that argument unset. Yes, this is being handled by "parse-options.c". But With the `no-` prefix it won't leave the argument unset, but will explicitly unset it. So if the argument was already set to some value X, that value would be unset when parsing the `no-` variant. Patrick
On 8/13/24 3:17 AM, Patrick Steinhardt wrote: > We're leaking config values in git-gc(1) when those values are tracked > as strings. Introduce a new `gc_config_release()` function that releases > this memory to plug those leaks and release old values before populating > the config fields via `git_config_string()` et al. > > Note that there is one small gotcha here with the "--prune" option. Next > to passing a string, this option also accepts the "--no-prune" option > that overrides the default or configured value. We thus need to discern > between the option not having been passed by the user and the negative > variant of it. This is done by using a simple sentinel value that lets > us discern these cases. I'm glad to see that you are correcting this in the very next patch where I pointed out my concern about it. Excellent. > - const char *gc_log_expire; > - const char *prune_expire; > - const char *prune_worktrees_expire; > + char *gc_log_expire; > + char *prune_expire; > + char *prune_worktrees_expire; > - .gc_log_expire = "1.day.ago", \ > - .prune_expire = "2.weeks.ago", \ > - .prune_worktrees_expire = "3.months.ago", \ > + .gc_log_expire = xstrdup("1.day.ago"), \ > + .prune_expire = xstrdup("2.weeks.ago"), \ > + .prune_worktrees_expire = xstrdup("3.months.ago"), \ Using xstrdup() is now possible that these aren't globals. Good. > static void gc_config(struct gc_config *cfg) > { > const char *value; > + char *owned = NULL; > > if (!git_config_get_value("gc.packrefs", &value)) { > if (value && !strcmp(value, "notbare")) > @@ -185,15 +195,34 @@ static void gc_config(struct gc_config *cfg) > git_config_get_bool("gc.autodetach", &cfg->detach_auto); > git_config_get_bool("gc.cruftpacks", &cfg->cruft_packs); > git_config_get_ulong("gc.maxcruftsize", &cfg->max_cruft_size); > - git_config_get_expiry("gc.pruneexpire", (char **) &cfg->prune_expire); > - git_config_get_expiry("gc.worktreepruneexpire", (char **) &cfg->prune_worktrees_expire); > - git_config_get_expiry("gc.logexpiry", (char **) &cfg->gc_log_expire); > + > + if (!git_config_get_expiry("gc.pruneexpire", &owned)) { > + free(cfg->prune_expire); > + cfg->prune_expire = owned; > + } Ah. Good logic of "if I get a (possibly NULL) result, then free the old value before setting the new one". > diff --git a/t/t5304-prune.sh b/t/t5304-prune.sh > index 1f1f664871..e641df0116 100755 > --- a/t/t5304-prune.sh > +++ b/t/t5304-prune.sh > @@ -7,6 +7,7 @@ test_description='prune' > GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main > export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME > > +TEST_PASSES_SANITIZE_LEAK=true Fantastic. Thanks, -Stolee
diff --git a/builtin/gc.c b/builtin/gc.c index eee7401647..a93cfa147e 100644 --- a/builtin/gc.c +++ b/builtin/gc.c @@ -139,9 +139,9 @@ struct gc_config { int gc_auto_threshold; int gc_auto_pack_limit; int detach_auto; - const char *gc_log_expire; - const char *prune_expire; - const char *prune_worktrees_expire; + char *gc_log_expire; + char *prune_expire; + char *prune_worktrees_expire; char *repack_filter; char *repack_filter_to; unsigned long big_pack_threshold; @@ -157,15 +157,25 @@ struct gc_config { .gc_auto_threshold = 6700, \ .gc_auto_pack_limit = 50, \ .detach_auto = 1, \ - .gc_log_expire = "1.day.ago", \ - .prune_expire = "2.weeks.ago", \ - .prune_worktrees_expire = "3.months.ago", \ + .gc_log_expire = xstrdup("1.day.ago"), \ + .prune_expire = xstrdup("2.weeks.ago"), \ + .prune_worktrees_expire = xstrdup("3.months.ago"), \ .max_delta_cache_size = DEFAULT_DELTA_CACHE_SIZE, \ } +static void gc_config_release(struct gc_config *cfg) +{ + free(cfg->gc_log_expire); + free(cfg->prune_expire); + free(cfg->prune_worktrees_expire); + free(cfg->repack_filter); + free(cfg->repack_filter_to); +} + static void gc_config(struct gc_config *cfg) { const char *value; + char *owned = NULL; if (!git_config_get_value("gc.packrefs", &value)) { if (value && !strcmp(value, "notbare")) @@ -185,15 +195,34 @@ static void gc_config(struct gc_config *cfg) git_config_get_bool("gc.autodetach", &cfg->detach_auto); git_config_get_bool("gc.cruftpacks", &cfg->cruft_packs); git_config_get_ulong("gc.maxcruftsize", &cfg->max_cruft_size); - git_config_get_expiry("gc.pruneexpire", (char **) &cfg->prune_expire); - git_config_get_expiry("gc.worktreepruneexpire", (char **) &cfg->prune_worktrees_expire); - git_config_get_expiry("gc.logexpiry", (char **) &cfg->gc_log_expire); + + if (!git_config_get_expiry("gc.pruneexpire", &owned)) { + free(cfg->prune_expire); + cfg->prune_expire = owned; + } + + if (!git_config_get_expiry("gc.worktreepruneexpire", &owned)) { + free(cfg->prune_worktrees_expire); + cfg->prune_worktrees_expire = owned; + } + + if (!git_config_get_expiry("gc.logexpiry", &owned)) { + free(cfg->gc_log_expire); + cfg->gc_log_expire = owned; + } git_config_get_ulong("gc.bigpackthreshold", &cfg->big_pack_threshold); git_config_get_ulong("pack.deltacachesize", &cfg->max_delta_cache_size); - git_config_get_string("gc.repackfilter", &cfg->repack_filter); - git_config_get_string("gc.repackfilterto", &cfg->repack_filter_to); + if (!git_config_get_string("gc.repackfilter", &owned)) { + free(cfg->repack_filter); + cfg->repack_filter = owned; + } + + if (!git_config_get_string("gc.repackfilterto", &owned)) { + free(cfg->repack_filter_to); + cfg->repack_filter_to = owned; + } git_config(git_default_config, NULL); } @@ -644,12 +673,15 @@ int cmd_gc(int argc, const char **argv, const char *prefix) struct child_process rerere_cmd = CHILD_PROCESS_INIT; struct maintenance_run_opts opts = {0}; struct gc_config cfg = GC_CONFIG_INIT; + const char *prune_expire_sentinel = "sentinel"; + const char *prune_expire_arg = prune_expire_sentinel; + int ret; struct option builtin_gc_options[] = { OPT__QUIET(&quiet, N_("suppress progress reporting")), - { OPTION_STRING, 0, "prune", &cfg.prune_expire, N_("date"), + { OPTION_STRING, 0, "prune", &prune_expire_arg, N_("date"), N_("prune unreferenced objects"), - PARSE_OPT_OPTARG, NULL, (intptr_t)cfg.prune_expire }, + PARSE_OPT_OPTARG, NULL, (intptr_t)prune_expire_arg }, OPT_BOOL(0, "cruft", &cfg.cruft_packs, N_("pack unreferenced objects separately")), OPT_MAGNITUDE(0, "max-cruft-size", &cfg.max_cruft_size, N_("with --cruft, limit the size of new cruft packs")), @@ -673,8 +705,8 @@ int cmd_gc(int argc, const char **argv, const char *prefix) strvec_pushl(&prune_worktrees, "worktree", "prune", "--expire", NULL); strvec_pushl(&rerere, "rerere", "gc", NULL); - /* default expiry time, overwritten in gc_config */ gc_config(&cfg); + if (parse_expiry_date(cfg.gc_log_expire, &gc_log_expire_time)) die(_("failed to parse gc.logExpiry value %s"), cfg.gc_log_expire); @@ -686,6 +718,10 @@ int cmd_gc(int argc, const char **argv, const char *prefix) if (argc > 0) usage_with_options(builtin_gc_usage, builtin_gc_options); + if (prune_expire_arg != prune_expire_sentinel) { + free(cfg.prune_expire); + cfg.prune_expire = xstrdup_or_null(prune_expire_arg); + } if (cfg.prune_expire && parse_expiry_date(cfg.prune_expire, &dummy)) die(_("failed to parse prune expiry value %s"), cfg.prune_expire); @@ -703,8 +739,11 @@ int cmd_gc(int argc, const char **argv, const char *prefix) /* * Auto-gc should be least intrusive as possible. */ - if (!need_to_gc(&cfg)) - return 0; + if (!need_to_gc(&cfg)) { + ret = 0; + goto out; + } + if (!quiet) { if (cfg.detach_auto) fprintf(stderr, _("Auto packing the repository in background for optimum performance.\n")); @@ -713,17 +752,22 @@ int cmd_gc(int argc, const char **argv, const char *prefix) fprintf(stderr, _("See \"git help gc\" for manual housekeeping.\n")); } if (cfg.detach_auto) { - int ret = report_last_gc_error(); - - if (ret == 1) + ret = report_last_gc_error(); + if (ret == 1) { /* Last gc --auto failed. Skip this one. */ - return 0; - else if (ret) + ret = 0; + goto out; + + } else if (ret) { /* an I/O error occurred, already reported */ - return ret; + goto out; + } + + if (lock_repo_for_gc(force, &pid)) { + ret = 0; + goto out; + } - if (lock_repo_for_gc(force, &pid)) - return 0; gc_before_repack(&opts, &cfg); /* dies on failure */ delete_tempfile(&pidfile); @@ -749,8 +793,11 @@ int cmd_gc(int argc, const char **argv, const char *prefix) name = lock_repo_for_gc(force, &pid); if (name) { - if (opts.auto_flag) - return 0; /* be quiet on --auto */ + if (opts.auto_flag) { + ret = 0; + goto out; /* be quiet on --auto */ + } + die(_("gc is already running on machine '%s' pid %"PRIuMAX" (use --force if not)"), name, (uintmax_t)pid); } @@ -826,6 +873,8 @@ int cmd_gc(int argc, const char **argv, const char *prefix) if (!daemonized) unlink(git_path("gc.log")); +out: + gc_config_release(&cfg); return 0; } @@ -1511,6 +1560,8 @@ static int maintenance_run(int argc, const char **argv, const char *prefix) PARSE_OPT_NONEG, task_option_parse), OPT_END() }; + int ret; + memset(&opts, 0, sizeof(opts)); opts.quiet = !isatty(2); @@ -1532,7 +1583,10 @@ static int maintenance_run(int argc, const char **argv, const char *prefix) if (argc != 0) usage_with_options(builtin_maintenance_run_usage, builtin_maintenance_run_options); - return maintenance_run_tasks(&opts, &cfg); + + ret = maintenance_run_tasks(&opts, &cfg); + gc_config_release(&cfg); + return ret; } static char *get_maintpath(void) diff --git a/t/t5304-prune.sh b/t/t5304-prune.sh index 1f1f664871..e641df0116 100755 --- a/t/t5304-prune.sh +++ b/t/t5304-prune.sh @@ -7,6 +7,7 @@ test_description='prune' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh day=$((60*60*24))
We're leaking config values in git-gc(1) when those values are tracked as strings. Introduce a new `gc_config_release()` function that releases this memory to plug those leaks and release old values before populating the config fields via `git_config_string()` et al. Note that there is one small gotcha here with the "--prune" option. Next to passing a string, this option also accepts the "--no-prune" option that overrides the default or configured value. We thus need to discern between the option not having been passed by the user and the negative variant of it. This is done by using a simple sentinel value that lets us discern these cases. Signed-off-by: Patrick Steinhardt <ps@pks.im> --- builtin/gc.c | 108 +++++++++++++++++++++++++++++++++++------------ t/t5304-prune.sh | 1 + 2 files changed, 82 insertions(+), 27 deletions(-)