[v3,4/4] maintenance: optionally use systemd timers on Linux

Message ID	20210520221359.75615-5-lenaic@lhuard.fr (mailing list archive)
State	New, archived
Headers	show Return-Path: <git-owner@kernel.org> From: =?utf-8?b?TMOpbmHDr2MgSHVhcmQ=?= <lenaic@lhuard.fr> To: git@vger.kernel.org Cc: Junio C Hamano <gitster@pobox.com>, Derrick Stolee <dstolee@microsoft.com>, Eric Sunshine <sunshine@sunshineco.com>, =?utf-8?b?xJBvw6BuIFRy4bqnbiBDw7Ru?= =?utf-8?b?ZyBEYW5o?= <congdanhqx@gmail.com>, Felipe Contreras <felipe.contreras@gmail.com>, Phillip Wood <phillip.wood123@gmail.com>, =?utf-8?q?Martin_=C3=85gren?= <martin.agren@gmail.com>, =?utf-8?b?w4Z2YXIg?= =?utf-8?b?QXJuZmrDtnLDsCBCamFybWFzb24=?= <avarab@gmail.com>, Bagas Sanjaya <bagasdotme@gmail.com>, "brian m . carlson" <sandals@crustytoothpaste.net>, =?utf-8?b?TMOpbmHDr2Mg?= =?utf-8?b?SHVhcmQ=?= <lenaic@lhuard.fr> Subject: [PATCH v3 4/4] maintenance: optionally use systemd timers on Linux Date: Fri, 21 May 2021 00:13:59 +0200 Message-Id: <20210520221359.75615-5-lenaic@lhuard.fr> In-Reply-To: <20210520221359.75615-1-lenaic@lhuard.fr> References: <20210509213217.449489-1-lenaic@lhuard.fr> <20210520221359.75615-1-lenaic@lhuard.fr> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Precedence: bulk
Series	maintenance: use systemd timers on Linux \| expand [v2,0/1] maintenance: use systemd timers on Linux [v2,1/1] maintenance: use systemd timers on Linux [v3,2/4] maintenance: introduce ENABLE/DISABLE for code clarity [v3,4/4] maintenance: optionally use systemd timers on Linux

diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt index 7c4bb38a2f..50179e010f 100644 --- a/Documentation/git-maintenance.txt +++ b/Documentation/git-maintenance.txt @@ -181,16 +181,19 @@ OPTIONS `maintenance.<task>.enabled` configured as `true` are considered. See the 'TASKS' section for the list of accepted `<task>` values. ---scheduler=auto|crontab|launchctl|schtasks:: +--scheduler=auto|crontab|systemd-timer|launchctl|schtasks:: When combined with the `start` subcommand, specify the scheduler to use to run the hourly, daily and weekly executions of `git maintenance run`. The possible values for `<scheduler>` depend on the system: `crontab` - is available on POSIX systems, `launchctl` is available on - MacOS and `schtasks` is available on Windows. + is available on POSIX systems, `systemd-timer` is available on Linux + systems, `launchctl` is available on MacOS and `schtasks` is available + on Windows. By default or when `auto` is specified, a suitable scheduler for the system is used. On MacOS, `launchctl` is used. On Windows, - `schtasks` is used. On all other systems, `crontab` is used. + `schtasks` is used. On Linux, `systemd-timer` is used if user systemd + timers are available, otherwise, `crontab` is used. On all other systems, + `crontab` is used. TROUBLESHOOTING @@ -290,6 +293,52 @@ schedule to ensure you are executing the correct binaries in your schedule. +BACKGROUND MAINTENANCE ON LINUX SYSTEMD SYSTEMS +----------------------------------------------- + +While Linux supports `cron`, depending on the distribution, `cron` may +be an optional package not necessarily installed. On modern Linux +distributions, systemd timers are superseding it. + +If user systemd timers are available, they will be used as a replacement +of `cron`. + +In this case, `git maintenance start` will create user systemd timer units +and start the timers. The current list of user-scheduled tasks can be found +by running `systemctl --user list-timers`. The timers written by `git +maintenance start` are similar to this: + +----------------------------------------------------------------------- +$ systemctl --user list-timers +NEXT LEFT LAST PASSED UNIT ACTIVATES +Thu 2021-04-29 19:00:00 CEST 42min left Thu 2021-04-29 18:00:11 CEST 17min ago git-maintenance@hourly.timer git-maintenance@hourly.service +Fri 2021-04-30 00:00:00 CEST 5h 42min left Thu 2021-04-29 00:00:11 CEST 18h ago git-maintenance@daily.timer git-maintenance@daily.service +Mon 2021-05-03 00:00:00 CEST 3 days left Mon 2021-04-26 00:00:11 CEST 3 days ago git-maintenance@weekly.timer git-maintenance@weekly.service +----------------------------------------------------------------------- + +One timer is registered for each `--schedule=<frequency>` option. + +The definition of the systemd units can be inspected in the following files: + +----------------------------------------------------------------------- +~/.config/systemd/user/git-maintenance@.timer +~/.config/systemd/user/git-maintenance@.service +~/.config/systemd/user/timers.target.wants/git-maintenance@hourly.timer +~/.config/systemd/user/timers.target.wants/git-maintenance@daily.timer +~/.config/systemd/user/timers.target.wants/git-maintenance@weekly.timer +----------------------------------------------------------------------- + +`git maintenance start` will overwrite these files and start the timer +again with `systemctl --user`, so any customization should be done by +creating a drop-in file, i.e. a `.conf` suffixed file in the +`~/.config/systemd/user/git-maintenance@.service.d` directory. + +`git maintenance stop` will stop the user systemd timers and delete +the above mentioned files. + +For more details, see `systemd.timer(5)`. + + BACKGROUND MAINTENANCE ON MACOS SYSTEMS --------------------------------------- diff --git a/builtin/gc.c b/builtin/gc.c index bf21cec059..d2432ee04f 100644 --- a/builtin/gc.c +++ b/builtin/gc.c @@ -2079,10 +2079,174 @@ static int crontab_update_schedule(enum enable_or_disable run_maintenance, return result; } +static int is_systemd_timer_available(void) +{ + const char *cmd = "systemctl"; + int is_available; + static int cached_result = -1; +#ifdef __linux__ + struct child_process child = CHILD_PROCESS_INIT; +#endif + + if (cached_result != -1) + return cached_result; + + if (get_schedule_cmd(&cmd, &is_available)) + return is_available; + +#ifdef __linux__ + strvec_split(&child.args, cmd); + strvec_pushl(&child.args, "--user", "list-timers", NULL); + child.no_stdin = 1; + child.no_stdout = 1; + child.no_stderr = 1; + child.silent_exec_failure = 1; + + if (start_command(&child)) { + cached_result = 0; + return cached_result; + } + if (finish_command(&child)) { + cached_result = 0; + return cached_result; + } + cached_result = 1; + return cached_result; +#else + return 0; +#endif +} + +static char *xdg_config_home_systemd(const char *filename) +{ + assert(filename); + return xdg_config_home("systemd/user/%s", filename); +} + +static int systemd_timer_enable_unit(int enable, + enum schedule_priority schedule) +{ + const char *cmd = "systemctl"; + struct child_process child = CHILD_PROCESS_INIT; + const char *frequency = get_frequency(schedule); + + get_schedule_cmd(&cmd, NULL); + strvec_split(&child.args, cmd); + strvec_pushl(&child.args, "--user", enable ? "enable" : "disable", + "--now", NULL); + strvec_pushf(&child.args, "git-maintenance@%s.timer", frequency); + + if (start_command(&child)) + die(_("failed to run systemctl")); + return finish_command(&child); +} + +static int systemd_timer_delete_unit_templates(void) +{ + char *filename = xdg_config_home_systemd("git-maintenance@.timer"); + unlink(filename); + free(filename); + + filename = xdg_config_home_systemd("git-maintenance@.service"); + unlink(filename); + free(filename); + + return 0; +} + +static int systemd_timer_delete_units(void) +{ + return systemd_timer_enable_unit(DISABLE, SCHEDULE_HOURLY) || + systemd_timer_enable_unit(DISABLE, SCHEDULE_DAILY) || + systemd_timer_enable_unit(DISABLE, SCHEDULE_WEEKLY) || + systemd_timer_delete_unit_templates(); +} + +static int systemd_timer_write_unit_templates(const char *exec_path) +{ + char *filename; + FILE *file; + const char *unit; + + filename = xdg_config_home_systemd("git-maintenance@.timer"); + if (safe_create_leading_directories(filename)) + die(_("failed to create directories for '%s'"), filename); + file = xfopen(filename, "w"); + FREE_AND_NULL(filename); + + unit = "# This file was created and is maintained by Git.\n" + "# Any edits made in this file might be replaced in the future\n" + "# by a Git command.\n" + "\n" + "[Unit]\n" + "Description=Optimize Git repositories data\n" + "\n" + "[Timer]\n" + "OnCalendar=%i\n" + "Persistent=true\n" + "\n" + "[Install]\n" + "WantedBy=timers.target\n"; + fputs(unit, file); + fclose(file); + + filename = xdg_config_home_systemd("git-maintenance@.service"); + file = xfopen(filename, "w"); + free(filename); + + unit = "# This file was created and is maintained by Git.\n" + "# Any edits made in this file might be replaced in the future\n" + "# by a Git command.\n" + "\n" + "[Unit]\n" + "Description=Optimize Git repositories data\n" + "\n" + "[Service]\n" + "Type=oneshot\n" + "ExecStart=\"%s/git\" --exec-path=\"%s\" for-each-repo --config=maintenance.repo maintenance run --schedule=%%i\n" + "LockPersonality=yes\n" + "MemoryDenyWriteExecute=yes\n" + "NoNewPrivileges=yes\n" + "RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6\n" + "RestrictNamespaces=yes\n" + "RestrictRealtime=yes\n" + "RestrictSUIDSGID=yes\n" + "SystemCallArchitectures=native\n" + "SystemCallFilter=@system-service\n"; + fprintf(file, unit, exec_path, exec_path); + fclose(file); + + return 0; +} + +static int systemd_timer_setup_units(void) +{ + const char *exec_path = git_exec_path(); + + return systemd_timer_write_unit_templates(exec_path) || + systemd_timer_enable_unit(ENABLE, SCHEDULE_HOURLY) || + systemd_timer_enable_unit(ENABLE, SCHEDULE_DAILY) || + systemd_timer_enable_unit(ENABLE, SCHEDULE_WEEKLY); +} + +static int systemd_timer_update_schedule(enum enable_or_disable run_maintenance, + int fd) +{ + switch (run_maintenance) { + case ENABLE: + return systemd_timer_setup_units(); + case DISABLE: + return systemd_timer_delete_units(); + default: + BUG("invalid enable_or_disable value"); + } +} + enum scheduler { SCHEDULER_INVALID = -1, SCHEDULER_AUTO, SCHEDULER_CRON, + SCHEDULER_SYSTEMD, SCHEDULER_LAUNCHCTL, SCHEDULER_SCHTASKS, }; @@ -2097,6 +2261,11 @@ static const struct { .is_available = is_crontab_available, .update_schedule = crontab_update_schedule, }, + [SCHEDULER_SYSTEMD] = { + .name = "systemctl", + .is_available = is_systemd_timer_available, + .update_schedule = systemd_timer_update_schedule, + }, [SCHEDULER_LAUNCHCTL] = { .name = "launchctl", .is_available = is_launchctl_available, @@ -2117,6 +2286,9 @@ static enum scheduler parse_scheduler(const char *value) return SCHEDULER_AUTO; else if (!strcasecmp(value, "cron") || !strcasecmp(value, "crontab")) return SCHEDULER_CRON; + else if (!strcasecmp(value, "systemd") || + !strcasecmp(value, "systemd-timer")) + return SCHEDULER_SYSTEMD; else if (!strcasecmp(value, "launchctl")) return SCHEDULER_LAUNCHCTL; else if (!strcasecmp(value, "schtasks")) @@ -2155,6 +2327,15 @@ static void resolve_auto_scheduler(enum scheduler *scheduler) *scheduler = SCHEDULER_SCHTASKS; return; +#elif defined(__linux__) + if (is_systemd_timer_available()) + *scheduler = SCHEDULER_SYSTEMD; + else if (is_crontab_available()) + *scheduler = SCHEDULER_CRON; + else + die(_("neither systemd timers nor crontab are available")); + return; + #else *scheduler = SCHEDULER_CRON; return; diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh index 9eac260307..c8a6f19ebc 100755 --- a/t/t7900-maintenance.sh +++ b/t/t7900-maintenance.sh @@ -20,6 +20,18 @@ test_xmllint () { fi } +test_lazy_prereq SYSTEMD_ANALYZE ' + systemd-analyze --help >out && + grep verify out +' + +test_systemd_analyze_verify () { + if test_have_prereq SYSTEMD_ANALYZE + then + systemd-analyze verify "$@" + fi +} + test_expect_success 'help text' ' test_expect_code 129 git maintenance -h 2>err && test_i18ngrep "usage: git maintenance <subcommand>" err && @@ -628,14 +640,54 @@ test_expect_success 'start and stop Windows maintenance' ' test_cmp expect args ' +test_expect_success 'start and stop Linux/systemd maintenance' ' + write_script print-args <<-\EOF && + printf "%s\n" "$*" >>args + EOF + + XDG_CONFIG_HOME="$PWD" && + export XDG_CONFIG_HOME && + rm -f args && + GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args" git maintenance start --scheduler=systemd-timer && + + # start registers the repo + git config --get --global --fixed-value maintenance.repo "$(pwd)" && + + test_systemd_analyze_verify "systemd/user/git-maintenance@.service" && + + printf -- "--user enable --now git-maintenance@%s.timer\n" hourly daily weekly >expect && + test_cmp expect args && + + rm -f args && + GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args" git maintenance stop && + + # stop does not unregister the repo + git config --get --global --fixed-value maintenance.repo "$(pwd)" && + + test_path_is_missing "systemd/user/git-maintenance@.timer" && + test_path_is_missing "systemd/user/git-maintenance@.service" && + + printf -- "--user disable --now git-maintenance@%s.timer\n" hourly daily weekly >expect && + test_cmp expect args +' + test_expect_success 'start and stop when several schedulers are available' ' write_script print-args <<-\EOF && printf "%s\n" "$*" | sed "s:gui/[0-9][0-9]*:gui/[UID]:; s:$schtasks /create .* /xml$.*:\1:;" >>args EOF rm -f args && - GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=launchctl && - rm -f expect && + GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args systemctl,launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=systemd-timer && + printf -- "systemctl --user enable --now git-maintenance@%s.timer\n" hourly daily weekly >expect && + printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \ + hourly daily weekly >>expect && + printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \ + hourly daily weekly >>expect && + test_cmp expect args && + + rm -f args && + GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args systemctl,launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=launchctl && + printf -- "systemctl --user disable --now git-maintenance@%s.timer\n" hourly daily weekly >expect && for frequency in hourly daily weekly do PLIST="$pfx/Library/LaunchAgents/org.git-scm.git.$frequency.plist" && @@ -647,17 +699,19 @@ test_expect_success 'start and stop when several schedulers are available' ' test_cmp expect args && rm -f args && - GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=schtasks && + GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args systemctl,launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance start --scheduler=schtasks && + printf -- "systemctl --user disable --now git-maintenance@%s.timer\n" hourly daily weekly >expect && printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \ - hourly daily weekly >expect && + hourly daily weekly >>expect && printf "schtasks /create /tn Git Maintenance (%s) /f /xml\n" \ hourly daily weekly >>expect && test_cmp expect args && rm -f args && - GIT_TEST_MAINT_SCHEDULER="launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance stop && + GIT_TEST_MAINT_SCHEDULER="systemctl:./print-args systemctl,launchctl:./print-args launchctl,schtasks:./print-args schtasks" git maintenance stop && + printf -- "systemctl --user disable --now git-maintenance@%s.timer\n" hourly daily weekly >expect && printf "launchctl bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \ - hourly daily weekly >expect && + hourly daily weekly >>expect && printf "schtasks /delete /tn Git Maintenance (%s) /f\n" \ hourly daily weekly >>expect && test_cmp expect args

[v3,4/4] maintenance: optionally use systemd timers on Linux

Commit Message

Comments

Patch