From patchwork Fri Nov 13 14:00:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 11903553 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8DF32C388F7 for ; Fri, 13 Nov 2020 14:00:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 499CE2222F for ; Fri, 13 Nov 2020 14:00:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="cB6UbvQv" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726778AbgKMOA3 (ORCPT ); Fri, 13 Nov 2020 09:00:29 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54184 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726498AbgKMOA2 (ORCPT ); Fri, 13 Nov 2020 09:00:28 -0500 Received: from mail-wm1-x341.google.com (mail-wm1-x341.google.com [IPv6:2a00:1450:4864:20::341]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 152C6C0617A6 for ; Fri, 13 Nov 2020 06:00:28 -0800 (PST) Received: by mail-wm1-x341.google.com with SMTP id a3so8574668wmb.5 for ; Fri, 13 Nov 2020 06:00:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=Nly5aoUwoSfFzapqnHpVG8ijbTA7fs42NqNdOyOuJhA=; b=cB6UbvQv6yy64uWWrwYsnDIFoBbU2JcY7BPIG10EnGOwahXt732xoYUcBucaaeRufG 9z8KMDkRrSgzhyjpGO+zbDpwxGp60bKhILdScWI7x036C6RUyCVlBDQ3p7TvY1kYoYOR /HP1b0EhLp63hNJ+IChnKnator65VSV6ourjPbqO8O2+JgJSdkAlrGf4CJMT49gEJNgx N1P/tYdX/UUTxit2yFWf3ZtCHNaTFpA7aM/WazNhpPtR9bkj9hBDB5OC6TR6TXJpKe1D bXOCJEga79M8hCxYQ5enVMFTajpM4pWZ3qS8Js+7LPo54pKojqUE8aFGXhgK1jpEsz8m qcmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=Nly5aoUwoSfFzapqnHpVG8ijbTA7fs42NqNdOyOuJhA=; b=qw7WJfjrpDTzQa5BgJyKUcSw4ZC2jZz+ywHhMt8L1VHC2oxHQAtrcQ24xXpAVucmmL IuspRjDHJhEbMwl61LjQEIBOeISEh8xJ6jVisIGlr8od3IGzzNh9drD+sms50hrb+RYt MQlPMMYBsA3rgXDRtlcuVNlVS2ZaGnGK9XM2eWwSx3BkKJ56fHSw68pCGuyxlIZBk9Zb D0mZZ+G/W9TA/WY+mdKyly6Qur6bzTGQreZVu8Y7nWvsjkPVW2sEaXpDJa8U0IAsQq3Q aK8ph/a+2t5HEgWqhrPT85x5N7ku2NZ1WDnUzR+hMAIdo1UVcgc7t2SemwDZXDcIE8j+ Ag9A== X-Gm-Message-State: AOAM530n+0hI8xtiKcrpQAyoKJhGXBpBPcavArSMXNZ56BHPJDDKZVQE 0f7q8t+raTzzUwXGixu0+ZBVfiPUm+M= X-Google-Smtp-Source: ABdhPJxu8pD63PVLeuVGochyhLhkJkqVR7p3INo4SN93pUpOMKcSJ5ilnIbeJAqbiVP2F4Vb4+YPtw== X-Received: by 2002:a1c:9c53:: with SMTP id f80mr2623981wme.19.1605276026620; Fri, 13 Nov 2020 06:00:26 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id 71sm11609927wrm.20.2020.11.13.06.00.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Nov 2020 06:00:26 -0800 (PST) Message-Id: In-Reply-To: References: Date: Fri, 13 Nov 2020 14:00:21 +0000 Subject: [PATCH v3 1/4] maintenance: extract platform-specific scheduling Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Eric Sunshine , Derrick Stolee , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee The existing schedule mechanism using 'cron' is supported by POSIX platforms, but not Windows. It also works slightly differently on macOS to significant detriment of the user experience. To allow for new implementations on these platforms, extract a method that performs the platform-specific scheduling mechanism. This will be swapped at compile time with new implementations on specialized platforms. Signed-off-by: Derrick Stolee --- builtin/gc.c | 38 +++++++++++++++++++++----------------- 1 file changed, 21 insertions(+), 17 deletions(-) diff --git a/builtin/gc.c b/builtin/gc.c index e3098ef6a1..c1f7d9bdc2 100644 --- a/builtin/gc.c +++ b/builtin/gc.c @@ -1494,7 +1494,7 @@ static int maintenance_unregister(void) #define BEGIN_LINE "# BEGIN GIT MAINTENANCE SCHEDULE" #define END_LINE "# END GIT MAINTENANCE SCHEDULE" -static int update_background_schedule(int run_maintenance) +static int platform_update_schedule(int run_maintenance, int fd) { int result = 0; int in_old_region = 0; @@ -1503,11 +1503,6 @@ static int update_background_schedule(int run_maintenance) FILE *cron_list, *cron_in; const char *crontab_name; struct strbuf line = STRBUF_INIT; - struct lock_file lk; - char *lock_path = xstrfmt("%s/schedule", the_repository->objects->odb->path); - - if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0) - return error(_("another process is scheduling background maintenance")); crontab_name = getenv("GIT_TEST_CRONTAB"); if (!crontab_name) @@ -1516,12 +1511,11 @@ static int update_background_schedule(int run_maintenance) strvec_split(&crontab_list.args, crontab_name); strvec_push(&crontab_list.args, "-l"); crontab_list.in = -1; - crontab_list.out = dup(lk.tempfile->fd); + crontab_list.out = dup(fd); crontab_list.git_cmd = 0; if (start_command(&crontab_list)) { - result = error(_("failed to run 'crontab -l'; your system might not support 'cron'")); - goto cleanup; + return error(_("failed to run 'crontab -l'; your system might not support 'cron'")); } /* Ignore exit code, as an empty crontab will return error. */ @@ -1531,7 +1525,7 @@ static int update_background_schedule(int run_maintenance) * Read from the .lock file, filtering out the old * schedule while appending the new schedule. */ - cron_list = fdopen(lk.tempfile->fd, "r"); + cron_list = fdopen(fd, "r"); rewind(cron_list); strvec_split(&crontab_edit.args, crontab_name); @@ -1539,8 +1533,7 @@ static int update_background_schedule(int run_maintenance) crontab_edit.git_cmd = 0; if (start_command(&crontab_edit)) { - result = error(_("failed to run 'crontab'; your system might not support 'cron'")); - goto cleanup; + return error(_("failed to run 'crontab'; your system might not support 'cron'")); } cron_in = fdopen(crontab_edit.in, "w"); @@ -1586,13 +1579,24 @@ static int update_background_schedule(int run_maintenance) close(crontab_edit.in); done_editing: - if (finish_command(&crontab_edit)) { + if (finish_command(&crontab_edit)) result = error(_("'crontab' died")); - goto cleanup; - } - fclose(cron_list); + else + fclose(cron_list); + return result; +} + +static int update_background_schedule(int run_maintenance) +{ + int result; + struct lock_file lk; + char *lock_path = xstrfmt("%s/schedule", the_repository->objects->odb->path); + + if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0) + return error(_("another process is scheduling background maintenance")); + + result = platform_update_schedule(run_maintenance, lk.tempfile->fd); -cleanup: rollback_lock_file(&lk); return result; } From patchwork Fri Nov 13 14:00:22 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 11903557 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 02C67C388F7 for ; Fri, 13 Nov 2020 14:00:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A16402222F for ; Fri, 13 Nov 2020 14:00:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ZSwBBq08" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726885AbgKMOAe (ORCPT ); Fri, 13 Nov 2020 09:00:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54194 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726418AbgKMOAa (ORCPT ); Fri, 13 Nov 2020 09:00:30 -0500 Received: from mail-wm1-x343.google.com (mail-wm1-x343.google.com [IPv6:2a00:1450:4864:20::343]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 53D06C0617A6 for ; Fri, 13 Nov 2020 06:00:30 -0800 (PST) Received: by mail-wm1-x343.google.com with SMTP id c9so8227999wml.5 for ; Fri, 13 Nov 2020 06:00:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=c3ckQj1fIW3U8Ln1uReV5M8x5QIwk6yY6Ii5rpTvUG4=; b=ZSwBBq08ZNRlLeb8PNHBWrfwn4rR4puzgjnl8g+UI0Velou0rZIIFbcu9aIHdbJqvn xGfUobRMjOPl7NGe5p6nKsnrzStf6JbX1DB1KSctX4TFBbMJ56utIEwr/rPn2NvbbeJX 74JG4LOy+0gxW+xoSAPdHUndr6EhIERC2BlKybz7CVjmoZVFKxc79UcdbPrirV83Hrku aKWAvGCRTMaCvyxby7fXoVnElbjjCvaXkN7ZhG61eCuCNLyeHpqv8T6Fx7QtwEYm1kPU hi2FzU3FBVz15bg8vpY4DoyArlssYhcnGsZ/72UFvA7D/Zn9QppiDVynPaKQ/j3qSBP/ iONQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=c3ckQj1fIW3U8Ln1uReV5M8x5QIwk6yY6Ii5rpTvUG4=; b=TCYct1glda7Nb5X733ML2YQiIJdN0VVbn0sNexQBFWuXLw36mFOY6S7Sib5sA67Lao DR4i6TfPqdvVGjvh9BSgzmnXzBzo9eS7tuRACEdGBtscJJF3JlAXtZ9muV93u5mAHMTD 3VVmSaBgfoA1BTt+y1bhZzJ+x70fXOSRJa4wjK0ej2X0Go4f/obbXNkXXTkht0RsPA52 a4Rurqq9BjkcCc0+Us/ajAzIIDPNQz26/rrEQeumNk3/36BPlRSbeFBhbjD01ApsoEmi RpOBpDys1H1QhONDH9pybOQSggYlqXzmCoP7zTIUsgo7Leqijc9BIv6ZFV5OrcE2ui6v hVrg== X-Gm-Message-State: AOAM533eu1GaD7Z0jFdQtqw527WdeG1SlRhWgk1XOZJ41xHn88tWNAPI 32UyOjrEkf8NTPongp5ABVpEANJnfGg= X-Google-Smtp-Source: ABdhPJwoYR8oIh/Lkobp9FbCU/nm/nuCSOPZzjpPkliu2UmnrqzQby+utt0DypleA1H1fsIHIuewYg== X-Received: by 2002:a1c:658b:: with SMTP id z133mr2604153wmb.1.1605276028829; Fri, 13 Nov 2020 06:00:28 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id s188sm10450136wmf.45.2020.11.13.06.00.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Nov 2020 06:00:27 -0800 (PST) Message-Id: <0dfe53092ea507e74260f2638c46c618fa554c94.1605276024.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Fri, 13 Nov 2020 14:00:22 +0000 Subject: [PATCH v3 2/4] maintenance: include 'cron' details in docs Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Eric Sunshine , Derrick Stolee , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Advanced and expert users may want to know how 'git maintenance start' schedules background maintenance in order to customize their own schedules beyond what the maintenance.* config values allow. Start a new set of sections in git-maintenance.txt that describe how 'cron' is used to run these tasks. This is particularly valuable for users who want to inspect what Git is doing or for users who want to customize the schedule further. Having a baseline can provide a way forward for users who have never worked with cron schedules. Helped-by: Eric Sunshine Signed-off-by: Derrick Stolee --- Documentation/git-maintenance.txt | 54 +++++++++++++++++++++++++++++++ 1 file changed, 54 insertions(+) diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt index 6fec1eb8dc..1aa1112418 100644 --- a/Documentation/git-maintenance.txt +++ b/Documentation/git-maintenance.txt @@ -218,6 +218,60 @@ Further, the `git gc` command should not be combined with but does not take the lock in the same way as `git maintenance run`. If possible, use `git maintenance run --task=gc` instead of `git gc`. +The following sections describe the mechanisms put in place to run +background maintenance by `git maintenance start` and how to customize +them. + +BACKGROUND MAINTENANCE ON POSIX SYSTEMS +--------------------------------------- + +The standard mechanism for scheduling background tasks on POSIX systems +is cron(8). This tool executes commands based on a given schedule. The +current list of user-scheduled tasks can be found by running `crontab -l`. +The schedule written by `git maintenance start` is similar to this: + +----------------------------------------------------------------------- +# BEGIN GIT MAINTENANCE SCHEDULE +# The following schedule was created by Git +# Any edits made in this region might be +# replaced in the future by a Git command. + +0 1-23 * * * "//git" --exec-path="/" for-each-repo --config=maintenance.repo maintenance run --schedule=hourly +0 0 * * 1-6 "//git" --exec-path="/" for-each-repo --config=maintenance.repo maintenance run --schedule=daily +0 0 * * 0 "//git" --exec-path="/" for-each-repo --config=maintenance.repo maintenance run --schedule=weekly + +# END GIT MAINTENANCE SCHEDULE +----------------------------------------------------------------------- + +The comments are used as a region to mark the schedule as written by Git. +Any modifications within this region will be completely deleted by +`git maintenance stop` or overwritten by `git maintenance start`. + +The `crontab` entry specifies the full path of the `git` executable to +ensure that the executed `git` command is the same one with which +`git maintenance start` was issued independent of `PATH`. If the same user +runs `git maintenance start` with multiple Git executables, then only the +latest executable is used. + +These commands use `git for-each-repo --config=maintenance.repo` to run +`git maintenance run --schedule=` on each repository listed in +the multi-valued `maintenance.repo` config option. These are typically +loaded from the user-specific global config. The `git maintenance` process +then determines which maintenance tasks are configured to run on each +repository with each `` using the `maintenance..schedule` +config options. These values are loaded from the global or repository +config values. + +If the config values are insufficient to achieve your desired background +maintenance schedule, then you can create your own schedule. If you run +`crontab -e`, then an editor will load with your user-specific `cron` +schedule. In that editor, you can add your own schedule lines. You could +start by adapting the default schedule listed earlier, or you could read +the crontab(5) documentation for advanced scheduling techniques. Please +do use the full path and `--exec-path` techniques from the default +schedule to ensure you are executing the correct binaries in your +schedule. + GIT --- From patchwork Wed Nov 4 20:06:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 11882037 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DA0ADC4742C for ; Wed, 4 Nov 2020 20:06:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5B69D2076D for ; Wed, 4 Nov 2020 20:06:17 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="iSj/s+sw" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731443AbgKDUGQ (ORCPT ); Wed, 4 Nov 2020 15:06:16 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42194 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731404AbgKDUGO (ORCPT ); Wed, 4 Nov 2020 15:06:14 -0500 Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com [IPv6:2a00:1450:4864:20::435]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1CF01C0613D3 for ; Wed, 4 Nov 2020 12:06:14 -0800 (PST) Received: by mail-wr1-x435.google.com with SMTP id n18so23388354wrs.5 for ; Wed, 04 Nov 2020 12:06:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=T+212svS589+wgo/bh0IJ/IgpZ5v4kEWB5VQsSADDPU=; b=iSj/s+swUQurGcAGYI7D8WEgAs0kGaZs4F7XVptIxoSCOOjTM7Iu3DPI+jlpfCBy6t vZM+5Db+h3XenRzPolPHH2njton4tLFpIWdpgilAeMhQhjamKNrYckjqvnOmDKHht2M7 XWm36HiC/fSyD5x5KhDfn3g4ETWutpoksy26iM52dzSu6jG57EkYfxlLUuQwJ5kRiwyi OunQfsDEADLTlz5KyOBMCsbKLYTisqg4vYLSn/6oMiXUVGACQCOUGMwmuYpyQbbDEP1q H2wDGmkpHEsG3fiDNwkCu7QjahpYVIpOIMnK8GwCIv6q6Sg+fX/JkLcxWy3W90GBQIv6 hvJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=T+212svS589+wgo/bh0IJ/IgpZ5v4kEWB5VQsSADDPU=; b=i7pwnN03GJb7magnvj3Ya953B+mG5osmamtjDF1Hwo5xKY83d+fURfARSRjYsw59sG S/jpeE962s/GjVEfoQpMmmkiA2iwiMKq3aLukDYtmI//7Dithg816m1nKkJ1AeDXxLpF DA7Kf/XThJEl4X8Ck1LlxhqTWJkFkxkGrdzs4FtD31h2q+lwyV8unDWUjx56DkOIVAdU Or/Rwd1f3cLarBMr6PrKo9eQEm2pYbzIYAb4UWwk5kO9BSLsTRNRc+AbYu02GUOh+LJ8 CIXbQHLM0qb42Rg+dGaLPOb0Fe+iaiGyDzPwg7wTYUNK/+uTMuRC4QU5HvCAdXvnqdiu 9lDA== X-Gm-Message-State: AOAM533PRNlG8TOcueE4SyI60PTJf3zvkFmKfAzQJXsvTJThbWsri2hi j0xPZlB6ZPlTHg65HrmjdyDaTzVFizQ= X-Google-Smtp-Source: ABdhPJz3Hh5P6BJvdfmrOfKa4xxiohBoXBunmAVXKm9pya8nWmQQ5qDwHhtzGP62Zb7VkAJtL+hE5A== X-Received: by 2002:adf:ef0d:: with SMTP id e13mr33755703wro.24.1604520372299; Wed, 04 Nov 2020 12:06:12 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id o184sm3851900wmo.37.2020.11.04.12.06.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Nov 2020 12:06:11 -0800 (PST) Message-Id: <0fafd75d100f343f7cff6471772ed9b12793f81e.1604520368.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Wed, 04 Nov 2020 20:06:07 +0000 Subject: [PATCH v2 3/4] maintenance: use launchctl on macOS Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: jrnieder@gmail.com, jonathantanmy@google.com, sluongng@gmail.com, Derrick Stolee , =?utf-8?b?xJBvw6BuIFRy4bqnbiBDw7RuZw==?= Danh , Martin =?utf-8?b?w4VncmVu?= , Eric Sunshine , Derrick Stolee , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee The existing mechanism for scheduling background maintenance is done through cron. The 'crontab -e' command allows updating the schedule while cron itself runs those commands. While this is technically supported by macOS, it has some significant deficiencies: 1. Every run of 'crontab -e' must request elevated privileges through the user interface. When running 'git maintenance start' from the Terminal app, it presents a dialog box saying "Terminal.app would like to administer your computer. Administration can include modifying passwords, networking, and system settings." This is more alarming than what we are hoping to achieve. If this alert had some information about how "git" is trying to run "crontab" then we would have some reason to believe that this dialog might be fine. However, it also doesn't help that some scenarios just leave Git waiting for a response without presenting anything to the user. I experienced this when executing the command from a Bash terminal view inside Visual Studio Code. 2. While cron initializes a user environment enough for "git config --global --show-origin" to show the correct config file information, it does not set up the environment enough for Git Credential Manager Core to load credentials during a 'prefetch' task. My prefetches against private repositories required re-authenticating through UI pop-ups in a way that should not be required. The solution is to switch from cron to the Apple-recommended [1] 'launchd' tool. [1] https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/ScheduledJobs.html The basics of this tool is that we need to create XML-formatted "plist" files inside "~/Library/LaunchAgents/" and then use the 'launchctl' tool to make launchd aware of them. The plist files include all of the scheduling information, along with the command-line arguments split across an array of tags. For example, here is my plist file for the weekly scheduled tasks: Labelorg.git-scm.git.weekly ProgramArguments /usr/local/libexec/git-core/git --exec-path=/usr/local/libexec/git-core for-each-repo --config=maintenance.repo maintenance run --schedule=weekly StartCalendarInterval Day0 Hour0 Minute0 The schedules for the daily and hourly tasks are more complicated since we need to use an array for the StartCalendarInterval with an entry for each of the six days other than the 0th day (to avoid colliding with the weekly task), and each of the 23 hours other than the 0th hour (to avoid colliding with the daily task). The "Label" value is currently filled with "org.git-scm.git.X" where X is the frequency. We need a different plist file for each frequency. The launchctl command needs to be aligned with a user id in order to initialize the command environment. This must be done using the 'launchctl bootstrap' subcommand. This subcommand is new as of macOS 10.11, which was released in September 2015. Before that release the 'launchctl load' subcommand was recommended. The best source of information on this transition I have seen is available at [2]. [2] https://babodee.wordpress.com/2016/04/09/launchctl-2-0-syntax/ To remove a schedule, we must run 'launchctl bootout' with a valid plist file. We also need to 'bootout' a task before the 'bootstrap' subcommand will succeed, if such a task already exists. We can verify the commands that were run by 'git maintenance start' and 'git maintenance stop' by injecting a script that writes the command-line arguments into GIT_TEST_CRONTAB. An earlier version of this patch accidentally had an opening "" tag when it should have had a closing "" tag. This was caught during manual testing with actual 'launchctl' commands, but we do not want to update developers' tasks when running tests. It appears that macOS includes the "xmllint" tool which can verify the XML format, so call it from the macOS-specific tests to ensure the .plist files are well-formatted. Helped-by: Eric Sunshine Signed-off-by: Derrick Stolee --- Documentation/git-maintenance.txt | 43 ++++++ builtin/gc.c | 209 ++++++++++++++++++++++++++++++ t/t7900-maintenance.sh | 53 +++++++- t/test-lib.sh | 4 + 4 files changed, 306 insertions(+), 3 deletions(-) diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt index 4c7aac877d..451ebac131 100644 --- a/Documentation/git-maintenance.txt +++ b/Documentation/git-maintenance.txt @@ -273,6 +273,49 @@ for advanced scheduling techniques. Please do use the full path and executing the correct binaries in your schedule. +BACKGROUND MAINTENANCE ON MACOS SYSTEMS +--------------------------------------- + +While macOS technically supports `cron`, using `crontab -e` requires +elevated privileges and the executed process do not have a full user +context. Without a full user context, Git and its credential helpers +cannot access stored credentials, so some maintenance tasks are not +functional. + +Instead, `git maintenance start` interacts with the `launchctl` tool, +which is the recommended way to +https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/ScheduledJobs.html[schedule timed jobs in macOS]. + +Scheduling maintenance through `git maintenance (start|stop)` requires +some `launchctl` features available only in macOS 10.11 or later. + +Your user-specific scheduled tasks are stored as XML-formatted `.plist` +files in `~/Library/LaunchAgents/`. You can see the currently-registered +tasks using the following command: + +----------------------------------------------------------------------- +$ ls ~/Library/LaunchAgents/ | grep org.git-scm.git +org.git-scm.git.daily.plist +org.git-scm.git.hourly.plist +org.git-scm.git.weekly.plist +----------------------------------------------------------------------- + +One task is registered for each `--schedule=` option. To +inspect how the XML format describes each schedule, open one of these +`.plist` files in an editor and inspect the `` element following +the `StartCalendarInterval` element. + +`git maintenance start` will overwrite these files and register the +tasks again with `launchctl`, so any customizations should be done by +creating your own `.plist` files with distinct names. Similarly, the +`git maintenance stop` command will unregister the tasks with `launchctl` +and delete the `.plist` files. + +To create more advanced customizations to your background tasks, see +https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/CreatingLaunchdJobs.html#//apple_ref/doc/uid/TP40001762-104142[the `launchctl` documentation] +for more information. + + GIT --- Part of the linkgit:git[1] suite diff --git a/builtin/gc.c b/builtin/gc.c index c1f7d9bdc2..7604064a8d 100644 --- a/builtin/gc.c +++ b/builtin/gc.c @@ -1491,6 +1491,214 @@ static int maintenance_unregister(void) return run_command(&config_unset); } +#if defined(__APPLE__) + +static char *get_service_name(const char *frequency) +{ + struct strbuf label = STRBUF_INIT; + strbuf_addf(&label, "org.git-scm.git.%s", frequency); + return strbuf_detach(&label, NULL); +} + +static char *get_service_filename(const char *name) +{ + char *expanded; + struct strbuf filename = STRBUF_INIT; + strbuf_addf(&filename, "~/Library/LaunchAgents/%s.plist", name); + + expanded = expand_user_path(filename.buf, 1); + if (!expanded) + die(_("failed to expand path '%s'"), filename.buf); + + strbuf_release(&filename); + return expanded; +} + +static const char *get_frequency(enum schedule_priority schedule) +{ + switch (schedule) { + case SCHEDULE_HOURLY: + return "hourly"; + case SCHEDULE_DAILY: + return "daily"; + case SCHEDULE_WEEKLY: + return "weekly"; + default: + BUG("invalid schedule %d", schedule); + } +} + +static char *get_uid(void) +{ + struct strbuf output = STRBUF_INIT; + struct child_process id = CHILD_PROCESS_INIT; + + strvec_pushl(&id.args, "/usr/bin/id", "-u", NULL); + if (capture_command(&id, &output, 0)) + die(_("failed to discover user id")); + + strbuf_trim_trailing_newline(&output); + return strbuf_detach(&output, NULL); +} + +static int bootout(const char *filename) +{ + int result; + struct strvec args = STRVEC_INIT; + char *uid = get_uid(); + const char *launchctl = getenv("GIT_TEST_CRONTAB"); + if (!launchctl) + launchctl = "/bin/launchctl"; + + strvec_split(&args, launchctl); + strvec_push(&args, "bootout"); + strvec_pushf(&args, "gui/%s", uid); + strvec_push(&args, filename); + + result = run_command_v_opt(args.v, 0); + + strvec_clear(&args); + free(uid); + return result; +} + +static int bootstrap(const char *filename) +{ + int result; + struct strvec args = STRVEC_INIT; + char *uid = get_uid(); + const char *launchctl = getenv("GIT_TEST_CRONTAB"); + if (!launchctl) + launchctl = "/bin/launchctl"; + + strvec_split(&args, launchctl); + strvec_push(&args, "bootstrap"); + strvec_pushf(&args, "gui/%s", uid); + strvec_push(&args, filename); + + result = run_command_v_opt(args.v, 0); + + strvec_clear(&args); + free(uid); + return result; +} + +static int remove_plist(enum schedule_priority schedule) +{ + const char *frequency = get_frequency(schedule); + char *name = get_service_name(frequency); + char *filename = get_service_filename(name); + int result = bootout(filename); + free(filename); + free(name); + return result; +} + +static int remove_plists(void) +{ + return remove_plist(SCHEDULE_HOURLY) || + remove_plist(SCHEDULE_DAILY) || + remove_plist(SCHEDULE_WEEKLY); +} + +static int schedule_plist(const char *exec_path, enum schedule_priority schedule) +{ + FILE *plist; + int i; + const char *preamble, *repeat; + const char *frequency = get_frequency(schedule); + char *name = get_service_name(frequency); + char *filename = get_service_filename(name); + + if (safe_create_leading_directories(filename)) + die(_("failed to create directories for '%s'"), filename); + plist = fopen(filename, "w"); + + if (!plist) + die(_("failed to open '%s'"), filename); + + preamble = "\n" + "\n" + "" + "\n" + "Label%s\n" + "ProgramArguments\n" + "\n" + "%s/git\n" + "--exec-path=%s\n" + "for-each-repo\n" + "--config=maintenance.repo\n" + "maintenance\n" + "run\n" + "--schedule=%s\n" + "\n" + "StartCalendarInterval\n" + "\n"; + fprintf(plist, preamble, name, exec_path, exec_path, frequency); + + switch (schedule) { + case SCHEDULE_HOURLY: + repeat = "\n" + "Hour%d\n" + "Minute0\n" + "\n"; + for (i = 1; i <= 23; i++) + fprintf(plist, repeat, i); + break; + + case SCHEDULE_DAILY: + repeat = "\n" + "Day%d\n" + "Hour0\n" + "Minute0\n" + "\n"; + for (i = 1; i <= 6; i++) + fprintf(plist, repeat, i); + break; + + case SCHEDULE_WEEKLY: + fprintf(plist, + "\n" + "Day0\n" + "Hour0\n" + "Minute0\n" + "\n"); + break; + + default: + /* unreachable */ + break; + } + fprintf(plist, "\n\n\n"); + + /* bootout might fail if not already running, so ignore */ + bootout(filename); + if (bootstrap(filename)) + die(_("failed to bootstrap service %s"), filename); + + fclose(plist); + free(filename); + free(name); + return 0; +} + +static int add_plists(void) +{ + const char *exec_path = git_exec_path(); + + return schedule_plist(exec_path, SCHEDULE_HOURLY) || + schedule_plist(exec_path, SCHEDULE_DAILY) || + schedule_plist(exec_path, SCHEDULE_WEEKLY); +} + +static int platform_update_schedule(int run_maintenance, int fd) +{ + if (run_maintenance) + return add_plists(); + else + return remove_plists(); +} +#else #define BEGIN_LINE "# BEGIN GIT MAINTENANCE SCHEDULE" #define END_LINE "# END GIT MAINTENANCE SCHEDULE" @@ -1585,6 +1793,7 @@ static int platform_update_schedule(int run_maintenance, int fd) fclose(cron_list); return result; } +#endif static int update_background_schedule(int run_maintenance) { diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh index 20184e96e1..1c43b34a93 100755 --- a/t/t7900-maintenance.sh +++ b/t/t7900-maintenance.sh @@ -367,7 +367,7 @@ test_expect_success 'register and unregister' ' test_cmp before actual ' -test_expect_success 'start from empty cron table' ' +test_expect_success !MACOS_MAINTENANCE 'start from empty cron table' ' GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance start && # start registers the repo @@ -378,7 +378,7 @@ test_expect_success 'start from empty cron table' ' grep "for-each-repo --config=maintenance.repo maintenance run --schedule=weekly" cron.txt ' -test_expect_success 'stop from existing schedule' ' +test_expect_success !MACOS_MAINTENANCE 'stop from existing schedule' ' GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance stop && # stop does not unregister the repo @@ -389,12 +389,59 @@ test_expect_success 'stop from existing schedule' ' test_must_be_empty cron.txt ' -test_expect_success 'start preserves existing schedule' ' +test_expect_success !MACOS_MAINTENANCE 'start preserves existing schedule' ' echo "Important information!" >cron.txt && GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance start && grep "Important information!" cron.txt ' +test_expect_success MACOS_MAINTENANCE 'start and stop macOS maintenance' ' + echo "#!/bin/sh\necho \$@ >>args" >print-args && + chmod a+x print-args && + + rm -f args && + GIT_TEST_CRONTAB="./print-args" git maintenance start && + + # start registers the repo + git config --get --global maintenance.repo "$(pwd)" && + + # ~/Library/LaunchAgents + ls "$HOME/Library/LaunchAgents" >actual && + cat >expect <<-\EOF && + org.git-scm.git.daily.plist + org.git-scm.git.hourly.plist + org.git-scm.git.weekly.plist + EOF + test_cmp expect actual && + + rm expect && + for frequency in hourly daily weekly + do + PLIST="$HOME/Library/LaunchAgents/org.git-scm.git.$frequency.plist" && + xmllint "$PLIST" >/dev/null && + grep schedule=$frequency "$PLIST" && + echo "bootout gui/$UID $PLIST" >>expect && + echo "bootstrap gui/$UID $PLIST" >>expect || return 1 + done && + test_cmp expect args && + + rm -f args && + GIT_TEST_CRONTAB="./print-args" git maintenance stop && + + # stop does not unregister the repo + git config --get --global maintenance.repo "$(pwd)" && + + # stop does not remove plist files, but boots them out + rm expect && + for frequency in hourly daily weekly + do + PLIST="$HOME/Library/LaunchAgents/org.git-scm.git.$frequency.plist" && + grep schedule=$frequency "$PLIST" && + echo "bootout gui/$UID $PLIST" >>expect || return 1 + done && + test_cmp expect args +' + test_expect_success 'register preserves existing strategy' ' git config maintenance.strategy none && git maintenance register && diff --git a/t/test-lib.sh b/t/test-lib.sh index 4a60d1ed76..620ffbf3af 100644 --- a/t/test-lib.sh +++ b/t/test-lib.sh @@ -1703,6 +1703,10 @@ test_lazy_prereq REBASE_P ' test -z "$GIT_TEST_SKIP_REBASE_P" ' +test_lazy_prereq MACOS_MAINTENANCE ' + launchctl list +' + # Ensure that no test accidentally triggers a Git command # that runs 'crontab', affecting a user's cron schedule. # Tests that verify the cron integration must set this locally From patchwork Fri Nov 13 14:00:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Derrick Stolee X-Patchwork-Id: 11903559 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 32893C4742C for ; Fri, 13 Nov 2020 14:00:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E5D1D20797 for ; Fri, 13 Nov 2020 14:00:38 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="FhSfnYO3" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726498AbgKMOAh (ORCPT ); Fri, 13 Nov 2020 09:00:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54202 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726791AbgKMOAc (ORCPT ); Fri, 13 Nov 2020 09:00:32 -0500 Received: from mail-wr1-x444.google.com (mail-wr1-x444.google.com [IPv6:2a00:1450:4864:20::444]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1F629C061A04 for ; Fri, 13 Nov 2020 06:00:32 -0800 (PST) Received: by mail-wr1-x444.google.com with SMTP id p8so10037529wrx.5 for ; Fri, 13 Nov 2020 06:00:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=fTEjAtkU6V7Q0ym/yQsJ8oc/O6O+oT50gXUWwlJA/2o=; b=FhSfnYO3NSv7JvyH40xAiJ8zSvN/YNcN9GWy4SYM7w1zHfgFuqZsfDfyZ8FkqU2H1M SAfGrVu87KqRNtsYMWP/4nvfdez+w9pxPoUF8JmF7f3cdL6ImqHVLj7Yuzs+dJt7VVO9 1MIcZrTiqEYi5wJiklUUikptJJ55TmpG+wQEfpXuUso0o812K7Eh9QIA+UZnkKDEX10u zV9k9Sq8mQaE7/04vXA+4xwi6xFffMs6Is57zQYADqz818KO7v6y2uZbwpsb4zHJrhHy R8zViWbWRnEk0ts9uDTo44MTUCmBjk0qPamkbuJGzwnvw4BHUWiPjDeYP7SaEQ7cIEDC sNPw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=fTEjAtkU6V7Q0ym/yQsJ8oc/O6O+oT50gXUWwlJA/2o=; b=AWpDZn2UhXDwgC1DjM+s2W9qPU9b948yAsjuCQsjYldgkMKKS8vlOkqwcZNtNupJc1 dhXXmnNJmSjYJahD6bEDjjP7/VpiRDTytwVgV+9Jq1cBW640aFThAgtd45sl9A9DUfJt 7PUnWdFGOgf6gPgM3g2wTiQTD97BVp1H31ld3bH7BZKL9OjxhH3KEU2JK0+iE9ZznpdU jOMzBovavM7zFp5yqoDSJkmKtoZL4tLrn6fkRVFXB2vLeK5fdeDtXw+1vi9U2jsxcM7v nmruxpzLCfMFNY7jPZfO/K89kSuIgckNwvoNFLNW037tz7EYE/qjXIsSCwG2YrAdYugu lvNw== X-Gm-Message-State: AOAM5321IFuO4J9p0YXgPDZpXaN2520pZJ/iDNzIh12az2efEETeAzFD eot2F3pkSOe4umxrrpkBZL702wGagfU= X-Google-Smtp-Source: ABdhPJx+fvTvZXpKB/RTtwdEtwc/aCzBHVETfpRbyH6/EWcxfJQ+DVIaIwr9/lC556dXIqi7AmlhKg== X-Received: by 2002:a5d:43c6:: with SMTP id v6mr3673963wrr.295.1605276030469; Fri, 13 Nov 2020 06:00:30 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id m3sm6780115wrv.6.2020.11.13.06.00.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Nov 2020 06:00:29 -0800 (PST) Message-Id: In-Reply-To: References: Date: Fri, 13 Nov 2020 14:00:24 +0000 Subject: [PATCH v3 4/4] maintenance: use Windows scheduled tasks Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Eric Sunshine , Derrick Stolee , Derrick Stolee , Derrick Stolee Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Derrick Stolee From: Derrick Stolee Git's background maintenance uses cron by default, but this is not available on Windows. Instead, integrate with Task Scheduler. Tasks can be scheduled using the 'schtasks' command. There are several command-line options that can allow for some advanced scheduling, but unfortunately these seem to all require authenticating using a password. Instead, use the "/xml" option to pass an XML file that contains the configuration for the necessary schedule. These XML files are based on some that I exported after constructing a schedule in the Task Scheduler GUI. These options only run background maintenance when the user is logged in, and more fields are populated with the current username and SID at run-time by 'schtasks'. There is a deficiency in the current design. Windows has two kinds of applications: GUI applications that start by "winmain()" and console applications that start by "main()". Console applications are attached to a new Console window if they are not already associated with a GUI application. This means that every hour the scheudled task launches a command window for the scheduled tasks. Not only is this visually obtrusive, but it also takes focus from whatever else the user is doing! A simple fix would be to insert a GUI application that acts as a shim between the scheduled task and Git. This is currently possible in Git for Windows by setting the tag equal to C:\Program Files\Git\git-bash.exe with options "--hide --no-needs-console --command=cmd\git.exe" followed by the arguments currently used. Since git-bash.exe is not included in Windows builds of core Git, I chose to leave out this feature. My plan is to submit a small patch to Git for Windows that converts the use of git.exe with this use of git-bash.exe in the short term. In the long term, we can consider creating this GUI shim application within core Git, perhaps in contrib/. Helped-by: Eric Sunshine Signed-off-by: Derrick Stolee --- Documentation/git-maintenance.txt | 22 ++++ builtin/gc.c | 184 ++++++++++++++++++++++++++++++ t/t7900-maintenance.sh | 33 +++++- 3 files changed, 236 insertions(+), 3 deletions(-) diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt index 5f8f63f098..6970f2b898 100644 --- a/Documentation/git-maintenance.txt +++ b/Documentation/git-maintenance.txt @@ -313,6 +313,28 @@ To create more advanced customizations to your background tasks, see launchctl.plist(5) for more information. +BACKGROUND MAINTENANCE ON WINDOWS SYSTEMS +----------------------------------------- + +Windows does not support `cron` and instead has its own system for +scheduling background tasks. The `git maintenance start` command uses +the `schtasks` command to submit tasks to this system. You can inspect +all background tasks using the Task Scheduler application. The tasks +added by Git have names of the form `Git Maintenance ()`. +The Task Scheduler GUI has ways to inspect these tasks, but you can also +export the tasks to XML files and view the details there. + +Note that since Git is a console application, these background tasks +create a console window visible to the current user. This can be changed +manually by selecting the "Run whether user is logged in or not" option +in Task Scheduler. This change requires a password input, which is why +`git maintenance start` does not select it by default. + +If you want to customize the background tasks, please rename the tasks +so future calls to `git maintenance (start|stop)` do not overwrite your +custom tasks. + + GIT --- Part of the linkgit:git[1] suite diff --git a/builtin/gc.c b/builtin/gc.c index da2c892f68..76a3afa20a 100644 --- a/builtin/gc.c +++ b/builtin/gc.c @@ -1684,6 +1684,190 @@ static int platform_update_schedule(int run_maintenance, int fd) else return remove_plists(); } + +#elif defined(GIT_WINDOWS_NATIVE) + +static const char *get_frequency(enum schedule_priority schedule) +{ + switch (schedule) { + case SCHEDULE_HOURLY: + return "hourly"; + case SCHEDULE_DAILY: + return "daily"; + case SCHEDULE_WEEKLY: + return "weekly"; + default: + BUG("invalid schedule %d", schedule); + } +} + +static char *get_task_name(const char *frequency) +{ + struct strbuf label = STRBUF_INIT; + strbuf_addf(&label, "Git Maintenance (%s)", frequency); + return strbuf_detach(&label, NULL); +} + +static int remove_task(enum schedule_priority schedule) +{ + int result; + struct strvec args = STRVEC_INIT; + const char *frequency = get_frequency(schedule); + char *name = get_task_name(frequency); + const char *schtasks = getenv("GIT_TEST_CRONTAB"); + if (!schtasks) + schtasks = "schtasks"; + + strvec_split(&args, schtasks); + strvec_pushl(&args, "/delete", "/tn", name, "/f", NULL); + + result = run_command_v_opt(args.v, 0); + + strvec_clear(&args); + free(name); + return result; +} + +static int remove_scheduled_tasks(void) +{ + return remove_task(SCHEDULE_HOURLY) || + remove_task(SCHEDULE_DAILY) || + remove_task(SCHEDULE_WEEKLY); +} + +static int schedule_task(const char *exec_path, enum schedule_priority schedule) +{ + int result; + struct child_process child = CHILD_PROCESS_INIT; + const char *xml, *schtasks; + char *xmlpath, *tempDir; + FILE *xmlfp; + const char *frequency = get_frequency(schedule); + char *name = get_task_name(frequency); + + tempDir = xstrfmt("%s/temp", the_repository->objects->odb->path); + xmlpath = xstrfmt("%s/schedule-%s.xml", tempDir, frequency); + safe_create_leading_directories(xmlpath); + xmlfp = xfopen(xmlpath, "w"); + + xml = "\n" + "\n" + "\n" + "\n"; + fprintf(xmlfp, xml); + + switch (schedule) { + case SCHEDULE_HOURLY: + fprintf(xmlfp, + "2020-01-01T01:00:00\n" + "true\n" + "\n" + "1\n" + "\n" + "\n" + "PT1H\n" + "PT23H\n" + "false\n" + "\n"); + break; + + case SCHEDULE_DAILY: + fprintf(xmlfp, + "2020-01-01T00:00:00\n" + "true\n" + "\n" + "\n" + "\n" + "\n" + "\n" + "\n" + "\n" + "\n" + "\n" + "1\n" + "\n"); + break; + + case SCHEDULE_WEEKLY: + fprintf(xmlfp, + "2020-01-01T00:00:00\n" + "true\n" + "\n" + "\n" + "\n" + "\n" + "1\n" + "\n"); + break; + + default: + break; + } + + xml= "\n" + "\n" + "\n" + "\n" + "InteractiveToken\n" + "LeastPrivilege\n" + "\n" + "\n" + "\n" + "IgnoreNew\n" + "true\n" + "true\n" + "true\n" + "false\n" + "PT72H\n" + "7\n" + "\n" + "\n" + "\n" + "\"%s\\git.exe\"\n" + "--exec-path=\"%s\" for-each-repo --config=maintenance.repo maintenance run --schedule=%s\n" + "\n" + "\n" + "\n"; + fprintf(xmlfp, xml, exec_path, exec_path, frequency); + fclose(xmlfp); + + schtasks = getenv("GIT_TEST_CRONTAB"); + if (!schtasks) + schtasks = "schtasks"; + strvec_split(&child.args, schtasks); + strvec_pushl(&child.args, "/create", "/tn", name, "/f", "/xml", xmlpath, NULL); + + child.no_stdout = 1; + child.no_stderr = 1; + + if (start_command(&child)) + die(_("failed to start schtasks")); + result = finish_command(&child); + + unlink(xmlpath); + rmdir(tempDir); + free(xmlpath); + free(name); + return result; +} + +static int add_scheduled_tasks(void) +{ + const char *exec_path = git_exec_path(); + + return schedule_task(exec_path, SCHEDULE_HOURLY) || + schedule_task(exec_path, SCHEDULE_DAILY) || + schedule_task(exec_path, SCHEDULE_WEEKLY); +} + +static int platform_update_schedule(int run_maintenance, int fd) +{ + if (run_maintenance) + return add_scheduled_tasks(); + else + return remove_scheduled_tasks(); +} + #else #define BEGIN_LINE "# BEGIN GIT MAINTENANCE SCHEDULE" #define END_LINE "# END GIT MAINTENANCE SCHEDULE" diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh index 29d340a828..0dc2479117 100755 --- a/t/t7900-maintenance.sh +++ b/t/t7900-maintenance.sh @@ -367,7 +367,7 @@ test_expect_success 'register and unregister' ' test_cmp before actual ' -test_expect_success !MACOS_MAINTENANCE 'start from empty cron table' ' +test_expect_success !MACOS_MAINTENANCE,!MINGW 'start from empty cron table' ' GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance start && # start registers the repo @@ -378,7 +378,7 @@ test_expect_success !MACOS_MAINTENANCE 'start from empty cron table' ' grep "for-each-repo --config=maintenance.repo maintenance run --schedule=weekly" cron.txt ' -test_expect_success !MACOS_MAINTENANCE 'stop from existing schedule' ' +test_expect_success !MACOS_MAINTENANCE,!MINGW 'stop from existing schedule' ' GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance stop && # stop does not unregister the repo @@ -389,7 +389,7 @@ test_expect_success !MACOS_MAINTENANCE 'stop from existing schedule' ' test_must_be_empty cron.txt ' -test_expect_success !MACOS_MAINTENANCE 'start preserves existing schedule' ' +test_expect_success !MACOS_MAINTENANCE,!MINGW 'start preserves existing schedule' ' echo "Important information!" >cron.txt && GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance start && grep "Important information!" cron.txt @@ -437,6 +437,33 @@ test_expect_success MACOS_MAINTENANCE 'start and stop macOS maintenance' ' test_line_count = 0 actual ' +test_expect_success MINGW 'start and stop Windows maintenance' ' + write_script print-args <<-\EOF && + echo $* >>args + EOF + + rm -f args && + GIT_TEST_CRONTAB="/bin/sh print-args" git maintenance start && + + # start registers the repo + git config --get --global maintenance.repo "$(pwd)" && + + printf "/create /tn Git Maintenance (%s) /f /xml .git/objects/temp/schedule-%s.xml\n" \ + hourly hourly daily daily weekly weekly >expect && + test_cmp expect args && + + rm -f args && + GIT_TEST_CRONTAB="/bin/sh print-args" git maintenance stop && + + # stop does not unregister the repo + git config --get --global maintenance.repo "$(pwd)" && + + rm expect && + printf "/delete /tn Git Maintenance (%s) /f\n" \ + hourly daily weekly >expect && + test_cmp expect args +' + test_expect_success 'register preserves existing strategy' ' git config maintenance.strategy none && git maintenance register &&