From patchwork Tue Feb 15 15:59:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff Hostetler X-Patchwork-Id: 12747256 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 73B06C433F5 for ; Tue, 15 Feb 2022 15:59:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241089AbiBOP7z (ORCPT ); Tue, 15 Feb 2022 10:59:55 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:56016 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241073AbiBOP7u (ORCPT ); Tue, 15 Feb 2022 10:59:50 -0500 Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 670ACBD2FC for ; Tue, 15 Feb 2022 07:59:39 -0800 (PST) Received: by mail-wm1-x334.google.com with SMTP id k3-20020a1ca103000000b0037bdea84f9cso1705526wme.1 for ; Tue, 15 Feb 2022 07:59:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=sKPEzHAsshFqncts38/XgWTJRQpaIG5aJT7K6hJZ6fI=; b=lx+LWu298GuK/hUr5jDlrbmPyXLu57uYyjX/Cl+/moidv/UNXt89g4m6cjFdHCyKlb rRMuhSPnA9UKF8zhNdBSVh0y8XxuKx19TC0K31q/xmcJTPUKvvnZji94U7KKCBmpRBeo 9+/p5KK8Qqs7Nn/04vxQGcMGOwJHxa+34JyvmTT5/Vmn8o40XSK5eK8lJSngb5fyuhxs yMdP0d2TECRrkD21+TccBEy2FrZdukWffN9MPGnBorjqine2DhfyqH+ON5GdzxLIJ9Lu OsOC03lqHE38+kzs11QhPLlQkmGrRGagZ+o3p4XmJx0+v9yYW3EuowPYhZLvCEO9BNGm fJlg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=sKPEzHAsshFqncts38/XgWTJRQpaIG5aJT7K6hJZ6fI=; b=Gkz5YG+bziOygB5W/I3ZBt+YeQJELTMXnOP5H9mtKb/qqS2qptGaIHAmZ7+71ehVvt +g0TAsBMvd1kyfaY/fEqzYR1zrtVQ4/JJAltwStXEBf5AoRuv3SNvqDgJuMDK4xdjqP0 VFzAdacSH5Znsod4en0Cw0TsLzV+zv509mFczyXkmu51IlG9daWGIa4qcBqwxQO5aNAT WFXTlLVANNpE6zKjWClxLwV9G/KYTwunQMMz4QyzV7HZuTufeK/9ytkZmlKvptQB9tlI G0Ts5A55X3lD5suNV9w4yQ4A/yDpJN96u5LqBwsF7A7iNRNMglQJpNMr/KQHkX2V9kIb t40g== X-Gm-Message-State: AOAM533zmA2trvitI7JEKWHewcO5ePNO5a3oCWbmWNwuUFF7mXJOjYJx 6gTFf9A4uFPup+PFrw/cfm5uSTcsyvA= X-Google-Smtp-Source: ABdhPJzYyZpQf44l8V8Yps5ezKhA0Le9u2mff4RCS+D/bXpQlBfxtB7kR2ZwIoiwclzcG1QNFjE+OQ== X-Received: by 2002:a05:600c:3b87:b0:37c:af2b:4f36 with SMTP id n7-20020a05600c3b8700b0037caf2b4f36mr3642933wms.132.1644940777221; Tue, 15 Feb 2022 07:59:37 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id w18sm16188487wrl.62.2022.02.15.07.59.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Feb 2022 07:59:36 -0800 (PST) Message-Id: <23f38338cecd270c15fd1bba42de79c53e14ca2a.1644940773.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 15 Feb 2022 15:59:11 +0000 Subject: [PATCH 01/23] fsm-listen-win32: handle shortnames Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Jeff Hostetler , Jeff Hostetler Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Jeff Hostetler From: Jeff Hostetler Teach FSMonitor daemon on Windows to recognize shortname paths as aliases of normal longname paths. FSMonitor clients, such as `git status`, should receive the longname spelling of changed files (when possible). Sometimes we receive FS events using the shortname, such as when a CMD shell runs "RENAME GIT~1 FOO" or "RMDIR GIT~1". The FS notification arrives using whatever combination of long and shortnames were used by the other process. (Shortnames do seem to be case normalized, however.) Use Windows GetLongPathNameW() to try to map the pathname spelling in the notification event into the normalized longname spelling. (This can fail if the file/directory is deleted, moved, or renamed, because we are asking the FS for the mapping in response to the event and after it has already happened, but we try.) Special case the shortname spelling of ".git" to avoid under-reporting these events. Signed-off-by: Jeff Hostetler --- compat/fsmonitor/fsm-listen-win32.c | 359 +++++++++++++++++++++++----- t/t7527-builtin-fsmonitor.sh | 65 +++++ 2 files changed, 370 insertions(+), 54 deletions(-) diff --git a/compat/fsmonitor/fsm-listen-win32.c b/compat/fsmonitor/fsm-listen-win32.c index c2d11acbc1e..eb407b0748c 100644 --- a/compat/fsmonitor/fsm-listen-win32.c +++ b/compat/fsmonitor/fsm-listen-win32.c @@ -25,6 +25,9 @@ struct one_watch DWORD count; struct strbuf path; + wchar_t wpath_longname[MAX_PATH + 1]; + DWORD wpath_longname_len; + HANDLE hDir; HANDLE hEvent; OVERLAPPED overlapped; @@ -34,6 +37,21 @@ struct one_watch * need to later call GetOverlappedResult() and possibly CancelIoEx(). */ BOOL is_active; + + /* + * Are shortnames enabled on the containing drive? This is + * always true for "C:/" drives and usually never true for + * other drives. + * + * We only set this for the worktree because we only need to + * convert shortname paths to longname paths for items we send + * to clients. (We don't care about shortname expansion for + * paths inside a GITDIR because we never send them to + * clients.) + */ + BOOL has_shortnames; + BOOL has_tilda; + wchar_t dotgit_shortname[16]; /* for 8.3 name */ }; struct fsmonitor_daemon_backend_data @@ -51,17 +69,18 @@ struct fsmonitor_daemon_backend_data }; /* - * Convert the WCHAR path from the notification into UTF8 and - * then normalize it. + * Convert the WCHAR path from the event into UTF8 and normalize it. + * + * `wpath_len` is in WCHARS not bytes. */ -static int normalize_path_in_utf8(FILE_NOTIFY_INFORMATION *info, +static int normalize_path_in_utf8(wchar_t *wpath, DWORD wpath_len, struct strbuf *normalized_path) { int reserve; int len = 0; strbuf_reset(normalized_path); - if (!info->FileNameLength) + if (!wpath_len) goto normalize; /* @@ -70,12 +89,12 @@ static int normalize_path_in_utf8(FILE_NOTIFY_INFORMATION *info, * sequence of 2 UTF8 characters. That should let us * avoid ERROR_INSUFFICIENT_BUFFER 99.9+% of the time. */ - reserve = info->FileNameLength + 1; + reserve = 2 * wpath_len + 1; strbuf_grow(normalized_path, reserve); for (;;) { - len = WideCharToMultiByte(CP_UTF8, 0, info->FileName, - info->FileNameLength / sizeof(WCHAR), + len = WideCharToMultiByte(CP_UTF8, 0, + wpath, wpath_len, normalized_path->buf, strbuf_avail(normalized_path) - 1, NULL, NULL); @@ -83,9 +102,7 @@ static int normalize_path_in_utf8(FILE_NOTIFY_INFORMATION *info, goto normalize; if (GetLastError() != ERROR_INSUFFICIENT_BUFFER) { error("[GLE %ld] could not convert path to UTF-8: '%.*ls'", - GetLastError(), - (int)(info->FileNameLength / sizeof(WCHAR)), - info->FileName); + GetLastError(), (int)wpath_len, wpath); return -1; } @@ -98,6 +115,148 @@ normalize: return strbuf_normalize_path(normalized_path); } +/* + * See if the worktree root directory has shortnames enabled. + * This will help us decide if we need to do an expensive shortname + * to longname conversion on every notification event. + * + * We do not want to create a file to test this, so we assume that the + * root directory contains a ".git" file or directory. (Out caller + * only calls us for the worktree root, so this should be fine.) + * + * Remember the spelling of the shortname for ".git" if it exists. + */ +static void check_for_shortnames(struct one_watch *watch) +{ + wchar_t buf_in[MAX_PATH + 1]; + wchar_t buf_out[MAX_PATH + 1]; + wchar_t *last_slash = NULL; + wchar_t *last_bslash = NULL; + wchar_t *last; + + /* build L"/.git" */ + wcscpy(buf_in, watch->wpath_longname); + wcscpy(buf_in + watch->wpath_longname_len, L".git"); + + if (!GetShortPathNameW(buf_in, buf_out, MAX_PATH)) + return; + + last_slash = wcsrchr(buf_out, L'/'); + last_bslash = wcsrchr(buf_out, L'\\'); + if (last_slash > last_bslash) + last = last_slash + 1; + else if (last_bslash) + last = last_bslash + 1; + else + last = buf_out; + + if (!wcscmp(last, L".git")) + return; + + watch->has_shortnames = 1; + wcsncpy(watch->dotgit_shortname, last, + ARRAY_SIZE(watch->dotgit_shortname)); + + /* + * The shortname for ".git" is usually of the form "GIT~1", so + * we should be able to avoid shortname to longname mapping on + * every notification event if the source string does not + * contain a "~". + * + * However, the documentation for GetLongPathNameW() says + * that there are filesystems that don't follow that pattern + * and warns against this optimization. + * + * Lets test this. + */ + if (wcschr(watch->dotgit_shortname, L'~')) + watch->has_tilda = 1; +} + +enum get_relative_result { + GRR_NO_CONVERSION_NEEDED, + GRR_HAVE_CONVERSION, + GRR_SHUTDOWN, +}; + +/* + * Info notification paths are relative to the root of the watch. + * If our CWD is still at the root, then we can use relative paths + * to convert from shortnames to longnames. If our process has a + * different CWD, then we need to construct an absolute path, do + * the conversion, and then return the root-relative portion. + * + * We use the longname form of the root as our basis and assume that + * it already has a trailing slash. + * + * `wpath_len` is in WCHARS not bytes. + */ +static enum get_relative_result get_relative_longname( + struct one_watch *watch, + const wchar_t *wpath, DWORD wpath_len, + wchar_t *wpath_longname) +{ + wchar_t buf_in[2 * MAX_PATH + 1]; + wchar_t buf_out[MAX_PATH + 1]; + DWORD root_len; + + /* Build L"/" */ + root_len = watch->wpath_longname_len; + wcsncpy(buf_in, watch->wpath_longname, root_len); + wcsncpy(buf_in + root_len, wpath, wpath_len); + buf_in[root_len + wpath_len] = 0; + + /* + * We don't actually know if the source pathname is a + * shortname or a longname. This routine allows either to be + * given as input. + */ + if (!GetLongPathNameW(buf_in, buf_out, MAX_PATH)) { + /* + * The shortname to longname conversion can fail for + * various reasons, for example if the file has been + * deleted. (That is, if we just received a + * delete-file notification event and the file is + * already gone, we can't ask the file system to + * lookup the longname for it. Likewise, for moves + * and renames where we are given the old name.) + * + * NEEDSWORK: Since deleting or moving a file or + * directory by shortname is rather obscure, I'm going + * ignore the failure and ask the caller to report the + * original relative path. This seemds kinder than + * failing here and forcing a resync. + */ + return GRR_NO_CONVERSION_NEEDED; + } + + if (!wcscmp(buf_in, buf_out)) { + /* + * The path does not have a shortname alias. + */ + return GRR_NO_CONVERSION_NEEDED; + } + + if (wcsncmp(buf_in, buf_out, root_len)) { + /* + * The spelling of the root directory portion of the computed + * longname has changed. This should not happen. Basically, + * it means that we don't know where (without recomputing the + * longname of just the root directory) to split out the + * relative path. Since this should not happen, I'm just + * going to let this fail and force a shutdown (because all + * subsequent events are probably going to see the same + * mismatch). + */ + return GRR_SHUTDOWN; + } + + /* Return the worktree root-relative portion of the longname. */ + + wcscpy(wpath_longname, buf_out + root_len); + return GRR_HAVE_CONVERSION; +} + void fsm_listen__stop_async(struct fsmonitor_daemon_state *state) { SetEvent(state->backend_data->hListener[LISTENER_SHUTDOWN]); @@ -111,7 +270,9 @@ static struct one_watch *create_watch(struct fsmonitor_daemon_state *state, DWORD share_mode = FILE_SHARE_WRITE | FILE_SHARE_READ | FILE_SHARE_DELETE; HANDLE hDir; - wchar_t wpath[MAX_PATH]; + DWORD len_longname; + wchar_t wpath[MAX_PATH + 1]; + wchar_t wpath_longname[MAX_PATH + 1]; if (xutftowcs_path(wpath, path) < 0) { error(_("could not convert to wide characters: '%s'"), path); @@ -128,6 +289,20 @@ static struct one_watch *create_watch(struct fsmonitor_daemon_state *state, return NULL; } + if (!GetLongPathNameW(wpath, wpath_longname, MAX_PATH)) { + error(_("[GLE %ld] could not get longname of '%s'"), + GetLastError(), path); + CloseHandle(hDir); + return NULL; + } + + len_longname = wcslen(wpath_longname); + if (wpath_longname[len_longname - 1] != L'/' && + wpath_longname[len_longname - 1] != L'\\') { + wpath_longname[len_longname++] = L'/'; + wpath_longname[len_longname] = 0; + } + CALLOC_ARRAY(watch, 1); watch->buf_len = sizeof(watch->buffer); /* assume full MAX_RDCW_BUF */ @@ -135,6 +310,9 @@ static struct one_watch *create_watch(struct fsmonitor_daemon_state *state, strbuf_init(&watch->path, 0); strbuf_addstr(&watch->path, path); + wcscpy(watch->wpath_longname, wpath_longname); + watch->wpath_longname_len = len_longname; + watch->hDir = hDir; watch->hEvent = CreateEvent(NULL, TRUE, FALSE, NULL); @@ -258,6 +436,62 @@ static void cancel_rdcw_watch(struct one_watch *watch) watch->is_active = FALSE; } +/* + * Process a single relative pathname event. + * Return 1 if we should shutdown. + */ +static int process_1_worktree_event( + struct string_list *cookie_list, + struct fsmonitor_batch **batch, + const struct strbuf *path, + enum fsmonitor_path_type t, + DWORD info_action) +{ + const char *slash; + + switch (t) { + case IS_INSIDE_DOT_GIT_WITH_COOKIE_PREFIX: + /* special case cookie files within .git */ + + /* Use just the filename of the cookie file. */ + slash = find_last_dir_sep(path->buf); + string_list_append(cookie_list, + slash ? slash + 1 : path->buf); + break; + + case IS_INSIDE_DOT_GIT: + /* ignore everything inside of "/.git/" */ + break; + + case IS_DOT_GIT: + /* "/.git" was deleted (or renamed away) */ + if ((info_action == FILE_ACTION_REMOVED) || + (info_action == FILE_ACTION_RENAMED_OLD_NAME)) { + trace2_data_string("fsmonitor", NULL, + "fsm-listen/dotgit", + "removed"); + return 1; + } + break; + + case IS_WORKDIR_PATH: + /* queue normal pathname */ + if (!*batch) + *batch = fsmonitor_batch__new(); + fsmonitor_batch__add_path(*batch, path->buf); + break; + + case IS_GITDIR: + case IS_INSIDE_GITDIR: + case IS_INSIDE_GITDIR_WITH_COOKIE_PREFIX: + default: + BUG("unexpected path classification '%d' for '%s'", + t, path->buf); + } + + return 0; +} + /* * Process filesystem events that happen anywhere (recursively) under the * root directory. For a normal working directory, this includes @@ -274,6 +508,7 @@ static int process_worktree_events(struct fsmonitor_daemon_state *state) struct string_list cookie_list = STRING_LIST_INIT_DUP; struct fsmonitor_batch *batch = NULL; const char *p = watch->buffer; + wchar_t wpath_longname[MAX_PATH + 1]; /* * If the kernel gets more events than will fit in the kernel @@ -306,54 +541,63 @@ static int process_worktree_events(struct fsmonitor_daemon_state *state) */ for (;;) { FILE_NOTIFY_INFORMATION *info = (void *)p; - const char *slash; + wchar_t *wpath = info->FileName; + DWORD wpath_len = info->FileNameLength / sizeof(WCHAR); enum fsmonitor_path_type t; + enum get_relative_result grr; + + if (watch->has_shortnames) { + if (!wcscmp(wpath, watch->dotgit_shortname)) { + /* + * This event exactly matches the + * spelling of the shortname of + * ".git", so we can skip some steps. + * + * (This case is odd because the user + * can "rm -rf GIT~1" and we cannot + * use the filesystem to map it back + * to ".git".) + */ + strbuf_reset(&path); + strbuf_addstr(&path, ".git"); + t = IS_DOT_GIT; + goto process_it; + } - strbuf_reset(&path); - if (normalize_path_in_utf8(info, &path) == -1) - goto skip_this_path; - - t = fsmonitor_classify_path_workdir_relative(path.buf); - - switch (t) { - case IS_INSIDE_DOT_GIT_WITH_COOKIE_PREFIX: - /* special case cookie files within .git */ - - /* Use just the filename of the cookie file. */ - slash = find_last_dir_sep(path.buf); - string_list_append(&cookie_list, - slash ? slash + 1 : path.buf); - break; - - case IS_INSIDE_DOT_GIT: - /* ignore everything inside of "/.git/" */ - break; + if (watch->has_tilda && !wcschr(wpath, L'~')) { + /* + * Shortnames on this filesystem have tildas + * and the notification path does not have + * one, so we assume that it is a longname. + */ + goto normalize_it; + } - case IS_DOT_GIT: - /* "/.git" was deleted (or renamed away) */ - if ((info->Action == FILE_ACTION_REMOVED) || - (info->Action == FILE_ACTION_RENAMED_OLD_NAME)) { - trace2_data_string("fsmonitor", NULL, - "fsm-listen/dotgit", - "removed"); + grr = get_relative_longname(watch, wpath, wpath_len, + wpath_longname); + switch (grr) { + case GRR_NO_CONVERSION_NEEDED: /* use info buffer as is */ + break; + case GRR_HAVE_CONVERSION: + wpath = wpath_longname; + wpath_len = wcslen(wpath); + break; + default: + case GRR_SHUTDOWN: goto force_shutdown; } - break; + } - case IS_WORKDIR_PATH: - /* queue normal pathname */ - if (!batch) - batch = fsmonitor_batch__new(); - fsmonitor_batch__add_path(batch, path.buf); - break; +normalize_it: + if (normalize_path_in_utf8(wpath, wpath_len, &path) == -1) + goto skip_this_path; - case IS_GITDIR: - case IS_INSIDE_GITDIR: - case IS_INSIDE_GITDIR_WITH_COOKIE_PREFIX: - default: - BUG("unexpected path classification '%d' for '%s'", - t, path.buf); - } + t = fsmonitor_classify_path_workdir_relative(path.buf); + +process_it: + if (process_1_worktree_event(&cookie_list, &batch, &path, t, + info->Action)) + goto force_shutdown; skip_this_path: if (!info->NextEntryOffset) @@ -382,6 +626,9 @@ force_shutdown: * Note that we DO NOT get filesystem events on the external * itself (it is not inside something that we are watching). In particular, * we do not get an event if the external is deleted. + * + * Also, we do not care about shortnames within the external , since + * we never send these paths to clients. */ static int process_gitdir_events(struct fsmonitor_daemon_state *state) { @@ -403,8 +650,10 @@ static int process_gitdir_events(struct fsmonitor_daemon_state *state) const char *slash; enum fsmonitor_path_type t; - strbuf_reset(&path); - if (normalize_path_in_utf8(info, &path) == -1) + if (normalize_path_in_utf8( + info->FileName, + info->FileNameLength / sizeof(WCHAR), + &path) == -1) goto skip_this_path; t = fsmonitor_classify_path_gitdir_relative(path.buf); @@ -538,6 +787,8 @@ int fsm_listen__ctor(struct fsmonitor_daemon_state *state) if (!data->watch_worktree) goto failed; + check_for_shortnames(data->watch_worktree); + if (state->nr_paths_watching > 1) { data->watch_gitdir = create_watch(state, state->path_gitdir_watch.buf); diff --git a/t/t7527-builtin-fsmonitor.sh b/t/t7527-builtin-fsmonitor.sh index 0ccbfb9616f..dbca7f835eb 100755 --- a/t/t7527-builtin-fsmonitor.sh +++ b/t/t7527-builtin-fsmonitor.sh @@ -123,6 +123,71 @@ test_expect_success 'implicit daemon stop (rename .git)' ' test_must_fail git -C test_implicit_2 fsmonitor--daemon status ' +# File systems on Windows may or may not have shortnames. +# This is a volume-specific setting on modern systems. +# "C:/" drives are required to have them enabled. Other +# hard drives default to disabled. +# +# This is a crude test to see if shortnames are enabled +# on the volume containing the test directory. It is +# crude, but it does not require elevation like `fsutil`. +# +test_lazy_prereq SHORTNAMES ' + mkdir .foo && + test -d "FOO~1" +' + +# Here we assume that the shortname of ".git" is "GIT~1". +test_expect_success MINGW,SHORTNAMES 'implicit daemon stop (rename GIT~1)' ' + test_when_finished "stop_daemon_delete_repo test_implicit_1s" && + + git init test_implicit_1s && + + start_daemon test_implicit_1s && + + # renaming the .git directory will implicitly stop the daemon. + # this moves {.git, GIT~1} to {.gitxyz, GITXYZ~1}. + # the rename-from FS Event will contain the shortname. + # + mv test_implicit_1s/GIT~1 test_implicit_1s/.gitxyz && + + sleep 1 && + # put it back so that our status will not crawl out to our + # parent directory. + # this moves {.gitxyz, GITXYZ~1} to {.git, GIT~1}. + mv test_implicit_1s/.gitxyz test_implicit_1s/.git && + + test_must_fail git -C test_implicit_1s fsmonitor--daemon status +' + +# Here we first create a file with LONGNAME of "GIT~1" before +# we create the repo. This will cause the shortname of ".git" +# to be "GIT~2". +test_expect_success MINGW,SHORTNAMES 'implicit daemon stop (rename GIT~2)' ' + test_when_finished "stop_daemon_delete_repo test_implicit_1s2" && + + mkdir test_implicit_1s2 && + echo HELLO >test_implicit_1s2/GIT~1 && + git init test_implicit_1s2 && + + test_path_is_file test_implicit_1s2/GIT~1 && + test_path_is_dir test_implicit_1s2/GIT~2 && + + start_daemon test_implicit_1s2 && + + # renaming the .git directory will implicitly stop the daemon. + # the rename-from FS Event will contain the shortname. + # + mv test_implicit_1s2/GIT~2 test_implicit_1s2/.gitxyz && + + sleep 1 && + # put it back so that our status will not crawl out to our + # parent directory. + mv test_implicit_1s2/.gitxyz test_implicit_1s2/.git && + + test_must_fail git -C test_implicit_1s2 fsmonitor--daemon status +' + test_expect_success 'cannot start multiple daemons' ' test_when_finished "stop_daemon_delete_repo test_multiple" && From patchwork Tue Feb 15 15:59:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff Hostetler X-Patchwork-Id: 12747255 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A3095C433F5 for ; Tue, 15 Feb 2022 15:59:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241082AbiBOP7v (ORCPT ); Tue, 15 Feb 2022 10:59:51 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:56004 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237550AbiBOP7t (ORCPT ); Tue, 15 Feb 2022 10:59:49 -0500 Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com [IPv6:2a00:1450:4864:20::42a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9D8A8BD2FA for ; Tue, 15 Feb 2022 07:59:39 -0800 (PST) Received: by mail-wr1-x42a.google.com with SMTP id h6so32893898wrb.9 for ; Tue, 15 Feb 2022 07:59:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=mhF4CuQxafkn47pjkZom0XYeBg1uoIRwdMOe7ojOkSE=; b=ZaWr8uH24kzlYbxholEeBzf96FHwKqiJw0BRm9f9mtjQMF4sagh4mvVxByOi0Mpx/z LGLuP1EUPINEbcEQhNJlrBuaMTlRRMKNo5bvXQmyOVYb8OlU6Aq2C0Ty6A2zpAP0IT5v 6Hi/ycbgNzofM6AURPFyOJ3sv+VONv1JnSnZmQ5HHHfOI9eMmNZ2rX4siEhvYZX306qn S66ND4/mOqfUaydvMeOCMyqtvUj9An4e0k2oY6O4qNFHLM0lvaBHFl6NNJviPeBre3N+ VACUfhjql3ApnPqXLRBmIhRiKPvVdmByAQBqVKVF1SHRKdD3eJ3GLcGVEyHiFBaxP21d v9iQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=mhF4CuQxafkn47pjkZom0XYeBg1uoIRwdMOe7ojOkSE=; b=cF+F94yQ0L20FMKwVgG7jRUHLgoWcz7QOORyrPFHS4zrAq/lPoRN4dz1Kl38A4dOqD vs557BNNKCIQ7aP0uMoTetcsGq55CkDHi1UbazuAAy9GkPd0qjhxscwRkQUyFzaUWIHL i4E+f+p0SC0B3aq1v/0LqN+MdwhatyaKlMlv94lCBm7ivdffkOj/T3auKtyvEp/s1Tha v6jTJdCEp6iQX0SGSthgXsSeQjngXkDxDqvurC1zmIQB+CFKTTNQC8xza+F6/PJWga5Y hUU9kJr10Ip+h7fezOl6peTeSXWl0MSzqaGIUM0Z2j8arDwxOqaAdTAV5JapGQsVZbTr 8EVA== X-Gm-Message-State: AOAM533DraTkxXQedOxFOxTpLamsYAhopbGAm04B9D0iEpNpqlwcZke8 Fh0vdT6tnB3Eh2Il0C007EYHDDh9QKQ= X-Google-Smtp-Source: ABdhPJwqjMaHpr+o59fbeQsNjQkIWCE2RvjpE70OIOKaQRJPOgjoTjmmVqheguc7z4I5e3AKqMIquw== X-Received: by 2002:a5d:4903:: with SMTP id x3mr3749851wrq.363.1644940778013; Tue, 15 Feb 2022 07:59:38 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id h6sm6625573wmq.8.2022.02.15.07.59.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Feb 2022 07:59:37 -0800 (PST) Message-Id: In-Reply-To: References: Date: Tue, 15 Feb 2022 15:59:12 +0000 Subject: [PATCH 02/23] t7527: test FS event reporing on macOS WRT case and Unicode Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Jeff Hostetler , Jeff Hostetler Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Jeff Hostetler From: Jeff Hostetler Confirm that macOS FS events are reported with a normalized spelling. APFS (and/or HFS+) is case-insensitive. This means that case-independent lookups ( [ -d .git ] and [ -d .GIT ] ) should both succeed. But that doesn't tell us how FS events are reported if we try "rm -rf .git" versus "rm -rf .GIT". Are the events reported using the on-disk spelling of the pathname or in the spelling used by the command. NEEDSWORK: I was only able to test case. It would be nice to add tests that use different Unicode spellings/normalizations and understand the differences between APFS and HFS+ in this area. We should confirm that the spelling of the workdir paths that the daemon sends to clients are always properly normalized. Signed-off-by: Jeff Hostetler --- t/t7527-builtin-fsmonitor.sh | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/t/t7527-builtin-fsmonitor.sh b/t/t7527-builtin-fsmonitor.sh index dbca7f835eb..1fdabfc4f1e 100755 --- a/t/t7527-builtin-fsmonitor.sh +++ b/t/t7527-builtin-fsmonitor.sh @@ -188,6 +188,36 @@ test_expect_success MINGW,SHORTNAMES 'implicit daemon stop (rename GIT~2)' ' test_must_fail git -C test_implicit_1s2 fsmonitor--daemon status ' +# Confirm that MacOS hides all of the Unicode normalization and/or +# case folding from the FS events. That is, are the pathnames in the +# FS events reported using the spelling on the disk or in the spelling +# used by the other process. +# +# Note that we assume that the filesystem is set to case insensitive. +# +# NEEDSWORK: APFS handles Unicode and Unicode normalization +# differently than HFS+. I only have an APFS partition, so +# more testing here would be helpful. +# + +# Rename .git using alternate spelling and confirm that the daemon +# sees the event using the correct spelling and shutdown. +test_expect_success UTF8_NFD_TO_NFC 'MacOS event spelling (rename .GIT)' ' + test_when_finished "stop_daemon_delete_repo test_apfs" && + + git init test_apfs && + start_daemon test_apfs && + + test_path_is_dir test_apfs/.git && + test_path_is_dir test_apfs/.GIT && + + mv test_apfs/.GIT test_apfs/.FOO && + sleep 1 && + mv test_apfs/.FOO test_apfs/.git && + + test_must_fail git -C test_apfs fsmonitor--daemon status +' + test_expect_success 'cannot start multiple daemons' ' test_when_finished "stop_daemon_delete_repo test_multiple" && From patchwork Tue Feb 15 15:59:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff Hostetler X-Patchwork-Id: 12747257 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5A140C433FE for ; Tue, 15 Feb 2022 15:59:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241094AbiBOP74 (ORCPT ); Tue, 15 Feb 2022 10:59:56 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:56022 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241075AbiBOP7u (ORCPT ); Tue, 15 Feb 2022 10:59:50 -0500 Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9B7E7BD8B4 for ; Tue, 15 Feb 2022 07:59:40 -0800 (PST) Received: by mail-wr1-x432.google.com with SMTP id i14so32845024wrc.10 for ; Tue, 15 Feb 2022 07:59:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=AYYV/9S1Pxi+/nQo2/9PJccjytswo1D3DmUghvHX2iw=; b=E+0/MQZwUuCdeg4tMFIYo8vO7zKJu1DU2Daa7m72eNH6N6GskbsNn2f4OJS7BXeUZk nbzX3qdD0Fqkq7NQ55rd9EsWQhNPrucAMTCnn8N7SjRS07OYREgWUM06U9pKX7lnolZZ HipLgR3ssO5u4LLGbEouq0LNH7lwu0sSkCL/hI16PuND7Pc9R3XE4o6D4TncI7S2k/5w JULwvs1Dp4dRRlUDu9T6rQwGq/IC+ZDCM0EsMDaqovYa+w6xKNdP24nSkpgs7sARhv92 CV9+8zqsaye7xIqpL8ulS6RZDh1uXS7kwuNNaMiKVvt/GK4YQHqDV+THTuRJ3XeMEUgw gyIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=AYYV/9S1Pxi+/nQo2/9PJccjytswo1D3DmUghvHX2iw=; b=TTr4Zu62zh8GapXjQaF4bRpxiPK7u17Qm9149Lqoj9/3Z3idjpR9Tc3ju1UDU5pUnx buKqeqKfB0OO2dlNcjdZIHr5BLtiHJotXLuqkbomShUmLtPXICudXjwTQC38OCWlrmm1 kePsZv4o8AkcoHnnz/YwO5o/pjM1ANjHFb/m2ntbmlBv0JC0tiB5HCQQDLGpwVZl/d02 agypOdK9uXDX5GIV2ojM5eVP9W6yyssVnxtBCD8A2nrNUaxajHciP2QEnfYhGY2s58Mf wvC0/9Boegi+rQsFNOt+3Fz3TgAicVbRNStronuktnWBuopXO9hHRfdpdzgHAS0KNEdA d8jg== X-Gm-Message-State: AOAM533oiOkszHUjMi8DjU/Wzi7kr3UNz5tp5E1P5lKSZp6jJZTJqJVl hdCAWRpqD6kNqWtflulDPpWTCuUt/wc= X-Google-Smtp-Source: ABdhPJxf8O0cBF+vrPAcF7J3CemMicLoUyUv/D0eGx6LslEk32t0cDcfIq8Pd2FJ2cHXh8+HyC+63w== X-Received: by 2002:adf:d0c9:: with SMTP id z9mr3790529wrh.245.1644940778990; Tue, 15 Feb 2022 07:59:38 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id z17sm15798134wmf.11.2022.02.15.07.59.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Feb 2022 07:59:38 -0800 (PST) Message-Id: <9af952e4d170d71d66db721368af42ae5ce0fca8.1644940773.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 15 Feb 2022 15:59:13 +0000 Subject: [PATCH 03/23] t7527: test builtin FSMonitor watching repos with unicode paths Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Jeff Hostetler , Jeff Hostetler Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Jeff Hostetler From: Jeff Hostetler Create some test repos with UTF8 pathnames and verify that the builtin FSMonitor can watch them. This test is mainly for Windows where we need to avoid `*A()` routines. Signed-off-by: Jeff Hostetler --- t/t7527-builtin-fsmonitor.sh | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/t/t7527-builtin-fsmonitor.sh b/t/t7527-builtin-fsmonitor.sh index 1fdabfc4f1e..c0145544ccb 100755 --- a/t/t7527-builtin-fsmonitor.sh +++ b/t/t7527-builtin-fsmonitor.sh @@ -696,4 +696,27 @@ do done done +# Test Unicode UTF-8 characters in the pathname of the working +# directory. Use of "*A()" routines rather than "*W()" routines +# on Windows can sometimes lead to odd failures. +# +u1=$(printf "u_c3_a6__\xC3\xA6") +u2=$(printf "u_e2_99_ab__\xE2\x99\xAB") +u_values="$u1 $u2" +for u in $u_values +do + test_expect_success "Unicode path: $u" ' + test_when_finished "stop_daemon_delete_repo $u" && + + git init "$u" && + echo 1 >"$u"/file1 && + git -C "$u" add file1 && + git -C "$u" config core.fsmonitor true && + + start_daemon "$u" && + git -C "$u" status >actual && + grep "new file: file1" actual + ' +done + test_done From patchwork Tue Feb 15 15:59:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff Hostetler X-Patchwork-Id: 12747258 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 49EAFC433EF for ; Tue, 15 Feb 2022 15:59:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241115AbiBOP76 (ORCPT ); Tue, 15 Feb 2022 10:59:58 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:56066 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241084AbiBOP7y (ORCPT ); Tue, 15 Feb 2022 10:59:54 -0500 Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B5720BDE4C for ; Tue, 15 Feb 2022 07:59:41 -0800 (PST) Received: by mail-wr1-x42f.google.com with SMTP id h6so32894105wrb.9 for ; Tue, 15 Feb 2022 07:59:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=WCRJGBzKu4QyjQLrcDwJxN00AqoRiU+DAZFB0l/UuiE=; b=AJK0Y3RAkaqr7U4LjaZFuqc0XmlDNZApJq6JlT3PCeYob53uE38SIQYSbc14qxY22d uagAo0W4Kox2s6V5uY6i2FOdtvwxGzypc3O4ZqA3+JlNHIY/2DVaxyGWd6hdxO8dNjgB OzzEst/IFr/OkZAOev7S4sdngHHVjUVS8KyLmFbYwQPhzpLplTCJaOP3k+aaXXc7GDKl zR+i7Cs3Xp/3GBCiC0YZrp06X65SclFdQGJKvO6UPqPAldHJRlt4InYukP6gKA+x289z WqI+wWBv+S6dA1lXlwXKDFMb0hqCTWAtVQdG0WPjMev/IzbZcgzLbgv3pOuiWjTqIfRK /Aew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=WCRJGBzKu4QyjQLrcDwJxN00AqoRiU+DAZFB0l/UuiE=; b=YNMcIlBGBZYbvL8IDczrZF6gV1xDee45R0tSAPilom4a07cq6ecCpcwYkpaJhl1LCA 4lBV7gcq5S+jCFo+5XnQW7v7FzqBWwMv9yLfFcxQWv+gm4YeZTUb6IfE8TwvrI0MLaL3 g2TTXK1rWyGxtV5GW21f9DbIDe6XUIHaAIN1UwQ0SfUDdakQoSEI6/KAp/v+1a/Xl8oN lWdA6vYfMhnR+F7fiMc8Gg2SBcrn6F96UbcEA/uSbHY679l/0F5tySo85l/zZX5jmvj3 YItkd6lRyZgux6IiK+biy2ceXHLbK5ZAGXSBoifg5OG9YL/Zy5LKsvEEhdLzy3K/ocX9 NBMA== X-Gm-Message-State: AOAM531wb3IyF18E0bvU3Qxwd41yDHBCBGmWpKlVVIOATEjpQRhe5x5R +d0FQAEwTHgc/vY2woV4TKtgT+bMJvI= X-Google-Smtp-Source: ABdhPJxe5yC5R5mHlX+Q+uHYRkhZWMgqTtzrbE6Cwif1kJezhIOheIcuNMh+me6LarxjeSZNnYOzaw== X-Received: by 2002:adf:eed0:: with SMTP id a16mr3710084wrp.163.1644940780014; Tue, 15 Feb 2022 07:59:40 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id m8sm9688160wms.4.2022.02.15.07.59.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Feb 2022 07:59:39 -0800 (PST) Message-Id: <9efdbe28223e371dc22b4ba32fc791729af5577a.1644940773.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 15 Feb 2022 15:59:14 +0000 Subject: [PATCH 04/23] t/helper/fsmonitor-client: create stress test Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Jeff Hostetler , Jeff Hostetler Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Jeff Hostetler From: Jeff Hostetler Create a stress test to hammer on the fsmonitor daemon. Create a client-side thread pool of n threads and have each of them make m requests as fast as they can. NEEDSWORK: This is just the client-side thread pool and is useful for interactive testing and experimentation. We need to add a script test to drive this. Signed-off-by: Jeff Hostetler --- t/helper/test-fsmonitor-client.c | 105 +++++++++++++++++++++++++++++++ 1 file changed, 105 insertions(+) diff --git a/t/helper/test-fsmonitor-client.c b/t/helper/test-fsmonitor-client.c index f7a5b3a32fa..9dd2f9af553 100644 --- a/t/helper/test-fsmonitor-client.c +++ b/t/helper/test-fsmonitor-client.c @@ -7,6 +7,8 @@ #include "cache.h" #include "parse-options.h" #include "fsmonitor-ipc.h" +#include "thread-utils.h" +#include "trace2.h" #ifndef HAVE_FSMONITOR_DAEMON_BACKEND int cmd__fsmonitor_client(int argc, const char **argv) @@ -79,20 +81,120 @@ static int do_send_flush(void) return 0; } +struct hammer_thread_data +{ + pthread_t pthread_id; + int thread_nr; + + int nr_requests; + const char *token; + + int sum_successful; + int sum_errors; +}; + +static void *hammer_thread_proc(void *_hammer_thread_data) +{ + struct hammer_thread_data *data = _hammer_thread_data; + struct strbuf answer = STRBUF_INIT; + int k; + int ret; + + trace2_thread_start("hammer"); + + for (k = 0; k < data->nr_requests; k++) { + strbuf_reset(&answer); + + ret = fsmonitor_ipc__send_query(data->token, &answer); + if (ret < 0) + data->sum_errors++; + else + data->sum_successful++; + } + + strbuf_release(&answer); + trace2_thread_exit(); + return NULL; +} + +/* + * Start a pool of client threads that will each send a series of + * commands to the daemon. + * + * The goal is to overload the daemon with a sustained series of + * concurrent requests. + */ +static int do_hammer(const char *token, int nr_threads, int nr_requests) +{ + struct hammer_thread_data *data = NULL; + int k; + int sum_join_errors = 0; + int sum_commands = 0; + int sum_errors = 0; + + if (!token || !*token) + token = get_token_from_index(); + if (nr_threads < 1) + nr_threads = 1; + if (nr_requests < 1) + nr_requests = 1; + + CALLOC_ARRAY(data, nr_threads); + + for (k = 0; k < nr_threads; k++) { + struct hammer_thread_data *p = &data[k]; + p->thread_nr = k; + p->nr_requests = nr_requests; + p->token = token; + + if (pthread_create(&p->pthread_id, NULL, hammer_thread_proc, p)) { + warning("failed to create thread[%d] skipping remainder", k); + nr_threads = k; + break; + } + } + + for (k = 0; k < nr_threads; k++) { + struct hammer_thread_data *p = &data[k]; + + if (pthread_join(p->pthread_id, NULL)) + sum_join_errors++; + sum_commands += p->sum_successful; + sum_errors += p->sum_errors; + } + + fprintf(stderr, "HAMMER: [threads %d][requests %d] [ok %d][err %d][join %d]\n", + nr_threads, nr_requests, sum_commands, sum_errors, sum_join_errors); + + free(data); + + /* + * TODO Decide if/when to return an error or call die(). + */ + return 0; +} + int cmd__fsmonitor_client(int argc, const char **argv) { const char *subcmd; const char *token = NULL; + int nr_threads = 1; + int nr_requests = 1; const char * const fsmonitor_client_usage[] = { N_("test-helper fsmonitor-client query []"), N_("test-helper fsmonitor-client flush"), + N_("test-helper fsmonitor-client hammer [] [] []"), NULL, }; struct option options[] = { OPT_STRING(0, "token", &token, N_("token"), N_("command token to send to the server")), + + OPT_INTEGER(0, "threads", &nr_threads, N_("number of client threads")), + OPT_INTEGER(0, "requests", &nr_requests, N_("number of requests per thread")), + OPT_END() }; @@ -116,6 +218,9 @@ int cmd__fsmonitor_client(int argc, const char **argv) if (!strcmp(subcmd, "flush")) return !!do_send_flush(); + if (!strcmp(subcmd, "hammer")) + return !!do_hammer(token, nr_threads, nr_requests); + die("Unhandled subcommand: '%s'", subcmd); } #endif From patchwork Tue Feb 15 15:59:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff Hostetler X-Patchwork-Id: 12747261 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3545AC433FE for ; Tue, 15 Feb 2022 15:59:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241131AbiBOQAB (ORCPT ); Tue, 15 Feb 2022 11:00:01 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:56068 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241085AbiBOP7y (ORCPT ); Tue, 15 Feb 2022 10:59:54 -0500 Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com [IPv6:2a00:1450:4864:20::42b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D9C0CBDE4E for ; Tue, 15 Feb 2022 07:59:42 -0800 (PST) Received: by mail-wr1-x42b.google.com with SMTP id v12so32949656wrv.2 for ; Tue, 15 Feb 2022 07:59:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=zRljM25kO7hXvbxx8zIAQyZPmbHLjyw9d9EA8U8Zs3g=; b=TMrZx3EmKC1ALXwZZnZYoU3C+m0ItdBDOziHLVrgQSIA60Qf4mFJtbqkgTOVLkKex4 PxQexj1RJlgXtXEES2eUOmhoVJRTxzzzy+bQCERQVEXL6bnFz3a3oa3P+FA6fM0N6K1A H8aTJVIHQAlmqbzMiMoUe0itwBcWEIlbPf7x4nnlDq/AOh+y6b51FSq7hYjWGA27DVeX CRlZOqFSDY8EcUqbO6ArbpbF9AHAW/1cRCH2xJsraKeUUdTo9p0AXrWzdWQW54bus91a Zgu+3yNkM/M7ncwCYW0vR7HbL2wYAZSDv+mMsBBMC/m+LMKylwA1DpZtaHg6UlqanTnz p2ww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=zRljM25kO7hXvbxx8zIAQyZPmbHLjyw9d9EA8U8Zs3g=; b=5p/optac3boKhoytyeJvhXXx92uxzLvpK3M/wUdw1elSygAV03G+NzDPjyFuJDQDHQ rWfJ1FL1MxK+LgeOCKqjJrfp8j1Cm8fyDkGCvZH9WsP2mup8dqYSn86Ku75klf6cHB8r HwbPxIxnl2jDX3twcwnbCR0M4WF6hvDPoeVpDlCIutcfVDPokwGqyHxmqbesJ6ImB1bK wrO9hhByYnr1L+aW0/1/71jTc3Q+6zPiAi/0Z0Ih5kCvih3Q/33T2wwmGjSmHtT+gDAc ozCi5MhvMa9lz0c6YejO0nKjJRbINSL0A2ALubsTdiuA/Lql5i4tkCDgDTcQVhcngAtn 49oA== X-Gm-Message-State: AOAM533v3W5IErC+gcKBE+YpHDrciFGpiqMayWhF662vRMgGKXXf15VV NYKiqBflEhIwuhLDXmffwRypfeO1Fhs= X-Google-Smtp-Source: ABdhPJxyRYKbIAL0YxW/Ae5sX+5VKE66NW5Fdy7HaiyIK1rLAANUmTPwu+pSupu9OrBLZYAj1/Jsrg== X-Received: by 2002:a05:6000:1542:: with SMTP id 2mr3827524wry.607.1644940781253; Tue, 15 Feb 2022 07:59:41 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id bd20sm1524793wmb.20.2022.02.15.07.59.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Feb 2022 07:59:40 -0800 (PST) Message-Id: <44cc61e186cb65fa6b2c1d5a0f080fc0b2265e57.1644940773.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 15 Feb 2022 15:59:15 +0000 Subject: [PATCH 05/23] fsmonitor-settings: bare repos are incompatible with FSMonitor Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Jeff Hostetler , Jeff Hostetler Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Jeff Hostetler From: Jeff Hostetler Bare repos do not have a worktree, so there is nothing for the daemon watch. Signed-off-by: Jeff Hostetler --- builtin/fsmonitor--daemon.c | 11 +++++++ builtin/update-index.c | 9 ++++++ fsmonitor-settings.c | 62 +++++++++++++++++++++++++++++++++++++ fsmonitor-settings.h | 11 +++++++ t/t7519-status-fsmonitor.sh | 26 ++++++++++++++++ 5 files changed, 119 insertions(+) diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c index 6011fe42ee0..899355c55aa 100644 --- a/builtin/fsmonitor--daemon.c +++ b/builtin/fsmonitor--daemon.c @@ -1424,6 +1424,17 @@ int cmd_fsmonitor__daemon(int argc, const char **argv, const char *prefix) die(_("invalid 'ipc-threads' value (%d)"), fsmonitor__ipc_threads); + prepare_repo_settings(the_repository); + fsm_settings__set_ipc(the_repository); + + if (fsm_settings__get_mode(the_repository) == FSMONITOR_MODE_INCOMPATIBLE) { + struct strbuf buf_reason = STRBUF_INIT; + fsm_settings__get_reason(the_repository, &buf_reason); + error("%s '%s'", buf_reason.buf, xgetcwd()); + strbuf_release(&buf_reason); + return -1; + } + if (!strcmp(subcmd, "start")) return !!try_to_start_background_daemon(); diff --git a/builtin/update-index.c b/builtin/update-index.c index 1232f0d214e..9a2c45f2e56 100644 --- a/builtin/update-index.c +++ b/builtin/update-index.c @@ -1215,6 +1215,15 @@ int cmd_update_index(int argc, const char **argv, const char *prefix) if (fsmonitor > 0) { enum fsmonitor_mode fsm_mode = fsm_settings__get_mode(r); + + if (fsm_mode == FSMONITOR_MODE_INCOMPATIBLE) { + struct strbuf buf_reason = STRBUF_INIT; + fsm_settings__get_reason(r, &buf_reason); + error("%s", buf_reason.buf); + strbuf_release(&buf_reason); + return -1; + } + if (fsm_mode == FSMONITOR_MODE_DISABLED) { advise(_("core.fsmonitor is unset; " "set it if you really want to " diff --git a/fsmonitor-settings.c b/fsmonitor-settings.c index eb4d661c81e..0fc5566eb8a 100644 --- a/fsmonitor-settings.c +++ b/fsmonitor-settings.c @@ -9,9 +9,33 @@ */ struct fsmonitor_settings { enum fsmonitor_mode mode; + enum fsmonitor_reason reason; char *hook_path; }; +static void set_incompatible(struct repository *r, + enum fsmonitor_reason reason) +{ + struct fsmonitor_settings *s = r->settings.fsmonitor; + + s->mode = FSMONITOR_MODE_INCOMPATIBLE; + s->reason = reason; +} + +static int check_for_incompatible(struct repository *r) +{ + if (!r->worktree) { + /* + * Bare repositories don't have a working directory and + * therefore have nothing to watch. + */ + set_incompatible(r, FSMONITOR_REASON_BARE); + return 1; + } + + return 0; +} + static void lookup_fsmonitor_settings(struct repository *r) { struct fsmonitor_settings *s; @@ -78,6 +102,9 @@ void fsm_settings__set_ipc(struct repository *r) { lookup_fsmonitor_settings(r); + if (check_for_incompatible(r)) + return; + r->settings.fsmonitor->mode = FSMONITOR_MODE_IPC; FREE_AND_NULL(r->settings.fsmonitor->hook_path); } @@ -86,6 +113,9 @@ void fsm_settings__set_hook(struct repository *r, const char *path) { lookup_fsmonitor_settings(r); + if (check_for_incompatible(r)) + return; + r->settings.fsmonitor->mode = FSMONITOR_MODE_HOOK; FREE_AND_NULL(r->settings.fsmonitor->hook_path); r->settings.fsmonitor->hook_path = strdup(path); @@ -96,5 +126,37 @@ void fsm_settings__set_disabled(struct repository *r) lookup_fsmonitor_settings(r); r->settings.fsmonitor->mode = FSMONITOR_MODE_DISABLED; + r->settings.fsmonitor->reason = FSMONITOR_REASON_ZERO; FREE_AND_NULL(r->settings.fsmonitor->hook_path); } + +static void create_reason_message(struct repository *r, + struct strbuf *buf_reason) +{ + struct fsmonitor_settings *s = r->settings.fsmonitor; + + switch (s->reason) { + case FSMONITOR_REASON_ZERO: + return; + + case FSMONITOR_REASON_BARE: + strbuf_addstr(buf_reason, + _("bare repos are incompatible with fsmonitor")); + return; + + default: + BUG("Unhandled case in create_reason_message '%d'", s->reason); + } +} + +enum fsmonitor_reason fsm_settings__get_reason(struct repository *r, + struct strbuf *buf_reason) +{ + lookup_fsmonitor_settings(r); + + strbuf_reset(buf_reason); + if (r->settings.fsmonitor->mode == FSMONITOR_MODE_INCOMPATIBLE) + create_reason_message(r, buf_reason); + + return r->settings.fsmonitor->reason; +} diff --git a/fsmonitor-settings.h b/fsmonitor-settings.h index a4c5d7b4889..e5f145a2f79 100644 --- a/fsmonitor-settings.h +++ b/fsmonitor-settings.h @@ -4,17 +4,28 @@ struct repository; enum fsmonitor_mode { + FSMONITOR_MODE_INCOMPATIBLE = -1, /* see _reason */ FSMONITOR_MODE_DISABLED = 0, FSMONITOR_MODE_HOOK = 1, /* core.fsmonitor= */ FSMONITOR_MODE_IPC = 2, /* core.fsmonitor= */ }; +/* + * Incompatibility reasons. + */ +enum fsmonitor_reason { + FSMONITOR_REASON_ZERO = 0, + FSMONITOR_REASON_BARE = 1, +}; + void fsm_settings__set_ipc(struct repository *r); void fsm_settings__set_hook(struct repository *r, const char *path); void fsm_settings__set_disabled(struct repository *r); enum fsmonitor_mode fsm_settings__get_mode(struct repository *r); const char *fsm_settings__get_hook_path(struct repository *r); +enum fsmonitor_reason fsm_settings__get_reason(struct repository *r, + struct strbuf *buf_reason); struct fsmonitor_settings; diff --git a/t/t7519-status-fsmonitor.sh b/t/t7519-status-fsmonitor.sh index f488d930dfd..3c4e6f5f89c 100755 --- a/t/t7519-status-fsmonitor.sh +++ b/t/t7519-status-fsmonitor.sh @@ -55,6 +55,32 @@ test_lazy_prereq UNTRACKED_CACHE ' test $ret -ne 1 ' +# Test that we detect and disallow repos that are incompatible with FSMonitor. +test_expect_success 'incompatible bare repo' ' + test_when_finished "rm -rf ./bare-clone actual expect" && + git init --bare bare-clone && + cat >expect <<-\EOF && + error: bare repos are incompatible with fsmonitor + EOF + + test_must_fail \ + git -C ./bare-clone -c core.fsmonitor=foo \ + update-index --fsmonitor 2>actual && + test_cmp expect actual && + + test_must_fail \ + git -C ./bare-clone -c core.fsmonitor=true \ + update-index --fsmonitor 2>actual && + test_cmp expect actual +' + +test_expect_success FSMONITOR_DAEMON 'run fsmonitor-daemon in bare repo' ' + test_when_finished "rm -rf ./bare-clone actual" && + git init --bare bare-clone && + test_must_fail git -C ./bare-clone fsmonitor--daemon run 2>actual && + grep "bare repos are incompatible with fsmonitor" actual +' + test_expect_success 'setup' ' mkdir -p .git/hooks && : >tracked && From patchwork Tue Feb 15 15:59:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff Hostetler X-Patchwork-Id: 12747260 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AECCAC433F5 for ; Tue, 15 Feb 2022 15:59:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241129AbiBOP77 (ORCPT ); Tue, 15 Feb 2022 10:59:59 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:56070 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241072AbiBOP7y (ORCPT ); Tue, 15 Feb 2022 10:59:54 -0500 Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D5555BDE59 for ; Tue, 15 Feb 2022 07:59:43 -0800 (PST) Received: by mail-wr1-x429.google.com with SMTP id d27so29119671wrb.5 for ; Tue, 15 Feb 2022 07:59:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=ql3BwyKnQQmRB4gTQ2bCfGNXZ0vsfhL/9Gg63XIFLq8=; b=Dg5yFaJNcJ7FJjlk6qFUpYSdMWfzdXn+8QWHLYWla37GqyQ/whkUmyn6YwAQuVdhyx ag2YsjbarLffxanrU4fKjxG6n/QvQf4rp8XEMRwJMKC3NeFZGFpLEuKHBwBX+SkDPAfy tm7accS/1nqKjlS5+xGaWtSWjQpcQVzIuu2CTSiJ4izodNIttP4VBntq0K9RXlIueIHu Ch0RFOtGP8uV13MA9aFqWsU/Za0l5HZNFDwYg9Ic8nHZ9TO76waV5kJJPlfPi/BtDQ3P Bs08aIOyL0k+xr6mTVer6XFee5Sa91drqfnjOxngdSCQVbIm0ofBMOPbp1AS6EibBgC/ p8Vw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=ql3BwyKnQQmRB4gTQ2bCfGNXZ0vsfhL/9Gg63XIFLq8=; b=vNoR20wQc2R4FZFDXbGQSsJzaSHkr9nfih4N/009o0iBdSotud9dZy1W5IJudYel7F VsO+lmDfJNCnz9KTZecvl8TZMtv8YZ5oEbGZbctz4VaE6bT6bOLTO1zBMY8dvzcYaCvJ wByAmDWh1kU8hljqWR9zlBMxXwP/gkigjgriN5OKoyTvA+krS+ifBxqbKvw5Tc96bDI+ XA2cWlM/wRdwj3A++LruvKfi3mksqduxbiWZaDnT812mHFUOKiIcmV34oN4jkIjsxRaN NVtH6pqBvq6jHQ8RyPdurm/y4NkIyeg4obPCjkXGuLjYl9foKlOnXXhAeuGEUZe2lVj8 HW/w== X-Gm-Message-State: AOAM532xRksmN1sspP8M4GJikNlquXpEuhWIYHnTTHC3VZgJbOgUlzAo tS9Zwv7pm2UykKKy0EvE/+ZRAtFYeUA= X-Google-Smtp-Source: ABdhPJzdhqM19kGMJQGNfBfqKIaOuI1dVDg1Ktx1QMO+MDZUuJgGa148CFrt0bRvQAaIvjNTDLzK3w== X-Received: by 2002:a5d:6681:: with SMTP id l1mr3513961wru.633.1644940782170; Tue, 15 Feb 2022 07:59:42 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id s7sm3990259wro.104.2022.02.15.07.59.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Feb 2022 07:59:41 -0800 (PST) Message-Id: <4715677f85ff3f47b437f82e2b82a8b5dd371c99.1644940773.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 15 Feb 2022 15:59:16 +0000 Subject: [PATCH 06/23] fsmonitor-settings: stub in platform-specific incompatibility checking Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Jeff Hostetler , Jeff Hostetler Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Jeff Hostetler From: Jeff Hostetler Extend generic incompatibility checkout with platform-specific mechanism. Stub in Win32 version. In the existing fsmonitor-settings code we have a way to mark types of repos as incompatible with fsmonitor (whether via the hook and ipc APIs). For example, we do this for bare repos, since there are no files to watch. Extend this exclusion mechanism for platfor-specific reasons. This commit just creates the framework and adds a stub for Win32. Signed-off-by: Jeff Hostetler --- Makefile | 13 +++++++++++++ compat/fsmonitor/fsm-settings-win32.c | 9 +++++++++ config.mak.uname | 4 ++++ contrib/buildsystems/CMakeLists.txt | 3 +++ fsmonitor-settings.c | 12 ++++++++++++ fsmonitor-settings.h | 13 +++++++++++++ 6 files changed, 54 insertions(+) create mode 100644 compat/fsmonitor/fsm-settings-win32.c diff --git a/Makefile b/Makefile index 5ad0cef69f4..ed3d5231ea1 100644 --- a/Makefile +++ b/Makefile @@ -470,6 +470,11 @@ all:: # `compat/fsmonitor/fsm-listen-.c` that implements the # `fsm_listen__*()` routines. # +# If your platform has os-specific ways to tell if a repo is incompatible with +# fsmonitor (whether the hook or ipc daemon version), set FSMONITOR_OS_SETTINGS +# to the "" of the corresponding `compat/fsmonitor/fsm-settings-.c` +# that implements the `fsm_os_settings__*()` routines. +# # Define DEVELOPER to enable more compiler warnings. Compiler version # and family are auto detected, but could be overridden by defining # COMPILER_FEATURES (see config.mak.dev). You can still set @@ -1941,6 +1946,11 @@ ifdef FSMONITOR_DAEMON_BACKEND COMPAT_OBJS += compat/fsmonitor/fsm-listen-$(FSMONITOR_DAEMON_BACKEND).o endif +ifdef FSMONITOR_OS_SETTINGS + COMPAT_CFLAGS += -DHAVE_FSMONITOR_OS_SETTINGS + COMPAT_OBJS += compat/fsmonitor/fsm-settings-$(FSMONITOR_OS_SETTINGS).o +endif + ifeq ($(TCLTK_PATH),) NO_TCLTK = NoThanks endif @@ -2819,6 +2829,9 @@ GIT-BUILD-OPTIONS: FORCE ifdef FSMONITOR_DAEMON_BACKEND @echo FSMONITOR_DAEMON_BACKEND=\''$(subst ','\'',$(subst ','\'',$(FSMONITOR_DAEMON_BACKEND)))'\' >>$@+ endif +ifdef FSMONITOR_OS_SETTINGS + @echo FSMONITOR_OS_SETTINGS=\''$(subst ','\'',$(subst ','\'',$(FSMONITOR_OS_SETTINGS)))'\' >>$@+ +endif ifdef TEST_OUTPUT_DIRECTORY @echo TEST_OUTPUT_DIRECTORY=\''$(subst ','\'',$(subst ','\'',$(TEST_OUTPUT_DIRECTORY)))'\' >>$@+ endif diff --git a/compat/fsmonitor/fsm-settings-win32.c b/compat/fsmonitor/fsm-settings-win32.c new file mode 100644 index 00000000000..176a6f5726c --- /dev/null +++ b/compat/fsmonitor/fsm-settings-win32.c @@ -0,0 +1,9 @@ +#include "cache.h" +#include "config.h" +#include "repository.h" +#include "fsmonitor-settings.h" + +enum fsmonitor_reason fsm_os__incompatible(struct repository *r) +{ + return FSMONITOR_REASON_ZERO; +} diff --git a/config.mak.uname b/config.mak.uname index 81f21d9e201..219a04b4c66 100644 --- a/config.mak.uname +++ b/config.mak.uname @@ -441,6 +441,8 @@ ifeq ($(uname_S),Windows) # These are always available, so we do not have to conditionally # support it. FSMONITOR_DAEMON_BACKEND = win32 + FSMONITOR_OS_SETTINGS = win32 + NO_SVN_TESTS = YesPlease RUNTIME_PREFIX = YesPlease HAVE_WPGMPTR = YesWeDo @@ -629,6 +631,8 @@ ifeq ($(uname_S),MINGW) # These are always available, so we do not have to conditionally # support it. FSMONITOR_DAEMON_BACKEND = win32 + FSMONITOR_OS_SETTINGS = win32 + RUNTIME_PREFIX = YesPlease HAVE_WPGMPTR = YesWeDo NO_ST_BLOCKS_IN_STRUCT_STAT = YesPlease diff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt index 41395705ac6..7b8ea41967b 100644 --- a/contrib/buildsystems/CMakeLists.txt +++ b/contrib/buildsystems/CMakeLists.txt @@ -289,6 +289,9 @@ if(SUPPORTS_SIMPLE_IPC) if(CMAKE_SYSTEM_NAME STREQUAL "Windows") add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-win32.c) + + add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-win32.c) elseif(CMAKE_SYSTEM_NAME STREQUAL "Darwin") add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-darwin.c) diff --git a/fsmonitor-settings.c b/fsmonitor-settings.c index 0fc5566eb8a..e445572354e 100644 --- a/fsmonitor-settings.c +++ b/fsmonitor-settings.c @@ -33,6 +33,18 @@ static int check_for_incompatible(struct repository *r) return 1; } +#ifdef HAVE_FSMONITOR_OS_SETTINGS + { + enum fsmonitor_reason reason; + + reason = fsm_os__incompatible(r); + if (reason != FSMONITOR_REASON_ZERO) { + set_incompatible(r, reason); + return 1; + } + } +#endif + return 0; } diff --git a/fsmonitor-settings.h b/fsmonitor-settings.h index e5f145a2f79..b52bf8edaf1 100644 --- a/fsmonitor-settings.h +++ b/fsmonitor-settings.h @@ -29,4 +29,17 @@ enum fsmonitor_reason fsm_settings__get_reason(struct repository *r, struct fsmonitor_settings; +#ifdef HAVE_FSMONITOR_OS_SETTINGS +/* + * Ask platform-specific code whether the repository is incompatible + * with fsmonitor (both hook and ipc modes). For example, if the working + * directory is on a remote volume and mounted via a technology that does + * not support notification events. + * + * fsm_os__* routines should considered private to fsm_settings__ + * routines. + */ +enum fsmonitor_reason fsm_os__incompatible(struct repository *r); +#endif /* HAVE_FSMONITOR_OS_SETTINGS */ + #endif /* FSMONITOR_SETTINGS_H */ From patchwork Tue Feb 15 15:59:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff Hostetler X-Patchwork-Id: 12747259 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9B941C433FE for ; Tue, 15 Feb 2022 15:59:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241122AbiBOP77 (ORCPT ); Tue, 15 Feb 2022 10:59:59 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:56072 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241088AbiBOP7y (ORCPT ); Tue, 15 Feb 2022 10:59:54 -0500 Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 95348BDE62 for ; Tue, 15 Feb 2022 07:59:44 -0800 (PST) Received: by mail-wr1-x432.google.com with SMTP id i14so32845370wrc.10 for ; Tue, 15 Feb 2022 07:59:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=rKlKildQJZ1+OIIunPCbX80u3V0q8AF3Opxe15VhUHs=; b=JtoQxUs9HMe7PnmlVaEfws8HTjdy3W/2nOutTZd7c4s3ZJl0Wf4fHegy55g1Z6YE6m RHjcYnnw+7XVGl3DGCEILpZ4mjPVDl3JwbeIed4KDfFmAkVF6JsPksuN+xdB4QlaxH7D 83p2xtj6C/lDuC7TO3G3FR8RPRCRuxdkn1k3LgUq6B/ISnADOJr71tKGHu0P4wc8sZMT XnOdcb67V+ix7eT4NvT6AKAe5svgXgrMo+wRxNhzQ633QpJpz1gjS3cYHi9vAsNv4tl6 w1GAn+A7nPNsOjwChrbH7Wzo4hpCxXRjHouotFGHZyTOi5+KdoMAsnISzXoXcHA3mHA1 6STw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=rKlKildQJZ1+OIIunPCbX80u3V0q8AF3Opxe15VhUHs=; b=osbYTC4rxzAnBhRp83lru54PT4MEioqea8718XGZw11YbuDI+A7OqMvV3CpTiYmK7U FdL0FjV/v7VdyBFSoaBZLBBoDBwHuWQ+u8zpO6sobqoKoZqyqQZOJj5AZfnoFhK6wrjk pDXIUvjVx4bzqFTY46NgHWcf0gqN0xb73V55lKy+pUk6Jd2r4i8kRR02z6rFE6q5hiIl ++m/RqZYTi3GliVWGWwlPOnUkGIzb9Imx4tcZMaFQ41Jt3BkyvTdAy27FY4efJrP00My nWmoL+IPsjQXS5//MYbnJQfPfRrHSmNFKFuBkaXj9nzmwmK4oDZK43BtCbGH+nzgB4b4 RoCQ== X-Gm-Message-State: AOAM530wkF9G9Rv0N3q1dhiNLIgFzFecGOxBZnMyIgikYrzztF/fiqjv qBu2Jv7AIXVjEM1+cUCmMdaKbCMJs5o= X-Google-Smtp-Source: ABdhPJz8dfKuwyWyelRE9kXE413wkl9kdQjlSLYStNwx9L0RVAgqlCoWkXA3+EdVQrJKlAj12Qc0Tw== X-Received: by 2002:a5d:4c89:: with SMTP id z9mr3922987wrs.365.1644940783065; Tue, 15 Feb 2022 07:59:43 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id d14sm16623913wrv.50.2022.02.15.07.59.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Feb 2022 07:59:42 -0800 (PST) Message-Id: <4e856d60e385d64158f17ec1226f97eb323bc55e.1644940774.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 15 Feb 2022 15:59:17 +0000 Subject: [PATCH 07/23] fsmonitor-settings: virtual repos are incompatible with FSMonitor Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Jeff Hostetler , Jeff Hostetler Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Jeff Hostetler From: Jeff Hostetler Virtual repos, such as GVFS (aka VFS for Git), are incompatible with FSMonitor. Signed-off-by: Jeff Hostetler --- compat/fsmonitor/fsm-settings-win32.c | 26 ++++++++++++++++++++++++++ fsmonitor-settings.c | 5 +++++ fsmonitor-settings.h | 1 + t/t7519-status-fsmonitor.sh | 9 +++++++++ 4 files changed, 41 insertions(+) diff --git a/compat/fsmonitor/fsm-settings-win32.c b/compat/fsmonitor/fsm-settings-win32.c index 176a6f5726c..7caa79570af 100644 --- a/compat/fsmonitor/fsm-settings-win32.c +++ b/compat/fsmonitor/fsm-settings-win32.c @@ -3,7 +3,33 @@ #include "repository.h" #include "fsmonitor-settings.h" +/* + * GVFS (aka VFS for Git) is incompatible with FSMonitor. + * + * Granted, core Git does not know anything about GVFS and we + * shouldn't make assumptions about a downstream feature, but users + * can install both versions. And this can lead to incorrect results + * from core Git commands. So, without bringing in any of the GVFS + * code, do a simple config test for a published config setting. (We + * do not look at the various *_TEST_* environment variables.) + */ +static enum fsmonitor_reason is_virtual(struct repository *r) +{ + const char *const_str; + + if (!repo_config_get_value(r, "core.virtualfilesystem", &const_str)) + return FSMONITOR_REASON_VIRTUAL; + + return FSMONITOR_REASON_ZERO; +} + enum fsmonitor_reason fsm_os__incompatible(struct repository *r) { + enum fsmonitor_reason reason; + + reason = is_virtual(r); + if (reason) + return reason; + return FSMONITOR_REASON_ZERO; } diff --git a/fsmonitor-settings.c b/fsmonitor-settings.c index e445572354e..bb2ddd2457f 100644 --- a/fsmonitor-settings.c +++ b/fsmonitor-settings.c @@ -156,6 +156,11 @@ static void create_reason_message(struct repository *r, _("bare repos are incompatible with fsmonitor")); return; + case FSMONITOR_REASON_VIRTUAL: + strbuf_addstr(buf_reason, + _("virtual repos are incompatible with fsmonitor")); + return; + default: BUG("Unhandled case in create_reason_message '%d'", s->reason); } diff --git a/fsmonitor-settings.h b/fsmonitor-settings.h index b52bf8edaf1..c169683bf2d 100644 --- a/fsmonitor-settings.h +++ b/fsmonitor-settings.h @@ -16,6 +16,7 @@ enum fsmonitor_mode { enum fsmonitor_reason { FSMONITOR_REASON_ZERO = 0, FSMONITOR_REASON_BARE = 1, + FSMONITOR_REASON_VIRTUAL = 2, }; void fsm_settings__set_ipc(struct repository *r); diff --git a/t/t7519-status-fsmonitor.sh b/t/t7519-status-fsmonitor.sh index 3c4e6f5f89c..d7540931a16 100755 --- a/t/t7519-status-fsmonitor.sh +++ b/t/t7519-status-fsmonitor.sh @@ -81,6 +81,15 @@ test_expect_success FSMONITOR_DAEMON 'run fsmonitor-daemon in bare repo' ' grep "bare repos are incompatible with fsmonitor" actual ' +test_expect_success MINGW,FSMONITOR_DAEMON 'run fsmonitor-daemon in virtual repo' ' + test_when_finished "rm -rf ./fake-virtual-clone actual" && + git init fake-virtual-clone && + test_must_fail git -C ./fake-virtual-clone \ + -c core.virtualfilesystem=true \ + fsmonitor--daemon run 2>actual && + grep "virtual repos are incompatible with fsmonitor" actual +' + test_expect_success 'setup' ' mkdir -p .git/hooks && : >tracked && From patchwork Tue Feb 15 15:59:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff Hostetler X-Patchwork-Id: 12747262 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 12FC4C433EF for ; Tue, 15 Feb 2022 15:59:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241132AbiBOQAD (ORCPT ); Tue, 15 Feb 2022 11:00:03 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:56088 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241090AbiBOP7z (ORCPT ); Tue, 15 Feb 2022 10:59:55 -0500 Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8899BBD2FA for ; Tue, 15 Feb 2022 07:59:45 -0800 (PST) Received: by mail-wr1-x429.google.com with SMTP id e3so32988643wra.0 for ; Tue, 15 Feb 2022 07:59:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=BbYop8GOrn8UsIsEQsfMq536DDd+W/wsj9wQHKVyvIQ=; b=MaEnOtS0GnnYIHkyv4WXWIY1aLN9X4acO7SBf6xlaDhEivSskRV/N14AtFfqko1ruY W3rXShJx02Xv6YLcsy6zFYJEbdeVHwiMhEtxGdGihk+F9oj2ghTOAp1cNAGHxzrhbT26 KWAUjPtQ2xlS6dTcH7Qn+XVpdb6yzn6jv2Y55HIHG/YpuUP9oDMCSJFMmrYSHrCCpHTf E5hCPDGGkOt8Y0aGaDk86KQNINc9wVx2Aob7Z+Zcu3vdx+uUl6Bi3HdIC5B6C2Hf9Txd 3+p6cZu58yyIj9pfTgjWVIj5jFHiljOIAmV+7EWc0dR4w8FDfz4vFx/4AYb1h2v3cfC9 sGKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=BbYop8GOrn8UsIsEQsfMq536DDd+W/wsj9wQHKVyvIQ=; b=GhaFGVXQBFpmhOrjPWOS7LvnxpBNYKjrVhDhrjBssrbp+xLH6mAlW1/HtKrPX9atil exr3D9KMagGPxMb6xE9EaZPI+hNM2CA1jEWnvmMxQRlQ0OrCIj0ktDVyWGW6EaJsIzWW ZHbB5BBWo6UYU1kYdvlLHBvFy+/+RgN3tJF9BYqUH7KH6et+j/P96hc6zf1MHAMiPNrT u1V7oBTmV/waEbbwu/rNhTjpdDgqySNFaLnTBT3mYwDEIcGvBsdaPytvhRX4XIbnN+7O xXAxgH0UDZkRw2H8dA1BWtKOfKxYSo2Xca6YwxlzaFQ6Z1MkISKGqleqfThsg6AWR7A5 b4lQ== X-Gm-Message-State: AOAM530u1mzcrlG8qweit9ZIn9Q4qBxBkh5/H0oXu4P/yU/stUutMdS+ 589RbXnntFwshGMP21uVLw/WtenAeWA= X-Google-Smtp-Source: ABdhPJyFQchUWN4tKbrQywnhDXJVuVruyYR0ArNqsoXoMh+kOpfWCZ+n+oDG/KuXIzqK9N/6z2aEyQ== X-Received: by 2002:adf:d4cb:: with SMTP id w11mr3728624wrk.221.1644940784009; Tue, 15 Feb 2022 07:59:44 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id d14sm20657251wri.93.2022.02.15.07.59.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Feb 2022 07:59:43 -0800 (PST) Message-Id: In-Reply-To: References: Date: Tue, 15 Feb 2022 15:59:18 +0000 Subject: [PATCH 08/23] fsmonitor-settings: stub in macOS-specific incompatibility checking Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Jeff Hostetler , Jeff Hostetler Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Jeff Hostetler From: Jeff Hostetler Signed-off-by: Jeff Hostetler --- compat/fsmonitor/fsm-settings-darwin.c | 9 +++++++++ config.mak.uname | 1 + contrib/buildsystems/CMakeLists.txt | 3 +++ 3 files changed, 13 insertions(+) create mode 100644 compat/fsmonitor/fsm-settings-darwin.c diff --git a/compat/fsmonitor/fsm-settings-darwin.c b/compat/fsmonitor/fsm-settings-darwin.c new file mode 100644 index 00000000000..176a6f5726c --- /dev/null +++ b/compat/fsmonitor/fsm-settings-darwin.c @@ -0,0 +1,9 @@ +#include "cache.h" +#include "config.h" +#include "repository.h" +#include "fsmonitor-settings.h" + +enum fsmonitor_reason fsm_os__incompatible(struct repository *r) +{ + return FSMONITOR_REASON_ZERO; +} diff --git a/config.mak.uname b/config.mak.uname index 219a04b4c66..529d58aae5d 100644 --- a/config.mak.uname +++ b/config.mak.uname @@ -159,6 +159,7 @@ ifeq ($(uname_S),Darwin) ifndef NO_PTHREADS ifndef NO_UNIX_SOCKETS FSMONITOR_DAEMON_BACKEND = darwin + FSMONITOR_OS_SETTINGS = darwin endif endif diff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt index 7b8ea41967b..fc70dd2fc1d 100644 --- a/contrib/buildsystems/CMakeLists.txt +++ b/contrib/buildsystems/CMakeLists.txt @@ -295,6 +295,9 @@ if(SUPPORTS_SIMPLE_IPC) elseif(CMAKE_SYSTEM_NAME STREQUAL "Darwin") add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-darwin.c) + + add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-darwin.c) endif() endif() From patchwork Tue Feb 15 15:59:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff Hostetler X-Patchwork-Id: 12747263 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6BB92C433F5 for ; Tue, 15 Feb 2022 16:00:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241142AbiBOQAR (ORCPT ); Tue, 15 Feb 2022 11:00:17 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:56106 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241095AbiBOP75 (ORCPT ); Tue, 15 Feb 2022 10:59:57 -0500 Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6FADBBD2FE for ; Tue, 15 Feb 2022 07:59:46 -0800 (PST) Received: by mail-wr1-x429.google.com with SMTP id w11so32907727wra.4 for ; Tue, 15 Feb 2022 07:59:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=KA8mn4mhpWKDhLHwnGplr1Fa1mftVzp7hVz63ubKL28=; b=iLjQsWXWXbQv4nR2IzaQldWS6q5xCIGRfrIV+TWLLHN9kGpkxiCr7UyQ4Z/C3qCQmI g1M7syS5yTRyytv+ofnooQ+iyPJ3c3QVvauAUacbHIk1FtjCHZf/BRevQol9ceQlaEhP hTvgyLb3VyYZFDFkE77qcgOtCA+ufPn1BvMDPxewnPSR6o051zHS2ONxOyhOo4nCuuGh GFAq29WM+JzHbUpiipXWJm2DnK6+m1gc3deV8Jnsy+P21Csexdg8MKumswTsTOky/JCu ffp2XvEwdosovUHxoM0PCIQCwH27pThmm//bqG+KS7xtH/IIrejtebxdtfxyqKr3tDW1 s92w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=KA8mn4mhpWKDhLHwnGplr1Fa1mftVzp7hVz63ubKL28=; b=aMSph9W4oFueK5vKO1dqrUewy7VviBE2SASNmkU0BnEXe5iEzFzSuSjGAGQUOAf0aO +s9njyGpx+HK2AruVQIhwEe+XJAGZm7Czy5cRQEyLYJxCA+oOC1ootvyyuyh5UdtZryL VFWWeuXq1BjAGTuCGUu5nxIp7zD7fJlGu/BxJs0HbsepAEfSWrlU+Rk3qIWUXnMNKR/V 0df4ekWkX0PYETHgGNyJDhjK2yF+uiYDM8aEY/U3KLibpGU6iKu2xSJFwyC3ZLzqll4v r6m4ZMAxqaR26jdOdkoFPQ7fsMPU4/Lk/6jX81SBETUBSNWAiYr6Y1apYnht4Hcvr/cD 7uCQ== X-Gm-Message-State: AOAM5318bc35zO8V1rMBtuPvnzrqAW9m3G8cpqjf+pJYr/8gWybBZpEs ALyGJJ9IxNorKWYebHtxsMx5v0lp+H8= X-Google-Smtp-Source: ABdhPJxRXoFl+ONuQDW2jxl7QGZxOW3y8J44r08Unjg3sHD548lFtLrEYYKWJAWY/NQBM4Tq8/QVyQ== X-Received: by 2002:adf:e98f:: with SMTP id h15mr3749489wrm.93.1644940784800; Tue, 15 Feb 2022 07:59:44 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id bg26sm17399012wmb.48.2022.02.15.07.59.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Feb 2022 07:59:44 -0800 (PST) Message-Id: <412fbc45868e7f2a05a03e424584b53ded4842e9.1644940774.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 15 Feb 2022 15:59:19 +0000 Subject: [PATCH 09/23] fsmonitor-settings: remote repos on macOS are incompatible with FSMonitor Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Jeff Hostetler , Jeff Hostetler Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Jeff Hostetler From: Jeff Hostetler Teach Git to detect remote working directories on macOS and mark them as incompatible with FSMonitor. With this, `git fsmonitor--daemon run` will error out with a message like it does for bare repos. Client commands, like `git status`, will not attempt to start the daemon. Signed-off-by: Jeff Hostetler --- compat/fsmonitor/fsm-settings-darwin.c | 66 ++++++++++++++++++++++++++ fsmonitor-settings.c | 5 ++ fsmonitor-settings.h | 1 + 3 files changed, 72 insertions(+) diff --git a/compat/fsmonitor/fsm-settings-darwin.c b/compat/fsmonitor/fsm-settings-darwin.c index 176a6f5726c..c3b719d6fb0 100644 --- a/compat/fsmonitor/fsm-settings-darwin.c +++ b/compat/fsmonitor/fsm-settings-darwin.c @@ -2,8 +2,74 @@ #include "config.h" #include "repository.h" #include "fsmonitor-settings.h" +#include "fsmonitor.h" +#include +#include + +/* + * Remote working directories are problematic for FSMonitor. + * + * The underlying file system on the server machine and/or the remote + * mount type (NFS, SAMBA, etc.) dictates whether notification events + * are available at all to remote client machines. + * + * Kernel differences between the server and client machines also + * dictate the how (buffering, frequency, de-dup) the events are + * delivered to client machine processes. + * + * A client machine (such as a laptop) may choose to suspend/resume + * and it is unclear (without lots of testing) whether the watcher can + * resync after a resume. We might be able to treat this as a normal + * "events were dropped by the kernel" event and do our normal "flush + * and resync" --or-- we might need to close the existing (zombie?) + * notification fd and create a new one. + * + * In theory, the above issues need to be addressed whether we are + * using the Hook or IPC API. + * + * For the builtin FSMonitor, we create the Unix domain socket for the + * IPC in the .git directory. If the working directory is remote, + * then the socket will be created on the remote file system. This + * can fail if the remote file system does not support UDS file types + * (e.g. smbfs to a Windows server) or if the remote kernel does not + * allow a non-local process to bind() the socket. (These problems + * could be fixed by moving the UDS out of the .git directory and to a + * well-known local directory on the client machine, but care should + * be taken to ensure that $HOME is actually local and not a managed + * file share.) + * + * So (for now at least), mark remote working directories as + * incompatible. + */ +static enum fsmonitor_reason is_remote(struct repository *r) +{ + struct statfs fs; + + if (statfs(r->worktree, &fs) == -1) { + int saved_errno = errno; + trace_printf_key(&trace_fsmonitor, "statfs('%s') failed: %s", + r->worktree, strerror(saved_errno)); + errno = saved_errno; + return FSMONITOR_REASON_ZERO; + } + + trace_printf_key(&trace_fsmonitor, + "statfs('%s') [type 0x%08x][flags 0x%08x] '%s'", + r->worktree, fs.f_type, fs.f_flags, fs.f_fstypename); + + if (!(fs.f_flags & MNT_LOCAL)) + return FSMONITOR_REASON_REMOTE; + + return FSMONITOR_REASON_ZERO; +} enum fsmonitor_reason fsm_os__incompatible(struct repository *r) { + enum fsmonitor_reason reason; + + reason = is_remote(r); + if (reason) + return reason; + return FSMONITOR_REASON_ZERO; } diff --git a/fsmonitor-settings.c b/fsmonitor-settings.c index bb2ddd2457f..de69ace246a 100644 --- a/fsmonitor-settings.c +++ b/fsmonitor-settings.c @@ -161,6 +161,11 @@ static void create_reason_message(struct repository *r, _("virtual repos are incompatible with fsmonitor")); return; + case FSMONITOR_REASON_REMOTE: + strbuf_addstr(buf_reason, + _("remote repos are incompatible with fsmonitor")); + return; + default: BUG("Unhandled case in create_reason_message '%d'", s->reason); } diff --git a/fsmonitor-settings.h b/fsmonitor-settings.h index c169683bf2d..fca25887c0f 100644 --- a/fsmonitor-settings.h +++ b/fsmonitor-settings.h @@ -17,6 +17,7 @@ enum fsmonitor_reason { FSMONITOR_REASON_ZERO = 0, FSMONITOR_REASON_BARE = 1, FSMONITOR_REASON_VIRTUAL = 2, + FSMONITOR_REASON_REMOTE = 3, }; void fsm_settings__set_ipc(struct repository *r); From patchwork Tue Feb 15 15:59:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff Hostetler X-Patchwork-Id: 12747264 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 30AB2C433FE for ; Tue, 15 Feb 2022 16:00:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238291AbiBOQAV (ORCPT ); Tue, 15 Feb 2022 11:00:21 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:56110 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241096AbiBOP75 (ORCPT ); Tue, 15 Feb 2022 10:59:57 -0500 Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com [IPv6:2a00:1450:4864:20::333]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3827CBD8B4 for ; Tue, 15 Feb 2022 07:59:47 -0800 (PST) Received: by mail-wm1-x333.google.com with SMTP id y6-20020a7bc186000000b0037bdc5a531eso1561354wmi.0 for ; Tue, 15 Feb 2022 07:59:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=EE1kU+RonBw911SS5R75P4cGS7NU2tc6PRM8jsMPKG8=; b=kvCLeEEdYYl/Tf2U57F4VKLnLyFbwGFB05i9zTSYzopC7h7l+VyVWDsR26JAVjFV4z vt+wH5RCuiYdf1qKTovJJ1pR4xoQFcJ8ljlcJRrmCUexeem6z9pmW8P6+byKHInc8ciy Y+mO+7fL//0k19TXJXCDITtH3EbnBFHaPIFL03uVvfmjcG/K0wDUorDhwKBTUQwdbc/P bsBWXQVEGzy03Z+EjJVj13C/GWCXJTdmFpA/AhIsuAnr/wnm6/4EUpArWXm/XH7UyvgN 7d5K/V9Adm+DULqkbjqjKRLtwgOTNUXH6n62WZABx9iV40S4+SF/dh3Ivd4Q7It5GZYz 7COw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=EE1kU+RonBw911SS5R75P4cGS7NU2tc6PRM8jsMPKG8=; b=O+l3mX1+pLuLdtjtVl9w3ZLy388yLtQ8YdZTNjyzMMzdMvdsG9b55Js3ONvZOTmBXI pKc4c7JpL92X5pBdY8hesT5fNjlkHXVvt8gbq8XDWiFPeQnjbWPsH2dHhP2fl81Eh8Uo qxmutE/8te9oPJe+pYSCrVVYejrh4JjRXSLLUsRIRNPTn7WG080lSUkFHE9smflLEvcG qsP4HUJDP0y7KKu/q+gTnnq5G7zdgSwK8cRotW5FmHJrqSq1jEhpzPE8RwEDhc+l0tHd sfsg7757Ed5FLIgrUxRR0f9OzujcIJTKe4RhAdCwZAatZkPEPDS/xAn9cJ/JpZxT8hjC QEoQ== X-Gm-Message-State: AOAM532GADzpAnwBOkBdaxDjteyzOfteV74/q1tszvaXPvmnHhAOrdNs hnLz5+Fyvsn0Q0flqOiBnSwyotDrMh8= X-Google-Smtp-Source: ABdhPJzyKYBte6L9s+GMKAukn/q7+9Ff8Y17kuBgO0ZqRJa/7waEwxUgXKsz9m4XM8dH74MbY8BQEg== X-Received: by 2002:a05:600c:3795:: with SMTP id o21mr3572744wmr.86.1644940785613; Tue, 15 Feb 2022 07:59:45 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id y6sm8719565wrd.30.2022.02.15.07.59.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Feb 2022 07:59:45 -0800 (PST) Message-Id: In-Reply-To: References: Date: Tue, 15 Feb 2022 15:59:20 +0000 Subject: [PATCH 10/23] fsmonitor-settings: remote repos on Windows are incompatible with FSMonitor Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Jeff Hostetler , Jeff Hostetler Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Jeff Hostetler From: Jeff Hostetler Teach Git to detect remote working directories on Windows and mark them as incompatible with FSMonitor. With this `git fsmonitor--daemon run` will error out with a message like it does for bare repos. Client commands, such as `git status`, will not attempt to start the daemon. Signed-off-by: Jeff Hostetler --- compat/fsmonitor/fsm-settings-win32.c | 102 ++++++++++++++++++++++++++ 1 file changed, 102 insertions(+) diff --git a/compat/fsmonitor/fsm-settings-win32.c b/compat/fsmonitor/fsm-settings-win32.c index 7caa79570af..77156c0ca20 100644 --- a/compat/fsmonitor/fsm-settings-win32.c +++ b/compat/fsmonitor/fsm-settings-win32.c @@ -2,6 +2,7 @@ #include "config.h" #include "repository.h" #include "fsmonitor-settings.h" +#include "fsmonitor.h" /* * GVFS (aka VFS for Git) is incompatible with FSMonitor. @@ -23,6 +24,103 @@ static enum fsmonitor_reason is_virtual(struct repository *r) return FSMONITOR_REASON_ZERO; } +/* + * Remote working directories are problematic for FSMonitor. + * + * The underlying file system on the server machine and/or the remote + * mount type dictates whether notification events are available at + * all to remote client machines. + * + * Kernel differences between the server and client machines also + * dictate the how (buffering, frequency, de-dup) the events are + * delivered to client machine processes. + * + * A client machine (such as a laptop) may choose to suspend/resume + * and it is unclear (without lots of testing) whether the watcher can + * resync after a resume. We might be able to treat this as a normal + * "events were dropped by the kernel" event and do our normal "flush + * and resync" --or-- we might need to close the existing (zombie?) + * notification fd and create a new one. + * + * In theory, the above issues need to be addressed whether we are + * using the Hook or IPC API. + * + * So (for now at least), mark remote working directories as + * incompatible. + * + * Notes for testing: + * + * (a) Windows allows a network share to be mapped to a drive letter. + * (This is the normal method to access it.) + * + * $ NET USE Z: \\server\share + * $ git -C Z:/repo status + * + * (b) Windows allows a network share to be referenced WITHOUT mapping + * it to drive letter. + * + * $ NET USE \\server\share\dir + * $ git -C //server/share/repo status + * + * (c) Windows allows "SUBST" to create a fake drive mapping to an + * arbitrary path (which may be remote) + * + * $ SUBST Q: Z:\repo + * $ git -C Q:/ status + * + * (d) Windows allows a directory symlink to be created on a local + * file system that points to a remote repo. + * + * $ mklink /d ./link //server/share/repo + * $ git -C ./link status + */ +static enum fsmonitor_reason is_remote(struct repository *r) +{ + wchar_t wpath[MAX_PATH]; + wchar_t wfullpath[MAX_PATH]; + size_t wlen; + UINT driveType; + + /* + * Do everything in wide chars because the drive letter might be + * a multi-byte sequence. See win32_has_dos_drive_prefix(). + */ + if (xutftowcs_path(wpath, r->worktree) < 0) + return FSMONITOR_REASON_ZERO; + + /* + * GetDriveTypeW() requires a final slash. We assume that the + * worktree pathname points to an actual directory. + */ + wlen = wcslen(wpath); + if (wpath[wlen - 1] != L'\\' && wpath[wlen - 1] != L'/') { + wpath[wlen++] = L'\\'; + wpath[wlen] = 0; + } + + /* + * Normalize the path. If nothing else, this converts forward + * slashes to backslashes. This is essential to get GetDriveTypeW() + * correctly handle some UNC "\\server\share\..." paths. + */ + if (!GetFullPathNameW(wpath, MAX_PATH, wfullpath, NULL)) + return FSMONITOR_REASON_ZERO; + + driveType = GetDriveTypeW(wfullpath); + trace_printf_key(&trace_fsmonitor, + "DriveType '%s' L'%ls' (%u)", + r->worktree, wfullpath, driveType); + + if (driveType == DRIVE_REMOTE) { + trace_printf_key(&trace_fsmonitor, + "is_remote('%s') true", + r->worktree); + return FSMONITOR_REASON_REMOTE; + } + + return FSMONITOR_REASON_ZERO; +} + enum fsmonitor_reason fsm_os__incompatible(struct repository *r) { enum fsmonitor_reason reason; @@ -31,5 +129,9 @@ enum fsmonitor_reason fsm_os__incompatible(struct repository *r) if (reason) return reason; + reason = is_remote(r); + if (reason) + return reason; + return FSMONITOR_REASON_ZERO; } From patchwork Tue Feb 15 15:59:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff Hostetler X-Patchwork-Id: 12747266 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7A3C5C433F5 for ; Tue, 15 Feb 2022 16:00:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241161AbiBOQAa (ORCPT ); Tue, 15 Feb 2022 11:00:30 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:56126 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241109AbiBOP76 (ORCPT ); Tue, 15 Feb 2022 10:59:58 -0500 Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com [IPv6:2a00:1450:4864:20::42e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EB1D6BDE4C for ; Tue, 15 Feb 2022 07:59:47 -0800 (PST) Received: by mail-wr1-x42e.google.com with SMTP id h6so32894576wrb.9 for ; Tue, 15 Feb 2022 07:59:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=4j62eB1Kl2emaHdog2W10LxWoiaSgrRC86C2r+OLHCg=; b=WN/M7o8nakMHmAlgJh0y+LR+HUc4iCDQ4psiNjVR2Eb2HZFtyFKIITdM9nKTEhWZkz vVND1stjP2awv2FwYpNB1r9JmEwq1W/5+YwO6DeKlQ5AYRZd32KgrFxOMTRRPlqLX4IV 191E2n8LXoZdWtkgNLTv/z+DuvSQavsXfiW/xlOvzpoyCebF+ZIGB3ua+tfuXvSYNBXx fjKx230tvI2xDTZnSTlqjhevGyn5qYdfLE3qwSv421ErX2433CPw2h1CM4mZISZtRPBw vYuM/K/lIufdkIZ9OMpFxjcHFxdVynkBwNVfGfBnMq3HCloEPW3C8ND80ecXpfrIZhcY 1Y6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=4j62eB1Kl2emaHdog2W10LxWoiaSgrRC86C2r+OLHCg=; b=BB77RlC/1iET7glusmRtLL6z1o18Xj8Tb1vbyAZHkINRFvW9iQBj5gKmrM4DP07c7w eQj3Zoexlkgev0hfhF8eF089keMbKy3dgq9o8aEP1IOCOnpPxzRXWYATHnoTXu00ida7 sI4EgFGqrDAdkha75/SokVSCWGC8OqvRFma6vIf+8TLGVJUsQbOou5gR1PUXMJtzW1iG z8q2VRlkrMqsW0FFVBYFaUA0BLTchfEDAhJ9OzBLFHh82tqUbBYaW7YPI/ps+nBxF4fz ZoByK4v50I1w+jYKJTfz9O/ozN5U8IgTTrzCi+e7tpxsobdZSa/RepljDbM5gJ+/DNLk HUjw== X-Gm-Message-State: AOAM533cgsjjuGwdObdTP9kERQS41WLAwQMfRwVe/n/s96W0+dz+DNxJ Bd8yLu+gIu67i/6wHAGLeI26O2glqRY= X-Google-Smtp-Source: ABdhPJxnYFo0qSQDuvqbiL7fxwz+cPeVL8DIdhjER/uq5wcOWLMb8uJx99whkwLJ+mBRC+j/v1n1FA== X-Received: by 2002:adf:d1c6:: with SMTP id b6mr3633233wrd.320.1644940786387; Tue, 15 Feb 2022 07:59:46 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id l11sm22004333wry.77.2022.02.15.07.59.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Feb 2022 07:59:45 -0800 (PST) Message-Id: In-Reply-To: References: Date: Tue, 15 Feb 2022 15:59:21 +0000 Subject: [PATCH 11/23] unpack-trees: initialize fsmonitor_has_run_once in o->result Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Jeff Hostetler , Jeff Hostetler Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Jeff Hostetler From: Jeff Hostetler Initialize `o->result.fsmonitor_has_run_once` based upon value in `o->src_index->fsmonitor_has_run_once` to prevent a second fsmonitor query during the tree traversal and possibly getting a skewed view of the working directory. The checkout code has already talked to the fsmonitor and the traversal is updating the index as it traverses, so there is no need to query the fsmonitor. Signed-off-by: Jeff Hostetler --- unpack-trees.c | 1 + 1 file changed, 1 insertion(+) diff --git a/unpack-trees.c b/unpack-trees.c index a7e1712d236..13e2e5de82c 100644 --- a/unpack-trees.c +++ b/unpack-trees.c @@ -1756,6 +1756,7 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options o->result.fsmonitor_last_update = xstrdup_or_null(o->src_index->fsmonitor_last_update); + o->result.fsmonitor_has_run_once = o->src_index->fsmonitor_has_run_once; /* * Sparse checkout loop #1: set NEW_SKIP_WORKTREE on existing entries From patchwork Tue Feb 15 15:59:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff Hostetler X-Patchwork-Id: 12747265 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 156BDC433EF for ; Tue, 15 Feb 2022 16:00:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241090AbiBOQAb (ORCPT ); Tue, 15 Feb 2022 11:00:31 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:56136 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241119AbiBOP77 (ORCPT ); Tue, 15 Feb 2022 10:59:59 -0500 Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com [IPv6:2a00:1450:4864:20::335]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D42C8BDE59 for ; Tue, 15 Feb 2022 07:59:48 -0800 (PST) Received: by mail-wm1-x335.google.com with SMTP id q198-20020a1ca7cf000000b0037bb52545c6so1812650wme.1 for ; Tue, 15 Feb 2022 07:59:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=SOxTtAs3wZOBGCxCwnBp9XRj5HCeVuYdhbP7LLvdJEw=; b=o/uH7BGY8F4zccmIk/lHijGezim/51aXn/KT5yqcBC/hAyjNDd1bJBNOxUYZvbzjiR lSTlxjec7/gPIAmLZ4pLDNhtLbGrbYZGacKNGi7MMYCfo9KBvAHRTJnKywqDvZ13T04J V0YsXeSHSUt35QSy7CTjLR6dj5C89OHNwyOMFmPCFMGxQJPxGPapfJ6BuqkO/cQZkDZt LRl5MuuCG568bIjF0GmGzvqFmLN00+ix1bYQthnrCWQdttid5m96vpy02dd1zd/ug49y LGvXGJIE24OHtI9ft4rqdgIHM6z6fgCnmtEz7ot+InqMVRkvtnCFIGrCKdUP1VmvihhW zTiQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=SOxTtAs3wZOBGCxCwnBp9XRj5HCeVuYdhbP7LLvdJEw=; b=eqxeBcCqLvLXJuRs+NN70GQf2j3fkaQOwL9hj2prEOi0MZTFpfkhflm7xNIrjJQe62 uTu/3kR+la5p39IK05Zp19fku7EAwflUpaH2Z3WNA0l95sSnVwuM1jE+n0cNCdtc4ogt NgUEWQPrB81j2iLAy7el4eaTgh+Vo//UE6yA+fFyd3wL26TP+ETcrPyNdjivtCJDUlpo bWQsk6w3keniFGk4mnOA8P8ZWw60l2SAmIJBnaXXEqwvinces/NpMtUlWE09krmsgiK9 jeSQmtOKgOGoVlzSsMPbZaXZSOiWuQjWfpH9hzwnr07m+eDR434mRl01LywGIPjm4gZI Cdvg== X-Gm-Message-State: AOAM531VEx5ivELQmI6cDPB5j5Ve3YXOptjQf3jf7IO0OGFJI2Cxf0xw SngGYJP2GNKlRGJv7ZcCDnnSgKPtOqA= X-Google-Smtp-Source: ABdhPJwuVcB3H3riz2p0vEbWEgDr990gd3NJnYjgqe4/T9ZYcpHwbe2Dt+debIjN5mXB+d60jPbqjg== X-Received: by 2002:a05:600c:1f05:b0:37b:d9aa:e210 with SMTP id bd5-20020a05600c1f0500b0037bd9aae210mr3640884wmb.63.1644940787244; Tue, 15 Feb 2022 07:59:47 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id g22sm15573392wmh.7.2022.02.15.07.59.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Feb 2022 07:59:46 -0800 (PST) Message-Id: <71babe7243a312c024e6bc6f7c54357cac12c9b1.1644940774.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 15 Feb 2022 15:59:22 +0000 Subject: [PATCH 12/23] fsm-listen-darwin: ignore FSEvents caused by xattr changes on macOS Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Jeff Hostetler , Jeff Hostetler Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Jeff Hostetler From: Jeff Hostetler Ignore FSEvents resulting from `xattr` changes. Git does not care about xattr's or changes to xattr's, so don't waste time collecting these events in the daemon nor transmitting them to clients. Various security tools add xattrs to files and/or directories, such as to mark them as having been downloaded. We should ignore these events since it doesn't affect the content of the file/directory or the normal meta-data that Git cares about. Signed-off-by: Jeff Hostetler --- compat/fsmonitor/fsm-listen-darwin.c | 34 +++++++++++++++++++++++++++- 1 file changed, 33 insertions(+), 1 deletion(-) diff --git a/compat/fsmonitor/fsm-listen-darwin.c b/compat/fsmonitor/fsm-listen-darwin.c index 2aefdc14d89..79d08517d7b 100644 --- a/compat/fsmonitor/fsm-listen-darwin.c +++ b/compat/fsmonitor/fsm-listen-darwin.c @@ -172,7 +172,7 @@ static void log_flags_set(const char *path, const FSEventStreamEventFlags flag) if (flag & kFSEventStreamEventFlagItemCloned) strbuf_addstr(&msg, "ItemCloned|"); - trace_printf_key(&trace_fsmonitor, "fsevent: '%s', flags=%u %s", + trace_printf_key(&trace_fsmonitor, "fsevent: '%s', flags=0x%x %s", path, flag, msg.buf); strbuf_release(&msg); @@ -197,6 +197,31 @@ static int ef_is_dropped(const FSEventStreamEventFlags ef) ef & kFSEventStreamEventFlagUserDropped); } +/* + * If an `xattr` change is the only reason we received this event, + * then silently ignore it. Git doesn't care about xattr's. We + * have to be careful here because the kernel can combine multiple + * events for a single path. And because events always have certain + * bits set, such as `ItemIsFile` or `ItemIsDir`. + * + * Return 1 if we should ignore it. + */ +static int ef_ignore_xattr(const FSEventStreamEventFlags ef) +{ + static const FSEventStreamEventFlags mask = + kFSEventStreamEventFlagItemChangeOwner | + kFSEventStreamEventFlagItemCreated | + kFSEventStreamEventFlagItemFinderInfoMod | + kFSEventStreamEventFlagItemInodeMetaMod | + kFSEventStreamEventFlagItemModified | + kFSEventStreamEventFlagItemRemoved | + kFSEventStreamEventFlagItemRenamed | + kFSEventStreamEventFlagItemXattrMod | + kFSEventStreamEventFlagItemCloned; + + return ((ef & mask) == kFSEventStreamEventFlagItemXattrMod); +} + static void fsevent_callback(ConstFSEventStreamRef streamRef, void *ctx, size_t num_of_events, @@ -262,6 +287,13 @@ static void fsevent_callback(ConstFSEventStreamRef streamRef, continue; } + if (ef_ignore_xattr(event_flags[k])) { + trace_printf_key(&trace_fsmonitor, + "ignore-xattr: '%s', flags=0x%x", + path_k, event_flags[k]); + continue; + } + switch (fsmonitor_classify_path_absolute(state, path_k)) { case IS_INSIDE_DOT_GIT_WITH_COOKIE_PREFIX: From patchwork Tue Feb 15 15:59:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff Hostetler X-Patchwork-Id: 12747267 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 300CAC4332F for ; Tue, 15 Feb 2022 16:00:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241154AbiBOQAc (ORCPT ); Tue, 15 Feb 2022 11:00:32 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:56322 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241133AbiBOQAM (ORCPT ); Tue, 15 Feb 2022 11:00:12 -0500 Received: from mail-wr1-x42e.google.com (mail-wr1-x42e.google.com [IPv6:2a00:1450:4864:20::42e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 731C5BDE62 for ; Tue, 15 Feb 2022 07:59:50 -0800 (PST) Received: by mail-wr1-x42e.google.com with SMTP id s10so19371701wrb.1 for ; Tue, 15 Feb 2022 07:59:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=loaJkqv8AAh0TwehIgIIWjp9ODA9lqvvgmtTriOysSs=; b=Hiaj5jf9owpQcIA0CYVSm9idI/CT7BoMeoAcWPZkrwF6kvf4pWQGgRo1ryN2AG8zL1 njNt5eNE6tAtCB3tvy0YkUlLlcjpylpfNDkV0uH0X4TA9hBpH/oQRdMPLaE+i87JuLZD 0HtGALBMj+dZn5Cq+wwKfhSind6mJmUkNL7kIMEuvytyczX31jBQAJn7HoHA3Hyoc3Rz 0Qk1iDq85KOJ+jqcV5oixiDZriwe7RZjeDidRsuF+vKQh9bDXZLQVTBSh0OUoMN+/2VN C4wleNXHSeidetle9QO5+w0HMn+o62yhnh1fHt9XBmoS5rfDDb6rxaAN6TOnhJ1luh7W 2Wgw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=loaJkqv8AAh0TwehIgIIWjp9ODA9lqvvgmtTriOysSs=; b=UYLsDJdiDFSiYUoCEvFTM6ROv6Ya20cES01tsx3Z6RdYixYfC+xXoMFxV3t/dnhIoU 8lMdacadQIwwqwlORT3lVLIt+YaLL56sII3inK+rz1s5YHRh4t9kF+UYylbhT1wlFyYP rgtTV8A6/So+lamsKk8h7m61XerYOZVfhKClk44zxvSakMrhNtWJ414l1HXDkxlnDe28 xd2RPVi1KeYuTYKsrGuW+HNw2Pz+z+MqS2dPmX07Y+DXl1MMWihBYqhGiwQung4+uvDc n1Y1iKc4cxnnyH9+gEyVKgpoH3jNMvK9SaoFFfiZ/7vPnM9oVSDjOQeTmQT+NzJzMMp/ WbcA== X-Gm-Message-State: AOAM532j96zHK413rrtnmTyCU+qtJ+DqW1qqupYWK07MB/ADFBk6uLcV VP6GuZdTCmtwg4MAIohv9HsolRqVBUI= X-Google-Smtp-Source: ABdhPJzfQv4luiu3qB+2f6sc6sL5av2u4oisuBA0Ln7owu1J0DeoxvexJXKv44t9XOa1vF8VgBV3aQ== X-Received: by 2002:adf:e0c9:: with SMTP id m9mr3951190wri.435.1644940788815; Tue, 15 Feb 2022 07:59:48 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id ba14sm10735153wrb.56.2022.02.15.07.59.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Feb 2022 07:59:48 -0800 (PST) Message-Id: In-Reply-To: References: Date: Tue, 15 Feb 2022 15:59:23 +0000 Subject: [PATCH 13/23] fsmonitor--daemon: print start message only if fsmonitor.announceStartup Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Jeff Hostetler , Jeff Hostetler Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Jeff Hostetler From: Jeff Hostetler Teach fsmonitor--daemon to print a startup message only when `fsmonitor.announceStartup` is true. This setting is false by default so that the output of client commands, like `git status`, is not changed if the daemon is implicitly started. The message is conditionally printed by "run" and "start" subcommands and is sent to stderr. It contains the path to the work tree root. Signed-off-by: Jeff Hostetler --- builtin/fsmonitor--daemon.c | 29 +++++++++++++++++++++++------ 1 file changed, 23 insertions(+), 6 deletions(-) diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c index 899355c55aa..dd0561cfc51 100644 --- a/builtin/fsmonitor--daemon.c +++ b/builtin/fsmonitor--daemon.c @@ -27,6 +27,9 @@ static int fsmonitor__ipc_threads = 8; #define FSMONITOR__START_TIMEOUT "fsmonitor.starttimeout" static int fsmonitor__start_timeout_sec = 60; +#define FSMONITOR__ANNOUNCE_STARTUP "fsmonitor.announcestartup" +static int fsmonitor__announce_startup = 0; + static int fsmonitor_config(const char *var, const char *value, void *cb) { if (!strcmp(var, FSMONITOR__IPC_THREADS)) { @@ -47,6 +50,16 @@ static int fsmonitor_config(const char *var, const char *value, void *cb) return 0; } + if (!strcmp(var, FSMONITOR__ANNOUNCE_STARTUP)) { + int is_bool; + int i = git_config_bool_or_int(var, value, &is_bool); + if (i < 0) + return error(_("value of '%s' not bool or int: %d"), + var, i); + fsmonitor__announce_startup = i; + return 0; + } + return git_default_config(var, value, cb); } @@ -1307,9 +1320,11 @@ static int try_to_run_foreground_daemon(int free_console) die("fsmonitor--daemon is already running '%s'", the_repository->worktree); - printf(_("running fsmonitor-daemon in '%s'\n"), - the_repository->worktree); - fflush(stdout); + if (fsmonitor__announce_startup) { + fprintf(stderr, _("running fsmonitor-daemon in '%s'\n"), + the_repository->worktree); + fflush(stderr); + } #ifdef GIT_WINDOWS_NATIVE if (free_console) @@ -1360,9 +1375,11 @@ static int try_to_start_background_daemon(void) die("fsmonitor--daemon is already running '%s'", the_repository->worktree); - printf(_("starting fsmonitor-daemon in '%s'\n"), - the_repository->worktree); - fflush(stdout); + if (fsmonitor__announce_startup) { + fprintf(stderr, _("starting fsmonitor-daemon in '%s'\n"), + the_repository->worktree); + fflush(stderr); + } cp.git_cmd = 1; From patchwork Tue Feb 15 15:59:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff Hostetler X-Patchwork-Id: 12747268 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 968BAC433F5 for ; Tue, 15 Feb 2022 16:00:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241137AbiBOQAe (ORCPT ); Tue, 15 Feb 2022 11:00:34 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:56124 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238585AbiBOQAR (ORCPT ); Tue, 15 Feb 2022 11:00:17 -0500 Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com [IPv6:2a00:1450:4864:20::436]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 83C16BDE75 for ; Tue, 15 Feb 2022 07:59:51 -0800 (PST) Received: by mail-wr1-x436.google.com with SMTP id j26so22057661wrb.7 for ; Tue, 15 Feb 2022 07:59:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=xRVyFwV/KR6l0CAmUoatgRiqQpA8JXkdhfK/5H7F/hg=; b=F9BT6L10n5Ec0FUaTm+i/h3Mec1PvOm5U0/08vKwywKKqFWRAI7W7E/X84Gg5Bg6ii aOMnyF4Nlcv80T/LC09cvTALxxCRFsKeztK4axDimbvNiXWgZqWb77eA/JqMNbBAdHq+ PVsXZunSwKKUoBMtaGGepxQZV65WPk0cjZ6+XbQ1YVtmEOCx8FWdRKbI2rPjY6tVeJfs XXsRIzBAoiRTC8WBUhu/ojkooF1v6MVlxWkmCOYXgsHuO5QatATVaSC0kBFajhMpXcf+ fGXg7J7hF1mwIMmL6ys/kVvld8w5P2zw5iSngcRJTHb9L5fVxOb+AZy8r+OupdRPWg3e tVYg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=xRVyFwV/KR6l0CAmUoatgRiqQpA8JXkdhfK/5H7F/hg=; b=Wul1qHpOZ/cmZXwUtA95qQjPe1TDfkVIFzkJ9OekfEplKhXoQ6LVgVaWFFt8R2fr+D F8khKltzpF6bPYzmO7KDWhx0ujOOhah/bVSIgov05SVkxxBI5BGcYg4zn/6MMRmMGQVq wD4TATLQ6pYewGqURNZZij2MkiBVVl0bew67oUegV70xeI+c8wsZHnSMCZH6Glc6tkRk A00Waskr69WU/5LuqTEdmUK9Acqp5LeXAHHTucxEcM6buK21ZC26gcCM3TMrxOy73oOO 4tI/gSXuLaF+FSbVpLxfTGSFJzKsSQcy/fVzEHi6ehgL9MfuUSLsAKlg3/StbgzKnede K4xg== X-Gm-Message-State: AOAM532f8SiHEfWt1DLjS72y0722pX/ocZ2DmjT4krr5XySP03KuiUqg 84X/9MzCBkYvthLSRSDTyM66FMq5nFk= X-Google-Smtp-Source: ABdhPJx9sPXK+faen9l9VyeXPhNzDbSj8k2/lNoHi4+PnIHwmpvVmTBPXuAjm2KGRu72qSfObUqOYQ== X-Received: by 2002:a5d:62c4:: with SMTP id o4mr3745400wrv.319.1644940789893; Tue, 15 Feb 2022 07:59:49 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id q13sm33716280wrd.78.2022.02.15.07.59.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Feb 2022 07:59:49 -0800 (PST) Message-Id: <95cdff22be0b0ee6f364dfed031836ed2327cadb.1644940774.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 15 Feb 2022 15:59:24 +0000 Subject: [PATCH 14/23] fsmonitor--daemon: cd out of worktree root Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Jeff Hostetler , Jeff Hostetler Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Jeff Hostetler From: Jeff Hostetler Teach the fsmonitor--daemon to CD outside of the worktree before starting up. The common Git startup mechanism causes the CWD of the daemon process to be in the root of the worktree. On Windows, this causes the daemon process to hold a locked handle on the CWD and prevents other processes from moving or deleting the worktree while the daemon is running. CD to HOME before entering main event loops. Signed-off-by: Jeff Hostetler --- builtin/fsmonitor--daemon.c | 32 +++++++++++++++++++++++++++-- compat/fsmonitor/fsm-listen-win32.c | 22 ++++++++++++++------ fsmonitor--daemon.h | 1 + 3 files changed, 47 insertions(+), 8 deletions(-) diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c index dd0561cfc51..e9b3f44d791 100644 --- a/builtin/fsmonitor--daemon.c +++ b/builtin/fsmonitor--daemon.c @@ -1169,11 +1169,11 @@ static int fsmonitor_run_daemon_1(struct fsmonitor_daemon_state *state) * before we need it. */ if (ipc_server_run_async(&state->ipc_server_data, - fsmonitor_ipc__get_path(), &ipc_opts, + state->path_ipc.buf, &ipc_opts, handle_client, state)) return error_errno( _("could not start IPC thread pool on '%s'"), - fsmonitor_ipc__get_path()); + state->path_ipc.buf); /* * Start the fsmonitor listener thread to collect filesystem @@ -1208,6 +1208,7 @@ static int fsmonitor_run_daemon_1(struct fsmonitor_daemon_state *state) static int fsmonitor_run_daemon(void) { struct fsmonitor_daemon_state state; + const char *home; int err; memset(&state, 0, sizeof(state)); @@ -1277,6 +1278,15 @@ static int fsmonitor_run_daemon(void) strbuf_addch(&state.path_cookie_prefix, '/'); + /* + * We create a named-pipe or unix domain socket inside of the + * ".git" directory. (Well, on Windows, we base our named + * pipe in the NPFS on the absolute path of the git + * directory.) + */ + strbuf_init(&state.path_ipc, 0); + strbuf_addstr(&state.path_ipc, absolute_path(fsmonitor_ipc__get_path())); + /* * Confirm that we can create platform-specific resources for the * filesystem listener before we bother starting all the threads. @@ -1286,6 +1296,23 @@ static int fsmonitor_run_daemon(void) goto done; } + /* + * CD out of the worktree root directory. + * + * The common Git startup mechanism causes our CWD to be the + * root of the worktree. On Windows, this causes our process + * to hold a locked handle on the CWD. This prevents the + * worktree from being moved or deleted while the daemon is + * running. + * + * We assume that our FS and IPC listener threads have either + * opened all of the handles that they need or will do + * everything using absolute paths. + */ + home = getenv("HOME"); + if (home && *home && chdir(home)) + die_errno("could not cd home '%s'", home); + err = fsmonitor_run_daemon_1(&state); done: @@ -1298,6 +1325,7 @@ done: strbuf_release(&state.path_worktree_watch); strbuf_release(&state.path_gitdir_watch); strbuf_release(&state.path_cookie_prefix); + strbuf_release(&state.path_ipc); /* * NEEDSWORK: Consider "rm -rf /" diff --git a/compat/fsmonitor/fsm-listen-win32.c b/compat/fsmonitor/fsm-listen-win32.c index eb407b0748c..6985903c95b 100644 --- a/compat/fsmonitor/fsm-listen-win32.c +++ b/compat/fsmonitor/fsm-listen-win32.c @@ -398,12 +398,22 @@ static int recv_rdcw_watch(struct one_watch *watch) } /* - * NEEDSWORK: If an external is deleted, the above - * returns an error. I'm not sure that there's anything that - * we can do here other than failing -- the /.git - * link file would be broken anyway. We might try to check - * for that and return a better error message, but I'm not - * sure it is worth it. + * GetOverlappedResult() fails if the watched directory is + * deleted while we were waiting for an overlapped IO to + * complete. The documentation did not list specific errors, + * but I observed ERROR_ACCESS_DENIED (0x05) errors during + * testing. + * + * Note that we only get notificaiton events for events + * *within* the directory, not *on* the directory itself. + * (These might be properies of the parent directory, for + * example). + * + * NEEDSWORK: We might try to check for the deleted directory + * case and return a better error message, but I'm not sure it + * is worth it. + * + * Shutdown if we get any error. */ error("GetOverlappedResult failed on '%s' [GLE %ld]", diff --git a/fsmonitor--daemon.h b/fsmonitor--daemon.h index c16ef095688..a777c3a0590 100644 --- a/fsmonitor--daemon.h +++ b/fsmonitor--daemon.h @@ -54,6 +54,7 @@ struct fsmonitor_daemon_state { struct fsmonitor_daemon_backend_data *backend_data; struct ipc_server_data *ipc_server_data; + struct strbuf path_ipc; }; /* From patchwork Tue Feb 15 15:59:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff Hostetler X-Patchwork-Id: 12747269 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B005FC433FE for ; Tue, 15 Feb 2022 16:00:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241160AbiBOQAh (ORCPT ); Tue, 15 Feb 2022 11:00:37 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:56126 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241096AbiBOQA3 (ORCPT ); Tue, 15 Feb 2022 11:00:29 -0500 Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com [IPv6:2a00:1450:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 54B18BF52E for ; Tue, 15 Feb 2022 07:59:52 -0800 (PST) Received: by mail-wr1-x42d.google.com with SMTP id h6so32894874wrb.9 for ; Tue, 15 Feb 2022 07:59:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=aaxh25XeSH8sX61oAvJCISnnCm9H/WdtdhYBRpy7XFo=; b=aSAsOp5qGBqqwAunQzbuycz8gm5V8NnFIo6W5JAizs90wZX4x4TtlTcA5jTLhvyLXb jm3r/5rnsICcyP4nwYIFdFiajN87eIuutIMRS4LtZ/YzX1rzTVGKjpihIjCPK84mE2t8 lO2AEh+qjb8sWesHIJnhgnu1gvYhSeCEjwlUPKudYzG9Ru7rrUB+aJw/C1PHJMJlF/vk BfELED9HKZV08d4s4/VPGaLQnuoOO/CSEPXIm3mjFR0ZKEHo2CP11gDYQ8Ip1jejVgmF TM0Hu9ZedeooekFoSHcdvRXFscmnZIcOU7FhLYxMUg6aUB1WKk7RS5moLQ6TCcwANaj7 jECA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=aaxh25XeSH8sX61oAvJCISnnCm9H/WdtdhYBRpy7XFo=; b=DsNhdroJuitz6Ofuk4vgshme3vPLd859SjZVDbOSDihrjdsOFv41OR9q2XJSf+nxoJ 7tVKlFLTwrThuEDjcyPluwDsCI6h8rFn6NSKHYZA+fq8cE5N7JGS9RpvMbrcy4E0wEQD E51wvKQYVsmqIGvoCStGA3AFBcCo8aJY17pKkg6dOuHjSccW9rxPT2HvANd81zfbl58j NUggdByX3GMoBsW78nyVGf51yOMTdDyZS1lECQaThPw8I4KULkIuIJXdY/Z5GJ8auuBA QAfHLytucoOxVNIXOhz7Kxya4IS76TZ7JFI5Z2/g4Nr6Gd8oT8kN3uwVEUSGjtYMhnpq z+mw== X-Gm-Message-State: AOAM532R1L0xOnEewiP3sdYmxssFHERhQcIP7SCiF5jylzftajyYWmef JVgTMKF2/xxAf3R04dV4fq2uieouQxU= X-Google-Smtp-Source: ABdhPJxwU9CTtHhCPdhlYzOAHNQcsFn0+cE00elCYyLh1iXZhihOTmr7ZRxSE6ItL4L+5Auh+Tvseg== X-Received: by 2002:a5d:64e5:: with SMTP id g5mr3754751wri.361.1644940790780; Tue, 15 Feb 2022 07:59:50 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id q2sm32178592wrw.14.2022.02.15.07.59.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Feb 2022 07:59:50 -0800 (PST) Message-Id: <6b5642f7770d95f318eb04066e0291072bd8814e.1644940774.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 15 Feb 2022 15:59:25 +0000 Subject: [PATCH 15/23] fsmonitor--daemon: prepare for adding health thread Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Jeff Hostetler , Jeff Hostetler Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Jeff Hostetler From: Jeff Hostetler Refactor daemon thread startup to make it easier to start a third thread class to monitor the health of the daemon. Signed-off-by: Jeff Hostetler --- builtin/fsmonitor--daemon.c | 27 ++++++++++++++++++++------- 1 file changed, 20 insertions(+), 7 deletions(-) diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c index e9b3f44d791..f42fb2ab626 100644 --- a/builtin/fsmonitor--daemon.c +++ b/builtin/fsmonitor--daemon.c @@ -1162,6 +1162,8 @@ static int fsmonitor_run_daemon_1(struct fsmonitor_daemon_state *state) */ .uds_disallow_chdir = 0 }; + int listener_started = 0; + int err = 0; /* * Start the IPC thread pool before the we've started the file @@ -1182,15 +1184,20 @@ static int fsmonitor_run_daemon_1(struct fsmonitor_daemon_state *state) if (pthread_create(&state->listener_thread, NULL, fsm_listen__thread_proc, state) < 0) { ipc_server_stop_async(state->ipc_server_data); - ipc_server_await(state->ipc_server_data); - - return error(_("could not start fsmonitor listener thread")); + err = error(_("could not start fsmonitor listener thread")); + goto cleanup; } + listener_started = 1; /* * The daemon is now fully functional in background threads. + * Our primary thread should now just wait while the threads + * do all the work. + */ +cleanup: + /* * Wait for the IPC thread pool to shutdown (whether by client - * request or from filesystem activity). + * request, from filesystem activity, or an error). */ ipc_server_await(state->ipc_server_data); @@ -1199,10 +1206,16 @@ static int fsmonitor_run_daemon_1(struct fsmonitor_daemon_state *state) * event from the IPC thread pool, but it doesn't hurt to tell * it again. And wait for it to shutdown. */ - fsm_listen__stop_async(state); - pthread_join(state->listener_thread, NULL); + if (listener_started) { + fsm_listen__stop_async(state); + pthread_join(state->listener_thread, NULL); + } - return state->error_code; + if (err) + return err; + if (state->error_code) + return state->error_code; + return 0; } static int fsmonitor_run_daemon(void) From patchwork Tue Feb 15 15:59:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff Hostetler X-Patchwork-Id: 12747270 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4C862C433EF for ; Tue, 15 Feb 2022 16:00:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241172AbiBOQAh (ORCPT ); Tue, 15 Feb 2022 11:00:37 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:56432 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241143AbiBOQA3 (ORCPT ); Tue, 15 Feb 2022 11:00:29 -0500 Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 69502C0539 for ; Tue, 15 Feb 2022 07:59:53 -0800 (PST) Received: by mail-wr1-x434.google.com with SMTP id w11so32908295wra.4 for ; Tue, 15 Feb 2022 07:59:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=h0VCBKNuQKteyZmQRJKVR+0JTcjzU388Wm7M0cQcvBc=; b=hw38NbAvwdsmspJU7qbF8ONPRzujZIsLh8Uee01Fe2x146GuVELQIUCRu1Oea0UaBP VTONnCodnKe1Y/DSlxwic154Ej4F9zdWEvummXSBQkBmeU5JHsiprMK0tADE+3q1dvRt 4C0qfvGT6tP7x4M0fybsWoyx6UFz79AQlhKSXlCmXXc9IqBfbSjoIgQYDACKjm5C94/5 hzZz9G014kbbJ58waZMrv56dAmlaGGFBM/HHcduXkyKH/8HTzoGGfjCTZ0DKyEvvOgpy JgMvQjG2znFnWYzbYAcAvDvK97aZym3UIIoLYlkEl8miV0/46NxlMZCVufFBqTu9bBjZ s/oA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=h0VCBKNuQKteyZmQRJKVR+0JTcjzU388Wm7M0cQcvBc=; b=xjobEfhOJ5vgzOuF30fkOsIpE8Gq5BpznUEgZ+ETdHtmtmhqcKAJkRrd+51tWw6X+q J6G1Dp+FSlm64bPqdhdgcioerU9kFFi1qEY3hQy8GV+YT8Z1OjyXvZgllY77Dngb5YjB LFaBrVYp1dTiZGW73Y0GAv8qUg0kvw16O4no+myjXpRcIksXvdKkou9hoywBKyscn2Lo dljD0zhT10lHFq42nAwzRM8fFWrhyio4bnHu9foRxWslnVFWusPE3dpEpxYogbA7wcNB JwK6xIF94BxqykXGK97F6MStsfHVKyqZzL51bo+WD1HZgBXsES5uT2UwO2lU6AeGXIiN 34QQ== X-Gm-Message-State: AOAM531w0Q0Blf7vuNexjuH6JEHwzVdE/CJIPCDpAykJW1YoZJasa4Cw R8kHmK1UdtUTCWBijDr2FJzacAW/Efk= X-Google-Smtp-Source: ABdhPJyazmjuZsn1SNNnnAg17jXmIQQ1XrnKvWHYSWVKxlcJUDr8K/rYPBZ2rPPW31W3fWt/2T/s/Q== X-Received: by 2002:a05:6000:2c8:: with SMTP id o8mr3906841wry.378.1644940791696; Tue, 15 Feb 2022 07:59:51 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id i3sm35239721wrq.72.2022.02.15.07.59.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Feb 2022 07:59:51 -0800 (PST) Message-Id: <63c502c20bd995636d551d7d24d5885c40bfda19.1644940774.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 15 Feb 2022 15:59:26 +0000 Subject: [PATCH 16/23] fsmonitor--daemon: rename listener thread related variables Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Jeff Hostetler , Jeff Hostetler Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Jeff Hostetler From: Jeff Hostetler Rename platform-specific listener thread related variables and data types as we prepare to add another backend thread type. [] `struct fsmonitor_daemon_backend_data` becomes `struct fsm_listen_data` [] `state->backend_data` becomes `state->listen_data` [] `state->error_code` becomes `state->listen_error_code` Signed-off-by: Jeff Hostetler --- builtin/fsmonitor--daemon.c | 6 +++--- compat/fsmonitor/fsm-listen-darwin.c | 30 ++++++++++++++-------------- compat/fsmonitor/fsm-listen-win32.c | 28 +++++++++++++------------- compat/fsmonitor/fsm-listen.h | 2 +- fsmonitor--daemon.h | 6 +++--- 5 files changed, 36 insertions(+), 36 deletions(-) diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c index f42fb2ab626..f3fb865a9d8 100644 --- a/builtin/fsmonitor--daemon.c +++ b/builtin/fsmonitor--daemon.c @@ -1213,8 +1213,8 @@ cleanup: if (err) return err; - if (state->error_code) - return state->error_code; + if (state->listen_error_code) + return state->listen_error_code; return 0; } @@ -1229,7 +1229,7 @@ static int fsmonitor_run_daemon(void) hashmap_init(&state.cookies, cookies_cmp, NULL, 0); pthread_mutex_init(&state.main_lock, NULL); pthread_cond_init(&state.cookies_cond, NULL); - state.error_code = 0; + state.listen_error_code = 0; state.current_token_data = fsmonitor_new_token_data(); /* Prepare to (recursively) watch the directory. */ diff --git a/compat/fsmonitor/fsm-listen-darwin.c b/compat/fsmonitor/fsm-listen-darwin.c index 79d08517d7b..87a8476b09f 100644 --- a/compat/fsmonitor/fsm-listen-darwin.c +++ b/compat/fsmonitor/fsm-listen-darwin.c @@ -99,7 +99,7 @@ void FSEventStreamRelease(FSEventStreamRef stream); #include "fsm-listen.h" #include "fsmonitor--daemon.h" -struct fsmonitor_daemon_backend_data +struct fsm_listen_data { CFStringRef cfsr_worktree_path; CFStringRef cfsr_gitdir_path; @@ -230,7 +230,7 @@ static void fsevent_callback(ConstFSEventStreamRef streamRef, const FSEventStreamEventId event_ids[]) { struct fsmonitor_daemon_state *state = ctx; - struct fsmonitor_daemon_backend_data *data = state->backend_data; + struct fsm_listen_data *data = state->listen_data; char **paths = (char **)event_paths; struct fsmonitor_batch *batch = NULL; struct string_list cookie_list = STRING_LIST_INIT_DUP; @@ -419,11 +419,11 @@ int fsm_listen__ctor(struct fsmonitor_daemon_state *state) NULL, NULL }; - struct fsmonitor_daemon_backend_data *data; + struct fsm_listen_data *data; const void *dir_array[2]; CALLOC_ARRAY(data, 1); - state->backend_data = data; + state->listen_data = data; data->cfsr_worktree_path = CFStringCreateWithCString( NULL, state->path_worktree_watch.buf, kCFStringEncodingUTF8); @@ -455,18 +455,18 @@ int fsm_listen__ctor(struct fsmonitor_daemon_state *state) failed: error("Unable to create FSEventStream."); - FREE_AND_NULL(state->backend_data); + FREE_AND_NULL(state->listen_data); return -1; } void fsm_listen__dtor(struct fsmonitor_daemon_state *state) { - struct fsmonitor_daemon_backend_data *data; + struct fsm_listen_data *data; - if (!state || !state->backend_data) + if (!state || !state->listen_data) return; - data = state->backend_data; + data = state->listen_data; if (data->stream) { if (data->stream_started) @@ -476,14 +476,14 @@ void fsm_listen__dtor(struct fsmonitor_daemon_state *state) FSEventStreamRelease(data->stream); } - FREE_AND_NULL(state->backend_data); + FREE_AND_NULL(state->listen_data); } void fsm_listen__stop_async(struct fsmonitor_daemon_state *state) { - struct fsmonitor_daemon_backend_data *data; + struct fsm_listen_data *data; - data = state->backend_data; + data = state->listen_data; data->shutdown_style = SHUTDOWN_EVENT; CFRunLoopStop(data->rl); @@ -491,9 +491,9 @@ void fsm_listen__stop_async(struct fsmonitor_daemon_state *state) void fsm_listen__loop(struct fsmonitor_daemon_state *state) { - struct fsmonitor_daemon_backend_data *data; + struct fsm_listen_data *data; - data = state->backend_data; + data = state->listen_data; data->rl = CFRunLoopGetCurrent(); @@ -510,7 +510,7 @@ void fsm_listen__loop(struct fsmonitor_daemon_state *state) switch (data->shutdown_style) { case FORCE_ERROR_STOP: - state->error_code = -1; + state->listen_error_code = -1; /* fall thru */ case FORCE_SHUTDOWN: ipc_server_stop_async(state->ipc_server_data); @@ -522,7 +522,7 @@ void fsm_listen__loop(struct fsmonitor_daemon_state *state) return; force_error_stop_without_loop: - state->error_code = -1; + state->listen_error_code = -1; ipc_server_stop_async(state->ipc_server_data); return; } diff --git a/compat/fsmonitor/fsm-listen-win32.c b/compat/fsmonitor/fsm-listen-win32.c index 6985903c95b..8406f8277dc 100644 --- a/compat/fsmonitor/fsm-listen-win32.c +++ b/compat/fsmonitor/fsm-listen-win32.c @@ -54,7 +54,7 @@ struct one_watch wchar_t dotgit_shortname[16]; /* for 8.3 name */ }; -struct fsmonitor_daemon_backend_data +struct fsm_listen_data { struct one_watch *watch_worktree; struct one_watch *watch_gitdir; @@ -259,7 +259,7 @@ static enum get_relative_result get_relative_longname( void fsm_listen__stop_async(struct fsmonitor_daemon_state *state) { - SetEvent(state->backend_data->hListener[LISTENER_SHUTDOWN]); + SetEvent(state->listen_data->hListener[LISTENER_SHUTDOWN]); } static struct one_watch *create_watch(struct fsmonitor_daemon_state *state, @@ -333,7 +333,7 @@ static void destroy_watch(struct one_watch *watch) free(watch); } -static int start_rdcw_watch(struct fsmonitor_daemon_backend_data *data, +static int start_rdcw_watch(struct fsm_listen_data *data, struct one_watch *watch) { DWORD dwNotifyFilter = @@ -512,7 +512,7 @@ static int process_1_worktree_event( */ static int process_worktree_events(struct fsmonitor_daemon_state *state) { - struct fsmonitor_daemon_backend_data *data = state->backend_data; + struct fsm_listen_data *data = state->listen_data; struct one_watch *watch = data->watch_worktree; struct strbuf path = STRBUF_INIT; struct string_list cookie_list = STRING_LIST_INIT_DUP; @@ -642,7 +642,7 @@ force_shutdown: */ static int process_gitdir_events(struct fsmonitor_daemon_state *state) { - struct fsmonitor_daemon_backend_data *data = state->backend_data; + struct fsm_listen_data *data = state->listen_data; struct one_watch *watch = data->watch_gitdir; struct strbuf path = STRBUF_INIT; struct string_list cookie_list = STRING_LIST_INIT_DUP; @@ -700,11 +700,11 @@ skip_this_path: void fsm_listen__loop(struct fsmonitor_daemon_state *state) { - struct fsmonitor_daemon_backend_data *data = state->backend_data; + struct fsm_listen_data *data = state->listen_data; DWORD dwWait; int result; - state->error_code = 0; + state->listen_error_code = 0; if (start_rdcw_watch(data, data->watch_worktree) == -1) goto force_error_stop; @@ -769,7 +769,7 @@ void fsm_listen__loop(struct fsmonitor_daemon_state *state) } force_error_stop: - state->error_code = -1; + state->listen_error_code = -1; force_shutdown: /* @@ -786,7 +786,7 @@ clean_shutdown: int fsm_listen__ctor(struct fsmonitor_daemon_state *state) { - struct fsmonitor_daemon_backend_data *data; + struct fsm_listen_data *data; CALLOC_ARRAY(data, 1); @@ -819,7 +819,7 @@ int fsm_listen__ctor(struct fsmonitor_daemon_state *state) data->nr_listener_handles++; } - state->backend_data = data; + state->listen_data = data; return 0; failed: @@ -832,16 +832,16 @@ failed: void fsm_listen__dtor(struct fsmonitor_daemon_state *state) { - struct fsmonitor_daemon_backend_data *data; + struct fsm_listen_data *data; - if (!state || !state->backend_data) + if (!state || !state->listen_data) return; - data = state->backend_data; + data = state->listen_data; CloseHandle(data->hEventShutdown); destroy_watch(data->watch_worktree); destroy_watch(data->watch_gitdir); - FREE_AND_NULL(state->backend_data); + FREE_AND_NULL(state->listen_data); } diff --git a/compat/fsmonitor/fsm-listen.h b/compat/fsmonitor/fsm-listen.h index f0539349baf..41650bf8972 100644 --- a/compat/fsmonitor/fsm-listen.h +++ b/compat/fsmonitor/fsm-listen.h @@ -33,7 +33,7 @@ void fsm_listen__dtor(struct fsmonitor_daemon_state *state); * do so if the listener thread receives a normal shutdown signal from * the IPC layer.) * - * It should set `state->error_code` to -1 if the daemon should exit + * It should set `state->listen_error_code` to -1 if the daemon should exit * with an error. */ void fsm_listen__loop(struct fsmonitor_daemon_state *state); diff --git a/fsmonitor--daemon.h b/fsmonitor--daemon.h index a777c3a0590..f7de7882517 100644 --- a/fsmonitor--daemon.h +++ b/fsmonitor--daemon.h @@ -33,7 +33,7 @@ void fsmonitor_batch__free_list(struct fsmonitor_batch *batch); */ void fsmonitor_batch__add_path(struct fsmonitor_batch *batch, const char *path); -struct fsmonitor_daemon_backend_data; /* opaque platform-specific data */ +struct fsm_listen_data; /* opaque platform-specific data for listener thread */ struct fsmonitor_daemon_state { pthread_t listener_thread; @@ -50,8 +50,8 @@ struct fsmonitor_daemon_state { int cookie_seq; struct hashmap cookies; - int error_code; - struct fsmonitor_daemon_backend_data *backend_data; + int listen_error_code; + struct fsm_listen_data *listen_data; struct ipc_server_data *ipc_server_data; struct strbuf path_ipc; From patchwork Tue Feb 15 15:59:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff Hostetler X-Patchwork-Id: 12747276 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6BA0AC43219 for ; Tue, 15 Feb 2022 16:00:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241242AbiBOQAv (ORCPT ); Tue, 15 Feb 2022 11:00:51 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:56320 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241149AbiBOQA3 (ORCPT ); Tue, 15 Feb 2022 11:00:29 -0500 Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com [IPv6:2a00:1450:4864:20::32a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 50C3CC0846 for ; Tue, 15 Feb 2022 07:59:54 -0800 (PST) Received: by mail-wm1-x32a.google.com with SMTP id d14-20020a05600c34ce00b0037bf4d14dc7so1705274wmq.3 for ; Tue, 15 Feb 2022 07:59:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=TAUHD55rHQCQbT+0+ox3IidX3Y7QpfUp5PFWbkeQsuY=; b=Q4MFY1jLscN9HrXCoX+tadxL/UpTjP8Evn1lvbj/jELk5AhH4FlDN4RFn9RiU5+qiW jbC4yXXe+8hRSagUVJubbNt9AfAzOfftBpCi+oaov+7OJVR86ANm0FjnA0r39XfmPa6z 3puk52R73h804vVVtvoT2qqA6KcaNuutNoRlpe3mp7hriMoN+1UT1Dxf8o+sladXg1Yq HodiQL0fNboNEtjZsnUgWztlab2nWu1eQnuB19ZCcAwNSaIu7kL9uGj/12L2/DzmcMTf LH60ox4HdUPoBDH3litHWuS5gYEKaDWsQ+05UfeLaJGLxvstdbk7fu1kuHyaXba5lDEw /w6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=TAUHD55rHQCQbT+0+ox3IidX3Y7QpfUp5PFWbkeQsuY=; b=kWI/njg2Rq0ytaFXgVdXIbnwlLFDg7aAddjyctUpaPyrAVUJERT9GZGy+0QiAqQtn8 J/YlRhsKdNTLlwd5Z+rVHsztoV5RwO/N64H+VocjGdXdEDMgDzFwP4OLoD3vRgNzXygK RQldVfPixFLegF1aOZSyBDGgJVOT2hBeAlXEgXDeodZqt+IspEyz5JVgb0R5qo32VsNA n6wtTiCXB1GcO2dbxKmOUfZL/NDyB+CYu2tyDb0kfsNNeEBJGIw1W9mDgbjLyUGYPnH4 ILJY9kV3IOKOlhIXChmM1GMg1bYfniB4i9bOQ6AOv8lD2yVN/K/CktTD/ycBgnsC+CCA 91mA== X-Gm-Message-State: AOAM533UA0jqSRVVeL5t+5tIGD0AsBsoo5e0kjAF+M2VQaDKwHLDQ/l+ NZLj1fQol6uiLxq7ntc8Kn9e3hrwdjk= X-Google-Smtp-Source: ABdhPJwddKbbOumOZ5hF5pKKIYUu3d+TFZLd88+RygiI3RhbxvzoHDsHcpWjBM5jlAyC7XWkBeG4cw== X-Received: by 2002:a05:600c:414c:b0:37b:c80e:e3dc with SMTP id h12-20020a05600c414c00b0037bc80ee3dcmr3636046wmm.116.1644940792605; Tue, 15 Feb 2022 07:59:52 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id k34sm15269438wms.35.2022.02.15.07.59.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Feb 2022 07:59:52 -0800 (PST) Message-Id: <4a77f5b1fdef3af20fbfefccccc5fee194eb3422.1644940774.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 15 Feb 2022 15:59:27 +0000 Subject: [PATCH 17/23] fsmonitor--daemon: stub in health thread Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Jeff Hostetler , Jeff Hostetler Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Jeff Hostetler From: Jeff Hostetler Create another thread to watch over the daemon process and automatically shut it down if necessary. This commit creates the basic framework for a "health" thread to monitor the daemon and/or the file system. Later commits will add platform-specific code to do the actual work. The "health" thread is intended to monitor conditions that would be difficult to track inside the IPC thread pool and/or the file system listener threads. For example, when there are file system events outside of the watched worktree root or if we want to have an idle-timeout auto-shutdown feature. Signed-off-by: Jeff Hostetler --- Makefile | 6 ++- builtin/fsmonitor--daemon.c | 39 +++++++++++++++ compat/fsmonitor/fsm-health-darwin.c | 24 ++++++++++ compat/fsmonitor/fsm-health-win32.c | 72 ++++++++++++++++++++++++++++ compat/fsmonitor/fsm-health.h | 47 ++++++++++++++++++ contrib/buildsystems/CMakeLists.txt | 2 + fsmonitor--daemon.h | 4 ++ 7 files changed, 192 insertions(+), 2 deletions(-) create mode 100644 compat/fsmonitor/fsm-health-darwin.c create mode 100644 compat/fsmonitor/fsm-health-win32.c create mode 100644 compat/fsmonitor/fsm-health.h diff --git a/Makefile b/Makefile index ed3d5231ea1..85a61934f21 100644 --- a/Makefile +++ b/Makefile @@ -467,8 +467,9 @@ all:: # # If your platform supports a built-in fsmonitor backend, set # FSMONITOR_DAEMON_BACKEND to the "" of the corresponding -# `compat/fsmonitor/fsm-listen-.c` that implements the -# `fsm_listen__*()` routines. +# `compat/fsmonitor/fsm-listen-.c` and +# `compat/fsmonitor/fsm-health-.c` files +# that implement the `fsm_listen__*()` and `fsm_health__*()` routines. # # If your platform has os-specific ways to tell if a repo is incompatible with # fsmonitor (whether the hook or ipc daemon version), set FSMONITOR_OS_SETTINGS @@ -1944,6 +1945,7 @@ endif ifdef FSMONITOR_DAEMON_BACKEND COMPAT_CFLAGS += -DHAVE_FSMONITOR_DAEMON_BACKEND COMPAT_OBJS += compat/fsmonitor/fsm-listen-$(FSMONITOR_DAEMON_BACKEND).o + COMPAT_OBJS += compat/fsmonitor/fsm-health-$(FSMONITOR_DAEMON_BACKEND).o endif ifdef FSMONITOR_OS_SETTINGS diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c index f3fb865a9d8..591433e897d 100644 --- a/builtin/fsmonitor--daemon.c +++ b/builtin/fsmonitor--daemon.c @@ -3,6 +3,7 @@ #include "parse-options.h" #include "fsmonitor.h" #include "fsmonitor-ipc.h" +#include "compat/fsmonitor/fsm-health.h" #include "compat/fsmonitor/fsm-listen.h" #include "fsmonitor--daemon.h" #include "simple-ipc.h" @@ -1124,6 +1125,18 @@ void fsmonitor_publish(struct fsmonitor_daemon_state *state, pthread_mutex_unlock(&state->main_lock); } +static void *fsm_health__thread_proc(void *_state) +{ + struct fsmonitor_daemon_state *state = _state; + + trace2_thread_start("fsm-health"); + + fsm_health__loop(state); + + trace2_thread_exit(); + return NULL; +} + static void *fsm_listen__thread_proc(void *_state) { struct fsmonitor_daemon_state *state = _state; @@ -1162,6 +1175,7 @@ static int fsmonitor_run_daemon_1(struct fsmonitor_daemon_state *state) */ .uds_disallow_chdir = 0 }; + int health_started = 0; int listener_started = 0; int err = 0; @@ -1189,6 +1203,17 @@ static int fsmonitor_run_daemon_1(struct fsmonitor_daemon_state *state) } listener_started = 1; + /* + * Start the health thread to watch over our process. + */ + if (pthread_create(&state->health_thread, NULL, + fsm_health__thread_proc, state) < 0) { + ipc_server_stop_async(state->ipc_server_data); + err = error(_("could not start fsmonitor health thread")); + goto cleanup; + } + health_started = 1; + /* * The daemon is now fully functional in background threads. * Our primary thread should now just wait while the threads @@ -1211,10 +1236,17 @@ cleanup: pthread_join(state->listener_thread, NULL); } + if (health_started) { + fsm_health__stop_async(state); + pthread_join(state->health_thread, NULL); + } + if (err) return err; if (state->listen_error_code) return state->listen_error_code; + if (state->health_error_code) + return state->health_error_code; return 0; } @@ -1230,6 +1262,7 @@ static int fsmonitor_run_daemon(void) pthread_mutex_init(&state.main_lock, NULL); pthread_cond_init(&state.cookies_cond, NULL); state.listen_error_code = 0; + state.health_error_code = 0; state.current_token_data = fsmonitor_new_token_data(); /* Prepare to (recursively) watch the directory. */ @@ -1309,6 +1342,11 @@ static int fsmonitor_run_daemon(void) goto done; } + if (fsm_health__ctor(&state)) { + err = error(_("could not initialize health thread")); + goto done; + } + /* * CD out of the worktree root directory. * @@ -1332,6 +1370,7 @@ done: pthread_cond_destroy(&state.cookies_cond); pthread_mutex_destroy(&state.main_lock); fsm_listen__dtor(&state); + fsm_health__dtor(&state); ipc_server_free(state.ipc_server_data); diff --git a/compat/fsmonitor/fsm-health-darwin.c b/compat/fsmonitor/fsm-health-darwin.c new file mode 100644 index 00000000000..b9f709e8548 --- /dev/null +++ b/compat/fsmonitor/fsm-health-darwin.c @@ -0,0 +1,24 @@ +#include "cache.h" +#include "config.h" +#include "fsmonitor.h" +#include "fsm-health.h" +#include "fsmonitor--daemon.h" + +int fsm_health__ctor(struct fsmonitor_daemon_state *state) +{ + return 0; +} + +void fsm_health__dtor(struct fsmonitor_daemon_state *state) +{ + return; +} + +void fsm_health__loop(struct fsmonitor_daemon_state *state) +{ + return; +} + +void fsm_health__stop_async(struct fsmonitor_daemon_state *state) +{ +} diff --git a/compat/fsmonitor/fsm-health-win32.c b/compat/fsmonitor/fsm-health-win32.c new file mode 100644 index 00000000000..94b1d020f25 --- /dev/null +++ b/compat/fsmonitor/fsm-health-win32.c @@ -0,0 +1,72 @@ +#include "cache.h" +#include "config.h" +#include "fsmonitor.h" +#include "fsm-health.h" +#include "fsmonitor--daemon.h" + +struct fsm_health_data +{ + HANDLE hEventShutdown; + + HANDLE hHandles[1]; /* the array does not own these handles */ +#define HEALTH_SHUTDOWN 0 + int nr_handles; /* number of active event handles */ +}; + +int fsm_health__ctor(struct fsmonitor_daemon_state *state) +{ + struct fsm_health_data *data; + + CALLOC_ARRAY(data, 1); + + data->hEventShutdown = CreateEvent(NULL, TRUE, FALSE, NULL); + + data->hHandles[HEALTH_SHUTDOWN] = data->hEventShutdown; + data->nr_handles++; + + state->health_data = data; + return 0; +} + +void fsm_health__dtor(struct fsmonitor_daemon_state *state) +{ + struct fsm_health_data *data; + + if (!state || !state->health_data) + return; + + data = state->health_data; + + CloseHandle(data->hEventShutdown); + + FREE_AND_NULL(state->health_data); +} + +void fsm_health__loop(struct fsmonitor_daemon_state *state) +{ + struct fsm_health_data *data = state->health_data; + + for (;;) { + DWORD dwWait = WaitForMultipleObjects(data->nr_handles, + data->hHandles, + FALSE, INFINITE); + + if (dwWait == WAIT_OBJECT_0 + HEALTH_SHUTDOWN) + goto clean_shutdown; + + error(_("health thread wait failed [GLE %ld]"), + GetLastError()); + goto force_error_stop; + } + +force_error_stop: + state->health_error_code = -1; + ipc_server_stop_async(state->ipc_server_data); +clean_shutdown: + return; +} + +void fsm_health__stop_async(struct fsmonitor_daemon_state *state) +{ + SetEvent(state->health_data->hHandles[HEALTH_SHUTDOWN]); +} diff --git a/compat/fsmonitor/fsm-health.h b/compat/fsmonitor/fsm-health.h new file mode 100644 index 00000000000..45547ba9380 --- /dev/null +++ b/compat/fsmonitor/fsm-health.h @@ -0,0 +1,47 @@ +#ifndef FSM_HEALTH_H +#define FSM_HEALTH_H + +/* This needs to be implemented by each backend */ + +#ifdef HAVE_FSMONITOR_DAEMON_BACKEND + +struct fsmonitor_daemon_state; + +/* + * Initialize platform-specific data for the fsmonitor health thread. + * This will be called from the main thread PRIOR to staring the + * thread. + * + * Returns 0 if successful. + * Returns -1 otherwise. + */ +int fsm_health__ctor(struct fsmonitor_daemon_state *state); + +/* + * Cleanup platform-specific data for the health thread. + * This will be called from the main thread AFTER joining the thread. + */ +void fsm_health__dtor(struct fsmonitor_daemon_state *state); + +/* + * The main body of the platform-specific event loop to monitor the + * health of the daemon process. This will run in the health thread. + * + * The health thread should call `ipc_server_stop_async()` if it needs + * to cause a shutdown. (It should NOT do so if it receives a shutdown + * shutdown signal.) + * + * It should set `state->health_error_code` to -1 if the daemon should exit + * with an error. + */ +void fsm_health__loop(struct fsmonitor_daemon_state *state); + +/* + * Gently request that the health thread shutdown. + * It does not wait for it to stop. The caller should do a JOIN + * to wait for it. + */ +void fsm_health__stop_async(struct fsmonitor_daemon_state *state); + +#endif /* HAVE_FSMONITOR_DAEMON_BACKEND */ +#endif /* FSM_HEALTH_H */ diff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt index fc70dd2fc1d..80704406895 100644 --- a/contrib/buildsystems/CMakeLists.txt +++ b/contrib/buildsystems/CMakeLists.txt @@ -289,12 +289,14 @@ if(SUPPORTS_SIMPLE_IPC) if(CMAKE_SYSTEM_NAME STREQUAL "Windows") add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-win32.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-win32.c) add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-win32.c) elseif(CMAKE_SYSTEM_NAME STREQUAL "Darwin") add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-darwin.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-darwin.c) add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-darwin.c) diff --git a/fsmonitor--daemon.h b/fsmonitor--daemon.h index f7de7882517..716e0e4d28d 100644 --- a/fsmonitor--daemon.h +++ b/fsmonitor--daemon.h @@ -34,9 +34,11 @@ void fsmonitor_batch__free_list(struct fsmonitor_batch *batch); void fsmonitor_batch__add_path(struct fsmonitor_batch *batch, const char *path); struct fsm_listen_data; /* opaque platform-specific data for listener thread */ +struct fsm_health_data; /* opaque platform-specific data for health thread */ struct fsmonitor_daemon_state { pthread_t listener_thread; + pthread_t health_thread; pthread_mutex_t main_lock; struct strbuf path_worktree_watch; @@ -51,7 +53,9 @@ struct fsmonitor_daemon_state { struct hashmap cookies; int listen_error_code; + int health_error_code; struct fsm_listen_data *listen_data; + struct fsm_health_data *health_data; struct ipc_server_data *ipc_server_data; struct strbuf path_ipc; From patchwork Tue Feb 15 15:59:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff Hostetler X-Patchwork-Id: 12747277 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5B5BBC4321E for ; Tue, 15 Feb 2022 16:00:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241237AbiBOQAu (ORCPT ); Tue, 15 Feb 2022 11:00:50 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:56452 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241150AbiBOQA3 (ORCPT ); Tue, 15 Feb 2022 11:00:29 -0500 Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3703CC084E for ; Tue, 15 Feb 2022 07:59:55 -0800 (PST) Received: by mail-wr1-x434.google.com with SMTP id s10so19372122wrb.1 for ; Tue, 15 Feb 2022 07:59:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=lAazD7H5A5l+Nok2+GMiSl0y/i2VXNjLVlzgEY8/SB4=; b=gxZYeo0cX+c1cnjfhSa1RCryFFozcWwZNEImLk4+CMv9DWXaa6/tw8HXrxaRjzwjB8 89I/Z2lh9jcICB7liZn/yVpW3ZZPb9ufsoFA5ylmEz5umHfWCbw/i8VHZs/yLRGV5OMm 97D54bwuWlF+DlaJVJSbP+SsQLKZt3pK35TcWFW7jDDsvLJvP1+72jBA5OY0pPC81JS0 8Z5Yj2pD9S65iVtylOul0jVdjPXGrHPYA7jl/WZZfv1szQps1KjEFbSKuGGc6vkaeC+U ivHOHMRisS42L5ymitylpvyvHwy4W/wJjBacBkrNXVct65dGYMa7dQHzqBnaCf6hYzdw XkPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=lAazD7H5A5l+Nok2+GMiSl0y/i2VXNjLVlzgEY8/SB4=; b=qxEPnXaQnuhGIl1hO9LiyjdCzXy/mn9qhcWGo484UEAk6r6ZMJvvXyJWfAfCzImwyO CPaVard41JyFIq1xvLPqIy/nqo6k6ZRkl6r+TVKmOxZI+aW8AHow86/cR2PQ4LkSqOIm 7iU+JvQ0cqnkfJ1Z1HxmA3HTbK73p+dRhqFAbxeTmTZjYvEIEQx4KFSZnz42rTkAsaRa LjOb3XlR1TLU9a2/Twmk4z+3s0adefLYZ8lzyORf2fMBtB+0gXj8jMXiBpMxszhDBdVo 1ixKaNbrVheU7Y1C7F4KdlJmNmhIJD4dqgJl2drcXCCRHoILam7tDSSmQvQTDOYdvsa0 /77g== X-Gm-Message-State: AOAM532K9KpAnRaOBbyzxgwGYmVAYuQC5sADeb8yY8chlRFYtkegAGLX yng5ysNrCRM4pS8FfjDKI5/aYsPG2Vc= X-Google-Smtp-Source: ABdhPJwJ4QM/OCYWlk9x3L3i3eL/agc4DRI7b7DGfaOVUJ3aFsGtBHMjdyuGqtUwBFS3TgpbgVKQcQ== X-Received: by 2002:adf:face:: with SMTP id a14mr3783354wrs.254.1644940793473; Tue, 15 Feb 2022 07:59:53 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id n6sm15704214wru.114.2022.02.15.07.59.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Feb 2022 07:59:52 -0800 (PST) Message-Id: In-Reply-To: References: Date: Tue, 15 Feb 2022 15:59:28 +0000 Subject: [PATCH 18/23] fsm-health-win32: add framework to monitor daemon health Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Jeff Hostetler , Jeff Hostetler Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Jeff Hostetler From: Jeff Hostetler Create framework in Win32 version of the "health" thread to periodically inspect the system and shutdown if warranted. This version just includes the setup for the timeout in WaitForMultipleObjects() and calls the (currently empty) table of functions. A later commit will add functions to the table to actually inspect the system. Signed-off-by: Jeff Hostetler --- compat/fsmonitor/fsm-health-win32.c | 54 ++++++++++++++++++++++++++++- 1 file changed, 53 insertions(+), 1 deletion(-) diff --git a/compat/fsmonitor/fsm-health-win32.c b/compat/fsmonitor/fsm-health-win32.c index 94b1d020f25..3c3453369cd 100644 --- a/compat/fsmonitor/fsm-health-win32.c +++ b/compat/fsmonitor/fsm-health-win32.c @@ -4,6 +4,40 @@ #include "fsm-health.h" #include "fsmonitor--daemon.h" +/* + * Every minute wake up and test our health. + */ +#define WAIT_FREQ_MS (60 * 1000) + +enum interval_fn_ctx { CTX_INIT = 0, CTX_TERM, CTX_TIMER }; + +typedef int (interval_fn)(struct fsmonitor_daemon_state *state, + enum interval_fn_ctx ctx); + +static interval_fn *table[] = { + NULL, /* must be last */ +}; + +/* + * Call all of the functions in the table. + * Shortcut and return first error. + * + * Return 0 if all succeeded. + */ +static int call_all(struct fsmonitor_daemon_state *state, + enum interval_fn_ctx ctx) +{ + int k; + + for (k = 0; table[k]; k++) { + int r = table[k](state, ctx); + if (r) + return r; + } + + return 0; +} + struct fsm_health_data { HANDLE hEventShutdown; @@ -45,15 +79,31 @@ void fsm_health__dtor(struct fsmonitor_daemon_state *state) void fsm_health__loop(struct fsmonitor_daemon_state *state) { struct fsm_health_data *data = state->health_data; + int r; + + r = call_all(state, CTX_INIT); + if (r < 0) + goto force_error_stop; + if (r > 0) + goto force_shutdown; for (;;) { DWORD dwWait = WaitForMultipleObjects(data->nr_handles, data->hHandles, - FALSE, INFINITE); + FALSE, WAIT_FREQ_MS); if (dwWait == WAIT_OBJECT_0 + HEALTH_SHUTDOWN) goto clean_shutdown; + if (dwWait == WAIT_TIMEOUT) { + r = call_all(state, CTX_TIMER); + if (r < 0) + goto force_error_stop; + if (r > 0) + goto force_shutdown; + continue; + } + error(_("health thread wait failed [GLE %ld]"), GetLastError()); goto force_error_stop; @@ -61,8 +111,10 @@ void fsm_health__loop(struct fsmonitor_daemon_state *state) force_error_stop: state->health_error_code = -1; +force_shutdown: ipc_server_stop_async(state->ipc_server_data); clean_shutdown: + call_all(state, CTX_TERM); return; } From patchwork Tue Feb 15 15:59:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff Hostetler X-Patchwork-Id: 12747271 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8CB5EC433F5 for ; Tue, 15 Feb 2022 16:00:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241190AbiBOQAj (ORCPT ); Tue, 15 Feb 2022 11:00:39 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:56460 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241156AbiBOQAa (ORCPT ); Tue, 15 Feb 2022 11:00:30 -0500 Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com [IPv6:2a00:1450:4864:20::333]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6A57EC0853 for ; Tue, 15 Feb 2022 07:59:56 -0800 (PST) Received: by mail-wm1-x333.google.com with SMTP id q198-20020a1ca7cf000000b0037bb52545c6so1812998wme.1 for ; Tue, 15 Feb 2022 07:59:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=UkMM7o067DX4pAl9q6UWTehwf7NH/z8tAqzRF88w+U0=; b=jp94jZNW8WpnN2MjgKfa/DEDZFfXAZcPD7WdYPEpKff6xR+EW/QgIgLCX4F7fF7LQd IRfzg3ZmWCfKVc852RDO4pszPlg37Uus6XZwd2RE/fEVM65PHjcci/vLHlvgQ2/aRdV0 eHPesajT9NeFDMWBxpk0QFEQlrLKjWuZp+2zBZELX6VoAzgzeR4MUr0H643CmNLdkwuD 9Ty9uB0U1klWXVvKcKSUDenW8tFkfYxFI4H5+MvygOGJn3DaLlnLmaq7CCmEIhWo2fee jSvVxEzxVQMB8rVNfV+kXJt7Wu3QAA4RqzxcT7ES9K3TXhNp4Cefysj/PH3pbJ1NkLWv yyHA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=UkMM7o067DX4pAl9q6UWTehwf7NH/z8tAqzRF88w+U0=; b=tTpnwxsifyf47J+ni8dm+ul0VfZKVJx+jBrFajZHJRe54QBT/bvgmbtNtHt9mLzDv2 0NjlwAJFvFVrAHtnQjWxAZVDlem/TVM42/hMuyERPKCvzz4j+NPmcVLyW4goQJ0zKSrj t6xDdeePUSf34ClVEDgdDVUqAIj78SGeyVzO4TuVAA8prlP8TCbd43LwrU71umqs10/v PB12LHNflbfbQPaiHisOI1DJChxW8GiDQXjt8ZCaYeKeEpMo6ITA1EPjKE5u17dpCoWw 2ONA7YeHcVNGS307V88fgL9cJKZynwa/egYXPdVtex7dz6w3w5AjrtxgFSA6gYa/ZlDD FTRA== X-Gm-Message-State: AOAM532aMoqETkQwLuLwzE8marVsaQROzorde53vMs9KEEmnz+DMncC9 w98BlMKx3NALig13TDRwWT+KMwQopYg= X-Google-Smtp-Source: ABdhPJwrA2Ye8lQKXs4Na/JBGw0XObFmbagznt9L0j2/sGFq66Ydw0RP2H6c5lVL7u7ji0c3jcwhvA== X-Received: by 2002:a05:600c:a4c:: with SMTP id c12mr3630833wmq.48.1644940794477; Tue, 15 Feb 2022 07:59:54 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id n11sm16726368wms.13.2022.02.15.07.59.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Feb 2022 07:59:53 -0800 (PST) Message-Id: <023fcd6e2b1163ab3d23b0d5933c14586d814ce0.1644940774.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 15 Feb 2022 15:59:29 +0000 Subject: [PATCH 19/23] fsm-health-win32: force shutdown daemon if worktree root moves Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Jeff Hostetler , Jeff Hostetler Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Jeff Hostetler From: Jeff Hostetler Force shutdown fsmonitor daemon if the worktree root directory is moved, renamed, or deleted. Signed-off-by: Jeff Hostetler --- compat/fsmonitor/fsm-health-win32.c | 133 ++++++++++++++++++++++++++++ 1 file changed, 133 insertions(+) diff --git a/compat/fsmonitor/fsm-health-win32.c b/compat/fsmonitor/fsm-health-win32.c index 3c3453369cd..2526ad9194f 100644 --- a/compat/fsmonitor/fsm-health-win32.c +++ b/compat/fsmonitor/fsm-health-win32.c @@ -14,7 +14,10 @@ enum interval_fn_ctx { CTX_INIT = 0, CTX_TERM, CTX_TIMER }; typedef int (interval_fn)(struct fsmonitor_daemon_state *state, enum interval_fn_ctx ctx); +static interval_fn has_worktree_moved; + static interval_fn *table[] = { + has_worktree_moved, NULL, /* must be last */ }; @@ -45,6 +48,12 @@ struct fsm_health_data HANDLE hHandles[1]; /* the array does not own these handles */ #define HEALTH_SHUTDOWN 0 int nr_handles; /* number of active event handles */ + + struct wt_moved + { + wchar_t wpath[MAX_PATH + 1]; + BY_HANDLE_FILE_INFORMATION bhfi; + } wt_moved; }; int fsm_health__ctor(struct fsmonitor_daemon_state *state) @@ -76,6 +85,130 @@ void fsm_health__dtor(struct fsmonitor_daemon_state *state) FREE_AND_NULL(state->health_data); } +static int lookup_bhfi(wchar_t *wpath, + BY_HANDLE_FILE_INFORMATION *bhfi) +{ + DWORD desired_access = FILE_LIST_DIRECTORY; + DWORD share_mode = + FILE_SHARE_WRITE | FILE_SHARE_READ | FILE_SHARE_DELETE; + HANDLE hDir; + + hDir = CreateFileW(wpath, desired_access, share_mode, NULL, + OPEN_EXISTING, FILE_FLAG_BACKUP_SEMANTICS, NULL); + if (hDir == INVALID_HANDLE_VALUE) { + error(_("[GLE %ld] health thread could not open '%ls'"), + GetLastError(), wpath); + return -1; + } + + if (!GetFileInformationByHandle(hDir, bhfi)) { + error(_("[GLE %ld] health thread getting BHFI for '%ls'"), + GetLastError(), wpath); + CloseHandle(hDir); + return -1; + } + + CloseHandle(hDir); + return 0; +} + +static int bhfi_eq(const BY_HANDLE_FILE_INFORMATION *bhfi_1, + const BY_HANDLE_FILE_INFORMATION *bhfi_2) +{ + return (bhfi_1->dwVolumeSerialNumber == bhfi_2->dwVolumeSerialNumber && + bhfi_1->nFileIndexHigh == bhfi_2->nFileIndexHigh && + bhfi_1->nFileIndexLow == bhfi_2->nFileIndexLow); +} + +/* + * Shutdown if the original worktree root directory been deleted, + * moved, or renamed? + * + * Since the main thread did a "chdir(getenv($HOME))" and our CWD + * is not in the worktree root directory and because the listener + * thread added FILE_SHARE_DELETE to the watch handle, it is possible + * for the root directory to be moved or deleted while we are still + * watching it. We want to detect that here and force a shutdown. + * + * Granted, a delete MAY cause some operations to fail, such as + * GetOverlappedResult(), but it is not guaranteed. And because + * ReadDirectoryChangesW() only reports on changes *WITHIN* the + * directory, not changes *ON* the directory, our watch will not + * receive a delete event for it. + * + * A move/rename of the worktree root will also not generate an event. + * And since the listener thread already has an open handle, it may + * continue to receive events for events within the directory. + * However, the pathname of the named-pipe was constructed using the + * original location of the worktree root. (Remember named-pipes are + * stored in the NPFS and not in the actual file system.) Clients + * trying to talk to the worktree after the move/rename will not + * reach our daemon process, since we're still listening on the + * pipe with original path. + * + * Furthermore, if the user does something like: + * + * $ mv repo repo.old + * $ git init repo + * + * A new daemon cannot be started in the new instance of "repo" + * because the named-pipe is still being used by the daemon on + * the original instance. + * + * So, detect move/rename/delete and shutdown. This should also + * handle unsafe drive removal. + * + * We use the file system unique ID to distinguish the original + * directory instance from a new instance and force a shutdown + * if the unique ID changes. + * + * Since a worktree move/rename/delete/unmount doesn't happen + * that often (and we can't get an immediate event anyway), we + * use a timeout and periodically poll it. + */ +static int has_worktree_moved(struct fsmonitor_daemon_state *state, + enum interval_fn_ctx ctx) +{ + struct fsm_health_data *data = state->health_data; + BY_HANDLE_FILE_INFORMATION bhfi; + int r; + + switch (ctx) { + case CTX_TERM: + return 0; + + case CTX_INIT: + if (xutftowcs_path(data->wt_moved.wpath, + state->path_worktree_watch.buf) < 0) { + error(_("could not convert to wide characters: '%s'"), + state->path_worktree_watch.buf); + return -1; + } + + /* + * On the first call we lookup the unique sequence ID for + * the worktree root directory. + */ + return lookup_bhfi(data->wt_moved.wpath, &data->wt_moved.bhfi); + + case CTX_TIMER: + r = lookup_bhfi(data->wt_moved.wpath, &bhfi); + if (r) + return r; + if (!bhfi_eq(&data->wt_moved.bhfi, &bhfi)) { + error(_("BHFI changed '%ls'"), data->wt_moved.wpath); + return -1; + } + return 0; + + default: + die("unhandled case in 'has_worktree_moved': %d", + (int)ctx); + } + + return 0; +} + void fsm_health__loop(struct fsmonitor_daemon_state *state) { struct fsm_health_data *data = state->health_data; From patchwork Tue Feb 15 15:59:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff Hostetler X-Patchwork-Id: 12747273 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2B10BC43217 for ; Tue, 15 Feb 2022 16:00:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241221AbiBOQAs (ORCPT ); Tue, 15 Feb 2022 11:00:48 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:56110 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241157AbiBOQAa (ORCPT ); Tue, 15 Feb 2022 11:00:30 -0500 Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E5657C087C for ; Tue, 15 Feb 2022 07:59:56 -0800 (PST) Received: by mail-wr1-x429.google.com with SMTP id j26so22058189wrb.7 for ; Tue, 15 Feb 2022 07:59:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=bVQOuNDx/E8BIA67X3H7/Oi34m9u80FBN5XoTePZszA=; b=SoUkOs0YOari/H9tPk2p3RtCyu/V2RakzgYl8UU5M4bPSW/pfrP8WNDGH9rYSd2dA8 eXincdf69izbIn1wrmhrxK/I1eYQbWEOk66+MAqsTxocbvEUwhciS0mGX7khpvLK+DwB fhzGLaKqwI1H8niWaMa/9r5ymDiMPEnsXGzYY7Y+0zHy9Wz3zkXz/aITtttxSR+FnLlr 0rnOscnZ3VAXnUT9Q4XSZZISKtxr4Ty1KiKRTcuoZ3vwT/HihiyHWqj4jX5FndSjLugh YRsbCcMenJyPoUF+85XDgLxx5dwAKeL3q6w2FcspWaJTS0q3EOGkF7b4P0+n2d+05O5g Dinw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=bVQOuNDx/E8BIA67X3H7/Oi34m9u80FBN5XoTePZszA=; b=WPxQkciAcfsGx6+uNcIGw2lpPzM4ySaqq6MN8WE9PeJY1rXTKmunfs9jGfb5p7Hjku vvMiKbTTkqXVs5l7E/H3FywHcG5lG/3eRpy6B4wN2sITLeESr9MM9ExqiW/C3ZkssyFu k3INBNCD7ehF0C9s7YDo11xwV0OUH8oSGNUu3Di6jo8TQyw5+ABAW5ZOayTJ6AJM7gCF QDHr0sX7ORgBJ/OWF5xLRZ+qFSADpY0/5tgAEm9Yi6tAAGkx+hb7jXGpFvshGo1Ju4lu GVx30BZOTvol78YPSKgqjACJpp2cUfqLQ/QLEzayMDfd+EC6RjA7e4jCVFkodH6wXzh4 dksQ== X-Gm-Message-State: AOAM532S3KPfRHOudoEOgJvLcQyDUPtjAq0G3BNz3uBCcRubT/JWWlR0 YpFtJ0MmFQ2lUfoMADCJfD/3wcHmf6I= X-Google-Smtp-Source: ABdhPJyC15If1TSbIFmkDNdX2GZ2TFzbJ1MemMY1AFSMC/aykzkOedo2M7pJuXyEcpyPqrfxljiYPQ== X-Received: by 2002:adf:cd0f:: with SMTP id w15mr3655447wrm.335.1644940795325; Tue, 15 Feb 2022 07:59:55 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id r12sm10854977wmq.33.2022.02.15.07.59.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Feb 2022 07:59:54 -0800 (PST) Message-Id: <496b3da35d0f9b8c68c6bc6c2bf2ca93ffe224b9.1644940774.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 15 Feb 2022 15:59:30 +0000 Subject: [PATCH 20/23] fsm-listen-darwin: shutdown daemon if worktree root is moved/renamed Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Jeff Hostetler , Jeff Hostetler Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Jeff Hostetler From: Jeff Hostetler Teach the listener thread to shutdown the daemon if the spelling of the worktree root directory changes. Signed-off-by: Jeff Hostetler --- compat/fsmonitor/fsm-listen-darwin.c | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/compat/fsmonitor/fsm-listen-darwin.c b/compat/fsmonitor/fsm-listen-darwin.c index 87a8476b09f..d3afbbc53d6 100644 --- a/compat/fsmonitor/fsm-listen-darwin.c +++ b/compat/fsmonitor/fsm-listen-darwin.c @@ -178,6 +178,11 @@ static void log_flags_set(const char *path, const FSEventStreamEventFlags flag) strbuf_release(&msg); } +static int ef_is_root_changed(const FSEventStreamEventFlags ef) +{ + return (ef & kFSEventStreamEventFlagRootChanged); +} + static int ef_is_root_delete(const FSEventStreamEventFlags ef) { return (ef & kFSEventStreamEventFlagItemIsDir && @@ -287,6 +292,26 @@ static void fsevent_callback(ConstFSEventStreamRef streamRef, continue; } + if (ef_is_root_changed(event_flags[k])) { + /* + * The spelling of the pathname of the root directory + * has changed. This includes the name of the root + * directory itself of of any parent directory in the + * path. + * + * (There may be other conditions that throw this, + * but I couldn't find any information on it.) + * + * Force a shutdown now and avoid things getting + * out of sync. The Unix domain socket is inside + * the .git directory and a spelling change will make + * it hard for clients to rendezvous with us. + */ + trace_printf_key(&trace_fsmonitor, + "event: root changed"); + goto force_shutdown; + } + if (ef_ignore_xattr(event_flags[k])) { trace_printf_key(&trace_fsmonitor, "ignore-xattr: '%s', flags=0x%x", From patchwork Tue Feb 15 15:59:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff Hostetler X-Patchwork-Id: 12747275 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 19D21C4332F for ; Tue, 15 Feb 2022 16:00:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241215AbiBOQAo (ORCPT ); Tue, 15 Feb 2022 11:00:44 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:56124 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233619AbiBOQAa (ORCPT ); Tue, 15 Feb 2022 11:00:30 -0500 Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 461C3C1CB8 for ; Tue, 15 Feb 2022 07:59:58 -0800 (PST) Received: by mail-wr1-x42c.google.com with SMTP id j26so22058284wrb.7 for ; Tue, 15 Feb 2022 07:59:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=O9uSUdVmjl0GNVDtJppFKUNv7/uBemAS63Cuf7uBWG0=; b=bphIfmQCkFTz+JHHrzm9dXUpi0qcNkeEeleizjCbiN5MxvdNnV5wQPFYFM+HFKw4DZ 8t8d/ObbrhAaZWOKNIznW9ORajvu3RmoPxKlr5cDliYAvfeqMgDyBduDXa/9Tbglx4TT UqSqL0gUzKr0t30s1Y3TxwvQG+WGu3+DeSemCPtVxTV3XOu4KtIf8QzWsqPNc+cAbwJD sZ08gtdXQTX4IoTtBTiY5yWNRkW1huLo1hA9WqV16HypWcZox3QtyR8rsspLASJaU75R Mr2jrSPgTAYJ/FC1NVaEZYv1Vw8AW2gqsSyDIvQyWzBplviH25l7/7QTWrLo89hEegwg 72tA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=O9uSUdVmjl0GNVDtJppFKUNv7/uBemAS63Cuf7uBWG0=; b=1m10lhyYH7Fb+RO6wH1J8snY+q5ovJg3IP+TfdiwdoEl9C3H/Zm8EElGsx5U1Z0WEj v6rbebAmuXFF9tyVBkBdEWYncyN8eiAq5hav0a13meVt1/zM3O5bmUOhYJcZhjQuPWse A58eyEA5rdYhfTZmyu24ZZEMpeYqbq9KPqoDhhKJHGKZIFyD5BwScOvhl89pk8PRLXEc jkyTJLrGx93Khsfhs7OZBOvWSAqIsyzRuXrMX4j7LAVO5lv/IfVkvqiMAOFi9fMUpt4R daH7FgRpDhgeGahBsW7iUVBo0ypS5zSKGMG9C3chgNafMsEPiKh7itbH4Fx5+jp2lISw stYQ== X-Gm-Message-State: AOAM5334eyZGqOjdZJjQNlxQg29RZgpdmoeRUFVIMqUEyD3pS4U9w/zQ XfaikW/Or/t/V9BNBlKGr465WmCssg8= X-Google-Smtp-Source: ABdhPJys/vrUUZfQvhLMAkUD1Kjs1uAjytM+Br6kxgoZSrRHvkHeSeH97eZADz2zZJQcd8FCHbR30g== X-Received: by 2002:a5d:554e:: with SMTP id g14mr3680888wrw.289.1644940796166; Tue, 15 Feb 2022 07:59:56 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id e8sm18097689wru.37.2022.02.15.07.59.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Feb 2022 07:59:55 -0800 (PST) Message-Id: <07a9c7542be79fb60861253848eca258bec87694.1644940774.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Tue, 15 Feb 2022 15:59:31 +0000 Subject: [PATCH 21/23] fsmonitor: optimize processing of directory events Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Jeff Hostetler , Jeff Hostetler Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Jeff Hostetler From: Jeff Hostetler Teach Git to perform binary search over the cache-entries for a directory notification and then linearly scan forward to find the immediate children. Previously, when the FSMonitor reported a modified directory Git would perform a linear search on the entire cache-entry array for all entries matching that directory prefix and invalidate them. Since the cache-entry array is already sorted, we can use a binary search to find the first matching entry and then only linearly walk forward and invalidate entries until the prefix changes. Also, the original code would invalidate anything having the same directory prefix. Since a directory event should only be received for items that are immediately within the directory (and not within sub-directories of it), only invalidate those entries and not the whole subtree. Signed-off-by: Jeff Hostetler --- fsmonitor.c | 71 ++++++++++++++++++++++++++++++++++++++++------------- 1 file changed, 54 insertions(+), 17 deletions(-) diff --git a/fsmonitor.c b/fsmonitor.c index 8e3499d0667..be08861a18e 100644 --- a/fsmonitor.c +++ b/fsmonitor.c @@ -184,30 +184,68 @@ static int query_fsmonitor_hook(struct repository *r, static void fsmonitor_refresh_callback(struct index_state *istate, char *name) { int i, len = strlen(name); - if (name[len - 1] == '/') { + int pos = index_name_pos(istate, name, len); + + trace_printf_key(&trace_fsmonitor, + "fsmonitor_refresh_callback '%s' (pos %d)", + name, pos); + if (name[len - 1] == '/') { /* - * TODO We should binary search to find the first path with - * TODO this directory prefix. Then linearly update entries - * TODO while the prefix matches. Taking care to search without - * TODO the trailing slash -- because '/' sorts after a few - * TODO interesting special chars, like '.' and ' '. + * The daemon can decorate directory events, such as + * moves or renames, with a trailing slash if the OS + * FS Event contains sufficient information, such as + * MacOS. + * + * Use this to invalidate the entire cone under that + * directory. + * + * We do not expect an exact match because the index + * does not normally contain directory entries, so we + * start at the insertion point and scan. */ + if (pos < 0) + pos = -pos - 1; /* Mark all entries for the folder invalid */ - for (i = 0; i < istate->cache_nr; i++) { - if (istate->cache[i]->ce_flags & CE_FSMONITOR_VALID && - starts_with(istate->cache[i]->name, name)) - istate->cache[i]->ce_flags &= ~CE_FSMONITOR_VALID; + for (i = pos; i < istate->cache_nr; i++) { + if (!starts_with(istate->cache[i]->name, name)) + break; + istate->cache[i]->ce_flags &= ~CE_FSMONITOR_VALID; } - /* Need to remove the / from the path for the untracked cache */ + + /* + * We need to remove the traling "/" from the path + * for the untracked cache. + */ name[len - 1] = '\0'; + } else if (pos >= 0) { + /* + * We have an exact match for this path and can just + * invalidate it. + */ + istate->cache[pos]->ce_flags &= ~CE_FSMONITOR_VALID; } else { - int pos = index_name_pos(istate, name, strlen(name)); - - if (pos >= 0) { - struct cache_entry *ce = istate->cache[pos]; - ce->ce_flags &= ~CE_FSMONITOR_VALID; + /* + * The path is not a tracked file -or- it is a + * directory event on a platform that cannot + * distinguish between file and directory events in + * the event handler, such as Windows. + * + * Scan as if it is a directory and invalidate the + * cone under it. (But remember to ignore items + * between "name" and "name/", such as "name-" and + * "name.". + */ + pos = -pos - 1; + + for (i = pos; i < istate->cache_nr; i++) { + if (!starts_with(istate->cache[i]->name, name)) + break; + if ((unsigned char)istate->cache[i]->name[len] > '/') + break; + if (istate->cache[i]->name[len] == '/') + istate->cache[i]->ce_flags &= ~CE_FSMONITOR_VALID; } } @@ -215,7 +253,6 @@ static void fsmonitor_refresh_callback(struct index_state *istate, char *name) * Mark the untracked cache dirty even if it wasn't found in the index * as it could be a new untracked file. */ - trace_printf_key(&trace_fsmonitor, "fsmonitor_refresh_callback '%s'", name); untracked_cache_invalidate_path(istate, name, 0); } From patchwork Tue Feb 15 15:59:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff Hostetler X-Patchwork-Id: 12747272 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 533C6C433FE for ; Tue, 15 Feb 2022 16:00:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241208AbiBOQAl (ORCPT ); Tue, 15 Feb 2022 11:00:41 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:56368 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241162AbiBOQAa (ORCPT ); Tue, 15 Feb 2022 11:00:30 -0500 Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2BD49C1CBC for ; Tue, 15 Feb 2022 07:59:59 -0800 (PST) Received: by mail-wr1-x431.google.com with SMTP id j26so22058351wrb.7 for ; Tue, 15 Feb 2022 07:59:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=zNfbcs/Jm+I3NHuyz1kCz41o5u4WnxVLf6/YW0I/FMw=; b=IxP5/pA1C8Gh8BOYzBIvepkf6Y66w8uNmqoY1hea+zwNBb4ubAM1VZXcbQ/t6Ale5B AOCkuFWKqODeMAcqQqxZFB40/TJ/Ekwjyz24TF7GtTrIVS9cg+FHYQdsKimS48dxPBjN dPQ9yIdv3f9Gr+/DDwuBSmzpos+wC+CpnuHtAsb3OhLxXaoCo+5RQUIM3+um63XrIPod Ck6iXDICXkqX+wC49fjGPvMGCVR2X5BzegW4OGqIKAYO+oTnJFi8E7dZ+o91phFe0BCm TmLrymMUqSlvKj+WSQHUZkcb0A8gkvkHjj2D09+n1v6m66kP66wEZQ3tWru6CICwJkEa Xzag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=zNfbcs/Jm+I3NHuyz1kCz41o5u4WnxVLf6/YW0I/FMw=; b=xPjxUK+FdBntY4L/2ktnQca6C2hf0RdTM4vKpmTHjDZLt8Y9TMuldWm8VqzXaNrTOl ZiFr/tNjmm9LcLiaeBDDJMzjTlGrOovhR+vTCpMiuULwhrlc2xWaSZlDbDozmLKdouje v9MUjQV11XN0skY/e03JTF4oZo6Mc5tCbt2L1fyv2FiSnagjL0Krg6QFEsUcTQbTg9Xd GWDVqy5wCa4jnx7hmu7IPRRNJneGvcNs3bR5Sbk9gFP6moKtMLVBliBYbC95DztMoJpH 0mBvisNNF8NPy0HU1dDidXxgBbb97UAIFDcbLfGGXcMZE5gkrKyunTBusrdPIgIxjKDb FpBg== X-Gm-Message-State: AOAM531UOIo/7Co2e/tIr/DFehtQJRNc2+iuiQITtteVcMcYCsraJcQi yf5OCyG6+aqDGTvEryDSRTZm+mIndLs= X-Google-Smtp-Source: ABdhPJzcOLVYAXkhDmGqITqYqgKZe8QnQlqupDgenfrrFXZV6ylCk7bQkCBod13p1fkA9xW7kaK6Yg== X-Received: by 2002:adf:d08d:: with SMTP id y13mr3651409wrh.445.1644940797583; Tue, 15 Feb 2022 07:59:57 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id x7sm25722941wro.21.2022.02.15.07.59.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Feb 2022 07:59:57 -0800 (PST) Message-Id: In-Reply-To: References: Date: Tue, 15 Feb 2022 15:59:32 +0000 Subject: [PATCH 22/23] t7527: FSMonitor tests for directory moves Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Jeff Hostetler , Jeff Hostetler Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Jeff Hostetler From: Jeff Hostetler Create unit tests to move a directory. Verify that `git status` gives the same result with and without FSMonitor enabled. NEEDSWORK: This test exposes a bug in the untracked-cache on Windows when FSMonitor is disabled. These are commented out for the moment. Signed-off-by: Jeff Hostetler --- t/t7527-builtin-fsmonitor.sh | 39 ++++++++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+) diff --git a/t/t7527-builtin-fsmonitor.sh b/t/t7527-builtin-fsmonitor.sh index c0145544ccb..408d614b28b 100755 --- a/t/t7527-builtin-fsmonitor.sh +++ b/t/t7527-builtin-fsmonitor.sh @@ -261,6 +261,16 @@ test_expect_success 'setup' ' trace* EOF + mkdir -p T1/T2/T3/T4 && + echo 1 >T1/F1 && + echo 1 >T1/T2/F1 && + echo 1 >T1/T2/T3/F1 && + echo 1 >T1/T2/T3/T4/F1 && + echo 2 >T1/F2 && + echo 2 >T1/T2/F2 && + echo 2 >T1/T2/T3/F2 && + echo 2 >T1/T2/T3/T4/F2 && + git -c core.fsmonitor=false add . && test_tick && git -c core.fsmonitor=false commit -m initial && @@ -354,6 +364,19 @@ verify_status () { echo HELLO AFTER } +move_directory_contents_deeper() { + mkdir T1/_new_ + mv T1/[A-Z]* T1/_new_ +} + +move_directory_up() { + mv T1/T2/T3 T1 +} + +move_directory() { + mv T1/T2/T3 T1/T2/NewT3 +} + # The next few test cases confirm that our fsmonitor daemon sees each type # of OS filesystem notification that we care about. At this layer we just # ensure we are getting the OS notifications and do not try to confirm what @@ -685,6 +708,22 @@ do matrix_try $uc_val $fsm_val file_to_directory matrix_try $uc_val $fsm_val directory_to_file + # NEEDSWORK: On Windows the untracked-cache is buggy when FSMonitor + # is DISABLED. Turn off a few test that cause it problems until + # we can debug it. + # + try_moves="true" + test_have_prereq UNTRACKED_CACHE,WINDOWS && \ + test $uc_val = true && \ + test $fsm_val = false && \ + try_moves="false" + if test $try_moves = true + then + matrix_try $uc_val $fsm_val move_directory_contents_deeper + matrix_try $uc_val $fsm_val move_directory_up + matrix_try $uc_val $fsm_val move_directory + fi + if test $fsm_val = true then test_expect_success "Matrix[uc:$uc_val][fsm:$fsm_val] disable fsmonitor at end" ' From patchwork Tue Feb 15 15:59:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeff Hostetler X-Patchwork-Id: 12747274 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 49DB8C433F5 for ; Tue, 15 Feb 2022 16:00:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241231AbiBOQAt (ORCPT ); Tue, 15 Feb 2022 11:00:49 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:56468 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241165AbiBOQAa (ORCPT ); Tue, 15 Feb 2022 11:00:30 -0500 Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com [IPv6:2a00:1450:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2D249C2E7D for ; Tue, 15 Feb 2022 08:00:00 -0800 (PST) Received: by mail-wr1-x433.google.com with SMTP id d27so29121016wrb.5 for ; Tue, 15 Feb 2022 08:00:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=3IAiMHgRwhfbvkUdsr17fwV8M0Qk+EnCGfvQEEa+RAk=; b=RXhEmrZEDSRTnfuDRizdAHxSq2ka8qnyCQ7Icvp6x8Bwrkb/RuVXDXYOGhtMl1d2IY NKWvUXM/C5X9XdppaQ9YmcANCslVh/YHlmWE7o7qyvIxfMwRvUliVD+41hulYoMzn3p3 Pl7qVemnnutX91xFbeGVbsfMgRdNVTckOi6q1P5LapneHS7pad4BHLIaF92X1BM6Rw2W fF2yNXCXsGKntgMUOZkwy6RclGg8uKJA+h8OO+U7P06iFj6361lXKFaJz65UkjD/Qm9o zXW+figy58C+cEWQ4Nqs1deOOC8bJfnJko3ktAyAM3Vdgc0w+aH1MHlZ+/WQlyV6CtoO C8zQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=3IAiMHgRwhfbvkUdsr17fwV8M0Qk+EnCGfvQEEa+RAk=; b=ZPlqu66Mni+o/jnJao9Yv4ep/EYCzROJgsCB6RIxPm86h8YAUh8bZI2uKUldZ4KXfN OBwaaM98kHqquAvJrtaMc5u6aW3EvAHWzxkN2v29GFwpNck0cu/C7o7F9q8qIq2hbXlw CwzxEnR4vUYlYf5LAIgmFHfHStO10Xb4BlMlN+NO4hodbXW/K6489oIdhDmQcSjZBHoa lR9cQ9p+JNxd22GkZtPA6vOl3Y6TzE3+n6EuBfsfTAoo0IWBkgZT4VSN2FpZGPeUTTPV glDQ/eU2t5IHx7w457QM2dxKyHe91n0rDs8cTS6aTvZO3w7D8sMm5J4uvn+8QSalAUXc O6+Q== X-Gm-Message-State: AOAM532ALexuDfDlXeixZSfpA3N4pX4cejF7hr4igSjraWpIUEoOhmHX ZiL0TocOV9yebC262KWxh9RyYNvmAJA= X-Google-Smtp-Source: ABdhPJzwLcXydnn1drnmk+L9+Jgooh7E8OnpDelYSTRoP8nFuim5DKxsc7o3nXp+BqlXUMUC6KBZOg== X-Received: by 2002:adf:f9cf:: with SMTP id w15mr3793447wrr.109.1644940798508; Tue, 15 Feb 2022 07:59:58 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id h6sm6626212wmq.8.2022.02.15.07.59.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Feb 2022 07:59:58 -0800 (PST) Message-Id: In-Reply-To: References: Date: Tue, 15 Feb 2022 15:59:33 +0000 Subject: [PATCH 23/23] t/perf/p7527: add perf test for builtin FSMonitor Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Jeff Hostetler , Jeff Hostetler Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Jeff Hostetler From: Jeff Hostetler Signed-off-by: Jeff Hostetler --- t/perf/p7527-builtin-fsmonitor.sh | 257 ++++++++++++++++++++++++++++++ 1 file changed, 257 insertions(+) create mode 100755 t/perf/p7527-builtin-fsmonitor.sh diff --git a/t/perf/p7527-builtin-fsmonitor.sh b/t/perf/p7527-builtin-fsmonitor.sh new file mode 100755 index 00000000000..9338b9ea008 --- /dev/null +++ b/t/perf/p7527-builtin-fsmonitor.sh @@ -0,0 +1,257 @@ +#!/bin/sh + +test_description="Perf test for the builtin FSMonitor" + +. ./perf-lib.sh + +if ! test_have_prereq FSMONITOR_DAEMON +then + skip_all="fsmonitor--daemon is not supported on this platform" + test_done +fi + +test_lazy_prereq UNTRACKED_CACHE ' + { git update-index --test-untracked-cache; ret=$?; } && + test $ret -ne 1 +' + +# Lie to perf-lib and ask for a new empty repo and avoid +# the complaints about GIT_PERF_REPO not being big enough +# the perf hit when GIT_PERF_LARGE_REPO is copied into +# the trash directory. +# +# NEEDSWORK: It would be nice if perf-lib had an option to +# "borrow" an existing large repo (especially for gigantic +# monorepos) and use it in-place. For now, fake it here. +# +test_perf_fresh_repo + + +# Use a generated synthetic monorepo. If it doesn't exist, we will +# generate it. If it does exist, we will put it in a known state +# before we start our timings. +# +PARAM_D=5 +PARAM_W=10 +PARAM_F=9 + +PARAMS="$PARAM_D"."$PARAM_W"."$PARAM_F" + +BALLAST_BR=p0006-ballast +export BALLAST_BR + +TMP_BR=tmp_br +export TMP_BR + +REPO=../repos/gen-many-files-"$PARAMS".git +export REPO + +if ! test -d $REPO +then + (cd ../repos; ./many-files.sh -d $PARAM_D -w $PARAM_W -f $PARAM_F) +fi + + +enable_uc () { + git -C $REPO config core.untrackedcache true + git -C $REPO update-index --untracked-cache + git -C $REPO status >/dev/null 2>&1 +} + +disable_uc () { + git -C $REPO config core.untrackedcache false + git -C $REPO update-index --no-untracked-cache + git -C $REPO status >/dev/null 2>&1 +} + +start_fsm () { + git -C $REPO fsmonitor--daemon start + git -C $REPO fsmonitor--daemon status + git -C $REPO config core.fsmonitor true + git -C $REPO update-index --fsmonitor + git -C $REPO status >/dev/null 2>&1 +} + +stop_fsm () { + git -C $REPO config --unset core.fsmonitor + git -C $REPO update-index --no-fsmonitor + test_might_fail git -C $REPO fsmonitor--daemon stop 2>/dev/null + git -C $REPO status >/dev/null 2>&1 +} + + +# Ensure that FSMonitor is turned off on the borrowed repo. +# +test_expect_success "Setup borrowed repo (fsm+uc)" " + stop_fsm && + disable_uc +" + +# Also ensure that it starts in a known state. +# +# Because we assume that $GIT_PERF_REPEAT_COUNT > 1, we are not going to time +# the ballast checkout, since only the first invocation does any work and the +# subsequent ones just print "already on branch" and quit, so the reported +# time is not useful. +# +# Create a temp branch and do all work relative to it so that we don't +# accidentially alter the real ballast branch. +# +test_expect_success "Setup borrowed repo (temp ballast branch)" " + test_might_fail git -C $REPO checkout $BALLAST_BR && + test_might_fail git -C $REPO reset --hard && + git -C $REPO clean -d -f && + test_might_fail git -C $REPO branch -D $TMP_BR && + git -C $REPO branch $TMP_BR $BALLAST_BR && + git -C $REPO checkout $TMP_BR +" + + +echo Data >data.txt + +# NEEDSWORK: We assume that $GIT_PERF_REPEAT_COUNT > 1. With +# FSMonitor enabled, we can get a skewed view of status times, since +# the index MAY (or may not) be updated after the first invocation +# which will update the FSMonitor Token, so the subsequent invocations +# may get a smaller response from the daemon. +# +do_status () { + msg=$1 + + test_perf "$msg" " + git -C $REPO status >/dev/null 2>&1 + " +} + +do_matrix () { + uc=$1 + fsm=$2 + + t="[uc $uc][fsm $fsm]" + MATRIX_BR="$TMP_BR-$uc-$fsm" + + test_expect_success "$t Setup matrix branch" " + git -C $REPO clean -d -f && + git -C $REPO checkout $TMP_BR && + test_might_fail git -C $REPO branch -D $MATRIX_BR && + git -C $REPO branch $MATRIX_BR $TMP_BR && + git -C $REPO checkout $MATRIX_BR + " + + if test $uc = true + then + enable_uc + else + disable_uc + fi + + if test $fsm = true + then + start_fsm + else + stop_fsm + fi + + do_status "$t status after checkout" + + # Modify many files in the matrix branch. + # Stage them. + # Commit them. + # Rollback. + # + test_expect_success "$t modify tracked files" " + find $REPO -name file1 -exec cp data.txt {} \\; + " + + do_status "$t status after big change" + + # Don't bother timing the "add" because _REPEAT_COUNT + # issue described above. + # + test_expect_success "$t add all" " + git -C $REPO add -A + " + + do_status "$t status after add all" + + test_expect_success "$t add dot" " + git -C $REPO add . + " + + do_status "$t status after add dot" + + test_expect_success "$t commit staged" " + git -C $REPO commit -a -m data + " + + do_status "$t status after commit" + + test_expect_success "$t reset HEAD~1 hard" " + git -C $REPO reset --hard HEAD~1 >/dev/null 2>&1 + " + + do_status "$t status after reset hard" + + # Create some untracked files. + # + test_expect_success "$t create untracked files" " + cp -R $REPO/ballast/dir1 $REPO/ballast/xxx1 + " + + do_status "$t status after create untracked files" + + # Remove the new untracked files. + # + test_expect_success "$t clean -df" " + git -C $REPO clean -d -f + " + + do_status "$t status after clean" + + if test $fsm = true + then + stop_fsm + fi +} + +# Begin testing each case in the matrix that we care about. +# +uc_values="false" +test_have_prereq UNTRACKED_CACHE && uc_values="false true" + +fsm_values="false true" + +for uc_val in $uc_values +do + for fsm_val in $fsm_values + do + do_matrix $uc_val $fsm_val + done +done + +cleanup () { + uc=$1 + fsm=$2 + + MATRIX_BR="$TMP_BR-$uc-$fsm" + + test_might_fail git -C $REPO branch -D $MATRIX_BR +} + + +# We're borrowing this repo. We should leave it in a clean state. +# +test_expect_success "Cleanup temp and matrix branches" " + git -C $REPO clean -d -f && + test_might_fail git -C $REPO checkout $BALLAST_BR && + test_might_fail git -C $REPO branch -D $TMP_BR && + for uc_val in $uc_values + do + for fsm_val in $fsm_values + do + cleanup $uc_val $fsm_val + done + done +" + +test_done