From patchwork Thu Dec 19 21:28:19 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Koji Nakamaru via GitGitGadget X-Patchwork-Id: 11304413 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BFEF41593 for ; Thu, 19 Dec 2019 21:28:33 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 9378E2467F for ; Thu, 19 Dec 2019 21:28:33 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="VsvgPUoO" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727260AbfLSV2d (ORCPT ); Thu, 19 Dec 2019 16:28:33 -0500 Received: from mail-ed1-f68.google.com ([209.85.208.68]:33383 "EHLO mail-ed1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727105AbfLSV2c (ORCPT ); Thu, 19 Dec 2019 16:28:32 -0500 Received: by mail-ed1-f68.google.com with SMTP id r21so6316359edq.0 for ; Thu, 19 Dec 2019 13:28:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=RjUWfqMLtbjNIPA16XEQPNcw4ayZ9uv9uVm9KQNo/JQ=; b=VsvgPUoODUZZ+2sHwbAIEWwiPIWjSgHNif5ONzQInZQtakDQ2JOx54WToVsPIUi8Ix UShDnkPs9ugR2yHFXJ1xUBzchSJICZAx1h30Je7X+VIJXG1zmXCLQIFET4yjMxFuFSXf WkSdpi4N5W7OTKA/BbwjUqfhxLMJcBGy1YiFf2Dj1Pd08f6VTE3+qpFKzhCjN6hP25mx K2bGyPdde6v2u0iqaoHou12gxsVoY8WItOayN4QjPvIArqIEgcHrOkBHIorokrltts10 B+//iG1IsTWSBL7dHjP/GGCpD45rIjR9gRSNF0A8dxPZsYgQgHVnsm4z6Tf4k2x71IFc wKQQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=RjUWfqMLtbjNIPA16XEQPNcw4ayZ9uv9uVm9KQNo/JQ=; b=UimJXsBDnLM5+pEtPKbSGbu5vn56pcFrHxrULMN/BDXWEVRngInF+ASAM1lsBQDzHV 15zgENCCTIRSGSC4j8rGmg11+9tBe10A5tt3OHuiZLn5iLtDddKujn2pbelnPmGe5oVe 5V9w0VL6qRnPi3HzhWpZ51Ws/aG4nRGPsnUjorheKZUthJzpNOiCdoy/W84VaLy/3hPu 4mznI/NRGQsOG4VKMChP0KkxNikEj96ghvVSsenFpdZqvBo2kUOY07J31uaYP7OwG4HS FLMtSCcYtnqgBYZQe1dfokIZT9U2BK+MpxpQfECcyt0hIKx9BEenw2s//+FTSN/UVrFR 5H5Q== X-Gm-Message-State: APjAAAVoQXcxRSZo8KEzQS6Lyt6+jjDcwSEdAv2qkNxR9lS1gdM1LRTF XGArI1g04VG3Lrcb77Z+9fU6DCug X-Google-Smtp-Source: APXvYqzUYJMVbb3CK3YXRxG5R0eVV8DxHWyVmT+DHoBp1FnKocSoA3u4Waf2A9UNb1WyXRZsZmVqyA== X-Received: by 2002:a17:906:9615:: with SMTP id s21mr12112694ejx.20.1576790909294; Thu, 19 Dec 2019 13:28:29 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id s19sm683911edr.55.2019.12.19.13.28.28 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 19 Dec 2019 13:28:28 -0800 (PST) Message-Id: <6d659b230239b530c90fbcc15c9a869f8d5dd5c5.1576790906.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Elijah Newren via GitGitGadget" Date: Thu, 19 Dec 2019 21:28:19 +0000 Subject: [PATCH v5 1/8] t3011: demonstrate directory traversal failures Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: blees@dcon.de, gitster@pobox.com, kyle@kyleam.com, sxlijin@gmail.com, Junio C Hamano , Elijah Newren Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Elijah Newren Add several tests demonstrating directory traversal failures of various sorts in dir.c (and one similar looking test that turns out to be a git_fnmatch bug). A lot of these tests look like near duplicates of each other, but an optimization path in dir.c to pre-descend into a common prefix and the specialized treatment of trailing slashes in dir.c mean the tiny differences are sometimes important and potentially cause different codepaths to be explored. Of the 7 failing tests, 2 are new to git-2.24.0 (tweaked by side effects of the en/clean-nested-with-ignored-topic); the other 5 also failed under git-2.23.0 and earlier. Signed-off-by: Elijah Newren --- ...common-prefixes-and-directory-traversal.sh | 209 ++++++++++++++++++ 1 file changed, 209 insertions(+) create mode 100755 t/t3011-common-prefixes-and-directory-traversal.sh diff --git a/t/t3011-common-prefixes-and-directory-traversal.sh b/t/t3011-common-prefixes-and-directory-traversal.sh new file mode 100755 index 0000000000..54f80c62b8 --- /dev/null +++ b/t/t3011-common-prefixes-and-directory-traversal.sh @@ -0,0 +1,209 @@ +#!/bin/sh + +test_description='directory traversal handling, especially with common prefixes' + +. ./test-lib.sh + +test_expect_success 'setup' ' + test_commit hello && + + >empty && + mkdir untracked_dir && + >untracked_dir/empty && + git init untracked_repo && + >untracked_repo/empty && + + cat <<-EOF >.gitignore && + ignored + an_ignored_dir/ + EOF + mkdir an_ignored_dir && + mkdir an_untracked_dir && + >an_ignored_dir/ignored && + >an_ignored_dir/untracked && + >an_untracked_dir/ignored && + >an_untracked_dir/untracked +' + +test_expect_success 'git ls-files -o shows the right entries' ' + cat <<-EOF >expect && + .gitignore + actual + an_ignored_dir/ignored + an_ignored_dir/untracked + an_untracked_dir/ignored + an_untracked_dir/untracked + empty + expect + untracked_dir/empty + untracked_repo/ + EOF + git ls-files -o >actual && + test_cmp expect actual +' + +test_expect_success 'git ls-files -o --exclude-standard shows the right entries' ' + cat <<-EOF >expect && + .gitignore + actual + an_untracked_dir/untracked + empty + expect + untracked_dir/empty + untracked_repo/ + EOF + git ls-files -o --exclude-standard >actual && + test_cmp expect actual +' + +test_expect_success 'git ls-files -o untracked_dir recurses' ' + echo untracked_dir/empty >expect && + git ls-files -o untracked_dir >actual && + test_cmp expect actual +' + +test_expect_success 'git ls-files -o untracked_dir/ recurses' ' + echo untracked_dir/empty >expect && + git ls-files -o untracked_dir/ >actual && + test_cmp expect actual +' + +test_expect_success 'git ls-files -o --directory untracked_dir does not recurse' ' + echo untracked_dir/ >expect && + git ls-files -o --directory untracked_dir >actual && + test_cmp expect actual +' + +test_expect_failure 'git ls-files -o --directory untracked_dir/ does not recurse' ' + echo untracked_dir/ >expect && + git ls-files -o --directory untracked_dir/ >actual && + test_cmp expect actual +' + +test_expect_success 'git ls-files -o untracked_repo does not recurse' ' + echo untracked_repo/ >expect && + git ls-files -o untracked_repo >actual && + test_cmp expect actual +' + +test_expect_failure 'git ls-files -o untracked_repo/ does not recurse' ' + echo untracked_repo/ >expect && + git ls-files -o untracked_repo/ >actual && + test_cmp expect actual +' + +test_expect_failure 'git ls-files -o untracked_dir untracked_repo recurses into untracked_dir only' ' + cat <<-EOF >expect && + untracked_dir/empty + untracked_repo/ + EOF + git ls-files -o untracked_dir untracked_repo >actual && + test_cmp expect actual +' + +test_expect_success 'git ls-files -o untracked_dir/ untracked_repo/ recurses into untracked_dir only' ' + cat <<-EOF >expect && + untracked_dir/empty + untracked_repo/ + EOF + git ls-files -o untracked_dir/ untracked_repo/ >actual && + test_cmp expect actual +' + +test_expect_failure 'git ls-files -o --directory untracked_dir untracked_repo does not recurse' ' + cat <<-EOF >expect && + untracked_dir/ + untracked_repo/ + EOF + git ls-files -o --directory untracked_dir untracked_repo >actual && + test_cmp expect actual +' + +test_expect_success 'git ls-files -o --directory untracked_dir/ untracked_repo/ does not recurse' ' + cat <<-EOF >expect && + untracked_dir/ + untracked_repo/ + EOF + git ls-files -o --directory untracked_dir/ untracked_repo/ >actual && + test_cmp expect actual +' + +test_expect_success 'git ls-files -o .git shows nothing' ' + git ls-files -o .git >actual && + test_must_be_empty actual +' + +test_expect_failure 'git ls-files -o .git/ shows nothing' ' + git ls-files -o .git/ >actual && + test_must_be_empty actual +' + +test_expect_success FUNNYNAMES 'git ls-files -o untracked_* recurses appropriately' ' + mkdir "untracked_*" && + >"untracked_*/empty" && + + cat <<-EOF >expect && + untracked_*/empty + untracked_dir/empty + untracked_repo/ + EOF + git ls-files -o "untracked_*" >actual && + test_cmp expect actual +' + +# It turns out fill_directory returns the right paths, but ls-files' post-call +# filtering in show_dir_entry() via calling dir_path_match() which ends up +# in git_fnmatch() has logic for PATHSPEC_ONESTAR that assumes the pathspec +# must match the full path; it doesn't check it for matching a leading +# directory. +test_expect_failure FUNNYNAMES 'git ls-files -o untracked_*/ recurses appropriately' ' + cat <<-EOF >expect && + untracked_*/empty + untracked_dir/empty + untracked_repo/ + EOF + git ls-files -o "untracked_*/" >actual && + test_cmp expect actual +' + +test_expect_success FUNNYNAMES 'git ls-files -o --directory untracked_* does not recurse' ' + cat <<-EOF >expect && + untracked_*/ + untracked_dir/ + untracked_repo/ + EOF + git ls-files -o --directory "untracked_*" >actual && + test_cmp expect actual +' + +test_expect_success FUNNYNAMES 'git ls-files -o --directory untracked_*/ does not recurse' ' + cat <<-EOF >expect && + untracked_*/ + untracked_dir/ + untracked_repo/ + EOF + git ls-files -o --directory "untracked_*/" >actual && + test_cmp expect actual +' + +test_expect_success 'git ls-files -o consistent between one or two dirs' ' + git ls-files -o --exclude-standard an_ignored_dir/ an_untracked_dir/ >tmp && + ! grep ^an_ignored_dir/ tmp >expect && + git ls-files -o --exclude-standard an_ignored_dir/ >actual && + test_cmp expect actual +' + +# ls-files doesn't have a way to request showing both untracked and ignored +# files at the same time, so use `git status --ignored` +test_expect_failure 'git status --ignored shows same files under dir with or without pathspec' ' + cat <<-EOF >expect && + ?? an_untracked_dir/ + !! an_untracked_dir/ignored + EOF + git status --porcelain --ignored >output && + grep an_untracked_dir output >expect && + git status --porcelain --ignored an_untracked_dir/ >actual && + test_cmp expect actual +' + +test_done From patchwork Thu Dec 19 21:28:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Koji Nakamaru via GitGitGadget X-Patchwork-Id: 11304423 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1F1591593 for ; Thu, 19 Dec 2019 21:28:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id F0BA624679 for ; Thu, 19 Dec 2019 21:28:37 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="NeWv1e5Q" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727186AbfLSV2c (ORCPT ); Thu, 19 Dec 2019 16:28:32 -0500 Received: from mail-ed1-f65.google.com ([209.85.208.65]:43138 "EHLO mail-ed1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726880AbfLSV2c (ORCPT ); Thu, 19 Dec 2019 16:28:32 -0500 Received: by mail-ed1-f65.google.com with SMTP id dc19so6291772edb.10 for ; Thu, 19 Dec 2019 13:28:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=O5bdAKERjFqZXYHkEGuYnzcy5g3lzmd91EZmqMjMLxI=; b=NeWv1e5QDv0+ikTrCaAiIv8RwsazFOO/R9WYl57oqw3GZzPTjn7N3riOY81uLSyfT0 igUN7ELuNBaMHBenVZEP5j9j1I8hzmzYCv0NC2Ns5CkqZHl4IFU7ABaI2a0AwbOnVk25 530DyRIPQXCWADUyC4u8IKHO4iOJfza8IZ6LjPLJWEgK4FWOpKEyN8FYEZ/8s5OAOCky apINquSpu8MagCX7kVrdLRvbCBwysa/21nrAaVV74Dvli5szHovaFvvqY/rHCcmre8W1 dH4+FO/0op22EqO9YhqJeyyyeYxTY6t4vKc4D71B9Gj0BeCVGm8CRI+kiY5vckP+CqSj mktw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=O5bdAKERjFqZXYHkEGuYnzcy5g3lzmd91EZmqMjMLxI=; b=GvmK59xrU4d8TOTZh/X7Ub4tfe/pz1gAWxlaG7v1qxMp4FeaPXxFeMR/L3bAYoLVZQ OfSypFzmD4EqHtFmJ1KVoiJobCe5/MIz+t7pVlhHSZGw8QfaDJIy2WZCsKt3pmIl/5Eo OhrvcueTMqzMYC3+uPrep54ruSOf8vIAp7q5iGZLzOJTgEfzsdQ0He6WjZgavSQwcR+e HZ3VMtA4eJXALu2K7Dd2atWKfX0k7eKHz8Ntz4U0PzsQgJ3W+BoY3FQhIKsFPypYbj/4 2kjg+eoT9u8awN6qtMnjev5Y8V2fOgr0StqZcxCsI5RHFhLUn5xqanqX5XBNJPqf2Inu lvSw== X-Gm-Message-State: APjAAAVUSHvz92B8MJsOJ1yemRRtx6mmh+5AaKDgMrFw9LyoLVeu9FTt evuYg+JoE2QDbUaf0dF1GKY5wg/J X-Google-Smtp-Source: APXvYqybXMm3UXLxcr8Cw1VwfjNshh8F1mLbLiib+dKI0WzKbCSr/m5Y7NCLzinWiP5DM4tw+/9hwQ== X-Received: by 2002:a17:906:5358:: with SMTP id j24mr12396133ejo.44.1576790910174; Thu, 19 Dec 2019 13:28:30 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id l11sm768667ejq.4.2019.12.19.13.28.29 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 19 Dec 2019 13:28:29 -0800 (PST) Message-Id: <79f2b561742372234a52dc7194dd72026c6e96ec.1576790906.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Elijah Newren via GitGitGadget" Date: Thu, 19 Dec 2019 21:28:20 +0000 Subject: [PATCH v5 2/8] Revert "dir.c: make 'git-status --ignored' work within leading directories" Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: blees@dcon.de, gitster@pobox.com, kyle@kyleam.com, sxlijin@gmail.com, Junio C Hamano , Elijah Newren Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Elijah Newren Commit be8a84c52669 ("dir.c: make 'git-status --ignored' work within leading directories", 2013-04-15) noted that git status --ignored would not list ignored files and directories within if was untracked, and modified the behavior to make it show them. However, it did so via a hack that broke consistency; it would show paths under differently than a simple git status --ignored | grep would show them. A correct fix is slightly more involved, and complicated slightly by this hack, so we revert this commit (but keep corrected versions of the testcases) and will later fix the original bug with a subsequent patch. Some history may be helpful: A very, very similar case to the commit we are reverting was raised in commit 48ffef966c76 ("ls-files: fix overeager pathspec optimization", 2010-01-08); but it actually went in somewhat the opposite direction. In that commit, it mentioned how git ls-files -o --exclude-standard t/ used to show untracked files under t/ even when t/ was ignored, and then changed the behavior to stop showing untracked files under an ignored directory. More importantly, this commit considered keeping this behavior but noted that it would be inconsistent with the behavior when multiple pathspecs were specified and thus rejected it. The reason for this whole inconsistency when one pathspec is specified versus zero or two is because common prefixes of pathspecs are sent through a different set of checks (in treat_leading_path()) than normal file/directory traversal (those go through read_directory_recursive() and treat_path()). As such, for consistency, one needs to check that both codepaths produce the same result. Revert commit be8a84c526691667fc04a8241d93a3de1de298ab, except instead of removing the testcase it added, modify it to check for correct and consistent behavior. A subsequent patch in this series will fix the testcase. Signed-off-by: Elijah Newren --- dir.c | 3 --- t/t7061-wtstatus-ignore.sh | 9 +++++++-- 2 files changed, 7 insertions(+), 5 deletions(-) diff --git a/dir.c b/dir.c index 61f559f980..0dd5266629 100644 --- a/dir.c +++ b/dir.c @@ -2083,14 +2083,12 @@ static int treat_leading_path(struct dir_struct *dir, struct strbuf sb = STRBUF_INIT; int baselen, rc = 0; const char *cp; - int old_flags = dir->flags; while (len && path[len - 1] == '/') len--; if (!len) return 1; baselen = 0; - dir->flags &= ~DIR_SHOW_OTHER_DIRECTORIES; while (1) { cp = path + baselen + !!baselen; cp = memchr(cp, '/', path + len - cp); @@ -2113,7 +2111,6 @@ static int treat_leading_path(struct dir_struct *dir, } } strbuf_release(&sb); - dir->flags = old_flags; return rc; } diff --git a/t/t7061-wtstatus-ignore.sh b/t/t7061-wtstatus-ignore.sh index 0c394cf995..84366050da 100755 --- a/t/t7061-wtstatus-ignore.sh +++ b/t/t7061-wtstatus-ignore.sh @@ -43,11 +43,16 @@ test_expect_success 'status untracked directory with --ignored -u' ' test_cmp expected actual ' cat >expected <<\EOF -?? untracked/uncommitted +?? untracked/ !! untracked/ignored EOF -test_expect_success 'status prefixed untracked directory with --ignored' ' +test_expect_failure 'status of untracked directory with --ignored works with or without prefix' ' + git status --porcelain --ignored >tmp && + grep untracked/ tmp >actual && + rm tmp && + test_cmp expected actual && + git status --porcelain --ignored untracked/ >actual && test_cmp expected actual ' From patchwork Thu Dec 19 21:28:21 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Koji Nakamaru via GitGitGadget X-Patchwork-Id: 11304421 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E8E1213B6 for ; Thu, 19 Dec 2019 21:28:37 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id C4C9324679 for ; Thu, 19 Dec 2019 21:28:37 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Be9I1Hny" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727330AbfLSV2e (ORCPT ); Thu, 19 Dec 2019 16:28:34 -0500 Received: from mail-ed1-f68.google.com ([209.85.208.68]:42037 "EHLO mail-ed1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726963AbfLSV2c (ORCPT ); Thu, 19 Dec 2019 16:28:32 -0500 Received: by mail-ed1-f68.google.com with SMTP id e10so6298059edv.9 for ; Thu, 19 Dec 2019 13:28:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=H1rfc3Xakcj/y7/qH6kpuw834RYNToBjdWGDvsRGWdg=; b=Be9I1Hnym8k1pBm8KNIgTAARpJiut0zWKEbm6sPjqgQf30SvKaWgdjmWSKeC83Ai/L 8sLZ5TprAiraOMujLUrH1DjKL6b3TYSGMD/LSEsGcY1vrmc81Dcmp/5PaRCQf1YmXNWK J3qbRK8JrModu/7BLmZ01IAPRjd7eJYoXmXYpJbIyEiQVWeCvJcy4PXKoqQM4qv6KJMW moa/m28Lfc3VIE/sH1lcrKBPLEPNOV6ci1dZDctAI8B2YuwNDTwOH6s/heamJ7nnoBz0 67pVvU6IjmSiozTj9/+mfAEpnhNkksGDCd4TdcTrgWCNDMls69PoVHgESbewhW4viJxj uMFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=H1rfc3Xakcj/y7/qH6kpuw834RYNToBjdWGDvsRGWdg=; b=LvsUHVo3BQ3Kb/kdUHFF6iaNUhz0RFaPNmcqtqbOKxGjP68owQMjlREAnpHEVoop6C USWKgJlv+cjS2+7aZitbCklAZJ15Ld8ea4XJJrQPb4W0FCPoT5D0paLu69s+cG9WxU6k oqgjPlwbmM52h8jrMo4em9MtJeTWCEFvLwaWq9xEroSffxlRbTYBJtWtqqLjBrQpNfdq YidRfWeRwCXFzVrELPVfestogBL5CtMERgm7f2q5xhwILlXr90xjGx2BYrl8bLUpgpam RvY+8tktmjZOQBjx5bR26jJ/JxgZqVrApp7BCAVPO3kFhsrDhtIJM/clyRxDeidzPXLy 9ELw== X-Gm-Message-State: APjAAAXDfg77jJigST0j39xLNZNN0/rs599pamBOYqpqwLsajT3VaRSF op/lOXDiW/0ZJotcDCZiaDjDlEsa X-Google-Smtp-Source: APXvYqxBGKnAVU5TdSBM7UPxqYZfKurWyIrKjI1eDRtWvYEmcyi+mX5HANtowU1oFWnNU1dH0wX5PQ== X-Received: by 2002:a17:906:7e41:: with SMTP id z1mr11865966ejr.23.1576790911016; Thu, 19 Dec 2019 13:28:31 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id m24sm679211edr.83.2019.12.19.13.28.30 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 19 Dec 2019 13:28:30 -0800 (PST) Message-Id: In-Reply-To: References: From: "Elijah Newren via GitGitGadget" Date: Thu, 19 Dec 2019 21:28:21 +0000 Subject: [PATCH v5 3/8] dir: remove stray quote character in comment Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: blees@dcon.de, gitster@pobox.com, kyle@kyleam.com, sxlijin@gmail.com, Junio C Hamano , Elijah Newren Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Elijah Newren Signed-off-by: Elijah Newren --- dir.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/dir.c b/dir.c index 0dd5266629..5dacacd469 100644 --- a/dir.c +++ b/dir.c @@ -373,7 +373,7 @@ static int match_pathspec_item(const struct index_state *istate, !ps_strncmp(item, match, name, namelen)) return MATCHED_RECURSIVELY_LEADING_PATHSPEC; - /* name" doesn't match up to the first wild character */ + /* name doesn't match up to the first wild character */ if (item->nowildcard_len < item->len && ps_strncmp(item, match, name, item->nowildcard_len - prefix)) From patchwork Thu Dec 19 21:28:22 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Koji Nakamaru via GitGitGadget X-Patchwork-Id: 11304419 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B50C3109A for ; Thu, 19 Dec 2019 21:28:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 948782467B for ; Thu, 19 Dec 2019 21:28:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Zit9YTLa" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727368AbfLSV2f (ORCPT ); Thu, 19 Dec 2019 16:28:35 -0500 Received: from mail-ed1-f68.google.com ([209.85.208.68]:43143 "EHLO mail-ed1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727024AbfLSV2e (ORCPT ); Thu, 19 Dec 2019 16:28:34 -0500 Received: by mail-ed1-f68.google.com with SMTP id dc19so6291829edb.10 for ; Thu, 19 Dec 2019 13:28:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=xbAMdwUoczvFExTFmHXxzR/2/texrkL36eKuhAIvlbk=; b=Zit9YTLaOoVTXTVgV5L1ZJL3h0ymn9DyXBvDKeA1n7kMdL/htDgY9YEzvdsH/Tf4GV bz+5mvibC/4DfJ8L7wugVTwqaGO2/E8vUMHXbRjVh9+ex0hsJyTNH5sYDTRL5uIWndwd 5/BRc0u7lDwbc1CQ6WculLbLvotO5d02/5G6+31Pm0KEIzgQzkzQAlAyeBOiDFWyzRUn nuOjyDDLXi0ltDcKGBH2Ou35VREeaUiV8RgEMFjpknX2Fp6Hj/6coWzywW+ASZMIkB7d 1cgYveo0rF/WVj7qMGEJe2M7zSPtlsQ3L10+kup6YQbtk4mBY7ttmTGbyDECVRpvrOaf E1pg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=xbAMdwUoczvFExTFmHXxzR/2/texrkL36eKuhAIvlbk=; b=iaNz7sKoVKNozwI+Vek0ZbsRpLixQWU+MBBttfiseAVkFUiUGRvutfrf/r5FgK8Jqr GX7/akzYVJNKaallFPL6LbVoMMswSr232c++Z13jMGTJxAJx0EofTLPj9jjjOTMrxWBP wBMk87hkJHCV1vbfLxSr4N/FzdCEeM9x5NhWW4A2qjUXK6aMrEtxjVZUH6yupNQxo7p5 S88MJ0SCbrAWVF3tBRJXteBRcFe6vubWcO1n1pb+tn2NocidDrnqihucjl9uYQThWpCL IqU0b0bMLUrG+UXRrnf6KGBRtxDjpHHusabo4FFFNp3d9oOBspOFY0e3IdhA/3YrMmr4 qI3g== X-Gm-Message-State: APjAAAXmRzwNFExj/s/9IW9vJEeRJePVTF1ZX1xtMqJaB+egJH/eH3yO 9BaqCaF81dwgAa9s8XILQJD4k6tr X-Google-Smtp-Source: APXvYqwXuXb5pnUpJbgZPjPbWH1K81957LWlxeyZu3/HSEustWBjh5ergdBmycWTzlOndJYmsDgGjQ== X-Received: by 2002:a17:906:1646:: with SMTP id n6mr11790768ejd.296.1576790911751; Thu, 19 Dec 2019 13:28:31 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id p6sm790539eja.63.2019.12.19.13.28.31 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 19 Dec 2019 13:28:31 -0800 (PST) Message-Id: <8d2d98eec3c620c55100a322087ef83e4f51c8cf.1576790906.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Elijah Newren via GitGitGadget" Date: Thu, 19 Dec 2019 21:28:22 +0000 Subject: [PATCH v5 4/8] dir: exit before wildcard fall-through if there is no wildcard Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: blees@dcon.de, gitster@pobox.com, kyle@kyleam.com, sxlijin@gmail.com, Junio C Hamano , Elijah Newren Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Elijah Newren The DO_MATCH_LEADING_PATHSPEC had a fall-through case for if there was a wildcard, noting that we don't yet have enough information to determine if a further paths under the current directory might match due to the presence of wildcards. But if we have no wildcards in our pathspec, then we shouldn't get to that fall-through case. Signed-off-by: Elijah Newren --- dir.c | 7 +++++++ t/t3011-common-prefixes-and-directory-traversal.sh | 4 ++-- 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/dir.c b/dir.c index 5dacacd469..517a569e10 100644 --- a/dir.c +++ b/dir.c @@ -379,6 +379,13 @@ static int match_pathspec_item(const struct index_state *istate, item->nowildcard_len - prefix)) return 0; + /* + * name has no wildcard, and it didn't match as a leading + * pathspec so return. + */ + if (item->nowildcard_len == item->len) + return 0; + /* * Here is where we would perform a wildmatch to check if * "name" can be matched as a directory (or a prefix) against diff --git a/t/t3011-common-prefixes-and-directory-traversal.sh b/t/t3011-common-prefixes-and-directory-traversal.sh index 54f80c62b8..d6e161ddd8 100755 --- a/t/t3011-common-prefixes-and-directory-traversal.sh +++ b/t/t3011-common-prefixes-and-directory-traversal.sh @@ -92,7 +92,7 @@ test_expect_failure 'git ls-files -o untracked_repo/ does not recurse' ' test_cmp expect actual ' -test_expect_failure 'git ls-files -o untracked_dir untracked_repo recurses into untracked_dir only' ' +test_expect_success 'git ls-files -o untracked_dir untracked_repo recurses into untracked_dir only' ' cat <<-EOF >expect && untracked_dir/empty untracked_repo/ @@ -110,7 +110,7 @@ test_expect_success 'git ls-files -o untracked_dir/ untracked_repo/ recurses int test_cmp expect actual ' -test_expect_failure 'git ls-files -o --directory untracked_dir untracked_repo does not recurse' ' +test_expect_success 'git ls-files -o --directory untracked_dir untracked_repo does not recurse' ' cat <<-EOF >expect && untracked_dir/ untracked_repo/ From patchwork Thu Dec 19 21:28:23 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Koji Nakamaru via GitGitGadget X-Patchwork-Id: 11304417 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0FE01109A for ; Thu, 19 Dec 2019 21:28:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id E26D92467B for ; Thu, 19 Dec 2019 21:28:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="nNABYYbE" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727354AbfLSV2e (ORCPT ); Thu, 19 Dec 2019 16:28:34 -0500 Received: from mail-ed1-f65.google.com ([209.85.208.65]:36302 "EHLO mail-ed1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727105AbfLSV2e (ORCPT ); Thu, 19 Dec 2019 16:28:34 -0500 Received: by mail-ed1-f65.google.com with SMTP id j17so6311975edp.3 for ; Thu, 19 Dec 2019 13:28:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=+CgvyKrM09apU1qkOsJft1P4o4wqjmDRS+ywvTmG7xU=; b=nNABYYbE3+GPc7LyEEx8auhXBi5tEhWgGacE85cvuUVvSOD6z7cSVNBydsAC9Y2eyT 87RdCwjH3IoXO0DzwZTFpiUleCLrF9JMtyVKAJKaNNxZvUQPBQqpbLeXdnTd6uT3qARm FMV479Sy7ec3qazzLGoFLIpzSbd+GHCzA2BjvUTF84PXg7fP5jlHvaLXATBth40Xae5A x5uX4ZBz+GkbbONy5FWebRjOJj8fSXRbOEoOQuEJ3qmTHMuUdBNc2JJCoGVdifQby9PZ 1kVAkAFZFIUyANZ/dsT5G+6TUT0KdzanjxEleS2bCwChNURFWwlGWBv+hKeERTsyUYaP gIdg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=+CgvyKrM09apU1qkOsJft1P4o4wqjmDRS+ywvTmG7xU=; b=Qzdtr7ZoTT/vHd8tx/XnrlCt2ggv4LW7Ir1gZpG6BtP8DxUvizPbH3RLHck1q/E0SM sRFrLvsCbFqk+fj9Dz5yBrrTtn+WU39D2nH+KvKIS2PPxpBiq0QI0CHoDuafGjHIE3BM 3L4M28zh6anTkytPrj4IncCu4jjDd8YYgyXjnsQasY7WGTcBO65wM9ptmqCg8dMOsX9o 8QBh8INeVK6IDTAoscmgi8wsdueocp8iTUrPhHXzePdp6Xa+/QWv9heOL5p7M8W6FyDw iNHoY6HIFg5scFL3Wm/GZ1PR9+vLc25g8MrfkumWc2cvlrpz/WuTuIqr1s8sIxcFLqtW T4hg== X-Gm-Message-State: APjAAAUB2qpVjoxNtdiRBDRFZdk69qwvGGCYBWN4ndHZ9dPsputKOxju UuGQAUWrJ+Ckw26DNk85xBhfvgIQ X-Google-Smtp-Source: APXvYqyJH/bzIs9V9huzy1fioUZ9do4wXP0hBemQENpbXP9a2xYYBOad5r0ySHvxL+2N0ptJyOVWhQ== X-Received: by 2002:a17:906:4749:: with SMTP id j9mr12274487ejs.132.1576790912464; Thu, 19 Dec 2019 13:28:32 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id k15sm763440ejc.35.2019.12.19.13.28.31 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 19 Dec 2019 13:28:32 -0800 (PST) Message-Id: In-Reply-To: References: From: "Elijah Newren via GitGitGadget" Date: Thu, 19 Dec 2019 21:28:23 +0000 Subject: [PATCH v5 5/8] dir: break part of read_directory_recursive() out for reuse Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: blees@dcon.de, gitster@pobox.com, kyle@kyleam.com, sxlijin@gmail.com, Junio C Hamano , Elijah Newren Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Elijah Newren Create an add_path_to_appropriate_result_list() function from the code at the end of read_directory_recursive() so we can use it elsewhere. Signed-off-by: Elijah Newren --- dir.c | 60 ++++++++++++++++++++++++++++++++++++----------------------- 1 file changed, 37 insertions(+), 23 deletions(-) diff --git a/dir.c b/dir.c index 517a569e10..645b44ea64 100644 --- a/dir.c +++ b/dir.c @@ -1932,6 +1932,40 @@ static void close_cached_dir(struct cached_dir *cdir) } } +static void add_path_to_appropriate_result_list(struct dir_struct *dir, + struct untracked_cache_dir *untracked, + struct cached_dir *cdir, + struct index_state *istate, + struct strbuf *path, + int baselen, + const struct pathspec *pathspec, + enum path_treatment state) +{ + /* add the path to the appropriate result list */ + switch (state) { + case path_excluded: + if (dir->flags & DIR_SHOW_IGNORED) + dir_add_name(dir, istate, path->buf, path->len); + else if ((dir->flags & DIR_SHOW_IGNORED_TOO) || + ((dir->flags & DIR_COLLECT_IGNORED) && + exclude_matches_pathspec(path->buf, path->len, + pathspec))) + dir_add_ignored(dir, istate, path->buf, path->len); + break; + + case path_untracked: + if (dir->flags & DIR_SHOW_IGNORED) + break; + dir_add_name(dir, istate, path->buf, path->len); + if (cdir->fdir) + add_untracked(untracked, path->buf + baselen); + break; + + default: + break; + } +} + /* * Read a directory tree. We currently ignore anything but * directories, regular files and symlinks. That's because git @@ -2035,29 +2069,9 @@ static enum path_treatment read_directory_recursive(struct dir_struct *dir, continue; } - /* add the path to the appropriate result list */ - switch (state) { - case path_excluded: - if (dir->flags & DIR_SHOW_IGNORED) - dir_add_name(dir, istate, path.buf, path.len); - else if ((dir->flags & DIR_SHOW_IGNORED_TOO) || - ((dir->flags & DIR_COLLECT_IGNORED) && - exclude_matches_pathspec(path.buf, path.len, - pathspec))) - dir_add_ignored(dir, istate, path.buf, path.len); - break; - - case path_untracked: - if (dir->flags & DIR_SHOW_IGNORED) - break; - dir_add_name(dir, istate, path.buf, path.len); - if (cdir.fdir) - add_untracked(untracked, path.buf + baselen); - break; - - default: - break; - } + add_path_to_appropriate_result_list(dir, untracked, &cdir, + istate, &path, baselen, + pathspec, state); } close_cached_dir(&cdir); out: From patchwork Thu Dec 19 21:28:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Koji Nakamaru via GitGitGadget X-Patchwork-Id: 11304425 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 81D2D109A for ; Thu, 19 Dec 2019 21:28:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 579172467F for ; Thu, 19 Dec 2019 21:28:41 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="f5l17HYu" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727386AbfLSV2k (ORCPT ); Thu, 19 Dec 2019 16:28:40 -0500 Received: from mail-ed1-f66.google.com ([209.85.208.66]:38204 "EHLO mail-ed1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727328AbfLSV2g (ORCPT ); Thu, 19 Dec 2019 16:28:36 -0500 Received: by mail-ed1-f66.google.com with SMTP id i16so6302828edr.5 for ; Thu, 19 Dec 2019 13:28:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=+YowHTlhK3qmCB2nfYJMdfk75zABXnHI8T9LUDhePM8=; b=f5l17HYuWPfm3Etc3BK2d6I24mKI79ojdVLa2Hg7YwFa8NLsiZ5ZBABokmQ5L5pCIb LCmIbJ3l6jDfhrMaWe0s9VNy8/mbmuNXMsYKqwJaiDAPYxE8cQ98QfbZgIFhkxT6BoVu XZEAHsqanjribWxOB3UD4UxznP19GPb4ciH4C44cg46lT+JK1TqgWpvYII/p7q1gqt1n uXViHjzn+fzayzpk2+rPb+6IaiIwY5neR3y9hiOd57QfU13UO+5F4+KhnYMyxf18/zLz F6Zs5luL+DDKl6xFT9g3epOyl6qLpUUwPRxjR7mCkPHVvSYTYG1kaROEazdHdEMyp5uJ /CJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=+YowHTlhK3qmCB2nfYJMdfk75zABXnHI8T9LUDhePM8=; b=o8hkwSWA9oT508GdYaTmafHyOevbDwcGYzhSvR8nF6krC6MlYVzVk4MPAjNZnSPnmh AhDhp6mIsMkfbvFhwOPY94A7ihh16QzPoFFCEAevlnqkXtuo+wm0mGo1oyM61pF43551 wnuCtpNcG+ZS6Ovyj8KnFWh6iMk4CC1Uiz5EDL8KyQo5XeWy6C8B/reZkESBvly7Xvdg idbIqzx5ibqSvDp51TBFDMYtpdFLpFEgCJ6dNk0f8ph7TZdDhOSgwYeZ5N59NSApXOdE CvgvobOJAbyDW0tCINbsYmtnGAnCEaIAzzQZyY20lf+41eSy6ZtMRrXBhe64Puv7KzH6 v9Hg== X-Gm-Message-State: APjAAAUMRg+znKgbEJQ8uQ7rHGBsG/FNfHS2pEJVDWfAtQhODEp6IasL QjIonaD+ZgCzV4mN9OcFFhf9AwUL X-Google-Smtp-Source: APXvYqyEKEBy0AD6PCVoY32uA0ss3MrjN5PwKnp0QAaxyFcWdXS5KfE5kDxm51M/MHMw+O6aViPjwQ== X-Received: by 2002:a50:ec1a:: with SMTP id g26mr11930103edr.164.1576790913242; Thu, 19 Dec 2019 13:28:33 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id by2sm766874ejb.45.2019.12.19.13.28.32 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 19 Dec 2019 13:28:32 -0800 (PST) Message-Id: <97e145489d275da2d39179f8d8d928a47936377b.1576790906.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Elijah Newren via GitGitGadget" Date: Thu, 19 Dec 2019 21:28:24 +0000 Subject: [PATCH v5 6/8] dir: fix checks on common prefix directory Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: blees@dcon.de, gitster@pobox.com, kyle@kyleam.com, sxlijin@gmail.com, Junio C Hamano , Elijah Newren Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Elijah Newren Many years ago, the directory traversing logic had an optimization that would always recurse into any directory that was a common prefix of all the pathspecs without walking the leading directories to get down to the desired directory. Thus, git ls-files -o .git/ # case A would notice that .git/ was a common prefix of all pathspecs (since it is the only pathspec listed), and then traverse into it and start showing unknown files under that directory. Unfortunately, .git/ is not a directory we should be traversing into, which made this optimization problematic. This also affected cases like git ls-files -o --exclude-standard t/ # case B where t/ was in the .gitignore file and thus isn't interesting and shouldn't be recursed into. It also affected cases like git ls-files -o --directory untracked_dir/ # case C where untracked_dir/ is indeed untracked and thus interesting, but the --directory flag means we only want to show the directory itself, not recurse into it and start listing untracked files below it. The case B class of bugs were noted and fixed in commits 16e2cfa90993 ("read_directory(): further split treat_path()", 2010-01-08) and 48ffef966c76 ("ls-files: fix overeager pathspec optimization", 2010-01-08), with the idea being that we first wanted to check whether the common prefix was interesting. The former patch noted that treat_path() couldn't be used when checking the common prefix because treat_path() requires a dir_entry() and we haven't read any directories at the point we are checking the common prefix. So, that patch split treat_one_path() out of treat_path(). The latter patch then created a new treat_leading_path() which duplicated by hand the bits of treat_path() that couldn't be broken out and then called treat_one_path() for the remainder. There were three problems with this approach: * The duplicated logic in treat_leading_path() accidentally missed the check for special paths (such as is_dot_or_dotdot and matching ".git"), causing case A types of bugs to continue to be an issue. * The treat_leading_path() logic assumed we should traverse into anything where path_treatment was not path_none, i.e. it perpetuated class C types of bugs. * It meant we had split logic that needed to kept in sync, running the risk that people introduced new inconsistencies (such as in commit be8a84c52669, which we reverted earlier in this series, or in commit df5bcdf83ae which we'll fix in a subsequent commit) Fix most these problems by making treat_leading_path() not only loop over each leading path component, but calling treat_path() directly on each. To do so, we have to create a synthetic dir_entry, but that only takes a few lines. Then, pay attention to the path_treatment result we get from treat_path() and don't treat path_excluded, path_untracked, and path_recurse all the same as path_recurse. This leaves one remaining problem, the new inconsistency from commit df5bcdf83ae. That will be addressed in a subsequent commit. Signed-off-by: Elijah Newren --- dir.c | 67 ++++++++++++++++--- ...common-prefixes-and-directory-traversal.sh | 6 +- 2 files changed, 59 insertions(+), 14 deletions(-) diff --git a/dir.c b/dir.c index 645b44ea64..a42cc2aa8c 100644 --- a/dir.c +++ b/dir.c @@ -2102,37 +2102,82 @@ static int treat_leading_path(struct dir_struct *dir, const struct pathspec *pathspec) { struct strbuf sb = STRBUF_INIT; - int baselen, rc = 0; + int prevlen, baselen; const char *cp; + struct cached_dir cdir; + struct dirent *de; + enum path_treatment state = path_none; + + /* + * For each directory component of path, we are going to check whether + * that path is relevant given the pathspec. For example, if path is + * foo/bar/baz/ + * then we will ask treat_path() whether we should go into foo, then + * whether we should go into bar, then whether baz is relevant. + * Checking each is important because e.g. if path is + * .git/info/ + * then we need to check .git to know we shouldn't traverse it. + * If the return from treat_path() is: + * * path_none, for any path, we return false. + * * path_recurse, for all path components, we return true + * * for some intermediate component, we make sure + * to add that path to the relevant list but return false + * signifying that we shouldn't recurse into it. + */ while (len && path[len - 1] == '/') len--; if (!len) return 1; + + /* + * We need a manufactured dirent with sufficient space to store a + * leading directory component of path in its d_name. Here, we + * assume that the dirent's d_name is either declared as + * char d_name[BIG_ENOUGH] + * or that it is declared at the end of the struct as + * char d_name[] + * For either case, padding with len+1 bytes at the end will ensure + * sufficient storage space. + */ + de = xcalloc(1, sizeof(struct dirent)+len+1); + memset(&cdir, 0, sizeof(cdir)); + cdir.de = de; +#if defined(DT_UNKNOWN) && !defined(NO_D_TYPE_IN_DIRENT) + de->d_type = DT_DIR; +#endif baselen = 0; + prevlen = 0; while (1) { - cp = path + baselen + !!baselen; + prevlen = baselen + !!baselen; + cp = path + prevlen; cp = memchr(cp, '/', path + len - cp); if (!cp) baselen = len; else baselen = cp - path; - strbuf_setlen(&sb, 0); + strbuf_reset(&sb); strbuf_add(&sb, path, baselen); if (!is_directory(sb.buf)) break; - if (simplify_away(sb.buf, sb.len, pathspec)) - break; - if (treat_one_path(dir, NULL, istate, &sb, baselen, pathspec, - DT_DIR, NULL) == path_none) + strbuf_reset(&sb); + strbuf_add(&sb, path, prevlen); + memcpy(de->d_name, path+prevlen, baselen-prevlen); + de->d_name[baselen-prevlen] = '\0'; + state = treat_path(dir, NULL, &cdir, istate, &sb, prevlen, + pathspec); + if (state != path_recurse) break; /* do not recurse into it */ - if (len <= baselen) { - rc = 1; + if (len <= baselen) break; /* finished checking */ - } } + add_path_to_appropriate_result_list(dir, NULL, &cdir, istate, + &sb, baselen, pathspec, + state); + + free(de); strbuf_release(&sb); - return rc; + return state == path_recurse; } static const char *get_ident_string(void) diff --git a/t/t3011-common-prefixes-and-directory-traversal.sh b/t/t3011-common-prefixes-and-directory-traversal.sh index d6e161ddd8..098fddc75b 100755 --- a/t/t3011-common-prefixes-and-directory-traversal.sh +++ b/t/t3011-common-prefixes-and-directory-traversal.sh @@ -74,7 +74,7 @@ test_expect_success 'git ls-files -o --directory untracked_dir does not recurse' test_cmp expect actual ' -test_expect_failure 'git ls-files -o --directory untracked_dir/ does not recurse' ' +test_expect_success 'git ls-files -o --directory untracked_dir/ does not recurse' ' echo untracked_dir/ >expect && git ls-files -o --directory untracked_dir/ >actual && test_cmp expect actual @@ -86,7 +86,7 @@ test_expect_success 'git ls-files -o untracked_repo does not recurse' ' test_cmp expect actual ' -test_expect_failure 'git ls-files -o untracked_repo/ does not recurse' ' +test_expect_success 'git ls-files -o untracked_repo/ does not recurse' ' echo untracked_repo/ >expect && git ls-files -o untracked_repo/ >actual && test_cmp expect actual @@ -133,7 +133,7 @@ test_expect_success 'git ls-files -o .git shows nothing' ' test_must_be_empty actual ' -test_expect_failure 'git ls-files -o .git/ shows nothing' ' +test_expect_success 'git ls-files -o .git/ shows nothing' ' git ls-files -o .git/ >actual && test_must_be_empty actual ' From patchwork Thu Dec 19 21:28:25 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Koji Nakamaru via GitGitGadget X-Patchwork-Id: 11304429 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 81C3413B6 for ; Thu, 19 Dec 2019 21:28:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 546602467B for ; Thu, 19 Dec 2019 21:28:43 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="LZpoOAbs" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727380AbfLSV2k (ORCPT ); Thu, 19 Dec 2019 16:28:40 -0500 Received: from mail-ed1-f68.google.com ([209.85.208.68]:40394 "EHLO mail-ed1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727344AbfLSV2g (ORCPT ); Thu, 19 Dec 2019 16:28:36 -0500 Received: by mail-ed1-f68.google.com with SMTP id b8so6294362edx.7 for ; Thu, 19 Dec 2019 13:28:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=UVu2aHob7mXX34a7VVx/wP35ouJz71qPPyqGeHesuRg=; b=LZpoOAbs0ULO/HnWXPbUOq0WiO2+4wrUUi97Ln2Z0XE2TgGa+LOjQuD/zPr7YhaQPN h+ODZFnLzgqSB0yNj/SlMsJMde71ihXBc27FtfIZI8WhFdjIamxHogM7WmNAe/lTu/3Q WZcC6Gb5CCip99ulJW3OIbRE4GC959Zuaaep+T7SI/aJrt7ZQVzklMW60F/SPUBqPs1h 1O7H8XzA8kr+Fc7EyO8BlDNPloUWZs6J9lKyPV997rS7AOZqypxEGBej4wzSGE2u3+dX rzo7bKQloPRx9v9Yl1kYXRHLOGeSnj8tZatQ1EF8pj2kBCzR8Yp+JpRWI+qSnEmWRJSZ 2Vug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=UVu2aHob7mXX34a7VVx/wP35ouJz71qPPyqGeHesuRg=; b=DBv2513tN2IayZJFbGedA0/Vkedm/tdLB+KGfEOXv6CLkSM9rq9QhU2RDfW9K/aPq5 Dd5gEHsJlZkL8JQsbT486aUvqe7sUkQzceJZs9JfquG41ftnS+0UZ8xfX+GSntGWgIqM RNmVWyIwKocDXeg7yrmNhTnFk9Oi1DDXUFDwZCnL0zgxybSnyx97QLvHQgdcyXLm4EWf 7S03dgtHLSpiAk2plXkHDFVChTzAxjhlENtFQ3zjxfIRVWSxC8QW2Dfv4ZmWc/WcvV3/ TStLcOIBtGdRJWk8Omjk3jRgG5yuwoKVYl6rH9g4Kb3nBWn4RbV6dnS5YoFLEw9NM8DA jq8A== X-Gm-Message-State: APjAAAVuYef3GoYlQMg/Slxb1rwVRMbG6YxDmNfATPp/KGMX8NYAP73k oYWJqH+S5NBEeiHGoKgsNAHGou2g X-Google-Smtp-Source: APXvYqzcoIJ8udM/q/WE6g71qTLoXs92baNfBpuypGpkd/EEejV/CFFM0Eu4VEyzfmbTHFUlkiFjlA== X-Received: by 2002:a17:906:1f94:: with SMTP id t20mr12276098ejr.5.1576790913863; Thu, 19 Dec 2019 13:28:33 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id r9sm758879ejx.31.2019.12.19.13.28.33 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 19 Dec 2019 13:28:33 -0800 (PST) Message-Id: <5275e6d7f0cc70468ecd76a88e0afd03339583a0.1576790906.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Elijah Newren via GitGitGadget" Date: Thu, 19 Dec 2019 21:28:25 +0000 Subject: [PATCH v5 7/8] dir: synchronize treat_leading_path() and read_directory_recursive() Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: blees@dcon.de, gitster@pobox.com, kyle@kyleam.com, sxlijin@gmail.com, Junio C Hamano , Elijah Newren Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Elijah Newren Our optimization to avoid calling into read_directory_recursive() when all pathspecs have a common leading directory mean that we need to match the logic that read_directory_recursive() would use if we had just called it from the root. Since it does more than call treat_path() we need to copy that same logic. Alternatively, we could try to change treat_path to return path_recurse for an untracked directory under the given special circumstances that this logic checks for, but a simple switch results in many test failures such as 'git clean -d' not wiping out untracked but empty directories. To work around that, we'd need the caller of treat_path to check for path_recurse and sometimes special case it into path_untracked. In other words, we'd still have extra logic in both places. Needing to duplicate logic like this means it is guaranteed someone will eventually need to make further changes and forget to update both locations. It is tempting to just nuke the leading_directory special casing to avoid such bugs and simplify the code, but unpack_trees' verify_clean_subdirectory() also calls read_directory() and does so with a non-empty leading path, so I'm hesitant to try to restructure further. Add obnoxious warnings to treat_leading_path() and read_directory_recursive() to try to warn people of such problems. Signed-off-by: Elijah Newren --- dir.c | 30 +++++++++++++++++++ ...common-prefixes-and-directory-traversal.sh | 2 +- t/t7061-wtstatus-ignore.sh | 2 +- 3 files changed, 32 insertions(+), 2 deletions(-) diff --git a/dir.c b/dir.c index a42cc2aa8c..357f9593c4 100644 --- a/dir.c +++ b/dir.c @@ -1990,6 +1990,15 @@ static enum path_treatment read_directory_recursive(struct dir_struct *dir, struct untracked_cache_dir *untracked, int check_only, int stop_at_first_file, const struct pathspec *pathspec) { + /* + * WARNING WARNING WARNING: + * + * Any updates to the traversal logic here may need corresponding + * updates in treat_leading_path(). See the commit message for the + * commit adding this warning as well as the commit preceding it + * for details. + */ + struct cached_dir cdir; enum path_treatment state, subdir_state, dir_state = path_none; struct strbuf path = STRBUF_INIT; @@ -2101,6 +2110,15 @@ static int treat_leading_path(struct dir_struct *dir, const char *path, int len, const struct pathspec *pathspec) { + /* + * WARNING WARNING WARNING: + * + * Any updates to the traversal logic here may need corresponding + * updates in treat_leading_path(). See the commit message for the + * commit adding this warning as well as the commit preceding it + * for details. + */ + struct strbuf sb = STRBUF_INIT; int prevlen, baselen; const char *cp; @@ -2166,6 +2184,18 @@ static int treat_leading_path(struct dir_struct *dir, de->d_name[baselen-prevlen] = '\0'; state = treat_path(dir, NULL, &cdir, istate, &sb, prevlen, pathspec); + if (state == path_untracked && + get_dtype(cdir.de, istate, sb.buf, sb.len) == DT_DIR && + (dir->flags & DIR_SHOW_IGNORED_TOO || + do_match_pathspec(istate, pathspec, sb.buf, sb.len, + baselen, NULL, DO_MATCH_LEADING_PATHSPEC) == MATCHED_RECURSIVELY_LEADING_PATHSPEC)) { + add_path_to_appropriate_result_list(dir, NULL, &cdir, + istate, + &sb, baselen, + pathspec, state); + state = path_recurse; + } + if (state != path_recurse) break; /* do not recurse into it */ if (len <= baselen) diff --git a/t/t3011-common-prefixes-and-directory-traversal.sh b/t/t3011-common-prefixes-and-directory-traversal.sh index 098fddc75b..3da5b2b6e7 100755 --- a/t/t3011-common-prefixes-and-directory-traversal.sh +++ b/t/t3011-common-prefixes-and-directory-traversal.sh @@ -195,7 +195,7 @@ test_expect_success 'git ls-files -o consistent between one or two dirs' ' # ls-files doesn't have a way to request showing both untracked and ignored # files at the same time, so use `git status --ignored` -test_expect_failure 'git status --ignored shows same files under dir with or without pathspec' ' +test_expect_success 'git status --ignored shows same files under dir with or without pathspec' ' cat <<-EOF >expect && ?? an_untracked_dir/ !! an_untracked_dir/ignored diff --git a/t/t7061-wtstatus-ignore.sh b/t/t7061-wtstatus-ignore.sh index 84366050da..e4cf5484f9 100755 --- a/t/t7061-wtstatus-ignore.sh +++ b/t/t7061-wtstatus-ignore.sh @@ -47,7 +47,7 @@ cat >expected <<\EOF !! untracked/ignored EOF -test_expect_failure 'status of untracked directory with --ignored works with or without prefix' ' +test_expect_success 'status of untracked directory with --ignored works with or without prefix' ' git status --porcelain --ignored >tmp && grep untracked/ tmp >actual && rm tmp && From patchwork Thu Dec 19 21:28:26 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Koji Nakamaru via GitGitGadget X-Patchwork-Id: 11304427 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 69D2913B6 for ; Thu, 19 Dec 2019 21:28:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 476892467F for ; Thu, 19 Dec 2019 21:28:42 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="XEBntyGX" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727393AbfLSV2k (ORCPT ); Thu, 19 Dec 2019 16:28:40 -0500 Received: from mail-ed1-f66.google.com ([209.85.208.66]:40394 "EHLO mail-ed1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727359AbfLSV2g (ORCPT ); Thu, 19 Dec 2019 16:28:36 -0500 Received: by mail-ed1-f66.google.com with SMTP id b8so6294390edx.7 for ; Thu, 19 Dec 2019 13:28:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=MmYpkLT1ZacSJKjEx9Dyv7Qs23ty5Ec5tk1JIoopM4Y=; b=XEBntyGXIsZ+QQBh6zLo9+SZfDn2Uc8mx3muVxVkfZtljLEa9DZJ/gPywxtPCOPa08 zuic50+iuSL/OACRK+kw2AeSE4qaa0tfcQIv1+yvjpAHJX+E+qZBlANJTquC+UXCsWh6 dzWOXcakZzohXLtYncKav8sAoflax7RgIAVrBmbbG8sCgoQ5pZ4ZEENN7lGFeY3xnZxO IhzW4ya4LYyq0Sa2tKnPcOj+WgSHB1mi9E9YwxZOrPQtRMpxsVGGtnHNrPT/6Wy9Pdhz IjKibURUejzqf60p/AiezLrHiSBU9dJSapTDrq5MFBkPAZSLrtOYBs/mpU2XER2DFM30 yZ7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=MmYpkLT1ZacSJKjEx9Dyv7Qs23ty5Ec5tk1JIoopM4Y=; b=DOvpqLXA/j1SEB/wJJm9+WhdKysb03S4HQR27l4SRiNqjRL7sE69Xdz13pYTT4LRCC 03I0xwSgqWUuYAljYVPpWV3xfRSipkrmxlu8773vEyZsADMoHjIRuSY+NPAl+ix19ttD dyVY/1lS+TRzGqO9cARzaSx8F1aqLwHAEeES3/howL6Ww93ztubseBVvk0CqArkbkg8h nbqdyOL+Y3Hgii+SkL0mQdj5ceC5KMKEdE1peuBYe8aa7P+xHvDa2cW8bCSzQzrLCzGa Wgaq0CDhe8DJZjJk1xpyX7aej0w77grkC8OxH5YTks4FZ/B65meUiIVIC9o2mOkWTx73 kZsA== X-Gm-Message-State: APjAAAVDuuoq9jeWzDgWx7NnSw3UVDF45kbzQzEIcsEO7LfhBvEjkM4a gjzvfHFmWxwoPgoY3aHthUJPXSsr X-Google-Smtp-Source: APXvYqyGyUYGV961TCmYp38vjQ4fPhhJSfbGAZL/MUHjcKN/Z5TR2pV+pT+mqRqg94FgooFpN1DwBg== X-Received: by 2002:aa7:ce13:: with SMTP id d19mr11556291edv.296.1576790914635; Thu, 19 Dec 2019 13:28:34 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id y21sm674388edu.70.2019.12.19.13.28.34 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 19 Dec 2019 13:28:34 -0800 (PST) Message-Id: In-Reply-To: References: From: "Elijah Newren via GitGitGadget" Date: Thu, 19 Dec 2019 21:28:26 +0000 Subject: [PATCH v5 8/8] dir: consolidate similar code in treat_directory() Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: blees@dcon.de, gitster@pobox.com, kyle@kyleam.com, sxlijin@gmail.com, Junio C Hamano , Elijah Newren Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Elijah Newren Both the DIR_SKIP_NESTED_GIT and DIR_NO_GITLINKS cases were checking for whether a path was actually a nonbare repository. That code could be shared, with just the result of how to act differing between the two cases. Signed-off-by: Elijah Newren --- dir.c | 18 +++++++----------- 1 file changed, 7 insertions(+), 11 deletions(-) diff --git a/dir.c b/dir.c index 357f9593c4..e1b74f6478 100644 --- a/dir.c +++ b/dir.c @@ -1461,6 +1461,8 @@ static enum path_treatment treat_directory(struct dir_struct *dir, const char *dirname, int len, int baselen, int exclude, const struct pathspec *pathspec) { + int nested_repo = 0; + /* The "len-1" is to strip the final '/' */ switch (directory_exists_in_index(istate, dirname, len-1)) { case index_directory: @@ -1470,15 +1472,16 @@ static enum path_treatment treat_directory(struct dir_struct *dir, return path_none; case index_nonexistent: - if (dir->flags & DIR_SKIP_NESTED_GIT) { - int nested_repo; + if ((dir->flags & DIR_SKIP_NESTED_GIT) || + !(dir->flags & DIR_NO_GITLINKS)) { struct strbuf sb = STRBUF_INIT; strbuf_addstr(&sb, dirname); nested_repo = is_nonbare_repository_dir(&sb); strbuf_release(&sb); - if (nested_repo) - return path_none; } + if (nested_repo) + return ((dir->flags & DIR_SKIP_NESTED_GIT) ? path_none : + (exclude ? path_excluded : path_untracked)); if (dir->flags & DIR_SHOW_OTHER_DIRECTORIES) break; @@ -1506,13 +1509,6 @@ static enum path_treatment treat_directory(struct dir_struct *dir, return path_none; } - if (!(dir->flags & DIR_NO_GITLINKS)) { - struct strbuf sb = STRBUF_INIT; - strbuf_addstr(&sb, dirname); - if (is_nonbare_repository_dir(&sb)) - return exclude ? path_excluded : path_untracked; - strbuf_release(&sb); - } return path_recurse; }