From patchwork Tue Dec 10 20:00:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Arver via GitGitGadget X-Patchwork-Id: 11283159 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2A5CE109A for ; Tue, 10 Dec 2019 20:00:37 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id F3C7C20838 for ; Tue, 10 Dec 2019 20:00:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="QW+t/up6" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726691AbfLJUAg (ORCPT ); Tue, 10 Dec 2019 15:00:36 -0500 Received: from mail-wr1-f66.google.com ([209.85.221.66]:42163 "EHLO mail-wr1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726631AbfLJUAd (ORCPT ); Tue, 10 Dec 2019 15:00:33 -0500 Received: by mail-wr1-f66.google.com with SMTP id q6so32248wro.9 for ; Tue, 10 Dec 2019 12:00:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=RjUWfqMLtbjNIPA16XEQPNcw4ayZ9uv9uVm9KQNo/JQ=; b=QW+t/up6z+MzyKhFXszBmJKNN8czTsN13+8mBu5pShpAqHX+Fe04JMFmWr4dbI2pmO go9esoKCfZzFKG1aDQc99tf2JKONuFeBPAJzTrGsqAc1zVVdgWFkcgyCM2xJvuFAM8Jb i1y04mgpgQrl3mzT4wFILdSeJuJSo67msnUVscjjf1bRL6B+ZFFo8YidasPiK09cr8ul 9k2m69fSdYW/WEF6HoJDWYIs9JBS9u5OC8r+OZ9++BqJkAHlhvXoFJDaTuuDO69Z+quR qZPWjpVDCJ3PRoGhLDhZWsDKHFbEoVWCH6po60ihSDqKw4R7jD3itwKnuaj3BsC/gmnz lX6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=RjUWfqMLtbjNIPA16XEQPNcw4ayZ9uv9uVm9KQNo/JQ=; b=b+5VkJj295p3iIyyNyILgqB3HYInj6XZr7MkQXY92VA2wtqF44eC3+YzhNTJqk190G //+niFECEAQxljGXm8HNkhLwD8ZAAH5f5Y+knefkRDFWLvGdBThsBEO/LbsJujt7yWhh x0J/K7fdyjuEYkD/KNCtHyRQieLQWgV9DCRNw+4OFpbMkkUVktobrcUFotiMAdudZkxP BOMyzfA1o3XA3n6A/6NWb7FyXHjrUiCR8SN02hzisARbhSYEGTKlac84VPC3TzrZoJZU Jt3vestxPHJqhcTXNKK2hGgUwvFPb84ipgN3m9JN5AuxEp/z4dvaNqKayjV5ZPtD2QiB ZkSQ== X-Gm-Message-State: APjAAAVB8p+EvsE5JIWaDErmDPw45B73LbVtA5u4hmJjadEKCoCaEkTe mXpsapdm/x7g/IIpQmAEqaiuvrO4 X-Google-Smtp-Source: APXvYqz8n1DKvmhyfr4XkVSTcMDY+FANI46sUVTGZpiUZqg0oFLXnZbabJzTEpi8FXBOIjKABHRxCw== X-Received: by 2002:a05:6000:12ce:: with SMTP id l14mr5557114wrx.342.1576008029945; Tue, 10 Dec 2019 12:00:29 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id z189sm4559181wmc.2.2019.12.10.12.00.29 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 10 Dec 2019 12:00:29 -0800 (PST) Message-Id: <6d659b230239b530c90fbcc15c9a869f8d5dd5c5.1576008027.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Elijah Newren via GitGitGadget" Date: Tue, 10 Dec 2019 20:00:20 +0000 Subject: [PATCH v2 1/8] t3011: demonstrate directory traversal failures Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: blees@dcon.de, gitster@pobox.com, kyle@kyleam.com, sxlijin@gmail.com, Junio C Hamano , Elijah Newren Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Elijah Newren Add several tests demonstrating directory traversal failures of various sorts in dir.c (and one similar looking test that turns out to be a git_fnmatch bug). A lot of these tests look like near duplicates of each other, but an optimization path in dir.c to pre-descend into a common prefix and the specialized treatment of trailing slashes in dir.c mean the tiny differences are sometimes important and potentially cause different codepaths to be explored. Of the 7 failing tests, 2 are new to git-2.24.0 (tweaked by side effects of the en/clean-nested-with-ignored-topic); the other 5 also failed under git-2.23.0 and earlier. Signed-off-by: Elijah Newren --- ...common-prefixes-and-directory-traversal.sh | 209 ++++++++++++++++++ 1 file changed, 209 insertions(+) create mode 100755 t/t3011-common-prefixes-and-directory-traversal.sh diff --git a/t/t3011-common-prefixes-and-directory-traversal.sh b/t/t3011-common-prefixes-and-directory-traversal.sh new file mode 100755 index 0000000000..54f80c62b8 --- /dev/null +++ b/t/t3011-common-prefixes-and-directory-traversal.sh @@ -0,0 +1,209 @@ +#!/bin/sh + +test_description='directory traversal handling, especially with common prefixes' + +. ./test-lib.sh + +test_expect_success 'setup' ' + test_commit hello && + + >empty && + mkdir untracked_dir && + >untracked_dir/empty && + git init untracked_repo && + >untracked_repo/empty && + + cat <<-EOF >.gitignore && + ignored + an_ignored_dir/ + EOF + mkdir an_ignored_dir && + mkdir an_untracked_dir && + >an_ignored_dir/ignored && + >an_ignored_dir/untracked && + >an_untracked_dir/ignored && + >an_untracked_dir/untracked +' + +test_expect_success 'git ls-files -o shows the right entries' ' + cat <<-EOF >expect && + .gitignore + actual + an_ignored_dir/ignored + an_ignored_dir/untracked + an_untracked_dir/ignored + an_untracked_dir/untracked + empty + expect + untracked_dir/empty + untracked_repo/ + EOF + git ls-files -o >actual && + test_cmp expect actual +' + +test_expect_success 'git ls-files -o --exclude-standard shows the right entries' ' + cat <<-EOF >expect && + .gitignore + actual + an_untracked_dir/untracked + empty + expect + untracked_dir/empty + untracked_repo/ + EOF + git ls-files -o --exclude-standard >actual && + test_cmp expect actual +' + +test_expect_success 'git ls-files -o untracked_dir recurses' ' + echo untracked_dir/empty >expect && + git ls-files -o untracked_dir >actual && + test_cmp expect actual +' + +test_expect_success 'git ls-files -o untracked_dir/ recurses' ' + echo untracked_dir/empty >expect && + git ls-files -o untracked_dir/ >actual && + test_cmp expect actual +' + +test_expect_success 'git ls-files -o --directory untracked_dir does not recurse' ' + echo untracked_dir/ >expect && + git ls-files -o --directory untracked_dir >actual && + test_cmp expect actual +' + +test_expect_failure 'git ls-files -o --directory untracked_dir/ does not recurse' ' + echo untracked_dir/ >expect && + git ls-files -o --directory untracked_dir/ >actual && + test_cmp expect actual +' + +test_expect_success 'git ls-files -o untracked_repo does not recurse' ' + echo untracked_repo/ >expect && + git ls-files -o untracked_repo >actual && + test_cmp expect actual +' + +test_expect_failure 'git ls-files -o untracked_repo/ does not recurse' ' + echo untracked_repo/ >expect && + git ls-files -o untracked_repo/ >actual && + test_cmp expect actual +' + +test_expect_failure 'git ls-files -o untracked_dir untracked_repo recurses into untracked_dir only' ' + cat <<-EOF >expect && + untracked_dir/empty + untracked_repo/ + EOF + git ls-files -o untracked_dir untracked_repo >actual && + test_cmp expect actual +' + +test_expect_success 'git ls-files -o untracked_dir/ untracked_repo/ recurses into untracked_dir only' ' + cat <<-EOF >expect && + untracked_dir/empty + untracked_repo/ + EOF + git ls-files -o untracked_dir/ untracked_repo/ >actual && + test_cmp expect actual +' + +test_expect_failure 'git ls-files -o --directory untracked_dir untracked_repo does not recurse' ' + cat <<-EOF >expect && + untracked_dir/ + untracked_repo/ + EOF + git ls-files -o --directory untracked_dir untracked_repo >actual && + test_cmp expect actual +' + +test_expect_success 'git ls-files -o --directory untracked_dir/ untracked_repo/ does not recurse' ' + cat <<-EOF >expect && + untracked_dir/ + untracked_repo/ + EOF + git ls-files -o --directory untracked_dir/ untracked_repo/ >actual && + test_cmp expect actual +' + +test_expect_success 'git ls-files -o .git shows nothing' ' + git ls-files -o .git >actual && + test_must_be_empty actual +' + +test_expect_failure 'git ls-files -o .git/ shows nothing' ' + git ls-files -o .git/ >actual && + test_must_be_empty actual +' + +test_expect_success FUNNYNAMES 'git ls-files -o untracked_* recurses appropriately' ' + mkdir "untracked_*" && + >"untracked_*/empty" && + + cat <<-EOF >expect && + untracked_*/empty + untracked_dir/empty + untracked_repo/ + EOF + git ls-files -o "untracked_*" >actual && + test_cmp expect actual +' + +# It turns out fill_directory returns the right paths, but ls-files' post-call +# filtering in show_dir_entry() via calling dir_path_match() which ends up +# in git_fnmatch() has logic for PATHSPEC_ONESTAR that assumes the pathspec +# must match the full path; it doesn't check it for matching a leading +# directory. +test_expect_failure FUNNYNAMES 'git ls-files -o untracked_*/ recurses appropriately' ' + cat <<-EOF >expect && + untracked_*/empty + untracked_dir/empty + untracked_repo/ + EOF + git ls-files -o "untracked_*/" >actual && + test_cmp expect actual +' + +test_expect_success FUNNYNAMES 'git ls-files -o --directory untracked_* does not recurse' ' + cat <<-EOF >expect && + untracked_*/ + untracked_dir/ + untracked_repo/ + EOF + git ls-files -o --directory "untracked_*" >actual && + test_cmp expect actual +' + +test_expect_success FUNNYNAMES 'git ls-files -o --directory untracked_*/ does not recurse' ' + cat <<-EOF >expect && + untracked_*/ + untracked_dir/ + untracked_repo/ + EOF + git ls-files -o --directory "untracked_*/" >actual && + test_cmp expect actual +' + +test_expect_success 'git ls-files -o consistent between one or two dirs' ' + git ls-files -o --exclude-standard an_ignored_dir/ an_untracked_dir/ >tmp && + ! grep ^an_ignored_dir/ tmp >expect && + git ls-files -o --exclude-standard an_ignored_dir/ >actual && + test_cmp expect actual +' + +# ls-files doesn't have a way to request showing both untracked and ignored +# files at the same time, so use `git status --ignored` +test_expect_failure 'git status --ignored shows same files under dir with or without pathspec' ' + cat <<-EOF >expect && + ?? an_untracked_dir/ + !! an_untracked_dir/ignored + EOF + git status --porcelain --ignored >output && + grep an_untracked_dir output >expect && + git status --porcelain --ignored an_untracked_dir/ >actual && + test_cmp expect actual +' + +test_done From patchwork Tue Dec 10 20:00:21 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Arver via GitGitGadget X-Patchwork-Id: 11283155 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BDA0C1593 for ; Tue, 10 Dec 2019 20:00:34 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 9BFF520836 for ; Tue, 10 Dec 2019 20:00:34 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="BKoKjdsO" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726913AbfLJUAd (ORCPT ); Tue, 10 Dec 2019 15:00:33 -0500 Received: from mail-wr1-f66.google.com ([209.85.221.66]:46844 "EHLO mail-wr1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726100AbfLJUAd (ORCPT ); Tue, 10 Dec 2019 15:00:33 -0500 Received: by mail-wr1-f66.google.com with SMTP id z7so21415338wrl.13 for ; Tue, 10 Dec 2019 12:00:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=O5bdAKERjFqZXYHkEGuYnzcy5g3lzmd91EZmqMjMLxI=; b=BKoKjdsOHCFdOuxzdR5NQldLoN8eR9v+zF/PXXPy28aiKoQ07QMVZY1+54IBNgYxQj aeDh8f1cjHzxZ952y+qiQfPIvf14IjLVK/we/3l2yJuKnSlZFrduJqev0IR/1JGy6i44 fVyt5q1/i/tcbKfR2QjgQaJ+u+0hjl+3wKMuyZzcEDZYMc8lxWu1AmOw4G9hnfq/3WTT UOVGHTMhfmPlH9znmu1eMagNxnNib/rQGr75tAyrEMOnNRyyrXhKR0mlGy75cloUlAnC RXzfOamEedyukWYgpQyKM0LOAzqXfhUw7t5FaPvAyP7frFmhQlpLhvrMNyGIVYBmChHW 1IEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=O5bdAKERjFqZXYHkEGuYnzcy5g3lzmd91EZmqMjMLxI=; b=CfyLmQkaSjwpi0XUjuwvRNLCjjZ7Kz+YefexOy4g5XT/voEtCtqEj/ezkuFPJ4oYhu kh8iZyaF0eEp8rJEmFoaPSXPwRnHPbOfMZc3r6ecdznXBK7QRktiFAnb7UESs/DrfWcY wo30aR8WdGVS4EdLVRRrYHz2Cr6re4jCTpdDRFdf01DHAXUzZhvXX7rhqwjtyvPHj6Fu i/lQD8+W9haobtcDFVVXzADa3uj2GfUwjGxmA9rhXzCEMh0s62Me2DT/bNz2lGb7o6i8 FyHySmV9ReDn66vEenjQWYu3YNMmVu8QgV0WIV787NaCmicVc78ChjQ7M3zHEgDEzad8 z+bQ== X-Gm-Message-State: APjAAAXOh4hScClHXcgFTH6VckeyhxfeM0ZVQY9hOjtDFyf0W2qziWHA wrXpZ4r0gCIQKccUbpoqalAdDyev X-Google-Smtp-Source: APXvYqyZs6dGOQwFoFh7+9vZJU9UNRbuu8oTV2G9VHLjOUTNWTxR7FY770cMUh3PpZ4Kl74UpXQS9A== X-Received: by 2002:a5d:6144:: with SMTP id y4mr5327069wrt.367.1576008030887; Tue, 10 Dec 2019 12:00:30 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id b67sm4417387wmc.38.2019.12.10.12.00.30 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 10 Dec 2019 12:00:30 -0800 (PST) Message-Id: <79f2b561742372234a52dc7194dd72026c6e96ec.1576008027.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Elijah Newren via GitGitGadget" Date: Tue, 10 Dec 2019 20:00:21 +0000 Subject: [PATCH v2 2/8] Revert "dir.c: make 'git-status --ignored' work within leading directories" Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: blees@dcon.de, gitster@pobox.com, kyle@kyleam.com, sxlijin@gmail.com, Junio C Hamano , Elijah Newren Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Elijah Newren Commit be8a84c52669 ("dir.c: make 'git-status --ignored' work within leading directories", 2013-04-15) noted that git status --ignored would not list ignored files and directories within if was untracked, and modified the behavior to make it show them. However, it did so via a hack that broke consistency; it would show paths under differently than a simple git status --ignored | grep would show them. A correct fix is slightly more involved, and complicated slightly by this hack, so we revert this commit (but keep corrected versions of the testcases) and will later fix the original bug with a subsequent patch. Some history may be helpful: A very, very similar case to the commit we are reverting was raised in commit 48ffef966c76 ("ls-files: fix overeager pathspec optimization", 2010-01-08); but it actually went in somewhat the opposite direction. In that commit, it mentioned how git ls-files -o --exclude-standard t/ used to show untracked files under t/ even when t/ was ignored, and then changed the behavior to stop showing untracked files under an ignored directory. More importantly, this commit considered keeping this behavior but noted that it would be inconsistent with the behavior when multiple pathspecs were specified and thus rejected it. The reason for this whole inconsistency when one pathspec is specified versus zero or two is because common prefixes of pathspecs are sent through a different set of checks (in treat_leading_path()) than normal file/directory traversal (those go through read_directory_recursive() and treat_path()). As such, for consistency, one needs to check that both codepaths produce the same result. Revert commit be8a84c526691667fc04a8241d93a3de1de298ab, except instead of removing the testcase it added, modify it to check for correct and consistent behavior. A subsequent patch in this series will fix the testcase. Signed-off-by: Elijah Newren --- dir.c | 3 --- t/t7061-wtstatus-ignore.sh | 9 +++++++-- 2 files changed, 7 insertions(+), 5 deletions(-) diff --git a/dir.c b/dir.c index 61f559f980..0dd5266629 100644 --- a/dir.c +++ b/dir.c @@ -2083,14 +2083,12 @@ static int treat_leading_path(struct dir_struct *dir, struct strbuf sb = STRBUF_INIT; int baselen, rc = 0; const char *cp; - int old_flags = dir->flags; while (len && path[len - 1] == '/') len--; if (!len) return 1; baselen = 0; - dir->flags &= ~DIR_SHOW_OTHER_DIRECTORIES; while (1) { cp = path + baselen + !!baselen; cp = memchr(cp, '/', path + len - cp); @@ -2113,7 +2111,6 @@ static int treat_leading_path(struct dir_struct *dir, } } strbuf_release(&sb); - dir->flags = old_flags; return rc; } diff --git a/t/t7061-wtstatus-ignore.sh b/t/t7061-wtstatus-ignore.sh index 0c394cf995..84366050da 100755 --- a/t/t7061-wtstatus-ignore.sh +++ b/t/t7061-wtstatus-ignore.sh @@ -43,11 +43,16 @@ test_expect_success 'status untracked directory with --ignored -u' ' test_cmp expected actual ' cat >expected <<\EOF -?? untracked/uncommitted +?? untracked/ !! untracked/ignored EOF -test_expect_success 'status prefixed untracked directory with --ignored' ' +test_expect_failure 'status of untracked directory with --ignored works with or without prefix' ' + git status --porcelain --ignored >tmp && + grep untracked/ tmp >actual && + rm tmp && + test_cmp expected actual && + git status --porcelain --ignored untracked/ >actual && test_cmp expected actual ' From patchwork Tue Dec 10 20:00:22 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Arver via GitGitGadget X-Patchwork-Id: 11283163 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E4DCE1593 for ; Tue, 10 Dec 2019 20:00:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id C31A82077B for ; Tue, 10 Dec 2019 20:00:39 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="uct62cGw" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727064AbfLJUAf (ORCPT ); Tue, 10 Dec 2019 15:00:35 -0500 Received: from mail-wr1-f67.google.com ([209.85.221.67]:39406 "EHLO mail-wr1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725999AbfLJUAd (ORCPT ); Tue, 10 Dec 2019 15:00:33 -0500 Received: by mail-wr1-f67.google.com with SMTP id y11so21493419wrt.6 for ; Tue, 10 Dec 2019 12:00:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=H1rfc3Xakcj/y7/qH6kpuw834RYNToBjdWGDvsRGWdg=; b=uct62cGwhQxj7rBRZqbJvB+i/1SmS9Bh1eTx1CmmjS4qDYAoL2crQyWBUafh146V/G 49TCUATPbNOSkGecozq5pSQ03oT3A8dBf1eJZCJgWEdvEuipT4aB7rAg0ryd9Hxeri8Y mZxYWFMSWGxPaLMOnk9sVZbgHnI+E66rLw15Om+d+ceodJjYXVpjE96LgN1Qn48NT55i QxAjbDnSkKWUQT56/gDcNybkOXOdhVRxHjSqtJlfqgD6jXesOjwwKJR8yokzAjyfXqdR Af8wIxYPai+f2apU+To4m+HuGNWNc7eSEYfWWUTY0R+pqcYQfcJWDoXsU4XqfsmLgIlh ltLQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=H1rfc3Xakcj/y7/qH6kpuw834RYNToBjdWGDvsRGWdg=; b=FUBiG2ikNQ8pqJDFgN9OQiwVAQv1RHRx+xquC1m7xMbRGQHM4cHHWkScb29g2gfqDN evwiuIVpL7BVdRzK6tYuOm+Uw4jifgvUp63RknxD1EnlTKkq1NBoihi1f9WzPMsheyTv g/A1lykynIphzEtz5zh4i4vvK7Vq4myJ8/U+8H3FNjV4SHOVIvXvwxXlhWHnYf1nh/gF DYx/U7NqBqZ8ORbe46TKfPAlB09kDA5Ee7JezVCFvijy4TD6QjVhjvJPWXVplmGK7qmp nLjCcl404OO8Sdu2celU7mXI5lOb8rh6NCL72pGrDJ0YbemmNal6U2qZLkUwe7BQwhJx oLPA== X-Gm-Message-State: APjAAAV0poiFBxtbHeOOcBYvugDFdTHlF1Q6Th4bVWI6VmaUlQXlhxYS NJZI7NUCQv3Q+0ZK6QHowFKlg0QG X-Google-Smtp-Source: APXvYqyF4gYUhlKL9tLCsfbH8YbCxQAQhdtndtDbTgF09fMWLWAHreP25dW5xZdhNV/BYsqrr4oxcQ== X-Received: by 2002:adf:fe8c:: with SMTP id l12mr4881837wrr.215.1576008031802; Tue, 10 Dec 2019 12:00:31 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id 60sm4610644wrn.86.2019.12.10.12.00.31 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 10 Dec 2019 12:00:31 -0800 (PST) Message-Id: In-Reply-To: References: From: "Elijah Newren via GitGitGadget" Date: Tue, 10 Dec 2019 20:00:22 +0000 Subject: [PATCH v2 3/8] dir: remove stray quote character in comment Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: blees@dcon.de, gitster@pobox.com, kyle@kyleam.com, sxlijin@gmail.com, Junio C Hamano , Elijah Newren Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Elijah Newren Signed-off-by: Elijah Newren --- dir.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/dir.c b/dir.c index 0dd5266629..5dacacd469 100644 --- a/dir.c +++ b/dir.c @@ -373,7 +373,7 @@ static int match_pathspec_item(const struct index_state *istate, !ps_strncmp(item, match, name, namelen)) return MATCHED_RECURSIVELY_LEADING_PATHSPEC; - /* name" doesn't match up to the first wild character */ + /* name doesn't match up to the first wild character */ if (item->nowildcard_len < item->len && ps_strncmp(item, match, name, item->nowildcard_len - prefix)) From patchwork Tue Dec 10 20:00:23 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Arver via GitGitGadget X-Patchwork-Id: 11283171 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7B9A1109A for ; Tue, 10 Dec 2019 20:00:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 59B6220838 for ; Tue, 10 Dec 2019 20:00:47 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Zm1kp6od" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726674AbfLJUAf (ORCPT ); Tue, 10 Dec 2019 15:00:35 -0500 Received: from mail-wr1-f65.google.com ([209.85.221.65]:44554 "EHLO mail-wr1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726884AbfLJUAe (ORCPT ); Tue, 10 Dec 2019 15:00:34 -0500 Received: by mail-wr1-f65.google.com with SMTP id q10so21512083wrm.11 for ; Tue, 10 Dec 2019 12:00:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=xbAMdwUoczvFExTFmHXxzR/2/texrkL36eKuhAIvlbk=; b=Zm1kp6od8k/PTUGlhnBZRXbX+fPM2LUCh6w6ZzLkbIds16Jy/eypmlY43xViRDSBye fx690S/kdp0NceEAgoBBzvtu2KsOYvPJs7vMF20ZOjN/lpvZaaUCcF1RJdouUGcO5rJC 5YaPKRejDTs0QBHCkQ3uiEz6aBJnVT4TCNys4kdXmFpYf0ohYJSDp/tUFQORrm+ncdZp wtfib8qdj2bpykeBLk/Z+yVVMhXekuvd9XbDucfR69Dg0+wqm/cz5kGqfRK7LIb2NFyJ rLoegMLhDlEPvQEV5WnWKqRqev+0rQtQ72WApbGMihXRo2F3/Y4mGAbXn4JP7INPjRhp 3cgg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=xbAMdwUoczvFExTFmHXxzR/2/texrkL36eKuhAIvlbk=; b=NPjXjkfo7tKzJSkb8Z7hM/d9HEGk5fepJ5in1rcQosFbzzs6DAaKb+5WoDZRCN4A/6 urwm218ELye8cBs0t2EF00aIjMiQLCMU3IIWUyiVPDZib1fdJQ1t3g2xR3DtELK5b8HX oVbcxykSYGQHWI2/Uoozp3TktxDguFnbL0xGCADWNltkguyHYCdBEvGf0YXwRIR39Brt zAEHmR/zaAGZ202HpqZRhrn6Xl9gFmuEqkwYHH6ERrwUJK5O9bxwd1NjwSzZe5fNFcvl sM6VcA7Vo1xZqtUNMlV/2b7s/FlqN7TymJJ8OoeHwuZiniirn36rsNGgdjgdkl/Q/Sxu nM4Q== X-Gm-Message-State: APjAAAWpYXy3Uw98JrBIuADU7vaj+zhEFHyVgJFDKXX83G8gLaPBzi6W X40Jd37iMq8I/3d8VYzpYeC7USpq X-Google-Smtp-Source: APXvYqxaUtYzMwBZQRRHCa9/QP6WLY0RIxzn0P6ViodAIt5RwLUvPSfb+RmO434P0XcOd2GM3noIYA== X-Received: by 2002:adf:ec48:: with SMTP id w8mr5313469wrn.19.1576008032492; Tue, 10 Dec 2019 12:00:32 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id h2sm4480990wrv.66.2019.12.10.12.00.31 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 10 Dec 2019 12:00:32 -0800 (PST) Message-Id: <8d2d98eec3c620c55100a322087ef83e4f51c8cf.1576008027.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Elijah Newren via GitGitGadget" Date: Tue, 10 Dec 2019 20:00:23 +0000 Subject: [PATCH v2 4/8] dir: exit before wildcard fall-through if there is no wildcard Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: blees@dcon.de, gitster@pobox.com, kyle@kyleam.com, sxlijin@gmail.com, Junio C Hamano , Elijah Newren Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Elijah Newren The DO_MATCH_LEADING_PATHSPEC had a fall-through case for if there was a wildcard, noting that we don't yet have enough information to determine if a further paths under the current directory might match due to the presence of wildcards. But if we have no wildcards in our pathspec, then we shouldn't get to that fall-through case. Signed-off-by: Elijah Newren --- dir.c | 7 +++++++ t/t3011-common-prefixes-and-directory-traversal.sh | 4 ++-- 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/dir.c b/dir.c index 5dacacd469..517a569e10 100644 --- a/dir.c +++ b/dir.c @@ -379,6 +379,13 @@ static int match_pathspec_item(const struct index_state *istate, item->nowildcard_len - prefix)) return 0; + /* + * name has no wildcard, and it didn't match as a leading + * pathspec so return. + */ + if (item->nowildcard_len == item->len) + return 0; + /* * Here is where we would perform a wildmatch to check if * "name" can be matched as a directory (or a prefix) against diff --git a/t/t3011-common-prefixes-and-directory-traversal.sh b/t/t3011-common-prefixes-and-directory-traversal.sh index 54f80c62b8..d6e161ddd8 100755 --- a/t/t3011-common-prefixes-and-directory-traversal.sh +++ b/t/t3011-common-prefixes-and-directory-traversal.sh @@ -92,7 +92,7 @@ test_expect_failure 'git ls-files -o untracked_repo/ does not recurse' ' test_cmp expect actual ' -test_expect_failure 'git ls-files -o untracked_dir untracked_repo recurses into untracked_dir only' ' +test_expect_success 'git ls-files -o untracked_dir untracked_repo recurses into untracked_dir only' ' cat <<-EOF >expect && untracked_dir/empty untracked_repo/ @@ -110,7 +110,7 @@ test_expect_success 'git ls-files -o untracked_dir/ untracked_repo/ recurses int test_cmp expect actual ' -test_expect_failure 'git ls-files -o --directory untracked_dir untracked_repo does not recurse' ' +test_expect_success 'git ls-files -o --directory untracked_dir untracked_repo does not recurse' ' cat <<-EOF >expect && untracked_dir/ untracked_repo/ From patchwork Tue Dec 10 20:00:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Arver via GitGitGadget X-Patchwork-Id: 11283167 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E9B6F138C for ; Tue, 10 Dec 2019 20:00:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id C779D2077B for ; Tue, 10 Dec 2019 20:00:45 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="QX3Q/3ik" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727109AbfLJUAl (ORCPT ); Tue, 10 Dec 2019 15:00:41 -0500 Received: from mail-wr1-f65.google.com ([209.85.221.65]:36713 "EHLO mail-wr1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726100AbfLJUAf (ORCPT ); Tue, 10 Dec 2019 15:00:35 -0500 Received: by mail-wr1-f65.google.com with SMTP id z3so21550327wru.3 for ; Tue, 10 Dec 2019 12:00:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=+CgvyKrM09apU1qkOsJft1P4o4wqjmDRS+ywvTmG7xU=; b=QX3Q/3ikcZVVB3OiemY1ggPzZ1sIEnyUMW+gm9MGMn6b2epSo2Sf/crBZN7znIehhy pfRXSA5V9tqc0DPaoWf2QXuVVm2UzBOjF6AqcU+840kgj+kYUBBpPmnTAOC8g9YkwD9h uwLRcMv9rlpexjRL37mSySsIwNtJOLYTvHW5W6iRFiBQ3jONI9eWBNJTAgzE3xkA/LPj g3UyVy1AOiw2XXlLYuykvdlbT4sWPPxElZ6fc9849LVZ1zoDRlWESkzTir1VMDLDnP8Z R5T3e59hLNDHoto9RhcijTZl3R33SdoXhUA4C+2G7QzsfRzGrqcPG066U76BD8Ya4WI9 9fZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=+CgvyKrM09apU1qkOsJft1P4o4wqjmDRS+ywvTmG7xU=; b=mcAgc5OYjlW9axlU+sLLaGknP8HoJeB5A/QzTaD8TW4IYKSwjRAk/9zewUk3g/ffvI HKZpD8faD4b4I6FfaZ0wvbMEztB3eXlygbafKEkqHf7944OZ9QbmZjwn2CMHvEp/lSeH J7PyBC8RdEl+oTeGClAjMWJUqFBh0zQhJELWsKTBJtL+GxkhJ5am9DpB9tfR2IqqrbIC auzkYPIKzSLvMXG1/e/mCc/PsNcURikI3/VuAx/q3F67WmXWmL/SaflGaIdmWVSDzmm9 xHEAZvfc2yBmsmuhj6gpnzLqcwJZW/samtyClIbCvPLRPgWvhTyHkUNq+dKl/xz0g2/1 gRIw== X-Gm-Message-State: APjAAAX+jpxNG6xIkkz25r3O+muT+cFcSH1eYbCmQVPCO3DQr+px0AVu A2fVxdpEXt+mAoje4de+rcEH9KXF X-Google-Smtp-Source: APXvYqzb/fOTEoyxzqOF+5uAeoMivyYMl+SN5B0bYF9MxmRYlcnsI7E9WpMbo5865xb52v4ayUFarg== X-Received: by 2002:adf:c54e:: with SMTP id s14mr5496353wrf.385.1576008033399; Tue, 10 Dec 2019 12:00:33 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id z189sm4559344wmc.2.2019.12.10.12.00.32 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 10 Dec 2019 12:00:33 -0800 (PST) Message-Id: In-Reply-To: References: From: "Elijah Newren via GitGitGadget" Date: Tue, 10 Dec 2019 20:00:24 +0000 Subject: [PATCH v2 5/8] dir: break part of read_directory_recursive() out for reuse Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: blees@dcon.de, gitster@pobox.com, kyle@kyleam.com, sxlijin@gmail.com, Junio C Hamano , Elijah Newren Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Elijah Newren Create an add_path_to_appropriate_result_list() function from the code at the end of read_directory_recursive() so we can use it elsewhere. Signed-off-by: Elijah Newren --- dir.c | 60 ++++++++++++++++++++++++++++++++++++----------------------- 1 file changed, 37 insertions(+), 23 deletions(-) diff --git a/dir.c b/dir.c index 517a569e10..645b44ea64 100644 --- a/dir.c +++ b/dir.c @@ -1932,6 +1932,40 @@ static void close_cached_dir(struct cached_dir *cdir) } } +static void add_path_to_appropriate_result_list(struct dir_struct *dir, + struct untracked_cache_dir *untracked, + struct cached_dir *cdir, + struct index_state *istate, + struct strbuf *path, + int baselen, + const struct pathspec *pathspec, + enum path_treatment state) +{ + /* add the path to the appropriate result list */ + switch (state) { + case path_excluded: + if (dir->flags & DIR_SHOW_IGNORED) + dir_add_name(dir, istate, path->buf, path->len); + else if ((dir->flags & DIR_SHOW_IGNORED_TOO) || + ((dir->flags & DIR_COLLECT_IGNORED) && + exclude_matches_pathspec(path->buf, path->len, + pathspec))) + dir_add_ignored(dir, istate, path->buf, path->len); + break; + + case path_untracked: + if (dir->flags & DIR_SHOW_IGNORED) + break; + dir_add_name(dir, istate, path->buf, path->len); + if (cdir->fdir) + add_untracked(untracked, path->buf + baselen); + break; + + default: + break; + } +} + /* * Read a directory tree. We currently ignore anything but * directories, regular files and symlinks. That's because git @@ -2035,29 +2069,9 @@ static enum path_treatment read_directory_recursive(struct dir_struct *dir, continue; } - /* add the path to the appropriate result list */ - switch (state) { - case path_excluded: - if (dir->flags & DIR_SHOW_IGNORED) - dir_add_name(dir, istate, path.buf, path.len); - else if ((dir->flags & DIR_SHOW_IGNORED_TOO) || - ((dir->flags & DIR_COLLECT_IGNORED) && - exclude_matches_pathspec(path.buf, path.len, - pathspec))) - dir_add_ignored(dir, istate, path.buf, path.len); - break; - - case path_untracked: - if (dir->flags & DIR_SHOW_IGNORED) - break; - dir_add_name(dir, istate, path.buf, path.len); - if (cdir.fdir) - add_untracked(untracked, path.buf + baselen); - break; - - default: - break; - } + add_path_to_appropriate_result_list(dir, untracked, &cdir, + istate, &path, baselen, + pathspec, state); } close_cached_dir(&cdir); out: From patchwork Tue Dec 10 20:00:25 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Arver via GitGitGadget X-Patchwork-Id: 11283161 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 50708109A for ; Tue, 10 Dec 2019 20:00:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 1BFA92077B for ; Tue, 10 Dec 2019 20:00:39 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="X6xXv7Vq" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727081AbfLJUAh (ORCPT ); Tue, 10 Dec 2019 15:00:37 -0500 Received: from mail-wr1-f67.google.com ([209.85.221.67]:38298 "EHLO mail-wr1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726783AbfLJUAg (ORCPT ); Tue, 10 Dec 2019 15:00:36 -0500 Received: by mail-wr1-f67.google.com with SMTP id y17so21535211wrh.5 for ; Tue, 10 Dec 2019 12:00:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=RSA39FD61/54jfe87pt4sZEjUV+TsBmB9NCWY4MD0PM=; b=X6xXv7VqiarpNhWIo1V4ihiR+dKuiskw/H1vyISJygXlkYU/W91LfCMoMSX1drpV0c UOSHSFfH6QTRXdrotYyNLAL2eotFk36LAG3hr7hpDeKLFV1XgS3fOXUdOjXc+YfZ6vLt ipSq6DIcmMm3nJcAulKocjqITJLD817RiM/YCJZDk4nSMWW+ixLO5b/KZnYjuqOogAOq gpP9+UA91BkHPUPVMS7VhNqufgxuuTYsaGjZhcNGqvFQf/sREuzdJjNV4Jc0cZfjrjpN vhHCPaCh76LPWSuRmkq2+xABwmYVbVUBPuntavGtYeSn53Xw5kyZh1W1wWnzhclnQQSF cKzA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=RSA39FD61/54jfe87pt4sZEjUV+TsBmB9NCWY4MD0PM=; b=W4VWR3BbI5WV2qRdDMHbATztMTba7I2BN7P73YvERRlN0UMIq8pC1P+rNu9irbZoR3 AYQ30CbDytmRW4qfgg0kUlZdJaWIWumAvxwqYFJtYu8cK22EDGQK8DWTdD8Ls6004pAb psDdchXC28l8YFJ/rYpdMWrHcYMkjvhFLy959oJ84Bc5bg7/uoSMS+p4TLyz7mSgeMmk ZcLpEO4jfKOTCGKs/4tWkyFBoE3KIHVayT/ramS8sEEXyQTcK3PSa4mjyuQBlDBbfJ2v e4zwSyDFWOfyE4nKTZtxExMf8PRUCWD6DDpAfwx7uG5oSsCyDmfds+Do75MUrWf0pRNp usgg== X-Gm-Message-State: APjAAAUgfs+SyoR/AsUh4cvISCO65w5WoVzNfe2fe2EEKz78vYbkfoA7 w6q9+5JBFXK0Zt8moYpB8vDvoR80 X-Google-Smtp-Source: APXvYqytg45YlFRVICP9sjVCBxweCCwOY9KYRkvYsiUNZbZsTG5X9dmBxg9RD++e/CkL34Tq0gm02w== X-Received: by 2002:a5d:4f90:: with SMTP id d16mr5031046wru.395.1576008034152; Tue, 10 Dec 2019 12:00:34 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id i5sm4373703wrv.34.2019.12.10.12.00.33 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 10 Dec 2019 12:00:33 -0800 (PST) Message-Id: <9839aca00a10b16d96c47db631ac025281ffc864.1576008027.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Elijah Newren via GitGitGadget" Date: Tue, 10 Dec 2019 20:00:25 +0000 Subject: [PATCH v2 6/8] dir: fix checks on common prefix directory Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: blees@dcon.de, gitster@pobox.com, kyle@kyleam.com, sxlijin@gmail.com, Junio C Hamano , Elijah Newren Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Elijah Newren Many years ago, the directory traversing logic had an optimization that would always recurse into any directory that was a common prefix of all the pathspecs without walking the leading directories to get down to the desired directory. Thus, git ls-files -o .git/ # case A would notice that .git/ was a common prefix of all pathspecs (since it is the only pathspec listed), and then traverse into it and start showing unknown files under that directory. Unfortunately, .git/ is not a directory we should be traversing into, which made this optimization problematic. This also affected cases like git ls-files -o --exclude-standard t/ # case B where t/ was in the .gitignore file and thus isn't interesting and shouldn't be recursed into. It also affected cases like git ls-files -o --directory untracked_dir/ # case C where untracked_dir/ is indeed untracked and thus interesting, but the --directory flag means we only want to show the directory itself, not recurse into it and start listing untracked files below it. The case B class of bugs were noted and fixed in commits 16e2cfa90993 ("read_directory(): further split treat_path()", 2010-01-08) and 48ffef966c76 ("ls-files: fix overeager pathspec optimization", 2010-01-08), with the idea being that we first wanted to check whether the common prefix was interesting. The former patch noted that treat_path() couldn't be used when checking the common prefix because treat_path() requires a dir_entry() and we haven't read any directories at the point we are checking the common prefix. So, that patch split treat_one_path() out of treat_path(). The latter patch then created a new treat_leading_path() which duplicated by hand the bits of treat_path() that couldn't be broken out and then called treat_one_path() for the remainder. There were three problems with this approach: * The duplicated logic in treat_leading_path() accidentally missed the check for special paths (such as is_dot_or_dotdot and matching ".git"), causing case A types of bugs to continue to be an issue. * The treat_leading_path() logic assumed we should traverse into anything where path_treatment was not path_none, i.e. it perpetuated class C types of bugs. * It meant we had split logic that needed to kept in sync, running the risk that people introduced new inconsistencies (such as in commit be8a84c52669, which we reverted earlier in this series, or in commit df5bcdf83ae which we'll fix in a subsequent commit) Fix most these problems by making treat_leading_path() not only loop over each leading path component, but calling treat_path() directly on each. To do so, we have to create a synthetic dir_entry, but that only takes a few lines. Then, pay attention to the path_treatment result we get from treat_path() and don't treat path_excluded, path_untracked, and path_recurse all the same as path_recurse. This leaves one remaining problem, the new inconsistency from commit df5bcdf83ae. That will be addressed in a subsequent commit. Signed-off-by: Elijah Newren --- dir.c | 54 +++++++++++++++---- ...common-prefixes-and-directory-traversal.sh | 6 +-- 2 files changed, 46 insertions(+), 14 deletions(-) diff --git a/dir.c b/dir.c index 645b44ea64..9c71a9ac21 100644 --- a/dir.c +++ b/dir.c @@ -2102,37 +2102,69 @@ static int treat_leading_path(struct dir_struct *dir, const struct pathspec *pathspec) { struct strbuf sb = STRBUF_INIT; - int baselen, rc = 0; + int prevlen, baselen; const char *cp; + struct cached_dir cdir; + struct dirent de; + enum path_treatment state = path_none; + + /* + * For each directory component of path, we are going to check whether + * that path is relevant given the pathspec. For example, if path is + * foo/bar/baz/ + * then we will ask treat_path() whether we should go into foo, then + * whether we should go into bar, then whether baz is relevant. + * Checking each is important because e.g. if path is + * .git/info/ + * then we need to check .git to know we shouldn't traverse it. + * If the return from treat_path() is: + * * path_none, for any path, we return false. + * * path_recurse, for all path components, we return true + * * for some intermediate component, we make sure + * to add that path to the relevant list but return false + * signifying that we shouldn't recurse into it. + */ while (len && path[len - 1] == '/') len--; if (!len) return 1; + + memset(&cdir, 0, sizeof(cdir)); + memset(&de, 0, sizeof(de)); + cdir.de = &de; + de.d_type = DT_DIR; baselen = 0; + prevlen = 0; while (1) { - cp = path + baselen + !!baselen; + prevlen = baselen + !!baselen; + cp = path + prevlen; cp = memchr(cp, '/', path + len - cp); if (!cp) baselen = len; else baselen = cp - path; - strbuf_setlen(&sb, 0); + strbuf_reset(&sb); strbuf_add(&sb, path, baselen); if (!is_directory(sb.buf)) break; - if (simplify_away(sb.buf, sb.len, pathspec)) - break; - if (treat_one_path(dir, NULL, istate, &sb, baselen, pathspec, - DT_DIR, NULL) == path_none) + strbuf_reset(&sb); + strbuf_add(&sb, path, prevlen); + memcpy(de.d_name, path+prevlen, baselen-prevlen); + de.d_name[baselen-prevlen] = '\0'; + state = treat_path(dir, NULL, &cdir, istate, &sb, prevlen, + pathspec); + if (state != path_recurse) break; /* do not recurse into it */ - if (len <= baselen) { - rc = 1; + if (len <= baselen) break; /* finished checking */ - } } + add_path_to_appropriate_result_list(dir, NULL, &cdir, istate, + &sb, baselen, pathspec, + state); + strbuf_release(&sb); - return rc; + return state == path_recurse; } static const char *get_ident_string(void) diff --git a/t/t3011-common-prefixes-and-directory-traversal.sh b/t/t3011-common-prefixes-and-directory-traversal.sh index d6e161ddd8..098fddc75b 100755 --- a/t/t3011-common-prefixes-and-directory-traversal.sh +++ b/t/t3011-common-prefixes-and-directory-traversal.sh @@ -74,7 +74,7 @@ test_expect_success 'git ls-files -o --directory untracked_dir does not recurse' test_cmp expect actual ' -test_expect_failure 'git ls-files -o --directory untracked_dir/ does not recurse' ' +test_expect_success 'git ls-files -o --directory untracked_dir/ does not recurse' ' echo untracked_dir/ >expect && git ls-files -o --directory untracked_dir/ >actual && test_cmp expect actual @@ -86,7 +86,7 @@ test_expect_success 'git ls-files -o untracked_repo does not recurse' ' test_cmp expect actual ' -test_expect_failure 'git ls-files -o untracked_repo/ does not recurse' ' +test_expect_success 'git ls-files -o untracked_repo/ does not recurse' ' echo untracked_repo/ >expect && git ls-files -o untracked_repo/ >actual && test_cmp expect actual @@ -133,7 +133,7 @@ test_expect_success 'git ls-files -o .git shows nothing' ' test_must_be_empty actual ' -test_expect_failure 'git ls-files -o .git/ shows nothing' ' +test_expect_success 'git ls-files -o .git/ shows nothing' ' git ls-files -o .git/ >actual && test_must_be_empty actual ' From patchwork Tue Dec 10 20:00:26 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Arver via GitGitGadget X-Patchwork-Id: 11283173 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 53679109A for ; Tue, 10 Dec 2019 20:00:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 2894C2077B for ; Tue, 10 Dec 2019 20:00:49 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="fnHvGgJh" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726647AbfLJUAs (ORCPT ); Tue, 10 Dec 2019 15:00:48 -0500 Received: from mail-wm1-f65.google.com ([209.85.128.65]:52205 "EHLO mail-wm1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726631AbfLJUAh (ORCPT ); Tue, 10 Dec 2019 15:00:37 -0500 Received: by mail-wm1-f65.google.com with SMTP id d73so1335465wmd.1 for ; Tue, 10 Dec 2019 12:00:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=QlQlvIUb7pVmcwx0APSrUZaVTcp4QONqDKymM3daESw=; b=fnHvGgJhrjd1XijPU0fcwqcRZH2iW1HaYvdPHWthY5Asuu61mcGZTAX3FEyrwtKxRj T7BGKZcUTudS6p6hJCc+YDCP4o6JVg13TvoY3twlsaV7Z+V8WRIkc5BDDA5Y93RCDinG oui4L3/7J0S10oaN6MWGGyQ82+Ij8OdVtHIF2zuwSkpJ2CldO8xzLkNgnJR7HKHflr5M efBD0UpViYVsSUSqnff/Kino0yCA4Mn+soHWqiTxN8jWJL2JxOsnEXtzOoHq6+NTqBVN AC/bw6AseBdHu0ayzFAb5QZ5h9eCWdoVzRNoC3l7mJTXsP1gS2wT1RS8Jv+r8lAQUqnr as3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=QlQlvIUb7pVmcwx0APSrUZaVTcp4QONqDKymM3daESw=; b=UEJGqpw44VGA2kh9FxkrRHvFf/4dCQw9BaOkQiAV/5E1oqQJvPQFwnaNzHsnxTGll2 +3RT/n3m/NEjB4Ar8zy3LSVoi5jGz2bEzodvFCMoSM336SGQ5v93MYVfa5g96IijcVGT hwXtQ3gOibOaXolvmb+NYKKFZ097N848UKyjdBqeFZaXEvGGjaL9FQY5wvzXVllAafDa zbniWUSQlShQtdU+WaciYvvMezri/g/ChJ27/cpZcPb+FSulMn3HAPU1KgurfQTmZzp6 ZU58xsUUotyhWhfXXYQChffcTjoUI8p1cXHTiCtgmbcmJkM5V8ViNzt/0Z9GMGFcN0+t ZHzw== X-Gm-Message-State: APjAAAVaP/WBzxuveYzsAlQf7eRo1cRq9uUXAV1lNjBetQkjAGWyWQWh IyKhmaMCO9/i9sOZQ56etF8ANdBb X-Google-Smtp-Source: APXvYqzfNlxPb7+f80OVx6bUpMMnuBOHYvqT+unWFmj/3ciSoGpxA9LIJLcLtRq3GacUxKL8yb6GbQ== X-Received: by 2002:a05:600c:214f:: with SMTP id v15mr7215977wml.110.1576008035207; Tue, 10 Dec 2019 12:00:35 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id s82sm4323098wms.28.2019.12.10.12.00.34 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 10 Dec 2019 12:00:34 -0800 (PST) Message-Id: In-Reply-To: References: From: "Elijah Newren via GitGitGadget" Date: Tue, 10 Dec 2019 20:00:26 +0000 Subject: [PATCH v2 7/8] dir: synchronize treat_leading_path() and read_directory_recursive() Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: blees@dcon.de, gitster@pobox.com, kyle@kyleam.com, sxlijin@gmail.com, Junio C Hamano , Elijah Newren Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Elijah Newren Our optimization to avoid calling into read_directory_recursive() when all pathspecs have a common leading directory mean that we need to match the logic that read_directory_recursive() would use if we had just called it from the root. Since it does more than call treat_path() we need to copy that same logic. Alternatively, we could try to change treat_path to return path_recurse for an untracked directory under the given special circumstances that this logic checks for, but a simple switch results in many test failures such as 'git clean -d' not wiping out untracked but empty directories. To work around that, we'd need the caller of treat_path to check for path_recurse and sometimes special case it into path_untracked. In other words, we'd still have extra logic in both places. Needing to duplicate logic like this means it is guaranteed someone will eventually need to make further changes and forget to update both locations. It is tempting to just nuke the leading_directory special casing to avoid such bugs and simplify the code, but unpack_trees' verify_clean_subdirectory() also calls read_directory() and does so with a non-empty leading path, so I'm hesitant to try to restructure further. Add obnoxious warnings to treat_leading_path() and read_directory_recursive() to try to warn people of such problems. Signed-off-by: Elijah Newren --- dir.c | 30 +++++++++++++++++++ ...common-prefixes-and-directory-traversal.sh | 2 +- t/t7061-wtstatus-ignore.sh | 2 +- 3 files changed, 32 insertions(+), 2 deletions(-) diff --git a/dir.c b/dir.c index 9c71a9ac21..bb6e481909 100644 --- a/dir.c +++ b/dir.c @@ -1990,6 +1990,15 @@ static enum path_treatment read_directory_recursive(struct dir_struct *dir, struct untracked_cache_dir *untracked, int check_only, int stop_at_first_file, const struct pathspec *pathspec) { + /* + * WARNING WARNING WARNING: + * + * Any updates to the traversal logic here may need corresponding + * updates in treat_leading_path(). See the commit message for the + * commit adding this warning as well as the commit preceding it + * for details. + */ + struct cached_dir cdir; enum path_treatment state, subdir_state, dir_state = path_none; struct strbuf path = STRBUF_INIT; @@ -2101,6 +2110,15 @@ static int treat_leading_path(struct dir_struct *dir, const char *path, int len, const struct pathspec *pathspec) { + /* + * WARNING WARNING WARNING: + * + * Any updates to the traversal logic here may need corresponding + * updates in treat_leading_path(). See the commit message for the + * commit adding this warning as well as the commit preceding it + * for details. + */ + struct strbuf sb = STRBUF_INIT; int prevlen, baselen; const char *cp; @@ -2154,6 +2172,18 @@ static int treat_leading_path(struct dir_struct *dir, de.d_name[baselen-prevlen] = '\0'; state = treat_path(dir, NULL, &cdir, istate, &sb, prevlen, pathspec); + if (state == path_untracked && + get_dtype(cdir.de, istate, sb.buf, sb.len) == DT_DIR && + (dir->flags & DIR_SHOW_IGNORED_TOO || + do_match_pathspec(istate, pathspec, sb.buf, sb.len, + baselen, NULL, DO_MATCH_LEADING_PATHSPEC) == MATCHED_RECURSIVELY_LEADING_PATHSPEC)) { + add_path_to_appropriate_result_list(dir, NULL, &cdir, + istate, + &sb, baselen, + pathspec, state); + state = path_recurse; + } + if (state != path_recurse) break; /* do not recurse into it */ if (len <= baselen) diff --git a/t/t3011-common-prefixes-and-directory-traversal.sh b/t/t3011-common-prefixes-and-directory-traversal.sh index 098fddc75b..3da5b2b6e7 100755 --- a/t/t3011-common-prefixes-and-directory-traversal.sh +++ b/t/t3011-common-prefixes-and-directory-traversal.sh @@ -195,7 +195,7 @@ test_expect_success 'git ls-files -o consistent between one or two dirs' ' # ls-files doesn't have a way to request showing both untracked and ignored # files at the same time, so use `git status --ignored` -test_expect_failure 'git status --ignored shows same files under dir with or without pathspec' ' +test_expect_success 'git status --ignored shows same files under dir with or without pathspec' ' cat <<-EOF >expect && ?? an_untracked_dir/ !! an_untracked_dir/ignored diff --git a/t/t7061-wtstatus-ignore.sh b/t/t7061-wtstatus-ignore.sh index 84366050da..e4cf5484f9 100755 --- a/t/t7061-wtstatus-ignore.sh +++ b/t/t7061-wtstatus-ignore.sh @@ -47,7 +47,7 @@ cat >expected <<\EOF !! untracked/ignored EOF -test_expect_failure 'status of untracked directory with --ignored works with or without prefix' ' +test_expect_success 'status of untracked directory with --ignored works with or without prefix' ' git status --porcelain --ignored >tmp && grep untracked/ tmp >actual && rm tmp && From patchwork Tue Dec 10 20:00:27 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Arver via GitGitGadget X-Patchwork-Id: 11283165 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7EC531593 for ; Tue, 10 Dec 2019 20:00:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 5D514207FF for ; Tue, 10 Dec 2019 20:00:44 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="HgCTp7xg" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727117AbfLJUAl (ORCPT ); Tue, 10 Dec 2019 15:00:41 -0500 Received: from mail-wr1-f66.google.com ([209.85.221.66]:41112 "EHLO mail-wr1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727077AbfLJUAi (ORCPT ); Tue, 10 Dec 2019 15:00:38 -0500 Received: by mail-wr1-f66.google.com with SMTP id c9so21526747wrw.8 for ; Tue, 10 Dec 2019 12:00:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=zze0uH3Nj4ZOmvZ1O0g/xeKVlrddGGSxpyF/tpjrz40=; b=HgCTp7xgXoz8tKiKqtfXE7pfEr3f4KBzDz6ZgFO6JxXdZnO75gqsmjlvAFH5NnXUnd 5eVUxAK+/hj+0qsRDgiCjQTe1qSzPrWjoaNTi5oYwgQXXEy0/r+TaDQB/xDOLko3ev98 eJCOCKc9kBUxpsj7A394bz3g42UTrY7rx6KCcmTnr/4WxL3nC6B0b56p8amI8IbqfUqZ e16Xs3t0UHYIkWFInVgJfA+PBdpox7OJ/HyJ5dL0Jx6tN1rlG+tnZChasLiz2VB2eB0E mrqK4w3PZSKR849jZGOiK9HLXAmnascvMh6hicJeMq02UtLo6QKDVgMkLicLsphDDcfK AvlQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=zze0uH3Nj4ZOmvZ1O0g/xeKVlrddGGSxpyF/tpjrz40=; b=Ta+GQw7+uQPe0yUsC+jbqAl72w78AVFGL13bLcGCBbGeSdWNh0UT7f66GPb++KS9nQ WX1vBLRnqBbbiTkOpdRDu8RRyLIdDW0NN60vcwVwFunRrqpF2PAf+gGfimHPwMl++qHv gOsQYoeUdNJcxQUQX8HjU27LHUTE6qoWYrbQG6ADH8PSp9/L3+gIlQ2cNG75cxd1jwOt fv3JWL44+r6CpVaDiaHf/Fx3S05KR6OfcxgIF9wWZzNH7UhlBu3zA6iPJPmg9nLluyoe PALe6eK28Qn8JoheHNRy0T1JA5+H8Bh1JSTiO9Yfc5nk4g/2c0YgQ0prjNEcO9FT+z52 xJMg== X-Gm-Message-State: APjAAAUMrvJNItveUqYFKuLuJCpoC6nZs5D8YP5F25rNmEhyVuFLdNfz bEyeVhr52e84k3ERUnorfZHbat4h X-Google-Smtp-Source: APXvYqwq2eK4vao8wmsrRccwKufI20dX0+eZ5qn0vZK6ACm7nYAck0EnJK9Xxvp3XMp9Ip0q0uwD5g== X-Received: by 2002:adf:fbc9:: with SMTP id d9mr5029941wrs.20.1576008035971; Tue, 10 Dec 2019 12:00:35 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id w19sm4093039wmc.22.2019.12.10.12.00.35 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 10 Dec 2019 12:00:35 -0800 (PST) Message-Id: <77b57e44fdd819536c9be37a6190a79792efbccb.1576008027.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Elijah Newren via GitGitGadget" Date: Tue, 10 Dec 2019 20:00:27 +0000 Subject: [PATCH v2 8/8] dir: consolidate similar code in treat_directory() Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: blees@dcon.de, gitster@pobox.com, kyle@kyleam.com, sxlijin@gmail.com, Junio C Hamano , Elijah Newren Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Elijah Newren Both the DIR_SKIP_NESTED_GIT and DIR_NO_GITLINKS cases were checking for whether a path was actually a nonbare repository. That code could be shared, with just the result of how to act differing between the two cases. Signed-off-by: Elijah Newren --- dir.c | 18 +++++++----------- 1 file changed, 7 insertions(+), 11 deletions(-) diff --git a/dir.c b/dir.c index bb6e481909..04541b798b 100644 --- a/dir.c +++ b/dir.c @@ -1461,6 +1461,8 @@ static enum path_treatment treat_directory(struct dir_struct *dir, const char *dirname, int len, int baselen, int exclude, const struct pathspec *pathspec) { + int nested_repo = 0; + /* The "len-1" is to strip the final '/' */ switch (directory_exists_in_index(istate, dirname, len-1)) { case index_directory: @@ -1470,15 +1472,16 @@ static enum path_treatment treat_directory(struct dir_struct *dir, return path_none; case index_nonexistent: - if (dir->flags & DIR_SKIP_NESTED_GIT) { - int nested_repo; + if ((dir->flags & DIR_SKIP_NESTED_GIT) || + !(dir->flags & DIR_NO_GITLINKS)) { struct strbuf sb = STRBUF_INIT; strbuf_addstr(&sb, dirname); nested_repo = is_nonbare_repository_dir(&sb); strbuf_release(&sb); - if (nested_repo) - return path_none; } + if (nested_repo) + return ((dir->flags & DIR_SKIP_NESTED_GIT) ? path_none : + (exclude ? path_excluded : path_untracked)); if (dir->flags & DIR_SHOW_OTHER_DIRECTORIES) break; @@ -1506,13 +1509,6 @@ static enum path_treatment treat_directory(struct dir_struct *dir, return path_none; } - if (!(dir->flags & DIR_NO_GITLINKS)) { - struct strbuf sb = STRBUF_INIT; - strbuf_addstr(&sb, dirname); - if (is_nonbare_repository_dir(&sb)) - return exclude ? path_excluded : path_untracked; - strbuf_release(&sb); - } return path_recurse; }