From patchwork Wed Dec 18 19:29:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Koji Nakamaru via GitGitGadget X-Patchwork-Id: 11301735 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 297CE1892 for ; Wed, 18 Dec 2019 19:29:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id F156B206D7 for ; Wed, 18 Dec 2019 19:29:52 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="VRlxFmpR" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727560AbfLRT3w (ORCPT ); Wed, 18 Dec 2019 14:29:52 -0500 Received: from mail-wr1-f65.google.com ([209.85.221.65]:36581 "EHLO mail-wr1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727185AbfLRT3v (ORCPT ); Wed, 18 Dec 2019 14:29:51 -0500 Received: by mail-wr1-f65.google.com with SMTP id z3so3563332wru.3 for ; Wed, 18 Dec 2019 11:29:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=RjUWfqMLtbjNIPA16XEQPNcw4ayZ9uv9uVm9KQNo/JQ=; b=VRlxFmpReAXrMfiBb6fR1RclV/+pc7/0Tlv6HqSg630pT4CXohbCzubNiZIakZG5l2 gIcaLZiIgzVqdUEn0JTHpds8LDacC7NuryJCKj+lZJeqybIjKsMMNYLdNCsPxi232B7p 0WhTYBtYBcJSdebo2B20IHTIjoHNUv7yHcqibIW0Ws3m7u+OQGfvUi1nqb8ouUYA9FXe tCm3zag6ZidSp87Hikeccq6cgqHU4sos26OE3TQg3s2uSxvJ9u6vE+ReRsMBhBj5hW0B 2nU5l8IHl3eRevLk4DyUy+ZLfcrRtu65M0e0LmInHy2+a3FXbAPfqq7nWhXHQcJoCt5I /Uww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=RjUWfqMLtbjNIPA16XEQPNcw4ayZ9uv9uVm9KQNo/JQ=; b=VszqLD6Cjg/atXtOwn5l9irdkCnN4E5jEV3ldx7bqFXyxLwd/9Hy5iD+3thg1JPk/E dEsFKNFKJkkyHLFR+s/ToKe2UMjBtGWqO+oMuMBpRkyaz9yRC9bBEL2zPXfL0KTtagBt Xhe6MDnW/pXe8oMZXITJyS0bcFc3MQtORCUtY36L7sgUTbkzqKDX8nMZ0KrOzjsMLi4f fr439WRsFSCrxs2yQNIlCACq/Yui+A9lBNwZ2GruEONLAmCf9JveI+Yj4HFgX9l5au7l SLRN5n7gFcP8qrMKhBQROXkqU4fDAJ9pSg6pUJvRIM1/T6xh3JHkhiuwGTetnilKh3IS +TFA== X-Gm-Message-State: APjAAAWQtadJVcAd2RuzNr5H774dUydHRDdzEDJux41vk3cxq1i3Nbop 7QkHM5FA89lci+KcCSdPMR8vCJOt X-Google-Smtp-Source: APXvYqwrjfsVpoxvfDhHTOABnv5UUc7PgZFM/RbWaQ08vOngY7hrSOb2FuEYSvOPjzV2/FD8LzX03Q== X-Received: by 2002:adf:eb51:: with SMTP id u17mr4681922wrn.29.1576697389232; Wed, 18 Dec 2019 11:29:49 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id a1sm3571663wrr.80.2019.12.18.11.29.48 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 18 Dec 2019 11:29:48 -0800 (PST) Message-Id: <6d659b230239b530c90fbcc15c9a869f8d5dd5c5.1576697386.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Elijah Newren via GitGitGadget" Date: Wed, 18 Dec 2019 19:29:39 +0000 Subject: [PATCH v4 1/8] t3011: demonstrate directory traversal failures Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: blees@dcon.de, gitster@pobox.com, kyle@kyleam.com, sxlijin@gmail.com, Junio C Hamano , Elijah Newren Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Elijah Newren Add several tests demonstrating directory traversal failures of various sorts in dir.c (and one similar looking test that turns out to be a git_fnmatch bug). A lot of these tests look like near duplicates of each other, but an optimization path in dir.c to pre-descend into a common prefix and the specialized treatment of trailing slashes in dir.c mean the tiny differences are sometimes important and potentially cause different codepaths to be explored. Of the 7 failing tests, 2 are new to git-2.24.0 (tweaked by side effects of the en/clean-nested-with-ignored-topic); the other 5 also failed under git-2.23.0 and earlier. Signed-off-by: Elijah Newren --- ...common-prefixes-and-directory-traversal.sh | 209 ++++++++++++++++++ 1 file changed, 209 insertions(+) create mode 100755 t/t3011-common-prefixes-and-directory-traversal.sh diff --git a/t/t3011-common-prefixes-and-directory-traversal.sh b/t/t3011-common-prefixes-and-directory-traversal.sh new file mode 100755 index 0000000000..54f80c62b8 --- /dev/null +++ b/t/t3011-common-prefixes-and-directory-traversal.sh @@ -0,0 +1,209 @@ +#!/bin/sh + +test_description='directory traversal handling, especially with common prefixes' + +. ./test-lib.sh + +test_expect_success 'setup' ' + test_commit hello && + + >empty && + mkdir untracked_dir && + >untracked_dir/empty && + git init untracked_repo && + >untracked_repo/empty && + + cat <<-EOF >.gitignore && + ignored + an_ignored_dir/ + EOF + mkdir an_ignored_dir && + mkdir an_untracked_dir && + >an_ignored_dir/ignored && + >an_ignored_dir/untracked && + >an_untracked_dir/ignored && + >an_untracked_dir/untracked +' + +test_expect_success 'git ls-files -o shows the right entries' ' + cat <<-EOF >expect && + .gitignore + actual + an_ignored_dir/ignored + an_ignored_dir/untracked + an_untracked_dir/ignored + an_untracked_dir/untracked + empty + expect + untracked_dir/empty + untracked_repo/ + EOF + git ls-files -o >actual && + test_cmp expect actual +' + +test_expect_success 'git ls-files -o --exclude-standard shows the right entries' ' + cat <<-EOF >expect && + .gitignore + actual + an_untracked_dir/untracked + empty + expect + untracked_dir/empty + untracked_repo/ + EOF + git ls-files -o --exclude-standard >actual && + test_cmp expect actual +' + +test_expect_success 'git ls-files -o untracked_dir recurses' ' + echo untracked_dir/empty >expect && + git ls-files -o untracked_dir >actual && + test_cmp expect actual +' + +test_expect_success 'git ls-files -o untracked_dir/ recurses' ' + echo untracked_dir/empty >expect && + git ls-files -o untracked_dir/ >actual && + test_cmp expect actual +' + +test_expect_success 'git ls-files -o --directory untracked_dir does not recurse' ' + echo untracked_dir/ >expect && + git ls-files -o --directory untracked_dir >actual && + test_cmp expect actual +' + +test_expect_failure 'git ls-files -o --directory untracked_dir/ does not recurse' ' + echo untracked_dir/ >expect && + git ls-files -o --directory untracked_dir/ >actual && + test_cmp expect actual +' + +test_expect_success 'git ls-files -o untracked_repo does not recurse' ' + echo untracked_repo/ >expect && + git ls-files -o untracked_repo >actual && + test_cmp expect actual +' + +test_expect_failure 'git ls-files -o untracked_repo/ does not recurse' ' + echo untracked_repo/ >expect && + git ls-files -o untracked_repo/ >actual && + test_cmp expect actual +' + +test_expect_failure 'git ls-files -o untracked_dir untracked_repo recurses into untracked_dir only' ' + cat <<-EOF >expect && + untracked_dir/empty + untracked_repo/ + EOF + git ls-files -o untracked_dir untracked_repo >actual && + test_cmp expect actual +' + +test_expect_success 'git ls-files -o untracked_dir/ untracked_repo/ recurses into untracked_dir only' ' + cat <<-EOF >expect && + untracked_dir/empty + untracked_repo/ + EOF + git ls-files -o untracked_dir/ untracked_repo/ >actual && + test_cmp expect actual +' + +test_expect_failure 'git ls-files -o --directory untracked_dir untracked_repo does not recurse' ' + cat <<-EOF >expect && + untracked_dir/ + untracked_repo/ + EOF + git ls-files -o --directory untracked_dir untracked_repo >actual && + test_cmp expect actual +' + +test_expect_success 'git ls-files -o --directory untracked_dir/ untracked_repo/ does not recurse' ' + cat <<-EOF >expect && + untracked_dir/ + untracked_repo/ + EOF + git ls-files -o --directory untracked_dir/ untracked_repo/ >actual && + test_cmp expect actual +' + +test_expect_success 'git ls-files -o .git shows nothing' ' + git ls-files -o .git >actual && + test_must_be_empty actual +' + +test_expect_failure 'git ls-files -o .git/ shows nothing' ' + git ls-files -o .git/ >actual && + test_must_be_empty actual +' + +test_expect_success FUNNYNAMES 'git ls-files -o untracked_* recurses appropriately' ' + mkdir "untracked_*" && + >"untracked_*/empty" && + + cat <<-EOF >expect && + untracked_*/empty + untracked_dir/empty + untracked_repo/ + EOF + git ls-files -o "untracked_*" >actual && + test_cmp expect actual +' + +# It turns out fill_directory returns the right paths, but ls-files' post-call +# filtering in show_dir_entry() via calling dir_path_match() which ends up +# in git_fnmatch() has logic for PATHSPEC_ONESTAR that assumes the pathspec +# must match the full path; it doesn't check it for matching a leading +# directory. +test_expect_failure FUNNYNAMES 'git ls-files -o untracked_*/ recurses appropriately' ' + cat <<-EOF >expect && + untracked_*/empty + untracked_dir/empty + untracked_repo/ + EOF + git ls-files -o "untracked_*/" >actual && + test_cmp expect actual +' + +test_expect_success FUNNYNAMES 'git ls-files -o --directory untracked_* does not recurse' ' + cat <<-EOF >expect && + untracked_*/ + untracked_dir/ + untracked_repo/ + EOF + git ls-files -o --directory "untracked_*" >actual && + test_cmp expect actual +' + +test_expect_success FUNNYNAMES 'git ls-files -o --directory untracked_*/ does not recurse' ' + cat <<-EOF >expect && + untracked_*/ + untracked_dir/ + untracked_repo/ + EOF + git ls-files -o --directory "untracked_*/" >actual && + test_cmp expect actual +' + +test_expect_success 'git ls-files -o consistent between one or two dirs' ' + git ls-files -o --exclude-standard an_ignored_dir/ an_untracked_dir/ >tmp && + ! grep ^an_ignored_dir/ tmp >expect && + git ls-files -o --exclude-standard an_ignored_dir/ >actual && + test_cmp expect actual +' + +# ls-files doesn't have a way to request showing both untracked and ignored +# files at the same time, so use `git status --ignored` +test_expect_failure 'git status --ignored shows same files under dir with or without pathspec' ' + cat <<-EOF >expect && + ?? an_untracked_dir/ + !! an_untracked_dir/ignored + EOF + git status --porcelain --ignored >output && + grep an_untracked_dir output >expect && + git status --porcelain --ignored an_untracked_dir/ >actual && + test_cmp expect actual +' + +test_done From patchwork Wed Dec 18 19:29:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Koji Nakamaru via GitGitGadget X-Patchwork-Id: 11301733 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BBC691593 for ; Wed, 18 Dec 2019 19:29:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 99AA620674 for ; Wed, 18 Dec 2019 19:29:52 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ANoIYT9R" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727557AbfLRT3w (ORCPT ); Wed, 18 Dec 2019 14:29:52 -0500 Received: from mail-wm1-f65.google.com ([209.85.128.65]:36933 "EHLO mail-wm1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727484AbfLRT3v (ORCPT ); Wed, 18 Dec 2019 14:29:51 -0500 Received: by mail-wm1-f65.google.com with SMTP id f129so3194193wmf.2 for ; Wed, 18 Dec 2019 11:29:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=O5bdAKERjFqZXYHkEGuYnzcy5g3lzmd91EZmqMjMLxI=; b=ANoIYT9RmotzFGAC7755K52g45bkv6qRWoMjZrqepOAUOQ6m1O8vLE3BzfOd2L0pfO 5cYk+L4NJ25b/M9V55UoLN/nwifi1XkBRwzlPNmyQIUz9rzVV8YpJcxEMtwR9Y3U1PkT T6nIrzQokva/G53uM4gktKBDjoPgsjZtq3m8LIjDo1LbrpMZRA+jrPcAMruo/IUnsb3V 7OLE2yQZm7q4Ao8i0YxzMSAShTSJQTdIpEAkoe0qTFrYYtDKHoNdB7TS9RRD5UNNXEPE WB7k+imzLZtm6RBmBN3vMMYeZTlUHhk3vQQzCTP2/60NMganCcL7yUyQaTU/kTDjO/Gm 477w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=O5bdAKERjFqZXYHkEGuYnzcy5g3lzmd91EZmqMjMLxI=; b=eJ+ktsq4oIvXEUBGGtr7AaZ3tNj6TbGxVjWADm68uOj8qkP1plN9rm6zT2H1KLdWqI N+n1rrubeclMntdM5CCgSlBJJYqWLAoM5dkESWb2GvZRzAfob6nYvBw8asLgCI4708qz MeNDDLlcGTPKDt/XOe38BuOzepdHneJccPVkXxeMMzpY/BsAMqwOrY6YxIlXh4X4RKun JRBGlUmFiForCTEuiGbmJtaWngURCp0rU8ht2xkMU9HQiDYV9ko1a5agZtHCtDJoqyap q2DQFAWVZtP/gICkZMPAuZuMUGRhCyKpimB9VFvLuAZw8wKK/LC3W46NLENqsQvzcZ4f vIrw== X-Gm-Message-State: APjAAAUr8K9yHGnBhNB2Kdfr+B5q/WQz5QsJnRrtYIWYMFSTRvlnLbFj pdF5+E+wSLQnGpX3EbF27GKu2JWb X-Google-Smtp-Source: APXvYqweZZgwtAyqq0dvyQRCc/DOwYWKlmUgMoHrQVEr6WW539Oc7XF/ll2/JlGj1FKavfCnn0a4cg== X-Received: by 2002:a1c:a543:: with SMTP id o64mr4897194wme.73.1576697390002; Wed, 18 Dec 2019 11:29:50 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id v83sm3574305wmg.16.2019.12.18.11.29.49 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 18 Dec 2019 11:29:49 -0800 (PST) Message-Id: <79f2b561742372234a52dc7194dd72026c6e96ec.1576697386.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Elijah Newren via GitGitGadget" Date: Wed, 18 Dec 2019 19:29:40 +0000 Subject: [PATCH v4 2/8] Revert "dir.c: make 'git-status --ignored' work within leading directories" Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: blees@dcon.de, gitster@pobox.com, kyle@kyleam.com, sxlijin@gmail.com, Junio C Hamano , Elijah Newren Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Elijah Newren Commit be8a84c52669 ("dir.c: make 'git-status --ignored' work within leading directories", 2013-04-15) noted that git status --ignored would not list ignored files and directories within if was untracked, and modified the behavior to make it show them. However, it did so via a hack that broke consistency; it would show paths under differently than a simple git status --ignored | grep would show them. A correct fix is slightly more involved, and complicated slightly by this hack, so we revert this commit (but keep corrected versions of the testcases) and will later fix the original bug with a subsequent patch. Some history may be helpful: A very, very similar case to the commit we are reverting was raised in commit 48ffef966c76 ("ls-files: fix overeager pathspec optimization", 2010-01-08); but it actually went in somewhat the opposite direction. In that commit, it mentioned how git ls-files -o --exclude-standard t/ used to show untracked files under t/ even when t/ was ignored, and then changed the behavior to stop showing untracked files under an ignored directory. More importantly, this commit considered keeping this behavior but noted that it would be inconsistent with the behavior when multiple pathspecs were specified and thus rejected it. The reason for this whole inconsistency when one pathspec is specified versus zero or two is because common prefixes of pathspecs are sent through a different set of checks (in treat_leading_path()) than normal file/directory traversal (those go through read_directory_recursive() and treat_path()). As such, for consistency, one needs to check that both codepaths produce the same result. Revert commit be8a84c526691667fc04a8241d93a3de1de298ab, except instead of removing the testcase it added, modify it to check for correct and consistent behavior. A subsequent patch in this series will fix the testcase. Signed-off-by: Elijah Newren --- dir.c | 3 --- t/t7061-wtstatus-ignore.sh | 9 +++++++-- 2 files changed, 7 insertions(+), 5 deletions(-) diff --git a/dir.c b/dir.c index 61f559f980..0dd5266629 100644 --- a/dir.c +++ b/dir.c @@ -2083,14 +2083,12 @@ static int treat_leading_path(struct dir_struct *dir, struct strbuf sb = STRBUF_INIT; int baselen, rc = 0; const char *cp; - int old_flags = dir->flags; while (len && path[len - 1] == '/') len--; if (!len) return 1; baselen = 0; - dir->flags &= ~DIR_SHOW_OTHER_DIRECTORIES; while (1) { cp = path + baselen + !!baselen; cp = memchr(cp, '/', path + len - cp); @@ -2113,7 +2111,6 @@ static int treat_leading_path(struct dir_struct *dir, } } strbuf_release(&sb); - dir->flags = old_flags; return rc; } diff --git a/t/t7061-wtstatus-ignore.sh b/t/t7061-wtstatus-ignore.sh index 0c394cf995..84366050da 100755 --- a/t/t7061-wtstatus-ignore.sh +++ b/t/t7061-wtstatus-ignore.sh @@ -43,11 +43,16 @@ test_expect_success 'status untracked directory with --ignored -u' ' test_cmp expected actual ' cat >expected <<\EOF -?? untracked/uncommitted +?? untracked/ !! untracked/ignored EOF -test_expect_success 'status prefixed untracked directory with --ignored' ' +test_expect_failure 'status of untracked directory with --ignored works with or without prefix' ' + git status --porcelain --ignored >tmp && + grep untracked/ tmp >actual && + rm tmp && + test_cmp expected actual && + git status --porcelain --ignored untracked/ >actual && test_cmp expected actual ' From patchwork Wed Dec 18 19:29:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Koji Nakamaru via GitGitGadget X-Patchwork-Id: 11301739 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 118F914E3 for ; Wed, 18 Dec 2019 19:29:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id E2F27206D7 for ; Wed, 18 Dec 2019 19:29:54 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="H8eH2Rd8" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727573AbfLRT3y (ORCPT ); Wed, 18 Dec 2019 14:29:54 -0500 Received: from mail-wr1-f65.google.com ([209.85.221.65]:37700 "EHLO mail-wr1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727538AbfLRT3w (ORCPT ); Wed, 18 Dec 2019 14:29:52 -0500 Received: by mail-wr1-f65.google.com with SMTP id w15so3555183wru.4 for ; Wed, 18 Dec 2019 11:29:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=H1rfc3Xakcj/y7/qH6kpuw834RYNToBjdWGDvsRGWdg=; b=H8eH2Rd8eIkjZ03b4g+SLET4Dfy7r9o4BOwRHA5q7+pEG/3CAGgI2BB6qczT1Qhm8i yLGGHq4u4ZxKQCfV4QryFah5hIEenJBXNw8tZ0SoIUaFTVsl+RHYGxdOBIxE/pmQURah DEYl2hF6ZvX4Oj8U/2C2jSdECB5pFxmUqO+UHrpdfHphmwM8NnZYjfDUajthD7yypgBJ G8fYWCZfoCM/O4mXdPyQ6LkDCQP1D4w+bxd2t1FVGG8laqgHg0mDTf2u4frP1+CCTSvr 8iXWkxiAcyH94YInsQ2c1H3HYe5NIG37aJPfw8HBTojOdz/9o21CBqMIfyWEG6EzU09h d+KA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=H1rfc3Xakcj/y7/qH6kpuw834RYNToBjdWGDvsRGWdg=; b=OySNqsL4NF+V6D5xXxATtyyncrsZQzn5TDEKeTs1aEf7zvy4O1d8a5ExHdcLfFz0Ou MycHx3vB3WtksENIQYNTyyTmEk72FagREsr2OI3nMqDcmh+jj3Wnn/GLXDRPjvginAEH mYlmNo1sb1q+wjxOv8Atwvo31i2vDSb3tR7pCrGwhs2gx3nRZufoOvCDjnMKxumDxzBY X8H2H/vZadb/V7EizbzjfYBQ4fE6PKe2MPnOFkHixAjzf22knwEvN+lh709ea3ypEniY qtCsGqmx59VS5UPCOxqu4ipNvi7F5zxTGhdmYAOGcLiDy+n4lw73KkyW0P5ZF9CNq2VE WUrg== X-Gm-Message-State: APjAAAV9sRdqkH6A1iRyj7DnWYpXdT7Lf7OHkPLkKh1KPVIy3WK1tm/Q klsfGsDgPrWVMnWgQgkmH9V15p1+ X-Google-Smtp-Source: APXvYqxfkaVuTg35V3LpDXIzU3wZXnG81plXHv0geJniwIdKl7+knghmZyVol/UJ5fgU7sfSbDuBvw== X-Received: by 2002:a5d:50d2:: with SMTP id f18mr4714356wrt.366.1576697390785; Wed, 18 Dec 2019 11:29:50 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id y22sm3465214wma.35.2019.12.18.11.29.50 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 18 Dec 2019 11:29:50 -0800 (PST) Message-Id: In-Reply-To: References: From: "Elijah Newren via GitGitGadget" Date: Wed, 18 Dec 2019 19:29:41 +0000 Subject: [PATCH v4 3/8] dir: remove stray quote character in comment Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: blees@dcon.de, gitster@pobox.com, kyle@kyleam.com, sxlijin@gmail.com, Junio C Hamano , Elijah Newren Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Elijah Newren Signed-off-by: Elijah Newren --- dir.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/dir.c b/dir.c index 0dd5266629..5dacacd469 100644 --- a/dir.c +++ b/dir.c @@ -373,7 +373,7 @@ static int match_pathspec_item(const struct index_state *istate, !ps_strncmp(item, match, name, namelen)) return MATCHED_RECURSIVELY_LEADING_PATHSPEC; - /* name" doesn't match up to the first wild character */ + /* name doesn't match up to the first wild character */ if (item->nowildcard_len < item->len && ps_strncmp(item, match, name, item->nowildcard_len - prefix)) From patchwork Wed Dec 18 19:29:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Koji Nakamaru via GitGitGadget X-Patchwork-Id: 11301741 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C3ADB14E3 for ; Wed, 18 Dec 2019 19:29:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id A14ED206D7 for ; Wed, 18 Dec 2019 19:29:55 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="eMxrW/nd" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727570AbfLRT3y (ORCPT ); Wed, 18 Dec 2019 14:29:54 -0500 Received: from mail-wm1-f65.google.com ([209.85.128.65]:39297 "EHLO mail-wm1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727121AbfLRT3x (ORCPT ); Wed, 18 Dec 2019 14:29:53 -0500 Received: by mail-wm1-f65.google.com with SMTP id 20so3069715wmj.4 for ; Wed, 18 Dec 2019 11:29:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=xbAMdwUoczvFExTFmHXxzR/2/texrkL36eKuhAIvlbk=; b=eMxrW/ndFjc70f5o/0z7zOE5eFOBUnADz+0Hvm8hwLaafNLi8/Krxu+jNySNTeNpTh 4OG8hFO1VXFfqzaI5iv+7JBHWuCDZ3NZ8hHofH/usY/8Mdtt8F+dq/bUTZAWt2XPOVS7 3RjcIxYnpnwoEv7JaqTa4aq+s8SYZJpRkNWwoT85D4pkA9bMVF+LB6+AxSDl6ho16BJj FQLAfczqTEq2K+oIIJrEoU0ya1nxg/n+jamXjsa/Lijbl1ppwVbUji5o9ztZtnlx4H6O 7no7cSDdhZbe9Sbiy2uAktuLF6CsrvwCtasxInMy+26o2GpLhimP1N/5dAUAnBYAlmmo qMiA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=xbAMdwUoczvFExTFmHXxzR/2/texrkL36eKuhAIvlbk=; b=s3kSIL+rnPaM0Kc6ByVpUqImm/+3cpI8SKl8FyeDZXA5u7T0qWafSMDhsNIjrwmgDw zFsW3zH8BOKM6rj34LswsyFvpVF//wwOKSjyfmTi8a7hY4sfKQeZUXcHzroq/4juK10T SUSARuFlEYalnUxuEBT3exG8g0yE7Zx3TaOXjtC+ALV8grq6godEVpFIiw2dOroU6x6w SHuVq6zSRynuaAGJTdMALWxiAXpakGyIOtfqbBb/uDpVQMHMe9eQAvVRTGObQ3y8pe2V Nx76Co/6cklE67c6cs/DfD+DixtOQjA0artT2XYokTHxshbZTzQamDHtGMnEu/DJOH47 kLCA== X-Gm-Message-State: APjAAAUNBj+xd3f8XfHETNqbuxsOnWgYqfUuSFCn0hk4tuhv+FUzn6q1 snoq53oAL7DbEbnjFosBAJc188U3 X-Google-Smtp-Source: APXvYqxkD961nzMx1zsG3CwuDWiUySlAlu9JJA3UBZHfA8wUxBOQPdggffY1utCXIurEF6sCCE/KVA== X-Received: by 2002:a1c:3187:: with SMTP id x129mr5186435wmx.91.1576697391478; Wed, 18 Dec 2019 11:29:51 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id c4sm3575175wml.7.2019.12.18.11.29.50 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 18 Dec 2019 11:29:51 -0800 (PST) Message-Id: <8d2d98eec3c620c55100a322087ef83e4f51c8cf.1576697386.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Elijah Newren via GitGitGadget" Date: Wed, 18 Dec 2019 19:29:42 +0000 Subject: [PATCH v4 4/8] dir: exit before wildcard fall-through if there is no wildcard Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: blees@dcon.de, gitster@pobox.com, kyle@kyleam.com, sxlijin@gmail.com, Junio C Hamano , Elijah Newren Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Elijah Newren The DO_MATCH_LEADING_PATHSPEC had a fall-through case for if there was a wildcard, noting that we don't yet have enough information to determine if a further paths under the current directory might match due to the presence of wildcards. But if we have no wildcards in our pathspec, then we shouldn't get to that fall-through case. Signed-off-by: Elijah Newren --- dir.c | 7 +++++++ t/t3011-common-prefixes-and-directory-traversal.sh | 4 ++-- 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/dir.c b/dir.c index 5dacacd469..517a569e10 100644 --- a/dir.c +++ b/dir.c @@ -379,6 +379,13 @@ static int match_pathspec_item(const struct index_state *istate, item->nowildcard_len - prefix)) return 0; + /* + * name has no wildcard, and it didn't match as a leading + * pathspec so return. + */ + if (item->nowildcard_len == item->len) + return 0; + /* * Here is where we would perform a wildmatch to check if * "name" can be matched as a directory (or a prefix) against diff --git a/t/t3011-common-prefixes-and-directory-traversal.sh b/t/t3011-common-prefixes-and-directory-traversal.sh index 54f80c62b8..d6e161ddd8 100755 --- a/t/t3011-common-prefixes-and-directory-traversal.sh +++ b/t/t3011-common-prefixes-and-directory-traversal.sh @@ -92,7 +92,7 @@ test_expect_failure 'git ls-files -o untracked_repo/ does not recurse' ' test_cmp expect actual ' -test_expect_failure 'git ls-files -o untracked_dir untracked_repo recurses into untracked_dir only' ' +test_expect_success 'git ls-files -o untracked_dir untracked_repo recurses into untracked_dir only' ' cat <<-EOF >expect && untracked_dir/empty untracked_repo/ @@ -110,7 +110,7 @@ test_expect_success 'git ls-files -o untracked_dir/ untracked_repo/ recurses int test_cmp expect actual ' -test_expect_failure 'git ls-files -o --directory untracked_dir untracked_repo does not recurse' ' +test_expect_success 'git ls-files -o --directory untracked_dir untracked_repo does not recurse' ' cat <<-EOF >expect && untracked_dir/ untracked_repo/ From patchwork Wed Dec 18 19:29:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Koji Nakamaru via GitGitGadget X-Patchwork-Id: 11301743 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B1BF414B7 for ; Wed, 18 Dec 2019 19:29:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 8F87D206D7 for ; Wed, 18 Dec 2019 19:29:56 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="O5l1JcIk" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727591AbfLRT3z (ORCPT ); Wed, 18 Dec 2019 14:29:55 -0500 Received: from mail-wr1-f68.google.com ([209.85.221.68]:41560 "EHLO mail-wr1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727185AbfLRT3y (ORCPT ); Wed, 18 Dec 2019 14:29:54 -0500 Received: by mail-wr1-f68.google.com with SMTP id c9so3535135wrw.8 for ; Wed, 18 Dec 2019 11:29:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=+CgvyKrM09apU1qkOsJft1P4o4wqjmDRS+ywvTmG7xU=; b=O5l1JcIknAlwSFUbXPiasoHyuf5xCOTOOkqi6bIGieIEOxeWfKfdv8oxviDfs+pbim hQkzZLA48k2bqvFh2R4+KCyIr70W2g5Nqpb/gESc45BDHDPImeJEuD5m/qexBi4voyvb QCjtrjGA+WIwi5RVUV05CGwUzFEGCqGE07LquAtGb2JiPWYQvv7yIDYAqsMfegTTr9MD Xb2u+cORtqJH3E4AtyYY6GWI/76BQNaLMuZZXZq/pgC3hscl5B9TlpmruEZd+QpFFJnl fSQ8IvXE3Qps7c6JkyUIcCRfYNJJ+VlxxS0GKSXHvxWF2lfhj3OzL47AGl1NG2ykdQQ4 vKqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=+CgvyKrM09apU1qkOsJft1P4o4wqjmDRS+ywvTmG7xU=; b=fp7FVG8S4DvpwZt2UHAGpt+bJy2L4eqrhbGxTYXVLKpLt146oqkD4Xgkfhov4YEze/ NfnuSOpFdAUEsTR32Wq0gZ4Q3miHn5yCLq3lmqGjrwemZUSvcjMQxavFzDpUnDZqOa5x Sty7J04sebdY7BNnlvFsq02lf7NbJPPDAlpHV8j+fS8xzSPFc2vx0jW6ZKGyEHL3rn9m mYU4AcgKWG+oYm28D6LyFlsG79n5kpZCy8Z2g5D1MVrdBq7GaQxIhHNJQOdMxbC+HIV6 7ecdKChV9YdlfzpR4kr2hQLne9UcyEKzlO47dpg1OvjkCmfBH5TFPbPHb5tKbs0SFBTX mhew== X-Gm-Message-State: APjAAAV4TR+hsyfrrQoKnlztjKjmI+yfzWED6Qqm5fuFtcXq6GTLerRb OBVQQK3XPjAmLbR7WR964qKy5tfl X-Google-Smtp-Source: APXvYqzqu2HOIYo0bVthV1sRQfnN8oZQnklfKXMAe7Nt9LA1pMy2ibHh65svbtkDV4j8ysu7mfw82A== X-Received: by 2002:adf:d0c1:: with SMTP id z1mr4838247wrh.371.1576697392175; Wed, 18 Dec 2019 11:29:52 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id z18sm3421984wmf.21.2019.12.18.11.29.51 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 18 Dec 2019 11:29:51 -0800 (PST) Message-Id: In-Reply-To: References: From: "Elijah Newren via GitGitGadget" Date: Wed, 18 Dec 2019 19:29:43 +0000 Subject: [PATCH v4 5/8] dir: break part of read_directory_recursive() out for reuse Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: blees@dcon.de, gitster@pobox.com, kyle@kyleam.com, sxlijin@gmail.com, Junio C Hamano , Elijah Newren Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Elijah Newren Create an add_path_to_appropriate_result_list() function from the code at the end of read_directory_recursive() so we can use it elsewhere. Signed-off-by: Elijah Newren --- dir.c | 60 ++++++++++++++++++++++++++++++++++++----------------------- 1 file changed, 37 insertions(+), 23 deletions(-) diff --git a/dir.c b/dir.c index 517a569e10..645b44ea64 100644 --- a/dir.c +++ b/dir.c @@ -1932,6 +1932,40 @@ static void close_cached_dir(struct cached_dir *cdir) } } +static void add_path_to_appropriate_result_list(struct dir_struct *dir, + struct untracked_cache_dir *untracked, + struct cached_dir *cdir, + struct index_state *istate, + struct strbuf *path, + int baselen, + const struct pathspec *pathspec, + enum path_treatment state) +{ + /* add the path to the appropriate result list */ + switch (state) { + case path_excluded: + if (dir->flags & DIR_SHOW_IGNORED) + dir_add_name(dir, istate, path->buf, path->len); + else if ((dir->flags & DIR_SHOW_IGNORED_TOO) || + ((dir->flags & DIR_COLLECT_IGNORED) && + exclude_matches_pathspec(path->buf, path->len, + pathspec))) + dir_add_ignored(dir, istate, path->buf, path->len); + break; + + case path_untracked: + if (dir->flags & DIR_SHOW_IGNORED) + break; + dir_add_name(dir, istate, path->buf, path->len); + if (cdir->fdir) + add_untracked(untracked, path->buf + baselen); + break; + + default: + break; + } +} + /* * Read a directory tree. We currently ignore anything but * directories, regular files and symlinks. That's because git @@ -2035,29 +2069,9 @@ static enum path_treatment read_directory_recursive(struct dir_struct *dir, continue; } - /* add the path to the appropriate result list */ - switch (state) { - case path_excluded: - if (dir->flags & DIR_SHOW_IGNORED) - dir_add_name(dir, istate, path.buf, path.len); - else if ((dir->flags & DIR_SHOW_IGNORED_TOO) || - ((dir->flags & DIR_COLLECT_IGNORED) && - exclude_matches_pathspec(path.buf, path.len, - pathspec))) - dir_add_ignored(dir, istate, path.buf, path.len); - break; - - case path_untracked: - if (dir->flags & DIR_SHOW_IGNORED) - break; - dir_add_name(dir, istate, path.buf, path.len); - if (cdir.fdir) - add_untracked(untracked, path.buf + baselen); - break; - - default: - break; - } + add_path_to_appropriate_result_list(dir, untracked, &cdir, + istate, &path, baselen, + pathspec, state); } close_cached_dir(&cdir); out: From patchwork Wed Dec 18 19:29:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Koji Nakamaru via GitGitGadget X-Patchwork-Id: 11301745 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CF5A614B7 for ; Wed, 18 Dec 2019 19:29:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id A3AE1206D7 for ; Wed, 18 Dec 2019 19:29:57 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="XEMPTIzp" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727598AbfLRT35 (ORCPT ); Wed, 18 Dec 2019 14:29:57 -0500 Received: from mail-wr1-f65.google.com ([209.85.221.65]:45331 "EHLO mail-wr1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727297AbfLRT3z (ORCPT ); Wed, 18 Dec 2019 14:29:55 -0500 Received: by mail-wr1-f65.google.com with SMTP id j42so3510542wrj.12 for ; Wed, 18 Dec 2019 11:29:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=90a8hTYzo1NrhO4e2Whvq4/bM4YvPykArFGr5qCbY7Y=; b=XEMPTIzp99H6fcfgott/HfDmE2EWwgY6KMx5T7mXbaFeTKzJ6HFgftfKYu1qyOiAsN qSinCCfZ7Lud9KLpAKeSQwUo2l496Gmubz1NoaHv4A2ujFZ/4g1crU2R7mxacArEmp+T xK8foR514qWUBjIB1xRhVQlqYqh0ZOLGO+Pfv8bB7PPdYF0ANp5M8I/KfKjAyrlSjF10 n1712Jr4Zi9pyGae37oZNkUhyKAP9Ve0bcWEul+trfDmpHKORCg8ukGK0lPxrgAebU8k xsC7LhZBMFZb23dkSG4uRy1hewPTl+Aj5o/oaJFDBDAGByMA+9V/oLqAlKkiYlFGKxbq BN+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=90a8hTYzo1NrhO4e2Whvq4/bM4YvPykArFGr5qCbY7Y=; b=r1mUy/qm5c27EM0F4263jsEhX1g+9/eB0NAFGVpvOPKH43mgdDkN65BMFROoZqe0R8 mZNcXYK0v59hM+wGHyLn72Cs2NNPOjMTGoj39PO+GWgRWnaICPoF22gS1p5V8n54bhhD gwP096FDML3fmaNO1f5WYmMvrTTYa8awby4TPEb/TWD+KNV1s0L+OqjBwNQvSv9iB9ty Fn9sXbvD5i8LRTfYA6NwYKr5bel4p9qscAFwr4kXmxz0VZe6CFNe54HptDQtiGwHHuND mlxv/kCErBQ8pj5EAINyUhY4I/SUsNYPoOsGP7A8nHBEYfQV2JY+JTKpchgI6SN42YS3 mw+A== X-Gm-Message-State: APjAAAUT5hATxYnlKNVeVmWHtJn3ym5hsoQJw7Mbe1MM1NPOuS60q/kH 4QxDwJ7wFfbuQOuWWWbFAvx7Ffm2 X-Google-Smtp-Source: APXvYqw5e+oleIbj5oNLhEg73m47gKtqVXmSUx6CkxkqPD8BHsHWVVTKKisabm4QBKtcLG1McUbw7w== X-Received: by 2002:a5d:6ac3:: with SMTP id u3mr4836398wrw.25.1576697392968; Wed, 18 Dec 2019 11:29:52 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id e16sm3544086wrs.73.2019.12.18.11.29.52 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 18 Dec 2019 11:29:52 -0800 (PST) Message-Id: <1f3978aa461929923eeb5037e69be6569f0ba331.1576697386.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Elijah Newren via GitGitGadget" Date: Wed, 18 Dec 2019 19:29:44 +0000 Subject: [PATCH v4 6/8] dir: fix checks on common prefix directory Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: blees@dcon.de, gitster@pobox.com, kyle@kyleam.com, sxlijin@gmail.com, Junio C Hamano , Elijah Newren Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Elijah Newren Many years ago, the directory traversing logic had an optimization that would always recurse into any directory that was a common prefix of all the pathspecs without walking the leading directories to get down to the desired directory. Thus, git ls-files -o .git/ # case A would notice that .git/ was a common prefix of all pathspecs (since it is the only pathspec listed), and then traverse into it and start showing unknown files under that directory. Unfortunately, .git/ is not a directory we should be traversing into, which made this optimization problematic. This also affected cases like git ls-files -o --exclude-standard t/ # case B where t/ was in the .gitignore file and thus isn't interesting and shouldn't be recursed into. It also affected cases like git ls-files -o --directory untracked_dir/ # case C where untracked_dir/ is indeed untracked and thus interesting, but the --directory flag means we only want to show the directory itself, not recurse into it and start listing untracked files below it. The case B class of bugs were noted and fixed in commits 16e2cfa90993 ("read_directory(): further split treat_path()", 2010-01-08) and 48ffef966c76 ("ls-files: fix overeager pathspec optimization", 2010-01-08), with the idea being that we first wanted to check whether the common prefix was interesting. The former patch noted that treat_path() couldn't be used when checking the common prefix because treat_path() requires a dir_entry() and we haven't read any directories at the point we are checking the common prefix. So, that patch split treat_one_path() out of treat_path(). The latter patch then created a new treat_leading_path() which duplicated by hand the bits of treat_path() that couldn't be broken out and then called treat_one_path() for the remainder. There were three problems with this approach: * The duplicated logic in treat_leading_path() accidentally missed the check for special paths (such as is_dot_or_dotdot and matching ".git"), causing case A types of bugs to continue to be an issue. * The treat_leading_path() logic assumed we should traverse into anything where path_treatment was not path_none, i.e. it perpetuated class C types of bugs. * It meant we had split logic that needed to kept in sync, running the risk that people introduced new inconsistencies (such as in commit be8a84c52669, which we reverted earlier in this series, or in commit df5bcdf83ae which we'll fix in a subsequent commit) Fix most these problems by making treat_leading_path() not only loop over each leading path component, but calling treat_path() directly on each. To do so, we have to create a synthetic dir_entry, but that only takes a few lines. Then, pay attention to the path_treatment result we get from treat_path() and don't treat path_excluded, path_untracked, and path_recurse all the same as path_recurse. This leaves one remaining problem, the new inconsistency from commit df5bcdf83ae. That will be addressed in a subsequent commit. Signed-off-by: Elijah Newren Signed-off-by: Junio C Hamano --- dir.c | 57 +++++++++++++++---- ...common-prefixes-and-directory-traversal.sh | 6 +- 2 files changed, 49 insertions(+), 14 deletions(-) diff --git a/dir.c b/dir.c index 645b44ea64..1de5d7ad33 100644 --- a/dir.c +++ b/dir.c @@ -2102,37 +2102,72 @@ static int treat_leading_path(struct dir_struct *dir, const struct pathspec *pathspec) { struct strbuf sb = STRBUF_INIT; - int baselen, rc = 0; + int prevlen, baselen; const char *cp; + struct cached_dir cdir; + struct dirent *de; + enum path_treatment state = path_none; + + /* + * For each directory component of path, we are going to check whether + * that path is relevant given the pathspec. For example, if path is + * foo/bar/baz/ + * then we will ask treat_path() whether we should go into foo, then + * whether we should go into bar, then whether baz is relevant. + * Checking each is important because e.g. if path is + * .git/info/ + * then we need to check .git to know we shouldn't traverse it. + * If the return from treat_path() is: + * * path_none, for any path, we return false. + * * path_recurse, for all path components, we return true + * * for some intermediate component, we make sure + * to add that path to the relevant list but return false + * signifying that we shouldn't recurse into it. + */ while (len && path[len - 1] == '/') len--; if (!len) return 1; + + de = xcalloc(1, sizeof(struct dirent)+len+1); + memset(&cdir, 0, sizeof(cdir)); + cdir.de = de; +#if defined(DT_UNKNOWN) && !defined(NO_D_TYPE_IN_DIRENT) + de->d_type = DT_DIR; +#endif baselen = 0; + prevlen = 0; while (1) { - cp = path + baselen + !!baselen; + prevlen = baselen + !!baselen; + cp = path + prevlen; cp = memchr(cp, '/', path + len - cp); if (!cp) baselen = len; else baselen = cp - path; - strbuf_setlen(&sb, 0); + strbuf_reset(&sb); strbuf_add(&sb, path, baselen); if (!is_directory(sb.buf)) break; - if (simplify_away(sb.buf, sb.len, pathspec)) - break; - if (treat_one_path(dir, NULL, istate, &sb, baselen, pathspec, - DT_DIR, NULL) == path_none) + strbuf_reset(&sb); + strbuf_add(&sb, path, prevlen); + memcpy(de->d_name, path+prevlen, baselen-prevlen); + de->d_name[baselen-prevlen] = '\0'; + state = treat_path(dir, NULL, &cdir, istate, &sb, prevlen, + pathspec); + if (state != path_recurse) break; /* do not recurse into it */ - if (len <= baselen) { - rc = 1; + if (len <= baselen) break; /* finished checking */ - } } + add_path_to_appropriate_result_list(dir, NULL, &cdir, istate, + &sb, baselen, pathspec, + state); + + free(de); strbuf_release(&sb); - return rc; + return state == path_recurse; } static const char *get_ident_string(void) diff --git a/t/t3011-common-prefixes-and-directory-traversal.sh b/t/t3011-common-prefixes-and-directory-traversal.sh index d6e161ddd8..098fddc75b 100755 --- a/t/t3011-common-prefixes-and-directory-traversal.sh +++ b/t/t3011-common-prefixes-and-directory-traversal.sh @@ -74,7 +74,7 @@ test_expect_success 'git ls-files -o --directory untracked_dir does not recurse' test_cmp expect actual ' -test_expect_failure 'git ls-files -o --directory untracked_dir/ does not recurse' ' +test_expect_success 'git ls-files -o --directory untracked_dir/ does not recurse' ' echo untracked_dir/ >expect && git ls-files -o --directory untracked_dir/ >actual && test_cmp expect actual @@ -86,7 +86,7 @@ test_expect_success 'git ls-files -o untracked_repo does not recurse' ' test_cmp expect actual ' -test_expect_failure 'git ls-files -o untracked_repo/ does not recurse' ' +test_expect_success 'git ls-files -o untracked_repo/ does not recurse' ' echo untracked_repo/ >expect && git ls-files -o untracked_repo/ >actual && test_cmp expect actual @@ -133,7 +133,7 @@ test_expect_success 'git ls-files -o .git shows nothing' ' test_must_be_empty actual ' -test_expect_failure 'git ls-files -o .git/ shows nothing' ' +test_expect_success 'git ls-files -o .git/ shows nothing' ' git ls-files -o .git/ >actual && test_must_be_empty actual ' From patchwork Wed Dec 18 19:29:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Koji Nakamaru via GitGitGadget X-Patchwork-Id: 11301749 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 326BF14E3 for ; Wed, 18 Dec 2019 19:30:01 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 07F04206D7 for ; Wed, 18 Dec 2019 19:30:01 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Q+rtrp7K" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727582AbfLRT37 (ORCPT ); Wed, 18 Dec 2019 14:29:59 -0500 Received: from mail-wm1-f67.google.com ([209.85.128.67]:39313 "EHLO mail-wm1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727538AbfLRT34 (ORCPT ); Wed, 18 Dec 2019 14:29:56 -0500 Received: by mail-wm1-f67.google.com with SMTP id 20so3069865wmj.4 for ; Wed, 18 Dec 2019 11:29:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=iz3flYHLH4WCewAEiphQi64vnf4FcOU8YCswG+AOn/Y=; b=Q+rtrp7K4zDBNAC2V3j4VuqI/Q44LiBBmjS14PftuKwhksfRXlpMmY+ghOi3sOv8MP OSHnwxdH8maUJWdo+FbG17CXyOToG9Na85kPi93Ec4H9pJZC5e2eAPX2GfNnPujG76gv WhNwOKZptrOzetw+orE+lRf9o8oCXJ/nqzxbEMUnJC9Zo3nxIH9BXWMAMl3+v3qnNeHN VkNkY9pk2maQ4m6yFHO4qozkuHgjZFKvAN/dC9hqvJcyY7LsJ+Qba0vZ6Y4FOu8Y8DxF 1wAOFYFCA+evSihwKoh0I8GmegPpLierc0d7f9BBvFNDhDQfGeOuduvLdbtGgbwojFHN bcAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=iz3flYHLH4WCewAEiphQi64vnf4FcOU8YCswG+AOn/Y=; b=hHfN9tpIrJRznEF2lddN1Tc0wFwF1py7ZGKnOz2+VOpvTsevaDNuZpB63iQfNEtQM3 8xNDsHqKIfqnOYllirU9nU3IV7Epd8Nd41vXoUk2IHcBUYnT5yJ4Yk0oMmd/b3/4Ozt3 dmYJ8nCpl5cNcyYdLtBlLPV8JeVV9ck/IqZmeTF17zr2Rg5ulGmxzptEJLcz9AifCt9d xi3fThvh3UvJ5Si8qF4wvFPMSLsffhYEGSc7LtuPKBQ6i9iNqUT4PKxVFAuerxIB49A4 2HjkvlW9llYyptN5zyHKgtW9FG891zDw0u04RW0hUSwCeCp8HhNhkeotZA/vk3+9OCBr 1xTw== X-Gm-Message-State: APjAAAXoYIJVvi88c3SXSI2PmVGbdKFUdR+RxKAQjA4zrgKkoLAsTw2i GZgsAnPBsBwgOzqfBsJ7vKA5QW/q X-Google-Smtp-Source: APXvYqx0FMmKft+lRUQXI17suerzzjDrM2d6pxQJ9vhPVz+6gPpOnynzUPJeTdaF5i3t+ZqbGmseWQ== X-Received: by 2002:a1c:ed0e:: with SMTP id l14mr4969391wmh.74.1576697393894; Wed, 18 Dec 2019 11:29:53 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id w17sm3596250wrt.89.2019.12.18.11.29.53 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 18 Dec 2019 11:29:53 -0800 (PST) Message-Id: <542c6e57927b968d58e4309e3a09f85914af0408.1576697387.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Elijah Newren via GitGitGadget" Date: Wed, 18 Dec 2019 19:29:45 +0000 Subject: [PATCH v4 7/8] dir: synchronize treat_leading_path() and read_directory_recursive() Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: blees@dcon.de, gitster@pobox.com, kyle@kyleam.com, sxlijin@gmail.com, Junio C Hamano , Elijah Newren Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Elijah Newren Our optimization to avoid calling into read_directory_recursive() when all pathspecs have a common leading directory mean that we need to match the logic that read_directory_recursive() would use if we had just called it from the root. Since it does more than call treat_path() we need to copy that same logic. Alternatively, we could try to change treat_path to return path_recurse for an untracked directory under the given special circumstances that this logic checks for, but a simple switch results in many test failures such as 'git clean -d' not wiping out untracked but empty directories. To work around that, we'd need the caller of treat_path to check for path_recurse and sometimes special case it into path_untracked. In other words, we'd still have extra logic in both places. Needing to duplicate logic like this means it is guaranteed someone will eventually need to make further changes and forget to update both locations. It is tempting to just nuke the leading_directory special casing to avoid such bugs and simplify the code, but unpack_trees' verify_clean_subdirectory() also calls read_directory() and does so with a non-empty leading path, so I'm hesitant to try to restructure further. Add obnoxious warnings to treat_leading_path() and read_directory_recursive() to try to warn people of such problems. Signed-off-by: Elijah Newren --- dir.c | 30 +++++++++++++++++++ ...common-prefixes-and-directory-traversal.sh | 2 +- t/t7061-wtstatus-ignore.sh | 2 +- 3 files changed, 32 insertions(+), 2 deletions(-) diff --git a/dir.c b/dir.c index 1de5d7ad33..f500fd9279 100644 --- a/dir.c +++ b/dir.c @@ -1990,6 +1990,15 @@ static enum path_treatment read_directory_recursive(struct dir_struct *dir, struct untracked_cache_dir *untracked, int check_only, int stop_at_first_file, const struct pathspec *pathspec) { + /* + * WARNING WARNING WARNING: + * + * Any updates to the traversal logic here may need corresponding + * updates in treat_leading_path(). See the commit message for the + * commit adding this warning as well as the commit preceding it + * for details. + */ + struct cached_dir cdir; enum path_treatment state, subdir_state, dir_state = path_none; struct strbuf path = STRBUF_INIT; @@ -2101,6 +2110,15 @@ static int treat_leading_path(struct dir_struct *dir, const char *path, int len, const struct pathspec *pathspec) { + /* + * WARNING WARNING WARNING: + * + * Any updates to the traversal logic here may need corresponding + * updates in treat_leading_path(). See the commit message for the + * commit adding this warning as well as the commit preceding it + * for details. + */ + struct strbuf sb = STRBUF_INIT; int prevlen, baselen; const char *cp; @@ -2156,6 +2174,18 @@ static int treat_leading_path(struct dir_struct *dir, de->d_name[baselen-prevlen] = '\0'; state = treat_path(dir, NULL, &cdir, istate, &sb, prevlen, pathspec); + if (state == path_untracked && + get_dtype(cdir.de, istate, sb.buf, sb.len) == DT_DIR && + (dir->flags & DIR_SHOW_IGNORED_TOO || + do_match_pathspec(istate, pathspec, sb.buf, sb.len, + baselen, NULL, DO_MATCH_LEADING_PATHSPEC) == MATCHED_RECURSIVELY_LEADING_PATHSPEC)) { + add_path_to_appropriate_result_list(dir, NULL, &cdir, + istate, + &sb, baselen, + pathspec, state); + state = path_recurse; + } + if (state != path_recurse) break; /* do not recurse into it */ if (len <= baselen) diff --git a/t/t3011-common-prefixes-and-directory-traversal.sh b/t/t3011-common-prefixes-and-directory-traversal.sh index 098fddc75b..3da5b2b6e7 100755 --- a/t/t3011-common-prefixes-and-directory-traversal.sh +++ b/t/t3011-common-prefixes-and-directory-traversal.sh @@ -195,7 +195,7 @@ test_expect_success 'git ls-files -o consistent between one or two dirs' ' # ls-files doesn't have a way to request showing both untracked and ignored # files at the same time, so use `git status --ignored` -test_expect_failure 'git status --ignored shows same files under dir with or without pathspec' ' +test_expect_success 'git status --ignored shows same files under dir with or without pathspec' ' cat <<-EOF >expect && ?? an_untracked_dir/ !! an_untracked_dir/ignored diff --git a/t/t7061-wtstatus-ignore.sh b/t/t7061-wtstatus-ignore.sh index 84366050da..e4cf5484f9 100755 --- a/t/t7061-wtstatus-ignore.sh +++ b/t/t7061-wtstatus-ignore.sh @@ -47,7 +47,7 @@ cat >expected <<\EOF !! untracked/ignored EOF -test_expect_failure 'status of untracked directory with --ignored works with or without prefix' ' +test_expect_success 'status of untracked directory with --ignored works with or without prefix' ' git status --porcelain --ignored >tmp && grep untracked/ tmp >actual && rm tmp && From patchwork Wed Dec 18 19:29:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Koji Nakamaru via GitGitGadget X-Patchwork-Id: 11301747 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3771114B7 for ; Wed, 18 Dec 2019 19:30:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 157FE206D7 for ; Wed, 18 Dec 2019 19:30:00 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="O7H4xgmL" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727606AbfLRT37 (ORCPT ); Wed, 18 Dec 2019 14:29:59 -0500 Received: from mail-wm1-f68.google.com ([209.85.128.68]:51063 "EHLO mail-wm1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727587AbfLRT34 (ORCPT ); Wed, 18 Dec 2019 14:29:56 -0500 Received: by mail-wm1-f68.google.com with SMTP id a5so3059573wmb.0 for ; Wed, 18 Dec 2019 11:29:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=OiVOjfhOoMJ2pyMlGtWTDqDyjMxgdcdniVdqMQnmLYQ=; b=O7H4xgmLgHMSEyKQnVxRB3O9lqTAmeG5aZhqLi5Yt9NupIWFHazpil0mb2BMMe5FSV UD6LX3f3vliPfliUDVEoZTDXTdxwvB64kegXE+joDZ4dyX5baG5FJmLsRNCtB3l/N4jW jq4uzmZW/Fb67PlUebo0sXoih3tQGwLS/JCXz3liQa5xWT98vtucoMQzXbMchs23jxzM uZz8bcwscp+m3nqgbLgdMJHnmO7zLcb3xZixMEfTQr+T9DruP5sVUj5spUVjlT3U3Fhz OD05DjWqDNUElXquF+5HBCdl57eOcdWJoQsVfe7iS5qBv01XHsyNWAI6TNDlGJiZme3D +XCQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=OiVOjfhOoMJ2pyMlGtWTDqDyjMxgdcdniVdqMQnmLYQ=; b=ioX9ymAr33Z+XjE6p52QGufosWHOxhyJmcClju2A0YhUNZ/hxfHD57kBsR7veO5KgZ ZnzyAeWLn/n3Hfb4n8+oSa0dTpJJRcJLdK9XACfQ7x2HNX7Azs+v4K3s/vEiwvVog4zk euTzWtHj0P6KzsncjhpmqDfmPYetx1QPRu9wyKnjY4zYlWkH5bkAyDhpHbAiLl/8h75x L2G+mLdMEKQJJ+SYjY+tfJMRYlLXcyZ/6xVkb6bV7i92SWhUmHVJQTVVbQ3jlmWIBlh/ abhOez3Fls2RXyGHgIOeHgd6t/Us+eEQjZxiyYfUI81PElfwriIBx3jjTeezxH/ds2dE 41TQ== X-Gm-Message-State: APjAAAXJCvpeHjpI4ZKl0xlgTu7dOZC1myx4WoNZZ6rn7Lqc/Git13ub HCo6OE/uvUNn8G0Xjyi/JPIM7XQG X-Google-Smtp-Source: APXvYqzEuA4PVheBRjO5qeCxZcGZ9FaLM5Io2I4CVHP7/2sU/UGgXnr0s/jgQhvqE1pOH3t/jU47og== X-Received: by 2002:a05:600c:409:: with SMTP id q9mr5111679wmb.19.1576697394615; Wed, 18 Dec 2019 11:29:54 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id i5sm3591690wml.31.2019.12.18.11.29.54 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 18 Dec 2019 11:29:54 -0800 (PST) Message-Id: <31079dc1cf10d031010846f1d1a0b30a05341943.1576697387.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Elijah Newren via GitGitGadget" Date: Wed, 18 Dec 2019 19:29:46 +0000 Subject: [PATCH v4 8/8] dir: consolidate similar code in treat_directory() Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: blees@dcon.de, gitster@pobox.com, kyle@kyleam.com, sxlijin@gmail.com, Junio C Hamano , Elijah Newren Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Elijah Newren Both the DIR_SKIP_NESTED_GIT and DIR_NO_GITLINKS cases were checking for whether a path was actually a nonbare repository. That code could be shared, with just the result of how to act differing between the two cases. Signed-off-by: Elijah Newren --- dir.c | 18 +++++++----------- 1 file changed, 7 insertions(+), 11 deletions(-) diff --git a/dir.c b/dir.c index f500fd9279..6912f7eb5e 100644 --- a/dir.c +++ b/dir.c @@ -1461,6 +1461,8 @@ static enum path_treatment treat_directory(struct dir_struct *dir, const char *dirname, int len, int baselen, int exclude, const struct pathspec *pathspec) { + int nested_repo = 0; + /* The "len-1" is to strip the final '/' */ switch (directory_exists_in_index(istate, dirname, len-1)) { case index_directory: @@ -1470,15 +1472,16 @@ static enum path_treatment treat_directory(struct dir_struct *dir, return path_none; case index_nonexistent: - if (dir->flags & DIR_SKIP_NESTED_GIT) { - int nested_repo; + if ((dir->flags & DIR_SKIP_NESTED_GIT) || + !(dir->flags & DIR_NO_GITLINKS)) { struct strbuf sb = STRBUF_INIT; strbuf_addstr(&sb, dirname); nested_repo = is_nonbare_repository_dir(&sb); strbuf_release(&sb); - if (nested_repo) - return path_none; } + if (nested_repo) + return ((dir->flags & DIR_SKIP_NESTED_GIT) ? path_none : + (exclude ? path_excluded : path_untracked)); if (dir->flags & DIR_SHOW_OTHER_DIRECTORIES) break; @@ -1506,13 +1509,6 @@ static enum path_treatment treat_directory(struct dir_struct *dir, return path_none; } - if (!(dir->flags & DIR_NO_GITLINKS)) { - struct strbuf sb = STRBUF_INIT; - strbuf_addstr(&sb, dirname); - if (is_nonbare_repository_dir(&sb)) - return exclude ? path_excluded : path_untracked; - strbuf_release(&sb); - } return path_recurse; }