From patchwork Mon Dec 9 20:47:38 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Arver via GitGitGadget X-Patchwork-Id: 11280417 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1E51F930 for ; Mon, 9 Dec 2019 20:47:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id E7D782071E for ; Mon, 9 Dec 2019 20:47:58 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="HfZ//fWc" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726917AbfLIUrw (ORCPT ); Mon, 9 Dec 2019 15:47:52 -0500 Received: from mail-wr1-f67.google.com ([209.85.221.67]:33644 "EHLO mail-wr1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726495AbfLIUru (ORCPT ); Mon, 9 Dec 2019 15:47:50 -0500 Received: by mail-wr1-f67.google.com with SMTP id b6so17730436wrq.0 for ; Mon, 09 Dec 2019 12:47:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=SmkT8r7D+uE7MPiEHrIRsFhOlsDnFfVFWAuCwELHwTc=; b=HfZ//fWc1vYj1DMXlYWCX1zd0N3yRi/mVl5g9ZH4O7yZ7C1MQ9Z+BYYCLYYIwkb37b m/ExYI8utFqp8bHP+AfEScZY/TqmtsDu1LDYaX1L9WvJCtjbVAPAz/KzBP1QW7tPjaJV uIOwtbOZXP6pXJz0ztHI1TTmiavaVxGHABd9fy1usNW75hb+JYp0gai8bMfmSpSNNk4O LX/4YVtxL5iWdpoXhW0759hMZ+6SwHcya6JkNJLAXV7ZcW9ET2DhvcIdkpGyLH3fqgaR 9Z/MSFmVU66AYKg1db3W8CLnGan66UDFNTzxvL8KFDY3HNzc4Sav4CcNSPbExw43H6ov AbrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=SmkT8r7D+uE7MPiEHrIRsFhOlsDnFfVFWAuCwELHwTc=; b=pIyqSirjJed11afcC05TGyi03bkM4K5tdSL1zlFKoaaIen0ipSfTdg/5G7eJgSoLNi /Smpst5vyFL8gagmNOcE5mo32kUobITnFe13ys8AOz4qa3yD9P6yFc3YFnqqg9xY4C1j jk6h4sJQK/QUGyD+8CeAEh39CyNyns+imZEwztS1FWoyN2/uUrvAUiQ775N6Bwr/Gm97 65scDQ4g1EWDmBoaJyyqKzwNC+c35LL6Lg3XWVXF5S9F+LMbLPJ7T6I4iXVhiRpMnrKd w6XNqydjMsXTvjeNnqyRMG0//xxSx8Gguo12Fclw9Gdq45hMxnRAULWIJ0CcuTGvmYSd G1kA== X-Gm-Message-State: APjAAAWk2pIFUMHxXkAQgP8oNYICXP4GLhnw9u7tipBGQ6sL1NRCQxN4 8UKR5yNNS710AZpfdLjGE+iCClCG X-Google-Smtp-Source: APXvYqxIUjr+h+yAbATOcrSkZ9U8H3ghGGU8AmOy3rWzgWLVCfAZCP3NaZ31qElAeCC87652BiYR3A== X-Received: by 2002:adf:fe43:: with SMTP id m3mr4246590wrs.213.1575924467737; Mon, 09 Dec 2019 12:47:47 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id q3sm742319wrn.33.2019.12.09.12.47.47 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 09 Dec 2019 12:47:47 -0800 (PST) Message-Id: <4b24ba9966d41558cc2382d81e6e76b4375b555c.1575924465.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Elijah Newren via GitGitGadget" Date: Mon, 09 Dec 2019 20:47:38 +0000 Subject: [PATCH 1/8] t3011: demonstrate directory traversal failures Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Elijah Newren Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Elijah Newren Add several tests demonstrating directory traversal failures of various sorts in dir.c (and one similar looking test that turns out to be a git_fnmatch bug). A lot of these tests look like near duplicates of each other, but an optimization path in dir.c to pre-descend into a common prefix and the specialized treatment of trailing slashes in dir.c mean the tiny differences are sometimes important and potentially cause different codepaths to be explored. Of the 7 failing tests, 2 are new to git-2.24.0 (tweaked by side effects of the en/clean-nested-with-ignored-topic); the other 5 also failed under git-2.23.0 and earlier. Signed-off-by: Elijah Newren --- ...common-prefixes-and-directory-traversal.sh | 193 ++++++++++++++++++ 1 file changed, 193 insertions(+) create mode 100755 t/t3011-common-prefixes-and-directory-traversal.sh diff --git a/t/t3011-common-prefixes-and-directory-traversal.sh b/t/t3011-common-prefixes-and-directory-traversal.sh new file mode 100755 index 0000000000..773d6038d1 --- /dev/null +++ b/t/t3011-common-prefixes-and-directory-traversal.sh @@ -0,0 +1,193 @@ +#!/bin/sh + +test_description='directory traversal handling, especially with common prefixes' + +. ./test-lib.sh + +test_expect_success 'setup' ' + test_commit hello && + + >empty && + mkdir untracked_dir && + >untracked_dir/empty && + git init untracked_repo && + >untracked_repo/empty && + + echo ignored >.gitignore && + echo an_ignored_dir/ >>.gitignore && + mkdir an_ignored_dir && + mkdir an_untracked_dir && + >an_ignored_dir/ignored && + >an_ignored_dir/untracked && + >an_untracked_dir/ignored && + >an_untracked_dir/untracked +' + +test_expect_success 'git ls-files -o shows the right entries' ' + cat <<-EOF >expect && + .gitignore + actual + an_ignored_dir/ignored + an_ignored_dir/untracked + an_untracked_dir/ignored + an_untracked_dir/untracked + empty + expect + untracked_dir/empty + untracked_repo/ + EOF + git ls-files -o >actual && + test_cmp expect actual +' + +test_expect_success 'git ls-files -o --exclude-standard shows the right entries' ' + cat <<-EOF >expect && + .gitignore + actual + an_untracked_dir/untracked + empty + expect + untracked_dir/empty + untracked_repo/ + EOF + git ls-files -o --exclude-standard >actual && + test_cmp expect actual +' + +test_expect_success 'git ls-files -o untracked_dir recurses' ' + echo untracked_dir/empty >expect && + git ls-files -o untracked_dir >actual && + test_cmp expect actual +' + +test_expect_success 'git ls-files -o untracked_dir/ recurses' ' + echo untracked_dir/empty >expect && + git ls-files -o untracked_dir/ >actual && + test_cmp expect actual +' + +test_expect_success 'git ls-files -o --directory untracked_dir does not recurse' ' + echo untracked_dir/ >expect && + git ls-files -o --directory untracked_dir >actual && + test_cmp expect actual +' + +test_expect_failure 'git ls-files -o --directory untracked_dir/ does not recurse' ' + echo untracked_dir/ >expect && + git ls-files -o --directory untracked_dir/ >actual && + test_cmp expect actual +' + +test_expect_success 'git ls-files -o untracked_repo does not recurse' ' + echo untracked_repo/ >expect && + git ls-files -o untracked_repo >actual && + test_cmp expect actual +' + +test_expect_failure 'git ls-files -o untracked_repo/ does not recurse' ' + echo untracked_repo/ >expect && + git ls-files -o untracked_repo/ >actual && + test_cmp expect actual +' + +test_expect_failure 'git ls-files -o untracked_dir untracked_repo recurses into untracked_dir only' ' + echo untracked_dir/empty >expect && + echo untracked_repo/ >>expect && + git ls-files -o untracked_dir untracked_repo >actual && + test_cmp expect actual +' + +test_expect_success 'git ls-files -o untracked_dir/ untracked_repo/ recurses into untracked_dir only' ' + echo untracked_dir/empty >expect && + echo untracked_repo/ >>expect && + git ls-files -o untracked_dir/ untracked_repo/ >actual && + test_cmp expect actual +' + +test_expect_failure 'git ls-files -o --directory untracked_dir untracked_repo does not recurse' ' + echo untracked_dir/ >expect && + echo untracked_repo/ >>expect && + git ls-files -o --directory untracked_dir untracked_repo >actual && + test_cmp expect actual +' + +test_expect_success 'git ls-files -o --directory untracked_dir/ untracked_repo/ does not recurse' ' + echo untracked_dir/ >expect && + echo untracked_repo/ >>expect && + git ls-files -o --directory untracked_dir/ untracked_repo/ >actual && + test_cmp expect actual +' + +test_expect_success 'git ls-files -o .git shows nothing' ' + >expect && + git ls-files -o .git >actual && + test_cmp expect actual +' + +test_expect_failure 'git ls-files -o .git/ shows nothing' ' + >expect && + git ls-files -o .git/ >actual && + test_cmp expect actual +' + +test_expect_success FUNNYNAMES 'git ls-files -o untracked_* recurses appropriately' ' + mkdir "untracked_*" && + >"untracked_*/empty" && + + echo "untracked_*/empty" >expect && + echo untracked_dir/empty >>expect && + echo untracked_repo/ >>expect && + git ls-files -o "untracked_*" >actual && + test_cmp expect actual +' + +# It turns out fill_directory returns the right paths, but ls-files' post-call +# filtering in show_dir_entry() via calling dir_path_match() which ends up +# in git_fnmatch() has logic for PATHSPEC_ONESTAR that assumes the pathspec +# must match the full path; it doesn't check it for matching a leading +# directory. +test_expect_failure FUNNYNAMES 'git ls-files -o untracked_*/ recurses appropriately' ' + echo "untracked_*/empty" >expect && + echo untracked_dir/empty >>expect && + echo untracked_repo/ >>expect && + git ls-files -o "untracked_*/" >actual && + test_cmp expect actual +' + +test_expect_success FUNNYNAMES 'git ls-files -o --directory untracked_* does not recurse' ' + echo "untracked_*/" >expect && + echo untracked_dir/ >>expect && + echo untracked_repo/ >>expect && + git ls-files -o --directory "untracked_*" >actual && + test_cmp expect actual +' + +test_expect_success FUNNYNAMES 'git ls-files -o --directory untracked_*/ does not recurse' ' + echo "untracked_*/" >expect && + echo untracked_dir/ >>expect && + echo untracked_repo/ >>expect && + git ls-files -o --directory "untracked_*/" >actual && + test_cmp expect actual +' + +test_expect_success 'git ls-files -o consistent between one or two dirs' ' + git ls-files -o --exclude-standard an_ignored_dir/ an_untracked_dir/ >tmp && + ! grep ^an_ignored_dir/ tmp >expect && + git ls-files -o --exclude-standard an_ignored_dir/ >actual && + test_cmp expect actual +' + +# ls-files doesn't have a way to request showing both untracked and ignored +# files at the same time, so use `git status --ignored` +test_expect_failure 'git status --ignored shows same files under dir with or without pathspec' ' + cat <<-EOF >expect && + ?? an_untracked_dir/ + !! an_untracked_dir/ignored + EOF + git status --porcelain --ignored >output && + grep an_untracked_dir output >expect && + git status --porcelain --ignored an_untracked_dir/ >actual && + test_cmp expect actual +' + +test_done From patchwork Mon Dec 9 20:47:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Arver via GitGitGadget X-Patchwork-Id: 11280403 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4259A1593 for ; Mon, 9 Dec 2019 20:47:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 212AD206E0 for ; Mon, 9 Dec 2019 20:47:51 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="h8YCWngw" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726818AbfLIUru (ORCPT ); Mon, 9 Dec 2019 15:47:50 -0500 Received: from mail-wm1-f66.google.com ([209.85.128.66]:36270 "EHLO mail-wm1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726483AbfLIUrt (ORCPT ); Mon, 9 Dec 2019 15:47:49 -0500 Received: by mail-wm1-f66.google.com with SMTP id p17so780462wma.1 for ; Mon, 09 Dec 2019 12:47:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=sv8Z1Qvh2PY479+74oo+SraQ5J/zsk70nUicKx+77sg=; b=h8YCWngwTRdLDFi72TIXSlpQFFEvz/eelTHpEmQCGmUqeBT1CdT8sprcxfAH/kICjE Iu/36PLdx2XfTOup328tVis+sO+mzGT4KTRanx3ynl1oWsag1ikV1xEmWwFhVJXbiB3q KEH6nXyO6vpn3casBE1zLb40AyyiBms8WEZAITR7L8sCLgtN6uvzw13+b58QFp1Tgy2N sy8ySj0t8C4QJ8+t1AYefXjnCnqFdqTJ7/o42PDg4Tccu4ylZzjnd6QAnFtS3DsgbRbA vsmXOTVB+uRApKSTh57TqYzrIej80FzD1wVnNkZ0jGJA1O1jhsQsD5L8KgbJuf7MByjc 1hZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=sv8Z1Qvh2PY479+74oo+SraQ5J/zsk70nUicKx+77sg=; b=ilQLfwueT76/g58pYpcJyGNc8n2wJ7ALudjsQYSrEhdyAOw4FADpMCu2h38D6a84eg Nt9f2dFAOXN2JlQDiZLcK9tEiJsm3V9qKHwu9H4yuAHduSLjGZ2mdHDgVeAYfTbM9ELw SsnxstskwljwHQRu/3RqEcHg7sNXe4ERbT+HCslioodqSRiS7ANtjOkiepbCjSUSZpE9 +SLrpyap8BpqIPDZw6E5MLUdRqlDBVZNZSrT4GGtle0Jxp8qGl6rH21nvqabQtr9tGrX NNTXOPZfU6fkAuqkHVfL8BmaeqJAdVXfB5BnlcHd3meQ8JOAkKE6g5d8hZSlwbHXmMln JMtw== X-Gm-Message-State: APjAAAXAIZD1EQFLuTxxn+k/F2JFsYmGf660YqT9L8SRD5OfxYyMRUIZ 0hPCQPcteZy8/tmDULm36e8ps5h7 X-Google-Smtp-Source: APXvYqwa2DfQHnHYP0DKdu2BNeqMPkJWVmqx/4FfZNZWVm3GoLbaSoXDQkX52ufP6e1RVtjepo+USg== X-Received: by 2002:a7b:c386:: with SMTP id s6mr949578wmj.105.1575924468506; Mon, 09 Dec 2019 12:47:48 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id v14sm704793wrm.28.2019.12.09.12.47.47 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 09 Dec 2019 12:47:48 -0800 (PST) Message-Id: In-Reply-To: References: From: "Elijah Newren via GitGitGadget" Date: Mon, 09 Dec 2019 20:47:39 +0000 Subject: [PATCH 2/8] Revert "dir.c: make 'git-status --ignored' work within leading directories" Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Elijah Newren Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Elijah Newren Commit be8a84c52669 ("dir.c: make 'git-status --ignored' work within leading directories", 2013-04-15) noted that git status --ignored would not list ignored files and directories within if was untracked, and modified the behavior to make it show them. However, it did so via a hack that broke consistency; it would show paths under differently than a simple git status --ignored | grep would show them. A correct fix is slightly more involved, and complicated slightly by this hack, so we revert this commit (but keep corrected versions of the testcases) and will later fix the original bug with a subsequent patch. Some history may be helpful: A very, very similar case to the commit we are reverting was raised in commit 48ffef966c76 ("ls-files: fix overeager pathspec optimization", 2010-01-08); but it actually went in somewhat the opposite direction. In that commit, it mentioned how git ls-files -o --exclude-standard t/ used to show untracked files under t/ even when t/ was ignored, and then changed the behavior to stop showing untracked files under an ignored directory. More importantly, this commit considered keeping this behavior but noted that it would be inconsistent with the behavior when multiple pathspecs were specified and thus rejected it. The reason for this whole inconsistency when one pathspec is specified versus zero or two is because common prefixes of pathspecs are sent through a different set of checks (in treat_leading_path()) than normal file/directory traversal (those go through read_directory_recursive() and treat_path()). As such, for consistency, one needs to check that both codepaths produce the same result. Revert commit be8a84c526691667fc04a8241d93a3de1de298ab, except instead of removing the testcase it added, modify it to check for correct and consistent behavior. A subsequent patch in this series will fix the testcase. Signed-off-by: Elijah Newren --- dir.c | 3 --- t/t7061-wtstatus-ignore.sh | 7 +++++-- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/dir.c b/dir.c index 61f559f980..0dd5266629 100644 --- a/dir.c +++ b/dir.c @@ -2083,14 +2083,12 @@ static int treat_leading_path(struct dir_struct *dir, struct strbuf sb = STRBUF_INIT; int baselen, rc = 0; const char *cp; - int old_flags = dir->flags; while (len && path[len - 1] == '/') len--; if (!len) return 1; baselen = 0; - dir->flags &= ~DIR_SHOW_OTHER_DIRECTORIES; while (1) { cp = path + baselen + !!baselen; cp = memchr(cp, '/', path + len - cp); @@ -2113,7 +2111,6 @@ static int treat_leading_path(struct dir_struct *dir, } } strbuf_release(&sb); - dir->flags = old_flags; return rc; } diff --git a/t/t7061-wtstatus-ignore.sh b/t/t7061-wtstatus-ignore.sh index 0c394cf995..ded7f97181 100755 --- a/t/t7061-wtstatus-ignore.sh +++ b/t/t7061-wtstatus-ignore.sh @@ -43,11 +43,14 @@ test_expect_success 'status untracked directory with --ignored -u' ' test_cmp expected actual ' cat >expected <<\EOF -?? untracked/uncommitted +?? untracked/ !! untracked/ignored EOF -test_expect_success 'status prefixed untracked directory with --ignored' ' +test_expect_failure 'status of untracked directory with --ignored works with or without prefix' ' + git status --porcelain --ignored | grep untracked/ >actual && + test_cmp expected actual && + git status --porcelain --ignored untracked/ >actual && test_cmp expected actual ' From patchwork Mon Dec 9 20:47:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Arver via GitGitGadget X-Patchwork-Id: 11280405 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7B0FE930 for ; Mon, 9 Dec 2019 20:47:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 560DD206E0 for ; Mon, 9 Dec 2019 20:47:53 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="RpxYq8tD" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726915AbfLIUrw (ORCPT ); Mon, 9 Dec 2019 15:47:52 -0500 Received: from mail-wm1-f65.google.com ([209.85.128.65]:56017 "EHLO mail-wm1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726598AbfLIUru (ORCPT ); Mon, 9 Dec 2019 15:47:50 -0500 Received: by mail-wm1-f65.google.com with SMTP id q9so801695wmj.5 for ; Mon, 09 Dec 2019 12:47:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=H1rfc3Xakcj/y7/qH6kpuw834RYNToBjdWGDvsRGWdg=; b=RpxYq8tDR3SDanmflqYAWCw6j1NaRw0Cb4rTRJg9IH1BH+8Oc2wY60JbpyN4G4kS90 ljyEVhQwmzPUAD8UFktu63nW2nU2Un/vPMx3LggwWFeh9XTWeELBdgOrBMbVBLDWsifL 2J5tckTjgq9t4drKk+OIEsipW5daeHoPqeP5fDuPHoSshCs8tH7ovVdL/t3xezONmUni KsavwZie8mxBLFDSVehcUshElSHs7AAWA+2NRTWsCD1imqbq37j4COCswEWv8whg17XR qAQap5VSOoUTtsk+t7Ri7bhxY5KShkpVA1FAMjFEdgbqMZAQ/ykMPe4AknVNW0xM1+J9 J8tw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=H1rfc3Xakcj/y7/qH6kpuw834RYNToBjdWGDvsRGWdg=; b=nzhxrDAhRknffGEBKNd99UeXIx21QCnK0b59ALP//1LQ4v4vR97Uab5m0GOiJcBZCF l4+t3Wa8nSK9HPW9YQafSxF98UKJ0SjKm0JN5BBkfgBXuWri3ho73bm5welkH/+5rAEd fP4PcIwb8S2KoLuzNUJRa53SE3qThwm39X8vq+lfyAKSXwhrIbsXLqmhkET/yRoliyN4 7eX0Y32+xjHdMOKaVuvhCen9D0WphfeV9j8OZMtsiYsybuNTlk0iwfgKKe0bbaOm9ZJR 2oRR/IDoVC11PkliDCBV3pVGZeCcTcDm1qfPzwhirclEB82msppnW9oZH40jDrit0zK+ S9Dg== X-Gm-Message-State: APjAAAXoGgPAjuaDfJ8v/9VwsQhfy3ZYqLFU/envNC5KZ8MuKCWTM+Kq g+bv80DCgLgI7vYz3k4pS6hjhL5a X-Google-Smtp-Source: APXvYqxKdSoJpKvNv3HhfNlKnxevKaIeOK6NOfXtIa5WTHWqDywPg4jJv/KHz9DirFRX0C8NPk+SdQ== X-Received: by 2002:a1c:730e:: with SMTP id d14mr967739wmb.88.1575924469141; Mon, 09 Dec 2019 12:47:49 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id v17sm667523wrt.91.2019.12.09.12.47.48 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 09 Dec 2019 12:47:48 -0800 (PST) Message-Id: In-Reply-To: References: From: "Elijah Newren via GitGitGadget" Date: Mon, 09 Dec 2019 20:47:40 +0000 Subject: [PATCH 3/8] dir: remove stray quote character in comment Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Elijah Newren Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Elijah Newren Signed-off-by: Elijah Newren --- dir.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/dir.c b/dir.c index 0dd5266629..5dacacd469 100644 --- a/dir.c +++ b/dir.c @@ -373,7 +373,7 @@ static int match_pathspec_item(const struct index_state *istate, !ps_strncmp(item, match, name, namelen)) return MATCHED_RECURSIVELY_LEADING_PATHSPEC; - /* name" doesn't match up to the first wild character */ + /* name doesn't match up to the first wild character */ if (item->nowildcard_len < item->len && ps_strncmp(item, match, name, item->nowildcard_len - prefix)) From patchwork Mon Dec 9 20:47:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Arver via GitGitGadget X-Patchwork-Id: 11280415 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 99A541593 for ; Mon, 9 Dec 2019 20:47:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 7A9C52071E for ; Mon, 9 Dec 2019 20:47:58 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="FexKu1kh" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726926AbfLIUrx (ORCPT ); Mon, 9 Dec 2019 15:47:53 -0500 Received: from mail-wm1-f66.google.com ([209.85.128.66]:56019 "EHLO mail-wm1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726483AbfLIUrv (ORCPT ); Mon, 9 Dec 2019 15:47:51 -0500 Received: by mail-wm1-f66.google.com with SMTP id q9so801720wmj.5 for ; Mon, 09 Dec 2019 12:47:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=IM8NXGxG48ELrMvEtU27DdAjZm+S48DBs3jTcX5uhok=; b=FexKu1kh5EOWKCtFetoafPI9P3zmdigVvkBPykfF0x5bv8DKkJYnZX9psE2PLZ2yv+ +7f2Zt7Not7N6W2NCVYRPAx1ZFm6+IftIHNihkFAIuMV4v1jZ9lfPIy0JqbDwXEDNKF/ p8V0K/NR4E3gJOWdYtYSYeXjmAkZYHLrOSMl/m3kip37lQ+RquHD98uWybHD/nqz22Qj sILo8EARj4Sp5h7rNW1LApIGo1hHMHb8/3nZnNAnDG1ut1Vs4fe4NuJi9Fh/XnqegXTS AVrdeZ4If5b3opevNY5yIkuRfdX+OrqET/KQg+pncIoZ8k3i47dWUYK5XgEI0MYiNDIO HPNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=IM8NXGxG48ELrMvEtU27DdAjZm+S48DBs3jTcX5uhok=; b=m9xjz3iFUs2Jud+idMzD1HUvI8eHGIpK3izJO9B1uInLYtrNAUq1JgQdcKFJSXKElQ Uf1LgcC+JdusgrT1bxrfTWjmumPUOtKzfRs6Hh3Sxf+yl4ZlwY7ZlJrM8yDAE593WNag JLXepVQ+Aggx2dLARHnzE48kqBdzr/kJT3YW21L1euPlgMvsk2qP5rBlEzgNIetkzAds ODi9In54PYokx5sHhlnIClWgKdUQeRbwhu08IAt5W5VR60W3khI82THKkvwpL5lpRlCd lQ/LoUneRvtTieClCyRfPsZNREdY0FfBnCgUBTFhyUBMVy7CdCJnq78c9BzjKe5CLxyz U2kw== X-Gm-Message-State: APjAAAUimGh54uTyp7e+MLsaftrDHwvGBcN/kzpnRxxbiC+5CBrHFj25 Cii27UzXdUHXMwZVtR3TNp7rrFsg X-Google-Smtp-Source: APXvYqwJsAnjDksU8IE5hXpyu7PgwI1kSIvUyrYxxrSkuze69GNPQh/gABS03zLRVYkcuD+4wjfTXw== X-Received: by 2002:a05:600c:224a:: with SMTP id a10mr966609wmm.143.1575924469810; Mon, 09 Dec 2019 12:47:49 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id s10sm745901wrw.12.2019.12.09.12.47.49 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 09 Dec 2019 12:47:49 -0800 (PST) Message-Id: In-Reply-To: References: From: "Elijah Newren via GitGitGadget" Date: Mon, 09 Dec 2019 20:47:41 +0000 Subject: [PATCH 4/8] dir: exit before wildcard fall-through if there is no wildcard Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Elijah Newren Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Elijah Newren The DO_MATCH_LEADING_PATHSPEC had a fall-through case for if there was a wildcard, noting that we don't yet have enough information to determine if a further paths under the current directory might match due to the presence of wildcards. But if we have no wildcards in our pathspec, then we shouldn't get to that fall-through case. Signed-off-by: Elijah Newren --- dir.c | 7 +++++++ t/t3011-common-prefixes-and-directory-traversal.sh | 4 ++-- 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/dir.c b/dir.c index 5dacacd469..517a569e10 100644 --- a/dir.c +++ b/dir.c @@ -379,6 +379,13 @@ static int match_pathspec_item(const struct index_state *istate, item->nowildcard_len - prefix)) return 0; + /* + * name has no wildcard, and it didn't match as a leading + * pathspec so return. + */ + if (item->nowildcard_len == item->len) + return 0; + /* * Here is where we would perform a wildmatch to check if * "name" can be matched as a directory (or a prefix) against diff --git a/t/t3011-common-prefixes-and-directory-traversal.sh b/t/t3011-common-prefixes-and-directory-traversal.sh index 773d6038d1..d4c06fcd76 100755 --- a/t/t3011-common-prefixes-and-directory-traversal.sh +++ b/t/t3011-common-prefixes-and-directory-traversal.sh @@ -90,7 +90,7 @@ test_expect_failure 'git ls-files -o untracked_repo/ does not recurse' ' test_cmp expect actual ' -test_expect_failure 'git ls-files -o untracked_dir untracked_repo recurses into untracked_dir only' ' +test_expect_success 'git ls-files -o untracked_dir untracked_repo recurses into untracked_dir only' ' echo untracked_dir/empty >expect && echo untracked_repo/ >>expect && git ls-files -o untracked_dir untracked_repo >actual && @@ -104,7 +104,7 @@ test_expect_success 'git ls-files -o untracked_dir/ untracked_repo/ recurses int test_cmp expect actual ' -test_expect_failure 'git ls-files -o --directory untracked_dir untracked_repo does not recurse' ' +test_expect_success 'git ls-files -o --directory untracked_dir untracked_repo does not recurse' ' echo untracked_dir/ >expect && echo untracked_repo/ >>expect && git ls-files -o --directory untracked_dir untracked_repo >actual && From patchwork Mon Dec 9 20:47:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Arver via GitGitGadget X-Patchwork-Id: 11280407 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E15DF138C for ; Mon, 9 Dec 2019 20:47:54 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id BD4772071E for ; Mon, 9 Dec 2019 20:47:54 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="nN5qTR43" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726932AbfLIUry (ORCPT ); Mon, 9 Dec 2019 15:47:54 -0500 Received: from mail-wr1-f66.google.com ([209.85.221.66]:43256 "EHLO mail-wr1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726874AbfLIUrx (ORCPT ); Mon, 9 Dec 2019 15:47:53 -0500 Received: by mail-wr1-f66.google.com with SMTP id d16so17664294wre.10 for ; Mon, 09 Dec 2019 12:47:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=+CgvyKrM09apU1qkOsJft1P4o4wqjmDRS+ywvTmG7xU=; b=nN5qTR43O7A7ycMNH5oaxA34tkT+OMiWWawKvzV2eesD+QSJtR4D6R28iA+Og6rB/v MrdJbsN7kT+Fn1IlGb/NA7YtIXLF96IwX4qpEd+/XcLnZpELJtcxWjKJK8swqhjatTZr DMcaHZjuKXsa9vhLjrjWsqcREOIS3ZkkrcehQhmDtoFrtpTCE2L4djBAelv9sNXxuijA xn59cXPPlWcSrHopj6uw+0eCVpKnAq2BsUuoKAAnNAOqhYXWe/XoiKuU+ycz6NqdsD0y ImkzzRQrxiBf9vGM/k/UPT1mSlfNKzhQJ1VEvSRmYR2kB22FT65Aaaqs8vaOfKho9jc6 d7WA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=+CgvyKrM09apU1qkOsJft1P4o4wqjmDRS+ywvTmG7xU=; b=DzyAQIIWHQEu47kH+4PW45RP/rYrcvQHoaeuYKlwsN7ahDZ9daqVd50TM74wFWHl06 Z5fwj0yTV0HF0gCZpqFrgt3NGZwBI5YIi6lQ9ohqof3EjJZOaczhm+PM67ghKwYufWpN NMytDnll8+S01uiMQzazr1F8OkbAj+mtf3/kbLsqXDKHllEWPIUuMM1zXTUA1x6rO6vB nISIKhJjpikUGlajTbEFEn1Uy9Z80ShfeKDpJAVwQBTTbiwIojGAwAabb/j7QIfRZD2S I4EtrcvMbdT2LPmuM1lSIfCTzvqfH5M965BgYYSt0STedO1BDWOSquBlqgJcUxBv/Yly +viA== X-Gm-Message-State: APjAAAUGp6nYJGwh0buxWZcvMyBIsuw9MFYv312+TzRZp1dpeNa9xXdf KChuuq0E+BPAZV+G70ADWGKSxpDl X-Google-Smtp-Source: APXvYqxToTJ2VS/lekdhYsFU1qmA1Lr7UJvj41s/16M6BIUevit0K7MA7whQEh2b0KEP1sSWqB2eYw== X-Received: by 2002:a5d:49c3:: with SMTP id t3mr4206882wrs.113.1575924470458; Mon, 09 Dec 2019 12:47:50 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id s65sm566690wmf.48.2019.12.09.12.47.49 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 09 Dec 2019 12:47:50 -0800 (PST) Message-Id: <738d9ae4c9ca0e54502b9ef606e2fc8dba9a94e1.1575924465.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Elijah Newren via GitGitGadget" Date: Mon, 09 Dec 2019 20:47:42 +0000 Subject: [PATCH 5/8] dir: break part of read_directory_recursive() out for reuse Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Elijah Newren Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Elijah Newren Create an add_path_to_appropriate_result_list() function from the code at the end of read_directory_recursive() so we can use it elsewhere. Signed-off-by: Elijah Newren --- dir.c | 60 ++++++++++++++++++++++++++++++++++++----------------------- 1 file changed, 37 insertions(+), 23 deletions(-) diff --git a/dir.c b/dir.c index 517a569e10..645b44ea64 100644 --- a/dir.c +++ b/dir.c @@ -1932,6 +1932,40 @@ static void close_cached_dir(struct cached_dir *cdir) } } +static void add_path_to_appropriate_result_list(struct dir_struct *dir, + struct untracked_cache_dir *untracked, + struct cached_dir *cdir, + struct index_state *istate, + struct strbuf *path, + int baselen, + const struct pathspec *pathspec, + enum path_treatment state) +{ + /* add the path to the appropriate result list */ + switch (state) { + case path_excluded: + if (dir->flags & DIR_SHOW_IGNORED) + dir_add_name(dir, istate, path->buf, path->len); + else if ((dir->flags & DIR_SHOW_IGNORED_TOO) || + ((dir->flags & DIR_COLLECT_IGNORED) && + exclude_matches_pathspec(path->buf, path->len, + pathspec))) + dir_add_ignored(dir, istate, path->buf, path->len); + break; + + case path_untracked: + if (dir->flags & DIR_SHOW_IGNORED) + break; + dir_add_name(dir, istate, path->buf, path->len); + if (cdir->fdir) + add_untracked(untracked, path->buf + baselen); + break; + + default: + break; + } +} + /* * Read a directory tree. We currently ignore anything but * directories, regular files and symlinks. That's because git @@ -2035,29 +2069,9 @@ static enum path_treatment read_directory_recursive(struct dir_struct *dir, continue; } - /* add the path to the appropriate result list */ - switch (state) { - case path_excluded: - if (dir->flags & DIR_SHOW_IGNORED) - dir_add_name(dir, istate, path.buf, path.len); - else if ((dir->flags & DIR_SHOW_IGNORED_TOO) || - ((dir->flags & DIR_COLLECT_IGNORED) && - exclude_matches_pathspec(path.buf, path.len, - pathspec))) - dir_add_ignored(dir, istate, path.buf, path.len); - break; - - case path_untracked: - if (dir->flags & DIR_SHOW_IGNORED) - break; - dir_add_name(dir, istate, path.buf, path.len); - if (cdir.fdir) - add_untracked(untracked, path.buf + baselen); - break; - - default: - break; - } + add_path_to_appropriate_result_list(dir, untracked, &cdir, + istate, &path, baselen, + pathspec, state); } close_cached_dir(&cdir); out: From patchwork Mon Dec 9 20:47:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Arver via GitGitGadget X-Patchwork-Id: 11280413 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 19129930 for ; Mon, 9 Dec 2019 20:47:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id E1D522071E for ; Mon, 9 Dec 2019 20:47:57 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="d3FXTArx" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726951AbfLIUr4 (ORCPT ); Mon, 9 Dec 2019 15:47:56 -0500 Received: from mail-wm1-f65.google.com ([209.85.128.65]:38214 "EHLO mail-wm1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726916AbfLIUrx (ORCPT ); Mon, 9 Dec 2019 15:47:53 -0500 Received: by mail-wm1-f65.google.com with SMTP id p17so761917wmi.3 for ; Mon, 09 Dec 2019 12:47:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=NSbAkiX42wwwydPZEFESNxKxxvsB/IftHv5iS3Cl7/I=; b=d3FXTArximyIcoqD8IzTI3sFzT7hjq/+GdgcLIr4e2IuWGJni4qN5WNWmIGM6dZZe4 A9rWFMpY0RhALG9/QkjFCnD5ZYYUaPogTRdXqe6tbkJRkih92bl4+7fmNiiWoz3RqvVS UwTwZjGIay9aTwRH16lupraJp5g3wnXUVk/jFJAOqATNX1Tc209twh8LAQFHcw0gyEyd m/h86iWKovIXwYv1lNemMUJudwUbz+yaZ8nEfbl1Yr7+BTlv+3Nkfyg8HO6RDUXUOAbp x46cv+aXOKpke06HoEZW4Y7cukKDUsq35c3LFskzW4YPuyDhWKAVnrUuf/XaEHGHkVYC 84IA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=NSbAkiX42wwwydPZEFESNxKxxvsB/IftHv5iS3Cl7/I=; b=rBXlT53k8+nTCZCy4qxI+EhLNrqDrYZMIm+mXppiM6SXb6TvoCxOXqU3lgFClvjazu PGpjiORfiR80fQuxNLGQCl4+wBQDIl2M0Lk0wdzcQFKCnT7nKeDk0MvEKyEThOFn56q9 eUEIhGfNZaCrS7gvWmsGC3QJG27+jvYAmgM2LtJ8Xm6jt7Niv+j2DSFumqJJiuR2Dxeb GYb2TqtiB1DYPlGyvCXQdhWB1aqZVE57yoAZhYV5v9Md+VKqLn1QW++PcejJbgfWOWlS PU2N3Q6O6IzeiR9WQTj44scclKWJY7omjX/ebm7hLBvSHLAb+sabKW/BFH//N3cxiZl4 4ojg== X-Gm-Message-State: APjAAAWjXxshzSfmcRYf/tzb/3DmtdhFVTNq/LQz9QvwV+dBvjRE9PBy geQHBmLIShCETE11b18c5kbC+k0s X-Google-Smtp-Source: APXvYqwZNJsoL1k5LSFLyLvyw5rO7aMRX9ZG02hIR0AMjIaLclQiqUQbhubnsyPjYnh5kn5fkFFccw== X-Received: by 2002:a05:600c:389:: with SMTP id w9mr966066wmd.5.1575924471230; Mon, 09 Dec 2019 12:47:51 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id a1sm681059wrr.80.2019.12.09.12.47.50 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 09 Dec 2019 12:47:50 -0800 (PST) Message-Id: In-Reply-To: References: From: "Elijah Newren via GitGitGadget" Date: Mon, 09 Dec 2019 20:47:43 +0000 Subject: [PATCH 6/8] dir: fix checks on common prefix directory Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Elijah Newren Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Elijah Newren Many years ago, the directory traversing logic had an optimization that would always recurse into any directory that was a common prefix of all the pathspecs without walking the leading directories to get down to the desired directory. Thus, git ls-files -o .git/ # case A would notice that .git/ was a common prefix of all pathspecs (since it is the only pathspec listed), and then traverse into it and start showing unknown files under that directory. Unfortunately, .git/ is not a directory we should be traversing into, which made this optimization problematic. This also affected cases like git ls-files -o --exclude-standard t/ # case B where t/ was in the .gitignore file and thus isn't interesting and shouldn't be recursed into. It also affected cases like git ls-files -o --directory untracked_dir/ # case C where untracked_dir/ is indeed untracked and thus interesting, but the --directory flag means we only want to show the directory itself, not recurse into it and start listing untracked files below it. The case B class of bugs were noted and fixed in commits 16e2cfa90993 ("read_directory(): further split treat_path()", 2010-01-08) and 48ffef966c76 ("ls-files: fix overeager pathspec optimization", 2010-01-08), with the idea being that we first wanted to check whether the common prefix was interesting. The former patch noted that treat_path() couldn't be used when checking the common prefix because treat_path() requires a dir_entry() and we haven't read any directories at the point we are checking the common prefix. So, that patch split treat_one_path() out of treat_path(). The latter patch then created a new treat_leading_path() which duplicated by hand the bits of treat_path() that couldn't be broken out and then called treat_one_path() for the remainder. There were three problems with this approach: * The duplicated logic in treat_leading_path() accidentally missed the check for special paths (such as is_dot_or_dotdot and matching ".git"), causing case A types of bugs to continue to be an issue. * The treat_leading_path() logic assumed we should traverse into anything where path_treatment was not path_none, i.e. it perpetuated class C types of bugs. * It meant we had split logic that needed to kept in sync, running the risk that people introduced new inconsistencies (such as in commit be8a84c52669, which we reverted earlier in this series, or in commit df5bcdf83ae which we'll fix in a subsequent commit) Fix most these problems by making treat_leading_path() not only loop over each leading path component, but calling treat_path() directly on each. To do so, we have to create a synthetic dir_entry, but that only takes a few lines. Then, pay attention to the path_treatment result we get from treat_path() and don't treat path_excluded, path_untracked, and path_recurse all the same as path_recurse. This leaves one remaining problem, the new inconsistency from commit df5bcdf83ae. That will be addressed in a subsequent commit. Signed-off-by: Elijah Newren --- dir.c | 54 +++++++++++++++---- ...common-prefixes-and-directory-traversal.sh | 6 +-- 2 files changed, 46 insertions(+), 14 deletions(-) diff --git a/dir.c b/dir.c index 645b44ea64..9c71a9ac21 100644 --- a/dir.c +++ b/dir.c @@ -2102,37 +2102,69 @@ static int treat_leading_path(struct dir_struct *dir, const struct pathspec *pathspec) { struct strbuf sb = STRBUF_INIT; - int baselen, rc = 0; + int prevlen, baselen; const char *cp; + struct cached_dir cdir; + struct dirent de; + enum path_treatment state = path_none; + + /* + * For each directory component of path, we are going to check whether + * that path is relevant given the pathspec. For example, if path is + * foo/bar/baz/ + * then we will ask treat_path() whether we should go into foo, then + * whether we should go into bar, then whether baz is relevant. + * Checking each is important because e.g. if path is + * .git/info/ + * then we need to check .git to know we shouldn't traverse it. + * If the return from treat_path() is: + * * path_none, for any path, we return false. + * * path_recurse, for all path components, we return true + * * for some intermediate component, we make sure + * to add that path to the relevant list but return false + * signifying that we shouldn't recurse into it. + */ while (len && path[len - 1] == '/') len--; if (!len) return 1; + + memset(&cdir, 0, sizeof(cdir)); + memset(&de, 0, sizeof(de)); + cdir.de = &de; + de.d_type = DT_DIR; baselen = 0; + prevlen = 0; while (1) { - cp = path + baselen + !!baselen; + prevlen = baselen + !!baselen; + cp = path + prevlen; cp = memchr(cp, '/', path + len - cp); if (!cp) baselen = len; else baselen = cp - path; - strbuf_setlen(&sb, 0); + strbuf_reset(&sb); strbuf_add(&sb, path, baselen); if (!is_directory(sb.buf)) break; - if (simplify_away(sb.buf, sb.len, pathspec)) - break; - if (treat_one_path(dir, NULL, istate, &sb, baselen, pathspec, - DT_DIR, NULL) == path_none) + strbuf_reset(&sb); + strbuf_add(&sb, path, prevlen); + memcpy(de.d_name, path+prevlen, baselen-prevlen); + de.d_name[baselen-prevlen] = '\0'; + state = treat_path(dir, NULL, &cdir, istate, &sb, prevlen, + pathspec); + if (state != path_recurse) break; /* do not recurse into it */ - if (len <= baselen) { - rc = 1; + if (len <= baselen) break; /* finished checking */ - } } + add_path_to_appropriate_result_list(dir, NULL, &cdir, istate, + &sb, baselen, pathspec, + state); + strbuf_release(&sb); - return rc; + return state == path_recurse; } static const char *get_ident_string(void) diff --git a/t/t3011-common-prefixes-and-directory-traversal.sh b/t/t3011-common-prefixes-and-directory-traversal.sh index d4c06fcd76..0151ea8b6d 100755 --- a/t/t3011-common-prefixes-and-directory-traversal.sh +++ b/t/t3011-common-prefixes-and-directory-traversal.sh @@ -72,7 +72,7 @@ test_expect_success 'git ls-files -o --directory untracked_dir does not recurse' test_cmp expect actual ' -test_expect_failure 'git ls-files -o --directory untracked_dir/ does not recurse' ' +test_expect_success 'git ls-files -o --directory untracked_dir/ does not recurse' ' echo untracked_dir/ >expect && git ls-files -o --directory untracked_dir/ >actual && test_cmp expect actual @@ -84,7 +84,7 @@ test_expect_success 'git ls-files -o untracked_repo does not recurse' ' test_cmp expect actual ' -test_expect_failure 'git ls-files -o untracked_repo/ does not recurse' ' +test_expect_success 'git ls-files -o untracked_repo/ does not recurse' ' echo untracked_repo/ >expect && git ls-files -o untracked_repo/ >actual && test_cmp expect actual @@ -124,7 +124,7 @@ test_expect_success 'git ls-files -o .git shows nothing' ' test_cmp expect actual ' -test_expect_failure 'git ls-files -o .git/ shows nothing' ' +test_expect_success 'git ls-files -o .git/ shows nothing' ' >expect && git ls-files -o .git/ >actual && test_cmp expect actual From patchwork Mon Dec 9 20:47:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Arver via GitGitGadget X-Patchwork-Id: 11280409 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5A412138C for ; Mon, 9 Dec 2019 20:47:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 2E533206E0 for ; Mon, 9 Dec 2019 20:47:56 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="mNmO9RRF" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726801AbfLIUry (ORCPT ); Mon, 9 Dec 2019 15:47:54 -0500 Received: from mail-wm1-f65.google.com ([209.85.128.65]:34200 "EHLO mail-wm1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726923AbfLIUrx (ORCPT ); Mon, 9 Dec 2019 15:47:53 -0500 Received: by mail-wm1-f65.google.com with SMTP id f4so745782wmj.1 for ; Mon, 09 Dec 2019 12:47:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=TxfuD3DIvF4/jJBYVQfQ67Z5+jOsIpci86DYhHcr4cs=; b=mNmO9RRFDOMdHN0fj0TpxkO12HcD2I6lx57tgQWRjdlbzHGNdE/se96JM978qzDB9w nagbATG799WDZAlWA81SZQ70fqFseuzuoK9yTuCm2DBhVAhxNETr0/gzbiCOfzZZC+jc 50y5oy1kWL2HrJPXGHKJXJOUGPHPVMx9dQkm/gB2ifgt/siyfo19P40fVNLEn6FxT3o4 ZW6IqVisaYi3NNntz5ETFHIhyMSkc5vIWf3GHyif8MuKkqwWTqI/1Q5uzM4f65gvnwIu 8uPI7y3SC2CjsYoziJT2yh69bWuxitJ6l9E06whIZpF1OV2zk1zE5blUS4/7OHWHipUR H1pQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=TxfuD3DIvF4/jJBYVQfQ67Z5+jOsIpci86DYhHcr4cs=; b=ADbteh8LTiTP872RhLObnNXdnmlR6B+Zs0sgcVnXRcbfkFIrgoeWOXIgGyGn57z4i/ Puj9cS8flNY/XAfJ4sSDJJE55njj69oyb3knKNuPjwSpbm7AFc5tx9neOvQ2HM2BCJ4m 98Oi/tkJWmJ7Uhpe13AVnfe/vJrDzZrIonEfIVuckGNytKyDpRJmxRbPdNFfNsCfQoHm JLh8sFoSia4tavFFwuh+xe/s+c4ptBj8g7ifVhPlu26uoq2laZ+F4hKgiysdTSvimJXk R8wFXzM0yLkKvToRaS31OIYOknAmxvr83wFyIKbNMibOQOtfN0f0Hz+dKeoeLI/W1+0g UQvw== X-Gm-Message-State: APjAAAWqzAOeo8Qi9ei2ELXfvXWKZqKucBuu7cZdoSAbeq4lj1BUGJdP pg9mku1EhL96lpCrV3wXvtgm2YQO X-Google-Smtp-Source: APXvYqxr1Z10Ve08Z0aA/Jobmb8LX9mx+QGxlNeeVQ6sq6GBUjdROH1K75vZGAi8M8nEmS3aKCTj8g== X-Received: by 2002:a05:600c:24b:: with SMTP id 11mr984858wmj.19.1575924471947; Mon, 09 Dec 2019 12:47:51 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id 2sm740961wrq.31.2019.12.09.12.47.51 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 09 Dec 2019 12:47:51 -0800 (PST) Message-Id: <4f8bf05d265bdf6c9242e21feeda14ad86dd0a4c.1575924466.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Elijah Newren via GitGitGadget" Date: Mon, 09 Dec 2019 20:47:44 +0000 Subject: [PATCH 7/8] dir: synchronize treat_leading_path() and read_directory_recursive() Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Elijah Newren Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Elijah Newren Our optimization to avoid calling into read_directory_recursive() when all pathspecs have a common leading directory mean that we need to match the logic that read_directory_recursive() would use if we had just called it from the root. Since it does more than call treat_path() we need to copy that same logic. Alternatively, we could try to change treat_path to return path_recurse for an untracked directory under the given special circumstances that this logic checks for, but a simple switch results in many test failures such as 'git clean -d' not wiping out untracked but empty directories. To work around that, we'd need the caller of treat_path to check for path_recurse and sometimes special case it into path_untracked. In other words, we'd still have extra logic in both places. Needing to duplicate logic like this means it is guaranteed someone will eventually need to make further changes and forget to update both locations. It is tempting to just nuke the leading_directory special casing to avoid such bugs and simplify the code, but unpack_trees' verify_clean_subdirectory() also calls read_directory() and does so with a non-empty leading path, so I'm hesitant to try to restructure further. Add obnoxious warnings to treat_leading_path() and read_directory_recursive() to try to warn people of such problems. Signed-off-by: Elijah Newren --- dir.c | 30 +++++++++++++++++++ ...common-prefixes-and-directory-traversal.sh | 2 +- t/t7061-wtstatus-ignore.sh | 2 +- 3 files changed, 32 insertions(+), 2 deletions(-) diff --git a/dir.c b/dir.c index 9c71a9ac21..bb6e481909 100644 --- a/dir.c +++ b/dir.c @@ -1990,6 +1990,15 @@ static enum path_treatment read_directory_recursive(struct dir_struct *dir, struct untracked_cache_dir *untracked, int check_only, int stop_at_first_file, const struct pathspec *pathspec) { + /* + * WARNING WARNING WARNING: + * + * Any updates to the traversal logic here may need corresponding + * updates in treat_leading_path(). See the commit message for the + * commit adding this warning as well as the commit preceding it + * for details. + */ + struct cached_dir cdir; enum path_treatment state, subdir_state, dir_state = path_none; struct strbuf path = STRBUF_INIT; @@ -2101,6 +2110,15 @@ static int treat_leading_path(struct dir_struct *dir, const char *path, int len, const struct pathspec *pathspec) { + /* + * WARNING WARNING WARNING: + * + * Any updates to the traversal logic here may need corresponding + * updates in treat_leading_path(). See the commit message for the + * commit adding this warning as well as the commit preceding it + * for details. + */ + struct strbuf sb = STRBUF_INIT; int prevlen, baselen; const char *cp; @@ -2154,6 +2172,18 @@ static int treat_leading_path(struct dir_struct *dir, de.d_name[baselen-prevlen] = '\0'; state = treat_path(dir, NULL, &cdir, istate, &sb, prevlen, pathspec); + if (state == path_untracked && + get_dtype(cdir.de, istate, sb.buf, sb.len) == DT_DIR && + (dir->flags & DIR_SHOW_IGNORED_TOO || + do_match_pathspec(istate, pathspec, sb.buf, sb.len, + baselen, NULL, DO_MATCH_LEADING_PATHSPEC) == MATCHED_RECURSIVELY_LEADING_PATHSPEC)) { + add_path_to_appropriate_result_list(dir, NULL, &cdir, + istate, + &sb, baselen, + pathspec, state); + state = path_recurse; + } + if (state != path_recurse) break; /* do not recurse into it */ if (len <= baselen) diff --git a/t/t3011-common-prefixes-and-directory-traversal.sh b/t/t3011-common-prefixes-and-directory-traversal.sh index 0151ea8b6d..d48ee1a320 100755 --- a/t/t3011-common-prefixes-and-directory-traversal.sh +++ b/t/t3011-common-prefixes-and-directory-traversal.sh @@ -179,7 +179,7 @@ test_expect_success 'git ls-files -o consistent between one or two dirs' ' # ls-files doesn't have a way to request showing both untracked and ignored # files at the same time, so use `git status --ignored` -test_expect_failure 'git status --ignored shows same files under dir with or without pathspec' ' +test_expect_success 'git status --ignored shows same files under dir with or without pathspec' ' cat <<-EOF >expect && ?? an_untracked_dir/ !! an_untracked_dir/ignored diff --git a/t/t7061-wtstatus-ignore.sh b/t/t7061-wtstatus-ignore.sh index ded7f97181..d8060a42e4 100755 --- a/t/t7061-wtstatus-ignore.sh +++ b/t/t7061-wtstatus-ignore.sh @@ -47,7 +47,7 @@ cat >expected <<\EOF !! untracked/ignored EOF -test_expect_failure 'status of untracked directory with --ignored works with or without prefix' ' +test_expect_success 'status of untracked directory with --ignored works with or without prefix' ' git status --porcelain --ignored | grep untracked/ >actual && test_cmp expected actual && From patchwork Mon Dec 9 20:47:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Arver via GitGitGadget X-Patchwork-Id: 11280411 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0A2FA930 for ; Mon, 9 Dec 2019 20:47:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id DD5372071E for ; Mon, 9 Dec 2019 20:47:56 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="kwuweCja" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726923AbfLIUr4 (ORCPT ); Mon, 9 Dec 2019 15:47:56 -0500 Received: from mail-wm1-f66.google.com ([209.85.128.66]:52621 "EHLO mail-wm1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726928AbfLIUry (ORCPT ); Mon, 9 Dec 2019 15:47:54 -0500 Received: by mail-wm1-f66.google.com with SMTP id p9so815696wmc.2 for ; Mon, 09 Dec 2019 12:47:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=zze0uH3Nj4ZOmvZ1O0g/xeKVlrddGGSxpyF/tpjrz40=; b=kwuweCjaJkU3vII5kt60YqdqslKQlYH09Nih/cp/dAK1ziyO/bTwMkbRSB8KV9exCo Nkg9l5kUG7sFNNdzzyk4WiQnEUVYMvkcE2iMPVHJqrIU40FY5x4ZiEHdlLOhCdAA/z8q cq6KtBB9ZU91AVbYJnr4SjsuYOy8GqO6ls7hqJGm6zhJULiPFWzOl0R8iteO+3WCq46Y Y15nbkyNarKdcOerJPYGeZOHQXxuTENSRNY+duTNx69S4GaoY6xr7LOfw2R7ZAYYYHIX e/pG3rHxKt/H5WFCNOx4pTg5x4F9hjciOfL7WmwRsvSIUzI5iqyCvb80K7tqB9hrCtCQ KHJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=zze0uH3Nj4ZOmvZ1O0g/xeKVlrddGGSxpyF/tpjrz40=; b=JZ8O2c7TO/T+YqbFjRzU1vBkSAK7Mr3UrobionNWb/HqWBR/pnqDchLrHkkGzLbo0y cWEvPkBEu3iqsGl70ovRvpXEgq9ec2BeOzJM8qxtFqJj5O2Axp3DjKdONEzB2gq25YMU QD1WecY0WLLp5kEdmBE0dZYd8lQZzsbNLSOWVA8CkiYELMvXWDOWgCDeHhIcmVGhoDwe M05AnzS0fFXVT0NOye9vOqSLwzQ4MaiXCOI3SZo5SsAFibgjJP6Oty2sHYCgpwNubmtE lH4+peFXrJP+MvoxptHPqB7pXXX6TNxMHok81jQvf9sfnO3UGyBS5V55cuxomjPybDiq LwTA== X-Gm-Message-State: APjAAAXnjsFOS5OK69Y/uJ/K+moSz2LYZxO42zdowH/QWX4U8sZLTI2m zb3B7SERlRPfnuIO4ypC3xXkw6o8 X-Google-Smtp-Source: APXvYqyn2t3MO0Pqy1G+0werV4PDGYwBlpCR2yqEdWND7eNL2rmgmqIGpJEeJUwlE7/H6FJLA+tc7g== X-Received: by 2002:a1c:2155:: with SMTP id h82mr1024901wmh.21.1575924472683; Mon, 09 Dec 2019 12:47:52 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id x11sm538002wmg.46.2019.12.09.12.47.52 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 09 Dec 2019 12:47:52 -0800 (PST) Message-Id: <2200bf144af2eab98919dd6ed6313d408b1cba1e.1575924466.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Elijah Newren via GitGitGadget" Date: Mon, 09 Dec 2019 20:47:45 +0000 Subject: [PATCH 8/8] dir: consolidate similar code in treat_directory() Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Junio C Hamano , Elijah Newren Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Elijah Newren Both the DIR_SKIP_NESTED_GIT and DIR_NO_GITLINKS cases were checking for whether a path was actually a nonbare repository. That code could be shared, with just the result of how to act differing between the two cases. Signed-off-by: Elijah Newren --- dir.c | 18 +++++++----------- 1 file changed, 7 insertions(+), 11 deletions(-) diff --git a/dir.c b/dir.c index bb6e481909..04541b798b 100644 --- a/dir.c +++ b/dir.c @@ -1461,6 +1461,8 @@ static enum path_treatment treat_directory(struct dir_struct *dir, const char *dirname, int len, int baselen, int exclude, const struct pathspec *pathspec) { + int nested_repo = 0; + /* The "len-1" is to strip the final '/' */ switch (directory_exists_in_index(istate, dirname, len-1)) { case index_directory: @@ -1470,15 +1472,16 @@ static enum path_treatment treat_directory(struct dir_struct *dir, return path_none; case index_nonexistent: - if (dir->flags & DIR_SKIP_NESTED_GIT) { - int nested_repo; + if ((dir->flags & DIR_SKIP_NESTED_GIT) || + !(dir->flags & DIR_NO_GITLINKS)) { struct strbuf sb = STRBUF_INIT; strbuf_addstr(&sb, dirname); nested_repo = is_nonbare_repository_dir(&sb); strbuf_release(&sb); - if (nested_repo) - return path_none; } + if (nested_repo) + return ((dir->flags & DIR_SKIP_NESTED_GIT) ? path_none : + (exclude ? path_excluded : path_untracked)); if (dir->flags & DIR_SHOW_OTHER_DIRECTORIES) break; @@ -1506,13 +1509,6 @@ static enum path_treatment treat_directory(struct dir_struct *dir, return path_none; } - if (!(dir->flags & DIR_NO_GITLINKS)) { - struct strbuf sb = STRBUF_INIT; - strbuf_addstr(&sb, dirname); - if (is_nonbare_repository_dir(&sb)) - return exclude ? path_excluded : path_untracked; - strbuf_release(&sb); - } return path_recurse; }