From patchwork Mon Apr 1 18:02:17 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Mazo, Andrey" X-Patchwork-Id: 10880427 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EF53D14DE for ; Mon, 1 Apr 2019 18:02:24 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DCA3428689 for ; Mon, 1 Apr 2019 18:02:24 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D053128764; Mon, 1 Apr 2019 18:02:24 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 549B228689 for ; Mon, 1 Apr 2019 18:02:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730971AbfDASCX (ORCPT ); Mon, 1 Apr 2019 14:02:23 -0400 Received: from mail-eopbgr810123.outbound.protection.outlook.com ([40.107.81.123]:61961 "EHLO NAM01-BY2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1729908AbfDASCV (ORCPT ); Mon, 1 Apr 2019 14:02:21 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kastle.onmicrosoft.com; s=selector1-checkvideo-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=fApAHcZQMtYa/QSLAHsE4NtFBjm4hfFqSIphn1aIuO0=; b=D8bMt3XsXp4EmUHiY17vY/6n60XbbzVOSL15KV17y1cEO/EDa/J9ZQSLwxJvgklNTzdFYNpTHtuX7C42f2xz6WEs4eqt3M5O7ihPh958qNrp7bfMgFtlKjSQzvrFU2pAAwGmi/Ks23ga3Z4OlhY7VpbhVHnCpaEvxrh+l533fI4= Received: from DM6PR08MB4956.namprd08.prod.outlook.com (20.176.115.217) by DM6PR08MB5610.namprd08.prod.outlook.com (20.178.26.91) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1750.20; Mon, 1 Apr 2019 18:02:17 +0000 Received: from DM6PR08MB4956.namprd08.prod.outlook.com ([fe80::1ca7:6cc5:a0b3:e164]) by DM6PR08MB4956.namprd08.prod.outlook.com ([fe80::1ca7:6cc5:a0b3:e164%3]) with mapi id 15.20.1750.021; Mon, 1 Apr 2019 18:02:17 +0000 From: "Mazo, Andrey" To: "git@vger.kernel.org" CC: "Mazo, Andrey" , Luke Diamand , Eric Sunshine , George Vanburgh , Lars Schneider , Miguel Torroja , Romain Merland , Vitor Antunes , Andrew Oakley , =?iso-8859-1?q?SZEDER_G=E1bor?= , Andrey Mazo , Junio C Hamano Subject: [PATCH v3 1/8] git-p4: detect/prevent infinite loop in gitCommitByP4Change() Thread-Topic: [PATCH v3 1/8] git-p4: detect/prevent infinite loop in gitCommitByP4Change() Thread-Index: AQHU6LULPukLbzpG80yBUB+l08v90g== Date: Mon, 1 Apr 2019 18:02:17 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BN6PR16CA0044.namprd16.prod.outlook.com (2603:10b6:405:14::30) To DM6PR08MB4956.namprd08.prod.outlook.com (2603:10b6:5:4b::25) authentication-results: spf=none (sender IP is ) smtp.mailfrom=amazo@checkvideo.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.19.2 x-originating-ip: [70.163.25.109] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 3c56b122-ac44-456d-add8-08d6b6cc2da2 x-microsoft-antispam: BCL:0;PCL:0;RULEID:(2390118)(7020095)(4652040)(8989299)(5600139)(711020)(4605104)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(2017052603328)(7193020);SRVR:DM6PR08MB5610; x-ms-traffictypediagnostic: DM6PR08MB5610: x-microsoft-antispam-prvs: x-forefront-prvs: 0994F5E0C5 x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(6029001)(136003)(366004)(396003)(346002)(376002)(39860400002)(199004)(189003)(486006)(102836004)(118296001)(86362001)(2906002)(50226002)(316002)(4326008)(54906003)(5660300002)(2351001)(105586002)(5640700003)(6486002)(26005)(106356001)(52116002)(186003)(8936002)(71190400001)(76176011)(68736007)(71200400001)(99286004)(6116002)(3846002)(97736004)(6436002)(6506007)(386003)(14454004)(478600001)(1730700003)(8676002)(81166006)(53936002)(7736002)(81156014)(66066001)(305945005)(7416002)(6512007)(2501003)(36756003)(476003)(14444005)(11346002)(256004)(446003)(6916009)(2616005)(25786009);DIR:OUT;SFP:1102;SCL:1;SRVR:DM6PR08MB5610;H:DM6PR08MB4956.namprd08.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; received-spf: None (protection.outlook.com: checkvideo.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: Y2W/dsKkl8o7rB92dDIGCHyAwGzxundE6CkTomLwzxb86je7i9dYLEnlG+XLF3XmrXPJLDz5FphtFC704+Cl+kYcsIecLZI0Bj/vZBoY/eZ07KNMvcX2ALUi6igvXCm1tQkGZgJSZ7CEEhSKseCQ/FyArXpK5tB4fTUuZJedsVOOs8LjDzZXzzgFgJax88bXtnKn+b/qVrCgPZE04ceABM6f2liOBP0U6kO6xaup2XaIwIV/1H/xMK8odCfsk9LhdaXlO4P2u2jqhd8osgWyqET6kZIaxI9Vk8Zv5WszdlD/64JM+V94CatyXBE63o54+22WxNJB0kLymhGaj8S/KvhtboEFsKzbgznjTk9672q75RyKOVMNfyMR47gzq06r+evfbw3NLHBivQut4SPsCj1pIKIEMDA7Hsy2F62OypM= MIME-Version: 1.0 X-OriginatorOrg: checkvideo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3c56b122-ac44-456d-add8-08d6b6cc2da2 X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Apr 2019 18:02:17.7545 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 7280061d-06ed-4a4e-a2b1-cc9ab5638c09 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR08MB5610 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Under certain circumstances, gitCommitByP4Change() can enter an infinite loop resulting in `git p4 sync` hanging forever. The problem is that `git rev-list --bisect ^` can return ``, which would result in reinspecting and potentially an infinite loop. This can happen when importing just a subset of P4 repository and/or with explicit "--changesfile" option. A real-life example: """ looking in ref refs/remotes/p4/mybranch for change 26894 using bisect... Reading pipe: git rev-parse refs/remotes/p4/mybranch trying: earliest latest 4daff81c520a82678e1ef347f2b5e97258101ae1 Reading pipe: git rev-list --bisect 4daff81c520a82678e1ef347f2b5e97258101ae1 Reading pipe: git cat-file commit 147f5d3292af2e1cc4a56a7b96db845144c68486 current change 25339 trying: earliest ^147f5d3292af2e1cc4a56a7b96db845144c68486 latest 4daff81c520a82678e1ef347f2b5e97258101ae1 Reading pipe: git rev-list --bisect 4daff81c520a82678e1ef347f2b5e97258101ae1 ^147f5d3292af2e1cc4a56a7b96db845144c68486 Reading pipe: git cat-file commit 51db83df9d588010d0bd995641c85aa0408a5bb9 current change 25420 trying: earliest ^51db83df9d588010d0bd995641c85aa0408a5bb9 latest 4daff81c520a82678e1ef347f2b5e97258101ae1 Reading pipe: git rev-list --bisect 4daff81c520a82678e1ef347f2b5e97258101ae1 ^51db83df9d588010d0bd995641c85aa0408a5bb9 Reading pipe: git cat-file commit e8f83909ceb570f5a7e48c2853f3c5d8207cea52 current change 25448 trying: earliest ^e8f83909ceb570f5a7e48c2853f3c5d8207cea52 latest 4daff81c520a82678e1ef347f2b5e97258101ae1 Reading pipe: git rev-list --bisect 4daff81c520a82678e1ef347f2b5e97258101ae1 ^e8f83909ceb570f5a7e48c2853f3c5d8207cea52 Reading pipe: git cat-file commit 09a48eb7acd594dce52e06681be9c366e1844d66 current change 25521 trying: earliest ^09a48eb7acd594dce52e06681be9c366e1844d66 latest 4daff81c520a82678e1ef347f2b5e97258101ae1 Reading pipe: git rev-list --bisect 4daff81c520a82678e1ef347f2b5e97258101ae1 ^09a48eb7acd594dce52e06681be9c366e1844d66 Reading pipe: git cat-file commit 4daff81c520a82678e1ef347f2b5e97258101ae1 current change 26907 trying: earliest ^09a48eb7acd594dce52e06681be9c366e1844d66 latest 4daff81c520a82678e1ef347f2b5e97258101ae1 Reading pipe: git rev-list --bisect 4daff81c520a82678e1ef347f2b5e97258101ae1 ^09a48eb7acd594dce52e06681be9c366e1844d66 Reading pipe: git cat-file commit 4daff81c520a82678e1ef347f2b5e97258101ae1 current change 26907 trying: earliest ^09a48eb7acd594dce52e06681be9c366e1844d66 latest 4daff81c520a82678e1ef347f2b5e97258101ae1 Reading pipe: git rev-list --bisect 4daff81c520a82678e1ef347f2b5e97258101ae1 ^09a48eb7acd594dce52e06681be9c366e1844d66 Reading pipe: git cat-file commit 4daff81c520a82678e1ef347f2b5e97258101ae1 current change 26907 ... """ The fix is two-fold: * detect an infinite loop and die right away instead of looping forever; * make sure, `git rev-list --bisect` can't return "latestCommit" again by excluding it from the rev-list range explicitly. Signed-off-by: Andrey Mazo --- Notes: I don't have a simple test-case for this yet, and I was able to perform a few complex initial `git p4 sync` runs without hitting this problem. I suspect, I had somehow messed up with branch definitions and --changesfile option at some point. git-p4.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/git-p4.py b/git-p4.py index 5b79920f46..c0a3068b6f 100755 --- a/git-p4.py +++ b/git-p4.py @@ -3323,11 +3323,13 @@ def gitCommitByP4Change(self, ref, change): return next if currentChange < change: earliestCommit = "^%s" % next else: - latestCommit = "%s" % next + if next == latestCommit: + die("Infinite loop while looking in ref %s for change %s. Check your branch mappings" % (ref, change)) + latestCommit = "%s^@" % next return "" def importNewBranch(self, branch, maxChange): # make fast-import flush all changes to disk and update the refs using the checkpoint From patchwork Mon Apr 1 18:02:21 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Mazo, Andrey" X-Patchwork-Id: 10880431 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C6663139A for ; Mon, 1 Apr 2019 18:02:26 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B474828689 for ; Mon, 1 Apr 2019 18:02:26 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A886028764; Mon, 1 Apr 2019 18:02:26 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 33C7628689 for ; Mon, 1 Apr 2019 18:02:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730824AbfDASCZ (ORCPT ); Mon, 1 Apr 2019 14:02:25 -0400 Received: from mail-eopbgr810110.outbound.protection.outlook.com ([40.107.81.110]:41696 "EHLO NAM01-BY2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1731037AbfDASCY (ORCPT ); Mon, 1 Apr 2019 14:02:24 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kastle.onmicrosoft.com; s=selector1-checkvideo-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=iXCzxs4EKeQPPOtg80VfgaDekIrWGbZji66f4q0MrFo=; b=SRrd2LC4sFoeFwHy11lYn2rNOopdPGigTRMjCiG0k5tXjhZp+Tz/2LTwk5SQiCeZ514GpcKp7OeI8Y9qSbcOwq1su7kzcBsbg8rV9sYPMpDSw3HG9ry4cPVpp+aY1VW2UyKnxA5fmggHoyMXGyu9nvq5neUn+kf+nYGBkcbDPGo= Received: from DM6PR08MB4956.namprd08.prod.outlook.com (20.176.115.217) by DM6PR08MB5610.namprd08.prod.outlook.com (20.178.26.91) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1750.20; Mon, 1 Apr 2019 18:02:21 +0000 Received: from DM6PR08MB4956.namprd08.prod.outlook.com ([fe80::1ca7:6cc5:a0b3:e164]) by DM6PR08MB4956.namprd08.prod.outlook.com ([fe80::1ca7:6cc5:a0b3:e164%3]) with mapi id 15.20.1750.021; Mon, 1 Apr 2019 18:02:21 +0000 From: "Mazo, Andrey" To: "git@vger.kernel.org" CC: "Mazo, Andrey" , Luke Diamand , Eric Sunshine , George Vanburgh , Lars Schneider , Miguel Torroja , Romain Merland , Vitor Antunes , Andrew Oakley , =?iso-8859-1?q?SZEDER_G=E1bor?= , Andrey Mazo , Junio C Hamano Subject: [PATCH v3 2/8] git-p4: add failing test for "git-p4: match branches case insensitively if configured" Thread-Topic: [PATCH v3 2/8] git-p4: add failing test for "git-p4: match branches case insensitively if configured" Thread-Index: AQHU6LUNbC8yZm9uaEqFmDinXPwhyA== Date: Mon, 1 Apr 2019 18:02:21 +0000 Message-ID: <68b68ce1e4782bba552a016867bfc629f0d5e24f.1554141338.git.amazo@checkvideo.com> References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BN6PR16CA0044.namprd16.prod.outlook.com (2603:10b6:405:14::30) To DM6PR08MB4956.namprd08.prod.outlook.com (2603:10b6:5:4b::25) authentication-results: spf=none (sender IP is ) smtp.mailfrom=amazo@checkvideo.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.19.2 x-originating-ip: [70.163.25.109] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 5e31b368-61ba-459e-4346-08d6b6cc3018 x-microsoft-antispam: BCL:0;PCL:0;RULEID:(2390118)(7020095)(4652040)(8989299)(5600139)(711020)(4605104)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(2017052603328)(7193020);SRVR:DM6PR08MB5610; x-ms-traffictypediagnostic: DM6PR08MB5610: x-microsoft-antispam-prvs: x-forefront-prvs: 0994F5E0C5 x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(136003)(366004)(396003)(346002)(376002)(39860400002)(199004)(189003)(486006)(102836004)(118296001)(86362001)(2906002)(50226002)(316002)(4326008)(54906003)(5660300002)(2351001)(105586002)(5640700003)(6486002)(26005)(106356001)(52116002)(186003)(8936002)(71190400001)(76176011)(68736007)(71200400001)(99286004)(6116002)(3846002)(97736004)(6436002)(6506007)(386003)(14454004)(478600001)(1730700003)(8676002)(81166006)(53936002)(7736002)(81156014)(66066001)(305945005)(7416002)(6512007)(2501003)(36756003)(476003)(14444005)(11346002)(256004)(446003)(6916009)(2616005)(25786009);DIR:OUT;SFP:1102;SCL:1;SRVR:DM6PR08MB5610;H:DM6PR08MB4956.namprd08.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; received-spf: None (protection.outlook.com: checkvideo.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: 1RyJI8gubOjL9Yz2BdVbk/gpcnBXNs72uh/rP8AKRX77tRV8oPNgH/dHI/Vnj4tFIG9eyE40dXbtqIEDEiOY7lkKtyqxgccWzqoHcClpLO3me4UKTxqKVgLxvL6Tx99jxlX6c3rxYUWNlHOLjdu/uv7izy1ZpetjlF9su8Y08x4uclJUg4lK+DzwVjRlFMYlacz6Xj4OMffejNhxPv9YwoGN8amZcXEh/EepDYPYal+Sex6CdxNvyO4B6qZzmuWHmUYx12688ctNyFDM9zhpnFQ6IuKWDnPux3ISfTsJEcAXza0iQDDLpst3nNA7M6c5+rjVMqP1sZoLcl3NDHroDNbhXmU1dEldRP+BA/wxV+haJVhAMcQhEgkaL/at/6GmVXoHyPni2HpJmu/2a4r7Wk/lM6PAKt+9hL6Xi+CFRCc= MIME-Version: 1.0 X-OriginatorOrg: checkvideo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5e31b368-61ba-459e-4346-08d6b6cc3018 X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Apr 2019 18:02:21.8335 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 7280061d-06ed-4a4e-a2b1-cc9ab5638c09 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR08MB5610 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP In preparation for a fix, add a failing test case to test that git-p4 doesn't fold the case in file paths when doing branch detection case insensitively. (i.e. when core.ignorecase is set) Signed-off-by: Andrey Mazo --- t/t9801-git-p4-branch.sh | 92 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 92 insertions(+) diff --git a/t/t9801-git-p4-branch.sh b/t/t9801-git-p4-branch.sh index 6a86d6996b..c48532e12b 100755 --- a/t/t9801-git-p4-branch.sh +++ b/t/t9801-git-p4-branch.sh @@ -608,10 +608,102 @@ test_expect_success 'Update a file in git side and submit to P4 using client vie cd branch1 && grep "client spec" file1 ) ' +test_expect_success 'restart p4d (case folding enabled)' ' + kill_p4d && + start_p4d -C1 +' + +# +# 1: //depot/main/mf1 +# 2: integrate //depot/main/... -> //depot/branch1/... +# 3: //depot/main/mf2 +# 4: //depot/BRANCH1/B1f3 +# 5: //depot/branch1/b1f4 +# +test_expect_success !CASE_INSENSITIVE_FS 'basic p4 branches for case folding' ' + ( + cd "$cli" && + mkdir -p main && + + echo mf1 >main/mf1 && + p4 add main/mf1 && + p4 submit -d "main/mf1" && + + p4 integrate //depot/main/... //depot/branch1/... && + p4 submit -d "integrate main to branch1" && + + echo mf2 >main/mf2 && + p4 add main/mf2 && + p4 submit -d "main/mf2" && + + mkdir BRANCH1 && + echo B1f3 >BRANCH1/B1f3 && + p4 add BRANCH1/B1f3 && + p4 submit -d "BRANCH1/B1f3" && + + echo b1f4 >branch1/b1f4 && + p4 add branch1/b1f4 && + p4 submit -d "branch1/b1f4" + ) +' + +# Check that files are properly split across branches when ignorecase is set +test_expect_failure !CASE_INSENSITIVE_FS 'git p4 clone, branchList branch definition, ignorecase' ' + test_when_finished cleanup_git && + test_create_repo "$git" && + ( + cd "$git" && + git config git-p4.branchList main:branch1 && + git config --type=bool core.ignoreCase true && + git p4 clone --dest=. --detect-branches //depot@all && + + git log --all --graph --decorate --stat && + + git reset --hard p4/master && + test_path_is_file mf1 && + test_path_is_file mf2 && + test_path_is_missing B1f3 && + test_path_is_missing b1f4 && + + git reset --hard p4/depot/branch1 && + test_path_is_file mf1 && + test_path_is_missing mf2 && + test_path_is_file B1f3 && + test_path_is_file b1f4 + ) +' + +# Check that files are properly split across branches when ignorecase is set, use-client-spec case +test_expect_failure !CASE_INSENSITIVE_FS 'git p4 clone with client-spec, branchList branch definition, ignorecase' ' + client_view "//depot/... //client/..." && + test_when_finished cleanup_git && + test_create_repo "$git" && + ( + cd "$git" && + git config git-p4.branchList main:branch1 && + git config --type=bool core.ignoreCase true && + git p4 clone --dest=. --use-client-spec --detect-branches //depot@all && + + git log --all --graph --decorate --stat && + + git reset --hard p4/master && + test_path_is_file mf1 && + test_path_is_file mf2 && + test_path_is_missing B1f3 && + test_path_is_missing b1f4 && + + git reset --hard p4/depot/branch1 && + test_path_is_file mf1 && + test_path_is_missing mf2 && + test_path_is_file B1f3 && + test_path_is_file b1f4 + ) +' + test_expect_success 'kill p4d' ' kill_p4d ' test_done From patchwork Mon Apr 1 18:02:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Mazo, Andrey" X-Patchwork-Id: 10880435 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DAF2D14DE for ; Mon, 1 Apr 2019 18:02:35 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C776D286A0 for ; Mon, 1 Apr 2019 18:02:35 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BB43B28689; Mon, 1 Apr 2019 18:02:35 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 04C3028689 for ; Mon, 1 Apr 2019 18:02:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731006AbfDASCd (ORCPT ); Mon, 1 Apr 2019 14:02:33 -0400 Received: from mail-eopbgr800114.outbound.protection.outlook.com ([40.107.80.114]:16704 "EHLO NAM03-DM3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1729644AbfDASCb (ORCPT ); Mon, 1 Apr 2019 14:02:31 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kastle.onmicrosoft.com; s=selector1-checkvideo-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=WLBJc3/6T45L7LU/ZLPvJFaGpn7iSfrlOaluT++5XzA=; b=zDC7ASafT5T1BoNtcQyw1oh2XPDW94BrG7hTpsAaYJj94onf9WuiUoiLCdBUuX+vEwhqVqhBoqUpnsWIF8VnVHy+Js7usG6Cd2UYTrZZ3W3F+AWQutWHPlnJAGYO+f8P/mOq023aj0OpSustSw2Y5t84UM0sRIHrOJfSnpdXsnM= Received: from DM6PR08MB4956.namprd08.prod.outlook.com (20.176.115.217) by DM6PR08MB4412.namprd08.prod.outlook.com (20.176.82.158) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1750.22; Mon, 1 Apr 2019 18:02:24 +0000 Received: from DM6PR08MB4956.namprd08.prod.outlook.com ([fe80::1ca7:6cc5:a0b3:e164]) by DM6PR08MB4956.namprd08.prod.outlook.com ([fe80::1ca7:6cc5:a0b3:e164%3]) with mapi id 15.20.1750.021; Mon, 1 Apr 2019 18:02:24 +0000 From: "Mazo, Andrey" To: "git@vger.kernel.org" CC: "Mazo, Andrey" , Luke Diamand , Eric Sunshine , George Vanburgh , Lars Schneider , Miguel Torroja , Romain Merland , Vitor Antunes , Andrew Oakley , =?iso-8859-1?q?SZEDER_G=E1bor?= , Andrey Mazo , Junio C Hamano Subject: [PATCH v3 3/8] git-p4: match branches case insensitively if configured Thread-Topic: [PATCH v3 3/8] git-p4: match branches case insensitively if configured Thread-Index: AQHU6LUPgABiTghHjkK8/wWQW9SQGQ== Date: Mon, 1 Apr 2019 18:02:24 +0000 Message-ID: <6eaad2582c14961ec682d299267b279ce16906ef.1554141338.git.amazo@checkvideo.com> References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BN6PR16CA0044.namprd16.prod.outlook.com (2603:10b6:405:14::30) To DM6PR08MB4956.namprd08.prod.outlook.com (2603:10b6:5:4b::25) authentication-results: spf=none (sender IP is ) smtp.mailfrom=amazo@checkvideo.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.19.2 x-originating-ip: [70.163.25.109] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 1dc4edde-b0b0-4b89-ec25-08d6b6cc316e x-microsoft-antispam: BCL:0;PCL:0;RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600139)(711020)(4605104)(2017052603328)(7193020);SRVR:DM6PR08MB4412; x-ms-traffictypediagnostic: DM6PR08MB4412: x-microsoft-antispam-prvs: x-forefront-prvs: 0994F5E0C5 x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(396003)(39860400002)(136003)(366004)(346002)(376002)(189003)(199004)(6116002)(118296001)(2351001)(66066001)(305945005)(105586002)(6916009)(3846002)(50226002)(97736004)(2501003)(106356001)(8936002)(81166006)(81156014)(7736002)(8676002)(2906002)(1730700003)(54906003)(71200400001)(71190400001)(316002)(14454004)(478600001)(14444005)(256004)(19627235002)(99286004)(446003)(6506007)(386003)(102836004)(52116002)(76176011)(486006)(476003)(2616005)(25786009)(11346002)(7416002)(6436002)(5660300002)(26005)(5640700003)(4326008)(53936002)(6512007)(86362001)(68736007)(36756003)(6486002)(186003);DIR:OUT;SFP:1102;SCL:1;SRVR:DM6PR08MB4412;H:DM6PR08MB4956.namprd08.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;MX:1;A:1; received-spf: None (protection.outlook.com: checkvideo.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: fiDBXrU0ImMYkwQPExTfhZS16OZF6c7kgLnk4s0kAknoFeiAw/0t5JtCvt79P8BEgoQIGhzywPUcIUaR2SbtvSrFVBJtiprTKrxLWqToUAamK3wcCIcyBZnEVR11RVgH0w7MLKjtyCet7yFy+oqcGbyiY2pqh8Aq47qLxGxfzwnvHz1VpS2M2VZoXVZPgB7ukfg0q2ElS4yKk2LNaQF3NGQu3GEItp1Gv3Bn63J866mD/wZi6TEGmDQ/cFigJwM6RLPi1MlVKC5gJnOAFT9rSTCo4PnxevT1RiYHH/6eF/WCxL3k+dAjXS4PcBCxcsUEMBKJzR1a5Ax9YWEvE3ucM/tLOfrveQiMTCT5QRDZCeE2sFgCOJ6+aP4CoIs7mYhwrL+xMIZay8gezJwFUqbLZxciA68/JlSWuiQAZCifLro= MIME-Version: 1.0 X-OriginatorOrg: checkvideo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1dc4edde-b0b0-4b89-ec25-08d6b6cc316e X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Apr 2019 18:02:24.1302 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 7280061d-06ed-4a4e-a2b1-cc9ab5638c09 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR08MB4412 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP git-p4 knows how to handle case insensitivity in file paths if core.ignorecase is set. However, when determining a branch for a file, it still does a case-sensitive prefix match. This may result in some file changes to be lost on import. For example, given the following commits 1. add //depot/main/file1 2. add //depot/DirA/file2 3. add //depot/dira/file3 4. add //depot/DirA/file4 and "branchList = main:DirA" branch mapping, commit 3 will be lost. So, do branch search case insensitively if running with core.ignorecase set. Teach splitFilesIntoBranches() to use the p4PathStartsWith() function for path prefix matches instead of always case-sensitive match. Signed-off-by: Andrey Mazo --- git-p4.py | 4 ++-- t/t9801-git-p4-branch.sh | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/git-p4.py b/git-p4.py index c0a3068b6f..f3e5ccb7af 100755 --- a/git-p4.py +++ b/git-p4.py @@ -2666,11 +2666,11 @@ def stripRepoPath(self, path, prefixes): # branch detection moves files up a level (the branch name) # from what client spec interpretation gives path = self.clientSpecDirs.map_in_client(path) if self.detectBranches: for b in self.knownBranches: - if path.startswith(b + "/"): + if p4PathStartsWith(path, b + "/"): path = path[len(b)+1:] elif self.keepRepoPath: # Preserve everything in relative path name except leading # //depot/; just look at first prefix as they all should @@ -2721,11 +2721,11 @@ def splitFilesIntoBranches(self, commit): relPath = self.stripRepoPath(path, self.depotPaths) for branch in self.knownBranches.keys(): # add a trailing slash so that a commit into qt/4.2foo # doesn't end up in qt/4.2, e.g. - if relPath.startswith(branch + "/"): + if p4PathStartsWith(relPath, branch + "/"): if branch not in branches: branches[branch] = [] branches[branch].append(file) break diff --git a/t/t9801-git-p4-branch.sh b/t/t9801-git-p4-branch.sh index c48532e12b..4779448b4c 100755 --- a/t/t9801-git-p4-branch.sh +++ b/t/t9801-git-p4-branch.sh @@ -648,11 +648,11 @@ test_expect_success !CASE_INSENSITIVE_FS 'basic p4 branches for case folding' ' p4 submit -d "branch1/b1f4" ) ' # Check that files are properly split across branches when ignorecase is set -test_expect_failure !CASE_INSENSITIVE_FS 'git p4 clone, branchList branch definition, ignorecase' ' +test_expect_success !CASE_INSENSITIVE_FS 'git p4 clone, branchList branch definition, ignorecase' ' test_when_finished cleanup_git && test_create_repo "$git" && ( cd "$git" && git config git-p4.branchList main:branch1 && @@ -674,11 +674,11 @@ test_expect_failure !CASE_INSENSITIVE_FS 'git p4 clone, branchList branch defini test_path_is_file b1f4 ) ' # Check that files are properly split across branches when ignorecase is set, use-client-spec case -test_expect_failure !CASE_INSENSITIVE_FS 'git p4 clone with client-spec, branchList branch definition, ignorecase' ' +test_expect_success !CASE_INSENSITIVE_FS 'git p4 clone with client-spec, branchList branch definition, ignorecase' ' client_view "//depot/... //client/..." && test_when_finished cleanup_git && test_create_repo "$git" && ( cd "$git" && From patchwork Mon Apr 1 18:02:26 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Mazo, Andrey" X-Patchwork-Id: 10880433 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9300B139A for ; Mon, 1 Apr 2019 18:02:31 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7FEFC28689 for ; Mon, 1 Apr 2019 18:02:31 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 73F1A286A0; Mon, 1 Apr 2019 18:02:31 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0810128803 for ; Mon, 1 Apr 2019 18:02:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731027AbfDASCa (ORCPT ); Mon, 1 Apr 2019 14:02:30 -0400 Received: from mail-eopbgr700094.outbound.protection.outlook.com ([40.107.70.94]:27833 "EHLO NAM04-SN1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1730884AbfDASC2 (ORCPT ); Mon, 1 Apr 2019 14:02:28 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kastle.onmicrosoft.com; s=selector1-checkvideo-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=hGU2xz9yoNd355zlBtTlFUkLXuZ3yqbgB4XOtyApIFY=; b=PKjq+yQLVM2KBhUuLPdcwmFbUf8PxRb0VwPcPc/sw/Xxx8UHoIFxMGbdc18CJFvv7QiDYq3Zgbhq8WpnOoiVsrpYVRHZTBq76vz/teTBqdlpnO4dy5Cqs22vXuj7y7tRJoB0LjjEtiXV8xCXwjxJZEGglCMDo92Hju4ehJJFRVc= Received: from DM6PR08MB4956.namprd08.prod.outlook.com (20.176.115.217) by DM6PR08MB5465.namprd08.prod.outlook.com (20.178.24.138) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1750.17; Mon, 1 Apr 2019 18:02:26 +0000 Received: from DM6PR08MB4956.namprd08.prod.outlook.com ([fe80::1ca7:6cc5:a0b3:e164]) by DM6PR08MB4956.namprd08.prod.outlook.com ([fe80::1ca7:6cc5:a0b3:e164%3]) with mapi id 15.20.1750.021; Mon, 1 Apr 2019 18:02:26 +0000 From: "Mazo, Andrey" To: "git@vger.kernel.org" CC: "Mazo, Andrey" , Luke Diamand , Eric Sunshine , George Vanburgh , Lars Schneider , Miguel Torroja , Romain Merland , Vitor Antunes , Andrew Oakley , =?iso-8859-1?q?SZEDER_G=E1bor?= , Andrey Mazo , Junio C Hamano Subject: [PATCH v3 4/8] git-p4: don't groom exclude path list on every commit Thread-Topic: [PATCH v3 4/8] git-p4: don't groom exclude path list on every commit Thread-Index: AQHU6LUQZzjar1DN4kml4TvFGRj2CA== Date: Mon, 1 Apr 2019 18:02:26 +0000 Message-ID: <1bd5e170e00956ba131cf57b680102610a1b4aa2.1554141338.git.amazo@checkvideo.com> References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BN6PR16CA0044.namprd16.prod.outlook.com (2603:10b6:405:14::30) To DM6PR08MB4956.namprd08.prod.outlook.com (2603:10b6:5:4b::25) authentication-results: spf=none (sender IP is ) smtp.mailfrom=amazo@checkvideo.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.19.2 x-originating-ip: [70.163.25.109] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 63bfcfaf-7e88-4d65-6cf3-08d6b6cc32b1 x-microsoft-antispam: BCL:0;PCL:0;RULEID:(2390118)(7020095)(4652040)(8989299)(5600139)(711020)(4605104)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(2017052603328)(7193020);SRVR:DM6PR08MB5465; x-ms-traffictypediagnostic: DM6PR08MB5465: x-microsoft-antispam-prvs: x-forefront-prvs: 0994F5E0C5 x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(376002)(366004)(396003)(346002)(136003)(39860400002)(189003)(199004)(105586002)(81156014)(2351001)(446003)(36756003)(76176011)(6916009)(11346002)(71190400001)(305945005)(4326008)(102836004)(478600001)(50226002)(7736002)(25786009)(6116002)(5660300002)(14454004)(2906002)(256004)(14444005)(3846002)(6436002)(1730700003)(53936002)(6512007)(26005)(5640700003)(71200400001)(66066001)(7416002)(186003)(6486002)(97736004)(86362001)(2501003)(81166006)(99286004)(316002)(6506007)(386003)(8676002)(54906003)(8936002)(486006)(68736007)(52116002)(2616005)(118296001)(476003)(106356001);DIR:OUT;SFP:1102;SCL:1;SRVR:DM6PR08MB5465;H:DM6PR08MB4956.namprd08.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;MX:1;A:1; received-spf: None (protection.outlook.com: checkvideo.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: AB+2b+rNxZwQxUrm9mswUlktd9+gkIpL6srUmThvS8oa9fzgop6nAJw+qKIEYF37T340TJKElmjX1iNTqksCAYaby4nj79S5fsRV/K1xQU7RJ/XEVsrjvSYIsC90No8ipandTp/pnz3lPcZYPx+wIUv/6ZNSuyzD0mheDRTvpXuHnjeLVaSOwl7YCpBUgiW6rYoxNGb6p+JBPuyxOjckjBFE165qwSCHFmGoHKzd2L37n3D5ftg543loyXMApZKAqULmAlrpkmHY7vKRlYVcVCSbkkuebsIhrMS32P/HuyNbP+WnxSavjNdBObuZpKe80NlINBsF16HmN8ZrxGkavqUK1YhkjahI4llI7LEH63DDguruuumiQ6BPBMjnqgAoOvViJczGmvfWOF2xOaZ77mjnlhPAcEZ/wVDF9hArxjI= MIME-Version: 1.0 X-OriginatorOrg: checkvideo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 63bfcfaf-7e88-4d65-6cf3-08d6b6cc32b1 X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Apr 2019 18:02:26.1837 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 7280061d-06ed-4a4e-a2b1-cc9ab5638c09 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR08MB5465 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Currently, `cloneExclude` array is being groomed (by removing trailing "...") on every changeset. (since `extractFilesFromCommit()` is called on every imported changeset) As a micro-optimization, do it once while parsing arguments. Also, prepend "/" and remove trailing "..." at the same time. Signed-off-by: Andrey Mazo --- git-p4.py | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/git-p4.py b/git-p4.py index f3e5ccb7af..7edcbad055 100755 --- a/git-p4.py +++ b/git-p4.py @@ -1314,11 +1314,11 @@ class Command: def __init__(self): self.usage = "usage: %prog [options]" self.needsGit = True self.verbose = False - # This is required for the "append" cloneExclude action + # This is required for the "append" update_shelve action def ensure_value(self, attr, value): if not hasattr(self, attr) or getattr(self, attr) is None: setattr(self, attr, value) return getattr(self, attr) @@ -2528,10 +2528,15 @@ def map_in_client(self, depot_path): return self.client_spec_path_cache[depot_path] die( "Error: %s is not found in client spec path" % depot_path ) return "" +def cloneExcludeCallback(option, opt_str, value, parser): + # prepend "/" because the first "/" was consumed as part of the option itself. + # ("-//depot/A/..." becomes "/depot/A/..." after option parsing) + parser.values.cloneExclude += ["/" + re.sub(r"\.\.\.$", "", value)] + class P4Sync(Command, P4UserMap): def __init__(self): Command.__init__(self) P4UserMap.__init__(self) @@ -2551,11 +2556,11 @@ def __init__(self): optparse.make_option("--keep-path", dest="keepRepoPath", action='store_true', help="Keep entire BRANCH/DIR/SUBDIR prefix during import"), optparse.make_option("--use-client-spec", dest="useClientSpec", action='store_true', help="Only sync files that are included in the Perforce Client Spec"), optparse.make_option("-/", dest="cloneExclude", - action="append", type="string", + action="callback", callback=cloneExcludeCallback, type="string", help="exclude depot path"), ] self.description = """Imports from Perforce into a git repository.\n example: //depot/my/project/ -- to import the current head @@ -2617,12 +2622,10 @@ def checkpoint(self): out = self.gitOutput.readline() if self.verbose: print("checkpoint finished: " + out) def extractFilesFromCommit(self, commit, shelved=False, shelved_cl = 0): - self.cloneExclude = [re.sub(r"\.\.\.$", "", path) - for path in self.cloneExclude] files = [] fnum = 0 while "depotFile%s" % fnum in commit: path = commit["depotFile%s" % fnum] @@ -3888,11 +3891,10 @@ def run(self, args): if not self.cloneDestination and len(depotPaths) > 1: self.cloneDestination = depotPaths[-1] depotPaths = depotPaths[:-1] - self.cloneExclude = ["/"+p for p in self.cloneExclude] for p in depotPaths: if not p.startswith("//"): sys.stderr.write('Depot paths must start with "//": %s\n' % p) return False From patchwork Mon Apr 1 18:02:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Mazo, Andrey" X-Patchwork-Id: 10880437 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7BD4414DE for ; Mon, 1 Apr 2019 18:02:38 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6846328689 for ; Mon, 1 Apr 2019 18:02:38 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5BCFE28764; Mon, 1 Apr 2019 18:02:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E425E28689 for ; Mon, 1 Apr 2019 18:02:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730844AbfDASCg (ORCPT ); Mon, 1 Apr 2019 14:02:36 -0400 Received: from mail-eopbgr790099.outbound.protection.outlook.com ([40.107.79.99]:44704 "EHLO NAM03-CO1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1730870AbfDASCd (ORCPT ); Mon, 1 Apr 2019 14:02:33 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kastle.onmicrosoft.com; s=selector1-checkvideo-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=nYS71XeoDaWAZXGpQQV29M+IKlqtdU6YHO8XpRABOsw=; b=WPPbODBd0vTwEZtJyY43kGERaFaEgVhENMQBnKDOzDUPe6NX2Q8PU92/NxzC2/1Yzc/ktnLDNhHwvjf/8HP1Hoqgb46qcydTsQh124Q6n+0ykuqoI0Pbj5NRZLUjaGB1xUI8R5t+xBCVHNntkKDxRQ943S3OsXhytT3tix+8AfM= Received: from DM6PR08MB4956.namprd08.prod.outlook.com (20.176.115.217) by DM6PR08MB5465.namprd08.prod.outlook.com (20.178.24.138) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1750.17; Mon, 1 Apr 2019 18:02:29 +0000 Received: from DM6PR08MB4956.namprd08.prod.outlook.com ([fe80::1ca7:6cc5:a0b3:e164]) by DM6PR08MB4956.namprd08.prod.outlook.com ([fe80::1ca7:6cc5:a0b3:e164%3]) with mapi id 15.20.1750.021; Mon, 1 Apr 2019 18:02:29 +0000 From: "Mazo, Andrey" To: "git@vger.kernel.org" CC: "Mazo, Andrey" , Luke Diamand , Eric Sunshine , George Vanburgh , Lars Schneider , Miguel Torroja , Romain Merland , Vitor Antunes , Andrew Oakley , =?iso-8859-1?q?SZEDER_G=E1bor?= , Andrey Mazo , Junio C Hamano Subject: [PATCH v3 5/8] git-p4: add failing test for "don't exclude other files with same prefix" Thread-Topic: [PATCH v3 5/8] git-p4: add failing test for "don't exclude other files with same prefix" Thread-Index: AQHU6LUS+SqAQ0SJYEq8tvgIcCeqUA== Date: Mon, 1 Apr 2019 18:02:29 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BN6PR16CA0044.namprd16.prod.outlook.com (2603:10b6:405:14::30) To DM6PR08MB4956.namprd08.prod.outlook.com (2603:10b6:5:4b::25) authentication-results: spf=none (sender IP is ) smtp.mailfrom=amazo@checkvideo.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.19.2 x-originating-ip: [70.163.25.109] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 8e6afa8e-9d1e-48bf-adfa-08d6b6cc34cd x-microsoft-antispam: BCL:0;PCL:0;RULEID:(2390118)(7020095)(4652040)(8989299)(5600139)(711020)(4605104)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(2017052603328)(7193020);SRVR:DM6PR08MB5465; x-ms-traffictypediagnostic: DM6PR08MB5465: x-microsoft-antispam-prvs: x-forefront-prvs: 0994F5E0C5 x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(376002)(366004)(396003)(346002)(136003)(39860400002)(189003)(199004)(105586002)(81156014)(2351001)(446003)(36756003)(76176011)(6916009)(11346002)(71190400001)(305945005)(4326008)(102836004)(478600001)(50226002)(7736002)(25786009)(6116002)(5660300002)(14454004)(2906002)(256004)(14444005)(3846002)(6436002)(1730700003)(53936002)(6512007)(26005)(5640700003)(71200400001)(66066001)(7416002)(186003)(6486002)(97736004)(86362001)(2501003)(81166006)(99286004)(316002)(6506007)(386003)(8676002)(54906003)(8936002)(486006)(68736007)(52116002)(2616005)(118296001)(476003)(106356001);DIR:OUT;SFP:1102;SCL:1;SRVR:DM6PR08MB5465;H:DM6PR08MB4956.namprd08.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;MX:1;A:1; received-spf: None (protection.outlook.com: checkvideo.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: WAJLnZXdlAdt9l5KQRqu/lZsWmz7e8psN8Lc2oLVN/Eyn3gjLWtLJgbOgTyLpXEguA/raEpk64vNNdRehlYkIIb1aKL8YVhnX546o2dhgpdWW1SgkyqmcOTy+2awNSRQKegl/d0V6XSBKK8NkrswDTNi57PMtu3W3mTKeCoyW5tsNjyVrV5SoBc9mk4/5JoX58RMwbRoLam34ekRiH4EG7o11+SyFHQzx/lqUhHOnh5BuDkK9Cprvi7s0kcXfaJhQDulqX98Da9MenrONV6lLAWTRRCE/zfxmyoqMwyBsd2qrVSOx4DoyaS8QQIPKnXpq3NN58Ky7PYDhvutXPuAh+qdqwdfwTC4cpxBIV6emCLf8ztNa/TC0CuibL+V9lp2VkWbOYFg14arAH/lcqWu3ZmvXmnpma+bbzXS5F+Y2kE= MIME-Version: 1.0 X-OriginatorOrg: checkvideo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8e6afa8e-9d1e-48bf-adfa-08d6b6cc34cd X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Apr 2019 18:02:29.7574 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 7280061d-06ed-4a4e-a2b1-cc9ab5638c09 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR08MB5465 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP In preparation for a fix, add a failing test case to test that git-p4 doesn't exclude files with the same prefix unintentionally when exclude paths are specified without a trailing /. I.e., don't exclude "//depot/file_dont_exclude" if run with "-//depot/file". or don't exclude "//depot/discard_file_not" if run with "-//depot/discard_file". Signed-off-by: Andrey Mazo --- t/t9817-git-p4-exclude.sh | 51 +++++++++++++++++++++++++++++++++++---- 1 file changed, 46 insertions(+), 5 deletions(-) diff --git a/t/t9817-git-p4-exclude.sh b/t/t9817-git-p4-exclude.sh index aac568eadf..1c22570797 100755 --- a/t/t9817-git-p4-exclude.sh +++ b/t/t9817-git-p4-exclude.sh @@ -20,49 +20,90 @@ test_expect_success 'create exclude repo' ' ( cd "$cli" && mkdir -p wanted discard && echo wanted >wanted/foo && echo discard >discard/foo && - p4 add wanted/foo discard/foo && + echo discard_file >discard_file && + echo discard_file_not >discard_file_not && + p4 add wanted/foo discard/foo discard_file discard_file_not && p4 submit -d "initial revision" ) ' test_expect_success 'check the repo was created correctly' ' test_when_finished cleanup_git && git p4 clone --dest="$git" //depot/...@all && ( cd "$git" && test_path_is_file wanted/foo && - test_path_is_file discard/foo + test_path_is_file discard/foo && + test_path_is_file discard_file && + test_path_is_file discard_file_not ) ' test_expect_success 'clone, excluding part of repo' ' test_when_finished cleanup_git && git p4 clone -//depot/discard/... --dest="$git" //depot/...@all && ( cd "$git" && test_path_is_file wanted/foo && - test_path_is_missing discard/foo + test_path_is_missing discard/foo && + test_path_is_file discard_file && + test_path_is_file discard_file_not + ) +' + +test_expect_failure 'clone, excluding single file, no trailing /' ' + test_when_finished cleanup_git && + git p4 clone -//depot/discard_file --dest="$git" //depot/...@all && + ( + cd "$git" && + test_path_is_file wanted/foo && + test_path_is_file discard/foo && + test_path_is_missing discard_file && + test_path_is_file discard_file_not ) ' test_expect_success 'clone, then sync with exclude' ' test_when_finished cleanup_git && git p4 clone -//depot/discard/... --dest="$git" //depot/...@all && ( cd "$cli" && - p4 edit wanted/foo discard/foo && + p4 edit wanted/foo discard/foo discard_file_not && date >>wanted/foo && date >>discard/foo && + date >>discard_file_not && p4 submit -d "updating" && cd "$git" && git p4 sync -//depot/discard/... && test_path_is_file wanted/foo && - test_path_is_missing discard/foo + test_path_is_missing discard/foo && + test_path_is_file discard_file && + test_path_is_file discard_file_not + ) +' + +test_expect_failure 'clone, then sync with exclude, no trailing /' ' + test_when_finished cleanup_git && + git p4 clone -//depot/discard/... -//depot/discard_file --dest="$git" //depot/...@all && + ( + cd "$cli" && + p4 edit wanted/foo discard/foo discard_file_not && + date >>wanted/foo && + date >>discard/foo && + date >>discard_file_not && + p4 submit -d "updating" && + + cd "$git" && + git p4 sync -//depot/discard/... -//depot/discard_file && + test_path_is_file wanted/foo && + test_path_is_missing discard/foo && + test_path_is_missing discard_file && + test_path_is_file discard_file_not ) ' test_expect_success 'kill p4d' ' kill_p4d From patchwork Mon Apr 1 18:02:32 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Mazo, Andrey" X-Patchwork-Id: 10880439 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 44953139A for ; Mon, 1 Apr 2019 18:02:39 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 31CDC28689 for ; Mon, 1 Apr 2019 18:02:39 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 25D5528764; Mon, 1 Apr 2019 18:02:39 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B1AAA28689 for ; Mon, 1 Apr 2019 18:02:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731036AbfDASCh (ORCPT ); Mon, 1 Apr 2019 14:02:37 -0400 Received: from mail-eopbgr790113.outbound.protection.outlook.com ([40.107.79.113]:6848 "EHLO NAM03-CO1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1729644AbfDASCf (ORCPT ); Mon, 1 Apr 2019 14:02:35 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kastle.onmicrosoft.com; s=selector1-checkvideo-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=GIvMw9O2+eRmUr65gU7GU4/HVjN8CnqIaf+WgPFnx/Y=; b=jPMaTI/zp8Nci4Ew94fTwxVUXxSXyd50XRdfnhyQ6HjQnpb7chiwE1n/ilal6PA2vid0kBUZBncGitVIY9cuk0gEpa2YnEb6thhpRzi7czYxgwuPedUPZkJQjCZqjRDQM9Kp52Knz7jYhih5G8e3B0iHSxMkm6lHLnkgmRazqQA= Received: from DM6PR08MB4956.namprd08.prod.outlook.com (20.176.115.217) by DM6PR08MB5465.namprd08.prod.outlook.com (20.178.24.138) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1750.17; Mon, 1 Apr 2019 18:02:33 +0000 Received: from DM6PR08MB4956.namprd08.prod.outlook.com ([fe80::1ca7:6cc5:a0b3:e164]) by DM6PR08MB4956.namprd08.prod.outlook.com ([fe80::1ca7:6cc5:a0b3:e164%3]) with mapi id 15.20.1750.021; Mon, 1 Apr 2019 18:02:33 +0000 From: "Mazo, Andrey" To: "git@vger.kernel.org" CC: "Mazo, Andrey" , Luke Diamand , Eric Sunshine , George Vanburgh , Lars Schneider , Miguel Torroja , Romain Merland , Vitor Antunes , Andrew Oakley , =?iso-8859-1?q?SZEDER_G=E1bor?= , Andrey Mazo , Junio C Hamano Subject: [PATCH v3 6/8] git-p4: don't exclude other files with same prefix Thread-Topic: [PATCH v3 6/8] git-p4: don't exclude other files with same prefix Thread-Index: AQHU6LUUUj4UZiRwYE2lNSWZ7nIpjA== Date: Mon, 1 Apr 2019 18:02:32 +0000 Message-ID: <035abfff2a20516bc13f6b2e219ff158490ceced.1554141338.git.amazo@checkvideo.com> References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BN6PR16CA0044.namprd16.prod.outlook.com (2603:10b6:405:14::30) To DM6PR08MB4956.namprd08.prod.outlook.com (2603:10b6:5:4b::25) authentication-results: spf=none (sender IP is ) smtp.mailfrom=amazo@checkvideo.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.19.2 x-originating-ip: [70.163.25.109] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 7df728bc-6ca2-477e-d907-08d6b6cc36b7 x-microsoft-antispam: BCL:0;PCL:0;RULEID:(2390118)(7020095)(4652040)(8989299)(5600139)(711020)(4605104)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(2017052603328)(7193020);SRVR:DM6PR08MB5465; x-ms-traffictypediagnostic: DM6PR08MB5465: x-microsoft-antispam-prvs: x-forefront-prvs: 0994F5E0C5 x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(376002)(366004)(396003)(346002)(136003)(39860400002)(189003)(199004)(105586002)(81156014)(2351001)(446003)(36756003)(76176011)(6916009)(11346002)(71190400001)(305945005)(4326008)(102836004)(478600001)(50226002)(7736002)(25786009)(6116002)(5660300002)(14454004)(2906002)(256004)(14444005)(3846002)(6436002)(1730700003)(53936002)(6512007)(26005)(5640700003)(71200400001)(66066001)(7416002)(186003)(6486002)(97736004)(86362001)(2501003)(81166006)(99286004)(316002)(6506007)(386003)(8676002)(54906003)(8936002)(486006)(68736007)(52116002)(2616005)(118296001)(476003)(106356001);DIR:OUT;SFP:1102;SCL:1;SRVR:DM6PR08MB5465;H:DM6PR08MB4956.namprd08.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;MX:1;A:1; received-spf: None (protection.outlook.com: checkvideo.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: COtB8Vcuj1s/hbC7UqV1bs8ANlJCmBxU6p5A7FUUR5/xC0Tu2j2XfAIvPC34iX7fDsVHZqiPCXjXsLZ92KTvm8xJPBgroolSuqpdkMwwRGJfLg39VsHEI3/EKCSpjSX0l0VbnluNBNqNZuL/LgqKcXzRrqEzAtWRPTyZuFrsuUrM72uj8fv8ad3r8P7cvqCHOmDgpHY8a+BJ3QLcp6/ojpi67/OQoeyAlyDNkiFf74EKJKDhnd99JC3TKNRp3YwpHmabP8G2BgPKmSLcHXa9OcbFc+b0x3ho8/Ez6FGNfjDvMRNGQPkKebaotjjmN91NUfktwAyWqONXD2if3zBhJWKnjGuE+7rcOqzJn8+n98e7tbCnfCz0ac6FDlJ9ihAcztgxvBAlKNQ31NWzH1SGQnwbLvTpMRqPiX5iMat9/lA= MIME-Version: 1.0 X-OriginatorOrg: checkvideo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 7df728bc-6ca2-477e-d907-08d6b6cc36b7 X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Apr 2019 18:02:32.9467 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 7280061d-06ed-4a4e-a2b1-cc9ab5638c09 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR08MB5465 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Make sure not to exclude files unintentionally if exclude paths are specified without a trailing /. I.e., don't exclude "//depot/file_dont_exclude" if run with "-//depot/file". Do this by ensuring that paths without a trailing "/" are only matched completely. Also, abort path search on the first match as a micro-optimization. Signed-off-by: Andrey Mazo --- git-p4.py | 21 ++++++++++++++------- t/t9817-git-p4-exclude.sh | 4 ++-- 2 files changed, 16 insertions(+), 9 deletions(-) diff --git a/git-p4.py b/git-p4.py index 7edcbad055..c47bd8c4d8 100755 --- a/git-p4.py +++ b/git-p4.py @@ -2621,22 +2621,29 @@ def checkpoint(self): self.gitStream.write("progress checkpoint\n\n") out = self.gitOutput.readline() if self.verbose: print("checkpoint finished: " + out) + def isPathWanted(self, path): + for p in self.cloneExclude: + if p.endswith("/"): + if p4PathStartsWith(path, p): + return False + # "-//depot/file1" without a trailing "/" should only exclude "file1", but not "file111" or "file1_dir/file2" + elif path.lower() == p.lower(): + return False + for p in self.depotPaths: + if p4PathStartsWith(path, p): + return True + return False + def extractFilesFromCommit(self, commit, shelved=False, shelved_cl = 0): files = [] fnum = 0 while "depotFile%s" % fnum in commit: path = commit["depotFile%s" % fnum] - - if [p for p in self.cloneExclude - if p4PathStartsWith(path, p)]: - found = False - else: - found = [p for p in self.depotPaths - if p4PathStartsWith(path, p)] + found = self.isPathWanted(path) if not found: fnum = fnum + 1 continue file = {} diff --git a/t/t9817-git-p4-exclude.sh b/t/t9817-git-p4-exclude.sh index 1c22570797..275dd30425 100755 --- a/t/t9817-git-p4-exclude.sh +++ b/t/t9817-git-p4-exclude.sh @@ -51,11 +51,11 @@ test_expect_success 'clone, excluding part of repo' ' test_path_is_file discard_file && test_path_is_file discard_file_not ) ' -test_expect_failure 'clone, excluding single file, no trailing /' ' +test_expect_success 'clone, excluding single file, no trailing /' ' test_when_finished cleanup_git && git p4 clone -//depot/discard_file --dest="$git" //depot/...@all && ( cd "$git" && test_path_is_file wanted/foo && @@ -83,11 +83,11 @@ test_expect_success 'clone, then sync with exclude' ' test_path_is_file discard_file && test_path_is_file discard_file_not ) ' -test_expect_failure 'clone, then sync with exclude, no trailing /' ' +test_expect_success 'clone, then sync with exclude, no trailing /' ' test_when_finished cleanup_git && git p4 clone -//depot/discard/... -//depot/discard_file --dest="$git" //depot/...@all && ( cd "$cli" && p4 edit wanted/foo discard/foo discard_file_not && From patchwork Mon Apr 1 18:02:35 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Mazo, Andrey" X-Patchwork-Id: 10880441 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B945D139A for ; Mon, 1 Apr 2019 18:02:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A707228689 for ; Mon, 1 Apr 2019 18:02:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9B35828764; Mon, 1 Apr 2019 18:02:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2310828689 for ; Mon, 1 Apr 2019 18:02:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731050AbfDASCl (ORCPT ); Mon, 1 Apr 2019 14:02:41 -0400 Received: from mail-eopbgr700122.outbound.protection.outlook.com ([40.107.70.122]:4813 "EHLO NAM04-SN1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1729644AbfDASCi (ORCPT ); Mon, 1 Apr 2019 14:02:38 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kastle.onmicrosoft.com; s=selector1-checkvideo-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4m+inUKchaFpUaP9pA04dauDSik59Fo32R/dgPzLyvk=; b=A4NzckrgEUQkkI5r7yEJhxgsNLCQQ3ojaKmDW4PJZynyR27IqQ+O2EF9KYMylaUIYFszWb23k5Zdq+JqLMEpbWfy0JSGmWZ68NNdC8+m3GyotDrtoWoxrREfG/iyndYF+313FzhqK0hsyhQHFlQNxmIDp6Q9+3Wf+8Z4N29bTKI= Received: from DM6PR08MB4956.namprd08.prod.outlook.com (20.176.115.217) by DM6PR08MB5465.namprd08.prod.outlook.com (20.178.24.138) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1750.17; Mon, 1 Apr 2019 18:02:35 +0000 Received: from DM6PR08MB4956.namprd08.prod.outlook.com ([fe80::1ca7:6cc5:a0b3:e164]) by DM6PR08MB4956.namprd08.prod.outlook.com ([fe80::1ca7:6cc5:a0b3:e164%3]) with mapi id 15.20.1750.021; Mon, 1 Apr 2019 18:02:35 +0000 From: "Mazo, Andrey" To: "git@vger.kernel.org" CC: "Mazo, Andrey" , Luke Diamand , Eric Sunshine , George Vanburgh , Lars Schneider , Miguel Torroja , Romain Merland , Vitor Antunes , Andrew Oakley , =?iso-8859-1?q?SZEDER_G=E1bor?= , Andrey Mazo , Junio C Hamano Subject: [PATCH v3 7/8] git-p4: add failing test for "git-p4: respect excluded paths when detecting branches" Thread-Topic: [PATCH v3 7/8] git-p4: add failing test for "git-p4: respect excluded paths when detecting branches" Thread-Index: AQHU6LUW4zWXpNn8rkaozp8QyCxVTA== Date: Mon, 1 Apr 2019 18:02:35 +0000 Message-ID: <2bde24b7e4d51e52614aaba1d38489cdf58c1543.1554141338.git.amazo@checkvideo.com> References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BN6PR16CA0044.namprd16.prod.outlook.com (2603:10b6:405:14::30) To DM6PR08MB4956.namprd08.prod.outlook.com (2603:10b6:5:4b::25) authentication-results: spf=none (sender IP is ) smtp.mailfrom=amazo@checkvideo.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.19.2 x-originating-ip: [70.163.25.109] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: d7b3ce08-a56b-4c69-e68a-08d6b6cc3873 x-microsoft-antispam: BCL:0;PCL:0;RULEID:(2390118)(7020095)(4652040)(8989299)(5600139)(711020)(4605104)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(2017052603328)(7193020);SRVR:DM6PR08MB5465; x-ms-traffictypediagnostic: DM6PR08MB5465: x-ms-exchange-purlcount: 1 x-microsoft-antispam-prvs: x-forefront-prvs: 0994F5E0C5 x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(376002)(366004)(396003)(346002)(136003)(39860400002)(189003)(199004)(105586002)(81156014)(2351001)(446003)(36756003)(76176011)(6916009)(11346002)(71190400001)(305945005)(4326008)(102836004)(478600001)(50226002)(7736002)(25786009)(53376002)(6116002)(5660300002)(14454004)(2906002)(256004)(14444005)(3846002)(6436002)(1730700003)(53936002)(6512007)(26005)(5640700003)(966005)(71200400001)(66066001)(7416002)(186003)(6486002)(97736004)(86362001)(2501003)(81166006)(99286004)(316002)(6506007)(386003)(8676002)(54906003)(8936002)(486006)(68736007)(52116002)(6306002)(2616005)(118296001)(476003)(106356001);DIR:OUT;SFP:1102;SCL:1;SRVR:DM6PR08MB5465;H:DM6PR08MB4956.namprd08.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;MX:1;A:1; received-spf: None (protection.outlook.com: checkvideo.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: T18HSIKRJlY/1bBkppZcwugA1dA+QriFKxW2Ca1D5DYHonzIBPY1cxHl1TDav00yak3rT4znZGs2TQTuiovSFWnoxcIvKvSkYf18sRaxmWeb0qSHA2MliiDumcGc+x2q0hI4ZbHYh0IPFcnIF66wlTDDiU3qYuyyWcYyoPJ7tubQG0QBXqCZbr2OuoQ9yAvDzw0zij+aMgxt+pRiRDBPvTnfQpZywSitDyt4Ool2lCZboIDaZ0trJKP0ahdCBaDJz3Mf3Fa2UCl9pXld3b89gVuD4AyWySRMUZ2C4Ik3aw3meD3CP1cq7gbHB00OU52ZeHeaHs/94LeQPhhp+6ebbJFv5xZ9LuWOO8KQWhjMIeVY6vb65qL2K6Veuh82u9wyX2QHaYcvS5OpbhWpKGT4EwEIy2hhJz7a7xMi41kT4Ls= MIME-Version: 1.0 X-OriginatorOrg: checkvideo.com X-MS-Exchange-CrossTenant-Network-Message-Id: d7b3ce08-a56b-4c69-e68a-08d6b6cc3873 X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Apr 2019 18:02:35.8809 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 7280061d-06ed-4a4e-a2b1-cc9ab5638c09 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR08MB5465 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP In preparation for a fix, add a failing test case to test that git-p4 doesn't exclude files despite being told to when handling multiple branches. I.e., it should exclude //depot/branch2/file2 when run with -//depot/branch2/file2, but doesn't do this right now. The test is based on 'git p4 clone complex branches' test with the following changes: * account for file3 moved from branch3 to branch4 in test 'git p4 submit to two branches in a single changelist'; * account for branch6 created in test 'git p4 clone file subset branch'; * file2 is expected to be missing from all branches due to explicit exclude. Signed-off-by: Andrey Mazo --- t/t9801-git-p4-branch.sh | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/t/t9801-git-p4-branch.sh b/t/t9801-git-p4-branch.sh index 4779448b4c..7530d22de2 100755 --- a/t/t9801-git-p4-branch.sh +++ b/t/t9801-git-p4-branch.sh @@ -409,10 +409,50 @@ test_expect_failure 'git p4 clone file subset branch' ' test_path_is_missing file2 && test_path_is_missing file3 ) ' +# Check that excluded files are omitted during import +test_expect_failure 'git p4 clone complex branches with excluded files' ' + test_when_finished cleanup_git && + test_create_repo "$git" && + ( + cd "$git" && + git config git-p4.branchList branch1:branch2 && + git config --add git-p4.branchList branch1:branch3 && + git config --add git-p4.branchList branch1:branch4 && + git config --add git-p4.branchList branch1:branch5 && + git config --add git-p4.branchList branch1:branch6 && + git p4 clone --dest=. --detect-branches -//depot/branch1/file2 -//depot/branch2/file2 -//depot/branch3/file2 -//depot/branch4/file2 -//depot/branch5/file2 -//depot/branch6/file2 //depot@all && + git log --all --graph --decorate --stat && + git reset --hard p4/depot/branch1 && + test_path_is_file file1 && + test_path_is_missing file2 && + test_path_is_file file3 && + git reset --hard p4/depot/branch2 && + test_path_is_file file1 && + test_path_is_missing file2 && + test_path_is_missing file3 && + git reset --hard p4/depot/branch3 && + test_path_is_file file1 && + test_path_is_missing file2 && + test_path_is_missing file3 && + git reset --hard p4/depot/branch4 && + test_path_is_file file1 && + test_path_is_missing file2 && + test_path_is_file file3 && + git reset --hard p4/depot/branch5 && + test_path_is_file file1 && + test_path_is_missing file2 && + test_path_is_file file3 && + git reset --hard p4/depot/branch6 && + test_path_is_file file1 && + test_path_is_missing file2 && + test_path_is_missing file3 + ) +' + # From a report in http://stackoverflow.com/questions/11893688 # where --use-client-spec caused branch prefixes not to be removed; # every file in git appeared into a subdirectory of the branch name. test_expect_success 'use-client-spec detect-branches setup' ' rm -rf "$cli" && From patchwork Mon Apr 1 18:02:38 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Mazo, Andrey" X-Patchwork-Id: 10880443 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E5A1814DE for ; Mon, 1 Apr 2019 18:02:44 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D2EEF28689 for ; Mon, 1 Apr 2019 18:02:44 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C6D1328764; Mon, 1 Apr 2019 18:02:44 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6EC0528689 for ; Mon, 1 Apr 2019 18:02:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731063AbfDASCn (ORCPT ); Mon, 1 Apr 2019 14:02:43 -0400 Received: from mail-eopbgr700105.outbound.protection.outlook.com ([40.107.70.105]:15392 "EHLO NAM04-SN1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1730977AbfDASCl (ORCPT ); Mon, 1 Apr 2019 14:02:41 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kastle.onmicrosoft.com; s=selector1-checkvideo-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=q25gsNN9/l2J5A1CKUHpx3C5mUfKkaMOPUZvbGFYSWM=; b=jv8jzSykgOfFGt6pU75NOtK7HvMKMLHEHWF+TTxNJqLOe1HAT4fMM7Vk0w8sCQ0MRlEptvrJ3P9EC1soU1zVyaIYnt5dc3IbpGsrzdbTsdXSAkmwbh/e4hALBzW2SlWv8zf2xk/OPyzBL9bTP9nXTFxgTUIBNsji0IOdOiemwKA= Received: from DM6PR08MB4956.namprd08.prod.outlook.com (20.176.115.217) by DM6PR08MB5465.namprd08.prod.outlook.com (20.178.24.138) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1750.17; Mon, 1 Apr 2019 18:02:38 +0000 Received: from DM6PR08MB4956.namprd08.prod.outlook.com ([fe80::1ca7:6cc5:a0b3:e164]) by DM6PR08MB4956.namprd08.prod.outlook.com ([fe80::1ca7:6cc5:a0b3:e164%3]) with mapi id 15.20.1750.021; Mon, 1 Apr 2019 18:02:38 +0000 From: "Mazo, Andrey" To: "git@vger.kernel.org" CC: "Mazo, Andrey" , Luke Diamand , Eric Sunshine , George Vanburgh , Lars Schneider , Miguel Torroja , Romain Merland , Vitor Antunes , Andrew Oakley , =?iso-8859-1?q?SZEDER_G=E1bor?= , Andrey Mazo , Junio C Hamano Subject: [PATCH v3 8/8] git-p4: respect excluded paths when detecting branches Thread-Topic: [PATCH v3 8/8] git-p4: respect excluded paths when detecting branches Thread-Index: AQHU6LUXzs31Ol25DkaeHUlhE2qkGg== Date: Mon, 1 Apr 2019 18:02:38 +0000 Message-ID: <6d3ffb98a7c94f664acc1bd29a429c006d77a30c.1554141338.git.amazo@checkvideo.com> References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BN6PR16CA0044.namprd16.prod.outlook.com (2603:10b6:405:14::30) To DM6PR08MB4956.namprd08.prod.outlook.com (2603:10b6:5:4b::25) authentication-results: spf=none (sender IP is ) smtp.mailfrom=amazo@checkvideo.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.19.2 x-originating-ip: [70.163.25.109] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 85b253bc-3680-4c56-1120-08d6b6cc3a2b x-microsoft-antispam: BCL:0;PCL:0;RULEID:(2390118)(7020095)(4652040)(8989299)(5600139)(711020)(4605104)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(2017052603328)(7193020);SRVR:DM6PR08MB5465; x-ms-traffictypediagnostic: DM6PR08MB5465: x-microsoft-antispam-prvs: x-forefront-prvs: 0994F5E0C5 x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(376002)(366004)(396003)(346002)(136003)(39860400002)(189003)(199004)(105586002)(81156014)(2351001)(446003)(36756003)(76176011)(6916009)(11346002)(71190400001)(305945005)(4326008)(102836004)(478600001)(50226002)(7736002)(25786009)(6116002)(5660300002)(14454004)(2906002)(256004)(14444005)(3846002)(6436002)(1730700003)(53936002)(6512007)(26005)(5640700003)(71200400001)(66066001)(7416002)(186003)(6486002)(97736004)(86362001)(2501003)(81166006)(99286004)(316002)(6506007)(386003)(8676002)(54906003)(8936002)(486006)(68736007)(52116002)(2616005)(118296001)(476003)(106356001);DIR:OUT;SFP:1102;SCL:1;SRVR:DM6PR08MB5465;H:DM6PR08MB4956.namprd08.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;MX:1;A:1; received-spf: None (protection.outlook.com: checkvideo.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: Pg08QWsMvhC/mfXfQW2xUJIB9kzVv2s9uMFZgTL7FqCaWXEWac4QOd7xIzHslRbqPsSI2btSMRTYBrpRSxWH71H6qcqLfJhX7/lvQbvPuwKJRlDNIcODuLNHyKX5RkAuj0RuS6NfYhquXaJ4yx9hDy4Sr1srQKKNYWLjV6/jkWpzKdDpX2B8A2i3FZfnhaeiAr0LNdPwMZLbmSxmStUME0BKrZxffawmWWHXs1LgorpRBcWgs/HUzG6gyneaNMBICD1o+mkPHQ2xg6NqQi50Cxlh0zhC0f414sMivudEUu+vYqsJKfdDr08uKNx28Hpa8nOXNTdn8WV1diVCz6RrT5Io5v4tKl2wfSIxAeBgab6I45q+2GZ/6jDIqjfqGVczDC18H2tVDz6Pl3pAZKGdoeQS6DqkGJKPOejSRJgbeOQ= MIME-Version: 1.0 X-OriginatorOrg: checkvideo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 85b253bc-3680-4c56-1120-08d6b6cc3a2b X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Apr 2019 18:02:38.7190 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 7280061d-06ed-4a4e-a2b1-cc9ab5638c09 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR08MB5465 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Currently, excluded paths are only handled in the following cases: * no branch detection; * branch detection with using clientspec. However, excluded paths are not respected in case of branch detection without using clientspec. Fix this by consulting the list of excluded paths when splitting files across branches. Signed-off-by: Andrey Mazo --- git-p4.py | 3 +-- t/t9801-git-p4-branch.sh | 2 +- 2 files changed, 2 insertions(+), 3 deletions(-) diff --git a/git-p4.py b/git-p4.py index c47bd8c4d8..96c4b78dc7 100755 --- a/git-p4.py +++ b/git-p4.py @@ -2708,12 +2708,11 @@ def splitFilesIntoBranches(self, commit): branches = {} fnum = 0 while "depotFile%s" % fnum in commit: path = commit["depotFile%s" % fnum] - found = [p for p in self.depotPaths - if p4PathStartsWith(path, p)] + found = self.isPathWanted(path) if not found: fnum = fnum + 1 continue file = {} diff --git a/t/t9801-git-p4-branch.sh b/t/t9801-git-p4-branch.sh index 7530d22de2..9654362052 100755 --- a/t/t9801-git-p4-branch.sh +++ b/t/t9801-git-p4-branch.sh @@ -410,11 +410,11 @@ test_expect_failure 'git p4 clone file subset branch' ' test_path_is_missing file3 ) ' # Check that excluded files are omitted during import -test_expect_failure 'git p4 clone complex branches with excluded files' ' +test_expect_success 'git p4 clone complex branches with excluded files' ' test_when_finished cleanup_git && test_create_repo "$git" && ( cd "$git" && git config git-p4.branchList branch1:branch2 &&