From patchwork Thu Mar 21 22:32:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Mazo, Andrey" X-Patchwork-Id: 10864453 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 83B1B922 for ; Thu, 21 Mar 2019 22:33:32 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 69F672A0B7 for ; Thu, 21 Mar 2019 22:33:32 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5CE5C2A530; Thu, 21 Mar 2019 22:33:32 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D0FC62A0B7 for ; Thu, 21 Mar 2019 22:33:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727194AbfCUWda (ORCPT ); Thu, 21 Mar 2019 18:33:30 -0400 Received: from mail-eopbgr720118.outbound.protection.outlook.com ([40.107.72.118]:2850 "EHLO NAM05-CO1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726840AbfCUWda (ORCPT ); Thu, 21 Mar 2019 18:33:30 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kastle.onmicrosoft.com; s=selector1-checkvideo-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=fApAHcZQMtYa/QSLAHsE4NtFBjm4hfFqSIphn1aIuO0=; b=mAZQivqGFUFXUtzxT+yMM+L/39Y+jBpcxhPWFE68oHhEqcAluzT8egHimxFD0qoGBj+PBZONAdSPqLOlMhVZa/Er1QpqmJEe54XUHU9sbX5Xih0+G4abKG3k2t0/dPetjQdHW+WeG/37t/g0CSYlUAZvtXrrSWYyOcnWbWp0uk4= Received: from DM6PR08MB4956.namprd08.prod.outlook.com (20.176.115.217) by DM6PR08MB4251.namprd08.prod.outlook.com (20.176.82.12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1730.15; Thu, 21 Mar 2019 22:32:45 +0000 Received: from DM6PR08MB4956.namprd08.prod.outlook.com ([fe80::75e5:faf9:290a:a244]) by DM6PR08MB4956.namprd08.prod.outlook.com ([fe80::75e5:faf9:290a:a244%3]) with mapi id 15.20.1709.015; Thu, 21 Mar 2019 22:32:45 +0000 From: "Mazo, Andrey" To: "git@vger.kernel.org" CC: "Mazo, Andrey" , "luke@diamand.org" , "sunshine@sunshineco.com" , "gvanburgh@bloomberg.net" , "larsxschneider@gmail.com" , "miguel.torroja@gmail.com" , "merlorom@yahoo.fr" , "vitor.hda@gmail.com" , "aoakley@roku.com" , "szeder.dev@gmail.com" , "ahippo@yandex.com" , "gitster@pobox.com" Subject: [PATCH v2 1/7] git-p4: detect/prevent infinite loop in gitCommitByP4Change() Thread-Topic: [PATCH v2 1/7] git-p4: detect/prevent infinite loop in gitCommitByP4Change() Thread-Index: AQHU4DYBurNperuERU6FwqqqyjyYfA== Date: Thu, 21 Mar 2019 22:32:45 +0000 Message-ID: <3ac39171d441b84a20d5e918a9995e8d8de627c5.1553207234.git.amazo@checkvideo.com> References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BN3PR05CA0023.namprd05.prod.outlook.com (2603:10b6:400::33) To DM6PR08MB4956.namprd08.prod.outlook.com (2603:10b6:5:4b::25) authentication-results: spf=none (sender IP is ) smtp.mailfrom=amazo@checkvideo.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.19.2 x-originating-ip: [70.163.25.109] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: c8420298-d124-4a2f-317b-08d6ae4d23a5 x-microsoft-antispam: BCL:0;PCL:0;RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600127)(711020)(4605104)(2017052603328)(7153060)(7193020);SRVR:DM6PR08MB4251; x-ms-traffictypediagnostic: DM6PR08MB4251: x-microsoft-antispam-prvs: x-forefront-prvs: 0983EAD6B2 x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(6029001)(396003)(346002)(136003)(39860400002)(376002)(366004)(199004)(189003)(3846002)(6116002)(26005)(7736002)(2616005)(6916009)(71200400001)(7416002)(4326008)(118296001)(6486002)(102836004)(11346002)(71190400001)(2351001)(1730700003)(81166006)(52116002)(5660300002)(105586002)(8936002)(316002)(66066001)(50226002)(81156014)(2906002)(106356001)(99286004)(14454004)(54906003)(36756003)(76176011)(8676002)(6506007)(386003)(446003)(486006)(14444005)(53936002)(6436002)(5640700003)(186003)(478600001)(476003)(86362001)(6512007)(68736007)(2501003)(256004)(25786009)(305945005)(97736004);DIR:OUT;SFP:1102;SCL:1;SRVR:DM6PR08MB4251;H:DM6PR08MB4956.namprd08.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; received-spf: None (protection.outlook.com: checkvideo.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: gT9blEmpKTpVNfIw1okIe5Yz8T+X3n2T5i/IuDy5AD5JdZTfZdqlit21REQLo22+ELiK7OmRIMk+pkU5sC0S4xjLcdH/Kg1eart2M3c/q1R9AALsdBUj9jFQ6Ku8RD9ENUb26UOBd00naJYVVn6S0H2PpRjmtjnYNLxnhs3x8uBzblmgMbCE7Q5nKENg4WN9aTkymUak0+KFlxO/J1u2RfozmbEhxZd1Pm1A2S1QuzLOPQItNdtu3il4iUzct0HNmef7+jQmmVMb+pRGgv0Y13ce/uSxnFvnRRRkcFiGoLz04bV/EIKCWwM4oYT4LBemEqtvT3oNL7SXqdTkzyZsSS9o8QJ1IDbcRFzuEYHIIueGSl4YnO6AF3huDn+Qts3yRcTrRKDIgWKc4NidNjlS9/t8SJ1RcwUv426s3KYq7sc= MIME-Version: 1.0 X-OriginatorOrg: checkvideo.com X-MS-Exchange-CrossTenant-Network-Message-Id: c8420298-d124-4a2f-317b-08d6ae4d23a5 X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Mar 2019 22:32:45.5939 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 7280061d-06ed-4a4e-a2b1-cc9ab5638c09 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR08MB4251 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Under certain circumstances, gitCommitByP4Change() can enter an infinite loop resulting in `git p4 sync` hanging forever. The problem is that `git rev-list --bisect ^` can return ``, which would result in reinspecting and potentially an infinite loop. This can happen when importing just a subset of P4 repository and/or with explicit "--changesfile" option. A real-life example: """ looking in ref refs/remotes/p4/mybranch for change 26894 using bisect... Reading pipe: git rev-parse refs/remotes/p4/mybranch trying: earliest latest 4daff81c520a82678e1ef347f2b5e97258101ae1 Reading pipe: git rev-list --bisect 4daff81c520a82678e1ef347f2b5e97258101ae1 Reading pipe: git cat-file commit 147f5d3292af2e1cc4a56a7b96db845144c68486 current change 25339 trying: earliest ^147f5d3292af2e1cc4a56a7b96db845144c68486 latest 4daff81c520a82678e1ef347f2b5e97258101ae1 Reading pipe: git rev-list --bisect 4daff81c520a82678e1ef347f2b5e97258101ae1 ^147f5d3292af2e1cc4a56a7b96db845144c68486 Reading pipe: git cat-file commit 51db83df9d588010d0bd995641c85aa0408a5bb9 current change 25420 trying: earliest ^51db83df9d588010d0bd995641c85aa0408a5bb9 latest 4daff81c520a82678e1ef347f2b5e97258101ae1 Reading pipe: git rev-list --bisect 4daff81c520a82678e1ef347f2b5e97258101ae1 ^51db83df9d588010d0bd995641c85aa0408a5bb9 Reading pipe: git cat-file commit e8f83909ceb570f5a7e48c2853f3c5d8207cea52 current change 25448 trying: earliest ^e8f83909ceb570f5a7e48c2853f3c5d8207cea52 latest 4daff81c520a82678e1ef347f2b5e97258101ae1 Reading pipe: git rev-list --bisect 4daff81c520a82678e1ef347f2b5e97258101ae1 ^e8f83909ceb570f5a7e48c2853f3c5d8207cea52 Reading pipe: git cat-file commit 09a48eb7acd594dce52e06681be9c366e1844d66 current change 25521 trying: earliest ^09a48eb7acd594dce52e06681be9c366e1844d66 latest 4daff81c520a82678e1ef347f2b5e97258101ae1 Reading pipe: git rev-list --bisect 4daff81c520a82678e1ef347f2b5e97258101ae1 ^09a48eb7acd594dce52e06681be9c366e1844d66 Reading pipe: git cat-file commit 4daff81c520a82678e1ef347f2b5e97258101ae1 current change 26907 trying: earliest ^09a48eb7acd594dce52e06681be9c366e1844d66 latest 4daff81c520a82678e1ef347f2b5e97258101ae1 Reading pipe: git rev-list --bisect 4daff81c520a82678e1ef347f2b5e97258101ae1 ^09a48eb7acd594dce52e06681be9c366e1844d66 Reading pipe: git cat-file commit 4daff81c520a82678e1ef347f2b5e97258101ae1 current change 26907 trying: earliest ^09a48eb7acd594dce52e06681be9c366e1844d66 latest 4daff81c520a82678e1ef347f2b5e97258101ae1 Reading pipe: git rev-list --bisect 4daff81c520a82678e1ef347f2b5e97258101ae1 ^09a48eb7acd594dce52e06681be9c366e1844d66 Reading pipe: git cat-file commit 4daff81c520a82678e1ef347f2b5e97258101ae1 current change 26907 ... """ The fix is two-fold: * detect an infinite loop and die right away instead of looping forever; * make sure, `git rev-list --bisect` can't return "latestCommit" again by excluding it from the rev-list range explicitly. Signed-off-by: Andrey Mazo --- Notes: I don't have a simple test-case for this yet, and I was able to perform a few complex initial `git p4 sync` runs without hitting this problem. I suspect, I had somehow messed up with branch definitions and --changesfile option at some point. git-p4.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/git-p4.py b/git-p4.py index 5b79920f46..c0a3068b6f 100755 --- a/git-p4.py +++ b/git-p4.py @@ -3323,11 +3323,13 @@ def gitCommitByP4Change(self, ref, change): return next if currentChange < change: earliestCommit = "^%s" % next else: - latestCommit = "%s" % next + if next == latestCommit: + die("Infinite loop while looking in ref %s for change %s. Check your branch mappings" % (ref, change)) + latestCommit = "%s^@" % next return "" def importNewBranch(self, branch, maxChange): # make fast-import flush all changes to disk and update the refs using the checkpoint From patchwork Thu Mar 21 22:32:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Mazo, Andrey" X-Patchwork-Id: 10864451 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5D34F1708 for ; Thu, 21 Mar 2019 22:32:59 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 44E2B2A0B7 for ; Thu, 21 Mar 2019 22:32:59 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 394D42A537; Thu, 21 Mar 2019 22:32:59 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CCAAA2A340 for ; Thu, 21 Mar 2019 22:32:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727166AbfCUWc5 (ORCPT ); Thu, 21 Mar 2019 18:32:57 -0400 Received: from mail-eopbgr740090.outbound.protection.outlook.com ([40.107.74.90]:6221 "EHLO NAM01-BN3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726681AbfCUWc4 (ORCPT ); Thu, 21 Mar 2019 18:32:56 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kastle.onmicrosoft.com; s=selector1-checkvideo-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zsXKuS0sLoF6vg9kwfz/4lj7MRt+ss9M7eELjPZ5gdM=; b=WeB3tEtSE7QTVQcqUUMPgn+yU7/gqeQnCXAU6LKLK3P05HsHsjDBdb5p37rTVyvwevY92sVGQxOLbBFgboGJei/x+pZvVSp3NYXb2xNXG6XwCxdkXObu1k05oCVt/DdeedOaZ+mG0wtYEiTWM2XDGbHAc7QwIIStgwPsRGgK984= Received: from DM6PR08MB4956.namprd08.prod.outlook.com (20.176.115.217) by DM6PR08MB4410.namprd08.prod.outlook.com (20.176.82.156) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1730.16; Thu, 21 Mar 2019 22:32:54 +0000 Received: from DM6PR08MB4956.namprd08.prod.outlook.com ([fe80::75e5:faf9:290a:a244]) by DM6PR08MB4956.namprd08.prod.outlook.com ([fe80::75e5:faf9:290a:a244%3]) with mapi id 15.20.1709.015; Thu, 21 Mar 2019 22:32:54 +0000 From: "Mazo, Andrey" To: "git@vger.kernel.org" CC: "Mazo, Andrey" , "luke@diamand.org" , "sunshine@sunshineco.com" , "gvanburgh@bloomberg.net" , "larsxschneider@gmail.com" , "miguel.torroja@gmail.com" , "merlorom@yahoo.fr" , "vitor.hda@gmail.com" , "aoakley@roku.com" , "szeder.dev@gmail.com" , "ahippo@yandex.com" , "gitster@pobox.com" Subject: [PATCH v2 2/7] git-p4: match branches case insensitively if configured Thread-Topic: [PATCH v2 2/7] git-p4: match branches case insensitively if configured Thread-Index: AQHU4DYGwpQ0yt3xbE+SDG6OAdz4sg== Date: Thu, 21 Mar 2019 22:32:54 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BN3PR05CA0023.namprd05.prod.outlook.com (2603:10b6:400::33) To DM6PR08MB4956.namprd08.prod.outlook.com (2603:10b6:5:4b::25) authentication-results: spf=none (sender IP is ) smtp.mailfrom=amazo@checkvideo.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.19.2 x-originating-ip: [70.163.25.109] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: fe3a9977-b3b7-4fec-6c8f-08d6ae4d28df x-microsoft-antispam: BCL:0;PCL:0;RULEID:(2390118)(7020095)(4652040)(8989299)(5600127)(711020)(4605104)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(2017052603328)(7153060)(7193020);SRVR:DM6PR08MB4410; x-ms-traffictypediagnostic: DM6PR08MB4410: x-microsoft-antispam-prvs: x-forefront-prvs: 0983EAD6B2 x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(396003)(39860400002)(346002)(366004)(136003)(376002)(189003)(199004)(26005)(2906002)(6436002)(68736007)(14444005)(7416002)(36756003)(6486002)(316002)(256004)(106356001)(66066001)(486006)(476003)(97736004)(11346002)(446003)(2501003)(118296001)(2616005)(105586002)(2351001)(54906003)(6506007)(386003)(6916009)(25786009)(8676002)(5660300002)(99286004)(76176011)(478600001)(102836004)(81166006)(81156014)(1730700003)(3846002)(6116002)(305945005)(71190400001)(71200400001)(5640700003)(4326008)(186003)(7736002)(86362001)(8936002)(53936002)(19627235002)(52116002)(50226002)(6512007)(14454004);DIR:OUT;SFP:1102;SCL:1;SRVR:DM6PR08MB4410;H:DM6PR08MB4956.namprd08.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;MX:1;A:1; received-spf: None (protection.outlook.com: checkvideo.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: sHzjkL+l39S4RSB3gL3brBMIh68345MGl9QS+PhWdfJnxc6fxD9QCULrtCBMcpip4fJOqsvXwLTBeywgR5WW+BsZgXGznreojWD8ERThuMS+IaALhg35Sk9FclGneTLguVZ0dFoQa1sB7spxHYi54uCIOgX+zqRsUR3XGOkSohMpRbYT+xyrpf7u8qPYCdC94HqEUc9mfiJf8d50WCEydTa3FoyGgpbufDjyNkUid8dP5glC0YESkXws5EOe6yT1eTojBK+fgvpvcqAU7+XQQxV9MJ8YNsRWTpXAQ5T1Lx0WanWiToDaGxPz0hYujY+pdXnJ47FoGIFiyQ0BDlqBDHeCsoPeLJIN6WOoz5KQx+/ryeUfwjZir07XzaIojMTh6vMS5zUidufl0gKcEDnCzag6sgtGdpA+yi3oGtQAzXQ= MIME-Version: 1.0 X-OriginatorOrg: checkvideo.com X-MS-Exchange-CrossTenant-Network-Message-Id: fe3a9977-b3b7-4fec-6c8f-08d6ae4d28df X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Mar 2019 22:32:54.3201 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 7280061d-06ed-4a4e-a2b1-cc9ab5638c09 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR08MB4410 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP git-p4 knows how to handle case insensitivity in file paths if core.ignorecase is set. However, when determining a branch for a file, it still does a case-sensitive prefix match. This may result in some file changes to be lost on import. For example, given the following commits 1. add //depot/main/file1 2. add //depot/DirA/file2 3. add //depot/dira/file3 4. add //depot/DirA/file4 and "branchList = main:DirA" branch mapping, commit 3 will be lost. So, do branch search case insensitively if running with core.ignorecase set. Teach splitFilesIntoBranches() to use the p4PathStartsWith() function for path prefix matches instead of always case-sensitive match. Signed-off-by: Andrey Mazo --- git-p4.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/git-p4.py b/git-p4.py index c0a3068b6f..91c610f960 100755 --- a/git-p4.py +++ b/git-p4.py @@ -2721,11 +2721,11 @@ def splitFilesIntoBranches(self, commit): relPath = self.stripRepoPath(path, self.depotPaths) for branch in self.knownBranches.keys(): # add a trailing slash so that a commit into qt/4.2foo # doesn't end up in qt/4.2, e.g. - if relPath.startswith(branch + "/"): + if p4PathStartsWith(relPath, branch + "/"): if branch not in branches: branches[branch] = [] branches[branch].append(file) break From patchwork Thu Mar 21 22:32:57 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Mazo, Andrey" X-Patchwork-Id: 10864455 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DB148922 for ; Thu, 21 Mar 2019 22:33:34 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C2DD32A0B7 for ; Thu, 21 Mar 2019 22:33:34 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B60CB2A530; Thu, 21 Mar 2019 22:33:34 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3E4792A0B7 for ; Thu, 21 Mar 2019 22:33:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727207AbfCUWdd (ORCPT ); Thu, 21 Mar 2019 18:33:33 -0400 Received: from mail-eopbgr720118.outbound.protection.outlook.com ([40.107.72.118]:2850 "EHLO NAM05-CO1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726460AbfCUWdc (ORCPT ); Thu, 21 Mar 2019 18:33:32 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kastle.onmicrosoft.com; s=selector1-checkvideo-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=RlH2x1R8E9eZa6pzO88bQsHk1eo441ZLR7kYd5RqXTc=; b=cQZ5nOdOHFE06Xg+KBwOpOy4I6Iyb40bA2GX9XYnSqfurccpKchUMN9C1EP3+1l9Uh9uHV68rCdEeZ4LS2TN4S7iywtZd3jjgIKInXAlHDfiMV120jVJtSdGf+5eDLGKBO3tP/uW42nS//LPZCwMZ15UUAGlVuo4Y4jmqbW8CIQ= Received: from DM6PR08MB4956.namprd08.prod.outlook.com (20.176.115.217) by DM6PR08MB4251.namprd08.prod.outlook.com (20.176.82.12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1730.15; Thu, 21 Mar 2019 22:32:57 +0000 Received: from DM6PR08MB4956.namprd08.prod.outlook.com ([fe80::75e5:faf9:290a:a244]) by DM6PR08MB4956.namprd08.prod.outlook.com ([fe80::75e5:faf9:290a:a244%3]) with mapi id 15.20.1709.015; Thu, 21 Mar 2019 22:32:57 +0000 From: "Mazo, Andrey" To: "git@vger.kernel.org" CC: "Mazo, Andrey" , "luke@diamand.org" , "sunshine@sunshineco.com" , "gvanburgh@bloomberg.net" , "larsxschneider@gmail.com" , "miguel.torroja@gmail.com" , "merlorom@yahoo.fr" , "vitor.hda@gmail.com" , "aoakley@roku.com" , "szeder.dev@gmail.com" , "ahippo@yandex.com" , "gitster@pobox.com" Subject: [PATCH v2 3/7] git-p4: don't groom exclude path list on every commit Thread-Topic: [PATCH v2 3/7] git-p4: don't groom exclude path list on every commit Thread-Index: AQHU4DYI1vGn3yFeRk27bGYJZI+79A== Date: Thu, 21 Mar 2019 22:32:57 +0000 Message-ID: <44fed954dc4ee7d98ce518c0665cc71a0751dd3b.1553207234.git.amazo@checkvideo.com> References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BN3PR05CA0023.namprd05.prod.outlook.com (2603:10b6:400::33) To DM6PR08MB4956.namprd08.prod.outlook.com (2603:10b6:5:4b::25) authentication-results: spf=none (sender IP is ) smtp.mailfrom=amazo@checkvideo.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.19.2 x-originating-ip: [70.163.25.109] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 72c1c5da-f34d-4dd0-4cfe-08d6ae4d2ab1 x-microsoft-antispam: BCL:0;PCL:0;RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600127)(711020)(4605104)(2017052603328)(7153060)(7193020);SRVR:DM6PR08MB4251; x-ms-traffictypediagnostic: DM6PR08MB4251: x-microsoft-antispam-prvs: x-forefront-prvs: 0983EAD6B2 x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(366004)(376002)(346002)(39860400002)(136003)(396003)(189003)(199004)(14444005)(6436002)(5640700003)(53936002)(36756003)(76176011)(386003)(486006)(446003)(6506007)(8676002)(68736007)(2501003)(97736004)(305945005)(256004)(25786009)(186003)(478600001)(476003)(86362001)(6512007)(71200400001)(4326008)(7416002)(6916009)(2616005)(11346002)(6486002)(102836004)(118296001)(3846002)(6116002)(26005)(7736002)(66066001)(50226002)(316002)(99286004)(14454004)(54906003)(2906002)(106356001)(81156014)(5660300002)(52116002)(71190400001)(1730700003)(81166006)(2351001)(8936002)(105586002);DIR:OUT;SFP:1102;SCL:1;SRVR:DM6PR08MB4251;H:DM6PR08MB4956.namprd08.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;MX:1;A:1; received-spf: None (protection.outlook.com: checkvideo.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: ZjeSYiCMk52cOxiSdvUt4e9IHbGxq+j5ZOYhXWxedZL0CRkC0Ib6vLwOAEa20XaWZjZSC0lQSSLuwLH6Q8G7+fevcRPLz2Aho3HoB0Vs4n3Hyv2PIyAQfPKG/nF71DZENoZoG4407ViRs/XxFw/h7WJOHg2qrB5xyyv7P5gdbDjJ4AjRDBZZ3UF3JuNIh7hXHGWUsUQRdlR+tVBSOzSfQjtNggxDaqMVarqTniui4ef1sGMda4FRUv8+6zz/GyBDxSfl4titUreySIzulsuHiJCvQKT1M+F8uXPra25h8WajXl/Ql5X/JS3S3COk8XZvRpab5HzeV+ngZ/tXVWUyLA96z+GtaK18t9TKr4/81x67uLAwFGtL/IGA6iQYhgPU/b1JTJep1FmFz7t+U0cTah6tx4wdIsZH9DxFRsdTJYU= MIME-Version: 1.0 X-OriginatorOrg: checkvideo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 72c1c5da-f34d-4dd0-4cfe-08d6ae4d2ab1 X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Mar 2019 22:32:57.5764 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 7280061d-06ed-4a4e-a2b1-cc9ab5638c09 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR08MB4251 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Currently, `cloneExclude` array is being groomed (by removing trailing "...") on every changeset. (since `extractFilesFromCommit()` is called on every imported changeset) As a micro-optimization, do it once while parsing arguments. Also, prepend "/" and remove trailing "..." at the same time. Signed-off-by: Andrey Mazo --- git-p4.py | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/git-p4.py b/git-p4.py index 91c610f960..a9f53e5b88 100755 --- a/git-p4.py +++ b/git-p4.py @@ -1314,11 +1314,11 @@ class Command: def __init__(self): self.usage = "usage: %prog [options]" self.needsGit = True self.verbose = False - # This is required for the "append" cloneExclude action + # This is required for the "append" update_shelve action def ensure_value(self, attr, value): if not hasattr(self, attr) or getattr(self, attr) is None: setattr(self, attr, value) return getattr(self, attr) @@ -2528,10 +2528,15 @@ def map_in_client(self, depot_path): return self.client_spec_path_cache[depot_path] die( "Error: %s is not found in client spec path" % depot_path ) return "" +def cloneExcludeCallback(option, opt_str, value, parser): + # prepend "/" because the first "/" was consumed as part of the option itself. + # ("-//depot/A/..." becomes "/depot/A/..." after option parsing) + parser.values.cloneExclude += ["/" + re.sub(r"\.\.\.$", "", value)] + class P4Sync(Command, P4UserMap): def __init__(self): Command.__init__(self) P4UserMap.__init__(self) @@ -2551,11 +2556,11 @@ def __init__(self): optparse.make_option("--keep-path", dest="keepRepoPath", action='store_true', help="Keep entire BRANCH/DIR/SUBDIR prefix during import"), optparse.make_option("--use-client-spec", dest="useClientSpec", action='store_true', help="Only sync files that are included in the Perforce Client Spec"), optparse.make_option("-/", dest="cloneExclude", - action="append", type="string", + action="callback", callback=cloneExcludeCallback, type="string", help="exclude depot path"), ] self.description = """Imports from Perforce into a git repository.\n example: //depot/my/project/ -- to import the current head @@ -2617,12 +2622,10 @@ def checkpoint(self): out = self.gitOutput.readline() if self.verbose: print("checkpoint finished: " + out) def extractFilesFromCommit(self, commit, shelved=False, shelved_cl = 0): - self.cloneExclude = [re.sub(r"\.\.\.$", "", path) - for path in self.cloneExclude] files = [] fnum = 0 while "depotFile%s" % fnum in commit: path = commit["depotFile%s" % fnum] @@ -3888,11 +3891,10 @@ def run(self, args): if not self.cloneDestination and len(depotPaths) > 1: self.cloneDestination = depotPaths[-1] depotPaths = depotPaths[:-1] - self.cloneExclude = ["/"+p for p in self.cloneExclude] for p in depotPaths: if not p.startswith("//"): sys.stderr.write('Depot paths must start with "//": %s\n' % p) return False From patchwork Thu Mar 21 22:33:00 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Mazo, Andrey" X-Patchwork-Id: 10864457 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 22A6F1390 for ; Thu, 21 Mar 2019 22:33:37 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0ACFC2A0B7 for ; Thu, 21 Mar 2019 22:33:37 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id F2B702A530; Thu, 21 Mar 2019 22:33:36 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 898A02A0B7 for ; Thu, 21 Mar 2019 22:33:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727215AbfCUWdf (ORCPT ); Thu, 21 Mar 2019 18:33:35 -0400 Received: from mail-eopbgr720118.outbound.protection.outlook.com ([40.107.72.118]:2850 "EHLO NAM05-CO1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726840AbfCUWde (ORCPT ); Thu, 21 Mar 2019 18:33:34 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kastle.onmicrosoft.com; s=selector1-checkvideo-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=nYS71XeoDaWAZXGpQQV29M+IKlqtdU6YHO8XpRABOsw=; b=IX3lm6BmgU8PhXz+5HAErSYYw47GwiZ2Qj9NhzCDiDailiSQA23yq4eJkf4zsxdX62CttUZvYxP0ExpeGRXIWjiYr36Fe2Flxpb/zhirLfYibHWSKCe/c6h62IYxWxAxCSeBCFtOReK5oQRdXXn5MP+4eBVmWAtwOVCozE+yOCo= Received: from DM6PR08MB4956.namprd08.prod.outlook.com (20.176.115.217) by DM6PR08MB4251.namprd08.prod.outlook.com (20.176.82.12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1730.15; Thu, 21 Mar 2019 22:33:09 +0000 Received: from DM6PR08MB4956.namprd08.prod.outlook.com ([fe80::75e5:faf9:290a:a244]) by DM6PR08MB4956.namprd08.prod.outlook.com ([fe80::75e5:faf9:290a:a244%3]) with mapi id 15.20.1709.015; Thu, 21 Mar 2019 22:33:09 +0000 From: "Mazo, Andrey" To: "git@vger.kernel.org" CC: "Mazo, Andrey" , "luke@diamand.org" , "sunshine@sunshineco.com" , "gvanburgh@bloomberg.net" , "larsxschneider@gmail.com" , "miguel.torroja@gmail.com" , "merlorom@yahoo.fr" , "vitor.hda@gmail.com" , "aoakley@roku.com" , "szeder.dev@gmail.com" , "ahippo@yandex.com" , "gitster@pobox.com" Subject: [PATCH v2 4/7] git-p4: add failing test for "don't exclude other files with same prefix" Thread-Topic: [PATCH v2 4/7] git-p4: add failing test for "don't exclude other files with same prefix" Thread-Index: AQHU4DYKa0qqowQcEUueNdbhPU+sRw== Date: Thu, 21 Mar 2019 22:33:00 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BN3PR05CA0023.namprd05.prod.outlook.com (2603:10b6:400::33) To DM6PR08MB4956.namprd08.prod.outlook.com (2603:10b6:5:4b::25) authentication-results: spf=none (sender IP is ) smtp.mailfrom=amazo@checkvideo.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.19.2 x-originating-ip: [70.163.25.109] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 0b7ce9d4-4b41-41c3-ad4b-08d6ae4d2c69 x-microsoft-antispam: BCL:0;PCL:0;RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600127)(711020)(4605104)(2017052603328)(7153060)(7193020);SRVR:DM6PR08MB4251; x-ms-traffictypediagnostic: DM6PR08MB4251: x-microsoft-antispam-prvs: x-forefront-prvs: 0983EAD6B2 x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(366004)(376002)(346002)(39860400002)(136003)(396003)(189003)(199004)(14444005)(6666004)(6436002)(5640700003)(53936002)(36756003)(76176011)(386003)(486006)(446003)(6506007)(8676002)(68736007)(2501003)(97736004)(305945005)(256004)(25786009)(186003)(478600001)(476003)(86362001)(6512007)(71200400001)(4326008)(7416002)(6916009)(2616005)(11346002)(6486002)(102836004)(118296001)(3846002)(6116002)(26005)(7736002)(66066001)(50226002)(316002)(99286004)(14454004)(54906003)(2906002)(106356001)(81156014)(5660300002)(52116002)(71190400001)(1730700003)(81166006)(2351001)(8936002)(105586002);DIR:OUT;SFP:1102;SCL:1;SRVR:DM6PR08MB4251;H:DM6PR08MB4956.namprd08.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;MX:1;A:1; received-spf: None (protection.outlook.com: checkvideo.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: Dhw66Jkk3hQlpMxJN0Uwr5KWDvjMa5ey1XtWY1HNC3bgpNRr5sGo0l1UnZF2zgWK+JV1EDU7EoZC5kV/sUpt+xRHk7rz8Vtl0uSCen6M6wg3l79wbtwcYouC1HhyxSj9t2OI1weUhW82FrcS66uFuG3nAHw+GactwqW6bdY4xA/wCrygj5o2uzfYCzit94AgkJkuzBKXhHJSu/9jFxphlulQ3iZ4nA5RJTcusXhiKBMz4fZ9FM5WBmXiRgi5p/KTd3zYsgNtG+TL4sYUlN2d3qwc7aGHzEWfk129w8gW5O1Xd6zgiemtGzeCVZ4FsGojYtbUTHR4wGGQmSN2UfouVwAnG863875f/YLYK/DmalwvVj0lxlvR5Q6OA5pkPjGWsBOKRPm6QYWhGAySDTyWxSuTyIBYFKh+0JttUiSF4ec= MIME-Version: 1.0 X-OriginatorOrg: checkvideo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0b7ce9d4-4b41-41c3-ad4b-08d6ae4d2c69 X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Mar 2019 22:33:00.2823 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 7280061d-06ed-4a4e-a2b1-cc9ab5638c09 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR08MB4251 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP In preparation for a fix, add a failing test case to test that git-p4 doesn't exclude files with the same prefix unintentionally when exclude paths are specified without a trailing /. I.e., don't exclude "//depot/file_dont_exclude" if run with "-//depot/file". or don't exclude "//depot/discard_file_not" if run with "-//depot/discard_file". Signed-off-by: Andrey Mazo --- t/t9817-git-p4-exclude.sh | 51 +++++++++++++++++++++++++++++++++++---- 1 file changed, 46 insertions(+), 5 deletions(-) diff --git a/t/t9817-git-p4-exclude.sh b/t/t9817-git-p4-exclude.sh index aac568eadf..1c22570797 100755 --- a/t/t9817-git-p4-exclude.sh +++ b/t/t9817-git-p4-exclude.sh @@ -20,49 +20,90 @@ test_expect_success 'create exclude repo' ' ( cd "$cli" && mkdir -p wanted discard && echo wanted >wanted/foo && echo discard >discard/foo && - p4 add wanted/foo discard/foo && + echo discard_file >discard_file && + echo discard_file_not >discard_file_not && + p4 add wanted/foo discard/foo discard_file discard_file_not && p4 submit -d "initial revision" ) ' test_expect_success 'check the repo was created correctly' ' test_when_finished cleanup_git && git p4 clone --dest="$git" //depot/...@all && ( cd "$git" && test_path_is_file wanted/foo && - test_path_is_file discard/foo + test_path_is_file discard/foo && + test_path_is_file discard_file && + test_path_is_file discard_file_not ) ' test_expect_success 'clone, excluding part of repo' ' test_when_finished cleanup_git && git p4 clone -//depot/discard/... --dest="$git" //depot/...@all && ( cd "$git" && test_path_is_file wanted/foo && - test_path_is_missing discard/foo + test_path_is_missing discard/foo && + test_path_is_file discard_file && + test_path_is_file discard_file_not + ) +' + +test_expect_failure 'clone, excluding single file, no trailing /' ' + test_when_finished cleanup_git && + git p4 clone -//depot/discard_file --dest="$git" //depot/...@all && + ( + cd "$git" && + test_path_is_file wanted/foo && + test_path_is_file discard/foo && + test_path_is_missing discard_file && + test_path_is_file discard_file_not ) ' test_expect_success 'clone, then sync with exclude' ' test_when_finished cleanup_git && git p4 clone -//depot/discard/... --dest="$git" //depot/...@all && ( cd "$cli" && - p4 edit wanted/foo discard/foo && + p4 edit wanted/foo discard/foo discard_file_not && date >>wanted/foo && date >>discard/foo && + date >>discard_file_not && p4 submit -d "updating" && cd "$git" && git p4 sync -//depot/discard/... && test_path_is_file wanted/foo && - test_path_is_missing discard/foo + test_path_is_missing discard/foo && + test_path_is_file discard_file && + test_path_is_file discard_file_not + ) +' + +test_expect_failure 'clone, then sync with exclude, no trailing /' ' + test_when_finished cleanup_git && + git p4 clone -//depot/discard/... -//depot/discard_file --dest="$git" //depot/...@all && + ( + cd "$cli" && + p4 edit wanted/foo discard/foo discard_file_not && + date >>wanted/foo && + date >>discard/foo && + date >>discard_file_not && + p4 submit -d "updating" && + + cd "$git" && + git p4 sync -//depot/discard/... -//depot/discard_file && + test_path_is_file wanted/foo && + test_path_is_missing discard/foo && + test_path_is_missing discard_file && + test_path_is_file discard_file_not ) ' test_expect_success 'kill p4d' ' kill_p4d From patchwork Thu Mar 21 22:33:02 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Mazo, Andrey" X-Patchwork-Id: 10864459 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2321F1390 for ; Thu, 21 Mar 2019 22:33:42 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 090EF2A0B7 for ; Thu, 21 Mar 2019 22:33:42 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id EE7FB2A530; Thu, 21 Mar 2019 22:33:41 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1154A2A0B7 for ; Thu, 21 Mar 2019 22:33:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727234AbfCUWdi (ORCPT ); Thu, 21 Mar 2019 18:33:38 -0400 Received: from mail-eopbgr720118.outbound.protection.outlook.com ([40.107.72.118]:2850 "EHLO NAM05-CO1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726840AbfCUWdi (ORCPT ); Thu, 21 Mar 2019 18:33:38 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kastle.onmicrosoft.com; s=selector1-checkvideo-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=CHqrEVUDqwRM9G4TICO4+0L6bjzLhdTt6paW6CviqPM=; b=v7iOvstSldqngfb1imCRmk3Z20MFxId2ydJ2BL7Pl3AyqXgSSaUjztMh/qitoHobJFTWbkdGBDUVchodCW4goAdmJ298NKBAf5KpyoWLxZDXDvKpPaerzSG8v5/qpSb8bfyZFBsGj5J8JgwciKVySKB/MvvsRXEI51CP0TDlaBM= Received: from DM6PR08MB4956.namprd08.prod.outlook.com (20.176.115.217) by DM6PR08MB4251.namprd08.prod.outlook.com (20.176.82.12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1730.15; Thu, 21 Mar 2019 22:33:10 +0000 Received: from DM6PR08MB4956.namprd08.prod.outlook.com ([fe80::75e5:faf9:290a:a244]) by DM6PR08MB4956.namprd08.prod.outlook.com ([fe80::75e5:faf9:290a:a244%3]) with mapi id 15.20.1709.015; Thu, 21 Mar 2019 22:33:10 +0000 From: "Mazo, Andrey" To: "git@vger.kernel.org" CC: "Mazo, Andrey" , "luke@diamand.org" , "sunshine@sunshineco.com" , "gvanburgh@bloomberg.net" , "larsxschneider@gmail.com" , "miguel.torroja@gmail.com" , "merlorom@yahoo.fr" , "vitor.hda@gmail.com" , "aoakley@roku.com" , "szeder.dev@gmail.com" , "ahippo@yandex.com" , "gitster@pobox.com" Subject: [PATCH v2 5/7] git-p4: don't exclude other files with same prefix Thread-Topic: [PATCH v2 5/7] git-p4: don't exclude other files with same prefix Thread-Index: AQHU4DYL82yhg3sxak2s9yXpzDSfVQ== Date: Thu, 21 Mar 2019 22:33:02 +0000 Message-ID: <3330f88a0d1ccd8aa1a376ee8c543690ac983958.1553207234.git.amazo@checkvideo.com> References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BN3PR05CA0023.namprd05.prod.outlook.com (2603:10b6:400::33) To DM6PR08MB4956.namprd08.prod.outlook.com (2603:10b6:5:4b::25) authentication-results: spf=none (sender IP is ) smtp.mailfrom=amazo@checkvideo.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.19.2 x-originating-ip: [70.163.25.109] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 71849c06-7e67-4cf8-a90a-08d6ae4d2ddb x-microsoft-antispam: BCL:0;PCL:0;RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600127)(711020)(4605104)(2017052603328)(7153060)(7193020);SRVR:DM6PR08MB4251; x-ms-traffictypediagnostic: DM6PR08MB4251: x-microsoft-antispam-prvs: x-forefront-prvs: 0983EAD6B2 x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(366004)(376002)(346002)(39860400002)(136003)(396003)(189003)(199004)(14444005)(6666004)(6436002)(5640700003)(53936002)(36756003)(76176011)(386003)(486006)(446003)(6506007)(8676002)(68736007)(2501003)(97736004)(305945005)(256004)(25786009)(186003)(478600001)(476003)(86362001)(6512007)(71200400001)(4326008)(7416002)(6916009)(2616005)(11346002)(6486002)(102836004)(118296001)(3846002)(6116002)(26005)(7736002)(66066001)(50226002)(316002)(99286004)(14454004)(54906003)(2906002)(106356001)(81156014)(5660300002)(52116002)(71190400001)(1730700003)(81166006)(2351001)(8936002)(105586002);DIR:OUT;SFP:1102;SCL:1;SRVR:DM6PR08MB4251;H:DM6PR08MB4956.namprd08.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;MX:1;A:1; received-spf: None (protection.outlook.com: checkvideo.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: s3Fo1w6EUZmjVpV6ftW8S4G9T5oDFOyI64XQdhubc7eKSaWAgWjem0U35X9Rn7F0Aa5efu+OhaawVw6w3amSGtNeSfgV0M/BclE079OhDiKf5TIGvjMurSRiE0oY1gSi211R05GOh0rG71F+kn3TZDWUSA5d6kNC7mqfg8GDricLmDCYUgIaaPZ9dryq8u6msWcrueo5wxELTpxgrOoZYS/IGnEXyhePO6B5hnQCPuugW6fpJyoYgpeyQI5Ygfj/jDPQyqgio6SFJCeUYM9YhNnjtHvAC/1WIFsgMUv8HJzV9jdC9of04gy2GAkPglxxuwPOTCVQgVh4QLh8em1UaAIEZQdMzTGzXITpXaE/XXKNC42nk8C7ErH7cX5+CmzLn3rgzhntcBNT4fhFo1UCrMxFKo7G5GXusX2gf2QaoUg= MIME-Version: 1.0 X-OriginatorOrg: checkvideo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 71849c06-7e67-4cf8-a90a-08d6ae4d2ddb X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Mar 2019 22:33:02.7070 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 7280061d-06ed-4a4e-a2b1-cc9ab5638c09 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR08MB4251 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Make sure not to exclude files unintentionally if exclude paths are specified without a trailing /. I.e., don't exclude "//depot/file_dont_exclude" if run with "-//depot/file". Do this by ensuring that paths without a trailing "/" are only matched completely. Also, abort path search on the first match as a micro-optimization. Signed-off-by: Andrey Mazo --- git-p4.py | 21 ++++++++++++++------- t/t9817-git-p4-exclude.sh | 4 ++-- 2 files changed, 16 insertions(+), 9 deletions(-) diff --git a/git-p4.py b/git-p4.py index a9f53e5b88..162877aa82 100755 --- a/git-p4.py +++ b/git-p4.py @@ -2621,22 +2621,29 @@ def checkpoint(self): self.gitStream.write("progress checkpoint\n\n") out = self.gitOutput.readline() if self.verbose: print("checkpoint finished: " + out) + def isPathWanted(self, path): + for p in self.cloneExclude: + if p.endswith("/"): + if p4PathStartsWith(path, p): + return False + # "-//depot/file1" without a trailing "/" should only exclude "file1", but not "file111" or "file1_dir/file2" + elif path.lower() == p.lower(): + return False + for p in self.depotPaths: + if p4PathStartsWith(path, p): + return True + return False + def extractFilesFromCommit(self, commit, shelved=False, shelved_cl = 0): files = [] fnum = 0 while "depotFile%s" % fnum in commit: path = commit["depotFile%s" % fnum] - - if [p for p in self.cloneExclude - if p4PathStartsWith(path, p)]: - found = False - else: - found = [p for p in self.depotPaths - if p4PathStartsWith(path, p)] + found = self.isPathWanted(path) if not found: fnum = fnum + 1 continue file = {} diff --git a/t/t9817-git-p4-exclude.sh b/t/t9817-git-p4-exclude.sh index 1c22570797..275dd30425 100755 --- a/t/t9817-git-p4-exclude.sh +++ b/t/t9817-git-p4-exclude.sh @@ -51,11 +51,11 @@ test_expect_success 'clone, excluding part of repo' ' test_path_is_file discard_file && test_path_is_file discard_file_not ) ' -test_expect_failure 'clone, excluding single file, no trailing /' ' +test_expect_success 'clone, excluding single file, no trailing /' ' test_when_finished cleanup_git && git p4 clone -//depot/discard_file --dest="$git" //depot/...@all && ( cd "$git" && test_path_is_file wanted/foo && @@ -83,11 +83,11 @@ test_expect_success 'clone, then sync with exclude' ' test_path_is_file discard_file && test_path_is_file discard_file_not ) ' -test_expect_failure 'clone, then sync with exclude, no trailing /' ' +test_expect_success 'clone, then sync with exclude, no trailing /' ' test_when_finished cleanup_git && git p4 clone -//depot/discard/... -//depot/discard_file --dest="$git" //depot/...@all && ( cd "$cli" && p4 edit wanted/foo discard/foo discard_file_not && From patchwork Thu Mar 21 22:33:05 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Mazo, Andrey" X-Patchwork-Id: 10864461 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0C2B9922 for ; Thu, 21 Mar 2019 22:33:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E742F2A0B7 for ; Thu, 21 Mar 2019 22:33:42 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DAEE82A530; Thu, 21 Mar 2019 22:33:42 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5CB612A0B7 for ; Thu, 21 Mar 2019 22:33:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727248AbfCUWdl (ORCPT ); Thu, 21 Mar 2019 18:33:41 -0400 Received: from mail-eopbgr720118.outbound.protection.outlook.com ([40.107.72.118]:2850 "EHLO NAM05-CO1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726840AbfCUWdl (ORCPT ); Thu, 21 Mar 2019 18:33:41 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kastle.onmicrosoft.com; s=selector1-checkvideo-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=CG6mGwI5K7t/lJrjTYG8/zk2pt4H62bpYTVKoEhwMto=; b=JPbaOxBvk1knM9kQktLnhDT/x7LpI72LB6HAqLkEBaQNtf4F888hGUVnW9J0Cy0B81P8m+TcxuTzFlX1hfefa15FFNEDc8B0yAjGmi1YcLo2ZkFv8rMyS0u/WrAxpzXQimEkdT3Sg9zMVCYvl6CJO5Er5aAzZhwFlKgVISJsqpM= Received: from DM6PR08MB4956.namprd08.prod.outlook.com (20.176.115.217) by DM6PR08MB4251.namprd08.prod.outlook.com (20.176.82.12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1730.15; Thu, 21 Mar 2019 22:33:10 +0000 Received: from DM6PR08MB4956.namprd08.prod.outlook.com ([fe80::75e5:faf9:290a:a244]) by DM6PR08MB4956.namprd08.prod.outlook.com ([fe80::75e5:faf9:290a:a244%3]) with mapi id 15.20.1709.015; Thu, 21 Mar 2019 22:33:10 +0000 From: "Mazo, Andrey" To: "git@vger.kernel.org" CC: "Mazo, Andrey" , "luke@diamand.org" , "sunshine@sunshineco.com" , "gvanburgh@bloomberg.net" , "larsxschneider@gmail.com" , "miguel.torroja@gmail.com" , "merlorom@yahoo.fr" , "vitor.hda@gmail.com" , "aoakley@roku.com" , "szeder.dev@gmail.com" , "ahippo@yandex.com" , "gitster@pobox.com" Subject: [PATCH v2 6/7] git-p4: add failing test for "git-p4: respect excluded paths when detecting branches" Thread-Topic: [PATCH v2 6/7] git-p4: add failing test for "git-p4: respect excluded paths when detecting branches" Thread-Index: AQHU4DYM86QPjCA0O0qhWIoiyPAfuQ== Date: Thu, 21 Mar 2019 22:33:05 +0000 Message-ID: <6170d45951d71171ea3ad502a3b2a5c5c55c12f8.1553207234.git.amazo@checkvideo.com> References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BN3PR05CA0023.namprd05.prod.outlook.com (2603:10b6:400::33) To DM6PR08MB4956.namprd08.prod.outlook.com (2603:10b6:5:4b::25) authentication-results: spf=none (sender IP is ) smtp.mailfrom=amazo@checkvideo.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.19.2 x-originating-ip: [70.163.25.109] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 1e65a0e1-8196-4bd2-a0eb-08d6ae4d2f4f x-microsoft-antispam: BCL:0;PCL:0;RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600127)(711020)(4605104)(2017052603328)(7153060)(7193020);SRVR:DM6PR08MB4251; x-ms-traffictypediagnostic: DM6PR08MB4251: x-ms-exchange-purlcount: 1 x-microsoft-antispam-prvs: x-forefront-prvs: 0983EAD6B2 x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(366004)(376002)(346002)(39860400002)(136003)(396003)(189003)(199004)(14444005)(6666004)(6436002)(5640700003)(53936002)(53376002)(36756003)(76176011)(386003)(486006)(446003)(6506007)(6306002)(8676002)(68736007)(2501003)(97736004)(305945005)(256004)(25786009)(186003)(478600001)(476003)(86362001)(6512007)(966005)(71200400001)(4326008)(7416002)(6916009)(2616005)(11346002)(6486002)(102836004)(118296001)(3846002)(6116002)(26005)(7736002)(66066001)(50226002)(316002)(99286004)(14454004)(54906003)(2906002)(106356001)(81156014)(5660300002)(52116002)(71190400001)(1730700003)(81166006)(2351001)(8936002)(105586002);DIR:OUT;SFP:1102;SCL:1;SRVR:DM6PR08MB4251;H:DM6PR08MB4956.namprd08.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;MX:1;A:1; received-spf: None (protection.outlook.com: checkvideo.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: +dukweoKO5nwfDu9GwitahAGLPUXigez3gc/rSoUKxZjoHz+chjLbnqgpUlHCv8Mx2TUwxEzd3P/0NghUbvYHvoPGwsgHXn9EMwjaOELLACVu1tFHUC+Nfggii4qi2spASmGaSvbwp6FWT7qheNrNztZQSB1iHuw1WuatekGjPYnNzQT9uqDIeV4Fln5TyHFWZWkYTxlPN5OyA5aOIl44zNL9/11IkHIq6FqZvJK8gcZX/AvlwH6lyEkQgcOAZ/pFiHsMcALeuGahW4w+xycTFGcTjzdQ0fOvRD+U2/vQd5fBSwdrfWZXTTviQE98TkcmaFSS7n2XVG/RD22infQ79QBrb4/R95SEIg6Zt01jH3fwfqM/VaVZyxLS1Puz8MY4cSpIJzSJ67WFDd5/T1ANPgKYPQx7BLsca3nwYeAPMg= MIME-Version: 1.0 X-OriginatorOrg: checkvideo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1e65a0e1-8196-4bd2-a0eb-08d6ae4d2f4f X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Mar 2019 22:33:05.1278 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 7280061d-06ed-4a4e-a2b1-cc9ab5638c09 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR08MB4251 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP In preparation for a fix, add a failing test case to test that git-p4 doesn't exclude files despite being told to when handling multiple branches. I.e., it should exclude //depot/branch2/file2 when run with -//depot/branch2/file2, but doesn't do this right now. The test is based on 'git p4 clone complex branches' test with the following changes: * account for file3 moved from branch3 to branch4 in test 'git p4 submit to two branches in a single changelist'; * account for branch6 created in test 'git p4 clone file subset branch'; * file2 is expected to be missing from all branches due to explicit exclude. Signed-off-by: Andrey Mazo --- t/t9801-git-p4-branch.sh | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/t/t9801-git-p4-branch.sh b/t/t9801-git-p4-branch.sh index 6a86d6996b..4729f470b2 100755 --- a/t/t9801-git-p4-branch.sh +++ b/t/t9801-git-p4-branch.sh @@ -409,10 +409,50 @@ test_expect_failure 'git p4 clone file subset branch' ' test_path_is_missing file2 && test_path_is_missing file3 ) ' +# Check that excluded files are omitted during import +test_expect_failure 'git p4 clone complex branches with excluded files' ' + test_when_finished cleanup_git && + test_create_repo "$git" && + ( + cd "$git" && + git config git-p4.branchList branch1:branch2 && + git config --add git-p4.branchList branch1:branch3 && + git config --add git-p4.branchList branch1:branch4 && + git config --add git-p4.branchList branch1:branch5 && + git config --add git-p4.branchList branch1:branch6 && + git p4 clone --dest=. --detect-branches -//depot/branch1/file2 -//depot/branch2/file2 -//depot/branch3/file2 -//depot/branch4/file2 -//depot/branch5/file2 -//depot/branch6/file2 //depot@all && + git log --all --graph --decorate --stat && + git reset --hard p4/depot/branch1 && + test_path_is_file file1 && + test_path_is_missing file2 && + test_path_is_file file3 && + git reset --hard p4/depot/branch2 && + test_path_is_file file1 && + test_path_is_missing file2 && + test_path_is_missing file3 && + git reset --hard p4/depot/branch3 && + test_path_is_file file1 && + test_path_is_missing file2 && + test_path_is_missing file3 && + git reset --hard p4/depot/branch4 && + test_path_is_file file1 && + test_path_is_missing file2 && + test_path_is_file file3 && + git reset --hard p4/depot/branch5 && + test_path_is_file file1 && + test_path_is_missing file2 && + test_path_is_file file3 && + git reset --hard p4/depot/branch6 && + test_path_is_file file1 && + test_path_is_missing file2 && + test_path_is_missing file3 + ) +' + # From a report in http://stackoverflow.com/questions/11893688 # where --use-client-spec caused branch prefixes not to be removed; # every file in git appeared into a subdirectory of the branch name. test_expect_success 'use-client-spec detect-branches setup' ' rm -rf "$cli" && From patchwork Thu Mar 21 22:33:07 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Mazo, Andrey" X-Patchwork-Id: 10864463 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BCDF8922 for ; Thu, 21 Mar 2019 22:33:44 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A3D662A0B7 for ; Thu, 21 Mar 2019 22:33:44 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 97F3E2A530; Thu, 21 Mar 2019 22:33:44 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3517A2A0B7 for ; Thu, 21 Mar 2019 22:33:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727257AbfCUWdn (ORCPT ); Thu, 21 Mar 2019 18:33:43 -0400 Received: from mail-eopbgr720118.outbound.protection.outlook.com ([40.107.72.118]:2850 "EHLO NAM05-CO1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727245AbfCUWdm (ORCPT ); Thu, 21 Mar 2019 18:33:42 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kastle.onmicrosoft.com; s=selector1-checkvideo-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=qafieZikvoznm5Z6VSOif/gKBU3KaMTwQzo7HdAT8Ks=; b=oyqn7Y957KWdII3VxlicdA1SPwMSEQL3OqS9Z2xKmdRqzCbOUF/6k5QO4f32b0IEtCCWEJorXowPh7sWU8yniY935GuChhVsPdBZ3j2Pkb2R8a/DJVWzrpXp6lRmqhsBBQxfl9N5Iiu771VpVmSPRX+YYhGOE2qBgWBfGpSNk60= Received: from DM6PR08MB4956.namprd08.prod.outlook.com (20.176.115.217) by DM6PR08MB4251.namprd08.prod.outlook.com (20.176.82.12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1730.15; Thu, 21 Mar 2019 22:33:11 +0000 Received: from DM6PR08MB4956.namprd08.prod.outlook.com ([fe80::75e5:faf9:290a:a244]) by DM6PR08MB4956.namprd08.prod.outlook.com ([fe80::75e5:faf9:290a:a244%3]) with mapi id 15.20.1709.015; Thu, 21 Mar 2019 22:33:11 +0000 From: "Mazo, Andrey" To: "git@vger.kernel.org" CC: "Mazo, Andrey" , "luke@diamand.org" , "sunshine@sunshineco.com" , "gvanburgh@bloomberg.net" , "larsxschneider@gmail.com" , "miguel.torroja@gmail.com" , "merlorom@yahoo.fr" , "vitor.hda@gmail.com" , "aoakley@roku.com" , "szeder.dev@gmail.com" , "ahippo@yandex.com" , "gitster@pobox.com" Subject: [PATCH v2 7/7] git-p4: respect excluded paths when detecting branches Thread-Topic: [PATCH v2 7/7] git-p4: respect excluded paths when detecting branches Thread-Index: AQHU4DYOWQvMGicNzEKZgMjiTfLXGA== Date: Thu, 21 Mar 2019 22:33:07 +0000 Message-ID: <758d8e84868edc4b53c382a74655048d69d187de.1553207234.git.amazo@checkvideo.com> References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BN3PR05CA0023.namprd05.prod.outlook.com (2603:10b6:400::33) To DM6PR08MB4956.namprd08.prod.outlook.com (2603:10b6:5:4b::25) authentication-results: spf=none (sender IP is ) smtp.mailfrom=amazo@checkvideo.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.19.2 x-originating-ip: [70.163.25.109] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: e6853f18-cb22-4383-28bc-08d6ae4d3083 x-microsoft-antispam: BCL:0;PCL:0;RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600127)(711020)(4605104)(2017052603328)(7153060)(7193020);SRVR:DM6PR08MB4251; x-ms-traffictypediagnostic: DM6PR08MB4251: x-microsoft-antispam-prvs: x-forefront-prvs: 0983EAD6B2 x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(366004)(376002)(346002)(39860400002)(136003)(396003)(189003)(199004)(14444005)(6666004)(6436002)(5640700003)(53936002)(36756003)(76176011)(386003)(486006)(446003)(6506007)(8676002)(68736007)(2501003)(97736004)(305945005)(256004)(25786009)(186003)(478600001)(476003)(86362001)(6512007)(71200400001)(4326008)(7416002)(6916009)(2616005)(11346002)(6486002)(102836004)(118296001)(3846002)(6116002)(26005)(7736002)(66066001)(50226002)(316002)(99286004)(14454004)(54906003)(2906002)(106356001)(81156014)(5660300002)(52116002)(71190400001)(1730700003)(81166006)(2351001)(8936002)(105586002);DIR:OUT;SFP:1102;SCL:1;SRVR:DM6PR08MB4251;H:DM6PR08MB4956.namprd08.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;MX:1;A:1; received-spf: None (protection.outlook.com: checkvideo.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: 82jwscr6OVVdrBEF26cEGTW8+8tQS1JuPMTzk28fPSN9oDxG5oC6D0JpPA19zbPGjkCayloQjc8D0BHLjw1QTc4uye4ErTTeKvxfoPh+IQKfLE8stvtOBsZUAwro27CQwexnGHLRqBPp8yjxZkRcnkHNuhXgKPENHosULNz+M6Enb/MoW/TzqjzgJ4iwd8WKEP7yz23FMT4IYjkg6Y6bAuQ87GbKSYT6Su2xdqlIFD4Fz/nczklvp0kpRh12WaBdlxNz7XzyonGFs/y081/S6Vl3F4Cjm361H8kH/vkw7ZiywGBEQhTFEG/zOo8Zel4BgaeLJOnqcFVQpcyvg5q4LyMM6/fU29y/hdefI4yw1HmKKlz/oD1xXDWSbFUYSQvi1Fpt/prsIofGm1VAAYRxpsuUmHpVx4d5VKGB12goPeo= MIME-Version: 1.0 X-OriginatorOrg: checkvideo.com X-MS-Exchange-CrossTenant-Network-Message-Id: e6853f18-cb22-4383-28bc-08d6ae4d3083 X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Mar 2019 22:33:07.1562 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 7280061d-06ed-4a4e-a2b1-cc9ab5638c09 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR08MB4251 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Currently, excluded paths are only handled in the following cases: * no branch detection; * branch detection with using clientspec. However, excluded paths are not respected in case of branch detection without using clientspec. Fix this by consulting the list of excluded paths when splitting files across branches. Signed-off-by: Andrey Mazo --- git-p4.py | 3 +-- t/t9801-git-p4-branch.sh | 2 +- 2 files changed, 2 insertions(+), 3 deletions(-) diff --git a/git-p4.py b/git-p4.py index 162877aa82..148ea6f1b0 100755 --- a/git-p4.py +++ b/git-p4.py @@ -2708,12 +2708,11 @@ def splitFilesIntoBranches(self, commit): branches = {} fnum = 0 while "depotFile%s" % fnum in commit: path = commit["depotFile%s" % fnum] - found = [p for p in self.depotPaths - if p4PathStartsWith(path, p)] + found = self.isPathWanted(path) if not found: fnum = fnum + 1 continue file = {} diff --git a/t/t9801-git-p4-branch.sh b/t/t9801-git-p4-branch.sh index 4729f470b2..62a3929d8e 100755 --- a/t/t9801-git-p4-branch.sh +++ b/t/t9801-git-p4-branch.sh @@ -410,11 +410,11 @@ test_expect_failure 'git p4 clone file subset branch' ' test_path_is_missing file3 ) ' # Check that excluded files are omitted during import -test_expect_failure 'git p4 clone complex branches with excluded files' ' +test_expect_success 'git p4 clone complex branches with excluded files' ' test_when_finished cleanup_git && test_create_repo "$git" && ( cd "$git" && git config git-p4.branchList branch1:branch2 &&