From patchwork Fri Mar 22 23:22:31 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Matheus Tavares X-Patchwork-Id: 10866629 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7A1711669 for ; Fri, 22 Mar 2019 23:22:55 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 608552AA13 for ; Fri, 22 Mar 2019 23:22:55 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 544892AA15; Fri, 22 Mar 2019 23:22:55 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BA2412AA1A for ; Fri, 22 Mar 2019 23:22:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727770AbfCVXWx (ORCPT ); Fri, 22 Mar 2019 19:22:53 -0400 Received: from mail-qk1-f195.google.com ([209.85.222.195]:46870 "EHLO mail-qk1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725982AbfCVXWx (ORCPT ); Fri, 22 Mar 2019 19:22:53 -0400 Received: by mail-qk1-f195.google.com with SMTP id s81so2208017qke.13 for ; Fri, 22 Mar 2019 16:22:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=usp-br.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=v3WXEr/6pVWhCvDNLtoEgQA3zfSOCN0wpPK3KquBOY4=; b=Xy4eCGefArLFkLl5lm0EyNah+0+rbJm99AfdhqGRUudGBavjKndz7LwgLDRH79v37d 8fBVWpbBe1cieQ19VC111q7fQPXAVU5QVjjvcM1DJKoU5R+UtzXkhjsAVPVzW8W36l8C qYMHY2Xds1QQ5oiqo7uLXsewDEkuaSXAzJqSPAhHSMlbrLKEhUSOYZVuDktWgfr2sNaW iLfNDLuYDO3eT38tkX+WTKpsYt5KFW+hoP8BoIdhcnsgq0996eTGFTnRsP5CLXVkrQcQ 0lf2CeTl3AjuekXDT5V0La9tQqfx60VXy0xdBwTEwH5RbctVPEg4jYdupOwteHSd+lln 7hqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=v3WXEr/6pVWhCvDNLtoEgQA3zfSOCN0wpPK3KquBOY4=; b=nsqW9RzL1t5vH11Yx/gQ5ObIw1C4Jcx9vH+SuC+6mXJkSAgBsfotzBytYTqJbq+8qn aFpoGu4cvwgI5R4+5fUT9CkirB7JeyCLkXfvOfVtyF8jWE7zqu3MBIeud3n2uq3CgRsP a2E5umFVbRW2/SCbi5sE0WboCCH99i+2KP+w3h/BRMXc+7Tx8HwYxS1r7UHesXcPktwT dLj6EzB5CW1j3+2UK7lOe29EtnsKx2qkcBjiSb3f7x39Q9t3a5tG1blx6lrOrYqe7VvA 120/ZrxW3pYSExh4Qgdmct1ZbvzRxvVpV/aTISpwsYICQ/7Zys/noFzmbngEeNLhS3R0 WvZA== X-Gm-Message-State: APjAAAWBvJeZ/aZ3iNo5rr9i4+7nau9rjB6bSuvib+augKJVugnJZ+dE bmc1fufDH1HQfezSuaHxXQo9Y2u9cww= X-Google-Smtp-Source: APXvYqyxhsVv0lyVtUJ6kR+wQd6ZokwmlD8rW/jOSO3/Df5x5tdNGV1apyLWZrz2SleWCAHrNRJakQ== X-Received: by 2002:a37:664b:: with SMTP id a72mr10161485qkc.57.1553296971632; Fri, 22 Mar 2019 16:22:51 -0700 (PDT) Received: from mango.spo.virtua.com.br ([2804:14c:81:942d::3]) by smtp.gmail.com with ESMTPSA id i68sm5002765qkf.42.2019.03.22.16.22.48 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 22 Mar 2019 16:22:51 -0700 (PDT) From: Matheus Tavares To: git@vger.kernel.org Cc: Thomas Gummerer , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= , Christian Couder , =?utf-8?b?Tmd1eeG7hW4gVGg=?= =?utf-8?b?w6FpIE5n4buNYyBEdXk=?= , kernel-usp@googlegroups.com, Alex Riesen , Junio C Hamano Subject: [GSoC][PATCH v4 1/7] clone: test for our behavior on odd objects/* content Date: Fri, 22 Mar 2019 20:22:31 -0300 Message-Id: <20190322232237.13293-2-matheus.bernardino@usp.br> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190322232237.13293-1-matheus.bernardino@usp.br> References: <20190226122829.19178-1-avarab@gmail.com> <20190322232237.13293-1-matheus.bernardino@usp.br> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Ævar Arnfjörð Bjarmason Add tests for what happens when we perform a local clone on a repo containing odd files at .git/object directory, such as symlinks to other dirs, or unknown files. I'm bending over backwards here to avoid a SHA1 dependency. See [1] for an earlier and simpler version that hardcoded a SHA-1s. This behavior has been the same for a *long* time, but hasn't been tested for. There's a good post-hoc argument to be made for copying over unknown things, e.g. I'd like a git version that doesn't know about the commit-graph to copy it under "clone --local" so a newer git version can make use of it. In follow-up commits we'll look at changing some of this behavior, but for now let's just assert it as-is so we'll notice what we'll change later. 1. https://public-inbox.org/git/20190226002625.13022-5-avarab@gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason Signed-off-by: Matheus Tavares Helped-by: Matheus Tavares --- t/t5604-clone-reference.sh | 116 +++++++++++++++++++++++++++++++++++++ 1 file changed, 116 insertions(+) diff --git a/t/t5604-clone-reference.sh b/t/t5604-clone-reference.sh index 4320082b1b..708b1a2c66 100755 --- a/t/t5604-clone-reference.sh +++ b/t/t5604-clone-reference.sh @@ -221,4 +221,120 @@ test_expect_success 'clone, dissociate from alternates' ' ( cd C && git fsck ) ' +test_expect_success 'setup repo with garbage in objects/*' ' + git init S && + ( + cd S && + test_commit A && + + cd .git/objects && + >.some-hidden-file && + >some-file && + mkdir .some-hidden-dir && + >.some-hidden-dir/some-file && + >.some-hidden-dir/.some-dot-file && + mkdir some-dir && + >some-dir/some-file && + >some-dir/.some-dot-file + ) +' + +test_expect_success 'clone a repo with garbage in objects/*' ' + for option in --local --no-hardlinks --shared --dissociate + do + git clone $option S S$option || return 1 && + git -C S$option fsck || return 1 + done && + find S-* -name "*some*" | sort >actual && + cat >expected <<-EOF && + S--dissociate/.git/objects/.some-hidden-file + S--dissociate/.git/objects/some-dir + S--dissociate/.git/objects/some-dir/.some-dot-file + S--dissociate/.git/objects/some-dir/some-file + S--dissociate/.git/objects/some-file + S--local/.git/objects/.some-hidden-file + S--local/.git/objects/some-dir + S--local/.git/objects/some-dir/.some-dot-file + S--local/.git/objects/some-dir/some-file + S--local/.git/objects/some-file + S--no-hardlinks/.git/objects/.some-hidden-file + S--no-hardlinks/.git/objects/some-dir + S--no-hardlinks/.git/objects/some-dir/.some-dot-file + S--no-hardlinks/.git/objects/some-dir/some-file + S--no-hardlinks/.git/objects/some-file + EOF + test_cmp expected actual +' + +test_expect_success SYMLINKS 'setup repo with manually symlinked dirs and unknown files at objects/' ' + git init T && + ( + cd T && + test_commit A && + git gc && + ( + cd .git/objects && + mv pack packs && + ln -s packs pack + ) && + test_commit B && + ( + cd .git/objects && + find ?? -type d >loose-dirs && + last_loose=$(tail -n 1 loose-dirs) && + rm -f loose-dirs && + mv $last_loose a-loose-dir && + ln -s a-loose-dir $last_loose && + find . -type f | sort >../../../T.objects-files.raw && + echo unknown_content> unknown_file + ) + ) && + git -C T fsck && + git -C T rev-list --all --objects >T.objects +' + + +test_expect_success SYMLINKS 'clone repo with symlinked dirs and unknown files at objects/' ' + for option in --local --no-hardlinks --shared --dissociate + do + git clone $option T T$option || return 1 && + git -C T$option fsck || return 1 && + git -C T$option rev-list --all --objects >T$option.objects && + test_cmp T.objects T$option.objects && + ( + cd T$option/.git/objects && + find . -type f | sort >../../../T$option.objects-files.raw + ) + done && + + for raw in $(ls T*.raw) + do + sed -e "s!/..\$!/X!; s!/../!/Y/!; s![0-9a-f]\{38,\}!Z!" \ + -e "/multi-pack-index/d" -e "/commit-graph/d" <$raw >$raw.de-sha || return 1 + done && + + cat >expected-files <<-EOF && + ./Y/Z + ./Y/Z + ./a-loose-dir/Z + ./Y/Z + ./info/packs + ./pack/pack-Z.idx + ./pack/pack-Z.pack + ./packs/pack-Z.idx + ./packs/pack-Z.pack + ./unknown_file + EOF + + for option in --local --dissociate --no-hardlinks + do + test_cmp expected-files T$option.objects-files.raw.de-sha || return 1 + done && + + cat >expected-files <<-EOF && + ./info/alternates + EOF + test_cmp expected-files T--shared.objects-files.raw +' + test_done From patchwork Fri Mar 22 23:22:32 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Matheus Tavares X-Patchwork-Id: 10866631 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2A2FF15AC for ; Fri, 22 Mar 2019 23:22:58 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1306F2AA12 for ; Fri, 22 Mar 2019 23:22:58 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 075372AA14; Fri, 22 Mar 2019 23:22:58 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6E0F22AA12 for ; Fri, 22 Mar 2019 23:22:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728219AbfCVXW4 (ORCPT ); Fri, 22 Mar 2019 19:22:56 -0400 Received: from mail-qt1-f194.google.com ([209.85.160.194]:43795 "EHLO mail-qt1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727799AbfCVXW4 (ORCPT ); Fri, 22 Mar 2019 19:22:56 -0400 Received: by mail-qt1-f194.google.com with SMTP id v32so4410309qtc.10 for ; Fri, 22 Mar 2019 16:22:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=usp-br.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=olCYosCGO89jmuFKSaVOQ23QWeJohzXASwr2uhs0YMQ=; b=UEF/vmsLsosXbKEWaCIMO+ayfCmEByfmDceKBl0OofC59pPDrsK11MamZzGbcMa+P2 nQcNWE/njqf9tBOgrZPOA78ESEWP0PoVN2kjz02UCchA52DiCL+yUrLPIAnAiwN4WlUA 0PUf+na0wY3FsCqVfBUgRXZDnEY59UPvtnQ3T3gHy1vI0dZ7RPxQX6byUKJLvf6j2dur 0dO1pfMZFIdQu8bgugHSCa266sEIg9LLDhHlZM/Ty4Y8SHnX9bSY/7eJNbYKqInmOJjt LoxngaZui4gTGm7pajlDaQa4SnhBZNMRqEnNfO1SCUNqQHlI7kSuRis2AxJwIQurXoBR tayw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=olCYosCGO89jmuFKSaVOQ23QWeJohzXASwr2uhs0YMQ=; b=DN5AGG52Ou+3RHNiBSWxWTdYBOhutAVcYG8LbmnZVbPYnWiRcKnxAk2hRiueyNgnWs fxKRZs7qqnW3v7r2yw9/0s65k8EyQc+kfPsO+P5Al1d85BZUvsBqn5LXkF59Z0evisEz UD+xHiUVS9SoZ2G1qKl46mcUkwD12ZJZVuoFBx91FiS3ueebjgzLzSpEwF6/1Tl/lb3j PBYzK1UnaxGRUOBD4+NhOnQmEPT7C3J6cHbqha7b1iMecjSGESoa9ca/LOCI2pJMhv70 J/vMHK3Iob3aPA7l0nPXEg+523H2yaWXhogsuxmUuSydwdhfEOgchKQ+Xno+O3Y3JWl8 ZYPw== X-Gm-Message-State: APjAAAWG5SZVeaec+uyyCbq3gw4Wz0dmarx329z5ZtLsY3wcye13KwGT yQqCGis1WkU3aVXortREFJcJEBGLQPw= X-Google-Smtp-Source: APXvYqzyyiGhcnjddBA0ckvP/IvhsdNX5dDA2UV5ABZM02AS25rdIV2tp0zObZo6yfMrlrP3N2dyGg== X-Received: by 2002:a0c:9528:: with SMTP id l37mr10256644qvl.243.1553296974922; Fri, 22 Mar 2019 16:22:54 -0700 (PDT) Received: from mango.spo.virtua.com.br ([2804:14c:81:942d::3]) by smtp.gmail.com with ESMTPSA id i68sm5002765qkf.42.2019.03.22.16.22.52 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 22 Mar 2019 16:22:54 -0700 (PDT) From: Matheus Tavares To: git@vger.kernel.org Cc: Thomas Gummerer , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= , Christian Couder , =?utf-8?b?Tmd1eeG7hW4gVGg=?= =?utf-8?b?w6FpIE5n4buNYyBEdXk=?= , kernel-usp@googlegroups.com, Benoit Pierre , Junio C Hamano Subject: [GSoC][PATCH v4 2/7] clone: better handle symlinked files at .git/objects/ Date: Fri, 22 Mar 2019 20:22:32 -0300 Message-Id: <20190322232237.13293-3-matheus.bernardino@usp.br> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190322232237.13293-1-matheus.bernardino@usp.br> References: <20190226122829.19178-1-avarab@gmail.com> <20190322232237.13293-1-matheus.bernardino@usp.br> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP There is currently an odd behaviour when locally clonning a repository with symlinks at .git/objects: using --no-hardlinks all symlinks are dereferenced but without it Git will try to hardlink the files with the link() function, which has an OS-specific behaviour on symlinks. On OSX and NetBSD, it creates a hardlink to the file pointed by the symlink whilst on GNU/Linux, it creates a hardlink to the symlink itself. On Manjaro GNU/Linux: $ touch a $ ln -s a b $ link b c $ ls -li a b c 155 [...] a 156 [...] b -> a 156 [...] c -> a But on NetBSD: $ ls -li a b c 2609160 [...] a 2609164 [...] b -> a 2609160 [...] c It's not good to have the result of a local clone to be OS-dependent and since the behaviour on GNU/Linux may result in broken symlinks, let's re-implement it with linkat() instead of link() using a flag to always follow symlinks and make the hardlink be to the pointed file. With this, besides standardizing the behaviour, no broken symlinks will be produced. Also, add tests for symlinked files at .git/objects/. Note: Git won't create symlinks at .git/objects itself, but it's better to handle this case and be friendly with users who manually create them. Signed-off-by: Matheus Tavares Signed-off-by: Ævar Arnfjörð Bjarmason Co-authored-by: Ævar Arnfjörð Bjarmason --- builtin/clone.c | 2 +- t/t5604-clone-reference.sh | 26 +++++++++++++++++++------- 2 files changed, 20 insertions(+), 8 deletions(-) diff --git a/builtin/clone.c b/builtin/clone.c index 50bde99618..b76f33c635 100644 --- a/builtin/clone.c +++ b/builtin/clone.c @@ -443,7 +443,7 @@ static void copy_or_link_directory(struct strbuf *src, struct strbuf *dest, if (unlink(dest->buf) && errno != ENOENT) die_errno(_("failed to unlink '%s'"), dest->buf); if (!option_no_hardlinks) { - if (!link(src->buf, dest->buf)) + if (!linkat(AT_FDCWD, src->buf, AT_FDCWD, dest->buf, AT_SYMLINK_FOLLOW)) continue; if (option_local > 0) die_errno(_("failed to create link '%s'"), dest->buf); diff --git a/t/t5604-clone-reference.sh b/t/t5604-clone-reference.sh index 708b1a2c66..76d45f1187 100755 --- a/t/t5604-clone-reference.sh +++ b/t/t5604-clone-reference.sh @@ -266,7 +266,7 @@ test_expect_success 'clone a repo with garbage in objects/*' ' test_cmp expected actual ' -test_expect_success SYMLINKS 'setup repo with manually symlinked dirs and unknown files at objects/' ' +test_expect_success SYMLINKS 'setup repo with manually symlinked or unknown files at objects/' ' git init T && ( cd T && @@ -282,10 +282,18 @@ test_expect_success SYMLINKS 'setup repo with manually symlinked dirs and unknow cd .git/objects && find ?? -type d >loose-dirs && last_loose=$(tail -n 1 loose-dirs) && - rm -f loose-dirs && mv $last_loose a-loose-dir && ln -s a-loose-dir $last_loose && + first_loose=$(head -n 1 loose-dirs) && + rm -f loose-dirs && + ( + cd $first_loose && + obj=$(ls *) && + mv $obj ../an-object && + ln -s ../an-object $obj + ) && find . -type f | sort >../../../T.objects-files.raw && + find . -type l | sort >../../../T.objects-symlinks.raw && echo unknown_content> unknown_file ) ) && @@ -294,7 +302,7 @@ test_expect_success SYMLINKS 'setup repo with manually symlinked dirs and unknow ' -test_expect_success SYMLINKS 'clone repo with symlinked dirs and unknown files at objects/' ' +test_expect_success SYMLINKS 'clone repo with symlinked or unknown files at objects/' ' for option in --local --no-hardlinks --shared --dissociate do git clone $option T T$option || return 1 && @@ -303,7 +311,8 @@ test_expect_success SYMLINKS 'clone repo with symlinked dirs and unknown files a test_cmp T.objects T$option.objects && ( cd T$option/.git/objects && - find . -type f | sort >../../../T$option.objects-files.raw + find . -type f | sort >../../../T$option.objects-files.raw && + find . -type l | sort >../../../T$option.objects-symlinks.raw ) done && @@ -317,6 +326,7 @@ test_expect_success SYMLINKS 'clone repo with symlinked dirs and unknown files a ./Y/Z ./Y/Z ./a-loose-dir/Z + ./an-object ./Y/Z ./info/packs ./pack/pack-Z.idx @@ -326,15 +336,17 @@ test_expect_success SYMLINKS 'clone repo with symlinked dirs and unknown files a ./unknown_file EOF - for option in --local --dissociate --no-hardlinks + for option in --local --no-hardlinks --dissociate do - test_cmp expected-files T$option.objects-files.raw.de-sha || return 1 + test_cmp expected-files T$option.objects-files.raw.de-sha || return 1 && + test_must_be_empty T$option.objects-symlinks.raw.de-sha || return 1 done && cat >expected-files <<-EOF && ./info/alternates EOF - test_cmp expected-files T--shared.objects-files.raw + test_cmp expected-files T--shared.objects-files.raw && + test_must_be_empty T--shared.objects-symlinks.raw ' test_done From patchwork Fri Mar 22 23:22:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matheus Tavares X-Patchwork-Id: 10866633 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 379C315AC for ; Fri, 22 Mar 2019 23:23:03 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1E0E22AA14 for ; Fri, 22 Mar 2019 23:23:03 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 07A1E2AA19; Fri, 22 Mar 2019 23:23:03 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D09612AA14 for ; Fri, 22 Mar 2019 23:23:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728337AbfCVXXA (ORCPT ); Fri, 22 Mar 2019 19:23:00 -0400 Received: from mail-qt1-f195.google.com ([209.85.160.195]:40174 "EHLO mail-qt1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727782AbfCVXW7 (ORCPT ); Fri, 22 Mar 2019 19:22:59 -0400 Received: by mail-qt1-f195.google.com with SMTP id x12so4436299qts.7 for ; Fri, 22 Mar 2019 16:22:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=usp-br.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=7WV1uTcn7eh12Y1cgNRgtyoHOOchVs7TOzTJlWTZ5k8=; b=yAfOWcuZN71ytM8Pns2CnllW3W04FBLX5ItBsBbkSeEIiLNa3+yU3X6pw3CYl0tCxQ HH7oO6wLXMDL6do+041/8xuFCb0eAKtLiZdoc6xzPGdSw/hD3kUjrdjMbZ5GCI5Y3aw1 ZsTNxrvl5FLTqBd2jOJm8opp1XX2Vn4J0S6laSbbZTUDcTOekfHXaA7rN9rQ/nwDvAjI Z24qh8OkGucDYyL5xMdZsCWB/o8GfDze20fLUv7Mv3KqqMbL6rA8nGWHYwUmq9i4SBGW T7MuceQpZhU8OuxlpA76ub3iLY/Xc5yLpo39MeFvAbl5RwSNxH7u1fZDcYFSPKS66pLF 2sAw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=7WV1uTcn7eh12Y1cgNRgtyoHOOchVs7TOzTJlWTZ5k8=; b=K1u0bIzvwtmsA3LpTwEeYu1TfeL0GK1NeDeYc0rP0nIv3Q03e7sSPMV+Xvr+FtD6PD 0/i0+32f0lCMPAFhjUiv+wMPLOQbzoeGh1yar1O8VrKdsHS/0mOnjkArAAbKpge/h2L2 T9U/W/g35FQmjyxsQ6EJIPHxxDC/gQ0PQKgbMyUcpHDq3nKIOmYnEaJXToGZakp/rdvf Kyrl61H7InwhA+Z1poiUBLYmNBnYrnnbYHR4wiTaZE2WKqIlBjuza3HqgG90uwvVJYyI fKLwrnNvqEb/yRSZMUf6XB4wbT4BTGfqVWITYRA7Bh5/whJ6lGpDdz7BYV1M5nFsWuaQ Kp9Q== X-Gm-Message-State: APjAAAXwHovOcDk7fOBLeTqqaxtpnX9snRRxZjOPEalGOg9eXmfq0N6I v3HejAgwrSoY2nUPfmKBiRvhxw8fqG4= X-Google-Smtp-Source: APXvYqxaNl4VrhiS/Cj4h7nUnzjr5YIYHQ2cU34ywU4VAGKYXoTwR5EqwWhVnGzohe8aiWMASFyl4A== X-Received: by 2002:a0c:9ba1:: with SMTP id o33mr10601574qve.15.1553296978406; Fri, 22 Mar 2019 16:22:58 -0700 (PDT) Received: from mango.spo.virtua.com.br ([2804:14c:81:942d::3]) by smtp.gmail.com with ESMTPSA id i68sm5002765qkf.42.2019.03.22.16.22.55 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 22 Mar 2019 16:22:57 -0700 (PDT) From: Matheus Tavares To: git@vger.kernel.org Cc: Thomas Gummerer , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= , Christian Couder , =?utf-8?b?Tmd1eeG7hW4gVGg=?= =?utf-8?b?w6FpIE5n4buNYyBEdXk=?= , kernel-usp@googlegroups.com, Michael Haggerty , Ramsay Jones , Junio C Hamano Subject: [GSoC][PATCH v4 3/7] dir-iterator: add flags parameter to dir_iterator_begin Date: Fri, 22 Mar 2019 20:22:33 -0300 Message-Id: <20190322232237.13293-4-matheus.bernardino@usp.br> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190322232237.13293-1-matheus.bernardino@usp.br> References: <20190226122829.19178-1-avarab@gmail.com> <20190322232237.13293-1-matheus.bernardino@usp.br> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add the possibility of giving flags to dir_iterator_begin to initialize a dir-iterator with special options. Currently possible flags are DIR_ITERATOR_PEDANTIC, which makes dir_iterator_advance abort imediatelly in the case of an error while trying to fetch next entry; and DIR_ITERATOR_FOLLOW_SYMLINKS, which makes the iteration follow symlinks to directories and include its contents in the iteration. These new flags will be used in a subsequent patch. Also adjust refs/files-backend.c to the new dir_iterator_begin signature. Signed-off-by: Matheus Tavares --- dir-iterator.c | 28 +++++++++++++++++++++++++--- dir-iterator.h | 39 +++++++++++++++++++++++++++++++++------ refs/files-backend.c | 2 +- 3 files changed, 59 insertions(+), 10 deletions(-) diff --git a/dir-iterator.c b/dir-iterator.c index f2dcd82fde..17aca8ea41 100644 --- a/dir-iterator.c +++ b/dir-iterator.c @@ -48,12 +48,16 @@ struct dir_iterator_int { * that will be included in this iteration. */ struct dir_iterator_level *levels; + + /* Combination of flags for this dir-iterator */ + unsigned flags; }; int dir_iterator_advance(struct dir_iterator *dir_iterator) { struct dir_iterator_int *iter = (struct dir_iterator_int *)dir_iterator; + int ret; while (1) { struct dir_iterator_level *level = @@ -71,6 +75,8 @@ int dir_iterator_advance(struct dir_iterator *dir_iterator) level->dir = opendir(iter->base.path.buf); if (!level->dir && errno != ENOENT) { + if (iter->flags & DIR_ITERATOR_PEDANTIC) + goto error_out; warning("error opening directory %s: %s", iter->base.path.buf, strerror(errno)); /* Popping the level is handled below */ @@ -122,6 +128,8 @@ int dir_iterator_advance(struct dir_iterator *dir_iterator) if (!de) { /* This level is exhausted; pop up a level. */ if (errno) { + if (iter->flags & DIR_ITERATOR_PEDANTIC) + goto error_out; warning("error reading directory %s: %s", iter->base.path.buf, strerror(errno)); } else if (closedir(level->dir)) @@ -138,11 +146,20 @@ int dir_iterator_advance(struct dir_iterator *dir_iterator) continue; strbuf_addstr(&iter->base.path, de->d_name); - if (lstat(iter->base.path.buf, &iter->base.st) < 0) { - if (errno != ENOENT) + + if (iter->flags & DIR_ITERATOR_FOLLOW_SYMLINKS) + ret = stat(iter->base.path.buf, &iter->base.st); + else + ret = lstat(iter->base.path.buf, &iter->base.st); + + if (ret < 0) { + if (errno != ENOENT) { + if (iter->flags & DIR_ITERATOR_PEDANTIC) + goto error_out; warning("error reading path '%s': %s", iter->base.path.buf, strerror(errno)); + } continue; } @@ -159,6 +176,10 @@ int dir_iterator_advance(struct dir_iterator *dir_iterator) return ITER_OK; } } + +error_out: + dir_iterator_abort(dir_iterator); + return ITER_ERROR; } int dir_iterator_abort(struct dir_iterator *dir_iterator) @@ -182,7 +203,7 @@ int dir_iterator_abort(struct dir_iterator *dir_iterator) return ITER_DONE; } -struct dir_iterator *dir_iterator_begin(const char *path) +struct dir_iterator *dir_iterator_begin(const char *path, unsigned flags) { struct dir_iterator_int *iter = xcalloc(1, sizeof(*iter)); struct dir_iterator *dir_iterator = &iter->base; @@ -195,6 +216,7 @@ struct dir_iterator *dir_iterator_begin(const char *path) ALLOC_GROW(iter->levels, 10, iter->levels_alloc); + iter->flags = flags; iter->levels_nr = 1; iter->levels[0].initialized = 0; diff --git a/dir-iterator.h b/dir-iterator.h index 970793d07a..890d5d8dbb 100644 --- a/dir-iterator.h +++ b/dir-iterator.h @@ -19,7 +19,7 @@ * A typical iteration looks like this: * * int ok; - * struct iterator *iter = dir_iterator_begin(path); + * struct iterator *iter = dir_iterator_begin(path, 0); * * while ((ok = dir_iterator_advance(iter)) == ITER_OK) { * if (want_to_stop_iteration()) { @@ -40,6 +40,20 @@ * dir_iterator_advance() again. */ +/* + * Flags for dir_iterator_begin: + * + * - DIR_ITERATOR_PEDANTIC: override dir-iterator's default behavior + * in case of an error while trying to fetch the next entry, which is + * to emit a warning and keep going. With this flag, resouces are + * freed and ITER_ERROR is return immediately. + * + * - DIR_ITERATOR_FOLLOW_SYMLINKS: make dir-iterator follow symlinks to + * directories, i.e., iterate over linked directories' contents. + */ +#define DIR_ITERATOR_PEDANTIC (1 << 0) +#define DIR_ITERATOR_FOLLOW_SYMLINKS (1 << 1) + struct dir_iterator { /* The current path: */ struct strbuf path; @@ -54,20 +68,28 @@ struct dir_iterator { /* The current basename: */ const char *basename; - /* The result of calling lstat() on path: */ + /* + * The result of calling lstat() on path or stat(), if the + * DIR_ITERATOR_FOLLOW_SYMLINKS flag was set at + * dir_iterator's initialization. + */ struct stat st; }; /* - * Start a directory iteration over path. Return a dir_iterator that - * holds the internal state of the iteration. + * Start a directory iteration over path with the combination of + * options specified by flags. Return a dir_iterator that holds the + * internal state of the iteration. * * The iteration includes all paths under path, not including path * itself and not including "." or ".." entries. * - * path is the starting directory. An internal copy will be made. + * Parameters are: + * - path is the starting directory. An internal copy will be made. + * - flags is a combination of the possible flags to initialize a + * dir-iterator or 0 for default behaviour. */ -struct dir_iterator *dir_iterator_begin(const char *path); +struct dir_iterator *dir_iterator_begin(const char *path, unsigned flags); /* * Advance the iterator to the first or next item and return ITER_OK. @@ -76,6 +98,11 @@ struct dir_iterator *dir_iterator_begin(const char *path); * dir_iterator and associated resources and return ITER_ERROR. It is * a bug to use iterator or call this function again after it has * returned ITER_DONE or ITER_ERROR. + * + * Note that whether dir-iterator will return ITER_ERROR when failing + * to fetch the next entry or just emit a warning and try to fetch the + * next is defined by the 'pedantic' option at dir-iterator's + * initialization. */ int dir_iterator_advance(struct dir_iterator *iterator); diff --git a/refs/files-backend.c b/refs/files-backend.c index ef053f716c..2ce9783097 100644 --- a/refs/files-backend.c +++ b/refs/files-backend.c @@ -2143,7 +2143,7 @@ static struct ref_iterator *reflog_iterator_begin(struct ref_store *ref_store, base_ref_iterator_init(ref_iterator, &files_reflog_iterator_vtable, 0); strbuf_addf(&sb, "%s/logs", gitdir); - iter->dir_iterator = dir_iterator_begin(sb.buf); + iter->dir_iterator = dir_iterator_begin(sb.buf, 0); iter->ref_store = ref_store; strbuf_release(&sb); From patchwork Fri Mar 22 23:22:34 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Matheus Tavares X-Patchwork-Id: 10866635 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CDB6E15AC for ; Fri, 22 Mar 2019 23:23:04 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B68A42A9F9 for ; Fri, 22 Mar 2019 23:23:04 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AB7012AA16; Fri, 22 Mar 2019 23:23:04 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4603A2A9F9 for ; Fri, 22 Mar 2019 23:23:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728436AbfCVXXD (ORCPT ); Fri, 22 Mar 2019 19:23:03 -0400 Received: from mail-qt1-f194.google.com ([209.85.160.194]:43805 "EHLO mail-qt1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727782AbfCVXXC (ORCPT ); Fri, 22 Mar 2019 19:23:02 -0400 Received: by mail-qt1-f194.google.com with SMTP id v32so4410478qtc.10 for ; Fri, 22 Mar 2019 16:23:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=usp-br.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=w31IgR5n4pwxD0WOvj4gks83n9A5TGB3dAAc+WgAmPY=; b=d7V3ZXWyptx2RPHy6T51wj9UysUp/DtZVRsiqD/0hpKFrPa7u0QQAwLRHqamxKd/dz VaeipOIFBG+SUWzY7M5vi3L6iHvZKM570j1GBH0xvT4ck2wDHlC2WAe1Fod27EJ/LC3e +4ngbTcdfyRxR4XQUyUqi0w+kJyZakjdZ23t/fy1YFAip5lLB+NG9eKZc5P8+6Jy9+wr g0Tt4uTpcVyv9g+2PgaHLRjEiaxvJMDIHJZFFJXv1zYkurzHXmCrmL5YZQrikGG3/npQ BFIXvrtX/jzIY0gFF9P6Vqm9icP8qxpfjn3z3KHXyWFbIoHiiiOFQ7kIMhaw5J8Nd4qJ KV4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=w31IgR5n4pwxD0WOvj4gks83n9A5TGB3dAAc+WgAmPY=; b=sE5PV/cqLyNJCl/KdHv0goAT3dOOQTqPXTZv+xB+sBy1PT49JxumeDmqQ5HMmTmkIe IdjRYM5B/1qcmaQR/QKw82NEl6IMgtGpvxOufe+xD9jPsoRiobDoq/AtkDRJZ2dq2CNg ZfCHmb/eRw3StsWThnRM9Uv19JnDVX5/PtleIUQge/XqL/rtQ9SQdbT5wP5ag/SRXAwI NbGNtTy+4e+0ofEOVS5CKAHpFTzzod7qrhAqZDthY1JMQk8SFVaIBi7hA6xsoIBMRuuL 4zSSxEQaGzGqA/BExLhso3zqdek1QXGmxQnQp+iD5II4n113vWn93O/M7MOEVE0xgeMj lx/w== X-Gm-Message-State: APjAAAWB9icqvBl8ea2+unA4L2DKqezMNEFmR/1xbKLy833TvZXPMQhE QrSHqf9UtqDgAyofVTxXj193MFjyzR8= X-Google-Smtp-Source: APXvYqyDqQ6gVYR/mvog4h8moLZcL/EWZbkyeRp6Jf09Ateb9hgErBGP/8PthYIlJ0+Gbv6tJPJ0dw== X-Received: by 2002:ac8:3821:: with SMTP id q30mr10356765qtb.73.1553296981330; Fri, 22 Mar 2019 16:23:01 -0700 (PDT) Received: from mango.spo.virtua.com.br ([2804:14c:81:942d::3]) by smtp.gmail.com with ESMTPSA id i68sm5002765qkf.42.2019.03.22.16.22.58 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 22 Mar 2019 16:23:00 -0700 (PDT) From: Matheus Tavares To: git@vger.kernel.org Cc: Thomas Gummerer , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= , Christian Couder , =?utf-8?b?Tmd1eeG7hW4gVGg=?= =?utf-8?b?w6FpIE5n4buNYyBEdXk=?= , kernel-usp@googlegroups.com, Benoit Pierre , Junio C Hamano Subject: [GSoC][PATCH v4 4/7] clone: copy hidden paths at local clone Date: Fri, 22 Mar 2019 20:22:34 -0300 Message-Id: <20190322232237.13293-5-matheus.bernardino@usp.br> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190322232237.13293-1-matheus.bernardino@usp.br> References: <20190226122829.19178-1-avarab@gmail.com> <20190322232237.13293-1-matheus.bernardino@usp.br> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Make the copy_or_link_directory function no longer skip hidden directories. This function, used to copy .git/objects, currently skips all hidden directories but not hidden files, which is an odd behaviour. The reason for that could be unintentional: probably the intention was to skip '.' and '..' only but it ended up accidentally skipping all directories starting with '.'. Besides being more natural, the new behaviour is more permissive to the user. Also adjust tests to reflect this behaviour change. Signed-off-by: Matheus Tavares Signed-off-by: Ævar Arnfjörð Bjarmason Co-authored-by: Ævar Arnfjörð Bjarmason --- builtin/clone.c | 2 +- t/t5604-clone-reference.sh | 9 +++++++++ 2 files changed, 10 insertions(+), 1 deletion(-) diff --git a/builtin/clone.c b/builtin/clone.c index b76f33c635..60c6780c06 100644 --- a/builtin/clone.c +++ b/builtin/clone.c @@ -428,7 +428,7 @@ static void copy_or_link_directory(struct strbuf *src, struct strbuf *dest, continue; } if (S_ISDIR(buf.st_mode)) { - if (de->d_name[0] != '.') + if (!is_dot_or_dotdot(de->d_name)) copy_or_link_directory(src, dest, src_repo, src_baselen); continue; diff --git a/t/t5604-clone-reference.sh b/t/t5604-clone-reference.sh index 76d45f1187..0992baa5ac 100755 --- a/t/t5604-clone-reference.sh +++ b/t/t5604-clone-reference.sh @@ -247,16 +247,25 @@ test_expect_success 'clone a repo with garbage in objects/*' ' done && find S-* -name "*some*" | sort >actual && cat >expected <<-EOF && + S--dissociate/.git/objects/.some-hidden-dir + S--dissociate/.git/objects/.some-hidden-dir/.some-dot-file + S--dissociate/.git/objects/.some-hidden-dir/some-file S--dissociate/.git/objects/.some-hidden-file S--dissociate/.git/objects/some-dir S--dissociate/.git/objects/some-dir/.some-dot-file S--dissociate/.git/objects/some-dir/some-file S--dissociate/.git/objects/some-file + S--local/.git/objects/.some-hidden-dir + S--local/.git/objects/.some-hidden-dir/.some-dot-file + S--local/.git/objects/.some-hidden-dir/some-file S--local/.git/objects/.some-hidden-file S--local/.git/objects/some-dir S--local/.git/objects/some-dir/.some-dot-file S--local/.git/objects/some-dir/some-file S--local/.git/objects/some-file + S--no-hardlinks/.git/objects/.some-hidden-dir + S--no-hardlinks/.git/objects/.some-hidden-dir/.some-dot-file + S--no-hardlinks/.git/objects/.some-hidden-dir/some-file S--no-hardlinks/.git/objects/.some-hidden-file S--no-hardlinks/.git/objects/some-dir S--no-hardlinks/.git/objects/some-dir/.some-dot-file From patchwork Fri Mar 22 23:22:35 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matheus Tavares X-Patchwork-Id: 10866637 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 273EE1669 for ; Fri, 22 Mar 2019 23:23:08 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1178E2AA10 for ; Fri, 22 Mar 2019 23:23:08 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 05E452AA14; Fri, 22 Mar 2019 23:23:08 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EB2CC2AA10 for ; Fri, 22 Mar 2019 23:23:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728445AbfCVXXF (ORCPT ); Fri, 22 Mar 2019 19:23:05 -0400 Received: from mail-qt1-f177.google.com ([209.85.160.177]:40884 "EHLO mail-qt1-f177.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727782AbfCVXXF (ORCPT ); Fri, 22 Mar 2019 19:23:05 -0400 Received: by mail-qt1-f177.google.com with SMTP id x12so4436469qts.7 for ; Fri, 22 Mar 2019 16:23:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=usp-br.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=QIiOCHYHZ4WP3/M+Kt6+3GSWZEEiYjUotGx83tH2z9g=; b=Zgz4vakZw69OYgsYX1KGcwHLdurG2TzpH5t3P+5x2FaGX7dwWtddj031JLDWNPgyb+ PVgbXi4dIJRJ3/JUvQkVvdQqOE18crAcggM1p/9O0yEiha5El7F10jmoFO/eXMbmqfIf /vX28BoKstarL21dIF446erNXTkS62ShnCoIep6BMoQxBcHQ9d7Ay98Wmg6zuHDps6/C 6kUaUQsHYdfTk7hSnWmqJCC1MxjODUQJwXKH7fm69j+m5a3UXUdYTF0G0ELd5VxY5oR8 Mdf7c+dXAftnOVGdotjJdRQFBdmKVQkxDbpX6TpPCP/Vsxc/lN9hpw/UyOhJWx407/hB uD6A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=QIiOCHYHZ4WP3/M+Kt6+3GSWZEEiYjUotGx83tH2z9g=; b=CxKepWWblKhqYRg6XlJsdRlH3XWNKIKz7gGF0mHK6HpkyOBJGJ6+Q9OVYHF+ERvqZF 0RtiMobwqrff7YEA5noZWeQ1t2/auWlijDmsFLYnlWtFH6SGKgntbCSrJasRVBtKR4bg QXlO4PB4FmHacC7jD3QxRbAoR1BsYyTfNV7WEBOi3eYGm6xKed+c7MHao2KE/rKi61s9 SUde/Hbp/zdf0hjX2lv9be6fUaMfWxqz8JRD4amImlfVGdDYERMIYG0rNb9aPnRs5OjQ XK3GOwR2l8DSL3pzuJ+1utk+kYhGrxxz2KKIL1Din7o1jUikHphDgNCKpeg7eR92nmFx m7sw== X-Gm-Message-State: APjAAAWPgF2BRqnWZuczuQdyy+ufx10es0P6PhCM+R4lp8OMb5AzbZoA s1BHCEpLztQ15ErWzjynM72F+p15HT4= X-Google-Smtp-Source: APXvYqzAgya1LLtFPLgSwkQYmMGdTatfzwdXbUE1+lm5Sds+p8I/jgt5cwM/X1vDMdEmRlC9ByBf0Q== X-Received: by 2002:ac8:33dd:: with SMTP id d29mr11005681qtb.320.1553296984397; Fri, 22 Mar 2019 16:23:04 -0700 (PDT) Received: from mango.spo.virtua.com.br ([2804:14c:81:942d::3]) by smtp.gmail.com with ESMTPSA id i68sm5002765qkf.42.2019.03.22.16.23.01 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 22 Mar 2019 16:23:03 -0700 (PDT) From: Matheus Tavares To: git@vger.kernel.org Cc: Thomas Gummerer , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= , Christian Couder , =?utf-8?b?Tmd1eeG7hW4gVGg=?= =?utf-8?b?w6FpIE5n4buNYyBEdXk=?= , kernel-usp@googlegroups.com, Benoit Pierre , Junio C Hamano Subject: [GSoC][PATCH v4 5/7] clone: extract function from copy_or_link_directory Date: Fri, 22 Mar 2019 20:22:35 -0300 Message-Id: <20190322232237.13293-6-matheus.bernardino@usp.br> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190322232237.13293-1-matheus.bernardino@usp.br> References: <20190226122829.19178-1-avarab@gmail.com> <20190322232237.13293-1-matheus.bernardino@usp.br> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Extract dir creation code snippet from copy_or_link_directory to its own function named mkdir_if_missing. This change will help removing copy_or_link_directory's explicit recursion, which will be done in a following patch. Also makes code more readable. Signed-off-by: Matheus Tavares --- builtin/clone.c | 24 ++++++++++++++++-------- 1 file changed, 16 insertions(+), 8 deletions(-) diff --git a/builtin/clone.c b/builtin/clone.c index 60c6780c06..c17bbf1bfc 100644 --- a/builtin/clone.c +++ b/builtin/clone.c @@ -392,6 +392,21 @@ static void copy_alternates(struct strbuf *src, struct strbuf *dst, fclose(in); } +static void mkdir_if_missing(const char *pathname, mode_t mode) +{ + struct stat st; + + if (!mkdir(pathname, mode)) + return; + + if (errno != EEXIST) + die_errno(_("failed to create directory '%s'"), pathname); + else if (stat(pathname, &st)) + die_errno(_("failed to stat '%s'"), pathname); + else if (!S_ISDIR(st.st_mode)) + die(_("%s exists and is not a directory"), pathname); +} + static void copy_or_link_directory(struct strbuf *src, struct strbuf *dest, const char *src_repo, int src_baselen) { @@ -404,14 +419,7 @@ static void copy_or_link_directory(struct strbuf *src, struct strbuf *dest, if (!dir) die_errno(_("failed to open '%s'"), src->buf); - if (mkdir(dest->buf, 0777)) { - if (errno != EEXIST) - die_errno(_("failed to create directory '%s'"), dest->buf); - else if (stat(dest->buf, &buf)) - die_errno(_("failed to stat '%s'"), dest->buf); - else if (!S_ISDIR(buf.st_mode)) - die(_("%s exists and is not a directory"), dest->buf); - } + mkdir_if_missing(dest->buf, 0777); strbuf_addch(src, '/'); src_len = src->len; From patchwork Fri Mar 22 23:22:36 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matheus Tavares X-Patchwork-Id: 10866639 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E2E791669 for ; Fri, 22 Mar 2019 23:23:10 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CBDAA2A480 for ; Fri, 22 Mar 2019 23:23:10 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BF78B2AA16; Fri, 22 Mar 2019 23:23:10 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 539B72AA1C for ; Fri, 22 Mar 2019 23:23:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728468AbfCVXXJ (ORCPT ); Fri, 22 Mar 2019 19:23:09 -0400 Received: from mail-qk1-f193.google.com ([209.85.222.193]:33918 "EHLO mail-qk1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727782AbfCVXXI (ORCPT ); Fri, 22 Mar 2019 19:23:08 -0400 Received: by mail-qk1-f193.google.com with SMTP id n68so2259772qka.1 for ; Fri, 22 Mar 2019 16:23:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=usp-br.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=N+PBpvo64IKr3yBPeTcCw2yZHKIa9a/SpKxRmBnM4yo=; b=Y88FikOCdjLPP6iY+pffNEAdUCBTaWK2i3Phh2E6o+EVfBKNsDQnv4ufqQLk8PhGOI Daf95rezm3D6pYjUyekMNibA8U8+ih0LuL6ZdK/G7saPQmniMDoPE+15AgnS229dntvf z1APM55NyzqUmfIj92eqkIAK//09pfEWmXjK76btEdrgVoAZmyjKA5O5mqoZYvK9hUsr EMPkteJS7kq2M75qJOADOV+ArZs8FCB68KaPLJLpjTKjaBHq7vLjwTW2FcK6oQPYQWCr EJTYhE4Y5G9Iy6f2gsFzRhhpyIbaFnr90fHHnDXgKaYRLK9zZZQqWdEXKm5D2mq5ZT3H TOGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=N+PBpvo64IKr3yBPeTcCw2yZHKIa9a/SpKxRmBnM4yo=; b=e3d7JzWKhJl0k9JucyfsYqzzUHWDgG+BTBsMy/uKJ7BAxWYuj661tYQjDOTMz/mUro 8Rm4pnyBIQpGWsgPiCp8pOzDnk12f3T8ajbJrMpfCheFN9wESN6kG+pIaln2rWdAC2sv OrXFcwS7dKHI1AQIkQcO1/2vCBjKXx8yqi/YvG+cy6pvdSuUsadu7pQ+pn65QIstybyM cxdbbU2k65kgO5Qz29JkpBtDEt73WYTKNl+vo+J7yGqhl9v/Plc562SkcdtFvrAIpudk P31pbcT9Z2eypHC4R6jAMAsHqgSx5MByBoo+Ygk7Xl0a5EVChXAA/nSIF+Um3/CXgS4B 610A== X-Gm-Message-State: APjAAAXBiTnHSIQz5m6trHmNn1tJP7b2ZOZQpRKdtc3VLxrvm8pkITS8 oDt7FYkvtSjsN+CPNymYTvo2y839uSM= X-Google-Smtp-Source: APXvYqyGwmZC6Q3njEWNET/yP2BZU+/AjoZafHcLM3D6UX07E4lAp6QECQDuEpxjl6glQup7a+6czw== X-Received: by 2002:a37:bd81:: with SMTP id n123mr9979686qkf.249.1553296987452; Fri, 22 Mar 2019 16:23:07 -0700 (PDT) Received: from mango.spo.virtua.com.br ([2804:14c:81:942d::3]) by smtp.gmail.com with ESMTPSA id i68sm5002765qkf.42.2019.03.22.16.23.04 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 22 Mar 2019 16:23:06 -0700 (PDT) From: Matheus Tavares To: git@vger.kernel.org Cc: Thomas Gummerer , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= , Christian Couder , =?utf-8?b?Tmd1eeG7hW4gVGg=?= =?utf-8?b?w6FpIE5n4buNYyBEdXk=?= , kernel-usp@googlegroups.com, Benoit Pierre , Junio C Hamano Subject: [GSoC][PATCH v4 6/7] clone: use dir-iterator to avoid explicit dir traversal Date: Fri, 22 Mar 2019 20:22:36 -0300 Message-Id: <20190322232237.13293-7-matheus.bernardino@usp.br> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190322232237.13293-1-matheus.bernardino@usp.br> References: <20190226122829.19178-1-avarab@gmail.com> <20190322232237.13293-1-matheus.bernardino@usp.br> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Replace usage of opendir/readdir/closedir API to traverse directories recursively, at copy_or_link_directory function, by the dir-iterator API. This simplifies the code and avoid recursive calls to copy_or_link_directory. This process also makes copy_or_link_directory call die() in case of an error on readdir or stat, inside dir_iterator_advance. Previously it would just print a warning for errors on stat and ignore errors on readdir, which isn't nice because a local git clone would end up successfully even though the .git/objects copy didn't fully succeeded. Signed-off-by: Matheus Tavares --- builtin/clone.c | 44 ++++++++++++++++++++++---------------------- 1 file changed, 22 insertions(+), 22 deletions(-) diff --git a/builtin/clone.c b/builtin/clone.c index c17bbf1bfc..4ee45e7862 100644 --- a/builtin/clone.c +++ b/builtin/clone.c @@ -23,6 +23,8 @@ #include "transport.h" #include "strbuf.h" #include "dir.h" +#include "dir-iterator.h" +#include "iterator.h" #include "sigchain.h" #include "branch.h" #include "remote.h" @@ -408,42 +410,36 @@ static void mkdir_if_missing(const char *pathname, mode_t mode) } static void copy_or_link_directory(struct strbuf *src, struct strbuf *dest, - const char *src_repo, int src_baselen) + const char *src_repo) { - struct dirent *de; - struct stat buf; int src_len, dest_len; - DIR *dir; - - dir = opendir(src->buf); - if (!dir) - die_errno(_("failed to open '%s'"), src->buf); + struct dir_iterator *iter; + int iter_status; + unsigned flags; mkdir_if_missing(dest->buf, 0777); + flags = DIR_ITERATOR_PEDANTIC | DIR_ITERATOR_FOLLOW_SYMLINKS; + iter = dir_iterator_begin(src->buf, flags); + strbuf_addch(src, '/'); src_len = src->len; strbuf_addch(dest, '/'); dest_len = dest->len; - while ((de = readdir(dir)) != NULL) { + while ((iter_status = dir_iterator_advance(iter)) == ITER_OK) { strbuf_setlen(src, src_len); - strbuf_addstr(src, de->d_name); + strbuf_addstr(src, iter->relative_path); strbuf_setlen(dest, dest_len); - strbuf_addstr(dest, de->d_name); - if (stat(src->buf, &buf)) { - warning (_("failed to stat %s\n"), src->buf); - continue; - } - if (S_ISDIR(buf.st_mode)) { - if (!is_dot_or_dotdot(de->d_name)) - copy_or_link_directory(src, dest, - src_repo, src_baselen); + strbuf_addstr(dest, iter->relative_path); + + if (S_ISDIR(iter->st.st_mode)) { + mkdir_if_missing(dest->buf, 0777); continue; } /* Files that cannot be copied bit-for-bit... */ - if (!strcmp(src->buf + src_baselen, "/info/alternates")) { + if (!strcmp(iter->relative_path, "info/alternates")) { copy_alternates(src, dest, src_repo); continue; } @@ -460,7 +456,11 @@ static void copy_or_link_directory(struct strbuf *src, struct strbuf *dest, if (copy_file_with_time(dest->buf, src->buf, 0666)) die_errno(_("failed to copy file to '%s'"), dest->buf); } - closedir(dir); + + if (iter_status != ITER_DONE) { + strbuf_setlen(src, src_len); + die(_("failed to iterate over '%s'"), src->buf); + } } static void clone_local(const char *src_repo, const char *dest_repo) @@ -478,7 +478,7 @@ static void clone_local(const char *src_repo, const char *dest_repo) get_common_dir(&dest, dest_repo); strbuf_addstr(&src, "/objects"); strbuf_addstr(&dest, "/objects"); - copy_or_link_directory(&src, &dest, src_repo, src.len); + copy_or_link_directory(&src, &dest, src_repo); strbuf_release(&src); strbuf_release(&dest); } From patchwork Fri Mar 22 23:22:37 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Matheus Tavares X-Patchwork-Id: 10866641 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F175B15AC for ; Fri, 22 Mar 2019 23:23:13 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DB3BB2AA13 for ; Fri, 22 Mar 2019 23:23:13 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CFD842AA15; Fri, 22 Mar 2019 23:23:13 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7E4142AA13 for ; Fri, 22 Mar 2019 23:23:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728484AbfCVXXM (ORCPT ); Fri, 22 Mar 2019 19:23:12 -0400 Received: from mail-qt1-f195.google.com ([209.85.160.195]:41943 "EHLO mail-qt1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728102AbfCVXXM (ORCPT ); Fri, 22 Mar 2019 19:23:12 -0400 Received: by mail-qt1-f195.google.com with SMTP id w30so4425205qta.8 for ; Fri, 22 Mar 2019 16:23:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=usp-br.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=CJihAWDQTLYbxPzuhbE4TElWGnCFpWOhxwA6JVS/bgg=; b=s5ZLRoeaGlB85210uZDHW+xWFIj1Eanq7XFM5Gou+deQe56bvPkQwxBBGvtKEIm1Tq cmnHDbtAG1BcNr6HAF+IuF4/KW8veanlU79lEMjTXyNO3bia3jJTNMVg5j0SJExJY5wE uPKKNiJJWdfVJ25uuelUtQcywzBjL77T/oruK3uhaCBMzzCVXDxLyQ9u6/Qv+Dlr44Eg WRvKjv0b2gnXOOqb68qWBpCUxpXf++wqdE2vUuYbHdDlu7af+LiIB4/1IjzaEhzfxs3g UBvMh16otVoCGItHFkKjnE4wehnm6oF1f7dtxMAxBavx3PPFLDvP3KT1UA2v2LxnthTc uZqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=CJihAWDQTLYbxPzuhbE4TElWGnCFpWOhxwA6JVS/bgg=; b=Q1isV2ZRLv0k/BVCOgbQE6VmR+/aJuC9o6WpN4PfrTdhJPdBjn/fw4LiA8WF+kjRp/ YZ9feZ/0u7bk+oC+3oo/5+8cMgoDycK0HeEEN93jheBjFT6XRDzB9f0CmUQMXYg5Z4El 61h5/tkAo8htVs7xa63F3uB1Co0IyvhQLQzIrY0fSWeXduod7/x6DylQ27BOKaTqmEd/ vuv+/lSiBEM3xnyVHZq6gS01yKCfhIT2Xn1d2yZ6MeVrXuTOK4U0wCK+wOmcppq5Fs8T 0pMTfCwVOIi+1NwnrxK+BnuA4guJOXdI8ZlEO/a+aMx2+4PCiN4ufB8HrLfXuuSlftG0 XQQw== X-Gm-Message-State: APjAAAVdIMcrcQMRaDoyd581T0VWepes02GcgZQOorWLcSZ3QJmKSNdg gkIA3uquWKy0/qmOIVl5sJdxB13K/Ew= X-Google-Smtp-Source: APXvYqxeLt7hFCNQqqPubJz7PL5KUY/q2eJOqhImN48sV9q9qjyM7Q3FXQnrGHRXkwf39uxzlKHRfw== X-Received: by 2002:a0c:d808:: with SMTP id h8mr10529076qvj.245.1553296990500; Fri, 22 Mar 2019 16:23:10 -0700 (PDT) Received: from mango.spo.virtua.com.br ([2804:14c:81:942d::3]) by smtp.gmail.com with ESMTPSA id i68sm5002765qkf.42.2019.03.22.16.23.07 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 22 Mar 2019 16:23:09 -0700 (PDT) From: Matheus Tavares To: git@vger.kernel.org Cc: Thomas Gummerer , =?utf-8?b?w4Z2YXIgQXJuZmrDtnI=?= =?utf-8?b?w7AgQmphcm1hc29u?= , Christian Couder , =?utf-8?b?Tmd1eeG7hW4gVGg=?= =?utf-8?b?w6FpIE5n4buNYyBEdXk=?= , kernel-usp@googlegroups.com, Benoit Pierre , Junio C Hamano Subject: [GSoC][PATCH v4 7/7] clone: Replace strcmp by fspathcmp Date: Fri, 22 Mar 2019 20:22:37 -0300 Message-Id: <20190322232237.13293-8-matheus.bernardino@usp.br> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190322232237.13293-1-matheus.bernardino@usp.br> References: <20190226122829.19178-1-avarab@gmail.com> <20190322232237.13293-1-matheus.bernardino@usp.br> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Replace the use of strcmp by fspathcmp at copy_or_link_directory, which is more permissive/friendly to case-insensitive file systems. Signed-off-by: Matheus Tavares Suggested-by: Nguyễn Thái Ngọc Duy --- builtin/clone.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/builtin/clone.c b/builtin/clone.c index 4ee45e7862..763ad5e31f 100644 --- a/builtin/clone.c +++ b/builtin/clone.c @@ -439,7 +439,7 @@ static void copy_or_link_directory(struct strbuf *src, struct strbuf *dest, } /* Files that cannot be copied bit-for-bit... */ - if (!strcmp(iter->relative_path, "info/alternates")) { + if (!fspathcmp(iter->relative_path, "info/alternates")) { copy_alternates(src, dest, src_repo); continue; }