From patchwork Tue Feb 26 00:26:19 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10829319 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0C9C5922 for ; Tue, 26 Feb 2019 00:26:52 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EA3222B9AE for ; Tue, 26 Feb 2019 00:26:51 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DC9272B99B; Tue, 26 Feb 2019 00:26:51 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 170F92B99B for ; Tue, 26 Feb 2019 00:26:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729336AbfBZA0t (ORCPT ); Mon, 25 Feb 2019 19:26:49 -0500 Received: from mail-wr1-f65.google.com ([209.85.221.65]:39017 "EHLO mail-wr1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728953AbfBZA0t (ORCPT ); Mon, 25 Feb 2019 19:26:49 -0500 Received: by mail-wr1-f65.google.com with SMTP id l5so11955681wrw.6 for ; Mon, 25 Feb 2019 16:26:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=osZrhocj2gzjlsu7cdenUckK+fi5gPbO0O8sqpgAlJ8=; b=Y3mrTOV2Pn2C0vTU/U1J0zVBVusS9UqBKCTs1sKEnQLf7lstn+n2askf0jLjBvQRFA n/GDDT87O4aMQlXcT+KobIL99t3xH4JApmvNiT6clxlcRyndyHaLbmrtCZmHIUrc4HL7 azaxjmUspUQ/afeJBbQqLsz5DAMk7HjE4eCZviz4I714dBD8+5jEz+x6ZNONPumWVbm0 I+KeLgokdu164XsmtrGF8hsgixSUvxpeh9rOboYQIrsXFqYeiWtdU1rqf1x+xZ5bDO2Y t4Zbm+fZjf2niJ9sly+c7g4thjsEsZJMCNaIkEzjfDrk8aGWozQcMN7O0o96vHjeRiPU zL4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=osZrhocj2gzjlsu7cdenUckK+fi5gPbO0O8sqpgAlJ8=; b=OyMtraz9iLeUBSN738PtNJk8n5qgeil0wkW/pTdbeihII5TsGx9eas1UWvWcggEQie DrkSh6eajJijqjlCqnfToHgX/02paGmd+ZHhE4mx1Gq0ZN91v1851VEqLc8YMFbiOY8e HMcpXB+vnpb4DZDat5fsyEZ1jCUKA/n+R/zlXMlLYA7IknLjcIMhDIq/O2iv3ET2AMMU 34vRzj5ZQ7KnLX9TXDQiriceGvD6YIEjEBAYrGW1251IixFB1Dw2E/RRDJEG2oG7MR+o MfuGAeAb3ZC4RnUtYk9PzPHj7VRkmmqxsghdWbHmI4Pd6ng78hFs3pEURHHUFlNHp2fJ QQiw== X-Gm-Message-State: AHQUAuZtc65xkrA3s2qg1nH3tbG5XYlSpNkNuvBmD39PivWOysh+XA18 K5Ae0LzthklPMD9gIiC5MFskgBTylu8= X-Google-Smtp-Source: AHgI3IbSAopzodnkrtrsO0eH7V4cOs1oCA4OIdSA08CmLXoYO9TGNE594Xr0Iq4YmNk78ipbbmkWMw== X-Received: by 2002:adf:e80c:: with SMTP id o12mr13724384wrm.114.1551140806849; Mon, 25 Feb 2019 16:26:46 -0800 (PST) Received: from vm.nix.is ([2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id e6sm11745708wrt.14.2019.02.25.16.26.46 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Mon, 25 Feb 2019 16:26:46 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmc=?= =?utf-8?b?4buNYyBEdXk=?= , Matheus Tavares , Thomas Gummerer , Christian Couder Subject: [WIP RFC PATCH 1/7] dir-iterator: add pedantic option to dir_iterator_begin Date: Tue, 26 Feb 2019 01:26:19 +0100 Message-Id: <20190226002625.13022-2-avarab@gmail.com> X-Mailer: git-send-email 2.21.0.rc2.261.ga7da99ff1b In-Reply-To: References: MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Matheus Tavares Add the pedantic option to dir-iterator's initialization function, dir_iterator_begin. When this option is set to true, dir_iterator_advance will immediately return ITER_ERROR when failing to fetch the next entry. When set to false, dir_iterator_advance will emit a warning and keep looking for the next entry. Also adjust refs/files-backend.c to the new dir_iterator_begin signature. Signed-off-by: Matheus Tavares --- dir-iterator.c | 23 +++++++++++++++++++++-- dir-iterator.h | 16 +++++++++++++--- refs/files-backend.c | 2 +- 3 files changed, 35 insertions(+), 6 deletions(-) diff --git a/dir-iterator.c b/dir-iterator.c index f2dcd82fde..070a656555 100644 --- a/dir-iterator.c +++ b/dir-iterator.c @@ -48,6 +48,13 @@ struct dir_iterator_int { * that will be included in this iteration. */ struct dir_iterator_level *levels; + + /* + * Boolean value to define dir-iterator's behaviour when failing to + * fetch next entry. See comments on dir_iterator_begin at + * dir-iterator.h + */ + int pedantic; }; int dir_iterator_advance(struct dir_iterator *dir_iterator) @@ -71,6 +78,8 @@ int dir_iterator_advance(struct dir_iterator *dir_iterator) level->dir = opendir(iter->base.path.buf); if (!level->dir && errno != ENOENT) { + if (iter->pedantic) + goto error_out; warning("error opening directory %s: %s", iter->base.path.buf, strerror(errno)); /* Popping the level is handled below */ @@ -122,6 +131,8 @@ int dir_iterator_advance(struct dir_iterator *dir_iterator) if (!de) { /* This level is exhausted; pop up a level. */ if (errno) { + if (iter->pedantic) + goto error_out; warning("error reading directory %s: %s", iter->base.path.buf, strerror(errno)); } else if (closedir(level->dir)) @@ -139,10 +150,13 @@ int dir_iterator_advance(struct dir_iterator *dir_iterator) strbuf_addstr(&iter->base.path, de->d_name); if (lstat(iter->base.path.buf, &iter->base.st) < 0) { - if (errno != ENOENT) + if (errno != ENOENT) { + if (iter->pedantic) + goto error_out; warning("error reading path '%s': %s", iter->base.path.buf, strerror(errno)); + } continue; } @@ -159,6 +173,10 @@ int dir_iterator_advance(struct dir_iterator *dir_iterator) return ITER_OK; } } + +error_out: + dir_iterator_abort(dir_iterator); + return ITER_ERROR; } int dir_iterator_abort(struct dir_iterator *dir_iterator) @@ -182,7 +200,7 @@ int dir_iterator_abort(struct dir_iterator *dir_iterator) return ITER_DONE; } -struct dir_iterator *dir_iterator_begin(const char *path) +struct dir_iterator *dir_iterator_begin(const char *path, int pedantic) { struct dir_iterator_int *iter = xcalloc(1, sizeof(*iter)); struct dir_iterator *dir_iterator = &iter->base; @@ -195,6 +213,7 @@ struct dir_iterator *dir_iterator_begin(const char *path) ALLOC_GROW(iter->levels, 10, iter->levels_alloc); + iter->pedantic = pedantic; iter->levels_nr = 1; iter->levels[0].initialized = 0; diff --git a/dir-iterator.h b/dir-iterator.h index 970793d07a..50ca8e1a27 100644 --- a/dir-iterator.h +++ b/dir-iterator.h @@ -19,7 +19,7 @@ * A typical iteration looks like this: * * int ok; - * struct iterator *iter = dir_iterator_begin(path); + * struct iterator *iter = dir_iterator_begin(path, 0); * * while ((ok = dir_iterator_advance(iter)) == ITER_OK) { * if (want_to_stop_iteration()) { @@ -65,9 +65,15 @@ struct dir_iterator { * The iteration includes all paths under path, not including path * itself and not including "." or ".." entries. * - * path is the starting directory. An internal copy will be made. + * Parameters are: + * - path is the starting directory. An internal copy will be made. + * - pedantic is a boolean value. If true, dir-iterator will free + * resources and return ITER_ERROR immediately, in case of an error + * while trying to fetch the next entry in dir_iterator_advance. If + * false, it will just emit a warning and keep looking for the next + * entry. */ -struct dir_iterator *dir_iterator_begin(const char *path); +struct dir_iterator *dir_iterator_begin(const char *path, int pedantic); /* * Advance the iterator to the first or next item and return ITER_OK. @@ -76,6 +82,10 @@ struct dir_iterator *dir_iterator_begin(const char *path); * dir_iterator and associated resources and return ITER_ERROR. It is * a bug to use iterator or call this function again after it has * returned ITER_DONE or ITER_ERROR. + * + * Note that whether dir_iterator_advance will return ITER_ERROR when + * failing to fetch the next entry or keep going is defined by the + * 'pedantic' option at dir-iterator's initialization. */ int dir_iterator_advance(struct dir_iterator *iterator); diff --git a/refs/files-backend.c b/refs/files-backend.c index dd8abe9185..c3d3b6c454 100644 --- a/refs/files-backend.c +++ b/refs/files-backend.c @@ -2143,7 +2143,7 @@ static struct ref_iterator *reflog_iterator_begin(struct ref_store *ref_store, base_ref_iterator_init(ref_iterator, &files_reflog_iterator_vtable, 0); strbuf_addf(&sb, "%s/logs", gitdir); - iter->dir_iterator = dir_iterator_begin(sb.buf); + iter->dir_iterator = dir_iterator_begin(sb.buf, 0); iter->ref_store = ref_store; strbuf_release(&sb); From patchwork Tue Feb 26 00:26:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10829321 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 440051575 for ; Tue, 26 Feb 2019 00:26:52 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 32CA82B99B for ; Tue, 26 Feb 2019 00:26:52 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 26D202B9AE; Tue, 26 Feb 2019 00:26:52 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D47872B99C for ; Tue, 26 Feb 2019 00:26:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729330AbfBZA0u (ORCPT ); Mon, 25 Feb 2019 19:26:50 -0500 Received: from mail-wm1-f68.google.com ([209.85.128.68]:40782 "EHLO mail-wm1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729351AbfBZA0u (ORCPT ); Mon, 25 Feb 2019 19:26:50 -0500 Received: by mail-wm1-f68.google.com with SMTP id t15so636591wmi.5 for ; Mon, 25 Feb 2019 16:26:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=e5fANClE8AMQfIJDXXjZsBHKE7PLkV25MTsKozomZAg=; b=X/bGZpUdxrYy2bo7/aS68Ka7u55JDIOSoX8ZUtQ1+Q5w173vYCWVQmqLluhy9Z99x6 9VQ8xMfLhJROZBPQVJfMRadlImPn2c1+VbxVZV9iGWukc7KALppp745d0fnk9dd0ETor 6cWF8d44xTKStV9QJSTho2nnPj9pvtso+PEOC1TQEekliyVrINluD5TiMVH/QOYQoF4f iqBAX5zD+TvQpalUeo+bUyhBIz90ohbmLsjUm1vrnZR/bd9/NPLcEBW0KUg2HXHpf8r4 Yj37fQd+7vapj1NLyj/BnI5lTr6HYvZ+ngKfysePdWR887izBEoVcic146aGpXWnooQx 5LzA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=e5fANClE8AMQfIJDXXjZsBHKE7PLkV25MTsKozomZAg=; b=Q4mxy5cNuJT+CasY1+7Thyg/btvcVW/PpBpi32xOM6tcHcnit/urqYnkS6O8gWzTIw 0DpKCdlU/AIRXE5XS0UGPXvQMQF58C4FOP0lynkwYWv4V9Teyttvx8JTMCnNsG7Sy8sW uEFcpt28yQft+9xyY4NojOvZu4IvCSjLc76rtyseq7CpMK+XYoo9FJNbcFCYQVKDwOxl 1tKoirs2VikVIL/0qnsXMM2i0AuclqnUTP4YQ7khc8nQ/VtVq7e75aMIJxA7ef2xmouV HsBOTwzZiKA/bNZKAyw3sZPQ/cs8uWbi0rLnH7rJofHzycmP8n30qD+JFncq9xFct8HQ U1Og== X-Gm-Message-State: AHQUAuYDVGWKjedvc5niwKWpJskkjhOa+FwOV+wyltZEvOL9z1BaF8zJ SEMxNiDbEmXSX+Oy74g3NUKjomqkbZ4= X-Google-Smtp-Source: AHgI3IbpVAxo3SpDzSpD7HKzYVbAX9MOHKPM3m7R4GHZ2td09yrB/eHzzQ53FAQTJfWG0fKAEW6PbQ== X-Received: by 2002:a1c:990c:: with SMTP id b12mr825490wme.106.1551140807961; Mon, 25 Feb 2019 16:26:47 -0800 (PST) Received: from vm.nix.is ([2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id e6sm11745708wrt.14.2019.02.25.16.26.46 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Mon, 25 Feb 2019 16:26:47 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmc=?= =?utf-8?b?4buNYyBEdXk=?= , Matheus Tavares , Thomas Gummerer , Christian Couder , =?utf-8?b?w4Z2YXIgQXJuZmo=?= =?utf-8?b?w7Zyw7AgQmphcm1hc29u?= Subject: [WIP RFC PATCH 2/7] dir-iterator: use stat() instead of lstat() Date: Tue, 26 Feb 2019 01:26:20 +0100 Message-Id: <20190226002625.13022-3-avarab@gmail.com> X-Mailer: git-send-email 2.21.0.rc2.261.ga7da99ff1b In-Reply-To: References: MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This is surely a horrible idea, but all tests pass with this (not that I trust them much). Doing this for later WIP use in clone.c. Signed-off-by: Ævar Arnfjörð Bjarmason --- dir-iterator.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/dir-iterator.c b/dir-iterator.c index 070a656555..6a9c0c4d08 100644 --- a/dir-iterator.c +++ b/dir-iterator.c @@ -149,7 +149,7 @@ int dir_iterator_advance(struct dir_iterator *dir_iterator) continue; strbuf_addstr(&iter->base.path, de->d_name); - if (lstat(iter->base.path.buf, &iter->base.st) < 0) { + if (stat(iter->base.path.buf, &iter->base.st) < 0) { if (errno != ENOENT) { if (iter->pedantic) goto error_out; From patchwork Tue Feb 26 00:26:21 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10829323 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 829281575 for ; Tue, 26 Feb 2019 00:26:55 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 714962B99B for ; Tue, 26 Feb 2019 00:26:55 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 654C82B9AE; Tue, 26 Feb 2019 00:26:55 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0DD082B99B for ; Tue, 26 Feb 2019 00:26:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729352AbfBZA0y (ORCPT ); Mon, 25 Feb 2019 19:26:54 -0500 Received: from mail-wr1-f67.google.com ([209.85.221.67]:39021 "EHLO mail-wr1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728953AbfBZA0v (ORCPT ); Mon, 25 Feb 2019 19:26:51 -0500 Received: by mail-wr1-f67.google.com with SMTP id l5so11955745wrw.6 for ; Mon, 25 Feb 2019 16:26:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=B0YQwzEt1WN9Hjju18SV/huk2vsIYdBsYKe2jpwGovI=; b=Y2hqQroSrXya+dLqdPlJVsgVqmAJ0jyBEPHY0FJRFZcellIojdFNehus1IiscqU3l1 N/ASkbVJhSfKVE9FO7xqR6aczyQ2x94M8IhHLSOGCMb2HvQ09nrLdjNlC21GBgMvE5n+ P744rzxxbX/OwrxUXoSRRM+E+1WCyPuZmcSN4u8Hl/idqn515jDagqbRyvXcVi5VmICP P0V71JH4mZi4b+3/vfYRJqtfxS13s5zYipLbJFNwJutDZ8xkFoMwk1w+qaepA2Sns/T2 KrgxB9SA+TyDxlRwjzihBvcq5N0Y7ZhhViHZVOl7Ev55UH3yHKxxLmhXtn7J0d3kSuQT iqWQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=B0YQwzEt1WN9Hjju18SV/huk2vsIYdBsYKe2jpwGovI=; b=YjDWu0fbxnF8t/hohUhLBtyY7oIXtdbcOMXglIXqRmpMXnIspoDlEj/HVokvWif/iy OuUyLz8Fcd0DN7SjC5Ctyqlr4L4TbkNiAqeb56E1ipTLfi+aoxTj/ASOIDpYfTHAstnV UOXSG6ivySiTCyX3gb4tNbJcRW2u8LCRcefkw82NANoB+q6ccVf0f82j2Zk5zQzdQQFG uBfzNgkkRlnyf0kDQry73yOgIYT5UI4mnVE1u/AaScUS+BUshaST35YPXhpqGecUNXl2 B/lPMRKgXgqHQiLLhJw5Jp42+UAMVS7wziQlGH4w63N2zZH1sfM8urYLK4KC+QWz1uho X/pw== X-Gm-Message-State: AHQUAuYVXanLhA2u2I6F9y7juAI3XzaBu8uDbLRKQQMkPvUi/VKJhVP0 p1gqcbt1fCwPaWGrybqSHfRrLjSJugY= X-Google-Smtp-Source: AHgI3IbXBI1XqSa/6mldsyMNiUJ0XvUkWIlxiYl1i3pGEZeRhy7jxVinyaASy5zMJ7HxCDtt1yLSlg== X-Received: by 2002:adf:f4c8:: with SMTP id h8mr15455541wrp.6.1551140809315; Mon, 25 Feb 2019 16:26:49 -0800 (PST) Received: from vm.nix.is ([2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id e6sm11745708wrt.14.2019.02.25.16.26.47 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Mon, 25 Feb 2019 16:26:48 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmc=?= =?utf-8?b?4buNYyBEdXk=?= , Matheus Tavares , Thomas Gummerer , Christian Couder Subject: [WIP RFC PATCH 3/7] clone: extract function from copy_or_link_directory Date: Tue, 26 Feb 2019 01:26:21 +0100 Message-Id: <20190226002625.13022-4-avarab@gmail.com> X-Mailer: git-send-email 2.21.0.rc2.261.ga7da99ff1b In-Reply-To: References: MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Matheus Tavares Extract dir creation code snippet from copy_or_link_directory to its own function named mkdir_if_missing. This change will help removing copy_or_link_directory's explicit recursion, which will be done in a following patch. Also makes code more readable. Signed-off-by: Matheus Tavares --- builtin/clone.c | 27 +++++++++++++++++++-------- 1 file changed, 19 insertions(+), 8 deletions(-) diff --git a/builtin/clone.c b/builtin/clone.c index 50bde99618..862d2ea69c 100644 --- a/builtin/clone.c +++ b/builtin/clone.c @@ -392,6 +392,24 @@ static void copy_alternates(struct strbuf *src, struct strbuf *dst, fclose(in); } +static void mkdir_if_missing(const char *pathname, mode_t mode) +{ + /* + * Create a dir at pathname unless there's already one. + */ + struct stat st; + + if (mkdir(pathname, mode)) { + if (errno != EEXIST) + die_errno(_("failed to create directory '%s'"), + pathname); + else if (stat(pathname, &st)) + die_errno(_("failed to stat '%s'"), pathname); + else if (!S_ISDIR(st.st_mode)) + die(_("%s exists and is not a directory"), pathname); + } +} + static void copy_or_link_directory(struct strbuf *src, struct strbuf *dest, const char *src_repo, int src_baselen) { @@ -404,14 +422,7 @@ static void copy_or_link_directory(struct strbuf *src, struct strbuf *dest, if (!dir) die_errno(_("failed to open '%s'"), src->buf); - if (mkdir(dest->buf, 0777)) { - if (errno != EEXIST) - die_errno(_("failed to create directory '%s'"), dest->buf); - else if (stat(dest->buf, &buf)) - die_errno(_("failed to stat '%s'"), dest->buf); - else if (!S_ISDIR(buf.st_mode)) - die(_("%s exists and is not a directory"), dest->buf); - } + mkdir_if_missing(dest->buf, 0777); strbuf_addch(src, '/'); src_len = src->len; From patchwork Tue Feb 26 00:26:22 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10829325 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 07C191575 for ; Tue, 26 Feb 2019 00:26:57 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EA4AA2B99B for ; Tue, 26 Feb 2019 00:26:56 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DEA532B9AE; Tue, 26 Feb 2019 00:26:56 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7DEE12B99B for ; Tue, 26 Feb 2019 00:26:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729343AbfBZA0x (ORCPT ); Mon, 25 Feb 2019 19:26:53 -0500 Received: from mail-wm1-f68.google.com ([209.85.128.68]:53291 "EHLO mail-wm1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729332AbfBZA0w (ORCPT ); Mon, 25 Feb 2019 19:26:52 -0500 Received: by mail-wm1-f68.google.com with SMTP id e74so682199wmg.3 for ; Mon, 25 Feb 2019 16:26:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=9vzRKvwGNszBCWl0xynz6wQvHytSZbm5ybB6pgFv9Z4=; b=bhuw39JheYX292rc0d0SkqgBy2nioYhNQfP+aXe1621vti5BW/+6gxG/vsdF+j467y XseVP3NA3dnnvittgNVtGOAHkIlqOvVqJrnCgdEu9abD8t0G6hXm0ig5wDg6ZFJoMSE8 7UZVPnxEnT9WOF7+ilTiwrdyVF0tvHEYTo25kTn3QjkbD789fLWWVa7m7VMCnH9Wpd94 PFqZS4v3D3dSbZOmSjYsRLE/pz7EAh8XhQCbRf58DC7neZxnsWP/QPm+1y1qXclH1B2z q2NQ3AeWGLWpyjipFmLcj82UiZdoYPhesgzxihPrc6D6asNAK+W7Bdc7AokQ61PcuLE3 YbvA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9vzRKvwGNszBCWl0xynz6wQvHytSZbm5ybB6pgFv9Z4=; b=HK4WGQLJgSMSW2RNy6jj46V8kUBSmsfJX2RE78pUlHUua0MxFhX5DWXliILGgym66X scbiFfFBcwhk2k2/hTUYI9l1Nbr13ek9I5xbf9RWdkycSLINnAnWB99TgIkrJy86H50i TsHtLTxuhaTNYzFj1t9IKG4BUneaA54Vbc+xBKyaxmx05pyBz9iKiQ4OzgrcfsRT52mq NW4OfVporzROgG5+L5u5+oHAtHctnkY/WTIt2DJW2UlkN9Q9fVG72iqj5T9xPhqFs4qp aJ2jFBWvaD37362MRpx/Vrs950kSuAqG7StfByR+U7FOJLRCZnFFEl/E4O/9/cTDEWle uwzA== X-Gm-Message-State: AHQUAuYHezohdmbMNtwsyYxuYsaBk8dh/yQuf00b7x1IFDhMXfNzzU+n OSnS8uJGPQckotlGaYVH8j0DHFybIV0= X-Google-Smtp-Source: AHgI3IbsSeR3AQOGubWF2yDRt2YjQe7mlUmho0vkm1BR+gnhyRrlA+KMZYV2CF+iN0AqiZmbl2l/UQ== X-Received: by 2002:a1c:80c4:: with SMTP id b187mr730886wmd.23.1551140810514; Mon, 25 Feb 2019 16:26:50 -0800 (PST) Received: from vm.nix.is ([2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id e6sm11745708wrt.14.2019.02.25.16.26.49 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Mon, 25 Feb 2019 16:26:49 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmc=?= =?utf-8?b?4buNYyBEdXk=?= , Matheus Tavares , Thomas Gummerer , Christian Couder , =?utf-8?b?w4Z2YXIgQXJuZmo=?= =?utf-8?b?w7Zyw7AgQmphcm1hc29u?= Subject: [WIP RFC PATCH 4/7] clone: test for our behavior on odd objects/* content Date: Tue, 26 Feb 2019 01:26:22 +0100 Message-Id: <20190226002625.13022-5-avarab@gmail.com> X-Mailer: git-send-email 2.21.0.rc2.261.ga7da99ff1b In-Reply-To: References: MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We've implicitly supported .git/objects/* content of symlinks since approximately forever, and when we do a copy of the repo we transfer those over, but aren't very consistent about other random stuff we find depending on if it's a "hidden" file or not. Let's add a test for that, which shouldn't read as an endorsement of what we're doing now, just asserts current behavior. Signed-off-by: Ævar Arnfjörð Bjarmason --- t/t5604-clone-reference.sh | 60 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 60 insertions(+) diff --git a/t/t5604-clone-reference.sh b/t/t5604-clone-reference.sh index 4320082b1b..6f9c77049e 100755 --- a/t/t5604-clone-reference.sh +++ b/t/t5604-clone-reference.sh @@ -221,4 +221,64 @@ test_expect_success 'clone, dissociate from alternates' ' ( cd C && git fsck ) ' +test_expect_success SHA1,SYMLINKS 'setup repo with manually symlinked objects/*' ' + git init S && + ( + cd S && + test_commit A && + git gc && + test_commit B && + ( + cd .git/objects && + mv 22/3b7836fb19fdf64ba2d3cd6173c6a283141f78 . && + ln -s ../3b7836fb19fdf64ba2d3cd6173c6a283141f78 22/ && + mv 40 forty && + ln -s forty 40 && + mv pack packs && + ln -s packs pack && + >.some-hidden-file && + >some-file && + mkdir .some-hidden-dir && + >.some-hidden-dir/some-file && + >.some-hidden-dir/.some-dot-file && + mkdir some-dir && + >some-dir/some-file && + >some-dir/.some-dot-file + ) + ) +' + +test_expect_success SHA1,SYMLINKS 'clone repo with manually symlinked objects/*' ' + for option in --local --no-hardlinks --shared --dissociate + do + git clone $option S S$option || return 1 && + git -C S$option fsck || return 1 + done && + find S-* -type l | sort >actual && + cat >expected <<-EOF && + S--dissociate/.git/objects/22/3b7836fb19fdf64ba2d3cd6173c6a283141f78 + S--local/.git/objects/22/3b7836fb19fdf64ba2d3cd6173c6a283141f78 + EOF + test_cmp expected actual && + find S-* -name "*some*" | sort >actual && + cat >expected <<-EOF && + S--dissociate/.git/objects/.some-hidden-file + S--dissociate/.git/objects/some-dir + S--dissociate/.git/objects/some-dir/.some-dot-file + S--dissociate/.git/objects/some-dir/some-file + S--dissociate/.git/objects/some-file + S--local/.git/objects/.some-hidden-file + S--local/.git/objects/some-dir + S--local/.git/objects/some-dir/.some-dot-file + S--local/.git/objects/some-dir/some-file + S--local/.git/objects/some-file + S--no-hardlinks/.git/objects/.some-hidden-file + S--no-hardlinks/.git/objects/some-dir + S--no-hardlinks/.git/objects/some-dir/.some-dot-file + S--no-hardlinks/.git/objects/some-dir/some-file + S--no-hardlinks/.git/objects/some-file + EOF + test_cmp expected actual +' + test_done From patchwork Tue Feb 26 00:26:23 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10829327 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1F349922 for ; Tue, 26 Feb 2019 00:26:58 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0BF722B99B for ; Tue, 26 Feb 2019 00:26:58 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 000B22B9AE; Tue, 26 Feb 2019 00:26:57 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7C29B2B99B for ; Tue, 26 Feb 2019 00:26:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729366AbfBZA04 (ORCPT ); Mon, 25 Feb 2019 19:26:56 -0500 Received: from mail-wr1-f65.google.com ([209.85.221.65]:39963 "EHLO mail-wr1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729333AbfBZA0y (ORCPT ); Mon, 25 Feb 2019 19:26:54 -0500 Received: by mail-wr1-f65.google.com with SMTP id q1so11961717wrp.7 for ; Mon, 25 Feb 2019 16:26:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=5Vaqn0NYc4sfnkvexQVfNnHA6qePpy/sFPXV9RNmxks=; b=Nf2CQsEyCbjksNq5G8PKriFACu9bYokJ3j+/9z4XPJOmw85IbBn++25wNmIGVBVPjB YBf0fqXdXJMEDhnKslfZYfh7tXPdZWeAeXBjuFIIkD3KHOIi8kJDstkl/kq3D6lZsmOz MokFUJgRdJwp+VZ+JWgRZl9XHB4jc/yFIjVo2odc1bNQqY8k7Sd023i7ozTLjrroDL1Z q6NpDrGoRq47BBm0BQr5BUpnepNILe4MFZK6ygD5K1b4MfAODtjr0pLo78GPUPh8Hz8B Cv742r21+ielLjs9pWAOxyjhqtUn3QePRbzdtU16jDTV2uDp/xt/67rmd7xcn+lrYpaC xbiA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=5Vaqn0NYc4sfnkvexQVfNnHA6qePpy/sFPXV9RNmxks=; b=tQkQbPxgndGSPmqgW+Pay/5PfR9iN4oGYZZECxDY+Zl6BOvYCH6z3qM6TMIiRW/Of6 zqmJ1u+3NlKMT45Qs+C6ucIo9iKWbSnJlFTL6S8o2E4SMcle9vcgPcadmATjG8tU7o/m SrutR+hT48T/BLN/WhJNQN5e39DdrS6p0at+0QkLgnJ1qRt9QEwANKz1ZRhW/L8qcjKR S9zlocK4MCRElVjfwShAqwVZgn59yaclKVMduTId14ti2IYm0OJHnD3fCT0x57obKvt3 JN7jr+N5GcJPNgtLea1o3UUo8F8bdnNwAKAAK6d0cCyZYe/F4O3+LIoTwud8CdlBvejx O8Vw== X-Gm-Message-State: AHQUAuaxEId/BvrxZgSqyzHW9ZL5HO1Ito+G4o1ImMCNCQHiuE63GZ56 of61Ygv0c5KGpdbVAGaSRlHj2o5vYwI= X-Google-Smtp-Source: AHgI3IbeVi179EUyg/5DfLTDGoVrpUgWOtFsewTryDNA+Le5Qbx+CMgQV6TAmSCnZ8sitv2coCMAgA== X-Received: by 2002:adf:8294:: with SMTP id 20mr14871414wrc.223.1551140811811; Mon, 25 Feb 2019 16:26:51 -0800 (PST) Received: from vm.nix.is ([2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id e6sm11745708wrt.14.2019.02.25.16.26.50 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Mon, 25 Feb 2019 16:26:50 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmc=?= =?utf-8?b?4buNYyBEdXk=?= , Matheus Tavares , Thomas Gummerer , Christian Couder , =?utf-8?b?w4Z2YXIgQXJuZmo=?= =?utf-8?b?w7Zyw7AgQmphcm1hc29u?= Subject: [WIP RFC PATCH 5/7] clone: use dir-iterator to avoid explicit dir traversal Date: Tue, 26 Feb 2019 01:26:23 +0100 Message-Id: <20190226002625.13022-6-avarab@gmail.com> X-Mailer: git-send-email 2.21.0.rc2.261.ga7da99ff1b In-Reply-To: References: MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Matheus Tavares Replace usage of opendir/readdir/closedir API to traverse directories recursively, at copy_or_link_directory function, by the dir-iterator API. This simplifies the code and avoid recursive calls to copy_or_link_directory. [Ævar: This should be bug-compatible with the existing "clone" behavior. The whole bit here with "iter->relative_path[0] == '.'" is a dirty hack. We don't copy dot-dirs, and then later on just blindly ignore ENOENT errors as we descend into them. That case really wants to be a is_dotdir_or_file_within() test instead] Now, copy_or_link_directory will call die() in case of an error on openddir, readdir or lstat, inside dir_iterator_advance. That means it will abort in case of an error trying to fetch any iteration entry. Signed-off-by: Matheus Tavares Signed-off-by: Ævar Arnfjörð Bjarmason --- builtin/clone.c | 55 +++++++++++++++++++++++++++++-------------------- 1 file changed, 33 insertions(+), 22 deletions(-) diff --git a/builtin/clone.c b/builtin/clone.c index 862d2ea69c..c32e9022b3 100644 --- a/builtin/clone.c +++ b/builtin/clone.c @@ -23,6 +23,8 @@ #include "transport.h" #include "strbuf.h" #include "dir.h" +#include "dir-iterator.h" +#include "iterator.h" #include "sigchain.h" #include "branch.h" #include "remote.h" @@ -411,42 +413,47 @@ static void mkdir_if_missing(const char *pathname, mode_t mode) } static void copy_or_link_directory(struct strbuf *src, struct strbuf *dest, - const char *src_repo, int src_baselen) + const char *src_repo) { - struct dirent *de; - struct stat buf; int src_len, dest_len; - DIR *dir; - - dir = opendir(src->buf); - if (!dir) - die_errno(_("failed to open '%s'"), src->buf); + struct dir_iterator *iter; + int iter_status; + struct stat st; mkdir_if_missing(dest->buf, 0777); + iter = dir_iterator_begin(src->buf, 1); + strbuf_addch(src, '/'); src_len = src->len; strbuf_addch(dest, '/'); dest_len = dest->len; - while ((de = readdir(dir)) != NULL) { + while ((iter_status = dir_iterator_advance(iter)) == ITER_OK) { strbuf_setlen(src, src_len); - strbuf_addstr(src, de->d_name); + strbuf_addstr(src, iter->relative_path); strbuf_setlen(dest, dest_len); - strbuf_addstr(dest, de->d_name); - if (stat(src->buf, &buf)) { + strbuf_addstr(dest, iter->relative_path); + + /* + * dir_iterator_advance already calls lstat to populate iter->st + * but, unlike stat, lstat does not checks for permissions on + * the given path. + */ + if (stat(src->buf, &st)) { warning (_("failed to stat %s\n"), src->buf); continue; } - if (S_ISDIR(buf.st_mode)) { - if (de->d_name[0] != '.') - copy_or_link_directory(src, dest, - src_repo, src_baselen); + + if (S_ISDIR(iter->st.st_mode)) { + if (iter->relative_path[0] == '.') + continue; + mkdir_if_missing(dest->buf, 0777); continue; } /* Files that cannot be copied bit-for-bit... */ - if (!strcmp(src->buf + src_baselen, "/info/alternates")) { + if (!strcmp(iter->relative_path, "info/alternates")) { copy_alternates(src, dest, src_repo); continue; } @@ -456,14 +463,18 @@ static void copy_or_link_directory(struct strbuf *src, struct strbuf *dest, if (!option_no_hardlinks) { if (!link(src->buf, dest->buf)) continue; - if (option_local > 0) - die_errno(_("failed to create link '%s'"), dest->buf); + if (option_local > 0 && errno != ENOENT) + warning_errno(_("failed to create link '%s'"), dest->buf); option_no_hardlinks = 1; } - if (copy_file_with_time(dest->buf, src->buf, 0666)) + if (copy_file_with_time(dest->buf, src->buf, 0666) && errno != ENOENT) die_errno(_("failed to copy file to '%s'"), dest->buf); } - closedir(dir); + + if (iter_status != ITER_DONE) { + strbuf_setlen(src, src_len); + die(_("failed to iterate over '%s'"), src->buf); + } } static void clone_local(const char *src_repo, const char *dest_repo) @@ -481,7 +492,7 @@ static void clone_local(const char *src_repo, const char *dest_repo) get_common_dir(&dest, dest_repo); strbuf_addstr(&src, "/objects"); strbuf_addstr(&dest, "/objects"); - copy_or_link_directory(&src, &dest, src_repo, src.len); + copy_or_link_directory(&src, &dest, src_repo); strbuf_release(&src); strbuf_release(&dest); } From patchwork Tue Feb 26 00:26:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10829331 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 63D6D1575 for ; Tue, 26 Feb 2019 00:27:01 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 51D1E2B99B for ; Tue, 26 Feb 2019 00:27:01 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 45E4A2B9AE; Tue, 26 Feb 2019 00:27:01 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D95692B99B for ; Tue, 26 Feb 2019 00:27:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729370AbfBZA06 (ORCPT ); Mon, 25 Feb 2019 19:26:58 -0500 Received: from mail-wr1-f67.google.com ([209.85.221.67]:35524 "EHLO mail-wr1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729348AbfBZA0z (ORCPT ); Mon, 25 Feb 2019 19:26:55 -0500 Received: by mail-wr1-f67.google.com with SMTP id t18so11958949wrx.2 for ; Mon, 25 Feb 2019 16:26:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=32aTWrOKLgNRXnOZkuCmjfI4neHybtVGVXDJqac5D7g=; b=QKmrFJa9NTWyokerYvJ53zLwlHcJ5+vPa8pIEYrNqyU7nIVbqjT9t46GCIf+9vbOPf nq25cyQ0UObw0w+ZDPaoQ95C2kaibSi2bMhLSvXNAlADg68LgxdZIszQbY4nJx6RoUIk hEf0NxuUnvjtsc8baebxRePCELemChzUEmA2HC9X3ZUuvCrleW+rpDZQyOuRw68t//l5 Ll1REom8Dm/gkIZwWFqO9Lota5KJLtVWzI8abc02+Fxrzy0ckbIwt+saxy+bZO4s8D3n Yu5e+OXX8oTSjbYNx8wYQrHwTnTr4kc7GSSP4zwYfusc5NJrUHeogAT2L7GQ3bUagpPm cNYQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=32aTWrOKLgNRXnOZkuCmjfI4neHybtVGVXDJqac5D7g=; b=VyLtvgv+j8HJf7h3rPxLDQFsf37F3ZiiEK4D7Oh2TCWaHdTJcVyv6I5W2lh/2nPAQW /t0vZBMQn11SAcbDylb/L3X2BtgJim2jj73vDfLXZVyp5ntzMUfKvSlQiZFEcbe1p7nq X74AaSm5Djz44TdCGfJ3HoDuy0cxiL6bcQdM103rj3uCeGlfBmOxVi76AH0HkCTpUMST 0xlfpmkeCtEQ5pXigg5gKVOVq8j4P6BWIUriYjs33mAVTDElcosyz9wkkI3sGZeDweHM 5LFauTL+QsrIChjpugjBzvwYpkwJ778P4JzrucMmNXQcGrYqvf4ZMXA59hdqGJaSOu/z nsug== X-Gm-Message-State: AHQUAuYSqrpOd0yrdhmPtBQo6INILVxCNdAMRd7473oAPkt1MrGssHAg pdMeldYSEaHvkD+dQhbcMaBUN9tXWIE= X-Google-Smtp-Source: AHgI3IYv+aQg4AeN7iBIFp7lW0qhqg6XdkbHh7DwEmVmf1jCKcwTTejLc4wA/HUfisecgCPJ1yb3aw== X-Received: by 2002:adf:dd8a:: with SMTP id x10mr15056059wrl.117.1551140813200; Mon, 25 Feb 2019 16:26:53 -0800 (PST) Received: from vm.nix.is ([2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id e6sm11745708wrt.14.2019.02.25.16.26.51 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Mon, 25 Feb 2019 16:26:52 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmc=?= =?utf-8?b?4buNYyBEdXk=?= , Matheus Tavares , Thomas Gummerer , Christian Couder , =?utf-8?b?w4Z2YXIgQXJuZmo=?= =?utf-8?b?w7Zyw7AgQmphcm1hc29u?= Subject: [WIP RFC PATCH 6/7] clone: stop ignoring dotdirs in --local etc. clone Date: Tue, 26 Feb 2019 01:26:24 +0100 Message-Id: <20190226002625.13022-7-avarab@gmail.com> X-Mailer: git-send-email 2.21.0.rc2.261.ga7da99ff1b In-Reply-To: References: MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This seems to never have been intentional, just a side-effect of how the existing code was written. Signed-off-by: Ævar Arnfjörð Bjarmason --- builtin/clone.c | 8 +++----- t/t5604-clone-reference.sh | 9 +++++++++ 2 files changed, 12 insertions(+), 5 deletions(-) diff --git a/builtin/clone.c b/builtin/clone.c index c32e9022b3..515dc91d63 100644 --- a/builtin/clone.c +++ b/builtin/clone.c @@ -446,8 +446,6 @@ static void copy_or_link_directory(struct strbuf *src, struct strbuf *dest, } if (S_ISDIR(iter->st.st_mode)) { - if (iter->relative_path[0] == '.') - continue; mkdir_if_missing(dest->buf, 0777); continue; } @@ -463,11 +461,11 @@ static void copy_or_link_directory(struct strbuf *src, struct strbuf *dest, if (!option_no_hardlinks) { if (!link(src->buf, dest->buf)) continue; - if (option_local > 0 && errno != ENOENT) - warning_errno(_("failed to create link '%s'"), dest->buf); + if (option_local > 0) + die_errno(_("failed to create link '%s'"), dest->buf); option_no_hardlinks = 1; } - if (copy_file_with_time(dest->buf, src->buf, 0666) && errno != ENOENT) + if (copy_file_with_time(dest->buf, src->buf, 0666)) die_errno(_("failed to copy file to '%s'"), dest->buf); } diff --git a/t/t5604-clone-reference.sh b/t/t5604-clone-reference.sh index 6f9c77049e..f1a8e74c44 100755 --- a/t/t5604-clone-reference.sh +++ b/t/t5604-clone-reference.sh @@ -262,16 +262,25 @@ test_expect_success SHA1,SYMLINKS 'clone repo with manually symlinked objects/*' test_cmp expected actual && find S-* -name "*some*" | sort >actual && cat >expected <<-EOF && + S--dissociate/.git/objects/.some-hidden-dir + S--dissociate/.git/objects/.some-hidden-dir/.some-dot-file + S--dissociate/.git/objects/.some-hidden-dir/some-file S--dissociate/.git/objects/.some-hidden-file S--dissociate/.git/objects/some-dir S--dissociate/.git/objects/some-dir/.some-dot-file S--dissociate/.git/objects/some-dir/some-file S--dissociate/.git/objects/some-file + S--local/.git/objects/.some-hidden-dir + S--local/.git/objects/.some-hidden-dir/.some-dot-file + S--local/.git/objects/.some-hidden-dir/some-file S--local/.git/objects/.some-hidden-file S--local/.git/objects/some-dir S--local/.git/objects/some-dir/.some-dot-file S--local/.git/objects/some-dir/some-file S--local/.git/objects/some-file + S--no-hardlinks/.git/objects/.some-hidden-dir + S--no-hardlinks/.git/objects/.some-hidden-dir/.some-dot-file + S--no-hardlinks/.git/objects/.some-hidden-dir/some-file S--no-hardlinks/.git/objects/.some-hidden-file S--no-hardlinks/.git/objects/some-dir S--no-hardlinks/.git/objects/some-dir/.some-dot-file From patchwork Tue Feb 26 00:26:25 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= X-Patchwork-Id: 10829329 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 738511575 for ; Tue, 26 Feb 2019 00:27:00 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 60F142B99B for ; Tue, 26 Feb 2019 00:27:00 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5509D2B9AE; Tue, 26 Feb 2019 00:27:00 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CA48D2B99B for ; Tue, 26 Feb 2019 00:26:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729376AbfBZA07 (ORCPT ); Mon, 25 Feb 2019 19:26:59 -0500 Received: from mail-wr1-f67.google.com ([209.85.221.67]:38687 "EHLO mail-wr1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728953AbfBZA05 (ORCPT ); Mon, 25 Feb 2019 19:26:57 -0500 Received: by mail-wr1-f67.google.com with SMTP id v13so11954801wrw.5 for ; Mon, 25 Feb 2019 16:26:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=zKIRNNhDBE5EinqM5VZ4bhKdPIiXbQLoYZbNewwpzfA=; b=p+ZB0gsLjTS6fzzvO66rZj3ubHSQgX3Rl1W8PI1PRMZvx7D+jAu00ceL9g+uZDaFc6 WHnoyefIqoY98wjXJL8UNdcrnUAUwqpQMtSwAmM/ia9Y8LDcrpn3lRVPtcpj6Cb134GV njXgEBLjKU+dkAAYkU8OLGEto5TWBRzPoXAzsiYRcj1DyGEvq7FUuejURiSkZ8d+WcU8 k9JKsF0RlOVZYw406FrDfe7DI6WPUmirk5wEGFotyF6f3lhaqlx3nVxI2Fb7F5DTC6AV lrZFQyy9J2ZApkG+ZDOZ05ww2LhdD4nLpTkZAefJNptJuP0NVj7Zg7xie954u2fJvAgJ unJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=zKIRNNhDBE5EinqM5VZ4bhKdPIiXbQLoYZbNewwpzfA=; b=njkKizXustTnA87306r6xQO71LNrL0VXaK09x6x2nT9mIoSiGWv7mGpx62OXF53k3y BbAtUk0uR/oD3qWJzi5/hQYorqDXDgg8mWyKbm2g9YquI62NXLnumgLdzB1+4FTMslaY pKMHmyfT53RGGJ7qjMMbJliPHjgt00ExwRPuhldHR9M+Nkxr6/MCDiEUDlRyW7TQJMFK g4iMiD6uOOd+0vPeciLl31r+TaQIXMSapJ4fN43WWa72re/A0LvBaRRv7rXQGUviNEYj LtAr28TFF5SnzoYbdxKzLMwNATLDDc39utuiBUlf/hMh3ozwNWq+CYB6yRzgQFvcqOGw 8oiQ== X-Gm-Message-State: AHQUAuaAZvRMOCp1wHn3VtOuESiYh8+Lhi7XJ5SIEyz0ITEKa+VuJ+cp YgADVGSb98Hk5Kj0xBBdhZLYkCGba7Q= X-Google-Smtp-Source: AHgI3IYg9vWy1xWgzlJtIjdh8P0ReaK+3028z2pVPzoHzDVlOHl3zW0uM3ZFxxyziFyFZV8KuyALdQ== X-Received: by 2002:adf:e98c:: with SMTP id h12mr15075359wrm.302.1551140814498; Mon, 25 Feb 2019 16:26:54 -0800 (PST) Received: from vm.nix.is ([2a01:4f8:120:2468::2]) by smtp.gmail.com with ESMTPSA id e6sm11745708wrt.14.2019.02.25.16.26.53 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Mon, 25 Feb 2019 16:26:53 -0800 (PST) From: =?utf-8?b?w4Z2YXIgQXJuZmrDtnLDsCBCamFybWFzb24=?= To: git@vger.kernel.org Cc: Junio C Hamano , =?utf-8?b?Tmd1eeG7hW4gVGjDoWkgTmc=?= =?utf-8?b?4buNYyBEdXk=?= , Matheus Tavares , Thomas Gummerer , Christian Couder , =?utf-8?b?w4Z2YXIgQXJuZmo=?= =?utf-8?b?w7Zyw7AgQmphcm1hc29u?= Subject: [WIP RFC PATCH 7/7] clone: break cloning repos that have symlinks in them Date: Tue, 26 Feb 2019 01:26:25 +0100 Message-Id: <20190226002625.13022-8-avarab@gmail.com> X-Mailer: git-send-email 2.21.0.rc2.261.ga7da99ff1b In-Reply-To: References: MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Revert back to the lstat() behavior in the dir_iterator interface we've had since it was added in 0fe5043dad ("dir_iterator: new API for iterating over a directory tree", 2016-06-18). No reflog test depends on it, so it's unclear if it's needed. Since clone now uses this it changes the longstanding behavior of how we just so happened to support objects dirs with symlinks in them. Why? I don't know. Just writing this up as an RFC with tests. Insert rationale here. Signed-off-by: Ævar Arnfjörð Bjarmason --- dir-iterator.c | 2 +- t/t5604-clone-reference.sh | 45 +++++++------------------------------- 2 files changed, 9 insertions(+), 38 deletions(-) diff --git a/dir-iterator.c b/dir-iterator.c index 6a9c0c4d08..070a656555 100644 --- a/dir-iterator.c +++ b/dir-iterator.c @@ -149,7 +149,7 @@ int dir_iterator_advance(struct dir_iterator *dir_iterator) continue; strbuf_addstr(&iter->base.path, de->d_name); - if (stat(iter->base.path.buf, &iter->base.st) < 0) { + if (lstat(iter->base.path.buf, &iter->base.st) < 0) { if (errno != ENOENT) { if (iter->pedantic) goto error_out; diff --git a/t/t5604-clone-reference.sh b/t/t5604-clone-reference.sh index f1a8e74c44..a4cd12643e 100755 --- a/t/t5604-clone-reference.sh +++ b/t/t5604-clone-reference.sh @@ -249,45 +249,16 @@ test_expect_success SHA1,SYMLINKS 'setup repo with manually symlinked objects/*' ' test_expect_success SHA1,SYMLINKS 'clone repo with manually symlinked objects/*' ' - for option in --local --no-hardlinks --shared --dissociate + for option in --local --no-hardlinks --dissociate do - git clone $option S S$option || return 1 && - git -C S$option fsck || return 1 + test_must_fail git clone $option S S$option 2>err || return 1 && + test_i18ngrep "the remote end hung up" err || return 1 done && - find S-* -type l | sort >actual && - cat >expected <<-EOF && - S--dissociate/.git/objects/22/3b7836fb19fdf64ba2d3cd6173c6a283141f78 - S--local/.git/objects/22/3b7836fb19fdf64ba2d3cd6173c6a283141f78 - EOF - test_cmp expected actual && - find S-* -name "*some*" | sort >actual && - cat >expected <<-EOF && - S--dissociate/.git/objects/.some-hidden-dir - S--dissociate/.git/objects/.some-hidden-dir/.some-dot-file - S--dissociate/.git/objects/.some-hidden-dir/some-file - S--dissociate/.git/objects/.some-hidden-file - S--dissociate/.git/objects/some-dir - S--dissociate/.git/objects/some-dir/.some-dot-file - S--dissociate/.git/objects/some-dir/some-file - S--dissociate/.git/objects/some-file - S--local/.git/objects/.some-hidden-dir - S--local/.git/objects/.some-hidden-dir/.some-dot-file - S--local/.git/objects/.some-hidden-dir/some-file - S--local/.git/objects/.some-hidden-file - S--local/.git/objects/some-dir - S--local/.git/objects/some-dir/.some-dot-file - S--local/.git/objects/some-dir/some-file - S--local/.git/objects/some-file - S--no-hardlinks/.git/objects/.some-hidden-dir - S--no-hardlinks/.git/objects/.some-hidden-dir/.some-dot-file - S--no-hardlinks/.git/objects/.some-hidden-dir/some-file - S--no-hardlinks/.git/objects/.some-hidden-file - S--no-hardlinks/.git/objects/some-dir - S--no-hardlinks/.git/objects/some-dir/.some-dot-file - S--no-hardlinks/.git/objects/some-dir/some-file - S--no-hardlinks/.git/objects/some-file - EOF - test_cmp expected actual + git clone --shared S S--shared && + find S--shared -type l | sort >actual && + test_must_be_empty actual && + find S--shared -name "*some*" | sort >actual && + test_must_be_empty actual ' test_done