From patchwork Sun Nov 11 06:23:03 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elijah Newren X-Patchwork-Id: 10677515 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8092346E4 for ; Sun, 11 Nov 2018 06:23:38 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 70CB72AFA1 for ; Sun, 11 Nov 2018 06:23:38 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 64E5E2B2DE; Sun, 11 Nov 2018 06:23:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C427E2AFEF for ; Sun, 11 Nov 2018 06:23:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727398AbeKKQK7 (ORCPT ); Sun, 11 Nov 2018 11:10:59 -0500 Received: from mail-ot1-f66.google.com ([209.85.210.66]:47031 "EHLO mail-ot1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727307AbeKKQK6 (ORCPT ); Sun, 11 Nov 2018 11:10:58 -0500 Received: by mail-ot1-f66.google.com with SMTP id w25so4533531otm.13 for ; Sat, 10 Nov 2018 22:23:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Dle/TW6DhgtBpPyGtMvUvHpq4r6OeTZX9pV+XBu/SUc=; b=Z0y3VDo5PJQ+7Wc7+IlHAlHxbpKfBa8QuZS/nruHpGIioxYceoYoyQfXnvtyh0m2/U FacW/I6bZA5b4lGQkXIErkrwI/609JfoSVPNjRfYu7BbBIBb8pedZ60UtmHFljt1gRpQ Ex29LLKFDkZjZL7BYdFFwNeTCI88t9+id95Vx8tcXZHN9a27yFZLy/6L1Rh0xEXdR36X csNgIYaNrffq5rqSM7Tgdp0wgi9MJOpVR5RleTzx0DIGDfBj5EsRPZDi2T77n3IJYNqW Y+/wDRQvS9VhoPMtjzs3MC2rblI62tEu8xtnAF+3kZcKRkrxAhpWqum/MkXM3S8+Jr/O lhkA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Dle/TW6DhgtBpPyGtMvUvHpq4r6OeTZX9pV+XBu/SUc=; b=Q+2ucOJI3lunY64Nhv75MhosQn9wjlvhFRuG8R+5v7F6/ywVfFsyUeZ1BCiOh//MkU PWNI+FUJ2DKD5/wasCCxZKCL2Hnqjkro8PJgM8siY4tYxgjQAh19KF6ffBd3VVnF+/EA 8n2+/HqpdTPB7p0WUV/HKw1ynkmh0C/1b+LdysGT4R4wWmY4U0QXcPWly9lF4oB2e0fR 7jn2NOdFCvdXl/1lokUxofr0irlojS7pavgjuj+lsr1L7E9wu86NSxXjDeq3ChZ5DE2G HZjKidoV2WnIqK7V41YXe97deNUj1hrYUjJvvN9LxuNznZi97Y+27ZwTaBNgqxiQoYDD fGXQ== X-Gm-Message-State: AGRZ1gI1eVLqcuzqPVB8xieq1iMvAvYN/H9fERrfpML3suI4tV0XXxg3 s0xZumI6NQlOOX+16hxenaDkgi8b X-Google-Smtp-Source: AJdET5fHxwvyOo3YXuF2fZ4ODu8AiVzBPdiCndHOlCi0uO55Yb9jbG6cwBGttOhNABsfm5R8L/W1iQ== X-Received: by 2002:a9d:738a:: with SMTP id j10mr9494838otk.188.1541917400060; Sat, 10 Nov 2018 22:23:20 -0800 (PST) Received: from tiger.attlocal.net ([2602:30a:2c28:20f0:7c1a:85e3:2ea9:5d7e]) by smtp.gmail.com with ESMTPSA id f34sm1286855otb.25.2018.11.10.22.23.18 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sat, 10 Nov 2018 22:23:19 -0800 (PST) From: Elijah Newren To: git@vger.kernel.org Cc: larsxschneider@gmail.com, sandals@crustytoothpaste.net, peff@peff.net, me@ttaylorr.com, jrnieder@gmail.com, Elijah Newren Subject: [PATCH 01/10] git-fast-import.txt: fix documentation for --quiet option Date: Sat, 10 Nov 2018 22:23:03 -0800 Message-Id: <20181111062312.16342-2-newren@gmail.com> X-Mailer: git-send-email 2.19.1.866.g82735bcbde In-Reply-To: <20181111062312.16342-1-newren@gmail.com> References: <20181111062312.16342-1-newren@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Signed-off-by: Elijah Newren --- Documentation/git-fast-import.txt | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/Documentation/git-fast-import.txt b/Documentation/git-fast-import.txt index e81117d27f..7ab97745a6 100644 --- a/Documentation/git-fast-import.txt +++ b/Documentation/git-fast-import.txt @@ -40,9 +40,10 @@ OPTIONS not contain the old commit). --quiet:: - Disable all non-fatal output, making fast-import silent when it - is successful. This option disables the output shown by - --stats. + Disable the output shown by --stats, making fast-import usually + be silent when it is successful. However, if the import stream + has directives intended to show user output (e.g. `progress` + directives), the corresponding messages will still be shown. --stats:: Display some basic statistics about the objects fast-import has From patchwork Sun Nov 11 06:23:04 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elijah Newren X-Patchwork-Id: 10677497 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F3911109C for ; Sun, 11 Nov 2018 06:23:36 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E405E2B69B for ; Sun, 11 Nov 2018 06:23:36 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D4DCA2B6F4; Sun, 11 Nov 2018 06:23:36 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E44712B2DE for ; Sun, 11 Nov 2018 06:23:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727440AbeKKQLA (ORCPT ); Sun, 11 Nov 2018 11:11:00 -0500 Received: from mail-ot1-f65.google.com ([209.85.210.65]:35681 "EHLO mail-ot1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727307AbeKKQK7 (ORCPT ); Sun, 11 Nov 2018 11:10:59 -0500 Received: by mail-ot1-f65.google.com with SMTP id 81so5291659otj.2 for ; Sat, 10 Nov 2018 22:23:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=GuoJPXfhSLP0jMlhy/1pwwfT7gnHll7lj2zkReDrCic=; b=ttOhZ4I2S+c+su5jAdL8CaRu/+VTy63MpaPM4zC49ufgZroMZQnbxocFaI8/AP/dnN SE4MH7J+ai2p/BwYw3019k6oODgj5vaHqQM8T0D+h/hdzMV1PhkxYfvOYKAUy45XD+gL ztclLFDLyJ6uchhsQeXOdDsEiFJ4Rqop8DbYXiEj3dIDJNM/dImZpBNiw0tZrBGgL557 S1UFILATj3JBwohpcj6Psmz0UaJRYuNRWBInETdytbi+aFuABBVjEGHDLygpnEIcifpf FMrqdFqHf5bYhw3Za7P8PTXo6tFgL6I7HLAkrsY5KFOVLh3qzxLMlJ1tU+BsIPQqEzha vUNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=GuoJPXfhSLP0jMlhy/1pwwfT7gnHll7lj2zkReDrCic=; b=gmBm5tAXa6OWJeIMeUekvWH/NSxIdyCt4LpBfBYw8/oWZFk/sy2i1n3LD+R8txwuj5 9fsjxuiVNVqCpxAHDr0AUIsAklspADNQcUum9dI0HZjse2lJZ7rmlfwPV5FvqVPnW3o2 JF7sOHTCivJuXUmc8F5Neg/uZ6v8vjgG+3djXf/k94AR80tLvKSyUYtjXLyismFTakQY MZ61yXfoCzfe4DuPRBTPPiI+JQMXzfwN9mRE9lsR3FbrR0rGIcYhDXixnKC2ie0+slZU IiG/C5YHg6OR4LHPH1fagdbUCdKn7/1e2ye4oNnTB/eyG6zp77BDmp3uwSo7RdVO1w5t dlEg== X-Gm-Message-State: AGRZ1gL4tsW55bSxswxboG9FgR1B/6E8gcmQ8sBUYUdzOrwB20vxLDkl t+Ghc8A0wsO1SKgpBVmpBjNU3JTn X-Google-Smtp-Source: AJdET5cvvkvGVDboIecU82hecuA/nFV+OeDW7sxL0wIbmRkGm6fSWOFUs6Q2JdhTunjewfmquk3vOg== X-Received: by 2002:a9d:5f06:: with SMTP id f6mr9594623oti.258.1541917401319; Sat, 10 Nov 2018 22:23:21 -0800 (PST) Received: from tiger.attlocal.net ([2602:30a:2c28:20f0:7c1a:85e3:2ea9:5d7e]) by smtp.gmail.com with ESMTPSA id f34sm1286855otb.25.2018.11.10.22.23.20 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sat, 10 Nov 2018 22:23:20 -0800 (PST) From: Elijah Newren To: git@vger.kernel.org Cc: larsxschneider@gmail.com, sandals@crustytoothpaste.net, peff@peff.net, me@ttaylorr.com, jrnieder@gmail.com, Elijah Newren Subject: [PATCH 02/10] git-fast-export.txt: clarify misleading documentation about rev-list args Date: Sat, 10 Nov 2018 22:23:04 -0800 Message-Id: <20181111062312.16342-3-newren@gmail.com> X-Mailer: git-send-email 2.19.1.866.g82735bcbde In-Reply-To: <20181111062312.16342-1-newren@gmail.com> References: <20181111062312.16342-1-newren@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Signed-off-by: Elijah Newren --- Documentation/git-fast-export.txt | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/Documentation/git-fast-export.txt b/Documentation/git-fast-export.txt index ce954be532..677510b7f7 100644 --- a/Documentation/git-fast-export.txt +++ b/Documentation/git-fast-export.txt @@ -119,7 +119,8 @@ marks the same across runs. 'git rev-list', that specifies the specific objects and references to export. For example, `master~10..master` causes the current master reference to be exported along with all objects - added since its 10th ancestor commit. + added since its 10th ancestor commit and all files common to + master\~9 and master~10. EXAMPLES -------- From patchwork Sun Nov 11 06:23:05 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elijah Newren X-Patchwork-Id: 10677509 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C980F139B for ; Sun, 11 Nov 2018 06:23:37 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BAD2A2AFA1 for ; Sun, 11 Nov 2018 06:23:37 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AF3832B2DE; Sun, 11 Nov 2018 06:23:37 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 10FF62B4E8 for ; Sun, 11 Nov 2018 06:23:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727462AbeKKQLB (ORCPT ); Sun, 11 Nov 2018 11:11:01 -0500 Received: from mail-oi1-f196.google.com ([209.85.167.196]:39032 "EHLO mail-oi1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727307AbeKKQLB (ORCPT ); Sun, 11 Nov 2018 11:11:01 -0500 Received: by mail-oi1-f196.google.com with SMTP id 192-v6so4763941oii.6 for ; Sat, 10 Nov 2018 22:23:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=9+LUPzmLQAjcx0lP9hKzfHOMaos3NdiJIOHNt17kb2s=; b=ke6YPX/do6LwPRqR1NFOVQ6XZJW08hWPIY7TKDPBzJ4WphBu9eiRJLe/6/wzEEqOZ9 TN8sHnp+gZjn4FxDfeujPUBOliGrYnXB59n/38xKb587xudlLeweK1LtH1YtiVmXodV4 6r/etM/0rhTRIdixHz2vtQpkEv5iIuUta0Tfd9PiVkmtgafHqfatK7ANcGNT6Erw9sXS DINdpFaVcV2K/tE/ErQoGl28usGVTtKzgWol7xZBrF7kxxpvRoPSLiLdrjT1ldQf7cMn APO9jv/G6Lu8RnWJxB2zcH2hp7BI8GI6+MUdNtIkpAXWnFz8tiQtoDHSTQXzaiUzAZPR 3BRg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9+LUPzmLQAjcx0lP9hKzfHOMaos3NdiJIOHNt17kb2s=; b=T8S5Ub1bgmt9TQUfEOWTFpCecdAPNAQh10+VMMRKc6tW9i4G/hym4a3m+HyUwBCz+r s0YzfZ1mIvFYEP5Aa54JP4nLbDZm4JhSIp2pNFmp8QHg+KZSb3vo27DUZt64Gkdoatmp PWYelabFfjA4S7i0qDI3CB6dDjjF7zX4nN/Cy3VtHhooKA4hsf74E3/zLHno6z8PekX1 9qTOZ0sY8AQLegUU/QWWztUzz9IkIVdnD7nxU8JHU/dhhshBBhQbM9hC+cvJKTclcCpQ NTt65tOZlpQrH0culTAY4y3lZAnj+xjgKARn6QzSs8lKZqcqXgPLIBZKuUOVNdBmzwAD Ld1w== X-Gm-Message-State: AGRZ1gJSdKOqVzRfJRNJTab6CqZnKDESZOux6SsuKI4KxcV3iwjJEMDs QiB4SXUppPn0iYK+Zi8MOTKasany X-Google-Smtp-Source: AJdET5fwV0RGet1utUelrWEHLEWhvTd9pWDpEs0MbW9noJUSDIeYba67ZjgGv8UO2dE4V0l4fvor5w== X-Received: by 2002:aca:62c3:: with SMTP id w186-v6mr8959267oib.320.1541917402585; Sat, 10 Nov 2018 22:23:22 -0800 (PST) Received: from tiger.attlocal.net ([2602:30a:2c28:20f0:7c1a:85e3:2ea9:5d7e]) by smtp.gmail.com with ESMTPSA id f34sm1286855otb.25.2018.11.10.22.23.21 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sat, 10 Nov 2018 22:23:22 -0800 (PST) From: Elijah Newren To: git@vger.kernel.org Cc: larsxschneider@gmail.com, sandals@crustytoothpaste.net, peff@peff.net, me@ttaylorr.com, jrnieder@gmail.com, Elijah Newren Subject: [PATCH 03/10] fast-export: use value from correct enum Date: Sat, 10 Nov 2018 22:23:05 -0800 Message-Id: <20181111062312.16342-4-newren@gmail.com> X-Mailer: git-send-email 2.19.1.866.g82735bcbde In-Reply-To: <20181111062312.16342-1-newren@gmail.com> References: <20181111062312.16342-1-newren@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP ABORT and ERROR happen to have the same value, but come from differnt enums. Use the one from the correct enum. Signed-off-by: Elijah Newren --- builtin/fast-export.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/builtin/fast-export.c b/builtin/fast-export.c index 456797c12a..1a299c2a21 100644 --- a/builtin/fast-export.c +++ b/builtin/fast-export.c @@ -752,7 +752,7 @@ static void handle_tag(const char *name, struct tag *tag) tagged_mark = get_object_mark(tagged); if (!tagged_mark) { switch(tag_of_filtered_mode) { - case ABORT: + case ERROR: die("tag %s tags unexported object; use " "--tag-of-filtered-object= to handle it", oid_to_hex(&tag->object.oid)); From patchwork Sun Nov 11 06:23:06 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elijah Newren X-Patchwork-Id: 10677511 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E1FC31926 for ; Sun, 11 Nov 2018 06:23:37 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D2B372AFA1 for ; Sun, 11 Nov 2018 06:23:37 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C4F0C2B688; Sun, 11 Nov 2018 06:23:37 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4664A2B67A for ; Sun, 11 Nov 2018 06:23:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727492AbeKKQLD (ORCPT ); Sun, 11 Nov 2018 11:11:03 -0500 Received: from mail-ot1-f68.google.com ([209.85.210.68]:45795 "EHLO mail-ot1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727307AbeKKQLD (ORCPT ); Sun, 11 Nov 2018 11:11:03 -0500 Received: by mail-ot1-f68.google.com with SMTP id 32so4721769ota.12 for ; Sat, 10 Nov 2018 22:23:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=zf8DEGUnzTd5YbCJ6JXA1bfCOXUQG87b2q3bqn4b1Js=; b=bpUblVRW+smTlzMT3Y3VHGEtHx8RB1VaBHsp0uFHmA9SdSkHGLoicsY6oTHFz8mb+T zxA18sxlgIoDyloO4hMkf8cbKmFJK/2UPiTrYezcodR6nP38ERtwBlDtBAODmrjfMM8K 58Xc3DoIxutZOE9AEwBGIQUnFz8nuNJl53i5+a+o/zhtB1w9v6lxTrX9nlhRM4Y7BnNt R14gseTT0wMGDUZD4tmx7K5cgcu80kWv8X7TLhsm5MjTl1qhKxiyntTrSMmr/tyHriez H2hSwkBqsoXfQ2fZOqyB+Ju7GBBPosYjfli5EH+sb8RwiIUnEwAL1ItJqS96RzrPAYpk k+iQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=zf8DEGUnzTd5YbCJ6JXA1bfCOXUQG87b2q3bqn4b1Js=; b=cOao1FbPCnnb2EHUMlvTjvVTSJuQo8DAGzXsxjnOY4jryz6Hl8V4zXYDDLHg8hpTlO dVm0qvetUVDESF0qCHnQz98O0FJkzwhkl5e/TjzwXRJiS7TeCz7AOjckIJUC25N4oe0A BC1E6V3+oA+zt9Jj/yOGp3bfLWpxiqCOVq/mtFQfarprkoPihnve5wWlWEqps9HPox59 8nynE5cInO+ob+WBU642tRa1FP/nzBLVLmmCyOQuNg6oD2epEBbwHt67yOFEgh2IZt9Y fWX8wHLBtguAetaFUwpD00QJaHHV17jDXTMh1EdMTrZl47tIBoRipTpt5wnEhqWk0oDI RjOA== X-Gm-Message-State: AGRZ1gJSeOw6EYbsb5Xiza9TtbWIO4sRP1mKlwUzTr075dBDCZdBj6ta pAlfYir0hPXkWC6qbLcGSlDCY7BN X-Google-Smtp-Source: AJdET5fHY0DCLFkMBxKW1T5eFq3/k+WDa0/+o/PTAzy9jPS6UaRBLhMrEZ45Bw5Rpt7qEVDH+mgr/w== X-Received: by 2002:a9d:7502:: with SMTP id r2mr9406542otk.273.1541917403924; Sat, 10 Nov 2018 22:23:23 -0800 (PST) Received: from tiger.attlocal.net ([2602:30a:2c28:20f0:7c1a:85e3:2ea9:5d7e]) by smtp.gmail.com with ESMTPSA id f34sm1286855otb.25.2018.11.10.22.23.22 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sat, 10 Nov 2018 22:23:23 -0800 (PST) From: Elijah Newren To: git@vger.kernel.org Cc: larsxschneider@gmail.com, sandals@crustytoothpaste.net, peff@peff.net, me@ttaylorr.com, jrnieder@gmail.com, Elijah Newren Subject: [PATCH 04/10] fast-export: avoid dying when filtering by paths and old tags exist Date: Sat, 10 Nov 2018 22:23:06 -0800 Message-Id: <20181111062312.16342-5-newren@gmail.com> X-Mailer: git-send-email 2.19.1.866.g82735bcbde In-Reply-To: <20181111062312.16342-1-newren@gmail.com> References: <20181111062312.16342-1-newren@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP If --tag-of-filtered-object=rewrite is specified along with a set of paths to limit what is exported, then any tags pointing to old commits that do not contain any of those specified paths cause problems. Since the old tagged commit is not exported, fast-export attempts to rewrite such tags to an ancestor commit which was exported. If no such commit exists, then fast-export currently die()s. Five years after the tag rewriting logic was added to fast-export (see commit 2d8ad4691921, "fast-export: Add a --tag-of-filtered-object option for newly dangling tags", 2009-06-25), fast-import gained the ability to delete refs (see commit 4ee1b225b99f, "fast-import: add support to delete refs", 2014-04-20), so now we do have a valid option to rewrite the tag to. Delete these tags instead of dying. Signed-off-by: Elijah Newren --- builtin/fast-export.c | 9 ++++++--- t/t9350-fast-export.sh | 20 ++++++++++++++++++++ 2 files changed, 26 insertions(+), 3 deletions(-) diff --git a/builtin/fast-export.c b/builtin/fast-export.c index 1a299c2a21..89de9d6400 100644 --- a/builtin/fast-export.c +++ b/builtin/fast-export.c @@ -774,9 +774,12 @@ static void handle_tag(const char *name, struct tag *tag) break; if (!(p->object.flags & TREESAME)) break; - if (!p->parents) - die("can't find replacement commit for tag %s", - oid_to_hex(&tag->object.oid)); + if (!p->parents) { + printf("reset %s\nfrom %s\n\n", + name, sha1_to_hex(null_sha1)); + free(buf); + return; + } p = p->parents->item; } tagged_mark = get_object_mark(&p->object); diff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh index 6a392e87bc..5bf21b4908 100755 --- a/t/t9350-fast-export.sh +++ b/t/t9350-fast-export.sh @@ -325,6 +325,26 @@ test_expect_success 'rewriting tag of filtered out object' ' ) ' +test_expect_success 'rewrite tag predating pathspecs to nothing' ' + test_create_repo rewrite_tag_predating_pathspecs && + ( + cd rewrite_tag_predating_pathspecs && + + touch ignored && + git add ignored && + test_commit initial && + + git tag -a -m "Some old tag" v0.0.0.0.0.0.1 && + + echo foo >bar && + git add bar && + test_commit add-bar && + + git fast-export --tag-of-filtered-object=rewrite --all -- bar >output && + grep -A 1 refs/tags/v0.0.0.0.0.0.1 output | grep -E ^from.0{40} + ) +' + cat > limit-by-paths/expected << EOF blob mark :1 From patchwork Sun Nov 11 06:23:07 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elijah Newren X-Patchwork-Id: 10677501 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 180E41926 for ; Sun, 11 Nov 2018 06:23:37 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 08BC72B2DE for ; Sun, 11 Nov 2018 06:23:37 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E44772B734; Sun, 11 Nov 2018 06:23:36 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 68CFE2B67F for ; Sun, 11 Nov 2018 06:23:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727497AbeKKQLE (ORCPT ); Sun, 11 Nov 2018 11:11:04 -0500 Received: from mail-oi1-f194.google.com ([209.85.167.194]:44012 "EHLO mail-oi1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727307AbeKKQLE (ORCPT ); Sun, 11 Nov 2018 11:11:04 -0500 Received: by mail-oi1-f194.google.com with SMTP id j202-v6so4745459oih.10 for ; Sat, 10 Nov 2018 22:23:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ogRkULu7I4a0e+eE8rK6ZT9RsXgWCpT/fWyGvrK7fZk=; b=RZCe0Wj98yLBi9YOf+l7ev9k+APpsBGxh97NyrI9SNBJtoy1vJ9fX3eskvfn5s0/43 VYV+Tu7SBPkAysRXNjs7/IY1zsVcrkAZYmj49dwwSOHHaVGpScQURG6q8c4y+5/QGMxv 19Dy6nlqRGLgzv/UIpHg/Ph0fEegmbDJD8vK7uLTjRgG4nL5BxcOeYwLul6+oOrmonBc 3ZPK0JoOKFy+VEE0TUjhvtJtNAMdHJm9qODmqs+WmX7pJge3+YOaVS+kqdN+8lUpbIUk 7gunoZFshozTTotKgDFqYe6mR+iBbxivJy4fsL3F+WSKs5KSIOm8pYhS+mnPrVQEMaWP d/Sw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ogRkULu7I4a0e+eE8rK6ZT9RsXgWCpT/fWyGvrK7fZk=; b=OLlzGvVBNjzJGCKGOYcgAyJgNg/J3RRtSJw3kOYD9iq0zMZdPRQkfXLELCvpp2sP+p sYp4AOzlvJ8vTdKNsHmr4vuJy/BRUrrgLaH970JNtpsrw9VAe6puS9aQcmt7vYfhwImv d0dOTP/Pwb8RXHx/KJyNIx4aDfDhnmwM9yb/EqbxbF9xEB4+Cop3vDawHwQ4B7wUD8Gd AMAY2Osjq/EgZScZoREkeo3H49jmr/NGqFUu/3siS7vcrKudRPUUHdYGEIHCi/OWCM// qkQ/RGSd3nJXrKv4yWkeA5KkXZVCunfegKxFeWEQ3QFXjJMzAjTixihsdAyRfGrsEYJ+ xF4w== X-Gm-Message-State: AGRZ1gI6f3fdN6ELfbcmVA4rMFQjD2wvx8vOj82WQ53wXStP3E0/8BDN 4nlO1+hyphykOHVmpns2O1bcMCQv X-Google-Smtp-Source: AJdET5fFEeEA6Cj4+a9ZYDpjiICN4OGb8OwlhoAMICf8+b67ahfCvUxebTCdvjwcRsA8L0t9pV+hew== X-Received: by 2002:aca:4543:: with SMTP id s64-v6mr2086789oia.278.1541917405184; Sat, 10 Nov 2018 22:23:25 -0800 (PST) Received: from tiger.attlocal.net ([2602:30a:2c28:20f0:7c1a:85e3:2ea9:5d7e]) by smtp.gmail.com with ESMTPSA id f34sm1286855otb.25.2018.11.10.22.23.24 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sat, 10 Nov 2018 22:23:24 -0800 (PST) From: Elijah Newren To: git@vger.kernel.org Cc: larsxschneider@gmail.com, sandals@crustytoothpaste.net, peff@peff.net, me@ttaylorr.com, jrnieder@gmail.com, Elijah Newren Subject: [PATCH 05/10] fast-export: move commit rewriting logic into a function for reuse Date: Sat, 10 Nov 2018 22:23:07 -0800 Message-Id: <20181111062312.16342-6-newren@gmail.com> X-Mailer: git-send-email 2.19.1.866.g82735bcbde In-Reply-To: <20181111062312.16342-1-newren@gmail.com> References: <20181111062312.16342-1-newren@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Logic to replace a filtered commit with an unfiltered ancestor is useful elsewhere; put it into a function we can call. Signed-off-by: Elijah Newren --- builtin/fast-export.c | 37 ++++++++++++++++++++++--------------- 1 file changed, 22 insertions(+), 15 deletions(-) diff --git a/builtin/fast-export.c b/builtin/fast-export.c index 89de9d6400..a3c044b0af 100644 --- a/builtin/fast-export.c +++ b/builtin/fast-export.c @@ -187,6 +187,22 @@ static int get_object_mark(struct object *object) return ptr_to_mark(decoration); } +static struct commit *rewrite_commit(struct commit *p) +{ + for (;;) { + if (p->parents && p->parents->next) + break; + if (p->object.flags & UNINTERESTING) + break; + if (!(p->object.flags & TREESAME)) + break; + if (!p->parents) + return NULL; + p = p->parents->item; + } + return p; +} + static void show_progress(void) { static int counter = 0; @@ -766,21 +782,12 @@ static void handle_tag(const char *name, struct tag *tag) oid_to_hex(&tag->object.oid), type_name(tagged->type)); } - p = (struct commit *)tagged; - for (;;) { - if (p->parents && p->parents->next) - break; - if (p->object.flags & UNINTERESTING) - break; - if (!(p->object.flags & TREESAME)) - break; - if (!p->parents) { - printf("reset %s\nfrom %s\n\n", - name, sha1_to_hex(null_sha1)); - free(buf); - return; - } - p = p->parents->item; + p = rewrite_commit((struct commit *)tagged); + if (!p) { + printf("reset %s\nfrom %s\n\n", + name, sha1_to_hex(null_sha1)); + free(buf); + return; } tagged_mark = get_object_mark(&p->object); } From patchwork Sun Nov 11 06:23:08 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elijah Newren X-Patchwork-Id: 10677503 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 54BC817D4 for ; Sun, 11 Nov 2018 06:23:37 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 40D532B2DE for ; Sun, 11 Nov 2018 06:23:37 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 350652B67F; Sun, 11 Nov 2018 06:23:37 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 895932B688 for ; Sun, 11 Nov 2018 06:23:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727509AbeKKQLF (ORCPT ); Sun, 11 Nov 2018 11:11:05 -0500 Received: from mail-oi1-f193.google.com ([209.85.167.193]:37713 "EHLO mail-oi1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727307AbeKKQLF (ORCPT ); Sun, 11 Nov 2018 11:11:05 -0500 Received: by mail-oi1-f193.google.com with SMTP id w66-v6so4762579oiw.4 for ; Sat, 10 Nov 2018 22:23:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Mms0aOjmTokxH/YVZ6NrGkicGMnzSg9TGErNjQ0QdY8=; b=eT90vv5fK9c7PFFrpuCXx4K6B6GZWhqBZ4cfYGw8Bxa9zaQXQ8zqTp5zcbk5zo1xZ7 VCqyzkHMkSYO50+3QpzkAiSW8csLdskw6yXXKgX5WV9xLpcD7rA7aPRoAxebygaMSHd6 1X8yAXOND4XOxJd6I8s8ju16+2NmTtkNUY3jmKGRgZXmS01kKllcushc7ueSY9jiRaIc DEZiarUEoxb/6xcSt0JZW9s1pHiAU9w03bvgi+xrqZLKJX3j5/mgSECyIZEBTvSoRkqg kjDWE39rZIt9NxbUaQP3iwOcNCFwbq2P0L1AX5FoMQCsMxoHzigMNRsQpidDeyi+IiYa e4bw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Mms0aOjmTokxH/YVZ6NrGkicGMnzSg9TGErNjQ0QdY8=; b=eYHjvPtFbfrZufQ3hPphiTZ/oUZX1fmf9xmc1eA+C650lDgYGCep/liCkvAfWsS7Fx cqIfTEo434SWSTXKk5NNSO1JYYE8a0XKnpwmu4kiEaV3kcHVIX25Fwk1IigK+Y2sQW8u zQVTiXiFpr5gz0hMj+J9+naMj8EPufTlTcvq7EwlyRB+8GdUJ96XACS98D0XqUYoO1Mm LrdPmbytuc16sDTitRvhk13LHUH46TDaPlRtw37u7DtsGH7OjBI67ufF84BK0AD3hwVK KX6Kv7Tt7H1Nw5+E7GihvfXgXQJSoTBtJFUqzchi2wTQvAbZDQHp0iLLCmzUH8Jnrrdu wQGg== X-Gm-Message-State: AGRZ1gJjSX/WrlPVXN5P/ufT2KEwSDGHsU/wNRSeH8NaqSZEci0ZZTAL k3W2W81IjqKp7xEl4ywYJYOzug44 X-Google-Smtp-Source: AJdET5cZMDIGua/GX0DCMb7DF++wamOzLStBt0qBKZ2Sx/BpI97IGSGFsdlX7WKYw+VS59eKdhlPJg== X-Received: by 2002:aca:e607:: with SMTP id d7-v6mr8135977oih.68.1541917406487; Sat, 10 Nov 2018 22:23:26 -0800 (PST) Received: from tiger.attlocal.net ([2602:30a:2c28:20f0:7c1a:85e3:2ea9:5d7e]) by smtp.gmail.com with ESMTPSA id f34sm1286855otb.25.2018.11.10.22.23.25 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sat, 10 Nov 2018 22:23:26 -0800 (PST) From: Elijah Newren To: git@vger.kernel.org Cc: larsxschneider@gmail.com, sandals@crustytoothpaste.net, peff@peff.net, me@ttaylorr.com, jrnieder@gmail.com, Elijah Newren Subject: [PATCH 06/10] fast-export: when using paths, avoid corrupt stream with non-existent mark Date: Sat, 10 Nov 2018 22:23:08 -0800 Message-Id: <20181111062312.16342-7-newren@gmail.com> X-Mailer: git-send-email 2.19.1.866.g82735bcbde In-Reply-To: <20181111062312.16342-1-newren@gmail.com> References: <20181111062312.16342-1-newren@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP If file paths are specified to fast-export and multiple refs point to a commit that does not touch any of the relevant file paths, then fast-export can hit problems. fast-export has a list of additional refs that it needs to explicitly set after exporting all blobs and commits, and when it tries to get_object_mark() on the relevant commit, it can get a mark of 0, i.e. "not found", because the commit in question did not touch the relevant paths and thus was not exported. Trying to import a stream with a mark corresponding to an unexported object will cause fast-import to crash. Avoid this problem by taking the commit the ref points to and finding an ancestor of it that was exported, and make the ref point to that commit instead. Signed-off-by: Elijah Newren --- builtin/fast-export.c | 13 ++++++++++++- t/t9350-fast-export.sh | 24 ++++++++++++++++++++++++ 2 files changed, 36 insertions(+), 1 deletion(-) diff --git a/builtin/fast-export.c b/builtin/fast-export.c index a3c044b0af..5648a8ce9c 100644 --- a/builtin/fast-export.c +++ b/builtin/fast-export.c @@ -900,7 +900,18 @@ static void handle_tags_and_duplicates(void) if (anonymize) name = anonymize_refname(name); /* create refs pointing to already seen commits */ - commit = (struct commit *)object; + commit = rewrite_commit((struct commit *)object); + if (!commit) { + /* + * Neither this object nor any of its + * ancestors touch any relevant paths, so + * it has been filtered to nothing. Delete + * it. + */ + printf("reset %s\nfrom %s\n\n", + name, sha1_to_hex(null_sha1)); + continue; + } printf("reset %s\nfrom :%d\n\n", name, get_object_mark(&commit->object)); show_progress(); diff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh index 5bf21b4908..dbb560c110 100755 --- a/t/t9350-fast-export.sh +++ b/t/t9350-fast-export.sh @@ -386,6 +386,30 @@ test_expect_success 'path limiting with import-marks does not lose unmodified fi grep file0 actual ' +test_expect_success 'avoid corrupt stream with non-existent mark' ' + test_create_repo avoid_non_existent_mark && + ( + cd avoid_non_existent_mark && + + touch important-path && + git add important-path && + test_commit initial && + + touch ignored && + git add ignored && + test_commit whatever && + + git branch A && + git branch B && + + echo foo >>important-path && + git add important-path && + test_commit more changes && + + git fast-export --all -- important-path | git fast-import --force + ) +' + test_expect_success 'full-tree re-shows unmodified files' ' git checkout -f simple && git fast-export --full-tree simple >actual && From patchwork Sun Nov 11 06:23:09 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elijah Newren X-Patchwork-Id: 10677513 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3625B109C for ; Sun, 11 Nov 2018 06:23:38 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 260572AFA1 for ; Sun, 11 Nov 2018 06:23:38 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 19C012B2DE; Sun, 11 Nov 2018 06:23:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AA6C72B68E for ; Sun, 11 Nov 2018 06:23:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727523AbeKKQLH (ORCPT ); Sun, 11 Nov 2018 11:11:07 -0500 Received: from mail-oi1-f194.google.com ([209.85.167.194]:41156 "EHLO mail-oi1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727307AbeKKQLH (ORCPT ); Sun, 11 Nov 2018 11:11:07 -0500 Received: by mail-oi1-f194.google.com with SMTP id g188-v6so4748673oif.8 for ; Sat, 10 Nov 2018 22:23:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=MBpZPg6l7CxjS3NxuCNj92+0MdQSMXusdWB2zYFHEOY=; b=eoGL37fGPSa6cIKVP6vxANuOtVrwCzu54dLLlRZRwd9bRQVOJd2T6cibGQTyD1kqFf LlyrGRhWzXEXBbZgfY8UpJpWZbCv5wBdb4Qo9/SSNMGo2kHUCMagi9LnxHgOi2iI4pbH ITeY9EuSuhA2hdBe/Uou7lv0pFrPnSoQT30RlRXsVko1XayxYJAi19Q0dtGJxJjcXiNe +39JknmQ1CVhn2wDPUQXFN4bhE0I3LwxdJt6a55tyPzTV+Db+pccXYVgan9Zp2grcmrS lQkPiJX0UtjA21fOYO07kwvvlKcNUybMBR+UIK/koC4lpxR3+kVWdfeH4KPFbqda8V1K WXdQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=MBpZPg6l7CxjS3NxuCNj92+0MdQSMXusdWB2zYFHEOY=; b=EWb8KMsuB2XOv8jlCj+FRTfP8XptDeSqhD7klWGWvZ493RHnFPbtV4wKbSN9LKNylA 2o+dnzyzc60ZR0cUX+u9/5hoD927xzZYRpQLkZqfTnjmp2WTvd/0P6nm2yvwj+v70Uwi AQhnjzQRXUr4OGwhBAvg6S6ui0RsVSpw7cuNgexs/AHK0gTVlqZU8ImgWwnontk/ck5C GNIz4TC1yxQabdPwpz8TWLyzu7ssof7OQvZzh3qi0is+Ewb3Ynw7FG2CZ7T4Lt+fSV6d S9VMEpPzdVhLfa8jpx8VnoJNz/a78eYDdXf4cycykmJ5cNUZZhoYCikfQMb9zPDTx3y0 MJcQ== X-Gm-Message-State: AGRZ1gJXnUEHLiOvB/F9/UigtWDhsD28OClkGgGBa0vDb55ShbWZQqOk xuaHv1URIAxEyZdaPCKnLBh/EKHG X-Google-Smtp-Source: AJdET5fLBIdx1+O8NdL/Ahh50H0gaJvy0OS1Kh6gG66FbLQfUpok0ICG+TIqsMMawbwROG6AxNTsDg== X-Received: by 2002:aca:ce47:: with SMTP id e68-v6mr8772718oig.83.1541917407754; Sat, 10 Nov 2018 22:23:27 -0800 (PST) Received: from tiger.attlocal.net ([2602:30a:2c28:20f0:7c1a:85e3:2ea9:5d7e]) by smtp.gmail.com with ESMTPSA id f34sm1286855otb.25.2018.11.10.22.23.26 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sat, 10 Nov 2018 22:23:27 -0800 (PST) From: Elijah Newren To: git@vger.kernel.org Cc: larsxschneider@gmail.com, sandals@crustytoothpaste.net, peff@peff.net, me@ttaylorr.com, jrnieder@gmail.com, Elijah Newren Subject: [PATCH 07/10] fast-export: ensure we export requested refs Date: Sat, 10 Nov 2018 22:23:09 -0800 Message-Id: <20181111062312.16342-8-newren@gmail.com> X-Mailer: git-send-email 2.19.1.866.g82735bcbde In-Reply-To: <20181111062312.16342-1-newren@gmail.com> References: <20181111062312.16342-1-newren@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP If file paths are specified to fast-export and a ref points to a commit that does not touch any of the relevant paths, then that ref would sometimes fail to be exported. (This depends on whether any ancestors of the commit which do touch the relevant paths would be exported with that same ref name or a different ref name.) To avoid this problem, put *all* specified refs into extra_refs to start, and then as we export each commit, remove the refname used in the 'commit $REFNAME' directive from extra_refs. Then, in handle_tags_and_duplicates() we know which refs actually do need a manual reset directive in order to be included. This means that we do need some special handling for excluded refs; e.g. if someone runs git fast-export ^master master then they've asked for master to be exported, but they have also asked for the commit which master points to and all of its history to be excluded. That logically means ref deletion. Previously, such refs were just silently omitted from being exported despite having been explicitly requested for export. Signed-off-by: Elijah Newren --- NOTE: I was hoping the strmap API proposal would materialize, but I either missed it or it hasn't shown up. The usage of string_list in this patch would be better replaced by what Peff suggested. builtin/fast-export.c | 48 +++++++++++++++++++++++++++++++----------- t/t9350-fast-export.sh | 16 +++++++++++--- 2 files changed, 49 insertions(+), 15 deletions(-) diff --git a/builtin/fast-export.c b/builtin/fast-export.c index 5648a8ce9c..0d0bbd9445 100644 --- a/builtin/fast-export.c +++ b/builtin/fast-export.c @@ -38,6 +38,7 @@ static int use_done_feature; static int no_data; static int full_tree; static struct string_list extra_refs = STRING_LIST_INIT_NODUP; +static struct string_list tag_refs = STRING_LIST_INIT_NODUP; static struct refspec refspecs = REFSPEC_INIT_FETCH; static int anonymize; static struct revision_sources revision_sources; @@ -611,6 +612,7 @@ static void handle_commit(struct commit *commit, struct rev_info *rev, export_blob(&diff_queued_diff.queue[i]->two->oid); refname = *revision_sources_at(&revision_sources, commit); + string_list_remove(&extra_refs, refname, 0); if (anonymize) { refname = anonymize_refname(refname); anonymize_ident_line(&committer, &committer_end); @@ -814,7 +816,7 @@ static struct commit *get_commit(struct rev_cmdline_entry *e, char *full_name) /* handle nested tags */ while (tag && tag->object.type == OBJ_TAG) { parse_object(the_repository, &tag->object.oid); - string_list_append(&extra_refs, full_name)->util = tag; + string_list_append(&tag_refs, full_name)->util = tag; tag = (struct tag *)tag->tagged; } if (!tag) @@ -873,25 +875,30 @@ static void get_tags_and_duplicates(struct rev_cmdline_info *info) } /* - * This ref will not be updated through a commit, lets make - * sure it gets properly updated eventually. + * Make sure this ref gets properly updated eventually, whether + * through a commit or manually at the end. */ - if (*revision_sources_at(&revision_sources, commit) || - commit->object.flags & SHOWN) + if (e->item->type != OBJ_TAG) string_list_append(&extra_refs, full_name)->util = commit; + if (!*revision_sources_at(&revision_sources, commit)) *revision_sources_at(&revision_sources, commit) = full_name; } + + string_list_sort(&extra_refs); + string_list_remove_duplicates(&extra_refs, 0); } -static void handle_tags_and_duplicates(void) +static void handle_tags_and_duplicates(struct string_list *extras) { struct commit *commit; int i; - for (i = extra_refs.nr - 1; i >= 0; i--) { - const char *name = extra_refs.items[i].string; - struct object *object = extra_refs.items[i].util; + for (i = extras->nr - 1; i >= 0; i--) { + const char *name = extras->items[i].string; + struct object *object = extras->items[i].util; + int mark; + switch (object->type) { case OBJ_TAG: handle_tag(name, (struct tag *)object); @@ -912,8 +919,24 @@ static void handle_tags_and_duplicates(void) name, sha1_to_hex(null_sha1)); continue; } - printf("reset %s\nfrom :%d\n\n", name, - get_object_mark(&commit->object)); + + mark = get_object_mark(&commit->object); + if (!mark) { + /* + * Getting here means we have a commit which + * was excluded by a negative refspec (e.g. + * fast-export ^master master). If the user + * wants the branch exported but every commit + * in its history to be deleted, that sounds + * like a ref deletion to me. + */ + printf("reset %s\nfrom %s\n\n", + name, sha1_to_hex(null_sha1)); + continue; + } + + printf("reset %s\nfrom :%d\n\n", name, mark + ); show_progress(); break; } @@ -1101,7 +1124,8 @@ int cmd_fast_export(int argc, const char **argv, const char *prefix) } } - handle_tags_and_duplicates(); + handle_tags_and_duplicates(&extra_refs); + handle_tags_and_duplicates(&tag_refs); handle_deletes(); if (export_filename && lastimportid != last_idnum) diff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh index dbb560c110..a0c93f2212 100755 --- a/t/t9350-fast-export.sh +++ b/t/t9350-fast-export.sh @@ -552,10 +552,20 @@ test_expect_success 'use refspec' ' test_cmp expected actual ' -test_expect_success 'delete refspec' ' +test_expect_success 'delete ref because entire history excluded' ' git branch to-delete && - git fast-export --refspec :refs/heads/to-delete to-delete ^to-delete > actual && - cat > expected <<-EOF && + git fast-export to-delete ^to-delete >actual && + cat >expected <<-EOF && + reset refs/heads/to-delete + from 0000000000000000000000000000000000000000 + + EOF + test_cmp expected actual +' + +test_expect_success 'delete refspec' ' + git fast-export --refspec :refs/heads/to-delete >actual && + cat >expected <<-EOF && reset refs/heads/to-delete from 0000000000000000000000000000000000000000 From patchwork Sun Nov 11 06:23:10 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elijah Newren X-Patchwork-Id: 10677507 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 92AFD46E4 for ; Sun, 11 Nov 2018 06:23:37 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7FDB92AFA1 for ; Sun, 11 Nov 2018 06:23:37 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 73D052B6F4; Sun, 11 Nov 2018 06:23:37 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CD3182AFA1 for ; Sun, 11 Nov 2018 06:23:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727531AbeKKQLI (ORCPT ); Sun, 11 Nov 2018 11:11:08 -0500 Received: from mail-oi1-f195.google.com ([209.85.167.195]:34917 "EHLO mail-oi1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727307AbeKKQLI (ORCPT ); Sun, 11 Nov 2018 11:11:08 -0500 Received: by mail-oi1-f195.google.com with SMTP id v198-v6so4772140oif.2 for ; Sat, 10 Nov 2018 22:23:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=wfH2+6YabaYG35LhvEhVXMJWWsSW3LIBAQR1xqoWPYA=; b=LUn5AOq2ruHO8QhkH+rLh6qv2VDU5R0JA9HJLlqlG6N+l5v46HdIEPukV9TrG+E2sw yX8oHar0pfUNPPDCVCPu9xKYWrWm+TYJu24Ybmp65zVBuKbqPzxLgaSrow6NPmCePyV3 5kLzNNcK5Suf10SdTEo30ZVjdRHlQrvK22EHk13vYFSsQPuuN9uFJ3goTxM3b4YGUJYt pIehvSa4TiPOg1yr7sZ0R209yw/xktFZGAbi31eX+zeIJ+CmpHj+gupQD/SVCCqVI79z pSlsI4s7bJbXqkl0CT+Eya7ERJc1db7BMgnvqjwvuSirV9zsa5Hmw4EPg5jCX3IbX7F9 +WJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=wfH2+6YabaYG35LhvEhVXMJWWsSW3LIBAQR1xqoWPYA=; b=ATctXdgTKQeBUXgQ7gi/GsGpyFOm7tJtcqXpXyDkgP1WW0SeHITEr9Nqm18gHhWQn2 088WtF1nVvaKJSIVGE0uU2j6KUvpznk2OPBQHnChw5+6roiLr8Kxtf/iKNUExxvBDNWe x35Xx7KMWjFSCCVtxid1a65jSWf/5q60JqO4Qbadw9gsX+ZFe+Juecj28nAPfb/wCJ6X s04GXM7vC/+HOe7apcHNALl2r9yjNTKVd6jiAIWJUtR7G/Si9BG64LuXnlN0N2nzhpHA r3c10m/JFSB2XRko3I/u9o2L5zPXtylDrn3Ol4fWEMKO2OqEvyBzELj7WcsmJp4Vz5u4 A9Ug== X-Gm-Message-State: AGRZ1gJK9rPKmO48MajP8hho90UVfehD4DpSf2vISGpqnRzbcHpYNkeb 79e3XLtSeEDh+lDZPaGSW+mMz8y6 X-Google-Smtp-Source: AJdET5eWhsFr40Kd25ZxXg6eajJDBmK5eZO0mYkN3ndEDZx+msGrB2toJuMHXB0Z6o90PmfFm5+byg== X-Received: by 2002:aca:5204:: with SMTP id g4-v6mr8961382oib.149.1541917409016; Sat, 10 Nov 2018 22:23:29 -0800 (PST) Received: from tiger.attlocal.net ([2602:30a:2c28:20f0:7c1a:85e3:2ea9:5d7e]) by smtp.gmail.com with ESMTPSA id f34sm1286855otb.25.2018.11.10.22.23.27 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sat, 10 Nov 2018 22:23:28 -0800 (PST) From: Elijah Newren To: git@vger.kernel.org Cc: larsxschneider@gmail.com, sandals@crustytoothpaste.net, peff@peff.net, me@ttaylorr.com, jrnieder@gmail.com, Elijah Newren Subject: [PATCH 08/10] fast-export: add --reference-excluded-parents option Date: Sat, 10 Nov 2018 22:23:10 -0800 Message-Id: <20181111062312.16342-9-newren@gmail.com> X-Mailer: git-send-email 2.19.1.866.g82735bcbde In-Reply-To: <20181111062312.16342-1-newren@gmail.com> References: <20181111062312.16342-1-newren@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP git filter-branch has a nifty feature allowing you to rewrite, e.g. just the last 8 commits of a linear history git filter-branch $OPTIONS HEAD~8..HEAD If you try the same with git fast-export, you instead get a history of only 8 commits, with HEAD~7 being rewritten into a root commit. There are two alternatives: 1) Don't use the negative revision specification, and when you're filtering the output to make modifications to the last 8 commits, just be careful to not modify any earlier commits somehow. 2) First run 'git fast-export --export-marks=somefile HEAD~8', then run 'git fast-export --import-marks=somefile HEAD~8..HEAD'. Both are more error prone than I'd like (the first for obvious reasons; with the second option I have sometimes accidentally included too many revisions in the first command and then found that the corresponding extra revisions were not exported by the second command and thus were not modified as I expected). Also, both are poor from a performance perspective. Add a new --reference-excluded-parents option which will cause fast-export to refer to commits outside the specified rev-list-args range by their sha1sum. Such a stream will only be useful in a repository which already contains the necessary commits (much like the restriction imposed when using --no-data). Signed-off-by: Elijah Newren --- Documentation/git-fast-export.txt | 16 ++++++++++-- builtin/fast-export.c | 42 +++++++++++++++++++++++-------- t/t9350-fast-export.sh | 11 ++++++++ 3 files changed, 57 insertions(+), 12 deletions(-) diff --git a/Documentation/git-fast-export.txt b/Documentation/git-fast-export.txt index 677510b7f7..2916096bdd 100644 --- a/Documentation/git-fast-export.txt +++ b/Documentation/git-fast-export.txt @@ -110,6 +110,17 @@ marks the same across runs. the shape of the history and stored tree. See the section on `ANONYMIZING` below. +--reference-excluded-parents:: + By default, running a command such as `git fast-export + master~5..master` will not include the commit master\~5 and + will make master\~4 no longer have master\~5 as a parent (though + both the old master\~4 and new master~4 will have all the same + files). Use --reference-excluded-parents to instead have the + the stream refer to commits in the excluded range of history + by their sha1sum. Note that the resulting stream can only be + used by a repository which already contains the necessary + parent commits. + --refspec:: Apply the specified refspec to each ref exported. Multiple of them can be specified. @@ -119,8 +130,9 @@ marks the same across runs. 'git rev-list', that specifies the specific objects and references to export. For example, `master~10..master` causes the current master reference to be exported along with all objects - added since its 10th ancestor commit and all files common to - master\~9 and master~10. + added since its 10th ancestor commit and (unless the + --reference-excluded-parents option is specified) all files + common to master\~9 and master~10. EXAMPLES -------- diff --git a/builtin/fast-export.c b/builtin/fast-export.c index 0d0bbd9445..ea9c5b1c00 100644 --- a/builtin/fast-export.c +++ b/builtin/fast-export.c @@ -37,6 +37,7 @@ static int fake_missing_tagger; static int use_done_feature; static int no_data; static int full_tree; +static int reference_excluded_commits; static struct string_list extra_refs = STRING_LIST_INIT_NODUP; static struct string_list tag_refs = STRING_LIST_INIT_NODUP; static struct refspec refspecs = REFSPEC_INIT_FETCH; @@ -596,7 +597,8 @@ static void handle_commit(struct commit *commit, struct rev_info *rev, message += 2; if (commit->parents && - get_object_mark(&commit->parents->item->object) != 0 && + (get_object_mark(&commit->parents->item->object) != 0 || + reference_excluded_commits) && !full_tree) { parse_commit_or_die(commit->parents->item); diff_tree_oid(get_commit_tree_oid(commit->parents->item), @@ -638,13 +640,21 @@ static void handle_commit(struct commit *commit, struct rev_info *rev, unuse_commit_buffer(commit, commit_buffer); for (i = 0, p = commit->parents; p; p = p->next) { - int mark = get_object_mark(&p->item->object); - if (!mark) + struct object *obj = &p->item->object; + int mark = get_object_mark(obj); + + if (!mark && !reference_excluded_commits) continue; if (i == 0) - printf("from :%d\n", mark); + printf("from "); + else + printf("merge "); + if (mark) + printf(":%d\n", mark); else - printf("merge :%d\n", mark); + printf("%s\n", sha1_to_hex(anonymize ? + anonymize_sha1(&obj->oid) : + obj->oid.hash)); i++; } @@ -925,13 +935,22 @@ static void handle_tags_and_duplicates(struct string_list *extras) /* * Getting here means we have a commit which * was excluded by a negative refspec (e.g. - * fast-export ^master master). If the user + * fast-export ^master master). If we are + * referencing excluded commits, set the ref + * to the exact commit. Otherwise, the user * wants the branch exported but every commit - * in its history to be deleted, that sounds - * like a ref deletion to me. + * in its history to be deleted, which basically + * just means deletion of the ref. */ - printf("reset %s\nfrom %s\n\n", - name, sha1_to_hex(null_sha1)); + if (!reference_excluded_commits) { + /* delete the ref */ + printf("reset %s\nfrom %s\n\n", + name, sha1_to_hex(null_sha1)); + continue; + } + /* set ref to commit using oid, not mark */ + printf("reset %s\nfrom %s\n\n", name, + sha1_to_hex(commit->object.oid.hash)); continue; } @@ -1068,6 +1087,9 @@ int cmd_fast_export(int argc, const char **argv, const char *prefix) OPT_STRING_LIST(0, "refspec", &refspecs_list, N_("refspec"), N_("Apply refspec to exported refs")), OPT_BOOL(0, "anonymize", &anonymize, N_("anonymize output")), + OPT_BOOL(0, "reference-excluded-parents", + &reference_excluded_commits, N_("Reference parents which are not in fast-export stream by sha1sum")), + OPT_END() }; diff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh index a0c93f2212..c2f40d6a40 100755 --- a/t/t9350-fast-export.sh +++ b/t/t9350-fast-export.sh @@ -66,6 +66,17 @@ test_expect_success 'fast-export master~2..master' ' ' +test_expect_success 'fast-export --reference-excluded-parents master~2..master' ' + + git fast-export --reference-excluded-parents master~2..master >actual && + grep commit.refs/heads/master actual >commit-count && + test_line_count = 2 commit-count && + sed "s/master/rewrite/" actual | + (cd new && + git fast-import && + test $MASTER = $(git rev-parse --verify refs/heads/rewrite)) +' + test_expect_success 'iso-8859-1' ' git config i18n.commitencoding ISO8859-1 && From patchwork Sun Nov 11 06:23:11 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elijah Newren X-Patchwork-Id: 10677517 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D464017D4 for ; Sun, 11 Nov 2018 06:23:38 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C0A692AFA1 for ; Sun, 11 Nov 2018 06:23:38 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B4C2E2AFEF; Sun, 11 Nov 2018 06:23:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EE2742B68F for ; Sun, 11 Nov 2018 06:23:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727539AbeKKQLJ (ORCPT ); Sun, 11 Nov 2018 11:11:09 -0500 Received: from mail-oi1-f173.google.com ([209.85.167.173]:43377 "EHLO mail-oi1-f173.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727524AbeKKQLJ (ORCPT ); Sun, 11 Nov 2018 11:11:09 -0500 Received: by mail-oi1-f173.google.com with SMTP id j202-v6so4745526oih.10 for ; Sat, 10 Nov 2018 22:23:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=nFevrcZ/T8fCkpNvRUt5EvBMQkHssSjGQ3IfS7WtzY4=; b=AM/y+awvC3p+T5AapK6N/7PilxV1HBNV5YRjvZHyatC+75ZnA4YGB2kcYzkXSgyQoK kUXC8yNpMZkMP1N9TNYf8oaAasYp39NpvRn+TGO7sxHpGV9tlBk/n+P5eQBctdW72i5l y0gFS3IDnS04AtZQXGI5DspY94Td2bzcGqW5PmyZ0wQPG0cl/vA8a6lWq63QYxtjwYuW RNjeH6HJBy14p03WHpM9wPlLws7uFx3At7Qn1nFjr1IUEgEaIRS3KJJWWhuHohjRjmk3 V/7DOz2JoYUu2eeCB+p+JIdtn4zTHgLsmXPhERw1NYmH2CkprEkC37dyyggBayGmaZNC VNww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=nFevrcZ/T8fCkpNvRUt5EvBMQkHssSjGQ3IfS7WtzY4=; b=OjvyrSrSdKZjeLl/OXGTdcTYRR7OD1NuzAf1bFvqqfFimNz5seKowL+oEXcdodTFHF fGvdk9zTQI7+u5kKo6EclP3/y/xOXS/S+QVXjee6wop/rSOHI06vfCwCYx2m97Wp5cz1 iOARbFAjV5r88/wIuJGKcMNPW5GDRwtLwOqJxfOkAEkg4qCYg4OZHQkX7TSiKNIGbhXL PDAZnKFmBdy9wlpG7yaaUlzmLjDXMucb10qoK8XcCRKiAsTngUPlEm4ZS69zyZTwZlf2 7bXmx8jQK1Y7YfzoePezttEsJQW8YyGVz9vK1+jk6Qs1pQ8Hi1URzA/vFVpZDEnOTzVS ensQ== X-Gm-Message-State: AGRZ1gIGyP5r4OE/L8t2inHIrbt+Ysku5g9o4vjBy8M8JaQpiYHQJLt+ 6Y53hdTwHl/zwestbeX8T56zFFfz X-Google-Smtp-Source: AJdET5f8j3lQdL/Jlqp9N++rxI5N9+/SI71kLayzHRTWhlETaQhwk0ZekCug+efIc/Oss+NkIW+4Mw== X-Received: by 2002:aca:eb4b:: with SMTP id j72-v6mr7975171oih.306.1541917410315; Sat, 10 Nov 2018 22:23:30 -0800 (PST) Received: from tiger.attlocal.net ([2602:30a:2c28:20f0:7c1a:85e3:2ea9:5d7e]) by smtp.gmail.com with ESMTPSA id f34sm1286855otb.25.2018.11.10.22.23.29 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sat, 10 Nov 2018 22:23:29 -0800 (PST) From: Elijah Newren To: git@vger.kernel.org Cc: larsxschneider@gmail.com, sandals@crustytoothpaste.net, peff@peff.net, me@ttaylorr.com, jrnieder@gmail.com, Elijah Newren Subject: [PATCH 09/10] fast-export: add a --show-original-ids option to show original names Date: Sat, 10 Nov 2018 22:23:11 -0800 Message-Id: <20181111062312.16342-10-newren@gmail.com> X-Mailer: git-send-email 2.19.1.866.g82735bcbde In-Reply-To: <20181111062312.16342-1-newren@gmail.com> References: <20181111062312.16342-1-newren@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Knowing the original names (hashes) of commits, blobs, and tags can sometimes enable post-filtering that would otherwise be difficult or impossible. In particular, the desire to rewrite commit messages which refer to other prior commits (on top of whatever other filtering is being done) is very difficult without knowing the original names of each commit. This commit teaches a new --show-original-ids option to fast-export which will make it add a 'originally ' line to blob, commits, and tags. It also teaches fast-import to parse (and ignore) such lines. Signed-off-by: Elijah Newren --- Documentation/git-fast-export.txt | 7 +++++++ builtin/fast-export.c | 20 +++++++++++++++----- fast-import.c | 17 +++++++++++++++++ t/t9350-fast-export.sh | 17 +++++++++++++++++ 4 files changed, 56 insertions(+), 5 deletions(-) diff --git a/Documentation/git-fast-export.txt b/Documentation/git-fast-export.txt index 2916096bdd..4e40f0b99a 100644 --- a/Documentation/git-fast-export.txt +++ b/Documentation/git-fast-export.txt @@ -121,6 +121,13 @@ marks the same across runs. used by a repository which already contains the necessary parent commits. +--show-original-ids:: + Add an extra directive to the output for commits and blobs, + `originally `. While such directives will likely be + ignored by importers such as git-fast-import, it may be useful + for intermediary filters (e.g. for rewriting commit messages + which refer to older commits, or for stripping blobs by id). + --refspec:: Apply the specified refspec to each ref exported. Multiple of them can be specified. diff --git a/builtin/fast-export.c b/builtin/fast-export.c index ea9c5b1c00..cc01dcc90c 100644 --- a/builtin/fast-export.c +++ b/builtin/fast-export.c @@ -38,6 +38,7 @@ static int use_done_feature; static int no_data; static int full_tree; static int reference_excluded_commits; +static int show_original_ids; static struct string_list extra_refs = STRING_LIST_INIT_NODUP; static struct string_list tag_refs = STRING_LIST_INIT_NODUP; static struct refspec refspecs = REFSPEC_INIT_FETCH; @@ -271,7 +272,10 @@ static void export_blob(const struct object_id *oid) mark_next_object(object); - printf("blob\nmark :%"PRIu32"\ndata %lu\n", last_idnum, size); + printf("blob\nmark :%"PRIu32"\n", last_idnum); + if (show_original_ids) + printf("originally %s\n", oid_to_hex(oid)); + printf("data %lu\n", size); if (size && fwrite(buf, size, 1, stdout) != 1) die_errno("could not write blob '%s'", oid_to_hex(oid)); printf("\n"); @@ -628,8 +632,10 @@ static void handle_commit(struct commit *commit, struct rev_info *rev, reencoded = reencode_string(message, "UTF-8", encoding); if (!commit->parents) printf("reset %s\n", refname); - printf("commit %s\nmark :%"PRIu32"\n%.*s\n%.*s\ndata %u\n%s", - refname, last_idnum, + printf("commit %s\nmark :%"PRIu32"\n", refname, last_idnum); + if (show_original_ids) + printf("originally %s\n", oid_to_hex(&commit->object.oid)); + printf("%.*s\n%.*s\ndata %u\n%s", (int)(author_end - author), author, (int)(committer_end - committer), committer, (unsigned)(reencoded @@ -807,8 +813,10 @@ static void handle_tag(const char *name, struct tag *tag) if (starts_with(name, "refs/tags/")) name += 10; - printf("tag %s\nfrom :%d\n%.*s%sdata %d\n%.*s\n", - name, tagged_mark, + printf("tag %s\nfrom :%d\n", name, tagged_mark); + if (show_original_ids) + printf("originally %s\n", oid_to_hex(&tag->object.oid)); + printf("%.*s%sdata %d\n%.*s\n", (int)(tagger_end - tagger), tagger, tagger == tagger_end ? "" : "\n", (int)message_size, (int)message_size, message ? message : ""); @@ -1089,6 +1097,8 @@ int cmd_fast_export(int argc, const char **argv, const char *prefix) OPT_BOOL(0, "anonymize", &anonymize, N_("anonymize output")), OPT_BOOL(0, "reference-excluded-parents", &reference_excluded_commits, N_("Reference parents which are not in fast-export stream by sha1sum")), + OPT_BOOL(0, "show-original-ids", &show_original_ids, + N_("Show original sha1sums of blobs/commits")), OPT_END() }; diff --git a/fast-import.c b/fast-import.c index 95600c78e0..232b6a8b8d 100644 --- a/fast-import.c +++ b/fast-import.c @@ -14,11 +14,13 @@ Format of STDIN stream: new_blob ::= 'blob' lf mark? + originally? file_content; file_content ::= data; new_commit ::= 'commit' sp ref_str lf mark? + originally? ('author' (sp name)? sp '<' email '>' sp when lf)? 'committer' (sp name)? sp '<' email '>' sp when lf commit_msg @@ -49,6 +51,7 @@ Format of STDIN stream: new_tag ::= 'tag' sp tag_str lf 'from' sp commit-ish lf + originally? ('tagger' (sp name)? sp '<' email '>' sp when lf)? tag_msg; tag_msg ::= data; @@ -73,6 +76,8 @@ Format of STDIN stream: data ::= (delimited_data | exact_data) lf?; + originally ::= 'originally' sp not_lf+ lf + # note: delim may be any string but must not contain lf. # data_line may contain any data but must not be exactly # delim. @@ -1968,6 +1973,13 @@ static void parse_mark(void) next_mark = 0; } +static void parse_original_identifier(void) +{ + const char *v; + if (skip_prefix(command_buf.buf, "originally ", &v)) + read_next_command(); +} + static int parse_data(struct strbuf *sb, uintmax_t limit, uintmax_t *len_res) { const char *data; @@ -2110,6 +2122,7 @@ static void parse_new_blob(void) { read_next_command(); parse_mark(); + parse_original_identifier(); parse_and_store_blob(&last_blob, NULL, next_mark); } @@ -2733,6 +2746,7 @@ static void parse_new_commit(const char *arg) read_next_command(); parse_mark(); + parse_original_identifier(); if (skip_prefix(command_buf.buf, "author ", &v)) { author = parse_ident(v); read_next_command(); @@ -2865,6 +2879,9 @@ static void parse_new_tag(const char *arg) die("Invalid ref name or SHA1 expression: %s", from); read_next_command(); + /* originally ... */ + parse_original_identifier(); + /* tagger ... */ if (skip_prefix(command_buf.buf, "tagger ", &v)) { tagger = parse_ident(v); diff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh index c2f40d6a40..5ad6669910 100755 --- a/t/t9350-fast-export.sh +++ b/t/t9350-fast-export.sh @@ -77,6 +77,23 @@ test_expect_success 'fast-export --reference-excluded-parents master~2..master' test $MASTER = $(git rev-parse --verify refs/heads/rewrite)) ' +test_expect_success 'fast-export --show-original-ids' ' + + git fast-export --show-original-ids master >output && + grep ^originally output| sed -e s/^originally.// | sort >actual && + git rev-list --objects master muss >objects-and-names && + awk "{print \$1}" objects-and-names | sort >commits-trees-blobs && + comm -23 actual commits-trees-blobs >unfound && + test_must_be_empty unfound +' + +test_expect_success 'fast-export --show-original-ids | git fast-import' ' + + git fast-export --show-original-ids master muss | git fast-import --quiet && + test $MASTER = $(git rev-parse --verify refs/heads/master) && + test $MUSS = $(git rev-parse --verify refs/tags/muss) +' + test_expect_success 'iso-8859-1' ' git config i18n.commitencoding ISO8859-1 && From patchwork Sun Nov 11 06:23:12 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elijah Newren X-Patchwork-Id: 10677505 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 791F24B7E for ; Sun, 11 Nov 2018 06:23:37 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 644482B67F for ; Sun, 11 Nov 2018 06:23:37 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 589732B2DE; Sun, 11 Nov 2018 06:23:37 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1CAF22B699 for ; Sun, 11 Nov 2018 06:23:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727550AbeKKQLL (ORCPT ); Sun, 11 Nov 2018 11:11:11 -0500 Received: from mail-oi1-f193.google.com ([209.85.167.193]:36337 "EHLO mail-oi1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727524AbeKKQLK (ORCPT ); Sun, 11 Nov 2018 11:11:10 -0500 Received: by mail-oi1-f193.google.com with SMTP id r127-v6so4765373oie.3 for ; Sat, 10 Nov 2018 22:23:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=mtfZyQCvGTLrItylsV4YbZAfxEuQI5GK14yAcs/anmQ=; b=NMw0IWbjMjR21qnKqJiO0VwYmHaQhAjsfzgANKDiFQ/XGihzwvp0KN5y10Hs13QcmC DZyP8nD/6oQQzXwDvrt+rruJDPt6HFlqkGum+w0TbeY+NhQRNaq7xs3FvC7cr/CkX87E OP0pj2lEd28hTzo9SksVOO8xxg1Ac7XKbYnf5ZoYHb/Fsj9KD6xVlu5uF32SorK42nCY y7lqVyjTWMYp1ImbbPFB5ojAKK+70PYsWEEu8nJSH0l2PtWQrxaryOvPnhP+EwC4H7RX XJ9VDRdvaWvuZ2JiIpEo8zgaCw8U9PFzvAz4m/BNPk3zIZpgd0k25Bhf7Aj4Wpkrph4r 82iA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=mtfZyQCvGTLrItylsV4YbZAfxEuQI5GK14yAcs/anmQ=; b=k9nN8UOpaZGAFTq0Wr9Vg3PvWdskde0DgGMmVasJsZXTIovwrcSWqopyqmOwHL0kXQ YRNJuBKXGv+gPdGv+bz4ku4E4UX02dPA4gBb8bvfdBQ4u5f8irS7lImr9JFojjXDrMC3 DZjQ4najfKYFK6A5KcC8XEgQ2I2xwRucKRVu2Nov/O8glLu8VaJxtYtnhEg7a2sM/Pz6 0PwNfEMjgzBvhgEQzB5prgQGFd+fHTRVZpbWmtGvOopB9EhQcsd5CZfc6T3lG4sIZ1Zz 6g+btoew97voqCKi/MMIKljGIkXshbKYcPUGLDK5vfPWQQ2xFfYfoIcwHTWfL1EkRIb6 mAow== X-Gm-Message-State: AGRZ1gLQR5iFAOIRlGBcXqz2NRL7kWN5ZYkdE58DYRXUmmEWNKZXfiIG 4l3I9MTeoGt56Bbf8RBcVm/n2stW X-Google-Smtp-Source: AJdET5d+8Lgtxg+vk3ZOw6EZlPE0gPTNH4/pgNlncX+Cs9ZLu6gUTWEz6PhdWKpidNoR4zDQOpDL4w== X-Received: by 2002:aca:3807:: with SMTP id f7-v6mr2068132oia.46.1541917411699; Sat, 10 Nov 2018 22:23:31 -0800 (PST) Received: from tiger.attlocal.net ([2602:30a:2c28:20f0:7c1a:85e3:2ea9:5d7e]) by smtp.gmail.com with ESMTPSA id f34sm1286855otb.25.2018.11.10.22.23.30 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sat, 10 Nov 2018 22:23:31 -0800 (PST) From: Elijah Newren To: git@vger.kernel.org Cc: larsxschneider@gmail.com, sandals@crustytoothpaste.net, peff@peff.net, me@ttaylorr.com, jrnieder@gmail.com, Elijah Newren Subject: [PATCH 10/10] fast-export: add --always-show-modify-after-rename Date: Sat, 10 Nov 2018 22:23:12 -0800 Message-Id: <20181111062312.16342-11-newren@gmail.com> X-Mailer: git-send-email 2.19.1.866.g82735bcbde In-Reply-To: <20181111062312.16342-1-newren@gmail.com> References: <20181111062312.16342-1-newren@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP fast-export output is traditionally used as an input to a fast-import program, but it is also useful to help gather statistics about the history of a repository (particularly when --no-data is also passed). For example, two of the types of information we may want to collect could include: 1) general information about renames that have occurred 2) what the biggest objects in a repository are and what names they appear under. The first bit of information can be gathered by just passing -M to fast-export. The second piece of information can partially be gotten from running git cat-file --batch-check --batch-all-objects However, that only shows what the biggest objects in the repository are and their sizes, not what names those objects appear as or what commits they were introduced in. We can get that information from fast-export, but when we only see R oldname newname instead of R oldname newname M 100644 $SHA1 newname then it makes the job more difficult. Add an option which allows us to force the latter output even when commits have exact renames of files. Signed-off-by: Elijah Newren --- Documentation/git-fast-export.txt | 11 ++++++++++ builtin/fast-export.c | 7 +++++- t/t9350-fast-export.sh | 36 +++++++++++++++++++++++++++++++ 3 files changed, 53 insertions(+), 1 deletion(-) diff --git a/Documentation/git-fast-export.txt b/Documentation/git-fast-export.txt index 4e40f0b99a..946a5aee1f 100644 --- a/Documentation/git-fast-export.txt +++ b/Documentation/git-fast-export.txt @@ -128,6 +128,17 @@ marks the same across runs. for intermediary filters (e.g. for rewriting commit messages which refer to older commits, or for stripping blobs by id). +--always-show-modify-after-rename:: + When a rename is detected, fast-export normally issues both a + 'R' (rename) and a 'M' (modify) directive. However, if the + contents of the old and new filename match exactly, it will + only issue the rename directive. Use this flag to have it + always issue the modify directive after the rename, which may + be useful for tools which are using the fast-export stream as + a mechanism for gathering statistics about a repository. Note + that this option only has effect when rename detection is + active (see the -M option). + --refspec:: Apply the specified refspec to each ref exported. Multiple of them can be specified. diff --git a/builtin/fast-export.c b/builtin/fast-export.c index cc01dcc90c..db606d1fd0 100644 --- a/builtin/fast-export.c +++ b/builtin/fast-export.c @@ -38,6 +38,7 @@ static int use_done_feature; static int no_data; static int full_tree; static int reference_excluded_commits; +static int always_show_modify_after_rename; static int show_original_ids; static struct string_list extra_refs = STRING_LIST_INIT_NODUP; static struct string_list tag_refs = STRING_LIST_INIT_NODUP; @@ -407,7 +408,8 @@ static void show_filemodify(struct diff_queue_struct *q, putchar('\n'); if (oideq(&ospec->oid, &spec->oid) && - ospec->mode == spec->mode) + ospec->mode == spec->mode && + !always_show_modify_after_rename) break; } /* fallthrough */ @@ -1099,6 +1101,9 @@ int cmd_fast_export(int argc, const char **argv, const char *prefix) &reference_excluded_commits, N_("Reference parents which are not in fast-export stream by sha1sum")), OPT_BOOL(0, "show-original-ids", &show_original_ids, N_("Show original sha1sums of blobs/commits")), + OPT_BOOL(0, "always-show-modify-after-rename", + &always_show_modify_after_rename, + N_("Always provide 'M' directive after 'R'")), OPT_END() }; diff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh index 5ad6669910..d0c30672ac 100755 --- a/t/t9350-fast-export.sh +++ b/t/t9350-fast-export.sh @@ -638,4 +638,40 @@ test_expect_success 'merge commit gets exported with --import-marks' ' ) ' +test_expect_success 'rename detection and --always-show-modify-after-rename' ' + test_create_repo renames && + ( + cd renames && + test_seq 0 9 >single_digit && + test_seq 10 98 >double_digit && + git add . && + git commit -m initial && + + echo 99 >>double_digit && + git mv single_digit single-digit && + git mv double_digit double-digit && + git add double-digit && + git commit -m renames && + + # First, check normal fast-export -M output + git fast-export -M --no-data master >out && + + grep double-digit out >out2 && + test_line_count = 2 out2 && + + grep single-digit out >out2 && + test_line_count = 1 out2 && + + # Now, test with --always-show-modify-after-rename; should + # have an extra "M" directive for "single-digit". + git fast-export -M --no-data --always-show-modify-after-rename master >out && + + grep double-digit out >out2 && + test_line_count = 2 out2 && + + grep single-digit out >out2 && + test_line_count = 2 out2 + ) +' + test_done