From patchwork Wed Sep 25 01:39:58 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elijah Newren X-Patchwork-Id: 11159889 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EFA7C1709 for ; Wed, 25 Sep 2019 01:40:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id CDEEC214AF for ; Wed, 25 Sep 2019 01:40:17 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="sFgHzcNH" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2411364AbfIYBkP (ORCPT ); Tue, 24 Sep 2019 21:40:15 -0400 Received: from mail-pg1-f195.google.com ([209.85.215.195]:37950 "EHLO mail-pg1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2389596AbfIYBkO (ORCPT ); Tue, 24 Sep 2019 21:40:14 -0400 Received: by mail-pg1-f195.google.com with SMTP id x10so2262653pgi.5 for ; Tue, 24 Sep 2019 18:40:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gzKAdVEBv8q/JAjAnX30HMBM8oMKpklxXex4+Wui2dM=; b=sFgHzcNH2N97GufRvc28nOGHmGvXnh1R/6TXHGcqU2ecR2Flb/7Ae1POkSIL0SYFge 6kPz2k1BvMYBE2lKMFZWq88nTUwzU0wKXBxcHMpPyCuHcZHu5ecmJhsfWMjxzhhd30MK ZlNjyFLqP1dswFsuDyWupcAWDMD7MdAqSzQkbf6xyRzdynQvZtfWAvba8TH/Hn6b+XYM NSuNLuGiSG3Kv1vVTl4HVZ0rspRBs+BhBkKUoxHFrfJxCTYV+2TOzxt1T5bkUk90+VGU i9dftEtfqOGK4MfDs5wp8o7BDEYQPQxeWaIJnbFpXFDdSa8GTBOgLWwFU+xWu+hQQ8dJ xzOA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gzKAdVEBv8q/JAjAnX30HMBM8oMKpklxXex4+Wui2dM=; b=DfnpgsecU4GdwqRUY4LCx06cK793E6IJ/ZskG48Ur/FwBNnG00vy9Zrblj6vOKUMFy mRh95rsoyaU8umJQwoz1mHG59wxBtcOWjunMhd161BmbCJOY8eUUjFGP9WTwVWKaqR81 K8izZiKugy4J5rcwsA2zJ+//+vcXCEVJbnf3/LEJqpyqsldP4y/F9cpMgaEWK/PvMiHX rnfbVoQEfkJyvnm163XbPKBj9oLVF3azJCMudXhBVZLxGU/PrMG7YLEBaxhtg3+iZLO2 AUMRP/6aJ2yvR9PQnDxzAqLKhQJSxOPob1Bp03bCPGrJHweXrCtfEF3/o0Tz+A4/hnkt n2Pg== X-Gm-Message-State: APjAAAXDZaH74JzykxVdmg+uUay2Q0JIc/QfVTfZ4r8EFML0EkOXG7PZ Pk9rWFqKH8WL4QZ9nlD8LMnFiyQP5JU= X-Google-Smtp-Source: APXvYqzgAJPME6TXCOROwpk8vgqmoJKWb7P6viMOFt5eafzC3xxmgzlMGHaXGLbHZViFRpfUhmng2A== X-Received: by 2002:a63:506:: with SMTP id 6mr2228017pgf.397.1569375613481; Tue, 24 Sep 2019 18:40:13 -0700 (PDT) Received: from newren2-linux.yojoe.local ([8.4.231.67]) by smtp.gmail.com with ESMTPSA id h4sm3759050pgg.81.2019.09.24.18.40.12 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 24 Sep 2019 18:40:13 -0700 (PDT) From: Elijah Newren To: git@vger.kernel.org Cc: gitster@pobox.com, Elijah Newren Subject: [PATCH 1/8] fast-export: fix exporting a tag and nothing else Date: Tue, 24 Sep 2019 18:39:58 -0700 Message-Id: <20190925014005.17056-2-newren@gmail.com> X-Mailer: git-send-email 2.23.0.177.g8af0b3ca64 In-Reply-To: <20190925014005.17056-1-newren@gmail.com> References: <20190925014005.17056-1-newren@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org fast-export allows specifying revision ranges, which can be used to export a tag without exporting the commit it tags. fast-export handled this rather poorly: it would emit a "from :0" directive. Since marks start at 1 and increase, this means it refers to an unknown commit and fast-import will choke on the input. When we are unable to look up a mark for the object being tagged, use a "from $HASH" directive instead to fix this problem. Note that this is quite similar to the behavior fast-export exhibits with commits and parents when --reference-excluded-parents is passed along with an excluded commit range. For tags of excluded commits we do not require the --reference-excluded-parents flag because we always have to tag something. By contrast, when dealing with commits, pruning a parent is always a viable option, so we need the flag to specify that parent pruning is not wanted. (It is slightly weird that --reference-excluded-parents isn't the default with a separate --prune-excluded-parents flag, but backward compatibility concerns resulted in the current defaults.) Signed-off-by: Elijah Newren --- builtin/fast-export.c | 7 ++++++- t/t9350-fast-export.sh | 13 +++++++++++++ 2 files changed, 19 insertions(+), 1 deletion(-) diff --git a/builtin/fast-export.c b/builtin/fast-export.c index f541f55d33..5822271c6b 100644 --- a/builtin/fast-export.c +++ b/builtin/fast-export.c @@ -860,7 +860,12 @@ static void handle_tag(const char *name, struct tag *tag) if (starts_with(name, "refs/tags/")) name += 10; - printf("tag %s\nfrom :%d\n", name, tagged_mark); + printf("tag %s\n", name); + if (tagged_mark) + printf("from :%d\n", tagged_mark); + else + printf("from %s\n", oid_to_hex(&tagged->oid)); + if (show_original_ids) printf("original-oid %s\n", oid_to_hex(&tag->object.oid)); printf("%.*s%sdata %d\n%.*s\n", diff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh index b4004e05c2..d32ff41859 100755 --- a/t/t9350-fast-export.sh +++ b/t/t9350-fast-export.sh @@ -53,6 +53,19 @@ test_expect_success 'fast-export | fast-import' ' ' +test_expect_success 'fast-export ^muss^{commit} muss' ' + git fast-export --tag-of-filtered-object=rewrite ^muss^{commit} muss >actual && + cat >expected <<-EOF && + tag muss + from $(git rev-parse --verify muss^{commit}) + $(git cat-file tag muss | grep tagger) + data 9 + valentin + + EOF + test_cmp expected actual +' + test_expect_success 'fast-export master~2..master' ' git fast-export master~2..master >actual && From patchwork Wed Sep 25 01:39:59 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elijah Newren X-Patchwork-Id: 11159891 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2820D1800 for ; Wed, 25 Sep 2019 01:40:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 05398214AF for ; Wed, 25 Sep 2019 01:40:18 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="mY4E+94y" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2411366AbfIYBkP (ORCPT ); Tue, 24 Sep 2019 21:40:15 -0400 Received: from mail-pf1-f193.google.com ([209.85.210.193]:40233 "EHLO mail-pf1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2389596AbfIYBkP (ORCPT ); Tue, 24 Sep 2019 21:40:15 -0400 Received: by mail-pf1-f193.google.com with SMTP id x127so2373753pfb.7 for ; Tue, 24 Sep 2019 18:40:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=xjLySYLBZHHft3O6gARhS8AXliebYlKeZZ9zMnuPoro=; b=mY4E+94y4qf8zonSjtixCY9JAu/9UPMndRDOlG8F2+MkC+eQsXMmfiurH4tboJ8GIO JlJV1puBjAt1TAjOHJKRV/PshVsFvA8qaACLHAUfpO87aMUFauwMFoJPybX7mX30c4j6 pCAnFiisZI7SzH1ctAIuY0WhOyeMLLKzsSghY9Y3cq4YmmZt6jV6wwjOaPPj490c+0MW 6E34cgIHQmH5nYZ+dQgTasuNMMKpgdwi66XdbX80bYLStpH735IIdjTNBTsQVvP5fE75 6pW9NB9Q4jgIgPrlOrUXCPDe1wEDN0g8uKy3jQD3bTeZkxajes7uP9iOOAPcoPd+p4Ee kK4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=xjLySYLBZHHft3O6gARhS8AXliebYlKeZZ9zMnuPoro=; b=doKPuM99fUn5N5j8C8zhssalPOuurauNyfsT3fdzKDV24sr9XAr7mLMkx7VMqQekaV ReHbm0W1dZ643MAOmK+BSemxv2ejmqPPFp57Onjt6+c13T5dsxqghE/CMDktC5Q61Zzo sv+Ka4DCke+AMxLN3ftzvEydLEADKKN2Xc+NbF7ScDw42GjMeQtkVlqJfASkNqRMeDmi D9fIFICfH3itgZ51TivHDm6MoHkuIaUylxoT8xgvKJt+Lk8ZvrYod8N/OefyEI/FR4Dd fw9mkxxO5kZvfxEla6Y0cEa0UyoIeOiPPKNgavRzxAU9C41aLJwjzE0/XNU7YEp/+f6i O0ZA== X-Gm-Message-State: APjAAAUVVypKmWU02QI8lYc398ZRKzbG5uZyeMNmVROTsyHs59vUqLaw v8d4+BIqfVOif5aMqGrKNyoThBY0y0U= X-Google-Smtp-Source: APXvYqwkreZZPlZM4GBEpoCB3kuzCsaYLJhNO7jRiWFk8xMbkhEfKhP/u1PWfgN5ikHMFZMWi8swPA== X-Received: by 2002:a63:4b42:: with SMTP id k2mr6219232pgl.35.1569375614316; Tue, 24 Sep 2019 18:40:14 -0700 (PDT) Received: from newren2-linux.yojoe.local ([8.4.231.67]) by smtp.gmail.com with ESMTPSA id h4sm3759050pgg.81.2019.09.24.18.40.13 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 24 Sep 2019 18:40:13 -0700 (PDT) From: Elijah Newren To: git@vger.kernel.org Cc: gitster@pobox.com, Elijah Newren Subject: [PATCH 2/8] fast-import: fix handling of deleted tags Date: Tue, 24 Sep 2019 18:39:59 -0700 Message-Id: <20190925014005.17056-3-newren@gmail.com> X-Mailer: git-send-email 2.23.0.177.g8af0b3ca64 In-Reply-To: <20190925014005.17056-1-newren@gmail.com> References: <20190925014005.17056-1-newren@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org If our input stream includes a tag which is later deleted, we were not properly deleting it. We did have a step which would delete it, but we left a tag in the tag list noting that it needed to be updated, and the updating of annotated tags occurred AFTER ref deletion. So, when we record that a tag needs to be deleted, also remove it from the list of annotated tags to update. While this has likely been something that has not happened in practice, it will come up more in order to support nested tags. For nested tags, we either need to give temporary names to the intermediate tags and then delete them, or else we need to use the final name for the intermediate tags. If we use the final name for the intermediate tags, then in order to keep the sanity check that someone doesn't try to update the same tag twice, we need to delete the ref after creating the intermediate tag. So, either way nested tags imply the need to delete temporary inner tag references. Signed-off-by: Elijah Newren --- fast-import.c | 29 +++++++++++++++++++++++++++++ t/t9300-fast-import.sh | 13 +++++++++++++ 2 files changed, 42 insertions(+) diff --git a/fast-import.c b/fast-import.c index b44d6a467e..dab905d667 100644 --- a/fast-import.c +++ b/fast-import.c @@ -2793,6 +2793,35 @@ static void parse_reset_branch(const char *arg) b = new_branch(arg); read_next_command(); parse_from(b); + if (b->delete && !strncmp(arg, "refs/tags/", 10)) { + /* + * Elsewhere, we call dump_branches() before dump_tags(), + * and dump_branches() will handle ref deletions first, so + * in order to make sure the deletion actually takes effect, + * we need to remove the tag from our list of tags to update. + * + * NEEDSWORK: replace list of tags with hashmap for faster + * deletion? + */ + struct strbuf tag_name = STRBUF_INIT; + struct tag *t, *prev = NULL; + for (t = first_tag; t; t = t->next_tag) { + strbuf_reset(&tag_name); + strbuf_addf(&tag_name, "refs/tags/%s", t->name); + if (!strcmp(arg, tag_name.buf)) + break; + prev = t; + } + if (t) { + if (prev) + prev->next_tag = t->next_tag; + else + first_tag = t->next_tag; + if (!t->next_tag) + last_tag = prev; + /* There is no mem_pool_free(t) function to call. */ + } + } if (command_buf.len > 0) unread_command_buf = 1; } diff --git a/t/t9300-fast-import.sh b/t/t9300-fast-import.sh index 141b7fa35e..74bc41333b 100755 --- a/t/t9300-fast-import.sh +++ b/t/t9300-fast-import.sh @@ -85,6 +85,15 @@ test_expect_success 'A: create pack from stdin' ' An annotated tag that annotates a blob. EOF + tag to-be-deleted + from :3 + data <expect <<-EOF && :2 $(git rev-parse --verify master:file2) From patchwork Wed Sep 25 01:40:00 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elijah Newren X-Patchwork-Id: 11159893 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2FB501709 for ; Wed, 25 Sep 2019 01:40:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 0C9B720874 for ; Wed, 25 Sep 2019 01:40:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="A6RRd7iY" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2411370AbfIYBkS (ORCPT ); Tue, 24 Sep 2019 21:40:18 -0400 Received: from mail-pg1-f173.google.com ([209.85.215.173]:39748 "EHLO mail-pg1-f173.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2389596AbfIYBkR (ORCPT ); Tue, 24 Sep 2019 21:40:17 -0400 Received: by mail-pg1-f173.google.com with SMTP id u17so2259208pgi.6 for ; Tue, 24 Sep 2019 18:40:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=OumwuTqVmtGx/iUW4iyLL8K9MG5ULwEe2xoWD+7MpXw=; b=A6RRd7iYnEi4ojOyzs8ukhAf2ikpbR50ygUM+axcJZidrO9zM9ApU8nXEcHD2BA9l/ JG9Ie/8YWTD8gAEzkNhNb2FmLRw1Hk0x8t0mddWiGjDynvyamzHv4y1OckhGs6tlWSTe e2KltquC2eSZ3J2HmmInYwHH77I8TeJob20uoJljuc5qgFY+qJZHtz/8K1JwHHTkutSe lkiNY0FSQlle9btoT2k20Kx8aq+znZUKndunnxfoNcccnAJgfXBcCy8eIZCHBkOp5xRb 53s7XtI7Prg4rvyxC+t72vfseuoIDuz4PSza3ixTRfqVInyfluzsJP8jMsWQMesVv/Jx mmkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=OumwuTqVmtGx/iUW4iyLL8K9MG5ULwEe2xoWD+7MpXw=; b=qXPOYtspd4c/CDZGmBYIPfB4kK+X1HBqaMVhhyULYVFLYDq77k5XZ8m1E+Z1UQjqew 2lZ9fjITWEnTvaGMtu6GhjsMreTz52eDD0aa3xB1KGhsvuavqxxU00knga3DXPd3ynUt aAbghhBz93fmKr5bp8kj2qsPRmJaiJzaNVIfA+gTMYXmK0tUPCW8kH5Z/p/Kest4zxKw mbJE5k2ImZkShlr80/jZynmdgVo55a9EUeykvQXll9MD7PujZAhvqjjCUBVq8ZeQECEw FePQao3JWD6FLr1m9GrBfpreHZC/nbRik2W0OT1hDipLM/XcBb4hUOmnpEg9RtXWedpu TYSw== X-Gm-Message-State: APjAAAW2ZFGBPnfQnVkDQ37/4qq6NpejE34qUodVN3eirbfvyQubG0jq zdDXV6YvsIlRW/kVIh/7ZxJsBR43Svg= X-Google-Smtp-Source: APXvYqxb/kJFW+gR/UavkU9wCWQmFUrZYS0mVCdjvD5jy42+pFgMXgG+WU+pKt85zIHYB9fvhxzmcQ== X-Received: by 2002:a17:90a:be08:: with SMTP id a8mr3525214pjs.92.1569375615255; Tue, 24 Sep 2019 18:40:15 -0700 (PDT) Received: from newren2-linux.yojoe.local ([8.4.231.67]) by smtp.gmail.com with ESMTPSA id h4sm3759050pgg.81.2019.09.24.18.40.14 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 24 Sep 2019 18:40:14 -0700 (PDT) From: Elijah Newren To: git@vger.kernel.org Cc: gitster@pobox.com, Elijah Newren Subject: [PATCH 3/8] fast-import: allow tags to be identified by mark labels Date: Tue, 24 Sep 2019 18:40:00 -0700 Message-Id: <20190925014005.17056-4-newren@gmail.com> X-Mailer: git-send-email 2.23.0.177.g8af0b3ca64 In-Reply-To: <20190925014005.17056-1-newren@gmail.com> References: <20190925014005.17056-1-newren@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Mark identifiers are used in fast-export and fast-import to provide a label to refer to earlier content. Blobs are given labels because they need to be referenced in the commits where they first appear with a given filename, and commits are given labels because they can be the parents of other commits. Tags were never given labels, probably because they were viewed as unnecessary, but that presents two problems: 1. It leaves us without a way of referring to previous tags if we want to create a tag of a tag (or higher nestings). 2. It leaves us with no way of recording that a tag has already been imported when using --export-marks and --import-marks. Fix these problems by allowing an optional mark label for tags. Signed-off-by: Elijah Newren --- Documentation/git-fast-import.txt | 1 + fast-import.c | 3 ++- t/t9300-fast-import.sh | 19 +++++++++++++++++++ 3 files changed, 22 insertions(+), 1 deletion(-) diff --git a/Documentation/git-fast-import.txt b/Documentation/git-fast-import.txt index 0bb276269e..4977869465 100644 --- a/Documentation/git-fast-import.txt +++ b/Documentation/git-fast-import.txt @@ -774,6 +774,7 @@ lightweight (non-annotated) tags see the `reset` command below. .... 'tag' SP LF + mark? 'from' SP LF original-oid? 'tagger' (SP )? SP LT GT SP LF diff --git a/fast-import.c b/fast-import.c index dab905d667..0271d81d0d 100644 --- a/fast-import.c +++ b/fast-import.c @@ -2713,6 +2713,7 @@ static void parse_new_tag(const char *arg) first_tag = t; last_tag = t; read_next_command(); + parse_mark(); /* from ... */ if (!skip_prefix(command_buf.buf, "from ", &from)) @@ -2769,7 +2770,7 @@ static void parse_new_tag(const char *arg) strbuf_addbuf(&new_data, &msg); free(tagger); - if (store_object(OBJ_TAG, &new_data, NULL, &t->oid, 0)) + if (store_object(OBJ_TAG, &new_data, NULL, &t->oid, next_mark)) t->pack_id = MAX_PACK_ID; else t->pack_id = pack_id; diff --git a/t/t9300-fast-import.sh b/t/t9300-fast-import.sh index 74bc41333b..3ad2b2f1ba 100755 --- a/t/t9300-fast-import.sh +++ b/t/t9300-fast-import.sh @@ -94,6 +94,23 @@ test_expect_success 'A: create pack from stdin' ' reset refs/tags/to-be-deleted from 0000000000000000000000000000000000000000 + tag nested + mark :6 + from :4 + data < X-Patchwork-Id: 11159895 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5B2891800 for ; Wed, 25 Sep 2019 01:40:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 3888C20874 for ; Wed, 25 Sep 2019 01:40:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="KNTL9Tbo" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2439274AbfIYBkT (ORCPT ); Tue, 24 Sep 2019 21:40:19 -0400 Received: from mail-pg1-f194.google.com ([209.85.215.194]:39868 "EHLO mail-pg1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2389596AbfIYBkS (ORCPT ); Tue, 24 Sep 2019 21:40:18 -0400 Received: by mail-pg1-f194.google.com with SMTP id u17so2259225pgi.6 for ; Tue, 24 Sep 2019 18:40:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ROkf6ojEQXuUls6wpph3o/ULzztKahIuSZCwrb7fI4c=; b=KNTL9TboBhFItokEgvxNXAyiMSmlUWykPTo8Z2sw6hu+bPgbqpu1S9xMa5NcsPwcTB rAgT5xuneM2zVfh0uflAcBxxBQmNfjP0sc2Kl2DbIc6FWqKQW7dPVG70TjAUWH4OBO/0 JgcwBjfhdLqa6YcdY023UAwZys49enfoaS05BUl2OeX9inmtgunJYqZcxU4TO2bYXQyH MSFHgKVB1B+V7jPgpS/C7hC46KXaX8PVPT/T/cLH8NK0Wl4wBP2NLN+uG91I4hare3wL nlMxBtqYldf0AnxX78pZWj6LGU0YJZk025QBY8gFkKcEaE/rQZTTYewuIkLbcJ37OcA/ dnSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ROkf6ojEQXuUls6wpph3o/ULzztKahIuSZCwrb7fI4c=; b=dWEK2d4y2/71OipYpdqlSS3WahILl/+bH3ChjSyDq5hm0lLBRuKbDYqJynZ8qkSBN1 1PLQcj2WXvCAmTaN5qAHqoJcKG1imv/Tu7TTlaq88ledydb3m/lVnUofWU6h9s+bMQ+L +z/3VExvp5Ri2z8CFfEJBJ0ueDP3v3BFbpUqOaH/OCbtyFtbYlSrtJ0JbhURe8+c7Clc iqbc0QXHXtyaxNasHjIB4lva2hhOI9pdhzrNmn2OmlKutt3+q7yLzObivTpi0wTcf1nT pA85GHZsJI0cvCaLHaiVSfZ7aJF6ektdvNxFnk8aRlLx4k3iExaq8e4QG6hLLygGFABc zlHQ== X-Gm-Message-State: APjAAAVjUV+c/p7veUpykGjaWjgDvXFZtO/JrblnvVnv+oK4bY8+vi2R tN+fsAgK72UoOwPPgVos484SV6sba+c= X-Google-Smtp-Source: APXvYqze95TBwbd8nIuhu8BfyXPSn998zUzIhdDjUNymphOr+Jb/KzNMXypqT+SAmCyX4ppxSxCV7Q== X-Received: by 2002:a63:5652:: with SMTP id g18mr5996240pgm.283.1569375616003; Tue, 24 Sep 2019 18:40:16 -0700 (PDT) Received: from newren2-linux.yojoe.local ([8.4.231.67]) by smtp.gmail.com with ESMTPSA id h4sm3759050pgg.81.2019.09.24.18.40.15 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 24 Sep 2019 18:40:15 -0700 (PDT) From: Elijah Newren To: git@vger.kernel.org Cc: gitster@pobox.com, Elijah Newren Subject: [PATCH 4/8] fast-import: add support for new 'alias' command Date: Tue, 24 Sep 2019 18:40:01 -0700 Message-Id: <20190925014005.17056-5-newren@gmail.com> X-Mailer: git-send-email 2.23.0.177.g8af0b3ca64 In-Reply-To: <20190925014005.17056-1-newren@gmail.com> References: <20190925014005.17056-1-newren@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org fast-export and fast-import have nice --import-marks flags which allow for incremental migrations. However, if there is a mark in fast-export's file of marks without a corresponding mark in the one for fast-import, then we run the risk that fast-export tries to send new objects relative to the mark it knows which fast-import does not, causing fast-import to fail. This arises in practice when there is a filter of some sort running between the fast-export and fast-import processes which prunes some commits programmatically. Provide such a filter with the ability to alias pruned commits to their most recent non-pruned ancestor. Signed-off-by: Elijah Newren --- Documentation/git-fast-import.txt | 22 +++++++++++ fast-import.c | 62 ++++++++++++++++++++++++++----- t/t9300-fast-import.sh | 5 +++ 3 files changed, 79 insertions(+), 10 deletions(-) diff --git a/Documentation/git-fast-import.txt b/Documentation/git-fast-import.txt index 4977869465..a3f1e0c5e4 100644 --- a/Documentation/git-fast-import.txt +++ b/Documentation/git-fast-import.txt @@ -337,6 +337,13 @@ and control the current import process. More detailed discussion `commit` command. This command is optional and is not needed to perform an import. +`alias`:: + Record that a mark refers to a given object without first + creating any new object. Using --import-marks and referring + to missing marks will cause fast-import to fail, so aliases + can provide a way to set otherwise pruned commits to a valid + value (e.g. the nearest non-pruned ancestor). + `checkpoint`:: Forces fast-import to close the current packfile, generate its unique SHA-1 checksum and index, and start a new packfile. @@ -914,6 +921,21 @@ a data chunk which does not have an LF as its last byte. + The `LF` after ` LF` is optional (it used to be required). +`alias` +~~~~~~~ +Record that a mark refers to a given object without first creating any +new object. + +.... + 'alias' LF + mark + 'to' SP LF + LF? +.... + +For a detailed description of `` see above under `from`. + + `checkpoint` ~~~~~~~~~~~~ Forces fast-import to close the current packfile, start a new one, and to diff --git a/fast-import.c b/fast-import.c index 0271d81d0d..8228cde759 100644 --- a/fast-import.c +++ b/fast-import.c @@ -2491,18 +2491,14 @@ static void parse_from_existing(struct branch *b) } } -static int parse_from(struct branch *b) +static int parse_objectish(struct branch *b, const char *objectish) { - const char *from; struct branch *s; struct object_id oid; - if (!skip_prefix(command_buf.buf, "from ", &from)) - return 0; - oidcpy(&oid, &b->branch_tree.versions[1].oid); - s = lookup_branch(from); + s = lookup_branch(objectish); if (b == s) die("Can't create a branch from itself: %s", b->name); else if (s) { @@ -2510,8 +2506,8 @@ static int parse_from(struct branch *b) oidcpy(&b->oid, &s->oid); oidcpy(&b->branch_tree.versions[0].oid, t); oidcpy(&b->branch_tree.versions[1].oid, t); - } else if (*from == ':') { - uintmax_t idnum = parse_mark_ref_eol(from); + } else if (*objectish == ':') { + uintmax_t idnum = parse_mark_ref_eol(objectish); struct object_entry *oe = find_mark(idnum); if (oe->type != OBJ_COMMIT) die("Mark :%" PRIuMAX " not a commit", idnum); @@ -2525,13 +2521,13 @@ static int parse_from(struct branch *b) } else parse_from_existing(b); } - } else if (!get_oid(from, &b->oid)) { + } else if (!get_oid(objectish, &b->oid)) { parse_from_existing(b); if (is_null_oid(&b->oid)) b->delete = 1; } else - die("Invalid ref name or SHA1 expression: %s", from); + die("Invalid ref name or SHA1 expression: %s", objectish); if (b->branch_tree.tree && !oideq(&oid, &b->branch_tree.versions[1].oid)) { release_tree_content_recursive(b->branch_tree.tree); @@ -2542,6 +2538,26 @@ static int parse_from(struct branch *b) return 1; } +static int parse_from(struct branch *b) +{ + const char *from; + + if (!skip_prefix(command_buf.buf, "from ", &from)) + return 0; + + return parse_objectish(b, from); +} + +static int parse_objectish_with_prefix(struct branch *b, const char *prefix) +{ + const char *base; + + if (!skip_prefix(command_buf.buf, prefix, &base)) + return 0; + + return parse_objectish(b, base); +} + static struct hash_list *parse_merge(unsigned int *count) { struct hash_list *list = NULL, **tail = &list, *n; @@ -3089,6 +3105,28 @@ static void parse_progress(void) skip_optional_lf(); } +static void parse_alias(void) +{ + struct object_entry *e; + struct branch b; + + skip_optional_lf(); + read_next_command(); + + /* mark ... */ + parse_mark(); + if (!next_mark) + die(_("Expected 'mark' command, got %s"), command_buf.buf); + + /* to ... */ + memset(&b, 0, sizeof(b)); + if (!parse_objectish_with_prefix(&b, "to ")) + die(_("Expected 'to' command, got %s"), command_buf.buf); + e = find_object(&b.oid); + assert(e); + insert_mark(next_mark, e); +} + static char* make_fast_import_path(const char *path) { if (!relative_marks_paths || is_absolute_path(path)) @@ -3216,6 +3254,8 @@ static int parse_one_feature(const char *feature, int from_stream) option_import_marks(arg, from_stream, 1); } else if (skip_prefix(feature, "export-marks=", &arg)) { option_export_marks(arg); + } else if (!strcmp(feature, "alias")) { + ; /* Don't die - this feature is supported */ } else if (!strcmp(feature, "get-mark")) { ; /* Don't die - this feature is supported */ } else if (!strcmp(feature, "cat-blob")) { @@ -3372,6 +3412,8 @@ int cmd_main(int argc, const char **argv) parse_checkpoint(); else if (!strcmp("done", command_buf.buf)) break; + else if (!strcmp("alias", command_buf.buf)) + parse_alias(); else if (starts_with(command_buf.buf, "progress ")) parse_progress(); else if (skip_prefix(command_buf.buf, "feature ", &v)) diff --git a/t/t9300-fast-import.sh b/t/t9300-fast-import.sh index 3ad2b2f1ba..41f2a1dad9 100755 --- a/t/t9300-fast-import.sh +++ b/t/t9300-fast-import.sh @@ -111,6 +111,10 @@ test_expect_success 'A: create pack from stdin' ' Tag of tag of our lovely commit EOF + alias + mark :8 + to :5 + INPUT_END git fast-import --export-marks=marks.out X-Patchwork-Id: 11159899 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B476117EE for ; Wed, 25 Sep 2019 01:40:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 914AB20874 for ; Wed, 25 Sep 2019 01:40:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="GeblFhwF" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2439283AbfIYBkU (ORCPT ); Tue, 24 Sep 2019 21:40:20 -0400 Received: from mail-pf1-f196.google.com ([209.85.210.196]:39162 "EHLO mail-pf1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2411372AbfIYBkT (ORCPT ); Tue, 24 Sep 2019 21:40:19 -0400 Received: by mail-pf1-f196.google.com with SMTP id v4so2377840pff.6 for ; Tue, 24 Sep 2019 18:40:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=fYj3rOK3QFnaJZwqKsXF5Oy4FMu8pe+1p3OJvZ6gWCk=; b=GeblFhwFvuXo0Ei9afEOJb/UbivdDPglFlakPkCA2dYxhnG5q9tuT4w8w5ZRNh8B50 P08FqjFno9xx0iuZmUSjiqKsWWljfkuao0ZIXP7V2PCXA9P1mYqd22vdCOzgmvuaLoHj DgIUvxfhAc40H1VWTGSpWfkoBz+0d96YDw2jaOPgiGUFDXqR/q1QeDCuj9YvkiOSdaxY gAnwgN2iiBdZNCFnOJfsiAWGcpTuNinbCRFH2Uelzjbfp98ejalIoSSrxQTjM/Bc4SI5 Fpm6EJ2j7+Av6pJIz8ZCyje9gbA3tax+vy5Bbz0GLIfNYK+MK1/qAqiX4I/nSEdz1niu IxlA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fYj3rOK3QFnaJZwqKsXF5Oy4FMu8pe+1p3OJvZ6gWCk=; b=dHQ25fiNCWrvpp010nVF0UVz/Lox2j1VHjSt00kJ3fH9DiaSnB8Ar3uD7gVESzXzgo gFUuHLLO26m6yyXp90hKu/8ISBM5UarTxMAJ1h+SXpv+YvxnQHsLAh6araQ3aw+r4ddC N65fCMysFVY9/fiALcYASddDzIGnV7QXFou5v384CctH9ws4ucbTfvARrL+bG3gsSoZ4 LlqPkNRnT8p+HcMdoTYs1WQKVcPBh2sxM+7CDbDZ8NVFoJo10tgmL4to0PjbTGUVEs8k P5j62zvjr8dGlSCPorgAOWVWW684AC1BOF/M+i1tZAjJTmw1OmJjhSkUFVnyBS/cqx6y Xtaw== X-Gm-Message-State: APjAAAVyqc1r6LbAaGhR2QBYy8jjjakaAWFvSUNeGD2yZku8Tt7AHfV9 72n4spw8iJ7LuOpYTNIGv+x8W+1NKTM= X-Google-Smtp-Source: APXvYqzzjWcMrBUDxPcsHaxzaSOmkf0as8nWgitBDfqvzMVoOPds35W8XKMxsw2Wy8kW/Tm+w5mfMg== X-Received: by 2002:a63:3c46:: with SMTP id i6mr6039149pgn.18.1569375617141; Tue, 24 Sep 2019 18:40:17 -0700 (PDT) Received: from newren2-linux.yojoe.local ([8.4.231.67]) by smtp.gmail.com with ESMTPSA id h4sm3759050pgg.81.2019.09.24.18.40.16 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 24 Sep 2019 18:40:16 -0700 (PDT) From: Elijah Newren To: git@vger.kernel.org Cc: gitster@pobox.com, Elijah Newren Subject: [PATCH 5/8] fast-export: add support for --import-marks-if-exists Date: Tue, 24 Sep 2019 18:40:02 -0700 Message-Id: <20190925014005.17056-6-newren@gmail.com> X-Mailer: git-send-email 2.23.0.177.g8af0b3ca64 In-Reply-To: <20190925014005.17056-1-newren@gmail.com> References: <20190925014005.17056-1-newren@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org fast-import has support for both an --import-marks flag and an --import-marks-if-exists flag; the latter of which will not die() if the file does not exist. fast-export only had support for an --import-marks flag; add an --import-marks-if-exists flag for consistency. Signed-off-by: Elijah Newren --- builtin/fast-export.c | 23 +++++++++++++++++++---- t/t9350-fast-export.sh | 10 ++++------ 2 files changed, 23 insertions(+), 10 deletions(-) diff --git a/builtin/fast-export.c b/builtin/fast-export.c index 5822271c6b..575e47833b 100644 --- a/builtin/fast-export.c +++ b/builtin/fast-export.c @@ -1052,11 +1052,16 @@ static void export_marks(char *file) error("Unable to write marks file %s.", file); } -static void import_marks(char *input_file) +static void import_marks(char *input_file, int check_exists) { char line[512]; - FILE *f = xfopen(input_file, "r"); + FILE *f; + struct stat sb; + + if (check_exists && stat(input_file, &sb)) + return; + f = xfopen(input_file, "r"); while (fgets(line, sizeof(line), f)) { uint32_t mark; char *line_end, *mark_end; @@ -1120,7 +1125,9 @@ int cmd_fast_export(int argc, const char **argv, const char *prefix) struct rev_info revs; struct object_array commits = OBJECT_ARRAY_INIT; struct commit *commit; - char *export_filename = NULL, *import_filename = NULL; + char *export_filename = NULL, + *import_filename = NULL, + *import_filename_if_exists = NULL; uint32_t lastimportid; struct string_list refspecs_list = STRING_LIST_INIT_NODUP; struct string_list paths_of_changed_objects = STRING_LIST_INIT_DUP; @@ -1140,6 +1147,10 @@ int cmd_fast_export(int argc, const char **argv, const char *prefix) N_("Dump marks to this file")), OPT_STRING(0, "import-marks", &import_filename, N_("file"), N_("Import marks from this file")), + OPT_STRING(0, "import-marks-if-exists", + &import_filename_if_exists, + N_("file"), + N_("Import marks from this file if it exists")), OPT_BOOL(0, "fake-missing-tagger", &fake_missing_tagger, N_("Fake a tagger when tags lack one")), OPT_BOOL(0, "full-tree", &full_tree, @@ -1187,8 +1198,12 @@ int cmd_fast_export(int argc, const char **argv, const char *prefix) if (use_done_feature) printf("feature done\n"); + if (import_filename && import_filename_if_exists) + die(_("Cannot pass both --import-marks and --import-marks-if-exists")); if (import_filename) - import_marks(import_filename); + import_marks(import_filename, 0); + else if (import_filename_if_exists) + import_marks(import_filename_if_exists, 1); lastimportid = last_idnum; if (import_filename && revs.prune_data.nr) diff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh index d32ff41859..ea84e2f173 100755 --- a/t/t9350-fast-export.sh +++ b/t/t9350-fast-export.sh @@ -580,17 +580,15 @@ test_expect_success 'fast-export quotes pathnames' ' ' test_expect_success 'test bidirectionality' ' - >marks-cur && - >marks-new && git init marks-test && - git fast-export --export-marks=marks-cur --import-marks=marks-cur --branches | \ - git --git-dir=marks-test/.git fast-import --export-marks=marks-new --import-marks=marks-new && + git fast-export --export-marks=marks-cur --import-marks-if-exists=marks-cur --branches | \ + git --git-dir=marks-test/.git fast-import --export-marks=marks-new --import-marks-if-exists=marks-new && (cd marks-test && git reset --hard && echo Wohlauf > file && git commit -a -m "back in time") && - git --git-dir=marks-test/.git fast-export --export-marks=marks-new --import-marks=marks-new --branches | \ - git fast-import --export-marks=marks-cur --import-marks=marks-cur + git --git-dir=marks-test/.git fast-export --export-marks=marks-new --import-marks-if-exists=marks-new --branches | \ + git fast-import --export-marks=marks-cur --import-marks-if-exists=marks-cur ' cat > expected << EOF From patchwork Wed Sep 25 01:40:03 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elijah Newren X-Patchwork-Id: 11159897 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 87C0F14ED for ; Wed, 25 Sep 2019 01:40:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 656F620874 for ; Wed, 25 Sep 2019 01:40:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Ur89iL0/" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2411375AbfIYBkT (ORCPT ); Tue, 24 Sep 2019 21:40:19 -0400 Received: from mail-pf1-f193.google.com ([209.85.210.193]:32810 "EHLO mail-pf1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2411371AbfIYBkS (ORCPT ); Tue, 24 Sep 2019 21:40:18 -0400 Received: by mail-pf1-f193.google.com with SMTP id q10so2399195pfl.0 for ; Tue, 24 Sep 2019 18:40:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=zeAhElhIbrDdu6uzDamVZo/HaatbOMWXxKYhqQLDmXI=; b=Ur89iL0/lPg28EMSmY87k3dRTyWpqGGd7o+8BOhwX0EtOmDCEDmQ/QAOvA3LN7Ftsj c4AC0+idZji6uNFKrL8QMr+y31DFXSo73Md9WJ7+uwmmh4eEhaI37dL/aRZIfaSxC/R/ nLB1vA+J0we1ndr3VWbDN0cssaLooKaO9oddx6PPmPC9TojDocZXALXBmwt/qewh+O5Z kJC3Hhczhaff5338yDHqvKM6bGpNXKrvT2b/FKomYnnCxt3lwH3odjwDJsPDf3VC7uuA VtRDRDZxJ8rFtVy20jrz1tijWid0M11Fn/uOXxuPJS6yPiAXteMbcumZoI2W3orl9ZNy MIlg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=zeAhElhIbrDdu6uzDamVZo/HaatbOMWXxKYhqQLDmXI=; b=MR9enWJzq9YqtDieJp9KrW6PV594QVwmxnZZFLZhgDzSB94Vov8XK6fLhQDOHCF6Hn UI+22R2hWkt6BoZgPRww6IQ7wBaeKvc9h/RQXLWhD0VbkxRIirGWsWYSHMvbPT02zE6b PZ4UaNJ26tUdr7dfPT64puJ1WX0yb/SHeQZ8974OjaPCwHOT7INPJPxsI2FjUcblAoSX Lvu0mBdlIIvcoa64m3mgbO4IYnySt+a4PZHs6Y1aCOkSieEDSzAibpF47EMg554M+AxL KIgdE5+7gc+f0nnnUxsjZBboDIfQRVVSIxxVbm8aZ5ZfMviWtBUuTdAjhtuTx4tBRv9K 6PIQ== X-Gm-Message-State: APjAAAVmgXNgTtAmms7sq0GiF+ICxZNMfvS+XCheDocI/kFHyMiRg8XA gD2YqtaKVn//v/K4lfnZq2No/PJi1DY= X-Google-Smtp-Source: APXvYqwLOPykI6E0GQlmtFOzzsOJAP28ZBsGoNWJp8/DzFAcvKgDXMZCWL+XRZVP8d/o9gTt5AC9fg== X-Received: by 2002:a17:90a:fc8:: with SMTP id 66mr3510462pjz.134.1569375618025; Tue, 24 Sep 2019 18:40:18 -0700 (PDT) Received: from newren2-linux.yojoe.local ([8.4.231.67]) by smtp.gmail.com with ESMTPSA id h4sm3759050pgg.81.2019.09.24.18.40.17 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 24 Sep 2019 18:40:17 -0700 (PDT) From: Elijah Newren To: git@vger.kernel.org Cc: gitster@pobox.com, Elijah Newren Subject: [PATCH 6/8] fast-export: allow user to request tags be marked with --mark-tags Date: Tue, 24 Sep 2019 18:40:03 -0700 Message-Id: <20190925014005.17056-7-newren@gmail.com> X-Mailer: git-send-email 2.23.0.177.g8af0b3ca64 In-Reply-To: <20190925014005.17056-1-newren@gmail.com> References: <20190925014005.17056-1-newren@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Add a new option, --mark-tags, which will output mark identifiers with each tag object. This improves the incremental export story with --export-marks since it will allow us to record that annotated tags have been exported, and it is also needed as a step towards supporting nested tags. Signed-off-by: Elijah Newren --- Documentation/git-fast-export.txt | 17 +++++++++++++---- builtin/fast-export.c | 7 +++++++ t/t9350-fast-export.sh | 14 ++++++++++++++ 3 files changed, 34 insertions(+), 4 deletions(-) diff --git a/Documentation/git-fast-export.txt b/Documentation/git-fast-export.txt index cc940eb9ad..c522b34f7b 100644 --- a/Documentation/git-fast-export.txt +++ b/Documentation/git-fast-export.txt @@ -75,11 +75,20 @@ produced incorrect results if you gave these options. Before processing any input, load the marks specified in . The input file must exist, must be readable, and must use the same format as produced by --export-marks. + +--mark-tags:: + In addition to labelling blobs and commits with mark ids, also + label tags. This is useful in conjunction with + `--export-marks` and `--import-marks`, and is also useful (and + necessary) for exporting of nested tags. It does not hurt + other cases and would be the default, but many fast-import + frontends are not prepared to accept tags with mark + identifiers. + -Any commits that have already been marked will not be exported again. -If the backend uses a similar --import-marks file, this allows for -incremental bidirectional exporting of the repository by keeping the -marks the same across runs. +Any commits (or tags) that have already been marked will not be +exported again. If the backend uses a similar --import-marks file, +this allows for incremental bidirectional exporting of the repository +by keeping the marks the same across runs. --fake-missing-tagger:: Some old repositories have tags without a tagger. The diff --git a/builtin/fast-export.c b/builtin/fast-export.c index 575e47833b..d32e1e9327 100644 --- a/builtin/fast-export.c +++ b/builtin/fast-export.c @@ -40,6 +40,7 @@ static int no_data; static int full_tree; static int reference_excluded_commits; static int show_original_ids; +static int mark_tags; static struct string_list extra_refs = STRING_LIST_INIT_NODUP; static struct string_list tag_refs = STRING_LIST_INIT_NODUP; static struct refspec refspecs = REFSPEC_INIT_FETCH; @@ -861,6 +862,10 @@ static void handle_tag(const char *name, struct tag *tag) if (starts_with(name, "refs/tags/")) name += 10; printf("tag %s\n", name); + if (mark_tags) { + mark_next_object(&tag->object); + printf("mark :%"PRIu32"\n", last_idnum); + } if (tagged_mark) printf("from :%d\n", tagged_mark); else @@ -1165,6 +1170,8 @@ int cmd_fast_export(int argc, const char **argv, const char *prefix) &reference_excluded_commits, N_("Reference parents which are not in fast-export stream by object id")), OPT_BOOL(0, "show-original-ids", &show_original_ids, N_("Show original object ids of blobs/commits")), + OPT_BOOL(0, "mark-tags", &mark_tags, + N_("Label tags with mark ids")), OPT_END() }; diff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh index ea84e2f173..b3fca6ffba 100755 --- a/t/t9350-fast-export.sh +++ b/t/t9350-fast-export.sh @@ -66,6 +66,20 @@ test_expect_success 'fast-export ^muss^{commit} muss' ' test_cmp expected actual ' +test_expect_success 'fast-export --mark-tags ^muss^{commit} muss' ' + git fast-export --mark-tags --tag-of-filtered-object=rewrite ^muss^{commit} muss >actual && + cat >expected <<-EOF && + tag muss + mark :1 + from $(git rev-parse --verify muss^{commit}) + $(git cat-file tag muss | grep tagger) + data 9 + valentin + + EOF + test_cmp expected actual +' + test_expect_success 'fast-export master~2..master' ' git fast-export master~2..master >actual && From patchwork Wed Sep 25 01:40:04 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elijah Newren X-Patchwork-Id: 11159901 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DE85F18A6 for ; Wed, 25 Sep 2019 01:40:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id BC9EB20874 for ; Wed, 25 Sep 2019 01:40:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="WddjyPQ+" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2442112AbfIYBkV (ORCPT ); Tue, 24 Sep 2019 21:40:21 -0400 Received: from mail-pg1-f196.google.com ([209.85.215.196]:45209 "EHLO mail-pg1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2439273AbfIYBkU (ORCPT ); Tue, 24 Sep 2019 21:40:20 -0400 Received: by mail-pg1-f196.google.com with SMTP id 4so2240211pgm.12 for ; Tue, 24 Sep 2019 18:40:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=bnMBZO0iYMIQ7/qnuO4E6933jCQOYpURaQW8mTFkgfI=; b=WddjyPQ+F7rhSp1caUNke1FiF6E49K3Dat77lwZH1JIEvO2r+n1jyqyPyLz3m+e1m8 1xvN3GoAT91kLuuyDOyOSxuUNIITDfrcs0cehRWH9+2ONVGSg63QO3pwSo2KB186RpH9 7Kr79z+Q6ZXQi4cAPq4zEKAPCCRaGKnDrjn1DL03Ei+B5QdESAP+Yiao2mMONbi1kSAX MUqqKbCxYFpE/tSHC+o9VB0FLnHloQTZFSaBCl2bt1VCFLch4EVfHV6duqt4846jOC3B Xst53VUstRFJ3hRnxq8ga4FfrZTaDQ/BQz8pNdhW9Tqbb5PzpVGPaU1ueMfUhXpY9hxG +45g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=bnMBZO0iYMIQ7/qnuO4E6933jCQOYpURaQW8mTFkgfI=; b=huF5mhpp0eHsUXMQtFo39dpm+UT9c3SVJGP5NJmn5ykpgf7Z40sbZQABgUT5c//WoI rdlTLLEwpEbI+pK91VMQ0Z5h9Z40tEpuUZ5dRVUZqVNyjVSL+B8Pnxj10IY0RZWp9nym WQUy1+FSvn6tSJ36Hcp18EjbHINRsPhp9798ZK+IUjY57mgW+d/k5R5XCAsxnr9DDdgd nggn9avZDJFvdiUJn9+rKN1zhm5ga7du9UFjdPhwr6JiOlwfcOhmSSDMeVT2FSQS+kc5 PEIWAdar92kev6FtivpJb0IMpm+2MJIWYm6BPOeNAx5vbwzx6OOtDwEapSo65P01pM66 fz9Q== X-Gm-Message-State: APjAAAX5RYhQKx/3pLSii9m/3SjlmeRUpTEMx+XfFnyY8iZ6o3CPJ4eb jmAzw/Iwmer642nx5N24HvKszOoPUEc= X-Google-Smtp-Source: APXvYqy0DbCP2yCU4o37wlpQDNx9z9GPJIA153yhkSnv27QwQShTX0eLi4bHX1xXhs9XBycXRaaZ3w== X-Received: by 2002:a17:90a:191d:: with SMTP id 29mr3515999pjg.60.1569375619155; Tue, 24 Sep 2019 18:40:19 -0700 (PDT) Received: from newren2-linux.yojoe.local ([8.4.231.67]) by smtp.gmail.com with ESMTPSA id h4sm3759050pgg.81.2019.09.24.18.40.18 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 24 Sep 2019 18:40:18 -0700 (PDT) From: Elijah Newren To: git@vger.kernel.org Cc: gitster@pobox.com, Elijah Newren Subject: [PATCH 7/8] t9350: add tests for tags of things other than a commit Date: Tue, 24 Sep 2019 18:40:04 -0700 Message-Id: <20190925014005.17056-8-newren@gmail.com> X-Mailer: git-send-email 2.23.0.177.g8af0b3ca64 In-Reply-To: <20190925014005.17056-1-newren@gmail.com> References: <20190925014005.17056-1-newren@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Multiple changes here: * add a test for a tag of a blob * add a test for a tag of a tag of a commit * add a comment to the tests for (possibly nested) tags of trees, making it clear that these tests are doing much less than you might expect Signed-off-by: Elijah Newren --- t/t9350-fast-export.sh | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh index b3fca6ffba..9ab281e4b9 100755 --- a/t/t9350-fast-export.sh +++ b/t/t9350-fast-export.sh @@ -540,10 +540,41 @@ test_expect_success 'tree_tag' ' ' # NEEDSWORK: not just check return status, but validate the output +# Note that these tests DO NOTHING other than print a warning that +# they are ommitting the one tag we asked them to export (because the +# tags resolve to a tree). They exist just to make sure we do not +# abort but instead just warn. test_expect_success 'tree_tag-obj' 'git fast-export tree_tag-obj' test_expect_success 'tag-obj_tag' 'git fast-export tag-obj_tag' test_expect_success 'tag-obj_tag-obj' 'git fast-export tag-obj_tag-obj' +test_expect_success 'handling tags of blobs' ' + git tag -a -m "Tag of a blob" blobtag $(git rev-parse master:file) && + git fast-export blobtag >actual && + cat >expect <<-EOF && + blob + mark :1 + data 9 + die Luft + + tag blobtag + from :1 + tagger $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE + data 14 + Tag of a blob + + EOF + test_cmp expect actual +' + +test_expect_failure 'handling nested tags' ' + git tag -a -m "This is a nested tag" nested muss && + git fast-export --mark-tags nested >output && + grep "^from $ZERO_OID$" output && + grep "^tag nested$" output >tag_lines && + test_line_count = 2 tag_lines +' + test_expect_success 'directory becomes symlink' ' git init dirtosymlink && git init result && From patchwork Wed Sep 25 01:40:05 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elijah Newren X-Patchwork-Id: 11159903 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 15D481850 for ; Wed, 25 Sep 2019 01:40:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id E787220874 for ; Wed, 25 Sep 2019 01:40:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="l5O1WH21" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2442127AbfIYBkW (ORCPT ); Tue, 24 Sep 2019 21:40:22 -0400 Received: from mail-pf1-f172.google.com ([209.85.210.172]:44325 "EHLO mail-pf1-f172.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2439285AbfIYBkV (ORCPT ); Tue, 24 Sep 2019 21:40:21 -0400 Received: by mail-pf1-f172.google.com with SMTP id q21so2365369pfn.11 for ; Tue, 24 Sep 2019 18:40:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Zlkoi6rTsj/rKOoXQBdiTxBZXu/PGCaRIpjv1vvMjKw=; b=l5O1WH21A8Ui4xyQHs+s+QSydvdbB+4/t1uF4KgJm3LWXgOHOe4L/I4J2hJDdc8VQI K5mdNqCG74CcQANSkLxTSuo42a/YZd+W/mbd5NrU2cssQzJiaQMZ22yn3daDu3ScNfQQ rWLnWBa0DN59njJvbbJoBeFh3RIBCnau6O7aeeV2P9mJPv3pjUqkV/2i3dMp+6PcJNCB mrIDvqZVWXZdqQWxaeZa6NRlpNXFygfoE46TWiQC1hUOpL8ViUYWPpd5k1Mn8mMTzVrG sBhKdX9Bn9XSac7fcz1YEcbP3toBPhsvDasu7U9F79fDQZU1IWt74pd+prdqwfke9WAb j9dQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Zlkoi6rTsj/rKOoXQBdiTxBZXu/PGCaRIpjv1vvMjKw=; b=tQaU5wiuJ3YAl146tpLH/NYs1iJO6z8JVgnrkW9LYgdRHURANC1RNBT+9hWw0i7fRs OQzl7U8PAcHV7ZJpyH8XPoCFjM0AsZcsDWkB1Oc5uxZyhh85dggw/DqTOYtI7Mun7ELn vmOC6kkj9rZCgvpPKRs4qyQNchIVedOsuAYW7EgqZlXtWobGJQ7CTgo6eBPDzlc+jlep OKYmTmwiXcWuwXjE4BfLrjEaLPPlnI5TP4DikDPSfwTIs+caDitYCyyXhIXiAU23DT1H aJQDiOk+D6s6A0THcW3iK2klJ3ZiX5pkK1qgtYpE3V/EMx33PV+LNDL+HddfZhnwcO1R O8XA== X-Gm-Message-State: APjAAAXszu2OcqQyxoz3Rt2HQV2ogZXMeYF6UsFk0bqz3iLW8BJ6MJNx FoMJ2xxyyu8EVUp4HlkFRPND9YNcjko= X-Google-Smtp-Source: APXvYqy04ELIcKxyr8mavXg5ZZOIcYeEQPz6rTyBit/0cNVZ5uozPmCAmtBmWlmSKUi1GHI7sFlD5w== X-Received: by 2002:a17:90a:db47:: with SMTP id u7mr3347482pjx.40.1569375620061; Tue, 24 Sep 2019 18:40:20 -0700 (PDT) Received: from newren2-linux.yojoe.local ([8.4.231.67]) by smtp.gmail.com with ESMTPSA id h4sm3759050pgg.81.2019.09.24.18.40.19 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 24 Sep 2019 18:40:19 -0700 (PDT) From: Elijah Newren To: git@vger.kernel.org Cc: gitster@pobox.com, Elijah Newren Subject: [PATCH 8/8] fast-export: handle nested tags Date: Tue, 24 Sep 2019 18:40:05 -0700 Message-Id: <20190925014005.17056-9-newren@gmail.com> X-Mailer: git-send-email 2.23.0.177.g8af0b3ca64 In-Reply-To: <20190925014005.17056-1-newren@gmail.com> References: <20190925014005.17056-1-newren@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Signed-off-by: Elijah Newren --- builtin/fast-export.c | 30 ++++++++++++++++++------------ t/t9350-fast-export.sh | 2 +- 2 files changed, 19 insertions(+), 13 deletions(-) diff --git a/builtin/fast-export.c b/builtin/fast-export.c index d32e1e9327..58a74de42a 100644 --- a/builtin/fast-export.c +++ b/builtin/fast-export.c @@ -843,22 +843,28 @@ static void handle_tag(const char *name, struct tag *tag) free(buf); return; case REWRITE: - if (tagged->type != OBJ_COMMIT) { - die("tag %s tags unexported %s!", - oid_to_hex(&tag->object.oid), - type_name(tagged->type)); - } - p = rewrite_commit((struct commit *)tagged); - if (!p) { - printf("reset %s\nfrom %s\n\n", - name, oid_to_hex(&null_oid)); - free(buf); - return; + if (tagged->type == OBJ_TAG && !mark_tags) { + die(_("Error: Cannot export nested tags unless --mark-tags is specified.")); + } else if (tagged->type == OBJ_COMMIT) { + p = rewrite_commit((struct commit *)tagged); + if (!p) { + printf("reset %s\nfrom %s\n\n", + name, oid_to_hex(&null_oid)); + free(buf); + return; + } + tagged_mark = get_object_mark(&p->object); + } else { + /* tagged->type is either OBJ_BLOB or OBJ_TAG */ + tagged_mark = get_object_mark(tagged); } - tagged_mark = get_object_mark(&p->object); } } + if (tagged->type == OBJ_TAG) { + printf("reset %s\nfrom %s\n\n", + name, oid_to_hex(&null_oid)); + } if (starts_with(name, "refs/tags/")) name += 10; printf("tag %s\n", name); diff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh index 9ab281e4b9..2e4e214815 100755 --- a/t/t9350-fast-export.sh +++ b/t/t9350-fast-export.sh @@ -567,7 +567,7 @@ test_expect_success 'handling tags of blobs' ' test_cmp expect actual ' -test_expect_failure 'handling nested tags' ' +test_expect_success 'handling nested tags' ' git tag -a -m "This is a nested tag" nested muss && git fast-export --mark-tags nested >output && grep "^from $ZERO_OID$" output &&