From patchwork Mon Sep 30 21:10:11 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elijah Newren X-Patchwork-Id: 11167601 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8B23613B1 for ; Mon, 30 Sep 2019 21:10:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 69813206BB for ; Mon, 30 Sep 2019 21:10:26 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="FL/ie33P" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732624AbfI3VKZ (ORCPT ); Mon, 30 Sep 2019 17:10:25 -0400 Received: from mail-pg1-f195.google.com ([209.85.215.195]:37897 "EHLO mail-pg1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732486AbfI3VKZ (ORCPT ); Mon, 30 Sep 2019 17:10:25 -0400 Received: by mail-pg1-f195.google.com with SMTP id x10so8053994pgi.5 for ; Mon, 30 Sep 2019 14:10:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=nXvlWDLlBYDCrkZHNlzFS2ZcuhsmhP0cLWtOEIhls4Y=; b=FL/ie33PkEbdx6mJON0YQxY4JRuiHrO1L3aN+Rx2Ibe3KfDMH467z+51WNbsWHGxvD xnaRMYWB+LZ0qkeq8IT7+lXrFSlgL+h1RkdQmCAEDX5kJFGOHriV+pm8BorXAgsKXrWQ VGsYOWsJx7YIebUJnKDkDc3EJua4DxZsORY+5invaoJu1pI7hxY7vfhnBmLk/EkmT1x1 fbBer9jmb5b2ilOVKw8R+HjwOqzdtg6yDhaOXfTwJANc3PZJoImxZXD/WO4mS0+3hViw GccyH+rqFErRq7gs5wSP19JD503xrttZYGFkMo+eAdnCj4kirJfH32uL3LKQcjEnozub O/TQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=nXvlWDLlBYDCrkZHNlzFS2ZcuhsmhP0cLWtOEIhls4Y=; b=P+d0Oz9+N64/dxVsLMSRT6O05Ea7fR+yI5smOPRdvC2BRWdqydZsypJ6ddDqDyy9fX 6yMBQ2amJyKfyZRdzggcb3/msgh48k1jIqxwq5/pT5QXZn4VwGFsrJ6XxF2MLat6H+H7 weEXUYHbCOMp9baUHxfdMwqapFZgz05SFjSwuBEMnkzurtNZ+5EQMgo/Ob+fMI/CgHhv UOmQXNwRj9/8dnXzhJPcY4R92k0kcBVyUuccfYjy73bOaS47QmRPJH7hBCyaZPlVsUf6 2jdG/fZrKjoAQ2JS7LEdJjEsQ2Ez4b/sMhLY9V+8nGMweyKtjzM7NaCAj194BPNFntoP 0kig== X-Gm-Message-State: APjAAAXDJq+vHncGS0j+c65vIXGh4TbUHkF+MD9uDozf8oKn9NdWUrMP RQWQSZdAFbzNCBlatOPfoWqQQdRr9pw= X-Google-Smtp-Source: APXvYqyI7XLWlSA5vDSITknNda3/ynllrH5fRVWsHHFu+akTMzKxqe/9XisXdlnyTmt0YVQq6v0gVw== X-Received: by 2002:a63:2406:: with SMTP id k6mr15495739pgk.420.1569877824074; Mon, 30 Sep 2019 14:10:24 -0700 (PDT) Received: from newren2-linux.yojoe.local ([8.4.231.67]) by smtp.gmail.com with ESMTPSA id w11sm13404609pfd.116.2019.09.30.14.10.22 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 30 Sep 2019 14:10:23 -0700 (PDT) From: Elijah Newren To: git@vger.kernel.org Cc: Junio C Hamano , Elijah Newren Subject: [PATCH v2 1/8] fast-export: fix exporting a tag and nothing else Date: Mon, 30 Sep 2019 14:10:11 -0700 Message-Id: <20190930211018.23633-2-newren@gmail.com> X-Mailer: git-send-email 2.23.0.264.gac739dbb79 In-Reply-To: <20190930211018.23633-1-newren@gmail.com> References: <20190925014005.17056-1-newren@gmail.com> <20190930211018.23633-1-newren@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org fast-export allows specifying revision ranges, which can be used to export a tag without exporting the commit it tags. fast-export handled this rather poorly: it would emit a "from :0" directive. Since marks start at 1 and increase, this means it refers to an unknown commit and fast-import will choke on the input. When we are unable to look up a mark for the object being tagged, use a "from $HASH" directive instead to fix this problem. Note that this is quite similar to the behavior fast-export exhibits with commits and parents when --reference-excluded-parents is passed along with an excluded commit range. For tags of excluded commits we do not require the --reference-excluded-parents flag because we always have to tag something. By contrast, when dealing with commits, pruning a parent is always a viable option, so we need the flag to specify that parent pruning is not wanted. (It is slightly weird that --reference-excluded-parents isn't the default with a separate --prune-excluded-parents flag, but backward compatibility concerns resulted in the current defaults.) Signed-off-by: Elijah Newren --- builtin/fast-export.c | 7 ++++++- t/t9350-fast-export.sh | 13 +++++++++++++ 2 files changed, 19 insertions(+), 1 deletion(-) diff --git a/builtin/fast-export.c b/builtin/fast-export.c index f541f55d33..5822271c6b 100644 --- a/builtin/fast-export.c +++ b/builtin/fast-export.c @@ -860,7 +860,12 @@ static void handle_tag(const char *name, struct tag *tag) if (starts_with(name, "refs/tags/")) name += 10; - printf("tag %s\nfrom :%d\n", name, tagged_mark); + printf("tag %s\n", name); + if (tagged_mark) + printf("from :%d\n", tagged_mark); + else + printf("from %s\n", oid_to_hex(&tagged->oid)); + if (show_original_ids) printf("original-oid %s\n", oid_to_hex(&tag->object.oid)); printf("%.*s%sdata %d\n%.*s\n", diff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh index b4004e05c2..d32ff41859 100755 --- a/t/t9350-fast-export.sh +++ b/t/t9350-fast-export.sh @@ -53,6 +53,19 @@ test_expect_success 'fast-export | fast-import' ' ' +test_expect_success 'fast-export ^muss^{commit} muss' ' + git fast-export --tag-of-filtered-object=rewrite ^muss^{commit} muss >actual && + cat >expected <<-EOF && + tag muss + from $(git rev-parse --verify muss^{commit}) + $(git cat-file tag muss | grep tagger) + data 9 + valentin + + EOF + test_cmp expected actual +' + test_expect_success 'fast-export master~2..master' ' git fast-export master~2..master >actual && From patchwork Mon Sep 30 21:10:12 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elijah Newren X-Patchwork-Id: 11167603 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DB14C16B1 for ; Mon, 30 Sep 2019 21:10:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id B983F205C9 for ; Mon, 30 Sep 2019 21:10:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Mws0J9Ve" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732626AbfI3VK0 (ORCPT ); Mon, 30 Sep 2019 17:10:26 -0400 Received: from mail-pf1-f196.google.com ([209.85.210.196]:36346 "EHLO mail-pf1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732486AbfI3VK0 (ORCPT ); Mon, 30 Sep 2019 17:10:26 -0400 Received: by mail-pf1-f196.google.com with SMTP id y22so6303418pfr.3 for ; Mon, 30 Sep 2019 14:10:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=6GhKKMc42LQZvrmWjslEfVGBS5TOHNJQfKnYhcBhBAc=; b=Mws0J9VeeFEWdETX36nxh5chZdiWr5lxGQDXqXNcmXXxeaWPHDqwnKHP+ncj5xlnbG 3uCm7N7m5CTmPhho7aTcXM0MFgWM9fRdsSxsZpfjmyHCx8691saecG949RWwPQfnl1Hl lKDEoYIfiCSTR/SzFV/lfHtrI5aD1idjKGKXnX4/Md86zxdV5Bas5uw0MOFkUI17AiW/ a56iKq4dadgWTH9T9/PNQk7ZlPKzcpG3/BD8yHk5tT3xBk3Y1hKoY9WBAj2NAM8SQ+XY QitCne+fymGgI9mzSNwfl74w8ijdnCxF2IfO5mKxjVbczvdGY5OFt15+btL/1rbnRCHP cj5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=6GhKKMc42LQZvrmWjslEfVGBS5TOHNJQfKnYhcBhBAc=; b=sORYhrRu9kv4b7Hdp9kT25Pc7rx+Ek+EDbGVhs5/5kv92RDk6g2k37R67EVE3og0XI fqvrk4X0DrQFgqzTp/FHRKLaotj7pjbkLsV6nNTfN7op6LdC0zjj0/OaNzeHIuGHpVTa xOVFw3U6UBr0N3L0QN0SvzRQyLK0oLgSE8IPJF4ZFoWwipcH4O36YP7AqLKc9Ucsz1YV JSWrbhed+EokhHuL5E+fsq33u9h3tB1rjyZUAoc9v52QvvavibKJDzg29asHVoC7cdaO BiXvZb5LKL/xe+ecYUb+aM+x1j+o7yxO0y+yiHVINoA5TC4J+D9kaleOo20RSC4/ZEbu tsHg== X-Gm-Message-State: APjAAAXEDsipWHS2PaAhdK2FBw8/0e98QCo0H9V/6DLJ7pBc4xUseCiB EHV0JUg4SsvU5e4fLWJS0f/ePBjyYro= X-Google-Smtp-Source: APXvYqz97XzH6MAxRILdEXk4GJVRkRmO96AKm2kfD7Xop2na7XA89eTrmTfWgxKqWI/zglRi20+YrA== X-Received: by 2002:a63:504b:: with SMTP id q11mr26165508pgl.188.1569877825130; Mon, 30 Sep 2019 14:10:25 -0700 (PDT) Received: from newren2-linux.yojoe.local ([8.4.231.67]) by smtp.gmail.com with ESMTPSA id w11sm13404609pfd.116.2019.09.30.14.10.24 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 30 Sep 2019 14:10:24 -0700 (PDT) From: Elijah Newren To: git@vger.kernel.org Cc: Junio C Hamano , Elijah Newren Subject: [PATCH v2 2/8] fast-import: fix handling of deleted tags Date: Mon, 30 Sep 2019 14:10:12 -0700 Message-Id: <20190930211018.23633-3-newren@gmail.com> X-Mailer: git-send-email 2.23.0.264.gac739dbb79 In-Reply-To: <20190930211018.23633-1-newren@gmail.com> References: <20190925014005.17056-1-newren@gmail.com> <20190930211018.23633-1-newren@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org If our input stream includes a tag which is later deleted, we were not properly deleting it. We did have a step which would delete it, but we left a tag in the tag list noting that it needed to be updated, and the updating of annotated tags occurred AFTER ref deletion. So, when we record that a tag needs to be deleted, also remove it from the list of annotated tags to update. While this has likely been something that has not happened in practice, it will come up more in order to support nested tags. For nested tags, we either need to give temporary names to the intermediate tags and then delete them, or else we need to use the final name for the intermediate tags. If we use the final name for the intermediate tags, then in order to keep the sanity check that someone doesn't try to update the same tag twice, we need to delete the ref after creating the intermediate tag. So, either way nested tags imply the need to delete temporary inner tag references. Signed-off-by: Elijah Newren --- fast-import.c | 29 +++++++++++++++++++++++++++++ t/t9300-fast-import.sh | 13 +++++++++++++ 2 files changed, 42 insertions(+) diff --git a/fast-import.c b/fast-import.c index b44d6a467e..546da3a938 100644 --- a/fast-import.c +++ b/fast-import.c @@ -2793,6 +2793,35 @@ static void parse_reset_branch(const char *arg) b = new_branch(arg); read_next_command(); parse_from(b); + if (b->delete && !strncmp(b->name, "refs/tags/", 10)) { + /* + * Elsewhere, we call dump_branches() before dump_tags(), + * and dump_branches() will handle ref deletions first, so + * in order to make sure the deletion actually takes effect, + * we need to remove the tag from our list of tags to update. + * + * NEEDSWORK: replace list of tags with hashmap for faster + * deletion? + */ + struct strbuf tag_name = STRBUF_INIT; + struct tag *t, *prev = NULL; + for (t = first_tag; t; t = t->next_tag) { + strbuf_reset(&tag_name); + strbuf_addf(&tag_name, "refs/tags/%s", t->name); + if (!strcmp(b->name, tag_name.buf)) + break; + prev = t; + } + if (t) { + if (prev) + prev->next_tag = t->next_tag; + else + first_tag = t->next_tag; + if (!t->next_tag) + last_tag = prev; + /* There is no mem_pool_free(t) function to call. */ + } + } if (command_buf.len > 0) unread_command_buf = 1; } diff --git a/t/t9300-fast-import.sh b/t/t9300-fast-import.sh index 141b7fa35e..74bc41333b 100755 --- a/t/t9300-fast-import.sh +++ b/t/t9300-fast-import.sh @@ -85,6 +85,15 @@ test_expect_success 'A: create pack from stdin' ' An annotated tag that annotates a blob. EOF + tag to-be-deleted + from :3 + data <expect <<-EOF && :2 $(git rev-parse --verify master:file2) From patchwork Mon Sep 30 21:10:13 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elijah Newren X-Patchwork-Id: 11167607 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B253E16B1 for ; Mon, 30 Sep 2019 21:10:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 9389D206BB for ; Mon, 30 Sep 2019 21:10:28 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="nLKR22IA" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732629AbfI3VK1 (ORCPT ); Mon, 30 Sep 2019 17:10:27 -0400 Received: from mail-pf1-f174.google.com ([209.85.210.174]:43782 "EHLO mail-pf1-f174.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732627AbfI3VK1 (ORCPT ); Mon, 30 Sep 2019 17:10:27 -0400 Received: by mail-pf1-f174.google.com with SMTP id a2so6286642pfo.10 for ; Mon, 30 Sep 2019 14:10:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=xw9PesypsbgQp7G0GGZXS1KbVnHdc8F6rT05h/ptazc=; b=nLKR22IADkXvFyayrR1nJZ309NTS1yl1mRB7d2iQijGN06h/m0WrxO9AZIyKZTuwtY bHDjgr1SZhVC/r9ST/AlJ//7mwPjX99MS6V5PDw4p8vl93tR8fKfqaGz6mW382RTY1IJ t6nGDHp1rrW7xLvnefh/NCSCVwXaS7ffPUHNoNkwO4/qfGrarkYpqJ89XKPFs9CAMAJl B3ZSqJuH7dKGgQtdBhkt96itzrpLVhpKYh8YMw5SaHnBYmVM8TxKeH7rIOBN7OP4dEId J2u5cn0wBZYpTLw1mcTzG3qwBiSGxuV2PA1QnTyH4Hl4EWyuS4JNDb/7KdKwTTtWK4WS bZzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=xw9PesypsbgQp7G0GGZXS1KbVnHdc8F6rT05h/ptazc=; b=hnEUdLzF3CURS2kGcCq6Z8WUMyWZJxxbFzi+X4lSn9QJe80lIcYTbk6PxwDQIFf1Vw jaSkhaYgx/Ws15ogAhgelU7Z91YG46uhf5AXChyXp4ytBmOgfeQPq4Vy9jZM5tiSE63F f+019F4G3apaQp/jA6GQwG5P2Ndu3aOAWajkJT/ktxCDl4cVNzIPvzxotUZ2+4mKst+d YIIVvGrURvsSqL+bOb2EcFFOuFwnNMDoPJ/c2dr+nrV35xKuu8DsvhCHPCzIpXH4guwz rashEnexBkXQT5/45hRz+HI85PSzOPlpwBPjex4NuLs9zYiD2qFh3ad8XYNmbzhPWOJ0 XUfQ== X-Gm-Message-State: APjAAAWcAQfEns/+nu1mPV8s/U0l8gWmM/f19MfX2Duf7dNsorcMiqUA 54FeaHpRb4tzom/y/OESblUzkhd0Ed0= X-Google-Smtp-Source: APXvYqz44YNW/eLMnzIoIv33QNlRmpeZ0TfApXIQQo0FQHH1yRlduQCnSYTMSvYa1iXdhLuf9w7pkw== X-Received: by 2002:a62:e917:: with SMTP id j23mr23719340pfh.50.1569877826223; Mon, 30 Sep 2019 14:10:26 -0700 (PDT) Received: from newren2-linux.yojoe.local ([8.4.231.67]) by smtp.gmail.com with ESMTPSA id w11sm13404609pfd.116.2019.09.30.14.10.25 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 30 Sep 2019 14:10:25 -0700 (PDT) From: Elijah Newren To: git@vger.kernel.org Cc: Junio C Hamano , Elijah Newren Subject: [PATCH v2 3/8] fast-import: allow tags to be identified by mark labels Date: Mon, 30 Sep 2019 14:10:13 -0700 Message-Id: <20190930211018.23633-4-newren@gmail.com> X-Mailer: git-send-email 2.23.0.264.gac739dbb79 In-Reply-To: <20190930211018.23633-1-newren@gmail.com> References: <20190925014005.17056-1-newren@gmail.com> <20190930211018.23633-1-newren@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Mark identifiers are used in fast-export and fast-import to provide a label to refer to earlier content. Blobs are given labels because they need to be referenced in the commits where they first appear with a given filename, and commits are given labels because they can be the parents of other commits. Tags were never given labels, probably because they were viewed as unnecessary, but that presents two problems: 1. It leaves us without a way of referring to previous tags if we want to create a tag of a tag (or higher nestings). 2. It leaves us with no way of recording that a tag has already been imported when using --export-marks and --import-marks. Fix these problems by allowing an optional mark label for tags. Signed-off-by: Elijah Newren --- Documentation/git-fast-import.txt | 1 + fast-import.c | 3 ++- t/t9300-fast-import.sh | 19 +++++++++++++++++++ 3 files changed, 22 insertions(+), 1 deletion(-) diff --git a/Documentation/git-fast-import.txt b/Documentation/git-fast-import.txt index 0bb276269e..4977869465 100644 --- a/Documentation/git-fast-import.txt +++ b/Documentation/git-fast-import.txt @@ -774,6 +774,7 @@ lightweight (non-annotated) tags see the `reset` command below. .... 'tag' SP LF + mark? 'from' SP LF original-oid? 'tagger' (SP )? SP LT GT SP LF diff --git a/fast-import.c b/fast-import.c index 546da3a938..0042440487 100644 --- a/fast-import.c +++ b/fast-import.c @@ -2713,6 +2713,7 @@ static void parse_new_tag(const char *arg) first_tag = t; last_tag = t; read_next_command(); + parse_mark(); /* from ... */ if (!skip_prefix(command_buf.buf, "from ", &from)) @@ -2769,7 +2770,7 @@ static void parse_new_tag(const char *arg) strbuf_addbuf(&new_data, &msg); free(tagger); - if (store_object(OBJ_TAG, &new_data, NULL, &t->oid, 0)) + if (store_object(OBJ_TAG, &new_data, NULL, &t->oid, next_mark)) t->pack_id = MAX_PACK_ID; else t->pack_id = pack_id; diff --git a/t/t9300-fast-import.sh b/t/t9300-fast-import.sh index 74bc41333b..3ad2b2f1ba 100755 --- a/t/t9300-fast-import.sh +++ b/t/t9300-fast-import.sh @@ -94,6 +94,23 @@ test_expect_success 'A: create pack from stdin' ' reset refs/tags/to-be-deleted from 0000000000000000000000000000000000000000 + tag nested + mark :6 + from :4 + data < X-Patchwork-Id: 11167609 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 27722912 for ; Mon, 30 Sep 2019 21:10:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id EA467205C9 for ; Mon, 30 Sep 2019 21:10:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="KxpuMEML" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732636AbfI3VKa (ORCPT ); Mon, 30 Sep 2019 17:10:30 -0400 Received: from mail-pg1-f195.google.com ([209.85.215.195]:43952 "EHLO mail-pg1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732627AbfI3VK2 (ORCPT ); Mon, 30 Sep 2019 17:10:28 -0400 Received: by mail-pg1-f195.google.com with SMTP id v27so8030960pgk.10 for ; Mon, 30 Sep 2019 14:10:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=GFKgcZauxsdso775qOgmHA1C2fKfz0Ftwuh6j+795w0=; b=KxpuMEML1vRfcGlRW0y5F9x5FgXBai9xQxtKVAagpGp48Ysdmj/ZXpjP3OWAsRaAXf oHTDP1QYxa6E41jOgCrGwBf3LvTJQAn/edYRNkxQPJkxm97rVcMIjGiFon9AQ0gqmuwI RNxO2bNwRUVhFtRES0Gkb24pBf3ILTUY1L5FkFmvNdHatjiq7ZgQ7x5R8HanSMfMV+M/ O5168usQxtAqehzKQT2dXpjbUfWxU78K783juEiuk9BiCeImrKJWXt6dpWwmJQy7tu+y C3e29pSEiRg9RR8KfR7CHfodaUXq7sZ9I/jeILg1lASZLZSnyE6uqjV2lPUEH7OMpfJL Zjkw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=GFKgcZauxsdso775qOgmHA1C2fKfz0Ftwuh6j+795w0=; b=e+Va3xmM7gqSUrCh69VwHloDN/oNbjfENdmMcaVFEObXH6dJ9tspdybq/QzsXdcFMX +48rfrdb1OYeFHE9trk7VlWhjsqK9DWioLQjmQ9ruBSAfP6P9QEo9CA5T0wUcXXTzXuo DvGv22QrcbPVi++vvYjRWXeav/Cmn82GDTjxD3nctvz26aSzsbsm3NlrzVhM+1WevP0L 9pXxcM2eZCx3ykb1AgS1zKWGCjTn2nwGRZMjlgMDrTbSErVPTHDky+hFX3e1QmF+RLQY RB4JN2a4Nc0F8xSqHjQzWKtgewwwxzI6hRafhPvJB5u/3o+Ev0b3U8ELJHx064ILAwmN epUw== X-Gm-Message-State: APjAAAVXUZxEIpS838df8ZxT5yJftqUKcGXngJoGNmNCRpp49+xy/YZP uODqsK58ikAqagtybKM22Lpb/UUhUls= X-Google-Smtp-Source: APXvYqy/SP23qbDjmKyGIJJJUwTmqJDJ9h2jpsCz8BGd386EHzB2dPthSZH6tSeNc8C8gaDi4wS6Ig== X-Received: by 2002:a65:6102:: with SMTP id z2mr26135164pgu.391.1569877827361; Mon, 30 Sep 2019 14:10:27 -0700 (PDT) Received: from newren2-linux.yojoe.local ([8.4.231.67]) by smtp.gmail.com with ESMTPSA id w11sm13404609pfd.116.2019.09.30.14.10.26 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 30 Sep 2019 14:10:26 -0700 (PDT) From: Elijah Newren To: git@vger.kernel.org Cc: Junio C Hamano , Elijah Newren Subject: [PATCH v2 4/8] fast-import: add support for new 'alias' command Date: Mon, 30 Sep 2019 14:10:14 -0700 Message-Id: <20190930211018.23633-5-newren@gmail.com> X-Mailer: git-send-email 2.23.0.264.gac739dbb79 In-Reply-To: <20190930211018.23633-1-newren@gmail.com> References: <20190925014005.17056-1-newren@gmail.com> <20190930211018.23633-1-newren@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org fast-export and fast-import have nice --import-marks flags which allow for incremental migrations. However, if there is a mark in fast-export's file of marks without a corresponding mark in the one for fast-import, then we run the risk that fast-export tries to send new objects relative to the mark it knows which fast-import does not, causing fast-import to fail. This arises in practice when there is a filter of some sort running between the fast-export and fast-import processes which prunes some commits programmatically. Provide such a filter with the ability to alias pruned commits to their most recent non-pruned ancestor. Signed-off-by: Elijah Newren --- Documentation/git-fast-import.txt | 22 +++++++++++ fast-import.c | 62 ++++++++++++++++++++++++++----- t/t9300-fast-import.sh | 5 +++ 3 files changed, 79 insertions(+), 10 deletions(-) diff --git a/Documentation/git-fast-import.txt b/Documentation/git-fast-import.txt index 4977869465..a3f1e0c5e4 100644 --- a/Documentation/git-fast-import.txt +++ b/Documentation/git-fast-import.txt @@ -337,6 +337,13 @@ and control the current import process. More detailed discussion `commit` command. This command is optional and is not needed to perform an import. +`alias`:: + Record that a mark refers to a given object without first + creating any new object. Using --import-marks and referring + to missing marks will cause fast-import to fail, so aliases + can provide a way to set otherwise pruned commits to a valid + value (e.g. the nearest non-pruned ancestor). + `checkpoint`:: Forces fast-import to close the current packfile, generate its unique SHA-1 checksum and index, and start a new packfile. @@ -914,6 +921,21 @@ a data chunk which does not have an LF as its last byte. + The `LF` after ` LF` is optional (it used to be required). +`alias` +~~~~~~~ +Record that a mark refers to a given object without first creating any +new object. + +.... + 'alias' LF + mark + 'to' SP LF + LF? +.... + +For a detailed description of `` see above under `from`. + + `checkpoint` ~~~~~~~~~~~~ Forces fast-import to close the current packfile, start a new one, and to diff --git a/fast-import.c b/fast-import.c index 0042440487..9a12850d16 100644 --- a/fast-import.c +++ b/fast-import.c @@ -2491,18 +2491,14 @@ static void parse_from_existing(struct branch *b) } } -static int parse_from(struct branch *b) +static int parse_objectish(struct branch *b, const char *objectish) { - const char *from; struct branch *s; struct object_id oid; - if (!skip_prefix(command_buf.buf, "from ", &from)) - return 0; - oidcpy(&oid, &b->branch_tree.versions[1].oid); - s = lookup_branch(from); + s = lookup_branch(objectish); if (b == s) die("Can't create a branch from itself: %s", b->name); else if (s) { @@ -2510,8 +2506,8 @@ static int parse_from(struct branch *b) oidcpy(&b->oid, &s->oid); oidcpy(&b->branch_tree.versions[0].oid, t); oidcpy(&b->branch_tree.versions[1].oid, t); - } else if (*from == ':') { - uintmax_t idnum = parse_mark_ref_eol(from); + } else if (*objectish == ':') { + uintmax_t idnum = parse_mark_ref_eol(objectish); struct object_entry *oe = find_mark(idnum); if (oe->type != OBJ_COMMIT) die("Mark :%" PRIuMAX " not a commit", idnum); @@ -2525,13 +2521,13 @@ static int parse_from(struct branch *b) } else parse_from_existing(b); } - } else if (!get_oid(from, &b->oid)) { + } else if (!get_oid(objectish, &b->oid)) { parse_from_existing(b); if (is_null_oid(&b->oid)) b->delete = 1; } else - die("Invalid ref name or SHA1 expression: %s", from); + die("Invalid ref name or SHA1 expression: %s", objectish); if (b->branch_tree.tree && !oideq(&oid, &b->branch_tree.versions[1].oid)) { release_tree_content_recursive(b->branch_tree.tree); @@ -2542,6 +2538,26 @@ static int parse_from(struct branch *b) return 1; } +static int parse_from(struct branch *b) +{ + const char *from; + + if (!skip_prefix(command_buf.buf, "from ", &from)) + return 0; + + return parse_objectish(b, from); +} + +static int parse_objectish_with_prefix(struct branch *b, const char *prefix) +{ + const char *base; + + if (!skip_prefix(command_buf.buf, prefix, &base)) + return 0; + + return parse_objectish(b, base); +} + static struct hash_list *parse_merge(unsigned int *count) { struct hash_list *list = NULL, **tail = &list, *n; @@ -3089,6 +3105,28 @@ static void parse_progress(void) skip_optional_lf(); } +static void parse_alias(void) +{ + struct object_entry *e; + struct branch b; + + skip_optional_lf(); + read_next_command(); + + /* mark ... */ + parse_mark(); + if (!next_mark) + die(_("Expected 'mark' command, got %s"), command_buf.buf); + + /* to ... */ + memset(&b, 0, sizeof(b)); + if (!parse_objectish_with_prefix(&b, "to ")) + die(_("Expected 'to' command, got %s"), command_buf.buf); + e = find_object(&b.oid); + assert(e); + insert_mark(next_mark, e); +} + static char* make_fast_import_path(const char *path) { if (!relative_marks_paths || is_absolute_path(path)) @@ -3216,6 +3254,8 @@ static int parse_one_feature(const char *feature, int from_stream) option_import_marks(arg, from_stream, 1); } else if (skip_prefix(feature, "export-marks=", &arg)) { option_export_marks(arg); + } else if (!strcmp(feature, "alias")) { + ; /* Don't die - this feature is supported */ } else if (!strcmp(feature, "get-mark")) { ; /* Don't die - this feature is supported */ } else if (!strcmp(feature, "cat-blob")) { @@ -3372,6 +3412,8 @@ int cmd_main(int argc, const char **argv) parse_checkpoint(); else if (!strcmp("done", command_buf.buf)) break; + else if (!strcmp("alias", command_buf.buf)) + parse_alias(); else if (starts_with(command_buf.buf, "progress ")) parse_progress(); else if (skip_prefix(command_buf.buf, "feature ", &v)) diff --git a/t/t9300-fast-import.sh b/t/t9300-fast-import.sh index 3ad2b2f1ba..41f2a1dad9 100755 --- a/t/t9300-fast-import.sh +++ b/t/t9300-fast-import.sh @@ -111,6 +111,10 @@ test_expect_success 'A: create pack from stdin' ' Tag of tag of our lovely commit EOF + alias + mark :8 + to :5 + INPUT_END git fast-import --export-marks=marks.out X-Patchwork-Id: 11167611 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B288016B1 for ; Mon, 30 Sep 2019 21:10:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 90F61206BB for ; Mon, 30 Sep 2019 21:10:31 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="JmrNJiz7" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732634AbfI3VKa (ORCPT ); Mon, 30 Sep 2019 17:10:30 -0400 Received: from mail-pg1-f194.google.com ([209.85.215.194]:35207 "EHLO mail-pg1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732631AbfI3VK3 (ORCPT ); Mon, 30 Sep 2019 17:10:29 -0400 Received: by mail-pg1-f194.google.com with SMTP id a24so8072215pgj.2 for ; Mon, 30 Sep 2019 14:10:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=HBylQdXHDFOeJEXetVuakLucf0tFU0UT7RS+6a+MhRw=; b=JmrNJiz7ebbXliEtTO2uwJsCzC3v6imsvdLhxATybX4qDg1Hvt98g695Ym9gnB301l tsaVqMjo3m3WQ6wyHPdAu5miyV45jW8dJ4pHiyBPPTeTEQDach+p5k42jQq6/+6LiCvC vLKcTRl1ICwDYtgFJxR9924Wqsc/32W/ecFtlpHNnf0ZAsTIsKVic4Dv/c/A4FumZole sU0yQwH3ocW7HjPFQvTS71dL5V1NFyOnlEEeGUSPF3oPcDgOewJ3mN+5YS7yVK7/sTCH 1wCMUKkyYXjH67B/z15wMU7zLXdNVtjurZviy112n/RzPfzKMN68jIiU6yV4fH5mwhuQ XzXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HBylQdXHDFOeJEXetVuakLucf0tFU0UT7RS+6a+MhRw=; b=P3Fz+AqtLSXZuAweH8YvKO7LhPpC0CK2ISjY4bsuUFGZLqGDp26mCIG1QE3Rbe1HTl 1VX0k1nDbL73dnVTRkAcvgz5UMHMNH2e/OKqLZ/lRUDJUHiZBgb91ZpuTprJqwDZIq+r 0fwuQIJPAj1/8643IxACPy4G8FtRYCImegU91a5QTzIh0UhsM9TX6X6+AsSfwe1Nv/iL aT3GyJNza9n3/c3s9mLvb3GZFAJadpzCV2ODHWshGpI54ErcV635kWNRTRHlJqftA1mh 0vSWs1QkN9toZaVnLtHLzHim6w1ilCAi+Fesy5MQvFzJkeLoUyuiQ06ai2+LYmd3AIQy 6/ow== X-Gm-Message-State: APjAAAXlIKP2309+L+Rt1GruuAkBX2WVQWy2tyhHJ81Zv854rwDgC2Mh R3S6AWf0JL5KCHHlhwYvSYdD5EsyBOo= X-Google-Smtp-Source: APXvYqzp1iBRQ9dZs67LlQs6eVS4yujFPmAQssGNBEUzAGdjyojLkFRTyin36u0BVONuFV/kGwI/gQ== X-Received: by 2002:a17:90a:cb16:: with SMTP id z22mr1449788pjt.70.1569877828484; Mon, 30 Sep 2019 14:10:28 -0700 (PDT) Received: from newren2-linux.yojoe.local ([8.4.231.67]) by smtp.gmail.com with ESMTPSA id w11sm13404609pfd.116.2019.09.30.14.10.27 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 30 Sep 2019 14:10:27 -0700 (PDT) From: Elijah Newren To: git@vger.kernel.org Cc: Junio C Hamano , Elijah Newren Subject: [PATCH v2 5/8] fast-export: add support for --import-marks-if-exists Date: Mon, 30 Sep 2019 14:10:15 -0700 Message-Id: <20190930211018.23633-6-newren@gmail.com> X-Mailer: git-send-email 2.23.0.264.gac739dbb79 In-Reply-To: <20190930211018.23633-1-newren@gmail.com> References: <20190925014005.17056-1-newren@gmail.com> <20190930211018.23633-1-newren@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org fast-import has support for both an --import-marks flag and an --import-marks-if-exists flag; the latter of which will not die() if the file does not exist. fast-export only had support for an --import-marks flag; add an --import-marks-if-exists flag for consistency. Signed-off-by: Elijah Newren --- builtin/fast-export.c | 23 +++++++++++++++++++---- t/t9350-fast-export.sh | 10 ++++------ 2 files changed, 23 insertions(+), 10 deletions(-) diff --git a/builtin/fast-export.c b/builtin/fast-export.c index 5822271c6b..575e47833b 100644 --- a/builtin/fast-export.c +++ b/builtin/fast-export.c @@ -1052,11 +1052,16 @@ static void export_marks(char *file) error("Unable to write marks file %s.", file); } -static void import_marks(char *input_file) +static void import_marks(char *input_file, int check_exists) { char line[512]; - FILE *f = xfopen(input_file, "r"); + FILE *f; + struct stat sb; + + if (check_exists && stat(input_file, &sb)) + return; + f = xfopen(input_file, "r"); while (fgets(line, sizeof(line), f)) { uint32_t mark; char *line_end, *mark_end; @@ -1120,7 +1125,9 @@ int cmd_fast_export(int argc, const char **argv, const char *prefix) struct rev_info revs; struct object_array commits = OBJECT_ARRAY_INIT; struct commit *commit; - char *export_filename = NULL, *import_filename = NULL; + char *export_filename = NULL, + *import_filename = NULL, + *import_filename_if_exists = NULL; uint32_t lastimportid; struct string_list refspecs_list = STRING_LIST_INIT_NODUP; struct string_list paths_of_changed_objects = STRING_LIST_INIT_DUP; @@ -1140,6 +1147,10 @@ int cmd_fast_export(int argc, const char **argv, const char *prefix) N_("Dump marks to this file")), OPT_STRING(0, "import-marks", &import_filename, N_("file"), N_("Import marks from this file")), + OPT_STRING(0, "import-marks-if-exists", + &import_filename_if_exists, + N_("file"), + N_("Import marks from this file if it exists")), OPT_BOOL(0, "fake-missing-tagger", &fake_missing_tagger, N_("Fake a tagger when tags lack one")), OPT_BOOL(0, "full-tree", &full_tree, @@ -1187,8 +1198,12 @@ int cmd_fast_export(int argc, const char **argv, const char *prefix) if (use_done_feature) printf("feature done\n"); + if (import_filename && import_filename_if_exists) + die(_("Cannot pass both --import-marks and --import-marks-if-exists")); if (import_filename) - import_marks(import_filename); + import_marks(import_filename, 0); + else if (import_filename_if_exists) + import_marks(import_filename_if_exists, 1); lastimportid = last_idnum; if (import_filename && revs.prune_data.nr) diff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh index d32ff41859..ea84e2f173 100755 --- a/t/t9350-fast-export.sh +++ b/t/t9350-fast-export.sh @@ -580,17 +580,15 @@ test_expect_success 'fast-export quotes pathnames' ' ' test_expect_success 'test bidirectionality' ' - >marks-cur && - >marks-new && git init marks-test && - git fast-export --export-marks=marks-cur --import-marks=marks-cur --branches | \ - git --git-dir=marks-test/.git fast-import --export-marks=marks-new --import-marks=marks-new && + git fast-export --export-marks=marks-cur --import-marks-if-exists=marks-cur --branches | \ + git --git-dir=marks-test/.git fast-import --export-marks=marks-new --import-marks-if-exists=marks-new && (cd marks-test && git reset --hard && echo Wohlauf > file && git commit -a -m "back in time") && - git --git-dir=marks-test/.git fast-export --export-marks=marks-new --import-marks=marks-new --branches | \ - git fast-import --export-marks=marks-cur --import-marks=marks-cur + git --git-dir=marks-test/.git fast-export --export-marks=marks-new --import-marks-if-exists=marks-new --branches | \ + git fast-import --export-marks=marks-cur --import-marks-if-exists=marks-cur ' cat > expected << EOF From patchwork Mon Sep 30 21:10:16 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elijah Newren X-Patchwork-Id: 11167613 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C5C9113B1 for ; Mon, 30 Sep 2019 21:10:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id A3BC0205C9 for ; Mon, 30 Sep 2019 21:10:32 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="fAvsNC36" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732640AbfI3VKb (ORCPT ); Mon, 30 Sep 2019 17:10:31 -0400 Received: from mail-pf1-f193.google.com ([209.85.210.193]:36352 "EHLO mail-pf1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732637AbfI3VKb (ORCPT ); Mon, 30 Sep 2019 17:10:31 -0400 Received: by mail-pf1-f193.google.com with SMTP id y22so6303529pfr.3 for ; Mon, 30 Sep 2019 14:10:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=L4iMS/AKmnHMXzVmbj6TkOuwaH/sOX0DzGJ6qcjJNKU=; b=fAvsNC36iXVp3bK+nKh6QiDG1XWQmO2N5nQChhuhTXMwne3O9PibIURG3IW7h6RjAF Wxn8ihorIRvy7lhAemdPQNM5xqOkNFHMPqf5phNvXxdlCFM7xHt94BEP7rMyv+0EDCPL Myw5vJjul/5SNKB2bf9ahq1Upq3sDDv7Uxp6Ib7UVQ4jahMWEdWdsWJa/yJGKJh6jlj8 camYTAJCVU1hFl5xw2wR43KA05OKOaX/bprXhMxYmrDRKWSe72g76IEJyuBfS4eUFm/H Z9h5hmGOkQu2Q8gbMK1Lwk3WCAgUONrCpN7GLNGV0In6jWHcVpezNAifF15oKvvERs0F nokg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=L4iMS/AKmnHMXzVmbj6TkOuwaH/sOX0DzGJ6qcjJNKU=; b=Jwm79/MRk71zpBWB9IpuCRpuRX8tITTyWep7G2VmDieICkGZ1WUjNBk8wWqrSi8Eaa B/dIvK1zlea4sPk39k5eINwwFaVU9/9sZOad2+ZhmqQVUg9KrsfJU8VPtz4YBdJLLzVB OPaRyUn7F0EalUBtRj4pFCz2SNxlecrAZQOZt/SqKkqXt5ZcxSkPeZIR//dQ+lPumcBA evqh/CtGYWb4oLMOUWmdWElaw8iHh1ETOy56oiSURcdOz6TynsrCQAmi8okbnMqtMXRl 3yvgJdlGUMC+nwwcnrPjLm/1h8cs6YGGTrRJCIibdqvY/Tep7b6q2Bo9YW6rQiGPMaE5 L4Sg== X-Gm-Message-State: APjAAAXlB9jOS2YNHwv7enEZVqGvVl9dFpLN5BUhTSbDZYWa0WdiIVqJ Ir+p/PCA15wo6VkjOeBmhd8Tm9A+QfE= X-Google-Smtp-Source: APXvYqyVJ8Z1Dx1oSW/PIddJHG+ENkccI1sKG0iRkd0hpol5INx3hRKDGK8xCRhf0+Wj85tqv/Xi6w== X-Received: by 2002:a17:90a:e017:: with SMTP id u23mr1298797pjy.55.1569877829431; Mon, 30 Sep 2019 14:10:29 -0700 (PDT) Received: from newren2-linux.yojoe.local ([8.4.231.67]) by smtp.gmail.com with ESMTPSA id w11sm13404609pfd.116.2019.09.30.14.10.28 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 30 Sep 2019 14:10:28 -0700 (PDT) From: Elijah Newren To: git@vger.kernel.org Cc: Junio C Hamano , Elijah Newren Subject: [PATCH v2 6/8] fast-export: allow user to request tags be marked with --mark-tags Date: Mon, 30 Sep 2019 14:10:16 -0700 Message-Id: <20190930211018.23633-7-newren@gmail.com> X-Mailer: git-send-email 2.23.0.264.gac739dbb79 In-Reply-To: <20190930211018.23633-1-newren@gmail.com> References: <20190925014005.17056-1-newren@gmail.com> <20190930211018.23633-1-newren@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Add a new option, --mark-tags, which will output mark identifiers with each tag object. This improves the incremental export story with --export-marks since it will allow us to record that annotated tags have been exported, and it is also needed as a step towards supporting nested tags. Signed-off-by: Elijah Newren --- Documentation/git-fast-export.txt | 17 +++++++++++++---- builtin/fast-export.c | 7 +++++++ t/t9350-fast-export.sh | 14 ++++++++++++++ 3 files changed, 34 insertions(+), 4 deletions(-) diff --git a/Documentation/git-fast-export.txt b/Documentation/git-fast-export.txt index cc940eb9ad..c522b34f7b 100644 --- a/Documentation/git-fast-export.txt +++ b/Documentation/git-fast-export.txt @@ -75,11 +75,20 @@ produced incorrect results if you gave these options. Before processing any input, load the marks specified in . The input file must exist, must be readable, and must use the same format as produced by --export-marks. + +--mark-tags:: + In addition to labelling blobs and commits with mark ids, also + label tags. This is useful in conjunction with + `--export-marks` and `--import-marks`, and is also useful (and + necessary) for exporting of nested tags. It does not hurt + other cases and would be the default, but many fast-import + frontends are not prepared to accept tags with mark + identifiers. + -Any commits that have already been marked will not be exported again. -If the backend uses a similar --import-marks file, this allows for -incremental bidirectional exporting of the repository by keeping the -marks the same across runs. +Any commits (or tags) that have already been marked will not be +exported again. If the backend uses a similar --import-marks file, +this allows for incremental bidirectional exporting of the repository +by keeping the marks the same across runs. --fake-missing-tagger:: Some old repositories have tags without a tagger. The diff --git a/builtin/fast-export.c b/builtin/fast-export.c index 575e47833b..d32e1e9327 100644 --- a/builtin/fast-export.c +++ b/builtin/fast-export.c @@ -40,6 +40,7 @@ static int no_data; static int full_tree; static int reference_excluded_commits; static int show_original_ids; +static int mark_tags; static struct string_list extra_refs = STRING_LIST_INIT_NODUP; static struct string_list tag_refs = STRING_LIST_INIT_NODUP; static struct refspec refspecs = REFSPEC_INIT_FETCH; @@ -861,6 +862,10 @@ static void handle_tag(const char *name, struct tag *tag) if (starts_with(name, "refs/tags/")) name += 10; printf("tag %s\n", name); + if (mark_tags) { + mark_next_object(&tag->object); + printf("mark :%"PRIu32"\n", last_idnum); + } if (tagged_mark) printf("from :%d\n", tagged_mark); else @@ -1165,6 +1170,8 @@ int cmd_fast_export(int argc, const char **argv, const char *prefix) &reference_excluded_commits, N_("Reference parents which are not in fast-export stream by object id")), OPT_BOOL(0, "show-original-ids", &show_original_ids, N_("Show original object ids of blobs/commits")), + OPT_BOOL(0, "mark-tags", &mark_tags, + N_("Label tags with mark ids")), OPT_END() }; diff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh index ea84e2f173..b3fca6ffba 100755 --- a/t/t9350-fast-export.sh +++ b/t/t9350-fast-export.sh @@ -66,6 +66,20 @@ test_expect_success 'fast-export ^muss^{commit} muss' ' test_cmp expected actual ' +test_expect_success 'fast-export --mark-tags ^muss^{commit} muss' ' + git fast-export --mark-tags --tag-of-filtered-object=rewrite ^muss^{commit} muss >actual && + cat >expected <<-EOF && + tag muss + mark :1 + from $(git rev-parse --verify muss^{commit}) + $(git cat-file tag muss | grep tagger) + data 9 + valentin + + EOF + test_cmp expected actual +' + test_expect_success 'fast-export master~2..master' ' git fast-export master~2..master >actual && From patchwork Mon Sep 30 21:10:17 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elijah Newren X-Patchwork-Id: 11167615 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CFAFD13B1 for ; Mon, 30 Sep 2019 21:10:33 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id AE67B205C9 for ; Mon, 30 Sep 2019 21:10:33 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Vr8eFYZ+" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732643AbfI3VKc (ORCPT ); Mon, 30 Sep 2019 17:10:32 -0400 Received: from mail-pg1-f196.google.com ([209.85.215.196]:43959 "EHLO mail-pg1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732627AbfI3VKb (ORCPT ); Mon, 30 Sep 2019 17:10:31 -0400 Received: by mail-pg1-f196.google.com with SMTP id v27so8031038pgk.10 for ; Mon, 30 Sep 2019 14:10:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=5EXF38cBCFVW4sx/PZ/aO7t14RV3nIGLVYjD0I+9JvE=; b=Vr8eFYZ+8OqjUX0YR896dNy8e2hmBnXp3fEhM+A4dr/D/vugnxaYCWfPDD8nj1Rv3J UwiruQ8po+xB/LFG14Z1tyF50p3vZrGeAL6FzaAVuFXZ1EEQ2SMj+ndivv/cNJSc8SLx HUjc9RKKLs9Es9W4l2hI9zO1u5Q/kHsNyHAcpNl8RbVfXr3l/SnRDN7mFhCP8zhlr4J3 9NCbAycLfGVN1njfqIonb4IE1YDj3KkYFrof8BQMWEzosN9df3zcxT+yfyK425jexKYA w/dEd9eXVE9+nkOsNM+rrKmNCYGYYXfa2m54Lc9/y2jbi3LRqnU0Mu5lsyDC7bW5H2I2 Ez1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=5EXF38cBCFVW4sx/PZ/aO7t14RV3nIGLVYjD0I+9JvE=; b=N/2Js7qn/kCuZt4frzA/npHJk+sTdcX9aS/FX1t3dIVhfVx+OK94Qw0XTYxCPQIDus Czj9RkeiRJvRkG3dXpkWjl0zRlaQ0fLTieAHCCps/1IttGdSRMEWM5b+3hJ48kJlTuYj 3Xvl+TNXRlypfR4byxdcpBvk95dLcGO2liw8Z0q6kqE3RY5uykw5yL65SQjseHl3ebFG 3tP6+1wTem5ipcIZ7m2yN5SJdFUe41KdWpyaaY1VGK4ouSjQfvYodQW5rKJQviM5xcj1 5v+gcABGjMJmFQQ2Ap5txjdYfpPMPzMsvQETRO4G8sBlYOwOW2duNV7cpxyBdMDjM/wd wzIg== X-Gm-Message-State: APjAAAXuFld8/8ag6+7hlV2Nc9A04rIKKaP0KodhyOTJ8XSJqF5XK4CY RA14DE/seSvsmPlF51lGT4rno9xdDOI= X-Google-Smtp-Source: APXvYqwXW6rLE7oukhWJ3WmchWidTkikCi9jhQtOcfODy28wIwuM/XAqGPniGkW7b2evJS5Iw2bbSg== X-Received: by 2002:a63:d010:: with SMTP id z16mr26145287pgf.327.1569877830327; Mon, 30 Sep 2019 14:10:30 -0700 (PDT) Received: from newren2-linux.yojoe.local ([8.4.231.67]) by smtp.gmail.com with ESMTPSA id w11sm13404609pfd.116.2019.09.30.14.10.29 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 30 Sep 2019 14:10:29 -0700 (PDT) From: Elijah Newren To: git@vger.kernel.org Cc: Junio C Hamano , Elijah Newren Subject: [PATCH v2 7/8] t9350: add tests for tags of things other than a commit Date: Mon, 30 Sep 2019 14:10:17 -0700 Message-Id: <20190930211018.23633-8-newren@gmail.com> X-Mailer: git-send-email 2.23.0.264.gac739dbb79 In-Reply-To: <20190930211018.23633-1-newren@gmail.com> References: <20190925014005.17056-1-newren@gmail.com> <20190930211018.23633-1-newren@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Multiple changes here: * add a test for a tag of a blob * add a test for a tag of a tag of a commit * add a comment to the tests for (possibly nested) tags of trees, making it clear that these tests are doing much less than you might expect Signed-off-by: Elijah Newren --- t/t9350-fast-export.sh | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh index b3fca6ffba..9ab281e4b9 100755 --- a/t/t9350-fast-export.sh +++ b/t/t9350-fast-export.sh @@ -540,10 +540,41 @@ test_expect_success 'tree_tag' ' ' # NEEDSWORK: not just check return status, but validate the output +# Note that these tests DO NOTHING other than print a warning that +# they are ommitting the one tag we asked them to export (because the +# tags resolve to a tree). They exist just to make sure we do not +# abort but instead just warn. test_expect_success 'tree_tag-obj' 'git fast-export tree_tag-obj' test_expect_success 'tag-obj_tag' 'git fast-export tag-obj_tag' test_expect_success 'tag-obj_tag-obj' 'git fast-export tag-obj_tag-obj' +test_expect_success 'handling tags of blobs' ' + git tag -a -m "Tag of a blob" blobtag $(git rev-parse master:file) && + git fast-export blobtag >actual && + cat >expect <<-EOF && + blob + mark :1 + data 9 + die Luft + + tag blobtag + from :1 + tagger $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE + data 14 + Tag of a blob + + EOF + test_cmp expect actual +' + +test_expect_failure 'handling nested tags' ' + git tag -a -m "This is a nested tag" nested muss && + git fast-export --mark-tags nested >output && + grep "^from $ZERO_OID$" output && + grep "^tag nested$" output >tag_lines && + test_line_count = 2 tag_lines +' + test_expect_success 'directory becomes symlink' ' git init dirtosymlink && git init result && From patchwork Mon Sep 30 21:10:18 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elijah Newren X-Patchwork-Id: 11167617 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B202E912 for ; Mon, 30 Sep 2019 21:10:34 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 915A9206BB for ; Mon, 30 Sep 2019 21:10:34 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="cDNWNt3p" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732645AbfI3VKd (ORCPT ); Mon, 30 Sep 2019 17:10:33 -0400 Received: from mail-pf1-f175.google.com ([209.85.210.175]:33838 "EHLO mail-pf1-f175.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732641AbfI3VKd (ORCPT ); Mon, 30 Sep 2019 17:10:33 -0400 Received: by mail-pf1-f175.google.com with SMTP id b128so6312108pfa.1 for ; Mon, 30 Sep 2019 14:10:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=pNW/5d03kX6rzb1J40jixnTNQ6JLAXVHeI36wT/1qwg=; b=cDNWNt3p6S9nu8MKCi6b/iFJA2GVAJgApBFwjUyDy1biZwWKFcyVlF0MdSoU85DSM9 fWNwAAPaOdEeIv/bEmIn/CnDYen8Dsn4HIHWzxKlJoVPFrtzJUNCv14JNoZtLVvuxaN6 5NkBWHFI2T3mHqqUXQVIT8IDK3PL0rpLDluYn38T/+4gSa+srzz39CszMNttPbfWBspB DlnAPvUenqcvgPQ69xzrQ1J2iiJxghv7LCnsmestjsh736tTEndgfE1rgnyBgQr1Arlx F/C5Bj2ixbqDX7+RShbjcDkoS7rbtNUlbXG43+ahpfdFOSg5nA/TxtxK9xg0iD2T//w5 xXHQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pNW/5d03kX6rzb1J40jixnTNQ6JLAXVHeI36wT/1qwg=; b=QIhy70TAfNSHdsC7bPZDA+lY250Cdz29fT+LL6Buq7nwpa1bVsglQ9Zixc85wKfABU yHj6GCAL64YLYPuI3xImsS4+HIisU3R8u37JO8f+tCpZkyuI9z+lc3EZ51s1+jboOhXC +0aGycBY1zPFNpXhcSFiVLYsHnLnGE1yjodjUsHItiW/JbJVWvic9ifhw6UZacCkYwOO vS4c0VUuQjO+l8pWIk41iv1hd097Fq02WealalLRO83/f45ES9MNYqDjtVHQ1QKeQptd 5+CWle7hNr/22QRio2ZIzVQ2BeDEJcZCxRzQClqiGNZNnnFmUylJ2h3axNa/CmlEECLp GyAg== X-Gm-Message-State: APjAAAVyWGFgxV+rR17GdXUytOIKJYVXFQ9iwA8hQaJduqgvmsUxn6QN OsP/T6oadGzARkajsvB2Dnk11tf+m3k= X-Google-Smtp-Source: APXvYqzMqg6HE9rB7xD2pCjUI1VRxtNO0ZlkG7qqtps1FDZoFP6ijjJ38WsLvmvCTCD1nIxHBaNomA== X-Received: by 2002:a63:cd04:: with SMTP id i4mr26263851pgg.21.1569877831639; Mon, 30 Sep 2019 14:10:31 -0700 (PDT) Received: from newren2-linux.yojoe.local ([8.4.231.67]) by smtp.gmail.com with ESMTPSA id w11sm13404609pfd.116.2019.09.30.14.10.30 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 30 Sep 2019 14:10:30 -0700 (PDT) From: Elijah Newren To: git@vger.kernel.org Cc: Junio C Hamano , Elijah Newren Subject: [PATCH v2 8/8] fast-export: handle nested tags Date: Mon, 30 Sep 2019 14:10:18 -0700 Message-Id: <20190930211018.23633-9-newren@gmail.com> X-Mailer: git-send-email 2.23.0.264.gac739dbb79 In-Reply-To: <20190930211018.23633-1-newren@gmail.com> References: <20190925014005.17056-1-newren@gmail.com> <20190930211018.23633-1-newren@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Signed-off-by: Elijah Newren --- builtin/fast-export.c | 30 ++++++++++++++++++------------ t/t9350-fast-export.sh | 2 +- 2 files changed, 19 insertions(+), 13 deletions(-) diff --git a/builtin/fast-export.c b/builtin/fast-export.c index d32e1e9327..58a74de42a 100644 --- a/builtin/fast-export.c +++ b/builtin/fast-export.c @@ -843,22 +843,28 @@ static void handle_tag(const char *name, struct tag *tag) free(buf); return; case REWRITE: - if (tagged->type != OBJ_COMMIT) { - die("tag %s tags unexported %s!", - oid_to_hex(&tag->object.oid), - type_name(tagged->type)); - } - p = rewrite_commit((struct commit *)tagged); - if (!p) { - printf("reset %s\nfrom %s\n\n", - name, oid_to_hex(&null_oid)); - free(buf); - return; + if (tagged->type == OBJ_TAG && !mark_tags) { + die(_("Error: Cannot export nested tags unless --mark-tags is specified.")); + } else if (tagged->type == OBJ_COMMIT) { + p = rewrite_commit((struct commit *)tagged); + if (!p) { + printf("reset %s\nfrom %s\n\n", + name, oid_to_hex(&null_oid)); + free(buf); + return; + } + tagged_mark = get_object_mark(&p->object); + } else { + /* tagged->type is either OBJ_BLOB or OBJ_TAG */ + tagged_mark = get_object_mark(tagged); } - tagged_mark = get_object_mark(&p->object); } } + if (tagged->type == OBJ_TAG) { + printf("reset %s\nfrom %s\n\n", + name, oid_to_hex(&null_oid)); + } if (starts_with(name, "refs/tags/")) name += 10; printf("tag %s\n", name); diff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh index 9ab281e4b9..2e4e214815 100755 --- a/t/t9350-fast-export.sh +++ b/t/t9350-fast-export.sh @@ -567,7 +567,7 @@ test_expect_success 'handling tags of blobs' ' test_cmp expect actual ' -test_expect_failure 'handling nested tags' ' +test_expect_success 'handling nested tags' ' git tag -a -m "This is a nested tag" nested muss && git fast-export --mark-tags nested >output && grep "^from $ZERO_OID$" output &&