From patchwork Thu Oct 3 20:27:02 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elijah Newren X-Patchwork-Id: 11173317 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4D2181709 for ; Thu, 3 Oct 2019 20:27:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 2C72821783 for ; Thu, 3 Oct 2019 20:27:19 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="PQT+s3PN" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2387938AbfJCU1R (ORCPT ); Thu, 3 Oct 2019 16:27:17 -0400 Received: from mail-pf1-f196.google.com ([209.85.210.196]:39252 "EHLO mail-pf1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726523AbfJCU1R (ORCPT ); Thu, 3 Oct 2019 16:27:17 -0400 Received: by mail-pf1-f196.google.com with SMTP id v4so2460955pff.6 for ; Thu, 03 Oct 2019 13:27:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=RqDXPrQpETSbRU+UZBVqA+4jkwRvX1lVJf8AsW6sgtc=; b=PQT+s3PNOa8K18qubXalFjEOAocsQz8kbys5BnKOpq5rWsbu6CYaglsOriC0d9fEvT MNaeo2Hm5hdDw6zLbcJMRa0u1LYBQpX8VMjwsBCAmzpaG0yebGEU8z4MCTEwOrKiDeN+ WjDnp9mcC5JEHz3Tu7Z9P5/Dptqgn2wkdnOesnF1kPEqCcjf0WSVGaGyjJwycByJ5IaH LIW4AfZ6MlATEsk7KMyjeSx7f2hpcDcqg2LnOROCOVGZP3cAH1yM3xaBgtDwksVIW6qi tq2ns+CujhYHIm3Lk1F3nfkW14VCHpnf1IwO/XySQdY9anAdHWYmZqv3kGPaL8lxUWP6 YzdQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=RqDXPrQpETSbRU+UZBVqA+4jkwRvX1lVJf8AsW6sgtc=; b=gx+y9lBosfFJyy16EKpqLaAbmjmMG6TxyV3E1G1Q55rtRahy2geaV+YaacF8TKQYkk d6xRFc6CN6Zs3LDXd38sKoBTGH+3JKjaZaw8TI+i4nL4UVX9jAiBXv2gJbzyN9HSvdMP NACjI4oo4KGZj3Kbwsx1gDLlyQwknCxj+hGBOL9HAVeHsTmOuhxAnKSccpBBOtOEHAzb Z4xovfLrpdQGzss8CvZcU3M++i154Wn0TUW/bQfHgpcR495xqU2OeltLo387+OqIN6xO 0amhDeGsSBTY/GUJ/ossaG7YihwhgxVd4qtesDB9cqcFjN6aCSNoEY7LXseC4uqZ4Ln8 KuBw== X-Gm-Message-State: APjAAAWV6vLAQqECI7OfyBt6wbbLwDdbM5OZ6AlfSkW0pqkZECJPBWWD 0Dm3+dLjiKX53G94hx4xToc= X-Google-Smtp-Source: APXvYqyJgmT3L2axeRvI7I2szpwc6ujfzxHSXsknhiL3JXdc52h/NUPvEZIXx/T42uE/ntM8pvDt7Q== X-Received: by 2002:a62:aa0f:: with SMTP id e15mr12632849pff.160.1570134436739; Thu, 03 Oct 2019 13:27:16 -0700 (PDT) Received: from newren2-linux.yojoe.local ([8.4.231.67]) by smtp.gmail.com with ESMTPSA id c64sm4055498pfc.19.2019.10.03.13.27.15 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 03 Oct 2019 13:27:15 -0700 (PDT) From: Elijah Newren To: Junio C Hamano Cc: git@vger.kernel.org, =?utf-8?q?Ren=C3=A9_Scharfe?= , Elijah Newren Subject: [PATCH -v3 1/8] fast-export: fix exporting a tag and nothing else Date: Thu, 3 Oct 2019 13:27:02 -0700 Message-Id: <20191003202709.26279-2-newren@gmail.com> X-Mailer: git-send-email 2.23.0.264.g3b9f7f2fc6 In-Reply-To: <20191003202709.26279-1-newren@gmail.com> References: <20190930211018.23633-1-newren@gmail.com> <20191003202709.26279-1-newren@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org fast-export allows specifying revision ranges, which can be used to export a tag without exporting the commit it tags. fast-export handled this rather poorly: it would emit a "from :0" directive. Since marks start at 1 and increase, this means it refers to an unknown commit and fast-import will choke on the input. When we are unable to look up a mark for the object being tagged, use a "from $HASH" directive instead to fix this problem. Note that this is quite similar to the behavior fast-export exhibits with commits and parents when --reference-excluded-parents is passed along with an excluded commit range. For tags of excluded commits we do not require the --reference-excluded-parents flag because we always have to tag something. By contrast, when dealing with commits, pruning a parent is always a viable option, so we need the flag to specify that parent pruning is not wanted. (It is slightly weird that --reference-excluded-parents isn't the default with a separate --prune-excluded-parents flag, but backward compatibility concerns resulted in the current defaults.) Signed-off-by: Elijah Newren --- builtin/fast-export.c | 7 ++++++- t/t9350-fast-export.sh | 13 +++++++++++++ 2 files changed, 19 insertions(+), 1 deletion(-) diff --git a/builtin/fast-export.c b/builtin/fast-export.c index f541f55d33..5822271c6b 100644 --- a/builtin/fast-export.c +++ b/builtin/fast-export.c @@ -860,7 +860,12 @@ static void handle_tag(const char *name, struct tag *tag) if (starts_with(name, "refs/tags/")) name += 10; - printf("tag %s\nfrom :%d\n", name, tagged_mark); + printf("tag %s\n", name); + if (tagged_mark) + printf("from :%d\n", tagged_mark); + else + printf("from %s\n", oid_to_hex(&tagged->oid)); + if (show_original_ids) printf("original-oid %s\n", oid_to_hex(&tag->object.oid)); printf("%.*s%sdata %d\n%.*s\n", diff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh index b4004e05c2..d32ff41859 100755 --- a/t/t9350-fast-export.sh +++ b/t/t9350-fast-export.sh @@ -53,6 +53,19 @@ test_expect_success 'fast-export | fast-import' ' ' +test_expect_success 'fast-export ^muss^{commit} muss' ' + git fast-export --tag-of-filtered-object=rewrite ^muss^{commit} muss >actual && + cat >expected <<-EOF && + tag muss + from $(git rev-parse --verify muss^{commit}) + $(git cat-file tag muss | grep tagger) + data 9 + valentin + + EOF + test_cmp expected actual +' + test_expect_success 'fast-export master~2..master' ' git fast-export master~2..master >actual && From patchwork Thu Oct 3 20:27:03 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Elijah Newren X-Patchwork-Id: 11173319 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 081FE1599 for ; Thu, 3 Oct 2019 20:27:20 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id DADBA20862 for ; Thu, 3 Oct 2019 20:27:19 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="cV7lFAgT" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388116AbfJCU1T (ORCPT ); Thu, 3 Oct 2019 16:27:19 -0400 Received: from mail-pf1-f196.google.com ([209.85.210.196]:43209 "EHLO mail-pf1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726523AbfJCU1S (ORCPT ); Thu, 3 Oct 2019 16:27:18 -0400 Received: by mail-pf1-f196.google.com with SMTP id a2so2443867pfo.10 for ; Thu, 03 Oct 2019 13:27:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=LjA68EJaDp/1KoTqrjA4J1/dYvTyr0t8uRhkTtuzPsg=; b=cV7lFAgTiBpDFjVHBYaNeuHCBhOTfD5GC1l9yVyFITbJs424611kgMKdBKku+EtKfa ULEi1DfOd/AJ5H+7NRbSYatDf+Ow8dKYbp9srlt++KzNSNr/Srm53J1NOQ/p2ADUnxt9 HEL4ptVCbkOS2EKeztdQEsWbw7DicYtDMk6y0l9KqMzOOb8ic2sYZFM6gKgC3mPKMS58 poWc3esBxsPL6FmDN8KDGb28cswt1FQHQQsywMnPdjqy9mkQaF1CXQuS37x7Rx4ehn+0 TrWQRCpm6pWeIXMEfrwOBvSqU8wWmcIyPJx1MV57p0tm4U9Euocz3Go/Kxry8BiTHKtX SFCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=LjA68EJaDp/1KoTqrjA4J1/dYvTyr0t8uRhkTtuzPsg=; b=AYwRTSGguJgEUxXizOyR66z/lcMJvUfww3cdvqoMwyNwROba3iFodjf3/tgQJfTcTE 1rk7MqoH0dP6p1lzX2jpQLovnTQ/C4lI2+4hAvbXnNSO/5q6NY7s/IH1Ej/NPsNuny29 xt0PdMNpCQw1EGoNeCtgiWurHAbOjWP1XvQIpo9Wi1jVXcdpcMx7vHs8GrJuf8IVOMpB 8onZob/tkuwuNY3oPbngrHGUEzvVC1Jt9k8AtGNfPDSPnZyw6gwunyfTsSC3VVvcVpTn PVVUjXn3jGcnIxJ2UrfelB4ZkPFAlFP64olkhUEptoTOsiEaR1Z7LKnHFJ/wpm4ZgoQe YNWw== X-Gm-Message-State: APjAAAWYmNYTKxOLi+Wb8iJZqynAOECmXLRptf8/WdzHjgQlLFogioee tKbDpvjO/g5E6ZljNqrCnSU= X-Google-Smtp-Source: APXvYqx1wD7P2BzAcxukTqjX30hdwp7nmKjUht/xI4BhHY6hjvE/OZrpqLi1dnk03WmCgji8koTbwA== X-Received: by 2002:a65:6844:: with SMTP id q4mr11976648pgt.274.1570134437719; Thu, 03 Oct 2019 13:27:17 -0700 (PDT) Received: from newren2-linux.yojoe.local ([8.4.231.67]) by smtp.gmail.com with ESMTPSA id c64sm4055498pfc.19.2019.10.03.13.27.16 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 03 Oct 2019 13:27:17 -0700 (PDT) From: Elijah Newren To: Junio C Hamano Cc: git@vger.kernel.org, =?utf-8?q?Ren=C3=A9_Scharfe?= , Elijah Newren Subject: [PATCH -v3 2/8] fast-import: fix handling of deleted tags Date: Thu, 3 Oct 2019 13:27:03 -0700 Message-Id: <20191003202709.26279-3-newren@gmail.com> X-Mailer: git-send-email 2.23.0.264.g3b9f7f2fc6 In-Reply-To: <20191003202709.26279-1-newren@gmail.com> References: <20190930211018.23633-1-newren@gmail.com> <20191003202709.26279-1-newren@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org If our input stream includes a tag which is later deleted, we were not properly deleting it. We did have a step which would delete it, but we left a tag in the tag list noting that it needed to be updated, and the updating of annotated tags occurred AFTER ref deletion. So, when we record that a tag needs to be deleted, also remove it from the list of annotated tags to update. While this has likely been something that has not happened in practice, it will come up more in order to support nested tags. For nested tags, we either need to give temporary names to the intermediate tags and then delete them, or else we need to use the final name for the intermediate tags. If we use the final name for the intermediate tags, then in order to keep the sanity check that someone doesn't try to update the same tag twice, we need to delete the ref after creating the intermediate tag. So, either way nested tags imply the need to delete temporary inner tag references. Helped-by: René Scharfe Signed-off-by: Elijah Newren --- fast-import.c | 27 +++++++++++++++++++++++++++ t/t9300-fast-import.sh | 13 +++++++++++++ 2 files changed, 40 insertions(+) diff --git a/fast-import.c b/fast-import.c index b44d6a467e..caae0819f5 100644 --- a/fast-import.c +++ b/fast-import.c @@ -2778,6 +2778,7 @@ static void parse_new_tag(const char *arg) static void parse_reset_branch(const char *arg) { struct branch *b; + const char *tag_name; b = lookup_branch(arg); if (b) { @@ -2793,6 +2794,32 @@ static void parse_reset_branch(const char *arg) b = new_branch(arg); read_next_command(); parse_from(b); + if (b->delete && skip_prefix(b->name, "refs/tags/", &tag_name)) { + /* + * Elsewhere, we call dump_branches() before dump_tags(), + * and dump_branches() will handle ref deletions first, so + * in order to make sure the deletion actually takes effect, + * we need to remove the tag from our list of tags to update. + * + * NEEDSWORK: replace list of tags with hashmap for faster + * deletion? + */ + struct tag *t, *prev = NULL; + for (t = first_tag; t; t = t->next_tag) { + if (!strcmp(t->name, tag_name)) + break; + prev = t; + } + if (t) { + if (prev) + prev->next_tag = t->next_tag; + else + first_tag = t->next_tag; + if (!t->next_tag) + last_tag = prev; + /* There is no mem_pool_free(t) function to call. */ + } + } if (command_buf.len > 0) unread_command_buf = 1; } diff --git a/t/t9300-fast-import.sh b/t/t9300-fast-import.sh index 141b7fa35e..74bc41333b 100755 --- a/t/t9300-fast-import.sh +++ b/t/t9300-fast-import.sh @@ -85,6 +85,15 @@ test_expect_success 'A: create pack from stdin' ' An annotated tag that annotates a blob. EOF + tag to-be-deleted + from :3 + data <expect <<-EOF && :2 $(git rev-parse --verify master:file2) From patchwork Thu Oct 3 20:27:04 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elijah Newren X-Patchwork-Id: 11173323 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 99C111599 for ; Thu, 3 Oct 2019 20:27:22 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 791882133F for ; Thu, 3 Oct 2019 20:27:22 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="T3V/csI8" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388533AbfJCU1W (ORCPT ); Thu, 3 Oct 2019 16:27:22 -0400 Received: from mail-pl1-f174.google.com ([209.85.214.174]:41400 "EHLO mail-pl1-f174.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726523AbfJCU1V (ORCPT ); Thu, 3 Oct 2019 16:27:21 -0400 Received: by mail-pl1-f174.google.com with SMTP id t10so2031488plr.8 for ; Thu, 03 Oct 2019 13:27:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=vsL9D9C+xGSJMKgH6ztcRLf/tZGCHwzhHGJAAg/eKwM=; b=T3V/csI8ONqMlkPpLox4B7VW1TZuqeTG+/XryTG3UWxP6iVL321a5oq3nggU2+4z9t mxm6KyBBHmrfBBBtwldT9a06FVv9ZKQRp6IOBflSbyc/0EakQ3H2n5p14zwYa8exaECt IuAHtYv/8O6TSihFTlia43IZy+C8/duRIUGFhyH2rocz0+pWQNOyO8cM9k/7BJgHAmt4 xE9kCAmfTxHQa6zHl5sBPaIZgjOfIEuKvLLyLclM7VZfojrBBKH3wbbH2t6Jj4ARex1F ipmiSNtj0MoeBwya0NnXUSG1vgbl0DvHP8wJ8GUIzifFcTivypE5FPdpb38WWXfDD938 lttw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=vsL9D9C+xGSJMKgH6ztcRLf/tZGCHwzhHGJAAg/eKwM=; b=ZpG/g4ZQWHZi2aEMGtGTEU08+F04nsiTHEjzZGflZUqU2enmCXFs56ou8SrwRxeQZj 9UkLlB6A9EQ3MTmEPywy4vkNZYOaW+szjSczxH82XBU39V/4o7D7QS49jmGHfoZhBRE5 F7sF48xuvj2X2DUadqIvsIAYvJyAqw83yQPOi+p9Z/qyF0wyZW8E3NVuuv1WKgxwH6NS XeYbzPgPUE6uAMuS39SG9ZYwZamOZGTB9WbjcqObcXiIi1x0ZzMWcG5Y/+s+NPFNxBPO vhr1azVsxQmjs2Pp2KUFC4Nnf0nbjRUL6r2HeKAaJqN5Goekv7wJOwR+qGdRDpHrnU+j pXYQ== X-Gm-Message-State: APjAAAUHIAznZdXWKvARHA6EiluQtrWA9htUHw88t1zazaYxKkObtiSI MCHhKg2dwCI5B1FwZfr/BXU= X-Google-Smtp-Source: APXvYqxU1s2z7JHSCsi359ZNplmvGU+cQlq85AQAjFQN+D25LVxtYTAW+E97Z8z6R18kF4/H+Qec4Q== X-Received: by 2002:a17:902:6b4c:: with SMTP id g12mr11748042plt.80.1570134438812; Thu, 03 Oct 2019 13:27:18 -0700 (PDT) Received: from newren2-linux.yojoe.local ([8.4.231.67]) by smtp.gmail.com with ESMTPSA id c64sm4055498pfc.19.2019.10.03.13.27.17 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 03 Oct 2019 13:27:18 -0700 (PDT) From: Elijah Newren To: Junio C Hamano Cc: git@vger.kernel.org, =?utf-8?q?Ren=C3=A9_Scharfe?= , Elijah Newren Subject: [PATCH -v3 3/8] fast-import: allow tags to be identified by mark labels Date: Thu, 3 Oct 2019 13:27:04 -0700 Message-Id: <20191003202709.26279-4-newren@gmail.com> X-Mailer: git-send-email 2.23.0.264.g3b9f7f2fc6 In-Reply-To: <20191003202709.26279-1-newren@gmail.com> References: <20190930211018.23633-1-newren@gmail.com> <20191003202709.26279-1-newren@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Mark identifiers are used in fast-export and fast-import to provide a label to refer to earlier content. Blobs are given labels because they need to be referenced in the commits where they first appear with a given filename, and commits are given labels because they can be the parents of other commits. Tags were never given labels, probably because they were viewed as unnecessary, but that presents two problems: 1. It leaves us without a way of referring to previous tags if we want to create a tag of a tag (or higher nestings). 2. It leaves us with no way of recording that a tag has already been imported when using --export-marks and --import-marks. Fix these problems by allowing an optional mark label for tags. Signed-off-by: Elijah Newren --- Documentation/git-fast-import.txt | 1 + fast-import.c | 3 ++- t/t9300-fast-import.sh | 19 +++++++++++++++++++ 3 files changed, 22 insertions(+), 1 deletion(-) diff --git a/Documentation/git-fast-import.txt b/Documentation/git-fast-import.txt index 0bb276269e..4977869465 100644 --- a/Documentation/git-fast-import.txt +++ b/Documentation/git-fast-import.txt @@ -774,6 +774,7 @@ lightweight (non-annotated) tags see the `reset` command below. .... 'tag' SP LF + mark? 'from' SP LF original-oid? 'tagger' (SP )? SP LT GT SP LF diff --git a/fast-import.c b/fast-import.c index caae0819f5..5b9e9e3b02 100644 --- a/fast-import.c +++ b/fast-import.c @@ -2713,6 +2713,7 @@ static void parse_new_tag(const char *arg) first_tag = t; last_tag = t; read_next_command(); + parse_mark(); /* from ... */ if (!skip_prefix(command_buf.buf, "from ", &from)) @@ -2769,7 +2770,7 @@ static void parse_new_tag(const char *arg) strbuf_addbuf(&new_data, &msg); free(tagger); - if (store_object(OBJ_TAG, &new_data, NULL, &t->oid, 0)) + if (store_object(OBJ_TAG, &new_data, NULL, &t->oid, next_mark)) t->pack_id = MAX_PACK_ID; else t->pack_id = pack_id; diff --git a/t/t9300-fast-import.sh b/t/t9300-fast-import.sh index 74bc41333b..3ad2b2f1ba 100755 --- a/t/t9300-fast-import.sh +++ b/t/t9300-fast-import.sh @@ -94,6 +94,23 @@ test_expect_success 'A: create pack from stdin' ' reset refs/tags/to-be-deleted from 0000000000000000000000000000000000000000 + tag nested + mark :6 + from :4 + data < X-Patchwork-Id: 11173325 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DA6AF1599 for ; Thu, 3 Oct 2019 20:27:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id B00E62133F for ; Thu, 3 Oct 2019 20:27:23 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="b+C4lyzp" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388208AbfJCU1V (ORCPT ); Thu, 3 Oct 2019 16:27:21 -0400 Received: from mail-pf1-f195.google.com ([209.85.210.195]:45002 "EHLO mail-pf1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725957AbfJCU1V (ORCPT ); Thu, 3 Oct 2019 16:27:21 -0400 Received: by mail-pf1-f195.google.com with SMTP id q21so2434754pfn.11 for ; Thu, 03 Oct 2019 13:27:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=dwpZ4CQoh5sZD9dl6KY8XiBUL7315KuNsqSRm2x251E=; b=b+C4lyzpu7A7SphWX2zJRGzAvbQXnx4vPGLuD/BOGOs1YD5igKss6LC+KbDyXubrrv /my6RbZFHvcr6gZ2wjk9SF5LpyfUFS41jLQuLmTWdkBC4lTc6qeLfCKXo+PUteoMU7fb DC5Jw3IxIfFOD6gz7LkYFVAwkBcGH5I423cDdeetBncF8FqTPk2sK6vdrcdznLh38cj1 1c51YvJKJzKuW+Q6TBXGHvlMBYMwunTCtWKLzqoswsWM2Dgr6yZnxgSUrPPVHz98c7NG gbudcYc7of8EgLIJm8LrpaNU1GBFTQZoVYqAEDIj1DZg5SQwCqQ/GTFkJf9+ZlRE6Wh4 hkVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=dwpZ4CQoh5sZD9dl6KY8XiBUL7315KuNsqSRm2x251E=; b=EGVsgj+yhVEIj8h3S8NiOSYnfYSkt52+mhxBMDFI9ffzKoDEfRKy+lYwUgf8utZAN7 /2aZBSdr0wmvWNCUJU9dptlFcEha1kPk6YRgUOBVFrDSxz35TWBSHOslKNnBBLmjKz8K Xk+UTXif1TZh7k40RyYuZ7f+tR4bW3EStsg1ZwExvlMFkzfs+pWYuTTLwUuqx3PbKlIe +pu3yy+PwD8JYo22xCqUmLExkh6mqiPVBB40XK3Ax8XjvJeV+9JTkw/35AdaaRHOtUgX A76513HNbULLANHdNYCXnSOkbFA3azWGC7WIJGiEf55hl0S7jUyCHhZA/VT1KCFjLfXX k8eA== X-Gm-Message-State: APjAAAV6rqzxVFbGOA1QrlMbfvDIclmrlDanEUczZf6wwEdET1aHdH5p 2UiszjVdQGE/vhG4aDyuHZc= X-Google-Smtp-Source: APXvYqwXSNcl4PbjFi2uPfbbZhfSRn7rd/SbHYs+hqQfTAmspWqjtI3X50wHTkxarZXxxQBn3zjzbQ== X-Received: by 2002:a62:5c82:: with SMTP id q124mr13098735pfb.177.1570134440006; Thu, 03 Oct 2019 13:27:20 -0700 (PDT) Received: from newren2-linux.yojoe.local ([8.4.231.67]) by smtp.gmail.com with ESMTPSA id c64sm4055498pfc.19.2019.10.03.13.27.18 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 03 Oct 2019 13:27:19 -0700 (PDT) From: Elijah Newren To: Junio C Hamano Cc: git@vger.kernel.org, =?utf-8?q?Ren=C3=A9_Scharfe?= , Elijah Newren Subject: [PATCH -v3 4/8] fast-import: add support for new 'alias' command Date: Thu, 3 Oct 2019 13:27:05 -0700 Message-Id: <20191003202709.26279-5-newren@gmail.com> X-Mailer: git-send-email 2.23.0.264.g3b9f7f2fc6 In-Reply-To: <20191003202709.26279-1-newren@gmail.com> References: <20190930211018.23633-1-newren@gmail.com> <20191003202709.26279-1-newren@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org fast-export and fast-import have nice --import-marks flags which allow for incremental migrations. However, if there is a mark in fast-export's file of marks without a corresponding mark in the one for fast-import, then we run the risk that fast-export tries to send new objects relative to the mark it knows which fast-import does not, causing fast-import to fail. This arises in practice when there is a filter of some sort running between the fast-export and fast-import processes which prunes some commits programmatically. Provide such a filter with the ability to alias pruned commits to their most recent non-pruned ancestor. Signed-off-by: Elijah Newren --- Documentation/git-fast-import.txt | 22 +++++++++++ fast-import.c | 62 ++++++++++++++++++++++++++----- t/t9300-fast-import.sh | 5 +++ 3 files changed, 79 insertions(+), 10 deletions(-) diff --git a/Documentation/git-fast-import.txt b/Documentation/git-fast-import.txt index 4977869465..a3f1e0c5e4 100644 --- a/Documentation/git-fast-import.txt +++ b/Documentation/git-fast-import.txt @@ -337,6 +337,13 @@ and control the current import process. More detailed discussion `commit` command. This command is optional and is not needed to perform an import. +`alias`:: + Record that a mark refers to a given object without first + creating any new object. Using --import-marks and referring + to missing marks will cause fast-import to fail, so aliases + can provide a way to set otherwise pruned commits to a valid + value (e.g. the nearest non-pruned ancestor). + `checkpoint`:: Forces fast-import to close the current packfile, generate its unique SHA-1 checksum and index, and start a new packfile. @@ -914,6 +921,21 @@ a data chunk which does not have an LF as its last byte. + The `LF` after ` LF` is optional (it used to be required). +`alias` +~~~~~~~ +Record that a mark refers to a given object without first creating any +new object. + +.... + 'alias' LF + mark + 'to' SP LF + LF? +.... + +For a detailed description of `` see above under `from`. + + `checkpoint` ~~~~~~~~~~~~ Forces fast-import to close the current packfile, start a new one, and to diff --git a/fast-import.c b/fast-import.c index 5b9e9e3b02..ac368b3e2b 100644 --- a/fast-import.c +++ b/fast-import.c @@ -2491,18 +2491,14 @@ static void parse_from_existing(struct branch *b) } } -static int parse_from(struct branch *b) +static int parse_objectish(struct branch *b, const char *objectish) { - const char *from; struct branch *s; struct object_id oid; - if (!skip_prefix(command_buf.buf, "from ", &from)) - return 0; - oidcpy(&oid, &b->branch_tree.versions[1].oid); - s = lookup_branch(from); + s = lookup_branch(objectish); if (b == s) die("Can't create a branch from itself: %s", b->name); else if (s) { @@ -2510,8 +2506,8 @@ static int parse_from(struct branch *b) oidcpy(&b->oid, &s->oid); oidcpy(&b->branch_tree.versions[0].oid, t); oidcpy(&b->branch_tree.versions[1].oid, t); - } else if (*from == ':') { - uintmax_t idnum = parse_mark_ref_eol(from); + } else if (*objectish == ':') { + uintmax_t idnum = parse_mark_ref_eol(objectish); struct object_entry *oe = find_mark(idnum); if (oe->type != OBJ_COMMIT) die("Mark :%" PRIuMAX " not a commit", idnum); @@ -2525,13 +2521,13 @@ static int parse_from(struct branch *b) } else parse_from_existing(b); } - } else if (!get_oid(from, &b->oid)) { + } else if (!get_oid(objectish, &b->oid)) { parse_from_existing(b); if (is_null_oid(&b->oid)) b->delete = 1; } else - die("Invalid ref name or SHA1 expression: %s", from); + die("Invalid ref name or SHA1 expression: %s", objectish); if (b->branch_tree.tree && !oideq(&oid, &b->branch_tree.versions[1].oid)) { release_tree_content_recursive(b->branch_tree.tree); @@ -2542,6 +2538,26 @@ static int parse_from(struct branch *b) return 1; } +static int parse_from(struct branch *b) +{ + const char *from; + + if (!skip_prefix(command_buf.buf, "from ", &from)) + return 0; + + return parse_objectish(b, from); +} + +static int parse_objectish_with_prefix(struct branch *b, const char *prefix) +{ + const char *base; + + if (!skip_prefix(command_buf.buf, prefix, &base)) + return 0; + + return parse_objectish(b, base); +} + static struct hash_list *parse_merge(unsigned int *count) { struct hash_list *list = NULL, **tail = &list, *n; @@ -3087,6 +3103,28 @@ static void parse_progress(void) skip_optional_lf(); } +static void parse_alias(void) +{ + struct object_entry *e; + struct branch b; + + skip_optional_lf(); + read_next_command(); + + /* mark ... */ + parse_mark(); + if (!next_mark) + die(_("Expected 'mark' command, got %s"), command_buf.buf); + + /* to ... */ + memset(&b, 0, sizeof(b)); + if (!parse_objectish_with_prefix(&b, "to ")) + die(_("Expected 'to' command, got %s"), command_buf.buf); + e = find_object(&b.oid); + assert(e); + insert_mark(next_mark, e); +} + static char* make_fast_import_path(const char *path) { if (!relative_marks_paths || is_absolute_path(path)) @@ -3214,6 +3252,8 @@ static int parse_one_feature(const char *feature, int from_stream) option_import_marks(arg, from_stream, 1); } else if (skip_prefix(feature, "export-marks=", &arg)) { option_export_marks(arg); + } else if (!strcmp(feature, "alias")) { + ; /* Don't die - this feature is supported */ } else if (!strcmp(feature, "get-mark")) { ; /* Don't die - this feature is supported */ } else if (!strcmp(feature, "cat-blob")) { @@ -3370,6 +3410,8 @@ int cmd_main(int argc, const char **argv) parse_checkpoint(); else if (!strcmp("done", command_buf.buf)) break; + else if (!strcmp("alias", command_buf.buf)) + parse_alias(); else if (starts_with(command_buf.buf, "progress ")) parse_progress(); else if (skip_prefix(command_buf.buf, "feature ", &v)) diff --git a/t/t9300-fast-import.sh b/t/t9300-fast-import.sh index 3ad2b2f1ba..41f2a1dad9 100755 --- a/t/t9300-fast-import.sh +++ b/t/t9300-fast-import.sh @@ -111,6 +111,10 @@ test_expect_success 'A: create pack from stdin' ' Tag of tag of our lovely commit EOF + alias + mark :8 + to :5 + INPUT_END git fast-import --export-marks=marks.out X-Patchwork-Id: 11173327 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5A10C1599 for ; Thu, 3 Oct 2019 20:27:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 39F9A2133F for ; Thu, 3 Oct 2019 20:27:26 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="XHhiy2ij" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388658AbfJCU1Z (ORCPT ); Thu, 3 Oct 2019 16:27:25 -0400 Received: from mail-pl1-f170.google.com ([209.85.214.170]:39373 "EHLO mail-pl1-f170.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725957AbfJCU1W (ORCPT ); Thu, 3 Oct 2019 16:27:22 -0400 Received: by mail-pl1-f170.google.com with SMTP id s17so2036313plp.6 for ; Thu, 03 Oct 2019 13:27:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=clCxo/v0vvXe0UYUTS3MHzveeiCzVx7QaMSNJOXMc9M=; b=XHhiy2ijECniVrXmFJ9XYxYAp+G7qWv/LJP+pAqjpV+vs0EdS6XTNlEUFKCFEEBm1w U/ot52Bkts+rWTl6L2yUcYXLJJxJ6kEeN1Ok2VY6+oXOW4Ftxxvywddb2oAQ6YnwVXIQ 1QXITjQnisu2Zdo9LjHi9OC9q98/7gkC5+OfCqdBOX57WZBsrMP8My8D3L/KbisMcYQq dloffQU+Sh7bkGEPWGGnSOFUY5w8eqdzYhp2tBTjQHQzkycDmhHBxHFduFet0V+KfJfB RyRdVZdwjq46HnmJLxWKJQb31kWyiqGlkXLQSW1O8+53Po7YAx7Z44lEI7pZSlD/QtdK Nf8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=clCxo/v0vvXe0UYUTS3MHzveeiCzVx7QaMSNJOXMc9M=; b=UjztezwWj8fChUQrxVklL1+0OalRtYfwx/HAssqSnP5V1opD08/WQtswlOfYnGLdAq uOtTn3dJwxFfyI4YjfZCmolV+7aRZepQAYeYSZ74vrdFwBNvHXUgZkl0TWC3sr739f5l 7EpB+cMHOWFs37et5fpUryZeXY0CVgRa60Kc9+10oMctA3YTVzkQO7gLZk4emKO2nGni LK/gZMO4e6huB97r1ySaEESYez0HBmTF85JN14zPQxoI/1sJimF4SfC/C7W27n0gdNwq XLmXJUxV8oRL3T0WQSOkztPlMoI5o3inAdHTMT6Y5DRyM4O3jbcl9ju/coqnN0xWKica vyGw== X-Gm-Message-State: APjAAAVExn5Qu1geZ0faPvfyROayGpFG4GJHDICOzw4NUN1dOMmG6Mvy 0a3f3QkemocMMmRxY/0+lfw= X-Google-Smtp-Source: APXvYqylMOsn2uISSMiJ5svUWuzjiHJJuag5hAmNL7NdNa+RABN9n1AUn6272hvWPHLijAOsv6eT8g== X-Received: by 2002:a17:902:ff18:: with SMTP id f24mr11439664plj.173.1570134441127; Thu, 03 Oct 2019 13:27:21 -0700 (PDT) Received: from newren2-linux.yojoe.local ([8.4.231.67]) by smtp.gmail.com with ESMTPSA id c64sm4055498pfc.19.2019.10.03.13.27.20 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 03 Oct 2019 13:27:20 -0700 (PDT) From: Elijah Newren To: Junio C Hamano Cc: git@vger.kernel.org, =?utf-8?q?Ren=C3=A9_Scharfe?= , Elijah Newren Subject: [PATCH -v3 5/8] fast-export: add support for --import-marks-if-exists Date: Thu, 3 Oct 2019 13:27:06 -0700 Message-Id: <20191003202709.26279-6-newren@gmail.com> X-Mailer: git-send-email 2.23.0.264.g3b9f7f2fc6 In-Reply-To: <20191003202709.26279-1-newren@gmail.com> References: <20190930211018.23633-1-newren@gmail.com> <20191003202709.26279-1-newren@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org fast-import has support for both an --import-marks flag and an --import-marks-if-exists flag; the latter of which will not die() if the file does not exist. fast-export only had support for an --import-marks flag; add an --import-marks-if-exists flag for consistency. Signed-off-by: Elijah Newren --- builtin/fast-export.c | 23 +++++++++++++++++++---- t/t9350-fast-export.sh | 10 ++++------ 2 files changed, 23 insertions(+), 10 deletions(-) diff --git a/builtin/fast-export.c b/builtin/fast-export.c index 5822271c6b..575e47833b 100644 --- a/builtin/fast-export.c +++ b/builtin/fast-export.c @@ -1052,11 +1052,16 @@ static void export_marks(char *file) error("Unable to write marks file %s.", file); } -static void import_marks(char *input_file) +static void import_marks(char *input_file, int check_exists) { char line[512]; - FILE *f = xfopen(input_file, "r"); + FILE *f; + struct stat sb; + + if (check_exists && stat(input_file, &sb)) + return; + f = xfopen(input_file, "r"); while (fgets(line, sizeof(line), f)) { uint32_t mark; char *line_end, *mark_end; @@ -1120,7 +1125,9 @@ int cmd_fast_export(int argc, const char **argv, const char *prefix) struct rev_info revs; struct object_array commits = OBJECT_ARRAY_INIT; struct commit *commit; - char *export_filename = NULL, *import_filename = NULL; + char *export_filename = NULL, + *import_filename = NULL, + *import_filename_if_exists = NULL; uint32_t lastimportid; struct string_list refspecs_list = STRING_LIST_INIT_NODUP; struct string_list paths_of_changed_objects = STRING_LIST_INIT_DUP; @@ -1140,6 +1147,10 @@ int cmd_fast_export(int argc, const char **argv, const char *prefix) N_("Dump marks to this file")), OPT_STRING(0, "import-marks", &import_filename, N_("file"), N_("Import marks from this file")), + OPT_STRING(0, "import-marks-if-exists", + &import_filename_if_exists, + N_("file"), + N_("Import marks from this file if it exists")), OPT_BOOL(0, "fake-missing-tagger", &fake_missing_tagger, N_("Fake a tagger when tags lack one")), OPT_BOOL(0, "full-tree", &full_tree, @@ -1187,8 +1198,12 @@ int cmd_fast_export(int argc, const char **argv, const char *prefix) if (use_done_feature) printf("feature done\n"); + if (import_filename && import_filename_if_exists) + die(_("Cannot pass both --import-marks and --import-marks-if-exists")); if (import_filename) - import_marks(import_filename); + import_marks(import_filename, 0); + else if (import_filename_if_exists) + import_marks(import_filename_if_exists, 1); lastimportid = last_idnum; if (import_filename && revs.prune_data.nr) diff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh index d32ff41859..ea84e2f173 100755 --- a/t/t9350-fast-export.sh +++ b/t/t9350-fast-export.sh @@ -580,17 +580,15 @@ test_expect_success 'fast-export quotes pathnames' ' ' test_expect_success 'test bidirectionality' ' - >marks-cur && - >marks-new && git init marks-test && - git fast-export --export-marks=marks-cur --import-marks=marks-cur --branches | \ - git --git-dir=marks-test/.git fast-import --export-marks=marks-new --import-marks=marks-new && + git fast-export --export-marks=marks-cur --import-marks-if-exists=marks-cur --branches | \ + git --git-dir=marks-test/.git fast-import --export-marks=marks-new --import-marks-if-exists=marks-new && (cd marks-test && git reset --hard && echo Wohlauf > file && git commit -a -m "back in time") && - git --git-dir=marks-test/.git fast-export --export-marks=marks-new --import-marks=marks-new --branches | \ - git fast-import --export-marks=marks-cur --import-marks=marks-cur + git --git-dir=marks-test/.git fast-export --export-marks=marks-new --import-marks-if-exists=marks-new --branches | \ + git fast-import --export-marks=marks-cur --import-marks-if-exists=marks-cur ' cat > expected << EOF From patchwork Thu Oct 3 20:27:07 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elijah Newren X-Patchwork-Id: 11173333 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E07351709 for ; Thu, 3 Oct 2019 20:27:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id BFDE42133F for ; Thu, 3 Oct 2019 20:27:28 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="lgMjcs2x" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388650AbfJCU1Z (ORCPT ); Thu, 3 Oct 2019 16:27:25 -0400 Received: from mail-pl1-f182.google.com ([209.85.214.182]:33985 "EHLO mail-pl1-f182.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726523AbfJCU1X (ORCPT ); Thu, 3 Oct 2019 16:27:23 -0400 Received: by mail-pl1-f182.google.com with SMTP id k7so2052052pll.1 for ; Thu, 03 Oct 2019 13:27:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=HPxq0xCDR0YGeHuIUddKZh5KzJnCYRYH327zQ62XhHw=; b=lgMjcs2x2071nE7lSlazzlpAfEk5A0ArIqUw/09o08P5d0iQaJXFuMt8v7iYUDQcys fO07ZZgaVE8+JIVZLrGn4q2w/LzULfrO35FJcp4qWsXES88p0pW9/xmn5I73OVKVkl4H H0F4RoXvq+5ybzg+hsiSbNFC5putebvAfmO0fO5Cd2Ekl/Zs3pltLvNHHVwn65A6tGdi N85bO/llJB2qGV7KQFURkski3JA9OFDiH9XbyjIESbUeicO08oAHEKJyUY2D4DI2noUb nZz5V/MHfDEq1Egyda9QA3dOU02N1FeXWDHFfAfWGGuo6JNDSFJdysMijc7UyBsZL91C uYhw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HPxq0xCDR0YGeHuIUddKZh5KzJnCYRYH327zQ62XhHw=; b=H3XZGLT1upKiRHUNFxaNo8+5WA2AvN3ZkZ4fUY7LChJMVIolQtLnA0QiKtoqbCyUpM EY1yUKwGJmJ12w+9JPvhRHTrqcfvqyfQtlyPBcdDF2qyPwwfBDl0SD/ublL6gBiB41Yy vownmHfvBv5LrpgOc5HoA1ZOHLSuqgNy0n1aQdcNCKwZc324idwWikvq0gEVkSv4n8y7 IrXLWMD3KFkUYkuuLZw1b3teH+A7mnA/uf62ZCA5i6oKh2MduV3e2enzl3xWxO0uVFsU i1d9pfA7C3vYdV8GG5s8P6zMW8nVMNfo579QfVIy3KFqp7bdGbP8PiuzP3jBnyzUdxkQ A9BA== X-Gm-Message-State: APjAAAUhD3r6GSM9fEYrHCBejTUuWTlRGu50CtXSR5UHm4YSnsooKBMb WaqLbK85X5i5t3tTes/Xl/M= X-Google-Smtp-Source: APXvYqzxzlPMCmF7VHH7a8ywTFlOughGEoqoN2LsXpvALGRi8Le5Qa7R1F6kneFOmLJ45QkKi9WyDA== X-Received: by 2002:a17:902:144:: with SMTP id 62mr11442740plb.283.1570134442168; Thu, 03 Oct 2019 13:27:22 -0700 (PDT) Received: from newren2-linux.yojoe.local ([8.4.231.67]) by smtp.gmail.com with ESMTPSA id c64sm4055498pfc.19.2019.10.03.13.27.21 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 03 Oct 2019 13:27:21 -0700 (PDT) From: Elijah Newren To: Junio C Hamano Cc: git@vger.kernel.org, =?utf-8?q?Ren=C3=A9_Scharfe?= , Elijah Newren Subject: [PATCH -v3 6/8] fast-export: allow user to request tags be marked with --mark-tags Date: Thu, 3 Oct 2019 13:27:07 -0700 Message-Id: <20191003202709.26279-7-newren@gmail.com> X-Mailer: git-send-email 2.23.0.264.g3b9f7f2fc6 In-Reply-To: <20191003202709.26279-1-newren@gmail.com> References: <20190930211018.23633-1-newren@gmail.com> <20191003202709.26279-1-newren@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Add a new option, --mark-tags, which will output mark identifiers with each tag object. This improves the incremental export story with --export-marks since it will allow us to record that annotated tags have been exported, and it is also needed as a step towards supporting nested tags. Signed-off-by: Elijah Newren --- Documentation/git-fast-export.txt | 17 +++++++++++++---- builtin/fast-export.c | 7 +++++++ t/t9350-fast-export.sh | 14 ++++++++++++++ 3 files changed, 34 insertions(+), 4 deletions(-) diff --git a/Documentation/git-fast-export.txt b/Documentation/git-fast-export.txt index cc940eb9ad..c522b34f7b 100644 --- a/Documentation/git-fast-export.txt +++ b/Documentation/git-fast-export.txt @@ -75,11 +75,20 @@ produced incorrect results if you gave these options. Before processing any input, load the marks specified in . The input file must exist, must be readable, and must use the same format as produced by --export-marks. + +--mark-tags:: + In addition to labelling blobs and commits with mark ids, also + label tags. This is useful in conjunction with + `--export-marks` and `--import-marks`, and is also useful (and + necessary) for exporting of nested tags. It does not hurt + other cases and would be the default, but many fast-import + frontends are not prepared to accept tags with mark + identifiers. + -Any commits that have already been marked will not be exported again. -If the backend uses a similar --import-marks file, this allows for -incremental bidirectional exporting of the repository by keeping the -marks the same across runs. +Any commits (or tags) that have already been marked will not be +exported again. If the backend uses a similar --import-marks file, +this allows for incremental bidirectional exporting of the repository +by keeping the marks the same across runs. --fake-missing-tagger:: Some old repositories have tags without a tagger. The diff --git a/builtin/fast-export.c b/builtin/fast-export.c index 575e47833b..d32e1e9327 100644 --- a/builtin/fast-export.c +++ b/builtin/fast-export.c @@ -40,6 +40,7 @@ static int no_data; static int full_tree; static int reference_excluded_commits; static int show_original_ids; +static int mark_tags; static struct string_list extra_refs = STRING_LIST_INIT_NODUP; static struct string_list tag_refs = STRING_LIST_INIT_NODUP; static struct refspec refspecs = REFSPEC_INIT_FETCH; @@ -861,6 +862,10 @@ static void handle_tag(const char *name, struct tag *tag) if (starts_with(name, "refs/tags/")) name += 10; printf("tag %s\n", name); + if (mark_tags) { + mark_next_object(&tag->object); + printf("mark :%"PRIu32"\n", last_idnum); + } if (tagged_mark) printf("from :%d\n", tagged_mark); else @@ -1165,6 +1170,8 @@ int cmd_fast_export(int argc, const char **argv, const char *prefix) &reference_excluded_commits, N_("Reference parents which are not in fast-export stream by object id")), OPT_BOOL(0, "show-original-ids", &show_original_ids, N_("Show original object ids of blobs/commits")), + OPT_BOOL(0, "mark-tags", &mark_tags, + N_("Label tags with mark ids")), OPT_END() }; diff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh index ea84e2f173..b3fca6ffba 100755 --- a/t/t9350-fast-export.sh +++ b/t/t9350-fast-export.sh @@ -66,6 +66,20 @@ test_expect_success 'fast-export ^muss^{commit} muss' ' test_cmp expected actual ' +test_expect_success 'fast-export --mark-tags ^muss^{commit} muss' ' + git fast-export --mark-tags --tag-of-filtered-object=rewrite ^muss^{commit} muss >actual && + cat >expected <<-EOF && + tag muss + mark :1 + from $(git rev-parse --verify muss^{commit}) + $(git cat-file tag muss | grep tagger) + data 9 + valentin + + EOF + test_cmp expected actual +' + test_expect_success 'fast-export master~2..master' ' git fast-export master~2..master >actual && From patchwork Thu Oct 3 20:27:08 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elijah Newren X-Patchwork-Id: 11173329 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 411461599 for ; Thu, 3 Oct 2019 20:27:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 20BA32133F for ; Thu, 3 Oct 2019 20:27:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ZGiYc/MN" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388654AbfJCU1Z (ORCPT ); Thu, 3 Oct 2019 16:27:25 -0400 Received: from mail-pg1-f172.google.com ([209.85.215.172]:44921 "EHLO mail-pg1-f172.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388597AbfJCU1Y (ORCPT ); Thu, 3 Oct 2019 16:27:24 -0400 Received: by mail-pg1-f172.google.com with SMTP id i14so2393419pgt.11 for ; Thu, 03 Oct 2019 13:27:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=8K3d17QGmELONI57a8uJFQsHlRUn6PkHEfuZqdESHHo=; b=ZGiYc/MNUfS9XAb9tuMp8iU7pAQEgTWU4F8nVIUbyOuEB+L54caYmRn9GMBJKm9QW5 Vb5Jb3VyFqpw5M2jjuCG6leQPNQk9yQufya15EznIWLEyDhJy2MoryB07JS+kb6GwXFF yzIGKIQhNngQMiBLFQ2tqj1kSJXrSrH7Mh8Fxos2XgbYDsKwT5ARU13OZE6fnYjznrcL WWpjcBwmG3HNUovaj0y8BDzLqa+iOD97PvjWBWp7UVRzHDY2H7FMv4Hpeo+n38/ehUXB lOjziHUgnny4OE7Va0AJtBqkb7OtH1nU3Npc5KWkterUPBZxzGwI7P013lD8LUEPsqdo SPGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=8K3d17QGmELONI57a8uJFQsHlRUn6PkHEfuZqdESHHo=; b=JZN2HW8PqgV+ZDx+ooX2qdHZanCaJEVw6zRcHteefsPZrwfIRWT/QMjyJNYNj3A7rz f0FGa6v5S35ss0/WT7qX2b08COaaoNhsPDtZkJSjw9UgT0//nHgVzuwsmdg9iiHXJpMV 3fLLOoV8+Bn4ePVavrnZsOs+ldCA4u+ag1ye3zvu3X0qrsZLgCdCaACT2WTzzdJzuB+x UdizC4qgOHWiHt3oI0ehPAzoB9qY84uhT0xZUkX3yombojkTA1fS+6s6ThHbFpB3oHX+ A/dYuhdXeN55SSPtv6b3gbCmdbYdvR93tXT19t14ReS/VcgU3+gBJ8fujMy1eqv/FcRH SAag== X-Gm-Message-State: APjAAAV4+72kUUsuPGCFJIyx1UekJxDAvyrIkkCO4uCAYFdzRtxQDFcs oSE+9ltlgLapzqRxFFIZwWg= X-Google-Smtp-Source: APXvYqzI5h5LB124bYnH/lWR0GXkBYz+uhmD05l4g+Bb5ZSlRf+XbEhAnvA9vX15cQyh8tWbnauO/Q== X-Received: by 2002:a17:90a:8d0c:: with SMTP id c12mr12148465pjo.112.1570134443198; Thu, 03 Oct 2019 13:27:23 -0700 (PDT) Received: from newren2-linux.yojoe.local ([8.4.231.67]) by smtp.gmail.com with ESMTPSA id c64sm4055498pfc.19.2019.10.03.13.27.22 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 03 Oct 2019 13:27:22 -0700 (PDT) From: Elijah Newren To: Junio C Hamano Cc: git@vger.kernel.org, =?utf-8?q?Ren=C3=A9_Scharfe?= , Elijah Newren Subject: [PATCH -v3 7/8] t9350: add tests for tags of things other than a commit Date: Thu, 3 Oct 2019 13:27:08 -0700 Message-Id: <20191003202709.26279-8-newren@gmail.com> X-Mailer: git-send-email 2.23.0.264.g3b9f7f2fc6 In-Reply-To: <20191003202709.26279-1-newren@gmail.com> References: <20190930211018.23633-1-newren@gmail.com> <20191003202709.26279-1-newren@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Multiple changes here: * add a test for a tag of a blob * add a test for a tag of a tag of a commit * add a comment to the tests for (possibly nested) tags of trees, making it clear that these tests are doing much less than you might expect Signed-off-by: Elijah Newren --- t/t9350-fast-export.sh | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh index b3fca6ffba..9ab281e4b9 100755 --- a/t/t9350-fast-export.sh +++ b/t/t9350-fast-export.sh @@ -540,10 +540,41 @@ test_expect_success 'tree_tag' ' ' # NEEDSWORK: not just check return status, but validate the output +# Note that these tests DO NOTHING other than print a warning that +# they are ommitting the one tag we asked them to export (because the +# tags resolve to a tree). They exist just to make sure we do not +# abort but instead just warn. test_expect_success 'tree_tag-obj' 'git fast-export tree_tag-obj' test_expect_success 'tag-obj_tag' 'git fast-export tag-obj_tag' test_expect_success 'tag-obj_tag-obj' 'git fast-export tag-obj_tag-obj' +test_expect_success 'handling tags of blobs' ' + git tag -a -m "Tag of a blob" blobtag $(git rev-parse master:file) && + git fast-export blobtag >actual && + cat >expect <<-EOF && + blob + mark :1 + data 9 + die Luft + + tag blobtag + from :1 + tagger $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE + data 14 + Tag of a blob + + EOF + test_cmp expect actual +' + +test_expect_failure 'handling nested tags' ' + git tag -a -m "This is a nested tag" nested muss && + git fast-export --mark-tags nested >output && + grep "^from $ZERO_OID$" output && + grep "^tag nested$" output >tag_lines && + test_line_count = 2 tag_lines +' + test_expect_success 'directory becomes symlink' ' git init dirtosymlink && git init result && From patchwork Thu Oct 3 20:27:09 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elijah Newren X-Patchwork-Id: 11173331 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3DCD21599 for ; Thu, 3 Oct 2019 20:27:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 1BA1820862 for ; Thu, 3 Oct 2019 20:27:28 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="TcHsGuVJ" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388708AbfJCU11 (ORCPT ); Thu, 3 Oct 2019 16:27:27 -0400 Received: from mail-pl1-f179.google.com ([209.85.214.179]:40434 "EHLO mail-pl1-f179.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725957AbfJCU10 (ORCPT ); Thu, 3 Oct 2019 16:27:26 -0400 Received: by mail-pl1-f179.google.com with SMTP id d22so2033136pll.7 for ; Thu, 03 Oct 2019 13:27:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=cFEKpUTHMK4jEjdfb9vYUj3CqxWL60eZhQIW6bagyNY=; b=TcHsGuVJ760p1qzMHLtdON0dgZ9sCofeD4+oppV9hPn2/eZ6inqSOoIcALK4kKdY3i po04m7ukOMvzYaAy5SC50Brt5CKnOvlUp6+cLRSAuXtd/VKclssPSSZCCa4XoJGPKp++ 1XZK8MQis49X51N/vbySQp6A5JqgOiV0tKO+rFsudHNZTNx1LS/jTtnrKBhGCcmg9QmQ lSxf8lfwsO558xR+fXpK1zDF/aE7Kcz0pu+K7I27rFIHVXwXZnhbWv8rI18dkGGcMmBZ DmWVxfF3JnKbHj0H3/+pfUxM/lB7r8+BcaLf3kWh1FlVnN1e3BpiYiWL7ataoa0K8bD1 ScGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=cFEKpUTHMK4jEjdfb9vYUj3CqxWL60eZhQIW6bagyNY=; b=tORlHiJusIlZumOC7p3UwjAA6Y+jNyc486048xsnJ7ilocke/8YQuF94M3ys+lgxBj EcvYq/IXOtgGCpVT0Q57tNha5ADNrCTAFERuhe7isxfJscA1wocICMlyJ1ADxLFnljQK 7+/1JY9ulJ4IfQspDR61WoL5tKr9D7II/bykBqcYDTqXbcrtc4ucmnZ3uT8TpsNW9kUv MM5t/Gnu8R4iPcFStXq0hcGiv2qeMcRHyqS1YMHXnuo0reOHLYfUhjD8vmAUgL0wZFnF E4pvfP/DfY/IdwLdtY6vSbgrB3YJ0ir4lj33m+o4Kyej/k6DNSCAQpmym2pe7LtLhthB fzyg== X-Gm-Message-State: APjAAAWxwoZgQDjVDy7L+A/ovfknEOEOIRrMHtdbniUWvVnxTHKgkrNn t34eYEbgh8+r+BUsLCMsD2I= X-Google-Smtp-Source: APXvYqz/S0T/i3Qt1m6a1920bMRuXzq1SCQk9RXdcbVLZUinlAsg857IGb26BovaSGmH/69ArUUykQ== X-Received: by 2002:a17:902:9a92:: with SMTP id w18mr11332029plp.201.1570134444284; Thu, 03 Oct 2019 13:27:24 -0700 (PDT) Received: from newren2-linux.yojoe.local ([8.4.231.67]) by smtp.gmail.com with ESMTPSA id c64sm4055498pfc.19.2019.10.03.13.27.23 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 03 Oct 2019 13:27:23 -0700 (PDT) From: Elijah Newren To: Junio C Hamano Cc: git@vger.kernel.org, =?utf-8?q?Ren=C3=A9_Scharfe?= , Elijah Newren Subject: [PATCH -v3 8/8] fast-export: handle nested tags Date: Thu, 3 Oct 2019 13:27:09 -0700 Message-Id: <20191003202709.26279-9-newren@gmail.com> X-Mailer: git-send-email 2.23.0.264.g3b9f7f2fc6 In-Reply-To: <20191003202709.26279-1-newren@gmail.com> References: <20190930211018.23633-1-newren@gmail.com> <20191003202709.26279-1-newren@gmail.com> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Signed-off-by: Elijah Newren --- builtin/fast-export.c | 30 ++++++++++++++++++------------ t/t9350-fast-export.sh | 2 +- 2 files changed, 19 insertions(+), 13 deletions(-) diff --git a/builtin/fast-export.c b/builtin/fast-export.c index d32e1e9327..58a74de42a 100644 --- a/builtin/fast-export.c +++ b/builtin/fast-export.c @@ -843,22 +843,28 @@ static void handle_tag(const char *name, struct tag *tag) free(buf); return; case REWRITE: - if (tagged->type != OBJ_COMMIT) { - die("tag %s tags unexported %s!", - oid_to_hex(&tag->object.oid), - type_name(tagged->type)); - } - p = rewrite_commit((struct commit *)tagged); - if (!p) { - printf("reset %s\nfrom %s\n\n", - name, oid_to_hex(&null_oid)); - free(buf); - return; + if (tagged->type == OBJ_TAG && !mark_tags) { + die(_("Error: Cannot export nested tags unless --mark-tags is specified.")); + } else if (tagged->type == OBJ_COMMIT) { + p = rewrite_commit((struct commit *)tagged); + if (!p) { + printf("reset %s\nfrom %s\n\n", + name, oid_to_hex(&null_oid)); + free(buf); + return; + } + tagged_mark = get_object_mark(&p->object); + } else { + /* tagged->type is either OBJ_BLOB or OBJ_TAG */ + tagged_mark = get_object_mark(tagged); } - tagged_mark = get_object_mark(&p->object); } } + if (tagged->type == OBJ_TAG) { + printf("reset %s\nfrom %s\n\n", + name, oid_to_hex(&null_oid)); + } if (starts_with(name, "refs/tags/")) name += 10; printf("tag %s\n", name); diff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh index 9ab281e4b9..2e4e214815 100755 --- a/t/t9350-fast-export.sh +++ b/t/t9350-fast-export.sh @@ -567,7 +567,7 @@ test_expect_success 'handling tags of blobs' ' test_cmp expect actual ' -test_expect_failure 'handling nested tags' ' +test_expect_success 'handling nested tags' ' git tag -a -m "This is a nested tag" nested muss && git fast-export --mark-tags nested >output && grep "^from $ZERO_OID$" output &&