From patchwork Tue Mar 26 17:53:25 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Tan X-Patchwork-Id: 10871793 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5BF4B1669 for ; Tue, 26 Mar 2019 17:53:35 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4AC0728CF6 for ; Tue, 26 Mar 2019 17:53:35 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3EDF228D48; Tue, 26 Mar 2019 17:53:35 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DD4AB28CF6 for ; Tue, 26 Mar 2019 17:53:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732073AbfCZRxd (ORCPT ); Tue, 26 Mar 2019 13:53:33 -0400 Received: from mail-ot1-f74.google.com ([209.85.210.74]:54854 "EHLO mail-ot1-f74.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731832AbfCZRxd (ORCPT ); Tue, 26 Mar 2019 13:53:33 -0400 Received: by mail-ot1-f74.google.com with SMTP id b10so8778770oti.21 for ; Tue, 26 Mar 2019 10:53:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=+bkIvXoNoOurawCqbJiHmlSqlAH1NhigNDZ29M/T0Xs=; b=XFnQRTGyjzrZhJRDvKcl6dFKKWJ7AJZTIEw4lBrsONqfulJCentpHFnX5A8RRpncJ3 G3pOorowmnq8naYryOZgisJTqCQP1raVbfnIKoGzAHOcFYQ5DMa6oOsKeIJ7VwkvuYhW /dQeCgIaVnn0Io4Luw4UWOJTioxds0KUMyp0vtwoPFta9pXooEVRPbgTwI5kSTLpzMw2 4t2+Ih91WjTDt8IPsLo47dtYNxRn/PYjPwnSm/J5BzbAwpUOE9c1SlMYrmATOFf31VoG Vr69brRgBki70KHCfbwo5z1r6XJOoMzRGfHP15uGcyFgW+hOVWLlxUUJecdZcshf32u3 MpkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=+bkIvXoNoOurawCqbJiHmlSqlAH1NhigNDZ29M/T0Xs=; b=AYCVlDGi2xYmYy1a14INjar/jrHORZlRksJ9UQ6uuvc77LDjDx8g8Ayg0ZuM7XDA8u mbPP8JA8PLkG3UZgNgZSc/iKgqjKDe5kdt+TrKtmI+/hie6LMvxpeHc1YqB4zqDJ1BK9 y+IQP2LR8ankRsiV1Q+bSRMlFLkw2C59uLWFQ9CpDIslSXGMy01WzXrTrIf71vWcSjOd FynA4isujqoMgj6SaUyoTHiPwolrq/21LL48Yooeh9RHNAWSnGHkDv6b0C5KrDh6Hqa+ Ii3Udc6lOmuCmG2I4O1aaaO8dzSqwK56ayZLaueEZHCpU/hYZqdA74yU2WLK/uPI7wsg aVFw== X-Gm-Message-State: APjAAAVCZ+lbZavlxV7V6dtp5cyuJVrtJodMjH0u/yziJcClbf76QdLV XRouwfV1Y6kPh+G+SLdMg1qUADIHNpFK/GDxRcfQv7ThMUN5MXAUB/srwKORtZyTG01rCaXlEH2 kVwPpWan39nA4GK6VPSjrk5yIIvDCAlYuPkQw1PDY+VEYAi9TSDa5/IuVAShz23ryhK0NkRhGAg vz X-Google-Smtp-Source: APXvYqxKrMEFPJYGwAUtqrVMSi0uqJxeyj1uF1lVh98K3qr8HoYjuLOI0Jw6S4oeSlixTWvVhgIPxw/pbwjjQ3fFOqdH X-Received: by 2002:a9d:4044:: with SMTP id o4mr4504022oti.10.1553622812744; Tue, 26 Mar 2019 10:53:32 -0700 (PDT) Date: Tue, 26 Mar 2019 10:53:25 -0700 In-Reply-To: Message-Id: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.21.0.155.ge902e9bcae.dirty Subject: [PATCH v2 1/2] fetch-pack: call prepare_shallow_info only if v0 From: Jonathan Tan To: git@vger.kernel.org Cc: Jonathan Tan , peff@peff.net, pclouds@gmail.com Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP In fetch_pack(), be clearer that there is no shallow information before the fetch when v2 is used - memset the struct shallow_info to 0 instead of calling prepare_shallow_info(). This patch is in preparation for a future patch in which a v2 fetch might call prepare_shallow_info() after shallow info has been retrieved during the fetch, so I needed to ensure that prepare_shallow_info() is not called before the fetch. Signed-off-by: Jonathan Tan --- commit.h | 4 ++++ fetch-pack.c | 10 +++++++--- 2 files changed, 11 insertions(+), 3 deletions(-) diff --git a/commit.h b/commit.h index 42728c2906..a3f2b2eddb 100644 --- a/commit.h +++ b/commit.h @@ -257,6 +257,10 @@ extern void setup_alternate_shallow(struct lock_file *shallow_lock, extern const char *setup_temporary_shallow(const struct oid_array *extra); extern void advertise_shallow_grafts(int); +/* + * Initialize with prepare_shallow_info() or zero-initialize (equivalent to + * prepare_shallow_info with a NULL oid_array). + */ struct shallow_info { struct oid_array *shallow; int *ours, nr_ours; diff --git a/fetch-pack.c b/fetch-pack.c index e69993b2eb..a0eb268dfc 100644 --- a/fetch-pack.c +++ b/fetch-pack.c @@ -1648,13 +1648,17 @@ struct ref *fetch_pack(struct fetch_pack_args *args, packet_flush(fd[1]); die(_("no matching remote head")); } - prepare_shallow_info(&si, shallow); - if (version == protocol_v2) + if (version == protocol_v2) { + if (shallow && shallow->nr) + BUG("Protocol V2 does not provide shallows at this point in the fetch"); + memset(&si, 0, sizeof(si)); ref_cpy = do_fetch_pack_v2(args, fd, ref, sought, nr_sought, pack_lockfile); - else + } else { + prepare_shallow_info(&si, shallow); ref_cpy = do_fetch_pack(args, fd, ref, sought, nr_sought, &si, pack_lockfile); + } reprepare_packed_git(the_repository); if (!args->cloning && args->deepen) { From patchwork Tue Mar 26 17:53:26 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Tan X-Patchwork-Id: 10871795 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BE395922 for ; Tue, 26 Mar 2019 17:53:37 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id ACA1B28CF6 for ; Tue, 26 Mar 2019 17:53:37 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A0B1F28D48; Tue, 26 Mar 2019 17:53:37 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0881F28CF6 for ; Tue, 26 Mar 2019 17:53:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732084AbfCZRxg (ORCPT ); Tue, 26 Mar 2019 13:53:36 -0400 Received: from mail-pl1-f202.google.com ([209.85.214.202]:50815 "EHLO mail-pl1-f202.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727492AbfCZRxf (ORCPT ); Tue, 26 Mar 2019 13:53:35 -0400 Received: by mail-pl1-f202.google.com with SMTP id b10so2582575plb.17 for ; Tue, 26 Mar 2019 10:53:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=KtrLp1nt1otiM2BjvOWUfQWRg943+4IuhVcK+NueVSM=; b=vz2AetqFQ5EQhWBm8QaPXQszg546nbinU6HTq5KiILkFTqoIbv/LF13EYpHGVMn/VG GoWFcc3pIHvs9tXSmoHPGlAdmKTiRcxj7TzL7PpRk83MXEQcvIpeLlBM16V77TD9voan g0YdIcxil20jTPC1rvgll8lB67RbO+HdoI1rFDk+Rl4KBr7oA3PVirMD+c8iyFY3VgFl Y88ZpsGXrfaBeXQAt1xdY2zGcCCzUXPBOoBlmbWDnHYBaHLCQmQClXDbLLQYwX6nVNP1 EtkzNyTud5Zo3YZ01w9cDYbMde6+tO2PLTsg1KLkYF++hhOxmx2h75s1e9IS8YsYhEyy p9OA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=KtrLp1nt1otiM2BjvOWUfQWRg943+4IuhVcK+NueVSM=; b=IDXh2oO7oIcZ7Vw/mcoVal6DwaCDyOPeVwGExdL18lOVkMQrxt/m2eq9iPc8dccjCc vuGAZAJuW7M/uzcmruNhBJrNC+1YHnKA6XSVDrjeyGgpEMIw/dQL8HdVqmhrir8zqK9S V8YHyA1wm1IRiKIftrufqNhtptCE0jkqdVmhcKDBC/V2lqh0UMPuFuh68syR0cUfYGgl h49VMuXNUIzzeNZuUSrDkkvo71HM/glhTz8hW1CtsrkyAFgd/uwU9GNPIHpgs7lKrPrt DsPy3n0+1ZFjw56USEoyzKGJk5nyk7e1FEm0PBYCQtwh043VQb8XV2+bkpHdUKpbPyvn CN+A== X-Gm-Message-State: APjAAAWcvvJObPhL0MNNfEfDz0TBXcAAnCO+X7aOUf+mZvlHPP3MosjS 15f8zaxpmV1b2Xi0XqOPzE65CwkKqJ77bc/4L67CoLU7CbxpYRX2mpKUET3fcdZpewAu7tKTF9B bKwCTc0TRQUKA8XCGJ4p26ldmVp/auWjEAxvT1zlgaB7qGf+75gXS0WUJJHcQTRW5lQt59wUopG 38 X-Google-Smtp-Source: APXvYqzYbctq2n/o0ytgjHrS/6Hwx03djAba/hXLhZQxETU/P/mo9ROoINWiweZUBC0dkg3YcdHJPfyfkjpibsN4mjUA X-Received: by 2002:a65:6548:: with SMTP id a8mr1672371pgw.442.1553622815029; Tue, 26 Mar 2019 10:53:35 -0700 (PDT) Date: Tue, 26 Mar 2019 10:53:26 -0700 In-Reply-To: Message-Id: <943b1cbc61dbcc498a9e34d7d39dc8617cb99195.1553622679.git.jonathantanmy@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.21.0.155.ge902e9bcae.dirty Subject: [PATCH v2 2/2] fetch-pack: respect --no-update-shallow in v2 From: Jonathan Tan To: git@vger.kernel.org Cc: Jonathan Tan , peff@peff.net, pclouds@gmail.com Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP In protocol v0, when sending "shallow" lines, the server distinguishes between lines caused by the remote repo being shallow and lines caused by client-specified depth settings. Unless "--update-shallow" is specified, there is a difference in behavior: refs that reach the former "shallow" lines, but not the latter, are rejected. But in v2, the server does not, and the client treats all "shallow" lines like lines caused by client-specified depth settings. Full restoration of v0 functionality is not possible without protocol change, but we can implement a heuristic: if we specify any depth setting, treat all "shallow" lines like lines caused by client-specified depth settings (that is, unaffected by "--no-update-shallow"), but otherwise, treat them like lines caused by the remote repo being shallow (that is, affected by "--no-update-shallow"). This restores most of v0 behavior, except in the case where a client fetches from a shallow repository with depth settings. This patch causes a test that previously failed with GIT_TEST_PROTOCOL_VERSION=2 to pass. Signed-off-by: Jonathan Tan --- fetch-pack.c | 43 +++++++++++++++++++++++++++++++++++-------- 1 file changed, 35 insertions(+), 8 deletions(-) diff --git a/fetch-pack.c b/fetch-pack.c index a0eb268dfc..4831eb6a25 100644 --- a/fetch-pack.c +++ b/fetch-pack.c @@ -1253,9 +1253,11 @@ static int process_acks(struct fetch_negotiator *negotiator, } static void receive_shallow_info(struct fetch_pack_args *args, - struct packet_reader *reader) + struct packet_reader *reader, + struct oid_array *shallows, + struct shallow_info *si) { - int line_received = 0; + int unshallow_received = 0; process_section_header(reader, "shallow-info", 0); while (packet_reader_read(reader) == PACKET_READ_NORMAL) { @@ -1265,8 +1267,7 @@ static void receive_shallow_info(struct fetch_pack_args *args, if (skip_prefix(reader->line, "shallow ", &arg)) { if (get_oid_hex(arg, &oid)) die(_("invalid shallow line: %s"), reader->line); - register_shallow(the_repository, &oid); - line_received = 1; + oid_array_append(shallows, &oid); continue; } if (skip_prefix(reader->line, "unshallow ", &arg)) { @@ -1279,7 +1280,7 @@ static void receive_shallow_info(struct fetch_pack_args *args, die(_("error in object: %s"), reader->line); if (unregister_shallow(&oid)) die(_("no shallow found: %s"), reader->line); - line_received = 1; + unshallow_received = 1; continue; } die(_("expected shallow/unshallow, got %s"), reader->line); @@ -1289,10 +1290,31 @@ static void receive_shallow_info(struct fetch_pack_args *args, reader->status != PACKET_READ_DELIM) die(_("error processing shallow info: %d"), reader->status); - if (line_received) { + if (args->deepen || unshallow_received) { + /* + * Treat these as shallow lines caused by our depth settings. + * In v0, these lines cannot cause refs to be rejected; do the + * same. + */ + int i; + + for (i = 0; i < shallows->nr; i++) + register_shallow(the_repository, &shallows->oid[i]); setup_alternate_shallow(&shallow_lock, &alternate_shallow_file, NULL); args->deepen = 1; + } else if (shallows->nr) { + /* + * Treat these as shallow lines caused by the remote being + * shallow. In v0, remote refs that reach these objects are + * rejected (unless --update-shallow is set); do the same. + */ + prepare_shallow_info(si, shallows); + if (si->nr_ours || si->nr_theirs) + alternate_shallow_file = + setup_temporary_shallow(si->shallow); + else + alternate_shallow_file = NULL; } else { alternate_shallow_file = NULL; } @@ -1337,6 +1359,8 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args, int fd[2], const struct ref *orig_ref, struct ref **sought, int nr_sought, + struct oid_array *shallows, + struct shallow_info *si, char **pack_lockfile) { struct ref *ref = copy_ref_list(orig_ref); @@ -1411,7 +1435,7 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args, case FETCH_GET_PACK: /* Check for shallow-info section */ if (process_section_header(&reader, "shallow-info", 1)) - receive_shallow_info(args, &reader); + receive_shallow_info(args, &reader, shallows, si); if (process_section_header(&reader, "wanted-refs", 1)) receive_wanted_refs(&reader, sought, nr_sought); @@ -1625,6 +1649,7 @@ struct ref *fetch_pack(struct fetch_pack_args *args, { struct ref *ref_cpy; struct shallow_info si; + struct oid_array shallows_scratch = OID_ARRAY_INIT; fetch_pack_setup(); if (nr_sought) @@ -1649,10 +1674,11 @@ struct ref *fetch_pack(struct fetch_pack_args *args, die(_("no matching remote head")); } if (version == protocol_v2) { - if (shallow && shallow->nr) + if (shallow->nr) BUG("Protocol V2 does not provide shallows at this point in the fetch"); memset(&si, 0, sizeof(si)); ref_cpy = do_fetch_pack_v2(args, fd, ref, sought, nr_sought, + &shallows_scratch, &si, pack_lockfile); } else { prepare_shallow_info(&si, shallow); @@ -1680,6 +1706,7 @@ struct ref *fetch_pack(struct fetch_pack_args *args, update_shallow(args, sought, nr_sought, &si); cleanup: clear_shallow_info(&si); + oid_array_clear(&shallows_scratch); return ref_cpy; }