From patchwork Tue Nov 3 00:26:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Tan X-Patchwork-Id: 11875621 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-20.4 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7E520C00A89 for ; Tue, 3 Nov 2020 00:27:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 384F922268 for ; Tue, 3 Nov 2020 00:27:12 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="cZ1Dgy32" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726877AbgKCA1L (ORCPT ); Mon, 2 Nov 2020 19:27:11 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57862 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725940AbgKCA1L (ORCPT ); Mon, 2 Nov 2020 19:27:11 -0500 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E1B18C0617A6 for ; Mon, 2 Nov 2020 16:27:10 -0800 (PST) Received: by mail-yb1-xb4a.google.com with SMTP id j19so15991943ybg.6 for ; Mon, 02 Nov 2020 16:27:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=JvrsL1CcVJLTqs272iqMdhgxjX9sVSShYbAQ9uxiXgc=; b=cZ1Dgy32E3Mon5rK5Uhg9axw0O6EaILEGtHZga39wM0l1MXPpzvYTMKEdExGxJZs44 v4sQzIp1aH6gTBJ71gGSef9vjLCRkZ3+GSwKduAl7418RT6TlxJe7scqjBvgSdMwH7ry kcTTbN6q7s8pbdRWDQcOqunIpVmYxuTt4RCghDDG5A+aOOv3l7ElDGwIXPpZpjGrjHGp NZOstWOSJE9f3MBHIBxT6LAtRwdgMvji1Urh1wDapsaImBTNK0wNZLnF8MTUPy9XFoB0 X6voJThuRu0XmfRSyL8OmgJzPu1t3rfhNgysnRlzJDu3q6h/opeGoLRFahjOb241SYcL liIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=JvrsL1CcVJLTqs272iqMdhgxjX9sVSShYbAQ9uxiXgc=; b=MqYm2ndc1UZUf+iiiNaeAKcvUh9oe/YoMm1TEmzKBdwOndyVYvyDMWWW6PI1mZlclI rJ0PtjDE2upN1mmV+Tl5wq7sLEEUGrPNzL3sQIEFGQwQtikrAZBsgXEDcMzeblJ5gm1J 1jWemJkzK8DWaqGLGIn+rdxxrdjUVY0bcqexEz+XzVPoI+VoGwFm2BGkEOa8pjjwxWg/ q3D0mQUc58stD71jlee13WIgCFGbB6EM7BfdfqlpWfP5n89INurGYmLAeVkffaBD5/hj NiElLKnWOhnSYii+0GWBn7OjSxo/KF6e4QF2wf42OzIwA/fRphGlStNOvAQmMb39s+7J h3uQ== X-Gm-Message-State: AOAM532uwGbu0pjH7PETKL/o/Lbvlhh6+MyZSLCLE/Ew8gD4NMlXTHEI jxT3vypc9AHnno3KDeSArz4FClx7BGkeKNMmMW1qaQVKqilyyZMsDPVsBTfCM3Vbe931YjyiY7K fD21DYVqX96z5CP0jUqQ8hYlDRpvqjmBcn/fiM69S4d7dIIxH9ujX5QfTH1hnz8cUiBNVxuT0hc E2 X-Google-Smtp-Source: ABdhPJzTQGlM1vtcHLkNke/PFlicGR7Bt5KUzG+Qr8xqdzCRs8YAKKA8ilDVfbq4Y9mhAK6hRBebmvQhuSQ2PPQ2911C Sender: "jonathantanmy via sendgmr" X-Received: from twelve4.c.googlers.com ([fda3:e722:ac3:10:24:72f4:c0a8:437a]) (user=jonathantanmy job=sendgmr) by 2002:a25:dc3:: with SMTP id 186mr25946535ybn.207.1604363230136; Mon, 02 Nov 2020 16:27:10 -0800 (PST) Date: Mon, 2 Nov 2020 16:26:10 -0800 In-Reply-To: Message-Id: <7893cfba05a663370f353cdf60aa1df24b386c04.1604362701.git.jonathantanmy@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.1.341.ge80a0c044ae-goog Subject: [PATCH 1/4] connect: refactor building of Extra Parameters From: Jonathan Tan To: git@vger.kernel.org Cc: Jonathan Tan Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Build the Extra Parameters string ("version=%d") in only one place (git_connect()) instead of two (git_connect() and push_ssh_options()). This is in preparation for a subsequent patch, which will change how the Extra Parameters are built. Signed-off-by: Jonathan Tan --- connect.c | 31 ++++++++++++++++++------------- 1 file changed, 18 insertions(+), 13 deletions(-) diff --git a/connect.c b/connect.c index 8b8f56cf6d..5221f1764c 100644 --- a/connect.c +++ b/connect.c @@ -1201,14 +1201,13 @@ static struct child_process *git_connect_git(int fd[2], char *hostandport, */ static void push_ssh_options(struct strvec *args, struct strvec *env, enum ssh_variant variant, const char *port, - enum protocol_version version, int flags) + const char *extra_parameters, int flags) { - if (variant == VARIANT_SSH && - version > 0) { + if (variant == VARIANT_SSH && extra_parameters) { strvec_push(args, "-o"); strvec_push(args, "SendEnv=" GIT_PROTOCOL_ENVIRONMENT); - strvec_pushf(env, GIT_PROTOCOL_ENVIRONMENT "=version=%d", - version); + strvec_pushf(env, GIT_PROTOCOL_ENVIRONMENT "=%s", + extra_parameters); } if (flags & CONNECT_IPV4) { @@ -1261,7 +1260,7 @@ static void push_ssh_options(struct strvec *args, struct strvec *env, /* Prepare a child_process for use by Git's SSH-tunneled transport. */ static void fill_ssh_args(struct child_process *conn, const char *ssh_host, - const char *port, enum protocol_version version, + const char *port, const char *extra_parameters, int flags) { const char *ssh; @@ -1296,14 +1295,14 @@ static void fill_ssh_args(struct child_process *conn, const char *ssh_host, strvec_push(&detect.args, ssh); strvec_push(&detect.args, "-G"); push_ssh_options(&detect.args, &detect.env_array, - VARIANT_SSH, port, version, flags); + VARIANT_SSH, port, extra_parameters, flags); strvec_push(&detect.args, ssh_host); variant = run_command(&detect) ? VARIANT_SIMPLE : VARIANT_SSH; } strvec_push(&conn->args, ssh); - push_ssh_options(&conn->args, &conn->env_array, variant, port, version, flags); + push_ssh_options(&conn->args, &conn->env_array, variant, port, extra_parameters, flags); strvec_push(&conn->args, ssh_host); } @@ -1352,6 +1351,7 @@ struct child_process *git_connect(int fd[2], const char *url, } else { struct strbuf cmd = STRBUF_INIT; const char *const *var; + struct strbuf extra_parameters = STRBUF_INIT; conn = xmalloc(sizeof(*conn)); child_process_init(conn); @@ -1367,6 +1367,9 @@ struct child_process *git_connect(int fd[2], const char *url, for (var = local_repo_env; *var; var++) strvec_push(&conn->env_array, *var); + if (version > 0) + strbuf_addf(&extra_parameters, "version=%d", version); + conn->use_shell = 1; conn->in = conn->out = -1; if (protocol == PROTO_SSH) { @@ -1392,15 +1395,16 @@ struct child_process *git_connect(int fd[2], const char *url, return NULL; } conn->trace2_child_class = "transport/ssh"; - fill_ssh_args(conn, ssh_host, port, version, flags); + fill_ssh_args(conn, ssh_host, port, + extra_parameters.len ? extra_parameters.buf : NULL, + flags); } else { transport_check_allowed("file"); conn->trace2_child_class = "transport/file"; - if (version > 0) { + if (extra_parameters.len) strvec_pushf(&conn->env_array, - GIT_PROTOCOL_ENVIRONMENT "=version=%d", - version); - } + GIT_PROTOCOL_ENVIRONMENT "=%s", + extra_parameters.buf); } strvec_push(&conn->args, cmd.buf); @@ -1409,6 +1413,7 @@ struct child_process *git_connect(int fd[2], const char *url, fd[0] = conn->out; /* read from child's stdout */ fd[1] = conn->in; /* write to child's stdin */ + strbuf_release(&extra_parameters); strbuf_release(&cmd); } free(hostandport); From patchwork Tue Nov 3 00:26:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Tan X-Patchwork-Id: 11875617 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-20.4 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3DB29C388F2 for ; Tue, 3 Nov 2020 00:27:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D840722268 for ; Tue, 3 Nov 2020 00:27:13 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="eqOAI+iR" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727063AbgKCA1N (ORCPT ); Mon, 2 Nov 2020 19:27:13 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57870 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725940AbgKCA1M (ORCPT ); Mon, 2 Nov 2020 19:27:12 -0500 Received: from mail-pf1-x449.google.com (mail-pf1-x449.google.com [IPv6:2607:f8b0:4864:20::449]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 69FB7C0617A6 for ; Mon, 2 Nov 2020 16:27:12 -0800 (PST) Received: by mail-pf1-x449.google.com with SMTP id m5so8476448pfk.5 for ; Mon, 02 Nov 2020 16:27:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=E3c2Of8bOSTUT+E5j9BMTTuG1AnorX3mUa8Idm4u03U=; b=eqOAI+iRRIqHeP+zQcezwHB76y7SXtOmXzqn8KRjR1IbvZK8LS3A0yc4zrvS2NUiSh 4uaOqfWLdBVTZ14kr4urSDWW/bE4UFzjgLj0jL3lknb/dSGE/1RAESgyttWyxptVW/ZA JZOvo8hOUxYCQSBMSlQhzPQtsSTW4YeiMY/kLSD7lZBTLNuxFmJqjJnCVS6DBXVVcZ2M LC7ZMUhHx/GrAcV1NabsI1Nzmc0jxDhQhPbAGxIWsDxs80QNVIqvAWPHntMxzsEw1e+F jl4xtB5r8BIgGEiC17Scjb00qPvA99V5182a0DBZsfHnPGDrpg3KnrE7oCQX6EqRvP/s vHFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=E3c2Of8bOSTUT+E5j9BMTTuG1AnorX3mUa8Idm4u03U=; b=FI4O4Nws0y9i0MUPpe+Qk3/6B4bcgZcfil9SW0TIC3RIR/oW9SFosOmJ7SjxT16GNe LbTjs7yQpywraFz+5dvPkDmhRHrGtP9Hs6FFBZgTamxNjUk99ch3yq1n2sGxm0snhslD /lkaRe0vFpQyZM1R0SF73MPOTlKVa+x5EjXSikesZZUbkvJy/LnDlloCs2LGUCDyCOC4 tn9VY6IriS1191cf0dNwTbDHn/tKYWsBUOvPtOmrwxH9xTxmnPDCjMBswVkwRUxFPZzh n9FZtIq2mQMbvXW5PVlONyAhOjrD4G2LVYDKUUNpPbTPcvs5QgndmNuwvqef7XXkEYTO w8HQ== X-Gm-Message-State: AOAM532hhAPbwTTChJaUKrRIB1WQvEduiicDdXPXZlWz3zr5fqpyDVLf 2CUFa132w4zvR6AzLG4Y2iBR4r70B0bB5k8hG/2obHJdI5uQxQsroNjbdynt+emoaLwQRsmv8xv aVfMQpWxYkNSHncpZpwZXhy8QSUH4RVkII2K9gPvVyZ/fJUqMfr0xKbSltK2dztyIXos8akVYvP qB X-Google-Smtp-Source: ABdhPJw1yaTYVE2cxn5YB9B72A5zK4cYggtX80j7R+VNXi2bijd/w5KpkqDRSyX6KH0UsjXzMGhh3bEcbp9xVqN1Bd5d Sender: "jonathantanmy via sendgmr" X-Received: from twelve4.c.googlers.com ([fda3:e722:ac3:10:24:72f4:c0a8:437a]) (user=jonathantanmy job=sendgmr) by 2002:a62:63c6:0:b029:18a:8d05:5bdc with SMTP id x189-20020a6263c60000b029018a8d055bdcmr14365484pfb.37.1604363231813; Mon, 02 Nov 2020 16:27:11 -0800 (PST) Date: Mon, 2 Nov 2020 16:26:11 -0800 In-Reply-To: Message-Id: <148e39960a2185d2355cdfe34f8856e708fb1b80.1604362701.git.jonathantanmy@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.1.341.ge80a0c044ae-goog Subject: [PATCH 2/4] push: teach --base for ssh:// and file:// From: Jonathan Tan To: git@vger.kernel.org Cc: Jonathan Tan Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org For the ssh:// and file:// transports, teach push the "--base" parameter, which indicates an ancestor of the commits to be pushed that is believed to be known by the server. If specified, it will be communicated to receive-pack through an Extra Parameter. If the repository served by receive-pack indeed has that object, receive-pack will send an abbreviated ref advertisement containing only that object instead of the usual ref advertisement, reducing the size of the ref advertisement and, potentially, the size of the packfile to be sent. This is supported both on protocol v0 and v1. (Protocol v1 is the same as protocol v0, except that the client sends "version=1" as an Extra Parameter and the server responds with "version 1".) Support for http(s):// will be added in a subsequent patch. Signed-off-by: Jonathan Tan --- builtin/fetch-pack.c | 2 +- builtin/push.c | 12 +++++-- builtin/receive-pack.c | 46 +++++++++++++++++++++--- builtin/send-pack.c | 2 +- builtin/upload-pack.c | 2 +- connect.c | 8 ++++- connect.h | 4 ++- http-backend.c | 2 +- protocol.c | 5 ++- protocol.h | 5 ++- t/t5700-protocol-v1.sh | 80 ++++++++++++++++++++++++++++++++++++++++++ transport.c | 14 ++++++-- transport.h | 9 +++++ 13 files changed, 175 insertions(+), 16 deletions(-) diff --git a/builtin/fetch-pack.c b/builtin/fetch-pack.c index 58b7c1fbdc..9452969319 100644 --- a/builtin/fetch-pack.c +++ b/builtin/fetch-pack.c @@ -207,7 +207,7 @@ int cmd_fetch_pack(int argc, const char **argv, const char *prefix) if (args.diag_url) flags |= CONNECT_DIAG_URL; conn = git_connect(fd, dest, args.uploadpack, - flags); + flags, NULL); if (!conn) return args.diag_url ? 0 : 1; } diff --git a/builtin/push.c b/builtin/push.c index 6da3a8e5d3..a9f3278a24 100644 --- a/builtin/push.c +++ b/builtin/push.c @@ -381,7 +381,8 @@ static int push_with_options(struct transport *transport, struct refspec *rs, static int do_push(int flags, const struct string_list *push_options, - struct remote *remote) + struct remote *remote, + const char *push_base) { int i, errs; const char **url; @@ -405,6 +406,8 @@ static int do_push(int flags, transport_get(remote, url[i]); if (flags & TRANSPORT_PUSH_OPTIONS) transport->push_options = push_options; + if (push_base) + transport_set_option(transport, TRANS_OPT_PUSH_BASE, push_base); if (push_with_options(transport, push_refspec, flags)) errs++; } @@ -413,6 +416,8 @@ static int do_push(int flags, transport_get(remote, NULL); if (flags & TRANSPORT_PUSH_OPTIONS) transport->push_options = push_options; + if (push_base) + transport_set_option(transport, TRANS_OPT_PUSH_BASE, push_base); if (push_with_options(transport, push_refspec, flags)) errs++; } @@ -526,6 +531,7 @@ int cmd_push(int argc, const char **argv, const char *prefix) struct string_list *push_options; const struct string_list_item *item; struct remote *remote; + const char *push_base = NULL; struct option options[] = { OPT__VERBOSITY(&verbosity), @@ -562,6 +568,8 @@ int cmd_push(int argc, const char **argv, const char *prefix) TRANSPORT_FAMILY_IPV4), OPT_SET_INT('6', "ipv6", &family, N_("use IPv6 addresses only"), TRANSPORT_FAMILY_IPV6), + OPT_STRING(0, "base", &push_base, N_("revision"), + N_("ancestor of commits to be pushed that is believed to be known by the server")), OPT_END() }; @@ -629,7 +637,7 @@ int cmd_push(int argc, const char **argv, const char *prefix) if (strchr(item->string, '\n')) die(_("push options must not have new line characters")); - rc = do_push(flags, push_options, remote); + rc = do_push(flags, push_options, remote, push_base); string_list_clear(&push_options_cmdline, 0); string_list_clear(&push_options_config, 0); if (rc == -1) diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c index bb9909c52e..274f39c944 100644 --- a/builtin/receive-pack.c +++ b/builtin/receive-pack.c @@ -313,13 +313,38 @@ static void show_one_alternate_ref(const struct object_id *oid, show_ref(".have", oid); } -static void write_head_info(void) +struct check_connected_for_one_args { + struct object_id *base; + unsigned already_emitted : 1; +}; +static int check_connected_for_one(void *args, struct object_id *oid) +{ + struct check_connected_for_one_args *a = args; + + if (a->already_emitted) + return -1; + oidcpy(oid, a->base); + a->already_emitted = 1; + return 0; +} + +static void write_head_info(struct object_id *base) { static struct oidset seen = OIDSET_INIT; + if (base) { + struct check_connected_for_one_args args = {base}; + + if (!check_connected(check_connected_for_one, &args, NULL)) { + show_ref(".have", base); + goto refs_shown; + } + } + for_each_ref(show_ref_cb, &seen); for_each_alternate_ref(show_one_alternate_ref, &seen); oidset_clear(&seen); +refs_shown: if (!sent_capabilities) show_ref("capabilities^{}", &null_oid); @@ -2417,6 +2442,7 @@ int cmd_receive_pack(int argc, const char **argv, const char *prefix) struct oid_array ref = OID_ARRAY_INIT; struct shallow_info si; struct packet_reader reader; + struct strbuf base_sb = STRBUF_INIT; struct option options[] = { OPT__QUIET(&quiet, N_("quiet")), @@ -2451,7 +2477,7 @@ int cmd_receive_pack(int argc, const char **argv, const char *prefix) else if (0 <= receive_unpack_limit) unpack_limit = receive_unpack_limit; - switch (determine_protocol_version_server()) { + switch (determine_protocol_version_server(&base_sb)) { case protocol_v2: /* * push support for protocol v2 has not been implemented yet, @@ -2474,10 +2500,21 @@ int cmd_receive_pack(int argc, const char **argv, const char *prefix) } if (advertise_refs || !stateless_rpc) { - write_head_info(); + if (base_sb.len) { + struct object_id oid; + const char *p; + + if (parse_oid_hex(base_sb.buf, &oid, &p) || *p != '\0') + die("invalid base"); + write_head_info(&oid); + } else { + write_head_info(NULL); + } } - if (advertise_refs) + if (advertise_refs) { + strbuf_release(&base_sb); return 0; + } packet_reader_init(&reader, 0, NULL, 0, PACKET_READ_CHOMP_NEWLINE | @@ -2540,6 +2577,7 @@ int cmd_receive_pack(int argc, const char **argv, const char *prefix) } if (use_sideband) packet_flush(1); + strbuf_release(&base_sb); oid_array_clear(&shallow); oid_array_clear(&ref); free((void *)push_cert_nonce); diff --git a/builtin/send-pack.c b/builtin/send-pack.c index 7af148d733..1557db7a42 100644 --- a/builtin/send-pack.c +++ b/builtin/send-pack.c @@ -262,7 +262,7 @@ int cmd_send_pack(int argc, const char **argv, const char *prefix) fd[1] = 1; } else { conn = git_connect(fd, dest, receivepack, - args.verbose ? CONNECT_VERBOSE : 0); + args.verbose ? CONNECT_VERBOSE : 0, NULL); } packet_reader_init(&reader, fd[0], NULL, 0, diff --git a/builtin/upload-pack.c b/builtin/upload-pack.c index 6da8fa2607..32aaa7742c 100644 --- a/builtin/upload-pack.c +++ b/builtin/upload-pack.c @@ -48,7 +48,7 @@ int cmd_upload_pack(int argc, const char **argv, const char *prefix) if (!enter_repo(dir, strict)) die("'%s' does not appear to be a git repository", dir); - switch (determine_protocol_version_server()) { + switch (determine_protocol_version_server(NULL)) { case protocol_v2: serve_opts.advertise_capabilities = opts.advertise_refs; serve_opts.stateless_rpc = opts.stateless_rpc; diff --git a/connect.c b/connect.c index 5221f1764c..fa5b7ea886 100644 --- a/connect.c +++ b/connect.c @@ -1318,7 +1318,8 @@ static void fill_ssh_args(struct child_process *conn, const char *ssh_host, * the connection failed). */ struct child_process *git_connect(int fd[2], const char *url, - const char *prog, int flags) + const char *prog, int flags, + const char *other_extra_parameters) { char *hostandport, *path; struct child_process *conn; @@ -1369,6 +1370,11 @@ struct child_process *git_connect(int fd[2], const char *url, if (version > 0) strbuf_addf(&extra_parameters, "version=%d", version); + if (other_extra_parameters) { + if (extra_parameters.len) + strbuf_addch(&extra_parameters, ':'); + strbuf_addstr(&extra_parameters, other_extra_parameters); + } conn->use_shell = 1; conn->in = conn->out = -1; diff --git a/connect.h b/connect.h index c53586e929..6d502d4224 100644 --- a/connect.h +++ b/connect.h @@ -7,7 +7,9 @@ #define CONNECT_DIAG_URL (1u << 1) #define CONNECT_IPV4 (1u << 2) #define CONNECT_IPV6 (1u << 3) -struct child_process *git_connect(int fd[2], const char *url, const char *prog, int flags); +struct child_process *git_connect(int fd[2], const char *url, + const char *prog, int flags, + const char *other_extra_parameters); int finish_connect(struct child_process *conn); int git_connection_is_socket(struct child_process *conn); int server_supports(const char *feature); diff --git a/http-backend.c b/http-backend.c index a03b4bae22..7cd684c8b5 100644 --- a/http-backend.c +++ b/http-backend.c @@ -544,7 +544,7 @@ static void get_info_refs(struct strbuf *hdr, char *arg) end_headers(hdr); - if (determine_protocol_version_server() != protocol_v2) { + if (determine_protocol_version_server(NULL) != protocol_v2) { packet_write_fmt(1, "# service=git-%s\n", svc->name); packet_flush(1); } diff --git a/protocol.c b/protocol.c index 052d7edbb9..ad653b7521 100644 --- a/protocol.c +++ b/protocol.c @@ -42,7 +42,7 @@ enum protocol_version get_protocol_version_config(void) return protocol_v2; } -enum protocol_version determine_protocol_version_server(void) +enum protocol_version determine_protocol_version_server(struct strbuf *base_sb) { const char *git_protocol = getenv(GIT_PROTOCOL_ENVIRONMENT); enum protocol_version version = protocol_v0; @@ -67,6 +67,9 @@ enum protocol_version determine_protocol_version_server(void) v = parse_protocol_version(value); if (v > version) version = v; + } else if (skip_prefix(item->string, "base=", &value)) { + if (base_sb) + strbuf_addstr(base_sb, value); } } diff --git a/protocol.h b/protocol.h index cef1a4a01c..e418b92049 100644 --- a/protocol.h +++ b/protocol.h @@ -22,8 +22,11 @@ enum protocol_version get_protocol_version_config(void); * by setting appropriate values for the key 'version'. If a client doesn't * request a particular protocol version, a default of 'protocol_v0' will be * used. + * + * If base_sb is not NULL and an extra parameter "base" is specified, this + * function will append its value to base_sb. */ -enum protocol_version determine_protocol_version_server(void); +enum protocol_version determine_protocol_version_server(struct strbuf *base_sb); /* * Used by a client to determine which protocol version the server is speaking diff --git a/t/t5700-protocol-v1.sh b/t/t5700-protocol-v1.sh index 022901b9eb..22459d37f5 100755 --- a/t/t5700-protocol-v1.sh +++ b/t/t5700-protocol-v1.sh @@ -146,6 +146,56 @@ test_expect_success 'push with file:// using protocol v1' ' grep "push< version 1" log ' +test_expect_success 'push with file:// using protocol v1 and --base' ' + test_commit -C file_child four && + COMMON_HASH=$(git -C file_child rev-parse three) && + + # Push to another branch, as the target repository has the + # master branch checked out and we cannot push into it. + GIT_TRACE_PACKET=1 git -C file_child -c protocol.version=1 \ + push --base=three origin HEAD:client_branch_four 2>log && + + # Server responded using protocol v1 + grep "push< version 1" log && + # Server advertised only the expected object + grep "$COMMON_HASH .have" log +' + +test_expect_success 'push with file:// using protocol v0 and --base' ' + test_commit -C file_child five && + COMMON_HASH=$(git -C file_child rev-parse four) && + + # Push to another branch, as the target repository has the + # master branch checked out and we cannot push into it. + GIT_TRACE_PACKET=1 git -C file_child -c protocol.version=0 \ + push --base=four origin HEAD:client_branch_five 2>log && + + # Server did not respond with any version + ! grep "push< version" log && + # Server advertised only the expected object + grep "$COMMON_HASH .have" log +' + +test_expect_success 'push with invalid --base' ' + test_commit -C file_child six && + + # Server does not have "six". + test_must_fail git -C file_child -c protocol.version=0 \ + push --base=an_invalid_object origin HEAD:client_branch_six 2>log && + grep "is not a valid object" log +' + +test_expect_success 'push with --base that does not exist on server' ' + COMMON_HASH=$(git -C file_child rev-parse six) && + + # The push still succeeds. + GIT_TRACE_PACKET=1 git -C file_child -c protocol.version=0 \ + push --base=six origin HEAD:client_branch_six 2>log && + + # Server did not advertise "six", since it does not know it + ! grep "$COMMON_HASH .have" log +' + # Test protocol v1 with 'ssh://' transport # test_expect_success 'setup ssh wrapper' ' @@ -226,6 +276,36 @@ test_expect_success 'push with ssh:// using protocol v1' ' grep "push< version 1" log ' +test_expect_success 'push with ssh:// using protocol v1 and --base' ' + test_commit -C ssh_child four && + COMMON_HASH=$(git -C ssh_child rev-parse three) && + + # Push to another branch, as the target repository has the + # master branch checked out and we cannot push into it. + GIT_TRACE_PACKET=1 git -C ssh_child -c protocol.version=1 \ + push --base="$COMMON_HASH" origin HEAD:client_branch_four 2>log && + + # Server responded using protocol v1 + grep "push< version 1" log && + # Server advertised only the expected object + grep "$COMMON_HASH .have" log +' + +test_expect_success 'push with ssh:// using protocol v0 and --base' ' + test_commit -C ssh_child five && + COMMON_HASH=$(git -C ssh_child rev-parse four) && + + # Push to another branch, as the target repository has the + # master branch checked out and we cannot push into it. + GIT_TRACE_PACKET=1 git -C ssh_child -c protocol.version=0 \ + push --base="$COMMON_HASH" origin HEAD:client_branch_five 2>log && + + # Server did not respond with any version + ! grep "push< version" log && + # Server advertised only the expected object + grep "$COMMON_HASH .have" log +' + # Test protocol v1 with 'http://' transport # . "$TEST_DIRECTORY"/lib-httpd.sh diff --git a/transport.c b/transport.c index ffe2115845..531ca0a834 100644 --- a/transport.c +++ b/transport.c @@ -236,6 +236,10 @@ static int set_git_option(struct git_transport_options *opts, list_objects_filter_die_if_populated(&opts->filter_options); parse_list_objects_filter(&opts->filter_options, value); return 0; + } else if (!strcmp(name, TRANS_OPT_PUSH_BASE)) { + if (get_oid(value, &opts->push_base)) + die(_("transport: '%s' is not a valid object"), value); + return 0; } return 1; } @@ -244,6 +248,7 @@ static int connect_setup(struct transport *transport, int for_push) { struct git_transport_data *data = transport->data; int flags = transport->verbose > 0 ? CONNECT_VERBOSE : 0; + char *extra_parameters = NULL; if (data->conn) return 0; @@ -254,11 +259,16 @@ static int connect_setup(struct transport *transport, int for_push) case TRANSPORT_FAMILY_IPV6: flags |= CONNECT_IPV6; break; } + if (!is_null_oid(&data->options.push_base)) + extra_parameters = xstrfmt("base=%s", + oid_to_hex(&data->options.push_base)); + data->conn = git_connect(data->fd, transport->url, for_push ? data->options.receivepack : data->options.uploadpack, - flags); + flags, extra_parameters); + free(extra_parameters); return 0; } @@ -815,7 +825,7 @@ static int connect_git(struct transport *transport, const char *name, { struct git_transport_data *data = transport->data; data->conn = git_connect(data->fd, transport->url, - executable, 0); + executable, 0, NULL); fd[0] = data->fd[0]; fd[1] = data->fd[1]; return 0; diff --git a/transport.h b/transport.h index ca409ea1e4..ce51edd673 100644 --- a/transport.h +++ b/transport.h @@ -46,6 +46,12 @@ struct git_transport_options { * transport_set_option(). */ struct oid_array *negotiation_tips; + + /* + * When pushing, if this is not a null OID, indicates an ancestor of + * the commits to be pushed that is believed to be known by the server. + */ + struct object_id push_base; }; enum transport_family { @@ -208,6 +214,9 @@ void transport_check_allowed(const char *type); /* Request atomic (all-or-nothing) updates when pushing */ #define TRANS_OPT_ATOMIC "atomic" +/* See "push_base" in struct git_transport_options */ +#define TRANS_OPT_PUSH_BASE "push-base" + /** * Returns 0 if the option was used, non-zero otherwise. Prints a * message to stderr if the option is not used. From patchwork Tue Nov 3 00:26:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Tan X-Patchwork-Id: 11875619 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-20.4 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8DF88C00A89 for ; Tue, 3 Nov 2020 00:27:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4569422268 for ; Tue, 3 Nov 2020 00:27:16 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="jivpCSFT" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727142AbgKCA1P (ORCPT ); Mon, 2 Nov 2020 19:27:15 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57876 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725940AbgKCA1O (ORCPT ); Mon, 2 Nov 2020 19:27:14 -0500 Received: from mail-qt1-x849.google.com (mail-qt1-x849.google.com [IPv6:2607:f8b0:4864:20::849]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 35C5BC0617A6 for ; Mon, 2 Nov 2020 16:27:14 -0800 (PST) Received: by mail-qt1-x849.google.com with SMTP id z12so9140282qto.4 for ; Mon, 02 Nov 2020 16:27:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=wMB7rRaE+f+qYdBZR4vOfyPKyZuUBfNajYcnOZdqbaI=; b=jivpCSFTp/jYm8Kyt0ajhjMFObYknno0UeVp+4ECh2AZc6SF19Kh7tGu8cWtX+/5tj +XSKhWRg8eay/3OylHJnZlLTIyMZ4toLZ2Yr7XidbUkVOA67xZiRqzVNTI8r6xC5Y4if uzsW0ehxwiTX9h+1Ory3BB2OgIgqFdnIqYEhma4mUvx1sbTbGgQg4Tf3LeTQEYghmXOB OPROCavkiMVEd609zkFHQDre6uEwgnwz9/qsWaImv6x3LYQUGtY6v55jbXgOQp3tGy3q nTwtK6xdiDN5APEh+ebcjCNrBz4ZnMA6PgRQRiwtjeiPCzQCU9CnbFw4qKXcf3ua6VSS XItQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=wMB7rRaE+f+qYdBZR4vOfyPKyZuUBfNajYcnOZdqbaI=; b=sHu8PRR6MzjaVHzsQUWiqbrPexWpJlLCd7fWTMgmh9a2BSSN67CFQQ0CigOYGWC+xQ bQF+zUE/+jcprCkLiRt02aurN/QLo43r8qeptFo4ctnqPbily/CPJ5gU+bgK48rK+xCC Q9uOqG2jnq3SXwz9OWaH3xygHLKUCV/4vStk2LTUBfEX6Ss3mxejWZnHttPds4dgKodU c5TdzpJzpjTu+wf551OjjE12GPPeyFNQbOVdy6tZ4+7V8z8TxaLpSCJc4ZS8X46H6Z6C kz/qZzu8MLCr0JOeXFI06jKpoNW+39I3Fr07pwQNyDTT0lGWPLgPGxRPwulX14OExRkd APnw== X-Gm-Message-State: AOAM533LUNXACR3EH4DQLgJPEWPnZnM1JAz5Bdj/0rR85dO3rCsqsYtU 0tkKHxo6YSlz82OgkYRmnbD+xZYdodDl4AX/AIUwqBNBoVdQrhNd3QO4WNusfM/9r5miahF4xEN vOKL8VK/XnFT9AWqCSBTyfl+Jbx9bj8hENEDbGTng6Q9hrxiXATVpE5lBwr65q0j/3WZSjLt/YG ra X-Google-Smtp-Source: ABdhPJwiaD3ERVkVxvU6+jmklynqtNDMbUNf9S8+JXe5zjvWHlAtymt09Rh0eB5uNNmb+4L/dWV6wPzQGxDXDvTesfpn Sender: "jonathantanmy via sendgmr" X-Received: from twelve4.c.googlers.com ([fda3:e722:ac3:10:24:72f4:c0a8:437a]) (user=jonathantanmy job=sendgmr) by 2002:a0c:b525:: with SMTP id d37mr24922151qve.31.1604363233313; Mon, 02 Nov 2020 16:27:13 -0800 (PST) Date: Mon, 2 Nov 2020 16:26:12 -0800 In-Reply-To: Message-Id: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.1.341.ge80a0c044ae-goog Subject: [PATCH 3/4] remote-curl: teach --base for http(s):// From: Jonathan Tan To: git@vger.kernel.org Cc: Jonathan Tan Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org For the http(s):// transports, teach push the "--base" parameter, which indicates an ancestor of the commits to be pushed that is believed to be known by the server. This is done through a new remote helper capability and option "push-base". If a remote helper advertises this capability, then it also supports the option of the same name. Git can then use this option to inform the remote helper of a base that the user has specified. See the commit message of this commit's parent for more information. Signed-off-by: Jonathan Tan --- Documentation/gitremote-helpers.txt | 8 ++++++++ remote-curl.c | 25 +++++++++++++++++------- t/t5700-protocol-v1.sh | 30 +++++++++++++++++++++++++++++ transport-helper.c | 15 +++++++++++++++ 4 files changed, 71 insertions(+), 7 deletions(-) diff --git a/Documentation/gitremote-helpers.txt b/Documentation/gitremote-helpers.txt index 6f1e269ae4..b4eff16a59 100644 --- a/Documentation/gitremote-helpers.txt +++ b/Documentation/gitremote-helpers.txt @@ -116,6 +116,9 @@ Supported commands: 'stateless-connect'. + Supported commands: 'list for-push', 'push'. +'push-base':: + Indicates that this remote helper supports the 'push-base' option. + 'export':: Can discover remote refs and push specified objects from a fast-import stream to remote refs. @@ -541,6 +544,11 @@ set by Git if the remote helper has the 'option' capability. If set to an algorithm, indicate that the caller wants to interact with the remote side using that algorithm. +'option push-base' :: + Only supported if this remote helper advertises the 'push-base' + capability. Indicate that is an ancestor of the commits to be + pushed, and it is believed to be known by the server. + SEE ALSO -------- linkgit:git-remote[1] diff --git a/remote-curl.c b/remote-curl.c index 32cc4a0c55..7919e4ebcf 100644 --- a/remote-curl.c +++ b/remote-curl.c @@ -29,6 +29,7 @@ struct options { struct string_list deepen_not; struct string_list push_options; char *filter; + struct object_id push_base; unsigned progress : 1, check_self_contained_and_connected : 1, cloning : 1, @@ -205,6 +206,10 @@ static int set_option(const char *name, const char *value) options.hash_algo = &hash_algos[algo]; } return 0; + } else if (!strcmp(name, "push-base")) { + if (get_oid(value, &options.push_base)) + die(_("%s is not a valid object"), value); + return 0; } else { return 1 /* unsupported */; } @@ -364,16 +369,21 @@ static int show_http_message(struct strbuf *type, struct strbuf *charset, } static int get_protocol_http_header(enum protocol_version version, + const struct object_id *base, struct strbuf *header) { - if (version > 0) { + if (version > 0) strbuf_addf(header, GIT_PROTOCOL_HEADER ": version=%d", version); - - return 1; + if (!is_null_oid(base)) { + if (version > 0) + strbuf_addch(header, ':'); + else + strbuf_addstr(header, GIT_PROTOCOL_HEADER ": "); + strbuf_addf(header, "base=%s", oid_to_hex(base)); } - return 0; + return !!(version > 0 || !(is_null_oid(base))); } static void check_smart_http(struct discovery *d, const char *service, @@ -469,7 +479,7 @@ static struct discovery *discover_refs(const char *service, int for_push) version = protocol_v0; /* Add the extra Git-Protocol header */ - if (get_protocol_http_header(version, &protocol_header)) + if (get_protocol_http_header(version, &options.push_base, &protocol_header)) string_list_append(&extra_headers, protocol_header.buf); memset(&http_options, 0, sizeof(http_options)); @@ -1074,7 +1084,7 @@ static int rpc_service(struct rpc_state *rpc, struct discovery *heads, strbuf_addf(&buf, "Accept: application/x-%s-result", svc); rpc->hdr_accept = strbuf_detach(&buf, NULL); - if (get_protocol_http_header(heads->version, &buf)) + if (get_protocol_http_header(heads->version, &options.push_base, &buf)) rpc->protocol_header = strbuf_detach(&buf, NULL); else rpc->protocol_header = NULL; @@ -1406,7 +1416,7 @@ static int stateless_connect(const char *service_name) rpc.service_url = xstrfmt("%s%s", url.buf, rpc.service_name); rpc.hdr_content_type = xstrfmt("Content-Type: application/x-%s-request", rpc.service_name); rpc.hdr_accept = xstrfmt("Accept: application/x-%s-result", rpc.service_name); - if (get_protocol_http_header(discover->version, &buf)) { + if (get_protocol_http_header(discover->version, &options.push_base, &buf)) { rpc.protocol_header = strbuf_detach(&buf, NULL); } else { rpc.protocol_header = NULL; @@ -1538,6 +1548,7 @@ int cmd_main(int argc, const char **argv) printf("push\n"); printf("check-connectivity\n"); printf("object-format\n"); + printf("push-base\n"); printf("\n"); fflush(stdout); } else if (skip_prefix(buf.buf, "stateless-connect ", &arg)) { diff --git a/t/t5700-protocol-v1.sh b/t/t5700-protocol-v1.sh index 22459d37f5..4ee29c1be0 100755 --- a/t/t5700-protocol-v1.sh +++ b/t/t5700-protocol-v1.sh @@ -373,6 +373,36 @@ test_expect_success 'push with http:// using protocol v1' ' grep "git< version 1" log ' +test_expect_success 'push with http:// using protocol v1 and --base' ' + test_commit -C http_child four && + COMMON_HASH=$(git -C http_child rev-parse three) && + + # Push to another branch, as the target repository has the + # master branch checked out and we cannot push into it. + GIT_TRACE_PACKET=1 test_might_fail git -C http_child -c protocol.version=1 \ + push --base=three origin HEAD:client_branch_four 2>log && + + # Server responded using protocol v1 + grep "git< version 1" log && + # Server advertised only the expected object + grep "$COMMON_HASH .have" log +' + +test_expect_success 'push with http:// using protocol v0 and --base' ' + test_commit -C http_child five && + COMMON_HASH=$(git -C http_child rev-parse four) && + + # Push to another branch, as the target repository has the + # master branch checked out and we cannot push into it. + GIT_TRACE_PACKET=1 git -C http_child -c protocol.version=0 \ + push --base=four origin HEAD:client_branch_five 2>log && + + # Server did not respond with any version + ! grep "git< version" log && + # Server advertised only the expected object + grep "$COMMON_HASH .have" log +' + # DO NOT add non-httpd-specific tests here, because the last part of this # test script is only executed when httpd is available and enabled. diff --git a/transport-helper.c b/transport-helper.c index b573b6c730..15d9527419 100644 --- a/transport-helper.c +++ b/transport-helper.c @@ -27,6 +27,7 @@ struct helper_data { export : 1, option : 1, push : 1, + push_base : 1, connect : 1, stateless_connect : 1, signed_tags : 1, @@ -186,6 +187,8 @@ static struct child_process *get_helper(struct transport *transport) data->option = 1; else if (!strcmp(capname, "push")) data->push = 1; + else if (!strcmp(capname, "push-base")) + data->push_base = 1; else if (!strcmp(capname, "import")) data->import = 1; else if (!strcmp(capname, "bidi-import")) @@ -1183,6 +1186,18 @@ static struct ref *get_refs_list_using_list(struct transport *transport, exit(128); } + if (!is_null_oid(&data->transport_options.push_base)) { + if (data->push_base) { + write_str_in_full(helper->in, "option push-base "); + write_str_in_full(helper->in, oid_to_hex(&data->transport_options.push_base)); + write_str_in_full(helper->in, "\n"); + if (recvline(data, &buf) || strcmp(buf.buf, "ok")) + exit(128); + } else { + warning(_("transport does not support --base")); + } + } + if (data->push && for_push) write_str_in_full(helper->in, "list for-push\n"); else From patchwork Tue Nov 3 00:26:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Tan X-Patchwork-Id: 11875625 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-25.4 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8D57CC4742C for ; Tue, 3 Nov 2020 00:27:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5050E22268 for ; Tue, 3 Nov 2020 00:27:17 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="a79WY+Yu" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727202AbgKCA1Q (ORCPT ); Mon, 2 Nov 2020 19:27:16 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57882 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725940AbgKCA1P (ORCPT ); Mon, 2 Nov 2020 19:27:15 -0500 Received: from mail-pl1-x649.google.com (mail-pl1-x649.google.com [IPv6:2607:f8b0:4864:20::649]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A944CC0617A6 for ; Mon, 2 Nov 2020 16:27:15 -0800 (PST) Received: by mail-pl1-x649.google.com with SMTP id y9so9550288pll.18 for ; Mon, 02 Nov 2020 16:27:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=I8LOc03FQsEbG+peTCv8TKVa0w7CTfBWvRVZBeYaACY=; b=a79WY+Yu7sRwHIfDCLkpxomiqkiUvWFcpCta0OPk06TapH2D1EnI2XXSoi8OxE3PPg +LlaXBSowZxra6rS9v8XVSXizdl5tg4DrdunrlqZCOkfgLWsxGyU69Uh3SyMEqXMECqK rKko9k0vC+IxQA9CvXRxo8bOmuzGq78X16BhLzhH8Oer1TU4uiNdqJz3shq9Gg7EBP48 CQekzjVxIbQtS2/lCVDEJmnOiID+UIXFYQW5YuwemnakddqDhHnhBPXilMFgXEW0YsiZ tffZBWvzOObUaXX4dHaDxEdlKwvXCLsYuhi+cjr29kaLUzfl9oZBrgKe28vw6soofSlk WbAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=I8LOc03FQsEbG+peTCv8TKVa0w7CTfBWvRVZBeYaACY=; b=XrBGDVHWzvS2aTQcdxwhX44v/mbTXFOlAtn2YKvZkkAX5I/FnnFyEMRq8d9PZF4BAJ 8v5DJ5rR+GMGOAIoHELrzK377T2OBVdwRhsi1Az09Jyl0jpSVU7wnnbKK0ag8353Hj3O DjIYxA38tgu8w59HF+FpaxCHx9KPj5PX2WHvs4MOeRSsf6Hro46dY4Mu/hJYLhYoaZRc Zk+7PTOZR0pTXWQglRVg+q0GxDzxB8iEUnZG4BXnDeVPQinTRT2eZlkneWmfX5Y/MxxJ NFeGsWzJpvBffnnS8LmwieyHd0hZkxemuIF/flOqbKWCkHI0i8/BRNBWZrUeDpETtpIs Wtpw== X-Gm-Message-State: AOAM533rywOiOb7CNbLeyPJj3NV6l2kZpzII+42Iw0PDFklQTdjHESgt k1j4deAfeZrB/fXeTBnoxTolle47q0xo9pJuoAi0ZchIi1kCtm7EJG4cQvdMb9Njw/Gh2aQwbID vPQV0KZCC7QH/JBUJHOkdOTZf2bIqeAu6aFb+pb+7q0m+eO3QLVa0IVFZrH9gIFfV+UXu9l/+Sy 8P X-Google-Smtp-Source: ABdhPJyMhVmmnR3nOIR1uI9y5eKu1rs3ogypp9aqmnFLS+vZYhHGvUZg8pvGAvsB+8gT+1frLPQWLEeA+fxZ5gC0XstI Sender: "jonathantanmy via sendgmr" X-Received: from twelve4.c.googlers.com ([fda3:e722:ac3:10:24:72f4:c0a8:437a]) (user=jonathantanmy job=sendgmr) by 2002:a17:902:d710:b029:d3:7e54:96d8 with SMTP id w16-20020a170902d710b02900d37e5496d8mr23385377ply.65.1604363234958; Mon, 02 Nov 2020 16:27:14 -0800 (PST) Date: Mon, 2 Nov 2020 16:26:13 -0800 In-Reply-To: Message-Id: <6250c13897e3cc01f247d80c148cf8dc5e7f3ad0.1604362701.git.jonathantanmy@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.1.341.ge80a0c044ae-goog Subject: [PATCH 4/4] Doc: push with --base From: Jonathan Tan To: git@vger.kernel.org Cc: Jonathan Tan Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Includes protocol documentation and a design document. Signed-off-by: Jonathan Tan --- Documentation/technical/pack-protocol.txt | 10 ++-- Documentation/technical/push-with-base.txt | 61 ++++++++++++++++++++++ 2 files changed, 68 insertions(+), 3 deletions(-) create mode 100644 Documentation/technical/push-with-base.txt diff --git a/Documentation/technical/pack-protocol.txt b/Documentation/technical/pack-protocol.txt index e13a2c064d..0485616701 100644 --- a/Documentation/technical/pack-protocol.txt +++ b/Documentation/technical/pack-protocol.txt @@ -59,9 +59,9 @@ Parameters", and are supported by the Git, SSH, and HTTP protocols. Each Extra Parameter takes the form of `=` or ``. Servers that receive any such Extra Parameters MUST ignore all -unrecognized keys. Currently, the only Extra Parameter recognized is -"version" with a value of '1' or '2'. See protocol-v2.txt for more -information on protocol version 2. +unrecognized keys. Currently, the only Extra Parameters recognized are +"version" with a value of '1' or '2' and, for push, "base" with an OID. See +protocol-v2.txt for more information on protocol version 2. Git Transport ------------- @@ -506,6 +506,10 @@ real difference is that the capability listing is different - the only possible values are 'report-status', 'report-status-v2', 'delete-refs', 'ofs-delta', 'atomic' and 'push-options'. +If a "base=" Extra Parameter was sent by the client, and the +server recognizes that object, the server MAY send " .have" in +lieu of all the reference obj-ids and names. + Reference Update Request and Packfile Transfer ---------------------------------------------- diff --git a/Documentation/technical/push-with-base.txt b/Documentation/technical/push-with-base.txt new file mode 100644 index 0000000000..d56aa7f900 --- /dev/null +++ b/Documentation/technical/push-with-base.txt @@ -0,0 +1,61 @@ +Push with base design notes +=========================== + +This feature allows clients, when pushing, to indicate that a +certain object is an ancestor of all pushed commits and that they +believe that the server knows of this object. This in turn allows +servers to send an abbreviated ref advertisement containing only that +object. + +Besides bandwidth savings, this also ensures that the ref +advertisement contains information relevant to the client. For +example, at least one project (Gerrit [1]) have included workarounds +to send ancestors of refs that move often, even though the ref +advertisement is only meant to contain refs. + +[1] https://gerrit.googlesource.com/gerrit/+/refs/heads/master/java/com/google/gerrit/server/git/receive/HackPushNegotiateHook.java + + +Design overview +--------------- + +The "base" being sent is sent as an Extra Parameter, supported in the +git://, ssh://, and http(s):// protocols. By sending it as an Extra +Parameter, the server is aware of this parameter before it generates +the ref advertisement, thus making it able to tailor the ref +advertisement accordingly. Sending it as an Extra Parameter also makes +this protocol backwards-compatible, as servers will ignore any Extra +Parameters they do not understand. (The push will then proceed as if +neither party had this feature.) + +The remote helper protocol has been extended to support the +"push-base" capability and an option of the same name. When a remote +helper advertises this capability, it thus indicates that it supports +this option. Git then will send "option push-base" if the user +specifies it when invoking "git push". + +The remote-curl remote helper bundled with Git has been updated to +support this capability and option. + + +Future work +----------- + +In the future, we might want a way to automatically determine the base +instead of always having the user specify it. However, this does not +make obsolete any of the current work - once the base is automatically +determined, we still need this protocol to communicate it to the +server, and allowing the user to specify the base manually is still +useful. + + +Alternatives +------------ + +- Making a more substantial protocol change like "fetch" protocol v2. + This would eliminate the need for some of the remote helper updates; + as part of the protocol change, the protocol could be made to + support "stateless-connect" and thus no remote helper updates (like + "push-base") would be needed. For "fetch", the protocol change has + enabled features like wanted-refs and packfile-uris, but I do not + have any similar ideas in mind for "push".