From patchwork Fri Jun 19 17:55:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11614831 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1AFAD618 for ; Fri, 19 Jun 2020 17:57:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id F373721531 for ; Fri, 19 Jun 2020 17:57:02 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="zulGy1w1" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2436496AbgFSR5B (ORCPT ); Fri, 19 Jun 2020 13:57:01 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:39614 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2404689AbgFSR4p (ORCPT ); Fri, 19 Jun 2020 13:56:45 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:7d4e:cde:7c41:71c2]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 3293D60791; Fri, 19 Jun 2020 17:56:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1592589373; bh=wLSEs4YSnnzYOGxzW174eZGSDZmYUM3piJq4zdKUr5M=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=zulGy1w1e8m2XM7W4SIoPFm6tBVqmMQ2Pnx1Dmp8pmNrR3jfP32Nl9wBD2/K6FmTq VAuU8sCUjVs/syK0krVh8GiL3AnBqBcvn4W3txBPlL3+nkFAWnD5JlKl8DkTgO6QFq l9+nAHk97dMMcrWkmPX3lFP+jjrpI6FfPUuSQULIFnVm+GJPAgoEBI5zwPJrRGS7oz QDh0aT5ZYFlCaD1/kYZak0cX8DqL3J4l4PlaMraPkNS7RtLflutYw75tVeCbseOnKG v+FpFbbR0gqT7m+6Z6UWri/e4cdC7fNAcq2mVrx7GXTGDsElkcycMfLiYlJx+h2S22 ZhQsgVrXxhc6H+YceOwLcbWMZCWOGuJ0lYFXWgYlXz7/kqbT/H8fw7Q6LcjG54rRHs CHeQEvgpP6+Eg6NaOXPi0X2lqjAfvJGvc+rY+2E4V2xl7bFuH7awT/FWtViFFEFvZL waO4dNKlTW2ob4tD5JxPCG9ONV0oz7nqz1yvYyW2xurQYOATIDm From: "brian m. carlson" To: Cc: Junio C Hamano , =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v3 01/44] t1050: match object ID paths in a hash-insensitive way Date: Fri, 19 Jun 2020 17:55:18 +0000 Message-Id: <20200619175601.569856-2-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.278.ge193c7cf3a9 In-Reply-To: <20200619175601.569856-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200619175601.569856-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org The pattern here looking for failures is specific to SHA-1. Let's create a variable that matches the regex or glob pattern for a path within the objects directory. Signed-off-by: brian m. carlson --- t/t1050-large.sh | 2 +- t/test-lib.sh | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/t/t1050-large.sh b/t/t1050-large.sh index 184b479a21..7f88ea07c2 100755 --- a/t/t1050-large.sh +++ b/t/t1050-large.sh @@ -64,7 +64,7 @@ test_expect_success 'add a large file or two' ' test $count = 1 && cnt=$(git show-index <"$idx" | wc -l) && test $cnt = 2 && - for l in .git/objects/??/?????????????????????????????????????? + for l in .git/objects/$OIDPATH_REGEX do test_path_is_file "$l" || continue bad=t diff --git a/t/test-lib.sh b/t/test-lib.sh index dbc027ff26..618a7c8d5b 100644 --- a/t/test-lib.sh +++ b/t/test-lib.sh @@ -1414,6 +1414,7 @@ test_oid_init ZERO_OID=$(test_oid zero) OID_REGEX=$(echo $ZERO_OID | sed -e 's/0/[0-9a-f]/g') +OIDPATH_REGEX=$(test_oid_to_path $ZERO_OID | sed -e 's/0/[0-9a-f]/g') EMPTY_TREE=$(test_oid empty_tree) EMPTY_BLOB=$(test_oid empty_blob) _z40=$ZERO_OID From patchwork Fri Jun 19 17:55:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11614795 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 977C413A0 for ; Fri, 19 Jun 2020 17:56:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6CC8220786 for ; Fri, 19 Jun 2020 17:56:17 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="znvSeBHK" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2392564AbgFSR4Q (ORCPT ); Fri, 19 Jun 2020 13:56:16 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:39410 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390892AbgFSR4P (ORCPT ); Fri, 19 Jun 2020 13:56:15 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:7d4e:cde:7c41:71c2]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id AF23D60A5C; Fri, 19 Jun 2020 17:56:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1592589374; bh=vM7pid11nQsZOGNYd0VAPXG+qD4DXfjxnlg7+N9j2CA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=znvSeBHKHNmgclWrBuSKaIfsgR7Cps42IlgE267LcFVma/GkOQz1QmgKlUXQshgD7 Nznjrob4TwdpsuwyBILGPk2/1x0KUq2WV2zdaaEP1/TNpTP7QKx5fsQt1lr7V+c4eI XLApLtqDw+1HxmFjnqqGdN0jDGwnYQLljPBTkh4sR9QJ5KTgvbzLUKkdCvSP+N0eKV NcMaMpRFVHFkNP2HFQk1/OejzdqViyKqANp5Rgu222w/dfGW7iScZq3I30vTscsR88 0JUq3dMTXKnKGRPxjMmKkQmVIxHdE3QqpDcGRVEGt68DXmXUliYfL6wa7iVOpwRj4j GzTPAsrz9SeLmNuVnFC5AtLfqtHFDn8jzg/fjy3hcws2lSUVr6qGo+ObkWgwetH1Fh ZM3Ipi+aA20SwaSbfdTEtRudtbe1JWuyk9+gr5cZ+tapP7QY9TyXNKd+rxta93v5Ei NpqRwglhfh7j7o9wPCJZVl/HZj7hColGqGU8riHm4t1H63Kl1sg From: "brian m. carlson" To: Cc: Junio C Hamano , =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v3 02/44] Documentation: document v1 protocol object-format capability Date: Fri, 19 Jun 2020 17:55:19 +0000 Message-Id: <20200619175601.569856-3-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.278.ge193c7cf3a9 In-Reply-To: <20200619175601.569856-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200619175601.569856-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Document a capability that indicates which hash algorithms are in use by both sides of a remote connection. Use the term "object-format", since this is the term used for the repository extension as well. Signed-off-by: brian m. carlson --- Documentation/technical/protocol-capabilities.txt | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/Documentation/technical/protocol-capabilities.txt b/Documentation/technical/protocol-capabilities.txt index 2b267c0da6..36ccd14f97 100644 --- a/Documentation/technical/protocol-capabilities.txt +++ b/Documentation/technical/protocol-capabilities.txt @@ -176,6 +176,21 @@ agent strings are purely informative for statistics and debugging purposes, and MUST NOT be used to programmatically assume the presence or absence of particular features. +object-format +------------- + +This capability, which takes a hash algorithm as an argument, indicates +that the server supports the given hash algorithms. It may be sent +multiple times; if so, the first one given is the one used in the ref +advertisement. + +When provided by the client, this indicates that it intends to use the +given hash algorithm to communicate. The algorithm provided must be one +that the server supports. + +If this capability is not provided, it is assumed that the only +supported algorithm is SHA-1. + symref ------ From patchwork Fri Jun 19 17:55:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11614799 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BBCF4618 for ; Fri, 19 Jun 2020 17:56:22 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A56BB20786 for ; Fri, 19 Jun 2020 17:56:22 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="pKrleMJd" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2393701AbgFSR4S (ORCPT ); Fri, 19 Jun 2020 13:56:18 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:39416 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2391253AbgFSR4P (ORCPT ); Fri, 19 Jun 2020 13:56:15 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:7d4e:cde:7c41:71c2]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 3B01060A5D; Fri, 19 Jun 2020 17:56:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1592589374; bh=vkuc2K9UfbrN+eJ7AOxlCMVFg/ows5cpcDfXPInHyU8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=pKrleMJd+UTvzWlF8ae4EfOjC4buj/nyBIWxS96iMnjZpM7NuVG1Pq+jkcOy6xU9P WWAZ3GdsdgnggEuA7qCzH8kznyYCeW6I6RgsynYhi/BPbzNgfhe7+ncXUXYc0np6d+ TKV8qYQYdi46Yc4GP8aL8LvKvVQDgU5e7cPuW8Xa5H4vJBUNIATuge++i6zk3nywBn jdhAe3rOg+FRMW8DhBzDxj2V9xGpRus9csxO3ZVF2iqQ7N5fR3/o21u56OIBRNhFYB B0VdSV4fGRI4H6VRAkd1GiGCyQH8jGEmmapflRwXcb+8zZzDIJQR3lPhhG7SKPifGu F4T++P2rY4RRWWmn7nQOi/vAPPQayykrHtjdmiZkWCl4xMIs03OMklGbY6kMlxHsdo 0R98lEjU+zJ5CWk0EZGZbKXRh14kgNNX+Dx5pdU1xQVlVhksNfzenkD3LIq8n7mkAd JP3fVfqNCzxgK/hrNsYUkjQB9bjGtZAhaJmmpgrur+PrR2eyBBS From: "brian m. carlson" To: Cc: Junio C Hamano , =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v3 03/44] connect: have ref processing code take struct packet_reader Date: Fri, 19 Jun 2020 17:55:20 +0000 Message-Id: <20200619175601.569856-4-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.278.ge193c7cf3a9 In-Reply-To: <20200619175601.569856-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200619175601.569856-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In a future patch, we'll want to access multiple members from struct packet_reader when parsing references. Therefore, have the ref parsing code take pointers to struct reader instead of having to pass multiple arguments to each function. Rename the len variable to "linelen" to make it clearer what the variable does in light of the variable change. Signed-off-by: brian m. carlson --- connect.c | 27 ++++++++++++++++----------- 1 file changed, 16 insertions(+), 11 deletions(-) diff --git a/connect.c b/connect.c index 0df45a1108..e66e779ebd 100644 --- a/connect.c +++ b/connect.c @@ -205,17 +205,19 @@ static void annotate_refs_with_symref_info(struct ref *ref) string_list_clear(&symref, 0); } -static void process_capabilities(const char *line, int *len) +static void process_capabilities(struct packet_reader *reader, int *linelen) { + const char *line = reader->line; int nul_location = strlen(line); - if (nul_location == *len) + if (nul_location == *linelen) return; server_capabilities_v1 = xstrdup(line + nul_location + 1); - *len = nul_location; + *linelen = nul_location; } -static int process_dummy_ref(const char *line) +static int process_dummy_ref(const struct packet_reader *reader) { + const char *line = reader->line; struct object_id oid; const char *name; @@ -235,9 +237,11 @@ static void check_no_capabilities(const char *line, int len) line + strlen(line)); } -static int process_ref(const char *line, int len, struct ref ***list, - unsigned int flags, struct oid_array *extra_have) +static int process_ref(const struct packet_reader *reader, int len, + struct ref ***list, unsigned int flags, + struct oid_array *extra_have) { + const char *line = reader->line; struct object_id old_oid; const char *name; @@ -261,9 +265,10 @@ static int process_ref(const char *line, int len, struct ref ***list, return 1; } -static int process_shallow(const char *line, int len, +static int process_shallow(const struct packet_reader *reader, int len, struct oid_array *shallow_points) { + const char *line = reader->line; const char *arg; struct object_id old_oid; @@ -317,20 +322,20 @@ struct ref **get_remote_heads(struct packet_reader *reader, switch (state) { case EXPECTING_FIRST_REF: - process_capabilities(reader->line, &len); - if (process_dummy_ref(reader->line)) { + process_capabilities(reader, &len); + if (process_dummy_ref(reader)) { state = EXPECTING_SHALLOW; break; } state = EXPECTING_REF; /* fallthrough */ case EXPECTING_REF: - if (process_ref(reader->line, len, &list, flags, extra_have)) + if (process_ref(reader, len, &list, flags, extra_have)) break; state = EXPECTING_SHALLOW; /* fallthrough */ case EXPECTING_SHALLOW: - if (process_shallow(reader->line, len, shallow_points)) + if (process_shallow(reader, len, shallow_points)) break; die(_("protocol error: unexpected '%s'"), reader->line); case EXPECTING_DONE: From patchwork Fri Jun 19 17:55:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11614851 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E29E213A0 for ; Fri, 19 Jun 2020 17:57:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C8EC720776 for ; Fri, 19 Jun 2020 17:57:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="ou+MNh76" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2436519AbgFSR5Z (ORCPT ); Fri, 19 Jun 2020 13:57:25 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:39624 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2404749AbgFSR4q (ORCPT ); Fri, 19 Jun 2020 13:56:46 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:7d4e:cde:7c41:71c2]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id B6F4160A5E; Fri, 19 Jun 2020 17:56:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1592589375; bh=VRoONdvcbARP5+3wr28J+/lkAxkxEY4gUpFmTPvhqlU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=ou+MNh767hw6VmhymtyGEFb+BR67zJBGeu+mU4sAe367aFxImUuxQgm4Hwrnyemnb PpmbcwS53G2c9jE4CoC5qSutoaJtLJh8HMVtqdP7bN0gpbhc0n14Xt3zSBapUsB+aK vGNnEx1zLWTDyIDZTe3o3sERqktXGFeiedt8SqMc/GwIXWJgK3uJV6yWFkvZ8z3PF6 zv27MM6Q9e1EczuXamTdTqRqpXGr6mBg/FFKH9X6vytRzPHXzOpPjnhG08+fCMbn+H nbIHYnx2kszV/yKeP2yBOZTxDiSyK4ChOyxWCa5qrgKWJ5Y/EOCYA2cRE+zr49pSoG caivI5+PHuaibEsxZS51JneVgsfK1kd95uGNL9bOCr/CFzMI4mLH+CpGNrV0KPigVi SnrcccuAOIwHqiaPuftqsDp/yT5zBHccg5g9NpYnL2DV4V/pGXUVz+6jbDlZWRlcSD ytG49RIDcjsdNKZSSHYUbUcVJPpKYu4UlsKxJbDhvegYzES6Q/p From: "brian m. carlson" To: Cc: Junio C Hamano , =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v3 04/44] wrapper: add function to compare strings with different NUL termination Date: Fri, 19 Jun 2020 17:55:21 +0000 Message-Id: <20200619175601.569856-5-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.278.ge193c7cf3a9 In-Reply-To: <20200619175601.569856-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200619175601.569856-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When parsing capabilities for the pack protocol, there are times we'll want to compare the value of a capability to a NUL-terminated string. Since the data we're reading will be space-terminated, not NUL-terminated, we need a function that compares the two strings, but also checks that they're the same length. Otherwise, if we used strncmp to compare these strings, we might accidentally accept a parameter that was a prefix of the expected value. Add a function, xstrncmpz, that takes a NUL-terminated string and a non-NUL-terminated string, plus a length, and compares them, ensuring that they are the same length. Signed-off-by: brian m. carlson --- git-compat-util.h | 6 ++++++ wrapper.c | 8 ++++++++ 2 files changed, 14 insertions(+) diff --git a/git-compat-util.h b/git-compat-util.h index a73632e8e4..5637114b8d 100644 --- a/git-compat-util.h +++ b/git-compat-util.h @@ -868,6 +868,12 @@ char *xgetcwd(void); FILE *fopen_for_writing(const char *path); FILE *fopen_or_warn(const char *path, const char *mode); +/* + * Like strncmp, but only return zero if s is NUL-terminated and exactly len + * characters long. If it is not, consider it greater than t. + */ +int xstrncmpz(const char *s, const char *t, size_t len); + /* * FREE_AND_NULL(ptr) is like free(ptr) followed by ptr = NULL. Note * that ptr is used twice, so don't pass e.g. ptr++. diff --git a/wrapper.c b/wrapper.c index 3a1c0e0526..4ff4a9c3db 100644 --- a/wrapper.c +++ b/wrapper.c @@ -105,6 +105,14 @@ char *xstrndup(const char *str, size_t len) return xmemdupz(str, p ? p - str : len); } +int xstrncmpz(const char *s, const char *t, size_t len) +{ + int res = strncmp(s, t, len); + if (res) + return res; + return s[len] == '\0' ? 0 : 1; +} + void *xrealloc(void *ptr, size_t size) { void *ret; From patchwork Fri Jun 19 17:55:22 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11614843 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id ECEF11731 for ; Fri, 19 Jun 2020 17:57:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D4D3620DD4 for ; Fri, 19 Jun 2020 17:57:19 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="CaMnhLis" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729513AbgFSR5S (ORCPT ); Fri, 19 Jun 2020 13:57:18 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:39634 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2404763AbgFSR4s (ORCPT ); Fri, 19 Jun 2020 13:56:48 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:7d4e:cde:7c41:71c2]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 3DE7360A60; Fri, 19 Jun 2020 17:56:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1592589375; bh=tcIo6T7g/Bgpnie25bur+FqoLMQVSB4Gb07k96qKCxE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=CaMnhLis7nUL1h5dXHGmWKgtYoQks3BWniHNCw20NHbSUAbI0JksrBK4NhwmtzJ4N 4nC6VieRyL+w2GBXwV+0qBld75A4VQE694pTQ/i5qerkrkmw8H4dgPfm1etYl7IGY4 8z77JsLYePdfdiDlVShxT1+nJbvS4wzqE+eYUkPxyPfoPRxYkgSRYiLnlVRAsEq5P2 E01LtsBexfVuyWG/+bJaQOSWVI9CF42FE1YYlqejaM2aIgBuu3xP1ICWJ/9R32OKb4 RPEKQJZA4VpLEkWsALBmdNW26PB6rIoZgTsAbQV4P3xqU6TNCewHJZutHX50NwzIqO MDQ9mFfLMzGYBB7Md62mOFTa4nsxtnDpWM3cEZDTzz8iQmQNIdy69TxdMKQlNzM64y 1+xSM2oKADgCtS8pYo5+to1y88d6ZOiL3+Uvcjs0kXwshH4OwPaXiI1OitB+YTMrLu IRg98OI9EA6n1obmC4Srhr10HlYORIGjkZL1Z6Ou2akfzZ+Je8r From: "brian m. carlson" To: Cc: Junio C Hamano , =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v3 05/44] remote: advertise the object-format capability on the server side Date: Fri, 19 Jun 2020 17:55:22 +0000 Message-Id: <20200619175601.569856-6-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.278.ge193c7cf3a9 In-Reply-To: <20200619175601.569856-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200619175601.569856-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Advertise the current hash algorithm in use by using the object-format capability as part of the ref advertisement. Signed-off-by: brian m. carlson --- builtin/receive-pack.c | 1 + upload-pack.c | 3 ++- 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c index ea3d0f01af..4ffa501dce 100644 --- a/builtin/receive-pack.c +++ b/builtin/receive-pack.c @@ -249,6 +249,7 @@ static void show_ref(const char *path, const struct object_id *oid) strbuf_addf(&cap, " push-cert=%s", push_cert_nonce); if (advertise_push_options) strbuf_addstr(&cap, " push-options"); + strbuf_addf(&cap, " object-format=%s", the_hash_algo->name); strbuf_addf(&cap, " agent=%s", git_user_agent_sanitized()); packet_write_fmt(1, "%s %s%c%s\n", oid_to_hex(oid), path, 0, cap.buf); diff --git a/upload-pack.c b/upload-pack.c index 401c9e6c4b..a72eef9d14 100644 --- a/upload-pack.c +++ b/upload-pack.c @@ -1063,7 +1063,7 @@ static int send_ref(const char *refname, const struct object_id *oid, struct strbuf symref_info = STRBUF_INIT; format_symref_info(&symref_info, &data->symref); - packet_write_fmt(1, "%s %s%c%s%s%s%s%s%s agent=%s\n", + packet_write_fmt(1, "%s %s%c%s%s%s%s%s%s object-format=%s agent=%s\n", oid_to_hex(oid), refname_nons, 0, capabilities, (allow_unadvertised_object_request & ALLOW_TIP_SHA1) ? @@ -1073,6 +1073,7 @@ static int send_ref(const char *refname, const struct object_id *oid, data->stateless_rpc ? " no-done" : "", symref_info.buf, allow_filter ? " filter" : "", + the_hash_algo->name, git_user_agent_sanitized()); strbuf_release(&symref_info); } else { From patchwork Fri Jun 19 17:55:23 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11614845 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A6B47618 for ; Fri, 19 Jun 2020 17:57:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8F44C20DD4 for ; Fri, 19 Jun 2020 17:57:23 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="JBfXJK3q" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2436510AbgFSR5U (ORCPT ); Fri, 19 Jun 2020 13:57:20 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:39636 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2404762AbgFSR4s (ORCPT ); Fri, 19 Jun 2020 13:56:48 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:7d4e:cde:7c41:71c2]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id BA51260A62; Fri, 19 Jun 2020 17:56:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1592589376; bh=+voCdiSirAgtqb9/rDB3ZWxhycsHIe24RZ1Pp/Jw/EQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=JBfXJK3qqAclWDUHqqFn/NI/ZEfe9WBEo7yLOdGeXJmj9SSSoK8FzFriuV9Wcy3DX bC8lltC31mtHmtnlapKxR/voVsu+WUQQ+xnKvgap49yrxAlL9Y1jKDqQsfP3a/d4R6 LTWbfX4ylEy+STMlvDB82M2mhf9P8gAAKj7aABz38rKJhMQw+g7BmJRmM/+2ng4KXa oD52dQ6qh7GT/lbrWBLWA92+wwOeWiBRW0deHKGVHHC6m4b4/D8wSrB/7pf4YG+Hme amh3r0iPgpgVsr5sWG9sDd2AseJseGTfY4gOh1Scoduposb1CyoX1LKZC4ulBwlZBb H2aIilwK2YrkTlZYwKaowTNTZ/Km+VQ92tet3E854RJeUKbFDesNLFEnQYwmJRqmft wtP3KmGclOLtfGcCRY5wR1NZqmI5KjQnwrvIb3YtWFeq30NrOMGdPAM/7cdPQWuFbF KkVsdULdjAWG28WZyzTvzA/t5Vbaj2dcKLmCAwcV5iXYEWt1eNy From: "brian m. carlson" To: Cc: Junio C Hamano , =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v3 06/44] connect: add function to parse multiple v1 capability values Date: Fri, 19 Jun 2020 17:55:23 +0000 Message-Id: <20200619175601.569856-7-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.278.ge193c7cf3a9 In-Reply-To: <20200619175601.569856-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200619175601.569856-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In a capability response, we can have multiple symref entries. In the future, we will also allow for multiple hash algorithms to be specified. To avoid duplication, expand the parse_feature_value function to take an optional offset where the parsing should begin next time. Add a wrapper function that allows us to query the next server feature value, and use it in the existing symref parsing code. Signed-off-by: brian m. carlson --- connect.c | 30 +++++++++++++++++++++--------- 1 file changed, 21 insertions(+), 9 deletions(-) diff --git a/connect.c b/connect.c index e66e779ebd..d30c637ab3 100644 --- a/connect.c +++ b/connect.c @@ -18,7 +18,8 @@ static char *server_capabilities_v1; static struct argv_array server_capabilities_v2 = ARGV_ARRAY_INIT; -static const char *parse_feature_value(const char *, const char *, int *); +static const char *parse_feature_value(const char *, const char *, int *, int *); +static const char *next_server_feature_value(const char *feature, int *len, int *offset); static int check_ref(const char *name, unsigned int flags) { @@ -181,17 +182,16 @@ static void parse_one_symref_info(struct string_list *symref, const char *val, i static void annotate_refs_with_symref_info(struct ref *ref) { struct string_list symref = STRING_LIST_INIT_DUP; - const char *feature_list = server_capabilities_v1; + int offset = 0; - while (feature_list) { + while (1) { int len; const char *val; - val = parse_feature_value(feature_list, "symref", &len); + val = next_server_feature_value("symref", &len, &offset); if (!val) break; parse_one_symref_info(&symref, val, len); - feature_list = val + 1; } string_list_sort(&symref); @@ -468,7 +468,7 @@ struct ref **get_remote_refs(int fd_out, struct packet_reader *reader, return list; } -static const char *parse_feature_value(const char *feature_list, const char *feature, int *lenp) +static const char *parse_feature_value(const char *feature_list, const char *feature, int *lenp, int *offset) { int len; @@ -476,6 +476,8 @@ static const char *parse_feature_value(const char *feature_list, const char *fea return NULL; len = strlen(feature); + if (offset) + feature_list += *offset; while (*feature_list) { const char *found = strstr(feature_list, feature); if (!found) @@ -490,9 +492,14 @@ static const char *parse_feature_value(const char *feature_list, const char *fea } /* feature with a value (e.g., "agent=git/1.2.3") */ else if (*value == '=') { + int end; + value++; + end = strcspn(value, " \t\n"); if (lenp) - *lenp = strcspn(value, " \t\n"); + *lenp = end; + if (offset) + *offset = value + end - feature_list; return value; } /* @@ -507,12 +514,17 @@ static const char *parse_feature_value(const char *feature_list, const char *fea int parse_feature_request(const char *feature_list, const char *feature) { - return !!parse_feature_value(feature_list, feature, NULL); + return !!parse_feature_value(feature_list, feature, NULL, NULL); +} + +static const char *next_server_feature_value(const char *feature, int *len, int *offset) +{ + return parse_feature_value(server_capabilities_v1, feature, len, offset); } const char *server_feature_value(const char *feature, int *len) { - return parse_feature_value(server_capabilities_v1, feature, len); + return parse_feature_value(server_capabilities_v1, feature, len, NULL); } int server_supports(const char *feature) From patchwork Fri Jun 19 17:55:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11614841 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 56FFC13A0 for ; Fri, 19 Jun 2020 17:57:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3D86821527 for ; Fri, 19 Jun 2020 17:57:19 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="w4UZgVoO" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2405302AbgFSR5R (ORCPT ); Fri, 19 Jun 2020 13:57:17 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:39638 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2404770AbgFSR4t (ORCPT ); Fri, 19 Jun 2020 13:56:49 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:7d4e:cde:7c41:71c2]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 4DB4860A63; Fri, 19 Jun 2020 17:56:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1592589376; bh=Vz8VJsqxAFBjylJ4xGpQCBj+IfUz8i1+hfPP9jT/Ca8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=w4UZgVoOOyHaaWW/72SHzXjWTHNpCyhRp0dERfUDtdbgyfUDiKbFyVF6INb/q5oSG Mqn4Wlm6qXt9Ha4+4EJEmizAEJEu54fzFKUx7GCITMy6NaGfaEfoKdj32z9vC2wOlo ruOWKqGrhzny9sjxmrI8WVUSRqBd+XfqsK4NKyH1361QvfavAWgapkNWlChByHUdeT oSQQDeOiLgmS8KfWpbQyBK2h/ITZ6ROLf9KJ/5k2eyVq0xmfQdfImgxA1qXycG5YIy SIarb5HUFue3r0kGp0NX+LhKUbLkCnntTzb9swMM5HrravlLN1I/LD3TcPpf38Vm7T cVlXrv10rxw60fiycW4AQBnJLU9416zZ2zVcxhpcaiWKKdXRgsyCchOilLW/AjmsRH +zZGA60q9YvrBbTKILAi1H/L+LDt+zg1aiXcTQ36AyPWmfVHkoFYLVL1rAw5o3xd5g fJVHSpK6DpwpJOey+0VXY65vL07AE/3+TCmt2MyZ8GkHd1DjjZ+ From: "brian m. carlson" To: Cc: Junio C Hamano , =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v3 07/44] connect: add function to fetch value of a v2 server capability Date: Fri, 19 Jun 2020 17:55:24 +0000 Message-Id: <20200619175601.569856-8-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.278.ge193c7cf3a9 In-Reply-To: <20200619175601.569856-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200619175601.569856-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org So far in protocol v2, all of our server capabilities that have values have not had values that we've been interested in parsing. For example, we receive but ignore the agent value. However, in a future commit, we're going to want to parse out the value of a server capability. To make this easy, add a function, server_feature_v2, that can fetch the value provided as part of the server capability. Signed-off-by: brian m. carlson --- connect.c | 15 +++++++++++++++ connect.h | 1 + 2 files changed, 16 insertions(+) diff --git a/connect.c b/connect.c index d30c637ab3..2cded78b0a 100644 --- a/connect.c +++ b/connect.c @@ -84,6 +84,21 @@ int server_supports_v2(const char *c, int die_on_error) return 0; } +int server_feature_v2(const char *c, const char **v) +{ + int i; + + for (i = 0; i < server_capabilities_v2.argc; i++) { + const char *out; + if (skip_prefix(server_capabilities_v2.argv[i], c, &out) && + (*out == '=')) { + *v = out + 1; + return 1; + } + } + return 0; +} + int server_supports_feature(const char *c, const char *feature, int die_on_error) { diff --git a/connect.h b/connect.h index 235bc66254..88702fdd17 100644 --- a/connect.h +++ b/connect.h @@ -19,6 +19,7 @@ struct packet_reader; enum protocol_version discover_version(struct packet_reader *reader); int server_supports_v2(const char *c, int die_on_error); +int server_feature_v2(const char *c, const char **v); int server_supports_feature(const char *c, const char *feature, int die_on_error); From patchwork Fri Jun 19 17:55:25 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11614881 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9FB82618 for ; Fri, 19 Jun 2020 17:57:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 866D021527 for ; Fri, 19 Jun 2020 17:57:56 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="yacW+pcl" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2393758AbgFSR4Z (ORCPT ); Fri, 19 Jun 2020 13:56:25 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:39446 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390892AbgFSR4S (ORCPT ); Fri, 19 Jun 2020 13:56:18 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:7d4e:cde:7c41:71c2]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 2542460A64; Fri, 19 Jun 2020 17:56:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1592589377; bh=A9L/mi6WLEtEwD2dFxGufSJkHStABYOr6fMktVS0ij8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=yacW+pcljV2Hj/2DGQ5ZAqFmTMxfvLbbDP7dYjrn1Eki304ly+UqHD2DRibx2bKUF 2Q6jGf1qCm24GKXr2oxkd+19MXaZsi9bD2RPA8726JHWWERejMtlN+OmgHjeQHZxxJ 0p8/3SADUEYtBIJY/zJG0VIXoz2dHAvl2HhFd9SBRvNYEYyjRqtChuLZYoI6mP8mgi IL6X/Wwa9KNlm2rkMqG/DWJJTV/ErcPtIpv9MSFXTh3eku5qGD1+X/1B7dUCbk68OM IvKWCl9EQl50ws6z+q7CK5vd/1vyuQuq9ty2Fn6FNybBpBqfxxe5fCoD1hM0v9DaRY zJWSfBLA0BXBfq/yRvcFs8jMmvh3EC2qRdJpMQG4D8HfKhsux/ncJ6y+Qt9ihZycbL sE+QPp1762ue01nrTUkplBACdZTrgULnQMpEWaDRRY9LPcn1zyA5OrNo9Rs7eziONC iWPGZDFE17IfzAWSPJkTC8CML9Avu7Fx/Hx9zv14RuvnYHganIu From: "brian m. carlson" To: Cc: Junio C Hamano , =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v3 08/44] pkt-line: add a member for hash algorithm Date: Fri, 19 Jun 2020 17:55:25 +0000 Message-Id: <20200619175601.569856-9-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.278.ge193c7cf3a9 In-Reply-To: <20200619175601.569856-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200619175601.569856-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Add a member for the hash algorithm currently in use to the packet reader so it can parse references correctly. Signed-off-by: brian m. carlson --- pkt-line.c | 1 + pkt-line.h | 3 +++ 2 files changed, 4 insertions(+) diff --git a/pkt-line.c b/pkt-line.c index 8f9bc68ee2..844c253ccd 100644 --- a/pkt-line.c +++ b/pkt-line.c @@ -490,6 +490,7 @@ void packet_reader_init(struct packet_reader *reader, int fd, reader->buffer_size = sizeof(packet_buffer); reader->options = options; reader->me = "git"; + reader->hash_algo = &hash_algos[GIT_HASH_SHA1]; } enum packet_read_status packet_reader_read(struct packet_reader *reader) diff --git a/pkt-line.h b/pkt-line.h index 5b373fe4cd..8c90daa59e 100644 --- a/pkt-line.h +++ b/pkt-line.h @@ -177,6 +177,9 @@ struct packet_reader { unsigned use_sideband : 1; const char *me; + + /* hash algorithm in use */ + const struct git_hash_algo *hash_algo; }; /* From patchwork Fri Jun 19 17:55:26 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11614837 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3DB4213A0 for ; Fri, 19 Jun 2020 17:57:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 20C1D20776 for ; Fri, 19 Jun 2020 17:57:15 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="kyKHiAfX" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2405263AbgFSR5N (ORCPT ); Fri, 19 Jun 2020 13:57:13 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:39640 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2404806AbgFSR4t (ORCPT ); Fri, 19 Jun 2020 13:56:49 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:7d4e:cde:7c41:71c2]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id A093A60A66; Fri, 19 Jun 2020 17:56:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1592589378; bh=vQNbFM+sOv8/dUxFZz1qhNxgDyUocSrlxBAs+oTyN7Y=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=kyKHiAfXvvkPhDWFy39R5zKIG/Ax5gYrBb3XDMlO4eWrHsr0+4zh1AQsTiKVD2RSk 7YPJzK9YKqLZADwJ4pM/A6dlUQy3tzehv22CY7HKHXHgaBOW0D4mHih/qTT1xVK+dl L+KZRsfCSGkqIN+pwRHe1ezsIDNeF/l08YvHk0hgUhCFbCGLgLEDEgeofTmaNKK+MM kyu5tinvG5tkg574nHazpEV+APL50eZZXHHQhTPvDqf0k2aStO+xLXM1Jnms3R1mJM SRix0WneA7DjoZdkZBnPALph8YJglJLGx2hbSk0nv4TfciB2NFeuM6HoaUOk4c4igL r1P1YXYEUnMINUGFuB1grmNgGx+PpOyqMHb6bYSFq2XdBGCb/cwDiHbI4NyjGlzkIk 4D1Uod19L899Nj9jAdNJVWNCUdrZGrC537inObjzW9PSFdPNfUlJc+ElzfEdN1x4fZ Q3v+gGKEuZD1N+R1+Kmow86GQykgjHP6RXuTThQllPHE3jZwv5M From: "brian m. carlson" To: Cc: Junio C Hamano , =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v3 09/44] transport: add a hash algorithm member Date: Fri, 19 Jun 2020 17:55:26 +0000 Message-Id: <20200619175601.569856-10-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.278.ge193c7cf3a9 In-Reply-To: <20200619175601.569856-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200619175601.569856-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When connecting to a remote system, we need to know what hash algorithm it will be using to talk to us. Add a hash_algo member to struct transport and add a function to read this data from the transport object. Signed-off-by: brian m. carlson --- transport.c | 8 ++++++++ transport.h | 8 ++++++++ 2 files changed, 16 insertions(+) diff --git a/transport.c b/transport.c index 7d50c502ad..a016f41702 100644 --- a/transport.c +++ b/transport.c @@ -312,6 +312,7 @@ static struct ref *handshake(struct transport *transport, int for_push, BUG("unknown protocol version"); } data->got_remote_heads = 1; + transport->hash_algo = reader.hash_algo; if (reader.line_peeked) BUG("buffer must be empty at the end of handshake()"); @@ -988,9 +989,16 @@ struct transport *transport_get(struct remote *remote, const char *url) ret->smart_options->receivepack = remote->receivepack; } + ret->hash_algo = &hash_algos[GIT_HASH_SHA1]; + return ret; } +const struct git_hash_algo *transport_get_hash_algo(struct transport *transport) +{ + return transport->hash_algo; +} + int transport_set_option(struct transport *transport, const char *name, const char *value) { diff --git a/transport.h b/transport.h index 4298c855be..2a9f96c05a 100644 --- a/transport.h +++ b/transport.h @@ -115,6 +115,8 @@ struct transport { struct git_transport_options *smart_options; enum transport_family family; + + const struct git_hash_algo *hash_algo; }; #define TRANSPORT_PUSH_ALL (1<<0) @@ -243,6 +245,12 @@ int transport_push(struct repository *repo, const struct ref *transport_get_remote_refs(struct transport *transport, const struct argv_array *ref_prefixes); +/* + * Fetch the hash algorithm used by a remote. + * + * This can only be called after fetching the remote refs. + */ +const struct git_hash_algo *transport_get_hash_algo(struct transport *transport); int transport_fetch_refs(struct transport *transport, struct ref *refs); void transport_unlock_pack(struct transport *transport); int transport_disconnect(struct transport *transport); From patchwork Fri Jun 19 17:55:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11614801 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 60CA4618 for ; Fri, 19 Jun 2020 17:56:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 471CE21527 for ; Fri, 19 Jun 2020 17:56:26 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="W1coQgEz" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2403903AbgFSR4Z (ORCPT ); Fri, 19 Jun 2020 13:56:25 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:39458 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2391253AbgFSR4V (ORCPT ); Fri, 19 Jun 2020 13:56:21 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:7d4e:cde:7c41:71c2]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 27C5460A67; Fri, 19 Jun 2020 17:56:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1592589378; bh=ORMmguZtrHvCLzay+DZGDcwD5Tww9WG4e4A+bbiby4g=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=W1coQgEzoJjFNYESwXYZCo+eBCt7h4bvNXHNg5bX7kimrDd+EaCa7+1Y0chz9GaP3 8eDfsi5lDL2E3vS91e+pTLn/GoD1HjDD7QPiITiFUTMZ9Yf/vs6jESsK2ZHfQ7o2I3 mB0M+m/9CgTj+L05V642WJLQqQoX3rDP+MtUvTEvHA+VDoma8zcjHdAkPUHkvInDwg RXC/qsuIT2U5RuIxKNMpiNgB656AtXLSNfaC7opbzxBJN+4L51LoSfGcjEHCUSEzYe iPND/MuwA9oxg5/UwkwrZ52ZLs3yRmVSs9QpJVU3hu0+UotU8vglT+adPauWCoyTLa gmJNFNJlnnsKKJGZsjryBrojoK1RU/yRIb0WUTO2SUMP5yvv7ZzQKGDHKGaJJWmJGK R8hIClf/ceNxoPRvdASp7tyGzt64z+a6hubrmRd8Ja8kQcg3nG0wMwZS1ISqqeJHSK RyWy+LT++UQmy7+0QUm95EvgOf9TP54LndgfoMfrhGhGkDn2IHB From: "brian m. carlson" To: Cc: Junio C Hamano , =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v3 10/44] connect: add function to detect supported v1 hash functions Date: Fri, 19 Jun 2020 17:55:27 +0000 Message-Id: <20200619175601.569856-11-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.278.ge193c7cf3a9 In-Reply-To: <20200619175601.569856-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200619175601.569856-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Add a function, server_supports_hash, to see if the remote server supports a particular hash algorithm when speaking protocol v1. Signed-off-by: brian m. carlson --- connect.c | 22 ++++++++++++++++++++++ connect.h | 1 + 2 files changed, 23 insertions(+) diff --git a/connect.c b/connect.c index 2cded78b0a..a52b038865 100644 --- a/connect.c +++ b/connect.c @@ -527,6 +527,28 @@ static const char *parse_feature_value(const char *feature_list, const char *fea return NULL; } +int server_supports_hash(const char *desired, int *feature_supported) +{ + int offset = 0; + int len; + const char *hash; + + hash = next_server_feature_value("object-format", &len, &offset); + if (feature_supported) + *feature_supported = !!hash; + if (!hash) { + hash = hash_algos[GIT_HASH_SHA1].name; + len = strlen(hash); + } + while (hash) { + if (!xstrncmpz(desired, hash, len)) + return 1; + + hash = next_server_feature_value("object-format", &len, &offset); + } + return 0; +} + int parse_feature_request(const char *feature_list, const char *feature) { return !!parse_feature_value(feature_list, feature, NULL, NULL); diff --git a/connect.h b/connect.h index 88702fdd17..c53976f7ec 100644 --- a/connect.h +++ b/connect.h @@ -18,6 +18,7 @@ int url_is_local_not_ssh(const char *url); struct packet_reader; enum protocol_version discover_version(struct packet_reader *reader); +int server_supports_hash(const char *desired, int *feature_supported); int server_supports_v2(const char *c, int die_on_error); int server_feature_v2(const char *c, const char **v); int server_supports_feature(const char *c, const char *feature, From patchwork Fri Jun 19 17:55:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11614873 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 71234618 for ; Fri, 19 Jun 2020 17:57:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4F34521527 for ; Fri, 19 Jun 2020 17:57:44 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="joBQjlY5" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2404229AbgFSR41 (ORCPT ); Fri, 19 Jun 2020 13:56:27 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:39446 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2393707AbgFSR40 (ORCPT ); Fri, 19 Jun 2020 13:56:26 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:7d4e:cde:7c41:71c2]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id A03C160A68; Fri, 19 Jun 2020 17:56:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1592589378; bh=edUdmuqwqgrI4d7hU9gk2tehK/HwKep/ouFUbgKC4Xo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=joBQjlY5agjFH8lEmQiaCo9BWLEGrMz90vxDI3kGsFfqCNeU/i+5p9k+Usdb5Z5jt tpxWPqSS3sAG10QA7xPkwfzQLxcN6VU3jWdo7Rmq7yeBOqPU8XuOp2M45WRbIvGjr8 G0WWc6kRhuTj1k3Yc8L7lihvsaZwUhGiPDiM1Y2GCNn5GhpjUBS1vBz0lJF0amK0ft 0+juaw/xM5oEY825l0uuHFT0Pepd3Bc4yzElyLut59bsfxR09nwqeqWnRkYDXKFLvm NSRlPPKN7N7bEU7+3g27LxSBHRr0ShCKyL+K8sSshVO7vRsyT38YwKOYRFLAuO+2zt PhRuze3lIQoEsvlTP/1VzRAXJ7myZXYJRzbsT7o8iOK6wszXCo+PdZdSpqYmh7e6hu /u360ij7smoC9s2NJ3Dp00LgWBh8LPRzfSZWedjUZNy2dPVDBjLiESfSNN/OCzAmgO wwG6GcYUAu3AvPoHAx9gmpaqKwCHEDj2OrOBdt+N2FgE5SSHtOO From: "brian m. carlson" To: Cc: Junio C Hamano , =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v3 11/44] send-pack: detect when the server doesn't support our hash Date: Fri, 19 Jun 2020 17:55:28 +0000 Message-Id: <20200619175601.569856-12-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.278.ge193c7cf3a9 In-Reply-To: <20200619175601.569856-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200619175601.569856-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Detect when the server doesn't support our hash algorithm and abort. If the server does support our hash, advertise it as part of our capabilities. Signed-off-by: brian m. carlson --- send-pack.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/send-pack.c b/send-pack.c index 0abee22283..02aefcb08e 100644 --- a/send-pack.c +++ b/send-pack.c @@ -363,6 +363,7 @@ int send_pack(struct send_pack_args *args, int atomic_supported = 0; int use_push_options = 0; int push_options_supported = 0; + int object_format_supported = 0; unsigned cmds_sent = 0; int ret; struct async demux; @@ -389,6 +390,9 @@ int send_pack(struct send_pack_args *args, if (server_supports("push-options")) push_options_supported = 1; + if (!server_supports_hash(the_hash_algo->name, &object_format_supported)) + die(_("the receiving end does not support this repository's hash algorithm")); + if (args->push_cert != SEND_PACK_PUSH_CERT_NEVER) { int len; push_cert_nonce = server_feature_value("push-cert", &len); @@ -429,6 +433,8 @@ int send_pack(struct send_pack_args *args, strbuf_addstr(&cap_buf, " atomic"); if (use_push_options) strbuf_addstr(&cap_buf, " push-options"); + if (object_format_supported) + strbuf_addf(&cap_buf, " object-format=%s", the_hash_algo->name); if (agent_supported) strbuf_addf(&cap_buf, " agent=%s", git_user_agent_sanitized()); From patchwork Fri Jun 19 17:55:29 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11614883 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C61FB618 for ; Fri, 19 Jun 2020 17:57:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AD30F21527 for ; Fri, 19 Jun 2020 17:57:57 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="Qd+OUti7" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2436566AbgFSR55 (ORCPT ); Fri, 19 Jun 2020 13:57:57 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:39508 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2393724AbgFSR40 (ORCPT ); Fri, 19 Jun 2020 13:56:26 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:7d4e:cde:7c41:71c2]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 2540260A69; Fri, 19 Jun 2020 17:56:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1592589379; bh=cEB7j441pJ7ZBQHDPn+LHn/tZPNuXvawa00Qmlr9GZs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=Qd+OUti7fdwDcDYJXuVf6MiuNv7GHz45MpwHC2kMlJ+K1reSLzVqyWWomJhi/Aa/t 6PbyqB4IyhM+r9zkpMGXE/dI+WYmutG7MRUgbadvcchvIAMAjJ4u3hSA9QZ3Ipkcnj SJCezvGzd57FD4oFfIZOJTultTqMSqbASvenAbyvjbpp/1UDVTCJ8Hn1v2lCLjNcjH 4yJMC/W3ZgGbttRA0iTwajyIF9jKqwWBocp7bHQWAgpf4psq6kTVnIDSoewMC/DbDU 6XYpJwEjF79/wI1CaTXBG5hcCfehsn6Q77B8XgKs+iPf5CDHs1FmI7C+SGVgNK5ZqA qUDuflTemmULkNsqVh+0OV0rmd4KAbVENmxvwEHN/C8mfMeDUuhTNo02HsyZgtO9yD bTi+n1rq89GeLixa7dCmuccpMRlcmVTOWeEWwruYtpDrf+BMT20fOxeiL0+ilXZ1wi ITxqqFMQrzF4z+xnyQM3uPWV/rHgbipTVzB1FC3/pIUXmz7/2+r From: "brian m. carlson" To: Cc: Junio C Hamano , =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v3 12/44] connect: make parse_feature_value extern Date: Fri, 19 Jun 2020 17:55:29 +0000 Message-Id: <20200619175601.569856-13-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.278.ge193c7cf3a9 In-Reply-To: <20200619175601.569856-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200619175601.569856-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org We're going to be using this function in other files, so no longer mark this function static. Signed-off-by: brian m. carlson --- connect.c | 3 +-- connect.h | 1 + 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/connect.c b/connect.c index a52b038865..8f70a05699 100644 --- a/connect.c +++ b/connect.c @@ -18,7 +18,6 @@ static char *server_capabilities_v1; static struct argv_array server_capabilities_v2 = ARGV_ARRAY_INIT; -static const char *parse_feature_value(const char *, const char *, int *, int *); static const char *next_server_feature_value(const char *feature, int *len, int *offset); static int check_ref(const char *name, unsigned int flags) @@ -483,7 +482,7 @@ struct ref **get_remote_refs(int fd_out, struct packet_reader *reader, return list; } -static const char *parse_feature_value(const char *feature_list, const char *feature, int *lenp, int *offset) +const char *parse_feature_value(const char *feature_list, const char *feature, int *lenp, int *offset) { int len; diff --git a/connect.h b/connect.h index c53976f7ec..c53586e929 100644 --- a/connect.h +++ b/connect.h @@ -19,6 +19,7 @@ struct packet_reader; enum protocol_version discover_version(struct packet_reader *reader); int server_supports_hash(const char *desired, int *feature_supported); +const char *parse_feature_value(const char *feature_list, const char *feature, int *lenp, int *offset); int server_supports_v2(const char *c, int die_on_error); int server_feature_v2(const char *c, const char **v); int server_supports_feature(const char *c, const char *feature, From patchwork Fri Jun 19 17:55:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11614879 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AFF4E618 for ; Fri, 19 Jun 2020 17:57:54 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8DC1921527 for ; Fri, 19 Jun 2020 17:57:54 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="0vkz0LMd" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2404146AbgFSR41 (ORCPT ); Fri, 19 Jun 2020 13:56:27 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:39458 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2393718AbgFSR4Z (ORCPT ); Fri, 19 Jun 2020 13:56:25 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:7d4e:cde:7c41:71c2]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 9E4F560A6A; Fri, 19 Jun 2020 17:56:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1592589380; bh=kxdnbajG1wJnUj09iiv6Qr7Gsgj4V99GqZeQ8laS6MM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=0vkz0LMdUX9LyCH91NFRQdM3NpDyreHoG88NjEImj8xHxh1ISnRquS0RITPE+TKtI fSGfKW887Ktx7PO+zEVNTQxELN5Kf/NvSoXfom6guj7/QVvrayB9Tb0dsLV4lAycWD n1MMF9t3wwto7PFJEjSXxHke/8nZjRt02PKxNHy2rBnKbq9+wOvEo27Gml9Z+PWgik z0qRfcRd3d2CaCFFYA9WFOf6tx7aryDHZWPptnBpWNM54ryi05uFJXxucV5IMvPz74 iUQGq/UU4NlYhm85RqGFReAHb+BrIP4Ylqq5QHNxCKvGy8Vkj9xmF/ZfkOaZOURsJn BC5yDjyx3SJZzVlh0aUpn4u5AEFSvRPnwq3xSwUHmXO+8M2jSnMKyitO91YGlAbBTf BNQRhr1m0Dg7ESZW4RSCzma3puP1Huik5/S2yrfbnJkQgnaRzpoMTvgNYtUN9LoomT l4Fp2X+alVkhpFqv7O+eaJMYC5jPQkOvXYwAVThD7uiUfrT8OC5 From: "brian m. carlson" To: Cc: Junio C Hamano , =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v3 13/44] fetch-pack: detect when the server doesn't support our hash Date: Fri, 19 Jun 2020 17:55:30 +0000 Message-Id: <20200619175601.569856-14-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.278.ge193c7cf3a9 In-Reply-To: <20200619175601.569856-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200619175601.569856-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Detect when the server doesn't support our hash algorithm and abort. Signed-off-by: brian m. carlson --- fetch-pack.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/fetch-pack.c b/fetch-pack.c index d8bbf45ee2..c090030680 100644 --- a/fetch-pack.c +++ b/fetch-pack.c @@ -1040,6 +1040,8 @@ static struct ref *do_fetch_pack(struct fetch_pack_args *args, print_verbose(args, _("Server supports %s"), "deepen-relative"); else if (args->deepen_relative) die(_("Server does not support --deepen")); + if (!server_supports_hash(the_hash_algo->name, NULL)) + die(_("Server does not support this repository's object format")); if (!args->no_dependents) { mark_complete_and_common_ref(negotiator, args, &ref); From patchwork Fri Jun 19 17:55:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11614839 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 62684618 for ; Fri, 19 Jun 2020 17:57:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4AA1320DD4 for ; Fri, 19 Jun 2020 17:57:18 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="B2/E94yv" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2405240AbgFSR5N (ORCPT ); Fri, 19 Jun 2020 13:57:13 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:39652 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726838AbgFSR45 (ORCPT ); Fri, 19 Jun 2020 13:56:57 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:7d4e:cde:7c41:71c2]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 30E2060A6B; Fri, 19 Jun 2020 17:56:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1592589380; bh=lgxPzgzAwkZLununwCcPrRYoAXzLeH7+ltMQ+PAhsc8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=B2/E94yvkbc4Fb0bCF4UypBGEf6Lhj4HT0bP7DEqpqfKMK9UnWrUsucXFNkDbA0PP jOp/3r5mk5HMEzYSbBElAYol3F+4QsJ6ITZP26pATwm6S+GTTLgejICpFwAQ9RRVJ5 /N3xBXT8NntRytwrL6epSEy1zLPQlzw03T5we645dmJqqPQR3LLOVFvPz8YXVDTFz7 angFfUKnq04LP4yhxmrWNpkveVJsA7+t1AkaTDOpCvM1nmGPrOLFg8HxSfYHxGHctt ElHYRiEViqWrCl2DjK9MG7x59TGoFbUmvppifwF2uRannlBdRdU+Za2yOVLUznZkZn 899GQ8Tg+w6o8ueG8q8zugW1353FhETpXRxSSvp2SSnrWxF677D7w/3pHqUqQrgnad qTCqQ7+kNV8uZRl+0Pyk7Gh9+11V1SXT9DvGVzP6XqtXz9Dq/8MECtl6L3hJt0BnET l5ndn7NWYk8x+n1DDKwK9lyqU+vzE5i43l7yABjHMxxTEkD+aBF From: "brian m. carlson" To: Cc: Junio C Hamano , =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v3 14/44] connect: detect algorithm when fetching refs Date: Fri, 19 Jun 2020 17:55:31 +0000 Message-Id: <20200619175601.569856-15-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.278.ge193c7cf3a9 In-Reply-To: <20200619175601.569856-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200619175601.569856-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org If we're fetching refs, detect the hash algorithm and parse the refs using that algorithm. As mentioned in the documentation, if multiple versions of the object-format capability are provided, we use the first. No known implementation supports multiple algorithms now, but they may in the future. Signed-off-by: brian m. carlson --- connect.c | 21 +++++++++++++++++---- 1 file changed, 17 insertions(+), 4 deletions(-) diff --git a/connect.c b/connect.c index 8f70a05699..b6e110cb24 100644 --- a/connect.c +++ b/connect.c @@ -221,12 +221,25 @@ static void annotate_refs_with_symref_info(struct ref *ref) static void process_capabilities(struct packet_reader *reader, int *linelen) { + const char *feat_val; + int feat_len; const char *line = reader->line; int nul_location = strlen(line); if (nul_location == *linelen) return; server_capabilities_v1 = xstrdup(line + nul_location + 1); *linelen = nul_location; + + feat_val = server_feature_value("object-format", &feat_len); + if (feat_val) { + char *hash_name = xstrndup(feat_val, feat_len); + int hash_algo = hash_algo_by_name(hash_name); + if (hash_algo != GIT_HASH_UNKNOWN) + reader->hash_algo = &hash_algos[hash_algo]; + free(hash_name); + } else { + reader->hash_algo = &hash_algos[GIT_HASH_SHA1]; + } } static int process_dummy_ref(const struct packet_reader *reader) @@ -235,7 +248,7 @@ static int process_dummy_ref(const struct packet_reader *reader) struct object_id oid; const char *name; - if (parse_oid_hex(line, &oid, &name)) + if (parse_oid_hex_algop(line, &oid, &name, reader->hash_algo)) return 0; if (*name != ' ') return 0; @@ -259,7 +272,7 @@ static int process_ref(const struct packet_reader *reader, int len, struct object_id old_oid; const char *name; - if (parse_oid_hex(line, &old_oid, &name)) + if (parse_oid_hex_algop(line, &old_oid, &name, reader->hash_algo)) return 0; if (*name != ' ') return 0; @@ -271,7 +284,7 @@ static int process_ref(const struct packet_reader *reader, int len, die(_("protocol error: unexpected capabilities^{}")); } else if (check_ref(name, flags)) { struct ref *ref = alloc_ref(name); - oidcpy(&ref->old_oid, &old_oid); + memcpy(ref->old_oid.hash, old_oid.hash, reader->hash_algo->rawsz); **list = ref; *list = &ref->next; } @@ -289,7 +302,7 @@ static int process_shallow(const struct packet_reader *reader, int len, if (!skip_prefix(line, "shallow ", &arg)) return 0; - if (get_oid_hex(arg, &old_oid)) + if (get_oid_hex_algop(arg, &old_oid, reader->hash_algo)) die(_("protocol error: expected shallow sha-1, got '%s'"), arg); if (!shallow_points) die(_("repository on the other end cannot be shallow")); From patchwork Fri Jun 19 17:55:32 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11614865 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8BF84618 for ; Fri, 19 Jun 2020 17:57:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 739A920DD4 for ; Fri, 19 Jun 2020 17:57:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="q1mF++TP" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2404156AbgFSR4c (ORCPT ); Fri, 19 Jun 2020 13:56:32 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:39458 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2393787AbgFSR41 (ORCPT ); Fri, 19 Jun 2020 13:56:27 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:7d4e:cde:7c41:71c2]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id AECD360A6C; Fri, 19 Jun 2020 17:56:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1592589381; bh=smmJElpsjDUJ3khU5WIlomu9EU1J8x04QmZc/+KQYhc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=q1mF++TPP+JMA/Fbbir+ickZxheOH9jo8UXyHHjlARW1+rqv6fwBASpLHMrgQWZOV Ly5ts6Cl/kuaOiMoQOJnAYGCPYv/+Zaw5Fpil+v42TnEprf7RzFyhv1ycO1su3wqZa kQ58jYHHn2V01ESZK9CgEJWb3fj57uGLo7lpJzgkji+v0A+/HDInNKdhtrel9oSykh E36x+Ogp2Ys4zAvSb5FjNhP8LVOM08+720FJhyAmc9krViafuf4x8+0KESiEbhdJXZ zxO11DSKGbaH8tkjWRUsP9/EAb92/B8dXFWLXeTavfSM0Az4iUp0FTffW93PV0fPkx ElFWELdD9iJkOPYivdwuFssIY9nWetFQZBNRpQhpOzO1ZMemWr/Nv1bfKIQJ+jPAhe QR2fUy7dXYh0ExjfTd9piDXVSH1tSbDBI3xYLlGBZz2uHk0D2+jxOLyngscrR8drBB eNJADezk5VEcpN+3Eye7AWs8vbp9ADKUt1aIkh49/BNN5RMTfap From: "brian m. carlson" To: Cc: Junio C Hamano , =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v3 15/44] builtin/receive-pack: detect when the server doesn't support our hash Date: Fri, 19 Jun 2020 17:55:32 +0000 Message-Id: <20200619175601.569856-16-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.278.ge193c7cf3a9 In-Reply-To: <20200619175601.569856-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200619175601.569856-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Detect when the server doesn't support our hash algorithm and abort. Signed-off-by: brian m. carlson --- builtin/receive-pack.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c index 4ffa501dce..d43663bb0a 100644 --- a/builtin/receive-pack.c +++ b/builtin/receive-pack.c @@ -1625,6 +1625,8 @@ static struct command *read_head_info(struct packet_reader *reader, linelen = strlen(reader->line); if (linelen < reader->pktlen) { const char *feature_list = reader->line + linelen + 1; + const char *hash = NULL; + int len = 0; if (parse_feature_request(feature_list, "report-status")) report_status = 1; if (parse_feature_request(feature_list, "side-band-64k")) @@ -1637,6 +1639,13 @@ static struct command *read_head_info(struct packet_reader *reader, if (advertise_push_options && parse_feature_request(feature_list, "push-options")) use_push_options = 1; + hash = parse_feature_value(feature_list, "object-format", &len, NULL); + if (!hash) { + hash = hash_algos[GIT_HASH_SHA1].name; + len = strlen(hash); + } + if (xstrncmpz(the_hash_algo->name, hash, len)) + die("error: unsupported object format '%s'", hash); } if (!strcmp(reader->line, "push-cert")) { From patchwork Fri Jun 19 17:55:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11614835 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 86786618 for ; Fri, 19 Jun 2020 17:57:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6E44020DD4 for ; Fri, 19 Jun 2020 17:57:06 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="Z5NM84vT" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2405187AbgFSR5F (ORCPT ); Fri, 19 Jun 2020 13:57:05 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:39658 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2404868AbgFSR46 (ORCPT ); Fri, 19 Jun 2020 13:56:58 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:7d4e:cde:7c41:71c2]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 34FD360A5C; Fri, 19 Jun 2020 17:56:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1592589381; bh=WY3PBUcXZxcwrx1+8moTMqCV5IR1KmrTN41x3B0zcEY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=Z5NM84vTL6awN91aymY0dbhXV9eZutiQfrT5ytGvLxDoYsLNAd1Ajb/WEsfTh1KxH 7uJeMBkn/E36FjiKNaHluGcQPEGqAhG5KG+n9zl//Q2oD6TBTWqRNH6FglH8bayIyN K9Yr03StnX70O7258iNzAoV60FNRNFRa2mcWLsES8akIMs1UuesW+SAZH0++UmOxAB Gs9/SzoeHwrOl59kHrWHDxo3/xMeHaMhcArZDAHdFQ/b8ZV4HAjes3BG9PuziUV+TE 0vVTPfbFeBMcY9ZLMdhWsdzhBND2UT8eJuSJ41Nu4ok85syXPPwqQepYcpRJjLH63/ hQqiQDyovsMiaCQmQAan+UY51YV6xPs3zoinE4q2HDR/sDrO5h/uq9X/q02wwB+y5W C3un+8PdTkGiCUQrEK5TWW7+Ru/qdjHflWOdMVK00/h+cVleflmS3y8QbxXKDS4MJs ST/XpWP/u6xWgPYCOWw7USbqBN3N8VQQ7IkVoArhhT2hzREmSCH From: "brian m. carlson" To: Cc: Junio C Hamano , =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v3 16/44] docs: update remote helper docs for object-format extensions Date: Fri, 19 Jun 2020 17:55:33 +0000 Message-Id: <20200619175601.569856-17-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.278.ge193c7cf3a9 In-Reply-To: <20200619175601.569856-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200619175601.569856-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Update the remote helper docs to document the object-format extensions we will implement in remote-curl and the transport helper code shortly. Signed-off-by: brian m. carlson --- Documentation/gitremote-helpers.txt | 33 +++++++++++++++++++++++++---- 1 file changed, 29 insertions(+), 4 deletions(-) diff --git a/Documentation/gitremote-helpers.txt b/Documentation/gitremote-helpers.txt index 93baeeb029..6f1e269ae4 100644 --- a/Documentation/gitremote-helpers.txt +++ b/Documentation/gitremote-helpers.txt @@ -238,6 +238,9 @@ the remote repository. `--signed-tags=verbatim` to linkgit:git-fast-export[1]. In the absence of this capability, Git will use `--signed-tags=warn-strip`. +'object-format':: + This indicates that the helper is able to interact with the remote + side using an explicit hash algorithm extension. COMMANDS @@ -257,12 +260,14 @@ Support for this command is mandatory. 'list':: Lists the refs, one per line, in the format " [ ...]". The value may be a hex sha1 hash, "@" for - a symref, or "?" to indicate that the helper could not get the - value of the ref. A space-separated list of attributes follows - the name; unrecognized attributes are ignored. The list ends - with a blank line. + a symref, ": " for a key-value pair, or + "?" to indicate that the helper could not get the value of the + ref. A space-separated list of attributes follows the name; + unrecognized attributes are ignored. The list ends with a + blank line. + See REF LIST ATTRIBUTES for a list of currently defined attributes. +See REF LIST KEYWORDS for a list of currently defined keywords. + Supported if the helper has the "fetch" or "import" capability. @@ -432,6 +437,18 @@ attributes are defined. This ref is unchanged since the last import or fetch, although the helper cannot necessarily determine what value that produced. +REF LIST KEYWORDS +----------------- + +The 'list' command may produce a list of key-value pairs. +The following keys are defined. + +'object-format':: + The refs are using the given hash algorithm. This keyword is only + used if the server and client both support the object-format + extension. + + OPTIONS ------- @@ -516,6 +533,14 @@ set by Git if the remote helper has the 'option' capability. transaction. If successful, all refs will be updated, or none will. If the remote side does not support this capability, the push will fail. +'option object-format' {'true'|algorithm}:: + If 'true', indicate that the caller wants hash algorithm information + to be passed back from the remote. This mode is used when fetching + refs. ++ +If set to an algorithm, indicate that the caller wants to interact with +the remote side using that algorithm. + SEE ALSO -------- linkgit:git-remote[1] From patchwork Fri Jun 19 17:55:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11614803 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C265C618 for ; Fri, 19 Jun 2020 17:56:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AA7E620DD4 for ; Fri, 19 Jun 2020 17:56:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="Lj8UJAX+" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2404547AbgFSR4f (ORCPT ); Fri, 19 Jun 2020 13:56:35 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:39446 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2393840AbgFSR41 (ORCPT ); Fri, 19 Jun 2020 13:56:27 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:7d4e:cde:7c41:71c2]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id ADCB960A6D; Fri, 19 Jun 2020 17:56:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1592589382; bh=z/hbO9Pa9kw3i3ydW7bPReUQR2B2iix4X60keH+Bypw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=Lj8UJAX++WYB0FVPztFOWzVWb/VOYhNgu5InsfA7cdf4ayulYTp/evDwsV8ZAeOEi ownh8WtGba34MJZhNlG3Z62NUTn5STvP+nC4GtDS3vIJr9Xt9arwXGHjvEA/lG+1SH 18+tdxt3EjrO7rM3k6hmmicQz1DOdVQVaQiolOhjj7KV1TAj1ywfch16SLfBhzmWDt C0FcJbWRmjX51bZg8IdoOhCGn4dfxJGq2fu/wOjU/cPnCjMQJiZmO7Yd8w7VCUBoLA UK2Sddv0XTQoQ37KQ32AWoKLwn5VYeBjT2gLjb9mJBx4LaAByyCh7XgycOIAQLiq3A ZuJ9+B/ofcm8OvhiR8olLiw5YcNkdCl1uK5eUS5RxcLZ6lyOo/UPVYntOFMmIBnIuW DUe4ug4v7Ymqs/Y4qNdcjp0cwqVKOo8WVc8yUtavYlfSBbBMh/JoG/rfYELZYo8JJG jDf47Y3utgn/BwKDciG7JXke9Ga21poz5n+421gq7M21FQel8PZ From: "brian m. carlson" To: Cc: Junio C Hamano , =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v3 17/44] transport-helper: implement object-format extensions Date: Fri, 19 Jun 2020 17:55:34 +0000 Message-Id: <20200619175601.569856-18-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.278.ge193c7cf3a9 In-Reply-To: <20200619175601.569856-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200619175601.569856-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Implement the object-format extensions that let us determine the hash algorithm in use when pushing or pulling data. Signed-off-by: brian m. carlson --- transport-helper.c | 24 ++++++++++++++++++++++-- 1 file changed, 22 insertions(+), 2 deletions(-) diff --git a/transport-helper.c b/transport-helper.c index a46afcb69d..ae33b0eea7 100644 --- a/transport-helper.c +++ b/transport-helper.c @@ -32,7 +32,8 @@ struct helper_data { signed_tags : 1, check_connectivity : 1, no_disconnect_req : 1, - no_private_update : 1; + no_private_update : 1, + object_format : 1; /* * As an optimization, the transport code may invoke fetch before @@ -207,6 +208,8 @@ static struct child_process *get_helper(struct transport *transport) data->import_marks = xstrdup(arg); } else if (starts_with(capname, "no-private-update")) { data->no_private_update = 1; + } else if (starts_with(capname, "object-format")) { + data->object_format = 1; } else if (mandatory) { die(_("unknown mandatory capability %s; this remote " "helper probably needs newer version of Git"), @@ -1103,6 +1106,12 @@ static struct ref *get_refs_list_using_list(struct transport *transport, data->get_refs_list_called = 1; helper = get_helper(transport); + if (data->object_format) { + write_str_in_full(helper->in, "option object-format\n"); + if (recvline(data, &buf) || strcmp(buf.buf, "ok")) + exit(128); + } + if (data->push && for_push) write_str_in_full(helper->in, "list for-push\n"); else @@ -1115,6 +1124,17 @@ static struct ref *get_refs_list_using_list(struct transport *transport, if (!*buf.buf) break; + else if (buf.buf[0] == ':') { + const char *value; + if (skip_prefix(buf.buf, ":object-format ", &value)) { + int algo = hash_algo_by_name(value); + if (algo == GIT_HASH_UNKNOWN) + die(_("unsupported object format '%s'"), + value); + transport->hash_algo = &hash_algos[algo]; + } + continue; + } eov = strchr(buf.buf, ' '); if (!eov) @@ -1127,7 +1147,7 @@ static struct ref *get_refs_list_using_list(struct transport *transport, if (buf.buf[0] == '@') (*tail)->symref = xstrdup(buf.buf + 1); else if (buf.buf[0] != '?') - get_oid_hex(buf.buf, &(*tail)->old_oid); + get_oid_hex_algop(buf.buf, &(*tail)->old_oid, transport->hash_algo); if (eon) { if (has_attribute(eon + 1, "unchanged")) { (*tail)->status |= REF_STATUS_UPTODATE; From patchwork Fri Jun 19 17:55:35 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11614867 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 68527618 for ; Fri, 19 Jun 2020 17:57:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4F75621527 for ; Fri, 19 Jun 2020 17:57:41 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="ZOwvPRDb" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2393707AbgFSR4b (ORCPT ); Fri, 19 Jun 2020 13:56:31 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:39524 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2404156AbgFSR41 (ORCPT ); Fri, 19 Jun 2020 13:56:27 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:7d4e:cde:7c41:71c2]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 36C6C60A6E; Fri, 19 Jun 2020 17:56:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1592589382; bh=ZGO48bjiov/t+FoF2DaOJBY6c+OxY4OgRGJB75P8hzc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=ZOwvPRDbnkwXH2v4sx7ftnrcVL6ZhDwvyjOBDurwsW4SkDjRpJ36Hi9TZ06ZTtFTB P21k6THDgI/OCbidm+NLYVcZ3EEfnKRjJDqzD4JX9Zy3F08fq2BtlgPShappFsx6LO Sjgct77xtsM71oMQB+2BVkceLPAOGGGApNtGQEvnzBUSpHbOgrFZRRs6rLamtKt1PJ jddQJqBmVwBKOjlCyGdomP18YXkXul1iftml8yfRAU4owI7B5sHtr35A+LCR9RT2xK Po7F0+0KhjBDhIywpc/IzokO9Iq1Ww6FNfFWDolm/JkUQjRCOVjl4TYwNsvvnt7Mo/ /t57FlZfSpFF050YqGED9bvX6u1Vq0KsTFvNdqkyuTOzKH2qbD8NZZHFaSgAAZf9xg JD2jyGnyqt0s+pMh3tlg4IVXEevqWCUrwkrrCeDz3cyWx7Gapuuj+I5XxkDnU39dJP GxQnhL58VeTGzOhiMVrMk/BYTZa9OrySJsENemxvpyavv79JNUD From: "brian m. carlson" To: Cc: Junio C Hamano , =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v3 18/44] remote-curl: implement object-format extensions Date: Fri, 19 Jun 2020 17:55:35 +0000 Message-Id: <20200619175601.569856-19-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.278.ge193c7cf3a9 In-Reply-To: <20200619175601.569856-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200619175601.569856-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Implement the object-format extensions that let us determine the hash algorithm in use when pushing, pulling, and fetching. Signed-off-by: brian m. carlson --- remote-curl.c | 19 ++++++++++++++++++- 1 file changed, 18 insertions(+), 1 deletion(-) diff --git a/remote-curl.c b/remote-curl.c index 75532a8bae..f0203547c5 100644 --- a/remote-curl.c +++ b/remote-curl.c @@ -41,7 +41,9 @@ struct options { deepen_relative : 1, from_promisor : 1, no_dependents : 1, - atomic : 1; + atomic : 1, + object_format : 1; + const struct git_hash_algo *hash_algo; }; static struct options options; static struct string_list cas_options = STRING_LIST_INIT_DUP; @@ -190,6 +192,16 @@ static int set_option(const char *name, const char *value) } else if (!strcmp(name, "filter")) { options.filter = xstrdup(value); return 0; + } else if (!strcmp(name, "object-format")) { + int algo; + options.object_format = 1; + if (strcmp(value, "true")) { + algo = hash_algo_by_name(value); + if (algo == GIT_HASH_UNKNOWN) + die("unknown object format '%s'", value); + options.hash_algo = &hash_algos[algo]; + } + return 0; } else { return 1 /* unsupported */; } @@ -231,6 +243,7 @@ static struct ref *parse_git_refs(struct discovery *heads, int for_push) case protocol_v0: get_remote_heads(&reader, &list, for_push ? REF_NORMAL : 0, NULL, &heads->shallow); + options.hash_algo = reader.hash_algo; break; case protocol_unknown_version: BUG("unknown protocol version"); @@ -509,6 +522,9 @@ static struct ref *get_refs(int for_push) static void output_refs(struct ref *refs) { struct ref *posn; + if (options.object_format && options.hash_algo) { + printf(":object-format %s\n", options.hash_algo->name); + } for (posn = refs; posn; posn = posn->next) { if (posn->symref) printf("@%s %s\n", posn->symref, posn->name); @@ -1499,6 +1515,7 @@ int cmd_main(int argc, const char **argv) printf("option\n"); printf("push\n"); printf("check-connectivity\n"); + printf("object-format\n"); printf("\n"); fflush(stdout); } else if (skip_prefix(buf.buf, "stateless-connect ", &arg)) { From patchwork Fri Jun 19 17:55:36 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11614877 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0052B618 for ; Fri, 19 Jun 2020 17:57:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D337621527 for ; Fri, 19 Jun 2020 17:57:48 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="g4NgoqiP" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2404059AbgFSR5q (ORCPT ); Fri, 19 Jun 2020 13:57:46 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:39508 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2404163AbgFSR42 (ORCPT ); Fri, 19 Jun 2020 13:56:28 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:7d4e:cde:7c41:71c2]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id AF5ED60A6F; Fri, 19 Jun 2020 17:56:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1592589383; bh=C0FilukGeoJOoC7fEq9AzoHQHWIHaLoxTcro18B7DmU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=g4NgoqiPijwSVRSK7ou1FLZ+qGtn+lXfyN/7abSM2OiIXZUpEQ0TWMWmn1SIlhDb7 qvHbJMdHrNdB2lg7CjinCX6+TZ8WCMxxlQvg4JfETrYvpbNmGfL5OspDYfhYptDg5T ITallqmExo+ZFAgOjbTrGvvGutvss5osiLSAgUO9Km/0XgzqtyfeHHbcB3TN70UT2R +Q7kbrhlGcD9av0dkpqbMtr6lgAbRxdCzuH44eV97roTRWg8NL5R7V/abJQI7QHIbQ qdNFXFbVEzOp1ybWRJYPN8aK6EKzJh4g3mCT94D+GMtvWJ1DE7DtiWkwmT35oUQKG3 9+BCPkVDG6GeA657kie4cA+tqItfevU2/NyBanH5sq45u7amCiWIz/ELbeJ2jNmx4o S50dY9NRRdZ+vKpymAtyi8bY7xXpn4E9Ttkh3dzFgyOWqBhZCClyvpMykTuOxPQNKD A2zrfErf4bfX5icy/sHxwhS7uX8L67pxNa4Xir0Muc5IPoiEMWC From: "brian m. carlson" To: Cc: Junio C Hamano , =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v3 19/44] builtin/clone: initialize hash algorithm properly Date: Fri, 19 Jun 2020 17:55:36 +0000 Message-Id: <20200619175601.569856-20-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.278.ge193c7cf3a9 In-Reply-To: <20200619175601.569856-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200619175601.569856-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When performing a clone, we don't know what hash algorithm the other end will support. Currently, we don't support fetching data belonging to a different algorithm, so we must know what algorithm the remote side is using in order to properly initialize the repository. We can know that only after fetching the refs, so if the remote side has any references, use that information to reinitialize the repository with the correct hash algorithm information. Signed-off-by: brian m. carlson --- builtin/clone.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/builtin/clone.c b/builtin/clone.c index 2a8e3aaaed..e3519a8355 100644 --- a/builtin/clone.c +++ b/builtin/clone.c @@ -1220,6 +1220,15 @@ int cmd_clone(int argc, const char **argv, const char *prefix) refs = transport_get_remote_refs(transport, &ref_prefixes); if (refs) { + int hash_algo = hash_algo_by_ptr(transport_get_hash_algo(transport)); + + /* + * Now that we know what algorithm the remote side is using, + * let's set ours to the same thing. + */ + initialize_repository_version(hash_algo); + repo_set_hash_algo(the_repository, hash_algo); + mapped_refs = wanted_peer_refs(refs, &remote->fetch); /* * transport_get_remote_refs() may return refs with null sha-1 From patchwork Fri Jun 19 17:55:37 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11614875 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CDD8C618 for ; Fri, 19 Jun 2020 17:57:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B19BB20DD4 for ; Fri, 19 Jun 2020 17:57:45 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="kqHA1Bjb" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2394668AbgFSR5o (ORCPT ); Fri, 19 Jun 2020 13:57:44 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:39526 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2404207AbgFSR42 (ORCPT ); Fri, 19 Jun 2020 13:56:28 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:7d4e:cde:7c41:71c2]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 34B7260A70; Fri, 19 Jun 2020 17:56:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1592589383; bh=YhcqSF7NVvsfDzpsd5p3oihZbPBd50K1aBOeC4FnGdY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=kqHA1BjbFQn+p0oWMsXmPnOzPX0mRq4ZG2o+2MgYSwNrF8HI+oP9a4HIF/dq84dOj QP7bkz6uuNWVtUoRgJ4qcCPDUutHKBPy6F34Vl180/nQmkZHx0Hvyeqx8R0PfsILXL JeWljfwgthobGPBG+V7OFS5EBVn3p43O5pKKo5kDBqn3FZoQpufgacr0sOMxyNbM1e ptYRJiZICMolaph1R6ttRtnYt+fUi1kfgv2bvuJkNtVSA79n6CXPVkV4QfaCNI6fsU C5QszsN3BmGjlSTpWYETmb4wGWryhZguYo5PNVTbJVyKOpmYXbThZs+NF4OyJ00emf ae14lYvlRs/Kn7nwcpJJe4OADH3B5GSNBGlwIAflzpzuIJYqPqd/8rQpipD2XvL1m6 HBkuc/OqMEfIgi9w651TZm2zWpwN33j2KDK/D6TIGTLrx8IIrj6GTZlnNYfHl7FMcL 5uftcBEl5iFye/HJ1hTnuC+OHRrGbrc8YWXI/1jrq/LFms/gfIg From: "brian m. carlson" To: Cc: Junio C Hamano , =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v3 20/44] t5562: pass object-format in synthesized test data Date: Fri, 19 Jun 2020 17:55:37 +0000 Message-Id: <20200619175601.569856-21-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.278.ge193c7cf3a9 In-Reply-To: <20200619175601.569856-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200619175601.569856-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Ensure that we pass the object-format capability in the synthesized test data so that this test works with algorithms other than SHA-1. In addition, add an additional test using the old data for when we're using SHA-1 so that we can be sure that we preserve backwards compatibility with servers not offering the object-format capability. Signed-off-by: brian m. carlson --- t/t5562-http-backend-content-length.sh | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/t/t5562-http-backend-content-length.sh b/t/t5562-http-backend-content-length.sh index 3f4ac71f83..c6ec625497 100755 --- a/t/t5562-http-backend-content-length.sh +++ b/t/t5562-http-backend-content-length.sh @@ -46,6 +46,7 @@ ssize_b100dots() { } test_expect_success 'setup' ' + test_oid_init && HTTP_CONTENT_ENCODING="identity" && export HTTP_CONTENT_ENCODING && git config http.receivepack true && @@ -62,8 +63,8 @@ test_expect_success 'setup' ' test_copy_bytes 10 fetch_body.trunc && hash_next=$(git commit-tree -p HEAD -m next HEAD^{tree}) && { - printf "%s %s refs/heads/newbranch\\0report-status\\n" \ - "$ZERO_OID" "$hash_next" | packetize && + printf "%s %s refs/heads/newbranch\\0report-status object-format=%s\\n" \ + "$ZERO_OID" "$hash_next" "$(test_oid algo)" | packetize && printf 0000 && echo "$hash_next" | git pack-objects --stdout } >push_body && From patchwork Fri Jun 19 17:55:38 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11614807 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6DF6213A0 for ; Fri, 19 Jun 2020 17:56:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4415120786 for ; Fri, 19 Jun 2020 17:56:41 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="pT0ZQv9g" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2394079AbgFSR4k (ORCPT ); Fri, 19 Jun 2020 13:56:40 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:39552 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2404378AbgFSR4c (ORCPT ); Fri, 19 Jun 2020 13:56:32 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:7d4e:cde:7c41:71c2]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id C1B7260A71; Fri, 19 Jun 2020 17:56:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1592589384; bh=c1M2lsVO9QupQR/ILz768yfVAsI0fSyuIBI+dEuXceU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=pT0ZQv9ga1Rq0v3Zw2QdKxjHitjd716UNJVN1yN9Ij2HyEGLyoUp5caONFA9KgDJ9 2/rbsuGopRzte8aqREwQuf2hE6t6s8fzX1Qq6+oCwsa4BanLNhMVZXPsguOhUmJSOI jsDFQA0C5ATP5l6jY20AhhKTbc9QLugJ3auhoYqZvZXkEJ6fWubpWYCARgBaK5g/sY HLh1HUmpAynkosN0kxUZeTtv31tEHf1tNGocUWJjk94ytNsB+lD6+3ilu7TS7eJ8Z0 dP/LN5TNWn0ibUZSRkFbHJQUSSLrxvB9UDHqvb2UE8K/ZVjCxuUaNq1VP4vJwB+lSd anxUpx55GCckhDRIu3sEodivq1ZGgjwfVV8FtfJC4MNoVgcZKTRpt5bP98JqXwxDWO qVw7Dhyvvbq3ef0j1JydARSBBhPMjhd93DhG4/GzTlVOnphBeKKZxUi3Kp2K91bxoK EXZ5b5MPRJhce0ehJSljYzQLctlB+U0hVGb1AGAIXmbCrtlK7Yf From: "brian m. carlson" To: Cc: Junio C Hamano , =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v3 21/44] fetch-pack: parse and advertise the object-format capability Date: Fri, 19 Jun 2020 17:55:38 +0000 Message-Id: <20200619175601.569856-22-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.278.ge193c7cf3a9 In-Reply-To: <20200619175601.569856-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200619175601.569856-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Parse the server's object-format capability and respond accordingly, dying if there is a mismatch. Signed-off-by: brian m. carlson --- fetch-pack.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/fetch-pack.c b/fetch-pack.c index c090030680..7e58f295f5 100644 --- a/fetch-pack.c +++ b/fetch-pack.c @@ -1180,6 +1180,7 @@ static int send_fetch_request(struct fetch_negotiator *negotiator, int fd_out, int sideband_all, int seen_ack) { int ret = 0; + const char *hash_name; struct strbuf req_buf = STRBUF_INIT; if (server_supports_v2("fetch", 1)) @@ -1194,6 +1195,17 @@ static int send_fetch_request(struct fetch_negotiator *negotiator, int fd_out, args->server_options->items[i].string); } + if (server_feature_v2("object-format", &hash_name)) { + int hash_algo = hash_algo_by_name(hash_name); + if (hash_algo_by_ptr(the_hash_algo) != hash_algo) + die(_("mismatched algorithms: client %s; server %s"), + the_hash_algo->name, hash_name); + packet_write_fmt(fd_out, "object-format=%s", the_hash_algo->name); + } else if (hash_algo_by_ptr(the_hash_algo) != GIT_HASH_SHA1) { + die(_("the server does not support algorithm '%s'"), + the_hash_algo->name); + } + packet_buf_delim(&req_buf); if (args->use_thin_pack) packet_buf_write(&req_buf, "thin-pack"); From patchwork Fri Jun 19 17:55:39 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11614805 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4E65913A0 for ; Fri, 19 Jun 2020 17:56:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2D80A20B80 for ; Fri, 19 Jun 2020 17:56:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="aeE7cA3c" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2404635AbgFSR4i (ORCPT ); Fri, 19 Jun 2020 13:56:38 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:39458 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2404364AbgFSR4a (ORCPT ); Fri, 19 Jun 2020 13:56:30 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:7d4e:cde:7c41:71c2]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 4A38D60A74; Fri, 19 Jun 2020 17:56:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1592589384; bh=5tWITec1XvUpGU34jbqxhafBFLo7Ap2izOu3L993MKI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=aeE7cA3cOEvGP+cwwYgJvP01wCXIVf0MDtD2KxyY4aTYxXPk0y+z8xRTCCPOnuFy2 9HCnZWPWDCS9ku28FVbQTm5JEOYks7iymOmtXsq35A5jRtZ+WaDnXZqKcCerZ/UnF4 tJn5b0wsmhr9lUWd5/1QVZEV4O1yTNl4Zy0Nvw4feGkcEnIDbo5fVoPd13rV0g8DYf 1fdiN4qk8NPQ8HnopH4odwlCVBr2xuMM2DNkn9zB50GtjtXiYnRvXlR24JJ6RdJNlR +3ga7xa3xLjPedzV+hAtIe+g9i8hxZTBlzR3TbYxSklK7BFCYnjVMypDh+gUME2wXd dqDu52cEbuzfoRG5gOcss5CWlRqjjYYh92c8Rdo/R+sZRNSejPY3SEjUDmB2O+52lv 2VDQqImOdEI9z39rGOZS9uPfnIJbouWgMpf4VGvRFSdrW+nhjFwaWlZca4wdXCTuHX znYjDhboPODNzQKhMTocEo5cnORcX1LuaEEpGAgtewMZPzeNfDT From: "brian m. carlson" To: Cc: Junio C Hamano , =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v3 22/44] setup: set the_repository's hash algo when checking format Date: Fri, 19 Jun 2020 17:55:39 +0000 Message-Id: <20200619175601.569856-23-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.278.ge193c7cf3a9 In-Reply-To: <20200619175601.569856-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200619175601.569856-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When we're checking the repository's format, set the hash algorithm at the same time. This ensures that we perform a suitable initialization early enough to avoid confusing any parts of the code. If we defer until later, we can end up with portions of the code which are confused about the hash algorithm, resulting in segfaults when working with SHA-256 repositories. Signed-off-by: brian m. carlson --- setup.c | 1 + 1 file changed, 1 insertion(+) diff --git a/setup.c b/setup.c index 65fe5ecefb..019a1c6367 100644 --- a/setup.c +++ b/setup.c @@ -1273,6 +1273,7 @@ void check_repository_format(struct repository_format *fmt) fmt = &repo_fmt; check_repository_format_gently(get_git_dir(), fmt, NULL); startup_info->have_repository = 1; + repo_set_hash_algo(the_repository, fmt->hash_algo); clear_repository_format(&repo_fmt); } From patchwork Fri Jun 19 17:55:40 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11614869 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4B0EA13A0 for ; Fri, 19 Jun 2020 17:57:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 27AA421527 for ; Fri, 19 Jun 2020 17:57:42 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="twwHRfUe" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2394009AbgFSR5k (ORCPT ); Fri, 19 Jun 2020 13:57:40 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:39554 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2404437AbgFSR4c (ORCPT ); Fri, 19 Jun 2020 13:56:32 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:7d4e:cde:7c41:71c2]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id C75BC60A72; Fri, 19 Jun 2020 17:56:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1592589385; bh=MMfJEd2BJ7WjAZeGiNm4voySOTAFCGpDlJvsq+dHUZY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=twwHRfUeIZvC+/Z1aU2dDbuoKuHDIJJpU8FvtagY1mqw+LZc974gVvussVIrWCIpD QkGlL5o/c6SZm5ZLmB51QTcz+0mhwK0YR0Vgo5W9aqhCU4rLN85YEifYzPRbrB7WlW DbjJGNlKen+sew4hNM+TDQZxQTculbnBgn9liO5Np19VE6q4fLzvOxSErvQbDwubki SRpHZUzW0Vi0j03Wnaq80IlAVzi/kL7G09iocvJ2qKt8PRVVZ4zcAfMOZjYChkjwaz Ogu6Coz7VSQuAx2tC1DrLe93GQ2ssf78DtNcrezK2DDQJ07TYi/52yMGZMjflgFr18 jc5JgVEKViU6JseVc26ncHmVbP/w+8zF0SBQns4kxrwsAjAX0m5/RmvV+mlpvEYDMn B3I8RNJ7POr6T3VlEKyf8tYIhyw/SSJmcB9W68B9bQvLFmo4oXaglgpQLz/ho1X0hy VAI2nUI+7auKWoZ0OO6VY1G+ZZMw6VuXnZwR0VYJsMqP0pLoSO1 From: "brian m. carlson" To: Cc: Junio C Hamano , =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v3 23/44] t3200: mark assertion with SHA1 prerequisite Date: Fri, 19 Jun 2020 17:55:40 +0000 Message-Id: <20200619175601.569856-24-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.278.ge193c7cf3a9 In-Reply-To: <20200619175601.569856-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200619175601.569856-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org One of the test assertions in this test checks that git branch -m works even without a .git/config file. However, if the repository requires configuration extensions, such as because it uses a non-SHA-1 algorithm, this assertion will fail. Mark the assertion as requiring SHA-1. Signed-off-by: brian m. carlson --- t/t3200-branch.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/t/t3200-branch.sh b/t/t3200-branch.sh index 411a70b0ce..2a3fedc6b0 100755 --- a/t/t3200-branch.sh +++ b/t/t3200-branch.sh @@ -402,7 +402,7 @@ EOF mv .git/config .git/config-saved -test_expect_success 'git branch -m q q2 without config should succeed' ' +test_expect_success SHA1 'git branch -m q q2 without config should succeed' ' git branch -m q q2 && git branch -m q2 q ' From patchwork Fri Jun 19 17:55:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11614871 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A1C2613A0 for ; Fri, 19 Jun 2020 17:57:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8AAAE21527 for ; Fri, 19 Jun 2020 17:57:43 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="d5AyOW8o" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2394473AbgFSR5m (ORCPT ); Fri, 19 Jun 2020 13:57:42 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:39524 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2404431AbgFSR4c (ORCPT ); Fri, 19 Jun 2020 13:56:32 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:7d4e:cde:7c41:71c2]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 62DF960A64; Fri, 19 Jun 2020 17:56:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1592589385; bh=+4Z4pf0T9fzuVPGr+GKKdEY7iEgxxGa0MzEFtugw7Z8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=d5AyOW8oLGefHdH8b2cEZhvr3R2VFT+5FrWD3RMCt7g+zzoMYV9R6t3akCpYKAPjI YktWW98Y0SdpzElaSBrm2wuXSLze9ZsfLnPR8dgN7AaaRFCQ/HewZ7vj1xzuRP4hsS GY+g7Ov2NikfWuRfPo+gSCHFIbsM1LRu6+TqeN5PKev6QQxviHEmD4CgzxHsZ4YV0R vkFdMpOzQLl5j5TEfP6ygdCjQJeXETFMsnvR1qm57xZNk7dg/ig/wCF9GQ9O04JZQP K2vfJp106SO2uKsiqWV6S3mN0lIp4ao7drmfrzIx6+ymlIZNOxnfOWfqb61YMWXWlZ lDhTGWMUG1aPa6d2k9FifEw4sLdEmkaSIXp32Uk2IhhtJuXAc8S6nN3RiIOsdONs4/ a4i5cFHSF5FvdTIy7YG6ywZS3uLZY+s0AWHEqUzvF58i/MSzGL09IP6oEQpNAc+gfm tCJw2ijsLdW9Lg0Rv3RvL8gstuVITrMsWL5VBd4HwhbjgD76POk From: "brian m. carlson" To: Cc: Junio C Hamano , =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v3 24/44] packfile: compute and use the index CRC offset Date: Fri, 19 Jun 2020 17:55:41 +0000 Message-Id: <20200619175601.569856-25-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.278.ge193c7cf3a9 In-Reply-To: <20200619175601.569856-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200619175601.569856-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Both v2 pack index files and the v3 format specified as part of the NewHash work have similar data starting at the CRC table. Much of the existing code wants to read either this table or the offset entries following it, and in doing so computes the offset each time. In order to share as much code between v2 and v3, compute the offset of the CRC table and store it when the pack is opened. Use this value to compute offsets to not only the CRC table, but to the offset entries beyond it. Signed-off-by: brian m. carlson --- builtin/index-pack.c | 6 +----- object-store.h | 1 + packfile.c | 1 + 3 files changed, 3 insertions(+), 5 deletions(-) diff --git a/builtin/index-pack.c b/builtin/index-pack.c index f176dd28c8..7bea1fba52 100644 --- a/builtin/index-pack.c +++ b/builtin/index-pack.c @@ -1555,13 +1555,9 @@ static void read_v2_anomalous_offsets(struct packed_git *p, { const uint32_t *idx1, *idx2; uint32_t i; - const uint32_t hashwords = the_hash_algo->rawsz / sizeof(uint32_t); /* The address of the 4-byte offset table */ - idx1 = (((const uint32_t *)p->index_data) - + 2 /* 8-byte header */ - + 256 /* fan out */ - + hashwords * p->num_objects /* object ID table */ + idx1 = (((const uint32_t *)((const uint8_t *)p->index_data + p->crc_offset)) + p->num_objects /* CRC32 table */ ); diff --git a/object-store.h b/object-store.h index d1e490f203..f439d47af8 100644 --- a/object-store.h +++ b/object-store.h @@ -70,6 +70,7 @@ struct packed_git { size_t index_size; uint32_t num_objects; uint32_t num_bad_objects; + uint32_t crc_offset; unsigned char *bad_object_sha1; int index_version; time_t mtime; diff --git a/packfile.c b/packfile.c index f4e752996d..6ab5233613 100644 --- a/packfile.c +++ b/packfile.c @@ -178,6 +178,7 @@ int load_idx(const char *path, const unsigned int hashsz, void *idx_map, */ (sizeof(off_t) <= 4)) return error("pack too large for current definition of off_t in %s", path); + p->crc_offset = 8 + 4 * 256 + nr * hashsz; } p->index_version = version; From patchwork Fri Jun 19 17:55:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11614811 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3991413A0 for ; Fri, 19 Jun 2020 17:56:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0FFE220B80 for ; Fri, 19 Jun 2020 17:56:45 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="PvvEW36p" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2404708AbgFSR4o (ORCPT ); Fri, 19 Jun 2020 13:56:44 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:39446 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2404451AbgFSR4d (ORCPT ); Fri, 19 Jun 2020 13:56:33 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:7d4e:cde:7c41:71c2]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id E02EF60A67; Fri, 19 Jun 2020 17:56:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1592589386; bh=pKT7D8FfW9+6hGmCBXYrcHQTegDW9wGONuHCaQuqves=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=PvvEW36psndSOnlGSOG0esy5igCLBbXFhJSMhjNHeqDRWWk0Pt1EzSviqxkPm6J/v R4QodpCyZynNnSZhEs0igIhXUFJuMFI4PM/uwasO7P2xYlTUEUb/Y5aY/CdcUbu2Gu cVrV9LiUWqmk7hDRxbXtSuzR3oYCsoErnwArcbDjSsH1OFJou1/5wIxp4tonKO33S4 mCW3QN1tg9Cb6mcRAq3yZWs4hFu4EVNv6fE/C/1nIpEjnG71U+xZ1fkxxI2FbjG+2X m4WAjkos2tYGjR0M4fz6QHmHnKEg7vdP951/TYdY7UjgIkC9ja7U5Uh2AuY5HXl8fo b18/2AyXPVCzLERjQkfG9odYkuTJlx3Hrrw2X4o0TWRGf1tz3d2J1wXMFDXOlFrrmu FwouR+o2St9g37CXLPBEXgu9evM+FqeaEvVSLT/xxQXiZpXKh88CYZ6ggCF1lYLWxg JWoF92GhoVWmck02I+qXQU4MPEH/V88+pG0eRubQ4youe++CUXj From: "brian m. carlson" To: Cc: Junio C Hamano , =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v3 25/44] t5302: modernize test formatting Date: Fri, 19 Jun 2020 17:55:42 +0000 Message-Id: <20200619175601.569856-26-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.278.ge193c7cf3a9 In-Reply-To: <20200619175601.569856-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200619175601.569856-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Our style these days is to place the description and the opening quote of the body on the same line as test_expect_success (if it fits), to place the trailing quote on a line by itself after the body, and to use tabs. Since we're going to be making several significant changes to this test, modernize the style to aid in readability of the subsequent patches. This patch should have no functional change. Signed-off-by: brian m. carlson --- t/t5302-pack-index.sh | 360 +++++++++++++++++++++--------------------- 1 file changed, 184 insertions(+), 176 deletions(-) diff --git a/t/t5302-pack-index.sh b/t/t5302-pack-index.sh index ad07f2f7fc..8981c9b90e 100755 --- a/t/t5302-pack-index.sh +++ b/t/t5302-pack-index.sh @@ -7,65 +7,65 @@ test_description='pack index with 64-bit offsets and object CRC' . ./test-lib.sh test_expect_success 'setup' ' - test_oid_init && - rawsz=$(test_oid rawsz) && - rm -rf .git && - git init && - git config pack.threads 1 && - i=1 && - while test $i -le 100 - do - iii=$(printf '%03i' $i) - test-tool genrandom "bar" 200 > wide_delta_$iii && - test-tool genrandom "baz $iii" 50 >> wide_delta_$iii && - test-tool genrandom "foo"$i 100 > deep_delta_$iii && - test-tool genrandom "foo"$(expr $i + 1) 100 >> deep_delta_$iii && - test-tool genrandom "foo"$(expr $i + 2) 100 >> deep_delta_$iii && - echo $iii >file_$iii && - test-tool genrandom "$iii" 8192 >>file_$iii && - git update-index --add file_$iii deep_delta_$iii wide_delta_$iii && - i=$(expr $i + 1) || return 1 - done && - { echo 101 && test-tool genrandom 100 8192; } >file_101 && - git update-index --add file_101 && - tree=$(git write-tree) && - commit=$(git commit-tree $tree obj-list && - git update-ref HEAD $commit + test_oid_init && + rawsz=$(test_oid rawsz) && + rm -rf .git && + git init && + git config pack.threads 1 && + i=1 && + while test $i -le 100 + do + iii=$(printf '%03i' $i) + test-tool genrandom "bar" 200 > wide_delta_$iii && + test-tool genrandom "baz $iii" 50 >> wide_delta_$iii && + test-tool genrandom "foo"$i 100 > deep_delta_$iii && + test-tool genrandom "foo"$(expr $i + 1) 100 >> deep_delta_$iii && + test-tool genrandom "foo"$(expr $i + 2) 100 >> deep_delta_$iii && + echo $iii >file_$iii && + test-tool genrandom "$iii" 8192 >>file_$iii && + git update-index --add file_$iii deep_delta_$iii wide_delta_$iii && + i=$(expr $i + 1) || return 1 + done && + { echo 101 && test-tool genrandom 100 8192; } >file_101 && + git update-index --add file_101 && + tree=$(git write-tree) && + commit=$(git commit-tree $tree obj-list && + git update-ref HEAD $commit ' -test_expect_success \ - 'pack-objects with index version 1' \ - 'pack1=$(git pack-objects --index-version=1 test-1 &1) || ! (echo "$msg" | grep "pack too large .* off_t") @@ -91,21 +91,21 @@ else say "# skipping tests concerning 64-bit offsets" fi -test_expect_success OFF64_T \ - 'index v2: verify a pack with some 64-bit offsets' \ - 'git verify-pack -v "test-3-${pack3}.pack"' +test_expect_success OFF64_T 'index v2: verify a pack with some 64-bit offsets' ' + git verify-pack -v "test-3-${pack3}.pack" +' -test_expect_success OFF64_T \ - '64-bit offsets: should be different from previous index v2 results' \ - '! cmp "test-2-${pack2}.idx" "test-3-${pack3}.idx"' +test_expect_success OFF64_T '64-bit offsets: should be different from previous index v2 results' ' + ! cmp "test-2-${pack2}.idx" "test-3-${pack3}.idx" +' -test_expect_success OFF64_T \ - 'index v2: force some 64-bit offsets with index-pack' \ - 'git index-pack --index-version=2,0x40000 -o 3.idx "test-1-${pack1}.pack"' +test_expect_success OFF64_T 'index v2: force some 64-bit offsets with index-pack' ' + git index-pack --index-version=2,0x40000 -o 3.idx "test-1-${pack1}.pack" +' -test_expect_success OFF64_T \ - '64-bit offsets: index-pack result should match pack-objects one' \ - 'cmp "test-3-${pack3}.idx" "3.idx"' +test_expect_success OFF64_T '64-bit offsets: index-pack result should match pack-objects one' ' + cmp "test-3-${pack3}.idx" "3.idx" +' test_expect_success OFF64_T 'index-pack --verify on 64-bit offset v2 (cheat)' ' # This cheats by knowing which lower offset should still be encoded @@ -120,135 +120,143 @@ test_expect_success OFF64_T 'index-pack --verify on 64-bit offset v2' ' # returns the object number for given object in given pack index index_obj_nr() { - idx_file=$1 - object_sha1=$2 - nr=0 - git show-index < $idx_file | - while read offs sha1 extra - do - nr=$(($nr + 1)) - test "$sha1" = "$object_sha1" || continue - echo "$(($nr - 1))" - break - done + idx_file=$1 + object_sha1=$2 + nr=0 + git show-index < $idx_file | + while read offs sha1 extra + do + nr=$(($nr + 1)) + test "$sha1" = "$object_sha1" || continue + echo "$(($nr - 1))" + break + done } # returns the pack offset for given object as found in given pack index index_obj_offset() { - idx_file=$1 - object_sha1=$2 - git show-index < $idx_file | grep $object_sha1 | - ( read offs extra && echo "$offs" ) + idx_file=$1 + object_sha1=$2 + git show-index < $idx_file | grep $object_sha1 | + ( read offs extra && echo "$offs" ) } -test_expect_success \ - '[index v1] 1) stream pack to repository' \ - 'git index-pack --index-version=1 --stdin < "test-1-${pack1}.pack" && - git prune-packed && - git count-objects | ( read nr rest && test "$nr" -eq 1 ) && - cmp "test-1-${pack1}.pack" ".git/objects/pack/pack-${pack1}.pack" && - cmp "test-1-${pack1}.idx" ".git/objects/pack/pack-${pack1}.idx"' +test_expect_success '[index v1] 1) stream pack to repository' ' + git index-pack --index-version=1 --stdin < "test-1-${pack1}.pack" && + git prune-packed && + git count-objects | ( read nr rest && test "$nr" -eq 1 ) && + cmp "test-1-${pack1}.pack" ".git/objects/pack/pack-${pack1}.pack" && + cmp "test-1-${pack1}.idx" ".git/objects/pack/pack-${pack1}.idx" +' test_expect_success \ - '[index v1] 2) create a stealth corruption in a delta base reference' \ - '# This test assumes file_101 is a delta smaller than 16 bytes. - # It should be against file_100 but we substitute its base for file_099 - sha1_101=$(git hash-object file_101) && - sha1_099=$(git hash-object file_099) && - offs_101=$(index_obj_offset 1.idx $sha1_101) && - nr_099=$(index_obj_nr 1.idx $sha1_099) && - chmod +w ".git/objects/pack/pack-${pack1}.pack" && - recordsz=$((rawsz + 4)) && - dd of=".git/objects/pack/pack-${pack1}.pack" seek=$(($offs_101 + 1)) \ - if=".git/objects/pack/pack-${pack1}.idx" \ - skip=$((4 + 256 * 4 + $nr_099 * recordsz)) \ - bs=1 count=$rawsz conv=notrunc && - git cat-file blob $sha1_101 > file_101_foo1' + '[index v1] 2) create a stealth corruption in a delta base reference' ' + # This test assumes file_101 is a delta smaller than 16 bytes. + # It should be against file_100 but we substitute its base for file_099 + sha1_101=$(git hash-object file_101) && + sha1_099=$(git hash-object file_099) && + offs_101=$(index_obj_offset 1.idx $sha1_101) && + nr_099=$(index_obj_nr 1.idx $sha1_099) && + chmod +w ".git/objects/pack/pack-${pack1}.pack" && + recordsz=$((rawsz + 4)) && + dd of=".git/objects/pack/pack-${pack1}.pack" seek=$(($offs_101 + 1)) \ + if=".git/objects/pack/pack-${pack1}.idx" \ + skip=$((4 + 256 * 4 + $nr_099 * recordsz)) \ + bs=1 count=$rawsz conv=notrunc && + git cat-file blob $sha1_101 > file_101_foo1 +' test_expect_success \ - '[index v1] 3) corrupted delta happily returned wrong data' \ - 'test -f file_101_foo1 && ! cmp file_101 file_101_foo1' + '[index v1] 3) corrupted delta happily returned wrong data' ' + test -f file_101_foo1 && ! cmp file_101 file_101_foo1 +' test_expect_success \ - '[index v1] 4) confirm that the pack is actually corrupted' \ - 'test_must_fail git fsck --full $commit' + '[index v1] 4) confirm that the pack is actually corrupted' ' + test_must_fail git fsck --full $commit +' test_expect_success \ - '[index v1] 5) pack-objects happily reuses corrupted data' \ - 'pack4=$(git pack-objects test-4 file_101_foo2 +' test_expect_success \ - '[index v2] 1) stream pack to repository' \ - 'rm -f .git/objects/pack/* && - git index-pack --index-version=2 --stdin < "test-1-${pack1}.pack" && - git prune-packed && - git count-objects | ( read nr rest && test "$nr" -eq 1 ) && - cmp "test-1-${pack1}.pack" ".git/objects/pack/pack-${pack1}.pack" && - cmp "test-2-${pack1}.idx" ".git/objects/pack/pack-${pack1}.idx"' + '[index v2] 3) corrupted delta happily returned wrong data' ' + test -f file_101_foo2 && ! cmp file_101 file_101_foo2 +' test_expect_success \ - '[index v2] 2) create a stealth corruption in a delta base reference' \ - '# This test assumes file_101 is a delta smaller than 16 bytes. - # It should be against file_100 but we substitute its base for file_099 - sha1_101=$(git hash-object file_101) && - sha1_099=$(git hash-object file_099) && - offs_101=$(index_obj_offset 1.idx $sha1_101) && - nr_099=$(index_obj_nr 1.idx $sha1_099) && - chmod +w ".git/objects/pack/pack-${pack1}.pack" && - dd of=".git/objects/pack/pack-${pack1}.pack" seek=$(($offs_101 + 1)) \ - if=".git/objects/pack/pack-${pack1}.idx" \ - skip=$((8 + 256 * 4 + $nr_099 * rawsz)) \ - bs=1 count=$rawsz conv=notrunc && - git cat-file blob $sha1_101 > file_101_foo2' + '[index v2] 4) confirm that the pack is actually corrupted' ' + test_must_fail git fsck --full $commit +' test_expect_success \ - '[index v2] 3) corrupted delta happily returned wrong data' \ - 'test -f file_101_foo2 && ! cmp file_101 file_101_foo2' + '[index v2] 5) pack-objects refuses to reuse corrupted data' ' + test_must_fail git pack-objects test-5 /dev/null || exit 1 - done /dev/null || exit 1 + done wrong-tag <wrong-tag <err && - grep "^warning:.* expected .tagger. line" err + tag=$(git hash-object -t tag -w --stdin err && + grep "^warning:.* expected .tagger. line" err ' test_expect_success 'index-pack --fsck-objects also warns upon missing tagger in tag' ' - git index-pack --fsck-objects tag-test-${pack1}.pack 2>err && - grep "^warning:.* expected .tagger. line" err + git index-pack --fsck-objects tag-test-${pack1}.pack 2>err && + grep "^warning:.* expected .tagger. line" err ' test_done From patchwork Fri Jun 19 17:55:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11614809 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 48291618 for ; Fri, 19 Jun 2020 17:56:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2FE2F20776 for ; Fri, 19 Jun 2020 17:56:44 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="MG2DJUHT" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2404644AbgFSR4l (ORCPT ); Fri, 19 Jun 2020 13:56:41 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:39560 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732259AbgFSR4d (ORCPT ); Fri, 19 Jun 2020 13:56:33 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:7d4e:cde:7c41:71c2]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 71D4060A75; Fri, 19 Jun 2020 17:56:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1592589386; bh=KCNvkesVvwvNQy/hd5h5tkTw+MplM9cz3ZpWZNGYF3U=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=MG2DJUHTEIiSIgr4CWVIKNL1aOyKPZrV6A5ayyWE+20NWceCIi0OPSFj+aB/mycUx GrodMJPf34iWGoZsI50Er0N8CMVP5ioGUY+ZNRtpiCMUIhJIzCRcQVdkhkYM5DGMwI pzJDt9AoS55ZZkhGoVtWBppDbVDMKGNd8M0b5cdI9qTzdlkaFxXcqCd4dtPl61qk5c Ifh+nkxROxJhMA5QwYXRcaoGHzWQWldXjJk5KPCAOD9M2H1k3NIu0rvc2myA0mvtCf 4/NuG3n5mfpP8bjm5h3RJFJudntUdb5YYdaKIFCXp0kIg1Lbavo5/2CoqaPv0jUm50 +GeZvMebid4CriUd1PEfihGoNDmzTeXmSghsWnzw7hCrvb7UZJ2jd3b7euaLYjiwbe 0IX0BunhqcDj0NWPWqKr0VRALeNesZIdkkMpPe+8p3LBkjF0r+qBLuLR9vyyB0G2Zu MHDQKkckWEmSJTOO1w4ekFgI1+Z/fgfvfswuuCGBvjbaFL+F9hz From: "brian m. carlson" To: Cc: Junio C Hamano , =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v3 26/44] builtin/show-index: provide options to determine hash algo Date: Fri, 19 Jun 2020 17:55:43 +0000 Message-Id: <20200619175601.569856-27-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.278.ge193c7cf3a9 In-Reply-To: <20200619175601.569856-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200619175601.569856-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org show-index is capable of reading any possible index file whether or not the index is inside a repository. However, because our index files lack metadata about the hash algorithm in use, it's not possible to autodetect the algorithm that a particular index file is using. In order to allow us to read index files of any algorithm, let's set up the .git directory gently so that we default to the algorithm for the current repository, and add an --object-format option to allow users to override this setting and continue to run show-index outside of a repository altogether. Let's also document this new option so that people can find it and use it. Signed-off-by: brian m. carlson --- Documentation/git-show-index.txt | 11 ++++++++++- builtin/show-index.c | 29 ++++++++++++++++++++++++----- git.c | 2 +- 3 files changed, 35 insertions(+), 7 deletions(-) diff --git a/Documentation/git-show-index.txt b/Documentation/git-show-index.txt index 424e4ba84c..39b1d8eaa1 100644 --- a/Documentation/git-show-index.txt +++ b/Documentation/git-show-index.txt @@ -9,7 +9,7 @@ git-show-index - Show packed archive index SYNOPSIS -------- [verse] -'git show-index' +'git show-index' [--object-format=] DESCRIPTION @@ -36,6 +36,15 @@ Note that you can get more information on a packfile by calling linkgit:git-verify-pack[1]. However, as this command considers only the index file itself, it's both faster and more flexible. +OPTIONS +------- + +--object-format=:: + Specify the given object format (hash algorithm) for the index file. The + valid values are 'sha1' and (if enabled) 'sha256'. The default is the + algorithm for the current repository (set by `extensions.objectFormat`), or + 'sha1' if no value is set or outside a repository.. + GIT --- Part of the linkgit:git[1] suite diff --git a/builtin/show-index.c b/builtin/show-index.c index 0826f6a5a2..8106b03a6b 100644 --- a/builtin/show-index.c +++ b/builtin/show-index.c @@ -1,9 +1,12 @@ #include "builtin.h" #include "cache.h" #include "pack.h" +#include "parse-options.h" -static const char show_index_usage[] = -"git show-index"; +static const char *const show_index_usage[] = { + "git show-index [--object-format=]", + NULL +}; int cmd_show_index(int argc, const char **argv, const char *prefix) { @@ -11,10 +14,26 @@ int cmd_show_index(int argc, const char **argv, const char *prefix) unsigned nr; unsigned int version; static unsigned int top_index[256]; - const unsigned hashsz = the_hash_algo->rawsz; + unsigned hashsz; + const char *hash_name = NULL; + int hash_algo; + const struct option show_index_options[] = { + OPT_STRING(0, "object-format", &hash_name, N_("hash-algorithm"), + N_("specify the hash algorithm to use")), + OPT_END() + }; + + argc = parse_options(argc, argv, prefix, show_index_options, show_index_usage, 0); + + if (hash_name) { + hash_algo = hash_algo_by_name(hash_name); + if (hash_algo == GIT_HASH_UNKNOWN) + die(_("Unknown hash algorithm")); + repo_set_hash_algo(the_repository, hash_algo); + } + + hashsz = the_hash_algo->rawsz; - if (argc != 1) - usage(show_index_usage); if (fread(top_index, 2 * 4, 1, stdin) != 1) die("unable to read header"); if (top_index[0] == htonl(PACK_IDX_SIGNATURE)) { diff --git a/git.c b/git.c index a2d337eed7..2f021b97f3 100644 --- a/git.c +++ b/git.c @@ -574,7 +574,7 @@ static struct cmd_struct commands[] = { { "shortlog", cmd_shortlog, RUN_SETUP_GENTLY | USE_PAGER }, { "show", cmd_show, RUN_SETUP }, { "show-branch", cmd_show_branch, RUN_SETUP }, - { "show-index", cmd_show_index }, + { "show-index", cmd_show_index, RUN_SETUP_GENTLY }, { "show-ref", cmd_show_ref, RUN_SETUP }, { "sparse-checkout", cmd_sparse_checkout, RUN_SETUP | NEED_WORK_TREE }, { "stage", cmd_add, RUN_SETUP | NEED_WORK_TREE }, From patchwork Fri Jun 19 17:55:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11614855 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8802413A0 for ; Fri, 19 Jun 2020 17:57:37 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 643B921527 for ; Fri, 19 Jun 2020 17:57:37 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="IGN/Mtfz" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2405345AbgFSR5g (ORCPT ); Fri, 19 Jun 2020 13:57:36 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:39508 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2393389AbgFSR4c (ORCPT ); Fri, 19 Jun 2020 13:56:32 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:7d4e:cde:7c41:71c2]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 18DEC60A76; Fri, 19 Jun 2020 17:56:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1592589387; bh=VVQ0dANHQNwdWuSaocgt9e3EABF8vgKVBdEBtDLaQuI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=IGN/Mtfz9MXEDzUvccEC3EvEZ+JMkb6RavZWs23ZVzUg0D6min4Gc4BioHnwPn1hK 0yP1pbD6AtLPlDdABgqDRpmd3eBUS4QKzXpX76BqsZg0OueZbno/7vUrg/ngbQorYU wgLRyq2poacWHFEUn5jDNulvqemDBPNu6P7HXS2tk6kLLPUxZHXDC865WP00LYuvxk +IWnSXK0mXHlUJwo85c6aYPQ+uOIPkmH9+HT6G0/NxeIzIj7GawEGTMAUmyje61yRa DPsJJMpvgwDjHF3rOO12rRuqO9vkBBFWDuUW1p7+yOJ4dB8oBjkTN7XCY3WEycUGXh Ptcfnyjsu9MLoKY4yNUPERK6XhxaskK0fQA8On7EaSKCrdWQUGIgEUd/METxFLQyZy C+37IMYxbmIFuGT4CVXGADW94Ag/0LoI5aKqOfm/S+eYnFTlpyKgkiUqv+IUCKIZr2 j6bDJu8Q7EjxwIiRK7S+JtZOIcz8vAAZkf09jiZkGB5JMjA3boD From: "brian m. carlson" To: Cc: Junio C Hamano , =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v3 27/44] t1302: expect repo format version 1 for SHA-256 Date: Fri, 19 Jun 2020 17:55:44 +0000 Message-Id: <20200619175601.569856-28-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.278.ge193c7cf3a9 In-Reply-To: <20200619175601.569856-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200619175601.569856-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When using SHA-256, we need to take advantage of the extensions section in the config file, so we need to use repository format version 1. Update the test to look for the correct value. Note that test_oid produces a value without a trailing newline, so use echo to ensure we print a trailing newline to compare it correctly against the actual results. Signed-off-by: brian m. carlson --- t/t1302-repo-version.sh | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/t/t1302-repo-version.sh b/t/t1302-repo-version.sh index ce4cff13bb..d60c042ce8 100755 --- a/t/t1302-repo-version.sh +++ b/t/t1302-repo-version.sh @@ -8,6 +8,10 @@ test_description='Test repository version check' . ./test-lib.sh test_expect_success 'setup' ' + test_oid_cache <<-\EOF && + version sha1:0 + version sha256:1 + EOF cat >test.patch <<-\EOF && diff --git a/test.txt b/test.txt new file mode 100644 @@ -23,7 +27,7 @@ test_expect_success 'setup' ' ' test_expect_success 'gitdir selection on normal repos' ' - echo 0 >expect && + echo $(test_oid version) >expect && git config core.repositoryformatversion >actual && git -C test config core.repositoryformatversion >actual2 && test_cmp expect actual && From patchwork Fri Jun 19 17:55:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11614825 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1BE10618 for ; Fri, 19 Jun 2020 17:56:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0039020776 for ; Fri, 19 Jun 2020 17:56:57 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="lTYUe+ot" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2404873AbgFSR44 (ORCPT ); Fri, 19 Jun 2020 13:56:56 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:39572 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2393718AbgFSR4g (ORCPT ); Fri, 19 Jun 2020 13:56:36 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:7d4e:cde:7c41:71c2]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 9177E60A68; Fri, 19 Jun 2020 17:56:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1592589387; bh=Cok++fiK9k9qHlhEU7BdUjv5FBYoGOPqOeNammZJSgs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=lTYUe+otxBa8YS/d3+kMsYBzt47hkdXucAJOKfGh8z9vvNo4NXiSWEnjQ/BonbL2j b+L06KFiR8kp46uoQ2/tYFg+yiyAa6et62HsXeDDzibBjYlkFT6oXX0xE5cVXWC5ba 7HDkrVjezaVKCCP3TaRPBVXorOFmu7zWKD51h0J4so5vE7Xzz/a012VhQtWro30pld oh740ImlvyGI/eyN3e/Cg6cgUST40ngAZg1F8mDN6i9zenO+gto+Cm2KUMi7Mi2F+G Sw+rBY0YwTLrQLhCSBIE48UCDYeHQZKzkCKJmEp2HJHZeRBL2PoqWrGKgse8lVgbF3 Bzd/TQBkM07YB7xdErbfgUH4BATj9cXEVBywFjOxSAr1hLnrKD7HEzsF8wkKLuvhww PMhggf0vLJ7hjNY5ANGeWADetdBBHGQaigZZC8u2B+yfCopluWhOZ5pMWovHRwBYz5 6ngCfF3IJwdinsU3wxQyqGIqk9VA+Lap43xeWLCQCWD+AIpppc0 From: "brian m. carlson" To: Cc: Junio C Hamano , =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v3 28/44] Documentation/technical: document object-format for protocol v2 Date: Fri, 19 Jun 2020 17:55:45 +0000 Message-Id: <20200619175601.569856-29-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.278.ge193c7cf3a9 In-Reply-To: <20200619175601.569856-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200619175601.569856-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Document the object-format extension for protocol v2. Signed-off-by: brian m. carlson --- Documentation/technical/protocol-v2.txt | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/Documentation/technical/protocol-v2.txt b/Documentation/technical/protocol-v2.txt index 3996d70891..b288df7ed7 100644 --- a/Documentation/technical/protocol-v2.txt +++ b/Documentation/technical/protocol-v2.txt @@ -455,3 +455,12 @@ included in a request. This is done by sending each option as a a request. The provided options must not contain a NUL or LF character. + + object-format +~~~~~~~~~~~~~~~ + +The server can advertise the `object-format` capability with a value `X` (in the +form `object-format=X`) to notify the client that the server is able to deal +with objects using hash algorithm X. If not specified, the server is assumed to +only handle SHA-1. If the client would like to use a hash algorithm other than +SHA-1, it should specify its object-format string. From patchwork Fri Jun 19 17:55:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11614817 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 184B913A0 for ; Fri, 19 Jun 2020 17:56:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EB0BE20DD4 for ; Fri, 19 Jun 2020 17:56:48 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="MChPRlvG" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2404795AbgFSR4s (ORCPT ); Fri, 19 Jun 2020 13:56:48 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:39458 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2393914AbgFSR4g (ORCPT ); Fri, 19 Jun 2020 13:56:36 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:7d4e:cde:7c41:71c2]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 1549960A77; Fri, 19 Jun 2020 17:56:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1592589388; bh=gcmPlShg+W0YG/VVs2GsdWxNbZDEeQ+gaiDzhdzhpqQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=MChPRlvG9H4vqXqH2/p/LBp0uqGAmac8SYHoUUsBTgjAtscYwhbSyEEX4ZSZwTCtZ z3daRE+pu4YrBuL+ZYLBCt4atnCah0HvHzxbHJ864pmyqM4BwOGqd4ES1Gr/hTtKV0 ssfh5qMoSPg5PTBk1lC75hS+IrvNPFQ68AqhpmYr8rCW/C3HLnKBEwXfT5o6v+at8r 8ZbHPSzVz/hIYrRAS3P8hZzp/3Y6Ntp5oga0w0EJL6jkB29DwP37Qp+4ltaGcsGAVy HoS9ng8RDBADrgBMc61YG6/tT4DBJKdx424vjZO7R4uWCz/sg5MDu0FZIWagmX8Wdx +9BuYxObnEzWG8hw9/IT/9M0tX/HCwv/5hY0vPA3OmFiYW0I+WhSyC57+geAObRGnY XZJv4aW6MzfhpKIfxfYgd593j5UGBKM3zVi3yLejy1jCjjrLYAzBHBP/DRnLkJ8yg1 n9RvhpO6lEmUvSEpBGzoM2iVMkyrJflRCqeW9SqQs/kbp5YfaJh From: "brian m. carlson" To: Cc: Junio C Hamano , =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v3 29/44] connect: pass full packet reader when parsing v2 refs Date: Fri, 19 Jun 2020 17:55:46 +0000 Message-Id: <20200619175601.569856-30-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.278.ge193c7cf3a9 In-Reply-To: <20200619175601.569856-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200619175601.569856-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When we're parsing refs, we need to know not only what the line we're parsing is, but also the hash algorithm we should use to parse it, which is stored in the reader object. Pass the packet reader object through to the protocol v2 ref parsing function. Signed-off-by: brian m. carlson --- connect.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/connect.c b/connect.c index b6e110cb24..320cc2016d 100644 --- a/connect.c +++ b/connect.c @@ -376,7 +376,7 @@ struct ref **get_remote_heads(struct packet_reader *reader, } /* Returns 1 when a valid ref has been added to `list`, 0 otherwise */ -static int process_ref_v2(const char *line, struct ref ***list) +static int process_ref_v2(struct packet_reader *reader, struct ref ***list) { int ret = 1; int i = 0; @@ -384,6 +384,7 @@ static int process_ref_v2(const char *line, struct ref ***list) struct ref *ref; struct string_list line_sections = STRING_LIST_INIT_DUP; const char *end; + const char *line = reader->line; /* * Ref lines have a number of fields which are space deliminated. The @@ -482,7 +483,7 @@ struct ref **get_remote_refs(int fd_out, struct packet_reader *reader, /* Process response from server */ while (packet_reader_read(reader) == PACKET_READ_NORMAL) { - if (!process_ref_v2(reader->line, &list)) + if (!process_ref_v2(reader, &list)) die(_("invalid ls-refs response: %s"), reader->line); } From patchwork Fri Jun 19 17:55:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11614815 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4CB3D1731 for ; Fri, 19 Jun 2020 17:56:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 319AC20DD4 for ; Fri, 19 Jun 2020 17:56:48 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="fr9+qBaa" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2404736AbgFSR4p (ORCPT ); Fri, 19 Jun 2020 13:56:45 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:39526 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2393893AbgFSR4g (ORCPT ); Fri, 19 Jun 2020 13:56:36 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:7d4e:cde:7c41:71c2]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 8FF6A60A78; Fri, 19 Jun 2020 17:56:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1592589388; bh=VEFsq6X+8mMwNvzoUhrFq5r46eaowPx57ncl9Z6sta8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=fr9+qBaahR9AgquiHWYyWorOzxaKKZb6QRLjt1MdMqpDUKIhf8G+8fTvvii0JgkPF /X5QmrrAGZyGVDdkyefU5E3m4oz22HLmgwsXnSQ43vsYu6uAEhL7CxQotBwRYoVJ1S PttPklZrXgX3E695IIK26CWivE5bvkqrmfdaf5QM+waNrrLR+TCwvlhLp3XCvEqFce sf133cYxVzY8mrQNkTSqzJhcCE3OzxkrQbHVp9NwyzYV2PGWoA7HLZXmckQwQcSlx8 QW62tXiS9iDoXlFzEes2KXOfygDnn3j1i3erhKCHzlfxzEbzQTxALZGwaVKR8LCLTW aVAKO5+o2Xy2jlsi5wRH8HzqP5yRYkVqUbOM5rYk/fBe66uOfGdwmrdYlKDrDfGGm3 iadqacM/iod3c9fT44NOMs0GJ/xUGXol5QXhbQt7b3two6gOBUo6k7oSpae4OrHCoP StzpR/SxeQSQVtmS4L2O/MmhFs+S+Y+oNsqeADlPjn6geJClEqP From: "brian m. carlson" To: Cc: Junio C Hamano , =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v3 30/44] connect: parse v2 refs with correct hash algorithm Date: Fri, 19 Jun 2020 17:55:47 +0000 Message-Id: <20200619175601.569856-31-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.278.ge193c7cf3a9 In-Reply-To: <20200619175601.569856-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200619175601.569856-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When using protocol v2, we need to know what hash algorithm is used by the remote end. See if the server has sent us an object-format capability, and if so, use it to determine the hash algorithm in use and set that value in the packet reader. Parse the refs using this algorithm. Signed-off-by: brian m. carlson --- connect.c | 21 ++++++++++++++++----- 1 file changed, 16 insertions(+), 5 deletions(-) diff --git a/connect.c b/connect.c index 320cc2016d..e6cf2f8dc4 100644 --- a/connect.c +++ b/connect.c @@ -284,7 +284,7 @@ static int process_ref(const struct packet_reader *reader, int len, die(_("protocol error: unexpected capabilities^{}")); } else if (check_ref(name, flags)) { struct ref *ref = alloc_ref(name); - memcpy(ref->old_oid.hash, old_oid.hash, reader->hash_algo->rawsz); + oidcpy(&ref->old_oid, &old_oid); **list = ref; *list = &ref->next; } @@ -397,7 +397,7 @@ static int process_ref_v2(struct packet_reader *reader, struct ref ***list) goto out; } - if (parse_oid_hex(line_sections.items[i++].string, &old_oid, &end) || + if (parse_oid_hex_algop(line_sections.items[i++].string, &old_oid, &end, reader->hash_algo) || *end) { ret = 0; goto out; @@ -405,7 +405,7 @@ static int process_ref_v2(struct packet_reader *reader, struct ref ***list) ref = alloc_ref(line_sections.items[i++].string); - oidcpy(&ref->old_oid, &old_oid); + memcpy(ref->old_oid.hash, old_oid.hash, reader->hash_algo->rawsz); **list = ref; *list = &ref->next; @@ -418,7 +418,8 @@ static int process_ref_v2(struct packet_reader *reader, struct ref ***list) struct object_id peeled_oid; char *peeled_name; struct ref *peeled; - if (parse_oid_hex(arg, &peeled_oid, &end) || *end) { + if (parse_oid_hex_algop(arg, &peeled_oid, &end, + reader->hash_algo) || *end) { ret = 0; goto out; } @@ -426,7 +427,8 @@ static int process_ref_v2(struct packet_reader *reader, struct ref ***list) peeled_name = xstrfmt("%s^{}", ref->name); peeled = alloc_ref(peeled_name); - oidcpy(&peeled->old_oid, &peeled_oid); + memcpy(peeled->old_oid.hash, peeled_oid.hash, + reader->hash_algo->rawsz); **list = peeled; *list = &peeled->next; @@ -456,6 +458,7 @@ struct ref **get_remote_refs(int fd_out, struct packet_reader *reader, int stateless_rpc) { int i; + const char *hash_name; *list = NULL; if (server_supports_v2("ls-refs", 1)) @@ -464,6 +467,14 @@ struct ref **get_remote_refs(int fd_out, struct packet_reader *reader, if (server_supports_v2("agent", 0)) packet_write_fmt(fd_out, "agent=%s", git_user_agent_sanitized()); + if (server_feature_v2("object-format", &hash_name)) { + int hash_algo = hash_algo_by_name(hash_name); + if (hash_algo == GIT_HASH_UNKNOWN) + die(_("unknown object format '%s' specified by server"), hash_name); + reader->hash_algo = &hash_algos[hash_algo]; + packet_write_fmt(fd_out, "object-format=%s", reader->hash_algo->name); + } + if (server_options && server_options->nr && server_supports_v2("server-option", 1)) for (i = 0; i < server_options->nr; i++) From patchwork Fri Jun 19 17:55:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11614819 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7259D618 for ; Fri, 19 Jun 2020 17:56:50 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5308320DD4 for ; Fri, 19 Jun 2020 17:56:50 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="hdv/hQ6G" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2404843AbgFSR4t (ORCPT ); Fri, 19 Jun 2020 13:56:49 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:39574 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2393935AbgFSR4g (ORCPT ); Fri, 19 Jun 2020 13:56:36 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:7d4e:cde:7c41:71c2]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 1461A60A79; Fri, 19 Jun 2020 17:56:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1592589389; bh=f/I+eENOG6oisCR05Dy/FbhAHw2+T1dtP8w4hBm/8KM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=hdv/hQ6Gco+2JKo+Q9Im+ZS87li559Rw1MmAYUiMQ0hgKqdjYzkAJGyasoczWmNPO czouGb35f/WzGzlhWY7Z5+l4QTYOMTX4seiTPmf9rQkaD4yiGsVgHHd36LQTbeNnO4 smHQSU9W10mZFbkHvqri/xxdJTWG+lfiU6YEJBWQKMVTAoliEfmLMbdwxVHjrZBrXp OCcSIHwniGAQsxTn/skbbyYErZBeOwK3kWPU7bpKkdSzJOndMibx2vPvXE/JYTpQCo DjtNqO9/IPxqKFocUYTk4VidWkhertdXbd3qRC2l5NG6S4WxTOvnQEJKXfVMFfBBTj YK+kqNsnrTSATYr34mlfJVIwWR267oO6EO0D9c+YVw7NwSgtZomJtEpiM96qLZ8K3e 6XOB0Z4wc7C83fcqjbT/B3e1d2Zk2s0K2g3Nb2/nde3dnRsVPuZEKg1W1rMPOqWyKz Xhe+EvhRuPdJNKR4smMNk56lZGh3HXN0UiUIFzEVfdILPcMmxv+ From: "brian m. carlson" To: Cc: Junio C Hamano , =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v3 31/44] serve: advertise object-format capability for protocol v2 Date: Fri, 19 Jun 2020 17:55:48 +0000 Message-Id: <20200619175601.569856-32-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.278.ge193c7cf3a9 In-Reply-To: <20200619175601.569856-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200619175601.569856-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In order to communicate the protocol supported by the server side, add support for advertising the object-format capability. We check that the client side sends us an identical algorithm if it sends us its own object-format capability, and assume it speaks SHA-1 if not. In the test, when we're using an algorithm other than SHA-1, we need to specify the algorithm in use so we don't get a failure with an "unknown format" message. Add a test that we handle a mismatched algorithm. Remove the test_oid_init call since it's no longer necessary. Signed-off-by: brian m. carlson --- connect.c | 2 ++ serve.c | 27 +++++++++++++++++++++++++++ t/t5701-git-serve.sh | 25 +++++++++++++++++++++++++ 3 files changed, 54 insertions(+) diff --git a/connect.c b/connect.c index e6cf2f8dc4..e0d5b9fee0 100644 --- a/connect.c +++ b/connect.c @@ -473,6 +473,8 @@ struct ref **get_remote_refs(int fd_out, struct packet_reader *reader, die(_("unknown object format '%s' specified by server"), hash_name); reader->hash_algo = &hash_algos[hash_algo]; packet_write_fmt(fd_out, "object-format=%s", reader->hash_algo->name); + } else { + reader->hash_algo = &hash_algos[GIT_HASH_SHA1]; } if (server_options && server_options->nr && diff --git a/serve.c b/serve.c index c046926ba1..fbd2fcdfb5 100644 --- a/serve.c +++ b/serve.c @@ -22,6 +22,14 @@ static int agent_advertise(struct repository *r, return 1; } +static int object_format_advertise(struct repository *r, + struct strbuf *value) +{ + if (value) + strbuf_addstr(value, r->hash_algo->name); + return 1; +} + struct protocol_capability { /* * The name of the capability. The server uses this name when @@ -57,6 +65,7 @@ static struct protocol_capability capabilities[] = { { "ls-refs", always_advertise, ls_refs }, { "fetch", upload_pack_advertise, upload_pack_v2 }, { "server-option", always_advertise, NULL }, + { "object-format", object_format_advertise, NULL }, }; static void advertise_capabilities(void) @@ -153,6 +162,22 @@ int has_capability(const struct argv_array *keys, const char *capability, return 0; } +static void check_algorithm(struct repository *r, struct argv_array *keys) +{ + int client = GIT_HASH_SHA1, server = hash_algo_by_ptr(r->hash_algo); + const char *algo_name; + + if (has_capability(keys, "object-format", &algo_name)) { + client = hash_algo_by_name(algo_name); + if (client == GIT_HASH_UNKNOWN) + die("unknown object format '%s'", algo_name); + } + + if (client != server) + die("mismatched object format: server %s; client %s\n", + r->hash_algo->name, hash_algos[client].name); +} + enum request_state { PROCESS_REQUEST_KEYS, PROCESS_REQUEST_DONE, @@ -225,6 +250,8 @@ static int process_request(void) if (!command) die("no command requested"); + check_algorithm(the_repository, &keys); + command->command(the_repository, &keys, &reader); argv_array_clear(&keys); diff --git a/t/t5701-git-serve.sh b/t/t5701-git-serve.sh index ffb9613885..a1f5fdc9fd 100755 --- a/t/t5701-git-serve.sh +++ b/t/t5701-git-serve.sh @@ -5,12 +5,17 @@ test_description='test protocol v2 server commands' . ./test-lib.sh test_expect_success 'test capability advertisement' ' + test_oid_cache <<-EOF && + wrong_algo sha1:sha256 + wrong_algo sha256:sha1 + EOF cat >expect <<-EOF && version 2 agent=git/$(git version | cut -d" " -f3) ls-refs fetch=shallow server-option + object-format=$(test_oid algo) 0000 EOF @@ -45,6 +50,7 @@ test_expect_success 'request invalid capability' ' test_expect_success 'request with no command' ' test-tool pkt-line pack >in <<-EOF && agent=git/test + object-format=$(test_oid algo) 0000 EOF test_must_fail test-tool serve-v2 --stateless-rpc 2>err in <<-EOF && command=foo + object-format=$(test_oid algo) agent=git/test 0000 EOF @@ -61,6 +68,17 @@ test_expect_success 'request invalid command' ' test_i18ngrep "invalid command" err ' +test_expect_success 'wrong object-format' ' + test-tool pkt-line pack >in <<-EOF && + command=fetch + agent=git/test + object-format=$(test_oid wrong_algo) + 0000 + EOF + test_must_fail test-tool serve-v2 --stateless-rpc 2>err in <<-EOF && command=ls-refs + object-format=$(test_oid algo) 0000 EOF @@ -96,6 +115,7 @@ test_expect_success 'basics of ls-refs' ' test_expect_success 'basic ref-prefixes' ' test-tool pkt-line pack >in <<-EOF && command=ls-refs + object-format=$(test_oid algo) 0001 ref-prefix refs/heads/master ref-prefix refs/tags/one @@ -116,6 +136,7 @@ test_expect_success 'basic ref-prefixes' ' test_expect_success 'refs/heads prefix' ' test-tool pkt-line pack >in <<-EOF && command=ls-refs + object-format=$(test_oid algo) 0001 ref-prefix refs/heads/ 0000 @@ -136,6 +157,7 @@ test_expect_success 'refs/heads prefix' ' test_expect_success 'peel parameter' ' test-tool pkt-line pack >in <<-EOF && command=ls-refs + object-format=$(test_oid algo) 0001 peel ref-prefix refs/tags/ @@ -157,6 +179,7 @@ test_expect_success 'peel parameter' ' test_expect_success 'symrefs parameter' ' test-tool pkt-line pack >in <<-EOF && command=ls-refs + object-format=$(test_oid algo) 0001 symrefs ref-prefix refs/heads/ @@ -178,6 +201,7 @@ test_expect_success 'symrefs parameter' ' test_expect_success 'sending server-options' ' test-tool pkt-line pack >in <<-EOF && command=ls-refs + object-format=$(test_oid algo) server-option=hello server-option=world 0001 @@ -200,6 +224,7 @@ test_expect_success 'unexpected lines are not allowed in fetch request' ' test-tool pkt-line pack >in <<-EOF && command=fetch + object-format=$(test_oid algo) 0001 this-is-not-a-command 0000 From patchwork Fri Jun 19 17:55:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11614833 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7AF11618 for ; Fri, 19 Jun 2020 17:57:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 61E1820DD4 for ; Fri, 19 Jun 2020 17:57:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="GsIuRvCW" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2405097AbgFSR5E (ORCPT ); Fri, 19 Jun 2020 13:57:04 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:39670 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2436502AbgFSR5C (ORCPT ); Fri, 19 Jun 2020 13:57:02 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:7d4e:cde:7c41:71c2]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 8E19A60A7A; Fri, 19 Jun 2020 17:56:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1592589389; bh=Mo6wZuC9p3nChbBxsGl44GyZeJquT3ZYpX4+z7OuAXU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=GsIuRvCWMs30DZDFe02VabblGB/nwN2PldnyBNZet3/IEiahaG6eKtOkK92a9QSXI vyh/sJKfaeYG+dVkYe6O/vdjGArWzF6sUq3nkMjK0quQ/oSffnvfXAg2jtwDsyb+jj x8HLAunx0FflHO6sUN7eZN85CA8wbDDATlYLY53bTzUBmuYDiR8chjAkIWsc2bODKD J5iF8v1BjDcWYWyoVPF5d2QUHybyyVpWcvF/oY6TUbcT4NM0DuGeUDGbkoDtRVebcW NouY6XvTvNGORAUIFia7q3NgEDOaev+BooZMTfiMStZmdnoIVwOCEAfnWRcsa+34r7 qu0mB2WFzJR4rnRseNDtJ6VBCz1pCXup4wH3lae1G3BDWHeI+umdj4q3Y2twqIwzMq lX/jeu8dIT9oXpqzZXGIf88bDGtlmTr60DNCMUEwZnv4s8XJEeOu6nGkUh8xbq0cDL ZzYz5OjPpD2xrTrc3M7D+uz/IhYdmGHjOwku22ui9TlMJN4OdsJ From: "brian m. carlson" To: Cc: Junio C Hamano , =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v3 32/44] t5500: make hash independent Date: Fri, 19 Jun 2020 17:55:49 +0000 Message-Id: <20200619175601.569856-33-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.278.ge193c7cf3a9 In-Reply-To: <20200619175601.569856-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200619175601.569856-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org This test has hard-coded pkt-lines with object IDs. The pkt-line lengths necessarily differ between hash algorithms, so generate these lines with the packetize helper so they're always the right size. In addition, we will require an object-format capability for SHA-256, so pass that capability on to the upload-pack process. Signed-off-by: brian m. carlson --- t/t5500-fetch-pack.sh | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/t/t5500-fetch-pack.sh b/t/t5500-fetch-pack.sh index 8c54e34ef1..dfed113247 100755 --- a/t/t5500-fetch-pack.sh +++ b/t/t5500-fetch-pack.sh @@ -871,9 +871,10 @@ test_expect_success 'shallow since with commit graph and already-seen commit' ' GIT_PROTOCOL=version=2 git upload-pack . <<-EOF >/dev/null 0012command=fetch + $(echo "object-format=$(test_oid algo)" | packetize) 00010013deepen-since 1 - 0032want $(git rev-parse other) - 0032have $(git rev-parse master) + $(echo "want $(git rev-parse other)" | packetize) + $(echo "have $(git rev-parse master)" | packetize) 0000 EOF ) From patchwork Fri Jun 19 17:55:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11614823 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D3987618 for ; Fri, 19 Jun 2020 17:56:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B1C2120776 for ; Fri, 19 Jun 2020 17:56:55 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="fMGArPas" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2404852AbgFSR4v (ORCPT ); Fri, 19 Jun 2020 13:56:51 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:39552 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2404496AbgFSR4g (ORCPT ); Fri, 19 Jun 2020 13:56:36 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:7d4e:cde:7c41:71c2]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 160E160A6D; Fri, 19 Jun 2020 17:56:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1592589390; bh=MWvvjYEo5Rp8KtFH0luFagYjxGUaEerSNE9nw0RTEhw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=fMGArPasp2HdkA0bbE1AYk2zs72buhZbqR5Dgu6UhQyhjOhaTBpAQmGTZHNiQ0cqq oKfo8CGGF8FS4ss8/1qzQErdWsKOksRrFVFCGoXIf+lCGK43SoXTUjTuf6BLt3fxvi ha0owgYoAgJSMULA7U1j6jVGcxej2c6Vi3DHIdwsaQR5n8bs5CK1ZS/YlCkr9+5u/T h4GsRSbYZAZb6dQUNTVd05MVK7xhoriNiv2Jh07GR03ca8G09TrwAtciczPskGlVdY WeZI+mm7TrTRS83QTc1dgQp7Yug8atH+ZhvP00JNycZsLt7deLM5HvECmkc26+lXM4 oNxhG8n9NOObJbEGn5crIV3xGsLkmYfFMOyRXYgiXfiDVoFH3vXbstljDuzNFuv3K9 nD9ZmpAVLcTaE++l6pgrNzFqT6OwPJkPj/xuXl2YyPnNGC+7GQGSc6apC3BLGqLS5f yhD6Pg9D6c6mVM/wxudpWZkNotVRP/I4KdiehMvNv9Wq4gyGFDs From: "brian m. carlson" To: Cc: Junio C Hamano , =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v3 33/44] builtin/ls-remote: initialize repository based on fetch Date: Fri, 19 Jun 2020 17:55:50 +0000 Message-Id: <20200619175601.569856-34-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.278.ge193c7cf3a9 In-Reply-To: <20200619175601.569856-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200619175601.569856-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org ls-remote may or may not operate within a repository, and as such will not have been initialized with the repository's hash algorithm. Even if it were, the remote side could be using a different algorithm and we would still want to display those refs properly. Find the hash algorithm used by the remote side by querying the transport object and set our hash algorithm accordingly. Without this change, if the remote side is using SHA-256, we truncate the refs to 40 hex characters, since that's the length of the default hash algorithm (SHA-1). Note that technically this is not a correct setting of the repository hash algorithm since, if we are in a repository, it might be one of a different hash algorithm from the remote side. However, our current code paths don't handle multiple algorithms and won't for some time, so this is the best we can do. We rely on the fact that ls-remote never modifies the current repository, which is a reasonable assumption to make. Signed-off-by: brian m. carlson --- builtin/ls-remote.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/builtin/ls-remote.c b/builtin/ls-remote.c index 6ef519514b..3a4dd12903 100644 --- a/builtin/ls-remote.c +++ b/builtin/ls-remote.c @@ -118,6 +118,10 @@ int cmd_ls_remote(int argc, const char **argv, const char *prefix) transport->server_options = &server_options; ref = transport_get_remote_refs(transport, &ref_prefixes); + if (ref) { + int hash_algo = hash_algo_by_ptr(transport_get_hash_algo(transport)); + repo_set_hash_algo(the_repository, hash_algo); + } if (transport_disconnect(transport)) { UNLEAK(sorting); return 1; From patchwork Fri Jun 19 17:55:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11614813 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2A936618 for ; Fri, 19 Jun 2020 17:56:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1193620DD4 for ; Fri, 19 Jun 2020 17:56:48 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="mkzvlWav" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2404757AbgFSR4q (ORCPT ); Fri, 19 Jun 2020 13:56:46 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:39596 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2404498AbgFSR4g (ORCPT ); Fri, 19 Jun 2020 13:56:36 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:7d4e:cde:7c41:71c2]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 8FA0260A7B; Fri, 19 Jun 2020 17:56:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1592589390; bh=TEKon/4etoHuZ6CSdrrFzoql3Vs+ljSPCfqhYISu0ns=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=mkzvlWavD+sGv/lvqJFlDInAkBm3uxLQOGaO8jqoNggAqBXx5WBXBNL40gfNkh+pr YnGIyIaO7oEhRbXVL01QsKrnq5UAbwr+BO9pfwBwa9Sudz6MccCgdNqnRks6GhgesK PBKpYrsssozNZYxYZmfvb8OAPWk9uhaYdZLNBai+PtLq9KG86UGFCtBONFYtfkNdSt Yx7SWwd/BY8/nlEQUxZPCZ1G0cejYleUu9f47eOXhsNGzcQAGwI1vkDTUXIjld/Dx6 YWhtnrR/9yd+q6zR/g79pkGemgo/SvP7eT9Eak2+ArQKfDtOtKlu7iu8ye3mcGX71s Fd4BtatlquajbHYyJq0fVcSSYi4byOYqck3HpJAfy/pStNOkL1Dz8fNAMRc6Lz/jPj ephsm5WXnTw6fQMj5Z0GhWQIKMyz9FY8Qd4Uma7d5KIMWSWo/ov1qvBYMiE4I6F7rR OvhvL2wFN7A7ug9YV4XXg+p7aiP09T94tHEvI0MN0OwGj8n3Xnv From: "brian m. carlson" To: Cc: Junio C Hamano , =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v3 34/44] remote-curl: detect algorithm for dumb HTTP by size Date: Fri, 19 Jun 2020 17:55:51 +0000 Message-Id: <20200619175601.569856-35-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.278.ge193c7cf3a9 In-Reply-To: <20200619175601.569856-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200619175601.569856-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When reading the info/refs file for a repository, we have no explicit way to detect which hash algorithm is in use because the file doesn't provide one. Detect the hash algorithm in use by the size of the first object ID. If we have an empty repository, we don't know what the hash algorithm is on the remote side, so default to whatever the local side has configured. Without doing this, we cannot clone an empty repository since we don't know its hash algorithm. Test this case appropriately, since we currently have no tests for cloning an empty repository with the dumb HTTP protocol. We anonymize the URL like elsewhere in the function in case the user has decided to include a secret in the URL. Signed-off-by: brian m. carlson --- remote-curl.c | 23 +++++++++++++++++++++-- t/t5550-http-fetch-dumb.sh | 18 ++++++++++++++++++ 2 files changed, 39 insertions(+), 2 deletions(-) diff --git a/remote-curl.c b/remote-curl.c index f0203547c5..e666845d9d 100644 --- a/remote-curl.c +++ b/remote-curl.c @@ -252,6 +252,19 @@ static struct ref *parse_git_refs(struct discovery *heads, int for_push) return list; } +static const struct git_hash_algo *detect_hash_algo(struct discovery *heads) +{ + const char *p = memchr(heads->buf, '\t', heads->len); + int algo; + if (!p) + return the_hash_algo; + + algo = hash_algo_by_length((p - heads->buf) / 2); + if (algo == GIT_HASH_UNKNOWN) + return NULL; + return &hash_algos[algo]; +} + static struct ref *parse_info_refs(struct discovery *heads) { char *data, *start, *mid; @@ -262,6 +275,12 @@ static struct ref *parse_info_refs(struct discovery *heads) struct ref *ref = NULL; struct ref *last_ref = NULL; + options.hash_algo = detect_hash_algo(heads); + if (!options.hash_algo) + die("%sinfo/refs not valid: could not determine hash algorithm; " + "is this a git repository?", + transport_anonymize_url(url.buf)); + data = heads->buf; start = NULL; mid = data; @@ -272,13 +291,13 @@ static struct ref *parse_info_refs(struct discovery *heads) if (data[i] == '\t') mid = &data[i]; if (data[i] == '\n') { - if (mid - start != the_hash_algo->hexsz) + if (mid - start != options.hash_algo->hexsz) die(_("%sinfo/refs not valid: is this a git repository?"), transport_anonymize_url(url.buf)); data[i] = 0; ref_name = mid + 1; ref = alloc_ref(ref_name); - get_oid_hex(start, &ref->old_oid); + get_oid_hex_algop(start, &ref->old_oid, options.hash_algo); if (!refs) refs = ref; if (last_ref) diff --git a/t/t5550-http-fetch-dumb.sh b/t/t5550-http-fetch-dumb.sh index 50485300eb..e57716bacd 100755 --- a/t/t5550-http-fetch-dumb.sh +++ b/t/t5550-http-fetch-dumb.sh @@ -50,6 +50,24 @@ test_expect_success 'create password-protected repository' ' "$HTTPD_DOCUMENT_ROOT_PATH/auth/dumb/repo.git" ' +test_expect_success 'create empty remote repository' ' + git init --bare "$HTTPD_DOCUMENT_ROOT_PATH/empty.git" && + (cd "$HTTPD_DOCUMENT_ROOT_PATH/empty.git" && + mkdir -p hooks && + write_script "hooks/post-update" <<-\EOF && + exec git update-server-info + EOF + hooks/post-update + ) +' + +test_expect_success 'empty dumb HTTP repository has default hash algorithm' ' + test_when_finished "rm -fr clone-empty" && + git clone $HTTPD_URL/dumb/empty.git clone-empty && + git -C clone-empty rev-parse --show-object-format >empty-format && + test "$(cat empty-format)" = "$(test_oid algo)" +' + setup_askpass_helper test_expect_success 'cloning password-protected repository can fail' ' From patchwork Fri Jun 19 17:55:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11614853 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8BFDF618 for ; Fri, 19 Jun 2020 17:57:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 69DFD20776 for ; Fri, 19 Jun 2020 17:57:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="DO4jS+4L" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2436564AbgFSR5f (ORCPT ); Fri, 19 Jun 2020 13:57:35 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:39446 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2404581AbgFSR4h (ORCPT ); Fri, 19 Jun 2020 13:56:37 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:7d4e:cde:7c41:71c2]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 1788660A6E; Fri, 19 Jun 2020 17:56:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1592589391; bh=wSBvyjey21gbVX/7xAGCthcffF4q4jyfM63IKoHZXnw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=DO4jS+4L1L/AxhqgRdZNxA8SngmpyMRCmbEPNBvAULIlQJS86l5YMlT+vL0CxmOV9 T6ODUVefOlfEyS8idxE1P3UshNcm52xd0lku12lzzgimFwBXslpvPw8FoIunMROl5p BQDpzHZe00eGiZUgD/HEuXNPU18Oyc7yAiHO1+reElcerWQ9+GoZZV7xeCr8ul2paj 2vlEMeiLVhoUpLIBJvasxsLdwEZueDAJ7kUKexByQRRoYc2W4KGzU8Ig6tToloix59 Xg8OO+M/GhgtfXJLcZxc0QX8KbWgNjmGhh9GeTOsx6+8oIr4UWhuku0Pb9DmN9GFCh sgiUih7qpNCOl+v3KAPQKTaVl/StFOUSQkoRRi2urZ7W4BMs+q7MwKqabZISXS2IOo bpgVhQXZRHBlj33ASp+Y34X4XEzM/LME8OqZUTinSaOso8Jv5aBWt2HKgg1B2s1oqH zygVwu+I5H2vWTWtSU+DhqthEcCoFFCTCt2qOBVXfav/jTuvBM2 From: "brian m. carlson" To: Cc: Junio C Hamano , =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v3 35/44] builtin/index-pack: add option to specify hash algorithm Date: Fri, 19 Jun 2020 17:55:52 +0000 Message-Id: <20200619175601.569856-36-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.278.ge193c7cf3a9 In-Reply-To: <20200619175601.569856-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200619175601.569856-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org git index-pack is usually run in a repository, but need not be. Since packs don't contains information on the algorithm in use, instead relying on context, add an option to index-pack to tell it which one we're using in case someone runs it outside of a repository. Since using --stdin necessarily implies a repository, don't allow specifying an object format if it's provided to prevent users from passing an option that won't work. Add documentation for this option. Signed-off-by: brian m. carlson --- Documentation/git-index-pack.txt | 8 ++++++++ builtin/index-pack.c | 8 ++++++++ 2 files changed, 16 insertions(+) diff --git a/Documentation/git-index-pack.txt b/Documentation/git-index-pack.txt index d5b7560bfe..9316d9a80b 100644 --- a/Documentation/git-index-pack.txt +++ b/Documentation/git-index-pack.txt @@ -93,6 +93,14 @@ OPTIONS --max-input-size=:: Die, if the pack is larger than . +--object-format=:: + Specify the given object format (hash algorithm) for the pack. The valid + values are 'sha1' and (if enabled) 'sha256'. The default is the algorithm for + the current repository (set by `extensions.objectFormat`), or 'sha1' if no + value is set or outside a repository. ++ +This option cannot be used with --stdin. + NOTES ----- diff --git a/builtin/index-pack.c b/builtin/index-pack.c index 7bea1fba52..f865666db9 100644 --- a/builtin/index-pack.c +++ b/builtin/index-pack.c @@ -1667,6 +1667,7 @@ int cmd_index_pack(int argc, const char **argv, const char *prefix) unsigned char pack_hash[GIT_MAX_RAWSZ]; unsigned foreign_nr = 1; /* zero is a "good" value, assume bad */ int report_end_of_input = 0; + int hash_algo = 0; /* * index-pack never needs to fetch missing objects except when @@ -1760,6 +1761,11 @@ int cmd_index_pack(int argc, const char **argv, const char *prefix) die(_("bad %s"), arg); } else if (skip_prefix(arg, "--max-input-size=", &arg)) { max_input_size = strtoumax(arg, NULL, 10); + } else if (skip_prefix(arg, "--object-format=", &arg)) { + hash_algo = hash_algo_by_name(arg); + if (hash_algo == GIT_HASH_UNKNOWN) + die(_("unknown hash algorithm '%s'"), arg); + repo_set_hash_algo(the_repository, hash_algo); } else usage(index_pack_usage); continue; @@ -1776,6 +1782,8 @@ int cmd_index_pack(int argc, const char **argv, const char *prefix) die(_("--fix-thin cannot be used without --stdin")); if (from_stdin && !startup_info->have_repository) die(_("--stdin requires a git repository")); + if (from_stdin && hash_algo) + die(_("--object-format cannot be used with --stdin")); if (!index_name && pack_name) index_name = derive_filename(pack_name, "idx", &index_name_buf); From patchwork Fri Jun 19 17:55:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11614861 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8FE8F13A0 for ; Fri, 19 Jun 2020 17:57:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6807621527 for ; Fri, 19 Jun 2020 17:57:39 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="sRT5PH0G" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2393787AbgFSR5c (ORCPT ); Fri, 19 Jun 2020 13:57:32 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:39554 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2404568AbgFSR4l (ORCPT ); Fri, 19 Jun 2020 13:56:41 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:7d4e:cde:7c41:71c2]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 9072B60A70; Fri, 19 Jun 2020 17:56:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1592589391; bh=QhWwx5JrFjkmi9fYf3BLeqi+AU9uqTKM8zuKL+dV4ZA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=sRT5PH0GRSvX9ZtP2neSToPeVAOl5rv3xTrhLPAoHOI56HjWgujEZ+F49xnazoDOR l7GUrLPVd491RrVwe4R0aYCis2Vl0CCqomcpG2V5zEi82ftz4ea3VQiw5e+7rHiWgX 3YNaNgdMCWBjhHdOkpSyv1TyZjfhIsXy8I/jGFx3t2E2rv46INOSp6VjhVig/GmgD5 aqkx0Sazsa1BjNO4qBN7QM6/biLSUnP5qi9jqxa4o5igxfh0cBy0eeQISRYkkQU3Oh j9cF1QyKVVvm54TO6NBXL//SVlxndMOdI1FBHbPnIPLcqL2yjZRCuG0r5qLhUTBe5L uMc0g7ygMfeu3L2q5OgZ1EGRRCSoGlzmplenIwrJ2kayoj7BPjAqCxfHWiRJ0oFj3o Q0rmQtEcjLBZoh1c9u1eDOkwKFgaYrByehARMIuMyUcGg0fyJBR6WhTspx5ovj84ud 05rjgnzvyfQo+D0Si1KGRYFzFCbvW2rFvzM+EH8srDOwXIyphJ0 From: "brian m. carlson" To: Cc: Junio C Hamano , =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v3 36/44] t1050: pass algorithm to index-pack when outside repo Date: Fri, 19 Jun 2020 17:55:53 +0000 Message-Id: <20200619175601.569856-37-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.278.ge193c7cf3a9 In-Reply-To: <20200619175601.569856-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200619175601.569856-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When outside a repository, git index-pack is unable to guess the hash algorithm in use for a pack, since packs don't contain any information on the algorithm in use. Pass an option to index-pack to help it out in this test. Signed-off-by: brian m. carlson --- t/t1050-large.sh | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/t/t1050-large.sh b/t/t1050-large.sh index 7f88ea07c2..6a56d1ca24 100755 --- a/t/t1050-large.sh +++ b/t/t1050-large.sh @@ -12,6 +12,7 @@ file_size () { } test_expect_success setup ' + test_oid_init && # clone does not allow us to pass core.bigfilethreshold to # new repos, so set core.bigfilethreshold globally git config --global core.bigfilethreshold 200k && @@ -177,7 +178,8 @@ test_expect_success 'git-show a large file' ' test_expect_success 'index-pack' ' git clone file://"$(pwd)"/.git foo && - GIT_DIR=non-existent git index-pack --strict --verify foo/.git/objects/pack/*.pack + GIT_DIR=non-existent git index-pack --object-format=$(test_oid algo) \ + --strict --verify foo/.git/objects/pack/*.pack ' test_expect_success 'repack' ' From patchwork Fri Jun 19 17:55:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11614821 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3600E618 for ; Fri, 19 Jun 2020 17:56:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 12FFA21531 for ; Fri, 19 Jun 2020 17:56:51 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="w8sGFVO+" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2392011AbgFSR4t (ORCPT ); Fri, 19 Jun 2020 13:56:49 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:39560 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2404585AbgFSR4h (ORCPT ); Fri, 19 Jun 2020 13:56:37 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:7d4e:cde:7c41:71c2]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 1491360A7C; Fri, 19 Jun 2020 17:56:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1592589392; bh=9/XCfoD9D8bLE+oXVKT1bfTngQ0zsWd7xXxEJCNWFvs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=w8sGFVO+s9/ANQ+ONL6BgzycPXT4RflGYlaodsT47/C2YyT1WDAV8eYw9DxCUNxZO TPpBGqb314VzIuGT0MVIRuuoJWhW3Hxv0XQW8EvC3OaOOEu4YC6wmr7kbPtuvlZjxD EifToHGeHZsTKhnVX0oXE2P4kP9v57H5g/LCgynZ+F5/qm4KQRvrf4N6Quo9jZNJNQ nSEZz9PVTH63DtT/IvfJM4mkyQ/pvVJhENfMOxRqL2MUZL1lgtiNhD14tD/XZhzw8N dfpRrGtyXB1uALTbZkao2DfNwfEjgaN3ClZlcWFs2EX5fYdumz/R2QZ4UoxbU/OD/a k2r4GGdGmy2m20a93wBe2yarYJlh9Gmhr5uNCPxa8qb5hpY8PmTGvSP5e7EXktZGUf 1j9fy5SZOtc0qgH/0LPA3UWZgLt7qGBQlesQEJqhP8u+1J0TTR2kms0rK0XKcNbwkw m89DOvRmtZB8fuQvYDC61/5WOoXWu1dP+p57YMmFcN56i4n4EUa From: "brian m. carlson" To: Cc: Junio C Hamano , =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v3 37/44] remote-curl: avoid truncating refs with ls-remote Date: Fri, 19 Jun 2020 17:55:54 +0000 Message-Id: <20200619175601.569856-38-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.278.ge193c7cf3a9 In-Reply-To: <20200619175601.569856-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200619175601.569856-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Normally, the remote-curl transport helper is aware of the hash algorithm we're using because we're in a repo with the appropriate hash algorithm set. However, when using git ls-remote outside of a repository, we won't have initialized the hash algorithm properly, so use hash_to_hex_algop to print the ref corresponding to the algorithm we've detected. Signed-off-by: brian m. carlson --- remote-curl.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/remote-curl.c b/remote-curl.c index e666845d9d..5cbc6e5002 100644 --- a/remote-curl.c +++ b/remote-curl.c @@ -548,7 +548,9 @@ static void output_refs(struct ref *refs) if (posn->symref) printf("@%s %s\n", posn->symref, posn->name); else - printf("%s %s\n", oid_to_hex(&posn->old_oid), posn->name); + printf("%s %s\n", hash_to_hex_algop(posn->old_oid.hash, + options.hash_algo), + posn->name); } printf("\n"); fflush(stdout); From patchwork Fri Jun 19 17:55:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11614859 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 658E1618 for ; Fri, 19 Jun 2020 17:57:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4C6D520DD4 for ; Fri, 19 Jun 2020 17:57:38 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="AyzawqrG" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2436518AbgFSR5d (ORCPT ); Fri, 19 Jun 2020 13:57:33 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:39524 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2404587AbgFSR4k (ORCPT ); Fri, 19 Jun 2020 13:56:40 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:7d4e:cde:7c41:71c2]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 8E6BC60A7D; Fri, 19 Jun 2020 17:56:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1592589393; bh=2AW4Ophh+GDqfkkvEpKmw/NRFQar3284kGJCZ5OkO5M=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=AyzawqrGLUKR9szMQGA5M0mQsqv6qjHpg+T8yYFx5+rrTkaafSc2jp/UOAgV7uJ8Z UwCWKrwm0aL/wiRbp900g9X6PcHwmfAP/I97yd9nYDufPhwe2I/uJy+sxaUAi8lkSh Sl2wDMjiav+Mcf1/ygmeb4lI24w1rPw/JazYwjDAJKkJTX9qCsALmhdjwOaJiDH9VX E2dILBiaqAdO/gQpoJPJncAertRm2hsCPG5XgG4/a0JYlkOLv1eFpxAazPUDyMuoXY y5Ush/jiQdnotejI9C7WVZ3plOkP9JaGw+SbfErvd/kDu06e3DNwy6bATRtCe//VxP mJ8G0jVDRQDucpgRpr/bC50RrG0neKJq0rLTX7veGTPi+eKSNLly2WRSNoJycxKcup im4IELvzEs0UQHjSbG8lWPtQKFL4LccoDE6Y6818qYknWa02aqoNUwqeOYXlkJAMYG r2D5FTUyGTSB99zZADTdeykSBB0Ts2KtEcxT9f1c6Ssdd65vFKY From: "brian m. carlson" To: Cc: Junio C Hamano , =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v3 38/44] t/helper: initialize the repository for test-sha1-array Date: Fri, 19 Jun 2020 17:55:55 +0000 Message-Id: <20200619175601.569856-39-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.278.ge193c7cf3a9 In-Reply-To: <20200619175601.569856-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200619175601.569856-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org test-sha1-array uses the_hash_algo under the hood. Since t0064 wants to use the value that is correct for the hash algorithm that we're testing, make sure the test helper initializes the repository to set the_hash_algo correctly. Signed-off-by: brian m. carlson --- t/helper/test-oid-array.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/t/helper/test-oid-array.c b/t/helper/test-oid-array.c index ce9fd5f091..b16cd0b11b 100644 --- a/t/helper/test-oid-array.c +++ b/t/helper/test-oid-array.c @@ -12,6 +12,9 @@ int cmd__oid_array(int argc, const char **argv) { struct oid_array array = OID_ARRAY_INIT; struct strbuf line = STRBUF_INIT; + int nongit_ok; + + setup_git_directory_gently(&nongit_ok); while (strbuf_getline(&line, stdin) != EOF) { const char *arg; From patchwork Fri Jun 19 17:55:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11614827 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 205FC618 for ; Fri, 19 Jun 2020 17:56:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0610520DD4 for ; Fri, 19 Jun 2020 17:56:59 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="SU3Nrll5" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2394116AbgFSR46 (ORCPT ); Fri, 19 Jun 2020 13:56:58 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:39508 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2404586AbgFSR4i (ORCPT ); Fri, 19 Jun 2020 13:56:38 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:7d4e:cde:7c41:71c2]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 2B8EC60A74; Fri, 19 Jun 2020 17:56:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1592589393; bh=POwUn8s/1JQRZ58gbQ+WBHHk/Mx/QgCRkJQxxE8tgNg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=SU3Nrll5ux/q92tWrFGJKNvAjny+l4p49Vw6Jnfi//EJY4c2dZQzik3FpD11utYxi 0ZRbz5fBlfi2pCfsS+NnUMlI1Coq52qfovBr9a2W9sfFUDD8welHUdcBF+1KIPGlgA jPhdNUzCJzL7dL8dCt0wwuhE2jX0SMngQ46zjvmja8upEwPbLmlT1urUsmRI37PoK/ 81Hj+ZqTEh0pfIQ04uwCTt+WF1Joyr4djm9c75mqVoeCXwY+66hXDevRCKDExAU6PO TYpUmXw031RTsYcWG4I4JYZGl9UFO1kePGVaot/R+egdh0ZIonixDvOJksrz528FOv ImYy5dnlFMI1Pf4eHrDuYrGD9GGelWbMs5ci/aPNK04ZRF0pNLw4m5sGnheRjFWSpU Nj8BFOx6rd/e3CRuUPN0PPMWjoGs12fN+SdXhMetpxtZvfAJJPGwc37XxS+ZD1QTIJ F9PkqV2FTEwVE3KgqgsnOQHFZEWYCBE9Mg4vrY/zzdh2ojXMJVj From: "brian m. carlson" To: Cc: Junio C Hamano , =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v3 39/44] t5702: offer an object-format capability in the test Date: Fri, 19 Jun 2020 17:55:56 +0000 Message-Id: <20200619175601.569856-40-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.278.ge193c7cf3a9 In-Reply-To: <20200619175601.569856-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200619175601.569856-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In order to make this test work with SHA-256, offer an object-format capability so that both sides use the same algorithm. Signed-off-by: brian m. carlson --- t/t5702-protocol-v2.sh | 2 ++ 1 file changed, 2 insertions(+) diff --git a/t/t5702-protocol-v2.sh b/t/t5702-protocol-v2.sh index 8da65e60de..63f425bbad 100755 --- a/t/t5702-protocol-v2.sh +++ b/t/t5702-protocol-v2.sh @@ -13,6 +13,7 @@ start_git_daemon --export-all --enable=receive-pack daemon_parent=$GIT_DAEMON_DOCUMENT_ROOT_PATH/parent test_expect_success 'create repo to be served by git-daemon' ' + test_oid_init && git init "$daemon_parent" && test_commit -C "$daemon_parent" one ' @@ -394,6 +395,7 @@ test_expect_success 'even with handcrafted request, filter does not work if not # Custom request that tries to filter even though it is not advertised. test-tool pkt-line pack >in <<-EOF && command=fetch + object-format=$(test_oid algo) 0001 want $(git -C server rev-parse master) filter blob:none From patchwork Fri Jun 19 17:55:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11614857 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 32B551731 for ; Fri, 19 Jun 2020 17:57:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1B9F521527 for ; Fri, 19 Jun 2020 17:57:38 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="TU2zNGgD" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2436556AbgFSR5e (ORCPT ); Fri, 19 Jun 2020 13:57:34 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:39526 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390614AbgFSR4k (ORCPT ); Fri, 19 Jun 2020 13:56:40 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:7d4e:cde:7c41:71c2]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id A4BC960A71; Fri, 19 Jun 2020 17:56:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1592589394; bh=uDVYhPzZg5oVXaxsTp1TA02b1tBgrRU+5xBUU4TII00=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=TU2zNGgDHfvkewJU4+T9PGhX74jq6cnEniyUjwGAx+Yq3jJVSWGHXsKtJxnA2VgF8 OwhtGSy6bpDko3G/0rf3Cx1J99gCVwBaf1KcDIiL1uyla5hFgMVuxYOSKd0bFZ+p51 TETgZIckVrJhytNOU1TWmVcqjqHQqOsZOrglM18MMLDG67nIh+SCIdk/CVaBsripeq gnbbJm+GxE+dWbNB9ue7f1oq2UUtEHSBALnRM1vDhMERudRflYiv5PJ3eAwQIlo58I qbIR3P8jYGqa38N7Xbp9dxcdTEyF/y1bR9dHOlzwhe9YdBHW4AKjN2WisFyeb8PDml BoAXtyN12Neqb5G1jHUwg1z0RtkqXqQvv1J8iJLhExrulQdW3h//hW18e/z9lXwvaT kf+W6cio1SdOLuDTNrdBFxFMUCbgLd+ngfG9HzLVoR11NIJhAJuASz+OZ4kc4T9vGc eIVysORicuQzy/0+iKVMtWUMGXGG3D8nmsjTmXXNfSkUIuAghRj From: "brian m. carlson" To: Cc: Junio C Hamano , =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v3 40/44] t5703: use object-format serve option Date: Fri, 19 Jun 2020 17:55:57 +0000 Message-Id: <20200619175601.569856-41-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.278.ge193c7cf3a9 In-Reply-To: <20200619175601.569856-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200619175601.569856-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When we're using an algorithm other than SHA-1, we need to specify the algorithm in use so we don't get a failure with an "unknown format" message. Add a wrapper function that specifies this header if required. Skip specifying this header for SHA-1 to test that it works both with an without this header. Signed-off-by: brian m. carlson --- t/t5703-upload-pack-ref-in-want.sh | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/t/t5703-upload-pack-ref-in-want.sh b/t/t5703-upload-pack-ref-in-want.sh index 92ad5eeec0..748282f058 100755 --- a/t/t5703-upload-pack-ref-in-want.sh +++ b/t/t5703-upload-pack-ref-in-want.sh @@ -27,6 +27,15 @@ check_output () { test_cmp sorted_commits actual_commits } +write_command () { + echo "command=$1" + + if test "$(test_oid algo)" != sha1 + then + echo "object-format=$(test_oid algo)" + fi +} + # c(o/foo) d(o/bar) # \ / # b e(baz) f(master) @@ -65,7 +74,7 @@ test_expect_success 'config controls ref-in-want advertisement' ' test_expect_success 'invalid want-ref line' ' test-tool pkt-line pack >in <<-EOF && - command=fetch + $(write_command fetch) 0001 no-progress want-ref refs/heads/non-existent @@ -86,7 +95,7 @@ test_expect_success 'basic want-ref' ' oid=$(git rev-parse a) && test-tool pkt-line pack >in <<-EOF && - command=fetch + $(write_command fetch) 0001 no-progress want-ref refs/heads/master @@ -110,7 +119,7 @@ test_expect_success 'multiple want-ref lines' ' oid=$(git rev-parse b) && test-tool pkt-line pack >in <<-EOF && - command=fetch + $(write_command fetch) 0001 no-progress want-ref refs/heads/o/foo @@ -132,7 +141,7 @@ test_expect_success 'mix want and want-ref' ' git rev-parse e f >expected_commits && test-tool pkt-line pack >in <<-EOF && - command=fetch + $(write_command fetch) 0001 no-progress want-ref refs/heads/master @@ -155,7 +164,7 @@ test_expect_success 'want-ref with ref we already have commit for' ' oid=$(git rev-parse c) && test-tool pkt-line pack >in <<-EOF && - command=fetch + $(write_command fetch) 0001 no-progress want-ref refs/heads/o/foo From patchwork Fri Jun 19 17:55:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11614829 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5F7F313A0 for ; Fri, 19 Jun 2020 17:57:01 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 47B1F21527 for ; Fri, 19 Jun 2020 17:57:01 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="nzbWV+7S" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2436489AbgFSR47 (ORCPT ); Fri, 19 Jun 2020 13:56:59 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:39596 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2393787AbgFSR4l (ORCPT ); Fri, 19 Jun 2020 13:56:41 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:7d4e:cde:7c41:71c2]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 297A560A73; Fri, 19 Jun 2020 17:56:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1592589394; bh=hYAhEgO21Q1ObbuQ8R0buMjNd5pGl8EufAeC4U72FN8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=nzbWV+7S2WX374uEW2Wh/yIjnIUWX8/tlH54Oaz4pIRPAhBTVcFXrJYDzOwC3EL+C rx5MxzyYZ7VRS3+hs1FXqEBuKhYfnFXYgNh9dZGOhCiFcoTyiXTDJPsr/PYWrN3W5c 97fE5FgYtbwS0Vsd5zZbffdyvIXE3V6NB7uB9qppO3V83R8cA5dIQspYDcmaIjn3Qk R1VN2ppOol9wpH/DBiF+gVgYuSGGcFqeY3CFCd0caYsDOMsmorXDIZgPAWYCK76zO5 GEmy/HyPBrN2ZrMqdzmRbCvORD9u0ffGmKICnTub+Gjy1eZ57flnOopV4T/FCrWVwK b9sekgPKEyr/Khjz+gyJU//PoWS7+id4UeeURQIrvEq2OGxd1xvIicxBoNrfoqhkW/ mQdAmXGGY7kohHO1DpJRULdfxNw0ylSi+dRnM7lnu2fFxnCHQQ7OAzR6L1gVlN/ivn IdfC1P+0xIE3uXoijgZK8QlOY5LgJ6kkm541n9QuFYvkW5ZiWPq From: "brian m. carlson" To: Cc: Junio C Hamano , =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v3 41/44] t5704: send object-format capability with SHA-256 Date: Fri, 19 Jun 2020 17:55:58 +0000 Message-Id: <20200619175601.569856-42-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.278.ge193c7cf3a9 In-Reply-To: <20200619175601.569856-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200619175601.569856-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When we speak protocol v2 in this test, we must pass the object-format header if the algorithm is not SHA-1. Otherwise, git upload-pack fails because the hash algorithm doesn't match and not because we've failed to speak the protocol correctly. Pass the header so that our assertions test what we're really interested in. Signed-off-by: brian m. carlson --- t/t5704-protocol-violations.sh | 2 ++ 1 file changed, 2 insertions(+) diff --git a/t/t5704-protocol-violations.sh b/t/t5704-protocol-violations.sh index 950cfb21fe..5c941949b9 100755 --- a/t/t5704-protocol-violations.sh +++ b/t/t5704-protocol-violations.sh @@ -9,6 +9,7 @@ making sure that we do not segfault or otherwise behave badly.' test_expect_success 'extra delim packet in v2 ls-refs args' ' { packetize command=ls-refs && + packetize "object-format=$(test_oid algo)" && printf 0001 && # protocol expects 0000 flush here printf 0001 @@ -21,6 +22,7 @@ test_expect_success 'extra delim packet in v2 ls-refs args' ' test_expect_success 'extra delim packet in v2 fetch args' ' { packetize command=fetch && + packetize "object-format=$(test_oid algo)" && printf 0001 && # protocol expects 0000 flush here printf 0001 From patchwork Fri Jun 19 17:55:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11614863 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id ACD061731 for ; Fri, 19 Jun 2020 17:57:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9214120776 for ; Fri, 19 Jun 2020 17:57:39 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="rgAsq1ZY" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2394162AbgFSR5c (ORCPT ); Fri, 19 Jun 2020 13:57:32 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:39458 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2394009AbgFSR4l (ORCPT ); Fri, 19 Jun 2020 13:56:41 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:7d4e:cde:7c41:71c2]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id A242560A7E; Fri, 19 Jun 2020 17:56:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1592589395; bh=0kTnW+si/14E4LvMgtbMNpi0j1rIx5tw+Yjkj/LLNLQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=rgAsq1ZYKLEcTp2nsBf8ziNC/hKH/VKkKd7k0sYFZW8HbN/9vUJoK0ihj47tBbliF kb7i+EdFMxQr58sVTg3/qyYahsdXZmJ6q0VCM4JSqS92J//B2UsYPlKk59tJehdq4H gqIgBoDETj92iR5B2gTF2ndXVlZXtpGEOwzH1Ku5FU/DrYNQ/Q87K8zGWyYBKkz/Pd NaYJXx5Fws2M8Yx7LdpkuWFxl81YQPsKjeeTr2xdvEQ2vh01bQ9dwFvKcjP/x1Ydga BO+eEgrPY/8fW58UJEj6lh+2HQ7TVI2dpB1R33AS8FQkvCx7CEz/teBzf7o4PM9o3/ hpdFugpjpkbfsvAtKW/BTXcsdt8uZgUjOK6kePGYKa10Yihs5SkHxPMOKLSF8hwaq5 nq1CeKfETFvJlKhvwBfDKTyUx4G8ol0D3rSQhIofAWyWq142vApbp2IRE6P1kHnDRR RNvBTqMhfu/rpSr/97tr1CNPxPXLbO6/dIsoxxX2rrsFw0UtSg/ From: "brian m. carlson" To: Cc: Junio C Hamano , =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v3 42/44] t5300: pass --object-format to git index-pack Date: Fri, 19 Jun 2020 17:55:59 +0000 Message-Id: <20200619175601.569856-43-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.278.ge193c7cf3a9 In-Reply-To: <20200619175601.569856-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200619175601.569856-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org git index-pack by default reads the repository to determine the object format. However, when outside of a repository, it's necessary to specify the hash algorithm in use so that the pack can be properly indexed. Add an --object-format argument when invoking git index-pack outside of a repository. Signed-off-by: brian m. carlson --- t/t5300-pack-object.sh | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/t/t5300-pack-object.sh b/t/t5300-pack-object.sh index 410a09b0dd..746cdb626e 100755 --- a/t/t5300-pack-object.sh +++ b/t/t5300-pack-object.sh @@ -12,7 +12,8 @@ TRASH=$(pwd) test_expect_success \ 'setup' \ - 'rm -f .git/index* && + 'test_oid_init && + rm -f .git/index* && perl -e "print \"a\" x 4096;" > a && perl -e "print \"b\" x 4096;" > b && perl -e "print \"c\" x 4096;" > c && @@ -412,18 +413,18 @@ test_expect_success 'set up pack for non-repo tests' ' ' test_expect_success 'index-pack --stdin complains of non-repo' ' - nongit test_must_fail git index-pack --stdin works in non-repo' ' - nongit git index-pack ../foo.pack && + nongit git index-pack --object-format=$(test_oid algo) ../foo.pack && test_path_is_file foo.idx ' test_expect_success 'index-pack --strict works in non-repo' ' rm -f foo.idx && - nongit git index-pack --strict ../foo.pack && + nongit git index-pack --strict --object-format=$(test_oid algo) ../foo.pack && test_path_is_file foo.idx ' From patchwork Fri Jun 19 17:56:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11614849 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 73FAF13A0 for ; Fri, 19 Jun 2020 17:57:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5BB4520776 for ; Fri, 19 Jun 2020 17:57:28 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="nON9jFYG" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2404868AbgFSR50 (ORCPT ); Fri, 19 Jun 2020 13:57:26 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:39572 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2404667AbgFSR4p (ORCPT ); Fri, 19 Jun 2020 13:56:45 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:7d4e:cde:7c41:71c2]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 2908360A7F; Fri, 19 Jun 2020 17:56:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1592589395; bh=n4cOXLIJFJySsqXcnB6iIuq2selM+4nlbmqMC5jy41g=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=nON9jFYGQFy2ptGg6hEj1zKlQhDDpIOq45e1o/LB5tfRWlPydPCLHDwi4wx8BYZB2 fNLjGfH6li6S5VulqM6QXfc7lbSHNPc+vWdWb5vaQaPY1sYe+3sD/cYpJUO4g6jyUA 1wpph0PVfuD4OmUa4oTQAqlfdVBV3B16BYSF8sqV6J5nXrspiTK2Qm46wCe6zfiGU+ Nwr3J+h0E7vCY+cWu2zIx9heNuEYdHughfAwOxRwuwTgGQwXmfz7SYUAVeEoCIrhia fN7/E4YMbsIq9ECgpo8fAdvPF4FaIul36VBr3y/8P2HI+LRNwnd0a2R+40+SyRUx6g 11a/W7iPkexCmm59/33DldZTRpG4IycP6/bGRvDHWpUp69KXHSpZSpt/HP4BcbNKtZ EfUYdruvfrCICxmH4c4iKX2ktDe8VLyuCFkMxFo+v5cMzn1u3sSPNmASwmiGN/ihsI IDFtP19m5kEBm/Oqrs8rbu4Imz6SYlqX1c+aC6EPNS/EgZ0iePG From: "brian m. carlson" To: Cc: Junio C Hamano , =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v3 43/44] bundle: detect hash algorithm when reading refs Date: Fri, 19 Jun 2020 17:56:00 +0000 Message-Id: <20200619175601.569856-44-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.278.ge193c7cf3a9 In-Reply-To: <20200619175601.569856-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200619175601.569856-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Much like with the dumb HTTP transport, there isn't a way to explicitly specify the hash algorithm when dealing with a bundle, so detect the algorithm based on the length of the object IDs in the prerequisites and ref advertisements. Signed-off-by: brian m. carlson --- bundle.c | 22 +++++++++++++++++++++- bundle.h | 1 + transport.c | 10 ++++++++-- 3 files changed, 30 insertions(+), 3 deletions(-) diff --git a/bundle.c b/bundle.c index 99439e07a1..2a0d744d3f 100644 --- a/bundle.c +++ b/bundle.c @@ -23,6 +23,17 @@ static void add_to_ref_list(const struct object_id *oid, const char *name, list->nr++; } +static const struct git_hash_algo *detect_hash_algo(struct strbuf *buf) +{ + size_t len = strcspn(buf->buf, " \n"); + int algo; + + algo = hash_algo_by_length(len / 2); + if (algo == GIT_HASH_UNKNOWN) + return NULL; + return &hash_algos[algo]; +} + static int parse_bundle_header(int fd, struct bundle_header *header, const char *report_path) { @@ -52,12 +63,21 @@ static int parse_bundle_header(int fd, struct bundle_header *header, } strbuf_rtrim(&buf); + if (!header->hash_algo) { + header->hash_algo = detect_hash_algo(&buf); + if (!header->hash_algo) { + error(_("unknown hash algorithm length")); + status = -1; + break; + } + } + /* * Tip lines have object name, SP, and refname. * Prerequisites have object name that is optionally * followed by SP and subject line. */ - if (parse_oid_hex(buf.buf, &oid, &p) || + if (parse_oid_hex_algop(buf.buf, &oid, &p, header->hash_algo) || (*p && !isspace(*p)) || (!is_prereq && !*p)) { if (report_path) diff --git a/bundle.h b/bundle.h index ceab0c7475..2dc9442024 100644 --- a/bundle.h +++ b/bundle.h @@ -15,6 +15,7 @@ struct ref_list { struct bundle_header { struct ref_list prerequisites; struct ref_list references; + const struct git_hash_algo *hash_algo; }; int is_bundle(const char *path, int quiet); diff --git a/transport.c b/transport.c index a016f41702..b255f123c0 100644 --- a/transport.c +++ b/transport.c @@ -143,6 +143,9 @@ static struct ref *get_refs_from_bundle(struct transport *transport, data->fd = read_bundle_header(transport->url, &data->header); if (data->fd < 0) die(_("could not read bundle '%s'"), transport->url); + + transport->hash_algo = data->header.hash_algo; + for (i = 0; i < data->header.references.nr; i++) { struct ref_list_entry *e = data->header.references.list + i; struct ref *ref = alloc_ref(e->name); @@ -157,11 +160,14 @@ static int fetch_refs_from_bundle(struct transport *transport, int nr_heads, struct ref **to_fetch) { struct bundle_transport_data *data = transport->data; + int ret; if (!data->get_refs_from_bundle_called) get_refs_from_bundle(transport, 0, NULL); - return unbundle(the_repository, &data->header, data->fd, - transport->progress ? BUNDLE_VERBOSE : 0); + ret = unbundle(the_repository, &data->header, data->fd, + transport->progress ? BUNDLE_VERBOSE : 0); + transport->hash_algo = data->header.hash_algo; + return ret; } static int close_bundle(struct transport *transport) From patchwork Fri Jun 19 17:56:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11614847 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B1BC3618 for ; Fri, 19 Jun 2020 17:57:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9055E20DD4 for ; Fri, 19 Jun 2020 17:57:26 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="rw0bZhNO" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2404686AbgFSR5Z (ORCPT ); Fri, 19 Jun 2020 13:57:25 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:39552 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2404705AbgFSR4p (ORCPT ); Fri, 19 Jun 2020 13:56:45 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:7d4e:cde:7c41:71c2]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id A048760A76; Fri, 19 Jun 2020 17:56:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1592589396; bh=73juDmT7AedVAFtoq2gbukr8T2H8y8stjGC9wi2bUQQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=rw0bZhNOvWv4YObOWax31THNIbgsO9d6vx/uyY75MVVHdJSakMwfAK2gPEUDrJQJN 8xHH/42tv5oB0xfZ3ezcdzuhk3DPedcZ3LsYOdWQALPVc0O/fbogvTXqMsaaHAJ04G 4zMrzc6zVdm3O4xphKbGA0hMX+mNFbYYUfLsbN4R1e8I0pu5R4G8AUMF1zrUAoVy85 vTJmrBJcjv+nU+p3Av/KT6RCZthk3g1Y9AS/9DyM5pVzOn0QIjV16dHm4myQKgTW3o nPmHiHXqtlxtbnpaRYkjOVoiDJNb6RhEpNJt4Cc8Fa4+KwLpC8RUodPlgMKyxkHAmt QghYlNMVNdKVzzH8IOYh8LtTq6QdU3FNSM1dWTL1YCjRGpj1D0BI4353tm04QV3QDr FOYMpR6hlj8cfVDKHTPa+w9WcMNPsnyHxv72jQ3qY2n3QVORtt7DWRDqRgzgk4lEiB C6yDl3tEsDE06dd0KcKmLGcDkYqQWMZwgUcGQnxSc8XxIQfoOnc From: "brian m. carlson" To: Cc: Junio C Hamano , =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v3 44/44] remote-testgit: adapt for object-format Date: Fri, 19 Jun 2020 17:56:01 +0000 Message-Id: <20200619175601.569856-45-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.278.ge193c7cf3a9 In-Reply-To: <20200619175601.569856-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200619175601.569856-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When using an algorithm other than SHA-1, we need the remote helper to advertise support for the object-format extension and provide information back to us so that we can properly parse refs and return data. Ensure that the test remote helper understands these extensions. Signed-off-by: brian m. carlson --- t/t5801/git-remote-testgit | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/t/t5801/git-remote-testgit b/t/t5801/git-remote-testgit index 6b9f0b5dc7..1544d6dc6b 100755 --- a/t/t5801/git-remote-testgit +++ b/t/t5801/git-remote-testgit @@ -52,9 +52,11 @@ do test -n "$GIT_REMOTE_TESTGIT_SIGNED_TAGS" && echo "signed-tags" test -n "$GIT_REMOTE_TESTGIT_NO_PRIVATE_UPDATE" && echo "no-private-update" echo 'option' + echo 'object-format' echo ;; list) + echo ":object-format $(git rev-parse --show-object-format=storage)" git for-each-ref --format='? %(refname)' 'refs/heads/' 'refs/tags/' head=$(git symbolic-ref HEAD) echo "@$head HEAD" @@ -139,6 +141,10 @@ do test $val = "true" && force="true" || force= echo "ok" ;; + object-format) + test $val = "true" && object_format="true" || object_format= + echo "ok" + ;; *) echo "unsupported" ;;