From patchwork Mon May 25 19:58:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11569329 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 14E0E739 for ; Mon, 25 May 2020 19:59:50 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id F140220787 for ; Mon, 25 May 2020 19:59:49 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="vXQhq1on" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390387AbgEYT7q (ORCPT ); Mon, 25 May 2020 15:59:46 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38678 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390354AbgEYT7o (ORCPT ); Mon, 25 May 2020 15:59:44 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 419866081B; Mon, 25 May 2020 19:59:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1590436783; bh=hh0AAPebP5FiADzq2rtNsAQTbv06qUSl3vhXg+93iek=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=vXQhq1onyb6tdMWILgHHVIIa36sdR4Q8eLKdIZUPe/CTM3P2BwvwddvdWKnGr5+YX Px6ECsZ03mFQZ68gBHqXc2EBhi06PKhEKSMXXx0syvEGUmZ7ubjwARVfZ1PklO+eF2 +yGkLqSB3iNm/3tU7GCDhTihePfC7+HOFBm8+qQZXBkGBnX3514kOutnBi4QBioTjJ 7UBa+4tL1t5OWeORIxFUN3yygyk/YhJ6bHw1fvY8j/LJXMPYaN0J/URzCKPdh5vFzW //YHkcjgOP7e/l12mT5i2Bzf3VlobX1qE5cI3fvDIL0wHRKDStITMvNE9S6SOgCYHd Vdvcf6fOvb+cby/D/CyjBYF9Gesdk74it1s6mBRT23sFz/DghVQqgy/jaJasqTmKNL Q+J5CwJBGz+LyFzPSP45axvyTPiYDdajMKmPJ0Njccq5Bwa6wtPFzCvwVK9dbmRNNL ByXrsanhZzJvWJZKtZrBo9mJG2/Bx5MLqhGems0RyXrGkrcG/KU From: "brian m. carlson" To: Cc: =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v2 01/44] t1050: match object ID paths in a hash-insensitive way Date: Mon, 25 May 2020 19:58:47 +0000 Message-Id: <20200525195930.309665-2-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.rc0.183.gde8f92d652 In-Reply-To: <20200525195930.309665-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200525195930.309665-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org The pattern here looking for failures is specific to SHA-1. Let's create a variable that matches the regex or glob pattern for a path within the objects directory. Signed-off-by: brian m. carlson --- t/t1050-large.sh | 2 +- t/test-lib.sh | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/t/t1050-large.sh b/t/t1050-large.sh index 184b479a21..7f88ea07c2 100755 --- a/t/t1050-large.sh +++ b/t/t1050-large.sh @@ -64,7 +64,7 @@ test_expect_success 'add a large file or two' ' test $count = 1 && cnt=$(git show-index <"$idx" | wc -l) && test $cnt = 2 && - for l in .git/objects/??/?????????????????????????????????????? + for l in .git/objects/$OIDPATH_REGEX do test_path_is_file "$l" || continue bad=t diff --git a/t/test-lib.sh b/t/test-lib.sh index d36b6ddc62..5c65c3e26c 100644 --- a/t/test-lib.sh +++ b/t/test-lib.sh @@ -1414,6 +1414,7 @@ test_oid_init ZERO_OID=$(test_oid zero) OID_REGEX=$(echo $ZERO_OID | sed -e 's/0/[0-9a-f]/g') +OIDPATH_REGEX=$(test_oid_to_path $ZERO_OID | sed -e 's/0/[0-9a-f]/g') EMPTY_TREE=$(test_oid empty_tree) EMPTY_BLOB=$(test_oid empty_blob) _z40=$ZERO_OID From patchwork Mon May 25 19:58:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11569331 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3CA6090 for ; Mon, 25 May 2020 19:59:50 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1AE2120776 for ; Mon, 25 May 2020 19:59:50 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="xX0E7cnZ" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390391AbgEYT7r (ORCPT ); Mon, 25 May 2020 15:59:47 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38682 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390379AbgEYT7p (ORCPT ); Mon, 25 May 2020 15:59:45 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id AB5566081C; Mon, 25 May 2020 19:59:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1590436783; bh=vM7pid11nQsZOGNYd0VAPXG+qD4DXfjxnlg7+N9j2CA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=xX0E7cnZlerX8WEOTNMscRVJhpul6+tMcRjabuSralG6TIHBdHigIiybaPVyLwnij sYpo9H5hWNjMYAg/kKHjWDga52K26SW/qTT+BeFt/X/tSvnVDSSZPkg9NEhZO8bDZ2 FJ0Msi09CLv3fZ8pWqNjFWYcmylJlNUu5XFHLAWZA/ZaoyD3M/0+yue6SyqNfQeI1h Umt5ktxByMPndexVwIYzhxGab3PrxdIRPuVkpsZ/Yt4Ir6E1HjttnmXqyilfEYNpkf g4X8D9OyqbtFx2M5jQm/EovZ1Lq8HuPO/eEfYSuiaD36PS/Vd0e2Fb/djv+VHozYlE It+YTT422iLzZ2bECdb1zFNC9VNSvnb2KucIxSkywm6ZX93+4flUDJjGuG3utGi+h+ /pWVaSB9p++mfb1krZPaHbcrBNBn6E7/ffQj7+X2rJrrYERvtPzWS/rTuc/oV4FZCi +CnAWITq8qU4Tv8wqSFMcEHpVTGhyPA5heur+HEtaJbO4iJ15s4 From: "brian m. carlson" To: Cc: =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v2 02/44] Documentation: document v1 protocol object-format capability Date: Mon, 25 May 2020 19:58:48 +0000 Message-Id: <20200525195930.309665-3-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.rc0.183.gde8f92d652 In-Reply-To: <20200525195930.309665-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200525195930.309665-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Document a capability that indicates which hash algorithms are in use by both sides of a remote connection. Use the term "object-format", since this is the term used for the repository extension as well. Signed-off-by: brian m. carlson --- Documentation/technical/protocol-capabilities.txt | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/Documentation/technical/protocol-capabilities.txt b/Documentation/technical/protocol-capabilities.txt index 2b267c0da6..36ccd14f97 100644 --- a/Documentation/technical/protocol-capabilities.txt +++ b/Documentation/technical/protocol-capabilities.txt @@ -176,6 +176,21 @@ agent strings are purely informative for statistics and debugging purposes, and MUST NOT be used to programmatically assume the presence or absence of particular features. +object-format +------------- + +This capability, which takes a hash algorithm as an argument, indicates +that the server supports the given hash algorithms. It may be sent +multiple times; if so, the first one given is the one used in the ref +advertisement. + +When provided by the client, this indicates that it intends to use the +given hash algorithm to communicate. The algorithm provided must be one +that the server supports. + +If this capability is not provided, it is assumed that the only +supported algorithm is SHA-1. + symref ------ From patchwork Mon May 25 19:58:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11569337 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4BE5A90 for ; Mon, 25 May 2020 19:59:54 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3325B2075F for ; Mon, 25 May 2020 19:59:54 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="YVRMivrQ" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390415AbgEYT7x (ORCPT ); Mon, 25 May 2020 15:59:53 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38686 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390316AbgEYT7p (ORCPT ); Mon, 25 May 2020 15:59:45 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 1E374609CF; Mon, 25 May 2020 19:59:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1590436784; bh=fK4p8fIWd9Go9/zHEmibNYwlQh8e8IryNd34Ad991b8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=YVRMivrQN9A5Q8VUvIm6xar2DncDaMlR/AKl8S5X7N/j9b6k+ZfmEsZzIYMU/Du7R NCeeFDDm3LAo0n7ej/+X3S4dVlpVnQda4lRYdQdkl78YhkrecPoCw374KNwKEMFzDs OzjmFKxOjB6yA07Kjtq1G9YW17OsnlVigdJ9AK6uNvVBQwAsqfKkOjTT8jVtVnWJKj cmi7UDzj4ekZk0KAEiIC3VRcfup+slUKEFeOnEE/lq+4Y/M9jnNbBmhJEnA06qv/cf 77vI/tg63nozM5fnQixqpp9f+M2QMPVeWzasRuAoGpaTCGq0HvHe4qMC43Il5B8wsv cjcy69jvsuNPAPxdjI98cqoVVwXKiYOln4qPrtIPMLPg68FNKp4l/aeyo5/TFisoI4 T3SeLMDwKa6B5YPdMI4svq4oqjUzMm6TDrAMWS4t08AcYFLIs6d9hbBhvraEBMx4QM oFaHyGpQ+YnetvmQc5YC3OVWBhA/cBebZBXwsWbUNaz1WKVbhVz From: "brian m. carlson" To: Cc: =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v2 03/44] connect: have ref processing code take struct packet_reader Date: Mon, 25 May 2020 19:58:49 +0000 Message-Id: <20200525195930.309665-4-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.rc0.183.gde8f92d652 In-Reply-To: <20200525195930.309665-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200525195930.309665-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In a future patch, we'll want to access multiple members from struct packet_reader when parsing references. Therefore, have the ref parsing code take pointers to struct reader instead of having to pass multiple arguments to each function. Rename the len variable to "linelen" to make it clearer what the variable does in light of the variable change. Signed-off-by: brian m. carlson --- connect.c | 27 ++++++++++++++++----------- 1 file changed, 16 insertions(+), 11 deletions(-) diff --git a/connect.c b/connect.c index 23013c6344..ccc5274189 100644 --- a/connect.c +++ b/connect.c @@ -204,17 +204,19 @@ static void annotate_refs_with_symref_info(struct ref *ref) string_list_clear(&symref, 0); } -static void process_capabilities(const char *line, int *len) +static void process_capabilities(struct packet_reader *reader, int *linelen) { + const char *line = reader->line; int nul_location = strlen(line); - if (nul_location == *len) + if (nul_location == *linelen) return; server_capabilities_v1 = xstrdup(line + nul_location + 1); - *len = nul_location; + *linelen = nul_location; } -static int process_dummy_ref(const char *line) +static int process_dummy_ref(const struct packet_reader *reader) { + const char *line = reader->line; struct object_id oid; const char *name; @@ -234,9 +236,11 @@ static void check_no_capabilities(const char *line, int len) line + strlen(line)); } -static int process_ref(const char *line, int len, struct ref ***list, - unsigned int flags, struct oid_array *extra_have) +static int process_ref(const struct packet_reader *reader, int len, + struct ref ***list, unsigned int flags, + struct oid_array *extra_have) { + const char *line = reader->line; struct object_id old_oid; const char *name; @@ -260,9 +264,10 @@ static int process_ref(const char *line, int len, struct ref ***list, return 1; } -static int process_shallow(const char *line, int len, +static int process_shallow(const struct packet_reader *reader, int len, struct oid_array *shallow_points) { + const char *line = reader->line; const char *arg; struct object_id old_oid; @@ -315,20 +320,20 @@ struct ref **get_remote_heads(struct packet_reader *reader, switch (state) { case EXPECTING_FIRST_REF: - process_capabilities(reader->line, &len); - if (process_dummy_ref(reader->line)) { + process_capabilities(reader, &len); + if (process_dummy_ref(reader)) { state = EXPECTING_SHALLOW; break; } state = EXPECTING_REF; /* fallthrough */ case EXPECTING_REF: - if (process_ref(reader->line, len, &list, flags, extra_have)) + if (process_ref(reader, len, &list, flags, extra_have)) break; state = EXPECTING_SHALLOW; /* fallthrough */ case EXPECTING_SHALLOW: - if (process_shallow(reader->line, len, shallow_points)) + if (process_shallow(reader, len, shallow_points)) break; die(_("protocol error: unexpected '%s'"), reader->line); case EXPECTING_DONE: From patchwork Mon May 25 19:58:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11569333 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 840DA1392 for ; Mon, 25 May 2020 19:59:50 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6D2BA2073B for ; Mon, 25 May 2020 19:59:50 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="vUAWAr/N" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390404AbgEYT7t (ORCPT ); Mon, 25 May 2020 15:59:49 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38690 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390380AbgEYT7q (ORCPT ); Mon, 25 May 2020 15:59:46 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 873A5609D3; Mon, 25 May 2020 19:59:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1590436784; bh=VRoONdvcbARP5+3wr28J+/lkAxkxEY4gUpFmTPvhqlU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=vUAWAr/NW5X/HydkOzaG94c1HwVZyJj4QAwlXI9eb5QZVEeErdBFddj6IW2Q4RBkj 6PCqQKZMu9dRGZz9IyydywpDKHLEdnvR4+TGBIW57bvCoKsk/BiK+xDuHqolQKr1Me QHOLeDeQjakpSPQHvzzMykaESI4mppCCxb4qUM4vwvV5iDP6/vDvFODDKFASmMBfyp sqh8rM6tbohdfAtduoXnoQFiQV88CNFUCMvpU8WLUa2+YHzbBJSI7wsrMC0mXu4DUJ uV/hOs49Kf6qwWna+IkGHRkoMRQrGR9t12zsn/ELds5dYCKQkEzHWhfBjyrO+WuoIw TWAbjKUbXtX4QfEEqoAUc6cLB6TasuWTxVK6YzLUMJkTNbYRsyEhxsKgg4FRPZJ2oX 5pbnV1/mOJ2+2N8Qnv1NwB/Bm64vfbPJ7h8bBC16jMcZfQO4+FXo6NwvkaV2hRWIBZ pl/BiCtWktjxQOTMwUkS+zsamZb9xeWIZYpz71sUXernnWvqcJ4 From: "brian m. carlson" To: Cc: =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v2 04/44] wrapper: add function to compare strings with different NUL termination Date: Mon, 25 May 2020 19:58:50 +0000 Message-Id: <20200525195930.309665-5-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.rc0.183.gde8f92d652 In-Reply-To: <20200525195930.309665-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200525195930.309665-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When parsing capabilities for the pack protocol, there are times we'll want to compare the value of a capability to a NUL-terminated string. Since the data we're reading will be space-terminated, not NUL-terminated, we need a function that compares the two strings, but also checks that they're the same length. Otherwise, if we used strncmp to compare these strings, we might accidentally accept a parameter that was a prefix of the expected value. Add a function, xstrncmpz, that takes a NUL-terminated string and a non-NUL-terminated string, plus a length, and compares them, ensuring that they are the same length. Signed-off-by: brian m. carlson --- git-compat-util.h | 6 ++++++ wrapper.c | 8 ++++++++ 2 files changed, 14 insertions(+) diff --git a/git-compat-util.h b/git-compat-util.h index a73632e8e4..5637114b8d 100644 --- a/git-compat-util.h +++ b/git-compat-util.h @@ -868,6 +868,12 @@ char *xgetcwd(void); FILE *fopen_for_writing(const char *path); FILE *fopen_or_warn(const char *path, const char *mode); +/* + * Like strncmp, but only return zero if s is NUL-terminated and exactly len + * characters long. If it is not, consider it greater than t. + */ +int xstrncmpz(const char *s, const char *t, size_t len); + /* * FREE_AND_NULL(ptr) is like free(ptr) followed by ptr = NULL. Note * that ptr is used twice, so don't pass e.g. ptr++. diff --git a/wrapper.c b/wrapper.c index 3a1c0e0526..4ff4a9c3db 100644 --- a/wrapper.c +++ b/wrapper.c @@ -105,6 +105,14 @@ char *xstrndup(const char *str, size_t len) return xmemdupz(str, p ? p - str : len); } +int xstrncmpz(const char *s, const char *t, size_t len) +{ + int res = strncmp(s, t, len); + if (res) + return res; + return s[len] == '\0' ? 0 : 1; +} + void *xrealloc(void *ptr, size_t size) { void *ret; From patchwork Mon May 25 19:58:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11569375 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 609DD912 for ; Mon, 25 May 2020 20:00:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 49B342075F for ; Mon, 25 May 2020 20:00:19 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="vU+/RdQI" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390563AbgEYUAS (ORCPT ); Mon, 25 May 2020 16:00:18 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38784 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390542AbgEYUAQ (ORCPT ); Mon, 25 May 2020 16:00:16 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id EF0DC609D4; Mon, 25 May 2020 19:59:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1590436785; bh=MiR21CjvndjGJN/0B3f1TGsex3CuTAO2yUZeWjGSHkg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=vU+/RdQIfPf4uipC9b9M636zCpnXXjlPACJnbWsuiI7CJYCSHdyZmI3o/eCdKlErj nyzBvPv9zYC6y3ojqOO5E5Oel9BKXPs9ai+AJbAphH4ZjB67pRd2DYaFPaBuEx5ROS Wx/HWRZLFAFpC2Wkt/AQ2kpmTbqA07zkRjegIzuGIGDPA1zMztdT2kR7xGxAJnRUq6 m4xkENSPLKmpeVLdA4+tkRrdXeDJaHUTd+dgX+gtk7+pNDUKTURI/L8OpRi5GAkCCo Tfc2X7vvMwjxr2AzihmcNwa+3unY+i+i4NhpvbRb6zS+AXvwF9dNaWM4ALLPDEVjHS Ex4lBzpV0JJM6KJtkBBMuGcR/r6yJbvMjFrjLulmfFzh5sifm+mAxc7LM39H2ByYBZ BbuIxXGirrYUWiA6e3ZU7xxNHFLpf7vs5aRiJJe0qkLAITjo9nLqpPkApecYuOW6W1 SUOleYl6KMBo/Qfrr9yfI+lwKfDcMTAnHp3E/f2lNGsCVQ87s4i From: "brian m. carlson" To: Cc: =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v2 05/44] remote: advertise the object-format capability on the server side Date: Mon, 25 May 2020 19:58:51 +0000 Message-Id: <20200525195930.309665-6-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.rc0.183.gde8f92d652 In-Reply-To: <20200525195930.309665-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200525195930.309665-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Advertise the current hash algorithm in use by using the object-format capability as part of the ref advertisement. Signed-off-by: brian m. carlson --- builtin/receive-pack.c | 1 + upload-pack.c | 3 ++- 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c index ea3d0f01af..4ffa501dce 100644 --- a/builtin/receive-pack.c +++ b/builtin/receive-pack.c @@ -249,6 +249,7 @@ static void show_ref(const char *path, const struct object_id *oid) strbuf_addf(&cap, " push-cert=%s", push_cert_nonce); if (advertise_push_options) strbuf_addstr(&cap, " push-options"); + strbuf_addf(&cap, " object-format=%s", the_hash_algo->name); strbuf_addf(&cap, " agent=%s", git_user_agent_sanitized()); packet_write_fmt(1, "%s %s%c%s\n", oid_to_hex(oid), path, 0, cap.buf); diff --git a/upload-pack.c b/upload-pack.c index 0478bff3e7..636911fec5 100644 --- a/upload-pack.c +++ b/upload-pack.c @@ -1008,7 +1008,7 @@ static int send_ref(const char *refname, const struct object_id *oid, struct strbuf symref_info = STRBUF_INIT; format_symref_info(&symref_info, cb_data); - packet_write_fmt(1, "%s %s%c%s%s%s%s%s%s agent=%s\n", + packet_write_fmt(1, "%s %s%c%s%s%s%s%s%s object-format=%s agent=%s\n", oid_to_hex(oid), refname_nons, 0, capabilities, (allow_unadvertised_object_request & ALLOW_TIP_SHA1) ? @@ -1018,6 +1018,7 @@ static int send_ref(const char *refname, const struct object_id *oid, stateless_rpc ? " no-done" : "", symref_info.buf, allow_filter ? " filter" : "", + the_hash_algo->name, git_user_agent_sanitized()); strbuf_release(&symref_info); } else { From patchwork Mon May 25 19:58:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11569389 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A9C7C739 for ; Mon, 25 May 2020 20:00:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8F7652073B for ; Mon, 25 May 2020 20:00:32 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="HtJ5p1Oi" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390621AbgEYUAb (ORCPT ); Mon, 25 May 2020 16:00:31 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38796 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390545AbgEYUAR (ORCPT ); Mon, 25 May 2020 16:00:17 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 62A1C60CF2; Mon, 25 May 2020 19:59:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1590436785; bh=qQ80ZzeBSLgKo57xBL5U1/mAg+xY3uajsMWjI2OqhAg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=HtJ5p1OiBnGpzzz9b4o5Ax2Lppf9MoxdTPix4UnLEbwVPeMdd9oA+0CDaDifXEx8x lTLf20PPN3Z0w7d/aAOq/KdGsw071n5gQYU+tnfbMrQcQNEoRwxbUHtftRpMxfDR+B /DdKydg7UkEaGg5Ya2tFDWaSgJZ+LoZ9LkH5Md5m6D7sHPA9XobNEgW+idEf8oJLXG YbdNF1rGwttnJ8YHkWwoS875bLUGegI+ya8bzlD11lARcddbS/UPPghuWbH5vzRVFg zodCJaNttGMqm5KNFnQJWqBzdF+yZ4dF9pU2VhARtNLWgfT5WLI56p7S3DBlmDpD1S PlBhlkRMT16kJ8eRCwrIrPueNt+FQqbZBNvJouziuLx2VsqOaz/kK69bSDuNJPid8N 8TTveejBhb5m0OYWW9Kxgpieytwc9g3QgVSDE1kTlk/o3Pgeq7bVDg2gUiYmJWqSrs izIx38Yqgit2SIGeH5kuQUNLKxUSe3tOwF3VZ7TMbNMDjBgoCcq From: "brian m. carlson" To: Cc: =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v2 06/44] connect: add function to parse multiple v1 capability values Date: Mon, 25 May 2020 19:58:52 +0000 Message-Id: <20200525195930.309665-7-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.rc0.183.gde8f92d652 In-Reply-To: <20200525195930.309665-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200525195930.309665-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In a capability response, we can have multiple symref entries. In the future, we will also allow for multiple hash algorithms to be specified. To avoid duplication, expand the parse_feature_value function to take an optional offset where the parsing should begin next time. Add a wrapper function that allows us to query the next server feature value, and use it in the existing symref parsing code. Signed-off-by: brian m. carlson --- connect.c | 30 +++++++++++++++++++++--------- 1 file changed, 21 insertions(+), 9 deletions(-) diff --git a/connect.c b/connect.c index ccc5274189..2b55a32d4d 100644 --- a/connect.c +++ b/connect.c @@ -18,7 +18,8 @@ static char *server_capabilities_v1; static struct argv_array server_capabilities_v2 = ARGV_ARRAY_INIT; -static const char *parse_feature_value(const char *, const char *, int *); +static const char *parse_feature_value(const char *, const char *, int *, int *); +static const char *next_server_feature_value(const char *feature, int *len, int *offset); static int check_ref(const char *name, unsigned int flags) { @@ -180,17 +181,16 @@ static void parse_one_symref_info(struct string_list *symref, const char *val, i static void annotate_refs_with_symref_info(struct ref *ref) { struct string_list symref = STRING_LIST_INIT_DUP; - const char *feature_list = server_capabilities_v1; + int offset = 0; - while (feature_list) { + while (1) { int len; const char *val; - val = parse_feature_value(feature_list, "symref", &len); + val = next_server_feature_value("symref", &len, &offset); if (!val) break; parse_one_symref_info(&symref, val, len); - feature_list = val + 1; } string_list_sort(&symref); @@ -452,7 +452,7 @@ struct ref **get_remote_refs(int fd_out, struct packet_reader *reader, return list; } -static const char *parse_feature_value(const char *feature_list, const char *feature, int *lenp) +static const char *parse_feature_value(const char *feature_list, const char *feature, int *lenp, int *offset) { int len; @@ -460,6 +460,8 @@ static const char *parse_feature_value(const char *feature_list, const char *fea return NULL; len = strlen(feature); + if (offset) + feature_list += *offset; while (*feature_list) { const char *found = strstr(feature_list, feature); if (!found) @@ -474,9 +476,14 @@ static const char *parse_feature_value(const char *feature_list, const char *fea } /* feature with a value (e.g., "agent=git/1.2.3") */ else if (*value == '=') { + int end; + value++; + end = strcspn(value, " \t\n"); if (lenp) - *lenp = strcspn(value, " \t\n"); + *lenp = end; + if (offset) + *offset = value + end - feature_list; return value; } /* @@ -491,12 +498,17 @@ static const char *parse_feature_value(const char *feature_list, const char *fea int parse_feature_request(const char *feature_list, const char *feature) { - return !!parse_feature_value(feature_list, feature, NULL); + return !!parse_feature_value(feature_list, feature, NULL, NULL); +} + +static const char *next_server_feature_value(const char *feature, int *len, int *offset) +{ + return parse_feature_value(server_capabilities_v1, feature, len, offset); } const char *server_feature_value(const char *feature, int *len) { - return parse_feature_value(server_capabilities_v1, feature, len); + return parse_feature_value(server_capabilities_v1, feature, len, NULL); } int server_supports(const char *feature) From patchwork Mon May 25 19:58:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11569385 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8867C912 for ; Mon, 25 May 2020 20:00:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6F2162075F for ; Mon, 25 May 2020 20:00:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="WriC5kZ7" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390593AbgEYUA1 (ORCPT ); Mon, 25 May 2020 16:00:27 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38802 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390547AbgEYUAS (ORCPT ); Mon, 25 May 2020 16:00:18 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id CAB3960CF9; Mon, 25 May 2020 19:59:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1590436786; bh=o38jj4d7QtGrS5a2eNNJZkgmpX7AjSI6RHIhy2ooVys=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=WriC5kZ7GMWHBk+/hUJXlDEnfH5JgYEgd08WzjJaSLnFKl8HKkvROvS5RFIK48X3q FLiI+p3cJtxVXfvZp+f/opqbZVqce4/LEvlGXzj9Z6VbxvsPsc1CgMO4nwBU0hJXMW 7J9hLHBdis4h8S2RRO2/nG5AwTgKXJLCd8jKuffgZh+Rqtz+Cu7H1YsxVdyF6BsFSt dobtsqgu/vVzv5chVgO70ZHpR6BgQco8JmNZ66WNalyDaiWgjfHlyWKxTEO5HHK22o GXvnB3WW0h4e78yT84OI1/yiyz3Wku68oA5dhXIkJREGG8U8OwNa/23u95hQiXF2en 3M+8EUJKwfAwAfke2+EwtFPSmdv4IKN4By2jIOuDFAcZRxEs9pOassdTL6C5HLKvjc 1GgKjUMjtNs8u/wDQFvaBqUpALu7Tn2pdIaJwF4IKEzKAWhw2c+vICv/de4luIEU3e YultkEzRL9bGtVBsrXhncj7KgV4aJxU8iUY6n3rEKzB9SwOerjO From: "brian m. carlson" To: Cc: =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v2 07/44] connect: add function to fetch value of a v2 server capability Date: Mon, 25 May 2020 19:58:53 +0000 Message-Id: <20200525195930.309665-8-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.rc0.183.gde8f92d652 In-Reply-To: <20200525195930.309665-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200525195930.309665-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org So far in protocol v2, all of our server capabilities that have values have not had values that we've been interested in parsing. For example, we receive but ignore the agent value. However, in a future commit, we're going to want to parse out the value of a server capability. To make this easy, add a function, server_feature_v2, that can fetch the value provided as part of the server capability. Signed-off-by: brian m. carlson --- connect.c | 15 +++++++++++++++ connect.h | 1 + 2 files changed, 16 insertions(+) diff --git a/connect.c b/connect.c index 2b55a32d4d..ad0e4e8e56 100644 --- a/connect.c +++ b/connect.c @@ -84,6 +84,21 @@ int server_supports_v2(const char *c, int die_on_error) return 0; } +int server_feature_v2(const char *c, const char **v) +{ + int i; + + for (i = 0; i < server_capabilities_v2.argc; i++) { + const char *out; + if (skip_prefix(server_capabilities_v2.argv[i], c, &out) && + (*out == '=')) { + *v = out + 1; + return 1; + } + } + return 0; +} + int server_supports_feature(const char *c, const char *feature, int die_on_error) { diff --git a/connect.h b/connect.h index 5f2382e018..4d76a6017d 100644 --- a/connect.h +++ b/connect.h @@ -19,6 +19,7 @@ struct packet_reader; enum protocol_version discover_version(struct packet_reader *reader); int server_supports_v2(const char *c, int die_on_error); +int server_feature_v2(const char *c, const char **v); int server_supports_feature(const char *c, const char *feature, int die_on_error); From patchwork Mon May 25 19:58:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11569387 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8BD18739 for ; Mon, 25 May 2020 20:00:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 706162075F for ; Mon, 25 May 2020 20:00:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="jabJFqhM" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390597AbgEYUA2 (ORCPT ); Mon, 25 May 2020 16:00:28 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38806 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390550AbgEYUAS (ORCPT ); Mon, 25 May 2020 16:00:18 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 3F90D60CFE; Mon, 25 May 2020 19:59:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1590436786; bh=CS7+T0CXvW4akc/aSvSaw17PrrQZy823QbT+zlFofqQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=jabJFqhMxxm6lkEbIrjNbmicdHDvueU/K0zqKq5Y933A0Kd32mlDn65duwzDSinEq 02yKUZOOlvvbQZHTU5KBgIy84rTRqW/bYNKbrojscE2S30+viGzKmUbsbu7XLNndtR iljQBpiuxPAtHhXnPEexhmWKltM8JfRk7olY59FJgemLcYF7iRq8bDq6MnlbobACzA gh4wTZoHpvns7EtdkXPqQlkNHdsX8LgVK1Y+O4wJpTD8eUK74fdQckjN3V9GR2Pv27 i85Z/oIAUrZY7nPlqKbBXs8N7qh1XEzYFROC53ewz9SWH9dTofjqVnkKel4FZ9wQz6 QueJuDh2fedIT/8RF8OX6QSA1W/4mjWPY0PbYuX5HsVecljzZBMss+2jlQAd4VEiT/ l2NpYqLBE61cbgWstPgdSPtfzxvqX7GlKSu5eifn0HSnsHiwW10VVvB7Kr1l0hnVuL PNTcPvqlrZih+vgb80UgFpDjf4I/02qtS64yhlmBLfEKmW3/hAp From: "brian m. carlson" To: Cc: =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v2 08/44] pkt-line: add a member for hash algorithm Date: Mon, 25 May 2020 19:58:54 +0000 Message-Id: <20200525195930.309665-9-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.rc0.183.gde8f92d652 In-Reply-To: <20200525195930.309665-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200525195930.309665-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Add a member for the hash algorithm currently in use to the packet reader so it can parse references correctly. Signed-off-by: brian m. carlson --- pkt-line.c | 1 + pkt-line.h | 3 +++ 2 files changed, 4 insertions(+) diff --git a/pkt-line.c b/pkt-line.c index a0e87b1e81..a4aea075de 100644 --- a/pkt-line.c +++ b/pkt-line.c @@ -479,6 +479,7 @@ void packet_reader_init(struct packet_reader *reader, int fd, reader->buffer_size = sizeof(packet_buffer); reader->options = options; reader->me = "git"; + reader->hash_algo = &hash_algos[GIT_HASH_SHA1]; } enum packet_read_status packet_reader_read(struct packet_reader *reader) diff --git a/pkt-line.h b/pkt-line.h index fef3a0d792..4cd9435e9a 100644 --- a/pkt-line.h +++ b/pkt-line.h @@ -166,6 +166,9 @@ struct packet_reader { unsigned use_sideband : 1; const char *me; + + /* hash algorithm in use */ + const struct git_hash_algo *hash_algo; }; /* From patchwork Mon May 25 19:58:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11569381 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 50F40739 for ; Mon, 25 May 2020 20:00:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 390BA2075F for ; Mon, 25 May 2020 20:00:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="ZkJukFxO" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390577AbgEYUAZ (ORCPT ); Mon, 25 May 2020 16:00:25 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38812 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390557AbgEYUAS (ORCPT ); Mon, 25 May 2020 16:00:18 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id A6E5F60CFF; Mon, 25 May 2020 19:59:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1590436786; bh=smurCJGXxF6jSqhkjKD9sTFjDwJyxySvg1esDLljrV0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=ZkJukFxOwg8fOmzXu6LkBGJ6oDV7AI2N2sX4EEwk6rPlcqOftd+q1KqH9PX17wep0 V0bLscgYvhXzPLb265Sl14bWNgc+Rx5kqcpqyTPWxdiELQJB8AhhhHKcsfRQokY0aG 2red55z/AUVqXl8CDm1psJYTqjfhVihZZ23pVAFK6YeWMN8ilcwZNgU2pr2G50hgBU ff03FVwyLa0Be1i+W9HCqBzoF/v5hOdZRrU/EVAxXEzB5nHr3Ot2N7Sz/TNsHXz0Ed MBtmIReh4+0TS2KBvZeq4PEf2MNdHBtF7K2hpzjoaxdSWoT1vY/h1VtXu47n5qc04j t1jrjUAwnfccsOi0l8R5599ZZZjlyZhjIwi+PLIsXvFJ5yBIYU3j06QptDvlvhnbz8 oza57nGOumnRfglpt4v9XIAlTwb+bKDkg23bqBVW9FTNN0C3pHud+RR33/BpzPfou1 0UlZgFIlZlRVPhaxNR9e4vsZfDj/mztiFcxEBefgeD76TGtiFBU From: "brian m. carlson" To: Cc: =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v2 09/44] transport: add a hash algorithm member Date: Mon, 25 May 2020 19:58:55 +0000 Message-Id: <20200525195930.309665-10-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.rc0.183.gde8f92d652 In-Reply-To: <20200525195930.309665-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200525195930.309665-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When connecting to a remote system, we need to know what hash algorithm it will be using to talk to us. Add a hash_algo member to struct transport and add a function to read this data from the transport object. Signed-off-by: brian m. carlson --- transport.c | 8 ++++++++ transport.h | 8 ++++++++ 2 files changed, 16 insertions(+) diff --git a/transport.c b/transport.c index 15f5ba4e8f..b43d985f90 100644 --- a/transport.c +++ b/transport.c @@ -311,6 +311,7 @@ static struct ref *handshake(struct transport *transport, int for_push, BUG("unknown protocol version"); } data->got_remote_heads = 1; + transport->hash_algo = reader.hash_algo; if (reader.line_peeked) BUG("buffer must be empty at the end of handshake()"); @@ -996,9 +997,16 @@ struct transport *transport_get(struct remote *remote, const char *url) ret->smart_options->receivepack = remote->receivepack; } + ret->hash_algo = &hash_algos[GIT_HASH_SHA1]; + return ret; } +const struct git_hash_algo *transport_get_hash_algo(struct transport *transport) +{ + return transport->hash_algo; +} + int transport_set_option(struct transport *transport, const char *name, const char *value) { diff --git a/transport.h b/transport.h index 4298c855be..2a9f96c05a 100644 --- a/transport.h +++ b/transport.h @@ -115,6 +115,8 @@ struct transport { struct git_transport_options *smart_options; enum transport_family family; + + const struct git_hash_algo *hash_algo; }; #define TRANSPORT_PUSH_ALL (1<<0) @@ -243,6 +245,12 @@ int transport_push(struct repository *repo, const struct ref *transport_get_remote_refs(struct transport *transport, const struct argv_array *ref_prefixes); +/* + * Fetch the hash algorithm used by a remote. + * + * This can only be called after fetching the remote refs. + */ +const struct git_hash_algo *transport_get_hash_algo(struct transport *transport); int transport_fetch_refs(struct transport *transport, struct ref *refs); void transport_unlock_pack(struct transport *transport); int transport_disconnect(struct transport *transport); From patchwork Mon May 25 19:58:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11569383 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id ED9971392 for ; Mon, 25 May 2020 20:00:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D61442075F for ; Mon, 25 May 2020 20:00:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="0m/YnXvp" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390588AbgEYUA0 (ORCPT ); Mon, 25 May 2020 16:00:26 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38818 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390559AbgEYUAS (ORCPT ); Mon, 25 May 2020 16:00:18 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 189E460D00; Mon, 25 May 2020 19:59:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1590436787; bh=CNRYudmVcbOwLD3nR9jzlGoBvErmrez6UAMqXK+Jovs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=0m/YnXvplG9psvHY/IYffKOFjQ5N14kPaDKEBWUBTTvuC3x0J0HD2OsjwRSgOWJPT 7XKkgSDSOoxrA38ZTOhG7M/Wk1azP6nvHG1CBh9FFPtVxeqerpulL+rOWHyZYftNsS eN3H2b0wFx4eOnh+kO8GHz0LJ5hrlnnxwP5887b8HkcKhVEWqZyG/fuYdTGn+HfoC1 XIviSl7DoW0ggJBNaX7xhfiLOXqzdGy2YYhO8uMa8x3tB27Zjqbdyy0Slx+ioPYhu9 6/6Fk1mgUMl1SJ8db8fs5TQBToOcwP3Mq21C4zWMFbVaH+Q9iZ61N1Ob5DXIGTtF40 Hb4iMLHizdwltU+YZ5J5SsQ7QOK2pfkWhkDTsj/CPpp/DImmBOvZyMHOm5K3SDOHcZ Az2vLrY0v2lh83JqjXN+6gubn7LLtc2lgVcTTdiWZdI7OPXw+uE2AOBddD+rFYEUCt 6kVvf7UxMw2GUtFyjkO75lfwRp+U665Ae3v/KI7GYH0XZcnltjA From: "brian m. carlson" To: Cc: =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v2 10/44] connect: add function to detect supported v1 hash functions Date: Mon, 25 May 2020 19:58:56 +0000 Message-Id: <20200525195930.309665-11-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.rc0.183.gde8f92d652 In-Reply-To: <20200525195930.309665-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200525195930.309665-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Add a function, server_supports_hash, to see if the remote server supports a particular hash algorithm when speaking protocol v1. Signed-off-by: brian m. carlson --- connect.c | 22 ++++++++++++++++++++++ connect.h | 1 + 2 files changed, 23 insertions(+) diff --git a/connect.c b/connect.c index ad0e4e8e56..eaa13b41bb 100644 --- a/connect.c +++ b/connect.c @@ -511,6 +511,28 @@ static const char *parse_feature_value(const char *feature_list, const char *fea return NULL; } +int server_supports_hash(const char *desired, int *feature_supported) +{ + int offset = 0; + int len; + const char *hash; + + hash = next_server_feature_value("object-format", &len, &offset); + if (feature_supported) + *feature_supported = !!hash; + if (!hash) { + hash = hash_algos[GIT_HASH_SHA1].name; + len = strlen(hash); + } + while (hash) { + if (!xstrncmpz(desired, hash, len)) + return 1; + + hash = next_server_feature_value("object-format", &len, &offset); + } + return 0; +} + int parse_feature_request(const char *feature_list, const char *feature) { return !!parse_feature_value(feature_list, feature, NULL, NULL); diff --git a/connect.h b/connect.h index 4d76a6017d..fc75d6a457 100644 --- a/connect.h +++ b/connect.h @@ -18,6 +18,7 @@ int url_is_local_not_ssh(const char *url); struct packet_reader; enum protocol_version discover_version(struct packet_reader *reader); +int server_supports_hash(const char *desired, int *feature_supported); int server_supports_v2(const char *c, int die_on_error); int server_feature_v2(const char *c, const char **v); int server_supports_feature(const char *c, const char *feature, From patchwork Mon May 25 19:58:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11569347 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0059190 for ; Mon, 25 May 2020 20:00:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D26242075F for ; Mon, 25 May 2020 19:59:59 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="A6XJCRUT" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390452AbgEYT76 (ORCPT ); Mon, 25 May 2020 15:59:58 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38716 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390393AbgEYT7t (ORCPT ); Mon, 25 May 2020 15:59:49 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 7EF90609CF; Mon, 25 May 2020 19:59:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1590436787; bh=edUdmuqwqgrI4d7hU9gk2tehK/HwKep/ouFUbgKC4Xo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=A6XJCRUTQUKscCWf5X97I6xLbNgW7naieDvuPKDBa73ov++gSVHGGmesN0/nmn9Qg WK9WEpHWs7hB0OtYmgcmL7SJEkDQrFmHqzEO72vRo143dsLWQYfYMtNxxSeMt+JifL XR7cH+vhUmgsjDNGDOd+4+9At/lzqI9CAIzSRFCI5et7+3WjX9r7kVPz7fyY8lafPq 9jjKLUnL2CLcla5JOnuY+8ViG+PwqKpPhlrLPA13jT8JBLkFirNOqMJq17kyWnuQJ8 wWi8YjU4GozmXjkHC2bm6kjnnwabblRrOnNnzoLoz8Gp0vhs4ziieOe0xgQJU6R0/F 5/3Xc+2On+v77Dvq4GLDnwUAEhTPQK0zX/MUJkmJ9KGTD0M7CRoQUfJEktVwZaJoTH oLFsvioikQViBhvSNxRDltmRWXmCLEIDveXe9dLGX65Lpqk9G9MKdqlTf+nPsIhnpn 3+VjDZ0S0/0e0OOgzfigNJbECeXWm8jAWbHnXajFk/GQmR5lRn7 From: "brian m. carlson" To: Cc: =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v2 11/44] send-pack: detect when the server doesn't support our hash Date: Mon, 25 May 2020 19:58:57 +0000 Message-Id: <20200525195930.309665-12-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.rc0.183.gde8f92d652 In-Reply-To: <20200525195930.309665-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200525195930.309665-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Detect when the server doesn't support our hash algorithm and abort. If the server does support our hash, advertise it as part of our capabilities. Signed-off-by: brian m. carlson --- send-pack.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/send-pack.c b/send-pack.c index 0abee22283..02aefcb08e 100644 --- a/send-pack.c +++ b/send-pack.c @@ -363,6 +363,7 @@ int send_pack(struct send_pack_args *args, int atomic_supported = 0; int use_push_options = 0; int push_options_supported = 0; + int object_format_supported = 0; unsigned cmds_sent = 0; int ret; struct async demux; @@ -389,6 +390,9 @@ int send_pack(struct send_pack_args *args, if (server_supports("push-options")) push_options_supported = 1; + if (!server_supports_hash(the_hash_algo->name, &object_format_supported)) + die(_("the receiving end does not support this repository's hash algorithm")); + if (args->push_cert != SEND_PACK_PUSH_CERT_NEVER) { int len; push_cert_nonce = server_feature_value("push-cert", &len); @@ -429,6 +433,8 @@ int send_pack(struct send_pack_args *args, strbuf_addstr(&cap_buf, " atomic"); if (use_push_options) strbuf_addstr(&cap_buf, " push-options"); + if (object_format_supported) + strbuf_addf(&cap_buf, " object-format=%s", the_hash_algo->name); if (agent_supported) strbuf_addf(&cap_buf, " agent=%s", git_user_agent_sanitized()); From patchwork Mon May 25 19:58:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11569345 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B065F1392 for ; Mon, 25 May 2020 19:59:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 999F720776 for ; Mon, 25 May 2020 19:59:58 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="hyBXMI93" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390443AbgEYT75 (ORCPT ); Mon, 25 May 2020 15:59:57 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38718 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390395AbgEYT7t (ORCPT ); Mon, 25 May 2020 15:59:49 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id E781A609D3; Mon, 25 May 2020 19:59:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1590436788; bh=+VC8gXa5DUc8GKeavPNfMWykrG004iaSqUaxmId/s2c=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=hyBXMI93ALcLY8UBwKDkK3u2rFdTfM4jQMwiM5HD9PpXjag0602fP69cgMboG9Wma TVZ6KUgf8ihrPNigoISPsDiHaLmkP3quYqviXc+AsAV4W22lLurZ0XU1J1iWGxOZqR BdyPj3y1hW28WGqNUJYLwC0te1i0kUod3f7ekvoyGmdDYCxRBof5msLd7yhTlsCWkS D2P3NJ0u8CpQuunMd0XmJE//1GR/e1pkIiCLLWxHKwmFUIyBfEE2wefgC9SG52taAX GM7DEhXydH85Aet5OcaXMgJMzDb/QNxct0ecElTAEPBEWn5/oVBauTKTKPECIq4Ue9 pXUqk1HwIFQK5dUNr2Us0lcrrz9CBRlcOuzevqKwr1PAJN7LKBGr7/CbR9i8AhnmY7 34ns2CZLWKQioaDQ1SMSNlX9NVVJM5Rftm3M3nMFhs0sTn5OQ4p490FB4MTLddC4KE riuqOWcAxosUsX8cWgrLPpUHkcQUfEKwz3/lHiVe+2YyOLSwDcy From: "brian m. carlson" To: Cc: =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v2 12/44] connect: make parse_feature_value extern Date: Mon, 25 May 2020 19:58:58 +0000 Message-Id: <20200525195930.309665-13-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.rc0.183.gde8f92d652 In-Reply-To: <20200525195930.309665-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200525195930.309665-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org We're going to be using this function in other files, so no longer mark this function static. Signed-off-by: brian m. carlson --- connect.c | 3 +-- connect.h | 1 + 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/connect.c b/connect.c index eaa13b41bb..397fad7e32 100644 --- a/connect.c +++ b/connect.c @@ -18,7 +18,6 @@ static char *server_capabilities_v1; static struct argv_array server_capabilities_v2 = ARGV_ARRAY_INIT; -static const char *parse_feature_value(const char *, const char *, int *, int *); static const char *next_server_feature_value(const char *feature, int *len, int *offset); static int check_ref(const char *name, unsigned int flags) @@ -467,7 +466,7 @@ struct ref **get_remote_refs(int fd_out, struct packet_reader *reader, return list; } -static const char *parse_feature_value(const char *feature_list, const char *feature, int *lenp, int *offset) +const char *parse_feature_value(const char *feature_list, const char *feature, int *lenp, int *offset) { int len; diff --git a/connect.h b/connect.h index fc75d6a457..ace074dcb6 100644 --- a/connect.h +++ b/connect.h @@ -19,6 +19,7 @@ struct packet_reader; enum protocol_version discover_version(struct packet_reader *reader); int server_supports_hash(const char *desired, int *feature_supported); +const char *parse_feature_value(const char *feature_list, const char *feature, int *lenp, int *offset); int server_supports_v2(const char *c, int die_on_error); int server_feature_v2(const char *c, const char **v); int server_supports_feature(const char *c, const char *feature, From patchwork Mon May 25 19:58:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11569341 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6E4B390 for ; Mon, 25 May 2020 19:59:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4E7AE20776 for ; Mon, 25 May 2020 19:59:57 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="Ns/QWGMc" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390430AbgEYT7z (ORCPT ); Mon, 25 May 2020 15:59:55 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38720 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390398AbgEYT7t (ORCPT ); Mon, 25 May 2020 15:59:49 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 5AC0160D01; Mon, 25 May 2020 19:59:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1590436788; bh=TGhOMPz35SRRFxjnBdGILqgto6hXnSxBir9T4tn1EIU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=Ns/QWGMcac4ai4socEVvd4I6phcBLv7CXh6Hi5nHmWVg8mE5QzSakHbZqR0Rk1tZS XYo1M6Pta4JU+TZF6H6LeGVpV+cCphRCG3p0QvBufYK9hGoyC636xT19dLp26AJ0ER SArhMbEq2lQF06rC6zyCyJTg8UwIBfkTuZl6fOlKA2WTKWOrSZjMc2NB6WnIGER5p2 ExFT+pGpSI1d/j4SKJK3b1hPe5Jxz+ism/jUEyt0sdwQWyBSHJ3V1/3g2yjzDs7xZ/ +eRwj7qAlUleMeUeRR1Ifk8iF91QAVfgEsT11NdrM7/IOtWWUOA5WCJ84F8hL7hVFa MaBiuk2AIVDhR0SHqNFBNG0UjTwd7xmB74bLXMCV7xsvX7tFYLkfnKURnqykli9TOd Vp/en8oZhgV1fMfq5/zOG2BjQcGPbvIDdHMpc424D6NUqF4a00BRxujqTetSD0X89O xwvB18zuEt9iNgB3w/0s8A8XXL/c1tzkXDwcA8vgVusiR9VEn4o From: "brian m. carlson" To: Cc: =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v2 13/44] fetch-pack: detect when the server doesn't support our hash Date: Mon, 25 May 2020 19:58:59 +0000 Message-Id: <20200525195930.309665-14-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.rc0.183.gde8f92d652 In-Reply-To: <20200525195930.309665-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200525195930.309665-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Detect when the server doesn't support our hash algorithm and abort. Signed-off-by: brian m. carlson --- fetch-pack.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/fetch-pack.c b/fetch-pack.c index 7eaa19d7c1..34c339a5fe 100644 --- a/fetch-pack.c +++ b/fetch-pack.c @@ -1040,6 +1040,8 @@ static struct ref *do_fetch_pack(struct fetch_pack_args *args, print_verbose(args, _("Server supports %s"), "deepen-relative"); else if (args->deepen_relative) die(_("Server does not support --deepen")); + if (!server_supports_hash(the_hash_algo->name, NULL)) + die(_("Server does not support this repository's object format")); if (!args->no_dependents) { mark_complete_and_common_ref(negotiator, args, &ref); From patchwork Mon May 25 19:59:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11569339 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DBE6E90 for ; Mon, 25 May 2020 19:59:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C456A2075F for ; Mon, 25 May 2020 19:59:55 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="qMXvlARQ" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390427AbgEYT7z (ORCPT ); Mon, 25 May 2020 15:59:55 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38722 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390407AbgEYT7u (ORCPT ); Mon, 25 May 2020 15:59:50 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id E84B160D02; Mon, 25 May 2020 19:59:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1590436789; bh=FNoOQgqMLkejIe1GGHLaeHepK9RofMxJKd5qQBwijFM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=qMXvlARQYjheCb8K40raJI7m0ddwOofg7g/ZX7DnHcAZrYIILQvBF/Xa7cK8tgFed ZNCAWWUzvACIk/D4SgMqciOiJWG62r6rGpIlnzfQ3SihkyiiLSIba9EPq9T2/nYwf1 k0+Si4ergVQ2xIwO7lAvB93834myqOlgFJ4a6s2GJlGgkg/kehnBWhuHE0Crsr5Fm+ wksgYRilIu7iNZmbWEt2D+nWEHVH78t8hChIw4ED+MW8LvKa4Fjalohf2JbI5D3cDk X0JEnihRoBkZSyVzrvpXh4Yf3ckyQKn4JO0mu5fmnqxDIcFGMX9ZzvHI08/nyMLvq+ xupLSXFDs5AeeiGE9CDQcWM/ADih/7ptAqiO7X/rQQS9bKp4KvVfn2pPpbTdKIYo0o dbEXeqbk3P24QMG5vgFw4SLXibYm0OcbdfIgNC71RqYJkCWoUwYMGnJx2nkr2vntKX gLzyptG/xMqNaIIQB9AYYRqCijLJx3eVI7JJtoAqHEz/yKpZpSq From: "brian m. carlson" To: Cc: =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v2 14/44] connect: detect algorithm when fetching refs Date: Mon, 25 May 2020 19:59:00 +0000 Message-Id: <20200525195930.309665-15-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.rc0.183.gde8f92d652 In-Reply-To: <20200525195930.309665-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200525195930.309665-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org If we're fetching refs, detect the hash algorithm and parse the refs using that algorithm. As mentioned in the documentation, if multiple versions of the object-format capability are provided, we use the first. No known implementation supports multiple algorithms now, but they may in the future. Signed-off-by: brian m. carlson --- connect.c | 21 +++++++++++++++++---- 1 file changed, 17 insertions(+), 4 deletions(-) diff --git a/connect.c b/connect.c index 397fad7e32..915f1736a0 100644 --- a/connect.c +++ b/connect.c @@ -220,12 +220,25 @@ static void annotate_refs_with_symref_info(struct ref *ref) static void process_capabilities(struct packet_reader *reader, int *linelen) { + const char *feat_val; + int feat_len; const char *line = reader->line; int nul_location = strlen(line); if (nul_location == *linelen) return; server_capabilities_v1 = xstrdup(line + nul_location + 1); *linelen = nul_location; + + feat_val = server_feature_value("object-format", &feat_len); + if (feat_val) { + char *hash_name = xstrndup(feat_val, feat_len); + int hash_algo = hash_algo_by_name(hash_name); + if (hash_algo != GIT_HASH_UNKNOWN) + reader->hash_algo = &hash_algos[hash_algo]; + free(hash_name); + } else { + reader->hash_algo = &hash_algos[GIT_HASH_SHA1]; + } } static int process_dummy_ref(const struct packet_reader *reader) @@ -234,7 +247,7 @@ static int process_dummy_ref(const struct packet_reader *reader) struct object_id oid; const char *name; - if (parse_oid_hex(line, &oid, &name)) + if (parse_oid_hex_algop(line, &oid, &name, reader->hash_algo)) return 0; if (*name != ' ') return 0; @@ -258,7 +271,7 @@ static int process_ref(const struct packet_reader *reader, int len, struct object_id old_oid; const char *name; - if (parse_oid_hex(line, &old_oid, &name)) + if (parse_oid_hex_algop(line, &old_oid, &name, reader->hash_algo)) return 0; if (*name != ' ') return 0; @@ -270,7 +283,7 @@ static int process_ref(const struct packet_reader *reader, int len, die(_("protocol error: unexpected capabilities^{}")); } else if (check_ref(name, flags)) { struct ref *ref = alloc_ref(name); - oidcpy(&ref->old_oid, &old_oid); + memcpy(ref->old_oid.hash, old_oid.hash, reader->hash_algo->rawsz); **list = ref; *list = &ref->next; } @@ -288,7 +301,7 @@ static int process_shallow(const struct packet_reader *reader, int len, if (!skip_prefix(line, "shallow ", &arg)) return 0; - if (get_oid_hex(arg, &old_oid)) + if (get_oid_hex_algop(arg, &old_oid, reader->hash_algo)) die(_("protocol error: expected shallow sha-1, got '%s'"), arg); if (!shallow_points) die(_("repository on the other end cannot be shallow")); From patchwork Mon May 25 19:59:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11569413 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C284A912 for ; Mon, 25 May 2020 20:00:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AB4312075F for ; Mon, 25 May 2020 20:00:47 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="sVIodrIE" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390685AbgEYUAq (ORCPT ); Mon, 25 May 2020 16:00:46 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38716 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390412AbgEYT7y (ORCPT ); Mon, 25 May 2020 15:59:54 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 5BC3560D03; Mon, 25 May 2020 19:59:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1590436789; bh=smmJElpsjDUJ3khU5WIlomu9EU1J8x04QmZc/+KQYhc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=sVIodrIEk4bhl+4ckn4WE/NdlJv4wxe3/LqaSnespZepUOG/+r+Zba/aNpa2udHaO iKwVLPRIzwVpXSjfYRlbh6Bu9RyHHghbSe3SzxKSeDC1LhVth79U1QHFd8Rw/+HBJZ CshnRXF8sebja3ZQTxxCgtwRvFjF6LdFJQIbvrqq0j27RBVA98LDXx9TZpzlWn2QAY PKsNGQhzKZNu+cpuj8ltYGkZ8YCvObNgNQMCR8IAZVjat7QU3siayBjTxRv7DPgyRi 42wI6KUs51VrYo88Vl7yyz2SwFbqjganPpWzdlwYOSrIBUP9rAWkNR36q3ujFcdUWE U5s7ElBsUvm8vCIFN7l7KMg/Z/4F1Kv/5tx+6PIy6ItVMB8AqCGVUlINfWKhuTB9J6 2BDFmk++slY0abyMhbbTyy9v3S12XVImyR6XaN1/tAqg1D8G9diUlpUYi2QGDMtSly lRszxnyJDyjzxDiI29X1RZTtjrusBFmHJqiA0G7cxlI6tE2ZsRn From: "brian m. carlson" To: Cc: =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v2 15/44] builtin/receive-pack: detect when the server doesn't support our hash Date: Mon, 25 May 2020 19:59:01 +0000 Message-Id: <20200525195930.309665-16-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.rc0.183.gde8f92d652 In-Reply-To: <20200525195930.309665-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200525195930.309665-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Detect when the server doesn't support our hash algorithm and abort. Signed-off-by: brian m. carlson --- builtin/receive-pack.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c index 4ffa501dce..d43663bb0a 100644 --- a/builtin/receive-pack.c +++ b/builtin/receive-pack.c @@ -1625,6 +1625,8 @@ static struct command *read_head_info(struct packet_reader *reader, linelen = strlen(reader->line); if (linelen < reader->pktlen) { const char *feature_list = reader->line + linelen + 1; + const char *hash = NULL; + int len = 0; if (parse_feature_request(feature_list, "report-status")) report_status = 1; if (parse_feature_request(feature_list, "side-band-64k")) @@ -1637,6 +1639,13 @@ static struct command *read_head_info(struct packet_reader *reader, if (advertise_push_options && parse_feature_request(feature_list, "push-options")) use_push_options = 1; + hash = parse_feature_value(feature_list, "object-format", &len, NULL); + if (!hash) { + hash = hash_algos[GIT_HASH_SHA1].name; + len = strlen(hash); + } + if (xstrncmpz(the_hash_algo->name, hash, len)) + die("error: unsupported object format '%s'", hash); } if (!strcmp(reader->line, "push-cert")) { From patchwork Mon May 25 19:59:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11569343 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8C79C739 for ; Mon, 25 May 2020 19:59:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 743962075F for ; Mon, 25 May 2020 19:59:58 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="iezzkm4l" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390437AbgEYT74 (ORCPT ); Mon, 25 May 2020 15:59:56 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38718 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390411AbgEYT7y (ORCPT ); Mon, 25 May 2020 15:59:54 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id C27A060D04; Mon, 25 May 2020 19:59:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1590436790; bh=JYZIPaXyfRmBGoUSPO/BC8tfJm3FljXIDJUspJCISoE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=iezzkm4lXTUxYcHVj7+Mm0b0NUJesbkr54guJqloEGUN4dnJGGNv8nczUZ0fYG+kx EYrogpc4nGWDCEBpQdyZcvd/BBEree5F0QkRmGknwl6wZLdfRzgq0jnQok/gM24i4f 4olT0Dy5fZ+qGqZoiIHggKgWYUp220OHI8JfoskU6oOYUoKJd6QWcO7gVzQooH7zBJ hLD2OQL6teGJ2EKSN0HgZ1yGzRPnu4tD21M33B3HupXm5wpoVLP/Q/ijZ0HaeTIU/K diYmF6ObZInmnjFGLlK/sRMuVZFlbSbM3dri2jYUfynOuKSQSvPEWZ0TIKgtoWU2FM tlk/mOyNlbIjBeOjzZMX8y5bxKBo7OK8lcgTJzt9/nlzhtbW+RkdnhrJmfDb1DBHER /3u3nbJpYAxAd+W/aS/tCxjPoeHJ8DSGhqA5IJGDdstV0Mx6B3IcP3Sg2mk04Zrk9D flH61Ky5EOHAV/4vK+tjSt0frHxvQUjfTRqSodB7S88YMWHMiP6 From: "brian m. carlson" To: Cc: =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v2 16/44] docs: update remote helper docs for object-format extensions Date: Mon, 25 May 2020 19:59:02 +0000 Message-Id: <20200525195930.309665-17-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.rc0.183.gde8f92d652 In-Reply-To: <20200525195930.309665-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200525195930.309665-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Update the remote helper docs to document the object-format extensions we will implement in remote-curl and the transport helper code shortly. Signed-off-by: brian m. carlson --- Documentation/gitremote-helpers.txt | 33 +++++++++++++++++++++++++---- 1 file changed, 29 insertions(+), 4 deletions(-) diff --git a/Documentation/gitremote-helpers.txt b/Documentation/gitremote-helpers.txt index f48a031dc3..26f32e4421 100644 --- a/Documentation/gitremote-helpers.txt +++ b/Documentation/gitremote-helpers.txt @@ -238,6 +238,9 @@ the remote repository. `--signed-tags=verbatim` to linkgit:git-fast-export[1]. In the absence of this capability, Git will use `--signed-tags=warn-strip`. +'object-format':: + This indicates that the helper is able to interact with the remote + side using an explicit hash algorithm extension. COMMANDS @@ -257,12 +260,14 @@ Support for this command is mandatory. 'list':: Lists the refs, one per line, in the format " [ ...]". The value may be a hex sha1 hash, "@" for - a symref, or "?" to indicate that the helper could not get the - value of the ref. A space-separated list of attributes follows - the name; unrecognized attributes are ignored. The list ends - with a blank line. + a symref, ": " for a key-value pair, or + "?" to indicate that the helper could not get the value of the + ref. A space-separated list of attributes follows the name; + unrecognized attributes are ignored. The list ends with a + blank line. + See REF LIST ATTRIBUTES for a list of currently defined attributes. +See REF LIST KEYWORDS for a list of currently defined keywords. + Supported if the helper has the "fetch" or "import" capability. @@ -430,6 +435,18 @@ attributes are defined. This ref is unchanged since the last import or fetch, although the helper cannot necessarily determine what value that produced. +REF LIST KEYWORDS +----------------- + +The 'list' command may produce a list of key-value pairs. +The following keys are defined. + +'object-format':: + The refs are using the given hash algorithm. This keyword is only + used if the server and client both support the object-format + extension. + + OPTIONS ------- @@ -514,6 +531,14 @@ set by Git if the remote helper has the 'option' capability. transaction. If successful, all refs will be updated, or none will. If the remote side does not support this capability, the push will fail. +'option object-format' {'true'|algorithm}:: + If 'true', indicate that the caller wants hash algorithm information + to be passed back from the remote. This mode is used when fetching + refs. ++ +If set to an algorithm, indicate that the caller wants to interact with +the remote side using that algorithm. + SEE ALSO -------- linkgit:git-remote[1] From patchwork Mon May 25 19:59:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11569417 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 69AD6739 for ; Mon, 25 May 2020 20:00:50 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 541742075F for ; Mon, 25 May 2020 20:00:50 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="RSQLBHjy" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390678AbgEYUAp (ORCPT ); Mon, 25 May 2020 16:00:45 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38842 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390422AbgEYUAY (ORCPT ); Mon, 25 May 2020 16:00:24 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 3822A60D05; Mon, 25 May 2020 19:59:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1590436790; bh=z/hbO9Pa9kw3i3ydW7bPReUQR2B2iix4X60keH+Bypw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=RSQLBHjyvV3gJ+bkav/vM1OHeKc1S+Ae+xyDKFxG5W3XuFmVgKBNc5odBsr6PJM/s gX6jx9D+G8KWE4EzflMwPXpLiywFCWAR20u7FP16c+rFPgsNvr+JZ8I4EOq03Bbo97 tRR5/Vy6j4MHKj8vEGT86GgUsMlASJ7U23ibsTKqK/p1WbFShaSyZjCI42h8il/08d nHHi9iybbY+lU3VtQMOQrrpYeu0AOr6U97qIvtyTU65eBAXia9h1pZgKeuI5PtbHec OQTMam2hYALC12wRwjVrrLOEGt6C6N/uwWumI7M9DGARLrhzbqrfkvY05JHvmJuAis i79Y4QU7Vzog19fbn58G+NfWbPiD8qrxYM+D+FIJiEy6HQk0YTM3BfdcfmKkvp1eVJ W0M+tK8PfOjtfxL6GV8Yqxz4bcKGtnxLLY7pYBDV4P2e7zslJpeAEyCq2mxrQuKxGu 0cPC+vkkSKjComWV3SPfMph5mXOAPkiQcqLpiWVsqUczyyYi/Cc From: "brian m. carlson" To: Cc: =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v2 17/44] transport-helper: implement object-format extensions Date: Mon, 25 May 2020 19:59:03 +0000 Message-Id: <20200525195930.309665-18-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.rc0.183.gde8f92d652 In-Reply-To: <20200525195930.309665-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200525195930.309665-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Implement the object-format extensions that let us determine the hash algorithm in use when pushing or pulling data. Signed-off-by: brian m. carlson --- transport-helper.c | 24 ++++++++++++++++++++++-- 1 file changed, 22 insertions(+), 2 deletions(-) diff --git a/transport-helper.c b/transport-helper.c index a46afcb69d..ae33b0eea7 100644 --- a/transport-helper.c +++ b/transport-helper.c @@ -32,7 +32,8 @@ struct helper_data { signed_tags : 1, check_connectivity : 1, no_disconnect_req : 1, - no_private_update : 1; + no_private_update : 1, + object_format : 1; /* * As an optimization, the transport code may invoke fetch before @@ -207,6 +208,8 @@ static struct child_process *get_helper(struct transport *transport) data->import_marks = xstrdup(arg); } else if (starts_with(capname, "no-private-update")) { data->no_private_update = 1; + } else if (starts_with(capname, "object-format")) { + data->object_format = 1; } else if (mandatory) { die(_("unknown mandatory capability %s; this remote " "helper probably needs newer version of Git"), @@ -1103,6 +1106,12 @@ static struct ref *get_refs_list_using_list(struct transport *transport, data->get_refs_list_called = 1; helper = get_helper(transport); + if (data->object_format) { + write_str_in_full(helper->in, "option object-format\n"); + if (recvline(data, &buf) || strcmp(buf.buf, "ok")) + exit(128); + } + if (data->push && for_push) write_str_in_full(helper->in, "list for-push\n"); else @@ -1115,6 +1124,17 @@ static struct ref *get_refs_list_using_list(struct transport *transport, if (!*buf.buf) break; + else if (buf.buf[0] == ':') { + const char *value; + if (skip_prefix(buf.buf, ":object-format ", &value)) { + int algo = hash_algo_by_name(value); + if (algo == GIT_HASH_UNKNOWN) + die(_("unsupported object format '%s'"), + value); + transport->hash_algo = &hash_algos[algo]; + } + continue; + } eov = strchr(buf.buf, ' '); if (!eov) @@ -1127,7 +1147,7 @@ static struct ref *get_refs_list_using_list(struct transport *transport, if (buf.buf[0] == '@') (*tail)->symref = xstrdup(buf.buf + 1); else if (buf.buf[0] != '?') - get_oid_hex(buf.buf, &(*tail)->old_oid); + get_oid_hex_algop(buf.buf, &(*tail)->old_oid, transport->hash_algo); if (eon) { if (has_attribute(eon + 1, "unchanged")) { (*tail)->status |= REF_STATUS_UPTODATE; From patchwork Mon May 25 19:59:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11569415 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 812EF912 for ; Mon, 25 May 2020 20:00:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 690272075F for ; Mon, 25 May 2020 20:00:48 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="kO1P/QSO" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390690AbgEYUAr (ORCPT ); Mon, 25 May 2020 16:00:47 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38726 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390414AbgEYT7y (ORCPT ); Mon, 25 May 2020 15:59:54 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 9E9FF60D06; Mon, 25 May 2020 19:59:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1590436790; bh=5OqbgbgT9Ub0EVedjs2/QCeU8Qm807ZRLrM0Pa/VUQY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=kO1P/QSOO48Ovh6+FLiE4yJNsHjLE7SiX8GYkkrqYi9qvatGLPDsEelVvEiFQMkzK t1Gu5i+/+5koBHrow11ynL98l8xAsCrEonHF0m+bCRudYLJWc4WTkMSzjc2C/Noopv l1wyCRrssDsopuK8TN7S1sjeMYWbzJjrgy/bsAbEXk/1gXzO4A21F3QyZayotBwNwJ TuJ2NirmaXL3xD2DDJLD0SrmSVaPAwVl9sgOZXmpWvvZdGC7A48oCfAkAWVZkTSJYz i4u228/8x39/C9rdDlbTKui7h/5rLPgROzRa45cfLHHMgNCkgZc5zdevEaE5XcwbWj 4delEfT0DAdyokM0sFJvcnnWkGV3w15PJVUzqAuJR7nl8ojo7tpYy5vy8tMhrRnYx5 njnwPdriTRaJ1xsA6s8D94jxiJAOurXQDaXSnMRJJ64Gcm4wUQnOleNihaol1NlCqx R+0guUPasURLxUS4nyVjv3SG++7pjpFfFt44mxJNGbqP1dOYEEP From: "brian m. carlson" To: Cc: =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v2 18/44] remote-curl: implement object-format extensions Date: Mon, 25 May 2020 19:59:04 +0000 Message-Id: <20200525195930.309665-19-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.rc0.183.gde8f92d652 In-Reply-To: <20200525195930.309665-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200525195930.309665-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Implement the object-format extensions that let us determine the hash algorithm in use when pushing, pulling, and fetching. Signed-off-by: brian m. carlson --- remote-curl.c | 19 ++++++++++++++++++- 1 file changed, 18 insertions(+), 1 deletion(-) diff --git a/remote-curl.c b/remote-curl.c index 1c9aa3d0ab..3ed0dfec1b 100644 --- a/remote-curl.c +++ b/remote-curl.c @@ -41,7 +41,9 @@ struct options { deepen_relative : 1, from_promisor : 1, no_dependents : 1, - atomic : 1; + atomic : 1, + object_format : 1; + const struct git_hash_algo *hash_algo; }; static struct options options; static struct string_list cas_options = STRING_LIST_INIT_DUP; @@ -190,6 +192,16 @@ static int set_option(const char *name, const char *value) } else if (!strcmp(name, "filter")) { options.filter = xstrdup(value); return 0; + } else if (!strcmp(name, "object-format")) { + int algo; + options.object_format = 1; + if (strcmp(value, "true")) { + algo = hash_algo_by_name(value); + if (algo == GIT_HASH_UNKNOWN) + die("unknown object format '%s'", value); + options.hash_algo = &hash_algos[algo]; + } + return 0; } else { return 1 /* unsupported */; } @@ -231,6 +243,7 @@ static struct ref *parse_git_refs(struct discovery *heads, int for_push) case protocol_v0: get_remote_heads(&reader, &list, for_push ? REF_NORMAL : 0, NULL, &heads->shallow); + options.hash_algo = reader.hash_algo; break; case protocol_unknown_version: BUG("unknown protocol version"); @@ -509,6 +522,9 @@ static struct ref *get_refs(int for_push) static void output_refs(struct ref *refs) { struct ref *posn; + if (options.object_format && options.hash_algo) { + printf(":object-format %s\n", options.hash_algo->name); + } for (posn = refs; posn; posn = posn->next) { if (posn->symref) printf("@%s %s\n", posn->symref, posn->name); @@ -1439,6 +1455,7 @@ int cmd_main(int argc, const char **argv) printf("option\n"); printf("push\n"); printf("check-connectivity\n"); + printf("object-format\n"); printf("\n"); fflush(stdout); } else if (skip_prefix(buf.buf, "stateless-connect ", &arg)) { From patchwork Mon May 25 19:59:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11569393 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E8630739 for ; Mon, 25 May 2020 20:00:34 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C5C752071A for ; Mon, 25 May 2020 20:00:34 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="Mx/WiaVv" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390643AbgEYUAd (ORCPT ); Mon, 25 May 2020 16:00:33 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38722 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390418AbgEYT7y (ORCPT ); Mon, 25 May 2020 15:59:54 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 10B8B60D07; Mon, 25 May 2020 19:59:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1590436791; bh=sxSDhOXns0vesz2DtA7y4piK/iRpR30k7wtrsZNC1VU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=Mx/WiaVv9dKpzs0X329sUQKXSpqUj64H7ir46EQyHVw10NErcL0mB8ajU7O0FLw3J l7Z1BQryfxHyXT6z8RR0x+OLmJVVZkU+ybFx9jHu+P0fNEa1MqhRdaWBO8vGCfCiis C1fOFO7r+hfmoTGiv3nXPZtheW7JPs3NSutTf2rUgIQotyIYB/Kcpz6ZlCmvunC3Mz F5h0b+62g7s0cBIbefUByolb97uY42DJtvmUMghlkJvKx/tg9CsHip8GXMHnvnJamL EcAwx1cZcm9u/vv9Kiz0kq70UUKmpEXvECXaasW5gYzXCNH+xRxs+1Oevta+m7uvF5 YDvzKSVtqf6UbYkpHpn+jkf/08u6M6WGcmK/k6XKKvHe7Zx98scZzVtS2Uxx+xSAGC TzxJ8AwhBFL3wWbO9hy/lM+E7JEe9g+UULfxaKex1GT8bBG6wQ3XNSEaVO7vIMAOqN /spmlxAMF0d9VYtqY1JdQkjGPpWYFCS/KfxBk//Wn+DTQT+z6a3 From: "brian m. carlson" To: Cc: =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v2 19/44] builtin/clone: initialize hash algorithm properly Date: Mon, 25 May 2020 19:59:05 +0000 Message-Id: <20200525195930.309665-20-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.rc0.183.gde8f92d652 In-Reply-To: <20200525195930.309665-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200525195930.309665-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When performing a clone, we don't know what hash algorithm the other end will support. Currently, we don't support fetching data belonging to a different algorithm, so we must know what algorithm the remote side is using in order to properly initialize the repository. We can know that only after fetching the refs, so if the remote side has any references, use that information to reinitialize the repository with the correct hash algorithm information. Signed-off-by: brian m. carlson --- builtin/clone.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/builtin/clone.c b/builtin/clone.c index cb48a291ca..f27d38bc8e 100644 --- a/builtin/clone.c +++ b/builtin/clone.c @@ -1217,6 +1217,15 @@ int cmd_clone(int argc, const char **argv, const char *prefix) refs = transport_get_remote_refs(transport, &ref_prefixes); if (refs) { + int hash_algo = hash_algo_by_ptr(transport_get_hash_algo(transport)); + + /* + * Now that we know what algorithm the remote side is using, + * let's set ours to the same thing. + */ + initialize_repository_version(hash_algo); + repo_set_hash_algo(the_repository, hash_algo); + mapped_refs = wanted_peer_refs(refs, &remote->fetch); /* * transport_get_remote_refs() may return refs with null sha-1 From patchwork Mon May 25 19:59:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11569395 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 660201392 for ; Mon, 25 May 2020 20:00:35 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 44B352075F for ; Mon, 25 May 2020 20:00:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="kJ2oOfe5" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390605AbgEYUAa (ORCPT ); Mon, 25 May 2020 16:00:30 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38720 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390419AbgEYT7y (ORCPT ); Mon, 25 May 2020 15:59:54 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 7747B60D08; Mon, 25 May 2020 19:59:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1590436791; bh=YhcqSF7NVvsfDzpsd5p3oihZbPBd50K1aBOeC4FnGdY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=kJ2oOfe5LOyP18JbAU/o6AGF9eIHJuCvEGDdkRD8/DkuAukUzwTw1/yjcptXq2Oxj XlBo+PCUSgXBRh/L+Xh/rGeX0GBYvPsnm/rovGM7F4omUhj2yyd6it4VDEN5B0HC2t idPodP8EBCj+Dy6DP15Tifi9JVBsb7F3Z5v+lo8CDuq9ZG1BqL+BnxL1LodNc15G/H HVPt4lBm+2bOpqE+VwqtDXmKyyXf8Ol2joH8fEB18j+71OUwvHUziGr9xDnfJs40c/ Ds858uyHZ2IQPtv2hRKRVs7bnXru3IQ3l9j4CFAAuLcSn/k1QNxAkFktQae6yr17zH /mvF2S9HrczFMTwe0T2xeK8SOPuZ0HviN/C5CmhCIGZkktlp2Sv60YguNxNDgzmnSH fdf5oG/+mnGuAkcjjEvIq+COHgDdMSLEY+PfBsDy9RYT5SReF9UlBUqY731uYLy8c1 I0pudUcIX8wGD+HtCUCzP0nWMza4vEVSeGMDORbkKjn+5hSdwSW From: "brian m. carlson" To: Cc: =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v2 20/44] t5562: pass object-format in synthesized test data Date: Mon, 25 May 2020 19:59:06 +0000 Message-Id: <20200525195930.309665-21-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.rc0.183.gde8f92d652 In-Reply-To: <20200525195930.309665-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200525195930.309665-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Ensure that we pass the object-format capability in the synthesized test data so that this test works with algorithms other than SHA-1. In addition, add an additional test using the old data for when we're using SHA-1 so that we can be sure that we preserve backwards compatibility with servers not offering the object-format capability. Signed-off-by: brian m. carlson --- t/t5562-http-backend-content-length.sh | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/t/t5562-http-backend-content-length.sh b/t/t5562-http-backend-content-length.sh index 3f4ac71f83..c6ec625497 100755 --- a/t/t5562-http-backend-content-length.sh +++ b/t/t5562-http-backend-content-length.sh @@ -46,6 +46,7 @@ ssize_b100dots() { } test_expect_success 'setup' ' + test_oid_init && HTTP_CONTENT_ENCODING="identity" && export HTTP_CONTENT_ENCODING && git config http.receivepack true && @@ -62,8 +63,8 @@ test_expect_success 'setup' ' test_copy_bytes 10 fetch_body.trunc && hash_next=$(git commit-tree -p HEAD -m next HEAD^{tree}) && { - printf "%s %s refs/heads/newbranch\\0report-status\\n" \ - "$ZERO_OID" "$hash_next" | packetize && + printf "%s %s refs/heads/newbranch\\0report-status object-format=%s\\n" \ + "$ZERO_OID" "$hash_next" "$(test_oid algo)" | packetize && printf 0000 && echo "$hash_next" | git pack-objects --stdout } >push_body && From patchwork Mon May 25 19:59:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11569391 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D50FF739 for ; Mon, 25 May 2020 20:00:33 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B57702073B for ; Mon, 25 May 2020 20:00:33 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="k7seesj2" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390640AbgEYUAc (ORCPT ); Mon, 25 May 2020 16:00:32 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38728 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390316AbgEYT7y (ORCPT ); Mon, 25 May 2020 15:59:54 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id DDEEB60D09; Mon, 25 May 2020 19:59:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1590436792; bh=vP7nC3ffY6aR9uRzQGpX+7hdzpsv9yyEoRYuzrjpRQw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=k7seesj24Aa76Kh5UqpIOzIoDaqIqVR8tcpcJjtlUJc2Agsl2HC6Vqk4+cIE59pWK foma0JU3G8lsDCZ+4v3dNddG/qdr0J7FGAoLIQHqmzlgkKDuhkZiSBg/jGoDs9c5vO 917pRPYCVulTbw6z/+zIRuz1Dqscn8LgPkrh9k9xoiXRt6ndZbwewHERYlYqAPZH4E j268JaWvKn824DIcMCdp17yFAi2F+shwbMDE7r/j2phvj6Ne4dIKOCmKK6mShDfvaZ J10q2QxGGqRGF9boKwJNTRQrtjtCbhsFocHqr/6p27LMU8VldW4ZPiJtpV0GNb2Rnl JXAW5PSNhVwarQxdrywDsoopXVXp9ULB4UcTl8c0Pni3xyYngWi5T6FgA/MAzK6s/9 K7oaqtNRStPoNCfMDXqpauBkI6JdDqjyptt6ESob+tBvxUohJcCeBCS/XcCYo9vAF5 8zPLyb7x70312NqfJ7+0Es/accM9668nnXuJ6nF9wwIuVyQSGXg From: "brian m. carlson" To: Cc: =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v2 21/44] fetch-pack: parse and advertise the object-format capability Date: Mon, 25 May 2020 19:59:07 +0000 Message-Id: <20200525195930.309665-22-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.rc0.183.gde8f92d652 In-Reply-To: <20200525195930.309665-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200525195930.309665-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Parse the server's object-format capability and respond accordingly, dying if there is a mismatch. Signed-off-by: brian m. carlson --- fetch-pack.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/fetch-pack.c b/fetch-pack.c index 34c339a5fe..ecfb8c3cf5 100644 --- a/fetch-pack.c +++ b/fetch-pack.c @@ -1180,6 +1180,7 @@ static int send_fetch_request(struct fetch_negotiator *negotiator, int fd_out, int sideband_all, int seen_ack) { int ret = 0; + const char *hash_name; struct strbuf req_buf = STRBUF_INIT; if (server_supports_v2("fetch", 1)) @@ -1194,6 +1195,17 @@ static int send_fetch_request(struct fetch_negotiator *negotiator, int fd_out, args->server_options->items[i].string); } + if (server_feature_v2("object-format", &hash_name)) { + int hash_algo = hash_algo_by_name(hash_name); + if (hash_algo_by_ptr(the_hash_algo) != hash_algo) + die(_("mismatched algorithms: client %s; server %s"), + the_hash_algo->name, hash_name); + packet_write_fmt(fd_out, "object-format=%s", the_hash_algo->name); + } else if (hash_algo_by_ptr(the_hash_algo) != GIT_HASH_SHA1) { + die(_("the server does not support algorithm '%s'"), + the_hash_algo->name); + } + packet_buf_delim(&req_buf); if (args->use_thin_pack) packet_buf_write(&req_buf, "thin-pack"); From patchwork Mon May 25 19:59:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11569411 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B2883739 for ; Mon, 25 May 2020 20:00:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 91C152073B for ; Mon, 25 May 2020 20:00:46 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="UfukM/5j" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390680AbgEYUAq (ORCPT ); Mon, 25 May 2020 16:00:46 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38844 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390570AbgEYUAY (ORCPT ); Mon, 25 May 2020 16:00:24 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 53CBE60D0A; Mon, 25 May 2020 19:59:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1590436792; bh=5tWITec1XvUpGU34jbqxhafBFLo7Ap2izOu3L993MKI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=UfukM/5j5oXTgDcwVw2bFQ8SsTT5lyfqKlHaVPaGdk2aZP9ZMoxVdvzotVoxH/cnD b2l+h4HHPams+ZSo4fL5qlP/EMvOGdICAiXZlVPtYqNUcUy0g7N/S04ziNlnOg2L4O 4q4qsyPqP2Me2IwIPmCEeZvU5RM5TnbxbaeL5HmFY9lZ2PytY/B1VA732YtvpcoY7t ABvYcls48u6de2GTTqnkq+U/nvbNbIPmX78utaGU5LsGKcdYbRmEiQuLifbOI1nzat h3JpNCw9HgX065no15d54yN+/ll/KZFHCQ9UpUhOvQbbGAPYn7tVTuktkGTpqkpyaR E9t0luF1X3KswWktoWEZVYxpjArQxJvyn2X2R6LeiRraBnOxnxfrSsRRlim+CAZRsp lpfHTuEFmYMXb6LQ0xBJumqTZp156FNVlh/AIk5iKeXul43vvqHVtzBjbcSprmZqyd FxAsdFvO/7aB/IOXvrKTgaKWwpbwb/6pQY/S1rWt1frvsZbdMaL From: "brian m. carlson" To: Cc: =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v2 22/44] setup: set the_repository's hash algo when checking format Date: Mon, 25 May 2020 19:59:08 +0000 Message-Id: <20200525195930.309665-23-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.rc0.183.gde8f92d652 In-Reply-To: <20200525195930.309665-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200525195930.309665-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When we're checking the repository's format, set the hash algorithm at the same time. This ensures that we perform a suitable initialization early enough to avoid confusing any parts of the code. If we defer until later, we can end up with portions of the code which are confused about the hash algorithm, resulting in segfaults when working with SHA-256 repositories. Signed-off-by: brian m. carlson --- setup.c | 1 + 1 file changed, 1 insertion(+) diff --git a/setup.c b/setup.c index 65fe5ecefb..019a1c6367 100644 --- a/setup.c +++ b/setup.c @@ -1273,6 +1273,7 @@ void check_repository_format(struct repository_format *fmt) fmt = &repo_fmt; check_repository_format_gently(get_git_dir(), fmt, NULL); startup_info->have_repository = 1; + repo_set_hash_algo(the_repository, fmt->hash_algo); clear_repository_format(&repo_fmt); } From patchwork Mon May 25 19:59:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11569379 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 48BFA1392 for ; Mon, 25 May 2020 20:00:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3225A2071A for ; Mon, 25 May 2020 20:00:23 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="PZdJ2E1y" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390569AbgEYUAV (ORCPT ); Mon, 25 May 2020 16:00:21 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38718 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390422AbgEYT7z (ORCPT ); Mon, 25 May 2020 15:59:55 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id BC41B60D0B; Mon, 25 May 2020 19:59:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1590436793; bh=MMfJEd2BJ7WjAZeGiNm4voySOTAFCGpDlJvsq+dHUZY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=PZdJ2E1y28B+F9+GmIN3BCwy4uNl9OJLV6AIAp7FiQi3EOHNWvXesHNerAKj3jzcG zrd+9XfHUrxPUlh++IiU6HGtp4GxQ/KyYfQkppS2ZPZdUMhXBOkuZl/F3hJSC8v0MQ hW62QD2dC/3TD83Q2LsXsTcEFfOwGPWc+MM8+FSL6ch87Gq53bpnM9R8hPmyuc1iGQ 7U70tpE/VsY+rgF67esA0PL+MnpIQhPr0hRBqqFlSWwutQwob2pf04ghWpRRwO3bKY 0XCnaJOSDv7OKsB5rSZRhdfc4w1g0nZxsDAhBccfEbFPa1n/EJaeB7pHW2iH0J02Oy 9cEDCd0a03eJZysKE8+T2z5bWx14kPoG5aSvMFlaZEmQbnaFWS/oPeG5MKlX8jr2Y3 nnfeI1lWIGUUgvOpY77FCmLUSCx1I9UdLBM3Qj+GioeCv1UowaS6C+KmVGEnMKsUmB LssU4fT/MV7xrZV+heZ46AsGxB8XThQ+55T1dq+EE91qqm3MF/X From: "brian m. carlson" To: Cc: =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v2 23/44] t3200: mark assertion with SHA1 prerequisite Date: Mon, 25 May 2020 19:59:09 +0000 Message-Id: <20200525195930.309665-24-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.rc0.183.gde8f92d652 In-Reply-To: <20200525195930.309665-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200525195930.309665-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org One of the test assertions in this test checks that git branch -m works even without a .git/config file. However, if the repository requires configuration extensions, such as because it uses a non-SHA-1 algorithm, this assertion will fail. Mark the assertion as requiring SHA-1. Signed-off-by: brian m. carlson --- t/t3200-branch.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/t/t3200-branch.sh b/t/t3200-branch.sh index 411a70b0ce..2a3fedc6b0 100755 --- a/t/t3200-branch.sh +++ b/t/t3200-branch.sh @@ -402,7 +402,7 @@ EOF mv .git/config .git/config-saved -test_expect_success 'git branch -m q q2 without config should succeed' ' +test_expect_success SHA1 'git branch -m q q2 without config should succeed' ' git branch -m q q2 && git branch -m q2 q ' From patchwork Mon May 25 19:59:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11569409 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id ED465739 for ; Mon, 25 May 2020 20:00:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D64942073B for ; Mon, 25 May 2020 20:00:44 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="JDs/hfbb" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390675AbgEYUAn (ORCPT ); Mon, 25 May 2020 16:00:43 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38846 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390575AbgEYUAZ (ORCPT ); Mon, 25 May 2020 16:00:25 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 39BDE60D0C; Mon, 25 May 2020 19:59:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1590436793; bh=+4Z4pf0T9fzuVPGr+GKKdEY7iEgxxGa0MzEFtugw7Z8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=JDs/hfbb9aOzMpxWswecy58Lh9MwLU8gVDqSCY9xagAeFoLhkTJrqsyws04MmC0Zw lQ9BpmRGP2cfgBqSll3lPVVoJLEmmgNvErJVtZG/ZYf79yHirGOsRhq+oMc9NsP6IM lNiY0RRCv9Iw5qDans5pPlCz0+bPsxszEdXBsRiCp3Hw9gpAFgQTB6NOZ8jR+8PEAl wBCUdKwE+7pVt4C3bxjzJOofv76G6y3VV97RZUqGgiSEtgWbBwL/MS8bHDUzvvecK/ WT3/O7fuu/cOSWuBILBzW5Ft8grbKDTkV2zhmFqX9oQVEiF8IprYFCqKkIFxw7fO/O A2qPXLbq3N1/GLUOE2aNdiNTr4P3s44WoEbpL1IykZLPc6sitOjIf5UtNiRvjUzxfv IhQxyQHASAp3CAyJ1d+UPfOzqLdUIR06MPbbrdmIYEkisa6rXcrtRQ2XbGfO2T6MKT blOJppIOXRllm+KarjPLc8JWqoZbTag3NVrRxFUptWJyKgGPQtK From: "brian m. carlson" To: Cc: =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v2 24/44] packfile: compute and use the index CRC offset Date: Mon, 25 May 2020 19:59:10 +0000 Message-Id: <20200525195930.309665-25-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.rc0.183.gde8f92d652 In-Reply-To: <20200525195930.309665-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200525195930.309665-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Both v2 pack index files and the v3 format specified as part of the NewHash work have similar data starting at the CRC table. Much of the existing code wants to read either this table or the offset entries following it, and in doing so computes the offset each time. In order to share as much code between v2 and v3, compute the offset of the CRC table and store it when the pack is opened. Use this value to compute offsets to not only the CRC table, but to the offset entries beyond it. Signed-off-by: brian m. carlson --- builtin/index-pack.c | 6 +----- object-store.h | 1 + packfile.c | 1 + 3 files changed, 3 insertions(+), 5 deletions(-) diff --git a/builtin/index-pack.c b/builtin/index-pack.c index f176dd28c8..7bea1fba52 100644 --- a/builtin/index-pack.c +++ b/builtin/index-pack.c @@ -1555,13 +1555,9 @@ static void read_v2_anomalous_offsets(struct packed_git *p, { const uint32_t *idx1, *idx2; uint32_t i; - const uint32_t hashwords = the_hash_algo->rawsz / sizeof(uint32_t); /* The address of the 4-byte offset table */ - idx1 = (((const uint32_t *)p->index_data) - + 2 /* 8-byte header */ - + 256 /* fan out */ - + hashwords * p->num_objects /* object ID table */ + idx1 = (((const uint32_t *)((const uint8_t *)p->index_data + p->crc_offset)) + p->num_objects /* CRC32 table */ ); diff --git a/object-store.h b/object-store.h index d1e490f203..f439d47af8 100644 --- a/object-store.h +++ b/object-store.h @@ -70,6 +70,7 @@ struct packed_git { size_t index_size; uint32_t num_objects; uint32_t num_bad_objects; + uint32_t crc_offset; unsigned char *bad_object_sha1; int index_version; time_t mtime; diff --git a/packfile.c b/packfile.c index f4e752996d..6ab5233613 100644 --- a/packfile.c +++ b/packfile.c @@ -178,6 +178,7 @@ int load_idx(const char *path, const unsigned int hashsz, void *idx_map, */ (sizeof(off_t) <= 4)) return error("pack too large for current definition of off_t in %s", path); + p->crc_offset = 8 + 4 * 256 + nr * hashsz; } p->index_version = version; From patchwork Mon May 25 19:59:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11569351 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 37D7D739 for ; Mon, 25 May 2020 20:00:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1580C20776 for ; Mon, 25 May 2020 20:00:03 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="F4tY0/xk" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390462AbgEYUAB (ORCPT ); Mon, 25 May 2020 16:00:01 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38726 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390398AbgEYT75 (ORCPT ); Mon, 25 May 2020 15:59:57 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id A16EF60D0D; Mon, 25 May 2020 19:59:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1590436793; bh=pKT7D8FfW9+6hGmCBXYrcHQTegDW9wGONuHCaQuqves=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=F4tY0/xkTn4WN5dqEVudIm8qfepSqvlZihtv3f+SH2FKntKJZq7aalNemz3gSLee0 UfJznd1AchDHKIIabsLwOD1D2lYTX6ubqdSMprfXLXk0vHNpyaDbl0bF5req8Ff9S2 hh46XbytnR0jyrUuLjJ0ylU4AnVtCrOX0DiGSasccST78VIN7rrKHuE7y+nBbuN+am zBlwS/VXVJLWJpZHRl+f3Q4P7ghRPFdqZadpHl0UmhEs42Kxg8ie9JfYTzvSk1s2J+ 97HjCJ/DdJFRwU+7gxAfbAMeM3ZrByL7JEM0aAjuw+yQoRo94peRtWCjYZhWrbczsb rOY8QCCp0haSmRHM3By8HUmldLZLHxL9QeriC4r2egfTKnWzm28nuGee91btKnWBao qd5CtHzWWixpJISUwZH1eo5lrVWQImcvrye5OkYpCPkJhaYakmKtvfvP3ivtPCF+4K PlKFAEQmDu5BNKtNm2fHf0NOdPF1brTM1pUoVzoJMMi5PaFpFD+ From: "brian m. carlson" To: Cc: =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v2 25/44] t5302: modernize test formatting Date: Mon, 25 May 2020 19:59:11 +0000 Message-Id: <20200525195930.309665-26-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.rc0.183.gde8f92d652 In-Reply-To: <20200525195930.309665-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200525195930.309665-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Our style these days is to place the description and the opening quote of the body on the same line as test_expect_success (if it fits), to place the trailing quote on a line by itself after the body, and to use tabs. Since we're going to be making several significant changes to this test, modernize the style to aid in readability of the subsequent patches. This patch should have no functional change. Signed-off-by: brian m. carlson --- t/t5302-pack-index.sh | 360 +++++++++++++++++++++--------------------- 1 file changed, 184 insertions(+), 176 deletions(-) diff --git a/t/t5302-pack-index.sh b/t/t5302-pack-index.sh index ad07f2f7fc..8981c9b90e 100755 --- a/t/t5302-pack-index.sh +++ b/t/t5302-pack-index.sh @@ -7,65 +7,65 @@ test_description='pack index with 64-bit offsets and object CRC' . ./test-lib.sh test_expect_success 'setup' ' - test_oid_init && - rawsz=$(test_oid rawsz) && - rm -rf .git && - git init && - git config pack.threads 1 && - i=1 && - while test $i -le 100 - do - iii=$(printf '%03i' $i) - test-tool genrandom "bar" 200 > wide_delta_$iii && - test-tool genrandom "baz $iii" 50 >> wide_delta_$iii && - test-tool genrandom "foo"$i 100 > deep_delta_$iii && - test-tool genrandom "foo"$(expr $i + 1) 100 >> deep_delta_$iii && - test-tool genrandom "foo"$(expr $i + 2) 100 >> deep_delta_$iii && - echo $iii >file_$iii && - test-tool genrandom "$iii" 8192 >>file_$iii && - git update-index --add file_$iii deep_delta_$iii wide_delta_$iii && - i=$(expr $i + 1) || return 1 - done && - { echo 101 && test-tool genrandom 100 8192; } >file_101 && - git update-index --add file_101 && - tree=$(git write-tree) && - commit=$(git commit-tree $tree obj-list && - git update-ref HEAD $commit + test_oid_init && + rawsz=$(test_oid rawsz) && + rm -rf .git && + git init && + git config pack.threads 1 && + i=1 && + while test $i -le 100 + do + iii=$(printf '%03i' $i) + test-tool genrandom "bar" 200 > wide_delta_$iii && + test-tool genrandom "baz $iii" 50 >> wide_delta_$iii && + test-tool genrandom "foo"$i 100 > deep_delta_$iii && + test-tool genrandom "foo"$(expr $i + 1) 100 >> deep_delta_$iii && + test-tool genrandom "foo"$(expr $i + 2) 100 >> deep_delta_$iii && + echo $iii >file_$iii && + test-tool genrandom "$iii" 8192 >>file_$iii && + git update-index --add file_$iii deep_delta_$iii wide_delta_$iii && + i=$(expr $i + 1) || return 1 + done && + { echo 101 && test-tool genrandom 100 8192; } >file_101 && + git update-index --add file_101 && + tree=$(git write-tree) && + commit=$(git commit-tree $tree obj-list && + git update-ref HEAD $commit ' -test_expect_success \ - 'pack-objects with index version 1' \ - 'pack1=$(git pack-objects --index-version=1 test-1 &1) || ! (echo "$msg" | grep "pack too large .* off_t") @@ -91,21 +91,21 @@ else say "# skipping tests concerning 64-bit offsets" fi -test_expect_success OFF64_T \ - 'index v2: verify a pack with some 64-bit offsets' \ - 'git verify-pack -v "test-3-${pack3}.pack"' +test_expect_success OFF64_T 'index v2: verify a pack with some 64-bit offsets' ' + git verify-pack -v "test-3-${pack3}.pack" +' -test_expect_success OFF64_T \ - '64-bit offsets: should be different from previous index v2 results' \ - '! cmp "test-2-${pack2}.idx" "test-3-${pack3}.idx"' +test_expect_success OFF64_T '64-bit offsets: should be different from previous index v2 results' ' + ! cmp "test-2-${pack2}.idx" "test-3-${pack3}.idx" +' -test_expect_success OFF64_T \ - 'index v2: force some 64-bit offsets with index-pack' \ - 'git index-pack --index-version=2,0x40000 -o 3.idx "test-1-${pack1}.pack"' +test_expect_success OFF64_T 'index v2: force some 64-bit offsets with index-pack' ' + git index-pack --index-version=2,0x40000 -o 3.idx "test-1-${pack1}.pack" +' -test_expect_success OFF64_T \ - '64-bit offsets: index-pack result should match pack-objects one' \ - 'cmp "test-3-${pack3}.idx" "3.idx"' +test_expect_success OFF64_T '64-bit offsets: index-pack result should match pack-objects one' ' + cmp "test-3-${pack3}.idx" "3.idx" +' test_expect_success OFF64_T 'index-pack --verify on 64-bit offset v2 (cheat)' ' # This cheats by knowing which lower offset should still be encoded @@ -120,135 +120,143 @@ test_expect_success OFF64_T 'index-pack --verify on 64-bit offset v2' ' # returns the object number for given object in given pack index index_obj_nr() { - idx_file=$1 - object_sha1=$2 - nr=0 - git show-index < $idx_file | - while read offs sha1 extra - do - nr=$(($nr + 1)) - test "$sha1" = "$object_sha1" || continue - echo "$(($nr - 1))" - break - done + idx_file=$1 + object_sha1=$2 + nr=0 + git show-index < $idx_file | + while read offs sha1 extra + do + nr=$(($nr + 1)) + test "$sha1" = "$object_sha1" || continue + echo "$(($nr - 1))" + break + done } # returns the pack offset for given object as found in given pack index index_obj_offset() { - idx_file=$1 - object_sha1=$2 - git show-index < $idx_file | grep $object_sha1 | - ( read offs extra && echo "$offs" ) + idx_file=$1 + object_sha1=$2 + git show-index < $idx_file | grep $object_sha1 | + ( read offs extra && echo "$offs" ) } -test_expect_success \ - '[index v1] 1) stream pack to repository' \ - 'git index-pack --index-version=1 --stdin < "test-1-${pack1}.pack" && - git prune-packed && - git count-objects | ( read nr rest && test "$nr" -eq 1 ) && - cmp "test-1-${pack1}.pack" ".git/objects/pack/pack-${pack1}.pack" && - cmp "test-1-${pack1}.idx" ".git/objects/pack/pack-${pack1}.idx"' +test_expect_success '[index v1] 1) stream pack to repository' ' + git index-pack --index-version=1 --stdin < "test-1-${pack1}.pack" && + git prune-packed && + git count-objects | ( read nr rest && test "$nr" -eq 1 ) && + cmp "test-1-${pack1}.pack" ".git/objects/pack/pack-${pack1}.pack" && + cmp "test-1-${pack1}.idx" ".git/objects/pack/pack-${pack1}.idx" +' test_expect_success \ - '[index v1] 2) create a stealth corruption in a delta base reference' \ - '# This test assumes file_101 is a delta smaller than 16 bytes. - # It should be against file_100 but we substitute its base for file_099 - sha1_101=$(git hash-object file_101) && - sha1_099=$(git hash-object file_099) && - offs_101=$(index_obj_offset 1.idx $sha1_101) && - nr_099=$(index_obj_nr 1.idx $sha1_099) && - chmod +w ".git/objects/pack/pack-${pack1}.pack" && - recordsz=$((rawsz + 4)) && - dd of=".git/objects/pack/pack-${pack1}.pack" seek=$(($offs_101 + 1)) \ - if=".git/objects/pack/pack-${pack1}.idx" \ - skip=$((4 + 256 * 4 + $nr_099 * recordsz)) \ - bs=1 count=$rawsz conv=notrunc && - git cat-file blob $sha1_101 > file_101_foo1' + '[index v1] 2) create a stealth corruption in a delta base reference' ' + # This test assumes file_101 is a delta smaller than 16 bytes. + # It should be against file_100 but we substitute its base for file_099 + sha1_101=$(git hash-object file_101) && + sha1_099=$(git hash-object file_099) && + offs_101=$(index_obj_offset 1.idx $sha1_101) && + nr_099=$(index_obj_nr 1.idx $sha1_099) && + chmod +w ".git/objects/pack/pack-${pack1}.pack" && + recordsz=$((rawsz + 4)) && + dd of=".git/objects/pack/pack-${pack1}.pack" seek=$(($offs_101 + 1)) \ + if=".git/objects/pack/pack-${pack1}.idx" \ + skip=$((4 + 256 * 4 + $nr_099 * recordsz)) \ + bs=1 count=$rawsz conv=notrunc && + git cat-file blob $sha1_101 > file_101_foo1 +' test_expect_success \ - '[index v1] 3) corrupted delta happily returned wrong data' \ - 'test -f file_101_foo1 && ! cmp file_101 file_101_foo1' + '[index v1] 3) corrupted delta happily returned wrong data' ' + test -f file_101_foo1 && ! cmp file_101 file_101_foo1 +' test_expect_success \ - '[index v1] 4) confirm that the pack is actually corrupted' \ - 'test_must_fail git fsck --full $commit' + '[index v1] 4) confirm that the pack is actually corrupted' ' + test_must_fail git fsck --full $commit +' test_expect_success \ - '[index v1] 5) pack-objects happily reuses corrupted data' \ - 'pack4=$(git pack-objects test-4 file_101_foo2 +' test_expect_success \ - '[index v2] 1) stream pack to repository' \ - 'rm -f .git/objects/pack/* && - git index-pack --index-version=2 --stdin < "test-1-${pack1}.pack" && - git prune-packed && - git count-objects | ( read nr rest && test "$nr" -eq 1 ) && - cmp "test-1-${pack1}.pack" ".git/objects/pack/pack-${pack1}.pack" && - cmp "test-2-${pack1}.idx" ".git/objects/pack/pack-${pack1}.idx"' + '[index v2] 3) corrupted delta happily returned wrong data' ' + test -f file_101_foo2 && ! cmp file_101 file_101_foo2 +' test_expect_success \ - '[index v2] 2) create a stealth corruption in a delta base reference' \ - '# This test assumes file_101 is a delta smaller than 16 bytes. - # It should be against file_100 but we substitute its base for file_099 - sha1_101=$(git hash-object file_101) && - sha1_099=$(git hash-object file_099) && - offs_101=$(index_obj_offset 1.idx $sha1_101) && - nr_099=$(index_obj_nr 1.idx $sha1_099) && - chmod +w ".git/objects/pack/pack-${pack1}.pack" && - dd of=".git/objects/pack/pack-${pack1}.pack" seek=$(($offs_101 + 1)) \ - if=".git/objects/pack/pack-${pack1}.idx" \ - skip=$((8 + 256 * 4 + $nr_099 * rawsz)) \ - bs=1 count=$rawsz conv=notrunc && - git cat-file blob $sha1_101 > file_101_foo2' + '[index v2] 4) confirm that the pack is actually corrupted' ' + test_must_fail git fsck --full $commit +' test_expect_success \ - '[index v2] 3) corrupted delta happily returned wrong data' \ - 'test -f file_101_foo2 && ! cmp file_101 file_101_foo2' + '[index v2] 5) pack-objects refuses to reuse corrupted data' ' + test_must_fail git pack-objects test-5 /dev/null || exit 1 - done /dev/null || exit 1 + done wrong-tag <wrong-tag <err && - grep "^warning:.* expected .tagger. line" err + tag=$(git hash-object -t tag -w --stdin err && + grep "^warning:.* expected .tagger. line" err ' test_expect_success 'index-pack --fsck-objects also warns upon missing tagger in tag' ' - git index-pack --fsck-objects tag-test-${pack1}.pack 2>err && - grep "^warning:.* expected .tagger. line" err + git index-pack --fsck-objects tag-test-${pack1}.pack 2>err && + grep "^warning:.* expected .tagger. line" err ' test_done From patchwork Mon May 25 19:59:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11569407 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8B927739 for ; Mon, 25 May 2020 20:00:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7424B2073B for ; Mon, 25 May 2020 20:00:43 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="C6fpyAfY" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390668AbgEYUAm (ORCPT ); Mon, 25 May 2020 16:00:42 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38848 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390580AbgEYUA1 (ORCPT ); Mon, 25 May 2020 16:00:27 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 163EE60D0E; Mon, 25 May 2020 19:59:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1590436794; bh=KCNvkesVvwvNQy/hd5h5tkTw+MplM9cz3ZpWZNGYF3U=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=C6fpyAfYo8ypPvUBDGqQLlQKHZCOzhM8rSXJdR3IquXigmMScC0B8SGWAVkLRHvk+ 0dEyGMREsMkFA5WVS0OCzzOdbgvK9q+QwVr5ssxfKAYaDYsKCAHuViW4HpgFjUuHiW jwIZkR4NdBbF+pdpY32tbwBTGrOHtVpodVgk+DntSApoO3fSbczIHd/FuF7BjpxlFQ ZSIhP8xfy0DUAOV7mYsZqSX7jlGQ5cswEhDO9BNKNzkFekPfVeHFfhDJz+IgFGO6/O 25GmXAHAHerHGx2eGi6hFnVVz3zO8lG2t3FVN71ZsHBatDEymN9ZPFlNAogDLZHBEf Ic6mWnSVtdHIYZ4h7uJ14w4OZarVdsBRiPY2RCyS7ug9uXzQz8KJ/SrkObd5ixwLDO xubx/GNAL21Ato/f9cIvw5r0i3nPAadjqz7axkLhrRKsC3z9v7EEc+4Vx7iF4GjyRZ zydi7b3iQOGvpkyAdUsADM+2TVb6uOkVWbmGqSDCM9YPPJ3rZ/b From: "brian m. carlson" To: Cc: =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v2 26/44] builtin/show-index: provide options to determine hash algo Date: Mon, 25 May 2020 19:59:12 +0000 Message-Id: <20200525195930.309665-27-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.rc0.183.gde8f92d652 In-Reply-To: <20200525195930.309665-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200525195930.309665-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org show-index is capable of reading any possible index file whether or not the index is inside a repository. However, because our index files lack metadata about the hash algorithm in use, it's not possible to autodetect the algorithm that a particular index file is using. In order to allow us to read index files of any algorithm, let's set up the .git directory gently so that we default to the algorithm for the current repository, and add an --object-format option to allow users to override this setting and continue to run show-index outside of a repository altogether. Let's also document this new option so that people can find it and use it. Signed-off-by: brian m. carlson --- Documentation/git-show-index.txt | 11 ++++++++++- builtin/show-index.c | 29 ++++++++++++++++++++++++----- git.c | 2 +- 3 files changed, 35 insertions(+), 7 deletions(-) diff --git a/Documentation/git-show-index.txt b/Documentation/git-show-index.txt index 424e4ba84c..39b1d8eaa1 100644 --- a/Documentation/git-show-index.txt +++ b/Documentation/git-show-index.txt @@ -9,7 +9,7 @@ git-show-index - Show packed archive index SYNOPSIS -------- [verse] -'git show-index' +'git show-index' [--object-format=] DESCRIPTION @@ -36,6 +36,15 @@ Note that you can get more information on a packfile by calling linkgit:git-verify-pack[1]. However, as this command considers only the index file itself, it's both faster and more flexible. +OPTIONS +------- + +--object-format=:: + Specify the given object format (hash algorithm) for the index file. The + valid values are 'sha1' and (if enabled) 'sha256'. The default is the + algorithm for the current repository (set by `extensions.objectFormat`), or + 'sha1' if no value is set or outside a repository.. + GIT --- Part of the linkgit:git[1] suite diff --git a/builtin/show-index.c b/builtin/show-index.c index 0826f6a5a2..8106b03a6b 100644 --- a/builtin/show-index.c +++ b/builtin/show-index.c @@ -1,9 +1,12 @@ #include "builtin.h" #include "cache.h" #include "pack.h" +#include "parse-options.h" -static const char show_index_usage[] = -"git show-index"; +static const char *const show_index_usage[] = { + "git show-index [--object-format=]", + NULL +}; int cmd_show_index(int argc, const char **argv, const char *prefix) { @@ -11,10 +14,26 @@ int cmd_show_index(int argc, const char **argv, const char *prefix) unsigned nr; unsigned int version; static unsigned int top_index[256]; - const unsigned hashsz = the_hash_algo->rawsz; + unsigned hashsz; + const char *hash_name = NULL; + int hash_algo; + const struct option show_index_options[] = { + OPT_STRING(0, "object-format", &hash_name, N_("hash-algorithm"), + N_("specify the hash algorithm to use")), + OPT_END() + }; + + argc = parse_options(argc, argv, prefix, show_index_options, show_index_usage, 0); + + if (hash_name) { + hash_algo = hash_algo_by_name(hash_name); + if (hash_algo == GIT_HASH_UNKNOWN) + die(_("Unknown hash algorithm")); + repo_set_hash_algo(the_repository, hash_algo); + } + + hashsz = the_hash_algo->rawsz; - if (argc != 1) - usage(show_index_usage); if (fread(top_index, 2 * 4, 1, stdin) != 1) die("unable to read header"); if (top_index[0] == htonl(PACK_IDX_SIGNATURE)) { diff --git a/git.c b/git.c index a2d337eed7..2f021b97f3 100644 --- a/git.c +++ b/git.c @@ -574,7 +574,7 @@ static struct cmd_struct commands[] = { { "shortlog", cmd_shortlog, RUN_SETUP_GENTLY | USE_PAGER }, { "show", cmd_show, RUN_SETUP }, { "show-branch", cmd_show_branch, RUN_SETUP }, - { "show-index", cmd_show_index }, + { "show-index", cmd_show_index, RUN_SETUP_GENTLY }, { "show-ref", cmd_show_ref, RUN_SETUP }, { "sparse-checkout", cmd_sparse_checkout, RUN_SETUP | NEED_WORK_TREE }, { "stage", cmd_add, RUN_SETUP | NEED_WORK_TREE }, From patchwork Mon May 25 19:59:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11569349 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1CAD3739 for ; Mon, 25 May 2020 20:00:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EB3322075F for ; Mon, 25 May 2020 20:00:01 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="lf40IDtk" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390459AbgEYT77 (ORCPT ); Mon, 25 May 2020 15:59:59 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38716 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390431AbgEYT75 (ORCPT ); Mon, 25 May 2020 15:59:57 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 7EC8860D0F; Mon, 25 May 2020 19:59:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1590436794; bh=VVQ0dANHQNwdWuSaocgt9e3EABF8vgKVBdEBtDLaQuI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=lf40IDtkEP5vng/dKzEZfo5j5c0cN/xUbJqt3FGZoY4s1I8hopWclsx0PtSj1q83h DSkzocHKwmnzr7mvot9B3tLUIylCvE6y4/WssxnTfB2FkiosT1uHAe6qZDqKqbWkDp fpwQk805kDSKSs+zaE517+WTParCUO2Y+hWn+iIII3v2rmSjQQ2LLjiX6VlDQUKtR8 Tsi1BTIPOHNzE765KYtZFjX+FQrxNZX5vhxicOvniu0oopwFQ4ui1B8eQpe0qpBI4a XJqC31IBNBUKuotzll44/oWp7dfr1qlMISODT7l9QFb/JeenKG06ieHEfunIX7lEqP LhR7Tv+rgE7wrSng5nsk7NfkcwgifgD1uqYna1ZORqcsLZMyXORqkimsqTkRZHNExT W6Ubraefk9JVb3RFZ4NKFdWCqXJWhx/VIpf1ttGi67Vfa7ZfLiXdVh8JxfUIsqSirU tV9cZgCWVMsAStPzPzFrhBJo7hQqsMoUyR3BdpI9z0bAc6voPw6 From: "brian m. carlson" To: Cc: =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v2 27/44] t1302: expect repo format version 1 for SHA-256 Date: Mon, 25 May 2020 19:59:13 +0000 Message-Id: <20200525195930.309665-28-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.rc0.183.gde8f92d652 In-Reply-To: <20200525195930.309665-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200525195930.309665-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When using SHA-256, we need to take advantage of the extensions section in the config file, so we need to use repository format version 1. Update the test to look for the correct value. Note that test_oid produces a value without a trailing newline, so use echo to ensure we print a trailing newline to compare it correctly against the actual results. Signed-off-by: brian m. carlson --- t/t1302-repo-version.sh | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/t/t1302-repo-version.sh b/t/t1302-repo-version.sh index ce4cff13bb..d60c042ce8 100755 --- a/t/t1302-repo-version.sh +++ b/t/t1302-repo-version.sh @@ -8,6 +8,10 @@ test_description='Test repository version check' . ./test-lib.sh test_expect_success 'setup' ' + test_oid_cache <<-\EOF && + version sha1:0 + version sha256:1 + EOF cat >test.patch <<-\EOF && diff --git a/test.txt b/test.txt new file mode 100644 @@ -23,7 +27,7 @@ test_expect_success 'setup' ' ' test_expect_success 'gitdir selection on normal repos' ' - echo 0 >expect && + echo $(test_oid version) >expect && git config core.repositoryformatversion >actual && git -C test config core.repositoryformatversion >actual2 && test_cmp expect actual && From patchwork Mon May 25 19:59:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11569405 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 29324739 for ; Mon, 25 May 2020 20:00:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 10CFE2071A for ; Mon, 25 May 2020 20:00:42 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="Y1tghg9L" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390667AbgEYUAk (ORCPT ); Mon, 25 May 2020 16:00:40 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38850 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390582AbgEYUA1 (ORCPT ); Mon, 25 May 2020 16:00:27 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id E696860D10; Mon, 25 May 2020 19:59:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1590436795; bh=GiBRjTyb6hSilLNak5pYDTZoH59+gji1GPh9ELc0b1c=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=Y1tghg9L5cEu+QhNyqiUYq+cYUQ6dcvRXHlS3X4MGBGB2eKp68Y4kVK9d2MZp9wLQ PxPJA+Yca7hBvTdDkTYY5LquJI/Oo3fP8UnRA6Pb9Cuxf8+fOzG7qSEGyXEfID4HnR 5JPgx9zc+Et1Ljc0IUDRtYMVhI9MrQ1p70osOX5fbC2QR/6LxkUNayj2jce8MDirNy 59WeGZu2+adA8slXxaWchQhZfUDH1lkJSXQ9zzVdu3gEwsM0MQVcEFhpJnss8NZJzY tcyQjw6nNO1W/t/ilxcAWd7INwwdH+Dze+wF0ZvujQnxsBTnTqwODm0Ke/HAL2wFYk mKSDT5STeds/6mmuIb0jSID17MNze8CBxrD3LBVEAdowO0BvltpUphSLAC43dP9knh qtZxp/3cD9RTuXdkGcE4eaeiTiD87bYzfbsABKZMqEYMQZKgzyoIfTqxeMCPsdwZ5l OjyYOolGZ4zb5sMzQoehwtaUh8DsjA3aOOzDkBCruA/nVRISRGP From: "brian m. carlson" To: Cc: =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v2 28/44] Documentation/technical: document object-format for protocol v2 Date: Mon, 25 May 2020 19:59:14 +0000 Message-Id: <20200525195930.309665-29-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.rc0.183.gde8f92d652 In-Reply-To: <20200525195930.309665-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200525195930.309665-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Document the object-format extension for protocol v2. Signed-off-by: brian m. carlson --- Documentation/technical/protocol-v2.txt | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/Documentation/technical/protocol-v2.txt b/Documentation/technical/protocol-v2.txt index 7e3766cafb..107e421fb7 100644 --- a/Documentation/technical/protocol-v2.txt +++ b/Documentation/technical/protocol-v2.txt @@ -453,3 +453,12 @@ included in a request. This is done by sending each option as a a request. The provided options must not contain a NUL or LF character. + + object-format +~~~~~~~~~~~~~~~ + +The server can advertise the `object-format` capability with a value `X` (in the +form `object-format=X`) to notify the client that the server is able to deal +with objects using hash algorithm X. If not specified, the server is assumed to +only handle SHA-1. If the client would like to use a hash algorithm other than +SHA-1, it should specify its object-format string. From patchwork Mon May 25 19:59:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11569377 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2C147739 for ; Mon, 25 May 2020 20:00:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0B99D2073B for ; Mon, 25 May 2020 20:00:23 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="mpMLrE17" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390567AbgEYUAU (ORCPT ); Mon, 25 May 2020 16:00:20 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38728 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390435AbgEYT75 (ORCPT ); Mon, 25 May 2020 15:59:57 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 5AAB960D11; Mon, 25 May 2020 19:59:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1590436795; bh=5uPFqOaJshb/wWN27Pe6v8ivFx+zjEBHwTu4yPt9+nI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=mpMLrE17rtzG0jAC7EN7SnoWCkABIX/SYEIK5ErroJjoHaqMTeYxUMdtLfV/FQpB4 Cnck9U1kjA0fmZ60MWVSxilwUpSfWp7XQiwaWqBgyxPWIhSB85MjIGDwF9w4wSA20N DP1FnFQI/jKGDE4RYF/d5luFk4ht4vhRDdKOEGy68RIVASGs7tBf0m7NnJBXUt3+rn Fvvv8b38cN+K65lovHcrwBu8DVY778UgNXLrs6JjfHjalN/gtjEIixplS9FyyOVEDT G2fjXsyCyCSVWk6bw8oiKIdqpHQWVnu2DwBAWCx/HCXVwWjwKyZ2xm2f67YPzX0c1v AAwXxMtKnt54koHM0hOghJb2aqewRTgzFA59vWX4qDyFPXU0cO/7P5rS6E2AC6Rdwf p5CH959E4Z8zur/2mbDWGFSqYN+jLojej2KJm4bS2akxQhOWMlzBYz299BXT+uQm+f CT1SMGaPsNWmbDFvSDObI6UPaKAcqdaP2BkrVtNLyaoiZyIlh95 From: "brian m. carlson" To: Cc: =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v2 29/44] connect: pass full packet reader when parsing v2 refs Date: Mon, 25 May 2020 19:59:15 +0000 Message-Id: <20200525195930.309665-30-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.rc0.183.gde8f92d652 In-Reply-To: <20200525195930.309665-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200525195930.309665-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When we're parsing refs, we need to know not only what the line we're parsing is, but also the hash algorithm we should use to parse it, which is stored in the reader object. Pass the packet reader object through to the protocol v2 ref parsing function. Signed-off-by: brian m. carlson --- connect.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/connect.c b/connect.c index 915f1736a0..1d05bc56ed 100644 --- a/connect.c +++ b/connect.c @@ -374,7 +374,7 @@ struct ref **get_remote_heads(struct packet_reader *reader, } /* Returns 1 when a valid ref has been added to `list`, 0 otherwise */ -static int process_ref_v2(const char *line, struct ref ***list) +static int process_ref_v2(struct packet_reader *reader, struct ref ***list) { int ret = 1; int i = 0; @@ -382,6 +382,7 @@ static int process_ref_v2(const char *line, struct ref ***list) struct ref *ref; struct string_list line_sections = STRING_LIST_INIT_DUP; const char *end; + const char *line = reader->line; /* * Ref lines have a number of fields which are space deliminated. The @@ -469,7 +470,7 @@ struct ref **get_remote_refs(int fd_out, struct packet_reader *reader, /* Process response from server */ while (packet_reader_read(reader) == PACKET_READ_NORMAL) { - if (!process_ref_v2(reader->line, &list)) + if (!process_ref_v2(reader, &list)) die(_("invalid ls-refs response: %s"), reader->line); } From patchwork Mon May 25 19:59:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11569373 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7D198739 for ; Mon, 25 May 2020 20:00:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 651BC2075F for ; Mon, 25 May 2020 20:00:18 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="EEKNgyot" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390554AbgEYUAR (ORCPT ); Mon, 25 May 2020 16:00:17 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38722 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390436AbgEYT75 (ORCPT ); Mon, 25 May 2020 15:59:57 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id C263960D12; Mon, 25 May 2020 19:59:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1590436796; bh=xz69Nj1npcQo0xs5mic9Nu0mlG+XkphPAs5sMsgd0Jc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=EEKNgyotm5WVLyYfrS6iUmn+6ofpRgMEPBxVZZX46X2R4eCp9a/kSmAp41j+vx+yF BImpH72e0c4Ux9zxLxWIvPJukBi8aNTEsFMVnSuvnILMO9CQRzsbgEi6VvR5YH3Xa5 XMaqoaCR7uM28DHyQA7KZNX45O2HBcaNao5UfTdzQArwa0Wo9Bv0TF2NiNHpA+2WRb 3WIjz0xWmso6OBwJ4RGUO6WPgw+rBGFGmC1Nb9BvTNTanHSHKk+UoSMlt5Nn10WzPU /Fj/Mw0Y9OEeIf8ePxDvnttTvFiWyy+ameaNpRIIyr15D9O48+yCW7jQarGTHSfW4x nWWvKQX9TUGoZ5+0ZH9xI+JDGk+KgZGLsTLEzC/yf5Vki220angAT+4DKAp9X6yKSt g+8SERFcW40Dw1xvE45DTrQQMJLUobjdO1gWIvuxJqKEOP60uf15ZexjJ2GTjKNub4 JvfIUsak8m56zu6lxDDmvzS0vzIkI0dNjX7UIyIDpj6AQShUJdf From: "brian m. carlson" To: Cc: =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v2 30/44] connect: parse v2 refs with correct hash algorithm Date: Mon, 25 May 2020 19:59:16 +0000 Message-Id: <20200525195930.309665-31-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.rc0.183.gde8f92d652 In-Reply-To: <20200525195930.309665-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200525195930.309665-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When using protocol v2, we need to know what hash algorithm is used by the remote end. See if the server has sent us an object-format capability, and if so, use it to determine the hash algorithm in use and set that value in the packet reader. Parse the refs using this algorithm. Signed-off-by: brian m. carlson --- connect.c | 21 ++++++++++++++++----- 1 file changed, 16 insertions(+), 5 deletions(-) diff --git a/connect.c b/connect.c index 1d05bc56ed..66650ff2d3 100644 --- a/connect.c +++ b/connect.c @@ -283,7 +283,7 @@ static int process_ref(const struct packet_reader *reader, int len, die(_("protocol error: unexpected capabilities^{}")); } else if (check_ref(name, flags)) { struct ref *ref = alloc_ref(name); - memcpy(ref->old_oid.hash, old_oid.hash, reader->hash_algo->rawsz); + oidcpy(&ref->old_oid, &old_oid); **list = ref; *list = &ref->next; } @@ -395,7 +395,7 @@ static int process_ref_v2(struct packet_reader *reader, struct ref ***list) goto out; } - if (parse_oid_hex(line_sections.items[i++].string, &old_oid, &end) || + if (parse_oid_hex_algop(line_sections.items[i++].string, &old_oid, &end, reader->hash_algo) || *end) { ret = 0; goto out; @@ -403,7 +403,7 @@ static int process_ref_v2(struct packet_reader *reader, struct ref ***list) ref = alloc_ref(line_sections.items[i++].string); - oidcpy(&ref->old_oid, &old_oid); + memcpy(ref->old_oid.hash, old_oid.hash, reader->hash_algo->rawsz); **list = ref; *list = &ref->next; @@ -416,7 +416,8 @@ static int process_ref_v2(struct packet_reader *reader, struct ref ***list) struct object_id peeled_oid; char *peeled_name; struct ref *peeled; - if (parse_oid_hex(arg, &peeled_oid, &end) || *end) { + if (parse_oid_hex_algop(arg, &peeled_oid, &end, + reader->hash_algo) || *end) { ret = 0; goto out; } @@ -424,7 +425,8 @@ static int process_ref_v2(struct packet_reader *reader, struct ref ***list) peeled_name = xstrfmt("%s^{}", ref->name); peeled = alloc_ref(peeled_name); - oidcpy(&peeled->old_oid, &peeled_oid); + memcpy(peeled->old_oid.hash, peeled_oid.hash, + reader->hash_algo->rawsz); **list = peeled; *list = &peeled->next; @@ -443,6 +445,7 @@ struct ref **get_remote_refs(int fd_out, struct packet_reader *reader, const struct string_list *server_options) { int i; + const char *hash_name; *list = NULL; if (server_supports_v2("ls-refs", 1)) @@ -451,6 +454,14 @@ struct ref **get_remote_refs(int fd_out, struct packet_reader *reader, if (server_supports_v2("agent", 0)) packet_write_fmt(fd_out, "agent=%s", git_user_agent_sanitized()); + if (server_feature_v2("object-format", &hash_name)) { + int hash_algo = hash_algo_by_name(hash_name); + if (hash_algo == GIT_HASH_UNKNOWN) + die(_("unknown object format '%s' specified by server"), hash_name); + reader->hash_algo = &hash_algos[hash_algo]; + packet_write_fmt(fd_out, "object-format=%s", reader->hash_algo->name); + } + if (server_options && server_options->nr && server_supports_v2("server-option", 1)) for (i = 0; i < server_options->nr; i++) From patchwork Mon May 25 19:59:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11569403 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6C246912 for ; Mon, 25 May 2020 20:00:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4F4572075F for ; Mon, 25 May 2020 20:00:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="E1rHN6wG" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390661AbgEYUAj (ORCPT ); Mon, 25 May 2020 16:00:39 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38852 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390587AbgEYUA1 (ORCPT ); Mon, 25 May 2020 16:00:27 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 35EC360D13; Mon, 25 May 2020 19:59:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1590436796; bh=q3GU+ncEEcVJcUTylYh7psxfcHSiJXW8/1xfhvPzTD8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=E1rHN6wGGrnYn12O//LXfZgNNq5poX3ktJMq3mVr+zRrR+79HtgL2Y0EU2/xAND/0 H/Z3LAkbDwTYqBGZjW9HGawj5389Jym3cxD5C4WTh06Vgg0AWo9MSlkqI4pdzQ3RNK rHJYrHaFG1llUQ7mthSJ8yIv5PxzyStYq9HF3I9nGN78W/FlJWewcLjAk3I3e2Qlpe +hog1lOO1ptn/0Y55riiSFiBJhW25srEP2UJGuynpgH7gRD3kPbTXmgghDNnEyXVzq qUF2QiAsAS9ZpkvXIYmMtKHuwDmW+4BmO56jbr+/1h7aaosu638pXHWSupDDdY0y+S mkXz4EXQXBgFAIrdTBQ+rbZ49zorlydhAlYofo0ec2dGg38Crg5kLBDJuJMQvWxESF z3KpWkeCoLKe1cI+tnPXlKl644MX0cL2fp8WdoCJhVpQJQF27S8lSIVGU+vJAIjeCQ SoepYU9UUGjzRrckSJxyLPAjj3CCP7Zxv6oe1XbM0oMJhbid1+I From: "brian m. carlson" To: Cc: =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v2 31/44] serve: advertise object-format capability for protocol v2 Date: Mon, 25 May 2020 19:59:17 +0000 Message-Id: <20200525195930.309665-32-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.rc0.183.gde8f92d652 In-Reply-To: <20200525195930.309665-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200525195930.309665-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In order to communicate the protocol supported by the server side, add support for advertising the object-format capability. We check that the client side sends us an identical algorithm if it sends us its own object-format capability, and assume it speaks SHA-1 if not. In the test, when we're using an algorithm other than SHA-1, we need to specify the algorithm in use so we don't get a failure with an "unknown format" message. Add a test that we handle a mismatched algorithm. Remove the test_oid_init call since it's no longer necessary. Signed-off-by: brian m. carlson --- connect.c | 2 ++ serve.c | 27 +++++++++++++++++++++++++++ t/t5701-git-serve.sh | 25 +++++++++++++++++++++++++ 3 files changed, 54 insertions(+) diff --git a/connect.c b/connect.c index 66650ff2d3..2ada5b5161 100644 --- a/connect.c +++ b/connect.c @@ -460,6 +460,8 @@ struct ref **get_remote_refs(int fd_out, struct packet_reader *reader, die(_("unknown object format '%s' specified by server"), hash_name); reader->hash_algo = &hash_algos[hash_algo]; packet_write_fmt(fd_out, "object-format=%s", reader->hash_algo->name); + } else { + reader->hash_algo = &hash_algos[GIT_HASH_SHA1]; } if (server_options && server_options->nr && diff --git a/serve.c b/serve.c index 317256c1a4..7ab7807fef 100644 --- a/serve.c +++ b/serve.c @@ -22,6 +22,14 @@ static int agent_advertise(struct repository *r, return 1; } +static int object_format_advertise(struct repository *r, + struct strbuf *value) +{ + if (value) + strbuf_addstr(value, r->hash_algo->name); + return 1; +} + struct protocol_capability { /* * The name of the capability. The server uses this name when @@ -57,6 +65,7 @@ static struct protocol_capability capabilities[] = { { "ls-refs", always_advertise, ls_refs }, { "fetch", upload_pack_advertise, upload_pack_v2 }, { "server-option", always_advertise, NULL }, + { "object-format", object_format_advertise, NULL }, }; static void advertise_capabilities(void) @@ -153,6 +162,22 @@ int has_capability(const struct argv_array *keys, const char *capability, return 0; } +static void check_algorithm(struct repository *r, struct argv_array *keys) +{ + int client = GIT_HASH_SHA1, server = hash_algo_by_ptr(r->hash_algo); + const char *algo_name; + + if (has_capability(keys, "object-format", &algo_name)) { + client = hash_algo_by_name(algo_name); + if (client == GIT_HASH_UNKNOWN) + die("unknown object format '%s'", algo_name); + } + + if (client != server) + die("mismatched object format: server %s; client %s\n", + r->hash_algo->name, hash_algos[client].name); +} + enum request_state { PROCESS_REQUEST_KEYS, PROCESS_REQUEST_DONE, @@ -223,6 +248,8 @@ static int process_request(void) if (!command) die("no command requested"); + check_algorithm(the_repository, &keys); + command->command(the_repository, &keys, &reader); argv_array_clear(&keys); diff --git a/t/t5701-git-serve.sh b/t/t5701-git-serve.sh index ffb9613885..a1f5fdc9fd 100755 --- a/t/t5701-git-serve.sh +++ b/t/t5701-git-serve.sh @@ -5,12 +5,17 @@ test_description='test protocol v2 server commands' . ./test-lib.sh test_expect_success 'test capability advertisement' ' + test_oid_cache <<-EOF && + wrong_algo sha1:sha256 + wrong_algo sha256:sha1 + EOF cat >expect <<-EOF && version 2 agent=git/$(git version | cut -d" " -f3) ls-refs fetch=shallow server-option + object-format=$(test_oid algo) 0000 EOF @@ -45,6 +50,7 @@ test_expect_success 'request invalid capability' ' test_expect_success 'request with no command' ' test-tool pkt-line pack >in <<-EOF && agent=git/test + object-format=$(test_oid algo) 0000 EOF test_must_fail test-tool serve-v2 --stateless-rpc 2>err in <<-EOF && command=foo + object-format=$(test_oid algo) agent=git/test 0000 EOF @@ -61,6 +68,17 @@ test_expect_success 'request invalid command' ' test_i18ngrep "invalid command" err ' +test_expect_success 'wrong object-format' ' + test-tool pkt-line pack >in <<-EOF && + command=fetch + agent=git/test + object-format=$(test_oid wrong_algo) + 0000 + EOF + test_must_fail test-tool serve-v2 --stateless-rpc 2>err in <<-EOF && command=ls-refs + object-format=$(test_oid algo) 0000 EOF @@ -96,6 +115,7 @@ test_expect_success 'basics of ls-refs' ' test_expect_success 'basic ref-prefixes' ' test-tool pkt-line pack >in <<-EOF && command=ls-refs + object-format=$(test_oid algo) 0001 ref-prefix refs/heads/master ref-prefix refs/tags/one @@ -116,6 +136,7 @@ test_expect_success 'basic ref-prefixes' ' test_expect_success 'refs/heads prefix' ' test-tool pkt-line pack >in <<-EOF && command=ls-refs + object-format=$(test_oid algo) 0001 ref-prefix refs/heads/ 0000 @@ -136,6 +157,7 @@ test_expect_success 'refs/heads prefix' ' test_expect_success 'peel parameter' ' test-tool pkt-line pack >in <<-EOF && command=ls-refs + object-format=$(test_oid algo) 0001 peel ref-prefix refs/tags/ @@ -157,6 +179,7 @@ test_expect_success 'peel parameter' ' test_expect_success 'symrefs parameter' ' test-tool pkt-line pack >in <<-EOF && command=ls-refs + object-format=$(test_oid algo) 0001 symrefs ref-prefix refs/heads/ @@ -178,6 +201,7 @@ test_expect_success 'symrefs parameter' ' test_expect_success 'sending server-options' ' test-tool pkt-line pack >in <<-EOF && command=ls-refs + object-format=$(test_oid algo) server-option=hello server-option=world 0001 @@ -200,6 +224,7 @@ test_expect_success 'unexpected lines are not allowed in fetch request' ' test-tool pkt-line pack >in <<-EOF && command=fetch + object-format=$(test_oid algo) 0001 this-is-not-a-command 0000 From patchwork Mon May 25 19:59:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11569353 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 84112739 for ; Mon, 25 May 2020 20:00:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6B89F20776 for ; Mon, 25 May 2020 20:00:04 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="Pal+HS1V" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390469AbgEYUAD (ORCPT ); Mon, 25 May 2020 16:00:03 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38720 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390438AbgEYT76 (ORCPT ); Mon, 25 May 2020 15:59:58 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 9D56A60D14; Mon, 25 May 2020 19:59:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1590436796; bh=Mo6wZuC9p3nChbBxsGl44GyZeJquT3ZYpX4+z7OuAXU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=Pal+HS1VZGU1EhADht+iCnJ+7u16QcTNQjtspEGQoRRNJ99ZgspWE+NDSG/pziCff j8W5wQDYSHK18YP6W5kj58IzuMZz1GSUQ2kCY5miOh82wrmF3cGIpUiwCcZOkueOad AJt5yEny2VDCM6HuYXtoh5q3EQFpcWUQEJ6m7ZoQTphbqNcFrAjfLnHk1BGiCM44T9 1QBy4v6HMtrIJeFylMOzmC4Fe6sBTwzUBy5j5IbvXrAV/AuVnwtfUJ9Pn/9SJJkq+p x7/2Z1dMxBTHUv2IRc9Lh8s/kc2gkEMW1Pt2atZMflHeltY/aUBuTTZnQdlUQEkpvd GrpebolJiLWoRbEj5UU34cvZILgxvbE8ZF4hENgDFUc5Qc0QqNQwhuaf9Yzvn21m4k B6IUwdSg6hgX1wZu4MhOvse876lZL/rzkykwTpgeAsfx1YYUC5D603PiRoKK8CfKqF zOt8okS2+Ki0wu4vQQMOkSU9IvuBUQXQuDYLFo8ZAmwbkRASR+l From: "brian m. carlson" To: Cc: =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v2 32/44] t5500: make hash independent Date: Mon, 25 May 2020 19:59:18 +0000 Message-Id: <20200525195930.309665-33-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.rc0.183.gde8f92d652 In-Reply-To: <20200525195930.309665-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200525195930.309665-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org This test has hard-coded pkt-lines with object IDs. The pkt-line lengths necessarily differ between hash algorithms, so generate these lines with the packetize helper so they're always the right size. In addition, we will require an object-format capability for SHA-256, so pass that capability on to the upload-pack process. Signed-off-by: brian m. carlson --- t/t5500-fetch-pack.sh | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/t/t5500-fetch-pack.sh b/t/t5500-fetch-pack.sh index 8c54e34ef1..dfed113247 100755 --- a/t/t5500-fetch-pack.sh +++ b/t/t5500-fetch-pack.sh @@ -871,9 +871,10 @@ test_expect_success 'shallow since with commit graph and already-seen commit' ' GIT_PROTOCOL=version=2 git upload-pack . <<-EOF >/dev/null 0012command=fetch + $(echo "object-format=$(test_oid algo)" | packetize) 00010013deepen-since 1 - 0032want $(git rev-parse other) - 0032have $(git rev-parse master) + $(echo "want $(git rev-parse other)" | packetize) + $(echo "have $(git rev-parse master)" | packetize) 0000 EOF ) From patchwork Mon May 25 19:59:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11569371 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 92619912 for ; Mon, 25 May 2020 20:00:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 714042075F for ; Mon, 25 May 2020 20:00:17 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="htUN3E69" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390528AbgEYUAP (ORCPT ); Mon, 25 May 2020 16:00:15 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38718 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390441AbgEYT76 (ORCPT ); Mon, 25 May 2020 15:59:58 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 1174D60D15; Mon, 25 May 2020 19:59:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1590436797; bh=MWvvjYEo5Rp8KtFH0luFagYjxGUaEerSNE9nw0RTEhw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=htUN3E69PhnTVOd7HW/PuRFXEIbvGVjYC/fI4eA9RGuKRBa42ykRR1RGWjnIySj9q OYeYhMPfJehCL+RaA+pYbsisldRwrdld8RkswhqLSr0CabtV5mUVWUBnjFbPvlmjWq Gu0oO/sGtVnwOfueWbIXs6U0jty//FdrcKBLd7la8aCzYV4CJU3gLcIy6acHiz82Mz LOPa2RBRzYPD9SINfNQ8OjcmiTj845e0vn6pQrAs8aVnzHnl8OyB1HnL+wCul/Ckyh uAeM9fqpAibYv6ywIzEsNrZdmwjFJrmkWlYz0Wg1k8DiCDsGoV4r/Bc83oDKscgiXd VGvgyGBXYOLIggxJu7ZL0s4+A0pIVr/ScmDIFa8bn7WvWLPt7rwmd2FfOmtOlofwF3 afTwvGclDXwr+G2Abx5VfrKUH7OosV0uRRl8HrNgw9kuezjde/f27Jj7QF7panJAgO hiCya8dOKGAvXbLGSlAk/SGTEKCcsPsBeID/jYfLOQm6nLHoBmK From: "brian m. carlson" To: Cc: =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v2 33/44] builtin/ls-remote: initialize repository based on fetch Date: Mon, 25 May 2020 19:59:19 +0000 Message-Id: <20200525195930.309665-34-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.rc0.183.gde8f92d652 In-Reply-To: <20200525195930.309665-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200525195930.309665-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org ls-remote may or may not operate within a repository, and as such will not have been initialized with the repository's hash algorithm. Even if it were, the remote side could be using a different algorithm and we would still want to display those refs properly. Find the hash algorithm used by the remote side by querying the transport object and set our hash algorithm accordingly. Without this change, if the remote side is using SHA-256, we truncate the refs to 40 hex characters, since that's the length of the default hash algorithm (SHA-1). Note that technically this is not a correct setting of the repository hash algorithm since, if we are in a repository, it might be one of a different hash algorithm from the remote side. However, our current code paths don't handle multiple algorithms and won't for some time, so this is the best we can do. We rely on the fact that ls-remote never modifies the current repository, which is a reasonable assumption to make. Signed-off-by: brian m. carlson --- builtin/ls-remote.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/builtin/ls-remote.c b/builtin/ls-remote.c index 6ef519514b..3a4dd12903 100644 --- a/builtin/ls-remote.c +++ b/builtin/ls-remote.c @@ -118,6 +118,10 @@ int cmd_ls_remote(int argc, const char **argv, const char *prefix) transport->server_options = &server_options; ref = transport_get_remote_refs(transport, &ref_prefixes); + if (ref) { + int hash_algo = hash_algo_by_ptr(transport_get_hash_algo(transport)); + repo_set_hash_algo(the_repository, hash_algo); + } if (transport_disconnect(transport)) { UNLEAK(sorting); return 1; From patchwork Mon May 25 19:59:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11569355 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4D6F8912 for ; Mon, 25 May 2020 20:00:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 359572075F for ; Mon, 25 May 2020 20:00:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="JOIO/+uB" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390473AbgEYUAE (ORCPT ); Mon, 25 May 2020 16:00:04 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38716 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390445AbgEYT76 (ORCPT ); Mon, 25 May 2020 15:59:58 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 78D0D60D16; Mon, 25 May 2020 19:59:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1590436797; bh=BrGE5IRK4iD22ojUaw6vyETbInj2RPhBRq6D/5gJkk0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=JOIO/+uBSwvVusO7yiGWK8pJQsl7TqzOuQQpkhyXUjJ4EyhSkznN5QSG9icbY17Ml v03v8uogRa7HS4DSSEA+jjQ5l61TPtnwTvSc+TCu3K1AJccbUseb6bucKf/TV+ohIi /4Cx7fFz+8Ajyrr8QtvCX4i9RNUnTzWTrFfi8VsK/C7Qtn/LyGdIB2uAg1N7Pi8NMI wICeBykPe1j5mdM9urhPB2IZDipyzZb6esrI6JcR1O7UGQCRaZ7If+kXi/nTi6hCyz kfVGzBI0fPC99Wh9vT8/t0YWHugkpzhmvxVXrTAwiLe2OtuukQMzEeEc/Pp2pQA0Iy ImOePx9h8ud6h/RvM4RhSF5hj1KeZYiJz8ziB+gpdZQiDoOdN+zI0dUspUYrMnX722 ltSTaebN3N6Fs4+OzySjHgBnMIJ218IvEtuhyZLV2+iK+eALtA8SsnaqEDooyNbxmy rogVSQSk78KDXc2jeOyVe7f5fpZgmmmRB1VrNNTJky1pFPQ38Th From: "brian m. carlson" To: Cc: =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v2 34/44] remote-curl: detect algorithm for dumb HTTP by size Date: Mon, 25 May 2020 19:59:20 +0000 Message-Id: <20200525195930.309665-35-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.rc0.183.gde8f92d652 In-Reply-To: <20200525195930.309665-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200525195930.309665-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When reading the info/refs file for a repository, we have no explicit way to detect which hash algorithm is in use because the file doesn't provide one. Detect the hash algorithm in use by the size of the first object ID. We anonymize the URL like elsewhere in the function in case the user has decided to include a secret in the URL. Signed-off-by: brian m. carlson --- remote-curl.c | 23 +++++++++++++++++++++-- 1 file changed, 21 insertions(+), 2 deletions(-) diff --git a/remote-curl.c b/remote-curl.c index 3ed0dfec1b..617edfedf4 100644 --- a/remote-curl.c +++ b/remote-curl.c @@ -252,6 +252,19 @@ static struct ref *parse_git_refs(struct discovery *heads, int for_push) return list; } +static const struct git_hash_algo *detect_hash_algo(struct discovery *heads) +{ + const char *p = memchr(heads->buf, '\t', heads->len); + int algo; + if (!p) + return NULL; + + algo = hash_algo_by_length((p - heads->buf) / 2); + if (algo == GIT_HASH_UNKNOWN) + return NULL; + return &hash_algos[algo]; +} + static struct ref *parse_info_refs(struct discovery *heads) { char *data, *start, *mid; @@ -262,6 +275,12 @@ static struct ref *parse_info_refs(struct discovery *heads) struct ref *ref = NULL; struct ref *last_ref = NULL; + options.hash_algo = detect_hash_algo(heads); + if (!options.hash_algo) + die("%sinfo/refs not valid: could not determine hash algorithm; " + "is this a git repository?", + transport_anonymize_url(url.buf)); + data = heads->buf; start = NULL; mid = data; @@ -272,13 +291,13 @@ static struct ref *parse_info_refs(struct discovery *heads) if (data[i] == '\t') mid = &data[i]; if (data[i] == '\n') { - if (mid - start != the_hash_algo->hexsz) + if (mid - start != options.hash_algo->hexsz) die(_("%sinfo/refs not valid: is this a git repository?"), transport_anonymize_url(url.buf)); data[i] = 0; ref_name = mid + 1; ref = alloc_ref(ref_name); - get_oid_hex(start, &ref->old_oid); + get_oid_hex_algop(start, &ref->old_oid, options.hash_algo); if (!refs) refs = ref; if (last_ref) From patchwork Mon May 25 19:59:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11569357 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2C9DC912 for ; Mon, 25 May 2020 20:00:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 15D8E2075F for ; Mon, 25 May 2020 20:00:07 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="DObXe0jb" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390479AbgEYUAF (ORCPT ); Mon, 25 May 2020 16:00:05 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38728 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390446AbgEYT77 (ORCPT ); Mon, 25 May 2020 15:59:59 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id DF99960D17; Mon, 25 May 2020 19:59:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1590436798; bh=wSBvyjey21gbVX/7xAGCthcffF4q4jyfM63IKoHZXnw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=DObXe0jbWyKrbFQkyydgII4x9GcinnMkjw/eQ5UeXiTiwohHQgdDdmLAVlPkzNiNB fUKdvP2uw6/7glVWHcpt2mF0ulCvPd7YQjbEkosjD+op7gigA0Cac20pWgeWvZtkRR DgTn/8zXPDBISfv1diq3Lm1a27NYnSIFusGk6pTr7hSjisbtCugy909lftqXiLWo/8 7IWqhsIV877/WbWguXgTQlzcKfzV9gz6DjMY5f5LjNcuxCiWTlXXwneSbIWrgjVGNg H7QUBq0Mu2xqPi78A+hgmgp0rEaFSvTyMSGGEHZ+ctFOFyRYS3zpm0TIcVhzxPTHyk w+cIuKX+tTcnWiPzybf0qYw9/KJsAx1Nrc7Ti0js0EsPSPtxzeV2H2TI5oM8Cxp7Sa VbOY1U4qheuGHNVh09rnYb9LplF71kHh0oVRVVquDHp8IkB1vY3pT+wsjGV9INbHM4 g8a6HFe/mZlqrQv9Dsmov3VNUvrYb5+65XPaw6JGXfat50fntx4 From: "brian m. carlson" To: Cc: =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v2 35/44] builtin/index-pack: add option to specify hash algorithm Date: Mon, 25 May 2020 19:59:21 +0000 Message-Id: <20200525195930.309665-36-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.rc0.183.gde8f92d652 In-Reply-To: <20200525195930.309665-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200525195930.309665-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org git index-pack is usually run in a repository, but need not be. Since packs don't contains information on the algorithm in use, instead relying on context, add an option to index-pack to tell it which one we're using in case someone runs it outside of a repository. Since using --stdin necessarily implies a repository, don't allow specifying an object format if it's provided to prevent users from passing an option that won't work. Add documentation for this option. Signed-off-by: brian m. carlson --- Documentation/git-index-pack.txt | 8 ++++++++ builtin/index-pack.c | 8 ++++++++ 2 files changed, 16 insertions(+) diff --git a/Documentation/git-index-pack.txt b/Documentation/git-index-pack.txt index d5b7560bfe..9316d9a80b 100644 --- a/Documentation/git-index-pack.txt +++ b/Documentation/git-index-pack.txt @@ -93,6 +93,14 @@ OPTIONS --max-input-size=:: Die, if the pack is larger than . +--object-format=:: + Specify the given object format (hash algorithm) for the pack. The valid + values are 'sha1' and (if enabled) 'sha256'. The default is the algorithm for + the current repository (set by `extensions.objectFormat`), or 'sha1' if no + value is set or outside a repository. ++ +This option cannot be used with --stdin. + NOTES ----- diff --git a/builtin/index-pack.c b/builtin/index-pack.c index 7bea1fba52..f865666db9 100644 --- a/builtin/index-pack.c +++ b/builtin/index-pack.c @@ -1667,6 +1667,7 @@ int cmd_index_pack(int argc, const char **argv, const char *prefix) unsigned char pack_hash[GIT_MAX_RAWSZ]; unsigned foreign_nr = 1; /* zero is a "good" value, assume bad */ int report_end_of_input = 0; + int hash_algo = 0; /* * index-pack never needs to fetch missing objects except when @@ -1760,6 +1761,11 @@ int cmd_index_pack(int argc, const char **argv, const char *prefix) die(_("bad %s"), arg); } else if (skip_prefix(arg, "--max-input-size=", &arg)) { max_input_size = strtoumax(arg, NULL, 10); + } else if (skip_prefix(arg, "--object-format=", &arg)) { + hash_algo = hash_algo_by_name(arg); + if (hash_algo == GIT_HASH_UNKNOWN) + die(_("unknown hash algorithm '%s'"), arg); + repo_set_hash_algo(the_repository, hash_algo); } else usage(index_pack_usage); continue; @@ -1776,6 +1782,8 @@ int cmd_index_pack(int argc, const char **argv, const char *prefix) die(_("--fix-thin cannot be used without --stdin")); if (from_stdin && !startup_info->have_repository) die(_("--stdin requires a git repository")); + if (from_stdin && hash_algo) + die(_("--object-format cannot be used with --stdin")); if (!index_name && pack_name) index_name = derive_filename(pack_name, "idx", &index_name_buf); From patchwork Mon May 25 19:59:22 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11569369 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 60D78912 for ; Mon, 25 May 2020 20:00:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 45D322075F for ; Mon, 25 May 2020 20:00:15 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="sNdF6jlq" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390519AbgEYUAO (ORCPT ); Mon, 25 May 2020 16:00:14 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38722 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390451AbgEYT77 (ORCPT ); Mon, 25 May 2020 15:59:59 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 5245260D18; Mon, 25 May 2020 19:59:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1590436798; bh=QhWwx5JrFjkmi9fYf3BLeqi+AU9uqTKM8zuKL+dV4ZA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=sNdF6jlq2fH/+QuMoXHCxYKP4LbD/L9bd8DKG2xQOkJwMnM2DCDg7lDpq++JDxr1j KrE3uHfo3nxcJoijZhovKEHSYbq/l0JikTqVqcnoY6+gryQpjry8FWbswkK6qX14mA gyIvf4wigkpKQjTJDApU3DCyIHn3OlGcEXFGpYjWnt7Arc+b1cRDhcs92kOjohQ/4X +wUDgXivOwdwREqooyHVNjc1VIeSVNsbJOyVzxH3+qFogQhDpmEJ2I8hKsDbMT8mZM 8XPZ6Vp2yonxXB5SW4X4k8EuI4AWo2XTneIO9LzTrx/P9Ly4yHthnDcUA6cpDji1NB 1Al+Nmnlbe3BPgwdnISePOq9fXoec4vKpRwgwyL9t8xJXadqCvtZxtYzMnX4SMpCpO Y1yZOu27ZL9ta4s0L+7DlbN0bpmXYNkQ1K0Co2x8XEtE/WaSnOXfy8G9ByGSGpRXrK QKE4efkXuzgfkjN+mLVYosi2Yy/vsY65j7LvAAZJGU/5h/I8NXu From: "brian m. carlson" To: Cc: =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v2 36/44] t1050: pass algorithm to index-pack when outside repo Date: Mon, 25 May 2020 19:59:22 +0000 Message-Id: <20200525195930.309665-37-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.rc0.183.gde8f92d652 In-Reply-To: <20200525195930.309665-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200525195930.309665-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When outside a repository, git index-pack is unable to guess the hash algorithm in use for a pack, since packs don't contain any information on the algorithm in use. Pass an option to index-pack to help it out in this test. Signed-off-by: brian m. carlson --- t/t1050-large.sh | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/t/t1050-large.sh b/t/t1050-large.sh index 7f88ea07c2..6a56d1ca24 100755 --- a/t/t1050-large.sh +++ b/t/t1050-large.sh @@ -12,6 +12,7 @@ file_size () { } test_expect_success setup ' + test_oid_init && # clone does not allow us to pass core.bigfilethreshold to # new repos, so set core.bigfilethreshold globally git config --global core.bigfilethreshold 200k && @@ -177,7 +178,8 @@ test_expect_success 'git-show a large file' ' test_expect_success 'index-pack' ' git clone file://"$(pwd)"/.git foo && - GIT_DIR=non-existent git index-pack --strict --verify foo/.git/objects/pack/*.pack + GIT_DIR=non-existent git index-pack --object-format=$(test_oid algo) \ + --strict --verify foo/.git/objects/pack/*.pack ' test_expect_success 'repack' ' From patchwork Mon May 25 19:59:23 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11569367 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5CCC21392 for ; Mon, 25 May 2020 20:00:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4537B20776 for ; Mon, 25 May 2020 20:00:13 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="MQNTaNqH" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390505AbgEYUAM (ORCPT ); Mon, 25 May 2020 16:00:12 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38726 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390453AbgEYT77 (ORCPT ); Mon, 25 May 2020 15:59:59 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id B976C60D19; Mon, 25 May 2020 19:59:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1590436799; bh=wGkcIWlrzxwpvybjFe3BIl3Q7hJ8QqVrySX1zMLhLPs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=MQNTaNqHy+R4H7FLQuU5HC6oLYPoaCZFM3XHDO4Mj8fCLOC9Cm1MFxgttr2i+FKst VdeGiNIajYIUn1ZhybaAtUpxkWHDTlKmQuI5uKgTb2D/UGc7pQrqUlb+twTEdvHPmm gz4vn5DF5cObAXjF1GlWAFDahfDK15nKXA9qA6gzt6olhzFPtCRrsGLA2VRQXGrcC1 NLhlRZ7YjTko2LM07I3t/HcZiVF/z7wmg4voKXWbaJsw3oKRAr54qrpQRyBwzRSiBK A/anQ3UYBEOXRe+W/MrQg/OYBtZCxchNkePWjdzLuQjXkSFa2O1SJsP3/6uC1wjHkb BGinibg/vtoXI3qfnbWKZcooDY/WGXRR9lstNkBnNYtBabAyMr4Icd+USRHlnlr4tp 7am20RwnpA6QEH3tVFtlYpVlxPTSFvnDeTID2+fFf9TXRdR+WS4Omc1QCL12A7vKdy YWRyqhW/ZPUovX2aynfLYBmqY2BL/vPd4UiGOUfY1Ima/h+sy/F From: "brian m. carlson" To: Cc: =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v2 37/44] remote-curl: avoid truncating refs with ls-remote Date: Mon, 25 May 2020 19:59:23 +0000 Message-Id: <20200525195930.309665-38-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.rc0.183.gde8f92d652 In-Reply-To: <20200525195930.309665-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200525195930.309665-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Normally, the remote-curl transport helper is aware of the hash algorithm we're using because we're in a repo with the appropriate hash algorithm set. However, when using git ls-remote outside of a repository, we won't have initialized the hash algorithm properly, so use hash_to_hex_algop to print the ref corresponding to the algorithm we've detected. Signed-off-by: brian m. carlson --- remote-curl.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/remote-curl.c b/remote-curl.c index 617edfedf4..c077323008 100644 --- a/remote-curl.c +++ b/remote-curl.c @@ -548,7 +548,9 @@ static void output_refs(struct ref *refs) if (posn->symref) printf("@%s %s\n", posn->symref, posn->name); else - printf("%s %s\n", oid_to_hex(&posn->old_oid), posn->name); + printf("%s %s\n", hash_to_hex_algop(posn->old_oid.hash, + options.hash_algo), + posn->name); } printf("\n"); fflush(stdout); From patchwork Mon May 25 19:59:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11569365 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F3BE3739 for ; Mon, 25 May 2020 20:00:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D29C32075F for ; Mon, 25 May 2020 20:00:12 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="tXzUA+oA" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390499AbgEYUAL (ORCPT ); Mon, 25 May 2020 16:00:11 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38718 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390457AbgEYUAA (ORCPT ); Mon, 25 May 2020 16:00:00 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 2BEC460D1A; Mon, 25 May 2020 19:59:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1590436799; bh=2AW4Ophh+GDqfkkvEpKmw/NRFQar3284kGJCZ5OkO5M=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=tXzUA+oAWCN0wFWm0XkT7uNATTBZF0Ur5JLxIwX4GWrwxiwvH7N1LxMD7evJuztKN /A0k7nt3sH12fmxtSyjmJDH1ugQOwaM5dNLpON9+6Kz4tf2SGkPTnUd2zZOKGmBY/q mI0K6PfgsUoRkpFZ1ldLnPQqvvMYBxYQXU03GZy+xWDL6OWMBotcjuOnH192CJzSnr tcCWaz1TowAO+/0VGAwl9F95Q/U5G0d9x01OKzCDgRp1coW3eSAnKd3q7ne74WdH6K CEcJFuYXvn3BZAwRW7Zic1fnelzNZoUUzEJiwxX18YOVFVGderKR6p5UYVQ5Pflegx ndRw9+NhI9EM06EJ24FL/XlrqMRZAmfFEVyliUK7MDZljOP+HifOiBLU9TwSwOzVt+ 7x3vNrT79Emmfnon9vn+y9UdazlBZU4VGp8purhGpZ+9dV0P6+56ML3JgfmqvSBLtV 4JBdmPfbfadkowGylfCxAurzgguRVCfFUZlCgsDghN+WeAZBdq0 From: "brian m. carlson" To: Cc: =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v2 38/44] t/helper: initialize the repository for test-sha1-array Date: Mon, 25 May 2020 19:59:24 +0000 Message-Id: <20200525195930.309665-39-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.rc0.183.gde8f92d652 In-Reply-To: <20200525195930.309665-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200525195930.309665-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org test-sha1-array uses the_hash_algo under the hood. Since t0064 wants to use the value that is correct for the hash algorithm that we're testing, make sure the test helper initializes the repository to set the_hash_algo correctly. Signed-off-by: brian m. carlson --- t/helper/test-oid-array.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/t/helper/test-oid-array.c b/t/helper/test-oid-array.c index ce9fd5f091..b16cd0b11b 100644 --- a/t/helper/test-oid-array.c +++ b/t/helper/test-oid-array.c @@ -12,6 +12,9 @@ int cmd__oid_array(int argc, const char **argv) { struct oid_array array = OID_ARRAY_INIT; struct strbuf line = STRBUF_INIT; + int nongit_ok; + + setup_git_directory_gently(&nongit_ok); while (strbuf_getline(&line, stdin) != EOF) { const char *arg; From patchwork Mon May 25 19:59:25 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11569361 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2D64B912 for ; Mon, 25 May 2020 20:00:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 16F502075F for ; Mon, 25 May 2020 20:00:09 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="MmuhxG1p" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390487AbgEYUAI (ORCPT ); Mon, 25 May 2020 16:00:08 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38716 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390460AbgEYUAA (ORCPT ); Mon, 25 May 2020 16:00:00 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id A4DA960D1C; Mon, 25 May 2020 19:59:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1590436800; bh=74qW95Zvoq9O2u0va6v/nSh8paXnLt0Me5nQURj92D4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=MmuhxG1pQWW/NeS1jxfCvrVw3EVkgl6WIJrjp2twv3Gn7Lu2xEM+jzdDnggngrPmP MS1ID5sUMjvaCn3lv9oTNA0px6JS94y/1GcfybG+8ZMaY0tfd/vNepElEeCzyZV75d nCYiat+ZFffpX7Al1/3nK4DpIyMWImeq8jsOoSwRomlcUdF+ud4cpR0B7k50UR7Znx mqmLHGpXyRoDBulWlU9XY0UhVtuexm3KWb+4vw0D15N/2jZbk2C+qSINPalTNMDY7i emhBynLUVzLMa93UtlP0ZQupsl+LtaZU1UqFP97vmiClLVQhdPRhJYTD9cQe/Pvw/Z 9V7jzNdkwrHa4RWwQ3UjJsRgoXmInEsmT6gN9Sc0gkTMHIWegPR4M9yKfuJSdz4lcn H5RLI+2FG72G1F5nhLq4A2xuwt733P/fnF3tX4a/Zbgmf1FPG7sbY6Rdeb+jamWxlF nmfg7VTE+a9zbCibGagPmDYqKdWOlgO79ujN1COSTKxoDYL0b/5 From: "brian m. carlson" To: Cc: =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v2 39/44] t5702: offer an object-format capability in the test Date: Mon, 25 May 2020 19:59:25 +0000 Message-Id: <20200525195930.309665-40-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.rc0.183.gde8f92d652 In-Reply-To: <20200525195930.309665-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200525195930.309665-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In order to make this test work with SHA-256, offer an object-format capability so that both sides use the same algorithm. Signed-off-by: brian m. carlson --- t/t5702-protocol-v2.sh | 2 ++ 1 file changed, 2 insertions(+) diff --git a/t/t5702-protocol-v2.sh b/t/t5702-protocol-v2.sh index 5039e66dc4..116358b9ac 100755 --- a/t/t5702-protocol-v2.sh +++ b/t/t5702-protocol-v2.sh @@ -13,6 +13,7 @@ start_git_daemon --export-all --enable=receive-pack daemon_parent=$GIT_DAEMON_DOCUMENT_ROOT_PATH/parent test_expect_success 'create repo to be served by git-daemon' ' + test_oid_init && git init "$daemon_parent" && test_commit -C "$daemon_parent" one ' @@ -394,6 +395,7 @@ test_expect_success 'even with handcrafted request, filter does not work if not # Custom request that tries to filter even though it is not advertised. test-tool pkt-line pack >in <<-EOF && command=fetch + object-format=$(test_oid algo) 0001 want $(git -C server rev-parse master) filter blob:none From patchwork Mon May 25 19:59:26 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11569397 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1E4E4739 for ; Mon, 25 May 2020 20:00:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 06F4D2075F for ; Mon, 25 May 2020 20:00:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="WhDo629x" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390645AbgEYUAe (ORCPT ); Mon, 25 May 2020 16:00:34 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38854 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390608AbgEYUAb (ORCPT ); Mon, 25 May 2020 16:00:31 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 3064760D1D; Mon, 25 May 2020 20:00:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1590436800; bh=/DV+zliFMgvjyMrzpHbzlFIwrj3WnjgN0b/139NAErw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=WhDo629xpwNUZHR6JatbFTvG0Lz2HOs5ixBkoezlFeLZi27c3SxrqmNwqE19QmMqJ ckBXh1osid4Paa9wBSQaDfpMhyhmVDL7RPNVR017V26ykOEqRGYtVorR6aCrxjc43X hzw+XE5XVNkWSRUUbpmdaaQMsQGaUpuD2Bn/OXqMfZx74aaHXTtQHYzWNW8/4Sn2JW AGMLxMEYTYjPqMTxBANzAQvVF+TOPLZMwrZHUizRzYvdSmdLob1YU77hxho3umlZvo 3u/XTtmWnNvEfLRKIddtSJYei9h5VNjQegY71TLAOA2QlefmU7RLRlE5AbJruRWdWh /RvYHyKWmuq3WWRXLXQcyIuaECtz3KAqGOzGoA38d0rBFVfWWQD4gKZQOlAHMOyXZG GVaBkLRRqgQgYPvKsB5+iGPAwH2a3pVrgQ5Y+K3qKPhn29TMR5pgM9rGl+dMB8XQ5R pXfLOkv4yZStpQ7KYGAgM07DgON1ORZnCKbsF+hgWoOLNhd9yk7 From: "brian m. carlson" To: Cc: =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v2 40/44] t5703: use object-format serve option Date: Mon, 25 May 2020 19:59:26 +0000 Message-Id: <20200525195930.309665-41-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.rc0.183.gde8f92d652 In-Reply-To: <20200525195930.309665-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200525195930.309665-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When we're using an algorithm other than SHA-1, we need to specify the algorithm in use so we don't get a failure with an "unknown format" message. Add a wrapper function that specifies this header if required. Skip specifying this header for SHA-1 to test that it works both with an without this header. Signed-off-by: brian m. carlson --- t/t5703-upload-pack-ref-in-want.sh | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/t/t5703-upload-pack-ref-in-want.sh b/t/t5703-upload-pack-ref-in-want.sh index a34460f7d8..afe7f7f919 100755 --- a/t/t5703-upload-pack-ref-in-want.sh +++ b/t/t5703-upload-pack-ref-in-want.sh @@ -27,6 +27,15 @@ check_output () { test_cmp sorted_commits actual_commits } +write_command () { + echo "command=$1" + + if test "$(test_oid algo)" != sha1 + then + echo "object-format=$(test_oid algo)" + fi +} + # c(o/foo) d(o/bar) # \ / # b e(baz) f(master) @@ -62,7 +71,7 @@ test_expect_success 'config controls ref-in-want advertisement' ' test_expect_success 'invalid want-ref line' ' test-tool pkt-line pack >in <<-EOF && - command=fetch + $(write_command fetch) 0001 no-progress want-ref refs/heads/non-existent @@ -83,7 +92,7 @@ test_expect_success 'basic want-ref' ' oid=$(git rev-parse a) && test-tool pkt-line pack >in <<-EOF && - command=fetch + $(write_command fetch) 0001 no-progress want-ref refs/heads/master @@ -107,7 +116,7 @@ test_expect_success 'multiple want-ref lines' ' oid=$(git rev-parse b) && test-tool pkt-line pack >in <<-EOF && - command=fetch + $(write_command fetch) 0001 no-progress want-ref refs/heads/o/foo @@ -129,7 +138,7 @@ test_expect_success 'mix want and want-ref' ' git rev-parse e f >expected_commits && test-tool pkt-line pack >in <<-EOF && - command=fetch + $(write_command fetch) 0001 no-progress want-ref refs/heads/master @@ -152,7 +161,7 @@ test_expect_success 'want-ref with ref we already have commit for' ' oid=$(git rev-parse c) && test-tool pkt-line pack >in <<-EOF && - command=fetch + $(write_command fetch) 0001 no-progress want-ref refs/heads/o/foo From patchwork Mon May 25 19:59:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11569399 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2B102739 for ; Mon, 25 May 2020 20:00:37 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0919C2073B for ; Mon, 25 May 2020 20:00:37 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="WY5NG1rb" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390651AbgEYUAg (ORCPT ); Mon, 25 May 2020 16:00:36 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38856 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390616AbgEYUAc (ORCPT ); Mon, 25 May 2020 16:00:32 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 976E560D1F; Mon, 25 May 2020 20:00:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1590436800; bh=hYAhEgO21Q1ObbuQ8R0buMjNd5pGl8EufAeC4U72FN8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=WY5NG1rbXg5bR8BqX9jnJt2qAs58z4YxJ0dU7OKMQK1LP9fKWpcnniFTOk9stiGAm B0cWpCrTaL55/8pnBjY4aSxByjDuKxXotZ+6Ioh96sKnqPzgWhqViQOGeh8EfFvDqh vJ5bOvTe1eDLbWoFror+0NKkOFbqhUj7YdLTn38TNULewvRr8S6ulbZVGajMkJsbBR vWU8ObSjDx4ZXWkF+Jz+XA+jg85H75CsMyouURce/HUmRc7WhAWKGzBFTfoCU+nAeB opvz7ii7PzqmesZk3cEfF1TkakJFjkk3/6hD5cpOHzd/Mp3iQ/ifbmxpqEOEE8n766 QnBCe5Z9JJUJ1qBA5RzF0/YapwHozutcQ//AYz98rTnXW/bI6G5jdOr8SycHG0g59Y l57uloOwZgkKOGB0Ml/vo30HGcVUENIRsLVaa8MCyMTEDZuH24m18mRFAJG+h0oFRM 7RtbOru1yXZUjjS4WlmEpPeXRF5FHHFpUVZBcAcf73VaMtQel7u From: "brian m. carlson" To: Cc: =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v2 41/44] t5704: send object-format capability with SHA-256 Date: Mon, 25 May 2020 19:59:27 +0000 Message-Id: <20200525195930.309665-42-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.rc0.183.gde8f92d652 In-Reply-To: <20200525195930.309665-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200525195930.309665-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When we speak protocol v2 in this test, we must pass the object-format header if the algorithm is not SHA-1. Otherwise, git upload-pack fails because the hash algorithm doesn't match and not because we've failed to speak the protocol correctly. Pass the header so that our assertions test what we're really interested in. Signed-off-by: brian m. carlson --- t/t5704-protocol-violations.sh | 2 ++ 1 file changed, 2 insertions(+) diff --git a/t/t5704-protocol-violations.sh b/t/t5704-protocol-violations.sh index 950cfb21fe..5c941949b9 100755 --- a/t/t5704-protocol-violations.sh +++ b/t/t5704-protocol-violations.sh @@ -9,6 +9,7 @@ making sure that we do not segfault or otherwise behave badly.' test_expect_success 'extra delim packet in v2 ls-refs args' ' { packetize command=ls-refs && + packetize "object-format=$(test_oid algo)" && printf 0001 && # protocol expects 0000 flush here printf 0001 @@ -21,6 +22,7 @@ test_expect_success 'extra delim packet in v2 ls-refs args' ' test_expect_success 'extra delim packet in v2 fetch args' ' { packetize command=fetch && + packetize "object-format=$(test_oid algo)" && printf 0001 && # protocol expects 0000 flush here printf 0001 From patchwork Mon May 25 19:59:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11569401 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 963A4912 for ; Mon, 25 May 2020 20:00:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 747B22073B for ; Mon, 25 May 2020 20:00:39 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="lWgfWh40" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390657AbgEYUAh (ORCPT ); Mon, 25 May 2020 16:00:37 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38858 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390626AbgEYUAc (ORCPT ); Mon, 25 May 2020 16:00:32 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 0A23C60D20; Mon, 25 May 2020 20:00:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1590436801; bh=0kTnW+si/14E4LvMgtbMNpi0j1rIx5tw+Yjkj/LLNLQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=lWgfWh4052vO2yv4L+bARyg0AUBiVcFw8ErpK1eCqoZNtlhoxVjis88zH6S1FfvV3 pTUIY0nkvZBvlh/GGFZMVWN+LD0loS2kzexdd6oaQkpZjjlfMsUd07Xd2rj+Cy34Ap 5+d2FLX4igg4R8ZI4P8SnSk0n9w50hq49zaG+YTIVFWbEf4jjoPx+TXw/uQuNcupwV 4JD70eskY30dZd7RHB8l7R7k1RmDQAAuCxe/NxOUXkTp9u9DVtBXAkbO0p+Sjfj6sz y5h8plJp9sl1QWYZwCDsEwwAV8IQroD9r+R/MzNj9+RWqNF7A+679hHeLYs2quGWRh hvR4h7wMuvatktm5Mmeci5MFDj9Ur+ZiKPL/kjP6ky0miJsJcujD1MMBY2kmsZ0myo ZUOv8rypoO3H9NN3BrJBsfLqf9+p8liiHbIYp18QGVX+bCphOZ+d/uViA2ntiCdUJ0 1Mmtcg11LBrcD71gueI1yumw1+gjoKlibXkOrTCyjUTREU8bngH From: "brian m. carlson" To: Cc: =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v2 42/44] t5300: pass --object-format to git index-pack Date: Mon, 25 May 2020 19:59:28 +0000 Message-Id: <20200525195930.309665-43-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.rc0.183.gde8f92d652 In-Reply-To: <20200525195930.309665-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200525195930.309665-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org git index-pack by default reads the repository to determine the object format. However, when outside of a repository, it's necessary to specify the hash algorithm in use so that the pack can be properly indexed. Add an --object-format argument when invoking git index-pack outside of a repository. Signed-off-by: brian m. carlson --- t/t5300-pack-object.sh | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/t/t5300-pack-object.sh b/t/t5300-pack-object.sh index 410a09b0dd..746cdb626e 100755 --- a/t/t5300-pack-object.sh +++ b/t/t5300-pack-object.sh @@ -12,7 +12,8 @@ TRASH=$(pwd) test_expect_success \ 'setup' \ - 'rm -f .git/index* && + 'test_oid_init && + rm -f .git/index* && perl -e "print \"a\" x 4096;" > a && perl -e "print \"b\" x 4096;" > b && perl -e "print \"c\" x 4096;" > c && @@ -412,18 +413,18 @@ test_expect_success 'set up pack for non-repo tests' ' ' test_expect_success 'index-pack --stdin complains of non-repo' ' - nongit test_must_fail git index-pack --stdin works in non-repo' ' - nongit git index-pack ../foo.pack && + nongit git index-pack --object-format=$(test_oid algo) ../foo.pack && test_path_is_file foo.idx ' test_expect_success 'index-pack --strict works in non-repo' ' rm -f foo.idx && - nongit git index-pack --strict ../foo.pack && + nongit git index-pack --strict --object-format=$(test_oid algo) ../foo.pack && test_path_is_file foo.idx ' From patchwork Mon May 25 19:59:29 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11569359 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4D3CE739 for ; Mon, 25 May 2020 20:00:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 35A2320776 for ; Mon, 25 May 2020 20:00:08 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="KeTj/awl" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390482AbgEYUAH (ORCPT ); Mon, 25 May 2020 16:00:07 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38728 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390463AbgEYUAC (ORCPT ); Mon, 25 May 2020 16:00:02 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 7234360D1B; Mon, 25 May 2020 20:00:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1590436801; bh=VQHK4KZV/HJtohhf/r/fcuDQ02WfP9S43VaRaa73xx8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=KeTj/awlmSZCc1n4OsbgMCeirTBqxwTuWJDdgTMQG7aCx99d4lKYwlGwP6/YSYOWM 47jKdUyMGAW8OYe6+lB2vPx8XgAVKfPH828h8pksN/UUKTreZzRr/QNENTBP/VhBLZ H7ae9+sItJD6a6NOcDPHdcSrDIR33O9KbY8N78l2NdBq6cjaTk2r2tJppT5Gx4J5A7 BMZCQZL0OzJ3/QEgVZ98iRZeYOIXrUJi0QvWkKF7APPRNLtyRhLaFE+V8NJkbUiV5y V9PcehK7SVTu0yqv7r2r0ATJDVbIUHoltQ0g3KMAdf6AQqpS05gQtQUgL5NTjUZ391 syRCxttjlK1DLiuU+rKYIV+E6is4+76PilJrVT40/UaU+GlB9aBnCwVS4UUCKnhPei 6KcoxI6YTnMyeEZEwjajIC3wpElxL3EPiUC5NWsok3oqH8OltIao7yVPWCvUrn9Jkb uDykMgQckbaIsVY0m9pDY30B7mjzo48BZ7k1Fan+ru+6E/GObkq From: "brian m. carlson" To: Cc: =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v2 43/44] bundle: detect hash algorithm when reading refs Date: Mon, 25 May 2020 19:59:29 +0000 Message-Id: <20200525195930.309665-44-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.rc0.183.gde8f92d652 In-Reply-To: <20200525195930.309665-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200525195930.309665-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Much like with the dumb HTTP transport, there isn't a way to explicitly specify the hash algorithm when dealing with a bundle, so detect the algorithm based on the length of the object IDs in the prerequisites and ref advertisements. Signed-off-by: brian m. carlson --- bundle.c | 22 +++++++++++++++++++++- bundle.h | 1 + transport.c | 10 ++++++++-- 3 files changed, 30 insertions(+), 3 deletions(-) diff --git a/bundle.c b/bundle.c index 99439e07a1..2a0d744d3f 100644 --- a/bundle.c +++ b/bundle.c @@ -23,6 +23,17 @@ static void add_to_ref_list(const struct object_id *oid, const char *name, list->nr++; } +static const struct git_hash_algo *detect_hash_algo(struct strbuf *buf) +{ + size_t len = strcspn(buf->buf, " \n"); + int algo; + + algo = hash_algo_by_length(len / 2); + if (algo == GIT_HASH_UNKNOWN) + return NULL; + return &hash_algos[algo]; +} + static int parse_bundle_header(int fd, struct bundle_header *header, const char *report_path) { @@ -52,12 +63,21 @@ static int parse_bundle_header(int fd, struct bundle_header *header, } strbuf_rtrim(&buf); + if (!header->hash_algo) { + header->hash_algo = detect_hash_algo(&buf); + if (!header->hash_algo) { + error(_("unknown hash algorithm length")); + status = -1; + break; + } + } + /* * Tip lines have object name, SP, and refname. * Prerequisites have object name that is optionally * followed by SP and subject line. */ - if (parse_oid_hex(buf.buf, &oid, &p) || + if (parse_oid_hex_algop(buf.buf, &oid, &p, header->hash_algo) || (*p && !isspace(*p)) || (!is_prereq && !*p)) { if (report_path) diff --git a/bundle.h b/bundle.h index ceab0c7475..2dc9442024 100644 --- a/bundle.h +++ b/bundle.h @@ -15,6 +15,7 @@ struct ref_list { struct bundle_header { struct ref_list prerequisites; struct ref_list references; + const struct git_hash_algo *hash_algo; }; int is_bundle(const char *path, int quiet); diff --git a/transport.c b/transport.c index b43d985f90..38a432be69 100644 --- a/transport.c +++ b/transport.c @@ -143,6 +143,9 @@ static struct ref *get_refs_from_bundle(struct transport *transport, data->fd = read_bundle_header(transport->url, &data->header); if (data->fd < 0) die(_("could not read bundle '%s'"), transport->url); + + transport->hash_algo = data->header.hash_algo; + for (i = 0; i < data->header.references.nr; i++) { struct ref_list_entry *e = data->header.references.list + i; struct ref *ref = alloc_ref(e->name); @@ -157,11 +160,14 @@ static int fetch_refs_from_bundle(struct transport *transport, int nr_heads, struct ref **to_fetch) { struct bundle_transport_data *data = transport->data; + int ret; if (!data->get_refs_from_bundle_called) get_refs_from_bundle(transport, 0, NULL); - return unbundle(the_repository, &data->header, data->fd, - transport->progress ? BUNDLE_VERBOSE : 0); + ret = unbundle(the_repository, &data->header, data->fd, + transport->progress ? BUNDLE_VERBOSE : 0); + transport->hash_algo = data->header.hash_algo; + return ret; } static int close_bundle(struct transport *transport) From patchwork Mon May 25 19:59:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11569363 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1EFB6912 for ; Mon, 25 May 2020 20:00:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id F18FE20776 for ; Mon, 25 May 2020 20:00:11 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="yA64YthG" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390494AbgEYUAK (ORCPT ); Mon, 25 May 2020 16:00:10 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38722 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390398AbgEYUAD (ORCPT ); Mon, 25 May 2020 16:00:03 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id DC92360D21; Mon, 25 May 2020 20:00:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1590436802; bh=73juDmT7AedVAFtoq2gbukr8T2H8y8stjGC9wi2bUQQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=yA64YthGx2mmnTxrt/gPUQijBpuXkFUfWL7pahvAMA89ZNngYG3qzVVx0tyUjXpvx BGDqYd6xmCmf28PjPCL6Qp+CWI95ipfmdq9kCMsBAbDmRp/8htyxmaKl9bIvE+r7BW lfcuaREGuQmNDl7S//MwheYn1KfcMXHClju7OOeE3K3QMTLCeIUFa39ErmXSog17wI W5DEjxqW0y2fsDuQTCxKGv39PIgEqiC58fEou5tueZZeKF9SUR2Od7SCNStYAWHb9m FzDSDtDwAZLBt9ugFcxb6jTZOQNgQzgvINWsk4aXTIwIHwkbKqH2J1PSWrBm7ennfk ERbEO5cIsNdgLfcLxy/e50k1uHs20o2Ak90pLf2J+4oU2EPTKwtqi/TYhowtEajoaV icja2s4ZEzoSjqK+zdTCmhYOB0OgnPALlFgeVEQdgOK299t4+wQwHEC6zMVVZ9BgCe +160TZhgt33xr9jn1q8Mct/sCWZnrCrTqeCFEeogUeKXsaynRAv From: "brian m. carlson" To: Cc: =?utf-8?q?Martin_=C3=85gren?= Subject: [PATCH v2 44/44] remote-testgit: adapt for object-format Date: Mon, 25 May 2020 19:59:30 +0000 Message-Id: <20200525195930.309665-45-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.27.0.rc0.183.gde8f92d652 In-Reply-To: <20200525195930.309665-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> <20200525195930.309665-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When using an algorithm other than SHA-1, we need the remote helper to advertise support for the object-format extension and provide information back to us so that we can properly parse refs and return data. Ensure that the test remote helper understands these extensions. Signed-off-by: brian m. carlson --- t/t5801/git-remote-testgit | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/t/t5801/git-remote-testgit b/t/t5801/git-remote-testgit index 6b9f0b5dc7..1544d6dc6b 100755 --- a/t/t5801/git-remote-testgit +++ b/t/t5801/git-remote-testgit @@ -52,9 +52,11 @@ do test -n "$GIT_REMOTE_TESTGIT_SIGNED_TAGS" && echo "signed-tags" test -n "$GIT_REMOTE_TESTGIT_NO_PRIVATE_UPDATE" && echo "no-private-update" echo 'option' + echo 'object-format' echo ;; list) + echo ":object-format $(git rev-parse --show-object-format=storage)" git for-each-ref --format='? %(refname)' 'refs/heads/' 'refs/tags/' head=$(git symbolic-ref HEAD) echo "@$head HEAD" @@ -139,6 +141,10 @@ do test $val = "true" && force="true" || force= echo "ok" ;; + object-format) + test $val = "true" && object_format="true" || object_format= + echo "ok" + ;; *) echo "unsupported" ;;