From patchwork Wed May 13 00:53:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11544361 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 14DE11668 for ; Wed, 13 May 2020 00:54:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id F084920675 for ; Wed, 13 May 2020 00:54:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="FKoPJzi6" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731399AbgEMAyj (ORCPT ); Tue, 12 May 2020 20:54:39 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38092 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726031AbgEMAyj (ORCPT ); Tue, 12 May 2020 20:54:39 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 0A91760797; Wed, 13 May 2020 00:54:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1589331278; bh=Qe1db/Vkg2LYh4oBw4EKMmRj3837dhpHVhX2HJXmgH4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=FKoPJzi6tqQQOgzo4V7RRtTh9gVlddIobIA43mYMJuXHVc61J9tvqVZagnFNLT44j O388AFipQJDT25vEZgXPQiOxclCDhicozkxpwBlsgb8mjWBx25oGf7wj4Fe88bEdWp 07AnNLCHOSotf8lAuwUH3Agfp4UVz4Atuqc6xRzTy+jiiEw6kB4V9qjG+sTvZ3hwXh +hK79FnCNpI8LCCDTV7CkQUjKcdzL69OdreQX8DFmZ+h9i8RMtXinKROCBnnBAjTNq TlzdB/Z7HOBfXVrM5NlUOplaj5m3iaeSplQdHesCwtKrEkPzrUjSAU4cHZTmsDh6yz qOdyw6fbO07tLusHGzA80+WGXSkpmrWZmCyPcziPeKRmSVVWoU4O1Aw1vk4+jrUhPj Gbr8074GohH0m5+XT4hWM8aeCWmvG2rWe4/NXU9ipFm0dFrLM3gVBamBkan4ingH5s zV5h4sPA7+vd5M4kk1SS/BcGFlib/dc7veqScH2Z4qe8m9rFQed From: "brian m. carlson" To: Cc: Jonathan Tan Subject: [PATCH 01/44] t1050: match object ID paths in a hash-insensitive way Date: Wed, 13 May 2020 00:53:41 +0000 Message-Id: <20200513005424.81369-2-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be In-Reply-To: <20200513005424.81369-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org The pattern here looking for failures is specific to SHA-1. Let's create a variable that matches the regex or glob pattern for a path within the objects directory. Signed-off-by: brian m. carlson --- t/t1050-large.sh | 2 +- t/test-lib.sh | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/t/t1050-large.sh b/t/t1050-large.sh index 184b479a21..7f88ea07c2 100755 --- a/t/t1050-large.sh +++ b/t/t1050-large.sh @@ -64,7 +64,7 @@ test_expect_success 'add a large file or two' ' test $count = 1 && cnt=$(git show-index <"$idx" | wc -l) && test $cnt = 2 && - for l in .git/objects/??/?????????????????????????????????????? + for l in .git/objects/$OIDPATH_REGEX do test_path_is_file "$l" || continue bad=t diff --git a/t/test-lib.sh b/t/test-lib.sh index baf94546da..77e9a60fcb 100644 --- a/t/test-lib.sh +++ b/t/test-lib.sh @@ -1428,6 +1428,7 @@ test_oid_init ZERO_OID=$(test_oid zero) OID_REGEX=$(echo $ZERO_OID | sed -e 's/0/[0-9a-f]/g') +OIDPATH_REGEX=$(test_oid_to_path $ZERO_OID | sed -e 's/0/[0-9a-f]/g') EMPTY_TREE=$(test_oid empty_tree) EMPTY_BLOB=$(test_oid empty_blob) _z40=$ZERO_OID From patchwork Wed May 13 00:53:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11544363 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3A32C90 for ; Wed, 13 May 2020 00:54:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 19EA020675 for ; Wed, 13 May 2020 00:54:41 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="dYCTq259" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731619AbgEMAyk (ORCPT ); Tue, 12 May 2020 20:54:40 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38096 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729215AbgEMAyj (ORCPT ); Tue, 12 May 2020 20:54:39 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 7243F60892; Wed, 13 May 2020 00:54:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1589331278; bh=XnY1hBbXCVrgcS1UHcgKuO6fcYFCkpQ6glUvbqDZkr4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=dYCTq259dXN4jkLpSbv/4Mwnp2CQt/Vre8QGYoA5+77xAmSlKVJtWfIYQ2w91l+7G i+MW9GbwaFe24d+KITVkE/jBCT/IZIeukpISWRDbJvAo3Amg6wtVoZbvBGG0cRlFTO qMHwBX0mFacs3KzLGmWpz3LHL2XQPS80W80IQ/rXawgdBdy4V+1/dR2RXfnxH0hs15 1FcLj2m69Ghq8NUm71/aKyDMuFfdKpbSQ85/ntoL6cgF2/yx0OTTfIRS9tWcrTYAKX +GHI9gFG/YaT8MhYeKn7HuKg6dgAdpKab8d917qyRw7zQrPZJ8uxkm1b1mQZDoJSFV vufQdes7xssQSc5qmkUJwiaA0/Mv80n4yjhpUBd6ZaZud4XxvS1disbIRiMuAr3NxR DLVFmSFD8LBhwoLXmtuuASbLWllXIqU5fteeesOMKUwa6JC6P8o74Zn9GIkWQfNwd+ 6UlR3J31znXByiBsn4fNfypn6IOwYZOqJCVerS6/wehmIytVg7k From: "brian m. carlson" To: Cc: Jonathan Tan Subject: [PATCH 02/44] Documentation: document v1 protocol object-format capability Date: Wed, 13 May 2020 00:53:42 +0000 Message-Id: <20200513005424.81369-3-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be In-Reply-To: <20200513005424.81369-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Document a capability that indicates which hash algorithms are in use by both sides of a remote connection. Use the term "object-format", since this is the term used for the repository extension as well. Signed-off-by: brian m. carlson --- .../technical/protocol-capabilities.txt | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/Documentation/technical/protocol-capabilities.txt b/Documentation/technical/protocol-capabilities.txt index 2b267c0da6..026c42f86a 100644 --- a/Documentation/technical/protocol-capabilities.txt +++ b/Documentation/technical/protocol-capabilities.txt @@ -176,6 +176,21 @@ agent strings are purely informative for statistics and debugging purposes, and MUST NOT be used to programmatically assume the presence or absence of particular features. +object-format +------------- + +This capability, which takes a hash algorithm as an argument, indicates +that the server supports the given hash algorithms. It may be sent +multiple times; if so, the first one given is the one used in the ref +advertisement. + +When provided by the client, this indicates that it intends to use the +given hash algorithm to communicate. The algorithm provided must be one +that the server supports. + +If this capability is not provided, it is assumed that the only +supported algorithm is SHA-1. + symref ------ @@ -189,7 +204,6 @@ refs being sent. Clients MAY use the parameters from this capability to select the proper initial branch when cloning a repository. - shallow ------- From patchwork Wed May 13 00:53:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11544415 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6919F90 for ; Wed, 13 May 2020 00:55:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5392720753 for ; Wed, 13 May 2020 00:55:16 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="wYjcKk7g" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732017AbgEMAzO (ORCPT ); Tue, 12 May 2020 20:55:14 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38198 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731991AbgEMAzK (ORCPT ); Tue, 12 May 2020 20:55:10 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id D9DDF609CF; Wed, 13 May 2020 00:54:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1589331279; bh=4rEdmRrkp7YTyoWHj6LeQnNui2xThC7IUvb10raiKlY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=wYjcKk7gk+FQ3iJshZNTRgdJT5eNONMlgB9kgevRSBSIOY//cxS7JQ8FIr8B5tc5c xBOmh0RFYxdaX8FdavR9g9Mua9LZM2RRpWwPlpbZY+kSnQ1+b+N9O7GuLfvLC+p/fO 2lEYIaxGHbsHsflcubMp/Jd0bDalj+hgSqxkwORlryYigCYweRpnL+L2poyJ4QCU/K kUl0ZgncqKPeXDBNKkqWCAu6EqD2qaECDq78Udhuyw7FrXEUXUtrMod4r6uF04QLMr TfV5JAJwB2NCH6CwCPSF84YHBLkrDyT0UY1DmLN+c7Wi90SbAvrZvF3GYSLwKWZz2H Sy5/SqC/P9LrLh7NM8BPOAsG5E2BKEDWRFSDDviDW8glxpAxmdUndVyNTpFbmwUkd9 oSuSw0uRaTLuFfdtaO6oeEyXxtulIojmDQIrCEYA5VHoHdoR/lgbF3yzTgMyJwaOp2 +0rmGDm8CFRCV1AFn2aosJbCCVM6fbuHc8tL+RwwelzmbJcbbFu From: "brian m. carlson" To: Cc: Jonathan Tan Subject: [PATCH 03/44] connect: have ref processing code take struct packet_reader Date: Wed, 13 May 2020 00:53:43 +0000 Message-Id: <20200513005424.81369-4-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be In-Reply-To: <20200513005424.81369-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In a future patch, we'll want to access multiple members from struct packet_reader when parsing references. Therefore, have the ref parsing code take pointers to struct reader instead of having to pass multiple arguments to each function. Signed-off-by: brian m. carlson --- connect.c | 23 ++++++++++++++--------- 1 file changed, 14 insertions(+), 9 deletions(-) diff --git a/connect.c b/connect.c index 23013c6344..641388a766 100644 --- a/connect.c +++ b/connect.c @@ -204,8 +204,9 @@ static void annotate_refs_with_symref_info(struct ref *ref) string_list_clear(&symref, 0); } -static void process_capabilities(const char *line, int *len) +static void process_capabilities(struct packet_reader *reader, int *len) { + const char *line = reader->line; int nul_location = strlen(line); if (nul_location == *len) return; @@ -213,8 +214,9 @@ static void process_capabilities(const char *line, int *len) *len = nul_location; } -static int process_dummy_ref(const char *line) +static int process_dummy_ref(const struct packet_reader *reader) { + const char *line = reader->line; struct object_id oid; const char *name; @@ -234,9 +236,11 @@ static void check_no_capabilities(const char *line, int len) line + strlen(line)); } -static int process_ref(const char *line, int len, struct ref ***list, - unsigned int flags, struct oid_array *extra_have) +static int process_ref(const struct packet_reader *reader, int len, + struct ref ***list, unsigned int flags, + struct oid_array *extra_have) { + const char *line = reader->line; struct object_id old_oid; const char *name; @@ -260,9 +264,10 @@ static int process_ref(const char *line, int len, struct ref ***list, return 1; } -static int process_shallow(const char *line, int len, +static int process_shallow(const struct packet_reader *reader, int len, struct oid_array *shallow_points) { + const char *line = reader->line; const char *arg; struct object_id old_oid; @@ -315,20 +320,20 @@ struct ref **get_remote_heads(struct packet_reader *reader, switch (state) { case EXPECTING_FIRST_REF: - process_capabilities(reader->line, &len); - if (process_dummy_ref(reader->line)) { + process_capabilities(reader, &len); + if (process_dummy_ref(reader)) { state = EXPECTING_SHALLOW; break; } state = EXPECTING_REF; /* fallthrough */ case EXPECTING_REF: - if (process_ref(reader->line, len, &list, flags, extra_have)) + if (process_ref(reader, len, &list, flags, extra_have)) break; state = EXPECTING_SHALLOW; /* fallthrough */ case EXPECTING_SHALLOW: - if (process_shallow(reader->line, len, shallow_points)) + if (process_shallow(reader, len, shallow_points)) break; die(_("protocol error: unexpected '%s'"), reader->line); case EXPECTING_DONE: From patchwork Wed May 13 00:53:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11544365 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2CC081668 for ; Wed, 13 May 2020 00:54:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1604B20675 for ; Wed, 13 May 2020 00:54:44 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="Uc1YZH98" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731656AbgEMAyl (ORCPT ); Tue, 12 May 2020 20:54:41 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38104 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726031AbgEMAyk (ORCPT ); Tue, 12 May 2020 20:54:40 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 4CD62609D3; Wed, 13 May 2020 00:54:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1589331279; bh=OLURWUEYz9+b3/c+Xf6DUH98Mfh15gzkrC5937mfdYo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=Uc1YZH98UMifDUuPvWZTTKkO1LIaywXF0u4Qam/n+cpvcs4AajqtoatqtuaWXtNlH TvCPWsz8JVzJzd3QyyYulpn6AiQ7MoWgZ+Hu0VfkSwoYh0Jm5ly2Rc1w+lcnIBStZY KQlr4ooycjgzJuiYl2phc7NLnoeklSXihVC8cg2di1VXcQy1qBnEkJ18vszs4Ownqy YBDlGuNjgV29ibOQhA4twrfLjotpUmicqsjq7sj5ZL+8w15H9ur2pIUEDpifUeQyFD q8Gc2zHZOi2YXOT+o82nGfoNsVqcF4aqit1wSjCtHNduBKsx/kpyZdi3b+Qs/khQ7S px6ESfxkIdDREWQNzx7Uu1L/EJEziZ9q1f5XQTKEHQbtHHKu0OOesULXMIikbulnPk 7Hpv91dyFG/dyY603Awq0Y3BHePUedbocEqehnO10kdnWmQv5hAiRD9k/eJlpkuixy U3tEW8hZzJJR5sVLkt8yfm6wGDTUd8XebuXVWeE/Qb2DiiKMljh From: "brian m. carlson" To: Cc: Jonathan Tan Subject: [PATCH 04/44] wrapper: add function to compare strings with different NUL termination Date: Wed, 13 May 2020 00:53:44 +0000 Message-Id: <20200513005424.81369-5-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be In-Reply-To: <20200513005424.81369-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When parsing capabilities for the pack protocol, there are times we'll want to compare the value of a capability to a NUL-terminated string. Since the data we're reading will be space-terminated, not NUL-terminated, we need a function that compares the two strings, but also checks that they're the same length. Otherwise, if we used strncmp to compare these strings, we might accidentally accept a parameter that was a prefix of the expected value. Add a function, xstrncmpz, that takes a NUL-terminated string and a non-NUL-terminated string, plus a length, and compares them, ensuring that they are the same length. Signed-off-by: brian m. carlson --- git-compat-util.h | 2 ++ wrapper.c | 12 ++++++++++++ 2 files changed, 14 insertions(+) diff --git a/git-compat-util.h b/git-compat-util.h index 8ba576e81e..6503deb171 100644 --- a/git-compat-util.h +++ b/git-compat-util.h @@ -868,6 +868,8 @@ char *xgetcwd(void); FILE *fopen_for_writing(const char *path); FILE *fopen_or_warn(const char *path, const char *mode); +int xstrncmpz(const char *s, const char *t, size_t len); + /* * FREE_AND_NULL(ptr) is like free(ptr) followed by ptr = NULL. Note * that ptr is used twice, so don't pass e.g. ptr++. diff --git a/wrapper.c b/wrapper.c index 3a1c0e0526..15a09740e7 100644 --- a/wrapper.c +++ b/wrapper.c @@ -430,6 +430,18 @@ int xmkstemp(char *filename_template) return fd; } +/* + * Like strncmp, but only return zero if s is NUL-terminated and exactly len + * characters long. If it is not, consider it greater than t. + */ +int xstrncmpz(const char *s, const char *t, size_t len) +{ + int res = strncmp(s, t, len); + if (res) + return res; + return s[len] == '\0' ? 0 : 1; +} + /* Adapted from libiberty's mkstemp.c. */ #undef TMP_MAX From patchwork Wed May 13 00:53:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11544425 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2EC831668 for ; Wed, 13 May 2020 00:55:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0E8B420675 for ; Wed, 13 May 2020 00:55:23 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="OgQNuYsL" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732041AbgEMAzW (ORCPT ); Tue, 12 May 2020 20:55:22 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38208 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731999AbgEMAzK (ORCPT ); Tue, 12 May 2020 20:55:10 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id B2E5760797; Wed, 13 May 2020 00:54:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1589331280; bh=cCM6VaFve35hSJGWj+rO2ac6FpNeh7H9Z9FzADHZSbs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=OgQNuYsLY5EnMOZNoibG7xcGSpP0qfyizlNpxzxvw6aWWWoTZSUWRhA7Wm/ajtVAU qiX6hP83+nWwvnG+7DU3aDTDWRc0FEX2aQeGXe4f7GjJeIPJfXr2Rb9Q+tJ8su3iAf ueFpFO8aXery1dWaIzw/sMhqnCC98NQkOqEWBRLKb536FSqZtgehofQQxi5vSqdQY7 7vlZ1hwUIvbDTUVz1sdAwDPv+xKv9CrKsYZMEtxz4MUem9g8QKKRAMfeT56KoOzZPb R0ufsjfeCNdIQLLjI1UjH/IjRbg51kXEwdcm5cVBJeO5BCpha5kZIz/fPmeHU+bfzp j0o1B/9nl1xVfoJ7VxpzUUeAMZDELVgs1iuF7mBf5KT794ex8SGjbC+aYLD8KXXNzJ bBHh7VlKHgU15z8mYYpvPWtde4RG08rbIAo7fn0AbCRoWDkaoxekLZLd7f/wmRGC94 XBj8exo52zCj9CUOKVqOpgEJx1CMePrzZ46MfjNLwrpH+WYnxmI From: "brian m. carlson" To: Cc: Jonathan Tan Subject: [PATCH 05/44] remote: advertise the object-format capability on the server side Date: Wed, 13 May 2020 00:53:45 +0000 Message-Id: <20200513005424.81369-6-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be In-Reply-To: <20200513005424.81369-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Advertise the current hash algorithm in use by using the object-format capability as part of the ref advertisement. Signed-off-by: brian m. carlson --- builtin/receive-pack.c | 1 + upload-pack.c | 3 ++- 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c index d37ab776b3..a4159b559e 100644 --- a/builtin/receive-pack.c +++ b/builtin/receive-pack.c @@ -248,6 +248,7 @@ static void show_ref(const char *path, const struct object_id *oid) strbuf_addf(&cap, " push-cert=%s", push_cert_nonce); if (advertise_push_options) strbuf_addstr(&cap, " push-options"); + strbuf_addf(&cap, " object-format=%s", the_hash_algo->name); strbuf_addf(&cap, " agent=%s", git_user_agent_sanitized()); packet_write_fmt(1, "%s %s%c%s\n", oid_to_hex(oid), path, 0, cap.buf); diff --git a/upload-pack.c b/upload-pack.c index 902d0ad5e1..df6cb51db7 100644 --- a/upload-pack.c +++ b/upload-pack.c @@ -1005,7 +1005,7 @@ static int send_ref(const char *refname, const struct object_id *oid, struct strbuf symref_info = STRBUF_INIT; format_symref_info(&symref_info, cb_data); - packet_write_fmt(1, "%s %s%c%s%s%s%s%s%s agent=%s\n", + packet_write_fmt(1, "%s %s%c%s%s%s%s%s%s object-format=%s agent=%s\n", oid_to_hex(oid), refname_nons, 0, capabilities, (allow_unadvertised_object_request & ALLOW_TIP_SHA1) ? @@ -1015,6 +1015,7 @@ static int send_ref(const char *refname, const struct object_id *oid, stateless_rpc ? " no-done" : "", symref_info.buf, allow_filter ? " filter" : "", + the_hash_algo->name, git_user_agent_sanitized()); strbuf_release(&symref_info); } else { From patchwork Wed May 13 00:53:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11544367 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4867117EA for ; Wed, 13 May 2020 00:54:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 31D0520675 for ; Wed, 13 May 2020 00:54:44 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="TTITDEpF" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731726AbgEMAym (ORCPT ); Tue, 12 May 2020 20:54:42 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38112 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729215AbgEMAyl (ORCPT ); Tue, 12 May 2020 20:54:41 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 25EC7609D4; Wed, 13 May 2020 00:54:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1589331280; bh=TXV9vw3iyjpcOd1Px6pnQlIK7bO3DW+gVm2gat27M3k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=TTITDEpF5g8344D3WWnCJLf2ggbleNmGHx6Bi+BmYKrCW3s03mcObDZTrB5zu7Xmn hBa/3EEtzfnRuc6akuiRrGMP276YofU3smUmbHEzOpL/abrN2+Hys+0HLe31HyOw37 dd7qQ67aOM5K12ITq6nCkq0jmw5lYOUP/o9hxLOlToNQRfzaZcNGiu0asWD3Rex3Wg zwAaPgmAEjg6Hs4iTqDwW/jEvPogiLi4HABwWHKGVY9xi7EsLfIPfhlCSyCy2epQfP odWEkwOGcOIcBj2aYnrHrOwCwQO/8Q/SZcIjlwfTdbjGaaS7WZcrcOTtZjDxpbjTdX 87IGv1CZNA1eMAZgCU287U8WE6LtPbZZEDGXfz3QQkJ6Rlu9e1PH2WDmFu05zC7Ph+ k0fDC3fpv+OoseZwYn4BQPkNTx5pQ+/FtPSEy8Fsh6un+VeuuWAd/5EmYyOXGa2xLI xKXjmY6ZsDDvza0KRAciu4p3LEC9eGQIYV/xgKEjHr3+VUFnfdb From: "brian m. carlson" To: Cc: Jonathan Tan Subject: [PATCH 06/44] connect: add function to parse multiple v1 capability values Date: Wed, 13 May 2020 00:53:46 +0000 Message-Id: <20200513005424.81369-7-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be In-Reply-To: <20200513005424.81369-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In a capability response, we can have multiple symref entries. In the future, we will also allow for multiple hash algorithms to be specified. To avoid duplication, expand the parse_feature_value function to take an optional offset where the parsing should begin next time. Add a wrapper function that allows us to query the next server feature value, and use it in the existing symref parsing code. Signed-off-by: brian m. carlson --- connect.c | 30 +++++++++++++++++++++--------- 1 file changed, 21 insertions(+), 9 deletions(-) diff --git a/connect.c b/connect.c index 641388a766..4027fd4677 100644 --- a/connect.c +++ b/connect.c @@ -18,7 +18,8 @@ static char *server_capabilities_v1; static struct argv_array server_capabilities_v2 = ARGV_ARRAY_INIT; -static const char *parse_feature_value(const char *, const char *, int *); +static const char *parse_feature_value(const char *, const char *, int *, int *); +static const char *next_server_feature_value(const char *feature, int *len, int *offset); static int check_ref(const char *name, unsigned int flags) { @@ -180,17 +181,16 @@ static void parse_one_symref_info(struct string_list *symref, const char *val, i static void annotate_refs_with_symref_info(struct ref *ref) { struct string_list symref = STRING_LIST_INIT_DUP; - const char *feature_list = server_capabilities_v1; + int offset = 0; - while (feature_list) { + while (1) { int len; const char *val; - val = parse_feature_value(feature_list, "symref", &len); + val = next_server_feature_value("symref", &len, &offset); if (!val) break; parse_one_symref_info(&symref, val, len); - feature_list = val + 1; } string_list_sort(&symref); @@ -452,7 +452,7 @@ struct ref **get_remote_refs(int fd_out, struct packet_reader *reader, return list; } -static const char *parse_feature_value(const char *feature_list, const char *feature, int *lenp) +static const char *parse_feature_value(const char *feature_list, const char *feature, int *lenp, int *offset) { int len; @@ -460,6 +460,8 @@ static const char *parse_feature_value(const char *feature_list, const char *fea return NULL; len = strlen(feature); + if (offset) + feature_list += *offset; while (*feature_list) { const char *found = strstr(feature_list, feature); if (!found) @@ -474,9 +476,14 @@ static const char *parse_feature_value(const char *feature_list, const char *fea } /* feature with a value (e.g., "agent=git/1.2.3") */ else if (*value == '=') { + int end; + value++; + end = strcspn(value, " \t\n"); if (lenp) - *lenp = strcspn(value, " \t\n"); + *lenp = end; + if (offset) + *offset = value + end - feature_list; return value; } /* @@ -491,12 +498,17 @@ static const char *parse_feature_value(const char *feature_list, const char *fea int parse_feature_request(const char *feature_list, const char *feature) { - return !!parse_feature_value(feature_list, feature, NULL); + return !!parse_feature_value(feature_list, feature, NULL, NULL); +} + +static const char *next_server_feature_value(const char *feature, int *len, int *offset) +{ + return parse_feature_value(server_capabilities_v1, feature, len, offset); } const char *server_feature_value(const char *feature, int *len) { - return parse_feature_value(server_capabilities_v1, feature, len); + return parse_feature_value(server_capabilities_v1, feature, len, NULL); } int server_supports(const char *feature) From patchwork Wed May 13 00:53:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11544369 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6FBB690 for ; Wed, 13 May 2020 00:54:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4DADD20675 for ; Wed, 13 May 2020 00:54:44 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="V/HYMPhD" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731806AbgEMAym (ORCPT ); Tue, 12 May 2020 20:54:42 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38104 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731638AbgEMAyl (ORCPT ); Tue, 12 May 2020 20:54:41 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 8D742609D3; Wed, 13 May 2020 00:54:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1589331280; bh=LYaNLLLOZ0jl34fl6MqE/r8TKb7ZelEL1oJtQOx/hqU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=V/HYMPhDiN14FEEXsjM0HRszfAkv1WWlvaymCgJp6xXbNSHSSpe776jjCG0Uy91cN /jP6vpPkHpIFxiONiIfW+dLRTyyN5FxT0ogzc50E9k/6CjmG57rSwn+TCnT3mEYAJx RyzrRERVGidW85VS1MPMW8XFaQAxRVKp7B+jLEU2EPMIMbNamNEbyvRHne0Du6ZjYQ Itz/SsO4NxkfjhH5Nwz8MB3H0jhcjC0LKvy2D2UsLiXXRQXHEtqd5iinznrTM+Ixuj ATRLLDWeBdeLThU2VOHriLGyCI9OoaW3HhGmOSEgUCXacwklMm0bZPa0mDxkYwkYLP C/YtJCfvuogJkIREl2NFKP0gbYA9DJi/HAgBxySC2SrXf6UbLhBg3BgJoqsbSLboxH KdmKztuevZZZSMdIPEn6YBe+iOXo8Mxa3Gty4G9/ujZUj/zna1JKADcf0YWk3y0QOM Q744hXUo45DPG7c6N2wDv6nrS1GY+iRI+3RmnXUHtHFGWmLl1Bx From: "brian m. carlson" To: Cc: Jonathan Tan Subject: [PATCH 07/44] connect: add function to fetch value of a v2 server capability Date: Wed, 13 May 2020 00:53:47 +0000 Message-Id: <20200513005424.81369-8-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be In-Reply-To: <20200513005424.81369-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org So far in protocol v2, all of our server capabilities that have values have not had values that we've been interested in parsing. For example, we receive but ignore the agent value. However, in a future commit, we're going to want to parse out the value of a server capability. To make this easy, add a function, server_feature_v2, that can fetch the value provided as part of the server capability. Signed-off-by: brian m. carlson --- connect.c | 15 +++++++++++++++ connect.h | 1 + 2 files changed, 16 insertions(+) diff --git a/connect.c b/connect.c index 4027fd4677..4df9e77206 100644 --- a/connect.c +++ b/connect.c @@ -84,6 +84,21 @@ int server_supports_v2(const char *c, int die_on_error) return 0; } +int server_feature_v2(const char *c, const char **v) +{ + int i; + + for (i = 0; i < server_capabilities_v2.argc; i++) { + const char *out; + if (skip_prefix(server_capabilities_v2.argv[i], c, &out) && + (*out == '=')) { + *v = out + 1; + return 1; + } + } + return 0; +} + int server_supports_feature(const char *c, const char *feature, int die_on_error) { diff --git a/connect.h b/connect.h index 5f2382e018..4d76a6017d 100644 --- a/connect.h +++ b/connect.h @@ -19,6 +19,7 @@ struct packet_reader; enum protocol_version discover_version(struct packet_reader *reader); int server_supports_v2(const char *c, int die_on_error); +int server_feature_v2(const char *c, const char **v); int server_supports_feature(const char *c, const char *feature, int die_on_error); From patchwork Wed May 13 00:53:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11544371 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4D5D990 for ; Wed, 13 May 2020 00:54:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3400D2078C for ; Wed, 13 May 2020 00:54:45 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="G5pFvIV7" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731829AbgEMAyn (ORCPT ); Tue, 12 May 2020 20:54:43 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38118 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726031AbgEMAym (ORCPT ); Tue, 12 May 2020 20:54:42 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 01CB460CF2; Wed, 13 May 2020 00:54:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1589331281; bh=CS7+T0CXvW4akc/aSvSaw17PrrQZy823QbT+zlFofqQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=G5pFvIV7aJTum1AjrMhRfP+lVMECC0BN79RfVZdYlrs2SurHkBEse8Ldddh1s5/Nl VTxNv7SP46nN56o2fB6kdWLhsPntH78Z+QaQO1tVHuTgwZo2cewjW4ihjbZ0h8Vwsc P1mdC9emx6yBYb/8hZo3lgDFVNCMQZ3EGD+S4Cx8KOq7jm95tOmOUlg2DKmR9vPQrg 2Nwe9wxHdKyHzYo0J1apzpI6nHSYT1V5zl8UIYgqWq/NxKbdTGJl6aeLs7NstOZoEy tkF/0VXCzcvvPJOGKeP3hepeH3zlZjFOh7/u60bFra+qxBLSISJaf797i3bWUusVDB WuFLROFYngpuf1jtFGKYXYPACdkKPPHrNa0yH/lX5yOEm/qrn5kSejhnYSpQhFs/Uw mfihVLjIjKJOGdHgKgGsx0Xit603Ue0ESosaPFvVDBjqroEnjeTJsREvqUeBIaNgfS 6u4TkbQRHUjZUsO1IIl8pPmyx/qs7xCKwXGMsGfLByqGLzYHOa/ From: "brian m. carlson" To: Cc: Jonathan Tan Subject: [PATCH 08/44] pkt-line: add a member for hash algorithm Date: Wed, 13 May 2020 00:53:48 +0000 Message-Id: <20200513005424.81369-9-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be In-Reply-To: <20200513005424.81369-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Add a member for the hash algorithm currently in use to the packet reader so it can parse references correctly. Signed-off-by: brian m. carlson --- pkt-line.c | 1 + pkt-line.h | 3 +++ 2 files changed, 4 insertions(+) diff --git a/pkt-line.c b/pkt-line.c index a0e87b1e81..a4aea075de 100644 --- a/pkt-line.c +++ b/pkt-line.c @@ -479,6 +479,7 @@ void packet_reader_init(struct packet_reader *reader, int fd, reader->buffer_size = sizeof(packet_buffer); reader->options = options; reader->me = "git"; + reader->hash_algo = &hash_algos[GIT_HASH_SHA1]; } enum packet_read_status packet_reader_read(struct packet_reader *reader) diff --git a/pkt-line.h b/pkt-line.h index fef3a0d792..4cd9435e9a 100644 --- a/pkt-line.h +++ b/pkt-line.h @@ -166,6 +166,9 @@ struct packet_reader { unsigned use_sideband : 1; const char *me; + + /* hash algorithm in use */ + const struct git_hash_algo *hash_algo; }; /* From patchwork Wed May 13 00:53:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11544417 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2B28F1668 for ; Wed, 13 May 2020 00:55:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 141C72078C for ; Wed, 13 May 2020 00:55:18 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="kUPMNlJk" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732032AbgEMAzR (ORCPT ); Tue, 12 May 2020 20:55:17 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38232 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732004AbgEMAzM (ORCPT ); Tue, 12 May 2020 20:55:12 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 68CA760CF9; Wed, 13 May 2020 00:54:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1589331281; bh=smurCJGXxF6jSqhkjKD9sTFjDwJyxySvg1esDLljrV0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=kUPMNlJkE1KskRogfJ5atV+F5eRvyzwKJksI/H9Xse93wyDg5qy4ogP7OtL2Efgjc vEQyWTznDlsvua8boxWrgNsHHKMdn3xpoGqgBi6ZSHTKwg8H6ccZtV0HHsjJkDzanK ZQQf9yPWOcHqAW9v1zy9llTjRjsQsts9CbfhlsPOS7SWP1vCnQw7UDgsOOV9hEiM6x Nvh7jRlyMkbSWDnpeQYMEcKe0ui/jIzjkM8CT3bWbGArVNQ9Fan5yh6jj/PgvU9cza lImvw93t9LIdBxNarddEvrQTL243AqHAsP5MhhS+B3TOFFdRbiCiPznoU9Cw2PVMrd u8CfzNCH9dJQTvP6FLXZYBLx0tSj9hqY+3qrHoym0ZIl+R3/6wNpSqqQ5VdcFg06c5 mh/TENmZTIYB7GRIg5sLvJBYGlTnunEEOePXnoik6PoOqtKCkT90LRIzvwHWkXEJEm OlOnALAi1UYPnHW75x76Sy2mJBPaGt+tEhqq3DRicTfawm+xjaN From: "brian m. carlson" To: Cc: Jonathan Tan Subject: [PATCH 09/44] transport: add a hash algorithm member Date: Wed, 13 May 2020 00:53:49 +0000 Message-Id: <20200513005424.81369-10-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be In-Reply-To: <20200513005424.81369-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When connecting to a remote system, we need to know what hash algorithm it will be using to talk to us. Add a hash_algo member to struct transport and add a function to read this data from the transport object. Signed-off-by: brian m. carlson --- transport.c | 8 ++++++++ transport.h | 8 ++++++++ 2 files changed, 16 insertions(+) diff --git a/transport.c b/transport.c index 15f5ba4e8f..b43d985f90 100644 --- a/transport.c +++ b/transport.c @@ -311,6 +311,7 @@ static struct ref *handshake(struct transport *transport, int for_push, BUG("unknown protocol version"); } data->got_remote_heads = 1; + transport->hash_algo = reader.hash_algo; if (reader.line_peeked) BUG("buffer must be empty at the end of handshake()"); @@ -996,9 +997,16 @@ struct transport *transport_get(struct remote *remote, const char *url) ret->smart_options->receivepack = remote->receivepack; } + ret->hash_algo = &hash_algos[GIT_HASH_SHA1]; + return ret; } +const struct git_hash_algo *transport_get_hash_algo(struct transport *transport) +{ + return transport->hash_algo; +} + int transport_set_option(struct transport *transport, const char *name, const char *value) { diff --git a/transport.h b/transport.h index 4298c855be..2a9f96c05a 100644 --- a/transport.h +++ b/transport.h @@ -115,6 +115,8 @@ struct transport { struct git_transport_options *smart_options; enum transport_family family; + + const struct git_hash_algo *hash_algo; }; #define TRANSPORT_PUSH_ALL (1<<0) @@ -243,6 +245,12 @@ int transport_push(struct repository *repo, const struct ref *transport_get_remote_refs(struct transport *transport, const struct argv_array *ref_prefixes); +/* + * Fetch the hash algorithm used by a remote. + * + * This can only be called after fetching the remote refs. + */ +const struct git_hash_algo *transport_get_hash_algo(struct transport *transport); int transport_fetch_refs(struct transport *transport, struct ref *refs); void transport_unlock_pack(struct transport *transport); int transport_disconnect(struct transport *transport); From patchwork Wed May 13 00:53:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11544373 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C81B61668 for ; Wed, 13 May 2020 00:54:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A94C62078C for ; Wed, 13 May 2020 00:54:47 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="gYpCqkES" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731850AbgEMAyq (ORCPT ); Tue, 12 May 2020 20:54:46 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38112 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731692AbgEMAyn (ORCPT ); Tue, 12 May 2020 20:54:43 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id CF8CD60CFE; Wed, 13 May 2020 00:54:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1589331282; bh=GQ7uCyvva2Azpko5tSygkfZCc9f/ScMTtEFfAlmp7gw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=gYpCqkES7iHAnBbS+KGNHifXhUL2wLXlxoZba7RwmprfVFk8JftV0sYq5WLxYyw3+ 0eOC9OQxLjb7+zM6J2IJLOdMSTRZont/EyI820B4/LWzn7hLKZz4aFqqWdIfowVcvU gBGI0TaMFROSloplnmSI/PofnQ8Tf4Ymz16tkFoF2NdEdELG+cdDnQHwcWCqTda5b6 XlzovFnihmpf+V074ZOP4N7m0E+DpC4wzeudjwblTn6HBfLiPMvOs3tgeou6cQEfS+ QHN8W5EUzsGiJiqyZgco8/SSjJcaEypfdMD7W+Ad4+jRTnm20zeKmLKuKYmI/cMsIw zWqu+zm5NQZbUI/Ex703tOed/ie8OSS07CbhhHTeSZKVMgaj592TyqtPOT0qFka3So Rq8J+6qxN9YFrqy8WmcgS4mVFkCDNJA4QD7ZWvxHShGDkZFMD5tLzpjcBJIAc9A1Bl j7UiDEKRZGfGUyTRaf6IwXb3s3Ga9b/E1zLcj3c0ltezqMEypAD From: "brian m. carlson" To: Cc: Jonathan Tan Subject: [PATCH 10/44] connect: add function to detect supported v1 hash functions Date: Wed, 13 May 2020 00:53:50 +0000 Message-Id: <20200513005424.81369-11-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be In-Reply-To: <20200513005424.81369-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Add a function, server_supports_hash, to see if the remote server supports a particular hash algorithm when speaking protocol v1. Signed-off-by: brian m. carlson --- connect.c | 24 ++++++++++++++++++++++++ connect.h | 1 + 2 files changed, 25 insertions(+) diff --git a/connect.c b/connect.c index 4df9e77206..cb69aafe2c 100644 --- a/connect.c +++ b/connect.c @@ -511,6 +511,30 @@ static const char *parse_feature_value(const char *feature_list, const char *fea return NULL; } +int server_supports_hash(const char *desired, int *feature_supported) +{ + int offset = 0; + int len, found = 0; + const char *hash; + + hash = next_server_feature_value("object-format", &len, &offset); + if (feature_supported) + *feature_supported = !!hash; + if (!hash) { + hash = hash_algos[GIT_HASH_SHA1].name; + len = strlen(hash); + } + while (hash) { + if (!xstrncmpz(desired, hash, len)) + found = 1; + + if (found) + return 1; + hash = next_server_feature_value("object-format", &len, &offset); + } + return 0; +} + int parse_feature_request(const char *feature_list, const char *feature) { return !!parse_feature_value(feature_list, feature, NULL, NULL); diff --git a/connect.h b/connect.h index 4d76a6017d..fc75d6a457 100644 --- a/connect.h +++ b/connect.h @@ -18,6 +18,7 @@ int url_is_local_not_ssh(const char *url); struct packet_reader; enum protocol_version discover_version(struct packet_reader *reader); +int server_supports_hash(const char *desired, int *feature_supported); int server_supports_v2(const char *c, int die_on_error); int server_feature_v2(const char *c, const char **v); int server_supports_feature(const char *c, const char *feature, From patchwork Wed May 13 00:53:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11544461 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9043A90 for ; Wed, 13 May 2020 00:56:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6A6B320753 for ; Wed, 13 May 2020 00:56:02 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="cP8edmyU" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731868AbgEMAyr (ORCPT ); Tue, 12 May 2020 20:54:47 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38104 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731760AbgEMAyn (ORCPT ); Tue, 12 May 2020 20:54:43 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 4240D609D3; Wed, 13 May 2020 00:54:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1589331282; bh=vC3yULAhTXszSyPeUhFmr4d2WyU9ajPycevd18cAwS4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=cP8edmyU4SnO4mbI6AAFB7Qcqz/I3lMlEUqfLhu1+qlbQ15e+vUbtzsTS3QxcgzFU yIBwyDXxf1FZcVCkw9WFqysknhi7lULiIGailvUcdEI7DXmPHPad/adCGY9qodgMdF PfOOpZuo0+7vvWlS/F4O4dwdKWYi1U270YbQTHAgDMqHSFZC6/qAVVBACl7ZauOJhe 0i29MaGmrKC+bc1qAS7H76spzmuI0qlbKQmEIiq0NE4+M3oExOWoPHOshFT82CJxiD /NTwGlyp4TXY+nv9o5rblzzs+2kT69rwP9tZH86xZ5/KhmV9Z746s9B434z4kxisMP lXKJKE9RB+spqMxGlBTH72PHK+jPebcx3sh8iWVsom4DIg0SdTIb65T+vUcuaVNWqM 9azQFo3ZRonherxpnhJFIQdWm7z40erExasdMqQORQl0OJ87Zk3pWKDcDeGEydk3Fv WFrWQZQalA7PZF0001VJ5CjQtNRWXFYGlqXKvdpJwSUmB/eVTAZ From: "brian m. carlson" To: Cc: Jonathan Tan Subject: [PATCH 11/44] send-pack: detect when the server doesn't support our hash Date: Wed, 13 May 2020 00:53:51 +0000 Message-Id: <20200513005424.81369-12-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be In-Reply-To: <20200513005424.81369-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Detect when the server doesn't support our hash algorithm and abort. Signed-off-by: brian m. carlson --- send-pack.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/send-pack.c b/send-pack.c index d1b7edc995..fb037568a9 100644 --- a/send-pack.c +++ b/send-pack.c @@ -362,6 +362,7 @@ int send_pack(struct send_pack_args *args, int atomic_supported = 0; int use_push_options = 0; int push_options_supported = 0; + int object_format_supported = 0; unsigned cmds_sent = 0; int ret; struct async demux; @@ -388,6 +389,9 @@ int send_pack(struct send_pack_args *args, if (server_supports("push-options")) push_options_supported = 1; + if (!server_supports_hash(the_hash_algo->name, &object_format_supported)) + die(_("the receiving end does not support this repository's hash algorithm")); + if (args->push_cert != SEND_PACK_PUSH_CERT_NEVER) { int len; push_cert_nonce = server_feature_value("push-cert", &len); @@ -428,6 +432,8 @@ int send_pack(struct send_pack_args *args, strbuf_addstr(&cap_buf, " atomic"); if (use_push_options) strbuf_addstr(&cap_buf, " push-options"); + if (object_format_supported) + strbuf_addf(&cap_buf, " object-format=%s", the_hash_algo->name); if (agent_supported) strbuf_addf(&cap_buf, " agent=%s", git_user_agent_sanitized()); From patchwork Wed May 13 00:53:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11544377 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 151F890 for ; Wed, 13 May 2020 00:54:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E8F5B20675 for ; Wed, 13 May 2020 00:54:50 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="VbsNlKMg" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731878AbgEMAys (ORCPT ); Tue, 12 May 2020 20:54:48 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38118 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731807AbgEMAyo (ORCPT ); Tue, 12 May 2020 20:54:44 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id A981A60CFF; Wed, 13 May 2020 00:54:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1589331282; bh=W+hJvQuxy2ekMVK7JOUcC5uq+ZOoIQrZE58qJIsEcVQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=VbsNlKMgoDNUQR94OO460ZWqAeVzULLL90kDSTkYtHraKxaMHipoHbtZeXtLtR4BY ZIEOsaFYDvCwarbGOJf97AGQDYgdkGiU3lu9nN9quXVY/kODhi+ixVxJCHdz9e6CiI kGxPWf3lg6AEMadIfEy137xT8n2rmOgllzuLMu7heGyuM33NtvdE4iEnpnEKu5PAdi ljb1QR+6fskbzEmRpO5nEfUkiChbAC+VEhjddXRydKO/CzvJxOTGJt42znMaJBZiMO XO+J5YqPGFKGJUiXkLFkYZgfw21IVGpHX3KlHq5x4g87tdJLcj4upGpQXCwfKiWcPm XcRwEl77jbAMtLg5wEr9um3aOon766o8Bm0Bs/j7PcJg9KEMxuDKkGG90vKNXgo+tZ k7KOFOv137FWyOFQZ93egnSlaewNLUXogLJ0S+Q2KZ9vq3yTSoDfTT0IbJaeRrHgwm lfVtgblTjpARfpsr/LFE5Z2o0Fg7Ohk1zQsZ6/mLFAamlFxuBEF From: "brian m. carlson" To: Cc: Jonathan Tan Subject: [PATCH 12/44] connect: make parse_feature_value extern Date: Wed, 13 May 2020 00:53:52 +0000 Message-Id: <20200513005424.81369-13-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be In-Reply-To: <20200513005424.81369-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org We're going to be using this function in other files, so no longer mark this function static. Signed-off-by: brian m. carlson --- connect.c | 3 +-- connect.h | 1 + 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/connect.c b/connect.c index cb69aafe2c..511a069304 100644 --- a/connect.c +++ b/connect.c @@ -18,7 +18,6 @@ static char *server_capabilities_v1; static struct argv_array server_capabilities_v2 = ARGV_ARRAY_INIT; -static const char *parse_feature_value(const char *, const char *, int *, int *); static const char *next_server_feature_value(const char *feature, int *len, int *offset); static int check_ref(const char *name, unsigned int flags) @@ -467,7 +466,7 @@ struct ref **get_remote_refs(int fd_out, struct packet_reader *reader, return list; } -static const char *parse_feature_value(const char *feature_list, const char *feature, int *lenp, int *offset) +const char *parse_feature_value(const char *feature_list, const char *feature, int *lenp, int *offset) { int len; diff --git a/connect.h b/connect.h index fc75d6a457..81935a0f2a 100644 --- a/connect.h +++ b/connect.h @@ -19,6 +19,7 @@ struct packet_reader; enum protocol_version discover_version(struct packet_reader *reader); int server_supports_hash(const char *desired, int *feature_supported); +const char *parse_feature_value(const char *, const char *, int *, int *); int server_supports_v2(const char *c, int die_on_error); int server_feature_v2(const char *c, const char **v); int server_supports_feature(const char *c, const char *feature, From patchwork Wed May 13 00:53:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11544379 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3E4C01668 for ; Wed, 13 May 2020 00:54:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1D3C02078C for ; Wed, 13 May 2020 00:54:52 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="Uv4aKmLx" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731900AbgEMAyu (ORCPT ); Tue, 12 May 2020 20:54:50 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38132 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731826AbgEMAyo (ORCPT ); Tue, 12 May 2020 20:54:44 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 1D2B960D00; Wed, 13 May 2020 00:54:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1589331283; bh=zpnT9fWOaLV9mwawdgB5SvkQ6uw0gD9jYk+wL9jjyZo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=Uv4aKmLxHrl8sYwPSebcpzDc1MIRsJWyzcjNiVJ4sAlDiQf0Ir2akvl5x7A91BYh6 jbPHiWusKz0pyav6gCx9rbO1ddgM1nZrDwDIK26DuzntFJWjc9j68zBv1K69tDesLG 8diKvg+dSXadshk4msUAXeHxzsxoTsPTDLBTfRLwAchW/FvmcDTPsCyQHVm2V5FY3X 5PsmYV6y4HIrY+LP1ZHCEi+KCm0+oaVE37fd0OmK0ccyboRe8gEGhOfRAp+qtMiHz/ GPd5T2lWgWoEBCzfobyyJ+9fHJ+RXVPGt54EnEQUtr1DNPgshtl3PV5UpT1iavItJT L1zFdGFHfZq6tq29dkaNazqNWo8eRUhFsaExO3XYCijG62LcgMn3AaW1dxsObP6IkN BTFF6SC3rJEk4+Z4sB/fHk4TiXjH2DJ4/IM6CcFoszPtGGpjCnCxp5l+N8bU3s0tzz hv69dVnou9DG3hWXTYuXXqECJ5zFLKAASxVwOIDmPUQQcyfVhjy From: "brian m. carlson" To: Cc: Jonathan Tan Subject: [PATCH 13/44] fetch-pack: detect when the server doesn't support our hash Date: Wed, 13 May 2020 00:53:53 +0000 Message-Id: <20200513005424.81369-14-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be In-Reply-To: <20200513005424.81369-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Detect when the server doesn't support our hash algorithm and abort. Signed-off-by: brian m. carlson --- fetch-pack.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/fetch-pack.c b/fetch-pack.c index f73a2ce6cb..1d277190e7 100644 --- a/fetch-pack.c +++ b/fetch-pack.c @@ -1039,6 +1039,8 @@ static struct ref *do_fetch_pack(struct fetch_pack_args *args, print_verbose(args, _("Server supports %s"), "deepen-relative"); else if (args->deepen_relative) die(_("Server does not support --deepen")); + if (!server_supports_hash(the_hash_algo->name, NULL)) + die(_("Server does not support this repository's object format")); if (!args->no_dependents) { mark_complete_and_common_ref(negotiator, args, &ref); From patchwork Wed May 13 00:53:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11544423 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E758B17EA for ; Wed, 13 May 2020 00:55:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D15742078C for ; Wed, 13 May 2020 00:55:21 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="U+5oXqTt" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732026AbgEMAzQ (ORCPT ); Tue, 12 May 2020 20:55:16 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38244 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732016AbgEMAzP (ORCPT ); Tue, 12 May 2020 20:55:15 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 98B4660D01; Wed, 13 May 2020 00:54:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1589331283; bh=gbBVk96z+lg6+eQy2KlTt0kXUkZxVoEsEzOwqyymPIY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=U+5oXqTto1zxEY9jiRjKL/TVaFuu2mlbant1AytegZbzwFvb930JL7OKHt2fnsyHN eKDF8TSlaZs2Zc/59hpTbcozub4zaBTiD6uA4JzSzDmphF0+WBR5z3iuyUDUsd7w17 C+Mokj6MIE+9Q01qyKOO6VQZd5zxLn34KTB4xwwFy+Yb8Bf8brDuQo9abGWQYN36t6 WHv+n0f7YRJaixVrsYmdX7Q4swZBTzFWp1LhPimrJZwQcD1YBmQBLz25LMBh8nbYdR 5Z0AycrOmfUJ+D+lTpQpo4LLjVFvvCXrCzRcbecedjmYHI08K4vkiXFg842+VypWWk H0tJh+UTdz6X4MnBjx9QD7LJEm4N8w3PnKtQ4lUu8FO2JexOVOSjPmRphYWwiw9n4W seNgjRf/xkIxwcxLmsh+e6rmNaYzmQQUTr49UepWAQXodU8tONptuwr609ONcJNt/S LFIXrQuKmRk6eOPv/m+et5cb0mgU6va8/H2DDwQMv3sSdUUs7X1 From: "brian m. carlson" To: Cc: Jonathan Tan Subject: [PATCH 14/44] connect: detect algorithm when fetching refs Date: Wed, 13 May 2020 00:53:54 +0000 Message-Id: <20200513005424.81369-15-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be In-Reply-To: <20200513005424.81369-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org If we're fetching refs, detect the hash algorithm and parse the refs using that algorithm. Signed-off-by: brian m. carlson --- connect.c | 20 ++++++++++++++++---- 1 file changed, 16 insertions(+), 4 deletions(-) diff --git a/connect.c b/connect.c index 511a069304..a39b843589 100644 --- a/connect.c +++ b/connect.c @@ -220,12 +220,24 @@ static void annotate_refs_with_symref_info(struct ref *ref) static void process_capabilities(struct packet_reader *reader, int *len) { + const char *feat_val; + int feat_len; + int hash_algo; const char *line = reader->line; int nul_location = strlen(line); if (nul_location == *len) return; server_capabilities_v1 = xstrdup(line + nul_location + 1); *len = nul_location; + + feat_val = server_feature_value("object-format", &feat_len); + if (feat_val) { + char *hash_name = xstrndup(feat_val, feat_len); + hash_algo = hash_algo_by_name(hash_name); + if (hash_algo != GIT_HASH_UNKNOWN) + reader->hash_algo = &hash_algos[hash_algo]; + free(hash_name); + } } static int process_dummy_ref(const struct packet_reader *reader) @@ -234,7 +246,7 @@ static int process_dummy_ref(const struct packet_reader *reader) struct object_id oid; const char *name; - if (parse_oid_hex(line, &oid, &name)) + if (parse_oid_hex_algop(line, &oid, &name, reader->hash_algo)) return 0; if (*name != ' ') return 0; @@ -258,7 +270,7 @@ static int process_ref(const struct packet_reader *reader, int len, struct object_id old_oid; const char *name; - if (parse_oid_hex(line, &old_oid, &name)) + if (parse_oid_hex_algop(line, &old_oid, &name, reader->hash_algo)) return 0; if (*name != ' ') return 0; @@ -270,7 +282,7 @@ static int process_ref(const struct packet_reader *reader, int len, die(_("protocol error: unexpected capabilities^{}")); } else if (check_ref(name, flags)) { struct ref *ref = alloc_ref(name); - oidcpy(&ref->old_oid, &old_oid); + memcpy(ref->old_oid.hash, old_oid.hash, reader->hash_algo->rawsz); **list = ref; *list = &ref->next; } @@ -288,7 +300,7 @@ static int process_shallow(const struct packet_reader *reader, int len, if (!skip_prefix(line, "shallow ", &arg)) return 0; - if (get_oid_hex(arg, &old_oid)) + if (get_oid_hex_algop(arg, &old_oid, reader->hash_algo)) die(_("protocol error: expected shallow sha-1, got '%s'"), arg); if (!shallow_points) die(_("repository on the other end cannot be shallow")); From patchwork Wed May 13 00:53:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11544375 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E47DE1668 for ; Wed, 13 May 2020 00:54:50 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C151520675 for ; Wed, 13 May 2020 00:54:50 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="uDPg+MEs" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731888AbgEMAyt (ORCPT ); Tue, 12 May 2020 20:54:49 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38140 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726031AbgEMAyp (ORCPT ); Tue, 12 May 2020 20:54:45 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 0C5EA60D03; Wed, 13 May 2020 00:54:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1589331284; bh=4MbzblCa/1wSvFdfUtTRCZg6R1LpC0SauaVEGykTrhY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=uDPg+MEsw96TeuAoLJS5swP/bV0ppoXUmhJalZp7raSNiPQNgkI9+EJhUvISNEONS 5nyo6H/5Bfe1XlfFHbK4JV6mipOwmJdrsQlmWEkYnw1oul5bsL8YARcs+e6Xob+hfu Se46HjD5n20p32YR3OmmXwiwFCZDJjo1Dp+J/VBV7RjjPv3KQyyc15Y1R620jgdRJm sjJl7xmHnaKwFE+2pvgzEOrjyvWSx9aL8MrIFPCgxJzjZnEReJdSGBJ6UibTmzxCEI icEaCRX/EL7k1VxpM592vcM3o2guMctxeDVJyBUJRpb/RxJFDeoBLghT2qovpc2XeU 4nAN5RW7Am7qyfT6fb6DX+eSSSTIYbwttGYkyMEf2aBNJUX1CJxkM+yGlXhl+MmTQv fJ7l5AELFqW3GD2ZuKaEKq8p05SEvbVWnX656WGmU67qp67+2jrpHn7TGWF3biIS2h YEDisHdET1ce+Z5mrnwdF8bKn/d16UOUmFLIdP8CZMilPbyXbcG From: "brian m. carlson" To: Cc: Jonathan Tan Subject: [PATCH 15/44] builtin/receive-pack: detect when the server doesn't support our hash Date: Wed, 13 May 2020 00:53:55 +0000 Message-Id: <20200513005424.81369-16-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be In-Reply-To: <20200513005424.81369-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Detect when the server doesn't support our hash algorithm and abort. Signed-off-by: brian m. carlson --- builtin/receive-pack.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c index a4159b559e..8755fa2463 100644 --- a/builtin/receive-pack.c +++ b/builtin/receive-pack.c @@ -1624,6 +1624,8 @@ static struct command *read_head_info(struct packet_reader *reader, linelen = strlen(reader->line); if (linelen < reader->pktlen) { const char *feature_list = reader->line + linelen + 1; + const char *hash; + int len = 0; if (parse_feature_request(feature_list, "report-status")) report_status = 1; if (parse_feature_request(feature_list, "side-band-64k")) @@ -1636,6 +1638,13 @@ static struct command *read_head_info(struct packet_reader *reader, if (advertise_push_options && parse_feature_request(feature_list, "push-options")) use_push_options = 1; + hash = parse_feature_value(feature_list, "object-format", &len, NULL); + if (!hash) { + hash = hash_algos[GIT_HASH_SHA1].name; + len = strlen(hash); + } + if (xstrncmpz(the_hash_algo->name, hash, len)) + die("error: unsupported object format '%s'", hash); } if (!strcmp(reader->line, "push-cert")) { From patchwork Wed May 13 00:53:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11544419 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8BFD11668 for ; Wed, 13 May 2020 00:55:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 742F120753 for ; Wed, 13 May 2020 00:55:19 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="xzlhDjBB" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732033AbgEMAzS (ORCPT ); Tue, 12 May 2020 20:55:18 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38252 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731991AbgEMAzQ (ORCPT ); Tue, 12 May 2020 20:55:16 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 7362660D04; Wed, 13 May 2020 00:54:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1589331284; bh=JYZIPaXyfRmBGoUSPO/BC8tfJm3FljXIDJUspJCISoE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=xzlhDjBBHau0DIA5SctEp9ht5Xv2Rxd/aLFHWZ9cmtM4ZdR3CxXH60qjymeVYlc0x HSMRWq7m+ln7PeuTHd5ukrCvkmv0zNDuLcG8vJ2cGFcLuLC5qeFK6CcZJ3h1+DDKzc fMzN0iOOi2WSbHNWHxDJG6AuvVM+RmcS0Bjzn+ZxsD9rYQ+hxHbkewFmSEZlF+rDNb RlQyX6rJapjWl5WuIi10DV+Li+C2OT6cesD3qXFaQ43LqwNqT2u8Ncy0P6XjPIYcP/ RPHQcC/XLPU4cJYfZKhJsUJn1K5IO3xCADh/kOqI0maNiBvpNhfrY7IJxo0553stJw tbKxEx36q1gDD1Wc9zGkFDMWmsqjmhA1+NCVbbV61JV/t27QxV2B9LxU2jp9s9Jm4/ hCV/jjwkSrrzAs+5nruHIEL6/lKxVBJo8Sz3pOqBWkB49wDg7UDtQjJWMulVSdd6sg 7okZxJasw/YvxDjDOJFy2c7TfSccEOu7/EnKgCR9cKrOKcQ1xhR From: "brian m. carlson" To: Cc: Jonathan Tan Subject: [PATCH 16/44] docs: update remote helper docs for object-format extensions Date: Wed, 13 May 2020 00:53:56 +0000 Message-Id: <20200513005424.81369-17-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be In-Reply-To: <20200513005424.81369-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Update the remote helper docs to document the object-format extensions we will implement in remote-curl and the transport helper code shortly. Signed-off-by: brian m. carlson --- Documentation/gitremote-helpers.txt | 33 +++++++++++++++++++++++++---- 1 file changed, 29 insertions(+), 4 deletions(-) diff --git a/Documentation/gitremote-helpers.txt b/Documentation/gitremote-helpers.txt index f48a031dc3..26f32e4421 100644 --- a/Documentation/gitremote-helpers.txt +++ b/Documentation/gitremote-helpers.txt @@ -238,6 +238,9 @@ the remote repository. `--signed-tags=verbatim` to linkgit:git-fast-export[1]. In the absence of this capability, Git will use `--signed-tags=warn-strip`. +'object-format':: + This indicates that the helper is able to interact with the remote + side using an explicit hash algorithm extension. COMMANDS @@ -257,12 +260,14 @@ Support for this command is mandatory. 'list':: Lists the refs, one per line, in the format " [ ...]". The value may be a hex sha1 hash, "@" for - a symref, or "?" to indicate that the helper could not get the - value of the ref. A space-separated list of attributes follows - the name; unrecognized attributes are ignored. The list ends - with a blank line. + a symref, ": " for a key-value pair, or + "?" to indicate that the helper could not get the value of the + ref. A space-separated list of attributes follows the name; + unrecognized attributes are ignored. The list ends with a + blank line. + See REF LIST ATTRIBUTES for a list of currently defined attributes. +See REF LIST KEYWORDS for a list of currently defined keywords. + Supported if the helper has the "fetch" or "import" capability. @@ -430,6 +435,18 @@ attributes are defined. This ref is unchanged since the last import or fetch, although the helper cannot necessarily determine what value that produced. +REF LIST KEYWORDS +----------------- + +The 'list' command may produce a list of key-value pairs. +The following keys are defined. + +'object-format':: + The refs are using the given hash algorithm. This keyword is only + used if the server and client both support the object-format + extension. + + OPTIONS ------- @@ -514,6 +531,14 @@ set by Git if the remote helper has the 'option' capability. transaction. If successful, all refs will be updated, or none will. If the remote side does not support this capability, the push will fail. +'option object-format' {'true'|algorithm}:: + If 'true', indicate that the caller wants hash algorithm information + to be passed back from the remote. This mode is used when fetching + refs. ++ +If set to an algorithm, indicate that the caller wants to interact with +the remote side using that algorithm. + SEE ALSO -------- linkgit:git-remote[1] From patchwork Wed May 13 00:53:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11544469 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EBA331668 for ; Wed, 13 May 2020 00:56:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D31B620753 for ; Wed, 13 May 2020 00:56:08 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="CrKiqQSl" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732167AbgEMA4I (ORCPT ); Tue, 12 May 2020 20:56:08 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38112 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731683AbgEMAyq (ORCPT ); Tue, 12 May 2020 20:54:46 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id DA5E860D05; Wed, 13 May 2020 00:54:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1589331285; bh=z/hbO9Pa9kw3i3ydW7bPReUQR2B2iix4X60keH+Bypw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=CrKiqQSlWSav9pkfdO2+y/n1EL+HgTAV8XsvVc5nfN6L/xQesNUqRKycSIT+oVLE8 JAAGGCBzzvHJdme/D2SHuomSx7wkoJDdpPuFcDynEqyf25JBWZ661Qk4fcxHctrKaH cIakhm/8aKSdjLNPa2PzXN5CEYgI9WWv63VW7n/QV8zBxdzoZ9JSI/Ub6DNSfuR/bV kWzksl60wcIPooBlg+kYSd0RBqNF8y1ROiVYUY1pnNRE5M+0a7JnqvepWc14rjHAPn tcm8cC86yH27hh3WjQ+56fcn4rZxHy5K5q/ZgoxIJfsxN3C7eRatsyDWGl4Lp56gbB 1bHFm131ZhJZfrAVzIqnQjoZXYYOgXrr1CpIZZepPqMd+INNU5D+Db9hhGRIeu4ASt PlYdTlOKSO3UlWwOS7o5b2FGhouJ9gnXYbS5uGUJ7kYY37xr3f61Whi96gT4ZBeElA rDrwaXp1mpcZwD0BcNOfPolDxTgQoaaNbZAi8dsAa3i8I+tVDDp From: "brian m. carlson" To: Cc: Jonathan Tan Subject: [PATCH 17/44] transport-helper: implement object-format extensions Date: Wed, 13 May 2020 00:53:57 +0000 Message-Id: <20200513005424.81369-18-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be In-Reply-To: <20200513005424.81369-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Implement the object-format extensions that let us determine the hash algorithm in use when pushing or pulling data. Signed-off-by: brian m. carlson --- transport-helper.c | 24 ++++++++++++++++++++++-- 1 file changed, 22 insertions(+), 2 deletions(-) diff --git a/transport-helper.c b/transport-helper.c index a46afcb69d..ae33b0eea7 100644 --- a/transport-helper.c +++ b/transport-helper.c @@ -32,7 +32,8 @@ struct helper_data { signed_tags : 1, check_connectivity : 1, no_disconnect_req : 1, - no_private_update : 1; + no_private_update : 1, + object_format : 1; /* * As an optimization, the transport code may invoke fetch before @@ -207,6 +208,8 @@ static struct child_process *get_helper(struct transport *transport) data->import_marks = xstrdup(arg); } else if (starts_with(capname, "no-private-update")) { data->no_private_update = 1; + } else if (starts_with(capname, "object-format")) { + data->object_format = 1; } else if (mandatory) { die(_("unknown mandatory capability %s; this remote " "helper probably needs newer version of Git"), @@ -1103,6 +1106,12 @@ static struct ref *get_refs_list_using_list(struct transport *transport, data->get_refs_list_called = 1; helper = get_helper(transport); + if (data->object_format) { + write_str_in_full(helper->in, "option object-format\n"); + if (recvline(data, &buf) || strcmp(buf.buf, "ok")) + exit(128); + } + if (data->push && for_push) write_str_in_full(helper->in, "list for-push\n"); else @@ -1115,6 +1124,17 @@ static struct ref *get_refs_list_using_list(struct transport *transport, if (!*buf.buf) break; + else if (buf.buf[0] == ':') { + const char *value; + if (skip_prefix(buf.buf, ":object-format ", &value)) { + int algo = hash_algo_by_name(value); + if (algo == GIT_HASH_UNKNOWN) + die(_("unsupported object format '%s'"), + value); + transport->hash_algo = &hash_algos[algo]; + } + continue; + } eov = strchr(buf.buf, ' '); if (!eov) @@ -1127,7 +1147,7 @@ static struct ref *get_refs_list_using_list(struct transport *transport, if (buf.buf[0] == '@') (*tail)->symref = xstrdup(buf.buf + 1); else if (buf.buf[0] != '?') - get_oid_hex(buf.buf, &(*tail)->old_oid); + get_oid_hex_algop(buf.buf, &(*tail)->old_oid, transport->hash_algo); if (eon) { if (has_attribute(eon + 1, "unchanged")) { (*tail)->status |= REF_STATUS_UPTODATE; From patchwork Wed May 13 00:53:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11544391 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 11AF617EA for ; Wed, 13 May 2020 00:55:01 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EF3C72078C for ; Wed, 13 May 2020 00:55:00 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="XUjmdqnj" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731958AbgEMAzA (ORCPT ); Tue, 12 May 2020 20:55:00 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38104 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731837AbgEMAyr (ORCPT ); Tue, 12 May 2020 20:54:47 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 4E66F60D07; Wed, 13 May 2020 00:54:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1589331285; bh=5OqbgbgT9Ub0EVedjs2/QCeU8Qm807ZRLrM0Pa/VUQY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=XUjmdqnj+eXCYwXOxxtAk5gcYVOaxdr/0lo3Bi63PpIUtKwV8JBHcfZkOASGj8ZUj 3NBy6WkfXjDWGO3XOyozmLPlv/UGpY7Xw900C0cp5N39kTR8NFoKFJRL4ZLOjzDUBf 5l+Jfp6R3Kc8kr7nTKSyn33FF38CGRfvqPN1thLhZQdQnFT1TlmurFowvEn/MzY96b iP5ztLFyFUABrpos6u2/mRnlxyANELvlQuZjxUeE2oD+Z//EMw0V4mclHjYWLpDIuG V0HCaYTYUl3u/2WdEjIfjEpvDBvKETyP4otqRh8t2yHgU6SHZLZzqcFxmradZ398Li 1hs/vyAzX/bEA1/YcBw6pTiEjRqFj0gZLs3ZFrpCK0Y+I4nzQbo/Z+aGUr6dNYEkjI 1OQOGUw9JCR4GbYyaWRj8l1wkVpcwJzhHAggpYBqUpxiDG7vIa2Qy8BBgm9m9ebvLJ Sfn2pC6Yd7+u19pIgqogK11UV0x/e4Ced1KUplTjMfo77Yby+Bi From: "brian m. carlson" To: Cc: Jonathan Tan Subject: [PATCH 18/44] remote-curl: implement object-format extensions Date: Wed, 13 May 2020 00:53:58 +0000 Message-Id: <20200513005424.81369-19-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be In-Reply-To: <20200513005424.81369-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Implement the object-format extensions that let us determine the hash algorithm in use when pushing, pulling, and fetching. Signed-off-by: brian m. carlson --- remote-curl.c | 19 ++++++++++++++++++- 1 file changed, 18 insertions(+), 1 deletion(-) diff --git a/remote-curl.c b/remote-curl.c index 1c9aa3d0ab..3ed0dfec1b 100644 --- a/remote-curl.c +++ b/remote-curl.c @@ -41,7 +41,9 @@ struct options { deepen_relative : 1, from_promisor : 1, no_dependents : 1, - atomic : 1; + atomic : 1, + object_format : 1; + const struct git_hash_algo *hash_algo; }; static struct options options; static struct string_list cas_options = STRING_LIST_INIT_DUP; @@ -190,6 +192,16 @@ static int set_option(const char *name, const char *value) } else if (!strcmp(name, "filter")) { options.filter = xstrdup(value); return 0; + } else if (!strcmp(name, "object-format")) { + int algo; + options.object_format = 1; + if (strcmp(value, "true")) { + algo = hash_algo_by_name(value); + if (algo == GIT_HASH_UNKNOWN) + die("unknown object format '%s'", value); + options.hash_algo = &hash_algos[algo]; + } + return 0; } else { return 1 /* unsupported */; } @@ -231,6 +243,7 @@ static struct ref *parse_git_refs(struct discovery *heads, int for_push) case protocol_v0: get_remote_heads(&reader, &list, for_push ? REF_NORMAL : 0, NULL, &heads->shallow); + options.hash_algo = reader.hash_algo; break; case protocol_unknown_version: BUG("unknown protocol version"); @@ -509,6 +522,9 @@ static struct ref *get_refs(int for_push) static void output_refs(struct ref *refs) { struct ref *posn; + if (options.object_format && options.hash_algo) { + printf(":object-format %s\n", options.hash_algo->name); + } for (posn = refs; posn; posn = posn->next) { if (posn->symref) printf("@%s %s\n", posn->symref, posn->name); @@ -1439,6 +1455,7 @@ int cmd_main(int argc, const char **argv) printf("option\n"); printf("push\n"); printf("check-connectivity\n"); + printf("object-format\n"); printf("\n"); fflush(stdout); } else if (skip_prefix(buf.buf, "stateless-connect ", &arg)) { From patchwork Wed May 13 00:53:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11544421 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 50C8090 for ; Wed, 13 May 2020 00:55:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 310162078C for ; Wed, 13 May 2020 00:55:21 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="OQFv4vNP" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731991AbgEMAzT (ORCPT ); Tue, 12 May 2020 20:55:19 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38260 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732025AbgEMAzR (ORCPT ); Tue, 12 May 2020 20:55:17 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id B544160D08; Wed, 13 May 2020 00:54:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1589331286; bh=sxSDhOXns0vesz2DtA7y4piK/iRpR30k7wtrsZNC1VU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=OQFv4vNP8ccNz5sTdW5vVDOO33vd8MXwPFPNqOO89DqE4sFAcY1x1XheojgES2DAR I9PARfDSkQhWqzTx5AvI+DRBtq0I6eESmBk2lBY0TkjwahUgcwSlqRkKrvqnSMMSRb mmogPIiBGvlhE9SmAJkso8vxpdIgYXUfbAc75u09WQ21bSIMbpiTptXYoReuDOFY/C nbRrLeNuYofQPfK4fCAKxdRNwrGUwdXXcI9gmzUmCASkxWIDRBwO3SNLtcO8jQmOHO puRTosEe/jyEQQttbOl1waEf3+1YQWeRMjAxO2UpR4uUtJ/GaRRWaiR+yUBbs2CQtu mdBCp7geEqdZORAWD7zIZCXD9x/JQKCjgkaRHKIdVtzPWgIKH+PffSI7Nk5Ol+Il/d nSio2v6UpejDb3UqMJGvX/I7ixDTHdy2Ebn+D0Lj7nbYV7dqHaglE9A3WFe4gtDlcC ANcMeSlmmdVqxkxttHj2qRODbpOLUVz453gyPR5IcprRShiPomX From: "brian m. carlson" To: Cc: Jonathan Tan Subject: [PATCH 19/44] builtin/clone: initialize hash algorithm properly Date: Wed, 13 May 2020 00:53:59 +0000 Message-Id: <20200513005424.81369-20-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be In-Reply-To: <20200513005424.81369-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When performing a clone, we don't know what hash algorithm the other end will support. Currently, we don't support fetching data belonging to a different algorithm, so we must know what algorithm the remote side is using in order to properly initialize the repository. We can know that only after fetching the refs, so if the remote side has any references, use that information to reinitialize the repository with the correct hash algorithm information. Signed-off-by: brian m. carlson --- builtin/clone.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/builtin/clone.c b/builtin/clone.c index cb48a291ca..f27d38bc8e 100644 --- a/builtin/clone.c +++ b/builtin/clone.c @@ -1217,6 +1217,15 @@ int cmd_clone(int argc, const char **argv, const char *prefix) refs = transport_get_remote_refs(transport, &ref_prefixes); if (refs) { + int hash_algo = hash_algo_by_ptr(transport_get_hash_algo(transport)); + + /* + * Now that we know what algorithm the remote side is using, + * let's set ours to the same thing. + */ + initialize_repository_version(hash_algo); + repo_set_hash_algo(the_repository, hash_algo); + mapped_refs = wanted_peer_refs(refs, &remote->fetch); /* * transport_get_remote_refs() may return refs with null sha-1 From patchwork Wed May 13 00:54:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11544401 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C0CC390 for ; Wed, 13 May 2020 00:55:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AA5B620753 for ; Wed, 13 May 2020 00:55:06 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="Z5/gSQx7" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731948AbgEMAy6 (ORCPT ); Tue, 12 May 2020 20:54:58 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38156 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731842AbgEMAyr (ORCPT ); Tue, 12 May 2020 20:54:47 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 280A6609D3; Wed, 13 May 2020 00:54:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1589331286; bh=6rJkrrv0VdfZ0C61oMUrkMGcnJ88EB6+2EVJhWj50Pg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=Z5/gSQx7pC5rrfKJfyBoT4NpzQhRyFwY9YGplJVEkd9nndQ8QX3M73nlzP0q1185l 7QJhLEvZ9XWHzLNUt3t7sKk0dSg+zZfYAYzjHdGcC73ALQRNJoP6++WjxgQ1D6RhFr DvpJPkEyV88dsfEgRm8jQKt9mTrWwX9KY8ImmpyNfXmQTZF2sPWajClusYeyYgbnFo /DbHvH3MZIcJLi2DLa+8nnMYbb3JLf2FsTMBWQFj0U/QAVndOg3vC7Ij6s3e0U1Wjs no8UJ3nrvYd7vzq/8CXNoCI8hBtWlhCEpEC4T3XEBll5GJ9S4VzIolx6ou8xAudlf+ YmffVtBZ2VFWLCT5BBDFVqJbu0IK0id9q5fPY/VfGkWt3LBkArY672sxb8Z3xihYAl R90HyLO0pvK9l1S4bAHmYLmek87GOg8+Z2EjetePC4D5r3oz1JImQu/wf5mxSh5Uwc 4chL2L2T7SNDy+1gzuoMrzQhH8gq2VrvkVvgROD9sPGl43JNf4F From: "brian m. carlson" To: Cc: Jonathan Tan Subject: [PATCH 20/44] t5562: pass object-format in synthesized test data Date: Wed, 13 May 2020 00:54:00 +0000 Message-Id: <20200513005424.81369-21-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be In-Reply-To: <20200513005424.81369-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Ensure that we pass the object-format capability in the synthesized test data so that this test works with algorithms other than SHA-1. In addition, add an additional test using the old data for when we're using SHA-1 so that we can be sure that we preserve backwards compatibility with servers not offering the object-format capability. Signed-off-by: brian m. carlson --- t/t5562-http-backend-content-length.sh | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/t/t5562-http-backend-content-length.sh b/t/t5562-http-backend-content-length.sh index 3f4ac71f83..f508d4d449 100755 --- a/t/t5562-http-backend-content-length.sh +++ b/t/t5562-http-backend-content-length.sh @@ -46,6 +46,7 @@ ssize_b100dots() { } test_expect_success 'setup' ' + test_oid_init && HTTP_CONTENT_ENCODING="identity" && export HTTP_CONTENT_ENCODING && git config http.receivepack true && @@ -62,8 +63,8 @@ test_expect_success 'setup' ' test_copy_bytes 10 fetch_body.trunc && hash_next=$(git commit-tree -p HEAD -m next HEAD^{tree}) && { - printf "%s %s refs/heads/newbranch\\0report-status\\n" \ - "$ZERO_OID" "$hash_next" | packetize && + printf "%s %s refs/heads/newbranch\\0report-status object-format=%s\\n" \ + "$ZERO_OID" "$hash_next" "$(test_oid algo)" | packetize && printf 0000 && echo "$hash_next" | git pack-objects --stdout } >push_body && @@ -117,6 +118,15 @@ test_expect_success GZIP 'push plain' ' test_cmp act.head exp.head ' +test_expect_success GZIP 'push plain with SHA-1' ' + test_when_finished "git branch -D newbranch" && + test_http_env receive push_body && + verify_http_result "200 OK" && + git rev-parse newbranch >act.head && + echo "$hash_next" >exp.head && + test_cmp act.head exp.head +' + test_expect_success 'push plain truncated' ' test_http_env receive push_body.trunc && ! verify_http_result "200 OK" From patchwork Wed May 13 00:54:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11544389 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AC02E90 for ; Wed, 13 May 2020 00:55:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8648E2078C for ; Wed, 13 May 2020 00:55:00 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="mCuYul1N" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731957AbgEMAy7 (ORCPT ); Tue, 12 May 2020 20:54:59 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38118 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731856AbgEMAys (ORCPT ); Tue, 12 May 2020 20:54:48 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 8E39B60CFE; Wed, 13 May 2020 00:54:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1589331286; bh=5MOVjLFFh3lkhlysaSMSoRifk6+bxVZ1Bm4EklDEL8g=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=mCuYul1Na2NtRq6BKtJU7g5vy9mWq+zZifdvv7RuBSEMKwoNw79FWaDTERZNzzHC1 QTiy/feJc7slFHvGTPEsW25xLP8z+uCh2GY37ILQDFUXZjVUH2BdbaxhXIt2k0ps+A tY36zBTo9QVnNRIzdpbO7EX+S0iIhm7ks/7160Ms/LY4+9yYB0bvfxihCBOxXBOccm NcdaA6yNMHijO35WR/zPOsbIEnDsg9zfiQWaJaFeb5Qm1fwpQAHIMt0Txdsv28FLkv OJWSweU366OITaBrN54oIsPcLz9HjQioqkZ5YFZRs7CeJxDU5qnLFOR8OxDAtN+YBc N5uYU1sZ4geLgnNntIxuGaJ3S/v91JQ1z7rGbsP5pKUAN/7wTi/I8QiCXIaBYQ8Dov n9nQfXg2u8b+HROWr/DMGkZM/Tk+l/+ZzSSk+DXOzPssakyCkjLyyFig1+Nj/LeCD2 TrnPiwSmrYX52C304yMceTknzmtjH83Ufkkte0us6aWlJqwwLOE From: "brian m. carlson" To: Cc: Jonathan Tan Subject: [PATCH 21/44] t5704: send object-format capability with SHA-256 Date: Wed, 13 May 2020 00:54:01 +0000 Message-Id: <20200513005424.81369-22-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be In-Reply-To: <20200513005424.81369-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When we speak protocol v2 in this test, we must pass the object-format header if the algorithm is not SHA-1. Otherwise, git upload-pack fails because the hash algorithm doesn't match and not because we've failed to speak the protocol correctly. Pass the header so that our assertions test what we're really interested in. Signed-off-by: brian m. carlson --- t/t5704-protocol-violations.sh | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/t/t5704-protocol-violations.sh b/t/t5704-protocol-violations.sh index 950cfb21fe..47e78932b9 100755 --- a/t/t5704-protocol-violations.sh +++ b/t/t5704-protocol-violations.sh @@ -6,9 +6,20 @@ communications if the other side says something unexpected. We are mostly making sure that we do not segfault or otherwise behave badly.' . ./test-lib.sh +# If we don't print the object format, we'll fail for a spurious reason: the +# mismatched object format. +print_object_format () { + local algo=$(test_oid algo) && + if test "$algo" != "sha1" + then + packetize "object-format=$algo" + fi +} + test_expect_success 'extra delim packet in v2 ls-refs args' ' { packetize command=ls-refs && + print_object_format && printf 0001 && # protocol expects 0000 flush here printf 0001 @@ -21,6 +32,7 @@ test_expect_success 'extra delim packet in v2 ls-refs args' ' test_expect_success 'extra delim packet in v2 fetch args' ' { packetize command=fetch && + print_object_format && printf 0001 && # protocol expects 0000 flush here printf 0001 From patchwork Wed May 13 00:54:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11544385 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 92E261668 for ; Wed, 13 May 2020 00:54:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 722462078C for ; Wed, 13 May 2020 00:54:56 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="lEYGdzei" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731926AbgEMAyz (ORCPT ); Tue, 12 May 2020 20:54:55 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38132 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731857AbgEMAys (ORCPT ); Tue, 12 May 2020 20:54:48 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 00A7360D00; Wed, 13 May 2020 00:54:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1589331287; bh=H2xDoXKjs7NgXARqu+L/reTeQW0Z4AxMrSZY7HFrlT8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=lEYGdzeiRYX5tNECTxlE9Tz2yTI5uulSStvvQ+g26HqG9XhZpAhY99Q/qFyRuRPEs Ut30/tqi+yBiWKJw6QiYzPtcIbVAJYZu/Cs1QKcgzbBe72oEhJWqCMKmWEd7f0KjlJ 7yNYxUBC0mcanfAEObb0hKTGQ4VoJ1qUeGqusbT2K7QRHRStj62WD8Z95RrDNUd1PO XfsRwiaYsl66Gk/ia0X8WXS7bpOOv0vR2eFLwRatSRcR1+CcwBR62OPqKvSBvyU7y0 5oRNDSo0Wd/6u6MOss2FmbBOVPdLY0rnAfcOuhOQcVa2KKZrojHQp/D1khcI0H66+Y Wh4TdcHwSYZkbxsfvOlnOrzo695SFFaZ0V1eaew0wQMGdaCclLHstiYQvIA2bnrtNd 5c01IXzV/DiI+FFQNBuo68vdSeib3Q4avFSPdhFP69v2I3PxD/YIaRXoGgrqjiWEQA gd2Ffaa2gjs+a8G7yOx9iz0hp3/MMuBameVuJYnjGDpnX7eccin From: "brian m. carlson" To: Cc: Jonathan Tan Subject: [PATCH 22/44] fetch-pack: parse and advertise the object-format capability Date: Wed, 13 May 2020 00:54:02 +0000 Message-Id: <20200513005424.81369-23-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be In-Reply-To: <20200513005424.81369-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Parse the server's object-format capability and respond accordingly, dying if there is a mismatch. Signed-off-by: brian m. carlson --- fetch-pack.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/fetch-pack.c b/fetch-pack.c index 1d277190e7..3a48ed4b13 100644 --- a/fetch-pack.c +++ b/fetch-pack.c @@ -1179,6 +1179,7 @@ static int send_fetch_request(struct fetch_negotiator *negotiator, int fd_out, int sideband_all, int seen_ack) { int ret = 0; + const char *hash_name; struct strbuf req_buf = STRBUF_INIT; if (server_supports_v2("fetch", 1)) @@ -1193,6 +1194,17 @@ static int send_fetch_request(struct fetch_negotiator *negotiator, int fd_out, args->server_options->items[i].string); } + if (server_feature_v2("object-format", &hash_name)) { + int hash_algo = hash_algo_by_name(hash_name); + if (hash_algo_by_ptr(the_hash_algo) != hash_algo) + die(_("mismatched algorithms: client %s; server %s"), + the_hash_algo->name, hash_name); + packet_write_fmt(fd_out, "object-format=%s", the_hash_algo->name); + } + else if (hash_algo_by_ptr(the_hash_algo) != GIT_HASH_SHA1) + die(_("the server does not support algorithm '%s'"), + the_hash_algo->name); + packet_buf_delim(&req_buf); if (args->use_thin_pack) packet_buf_write(&req_buf, "thin-pack"); From patchwork Wed May 13 00:54:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11544387 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 57F631668 for ; Wed, 13 May 2020 00:54:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 37F3120753 for ; Wed, 13 May 2020 00:54:58 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="j1474I/s" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731940AbgEMAy4 (ORCPT ); Tue, 12 May 2020 20:54:56 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38140 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731871AbgEMAys (ORCPT ); Tue, 12 May 2020 20:54:48 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 6936760D02; Wed, 13 May 2020 00:54:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1589331287; bh=BfA/BVldsKduzgAB919RC6B20stnCTVSH4nJ7cJBXRQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=j1474I/sgiG9xck7CNGselBaAdzAbjQWwRRTyAYI/Cj1E5BXJbPYuTJnQbTFmGG6f 8UaNL5zfs3J7zJ3Dp8mq+CBkoVE76N9bUkZmpVC1CGxsMXNaI7gidpmU87mcNn0VXO qLeiN/j/Zy07w12NpfYNSmtzB9tz6JkoVJyDPRVoLZzPBchm/A87SWQZf7eIESV1tR IhnFbNzC3St7pwSE88yP4c4rKVPb7v+Xu8y5n4wXOok3Xe1p6RYkRTe5LzzRKG5BPW 83K70bGyvhTvQ0BWkg4ucTd2ren9IrFkLeHEpqFkdtJe3GrXZsvna+MnsgLeCyRbyk 8BFyuWs5gsFHRi7+e6/sodCpyLCewId8wrkBu6P825S4FKNjGu2K3EGv+XG3XX3hXr pYTKvaSXQEiTNnVprdPQ9s/Zt7TqIfjoJHMUioDp/d54JDTl0bOofU4uEBxiNjBqyB 10kvBp28IkLTXjC1Bq4Yf05u2jDQdX6ZkscOXpSTqZe2Rimvovi From: "brian m. carlson" To: Cc: Jonathan Tan Subject: [PATCH 23/44] setup: set the_repository's hash algo when checking format Date: Wed, 13 May 2020 00:54:03 +0000 Message-Id: <20200513005424.81369-24-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be In-Reply-To: <20200513005424.81369-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When we're checking the repository's format, set the hash algorithm at the same time. This ensures that we perform a suitable initialization early enough to avoid confusing any parts of the code. If we defer until later, we can end up with portions of the code which are confused about the hash algorithm, resulting in segfaults. Signed-off-by: brian m. carlson --- setup.c | 1 + 1 file changed, 1 insertion(+) diff --git a/setup.c b/setup.c index 65fe5ecefb..019a1c6367 100644 --- a/setup.c +++ b/setup.c @@ -1273,6 +1273,7 @@ void check_repository_format(struct repository_format *fmt) fmt = &repo_fmt; check_repository_format_gently(get_git_dir(), fmt, NULL); startup_info->have_repository = 1; + repo_set_hash_algo(the_repository, fmt->hash_algo); clear_repository_format(&repo_fmt); } From patchwork Wed May 13 00:54:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11544383 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 99DE990 for ; Wed, 13 May 2020 00:54:54 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7867A20675 for ; Wed, 13 May 2020 00:54:54 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="kbb6rto2" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731918AbgEMAyx (ORCPT ); Tue, 12 May 2020 20:54:53 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38158 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731882AbgEMAyt (ORCPT ); Tue, 12 May 2020 20:54:49 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id E1C3660D09; Wed, 13 May 2020 00:54:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1589331288; bh=MMfJEd2BJ7WjAZeGiNm4voySOTAFCGpDlJvsq+dHUZY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=kbb6rto2m9vTit12xDHRzkqNRzYtHtxeMCjk1UF76REQWpt/lWxWXjtYH2dHHXGEW vWO8CBliEKve0HvAu6jg9beKxA52MKC5aeVfu6+GbM+lXAsraWkoRBHlk2MHIxwtHE l79bRrfAW3VriojOXdjjsBR61dFkD0YatAdXIJ3noRRp9Ih3hSUZMy3cr3WdybhA3I WaeOQDXvfiZia9ybzdky9KAuuzBxW7AfrN+XUZEGEp5MzovKi6xP53rNYr3XZ686tP YGi4GAC6ECeYQ5QoX0jG2P3z2Mx8spUXtbNCGy+tb+BD5q9SWTDnnMUhPrqhznW8jv 3qo4JCNl5TLNEAbZXWblPL1thDupHeJlfzMKa+rPicD/8R1VZzEcwVR/jRiyWoLs7X GBeNzkmnYarEpjeUaQ0w1VWneOeeeWWap6SGOvO4jJ64Pg8Vns50/tlgXjdKZgIAkC 0dPl6e9wt5b34gGuWnRHiGZtQMhb9or77VFd+iy29I+31/4obL7 From: "brian m. carlson" To: Cc: Jonathan Tan Subject: [PATCH 24/44] t3200: mark assertion with SHA1 prerequisite Date: Wed, 13 May 2020 00:54:04 +0000 Message-Id: <20200513005424.81369-25-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be In-Reply-To: <20200513005424.81369-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org One of the test assertions in this test checks that git branch -m works even without a .git/config file. However, if the repository requires configuration extensions, such as because it uses a non-SHA-1 algorithm, this assertion will fail. Mark the assertion as requiring SHA-1. Signed-off-by: brian m. carlson --- t/t3200-branch.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/t/t3200-branch.sh b/t/t3200-branch.sh index 411a70b0ce..2a3fedc6b0 100755 --- a/t/t3200-branch.sh +++ b/t/t3200-branch.sh @@ -402,7 +402,7 @@ EOF mv .git/config .git/config-saved -test_expect_success 'git branch -m q q2 without config should succeed' ' +test_expect_success SHA1 'git branch -m q q2 without config should succeed' ' git branch -m q q2 && git branch -m q2 q ' From patchwork Wed May 13 00:54:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11544381 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5790C1668 for ; Wed, 13 May 2020 00:54:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 416BB20753 for ; Wed, 13 May 2020 00:54:53 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="l9+cl7fw" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731914AbgEMAyw (ORCPT ); Tue, 12 May 2020 20:54:52 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38160 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731885AbgEMAyu (ORCPT ); Tue, 12 May 2020 20:54:50 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 6A2B360D0A; Wed, 13 May 2020 00:54:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1589331288; bh=+4Z4pf0T9fzuVPGr+GKKdEY7iEgxxGa0MzEFtugw7Z8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=l9+cl7fwO7Pu2ns8X476W3uM0sVd+2NSCPqudlWFGl2Asja27jyGrgH81qEK8ndYb 1dpTp2iJZBqqoMeWKdOVjkaG7tnJ6yhya5YqgaRuMpa6Us9eLLsZ9KClOpJzVEKCR1 fWUJSWeaR+htLaULVVy5ApZ4IoPz8TG8o+o+C5BnBgctuBmq6xhrFRvi14fytc8eSo p2uaM0tdS2sQ8Hs10WB+P8ijgZV9Pp9OZUCWMTm5WlQLZ3oGwJ73VpYF8cShnKWvxW Q3E2Fi4tpRwjpE3gilx5wB4GkqFOqryBY6ERywgiULR+5Vza87tstWZN8Oz0aHlPwz cig0QC13ZXdmQZv1AQP7SPGR+NqdESyC7J1qrYI4jNjYZPC4CKi3Q9UwR6LGYuiS/Z CZOpV/yVf4GymD0RoEvGBq+4Bdn4Y23eeAg9OTn4Rfa8uWtddaChswXymQAorwwzDz pBbfyNXlXvNnYUL+AT1CFmDVlIr10Nrq0vObA6zuIkeTV4AiSab From: "brian m. carlson" To: Cc: Jonathan Tan Subject: [PATCH 25/44] packfile: compute and use the index CRC offset Date: Wed, 13 May 2020 00:54:05 +0000 Message-Id: <20200513005424.81369-26-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be In-Reply-To: <20200513005424.81369-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Both v2 pack index files and the v3 format specified as part of the NewHash work have similar data starting at the CRC table. Much of the existing code wants to read either this table or the offset entries following it, and in doing so computes the offset each time. In order to share as much code between v2 and v3, compute the offset of the CRC table and store it when the pack is opened. Use this value to compute offsets to not only the CRC table, but to the offset entries beyond it. Signed-off-by: brian m. carlson --- builtin/index-pack.c | 6 +----- object-store.h | 1 + packfile.c | 1 + 3 files changed, 3 insertions(+), 5 deletions(-) diff --git a/builtin/index-pack.c b/builtin/index-pack.c index f176dd28c8..7bea1fba52 100644 --- a/builtin/index-pack.c +++ b/builtin/index-pack.c @@ -1555,13 +1555,9 @@ static void read_v2_anomalous_offsets(struct packed_git *p, { const uint32_t *idx1, *idx2; uint32_t i; - const uint32_t hashwords = the_hash_algo->rawsz / sizeof(uint32_t); /* The address of the 4-byte offset table */ - idx1 = (((const uint32_t *)p->index_data) - + 2 /* 8-byte header */ - + 256 /* fan out */ - + hashwords * p->num_objects /* object ID table */ + idx1 = (((const uint32_t *)((const uint8_t *)p->index_data + p->crc_offset)) + p->num_objects /* CRC32 table */ ); diff --git a/object-store.h b/object-store.h index d1e490f203..f439d47af8 100644 --- a/object-store.h +++ b/object-store.h @@ -70,6 +70,7 @@ struct packed_git { size_t index_size; uint32_t num_objects; uint32_t num_bad_objects; + uint32_t crc_offset; unsigned char *bad_object_sha1; int index_version; time_t mtime; diff --git a/packfile.c b/packfile.c index f4e752996d..6ab5233613 100644 --- a/packfile.c +++ b/packfile.c @@ -178,6 +178,7 @@ int load_idx(const char *path, const unsigned int hashsz, void *idx_map, */ (sizeof(off_t) <= 4)) return error("pack too large for current definition of off_t in %s", path); + p->crc_offset = 8 + 4 * 256 + nr * hashsz; } p->index_version = version; From patchwork Wed May 13 00:54:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11544465 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 088C11668 for ; Wed, 13 May 2020 00:56:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DA1A720753 for ; Wed, 13 May 2020 00:56:03 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="loSlJQAS" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732163AbgEMA4C (ORCPT ); Tue, 12 May 2020 20:56:02 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38162 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731889AbgEMAyv (ORCPT ); Tue, 12 May 2020 20:54:51 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id D37F760D0B; Wed, 13 May 2020 00:54:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1589331289; bh=pKT7D8FfW9+6hGmCBXYrcHQTegDW9wGONuHCaQuqves=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=loSlJQASWO+gIaQpxCDzN40XrR+dARuZu0LJbzhRl2jSCusu0bnr6mQ3fwKLL7Cu0 ZhYrKt18Z/FE0ilo0csjJdFWzWjFZ1JmralkngHwJN1mBiTIhOTZSR1YHQBufQU2a5 2LRa3JCY7co4FIaTcZDepKd1Hdq3ZBFrzArU4AldwBsYAWw2XE4x2+kbSyOipZd4TU jxoZH0WWXAxed+hC+wywJIis0b45Zc4JS7unvkwlQjErRLj/xhLTH87rHn8jRzB98c zyAAJ/KRKoN5K4ZNjtbplVINzduSfCkVfo/WuvpVUJfXEzBD12lX50RQwBXIyZsAA6 bhRqipgLfchg8+o9CggZlJT3n0dBU/poEaT58zy0yqHcGts03uN1YEzVfZwHPi4uuY GBgmsGNN8UG4MW7TH3yPkzDauL6y5r1OMtbkEt0GAD24wRXjWtf/UGbWGYO+gQ35NN e2kvwrFbCIxEcxbG9XF70ODciCjcaWZc+a8Rw9vJCZq7vPxSon3 From: "brian m. carlson" To: Cc: Jonathan Tan Subject: [PATCH 26/44] t5302: modernize test formatting Date: Wed, 13 May 2020 00:54:06 +0000 Message-Id: <20200513005424.81369-27-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be In-Reply-To: <20200513005424.81369-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Our style these days is to place the description and the opening quote of the body on the same line as test_expect_success (if it fits), to place the trailing quote on a line by itself after the body, and to use tabs. Since we're going to be making several significant changes to this test, modernize the style to aid in readability of the subsequent patches. This patch should have no functional change. Signed-off-by: brian m. carlson --- t/t5302-pack-index.sh | 360 +++++++++++++++++++++--------------------- 1 file changed, 184 insertions(+), 176 deletions(-) diff --git a/t/t5302-pack-index.sh b/t/t5302-pack-index.sh index ad07f2f7fc..8981c9b90e 100755 --- a/t/t5302-pack-index.sh +++ b/t/t5302-pack-index.sh @@ -7,65 +7,65 @@ test_description='pack index with 64-bit offsets and object CRC' . ./test-lib.sh test_expect_success 'setup' ' - test_oid_init && - rawsz=$(test_oid rawsz) && - rm -rf .git && - git init && - git config pack.threads 1 && - i=1 && - while test $i -le 100 - do - iii=$(printf '%03i' $i) - test-tool genrandom "bar" 200 > wide_delta_$iii && - test-tool genrandom "baz $iii" 50 >> wide_delta_$iii && - test-tool genrandom "foo"$i 100 > deep_delta_$iii && - test-tool genrandom "foo"$(expr $i + 1) 100 >> deep_delta_$iii && - test-tool genrandom "foo"$(expr $i + 2) 100 >> deep_delta_$iii && - echo $iii >file_$iii && - test-tool genrandom "$iii" 8192 >>file_$iii && - git update-index --add file_$iii deep_delta_$iii wide_delta_$iii && - i=$(expr $i + 1) || return 1 - done && - { echo 101 && test-tool genrandom 100 8192; } >file_101 && - git update-index --add file_101 && - tree=$(git write-tree) && - commit=$(git commit-tree $tree obj-list && - git update-ref HEAD $commit + test_oid_init && + rawsz=$(test_oid rawsz) && + rm -rf .git && + git init && + git config pack.threads 1 && + i=1 && + while test $i -le 100 + do + iii=$(printf '%03i' $i) + test-tool genrandom "bar" 200 > wide_delta_$iii && + test-tool genrandom "baz $iii" 50 >> wide_delta_$iii && + test-tool genrandom "foo"$i 100 > deep_delta_$iii && + test-tool genrandom "foo"$(expr $i + 1) 100 >> deep_delta_$iii && + test-tool genrandom "foo"$(expr $i + 2) 100 >> deep_delta_$iii && + echo $iii >file_$iii && + test-tool genrandom "$iii" 8192 >>file_$iii && + git update-index --add file_$iii deep_delta_$iii wide_delta_$iii && + i=$(expr $i + 1) || return 1 + done && + { echo 101 && test-tool genrandom 100 8192; } >file_101 && + git update-index --add file_101 && + tree=$(git write-tree) && + commit=$(git commit-tree $tree obj-list && + git update-ref HEAD $commit ' -test_expect_success \ - 'pack-objects with index version 1' \ - 'pack1=$(git pack-objects --index-version=1 test-1 &1) || ! (echo "$msg" | grep "pack too large .* off_t") @@ -91,21 +91,21 @@ else say "# skipping tests concerning 64-bit offsets" fi -test_expect_success OFF64_T \ - 'index v2: verify a pack with some 64-bit offsets' \ - 'git verify-pack -v "test-3-${pack3}.pack"' +test_expect_success OFF64_T 'index v2: verify a pack with some 64-bit offsets' ' + git verify-pack -v "test-3-${pack3}.pack" +' -test_expect_success OFF64_T \ - '64-bit offsets: should be different from previous index v2 results' \ - '! cmp "test-2-${pack2}.idx" "test-3-${pack3}.idx"' +test_expect_success OFF64_T '64-bit offsets: should be different from previous index v2 results' ' + ! cmp "test-2-${pack2}.idx" "test-3-${pack3}.idx" +' -test_expect_success OFF64_T \ - 'index v2: force some 64-bit offsets with index-pack' \ - 'git index-pack --index-version=2,0x40000 -o 3.idx "test-1-${pack1}.pack"' +test_expect_success OFF64_T 'index v2: force some 64-bit offsets with index-pack' ' + git index-pack --index-version=2,0x40000 -o 3.idx "test-1-${pack1}.pack" +' -test_expect_success OFF64_T \ - '64-bit offsets: index-pack result should match pack-objects one' \ - 'cmp "test-3-${pack3}.idx" "3.idx"' +test_expect_success OFF64_T '64-bit offsets: index-pack result should match pack-objects one' ' + cmp "test-3-${pack3}.idx" "3.idx" +' test_expect_success OFF64_T 'index-pack --verify on 64-bit offset v2 (cheat)' ' # This cheats by knowing which lower offset should still be encoded @@ -120,135 +120,143 @@ test_expect_success OFF64_T 'index-pack --verify on 64-bit offset v2' ' # returns the object number for given object in given pack index index_obj_nr() { - idx_file=$1 - object_sha1=$2 - nr=0 - git show-index < $idx_file | - while read offs sha1 extra - do - nr=$(($nr + 1)) - test "$sha1" = "$object_sha1" || continue - echo "$(($nr - 1))" - break - done + idx_file=$1 + object_sha1=$2 + nr=0 + git show-index < $idx_file | + while read offs sha1 extra + do + nr=$(($nr + 1)) + test "$sha1" = "$object_sha1" || continue + echo "$(($nr - 1))" + break + done } # returns the pack offset for given object as found in given pack index index_obj_offset() { - idx_file=$1 - object_sha1=$2 - git show-index < $idx_file | grep $object_sha1 | - ( read offs extra && echo "$offs" ) + idx_file=$1 + object_sha1=$2 + git show-index < $idx_file | grep $object_sha1 | + ( read offs extra && echo "$offs" ) } -test_expect_success \ - '[index v1] 1) stream pack to repository' \ - 'git index-pack --index-version=1 --stdin < "test-1-${pack1}.pack" && - git prune-packed && - git count-objects | ( read nr rest && test "$nr" -eq 1 ) && - cmp "test-1-${pack1}.pack" ".git/objects/pack/pack-${pack1}.pack" && - cmp "test-1-${pack1}.idx" ".git/objects/pack/pack-${pack1}.idx"' +test_expect_success '[index v1] 1) stream pack to repository' ' + git index-pack --index-version=1 --stdin < "test-1-${pack1}.pack" && + git prune-packed && + git count-objects | ( read nr rest && test "$nr" -eq 1 ) && + cmp "test-1-${pack1}.pack" ".git/objects/pack/pack-${pack1}.pack" && + cmp "test-1-${pack1}.idx" ".git/objects/pack/pack-${pack1}.idx" +' test_expect_success \ - '[index v1] 2) create a stealth corruption in a delta base reference' \ - '# This test assumes file_101 is a delta smaller than 16 bytes. - # It should be against file_100 but we substitute its base for file_099 - sha1_101=$(git hash-object file_101) && - sha1_099=$(git hash-object file_099) && - offs_101=$(index_obj_offset 1.idx $sha1_101) && - nr_099=$(index_obj_nr 1.idx $sha1_099) && - chmod +w ".git/objects/pack/pack-${pack1}.pack" && - recordsz=$((rawsz + 4)) && - dd of=".git/objects/pack/pack-${pack1}.pack" seek=$(($offs_101 + 1)) \ - if=".git/objects/pack/pack-${pack1}.idx" \ - skip=$((4 + 256 * 4 + $nr_099 * recordsz)) \ - bs=1 count=$rawsz conv=notrunc && - git cat-file blob $sha1_101 > file_101_foo1' + '[index v1] 2) create a stealth corruption in a delta base reference' ' + # This test assumes file_101 is a delta smaller than 16 bytes. + # It should be against file_100 but we substitute its base for file_099 + sha1_101=$(git hash-object file_101) && + sha1_099=$(git hash-object file_099) && + offs_101=$(index_obj_offset 1.idx $sha1_101) && + nr_099=$(index_obj_nr 1.idx $sha1_099) && + chmod +w ".git/objects/pack/pack-${pack1}.pack" && + recordsz=$((rawsz + 4)) && + dd of=".git/objects/pack/pack-${pack1}.pack" seek=$(($offs_101 + 1)) \ + if=".git/objects/pack/pack-${pack1}.idx" \ + skip=$((4 + 256 * 4 + $nr_099 * recordsz)) \ + bs=1 count=$rawsz conv=notrunc && + git cat-file blob $sha1_101 > file_101_foo1 +' test_expect_success \ - '[index v1] 3) corrupted delta happily returned wrong data' \ - 'test -f file_101_foo1 && ! cmp file_101 file_101_foo1' + '[index v1] 3) corrupted delta happily returned wrong data' ' + test -f file_101_foo1 && ! cmp file_101 file_101_foo1 +' test_expect_success \ - '[index v1] 4) confirm that the pack is actually corrupted' \ - 'test_must_fail git fsck --full $commit' + '[index v1] 4) confirm that the pack is actually corrupted' ' + test_must_fail git fsck --full $commit +' test_expect_success \ - '[index v1] 5) pack-objects happily reuses corrupted data' \ - 'pack4=$(git pack-objects test-4 file_101_foo2 +' test_expect_success \ - '[index v2] 1) stream pack to repository' \ - 'rm -f .git/objects/pack/* && - git index-pack --index-version=2 --stdin < "test-1-${pack1}.pack" && - git prune-packed && - git count-objects | ( read nr rest && test "$nr" -eq 1 ) && - cmp "test-1-${pack1}.pack" ".git/objects/pack/pack-${pack1}.pack" && - cmp "test-2-${pack1}.idx" ".git/objects/pack/pack-${pack1}.idx"' + '[index v2] 3) corrupted delta happily returned wrong data' ' + test -f file_101_foo2 && ! cmp file_101 file_101_foo2 +' test_expect_success \ - '[index v2] 2) create a stealth corruption in a delta base reference' \ - '# This test assumes file_101 is a delta smaller than 16 bytes. - # It should be against file_100 but we substitute its base for file_099 - sha1_101=$(git hash-object file_101) && - sha1_099=$(git hash-object file_099) && - offs_101=$(index_obj_offset 1.idx $sha1_101) && - nr_099=$(index_obj_nr 1.idx $sha1_099) && - chmod +w ".git/objects/pack/pack-${pack1}.pack" && - dd of=".git/objects/pack/pack-${pack1}.pack" seek=$(($offs_101 + 1)) \ - if=".git/objects/pack/pack-${pack1}.idx" \ - skip=$((8 + 256 * 4 + $nr_099 * rawsz)) \ - bs=1 count=$rawsz conv=notrunc && - git cat-file blob $sha1_101 > file_101_foo2' + '[index v2] 4) confirm that the pack is actually corrupted' ' + test_must_fail git fsck --full $commit +' test_expect_success \ - '[index v2] 3) corrupted delta happily returned wrong data' \ - 'test -f file_101_foo2 && ! cmp file_101 file_101_foo2' + '[index v2] 5) pack-objects refuses to reuse corrupted data' ' + test_must_fail git pack-objects test-5 /dev/null || exit 1 - done /dev/null || exit 1 + done wrong-tag <wrong-tag <err && - grep "^warning:.* expected .tagger. line" err + tag=$(git hash-object -t tag -w --stdin err && + grep "^warning:.* expected .tagger. line" err ' test_expect_success 'index-pack --fsck-objects also warns upon missing tagger in tag' ' - git index-pack --fsck-objects tag-test-${pack1}.pack 2>err && - grep "^warning:.* expected .tagger. line" err + git index-pack --fsck-objects tag-test-${pack1}.pack 2>err && + grep "^warning:.* expected .tagger. line" err ' test_done From patchwork Wed May 13 00:54:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11544427 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AB9A81668 for ; Wed, 13 May 2020 00:55:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9238120753 for ; Wed, 13 May 2020 00:55:26 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="g//Vdc4S" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732060AbgEMAzY (ORCPT ); Tue, 12 May 2020 20:55:24 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38112 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726031AbgEMAyv (ORCPT ); Tue, 12 May 2020 20:54:51 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 57C5360CFF; Wed, 13 May 2020 00:54:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1589331289; bh=FIDfPhqmGD6F3kAFqrdeqRinJc9gYjcu7fAYule9rDE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=g//Vdc4SFCIM/FT7JuB4UDxM3d32YbCB4LhAFBO+7IHmnxmXudNB5NHdhMM7CT8Q3 Nr/yvHG/p34CEv9lVdSlzZjZ4aqe5W3yAVkdMcXmxS3ruerrstEEnZ5VmmHP8i4qdZ UpDO/l1rM/G09NDbTmrBQzkKoOYLyglU4eLUwTNEU5QyhlXTpJfhuKGXHScE43jBQ6 JU0r0iQQ0K6YFYe5rQPl/dE+P5IQlDgqV2UZBgSXe6E62L1ltCpmFKpUmqd8b2q05Y MRSgvz1Pm1828WMvfnzd5b4PvcFoEkTPxlZMa0JcToS9tOKni42jeAnkWt2Mr5Ww1w 1sGuF7dNro5KiA+DNtMo0J99gu8eOpyVkPl/2RY4zPaXF2Gv9BynEjVdFl2BcRaXoO T7Ne5uT2qXQisz9Ol/biYef5xVKb87wDU+RRHb3wYCkY51/OLfTxhDMMic/N5BRdPb nyMYulxLNhg3IteFwtyLi51PgnjFh471UjdLLlizholtqCyKfYx From: "brian m. carlson" To: Cc: Jonathan Tan Subject: [PATCH 27/44] builtin/show-index: provide options to determine hash algo Date: Wed, 13 May 2020 00:54:07 +0000 Message-Id: <20200513005424.81369-28-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be In-Reply-To: <20200513005424.81369-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org It's possible to use a variety of index formats with show-index, and we need a way to indicate the hash algorithm which is in use for a particular index we'd like to show. Default to using the value for the repository we're in by calling setup_git_directory_gently, and allow overriding it by using a --hash argument. Signed-off-by: brian m. carlson --- builtin/show-index.c | 29 ++++++++++++++++++++++++----- git.c | 2 +- 2 files changed, 25 insertions(+), 6 deletions(-) diff --git a/builtin/show-index.c b/builtin/show-index.c index 0826f6a5a2..ebfa2e9abd 100644 --- a/builtin/show-index.c +++ b/builtin/show-index.c @@ -1,9 +1,12 @@ #include "builtin.h" #include "cache.h" #include "pack.h" +#include "parse-options.h" -static const char show_index_usage[] = -"git show-index"; +static const char *const show_index_usage[] = { + "git show-index [--hash=HASH]", + NULL +}; int cmd_show_index(int argc, const char **argv, const char *prefix) { @@ -11,10 +14,26 @@ int cmd_show_index(int argc, const char **argv, const char *prefix) unsigned nr; unsigned int version; static unsigned int top_index[256]; - const unsigned hashsz = the_hash_algo->rawsz; + unsigned hashsz; + const char *hash_name = NULL; + int hash_algo; + const struct option show_index_options[] = { + OPT_STRING(0, "hash", &hash_name, N_("hash"), + N_("specify the hash algorithm to use")), + OPT_END() + }; + + argc = parse_options(argc, argv, prefix, show_index_options, show_index_usage, 0); + + if (hash_name) { + hash_algo = hash_algo_by_name(hash_name); + if (hash_algo == GIT_HASH_UNKNOWN) + die(_("Unknown hash algorithm")); + repo_set_hash_algo(the_repository, hash_algo); + } + + hashsz = the_hash_algo->rawsz; - if (argc != 1) - usage(show_index_usage); if (fread(top_index, 2 * 4, 1, stdin) != 1) die("unable to read header"); if (top_index[0] == htonl(PACK_IDX_SIGNATURE)) { diff --git a/git.c b/git.c index 2e4efb4ff0..e53e8159a2 100644 --- a/git.c +++ b/git.c @@ -573,7 +573,7 @@ static struct cmd_struct commands[] = { { "shortlog", cmd_shortlog, RUN_SETUP_GENTLY | USE_PAGER }, { "show", cmd_show, RUN_SETUP }, { "show-branch", cmd_show_branch, RUN_SETUP }, - { "show-index", cmd_show_index }, + { "show-index", cmd_show_index, RUN_SETUP_GENTLY }, { "show-ref", cmd_show_ref, RUN_SETUP }, { "sparse-checkout", cmd_sparse_checkout, RUN_SETUP | NEED_WORK_TREE }, { "stage", cmd_add, RUN_SETUP | NEED_WORK_TREE }, From patchwork Wed May 13 00:54:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11544463 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 20A1E17EA for ; Wed, 13 May 2020 00:56:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id F246220753 for ; Wed, 13 May 2020 00:56:02 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="JB68NXyr" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732082AbgEMA4C (ORCPT ); Tue, 12 May 2020 20:56:02 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38264 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731966AbgEMAzV (ORCPT ); Tue, 12 May 2020 20:55:21 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id BECB660D0C; Wed, 13 May 2020 00:54:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1589331290; bh=VVQ0dANHQNwdWuSaocgt9e3EABF8vgKVBdEBtDLaQuI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=JB68NXyr87VdfTUadG2MV+hcOQb7Tbrd6gz0pm7ST1gZHtxiGUH7tVXkkjQ3hrgiv 26PfP9Z9i3Ppdox7kSzm2Xcu5GhcStHhTB0DHS9xx6slKsg7gw9GMnr7QUq1JKDO8C rmzOi9Gq1IT7Js/F5pV1vWgIGgY8ygFOnNaeNyFnkN4kMHVjYhQacwXX4wT2YXvUzP rYjnfLJLQ6FTYDlldhAhBLRvNADCh1a+xtihiFvzqgzNTQq6hmVOWdK1B4TfyMv5Ll OXv69TgVidPno4BsTeugxcEHfht5h3r73WN87bS5/6obkguBS6Z46SCzX9vjrfqNfc OOcSMyxvAf6ODyy81qk2JfUBDH11vTGVm7Om+QkWLhOLNuVy/3pbOY6jwjooFje67T oY7qDXRuC7KvnvcSQPF8mZSczCL4oc6XLi0uMcx4UmkS7eXxrxyMopDmdGQA+oXhuP yYAK19VNhazB6dj96ePNHNQ6hA3s/ArcJ7eoDPFvQHnlD2XXexV From: "brian m. carlson" To: Cc: Jonathan Tan Subject: [PATCH 28/44] t1302: expect repo format version 1 for SHA-256 Date: Wed, 13 May 2020 00:54:08 +0000 Message-Id: <20200513005424.81369-29-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be In-Reply-To: <20200513005424.81369-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When using SHA-256, we need to take advantage of the extensions section in the config file, so we need to use repository format version 1. Update the test to look for the correct value. Note that test_oid produces a value without a trailing newline, so use echo to ensure we print a trailing newline to compare it correctly against the actual results. Signed-off-by: brian m. carlson --- t/t1302-repo-version.sh | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/t/t1302-repo-version.sh b/t/t1302-repo-version.sh index ce4cff13bb..d60c042ce8 100755 --- a/t/t1302-repo-version.sh +++ b/t/t1302-repo-version.sh @@ -8,6 +8,10 @@ test_description='Test repository version check' . ./test-lib.sh test_expect_success 'setup' ' + test_oid_cache <<-\EOF && + version sha1:0 + version sha256:1 + EOF cat >test.patch <<-\EOF && diff --git a/test.txt b/test.txt new file mode 100644 @@ -23,7 +27,7 @@ test_expect_success 'setup' ' ' test_expect_success 'gitdir selection on normal repos' ' - echo 0 >expect && + echo $(test_oid version) >expect && git config core.repositoryformatversion >actual && git -C test config core.repositoryformatversion >actual2 && test_cmp expect actual && From patchwork Wed May 13 00:54:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11544467 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 08EFF1668 for ; Wed, 13 May 2020 00:56:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DC1A420753 for ; Wed, 13 May 2020 00:56:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="uLbSdOBr" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732165AbgEMA4E (ORCPT ); Tue, 12 May 2020 20:56:04 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38104 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731897AbgEMAyv (ORCPT ); Tue, 12 May 2020 20:54:51 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 31B9860D0E; Wed, 13 May 2020 00:54:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1589331290; bh=GiBRjTyb6hSilLNak5pYDTZoH59+gji1GPh9ELc0b1c=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=uLbSdOBrecSer81so2pdw8lLQELARAfEvf07W9z2tmqQ0IQjIqskXOlWahwj57dUu hx7mUhMWuVmWQY8hlw6i6cahJaKLGLxTQS4UGOTVUEMbjk9SMqEF0m1U7arPbzJJMK hJ5JlZ1dk4ZKxhfMJGnm5vcM4g+LSWEeIYvYT1bux3Y/sbkb8s9cdDGlA9X207j3oR XzL64gJvd3d4lgixcjQyI8/xpnPaj1lBmWG01Mow8AiXN2g6hPmZ+BnXzPPIKvP7Tp qPyF8IrrJZC6PNUYteDU2ptO6CyErKeBGvNfm6Zgn0pFQBQZqkWTY2Jpnzmw3RSJJz IWw0g1mNhWZPydyNiEqz7Zm7E/RXNTJ7VrxpuwpozoBMgIQXBxNBXEZ2Cf1vlLQZEO utgYIaTUYk7np8FNnl4bMHLTTiXE2+93NFJJ0EXHa5h9voVJfa5aJen/X9rE120yKk kANeDoK5feYjLZLxqv9KtJNb944nj9WsoJKaaxABnDjmqSDJnH2 From: "brian m. carlson" To: Cc: Jonathan Tan Subject: [PATCH 29/44] Documentation/technical: document object-format for protocol v2 Date: Wed, 13 May 2020 00:54:09 +0000 Message-Id: <20200513005424.81369-30-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be In-Reply-To: <20200513005424.81369-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Document the object-format extension for protocol v2. Signed-off-by: brian m. carlson --- Documentation/technical/protocol-v2.txt | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/Documentation/technical/protocol-v2.txt b/Documentation/technical/protocol-v2.txt index 7e3766cafb..107e421fb7 100644 --- a/Documentation/technical/protocol-v2.txt +++ b/Documentation/technical/protocol-v2.txt @@ -453,3 +453,12 @@ included in a request. This is done by sending each option as a a request. The provided options must not contain a NUL or LF character. + + object-format +~~~~~~~~~~~~~~~ + +The server can advertise the `object-format` capability with a value `X` (in the +form `object-format=X`) to notify the client that the server is able to deal +with objects using hash algorithm X. If not specified, the server is assumed to +only handle SHA-1. If the client would like to use a hash algorithm other than +SHA-1, it should specify its object-format string. From patchwork Wed May 13 00:54:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11544393 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2CBBD90 for ; Wed, 13 May 2020 00:55:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0AF5B2078C for ; Wed, 13 May 2020 00:55:02 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="iAFVCi52" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731962AbgEMAzB (ORCPT ); Tue, 12 May 2020 20:55:01 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38156 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731906AbgEMAyw (ORCPT ); Tue, 12 May 2020 20:54:52 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 9833260D0D; Wed, 13 May 2020 00:54:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1589331290; bh=QofIfR5pOyf6mrSUvF27hy9tTvuhWy8Qz5L3I6VaToU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=iAFVCi52+wITCjGnqor5x1k533AP6tbjBbyiqcGTPmqDMHJs035iwi084FGY0canP JTkOM37fdKsYjCJJtQguVnTEDPwJmNn7tgJOxuIlKtBxTsWnzP5JYfjM2O0N6SVvNh Oj3tLnrxBMfK66X9R8UUlLfVcL3XvSZiof+ROYr9OYlJ7t+vqpLWpPrebNgcIRchBe Rc9sI0rR2BAsNSOIbQd5Nrv00dyTTA8BMU5n+SP0nGC9ewt1modkY5FyoYncVSop2K YLWcoW9BjRaUKDnWOfmjZ8feqlvMGKleXiXV4Clv1p8Etgf1VLO0vzcAk82mMdzqSH l/AqDDVPIocPCuDEItLql/CWWfgb/OFcyX91eGwXI6Wd/bf7+kC5z91viqCcQqmJHZ AKbiDVXTHXsro2xpeh4nJvKC0UWbwgPu1wV832sEnItxFXTvA+QKvaDH+b2/34pqkY EWItzslWrNg6Jf62Emh/j9F958fMpeWMiPkgUEIZF2xj5UJAtNj From: "brian m. carlson" To: Cc: Jonathan Tan Subject: [PATCH 30/44] connect: pass full packet reader when parsing v2 refs Date: Wed, 13 May 2020 00:54:10 +0000 Message-Id: <20200513005424.81369-31-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be In-Reply-To: <20200513005424.81369-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When we're parsing refs, we need to know not only what the line we're parsing is, but also the hash algorithm we should use to parse it, which is stored in the reader object. Pass the packet reader object through to the protocol v2 ref parsing function. Signed-off-by: brian m. carlson --- connect.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/connect.c b/connect.c index a39b843589..bb4bc4ab7f 100644 --- a/connect.c +++ b/connect.c @@ -373,7 +373,7 @@ struct ref **get_remote_heads(struct packet_reader *reader, } /* Returns 1 when a valid ref has been added to `list`, 0 otherwise */ -static int process_ref_v2(const char *line, struct ref ***list) +static int process_ref_v2(struct packet_reader *reader, struct ref ***list) { int ret = 1; int i = 0; @@ -381,6 +381,7 @@ static int process_ref_v2(const char *line, struct ref ***list) struct ref *ref; struct string_list line_sections = STRING_LIST_INIT_DUP; const char *end; + const char *line = reader->line; /* * Ref lines have a number of fields which are space deliminated. The @@ -466,9 +467,10 @@ struct ref **get_remote_refs(int fd_out, struct packet_reader *reader, } packet_flush(fd_out); + /* Process response from server */ while (packet_reader_read(reader) == PACKET_READ_NORMAL) { - if (!process_ref_v2(reader->line, &list)) + if (!process_ref_v2(reader, &list)) die(_("invalid ls-refs response: %s"), reader->line); } From patchwork Wed May 13 00:54:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11544459 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4EA5A90 for ; Wed, 13 May 2020 00:55:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3457C2078C for ; Wed, 13 May 2020 00:55:59 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="tL/0oXLP" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732053AbgEMAzX (ORCPT ); Tue, 12 May 2020 20:55:23 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38118 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731908AbgEMAyw (ORCPT ); Tue, 12 May 2020 20:54:52 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 0DCA660D0F; Wed, 13 May 2020 00:54:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1589331291; bh=JSgB3nukR2NV7duJV991p0Ltx9V/rSGJxeO51RyxAVw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=tL/0oXLPk/J5f/m8sroDrv+YddXjo8WDpN6Jc8yVcMuCGHOzuazrj+q/N7Eq9/CUp JeJsZV3x6W0A4+afOc7E8LH0yw0Cx6b+pCpzW0xdX9v0DJRyjZPLq5KvHlwW7DVISC jdmhFfHC43sRlXeosdYWyS+KuAd86KA8GWuJV1FjVt9c+hPcuDcch9pysMT1RDbIS4 lxEdq4GDqjJjyOyNG/VgFkTqxye25ru8pV/B7oujwFrB6EiuAlS+gvHFb7h8EoyOTj w+0eCCpiioyUx+R4U8f8cktd0BhRP/uhFthVZtBMkJ9LlNGeumGHt44kcq457tYJ8i Yerd1z1dBjsfQeRpiV7e2pQSF+ZAlSKF1cSndP+vMk4GJwlMMGSLBk5Edob2nhJQmW RIu7sQEyvrazv9CHhXSk0ZG20AkN5kPpg6U5AlowXhgJBtsGpU9diOIJoCqoubKK/W WHGYYSvYQPgVhsV7YJsx/xj9yeHkdTlyefYLldC3U69LW2rxH8X From: "brian m. carlson" To: Cc: Jonathan Tan Subject: [PATCH 31/44] connect: parse v2 refs with correct hash algorithm Date: Wed, 13 May 2020 00:54:11 +0000 Message-Id: <20200513005424.81369-32-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be In-Reply-To: <20200513005424.81369-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When using protocol v2, we need to know what hash algorithm is used by the remote end. See if the server has sent us an object-format capability, and if so, use it to determine the hash algorithm in use and set that value in the packet reader. Parse the refs using this algorithm. Note that we use memcpy instead of oidcpy for copying values, since oidcpy is intentionally limited to the current hash algorithm length, and the copy will be too short if the server side uses SHA-256 but the client side has not had a repository set up (and therefore defaults to SHA-1). Signed-off-by: brian m. carlson --- connect.c | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-) diff --git a/connect.c b/connect.c index bb4bc4ab7f..4e6462e52f 100644 --- a/connect.c +++ b/connect.c @@ -394,7 +394,7 @@ static int process_ref_v2(struct packet_reader *reader, struct ref ***list) goto out; } - if (parse_oid_hex(line_sections.items[i++].string, &old_oid, &end) || + if (parse_oid_hex_algop(line_sections.items[i++].string, &old_oid, &end, reader->hash_algo) || *end) { ret = 0; goto out; @@ -402,7 +402,7 @@ static int process_ref_v2(struct packet_reader *reader, struct ref ***list) ref = alloc_ref(line_sections.items[i++].string); - oidcpy(&ref->old_oid, &old_oid); + memcpy(ref->old_oid.hash, old_oid.hash, reader->hash_algo->rawsz); **list = ref; *list = &ref->next; @@ -415,7 +415,8 @@ static int process_ref_v2(struct packet_reader *reader, struct ref ***list) struct object_id peeled_oid; char *peeled_name; struct ref *peeled; - if (parse_oid_hex(arg, &peeled_oid, &end) || *end) { + if (parse_oid_hex_algop(arg, &peeled_oid, &end, + reader->hash_algo) || *end) { ret = 0; goto out; } @@ -423,7 +424,8 @@ static int process_ref_v2(struct packet_reader *reader, struct ref ***list) peeled_name = xstrfmt("%s^{}", ref->name); peeled = alloc_ref(peeled_name); - oidcpy(&peeled->old_oid, &peeled_oid); + memcpy(peeled->old_oid.hash, peeled_oid.hash, + reader->hash_algo->rawsz); **list = peeled; *list = &peeled->next; @@ -442,6 +444,7 @@ struct ref **get_remote_refs(int fd_out, struct packet_reader *reader, const struct string_list *server_options) { int i; + const char *hash_name; *list = NULL; if (server_supports_v2("ls-refs", 1)) @@ -450,6 +453,14 @@ struct ref **get_remote_refs(int fd_out, struct packet_reader *reader, if (server_supports_v2("agent", 0)) packet_write_fmt(fd_out, "agent=%s", git_user_agent_sanitized()); + if (server_feature_v2("object-format", &hash_name)) { + int hash_algo = hash_algo_by_name(hash_name); + if (hash_algo == GIT_HASH_UNKNOWN) + die(_("unknown object format '%s' specified by server"), hash_name); + reader->hash_algo = &hash_algos[hash_algo]; + packet_write_fmt(fd_out, "object-format=%s", reader->hash_algo->name); + } + if (server_options && server_options->nr && server_supports_v2("server-option", 1)) for (i = 0; i < server_options->nr; i++) From patchwork Wed May 13 00:54:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11544397 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 937E31668 for ; Wed, 13 May 2020 00:55:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 783CA20675 for ; Wed, 13 May 2020 00:55:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="xPYgU9mg" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731971AbgEMAzD (ORCPT ); Tue, 12 May 2020 20:55:03 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38140 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731912AbgEMAyx (ORCPT ); Tue, 12 May 2020 20:54:53 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 758C960D10; Wed, 13 May 2020 00:54:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1589331291; bh=i88gs3LGI63F5dBfTz3y5WYylMYflhUfS0VuYjhQUUs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=xPYgU9mgreT1DznvkiC8TUVVA+xay63M6fOuumXZpAEQ1pC1NfKRk53IDP9Nq/wG/ CV+gtrDHKNAAdtpSwOoBq9f1bRfu1HNo9yTQRKt3qvwtusySbH9em6BIzWs0QOaj9N 1kO82kjo6jPHkRVvhq5910ZmOqNlFEmfsWA1vIi4yisLT8PoY1WnB9fnAq+lqJCIDy 46CWNDf7HOD8AzGnc+Gm8GAHPKw7Iy+ZWZfwp+dPY91pkjzCEG0b76lnoWylYxI2fT aFSuUP7+xpSuWunULLHaFBQ/A/P3rmyfSmKa8d9korlF/pH5R5MSkJhy9ZwP3G+0ft F4d+Ho7jLqLdeaQw4xTmejf7gPWvESAyXgO0ynLEzv1mH2HPVaMBgiJLfGIbJ5IZnK NrsKSWZuviLIXShNGc62FOOscbDkOD2zKOHVkVEk5m3E18marrqXD3EYkWAVzx7FHj MjGfAZtvPJQ3gPLTfd55gGlpE4fKdhedBJXVXQPS6N9dfhW4QU9 From: "brian m. carlson" To: Cc: Jonathan Tan Subject: [PATCH 32/44] serve: advertise object-format capability for protocol v2 Date: Wed, 13 May 2020 00:54:12 +0000 Message-Id: <20200513005424.81369-33-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be In-Reply-To: <20200513005424.81369-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In order to communicate the protocol supported by the server side, add support for advertising the object-format capability. We check that the client side sends us an identical algorithm if it sends us its own object-format capability, and assume it speaks SHA-1 if not. In the test, when we're using an algorithm other than SHA-1, we need to specify the algorithm in use so we don't get a failure with an "unknown format" message. Add a wrapper function that specifies this header if required. Skip specifying this header for SHA-1 to test that it works both with and without this header. Signed-off-by: brian m. carlson --- serve.c | 27 +++++++++++++++++++++++++++ t/t5701-git-serve.sh | 28 ++++++++++++++++++++-------- 2 files changed, 47 insertions(+), 8 deletions(-) diff --git a/serve.c b/serve.c index 317256c1a4..7ab7807fef 100644 --- a/serve.c +++ b/serve.c @@ -22,6 +22,14 @@ static int agent_advertise(struct repository *r, return 1; } +static int object_format_advertise(struct repository *r, + struct strbuf *value) +{ + if (value) + strbuf_addstr(value, r->hash_algo->name); + return 1; +} + struct protocol_capability { /* * The name of the capability. The server uses this name when @@ -57,6 +65,7 @@ static struct protocol_capability capabilities[] = { { "ls-refs", always_advertise, ls_refs }, { "fetch", upload_pack_advertise, upload_pack_v2 }, { "server-option", always_advertise, NULL }, + { "object-format", object_format_advertise, NULL }, }; static void advertise_capabilities(void) @@ -153,6 +162,22 @@ int has_capability(const struct argv_array *keys, const char *capability, return 0; } +static void check_algorithm(struct repository *r, struct argv_array *keys) +{ + int client = GIT_HASH_SHA1, server = hash_algo_by_ptr(r->hash_algo); + const char *algo_name; + + if (has_capability(keys, "object-format", &algo_name)) { + client = hash_algo_by_name(algo_name); + if (client == GIT_HASH_UNKNOWN) + die("unknown object format '%s'", algo_name); + } + + if (client != server) + die("mismatched object format: server %s; client %s\n", + r->hash_algo->name, hash_algos[client].name); +} + enum request_state { PROCESS_REQUEST_KEYS, PROCESS_REQUEST_DONE, @@ -223,6 +248,8 @@ static int process_request(void) if (!command) die("no command requested"); + check_algorithm(the_repository, &keys); + command->command(the_repository, &keys, &reader); argv_array_clear(&keys); diff --git a/t/t5701-git-serve.sh b/t/t5701-git-serve.sh index ffb9613885..bcb6453ae3 100755 --- a/t/t5701-git-serve.sh +++ b/t/t5701-git-serve.sh @@ -4,13 +4,24 @@ test_description='test protocol v2 server commands' . ./test-lib.sh +write_command () { + echo "command=$1" + + if test "$(test_oid algo)" != sha1 + then + echo "object-format=$(test_oid algo)" + fi +} + test_expect_success 'test capability advertisement' ' + test_oid_init && cat >expect <<-EOF && version 2 agent=git/$(git version | cut -d" " -f3) ls-refs fetch=shallow server-option + object-format=$(test_oid algo) 0000 EOF @@ -45,6 +56,7 @@ test_expect_success 'request invalid capability' ' test_expect_success 'request with no command' ' test-tool pkt-line pack >in <<-EOF && agent=git/test + object-format=$(test_oid algo) 0000 EOF test_must_fail test-tool serve-v2 --stateless-rpc 2>err in <<-EOF && - command=foo + $(write_command foo) agent=git/test 0000 EOF @@ -73,7 +85,7 @@ test_expect_success 'setup some refs and tags' ' test_expect_success 'basics of ls-refs' ' test-tool pkt-line pack >in <<-EOF && - command=ls-refs + $(write_command ls-refs) 0000 EOF @@ -95,7 +107,7 @@ test_expect_success 'basics of ls-refs' ' test_expect_success 'basic ref-prefixes' ' test-tool pkt-line pack >in <<-EOF && - command=ls-refs + $(write_command ls-refs) 0001 ref-prefix refs/heads/master ref-prefix refs/tags/one @@ -115,7 +127,7 @@ test_expect_success 'basic ref-prefixes' ' test_expect_success 'refs/heads prefix' ' test-tool pkt-line pack >in <<-EOF && - command=ls-refs + $(write_command ls-refs) 0001 ref-prefix refs/heads/ 0000 @@ -135,7 +147,7 @@ test_expect_success 'refs/heads prefix' ' test_expect_success 'peel parameter' ' test-tool pkt-line pack >in <<-EOF && - command=ls-refs + $(write_command ls-refs) 0001 peel ref-prefix refs/tags/ @@ -156,7 +168,7 @@ test_expect_success 'peel parameter' ' test_expect_success 'symrefs parameter' ' test-tool pkt-line pack >in <<-EOF && - command=ls-refs + $(write_command ls-refs) 0001 symrefs ref-prefix refs/heads/ @@ -177,7 +189,7 @@ test_expect_success 'symrefs parameter' ' test_expect_success 'sending server-options' ' test-tool pkt-line pack >in <<-EOF && - command=ls-refs + $(write_command ls-refs) server-option=hello server-option=world 0001 @@ -199,7 +211,7 @@ test_expect_success 'unexpected lines are not allowed in fetch request' ' git init server && test-tool pkt-line pack >in <<-EOF && - command=fetch + $(write_command fetch) 0001 this-is-not-a-command 0000 From patchwork Wed May 13 00:54:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11544429 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D1DFC90 for ; Wed, 13 May 2020 00:55:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B0E8B20753 for ; Wed, 13 May 2020 00:55:26 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="WQcl2WmX" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732061AbgEMAzZ (ORCPT ); Tue, 12 May 2020 20:55:25 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38266 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732042AbgEMAzX (ORCPT ); Tue, 12 May 2020 20:55:23 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id DDB4660D11; Wed, 13 May 2020 00:54:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1589331292; bh=lDj+6iVRyV36yqcSu1WE1ftEbOZxQhSslHVg1FJqDT8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=WQcl2WmXEHKfs4tFf8Svc4+BJFkocJgLpQzeauAsqqX+2g2d3c8ahcgG9JWBANV4g Nu+xNUc1zplGwX6rpqoeVHK4yrbJqGzhc7xXH2U7BX2lSkURfU5b2Jp1gjyKRRgCR7 l11NX2B+c0oQqWNBCZZF9iqx4HeA2lPWmibxo8lHEf/bakfrELXbEoCnpZOiwi8LzA rVhxWUbzQgIJyN7rj44pUye3f1Nqqsh8Im/MRz9ajw8kQZYgOmVHKFXvggk+Di4kvx Bo3qZmDtx4U9PypYF4sR+5n8/VHU+OraVw3bsQ7Qu4h3aIpONYePx81Wcu8PvHQ5wq 52ekSbk9ICgVJTPFPRlkuZmEhqL8GySAdk00bU+a+vrZ/x3iCIfhwPybcQAEX7zZHI BXZ4qF5u0McHpq2TBKH7Jl/uxCpnnj9+i8rN5Pu3eWLdqUpXQwZkStCATIBZ18kFOb V8eB5GuReSLb0gjy4ZyKd7MSgOql+DCBaIhGLzg3RuZtJIryoVB From: "brian m. carlson" To: Cc: Jonathan Tan Subject: [PATCH 33/44] t5500: make hash independent Date: Wed, 13 May 2020 00:54:13 +0000 Message-Id: <20200513005424.81369-34-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be In-Reply-To: <20200513005424.81369-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org This test has hard-coded pkt-lines with object IDs. The pkt-line lengths necessarily differ between hash algorithms, so generate these lines with the packetize helper so they're always the right size. In addition, we will require an object-format capability for SHA-256, so pass that capability on to the upload-pack process. Signed-off-by: brian m. carlson --- t/t5500-fetch-pack.sh | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/t/t5500-fetch-pack.sh b/t/t5500-fetch-pack.sh index 52dd1a688c..8fee99ecfb 100755 --- a/t/t5500-fetch-pack.sh +++ b/t/t5500-fetch-pack.sh @@ -871,9 +871,10 @@ test_expect_success 'shallow since with commit graph and already-seen commit' ' GIT_PROTOCOL=version=2 git upload-pack . <<-EOF >/dev/null 0012command=fetch + $(echo "object-format=$(test_oid algo)" | packetize) 00010013deepen-since 1 - 0032want $(git rev-parse other) - 0032have $(git rev-parse master) + $(echo "want $(git rev-parse other)" | packetize) + $(echo "have $(git rev-parse master)" | packetize) 0000 EOF ) From patchwork Wed May 13 00:54:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11544441 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A93F790 for ; Wed, 13 May 2020 00:55:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8795A20753 for ; Wed, 13 May 2020 00:55:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="V9sYlPqZ" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732096AbgEMAzd (ORCPT ); Tue, 12 May 2020 20:55:33 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38268 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732044AbgEMAzX (ORCPT ); Tue, 12 May 2020 20:55:23 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 4FDC060D13; Wed, 13 May 2020 00:54:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1589331292; bh=ncte7hr0FKpv6U+a48mfFqqVQlIIMBOljR5O5MmRll4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=V9sYlPqZkrOk6JlOqCGxE/jzCdYHNMrLB0/uRFKZ3fWQviUvwOmM734JBlZbmgH1F PdH0QVsacL4e4Lh/yA8HT+00KtnLJcjJQtJ+rnqpkZXq9BFlSS/eeYVJDFSalyCzrz T4m7XxkYE3ScyPhoUUjl6zHtHdR8xPlmlYBFwDcwix7Isckz0iwZNqGs8DPR+/q69m gxLsuk66FaBRaCHUQr3zcDa8aCSpzkSEsadDeyfgnqk0RvYvlxcEjnZcQTekEAPnvT 8vzDZ8XO7fKKA/C8RrHHN9CppGR/mJXysoVeYiflaRLXvZ7fM66OnJKHGkY2PjVpuq qmGA0dBxHTHPAg8BRzR1VGbu+tJ+Myz40setkDAAptxHbZICxPNcxuB2SAViqJWIMK Pze1Im8pyPjknayMHpMopT8KlUKAVE8OwKdeUcaeJqiOT3ANYeQHzMBD3L+mbNkRDS oIWfrnkObdmRy/qyUK23ywHVHcn7DeRGa+tQ8vGavG29esWoCdf From: "brian m. carlson" To: Cc: Jonathan Tan Subject: [PATCH 34/44] builtin/ls-remote: initialize repository based on fetch Date: Wed, 13 May 2020 00:54:14 +0000 Message-Id: <20200513005424.81369-35-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be In-Reply-To: <20200513005424.81369-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org ls-remote may or may not operate within a repository, and as such will not have been initialized with the repository's hash algorithm. Even if it were, the remote side could be using a different algorithm and we would still want to display those refs properly. Find the hash algorithm used by the remote side by querying the transport object and set our hash algorithm accordingly. Without this change, if the remote side is using SHA-256, we truncate the refs to 40 hex characters, since that's the length of the default hash algorithm (SHA-1). Signed-off-by: brian m. carlson --- builtin/ls-remote.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/builtin/ls-remote.c b/builtin/ls-remote.c index 6ef519514b..3a4dd12903 100644 --- a/builtin/ls-remote.c +++ b/builtin/ls-remote.c @@ -118,6 +118,10 @@ int cmd_ls_remote(int argc, const char **argv, const char *prefix) transport->server_options = &server_options; ref = transport_get_remote_refs(transport, &ref_prefixes); + if (ref) { + int hash_algo = hash_algo_by_ptr(transport_get_hash_algo(transport)); + repo_set_hash_algo(the_repository, hash_algo); + } if (transport_disconnect(transport)) { UNLEAK(sorting); return 1; From patchwork Wed May 13 00:54:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11544433 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0B4E317EA for ; Wed, 13 May 2020 00:55:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E83FC2312B for ; Wed, 13 May 2020 00:55:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="NZbfCSFc" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732063AbgEMAz2 (ORCPT ); Tue, 12 May 2020 20:55:28 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38270 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732052AbgEMAzZ (ORCPT ); Tue, 12 May 2020 20:55:25 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id B68B160D14; Wed, 13 May 2020 00:54:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1589331293; bh=3bTsmEHMqlsl1PYPzdiFXgmzIHg++osLP7PoCkW072Y=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=NZbfCSFcMY7YemJiHYRIW3fhPmktFWK1iAOAYTh4fJHczkDKCX2i/qQTIOZWsDznk nCVyCAn9h6o3Hi3YxMFhSoDSkVwRosUCnyjlOWm4LV04vCgXw9wSxByDShQPhccZem SMZbtbAAmWHxPyaJpE7xnrj1ufEVE2Tk1YQ1xbcvqucBIOAYGems05fjSpedjb4QqH x/a3MHxOXAH1tptd0TnbYPP9V0e2fUD6kU891tC6IYpTA/YSog0W97rSKxgQev/OB2 i2fkO7ZnVJ+U2RuEoENwvNxBI6Vg3+pn9K60JEmKul/J767kVyi1kQ+aL3+zi2LdMi 8UuJ3ZcrB58+clykwuxEuqGABulpgjJRA4kSuS+qBIvfWaWUvVSCuTkw4apZQcZNV1 YoubuqFjBeOsdU++jjAgpcz/iQmmiTbO4Fn8f5LOIoC9QjfzQkIpnOP/IJ8j2cvdJ7 wbNqOP0XIIUhRvNFL1jaXLO4d7B9f6VLYS1fLi45ge4vkPDyg8A From: "brian m. carlson" To: Cc: Jonathan Tan Subject: [PATCH 35/44] remote-curl: detect algorithm for dumb HTTP by size Date: Wed, 13 May 2020 00:54:15 +0000 Message-Id: <20200513005424.81369-36-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be In-Reply-To: <20200513005424.81369-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When reading the info/refs file for a repository, we have no explicit way to detect which hash algorithm is in use because the file doesn't provide one. Detect the hash algorithm in use by the size of the first object ID. Signed-off-by: brian m. carlson --- remote-curl.c | 23 +++++++++++++++++++++-- 1 file changed, 21 insertions(+), 2 deletions(-) diff --git a/remote-curl.c b/remote-curl.c index 3ed0dfec1b..35275b42e9 100644 --- a/remote-curl.c +++ b/remote-curl.c @@ -252,6 +252,19 @@ static struct ref *parse_git_refs(struct discovery *heads, int for_push) return list; } +static const struct git_hash_algo *detect_hash_algo(struct discovery *heads) +{ + const char *p = memchr(heads->buf, '\t', heads->len); + int algo; + if (!p) + return NULL; + + algo = hash_algo_by_length((p - heads->buf) / 2); + if (algo == GIT_HASH_UNKNOWN) + return NULL; + return &hash_algos[algo]; +} + static struct ref *parse_info_refs(struct discovery *heads) { char *data, *start, *mid; @@ -262,6 +275,12 @@ static struct ref *parse_info_refs(struct discovery *heads) struct ref *ref = NULL; struct ref *last_ref = NULL; + options.hash_algo = detect_hash_algo(heads); + if (!options.hash_algo) + die("%sinfo/refs not valid: could not determine hash algorithm; " + "is this a git repository?", + url.buf); + data = heads->buf; start = NULL; mid = data; @@ -272,13 +291,13 @@ static struct ref *parse_info_refs(struct discovery *heads) if (data[i] == '\t') mid = &data[i]; if (data[i] == '\n') { - if (mid - start != the_hash_algo->hexsz) + if (mid - start != options.hash_algo->hexsz) die(_("%sinfo/refs not valid: is this a git repository?"), transport_anonymize_url(url.buf)); data[i] = 0; ref_name = mid + 1; ref = alloc_ref(ref_name); - get_oid_hex(start, &ref->old_oid); + get_oid_hex_algop(start, &ref->old_oid, options.hash_algo); if (!refs) refs = ref; if (last_ref) From patchwork Wed May 13 00:54:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11544395 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1070690 for ; Wed, 13 May 2020 00:55:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E387B20753 for ; Wed, 13 May 2020 00:55:03 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="IpA1MCh2" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731965AbgEMAzC (ORCPT ); Tue, 12 May 2020 20:55:02 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38172 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731920AbgEMAyy (ORCPT ); Tue, 12 May 2020 20:54:54 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 28AE860D15; Wed, 13 May 2020 00:54:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1589331293; bh=Tngbtm4HDKwryfMFXEnLa+X54TrkAxC2lx+1IfD4NYk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=IpA1MCh2EDXWE9qJ0UA3eaEzXWz3+U/aD5vtxmSvtkZk27Ka2GDJi6guMjYGfGzCP WHcG+OKs89LJVD9v123cl4GWc4OzLr63vGtUK+TnAkvZ9kEj54Zu5M4gZhc2z6sOf4 r3poEwZ8OfoVb0WzjpsOgpKhESyuSwqFUuJAIEpNKESdeAVTDdjIgiu0IL8/jtuyYJ t05vv8If9xNsxME0K0jn1mXd+5n5jKigF4VnNf0tkGOcb3/Y3/6e4t5P5t4E6lJv9j L/zX2nM3ohEskxZ+XAkEjkV5gzczmaDp+7hIGLCTWyI623WnJBnW4Gob4aINgoKNZK PWf/HllGzrDYuus3nqivCO5Qy9arkJjOc6IjG/0h0EK69TH5umcRJFPaCIYqEqvGZn YSbwblUVH5xitOGdqFsiuetJaeU8b/BzkNQiQwPohxElZqa/kkITl9MU5kCTuNwYWR cFMTWyo1bPdGwtI8367cUYuAgkNUs53NKUemOsBRnwP9f2Nn7AP From: "brian m. carlson" To: Cc: Jonathan Tan Subject: [PATCH 36/44] builtin/index-pack: add option to specify hash algorithm Date: Wed, 13 May 2020 00:54:16 +0000 Message-Id: <20200513005424.81369-37-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be In-Reply-To: <20200513005424.81369-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org git index-pack is usually run in a repository, but need not be. Since packs don't contains information on the algorithm in use, instead relying on context, add an option to index-pack to tell it which one we're using in case someone runs it outside of a repository. Signed-off-by: brian m. carlson --- builtin/index-pack.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/builtin/index-pack.c b/builtin/index-pack.c index 7bea1fba52..89f4962a00 100644 --- a/builtin/index-pack.c +++ b/builtin/index-pack.c @@ -1760,6 +1760,11 @@ int cmd_index_pack(int argc, const char **argv, const char *prefix) die(_("bad %s"), arg); } else if (skip_prefix(arg, "--max-input-size=", &arg)) { max_input_size = strtoumax(arg, NULL, 10); + } else if (skip_prefix(arg, "--object-format=", &arg)) { + int hash_algo = hash_algo_by_name(arg); + if (hash_algo == GIT_HASH_UNKNOWN) + die(_("unknown hash algorithm '%s'"), arg); + repo_set_hash_algo(the_repository, hash_algo); } else usage(index_pack_usage); continue; From patchwork Wed May 13 00:54:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11544435 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9E4C090 for ; Wed, 13 May 2020 00:55:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7D47223126 for ; Wed, 13 May 2020 00:55:32 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="SgTdrtwF" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732090AbgEMAzb (ORCPT ); Tue, 12 May 2020 20:55:31 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38272 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732057AbgEMAzZ (ORCPT ); Tue, 12 May 2020 20:55:25 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 8FB2160D06; Wed, 13 May 2020 00:54:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1589331293; bh=QhWwx5JrFjkmi9fYf3BLeqi+AU9uqTKM8zuKL+dV4ZA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=SgTdrtwFErQZF+7LDlnQeltYPjk9usGWQhIE3g1qnipmlUpooqy/WY1/5hOHe0WJ8 hGsw1YLGeax1JhcHqqVaOV0qZhRTwE5drOn/j9tmRa7ufmaaM/6yK7Cwy2r3jzeRNX lBEMyI+v+tXkUT+IVJMxrNJTWAaQp5RcbmI1bKfzdUMXUvbHLd5V/s7DWhlNBeQwAO +ETNZnRCSWPhYep4V37mE4Rfzik1+LILGFVAgG5c07Z5KHaSIX9uNhv+B2MmceEJvR TbayKhJo79gMndxm8rgQm7sc5ehJYAYnf56qwtf8zK+oW3wTjn6Tlkfq+hOuqfnQ1Y IGRIOd7IPAhlFFSUNhWny8gdHvZT0eEbScsvnEIcoRiMUE8ChOgKfrl3Ikideq/H0i MNruH9hpOiR0RM2ACguWsqEP6ypAD9JPODDrBMc3TDaYQAfYaInsA2ygjzm6KjY1yf oOHmZv8vQOI5AwFd28UeflKCLpxX3kmJTEanYrigLuz0JVcUXFs From: "brian m. carlson" To: Cc: Jonathan Tan Subject: [PATCH 37/44] t1050: pass algorithm to index-pack when outside repo Date: Wed, 13 May 2020 00:54:17 +0000 Message-Id: <20200513005424.81369-38-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be In-Reply-To: <20200513005424.81369-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When outside a repository, git index-pack is unable to guess the hash algorithm in use for a pack, since packs don't contain any information on the algorithm in use. Pass an option to index-pack to help it out in this test. Signed-off-by: brian m. carlson --- t/t1050-large.sh | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/t/t1050-large.sh b/t/t1050-large.sh index 7f88ea07c2..6a56d1ca24 100755 --- a/t/t1050-large.sh +++ b/t/t1050-large.sh @@ -12,6 +12,7 @@ file_size () { } test_expect_success setup ' + test_oid_init && # clone does not allow us to pass core.bigfilethreshold to # new repos, so set core.bigfilethreshold globally git config --global core.bigfilethreshold 200k && @@ -177,7 +178,8 @@ test_expect_success 'git-show a large file' ' test_expect_success 'index-pack' ' git clone file://"$(pwd)"/.git foo && - GIT_DIR=non-existent git index-pack --strict --verify foo/.git/objects/pack/*.pack + GIT_DIR=non-existent git index-pack --object-format=$(test_oid algo) \ + --strict --verify foo/.git/objects/pack/*.pack ' test_expect_success 'repack' ' From patchwork Wed May 13 00:54:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11544431 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DDAD91668 for ; Wed, 13 May 2020 00:55:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C581323126 for ; Wed, 13 May 2020 00:55:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="Ft0rWhKt" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732068AbgEMAz3 (ORCPT ); Tue, 12 May 2020 20:55:29 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38274 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732059AbgEMAzZ (ORCPT ); Tue, 12 May 2020 20:55:25 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 0278D60D16; Wed, 13 May 2020 00:54:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1589331294; bh=Q7QGgr71ZvOWyLOax8q5JM3VXmokxTeKAMbY9xJ0AFs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=Ft0rWhKtRzrZfRDJSfY78A2YQnye4joRPHAdyvCZRBRfUIqtl7jndUksNw2wWJx8a ZoIc6H/v5+9CYUGEwv9xYaM24yaQSFYpu08li1cuQaJjMuopb7944jAQl1jnEprvyQ 1W6lwZifVe1bwy7oDMhuUWhKEiubAVfzv+J+IZgmAgulVyYZ2V/xr79Egfj4yWA8v8 fRptg75qTicIYNiuwWMhntUCEOphVboG0ihOk7+JOmn9JQHgiqE6ChHiyRUPWH/gUO rUJ0bSvlFJpboqK8gc/OwaPCJJbCkxFuR5s2S3VIq0kf2vKVyWXfwUk/7STG2wnUAN QPixSL0AIzn/vD13nQyN77ARqBy+MUSjucKq9OWSR7swVBGJfvY2weR1iXGPnOXQC8 DsMihB5n2VG75d6+Z0x9LB1fOrAE9H7cvFnSibrDxrG2w77osq9HyF2ZxDurtD3Zit WyqkCy1ZkRWZho24h2VFSM28QwxspqX1q2SLA5n/qASpodbl447 From: "brian m. carlson" To: Cc: Jonathan Tan Subject: [PATCH 38/44] remote-curl: avoid truncating refs with ls-remote Date: Wed, 13 May 2020 00:54:18 +0000 Message-Id: <20200513005424.81369-39-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be In-Reply-To: <20200513005424.81369-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Normally, the remote-curl transport helper is aware of the hash algorithm we're using because we're in a repo with the appropriate hash algorithm set. However, when using git ls-remote outside of a repository, we won't have initialized the hash algorithm properly, so use hash_to_hex_algop to print the ref corresponding to the algorithm we've detected. Signed-off-by: brian m. carlson --- remote-curl.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/remote-curl.c b/remote-curl.c index 35275b42e9..9808e53182 100644 --- a/remote-curl.c +++ b/remote-curl.c @@ -548,7 +548,9 @@ static void output_refs(struct ref *refs) if (posn->symref) printf("@%s %s\n", posn->symref, posn->name); else - printf("%s %s\n", oid_to_hex(&posn->old_oid), posn->name); + printf("%s %s\n", hash_to_hex_algop(posn->old_oid.hash, + options.hash_algo), + posn->name); } printf("\n"); fflush(stdout); From patchwork Wed May 13 00:54:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11544411 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CBE191668 for ; Wed, 13 May 2020 00:55:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A86B720753 for ; Wed, 13 May 2020 00:55:12 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="gHH/IS2E" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732003AbgEMAzL (ORCPT ); Tue, 12 May 2020 20:55:11 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38162 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731928AbgEMAy4 (ORCPT ); Tue, 12 May 2020 20:54:56 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 6943C60D18; Wed, 13 May 2020 00:54:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1589331294; bh=2AW4Ophh+GDqfkkvEpKmw/NRFQar3284kGJCZ5OkO5M=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=gHH/IS2EE+3iJl4GVWPEJMbnoOKbR8Z006bAWnZcxrTlBh95Gyr/IqyQwMGbW4ijG nm3N0vpkmi3Z0hjV9wArvBqaeF0IsYURZm31ydjhEhN1xq+sGShajyfeBQZDPutjtH VhyE0nSPuXB706oDk9hz6Z4RGvO4PZ+zAS1ig2zKehxsQ9tC3cC6nZs+vJ+wvrI4ni tTsZ+xMAsntZDGUVOpyRTjqYIla8LYobHwRsaQKPHKxOXP4jWYpUNrLe+6B5BOpZq1 G3ShsyHHQARzwfH8Y+IJ4tPzPOTH/zOJMu64wdZ7T3GAqKSAN4fdrrwlPjMruTNR7l +uzqqM76aGRzaQtADIIP0FlKHLQDpCmIDrlWEjlbAtVOBKL2uFHQAl9R4dMJlsW5vW Aph+HOooQ9sZaHJN5L6MSvSgqlj4fMKaGeRgIhnisDqpeZh3qLDI9iAhAn0/ZNY/SX RGbCe0yYg9YmdCeZ72A/Cei9170iuxS6UZbyn4faWaa4ndvvkUI From: "brian m. carlson" To: Cc: Jonathan Tan Subject: [PATCH 39/44] t/helper: initialize the repository for test-sha1-array Date: Wed, 13 May 2020 00:54:19 +0000 Message-Id: <20200513005424.81369-40-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be In-Reply-To: <20200513005424.81369-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org test-sha1-array uses the_hash_algo under the hood. Since t0064 wants to use the value that is correct for the hash algorithm that we're testing, make sure the test helper initializes the repository to set the_hash_algo correctly. Signed-off-by: brian m. carlson --- t/helper/test-oid-array.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/t/helper/test-oid-array.c b/t/helper/test-oid-array.c index ce9fd5f091..b16cd0b11b 100644 --- a/t/helper/test-oid-array.c +++ b/t/helper/test-oid-array.c @@ -12,6 +12,9 @@ int cmd__oid_array(int argc, const char **argv) { struct oid_array array = OID_ARRAY_INIT; struct strbuf line = STRBUF_INIT; + int nongit_ok; + + setup_git_directory_gently(&nongit_ok); while (strbuf_getline(&line, stdin) != EOF) { const char *arg; From patchwork Wed May 13 00:54:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11544409 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6652A90 for ; Wed, 13 May 2020 00:55:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4461F2078C for ; Wed, 13 May 2020 00:55:11 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="ym6CIXKb" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732001AbgEMAzK (ORCPT ); Tue, 12 May 2020 20:55:10 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38104 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731930AbgEMAy4 (ORCPT ); Tue, 12 May 2020 20:54:56 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id F0ACF60D17; Wed, 13 May 2020 00:54:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1589331295; bh=74qW95Zvoq9O2u0va6v/nSh8paXnLt0Me5nQURj92D4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=ym6CIXKbbHwiSIlUAGzXK2PNvPls2P2glT6Og0A1Tw8e83i4ajYwN0qjAO31FdrcU 1C+q+QloxSzg9zgaq84BuibIbWtaJ1dxLG2BUV6C/hyPvJqidF5SDpaJTbMCBsd3n1 2XnV+AC6w8EgIO831AxyF6ip8ikmeSqDE/y+mTyLj41jSG3aRDfbMrRw20ckGw3+W+ lGT25QOCKzU92mcSxK9E3FKDFgl8V85r+8++OikeUcL25tdg0TDS5BDk6BDbcQOgkI 5hHtUViimJ9PjkHeeRv5TmchViJhZJwjw4Di5VoeNlC6gX8IaeKC0Pv6N3m8AJo3hc kzv95vd9b0LrVfzX47BIn430dRt79SG6D20Oy+x9kQV2j0B2nRsC0CR1HYZDfuPTkC 0sq/HbZajraRuYyBjaoXdYdihb//qcYXFgTkxXfpqVOHHo6h3HG3DtTmQCfYq+hqll FHMIn4GNN9XtsKArGrD83ItyHvnHkV3h71hCZofkdbTVvrlrok4 From: "brian m. carlson" To: Cc: Jonathan Tan Subject: [PATCH 40/44] t5702: offer an object-format capability in the test Date: Wed, 13 May 2020 00:54:20 +0000 Message-Id: <20200513005424.81369-41-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be In-Reply-To: <20200513005424.81369-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org In order to make this test work with SHA-256, offer an object-format capability so that both sides use the same algorithm. Signed-off-by: brian m. carlson --- t/t5702-protocol-v2.sh | 2 ++ 1 file changed, 2 insertions(+) diff --git a/t/t5702-protocol-v2.sh b/t/t5702-protocol-v2.sh index 5039e66dc4..116358b9ac 100755 --- a/t/t5702-protocol-v2.sh +++ b/t/t5702-protocol-v2.sh @@ -13,6 +13,7 @@ start_git_daemon --export-all --enable=receive-pack daemon_parent=$GIT_DAEMON_DOCUMENT_ROOT_PATH/parent test_expect_success 'create repo to be served by git-daemon' ' + test_oid_init && git init "$daemon_parent" && test_commit -C "$daemon_parent" one ' @@ -394,6 +395,7 @@ test_expect_success 'even with handcrafted request, filter does not work if not # Custom request that tries to filter even though it is not advertised. test-tool pkt-line pack >in <<-EOF && command=fetch + object-format=$(test_oid algo) 0001 want $(git -C server rev-parse master) filter blob:none From patchwork Wed May 13 00:54:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11544405 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8DD991668 for ; Wed, 13 May 2020 00:55:10 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 764AE20675 for ; Wed, 13 May 2020 00:55:10 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="Nto2EaeW" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731989AbgEMAzJ (ORCPT ); Tue, 12 May 2020 20:55:09 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38112 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731937AbgEMAy5 (ORCPT ); Tue, 12 May 2020 20:54:57 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 640C460D19; Wed, 13 May 2020 00:54:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1589331295; bh=/DV+zliFMgvjyMrzpHbzlFIwrj3WnjgN0b/139NAErw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=Nto2EaeWYELOu5hFIxlXH5CvP1tsUvEnAFdfWaOQL6p+U2lcR1Y+rdZbWPZV37455 TmNmUbpa+6HEAC6AD7whnIyHlRgoYLCgb/2SdsGwuTIo0Ckbltg1UUfQ0DT/6qLee5 vA8vUBpPLJZQwnMOH8/2nrnnx+f+HVgZIx2aritAPtUqmfXQjoBZOP56aIMcJg+ZLy 6A8dFKt+iRWtoDfHJqpMN9NxWMXfrRcCzV7jVXEbnwB4ESICqHCeEMdQiqJAHs3m/e 82Wox0tRWrZvhSXbv8gWgaHjhGzWLKk/nLqLY0JoVmZovnNfPeCa2jUn8HKfDkNTNu I1FVdWs5hxdx5kkpopHh+NEJxSO0unUyOLyuCS9gCZ3ff7Y+c6djoj2x6FPZadEJ0+ Zk/IG8ZyvF1Y8Y1bW/o+5RlOQIsXqGzuwgLSepI2UMNjOKnduEgFe5x+NKY8yl+n1N SXAt85X4WPAfcuMTblCzkID71zeURRpr3FSCIfst4lYnUja7clU From: "brian m. carlson" To: Cc: Jonathan Tan Subject: [PATCH 41/44] t5703: use object-format serve option Date: Wed, 13 May 2020 00:54:21 +0000 Message-Id: <20200513005424.81369-42-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be In-Reply-To: <20200513005424.81369-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When we're using an algorithm other than SHA-1, we need to specify the algorithm in use so we don't get a failure with an "unknown format" message. Add a wrapper function that specifies this header if required. Skip specifying this header for SHA-1 to test that it works both with an without this header. Signed-off-by: brian m. carlson --- t/t5703-upload-pack-ref-in-want.sh | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/t/t5703-upload-pack-ref-in-want.sh b/t/t5703-upload-pack-ref-in-want.sh index a34460f7d8..afe7f7f919 100755 --- a/t/t5703-upload-pack-ref-in-want.sh +++ b/t/t5703-upload-pack-ref-in-want.sh @@ -27,6 +27,15 @@ check_output () { test_cmp sorted_commits actual_commits } +write_command () { + echo "command=$1" + + if test "$(test_oid algo)" != sha1 + then + echo "object-format=$(test_oid algo)" + fi +} + # c(o/foo) d(o/bar) # \ / # b e(baz) f(master) @@ -62,7 +71,7 @@ test_expect_success 'config controls ref-in-want advertisement' ' test_expect_success 'invalid want-ref line' ' test-tool pkt-line pack >in <<-EOF && - command=fetch + $(write_command fetch) 0001 no-progress want-ref refs/heads/non-existent @@ -83,7 +92,7 @@ test_expect_success 'basic want-ref' ' oid=$(git rev-parse a) && test-tool pkt-line pack >in <<-EOF && - command=fetch + $(write_command fetch) 0001 no-progress want-ref refs/heads/master @@ -107,7 +116,7 @@ test_expect_success 'multiple want-ref lines' ' oid=$(git rev-parse b) && test-tool pkt-line pack >in <<-EOF && - command=fetch + $(write_command fetch) 0001 no-progress want-ref refs/heads/o/foo @@ -129,7 +138,7 @@ test_expect_success 'mix want and want-ref' ' git rev-parse e f >expected_commits && test-tool pkt-line pack >in <<-EOF && - command=fetch + $(write_command fetch) 0001 no-progress want-ref refs/heads/master @@ -152,7 +161,7 @@ test_expect_success 'want-ref with ref we already have commit for' ' oid=$(git rev-parse c) && test-tool pkt-line pack >in <<-EOF && - command=fetch + $(write_command fetch) 0001 no-progress want-ref refs/heads/o/foo From patchwork Wed May 13 00:54:22 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11544407 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B770F17EA for ; Wed, 13 May 2020 00:55:10 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 927702078C for ; Wed, 13 May 2020 00:55:10 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="mewEXEbS" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731986AbgEMAzH (ORCPT ); Tue, 12 May 2020 20:55:07 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38118 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731938AbgEMAy6 (ORCPT ); Tue, 12 May 2020 20:54:58 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id CBA4360D1B; Wed, 13 May 2020 00:54:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1589331296; bh=0kTnW+si/14E4LvMgtbMNpi0j1rIx5tw+Yjkj/LLNLQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=mewEXEbSORqDeyGJAZVPtt8y/0JSQXEUVE+eMlIdqzED5YdO55WPITbO/sEo0Is2M EfAB369+w5vhnhoD7YNZswf973OzCctSXi39BnbCz4ex9JYFzMiU86qTOYPMAz8vKL ip8cNXiCp3ypeHmMZmVVqaqJnRbfYBUuJZtPfzcGcqa1XZi6+2DqsBc32GQtwRO14j OoV/IwvEKkccv8LWO77C4SfTd9k99neSHUErwifnsMQkqQtUikAM+RU7cHUCIquZ10 9hlNrFOz498vFGiGI1aigKEqvzhIJz0m1aOXORqSJ+/lb+7qfWlupec8nER6E0U7I/ iZWs9wvK2tXSofUVsSFzhTiWRJjcHJuB7NSRVG8JmlnkX896nTM/L292RIT0p9OeWy nEz2yWoMgQc6yMsVYwZqWY1WpeRLJse4T7dnXvKTNSIRMxSV0pCFZZFEp8J5GUlZVy xiqCzksJUGdzPb42g7uDkrBJAESdEwljKL/ENVOghijYctUCQQ8 From: "brian m. carlson" To: Cc: Jonathan Tan Subject: [PATCH 42/44] t5300: pass --object-format to git index-pack Date: Wed, 13 May 2020 00:54:22 +0000 Message-Id: <20200513005424.81369-43-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be In-Reply-To: <20200513005424.81369-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org git index-pack by default reads the repository to determine the object format. However, when outside of a repository, it's necessary to specify the hash algorithm in use so that the pack can be properly indexed. Add an --object-format argument when invoking git index-pack outside of a repository. Signed-off-by: brian m. carlson --- t/t5300-pack-object.sh | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/t/t5300-pack-object.sh b/t/t5300-pack-object.sh index 410a09b0dd..746cdb626e 100755 --- a/t/t5300-pack-object.sh +++ b/t/t5300-pack-object.sh @@ -12,7 +12,8 @@ TRASH=$(pwd) test_expect_success \ 'setup' \ - 'rm -f .git/index* && + 'test_oid_init && + rm -f .git/index* && perl -e "print \"a\" x 4096;" > a && perl -e "print \"b\" x 4096;" > b && perl -e "print \"c\" x 4096;" > c && @@ -412,18 +413,18 @@ test_expect_success 'set up pack for non-repo tests' ' ' test_expect_success 'index-pack --stdin complains of non-repo' ' - nongit test_must_fail git index-pack --stdin works in non-repo' ' - nongit git index-pack ../foo.pack && + nongit git index-pack --object-format=$(test_oid algo) ../foo.pack && test_path_is_file foo.idx ' test_expect_success 'index-pack --strict works in non-repo' ' rm -f foo.idx && - nongit git index-pack --strict ../foo.pack && + nongit git index-pack --strict --object-format=$(test_oid algo) ../foo.pack && test_path_is_file foo.idx ' From patchwork Wed May 13 00:54:23 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11544403 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2272F90 for ; Wed, 13 May 2020 00:55:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 09C7A20753 for ; Wed, 13 May 2020 00:55:08 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="aA86K8Nd" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731982AbgEMAzG (ORCPT ); Tue, 12 May 2020 20:55:06 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38156 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731941AbgEMAy6 (ORCPT ); Tue, 12 May 2020 20:54:58 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id 3EBD260D12; Wed, 13 May 2020 00:54:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1589331296; bh=VQHK4KZV/HJtohhf/r/fcuDQ02WfP9S43VaRaa73xx8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=aA86K8NdVHzhdOGGKre3xdV4INo4BVBbw1L/BXjyT10R3pIMWJWQ5SZjA8X0xHfWk pSB0JOheoQK3Zw1jCKyzvp3a5GrHS49DOkDhUTthW/i8QgniAl2TUL9HEbBem2DeII yRyQLT3kUcu4PNR3xY690k2VvH7OTccC3tgIyF9UWHiDGUXfPv81Znx2ahIhKUddSA wZQFURZHi3ecXhgg46vljiPVaH64j4Sj/6vy/4b9cPMwLmea5hcYjVkbmpIOQEYDRq 0l9J1E5Hi8MAf8oplPcOvEQqQlotDNG4yjxq8fq7jTzpup+7oxIwKcK+Va+MWDy/V1 YqcH7E8JYYDohO4ZwMYnzR3uHNc7Ygkrn2lRohATDa3Azg7LCi/qah7eq2SQKIik4D U+C9WLhjfx7fIEataehVQiPeVaq/j9ix9UIIXCHtM0Sm5vWBt3Wi85PW7Q2E5Zidgp dy/NpVQLuCEC2BxpO8pG9wp053zoZp0RZqOD9diWmAzyrjnDepT From: "brian m. carlson" To: Cc: Jonathan Tan Subject: [PATCH 43/44] bundle: detect hash algorithm when reading refs Date: Wed, 13 May 2020 00:54:23 +0000 Message-Id: <20200513005424.81369-44-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be In-Reply-To: <20200513005424.81369-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Much like with the dumb HTTP transport, there isn't a way to explicitly specify the hash algorithm when dealing with a bundle, so detect the algorithm based on the length of the object IDs in the prerequisites and ref advertisements. Signed-off-by: brian m. carlson --- bundle.c | 22 +++++++++++++++++++++- bundle.h | 1 + transport.c | 10 ++++++++-- 3 files changed, 30 insertions(+), 3 deletions(-) diff --git a/bundle.c b/bundle.c index 99439e07a1..2a0d744d3f 100644 --- a/bundle.c +++ b/bundle.c @@ -23,6 +23,17 @@ static void add_to_ref_list(const struct object_id *oid, const char *name, list->nr++; } +static const struct git_hash_algo *detect_hash_algo(struct strbuf *buf) +{ + size_t len = strcspn(buf->buf, " \n"); + int algo; + + algo = hash_algo_by_length(len / 2); + if (algo == GIT_HASH_UNKNOWN) + return NULL; + return &hash_algos[algo]; +} + static int parse_bundle_header(int fd, struct bundle_header *header, const char *report_path) { @@ -52,12 +63,21 @@ static int parse_bundle_header(int fd, struct bundle_header *header, } strbuf_rtrim(&buf); + if (!header->hash_algo) { + header->hash_algo = detect_hash_algo(&buf); + if (!header->hash_algo) { + error(_("unknown hash algorithm length")); + status = -1; + break; + } + } + /* * Tip lines have object name, SP, and refname. * Prerequisites have object name that is optionally * followed by SP and subject line. */ - if (parse_oid_hex(buf.buf, &oid, &p) || + if (parse_oid_hex_algop(buf.buf, &oid, &p, header->hash_algo) || (*p && !isspace(*p)) || (!is_prereq && !*p)) { if (report_path) diff --git a/bundle.h b/bundle.h index ceab0c7475..2dc9442024 100644 --- a/bundle.h +++ b/bundle.h @@ -15,6 +15,7 @@ struct ref_list { struct bundle_header { struct ref_list prerequisites; struct ref_list references; + const struct git_hash_algo *hash_algo; }; int is_bundle(const char *path, int quiet); diff --git a/transport.c b/transport.c index b43d985f90..38a432be69 100644 --- a/transport.c +++ b/transport.c @@ -143,6 +143,9 @@ static struct ref *get_refs_from_bundle(struct transport *transport, data->fd = read_bundle_header(transport->url, &data->header); if (data->fd < 0) die(_("could not read bundle '%s'"), transport->url); + + transport->hash_algo = data->header.hash_algo; + for (i = 0; i < data->header.references.nr; i++) { struct ref_list_entry *e = data->header.references.list + i; struct ref *ref = alloc_ref(e->name); @@ -157,11 +160,14 @@ static int fetch_refs_from_bundle(struct transport *transport, int nr_heads, struct ref **to_fetch) { struct bundle_transport_data *data = transport->data; + int ret; if (!data->get_refs_from_bundle_called) get_refs_from_bundle(transport, 0, NULL); - return unbundle(the_repository, &data->header, data->fd, - transport->progress ? BUNDLE_VERBOSE : 0); + ret = unbundle(the_repository, &data->header, data->fd, + transport->progress ? BUNDLE_VERBOSE : 0); + transport->hash_algo = data->header.hash_algo; + return ret; } static int close_bundle(struct transport *transport) From patchwork Wed May 13 00:54:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "brian m. carlson" X-Patchwork-Id: 11544399 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 785201668 for ; Wed, 13 May 2020 00:55:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 561602078C for ; Wed, 13 May 2020 00:55:06 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (3072-bit key) header.d=crustytoothpaste.net header.i=@crustytoothpaste.net header.b="kqF4LMWP" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731977AbgEMAzF (ORCPT ); Tue, 12 May 2020 20:55:05 -0400 Received: from injection.crustytoothpaste.net ([192.241.140.119]:38140 "EHLO injection.crustytoothpaste.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731946AbgEMAy6 (ORCPT ); Tue, 12 May 2020 20:54:58 -0400 Received: from camp.crustytoothpaste.net (unknown [IPv6:2001:470:b978:101:b610:a2f0:36c1:12e3]) (using TLSv1.2 with cipher ECDHE-RSA-CHACHA20-POLY1305 (256/256 bits)) (No client certificate requested) by injection.crustytoothpaste.net (Postfix) with ESMTPSA id A5DAC60D1A; Wed, 13 May 2020 00:54:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=crustytoothpaste.net; s=default; t=1589331296; bh=73juDmT7AedVAFtoq2gbukr8T2H8y8stjGC9wi2bUQQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Reply-To: Subject:Date:To:CC:Resent-Date:Resent-From:Resent-To:Resent-Cc: In-Reply-To:References:Content-Type:Content-Disposition; b=kqF4LMWPH8twVcsAIivho/qlwBS1GDqrxz8yiclkyA5CdCbfd9378gmudFvtRui2F a1IZjnGKgMiNL/zHr6UJ4kh9l31FRtzILNTsGQhSUp3tJfNjL2s2zwquhgNMfKVH8A VV6IsuWLgOlMfdu84ia+u6AxQvoCEvBbP2fAEuuQyqqYmwCpi+377YgCoeGW73LzKj UPzxaQeOUoj8ckXzTOG2P6YLkjFR6F0N8fVgg5N9SVxljCGPm2yOOt4d2HFmy7TbJV WezbbeCO9vzzRMuxyUPNZbkfHFfnVk2xn/F4jKkCtXtS4YliSXznPj6U4Ot54JWpSW /Z7yhAZa+WgHkuWkq7vWLV7K89CfK6KNwh/fpbTw2pGufoXWDEACjIddhbBWttTDz/ BOeyNTT8Zs6aJqBs3HiDq0YcQQLskD/ha9o0nbjhlOTfSUfPYVciQvJUPCqvN+E+NM 55aeWdcWQOAZdVEu5mrifwwNFZw18mzQPrIEsjJJVkruSQuwg8O From: "brian m. carlson" To: Cc: Jonathan Tan Subject: [PATCH 44/44] remote-testgit: adapt for object-format Date: Wed, 13 May 2020 00:54:24 +0000 Message-Id: <20200513005424.81369-45-sandals@crustytoothpaste.net> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be In-Reply-To: <20200513005424.81369-1-sandals@crustytoothpaste.net> References: <20200513005424.81369-1-sandals@crustytoothpaste.net> MIME-Version: 1.0 Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org When using an algorithm other than SHA-1, we need the remote helper to advertise support for the object-format extension and provide information back to us so that we can properly parse refs and return data. Ensure that the test remote helper understands these extensions. Signed-off-by: brian m. carlson --- t/t5801/git-remote-testgit | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/t/t5801/git-remote-testgit b/t/t5801/git-remote-testgit index 6b9f0b5dc7..1544d6dc6b 100755 --- a/t/t5801/git-remote-testgit +++ b/t/t5801/git-remote-testgit @@ -52,9 +52,11 @@ do test -n "$GIT_REMOTE_TESTGIT_SIGNED_TAGS" && echo "signed-tags" test -n "$GIT_REMOTE_TESTGIT_NO_PRIVATE_UPDATE" && echo "no-private-update" echo 'option' + echo 'object-format' echo ;; list) + echo ":object-format $(git rev-parse --show-object-format=storage)" git for-each-ref --format='? %(refname)' 'refs/heads/' 'refs/tags/' head=$(git symbolic-ref HEAD) echo "@$head HEAD" @@ -139,6 +141,10 @@ do test $val = "true" && force="true" || force= echo "ok" ;; + object-format) + test $val = "true" && object_format="true" || object_format= + echo "ok" + ;; *) echo "unsupported" ;;