From patchwork Wed Sep 27 19:55:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13401611 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 829E7E82CCC for ; Wed, 27 Sep 2023 19:55:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229719AbjI0Tz4 (ORCPT ); Wed, 27 Sep 2023 15:55:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57678 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229464AbjI0Tzy (ORCPT ); Wed, 27 Sep 2023 15:55:54 -0400 Received: from mail-io1-xd35.google.com (mail-io1-xd35.google.com [IPv6:2607:f8b0:4864:20::d35]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0B27B10E for ; Wed, 27 Sep 2023 12:55:52 -0700 (PDT) Received: by mail-io1-xd35.google.com with SMTP id ca18e2360f4ac-79fca042ec0so242141839f.3 for ; Wed, 27 Sep 2023 12:55:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1695844551; x=1696449351; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=qOgoZtsJ9EsnKCpmBsoG93+BKGzH8VZ6zQpfUKUiUoY=; b=jiSighZQGZl/BV4RzURYZccU920bXzxt2rbY1VlATydozAk77Xv0hj8Q8qBXxn0OAF rsE/QQIkArghIIW7WaEXTMfrFzCKxRnRY7VA6kiMzScybY8kIMtAozLRJQyqFH3cMTR9 1efjnplSWLF6YBx1su6f6//bsZ5Y3izTFw0ufyPbBTxQe02nBCq7y5T2ppu3w4zuh2BB ZifU7Uw8reK68RO+Khol+DUclBIt6xU9H56nzQPpkUvA/Tb3BuonAYZrizMt/y9ilBo4 rZhi2/mxM0gpi0K3npkazvmOl1253bCFMZUBx9cWYGMNFCPfOLov1aSCXlGMWmcsQNz2 zLWA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695844551; x=1696449351; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qOgoZtsJ9EsnKCpmBsoG93+BKGzH8VZ6zQpfUKUiUoY=; b=mxlNJZTV55bt9OCCIV/96mKxFZ92LQ25ZvI2FtW6DsDPWC6pOIA3BQlAubTnjU5yif xD3ufx9ZpLyhdKOO/p/2yhWl0SApwnSMtKlWwo2TlU4MD4uRvjnhHkxNvP/ygzSbqPZW lkxVwY3+GKA4eAC52EXEwB19nuWo3MtDbmgj+XA2yQUWLrNCrrO8p49/XEgztlq4QtaZ jqfrOcArR2t6cc6kgAWiiP63Z/D3BRJ3/+hYNiWls+Z0DggX8wLw6AnEMHWOJOIkfe4Q JZrnfkMiv0BoCg/NzkSkz9fHmMUz9Tv+azlafbvHVyHyYUlS+3pdF9E23YMcWthjNyEC iyjQ== X-Gm-Message-State: AOJu0YyKpAfaVGJFO+kDvy8wH4gnpA8myKpL+k667eQ1XrtQYGUM8aAM aWUDsEqDWsOrqHMJ8pWYuSdLr58Mlo9k3g== X-Google-Smtp-Source: AGHT+IFxTMT0GqSJOfsHhgvqMSeLWW0Ec8grjyLWQdMrtEXSFnXDi+uSD9wZkZbmJ/zwfXj2KFE9aw== X-Received: by 2002:a6b:fd05:0:b0:79f:ce11:c1b0 with SMTP id c5-20020a6bfd05000000b0079fce11c1b0mr3376613ioi.6.1695844551344; Wed, 27 Sep 2023 12:55:51 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id o23-20020a6b5a17000000b007836c7e8dccsm2935628iob.17.2023.09.27.12.55.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Sep 2023 12:55:50 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , "Eric W. Biederman" Subject: [PATCH 01/30] object-file-convert: Stubs for converting from one object format to another Date: Wed, 27 Sep 2023 14:55:08 -0500 Message-Id: <20230927195537.1682-1-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> References: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "Eric W. Biederman" Two basic functions are provided: - convert_object_file Takes an object file it's type and hash algorithm and converts it into the equivalent object file that would have been generated with hash algorithm "to". For blob objects there is no converstion to be done and it is an error to use this function on them. For commit, tree, and tag objects embedded oids are replaced by the oids of the objects they refer to with those objects and their object ids reencoded in with the hash algorithm "to". Signatures are rearranged so that they remain valid after the object has been reencoded. - repo_oid_to_algop which takes an oid that refers to an object file and returns the oid of the equavalent object file generated with the target hash algorithm. The pair of files object-file-convert.c and object-file-convert.h are introduced to hold as much of this logic as possible to keep this conversion logic cleanly separated from everything else and in the hopes that someday the code will be clean enough git can support compiling out support for sha1 and the various conversion functions. Signed-off-by: "Eric W. Biederman" --- Makefile | 1 + object-file-convert.c | 57 +++++++++++++++++++++++++++++++++++++++++++ object-file-convert.h | 24 ++++++++++++++++++ 3 files changed, 82 insertions(+) create mode 100644 object-file-convert.c create mode 100644 object-file-convert.h diff --git a/Makefile b/Makefile index 577630936535..f7e824f25cda 100644 --- a/Makefile +++ b/Makefile @@ -1073,6 +1073,7 @@ LIB_OBJS += notes-cache.o LIB_OBJS += notes-merge.o LIB_OBJS += notes-utils.o LIB_OBJS += notes.o +LIB_OBJS += object-file-convert.o LIB_OBJS += object-file.o LIB_OBJS += object-name.o LIB_OBJS += object.o diff --git a/object-file-convert.c b/object-file-convert.c new file mode 100644 index 000000000000..ba3e18f6af44 --- /dev/null +++ b/object-file-convert.c @@ -0,0 +1,57 @@ +#include "git-compat-util.h" +#include "gettext.h" +#include "strbuf.h" +#include "repository.h" +#include "hash-ll.h" +#include "object.h" +#include "object-file-convert.h" + +int repo_oid_to_algop(struct repository *repo, const struct object_id *src, + const struct git_hash_algo *to, struct object_id *dest) +{ + /* + * If the source alogirthm is not set, then we're using the + * default hash algorithm for that object. + */ + const struct git_hash_algo *from = + src->algo ? &hash_algos[src->algo] : repo->hash_algo; + + if (from == to) { + if (src != dest) + oidcpy(dest, src); + return 0; + } + return -1; +} + +int convert_object_file(struct strbuf *outbuf, + const struct git_hash_algo *from, + const struct git_hash_algo *to, + const void *buf, size_t len, + enum object_type type, + int gentle) +{ + int ret; + + /* Don't call this function when no conversion is necessary */ + if ((from == to) || (type == OBJ_BLOB)) + die("Refusing noop object file conversion"); + + switch (type) { + case OBJ_COMMIT: + case OBJ_TREE: + case OBJ_TAG: + default: + /* Not implemented yet, so fail. */ + ret = -1; + break; + } + if (!ret) + return 0; + if (gentle) { + strbuf_release(outbuf); + return ret; + } + die(_("Failed to convert object from %s to %s"), + from->name, to->name); +} diff --git a/object-file-convert.h b/object-file-convert.h new file mode 100644 index 000000000000..a4f802aa8eea --- /dev/null +++ b/object-file-convert.h @@ -0,0 +1,24 @@ +#ifndef OBJECT_CONVERT_H +#define OBJECT_CONVERT_H + +struct repository; +struct object_id; +struct git_hash_algo; +struct strbuf; +#include "object.h" + +int repo_oid_to_algop(struct repository *repo, const struct object_id *src, + const struct git_hash_algo *to, struct object_id *dest); + +/* + * Convert an object file from one hash algorithm to another algorithm. + * Return -1 on failure, 0 on success. + */ +int convert_object_file(struct strbuf *outbuf, + const struct git_hash_algo *from, + const struct git_hash_algo *to, + const void *buf, size_t len, + enum object_type type, + int gentle); + +#endif /* OBJECT_CONVERT_H */ From patchwork Wed Sep 27 19:55:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13401612 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 71C14E82CCD for ; Wed, 27 Sep 2023 19:55:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229736AbjI0Tz6 (ORCPT ); Wed, 27 Sep 2023 15:55:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57692 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229592AbjI0Tzz (ORCPT ); Wed, 27 Sep 2023 15:55:55 -0400 Received: from mail-io1-xd2a.google.com (mail-io1-xd2a.google.com [IPv6:2607:f8b0:4864:20::d2a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A0257114 for ; Wed, 27 Sep 2023 12:55:52 -0700 (PDT) Received: by mail-io1-xd2a.google.com with SMTP id ca18e2360f4ac-79fb70402a7so294042739f.0 for ; Wed, 27 Sep 2023 12:55:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1695844552; x=1696449352; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=gKRqigMmBJjt+jVVA0IpgumX4pWEUQkfZwHO4CKzSaU=; b=SdAzkteXgf4kb3EYLSAztXqgTenz7Y04NYOk89UHTqvjz1Xyp4pQLGRbEhp5aoCf3o ZJbfcuZURILb5j3Y/d1ZPvZM2900OUwYHaJ8miyaifAbo/2YWjy/3ta+c0EtRBLC45FH akPZODOPAYLx/gOjuz46hqC9rIjLqVkxfbmtNKfFKgskmOaVLFsbFJtgjBYRbQmO3yES elw/ArO4IZ1USy3R0O9ZOK678nwKWJOfzPPqe8IIyMNu244XginrzfljAIy6yFRsr2Ea +Iam2EHQ+bA05hxV7GsKTrWCVjJCT4FUTDzUknsjv+1M5UWhIMlt7tkrpieXhzp2nqZU ndew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695844552; x=1696449352; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=gKRqigMmBJjt+jVVA0IpgumX4pWEUQkfZwHO4CKzSaU=; b=kr/ClcDMIEPTztbPIlvIuu3mIntFA21RE9Y6vNItXLRPXJdwlwRQJljpBnnu2RTkhZ bkIBr8xIwqA9bJXBlQNNzigPQCNZMZN8j4058M6CQAnCnWhg4unfdD1niil8tE3sEQUd QYRwf4jWTwQZmjD1LOWn+0x1vgyYEYz1MMofq62hT6KQD8lhmKDQRSdy/xFPUrleL9Hh 6Hq0Z1k/Vu189VlgWUpQJR3cUZeOGIqf4EbUBZ3rW1TDfN/1kk1xd2ICJGl0SRoM3UtP 1YCng0vfLFQ4emzld7hSm9ePsxUYPBCEG4SybEhGBUYqrwUOJsmnDPFbQFtUGkKNEp3j NA+A== X-Gm-Message-State: AOJu0YwjhEafA9D1aPeepTw75WZWndaUKiGT2kjaMUYMsZXFL/g0oETV preW9sH6iuHckXqbhqopUV4= X-Google-Smtp-Source: AGHT+IFeS2lbywd2eWZoIlw8tJnWosFjWhuwGNOTsS9MenJFgHMpv5VfqZoLuypLm+WlXJOpxoysvg== X-Received: by 2002:a6b:ea0a:0:b0:79f:9574:b93c with SMTP id m10-20020a6bea0a000000b0079f9574b93cmr2949717ioc.0.1695844552031; Wed, 27 Sep 2023 12:55:52 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id o23-20020a6b5a17000000b007836c7e8dccsm2935628iob.17.2023.09.27.12.55.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Sep 2023 12:55:51 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , "Eric W. Biederman" Subject: [PATCH 02/30] oid-array: Teach oid-array to handle multiple kinds of oids Date: Wed, 27 Sep 2023 14:55:09 -0500 Message-Id: <20230927195537.1682-2-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> References: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "Eric W. Biederman" While looking at how to handle input of both SHA-1 and SHA-256 oids in get_oid_with_context, I realized that the oid_array in repo_for_each_abbrev might have more than one kind of oid stored in it simulataneously. Update to oid_array_append to ensure that oids added to an oid array always have an algorithm set. Update void_hashcmp to first verify two oids use the same hash algorithm before comparing them to each other. With that oid-array should be safe to use with differnt kinds of oids simultaneously. Signed-off-by: "Eric W. Biederman" --- oid-array.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/oid-array.c b/oid-array.c index 8e4717746c31..1f36651754ed 100644 --- a/oid-array.c +++ b/oid-array.c @@ -6,12 +6,20 @@ void oid_array_append(struct oid_array *array, const struct object_id *oid) { ALLOC_GROW(array->oid, array->nr + 1, array->alloc); oidcpy(&array->oid[array->nr++], oid); + if (!oid->algo) + oid_set_algo(&array->oid[array->nr - 1], the_hash_algo); array->sorted = 0; } -static int void_hashcmp(const void *a, const void *b) +static int void_hashcmp(const void *va, const void *vb) { - return oidcmp(a, b); + const struct object_id *a = va, *b = vb; + int ret; + if (a->algo == b->algo) + ret = oidcmp(a, b); + else + ret = a->algo > b->algo ? 1 : -1; + return ret; } void oid_array_sort(struct oid_array *array) From patchwork Wed Sep 27 19:55:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13401614 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 556D5E82CCC for ; Wed, 27 Sep 2023 19:56:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229761AbjI0T4A (ORCPT ); Wed, 27 Sep 2023 15:56:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55384 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229664AbjI0Tz4 (ORCPT ); Wed, 27 Sep 2023 15:55:56 -0400 Received: from mail-io1-xd32.google.com (mail-io1-xd32.google.com [IPv6:2607:f8b0:4864:20::d32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 82DBC11D for ; Wed, 27 Sep 2023 12:55:53 -0700 (PDT) Received: by mail-io1-xd32.google.com with SMTP id ca18e2360f4ac-79f915e5b47so408188439f.2 for ; Wed, 27 Sep 2023 12:55:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1695844553; x=1696449353; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=+g7ptA90ca7LQhvrnO2UwzibPbMQdF9Tu/x7XmFUxR0=; b=i1VSSjTuv4iJhifPmuc1gWzqU1/RB7cMIiJRT6cXNlDYz+elhCkODLEEUvueaZakZZ io7ByZGduwLO7S3xBzk8l0xiyyv7zmBZJs4vyeyl0KPgAXE9J4nZiEX41UfBLcT1e95I lBB0WAVEBGJTsQpI6YVf+6om+i3KzFYZjZGun3gWNJ6WTBBxUQ6CELovA9BcnXMsSeNZ v2xQtf1iovvhf6I4vT94MD3QYEpGMq/jzaDDgfqF5YW4Q9VDd3G3WBkLzsvvIuYI4E6A avfy8mxEHrQ9tqTMrWKJUi303NzX7yo0uro2LS95O3oU5ArbugJTuRun+yyjB4OAzjaz Mmng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695844553; x=1696449353; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=+g7ptA90ca7LQhvrnO2UwzibPbMQdF9Tu/x7XmFUxR0=; b=DNZFG71u5s7YnElr9rLFvB8Wr++9ye2MrFL1ufYaMb7wnKl71ojFwghHxhyHLrbSB/ 4+P+vSwg+/9KfKG9w9Bh0fYBTyskw3V24trVPrKNGgl4SPM5UYQBooX1l4dJxBRJjjuQ rurJpJ4Ma7F5D60qQb2tR+5bYH8Tsmo5ULo2dOLv72fMuZuTCXiEAUt6ykxAzHBVZYHV 1w74m33XfMiwoWz0RxgOOcRb5RGOFlk7gG5RX3YFXsiHJ+KCw8ufSanOm5JvSgMwWSP9 UkNH11PjrQ8M7oBxy5UZsF63HumtaZVAPn6vz5r7L/ItGUbqtBvVmbuwgTzY+MM8WSvC 0lrQ== X-Gm-Message-State: AOJu0YwABhVKuwyMWzXhs07Z/m5c08VlfDdQl/dUVh8NIE62HrOSv0hw uFYW6KPKXY52W4cJna6bMzk= X-Google-Smtp-Source: AGHT+IGxE4yDV6wSykql2N9/0Fl0+NIiuVR80j7DnQt6bcv0AzQGabouIYoIfQczGlACBw7lGT1nZw== X-Received: by 2002:a6b:f012:0:b0:798:312a:5403 with SMTP id w18-20020a6bf012000000b00798312a5403mr3617269ioc.19.1695844552886; Wed, 27 Sep 2023 12:55:52 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id o23-20020a6b5a17000000b007836c7e8dccsm2935628iob.17.2023.09.27.12.55.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Sep 2023 12:55:52 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , "Eric W. Biederman" Subject: [PATCH 03/30] object-names: Support input of oids in any supported hash Date: Wed, 27 Sep 2023 14:55:10 -0500 Message-Id: <20230927195537.1682-3-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> References: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "Eric W. Biederman" Support short oids encoded in any algorithm, while ensuring enough of the oid is specified to disambiguate between all of the oids in the repository encoded in any algorithm. By default have the code continue to only accept oids specified in the storage hash algorithm of the repository, but when something is ambiguous display all of the possible oids from any oid encoding. A new flag is added GET_OID_HASH_ANY that when supplied causes the code to accept oids specified in any hash algorithm, and to return the oids that were resolved. This implements the functionality that allows both SHA-1 and SHA-256 object names, from the "Object names on the command line" section of the hash function transition document. Care is taken in get_short_oid so that when the result is ambiguous the output remains the same of GIT_OID_HASH_ANY was not supplied. If GET_OID_HASH_ANY was supplied objects of any hash algorithm that match the prefix are displayed. This required updating repo_for_each_abbrev to give it a parameter so that it knows to look at all hash algorithms. Signed-off-by: "Eric W. Biederman" --- builtin/rev-parse.c | 2 +- hash-ll.h | 1 + object-name.c | 49 +++++++++++++++++++++++++++++++++++---------- object-name.h | 3 ++- 4 files changed, 42 insertions(+), 13 deletions(-) diff --git a/builtin/rev-parse.c b/builtin/rev-parse.c index fde8861ca4e0..43e96765400c 100644 --- a/builtin/rev-parse.c +++ b/builtin/rev-parse.c @@ -882,7 +882,7 @@ int cmd_rev_parse(int argc, const char **argv, const char *prefix) continue; } if (skip_prefix(arg, "--disambiguate=", &arg)) { - repo_for_each_abbrev(the_repository, arg, + repo_for_each_abbrev(the_repository, arg, the_hash_algo, show_abbrev, NULL); continue; } diff --git a/hash-ll.h b/hash-ll.h index 10d84cc20888..2cfde63ae1cf 100644 --- a/hash-ll.h +++ b/hash-ll.h @@ -145,6 +145,7 @@ struct object_id { #define GET_OID_RECORD_PATH 0200 #define GET_OID_ONLY_TO_DIE 04000 #define GET_OID_REQUIRE_PATH 010000 +#define GET_OID_HASH_ANY 020000 #define GET_OID_DISAMBIGUATORS \ (GET_OID_COMMIT | GET_OID_COMMITTISH | \ diff --git a/object-name.c b/object-name.c index 0bfa29dbbfe9..976b7106821b 100644 --- a/object-name.c +++ b/object-name.c @@ -25,6 +25,7 @@ #include "midx.h" #include "commit-reach.h" #include "date.h" +#include "object-file-convert.h" static int get_oid_oneline(struct repository *r, const char *, struct object_id *, struct commit_list *); @@ -49,6 +50,7 @@ struct disambiguate_state { static void update_candidates(struct disambiguate_state *ds, const struct object_id *current) { + /* The hash algorithm of the current has already been filtered */ if (ds->always_call_fn) { ds->ambiguous = ds->fn(ds->repo, current, ds->cb_data) ? 1 : 0; return; @@ -134,6 +136,8 @@ static void unique_in_midx(struct multi_pack_index *m, { uint32_t num, i, first = 0; const struct object_id *current = NULL; + int len = ds->len > ds->repo->hash_algo->hexsz ? + ds->repo->hash_algo->hexsz : ds->len; num = m->num_objects; if (!num) @@ -149,7 +153,7 @@ static void unique_in_midx(struct multi_pack_index *m, for (i = first; i < num && !ds->ambiguous; i++) { struct object_id oid; current = nth_midxed_object_oid(&oid, m, i); - if (!match_hash(ds->len, ds->bin_pfx.hash, current->hash)) + if (!match_hash(len, ds->bin_pfx.hash, current->hash)) break; update_candidates(ds, current); } @@ -159,6 +163,8 @@ static void unique_in_pack(struct packed_git *p, struct disambiguate_state *ds) { uint32_t num, i, first = 0; + int len = ds->len > ds->repo->hash_algo->hexsz ? + ds->repo->hash_algo->hexsz : ds->len; if (p->multi_pack_index) return; @@ -177,7 +183,7 @@ static void unique_in_pack(struct packed_git *p, for (i = first; i < num && !ds->ambiguous; i++) { struct object_id oid; nth_packed_object_id(&oid, p, i); - if (!match_hash(ds->len, ds->bin_pfx.hash, oid.hash)) + if (!match_hash(len, ds->bin_pfx.hash, oid.hash)) break; update_candidates(ds, &oid); } @@ -188,6 +194,10 @@ static void find_short_packed_object(struct disambiguate_state *ds) struct multi_pack_index *m; struct packed_git *p; + /* Skip, unless oids from the storage hash algorithm are wanted */ + if (ds->bin_pfx.algo && (&hash_algos[ds->bin_pfx.algo] != ds->repo->hash_algo)) + return; + for (m = get_multi_pack_index(ds->repo); m && !ds->ambiguous; m = m->next) unique_in_midx(m, ds); @@ -326,11 +336,12 @@ int set_disambiguate_hint_config(const char *var, const char *value) static int init_object_disambiguation(struct repository *r, const char *name, int len, + const struct git_hash_algo *algo, struct disambiguate_state *ds) { int i; - if (len < MINIMUM_ABBREV || len > the_hash_algo->hexsz) + if (len < MINIMUM_ABBREV || len > GIT_MAX_HEXSZ) return -1; memset(ds, 0, sizeof(*ds)); @@ -357,6 +368,7 @@ static int init_object_disambiguation(struct repository *r, ds->len = len; ds->hex_pfx[len] = '\0'; ds->repo = r; + ds->bin_pfx.algo = algo ? hash_algo_by_ptr(algo) : GIT_HASH_UNKNOWN; prepare_alt_odb(r); return 0; } @@ -491,9 +503,10 @@ static int repo_collect_ambiguous(struct repository *r UNUSED, return collect_ambiguous(oid, data); } -static int sort_ambiguous(const void *a, const void *b, void *ctx) +static int sort_ambiguous(const void *va, const void *vb, void *ctx) { struct repository *sort_ambiguous_repo = ctx; + const struct object_id *a = va, *b = vb; int a_type = oid_object_info(sort_ambiguous_repo, a, NULL); int b_type = oid_object_info(sort_ambiguous_repo, b, NULL); int a_type_sort; @@ -503,8 +516,13 @@ static int sort_ambiguous(const void *a, const void *b, void *ctx) * Sorts by hash within the same object type, just as * oid_array_for_each_unique() would do. */ - if (a_type == b_type) - return oidcmp(a, b); + if (a_type == b_type) { + /* Is the hash algorithm the same? */ + if (a->algo == b->algo) + return oidcmp(a, b); + else + return a->algo > b->algo ? 1 : -1; + } /* * Between object types show tags, then commits, and finally @@ -533,8 +551,12 @@ static enum get_oid_result get_short_oid(struct repository *r, int status; struct disambiguate_state ds; int quietly = !!(flags & GET_OID_QUIETLY); + const struct git_hash_algo *algo = r->hash_algo; + + if (flags & GET_OID_HASH_ANY) + algo = NULL; - if (init_object_disambiguation(r, name, len, &ds) < 0) + if (init_object_disambiguation(r, name, len, algo, &ds) < 0) return -1; if (HAS_MULTI_BITS(flags & GET_OID_DISAMBIGUATORS)) @@ -553,6 +575,7 @@ static enum get_oid_result get_short_oid(struct repository *r, else ds.fn = default_disambiguate_hint; + find_short_object_filename(&ds); find_short_packed_object(&ds); status = finish_object_disambiguation(&ds, oid); @@ -588,7 +611,7 @@ static enum get_oid_result get_short_oid(struct repository *r, if (!ds.ambiguous) ds.fn = NULL; - repo_for_each_abbrev(r, ds.hex_pfx, collect_ambiguous, &collect); + repo_for_each_abbrev(r, ds.hex_pfx, algo, collect_ambiguous, &collect); sort_ambiguous_oid_array(r, &collect); if (oid_array_for_each(&collect, show_ambiguous_object, &out)) @@ -610,15 +633,17 @@ static enum get_oid_result get_short_oid(struct repository *r, } int repo_for_each_abbrev(struct repository *r, const char *prefix, + const struct git_hash_algo *algo, each_abbrev_fn fn, void *cb_data) { struct oid_array collect = OID_ARRAY_INIT; struct disambiguate_state ds; int ret; - if (init_object_disambiguation(r, prefix, strlen(prefix), &ds) < 0) + if (init_object_disambiguation(r, prefix, strlen(prefix), algo, &ds) < 0) return -1; + ds.bin_pfx.algo = GIT_HASH_UNKNOWN; ds.always_call_fn = 1; ds.fn = repo_collect_ambiguous; ds.cb_data = &collect; @@ -787,10 +812,12 @@ void strbuf_add_unique_abbrev(struct strbuf *sb, const struct object_id *oid, int repo_find_unique_abbrev_r(struct repository *r, char *hex, const struct object_id *oid, int len) { + const struct git_hash_algo *algo = + oid->algo ? &hash_algos[oid->algo] : r->hash_algo; struct disambiguate_state ds; struct min_abbrev_data mad; struct object_id oid_ret; - const unsigned hexsz = r->hash_algo->hexsz; + const unsigned hexsz = algo->hexsz; if (len < 0) { unsigned long count = repo_approximate_object_count(r); @@ -826,7 +853,7 @@ int repo_find_unique_abbrev_r(struct repository *r, char *hex, find_abbrev_len_packed(&mad); - if (init_object_disambiguation(r, hex, mad.cur_len, &ds) < 0) + if (init_object_disambiguation(r, hex, mad.cur_len, algo, &ds) < 0) return -1; ds.fn = repo_extend_abbrev_len; diff --git a/object-name.h b/object-name.h index 9ae522307148..064ddc97d1fe 100644 --- a/object-name.h +++ b/object-name.h @@ -67,7 +67,8 @@ enum get_oid_result get_oid_with_context(struct repository *repo, const char *st typedef int each_abbrev_fn(const struct object_id *oid, void *); -int repo_for_each_abbrev(struct repository *r, const char *prefix, each_abbrev_fn, void *); +int repo_for_each_abbrev(struct repository *r, const char *prefix, + const struct git_hash_algo *algo, each_abbrev_fn, void *); int set_disambiguate_hint_config(const char *var, const char *value); From patchwork Wed Sep 27 19:55:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13401613 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B39D9E82CCE for ; Wed, 27 Sep 2023 19:55:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229747AbjI0Tz6 (ORCPT ); Wed, 27 Sep 2023 15:55:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55382 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229595AbjI0Tz4 (ORCPT ); Wed, 27 Sep 2023 15:55:56 -0400 Received: from mail-io1-xd2f.google.com (mail-io1-xd2f.google.com [IPv6:2607:f8b0:4864:20::d2f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5917F11F for ; Wed, 27 Sep 2023 12:55:54 -0700 (PDT) Received: by mail-io1-xd2f.google.com with SMTP id ca18e2360f4ac-79f95cd15dfso382728039f.0 for ; Wed, 27 Sep 2023 12:55:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1695844553; x=1696449353; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=sXaGbPC9QegDS+5xxfdJtEA4+TpWeLkpFBb18S/QmhA=; b=IvS8KVryFHvbdOB28DrOPjDX3mxTKDzM0jj166yGoU9XxshDEtLxFdWVadr+IbeHO1 Wn6aG+kEuPE4cTPSgnvqSUBSg8n/3PDl6+VL/0NJ/TZvVWI92PQDfk2o44Xnpl/KLjzX rvmgbqEhXkiCNOhjqOBrSUMqXlEnKzeEn2pKy33J+FwPweYYeckM03zeRTxxTWClClV5 DO+lNtELRoahr0OT8AXEkt0qxD4kddaAz/o3QCtYLePFKViNd9pEPiNmPEL0saAJ8LBg BZarijM20a4AUtuNi0oR0w6elHPEUjCUEsThK1vzEb9JIgwXjO6oZQaJh7iPLWzJ/iyK JEiQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695844553; x=1696449353; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=sXaGbPC9QegDS+5xxfdJtEA4+TpWeLkpFBb18S/QmhA=; b=a//s2YdAHCzVBG6TAIMXIhA4DUzUDc5cQzIUKrrvJ9uqaH80b4O4pkszXx2TjyuO2Y p0MTiz6Ptdnuu1lIaIy150vTE9Y8s8lbU7FWb9Y1crZbstj1KOA912Cm9oLXrctY49wR na5gCn4j3OCeVnjI05I4OY6i9Z5XQbjMlSm4YxCpBWU65zHEHiWvFjMk0XDjEvST+6p/ ffw0ju+fiH/HEHnKg7lhcs/WJ/BJU5AnK01u1dqIxphIWW63DYXgQMABSRzf+dvpoFxm QCn8KzuDV7GmQRs7KD3AORbhg9FwK7EJJNheiVHJwyJmZZNr8e+OZPHHvm4UK/BTiBm+ 7OmA== X-Gm-Message-State: AOJu0Yzs4jpLYYuJDBlC0wnP8gEuwg9zj68BtTM4WgVCcJlhpSRLvaVh KaYRTE1Kyk9FVj4JC68pCEM/+DgFLYZdLw== X-Google-Smtp-Source: AGHT+IGYkugShUH6OcuzqyfbNmSKCnPslRTRvynNdT7dgFX/DFAZh50njCQ7SUNCdBzoNnVCj1hvOA== X-Received: by 2002:a5e:8705:0:b0:79f:a25b:51c with SMTP id y5-20020a5e8705000000b0079fa25b051cmr3394919ioj.11.1695844553672; Wed, 27 Sep 2023 12:55:53 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id o23-20020a6b5a17000000b007836c7e8dccsm2935628iob.17.2023.09.27.12.55.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Sep 2023 12:55:53 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , "Eric W. Biederman" Subject: [PATCH 04/30] repository: add a compatibility hash algorithm Date: Wed, 27 Sep 2023 14:55:11 -0500 Message-Id: <20230927195537.1682-4-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> References: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "Eric W. Biederman" We currently have support for using a full stage 4 SHA-256 implementation. However, we'd like to support interoperability with SHA-1 repositories as well. The transition plan anticipates a compatibility hash algorithm configuration option that we can use to implement support for this. Let's add an element to the repository structure that indicates the compatibility hash algorithm so we can use it when we need to consider interoperability between algorithms. Add a helper function repo_set_compat_hash_algo that takes a compatibility hash algorithm and sets "repo->compat_hash_algo". If GIT_HASH_UNKNOWN is passed as the compatibilty hash algorithm "repo->compat_hash_algo" is set to NULL. For now, the code always results in "repo->compat_hash_algo" always being set to NULL, but that will change once a configuration option is added. Inspired-by: brian m. carlson Signed-off-by: Eric W. Biederman --- repository.c | 8 ++++++++ repository.h | 4 ++++ setup.c | 3 +++ 3 files changed, 15 insertions(+) diff --git a/repository.c b/repository.c index a7679ceeaa45..80252b79e93e 100644 --- a/repository.c +++ b/repository.c @@ -104,6 +104,13 @@ void repo_set_hash_algo(struct repository *repo, int hash_algo) repo->hash_algo = &hash_algos[hash_algo]; } +void repo_set_compat_hash_algo(struct repository *repo, int algo) +{ + if (hash_algo_by_ptr(repo->hash_algo) == algo) + BUG("hash_algo and compat_hash_algo match"); + repo->compat_hash_algo = algo ? &hash_algos[algo] : NULL; +} + /* * Attempt to resolve and set the provided 'gitdir' for repository 'repo'. * Return 0 upon success and a non-zero value upon failure. @@ -184,6 +191,7 @@ int repo_init(struct repository *repo, goto error; repo_set_hash_algo(repo, format.hash_algo); + repo_set_compat_hash_algo(repo, GIT_HASH_UNKNOWN); repo->repository_format_worktree_config = format.worktree_config; /* take ownership of format.partial_clone */ diff --git a/repository.h b/repository.h index 5f18486f6465..bf3fc601cc53 100644 --- a/repository.h +++ b/repository.h @@ -160,6 +160,9 @@ struct repository { /* Repository's current hash algorithm, as serialized on disk. */ const struct git_hash_algo *hash_algo; + /* Repository's compatibility hash algorithm. */ + const struct git_hash_algo *compat_hash_algo; + /* A unique-id for tracing purposes. */ int trace2_repo_id; @@ -199,6 +202,7 @@ void repo_set_gitdir(struct repository *repo, const char *root, const struct set_gitdir_args *extra_args); void repo_set_worktree(struct repository *repo, const char *path); void repo_set_hash_algo(struct repository *repo, int algo); +void repo_set_compat_hash_algo(struct repository *repo, int compat_algo); void initialize_the_repository(void); RESULT_MUST_BE_USED int repo_init(struct repository *r, const char *gitdir, const char *worktree); diff --git a/setup.c b/setup.c index ef9f79b8885e..deb5a33fe9e1 100644 --- a/setup.c +++ b/setup.c @@ -1572,6 +1572,8 @@ const char *setup_git_directory_gently(int *nongit_ok) } if (startup_info->have_repository) { repo_set_hash_algo(the_repository, repo_fmt.hash_algo); + repo_set_compat_hash_algo(the_repository, + GIT_HASH_UNKNOWN); the_repository->repository_format_worktree_config = repo_fmt.worktree_config; /* take ownership of repo_fmt.partial_clone */ @@ -1665,6 +1667,7 @@ void check_repository_format(struct repository_format *fmt) check_repository_format_gently(get_git_dir(), fmt, NULL); startup_info->have_repository = 1; repo_set_hash_algo(the_repository, fmt->hash_algo); + repo_set_compat_hash_algo(the_repository, GIT_HASH_UNKNOWN); the_repository->repository_format_worktree_config = fmt->worktree_config; the_repository->repository_format_partial_clone = From patchwork Wed Sep 27 19:55:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13401615 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 80830E82CCE for ; Wed, 27 Sep 2023 19:56:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229765AbjI0T4C (ORCPT ); Wed, 27 Sep 2023 15:56:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55386 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229721AbjI0Tz5 (ORCPT ); Wed, 27 Sep 2023 15:55:57 -0400 Received: from mail-io1-xd2f.google.com (mail-io1-xd2f.google.com [IPv6:2607:f8b0:4864:20::d2f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 40476CC for ; Wed, 27 Sep 2023 12:55:55 -0700 (PDT) Received: by mail-io1-xd2f.google.com with SMTP id ca18e2360f4ac-79f909071c3so407928539f.0 for ; Wed, 27 Sep 2023 12:55:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1695844554; x=1696449354; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=h4LK71V82XDfAsC4icL5Dk5UxaffRPCmxGntq7sDdQE=; b=cFzrnMXrc7DZXf0xdAzdNyMswFOqmGwboHjpfTBDlM3bQz3RPnk3Ma8DeUj6xmIW8w FsZ9HZONWqhFEjXJiIptyeqezWwlLdd9T9rqN7kkLFxyFwXfrXfNCs02mdWuokDn6uOv MNMP71iBUtqyGBFNuh/WSGeAKqlSEaka0kR9+IAiLRfQhTkSRanRQ7VUGC5tH3ODZJyf /8BhYAHGbYanE5Ru24sgl4+wkvgosZTxhKT9bHsZIhxnHpXAfuSVsK7STpefI7ceaFXN TRfen+eQnkdjg/BfVz3mhplnrhcwbodeWqPM9TzTZMdmfrVS5lEIoTeNZp5aDjDPXaWo haHA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695844554; x=1696449354; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=h4LK71V82XDfAsC4icL5Dk5UxaffRPCmxGntq7sDdQE=; b=dn0gg/kMgVCX1xgrQZcuPxwgv8cr0nLFHlsk5uqPY0PfiaFI6ywuVFtPBhG6LkyMUB gGtXOGuIBw8zPCz/QxJhM37h+jOnBeFGuEJCp5JZysy/CAPUfXwlExjTuYjsqP847cWy kiYxJA5y8tw7Uyfvo4eIHw9TsosWH7+mDDEnUoSuoEfzZahfCnwOJsodWFVgZcBXB5Vw ulhYeo463A0LaAM+qRXinSC7GJWRZGJukrKdKg4ebgjW8L/FjIIiH3VwNg6eE3t4+OCo PmH3OPjhkiAB0IWKAsRAo1NDWRsKfUMSVjzj+92XjzQah8phAMouapeGpZuNjHQbbIjp 7RDw== X-Gm-Message-State: AOJu0Yw5JoVIkkt8sOq5DZ+2laSHnjau7aX0Qd7Vu/kRIdKEkQWkHf42 1iGGQV861fL4cSBE6+YTIEA= X-Google-Smtp-Source: AGHT+IHVr4sGDUvTJpmhO6IVmxTD6lLESM6TPujoDLBnnRBAC85/l48X0Q+eqtchMEKrAkeHrqygdg== X-Received: by 2002:a6b:7301:0:b0:785:ca70:46a2 with SMTP id e1-20020a6b7301000000b00785ca7046a2mr3565772ioh.4.1695844554569; Wed, 27 Sep 2023 12:55:54 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id o23-20020a6b5a17000000b007836c7e8dccsm2935628iob.17.2023.09.27.12.55.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Sep 2023 12:55:54 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , "Eric W . Biederman" Subject: [PATCH 05/30] loose: add a mapping between SHA-1 and SHA-256 for loose objects Date: Wed, 27 Sep 2023 14:55:12 -0500 Message-Id: <20230927195537.1682-5-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> References: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "brian m. carlson" As part of the transition plan, we'd like to add a file in the .git directory that maps loose objects between SHA-1 and SHA-256. Let's implement the specification in the transition plan and store this data on a per-repository basis in struct repository. Signed-off-by: brian m. carlson Signed-off-by: Eric W. Biederman --- Makefile | 1 + loose.c | 245 ++++++++++++++++++++++++++++++++++++++++++ loose.h | 22 ++++ object-file-convert.c | 14 ++- object-store-ll.h | 3 + object.c | 2 + repository.c | 6 ++ 7 files changed, 292 insertions(+), 1 deletion(-) create mode 100644 loose.c create mode 100644 loose.h diff --git a/Makefile b/Makefile index f7e824f25cda..3c18664def9a 100644 --- a/Makefile +++ b/Makefile @@ -1053,6 +1053,7 @@ LIB_OBJS += list-objects-filter.o LIB_OBJS += list-objects.o LIB_OBJS += lockfile.o LIB_OBJS += log-tree.o +LIB_OBJS += loose.o LIB_OBJS += ls-refs.o LIB_OBJS += mailinfo.o LIB_OBJS += mailmap.o diff --git a/loose.c b/loose.c new file mode 100644 index 000000000000..28d11593b2ea --- /dev/null +++ b/loose.c @@ -0,0 +1,245 @@ +#include "git-compat-util.h" +#include "hash.h" +#include "path.h" +#include "object-store.h" +#include "hex.h" +#include "wrapper.h" +#include "gettext.h" +#include "loose.h" +#include "lockfile.h" + +static const char *loose_object_header = "# loose-object-idx\n"; + +static inline int should_use_loose_object_map(struct repository *repo) +{ + return repo->compat_hash_algo && repo->gitdir; +} + +void loose_object_map_init(struct loose_object_map **map) +{ + struct loose_object_map *m; + m = xmalloc(sizeof(**map)); + m->to_compat = kh_init_oid_map(); + m->to_storage = kh_init_oid_map(); + *map = m; +} + +static int insert_oid_pair(kh_oid_map_t *map, const struct object_id *key, const struct object_id *value) +{ + khiter_t pos; + int ret; + struct object_id *stored; + + pos = kh_put_oid_map(map, *key, &ret); + + /* This item already exists in the map. */ + if (ret == 0) + return 0; + + stored = xmalloc(sizeof(*stored)); + oidcpy(stored, value); + kh_value(map, pos) = stored; + return 1; +} + +static int load_one_loose_object_map(struct repository *repo, struct object_directory *dir) +{ + struct strbuf buf = STRBUF_INIT, path = STRBUF_INIT; + FILE *fp; + + if (!dir->loose_map) + loose_object_map_init(&dir->loose_map); + + insert_oid_pair(dir->loose_map->to_compat, repo->hash_algo->empty_tree, repo->compat_hash_algo->empty_tree); + insert_oid_pair(dir->loose_map->to_storage, repo->compat_hash_algo->empty_tree, repo->hash_algo->empty_tree); + + insert_oid_pair(dir->loose_map->to_compat, repo->hash_algo->empty_blob, repo->compat_hash_algo->empty_blob); + insert_oid_pair(dir->loose_map->to_storage, repo->compat_hash_algo->empty_blob, repo->hash_algo->empty_blob); + + insert_oid_pair(dir->loose_map->to_compat, repo->hash_algo->null_oid, repo->compat_hash_algo->null_oid); + insert_oid_pair(dir->loose_map->to_storage, repo->compat_hash_algo->null_oid, repo->hash_algo->null_oid); + + strbuf_git_common_path(&path, repo, "objects/loose-object-idx"); + fp = fopen(path.buf, "rb"); + if (!fp) + return 0; + + errno = 0; + if (strbuf_getwholeline(&buf, fp, '\n') || strcmp(buf.buf, loose_object_header)) + goto err; + while (!strbuf_getline_lf(&buf, fp)) { + const char *p; + struct object_id oid, compat_oid; + if (parse_oid_hex_algop(buf.buf, &oid, &p, repo->hash_algo) || + *p++ != ' ' || + parse_oid_hex_algop(p, &compat_oid, &p, repo->compat_hash_algo) || + p != buf.buf + buf.len) + goto err; + insert_oid_pair(dir->loose_map->to_compat, &oid, &compat_oid); + insert_oid_pair(dir->loose_map->to_storage, &compat_oid, &oid); + } + + strbuf_release(&buf); + strbuf_release(&path); + return errno ? -1 : 0; +err: + strbuf_release(&buf); + strbuf_release(&path); + return -1; +} + +int repo_read_loose_object_map(struct repository *repo) +{ + struct object_directory *dir; + + if (!should_use_loose_object_map(repo)) + return 0; + + prepare_alt_odb(repo); + + for (dir = repo->objects->odb; dir; dir = dir->next) { + if (load_one_loose_object_map(repo, dir) < 0) { + return -1; + } + } + return 0; +} + +int repo_write_loose_object_map(struct repository *repo) +{ + kh_oid_map_t *map = repo->objects->odb->loose_map->to_compat; + struct lock_file lock; + int fd; + khiter_t iter; + struct strbuf buf = STRBUF_INIT, path = STRBUF_INIT; + + if (!should_use_loose_object_map(repo)) + return 0; + + strbuf_git_common_path(&path, repo, "objects/loose-object-idx"); + fd = hold_lock_file_for_update_timeout(&lock, path.buf, LOCK_DIE_ON_ERROR, -1); + iter = kh_begin(map); + if (write_in_full(fd, loose_object_header, strlen(loose_object_header)) < 0) + goto errout; + + for (; iter != kh_end(map); iter++) { + if (kh_exist(map, iter)) { + if (oideq(&kh_key(map, iter), the_hash_algo->empty_tree) || + oideq(&kh_key(map, iter), the_hash_algo->empty_blob)) + continue; + strbuf_addf(&buf, "%s %s\n", oid_to_hex(&kh_key(map, iter)), oid_to_hex(kh_value(map, iter))); + if (write_in_full(fd, buf.buf, buf.len) < 0) + goto errout; + strbuf_reset(&buf); + } + } + strbuf_release(&buf); + if (commit_lock_file(&lock) < 0) { + error_errno(_("could not write loose object index %s"), path.buf); + strbuf_release(&path); + return -1; + } + strbuf_release(&path); + return 0; +errout: + rollback_lock_file(&lock); + strbuf_release(&buf); + error_errno(_("failed to write loose object index %s\n"), path.buf); + strbuf_release(&path); + return -1; +} + +static int write_one_object(struct repository *repo, const struct object_id *oid, + const struct object_id *compat_oid) +{ + struct lock_file lock; + int fd; + struct stat st; + struct strbuf buf = STRBUF_INIT, path = STRBUF_INIT; + + strbuf_git_common_path(&path, repo, "objects/loose-object-idx"); + hold_lock_file_for_update_timeout(&lock, path.buf, LOCK_DIE_ON_ERROR, -1); + + fd = open(path.buf, O_WRONLY | O_CREAT | O_APPEND, 0666); + if (fd < 0) + goto errout; + if (fstat(fd, &st) < 0) + goto errout; + if (!st.st_size && write_in_full(fd, loose_object_header, strlen(loose_object_header)) < 0) + goto errout; + + strbuf_addf(&buf, "%s %s\n", oid_to_hex(oid), oid_to_hex(compat_oid)); + if (write_in_full(fd, buf.buf, buf.len) < 0) + goto errout; + if (close(fd)) + goto errout; + adjust_shared_perm(path.buf); + rollback_lock_file(&lock); + strbuf_release(&buf); + strbuf_release(&path); + return 0; +errout: + error_errno(_("failed to write loose object index %s\n"), path.buf); + close(fd); + rollback_lock_file(&lock); + strbuf_release(&buf); + strbuf_release(&path); + return -1; +} + +int repo_add_loose_object_map(struct repository *repo, const struct object_id *oid, + const struct object_id *compat_oid) +{ + int inserted = 0; + + if (!should_use_loose_object_map(repo)) + return 0; + + inserted |= insert_oid_pair(repo->objects->odb->loose_map->to_compat, oid, compat_oid); + inserted |= insert_oid_pair(repo->objects->odb->loose_map->to_storage, compat_oid, oid); + if (inserted) + return write_one_object(repo, oid, compat_oid); + return 0; +} + +int repo_loose_object_map_oid(struct repository *repo, + const struct object_id *src, + const struct git_hash_algo *to, + struct object_id *dest) + +{ + struct object_directory *dir; + kh_oid_map_t *map; + khiter_t pos; + + for (dir = repo->objects->odb; dir; dir = dir->next) { + struct loose_object_map *loose_map = dir->loose_map; + if (!loose_map) + continue; + map = (to == repo->compat_hash_algo) ? + loose_map->to_compat : + loose_map->to_storage; + pos = kh_get_oid_map(map, *src); + if (pos < kh_end(map)) { + oidcpy(dest, kh_value(map, pos)); + return 0; + } + } + return -1; +} + +void loose_object_map_clear(struct loose_object_map **map) +{ + struct loose_object_map *m = *map; + struct object_id *oid; + + if (!m) + return; + + kh_foreach_value(m->to_compat, oid, free(oid)); + kh_foreach_value(m->to_storage, oid, free(oid)); + kh_destroy_oid_map(m->to_compat); + kh_destroy_oid_map(m->to_storage); + free(m); + *map = NULL; +} diff --git a/loose.h b/loose.h new file mode 100644 index 000000000000..2c2957072c5f --- /dev/null +++ b/loose.h @@ -0,0 +1,22 @@ +#ifndef LOOSE_H +#define LOOSE_H + +#include "khash.h" + +struct loose_object_map { + kh_oid_map_t *to_compat; + kh_oid_map_t *to_storage; +}; + +void loose_object_map_init(struct loose_object_map **map); +void loose_object_map_clear(struct loose_object_map **map); +int repo_loose_object_map_oid(struct repository *repo, + const struct object_id *src, + const struct git_hash_algo *dest_algo, + struct object_id *dest); +int repo_add_loose_object_map(struct repository *repo, const struct object_id *oid, + const struct object_id *compat_oid); +int repo_read_loose_object_map(struct repository *repo); +int repo_write_loose_object_map(struct repository *repo); + +#endif diff --git a/object-file-convert.c b/object-file-convert.c index ba3e18f6af44..4d62ed192bf0 100644 --- a/object-file-convert.c +++ b/object-file-convert.c @@ -4,6 +4,7 @@ #include "repository.h" #include "hash-ll.h" #include "object.h" +#include "loose.h" #include "object-file-convert.h" int repo_oid_to_algop(struct repository *repo, const struct object_id *src, @@ -21,7 +22,18 @@ int repo_oid_to_algop(struct repository *repo, const struct object_id *src, oidcpy(dest, src); return 0; } - return -1; + if (repo_loose_object_map_oid(repo, src, to, dest)) { + /* + * We may have loaded the object map at repo initialization but + * another process (perhaps upstream of a pipe from us) may have + * written a new object into the map. If the object is missing, + * let's reload the map to see if the object has appeared. + */ + repo_read_loose_object_map(repo); + if (repo_loose_object_map_oid(repo, src, to, dest)) + return -1; + } + return 0; } int convert_object_file(struct strbuf *outbuf, diff --git a/object-store-ll.h b/object-store-ll.h index 26a3895c821c..bc76d6bec80d 100644 --- a/object-store-ll.h +++ b/object-store-ll.h @@ -26,6 +26,9 @@ struct object_directory { uint32_t loose_objects_subdir_seen[8]; /* 256 bits */ struct oidtree *loose_objects_cache; + /* Map between object IDs for loose objects. */ + struct loose_object_map *loose_map; + /* * This is a temporary object store created by the tmp_objdir * facility. Disable ref updates since the objects in the store diff --git a/object.c b/object.c index 2c61e4c86217..186a0a47c0fb 100644 --- a/object.c +++ b/object.c @@ -13,6 +13,7 @@ #include "alloc.h" #include "packfile.h" #include "commit-graph.h" +#include "loose.h" unsigned int get_max_object_index(void) { @@ -540,6 +541,7 @@ void free_object_directory(struct object_directory *odb) { free(odb->path); odb_clear_loose_cache(odb); + loose_object_map_clear(&odb->loose_map); free(odb); } diff --git a/repository.c b/repository.c index 80252b79e93e..6214f61cf4e7 100644 --- a/repository.c +++ b/repository.c @@ -14,6 +14,7 @@ #include "read-cache-ll.h" #include "remote.h" #include "setup.h" +#include "loose.h" #include "submodule-config.h" #include "sparse-index.h" #include "trace2.h" @@ -109,6 +110,8 @@ void repo_set_compat_hash_algo(struct repository *repo, int algo) if (hash_algo_by_ptr(repo->hash_algo) == algo) BUG("hash_algo and compat_hash_algo match"); repo->compat_hash_algo = algo ? &hash_algos[algo] : NULL; + if (repo->compat_hash_algo) + repo_read_loose_object_map(repo); } /* @@ -201,6 +204,9 @@ int repo_init(struct repository *repo, if (worktree) repo_set_worktree(repo, worktree); + if (repo->compat_hash_algo) + repo_read_loose_object_map(repo); + clear_repository_format(&format); return 0; From patchwork Wed Sep 27 19:55:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13401616 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 05299E82CCC for ; Wed, 27 Sep 2023 19:56:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229780AbjI0T4E (ORCPT ); Wed, 27 Sep 2023 15:56:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55404 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229744AbjI0Tz6 (ORCPT ); Wed, 27 Sep 2023 15:55:58 -0400 Received: from mail-io1-xd32.google.com (mail-io1-xd32.google.com [IPv6:2607:f8b0:4864:20::d32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0614B10E for ; Wed, 27 Sep 2023 12:55:56 -0700 (PDT) Received: by mail-io1-xd32.google.com with SMTP id ca18e2360f4ac-77acb04309dso426148839f.2 for ; Wed, 27 Sep 2023 12:55:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1695844555; x=1696449355; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=s14KnSR2o+bXyNQ4jzeNB/49zdtpSQdBJXmr08CaAVQ=; b=liRtTCPiisF8SWa9T/j4PvSYrOvVuyYGlC3G9NNEXpvQ7hHSH+S9X00XZutmK+A+H0 7XH4YW9riPhL2ADSGFkC4MwZchnM54P2kVbh5bCG21Zk2JWCQOMirHTQkhghdhWyFzmW 57tjQ/5x3Kl3YfFtF6KgVNEORMAO5NbhqCP2D8h7MD4SzK+VLO7F+2/amj4l+ZxDp1/8 VQ9QbxO2yaSMYGUK+iyM7aABe8gBXcmLcBcKFkjILrdp6vT8N4oBX2nVYPbigrFBpDKT Q7NbWggC7rlwEnxcC1dj4p2tjLpCyXi9ZsKhygQ5lHWbfZ3qmYGKEc09JOsQ4fICIhQR 4/FQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695844555; x=1696449355; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=s14KnSR2o+bXyNQ4jzeNB/49zdtpSQdBJXmr08CaAVQ=; b=H/Kb3VbL9aBj294UPERovQTm0u8tt7wB+YHOmp6Sbu+yvs+h8ilqKSu1CDnBvDjikb zNKrWLST/DocoapGgwkjw9XJ7tBiXaOJCUI+p1+Ib5AUEQfnOcJZJsOUl6AAvB004aVE EAHpxhKzI5sU0uw0D1rjUSNWpGV5jS7FhZmQWwEyc/R9SGT/FWGeE8Y03RTcOJUYQgz9 SBkNr4DwDLyextUijuoKeG2V9wXeWnhiIfCaXxatiG+gLPLN9ybY63Ia73IsE0qAuiJR QmERoiRd9FmYrgi7op3w0aEZbHcPgxhtvzXcZB+YwoXXTdUtbpI4vx9S1f0FgaULk47k QsVQ== X-Gm-Message-State: AOJu0YyHd9PuPM+/pkfrbxXgJByVIaLHoPffK7xc1I6If/LZtSGg8ws0 jM4Z2H20jy+JtVIaHtld/lw= X-Google-Smtp-Source: AGHT+IEE3qsI/4f7ojj5wHmRSCN7HJfGhnu/YibOFpu3H/79THNpzF9QG/PMwYQvl4UJ7eRjF4qwVw== X-Received: by 2002:a5d:9e13:0:b0:785:ff35:f340 with SMTP id h19-20020a5d9e13000000b00785ff35f340mr3457344ioh.14.1695844555381; Wed, 27 Sep 2023 12:55:55 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id o23-20020a6b5a17000000b007836c7e8dccsm2935628iob.17.2023.09.27.12.55.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Sep 2023 12:55:54 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , "Eric W. Biederman" Subject: [PATCH 06/30] loose: Compatibilty short name support Date: Wed, 27 Sep 2023 14:55:13 -0500 Message-Id: <20230927195537.1682-6-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> References: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "Eric W. Biederman" Update loose_objects_cache when udpating the loose objects map. This oidtree is used to discover which oids are possibilities when resolving short names, and it can support a mixture of sha1 and sha256 oids. With this any oid recorded objects/loose-objects-idx is usable for resolving an oid to an object. To make this maintainable a helper insert_loose_map is factored out of load_one_loose_object_map and repo_add_loose_object_map, and then modified to also update the loose_objects_cache. Signed-off-by: "Eric W. Biederman" --- loose.c | 37 +++++++++++++++++++++++++------------ 1 file changed, 25 insertions(+), 12 deletions(-) diff --git a/loose.c b/loose.c index 28d11593b2ea..bb50d43cd1d9 100644 --- a/loose.c +++ b/loose.c @@ -7,6 +7,7 @@ #include "gettext.h" #include "loose.h" #include "lockfile.h" +#include "oidtree.h" static const char *loose_object_header = "# loose-object-idx\n"; @@ -42,6 +43,21 @@ static int insert_oid_pair(kh_oid_map_t *map, const struct object_id *key, const return 1; } +static int insert_loose_map(struct object_directory *odb, + const struct object_id *oid, + const struct object_id *compat_oid) +{ + struct loose_object_map *map = odb->loose_map; + int inserted = 0; + + inserted |= insert_oid_pair(map->to_compat, oid, compat_oid); + inserted |= insert_oid_pair(map->to_storage, compat_oid, oid); + if (inserted) + oidtree_insert(odb->loose_objects_cache, compat_oid); + + return inserted; +} + static int load_one_loose_object_map(struct repository *repo, struct object_directory *dir) { struct strbuf buf = STRBUF_INIT, path = STRBUF_INIT; @@ -49,15 +65,14 @@ static int load_one_loose_object_map(struct repository *repo, struct object_dire if (!dir->loose_map) loose_object_map_init(&dir->loose_map); + if (!dir->loose_objects_cache) { + ALLOC_ARRAY(dir->loose_objects_cache, 1); + oidtree_init(dir->loose_objects_cache); + } - insert_oid_pair(dir->loose_map->to_compat, repo->hash_algo->empty_tree, repo->compat_hash_algo->empty_tree); - insert_oid_pair(dir->loose_map->to_storage, repo->compat_hash_algo->empty_tree, repo->hash_algo->empty_tree); - - insert_oid_pair(dir->loose_map->to_compat, repo->hash_algo->empty_blob, repo->compat_hash_algo->empty_blob); - insert_oid_pair(dir->loose_map->to_storage, repo->compat_hash_algo->empty_blob, repo->hash_algo->empty_blob); - - insert_oid_pair(dir->loose_map->to_compat, repo->hash_algo->null_oid, repo->compat_hash_algo->null_oid); - insert_oid_pair(dir->loose_map->to_storage, repo->compat_hash_algo->null_oid, repo->hash_algo->null_oid); + insert_loose_map(dir, repo->hash_algo->empty_tree, repo->compat_hash_algo->empty_tree); + insert_loose_map(dir, repo->hash_algo->empty_blob, repo->compat_hash_algo->empty_blob); + insert_loose_map(dir, repo->hash_algo->null_oid, repo->compat_hash_algo->null_oid); strbuf_git_common_path(&path, repo, "objects/loose-object-idx"); fp = fopen(path.buf, "rb"); @@ -75,8 +90,7 @@ static int load_one_loose_object_map(struct repository *repo, struct object_dire parse_oid_hex_algop(p, &compat_oid, &p, repo->compat_hash_algo) || p != buf.buf + buf.len) goto err; - insert_oid_pair(dir->loose_map->to_compat, &oid, &compat_oid); - insert_oid_pair(dir->loose_map->to_storage, &compat_oid, &oid); + insert_loose_map(dir, &oid, &compat_oid); } strbuf_release(&buf); @@ -195,8 +209,7 @@ int repo_add_loose_object_map(struct repository *repo, const struct object_id *o if (!should_use_loose_object_map(repo)) return 0; - inserted |= insert_oid_pair(repo->objects->odb->loose_map->to_compat, oid, compat_oid); - inserted |= insert_oid_pair(repo->objects->odb->loose_map->to_storage, compat_oid, oid); + inserted = insert_loose_map(repo->objects->odb, oid, compat_oid); if (inserted) return write_one_object(repo, oid, compat_oid); return 0; From patchwork Wed Sep 27 19:55:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13401618 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6B7CBE82CCC for ; Wed, 27 Sep 2023 19:56:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229806AbjI0T4H (ORCPT ); Wed, 27 Sep 2023 15:56:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55418 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229748AbjI0Tz6 (ORCPT ); Wed, 27 Sep 2023 15:55:58 -0400 Received: from mail-io1-xd2a.google.com (mail-io1-xd2a.google.com [IPv6:2607:f8b0:4864:20::d2a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BE502114 for ; Wed, 27 Sep 2023 12:55:56 -0700 (PDT) Received: by mail-io1-xd2a.google.com with SMTP id ca18e2360f4ac-79f9acc857cso368627139f.2 for ; Wed, 27 Sep 2023 12:55:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1695844556; x=1696449356; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4i2Jw7pqjoHrhMzhVxMpNs0CcVLc0Uwjk9WWF7/oAwE=; b=nqEH8majcXvaZO61oXChHpDvaeyMVb3q1ClZwwLhxP/ygkn4PDwOzMo9laEQFIgOW9 vRCyeuE9euaiSEix3dAH4st4JnwWgZoMFd69c4oAfhDmLHx/jtdH2kLxdCZWjZEW9xMV KGY51EIj9W7gOp85fgb3nADMfO91+ecwhU/ujAchUw57aa50zm2pGNLQO5UDMRpij3yT L5fL4UHrG4zKXxjmmg4IkLach/Of/HJtu6qlq95ZnCCBEqpXkRx7tVot9jJNEeVRvIEl yjOL6mnm2MIQy2QCsSWxEjBI124ka0MgN+wUFH/xngKYuC1AX1FqCtlCoQW97fODpJWB B7Uw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695844556; x=1696449356; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4i2Jw7pqjoHrhMzhVxMpNs0CcVLc0Uwjk9WWF7/oAwE=; b=trWO4R7KXudYLwGVGbvlHb2cKfv7rq0sj+7keAHz9P0rEyoXmbyp52RYDrDifE7mtk 1zqsXYByMI2zSQ7hB4vd/4+j7GJoJSqOWjB9b/r6Zcm8HebFHlfaqXmpc6uaBpRUe7qG BwJOoR7GsQqL/25JwzeN0XVaDHppiraFrAzqtiWjcx09wafXf0GKXKqXyQH/PtS1HIch ZJ70qy2gj0BXfATkocVLeB43GIKClXjaITKJVhg1fjRzkDRcz4aPSXFf20Wk2LRQEXKZ bVDndguD96QEUiYvBxVS14cukB/RdX8tHlygqOOP6MFSkQBfGpemSBA37teWdTtwybvL +DGg== X-Gm-Message-State: AOJu0YzbR/A2jkEY4p/CEIOBVUdyngEv1APgyj8skl2wU5qRKJDe1IHj bdpU0EGgCSo3uqApSc6iY7w= X-Google-Smtp-Source: AGHT+IHZvDkNj7emj5Nys30kIQ8rkf7cpmcwXbIt4Pm2uhQZAsOA70WBHBHAgFyjff8/ZM6k2AZLWg== X-Received: by 2002:a5e:dd48:0:b0:79a:bbd2:8f5a with SMTP id u8-20020a5edd48000000b0079abbd28f5amr3542773iop.12.1695844556076; Wed, 27 Sep 2023 12:55:56 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id o23-20020a6b5a17000000b007836c7e8dccsm2935628iob.17.2023.09.27.12.55.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Sep 2023 12:55:55 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , "Eric W. Biederman" Subject: [PATCH 07/30] object-file: Update the loose object map when writing loose objects Date: Wed, 27 Sep 2023 14:55:14 -0500 Message-Id: <20230927195537.1682-7-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> References: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "Eric W. Biederman" To implement SHA1 compatibility on SHA256 repositories the loose object map needs to be updated whenver a loose object is written. Updating the loose object map this way allows git to support the old hash algorithm in constant time. The functions write_loose_object, and stream_loose_object are the only two functions that write to the loose object store. Update stream_loose_object to compute the compatibiilty hash, update the loose object, and then call repo_add_loose_object_map to update the loose object map. Update write_object_file_flags to convert the object into it's compatibility encoding, hash the compatibility encoding, write the object, and then update the loose object map. Update force_object_loose to lookup the hash of the compatibility encoding, write the loose object, and then update the loose object map. Update write_object_file_literally to convert the object into it's compatibility hash encoding, hash the compatibility enconding, write the object, and then update the loose object map, when the type string is a known type. For objects with an unknown type this results in a partially broken repository, as the objects are not mapped. The point of write_object_file_literally is to generate a partially broken repository for testing. For testing skipping writing the loose object map is much more useful than refusing to write the broken object at all. Except that the loose objects are updated before the loose object map I have not done any analysis to see how robust this scheme is in the event of failure. Signed-off-by: "Eric W. Biederman" --- object-file.c | 113 ++++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 95 insertions(+), 18 deletions(-) diff --git a/object-file.c b/object-file.c index 7c7afe579364..4ad31f25c555 100644 --- a/object-file.c +++ b/object-file.c @@ -43,6 +43,8 @@ #include "setup.h" #include "submodule.h" #include "fsck.h" +#include "loose.h" +#include "object-file-convert.h" /* The maximum size for an object header. */ #define MAX_HEADER_LEN 32 @@ -1952,9 +1954,12 @@ static int start_loose_object_common(struct strbuf *tmp_file, const char *filename, unsigned flags, git_zstream *stream, unsigned char *buf, size_t buflen, - git_hash_ctx *c, + git_hash_ctx *c, git_hash_ctx *compat_c, char *hdr, int hdrlen) { + struct repository *repo = the_repository; + const struct git_hash_algo *algo = repo->hash_algo; + const struct git_hash_algo *compat = repo->compat_hash_algo; int fd; fd = create_tmpfile(tmp_file, filename); @@ -1974,14 +1979,18 @@ static int start_loose_object_common(struct strbuf *tmp_file, git_deflate_init(stream, zlib_compression_level); stream->next_out = buf; stream->avail_out = buflen; - the_hash_algo->init_fn(c); + algo->init_fn(c); + if (compat && compat_c) + compat->init_fn(compat_c); /* Start to feed header to zlib stream */ stream->next_in = (unsigned char *)hdr; stream->avail_in = hdrlen; while (git_deflate(stream, 0) == Z_OK) ; /* nothing */ - the_hash_algo->update_fn(c, hdr, hdrlen); + algo->update_fn(c, hdr, hdrlen); + if (compat && compat_c) + compat->update_fn(compat_c, hdr, hdrlen); return fd; } @@ -1990,16 +1999,21 @@ static int start_loose_object_common(struct strbuf *tmp_file, * Common steps for the inner git_deflate() loop for writing loose * objects. Returns what git_deflate() returns. */ -static int write_loose_object_common(git_hash_ctx *c, +static int write_loose_object_common(git_hash_ctx *c, git_hash_ctx *compat_c, git_zstream *stream, const int flush, unsigned char *in0, const int fd, unsigned char *compressed, const size_t compressed_len) { + struct repository *repo = the_repository; + const struct git_hash_algo *algo = repo->hash_algo; + const struct git_hash_algo *compat = repo->compat_hash_algo; int ret; ret = git_deflate(stream, flush ? Z_FINISH : 0); - the_hash_algo->update_fn(c, in0, stream->next_in - in0); + algo->update_fn(c, in0, stream->next_in - in0); + if (compat && compat_c) + compat->update_fn(compat_c, in0, stream->next_in - in0); if (write_in_full(fd, compressed, stream->next_out - compressed) < 0) die_errno(_("unable to write loose object file")); stream->next_out = compressed; @@ -2014,15 +2028,21 @@ static int write_loose_object_common(git_hash_ctx *c, * - End the compression of zlib stream. * - Get the calculated oid to "oid". */ -static int end_loose_object_common(git_hash_ctx *c, git_zstream *stream, - struct object_id *oid) +static int end_loose_object_common(git_hash_ctx *c, git_hash_ctx *compat_c, + git_zstream *stream, struct object_id *oid, + struct object_id *compat_oid) { + struct repository *repo = the_repository; + const struct git_hash_algo *algo = repo->hash_algo; + const struct git_hash_algo *compat = repo->compat_hash_algo; int ret; ret = git_deflate_end_gently(stream); if (ret != Z_OK) return ret; - the_hash_algo->final_oid_fn(oid, c); + algo->final_oid_fn(oid, c); + if (compat && compat_c) + compat->final_oid_fn(compat_oid, compat_c); return Z_OK; } @@ -2046,7 +2066,7 @@ static int write_loose_object(const struct object_id *oid, char *hdr, fd = start_loose_object_common(&tmp_file, filename.buf, flags, &stream, compressed, sizeof(compressed), - &c, hdr, hdrlen); + &c, NULL, hdr, hdrlen); if (fd < 0) return -1; @@ -2056,14 +2076,14 @@ static int write_loose_object(const struct object_id *oid, char *hdr, do { unsigned char *in0 = stream.next_in; - ret = write_loose_object_common(&c, &stream, 1, in0, fd, + ret = write_loose_object_common(&c, NULL, &stream, 1, in0, fd, compressed, sizeof(compressed)); } while (ret == Z_OK); if (ret != Z_STREAM_END) die(_("unable to deflate new object %s (%d)"), oid_to_hex(oid), ret); - ret = end_loose_object_common(&c, &stream, ¶no_oid); + ret = end_loose_object_common(&c, NULL, &stream, ¶no_oid, NULL); if (ret != Z_OK) die(_("deflateEnd on object %s failed (%d)"), oid_to_hex(oid), ret); @@ -2108,10 +2128,12 @@ static int freshen_packed_object(const struct object_id *oid) int stream_loose_object(struct input_stream *in_stream, size_t len, struct object_id *oid) { + const struct git_hash_algo *compat = the_repository->compat_hash_algo; + struct object_id compat_oid; int fd, ret, err = 0, flush = 0; unsigned char compressed[4096]; git_zstream stream; - git_hash_ctx c; + git_hash_ctx c, compat_c; struct strbuf tmp_file = STRBUF_INIT; struct strbuf filename = STRBUF_INIT; int dirlen; @@ -2135,7 +2157,7 @@ int stream_loose_object(struct input_stream *in_stream, size_t len, */ fd = start_loose_object_common(&tmp_file, filename.buf, 0, &stream, compressed, sizeof(compressed), - &c, hdr, hdrlen); + &c, &compat_c, hdr, hdrlen); if (fd < 0) { err = -1; goto cleanup; @@ -2153,7 +2175,7 @@ int stream_loose_object(struct input_stream *in_stream, size_t len, if (in_stream->is_finished) flush = 1; } - ret = write_loose_object_common(&c, &stream, flush, in0, fd, + ret = write_loose_object_common(&c, &compat_c, &stream, flush, in0, fd, compressed, sizeof(compressed)); /* * Unlike write_loose_object(), we do not have the entire @@ -2176,7 +2198,7 @@ int stream_loose_object(struct input_stream *in_stream, size_t len, */ if (ret != Z_STREAM_END) die(_("unable to stream deflate new object (%d)"), ret); - ret = end_loose_object_common(&c, &stream, oid); + ret = end_loose_object_common(&c, &compat_c, &stream, oid, &compat_oid); if (ret != Z_OK) die(_("deflateEnd on stream object failed (%d)"), ret); close_loose_object(fd, tmp_file.buf); @@ -2203,6 +2225,8 @@ int stream_loose_object(struct input_stream *in_stream, size_t len, } err = finalize_object_file(tmp_file.buf, filename.buf); + if (!err && compat) + err = repo_add_loose_object_map(the_repository, oid, &compat_oid); cleanup: strbuf_release(&tmp_file); strbuf_release(&filename); @@ -2213,17 +2237,38 @@ int write_object_file_flags(const void *buf, unsigned long len, enum object_type type, struct object_id *oid, unsigned flags) { + struct repository *repo = the_repository; + const struct git_hash_algo *algo = repo->hash_algo; + const struct git_hash_algo *compat = repo->compat_hash_algo; + struct object_id compat_oid; char hdr[MAX_HEADER_LEN]; int hdrlen = sizeof(hdr); + /* Generate compat_oid */ + if (compat) { + if (type == OBJ_BLOB) + hash_object_file(compat, buf, len, type, &compat_oid); + else { + struct strbuf converted = STRBUF_INIT; + convert_object_file(&converted, algo, compat, + buf, len, type, 0); + hash_object_file(compat, converted.buf, converted.len, + type, &compat_oid); + strbuf_release(&converted); + } + } + /* Normally if we have it in the pack then we do not bother writing * it out into .git/objects/??/?{38} file. */ - write_object_file_prepare(the_hash_algo, buf, len, type, oid, hdr, - &hdrlen); + write_object_file_prepare(algo, buf, len, type, oid, hdr, &hdrlen); if (freshen_packed_object(oid) || freshen_loose_object(oid)) return 0; - return write_loose_object(oid, hdr, hdrlen, buf, len, 0, flags); + if (write_loose_object(oid, hdr, hdrlen, buf, len, 0, flags)) + return -1; + if (compat) + return repo_add_loose_object_map(repo, oid, &compat_oid); + return 0; } int write_object_file_literally(const void *buf, unsigned long len, @@ -2231,7 +2276,27 @@ int write_object_file_literally(const void *buf, unsigned long len, unsigned flags) { char *header; + struct repository *repo = the_repository; + const struct git_hash_algo *algo = repo->hash_algo; + const struct git_hash_algo *compat = repo->compat_hash_algo; + struct object_id compat_oid; int hdrlen, status = 0; + int compat_type = -1; + + if (compat) { + compat_type = type_from_string_gently(type, -1, 1); + if (compat_type == OBJ_BLOB) + hash_object_file(compat, buf, len, compat_type, + &compat_oid); + else if (compat_type != -1) { + struct strbuf converted = STRBUF_INIT; + convert_object_file(&converted, algo, compat, + buf, len, compat_type, 0); + hash_object_file(compat, converted.buf, converted.len, + compat_type, &compat_oid); + strbuf_release(&converted); + } + } /* type string, SP, %lu of the length plus NUL must fit this */ hdrlen = strlen(type) + MAX_HEADER_LEN; @@ -2244,6 +2309,8 @@ int write_object_file_literally(const void *buf, unsigned long len, if (freshen_packed_object(oid) || freshen_loose_object(oid)) goto cleanup; status = write_loose_object(oid, header, hdrlen, buf, len, 0, 0); + if (compat_type != -1) + return repo_add_loose_object_map(repo, oid, &compat_oid); cleanup: free(header); @@ -2252,9 +2319,12 @@ int write_object_file_literally(const void *buf, unsigned long len, int force_object_loose(const struct object_id *oid, time_t mtime) { + struct repository *repo = the_repository; + const struct git_hash_algo *compat = repo->compat_hash_algo; void *buf; unsigned long len; struct object_info oi = OBJECT_INFO_INIT; + struct object_id compat_oid; enum object_type type; char hdr[MAX_HEADER_LEN]; int hdrlen; @@ -2267,8 +2337,15 @@ int force_object_loose(const struct object_id *oid, time_t mtime) oi.contentp = &buf; if (oid_object_info_extended(the_repository, oid, &oi, 0)) return error(_("cannot read object for %s"), oid_to_hex(oid)); + if (compat) { + if (repo_oid_to_algop(repo, oid, compat, &compat_oid)) + return error(_("cannot map object %s to %s"), + oid_to_hex(oid), compat->name); + } hdrlen = format_object_header(hdr, sizeof(hdr), type, len); ret = write_loose_object(oid, hdr, hdrlen, buf, len, mtime, 0); + if (!ret && compat) + ret = repo_add_loose_object_map(the_repository, oid, &compat_oid); free(buf); return ret; From patchwork Wed Sep 27 19:55:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13401617 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 129CCE82CCE for ; Wed, 27 Sep 2023 19:56:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229783AbjI0T4G (ORCPT ); Wed, 27 Sep 2023 15:56:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55408 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229706AbjI0Tz6 (ORCPT ); Wed, 27 Sep 2023 15:55:58 -0400 Received: from mail-io1-xd32.google.com (mail-io1-xd32.google.com [IPv6:2607:f8b0:4864:20::d32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AAB63CC for ; Wed, 27 Sep 2023 12:55:57 -0700 (PDT) Received: by mail-io1-xd32.google.com with SMTP id ca18e2360f4ac-79fce245bf6so234926339f.1 for ; Wed, 27 Sep 2023 12:55:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1695844557; x=1696449357; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ifUaMY1sOh9/0SjHK+KXHM4A5XqXE/ubCaNK1KkvCYI=; b=HyBLyl9ffxGC3cXDAgRyphvZ6qAH2XS7We/AKUmyb4UchtWWC5s/M5akBxX5MsGc7W PUqmAJUGywJwdC0NsJh8mZ/cQf61Rlrf7mMsqhf745ywsJZBDs2EmvLpxoTSEp4kNba4 ICuAK+qmtUrmfaaRax+xwSitg2J+ARWMxr/G64fKBWFAOhxEe8kGgOKrCNV6W3pStLEQ c6/3mDFuVy51bdhcEjhAAJK9+nekX6sVbz02lsiZrshvjSK5CBOkaAMR5IKBBmyrFUkk vTB9GP0eveJCDlH9QucF1pwe9obrpLpePDSdyIS2MvOnOCzp3kUoDFLY3TlDfHcZo3w9 V4mQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695844557; x=1696449357; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ifUaMY1sOh9/0SjHK+KXHM4A5XqXE/ubCaNK1KkvCYI=; b=SLZxrOxPpKgMV9JTUkUu4q7E0Dhc6lbPjGoMX3I6x9pVCyjkJeJyL7eipPhivc4MnY t+eQT5DqsfAVh7mz7PWyefHdlv8Xe2Uc7tvmVshLYt1gjdVYtFQ0ATvmhaFKXkbAswA7 JRstb0Ur1+x58zX/zi4ae5+JhmsI0eY3yKfsw3GN3lXfWvKIRbGVBEAsjzftpBW2AZ3j YDNm+/9fIHny5Zztji1ClceKM6zW7Gho7XBjVEWELaSgtdZK4Lpl43NIRv+iJF/HUdgx 4fzeklaHk7nn7OyoLQv162stivK27mLY3u5I9aKCfz0kHumZD5bF3bzXRXrl1VFksMBt LkPQ== X-Gm-Message-State: AOJu0YxqNpohFT670bSUPClXisprpd9rywTJMRoRCREQUFqU7NfPQSzw 7v5RbhrOH1IGWvZB51xwv/A= X-Google-Smtp-Source: AGHT+IHU0aO3SA6xhsTKzmP1pHPUH/XoBeCqeJwiLxCyHoj4y39+B0ZKmbESboNOR+i+DK24BOZwig== X-Received: by 2002:a6b:7a49:0:b0:79e:4abd:7be3 with SMTP id k9-20020a6b7a49000000b0079e4abd7be3mr3098403iop.15.1695844556878; Wed, 27 Sep 2023 12:55:56 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id o23-20020a6b5a17000000b007836c7e8dccsm2935628iob.17.2023.09.27.12.55.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Sep 2023 12:55:56 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , "Eric W. Biederman" Subject: [PATCH 08/30] object-file: Add a compat_oid_in parameter to write_object_file_flags Date: Wed, 27 Sep 2023 14:55:15 -0500 Message-Id: <20230927195537.1682-8-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> References: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "Eric W. Biederman" To create the proper signatures for commit objects both versions of the commit object need to be generated and signed. After that it is a waste to throw away the work of generating the compatibility hash so update write_object_file_flags to take a compatibility hash input parameter that it can use to skip the work of generating the compatability hash. Update the places that don't generate the compatability hash to pass NULL so it is easy to tell write_object_file_flags should not attempt to use their compatability hash. Signed-off-by: "Eric W. Biederman" --- cache-tree.c | 2 +- object-file.c | 6 ++++-- object-store-ll.h | 4 ++-- 3 files changed, 7 insertions(+), 5 deletions(-) diff --git a/cache-tree.c b/cache-tree.c index 641427ed410a..ddc7d3d86959 100644 --- a/cache-tree.c +++ b/cache-tree.c @@ -448,7 +448,7 @@ static int update_one(struct cache_tree *it, hash_object_file(the_hash_algo, buffer.buf, buffer.len, OBJ_TREE, &it->oid); } else if (write_object_file_flags(buffer.buf, buffer.len, OBJ_TREE, - &it->oid, flags & WRITE_TREE_SILENT + &it->oid, NULL, flags & WRITE_TREE_SILENT ? HASH_SILENT : 0)) { strbuf_release(&buffer); return -1; diff --git a/object-file.c b/object-file.c index 4ad31f25c555..d66d11890696 100644 --- a/object-file.c +++ b/object-file.c @@ -2235,7 +2235,7 @@ int stream_loose_object(struct input_stream *in_stream, size_t len, int write_object_file_flags(const void *buf, unsigned long len, enum object_type type, struct object_id *oid, - unsigned flags) + struct object_id *compat_oid_in, unsigned flags) { struct repository *repo = the_repository; const struct git_hash_algo *algo = repo->hash_algo; @@ -2246,7 +2246,9 @@ int write_object_file_flags(const void *buf, unsigned long len, /* Generate compat_oid */ if (compat) { - if (type == OBJ_BLOB) + if (compat_oid_in) + oidcpy(&compat_oid, compat_oid_in); + else if (type == OBJ_BLOB) hash_object_file(compat, buf, len, type, &compat_oid); else { struct strbuf converted = STRBUF_INIT; diff --git a/object-store-ll.h b/object-store-ll.h index bc76d6bec80d..c5f2bb2fc2fe 100644 --- a/object-store-ll.h +++ b/object-store-ll.h @@ -255,11 +255,11 @@ void hash_object_file(const struct git_hash_algo *algo, const void *buf, int write_object_file_flags(const void *buf, unsigned long len, enum object_type type, struct object_id *oid, - unsigned flags); + struct object_id *comapt_oid_in, unsigned flags); static inline int write_object_file(const void *buf, unsigned long len, enum object_type type, struct object_id *oid) { - return write_object_file_flags(buf, len, type, oid, 0); + return write_object_file_flags(buf, len, type, oid, NULL, 0); } int write_object_file_literally(const void *buf, unsigned long len, From patchwork Wed Sep 27 19:55:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13401619 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 19B4FE82CCE for ; Wed, 27 Sep 2023 19:56:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229706AbjI0T4J (ORCPT ); Wed, 27 Sep 2023 15:56:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55432 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229721AbjI0T4D (ORCPT ); Wed, 27 Sep 2023 15:56:03 -0400 Received: from mail-io1-xd2d.google.com (mail-io1-xd2d.google.com [IPv6:2607:f8b0:4864:20::d2d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 490D611D for ; Wed, 27 Sep 2023 12:55:58 -0700 (PDT) Received: by mail-io1-xd2d.google.com with SMTP id ca18e2360f4ac-76c64da0e46so425665239f.0 for ; Wed, 27 Sep 2023 12:55:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1695844557; x=1696449357; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4CYL5/gskRTdwmKU9Ju2COcf8vtvsc9L64wzn6/3rx8=; b=mKzxPVn6seGfOiFPp5nP74wUpVs9MLednrS6IKs4db5BcPWgA8IFZm0rdNW9Do9tiL Qvv4OVpNzgbTqe4pEOYS7UOt4P1JOXfDhQWlIuTWzBa8bxBcQzmdbRJJB7kF2AnF3fub +wcEnpKlMJvtzvhH7UvvhJSOSfW4T5FZ1msWl6OMn+5484JGMcCayjEIaZUZNjeS/UDP SDfPluRgZ4vykdXK5yHt9/biF1GxdfnlOiH3ObhZwcOEcRtAN+0egwjW9mryOw5lh+Oy 5OesYS6N692PFkZR2EJXbUo7PbaByCV/aXOLbnnnusHIT8TjEBako+Y69HX2wsVZyJwH mAsw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695844557; x=1696449357; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4CYL5/gskRTdwmKU9Ju2COcf8vtvsc9L64wzn6/3rx8=; b=p8rsLurN6Lkl5Kxjo+XJArS5m6Qrrb8foJq8Rwp6fkJUFV4BMEs8Ze0dtIRwBTtkQ+ Ggx/Xni/b25QuL4c//xYfOOkQvby7C5AL2YLX9EAf/EYeE/u7UCgrkJ+vYWj52ClR2nl 8WIU8JQX5hPJhlpaf2eSz2yEJzIj2tIG9HMV8IgeQs08TfFAIj6Hsn+u5Q8FbssMnU3d tkWe5WQwVo/tIepFGPzmvchk+euxvSMbG3UVCy5AMvnhiy12BQm0EnoAofe49i9Sn3sq EC+VkPp6cBleE9gjpNko++c7Uw9tLp5K3z7yVKNYdFICtb2CIxGb5M45L/Wuetn30T50 J6+w== X-Gm-Message-State: AOJu0YwDBjBrkwHdg3U6vWTZNHGmhQ1b28Xr5m0GrwEH0G1r/XISKp4c G3sJGXRbd/b8vnhQIR/C5Uc= X-Google-Smtp-Source: AGHT+IHBalTRn7XtdowhPfzvHOD5JK7EnA2Q+ro1xDfB4HL0VK7SRDU3+2PNhFXB+6X9WuVlQXtHSw== X-Received: by 2002:a5e:8b0d:0:b0:79f:9b45:3365 with SMTP id g13-20020a5e8b0d000000b0079f9b453365mr2952851iok.17.1695844557583; Wed, 27 Sep 2023 12:55:57 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id o23-20020a6b5a17000000b007836c7e8dccsm2935628iob.17.2023.09.27.12.55.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Sep 2023 12:55:57 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , "Eric W . Biederman" Subject: [PATCH 09/30] commit: write commits for both hashes Date: Wed, 27 Sep 2023 14:55:16 -0500 Message-Id: <20230927195537.1682-9-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> References: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "brian m. carlson" When we write a commit, we include data that is specific to the hash algorithm, such as parents and the root tree. In order to write both a SHA-1 commit and a SHA-256 version, we need to convert between them. However, a straightforward conversion isn't necessarily what we want. When we sign a commit, we sign its data, so if we create a commit for SHA-256 and then write a SHA-1 version, we'll still have only signed the SHA-256 data. While this is valid, it would be better to sign both forms of data so people using SHA-1 can verify the signatures as well. Consequently, we don't want to use the standard mapping that occurs when we write an object. Instead, let's move most of the writing of the commit into a separate function which is agnostic of the hash algorithm and which simply writes into a buffer and specify both versions of the object ourselves. We can then call this function twice: once with the SHA-256 contents, and if SHA-1 is enabled, once with the SHA-1 contents. If we're signing the commit, we then sign both versions and append both signatures to both buffers. To produce a consistent hash, we always append the signatures in the order in which Git implemented them: first SHA-1, then SHA-256. In order to make this signing code work, we split the commit signing code into two functions, one which signs the buffer, and one which appends the signature. Signed-off-by: brian m. carlson Signed-off-by: Eric W. Biederman --- commit.c | 179 +++++++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 134 insertions(+), 45 deletions(-) diff --git a/commit.c b/commit.c index b3223478bc2a..46696ede8981 100644 --- a/commit.c +++ b/commit.c @@ -28,6 +28,7 @@ #include "shallow.h" #include "tree.h" #include "hook.h" +#include "object-file-convert.h" static struct commit_extra_header *read_commit_extra_header_lines(const char *buf, size_t len, const char **); @@ -1100,12 +1101,11 @@ static const char *gpg_sig_headers[] = { "gpgsig-sha256", }; -int sign_with_header(struct strbuf *buf, const char *keyid) +static int add_commit_signature(struct strbuf *buf, struct strbuf *sig, const struct git_hash_algo *algo) { - struct strbuf sig = STRBUF_INIT; int inspos, copypos; const char *eoh; - const char *gpg_sig_header = gpg_sig_headers[hash_algo_by_ptr(the_hash_algo)]; + const char *gpg_sig_header = gpg_sig_headers[hash_algo_by_ptr(algo)]; int gpg_sig_header_len = strlen(gpg_sig_header); /* find the end of the header */ @@ -1115,15 +1115,8 @@ int sign_with_header(struct strbuf *buf, const char *keyid) else inspos = eoh - buf->buf + 1; - if (!keyid || !*keyid) - keyid = get_signing_key(); - if (sign_buffer(buf, &sig, keyid)) { - strbuf_release(&sig); - return -1; - } - - for (copypos = 0; sig.buf[copypos]; ) { - const char *bol = sig.buf + copypos; + for (copypos = 0; sig->buf[copypos]; ) { + const char *bol = sig->buf + copypos; const char *eol = strchrnul(bol, '\n'); int len = (eol - bol) + !!*eol; @@ -1136,11 +1129,17 @@ int sign_with_header(struct strbuf *buf, const char *keyid) inspos += len; copypos += len; } - strbuf_release(&sig); return 0; } - +static int sign_commit_to_strbuf(struct strbuf *sig, struct strbuf *buf, const char *keyid) +{ + if (!keyid || !*keyid) + keyid = get_signing_key(); + if (sign_buffer(buf, sig, keyid)) + return -1; + return 0; +} int parse_signed_commit(const struct commit *commit, struct strbuf *payload, struct strbuf *signature, @@ -1599,70 +1598,160 @@ N_("Warning: commit message did not conform to UTF-8.\n" "You may want to amend it after fixing the message, or set the config\n" "variable i18n.commitEncoding to the encoding your project uses.\n"); -int commit_tree_extended(const char *msg, size_t msg_len, - const struct object_id *tree, - struct commit_list *parents, struct object_id *ret, - const char *author, const char *committer, - const char *sign_commit, - struct commit_extra_header *extra) +static void write_commit_tree(struct strbuf *buffer, const char *msg, size_t msg_len, + const struct object_id *tree, + const struct object_id *parents, size_t parents_len, + const char *author, const char *committer, + struct commit_extra_header *extra) { - int result; int encoding_is_utf8; - struct strbuf buffer; - - assert_oid_type(tree, OBJ_TREE); - - if (memchr(msg, '\0', msg_len)) - return error("a NUL byte in commit log message not allowed."); + size_t i; /* Not having i18n.commitencoding is the same as having utf-8 */ encoding_is_utf8 = is_encoding_utf8(git_commit_encoding); - strbuf_init(&buffer, 8192); /* should avoid reallocs for the headers */ - strbuf_addf(&buffer, "tree %s\n", oid_to_hex(tree)); + strbuf_init(buffer, 8192); /* should avoid reallocs for the headers */ + strbuf_addf(buffer, "tree %s\n", oid_to_hex(tree)); /* * NOTE! This ordering means that the same exact tree merged with a * different order of parents will be a _different_ changeset even * if everything else stays the same. */ - while (parents) { - struct commit *parent = pop_commit(&parents); - strbuf_addf(&buffer, "parent %s\n", - oid_to_hex(&parent->object.oid)); - } + for (i = 0; i < parents_len; i++) + strbuf_addf(buffer, "parent %s\n", oid_to_hex(&parents[i])); /* Person/date information */ if (!author) author = git_author_info(IDENT_STRICT); - strbuf_addf(&buffer, "author %s\n", author); + strbuf_addf(buffer, "author %s\n", author); if (!committer) committer = git_committer_info(IDENT_STRICT); - strbuf_addf(&buffer, "committer %s\n", committer); + strbuf_addf(buffer, "committer %s\n", committer); if (!encoding_is_utf8) - strbuf_addf(&buffer, "encoding %s\n", git_commit_encoding); + strbuf_addf(buffer, "encoding %s\n", git_commit_encoding); while (extra) { - add_extra_header(&buffer, extra); + add_extra_header(buffer, extra); extra = extra->next; } - strbuf_addch(&buffer, '\n'); + strbuf_addch(buffer, '\n'); /* And add the comment */ - strbuf_add(&buffer, msg, msg_len); + strbuf_add(buffer, msg, msg_len); +} - /* And check the encoding */ - if (encoding_is_utf8 && !verify_utf8(&buffer)) - fprintf(stderr, _(commit_utf8_warn)); +int commit_tree_extended(const char *msg, size_t msg_len, + const struct object_id *tree, + struct commit_list *parents, struct object_id *ret, + const char *author, const char *committer, + const char *sign_commit, + struct commit_extra_header *extra) +{ + struct repository *r = the_repository; + int result = 0; + int encoding_is_utf8; + struct strbuf buffer, compat_buffer; + struct strbuf sig = STRBUF_INIT, compat_sig = STRBUF_INIT; + struct object_id *parent_buf = NULL, *compat_oid = NULL; + struct object_id compat_oid_buf; + size_t i, nparents; + + /* Not having i18n.commitencoding is the same as having utf-8 */ + encoding_is_utf8 = is_encoding_utf8(git_commit_encoding); + + assert_oid_type(tree, OBJ_TREE); + + if (memchr(msg, '\0', msg_len)) + return error("a NUL byte in commit log message not allowed."); + + nparents = commit_list_count(parents); + parent_buf = xcalloc(nparents, sizeof(*parent_buf)); + for (i = 0; i < nparents; i++) { + struct commit *parent = pop_commit(&parents); + oidcpy(&parent_buf[i], &parent->object.oid); + } + + /* should avoid reallocs for the headers */ + strbuf_init(&buffer, 8192); + strbuf_init(&compat_buffer, 8192); - if (sign_commit && sign_with_header(&buffer, sign_commit)) { + write_commit_tree(&buffer, msg, msg_len, tree, parent_buf, nparents, author, committer, extra); + if (sign_commit && sign_commit_to_strbuf(&sig, &buffer, sign_commit)) { result = -1; goto out; } + if (r->compat_hash_algo) { + struct object_id mapped_tree; + struct object_id *mapped_parents = xcalloc(nparents, sizeof(*mapped_parents)); + if (repo_oid_to_algop(r, tree, r->compat_hash_algo, &mapped_tree)) { + result = -1; + free(mapped_parents); + goto out; + } + for (i = 0; i < nparents; i++) + if (repo_oid_to_algop(r, &parent_buf[i], r->compat_hash_algo, &mapped_parents[i])) { + result = -1; + free(mapped_parents); + goto out; + } + write_commit_tree(&compat_buffer, msg, msg_len, &mapped_tree, + mapped_parents, nparents, author, committer, extra); - result = write_object_file(buffer.buf, buffer.len, OBJ_COMMIT, ret); + if (sign_commit && sign_commit_to_strbuf(&compat_sig, &compat_buffer, sign_commit)) { + result = -1; + goto out; + } + } + + if (sign_commit) { + struct sig_pairs { + struct strbuf *sig; + const struct git_hash_algo *algo; + } bufs [2] = { + { &compat_sig, r->compat_hash_algo }, + { &sig, r->hash_algo }, + }; + int i; + + /* + * We write algorithms in the order they were implemented in + * Git to produce a stable hash when multiple algorithms are + * used. + */ + if (r->compat_hash_algo && hash_algo_by_ptr(bufs[0].algo) > hash_algo_by_ptr(bufs[1].algo)) + SWAP(bufs[0], bufs[1]); + + /* + * We traverse each algorithm in order, and apply the signature + * to each buffer. + */ + for (i = 0; i < ARRAY_SIZE(bufs); i++) { + if (!bufs[i].algo) + continue; + add_commit_signature(&buffer, bufs[i].sig, bufs[i].algo); + if (r->compat_hash_algo) + add_commit_signature(&compat_buffer, bufs[i].sig, bufs[i].algo); + } + } + + /* And check the encoding. */ + if (encoding_is_utf8 && (!verify_utf8(&buffer) || !verify_utf8(&compat_buffer))) + fprintf(stderr, _(commit_utf8_warn)); + + if (r->compat_hash_algo) { + hash_object_file(r->compat_hash_algo, compat_buffer.buf, compat_buffer.len, + OBJ_COMMIT, &compat_oid_buf); + compat_oid = &compat_oid_buf; + } + + result = write_object_file_flags(buffer.buf, buffer.len, OBJ_COMMIT, + ret, compat_oid, 0); out: strbuf_release(&buffer); + strbuf_release(&compat_buffer); + strbuf_release(&sig); + strbuf_release(&compat_sig); return result; } From patchwork Wed Sep 27 19:55:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13401620 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1813EE82CCC for ; Wed, 27 Sep 2023 19:56:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229793AbjI0T4M (ORCPT ); Wed, 27 Sep 2023 15:56:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55404 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229754AbjI0T4D (ORCPT ); Wed, 27 Sep 2023 15:56:03 -0400 Received: from mail-io1-xd2d.google.com (mail-io1-xd2d.google.com [IPv6:2607:f8b0:4864:20::d2d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0249911F for ; Wed, 27 Sep 2023 12:55:58 -0700 (PDT) Received: by mail-io1-xd2d.google.com with SMTP id ca18e2360f4ac-79fb64b5265so309748639f.1 for ; Wed, 27 Sep 2023 12:55:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1695844558; x=1696449358; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=XMgSh1BaVc3GwM9knHHc1gAtOQJjYyR6B0N6ASPctTI=; b=W8kQJQ8gOtwm0j7fCUe4R6HwWmz772fifTICab0tLiWen+EoIr8i7bbIe9UDEs5LNW Ze9TxnIrVHaolniv9O8anUw9XLTEI2hyV5TXbAAqFi96Bpdhc0JCcfBtWkaYI4H8om+U UoPQr57jOW844mJ4iGJV7sr69GO1/bakQgkcESB2bzeie8y2pG7RB+mhnG/WkH9YYueP An6fsGEcrUlF0NeTnxlhjk1ugl0XrEgYjRsD8ox7Fz03/AJSFsjJ9+//+QvYZ3VVTCge 7sSqlZ9cHQ+Rl2ODmvCVWSQfogFFKGHbd+Ks74645XihDnPGcnRt35nb7yAUjJ+PF3Op sU0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695844558; x=1696449358; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XMgSh1BaVc3GwM9knHHc1gAtOQJjYyR6B0N6ASPctTI=; b=mSVi7Tjmuyd4bwQvswmJ3GotFHaehnXZ8NG6LTUMBjlQuVGDkwQojrAvgFYGzPOx52 tz+l9UkM4iL0IyjPLm5G2uqR01WlMywUwBb/3C1qXBzc7ndntXBa1S1b0Fa1y5wNkPlo vdbla1jTAdHGmfa4HSFhB/rS60Cx+WoolpgFFz47tOSJZenjvdcfGDzI950EgxqfTCwY 00W5KCu7qarleoes2RJZ7u+JJYQEbx27976IkvxVnRJ6qajgnVl3fugfpo6aX4Oe3QX7 DI9gIBbyGrYXvSy+/4VQ4Fmtr0NPeRc0g/q9+NPyxL/Faj9reDcWAG4x92owA92Su/cE LE5g== X-Gm-Message-State: AOJu0YwWblnuRMfQIcAA07QpERpr7Ii9N3WboFBQmx7F3LEA3QrzZzNc Bes3wEY6I2hVn412YSIS38g= X-Google-Smtp-Source: AGHT+IF7mfCaWe4yh5eE2yMTebtgBObqeNpaMJH2LKbYva3cVeAHEXneRDSGQClXMphdDd8aVidsFg== X-Received: by 2002:a5d:9251:0:b0:792:9416:9c7f with SMTP id e17-20020a5d9251000000b0079294169c7fmr3353859iol.9.1695844558360; Wed, 27 Sep 2023 12:55:58 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id o23-20020a6b5a17000000b007836c7e8dccsm2935628iob.17.2023.09.27.12.55.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Sep 2023 12:55:57 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , "Eric W. Biederman" Subject: [PATCH 10/30] commit: Convert mergetag before computing the signature of a commit Date: Wed, 27 Sep 2023 14:55:17 -0500 Message-Id: <20230927195537.1682-10-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> References: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "Eric W. Biederman" It so happens that commit mergetag lines embed a tag object. So to compute the compatible signature of a commit object that has mergetag lines the compatible embedded tag must be computed first. Implement this by duplicating and converting the commit extra headers into the compatible version of the commit extra headers, that need to be passed to commit_tree_extended. To handle merge tags only the compatible extra headers need to be computed. Signed-off-by: Eric W. Biederman --- commit.c | 42 +++++++++++++++++++++++++++++++++++++++++- 1 file changed, 41 insertions(+), 1 deletion(-) diff --git a/commit.c b/commit.c index 46696ede8981..a6dac9a1957b 100644 --- a/commit.c +++ b/commit.c @@ -1355,6 +1355,39 @@ void append_merge_tag_headers(struct commit_list *parents, } } +static int convert_commit_extra_headers(struct commit_extra_header *orig, + struct commit_extra_header **result) +{ + const struct git_hash_algo *compat = the_repository->compat_hash_algo; + const struct git_hash_algo *algo = the_repository->hash_algo; + struct commit_extra_header *extra = NULL, **tail = &extra; + struct strbuf out = STRBUF_INIT; + while (orig) { + struct commit_extra_header *new; + CALLOC_ARRAY(new, 1); + if (!strcmp(orig->key, "mergetag")) { + if (convert_object_file(&out, algo, compat, + orig->value, orig->len, + OBJ_TAG, 1)) { + free(new); + free_commit_extra_headers(extra); + return -1; + } + new->key = xstrdup("mergetag"); + new->value = strbuf_detach(&out, &new->len); + } else { + new->key = xstrdup(orig->key); + new->len = orig->len; + new->value = xmemdupz(orig->value, orig->len); + } + *tail = new; + tail = &new->next; + orig = orig->next; + } + *result = extra; + return 0; +} + static void add_extra_header(struct strbuf *buffer, struct commit_extra_header *extra) { @@ -1682,6 +1715,7 @@ int commit_tree_extended(const char *msg, size_t msg_len, goto out; } if (r->compat_hash_algo) { + struct commit_extra_header *compat_extra = NULL; struct object_id mapped_tree; struct object_id *mapped_parents = xcalloc(nparents, sizeof(*mapped_parents)); if (repo_oid_to_algop(r, tree, r->compat_hash_algo, &mapped_tree)) { @@ -1695,8 +1729,14 @@ int commit_tree_extended(const char *msg, size_t msg_len, free(mapped_parents); goto out; } + if (convert_commit_extra_headers(extra, &compat_extra)) { + result = -1; + free(mapped_parents); + goto out; + } write_commit_tree(&compat_buffer, msg, msg_len, &mapped_tree, - mapped_parents, nparents, author, committer, extra); + mapped_parents, nparents, author, committer, compat_extra); + free_commit_extra_headers(compat_extra); if (sign_commit && sign_commit_to_strbuf(&compat_sig, &compat_buffer, sign_commit)) { result = -1; From patchwork Wed Sep 27 19:55:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13401621 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 41624E82CCE for ; Wed, 27 Sep 2023 19:56:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229795AbjI0T4O (ORCPT ); Wed, 27 Sep 2023 15:56:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55432 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229797AbjI0T4H (ORCPT ); Wed, 27 Sep 2023 15:56:07 -0400 Received: from mail-io1-xd30.google.com (mail-io1-xd30.google.com [IPv6:2607:f8b0:4864:20::d30]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B2183122 for ; Wed, 27 Sep 2023 12:55:59 -0700 (PDT) Received: by mail-io1-xd30.google.com with SMTP id ca18e2360f4ac-79f915e5b47so408194839f.2 for ; Wed, 27 Sep 2023 12:55:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1695844559; x=1696449359; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=102JTjdfwbYfwPCzdbqBjbe8OTap1CK+j6bzjqwhh/M=; b=T1Epyc6p6hmnquE+/adzvOAyK/jSU40VyLum4jKIvY9Ot+jCRXYbwAvw+VF9LYlm2O OVpVwBQobtzv5eDRUuKujWL/3oybd32hET1IloIB1icE7WkfvWGa7D6dg4Vh4tWpwoQj FGbdLkjUJ6mkZ6KB+S48MNu0KmJbClLytfcUQ+NgqEL+Wnl6Iwdw45vSgdQRPHq0q6K3 32OuWWZlwOF2oCqUAjqTR7LCfg3awez1RtnxBDVqI1ad6F/lwFTnc4xLjIvj1YabWSKc VHuQegHNcYOiONPhVXR7L5WzPMdp3Xw7/fVXKrDXDShhVOpL6Ymv8ckWaDjARDW3Su0h ZvFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695844559; x=1696449359; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=102JTjdfwbYfwPCzdbqBjbe8OTap1CK+j6bzjqwhh/M=; b=uqAX8SI19ScjFzTFvjUGDzn09PW7ypat8xKdKA6rLB0jCTFaZ9P+G9+I4rJhHh5mKJ VsDJHw7CVsUYx4tN8eoU4luUCl99hgrZZFV3u8zxHGmoPRT+6sV1DTHblj0Cbd00B7+b oGgziOEPuvbgqKgoOlLO0XZmPgR8VtNOAXGaH4WQSuJpXxldD3SezokdBmoEscx/8lUU fPYUqippFg5yoPvAg7sZhfYzT6wBd0kMGyDy++nXiiaWIBTQmU6gmpuHLjvEcjtgrcpf D0F5LqTWqIpwFYTNZlw1CTcZ8mLvk5vLAKz47b9zrGVyCvw2XfOFCqOA6mQ1lT3j1xUD CwlQ== X-Gm-Message-State: AOJu0YxtMrMwjf/QwK+WZPt1d6YJoy+qfvdr5TvD88ksHAW/J9hXTGB7 ClrjX+YInji8/DCZD8myPUo= X-Google-Smtp-Source: AGHT+IHGYYTDdsyZPdeRXcKyZLhXhTuyROgrFBCazNTvjJ0BwJQ+R/zVX5v/ceAFWj1AVwC/xWnnSg== X-Received: by 2002:a5e:a503:0:b0:791:60c:4b60 with SMTP id 3-20020a5ea503000000b00791060c4b60mr2766204iog.15.1695844559105; Wed, 27 Sep 2023 12:55:59 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id o23-20020a6b5a17000000b007836c7e8dccsm2935628iob.17.2023.09.27.12.55.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Sep 2023 12:55:58 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , "Eric W. Biederman" Subject: [PATCH 11/30] commit: Export add_header_signature to support handling signatures on tags Date: Wed, 27 Sep 2023 14:55:18 -0500 Message-Id: <20230927195537.1682-11-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> References: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "Eric W. Biederman" Rename add_commit_signature as add_header_signature, and expose it so that it can be used for converting tags from one object format to another. Inspired-by: brian m. carlson Signed-off-by: "Eric W. Biederman" --- commit.c | 6 +++--- commit.h | 1 + 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/commit.c b/commit.c index a6dac9a1957b..e75270171bc3 100644 --- a/commit.c +++ b/commit.c @@ -1101,7 +1101,7 @@ static const char *gpg_sig_headers[] = { "gpgsig-sha256", }; -static int add_commit_signature(struct strbuf *buf, struct strbuf *sig, const struct git_hash_algo *algo) +int add_header_signature(struct strbuf *buf, struct strbuf *sig, const struct git_hash_algo *algo) { int inspos, copypos; const char *eoh; @@ -1769,9 +1769,9 @@ int commit_tree_extended(const char *msg, size_t msg_len, for (i = 0; i < ARRAY_SIZE(bufs); i++) { if (!bufs[i].algo) continue; - add_commit_signature(&buffer, bufs[i].sig, bufs[i].algo); + add_header_signature(&buffer, bufs[i].sig, bufs[i].algo); if (r->compat_hash_algo) - add_commit_signature(&compat_buffer, bufs[i].sig, bufs[i].algo); + add_header_signature(&compat_buffer, bufs[i].sig, bufs[i].algo); } } diff --git a/commit.h b/commit.h index 28928833c544..03edcec0129f 100644 --- a/commit.h +++ b/commit.h @@ -370,5 +370,6 @@ int parse_buffer_signed_by_header(const char *buffer, struct strbuf *payload, struct strbuf *signature, const struct git_hash_algo *algop); +int add_header_signature(struct strbuf *buf, struct strbuf *sig, const struct git_hash_algo *algo); #endif /* COMMIT_H */ From patchwork Wed Sep 27 19:55:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13401622 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2576EE82CCD for ; Wed, 27 Sep 2023 19:56:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229800AbjI0T4P (ORCPT ); Wed, 27 Sep 2023 15:56:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50360 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229802AbjI0T4H (ORCPT ); Wed, 27 Sep 2023 15:56:07 -0400 Received: from mail-io1-xd33.google.com (mail-io1-xd33.google.com [IPv6:2607:f8b0:4864:20::d33]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 72560194 for ; Wed, 27 Sep 2023 12:56:00 -0700 (PDT) Received: by mail-io1-xd33.google.com with SMTP id ca18e2360f4ac-79f9acc857cso368633339f.2 for ; Wed, 27 Sep 2023 12:56:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1695844560; x=1696449360; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=WkhLsqo78Srsesp5Qst5wXtDzgDVzwUYy+O/4Wvj/rI=; b=MucLKlWE9wzdmO/8g1QmHYK5EHCAixSH+RqGehUzcRNAk+ARTXRpIzw2dRXRoBXgGY 6PzzN/GmDAnE3fkjvsQBy0NzrmnLyY6t9FC9BDHk6FWtz4o0kNd8siXxfo/3HTFBSZDF wrGgz944YHVI6/Kr2oVNBWTKfnNZqKub/G3TDFI7XecNuDxf7QN4G3f3ZSg46G7AUjWe ArKaOaYsnMkoYEtW5ydTaBV6S97QP69YYviy3mCWgdOPtHofIUc8tvnV82CEv52G/gvp HuQL+u3oYtLpxBIOnfoUHOTUTy7LukMGKWLNm80H3US1B0zKPZ6aYmeIBiphWHDiF466 bkPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695844560; x=1696449360; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=WkhLsqo78Srsesp5Qst5wXtDzgDVzwUYy+O/4Wvj/rI=; b=PMPHcfahLq1MYeY2DigsqT1Q8fG5yIFyi5A60LzM9P7g6Q1o15Uw90Ikox4xs1uu+8 71kqsNNHV+U6/5LVt7SdJXsuyFY7K3ueUBPOE1lT1Z7JDOx7e3IE8ukXuSsmZNJC647E iw8rNMQ8u0T4nveVc5fOEoU9UAHwFVa0R8HrtxYHTDVaRHgI8LVAMtmzHTlSo8oyfjod J/hVoPtWV5JE7kxMpsuyr9JOy0GhBamHPyuXUPOllsC7VpzZgx1byVq2k10M/Nw2BCK7 YUDwAnWNE4Cszi+y6E5csEnoT4rApiOqNODHjH5OrRqJqekeO7MwJWrpmXrE1NcB4iN1 0gbQ== X-Gm-Message-State: AOJu0YwTq1hO9+XB1HoLBbExyeWYck651A62wmRMy5iB+aPmFbjdbPWg w5y2JUbxjc5Evjop8InB9wk= X-Google-Smtp-Source: AGHT+IFnu5LD9rgA6y3mSKl/XdJ8FdaKzOEDT1sv7Sl8R/RQ7zTP+o5ngjJEQlWFDzOaYU+E4dwGVA== X-Received: by 2002:a6b:4902:0:b0:780:d76c:b640 with SMTP id u2-20020a6b4902000000b00780d76cb640mr2966545iob.9.1695844559802; Wed, 27 Sep 2023 12:55:59 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id o23-20020a6b5a17000000b007836c7e8dccsm2935628iob.17.2023.09.27.12.55.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Sep 2023 12:55:59 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , "Eric W. Biederman" Subject: [PATCH 12/30] tag: sign both hashes Date: Wed, 27 Sep 2023 14:55:19 -0500 Message-Id: <20230927195537.1682-12-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> References: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "Eric W. Biederman" When we write a tag the object oid is specific to the hash algorithm. This matters when a tag is signed. The hash transition plan calls for signatures on both the sha1 form and the sha256 form of the object, and for both of those signatures to live in the tag object. To generate tag object with multiple signatures, first compute the unsigned form of the tag, and then if the tag is being signed compute the unsigned form of the tag with the compatibilityr hash. Then compute compute the signatures of both buffers. Once the signatures are computed add them to both buffers. This allows computing the compatibility hash in do_sign, saving write_object_file the expense of recomputing the compatibility tag just to compute it's hash. Signed-off-by: Eric W. Biederman --- builtin/tag.c | 45 +++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 41 insertions(+), 4 deletions(-) diff --git a/builtin/tag.c b/builtin/tag.c index 3918eacbb57b..8c4bc28952c2 100644 --- a/builtin/tag.c +++ b/builtin/tag.c @@ -28,6 +28,7 @@ #include "ref-filter.h" #include "date.h" #include "write-or-die.h" +#include "object-file-convert.h" static const char * const git_tag_usage[] = { N_("git tag [-a | -s | -u ] [-f] [-m | -F ] [-e]\n" @@ -174,9 +175,43 @@ static int verify_tag(const char *name, const char *ref UNUSED, return 0; } -static int do_sign(struct strbuf *buffer) +static int do_sign(struct strbuf *buffer, struct object_id **compat_oid, + struct object_id *compat_oid_buf) { - return sign_buffer(buffer, buffer, get_signing_key()); + const struct git_hash_algo *compat = the_repository->compat_hash_algo; + struct strbuf sig = STRBUF_INIT, compat_sig = STRBUF_INIT; + struct strbuf compat_buf = STRBUF_INIT; + const char *keyid = get_signing_key(); + int ret = -1; + + if (sign_buffer(buffer, &sig, keyid)) + return -1; + + if (compat) { + const struct git_hash_algo *algo = the_repository->hash_algo; + + if (convert_object_file(&compat_buf, algo, compat, + buffer->buf, buffer->len, OBJ_TAG, 1)) + goto out; + if (sign_buffer(&compat_buf, &compat_sig, keyid)) + goto out; + add_header_signature(&compat_buf, &sig, algo); + strbuf_addbuf(&compat_buf, &compat_sig); + hash_object_file(compat, compat_buf.buf, compat_buf.len, + OBJ_TAG, compat_oid_buf); + *compat_oid = compat_oid_buf; + } + + if (compat_sig.len) + add_header_signature(buffer, &compat_sig, compat); + + strbuf_addbuf(buffer, &sig); + ret = 0; +out: + strbuf_release(&sig); + strbuf_release(&compat_sig); + strbuf_release(&compat_buf); + return ret; } static const char tag_template[] = @@ -249,9 +284,11 @@ static void write_tag_body(int fd, const struct object_id *oid) static int build_tag_object(struct strbuf *buf, int sign, struct object_id *result) { - if (sign && do_sign(buf) < 0) + struct object_id *compat_oid = NULL, compat_oid_buf; + if (sign && do_sign(buf, &compat_oid, &compat_oid_buf) < 0) return error(_("unable to sign the tag")); - if (write_object_file(buf->buf, buf->len, OBJ_TAG, result) < 0) + if (write_object_file_flags(buf->buf, buf->len, OBJ_TAG, result, + compat_oid, 0) < 0) return error(_("unable to write tag file")); return 0; } From patchwork Wed Sep 27 19:55:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13401624 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4DCABE82CCC for ; Wed, 27 Sep 2023 19:56:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229854AbjI0T4Y (ORCPT ); Wed, 27 Sep 2023 15:56:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55432 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229811AbjI0T4H (ORCPT ); Wed, 27 Sep 2023 15:56:07 -0400 Received: from mail-io1-xd31.google.com (mail-io1-xd31.google.com [IPv6:2607:f8b0:4864:20::d31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 36C3B199 for ; Wed, 27 Sep 2023 12:56:01 -0700 (PDT) Received: by mail-io1-xd31.google.com with SMTP id ca18e2360f4ac-79fa425855cso347633639f.2 for ; Wed, 27 Sep 2023 12:56:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1695844560; x=1696449360; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=rjIQESPk0o8Npu7i1ELfD9lAS93ZgVT64m2daSkMins=; b=S3fviLbHS8Xj0RKNxXdcSlTtS+s/jJ4R0eOkGgxUqStW685LkBBCOJ7JVXqVrlOH0B 7lLs4jrSppJ0BOpexY+xapreOGjRKtfl8rTQZUFw+RInQfRbl2Hy6tYzigEoNxf5orxC KarAylT6uWNn/4oHo+B0IJN3yv6l5hBrlpHXYdZKELvEpxq97wwSd62UHPCFBENQdRcG HZzALBS39cFFnbPvAKRBT3cvQpC6dNjpxs6nDMpDxg9aJe7l8vBkyA4CFnA61q46vwOu A8+bvUVQ09y/Kvt/DZslRdZApeMfSgBmv5bVPZvi7GUrqq7oNX9AEIvszMVsjtuTcB0k BT3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695844560; x=1696449360; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=rjIQESPk0o8Npu7i1ELfD9lAS93ZgVT64m2daSkMins=; b=XOTdPA60D622Xu8bDe2pbbliV3yjSNY4ksx5bpzOH6VE7LUAmekLmiyf7+btja7NIO AdA2xHz/6jB79tP0NZiXnwHvAo9MACHwJtRF4szFo6CGCAqZEzb5YCJoh+LBYMWPE1Iz qBn4awt55enQYuyqk/n2FJWmf1XoY42xyj3O00XJsMixGGSXLrR+UbhsA/nPNhaPj178 qPWv+WnxAK2pouuA4SjkStU8NKXdAQbfu3OW7vvxiwauGn7oKm8uHHYJ5gmkQhkXwIRU A0Tf6UTIQkIob866mtC2n1bAxorc6PdK+Xnu5RugM4MmEkDgkFt0AWoCrnqJYIbANdWj vG/A== X-Gm-Message-State: AOJu0YxLpFvQtJAg+kWsk5cdXEr9UlOeheHujz8+Z9xDTFVjwBa5x9P2 6V/T8NTnb1pmDYOZ8QrhzUiiVsTbgf8= X-Google-Smtp-Source: AGHT+IEcGoQmH5/VbnhNtAyFTHwh6d2WjynJFcWnLQc7IduD3KSAsvWNgSdLKxOEhgfzEgEQfJXgrQ== X-Received: by 2002:a6b:fe14:0:b0:787:4b5f:b6cf with SMTP id x20-20020a6bfe14000000b007874b5fb6cfmr3115746ioh.5.1695844560517; Wed, 27 Sep 2023 12:56:00 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id o23-20020a6b5a17000000b007836c7e8dccsm2935628iob.17.2023.09.27.12.55.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Sep 2023 12:56:00 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , "Eric W. Biederman" Subject: [PATCH 13/30] cache: add a function to read an OID of a specific algorithm Date: Wed, 27 Sep 2023 14:55:20 -0500 Message-Id: <20230927195537.1682-13-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> References: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "brian m. carlson" Currently, we always read a object ID of the current algorithm with oidread. However, once we start converting objects, we'll need to consider what happens when we want to read an object ID of a specific algorithm, such as the compatibility algorithm. To make this easier, let's define oidread_algop, which specifies which algorithm we should use for our object ID, and define oidread in terms of it. Signed-off-by: brian m. carlson Signed-off-by: "Eric W. Biederman" --- hash.h | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/hash.h b/hash.h index 615ae0691d07..e064807c1733 100644 --- a/hash.h +++ b/hash.h @@ -73,10 +73,15 @@ static inline void oidclr(struct object_id *oid) oid->algo = hash_algo_by_ptr(the_hash_algo); } +static inline void oidread_algop(struct object_id *oid, const unsigned char *hash, const struct git_hash_algo *algop) +{ + memcpy(oid->hash, hash, algop->rawsz); + oid->algo = hash_algo_by_ptr(algop); +} + static inline void oidread(struct object_id *oid, const unsigned char *hash) { - memcpy(oid->hash, hash, the_hash_algo->rawsz); - oid->algo = hash_algo_by_ptr(the_hash_algo); + oidread_algop(oid, hash, the_hash_algo); } static inline int is_empty_blob_sha1(const unsigned char *sha1) From patchwork Wed Sep 27 19:55:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13401623 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A3540E82CCE for ; Wed, 27 Sep 2023 19:56:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229770AbjI0T4V (ORCPT ); Wed, 27 Sep 2023 15:56:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55456 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229807AbjI0T4H (ORCPT ); Wed, 27 Sep 2023 15:56:07 -0400 Received: from mail-io1-xd2e.google.com (mail-io1-xd2e.google.com [IPv6:2607:f8b0:4864:20::d2e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 09F651A8 for ; Wed, 27 Sep 2023 12:56:01 -0700 (PDT) Received: by mail-io1-xd2e.google.com with SMTP id ca18e2360f4ac-79fa5d9f3a2so345792439f.3 for ; Wed, 27 Sep 2023 12:56:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1695844561; x=1696449361; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=FkazeKVM1RtP9Z70mx3eJTcTbac/plstEPcnoAezcv4=; b=CqNlzW8ZWz3V+7uyb9MjjtFfIUIBy8gVyc5QIofae8VHUNVS+MN8lJzyZFj8yCkoT2 iDdz8Zd5dZUAMNhKKiuuM+lG5YkC/deIcpEZIZzNrSl+ElksIYBmjsS1LRh0Fugofrtn OeQ54NtYAG5Dpa01wIeOMxqnNP30TEO2xAvq2tMlWKwAW1BQ0qwXyv/E5+lSQCeWTVET NekitbqY6zGI1wPaBF+taE3NbkCRVQ5E2QxazNG+/lXy1o/T9kcmKKdwBUEGHSnrdJEp ci6XWy6wN26yBKWuy6UZwkxNcMAGrx8DbqM4lNvGVkyiqKREcP2emuY4DP+Xa8elf2Fm 317A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695844561; x=1696449361; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FkazeKVM1RtP9Z70mx3eJTcTbac/plstEPcnoAezcv4=; b=fd5eEB7knNo/clvZcArIkTOAFq5WMTXZ7HXcmn6Z34GwHl72z0S9XI2NR1+U73gvWy m+72u3wIvRjW1NvR5fg6BBPitM+Mgc8PCr9z9rcqP3h5F4JZuSbewhDrRTtk33COM4he AQ/N+wLwTGN85/ty9c0Z3bgRvkRdc9Mptugbgs6gBGWILa2GsFFGgW1qT6TsP7EM7m5I ltcgqz5/4nIs77X3iviRamXdRwVaRvCZPITykoqgV80mIpbv688SqPHjYUeSEdHxyGXY 8kmTfiM+SlOzI/HIMt3ZDli88ksBfJAh5GWRM2BDas9KAvef2NQNHR7OZwMHSgZW7wbQ nArg== X-Gm-Message-State: AOJu0YyOVVkyHyRmShSgYs46p5H6QG2WPZR2ymSGPtNF0CUcBAFNj2FQ S1MteQm5dOluWpWknk6mmYY= X-Google-Smtp-Source: AGHT+IE7J/P/XkNeI/Xa6OOD8qbn/3pl5ygyfNPL9ZNV1vDBKQKOyd4MygSxmB4mY0uFSEiBZb6NtQ== X-Received: by 2002:a6b:fe14:0:b0:787:4b5f:b6cf with SMTP id x20-20020a6bfe14000000b007874b5fb6cfmr3115763ioh.5.1695844561172; Wed, 27 Sep 2023 12:56:01 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id o23-20020a6b5a17000000b007836c7e8dccsm2935628iob.17.2023.09.27.12.56.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Sep 2023 12:56:00 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , "Eric W. Biederman" Subject: [PATCH 14/30] object: Factor out parse_mode out of fast-import and tree-walk into in object.h Date: Wed, 27 Sep 2023 14:55:21 -0500 Message-Id: <20230927195537.1682-14-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> References: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "Eric W. Biederman" builtin/fast-import.c and tree-walk.c have almost identical version of get_mode. The two functions started out the same but have diverged slightly. The version in fast-import changed mode to a uint16_t to save memory. The version in tree-walk started erroring if no mode was present. As far as I can tell both of these changes are valid for both of the callers, so add the both changes and place the common parsing helper in object.h Rename the helper from get_mode to parse_mode so it does not conflict with another helper named get_mode in diff-no-index.c This will be used shortly in a new helper decode_tree_entry_raw which is used to compute cmpatibility objects as part of the sha256 transition. Signed-off-by: "Eric W. Biederman" --- builtin/fast-import.c | 18 ++---------------- object.h | 18 ++++++++++++++++++ tree-walk.c | 22 +++------------------- 3 files changed, 23 insertions(+), 35 deletions(-) diff --git a/builtin/fast-import.c b/builtin/fast-import.c index 4dbb10aff3da..2c645fcfbe3f 100644 --- a/builtin/fast-import.c +++ b/builtin/fast-import.c @@ -1235,20 +1235,6 @@ static void *gfi_unpack_entry( return unpack_entry(the_repository, p, oe->idx.offset, &type, sizep); } -static const char *get_mode(const char *str, uint16_t *modep) -{ - unsigned char c; - uint16_t mode = 0; - - while ((c = *str++) != ' ') { - if (c < '0' || c > '7') - return NULL; - mode = (mode << 3) + (c - '0'); - } - *modep = mode; - return str; -} - static void load_tree(struct tree_entry *root) { struct object_id *oid = &root->versions[1].oid; @@ -1286,7 +1272,7 @@ static void load_tree(struct tree_entry *root) t->entries[t->entry_count++] = e; e->tree = NULL; - c = get_mode(c, &e->versions[1].mode); + c = parse_mode(c, &e->versions[1].mode); if (!c) die("Corrupt mode in %s", oid_to_hex(oid)); e->versions[0].mode = e->versions[1].mode; @@ -2275,7 +2261,7 @@ static void file_change_m(const char *p, struct branch *b) struct object_id oid; uint16_t mode, inline_data = 0; - p = get_mode(p, &mode); + p = parse_mode(p, &mode); if (!p) die("Corrupt mode: %s", command_buf.buf); switch (mode) { diff --git a/object.h b/object.h index 114d45954d08..70c8d4ae63dc 100644 --- a/object.h +++ b/object.h @@ -190,6 +190,24 @@ void *create_object(struct repository *r, const struct object_id *oid, void *obj void *object_as_type(struct object *obj, enum object_type type, int quiet); + +static inline const char *parse_mode(const char *str, uint16_t *modep) +{ + unsigned char c; + unsigned int mode = 0; + + if (*str == ' ') + return NULL; + + while ((c = *str++) != ' ') { + if (c < '0' || c > '7') + return NULL; + mode = (mode << 3) + (c - '0'); + } + *modep = mode; + return str; +} + /* * Returns the object, having parsed it to find out what it is. * diff --git a/tree-walk.c b/tree-walk.c index 29ead71be173..3af50a01c2c7 100644 --- a/tree-walk.c +++ b/tree-walk.c @@ -10,27 +10,11 @@ #include "pathspec.h" #include "json-writer.h" -static const char *get_mode(const char *str, unsigned int *modep) -{ - unsigned char c; - unsigned int mode = 0; - - if (*str == ' ') - return NULL; - - while ((c = *str++) != ' ') { - if (c < '0' || c > '7') - return NULL; - mode = (mode << 3) + (c - '0'); - } - *modep = mode; - return str; -} - static int decode_tree_entry(struct tree_desc *desc, const char *buf, unsigned long size, struct strbuf *err) { const char *path; - unsigned int mode, len; + unsigned int len; + uint16_t mode; const unsigned hashsz = the_hash_algo->rawsz; if (size < hashsz + 3 || buf[size - (hashsz + 1)]) { @@ -38,7 +22,7 @@ static int decode_tree_entry(struct tree_desc *desc, const char *buf, unsigned l return -1; } - path = get_mode(buf, &mode); + path = parse_mode(buf, &mode); if (!path) { strbuf_addstr(err, _("malformed mode in tree entry")); return -1; From patchwork Wed Sep 27 19:55:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13401625 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 106E1E82CCD for ; Wed, 27 Sep 2023 19:56:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229872AbjI0T41 (ORCPT ); Wed, 27 Sep 2023 15:56:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50400 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229789AbjI0T4H (ORCPT ); Wed, 27 Sep 2023 15:56:07 -0400 Received: from mail-io1-xd34.google.com (mail-io1-xd34.google.com [IPv6:2607:f8b0:4864:20::d34]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B8C321B1 for ; Wed, 27 Sep 2023 12:56:02 -0700 (PDT) Received: by mail-io1-xd34.google.com with SMTP id ca18e2360f4ac-79fe6da0049so125355839f.1 for ; Wed, 27 Sep 2023 12:56:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1695844562; x=1696449362; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=r2xlBxGjJYJSOp2/X8A6tuYZr6hMyLXPzax8OFExRmc=; b=GgKzAEr32YuKh+QzJn75RgYNf8wiBwoy3nbWWtnJ2c7OM/OnC350Y1Gr9BenCcPGJL oajMdLnpyZoz21IEiajSzD5K9wO9lTAP4pUh1i1P8oAteBNC6YnylmsrmQJ2hHXHGUuR ByM0SIOB6TOcnPuNN4MB4sCkU82hktDsV8xvEus45oHUwuCK07LMPLne1J4/ffSCpTgh V7Hs1WWwRuRAzpDSiXGOD5L8oQP+zrX/75L4w4hT49TL/NuJx6ev5rhTMGMjQ0omqrKn +niurg/ScLHGyfPcKfZ8+mBsb8eGCj+7V0e2qvoXzt1Rz4bqISXIp2Xqsv+nCNfUEYsF XbYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695844562; x=1696449362; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=r2xlBxGjJYJSOp2/X8A6tuYZr6hMyLXPzax8OFExRmc=; b=Q1Y8y5jK+YZ7zWKS0qLgNgyxfMYLknMWv/uKcJtq+BhMXGprZU8PZ/t6kzynIl4N8f 2XqAD55sR7WTh2WXwkJSpgiKzTEqC1NIZKWGG4eMYHUugAQDQCpsaAY7qlN8oM2bPGUT 8O3no/lH0WrMTPGpEP+LjdEpYdzQyKAiKJkli/24I5p1EckcyPFhDQewMKsl7XcYQWbQ TbiryRZ3zTV2KGwQo+/alKNI2UC0v99709+l7NHk5bW50MkSYwlIwIll/qq4hEGfXye9 Ou9k/4dc+jz640AbDphgLNou5f/cKoPVTXL7UKGN+Iidz0YTFyx4C1ZQU+ubhu+mHr4E d7hA== X-Gm-Message-State: AOJu0YwTK/degxjhLJphdEcSoXfYaJ5vuGhFdinq+bHGQkAxuRtCIbF7 7wbXZ07/ljWsiPiusPPqOl8= X-Google-Smtp-Source: AGHT+IGEveuUAoMV796UogOJ50zCOP0FRN8LZ2GudBqVX3cHjhRlxURL+VHPYr2poyOQtQj+dBKt6A== X-Received: by 2002:a05:6602:2566:b0:79f:d4e6:5175 with SMTP id dj6-20020a056602256600b0079fd4e65175mr3237439iob.16.1695844561892; Wed, 27 Sep 2023 12:56:01 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id o23-20020a6b5a17000000b007836c7e8dccsm2935628iob.17.2023.09.27.12.56.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Sep 2023 12:56:01 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , "Eric W . Biederman" Subject: [PATCH 15/30] object-file-convert: add a function to convert trees between algorithms Date: Wed, 27 Sep 2023 14:55:22 -0500 Message-Id: <20230927195537.1682-15-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> References: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "brian m. carlson" In the future, we're going to want to provide SHA-256 repositories that have compatibility support for SHA-1 as well. In order to do so, we'll need to be able to convert tree objects from SHA-256 to SHA-1 by writing a tree with each SHA-256 object ID mapped to a SHA-1 object ID. We implement a function, convert_tree_object, that takes an existing tree buffer and writes it to a new strbuf, converting between algorithms. Let's make this function generic, because while we only need it to convert from the main algorithm to the compatibility algorithm now, we may need to do the other way around in the future, such as for transport. We avoid reusing the code in decode_tree_entry because that code normalizes data, and we don't want that here. We want to produce a complete round trip of data, so if, for example, the old entry had a wrongly zero-padded mode, we'd want to preserve that when converting to ensure a stable hash value. Signed-off-by: brian m. carlson Signed-off-by: Eric W. Biederman --- object-file-convert.c | 51 ++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 50 insertions(+), 1 deletion(-) diff --git a/object-file-convert.c b/object-file-convert.c index 4d62ed192bf0..a9e7a208a707 100644 --- a/object-file-convert.c +++ b/object-file-convert.c @@ -1,8 +1,10 @@ #include "git-compat-util.h" #include "gettext.h" #include "strbuf.h" +#include "hex.h" #include "repository.h" #include "hash-ll.h" +#include "hash.h" #include "object.h" #include "loose.h" #include "object-file-convert.h" @@ -36,6 +38,51 @@ int repo_oid_to_algop(struct repository *repo, const struct object_id *src, return 0; } +static int decode_tree_entry_raw(struct object_id *oid, const char **path, + size_t *len, const struct git_hash_algo *algo, + const char *buf, unsigned long size) +{ + uint16_t mode; + const unsigned hashsz = algo->rawsz; + + if (size < hashsz + 3 || buf[size - (hashsz + 1)]) { + return -1; + } + + *path = parse_mode(buf, &mode); + if (!*path || !**path) + return -1; + *len = strlen(*path) + 1; + + oidread_algop(oid, (const unsigned char *)*path + *len, algo); + return 0; +} + +static int convert_tree_object(struct strbuf *out, + const struct git_hash_algo *from, + const struct git_hash_algo *to, + const char *buffer, size_t size) +{ + const char *p = buffer, *end = buffer + size; + + while (p < end) { + struct object_id entry_oid, mapped_oid; + const char *path = NULL; + size_t pathlen; + + if (decode_tree_entry_raw(&entry_oid, &path, &pathlen, from, p, + end - p)) + return error(_("failed to decode tree entry")); + if (repo_oid_to_algop(the_repository, &entry_oid, to, &mapped_oid)) + return error(_("failed to map tree entry for %s"), oid_to_hex(&entry_oid)); + strbuf_add(out, p, path - p); + strbuf_add(out, path, pathlen); + strbuf_add(out, mapped_oid.hash, to->rawsz); + p = path + pathlen + from->rawsz; + } + return 0; +} + int convert_object_file(struct strbuf *outbuf, const struct git_hash_algo *from, const struct git_hash_algo *to, @@ -50,8 +97,10 @@ int convert_object_file(struct strbuf *outbuf, die("Refusing noop object file conversion"); switch (type) { - case OBJ_COMMIT: case OBJ_TREE: + ret = convert_tree_object(outbuf, from, to, buf, len); + break; + case OBJ_COMMIT: case OBJ_TAG: default: /* Not implemented yet, so fail. */ From patchwork Wed Sep 27 19:55:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13401626 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 277BEE82CCC for ; Wed, 27 Sep 2023 19:56:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229899AbjI0T4e (ORCPT ); Wed, 27 Sep 2023 15:56:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50436 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229821AbjI0T4I (ORCPT ); Wed, 27 Sep 2023 15:56:08 -0400 Received: from mail-io1-xd32.google.com (mail-io1-xd32.google.com [IPv6:2607:f8b0:4864:20::d32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 72EDB10E for ; Wed, 27 Sep 2023 12:56:03 -0700 (PDT) Received: by mail-io1-xd32.google.com with SMTP id ca18e2360f4ac-79fe87cd74eso115130639f.3 for ; Wed, 27 Sep 2023 12:56:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1695844562; x=1696449362; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=NxSownEw2GFkZ4s2nCmQWhrw2EH8D8Q1mcAw7DgyobA=; b=mo1KVCBFifmnL0i4C5b9H0vGR7ZP/+krIkxDzzB2fRP195XLvKEZkfgSsf0dHYV85z cjKsdN7UQn2P9LqOgQ/8PXknXxEP7JQjwIeauC9hRXkTayWHAiZj5E1UJC/wSWm+NxSI f+0s+8z34PxmSQDkUapzPgOHvTzwGPRm51nzhjJ0jh7yaGZLHI+Vf+WrLFqQ64F1lu0S jeaRwXboiLO7PkilmDk9T23vbwUP5uvz53xGCRbq1JyrWYq2gQaHUrIY9z/qbwIz32lj J6fK7GthcQPP2fnr6Sq07d0ofulXWDe0Ou32LlisGl7KvQjpgn1eTQe/ZBf4eDRWFljf ETaw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695844562; x=1696449362; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=NxSownEw2GFkZ4s2nCmQWhrw2EH8D8Q1mcAw7DgyobA=; b=CfW22o7EbbcyVYEByU7eOvzvtBzLvCgnKXEkFPwtIPoU4VvA45HpHnrrifNC1tr44C F2CctI4PpnIdOmu6lmRWwLZk6FYdcNMO9mqFKzKxpgVmVp9eviVS5OlvR80hHg1DhAbS +C2zicqu339oj7KTHfXvdTpdoZChRj+oxfpDkEjjqMKxNaZy1YXfMR/Vyhu2xTY8XkzS qScSzpJKKmZf4bVZJnIGUnDq+Xoeed6BaSZ/dYfo6Cqt7T5cq4lejNrNxGfI3z9URtYf ulPDoumwSvqUhvJ3avAOx4gOZ/mAS2siJihNn4GV38/LdWW2K7AEZQH0Mjt5yQx1gUQv L5LQ== X-Gm-Message-State: AOJu0YyJBc646Th+Eklt/c/l3ME5qz2R27O0vj8fdQlMHzj0DDkzuwxc iZIKLkpEJfeHixfQ3JWf5W4= X-Google-Smtp-Source: AGHT+IG4rywSWRMygJ5YmDaNpqHxWkild2bQhAJsJ7bfFt2NpZWbtQRRWMLj/VajUEXa+BxNG3HOUA== X-Received: by 2002:a6b:dc0c:0:b0:791:2db5:c779 with SMTP id s12-20020a6bdc0c000000b007912db5c779mr3499970ioc.10.1695844562528; Wed, 27 Sep 2023 12:56:02 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id o23-20020a6b5a17000000b007836c7e8dccsm2935628iob.17.2023.09.27.12.56.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Sep 2023 12:56:02 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , "Eric W . Biederman" Subject: [PATCH 16/30] object-file-convert: convert tag objects when writing Date: Wed, 27 Sep 2023 14:55:23 -0500 Message-Id: <20230927195537.1682-16-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> References: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "brian m. carlson" When writing a tag object in a repository with both SHA-1 and SHA-256, we'll need to convert our commit objects so that we can write the hash values for both into the repository. To do so, let's add a function to convert tag objects. Note that signatures for tag objects in the current algorithm trail the message, and those for the alternate algorithm are in headers. Therefore, we parse the tag object for both a trailing signature and a header and then, when writing the other format, swap the two around. Signed-off-by: brian m. carlson Signed-off-by: Eric W. Biederman --- object-file-convert.c | 52 ++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 51 insertions(+), 1 deletion(-) diff --git a/object-file-convert.c b/object-file-convert.c index a9e7a208a707..777ae5b58036 100644 --- a/object-file-convert.c +++ b/object-file-convert.c @@ -7,6 +7,8 @@ #include "hash.h" #include "object.h" #include "loose.h" +#include "commit.h" +#include "gpg-interface.h" #include "object-file-convert.h" int repo_oid_to_algop(struct repository *repo, const struct object_id *src, @@ -83,6 +85,52 @@ static int convert_tree_object(struct strbuf *out, return 0; } +static int convert_tag_object(struct strbuf *out, + const struct git_hash_algo *from, + const struct git_hash_algo *to, + const char *buffer, size_t size) +{ + struct strbuf payload = STRBUF_INIT, temp = STRBUF_INIT, oursig = STRBUF_INIT, othersig = STRBUF_INIT; + size_t payload_size; + struct object_id oid, mapped_oid; + const char *p; + + /* Add some slop for longer signature header in the new algorithm. */ + strbuf_grow(out, size + 7); + + /* Is there a signature for our algorithm? */ + payload_size = parse_signed_buffer(buffer, size); + strbuf_add(&payload, buffer, payload_size); + if (payload_size != size) { + /* Yes, there is. */ + strbuf_add(&oursig, buffer + payload_size, size - payload_size); + } + /* Now, is there a signature for the other algorithm? */ + if (parse_buffer_signed_by_header(payload.buf, payload.len, &temp, &othersig, to)) { + /* Yes, there is. */ + strbuf_swap(&payload, &temp); + strbuf_release(&temp); + } + + /* + * Our payload is now in payload and we may have up to two signatrures + * in oursig and othersig. + */ + if (strncmp(payload.buf, "object ", 7) || payload.buf[from->hexsz + 7] != '\n') + return error("bogus tag object"); + if (parse_oid_hex_algop(payload.buf + 7, &oid, &p, from) < 0) + return error("bad tag object ID"); + if (repo_oid_to_algop(the_repository, &oid, to, &mapped_oid)) + return error("unable to map tree %s in tag object", + oid_to_hex(&oid)); + strbuf_addf(out, "object %s", oid_to_hex(&mapped_oid)); + strbuf_add(out, p, payload.len - (p - payload.buf)); + strbuf_addbuf(out, &othersig); + if (oursig.len) + add_header_signature(out, &oursig, from); + return 0; +} + int convert_object_file(struct strbuf *outbuf, const struct git_hash_algo *from, const struct git_hash_algo *to, @@ -100,8 +148,10 @@ int convert_object_file(struct strbuf *outbuf, case OBJ_TREE: ret = convert_tree_object(outbuf, from, to, buf, len); break; - case OBJ_COMMIT: case OBJ_TAG: + ret = convert_tag_object(outbuf, from, to, buf, len); + break; + case OBJ_COMMIT: default: /* Not implemented yet, so fail. */ ret = -1; From patchwork Wed Sep 27 19:55:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13401627 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0863DE82CCD for ; Wed, 27 Sep 2023 19:56:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229933AbjI0T4f (ORCPT ); Wed, 27 Sep 2023 15:56:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35862 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229823AbjI0T4R (ORCPT ); Wed, 27 Sep 2023 15:56:17 -0400 Received: from mail-io1-xd29.google.com (mail-io1-xd29.google.com [IPv6:2607:f8b0:4864:20::d29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E59E61B4 for ; Wed, 27 Sep 2023 12:56:03 -0700 (PDT) Received: by mail-io1-xd29.google.com with SMTP id ca18e2360f4ac-79f915e5b47so408199439f.2 for ; Wed, 27 Sep 2023 12:56:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1695844563; x=1696449363; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=gr09Tj5ObzgTI5Xr9c8J1b1KKj4wL/brqhu1La2Ynvo=; b=YmcvESqGC24LRhq0SYTn1jbpA81q7dWX9Igr3PbyMQRA7UMQLeviDynHVOUGnRjEkJ +tPErGdpb6Plx+pMMpZf1uP1Hp+OxqxhJgqyV0AIJ+H3fPSFaOHD4dnDTciG1e5Ve28H O81iNmG05Rt9JloFvHDXBw00e+nL/5bnJwsS+G4LpQ7SN7XS+/8NL9B0OpMixDGKmGbs apLf6/zC3jVEHx9KNz45eaIog2fVyoIT1lwzTdoxCpRO13JOhxYMtRIlVMirYp8vuPAS MWvKcPXpAzL2/Teq8MmFSyIUsTzKJJ88Ya60elT5snWVJAEN5uI3HHbjrPeTgjQgUYfN uy+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695844563; x=1696449363; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=gr09Tj5ObzgTI5Xr9c8J1b1KKj4wL/brqhu1La2Ynvo=; b=Pz0kBpo7nPDAJtx1JC+EO/xbwO1kWVBFKlDG+VCQPYdGRlxvIJrbd4LuYzj2Ww/Imh QMGuJ9InWWuRXczZ3pkNqj+vI0KQoLxauXIeaJmQVRNi/R+eB3RTLKJFcAgiKhg7Jvjm bKG79L7CwJTi8LBvrIJNrnMykYEciuL2DT2pxpOlUDY1V6jLRirk+NnVPBgwrZ/onu6L z6kNSrR+8IG9+CH2GyQLyx3d3BfWixvaJAxoBaPqqD/q8NI+Xg+z7cYHAOqyAGWImBle wYyfpxUZn1LMJYpIYfBV15phXP2miqRVolIsImBxilVe2W/NQ/AJkMz0i6AVW2QlW+G4 ZfRg== X-Gm-Message-State: AOJu0YzYFZs3EA+vYA1Xpmwc/zV0aMg/39NlPIO1TCQMBR88UV6MAlNi oxjnEDYRoHs003iGrvp2mo4= X-Google-Smtp-Source: AGHT+IFt0BbfORI66Fs41244d3j2+XXr0kfajaWJO91AhSol9E7Jis2tFhjkrMEP/Jjw91SvI/XUhw== X-Received: by 2002:a6b:d002:0:b0:792:9ace:f7ba with SMTP id x2-20020a6bd002000000b007929acef7bamr2691207ioa.11.1695844563203; Wed, 27 Sep 2023 12:56:03 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id o23-20020a6b5a17000000b007836c7e8dccsm2935628iob.17.2023.09.27.12.56.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Sep 2023 12:56:02 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , "Eric W. Biederman" Subject: [PATCH 17/30] object-file-convert: Don't leak when converting tag objects Date: Wed, 27 Sep 2023 14:55:24 -0500 Message-Id: <20230927195537.1682-17-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> References: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "Eric W. Biederman" Upon close examination I discovered that while brian's code to convert tag objects was functionally correct, it leaked memory. Rearrange the code so that all error checking happens before any memory is allocated. Add code to release the temporary strbufs the code uses. The code pretty much assumes the tag object ends with a newline, so add an explict test to verify that is the case. Signed-off-by: Eric W. Biederman --- object-file-convert.c | 45 ++++++++++++++++++++++++------------------- 1 file changed, 25 insertions(+), 20 deletions(-) diff --git a/object-file-convert.c b/object-file-convert.c index 777ae5b58036..822be9d0fdb8 100644 --- a/object-file-convert.c +++ b/object-file-convert.c @@ -90,44 +90,49 @@ static int convert_tag_object(struct strbuf *out, const struct git_hash_algo *to, const char *buffer, size_t size) { - struct strbuf payload = STRBUF_INIT, temp = STRBUF_INIT, oursig = STRBUF_INIT, othersig = STRBUF_INIT; + struct strbuf payload = STRBUF_INIT, oursig = STRBUF_INIT, othersig = STRBUF_INIT; + const int entry_len = from->hexsz + 7; size_t payload_size; struct object_id oid, mapped_oid; const char *p; - /* Add some slop for longer signature header in the new algorithm. */ - strbuf_grow(out, size + 7); + /* Consume the object line */ + if ((entry_len >= size) || + memcmp(buffer, "object ", 7) || buffer[entry_len] != '\n') + return error("bogus tag object"); + if (parse_oid_hex_algop(buffer + 7, &oid, &p, from) < 0) + return error("bad tag object ID"); + if (repo_oid_to_algop(the_repository, &oid, to, &mapped_oid)) + return error("unable to map tree %s in tag object", + oid_to_hex(&oid)); + size -= ((p + 1) - buffer); + buffer = p + 1; /* Is there a signature for our algorithm? */ payload_size = parse_signed_buffer(buffer, size); - strbuf_add(&payload, buffer, payload_size); if (payload_size != size) { /* Yes, there is. */ strbuf_add(&oursig, buffer + payload_size, size - payload_size); } - /* Now, is there a signature for the other algorithm? */ - if (parse_buffer_signed_by_header(payload.buf, payload.len, &temp, &othersig, to)) { - /* Yes, there is. */ - strbuf_swap(&payload, &temp); - strbuf_release(&temp); - } + /* Now, is there a signature for the other algorithm? */ + parse_buffer_signed_by_header(buffer, payload_size, &payload, &othersig, to); /* * Our payload is now in payload and we may have up to two signatrures * in oursig and othersig. */ - if (strncmp(payload.buf, "object ", 7) || payload.buf[from->hexsz + 7] != '\n') - return error("bogus tag object"); - if (parse_oid_hex_algop(payload.buf + 7, &oid, &p, from) < 0) - return error("bad tag object ID"); - if (repo_oid_to_algop(the_repository, &oid, to, &mapped_oid)) - return error("unable to map tree %s in tag object", - oid_to_hex(&oid)); - strbuf_addf(out, "object %s", oid_to_hex(&mapped_oid)); - strbuf_add(out, p, payload.len - (p - payload.buf)); - strbuf_addbuf(out, &othersig); + + /* Add some slop for longer signature header in the new algorithm. */ + strbuf_grow(out, (7 + to->hexsz + 1) + size + 7); + strbuf_addf(out, "object %s\n", oid_to_hex(&mapped_oid)); + strbuf_addbuf(out, &payload); if (oursig.len) add_header_signature(out, &oursig, from); + strbuf_addbuf(out, &othersig); + + strbuf_release(&payload); + strbuf_release(&othersig); + strbuf_release(&oursig); return 0; } From patchwork Wed Sep 27 19:55:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13401628 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 56167E82CCC for ; Wed, 27 Sep 2023 19:56:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229847AbjI0T4h (ORCPT ); Wed, 27 Sep 2023 15:56:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35876 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229831AbjI0T4R (ORCPT ); Wed, 27 Sep 2023 15:56:17 -0400 Received: from mail-io1-xd2a.google.com (mail-io1-xd2a.google.com [IPv6:2607:f8b0:4864:20::d2a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8C8271BB for ; Wed, 27 Sep 2023 12:56:04 -0700 (PDT) Received: by mail-io1-xd2a.google.com with SMTP id ca18e2360f4ac-79faba5fe12so325523239f.3 for ; Wed, 27 Sep 2023 12:56:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1695844564; x=1696449364; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=LVMkft9MkWAd0VHZsoOdepZnaQ7p0NDdLzvbL/PLqXc=; b=QZbZ2yDCUCoz73IL5BarJpQbDfM9u8/rtmpQIxLZqhfG4A79AbzF3FxwGKt0UsqwYr dIESJyP/lGIowvIL6dOc1OSjoLz5NvYms31v0v3HZPpipHVWutzm63f9lz4LE9ppwvpH qmDZduVXJApH3NdQTxOC3z1+fU2bPW2HYUt3vg1lEU6mMgxL5Ot62s/HZRZgfZdDdl9h doMMCNIWosqHck3/Azv2r4QKfXKg00QJ20WNXE/rsw/7eyXSFPKse5nF23yxdnfWJv6h iPeCaMPkwjQaIOEJxhkMimOiuLxQlboEqbmJoAcyu/1B7TreNlYLxh3k47Mlaec1qPcv dGdA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695844564; x=1696449364; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LVMkft9MkWAd0VHZsoOdepZnaQ7p0NDdLzvbL/PLqXc=; b=wU6/4aBJgl2yBDsR0Q3o3CMBxd6GWVXjd46nRo1sWCQUy623SCEUGsT7dRXcjHMopJ MNvdQh5VHlEvo8LDs2pE9a4iy9zRB64q0OZzVehwGKKiCiud1kr9i24ps+t/sRowHltK ykHePSPv6zEvlwLlw3QRXBXySyK5a6EwnPi9qDS8PbarMEKDjpmiSn1e6sTLDxSYIcVs vr1RTBjR7ZMthZd0dA73qxGxteCWrW+bROHixygJgMjl8a1LzY9T3xPludQIwbfXX6bG lh2epoo4u+nTPeCTRt/DoHSG/6NVu4LSSMKO3qVeCf3wdAJhIrsPWZ0hQugXZOowzuMR cWhw== X-Gm-Message-State: AOJu0YwI59PhX3yQZ9h10fae7nlSyIktoaxJl/iIdfqHdD/RlBDehOyA r090/bv/+9Vn6itiJFsgjRk= X-Google-Smtp-Source: AGHT+IHXPI3jbS0CLal6t5sXxErHbsNbhv6v1hlZsvA8fl0euiGUQXyqtktofb7rjr9KnhYaaIBJCw== X-Received: by 2002:a6b:cf10:0:b0:790:6629:b222 with SMTP id o16-20020a6bcf10000000b007906629b222mr2696930ioa.5.1695844563891; Wed, 27 Sep 2023 12:56:03 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id o23-20020a6b5a17000000b007836c7e8dccsm2935628iob.17.2023.09.27.12.56.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Sep 2023 12:56:03 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , "Eric W . Biederman" Subject: [PATCH 18/30] object-file-convert: convert commit objects when writing Date: Wed, 27 Sep 2023 14:55:25 -0500 Message-Id: <20230927195537.1682-18-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> References: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "brian m. carlson" When writing a commit object in a repository with both SHA-1 and SHA-256, we'll need to convert our commit objects so that we can write the hash values for both into the repository. To do so, let's add a function to convert commit objects. Read the commit object and map the tree value and any of the parent values, and copy the rest of the commit through unmodified. Note that we don't need to modify the signature headers, because they are the same under both algorithms. Signed-off-by: brian m. carlson Signed-off-by: Eric W. Biederman --- object-file-convert.c | 46 ++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 45 insertions(+), 1 deletion(-) diff --git a/object-file-convert.c b/object-file-convert.c index 822be9d0fdb8..f53e14e5a170 100644 --- a/object-file-convert.c +++ b/object-file-convert.c @@ -136,6 +136,48 @@ static int convert_tag_object(struct strbuf *out, return 0; } +static int convert_commit_object(struct strbuf *out, + const struct git_hash_algo *from, + const struct git_hash_algo *to, + const char *buffer, size_t size) +{ + const char *tail = buffer; + const char *bufptr = buffer; + const int tree_entry_len = from->hexsz + 5; + const int parent_entry_len = from->hexsz + 7; + struct object_id oid, mapped_oid; + const char *p; + + tail += size; + if (tail <= bufptr + tree_entry_len + 1 || memcmp(bufptr, "tree ", 5) || + bufptr[tree_entry_len] != '\n') + return error("bogus commit object"); + if (parse_oid_hex_algop(bufptr + 5, &oid, &p, from) < 0) + return error("bad tree pointer"); + + if (repo_oid_to_algop(the_repository, &oid, to, &mapped_oid)) + return error("unable to map tree %s in commit object", + oid_to_hex(&oid)); + strbuf_addf(out, "tree %s\n", oid_to_hex(&mapped_oid)); + bufptr = p + 1; + + while (bufptr + parent_entry_len < tail && !memcmp(bufptr, "parent ", 7)) { + if (tail <= bufptr + parent_entry_len + 1 || + parse_oid_hex_algop(bufptr + 7, &oid, &p, from) || + *p != '\n') + return error("bad parents in commit"); + + if (repo_oid_to_algop(the_repository, &oid, to, &mapped_oid)) + return error("unable to map parent %s in commit object", + oid_to_hex(&oid)); + + strbuf_addf(out, "parent %s\n", oid_to_hex(&mapped_oid)); + bufptr = p + 1; + } + strbuf_add(out, bufptr, tail - bufptr); + return 0; +} + int convert_object_file(struct strbuf *outbuf, const struct git_hash_algo *from, const struct git_hash_algo *to, @@ -150,13 +192,15 @@ int convert_object_file(struct strbuf *outbuf, die("Refusing noop object file conversion"); switch (type) { + case OBJ_COMMIT: + ret = convert_commit_object(outbuf, from, to, buf, len); + break; case OBJ_TREE: ret = convert_tree_object(outbuf, from, to, buf, len); break; case OBJ_TAG: ret = convert_tag_object(outbuf, from, to, buf, len); break; - case OBJ_COMMIT: default: /* Not implemented yet, so fail. */ ret = -1; From patchwork Wed Sep 27 19:55:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13401629 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CFB8DE82CCE for ; Wed, 27 Sep 2023 19:56:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229938AbjI0T4j (ORCPT ); Wed, 27 Sep 2023 15:56:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35916 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229842AbjI0T4S (ORCPT ); Wed, 27 Sep 2023 15:56:18 -0400 Received: from mail-io1-xd2a.google.com (mail-io1-xd2a.google.com [IPv6:2607:f8b0:4864:20::d2a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 48E55CC7 for ; Wed, 27 Sep 2023 12:56:05 -0700 (PDT) Received: by mail-io1-xd2a.google.com with SMTP id ca18e2360f4ac-79fa425855cso347637739f.2 for ; Wed, 27 Sep 2023 12:56:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1695844564; x=1696449364; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=bhSn7P7v6ympvS94OsSp5itQ+Qg0o5Mo0tjhO0FEJvw=; b=HuZstK91NMmVGB2CG1oMa/6BAC2Ug8cOXmL09RsEzRu3R56R2GqnFORm+Bxl13GMER cmDmdAhWAGnsc/ttdNUBkbZdw5earXjvF1qozPf0/QpvNQNL5GGt51177iAPNlcknqPH KXLVtVG+8Hu6sjPuf8LKPljGghmF326beLgGXMNkLbMC61FFKACiY2HugcCYTyVoejzS Jl6CylbjHU5+3SxlwfibQQMrLs/uwuiGJwvbyOPPeilbnAH/Vsr9R0HpCFt6Ka6ikpFo WFULZgErwigkJiMqY9eS51ysd2BjoVDHtpsv3dTE9kiSnF+b2QE9MZgS69wonehvzlLI jp1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695844564; x=1696449364; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bhSn7P7v6ympvS94OsSp5itQ+Qg0o5Mo0tjhO0FEJvw=; b=f/2jhy8ZW+agqPTHJrfy0B4GhKjfG9wcFkcZM0nBc3TezvLegv0Os3F2adguW/axJu 772BD+yGy+Wok8IycmavklFXAeqbaZb4ZAQJZUJA5eHg3f4qoz7eFbKdtgPJ6K71l1SF 37cZ7jSg+AFFWs40eVz8reTk3wgf8S47TYKi+CissSd78qxMeo4QQZifQsCh2uY3U8qk mFmfvi2/daj7nVm+OGctbobo9ZivD/X9qMGSfJZVSATBNthFE5aWKU5GbIQnAbgjq2NK H+ILJz8NY9v1OqtIYAkjq09aAa+XpvV389JTr+DzyuCdp8RwQ2QQHdpEIS9VsnkXWj4r slNQ== X-Gm-Message-State: AOJu0YyAmTAUITiVr6BGTzAGDEvEIM4yuO1ruAHrE7NzhBfHeyCNiyj1 kpIhUDwHfSvN4XC//1ldt4nYToKcxHU= X-Google-Smtp-Source: AGHT+IEf1cCnoOZ+2HhXUsSxk1Ja2qcS4BWUYFM2AeqSHgmwpOJ/zm5W5coK3BgRtC8FgB30elQKNA== X-Received: by 2002:a6b:fe14:0:b0:787:4b5f:b6cf with SMTP id x20-20020a6bfe14000000b007874b5fb6cfmr3115877ioh.5.1695844564559; Wed, 27 Sep 2023 12:56:04 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id o23-20020a6b5a17000000b007836c7e8dccsm2935628iob.17.2023.09.27.12.56.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Sep 2023 12:56:04 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , "Eric W. Biederman" Subject: [PATCH 19/30] object-file-convert: Convert commits that embed signed tags Date: Wed, 27 Sep 2023 14:55:26 -0500 Message-Id: <20230927195537.1682-19-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> References: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "Eric W. Biederman" As mentioned in the hash function transition plan commit mergetag lines need to be handled. The commit mergetag lines embed an entire tag object in a commit object. Keep the implementation sane if not fast by unembedding the tag object, converting the tag object, and embedding the new tag object, in the new commit object. In the long run I don't expect any other approach is maintainable, as tag objects may be extended in ways that require additional translation. To keep the implementation of convert_commit_object maintainable I have modified convert_commit_object to process the lines in any order, and to fail on unknown lines. We can't know ahead of time if a new line might embed something that needs translation or not so it is better to fail and require the code to be updated instead of silently mistranslating objects. Signed-off-by: Eric W. Biederman --- object-file-convert.c | 104 +++++++++++++++++++++++++++++++++--------- 1 file changed, 82 insertions(+), 22 deletions(-) diff --git a/object-file-convert.c b/object-file-convert.c index f53e14e5a170..8ede9889a7ab 100644 --- a/object-file-convert.c +++ b/object-file-convert.c @@ -146,35 +146,95 @@ static int convert_commit_object(struct strbuf *out, const int tree_entry_len = from->hexsz + 5; const int parent_entry_len = from->hexsz + 7; struct object_id oid, mapped_oid; - const char *p; + const char *p, *eol; tail += size; - if (tail <= bufptr + tree_entry_len + 1 || memcmp(bufptr, "tree ", 5) || - bufptr[tree_entry_len] != '\n') - return error("bogus commit object"); - if (parse_oid_hex_algop(bufptr + 5, &oid, &p, from) < 0) - return error("bad tree pointer"); - if (repo_oid_to_algop(the_repository, &oid, to, &mapped_oid)) - return error("unable to map tree %s in commit object", - oid_to_hex(&oid)); - strbuf_addf(out, "tree %s\n", oid_to_hex(&mapped_oid)); - bufptr = p + 1; + while ((bufptr < tail) && (*bufptr != '\n')) { + eol = memchr(bufptr, '\n', tail - bufptr); + if (!eol) + return error(_("bad %s in commit"), "line"); + + if (((bufptr + 5) < eol) && !memcmp(bufptr, "tree ", 5)) + { + if (((bufptr + tree_entry_len) != eol) || + parse_oid_hex_algop(bufptr + 5, &oid, &p, from) || + (p != eol)) + return error(_("bad %s in commit"), "tree"); + + if (repo_oid_to_algop(the_repository, &oid, to, &mapped_oid)) + return error(_("unable to map %s %s in commit object"), + "tree", oid_to_hex(&oid)); + strbuf_addf(out, "tree %s\n", oid_to_hex(&mapped_oid)); + } + else if (((bufptr + 7) < eol) && !memcmp(bufptr, "parent ", 7)) + { + if (((bufptr + parent_entry_len) != eol) || + parse_oid_hex_algop(bufptr + 7, &oid, &p, from) || + (p != eol)) + return error(_("bad %s in commit"), "parent"); - while (bufptr + parent_entry_len < tail && !memcmp(bufptr, "parent ", 7)) { - if (tail <= bufptr + parent_entry_len + 1 || - parse_oid_hex_algop(bufptr + 7, &oid, &p, from) || - *p != '\n') - return error("bad parents in commit"); + if (repo_oid_to_algop(the_repository, &oid, to, &mapped_oid)) + return error(_("unable to map %s %s in commit object"), + "parent", oid_to_hex(&oid)); - if (repo_oid_to_algop(the_repository, &oid, to, &mapped_oid)) - return error("unable to map parent %s in commit object", - oid_to_hex(&oid)); + strbuf_addf(out, "parent %s\n", oid_to_hex(&mapped_oid)); + } + else if (((bufptr + 9) < eol) && !memcmp(bufptr, "mergetag ", 9)) + { + struct strbuf tag = STRBUF_INIT, new_tag = STRBUF_INIT; - strbuf_addf(out, "parent %s\n", oid_to_hex(&mapped_oid)); - bufptr = p + 1; + /* Recover the tag object from the mergetag */ + strbuf_add(&tag, bufptr + 9, (eol - (bufptr + 9)) + 1); + + bufptr = eol + 1; + while ((bufptr < tail) && (*bufptr == ' ')) { + eol = memchr(bufptr, '\n', tail - bufptr); + if (!eol) { + strbuf_release(&tag); + return error(_("bad %s in commit"), "mergetag continuation"); + } + strbuf_add(&tag, bufptr + 1, (eol - (bufptr + 1)) + 1); + bufptr = eol + 1; + } + + /* Compute the new tag object */ + if (convert_tag_object(&new_tag, from, to, tag.buf, tag.len)) { + strbuf_release(&tag); + strbuf_release(&new_tag); + return -1; + } + + /* Write the new mergetag */ + strbuf_addstr(out, "mergetag"); + strbuf_add_lines(out, " ", new_tag.buf, new_tag.len); + strbuf_release(&tag); + strbuf_release(&new_tag); + } + else if (((bufptr + 7) < tail) && !memcmp(bufptr, "author ", 7)) + strbuf_add(out, bufptr, (eol - bufptr) + 1); + else if (((bufptr + 10) < tail) && !memcmp(bufptr, "committer ", 10)) + strbuf_add(out, bufptr, (eol - bufptr) + 1); + else if (((bufptr + 9) < tail) && !memcmp(bufptr, "encoding ", 9)) + strbuf_add(out, bufptr, (eol - bufptr) + 1); + else if (((bufptr + 6) < tail) && !memcmp(bufptr, "gpgsig", 6)) + strbuf_add(out, bufptr, (eol - bufptr) + 1); + else { + /* Unknown line fail it might embed an oid */ + return -1; + } + /* Consume any trailing continuation lines */ + bufptr = eol + 1; + while ((bufptr < tail) && (*bufptr == ' ')) { + eol = memchr(bufptr, '\n', tail - bufptr); + if (!eol) + return error(_("bad %s in commit"), "continuation"); + strbuf_add(out, bufptr, (eol - bufptr) + 1); + bufptr = eol + 1; + } } - strbuf_add(out, bufptr, tail - bufptr); + if (bufptr < tail) + strbuf_add(out, bufptr, tail - bufptr); return 0; } From patchwork Wed Sep 27 19:55:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13401630 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 872A6E82CCF for ; Wed, 27 Sep 2023 19:56:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229939AbjI0T4l (ORCPT ); Wed, 27 Sep 2023 15:56:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35934 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229803AbjI0T4X (ORCPT ); Wed, 27 Sep 2023 15:56:23 -0400 Received: from mail-io1-xd33.google.com (mail-io1-xd33.google.com [IPv6:2607:f8b0:4864:20::d33]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ED875CCB for ; Wed, 27 Sep 2023 12:56:05 -0700 (PDT) Received: by mail-io1-xd33.google.com with SMTP id ca18e2360f4ac-79fa891b645so337332839f.0 for ; Wed, 27 Sep 2023 12:56:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1695844565; x=1696449365; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1a9rKHC3NFMW+Qi8DVW/6mu7Wi6x73PfiC0GhejhjQs=; b=m7FglewgD9vuKPO4njaeQAW53zkiNE/HoUtd+xGO6rJqRaPF203VSc2DkD0wgnhbzR O3ox1sk4cJnpKdWOGO18enEC7V+aa+n/hMVWM5ksm9OAfekIHlvdK6NdhbWGYpjQDgoN uCRuhmnvEwozsqsRAFsuvDX4bQv7lPyJjhN6u03MUxdF5U3rvB+y1aTjAaJ8b++TWT7B E3CCqrFNYhkruhOuVH8PTefbSjXyNac8CU4155D2x4BvGOkhzMn2OvxoRiM4Yuit069a hNj6G7i3jzGwFZUuP6gfwcE+T9bcmYrK9eDNVYlZD8/KrKJUTLWzX0mnMSI4I8p5P7sS iojg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695844565; x=1696449365; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1a9rKHC3NFMW+Qi8DVW/6mu7Wi6x73PfiC0GhejhjQs=; b=mvEGCI6EiZkOwzk7F6n8cC9y0SIaesxBQwjy7xtPB+KypEm1swhU8OGKJi5j4beIYx S0aZf0ahCij9D+YGcZHKDlHQzTZ1L7k00/Wh+gdX1pDd3Tf0WBBSzTLTqAt+0EhMM2KC 95AW8ercKbiTvFvuYX684hHk+tRv0vuTAajAFHosQviosQoGQE89tC3DBS7Y0kP0EwE8 iNGguiLFMwHF1836RtkIDZ4rg85Gd/2lDhfYxqaXz0ie0xArxSvDX/fmnSUfru5Quduq bBIUpL4w+duC/P8/qbt4oQdVIss7RfWI1AEErjZ4kNUA6KzbcNROuxkUn5ShDaIsLe1D zM4Q== X-Gm-Message-State: AOJu0YzQfS8nnLk2Gk+WKZpxEOWdU4JNm2w5JsImBh9fsu6L9y6W9hne 2CFf5HYIp0zNi6oHCEwaEjw07EwrA50= X-Google-Smtp-Source: AGHT+IHHgFpoMYHHv6Jr8iZeJsj+yvY9RRVOsK0hj7u0nPEVdibGKplXLEvQeIyLcxOiNdcndiQYEg== X-Received: by 2002:a5d:9251:0:b0:79f:e9ac:f60a with SMTP id e17-20020a5d9251000000b0079fe9acf60amr3052576iol.20.1695844565307; Wed, 27 Sep 2023 12:56:05 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id o23-20020a6b5a17000000b007836c7e8dccsm2935628iob.17.2023.09.27.12.56.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Sep 2023 12:56:04 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , "Eric W. Biederman" Subject: [PATCH 20/30] object-file: Update object_info_extended to reencode objects Date: Wed, 27 Sep 2023 14:55:27 -0500 Message-Id: <20230927195537.1682-20-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> References: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "Eric W. Biederman" oid_object_info_extended is updated to detect an oid encoding that does not match the current repository, use repo_oid_to_algop to find the correspoding oid in the current repository and to return the data for the oid. Signed-off-by: "Eric W. Biederman" --- object-file.c | 91 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 91 insertions(+) diff --git a/object-file.c b/object-file.c index d66d11890696..1601d624c9fd 100644 --- a/object-file.c +++ b/object-file.c @@ -1662,10 +1662,101 @@ static int do_oid_object_info_extended(struct repository *r, return 0; } +static int oid_object_info_convert(struct repository *r, + const struct object_id *input_oid, + struct object_info *input_oi, unsigned flags) +{ + const struct git_hash_algo *input_algo = &hash_algos[input_oid->algo]; + int do_die = flags & OBJECT_INFO_DIE_IF_CORRUPT; + struct strbuf type_name = STRBUF_INIT; + struct object_id oid, delta_base_oid; + struct object_info new_oi, *oi; + unsigned long size; + void *content; + int ret; + + if (repo_oid_to_algop(r, input_oid, the_hash_algo, &oid)) { + if (do_die) + die(_("missing mapping of %s to %s"), + oid_to_hex(input_oid), the_hash_algo->name); + return -1; + } + + /* Is new_oi needed? */ + oi = input_oi; + if (input_oi && (input_oi->delta_base_oid || input_oi->sizep || + input_oi->contentp)) { + new_oi = *input_oi; + /* Does delta_base_oid need to be converted? */ + if (input_oi->delta_base_oid) + new_oi.delta_base_oid = &delta_base_oid; + /* Will the attributes differ when converted? */ + if (input_oi->sizep || input_oi->contentp) { + new_oi.contentp = &content; + new_oi.sizep = &size; + new_oi.type_name = &type_name; + } + oi = &new_oi; + } + + ret = oid_object_info_extended(r, &oid, oi, flags); + if (ret) + return -1; + if (oi == input_oi) + return ret; + + if (new_oi.contentp) { + struct strbuf outbuf = STRBUF_INIT; + enum object_type type; + + type = type_from_string_gently(type_name.buf, type_name.len, + !do_die); + if (type == -1) + return -1; + if (type != OBJ_BLOB) { + ret = convert_object_file(&outbuf, + the_hash_algo, input_algo, + content, size, type, !do_die); + if (ret == -1) + return -1; + free(content); + size = outbuf.len; + content = strbuf_detach(&outbuf, NULL); + } + if (input_oi->sizep) + *input_oi->sizep = size; + if (input_oi->contentp) + *input_oi->contentp = content; + else + free(content); + if (input_oi->type_name) + *input_oi->type_name = type_name; + else + strbuf_release(&type_name); + } + if (new_oi.delta_base_oid == &delta_base_oid) { + if (repo_oid_to_algop(r, &delta_base_oid, input_algo, + input_oi->delta_base_oid)) { + if (do_die) + die(_("missing mapping of %s to %s"), + oid_to_hex(&delta_base_oid), + input_algo->name); + return -1; + } + } + input_oi->whence = new_oi.whence; + input_oi->u = new_oi.u; + return ret; +} + int oid_object_info_extended(struct repository *r, const struct object_id *oid, struct object_info *oi, unsigned flags) { int ret; + + if (oid->algo && (hash_algo_by_ptr(r->hash_algo) != oid->algo)) + return oid_object_info_convert(r, oid, oi, flags); + obj_read_lock(); ret = do_oid_object_info_extended(r, oid, oi, flags); obj_read_unlock(); From patchwork Wed Sep 27 19:55:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13401632 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DF54EE82CCC for ; Wed, 27 Sep 2023 19:56:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229966AbjI0T4t (ORCPT ); Wed, 27 Sep 2023 15:56:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50436 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229879AbjI0T4c (ORCPT ); Wed, 27 Sep 2023 15:56:32 -0400 Received: from mail-io1-xd32.google.com (mail-io1-xd32.google.com [IPv6:2607:f8b0:4864:20::d32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A684DCCF for ; Wed, 27 Sep 2023 12:56:06 -0700 (PDT) Received: by mail-io1-xd32.google.com with SMTP id ca18e2360f4ac-7927f24140eso412187039f.2 for ; Wed, 27 Sep 2023 12:56:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1695844566; x=1696449366; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=C+XvkI2q8HdgyNZqFquNWFe2nBLsYwqJ2GxQfllpj68=; b=DeYK+9eeqfO/4W0llSsxtOcXmR7FajwjCl1Y1+ImUNGGt44bDrkUyOraBfsVBfXzOa N/vif8yA0vQHZRKP7j6yCRK8IvpJn5eQReaCZV2HY1P6CntIOLpVJxbv4tmBZpEXtPMW hJHqAp5QoBSIKhgzitH0BZ5gfNwmL5yHDRqJtU7CeNwsDztszbgivts4YIaSZxFz3REp 3vkTsfG1jFR0zWXfTI8tqwynx2dP7hcG3mU04guKbV09G/AxVxm6NK2LMmwo68lZ1ZW7 TNlH2g7MgM4nT2mtR3x60z/AkthJVrGqaiq7QeDsdoJneuYUu++jyX0ubHbF4oAwXhul bEdA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695844566; x=1696449366; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=C+XvkI2q8HdgyNZqFquNWFe2nBLsYwqJ2GxQfllpj68=; b=Wf7fNx/iiuS1xO255yfimzNB0wz+HS9P33+ZtS+ElMbPwXnodhaVXhucaKBcynG0wv oHot894ab+B3tTa3TacaQSBmSqls1df2hCALSiGjZlAHs57DA3W9Xp4lqmLLxCrqasZ/ bG2FbE45ku6gOJhflML2mmFY5QV6QI62tlv5pN2YGzlXYUEwB80DG7x1sbBev8gV/0yq +SVvRT/tq2LWgQ1NXvjoUJAQ+v0rJtmsx/kbaGMgLO6Bzxf3Rqh2CeQM5pn37d0ZRkLo dQ9AkusmB/JkfEbtfMEWsZVlyx4Ejm9EE/VeIlH2MbJ5Cf8W7vvDegRt0oOhEhCprC4m Mkkw== X-Gm-Message-State: AOJu0YxN3jv3xpkTMtbzT/fv9MhdPYmoOn20Ki9l4G4iZsjQzOAgzahn MI4xDKLg0Bvvu65+w9pUqlM= X-Google-Smtp-Source: AGHT+IFCsZOxoqhIUmy9XumtBApasHB93gJFvBA0dQG+UCROekUqdIBE31zvhv20iNuIE6jpVwasgw== X-Received: by 2002:a6b:dd07:0:b0:79f:cb49:f476 with SMTP id f7-20020a6bdd07000000b0079fcb49f476mr2480117ioc.21.1695844565977; Wed, 27 Sep 2023 12:56:05 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id o23-20020a6b5a17000000b007836c7e8dccsm2935628iob.17.2023.09.27.12.56.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Sep 2023 12:56:05 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , "Eric W. Biederman" Subject: [PATCH 21/30] repository: Implement extensions.compatObjectFormat Date: Wed, 27 Sep 2023 14:55:28 -0500 Message-Id: <20230927195537.1682-21-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> References: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "brian m. carlson" Add a configuration option to enable updating and reading from compatibility hash maps when git accesses the reposotiry. Call the helper function repo_set_compat_hash_algo with the value that compatObjectFormat is set to. Signed-off-by: "Eric W. Biederman" --- Documentation/config/extensions.txt | 12 ++++++++++++ repository.c | 2 +- setup.c | 23 +++++++++++++++++++++-- setup.h | 1 + 4 files changed, 35 insertions(+), 3 deletions(-) diff --git a/Documentation/config/extensions.txt b/Documentation/config/extensions.txt index bccaec7a9636..9f72e6d9f4f1 100644 --- a/Documentation/config/extensions.txt +++ b/Documentation/config/extensions.txt @@ -7,6 +7,18 @@ Note that this setting should only be set by linkgit:git-init[1] or linkgit:git-clone[1]. Trying to change it after initialization will not work and will produce hard-to-diagnose issues. +extensions.compatObjectFormat:: + + Specify a compatitbility hash algorithm to use. The acceptable values + are `sha1` and `sha256`. The value specified must be different from the + value of extensions.objectFormat. This allows client level + interoperability between git repositories whose objectFormat matches + this compatObjectFormat. In particular when fully implemented the + pushes and pulls from a repository in whose objectFormat matches + compatObjectFormat. As well as being able to use oids encoded in + compatObjectFormat in addition to oids encoded with objectFormat to + locally specify objects. + extensions.worktreeConfig:: If enabled, then worktrees will load config settings from the `$GIT_DIR/config.worktree` file in addition to the diff --git a/repository.c b/repository.c index 6214f61cf4e7..9d91536b613b 100644 --- a/repository.c +++ b/repository.c @@ -194,7 +194,7 @@ int repo_init(struct repository *repo, goto error; repo_set_hash_algo(repo, format.hash_algo); - repo_set_compat_hash_algo(repo, GIT_HASH_UNKNOWN); + repo_set_compat_hash_algo(repo, format.compat_hash_algo); repo->repository_format_worktree_config = format.worktree_config; /* take ownership of format.partial_clone */ diff --git a/setup.c b/setup.c index deb5a33fe9e1..87b40472dbc5 100644 --- a/setup.c +++ b/setup.c @@ -598,6 +598,25 @@ static enum extension_result handle_extension(const char *var, } data->hash_algo = format; return EXTENSION_OK; + } else if (!strcmp(ext, "compatobjectformat")) { + struct string_list_item *item; + int format; + + if (!value) + return config_error_nonbool(var); + format = hash_algo_by_name(value); + if (format == GIT_HASH_UNKNOWN) + return error(_("invalid value for '%s': '%s'"), + "extensions.compatobjectformat", value); + /* For now only support compatObjectFormat being specified once. */ + for_each_string_list_item(item, &data->v1_only_extensions) { + if (!strcmp(item->string, "compatobjectformat")) + return error(_("'%s' already specified as '%s'"), + "extensions.compatobjectformat", + hash_algos[data->compat_hash_algo].name); + } + data->compat_hash_algo = format; + return EXTENSION_OK; } return EXTENSION_UNKNOWN; } @@ -1573,7 +1592,7 @@ const char *setup_git_directory_gently(int *nongit_ok) if (startup_info->have_repository) { repo_set_hash_algo(the_repository, repo_fmt.hash_algo); repo_set_compat_hash_algo(the_repository, - GIT_HASH_UNKNOWN); + repo_fmt.compat_hash_algo); the_repository->repository_format_worktree_config = repo_fmt.worktree_config; /* take ownership of repo_fmt.partial_clone */ @@ -1667,7 +1686,7 @@ void check_repository_format(struct repository_format *fmt) check_repository_format_gently(get_git_dir(), fmt, NULL); startup_info->have_repository = 1; repo_set_hash_algo(the_repository, fmt->hash_algo); - repo_set_compat_hash_algo(the_repository, GIT_HASH_UNKNOWN); + repo_set_compat_hash_algo(the_repository, fmt->compat_hash_algo); the_repository->repository_format_worktree_config = fmt->worktree_config; the_repository->repository_format_partial_clone = diff --git a/setup.h b/setup.h index 58fd2605dd26..5d678ceb8caa 100644 --- a/setup.h +++ b/setup.h @@ -86,6 +86,7 @@ struct repository_format { int worktree_config; int is_bare; int hash_algo; + int compat_hash_algo; int sparse_index; char *work_tree; struct string_list unknown_extensions; From patchwork Wed Sep 27 19:55:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13401631 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 85708E82CCD for ; Wed, 27 Sep 2023 19:56:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229941AbjI0T4m (ORCPT ); Wed, 27 Sep 2023 15:56:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43724 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229878AbjI0T4c (ORCPT ); Wed, 27 Sep 2023 15:56:32 -0400 Received: from mail-io1-xd29.google.com (mail-io1-xd29.google.com [IPv6:2607:f8b0:4864:20::d29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5ED2C126 for ; Wed, 27 Sep 2023 12:56:07 -0700 (PDT) Received: by mail-io1-xd29.google.com with SMTP id ca18e2360f4ac-79fce245bf6so234936039f.1 for ; Wed, 27 Sep 2023 12:56:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1695844566; x=1696449366; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Ah9lMgvMVvnNRn//l/9eCmgPadLsslMM4KL7LWB14S4=; b=VLA16e5Bj4ZL0Gx1KKNz31ByK4WqWd5ZA7ZlYrMGf/SWJ/f6QOb2NLpuJ4zZuHjySZ +uJpXY0ZcodzQiA/GigrgcTWuvSqhftiZuZCVlasQ2K0oLOeM5a5kT3snTYS5hOeoDMb X8sOEOw0UrSZCVHsv8cKIZqrgeNdsKg69Cr4IeAjwGnfHz1P69u6ALtHHS80zcEPx0bk cgpnC8h+ZHYQuQ5ZL6fvN+rJamlH2GyVT6VwUioDkOC4RrNZmRglb1en0g0bgc/rFrhK cYJfgQS2r4BzzQGW01fJQr9lyA/Uzqz7aTKB/qj97j49kGmDMaL6Nf2voxfpYiiUC3cs 8zfw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695844566; x=1696449366; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Ah9lMgvMVvnNRn//l/9eCmgPadLsslMM4KL7LWB14S4=; b=Khzp2V6z9IEyx8Z8xdW63s0a6qxb7APBguSaBSJb/3z/vIbPu9DspNo97y1pukxtk4 PIoq37Gymtf5GmP38FYSytVjE9whOJ3By9+gKl7RdXQa16kYVqKik2d7mSmeiovDtZk3 gSxJsn0stSRihEDI8JaFLcEGYz4CyLKIJwAfXBOdLRYE0fA9pydNh/0feokcGoho2Kp5 oelnLupJpzLa7CKMFX32CpBMeGx+o8nSI2aQLlswUYCIcFp7Kii3oE6KlEpBse5ms3II flw9uOskHH8AtfW9wsSP7x4QX/nqaVqNk4/vPw8SxkBQMFxQedev7l+X/sS7mwt79kAo H8/w== X-Gm-Message-State: AOJu0YyhUKqEWopMfzmN0y2F5KmCyrce82C46aIFs6sERIiuprkOSj6F 5vSbIv6+yEH4iVyOGvsBLe6WoOY61IA= X-Google-Smtp-Source: AGHT+IEH01IYQox6bknbY8GUZgezteVACSFM2QNRdDRVyWbTeo+7POg5TVX2E9crR2GGPCfS3HddQQ== X-Received: by 2002:a6b:dd07:0:b0:79f:cb49:f476 with SMTP id f7-20020a6bdd07000000b0079fcb49f476mr2480136ioc.21.1695844566667; Wed, 27 Sep 2023 12:56:06 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id o23-20020a6b5a17000000b007836c7e8dccsm2935628iob.17.2023.09.27.12.56.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Sep 2023 12:56:06 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , "Eric W. Biederman" Subject: [PATCH 22/30] rev-parse: Add an --output-object-format parameter Date: Wed, 27 Sep 2023 14:55:29 -0500 Message-Id: <20230927195537.1682-22-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> References: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "Eric W. Biederman" The new --output-object-format parameter returns the oid in the specified format. This is a generally useful plumbing facility. It is useful for writing test cases and for directly querying the translation maps. Signed-off-by: "Eric W. Biederman" --- Documentation/git-rev-parse.txt | 12 ++++++++++++ builtin/rev-parse.c | 23 +++++++++++++++++++++++ 2 files changed, 35 insertions(+) diff --git a/Documentation/git-rev-parse.txt b/Documentation/git-rev-parse.txt index f26a7591e373..f0f9021f2a5a 100644 --- a/Documentation/git-rev-parse.txt +++ b/Documentation/git-rev-parse.txt @@ -159,6 +159,18 @@ for another option. unfortunately named tag "master"), and show them as full refnames (e.g. "refs/heads/master"). +--output-object-format=(sha1|sha256|storage):: + + Allow oids to be input from any object format that the current + repository supports. + + Specifying "sha1" translates if necessary and returns a sha1 oid. + + Specifying "sha256" translates if necessary and returns a sha256 oid. + + Specifying "storage" translates if necessary and returns an oid in + encoded in the storage hash algorithm. + Options for Objects ~~~~~~~~~~~~~~~~~~~ diff --git a/builtin/rev-parse.c b/builtin/rev-parse.c index 43e96765400c..0ef3e658cc5b 100644 --- a/builtin/rev-parse.c +++ b/builtin/rev-parse.c @@ -25,6 +25,7 @@ #include "submodule.h" #include "commit-reach.h" #include "shallow.h" +#include "object-file-convert.h" #define DO_REVS 1 #define DO_NOREV 2 @@ -675,6 +676,8 @@ static void print_path(const char *path, const char *prefix, enum format_type fo int cmd_rev_parse(int argc, const char **argv, const char *prefix) { int i, as_is = 0, verify = 0, quiet = 0, revs_count = 0, type = 0; + const struct git_hash_algo *output_algo = NULL; + const struct git_hash_algo *compat = NULL; int did_repo_setup = 0; int has_dashdash = 0; int output_prefix = 0; @@ -746,6 +749,7 @@ int cmd_rev_parse(int argc, const char **argv, const char *prefix) prepare_repo_settings(the_repository); the_repository->settings.command_requires_full_index = 0; + compat = the_repository->compat_hash_algo; } if (!strcmp(arg, "--")) { @@ -833,6 +837,22 @@ int cmd_rev_parse(int argc, const char **argv, const char *prefix) flags |= GET_OID_QUIETLY; continue; } + if (opt_with_value(arg, "--output-object-format", &arg)) { + if (!arg) + die(_("no object format specified")); + if (!strcmp(arg, the_hash_algo->name) || + !strcmp(arg, "storage")) { + flags |= GET_OID_HASH_ANY; + output_algo = the_hash_algo; + continue; + } + else if (compat && !strcmp(arg, compat->name)) { + flags |= GET_OID_HASH_ANY; + output_algo = compat; + continue; + } + else die(_("unsupported object format: %s"), arg); + } if (opt_with_value(arg, "--short", &arg)) { filter &= ~(DO_FLAGS|DO_NOREV); verify = 1; @@ -1083,6 +1103,9 @@ int cmd_rev_parse(int argc, const char **argv, const char *prefix) } if (!get_oid_with_context(the_repository, name, flags, &oid, &unused)) { + if (output_algo) + repo_oid_to_algop(the_repository, &oid, + output_algo, &oid); if (verify) revs_count++; else From patchwork Wed Sep 27 19:55:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13401633 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B4EF6E82CCD for ; Wed, 27 Sep 2023 19:56:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229968AbjI0T4u (ORCPT ); Wed, 27 Sep 2023 15:56:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35862 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229882AbjI0T4d (ORCPT ); Wed, 27 Sep 2023 15:56:33 -0400 Received: from mail-io1-xd30.google.com (mail-io1-xd30.google.com [IPv6:2607:f8b0:4864:20::d30]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2249B193 for ; Wed, 27 Sep 2023 12:56:08 -0700 (PDT) Received: by mail-io1-xd30.google.com with SMTP id ca18e2360f4ac-79faba5fe12so325526139f.3 for ; Wed, 27 Sep 2023 12:56:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1695844567; x=1696449367; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=qNlEB4NzHehHwb5iJrKXvrOJbxoQMMja8pt5tTtXO1Q=; b=aVsPheVl51TrUqCIUujV+TF+ucYsSoZnrN+anrB0PWqXWQjHnUQ7fl57t55WEnnlIU lrCnt0irFGwybgKfNxXRJtVBg7kIJRDJq+495/l9msbAboQpUCEHTF3lQqq4tHOy2Zgn 8SSVY5IjFEwO+ZK0XuosqzG+Es/F72nryTMaUmjQuo6XyIn0vPol3Vz+zUYbwqm8LzXA 5li0Q3cGJmq7/zXgZrMZN9zyIiomKmHaXT4whMmwtG7FVzTAF28yReH+0Oa75mKni/GJ VwDrXxfRswWF8oyMrpxyjjNpll5Gc6AAK9WPDuBfYT7SX793HtT/adVB93obMgDfmusg 6D2w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695844567; x=1696449367; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qNlEB4NzHehHwb5iJrKXvrOJbxoQMMja8pt5tTtXO1Q=; b=V+uOSSH1vr6Ojk4I0OsNigAJOxv3Jp/pjOc6RDhAgkFMinXsqyv5f6NaXV3e1qOFhp LER+pTstRfYIP2f7WIHxvSo+EkUpQ+tB2ii8qKhd5JnjHzNK6kbbFjPO/jTq8IJp3Pz5 l4l7V/CeSBqsLMNyItOVpfYw6Mk0B3nDoWUm8oIzl8WP+cKot1KTTGEHYflsZhCcjwOk jF62oedRFT4wtQW+i808bYGBnlF2mCLIgpoTewEAZA/F4F9RY92XDbYSWyJ6MC6z1OR2 gYT9ZudxKXqv4VM5jeul3Z6Pm1JbqOtD8a1CCACCFMF2lfpjPFf1UwtuHzS9/cov7AUw 2xrA== X-Gm-Message-State: AOJu0Ywqea0+7sCoC1Hi2ZA/aMzQAttrMaLRAMMpF0E38bKtnj/rQPGx USEhHRdBpA3bRJdhS/DyAls= X-Google-Smtp-Source: AGHT+IFQOatpyhpwVGMbJebw4JPloD7iq617g1hugzw3c/U0TL59OSOFh/f8KHNC0Rs+e+cWalkw9Q== X-Received: by 2002:a05:6602:2756:b0:792:9406:a453 with SMTP id b22-20020a056602275600b007929406a453mr3260855ioe.14.1695844567486; Wed, 27 Sep 2023 12:56:07 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id o23-20020a6b5a17000000b007836c7e8dccsm2935628iob.17.2023.09.27.12.56.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Sep 2023 12:56:07 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , "Eric W. Biederman" Subject: [PATCH 23/30] builtin/cat-file: Let the oid determine the output algorithm Date: Wed, 27 Sep 2023 14:55:30 -0500 Message-Id: <20230927195537.1682-23-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> References: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "Eric W. Biederman" Use GET_OID_HASH_ANY when calling get_oid_with_context. This implements the semi-obvious behaviour that specifying a sha1 oid shows the output for a sha1 encoded object, and specifying a sha256 oid shows the output for a sha256 encoded object. This is useful for testing the the conversion of an object to an equivalent object encoded with a different hash function. Signed-off-by: "Eric W. Biederman" --- builtin/cat-file.c | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/builtin/cat-file.c b/builtin/cat-file.c index 694c8538df2f..e615d1f8e0da 100644 --- a/builtin/cat-file.c +++ b/builtin/cat-file.c @@ -107,7 +107,10 @@ static int cat_one_file(int opt, const char *exp_type, const char *obj_name, struct object_info oi = OBJECT_INFO_INIT; struct strbuf sb = STRBUF_INIT; unsigned flags = OBJECT_INFO_LOOKUP_REPLACE; - unsigned get_oid_flags = GET_OID_RECORD_PATH | GET_OID_ONLY_TO_DIE; + unsigned get_oid_flags = + GET_OID_RECORD_PATH | + GET_OID_ONLY_TO_DIE | + GET_OID_HASH_ANY; const char *path = force_path; const int opt_cw = (opt == 'c' || opt == 'w'); if (!path && opt_cw) @@ -223,7 +226,8 @@ static int cat_one_file(int opt, const char *exp_type, const char *obj_name, &size); const char *target; if (!skip_prefix(buffer, "object ", &target) || - get_oid_hex(target, &blob_oid)) + get_oid_hex_algop(target, &blob_oid, + &hash_algos[oid.algo])) die("%s not a valid tag", oid_to_hex(&oid)); free(buffer); } else @@ -512,7 +516,9 @@ static void batch_one_object(const char *obj_name, struct expand_data *data) { struct object_context ctx; - int flags = opt->follow_symlinks ? GET_OID_FOLLOW_SYMLINKS : 0; + int flags = + GET_OID_HASH_ANY | + (opt->follow_symlinks ? GET_OID_FOLLOW_SYMLINKS : 0); enum get_oid_result result; result = get_oid_with_context(the_repository, obj_name, From patchwork Wed Sep 27 19:55:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13401637 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A3C94E82CCE for ; Wed, 27 Sep 2023 19:57:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229831AbjI0T5A (ORCPT ); Wed, 27 Sep 2023 15:57:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35934 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229836AbjI0T4f (ORCPT ); Wed, 27 Sep 2023 15:56:35 -0400 Received: from mail-io1-xd34.google.com (mail-io1-xd34.google.com [IPv6:2607:f8b0:4864:20::d34]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1DE4A1AD for ; Wed, 27 Sep 2023 12:56:09 -0700 (PDT) Received: by mail-io1-xd34.google.com with SMTP id ca18e2360f4ac-79fa425855cso347642239f.2 for ; Wed, 27 Sep 2023 12:56:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1695844568; x=1696449368; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ctOzb5UNmN2By2/XBixPoWkARfwfWfm6uJO9oRBQccg=; b=JTGt5Vqz4F3of2nNW0/xtgFBUUcYQ+Hy5Ep6UEpdIFTTIHrHERQap3wjLM7bgTJYC9 Q7yBm2ytWoLCc+8vpzfvXYkNIrsCZZenuNgaSrqGYtYbr5J2QJS4T8kB55TV9d9S429H 7SqWEAoJUhr2rfG5UnWJQyEz96i0NW6HkfnY4zfd4KppemWtWoo5SAeQCHE6wSnfQpr6 waHZb2xqWxBeNw0z8oDX5ODc+3ffeyHLoSZDXu+t7Rd0o0O7Ecp6YA3MstsA5NT28vyh b33PpVy3cXvqRdrbDMsADKwL1p1MEVVk53w11Mi+RmSK42zASrbLUCGDdz87R9mlM95/ PUjA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695844568; x=1696449368; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ctOzb5UNmN2By2/XBixPoWkARfwfWfm6uJO9oRBQccg=; b=c/KQbokMs7vvoPL0cyZ83wXxLfQltTpMF5OahlSWL9lTH/Kp15sXVZf7X5K7dSUkHr Wn2ZrVzT5B0vNhoiDhcuWZXedarNySE/vbNUWYgfGdJz7XAPcrfoFe+dQqNNSfHMT8oF EMx8Q1anNtFzaL7t5Mk7nVQ0CcTM91vLlL+t99vPbwUV7P0iCNvZhCmZT7xU7tSMl2HC tKlNwCLCx2wOmKYZLZwYeEILiAVf1kv/7A92idHgbxxWpceX6V57me6K1iyhEYiFwVqQ 4nkYGaRaGT4ZBPAxnhRRV+x6XEXI5j+7YYzSiwk/lRIuYg9JJYoQhn21oz+TLXMr+AM1 YvgQ== X-Gm-Message-State: AOJu0YxjpW3e2e3JtwJaV9dvSZSvQT0+d/XVmoz30JbwdQaaFS6JKpGh Q2Ng4Hq1ajUfRY0WQn1WpJiAQPOWkPw= X-Google-Smtp-Source: AGHT+IEQjvgJMwmqXs06iFfZ4uTBPNchwNfMehIfY4qcM8Oxpr+S4WDkr9LErD2Y3d6RSewMrV8OcA== X-Received: by 2002:a5e:8912:0:b0:792:72b8:b8a with SMTP id k18-20020a5e8912000000b0079272b80b8amr3100381ioj.18.1695844568197; Wed, 27 Sep 2023 12:56:08 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id o23-20020a6b5a17000000b007836c7e8dccsm2935628iob.17.2023.09.27.12.56.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Sep 2023 12:56:07 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , "Eric W. Biederman" Subject: [PATCH 24/30] tree-walk: init_tree_desc take an oid to get the hash algorithm Date: Wed, 27 Sep 2023 14:55:31 -0500 Message-Id: <20230927195537.1682-24-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> References: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "Eric W. Biederman" To make it possible for git ls-tree to display the tree encoded in the hash algorithm of the oid specified to git ls-tree, update init_tree_desc to take as a parameter the oid of the tree object. Update all callers of init_tree_desc and init_tree_desc_gently to pass the oid of the tree object. Use the oid of the tree object to discover the hash algorithm of the oid and store that hash algorithm in struct tree_desc. Use the hash algorithm in decode_tree_entry and update_tree_entry_internal to handle reading a tree object encoded in a hash algorithm that differs from the repositories hash algorithm. Signed-off-by: "Eric W. Biederman" --- archive.c | 3 ++- builtin/am.c | 6 +++--- builtin/checkout.c | 8 +++++--- builtin/clone.c | 2 +- builtin/commit.c | 2 +- builtin/grep.c | 8 ++++---- builtin/merge.c | 3 ++- builtin/pack-objects.c | 6 ++++-- builtin/read-tree.c | 2 +- builtin/stash.c | 5 +++-- cache-tree.c | 2 +- delta-islands.c | 2 +- diff-lib.c | 2 +- fsck.c | 6 ++++-- http-push.c | 2 +- list-objects.c | 2 +- match-trees.c | 4 ++-- merge-ort.c | 11 ++++++----- merge-recursive.c | 2 +- merge.c | 3 ++- pack-bitmap-write.c | 2 +- packfile.c | 3 ++- reflog.c | 2 +- revision.c | 4 ++-- tree-walk.c | 36 +++++++++++++++++++++--------------- tree-walk.h | 7 +++++-- tree.c | 2 +- walker.c | 2 +- 28 files changed, 80 insertions(+), 59 deletions(-) diff --git a/archive.c b/archive.c index ca11db185b15..b10269aee7be 100644 --- a/archive.c +++ b/archive.c @@ -339,7 +339,8 @@ int write_archive_entries(struct archiver_args *args, opts.src_index = args->repo->index; opts.dst_index = args->repo->index; opts.fn = oneway_merge; - init_tree_desc(&t, args->tree->buffer, args->tree->size); + init_tree_desc(&t, &args->tree->object.oid, + args->tree->buffer, args->tree->size); if (unpack_trees(1, &t, &opts)) return -1; git_attr_set_direction(GIT_ATTR_INDEX); diff --git a/builtin/am.c b/builtin/am.c index 8bde034fae68..4dfd714b910e 100644 --- a/builtin/am.c +++ b/builtin/am.c @@ -1991,8 +1991,8 @@ static int fast_forward_to(struct tree *head, struct tree *remote, int reset) opts.reset = reset ? UNPACK_RESET_PROTECT_UNTRACKED : 0; opts.preserve_ignored = 0; /* FIXME: !overwrite_ignore */ opts.fn = twoway_merge; - init_tree_desc(&t[0], head->buffer, head->size); - init_tree_desc(&t[1], remote->buffer, remote->size); + init_tree_desc(&t[0], &head->object.oid, head->buffer, head->size); + init_tree_desc(&t[1], &remote->object.oid, remote->buffer, remote->size); if (unpack_trees(2, t, &opts)) { rollback_lock_file(&lock_file); @@ -2026,7 +2026,7 @@ static int merge_tree(struct tree *tree) opts.dst_index = &the_index; opts.merge = 1; opts.fn = oneway_merge; - init_tree_desc(&t[0], tree->buffer, tree->size); + init_tree_desc(&t[0], &tree->object.oid, tree->buffer, tree->size); if (unpack_trees(1, t, &opts)) { rollback_lock_file(&lock_file); diff --git a/builtin/checkout.c b/builtin/checkout.c index f53612f46870..03eff73fd031 100644 --- a/builtin/checkout.c +++ b/builtin/checkout.c @@ -701,7 +701,7 @@ static int reset_tree(struct tree *tree, const struct checkout_opts *o, info->commit ? &info->commit->object.oid : null_oid(), NULL); parse_tree(tree); - init_tree_desc(&tree_desc, tree->buffer, tree->size); + init_tree_desc(&tree_desc, &tree->object.oid, tree->buffer, tree->size); switch (unpack_trees(1, &tree_desc, &opts)) { case -2: *writeout_error = 1; @@ -815,10 +815,12 @@ static int merge_working_tree(const struct checkout_opts *opts, die(_("unable to parse commit %s"), oid_to_hex(old_commit_oid)); - init_tree_desc(&trees[0], tree->buffer, tree->size); + init_tree_desc(&trees[0], &tree->object.oid, + tree->buffer, tree->size); parse_tree(new_tree); tree = new_tree; - init_tree_desc(&trees[1], tree->buffer, tree->size); + init_tree_desc(&trees[1], &tree->object.oid, + tree->buffer, tree->size); ret = unpack_trees(2, trees, &topts); clear_unpack_trees_porcelain(&topts); diff --git a/builtin/clone.c b/builtin/clone.c index c6357af94989..79ceefb93995 100644 --- a/builtin/clone.c +++ b/builtin/clone.c @@ -737,7 +737,7 @@ static int checkout(int submodule_progress, int filter_submodules) if (!tree) die(_("unable to parse commit %s"), oid_to_hex(&oid)); parse_tree(tree); - init_tree_desc(&t, tree->buffer, tree->size); + init_tree_desc(&t, &tree->object.oid, tree->buffer, tree->size); if (unpack_trees(1, &t, &opts) < 0) die(_("unable to checkout working tree")); diff --git a/builtin/commit.c b/builtin/commit.c index 7da5f924484d..537319932b65 100644 --- a/builtin/commit.c +++ b/builtin/commit.c @@ -340,7 +340,7 @@ static void create_base_index(const struct commit *current_head) if (!tree) die(_("failed to unpack HEAD tree object")); parse_tree(tree); - init_tree_desc(&t, tree->buffer, tree->size); + init_tree_desc(&t, &tree->object.oid, tree->buffer, tree->size); if (unpack_trees(1, &t, &opts)) exit(128); /* We've already reported the error, finish dying */ } diff --git a/builtin/grep.c b/builtin/grep.c index 50e712a18479..0c2b8a376f8e 100644 --- a/builtin/grep.c +++ b/builtin/grep.c @@ -530,7 +530,7 @@ static int grep_submodule(struct grep_opt *opt, strbuf_addstr(&base, filename); strbuf_addch(&base, '/'); - init_tree_desc(&tree, data, size); + init_tree_desc(&tree, oid, data, size); hit = grep_tree(&subopt, pathspec, &tree, &base, base.len, object_type == OBJ_COMMIT); strbuf_release(&base); @@ -574,7 +574,7 @@ static int grep_cache(struct grep_opt *opt, data = repo_read_object_file(the_repository, &ce->oid, &type, &size); - init_tree_desc(&tree, data, size); + init_tree_desc(&tree, &ce->oid, data, size); hit |= grep_tree(opt, pathspec, &tree, &name, 0, 0); strbuf_setlen(&name, name_base_len); @@ -670,7 +670,7 @@ static int grep_tree(struct grep_opt *opt, const struct pathspec *pathspec, oid_to_hex(&entry.oid)); strbuf_addch(base, '/'); - init_tree_desc(&sub, data, size); + init_tree_desc(&sub, &entry.oid, data, size); hit |= grep_tree(opt, pathspec, &sub, base, tn_len, check_attr); free(data); @@ -714,7 +714,7 @@ static int grep_object(struct grep_opt *opt, const struct pathspec *pathspec, strbuf_add(&base, name, len); strbuf_addch(&base, ':'); } - init_tree_desc(&tree, data, size); + init_tree_desc(&tree, &obj->oid, data, size); hit = grep_tree(opt, pathspec, &tree, &base, base.len, obj->type == OBJ_COMMIT); strbuf_release(&base); diff --git a/builtin/merge.c b/builtin/merge.c index de68910177fb..718165d45917 100644 --- a/builtin/merge.c +++ b/builtin/merge.c @@ -704,7 +704,8 @@ static int read_tree_trivial(struct object_id *common, struct object_id *head, cache_tree_free(&the_index.cache_tree); for (i = 0; i < nr_trees; i++) { parse_tree(trees[i]); - init_tree_desc(t+i, trees[i]->buffer, trees[i]->size); + init_tree_desc(t+i, &trees[i]->object.oid, + trees[i]->buffer, trees[i]->size); } if (unpack_trees(nr_trees, t, &opts)) return -1; diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c index d2a162d52804..d34902002656 100644 --- a/builtin/pack-objects.c +++ b/builtin/pack-objects.c @@ -1756,7 +1756,8 @@ static void add_pbase_object(struct tree_desc *tree, tree = pbase_tree_get(&entry.oid); if (!tree) return; - init_tree_desc(&sub, tree->tree_data, tree->tree_size); + init_tree_desc(&sub, &tree->oid, + tree->tree_data, tree->tree_size); add_pbase_object(&sub, down, downlen, fullname); pbase_tree_put(tree); @@ -1816,7 +1817,8 @@ static void add_preferred_base_object(const char *name) } else { struct tree_desc tree; - init_tree_desc(&tree, it->pcache.tree_data, it->pcache.tree_size); + init_tree_desc(&tree, &it->pcache.oid, + it->pcache.tree_data, it->pcache.tree_size); add_pbase_object(&tree, name, cmplen, name); } } diff --git a/builtin/read-tree.c b/builtin/read-tree.c index 1fec702a04fa..24d6d156d3a2 100644 --- a/builtin/read-tree.c +++ b/builtin/read-tree.c @@ -264,7 +264,7 @@ int cmd_read_tree(int argc, const char **argv, const char *cmd_prefix) for (i = 0; i < nr_trees; i++) { struct tree *tree = trees[i]; parse_tree(tree); - init_tree_desc(t+i, tree->buffer, tree->size); + init_tree_desc(t+i, &tree->object.oid, tree->buffer, tree->size); } if (unpack_trees(nr_trees, t, &opts)) return 128; diff --git a/builtin/stash.c b/builtin/stash.c index fe64cde9ce30..9ee52af4d28e 100644 --- a/builtin/stash.c +++ b/builtin/stash.c @@ -285,7 +285,7 @@ static int reset_tree(struct object_id *i_tree, int update, int reset) if (parse_tree(tree)) return -1; - init_tree_desc(t, tree->buffer, tree->size); + init_tree_desc(t, &tree->object.oid, tree->buffer, tree->size); opts.head_idx = 1; opts.src_index = &the_index; @@ -871,7 +871,8 @@ static void diff_include_untracked(const struct stash_info *info, struct diff_op tree[i] = parse_tree_indirect(oid[i]); if (parse_tree(tree[i]) < 0) die(_("failed to parse tree")); - init_tree_desc(&tree_desc[i], tree[i]->buffer, tree[i]->size); + init_tree_desc(&tree_desc[i], &tree[i]->object.oid, + tree[i]->buffer, tree[i]->size); } unpack_tree_opt.head_idx = -1; diff --git a/cache-tree.c b/cache-tree.c index ddc7d3d86959..334973a01cee 100644 --- a/cache-tree.c +++ b/cache-tree.c @@ -770,7 +770,7 @@ static void prime_cache_tree_rec(struct repository *r, oidcpy(&it->oid, &tree->object.oid); - init_tree_desc(&desc, tree->buffer, tree->size); + init_tree_desc(&desc, &tree->object.oid, tree->buffer, tree->size); cnt = 0; while (tree_entry(&desc, &entry)) { if (!S_ISDIR(entry.mode)) diff --git a/delta-islands.c b/delta-islands.c index 5de5759f3f13..1ff3506b10f2 100644 --- a/delta-islands.c +++ b/delta-islands.c @@ -289,7 +289,7 @@ void resolve_tree_islands(struct repository *r, if (!tree || parse_tree(tree) < 0) die(_("bad tree object %s"), oid_to_hex(&ent->idx.oid)); - init_tree_desc(&desc, tree->buffer, tree->size); + init_tree_desc(&desc, &tree->object.oid, tree->buffer, tree->size); while (tree_entry(&desc, &entry)) { struct object *obj; diff --git a/diff-lib.c b/diff-lib.c index 6b0c6a7180cc..add323f5628d 100644 --- a/diff-lib.c +++ b/diff-lib.c @@ -558,7 +558,7 @@ static int diff_cache(struct rev_info *revs, opts.pathspec = &revs->diffopt.pathspec; opts.pathspec->recursive = 1; - init_tree_desc(&t, tree->buffer, tree->size); + init_tree_desc(&t, &tree->object.oid, tree->buffer, tree->size); return unpack_trees(1, &t, &opts); } diff --git a/fsck.c b/fsck.c index 2b1e348005b7..6b492a48da82 100644 --- a/fsck.c +++ b/fsck.c @@ -313,7 +313,8 @@ static int fsck_walk_tree(struct tree *tree, void *data, struct fsck_options *op return -1; name = fsck_get_object_name(options, &tree->object.oid); - if (init_tree_desc_gently(&desc, tree->buffer, tree->size, 0)) + if (init_tree_desc_gently(&desc, &tree->object.oid, + tree->buffer, tree->size, 0)) return -1; while (tree_entry_gently(&desc, &entry)) { struct object *obj; @@ -583,7 +584,8 @@ static int fsck_tree(const struct object_id *tree_oid, const char *o_name; struct name_stack df_dup_candidates = { NULL }; - if (init_tree_desc_gently(&desc, buffer, size, TREE_DESC_RAW_MODES)) { + if (init_tree_desc_gently(&desc, tree_oid, buffer, size, + TREE_DESC_RAW_MODES)) { retval += report(options, tree_oid, OBJ_TREE, FSCK_MSG_BAD_TREE, "cannot be parsed as a tree"); diff --git a/http-push.c b/http-push.c index a704f490fdb2..81c35b5e96f7 100644 --- a/http-push.c +++ b/http-push.c @@ -1308,7 +1308,7 @@ static struct object_list **process_tree(struct tree *tree, obj->flags |= SEEN; p = add_one_object(obj, p); - init_tree_desc(&desc, tree->buffer, tree->size); + init_tree_desc(&desc, &tree->object.oid, tree->buffer, tree->size); while (tree_entry(&desc, &entry)) switch (object_type(entry.mode)) { diff --git a/list-objects.c b/list-objects.c index e60a6cd5b46e..312335c8a7f2 100644 --- a/list-objects.c +++ b/list-objects.c @@ -97,7 +97,7 @@ static void process_tree_contents(struct traversal_context *ctx, enum interesting match = ctx->revs->diffopt.pathspec.nr == 0 ? all_entries_interesting : entry_not_interesting; - init_tree_desc(&desc, tree->buffer, tree->size); + init_tree_desc(&desc, &tree->object.oid, tree->buffer, tree->size); while (tree_entry(&desc, &entry)) { if (match != all_entries_interesting) { diff --git a/match-trees.c b/match-trees.c index 0885ac681cd5..3412b6a1401d 100644 --- a/match-trees.c +++ b/match-trees.c @@ -63,7 +63,7 @@ static void *fill_tree_desc_strict(struct tree_desc *desc, die("unable to read tree (%s)", oid_to_hex(hash)); if (type != OBJ_TREE) die("%s is not a tree", oid_to_hex(hash)); - init_tree_desc(desc, buffer, size); + init_tree_desc(desc, hash, buffer, size); return buffer; } @@ -194,7 +194,7 @@ static int splice_tree(const struct object_id *oid1, const char *prefix, buf = repo_read_object_file(the_repository, oid1, &type, &sz); if (!buf) die("cannot read tree %s", oid_to_hex(oid1)); - init_tree_desc(&desc, buf, sz); + init_tree_desc(&desc, oid1, buf, sz); rewrite_here = NULL; while (desc.size) { diff --git a/merge-ort.c b/merge-ort.c index 8631c997002d..3a5729c91e48 100644 --- a/merge-ort.c +++ b/merge-ort.c @@ -1679,9 +1679,10 @@ static int collect_merge_info(struct merge_options *opt, parse_tree(merge_base); parse_tree(side1); parse_tree(side2); - init_tree_desc(t + 0, merge_base->buffer, merge_base->size); - init_tree_desc(t + 1, side1->buffer, side1->size); - init_tree_desc(t + 2, side2->buffer, side2->size); + init_tree_desc(t + 0, &merge_base->object.oid, + merge_base->buffer, merge_base->size); + init_tree_desc(t + 1, &side1->object.oid, side1->buffer, side1->size); + init_tree_desc(t + 2, &side2->object.oid, side2->buffer, side2->size); trace2_region_enter("merge", "traverse_trees", opt->repo); ret = traverse_trees(NULL, 3, t, &info); @@ -4400,9 +4401,9 @@ static int checkout(struct merge_options *opt, unpack_opts.fn = twoway_merge; unpack_opts.preserve_ignored = 0; /* FIXME: !opts->overwrite_ignore */ parse_tree(prev); - init_tree_desc(&trees[0], prev->buffer, prev->size); + init_tree_desc(&trees[0], &prev->object.oid, prev->buffer, prev->size); parse_tree(next); - init_tree_desc(&trees[1], next->buffer, next->size); + init_tree_desc(&trees[1], &next->object.oid, next->buffer, next->size); ret = unpack_trees(2, trees, &unpack_opts); clear_unpack_trees_porcelain(&unpack_opts); diff --git a/merge-recursive.c b/merge-recursive.c index 6a4081bb0f52..93df9eecdd95 100644 --- a/merge-recursive.c +++ b/merge-recursive.c @@ -411,7 +411,7 @@ static inline int merge_detect_rename(struct merge_options *opt) static void init_tree_desc_from_tree(struct tree_desc *desc, struct tree *tree) { parse_tree(tree); - init_tree_desc(desc, tree->buffer, tree->size); + init_tree_desc(desc, &tree->object.oid, tree->buffer, tree->size); } static int unpack_trees_start(struct merge_options *opt, diff --git a/merge.c b/merge.c index b60925459c29..86179c34102d 100644 --- a/merge.c +++ b/merge.c @@ -81,7 +81,8 @@ int checkout_fast_forward(struct repository *r, } for (i = 0; i < nr_trees; i++) { parse_tree(trees[i]); - init_tree_desc(t+i, trees[i]->buffer, trees[i]->size); + init_tree_desc(t+i, &trees[i]->object.oid, + trees[i]->buffer, trees[i]->size); } memset(&opts, 0, sizeof(opts)); diff --git a/pack-bitmap-write.c b/pack-bitmap-write.c index f6757c3cbf20..9211e08f0127 100644 --- a/pack-bitmap-write.c +++ b/pack-bitmap-write.c @@ -366,7 +366,7 @@ static int fill_bitmap_tree(struct bitmap *bitmap, if (parse_tree(tree) < 0) die("unable to load tree object %s", oid_to_hex(&tree->object.oid)); - init_tree_desc(&desc, tree->buffer, tree->size); + init_tree_desc(&desc, &tree->object.oid, tree->buffer, tree->size); while (tree_entry(&desc, &entry)) { switch (object_type(entry.mode)) { diff --git a/packfile.c b/packfile.c index 9cc0a2e37a83..1fae0fcdd9e7 100644 --- a/packfile.c +++ b/packfile.c @@ -2250,7 +2250,8 @@ static int add_promisor_object(const struct object_id *oid, struct tree *tree = (struct tree *)obj; struct tree_desc desc; struct name_entry entry; - if (init_tree_desc_gently(&desc, tree->buffer, tree->size, 0)) + if (init_tree_desc_gently(&desc, &tree->object.oid, + tree->buffer, tree->size, 0)) /* * Error messages are given when packs are * verified, so do not print any here. diff --git a/reflog.c b/reflog.c index 9ad50e7d93e4..c6992a19268f 100644 --- a/reflog.c +++ b/reflog.c @@ -40,7 +40,7 @@ static int tree_is_complete(const struct object_id *oid) tree->buffer = data; tree->size = size; } - init_tree_desc(&desc, tree->buffer, tree->size); + init_tree_desc(&desc, &tree->object.oid, tree->buffer, tree->size); complete = 1; while (tree_entry(&desc, &entry)) { if (!repo_has_object_file(the_repository, &entry.oid) || diff --git a/revision.c b/revision.c index 2f4c53ea207b..a60dfc23a2a5 100644 --- a/revision.c +++ b/revision.c @@ -82,7 +82,7 @@ static void mark_tree_contents_uninteresting(struct repository *r, if (parse_tree_gently(tree, 1) < 0) return; - init_tree_desc(&desc, tree->buffer, tree->size); + init_tree_desc(&desc, &tree->object.oid, tree->buffer, tree->size); while (tree_entry(&desc, &entry)) { switch (object_type(entry.mode)) { case OBJ_TREE: @@ -189,7 +189,7 @@ static void add_children_by_path(struct repository *r, if (parse_tree_gently(tree, 1) < 0) return; - init_tree_desc(&desc, tree->buffer, tree->size); + init_tree_desc(&desc, &tree->object.oid, tree->buffer, tree->size); while (tree_entry(&desc, &entry)) { switch (object_type(entry.mode)) { case OBJ_TREE: diff --git a/tree-walk.c b/tree-walk.c index 3af50a01c2c7..0b44ec7c75ff 100644 --- a/tree-walk.c +++ b/tree-walk.c @@ -15,7 +15,7 @@ static int decode_tree_entry(struct tree_desc *desc, const char *buf, unsigned l const char *path; unsigned int len; uint16_t mode; - const unsigned hashsz = the_hash_algo->rawsz; + const unsigned hashsz = desc->algo->rawsz; if (size < hashsz + 3 || buf[size - (hashsz + 1)]) { strbuf_addstr(err, _("too-short tree object")); @@ -37,15 +37,19 @@ static int decode_tree_entry(struct tree_desc *desc, const char *buf, unsigned l desc->entry.path = path; desc->entry.mode = (desc->flags & TREE_DESC_RAW_MODES) ? mode : canon_mode(mode); desc->entry.pathlen = len - 1; - oidread(&desc->entry.oid, (const unsigned char *)path + len); + oidread_algop(&desc->entry.oid, (const unsigned char *)path + len, + desc->algo); return 0; } -static int init_tree_desc_internal(struct tree_desc *desc, const void *buffer, - unsigned long size, struct strbuf *err, +static int init_tree_desc_internal(struct tree_desc *desc, + const struct object_id *oid, + const void *buffer, unsigned long size, + struct strbuf *err, enum tree_desc_flags flags) { + desc->algo = (oid && oid->algo) ? &hash_algos[oid->algo] : the_hash_algo; desc->buffer = buffer; desc->size = size; desc->flags = flags; @@ -54,19 +58,21 @@ static int init_tree_desc_internal(struct tree_desc *desc, const void *buffer, return 0; } -void init_tree_desc(struct tree_desc *desc, const void *buffer, unsigned long size) +void init_tree_desc(struct tree_desc *desc, const struct object_id *tree_oid, + const void *buffer, unsigned long size) { struct strbuf err = STRBUF_INIT; - if (init_tree_desc_internal(desc, buffer, size, &err, 0)) + if (init_tree_desc_internal(desc, tree_oid, buffer, size, &err, 0)) die("%s", err.buf); strbuf_release(&err); } -int init_tree_desc_gently(struct tree_desc *desc, const void *buffer, unsigned long size, +int init_tree_desc_gently(struct tree_desc *desc, const struct object_id *oid, + const void *buffer, unsigned long size, enum tree_desc_flags flags) { struct strbuf err = STRBUF_INIT; - int result = init_tree_desc_internal(desc, buffer, size, &err, flags); + int result = init_tree_desc_internal(desc, oid, buffer, size, &err, flags); if (result) error("%s", err.buf); strbuf_release(&err); @@ -85,7 +91,7 @@ void *fill_tree_descriptor(struct repository *r, if (!buf) die("unable to read tree %s", oid_to_hex(oid)); } - init_tree_desc(desc, buf, size); + init_tree_desc(desc, oid, buf, size); return buf; } @@ -102,7 +108,7 @@ static void entry_extract(struct tree_desc *t, struct name_entry *a) static int update_tree_entry_internal(struct tree_desc *desc, struct strbuf *err) { const void *buf = desc->buffer; - const unsigned char *end = (const unsigned char *)desc->entry.path + desc->entry.pathlen + 1 + the_hash_algo->rawsz; + const unsigned char *end = (const unsigned char *)desc->entry.path + desc->entry.pathlen + 1 + desc->algo->rawsz; unsigned long size = desc->size; unsigned long len = end - (const unsigned char *)buf; @@ -611,7 +617,7 @@ int get_tree_entry(struct repository *r, retval = -1; } else { struct tree_desc t; - init_tree_desc(&t, tree, size); + init_tree_desc(&t, tree_oid, tree, size); retval = find_tree_entry(r, &t, name, oid, mode); } free(tree); @@ -654,7 +660,7 @@ enum get_oid_result get_tree_entry_follow_symlinks(struct repository *r, struct tree_desc t; int follows_remaining = GET_TREE_ENTRY_FOLLOW_SYMLINKS_MAX_LINKS; - init_tree_desc(&t, NULL, 0UL); + init_tree_desc(&t, NULL, NULL, 0UL); strbuf_addstr(&namebuf, name); oidcpy(¤t_tree_oid, tree_oid); @@ -690,7 +696,7 @@ enum get_oid_result get_tree_entry_follow_symlinks(struct repository *r, goto done; /* descend */ - init_tree_desc(&t, tree, size); + init_tree_desc(&t, ¤t_tree_oid, tree, size); } /* Handle symlinks to e.g. a//b by removing leading slashes */ @@ -724,7 +730,7 @@ enum get_oid_result get_tree_entry_follow_symlinks(struct repository *r, free(parent->tree); parents_nr--; parent = &parents[parents_nr - 1]; - init_tree_desc(&t, parent->tree, parent->size); + init_tree_desc(&t, &parent->oid, parent->tree, parent->size); strbuf_remove(&namebuf, 0, remainder ? 3 : 2); continue; } @@ -804,7 +810,7 @@ enum get_oid_result get_tree_entry_follow_symlinks(struct repository *r, contents_start = contents; parent = &parents[parents_nr - 1]; - init_tree_desc(&t, parent->tree, parent->size); + init_tree_desc(&t, &parent->oid, parent->tree, parent->size); strbuf_splice(&namebuf, 0, len, contents_start, link_len); if (remainder) diff --git a/tree-walk.h b/tree-walk.h index 74cdceb3fed2..cf54d01019e9 100644 --- a/tree-walk.h +++ b/tree-walk.h @@ -26,6 +26,7 @@ struct name_entry { * A semi-opaque data structure used to maintain the current state of the walk. */ struct tree_desc { + const struct git_hash_algo *algo; /* * pointer into the memory representation of the tree. It always * points at the current entry being visited. @@ -85,9 +86,11 @@ int update_tree_entry_gently(struct tree_desc *); * size parameters are assumed to be the same as the buffer and size * members of `struct tree`. */ -void init_tree_desc(struct tree_desc *desc, const void *buf, unsigned long size); +void init_tree_desc(struct tree_desc *desc, const struct object_id *tree_oid, + const void *buf, unsigned long size); -int init_tree_desc_gently(struct tree_desc *desc, const void *buf, unsigned long size, +int init_tree_desc_gently(struct tree_desc *desc, const struct object_id *oid, + const void *buf, unsigned long size, enum tree_desc_flags flags); /* diff --git a/tree.c b/tree.c index c745462f968e..44bcf728f10a 100644 --- a/tree.c +++ b/tree.c @@ -27,7 +27,7 @@ int read_tree_at(struct repository *r, if (parse_tree(tree)) return -1; - init_tree_desc(&desc, tree->buffer, tree->size); + init_tree_desc(&desc, &tree->object.oid, tree->buffer, tree->size); while (tree_entry(&desc, &entry)) { if (retval != all_entries_interesting) { diff --git a/walker.c b/walker.c index 65002a7220ad..c0fd632d921c 100644 --- a/walker.c +++ b/walker.c @@ -45,7 +45,7 @@ static int process_tree(struct walker *walker, struct tree *tree) if (parse_tree(tree)) return -1; - init_tree_desc(&desc, tree->buffer, tree->size); + init_tree_desc(&desc, &tree->object.oid, tree->buffer, tree->size); while (tree_entry(&desc, &entry)) { struct object *obj = NULL; From patchwork Wed Sep 27 19:55:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13401634 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 35A99E82CCD for ; Wed, 27 Sep 2023 19:56:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229910AbjI0T45 (ORCPT ); Wed, 27 Sep 2023 15:56:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43828 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229902AbjI0T4e (ORCPT ); Wed, 27 Sep 2023 15:56:34 -0400 Received: from mail-io1-xd36.google.com (mail-io1-xd36.google.com [IPv6:2607:f8b0:4864:20::d36]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DF1DACDC for ; Wed, 27 Sep 2023 12:56:09 -0700 (PDT) Received: by mail-io1-xd36.google.com with SMTP id ca18e2360f4ac-79fb70402a7so294061839f.0 for ; Wed, 27 Sep 2023 12:56:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1695844569; x=1696449369; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7dnxcgIKj3s0Ny3j12+gtpGpedSlcjdex5cQNRhJerE=; b=jBiMsu/n0gMoahCz9X5fGrW0mLntQP+/v4RFg4+UQHaueDh3bHTojaVarPvUYYzI3B GgWojBo8jQ659sEYWfZGSCyRaRikT6u45NNGBxWLJUt/11xfgNeXKF2aJlPvEY8hoqO7 8y8rcpiZXeggiSuCxZXUz+cflaIGVC+o0c3e1WuIIAQP03KKzjjeRypfGQg+GDsYMtRI PZU2rCV9sdVRg+JsXwnYKeprrLx2+c0yFu22wjVHSg8GWeUTb9r3BCGltbExHOT+B/xz TfOzUoCpSZUmGTZyNVjvrT5aADJXP1Ks4X22AdYypdx1r+Y8bEcdRInDsvmhBWR5VgRw bruw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695844569; x=1696449369; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7dnxcgIKj3s0Ny3j12+gtpGpedSlcjdex5cQNRhJerE=; b=beyxn7yPciazpyCrIA+yp8aYbKcwHC2NysOrPGIxGKr3RNut8hzf2nPvUeh/e1EeBT UCuUiFNPfMXKPYExK1fvIlvL78od0I7G4YhSWkpwNNhY3MQAmBWaU6/D/UrnMeCtZ2ZK ccApQy0L4ErCx0UPXm0Tlzzq6CK+MJe6lA60BJHGZOtlT/04ePQy5AbTelH5LNvX0w2l wpYIhEj+WgHPft+beKOkjbqtILqXoulZ8MYBi79dYsUdcyY0fOjODaILyyrV9BR88/57 in7ra0TpEm/kYEz1Mz2qqGXFu7y6ZPWY49kV3zaaBTNmcoVVUCXMRyjViSfeYRk46R84 8i+w== X-Gm-Message-State: AOJu0Yz/cAYTjD+1F3D6c1F+HzL9WC+3kIHS1eol+JPAujphBaOdBWVK TakmbSlIwt5o1SlOanySxyY= X-Google-Smtp-Source: AGHT+IHeTGAaND2c5MjcSWI05jYu1Z+g3gX0QKoky+j3Dh5S1X9X0JKCuMomC1OmDdmwolNSVGmdpg== X-Received: by 2002:a05:6602:39a:b0:791:1e87:b47e with SMTP id f26-20020a056602039a00b007911e87b47emr3459633iov.15.1695844569202; Wed, 27 Sep 2023 12:56:09 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id o23-20020a6b5a17000000b007836c7e8dccsm2935628iob.17.2023.09.27.12.56.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Sep 2023 12:56:08 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , "Eric W. Biederman" Subject: [PATCH 25/30] object-file: Handle compat objects in check_object_signature Date: Wed, 27 Sep 2023 14:55:32 -0500 Message-Id: <20230927195537.1682-25-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> References: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "Eric W. Biederman" Update check_object_signature to find the hash algorithm the exising signature uses, and to use the same hash algorithm when recomputing it to check the signature is valid. This will be useful when teaching git ls-tree to display objects encoded with the compat hash algorithm. Signed-off-by: "Eric W. Biederman" --- object-file.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/object-file.c b/object-file.c index 1601d624c9fd..df49d2239f24 100644 --- a/object-file.c +++ b/object-file.c @@ -1094,9 +1094,11 @@ int check_object_signature(struct repository *r, const struct object_id *oid, void *buf, unsigned long size, enum object_type type) { + const struct git_hash_algo *algo = + oid->algo ? &hash_algos[oid->algo] : r->hash_algo; struct object_id real_oid; - hash_object_file(r->hash_algo, buf, size, type, &real_oid); + hash_object_file(algo, buf, size, type, &real_oid); return !oideq(oid, &real_oid) ? -1 : 0; } From patchwork Wed Sep 27 19:55:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13401635 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B68BBE82CCC for ; Wed, 27 Sep 2023 19:56:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229807AbjI0T47 (ORCPT ); Wed, 27 Sep 2023 15:56:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35916 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229837AbjI0T4e (ORCPT ); Wed, 27 Sep 2023 15:56:34 -0400 Received: from mail-io1-xd31.google.com (mail-io1-xd31.google.com [IPv6:2607:f8b0:4864:20::d31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A6216CE3 for ; Wed, 27 Sep 2023 12:56:10 -0700 (PDT) Received: by mail-io1-xd31.google.com with SMTP id ca18e2360f4ac-79fa87d8f7aso343454439f.3 for ; Wed, 27 Sep 2023 12:56:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1695844570; x=1696449370; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=bH2hzoWai8vH7Zjh0kf9Emav2ZMVmyeQiZtNTadsv4M=; b=mOZRSvFuuuaYGmdH1Poe9JzBOzPCbeLgDDw0FnsFR0JADmHlB114XUYPZwfLUgP8Ts GBiPXh0guaopV7DdEukdVisJIAHtcARmjve4tocaJYWCWC+7tvp6+puiTCdteld9Cw6n 1dWb+Kwp8sCt5WpKz4Ez4J/6edi2/Hv7dw2iXgpKo6IrROXyeLCHtA2r/G1oDjQYFQz+ MPTGHybEOJk8yz0fQziSdn6PohAZHzTNhXVF1Hu/79k06ebwyVpUkU+Z5c8G74Foursl LkT3leu/LW7mopIUiACH2rFP8yf13pz4XKERf+F8ztONfeRUPM1/6U/OVxZnFUopdoT4 gvuA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695844570; x=1696449370; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bH2hzoWai8vH7Zjh0kf9Emav2ZMVmyeQiZtNTadsv4M=; b=P63i3dlgAgL9E7we5FA//ZhtGxLMz6YkAmMazTFC6m1en/d0hOCMLjVCdgmCB6BQU5 txgo8aBZZ2m5P3GCPcFlIE8UMGi5+oDSag429lQ2DNa4d6p5wmIAIOHbEJNUIwFWh6rg wDo/wAXkyRyEQIYfm+Cd4Hq4g+CNtzhUd8D0JbzO0xXCBLfVbjbifXFspiT0Aatrdc0L 8gCfgFZjzAG08VzVPptFXqGEH5Fdsve02TvEYMGlPd7mpCqKXJziB8dF0qxQp/H7bcOH OpK5UCII5zN/wkIa29fyIoxSzkR2spyVnQIPmgRp9NtMmuB5QCOTa3r/4T3R3LS3JSR0 JS5A== X-Gm-Message-State: AOJu0YzlomgwVY0N0ooxrTIPier58swJf4UmhV6RV0gnw0AMa8OobxPe rDY05gQHKi9KpfnaJ3AkAjM= X-Google-Smtp-Source: AGHT+IFAQ20+gJHatmpOOAW17oWZGaVhHsDUl1kxlPof4VE3IiOUQZ8yq1vpk99CWQ7tVh4dqFwUYg== X-Received: by 2002:a6b:4902:0:b0:780:d76c:b640 with SMTP id u2-20020a6b4902000000b00780d76cb640mr2966848iob.9.1695844569892; Wed, 27 Sep 2023 12:56:09 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id o23-20020a6b5a17000000b007836c7e8dccsm2935628iob.17.2023.09.27.12.56.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Sep 2023 12:56:09 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , "Eric W. Biederman" Subject: [PATCH 26/30] builtin/ls-tree: Let the oid determine the output algorithm Date: Wed, 27 Sep 2023 14:55:33 -0500 Message-Id: <20230927195537.1682-26-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> References: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "Eric W. Biederman" Update cmd_ls_tree to call get_oid_with_context and pass GET_OID_HASH_ANY instead of calling the simpler repo_get_oid. This implments in ls-tree the behavior that asking to display a sha1 hash displays the corrresponding sha1 encoded object and asking to display a sha256 hash displayes the corresponding sha256 encoded object. This is useful for testing the conversion of an object to an equivlanet object encoded with a different hash function. Signed-off-by: "Eric W. Biederman" --- builtin/ls-tree.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/builtin/ls-tree.c b/builtin/ls-tree.c index f558db5f3b80..71281ab705b6 100644 --- a/builtin/ls-tree.c +++ b/builtin/ls-tree.c @@ -376,6 +376,7 @@ int cmd_ls_tree(int argc, const char **argv, const char *prefix) OPT_END() }; struct ls_tree_cmdmode_to_fmt *m2f = ls_tree_cmdmode_format; + struct object_context obj_context; int ret; git_config(git_default_config, NULL); @@ -407,7 +408,9 @@ int cmd_ls_tree(int argc, const char **argv, const char *prefix) ls_tree_usage, ls_tree_options); if (argc < 1) usage_with_options(ls_tree_usage, ls_tree_options); - if (repo_get_oid(the_repository, argv[0], &oid)) + if (get_oid_with_context(the_repository, argv[0], + GET_OID_HASH_ANY, &oid, + &obj_context)) die("Not a valid object name %s", argv[0]); /* From patchwork Wed Sep 27 19:55:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13401636 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C581CE82CCC for ; Wed, 27 Sep 2023 19:57:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229837AbjI0T5C (ORCPT ); Wed, 27 Sep 2023 15:57:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35950 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229840AbjI0T4f (ORCPT ); Wed, 27 Sep 2023 15:56:35 -0400 Received: from mail-io1-xd2b.google.com (mail-io1-xd2b.google.com [IPv6:2607:f8b0:4864:20::d2b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9477BCE8 for ; Wed, 27 Sep 2023 12:56:11 -0700 (PDT) Received: by mail-io1-xd2b.google.com with SMTP id ca18e2360f4ac-79fca042ec0so242158739f.3 for ; Wed, 27 Sep 2023 12:56:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1695844570; x=1696449370; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/lEoBpkJLVvzE2xIITImYwnRjfOno4E1sdAiC3yAWps=; b=cE1v/nPY1K8jW13vvqDOoQZoFpLKKmzUsLHD3bTU7a6dL9R2fCwO6FONjQ2hVCUd+1 Sv/6swfPlnxcYPG2BPAHNXTb+jSjw3hslSycL7yhwFO0AH1XJSgxBJYMurjcDFUSiD2m N+QO5bw/A2Qtp8eppkUtttgKDkh7ssM+VWV0fHZ5DOYrxvfF5h1FcLrgxmgjYLisxoFv KPmfb56vPEPbTKeu14nmM9ofjxAT3SU7VV9P/rWCPYGEkrLLXVAmL09cILWH2xeJ1Bn3 qv+Isw/FiX7wr5nHIZR77dGWvM2oMXtwNoqkvFSVIV8Idy/hoNXjxYOFOCaDqTcnVSh2 d+HQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695844570; x=1696449370; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/lEoBpkJLVvzE2xIITImYwnRjfOno4E1sdAiC3yAWps=; b=XjFLGWJ3td1S+UlZOCLQOF5JkU4palKlBxPiY48T3gvTRgS/TKc3M9sTt1vLP/GJEG w84kRVyThCY1V6ooUz+MRPRba/bZksEnOBPuoSW51Y8BDXoLOpcsmCRTL2tzUtVZCpG8 oIVdNjhthtxBHz9yP4iiaqby/trNL7yB+8EzVECnXOxpd9CIxfOBrJyz3BdABfRIePr8 Th87JgbHopwXM55eUC4xgSWRa6C6YwYQvhmaKB89c3Eij1kq/U5cyesrYNMPkQ0G6BjS 1BXN9Y6lzfAS4Im5XppLSH9Uo+T05qRxZHkqejvd4ctWsI6KuMEFC4c6FJWvCrYF23+2 iu3Q== X-Gm-Message-State: AOJu0YxTs7YHHqaz/Dnopczq8BFgQKoy9bKoJhy8v82+8htmIs8Gz33P 95BHhtp9e6DEG4VymV6j+B9Pe8F0/UU= X-Google-Smtp-Source: AGHT+IEsjdD5PSpTih4Njd5AFGe3RGvqOsgXr7W7u9NlHatb3CfbIhsswlak8o9bYYINYCuvO94m3w== X-Received: by 2002:a5e:870e:0:b0:792:931e:a93a with SMTP id y14-20020a5e870e000000b00792931ea93amr3239509ioj.4.1695844570558; Wed, 27 Sep 2023 12:56:10 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id o23-20020a6b5a17000000b007836c7e8dccsm2935628iob.17.2023.09.27.12.56.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Sep 2023 12:56:10 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , "Eric W. Biederman" Subject: [PATCH 27/30] test-lib: Compute the compatibility hash so tests may use it Date: Wed, 27 Sep 2023 14:55:34 -0500 Message-Id: <20230927195537.1682-27-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> References: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "Eric W. Biederman" Inspired-by: brian m. carlson Signed-off-by: "Eric W. Biederman" --- t/test-lib-functions.sh | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh index 2f8868caa171..92b462e2e711 100644 --- a/t/test-lib-functions.sh +++ b/t/test-lib-functions.sh @@ -1599,7 +1599,16 @@ test_set_hash () { # Detect the hash algorithm in use. test_detect_hash () { - test_hash_algo="${GIT_TEST_DEFAULT_HASH:-sha1}" + case "$GIT_TEST_DEFAULT_HASH" in + "sha256") + test_hash_algo=sha256 + test_compat_hash_algo=sha1 + ;; + *) + test_hash_algo=sha1 + test_compat_hash_algo=sha256 + ;; + esac } # Load common hash metadata and common placeholder object IDs for use with @@ -1651,6 +1660,12 @@ test_oid () { local algo="${test_hash_algo}" && case "$1" in + --hash=storage) + algo="$test_hash_algo" && + shift;; + --hash=compat) + algo="$test_compat_hash_algo" && + shift;; --hash=*) algo="${1#--hash=}" && shift;; From patchwork Wed Sep 27 19:55:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13401639 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 587FDE82CCC for ; Wed, 27 Sep 2023 19:57:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229913AbjI0T5F (ORCPT ); Wed, 27 Sep 2023 15:57:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55466 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229843AbjI0T4f (ORCPT ); Wed, 27 Sep 2023 15:56:35 -0400 Received: from mail-io1-xd31.google.com (mail-io1-xd31.google.com [IPv6:2607:f8b0:4864:20::d31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 359BDCEF for ; Wed, 27 Sep 2023 12:56:12 -0700 (PDT) Received: by mail-io1-xd31.google.com with SMTP id ca18e2360f4ac-79f989480e0so376959639f.3 for ; Wed, 27 Sep 2023 12:56:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1695844571; x=1696449371; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Ku1SPtYJYS0jAS4sy8h0S6jPalVmdQPKuhDb6UU5UX8=; b=fibxaY6tQf0Gr2XNAUHMn/wEFNoE7iRAXeOM7u88JmlwkU4czHUMLU8WIvO6p5WDQ9 TlaTKN6Uu8kyVJkJurR53B8GoTqpTcF0EFiGzOEw7xUnvek3Pske0qJ0EYgaTlM1t2Ai /Bt7zezeaDGD7+gNjX7EyghSIofH8GOCZVsq3hJ9Enz+1Z1i+Emrq5Syck6V86Ejqp2C MPLm9JVE4aPa/Cyf21n1EmvxEZRHkJDBR0o1SEv6pCOlptV2PVAvWSeBBe4HU4R7stS1 z+7Q4whXgX9XU2v3eI15OVSXlIYH/URXme1r2nSuum7lxjg+JSKFZz72L7fMyjXb3F5w 90TQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695844571; x=1696449371; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Ku1SPtYJYS0jAS4sy8h0S6jPalVmdQPKuhDb6UU5UX8=; b=CZj2sISotysIptVhPyUtRlZHRxb71gW+2jw6DTlVLPSXaVGNkkA7iLms1DDcJj2s+T xrNjqhMFnTC99nRTCA/9eKG3CROyOH/NdrqwgAePR0sGL+2TXHTnIbw8qhxzChleSpyW La/KqikTtXFyrstHfJ4Q9DCBooqOr2Sv7MXc6OQRHaHg2LwhlAI/UONuI6TGDViZ4z98 M4BomfAchW3e8GV1kFqGTPuunR3Hk5Bqnorro2TAIFdc2uSLjU1ESrvIomiGWM+h/Lbi tyhseSsByU0wA85ieMay83F2LwOIMgwut2U5X5rPpJVpiHePpZobTz2TtBZzBHBUEqxK F0xQ== X-Gm-Message-State: AOJu0YxCPBR5O1pJ1lTvcGDytAvwOfmyPU9Id9ql7MtVzWGoYhJHmgwt DDsoCYMjs8a6p/Zt3AdIUlM= X-Google-Smtp-Source: AGHT+IG/jwix3gDvcQDj9ELUbtkzXjoRfXb7LwyRZyN7D2SN9rpOFhaXvHRoAzTfOTYFokW/kObIqg== X-Received: by 2002:a5d:8794:0:b0:790:f866:d71b with SMTP id f20-20020a5d8794000000b00790f866d71bmr3032874ion.13.1695844571296; Wed, 27 Sep 2023 12:56:11 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id o23-20020a6b5a17000000b007836c7e8dccsm2935628iob.17.2023.09.27.12.56.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Sep 2023 12:56:10 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , "Eric W. Biederman" Subject: [PATCH 28/30] t1006: Rename sha1 to oid Date: Wed, 27 Sep 2023 14:55:35 -0500 Message-Id: <20230927195537.1682-28-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> References: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "Eric W. Biederman" Before I extend this test, changing the naming of the relevant hash from sha1 to oid. Calling the hash sha1 is incorrect today as it can be either sha1 or sha256 depending on the value of GIT_DEFAULT_HASH_FUNCTION when the test is called. I plan to test sha1 and sha256 simultaneously in the same repository. Having a name like sha1 will be even more confusing. Signed-off-by: "Eric W. Biederman" --- t/t1006-cat-file.sh | 220 ++++++++++++++++++++++---------------------- 1 file changed, 110 insertions(+), 110 deletions(-) diff --git a/t/t1006-cat-file.sh b/t/t1006-cat-file.sh index d73a0be1b9d1..9b018b538950 100755 --- a/t/t1006-cat-file.sh +++ b/t/t1006-cat-file.sh @@ -112,65 +112,65 @@ strlen () { run_tests () { type=$1 - sha1=$2 + oid=$2 size=$3 content=$4 pretty_content=$5 - batch_output="$sha1 $type $size + batch_output="$oid $type $size $content" test_expect_success "$type exists" ' - git cat-file -e $sha1 + git cat-file -e $oid ' test_expect_success "Type of $type is correct" ' echo $type >expect && - git cat-file -t $sha1 >actual && + git cat-file -t $oid >actual && test_cmp expect actual ' test_expect_success "Size of $type is correct" ' echo $size >expect && - git cat-file -s $sha1 >actual && + git cat-file -s $oid >actual && test_cmp expect actual ' test_expect_success "Type of $type is correct using --allow-unknown-type" ' echo $type >expect && - git cat-file -t --allow-unknown-type $sha1 >actual && + git cat-file -t --allow-unknown-type $oid >actual && test_cmp expect actual ' test_expect_success "Size of $type is correct using --allow-unknown-type" ' echo $size >expect && - git cat-file -s --allow-unknown-type $sha1 >actual && + git cat-file -s --allow-unknown-type $oid >actual && test_cmp expect actual ' test -z "$content" || test_expect_success "Content of $type is correct" ' echo_without_newline "$content" >expect && - git cat-file $type $sha1 >actual && + git cat-file $type $oid >actual && test_cmp expect actual ' test_expect_success "Pretty content of $type is correct" ' echo_without_newline "$pretty_content" >expect && - git cat-file -p $sha1 >actual && + git cat-file -p $oid >actual && test_cmp expect actual ' test -z "$content" || test_expect_success "--batch output of $type is correct" ' echo "$batch_output" >expect && - echo $sha1 | git cat-file --batch >actual && + echo $oid | git cat-file --batch >actual && test_cmp expect actual ' test_expect_success "--batch-check output of $type is correct" ' - echo "$sha1 $type $size" >expect && - echo_without_newline $sha1 | git cat-file --batch-check >actual && + echo "$oid $type $size" >expect && + echo_without_newline $oid | git cat-file --batch-check >actual && test_cmp expect actual ' @@ -179,33 +179,33 @@ $content" test -z "$content" || test_expect_success "--batch-command $opt output of $type content is correct" ' echo "$batch_output" >expect && - test_write_lines "contents $sha1" | git cat-file --batch-command $opt >actual && + test_write_lines "contents $oid" | git cat-file --batch-command $opt >actual && test_cmp expect actual ' test_expect_success "--batch-command $opt output of $type info is correct" ' - echo "$sha1 $type $size" >expect && - test_write_lines "info $sha1" | + echo "$oid $type $size" >expect && + test_write_lines "info $oid" | git cat-file --batch-command $opt >actual && test_cmp expect actual ' done test_expect_success "custom --batch-check format" ' - echo "$type $sha1" >expect && - echo $sha1 | git cat-file --batch-check="%(objecttype) %(objectname)" >actual && + echo "$type $oid" >expect && + echo $oid | git cat-file --batch-check="%(objecttype) %(objectname)" >actual && test_cmp expect actual ' test_expect_success "custom --batch-command format" ' - echo "$type $sha1" >expect && - echo "info $sha1" | git cat-file --batch-command="%(objecttype) %(objectname)" >actual && + echo "$type $oid" >expect && + echo "info $oid" | git cat-file --batch-command="%(objecttype) %(objectname)" >actual && test_cmp expect actual ' test_expect_success '--batch-check with %(rest)' ' echo "$type this is some extra content" >expect && - echo "$sha1 this is some extra content" | + echo "$oid this is some extra content" | git cat-file --batch-check="%(objecttype) %(rest)" >actual && test_cmp expect actual ' @@ -216,7 +216,7 @@ $content" echo "$size" && echo "$content" } >expect && - echo $sha1 | git cat-file --batch="%(objectsize)" >actual && + echo $oid | git cat-file --batch="%(objectsize)" >actual && test_cmp expect actual ' @@ -226,25 +226,25 @@ $content" echo "$type" && echo "$content" } >expect && - echo $sha1 | git cat-file --batch="%(objecttype)" >actual && + echo $oid | git cat-file --batch="%(objecttype)" >actual && test_cmp expect actual ' } hello_content="Hello World" hello_size=$(strlen "$hello_content") -hello_sha1=$(echo_without_newline "$hello_content" | git hash-object --stdin) +hello_oid=$(echo_without_newline "$hello_content" | git hash-object --stdin) test_expect_success "setup" ' echo_without_newline "$hello_content" > hello && git update-index --add hello ' -run_tests 'blob' $hello_sha1 $hello_size "$hello_content" "$hello_content" +run_tests 'blob' $hello_oid $hello_size "$hello_content" "$hello_content" test_expect_success '--batch-command --buffer with flush for blob info' ' - echo "$hello_sha1 blob $hello_size" >expect && - test_write_lines "info $hello_sha1" "flush" | + echo "$hello_oid blob $hello_size" >expect && + test_write_lines "info $hello_oid" "flush" | GIT_TEST_CAT_FILE_NO_FLUSH_ON_EXIT=1 \ git cat-file --batch-command --buffer >actual && test_cmp expect actual @@ -252,38 +252,38 @@ test_expect_success '--batch-command --buffer with flush for blob info' ' test_expect_success '--batch-command --buffer without flush for blob info' ' touch output && - test_write_lines "info $hello_sha1" | + test_write_lines "info $hello_oid" | GIT_TEST_CAT_FILE_NO_FLUSH_ON_EXIT=1 \ git cat-file --batch-command --buffer >>output && test_must_be_empty output ' test_expect_success '--batch-check without %(rest) considers whole line' ' - echo "$hello_sha1 blob $hello_size" >expect && - git update-index --add --cacheinfo 100644 $hello_sha1 "white space" && + echo "$hello_oid blob $hello_size" >expect && + git update-index --add --cacheinfo 100644 $hello_oid "white space" && test_when_finished "git update-index --remove \"white space\"" && echo ":white space" | git cat-file --batch-check >actual && test_cmp expect actual ' -tree_sha1=$(git write-tree) +tree_oid=$(git write-tree) tree_size=$(($(test_oid rawsz) + 13)) -tree_pretty_content="100644 blob $hello_sha1 hello${LF}" +tree_pretty_content="100644 blob $hello_oid hello${LF}" -run_tests 'tree' $tree_sha1 $tree_size "" "$tree_pretty_content" +run_tests 'tree' $tree_oid $tree_size "" "$tree_pretty_content" commit_message="Initial commit" -commit_sha1=$(echo_without_newline "$commit_message" | git commit-tree $tree_sha1) +commit_oid=$(echo_without_newline "$commit_message" | git commit-tree $tree_oid) commit_size=$(($(test_oid hexsz) + 137)) -commit_content="tree $tree_sha1 +commit_content="tree $tree_oid author $GIT_AUTHOR_NAME <$GIT_AUTHOR_EMAIL> $GIT_AUTHOR_DATE committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE $commit_message" -run_tests 'commit' $commit_sha1 $commit_size "$commit_content" "$commit_content" +run_tests 'commit' $commit_oid $commit_size "$commit_content" "$commit_content" -tag_header_without_timestamp="object $hello_sha1 +tag_header_without_timestamp="object $hello_oid type blob tag hellotag tagger $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL>" @@ -292,14 +292,14 @@ tag_content="$tag_header_without_timestamp 0 +0000 $tag_description" -tag_sha1=$(echo_without_newline "$tag_content" | git hash-object -t tag --stdin -w) +tag_oid=$(echo_without_newline "$tag_content" | git hash-object -t tag --stdin -w) tag_size=$(strlen "$tag_content") -run_tests 'tag' $tag_sha1 $tag_size "$tag_content" "$tag_content" +run_tests 'tag' $tag_oid $tag_size "$tag_content" "$tag_content" test_expect_success "Reach a blob from a tag pointing to it" ' echo_without_newline "$hello_content" >expect && - git cat-file blob $tag_sha1 >actual && + git cat-file blob $tag_oid >actual && test_cmp expect actual ' @@ -308,31 +308,31 @@ do for opt in t s e p do test_expect_success "Passing -$opt with --$batch fails" ' - test_must_fail git cat-file --$batch -$opt $hello_sha1 + test_must_fail git cat-file --$batch -$opt $hello_oid ' test_expect_success "Passing --$batch with -$opt fails" ' - test_must_fail git cat-file -$opt --$batch $hello_sha1 + test_must_fail git cat-file -$opt --$batch $hello_oid ' done test_expect_success "Passing with --$batch fails" ' - test_must_fail git cat-file --$batch blob $hello_sha1 + test_must_fail git cat-file --$batch blob $hello_oid ' test_expect_success "Passing --$batch with fails" ' - test_must_fail git cat-file blob --$batch $hello_sha1 + test_must_fail git cat-file blob --$batch $hello_oid ' - test_expect_success "Passing sha1 with --$batch fails" ' - test_must_fail git cat-file --$batch $hello_sha1 + test_expect_success "Passing oid with --$batch fails" ' + test_must_fail git cat-file --$batch $hello_oid ' done for opt in t s e p do test_expect_success "Passing -$opt with --follow-symlinks fails" ' - test_must_fail git cat-file --follow-symlinks -$opt $hello_sha1 + test_must_fail git cat-file --follow-symlinks -$opt $hello_oid ' done @@ -360,12 +360,12 @@ test_expect_success "--batch-check for a non-existent hash" ' test_expect_success "--batch for an existent and a non-existent hash" ' cat >expect <<-EOF && - $tag_sha1 tag $tag_size + $tag_oid tag $tag_size $tag_content 0000000000000000000000000000000000000000 missing EOF - printf "$tag_sha1\n0000000000000000000000000000000000000000" >in && + printf "$tag_oid\n0000000000000000000000000000000000000000" >in && git cat-file --batch actual && test_cmp expect actual ' @@ -386,74 +386,74 @@ test_expect_success 'empty --batch-check notices missing object' ' test_cmp expect actual ' -batch_input="$hello_sha1 -$commit_sha1 -$tag_sha1 +batch_input="$hello_oid +$commit_oid +$tag_oid deadbeef " printf "%s\0" \ - "$hello_sha1 blob $hello_size" \ + "$hello_oid blob $hello_size" \ "$hello_content" \ - "$commit_sha1 commit $commit_size" \ + "$commit_oid commit $commit_size" \ "$commit_content" \ - "$tag_sha1 tag $tag_size" \ + "$tag_oid tag $tag_size" \ "$tag_content" \ "deadbeef missing" \ " missing" >batch_output -test_expect_success '--batch with multiple sha1s gives correct format' ' +test_expect_success '--batch with multiple oids gives correct format' ' tr "\0" "\n" expect && echo_without_newline "$batch_input" >in && git cat-file --batch actual && test_cmp expect actual ' -test_expect_success '--batch, -z with multiple sha1s gives correct format' ' +test_expect_success '--batch, -z with multiple oids gives correct format' ' echo_without_newline_nul "$batch_input" >in && tr "\0" "\n" expect && git cat-file --batch -z actual && test_cmp expect actual ' -test_expect_success '--batch, -Z with multiple sha1s gives correct format' ' +test_expect_success '--batch, -Z with multiple oids gives correct format' ' echo_without_newline_nul "$batch_input" >in && git cat-file --batch -Z actual && test_cmp batch_output actual ' -batch_check_input="$hello_sha1 -$tree_sha1 -$commit_sha1 -$tag_sha1 +batch_check_input="$hello_oid +$tree_oid +$commit_oid +$tag_oid deadbeef " printf "%s\0" \ - "$hello_sha1 blob $hello_size" \ - "$tree_sha1 tree $tree_size" \ - "$commit_sha1 commit $commit_size" \ - "$tag_sha1 tag $tag_size" \ + "$hello_oid blob $hello_size" \ + "$tree_oid tree $tree_size" \ + "$commit_oid commit $commit_size" \ + "$tag_oid tag $tag_size" \ "deadbeef missing" \ " missing" >batch_check_output -test_expect_success "--batch-check with multiple sha1s gives correct format" ' +test_expect_success "--batch-check with multiple oids gives correct format" ' tr "\0" "\n" expect && echo_without_newline "$batch_check_input" >in && git cat-file --batch-check actual && test_cmp expect actual ' -test_expect_success "--batch-check, -z with multiple sha1s gives correct format" ' +test_expect_success "--batch-check, -z with multiple oids gives correct format" ' tr "\0" "\n" expect && echo_without_newline_nul "$batch_check_input" >in && git cat-file --batch-check -z actual && test_cmp expect actual ' -test_expect_success "--batch-check, -Z with multiple sha1s gives correct format" ' +test_expect_success "--batch-check, -Z with multiple oids gives correct format" ' echo_without_newline_nul "$batch_check_input" >in && git cat-file --batch-check -Z actual && test_cmp batch_check_output actual @@ -480,18 +480,18 @@ test_expect_success FUNNYNAMES '--batch-check, -Z with newline in input' ' test_cmp expect actual ' -batch_command_multiple_info="info $hello_sha1 -info $tree_sha1 -info $commit_sha1 -info $tag_sha1 +batch_command_multiple_info="info $hello_oid +info $tree_oid +info $commit_oid +info $tag_oid info deadbeef" test_expect_success '--batch-command with multiple info calls gives correct format' ' cat >expect <<-EOF && - $hello_sha1 blob $hello_size - $tree_sha1 tree $tree_size - $commit_sha1 commit $commit_size - $tag_sha1 tag $tag_size + $hello_oid blob $hello_size + $tree_oid tree $tree_size + $commit_oid commit $commit_size + $tag_oid tag $tag_size deadbeef missing EOF @@ -512,19 +512,19 @@ test_expect_success '--batch-command with multiple info calls gives correct form test_cmp expect_nul actual ' -batch_command_multiple_contents="contents $hello_sha1 -contents $commit_sha1 -contents $tag_sha1 +batch_command_multiple_contents="contents $hello_oid +contents $commit_oid +contents $tag_oid contents deadbeef flush" test_expect_success '--batch-command with multiple command calls gives correct format' ' printf "%s\0" \ - "$hello_sha1 blob $hello_size" \ + "$hello_oid blob $hello_size" \ "$hello_content" \ - "$commit_sha1 commit $commit_size" \ + "$commit_oid commit $commit_size" \ "$commit_content" \ - "$tag_sha1 tag $tag_size" \ + "$tag_oid tag $tag_size" \ "$tag_content" \ "deadbeef missing" >expect_nul && tr "\0" "\n" expect && @@ -569,7 +569,7 @@ test_expect_success 'confirm that neither loose blob is a delta' ' # we will check only that one of the two objects is a delta # against the other, but not the order. We can do so by just # asking for the base of both, and checking whether either -# sha1 appears in the output. +# oid appears in the output. test_expect_success '%(deltabase) reports packed delta bases' ' git repack -ad && git cat-file --batch-check="%(deltabase)" actual && @@ -583,12 +583,12 @@ test_expect_success 'setup bogus data' ' bogus_short_type="bogus" && bogus_short_content="bogus" && bogus_short_size=$(strlen "$bogus_short_content") && - bogus_short_sha1=$(echo_without_newline "$bogus_short_content" | git hash-object -t $bogus_short_type --literally -w --stdin) && + bogus_short_oid=$(echo_without_newline "$bogus_short_content" | git hash-object -t $bogus_short_type --literally -w --stdin) && bogus_long_type="abcdefghijklmnopqrstuvwxyz1234679" && bogus_long_content="bogus" && bogus_long_size=$(strlen "$bogus_long_content") && - bogus_long_sha1=$(echo_without_newline "$bogus_long_content" | git hash-object -t $bogus_long_type --literally -w --stdin) + bogus_long_oid=$(echo_without_newline "$bogus_long_content" | git hash-object -t $bogus_long_type --literally -w --stdin) ' for arg1 in '' --allow-unknown-type @@ -608,9 +608,9 @@ do if test "$arg1" = "--allow-unknown-type" then - git cat-file $arg1 $arg2 $bogus_short_sha1 + git cat-file $arg1 $arg2 $bogus_short_oid else - test_must_fail git cat-file $arg1 $arg2 $bogus_short_sha1 >out 2>actual && + test_must_fail git cat-file $arg1 $arg2 $bogus_short_oid >out 2>actual && test_must_be_empty out && test_cmp expect actual fi @@ -620,21 +620,21 @@ do if test "$arg2" = "-p" then cat >expect <<-EOF - error: header for $bogus_long_sha1 too long, exceeds 32 bytes - fatal: Not a valid object name $bogus_long_sha1 + error: header for $bogus_long_oid too long, exceeds 32 bytes + fatal: Not a valid object name $bogus_long_oid EOF else cat >expect <<-EOF - error: header for $bogus_long_sha1 too long, exceeds 32 bytes + error: header for $bogus_long_oid too long, exceeds 32 bytes fatal: git cat-file: could not get object info EOF fi && if test "$arg1" = "--allow-unknown-type" then - git cat-file $arg1 $arg2 $bogus_short_sha1 + git cat-file $arg1 $arg2 $bogus_short_oid else - test_must_fail git cat-file $arg1 $arg2 $bogus_long_sha1 >out 2>actual && + test_must_fail git cat-file $arg1 $arg2 $bogus_long_oid >out 2>actual && test_must_be_empty out && test_cmp expect actual fi @@ -668,28 +668,28 @@ do done test_expect_success '-e is OK with a broken object without --allow-unknown-type' ' - git cat-file -e $bogus_short_sha1 + git cat-file -e $bogus_short_oid ' test_expect_success '-e can not be combined with --allow-unknown-type' ' - test_expect_code 128 git cat-file -e --allow-unknown-type $bogus_short_sha1 + test_expect_code 128 git cat-file -e --allow-unknown-type $bogus_short_oid ' test_expect_success '-p cannot print a broken object even with --allow-unknown-type' ' - test_must_fail git cat-file -p $bogus_short_sha1 && - test_expect_code 128 git cat-file -p --allow-unknown-type $bogus_short_sha1 + test_must_fail git cat-file -p $bogus_short_oid && + test_expect_code 128 git cat-file -p --allow-unknown-type $bogus_short_oid ' test_expect_success ' does not work with objects of broken types' ' cat >err.expect <<-\EOF && fatal: invalid object type "bogus" EOF - test_must_fail git cat-file $bogus_short_type $bogus_short_sha1 2>err.actual && + test_must_fail git cat-file $bogus_short_type $bogus_short_oid 2>err.actual && test_cmp err.expect err.actual ' test_expect_success 'broken types combined with --batch and --batch-check' ' - echo $bogus_short_sha1 >bogus-oid && + echo $bogus_short_oid >bogus-oid && cat >err.expect <<-\EOF && fatal: invalid object type @@ -711,52 +711,52 @@ test_expect_success 'the --allow-unknown-type option does not consider replaceme cat >expect <<-EOF && $bogus_short_type EOF - git cat-file -t --allow-unknown-type $bogus_short_sha1 >actual && + git cat-file -t --allow-unknown-type $bogus_short_oid >actual && test_cmp expect actual && # Create it manually, as "git replace" will die on bogus # types. head=$(git rev-parse --verify HEAD) && - test_when_finished "test-tool ref-store main delete-refs 0 msg refs/replace/$bogus_short_sha1" && - test-tool ref-store main update-ref msg "refs/replace/$bogus_short_sha1" $head $ZERO_OID REF_SKIP_OID_VERIFICATION && + test_when_finished "test-tool ref-store main delete-refs 0 msg refs/replace/$bogus_short_oid" && + test-tool ref-store main update-ref msg "refs/replace/$bogus_short_oid" $head $ZERO_OID REF_SKIP_OID_VERIFICATION && cat >expect <<-EOF && commit EOF - git cat-file -t --allow-unknown-type $bogus_short_sha1 >actual && + git cat-file -t --allow-unknown-type $bogus_short_oid >actual && test_cmp expect actual ' test_expect_success "Type of broken object is correct" ' echo $bogus_short_type >expect && - git cat-file -t --allow-unknown-type $bogus_short_sha1 >actual && + git cat-file -t --allow-unknown-type $bogus_short_oid >actual && test_cmp expect actual ' test_expect_success "Size of broken object is correct" ' echo $bogus_short_size >expect && - git cat-file -s --allow-unknown-type $bogus_short_sha1 >actual && + git cat-file -s --allow-unknown-type $bogus_short_oid >actual && test_cmp expect actual ' test_expect_success 'clean up broken object' ' - rm .git/objects/$(test_oid_to_path $bogus_short_sha1) + rm .git/objects/$(test_oid_to_path $bogus_short_oid) ' test_expect_success "Type of broken object is correct when type is large" ' echo $bogus_long_type >expect && - git cat-file -t --allow-unknown-type $bogus_long_sha1 >actual && + git cat-file -t --allow-unknown-type $bogus_long_oid >actual && test_cmp expect actual ' test_expect_success "Size of large broken object is correct when type is large" ' echo $bogus_long_size >expect && - git cat-file -s --allow-unknown-type $bogus_long_sha1 >actual && + git cat-file -s --allow-unknown-type $bogus_long_oid >actual && test_cmp expect actual ' test_expect_success 'clean up broken object' ' - rm .git/objects/$(test_oid_to_path $bogus_long_sha1) + rm .git/objects/$(test_oid_to_path $bogus_long_oid) ' test_expect_success 'cat-file -t and -s on corrupt loose object' ' @@ -853,7 +853,7 @@ test_expect_success 'prep for symlink tests' ' test_ln_s_add loop2 loop1 && git add morx dir/subdir/ind2 dir/ind1 && git commit -am "test" && - echo $hello_sha1 blob $hello_size >found + echo $hello_oid blob $hello_size >found ' test_expect_success 'git cat-file --batch-check --follow-symlinks works for non-links' ' @@ -941,7 +941,7 @@ test_expect_success 'git cat-file --batch-check --follow-symlinks works for dir/ echo HEAD:dirlink/morx >>expect && echo HEAD:dirlink/morx | git cat-file --batch-check --follow-symlinks >actual && test_cmp expect actual && - echo $hello_sha1 blob $hello_size >expect && + echo $hello_oid blob $hello_size >expect && echo HEAD:dirlink/ind1 | git cat-file --batch-check --follow-symlinks >actual && test_cmp expect actual ' From patchwork Wed Sep 27 19:55:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13401638 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 788A6E82CCF for ; Wed, 27 Sep 2023 19:57:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229863AbjI0T5D (ORCPT ); Wed, 27 Sep 2023 15:57:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43922 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229922AbjI0T4f (ORCPT ); Wed, 27 Sep 2023 15:56:35 -0400 Received: from mail-io1-xd33.google.com (mail-io1-xd33.google.com [IPv6:2607:f8b0:4864:20::d33]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 07766CF3 for ; Wed, 27 Sep 2023 12:56:13 -0700 (PDT) Received: by mail-io1-xd33.google.com with SMTP id ca18e2360f4ac-76c64da0e46so425681439f.0 for ; Wed, 27 Sep 2023 12:56:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1695844572; x=1696449372; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=iIp5xN1/FiLxLp7U+weLwdU6T91ZE64ygxLvw8ASvC4=; b=LN4Zqw5iGpx+OswWuyi5J0ACnb9+4wS28GNb12ZEks+ek3IeVP6+N7i1Zhak3NYvF/ i+/ynCJ2jv6Yzi5p6ATi3U9xuppVtE8Vex4staXxm9AOTwrPrV7XFGOPgWmclA/fJazh hPT+L7X8pYTW4d9rbTgyAoGwMok5JeKgix/RYexlAR8Tr6/bjtS3BXM14d2AqaGUTqFj 7oaQOvlxg5V9OTIpT0DTSjoV2Che4gPqc4/c2iLUazg58bjkRBYSlFUU+fHBRO9/KI6B JmDz5t+D0ewzOS8aZJRcPlPn/0xtGmRKnkSIPTJJ4t2V3a3ICwXf7Vfzh75zwxi91J+h nTDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695844572; x=1696449372; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=iIp5xN1/FiLxLp7U+weLwdU6T91ZE64ygxLvw8ASvC4=; b=RXCcyQKRq/3VEuL6rMTMAg+52dWvGq1gllz5lx+PLQJqkP0qGD9IOl/wOexVG84Tmz j7TGIZCxSbgeDg2eoBOS0rqOxhT6483JuMprXvpc5oPXnscwCA22C+juniRpesRlQCB+ 47BuoHIygqTsCy84nkwTffUb+kfq3ej7Yt4AHT18+fTWUTuOG4S4WGxy4uYi5SNo/wRT C8IsJgIwaehmaGpXijefPk9uKdEOSqWCNwGJTlgKUEBji6RFdBZQzhdWQWMoeZES4JBg Bg5YsJ+xLdZKcZCjBZ0TUoDkSsZK2zurR6+jCgKFuNfMhY1ezRfKujllZsLMBOcuoo0f lglA== X-Gm-Message-State: AOJu0YzP2ukwvagx5/etRFpN/W4cz48xJcpJi+hzRYn2AjuDFDjbTZTp VwfZxc4dt2FrksZ7pezZKXo= X-Google-Smtp-Source: AGHT+IFpu+Ghc/pWTm3/4DUfrfKzBQByIbKemQ5HIMzZYUrGUyaHBZnQMVmFd1VBlK/ycPzIPtuH+Q== X-Received: by 2002:a6b:d90f:0:b0:799:36c2:fa49 with SMTP id r15-20020a6bd90f000000b0079936c2fa49mr3014365ioc.15.1695844572225; Wed, 27 Sep 2023 12:56:12 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id o23-20020a6b5a17000000b007836c7e8dccsm2935628iob.17.2023.09.27.12.56.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Sep 2023 12:56:11 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , "Eric W. Biederman" Subject: [PATCH 29/30] t1006: Test oid compatibility with cat-file Date: Wed, 27 Sep 2023 14:55:36 -0500 Message-Id: <20230927195537.1682-29-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> References: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "Eric W. Biederman" Update the existing tests that are oid based to test that cat-file works correctly with the normal oid and the compat_oid. Signed-off-by: "Eric W. Biederman" --- t/t1006-cat-file.sh | 251 +++++++++++++++++++++++++++----------------- 1 file changed, 154 insertions(+), 97 deletions(-) diff --git a/t/t1006-cat-file.sh b/t/t1006-cat-file.sh index 9b018b538950..23d3d37283bb 100755 --- a/t/t1006-cat-file.sh +++ b/t/t1006-cat-file.sh @@ -236,27 +236,38 @@ hello_size=$(strlen "$hello_content") hello_oid=$(echo_without_newline "$hello_content" | git hash-object --stdin) test_expect_success "setup" ' + git config core.repositoryformatversion 1 && + git config extensions.objectformat $test_hash_algo && + git config extensions.compatobjectformat $test_compat_hash_algo && echo_without_newline "$hello_content" > hello && git update-index --add hello ' -run_tests 'blob' $hello_oid $hello_size "$hello_content" "$hello_content" +run_blob_tests () { + oid=$1 -test_expect_success '--batch-command --buffer with flush for blob info' ' - echo "$hello_oid blob $hello_size" >expect && - test_write_lines "info $hello_oid" "flush" | + run_tests 'blob' $oid $hello_size "$hello_content" "$hello_content" + + test_expect_success '--batch-command --buffer with flush for blob info' ' + echo "$oid blob $hello_size" >expect && + test_write_lines "info $oid" "flush" | GIT_TEST_CAT_FILE_NO_FLUSH_ON_EXIT=1 \ git cat-file --batch-command --buffer >actual && test_cmp expect actual -' + ' -test_expect_success '--batch-command --buffer without flush for blob info' ' + test_expect_success '--batch-command --buffer without flush for blob info' ' touch output && - test_write_lines "info $hello_oid" | + test_write_lines "info $oid" | GIT_TEST_CAT_FILE_NO_FLUSH_ON_EXIT=1 \ git cat-file --batch-command --buffer >>output && test_must_be_empty output -' + ' +} + +hello_compat_oid=$(git rev-parse --output-object-format=$test_compat_hash_algo $hello_oid) +run_blob_tests $hello_oid +run_blob_tests $hello_compat_oid test_expect_success '--batch-check without %(rest) considers whole line' ' echo "$hello_oid blob $hello_size" >expect && @@ -267,35 +278,58 @@ test_expect_success '--batch-check without %(rest) considers whole line' ' ' tree_oid=$(git write-tree) +tree_compat_oid=$(git rev-parse --output-object-format=$test_compat_hash_algo $tree_oid) tree_size=$(($(test_oid rawsz) + 13)) +tree_compat_size=$(($(test_oid --hash=compat rawsz) + 13)) tree_pretty_content="100644 blob $hello_oid hello${LF}" +tree_compat_pretty_content="100644 blob $hello_compat_oid hello${LF}" run_tests 'tree' $tree_oid $tree_size "" "$tree_pretty_content" +run_tests 'tree' $tree_compat_oid $tree_compat_size "" "$tree_compat_pretty_content" commit_message="Initial commit" commit_oid=$(echo_without_newline "$commit_message" | git commit-tree $tree_oid) +commit_compat_oid=$(git rev-parse --output-object-format=$test_compat_hash_algo $commit_oid) commit_size=$(($(test_oid hexsz) + 137)) +commit_compat_size=$(($(test_oid --hash=compat hexsz) + 137)) commit_content="tree $tree_oid author $GIT_AUTHOR_NAME <$GIT_AUTHOR_EMAIL> $GIT_AUTHOR_DATE committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE $commit_message" +commit_compat_content="tree $tree_compat_oid +author $GIT_AUTHOR_NAME <$GIT_AUTHOR_EMAIL> $GIT_AUTHOR_DATE +committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE + +$commit_message" + run_tests 'commit' $commit_oid $commit_size "$commit_content" "$commit_content" +run_tests 'commit' $commit_compat_oid $commit_compat_size "$commit_compat_content" "$commit_compat_content" -tag_header_without_timestamp="object $hello_oid -type blob +tag_header_without_oid="type blob tag hellotag tagger $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL>" +tag_header_without_timestamp="object $hello_oid +$tag_header_without_oid" +tag_compat_header_without_timestamp="object $hello_compat_oid +$tag_header_without_oid" tag_description="This is a tag" tag_content="$tag_header_without_timestamp 0 +0000 +$tag_description" +tag_compat_content="$tag_compat_header_without_timestamp 0 +0000 + $tag_description" tag_oid=$(echo_without_newline "$tag_content" | git hash-object -t tag --stdin -w) tag_size=$(strlen "$tag_content") +tag_compat_oid=$(git rev-parse --output-object-format=$test_compat_hash_algo $tag_oid) +tag_compat_size=$(strlen "$tag_compat_content") + run_tests 'tag' $tag_oid $tag_size "$tag_content" "$tag_content" +run_tests 'tag' $tag_compat_oid $tag_compat_size "$tag_compat_content" "$tag_compat_content" test_expect_success "Reach a blob from a tag pointing to it" ' echo_without_newline "$hello_content" >expect && @@ -303,37 +337,43 @@ test_expect_success "Reach a blob from a tag pointing to it" ' test_cmp expect actual ' -for batch in batch batch-check batch-command +for oid in $hello_oid $hello_compat_oid do - for opt in t s e p + for batch in batch batch-check batch-command do + for opt in t s e p + do test_expect_success "Passing -$opt with --$batch fails" ' - test_must_fail git cat-file --$batch -$opt $hello_oid + test_must_fail git cat-file --$batch -$opt $oid ' test_expect_success "Passing --$batch with -$opt fails" ' - test_must_fail git cat-file -$opt --$batch $hello_oid + test_must_fail git cat-file -$opt --$batch $oid ' - done + done - test_expect_success "Passing with --$batch fails" ' - test_must_fail git cat-file --$batch blob $hello_oid - ' + test_expect_success "Passing with --$batch fails" ' + test_must_fail git cat-file --$batch blob $oid + ' - test_expect_success "Passing --$batch with fails" ' - test_must_fail git cat-file blob --$batch $hello_oid - ' + test_expect_success "Passing --$batch with fails" ' + test_must_fail git cat-file blob --$batch $oid + ' - test_expect_success "Passing oid with --$batch fails" ' - test_must_fail git cat-file --$batch $hello_oid - ' + test_expect_success "Passing oid with --$batch fails" ' + test_must_fail git cat-file --$batch $oid + ' + done done -for opt in t s e p +for oid in $hello_oid $hello_compat_oid do - test_expect_success "Passing -$opt with --follow-symlinks fails" ' - test_must_fail git cat-file --follow-symlinks -$opt $hello_oid + for opt in t s e p + do + test_expect_success "Passing -$opt with --follow-symlinks fails" ' + test_must_fail git cat-file --follow-symlinks -$opt $oid ' + done done test_expect_success "--batch-check for a non-existent named object" ' @@ -386,112 +426,102 @@ test_expect_success 'empty --batch-check notices missing object' ' test_cmp expect actual ' -batch_input="$hello_oid -$commit_oid -$tag_oid +batch_tests () { + boid=$1 + loid=$2 + lsize=$3 + coid=$4 + csize=$5 + ccontent=$6 + toid=$7 + tsize=$8 + tcontent=$9 + + batch_input="$boid +$coid +$toid deadbeef " -printf "%s\0" \ - "$hello_oid blob $hello_size" \ + printf "%s\0" \ + "$boid blob $hello_size" \ "$hello_content" \ - "$commit_oid commit $commit_size" \ - "$commit_content" \ - "$tag_oid tag $tag_size" \ - "$tag_content" \ + "$coid commit $csize" \ + "$ccontent" \ + "$toid tag $tsize" \ + "$tcontent" \ "deadbeef missing" \ " missing" >batch_output -test_expect_success '--batch with multiple oids gives correct format' ' + test_expect_success '--batch with multiple oids gives correct format' ' tr "\0" "\n" expect && echo_without_newline "$batch_input" >in && git cat-file --batch actual && test_cmp expect actual -' + ' -test_expect_success '--batch, -z with multiple oids gives correct format' ' + test_expect_success '--batch, -z with multiple oids gives correct format' ' echo_without_newline_nul "$batch_input" >in && tr "\0" "\n" expect && git cat-file --batch -z actual && test_cmp expect actual -' + ' -test_expect_success '--batch, -Z with multiple oids gives correct format' ' + test_expect_success '--batch, -Z with multiple oids gives correct format' ' echo_without_newline_nul "$batch_input" >in && git cat-file --batch -Z actual && test_cmp batch_output actual -' + ' -batch_check_input="$hello_oid -$tree_oid -$commit_oid -$tag_oid +batch_check_input="$boid +$loid +$coid +$toid deadbeef " -printf "%s\0" \ - "$hello_oid blob $hello_size" \ - "$tree_oid tree $tree_size" \ - "$commit_oid commit $commit_size" \ - "$tag_oid tag $tag_size" \ + printf "%s\0" \ + "$boid blob $hello_size" \ + "$loid tree $lsize" \ + "$coid commit $csize" \ + "$toid tag $tsize" \ "deadbeef missing" \ " missing" >batch_check_output -test_expect_success "--batch-check with multiple oids gives correct format" ' + test_expect_success "--batch-check with multiple oids gives correct format" ' tr "\0" "\n" expect && echo_without_newline "$batch_check_input" >in && git cat-file --batch-check actual && test_cmp expect actual -' + ' -test_expect_success "--batch-check, -z with multiple oids gives correct format" ' + test_expect_success "--batch-check, -z with multiple oids gives correct format" ' tr "\0" "\n" expect && echo_without_newline_nul "$batch_check_input" >in && git cat-file --batch-check -z actual && test_cmp expect actual -' + ' -test_expect_success "--batch-check, -Z with multiple oids gives correct format" ' + test_expect_success "--batch-check, -Z with multiple oids gives correct format" ' echo_without_newline_nul "$batch_check_input" >in && git cat-file --batch-check -Z actual && test_cmp batch_check_output actual -' - -test_expect_success FUNNYNAMES 'setup with newline in input' ' - touch -- "newline${LF}embedded" && - git add -- "newline${LF}embedded" && - git commit -m "file with newline embedded" && - test_tick && - - printf "HEAD:newline${LF}embedded" >in -' - -test_expect_success FUNNYNAMES '--batch-check, -z with newline in input' ' - git cat-file --batch-check -z actual && - echo "$(git rev-parse "HEAD:newline${LF}embedded") blob 0" >expect && - test_cmp expect actual -' - -test_expect_success FUNNYNAMES '--batch-check, -Z with newline in input' ' - git cat-file --batch-check -Z actual && - printf "%s\0" "$(git rev-parse "HEAD:newline${LF}embedded") blob 0" >expect && - test_cmp expect actual -' + ' -batch_command_multiple_info="info $hello_oid -info $tree_oid -info $commit_oid -info $tag_oid +batch_command_multiple_info="info $boid +info $loid +info $coid +info $toid info deadbeef" -test_expect_success '--batch-command with multiple info calls gives correct format' ' + test_expect_success '--batch-command with multiple info calls gives correct format' ' cat >expect <<-EOF && - $hello_oid blob $hello_size - $tree_oid tree $tree_size - $commit_oid commit $commit_size - $tag_oid tag $tag_size + $boid blob $hello_size + $loid tree $lsize + $coid commit $csize + $toid tag $tsize deadbeef missing EOF @@ -510,22 +540,22 @@ test_expect_success '--batch-command with multiple info calls gives correct form git cat-file --batch-command --buffer -Z actual && test_cmp expect_nul actual -' + ' -batch_command_multiple_contents="contents $hello_oid -contents $commit_oid -contents $tag_oid +batch_command_multiple_contents="contents $boid +contents $coid +contents $toid contents deadbeef flush" -test_expect_success '--batch-command with multiple command calls gives correct format' ' + test_expect_success '--batch-command with multiple command calls gives correct format' ' printf "%s\0" \ - "$hello_oid blob $hello_size" \ + "$boid blob $hello_size" \ "$hello_content" \ - "$commit_oid commit $commit_size" \ - "$commit_content" \ - "$tag_oid tag $tag_size" \ - "$tag_content" \ + "$coid commit $csize" \ + "$ccontent" \ + "$toid tag $tsize" \ + "$tcontent" \ "deadbeef missing" >expect_nul && tr "\0" "\n" expect && @@ -543,6 +573,33 @@ test_expect_success '--batch-command with multiple command calls gives correct f git cat-file --batch-command --buffer -Z actual && test_cmp expect_nul actual + ' + +} + +batch_tests $hello_oid $tree_oid $tree_size $commit_oid $commit_size "$commit_content" $tag_oid $tag_size "$tag_content" +batch_tests $hello_compat_oid $tree_compat_oid $tree_compat_size $commit_compat_oid $commit_compat_size "$commit_compat_content" $tag_compat_oid $tag_compat_size "$tag_compat_content" + + +test_expect_success FUNNYNAMES 'setup with newline in input' ' + touch -- "newline${LF}embedded" && + git add -- "newline${LF}embedded" && + git commit -m "file with newline embedded" && + test_tick && + + printf "HEAD:newline${LF}embedded" >in +' + +test_expect_success FUNNYNAMES '--batch-check, -z with newline in input' ' + git cat-file --batch-check -z actual && + echo "$(git rev-parse "HEAD:newline${LF}embedded") blob 0" >expect && + test_cmp expect actual +' + +test_expect_success FUNNYNAMES '--batch-check, -Z with newline in input' ' + git cat-file --batch-check -Z actual && + printf "%s\0" "$(git rev-parse "HEAD:newline${LF}embedded") blob 0" >expect && + test_cmp expect actual ' test_expect_success 'setup blobs which are likely to delta' ' From patchwork Wed Sep 27 19:55:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Eric W. Biederman" X-Patchwork-Id: 13401640 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3BFB4E82CCE for ; Wed, 27 Sep 2023 19:57:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229920AbjI0T5G (ORCPT ); Wed, 27 Sep 2023 15:57:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43996 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229810AbjI0T4f (ORCPT ); Wed, 27 Sep 2023 15:56:35 -0400 Received: from mail-io1-xd2f.google.com (mail-io1-xd2f.google.com [IPv6:2607:f8b0:4864:20::d2f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A7D7ECFB for ; Wed, 27 Sep 2023 12:56:13 -0700 (PDT) Received: by mail-io1-xd2f.google.com with SMTP id ca18e2360f4ac-79fe612beabso130146939f.3 for ; Wed, 27 Sep 2023 12:56:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1695844573; x=1696449373; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=hQxjE9pH8qvGJyZahDG9JSV3KWmgPfFjKYyYvy3ZImg=; b=HkfPLV+UAB7rPu3UPIZnfwbMnsT1uSu8uOC85q4vDUkSfICC51DsRBaDoxAdeq7kzk DOw3btti4TxlIxMvjVVIUt6xWr0+ZLN09P0mhnpukv/cy79kuAR64MEaDC9FMsB6vepF t63pTrI827M5q31uwLCpmE1WSO4JtrkIj0kC69Vr+soMuUJVYgNSD3S7gJ9ZcH/tn8uC SG5MHdt6wiPYllF6QmdOVudF0v9t9uVlD/xMP/27HsbC+ODmMLY7pprAfdxqxAQ/MxhO pT8GA0Zs180wlnzZaIzdBNYpaN2jlCcoKB5xEQwFHKZPM66AM6nHnDNm5jYvg9+TY62S Z1Yg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695844573; x=1696449373; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hQxjE9pH8qvGJyZahDG9JSV3KWmgPfFjKYyYvy3ZImg=; b=nJ1vt8/gS30QeNvsK+39Gpwz1N18WCUdf8X8Qxqe3kRgTJfLpKcim8PtFN2hzkcHyI hH+vVXeLNg/zeNLyyJr5a4zBym9imTkifKtFpTlo+lc9XxUlNfkUuLrNpod41xcrYew7 LszmuZasmGMgbkwg1jGTto292781ofAtenLn9r7UXeKAnyUoYQ5sQT4mTxHducGZQF36 Ystz29iOu+22//5EWP4TFYn9hODTdca45xF77aclbfBgNc5RS9Bg0CKMx+qqPG6f0xi2 84fvBrWTZtM4+ir0kG6TQ2jVJj72bTL08MbZU8757hEw1hUQO+NtVasYm6b4DsWtPgAK vxtA== X-Gm-Message-State: AOJu0Yy+3VJpEsEhXIbV33wtazpYyR/4RbUfx6qOWa6/eW+Fr5bvAeLy fSqdv/tNxiBZSDg3gwmIZW82DUwPkCc= X-Google-Smtp-Source: AGHT+IFLMW32L3+sOAMe/wRxk05M1cWLdPBZMJK9+v4w9cb7SM2DWBsMuY3rbcKQDRBQy3i04en8Bg== X-Received: by 2002:a6b:dd14:0:b0:784:314f:8d68 with SMTP id f20-20020a6bdd14000000b00784314f8d68mr3037146ioc.1.1695844572935; Wed, 27 Sep 2023 12:56:12 -0700 (PDT) Received: from localhost.localdomain (ip68-227-168-167.om.om.cox.net. [68.227.168.167]) by smtp.gmail.com with ESMTPSA id o23-20020a6b5a17000000b007836c7e8dccsm2935628iob.17.2023.09.27.12.56.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Sep 2023 12:56:12 -0700 (PDT) From: "Eric W. Biederman" To: Junio C Hamano Cc: git@vger.kernel.org, "brian m. carlson" , "Eric W. Biederman" Subject: [PATCH 30/30] t1016-compatObjectFormat: Add tests to verify the conversion between objects Date: Wed, 27 Sep 2023 14:55:37 -0500 Message-Id: <20230927195537.1682-30-ebiederm@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> References: <87jzsbjt0a.fsf@gmail.froward.int.ebiederm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: "Eric W. Biederman" For now my strategy is simple. Create two identical repositories one in each format. Use fixed timestamps. Verify the dynamically computed compatibility objects from one repository match the objects stored in the other repository. A general limitation of this strategy is that the git when generating signed tags and commits with compatObjectFormat enabled will generate a signature for both formats. To overcome this limitation I have added "test-tool delete-gpgsig" that when fed an signed commit or tag with two signatures deletes one of the signatures. With that in place I can have "git commit" and "git tag" generate signed objects, have my tool delete one, and feed the new object into "git hash-object" to create the kinds of commits and tags git without compatObjectFormat enabled will generate. Signed-off-by: "Eric W. Biederman" --- Makefile | 1 + t/helper/test-delete-gpgsig.c | 62 ++++++++ t/helper/test-tool.c | 1 + t/helper/test-tool.h | 1 + t/t1016-compatObjectFormat.sh | 280 ++++++++++++++++++++++++++++++++++ t/t1016/gpg | 2 + 6 files changed, 347 insertions(+) create mode 100644 t/helper/test-delete-gpgsig.c create mode 100755 t/t1016-compatObjectFormat.sh create mode 100755 t/t1016/gpg diff --git a/Makefile b/Makefile index 3c18664def9a..3e4444fb9ab2 100644 --- a/Makefile +++ b/Makefile @@ -790,6 +790,7 @@ TEST_BUILTINS_OBJS += test-crontab.o TEST_BUILTINS_OBJS += test-csprng.o TEST_BUILTINS_OBJS += test-ctype.o TEST_BUILTINS_OBJS += test-date.o +TEST_BUILTINS_OBJS += test-delete-gpgsig.o TEST_BUILTINS_OBJS += test-delta.o TEST_BUILTINS_OBJS += test-dir-iterator.o TEST_BUILTINS_OBJS += test-drop-caches.o diff --git a/t/helper/test-delete-gpgsig.c b/t/helper/test-delete-gpgsig.c new file mode 100644 index 000000000000..e36831af03f6 --- /dev/null +++ b/t/helper/test-delete-gpgsig.c @@ -0,0 +1,62 @@ +#include "test-tool.h" +#include "gpg-interface.h" +#include "strbuf.h" + + +int cmd__delete_gpgsig(int argc, const char **argv) +{ + struct strbuf buf = STRBUF_INIT; + const char *pattern = "gpgsig"; + const char *bufptr, *tail, *eol; + int deleting = 0; + size_t plen; + + if (argc >= 2) { + pattern = argv[1]; + argv++; + argc--; + } + + plen = strlen(pattern); + strbuf_read(&buf, 0, 0); + + if (!strcmp(pattern, "trailer")) { + size_t payload_size = parse_signed_buffer(buf.buf, buf.len); + fwrite(buf.buf, 1, payload_size, stdout); + fflush(stdout); + return 0; + } + + bufptr = buf.buf; + tail = bufptr + buf.len; + + while (bufptr < tail) { + /* Find the end of the line */ + eol = memchr(bufptr, '\n', tail - bufptr); + if (!eol) + eol = tail; + + /* Drop continuation lines */ + if (deleting && (bufptr < eol) && (bufptr[0] == ' ')) { + bufptr = eol + 1; + continue; + } + deleting = 0; + + /* Does the line match the prefix? */ + if (((bufptr + plen) < eol) && + !memcmp(bufptr, pattern, plen) && + (bufptr[plen] == ' ')) { + deleting = 1; + bufptr = eol + 1; + continue; + } + + /* Print all other lines */ + fwrite(bufptr, 1, (eol - bufptr) + 1, stdout); + bufptr = eol + 1; + } + fflush(stdout); + + return 0; +} diff --git a/t/helper/test-tool.c b/t/helper/test-tool.c index abe8a785eb65..8b6c84f202d6 100644 --- a/t/helper/test-tool.c +++ b/t/helper/test-tool.c @@ -21,6 +21,7 @@ static struct test_cmd cmds[] = { { "csprng", cmd__csprng }, { "ctype", cmd__ctype }, { "date", cmd__date }, + { "delete-gpgsig", cmd__delete_gpgsig }, { "delta", cmd__delta }, { "dir-iterator", cmd__dir_iterator }, { "drop-caches", cmd__drop_caches }, diff --git a/t/helper/test-tool.h b/t/helper/test-tool.h index ea2672436c9a..76baaece35b9 100644 --- a/t/helper/test-tool.h +++ b/t/helper/test-tool.h @@ -15,6 +15,7 @@ int cmd__csprng(int argc, const char **argv); int cmd__ctype(int argc, const char **argv); int cmd__date(int argc, const char **argv); int cmd__delta(int argc, const char **argv); +int cmd__delete_gpgsig(int argc, const char **argv); int cmd__dir_iterator(int argc, const char **argv); int cmd__drop_caches(int argc, const char **argv); int cmd__dump_cache_tree(int argc, const char **argv); diff --git a/t/t1016-compatObjectFormat.sh b/t/t1016-compatObjectFormat.sh new file mode 100755 index 000000000000..bb558a1d562a --- /dev/null +++ b/t/t1016-compatObjectFormat.sh @@ -0,0 +1,280 @@ +#!/bin/sh +# +# Copyright (c) 2023 Eric Biederman +# + +test_description='Test how well compatObjectFormat works' + +. ./test-lib.sh +. "$TEST_DIRECTORY"/lib-gpg.sh + +# All of the follow variables must be defined in the environment: +# GIT_AUTHOR_NAME +# GIT_AUTHOR_EMAIL +# GIT_AUTHOR_DATE +# GIT_COMMITTER_NAME +# GIT_COMMITTER_EMAIL +# GIT_COMMITTER_DATE +# +# The test relies on these variables being set so that the two +# different commits in two different repositories encoded with two +# different hash functions result in the same content in the commits. +# This means that when the commit is translated between hash functions +# the commit is identical to the commit in the other repository. + +compat_hash () { + case "$1" in + "sha1") + echo "sha256" + ;; + "sha256") + echo "sha1" + ;; + esac +} + +hello_oid () { + case "$1" in + "sha1") + echo "$hello_sha1_oid" + ;; + "sha256") + echo "$hello_sha256_oid" + ;; + esac +} + +tree_oid () { + case "$1" in + "sha1") + echo "$tree_sha1_oid" + ;; + "sha256") + echo "$tree_sha256_oid" + ;; + esac +} + +commit_oid () { + case "$1" in + "sha1") + echo "$commit_sha1_oid" + ;; + "sha256") + echo "$commit_sha256_oid" + ;; + esac +} + +commit2_oid () { + case "$1" in + "sha1") + echo "$commit2_sha1_oid" + ;; + "sha256") + echo "$commit2_sha256_oid" + ;; + esac +} + +del_sigcommit () { + local delete=$1 + + if test "$delete" = "sha256" ; then + local pattern="gpgsig-sha256" + else + local pattern="gpgsig" + fi + test-tool delete-gpgsig "$pattern" +} + + +del_sigtag () { + local storage=$1 + local delete=$2 + + if test "$storage" = "$delete" ; then + local pattern="trailer" + elif test "$storage" = "sha256" ; then + local pattern="gpgsig" + else + local pattern="gpgsig-sha256" + fi + test-tool delete-gpgsig "$pattern" +} + +base=$(pwd) +for hash in sha1 sha256 +do + cd "$base" + mkdir -p repo-$hash + cd repo-$hash + + test_expect_success "setup $hash repository" ' + git init --object-format=$hash && + git config core.repositoryformatversion 1 && + git config extensions.objectformat $hash && + git config extensions.compatobjectformat $(compat_hash $hash) && + git config gpg.program $TEST_DIRECTORY/t1016/gpg && + echo "Hellow World!" > hello && + eval hello_${hash}_oid=$(git hash-object hello) && + git update-index --add hello && + git commit -m "Initial commit" && + eval commit_${hash}_oid=$(git rev-parse HEAD) && + eval tree_${hash}_oid=$(git rev-parse HEAD^{tree}) + ' + test_expect_success "create a $hash tagged blob" ' + git tag --no-sign -m "This is a tag" hellotag $(hello_oid $hash) && + eval hellotag_${hash}_oid=$(git rev-parse hellotag) + ' + test_expect_success "create a $hash tagged tree" ' + git tag --no-sign -m "This is a tag" treetag $(tree_oid $hash) && + eval treetag_${hash}_oid=$(git rev-parse treetag) + ' + test_expect_success "create a $hash tagged commit" ' + git tag --no-sign -m "This is a tag" committag $(commit_oid $hash) && + eval committag_${hash}_oid=$(git rev-parse committag) + ' + test_expect_success GPG2 "create a $hash signed commit" ' + git commit --gpg-sign --allow-empty -m "This is a signed commit" && + eval signedcommit_${hash}_oid=$(git rev-parse HEAD) + ' + test_expect_success GPG2 "create a $hash signed tag" ' + git tag -s -m "This is a signed tag" signedtag HEAD && + eval signedtag_${hash}_oid=$(git rev-parse signedtag) + ' + test_expect_success "create a $hash branch" ' + git checkout -b branch $(commit_oid $hash) && + echo "More more more give me more!" > more && + eval more_${hash}_oid=$(git hash-object more) && + echo "Another and another and another" > another && + eval another_${hash}_oid=$(git hash-object another) && + git update-index --add more another && + git commit -m "Add more files!" && + eval commit2_${hash}_oid=$(git rev-parse HEAD) && + eval tree2_${hash}_oid=$(git rev-parse HEAD^{tree}) + ' + test_expect_success GPG2 "create another $hash signed tag" ' + git tag -s -m "This is another signed tag" signedtag2 $(commit2_oid $hash) && + eval signedtag2_${hash}_oid=$(git rev-parse signedtag2) + ' + test_expect_success GPG2 "merge the $hash branches together" ' + git merge -S -m "merge some signed tags together" signedtag signedtag2 && + eval signedcommit2_${hash}_oid=$(git rev-parse HEAD) + ' + test_expect_success GPG2 "create additional $hash signed commits" ' + git commit --gpg-sign --allow-empty -m "This is an additional signed commit" && + git cat-file commit HEAD | del_sigcommit sha256 > "../${hash}_signedcommit3" && + git cat-file commit HEAD | del_sigcommit sha1 > "../${hash}_signedcommit4" && + eval signedcommit3_${hash}_oid=$(git hash-object -t commit -w ../${hash}_signedcommit3) && + eval signedcommit4_${hash}_oid=$(git hash-object -t commit -w ../${hash}_signedcommit4) + ' + test_expect_success GPG2 "create additional $hash signed tags" ' + git tag -s -m "This is an additional signed tag" signedtag34 HEAD && + git cat-file tag signedtag34 | del_sigtag "${hash}" sha256 > ../${hash}_signedtag3 && + git cat-file tag signedtag34 | del_sigtag "${hash}" sha1 > ../${hash}_signedtag4 && + eval signedtag3_${hash}_oid=$(git hash-object -t tag -w ../${hash}_signedtag3) && + eval signedtag4_${hash}_oid=$(git hash-object -t tag -w ../${hash}_signedtag4) + ' +done +cd "$base" + +compare_oids () { + test "$#" = 5 && { local PREREQ=$1; shift; } || PREREQ= + local type="$1" + local name="$2" + local sha1_oid="$3" + local sha256_oid="$4" + + echo ${sha1_oid} > ${name}_sha1_expected + echo ${sha256_oid} > ${name}_sha256_expected + echo ${type} > ${name}_type_expected + + git --git-dir=repo-sha1/.git rev-parse --output-object-format=sha256 ${sha1_oid} > ${name}_sha1_sha256_found + git --git-dir=repo-sha256/.git rev-parse --output-object-format=sha1 ${sha256_oid} > ${name}_sha256_sha1_found + local sha1_sha256_oid=$(cat ${name}_sha1_sha256_found) + local sha256_sha1_oid=$(cat ${name}_sha256_sha1_found) + + test_expect_success $PREREQ "Verify ${type} ${name}'s sha1 oid" ' + git --git-dir=repo-sha256/.git rev-parse --output-object-format=sha1 ${sha256_oid} > ${name}_sha1 && + test_cmp ${name}_sha1 ${name}_sha1_expected +' + + test_expect_success $PREREQ "Verify ${type} ${name}'s sha256 oid" ' + git --git-dir=repo-sha1/.git rev-parse --output-object-format=sha256 ${sha1_oid} > ${name}_sha256 && + test_cmp ${name}_sha256 ${name}_sha256_expected +' + + test_expect_success $PREREQ "Verify ${name}'s sha1 type" ' + git --git-dir=repo-sha1/.git cat-file -t ${sha1_oid} > ${name}_type1 && + git --git-dir=repo-sha256/.git cat-file -t ${sha256_sha1_oid} > ${name}_type2 && + test_cmp ${name}_type1 ${name}_type2 && + test_cmp ${name}_type1 ${name}_type_expected +' + + test_expect_success $PREREQ "Verify ${name}'s sha256 type" ' + git --git-dir=repo-sha256/.git cat-file -t ${sha256_oid} > ${name}_type3 && + git --git-dir=repo-sha1/.git cat-file -t ${sha1_sha256_oid} > ${name}_type4 && + test_cmp ${name}_type3 ${name}_type4 && + test_cmp ${name}_type3 ${name}_type_expected +' + + test_expect_success $PREREQ "Verify ${name}'s sha1 size" ' + git --git-dir=repo-sha1/.git cat-file -s ${sha1_oid} > ${name}_size1 && + git --git-dir=repo-sha256/.git cat-file -s ${sha256_sha1_oid} > ${name}_size2 && + test_cmp ${name}_size1 ${name}_size2 +' + + test_expect_success $PREREQ "Verify ${name}'s sha256 size" ' + git --git-dir=repo-sha256/.git cat-file -s ${sha256_oid} > ${name}_size3 && + git --git-dir=repo-sha1/.git cat-file -s ${sha1_sha256_oid} > ${name}_size4 && + test_cmp ${name}_size3 ${name}_size4 +' + + test_expect_success $PREREQ "Verify ${name}'s sha1 pretty content" ' + git --git-dir=repo-sha1/.git cat-file -p ${sha1_oid} > ${name}_content1 && + git --git-dir=repo-sha256/.git cat-file -p ${sha256_sha1_oid} > ${name}_content2 && + test_cmp ${name}_content1 ${name}_content2 +' + + test_expect_success $PREREQ "Verify ${name}'s sha256 pretty content" ' + git --git-dir=repo-sha256/.git cat-file -p ${sha256_oid} > ${name}_content3 && + git --git-dir=repo-sha1/.git cat-file -p ${sha1_sha256_oid} > ${name}_content4 && + test_cmp ${name}_content3 ${name}_content4 +' + + test_expect_success $PREREQ "Verify ${name}'s sha1 content" ' + git --git-dir=repo-sha1/.git cat-file ${type} ${sha1_oid} > ${name}_content5 && + git --git-dir=repo-sha256/.git cat-file ${type} ${sha256_sha1_oid} > ${name}_content6 && + test_cmp ${name}_content5 ${name}_content6 +' + + test_expect_success $PREREQ "Verify ${name}'s sha256 content" ' + git --git-dir=repo-sha256/.git cat-file ${type} ${sha256_oid} > ${name}_content7 && + git --git-dir=repo-sha1/.git cat-file ${type} ${sha1_sha256_oid} > ${name}_content8 && + test_cmp ${name}_content7 ${name}_content8 +' + +} + +compare_oids 'blob' hello "$hello_sha1_oid" "$hello_sha256_oid" +compare_oids 'tree' tree "$tree_sha1_oid" "$tree_sha256_oid" +compare_oids 'commit' commit "$commit_sha1_oid" "$commit_sha256_oid" +compare_oids GPG2 'commit' signedcommit "$signedcommit_sha1_oid" "$signedcommit_sha256_oid" +compare_oids 'tag' hellotag "$hellotag_sha1_oid" "$hellotag_sha256_oid" +compare_oids 'tag' treetag "$treetag_sha1_oid" "$treetag_sha256_oid" +compare_oids 'tag' committag "$committag_sha1_oid" "$committag_sha256_oid" +compare_oids GPG2 'tag' signedtag "$signedtag_sha1_oid" "$signedtag_sha256_oid" + +compare_oids 'blob' more "$more_sha1_oid" "$more_sha256_oid" +compare_oids 'blob' another "$another_sha1_oid" "$another_sha256_oid" +compare_oids 'tree' tree2 "$tree2_sha1_oid" "$tree2_sha256_oid" +compare_oids 'commit' commit2 "$commit2_sha1_oid" "$commit2_sha256_oid" +compare_oids GPG2 'tag' signedtag2 "$signedtag2_sha1_oid" "$signedtag2_sha256_oid" +compare_oids GPG2 'commit' signedcommit2 "$signedcommit2_sha1_oid" "$signedcommit2_sha256_oid" +compare_oids GPG2 'commit' signedcommit3 "$signedcommit3_sha1_oid" "$signedcommit3_sha256_oid" +compare_oids GPG2 'commit' signedcommit4 "$signedcommit4_sha1_oid" "$signedcommit4_sha256_oid" +compare_oids GPG2 'tag' signedtag3 "$signedtag3_sha1_oid" "$signedtag3_sha256_oid" +compare_oids GPG2 'tag' signedtag4 "$signedtag4_sha1_oid" "$signedtag4_sha256_oid" + +test_done diff --git a/t/t1016/gpg b/t/t1016/gpg new file mode 100755 index 000000000000..2601cb18a5b3 --- /dev/null +++ b/t/t1016/gpg @@ -0,0 +1,2 @@ +#!/bin/sh +exec gpg --faked-system-time "20230918T154812" "$@"